Showing posts with label statistics. Show all posts
Showing posts with label statistics. Show all posts

Friday, December 13, 2013

its a sexy job

Yesterday I was justifiably proud of myself. Yay yay. Jispa had to do a project in geography and it was about data and research. She and jasmine were like – this is right up Jasmine’s sleeve. Perfect. Well that put some extraordinary stress on me. I can do that in an instant. Jispa will have the best and sexiest project if I get involved but hell – that is not the job. I am not on trial here. Jispa is. She needs to learn and she needs to understand the magic of data and the bigger magic of showing the data in way that it tells stories.
Jispa had once told me that she doesn’t like me teaching because I can get very stressed out and upset when she doesn’t get the idea. She doesn’t like it. So I have been doing some deep amount of introspection. Why is that I am a great fun guy when I teach at the classrooms and such an ass for my daughter. I can have infinite patience with 60 kids but I lose it with one.  Shame on me. So I thought I probably need to change the setting. Can I use the whiteboard and teach like I teach in a classroom putting distance between the student and the teacher. Space in a classroom could be a liberating factor versus the direct and arm distance gaze (hence pressure) of a private tutor. Much like a book… what you need is also an ability to switch thoughts in the middle. Or like what Salman of the www.khanacademy.org founder says of why his youtube videos were such a hit with kids – the ability to pause and rewind.
Well I couldn’t pause and rewind or even give Jispa the button but I surely did put myself in a special mood before I started. I asked Jispa to sit on the table first and get her books and note book that has square checks on it. Then the BIG CHALLENGE. How do I explain to her without solving her problem? How do I assist her in thinking and put a basic process in place without giving out too much. Answer dena easy hai but to teach in a manner that she finds her own answers is extremely difficult. Admittedly this is the ultimate daddy teaching test. Because even in class in many ways I give a lot of answers, but homework help can never be giving answers. It has to be constructs as opposed to method. This time around it was even more difficult as the topic was not straight and especially since I have some strong and trained views on it. I pride myself in the firm for someone who not only understands data but I am especially good at structuring data so as to analyse it better. I have thought of terms and measured campaign along those lines. The very description of a term to be used helps in understanding the data in a way it makes sense. I am easily amongst the best in our office at this. My MIS over a period of time …. Ok ok… you get the point.
So how do I explain it to Jispa? So I was proud of the way I structured it for her. I sold her the magic of data. How data by itself means nothing. How more data would mean more of nothing? The context is a story. I taught her that data is nothing but a series of stories. These stories must be relevant, must have meaning must have context. Now in your project what stories do you want to tell and what is it that you must ask so that you know what to fill in the blanks. Then once a set of stories were told then the simple act of using existing data to tell more stories can be done – that ladies and gentleman is correlation without using the dirty.
I was so proud of myself. Now hopefully this weekend I will teach to draw up the loveliest questionnaire that is short but pertinent. And then after collection of data – the challenge will be to get her to be inspired to tell the best stories as possible. She shows a lot of promise and I am loving it. at the end of it I hope she not only loves the project but she loves doing a project with me.
There are two skills which will define the young individual of tomorrow – one is language. The ability to convince the other/ converse with another… the other is statistics. The world will belong to the statisticians. If you can’t read data you are dead. I am so happy with the chance to get her to love data that I am trying my best not to screw it up.
Here is Hal Varian on statistics and statistitians and how they are so sexy. Hal is the guy who worked with Google for their famous adwords bidding alogarithim. Basically – a very smart guy.

“I keep saying the sexy job in the next ten years will be statisticians. People think I’m joking, but who would’ve guessed that computer engineers would’ve been the sexy job of the 1990s? The ability to take data—to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it—that’s going to be a hugely important skill in the next decades, not only at the professional level but even at the educational level for elementary school kids, for high school kids, for college kids. Because now we really do have essentially free and ubiquitous data. So the complimentary scarce factor is the ability to understand that data and extract value from it.

I think statisticians are part of it, but it’s just a part. You also want to be able to visualize the data, communicate the data, and utilize it effectively. But I do think those skills—of being able to access, understand, and communicate the insights you get from data analysis—are going to be extremely important. Managers need to be able to access and understand the data themselves.


You always have this problem of being surrounded by “yes men” and people who want to predigest everything for you. In the old organization, you had to have this whole army of people digesting information to be able to feed it to the decision maker at the top. But that’s not the way it works anymore: the information can be available across the ranks, to everyone in the organization. And what you need to ensure is that people have access to the data they need to make their day-to-day decisions. And this can be done much more easily than it could be done in the past. And it really empowers the knowledge workers to work more effectively.”

Thursday, July 07, 2011

think right

I was talking to Jispa the other day and we were talking about the flat that we bought. So I was explaining to her that to buy the flat I had to take a loan. That set off a series of questions which culminated in me explaining to her the concept of how a bank works. I explained how the bank borrows money and pays interest and how it lends money and charges interest and the spread between the two. It was good fun. She seemed to have grasped the same. It is always an interesting conversation when I have to explain to her stuff. I kind of love taking complicated stuff and translating it to her knowledge and understanding. I yearn for the day she will be studying her rocket science and explaining it to me in a language I understand.
I have always taken pride in being able to understand complex stuff as its basic level. Like I still remember my statistics class and being able to correctly remember and apply some of its core principals. Especially the concept of the bell shaped curve. In office when we do our rating we are supposed to follow the standard distribution pattern while rating teams. It is pretty stupid and ananl in the way we implement this stuff. I have a team of around 14 heads. Now the idiots will make me rate them and expect me to rate them while following the curve. So not more than an x percentage must be rated outstanding and a y be rated below that and a z must be rated adversely. It’s stupid. A simple error of taking what is true at macro and expecting it to work in a micro environment. The first rule that most students of statistics will tell you is that standard normal curves tend to be formed at minimum sample size of 40. And that too at that level you only have 80percent confidence in the sample. Lower numbers do not throw the curve distribution. Good HR practice would have dictated that at the time of appraisal the organisation must follow a standard normal pattern for grading. So the corporate duds have mandated that every team must follow the pattern. So even my 14 head team would need to follow a statistical model while statistically it does not hold good.
Duh. Also this would be true in an environment that does not have external stimulus and bias. My team is lean and small. It enables me to focus on the lower level and manage them. I constantly attempt to attack and weed out non performance. I do not let non performance be tolerated and linger. Hence as a whole I would have a bias that ensures a larger than normal head. My variance would necessarily be less and hence the model itself designed to paint the complete picture of the organisation would fail. But it is difficult to explain this to corporate India. We are adept at making the stupid error of taking the macro picture and expecting it to work at the micro level. It’s like saying that every branch of the mango tree must produce a variation in fruit that is consistent with the variance in the tree as a whole.
This shit happens when you sleep in statistic class. I loved my professor. He used to try and make a sincere attempt to get the stuff into our brains. I remember trying to sit at the first one or two rows else the lecture risked getting lost on you. In the break we used to run to the wada pav guy and get some wadas to eat along with the spicy chutney. That also was designed to keep you alert. He tried his best and I tried my best to understand him. It was a sincere attempt.
So after all these years while I forgot most of my statistics I have been fortunate to know some of the basic stuff and as I grow older I meet people and understand further application of the stuff I learnt. I might forget what the chi square test really is but I know enough to know that mere correlation is not equal to causation. The basics like mean median and mode is something that I remember and I think at some level it helps. Especially when the quanti researchers come in to pitch some of their products. It fries my brain when they talk but at least after the deep frying some thing interesting comes out from it.
So with Jispa I try and get the basic thinking right. It’s important that the thought process works correctly. A broad and correct understanding of the concept is at times more than adequate than a detailed analysis. Sure there will be several opportunities to dive deeper into a subject matter but for most occasions an understanding that helps you talk and direct the subject matter experts would be more than useful. This is not an excuse to be a generalist. In fact one of the biggest regrets of my training is that I have not taken the opportunity to be an expert in any one field. I have been a stupid generalist. I wish I would have taken a science or an arts stream that would have helped me direct my energies towards a specialised field. Even if I would have done my management afterwards, I still wish I would have done at the least an engineering. In fact today I wish I would have followed my dad and done my graduation in civil engineering. I would have gotten great pleasure to be able to construct a building or something. Even I might have hardly ever gotten chance to build one, the knowledge of being able to build one would have been lovely.
Talking about Jispa knowing about interest. Once while we were giving her pocket money because she played her guitar I got a brain wave. While she should know and experience the joy of getting and spending money she must also experience the joy of saving money and getting interest on it. so if she spends her money she know that she will not only part with her money but also the future interest on it. So we decided to give her interest. For sake of calculation her interest is set at 10percent per month. Oh dam what a high rate of interest. She is a pathaan. But it’s ok I guess. The other day Jasmine decided to give Jispa the money we got from selling old newspapers. On hearing this I jumped up. “what are you doing? I am paying ten percent on that every month.” Jasmine had a good laugh. “who told you to pay interest to your daughter?” she asked.
Oh dam. My wise ideas will come back to give me a bite on my bum.
For the record, in the middle her Nanaji gave her some 750 bucks for helping him in some origami class. So today while I was paying her interest, I was like wow...she had calculated interest at 98.7 rupees. Jasmine later told me that she calculated interest herself. She figured out how much the amount should be ... all jasmine told her was you will get 10 bucks per hundred. Cool. next time around she will understand for herself the benefits of compounding.
Jispa is also prone to asking some really simple questions. One question she asked Jasmine. “Mom, if you knew John Abraham in college itself (he studied in the same college, one year older), why didn’t you take his autograph then itself?”

Another one that I was super impressed by - she called me while I was travelling. She asked me “dad, I was reading about Jesus Christ (amar chitra katha) and I was wondering if BC is before Christa and AD is after death, then what is the period between?” Dam cool. My history teacher, Mrs. Annie Mathews had taught us this in the fifth standard. AD is not after Death. It’s Anno Domini. I remember that class very well. I told Jispa that it’s not after death, it stands for Anno Domini. But somehow she remembers me teaching her in some museum that AD stood for after death. I wouldn’t normally teach her, as this is my favourite GK question, but if Jispa was so sure, then I let it rest. “sorry, if taught wrong, but here is the truth.” I then went online and explained to her what Anno Domini stood for (the year of the lord) and that in all probability there is a miscalculation and Jesus was not born in 0 BC.