Tuesday, December 27, 2011

Statistics: A Tricky Measure

“Statistics can prove anything, even the truth” - Noel Moynihan

Many people would agree with the statement, “Life would probably be less meaningful if we discount statistics from it”. Even if you were to disagree, I fervently believe that most professions of today wouldn’t even exist had we been less fascinated with statistics. And if you talk to anyone who has done research of some sort, they will confirm the importance of statistics.

So, why is statistics so important to us? Scholars believe that statistics plays a vital role in every field of human activity. Humans in general desire predictability, consistency, and assurance to make better decisions. Essentially statistics quantifies and represents these factors in a more visually comprehendible manner. In spite of this huge advantage that it offers, I’ve always felt that most outcomes based on statistics leave plenty of room for ambiguity. Consider the below scenario.

I recently read an article titled ‘disappointing facts in popular music’ in which the author had posted a fact that “Flo Rida’s (an American rapper & singer) song ‘Low’ sold same number of copies (8 million) as that of the Beatles’ (one of the greatest band of all time) song ‘Hey Jude’” . Now here’s some food for thought. People, who view this fact as a regrettable one, would claim that the quality of music has gone down since the 70’s. That could only be the possible explanation for Flo Rida’s song to match the success of the Beatles’ song. Those who like the rap music, prefer taking a tequila shot, and hit the dance floor, might actually feel that Flo Rida is equally good, if not better, than the Beatles and would cite the same statistics to validate their claim. Then there are those make analysis might include factors such as increase in population from the 70’s to now and the power of Internet to propagate popularity, and conclude other way. Probably, using the same statistics, we can argue from either side. A lot depends on which side are we standing.

For a long time I believed if there is someone who defied this ambiguity that statistics generates was the great cricketer Sir Don Bradman. Recently, a researcher published some facts, using the same available statistics, and concluded that Sir Don was second best batsman, not the best. That is when it became very clear to me that statistics operates more in tandem with ‘confirmation biases’ than logic. In such a case, relying blindly on statistics would seldom help us make effective decisions. The reason I call statistics as a tricky measure is because though there is this visible shortcoming, bright minds have always been able to use statistics effectively.

PS: This article has references to some subjects and people, and could be better understood through the below links:

The Beatles, Hey Jude, Flo Rida, Low, Don Bradman, Confirmation Bias

4 comments:

Sutapa Dey said...

Hmmmmmmmm a sort of technical blog from you :). Ok my questions and observations as follows:
"For a long time I believed if there is someone who defied this ambiguity that statistics generates was the great cricketer Sir Don Bradman" - I didn't get how Sir Bradman defied the ambiguity. As far as I understood, some researcher projected him as second best. But did Don Bradman do to defy the statistics :P.
Also, when it comes to relying on statistics for making effective decision, it is actually not that bad an idea. Because there may be cases where we do not have a bias. For example, say if we want to check out a popular cosmetic product, we go by user reviews for good and bad. That time, we hardly be biased or not biased to the product. ;) Hopefully I didn't confuse you with my contradictory statements. :P .......... But it was a nice change of genre for your blogs. :) I like it.

Anonymous said...

Agreed. Stats can be used as a means of summarizing things and giving a brief picture. But should never be taken as a whole entity especially when comparing two objects/people of different eras.
-Tejas

uday said...

I agree with Sidhu who says " statistics are like mini skirts, they hide more than they reveal " . They are ok for the first impression of a person or a product but after that we really can't say.

Vijay Nadadur said...

@Sutapa: I would hope that you went through the Wiki link for Sir Don Bradman. If not, please go through it. You will realize how dominant he was, not only in terms of cricket but in terms of any ball game ever played. Statistics do a fair amount of justice in his case, only if people didn't do their further analysis and change the way we interpret them

@Tej: Can't agree any more with your comment. Stats give an overview but are never substantial.

@Uday: Very good comment Highly. It's like, beyond one point stats cannot do any good.