Tuesday, December 27, 2011

Statistics: A Tricky Measure

“Statistics can prove anything, even the truth” - Noel Moynihan

Many people would agree with the statement, “Life would probably be less meaningful if we discount statistics from it”. Even if you were to disagree, I fervently believe that most professions of today wouldn’t even exist had we been less fascinated with statistics. And if you talk to anyone who has done research of some sort, they will confirm the importance of statistics.

So, why is statistics so important to us? Scholars believe that statistics plays a vital role in every field of human activity. Humans in general desire predictability, consistency, and assurance to make better decisions. Essentially statistics quantifies and represents these factors in a more visually comprehendible manner. In spite of this huge advantage that it offers, I’ve always felt that most outcomes based on statistics leave plenty of room for ambiguity. Consider the below scenario.

I recently read an article titled ‘disappointing facts in popular music’ in which the author had posted a fact that “Flo Rida’s (an American rapper & singer) song ‘Low’ sold same number of copies (8 million) as that of the Beatles’ (one of the greatest band of all time) song ‘Hey Jude’” . Now here’s some food for thought. People, who view this fact as a regrettable one, would claim that the quality of music has gone down since the 70’s. That could only be the possible explanation for Flo Rida’s song to match the success of the Beatles’ song. Those who like the rap music, prefer taking a tequila shot, and hit the dance floor, might actually feel that Flo Rida is equally good, if not better, than the Beatles and would cite the same statistics to validate their claim. Then there are those make analysis might include factors such as increase in population from the 70’s to now and the power of Internet to propagate popularity, and conclude other way. Probably, using the same statistics, we can argue from either side. A lot depends on which side are we standing.

For a long time I believed if there is someone who defied this ambiguity that statistics generates was the great cricketer Sir Don Bradman. Recently, a researcher published some facts, using the same available statistics, and concluded that Sir Don was second best batsman, not the best. That is when it became very clear to me that statistics operates more in tandem with ‘confirmation biases’ than logic. In such a case, relying blindly on statistics would seldom help us make effective decisions. The reason I call statistics as a tricky measure is because though there is this visible shortcoming, bright minds have always been able to use statistics effectively.

PS: This article has references to some subjects and people, and could be better understood through the below links:

The Beatles, Hey Jude, Flo Rida, Low, Don Bradman, Confirmation Bias