Bad Statistics Give Me Chest Pains
On the radio last week I heard a local news reporter say of Bill Clinton that "90% of some of his arteries were blocked" (prior to his quadruple bypass). I grimaced and groaned and I think I might have made some derogatory remark about the reporter's intelligence. Ninety percent of *some*???!!! Nothing like being exactly vague with your statistics. Note also that the reporter wasn't specific about which arteries...here we're talking about coronary arteries...not just any old arteries pushing blood around the body.
Most reports I heard were more accurate and more successful at communicating a technical statistic to the average "Joe" on the street. For example, this AP photo has a good caption that more accurately states the stat:
"Clinton was at high risk of a heart attack before his quadruple bypass surgery Monday, with several arteries well over 90 percent blocked"
Doing a bit of Googling turns up more inaccuracies in reports of the same story:
"His arteries were 90 percent blocked."
from KABC-TV Los Angeles: Doctors: Clinton Dodged a Major Bullet
...all of his arteries?
"Monday's surgery revealed that his arteries were 90 percent blocked."
from WCVB-TV Boston: After Clinton Scare, Docs Urge Heart Vigilance
...exactly 90%? And again, all of his arteries?
"Clinton remained in intensive care after cardiologists performed a four-hour operation Monday to bypass four clogged arteries. They were so severely blocked that less than 10 percent of the normal blood flow was getting to his heart..."
from New York Post Online: CLINTON'S RECOVERY GOES WELL, DOCS SAY
...Wow! We're not talking about a few arteries that are mostly blocked, we're talking about 90% less blood flow in total to the heart!!!???
When dealing with statistics, whether related to the health of a past President or a recent usability test, it's important to maintain accuracy. Don't try to quote an exact stat unless A) you can get the stat correct, and B) the audience will be able to follow and understand the stat's context and content.
It's one thing to say "in testing we found that most people didn't use the site map" and a totally different thing to say "our tests and third party research show that 73% of users will not find a site map or site index useful in locating detailed product information on consumer web sites." Stats can be difficult to understand, so sometimes having them in print or on a slide can help people understand the stats. (Note, that stat example is entirely made up...)
Finally, if you find yourself trying to summarize a stat, be careful that you're not changing the meaning (as many of the news reporters on the Clinton story did).