September 16, 2004

Bad Statistics Give Me Chest Pains

On the radio last week I heard a local news reporter say of Bill Clinton that "90% of some of his arteries were blocked" (prior to his quadruple bypass). I grimaced and groaned and I think I might have made some derogatory remark about the reporter's intelligence. Ninety percent of *some*???!!! Nothing like being exactly vague with your statistics. Note also that the reporter wasn't specific about which arteries...here we're talking about coronary arteries...not just any old arteries pushing blood around the body.

The facts:
Most reports I heard were more accurate and more successful at communicating a technical statistic to the average "Joe" on the street. For example, this AP photo has a good caption that more accurately states the stat:
"Clinton was at high risk of a heart attack before his quadruple bypass surgery Monday, with several arteries well over 90 percent blocked"

Doing a bit of Googling turns up more inaccuracies in reports of the same story:

"His arteries were 90 percent blocked."
from KABC-TV Los Angeles: Doctors: Clinton Dodged a Major Bullet
...all of his arteries?

"Monday's surgery revealed that his arteries were 90 percent blocked."
from WCVB-TV Boston: After Clinton Scare, Docs Urge Heart Vigilance
...exactly 90%? And again, all of his arteries?

"Clinton remained in intensive care after cardiologists performed a four-hour operation Monday to bypass four clogged arteries. They were so severely blocked that less than 10 percent of the normal blood flow was getting to his heart..."
from New York Post Online: CLINTON'S RECOVERY GOES WELL, DOCS SAY
...Wow! We're not talking about a few arteries that are mostly blocked, we're talking about 90% less blood flow in total to the heart!!!???


When dealing with statistics, whether related to the health of a past President or a recent usability test, it's important to maintain accuracy. Don't try to quote an exact stat unless A) you can get the stat correct, and B) the audience will be able to follow and understand the stat's context and content.

It's one thing to say "in testing we found that most people didn't use the site map" and a totally different thing to say "our tests and third party research show that 73% of users will not find a site map or site index useful in locating detailed product information on consumer web sites." Stats can be difficult to understand, so sometimes having them in print or on a slide can help people understand the stats. (Note, that stat example is entirely made up...)

Finally, if you find yourself trying to summarize a stat, be careful that you're not changing the meaning (as many of the news reporters on the Clinton story did).

September 15, 2004

UCD in Plain Language

An oddly titled article in Technology Marketing magazine - "Headline" - does a good job of describing key tenets of User Centered Design for web sites without getting into a bunch of usability lingo.

"Most sites are built backwards. They start with content. They organize the content. Then they publish the content and tell the audience it's there. This approach could be summarized in this way: 'Content, architecture, audience.' This is the most common process, and it results in sites that are impossible for you to manage and for your customers to navigate.

"Instead, try to start by defining the audiences for your content, then list the actions they want to take, and then you should develop the content that supports those actions. In summary: 'Audience, actions, content.' The audience and their actions determine the content. ...

"You already know whom you're trying to reach. Interview them to find out what actions they want to take and the content that will support those actions. What you learn will result in sites that are refreshingly intuitive for your audiences.

"The bad news? If you design your sites around the audience and their actions, the resulting content set will not include some of your favorite Content Babies. We all have them: that white paper we wrote until the wee hours of the morning, the brochure that took us a year to get approved, etc. We can't stand the thought of leaving our Content Babies out of our content set. But if they don't fit into the "Audience, actions, content model," you'll have to let them go, and replace them with the content that your audience actually wants."