False prophets of Usability - Part 1
In the past few years usability has become somewhat of a buzzword. That's both a good and a bad thing. What's bad is that the meaning of usability gets muddied, re-interpreted and sometimes even set aside. You have software companies selling "automated usability testing solutions" (no one really sells products anymore). You have traditional graphic designers passing themselves off as "interaction" this and "user experience" that. And of course most of the "buyers" don't know what is what -- they're just buying instant 'usability" -- or so they think. I'm not saying I'm perfect, but there are clearly some folks in the "usability game" who are just jumping on a bandwagon and bringing their own version of snake oil with them.

Here's an example:

NetConversions has some pretty neat software to sell. It's mostly event logging / data capture type of stuff. It could be very useful to a business or a usability engineer doing research. But, listen to this claim from the NetConversions site:

"True Usability™ is an innovative and rapid data driven method to test for website usability using your actual visitor traffic to improve the user experience and to optimize bottom-line results." [November 2005 Update: Netconversions is now "Atlas" and is still offering True Usability™]

Note that they've co-opted and trademarked "True Usability" as their product name. Of course this really has little to do with "truth" or "usability" as you'll soon see. They have a "white paper" on their site claiming to compare various "approaches to usability": heuristic evaluations (aka "experts"), focus groups, surveys, server logs, and of course "True Usability™". This supposed comparison has more errors in it than I can cover here, but here are the highlights:

Supposedly a heuristic evaluation is "an analysis of the site with respect to a set of usability guidelines. These usability guidelines are often based upon the expert’s past testing and consulting experiences." -- No, a heuristic evaluation is based on heuristics. A guideline review is based on guidelines, and most "experts" don't create their own guidelines from scratch. Clearly they don't know what they are talking about. Looking at their "about us" page, there isn't a person listed with a background in HCI, Usability Engineering or User Centered Design -- instead they are technologists and marketing data research types. Don't get me wrong -- I like those types of people, but I know usability professionals when I see them, and I don't see any at this company. If they're really selling "usability" expertise, shouldn't they show their qualifications in the field?

The "comparision" opens with a very nice quote about usability testing from the Industry Standard.-- evidently to promote NetConversions' product. Of course the types of usability testing Usability Engineers have been doing for decades -- involving humans observing and listening to users isn't even mentioned in this comparison. It seems anything that can help you evaluate usability of a system is now considered "usability testing".

Unfortunately NetConversions isn't the only company doing things along this line, and even many usability professionals can fall into the trap of promising to solve all of a customer's problems in one fell swoop. Usability (as a field) suffers when "experts" don't meet expectations. We have to set realistic expectations with clients, help them select the tools, methods and approaches that meet their needs. We can't over-promise and under-deliver. Perfect usability is a goal never attained, but great strides can be made in that direction if businesses, designers and developers work together.

There's plenty of bad usability out there -- more than enough for all the traditional usability professionals and the new entrants to the "usability game" alike. We can work together -- tool-makers and practitioners, researchers and designers, marketers and engineers. Let's just quit promising that we can get companies to the moon on the next bus leaving town.

Be realistic, helpful and truthful...and maybe Real Usability™ will happen.