February 07, 2003

Web users' navigation behaviors changing and findings from testing search query formulation
A new StatMarket survey shows a couple of interesting trends:

- 64% of users now go directly where they want to go by typing in URLs and using bookmarks, while only 35% get there via surfing (just following links)
- Similar surveys done in February 2002 and 2001 showed surfing much higher - at 47% and 52% respectively.
- 13% of users arrived at a site via a search engine, up from 8 percent in 2002. So, while search engine referrals are growing, it's link-to-link navigation that is decreasing

Here are a few excerpts from an IDG article covering the survey results:

"Meandering is decreasing. This is a sign that the market is maturing," said Geoff Johnston, vice president of product management for the StatMarket division of WebSideStory Inc. which released the report Thursday.

While the Internet's cornucopia of information may have left Net users starry-eyed in the early days, users are now determined to get down to business rather than browse."

Not only is search engine use growing, people are getting better at using them, said Matthew Berk, a senior analyst at Jupiter Research.

Johnston ... comparing the Internet to TV. "After a while you get tired of flipping through the channels and just turn to the programs you like," he said."


Are users really better at searching?
I don't agree that with the statement that "people are getting better at using" search engines. My take is Google has improved the effectiveness of web search engines -- making even their competitors serve web searchers better by focusing on improving search algorithms and ranking methods. Maybe a very small percentage of web searchers have become more savvy about search query formulation or which engine to use, but I think it's more likely that the usability improvements made in search engines have accounted for users trusting them to meet their needs more often.

My experience shows that most users aren't very sophisticated in their approach to using search. For example, when usability testing an intranet site recently, we wanted to test how people would try to find the site on an internal search engine. We asked people to search for the site, knowing that it wasn't in the search engine's index. Our goals were two-fold:
1) see what query terms users would enter so they we could make sure those terms worked later, and
2) see how many different terms users would try before giving up.

The results amazed me. Many users totally gave up after just one or two search attempts, and no one made more than four search attempts. It was a very useful exercise - we ended up with a huge list of key query terms that we could focus on -- making sure those queries would find the site in the future. We also learned just how important first and second search result sets are, and that users will NOT be patient or successful with a poor search engine.

You might give that exercise a try on your next effort. You just have to reassure the test participant so they don't feel bad about not completing the task. We actually admitted to our participants right after that task that there was no way they wouldv'e found the site that day. We then explained why we set the test up that way, and how their struggle with different queries helped us. We also reassured them that there were no more 'trick' tasks like that in the study. This seemed to help them regain any lost confidence.

Related:
- StatMarket page covering the survey results (Note StatMarket links haven't been reliable in the past.)
- Jakob Nielsen's Alertbox - Search: Visible and Simple - note Jakob talks about observing how effective users are at query formulation.
- 37signals E-Commerce Search Report - a review of 25 online retailers' search engines that identifies 22 best practices.

February 04, 2003

Mad as hell
"It's as if the user interface folks have had their clocks flashing 12:00 for two full decades."
- TechWeb: Putting A Bad Interface On Things