This is a translation of my Dutch article “Findability testing: de ontbrekende schakel“, originally published on 14 January 2008 at Frankwatching.com, a leading Dutch blog on digital trends and online marketing.
Usability testing has become a well-known evaluation method for websites and applications. It is valuable, but it doesn’t address an important aspect of ‘user experience’: the path to the website. “How do people search for my content and will they find my website?”
Many of my projects are aimed at (functional) designing a new or improved web site. If possible, I organize a usability test. We invite some people who represent the target group of the website, guide them to the homepage and give them several tasks, based on realistic or real scenario’s.
We learn how users navigate on the site, what they understand or not, where they get stuck, what they like and more. But, if we let them start at the homepage, we skip an important aspect of the user experience: how easily do people find the way to the website?
“Findability precedes usability
in the alphabet and on the web.
You can’t use what you can’t find.”
Peter Morville – “Ambient Findability“
When it is about how people enter a website, we often turn to web statistics, which tell us:
- how people enter the web site (directly, referring website or search engine).
- which search keywords are used to find and visit the site.
- which pages are the top entry pages.
You can also try querying the search engines yourself and see how well your web site is findable on each search phrase.
Both methods have their limits. Web stats only show information on people who actually visited your site. And querying search engines yourself says only how well you are found (and possibly visited) on keywords you can think of. Other people search different than you, especially people who are not familiar with your company or your jargon. And besides, how do you know if a top-10 position on Google guarantees visits? Do your domain, title and description trigger people to actually click?
To get a better picture of your actual findability, observe.
Case: Going to court
Rechtspraak.nl is the joint web site of all law courts, appeal courts and the supreme court in the Netherlands. Primarily aimed at legal professionals, but also increasingly relevant to the public. The Dutch Council for the Judiciary has started a project to better inform and prepare people who go to court.
Much information was already available, such as web stats, usability tests, surveys, e-mail questions, and some reports. However, we didn’t know how the public uses the internet when they prepare for court. For this, we started a new research project.
The setting was identical to that of a usability test: We invited people from the target group and seated them at a computer equipped with eye-tracking and other registration software. We gave them a fictitious subpoena and watched their reaction. Which questions arose? What actions would they take?
Then we asked them to try and find the answers to their questions online. The participants didn’t know which website was the research was for. They started on a blank page and were free to choose their own starting point and follow their own search strategy. All participants started their quest on Google, which was – considering the matter – to be expected. From there, their exploration started. I’d like to share some of our observations.
It became immediately apparent that the participants had a difficult time finding information on legal procedures and formalities. For non-jurists it’s an unknown and complex domain. Partly due to that, the participants’ search strategies differed much. Some copied words from the subpoena like “terechtzitting” (court session), others searched on the root of their problem, like “huurschuld” (rent arrears) or “echtscheiding” (divorce) and some even queried entire sentences: “Hoe gaat een terechtzitting in zijn werk” (How does a session of the court work).
For most search queries rechtspraak.nl didn’t appear in the results. This was not unexpected, because the participants’ queries were much more specific and concrete than the content on the website and because laymen use different words than jurists. This is valuable knowledge. If you know how people search for ‘your’ information, you can optimize your website for it.
People are not likely to question the quality of their own search queries, have much confidence in Google and don’t pay much attention to the sender or authority of a website. So people assume that any top search result is relevant. Sponsored links often look very relevant as well, because the query words are regularly included in the title of the result and because they appear at the top, before the organic results. In the research we saw how people clicked on top results (both sponsored and organic) that weren’t very relevant to them or authoritative.
The first sponsored link gets many clicks because the title matches the search query better than the first organic result.
A top position on Google doesn’t automatically mean that everyone finds your site. In the research people searched for “rechtbank Amsterdam” (court of Amsterdam). We observed some of them skipping the – top listed – official page on rechtspraak.nl to visit the Wikipedia entry on the court of Amsterdam (listed no. 3). Possible explanations for this behavior:
- People (even in the Netherlands) are more familiar with the brand ‘Wikipedia’ than with ‘Rechtspraak’ (judiciary).
- People have much confidence in the quality and authority of encyclopedias, including Wikipedia (although many Dutch jurists agree that some information on legal subjects in Wikipedia is erroneous or at least not accurate).
- The Wikipedia result title starts with “Rechtbank Amsterdam” (the search query), whereas the Rechtspraak.nl result title ends with those words. This might have an effect on the perceived relevance of each link.
Theoretically, the content of the short description of a result can be another reason why a lower ranked result is conceived to be more relevant than a higher ranked result. But this doesn’t apply to the example given, because the two descriptions are almost the same.
Furthermore it became apparent that for the public not the homepage, but the first page of a court is the most popular entrance. This is because this page type is one of the few for this website that people actually can find, using their own logic and wording.
All this is great input for the new information architecture of the website.
I have begun to call this method ‘Findability testing’. As far as I know, this term hasn’t been really used so far (Google found only 10 occurrences, against 1 million for ‘usability testing’), it covers the subject and it sounds like ‘usability testing’, which is ok because the method is much like usability testing.
Here is a first attempt to a definition:
Findability testing is a qualitative research method in which representative users freely search online (information) within a given focus area. The method is used to gain insight in:
(a) the information need and the consequent search strategy of the target group, and
(b) the findability of a specific website in this context.
Tests are conducted similar to usability testing:
- Recruit respondents who represent the target group.
- Arrange an informal and quiet setting for the test, making the respondents at ease, what hopefully stimulated more natural behavior.
- Work with realistic scenario’s, or even better: have the respondents suggest a scenario by wording their own information need.
- Steer as little as possible.
- Avoid communicating terms that the website you work for, uses.
- Do not immediately direct them to the internet, but start with “What are you going to do?”. Take multi-channel behavior into account: in some situations people prefer calling, visiting or consulting family and friends. In the second instance you can direct them to the internet: “Suppose it’s Sunday, they are closed and you need an answer right now; can you try and find your answer online?”.
- Occasionally ask questions to gain insight in the respondents’ search strategy.
- Ask questions after every visit of a website, to learn why they spent much/little time there, if the site was useful to them, etcetera.
- Consider using eye-tracking for even more insight in search en surfing behavior. Not to create heat maps, just to follow eye movement during the test.
I believe findability testing is a valuable addition to research methods like usability testing and web analytics, for designing and managing a successful website. Too often search engine optimization is being approached from the inside out and from the website’s statistics on visitors. But what if your visitors are just a fraction of your searchers? That is where findability testing comes in.