NPR logo Usability Best Practices

Metrics for Humans

Usability Best Practices

The Case For Formal Usability Testing

Consumers consistently attempt to keep pace with developing digital technologies by creating new ways to navigate through the computer or mobile devices. However, these new technologies develop at breakneck speeds thus the consumer may feel as if they are lagging behind everyone else in the technology space if they don't know how to operate the latest gadget. However, it is not their fault and companies need to pay closer attention to the consumer or end customer when they develop products. As we often say before every usability test, we are not testing you but testing our ability to deliver an easy-to-use product.

Thus usability testing is not only appropriate but necessary in today's ever-changing digital world. There are companies that believe quality assurance testing a new product or website is all that is necessary. Those who know the problem better than anyone should be able to find all of the errors, right? Not exactly; they know the product TOO well. Rather they need to be most interested in the experience of the end client. If the customer uses a website in a way the designers didn't intend, he or she will inevitably get lost, frustrated, or, worse, give up (which is the worst thing that can happen if you want the end client to actually use your tools).

The new buzzword in the research industry in the past few years seems to be "customer-centric." So what should be the steps in creating a satisfactory usability experience? Let's start with some definitions.

The On-on-One
First, usability testing is, at its most basic, a one-on-one interview. The moderator goes through a series of tasks and questions about the concept in front of a target audience member. In other words, ask someone who will most likely use the product a bunch of questions to make sure that they can not only understand what the product can do but also why it is doing it.

Guerilla Testing
Second, there are other forms of usability testing including guerilla testing, where the moderator does not formally set up testing with respondents but tries to find them "in the wild" to ask them a few simple questions. This test is useful but should only be used in early stages of a design as the proper audience has not been screened prior to testing and it is not usually conducted in a sterile environment.

These tests allow researchers to closely watch the actions and reactions of the respondent and to ask questions about why they performed certain actions. Metrics and click-throughs only provide some of the story and it would be hard to write a research report without asking the customer, "Why did you do that?"

Research Tools
As with any research project, formal usability testing requires two things: the proper tools and the right population for sampling purposes. A computer or paper drawing will work just fine for basic usability testing because it gives the respondent something to react to for the testing. However, if you want others to see what is happening or need a recording to review later, then a camera, software, and an Internet connection are required tools. Remote testing can be even easier as conferencing tools based on a web browser and a phone number are inexpensive (or free).

The most critical piece of formal usability testing, after an appropriate stimulus to test, is the right respondent. How much can you learn if you just bring in those individuals that are great with computers? How much of a potential risk are you taking if you only bring in loyal customers to your tests? These are only just a couple of a myriad of questions that you need to ask when you are developing your sample.

Sample Size
Many people still believe the law of large numbers is the only way to go when it comes to sample size. Unfortunately, this is not true with usability research because the aim is not to figure out the mean of a belief or attitude in the public today. The goal is to understand the usability of a particular product.

Jakob Nielsen, the father of usability as some call him, says that you only need to test five users to identify 85 percent of the usability issues of any given web project. Similar results have been identified in other product groups, but the main point is that you don't have to test a lot of people. If you bump it up to ten users, then you have most of your answers. Find a cost effective point and use it.

Moderator Guide
Lastly, develop a guide that not only probes the tasks for the moderator to accomplish but also explores the background and the "whys." If a researcher only conducts task-based questions then they will miss the reasons behind the movements. If the focus is only on the background then the results will be scattered and will not coalesce into a nice story on how well (or badly) the product works. The user does not have a context to perform any tasks and will randomly conduct them without any structure. The report would then closely match these results.

Usability research is critical as it helps companies and organizations make critical decisions on how to make their products better for their customers. Whether you are making a drill or website it is important that product owners have a firm grasp on what their customers want and how they operate their products.

Scott Vanderbilt is the Research Manager for Digital Media/Emerging Platforms.