Usability Testing Methods

When you’re launching your digital product, you need to know as much as possible about it. That includes insight into the design. Failing to test your design is like sifting through sand dunes blindfolded and handcuffed looking for a quarter. What you want instead is to test your design thoroughly and repeatedly until it satisfies your target audience. You need iterative usability testing. In this article, we will explore different types of usability testing methods and when to use which method.

So what is the purpose of Usability Testing? 

If the sand dune metaphor didn’t convince you, let’s dig into a bit more detailed justification.

Usability testing is an important part of your product’s lifecycle. It’s where you test if the design resonates — or not — with your preferred audience.

In essence, usability testing helps you save lots of money. Imagine moving straight into development with an untested design only to find out you need to do an ocean of changes for the design to match your target audience’s taste.

Don’t ever skip usability testing!

Before we delve deeper into the topic, check out some general info about usability testing:

  • Usability testing replicates how users behave inside an application. It can give you actionable and highly valuable insight before development.
  • Consider testing your designs very early on, starting with wireframes and prototypes. Validating the design should be an iterative process — you build the design based on the feedback you get from usability testing.

How to Prepare for a Usability Test?

All right, so I’ve got your attention. You know usability testing is important and all the good it does to your product, but you’re probably wondering how to conduct a usability test?

Let’s start with the basics first.

The first step is to agree with the stakeholders on the goals for usability testing. Discuss and work out what are their fears and concerns, as well as specific points to include in the testing.

Narrowing down the goals will help you figure out which usability testing method to use.

These goals can be to find:

  • How visitors navigate the product
  • How users perform a specific task
  • How long they perform a specific task
  • Whether the design and content are understandable with action items easy to locate

Why is determining goals important? Usability testing lets you measure design-focused and behavioral data about your design.

So if you want to know what people think about your design or product or collect attitudinal user information, you need usability testing and full-fledged user experience (UX) testing.

Some UX testing methods:

  • Surveys
  • A/B testing
  • Focus groups
  • Heat maps
  • Individual interviews

Note: Pairing usability testing with UX testing will give you a broader and more accurate picture of your product and design.

Usability Testing Methods

There are six methods of usability testing, each giving you a detailed insight into a range of questions regarding your target audience.

The usability testing method you choose will depend on your resources and your objectives.

#1. Lab-based and Guerrilla Testing

Lab-based in-person usability testing is one of the most effective methods to validate your design. It’s convenient — you invite participants to a single location. And effective — you have all the tools to gain in-depth insight.

But some projects require you to test the design in the user’s location or environment (because, e.g., the user setting is so specific that it can’t be replicated in a lab setting).

Keep in mind that usability testing in the user’s environment will likely take longer to complete than other test types.

#2. Moderated and unmoderated

Moderated usability tests are run by a facilitator who conducts the testing. The facilitator shows the participants how the test works and addresses their questions and uncertainties.

The benefit of moderated testing is that the researcher running the test can ask follow-up questions for clarification and elaboration. What’s more, participants can perform these tasks on their devices while connecting with their researcher telephonically or via a video conferencing app like Zoom and Skype.

As a result, moderated tests yield insight-rich results. However, the main drawback is the relatively high cost of organizing such testing (you need an experienced facilitator, venue or lab, and compensation for participants).

Unmoderated testing is done without a facilitator. Participants can test designs in a lab or at their location.

While the benefits of unmoderated testing are its low cost and quick results, the drawback is the accuracy of its results. Without a facilitator, there’s no option to ask custom follow-up questions for clarification.

Note: Cost-wise, unmoderated remote usability tests might cost you around 30% less than moderated ones. They will also take less time to complete, which can be a huge benefit when you need to validate your design quickly.

#3. In-person and remote

Remote Unmoderated - Flow of Information

In-person usability testing offers researchers greater context and insight — they can observe body language and facial expressions. It’s usually done during moderated lab-based testing but can also be run remotely by a facilitator via a tool.

While remote testing won’t give you as deep an insight as in-person testing, it will give you access to a greater number of participants scattered over different locations at a much lower cost.

Lookback dashboard for remote moderated usability testing
Lookback’s dashboard for remote moderated usability testing. Source: Lookback

It can also help you hire participants that are difficult to find, for example, during travel restrictions.

#4 Card sorting

Card sorting is a method for prioritizing content and features in user interface. You have to place the concepts on virtual note cards, and then participants are allowed to manipulate the cards into the groups and categories. After the cards are sorted, participants explain their logic.

#5 Session recording

Session recording is a method of recording the actions that real users take such as movement, scrolling while they interact with a site. Session recording data helps to understand what content/features are engaged most as well as what interaction problems users face while they interact with your app.

#6 Eye Tracking

This usability approach lets you observe and evaluate a user’s eye movements through the eye tracking technology. Researchers analyze skimming patterns to understand the direction in which a user is looking, the element they’re looking at, and for how long. It enables them to unveil usability challenges without interrupting a user’s natural flow.

When to use which usability testing methods?

Use moderated in-person or remote usability testing (either remote or in-person) to test your initial prototypes and wireframes. This way, you’ll be able to pinpoint usability issues in the design very early on.

Moderated usability testing is especially recommended for complex tasks when it’s necessary to ask follow-up questions and conduct in-depth interviews with users to understand their behavior.

Remote unmoderated tests are a go-to method when you’ve gathered initial feedback from in-person or remote moderated usability testing and need to improve the flow of the design.

You can use one of the many online tools for remote testing:

Some tools offer the recording of a session, with included commentary from the participants.

Sample remote usability testing tools:

How to recruit users for Usability Testing?

The key to meaningful usability testing is getting participants that are a close match to your target audience or personas.

Remember, in usability testing, we’re replicating the real-world use cases of your design, so the participants have to represent your audience.

This is especially important for specialized and unique designs — testers have to match your personas closely for valuable feedback.

That’s why you need to set up criteria for your participants. Use demographics from your personas.

Sample demographic criteria:

  • Age
  • Gender
  • Education
  • Occupation
  • Location
  • Income
  • Technical literacy

If you’re developing a product that caters to a specific group of professionals, e.g., healthcare providers include other criteria that help recruit best-matching testers.

Ideally, you’re looking at five testers, with three being the minimum.

Where to look for testers?

Hire your clients. Turning to your existing clients for help is a great way to get high-quality feedback from your perfect audience. Your past and current clients offer a wealth of insight, especially if you want to check recent changes in design.

Hire an agency. An agency might be your best shot when you’re just starting out and have no existing clients. An agency is also helpful if you need to run your design through a very specific target group, e.g., aerospace engineers.

Hire users through your website. If you already have a website, just include a pop-up or other easily noticeable call to action to ask visitors whether they would be willing to participate in testing.

Internet forums. If you don’t have the resources for a recruitment agency, you can look into specialist internet forums to hire professionals for your usability test. 

Your network or social media. You can consider recruiting connections from your network. But be careful to avoid connections who know you, as this could lead to biased results.

Recruitment tools. Recruitment tools will help you determine your ideal user base for usability testing. Recruitment tools are a great source of highly specific testers that are relatively easy to recruit via the platform.

Sample recruitment tools:

Recruiting testers for Usability Testing
Recruiting testers via User Interviews. Source: User Interviews

How to Run a Usability Test?

By now, you know the benefits of usability testing, what you want to learn, various usability testing methods, what type of test to run, and how to recruit testers.

“But, how to do usability testing?” you may ask

– Prepare test scenarios. These have to be vague — you don’t want to lead the testers in a specific direction nor do you want to give them instructions.

For example, a task to test the design for an e-commerce app would look like this:

  • Search for a product
  • Log in
  • Complete a payment
  • Complete a checkout

Tip: Avoid jamming as many tasks as possible — it’ll make drawing conclusions more difficult. Instead, prioritize tasks based on their importance in relation to your initial objectives. Assign no more than five tasks per single participant. 

The key is to keep the tasks simple and as short as possible.

But, you can and should give those tasks some context and make them feel natural. For example, “You want to buy your partner an anniversary gift. Browse for a coffee siphon for 3 cups.”

– Establish realistic goals for each task. For example, users should be able to complete the checkout in under one minute.

Usability Testing Questions: which ones to ask for best feedback?

A rule of thumb in usability testing, especially in the in-person moderated variation, is to always try to learn as much as possible about the experiences of testers.

This is done best via follow-up questions. Prod and encourage testers to reveal their motivations for performing steps in the way they did. When prompted, users are more likely to give a detailed explanation.

Questions like the ones below should help illuminate what the user is doing, and why:

  • Which of these two approaches/options do you find best? Why? – This is useful if you’re trying to determine the more appealing of multiple options.
  • Did you notice whether there was any other way to ___? – You are trying to determine why the user did one thing instead of another.
  • I noticed you did ____. Can you tell me why? – Follow up on any interesting behavior you observe during the test to get a better idea of the thought process behind the user’s actions.

Mobile Patterns - Write for Us

How to analyze and interpret the results?

After that first round of testing, you should have plenty of valuable data to analyze and draw conclusions for the next interactions of your designs.

Before you jump headlong into an analysis frenzy, take a step back and review the goals you’ve set out when preparing for a usability test.

What were the key areas you wanted to test? This step is essential because you’ll use those key areas to categorize the insights.

That said, go over the data and note down the issues that testers have cited the most. But remember, only focus on the biggest issues for now.

What to look for in the data:

  • Issues testers had when completing tasks
  • Steps users took to complete tasks
  • Responses to follow-up questions

With a summary of results ready, sit down with the design team — and preferably the development team, as the two should work together to agree on viable design suggestions — and brainstorm on the possible solutions.

Remember to prioritize issues according to your key areas. The goal now is to adjust the designs to improve the user experience

Design, test, implement, and iterate

Regular usability testing will help you develop your designs in tune with your audience’s needs and inherent motivations. Iteration-based design is key to building great digital products that engage users and boost customer retention.

Tips on usability testing methods:

  • If you’re just starting and have limited resources to launch a full-fledged moderated in-person testing session, try out some of the online usability testing tools listed in the article.
  • Or you can go for guerilla testing — simply ask random people on the street or in a coffee shop what they think about your design. While it won’t yield as much insight as regular usability testing, it’s still better than forgo testing at all.

Testing might seem difficult at first glance, but it’s effortless when you set up specific goals, pick the best usability testing method, create real-life scenarios, and aggregate results.

Maja Nowak

Maja Nowak oversees content production at Nomtek. We translate the wants of your customers into products that change how things are done. Our solutions help companies acquire new customers, showcase product capabilities, and convince investors. We’ll take your idea on a journey toward becoming a great digital product.

Previous ArticleNext Article

Leave a Reply

Your email address will not be published. Required fields are marked *