Usability Testing Methods: How to Choose the Right One

The main usability testing methods, what each is good for, how many users you need, and how to pick the right one for your question and budget.

Bretton Badenoch · AI researcher, University of Michigan · Founder, CanaryUsers··3 min read

The main usability testing methods split along two axes: moderated or unmoderated, and remote or in-person. Moderated tests let a facilitator probe in real time; unmoderated tests scale cheaply. You rarely need a crowd: testing with about five users per audience surfaces most of the serious problems.

What is usability testing?

Usability testing is watching real people attempt real tasks on your product to find where they struggle. The goal is not opinions, it is behavior: where people hesitate, misread, click the wrong thing, or give up. A handful of well-run sessions can reveal problems that months of internal debate miss.

How many users do you need?

Fewer than most people expect. Nielsen Norman Group's long-standing finding is that testing with about five users uncovers roughly 85% of usability problems, because the same issues repeat quickly. If you have several distinct audiences, test five per audience rather than fifteen of one. Spend the saved budget on more rounds, not more people per round.

Moderated vs unmoderated testing

This is the first choice to make.

Moderated Unmoderated
Who runs it A facilitator guides live The user works alone, recorded
Best for Complex flows, probing the "why" Quick, scaled, directional checks
Speed Slower, scheduled Fast, on the participant's schedule
Cost Higher NN/G estimates 20% to 40% lower

Moderated testing wins when you need to ask follow-up questions and understand reasoning. Unmoderated testing wins when you need many sessions quickly and the tasks are clear enough to stand on their own.

Remote vs in-person

Remote testing is now the default for most teams. It reaches participants anywhere and is cheaper and faster. In-person testing gives richer body-language signal and tight control of the environment, which matters for hardware, sensitive topics, or specialized users. For most software, remote is enough.

Common method types

  • Task-based usability testing: give a realistic task and watch the attempt. This is the core method.
  • First-click testing: measure where people click first. A correct first click strongly predicts task success.
  • Tree testing: test your navigation and labels with no visual design, to see whether people can find things.
  • Five-second testing: show a page briefly to check what people remember and understand.

Pick the method that matches your question. "Can they finish checkout?" calls for task-based testing; "is our menu labeled right?" calls for tree testing.

Where AI user testing fits

Recruiting and scheduling are the slow, expensive parts of every method above, and you cannot run any of them before you have a product and participants. AI user testing fills that gap: it sends simulated, behaviorally diverse users through your live page and flags likely friction in minutes, with no recruiting and no traffic. It does not replace watching real humans on high-stakes questions, but it is a strong fast first pass and a continuous check. You can run a free scan and see where simulated users drop off before real ones do.

The takeaway

There is no single best method, only the right one for your question and budget. Decide moderated vs unmoderated, remote vs in-person, and the method type, then keep the rounds small and frequent. Five users, run often, beats one big study a year.

Frequently asked questions

How many users do you need for usability testing?

About five per audience. Nielsen Norman Group found that five users uncover roughly 85% of usability problems, so it is better to run several small rounds than one large one.

What is the difference between moderated and unmoderated usability testing?

In moderated testing a facilitator guides the session live and can ask follow-up questions. In unmoderated testing the participant works alone while the session is recorded. Unmoderated is faster and cheaper; moderated gives deeper insight.

Is remote usability testing as good as in-person?

For most software, yes. Remote testing is faster, cheaper, and reaches more people. In-person still wins when body language, environment control, or specialized hardware matters.

Can AI replace usability testing?

Not fully. AI user testing is an excellent fast first pass and a continuous check, but watching real people remains the gold standard for high-stakes decisions.

Keep reading

Sources

Bretton Badenoch

Written by

Bretton Badenoch

AI researcher, University of Michigan · Founder, CanaryUsers

Bretton Badenoch is an AI researcher at the University of Michigan and the founder of CanaryUsers. His research is in machine learning and aging; he has also built and run several startups as "chief-everything-officer," shipping products and obsessing over why users drop off, the problem CanaryUsers now automates.