But how many users did you test that with?

How to convince stakeholders of the value of qualitative insight work.

When discussing and delivering research findings it can be tricky to successfully convey the value that can be achieved from a small sample size.

But, being able to explain that value goes a long way in establishing credibility for user research.

Is this scenario familiar to you?

You’re presenting the findings of customer research on the latest version of your designs. Some areas of the design need improving, but some performed really well and you are confident that they should be implemented in the next iteration of the product.

Someone interrupts: “I’m sorry… how many people did you test that with?”

The answer is often followed-up by: “Only 5 people? Is that enough?”

And then finally: “Should we really be making these changes to the product just based on the opinion of these 5 people?”

Maybe this is your first-time presenting research findings to your team. You feel pretty sure that your results are valid, but you aren’t sure of how to respond. Or maybe this is your hundredth-time presenting research findings, but you keep coming across this challenge.

Whether you are a beginner or an experienced design researcher you’re likely to come across this common question at some point in your career, if you haven’t already. We’re trying to provide a foundation of clarity and certainty, failing to do so may undermine our whole creative effort, resulting in a waste of time, resource, and money. I’ve had to answer this question hundreds of times, and want to share some tips on how to handle this question/situation when it happens.

Let’s spend a minute exploring the context of this question, and why it’s being asked, to understand what’s going on and how best to respond.

Think back to the last time that you were in a room where this happened:

In my experience, you will typically notice that this challenge comes from someone:

Why are they asking?

Often, if you take a closer look at who’s asking, you will see that there are three main characteristics that all of your sample-size doubters are likely to have in common. 

  1. They are busy people: You might have already told them what you’re doing and why, but they are likely managing multiple workstreams and have probably forgotten what’s happened before and what’s coming next in your specific programme.

  2. They are ‘numbers’ people: They are strategic thinkers, more used to working in the initial strategy phase and late launch phases of product development that rely on quantitative data and are focused on establishing risk, reward and levels of confidence in an idea or solution.

  3. They aren’t design folk: they don’t know much about the tools and activities of the design phases of a programme because they are not UX experts.

So how do you persuade them that there is value in what you are doing?

Remember, it's our duty as UX designers and design researchers to help other stakeholders around the business understand what we do and the value that it generates. So, take a deep breath and recognise that this question is actually presenting you with a brilliant opportunity to grow the knowledge of UX design tools and techniques in your organisation and can help to pave the way towards greater UX maturity.

Here are some tips on how to prepare for this type of meeting and respond to the question when it comes up: 

Now you should be better equipped to face your stakeholders confidently, but tactfully, when discussing and delivering research findings. Remember, not everyone understands the value of qualitative research findings and it can be tricky to convey results successfully, but answering challenges head-on will help you to communicate the value of your work and educate others about UX practices across your business.

Related articles