Leading questions can unintentionally reveal the type of answer a recruiter is looking for, encouraging participants to provide responses that help them qualify rather than answer honestly. This can result in recruiting people who are not representative of the intended audience. Effective screening ensures that user research is conducted with the right participants, leading to more reliable insights and better-informed design decisions.
Recruitment screening: avoiding leading questions
How to screen participants to get the best possible results in user research.
We are careful to avoid leading questions in research; are we doing the same when screening participants?
The way we ask questions in research is different from the way we do in everyday conversation. It has to be because the goal is not the exchange of ideas. It is gathering data.
Likewise, screener questions help us gather information to establish the suitability of participants. We may filter out people with irrelevant experience, exclusions that may skew data, and people who are trying to make a quick buck from research without any interest in providing genuine answers.
The way we ask screener questions therefore requires the same rigour as research to make sure we don’t inadvertently reveal the ’correct‘ response to potential respondents.
User research with non-representative participants is a waste of time and money. Here are some questioning methods from research which can be applied to the screening process to make sure you recruit the ‘right’ participants.
1. Leave the answer out of the question
Instead of “Are you a driver?” ask, “What do you do for work?”
Obscuring the purpose of the question makes things less straightforward so you may need to ask further questions to match against your recruitment criteria, but it should yield more reliable data than giving participants a 50/50 chance of getting the answer ’right’.
In this case for example, you might need to follow up with “What does that job involve?” or “What are you responsible for?” and decide if the answer corresponds with your criteria.
2. Don’t prompt the participant
“What apps do you use? Like Facebook, Instagram or Path, for example?”
Even for well-intentioned participants, it’s easy to latch on to what is given and cut off a train of thought. Ask “What apps do you use?” and wait.
Don’t read out multiple choices either.
3. Use triangulation questions to verify their claims
Triangulation is a way of using different sources or methods to check the credibility of research findings. Findings are likely to be credible if the same results are obtained from disparate sources.
In this context, we ask follow up questions to see if answers converge.
Sometimes, you get a feeling that an answer was shaky or you slipped up and might have given away the answer—a questionable “Err…Facebook?” perhaps. Trust your instincts and follow up with “What do you use Facebook for?” or “What did you last do on Facebook and when was this?” If they can’t decide (might have been a very long time ago) or worse, can’t name a correct Facebook function, you have your answer.
These techniques can quickly improve your pool of participants, but you may still find the odd miss-recruit that slipped through. Remember that as a researcher, you reserve the right to send a participant home if they have misrepresented themselves during screening.
It takes an attention to crafting questions and a bit of intuition to extract truthful answers. If you want to recruit better participants, don’t just check things off on your screener, ask questions just like you would in research.
Research Screening - FAQs
A leading question suggests or hints at a preferred answer. For example, directly asking a participant if they fit a specific profile may encourage them to respond in a way that increases their chances of being selected. Neutral screening questions are designed to gather information without revealing recruitment criteria, helping researchers identify suitable participants more accurately.
Recruiters can improve response quality by asking open-ended questions, avoiding prompts, and allowing participants to explain their experiences in their own words. Follow-up questions can help researchers better understand the participant's background and confirm whether their responses align with recruitment requirements. This approach reduces the likelihood of inaccurate or misleading answers.
Triangulation involves asking multiple questions about the same topic from different perspectives to confirm the accuracy of a participant's responses. Rather than relying on a single answer, recruiters look for consistency across several questions. This helps verify claims, identify uncertainties, and improve confidence that participants genuinely meet the research criteria.
Recruiting unsuitable participants can significantly reduce the value of a research study. Findings may not accurately represent the target audience, leading to misleading conclusions and ineffective design recommendations. Since business decisions are often influenced by research outcomes, participant quality plays a crucial role in ensuring research validity and credibility.
A structured recruitment process improves data quality, reduces bias, and increases confidence in research findings. When the right participants are selected, researchers gain insights that more accurately reflect real user behaviors, challenges, and needs. This enables organizations to make better decisions, reduce project risk, and create products and services that genuinely solve customer problems.