How to Run a Concept Test
Without Leading Participants or "Validating" Ideas
Hi, I’m Nikki. I run Drop In Research, where I help teams stop launching “meh” and start shipping what customers really need. I write about the conversations that change a roadmap, the questions that shake loose real insight, and the moves that get leadership leaning in. Bring me to your team.
Paid subscribers get the power tools: the UXR Tools Bundle with a full year of four top platforms free, plus all my Substack content, and a bangin’ Slack community where you can ask questions 24/7. Subscribe if you want your work to create change people can feel.
Concept testing can sometimes feel like a mystery. There were quite a few times in my career when teams put an idea in front of me and asked me to test it with users (well, to validate or ask for preference, which we don’t do, of course 😁). And for some time, I felt incredibly stuck when my teams made these requests.
The ideas weren’t solid enough to conduct a usability test.
They weren’t basic enough for me to conduct generative research.
These concepts were in the in-between (or the upside-down, if you will 😂).
Whenever a team came to me with these concepts, I cringed. I had no idea how to get them the information they needed without leading the participants and asking biased questions. I knew I shouldn’t be asking things like:
“Do you like this idea?”
“Would you use something like this?”
“How would you make this better?”
“Is this going in the right direction?”
But I wasn’t sure how else to engage with participants.
So, I tried to conduct usability tests on concepts. That ended as a major failure. The ideas were too early to test. Participants got lost and confused because there wasn’t a flow. And, to be honest, I had no idea what I was trying to test. My goals for those research projects were vague, and the findings were unhelpful.
We understood that people had a hard time navigating a loosely defined concept, but we still had no idea whether or not we were heading in the right direction. Each of those reports ended up disappointing not only my teams but also me.
It was by chance that I heard about the concept of concept testing (very meta, I know). At first, I wasn’t sold. How were we meant to evaluate concepts in an unbiased way? And how were we meant to investigate their reactions without relying too heavily on future-based data?
But, after some more disappointing results and failed usability tests on ideas, I finally decided to give concept testing a whirl. Admittedly, I wasn’t very skilled at conducting those tests, but with some practice and guidance, I finally understood the importance of concept testing. And, from there, I never looked back.
What is Concept Testing, Anyway?
Concept testing is one of those elusive methods that, I believe, we don’t discuss nearly enough, as it can be an extremely powerful tool to use early in the discovery process. Because it can be such an “in-between” method, we often skip it, going straight from generative research to usability testing.
However, concept testing definitely has its place in our process. The way I define it is:
Concept testing is a way to engage with participants to more deeply understand a specific problem and their current process through a stimuli (concept). Through concept testing, we gather feedback that allows us to gauge how aligned we are (or not) with participants’ mental models regarding an idea.
Within the scope of this definition, we are looking for immediate reactions and perceptions from our participants. We are looking to see how participants respond to the idea and where there are gaps or confusion about what we’ve put in front of them.
This is the crux of the definition and often where I can see concept testing go wrong (and where I’ve done it incorrectly before).
Where Concept Testing Goes Wrong
As I mentioned, it took me a good amount of practice to hone my concept testing skills. Because it is a less-discussed methodology, I struggled with finding the proper resources on how to conduct a concept test and what exactly I was looking for as an outcome.
Unfortunately, I see concept testing used a lot for things like:
Product/idea validation
Preference testing
A/B testing
Asking about future-based behavior
When I first started concept testing, I made these mistakes. I wanted the concept test to tell me whether or not participants liked the concept, if they would use it or not, and if I tested multiple concepts, which one they preferred.
The problem with all of the above is that concept testing is still a qualitative method. And with qualitative methods, we can’t answer these types of questions. Qualitative user research isn’t set up for success to answer “whether or not” or “if” or “preference” questions.
Qualitative research involves uncovering reactions, perceptions, feelings, and mental models. Concept testing should be no different.
When I first started conducting concept tests, I asked many of those questions, and the results were more disappointing than the usability tests I had attempted to run on the concepts.
What did it mean if people liked or disliked a concept? What did preference mean when it came to the concepts? How would we know people would actually use the product in the future? Not only that, but usually, during these tests, there can be social desirability bias present, where participants will tell you what you want to hear.
So, when I delivered my results, my teams weren’t always sure what actions to take. We knew that people liked the concept and which they preferred, but there was so little depth to the answers and so little action within the data that the teams ended up feeling just as stuck as before.
From there, I changed how I thought and approached concept tests, ensuring that I got my teams the data they needed without asking participants questions that could skew our decision-making.
If concept testing has ever made you freeze, this next part is the fix.
Paid subscribers get the full concept testing playbook:
A quick “Should we even run a concept test?” decision checklist
A goal-setting script to pull the real decision your team is trying to make
A concept setup guide (what to show, how low-fi to go, what to avoid)
A question bank built for concept testing (TEDW prompts + follow-up ladders)
Run-of-show templates for 60 and 90 minutes (1 concept, 2 concepts, 3 concepts)
Sample screener prompts to recruit people with the real problem, not hypothetical interest
A synthesis workflow tailored to concept tests (deductive tags + clustering pattern map)
An ideation workshop plan that turns findings into prototype-ready directions
Exclusively for paid subscribers
