Subscribe now

P.S. Paid subscribers get access to full archive, all content, a private Slack community, Substack lives, and a hub of templates,…

Read more

Subscribe now

P.S. Paid subscribers get access to full archive, all content, a private Slack community, Substack lives, and a hub of templates,…

Read more

Subscribe now

P.S. Paid subscribers get access to full archive, all content, a private Slack community, Substack lives, and a hub of templates, scripts, and mini-courses

A few months ago, I asked a room full of researchers a simple question:

“Who here is using AI in their research practice?”

Every single hand went up.

I wasn’t surprised. AI is everywhere right now, and researchers are nothing if not curious, so I followed up with, “What are you using it for?” and got the following:

• Cleaning up transcripts

• Writing first drafts of screeners

• Summarising my notes before synthesis

• Tidying up my reports

Useful stuff, genuinely, but also not what I’d call AI fluency.

Almost every researcher in that room thought they were using AI well. And in a surface-level sense, they were. But when I started asking follow-up questions—how do you verify the output? what’s your approach when it gets it wrong? what’s your policy on participant data? have you disclosed to stakeholders how AI was involved?—the answers got much quieter.

Nobody had a actual clear framework. But, hey, it’s not surprising. Nobody has ever actually defined what AI fluency looks like for us specifically.

This is now showing up in your job description

Let me tell you what prompted me to actually write this down.

Companies are creating frameworks for assessing AI fluency in every single hire, across every function. They’re not just asking “do you use AI?” They’re assessing mindset, strategy, building, and accountability. They’ve raising the minimum bar so that candidates who can’t demonstrate AI embedded in their core work don’t make it through.

PMs are being assessed on it. Engineers. Designers. Customer success. Marketing. Finance.

The frameworks they’re building for each of these roles are specific. The PM version talks about writing SQL, building dashboards, generating working prototypes. The engineering version talks about shipping AI-powered features. The marketing version talks about building automated content pipelines.

Nobody has built the researcher version. So researchers are either being assessed against generic criteria that doesn’t account for what our work actually involves, or worse, they’re walking into AI fluency conversations with nothing concrete to say.

If you’ve been in a job interview recently and someone has asked you “how do you use AI in your research practice?”, you’ve felt this. The question lands and you have a few seconds to figure out whether “I use it to clean up my transcripts” is going to cut it.

It isn’t. Not anymore.

AI fluency is no longer a nice-to-have that ambitious researchers can choose to develop. It’s becoming a baseline expectation and companies are increasingly building tests to measure it.

So let’s build the framework for what it actually looks like for us.

Why research is different (and why the stakes are higher)

Before I get into the levels, I want to make one argument clearly, because I think it’s the most important thing in this entire piece. AI fluency for researchers is not the same as AI fluency for any other role and the reason is accountability.

If a researcher uses AI badly, they risk presenting fabricated or distorted insights to a product team who will then use those insights to make real decisions. That’s a pretty big mistake and it can cascade in ways that are genuinely hard to trace back to the source.

I’ve seen it happen. A researcher uses AI to synthesise interview transcripts, takes the themes at face value, builds a report around them. The themes look right, they’re plausibly written, but they don’t fully reflect what participants actually said. They reflect what the AI predicted the themes would be, shaped by its training data, which is not the same as your eight participants and their specific context.

This is the GIGO principle playing out in research: garbage in, garbage out. The terrifying version for researchers isn’t that you feed garbage in. It’s that you feed good data in, and the process quietly introduces distortion you can’t see.

That’s why accountability has to be the centerpiece of any AI fluency framework for us. It’s not the most exciting component but it’s the one that separates researchers who are using AI well from researchers who are just using AI fast.

The four levels of UXR AI fluency

I’m going to describe each level in detail, across the full scope of research work. Read through all four before you try to self-assess, because it’s easy to over-identify with a higher level until you see the specifics written out.

Read more

Subscribe now

P.S. Paid subscribers get access to full archive, all content, a private Slack community, Substack lives, and a hub of templates, scripts, and mini-courses

I hate making deliverables. So I made Claude do it and accidentally built something better than any journey map I’ve ever made.

Research deliverables f*cking suck. I’m a words person. I do not have a design bone in my body, not even the tip of a pinky bone. I will write you a beautiful report. I will not make you a beautiful journey map and yet somehow half my job is making beautiful journey maps.

So when clients started asking for dynamic deliverables lately, my honest reaction was that I don’t even like static deliverables, how am I going to make dynamic ones?!?!?!? I’m not even dynamic.

Then I sat down with Claude Cowork one evening, uploaded two screenshots of a bogstandard journey map I made years ago at a now-defunct travel company, wrote an embarrassingly bad prompt (”how can we make this more dynamic”), and watched something kind of amazing happen. This video is me playing around, live, with very little plan. Things break. I yell at Claude (who, for the record, is a dude). I burn through some Lovable credits I was saving for actual work. And by the end I’ve got something I genuinely wish I’d had ten years ago.

What I cover:

1. From two flat screenshots to an interactive, segmented, three-persona journey map in one prompt. I fed Claude the world’s most standard journey map with goals, tasks, pain points, quotes, the usual, and a prompt that I am not proud of. Claude came back with a toggleable map across three user types (occasional, first-time, power), moments of truth, a backstage layer, channel dimensions, impact-vs-effort opportunities, and pain points ranked high, medium, and low. Not perfect but already more useful than anything I’ve ever built by hand.

2. The real journey is a mess and Claude can finally show that. Every journey map I’ve ever made has been a lie. Real people don’t walk in straight lines from Awareness to Conversion. They loop, they abandon, they come back three days later on a different device. We simplify, since showing the mess is genuinely hard and takes forever. I told Claude I wanted the realistic version. It gave me a more real scenario: 12 days, 7 sessions, 5 backtrack loops. You can scrub through it on a timeline. That sentence alone is more interesting to a stakeholder than my entire old journey map.

3. Pain points linked straight to Jira tickets. Opportunities linked to prototype prompts. The whole point of a journey map is to make someone do something. So I had Claude wire each pain point and opportunity to backlog tickets (manual now, Jira-connected later) and then try to link out to Lovable and Figma Make with a prompt pre-built to prototype the fix. Some of it 404’d. All of it pointed at something I couldn’t have dreamed of making myself a year ago.

4. This is a genuinely bad prompt and I want you to see it anyway. I’m a huge fan of prompt engineering. The prompts in this video are not that. I left them in on purpose. When you’re experimenting, perfect prompts aren’t the point, throwing stuff at the wall is. You do not need a flawless prompt to get something useful. You need to start.

5. The thing we used to beg others to build, we can now build ourselves. When I was making journey maps early in my career, I printed personas and taped them in bathrooms so people would actually look at them. That was the bar. Now I can hand a stakeholder something interactive they can click through, segment, and give feedback on without a wait. It’s not “AI replaces researchers,” it’s “researchers finally have the means, the power, and the resources to make the things we’ve always wanted to make.”

This is me experimenting live. A proper walkthrough with real prompt engineering is coming. I wanted to get this out now anyway, since it’s the most fun I’ve had with research deliverables in years, and I want you to go try it too.

Watch the full thing above then go make something ugly and cool.

Want Claude working like this without the “oh god what’s my prompt” moment?

I built the UXR Claude Skills Bundle for exactly this reason, 52 research skills installed directly into Claude, so the right framework shows up the moment you need it. Journey maps, TEDW interview guides, the insight formula, the Pyramid Principle for reports, the whole toolkit with no re-explaining, no starting from a blank chat every time.

One-time $49, lifetime access (updates included), and installs in 5 minutes.

Get the Bundle →

I’ve spent the past few months watching synthetic research show up everywhere. Conference talks. LinkedIn posts. Case studies. Every time I saw it, I had the same reaction. This is going to be a disaster.

Not because synthetic research is inherently bad, but because I know what happens when a new tool promises efficiency. Someone uses it to skip the real work. Someone points to synthetic data and says “we don’t need to do user research anymore!”

I’ve cleaned up those messes before. I’ve been in the room when analytics replaced interviews, when surveys replaced conversations, when speed became more important than understanding. Every single time, the promise was “this will make research faster and cheaper.”

Every single time, we lost something that mattered.

So when synthetic research started gaining traction, I braced for the worst.

Then I looked around and realized this isn’t a future problem. This is already happening. 73% of market researchers have already tried synthetic responses. A third are using it regularly. Companies like Booking.com and Dollar Shave Club aren’t experimenting with it, they’re building it into their research practice.

The market moved without me.

And I had to ask myself: am I avoiding this because it’s actually bad, or am I avoiding it because I’m scared of what it means?

From Panic to Openness

Qualtrics asked if I’d be willing to test their synthetic audiences feature and write about the experience. My first reaction was panic. Was I going to try something like this? And how would it go?

Through the anxiety came another thought. How will I ever know if I don’t experience this?

I’ve been that person before. I was deeply suspicious of surveys when I was a qualitative purist. I thought unmoderated testing would never work. I resisted Jobs to be Done because I worried it would replace other frameworks I valued. Every time, my fear was the same, that this new thing will make my skills less valuable. Every time, I pushed these methods away and, ultimately, my career suffered.

I eventually tried the above methodologies and learned that they don’t replace what I do. They just give me a different way to approach certain problems.

Maybe synthetic research was the same.

So I said yes.

What I Needed to Understand First

Before I ran anything, I had to understand what I was actually testing and what synthetic really was because, trust me, there are a lot of different definitions out there.

Not all synthetic is AI randomly generating responses based on what it thinks humans might say. That’s what generic LLMs do when you ask them to “pretend to be a user.” Those fall apart fast. The responses are too agreeable, they lack demographic variation, they miss all the nuance that shows up in real research. It’s like having someone pat you on the back and tell you, “yes, that is the best product idea ever!”

I found that Qualtrics works differently. It’s trained on millions of actual human survey responses collected over 25+ years of research. When you ask it a question, it’s not guessing or fabricating. It’s looking at patterns across massive amounts of real human data and predicting what humans would likely say to that question.

This kind of synthetic data is model-generated data that’s built to mimic how a real audience might respond. Think of it as a way to explore likely patterns without running a full study with live participants. The key idea here is that mimicry isn’t the same thing as replication. In research, replication is tied to real people, real sessions, and repeating a study to see if the outcome holds. Synthetic data isn’t trying to do that. It isn’t claiming to recreate lived context or match what real participants would say word-for-word. It’s built to give you a fast, pattern-based preview of how a defined audience is likely to react, so you can learn, compare options, and move with more confidence before you invest in full fieldwork.

That difference matters. One is AI making stuff up. The other is AI drawing on what humans actually said when asked similar questions.

Source

I still went in skeptical, but at least I understood I wasn’t testing random text generation. I was testing whether a model trained on real research data could show me something useful before I talked to a single person.

The Research Question

I picked a research question that felt like appropriate territory for testing synthetic: how do people make food decisions when they’re tired, stressed, and low on money?

This question type matched what synthetic is actually designed to handle. Synthetic performs well with perceptions, preferences, and intent-based questions.

As of right now, it doesn’t perform as well with past behaviors, detailed recall, and awareness questions. So I avoided anything like:

• “Tell me about the last time you ordered takeout”

• “Walk me through your typical dinner routine”

• “How often do you meal prep?”

Those questions need memory. They need a biography. They’re asking people to reconstruct events and explain what happened. That’s not what synthetic is for. And, for me, that actually allowed a sigh of relief. Those questions above are the bread and butter of my deep, qualitative research and they aren’t what I would be asking a synthetic panel. I could save those for human-based research.

Instead, I focused on the mental state that shapes decisions:

• What mindsets do people bring to food choices when they’re under pressure?

• What kinds of friction stop people from deciding at all?

• What does “good enough” mean when you’re exhausted and broke?

• What would actually help in those moments?

These questions are about recognizing patterns, not recalling specific events. They focus on mental shortcuts and trade-offs.

WIth this, I built a 13-question survey. Every question had clear scenario framing, a specific cognitive task, and distinct answer choices that felt similar to those I’d create in a standard survey.

I kept the screening broad, just “are you involved in household food decisions?” You can screen with synthetic, you just can’t go hyper-niche, just like in real research. Screening for moms, millennials, pet owners, households of 3+, specific income brackets? Fine. Screening for “left-handed surgeons from Omaha?” That’s not happening. It’s the same as human research, the more niche you go, the harder it gets.

I launched the survey on a Wednesday morning.

By noon, I had 500 responses. How’s that for speed?

What The Data Actually Showed Me

I was met with a dashboard of data which, as a qualitative researcher, can sometimes feel overwhelming, but I took a deep breath and scrolled through the results.

Each question was broken down in several ways:

I really wanted to read the results as I would a traditional survey that I sent to real humans to fill out. The force was real with this. But, I took a step back first and really thought about how I would analyze these results. I took the first question: Now imagine it’s the end of a long day. You’re hungry, tired, and don’t feel like cooking, but you also have a limited budget this week. Which option becomes more important in your decision?

When I look at a synthetic-based chart like this, the goal isn’t to pretend I’m staring at “real behavior.” I was looking at a pattern the model thinks is likely when people are tired, hungry, short on time, and trying not to overspend. It’s a stress-test scenario. The value is in seeing how priorities reorder when the pressure dials up.

This is how I read the chart:

“Choosing something filling, even if not healthy” gets the strongest pull. In a depleted moment, people want to feel fed. Satiety beats ideals, beats convenience, beats health. It becomes the anchor. If I were running real sessions, this is where I’d expect the richer stories on what “filling” means, where they go, what they reach for, which shortcuts they trust.

“Sticking to something low-cost” sits right behind fullness. Tired or not, budget pressure stays in the picture. This gave me a directional hint that cost is a constraint people keep even when their energy is shot. In live research, I’d probe how they manage that tension. What counts as “low cost” in their mind? What crosses the line?

“Using what I already have” lands in the middle. It tracks with what real people say, that the intention is there, and it matters, but it’s easier to abandon when they’re drained. Synthetic data is giving me the shape of that trade-off, not the emotional story behind it.

“Getting something fast, even if it costs more” was lower that I expected. With a tight budget in the prompt, synthetic response showed a prioritization of price and fullness over speed. That doesn’t mean speed stops mattering for real people. It just means, in this hypothetical, it’s not the dominant driver.

“Skipping a meal to save money” sits at the bottom, but it’s not zero. That’s a cue for segmentation, not a headline. In real research, this is where I’d ask one of those beautiful, open-ended questions from above, “Walk me through the moment you decide not to eat. What’s happening right before that choice?”

“Not much changes for me” shows up in every study, real or synthetic. It’s the group with strong routines or fixed habits. They stick to their script regardless of energy or budget shifts.

Here are some other ideas I discovered while looking through the results.

The Cost-Versus-Speed Tension

One question asked: “When you’re low on time or energy, which of these factors usually has the biggest impact on your food choice?”

The results came back nearly split: “Cost” at 143 responses (29%) and “Speed” at 135 responses (27%).

Synthetic responses showed me that when people are tired and resource-constrained, cost and speed genuinely compete as decision drivers in human behavior. Neither factor dominates cleanly in the model’s prediction because neither factor dominates cleanly in real human decision-making. Results from the synthetic data said that humans are split on this because it’s a genuine dilemma they navigate.

I might not have seen that tension as clearly if I’d just written the question and jumped straight to interviews. I might have assumed one factor would always win. I might have designed my interview guide assuming cost would be the primary driver and missed the whole speed dimension.

This became a key area to explore in follow-up human research. When does cost win? When does speed win? What makes someone prioritize one over the other? The synthetic data had flagged where genuine decision-making complexity lives before I’d talked to anyone.

The Emotional Overwhelm Discovery

Another question asked: “What would make you completely give up on choosing food and go to bed without eating?”

“Feeling overwhelmed or anxious” came back at 222 responses (44%), which was significantly higher than practical barriers like “not wanting to clean up after” (112 responses, 22%) or “realizing everything costs too much” (101 responses, 20%).

I’d designed that question thinking about practical constraints like money, effort, time. But the model highlighted emotional overwhelm as a much bigger driver of food decision abandonment than practical barriers.

I’d been thinking about this as a resource constraint problem. How do people choose food when money and time are limited? But the model was indicating that this is primarily an emotional regulation problem. People don’t give up because the barriers are too high, but they give up because making one more decision feels unbearable.

When I redesigned my interview guide for follow-up human research, I centered the emotional state instead of practical constraints. I made that the primary lens for exploration rather than treating it as a secondary factor.

The “Good Enough” Complexity

I asked: “Which of these feels most like a ‘good enough’ food decision when you’re tired, busy, and low on money?”

The distribution came back with no clear winner: “something I had already planned” (153 responses, 31%), “anything hot, filling, and settles my hunger” (138 responses, 28%), “something that is comforting or emotionally satisfying” (107 responses, 21%), “something that meets a low-cost threshold” (99 responses, 20%).

“Good enough” doesn’t mean one thing to humans. The model demonstrated that all of these definitions of “good enough” show up with roughly equal frequency, likely because they’re drawing on genuinely different frameworks depending on which resource (physical energy, emotional capacity, money, cognitive bandwidth) is most depleted in that specific moment.

When you’re physically depleted, “good enough” means something that fills you up and settles your hunger. When you’re emotionally exhausted, “good enough” means something that brings comfort. When you’re financially stressed, “good enough” means hitting a cost threshold. When you’re cognitively overloaded, “good enough” means not having to deviate from the plan you already made.

The flat distribution was the model telling me that I’d asked a question that collapses multiple real human patterns into one. When I designed follow-up human research, I split this into separate questions focused on physical needs, emotional needs, financial constraints, and cognitive load. That separation allowed for much clearer exploration of how different resource depletion states drive different definitions of “good enough.”

The synthetic data hadn’t just caught a poorly-worded question. It showed me I was conflating multiple human decision frameworks that needed to be understood separately.

While reviewing the results, something shifted. Not in a “this changes everything” kind of way, but in a “huh…this is actually interesting” kind of way. I started to see where this could fit into my workflow.

What This Actually Gave Me

After two days with the data, I had to ask: was this worth it?

I didn’t get finished, complete insights. I didn’t learn everything about how people make food decisions under stress. I wouldn’t feel comfortable walking into a meeting and presenting findings that would drive our next biggest product decision.

But I also didn’t waste anyone’s time. I didn’t recruit participants for a survey with broken logic. I didn’t schedule interviews based on assumptions that turned out to be wrong. I didn’t build a research plan around questions that collapsed multiple concepts.

Instead, I learned where the complexity was before talking to anyone. The cost-versus-speed split showed me there’s genuine tension in how people prioritize. The overwhelm spike showed me I’d underweighted emotional factors. The flat “good enough” distribution showed me I’d conflated multiple decision frameworks.

I redesigned my research approach with better questions. Instead of asking broadly about “good enough” decisions, I split it into questions about physical needs, emotional needs, and financial constraints. Instead of treating food decision abandonment as a practical problem, I centered on emotional regulation. Instead of assuming one factor would dominate the cost-versus-speed trade-off, I designed my interview guide to explore the tension.

I saved weeks of iteration time. If I’d skipped synthetic and gone straight to human research, I would have recruited eight people for pilot interviews, spent three sessions realizing my questions were off, spent three more sessions trying to figure out what I should have asked, and then recruited twelve more people for a second round with better questions. That’s 20 participants and three to four weeks of calendar time minimum. Synthetic compressed that learning cycle and helped to narrow the scope of a very broad study.

By the time I sat down with real participants, I wasn’t wondering if my questions made sense anymore. I was listening for the nuance synthetic can’t capture and the reasons why people make specific choices, with the emotional context around decisions, and the moments where behavior contradicts stated preferences.

I could focus on listening because I wasn’t worried about structure.

So, yes, to me this felt worth it.

A Note on Data Richness vs. Data Poorness

Data-rich organizations have massive amounts of customer data, years of research, thousands of completed studies. They’re not using synthetic because they have no data. They’re using it to drill deeper into the data they already have.

Data-poor organizations are the opposite. They are startups, small teams, companies entering new markets, researchers exploring completely new problem spaces. They don’t have years of existing research to draw from. They can’t afford to run massive studies. They need directional clarity fast, and they need it cheap.

That’s where synthetic becomes really interesting.

If you’re exploring a new market and you have zero data about how people in that space think, synthetic can give you initial pattern recognition based on what the model has seen across millions of responses in adjacent spaces. It’s not definitive and it’s not finished insight, but it’s enough to help you figure out which questions are worth asking humans and where the complexity might live.

Or take the “water cooler ask” scenario. Your VP drops by and asks, “Hey, do you think our customers would care about this feature?” You don’t have time to spin up a full study. You need directional sense in the next day or two. Synthetic can help you triage whether this is worth a real study, or can we deprioritize this?

Or feature prioritization when you have ten concepts and you need to narrow to three before you invest in validation research. Synthetic can help you screen fast without burning participant goodwill or budget on ideas that won’t make the cut.

Nobody’s replacing human research. They’re using synthetic to figure out where to focus their human research or to move faster with human research when they don’t have the luxury of time.

It’s triage. It’s prioritization. It’s the pre-work that helps you not waste resources on the wrong questions.

The Dance

For me, the most interesting use isn’t synthetic OR human. It’s the dance between them.

I could start with synthetic to map the territory quickly, like to see where tensions live, which concepts need exploration, what questions I’m not asking yet. Then bring in humans to understand the why behind those patterns. But it doesn’t stop there.

Sometimes human research surfaces something unexpected. Instead of just noting it and moving on, I could go back to synthetic to test whether that’s a broad pattern or an edge case specific to the five people I talked to. Then back to humans to dig into the nuance if it looks like a real pattern.

It’s not linear. It’s iterative.

With my food decision study, synthetic showed me the cost-versus-speed tension. I’d bring that to human interviews and discover, let’s say, that the tension resolves differently on weekdays versus weekends. I could go back to synthetic and test that hypothesis across a larger set of responses to see if it holds. Then return to humans with a more refined question about what drives the weekday-weekend difference.

Or the emotional overwhelm finding that synthetic flagged. I’d explore it with humans and learn that it’s specifically decision fatigue, not general stress. Back to synthetic to see if that distinction shows up in how people describe the overwhelm. Then back to humans to understand what actually helps with decision fatigue versus other types of stress.

The dance is what makes it work. Synthetic gives you speed and breadth to test hunches. Humans give you depth and context to understand what the patterns mean. You need both, and you need to know when to use which.

I didn’t see that coming. I thought I’d use synthetic once, pressure-test my questions, and move to humans. What I actually discovered is that the back-and-forth between them is where the real value sits.

What I’d Use This For

After this experience, I know exactly where synthetic fits in my practice.

Pre-research testing: Before I invest time and participant goodwill on a real survey, I can use synthetic to rehearse. Test if my questions make sense. See if different demographic groups might interpret things differently. Catch confusing wording or weak spots before anyone real has to deal with them. It’s like a dress rehearsal before opening night.

Follow-up exploration: After I collect real data and find something unexpected, I can use synthetic to explore “what if” scenarios without having to recruit a whole new round of participants. Test additional questions I forgot to include. Stretch my research budget by running quick follow-ups to probe areas where the human data pointed to something interesting but didn’t give me enough depth.

Concept screening and feature prioritization: When I have ten concepts and need to narrow to three before doing validation research, synthetic can help me screen fast. I’m not using it to make the final decision, but using it to figure out which three are worth the investment of real human research. Same with feature prioritization, I can quickly assess which features seem most valuable before committing resources to full validation.

The water cooler ask: When someone drops by with a question and I need directional sense-making fast, synthetic can help me triage. Is this worth a real study? Should we deprioritize this? Can I give them enough direction to make a next step without spinning up a whole research project?

Data-poor exploration: When I’m exploring a completely new problem space or market where I have zero existing research, synthetic can give me initial pattern recognition to help me figure out which questions are worth asking humans and where complexity might live.

What I absolutely wouldn’t use it for:

• Answering research questions on its own without human validation

• Understanding personal motivation, emotion, or lived context

• Making high-stakes product decisions without any human input

• Past behaviors, memory, or detailed recall questions

• Anything where getting it wrong would hurt people

• Replacing interviews, usability testing, or actual human research

The Fear I Started With

The fear of “is this going to replace me?” was never really about synthetic research. It was about value.

We worry that someone will decide our work isn’t necessary. That someone will find a faster, cheaper way to get “insights” and we’ll be left explaining why depth matters, why context matters, why you can’t replace human understanding with pattern recognition.

Using synthetic didn’t make that fear go away.

But it did clarify something for me. The work that matters, the listening, the probing, the synthesis of human stories into understanding, can’t be automated. A model trained on millions of survey responses can tell me where patterns exist. It can’t tell me why those patterns matter or what they mean for any individual person.

But the prep work like the question testing, the concept narrowing, the structural validation, can be faster. And when you speed up the parts that don’t require human voices, you create more space for the parts that do.

What I’d Tell Another Skeptical Researcher

If you’re reading this because someone asked you to try synthetic and you’re suspicious, I get it. I was there three months ago.

Keep your skepticism. Don’t let anyone tell you synthetic can replace human research. Don’t let anyone pressure you into treating AI-generated responses like they came from real people. But you can try it once with clear boundaries.

Pick a research question that’s attitudinal and forward-looking. Write 8-10 questions with clear scenarios and distinct options. Run synthetic. Look at the data for pattern flags and complexity signals, not finished insights.

Then ask yourself:

• Did this help me design better questions?

• Did it show me where complexity lives?

• Did it save me from wasting participant time on bad design?

• Did it help me figure out where to focus when I bring in humans?

If the answer is yes, you’ve found a tool that can make your human research stronger.

If the answer is no, you’ve confirmed your skepticism with evidence instead of assumptions.

Either way, you’ll know.

I’m still a qualitative researcher. I still need human voices. I still believe depth and context and lived experience are irreplaceable.

I just have one more tool for figuring out the right questions to ask before I involve the people who matter most.

• The fear and skepticism many researchers feel toward synthetic users, especially around job security and research quality.

• How a synthetic panel works in Qualtrics, step by step, including setup, question design, and early signals.

• The tension between stated advice and lived behavior in synthetic data, and how that tension becomes a clue for deeper human follow-up.

• How synthetic results can help shape hypotheses, narrow scope, and surface mental models worth examining with human participants.

• The role of experimentation, reality-checking, and ethical use when bringing synthetic insights into a human-centered research practice.

Key Takeaways:

1. Synthetic users aren’t a replacement, they’re a low-stakes way to surface potential thinking paths worth exploring. Fear of being replaced is real for many UXRs, but synthetic panels don’t replicate lived experience. They can spark ideas, highlight tension in responses, and point toward questions worth asking humans, but they don’t carry nuance, emotion, memory, or contradiction. They’re an extra tool, not a takeover. 

2. Synthetic panels help you see mental models earlier, especially the ones users rarely say out loud. The synthetic example in the video about routines revealed goal-driven thinking mixed with self-doubt, which is a pattern worth validating with real people. This gives researchers a head start when writing interview guides or structuring probes. It doesn’t give you truth, but it does give you direction.

3. Synthetic data is great for pressure-testing your own questions before running a study. I described how running a synthetic version of a study I’d previously done with humans showed where the survey and interview questions held up and where they needed tightening. This kind of dry-run can save time, catch weak spots, and help teams narrow scope before talking to real people.

4. Researchers still need to reality-check everything with humans. Synthetic outputs are predictions shaped by large datasets, not lived stories. Human sessions reveal timing, emotion, contradictions, and subtle meaning shifts that synthetic models can’t replicate. You can use synthetic to form hypotheses, but every hypothesis needs human evidence behind it. 

5. Ethical and intentional use must lead the way. Researchers should be the ones teaching teams how to use synthetic panels responsibly. That means knowing where they fit, where they fail, and how to protect user trust. Synthetic tools aren’t going anywhere, so UXRs benefit from learning how to guide their use with clarity and care.

The companion guide to synthetic users:

Want to learn even more about synthetic users? Check out the companion guide to this video which goes in-depth about responsible, intentional, and ethical synthetic user usage.

Try Qualtrics:

Want to try this out on Qualtrics? You can request a demo below:

Try Qualtrics

Interested in sponsoring the podcast?

Interested in sponsoring or advertising on this podcast? I’m always looking to partner with brands and businesses that align with my audience. Reach out to me at nikki@userresearchacademy.com to learn more about sponsorship opportunities!