
👋 Hey, I’m Nikki. Each week I write about UX research strategy, communicating impact, and using AI to do your best work.
What’s coming up
Come join my Introduction to Claude Agents 3-hour workshop on October 27th ($200 special just for you!) where you will learn everything you need to know to set up your first agent, including seeing exactly how I set mine up and examples of my own agents. Recording available for all sign-ups!
Diving into evidence strategy
Early in my career I wasted five months because I didn't clarify the goal or expected outcome of the research. And then I did it again, and again, and again, until I wanted to scream. My stakeholders would give a thumbs up and never use my insights and I’d have an absolute nightmare trying to figure out how I impacted anything at my performance reviews.
What had I actually been DOING? It was the worst feeling, and it continued for many years until I realized my research HAD to impact decisions.
Last week I wrote about the concept of decision research. If you missed it, you can catch up on it here, but today I want to dive into evidence strategy.
Evidence strategy is the first of its three practices and the one I always start with, since outcome and impact strategy have nothing to hold onto if the study isn't aimed at a real decision. It has three parts:
Framing the decision
Auditing what you already know
Agreeing how much evidence is enough
Frame the decision first
Whenever a request comes in, I write one sentence before I let myself think about a method. I use this formula:
[Owner] needs to decide [decision] by [deadline], choosing between [option A] and [option B]
For example, imagine I'm the researcher at TeamTicket, an app for booking tickets to events as a group. A product manager asks me for "some interviews about group bookings," and my sentence ends up looking like this one:
Priya, head of product, needs to decide whether to build split payments for group bookings by the Q2 roadmap review, choosing between building it this quarter and pushing it to next year.
If I can't fill in every bracket, I book a short call with the stakeholder before doing anything else, share my sentence and ask them to correct it. I've found that by sharing something already, it helps incredibly to surface the real deadline, and it often reveals that the decision is different from the one in the original request (ex: the team has already committed to building split payments and wants to know which payment providers to support).
The question I ask myself here is simple. Can I write the decision, owner and deadline in one sentence before choosing a method?
Audit what the organization already knows
I skipped this step for years, mostly since we usually had a deadline of last week and I was terrified if I didn’t research quickly enough, stakeholders would stop coming to me. However, auditing existing research and data is hugely important.
Since I got into this practice, we’ve skipped entire studies or completely changed the scope of a research question. Another benefit I found was when I thought I had to say no to a research study, I was able to do an audit and pull enough evidence for a stakeholder to move forward, even if just on a small piece of the project.
Now, before I scope anything, I look at the evidence that already exists. These are the sources I check:
past research reports and repository entries
product analytics and funnel data
support tickets and help centre searches
sales and customer success call notes
secondary research, such as industry reports and academic studies
what competitors are doing and how their users talk about it
I write my decision sentence at the top of a page, list the three to five questions I'd need answered to make it, then pull up at least three of these sources and write one line under each question saying what the source tells me and how old it is. If two sources agree and they're recent, I mark that question as answered. If only one source covers it, or nothing does, it stays open.
Then I make one of four calls:
I broaden the scope if the evidence shows the decision is bigger than anyone thought
Narrow it if half my questions are already answered
Change it if the evidence points at a different problem
Skip the study entirely if the decision can be made with what we have
Back at TeamTicket, a search through support tickets shows people asking how to pay friends back after booking, sales call notes mention a competitor that already offers split payments, and a study from last year found most groups have one organizer who pays upfront. So the team already knows people want a way to split costs. What nobody knows is whether organizers would use split payments or keep paying upfront and chasing friends later, and that single question becomes a much smaller, faster study.
When the audit tells me we don't need a study at all, I used to worry the stakeholder would think I was dodging work. Instead, I send them something like this:
Before setting up interviews, I looked at what we already have on [decision]. [Source] shows [finding] and [source] shows [finding], which gives us enough to decide between [option A] and [option B]. I recommend [option], and I'm happy to run a quick check on [open question] if you'd like more confidence.
I've found most stakeholders are relieved, since they get an answer weeks earlier than they expected.
This is where AI is a huge help—yay to properly using it for admin work!
I put past reports, ticket exports and call notes into a Claude Project and ask it to map what each source says against my decision sentence. Claude pulls everything together in a fraction of the time, while I decide what counts as evidence and what's missing. I always check the dates on whatever it surfaces, since Claude will treat a three-year-old study as current if you don't tell it the dates matter. Plus, reading the sources it picked teaches you a lot about what your organization has been paying attention to.
The question I ask myself here is whether I review what the organization already knows before every study, or only when I have time.
Agree how much evidence is enough
A small, reversible decision needs a lot less evidence than a big, expensive one. Most of the over-researching I've done came from treating every request like the big one, and my backlog never got any shorter.
So I score the decision on three things, each from 1 to 3:
Reversibility, from 1 (easy to undo next sprint) to 3 (very hard to undo)
Cost, from 1 (a few days of work) to 3 (a quarter or more of a team's time)
Reach, from 1 (a small slice of users) to 3 (everyone who uses the product)
Adding them up gives a stakes score between 3 and 9.
A score of 3 or 4 means existing evidence plus a light check is enough, 5 to 7 calls for one focused study, and 8 or 9 is where I bring out mixed methods and my full sample sizes. TeamTicket's split payments decision scores a 6, so one focused round of sessions with organizers is plenty.
Once I know the level, I agree the threshold with the decision owner before the study starts, using this script:
If we see [result], we'll [action]. If we see [result], we'll [action]. Before we act, we need [confidence level], which means [method and sample]
For TeamTicket, the filled-in version reads like this:
If most organizers say they'd keep paying upfront and chasing friends, we'll push split payments to next year. If most say they'd switch to split payments for their next booking, we'll build it this quarter. Before we act, we need a clear pattern across organizers, which means six moderated sessions with people who've organized at least two group bookings in the last six months
I always ask the decision owner to agree to this in writing, usually over Slack or email. It protects you from "we already knew that," since you agreed together what would change their mind. And when a study confirms the original plan, you are able to count that as a decision the research supported, rather than something people shrug off.
The question I ask myself here is whether my stakeholders agree, before the study, what result would change their minds.
Where to start
If you don't know which of the three to work on, think about your last few studies and pick the part you skipped most often. Working on one of them this month is plenty.
If you only try one thing from this piece, I recommend writing the decision sentence for the next study on your list before you open a single template. It takes five minutes and it will save you from the kind of five months I lost!
Grab this
I've turned everything from this piece into two downloads, so you can use whichever fits how you work—for paid subscribers only
The Evidence Strategy skill is a Claude skill that walks you through all three parts, one question at a time. You give it your request and upload your past reports, ticket exports and call notes, and it helps you write the decision sentence, scores every source you have, works out which questions still need research, and drafts the threshold agreement for your decision owner. At the end it hands you an evidence strategy summary you can paste straight into your research plan.
The Evidence Strategy Guide is a fillable PDF with the same steps laid out as a worksheet, for when you want to fill it in yourself or share it with your team. Inside you'll find:
the decision sentence formula with three worked examples
an evidence audit worksheet with a scoring rubric for every source
the gap count, so you know exactly which questions need new research
a stakes score rubric with an evidence level table for method and sample size
the threshold agreement script, plus a version for Slack
the red, yellow and green self-check
a standalone Claude prompt for the evidence audit, if you'd rather not install the skill
To add the skill, download the zip file, open Claude, go to Settings, then Capabilities, and upload it under Skills. Start a new chat and tell Claude you have a new research request you want to scope!
Please tweak the scores and levels in both to fit how your team works as it will end up being a lot more specific and helpful!
Exclusively for paid subscibrers: