What should AI do in research, and what should stay human?

Ask people to describe their kitchen and many will tell you it’s minimal. Ask them to photograph it and you’ll see a coffee maker, a toaster, a knife block, a dish rack, the mail and a phone charger.

We saw that gap again and again in a study this year, and finding it took two very different kinds of work. AI made it possible to run more than 500 interviews in five days and to go through more than 6,500 photos and videos. Researchers worked out what the gap meant, and what to do about it.

That is the division of labour research needs. AI should scale the evidence. People should own the conclusion.

Why is “will AI replace researchers?” the wrong question?

Most of the debate about AI in research is about replacement. Will AI moderators replace human ones? Will automated analysis replace the analyst? Will synthetic respondents replace participants? It’s the wrong frame, because research was never one job. It is two kinds of work bundled together.

The first kind gets better with more: more conversations, more languages, more hours in the day, more consistency from the first interview to the last. The second gets better with context: knowing which decision the research has to inform, what was tried before, what is at stake, and who will have to stand behind the answer.

AI is exceptional at the first kind. Much of the second is out of its reach, and not because it lacks nuance. A good AI interviewer probes, follows up and gets people to open up. The problem is that the context that decides what a finding means usually isn’t in the data at all.

What should AI do in research?

Give AI the work that gets better with more.

• Run the conversations nobody has the hours for. Hundreds of interviews in parallel, overnight, across time zones, each in the participant’s own language. A small team can’t moderate three hundred interviews in a week. An AI interviewer can, and every participant still gets a real conversation instead of a form.
• Ask the follow-up every time. The sixtieth interview gets the same attention as the first: no fatigue, no rushing the last session of the day.
• Remember everything. Read every transcript, every open answer, every photo and video, and find every time a topic came up, not just the times someone happens to remember.
• Count people, not mentions. One talkative participant who raises something eight times is one person, not a theme. Counting is tedious for people and easy for software, as long as it counts the right thing.
• Draft the first synthesis. Themes, sentiment, quotes and highlight reels in hours rather than weeks, so researchers start from a draft instead of a blank wall of sticky notes.

What should stay human?

Keep the work that depends on context outside the data, or that someone has to answer for.

• Choosing the question. An AI can answer a question perfectly and it can still be the wrong question. Only the team knows which decision the research is meant to inform.
• Choosing who to hear from. Who you recruit decides what you can learn. That is a judgement about the market, not a data problem.
• Weighing, not counting. AI counts; people weigh. Three enterprise customers describing a workaround for losing data can matter more than thirty people asking for dark mode.
• Knowing what isn’t in the transcripts. The product’s history, what was tried last year, what the team can actually ship, what leadership already believes. None of it is in the data, and all of it shapes what a finding means.
• Owning the call. Someone has to stand in the room and say “we’re dropping this feature”, and answer for it if it turns out to be wrong. An AI can’t be accountable. A researcher can.
• Caring for participants. Consent, sensitive topics, and knowing when a conversation should be held by a person rather than a machine.

What does this look like in a real study?

Earlier this year we ran a five-day home diary study for a large US retailer: more than 500 AI-moderated interviews in five days, with more than 6,500 photos and videos. Each day, participants showed us one room (living room, kitchen, primary bedroom, primary bathroom or dining room) with a photo and a short video, wrote about it, and answered follow-up questions from the AI interviewer.

Two choices in the study’s design, both made by the research team, made the evidence trustworthy. First, the AI interviewer never introduced style or design vocabulary. It didn’t say “minimalist” or “farmhouse” unless a participant did, so every style word in the data was the participant’s own. Second, the analysis looked past the words: alongside every written answer, a computer-vision pass went through the images, so what people showed could be set against what they said.

That is where the tension appeared. Participants who described their homes as minimal (“we like things clean”, “we try to keep counters clear”) photographed kitchens with coffee makers, toasters, knife blocks, dish racks, the mail, keys and phone chargers. It wasn’t a handful of households. The gap held across the sample. It was widest in the kitchen and almost absent in the bathroom, where there simply isn’t room for the two to diverge.

The obvious reading is a failed aspiration: people want minimalism and don’t manage it. Our researchers read it differently. It is a rule. Anything in daily use is exempt from the minimalist standard, and participants apply that rule without ever saying it out loud. That reading changes what a retailer should do about it.

Scale also kept us honest. One of our own going-in hypotheses about how people style their homes didn’t hold up across the sample, and the readout said so. That is the split in one study: AI made the evidence big enough to trust, and people decided what it meant.

Where does the handoff between AI and people break?

The split sounds tidy. In practice, most of the damage happens at the handoff, the moment AI output becomes something a person relies on. It’s also where “human in the loop” either means something or doesn’t. Five failures come up again and again.

• The summary nobody can check. AI writes a confident paragraph and nobody can see where it came from. The fix: every finding links back to the verbatim quotes behind it, so anyone can check it in seconds.
• Paraphrase drift. Summaries quietly put words in participants’ mouths: close to what they said, but not what they said. The fix: quotes stay quotes. If it is in quotation marks, a participant said it, word for word.
• The loud participant. One person who mentions something eight times becomes a “recurring theme”. The fix: count people, not mentions.
• Invented confidence. When the evidence is thin, a plausible answer is worse than none. The fix: AI should be allowed, and expected, to say there isn’t enough evidence.
• Rubber-stamping. A researcher who approves AI output without reading the quotes isn’t in the loop; they’re decoration. The fix: build the human step around real work (reading the evidence, weighing it, deciding), not a final click on approve.

How do you decide which research tasks to hand to AI?

Before handing a research task to AI, ask three questions:

1. Does it get better with more? More interviews, more languages, more hours, more consistency.
2. Can a person check the output against the source in seconds? If a finding can’t be traced to what participants actually said, it isn’t ready to be relied on.
3. Does it depend on context outside the data, or does someone have to answer for it?

If the first two answers are yes and the third is no, hand it over. If the third is yes, keep it, however good AI gets at the first two. Most real tasks pass through all three: AI drafts the themes, each theme links to its quotes, and a researcher decides which one goes in front of the leadership team.

The line will keep moving. Not long ago, transcription and translation were human work. The principle shouldn’t move with it.

Should AI replace the participants too?

Synthetic respondents, AI personas that answer the way a customer might, are turning up in more and more research tools. The same principle applies. They can help you rehearse a study: check that a discussion guide flows, find the questions that confuse, see what you should be ready to probe. But a synthetic respondent is only as good as the real people it was built from and checked against. It can extend research with real people; it can’t stand in for it. And it has no kitchen counter to photograph. What people actually do still has to come from people.

How does Convo split the work?

We built Convo around this split. The AI interviewer runs voice, video and chat interviews in more than 40 languages, as well as diary studies that collect photos and video over several days. The analysis turns responses into themes, sentiment, quotes and highlight reels as they come in. The Research Assistant answers questions across a study with counts of people and their exact words, so every finding can be traced back to the conversation it came from. Interviews your own team ran can be uploaded into the same study and analysed alongside the AI-moderated ones.

Your team keeps the work that needs context: what to ask, who to ask, what the findings mean and what happens next. When you want researchers on our side as well, our team can design, moderate and synthesise a study with you. For one team’s version of this in practice, read how Perspective pairs Convo conversations with its in-depth interviews.

Use AI to hear from more people than you ever could. Then do the part only you can: decide what it means.