Our UX research had a bottleneck everyone politely ignored. We'd collect great feedback — open-ended surveys, interview transcripts, the occasional emotional voicemail — and then our three-person team needed three to four weeks to grind 200 responses into themes. By the time the insights were ready, the product decision had usually already been made by someone in a hurry. Our research wasn't informing the roadmap. It was eulogizing it.
The reason it took weeks is that real thematic analysis is genuinely hard, not just tedious. You have to hold all the responses in your head at once, resist naming themes before you've seen everything, find actual patterns instead of confirming your hunches, and pick quotes that represent the middle rather than the loudest extremes. It's cognitively expensive work, and three brains can only do so much of it before the deadline laps them.
What I asked Claude to do
I took 200 responses to 'What is the main thing stopping you from using the product more?' and ran them through Claude Sonnet 4.6 with the Turn Raw Survey Responses into Themes You Can Trust prompt, goal set to 'identify retention barriers for the roadmap.' Sonnet 4.6 is the right model here — careful, good at long inputs, and it actually follows the 'read everything before grouping' instruction instead of pattern-matching on the first ten responses like an over-caffeinated intern. It returned six themes, each with verbatim quotes and an honest frequency band, plus a Key Tensions section and five ranked insights.
The Key Tensions section was the part that made my whole team physically lean toward the screen. It surfaced a contradiction we'd been circling and arguing about for a month without ever naming it cleanly: a Common cluster of users wanted more features, while another Common cluster said the product was already too complex to use. That's not noise to be averaged away — that's a real, structural product tension between power users and newcomers, and seeing it stated plainly, with verbatim quotes from both camps sitting right next to each other, ended a month-long argument faster than any of our actual meetings had managed. You cannot 'split the difference' on that tension; you have to choose, and naming it was the first step to choosing.
It's worth being clear about why I trust Claude Sonnet 4.6 here over a flashier reasoning model: thematic analysis doesn't need brilliance, it needs discipline and stamina. It needs something that will read all 200 responses without getting bored at response 60 and starting to skim, that will pull a real quote instead of a plausible-sounding invented one, and that will hold the boring 'read everything first' rule for the entire job. Sonnet 4.6 is calm, careful, and good at long inputs, which is exactly the temperament this task rewards. A model that's eager to leap to insight is a liability when the whole point is to not leap.
How I made sure it wasn't making things up
I do not trust a model's themes on faith, so we verified. Two teammates read the raw 200 responses cold and independently named their own themes. They matched the model on five of six. The sixth — a Minority theme about pricing confusion — both humans had missed, and on review it was legitimately there, just quiet. That's the dream outcome: the model caught a real minor pattern, and the humans confirmed the major ones. The verbatim-quote rule is what made verification fast — every theme came with the receipts, so checking it meant reading three real sentences, not trusting a vibe.
- Two humans independently matched 5 of 6 themes — the agreement is what earns trust
- The model surfaced a real Minority theme (pricing confusion) both humans skimmed past
- Verbatim quotes made verification a 3-sentence check instead of a re-read
- The Confidence Note flagged that our sample skewed toward power users — which changed how we framed every finding to leadership
That Confidence Note deserves a special mention. It pointed out that our respondents skewed heavily toward power users, which meant the 'too complex' theme was probably underrepresented relative to the silent newcomers who'd already churned and weren't around to answer surveys. We added that caveat to the deck, and it visibly changed how leadership weighed the feature-vs-simplicity debate. A confident analysis without that note would have quietly misled everyone.
The work that took three weeks now takes about three hours: one hour for the prompt, two hours for the team to verify, refine, and add the human judgment about what it all means. That's the right division of labor — the model reads and groups, the humans decide. Grab the Turn Raw Survey Responses into Themes You Can Trust prompt on Prompt Dock, point it at your backlog of open-ended responses, and get your research back in front of decisions instead of behind them. When a single response is a wall of text, I pre-clean it with my Messy Text to Clean Table, Zero Hallucinated Cells prompt first.