This AI Expert's Method Will Change How You Do Customer Research

AI for user research is unreliable. But Caitlin Sullivan, one of the world's leading experts in user research, knows exactly how to fix it. In this episode, she demos the complete workflow for analyzing surveys and interviews with AI, using Claude, Claude Code, and agentic workflows that cut analysis time in half without hallucinating. Complete write-up: https://www.news.aakashg.com/p/caitlin-sullivan-podcast ---- Timestamps: 0:00 - Intro 1:54 - What Good AI Research Actually Looks Like 8:22 - Step 0: Loading Context Into Claude 11:34 - Why Claude Is the Best Model for Analysis 16:12 - Step 1: Per-Participant Analysis Prompting 26:06 - Step 2: Verification & Contradiction Checking 34:51 - Survey Analysis: Why You Must Code First 46:18 - Adding Emotional Intensity Ratings 51:31 - Step 3: Auditing AI's Own Work 57:42 - Claude Code: The Agentic Parallel Version 1:09:01 - Final Output & Results ---- 🧠 Key Takeaways: 1. Replicate the human process - Good AI analysis mirrors how experienced researchers work: comb through data first, then synthesize. Never jump straight to "give me themes." 2. Use multi-step prompting - Load context in one prompt, run per-participant analysis in the next, then verify. Cramming everything into one prompt degrades quality. 3. Code before you count - For surveys, apply inductive coding labels to every response before asking for patterns. Skipping this step leads to miscategorized, unreliable results. 4. Always audit AI's work - Force the model to re-check its own analysis. It catches contradictions, overexaggerated intensity ratings, and miscoded responses regularly. 5. Claude wins on nuance, Gemini wins on frequency - Claude gives more thorough, complete analysis by default. Gemini surfaces top-frequency themes faster but misses smaller patterns. 6. Define everything explicitly - Quotes, ratings, emotional intensity levels, contradiction types. If you assume the model shares your definitions, you'll get inconsistent results. 7. Markdown files beat raw transcripts - Converting transcripts to structured markdown improves accuracy and helps you work around token limits on non-Max plans. 8. Parallelize with Claude Code agents - Set up agent markdown files for interview and survey analysis, then run both simultaneously. Cuts total analysis time in half again. ---- 🏆 Sponsors: 1. Maven: Get 15% off Caitlin’s courses with code AAKASHxMAVEN - https://bit.ly/4rHCCrb 2. Pendo: The #1 software experience management platform - http://www.pendo.io/aakash 3. Jira Product Discovery: Plan with purpose, ship with confidence - https://www.atlassian.com/software/jira/product-discovery 4. Kameleoon: AI experimentation platform - http://www.kameleoon.com/ 5. Amplitude: The market-leader in product analytics - https://amplitude.com/session-replay?utm_campaign=session-replay-launch-2025&utm_source=linkedin&utm_medium=organic-social&utm_content=productgrowthpodcast ---- 👨‍💻 Where to find Aakash: Twitter: https://www.x.com/aakashg0 LinkedIn: https://www.linkedin.com/in/aagupta/ Newsletter: https://www.news.aakashg.com Premium Bundle: https://bundle.aakashg.com Where to find Caitlin: LinkedIn: https://www.linkedin.com/in/caitlindsullivan/ Maven: https://bit.ly/4rHCCrb #aitools #userresearch ---- 🧠 About Product Growth: The world's largest podcast focused solely on product + growth, with over 200K+ listeners. 🔔 Subscribe and turn on notifications to get more videos like this.

Aakash GuptahostCaitlin Sullivanguest

Feb 12, 20261h 12mWatch on YouTube ↗

EPISODE INFO

Released: February 12, 2026
Duration: 1h 12m
Channel: Aakash Gupta
Watch on YouTube: ▶ Open ↗

EPISODE DESCRIPTION

AI for user research is unreliable. But Caitlin Sullivan, one of the world's leading experts in user research, knows exactly how to fix it. In this episode, she demos the complete workflow for analyzing surveys and interviews with AI, using Claude, Claude Code, and agentic workflows that cut analysis time in half without hallucinating. Complete write-up: https://www.news.aakashg.com/p/caitlin-sullivan-podcast ---- Timestamps: 0:00 - Intro 1:54 - What Good AI Research Actually Looks Like 8:22 - Step 0: Loading Context Into Claude 11:34 - Why Claude Is the Best Model for Analysis 16:12 - Step 1: Per-Participant Analysis Prompting 26:06 - Step 2: Verification & Contradiction Checking 34:51 - Survey Analysis: Why You Must Code First 46:18 - Adding Emotional Intensity Ratings 51:31 - Step 3: Auditing AI's Own Work 57:42 - Claude Code: The Agentic Parallel Version 1:09:01 - Final Output & Results ---- 🧠 Key Takeaways:
1. Replicate the human process - Good AI analysis mirrors how experienced researchers work: comb through data first, then synthesize. Never jump straight to "give me themes."
1. Use multi-step prompting - Load context in one prompt, run per-participant analysis in the next, then verify. Cramming everything into one prompt degrades quality.
1. Code before you count - For surveys, apply inductive coding labels to every response before asking for patterns. Skipping this step leads to miscategorized, unreliable results.
1. Always audit AI's work - Force the model to re-check its own analysis. It catches contradictions, overexaggerated intensity ratings, and miscoded responses regularly.
1. Claude wins on nuance, Gemini wins on frequency - Claude gives more thorough, complete analysis by default. Gemini surfaces top-frequency themes faster but misses smaller patterns.
1. Define everything explicitly - Quotes, ratings, emotional intensity levels, contradiction types. If you assume the model shares your definitions, you'll get inconsistent results.
1. Markdown files beat raw transcripts - Converting transcripts to structured markdown improves accuracy and helps you work around token limits on non-Max plans.
1. Parallelize with Claude Code agents - Set up agent markdown files for interview and survey analysis, then run both simultaneously. Cuts total analysis time in half again. ---- 🏆 Sponsors:
Maven: Get 15% off Caitlin’s courses with code AAKASHxMAVEN - https://bit.ly/4rHCCrb
Pendo: The #1 software experience management platform - http://www.pendo.io/aakash
Jira Product Discovery: Plan with purpose, ship with confidence - https://www.atlassian.com/software/jira/product-discovery
Kameleoon: AI experimentation platform - http://www.kameleoon.com/
Amplitude: The market-leader in product analytics - https://amplitude.com/session-replay?utm_campaign=session-replay-launch-2025&utm_source=linkedin&utm_medium=organic-social&utm_content=productgrowthpodcast
---- 👨‍💻 Where to find Aakash: Twitter: https://www.x.com/aakashg0 LinkedIn: https://www.linkedin.com/in/aagupta/ Newsletter: https://www.news.aakashg.com Premium Bundle: https://bundle.aakashg.com Where to find Caitlin: LinkedIn: https://www.linkedin.com/in/caitlindsullivan/ Maven: https://bit.ly/4rHCCrb #aitools #userresearch ---- 🧠 About Product Growth: The world's largest podcast focused solely on product + growth, with over 200K+ listeners. 🔔 Subscribe and turn on notifications to get more videos like this.

SPEAKERS

Aakash Gupta
host
Host of the channel/podcast and a product management professional who interviews experts on AI and customer research.
Caitlin Sullivan
guest
User research expert and leader who teaches rigorous AI-assisted customer research and qualitative analysis workflows.

EPISODE SUMMARY

In this episode of Aakash Gupta, featuring Aakash Gupta and Caitlin Sullivan, This AI Expert's Method Will Change How You Do Customer Research explores a rigorous, multi-step AI workflow for trustworthy customer research analysis Good AI research mirrors rigorous human research by separating analysis, verification, and synthesis rather than jumping straight to themes.

RELATED EPISODES