Skip to content

By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

How I AIHow I AI

I benchmarked the NEW Sonnet 5. The results shocked me.

I’ve been testing every major frontier model release since the start of the year, and when Anthropic dropped Sonnet 5, I wanted more than a vibe check.

Claire Vohost
Jun 30, 202625mWatch on YouTube

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

Great conversations run long. Their best ideas don't have to.

uListen is free on iPhone, iPad, Android, Chrome, and Claude.

See how

Episode Details

EPISODE INFO

Released
June 30, 2026
Duration
25m
Channel
How I AI
Watch on YouTube
▶ Open ↗

EPISODE DESCRIPTION

I’ve been testing every major frontier model release since the start of the year, and when Anthropic dropped Sonnet 5, I wanted more than a vibe check. I got tired of one-off tests I couldn’t repeat or compare over time, so I built something better: the How I AI Bench, a repeatable eval harness I constructed live using Claude Code while recording this episode. I ran Sonnet 5 blind against four other frontier models (Sonnet 4.6, Opus 4.8, GPT-5.5, and Gemini 3 Pro) across PRD quality, prototype generation, agentic task completion, and agent personality. The results were not what I expected. *What you’ll learn:*

  1. What Anthropic claims Sonnet 5 improves over Sonnet 4.6, and where the benchmark data actually backs that up
  2. How I built the How I AI Bench in under 45 minutes using Claude Code, starting from my own stored session history
  3. Why I combined human vibe scoring (70%) with LLM as judge scoring (30%) instead of trusting either alone
  4. How to set up a local HTML scoring page so you can rate AI outputs on gut feel and export those scores as JSON
  5. Which model I recommend for PRDs, which for complex prototypes, and which for chatting with an agent daily

*Brought to you by:* Runway—The creative AI platform for images, video and more: https://runwayml.com/howIAI Hyperagent—Deploy fleets of agents that handle real work: https://www.hyperagent.com/howiai *In this episode, we cover:* (00:00) Sonnet 5 is out (01:55) What Anthropic claims (04:02) Why I’m done with one-off vibe checks (05:05) Building the How I AI Bench live with Claude Code (07:42) The scoring system (10:43) Agent voice eval (11:57) Quick recap (13:58) Results: The How I AI index leaderboard (21:21) What I’m improving for the next run (22:16) Generating a Claire-weighted index (23:53) Model-by-task recommendations *Blog and detailed workflow walkthroughs from this episode:* Building a Custom Benchmark for Sonnet 5, and Why the Results Surprised Me: https://www.chatprd.ai/how-i-ai/sonnet-5-review-and-custom-benchmark ↳ How to Conduct a Blind ‘Vibe Check’ to Evaluate AI Model Quality: https://www.chatprd.ai/how-i-ai/workflows/how-to-conduct-a-blind-vibe-check-to-evaluate-ai-model-quality ↳ How to Build a Custom AI Model Benchmark Using Claude Code: https://www.chatprd.ai/how-i-ai/workflows/how-to-build-a-custom-ai-model-benchmark-using-claude-code ↳ How to Create a Weighted Index for AI Model Benchmark Results: https://www.chatprd.ai/how-i-ai/workflows/how-to-create-a-weighted-index-for-ai-model-benchmark-results *Tools referenced:*

*Other references:*

• SWE-bench Pro (agentic coding benchmark referenced): https://www.swebench.com/ *Where to find Claire Vo:* ChatPRD: https://www.chatprd.ai/ Website: https://clairevo.com/ LinkedIn: https://www.linkedin.com/in/clairevo/ X: https://x.com/clairevo _Production and marketing by https://penname.co/._ _For inquiries about sponsoring the podcast, email jordan@penname.co._

SPEAKERS

  • Claire Vo

    host

    Host of the "How I AI" show, covering practical AI tools and model evaluations.

EPISODE SUMMARY

In this episode of How I AI, featuring Claire Vo, I benchmarked the NEW Sonnet 5. The results shocked me. explores Claire benchmarks Sonnet 5, finds surprises, builds repeatable eval index Anthropic positions Claude Sonnet 5 as a more agentic, near-Opus-performance model at significantly lower cost, especially for long-running tool use and computer-use tasks.

RELATED EPISODES

GPT-5.6 Sol: Better AND cheaper than Fable

GPT-5.6 Sol: Better AND cheaper than Fable

What is an AI harness? I build one live in less than 30 minutes

What is an AI harness? I build one live in less than 30 minutes

How I run autonomous coding agents from my phone with OpenAI Symphony + Linear

How I run autonomous coding agents from my phone with OpenAI Symphony + Linear

How Gusto’s CTO uses Claude Code to ship like a startup

How Gusto’s CTO uses Claude Code to ship like a startup

GLM 5.2 is SO GOOD (and almost free)

GLM 5.2 is SO GOOD (and almost free)

How Mozilla Uses Claude Mythos to find Firefox bugs before hackers do

How Mozilla Uses Claude Mythos to find Firefox bugs before hackers do

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.