CHAPTERS
- 0:00 – 1:00
GPT-5.6 is back—Claire’s head-to-head: Sol vs. Claude Fable
Claire sets up the episode as a practical comparison of OpenAI’s GPT-5.6 lineup—especially Sol—against Anthropic’s Claude Fable. She frames the review around real work: PRDs, prototypes, debugging, and “agentic voice,” using her own benchmark rather than vibes alone.
- •Motivation: losing access to GPT-5.6 briefly made her realize how central it is to her workflow
- •What will be tested: PRDs, prototyping, debugging, and agentic voice/human-likeness
- •Core question: is Fable better, or does GPT-5.6 (especially Sol) win in practice?
- 1:00 – 2:01
The GPT-5.6 family: Sol (frontier), Terra (balanced), Luna (cheap/high-volume)
She summarizes OpenAI’s three GPT-5.6 variants and how to think about their intended use. The episode is primarily a “love letter” to Sol, but she notes where Terra and Luna fit.
- •Sol: the most capable “frontier” model and her favorite
- •Terra: efficiency + capability balance for everyday work
- •Luna: lower-cost option for high-volume tasks
- •Focus of the rest: Sol vs. Fable for Claire’s day-to-day output quality
- 2:01 – 3:32
Pricing reality check: Sol is cheaper than Fable (especially on output tokens)
Claire compares API pricing and highlights that Sol undercuts Fable on both input and output. She also notes subscription availability uncertainty and how that shapes adoption.
- •Sol pricing: ~$5/M input, ~$30/M output tokens
- •Fable pricing (at recording): ~$10/M input, ~$50/M output tokens
- •Subscription access volatility (especially on the Anthropic side) affects experimentation
- •Prediction: Sol’s pricing pressures Fable’s subscription inclusion
- 3:32 – 4:02
Official benchmarks vs. “what I care about”: why she built the How I AI benchmark
She briefly nods to OpenAI’s published evaluations (including security-focused ones), then pivots to her own rubric. The point: model usefulness should be measured on the tasks she actually does.
- •OpenAI’s claims: strong performance on terminal/security/cybersecurity evals
- •Growing emphasis on safeguards and security frameworks for frontier models
- •Claire’s stance: product work needs practical evals beyond academic benchmarks
- •Transition into her custom benchmark harness
- 4:02 – 7:05
Inside the How I AI benchmark: tasks, models tested, and judging approach
Claire explains the benchmark components and the evaluation setup. She combines an LLM-judge score with a manual “taste test,” reviewing artifacts like PRDs, wireframes, prototypes, and agent voice.
- •Task suite: PRDs, wireframes, full prototypes, code debugging, agentic voice
- •Models compared: Fable 5, Sonnet 5, GPT-5.6 Sol/Terra/Luna
- •Eval harness: runs prompts across models + LLM-based judge (GPT-5.5 as strict judge)
- •Human review layer: Claire reads outputs, inspects designs, assigns scores/notes
- 7:05 – 9:07
Claire-weighted leaderboard: Sol wins overall (and why her weighting matters)
She reveals the combined scoring method: 70% Claire, 30% machine judge. With that weighting, GPT-5.6 Sol leads by a notable margin; Fable remains competitive when she doesn’t have to converse with it.
- •Claire’s weighting: 70% her taste, 30% LLM judge
- •Result: GPT-5.6 Sol highest overall by a significant gap
- •Fable’s caveat: decent outputs, but she dislikes interacting with it conversationally
- •Terra/Luna: “fine work”; Sonnet 5 last overall but has niche strengths
- 9:07 – 11:39
Per-task winners: Sol for prototypes, Terra for PRDs, Sonnet for voice, Sonnet for debugging (per judge)
She breaks down winners by category rather than overall. The nuance: different models shine in different tasks, especially voice and debugging, even if Sol dominates prototyping.
- •Full-fidelity prototypes: Sol preferred most often
- •PRD writing: Terra is her favorite for streamlined business writing
- •Bug-hunting/debug eval: LLM judge favored Sonnet 5 (she’s less confident in this eval)
- •Agentic voice: Sonnet 5 feels most “human” (despite em-dash quirks)
- •Design aesthetic note: Claude-style “editorial beige/orange/serif” is recognizable but not her favorite
- 11:39 – 13:11
What Claire rewards (and hates): non-slop design, functionality, and crisp writing
Claire surfaces her qualitative rubric: uniqueness + functional UX for design, and succinct, direct writing. She’s explicitly punishing “AI slop,” including clichéd aesthetics, placeholder-y layouts, and em-dash-heavy tone.
- •Design positives: uniqueness, creativity, clear hierarchy, real functionality
- •Writing positives: frank, crisp, non-AI-sounding prose
- •Design negatives: “Claude design slop,” emojis-as-crutches, bad placeholders, weak typography
- •Voice negative: em-dashes and performative ‘assistant speak’
- 13:11 – 17:45
Full-fidelity prototype side-by-sides: Sol’s functionality and visual hierarchy beat Fable’s sameness
Through multiple examples, she argues Sol produces more opinionated, usable prototypes with better semantics and interactions. Fable is “serviceable” but more generic, less readable, and sometimes has layout/typography issues.
- •Ops/doc-scheduler dashboard: Sol’s neutral palette + semantic color + hierarchy feel more usable
- •Fable outputs: competent but harder to read and less distinct; occasional spacing/layout issues
- •Creative pack site: both strong, but Sol adds personality and better design affordances
- •Dev tools/incident triage: Sol’s point-of-view design + dense technical UI handling stands out
- •Noted “tell”: Sol frequently leans into forest-green styling
- 17:45 – 19:16
Wireframes + agentic voice: Sol stays functional, but Sonnet sounds more human
On wireframes, Claire prefers Sol’s clarity and usability for communicating complex flows. On voice, she dislikes Sol’s dramatic phrasing and em-dashes, giving Sonnet the edge for assistant-like conversation.
- •Wireframes: Sol is easier to read and more actionable for complex apps
- •Fable wireframes: less clear on what the user should do next
- •Agentic voice test prompts: meeting changes, “why did I start this company,” prod deploy tone
- •Sonnet wins voice: less awkward overall; Sol’s phrasing feels too performative
- 19:16 – 22:18
Why Sol feels better day-to-day: practical effectiveness vs. Fable’s pedantry
Claire shifts from benchmark artifacts to lived experience. Her core thesis: Fable may be “theoretically hyper-intelligent,” but Sol collaborates better, communicates more clearly, and unblocks real shipping work.
- •Sol writes/communicates like a normal collaborator; Fable feels inscrutable and overly technical
- •Fable: detailed and hardworking, but low collaboration/UX intuition in product-building contexts
- •Claire’s framing: “theoretically intelligent” vs. “practically effective”
- •Sol is more willing to loosen constraints to reach user value and ship
- 22:18 – 27:54
One-shot, zero-to-one building: ChatPRD rebuild ideas and a gamified kids’ homework app
She shows Sol’s strength at generating robust, shippable-feeling prototypes quickly. Examples include a forward-looking ChatPRD concept and a surprisingly complete homework/XP app with focus mode, rewards, and parent controls.
- •ChatPRD greenfield vision: clearer executive-style doc + prototype that supports decision-to-handoff flow
- •Gamified homework app: quests, focus mode timer, confetti, rewards marketplace, parent HQ admin panel
- •Sol listens to specifics (basketball vs. Minecraft motivations) and reflects them in UI and rewards
- •Result: not fully consumer-grade, but unusually polished for a single pass
- 27:54 – 31:58
Breaking through over-hardened systems: how Sol/Codex fixed what Fable ‘locked down’
Claire describes two projects where Fable’s insistence on precision and hardened constraints made systems brittle. Switching to Codex + GPT-5.6 helped her escape rigid loops and generate more useful product outputs.
- •Prototype tool tool-calling loop: Fable’s hardening left only GPT-5.5 able to run; other models failed
- •Switching to Codex/Sol: more flexible approach produced a working solution quickly (even if imperfect)
- •Insights/product-brain system: Fable demanded deterministic, always-cited, verifiable prose—misaligned with product goals
- •Sol: stops over-linting/pedantry and produces practical wiki-style outputs from mixed signals
- 31:58 – 35:02
Two bonus use cases: fast video clipping and “beast mode” browser automation with @Chrome
She closes with two workflows where Sol shines beyond prototyping: editing long recordings into social clips and high-throughput browser tasks. Codex + GPT-5.6 paired with browser control is described as a major productivity multiplier.
- •Video editing: upload a talk, request multiple social clips, iterate on pacing/format, then polish in CapCut
- •Browser use: @Chrome in Codex on logged-in pages to triage LinkedIn messages with strict criteria
- •Automation scale: processes hundreds of messages, replies selectively, handles forms and web app testing
- •Key takeaway: losing GPT-5.6 access made these workflows noticeably worse
- 35:02 – 36:40
Final recommendations: when to use Sol vs. Fable vs. Sonnet (and what she’ll publish next)
Claire summarizes where each model fits: Sol for web apps and shipping-oriented work, Fable for strong code output when conversation isn’t required, and Sonnet for assistant voice. She invites feedback and plans to publish the benchmark outputs.
- •Sol: best overall for her benchmark; great at web apps, practical unblocking, video clipping, browser use
- •Fable: strong technical output, but frustrating to collaborate with conversationally
- •Sonnet: best “agentic voice” personality for her OpenClaw assistant use
- •Next steps: publish artifacts on the ChatPRD blog; asks viewers what to add to the benchmark
