By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
Great conversations run long. Their best ideas don't have to.
uListen is free on iPhone, iPad, Android, Chrome, and Claude.
See howCHAPTERS
Jump to publisher timestampsChapter summaries
Prompt caching as the go-to optimization (opening takeaway)
Lucca opens with a clear claim: prompt caching is the first recommendation he gives customers because it meaningfully reduces costs. This sets up cost, performance, and workflow optimization as recurring themes for the rest of the discussion.
- •Lucca says many roads lead to prompt caching, even if not all.
- •He describes prompt caching as the first recommendation given to customers.
- •He frames caching as a meaningful lever for reducing costs.
Messages API as the cornerstone: turns, tool calls, and control knobs
Lucca describes the Messages API as the foundation of the Claude API, built around a familiar structure of user turns, assistant turns, and tool calls. He emphasizes that the API includes built-in “knobs” that enable behavior control beyond what prompting alone can do.
- •The Messages API is described as the cornerstone that can be customized to build many kinds of applications.
- •The core interaction structure includes user turns, assistant turns, and tool calls.
- •Built-in controls provide an “easier path” to desired app behavior versus building from first principles.
- •Lucca highlights “effort” as a knob that can make the same model feel like two different models.
- •He says these knobs unlock customization that isn’t available through prompting alone.
How feature decisions get made: tight loop between research and customer needs
Jake asks how the team decides what to ship, and Lucca explains it as a two-way conversation with researchers. Some capabilities originate in research exploration, while others are driven by customer requests that feed into future training and product work.
- •Lucca describes a two-way feedback loop with researchers.
- •New ideas can flow from research experiments into productized API features (example: noticing Claude improving at using computers).
- •Customer conversations also drive requests that become inputs for future model and API work.
- •He notes there’s a pipeline, so asks affect “new Claudes,” not older ones.
- •This ongoing loop is described as a major part of Lucca’s job.
Effort and mid-conversation system prompt changes—without breaking caching
Lucca points to two capabilities that seem small but matter significantly in practice: changing effort and system prompts mid-conversation. He notes that while developers can always change these settings, doing so typically breaks prompt caching—so enabling changes without losing caching is a key advantage.
- •Lucca highlights changing effort mid-conversation as highly empowering in real workflows.
- •He also highlights changing system prompts mid-conversation.
- •He notes these changes traditionally break prompt caching, which he calls a deal breaker for users.
- •Jake frames the goal as empowering the experience while keeping costs low and intelligence high.
- •Lucca gives an example: switching from high effort to low effort for an easier next step without losing cache savings.
From idea to launch: evals, validation, and customers running their own tests
Asked what it takes to move from idea to launch, Lucca centers the process on evaluation. Both internal testing and customer-specific evals matter, because the API is generic while each customer’s harness and personalization are unique.
- •Lucca says the first requirement after an idea is understanding how well Claude works, usually via evals at each step.
- •Jake emphasizes ensuring workflows don’t break as changes roll out.
- •Lucca says the team runs many evals and recommends customers run their own evals when adopting knobs.
- •He explains that because the API serves many use cases, customers must validate behavior for their specific context.
Model versioning vs API versioning: upgrades, rollouts, and harness measurement
Jake contrasts traditional API versioning with model-driven versioning, where the model string functions as a key version mechanism. The discussion frames versioning as intertwined with prompt stability, rollout switches, and the requirement that developers continuously test harness outcomes.
- •Jake describes traditional API versioning as important but painful, and says models create a different version mechanism.
- •He cites model strings, API versioning, and prompt stability as intertwined elements.
- •Jake says users are often willing to adjust prompts/harnesses because the value of intelligence improvements is high.
- •He emphasizes that harnesses still need to be tested and validated across changes.
- •Jake lists measurements developers should consider: tokens used, intelligence, and task outcomes.
Deprecations and migration paths: toggles, beta headers, and the end date reality
Lucca explains that additive capability changes are easy, while deprecating capabilities can be difficult when users rely on them. He describes strategies like offering toggles and rollback mechanisms temporarily, while acknowledging that migrations sometimes must end when old behaviors are no longer supported by training.
- •Lucca says implicit versioning is easy when adding capabilities, but hard when deprecating relied-upon behavior.
- •He gives an example of changing from extended thinking to adaptive thinking, while keeping a toggle to switch back for a time.
- •Jake mentions beta headers and other rollback mechanisms to provide time to update harnesses.
- •Lucca says at some point a harder deprecation is necessary when models stop being trained for old behavior.
- •He describes maintaining one or two models to allow gradual migration while changing what is recommended.
Developer upgrade loop: reevaluating harnesses as models outgrow the scaffolding
Lucca argues developers should periodically re-evaluate their entire setup as models change, including when using Claude Code. He suggests that as models become much stronger, existing harness logic can become the limiting factor, and forced migrations can prompt simplification and better caching (and therefore savings).
- •Lucca recommends periodically reevaluating workflows and setup as models change, including in products like Claude Code.
- •He says model improvements can change which parts of a harness matter or need personalization.
- •Lucca claims a much smarter model can make the harness itself the limitation.
- •He frames forced deprecations as a forcing function to reassess unnecessary harness pieces.
- •He links harness changes to improved prompt caching and cost savings.
Cost management: prompt caching mechanics and aiming for high cache rates
Jake shifts the conversation to cost controls, and Lucca reinforces prompt caching as the biggest lever. Lucca explains caching as reusing blocks from similar, append-style requests in agentic flows, and notes that customer conversations often focus quickly on cache rate optimization.
- •Jake calls prompt caching one of the largest levers for controlling cost.
- •Lucca reiterates prompt caching as the first recommendation for customers who haven’t optimized it.
- •He explains caching works well when a second request is very similar to the first—more like appending new information.
- •Lucca says teams ask customers how much they’ve worked on prompt caching rates.
- •He cites aiming for very high prompt caching rates as the “dream numbers” because it translates directly to cost improvements.
Cost per token vs cost per task: intelligence per token and model choice tradeoffs
Jake and Lucca discuss why cost per token alone can be misleading without considering the speed and effectiveness of outcomes. Jake introduces “intelligence per token” and argues that model selection and tuning should be guided by evals that reflect end outcomes and total tokens used, not only token price.
- •Lucca asks how to think about cost per task versus cost per token, noting features can change token usage without changing per-token pricing.
- •Jake says cost per token needs an added dimension: “intelligence per token,” or how quickly outcomes are achieved.
- •He compares choosing a higher-intelligence model versus a lower-intelligence model in terms of tokens needed for the same outcome.
- •Jake returns to the importance of evals to validate outcomes while shifting models or knobs.
- •Lucca notes a stronger (more expensive) model can be worth it if it solves a task you couldn’t solve before; Jake adds that improving lower-tier models can also enable downgrading for the same workflow.
Balancing access and safety: Mythos vs Fable, classifiers, and distillation concerns
Jake raises misuse risk and safety priorities, and Lucca describes the tension between broad access for beneficial use and limiting harmful use. Lucca explains a split between Mythos and Fable access paths, notes widespread classifier use, and highlights distillation prevention as a major safety concern, including recent “thinking protections.”
- •Lucca describes excitement about new model capabilities alongside concerns about what bad actors might do.
- •He says balancing broad access with safety is a major focus area and points to a clear split between Mythos and Fable models.
- •Lucca says every model in the API runs with many classifiers intended to prevent bad usage.
- •He describes Mythos access as requiring an extensive KYC process, while Fable is available to those who don’t go through it.
- •Lucca cites distillation risk as a serious concern and says the team rolled out changes described as “thinking protections” to prevent distillation.
Safeguards without breaking developer experience: data-driven iteration and tougher evals
Lucca explains that balancing safeguards with usability relies on extensive experimentation and measurement. Jake adds that as models improve, evals can saturate, requiring the team to design harder evals so progress remains measurable.
- •Jake asks how the team balances developer experience with safeguards.
- •Lucca says the team runs an “absurd amount” of evals and tests many internal variations before shipping one public choice.
- •He describes the approach as difficult but extremely data-driven, involving many people reviewing numbers over time.
- •Jake notes it can become necessary to make evals harder as models perform extremely well even after changes.
- •Lucca calls eval saturation a “funny, good problem” when an eval stops being meaningful because every model can solve it.
What’s next: quality bars, rerunning shelved features, and an agents-first API evolution
Lucca says the most exciting future direction is new capabilities enabled by increasing model intelligence, paired with a strict quality threshold before shipping. The conversation closes on how the API is designed for agentic flows, including cloud managed agents built on top of the Messages API, and a continued push toward agent abstractions, efficiency, and speed.
- •Lucca says new capabilities are the most exciting aspect of new model launches, driven by model intelligence.
- •He describes rerunning evals on previously unshipped features to see if they now meet the quality bar.
- •Lucca emphasizes that every API feature needs a “stamp of quality.”
- •He describes the API as designed primarily for agents, including cloud managed agents and Messages API features tuned for agentic workflows.
- •Jake points to first-party apps as demonstrations of what the Messages API can do and expects continued evolution toward agents, efficiency improvements, and faster experiences; Lucca notes cloud managed agents are built on top of the Messages API.
