CHAPTERS
Why Casetext Went All-In on GPT-4
Jake Heller explains his background founding Casetext and how early access to GPT-4 changed what the company believed was possible in legal AI. He frames GPT-4 as the first model that could perform complex legal work at roughly human-level quality—while being much faster and scalable.
From “Nice Demo” to Product Pivot: Building CoCounsel
He describes the internal realization that GPT-4 enabled the product customers had been requesting for years: a true AI assistant for lawyers. That realization triggered a major company pivot to build CoCounsel around GPT-4 capabilities.
Validation Moments and Market Impact (Bar Exam + Adoption)
Jake highlights concrete proof points that increased confidence in GPT-4’s legal aptitude, including their bar exam performance research. He ties these results to why customers trusted the system enough to adopt it for real work.
Acquisition by Thomson Reuters and Continued Iteration
He briefly covers the Thomson Reuters acquisition and uses it to contextualize why these lessons matter at scale. The team continued refining “prompt engineering” practices post-acquisition as usage expanded.
Context Engineering vs Prompt Engineering: The Framing
Jake challenges the terminology, arguing most useful prompts are ‘instructions + context.’ He suggests the label matters less than mastering the two components and how they interact.
Design the Customer Experience First (Skills, Tools, and UX)
He outlines the top-down approach they used: start by defining the user experience and the “skills” the product should offer. CoCounsel is described as a chat interface with tool-like skills mapped to what lawyers do in practice.
Decompose Each Skill Like the World’s Best Professional Would
For each skill, the team asked: how would the best lawyer in the world do this with unlimited time? He walks through a legal research workflow (clarify → generate queries → search → review → notes → final answer) as the blueprint for the AI pipeline.
Linear Pipelines vs Agentic Loops (When to Program vs Let It Roam)
Jake distinguishes predictable, linear tasks from tasks requiring iteration and backtracking. If the process is stable, hard-code step-by-step execution; if success depends on exploration, use more agentic looping behavior.
Making Prompts Actually Work: Evals-Driven Iteration
He presents a practical workflow: write a best-guess prompt, create a small set of tests (evals), then iterate relentlessly until it passes. Most teams quit at partial success; his argument is that reliability comes from grinding through failures with disciplined evaluation.
Why Many “Prompt Problems” Are Actually Context Problems (Retrieval/OCR)
Jake emphasizes that models often fail because the provided context is incomplete, wrong, or unreadable—not because the instructions are bad. He points to retrieval quality and OCR errors as frequent root causes and recommends inspecting the exact input the model sees.
Operational Discipline: Don’t Ship Until You’ve Earned Reliability
He argues the real differentiator is willingness to iterate obsessively—often for weeks—until prompts are robust. He suggests shipping only once you’re near-perfect on large eval sets, while still setting realistic expectations with customers.
Practical Prompting Tricks: Speed, Output Control, and Decomposition
Jake shares tactical techniques to improve latency, reduce cost, and make evaluation easier. These include forcing single-token outputs with stop sequences, asking for a final answer first for easier scoring, and splitting large tasks into smaller prompts.
Reinforcement Fine-Tuning, Model Selection, and Multi-Model Pipelines
He closes by recommending reinforcement fine-tuning (RFT) as more effective than older fine-tuning approaches, often requiring fewer examples but clearer objective grading. He also advocates mixing models across steps to balance cost and accuracy, potentially using a different model per micro-step.
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome