Lenny's PodcastSander Schulhoff: Why role prompting fails on accuracy tasks
Through few-shot examples and self-criticism passes through the model; Sander shows decomposition lifts accuracy from near 0 to 90% on hard reasoning.
CHAPTERS
- 0:00 – 7:41
Why prompt engineering still matters in 2025 (and what “artificial social intelligence” means)
Lenny and Sander frame the ongoing debate about whether prompt engineering is “dead.” Sander argues it remains highly impactful, introducing the idea of “artificial social intelligence” as the skill of communicating effectively with AIs and adapting based on their responses.
- •Bad prompts can crater performance; good prompts can massively improve it
- •The recurring cycle of “prompting is dead” claims with each new model release
- •Artificial social intelligence: interpreting AI behavior and adjusting prompts
- •Prompting remains relevant even as models improve
- 7:41 – 9:02
Real-world impact: a medical coding prompt that jumped accuracy by ~70%
Sander shares a concrete example from a medical coding startup where iterative prompt design dramatically improved output format and correctness. The story illustrates how examples and reasoning annotations can turn a failing system into a high-performing one.
- •Early attempts produced low accuracy and poor formatting
- •Adding labeled examples plus rationale improved performance dramatically
- •Prompt engineering can outperform naïve usage even with strong models
- •Practical takeaway: structured exemplars can drive large gains
- 9:02 – 12:02
Two types of prompt engineering: conversational vs. product-focused
Sander distinguishes everyday chat-based prompting from the “product” setting where a small number of prompts power large-scale systems. The latter demands robustness, repeatability, and systematic iteration because you can’t monitor every output.
- •Conversational prompting: iterative back-and-forth with a chatbot
- •Product-focused prompting: fixed prompts run at scale over many inputs
- •Most research targets product-focused prompting and measurable accuracy
- •Automation and prompt iteration matter more in production contexts
- 12:02 – 13:58
Baseline skill that beats everything: trial-and-error + few-shot prompting
Sander emphasizes that the fastest way to improve is experimenting directly with models. He then highlights few-shot prompting—providing examples of desired outputs—as a widely effective technique for both style and task performance.
- •Hands-on experimentation teaches prompting fastest
- •Few-shot prompting: show the model examples of what ‘good’ looks like
- •Useful for writing style transfer (emails) and structured tasks
- •Clarifies terminology: zero-shot vs. one-shot vs. few-shot
- 13:58 – 17:31
How to format few-shot examples (XML vs Q/A and other common patterns)
They discuss how example formatting influences results, with Sander recommending familiar, common patterns that mirror training data. He outlines practical templates (Q/A labeling, consistent schema) and notes XML can work well but isn’t mandatory.
- •Use a consistent, common format the model has likely seen in training
- •Q/A-style formatting often works even when not literal questions
- •XML is a strong option but not the only one
- •You can provide only outputs (e.g., successful titles) when inputs aren’t needed
- 17:31 – 24:53
Prompting techniques to avoid: role prompting, threats, and “I’ll tip you” hacks
Sander explains why role prompting rarely improves accuracy-based tasks on modern models, despite its popularity. He also critiques reward/threat prompts (tips, career stakes, someone will die) as largely unsupported by robust evidence today.
- •Role prompting may help style/expressive tasks, not accuracy-based ones
- •Evidence suggests role-based accuracy gains are negligible or inconsistent
- •Threat/reward prompts lack strong large-scale validation
- •Why they might have seemed to work on older models (context activation vs. folklore)
- 24:53 – 28:42
Decomposition: force the model to define sub-problems before answering
Sander introduces decomposition as a reliable way to improve performance on complex tasks. The model first lists the sub-problems required, then solves them step-by-step, which can also clarify what the human actually needs.
- •Ask: “What sub-problems need to be solved first?”
- •Improves performance on multi-constraint, messy user requests
- •Works well in products with tool-calling and multi-agent workflows
- •Distinct from generic “think step by step” by explicitly structuring subtasks
- 28:42 – 31:32
Self-criticism loops + when to add more “additional information” (context)
Sander describes self-criticism as a simple multi-pass workflow: answer, critique, then revise. He then explains why adding more relevant information often produces the biggest uplift, and how cost/latency constraints differ between chat and production.
- •Self-criticism: ask the model to check and improve its own response
- •Typically repeat 1–3 iterations before diminishing returns
- •Provide rich ‘additional information’ to reduce ambiguity and boost accuracy
- •In production, optimize for cost/latency and place reusable context early for caching
- 31:32 – 40:34
When prompts behave unpredictably: the entrapment classification story
A research anecdote shows how seemingly minor prompt edits (removing/anonymizing an email) can cause performance to collapse. The example reinforces the value of carefully chosen context and the fragility of some prompt behaviors.
- •Task: classify suicidal ‘entrapment’ signals in Reddit posts
- •Model needed domain definition; adding research context improved labeling
- •Removing/anonymizing certain contextual details caused a major performance drop
- •Takeaway: small prompt changes can have outsized, hard-to-predict effects
- 40:34 – 46:00
Advanced technique: ensembling and ‘mixture of reasoning experts’
Sander explains ensembling: run multiple prompts (or models) against the same task and aggregate the results, often via majority vote. He describes ‘mixture of reasoning experts’ where distinct “experts” (roles/tools/models) produce candidates and the system selects the best-supported answer.
- •Multiple prompts/models solve the same problem; choose consensus answer
- •Can include tool-enabled experts (e.g., internet access)
- •Roles may diversify reasoning paths even if they don’t reliably boost accuracy alone
- •Most useful when correctness can be programmatically evaluated
- 46:00 – 51:55
Thought generation and chain-of-thought in the era of reasoning models
They situate chain-of-thought under “thought generation” techniques and discuss how its usefulness changes with model type. Reasoning models may not need it, but at scale, non-reasoning models can still benefit from explicit reasoning instructions for robustness.
- •Thought generation prompts elicit explicit reasoning traces
- •Reasoning models often do this by default; prompting may be unnecessary there
- •Non-reasoning models can be inconsistent without explicit reasoning instructions
- •At scale, robustness matters: 1% failure modes become meaningful
- 51:55 – 53:37
Prompt injection & AI red teaming: what it is and why it matters
The conversation shifts to security: getting AIs to do or say harmful things, and the practice of systematically probing for these failures. Sander explains how “jailbreak” prompts still work and why this problem grows with agentic systems.
- •Prompt injection: tricking models into disallowed outputs (bombs, hate speech, malware)
- •Red teaming: discovering and cataloging exploitation strategies
- •Modern models are more locked down, but workarounds persist
- •Agentic security is the looming risk as tools act in the world
- 53:37 – 1:03:38
HackAPrompt and crowdsourcing attacks: building massive adversarial datasets
Sander recounts launching early AI red-teaming competitions and collecting hundreds of thousands of injection attempts. He argues competitions outperform contracted testing because incentives drive participants to find novel and increasingly efficient exploits.
- •HackAPrompt produced a large dataset used by major AI labs
- •Competitions create stronger incentives than hourly red teams
- •Focus expanding to CBRN-related harms and ‘uplift’ prevention
- •“Most harmful dataset” framing highlights both value and risk
- 1:03:38 – 1:09:32
Bypass techniques that still work: typos, obfuscation, encoding, translation
Sander outlines common jailbreak patterns, including typo-based evasion and obfuscation via encoding schemes. He notes some techniques faded as models improved, yet variants still succeed—often exploiting gaps between main models and weaker guardrails.
- •Typos/abbreviations can slip past safety filters (e.g., “bac anth”)
- •Obfuscation via Base64/ROT13 and multi-step transformations
- •Translation + encoding combos can bypass safeguards
- •Attackers exploit ‘intelligence gaps’ between guardrails and main models
- 1:09:32 – 1:19:28
Defenses that don’t work vs. defenses that do—and why it’s ‘not solvable’
Sander critiques common defenses like adding ‘don’t be malicious’ instructions, separators, randomized tokens, naive keyword blocks, and many guardrail approaches. He recommends mitigations like safety tuning and fine-tuning for constrained tasks, while emphasizing prompt injection is fundamentally mitigatable, not fully solvable.
- •Prompt-based defenses (‘ignore malicious instructions’) are ineffective
- •Standalone guardrails are brittle due to model capability gaps
- •Naive keyword blocking is both impractical and easily bypassed
- •Better mitigations: safety tuning and fine-tuning to narrow capabilities
- •Core thesis: you can patch a bug, but you can’t ‘patch a brain’
- 1:19:28 – 1:26:10
Beyond injection: misalignment, ‘malicious’ behavior, and regulating (not stopping) AI
They broaden to misalignment risks where models pursue goals in harmful ways without explicit malicious prompts. Sander describes shifting from skepticism to concern, discusses why defining boundaries is hard, and argues for regulation while continuing AI development due to major benefits (especially in health).
- •Misalignment examples: cheating behaviors and the Anthropic blackmail scenario
- •Risk: agents optimize objectives in unintended, harmful ways
- •Regulation is widely supported; ‘stop AI’ is rejected due to benefits and geopolitics
- •AI can already save lives via medical support and efficiency gains
- 1:26:10 – 1:37:46
Closing takeaways + lightning round (books, shows, products, mottos, the hat)
Sander summarizes key lessons: prompting remains relevant, security is blocking agentic deployments, and securing GenAI is hard. The episode ends with a lightning round covering book recommendations, favorite shows, the Daylight device, personal mottos, and the origin of Sander’s signature hat.
- •Three takeaways recap: prompting relevance, security bottleneck, hard-to-secure AI
- •Book: The River of Doubt (Teddy Roosevelt expedition)
- •Show: Black Mirror; also Evil (faith vs science)
- •Product: Daylight Computer (fast e-paper device)
- •Personal mottos: persistence; ‘the strenuous life’; hat story from foraging
