Skip to content
Lenny's PodcastLenny's Podcast

Sander Schulhoff: Why role prompting fails on accuracy tasks

Through few-shot examples and self-criticism passes through the model; Sander shows decomposition lifts accuracy from near 0 to 90% on hard reasoning.

Lenny RachitskyhostSander SchulhoffguestGuest (Vanta sponsor segment)guest
Jun 19, 20251h 37mWatch on YouTube ↗

CHAPTERS

  1. 0:00 – 4:56

    Introduction to Sander Schulhoff

  2. 4:56 – 9:01

    The importance of prompt engineering

  3. 9:01 – 12:02

    Two modes for thinking about prompt engineering

  4. 12:02 – 17:30

    Few-shot prompting

  5. 17:30 – 24:52

    Prompting techniques to avoid

  6. 24:52 – 28:26

    Decomposition

  7. 28:26 – 40:29

    Self-criticism and context

  8. 40:29 – 45:59

    Ensembling

  9. 45:59 – 48:23

    Thought generation

  10. 48:23 – 51:56

    Conversational vs. product-focused prompt engineering

  11. 51:56 – 53:37

    Introduction to prompt injection and red teaming

  12. 53:37 – 55:23

    AI red teaming competitions

  13. 55:23 – 1:03:39

    The growing importance of AI security

  14. 1:03:39 – 1:06:17

    Techniques to bypass AI safeguards

  15. 1:06:17 – 1:09:31

    Challenges in AI security and future outlook

  16. 1:09:31 – 1:13:18

    Common defenses to prompt injection that don't actually work

  17. 1:13:18 – 1:16:33

    Defenses that do work

  18. 1:16:33 – 1:19:29

    Misalignment and AI's potential risks

  19. 1:19:29 – 1:26:05

    Are LLMs behaving maliciously?

  20. 1:26:05 – 1:37:46

    Final thoughts and lightning round

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

Add to Chrome