Skip to content
Lenny's PodcastLenny's Podcast

Kevin Weil: Why evals are the new core skill in AI products

Through fine-tuning runs and writing evals against the fuzzy outputs; OpenAI builds at the edge of capabilities, betting on better models every two months.

Kevin WeilguestLenny Rachitskyhost
Apr 10, 20251h 31mWatch on YouTube ↗

CHAPTERS

  1. 0:00 – 5:16

    Kevin’s background

  2. 5:16 – 8:13

    OpenAI’s new image model

  3. 8:13 – 11:42

    The role of chief product officer at OpenAI

  4. 11:42 – 15:59

    His recruitment story and joining OpenAI

  5. 15:59 – 18:44

    Working at OpenAI

  6. 18:44 – 24:40

    The importance of evals in AI

  7. 24:40 – 26:34

    Opportunities in the space

  8. 26:34 – 29:47

    Shipping quickly and consistently

  9. 29:47 – 32:53

    Product reviews and iterative deployment

  10. 32:53 – 36:03

    Winning consumer awareness

  11. 36:03 – 40:56

    Designing thoughtful experiences

  12. 40:56 – 45:21

    Chat as an interface for AI

  13. 45:21 – 48:05

    Collaboration between researchers and product teams

  14. 48:05 – 53:06

    Hiring product managers at OpenAI

  15. 53:06 – 1:04:34

    How OpenAI uses AI: vibe coding, AI prototyping, and more

  16. 1:04:34 – 1:08:07

    Raising kids in an increasingly intelligent AI world

  17. 1:08:07 – 1:14:20

    Why Kevin feels optimistic about our AI future

  18. 1:14:20 – 1:17:58

    The AI model you're using today is the worst AI model you'll ever use

  19. 1:17:58 – 1:21:51

    Reflections on the Libra project

  20. 1:21:51 – 1:31:40

    Lightning round and final thoughts

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

Add to Chrome