Skip to content
Lenny's PodcastLenny's Podcast

Becoming evidence-guided | Itamar Gilad (Gmail, YouTube, Microsoft)

Itamar Gilad is a product coach, author, and speaker with over two decades of experience in senior product roles at Google, Microsoft, and various startups. He is also the author of Evidence-Guided: Creating High-Impact Products in the Face of Uncertainty and publishes a popular product management newsletter. In today’s episode, we discuss: • What it means to be “evidence-guided” • How to think about your KPIs as metric trees • How to prioritize ideas using the “confidence meter” • The GIST model for roadmapping • Common mistakes with ICE • Advice for using evidence to challenge gut-driven founders — Brought to you by Ezra—The leading full-body cancer screening company: http://www.ezra.com/lenny | Vanta—Automate compliance. Simplify security: https://vanta.com/lenny | LinkedIn Ads—Reach professionals and drive results for your business: https://www.linkedin.com/podlenny Find the full transcript at: https://www.lennysnewsletter.com/p/becoming-evidence-guided-itamar-gilad Where to find Itamar Gilad: • Twitter/X: https://twitter.com/ItamarGilad • LinkedIn: https://www.linkedin.com/in/itamargilad/ • Website: https://itamargilad.com/ Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • Twitter/X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ In this episode, we cover: (00:00) Itamar’s background (04:35) How his time working on Gmail shaped his philosophy of “opinion-based” development (08:35) Lessons from developing Gmail’s tabbed inbox (13:40) A brief overview of Itamar’s book, Evidence-Guided (14:30) Balancing founder creativity with an evidence-based approach (17:32) Advice on how to push back against founders (19:36) Signs you aren’t as evidence-guided as you may think (21:13) Itamar’s GIST model for becoming more evidence-guided (23:51) How to set overarching goals using his “value exchange loop” (28:45) North star metrics vs. KPIs (33:47) Using “ICE” to assess the value of ideas (37:39) Itamar’s confidence meter (44:28) Speed of delivery vs. speed of discovery (46:14) How to apply Itamar’s frameworks based on company type and stage (49:09) First steps in becoming more evidence-guided (50:21) Next steps in testing (55:41) The task layer in the GIST framework (1:02:54) Thoughts on roadmapping (1:04:56) How OKRs fit into the whole picture (1:07:11) Lightning round Referenced: • Itamar’s presentation slides: https://itamargilad.com/wp-content/uploads/2023/09/Podcast-Slides.pdf • What differentiates the highest-performing product teams | John Cutler (Amplitude, The Beautiful Mess): https://www.lennyspodcast.com/what-differentiates-the-highest-performing-product-teams-john-cutler-amplitude-the-beautiful-mess/ • Evidence-Guided: Creating High-Impact Products in the Face of Uncertainty: https://itamargilad.com/book-evidence-guided/ • The co-founders of Google in Forbes: https://www.forbes.com/profile/larry-page-and-sergey-brin • Kanban: https://www.atlassian.com/agile/kanban • Jira: https://www.atlassian.com/software/jira • The ultimate guide to OKRs | Christina Wodtke (Stanford): https://www.lennyspodcast.com/the-ultimate-guide-to-okrs-christina-wodtke-stanford/ • Amplitude: https://amplitude.com/ • The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon): https://www.lennyspodcast.com/the-ultimate-guide-to-ab-testing-ronny-kohavi-airbnb-microsoft-amazon/ • ICE framework: https://growthmethod.com/ice-framework/ • Sean Ellis on LinkedIn: https://www.linkedin.com/in/seanellis/ • RICE scoring model: https://www.productplan.com/glossary/rice-scoring-model/ • Idea Prioritization with ICE and the Confidence Meter: https://itamargilad.com/the-tool-that-will-help-you-choose-better-product-ideas/ • Assumptions Mapping: https://designsprintkit.withgoogle.com/methodology/phase2-define/assumptions-mapping • What is Dog Fooding, Fish Fooding a Product?: https://matt-rickard.com/fishfooding-dogfooding-product • SVPG books: https://www.svpg.com/books/ • The Lean series: https://theleanstartup.com/the-lean-series • Dreaming Spanish: https://www.youtube.com/c/DreamingSpanish • ElevenLabs: https://elevenlabs.io/ • Lennybot: https://www.lennybot.com/ Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. Lenny may be an investor in the companies discussed.

Itamar GiladguestLenny Rachitskyhost
Sep 20, 20231h 12mWatch on YouTube ↗

At a glance

WHAT IT’S REALLY ABOUT

From Opinions To Evidence: Building High-Impact Products That Work

  1. Lenny interviews Itamar Gilad, former Gmail and YouTube PM and author of *Evidence-Guided*, about moving from opinion-driven product development to an evidence-guided approach. Drawing on Google+ (a costly failure) and Gmail’s tabbed inbox (a massive success), Itamar shows how evidence, not conviction, should drive investment decisions. He introduces his GIST model—Goals, Ideas, Steps, Tasks—plus tools like metrics trees, ICE scoring, and the Confidence Meter to structure better decisions. The conversation focuses on practical ways teams can test ideas cheaply, align around value-creating metrics, and give product teams more autonomy while keeping leadership grounded in data.

IDEAS WORTH REMEMBERING

5 ideas

Opinion-based development is expensive and misleading, even at top companies.

Google+ consumed ~1,000 people over years based largely on leadership conviction and strategic fear of Facebook; users never truly wanted another social network, and the entire bet was ultimately rolled back.

Evidence-guided teams start from user outcomes and learn before they scale.

Gmail’s tabbed inbox began as a small, doubted idea; the team grounded it in a clear user problem (cluttered inboxes), ran cheap fake interfaces and internal tests, and only invested heavily after seeing strong user reactions and data.

Define success with a few shared metrics: value created and value captured.

Itamar recommends pairing a North Star metric (user value, e.g., WhatsApp messages sent, Airbnb nights booked) with a top business KPI (e.g., revenue) and breaking them into metrics trees to align teams, spot leverage points, and structure ownership.

Use ICE plus a Confidence Meter to prioritize ideas more objectively.

Rate ideas on Impact, Confidence, and Ease—but explicitly score confidence based on evidence type (from pure opinions at the bottom to strong tests and experiments at the top), limiting how far gut feel alone can boost an idea.

Validate assumptions cheaply before building full solutions.

Teams should progress from assessments and existing data to lightweight tests (fake doors, Wizard of Oz, usability tests), then to rough prototypes and limited rollouts, only moving to full-scale releases when evidence justifies the investment.

WORDS WORTH SAVING

5 quotes

For me, Google+ was the epitome of opinion-based development.

— Itamar Gilad

Every successful product company finds a way to balance human judgment with evidence.

— Itamar Gilad

You don’t need to obliterate opinion—just supercharge it with evidence.

— Itamar Gilad

The metric is not how fast we get bits into production; it’s how fast we get to outcomes.

— Itamar Gilad

If a boss gets mad at you for bringing real evidence, you should probably start polishing your résumé.

— Itamar Gilad

Google+ vs. Gmail tabbed inbox: opinion-based failure vs. evidence-guided successThe shift from opinion-based to evidence-guided product developmentThe GIST model: Goals, Ideas, Steps, TasksNorth Star metrics, top business KPIs, and metrics treesPrioritizing ideas with ICE and the Confidence MeterLow-cost validation and experimentation (fake doors, Wizard of Oz, dogfooding, A/B tests)Using GIST boards and outcome-focused roadmaps instead of feature roadmaps

High quality AI-generated summary created from speaker-labeled transcript.

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

Add to Chrome