Skip to content
Lenny's PodcastLenny's Podcast

Marketplace lessons from Uber, Airbnb, Bumble, and more | Ramesh Johari (Stanford professor)

Ramesh Johari is a professor at Stanford University focusing on data science methods and practice, as well as the design and operation of online markets and platforms. Beyond academia, Ramesh has advised some incredible startups, including Airbnb, Uber, Bumble, and Stitch Fix. Today we discuss: • What exactly a marketplace is, if you boil it down • What you need to get right to build a successful marketplace • How to optimize any marketplace • An easy litmus test to see if there’s an opportunity to build a marketplace in the space • The role of data science in successful marketplaces • Ramesh’s philosophy on experimentation and AI • Advice on implementing rating systems • Why learning isn’t free — Brought to you by Sanity—The most customizable content layer to power your growth engine: https://www.sanity.io/lenny | Hex—Helping teams ask and answer data questions by working together: https://www.hex.tech/lenny | Eppo—Run reliable, impactful experiments: https://www.geteppo.com/ Find the full transcript at: https://www.lennyspodcast.com/marketplace-lessons-from-uber-airbnb-bumble-and-more-ramesh-johari-stanford-professor-startup/ Where to find Ramesh Johari: • LinkedIn: https://www.linkedin.com/in/rameshjohari/ • Website: https://web.stanford.edu/~rjohari/ • X: https://twitter.com/rameshjohari Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ In this episode, we cover: (00:00) Ramesh’s background (04:31) A brief overview of what a marketplace is (08:10) The role of data science in marketplaces (11:21) Common flaws of marketplaces (16:43) Why every founder is a marketplace founder (20:26) How Substack increased value to creators by driving demand (20:58) An example of overcommitting at eBay (22:24) An easy litmus test for marketplaces  (25:52) Thoughts on employees vs. contractors (28:02) How to leverage data scientists to improve your marketplace (34:10) Correlation vs. causation (35:27) Decisions that should be made using data (39:29) Ramesh’s philosophy on experimentation (41:06) How to find a balance between running experiments and finding new opportunities (44:11) Badging in marketplaces (46:04) The “superhost” badge at Airbnb (49:59) How marketplaces are like a game of Whac-A-Mole (52:41) How to shift an organization’s focus from impact to learning (55:43) Frequentist vs. Bayesian A/B testing  (57:50) The idea that learning is costly (1:01:55) The basics of rating systems (1:04:41) The problem with averaging (1:07:14) Double-blind reviews at Airbnb (1:08:55) How large language models are affecting data science (1:11:27) Lightning round Referenced: • Riley Newman on LinkedIn: https://www.linkedin.com/in/rileynewman/ • Upwork (formerly Odesk): https://www.upwork.com/ • Ancient Agora: https://en.wikipedia.org/wiki/Ancient_Agora_of_Athens • Trajan’s Market: https://en.wikipedia.org/wiki/Trajan%27s_Market • Kayak: https://www.kayak.com/ • UrbanSitter: https://www.urbansitter.com/ • Thumbtack: https://www.thumbtack.com/ • Substack: https://substack.com/ • Ebay: https://www.ebay.com/ • Coase: “The Nature of the Firm”: https://en.wikipedia.org/wiki/The_Nature_of_the_Firm • Stitch Fix: https://www.stitchfix.com/ • A/B Testing with Fat Tails: https://www.journals.uchicago.edu/doi/abs/10.1086/710607 • The ultimate guide to A/B testing | Ronny Kohavi (Airbnb, Microsoft, Amazon): https://www.lennyspodcast.com/the-ultimate-guide-to-ab-testing-ronny-kohavi-airbnb-microsoft-amazon/ • Servaes Tholen on LinkedIn: https://www.linkedin.com/in/servaestholen/ • Bayesian A/B Testing: A More Calculated Approach to an A/B Test: https://blog.hubspot.com/marketing/bayesian-ab-testing • Designing Informative Rating Systems: Evidence from an Online Labor Market: https://arxiv.org/abs/1810.13028 • Reputation and Feedback Systems in Online Platform Markets: https://faculty.haas.berkeley.edu/stadelis/Annual_Review_Tadelis.pdf • How to Lie with Statistics: https://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/0393310728 • David Freedman’s books on Amazon: https://www.amazon.com/stores/David-Freedman/author/B001IGLSGA • Four Thousand Weeks: Time Management for Mortals: https://www.amazon.com/Four-Thousand-Weeks-Management-Mortals/dp/0374159122 • The Alpinist on Prime Video: https://www.amazon.com/Alpinist-Peter-Mortimer/dp/B09KYDWVVC • Only Murders in the Building on Hulu: https://www.hulu.com/series/only-murders-in-the-building-ef31c7e1-cd0f-4e07-848d-1cbfedb50ddf Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. Lenny may be an investor in the companies discussed.

Ramesh JohariguestLenny Rachitskyhost
Nov 8, 20231h 23mWatch on YouTube ↗

At a glance

WHAT IT’S REALLY ABOUT

Stanford expert reveals data, friction, and flywheels behind marketplaces

  1. Stanford professor Ramesh Johari explains that marketplaces like Uber and Airbnb don’t sell rides or rooms; they sell the removal of transaction friction for both sides of the market. He frames marketplace design as a continuous Whac-A-Mole exercise of reallocating attention and inventory, inevitably creating winners and losers with every product or algorithm change. A major throughline is the centrality of data and experimentation—moving from pure prediction to causal decision-making, embracing learning as a paid, intentional activity, and avoiding over-reliance on short-term, “winner/loser” A/B test thinking. The conversation also covers how to start a marketplace (hint: don’t start as a marketplace founder), how to design fairer rating systems, and how AI expands hypotheses while increasing the need for strong human judgment.

IDEAS WORTH REMEMBERING

5 ideas

Marketplaces sell friction reduction, not the underlying good or service.

Platforms like Uber and Airbnb earn their keep by removing transaction costs—helping riders find drivers and guests find hosts—so both supply and demand sides are the platform’s true customers.

Don’t start by thinking of yourself as a “marketplace founder.”

Until you have scaled liquidity on both sides, you’re not really a marketplace; focus first on solving a concrete, non-marketplace friction (trust, payments, workflows), then consider evolving into a platform once one side is meaningfully scaled.

Data work must prioritize decisions and causality, not just prediction.

Common machine learning tasks (like predicting who gets hired or who will churn) surface correlations, but business value comes from knowing how actions change outcomes—requiring causal inference and well-designed experiments.

Experiments should be hypothesis-driven learning tools, not just “win factories.”

A culture obsessed with A/B “wins” encourages safe, incremental tests run too long; reframing experiments around explicit hypotheses, learning value, and sometimes shorter, higher-variance tests unlocks bigger opportunities.

Learning is costly—and you should treat it as an intentional investment.

Every control-group user or holdout segment represents foregone revenue, but that cost buys clarity on what actually works; recognizing this explicitly helps organizations value experiments and avoid seeing non-winning tests as “waste.”

WORDS WORTH SAVING

5 quotes

Uber and Airbnb are selling you the taking away of something—the friction of finding a driver or a place to stay.

Ramesh Johari

A marketplace business never starts as a marketplace business.

Ramesh Johari

Prediction is inherently about correlation, but making decisions is about causation.

Ramesh Johari

Many of the changes that are most consequential create winners and losers, and rolling with those changes is about recognizing whether the winners you’ve created are more important to your business than the losers you’ve created in the process.

Ramesh Johari (via Sirbas Solan)

What AI has done is massively expanded the frontier of ideas we could think about—and that puts more pressure on the human, not less.

Ramesh Johari

What marketplaces actually sell: reducing transaction costs and frictionHow to think about being (or not being) a “marketplace founder”Data science foundations for marketplaces: matching, ranking, and feedback loopsPrediction vs. causal decision-making and the role of experimentationRisks of over-optimizing via A/B tests and how to design for learningDesigning rating and review systems, rating inflation, and fairnessAI’s impact on data science workflows and why humans matter more, not less

High quality AI-generated summary created from speaker-labeled transcript.

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

Add to Chrome