Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI

ℹ️ EPISODE INFO

Released

March 6, 2024

Duration

1h 37m

Channel

Latent Space

Watch on YouTube

▶ Open ↗

📝 EPISODE DESCRIPTION

Soumith Chintala needs no introduction in the ML world — his insights are incredibly accessible across Twitter, LinkedIn, podcasts, and conference talks (in this pod we’ll assume you’ll have caught up on his Gradient pod from last year). He’s well known as the creator of PyTorch, but he's also the Engineering Lead on AI Infra, PyTorch, and Generative AI at Meta. Soumith was one of the earliest supporters of Latent Space (and more recently AI News), and we were overjoyed to catch up with him on his latest SF visit for a braindump of the latest AI topics, reactions to some of our past guests, and why Open Source AI is personally so important to him. Timestamps: 00:00:00 Introductions 00:00:51 Extrinsic vs Intrinsic Success 00:02:58 Importance of Open Source and Its Impact 00:04:25 PyTorch vs TinyGrad 00:10:23 Why PyTorch is the Switzerland of frameworks 00:12:44 Modular's Mojo + PyTorch? 00:16:12 PyTorch vs Apple's MLX 00:19:46 FAIR / PyTorch Alumni 00:22:38 How can AI inference providers differentiate? 00:25:48 How to build good benchmarks and learnings from AnyScale's 00:29:51 Most interesting unexplored ideas 00:33:23 What people get wrong about synthetic data 00:42:00 Meta AI's evolution 00:45:20 How do you allocate 600,000 GPUs? 00:49:24 Even the GPU Rich are GPU Poor 00:55:49 Meta's MTIA silicon 00:58:56 Why we need open source 01:07:00 Open source's coordination problem for feedback gathering 01:21:16 Beyond text generation 01:29:02 Osmo and the Future of Smell Recognition Technology

🎤 SPEAKERS

• Alessio Fanelli

  host

  Partner and CTO in Residence at Decibel Partners; co-host of the Latent Space podcast.

• swyx

  host

  Founder of Osmo AI and co-host of the Latent Space podcast.

• Soumith Chintala

  guest

  Meta AI/FAIR researcher and a key creator/maintainer of PyTorch.

🧾 EPISODE SUMMARY

In this episode of Latent Space, featuring Alessio Fanelli and swyx, Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI explores Soumith Chintala on PyTorch tradeoffs, open models, and feedback loops Soumith Chintala contrasts intrinsic motivation with extrinsic success, saying happiness tracks intrinsic interest after a basic baseline is met.

🔗 RELATED EPISODES

Why Google failed to make GPT-3 -- with David Luan of Adept

A Comprehensive Overview of Large Language Models - Latent Space Paper Club

Making Transformers Sing - with Mikey Shulman of Suno

A Brief History of the Open Source AI Hacker - with Ben Firshman of Replicate

Truly Serverless Infra for AI Engineers - with Erik Bernhardsson of Modal

Building an open AI company - with Ce and Vipul of Together AI