How Open Source Became AI's Backbone | Inferact with a16z

Elena Burger and Matt Bornstein are joined by Simon Mo, co-founder and CEO of Inferact, the open-source inference engine powering many of today's most advanced AI applications. Together, they explore how open-source AI evolved from a research project into critical infrastructure, why inference has become one of the most important layers of the AI stack, and what it takes to bring frontier intelligence to developers around the world. The conversation covers vLLM's origins, the rise of open-weight models, why companies increasingly want control over their AI infrastructure, and how open-source inference enables the next generation of AI applications. They also discuss model licensing, the economics of open-weight AI, Kimi K3, distillation, AI infrastructure, and why Simon believes the gap between open and closed models is rapidly disappearing. Timestamps: 00:00 - Intro 01:00 - What Is vLLM & Why Serving LLMs Is a Fundamentally Different Problem 05:10 - When Open Source Became Critical Infrastructure 08:26 - Where vLLM Sits in the Stack 13:35 - The Open Weights Letter & Why Open AI Development Must Be Protected 16:42 - K3 Economics: Bridging the Gap Between Open & Proprietary 19:57 - Licensing Evolution: From Apache 2 to Commercial Terms 28:51 - Why Open Source Inference Is the Only Way to Scale Agents 32:21 - The Hugging Face Incident & Why Guardrails Break Down 36:57 - Building a Company from an Open Source Project 43:32 - The Distillation Debate: Is It Critical or Incidental? Resources: Follow Simon Mo on X: https://x.com/simon_mo_ Follow Matt Bornstein on X: https://x.com/BornsteinMatt Follow Elena Burger on X: https://x.com/VirtualElena Follow Inferact: https://x.com/inferact Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends! Find a16z on X: https://twitter.com/a16z Find a16z on LinkedIn: https://www.linkedin.com/company/a16z Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711 Follow our host: https://x.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures.

Matt BornsteinhostSimon MoguestElena Burgerhost

Aug 6, 202646mWatch on YouTube ↗

EPISODE INFO

Released: August 6, 2026
Duration: 46m
Channel: a16z
Watch on YouTube: ▶ Open ↗

EPISODE DESCRIPTION

Elena Burger and Matt Bornstein are joined by Simon Mo, co-founder and CEO of Inferact, the open-source inference engine powering many of today's most advanced AI applications. Together, they explore how open-source AI evolved from a research project into critical infrastructure, why inference has become one of the most important layers of the AI stack, and what it takes to bring frontier intelligence to developers around the world. The conversation covers vLLM's origins, the rise of open-weight models, why companies increasingly want control over their AI infrastructure, and how open-source inference enables the next generation of AI applications. They also discuss model licensing, the economics of open-weight AI, Kimi K3, distillation, AI infrastructure, and why Simon believes the gap between open and closed models is rapidly disappearing. Timestamps: 00:00 - Intro 01:00 - What Is vLLM & Why Serving LLMs Is a Fundamentally Different Problem 05:10 - When Open Source Became Critical Infrastructure 08:26 - Where vLLM Sits in the Stack 13:35 - The Open Weights Letter & Why Open AI Development Must Be Protected 16:42 - K3 Economics: Bridging the Gap Between Open & Proprietary 19:57 - Licensing Evolution: From Apache 2 to Commercial Terms 28:51 - Why Open Source Inference Is the Only Way to Scale Agents 32:21 - The Hugging Face Incident & Why Guardrails Break Down 36:57 - Building a Company from an Open Source Project 43:32 - The Distillation Debate: Is It Critical or Incidental? Resources: Follow Simon Mo on X: https://x.com/simon_mo_ Follow Matt Bornstein on X: https://x.com/BornsteinMatt Follow Elena Burger on X: https://x.com/VirtualElena Follow Inferact: https://x.com/inferact Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends! Find a16z on X: https://twitter.com/a16z Find a16z on LinkedIn: https://www.linkedin.com/company/a16z Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711 Follow our host: https://x.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures.

SPEAKERS

Matt Bornstein
host
a16z general partner and podcast host focused on software and infrastructure.
Simon Mo
guest
Co-founder of Inferact and lead maintainer of vLLM (open-source LLM inference engine).
Elena Burger
host
a16z podcast host and investor covering AI and infrastructure.

EPISODE SUMMARY

In this episode of a16z, featuring Matt Bornstein and Simon Mo, How Open Source Became AI's Backbone | Inferact with a16z explores how vLLM and open weights turned inference into AI infrastructure Serving LLMs is fundamentally different from traditional ML because inference must handle highly variable, non-deterministic workloads that demand sophisticated GPU scheduling, batching, and latency control.

RELATED EPISODES