Alex Wang: Why Data Not Compute is the Bottleneck to Foundation Model Performance | E1164

ℹ️ EPISODE INFO

Released

June 12, 2024

Duration

1h 6m

Channel

The Twenty Minute VC

Watch on YouTube

▶ Open ↗

📝 EPISODE DESCRIPTION

Alexandr Wang is the Founder and CEO @ Scale.ai, the company that allows you to make the best models with the best data. To date, Alex has raised $1.6BN for the company with a last reported valuation of $14BN earlier this year. Scale tripled their ARR in 2023 and is expected to hit $1.4BN in ARR by the end of 2024. Their investors include Accel, Index, Thrive, Founders Fund, Meta and Nvidia to name a few. ----------------------------------------------- Timestamps: (00:00) Intro (01:05) Diminishing Returns in AI Compute (09:08) Solving Reasoning to Overcome Limits (10:56) From Data Scarcity to Abundance (14:37) Challenges in Structuring Massive Enterprise Data (18:59) Fair Access to Proprietary Data for Models (22:02) Model Commoditization (26:51) Value Extraction Challenges in AI Commoditization (32:55) Navigating Data Regulatory Challenges for Innovation (36:53) A Military Asset in Global Conflict: China & Russia (42:49) The Future Landscape of Foundation Models (44:52) About Founder Brand & PR & Media (52:11) Hiring (01:00:41) Quick-Fire Round ----------------------------------------------- In Today’s Show with Alex Wang We Discuss:

1. Foundation Models: Diminishing Returns: What are the three core pillars that can meaningfully improve foundation models performance? Why is data the single largest bottleneck to the performance of models today? What data do we need to capture that we do not currently, that will have the biggest impact on model performance moving forward? Will we see the largest companies in the world revert back to on-prem with the increasing security challenges of migrating all customer data to foundation models?

1. AI: A Military Asset in Global Conflict: China + Russia Why does Alex believe that AI has the potential to be an even more powerful military asset than nuclear weapons? If this is the case, should we have open systems? Do we not have to have closed systems? Why does Alex believe that the CCP’s approach to industrial policy is better than anyone else’s? How does Alex evaluate the rise of Chinese EV car manufacturers in the last few years? Does Alex really believe that China is two years behind the US in the AI race?

1. “I Get Fairer Treatment in Congress than in the Press”: Why does Alex believe that the best PR is no PR? Why does Alex believe that he got fairer treatment in congress than he does in the media? Why does Alex believe that all founders should look to own their own distribution channels today?

1. Alex Wang: AMA: What are some of Alex’s biggest lessons from Patrick Collison on the impact that a hot company brand has on the ability for that company to hire the best? Does Alex think Trump is going to win? What would be the impact if he were to? Why does Alex believe that enterprise software will be changed forever in the next few years? What question is Alex never asked that he thinks he should be asked? ----------------------------------------------- Subscribe on Spotify: https://open.spotify.com/show/3j2KMcZTtgTNBKwtZBMHvl?si=85bc9196860e4466 Subscribe on Apple Podcasts: https://podcasts.apple.com/us/podcast/the-twenty-minute-vc-20vc-venture-capital-startup/id958230465 Follow Harry Stebbings on Twitter: https://twitter.com/HarryStebbings Follow Alexandr Wang on Twitter: https://twitter.com/alexandr_wang Follow 20VC on Instagram: https://www.instagram.com/20vchq Follow 20VC on TikTok: https://www.tiktok.com/@20vc_tok Visit our Website: https://www.20vc.com Subscribe to our Newsletter: https://www.thetwentyminutevc.com/contact ----------------------------------------------- #20vc #harrystebbings #alexandrwang #scaleai #openai #venturecapital #founder #chatgpt #ai #foundationmodels #china #military

🎤 SPEAKERS

• Alexandr (Alex) Wang

  guest

• Harry Stebbings

  host

🧾 EPISODE SUMMARY

In this episode of The Twenty Minute VC, featuring Alexandr (Alex) Wang and Harry Stebbings, Alex Wang: Why Data Not Compute is the Bottleneck to Foundation Model Performance | E1164 explores alex Wang: Data, Not Compute, Now Limits AI’s Next Breakthroughs Scale AI CEO Alex Wang argues that foundation model progress has plateaued not because of insufficient compute, but because the industry has hit a ‘data wall’ after exhausting most high-quality internet text. He introduces the concept of “frontier data” — rich, task- and reasoning-centric datasets drawn from enterprises, experts, agents, and synthetic generation — as the core bottleneck and main moat for future AI systems.

🔗 RELATED EPISODES

Inside Legora: Jude Law Generated $50M Pipeline | Are They Undervalued at $5.5BN? | Patrick Forquer

Cliff Weitzman: What I Learned from 100 of the World’s Top CEOs & Why Tokens Will Outspend Salaries

Shopify CEO on How AI is a Scapegoat for Mass Layoffs & Trump Derangement Syndrome in Canada

Inside Clay's Sales Playbook | Becca Lindquist

Jackie Reses: The Most Powerful Woman in Finance; Making $50BN for Yahoo on Alibaba | E996

Tom Hulme & Stan Boland: Lessons from Jensen Huang & How to Fix the UK Tech Ecosystem