DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters | Lex Fridman Podcast #459

The following is a conversation with Dylan Patel and Nathan Lambert. Dylan runs SemiAnalysis, a well-respected research and analysis company that specializes in semiconductors, GPUs, CPUs, and AI hardware in general. Nathan is a research scientist at the Allen Institute for AI, and is the author of the amazing blog on AI called Interconnects. They are both highly respected, read, and listened to by the experts, researchers, and engineers in the field of AI. And personally, I'm just a fan of the two of them. So, I used the DeepSeek moment that shook the AI world a bit as an opportunity to sit down with them and lay it all out. From DeepSeek, OpenAI, Google, xAI, Meta Anthropic, to NVIDIA and TSMC, and to US, China, Taiwan relations, and everything else that is happening at the cutting edge of AI. This conversation is a deep dive into many critical aspects of the AI industry. While it does get super technical, we tried to make sure that it's still accessible to folks outside of the AI field by defining terms, stating important concepts explicitly, spelling out acronyms, and in general, always moving across the several layers of abstraction and levels of detail. There is a lot of hype in the media about what AI is and isn't. The purpose of this podcast, in part, is to cut through the hype, through the bullshit, and the low resolution analysis, and to discuss in detail how stuff works and what the implications are. Let me also, if I may, comment on the new OpenAI o3 mini reasoning model, the release of which we were anticipating during the conversation, and it did indeed come out right after. Its capabilities and cost are on par with our expectations, as we stated. OpenAI o3 mini is indeed a great model, but it should be stated that, uh, DeepSeek R1 has similar performance on benchmarks, is still cheaper, and it reveals its chain of thought reasoning, which o3 mini does not. It only shows a summary of the reasoning. Plus, R1 is open weight and, uh, o3 mini is not. By the way, I got a chance to play with, uh, o3 mini, and uh, anecdotal vibe check-wise, I felt that o3 mini, specifically o3 mini high, is, uh, better than R1. Still, for me personally, I find that Claude Sonnet 3.5 is the best model for programming, except for tricky cases where I will use o1 pro to brainstorm. Either way, many more better AI models will come, including reasoning models, both from American and Chinese companies. They will continue to shift the cost curve. But the "DeepSeek moment" is indeed real. I think it will still be remembered five years from now as a pivotal event in tech history, due in part to the geopolitical implications, but for other reasons too, as we discuss in detail from many perspectives in this conversation. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description, and now, dear friends, here's Dylan Patel and Nathan Lambert. A lot of people are curious to understand China's DeepSeek AI models, so let's lay it out. Nathan, can you describe what DeepSeek V3 and DeepSeek R1 are, how they work, how they're trained? Let's, uh, look at the big picture and then we'll zoom in on the details.