Eiso Kant, CTO @Poolside: Raising $600M To Compete in the Race for AGI | E1211

1. 0:00 – 0:53

   Intro

   1. EKEiso Kant0:00

      (instrumental music plays) Who has earned the right to be in the race to AGI? And we're gonna look back on this moment 10 years from now, just like we would look back to the moment of mobile, internet, and realize that that was the moment where the table got set. You do not want to look back on that moment and not have given it everything you've got, because it's a race. And the latest $500 million round translates to us being able to be an entrant into the race. We don't get the luxury of stumbling on the capabilities race, or the go-to-market race.

   2. HSHarry Stebbings0:27

      Ready to go? (instrumental music plays) I so... Dude, I am so excited for this. This is also the first time that we've actually met in person. You are far more incredibly good-looking in person, so thank you so much for joining me today.

   3. EKEiso Kant0:48

      Well, thank you, Harry. It's a pleasure to be here. And it's- glad that we finally met in person. It's been a minute since we've known each other.

2. 0:53 – 4:42

   What is Poolside?

   1. EKEiso Kant0:53

   2. HSHarry Stebbings0:53

      Now, I want to just dive straight in. I think there's a lot of people looking at Poolside in the news and seeing the new round, going, "What is Poolside?" Can you just provide some context? What is Poolside? What do you do? And let's start there.

   3. EKEiso Kant1:08

      So Poolside's in the race towards AGI. We think the future is gonna play out that the gap between machine intelligence and human-level capabilities is gonna continue to desc- decrease. And... But our- the path towards that, in our opinion, is by focusing on building the most capable AI for software development. And all of this comes back to a s- a set of foundational beliefs that we have, that I would say are different than some of the other companies in this space, in terms of where both research is heading and where capabilities are heading. And so the term AGI is a loaded term, and the way that I like to kind of take the definition that is most commonly used is that at some point we are going to be in a world where, across all sets of capabilities that we have as human beings, machine intelligence is going to be as capable, and if not more capable than us, and surpass us. Now, our point of view is, is that that world is still quite a bit out, and that we are actually going to end up in a place before that, where we see human-level capabilities in areas that are massively economically valuable, and can drive abundance in the world for all of us, that are not gonna be equally distributed, not for every single thing. And what I mean by that is that if you think about foundation models today, and I have a kind of simple mental model about them, which is that we are taking large web-scale data, and we're compressing it into a neural net, and we're forcing generalization and learning. And this has led to things like incredible language understanding in these models. But it's also led to things where we look at it and we say, "These models are kind of dumb. Why aren't they able to do X, Y, or Z?" And our point of view is that the, the reason why they're not able to do X, Y, or Z has to do with how they learn, and the most important part, I think, of what I said is the scale of data. When we have web-scale data, we can get language understanding. But when we have areas where we have very little data, models really struggle to learn truly more capable areas. And I mean improvements in reasoning, improvements in planning capabilities, improvement in deep understanding of things. And so, while as humans we don't require so much data, the way to think about models is that they require magnitudes order more data to learn the same thing. Our focus is on software development and coding, and it's for a very specific reason. The world has already generated an incredibly large dataset of code. Uh, to put a little bit into context, like usable code for training, so what we refer to as about three trillion tokens, and if you look at kind of usable language in English on the internet for training, we're talking about anywhere between 10 and 15 trillion tokens. There's a massive amount of code that the world has developed. Over 400 million code bases are publicly on the internet. So why don't we have this incredible AI that's able to already, you know, do everything in coding? It's because coding is not just about the output of the work. Right? The code that we have online represents the final product, but it doesn't represent all of the thinking and actions that we took to get there. And that's the missing dataset. The missing dataset in the world to go from where models are today to being as capable as humans at building software is the dataset that represents being given the task, all of your intermediate reasoning and thinking, the steps that you do, the code that you write and try to run, and then it fails, and you learn from that- those interactions, and all the way to kind of getting that final product. And that intermediate dataset, that's what Poolside exists on

3. 4:42 – 8:58

   Capturing Iterative Thinking in Uncharted Data

   1. EKEiso Kant4:42

      creating.

   2. HSHarry Stebbings4:42

      So I immediately think, and you may kind of, um, chafe at this, but I immediately think of The Social Network, where they are drawing the algebraic equations on the windows-

   3. EKEiso Kant4:52

      (laughs) .

   4. HSHarry Stebbings4:52

      ... uh, and you see that in the early scenes. How do you capture that process iteration thinking in what is previously non-existent or non-captured data?

   5. EKEiso Kant5:04

      This is th- the right question. Now, the way that I think about it is that all of this got unlocked by DeepMind here in London, actually, in 2016. When DeepMind created AlphaGo, right, the- this incredible achievement of beating, you know, the world's best, you know, human player in Go, they did it in the following way. And, and I'm oversimplifying it for kind of brevity, but they first trained a neural net on all of the Go games that they could find online that had been played. And the outcoming Go player that the model, that the model had become was kind of average. So what they did next is they said, "Hey, Go is deterministic. There's a win and a loss scenario. Uh, and while there's close to infinite possible moves at every turn, or extremely large number," which was the par- part why we couldn't computationally brute force it, "we do have these outcomes, win or lose. It's deterministic." So they gave a model the Go game engine, and through the use of reinforcement learning, let it explore moves and play against itself.... and the model was learning from when it was winning and when it was losing, about which moves to pick at which turn of the game. And in the end, it created AlphaGo. Now, the way to think about it... And by the way, future versions were entirely in simulated environments. They didn't require any more the human played games to bootstrap it. And, and the reason... what kind of DeepMind did is, they saw a place where there wasn't extremely large-scale data, right? There wasn't so many Go games at all levels of, you know, complexity, uh, that a model could just learn to become as good. So they said, "Hey, we're in a simulatable domain, so we can simulate the data." And this is often what we refer to as synthetic data.

   6. HSHarry Stebbings6:38

      Mm.

   7. EKEiso Kant6:38

      Now, this is through the use of reinforcement learning, but at the end of the day, it's all, it's all data. Now, I'm giving this as a preamble to your question, because the way I think about the world is that there are problems that we cannot simulate. The real world is impossible to perfectly simulate. It's messy, it's multivariable. How do we deal with the real world when we're trying to m- close the gap between human capabilities and AI? We have to gather data. The best example of this is Elon and Tesla. Elon has put millions of cars on the road that are actually capturing every single engagement and disengagement with autopilot, and every single scenario, and then sending that back to Tesla to train increasingly more capable AI. And if you look at how full self-driving got more capable over the years, it's directly relational to it becoming more and more learning from the data, instead of rule-based, and more and more cars on the road. And so to me, Elon has won full self-driving. I think it's, it's inevitable that, that, uh, the most capable AI for full self-driving is coming out of Tesla, because they've been gathering and building up this dataset. And he needs to gather data, because it's non-simulatable. Now, this is the head fake behind Poolside. If you think about Go being deterministic, and you think about the other end, the real world being non-deterministic, where does code sit? Code sits a lot closer to being deterministic. It follows a set of rules. Every time it runs, it runs in exactly the same way. And so this is what we call execution feedback. What we're really known for is our work in reinforcement learning from code execution feedback. It's the way where we then take a model that we've trained from the ground up, we put it in an environment, say it's an environment with 130,000 real-world code bases, several orders of magnitude, the largest environment in the world, and we send the model off to explore different solutions to sets of tasks, and learn from when it actually passes the tests versus when it doesn't. And there's a lot more details behind this, but the way to think about it is if you can simulate it, you can actually build an extremely large dataset. And part of the things that we synthetically generate is not just the output code, but it's the intermediate thinking and reasoning to get to that output code. Because models today, and you can try this yourself by going online and chatting to any model, can actually produce their thinking. They're not very good at it yet. So what do you do when your thinking is not very good? You need feedback. In our case, deterministic feedback, code execution

4. 8:58 – 12:49

   The Biggest Bottleneck in AI Progress: Compute, Data, or Models?

   1. EKEiso Kant8:58

      feedback.

   2. HSHarry Stebbings8:58

      A lot of people break it down as compute, data, and then kind of models themselves.

   3. EKEiso Kant9:05

      Oh, well-

   4. HSHarry Stebbings9:05

      I mean, yeah, compute, data, and then algorithms, really. So if we take those three, how do you think about what the b- biggest bottleneck is today in the progression of models? Is it the data that we mentioned, or is it one of the other two?

   5. EKEiso Kant9:16

      We are making, in our space, especially I think post the ChatGPT moment, like incredible advancements in the algorithms that are making learning more efficient. Internally, I, I have this th- thing that I say to the team, and they're probably tired of me hearing, 'cause I say it every single day, I say, "All the work we do on foundation models, on one hand, is improving their compute efficiency for training or running them, or on the other hand, improving data." Now, the way to think about the algorithms and the improvement of compute efficiencies, that's table stakes. All of us, OpenAI, Anthropic, Google, et cetera, are doing this, and we're just constantly improving here. And it's engineering and research combined. But the real differentiation between two models is the data. But compute matters tremendously for data, because if you think about Poolside, and we spoke about, how do we get this data? And I mentioned the word synthetic, it means that we're generating it, which means that we're using models to generate data, to then actually use models to evaluate it, to then run it. And so, compute hugely matters on the side of the generation of data. But once we have all of this data, where we started today, we spoke about, you know, neural nets essentially being compression of data that forces and generalizes learning. Now, when we have small models, we are taking huge amounts of data and we're forcing this generalization of learning to happen in a very small space. And this is why we essentially see these difference in capabilities. Larger models require l- essentially, it's easier for them to generalize, because we're not forcing so much data into such a small cr- compression space. And so my personal mental model of this is that the scale of your models, this has been shown over and over again... By the way, we owe a debt of gratitude to Google, to OpenAI, for, you know, proving out the scaling laws, which essentially say, as we provide more data and more parameters, more skill, hence more compute for these models, we get more and more capable models. Now, there is a limit to that, most likely. If you think about it as a c- as the analogy to compression, right? Like your image that you had, you know, at high resolution compressed down to small resolution, the small models are the small resolution. We have generalization, but you're losing things. But in the infinite extreme, an infinitely large model wouldn't be doing any compression. So there is definitely a limit at some point to model size. But what underpins all of this, to directly answer your question, is the compute. And the compute really, really matters, and your own proprietary advantages in your applied research to get great data, or to gather it, matter equally as much. But if you don't have the compute, you're not in the race.

   6. HSHarry Stebbings11:52

      I wanna kind of unpack that kind of one by one. If we start, I mentioned the algos, I mentioned the data, I mentioned the compute. You said about kind of algos and how it improves model efficiency. Is there a limit to how efficient models can and will get, and does that kind of plateau at some point?

   7. EKEiso Kant12:08

      We are...... horribly inefficient at learning today. If you think about what drives efficiency of learning, it's the algorithms and it's the, and it's the hardware itself. And we've got probably decades, if not, you know, hundreds of years of, of improvements still left there, and different forms of it over time. If we look very practically, in the coming years, we are gonna see increasing advantages on the hardware, I'm gonna see increasing advantages on the algorithms. But I hope everyone takes away that this is table stakes. This is something that you have to do to be in this space, and you have to be excellent at it. It's not what differentiates you, it's what allows you to keep up with everyone

5. 12:49 – 15:45

   The Value of Synthetic Data

   1. EKEiso Kant12:49

      else.

   2. HSHarry Stebbings12:49

      On the synthetic data side, a lot of people use this as a catch-all for, like, "Oh, we've got a data shortage problem, but don't worry, synthetic data's here to save us." To what extent is all synthetic data equally valuable, or is it more valuable in certain industries versus others?

   3. EKEiso Kant13:06

      I think the biggest cognitive dissonance that people have around synthetic data is a model is generating data to then actually become smarter itself. Right? It feels like the snake eating itself. There's something that doesn't make sense in it. Now, the way that you need to look at that is that there's actually another step in that loop. There is something that determines if from all th- the data that the model generated... So think of this as a, in my domain, a software development. I have a task in a codebase, and the model generates 100 different solutions. Now, if I were to just feed those 100 different solutions back to the model in its training, the model won't get smarter. That's the f- the snake eating itself. But if you have something that can determine, an oracle of truth, that can help say, "This is better and this is worse," or, "This is correct and this is wrong," that's when you can actually use synthetic data. Now, in our case, that's executing the code. Did it actually run, did it pass the tests, that gives me validation that the reasoning and the output are correct.

   4. HSHarry Stebbings14:02

      So it's simply a case of objectivity, true or false, where there is a clear right and wrong.

   5. EKEiso Kant14:07

      Exactly. And, and look, and it doesn't have to even be clear right or wrong, it's about, is it better or worse? And this is where human feedback has been so valuable in AI. Right? There's this, there's huge amounts of human feedback that companies have been gathering, uh, to be- because when you don't have this perfect simulatable d- domain that we have as, as Poolside, you need to go to, who are the experts? Well, the experts are the humans. It's us. Right? We're building intelligence for us to represent the world that we have, to do the things that we're already doing. And so we need to gather that. So humans, they are training AI, and they're doing it very often on synthetic data, saying, "This was a better answer, this was a worse answer." Or annotating it. "Hey, you were almost right, but this is where you made a mistake." And the humans are actually, you know, giving the correct part of it. And so we're, we're combining deterministic feedback, we're combining human feedback, and we're doing that on outputs, but also on reasoning and thinking of these models that lead to those outputs. So all of us collectively, both with compute and with humans, are training AI to get better. I'm very glad that I'm very happy with where we sit with Poolside, because our view of Poolside is that the first majorly economically viable capability that is gonna close the gap between human intelligence and machine intelligence is software development. Why? Because I can simulate so much of this. I can scale it up with compute. While human feedback is a lot peskier, is a lot harder to scale up. And also is often not as deterministic as software development. Software development, we know if something does what we intend it to do. But it's quite hard to ask that same question, you know, to five doctors about, you know, a certain opinion they might have, or, you know, what answer

6. 15:45 – 18:21

   Scaling Laws in AI

   1. EKEiso Kant15:45

      do you prefer or not.

   2. HSHarry Stebbings15:46

      I want to get to the closing gap, but I do want to just discuss scaling laws before that. You mentioned it earlier. There's different opinions around this. A lot of people now have come to the conclusion that we haven't even touched the surface, and s- scaling laws have so much more room to play out. And others have a lot more n- negative views, bluntly. How do you feel about our- where we are in terms of scaling laws, and how much room we have to run?

   3. EKEiso Kant16:09

      So I think we are starting to understand that the scaling- the first version of the scaling laws that came out spoke about the amount of data we provided during training, th- and the size of the model. Right? And, and more data, longer training, and size of the model larger requires more compute. And so we often say the scaling laws are about applying more compute. And it's actually more correct than we initially realized, because the importance of synthetic data for models to get better is another form of using compute. But we're using it at inference time. We're running these models to generate these 100 solutions, generate 1,000, or 100, or 50. I think we have a lot of room still for scaling up models. We can do this by scaling up data, and we can do this by scaling up the size of the model. And I do think we're, we're, in this case, not so different than I think most of the major, you know, the major AI companies in this space, is that there's a lot of room to scale the number of parameters and size of models still.

   4. HSHarry Stebbings17:04

      Mm-hmm.

   5. EKEiso Kant17:04

      But there's something that we don't really talk about in our industry as much. We're training extremely large models. And by the way, we, until very recently, weren't even capable of doing so, 'cause we didn't have the compute and the capital. This is why our fundraise, you know, has been so important to us, so that we can have the capital to scale up. But what everyone does is, and, is that extremely large models we can't run cost-efficiently for our end users. If you have a multi-trillion-parameter model that is what we often, you know, often architected as an MoE, meaning that not all of those parameters activate during inference time, but are still very large, it's too expensive. Every request that you make to that model is not, you know, a couple of cents. And so you have to find a way to actually build models that you can actually run for customers. And so what happens in our, in our, in our industry, and, and this is our path as well, is you train a very large model, where you can clearly see that there's more capabilities in the model. And then what we call, we distill it down to a smaller model. Learning from data, models are really inefficient. But learning from data in combination with learning from a smarter, larger model, is actually quite efficient.And so, it's, we make really big things that become really smart. We then teach the smaller models to try to match as much of that intelligence as possible, which we can then economically viable, put in the market, and make

7. 18:21 – 22:10

   Projecting Model Costs Over the Next 12-24 Months

   1. EKEiso Kant18:21

      revenue from.

   2. HSHarry Stebbings18:22

      How do we expect the cost of models to change in the next 12 to 24 months?

   3. EKEiso Kant18:28

      So I think the cost of models... we should separate the price and the cost of models.

   4. HSHarry Stebbings18:32

      Okay.

   5. EKEiso Kant18:33

      If you look at what's happening in the world of general purpose LLMs, LLMs for everything, it's an incredibly competitive price war that's happening. And it's happening between the large hyperscalers, and it's happening between what I kind of refer to as the escape velocity AI companies, Anthropic and OpenAI. And- and then you throw in the mix, you know, the, the vendors that are, are putting up the open source models from Meta and such. I often think about, you know, what sits in that stack of costs. Well, what sits in the stack at a cost is a server, a box, the networking around it, a data center, uh, the chips, right, the GPUs, or, uh, and then the energy that goes into that. And everything after that is fundamentally, you know, marginal cost or variable cost that's of- of the running of the, of the models. So we have to think about who has the lowest cost profile in this space, right? Who has the- the cheapest first principles CapEx that they're doing to run these models? Well, that's the people who have as much of that vertically integrated, and who have as much of that infrastructure already online and brought into the world. And this really is the hyperscalers. This is Amazon in number one, this is Google in number t- uh, sorry, Microsoft in number two, Google in number three. But there's something interesting about all of those, is that each of those at different moments in time understood that they couldn't be reliant on hardware built by NVIDIA or AMD, by someone else. They had to also build their own. Furthest along that has been Google with their TPUs. I think it's currently at their fifth generation. They started early on this and they've been improving it ever since. Then you have Amazon who's been working on their Trainium and, and Inferentia chips, their Neuron cores for some time. And Amazon has an incredible background, by the way, in manufacturing chips. And this is something I don't think people take enough credit to, uh, because while Google has to work with Broadcom to be able to bring TPUs into the world, Amazon is working with the fabs directly, and they have an incredible skill, right? They've done an amazing job on- on- in- in- in the cloud. And Microsoft's still earlier in their own chip journey. Now, I know this is again a preamble to your question, and I'm sorry for- for bringing that this there all the time, but I think it's an important thing to understand, because when you are buying NVIDIA hardware, and you're putting it in a data center, and you're working with an Oracle, or you're working with whoever it is in the space, or even a Google or- or an Amazon or a Microsoft, you are taking that margin of that chip, right? The H100, the H200, the- the Blackwell generation coming up. And that has to be baked into your costs. The way that I think about this is at the extreme end of it, you know, Amazon, and Google, and- and- and Microsoft as well, as they come out with their own silicon, have a lot more margin to play with. And now then it comes down to business decisions. And right now, since this is a war, and it's a total, you know, like it's- it's an incredible race that's happening. I often refer to it as a- as a drunken bar fight that's happening in our industry, is that we are, you know, all these companies are massively incentivized to drop the cost of their models as quickly as possible. And they do that in two ways. They do that by cutting more and more of their margin down to their actual cost, right, their hardware. And then you can see a big difference between, you know, what an Amazon is able to do, and a Google, and a Microsoft, and an OpenAI, and an Anthropic. But they also do it on the intelligence layer. We spoke earlier about large capable models that distill down into smaller models. If you have the most intelligent, largest model, you can distill it down into a smaller model, and you can have advantages at that layer as well. So it's- it's not a single variable answer. My view is, though, that in- in the extreme of it, the compute margin, like the hardware margin really matters, uh, as this gets lower and lower in price. Uh, and this is what we've seen in cloud computing as well.

8. 22:10 – 29:36

   Future of Model Distillation

   1. EKEiso Kant22:10

   2. HSHarry Stebbings22:10

      Do you think in five years' time we will need to go through the process of distilling a larger model down to a smaller model, trying to get the best of it for the benefits of reducing cost for end consumer? Or actually, we'll have such efficient costing that actually it'll just be one model that we can apply?

   3. EKEiso Kant22:26

      I- I think we need to go back to... this is my personal view of how the world plays out. It's really easy to- to stay focused on the tactics and- and- and things that matter in this moment. And they're exactly the right questions of what- of what matter right now in- in the moment. But we are moving towards a place where we are closing this gap between human intelligence and machine intelligence. And I think it's gonna be an incredible amount of problems and challenges and places where we want to apply this intelligence. If I have a- a view on- on modern history... again, I'm taking you a bit off the side path here.

   4. HSHarry Stebbings23:05

      Sure.

   5. EKEiso Kant23:05

      If you look at what happened from the printing press, uh, onwards, is that what we've done is we've connected more and more people around intelligence. We went from, you know, the telephone to, you know, personal computer to the internet to the mobile phone. Fundamentally, what we've been able to do is we've been able to take hard challenges in the world, if that's cancer research or if that's even, you know, building a business, SaaS companies, anything. And we've been able to connect more and more people together to direct resources to those things. What we're fundamentally doing is we're bundling intelligence, right? More and more people got connected together, and I think that's the true underlying thing that has- has- has underpinned this technological, like, exponential curve run, right? I- I... If you think back about 100 years ago or 50 years ago, you can truly see it's an exponential. I don't think we want to live in any other moment in time. And the reason I mention this to your question is that I think we are now gonna go from a world where human intelligence and the amount of humans we had who, uh, was the entire bottleneck, to now we are having machine intelligence, and so we can pair investments in energy and chips and compute.... together with humans, and have an extreme explosion on this exponential of all of the places in the world where we wanna direct it to. I think there is a huge amount of places where this is gonna be valuable. We will figure out the compute efficiency along the way. The hardware will get more efficient, because that's capitalism. As the opportunity is big, we'll direct things to make it more efficient.

   6. HSHarry Stebbings24:34

      Speaking of, like, where it's valuable, you said... A- a- again, I'm jumping around the different notes, but you said about closing the gap, and specifically with regards to code, and we chatted earlier about that as- with regards to other industries. I think we chatted about voice recognition as an alternative. How do you think about this element of closing the gap and how that correlates to where value is and maybe where it isn't?

   7. EKEiso Kant24:54

      The way I think about this is, there's things in the world that today we consider economically valuable. So if we take what's economically valuable, the next thing we need to ask ourselves, "What's the gap between models today and human-level capabilities? And how large is that gap?" And in some cases, the gap is actually not that large anymore. We were talking earlier about speech recognition. Models today, in my opinion, are pretty much there. Maybe there's a tiny bit left to say, but we've closed that gap into an incredible amount. Uh, in other areas, the gap felt like it was gonna be impossible to close, but we're making a lot of progress. Come back to full self-driving, if you've been in your latest, you know, Tesla FSD update, that gap feels getting closer and closer to be closed. Now, there's other areas where the gap is really large still. I think software development, our domain, we think the gap is still very large, right? What models are able to do is they're massively useful assistants, and they drive massive economic value because of that. But between a model working with a developer today, there's a huge, huge gap, and we wanna get to a world where developers can work with models that are as capable as them, right? And potentially even one day more capable. Now, the reason I mention this is, so we've got the human capability aspect, right? What's the gap that's there? How economically valuable is the domain? Now, then there's a next area I think that you have to ask yourself is, "How easy is it gonna be to close that gap?" And that comes down to data, right? It- it- it's where can we get extremely large-scale, web-scaled data to be able to close that gap in areas where the intelligence gap is really big? Because the bigger the gap in intelligence today, and- and I know it's a broad word, the more data that we need to close it. And so if you kind of used this as a, "Where can we find the data, size related to how large the gap is between human and machine intelligence, and how economically val- val- valuable it is in the real world already," I think the intersection between those is the places where companies like us get to exist.

   8. HSHarry Stebbings26:49

      My immediate thought jumps to GitHub. Are GitHub not best place to do that?

   9. EKEiso Kant26:53

      GitHub today has this incredible dataset, which is all of the code, almost all of the code in the world. GitLab is a player, but only in the private side, right? What sits behind the- the accounts of developers. GitHub is massive in public code and it's massive in- in private code. But private code, no one's allowed to train on. Not us, not OpenAI. So all of us have access to the same public data, and it's the output data. And so there isn't an inherent advantage from a capabilities race perspective. And another thing that we frame in our company over and over again is there's a capabilities race in the world. And to your point earlier, we said there's four things that matter. Agree with you on the three, but I'm gonna add one, right? It's compute, it's data, it's proprietary applied research, it's the algorithms, and talent. Talent is absolutely key in this industry. Now, in the go-to-market race, it's talent first and foremost, but it's also product and distribution. And distribution, Microsoft definitely has an incredible positioning in the world.

   10. HSHarry Stebbings27:51

       Can I ask, in terms of the compute element that we didn't discuss, when we think about compute underpins all of this and a lot of the data challenges that we mentioned, I'm British, and so we're not very good at asking the direct-

   11. EKEiso Kant28:04

       (laughs)

   12. HSHarry Stebbings28:05

       ... is $600 million enough?

   13. EKEiso Kant28:07

       No. $600 million that- that we've raised till date, and- and the latest $500 million round, translates to us being able to be an entrant into the race. And what that means is that the 10,000 GPUs that we've now brought online this summer, you know, that- that came from this capital, allow us to make incredible advancements in model capabilities, uh, because of the, our ability to take reinforcement learning from code execution feedback and generate extremely large amounts of data, and then train very large models with it. And it is the- it is enough for this moment in time, but over time, it won't be enough.

   14. HSHarry Stebbings28:47

       How much do you think you'll need?

   15. EKEiso Kant28:49

       It's a very good question. I think we are... there are real physical, real-world constraints behind this. And so we've seen crazy numbers thrown out in our industry of, you know, compute cluster sizes and things like that. But the world actually still needs time to catch up with the real ability to do so. Today, interconnecting more than 32,000 GPUs is extremely challenging. We're starting to be able to possibly interconnect 100,000. But right now, a million GPU cluster, a 10 million GPU cluster for training of models has both true algorithmic things that we have to overcome to be able to do this, and also has actual, like, physical limitations still in the world. So we're not living in a world right now where, like, unlimited money can buy you unlimited advantages. It's why we get to exist, right? With 10,000

9. 29:36 – 31:35

   Does Cash Directly Correlate to Compute Access?

   1. EKEiso Kant29:36

      GPUs.

   2. HSHarry Stebbings29:37

      Does cash correlate to compute? And what I mean by that is, if you have cash, can you go to your store and say, "I want this amount of compute, please?" Or is it more than that?

   3. EKEiso Kant29:50

      It's the right question. Uh, I think, again, it depends on how much cash and how much compute. About a year and a half ago when we started as a company, there was a true imbalance between supply and demand in the world, that even as a- as, you know, frontier AI company starting this, everyone wants you to win.NVIDIA's incentivized the hyperscalers. Everyone is incentivized, actually, to make early-stage companies succeed with compute. It's a lot easier when you're an early-stage AI company, or frontier AI company in general, to get compute than it is when you're an enterprise, because they understand this is where the future is heading towards. But even then, there was a real mismatch between demand and supply, and we had to do an incredible amount of work of, of understanding the market, building relationships, and, and having plan A to Z to get there. In the last six months, I think the world has still a huge supply shortage, and we can, and we can see this. But when you are where if you're an early-stage startup, there's lots of paths for you. If you're a frontier AI company, you need to make decisions about: Who do you partner with? Who do you work with? How much do you do yourself? You need to... I'm making decisions today that will impact us on computing 12 or 18 months from now. It's very rare to be at early-stage companies where you have to make decisions right now that impact you, you know, on physical infrastructure a year and a year and a half later.

   4. HSHarry Stebbings31:05

      Have we seen that demand-supply imbalance change?

   5. EKEiso Kant31:08

      I don't think I sit in the perfect right place to have the, the full answer to that. I think the people who have the best answer to that are the people who are the hyperscalers, right? Uh, uh, and this is, I think, on the earnings calls of NVIDIA and, and, and Amazon and, and, and Google and Microsoft, I think you can get most of your, your view there. But from what I'm seeing right now, the world has still far more demand for GPU and, and, and, and GPU-like, uh, compute than, uh, that

10. 31:35 – 36:50

    Eiso’s Perspective on Larry Ellison’s $100B Foundation Model Entry Point

    1. EKEiso Kant31:35

       supply that's available.

    2. HSHarry Stebbings31:36

       Because of some of the egregious spending, uh, Larry Ellison said on a stage recently, "It will require $100 billion to enter the race, and that is the entry price." It was a really striking moment for me, where I was like, "Oh, my God. I, I don't know what the future holds." Do you agree with that as an entry price?

    3. EKEiso Kant31:57

       If you want to become, uh, a hyperscaler that is able to put data centers all over the world, with GPUs in it that is- that are, are going to allow you to serve these models to everyone, an infrastructure player, that's probably it. And that's probably just a starting point, right? If we look at the, the massive CapEx investments that, uh, all of the, the cloud companies are doing, uh, you know, they're, they're far above $100 billion when you look at them over the course of, you know, a couple of years. Now, in the race towards more and more capable AI, closing that gap between human intelligence and, and machine intelligence, uh, I think we are all pushing the frontier more and more possible, and we're seeing how that gap closes as we're scaling up our models and scaling up our data. But I don't think anyone has a definite answer of, how many dollars is it gonna take from here to there. If we knew that, we knew the outcomes. We're all on the frontier of what's possible right now.

    4. HSHarry Stebbings32:53

       You, you very s- wisely called it a drunken bar fight. One of my friends who runs one of the hyperscalers the other day said it's like the Manhattan Project, where kind of everyone's kinda actually trying to get out. (laughs)

    5. EKEiso Kant33:05

       (laughs)

    6. HSHarry Stebbings33:05

       But no one actually can, and it's far too late, and it's like chips are on the table, and we've gotta keep going.

    7. EKEiso Kant33:09

       (laughs)

    8. HSHarry Stebbings33:10

       How far in are we, do you think? Is this just the, the tip of the iceberg, and there is a huge amount left to be spent by the incumbents?

    9. EKEiso Kant33:19

       I think we need to separate spend from getting the world's most possible capable AI, by closing this gap, getting to AGI, closing the gap between human intelligence and-

    10. HSHarry Stebbings33:30

        Are, are they not the same thing?

    11. EKEiso Kant33:32

        ... the trained... They're not the same thing, because if you look at these models as investments that we're making to get intelligence out on the other end, that needs to be economically valuable to end users, right? With lots of layers and applications and things in between. The model, the creation of models is CapEx. Uh, the operating of them, the inference, the running of them, is OpEx. But the OpEx to run them requires extremely large-scale physical footprint in the world. Just very simply, if we would spend, you know, $100 making a model, and it will ever return $2 or $3, you know, in terms of value to the world, it makes no sense, right? The world will punish it. It won't exist. The huge scale-out that, that has to happen in the world for AI to become something that can tackle all of our world's problems, and can, and can feed in all of... from our, from our software to our, you know, to our daily lives, requires huge footprint to run these models. It requires massive inference, and that means it requires data centers all over the world, close to end users, right? Latency matters. And that requires a massive build-out. And I think this is the- one of the largest build-outs that we've seen in physical infrastructure since, you know, the, the last couple of decades in, in the cloud.

    12. HSHarry Stebbings34:43

        In terms of that kind of build-out of physical infrastructure, it was David Khan. I'm trying to get exactly what he said, just 'cause I don't wanna butcher it given it's someone else's quote. But he said that essentially you will never train a frontier model on the same data center twice, meaning that, you know, the evolution of models is now outpacing the development of data centers. Do you agree with him when you hear that?

    13. EKEiso Kant35:06

        We're in a world today where the amount of data centers that can hold and power, and have enough energy, uh, to power increasingly magnitude order larges of clusters, uh, is a very small number. Uh, and, and so I, I think he's absolutely right in this sense, that the data centers from, you know, two years ago versus the data centers in terms of size and power requirement that we're gonna see in the next two years look radically different, not just because the scale of number of servers and nodes that we're interconnecting. This is the difference between inference, right? For inference, we don't need all of the machines to be connected to each other or in the same place. For training, we need them all to be connected to each other in the same room, in the same place. And so that m- massively changes what a data center looks like.

    14. HSHarry Stebbings35:50

        I'm sorry. I, I think this show has done so well 'cause I ask questions that most people think.

    15. EKEiso Kant35:54

        (laughs) Please do.

    16. HSHarry Stebbings35:54

        Why do you need that for training and not for inference?

    17. EKEiso Kant35:57

        When we're scaling up the size of these models, and we're training them on more and more data, and we're using more and more compute for it, at every single step that we're taking...... in the learning, every, you know, set of samples of data that we show the model. We need them to communicate with each other and share what they've learned across the optimization landscape. And so this just means that if I would, you know, have two data centers that sit far away from each other, the- the- the amount of information that they have to share with each other, all of the different servers, and we're talking here thousands and tens of thousands soon of servers, you know, would make it so slow that it wouldn't be economically viable to train these models. Once I'm running a model, I'm using a lot less servers to run it. So think of it as having lots and lots of copies of the model during training over lots and lots of machines, that at every time they see data to learn, they need to communicate with each other to continue to improve in their

11. 36:50 – 38:51

    Eiso’s Outlook on Nvidia's Dominance and the Future of Compute

    1. EKEiso Kant36:50

       learning.

    2. HSHarry Stebbings36:50

       Can I ask you, when we look at kind of, uh, the build-out and the chips required and the compute required, to what extent is it a continuing NVIDIA monopoly, and to what extent is it a more even playing field?

    3. EKEiso Kant37:03

       The dynamic that exists in the world today is that NVIDIA has built this... I mean, I think we're- we all owe a debt of gratitude to NVIDIA. When I started in this space in 2016, and I was building Source, the first AI code company in the world as far, to the best of my knowledge, you know, we were stacking 1080 Ti chips in racks of servers at our office. And NVIDIA already back then understood that AI was going to be world-changing. And no other company, the exception of maybe Google, had that deep of a realization. But NVIDIA had massive conviction on this, and continued to double down here, and has made more and more incredible, you know, hardware. The company that fast followed on that was Google. That's why we're on the fifth generation of TPUs. And the company that fast followed on that was Amazon. The reason I- I mention those three specifically is that they are all building extremely large volume of chips, and constantly iterating on faster and better and better generations of chips, for training and for inference. They're for me the three primary players in the race, just from sheer volume of what they're producing in the fabs, versus what they're bringing online to end users. Now we have some other companies in this space, AMD, right? Competitor to NVIDIA. Doesn't have their own cloud, right? Is- is- is reliant on being in this competitive nature from a price perspective with NVIDIA. And so their ramp up is entirely determined by the demand of the world wanting to use their chips. The demand of the world for AI, for a Google and an Amazon with their own silicon, is not the demand for chips. It's the demand for AI. The way I look at this is that we're gonna be in a world where those three and possibly new entrants or possibly, you know, AMD may be catching up, but I think really those three and Microsoft with their own silicon one day, are going to be the driving force in this industry.

12. 38:51 – 46:06

    Has Innovation Stalled Awaiting Nvidia's Blackwell?

    1. EKEiso Kant38:51

    2. HSHarry Stebbings38:51

       To what extent has innovation in the space been held up by everyone awaiting (laughs) NVIDIA's new Blackwell?

    3. EKEiso Kant38:58

       I have to say that I was quite happy Blackwell was delayed. (laughs)

    4. HSHarry Stebbings39:01

       Why?

    5. EKEiso Kant39:01

       Because I'm training on H200s, and so the compute that I brought online in- in- at the end of August, these 10,000 H200s, uh, mean that the longer the next generation chips is delayed, it helps me in a competitive nature in the world. Also, there's a lot of marketing around the next generation of chips. And again, we have to separate training and inference. Pretty much what we've seen consistently with every two-year generation of training from NVIDIA is about a 2X, one and a half to 2X, you know, performance increase. But training is about 2X every two years. On inference though, I think there's a lot of hope on Blackwell, because it looks like w- for inference, Blackwell might potentially unlock a much, much larger game.

    6. HSHarry Stebbings39:44

       When Blackwell is released, are you forced, given the competitive nature of the landscape, to get Blackwell Two, and to spend hundreds of millions of dollars on Blackwell chips, upgrading from H200s?

    7. EKEiso Kant39:55

       The way we think about this, and I think the way to think about it, is that these chips, when we- they come two times more efficient. The operations we're doing on them is still the same. It's matrix multiplications and additions and such. It's- it's- it's- it's math that we're doing on these chips. And so the Blackwell generation, for us from a training perspective, doesn't unlock anything new. It just means that we have to... we can do more with a certain set of chips. My H200s become less valuable in the world, but it does not necessarily mean I have to go upgrade to the next generation.

    8. HSHarry Stebbings40:26

       You know, we mentioned Blackwell and what that will unlock. I think a lot of people have been waiting for GPT-5 for quite a long time. When you think about what GPT-5 needs to deliver, what does it need to deliver to be a step function change, and do you think it will?

    9. EKEiso Kant40:41

       I think GPT-5, what are won't deliver isn't the question we're gonna look back on in a decade from now. In a decade from now, we're gonna look back to this moment, and it's similar, I think, how we look back to the early days of the computer or the early days of the internet, the early days of Google and others, and realize that we didn't fully internalize yet how much the world is gonna unlock in value and abundance. We wrote this blog post when the fundraising announcement came out. We said, "Look, we're in this century. I think there's three mountains that humanity's gonna climb. AGI is one of the mountains. The other is energy, and the other is space." And so I think as we're gonna keep progressing, we're gonna keep looking at the next mountain, and we- from the top of that mountain, we look back, and we're gonna realize the ones before were exponentially smaller.

    10. HSHarry Stebbings41:29

        We mentioned, like, is 600 million enough for you? I'm- I'm being slightly unfair here, but I don't understand how 6 billion is enough for OpenAI. You mentioned them in the hyperscalers, but when you look at what Zuck has said he'll spend, what, you know, Google has said they'll spend, and Larry Page saying that he's willing to go bust in the race to win, and then Larry Ellison. I don't understand how 6 billion is anywhere near enough.

    11. EKEiso Kant41:55

        If we come back to the ingredients of the capabilities race, compute, talent, data, proprietary applied research, what we are gonna find is that for compute, dollars have a direct one-on-one effect. But when we look at data, when we look at proprietary applied research, and we look at talent...It is not as straightforward as dollars in, magic, you know, and success out on the other end of it. And I think we've had lots of examples in- in technology history already, where we have seen these giant, that seemed unbeatable, IBM in the early days of the personal computer. And so if we live in a world where we could perfectly translate dollars to successful outcomes, uh, then the numbers, you know, whoever can put more there is going to win. I think that in the race towards AGI, dollars are critical- critical for compute. And remember, there's still time constraints. There's real world physical constraints of how large we can make these compute clusters for training, and it's that time and physical constraint, right, the constraint of what the chip is able to do, what the networking is able to pass through, that allow companies like us to have time and to do things and build massive advantages on the data, on the talent, and on the proprietary applied research.

    12. HSHarry Stebbings43:22

        Is there such thing as proprietary knowledge in this market? Given the incestuous nature of jumping between companies and the knowledge that moves with those people, is there such thing as proprietary knowledge?

    13. EKEiso Kant43:32

        I think you're fair to say that a lot of knowledge moves around.

    14. HSHarry Stebbings43:37

        What do you make of large corporates funding these companies, and do you have corporates in Poolside?

    15. EKEiso Kant43:42

        If you look at our capital raise, our last $500 million round, you'll see that there's none of the- the big hyperscalers, Google, Microsoft, you know, Amazon, were part of the round.

    16. HSHarry Stebbings43:51

        Was that deliberate?

    17. EKEiso Kant43:52

        That was deliberate from us.

    18. HSHarry Stebbings43:54

        Why?

    19. EKEiso Kant43:54

        There's a future we see ahead of us for the world, and right now we see a path towards that future that we can do as a standalone company, and- and we have to acknowledge the fact that, you know, we're all in the same race. And so to me, you can make strategic decisions along the way, where you decide to say, "Hey, we're- we're partnering up together in- in one way or another," like in an eq- like you're referring to equity relationships. It wasn't something that we had to do at this point, and it's something that frankly, uh, we- we very consciously decided not to do right now. There is one corporate, uh, that became part of our round, and that was very deliberate, was NVIDIA, not in an outsized stake manner at all. Uh, and it's because we collaborate really closely with them on, you know, their- their next generation of their chips, on the software and such like that. And- and frankly, we- we- we're very grateful for that and- and wanted them part of the round. The nature of- of large technology companies choosing to invest, you know, in frontier AI companies is- is frankly the game theory optimal thing for them to do.

    20. HSHarry Stebbings45:01

        Do you think we will consin- continue to see the consolidation of smaller players, like in Flash and like Adept, like Character, continue to get acquired by the large incumbents?

    21. EKEiso Kant45:10

        I think there's very few left to be acquired, to be very honest.

    22. HSHarry Stebbings45:14

        Who is left? Cohere? One.

    23. EKEiso Kant45:19

        Cohere. Uh, Reka.

    24. HSHarry Stebbings45:22

        Reka?

    25. EKEiso Kant45:23

        Reka, R-E-K-A.

    26. HSHarry Stebbings45:25

        Wow.

    27. EKEiso Kant45:26

        Small but very capable team from what I can see from the outside.

    28. HSHarry Stebbings45:28

        Where are they?

    29. EKEiso Kant45:29

        Uh, I believe they're in- in Europe.

    30. HSHarry Stebbings45:31

        Wow.
13. 46:06 – 51:00

    OpenAI, Anthropic, or X.ai — Which to Buy and Why?

    1. EKEiso Kant46:06

       are still left.

    2. HSHarry Stebbings46:07

       Okay, I've asked many unfair questions. I'll keep going.

    3. EKEiso Kant46:09

       (laughs)

    4. HSHarry Stebbings46:09

       You can buy OpenAI at $156. You can buy... And actually, quite a lot of your investors said this was a great question...

    5. EKEiso Kant46:17

       (laughs)

    6. HSHarry Stebbings46:17

       ... which I agree with. Uh, so you can buy OpenAI at $156, Anthropic at $40, which is their suggested new round, or x.ai at $24. Which one do you buy and why?

    7. EKEiso Kant46:29

       I would love to spend a day with the current leadership team of every single one of these and then make a decision. They each have inherent advantages to them. xAI has understood that compute mattered, and there's an incredible team there, and- and did what no one really had done at that, you know, speed. They built 100,000 GPU, th- three 32K interconnected clusters, in Tennessee in the span of- of months. And xAI showed up with Elon's strength, the ability to build physical infrastructure incredibly fast in the world.

    8. HSHarry Stebbings47:05

       How does that compare to Anthropic's cluster capabilities?

    9. EKEiso Kant47:09

       You'll have to ask them.

    10. HSHarry Stebbings47:11

        (laughs)

    11. EKEiso Kant47:11

        OpenAI, I think has had the incredible ChatGPT moment, and has built this, you know, incredible business around both ChatGPT and- and- and the usage of their APIs. And- and is clearly, you know, ahead in revenue as others, like, has publicly stated. And then I think Anthropic has incredible thoughtful researchers and a very rigorous approach in what they do, uh, a very rigorous scientific approach in terms of moving things forward. And so, while I can see strengths in all three of those, uh, it would really h- be a day with the leadership team of each to determine where I would put my own money.

    12. HSHarry Stebbings47:48

        That's awesome. Luckily, it's not your own money. You're a venture investor for this.

    13. EKEiso Kant47:53

        (laughs)

    14. HSHarry Stebbings47:53

        And so which one would you go for?

    15. EKEiso Kant47:55

        I'm not a YOLO venture investor.

    16. HSHarry Stebbings47:56

        What would you do if you were Sam today? You've just raised six billion.

    17. EKEiso Kant48:00

        Look, I think Sam and OpenAI have understood the importance of compute and have understood the importance of data. And what I imagine, you know, that- that $6.6 billion is- is going towards is exactly those two things. I think it is tricky to be Sam today. I think it's tricky to be Sam today because...... general-purpose models that aim to be everything for everyone is a h- incredibly competitive market, and, and you find yourself with incredible pressures from all side. Uh, and you're building a platform and a consumer product at exactly the same time, and more so than that, you're building a consumer product that from the outside is seeming to be for everyone, and I think that's a really hard thing to do.

    18. HSHarry Stebbings48:51

        It's funny, there was a brilliant Elon Musk interview. I think it was with Rogan, and he says, like, "A lot of people think they'd like to be me. It's not that fun." (laughs) It, that one stuck with me. It, you actually really hear the sadness in his voice. I don't know if you've heard that interview.

    19. EKEiso Kant49:08

        It's one I think about a lot, to be very honest, and it, it's... You're getting me even witted. It's probably one of the only things you could've said that would've got me a bit emotional, 'cause I think about it a lot. I saw it many years ago.

    20. HSHarry Stebbings49:17

        Mm-hmm.

    21. EKEiso Kant49:18

        And I think I understand what he means very well. Uh, I was earlier today with a, with a founder I, I really respect, and we were funny enough talking a little bit about this, is that you don't get... Building, building what we're building, we're... It's not a choice, it's an obsession, and, and, and you bring everything you've got to it. And we were talking about, "Hey, how do you deal with waking up at 3:00 in the morning when your mind doesn't stop racing?" You know, we're, you know, expla- going back and forth and, and sharing e- each of our techniques, and, and probably (laughs) going back home tonight and trying them. I think Elon is one of the most impressive examples of someone who has done this for such a prolonged amount of time, at moments in time when the entire world, you know, refused to align around his view. I think there's, there's companies in the world that get built because they were at the right moment, at the right time, and there's companies in the world that get built that shouldn't have the right to exist, and... 'Cause everything is against them. And, and I think Elon has done that not once, he's done it multiple times. And, and it brings me to another quote that I heard on an interview from him, or about him. It was Peter Thiel, and Peter Thiel said recently on some interview, he said, "You know, when we, when we all worked with Elon, we thought he was crazy. He would take so much risk, and then he went and started, you know, Tesla and SpaceX, and we thought he was even crazier. And if one of those two companies would've worked out, we would've said he'd gotten lucky, but both of those companies worked out and beyond, and to extremes. So there's something Elon understands about risk that the rest of us don't." And for me, that's the second quote that's been on my mind most

14. 51:00 – 55:23

    Comparing Crypto & AI: Decentralization vs. Centralization

    1. EKEiso Kant51:00

       of this year.

    2. HSHarry Stebbings51:00

       Another fantastic Thiel quote is when he compared actually kind of crypto and AI. He said that crypto specifically really embodied decentralization, and if that were the case, then AI would embody centralization.

    3. EKEiso Kant51:14

       When I was in high school, uh, in 2008, my very first startup was a virtual digital currency. My views on crypto have changed a lot over the years. I think the, the, the notion of decentralization and, and the idea- and, and, and what it can mean for the world, that's why I got excited about it very early on, is incredible... Are incredible ideals. The problem I think that we've seen in crypto is a, is a quote that I learned, it might've been my high school economics professor, uh, and he said, "Bad money drives out good money." If you think about this in environments when, when bad actors, you know, come in, it drives out the good actors, 'cause we wanna be in environments with other good actors. And I think the promise of crypto, uh, started from great actors with, you know, great views on what the world could look like, and found itself, due to, you know, the incentives of, of the ability to make money very quickly in lots of distorted ways, to bring a lot of bad actors to it. And I think the bad actors have driven out a lot of the good actors over time. There are still a true amazing idealists in that space, but I think it's, it's a place where I think it's hard to be. Now, the thing in AI is that we don't have that. We have a l- we have a set of people around the world who all sh- fundamentally might disagree on how to get there, but all see that in 10, 15 years, we're gonna look back and realize we had this incredible shift in the world by being able to, you know, close the gap between machine and human intelligence. While that might drive today some centralization because of the sheer amount of resources required that are scarce, right? Capital's the least scarce part of this, right? The talent is scarce. The proprietary, you know, applied points of view and research, those are scarce, you know? Uh, and so I think that does lead itself to, to a small number of companies. I think we've seen that over and over in history. We look at when the, you know, we got the massive boom of automobiles, right? The hundreds of automobile companies that started and how many actually survived. We've had this over and over again, and so I don't think this is something new. What I would like to see, and what I think is going to happen, is that it's not just Google, Amazon, and Microsoft. It's going to be, you know, an OpenAI, an Anthropic, a Poolside, and a set of companies who are, who are able to get that massive escape velocity needed to sit alongside those companies and build the next, you know, generational businesses.

    4. HSHarry Stebbings53:52

       You mentioned bad actors there, and it made me think of tourists, and when I thought of tourists, for some reason I thought of, like, bluntly, and this sounds awful, but, like, people who are not in it for the long term or who are in it for, uh, a story. And a lot of, bluntly, public company CEOs and large company CEOs, and this is not tourists or bad actors at all, but they have to tell an AI story, and they have to show that they are spending money on AI and innovating in some way.My question to you is, when you, you- you mentioned obviously the GTM team build out. When you think about the revenues that we're seeing today, are we well past the experimental budget phase? Are we into true deployment, true commitment? How do you see that from enterprise communication?

    5. EKEiso Kant54:38

       So I think it depends on the use case. There's lots of experimental use cases still, and there's use cases that are far past the experimental side. AI for software developers. I don't think anyone in the world any more questions that software development moving forward is gonna be a, for the foreseeable future, a developer led, AI assisted world, an increasingly AI assisted world.

    6. HSHarry Stebbings55:00

       Which use case do you see that you least understand or think has long term potential?

    7. EKEiso Kant55:06

       I think there are use cases that are commoditizing very quickly. Speech recognition-

    8. HSHarry Stebbings55:12

       Mm-hmm.

    9. EKEiso Kant55:12

       ... I think is one of them. I think, uh, what are still a bit of a gap, I think image generation is one already where we're seeing more and more commoditization over

15. 55:23 – 59:01

    The Decision to Stay Europe-Based

    1. EKEiso Kant55:23

       time.

    2. HSHarry Stebbings55:23

       You've mentioned talent before being such a crucial part, but we haven't really unpacked, because we have discussed the models, the data, the compute. The talent perspective is one that you also have taken quite a different approach on. You- you're a European based company. The big question that a lot of your investors said that we have to discuss is, why did you decide to keep this as a European based company?

    3. EKEiso Kant55:43

       I wanna set the record straight. Uh, we're an American company, and we've got incredible people from all the way from San Francisco to Israel. A decision that we made early on is, we were actually, Jason and I, my co-founder and I, we were planning on building this company in the Bay Area. And we did the work in the first days at a company, and- and the work was, "Let's make a list of everyone we think, from both that we knew and also, like, externally, like, you know, on research papers an- and- and GitHub repos, that we think potentially could be great for us." And the list ended up at about 3,300 people. A lot of work done. And-

    4. HSHarry Stebbings56:19

       Fucking hell. (laughs) 3,300?

    5. EKEiso Kant56:21

       3,300 people that we saw ranged from having experience on distributor training, to GPU optimizations, to work on data, to reinforcement learning, you know, experienced with large language models. So, you know, the whole breadth of what it takes to build what we're building, uh, from a model perspective. In that list was a location column, and as you can expect, the number one represented geo in that was, was the Bay Area. Not even the United States, just purely the Bay Area. But what really was striking for us is that there was a huge part of that list that was not in the Bay Area. It was spread out across Europe and Israel, and it was spread out all over, from the UK, to Switzerland, to Tel Aviv, to Amsterdam, you know, Paris, like, all of these different places. And so while we couldn't see a clear, deep talent concentration in one place, probably UK actually being the one with the largest talent concentration, we didn't realize that it was probably worth spending some time talking to people there. We realized one thing. We said, "There's incredibly capable people here who wanna stay here geographically, they don't wanna move to the Bay Area to join some of the other companies in the space, but they're not finding massively ambitious, you know, young companies that they- that have huge visions to join." And so we saw that as an advantage, right? To- to- the- the thing, the four things in that, you know, capabilities race, we need to build unfair advantages for every single one of those. And so we said, "Great, let's build up talent, you know, here on this continent in Europe, as is we will in the United States." And, uh, and frankly, I'm very grateful that we did.

    6. HSHarry Stebbings57:58

       How many people do you have in London?

    7. EKEiso Kant58:00

       So London for us is about 15 people.

    8. HSHarry Stebbings58:02

       How many in Paris?

    9. EKEiso Kant58:05

       Uh, two.

    10. HSHarry Stebbings58:07

        Paris is meant to be the AI hub of Europe, no?

    11. EKEiso Kant58:10

        Where has talent historically been, even pre-ChatGPT moment, right? And talent in AI, and who helped build that talent in the space? The number one company we have to give credit to is DeepMind. DeepMind built an incredible talent base, and they built- they built it out of London. Meta did some work in building a very incredible talent base, and it did it between London and Paris, but in terms of when you look at it from a numbers perspective and- and sheer size of- of people, I think, you know, Google separately and DeepMind as- as part of Google, uh, had made much larger investments. And then there's another talent pool that we do- often talk about publicly that is just absolutely extraordinary, which is Yandex. Yandex built an incredible company in Russia with some of the world's most capable researchers and engineers, many of which have since left Russia and have kind of become a diaspora all over

16. 59:01 – 1:04:53

    Work Ethic & Work-Life Balance

    1. EKEiso Kant59:01

       Europe.

    2. HSHarry Stebbings59:01

       When we look at that talent, when we think about work ethic, it's one thing which Europe is often chastised for. In terms of work/life balance, how do you approach that and feel about implementing standards of work with teams?

    3. EKEiso Kant59:17

       There was a tweet, and if I recall correctly it was from Aaron, uh, Levi from Box-

    4. HSHarry Stebbings59:21

       Yeah.

    5. EKEiso Kant59:21

       ... uh, early on in- in post-ChatGPT moment. And- and he wrote something along the lines of, "If you feel like you're working extremely hard on reasonable hours as AI is now booming, you're probably right to do so." Because it's in these first years that, and now I'm probably going beyond what the tweet said, but I think it's in the first years, it's where the table gets set. Who has earned the right to be in the race to AGI? And the way that I've- I've always looked at this is, from a personal perspective, and so has my co-founder, and so has Margarita, like, is- has put us in a place where we're gonna look back on this moment 10 years from now, just like we would look back to the moment of mobile, internet, you know, first personal computer, and realize that that was the moment where the table got set. And you do not wanna l- you do not wanna look back on that moment and not have given it everything you've got, because it's a race. And look, most startups are not races. Most startups are against your- yourself.But AGI is a race, and so our view always has been, is that the team that we build is a team that is deeply passionate to be in that race, and, and knows that, and knows h- And, and frankly, when you decide to join a race and you're upfront about it, you decide to be, try to become the gold medalist in swimming, that means that there's sacrifices that come with that. You don't get to have it all. And, and so that's something that we've been super open about, with, with people from, you know, day zero. It's on our first intro call we talk about, like, "Do you wanna join a race?" And frankly, I have found no shortage of people in Europe that, or in America... There's a stereotype about Europe, but the fact of the matter is the people who wanna join races and do truly their life's work, they're built differently, and you can find them all over the world in every single country. You just gotta do the work to find them.

    6. HSHarry Stebbings1:01:20

       Chase Coleman had an interesting, uh, kind of stat, and it was in the two years subsequent the founding of Netscape, 1% of the value, enterprise value, of internet companies was created. 99% was in the chasm between that subsequent two years and now. Meaning, actually, it is such a long process, and so much is to come. Does that not go against the idea of it being a race, and is now different?

    7. EKEiso Kant1:01:48

       So I think, you know, there's the classic quote of, uh, you know, history doesn't repeat itself, it rhymes. It's failing for me from, from who it was. I think it was Mark Twain. And I think that might be the mistake that we're possibly making, looking at the past. And, and the reason that is, is because we're on an exponential, in terms of technological progress. I think in 1996 with Netscape, if I'm getting the year right, there wasn't this amount of people and capital that understood what the future might look like in the next 10 years, and it took some time to, to get there. And, and now I could be, I could be wrong about this, another thing that I could, could see as a possible avenue of why I tend to disagree is, there's a big difference between what was required to be built in 1996 versus what's required to be built today. You know, if I, if I bring it all the way back and try to steel man the opposite side of the argument is, maybe it's exactly that, and, and, and that the next couple of years are about these massive capabilities that were moving the world closer towards AGI. And then when you look at the following five or 10 years, it's true. The huge economic value that's gonna come from that will hug- of course surpass the economic value that we have. I think the economic value is gonna continue to surpass on the exponential that we're on. But what I don't agree with is that, uh, the companies that are being built today will not have a set of companies amongst them that will become the giants of the future that have helped enabled us.

    8. HSHarry Stebbings1:03:22

       I think the concern that I have is you will use a huge amount of dollars to get to a level of advancement and technology, that will then be leveraged by other people to build incredibly valuable companies. If we look at battery in particular, where there's kinda been unbelievable breakthroughs in battery technology, by companies that you will never have heard of that kind of got acqui-hired, went out of business, and then were bought for their IP. And it's a case of actually, it takes a huge amount of money to uncover new breakthroughs, and then those breakthroughs are taken by someone else.

    9. EKEiso Kant1:03:53

       So if, if we take the battery analogy, I would actually think a little bit about BYD. Started as a battery company, today is, right, the largest volume of electric car sold in the world. And, and I do think there is a lot to be said about deep vertical integration, right? And, and if we look at, look at Poolside, we're building foundation models with a mission towards AGI, we're right now in the world focused on bringing more and more capabilities of AI to software development, building a truly end-to-end business. Because I agree with you that the value is not going to only accumulate at the model layer. It's gonna accumulate all the way to the end user. And so in our point of view, the, the way that we kind of, I think, get to avoid, you know, what the future plays out in, in, in your hypothetical scenario, is just by truly doing it end to end. But I still actually look at this thinking that there will be more value built on top of us in the future than what we can possibly unlock only ourselves.

17. 1:04:53 – 1:06:48

    Is China 2 Years Behind Than Europe?

    1. EKEiso Kant1:04:53

    2. HSHarry Stebbings1:04:53

       Last one, and then we'll do a quick fire. You mentioned BYD. Unbelievable journey. Are China really two years behind the-

    3. EKEiso Kant1:05:00

       No. No, they're not. Uh, there's a couple of interesting things that might not be as obvious unless you're in our industry is, the research that still gets published openly, right, uh, that doesn't get held back, that is most interesting, is all coming out of China in, in, in vast spades and majority. Something that wouldn't necessarily be obvious, but if you think about the game theory optimal thing to do, because they're not on the forefront of the world scene of AI, actually opening up some of that research is the game theory optional, you know, optimal thing to do to be able to continue to attract talent. Because that's really what opening up your research does, right? It attracts talent to you. Um, no, I think, I think China is at an incredible level of capabilities, uh, and in no way should be discarded or thought of as years behind, uh, on AI or AGI progress. We're working on technologies that we can see have massive societal impact, and I think it's really important to be good stewards of that technology and that progress. And I think part of that is acknowledging that what we know about is the technology, what we know about is our users, our customers, but we sh- we should be, you know, very m- be careful in terms of trying to know what's best for the world, and how to think about massive geopolitical conflicts and things like that. The best thing that we can do as the West is to keep making it as attractive as possible for talent...... from China, we consider as a competitor, right, on a, on a very large scale macro, to come to our countries. The easier we make it, you know, for one of those four major ingredients in the capability race, and frankly, one of the most important ones, to, you know, help us accelerate, I think is probably the- the most practical advice

18. 1:06:48 – 1:17:42

    Quick-Fire Round

    1. EKEiso Kant1:06:48

       I can give.

    2. HSHarry Stebbings1:06:48

       I'm gonna do a quick fire round because I could talk to you all day. So I say a short statement, you give me your immediate thoughts. Does that sound okay?

    3. EKEiso Kant1:06:54

       Let's do it. Let's try it.

    4. HSHarry Stebbings1:06:54

       Okay, so what have you changed your mind on most in the last 12 months?

    5. EKEiso Kant1:06:59

       I think in the last 12 months, it's a continued realization of the importance of scale of data, uh, and not only compute.

    6. HSHarry Stebbings1:07:11

       Do you regret not selling Source to GitHub?

    7. EKEiso Kant1:07:14

       It was probably the dumbest financial decision of my life, considering it was an all-stock offer when GitHub sold to Microsoft, I think, less than a year later.

    8. HSHarry Stebbings1:07:21

       And it would have 3Xed the price.

    9. EKEiso Kant1:07:24

       Far higher than that. Uh, but I'm really grateful I didn't.

    10. HSHarry Stebbings1:07:29

        Why?

    11. EKEiso Kant1:07:30

        First and foremost, I'm sitting here.

    12. HSHarry Stebbings1:07:32

        Would you have done Poolside if you had sold Source?

    13. EKEiso Kant1:07:35

        I think the question is, what- what could I have been able to do continuing on Source mission? Because Source mission was the mission we're talking about today with Poolside. And back in 2016, there were very few people who believed it was ever possible for AI to write code. But no, I- I don't think... There- there's really no regrets there. I wouldn't be sitting where I am today, and I don't think I would have become the person that allows me to go build Poolside today. And frankly, I'm really grateful that that event did happen, because that's how I met my co-founder, right? That's how I met Jason. He was the CTO of GitHub at the time, and it started this many-year conversation on what the progress in AI looks like and- and its applicability to software development.

    14. HSHarry Stebbings1:08:15

        What do you think is the biggest misconception of AI in the next 10 years?

    15. EKEiso Kant1:08:19

        That it's going to... That progress is going to halt.

    16. HSHarry Stebbings1:08:22

        What would cause progress to halt?

    17. EKEiso Kant1:08:24

        Global conflict that disrupts the supply chain of chips.

    18. HSHarry Stebbings1:08:30

        That is fucking meaty for a quick fire round. I'm- I'm struggling here.

    19. EKEiso Kant1:08:33

        (laughs)

    20. HSHarry Stebbings1:08:33

        I really can't unpack that in- in 60 seconds.

    21. EKEiso Kant1:08:36

        (laughs)

    22. HSHarry Stebbings1:08:36

        If you could have any board member in the world, who would it be?

    23. EKEiso Kant1:08:39

        Mark Zuckerberg.

    24. HSHarry Stebbings1:08:41

        Why?

    25. EKEiso Kant1:08:41

        I think we all should give a lot of credit to Mark Zuckerberg in terms of having built an incredible company with a lot of conviction on what the future would look like when most people didn't agree with him. If you look at what he's done on AR and VR, uh, in the, uh, in- in the last decade, right? From buying Oculus to where it is today, when most of the world just wanted him to stop. I think that required an incredible ability to have conviction for a future that's gonna massively change the world. And I think to get to AGI requires an incredible conviction for a technology to change the world. He's one of the people who has, who has done that and then... And someone very different than the people I have currently on our board.

    26. HSHarry Stebbings1:09:29

        What's the worst thing that could happen for AI with regards to regulation?

    27. EKEiso Kant1:09:33

        The reality of regulation, in many cases, is that it becomes a, an expensive bureaucratic overhead, and that harms the most young startups. It doesn't harm companies that have raised massive amounts of capital.

    28. HSHarry Stebbings1:09:45

        What specific regulations should be taken away?

    29. EKEiso Kant1:09:48

        The world is finding a balance, uh, and the balance that I'd like to see that the world is... And I think it's moving towards it, is to regulate the end user application of AI, just like we've regulated the end user applications of any type of technology before. It's not the database that does harm, uh, it's, you know, how it's used. And so for me, I- I would love us for us to- to continue to hold companies massively accountable for the end use of their technology to- to users, to- to consumers. Less so trying to put, you know, limitations on, you know, how much compute you're allowed to train on or, you know, we are, we are, we're building tools that are closing this gap between human capabilities and machine intelligence. We are not building the Terminator.

    30. HSHarry Stebbings1:10:36

        What do you think of DST's Yuri Milner?

Episode duration: 1:19:00