No PriorsBeam: The Great American Open Model with ReflectionAI Co-Founder and CEO Misha Laskin
EVERY SPOKEN WORD
70 min read · 13,653 words- 0:00 – 0:05
Intro
- MLMisha Laskin
When you remove cyber offensive capabilities, you also remove cyber defensive capabilities. The state of the world
- 0:05 – 0:22
Misha Laskin Introduction
- MLMisha Laskin
today is that we have a few hundred safety researchers within closed labs that understand how these things work, and despite their best intentions, it's impossible to cover the long tail of unintended consequences that these systems might have. A very powerful closed model went and, like, hacked into another company, and the only way that company could remediate
- 0:22 – 1:30
Latest with ReflectionAI
- MLMisha Laskin
itself was by using open models to protect itself. That's the empirical evidence of the world that we're in. Linus's Law, with enough eyeballs, all bugs become shallow. I have the belief that with enough eyeballs, most security and safety vulnerabilities become shallow as well.
- SGSarah Guo
[upbeat music] Today, we're joined by Misha Laskin, the co-founder and CEO of Reflection AI. Reflection provides open-weight models to power the future of intelligence. Misha, prior, was a researcher at Google DeepMind and received his PhD in physics. Welcome to No Priors. Great to have you here today.
- MLMisha Laskin
Yeah, it's great to be here. Thanks for having me.
- SGSarah Guo
Could you give us an update on where things are at? I mean, obviously, you guys have been pioneering really interesting work in open-weights and open source models, particularly with a US bent. Uh, can you talk a bit what you've been up to and what you guys have been building?
- MLMisha Laskin
Yeah. So for the last 12 months, we set, right, the mission of the company to build frontier open intelligence and make it widely accessible. And effectively, it's been a sprint to set up a lab that is capable of, of doing such things. Uh, it, it kind of
- 1:30 – 3:19
Challenges Building an Open Model
- MLMisha Laskin
feels like, uh, maybe there's a Reid Hoffman quote that, you know, you're assembling a, a plane as you're flying it, uh, and that's, you know, that, that's been the case. We were maybe around 30 people about a year ago, uh, but it does take, uh, order of 100 plus, so you know, a couple hundred researchers and engineers to build one of these things, like a, like a true rocket ship project, and we're now at around 300 people and, you know, assembled all the teams on pre-training, mid-training, reinforcement learning, scaled up our, you know, trained our first models, uh, end to end, and, uh, just, you know, released a model called Beam, which is Reflection's first open model.
- SGSarah Guo
What, what has been the most challenging part of that? Because when people talk about scale or scaling up models or starting to build these models out, one issue is compute and procuring enough, particularly a training cluster that's, uh, cohesive in terms of the ability to use it. Um, second is, uh, talent, and obviously, there's been these huge sort of talent wars in terms of how much people will pay for researchers, et cetera. Uh, third is sort of scale of data. Like, what-- which of those has been the most limiting or the most challenging?
- MLMisha Laskin
What's been interesting about this experience is that I got asked this questions at, at, at a, at a all-hands at the company, "What's been the hardest thing, you know, in the-- in the last year?" And, uh, the reality is that everything is extremely hard.
- SGSarah Guo
[laughs]
- MLMisha Laskin
[laughs] Everything has been hard.
- SGSarah Guo
Startup, yeah.
- EGElad Gil
[laughs]
- MLMisha Laskin
Uh, and it's, it's also sometimes, you know, uh, people might ask, like, "Well, what's the thing that, you know, really, uh, made your models work as a, you know, better than others?" or something like this. And again, the, the answer is, like, everything, right? You, you actually have to get 30 things right, and that's why everything is hard because you have to get 30 things right. You have to get the talent. You have to retain the talent. You have to have a strong mission and culture, uh, that keeps the talent, uh, you know, working together and rowing
- 3:19 – 7:00
ReflectionAI’s Agentic Shift
- MLMisha Laskin
in the same boat. Uh, you have to get the data. You have to get the compute. You have to build the infrastructure, uh, to ensure that the compute is actually usable. Uh, and these are-- a lot of these things, when you're at an existing big lab, there have been years of build-out that enabled them to have this kind of stable surface. So I was at DeepMind before this, and a lot of the tools that we just take for granted, or I took for granted as a researcher because they just worked, you have to build. So I would say, yeah, everything has been hard, uh, but also very rewarding that, you know, it's, uh, that, that you can do it.
- EGElad Gil
Can you talk a little bit, you said about a year ago, you, you guys really committed to do this. Can you talk about the, the sort of, um, shift in ambition and commitment and what drove that?
- MLMisha Laskin
When we started the company about two and a half years ago, uh, our bet was-- So, uh, maybe I'll contextualize it, where the world was then. Uh, my co-founder, Yannis, and I had been working on the first series of Gemini models. We were working on the reinforcement learning team. Uh, Yannis was leading it. And we had just shipped the Gemini 1 and then 1.5 model, and that was, you know, that early era that was, like, similar to the first ChatGPT experience, which was primarily a chat experience. There was no coding really, no- nothing agentic. Uh, these models were just good at chat. And they were primarily, you know, 95% of the compute spent was spent on pre-training, so they're primarily pre-trained models. Um, we are reinforcement learning researchers. That's been our lineage. Uh, Yannis, my co-founder, was one of the founding engineers at DeepMind and, uh, was a key contributor to all the big RL projects that came out of that lab, including AlphaGo. And our bet was, because we were working on the RL team and we saw, you know, RL for aligning, uh, chat models works, you know. There's only so much you can align them, right? Uh, well, now there's a lot more, uh, [chuckles] you know, now that they're agents. But at the time, they were just chat. So RL was working, and we just thought that, well, if you take that and apply it to domains where, like mathematics, coding, like, you could, that you could make these systems agentic. Um, that was the big bet. And we thought that you could do this as an independent lab, um, much more capital efficiently because we started seeing open source, open, open models materialize. There was a small model from Mistral at the time. Llama 2 had just released, and we saw Llama 3 coming. Uh, and we-
- EGElad Gil
Which feels like ages ago.
- MLMisha Laskin
Yeah.
- EGElad Gil
[laughs]
- MLMisha Laskin
Yeah. This was a while back, I guess, uh, two years ago. [laughs]
- EGElad Gil
Uh.
- MLMisha Laskin
And- And we thought that there would be a great open model base that someone would build that, that we could build on and do our research and scale reinforcement learning. And over that first year of the company, what ended up happening was that, one, reinforcement learning started working faster than we thought it would, uh, both within, you know, our experiments, but also, uh, within industry. o1 came out at that time.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
We actually thought it would take longer just because the journey from ImageNet in 2012 to RL systems that worked at scale was about four years, so we thought maybe something similar. But every year moves faster than, uh, than my prediction the previous year. Uh, and so about a year into the company, we got to the point where there are actually-- like, all the good open models are coming from, from China. There were not really good, uh, you know, Western open models, and we needed one as a base for kind of the technology that we were building, uh, and decided for a number
- 7:00 – 9:32
Resources for Model Training
- MLMisha Laskin
of reasons, um, that the most impactful thing we could do, given the state of play, was to just build open models ourselves end-to-end and, you know, combine the RL bet together with just building models end-to-end. There were, you know, there-- I guess there were enterprise reasons, there were geopolitical reasons happening, but also there was the research reason that it turned out you actually do need to pre-train your model in order to make reinforcement learning work very well at scale. So you actually kind of do need to do... The, the things are just so tightly coupled that, uh, you, you do need to do both.
- SGSarah Guo
The conventional wisdom has been, uh, that pre-training at the scale of the frontier is, um, impossibly expensive, or at least extraordinarily so. Um, clearly there's been a huge shift of compute toward post-training. Uh, feel free not to answer this and we can just cut it, but, like, what do you-- how, how did you think about just the resources needed or the ratio, or did you have, like, design principles like, uh, around the model that made this feel tenable to you at the beginning? Or did you just say, "We need to do this. We'll go resource it"?
- MLMisha Laskin
Well, the, the nice thing about being an open model lab is that, uh, you can also talk about things openly. Uh, so you do need a lot of resources to, to build something meaningful. You don't need the same amount of resources. Uh, once, once you get to the frontier and you're really, really at the frontier of intelligence, then you need the same amount of resources another-- as any other frontier lab because you can kind of think about research as, um, exploration of new ideas versus, uh, execution of known ideas.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
Uh, part of why getting great talent in matters a lot is because you get to cut down on exploration and then you-- and focus on execution of the things that, that work. Uh, so all this is to say is that if you are, you know, catching up to the frontier, you can do it a lot more capital efficiently.
- SGSarah Guo
How much capital do you think it would take to catch up to the frontier?
- MLMisha Laskin
Well, I think that it depends. The frontier keeps moving-
- SGSarah Guo
Yeah
- MLMisha Laskin
... so each year it's actually more capital. So, um, and I'll get back to the question on the resourcing between, let's say, pre-training and, and post-training. I would say that a year ago or maybe 18 months ago, it would've been-- and I say orders, meaning that, you know, order of a hundred million, so hundreds of millions of dollars.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
I think that now or let's say even six months ago is probably order billion, so billions of dollars, single-digit billions. Uh, going into next year, I think order 10. You know, you can kind of think
- 9:32 – 11:43
Scaling Efficiency
- MLMisha Laskin
about it as for every generation of model, there's a four X multiplier in compute roughly, and one heuristic that I have is like, uh, when-- L-let's say, like, we, we talk about, like, the chips that are currently considered, you know, the ones that, you know, a frontier chip and maybe a couple years ago was an H100-
- SGSarah Guo
Yeah
- MLMisha Laskin
... and maybe it was a hundred thousand H100s. Then, you know, Astra, right, was trained and that was just a big run, right, on a hundred thousand Blackwells, which is roughly a four X multiplier on an H100. Now, the next generation is gonna be around a hundred thousand Vera Rubins-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... I believe. So that kind of gives you the scale, and it moves from hundreds of millions to billions to tens of billions.
- SGSarah Guo
Yeah. It seems like at some point that has to asymptote then because ultimately, relative to revenue scale of these companies, even potential revenue scale, you start to tap out in terms of the ability to invest against what you're actually gonna produce from an economic value perspective.
- MLMisha Laskin
Absolutely.
- SGSarah Guo
So do you think we're heading into that asymptote in terms of model size or at least training cluster size?
- MLMisha Laskin
Absolutely. I think that in terms of, uh, you know, there's only so much CapEx that you can do, right? And frontier labs are now putting hundreds of billions of dollars into it, so will they be putting trillions of dollars-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... you know, in, in the next year or two? I-- You know, it's probably un-- you know, unlikely, but there are a couple of things that are happening. One is that the use of that compute is becoming a lot more efficient-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... because the models are themselves becoming tools that help build themselves.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
So I think that when it was just, you know, human researchers doing the work, there's probably a seven X improvement, uh, each year, and you can actually track these things very technically. In, in pre-training, you do it by, you know, you have your pre-training loss and then you measure, you know, your compute efficiency gains against, you know, you made some changes. Now you have a new pre-training loss and, uh, it should be you got to the same point with less compute. And so when you say seven X, it's actually a very concrete thing that you're studying. Uh, and now I would say across the systems of, you know-- I mean, pre-training is probably a bit more hardened, but there's a lot of, um, a lot of headroom in reinforcement learning. We're probably at a point where it's, you know, 30X,
- 11:43 – 16:14
Training Beam
- MLMisha Laskin
like, efficiency depending how good your model is for actually improving itself. So that is one thing that's happening is that-
- SGSarah Guo
30X per year or over what?
- MLMisha Laskin
Yeah, something like that.
- SGSarah Guo
Per model generation?
- MLMisha Laskin
Yeah. I mean, it depends on, you know, what metric it is, but it, it roughly-- I do think that the speed is probably four or more times faster than researchers just doing it alone, and that'll probably accelerate. So the amount of intelligence you can extract Per training flop is, is increasing. So at some point, that asymptote is okay. And then also, like the amount of revenue that each flop can generate also increases, right, the more you pack intelligence. So going back to the question around how does this distribute across pre-training and, uh, reinforcement learning and to train Beam, which is a 500 billion parameter model total, 23B active, it was 6,000, uh, GB300s. We ran it, I think, for, for a few weeks, but now, you know, like with infrastructure efficiencies, we can do it in about 12 days, maybe less.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
Uh, so right, there's a coupling of scientific and infrastructure efficiencies. Um, reinforcement learning was a little over 10,000 GB300s for four weeks. So actually, there were more flops spent on reinforcement learning. Reinforcement learning jobs are, you know, more complex in the sense that you do both a lot of inference at scale with all sorts of sandboxes for agents and training.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
Um, th-this actually, the, the reason I got into AI was, uh, I saw my co-founder's work, um, y- on AlphaGo, and I was a physicist at the time. And, and there was a moment, like, there was a plot in AlphaGo where it just never stopped improving, right? It just kept improving, and they just cut it off at some point because what's the point of improving it further? You already beat the world champion.
- SGSarah Guo
Yeah.
- EGElad Gil
Mm-hmm.
- MLMisha Laskin
And I thought that, well, if you start apply- figure out how to apply that recipe to some, to stuff that is economically valuable, then well, at that point it becomes an economic question of-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... how much money do I wanna put in to get, to just keep improving the system. And we as a field are, are there. And one of the, I think, things that I'm really excited about in this open source project and the tech report that we'll have in it is that we'll, you know, describe how, how these things are built. And indeed, our, like, this reinforcement learning system never stopped learning. If you look at our plots, they just keep going up, and it's just a matter of compute basically, right, in terms of scaling it, uh, further. So that is to say, we as a field are there, where you have these RL systems that are very general, that don't really stop improving. You wanna make them a lot more efficient to extract the most intelligence from your compute. But that means that you move from, you know, a question of can you train these things to where can you get the data and what is economically valuable, and it's just an economics decision of how much money do I put into X to get some kind of improvement out of it, um, which is incredible. You know, that's, uh- [chuckles]
- SGSarah Guo
Yeah, yeah
- MLMisha Laskin
... it's, this was not true two years ago.
- EGElad Gil
What can you share about where you've already decided, like, we should turn the crank in terms of, uh, just continuing to invest in capability?
- MLMisha Laskin
This is almost to some kind of dissatisfaction to some scientists and that it's moved-- I mean, it is a science, but it's, you know, I would say m- feels more of an engineering discipline. Like, it seems more like a, again, the rocket ship analogy and the way that building rockets is a, is a science, but most people think about it as an engineering discipline with some scientific work in it, and that's roughly, um, how it feels. So I-- the stages are kind of set in that they're, the axes of scaling are today. And now there might be something kind of totally blue ocean that we don't know, and there are some new labs exploring it. But it's, um, you know, pre-training, synthetic data, reinforcement learning.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
Or in one-- or in some sense you can say just training and reinforcement learning, and there are architectural improvements that can be made. Um, there are, um, data improvements, algorithmic improvements, but there, it's not like a, it doesn't feel like a Wild West the way I felt five years ago or something. Like, when, like, the, the-- something that was kind of beautiful about both, I think, DeepMind and Google Brain, um, and even if you look at early OpenAI, is how diverse the bets were, and people were doing just crazy, fun stuff.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
And, uh, now it's really narrowed in and, and part of it there's, like, a, also a bit of a hardware lottery kind of thing, 'cause once something starts working, the hardware also starts co-optimizing against it, and so it's harder to find, even if you have
- 16:14 – 19:18
Where Model Value Comes From
- MLMisha Laskin
a great new idea, if it's not really a good fit for the hardware, um-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... you know, it doesn't make sense. So, so to answer your question, I don't s- there hasn't been an asymptote of how much, how much juice you can get out of better pre-training. Um, like, we're not seeing a slowdown in compute efficiency gains. You keep seeing stuff. Um, there's a lot, a lot of headroom in reinforcement learning. I think that that one is probably, um, there's a lot of stuff to be kind of discovered there. But it's, again, it feels more like a engineering discovery, um, than groundbreaking understanding of a new science or something like this.
- SGSarah Guo
Um, and you mentioned investing against, um, some economic activity or some productivity out of this that then allows you to sort of keep going from a capital, et cetera, perspective. Is there-- I, I know, for example, Anthropic placed a pretty early bet on code. Obviously, OpenAI kinda did that too. They originally had, you know, a coding model that they were working with GitHub Copilot on. Um, and then they, they obviously diversified into consumer and other areas. Um, are there specific application areas that you all are most focused on from the perspective of economic value creation? Is it code and agentic workflows? Is it something else? 'Cause that seems to be the basis for a lot. I'm just sorta curious-
- MLMisha Laskin
Yeah
- SGSarah Guo
... if there's a hypothesis on where your model will be used the most or where you need to direct it most.
- MLMisha Laskin
Yeah. I think the, the code and agentic stuff kind of sets a foundation for the intelligence, and the intelligence is ultimately... It's, it's interesting. It's jagged in the sense that, one, it's, it's generalized in a surprising, you know, in a surprising way. Like, the, um, I was surprised by how quickly our model got to a level of capability that, um, is, you know, fairly fresh and recent, right? That, uh, you know, other labs were discovering not so long ago. And so there is some kind of stitching, like generalization happening. Um, at the same time, it is somewhat jagged, where, um, it, it quickly adapts to, to data, um, once you have the right data for, you know, tasks you wanna do, but there is kind of a jaggedness about it. And it's even between, like, the various benchmarks. Like- The various versions of Terminal Bench, like you don't see clean generalization between different harnesses. Um, but once you have something working, you have a general agentic capability, and you get the data for a new harness, it actually starts adapting to it pretty quickly. So what that means is that you have this like pretty adaptive thing that you need to... And, and then it's a matter of like where are the economically valuable pools of data. That's, I mean, part of what makes open models, I think, powerful is that, well, enterprises and customers can take it and customize it for their own stuff and get something that is kind of Pareto optimal for their workloads, like the lowest cost for the highest performance. Um, and so it's an, it's an empirical, it, it's an empirical question. You go to kind of customers, you actually see what is valuable to them, and you see like whether it's possible to... And usually when it's economically valuable, like you can gen-generate data because you're, you're...
- 19:18 – 21:58
Beam’s Reasoning Efficiency
- MLMisha Laskin
It's not even that you're gonna be training on the customer's data, it's more that you'll be set-- you're setting up evaluations, and if you can set up a good evaluation for their tasks, then you can generate synthetic data that approximate it, and you get good generalization. And so concretely finance, like various know your customer flows and compliance flows in finance are very valuable. A lot of things, different things in cybersecurity, uh, cyber defense in particular are quite valuable. Legal, like there, there are all sorts of kind of, uh, uh, like agentic verticals like within enterprise that are, that are valuable and have a pretty similar pattern in terms of how you get something-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... uh, working there.
- SGSarah Guo
One thing that stands out on, on, uh, Beam is the reasoning efficiency of the model. Could you tell us a little bit more about that?
- MLMisha Laskin
So Beam is... So it's a model, right, that was trained for, uh, coding and agentic tasks. That's where it excels. Uh, and an important thing in model building is not just the capability, but how quickly an agent achieves a thing. So it's, uh, it translates to faster, you know, workload times, cheaper costs for customers. If you've ever sat around a, a, AI, you know, some, your favorite AI chat and asked it something, and it took it, you know, 10 minutes, um, much better if it does it in one. So Beam tends to be three to four times more efficient than, uh, models of the same, you know, capability class and, uh, much more efficient when it comes to even, you know, there are m-models that are larger out there, uh, where, you know, where the efficiency gains are then end up being something like 10X. Uh, and the reason, the reason it's so efficient is because, uh, we prioritized, well, both a strong pre-training base for reasoning, but then amplified it with reinforcement learning at what we believe to be the largest scale that's ever been done in open source. Um, I've, I've not seen a 10,000 GB 300 for four weeks run, um, documented yet. Uh, and reinforcement learning, what it, you know, when you-- The way you tend to set it up is that you wanna extract the maximum amount of capability in the least amount of time. And this is the same thing that happened in the previous systems like AlphaGo. The first AlphaGo agents were, um, pretty, you know, meandering in the way that they were solving the problem, and then by the time you got it to Lee Sedol level, it was just, you know, very smart in its, uh, kind of in its search, right? So, uh, that's really what enabled it to have that capability. So, you know, the more re-reinforcement learning you run, the higher the capability and the faster, uh, these, these systems sol-solve it.
- SGSarah Guo
Mm-hmm.
- EGElad Gil
Awesome. That seems very pragmatic.
- MLMisha Laskin
The, the intention, I guess because we are kind of RL believers, and so we always believe that that was the,
- 21:58 – 24:28
Monetizing Open Weight Models
- MLMisha Laskin
uh, the path to AGI was, you know, through reinforcement learning with actually a s-really strong base. We forget that n-the first AlphaGo systems, um, had-- They were trained on, uh, expert-amateur human games, so they did this imitation learning first and then reinforcement learning.
- EGElad Gil
Mm-hmm.
- MLMisha Laskin
So you kind of need those things working together and, uh, and a byproduct of it is that, um, you get this really economically kind of valuable thing that it's very reasoning efficient and makes a really nice workhorse model for enterprises and, um, and, and public sector sovereign.
- SGSarah Guo
And from a monetization perspective, uh, what is the pa-- There's different ways to commercialize both open source and open weight models, and Mistral and others were early in terms of different, uh, different, uh, approaches to doing so. Is there a specific direction that you all have been heading and that you can share?
- MLMisha Laskin
Yeah. I think that, uh, commercialization of, let's say, open and closed models is in some sense pretty similar. What you're really trying to do is you're trying to maximize inference. That's really what you're trying to do, and the difference is that, um, it's, uh, like basically like rental versus ownership inference. So an example is that when you're buying a token, you're renting like a piece of a whole stack, which is the harness, if it's, you know, like, uh, if there's an agentic harness in there, the model, the inference software, the cluster management software, the GPUs that it's running on, and all of that is sort of like, uh, amortized into a token, and you're renting a piece of that whole thing. When you want to own your intelligence for a number of reasons, uh, and I liken it to we start typically off renting, renting our apartments-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... uh, but then as you grow up, you wanna own a house for various reasons, even if it's a bit of a headache, right? Like you d-
- SGSarah Guo
Yeah
- MLMisha Laskin
... like it i- I mean-
- EGElad Gil
[chuckles]
- MLMisha Laskin
... uh, it can be. [chuckles]
- SGSarah Guo
Yeah.
- MLMisha Laskin
Uh, but, you know, as you get to a point where, you know, you're financially mature enough and you have a family and so forth, you wanna, you wanna own it. And, uh, AI has matured as a commercial, like i-in a commercial market, to the point where enterprises are spending a lot of money on renting their intelligence and then wanna start owning it for various, various, you know, reasons around control.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
And so going back to monetization, like what's the difference between a closed and an open model? Well, the mo-the open model is, you know, permissive, but in order for, you know, in order for a customer to make use of it, they need all the other stuff that went around it. Like the stuff that you kind of take for granted from a closed model You need the
- 24:28 – 30:54
Future of Open Versus Closed Tokens
- MLMisha Laskin
cluster management software, you need the inference software, you need the harness, and so forth. Um, and so we provide the, you know, to large enterprises, sovereigns, um, all the tools that they need to make kind of open model deployment successful. And even when we think about services is because a lot of, uh, enterprises, most enterprises need some hand-holding on this. We think about services as how do we go in, unlock really valuable use cases that drive a lot of, um, compute demand, right, inference demand, and in a sense, services in open model is like a demand driver for-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... uh, for a much, you know, for, for an inference business.
- EGElad Gil
What do you think, um, I won't ask you to project too far because it is just very hard, but over a one or two-year time horizon, the mixes of tokens that is open versus closed?
- MLMisha Laskin
Well, I think you're kind of starting to see the trajectory now, which is that, uh, maybe six months ago, it was majority closed, uh, minority open when you go to any gateway like Open Router or Vercel, and it's flipped almost exactly from 70/30 closed/open to 70/30 open/closed now. Uh, I think that's only going to accelerate. Uh, and I suspect that the world is gonna look not too dissimilar from operating systems where 95%-plus of servers, computers in the world run on an open-source operating system like Linux. That doesn't mean that the, the closed stuff is very valuable. Right. Microsoft is, and Apple, these are very extremely valuable companies, so the, the market for this stuff is, is really big. So I think that, um, we'll see most token demand going to open. A big difference here is that, uh, I think we'll also see a lot of, um, economic value going to open simply because, you know, even if the model is o- fully permissive or perhaps has a license on it, um, everything else you need to run it is still expensive. The compute is still expensive, so it's a... You're in a different world than, um, you know, hardware accelerators like GPUs and others are, are just-- It's a more expensive compute substrate than, uh, CPUs, right? And so you have a different kind of cloud that, that needs to be, uh, built around it. So I expect majority of tokens to be going to open source, uh, extremely valuable closed model companies to, to exist, um, a valuable ecosystem of open model companies and various distributors around them, um, to, to exist as well.
- SGSarah Guo
Do you have a sense of what proportion of the open tokens are, are RL'ed versus not, versus sort of base model?
- MLMisha Laskin
I think every single, like, open model that's, um, out there at this point has been RL'ed to some extent.
- SGSarah Guo
Oh, yeah. But I mean, for a specific use case or application.
- MLMisha Laskin
Yeah.
- EGElad Gil
In a, in a custom model-
- MLMisha Laskin
Yeah. So I think that-
- EGElad Gil
... as people term it today
- MLMisha Laskin
... what I've heard, kind of the statistics that, you know, some of, uh, like the kind of open model dedicated inference providers, uh, say, is that it's, uh, actually vast majority, 90%-plus, um, customized. But there's a caveat there because today the biggest customers of dedicated inference are AI natives-
- EGElad Gil
Mm-hmm
- MLMisha Laskin
... and maybe digital natives who have a whole product that is just, you know, one big customization of an open model. Like-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... you know, like a cursor or, um, you know, cognition, Abridge, Harvey, like these kinds of companies that have a product that's built around something that's big and customized. Um, so that would make sense then. Well, if you're supporting those workloads, the majority of them would be, um, fine-tuned. I actually have a different take of what's going to happen in enterprise. Um, I think that the majority of enterprise token consumption is going to come not from customized models, but from customized systems. Meaning you took an open model, you didn't actually fine-tune it yet, you just customized a system around it, like a, your agentic harness around it to make it work for some KYC flow or something like that. And then once you're sophisticated enough, you might start, you know, fine-tuning and moving to that. But I, I actually think that in, in enterprise it will be, it, it, it will be kind of reversed in the sense that the journey that AI natives went through, which was start with a closed model, then move to an open model, then customize it, they went through it very quickly-
- EGElad Gil
Mm-hmm
- MLMisha Laskin
... um, because they set up native products that are data collectors. Um, enterprises are mostly architecting this around their existing products and existing tools, and I think that there are a lot of frictions there to just do a straight shot to fine-tuning. So in enterprise, I suspect it'll, it'll be a bit different and-
- SGSarah Guo
And do you think they'll still start with state-of-the-art models or the, you know, the closed models, and then they'll move over, but the harness will be the mechanism by which they customize the open model?
- MLMisha Laskin
That has been our observation, that the point at which an enterprise is considering open models, and a lot of... and most of the enterprise we talk to are considering now, is, uh, by the time they've ramped up some significant workloads with closed models, uh, and right, it's, uh, I have not seen kind of a basically s- straight shot kind of sprint to an ownership market, but it's you have to go through the rental stage to become an owner.
- SGSarah Guo
It has to get a big enough cost driver for your enterprise, and then you say, "How do I cut costs? And this is valuable enough for me to keep it, and so then I switch over as effectively-"
- MLMisha Laskin
Yeah. If you're spending, let's say $100 million-plus, which is not uncommon at all on closed models a year, then you start thinking about, well, you know, maybe I should figure out like a more optimal way to do this. And, um, I think there's an interesting resurgence of, um, on-prem in the sense of, uh, also because there's a compute shortage. Uh, and so it sometimes it becomes... If you're an enterprise and you're consuming from your favorite hyperscaler, you know, someti- very often there's just not enough compute. It's hard to get good rates there. Uh, and so... But you're spending so much money, so you might go with like a, you know, an infrastructure provider like Dell and say, um, "I just wanna set up the bare metal." But I want to serve stuff into, into my enterprise, and then well, what happens then? Who helps them with that, with that layer between the bare metal and actually making them successful?
- SGSarah Guo
And you view that as your role in the open model world.
- MLMisha Laskin
Yes.
- SGSarah Guo
Yeah.
- MLMisha Laskin
Yeah. So I mean, we view our role as enabling enterprises to build successful solutions, so it is very- it is solutions-driven in the sense that the thing that enabled, uh, AI natives to adopt, uh, open
- 30:54 – 34:01
Competition with Open Models
- MLMisha Laskin
models well is building their own solution. But enterprises just need more, more, um, help there. Uh, and so it's really around going in, assessing what are the biggest value things you can do there, and a lot of these enterprises can go... Like, they might have built, you know, an agent that spans, you know, hundreds, hundreds of millions of customers, uh, on closed models that they really wanna scale out. Um, and so the, the scaling factor is very high. It's very hard to go in and just say, "We're gonna, uh, gonna give you inference." I, I've, I've not heard that wor- that, that has not worked, you know? So-
- SGSarah Guo
Yeah. Yeah. Yeah.
- EGElad Gil
The closed model ecosystem is a competitive one. It looks increasingly like the open model ecosystem will also be. What do you... I mean, tell me if you disagree with that-
- MLMisha Laskin
No, I agree
- EGElad Gil
... claim as well.
- MLMisha Laskin
It's, it's all very competitive.
- EGElad Gil
[chuckles] Yeah. Uh, I mean, large markets tend to be. Um, what, what do you think are the important dimensions of competition when you think about yourselves and, like, how you position in the long run?
- MLMisha Laskin
Yeah, well, I, I have kind of a, a pretty simple formula for, you know, ultimately the way anyone wins in this is by well, it's how much revenue, right, are you- how much durable revenue are you generating? And the formula is, um, what intelligence density are you able to offer times how much compute do you have times how much trust do you have with organizations that they would wanna work with you? Basically, how, how good are you at solving their problems? And because something that's interesting about this market that maybe you all know more, more than I do, I actually, you know, previous markets in terms of what were limiting factors, but compute is scarce, so there are, you know, a lot of things that previously were... And, and maybe this is a temporary thing, but I don't see this as being a temporary thing for some time. Uh, so in, in, in previous, um, I don't know, competitive instances, you, you might say, "Oh, there's like a, you know, these few players are all, like, undifferentiated," but then it turns out if you're really good and you have the compute, that it's okay. [chuckles]
- SGSarah Guo
[chuckles] Yeah.
- MLMisha Laskin
I, I mean, I see the whole space being competitive across the whole stack, and, uh, you know, th- there are arguments around well, this part is getting commoditized and that part is getting commoditized. In reality, everything is getting commoditized. Like, everything across the whole stack is so hypercompetitive that it's getting commoditized. And the model margins are gonna be compressed, right? There is an open model tension, right, with, uh, with closed models that does, I think, does lead to a, um, a margin compression.
- EGElad Gil
Mm-hmm.
- MLMisha Laskin
The application stuff, margins compress. The, you know, inference layer and the bare metal. So-
- EGElad Gil
We see a lot of the AI native companies negotiate their closed deals-
- MLMisha Laskin
Yeah
- EGElad Gil
... very aggressively because the open ecosystem ex- exists, so it's explicitly true. Yeah.
- MLMisha Laskin
So I think that for any one of these companies, and you can think about any even AI native company, as what is the intelligence density that they serve to customers? How many- how much compute do they have, and how much trust do they have with those customers because they're driving-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... successful solutions for them? Uh, which means that if, if those are the three factors, then, um, so
- 34:01 – 38:03
Chinese Open Source Model Ecosystem
- MLMisha Laskin
long as there are great open models, so so long as you have great intelligence density out there that is accessible, there should be many successful companies, um, because then it's a matter of how much compute can each one, you know, assemble. Now, we are actually in a world where when you have- when you're a great kind of intelligence builder, um, and you're able to... And this, this is kind of right what's happened with the closed companies, that your revenue really shoots up and, um, you're just able to amass more compute than anyone else, right? Then that puts you in a strategic, um, position. Um, but that is what I, I, I think is required to win. You have to, right, serve great intelligence, have a lot of compute, and, and have trusted customers.
- SGSarah Guo
Yeah. How, how do you think about, um... So if, if you look at the era that you mentioned at the very beginning of this conversation, you know, two years ago, a lot of the open-weights or open source models were, uh, Western in origin, so Mistral was developed in Europe. Obviously, Meta was a mix of Europe and the US. And what's happened over the last year or two is we've really seen the rise of, uh, Chinese open-weight models. In some cases, there's, um, the perspective that they've been distilling a lot off of the state-of-the-art models, and so that's part of what's allowed them to make very rapid progress. Um, the labs are now trying to now roll out, or the big sort of closed labs are trying to roll out, uh, tools to prevent as much distillation from happening or at least making it more challenging. What do you think happens over the next year or two in terms of Chinese open source or Chinese open-weight models? Do you think it remains where it's at? Do you think it evolves? Like, what happens?
- MLMisha Laskin
So I, I think the first thing is that the fact that great, a great open model ecosystem came from China is actually a massive benefit for the world. Like, the- there are so many companies in the West, right, that have been able to build more durable businesses as a result of that. So it's a, uh, there is a sense in which it's a, right, it's a very positive thing and a gift because you could imagine a world where such a thing didn't happen, and then there are just no open models, right? And there's a kind of a tall poppy syndrome that happens when if you're a application builder and you build something that's great, well, then all- it gets subsumed into, right, a closed model provider because, well, then you gen- it's a business, it needs to generate revenue. So I, I view it as positively. Um, the thing that is- the thing is that you don't want a world that is, um, kind of, uh, m- like monopolar in any given way, right? That you either have all sort of, uh, resources and compute concentrating across, uh, you know, um, closed labs or, like, a handful. Let's say if, if it was... I think if it was, like, 10, 20 closed labs, then fine, you have a competitive ecosystem, but it's one or two, that's a bit scarier. And similarly- You don't want, um, the source of intelligence that everyone else can build on when they wanna own it to be coming from one country. Ideally, it'd be also 10 or 20, but the capital expenditures are very high, so at least two would be great. And so I think it's, um, you know, extremely important that there's a ecosystem, a pa- like an ecosystem that competes with China in the West, but it's not, like, really a, like, West versus China thing. It's more of a, um... If there was, like, one country other than China where all the great open models are coming from, you'd probably also want some competitive tension. Competitive tension is just good. Uh, I think that the Chinese models will continue to be great. Um, they--there is an advantage that they have in, um... They have a number of advantages and a number of disadvantages, but the advantages are they certainly distill closed models at, you know, at industrial scale. That is, that is true. Um, they, you know, have access to, um, data that is, uh, much cheaper and, you know, free because you can, you, you can just train on, you know, PDFs that are copyrighted in China and it's okay. Um, just different regulation. Um, and I think that, uh... And there is a notion in which those companies do have some state support, like whether directly or indirectly for, um,
- 38:03 – 44:07
Will Chinese Models Remain Open
- MLMisha Laskin
you know, like, th-these, you know, these companies were not making any revenue for a long time.
- SGSarah Guo
Mm-hmm. Yeah, I always thought the Chinese open source was basically a subsidy by the Chinese government to US enterprise or to Western enterprise in terms of-
- MLMisha Laskin
Yeah
- SGSarah Guo
... if you actually looked at what was happening.
- MLMisha Laskin
Yeah, exactly. So and I think that continues. I mean, now these businesses are, like, uh, these companies are turning into businesses, um, and, you know, they're--while the models are open abroad, right, they are, uh, within China, they're kind of the equivalents of the closed model labs here. Um, uh, so, like, there are real businesses that are being built there, and I think that they'll continue building great models.
- SGSarah Guo
But do you think they're gonna keep them open here? In other words, why, why do that if you now have an economic driver and pattern? What do you think is the incentive for them to keep the models open in the West?
- MLMisha Laskin
In the West or in the-
- SGSarah Guo
In the West.
- MLMisha Laskin
Yeah. This is a, it's a really interesting question, right? I mean, so fundamentally, right, there's, um, there is a lot of demand from enterprise public sector sovereign for, um, Western open models for a number of reasons. Um, but there are kind of regulatory fears or uncertainty rather, and it, and it is the case that it's, you know, not, not just... You, you want not just a model, you know, but a partner that will help you kind of serve that model. So we have seen that there are, you know, most are Fortune 500. The foot--the open model footprint is actually pretty low today because, um, because of, uh, sort of resistance or aversion to Chinese models for a number of reasons that are both some are rational and some are irrational. Uh, and so it's very clear that building those things is important. And so if you can sustain an economic engine that builds... Well, one, builds a lot of trust that, you know, the enterprise wants to work with you as the model builder. Um, and, you know, two, enables you to secure compute, um, then, right, if there's a commercial engine around this stuff, then, uh-
- SGSarah Guo
Yes, I totally under-
- MLMisha Laskin
There's an incentive
- SGSarah Guo
... I totally agree that, um, having a Western open weights model that provides incremental services and potentially inference or other things over time is really valuable, and I know that that's the direction you all are heading in. I'm just a little bit curious in terms of what is the incentive system to keep the Chine- for the Chinese- To allow- ... model companies to continue- Models ... to leave the, the models open. Um-
- MLMisha Laskin
So the question is on the Chinese side.
- SGSarah Guo
Correct.
- MLMisha Laskin
But, but yeah.
- SGSarah Guo
Yeah.
- MLMisha Laskin
So I think there it's actually that my, my, my, um, hypothesis on that is that, uh, there's Ch- China's somewhat of a captive market that, you know, like you can build open models and still make a lot of money in that market.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
Like something that's interesting is that, right, in, uh, in the West, there are plenty of... I mean, there are great companies that, um, don't build open models, but serve them, right, into various companies. Um, I'm not really aware of that dynamic happening in China as much.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
You'd think it would because that's the... right? Like why, why not? So there's something interesting there where the, the open model providers are kind of the intelligence providers in that country.
- SGSarah Guo
Mm-hmm. Mm-hmm.
- MLMisha Laskin
And I think that the continuing to release great open models is very geopolitically advantageous to China-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... for a number of reasons, but one is that you want, you know, as a country, and this is for America and China and any other country that has the capabilities, that you want other countries building on your stuff. There have been previous escalations or tensions between the US and China in, you know, 5G fiber layout-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... um, and the Belt and Road Initiative. Uh, ultimately, right, when you go in and you provide something cheap to another country, it then, you know, gets locked into your infrastructure.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
There are, there are just commercial and geopolitical advantages for the infrastructure side.
- SGSarah Guo
So I think it's almost like rare earth minerals or something. It's you have a component that other people will use for all sorts of purposes, and therefore you have geopolitical leverage through that.
- 44:07 – 52:25
Safety and Open Models
- MLMisha Laskin
source, the world basically, right, like is American currency is like the widely established global currency. Um, free trade works because, you know, the, we have a navy that's everywhere. So that's been actually the American playbook for a long time. It's interesting that in this case, uh, I think the United States has been on the back foot when it comes to open source, but-
- SGSarah Guo
No longer.
- MLMisha Laskin
Yeah. Well, I think in eco- like we, we have to be clear, it's like, uh, the competitors in China are just really, really good. And so there's an ecosystem that is just starting to form now in the United States. But there is, you know, there is some catch-up to be had.
- SGSarah Guo
The biggest, um, criticism from the, uh, closed labs of, uh, open, open models tends to be safety and controllability. What's your philosophy on this? And is that like-
- MLMisha Laskin
Yeah
- SGSarah Guo
... a legitimate concern?
- MLMisha Laskin
Um, no, it's definitely a legitimate concern and I think that... So I do think that the safety worldview that has been established has been established, you know, from one particular perspective with one particular point of view that has become almost kind of, um, you know, kind of dogmatic. Like if we were looking at from first principles at AI before, you know, considering closed or open or what have you, this technology is getting built and you kind of had the foresight to know that, you know, it's gonna have these capabilities and so forth, and you are thinking about safety, right? How would you actually design it from scratch? Um, and there are... I mean, one way, one analogy to take it to is, well, software, you can kind of think about AI as a more advanced version of software. It has, you know, many similarities actually. It's like it is a digital, uh, utility. Um, and there were debates, by the way, around software being dangerous, uh, particularly in the early 1990s around strong encryption protocols. You know, like they were actually closed at the time and there were debates between the NSA and, uh, kind of civil, uh, liberty advocates around whether you should close it or open it, and, um, ultimately the decision after some, um, catastrophic failures on the closed side where effectively a small handful of engineers designed certain systems that had unintended consequences that they couldn't predict and got easily hacked effectively, um, that strong encryption protocols became open and that actually gave birth to the whole field of cybersecurity. So the way the world, the default state of the world is actually its openness is safety. That's, that's a default state that we'd be going into AI with if we were just thinking about continuation of technology. Um, and models actually have a lot of similarities, but kind of exacerbated, uh, to software in that the, the long tail of vulnerabilities is larger. Like you, it's, it's harder to understand than software because it's a black box and the long tail of vulnerabilities, and hence unintended consequences is larger. And Linus's law, like fou- fou- creator of Linux, is that with enough eyeballs, all bugs become shallow.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
And I have the belief that with enough eyeballs, um, most security and safety vulnerabilities become shallow as well.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
So like the state of the world today is that we have a few hundred safety researchers within closed labs that understand how these things work, and despite their best intentions, it is impossible to cover the long tail of vulnerabilities or unintended consequences that these systems might have. Which is why the symptom of this is that when we actually look at what major cyber, you know, security issues have surfaced, well, it's that a very powerful closed model had unintended consequences where it went and like hacked into another company.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
And the only way that company could remediate itself was by using open models to protect itself.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
Right? So that's the empirical evidence of the world that we're in.
- SGSarah Guo
When you say safety, by the way, 'cause I feel like people really conflate notions of safety, and safety means three or four different things to different people. There's safety in terms of cyber attacks or, you know, uh, the use of AI for hacking or other things. There's safety and, you know, I think this tends to be overstated in the short run around bioweaponry or, you know, terrorism and then there's sort of safety from the perspective of an existential threat to humanity. Um, and again, I, I feel like people kind of talk about these things as if they're one thing-
- MLMisha Laskin
Yeah
- SGSarah Guo
... and each one of these are sort of separable. So when you're talking about safety, are you addressing all three of those? Do you mainly mean... Uh, and by the way-
- MLMisha Laskin
Yeah
- SGSarah Guo
... I don't necessarily agree that these are all like true things that are gonna happen in any realistic timeframe. It's more just I'm, I'm a little bit curious how you think about the, the span of things that benefit from openness and benefit from these approaches.
- MLMisha Laskin
Yeah, I think on the, on the safety spectrum, there's basically a spectrum of reality to like empirical reality to theoretical, you know, scenarios, and I will grant that s- that AI There are sci-fi bits to it where something that was totally a theoretical thing last year is like a reality, which is like the cyber capabilities of these models, for example. So it's not to discount the theoretical stuff, but we have to be clear that there's, you know, real empirical stuff that's happening and that we can predict, you know, with some, you know, confidence will be happening in the next six months. And then there is like, uh, you know, maximal theoretical stuff.
- SGSarah Guo
Yeah, yeah. I mean, the, the prior versions of that, for example, would be the thought that the atmosphere would catch fire the first time we set off a nuclear weapon.
- MLMisha Laskin
Yes.
- SGSarah Guo
Right? So there's that theoretical fear. It didn't happen, uh, but there was a lot of churn amongst a small subset of researchers in the physics community around that as an example. So there, there, there's been a lot of these theoretical things that could happen. I remember there's also... When people talk about nanotech, they used to talk about gray goop and how you'd accidentally release a nanobot, and then suddenly it would dis- it would eat the entire world. Like, this was something that was discussed in the '90s in-
- MLMisha Laskin
Yeah
- SGSarah Guo
... in labs as people were, were working on, you know, microfluidics- [laughs]
- MLMisha Laskin
Yeah, exactly
- SGSarah Guo
... and things like that, so.
- MLMisha Laskin
The challenge with AI is that it's so top of mind, and it's permeated kind of everyday culture, and it's so easy to humanize that, uh, it's, it's kind of, I think, th- there... That it, it feels like there's a, from a perception perspective, almost like a not even uniform distribution around what safety means, but it's actually like peaked at the doomsday scenarios. Whereas the reality is that there's probably like a... The service will peak to address is like the reality, and then there's a decay into, into the theoretical. Um, but it doesn't help when, you know, when leaders of companies say like on the theoretical side, there's a ten percent chance that we all die, you know? [laughs] Uh, that, that, that doesn't help.
- SGSarah Guo
Yeah. That's not substantiated by any specific-
- 52:25 – 56:39
Debating Access to Powerful Tools
- MLMisha Laskin
the reality with these alignment, you know, issues, that they're probably gonna be very mundane and boring, where you've got a, you know, you've discovered a bunch of these vulnerabilities. You have data that patches them up. You have some a- algorithms or some... And when you say algorithms for detecting them, it's typically asking a language model to detect this thing.
- EGElad Gil
Yeah, it's a classic. [laughs]
- SGSarah Guo
Yeah.
- MLMisha Laskin
So there's the theoretical kind of, uh, alignment safety philosophical stuff, and then the mundane reality that you're just like patching up all sorts of bugs. And then it becomes a, well, shouldn't there be 100,000 like researchers and computer scientists looking and patching up these bugs? Wouldn't that be safer?
- EGElad Gil
I feel like there's a sort of important philosophical distinction, which it sounds like you're on one side of, but I'll let you be explicit about it, which is just, you know, even if they are very intelligent tools and if you just say we can man- as, as an ecosystem, manage the unintended consequences, we don't want individuals and businesses to have such powerful tools.
- MLMisha Laskin
That is kind of coming from a standpoint that, um, we can have access to powerful tools, but, you know-
- EGElad Gil
Yes
- MLMisha Laskin
... but other people can't because, you know, we're, we're, uh, effectively better, right? Like we're better at managing these tools. And, uh, well, it's kind of-
- EGElad Gil
To be fair to the claim, I think... This is not my belief. But I, uh, to be fair to the claim, it would be like the negative impact that any single person or company could have is like too large, and that's scary. But we can have them.
- MLMisha Laskin
Yeah.
- EGElad Gil
Yeah.
- MLMisha Laskin
Well, I mean, it's, uh, also like on a scale of like, uh, you know, weapons, right? Like that's a, um, s- you know, we have like the, the, the US government sells like jets into other countries, right? Uh, but I think that more broadly, uh, it's not so easy to take a model, especially a model that's been, um, even like an open model that's been, uh, you know, trained to be safe and, you know, like, uh, put in the compute and work and effort to, um, to make it very dangerous. Um, now I'm not saying it's impossible, but it's just like, uh, again, when we think about cybersecurity, um, very intelligent hackers can go take and wreak havoc, um, as they have. Uh, but it's you have an ecosystem where the defenses tend to outweigh the offenses. And again, like if we come to that as a default, that that's the way, you know, people secure software and that there are- Positive and bad actors on the internet, and ultimately the internet is an ecosystem where it's, it's kind of like having, you know, an ecosystem of white blood cells.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
And enabling more people to have the defensive capabilities actually helps you protect against the offensive capabilities. That's... Right, the... When we look at, again, the empirical reality of what happened is that it's really hard to separate cyber defense from offense. So when you remove cyber offensive capabilities, you also remove cyber defensive capabilities, and as a result, uh, the players who would wanna help defending are, are incapable of doing so.
- SGSarah Guo
And we saw that in some of the recent, um, things that happened with, uh, OpenAI Hugging Face incident where they reverted to using open source-
- MLMisha Laskin
Right
- SGSarah Guo
... uh, because they weren't able to use the existing state-of-the-art labs because of the guardrails-
- MLMisha Laskin
Right
- SGSarah Guo
... that were placed on the models, so.
- MLMisha Laskin
Yeah, I think that these, you know, arguments go into a place where they're very absolute, where you kind of say it, you know, these things are so powerful that no one ever should have access to them, right, except for us.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
And the reality is that that's never, you know... It's hard to, you know, except for, like, some really rare exceptions where there are actually not that many positives, where, uh, s- you know, where such absolutist perspectives worked. Um, and usually it's kind of the, the opposite. Like, one thing that... So I, I was born, um, in the last year of, uh, Soviet Russia's, uh, or so the Soviet Union's existence, uh, in Leningrad, um, and then a year later it became St. Petersburg.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
Uh, and, you know, and then we immigrated. Uh, but s- in the Communist Manifesto, in Lenin's Communist Manifesto, there's a statement around, "Hey,
- 56:39 – 59:59
AI and Scientific Progress
- MLMisha Laskin
we're building this, like, socialist state that's gonna benefit everyone, but we do need this temporary state of dictatorship where, you know, everything is centralized, we control everything. But the good news is that once the benefits of what we do are, like, evenly distributed, you won't even need us anymore."
- SGSarah Guo
Uh-huh.
- MLMisha Laskin
"Right? You won't even need the state anymore." So the, like, that is kind of what that thinking reminds me of is that, listen, we're gonna, we're gonna take care of everyone. Like, it's gonna be... Like, uh, there, there is gonna be a period where everything needs to be concentrated around us, but that's a safer, better thing. And at some point, the benefits are just gonna be so widely distributed that it won't even matter anymore. Um, the parallel to me is pretty striking.
- SGSarah Guo
What are you most excited about if you think ahead two to five years in the AI world? It's very hard to predict right now, but as you think ahead in terms of the curve of technology, the curve of adoption, uh, what do you think is the, some of the positive things that you view as coming?
- MLMisha Laskin
Like, there's so many positive things that are coming, and they have been coming, and an example is that I, I'm personally very excited about scientific progress. Uh, as... That's, you know, got into science as, you know, when we moved to the States, uh, got in- developed interest in physics and always kind of had this, like, lifelong pursuit of science since then. And one of the prompts that I was trying with language models over the last few years, uh, is giving it my PhD thesis, which granted a lot of, um, getting to the point where you have the right question of the PhD thesis is kind of the... That's a lot of the work. But then actually executing the rote work takes a really long time as well, and overall between finding the right question and executing the answer, right, a PhD, I think took a few years. Um, and so I was asking language models, uh, over the last couple of years, like, the, giving this prompt of my thesis. Um, couple years ago, well, it was just chat, so it couldn't do anything. Then a year ago, it started answering things I would say at an undergraduate level. Um, like that would've been my answer if this was, like, given to me as, like, a homework in undergraduate. Six months ago, um, it solved it. [laughs]
- SGSarah Guo
[laughs] Yeah.
- MLMisha Laskin
Right? At, at, like, at real PhD level, and, uh, solved it correctly, and now when I actually try it, it actually even gives me some, like, interesting, like, new information that, um, you know, I hadn't considered at the time. And so we've gone... So now you can-
- SGSarah Guo
Yeah
- MLMisha Laskin
... like, if you have the right question, you can just put it into, you know, a chat box and have it do, like, all the calculation and give you something really interesting.
- SGSarah Guo
That's so exciting.
- MLMisha Laskin
Right? Right. Which means that your iteration speed, like, why do you need a few years? Like, what's the point of a few-year PhD? You can, you can do a PhD a week, right? [laughs]
- SGSarah Guo
Yeah, similar to the recent news from OpenAI, I think it was the last day or two, in terms of all the various, uh, mathematical, uh, theorems that were proved in the last, you know, uh, couple weeks, uh, by OpenAI. So yeah, it's kinda crazy what's been happening.
- MLMisha Laskin
No, it's, uh, it's absolutely incredible, uh, what this is do- what this is gonna do for science. Um, I think that the other stuff I'm really excited about is, uh, you know, not just theoretical science where the environment is basically a, you know, a... well, what would be equivalent of a chalkboard or something. Um, but real world science, uh, life sciences, um,
- 59:59 – 1:01:32
Data Centers and Jobs
- MLMisha Laskin
material science, chemistry, I think there are a lot of places where you can have an AI interfacing with a real experiment, um, where you can set up proprietary data flywheels, and it's gonna be a bit slower, well, much slower than the chalkboard stuff, but, um, dramatically faster than what it was before. So, uh, the... I think we, like, kind of underestimate, continue to underestimate how dramatic of a shift this has been for software engineering and how much more you can build. Um, it's, um... I think it's an already very incredible and, and positive tool, um, in many ways. Also at the, at the blue collar, uh, kind of level as well, um, I had the, um... You know, I, I toured, uh, Stargate some weeks back, and the thing that struck me was the amount of, um, cars in the parking lots. There were so many people working there, [laughs] right? So these, like, data center projects create Tens of thousands of, of jobs, uh, that like are, are high paying, high paying jobs. Like to me, it kind of seems like data center now is kind of what a factory used to be, right? You have these like railroads and factories in the 20th century, um, like you had, you know, maybe like a shoe factory that, uh, makes a town, you know, produce kind of a, you know, vibrant economy within a town. And, uh, that is, at least when like I'm on the ground seeing like the stuff that's getting built, um, there is, there
- 1:01:32 – 1:05:03
Beam and Scientific Research
- MLMisha Laskin
is fear of data centers, but then the reality is you go on the ground and like it actually creates a lot of jobs, a lot of tax revenue for the local communities. Um, and while there are certain things that I think are, need to be done very tactfully, like these things are noisy, so you need to kind of, uh, insulate, you know, insulate the noise or put them in places that are less like close to residential. But it is certainly the case that, uh, for local economies, um, these things do create jobs, they generate tax, tax revenues, um, and just feel like what a factory used to be in terms of job creation.
- SGSarah Guo
How do you direct like the scientific and experimental effort now, like from, from Beam going forward? And, and within that, assuming that you are trying to use the models themselves to, um, improve your training effort, like what is the role of the science team today?
- MLMisha Laskin
The way we direct our, our research, and a lot of this, you know, falls under, uh, you know, my co-founder Yannis, who, uh, leads the research, research and technology, um, among other things. Um, it's, you know, it's really, um, can-- You know, there, there are a lot of things that we know work that, uh, you know, we weren't able to put in. It's just a timeline thing. Um, you know, when you-- We, we trained, so we, we, we trained the final model on, um, you know, the SpaceX cluster that we received in, in July. And so it was, uh, you know, a few weeks of pre-training, then, you know, a few weeks of, you know, synthetic data on our own, then shipping it. And so there's, uh, uh, a lot of stuff that didn't make it in that, uh, we're really excited about. And so, uh, right, there's a trade-off of, um, the careful execution of kind of known things or bets that have a bit less risk around them, and then the, uh, right, going on to kind of more risky bets. And so you kind of want to exhaust all the high impact things that you know work, um, and then start expanding, uh, some of the risk profile. Uh, and, and right, and the way you do this is that you, uh, on the risk profile, you, um, you have scaling experiments where you do stuff at small scale. Um, I think the interesting bit is that some interesting things don't emerge at all until you're at large scale, so there's a bit of an art and science of, you know, am I going to put in more compute to find out the next thing or go on to the next idea? Uh, as far as model, model-in-the-loop goes, uh, I find it to be very empowering for, for researchers because, uh, again, things like, uh, things that would have taken a long time individually now can be just done fairly, fairly quickly. So, uh, these models are fairly jagged in their intelligence, so you have to kind of feel like where they're, where, where they have kind of a tight loop and where there's, like human creativity is still important. But you kinda, you get a feel for it pretty quickly, and so the place where you can loop things, uh, you, you do that.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
Uh, and that, uh, especially, I mean, there are all sorts of things around hyperparameter searches and, uh, some of the more, uh, like granular things on infrastructure that these things can be helpful in. Um, but there's still-- I, I actually think they're very exciting tools because they enable a scientist to exercise their, their intuition. Uh, right? It's kind of like having a very fast and eager colleague that actually listens to you.
- SGSarah Guo
Mm.
- MLMisha Laskin
And, and if it's aligned, if the colleague is aligned correctly.
- SGSarah Guo
[laughs]
- 1:05:03 – 1:10:18
Research Head Count to Compute Ratio
- MLMisha Laskin
Uh, and, and so I, I see a research- researcher as just getting a lot more, um, like ability to move a lot faster than they have before, which is like why maybe the rate of improvement, you know, uh, annual was roughly, let's say seven X before when it was just, when it was just manual work, and now several times that, um, say maybe four to five X. Uh, and so it's actually really exciting for a researcher to have these tools. Um, there's this question around at what point do you not need a researcher in the loop?
- SGSarah Guo
I was gonna ask you, if you need one hundred people today, do you need twenty next year for the same effort?
- MLMisha Laskin
For the same effort, yes. But I think the, it- it's, uh-- But you're constantly expanding the ambition of what you can do. And so we, we find ourselves, uh, you know, constantly, you know, understaffed rather than, uh, than over. But there is actually-- I mean, there, there, there's basically a staff to sort of compute ratio that you need to keep in mind, so it's not like it just goes on forever. Like there isn't-- I think three thousand researchers would not be very helpful, but you know, a couple of hundred, very, very helpful.
- SGSarah Guo
Yeah. The argument I've heard is that given the amount of compute you need to allocate per person, and especially to your point, if, if you wanna scale certain experiments up over time, uh, and then coupled to that, often there's a bit of a, a distribution in terms of contributions. In other words, there's a handful of people who contribute the most from an idea perspective in every field, right?
- MLMisha Laskin
Yeah.
- SGSarah Guo
Physics, biology, whatever, eventually collapses into you wanna give every incremental piece of compute to the most productive subset of people, and so therefore you end up with some static or shrinking number of researchers over time. That'd be kind of the argument in the extreme in terms of where you end up. I'm not saying it's correct.
- MLMisha Laskin
Yeah.
- SGSarah Guo
I'm just saying that's kind of an argument I've heard, uh, being made.
- MLMisha Laskin
The argument definitely has merit. So these projects, actually they never needed like a crazy amount of researchers to, to really, you know, when you think about projects like AlphaGo at the time was maybe order of ten people. Uh, and now I think that Maybe for, for large efforts like the one we're doing, maybe an order of 100 or, or hundreds. So it's never been the case that you needed, um, an extraordinary number of people, and the question, you know, whether it collapses, you know, back into, back into 10 or not, it's, it's hard to say. I would estimate that there's gonna be some steady state in that kind of order, order of 100. There are just, um, there are just a lot of things to do, and that there will be... But there will be a lot of job creation in, um, applied research, like going and taking these things and actually applying them to real problems, I think there's going to be-
- SGSarah Guo
I think people really lose that when they talk about engineering and research in terms of the diffusion of those roles out into a broader swath of society and not just building language models.
- MLMisha Laskin
Yeah.
- SGSarah Guo
And so to your point, there's lots of other model types to build, there's lots of closed loop systems to build, but even when people talk about engineers and the potential displacement of engineers, and so far the evidence seems that many companies need more engineers versus fewer, there's also the diffusion of a certain level quality of engineer out to actually deploy the technology in organizations that never would have been able to access somebody-
- MLMisha Laskin
Right
- SGSarah Guo
... of that sort of talent level on a relative basis, or at least not at scale. And so I do think that's kind of lost as people think about the economy. They tend to centralize it into a handful of big tech companies instead of saying, "What is our overall GDP and how does that get transformed by the diffusion of people bringing this technology out?"
- MLMisha Laskin
I, I think that's correct. So the, again, like, the, the amount of researchers you need to do these things, it's not like it's changed dramatically over the years. It's always been, you know, big project, okay, around 100 people. I don't, I don't think you need the thousands on like a, on a big project, um, but maybe on different, you know, product surfaces, okay, then you need a bunch of people. Um, and what's interesting is that when, when you look at the distribution of why do you even need hundreds of people, and a lot of it is also you have teams that do evals for different things and collect data for different things. Like you, you know, when you're, when you're generating... It's not by accident these models have capabilities.
- SGSarah Guo
Mm-hmm.
- MLMisha Laskin
There, there are pods, right, of like five to 10 people that go and target a particular capability, and within coding there are five to 10 capabilities, right? Uh, so but what, what is that? So that's for bringing the general capabilities to the model. Um, but now when you wanna take that model and apply it to real world capabilities, you basically need a pod for any, any real world capability in there, and each enterprise has an, many, many of them. So I think that the job creation for this new type of forward deployed engineer that is a bit more scientific and, uh, evaluations oriented and knows, has some intuition for these models and the harnesses, uh, which is actually what the evaluation researcher does, right? So it's, it's actually the same skill set as the researcher building the core thing-
- SGSarah Guo
Mm-hmm
- MLMisha Laskin
... but just deployed to work on, on real problems. Um, and that is, you know, we'll take as many of those people as possible. Like, that's actually, it's more of a training gap on that. Like, uh, that is, like, a, you need to, you know, train engineers to, to become good at this, uh, than it is, you know, we, we would take as many as possible today.
- EGElad Gil
Congratulations, Misha, uh-
- MLMisha Laskin
Thank you
- EGElad Gil
... having the, uh,
- 1:10:18 – 1:10:46
Conclusion
- EGElad Gil
first, um, Western model that is, you know, in the, in the class of the frontier. It's just been a, a big step forward for the ecosystem.
- MLMisha Laskin
Thank you. Thank you so much.
- SGSarah Guo
Yeah, thanks for joining us. Yeah.
- EGElad Gil
Find us on Twitter @NoPriorsPod. Subscribe to our YouTube channel if you wanna see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.
Episode duration: 1:10:48
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode lx9RjE0GekI
