Skip to content
Superintelligence Needs the Physical World with CEO and Co-Founder of Arda Bob McGrew
This video isn’t embeddableWatch on YouTube →
No PriorsNo Priors

Superintelligence Needs the Physical World with CEO and Co-Founder of Arda Bob McGrew

Is the AI singularity actually coming? Bob McGrew, former Chief Research Officer at OpenAI, argues that superintelligence won’t arrive without changes in the physical world, and that the “hard takeoff” argument is wrong.

Bob McGrewguestSarah GuohostElad Gilhost
Nov 19, 202659mWatch on YouTube

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

Great conversations run long. Their best ideas don't have to.

uListen is free on iPhone, iPad, Android, Chrome, and Claude.

See how

Transcript

  1. 0:00 – 0:05

    Trailer

    1. BM

      If you look around outside, how different is anything from ten years ago before AI happened?

  2. 0:05 – 1:05

    Bob McGrew Introduction

    1. BM

      Not very different at all. The first time you were in your Waymo, you thought to yourself, "This is amazing." Probably the third time you're in your Waymo, you're like, "Man, this traffic sucks." We need to break out of this pure focus on the digital world and go to the physical world. I don't think you are actually gonna be able to get to superintelligence without changes in the physical world.

    2. SG

      [instrumental music] Hi, listeners. Welcome back to No Priors. Today, Elad and I are here with Bob McGrew, the CEO and co-founder of Arda. Bob was previously an early key exec at Palantir and the Chief Research Officer at OpenAI. We talk about making American manufacturing competitive with intelligence, why physical intelligence is necessary for superintelligence, and why the future is going to be machines that can make machines and explore the stars. Welcome, Bob.

    3. EG

      Bob, thanks so much for joining us today on No Priors.

    4. BM

      Great to be here.

    5. EG

      Uh, so you've had a ridiculous career. You were the Chief Research Officer at OpenAI, so you drove all the sort of research directions for the company, or at least set a lot of the agenda.

  3. 1:05 – 2:27

    Why work on the physical world during the AI revolution?

    1. EG

      You were at Palantir quite early and one of the main people there driving product and engineering. So you've had this amazing career spanning some of the most seminal moments of the digital world, and yet now you're working in the physical world. And it's right in the arrow when people are talking about RSI, which is recursive self-improvement, about how we will no longer have researchers and they'll be displaced by code and AI and, you know, AI building itself. Why do anything in the physical world right now as this digital revolution is happening?

    2. BM

      Well, you know, fundamentally, I don't think you are actually gonna be able to get to superintelligence without changes in the physical world. So, you know, the-- We've seen a huge amount of improvement from ChatGPT, from agents. You know, all of this is happening in our digital lives, and very little of that has actually affected the atoms of our lives. Like, if you look around outside, how different is anything from ten years ago before AI happened?

    3. EG

      Mm-hmm.

    4. BM

      You know? Not very different at all. You know, even when you're doing something that is genuinely very different, maybe you're in a self-driving car, right? You're, you know, in a self-driving car, in your Waymo, going around San Francisco. And the first time you were in your Waymo, you thought to yourself, "This is amazing."

    5. EG

      Mm-hmm.

    6. BM

      There's, there's no one driving it.

    7. EG

      Yeah.

    8. BM

      The, like, car is driving itself. And, like, probably the third time you're in your Waymo, you're like, "Traffic."

    9. EG

      Mm-hmm.

    10. BM

      "Man, this traffic sucks."

    11. EG

      Yeah. [laughs]

    12. BM

      Right? Because just... You know, you have something. Even, you know, a Waymo is, like-

    13. EG

      It cannot get outside. Yeah

    14. BM

      ... halfway towards the physical

  4. 2:27 – 5:48

    Why the hard takeoff argument is wrong

    1. BM

      world. But, like, you didn't change the car. Like, the car doesn't fly. You're still bounded by all of the physical barriers that, you know, made the car trips around San Francisco pretty, you know, slow even in the 1990s, right? And so, you know, fundamentally, we need to break out of, you know, this, this pure focus on the digital world and go to the physical world.

    2. EG

      So when you say you can't have superintelligence without changing the physical world, what does that actually mean?

    3. BM

      Yeah.

    4. EG

      What does that substantiate as?

    5. BM

      So, so maybe a good place to start here is I think there's this common perception and-- or there's a common argument, and I've, I've been hearing it since, like, 2018, which says that if you can just automate the work of the researcher, that you're gonna be able to unleash a runaway acceleration, runaway self-improvement loop-

    6. EG

      Mm-hmm

    7. BM

      ... and we're gonna have a singularity.

    8. SG

      Mm, this is the hard takeoff.

    9. BM

      Yeah, hard takeoff, exactly. And I think this is wrong. I think this, this argument is just straightforwardly wrong. So, you know, just to recapitulate the argument, it says, you know, okay, so we can automate 10% of a research time. That is gonna give, you know-- that's gonna give some improvement. We automate 50% of a researcher's time, now you have a 2x improvement, so now you're going 2x faster. And then you automate 90% of a researcher's time, you're going 10x faster. Ninety-nine percent automated, you're going 100x faster. And so by automating more and more of this job, you know, and eliminating this labor of the researcher, you're able to just, you know, zoom up and have runaway acceleration. But this is, this is, this is straightforwardly wrong because researchers are not the bottleneck, right? Like, the amount of time it takes in order for you to discover something is not primarily bounded by the number of researchers you have. Fundamentally, those researchers are running experiments.

    10. EG

      Mm-hmm.

    11. BM

      And so, you know, if you, if you actually think about the, the, you know, even today, right, even two years ago in 2024 when I was Chief Research Officer at OpenAI, like, the actual limiting factor was not the number of researchers we could hire. It was the amount of GPUs that we had on hand.

    12. EG

      Mm-hmm.

    13. BM

      It was the physical amount of compute that we were able to put out there. And, you know, was the compute accumulated in the right places? Was it connected together with the right networks? And so, um, you can, you know, completely automate researchers. This is a big effect, right? I'm not saying this is not a big deal. You know, maybe this is even-- maybe the ability to do automated research is gonna be as impactful as the ability to go from, you know, GPT-4o without reasoning to o3 with reasoning and tools. That was a huge one-time acceleration, but it's not RSI. So I think the, the right way to think about research is that, you know, fundamentally, there's a bunch of experiments. You have to run these experiments. And right now, there's a lot of waste. You have to-- You know, researchers have to do a lot of engineering. They have to pick the right experiments. They have to just, you know, go physically schedule them on the GPUs. And all of these lead to the GPUs being less effective at delivering new discoveries. And so, you know, the correct mental model is to look at the GPUs and to say, "Okay, well, you know, right now the GPUs are 35% of the way to optimal in order to be able to deliver discoveries. And if we can automate more and more, maybe we can get them to 90% of the way to optimal."

    14. EG

      Mm-hmm.

    15. BM

      But the difference between going from 90 to 99%

  5. 5:48 – 13:00

    Labs or startups – who will win in robotics and biology?

    1. BM

      of optimal, it's actually not a big difference. And so the-- we can automate researchers. It's a big one-time benefit, but it doesn't kick in a self-improvement cycle. Instead, it's one more, you know, S curve, one more big one-time improvement similar to reasoning that allows us to- Uh, you know, get closer to optimum-

    2. EG

      Mm-hmm

    3. BM

      ... and then we have to go do something else.

    4. EG

      Yeah, I feel like a lot of, um, AI-driven research is basically stacking S-curves of different types, or AI development, I should say, in general.

    5. BM

      Yeah.

    6. EG

      Uh, that also translates into other domains, though. There's robotics, there's biology. There's a few other places where the physical world is really, uh, you know, the primary limiter on progress, right? You have to run the biology experiment. You have to build the, you know, the motors for the, the sort of mechanical hand or whatever you're building. How do you think about the progress that the labs will drive versus startups will drive in the context of these other big markets where there's a strong research overlay or angle, but then there's sort of this physical world limitation that you have to overcome, or at least optimize against in parallel? Like, will it all... Will robotics and biology aggregate to the Anthropics and OpenAI's and Google's of the world and their subsidiaries, like Isomorphic, or do you think there'll be, like, a bunch of startups that drive this sort of progress?

    7. BM

      Yeah. Before we talk about market structure, I think, like, let's, let's think about what the actual, maybe the economic structure-

    8. EG

      Mm

    9. BM

      ... of this is. You know, the labs, if, if we think about what the labs are doing, they're really producing cheap intelligence. So what are the complements to intelligence? If we think about biology, or again, we think about, you know, running these experiments for superintelligence research, you know, the intelligence in terms of choosing which experiments to run, generating hypotheses, it's really a complement to the physical layer of having the wet labs-

    10. EG

      Mm

    11. BM

      ... of, you know, h- laying out the GPUs. And so, you know, when hypotheses become cheap-

    12. EG

      Mm

    13. BM

      ... then it's actually the physical part that becomes scarce. And so, you know, you know, if, if you're, if you get better and better at generating hypotheses, you're just gonna bang harder and harder against the limited number of wet labs that you have.

    14. EG

      Yeah.

    15. BM

      And so then, okay, so go back to the market structure. Well, you know, I think it could be either way, right? Uh, but then do Anthropic, do OpenAI, do they have a comparative advantage in building out wet labs?

    16. EG

      Mm-hmm.

    17. BM

      I think they probably don't.

    18. EG

      Did you consider doing other, working in other domains before you decided to go work on manufacturing and robotics?

    19. BM

      I did. Yeah, yeah. So when I left OpenAI, I think the one thing I knew was I didn't want to start another frontier lab.

    20. EG

      Mm-hmm.

    21. BM

      Like, in fact, maybe I wanted to be the only person that's ever left OpenAI-

    22. EG

      [laughs]

    23. BM

      ... and not started a new frontier lab. And, you know, I'd done that. And I wanted to f- to do something new. And I was basically, I was, I was captivated by this question that the models are so powerful, right? And they're really powerful now, but they were really powerful then, too. And yet they don't seem to be making that big a difference in the world. And why is that?

    24. EG

      You know, people, people love to talk about abundance as a concept in Silicon Valley. And I think it's a very interesting one because it's, it's a very physical word, right? When you think about abundance, you're like, "I have an abundance of food or housing or family," or something like that. Like, the way we experience the world around us is very physical. And I don't think anybody is like, "I necessarily have an abundance of digital work."

    25. BM

      Yeah. And, and, and, and, and in order to actually get something done, you know, in economics, I think it's called, like, an O-ring production function, but, like, you have to do all the things.

    26. EG

      Mm-hmm.

    27. BM

      And so if we're able to speed up the purely digital process of building housing, uh, well, that's, that turns out not to be very important, right? Actually, the hard part of building housing is getting the permits-

    28. EG

      Yeah

    29. BM

      ... which no amount of agents are gonna help you do, right? But also you have to, you know, do the construction work. Um, you know, I have a, I have a mental model that I think about for what it looks like to, you know, the, the impact of superintelligence, where people say, "Okay, well, if we can just have sufficiently smart agents, then, you know, we're gonna have Dyson spheres in just a few years."

    30. EG

      Mm-hmm.

  6. 13:00 – 16:33

    Why America designs products but doesn’t build them

    1. BM

      of ideas that are, that are actually very good.

    2. SG

      Mm-hmm.

    3. BM

      And just applying a lot more intelligence to figure out, like, is it a good idea or is it not a good idea? Is it gonna help you? You have to go run the experiment. And then maybe you do- were able to find proxies for this, which is great. As we find those proxies, then, you know, then we, we scale up to accomplish that, and then again, we're-

    4. SG

      At the physical

    5. BM

      ... banging hard on the physical world.

    6. SG

      Tell us more about what Arda is doing then specifically. Like, how is Arda approaching the physical world? What are some of the products or areas that you're focused on?

    7. BM

      Yeah. So, so let me start by, by talking about the problem we're trying to solve. So after I left OpenAI, um, you know, I, I sort of noticed that we are very good in America at designing products.

    8. SG

      Mm-hmm.

    9. BM

      Right? So the iPhone, you know, famously designed in California, manufactured in China. It says that on the back of your iPhone. Um, you know, we're amazing at designing defense materials, uh, but we don't manufacture very much of it here. And so I went around, spent a year talking to manufacturing leaders and asking them, like, "What's going on here?" And, you know, fundamentally, it comes down to, fundamentally it comes down to labor costs.

    10. SG

      Mm-hmm.

    11. BM

      And so if you wanna build something in America, one possibility is that you're in defense, in which case you can charge whatever you want, and if you're, if it works, the government will buy it. So that's great. Um, but if you're not in defense, then you need to build an automated line if you want to be able to build something cost competitively in the US. We're building all, we're designing all these innovative products. One part of designing an innovative product is that the product itself changes.

    12. SG

      Mm-hmm.

    13. BM

      And so famously, you know, each year the iPhone 70, has 75% of the parts in it change. Only about 25% stay the same. So that 25% is a highly automated line. The other 75% is assembled by hand. And the reason for that is that, um, if you were to try to build automation that can go assemble this, that automation is very specific. The way we build automation today, it's like a bespoke machine that solves one particular physical problem, and that machine doesn't generalize from one phone to another, and it doesn't even generalize from one generation of the iPhone to another. So going back to, you know, what we can build in America, well, if you're building a car, right? Great. Build a highly automated facility, spend, you know, a couple years and $200 million to make it happen, and then produce that same car with minor cosmetic updates for seven years. The economics work great.

    14. SG

      Mm-hmm.

    15. BM

      But if you wanna do that for an iPhone, first off, you have to have, you have to go from about 25% automated to about 90% automated for it to pencil. And then how are you gonna be able to replace all of that automation? Like, that's just a huge, incredible cost that you're not gonna be able to amortize unless the automation becomes more flexible. And then there's a second piece of it, which is that, who is actually going to be designing this automation? There are about a million people assembling iPhones or components in Asia. There are, alongside that, about 10,000 manufacturing engineers who are figuring out, what is the manufacturing process that we need to do? How do we stand up these assembly lines? How do we spin down these assembly lines when it comes time to spin up the next product? And so that's, you know, fundamentally the problem we're trying to solve, is how can you make it as easy to turn over an automated line-

    16. SG

      Mm-hmm

    17. BM

      ... as it is to change and retrain workers in a human plant?

    18. SG

      Mm-hmm.

    19. BM

      So you can retrain a human worker in a day.

  7. 16:33 – 18:19

    Who needs flexible automation: defense, consumer electronics, data centers

    1. BM

      Practically, manufacturers think they're extremely agile if they can bring a design change into production, even on a manual line, in seven days.

    2. SG

      Mm-hmm.

    3. BM

      We would like to be able to build a system, where we are building a system, that allows you to upload your product design into our cloud and then design and ultimately reconfigure an automated line in seven days.

    4. SG

      Putting aside the fact that everybody would like to be able to make whatever they want on demand as quickly as possible with, with that sort of pace of innovation, like, what are the types of products and customers where this matters the most from a business perspective? Where that high mix and speed to bringing up the line is really important to them? 'Cause you kinda said, like, for a car, it's kinda okay to have the same car for seven years.

    5. BM

      It basically, it's anywhere that you are doing highly innovative, frequently changing manufacturing, or where you have a lot of variance. Defense is, I think, a very interesting example. It's very close to my heart as, you know, a Palantir veteran. And, you know, in Ukraine, for example, you are now going, you're now spending only a few weeks from changes in the battlefield-

    6. SG

      Mm-hmm

    7. BM

      ... into physical changes in the design-

    8. SG

      Mm-hmm

    9. BM

      ... that then need to be pushed out back onto the battlefield. That is a very short amount of time, and it's driven by these competitive pressures-

    10. SG

      Yeah

    11. BM

      ... right, between Ukraine and Russia. Well-

    12. SG

      'Cause if, if your adversary can turn that around faster than you can.

    13. BM

      Exactly. Well, you know, there are also competitive pressures, although less violent ones, between Google and Apple or, you know, the Android ecosystem and the I- iOS ecosystem. And so, you know, as you are, uh, building these larger and larger markets and these more and more innovative products, what you wanna be able to do is just- ... crank up the pace of innovation.

  8. 18:19 – 21:07

    Humanoid robots vs. robotic work cells

    1. BM

      And so consumer electronics-

    2. EG

      Mm-hmm

    3. BM

      ... is a great example here. A lot of things that go into data centers need to be changed very frequently because chips change very frequently. I think that's, those are the kinds of industries, defense, consumer electronics, the data centers, and data center supply chain, where you see a lot of need to be able to turn things over quickly, and where, as a result, almost everything is built by hand.

    4. EG

      And when you talk about automating these lines, is that humanoid robotics? Is it just existing manufacturing technology repositioned? How do you think about the form that's gonna take?

    5. BM

      Yeah, I, I mean, the, the f- the, the true answer is that it's all of the above. But practically, what does it look like? Um, you know, I, and I, I would, I think there's, the, the most obvious thing is that you wanna be able to have reconfigurable robotic work cells-

    6. EG

      Mm-hmm

    7. BM

      ... that you can immediately flow a design change into an actual change that's happening, uh, in the factory. That's a hard problem. Obviously, there's a lot of people working on different variants of that.

    8. EG

      The re- the reason I ask about it is, uh, I feel like there's two theories of the world, and I'm not saying they're incompatible. One theory of the world is everything is dir- designed around people right now in terms of manufacturing lines, et cetera, and so therefore, the most natural thing to do is just copy a person, which means have a humanoid robot. The other viewpoint is while there's lots of these, um, sort of very common manufacturing processes that you need to do, a lot of them are done by hand because, to your point, it just, it's faster turn if you need to turn something around quickly, and those things could just be made into sort of stepwise sequential things like what we've done in automotive for welding arms or other things like that. And so I'm a little bit curious, like, where on that spectrum you think we are in the near term versus the long term.

    9. BM

      Yeah, so I, so I would, I would come in with two ideas. So first off, let's talk about the form.

    10. EG

      Mm-hmm.

    11. BM

      Form factor. So for the form factor, given what I just told you, we are trying to replace humans, right? We're not trying to replace additional forms of automated lines. We don't need to work in automotive where they have these, like, extremely large baroque robotics. Um, I think, you know, two arms and a camera-

    12. EG

      Mm-hmm

    13. BM

      ... is enough, in a lot of cases, actually one arm and a camera is enough, and maybe you need some more, some better sensing. Um, you know, uh, s- so from a form factor perspective, these lines, you know, you have a human standing there all the time. You don't need a humanoid robot. You just need an arm, so-

    14. EG

      And does the capability of the arms have to change dramatically? 'Cause people always talk about the sensitivity and fidelity of existing robotic arms versus humans and all the rest. Like, how much progress actually has to be made in terms of the componentry for the arm versus, hey, it's actually ready to go and it's just all about the software?

    15. BM

      Automated, existing automated lines are able to build all o- all of the products that we're talking about, or at least, like, 90% of them. You know, yes, wire harnessing is hard, but, you know, if you go look at something like the iPhone, and you ask, what is the barrier to doing the automation? It is not that robot arms, you know, can't, you know, sufficiently screw in every screw and, you know, fiddle with every- everything. That kind of sensitivity

  9. 21:07 – 25:20

    Open-world vs. closed-world robotics

    1. BM

      already exists.

    2. EG

      Mm-hmm.

    3. BM

      It's the economics because those arms and, you know, that, those, they call them CapEx, capital expenditures, those machines have to be redesigned and rebuilt every year. So I think the-

    4. EG

      They can do it. It's just too expensive to do it at the pace they-

    5. BM

      They can do it. It's just too expensive

    6. EG

      ... want. Yeah.

    7. BM

      Yeah. And so I think there you wanna actually switch from thinking about the embodiment to thinking about the intelligence layer. So, um, for the intelligence layer, I would distinguish f- robotics as, you know, some robotics problems are open world, and some robotics problems are closed world.

    8. EG

      Mm-hmm.

    9. BM

      So if you were trying to clean a bedroom, that is an open world robotics problem. You have never seen the items here. You know, in a lot of ways that we think about this, when you're deployed there, you've never been in this bedroom before. You're never gonna go back into this bedroom. Um, it's this, like, very long-horizon planning problem. Um, that's very hard, and it's very suited to, you know, a pure... You wanna think about this as, as, you know, the, the intelligence you need is very similar to human intelligence. A closed world, um, is one in which you actually control everything, and you're doing something very repetitively, and there's no novelty that's being introduced.

    10. EG

      Mm-hmm.

    11. BM

      Assembly lines are a closed world. That makes sense because we see that we're actually able to use sort of traditional automation approaches in order to automate them, and I think it's worth talking a little bit about what those look like.

    12. EG

      Mm.

    13. BM

      So, you know, suppose that I am trying to, uh, solve some particular p- problem with, with automation, um, again, traditional robotic automation. What they end up doing, I think the best analogy for this is a Rube Goldberg machine.

    14. EG

      Mm-hmm. [laughs]

    15. BM

      So I, I had to build one of these with my son who's, who's 13 for school, you know, and you're, you're like, "Okay, well, you know, the ball rolls down the hill, and it hits the pencil, and the pencil, like, pops the balloon." And, you know, instead of that, it's actually you have a chute and, you know, the, the nut goes down the chute and, you know, there's a cup, and the cup is vibrating, and because it's vibrating, uh, it, the nut drops out, and it lands on a fixture.

    16. EG

      Yeah.

    17. BM

      And the fixture locates it precisely.

    18. EG

      Mm-hmm.

    19. BM

      So now there's no, you know, uncertainty about where it is, and a robot arm, without any perception whatsoever, can come and pick up the nut-

    20. EG

      Mm-hmm. Mm-hmm

    21. BM

      ... and then they can go take it to, you know, the bolt that you're screwing the nut on, which is also on a fixture, is also precisely located, and then it can screw that on. And-

    22. EG

      This is basically deterministic.

    23. BM

      It's, it's all deterministic

    24. EG

      It's moving from this coordinate to this coordinate.

    25. BM

      Yeah. So what's hard about this, right? Like, why, what, why it is, why is this interesting? Like, why does this take six months in order to turn around? Well, there's two things you have to do. One is you have made a machine.

    26. EG

      Yeah.

    27. BM

      Right? So someone needs to go design that machine. That is not an easy process. That is a mechanical engineering task. It is a, a manufacturing engineering task. Um, well, we now have AI that can design machines, so that's one way we can make it faster. The other thing, uh, which, you know, really hit home to me as I was doing this Rube Goldberg project, is, like, you know, half the work is in the design. The other half is in the calibration.

    28. EG

      Mm.

    29. BM

      Right? So, you know, you gotta make sure the ball hits the pencil exactly right so it pops the balloon. Um, you know, similarly, you know, if the nut is, you know, a 10th of a millimeter off, it may not end up in the right place on the fixture, and then, you know, the arm may not be able to, uh, actually, you know, place it in the correct place on the device.

    30. SG

      That sounds like a relatively simple RL problem. No?

  10. 25:20 – 28:59

    Building with Anduril: from days to 90 minutes

    1. BM

      automating the simpler kinds of stations, and then over time you get into the ones that are really wacky where you've got, you know, elastic things that need to be stretched over a gasket. And again, though, speaking of closed world, in an open world, you know, maybe, you know, you miss a couple things in the bedroom and that's fine. In a closed world, actually, like, 99 or 99.9% of the time you gotta go construct the device that you are constructing.

    2. SG

      How tall is the, the hill to climb to creating a great manufacturing engineer? Like, how deep is that field? How far do you think you are?

    3. BM

      You know, the starting point for us, you know, we've been talking about designing automated lines. We've spent the last year partnered with Anduril on building a software and AI platform that allows manufacturing engineers to design lines. Now, you know, Anduril's a defense manufacturer. You know, they don't need to have automated lines, and a lot of the, the products they have aren't high enough rate yet to support au- traditional automated lines. And so what we've done is, you know, basically, so we went in there a year ago. We actually talked to them the day before we incorporated the company, and they told us about their challenges. They talked about the need for a single source of truth for manufacturing engineering, where one place where you actually encapsulated the manufacturing process and the manufacturing plan-

    4. SG

      Mm-hmm

    5. BM

      ... and a reconfigurable factory that would allow you to change the manufacturing process and then also spin up and spin down different lines based on demand. And so we adopted those as sort of our North Stars for automating manufacturing engineering. And, you know, we got access to their systems in January. In April, we had our first user. Um, so he was able to take a actuator that he was trying to build, and normally that would've taken him a few days, uh, in order to do the virtual build for this, and he was able to do that in 90 minutes using our software.

    6. SG

      Mm.

    7. BM

      And that was, you know, the very first thing we did. And since, you know, and that, that worked out really-- That saved them a lot of time, and they were very excited. And so since then we have been moving across all of their pro- We're now deployed across all of their product divisions. Uh, we've done work with them, you know, on everything from drones to missiles. Uh, you know, we're currently working with them on the Fury, uh, collaborative combat airplane. So we're taking this entire airplane and, you know, uh, taking that manufacturing process and making that legible to AI and then, you know, a lot of what we're doing is basically you encapsulate that manufacturing process. Now it's not scattered in 100 different documents in 100 different places, and so you can have an AI agent that actually is able to, to add value. So, you know, going back to what does it take to be, to your question, to what does it take in order to truly, you know, build an automated manufacturing engineer, w- we are lucky that we can start by saying, "Okay, we wanna be Cursor here," effectively. Noting that Cursor basically said, "Okay, we're gonna take this existing IDE, and we're gonna add an agent." There's no IDE for manufacturing engineers. So first we had to build that so that there's a true data structure, there's the equivalent of, of effectively Git that you can work with.

    8. SG

      Mm-hmm.

    9. BM

      And then we can apply the agent on top of that. And then now that we own and, and, well, now that we control, uh, in a, in a specific way the manufacturing process, we can, you know, move left and right of that. So now you can say, "Okay, well, this design, you know, is manufactured this way. If we wanted to manufacture it more cheaply, how would we have to change the design?" So we can send design for manufacturing feedback

  11. 28:59 – 33:15

    Palantir’s problem-first approach vs. OpenAI’s technology-first approach

    1. BM

      back into the design engineering, which is, again, a role of the manufacturing engineers-

    2. SG

      Mm-hmm

    3. BM

      ... but they're able to use our agent to do that much faster and more effectively. And similarly, we can then say, okay, we can move right and say, if we wanna have an automated station, how do we design an automated station that can produce one of these, you know, that can do one of these manufacturing steps?

    4. EG

      Yeah, this is, um, definitely the most pragmatic approach to this world that I've seen, and I feel like there's been a lot of iteration over time. I remember, like, a decade ago funding things that were gonna be like Git for hardware or other things that kind of impinge on this in terms of how do you store the knowledge and then how do you act on it, and it feels like the unlock is really happening with AI. One thing that I think is really pragmatic about what you've ori- what you've described to me in the past in terms of the approach that you're taking is you're also not going and trying to build massive models of your own, and instead you're saying, "How do we actually use things that are a bit off the shelf from an AI perspective to be able to really make this streamlined and fast and efficient and cost-effective?" Could you talk more about that choice of, like, building your own models versus, uh, trying to use a broader suite of things against that?

    5. BM

      Yeah. Well, I would start but just by the, from the perspective simplicity.

    6. EG

      Mm-hmm.

    7. BM

      Right? So, you know, when you wanna go solve a problem, you wanna use the simplest tool for the job. And, you know, maybe even to step back, so, you know, I had, you know, one career at Palantir, one career at OpenAI. At Palantir, we were relentlessly problem-focused. You know, we wanted to build the enterprise platform for the intelligence community, which ultimately became, you know, the enterprise platform for all enterprises.

    8. EG

      Uh-huh.

    9. BM

      You know, we didn't care what technology we needed, and we had to solve a lot of very hard technological problems. But we used the forward deployed engineering approach. We talked to our customers, we heard what they needed, we brought that back into the product, and we just relentlessly expanded that, but by focusing on the problem. At OpenAI, we took the opposite approach. We had no idea what any of this would be used for. Um, you know, we-- I remember having a conversation with Ilya once where, uh, we had just built GPT-3 and, you know, we, we were like, "This is pretty good, but it needs to be better." And Ilya said, "Well, Bob, how do you actually build a company?"

    10. EG

      Yeah.

    11. BM

      "Like, what would we need to do to make this product?" I said, "Well, you know, there's a standard approach that everybody in startups knows, which is you pick someone who, you know, has a real problem, you actually solve their problem, and then you expand from there." And Ilya said, "Ah. Yeah, that's very interesting."

    12. EG

      Yeah.

    13. BM

      "No, that's not gonna work for us."

    14. EG

      Yeah.

    15. BM

      And I said, "What? Well, well-

    16. EG

      Yeah

    17. BM

      ... I mean, thanks for, thanks for asking, but what, why won't this work?" And he's like, "Well, look, we could learn something by talking to one person, but instead, we have the ability to just, you know, 10X our training data, and then we'll learn so much more from everybody, and that's much more than what we can learn from one customer."

    18. EG

      Mm-hmm.

    19. BM

      So I think that's, that's the justification for why you wanna take a technology-first approach to your market because fundamentally, this technology has its own way of finding out about the world. Well, at Arda, we're taking a technology first-- we're taking a problem first approach. And so why are we not training models? I've trained models before. We don't need to do it. Um, for the, uh, the first step of understanding the, the manufacturing process, designing a line from that, that's something that we see that frontier models can do. Uh, when you start going into something like robotics, then, um, frontier models can do some of it. Um, but what, what we see is that basically the frontier models may work best as an orchestrator. And in fact, that's, that's actually a lot of what we're doing-

    20. EG

      Mm-hmm

    21. BM

      ... with, uh, manufacturing engineering, is that we have built, um, specialized algorithms. We use motion planning. Um, we have different tools that you can use to inspect the CAD. The frontier model is not adding all the value. They're an orchestrator of all of these different tools.

    22. EG

      Yeah, you actually see this a lot in the SaaS world too, where you use the SOTA model as the orchestrator, and then you actually have fine-tuned models or specific things that are very specialized for the application area. So it's interesting to see that it's also happened here.

    23. BM

      Yeah. And, and I, I think it's just like a good general-purpose mechanism for thinking about things. Ultimately, if you wanna have the most intelligent model, there will be, you know, w- a small number of, you know, very capital intensive, very smart models.

  12. 33:15 – 37:13

    From dropping out of Stanford to joining OpenAI

    1. BM

      Like, you know, we-- I don't think you're gonna have specialist models unless you're thinking about cost. But when you're trying to-- when you're not a frontier lab, it's actually extremely approachable-

    2. EG

      Mm-hmm

    3. BM

      ... to make, you know, small, special-purpose models that are better than, you know, what the off-the-shelf frontier models can do, at least for now. And training one of those models versus, like, writing a software package versus writing a tool, like, those are all kinda the same action.

    4. EG

      How did you, um, end up at OpenAI coming from Palantir? 'Cause to your point, you came from such a pragmatic world, and I remember visiting OpenAI in twenty eighteen, twenty nineteen, and at the time, there was a handful of, like, robotic arms that weren't doing much in the office, and there was a bunch of, um, the, a lot of the focus, I think, was on video games and sort of building, beating video games initially, and then it, and then you guys started on the transformer and sort of GPT train. But for a while, it was kind of this very broad-based research approach, let's try a couple things. At least that's how I, I saw it as an external viewer. Was that a big transition? Why even join an organization that was so radically different from what you were used to?

    5. BM

      Yeah. So, so I would, I would actually start the answer to this, uh, ten years prior. So I was doing my PhD at Stanford in the AI lab, um, and you know, this from two thousand two until I dropped out in two thousand five to join Palantir. And I remember vividly sitting around with my research group-

    6. EG

      Mm-hmm

    7. BM

      ... making fun of people who thought that neural networks would ever work.

    8. EG

      Uh-huh.

    9. BM

      Because, you know, there was no, you know, proof that they would work, uh, you know, unlike SVMs or regression, which have clear provable guarantees. Uh, and even if they did work, you couldn't figure out why they worked, and so, you know, like, why, why, why would you ever use these things when obviously, obviously all these other things work? And so I left Palantir be- I left, uh, and went to Palantir because I thought that AI wasn't really happening-

    10. EG

      Mm-hmm

    11. BM

      ... back in two thousand five. So spent ten years at Palantir, ran engineering and product management there. And, you know, around twenty fifteen, I'm looking around. It's been a few years since AlexNet happened. Uh, vision models are beginning to really work. Palantir has grown tremendously, and I'm, you know, thinking maybe it's time for me to move on. And, you know, I thought to myself, like, "Maybe AI is really working now, and where is it going to be an application of all the vision models that people were think-- Well, sure, you can go recognize people's faces in photos, and if you want that, you should work at Facebook, not start a company." But robotics seemed like maybe this would be the place. Um, I was very wrong about that. I spent a year and a half trying to get this to work. I taught myself, you know, deep learning from watching Andrej Karpathy's videos.

    12. EG

      Mm-hmm.

    13. BM

      Uh, you know, got to the point of teaching a robot to play checkers from vision. And having done that, I realized this is way too hard. This is not the right time for a startup. And so, uh, I knew some people and, uh, you know, got introduced to this nonprofit full of smart people that had the, you know, somewhat dramatic goal of building AGI.

    14. EG

      Mm-hmm.

    15. BM

      And I thought, "Well, this isn't a... I, I, I'm not gonna start a company. Why don't I go work for a nonprofit for a while?"

    16. EG

      Mm-hmm.

    17. BM

      And so I started there, and, you know, I didn't think I would stay for years. I expected I would be there for a couple years. I would learn from people like Ilya or Wojciech Zaremba or Andrej Karpathy, who were there at the time, and I ended up joining the robotics team. And the goal for the robotics team was to show cool advances in robotics- Mm-hmm ... in a way that demonstrated that deep learning was actually a real thing that people should pay attention to- Uh-huh ... that it might in fact lead to AGI. And so we settled on this project of have a humanoid robot hand, a shadow hand, uh, be able to solve Rubik's Cube. And the reason for that, the way the story behind this is that Wojciech Zaremba, who's one of the co-founders of OpenAI, but also the head of the robotics team, went around at a robotics conference and asked people, "What would just be absolutely mind-blowing to you if we solved it?" [laughs] Yeah. And, you know, people said, "Well,

  13. 37:13 – 40:28

    Why general robotics models suddenly work

    1. BM

      either pick and place, that can pick and place arbitrary objects- Yeah ... or, or doing anything with a robot hand." Yeah. And what would be the most impressive thing you could do with a robot hand? Yeah. I don't know, solve Rubik's Cube. Yeah. And so we spent two years for that, and it actually worked. Mm-hmm. Um, although I, I will... I'm now permanently not a believer in robotic hands- Yeah ... uh, after the experience of trying to make one do anything. Yeah. I hear the, I hear the engineering's better now, but I'm probably ruined for life.

    2. SG

      If you fast-forward all the way to this point where Astra and Fable look like they're showing, you know, signs of life that excite or freak out the robotics community in terms of actually being able to generate policy to do things in the real world from a general model, like what do you make of that? Is it, i- is that like the end of people training, um, models specifically in general robotics?

    3. BM

      Well, you know, the, the end of people training models probably depends on venture capital, so-

    4. SG

      [laughs]

    5. BM

      ... you know, that's more a question for you guys than for me. But like, why does this work, right? Like, yes, it's the bitter lesson, but fundamentally, you know, the, the challenge in robotics has always been that the amount of data that you have to work with is very small. Mm.

    6. SG

      Mm-hmm.

    7. BM

      And so you would see companies that, you know, we're going to solve robotics by teleoperation, and we're gonna build giant teleoperation labs. And, you know, then it's like, oh, well, we can use these UMIs, uh, you know, little devices like chopsticks that record, you know, people's operations, and then I can collect data 10 to 10X more cheaply. All of these are extremely scarce sources of data. I think about it like if we were trying to train GPT-4- Mm ... but we were doing it on top of, you know, if we had to pay, pay Scale AI to go have humans write all of the data. Like, we would never have really been able to train GPT-4. But fundamentally, you don't need to train the model on exactly what you're deploying on. What you need to do is generate really good representations of the world, and then you can fine-tune the models towards a particular problem. And so, you know, where do those representations of the world come from? They come from predicting the last few significant bits of, you know, some sort of real-world data that you're dealing with, which could be internet videos. Um, I'm sure there is, in fact, a tremendous amount of robot data that has been brought in, but I also suspect it's very small compared to the fraction of overall data in a model like Astra or Fable.

    8. SG

      Mm.

    9. BM

      And so as, you know, the models progress, as the data gets better, as the algorithm, the improvements change, um, the model becomes better at sort of eking out those last few bits of randomness and being able to predict those. And suddenly, now it's able to predict events in the physical world. And once you have those really good representations, then you can go target them at robotics, and suddenly robotics begins to work. I suspect, you know, the, the... for a long time, there was this theory of, you know, emergent capabilities that, you know... I remember watching on Twitter sometime where someone said, "Oh, you know, I watched GPT-4, and it can go debug, it can, like, iterate with a software," you know, uh, you, you, you can run this program, you can, it can, you know, look at the output, and it can go fix bugs, this crazy emergent capability. And I'm like, well, Jerry spent, like, two years-

    10. SG

      [laughs]

    11. BM

      ... working on that. Yeah, yeah. Right? That's, like, not as emergent as you think it is.

    12. SG

      [laughs]

    13. BM

      Yeah. And so,

  14. 40:28 – 42:46

    Implications for the robotics ecosystem

    1. BM

      you know, yes, you know, we know that there are robotics teams at OpenAI, there are robotics teams at Anthropic that are doing this. But my guess is that the, the actual motor of progress here is not that they collected a lot more robot data than the next people, but that the models were able to finally, you know, get down to, you know, the, the last bits of the physical data that they already had.

    2. SG

      What do you think that means for the overall robotics ecosystem? Do you expect, like, maybe that's also a question for entrepreneurs and venture capitalists, but, uh, what do you expect the player landscape to look like? When do you expect people to have robots in different use cases?

    3. BM

      Yeah, I think this really democratizes it. Um, I, I think there's an interesting tension here that I don't know how to resolve, which is, you know, for digital tasks, few-shot learning works very well, and also, you know, you, it's really easy to, to collect data that you can actually, that, that's similar in format to the data you're actually using. In robotics, you know, again, because of my suspicion that a lot of the value is not coming from the robot data in these models, um, I've always believed that you're probably gonna wanna be able to fine-tune in order to be able to work in any particular area. And so I think there's a tension here between something like Astra or Fable. You know, it's very good out of the box. Um, it's also slow. That can be overcome, but you can't fine-tune it. So it's very hard to fine-tune effectively on a specific use case. Mm-hmm. And so I think that's an opportunity for, uh, other robotics players, and I think if you, if you look at Pi or you look at General, it's like you can see them, you know, using this opportunity that they have to take data and use it to fine-tune against a specific use case. Mm-hmm. And so, you know, to, you know, the bitter lesson is very strong, and also the idea of training on the test set is very strong.

    4. SG

      [laughs]

    5. BM

      Yeah. Right? Both of those are very powerful forces, and I think we're gonna have to see where those come in. Um, I do think this, this does sort of leaping out of that frame, though, is that agentic robotics, I think, is an interesting way, like sort of in between these two things, where, um, you can think of these models as just one more tool- Mm-hmm ... that the agents have. If you're doing something like we are, where you're working with a closed

  15. 42:46 – 44:18

    On-device models, speed, and safety

    1. BM

      world, then- You have a lot of different tools. Now you, you know, now if there's one thing that requires an incredible amount of dex-dexterity, maybe you put Fable on that. But maybe you don't wanna do that because Fable's gonna be really slow.

    2. EG

      Yeah.

    3. BM

      So you don't wanna do that for everything. And so I think that, you know, if you think about these, now for people who are actually deploying these, there's many more tools-

    4. EG

      Mm-hmm

    5. BM

      ... that they can do, and I think that's a very powerful mechanism.

    6. EG

      Mm-hmm. Yeah. I think people also talk a little bit about, um, to your point, sort of, uh, public/open world versus private, more closed loop systems. And, and the sort of public open world, like you have a robot in somebody's home or walking down the street with them or something else, you can't have, like, the lag of an internet connection or an API call because you're worried that the robot may freeze, fall over, something could happen, and so you want the intelligence to be embodied on the device as much as possible, or at least certain control mechanisms to provide safety relative to the device. And that's the other aspect that people sometimes talk about in terms of, uh, state-of-the-art models versus specialized fine-tuned models that are just running on device and things like that, which is kind of interesting from a more safety perspective, I guess.

    7. BM

      Yeah. I think, I think that's a, an important thing. I mean, it's a, there's a safety perspective and also there's just a speed.

    8. EG

      Yeah.

    9. BM

      Right? Um, that you, you want, you know, your robotics typically to operate on a very fast clock cycle-

    10. EG

      Mm-hmm. Yeah

    11. BM

      ... that is challenging to do with these larger models.

    12. EG

      Mm-hmm.

    13. BM

      Um, you know, historically that is typically solved by distilling into smaller models-

    14. EG

      Yeah

    15. BM

      ... whether that's something that the end user is doing or whether, you know, you have your Astra and by the time

  16. 44:18 – 50:57

    How forward deployed engineering really works

    1. BM

      that gets turned into a Luna model, you know, it can still work on some of the most popular use cases, but it's able to operate, like, very quickly and very efficiently.

    2. EG

      Mm-hmm. Yeah.

    3. BM

      Actual edge on-device models I think are also, you know, likely to be very important here. Um, and they're gonna rely pro- typically they end up being solved by open source that, you know, then gets improved through one way or another-

    4. EG

      [laughs]

    5. BM

      ... uh, coming from data from the big labs.

    6. EG

      Data distillation. Yeah, yeah.

    7. SG

      So amazing run at Palantir and OpenAI. You're used to doing things relatively fast, um, including with, like, difficult ecosystems. Uh, it's conventional wisdom that working with large incumbents with physical world plans, trying to get them to change big processes is hard, right? Like, how do you think about the pace for Arda and what's possible here?

    8. BM

      Yeah. So there is just a huge difference between being, you know, developing technology for yourself as a first party, like, you know, building models for use in ChatGPT and, you know, building something for use in a large enterprise. And the, the, the way that we're approaching this is that we're using the forward deployed engineering model. And so I think, you know, sort of classically, uh, we developed this in Palantir, you know, way back in 2005. You know, the way I like to think about it is, you know, back in 2005, I spent a lot of time in secure spaces in Northern Virginia working with people with secret workflows that couldn't leave the building and had to figure out how to build a product for them, right? And, you know, turns out building a product for people that have secrets that they can't tell the world is actually really good training for building a product for any enterprise. Because any enterprise that works, like, if they're working, they have secrets, and they don't wanna tell the world everything about their IP. And so the outcome of that process was the forward deployed engineering model, which I think mostly Shyam Sankar, who's the CTO at Palantir, deserves the most credit for, but my job was to figure out how to build products for it. And so what that does is you basically take your engineers and you put them, you know, embed them with the customer. Um, I would not develop a feature unless I had a customer that was ready to use that feature. And so when I think about the product roadmap, a lot of the, the, the beginning point for the product roadmap is not, "Okay, I'm gonna go build this feature," it's, "How do I find a customer that needs this feature so I can go build it?" As a consequence, then you need to start embedding with understanding, you know, seeing these enterprises. You still wanna move fast, though. So how do you move fast? Well, typically you wanna start by finding a small number of power users. Um, you know, classically, uh, at the agency way back in the day, there was a woman named Sarah Adams that was, like, the only user of Palantir for about a year.

    9. SG

      [laughs]

    10. BM

      Um, and we learned-- I would go visit and, you know, I would meet her and, you know, she would be like, "This stuff doesn't work." And I'd be like, "Okay, I've gotta go fix this." And eventually it worked, and then suddenly lots more people used it, right? But it's the power of that just very small number of very strong users that are willing to fight through the pain with you. Um, you know, I told the story about our first user for Arda, our first user at Anduril of, uh, a sort of lead manufacturing engineer who was trying to do the virtual build for an actuator. Well, what I didn't say is that this was a tremendously frustrating process for him being the first user, and he was able to make, you know, tremendous improvements and, and be able to move very fast with that actuator. But it only works because our head of product manager was sitting right next to him, you know, and every time he did something, it didn't quite work the way he wanted it to. And, you know, our head of product manager was just sitting there, like, taking notes. And then two weeks later, all those things worked.

    11. SG

      Thank you, user number one.

    12. BM

      Exactly. Exactly. And so but, but, like, it's not like this process ends.

    13. EG

      Mm-hmm.

    14. BM

      Because, you know, six months later, we're at Anduril, they're like, "Okay, go see if we can figure out how to build missiles faster." Well, this only works if we find that power user who's on the missiles team, who's now interested in working on a product that has, uh, 100 times more parts than the actuator, two orders of magnitudes more parts than the actuator that we originally worked with. Yeah, once again, out of the box, the product doesn't quite work. You find that user, you f- you-- with that user, you fight through to make it work, and then they can introduce it to everybody else. And so you're, the, you know, uh, you know, classically y- when you're doing enterprise sales, you wanna find that visionary early on. You know, this is from Crossing the Chasm, the classic business book. Classically, that visionary is a sponsor, and you do need that sponsor. But what the book doesn't talk about is you actually also need a user, someone who's actually willing to use your software to go accomplish an outcome.

    15. EG

      And if you do both these things, then even very large organizations will work on their problems faster than you expect.

    16. BM

      Well, there's not really any such thing as a very large organization. Every, you know, it's... People will be like, "Well, what does OpenAI want?" It's like, OpenAI doesn't want anything. Sam wants something. Greg wants something. Like, you know, this person, you know, this person down here, this research scientist on this team wants something. And big organizations are just an assemblage of very small teams. And so, you know, you have to proceed problem by problem, team by team. You know, one of the, the really interesting ideas in the FDE approach is you're not just throwing the software over the wall to them, you are partnering with them to achieve an outcome. For example, you know, figure out how to build missiles faster. And you are not successful unless the customer is successful.

    17. EG

      Mm-hmm.

    18. BM

      And once you achieve that outcome, you can then look around at all the other outcomes, and you've earned the right to work on something bigger. In, in our case, maybe that's we go work on an airplane, we go work on the Fury with Anduril, right? But also it could be we work on an entirely different problem that's bigger in terms of dollars for them. It moves the needle more for them. And maybe they wouldn't be willing to trust you, to trust your startup to solve that problem unless you had demonstrated some form of excellence.

    19. EG

      Mm-hmm.

    20. BM

      And so, you know, really within... A, a lot of this within an enterprise is snowballing, you know, the, the successes you've made, snowballing the outcomes you can achieve, while still baking that into your product.

    21. EG

      Mm-hmm.

    22. BM

      'Cause I think the, the thing that a lot of people who try the FDE approach now get wrong is that they see this as, okay, well, I'm just deploying some engineers at the site, and they're gonna make the product work at, at the site. You know, sort of, somewhat famously, like, people didn't believe the FDE approach worked, right? Like, it was widely mocked-

    23. EG

      Mm

    24. BM

      ... in, you know, 2015 to 2020.

    25. EG

      Folks thought there would never be leverage on it.

    26. BM

      That's right, yeah.

    27. EG

      Yeah.

    28. BM

      These, these are just consultants. Like, oh, Palantir just sends a bunch of consultants to do things. And then people started doing it when Palantir became a $300 billion company and started to profit. And the difference

  17. 50:57 – 53:41

    Superintelligence out of the data center

    1. BM

      is that they weren't just consultants, right? The FDE strategy is a product strategy that teaches you how to build products. The FDEs are the gap between the product and the user, or they're, they're filling the gap between the product and the user, and if they have to keep filling the gap, you get no leverage. Instead, you have to productize what they're seeing and bake that back into what you're doing. And so that's what allows you to move quickly and dis- and, and discover new things and bake those into your products.

    2. EG

      So if we take all this to the extreme and you have a army of FDEs over time who are deploying across multiple industries, you're learning very quickly, you're iterating, you're building a lot of product that then gets reused at scale, where does this all go? Like, what is your vision for the endpoint of all this?

    3. BM

      Yeah. Well, I, I, I would go back to, to think about superintelligence at the beginning. So we started by talking about, you know, what it looks like to have digital-only superintelligence and, you know, yes, the need for, for laying out lots of compute, that's a requirement for superintelligence. But superintelligence also has consequences. So, you know, I love Dario's phrase of, you know, a country of geniuses in a data center. But if all they can do is sit in that data center and think, and, like, you can send them emails, and they can send you emails back-

    4. EG

      Uh-huh

    5. BM

      ... like, they're not gonna be able to do very much. They're kind of trapped in the data center. And so, okay, so intelligence has become cheap. What are the complements to intelligence? Well, perception on the one hand, and action. As we have all these AGIs, you know, right now humans are the eyes and hands of the intelligence that's in the data center.

    6. EG

      Mm-hmm.

    7. BM

      Right? And instead, what I think we're gonna be seeing over the next 10 years is this tremendous profusion of, you know, various different kinds of perception, various different kinds of manipulation. You know, there'll be, you know, drones and sidewalk robots. Every physical thing that we own will become, you know, perception-enabled and hooked up to the cloud and, you know, whatever it does will become controlled, uh, by AI. And then all sorts of things that we wanna do, we will be doing because there are robots that are out there doing these things for you. Some people think that they will just be an army of humanoid robots. I think instead we're gonna see just this tremendous profusion of different forms. We think that, you know, the world is, is made for humans, but in many ways, you know, if, if you look at sort of the broader world, it's actually, you know, we'll, we'll probably see robots that begin to look like all different kinds of animals, right? There will be robots that fly. There will be robots on four legs. There'll be robots that carry burdens. There will be robots that, uh, you know, plow fields. Um, I think we already have those, right, with tractors. Um, and so the, the, the s- part of the story

  18. 53:41 – 56:46

    Self-replicating factories and Von Neumann probes

    1. BM

      of superintelligence is not just, you know, creating superintelligence-

    2. EG

      Mm-hmm

    3. BM

      ... and having it live in the data center and maybe scheme about how to take over the world. It's you want it to actually be able to go out of the data center and help people and do things for us.

    4. EG

      Mm-hmm.

    5. BM

      And so, you know, that, I think, is the, the call to action for manufacturing and why what we're doing, what m- what this, you know, manufacturing writ large is doing in this world of superintelligence.

    6. EG

      You know, Von Neumann and, I don't remember when he came up with this, he came up with, uh, with, uh, Von Neumann probes, right? These self-replicating probes that you send out of space. They land on an asteroid, they mine it, or they land on a planet, they mine it, they produce more versions of themselves, and they keep spreading.

    7. BM

      Yeah.

    8. EG

      And so it's the way to effectively explore the universe at scale in a sort of distributed system way where you have things self-assemble and then keep going, and it's exponential, right? You land, you make 10 copies. Those 10 copies go off and make 10 more copies, and you kind of spread through the universe. Um, do you view that as an eventual manufacturing model on, on our planet? In other words, eventually do you have these self-replicating factories? Or what, what do you kind of view as the end state from an almost, uh, physical form perspective?

    9. BM

      Let's, let's, like, narrow in and think about this concretely, right? So, you know- Uh, you know, maybe you have a problem that you need to go build more bulldozers, and so you need to build a bulldozer factory, right? Well, right there you have a recursive loop-

    10. EG

      [laughs] Uh-huh

    11. BM

      ... where the, the right way to do this is that you're deploying your intelligence to go manufacture more things. Um, and so, you know, I think, I think very, very quickly we will see, you know, yes, uh, factories that build other factories.

    12. EG

      Mm-hmm.

    13. BM

      But, like, that, that's machine tools, right? Like, that's what we have.

    14. EG

      Yeah, yeah.

    15. BM

      Right? But the difference is doing this in an intelligent way-

    16. EG

      Mm-hmm

    17. BM

      ... where, you know, the... it is a tremendous amount of human work to be able to go build a factory. It is a tremendous amount of human work to be able to design machine tools for that factory.

    18. EG

      Mm-hmm.

    19. BM

      All of that, I think, is going to become increasingly automated, and, and it will have to in order to be able to support-

    20. EG

      Yeah

    21. BM

      ... the vast network of things that we need in order to-

    22. EG

      It's basically the digital singularity versus the physical singularity.

    23. BM

      Yeah. Or, or, and, and not even-

    24. EG

      Yeah

    25. BM

      ... versus, just alongside. Like, part of the digital singularity involves converting increasing fractions of the Earth's crust-

    26. EG

      Mm

    27. BM

      ... into data centers.

    28. EG

      Yeah.

    29. BM

      Right? Um, and maybe those data centers are gonna be in space, too. Um, and, like, that is a physical process, and it's gonna have to be directed by AI, just like increasing human civilization across the world was directed by human intelligence. So I think that's a, that's a very near-term outcome. You know, that's, that's-

    30. EG

      Mm

  19. 56:46 – 59:33

    Bob’s answer to the Fermi paradox

    1. BM

      remains as much as possible in control of, rather than something that, you know, where humanity is a little sideline that just remains here on Earth, uh, and, you know, watches a lot of, uh, videos, uh, which will probably will also be generated by AI.

    2. SG

      They're going out there to terraform for us so that we may also explore the stars.

    3. BM

      Exactly. I think we will also have a role to play, if we choose to take it, in getting to the stars.

    4. EG

      Why do you think, um... I mean, this is back to the Fermi paradox, right? To some extent, AI and superintelligence and AGI and these self-replicating factories, they all suggest we should have come across other intelligent life sooner, actually. Right? In other words, it increases the likelihood of running across it than if you just had, you know, our biological forms trying to make it through space on our own.

    5. BM

      Actually, I disagree.

    6. EG

      Oh, interesting.

    7. BM

      So I think this is the resolution to the Fermi paradox-

    8. EG

      Uh-huh

    9. BM

      ... which is, uh, if you think about it, what is the most effective, uh, you know, civilization spreading through space? It's one that, you know, goes to a star system, very quickly sets up a factory, builds a lot of self-replicating probes-

    10. EG

      Yeah

    11. BM

      ... sends those out. And so you have basically a wave of civilization threading, spreading through space at the speed of light.

    12. EG

      Yeah.

    13. BM

      And so why don't we see that? Because it's spreading through space at the speed of light.

    14. EG

      I see.

    15. BM

      And, you know, uh, we will only see it, you know, a few minutes or, or perhaps a few years before it arrives.

    16. EG

      Isn't the argument that it should have arrived by now? In other words, there's arguments in terms of what's the density of potential civilizations throughout space and the rate at which they arise, and then therefore if you send something out at the, you know, sub- sub-speed of light, what is the rate at which it would eventually get to you, et cetera. And so that would suggest sparseness in the universe, right? In terms of-

    17. BM

      Well, I think... I mean, I think this is a mystery, right?

    18. EG

      Yeah.

    19. BM

      Is, like, why are we the first ones?

    20. EG

      Uh-huh.

    21. BM

      Um, you know, and there's some evidence that says if you look at sort of the uncertainty of all the different factors, uh, in the Drake equation-

    22. EG

      Mm-hmm

    23. BM

      ... that you end up seeing that, okay, well, maybe actually it's quite unlikely.

    24. EG

      Uh-huh.

    25. BM

      And so it is plausible that we're the first-

    26. EG

      Yeah, depends on what you assume-

    27. BM

      ... civilization

    28. EG

      ... on that Drake equation. Yeah, exactly.

    29. BM

      Yeah.

    30. EG

      Yeah.

  20. 59:33 – 59:55

    Conclusion

    1. EG

      Amazing. Thank you for joining us today.

    2. BM

      Thanks for having me. This was fun.

    3. SG

      [upbeat music] Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you wanna see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.

Episode duration: 59:57

In the uListen app, the transcript lights up word by word as you listen. Get uListen

Transcript of episode _wLzJINt9iQ

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.