The Next Frontier of AI Is Spatial Intelligence | Fei-Fei Li on a16z

1. 0:00 – 1:08

   Intro

   1. FLFei-Fei Li0:00

      We are building the next frontier of AI, which is what we call spatial intelligence.

   2. YLYunzhu Li0:05

      At SceniX, we are developing what we call a real-to-sim-to-real pipeline. We can replace all the data, all the evaluation we need in the real environments by using the data that can generate at a scalable way in our digital world.

   3. FLFei-Fei Li0:19

      Think about human intelligence. We do a lot of simulation in our head. Why? There's a very important role simulation plays that real-world data doesn't play, which is counterfactual reasoning.

   4. YLYunzhu Li0:32

      What we are building is a consistent world. Consistent both over space, over time, over different viewpoints, and over different type of interactions. My North Star is I want the robot to work.

   5. FLFei-Fei Li0:43

      The world we live in can be multiverse, that we create technology to allow people, builders, developers to act within different spaces.

   6. MCMartin Casado0:53

      Do you believe we'll ever be able to build robots that have the power efficiency of a human being? How far away are we from this? Is this like five years or this is like never?

   7. FLFei-Fei Li1:03

      The TLDR is we-

2. 1:08 – 6:04

   World Labs & SceniX

   1. MCMartin Casado1:08

      All right, well, it's great to have you both here. So Fei-Fei, for the listeners that may not have the background, maybe you can give, uh, an overview of what World Labs does.

   2. FLFei-Fei Li1:18

      Yeah, well, World Lab has... is a two-year-old startup. I, I, I think we should just recognize it's a frontier model lab. It's, uh, we, we are building the next frontier of AI, which is what we call spatial intelligence. And, uh, s- spatial intelligence is about, um, creating AI that has the ability to generate, uh, understand, reason with, and interact with spaces, whether it's physical or virtual. And of course, uh, a means to an end towards spatial intelligence is building large world models, and that's what, uh, World Labs is mostly focused on.

   3. MCMartin Casado2:03

      Yeah. So you've been saying this since the very beginning, which is, um, you know, the machine's ability to perceive and reason about spaces and act on spaces.

   4. FLFei-Fei Li2:11

      Mm-hmm.

   5. MCMartin Casado2:12

      But, but I always had the assumption that the acting on spaces was some, like, long distant future thing, but now you're acquiring a robotics company, and so maybe talk a little bit about the timeliness of this and the intentions.

   6. FLFei-Fei Li2:24

      Yeah. So first of all, it doesn't just take robotics to act within spaces or to interact, right? I, I mean, look at the creative field, whether it's, uh, VFX or gaming and, or design. Many use cases you can create and act within virtual spaces. And World Labs' thesis has always been that, um, the, the, the world we live in can be multiverse, that we create technology to allow people, builders, developers to act within different spaces.

   7. MCMartin Casado2:59

      Yeah.

   8. FLFei-Fei Li3:00

      Having said that, the ability to act within the physical space is one of the most exciting and most profoundly important capability of the future AI world. So robotics is very much that. So World Lab has always believed that robotics is a important application as well as use case of, uh, spatial intelligence and world modeling. So by joining force with, uh, inviting SceniX and SceniX team to World Labs is part of our long-term vision and, uh, mission. We, we've always committed to that.

   9. MCMartin Casado3:42

      Amazing. So Yunzhu, you're the co-founder of SceniX, so maybe provide everyone with a quick, um, overview of your background and what SceniX does.

   10. YLYunzhu Li3:50

       Yeah. So I'm Yunzhu. So I'm currently co-founder of SceniX-

   11. MCMartin Casado3:54

       Yeah

   12. YLYunzhu Li3:54

       ... and also assistant professor at, uh, Columbia University.

   13. MCMartin Casado3:57

       Wow.

   14. YLYunzhu Li3:58

       So my research started from my PhD at MIT, and then postdoc with Fei-Fei-

   15. MCMartin Casado4:04

       Really?

   16. YLYunzhu Li4:04

       ... at, uh, Stanford University.

   17. FLFei-Fei Li4:05

       Yes.

   18. MCMartin Casado4:05

       That's great. Yeah, yeah.

   19. FLFei-Fei Li4:06

       The world is small.

   20. YLYunzhu Li4:07

       The world is small.

   21. MCMartin Casado4:07

       Yeah, yeah. It is.

   22. YLYunzhu Li4:08

       Throughout my career, my goal has been very simple.

   23. MCMartin Casado4:11

       Yeah.

   24. YLYunzhu Li4:11

       Trying to help the robots better perceive and interact with the physical world.

   25. MCMartin Casado4:14

       Yeah.

   26. YLYunzhu Li4:14

       So I'm a very practical person. I want my robot to work in the real physical environments.

   27. MCMartin Casado4:19

       Yeah.

   28. YLYunzhu Li4:19

       So for SceniX, the unique opportunity we see is that there has been a lot of, like, bottlenecks-

   29. MCMartin Casado4:25

       Mm

   30. YLYunzhu Li4:25

       ... right now we see faced by the developments of general purpose robots, especially around training and also around evaluations. So at SceniX, we are developing what we call a real-to-sim-to-real pipeline.
3. 6:04 – 7:13

   Marble & the Data Bottleneck in Robotics

   1. MCMartin Casado6:04

      Maybe, Fei-Fei, just quickly describe what Marble is.

   2. FLFei-Fei Li6:07

      Yeah. Marble is the code name for the base model that World Lab has been training and iterating on. The, the fundamental capability right now of Marble that is publicly released is to take a, a prompt.

   3. MCMartin Casado6:20

      Yeah.

   4. FLFei-Fei Li6:20

      It can be a image, it can be a, a, a few images and, uh, or a text, and turn that into a- ... geometrically consistent world that can be represented in 3D geometry, whether it's Gaussian splat or mesh. Really what SceniX team is doing is trying to solve this extremely difficult problem in robotics, which is the lack of data.

   5. MCMartin Casado6:46

      Mm.

   6. FLFei-Fei Li6:46

      The lack of data in training, the lack of data in, uh, evaluation. This is very, very different from language models-

   7. MCMartin Casado6:54

      Yeah

   8. FLFei-Fei Li6:54

      ... where data is abundant on the internet.

   9. MCMartin Casado6:57

      Yeah.

   10. FLFei-Fei Li6:57

       And we know that, um, in order for robotics to work, we have to somehow unlock the power of scaling law. But where does that come from? This is something that, that... it's a profound problem that everybody's battling with in-

   11. MCMartin Casado7:12

       Yeah

   12. FLFei-Fei Li7:12

       ... in robotics.

4. 7:13 – 10:55

   How the Two Teams Come Together

   1. MCMartin Casado7:13

      It'd actually be great to talk about the synergy.

   2. FLFei-Fei Li7:15

      Yeah.

   3. MCMartin Casado7:15

      Like, you have put together a very, very talented team. You have put together a very talented team, and so, like, to what extent is there overlap? To what extent is this an extension? Maybe talk a little bit about that.

   4. FLFei-Fei Li7:24

      Yeah, that's actually a-

   5. MCMartin Casado7:24

      Like how complementary it is.

   6. FLFei-Fei Li7:25

      It's, it's actually the sh- the TLDR is it's very complementary and with a shared mission. So Yunzhu is one of the three, uh, technical co-founders. The other two are Changxi Zheng, another Columbia professor who has been a world-class technologist in simulation.

   7. MCMartin Casado7:44

      Oh, wow.

   8. FLFei-Fei Li7:44

      And Changxi has his background in also, um, VFX. He worked at Weta. He-

   9. MCMartin Casado7:49

      Oh

   10. FLFei-Fei Li7:49

       ... worked at Tencent. He's been a entrepreneur. Uh, then there's Sonny Hu, who is a phenomenal engineering leader who was also in a, uh, startup, uh, that was acquired by Amazon many years ago.

   11. MCMartin Casado8:04

       Okay.

   12. FLFei-Fei Li8:04

       So he worked in many-

   13. MCMartin Casado8:05

       That's great

   14. FLFei-Fei Li8:06

       ... different tech stacks in the computer vision field in, uh, in Amazon. So when, when we started talking more seriously, I recognized that, uh, a couple of things that SceniX has from a talent point of view is extremely complementary to, to, uh, World Labs. One is obviously Yunzhu's incredible, um, uh, thought leadership and, and just technical prowess in robotics, right? So-

   15. MCMartin Casado8:34

       Sure

   16. FLFei-Fei Li8:34

       ... uh, from really from hardware, full stack robotics, and even when he was my postdoc at Stanford, at that time you already had your faculty offer.

   17. MCMartin Casado8:45

       Yes.

   18. FLFei-Fei Li8:45

       So you were there only for one year. I wanted you for more than one year, but he had to go become a, uh, have the real job. [chuckles] So, uh, he was a full stack researcher in, in robotics from modeling to, to hardware. Uh, and of course, uh, Yunzhu and his student, uh, students at SceniX was that pool of talent World Lab hasn't had yet. Then on the Changxi side is just incredible simulation, um, um, capability, right? He's such a senior, um, researcher and technologist in simulation and, and, uh, what World Labs is doing is very much, um, interfacing the world of simulation. So, so I think what they don't have, um, obviously, is on the generative model side as well as the-

   19. MCMartin Casado9:34

       Sure

   20. FLFei-Fei Li9:34

       ... computer vision 3D reconstruction side, we're also very strong at World Labs. So that's a technology that SceniX needs. So together these, uh, these two sides come together and make it much more complete.

   21. MCMartin Casado9:48

       Fei-Fei's motivation in this is like this is an extension, uh, and a complement to get into robotics. You know, having been in your situation, which is deciding when to sell a company, it would be great to hear from you on, like, how you think about joining World Labs and kind of the fit there and, like, why you made the decision to do it.

   22. YLYunzhu Li10:03

       Yeah. So at the very beginning, we were deciding, okay, do we want to just keep going? But after chatting with Fei-Fei, after seeing all the synergies that are happening in the middle, it just makes perfect sense for the forces to join each other. So in a sense, at SceniX, what we have been doing is real-to-sim-to-real, is to do dense reconstruction of the environment. So we capture the appearance of the environment, geometry of the environment, and also the dynamics of the environment, meaning how the environment is going to change when you apply actions. So this dense reconstruction right now is still a little bit on the heavier side. And what World Labs right now has been doing involves a lot of profound capabilities around sparse reconstruction and generations. So we see a lot of opportunities of leveraging like Marble and other like capabilities at World Labs in order to do very efficient reconstructions and modeling

5. 10:55 – 12:35

   Building a Foundation Model for Robotics

   1. YLYunzhu Li10:55

      of the environment.

   2. MCMartin Casado10:56

      So can we expect a foundation model for robotics from World Labs?

   3. FLFei-Fei Li11:01

      World Lab is building a foundation model, as you know, Martin. We're building a base model, and, uh, as the technology has been evolving, some of the most exciting base models are omni models, right? They take, they take multimodal input, they have multimodal outputs. And, uh, what is a foundation model for robotics? Uh, it's very likely gonna involve actions.

   4. MCMartin Casado11:27

      Mm-hmm.

   5. FLFei-Fei Li11:28

      It's very likely gonna involve the output of actions in addition to the state of the world, and we're definitely not ruling this out.

   6. MCMartin Casado11:38

      Yeah, great.

   7. YLYunzhu Li11:38

      So for example, for the foundation models, it's essentially needs to be a multimodal model.

   8. MCMartin Casado11:42

      Hmm.

   9. YLYunzhu Li11:43

      So it has to take into account from text, image, depths, and different kind of modalities.

   10. MCMartin Casado11:48

       Yeah.

   11. YLYunzhu Li11:48

       And action is a very, very important parts of that modalities.

   12. MCMartin Casado11:51

       Sure.

   13. YLYunzhu Li11:52

       So if you think about from actions as the inputs, that essentially a forward simulator that is going to predict how the environment is going to change when you apply a specific action. When the action is output, this is essentially a policy model-

   14. MCMartin Casado12:05

       Yeah

   15. YLYunzhu Li12:05

       ... that is trying to predict, given a specific goal, like what should be the action you take in the real environment to get you closer to that goal. So this kind of omni models actually can benefit a lot and actually provide huge amount of values for the robotics communities in trying to understand how to model the environments and at the same time how to act in the environments. And this can also act as a backbone for you to fine-tune into specific robotic applications to making sure it's really live up to the reliability and efficiency that's expected by the clients.

6. 12:35 – 19:38

   Video Models vs Real-to-Sim-to-Real

   1. MCMartin Casado12:35

      You know, if, uh, Yunzhu, if you don't, uh, if you don't mind a kind of a lay investor question, I see a lot of robotics companies, and a very popular approach right now for the robotics companies that come in is like, "We'll use a video model." You know, and like, you know, that's the, the predominant method where this is, you know, 3D and simulation, it's a very different approach. And so maybe you could contrast the, you know, this popular approach of just using video only versus kind of what the ambition here is.

   2. YLYunzhu Li13:03

      Yeah. So in order to create worlds where the robot can learn, the worlds, as I mentioned, need to capture the essential structure of the problem. And one of the very important necessary, like, requirements for those worlds will be consistency. So that is where I actually see there's very, very strong synergies with Marble, because what we are building is a consistent world. Consistent both over space, over time, over different viewpoints, and over different type of interactions. And Marble, the generated worlds from Marble, is also provides an infrastructure, a component of that entire worlds that we believe is necessary for the robot to learn. Imagine if a robot's pushing an object forwards, the object just magically disappear, which has been a problem of many of the existing, like, video prediction models.

   3. MCMartin Casado13:47

      Right.

   4. YLYunzhu Li13:47

      This one provides good enough signal-

   5. MCMartin Casado13:49

      Yeah

   6. YLYunzhu Li13:49

      ... for the robot to know, like, what is the right thing to do. But obviously, right now, there has been a lot of investigation on building better and better and stronger and stronger, like, video models. So we actually see a way where some of the infrastructure we build can provide as, uh, initial momentums, and to going through this data flywheel of going from this, like, a more simulation-driven models into, like, a robot policy models, which gonna c- do the execution in the real environment, collecting new data, the data will come back in, where the model doesn't necessarily have to be physics only or learning only, but somewhere in the middle, which be able to capture the essential structure of the problem, but at the same time, be able to scale and become better and better as you accumulate more data.

   7. MCMartin Casado14:30

      You know, um, I've worked now with Fei-Fei very closely for a while and, and, and you've always had this North Star which has driven this, and, you know, you've, you've articulated it variously as kind of 3D and, and in a number of other ways. And I'm just wondering, for you, is there also a similar philosophical North Star, or you're more the pragmatic, like I am? [laughs]

   8. FLFei-Fei Li14:51

      [laughs]

   9. MCMartin Casado14:51

      Like, build the system, like, do the thing.

   10. YLYunzhu Li14:54

       My North Star is to make robots work-

   11. MCMartin Casado14:56

       Amazing, yeah

   12. YLYunzhu Li14:56

       ... in the real environment. I'm a very practical person. I want the robot to work. One interesting thing that's actually coming from my collaborations with Fei-Fei during my postdoc, we are building this kind of benchmark. We actually send out surveys asking the general public what they want the robots to do for them.

   13. MCMartin Casado15:12

       Yeah.

   14. YLYunzhu Li15:12

       Among the southern tasks we collected, one-third of the tasks are about cleaning.

   15. MCMartin Casado15:17

       Yeah.

   16. YLYunzhu Li15:18

       People just don't like to do those, like, uh-

   17. MCMartin Casado15:20

       Yeah

   18. YLYunzhu Li15:20

       ... dull and dirty tasks.

   19. MCMartin Casado15:21

       Right.

   20. YLYunzhu Li15:22

       And those are the scenarios that we really want to making sure we have robotic solutions to deal with.

   21. FLFei-Fei Li15:27

       One thing I really like about SceniX, uh, Martin, especially, um, uh, continuing your question, there's a lot of robotics companies building models and all that. One thing I truly like about SceniX is, is Yunzhu and his co-founders have such an incredibly pragmatic approach to robotics. They-- [laughs] Especially they come from academia, right? Uh, Sunny doesn't, but Yunzhu and Chanshy come from ac-academia, but their first instinct is work with design part-partners and customers in real industry, whether it's, is, uh, labs at, um, industry labs or, or, uh, warehouses or, um, um-

   22. YLYunzhu Li16:08

       Electronics assembly

   23. FLFei-Fei Li16:09

       ... electronics as-- uh, you know, uh, assembly. That is such a refreshing, ac-actually, a refreshing way of approaching-

   24. MCMartin Casado16:17

       Yeah

   25. FLFei-Fei Li16:17

       ... robotics, and that, that really true-- made me very excited to work with them.

   26. MCMartin Casado16:22

       Maybe this is for Yunzhu, but I, I'll just be... This is, this is personal curiosity, which is, it seems to me that for, for robotics, you have to be pretty exact. I mean, not perfect, but pretty close. But for the creative use cases, which World Labs has done a lot of, you kind of don't need to [laughs] because, you know, I mean, you know, even sometimes, like, being wrong is stylistic or intentional or whatever. And so from a technical perspective, what is the challenge here for reconciling these two things? Or do they never get reconciled? Like, will there always be two points in the design space?

   27. YLYunzhu Li16:52

       So they will be, like, reconciled in the, in the long terms, of course. And, um, modeling of the environments, um, doesn't have to be perfect. The model doesn't have to be perfect in robotics.

   28. MCMartin Casado17:04

       And by the way, is there-- Again, this is pure curiosity, but is there, like, um, a bit more formal way to say that? Like, what does that mean not to be perfect? It has to be pretty close.

   29. YLYunzhu Li17:12

       Mm-hmm, mm-hmm. So, so let me put it this way. For example, models over the developments of all different kind of robotic applications has been a very important cornerstones.

   30. MCMartin Casado17:21

       Yeah.
7. 19:38 – 23:01

   Why Simulation is Essential for Robot Learning

   1. MCMartin Casado19:38

      Very good.

   2. FLFei-Fei Li19:39

      I want to add to this and be s- uh, slightly philosophical here, is there isn't a, a, uh, binary choice between simulation or no simulation. All this come, um, in together, um, to, to make robotics work. Think about human intelligence. We do a lot of simulation in our head. You know why? There's a very important role simulation plays that real world data doesn't play, which is counterfactual reasoning, is that you play out events that ca- hasn't happened or cannot happen, or you don't have enough data to make it happen in real world. And while, while you play it out, you learn how to act in it. Humans do this all the time. We probably don't... You know, we just-- I know you were at the World Cups. [laughs]

   3. MCMartin Casado20:30

      I was at the World Cup, yeah.

   4. FLFei-Fei Li20:31

      Congratulations to Spain winning. I'm sure in the planning of every game, there is simulation, whether it's digital or on the, on the whiteboard or whatever. That simulation, the role simulation plays, is counterfactual reasoning, and that's really important in robotics because we just do not have po-- cannot possibly have enough real world data for that. Here is a real life, uh, example, the, the industry of self-driving cars. Waymo has officially said they use billions of hours of simulation.

   5. MCMartin Casado21:06

      Yeah.

   6. FLFei-Fei Li21:06

      And, and actually, Waymo is more simulation heavy than just, uh, real world data heavy. So these are real examples and, and as you know, Martin, and Yunzhu too, cars are the simplest kind of robots.

   7. MCMartin Casado21:19

      Yeah, 2D, yeah.

   8. FLFei-Fei Li21:20

      Yeah. They-

   9. MCMartin Casado21:20

      Of course, yeah

   10. FLFei-Fei Li21:21

       ... so, so clearly, simulation plays a huge role in robotic learning.

   11. YLYunzhu Li21:26

       I also want to add to that. So like, uh, there are-- If you put things more specific, simulation can provide two levels of benefits. The first one is reliability and the second one is efficiency. So for reliability, if you're thinking about a robotic system working reliable in the real environments, you need data to provide systematic coverage of all the state space and the variations that robots might encounter. That's how you can learn of how that is robust. So with simulation, you can do systematic randomizations and control and the variations of lighting, frictions, geometries, object types, and also all different kind of physical parameters to making sure you have sufficient coverage of the state space. So this is what can give the robotic systems reliability. And second is about efficiency. So right now, many people are doing teleoperation, and if you look at many of the teleoperation device, imagining all the exoskeletons you are using, you're actually collecting the data at a speed that is actually slower-

   12. MCMartin Casado22:25

       Yeah

   13. YLYunzhu Li22:26

       ... than human actually doing the task.

   14. MCMartin Casado22:27

       [laughs]

   15. YLYunzhu Li22:27

       But for many of our clients-

   16. MCMartin Casado22:29

       Yeah

   17. YLYunzhu Li22:29

       ... human speed to them is no good enough.

   18. MCMartin Casado22:31

       Yeah.

   19. YLYunzhu Li22:31

       They want faster than human speeds.

   20. MCMartin Casado22:33

       Yeah.

   21. YLYunzhu Li22:33

       So for the robot to like, uh, move faster, it's not as simple as just drive the robot faster because the gravity doesn't change.

   22. MCMartin Casado22:39

       Yeah.

   23. YLYunzhu Li22:40

       But in simulation, you can do systematic speed up of the robot's behaviors to train the robots such that it considers all the dynamics changes of the environments. So this is what can give, like our clients, for them efficiency. So both for the reliability and efficiency, though there are some kind of like a very unique, like values where simulation can provide.

8. 23:01 – 29:17

   Training, Evaluation & Real Customer Use Cases

   1. MCMartin Casado23:01

      You've talked about the technology and the platform, what it does. Maybe talk about the specific use cases people use it for.

   2. YLYunzhu Li23:06

      Mm-hmm. There are essential, like two specific use cases, especially around both training-

   3. MCMartin Casado23:11

      Mm-hmm

   4. YLYunzhu Li23:11

      ... and also around evaluations.

   5. MCMartin Casado23:13

      Okay.

   6. YLYunzhu Li23:13

      Yeah. Starting from the evaluations.

   7. MCMartin Casado23:15

      Yeah.

   8. YLYunzhu Li23:15

      So evaluation is something like people o- o- often overlooked-

   9. MCMartin Casado23:19

      Yeah

   10. YLYunzhu Li23:20

       ... in the robotics. But if you are training like robotic models-

   11. MCMartin Casado23:23

       Yeah

   12. YLYunzhu Li23:23

       ... you have to know how well it works, and that is the only source of information for you to iterate.

   13. MCMartin Casado23:28

       Yeah. By the way-

   14. YLYunzhu Li23:29

       Your righteousness

   15. MCMartin Casado23:29

       ... a lot of [laughs] -- a lot-- every, every AI person really understands what evals are and uses it all the time. Non-AI people, it often means something a little different, so maybe it's even worth just describing specifically what you mean by evaluation.

   16. YLYunzhu Li23:41

       Okay. So what I mean by evaluation is you'll be able to understand, for these specific checkpoints, how well does it perform? Does it perform, for example, 95% of the time or 99.9% of the time?

   17. MCMartin Casado23:53

       Sure.

   18. YLYunzhu Li23:54

       And the key criteria people use in industry is how long does it take? How long in work-

   19. MCMartin Casado24:00

       Interesting

   20. YLYunzhu Li24:00

       ... work clock time does it take for you to distinguish between a checkpoint that is 90% from a checkpoint that is 90.92 points. And if you only do that in the real environment, this just takes so long-

   21. MCMartin Casado24:12

       Interesting

   22. YLYunzhu Li24:12

       ... for you to do the distinguishment. And if you really think about also the robotic evaluations right now people are doing in the real environments, the iteration speeds is multiple orders of magnitude slower-

   23. FLFei-Fei Li24:24

       Mm-hmm

   24. YLYunzhu Li24:24

       ... than iterations of those language models.

   25. FLFei-Fei Li24:26

       Yeah.

   26. YLYunzhu Li24:26

       So not only is like, uh, the robotic tasks very varied, very diverse and-

   27. MCMartin Casado24:31

       Oh, yeah, because like you actually have to do the thing.

   28. YLYunzhu Li24:33

       Yeah, you actually have to do the thing.

   29. MCMartin Casado24:33

       Yeah, yeah, yeah. Right, right, right.

   30. YLYunzhu Li24:35

       Like, like, place the glass.
9. 29:17 – 36:56

   Humanoids, Semi-Structured Environments & the Grand Challenge

   1. MCMartin Casado29:17

      You know, you, you, you have s- uh, told me, um, that you think, uh, a lot of the predictions around humanoids were a little bit aggressive, and we're likely to see more constrained rollouts, like warehouses or whatever. Can you talk a little bit about that and like how that impacts what you're gonna be tackling here at, uh-

   2. YLYunzhu Li29:35

      Yeah

   3. MCMartin Casado29:35

      ... the con- like World Labs?

   4. YLYunzhu Li29:37

      So that's a very good question. So if you look at, for example, all the, uh, progressions of robotic applications in the real environments, it has always followed the trend from going from fully structured environments into semi-structured environments, and then into unstructured environments.

   5. MCMartin Casado29:52

      Yeah.

   6. YLYunzhu Li29:53

      For fully structured environments, what we mean is that you have knowledge and the control over all the configurations within the environments.

   7. FLFei-Fei Li30:00

      Like factories.

   8. YLYunzhu Li30:00

      Like factories or, for example, car manufacturing lines.

   9. MCMartin Casado30:03

      Yeah.

   10. YLYunzhu Li30:03

       Those has been automated for decades.

   11. MCMartin Casado30:05

       Yeah, yeah, yeah.

   12. YLYunzhu Li30:05

       And then you have, for example, semi-structured environments, which you have certain controls over the environments. For example, like the Amazon, for the warehouses.

   13. MCMartin Casado30:14

       Yeah.

   14. FLFei-Fei Li30:14

       Mm-hmm.

   15. YLYunzhu Li30:14

       Or for example, like, uh, restaurants, hotels, where you have certain control over the environment to just make the task easier-

   16. MCMartin Casado30:21

       Yeah

   17. YLYunzhu Li30:21

       ... for your robots. But there are obviously many other, like objects.

   18. MCMartin Casado30:24

       Yeah.

   19. YLYunzhu Li30:24

       Or for example, clothes, those are the object you don't have control.

   20. MCMartin Casado30:27

       Yeah.

   21. YLYunzhu Li30:27

       And then for the unstructured environments, it's like your home and my homes.

   22. MCMartin Casado30:30

       Yeah.

   23. YLYunzhu Li30:31

       Those is, I would say, the, the, the grand challenge-

   24. MCMartin Casado30:34

       Yeah, especially-

   25. YLYunzhu Li30:35

       ... for robotics

   26. MCMartin Casado30:35

       ... especially my house, trust me. Three dogs.

   27. FLFei-Fei Li30:37

       [laughs]

   28. MCMartin Casado30:38

       Five-year-old.

   29. FLFei-Fei Li30:39

       Yes. Dogs.

   30. MCMartin Casado30:41

       Yeah.
10. 36:56 – 42:05

    Integration Plans & What Success Looks Like in Two Years

    1. MCMartin Casado36:56

       timeframe? How are you thinking about this, Fei-Fei? Is this something that integrates right away or is this kind of a separate longer term?

    2. FLFei-Fei Li37:03

       This is a great question. I think at this point, you know, Yunzhu, Changxi, Sunny, Justin, Ben, and, uh, have been talking about this. At this point, we are going to Take it thoughtfully. We're not rushing to integrate everything from code base to teams because I think SceniX does have a very, uh, um, well-thought and I wouldn't call it standalone completely, but fairly, uh, contained, um, tech stack, as well as their customers-

    3. MCMartin Casado37:37

       Yeah

    4. FLFei-Fei Li37:37

       ... as well as the, the, the kind of products they're building.

    5. MCMartin Casado37:41

       Yeah.

    6. FLFei-Fei Li37:41

       We're gonna take time. We definitely will s- we already have a simulation side as well as the, the potential base model, um, action condition model side. We already are starting to talk, and also they are using Marble as a internal customer. Uh, so we, we will be integrating, but, uh, we're not rushing to blend the team, uh, as like a full salad bowl. [chuckles]

    7. MCMartin Casado38:07

       Yeah. H-how are you thinking about geographies with this? Will SceniX move? Is it gonna stay in the same place?

    8. FLFei-Fei Li38:12

       We're gonna... Yunzhu is gonna move. [chuckles]

    9. MCMartin Casado38:14

       Oh, well, welcome.

    10. FLFei-Fei Li38:15

        Yeah. [chuckles]

    11. MCMartin Casado38:16

        Here. Good.

    12. YLYunzhu Li38:16

        I'm moving to San Francisco.

    13. MCMartin Casado38:17

        Yeah, Florence during the Renaissance. Perfect.

    14. FLFei-Fei Li38:19

        Um, I think we, World Labs is officially becoming a bi-coastal company where the headquarter is in San Francisco. I've, you know, I live in Palo Alto. I feel like I'm in a [chuckles] different state. We, we, but, uh, we, but, um, I'm actually excited that we're gonna have an office in New York-

    15. MCMartin Casado38:40

        Of course

    16. FLFei-Fei Li38:41

        ... that can help us to attract talent on the East Coast. And also we have been talking about making sure that in both offices we set up the robots so that we get to basically test out and, and, and, uh, mature our engineering stack so that we can work with robots remotely because we have to do that for, for our customers anyway.

    17. MCMartin Casado39:03

        So may-maybe just to be very concrete, Fei-Fei, maybe let's just pencil out, like what is the, the perfect success case in two years? Like what product do you have? Who's engaging with it? How do they use it? Just, just a crisp like what this becomes.

    18. FLFei-Fei Li39:16

        I will be very happy that, uh, SceniX team and World Labs team will have validated, um, customers in, in, um, a small number of important vertical use cases where our system, our infrastructure has proven to be truly beneficial to their automation needs, and these customers became our lighthouse examples to scale our business.

    19. MCMartin Casado39:54

        And how, how early-- let's say someone listening to this, uh, is running a robotics company. How, at what stage do they engage with World Labs?

    20. FLFei-Fei Li40:06

        World Labs. Yeah.

    21. MCMartin Casado40:06

        Is it really early on? Is it somewhere in the middle?

    22. YLYunzhu Li40:10

        So right now, um, for our customers-

    23. MCMartin Casado40:12

        Yeah

    24. YLYunzhu Li40:12

        ... because we are building this kind of real-to-sim-to-real pipelines-

    25. MCMartin Casado40:15

        Yeah

    26. YLYunzhu Li40:15

        ... like where the simulation is essentially the worlds-

    27. MCMartin Casado40:18

        Yeah

    28. YLYunzhu Li40:18

        ... we're gonna provide the training and evaluation grounds. Some customers, they need only the real-to-sim part. They want to digitalize the task they care about and be able to do the evaluations-

    29. MCMartin Casado40:28

        I see

    30. YLYunzhu Li40:28

        ... of their robotic systems.

Episode duration: 42:20