The Data Layer for the Robot Economy

1. 0:00 – 0:05

   Intro

   1. NDNicolas Dessaigne0:00

      [upbeat music]

2. 0:05 – 0:30

   Encord’s mission: an AI-native data layer for physical AI and robotics

   1. NDNicolas Dessaigne0:05

      Today I'm joined by Eric and Ulrik, the co-founders of Encord. They just announced a 60 million Series C led by Wellington Management. Congrats guys, and welcome to YC.

   2. UWUlrik Waage0:16

      Thank you, Nicolas.

   3. ELEric Landau0:16

      Thank you. Yeah.

   4. NDNicolas Dessaigne0:17

      So let's jump in. What is Encord? Can you tell us, uh, what you are doing today?

   5. UWUlrik Waage0:21

      Encord is AI-native data infrastructure, and, uh, we work with the world's top AI teams that are building different types of applications, predominantly in physical AI and robotics.

3. 0:30 – 0:51

   What “data infrastructure” means: curate, annotate, evaluate—and keep bad data out

   1. NDNicolas Dessaigne0:30

      So, so what is a data infrastructure? Like are you doing the labeling? Are you selling the data?

   2. UWUlrik Waage0:35

      Ultimately, when companies build models, they need to make sure that the, the data going into the model is the right data, so getting the right data in, keeping the wrong data out, and we build, uh, our platform, a universal data layer for physical AI to effectively create, manage, annotate, and evaluate the data.

4. 0:51 – 1:22

   Why the problem gets harder after deployment: continuous data improvements

   1. NDNicolas Dessaigne0:51

      And so what problem are you exactly solving for them?

   2. UWUlrik Waage0:54

      So ultimately, a model is only as good as the data it's trained on, and even like the slightest errors in the data set can influence and impact like how the model actually works in the real world, and that is an incredibly difficult problem to solve because ultimately like the, uh, data sets only really get larger as, uh-

   3. NDNicolas Dessaigne1:11

      Mm

   4. UWUlrik Waage1:11

      ... AI complexity and, uh, size of the model scales. Once the model's in production, they have to make sure that they continue to feed it with the correct data that k- keeps pushing the frontiers of the, of, of the model.

5. 1:22 – 2:19

   Founding story: spotting data as the defensible bottleneck (pre-ChatGPT)

   1. NDNicolas Dessaigne1:22

      All right. So before to explore more what Encord is today, let's go back to the, the founding of the company. Can you tell us more about how you came up with the idea, the founding story?

   2. UWUlrik Waage1:31

      So it was like right when AI was like starting to take off for the people that were, uh, taking note.

   3. NDNicolas Dessaigne1:36

      Yeah.

   4. UWUlrik Waage1:37

      So towards the end of the, the 2010s, um, I was doing a computer science masters at Imperial doing deep learning research. Eric was working at a high-frequency trading firm. I saw that out of the three ingredients for AI development, models, compute, and data, uh, the one that, uh, took the longest, we spent a bunch of time wrangling the data, spent a bunch of time cleaning the data, felt like the most, uh, defensible. Um, and so Eric had worked on big data systems and put thousands of models into production, and, uh, we saw that the way that AI development was being done was, again, sending all this data to-

   5. NDNicolas Dessaigne2:10

      Mm

   6. UWUlrik Waage2:10

      ... uh, the Philippines, getting it back, and we thought there had to be a better way. And so we, uh, decided to found Encord to solve, broadly speaking, the, the data problem in, in AI.

6. 2:19 – 3:13

   Early skepticism: raising money when AI data tooling looked “unsexy”

   1. NDNicolas Dessaigne2:19

      And that was all before ChatGPT, so really you kind of like, uh, anticipated. You were already seeing the future in a way.

   2. UWUlrik Waage2:26

      That's right.

   3. NDNicolas Dessaigne2:27

      Is that the right way to frame it?

   4. ELEric Landau2:28

      Uh, yeah, it was, um, it was definitely not the hot category when we were in YC.

   5. NDNicolas Dessaigne2:33

      Like-

   6. ELEric Landau2:33

      So-

   7. NDNicolas Dessaigne2:33

      Like what, uh, what did you believe early like that, that was, uh [sniffs] , that looked unsexy then but now is obvious?

   8. ELEric Landau2:41

      Uh, well, actually AI itself. Um, so in, in our batch, um, the, the main companies that were getting a lot of attention were fintech companies, they were crypto companies.

   9. NDNicolas Dessaigne2:51

      A lot of crypto back then, right?

   10. ELEric Landau2:52

       Yeah. Remote working was a huge one. Um, so we actually struggled to, to raise a seed round, and, uh, one of the funds that we get- got in like really late stages with-

   11. NDNicolas Dessaigne3:00

       Mm-hmm

   12. ELEric Landau3:00

       ... um, they e- ended up rejecting us 'cause they said the market wasn't gonna be big enough.

   13. UWUlrik Waage3:04

       [laughs] Yeah.

   14. ELEric Landau3:05

       Uh, and then their next investment was an Icelandic dating company. So, um, they thought that the Icelandic dating market was going to be bigger than, than the AI market.

7. 3:13 – 3:42

   Encord today: customers, scale, and traction across physical-world AI

   1. NDNicolas Dessaigne3:13

      And can you give us a sense of like the, the scale of the business today? Like how many customers do you have today?

   2. UWUlrik Waage3:19

      We work with more than 300, uh, AI teams. They range from companies in autonomous driving, companies like Toyota, um, some of the world's leading robotics companies, and everything that really touches the, the physical world. We're a team of 150 people between London and San Francisco, and, uh, we raised $110 million with a $60 million CVC that we just, uh, announced.

   3. ELEric Landau3:39

      Excellent.

   4. UWUlrik Waage3:40

      Um, so yeah, we're, uh, excited.

8. 3:42 – 4:43

   The first product: automating computer-vision annotation workflows

   1. NDNicolas Dessaigne3:42

      It took you some time to get, to get here, right? I remember you were like in the Winter '21 batch.

   2. ELEric Landau3:46

      Yeah.

   3. NDNicolas Dessaigne3:46

      And so that was even pre-ChatGPT release, right?

   4. ELEric Landau3:50

      Yeah.

   5. NDNicolas Dessaigne3:50

      Uh, what did you, uh, what did the first version of Encord look like?

   6. UWUlrik Waage3:55

      Well, so the, the, the first, uh, version of, of Encord was really a product to automate the annotation process-

   7. ELEric Landau4:00

      Yeah

   8. UWUlrik Waage4:00

      ... particularly for computer vision. So at the time, uh, the way that like companies build models, they would take some data, they would send it to the Philippines to get labeled, they send it back. And, uh, Eric and I, when we met, uh, that felt like a very like, um-

   9. ELEric Landau4:13

      Yeah

   10. UWUlrik Waage4:13

       ... slow and like tedious and extensive process.

   11. NDNicolas Dessaigne4:14

       What was market back then like? LLMs didn't really, wasn't, were not seen yet.

   12. ELEric Landau4:19

       Uh, GPT-2, GPT-3 maybe was, was around at the time.

   13. UWUlrik Waage4:22

       I think, uh, was it GPT-3 was announced when we were in YC.

   14. NDNicolas Dessaigne4:26

       So these-

   15. ELEric Landau4:26

       Yeah, it was YC

   16. NDNicolas Dessaigne4:26

       ... were not the data, like the people, uh, needing you necessarily, using you. Like who was the, the first customers?

   17. UWUlrik Waage4:35

       At the time, like, uh, language models were called NLP or like-

   18. NDNicolas Dessaigne4:38

       Yeah [laughs] .

   19. ELEric Landau4:38

       Yeah

   20. UWUlrik Waage4:38

       ... that was like what people were building, and then, uh, there was computer vision.

   21. NDNicolas Dessaigne4:41

       I remember you were segmenting images and so on.

   22. UWUlrik Waage4:43

       That's right.

9. 4:43 – 6:14

   ChatGPT changes the market: trust in AI-enabled automation emerges

   1. NDNicolas Dessaigne4:43

      And so what did the release of ChatGPT do to you? Like how much of a shock was that?

   2. ELEric Landau4:48

      Uh, so for us it was, um, a quite important moment. Before, so we came into YC to automate the annotation process, and one of the things that we did when we started was we came up with this methodology, we called it at the time micro models.

   3. NDNicolas Dessaigne5:02

      Okay.

   4. ELEric Landau5:02

      So kind of the opposite of a foundation model, like a very specific model that is specialized for a particular, uh, part of the data distribution. You can train off of two or three examples, and then you can use it to automate the annotation process, and you, um, kind of compile a bunch together. But what we were finding was that, uh, so we had this like fun technology, it was useful to automate the annotation process. People weren't actually using it, and they didn't trust AI to actually process their data because there was a lot of skepticism even for AI companies-

   5. NDNicolas Dessaigne5:30

      Okay

   6. ELEric Landau5:30

      ... on their data. And so what ChatGPT did was it showed that, "Oh, actually, um, you can trust AI for a lot of these use cases." And now we implicitly trust AI for many, many different a- applications.

   7. NDNicolas Dessaigne5:41

      That's the thing. So even the AI companies of that time-

   8. ELEric Landau5:44

      Yes

   9. NDNicolas Dessaigne5:44

      ... needed to see that consumer product-

   10. ELEric Landau5:46

       Yes

   11. NDNicolas Dessaigne5:46

       ... to start believing, believing in AI.

   12. ELEric Landau5:48

       They, they wanted humans to be on their data. They didn't trust AI to work with their data, and it wasn't until they saw that AI could work in-

   13. NDNicolas Dessaigne5:56

       Mm-hmm

   14. ELEric Landau5:56

       ... a lot more generalizable use cases that they started, uh, automating more-

   15. NDNicolas Dessaigne6:00

       It's kind of like helped you, uh-Educate the market-

   16. ELEric Landau6:03

       Yeah

   17. NDNicolas Dessaigne6:03

       ... on the value of AI-

   18. ELEric Landau6:05

       Yeah

   19. NDNicolas Dessaigne6:05

       ... for what you are doing. Uh, but did that mean also that you had to, uh, expand the product, or was your initial product still enough for that market?

10. 6:14 – 7:53

    Expanding to multimodal and physical AI: why embodied systems need a data layer

    1. ELEric Landau6:14

       When ChatGPT came out, we saw a big opportunity in multimodal AI in particular, uh, because-

    2. NDNicolas Dessaigne6:19

       Other, like not images, not video

    3. ELEric Landau6:21

       Not just images and videos, but image with text, with audio. Ultimately, a lot of, um, systems-

    4. NDNicolas Dessaigne6:26

       Mm-hmm

    5. ELEric Landau6:27

       ... of the future will be multimodal. Humans are multimodal. Uh, so we invested in, uh, multimodal applications soon after the, um, the kind of ChatGPT moment-

    6. NDNicolas Dessaigne6:36

       Yeah

    7. ELEric Landau6:37

       ... which has led us to, uh, being, you know, very well-versed in, in physical AI and, um, similar, uh, tech.

    8. NDNicolas Dessaigne6:42

       So it's kind of like the new, uh, the new thing for you, physical AI today, right?

    9. ELEric Landau6:46

       Physical AI has been around for some time, and-

    10. NDNicolas Dessaigne6:48

        Okay

    11. ELEric Landau6:48

        ... we've been, um, uh, working in it for, for a while. Uh, but I think what's changed is that the kind of generalizability of these, of these AI models have, um, uh, have been kind of unlocked by seeing that scaling laws work with LLMs.

    12. NDNicolas Dessaigne7:02

        Is that the new frontier for, for like models?

    13. ELEric Landau7:05

        Y- yeah, so what happened with digital AI and, and LLMs is that, um, there was a lot of data availability. And so, um, what it proved was that if you throw data and compute at a problem, then these systems can be extremely performant. And so the only, like, risk was to throw additional compute because you already had a lot of data in the internet data and, and human language. With physical AI, it's actually the opposite. Now we have all the compute infrastructure, but you need the data to actually get to the scaling law. So for us, it's a great moment because ultimately, to get to the ChatGPT moment for physical AI, you need the data to unlock it, and we are a data infrastructure company for, for physical AI.

    14. NDNicolas Dessaigne7:40

        So you're helping all of these physical AI companies to generate the data they need to train their model.

    15. ELEric Landau7:44

        That's right.

    16. NDNicolas Dessaigne7:45

        That, that's awesome. Uh, uh, and you're, you're also launching a new offer for that, right, to, to companies. Can you tell us more about what, uh, what it is?

11. 7:53 – 8:37

    New offering: Bay Area R&D facility for data collection and robot training environments

    1. UWUlrik Waage7:53

       We work with a lot of the world's leading physical AI companies, and one of the, uh, the parts of the, the training pipeline that we've sort of like stayed away from, at least up until recently, has been, um, pre-training and data collection. Pre-training and data collection in LLMs is very simple, right? You go and you scrape the internet, and then you can train, like, a big model, and that's how a lot of the, uh, language model companies kind of caught up with each other very, very quickly. For physical AI, like that's a much more difficult problem because you have to collect the real-world embodied data. And so we have, uh, opened a, an R&D facility in the Bay Area actually to, um, start building out, uh, robots, environments that they can like go and train the robots, quote, unquote.

    2. NDNicolas Dessaigne8:32

       But you don't build the robots. You just-

    3. UWUlrik Waage8:34

       No

    4. NDNicolas Dessaigne8:34

       ... provide the place to capture the data and work on it?

12. 8:37 – 9:53

    From training to operations: post-deployment observability and exception handling

    1. UWUlrik Waage8:37

       That's exactly right, yeah. So we don't, uh, build the robots, but we work with the robotics companies. Like, they need like different types of environments they need to train the robots in, and, um, that's, uh, obviously a huge opportunity. And then there's a second part of that which is the post-deployment phase, which is where I think we're getting to very soon in physical AI. So a lot of these robotics companies, they're, I think, getting pretty, pretty close to getting to market. I think there's a, a YC company actually that we work with that is already in, uh, production that we can, can also talk about. But once the robot's in production, they also need things like, uh, exception handling. They need, uh, help with like observability. They need a whole different types of, uh, services that will be coupling the real world with the, with the, with the digital world. And like we're building the real and, uh, physic- r- sorry, the real and digital world infrastructure to really make that happen.

    2. NDNicolas Dessaigne9:22

       So it's kind of like, uh, bring your own robot to your facility so that you are going to collect the data to train it, and then you are going to be able to QA once you-

    3. UWUlrik Waage9:30

       Yeah

    4. NDNicolas Dessaigne9:31

       ... got the model trained.

    5. ELEric Landau9:31

       Yeah, you get the, the data flywheel.

    6. NDNicolas Dessaigne9:34

       That's, that's awesome. So people cannot do that at home [laughs] in their own company?

    7. ELEric Landau9:38

       It, it's, it's operationally quite difficult to do, uh, especially to do at scale. Um, one, you need the software and the platform to be able to kind of work-

    8. NDNicolas Dessaigne9:46

       Yeah

    9. ELEric Landau9:46

       ... through the data, which is what we've spent the last-

    10. NDNicolas Dessaigne9:48

        Okay

    11. ELEric Landau9:48

        ... five years building, and so that gives us, um, a very strong competitive edge in the, in the space.

13. 9:53 – 11:03

    Why Encord can be a platform: the data flywheel and consolidated pipeline view

    1. NDNicolas Dessaigne9:53

       Do you get some, um, uh, what do you call that, network effect? Like, like learning from one company, uh, to help everyone else?

    2. UWUlrik Waage10:01

       Yeah, and I think it's, it's also more profound than that because, um, when we work with customers, they index all the data on our platform. They curate all the data on our platform. They annotate the data on our platform, and they embed their model to do the pre-labeling. So once you have like a complete view of your entire data flywheel, you can start to like actually automate the entire, like, stack, right? And so the faster you can get this, like, flywheel turning, the faster you can get your model to production, the faster you can get your product working, and the faster you can, like start to generate, uh, revenue. And you can only do that because you have a consolidated single view of your entire, like, um, model pipeline, all the way from pre-training to post-training to deployment.

    3. NDNicolas Dessaigne10:35

       Anything that's really different from what people were using for, like LLMs, for example? I c- I guess you get a lot more data volume?

    4. ELEric Landau10:45

       Yeah, there's a lot of complexity as well with just working with multiple modalities at once.

    5. NDNicolas Dessaigne10:49

       Mm-hmm.

    6. ELEric Landau10:49

       So, uh, text is low bandwidth. It's kind of easy to see and visualize. But video, sensor data, audio data, especially when you're working together in groups, is quite difficult to, um, to, to interact with a- at scale.

14. 11:03 – 12:20

    Humans in the loop: frontier labeling, supervision, and higher safety stakes

    1. NDNicolas Dessaigne11:03

       Speaking of that, like, uh, do, do you still need humans in the loop?

    2. UWUlrik Waage11:06

       You still do need humans in the loop, uh, for a few different things.

    3. NDNicolas Dessaigne11:09

       Okay.

    4. UWUlrik Waage11:09

       So, uh, principally, you need humans at the frontier. So what you're seeing now in language models is you have a bunch of people that are, um, going in and like doing reasoning problems and whatnot to kind of help make the models better. We're still like in the very infancy of physical AI, right? So like the types of tasks that people are doing is, are things like folding laundry, and like they're emptying the dishwasher. So we're still like-

    5. NDNicolas Dessaigne11:28

       That's more like, uh, for the training data.

    6. UWUlrik Waage11:30

       Exactly, so we're still at the infancy, whereas like we're... In language models, we're really pushing the frontier. Like the tasks are much more different.

    7. NDNicolas Dessaigne11:35

       It's more like a head start to learn than learn.

    8. ELEric Landau11:36

       I would even say we want humans in the loop because ultimately, like we want to be the managers and supervisors of, of these AI systems. So you want to leave a, a space for humans to be, and to have some control. Uh...

    9. UWUlrik Waage11:47

       Yeah, then exception handling.

    10. ELEric Landau11:48

        Yeah.

    11. UWUlrik Waage11:49

        So all the times where like, you know, the model gets something wrong, in language models and, and digital AI systems, the stakes are not that high, right? Because, uh, ChatGPT gives you like a bad output. You can give it a thumbs up or a thumbs down. Nothing breaks.

    12. ELEric Landau12:02

        [laughs]

    13. UWUlrik Waage12:02

        But if you have a model in the real world hallucinate, right? That's a self-driving car. That could be a drone that falls down from the sky. Like a, a, a bunch of different things can happen. So the error tolerance for physical AI systems are actually much lower, which again, like c-Increases the bar quite substantially for, like, the quality requirements of the data that they're, um, that they're trained on.

15. 12:20 – 13:47

    Customers and value proposition: speed to market and better models (with example)

    1. NDNicolas Dessaigne12:20

       So is that going to be, um... Today, your customers are these, um, robotic companies mostly then, or labs companies? Like, who are the-

    2. UWUlrik Waage12:29

       Yeah, across-

    3. NDNicolas Dessaigne12:29

       ... perfect customers for you?

    4. ELEric Landau12:30

       Um, many different embodied AI applications. So in robotics, self-driving cars, autonomous systems.

    5. NDNicolas Dessaigne12:36

       Yes.

    6. ELEric Landau12:36

       Uh, wherever AI is in the real world, that's a, a relevant customer for us.

    7. NDNicolas Dessaigne12:41

       And, uh, what are customers buying really here? Is that the speed, accelerating their, the building of their models? Is that kind of like the performance itself of the model, the workflows? Like, what, what really are they buying when they're using Encord?

    8. UWUlrik Waage12:55

       Yeah, so ultimately, like, they're buying, like, uh, getting their model to market faster-

    9. NDNicolas Dessaigne12:59

       Okay

    10. UWUlrik Waage12:59

        ... and making the model better. And the reason that we can help them do that is because we already have, like, all the infrastructure they need to, uh, get the right data, curate the data, annotate the data, and all the rest of it, so that they can focus on building what they do best, which is building a, an actual robot, building a self-driving car, and like, not building data infrastructure.

    11. NDNicolas Dessaigne13:18

        Is that, uh, kind of like, uh, the main, uh, the main use cases? Like, any, uh, any example companies you can share with us?

    12. UWUlrik Waage13:22

        Yeah, there, well, there's actually a YC company called, uh, Weave Robotics-

    13. NDNicolas Dessaigne13:25

        Okay

    14. UWUlrik Waage13:25

        ... which I think is the first company to build or bring an actual laundry-folding robot, uh, to market.

    15. NDNicolas Dessaigne13:31

        To market.

    16. UWUlrik Waage13:32

        And, uh, they're using us for exactly that. Um, so they bought our physical AI data platform, and they, um, uh, have been a very happy customer so far. And, uh, obviously also very excited to be working with them because they, uh, are, I think, like, the first, uh, consumer household-

    17. NDNicolas Dessaigne13:45

        That's exciting

    18. UWUlrik Waage13:45

        ... robot in production.

16. 13:47 – 18:42

    Series C rationale and the long-term vision: powering the robot economy + founder lessons

    1. NDNicolas Dessaigne13:47

       And so the other important news here that you announced recently was your big Series C, right?

    2. UWUlrik Waage13:52

       Yeah.

    3. NDNicolas Dessaigne13:53

       Uh, why now? What are you going to do with that money?

    4. ELEric Landau13:56

       Well, the physical AI kind of world is opening up now. There's a lot of attention that's going in the space, a lot of investment that's going in the space, and we have the best platform and solution for it.

    5. NDNicolas Dessaigne14:05

       Okay.

    6. ELEric Landau14:05

       So we just wanted to accelerate and make sure that we capture the, the full size of the opportunity.

    7. NDNicolas Dessaigne14:10

       Is that what, uh, investors, uh, saw in you, kind of like the, the right bet for that future vision of the world?

    8. ELEric Landau14:17

       Uh, I think so. And, uh, one thing that, um, is, is almost, like, silly to say is that, uh, for as much investment that's going into physical AI-

    9. NDNicolas Dessaigne14:26

       Yeah

    10. ELEric Landau14:26

        ... it still is actually small because-

    11. NDNicolas Dessaigne14:28

        Mm-hmm

    12. ELEric Landau14:28

        ... um, 80% of the world's economy is just moving things around in the physical world or doing things in the physical world. So it's, it's gonna be a just enormous economic opportunity.

    13. NDNicolas Dessaigne14:38

        And so we waited, like, what, 10 years, 15 years to get, uh, Cruise, uh, or Waymo, uh-

    14. UWUlrik Waage14:45

        Yeah

    15. NDNicolas Dessaigne14:45

        ... actually working. Uh, how long before we can, uh, get this new next generation of robots?

    16. ELEric Landau14:50

        There are a lot of robots that do work in, um, in factory settings and logistics. Some of our customers that, that we have are actually deploying at scale production. The generalizable robots that are in, um, human houses, like the humanoids, uh, that will maybe take a few, few more iterations.

    17. NDNicolas Dessaigne15:07

        [laughs]

    18. ELEric Landau15:07

        But, uh, I think it's probably going to have a similar trajectory as what self-driving cars had, where, um, there's a lot of attention, kind of, um-

    19. NDNicolas Dessaigne15:14

        It's hype cycle

    20. ELEric Landau15:14

        ... uh, hype now, and then there's gonna be some consolidation, uh, realize h- what it takes to actually get these to work, um, properly in production. And then in a few years, it'll be at the kind of slope of enlightenment, where, uh, everyone will have a, a humanoid in their house.

    21. NDNicolas Dessaigne15:27

        So not happening before the end of this year. Like, maybe take a little more time.

    22. ELEric Landau15:30

        Uh-

    23. UWUlrik Waage15:30

        We'll see.

    24. ELEric Landau15:30

        We'll see. Yeah, yeah.

    25. UWUlrik Waage15:32

        We'll see. [laughs]

    26. ELEric Landau15:32

        Yeah. Uh, everything is g- is moving much faster than we expected, and that was one of the things that we're very surprised by internally.

    27. NDNicolas Dessaigne15:37

        And what does it mean for you at Encord? Like, uh, where is the company going to be in a couple of years?

    28. ELEric Landau15:41

        We basically want every sing- single piece of physical AI data to go through our system, and we wanna work with every single physical AI company in the world. So similar to, like, how, um, Stripe wants every kind of financial transaction-

    29. NDNicolas Dessaigne15:55

        Mm

    30. ELEric Landau15:55

        ... to, to go through them. Uh, that's, that's where we wanna be in, in a few years.

Episode duration: 18:43