How Factory is building autonomous software with Claude

1. 0:00 – 0:19

   Intro

   1. MGMatan Grinberg0:00

      I think that's like some of the most human pieces of art or pieces of technology. Rethink what we've been doing before and figure out this entirely new angle, and we have time to do that by using models like Opus and Fable, where they take on a lot of the work that used to occupy so much of our time.

   2. KGKai Gill0:15

      [upbeat music]

2. 0:19 – 1:48

   From string theory to founding Factory

   1. KGKai Gill0:19

      Matan, super great to be here with you today. We were talking about this a little bit before we sat down, but you have such an unusual background. In twenty twenty-three, there's this pivotal moment where on one side of it, you were a PhD student studying the AdS/CFT correspondence in ten-dimensional black holes and string theory. On the other side of it, you become a founder of Factory. What happened?

   2. MGMatan Grinberg0:40

      Yeah. So, uh, I guess growing up in the Bay Area, the culture is startups, and I was, you know, a rebellious child. I was like: "I want nothing to do with startups. I'm gonna do the most opposite thing." So I got into theoretical physics, um, and did that for about, you know, twelve years.

   3. KGKai Gill0:57

      As one does. [laughs]

   4. MGMatan Grinberg0:57

      Yeah. Yeah. Yeah. Right. That dividing line between March and April of twenty twenty-three, I think that was the culmination of realizing that while theoretical physics and string theory is extremely beautiful, um, as you've noticed, you know, I was working on ten-dimensional black holes, and we don't live in ten dimensions, nor do we live near a black hole. But then I took a seminar in program synthesis, which we now call code generation, and that just completely nerd sniped me.

   5. KGKai Gill1:20

      [laughs]

   6. MGMatan Grinberg1:21

      Um, head over heels, was obsessed with that, and then it just became very clear in twenty twenty-three that to solve this problem, academia was not the place. You had to go into the industry. You had to work with some of the world's largest enterprises, some of the messiest code bases where some of the code is thirty years old, some of it it's brand new, some of it, you know, is in COBOL, half of it is in Python. Like, just this very weird dynamic situation is where the code generation problem becomes the most interesting.

3. 1:48 – 2:22

   The mission: autonomy for software engineering

   1. MGMatan Grinberg1:48

      The mission that we have in the very first deck that we ever made is the same as our mission today, which is to bring autonomy to software engineering. Software engineers are some of the best problem solvers in the world, but so much of their life, especially in like a large enterprise, is not as high leverage as it could be.

   2. KGKai Gill2:06

      You had this conviction from day one that this is gonna be an autonomous software factory, even though at the time the, the models weren't really there yet.

   3. MGMatan Grinberg2:14

      You know, models are gonna get better. They're gonna be able to do more and more of these tasks. Before we were selling division of agents. Now it's like everyone knows.

4. 2:22 – 2:58

   Lessons from being too early

   1. KGKai Gill2:22

      That two-year period between twenty twenty-three to twenty twenty-five was your journey through the desert. And I think you said elsewhere that sometimes being too early is the same as being wrong. What did that desert teach you?

   2. MGMatan Grinberg2:31

      I was obsessed with problems and, like, solving problems. That was all I cared about, so this is really the first job I've ever had. I knew nothing about anything relating to building a company, whether it's hiring, you know, engineering, building product, sales, marketing, none of it. If the timing was right and the world was ready, I was the wrong founder in my mind because I didn't know anything. We would have missed the opportunity. So it's kind of a blessing in disguise because I think it allowed me the time to grow and learn the things that really matter to now

5. 2:58 – 4:09

   Droids and the "dark factory"

   1. MGMatan Grinberg2:58

      scale.

   2. KGKai Gill2:58

      You've mentioned this concept of these droids working in a dark factory, lights off twenty-four hours a day, seven days a week.

   3. MGMatan Grinberg3:05

      Yep.

   4. KGKai Gill3:05

      How are you building the product now for, uh, that future?

   5. MGMatan Grinberg3:10

      So, like, as you get used to delegating tasks to agents that can go for days or weeks on end, you start getting used to the idea that, oh, there's something that you're not gonna go and touch for a couple days at a time. And so it slowly starts building that behavior change. And also when they see that the output is actually meaningful, they start to build some of that trust. Trust is going to become all the more important. You're now having systems that are doing things without you looking, without you necessarily explicitly approving every time. Having a, a software factory that is kind of always monitoring, seeing, "Oh, hey, we just deployed some change, and now the completion rate through this step has gone down. Let's go and spin up a sandbox where we can go and kind of test and see, oh, why did that happen? We found a candidate solution. Let me go ping you and show you what might fix it," and then kind of deploy it and monitor again. I think it's so important to meet our users where they are and, like, respect their intelligence as developers. They know their business more than we do, and the best we can do is, like, allow them to configure it and morph it to the silhouette of their organization.

6. 4:09 – 4:58

   A COBOL modernization with 90% less human effort

   1. KGKai Gill4:09

      Do you have any like hero examples of a droid doing something like really impressive or crazy end to end for a customer?

   2. MGMatan Grinberg4:16

      Every single bank has a, you know, backlog list of these crazy COBOL modernizations that they need to do, and it's all like no engineer wants to be assigned to this because that is like, "Oh, God, for the next year, my life is gonna suck dealing with this." With one such bank, we did one of these modernizations. As it was working, you know, it was extracting lessons. This is the way this bank does things. This is how we can learn from it in the future. Um, and that was like one of our engineers sitting next to some of their engineers to go and kind of do that first pass. And then after that, the next one, the amount of human involvement was like ninety percent less. That to them was like a magic moment. And there's like tears of joy in these engineers' eyes-

   3. KGKai Gill4:53

      [laughs]

   4. MGMatan Grinberg4:53

      ... because, like, working on those migrations is the worst.

7. 4:58 – 5:29

   When Droids need frontier models

   1. KGKai Gill4:58

      Where do you feel that the droids need to be very, very smart-

   2. MGMatan Grinberg5:03

      Yeah

   3. KGKai Gill5:03

      ... and they need the frontier?

   4. MGMatan Grinberg5:05

      If you have the CTO of a company implement every line of code, that is not an efficient use of their time.

   5. KGKai Gill5:11

      Right.

   6. MGMatan Grinberg5:11

      And it turns out that, like, typically, we will have structure where there is someone who has maybe more context about a problem or more history of, like, problem-solving, where they're gonna come up with a general, like, systems solution. It's sort of analogous here where I myself, like, if I'm doing planning or something like pretty complicated where it's like I need to orchestrate a lot of moving parts, I'll

8. 5:29 – 6:31

   What Opus unlocks: less human oversight

   1. MGMatan Grinberg5:29

      use Opus.

   2. KGKai Gill5:29

      What has Opus enabled you to build with droids that you weren't able to before?

   3. MGMatan Grinberg5:34

      I think the biggest unlock, and this is the most important part, part for us, is like reducing human oversight. Like, that planning is a lot more difficult, or you'll only find out if you went awry, like, billions of tokens down the line, right?

   4. KGKai Gill5:48

      Yeah.

   5. MGMatan Grinberg5:48

      And I think there's kind of like the gut sense of like you can kind of trust the plans a little bit more to the point where you literally might not even look at them, and you're like, "Yep, sounds great. Let's go and do it." That's been pretty cool to see. Before it was like, oh, you see it, but you're like really closely monitoring to now it's like... And it's only like a very crazy corner case where you'll go back and be like, "Wait, well, what did we do in the plan?" It rarely, rarely happens. There are certain things that are kind of like the brutal reality of working at a company where there are just things that you have to do that need to meet a certain kind of reliability threshold. And what's cool is with Opus and Fable- It takes a lot of those things that previously you had to spend a ridiculous amount of time doing, and it's like, "I got you covered there." You know, every company wants to ship 30 things, but they can only ship three in a quarter because of the resourcing that they have.

9. 6:31 – 7:11

   Shipping 300 things and rethinking what matters

   1. MGMatan Grinberg6:31

      Well, now you can ship 300 things.

   2. KGKai Gill6:33

      Yeah.

   3. MGMatan Grinberg6:33

      But like will those 300 things actually make a difference? Are they gonna actually make happier customers or better relationships with your customers or kind of your end users? And I think it shifts the question for the humans to now be like, "What really is our core competency? How can we deliver on that better?" And that is a thing that's really hard, 'cause sometimes we put ourselves in such a box of like, "This is all that we can do." To me it's very exciting because I think that's like some of the most human pieces of art or pieces of technology, rethink what we've been doing before and figure out this entirely new angle, and we have time to do that by using models like Opus and Fable where they take on a lot of the work that used to occupy so much

10. 7:11 – 8:12

    The Anthropic partnership

    1. MGMatan Grinberg7:11

       of our time.

    2. KGKai Gill7:11

       When you think of Anthropic and you think of that partnership, what's top of mind for you?

    3. MGMatan Grinberg7:15

       I think the biggest thing is like reliability and in terms of how we work. Like understanding that, you know, there are a lot of things that we have in common in terms of what we're excited about, and making sure that it's set up such that like, "Hey, here are the important things we should spend our time on. Here's the things that you guys should spend your time on." We very clearly want to see a world where like human potential is unlocked and like this ambition is raised and there are certain constraints that are required. Like there's some safety constraints that are required. There's some messaging that really matters. Sometimes these tools can be scary. Like they're very powerful and that is inherently scary, but it's also like a beautiful thing for the world that we should highlight the benefits that are coming from it and how Droid can be better with Claude and how Claude can be better with Droid. And it's really important that the people that we partner with take it as seriously as we do as like a today problem. Like some people are like, "Oh, you know, eventually it might be crazy." It's like, no, today this is like extremely important across every piece of this AI engineering puzzle.

11. 8:12 – 9:31

    What's blocking the dark factory

    1. KGKai Gill8:12

       So like looking ahead a year in advance when you have these like dark factories, we're not fully there yet, right? What are the blockers? What are like the bottlenecks right now?

    2. MGMatan Grinberg8:21

       The way I think about it, there are like three kind of concentric circles. At the core there is the model, then there's the harness, and then there's kind of like the company and the workflows of the company as a whole. And these are all growing very quickly, right? Like the model capabilities now versus two years ago, it's insane. Completely, completely different. A model today could do what a model plus a harness could do two years ago. It's a cool call to arms for everyone to raise the scale of their ambition, right? Because as models get better, if you are lazy and you like don't change your harness, like it's kind of, you know, it's gonna be superseded.

    3. KGKai Gill8:52

       As these models get smarter and smarter, they start to absorb more of the harness. How do you build for that? And then also how do you think about the leverage that that type of really intelligent model can give you?

    4. MGMatan Grinberg9:03

       We cannot settle. Like things are moving way too quickly. Like we need to always have like in one hand a sledgehammer to whatever we thought was like the best solution and like every month we need to rethink.

    5. KGKai Gill9:14

       Yeah.

    6. MGMatan Grinberg9:14

       Is this still optimal? As these models are getting better... Like and also having a deep partnership, right? Like understanding and having this relationship where we can see, hey, these are the types of problems they're thinking about so we can kind of trust, okay, this is gonna be going up there. Let's make sure we're filling this hole in the meantime so that we can kind of ride that wave together.

12. 9:31 – 10:36

    Advice for would-be founders

    1. KGKai Gill9:31

       I want to go back to the very beginning when we were talking about your pivotal moment that led you onto this path. There are a number of would-be founders watching this and their pivotal moment is right ahead of them. What advice would you have to those founders?

    2. MGMatan Grinberg9:45

       This is a bit funny, but anytime a person asks me if they should start a company, I always say no.

    3. KGKai Gill9:49

       Say no. Yeah. [laughs]

    4. MGMatan Grinberg9:49

       I always say no. Because if they need my affirmation, like, you know. I just like really getting obsessed with problems and then everything is just so much fun. Find the thing that you literally cannot stop thinking about and just keeps you up at night. And the beautiful thing about the era in which we're in now, you know, normally you could say, "Oh, like, you know, the thing that keeps me up at night is I want to be an astronaut," or whatever, and 50 years ago it's pretty unrealistic. Like it is extremely realistic today. And even like if you have the most out there like kind of crazy ideas or, and that's the only thing that keeps you up at night, it's actually way more achievable than it's ever been. Find the thing that you get obsessed about, the intellectual curiosity just pours out of you, and I think the rest kind of just will, will come from that.

    5. KGKai Gill10:31

       Thank you so much. This has been an amazing conversation. I really appreciate the time.

    6. MGMatan Grinberg10:35

       Thank you for having me. This was fun.

Episode duration: 10:38