How Jev Turns AI Into Software That Gets Things Done

Diogo Almeida 0:00 Where the fuck is all the automation?

Martin Casado 0:01 Yeah.

Diogo Almeida 0:01 AI is so unbelievably smart, and yet it's so useless at all other stuff.

Martin Casado 0:06 It doesn't matter how much AI coding agents you use, the software actually isn't getting better. [laughs]

Diogo Almeida 0:10 [laughs]

Martin Casado 0:11 Maybe you're writing it faster. It's like-

Diogo Almeida 0:12 Yes

Martin Casado 0:12 ... arguably getting worse.

Diogo Almeida 0:13 OpenAI has been trying to automate customer service since 2020. What I want instead is smart software. I want to expand what software itself can do, such that things that should be automatable can then be automatable.

Ben Horowitz 0:26 My favorite thing that you guys say is, "We build prod, not God."

Martin Casado 0:29 [laughs] So good.

Ben Horowitz 0:31 Because if we had any other kind of, like, big lab leader, even if they had joy, they would cover it up. [laughs]

Martin Casado 0:37 [laughs]

Ben Horowitz 0:37 And then your view is so different. You're like, "No, we're gonna create a way better world."

Diogo Almeida 0:43 For nuanced reasons, I don't think we are on the path of RSI. In the SaaSpocalypse story, um-

Ben Horowitz 0:52 Today-

Diogo Almeida 0:52 [laughs]

Ben Horowitz 0:53 ... we have the founder and leader of TypeSafe, Diogo, with us, um, who, uh, is a bit of a hero to both Martin and me.

Martin Casado 1:04 Yes.

Ben Horowitz 1:05 Um-

Martin Casado 1:05 Yes

Ben Horowitz 1:05 ... he is not only building, like, a really interesting product, uh, but creating what we think is a very important movement. So we're super excited about today. Welcome, Diogo.

Martin Casado 1:16 Thanks for coming. Yeah.

Diogo Almeida 1:16 Thank you.

Ben Horowitz 1:17 Yeah. And, uh, maybe you can give us kind of a, a brief on, um, just, you know, what is Jev? What is TypeSafe? Why is it important?

Diogo Almeida 1:29 Is this, uh, curse-friendly or no?

Ben Horowitz 1:31 Yeah. Yeah, yeah.

Martin Casado 1:32 Oh.

Diogo Almeida 1:32 Oh, okay. Cool.

Ben Horowitz 1:33 What the fuck are you talking about? [laughs]

Diogo Almeida 1:35 Oh, okay, okay, okay, cool. So I was actually asked-

Martin Casado 1:37 Anything else, and yeah, yeah

Diogo Almeida 1:38 ... uh, for, like, a elevator pitch, which I tend to ramble on, and I don't do well. But, like, I realized my favorite e- elevator pitch for Jev is-

Martin Casado 1:45 Oh, yeah

Diogo Almeida 1:45 ... where the fuck is all the automation?

Martin Casado 1:46 [laughs]

Diogo Almeida 1:47 Like, this is, like, so-

Martin Casado 1:48 Oh

Diogo Almeida 1:48 ... unbelievably tragic.

Martin Casado 1:50 Yes.

Diogo Almeida 1:50 You know? So much intelligence.

Martin Casado 1:51 Yeah.

Diogo Almeida 1:52 AI is so unbelievably smart, and yet so... I, I... Not that I hate on chatbots or coding agents. I love them myself, but it's like th- it's so useless at all other stuff, and it's, it's tragic. It's tragic-

Martin Casado 2:05 Yes

Diogo Almeida 2:05 ... so that, you know, we have so much, like, diamond in the rough, but not polished for work. That's, that's... But TypeSafe is making AI for software. You know-

Martin Casado 2:15 Yes

Diogo Almeida 2:15 ... we want to make AI powerful, not just for humans in the loop, but to actually build real software, and Jev, to us, is our first model in this whole space to make it-

Martin Casado 2:25 Yeah

Diogo Almeida 2:25 ... way, way better, to, like, make automation.

Ben Horowitz 2:28 Yeah, and so it, it's been interesting because it's kind of caught fire in software world. So the, you know, [laughs] one of the things that made us go, "What the hell's going on here?" is, like, every developer we know is calling us and going, "Oh, this is fricking awesome."

Martin Casado 2:42 It's great.

Ben Horowitz 2:43 Like, it's fast. It's, it's great. Everything's better. Um, and h- then how does that, uh... 'Cause, 'cause everybody thinks of, "Well, we've got Cloud Code. You know, we've got Codex. Don't we already have that?" Like, what's the difference, and then how does that lead to real automation?

Diogo Almeida 3:00 Ooh, I wish I had, like, a, some slopped visuals-

Ben Horowitz 3:05 [laughs] Yeah

Diogo Almeida 3:05 ... because I have, like, a favorite slopped visual for this.

Ben Horowitz 3:07 Yeah.

Diogo Almeida 3:07 So I like Cloud Code, uh, and Codex.

Ben Horowitz 3:09 Mm-hmm.

Diogo Almeida 3:10 Um, I love the description from Gary Tan and them. It's just-in-time software. You know, incredible way to describe what they're doing.

Ben Horowitz 3:17 Mm-hmm.

Diogo Almeida 3:17 Like, it makes software on the fly, and you can, like, program software in natural language, but it has the same expressive power of software.

Ben Horowitz 3:24 Mm-hmm.

Diogo Almeida 3:24 Um, what I want instead is smart software. Like, i- instead of, like, automating software engineering, I want to expand what software itself can do, such that things that co- should be automatable can then be automatable. And, like, in a more, um, flowery language, like, I wanna express things like intent. I want to, like, expand the vocabulary of what we can do, and I can talk about, like, all sorts of, like, weird sci-fi things I want.

Ben Horowitz 3:49 Hmm.

Diogo Almeida 3:49 But, like, programming is, like, hyper-specifying, like, valuable things and then infinitely replicating them. It's so freaking cool, and-

Ben Horowitz 3:57 Yeah

Diogo Almeida 3:58 ... I wanna just make that more, you know?

Ben Horowitz 4:00 Yeah. Oh, interesting. So, so one way to think about it is instead of kind of a tool that,

Ben Horowitz 4:07 uh, somewhat replaces a software engineer with a faster, maybe not even as good software engineer, what you're saying is, "No, no, no, we're gonna super empower the software engineers we have to write way, way better, more interesting things."

Martin Casado 4:25 Yeah. I, I actually-

Ben Horowitz 4:26 Yeah.

Martin Casado 4:26 I mean, yeah, yeah. So this is s- by the way, I, I just think so many people miss this point, and it's such-

Ben Horowitz 4:31 Yeah

Martin Casado 4:31 ... a subtle point, and it's so important to actually tease it out, which is, um, if you use something like Cloud Code or Codex, which is great, or Cursor, which is great-

Diogo Almeida 4:38 [laughs]

Ben Horowitz 4:39 Yeah

Martin Casado 4:39 ... they write code, but that code is the same thing a human being would have write. Maybe it's better, maybe it's worse, but it's basically still code, just like code looked 10 years ago.

Diogo Almeida 4:46 Yeah.

Ben Horowitz 4:47 Yeah.

Martin Casado 4:47 A- and the thing with Jev is whether or not you're Cloud Code or a human, you have this new primitive, this new thing that you stick in your code that actually expands, like, the power of software. So instead of, like, writing code, it is something that you include in your code.

Diogo Almeida 5:01 Hmm.

Martin Casado 5:02 Which-

Diogo Almeida 5:03 Go ahead. No, no, you go ahead. Go ahead

Martin Casado 5:03 ... no, no, you go ahead. Well, one, two, which by the way is interesting because it's this very powerful primitive, which would be great if you explain, but it's also a little bit different than, like, you know, how programmers think. For example, like, it has this notion of, like, you know, probabilities or, you know... And, like, so, you know, so maybe-

Ben Horowitz 5:19 Oh, so an intelligent layer inside the software.

Martin Casado 5:22 Yeah, think of, like, like a libr- yeah, like a library that you can, like, use natural language to describe what you want, and then you give it kind of a state machine, and then it will, will choose what to do with some confidence levels, which we kind of haven't really had before-

Diogo Almeida 5:36 Yeah. Yeah

Martin Casado 5:36 ... like, so ubiquitous. So maybe...

Diogo Almeida 5:39 Ooh, there's a lot of tricks there. I will-

Martin Casado 5:41 [laughs]

Diogo Almeida 5:42 ... jump into one thing first, which is-

Martin Casado 5:43 Yeah

Diogo Almeida 5:43 ... I love the first thing you said. Like, you know, in the direction of where the fuck is all the automation.

Martin Casado 5:49 Yeah.

Diogo Almeida 5:49 I love software so much.

Martin Casado 5:51 It's the best.

Diogo Almeida 5:51 I wish I could be writing it all day.

Martin Casado 5:53 Yes. [laughs]

Diogo Almeida 5:53 It's... Would not recommend being a CEO to people, but whatever.

Martin Casado 5:56 [laughs]

Diogo Almeida 5:57 Um, and also, like, it's- ... wild that AI is so cool and software has been unchanged in ten years.

Martin Casado 6:03 Yeah. No, exactly.

Diogo Almeida 6:04 You know, like-

Martin Casado 6:04 Yeah, yeah

Diogo Almeida 6:05 ... like, like that to me, like no one can like square this together, and the most we can do is add like a little chat bot in the side sometimes that can take actions, but not all actions, 'cause some of the actions are not reliable enough.

Martin Casado 6:15 Okay.

Diogo Almeida 6:15 So I just wanna give like that tiny aside. Um, I love the point... Uh, I'm gonna jump back to the point about like this is a little bit of a different way to think about it. Um, yes, I think that machine native doesn't exactly match bits perfectly, and like that's actually the art form that we are trying to do. Like I, in our onboarding on day one, I draw like the Venn diagram of like what AI is good at, what is valuable in code. We're in the middle.

Martin Casado 6:42 Yeah.

Diogo Almeida 6:42 So, you know-

Martin Casado 6:43 No

Diogo Almeida 6:43 ... we don't output like, you know, extrapolated floats, for example, 'cause like AI is just bad at that.

Martin Casado 6:48 Sure.

Diogo Almeida 6:49 You know? But things like probabilities are not exactly novel, and it's similar to the, "Is Jev just a classifier" argument.

Martin Casado 6:57 Yeah.

Diogo Almeida 6:57 Jev is absolutely a classifier.

Martin Casado 6:59 Yeah.

Diogo Almeida 7:00 You know, like classifiers are sick. Classifiers were-

Martin Casado 7:02 [laughs]

Diogo Almeida 7:02 ... they're designed to be useful.

Martin Casado 7:04 [laughs]

Diogo Almeida 7:04 You know, they're designed to be useful, and actually it's the same interface-

Martin Casado 7:06 Yeah

Diogo Almeida 7:07 ... as like some of those ML concepts.

Martin Casado 7:09 Of course.

Diogo Almeida 7:09 Because these came from like practical people who are trying to make systems work.

Martin Casado 7:13 No.

Diogo Almeida 7:14 And what I'm seeing is happening now is that Jev actually... I, uh, my guess is that Jev probably is better than having like an MLE team from 2019-

Martin Casado 7:25 Of course

Diogo Almeida 7:25 ... making the stuff for you, and you can just program it on the fly.

Martin Casado 7:28 Right.

Diogo Almeida 7:28 Who knows what could be built? 'Cause like there were not that many good MLE team, L- MLE teams in 2019-

Martin Casado 7:33 No, very true

Diogo Almeida 7:33 ... to build like narrow things and to be able to like collect data sets and measure it and all of that. And it is just the beginning. There's... I feel like there's way... Like, like-

Martin Casado 7:41 Well, by the way, to this point, do you think there's a slider bar here where like on one end is like language in, language out, like we have today. On the other end is like a, like an existing imperative program, and then you can kind of move between the two? Or do you think like this is like the point in the design space, which is language in, kind of state machine out-

Diogo Almeida 7:56 Yeah

Martin Casado 7:56 ... which is gonna like solidify as the general purpose thing for programmers?

Diogo Almeida 8:00 Ooh, that's a tricky one. So I will say the answer in my heart.

Martin Casado 8:05 Yeah.

Diogo Almeida 8:05 The answer in my heart is that it is a slider. So i- in... And actually, when I design for, uh, the properties we have, I might have made mistakes due to my personal preferences.

Martin Casado 8:15 Yeah.

Diogo Almeida 8:15 But like intelligence per dollar is my north star right now.

Martin Casado 8:17 Hundred percent.

Diogo Almeida 8:18 And it could be wrong, just to be clear.

Martin Casado 8:19 Mm-hmm.

Diogo Almeida 8:19 Intelligence per second might be more valuable in the short term. But like even like our interface, like calling the input state, this was intentional. Like it's to say that this-

Martin Casado 8:28 Oh, that's great. I didn't even catch that.

Diogo Almeida 8:29 Yes. It's meant to be the inside of programs.

Martin Casado 8:31 Oh, I love that. [laughs]

Diogo Almeida 8:32 So, so in my heart-

Martin Casado 8:34 That's so good

Diogo Almeida 8:34 ... uh, 'cause like, so we are really optimizing. A lot of the work I do is for even more complicated arrangements of the internals of program state. Can you put intelligence in there? I think that this is going to be an ever-present battle to have. I... You know, we're very intentional about our design. And also pragmatically, I think certain things happen, like it's easier to make an AI at-

Martin Casado 8:54 Yeah

Diogo Almeida 8:54 ... these milliseconds, so it'll be more like a database-

Martin Casado 8:56 Okay

Diogo Almeida 8:56 ... for a while than like a standard library thing. But I would love it to be a standard library thing too.

Martin Casado 9:01 Can I, can I just pull back very quick? Like what is the alchemy that creates a Diogo? I mean, like-

Diogo Almeida 9:05 [laughs]

Martin Casado 9:06 ... you speak-

Diogo Almeida 9:07 When a man and a woman love each other. [laughs]

Martin Casado 9:09 [laughs]

Diogo Almeida 9:10 Well, like, but listen, you speak like an AI researcher, you spe- speak like a systems person, and you speak like a programmer, and normally these things have been like not super overlapping, and like you're, you're taking, you know, AI, which we've been pushing towards, you know, being a being, and you're making it a programmer's tool. So maybe a little bit about your personal journey that... My history into AI is somewhat unorthodox. Um, I was a mathlete. Um, I was a award-winning mathlete. Um, the way I describe it is I was good enough at... This is cringe. I was good enough at math to get girls.

Martin Casado 9:41 [laughs]

Diogo Almeida 9:41 So that's quite good.

Martin Casado 9:43 I didn't know that was a thing.

Diogo Almeida 9:43 Yeah, you have to get, you have to get quite good. Um-

Martin Casado 9:47 And what kind of girls do you get when you're that good at math? That's an actually-

Diogo Almeida 9:50 Whoa, that's... I-

Martin Casado 9:51 Our audience needs to know. [laughs]

Diogo Almeida 9:53 Oh, no. [laughs]

Martin Casado 9:55 We have to inspire the youth here. [laughs]

Diogo Almeida 9:56 Don't do it. You- don't do it. It's not worth it. Just be cool and chill and interesting.

Martin Casado 10:03 Yes.

Diogo Almeida 10:04 And don't overcompensate. Wow, I can't believe I said that. Um- [laughs]

Martin Casado 10:07 [laughs] Okay

Diogo Almeida 10:08 ... so I was a mathlete.

Martin Casado 10:09 Yeah.

Diogo Almeida 10:10 But I actually never... Oh man, this also is a little cringe. I never really liked math. I never really tried. I was just like big fish in little pond. And to me, math was ac- I... Math was always the path I was set on, but I hated it because it was always about, like, winning competitions. But then computer science is actually a lot like math. It's basically like math, but cool and useful and fun and interesting, and I still love giving algorithms interviews. It's, um, is it the best thing for me to do? I don't know. But do I love it? Yes. And does it, like, allow me to, like, suss people out, out really well? Yes, it does. So I love computer science. I consider myself to be computer scientist much more before AI researcher, despite my history.

Martin Casado 10:50 Hmm.

Diogo Almeida 10:51 And, um, like what actually got me into it was I also won a Kaggle competition, not from sophisticated math, but from like just automating like the fuck out of it.

Martin Casado 11:00 [laughs] Nice.

Diogo Almeida 11:01 Um, you know, like just-

Martin Casado 11:02 That's good

Diogo Almeida 11:02 ... like more nested loops, more... You know, like, it, it, I solved it like a systems problem.

Martin Casado 11:07 Wow.

Diogo Almeida 11:07 You know? So that event- eventually got me, um... Like I was forced to speak at NeurIPS, normally an honor, but I hated it 'cause I just wanted to be in the mines.

Martin Casado 11:16 Was that from the Kaggle thing?

Diogo Almeida 11:17 Yes.

Martin Casado 11:18 Oh, wow.

Diogo Almeida 11:19 Yeah.

Martin Casado 11:19 I didn't know that.

Diogo Almeida 11:19 Actually, the Kaggle host of it was Isabelle Guyon, who was the co-inventor of the SVM. Actually, I think the first author of SVM.

Martin Casado 11:26 Oh, yeah, interesting.

Diogo Almeida 11:27 I'm not 100% sure on first author. Um, and she just basically saw that I was like this person who really didn't fit into the research community and then adopted me and then showed me ev- like, like it got me to meet all the AI people and then, you know, my career was just pushed into that direction.

Martin Casado 11:44 And from there, OpenAI?

Diogo Almeida 11:46 No, it was, like, a startup with Jeremy Howard. Um-

Martin Casado 11:52 No kidding.

Diogo Almeida 11:53 Yes.

Martin Casado 11:53 I love Jeremy.

Diogo Almeida 11:54 Yeah.

Martin Casado 11:54 Fantastic.

Diogo Almeida 11:55 Cool.

Martin Casado 11:55 Yeah.

Diogo Almeida 11:55 Um, and then Google Brain for a while.

Martin Casado 11:58 Wow.

Diogo Almeida 11:59 And then, uh, retire for a while.

Martin Casado 12:02 Yeah.

Diogo Almeida 12:02 And then eventually I was, like, just kind of- Tired of not doing anything, and I was like, "You know what? Actually, AI's pretty damn fun." And I joined OpenAI because of that reason, uh, and it worked out really well.

Ben Horowitz 12:17 Amazing.

Diogo Almeida 12:17 Really, really well.

Ben Horowitz 12:18 Yeah.

Martin Casado 12:18 Right.

Ben Horowitz 12:18 Incredible. So you, you said something there that is so, um, unusual in today's world, which is AI is really, really fun, and then the company has such a different demeanor and view of AI than every- everybody else, and, um, my favorite thing that you guys say is, "We build prod, not God." Um-

Martin Casado 12:42 [laughs] So good

Ben Horowitz 12:42 ... because if we had any other kind of, like, big lab leader, they'd be, like, trying to... Even if they had joy, they would cover that.

Martin Casado 12:50 [laughs]

Ben Horowitz 12:51 And then your view is so different. You're like, "No, we're gonna create a way better world, and it's gonna be awesome, and there's gonna be... Not only are there not gonna be less jobs, there'll be more jobs, and they'll be way better jobs, and everybody's gonna have a great time." And, like, just be- being around you, like, you clearly believe that. So, so tell us about that and, like, what this... 'Cause for us, you know, TypeSafe Jev, it's, it's more than a company. It's a, it's a whole movement towards a positive future that most people in the AI world kind of don't like.

Diogo Almeida 13:24 Yes.

Ben Horowitz 13:25 Or, or, or, or, or they're not with it.

Diogo Almeida 13:27 The-- I think they don't get it.

Ben Horowitz 13:29 Yes.

Diogo Almeida 13:29 You know, like, the, it's just a classifier complaint. It's, like, an ML-level concern-

Ben Horowitz 13:33 Yes. [laughs]

Diogo Almeida 13:33 ... while everyone else is having, like, a Jev party. Um [laughs]

Ben Horowitz 13:36 [laughs]

Martin Casado 13:37 [laughs]

Diogo Almeida 13:37 Because it's like, holy shit-

Ben Horowitz 13:38 True. Yeah, yeah, yeah

Diogo Almeida 13:39 ... like, we can do all the things that we-

Ben Horowitz 13:40 Yeah

Diogo Almeida 13:41 ... wanted to do, and I don't, I think if you don't, like, get developers, it'll be hard to understand what's really going on.

Ben Horowitz 13:46 Yeah.

Diogo Almeida 13:46 So 100% I agree with that. I do think that there's, like, a pretty negative world painted that I obviously disagree with.

Ben Horowitz 13:54 Mm-hmm.

Diogo Almeida 13:54 I think it's, really comes from this, like, um, you know, mono-model Kool-Aid that everyone believes. I think AI-

Ben Horowitz 14:01 Right. We write one big brain to rule them all.

Diogo Almeida 14:03 [laughs]

Martin Casado 14:05 [laughs]

Diogo Almeida 14:05 That's one, that's one way to-

Ben Horowitz 14:06 Yeah

Diogo Almeida 14:07 ... it sounds, uh, sounds much more ominous.

Ben Horowitz 14:09 Yes.

Martin Casado 14:09 Yeah.

Ben Horowitz 14:10 Yes, but that's, uh-

Diogo Almeida 14:11 That's what people hear

Ben Horowitz 14:11 ... that seems to be, yeah, that-

Martin Casado 14:12 For sure that's what people hear.

Ben Horowitz 14:12 Yeah, that's what people hear, yeah.

Diogo Almeida 14:14 But, you know, like, will that one-- Is that one brain really on the path to rule us all? Like, we have not automated really basic things that I don't think we want people to be doing.

Ben Horowitz 14:23 Yeah.

Diogo Almeida 14:23 You know? Like, it, there's lots of really, really basic stuff, and I think that, oh, man, it, it, it pains me when the world is discordant with the reality.

Ben Horowitz 14:34 Yeah.

Diogo Almeida 14:34 And, like, part of the pain is, you know, on the, "Where the fuck is all the automation?"

Ben Horowitz 14:38 Yeah.

Diogo Almeida 14:38 Like, how can we have AI be so freaking smart?

Ben Horowitz 14:42 Mm-hmm.

Diogo Almeida 14:42 And, like, there's so much, so much financial incentive to automate stuff. Like, yeah, you could make-

Ben Horowitz 14:47 Yeah

Diogo Almeida 14:47 ... an excuse for diffusion. I don't buy it at all.

Ben Horowitz 14:49 Yeah.

Diogo Almeida 14:50 I shouldn't name names, but, like, that obviously is not true. Part of the problem is, like, the discordance with the reality, and the fact that AI has, like, so much potential is what made it really tragic for me that we had-

Ben Horowitz 15:01 Mm-hmm

Diogo Almeida 15:01 ... not released this. So now it's a, now it's, like, a little bit of a party for me. But, like-

Ben Horowitz 15:05 Yeah. Well-

Diogo Almeida 15:05 ... I was afraid of AI winter

Ben Horowitz 15:07 ... and all Jev users are like, [laughs] there's the happy AI, the people on Jev-

Diogo Almeida 15:12 Yeah

Ben Horowitz 15:12 ... and then there's the morose AI, the people-

Diogo Almeida 15:15 Oh

Ben Horowitz 15:15 ... who are not.

Diogo Almeida 15:16 Yeah, yeah, yeah.

Ben Horowitz 15:16 It, it's, it's really, uh, it's quite a kind of fascinating dichotomy. It's, it is really re- Well, I'll give you... And, and to your automation point, I had a funny conversation this morning with, uh, David George, who runs our growth fund, 'cause we're talking about the new tools, and I was like, "Oh, have you tried the Muse thing?" He's like, "Oh, it's awesome." I was like, "What'd you do with it?" He said, "I finally canceled my New York Times subscription." [laughs]

Diogo Almeida 15:39 [laughs]

Martin Casado 15:40 [laughs]

Ben Horowitz 15:40 And I was like, "That, that is hard to do." [laughs]

Martin Casado 15:45 [laughs]

Ben Horowitz 15:46 But y-

Diogo Almeida 15:47 Yeah

Ben Horowitz 15:47 ... you know, it, it's a kind of a, it's a very tip of the iceberg of the things that are horrible things to do that, that we need to automate.

Diogo Almeida 15:56 I think that if we were going to be really intellectually honest, and we are really aiming for the North Star of automation, we cannot fall into the same anti-patterns that AI has fallen into, which is really, um, focusing on outliers and demos.

Ben Horowitz 16:09 Hmm.

Diogo Almeida 16:09 Right? Like, a lot of people ask me, like, "What are your favorite use cases?" And I'm like, "I'm not sure if they work." I want them to work in the background-

Ben Horowitz 16:15 [laughs]

Diogo Almeida 16:16 ... such that, like-

Ben Horowitz 16:16 Yeah

Diogo Almeida 16:16 ... someone would trust that to run and not page them, and, like, people can build on top of that too. And, like-

Ben Horowitz 16:22 Uh, composable. Composable AI.

Diogo Almeida 16:23 Composable, but, like, other things, like-

Ben Horowitz 16:25 Yeah

Diogo Almeida 16:25 ... safe, right?

Ben Horowitz 16:25 Mm-hmm.

Diogo Almeida 16:26 Like, it's a different type of safety where, like, i- if you want it to actually, like, run with resources associated with it, with access-

Ben Horowitz 16:32 Mm-hmm

Diogo Almeida 16:32 ... to things, you need guarantees for that, or, like, at least statistical guarantees.

Ben Horowitz 16:36 [laughs]

Diogo Almeida 16:36 And, um, like-

Ben Horowitz 16:37 So it doesn't go rogue, break and enter-

Diogo Almeida 16:39 [laughs]

Ben Horowitz 16:39 ... face, that type of thing.

Diogo Almeida 16:41 Well, I, I, I don't think our models will be doing that-

Ben Horowitz 16:43 Yes

Diogo Almeida 16:43 ... anytime soon-

Ben Horowitz 16:44 Yes

Diogo Almeida 16:44 ... unless someone, like, does the software to do that, which would be g-

Ben Horowitz 16:48 Their fault, yeah

Diogo Almeida 16:49 ... a very cool flex. Very cool flex.

Ben Horowitz 16:50 Yeah. [laughs]

Diogo Almeida 16:50 I should, I should figure out how to give credits for that. But, um, [laughs] but not in a way that we're not responsible.

Martin Casado 16:57 Right, right, right.

Diogo Almeida 16:57 [laughs]

Martin Casado 16:58 Right. I'm just curious, like, how, how long has this intuition been percolating? 'Cause I remember talking to you maybe in, was it 2017? It was so long-

Diogo Almeida 17:06 We, we did talk about that, yeah.

Martin Casado 17:08 Yeah, and, like, and then, like, a lot of these ideas were in y- you know, you were talking about data being important, and you're talking about, like, you know, you want to focus on the task and, like... But, like, so I, I just, like, you know... Was this, like, did you know that this was gonna end up being a classifier, or was this just an intuition that, like, there was just kind of another way to view this entire kind of AI movement?

Diogo Almeida 17:27 Mm-hmm.

Martin Casado 17:27 You know?

Diogo Almeida 17:28 So actually, a fun story about that chat in the talk from-

Martin Casado 17:32 Yeah

Diogo Almeida 17:32 ... 2017.

Martin Casado 17:33 Yeah.

Diogo Almeida 17:33 I think my talk was actually on a very similar theme. I think it was called something like AI Modular in Theory-

Martin Casado 17:40 Yeah

Diogo Almeida 17:40 ... and Flexible in Practice.

Martin Casado 17:41 Yeah, yeah.

Diogo Almeida 17:41 Which is very software systems-y.

Martin Casado 17:43 Yeah, yeah, totally. Yeah, yeah.

Diogo Almeida 17:43 So I'm a little bit consistent in that. I, I think that this really started right before ChatGPT. Um, like, right when we released these things, I did not have intuition about this, and honestly, I was not even... I was very, very pleasantly surprised by the generalization capabilities of RLHF.

Martin Casado 18:01 When is this?

Diogo Almeida 18:02 Must be end of 2021, like, uh-

Martin Casado 18:04 Yeah, okay

Diogo Almeida 18:05 ... fourth quarter of 2021.

Martin Casado 18:06 Yeah, yeah.

Diogo Almeida 18:06 Um, like, we were s- It was really, really general. Like, if you read the paper, it's unlike other papers that are, like, trying to prove their point. It was us actually, you know, scientific method-ish trying to disprove, like-

Martin Casado 18:19 Yeah

Diogo Almeida 18:19 ... is it cheating? And, uh- You know, my favorite query was, why is it important to eat socks before meditating? We'd made sure that was not on the internet beforehand.

Martin Casado 18:29 [laughs]

Diogo Almeida 18:29 And like the models were able to like make plausible hu-human-looking answers for this. And that to us in the team was the thing that clicked, like this is not cheating-

Martin Casado 18:37 No

Diogo Almeida 18:38 ... which you should always be afraid of cheating in, in ML.

Martin Casado 18:40 Right.

Diogo Almeida 18:40 And then what really got me burnt was I, we released it. Um, you know, we did a, you know... I'm obviously a big capabilities guy.

Martin Casado 18:49 Yeah.

Diogo Almeida 18:50 I did a lot to release that model. I, I really thought that the model had like a decent chance of being AGI, and when it didn't, that was like when my whole world came crashing down.

Martin Casado 19:01 Oh, interesting.

Diogo Almeida 19:01 And I was like, "Why?" And-

Martin Casado 19:02 Oh, so you were kind of on the other train for a bit.

Diogo Almeida 19:05 Which, uh, like the crazy train, or?

Martin Casado 19:06 Well, no, I'm just like-

Diogo Almeida 19:07 Oh, no offense. No

Martin Casado 19:08 ... RL, uh, no, no, no, RL generalizes, like maybe we have AGI, like...

Diogo Almeida 19:13 RLHF generalizes pretty well.

Martin Casado 19:14 Oh, yeah.

Diogo Almeida 19:14 RLVR is the thing that doesn't generalize as well-

Martin Casado 19:16 Right, right, right

Diogo Almeida 19:16 ... from what I've seen.

Martin Casado 19:17 Right.

Diogo Almeida 19:18 Um, and AGI in to-

Martin Casado 19:22 Well, as, I was just saying m-more, I mean, like, you know, you were behi- in ChatGPT, you were behind-

Diogo Almeida 19:27 Yeah

Martin Casado 19:27 ... the, these early GPTs. That was a very different goal, which is like creating a chatbot that will talk to the human being, but it's not a programmer's tool, et cetera. So I'm just wondering, like-

Diogo Almeida 19:34 Oh, well, actually, early, early like 2020, um, OpenAI, when we talked about AGI, people used to describe it as Ilya and Ebrieff statement. Um, so it's not-

Martin Casado 19:45 [laughs]

Diogo Almeida 19:45 It's, uh, it's kind of like a-

Martin Casado 19:47 Interesting

Diogo Almeida 19:47 ... but like, it's like part, uh, we were talking about OpenAI culture.

Martin Casado 19:51 Yeah.

Diogo Almeida 19:51 Part of it is that it's like intentionally vague, so it's a wide c-like tent-

Martin Casado 19:55 I see

Diogo Almeida 19:55 ... so that everyone can be inside of it.

Martin Casado 19:57 Mm.

Diogo Almeida 19:57 But like I am not... For nuanced reasons, I don't think we are on the path of RSI, and I still don't think we're on the path of RSI, and I did then. I do think that what OpenAI defined as AGI is extremely doable. Automating most of the world's economically valuable work-

Martin Casado 20:14 Yeah

Diogo Almeida 20:14 ... actually sounds-

Martin Casado 20:15 Yeah

Diogo Almeida 20:15 ... like, uh, oh man, I don't, like there's a, there's a lot of work out there. A lot of it is very rote and simple, and like by volume, in order to be able to like outsource work, you need like simple instructions that like basic people can do.

Martin Casado 20:30 Yeah.

Diogo Almeida 20:30 And as far as I can tell, the intelligence of that has been available in the models for like quite a while now.

Martin Casado 20:35 Yeah.

Diogo Almeida 20:35 And like my, oh man, you know, chip on my shoulder is like, "Why is this not available?" And, and then ev- the, the a- and then since RLHF, the, the AI industry just like kind of bifurcated into gigantic overpromise, underdeliver. I think GPT-3 was actually quite calibrated-

Martin Casado 20:52 Mm

Diogo Almeida 20:52 ... w- back in that day.

Martin Casado 20:53 Mm-hmm.

Diogo Almeida 20:53 But because humans evaluate how good the models are, it looks really good 'cause they are the judge, but we've been optimizing that judge instead of the automation part-

Martin Casado 21:02 Mm

Diogo Almeida 21:02 ... and that has been the missing thing. So I would say that it was really, really then that it like hit me, you know, like, "Why is this thing not more useful?"

Martin Casado 21:10 And so you think that the measure that we should have is to what extent can you automate actual productive tasks? Is that the-

Diogo Almeida 21:16 That, that's, I would like that. I would like that

Martin Casado 21:17 ... when you say overpro- when you say overpromise and underdeliver, that's the dimension in particular you're talking to, the ability to automate tasks.

Diogo Almeida 21:25 I, like, I think in my heart, it's like cool sci-fi. Um, you know, and I think that-

Martin Casado 21:29 Totally

Diogo Almeida 21:29 ... I think that that is the canary in the coal mine for cool sci-fi.

Martin Casado 21:33 I see.

Diogo Almeida 21:33 Like, are you really telling me that we, like math is solved, or like even like two years ago, GPQA-

Martin Casado 21:38 Mm-hmm

Diogo Almeida 21:39 ... that Google proof question answering is solved, but we still can't handle a drive-through, right?

Martin Casado 21:43 [laughs]

Diogo Almeida 21:44 Like, it's, it's, it's, it's, it's a very hard thing to hold in your head at once.

Martin Casado 21:48 Yeah.

Diogo Almeida 21:49 And I think a lot of people don't have good answers to that.

Martin Casado 21:52 Yeah.

Diogo Almeida 21:52 I agree.

Martin Casado 21:52 Can I, can I just test one thing which would, which may-

Diogo Almeida 21:54 Hell yeah

Martin Casado 21:54 ... which may not make sense, but I want to just-

Diogo Almeida 21:56 Hit me.

Martin Casado 21:57 I mean, isn't there an argument though that like

Martin Casado 22:00 the, the, the real, the distribution of the real world is, is, is different than the digital world, right? It's heavy tailed. There's a lot of exceptions. We don't have all the data. And I mean, it, couldn't it be the case that the reason we're not doing productive stuff in the real world is just like we're not, we don't have the data for that distribution. We're not training on that distribution, and this is why it's just been basically relegated to like these lower dimensional manifolds, like whatever math or code or...

Diogo Almeida 22:26 Um, I don't entirely buy the data argument, in my opinion. Um, I do believe that there's a, a long tail, for sure, like that would be kind of crazy-

Martin Casado 22:38 Yeah

Diogo Almeida 22:38 ... to deny. And I don't think that in my like canary in the coal mine situation, we need to automate that long tail. Like I think that-

Martin Casado 22:45 I see. Okay

Diogo Almeida 22:45 ... I think we need to be incredibly pragmatic on everything, and like building reliable software is always an investment.

Martin Casado 22:53 Yeah.

Diogo Almeida 22:53 Right? Like it, like, you know, what are the three great virtues of a programmer? Laziness to not to do it again, hubris, and [inhales] there was a third one.

Martin Casado 23:01 Oh, yeah. No, yeah, no, I remember. This is from the Pearl days.

Diogo Almeida 23:04 Yeah.

Martin Casado 23:05 Yeah, yeah.

Diogo Almeida 23:05 Yeah. There, there was a third one.

Martin Casado 23:06 Larry Wall, yeah.

Diogo Almeida 23:07 I, I wish, I wish I could remember it.

Martin Casado 23:08 Yeah, yeah.

Diogo Almeida 23:09 But like it's about like the laziness to like spend [chuckles] you know, like ten hours to do like the five-minute task instantly and to never have to do it again. Like, it only make, like it should be an ROI decision for people who like automate stuff. Like, I would just like it to be automatable.

Martin Casado 23:24 Yeah.

Diogo Almeida 23:25 And I think that people will just make n-like new kinds of work, he-hence the Jev in Jevens.

Martin Casado 23:30 Yeah.

Diogo Almeida 23:30 New kinds of work once that stuff is doable. But, uh, like, as like a, a benchmark, I feel like it's useful to see can we actually automate the stuff that it really, really looks like AI should be able to automate.

Martin Casado 23:42 I see.

Diogo Almeida 23:43 OpenAI has been trying to automate customer service since 2020.

Martin Casado 23:46 Yeah.

Diogo Almeida 23:46 You know, like, and it, it, it's, you know, like it's not. [laughs]

Martin Casado 23:51 Which is pretty amazing.

Diogo Almeida 23:52 [laughs] It, it's, it's wild, you know? It's wild.

Martin Casado 23:56 Well, and then inside, I mean, inside companies, um, there's very little that's automated right now.

Diogo Almeida 24:02 Yeah.

Martin Casado 24:02 Like, a-a-and the projects haven't worked. Um-

Diogo Almeida 24:05 Yeah

Martin Casado 24:05 ... other than pro-programming has worked amazing.

Diogo Almeida 24:08 Can you, can you maybe classify the types of problems you think that are easier to automate?

Martin Casado 24:13 Now, 'cause it, it was kind of interesting. So we've actually looked at support before the current generative wave, and it was interesting. You'd meet a company, and the company would say, um, "We, uh, we answer 95% of all, like, you know, like, help desk calls." And like, that is so many. But then you actually look at the data-

Ben Horowitz 24:30 They're all the same incident

Martin Casado 24:30 ... and you realize it's all password resets. [laughs] And then, like, but if you did it by, like, uniqueness, it was only something like 50% or something. So it just feels like when you're dealing with humans and natural systems, like there's just kind of this very kind of-

Ben Horowitz 24:42 Yeah

Martin Casado 24:42 ... you know, like a, you know, long tail of exceptions. And so to what extent did, like I, every, probably every hour I have somebody ping me on like, "I'm using Jev for this new use case." [laughs] I'm like, "I had no idea." You know, like, you know. And, uh, and so, like, to what extent did you even predict, like, the broad range of use case for it? Like, did you assume that was gonna happen? And have you been surprised by that or-

Diogo Almeida 25:03 Extremely surprised.

Martin Casado 25:04 Yeah.

Diogo Almeida 25:04 Did not assume it would happen. Uh, this launch was, like, not something... Like, if anyone expected this, they are probably insane.

Martin Casado 25:12 Yeah, yeah.

Diogo Almeida 25:13 Um, right? Like it is... I, I don't think someone could expect a ChatGPT for developers, 'cause ChatGPT was for-

Martin Casado 25:22 Yeah, sure

Diogo Almeida 25:22 ... you know, li-li-like, you know, the normal users. And it's weird. I actually don't even know what percentage of the people who are part of the Jev party are developers themselves.

Martin Casado 25:32 Oh, interesting.

Diogo Almeida 25:32 I can't imagine non-developers using it. I don't know how they would use it. Um, but even my non-developer friends are just, like, part of the party and Twitter and meme-ing and everything like that. So number one, phenomenal. Number two, um, this will be hard to convey in this short message because, like, it's been like blood, sweat, and tears for years now.

Martin Casado 25:55 Yeah.

Diogo Almeida 25:55 Like, the amount I care about reliability is, um, it's, it's a lot. Like, reliability is what this thing is. If you don't understand that, it'll be very hard to make like a copycat that's benchmaxed. Like-

Martin Casado 26:08 Mm-hmm

Diogo Almeida 26:08 ... it's... I feel like every nine of reliability is going to be so valuable for everyone, even if it's not the most valuable thing market cap-wise.

Martin Casado 26:17 Hmm.

Ben Horowitz 26:17 Yeah.

Diogo Almeida 26:17 Because it will just enable new applications and, like, we are fighting for, like, all sorts of, like, weird nines of reliability that, like, we don't even fully understand because we are just like, you know, s- like really getting the, this like electric motor of AI, of intelligence-

Martin Casado 26:32 Yeah

Diogo Almeida 26:32 ... like into people's, like, workstations, and they can figure out what to do with it.

Martin Casado 26:35 What, what does reliability mean in this context? Is this, is this like availability of the model, or is it like I call the model and it returns the same thing? Or like how, how do I think about reliability?

Diogo Almeida 26:46 Yeah, so-

Martin Casado 26:47 For something that's inherently kind of stochastic.

Diogo Almeida 26:49 So, uh, not so much the former thing.

Martin Casado 26:51 Okay.

Diogo Almeida 26:51 And the second thing is closer. Like, I would describe the first thing as kind of like uptime or SLAs.

Martin Casado 26:56 Okay.

Diogo Almeida 26:57 The second thing I would maybe call closer to determinism.

Martin Casado 27:00 Yeah.

Diogo Almeida 27:00 The, something thirdly, I would consider more like robustness. So robustness I would kind of describe as, um, similar intelligence every time.

Martin Casado 27:09 Oh, interesting.

Diogo Almeida 27:10 Yeah. So like not exactly determinism, 'cause I think determinism, it's useful for unit tests-

Martin Casado 27:14 Sure, sure

Diogo Almeida 27:14 ... but not real systems.

Martin Casado 27:15 Sure.

Diogo Almeida 27:15 Think about like if you add a UUID to a prompt, it should be the same 'cause it's the same functionally.

Martin Casado 27:20 Yeah, yeah.

Diogo Almeida 27:20 But it's not exactly deterministic.

Martin Casado 27:22 Right, right, right.

Diogo Almeida 27:22 I think that there's another layer of it that I don't really know what it's called yet. Like, maybe this is what I would call like some form of intelligence, which is it doesn't have to be the similar function every time, but it needs to be smart every time. You know, like w- if you were in that situation, would this be an understandable thing for a human to think? Because a developer can program around that and actually to me, the highest honor of reliability-

Martin Casado 27:48 Right

Diogo Almeida 27:48 ... will be to get to the point where people can program against Jev without making example queries. Like-

Martin Casado 27:54 Yeah

Diogo Almeida 27:54 ... you, when you just trust Jev-

Martin Casado 27:55 Hmm

Diogo Almeida 27:55 ... you'll be in like permaflow state just creating-

Martin Casado 27:58 [laughs]

Diogo Almeida 27:58 ... cra-crazy software. And like that's where a lot of-

Martin Casado 28:02 Yeah

Diogo Almeida 28:02 ... software is today, right? Like I-

Martin Casado 28:03 Yeah

Diogo Almeida 28:03 ... I, I don't think it's totally unrealistic, but I'm gonna, gonna be fighting for it.

Martin Casado 28:07 Do you, by the way, th-this is kind of a weird question, so like, I mean feel, feel free, like if, if it's too weird, just feel free. But, um, it occurs to me that actually the value of things like coding agents goes down if you have a primitive like this in a way, which is like you could be like, you know, whatever, some, you know, Codex builds all the software for me, but it doesn't actually use Jev, and so like the software itself it creates is somewhat limited. Or you could be like, "Okay, I as a human being, I will write the software without using a coding agent, but I've got this very generalized primitive that makes writing software easier." So like do you feel, like see a future where it's like the coding agents using Jev and then you're telling the coding agents, and then do you have like redundancy? Or do you feel it's like humans implement or-

Diogo Almeida 28:46 This is more of like a coding agent question than it is a Jev question.

Martin Casado 28:49 Oh yeah, for sure. Yeah, yeah.

Diogo Almeida 28:50 So my vibe is that, um, I'm not in the coding minds as much as I'd like to be.

Martin Casado 28:55 Yeah.

Diogo Almeida 28:55 So you are, you two might be in there more than I am-

Martin Casado 28:59 Yeah

Diogo Almeida 28:59 ... which is sad. But my experience is that they are really good at syntax and really-

Martin Casado 29:06 They're bad at semantics

Diogo Almeida 29:07 ... um, I would say incredibly bad at architecture.

Martin Casado 29:12 Yeah. True.

Diogo Almeida 29:12 Um, uh, so like to me, architecture is like the most human creative part of software.

Martin Casado 29:16 Mm-hmm.

Diogo Almeida 29:17 So I love using coding agents.

Martin Casado 29:19 Yeah.

Diogo Almeida 29:19 I think that, uh, Jev is almost certainly not in distribution. That would be spooky if they trained on our user data.

Martin Casado 29:25 Yeah.

Diogo Almeida 29:25 Um, so it's probably not. Um, but, uh, I think that when it is in distribution, I, I see no problem with like having it do the syntax. And the thing with architecture is that maybe the models are actually like not just crap at architecture, but maybe they're 50th percentile at architecture.

Martin Casado 29:41 Yeah. Hmm.

Diogo Almeida 29:41 And if you don't know anything about architecture, it would be fine. So these are all like gray area trade-offs in order for you to navigate, and sometimes speed is the, the knob for your company or project to turn. Like you're willing to do a 50th, like a 50th percentile architecture instead of a 60th because you want to move faster and have like Codex work overnight or something like that.

Martin Casado 30:00 Right, right, right, right, right. So-

Ben Horowitz 30:00 Yeah. W- actually, kind of along those lines, one of the interesting things or phenomenons in the market already is that, you know, when the coding agents came out, it was the SaaS apocalypse, and all their values dropped through the floor. And then when Jev came out, every SaaS company is like, "This is the greatest thing ever."

Martin Casado 30:17 It's amazing. Yeah.

Ben Horowitz 30:18 So explain that.

Diogo Almeida 30:21 I, I don't know what else to say, right?

Ben Horowitz 30:24 Yeah.

Diogo Almeida 30:24 Like, I, I think it's, like, quite natural.

Ben Horowitz 30:26 Mm-hmm.

Diogo Almeida 30:26 Like, in the SaaSpocalypse story-

Ben Horowitz 30:29 Mm

Diogo Almeida 30:29 ... um, the, the story that I feel like has panned out really poorly is that software is very cheap and perhaps easy to replicate, which-

Ben Horowitz 30:38 Mm-hmm

Diogo Almeida 30:38 ... I think I could, I could believe the former, I could not believe the latter because a lot of the stuff happens beneath the hood.

Ben Horowitz 30:44 Totally.

Diogo Almeida 30:44 Um, I'm maybe overly a software fanboy here. Um-

Ben Horowitz 30:48 Yeah, all of us. [laughs]

Diogo Almeida 30:49 Yeah. Okay, okay, okay. Um, I didn't know where you-

Ben Horowitz 30:51 Yeah

Diogo Almeida 30:52 ... might be in the coding agent fanboy-

Ben Horowitz 30:53 We, we have a lot of legacy around that.

Diogo Almeida 30:55 Yeah.

Ben Horowitz 30:55 100%, yeah.

Diogo Almeida 30:56 Yeah. So it, I don't think that really panned out, so SaaS seems like maybe the markets don't agree.

Ben Horowitz 31:02 Mm.

Diogo Almeida 31:02 But, like, I think SaaS is providing the same value it used to. Maybe the markets are just scared.

Ben Horowitz 31:06 Mm.

Diogo Almeida 31:07 Um, but I think that SaaS will be one of the largest winners of, like, the whole AI game.

Ben Horowitz 31:13 Mm-hmm.

Diogo Almeida 31:14 And I want to, like, work really, really well with, like, all the biggest, most boring, most, like, in the know of user problem-

Ben Horowitz 31:20 Mm-hmm

Diogo Almeida 31:20 ... SaaS companies because I think that they are the best positioned to know what workflows to automate, what do people need. Like, that's what-

Ben Horowitz 31:28 Mm

Diogo Almeida 31:28 ... what their bread and butter is, and to spend the big... Like, you know, software is always a CapEx investment.

Ben Horowitz 31:33 Mm.

Diogo Almeida 31:33 And, but, like, you spend it ahead of time in order to make this experience even better that gets-

Ben Horowitz 31:38 Right

Diogo Almeida 31:38 ... you know, like, distributed to all of that mass of users.

Ben Horowitz 31:40 Right.

Diogo Almeida 31:41 So I think that it's gonna be... I, I, I'm not gonna forecast anything about the financial markets-

Ben Horowitz 31:46 Yeah

Diogo Almeida 31:46 ... but I think for as far as, like, a capabilities games goes, it's gonna be like an inverse SaaSpocalypse, and I am-

Ben Horowitz 31:52 Uh, uh

Diogo Almeida 31:52 ... so jazzed about it. I, we, I should make a name.

Ben Horowitz 31:55 Yeah. Yeah, that, yeah, that, that, that...

Diogo Almeida 31:57 Yeah.

Ben Horowitz 31:57 You should have a name.

Martin Casado 31:57 SaaS- SaaSpalooza.

Diogo Almeida 31:59 Oh. [laughter] That's a little too fun. [laughs]

Ben Horowitz 32:04 Well, the SaaS applications are gonna all of a sudden get, like, dramatically more useful.

Diogo Almeida 32:08 Yeah.

Ben Horowitz 32:08 And by the way, y- you know, the, the, the, the kind of, uh, capital investment, like, so much of a, of a SaaS company's capital investment is li- actually getting to all the customers. And so if you've gotten to all the customers, and then you make, you know, not just put a chatbot on your SaaS product, but actually make the software, like, way, way better, um, th- that's a hell of a thing.

Diogo Almeida 32:30 I don't know if this is a realistic dream or not, but I think that there's a world where, like, the multi-choice, choice forms just disappear. You know?

Ben Horowitz 32:39 Mm.

Diogo Almeida 32:39 Like, I f- I feel like they are-

Ben Horowitz 32:41 Yeah.

Martin Casado 32:41 Yeah

Diogo Almeida 32:41 ... l- like, they are always, like, something mapping natural language that usually the software already has into, like, a Jev-like output, and I think that that is a-

Ben Horowitz 32:50 It's literally, it's literally from the '80s.

Diogo Almeida 32:51 [laughs]

Ben Horowitz 32:51 It's like, it's called, we used to call them 4GLs. Do you remember that, Martin?

Martin Casado 32:54 Yes. Yeah.

Ben Horowitz 32:55 [laughs] Fourth generation language.

Diogo Almeida 32:57 Yeah. Um, actually, also, I think this is from the '80s, this might be an insult. I wasn't born then.

Ben Horowitz 33:01 Yeah. Yes.

Diogo Almeida 33:01 Um, the, like, I think that do what I mean is going to be, like, be taken-

Ben Horowitz 33:06 Mm

Diogo Almeida 33:06 ... to the absolute next level.

Ben Horowitz 33:07 Yeah, yeah.

Martin Casado 33:08 Yeah, for sure, yeah.

Diogo Almeida 33:08 If I could shout out one Jev application, I don't know if it's reliable, so I can't promise anything.

Ben Horowitz 33:12 Yeah.

Diogo Almeida 33:12 But it was so freaking cool. Someone was using, like, a voice to control your computer, and it was basically constantly making decisions on, like, is this a command or is it inserting text?

Ben Horowitz 33:23 Oh, that's cool.

Diogo Almeida 33:23 Where is inserting text? Like-

Ben Horowitz 33:24 Wow

Diogo Almeida 33:24 ... like, that sounds so unbelievably cool.

Ben Horowitz 33:27 Oh, that is cool.

Diogo Almeida 33:28 Like, like, I feel like interfaces could, like, just completely change, and maybe we're gonna have to make it, like, cheaper or faster, you know?

Ben Horowitz 33:34 Yeah. Then you're at Star Trek.

Diogo Almeida 33:36 Oh. [laughs] Well...

Martin Casado 33:38 You, you know, there's-

Ben Horowitz 33:39 Yeah

Martin Casado 33:39 ... there's just such a profound intuition here, which is, um, um, you know, if, if you use AI today to generate software, right? You're still cr- creating the, the same software that you did before, but if you actually look at, like, the average PR for a large company, it's like 10 lines, right? Seriously. No, so we actually, we-

Diogo Almeida 33:57 I, I've been at Google

Martin Casado 33:58 ... we actually did this study.

Ben Horowitz 33:58 Yeah.

Martin Casado 33:59 So it's like 10 lines. So, like, you're automating 10 lines, and by the way, those 10 lines are, are like, you know, part of a learning from a customer or something.

Diogo Almeida 34:05 Mm-hmm.

Martin Casado 34:05 So, like, you're, you've kind of optimized something that's actually pretty minimal, um, but what it doesn't do is provide new capabilities to the software, right?

Diogo Almeida 34:12 Yeah.

Martin Casado 34:12 It's kind of automating this thing which, uh, in the limit ends up being relatively minor, and now, like, there's actually new capability, and so, like, it could just be the case that just software just actually gets better.

Diogo Almeida 34:22 Yep.

Martin Casado 34:22 And, and by the way, it didn't... Even b- before Jev, like, just, it didn't even occur to me that, like, it doesn't matter how much, you know, AI coding agents you use, the software actually isn't getting better. [laughs]

Diogo Almeida 34:32 Maybe.

Martin Casado 34:32 So maybe you're writing it faster.

Diogo Almeida 34:33 Yes.

Martin Casado 34:33 It's, like, arguably getting worse just because, like, there's less oversight. So I think this is-

Ben Horowitz 34:37 Well, and, and often in- more insecure.

Martin Casado 34:39 Yeah. [laughs] For, for sure, for sure. But you're a- but you're actually now can make an argument, like, like, like, like apps will have new functionalities as a result of this because there is this new primitive that you're providing that, I mean, like, in, in a way, like, it speaks natural languages and it can reason.

Diogo Almeida 34:53 Yep.

Martin Casado 34:54 But it marries that to a state machine.

Diogo Almeida 34:57 I...

Diogo Almeida 34:58 If people take that as a takeaway, that would be, like, the greatest compliment ever to what we are doing.

Ben Horowitz 35:03 Yeah.

Diogo Almeida 35:03 Like, I actually feel like it's almost too grand of a vision to expand beyond the three logic gates that we have-

Ben Horowitz 35:10 Yeah, yeah

Diogo Almeida 35:10 ... into, like... You know, it, our types are kind of like one s- of the same logic gate, but, like, one that's, like, a little brain in there.

Ben Horowitz 35:17 Yeah.

Diogo Almeida 35:17 Like, that would be the greatest compliment-

Ben Horowitz 35:19 Yeah

Diogo Almeida 35:19 ... to, like, the, the, the type safe legacy, 'cause, like, that is t- that's a very-

Ben Horowitz 35:23 Huge

Diogo Almeida 35:23 ... non-trivial, huge thing for the world. Um, I'm not gonna, like, overpromise, underdeliver that, but I will fight for that.

Martin Casado 35:30 Yeah, I mean, listen, I mean, there's, I think pretty open questions to what, like how deep can this get as far as, like, like, really serious stuff, like state consistency or durability or, like, real systems level stuff where you actually need to, like, provide strong guarantees. And so 100% this will change-

Diogo Almeida 35:46 Mm-hmm

Martin Casado 35:46 ... things like whatever, analyzing logs, analyzing emails, providing a UI, talking to the human. Like, that for sure, but, like, you know, you could argue that over time this becomes, like, a smart database.

Diogo Almeida 35:55 Yep, yep.

Martin Casado 35:55 Like, you know, and so-

Diogo Almeida 35:57 And, and also-

Ben Horowitz 35:57 Air traffic control system.

Martin Casado 35:59 Uh, anything, right?

Ben Horowitz 35:59 Which we really need. [laughs]

Martin Casado 36:01 True, true.

Diogo Almeida 36:02 A little scary.

Ben Horowitz 36:03 Yeah.

Diogo Almeida 36:03 Like, I think automate the easy work before the hard work is always my philosophy.

Martin Casado 36:07 Yeah, yeah.

Ben Horowitz 36:07 Yes.

Diogo Almeida 36:07 But I also think there's going to be, like, an entire era of probabilistic programming that's opened up. Like, my-

Martin Casado 36:15 By the way, you know there's a huge history of probabilistic programming-

Diogo Almeida 36:16 Oh, I do

Martin Casado 36:16 ... that basically died in, like, the '70s, right? [laughs]

Diogo Almeida 36:19 I, I'm familiar with it. I, I, I actually think it's going to be- Like with the same, like you could also call Jev like neuro-symbolic AI

Martin Casado 36:26 So your co-f- your co-founder Eric came from that background, he was telling me.

Diogo Almeida 36:30 Oh, cool.

Martin Casado 36:30 That kind of Bayesian-

Diogo Almeida 36:31 Oh, yes, yes, yes. He did a lot of biology stuff.

Martin Casado 36:34 Yeah.

Diogo Almeida 36:34 It, it goes up and down.

Martin Casado 36:35 Yeah.

Diogo Almeida 36:35 Uh, but like what I mean is a more-- I'm not a fan. My brand is pragmatism.

Martin Casado 36:42 Yeah.

Diogo Almeida 36:42 Incredible pragmatism.

Martin Casado 36:43 Yeah.

Diogo Almeida 36:43 I'm not a fan of like biologically inspired stuff-

Martin Casado 36:46 [chuckles]

Diogo Almeida 36:46 ... um, at all, ever.

Martin Casado 36:47 It's never worked. Have you ever noticed that?

Diogo Almeida 36:49 I mean-

Martin Casado 36:50 Or rarely

Diogo Almeida 36:50 ... I, I think it's never worked.

Martin Casado 36:52 Yeah.

Diogo Almeida 36:52 It's useful to motivate crazy people to work on things for decades-

Martin Casado 36:56 Yeah

Diogo Almeida 36:56 ... until it works, and then they refine it into like the engineering version.

Martin Casado 36:58 The, uh, the story of AI, totally. Neural nets for sure.

Diogo Almeida 37:01 Yes.

Martin Casado 37:01 Yeah.

Diogo Almeida 37:01 But like, you know, the, a lot of the stories about how it worked were not accurate, right?

Martin Casado 37:05 Yes.

Ben Horowitz 37:05 Yeah, yeah.

Diogo Almeida 37:05 So like the hierarchical features of applications really didn't end up working, 'cause like otherwise ResNets wouldn't have worked. Longer story. Um, I, I do think that it opens up like, like from a systems perspective, I'm not excited about this part 'cause it's really... I'm excited for the world, not about me programming this, 'cause it sounds like really complicated.

Martin Casado 37:22 Yeah.

Diogo Almeida 37:23 But I think that as we have like lots of intelligence at lots of, uh, like different cost and speed trade-offs, the super systems-y types will be b- making trade-offs at like, you know, like Jev's gonna be like a thousand times too smart for them. They just want like an approximate link-

Martin Casado 37:38 Yeah

Diogo Almeida 37:38 ... to have an approximate guess to like optimistically route here and there. It's gonna be like so crazy, the, this-

Martin Casado 37:44 Right

Diogo Almeida 37:44 ... type of stuff that's available in the extreme systems end.

Martin Casado 37:46 And the g- and the good news is we get to like rebuild systems again, which is great, right? We have a new-- No, seriously, we have a new primitive. It's kind of a new way of thinking about doing software. Like, I mean, we did this and we did this with the internet, you know.

Diogo Almeida 37:56 Mm-hmm.

Martin Casado 37:56 We did this mainframe to client server. I mean, we do this periodically. And, and by the way, just because of the, um, cybersecurity issues, we probably have to rebuild almost all the systems-

Ben Horowitz 38:08 Yeah, for sure

Martin Casado 38:09 ... uh, to, to just make them safe.

Ben Horowitz 38:12 I, I would think. I think it's pretty clear that there, like there's not, not-

Martin Casado 38:15 Or at least the critical, the-

Ben Horowitz 38:17 Yeah, yeah

Martin Casado 38:17 ... the critical infrastructure, for sure.

Ben Horowitz 38:19 Yeah, yeah, yeah. Yeah, yeah.

Martin Casado 38:20 Do, do you think about this more in terms of like apps, SaaS, analytics, or

Martin Casado 38:28 more in terms of like systems, foundations, or all the above?

Diogo Almeida 38:31 For what I would think of or how-

Martin Casado 38:33 Yeah, just general application for this. When you think about like, like you're working on Jev and like, and you kind of envision the people are adapting it, like how, how, you know, like do, do, or maybe do you even have an opinion?

Diogo Almeida 38:44 I have a little bit, and it's... So the way I think of it is a little like, uh, like deep into like the TCP guts. You know, like UDP, TCP, you know, like it's unreliable, it's unreliable.

Martin Casado 38:58 You're speaking, speaking my language. [laughs]

Diogo Almeida 38:59 Exactly. And, and like I, I, so like when I think of AI-

Martin Casado 39:01 Feel comfortable

Diogo Almeida 39:02 ... no, no, I mean, like when I think of AI, and this is why I care about intelligence per dollar, to be clear.

Martin Casado 39:06 Yeah.

Diogo Almeida 39:06 When I think... And, and how I got to this conclusion is I work backwards from AI-based economic revolution, AI everywhere, you know, sci-fi and everything, like all the software has AI all over the place. And I ask myself the question, what percentage of the calls to AI, imagine it's like a function-

Martin Casado 39:21 Mm-hmm

Diogo Almeida 39:21 ... which, uh, what percentage are like for human consumption, where you need like style and everything?

Martin Casado 39:26 Versus understanding.

Diogo Almeida 39:26 And, and, yeah.

Martin Casado 39:27 Mm-hmm.

Diogo Almeida 39:27 And it's gonna be like many nines, and actually from that same question, how many will be at the first layer-

Martin Casado 39:32 Yeah

Diogo Almeida 39:32 ... versus like deep in the guts?

Martin Casado 39:34 Yeah.

Diogo Almeida 39:34 Right? And I think that it's going to be many nines in the guts, but it will start at the first layer. But like we need to, if you don't aim for the guts... [chuckles] Wait, that sounds weird.

Martin Casado 39:41 [laughs]

Diogo Almeida 39:42 If you don't aim for the guts, you, you, it's, it's, it's going to take you a while to get there.

Martin Casado 39:45 Yeah.

Diogo Almeida 39:46 Right?

Martin Casado 39:46 I, I think people don't understand to what extent like AI was kind of ships in the night with software. Like even if you try to embed AI in software, it kind of like didn't behave, right? Because software doesn't really take natural languages, and you like do all this weird stuff. Like you stick in the prompt like, "Here's the JSON output that you want, and here's a schema," and it would never listen to it.

Diogo Almeida 40:04 Yeah.

Martin Casado 40:04 And so, and so what you ended up doing is just taking the output and giving it to a human. You're like, "To hell with it," right? It's like-

Diogo Almeida 40:08 Yes, or another LLM. That is what a while loop is, like the-

Martin Casado 40:10 Yeah, ex-

Diogo Almeida 40:10 ... agent while loop, right?

Martin Casado 40:11 Yeah, yeah.

Diogo Almeida 40:11 So it's like, it's like from first principles, it needs to be human in the loop, which is the chat.

Martin Casado 40:15 Yeah, yeah.

Diogo Almeida 40:15 Or an agent, which is the while loop, because like the natural language needs to be fed back into another.

Martin Casado 40:20 Yes, and I will say, I, I have watched this ha- There was almost like this kind of like five stages of grief. [laughs] Like, you know, people would pick up AI and like, "I'm gonna use this," you know, "within my software," right? And then, you know, and then it would like go with like, you know, whatever, denial, like try to make it work, [chuckles] and like anger. Then they go to acceptance, which is like, "Okay, never mind, I'm just gonna give this to another LLM, to a human being." So it's been very ships in the night, and I think this is the first time I have seen-

Diogo Almeida 40:44 Yeah

Martin Casado 40:44 ... when someone's like, "Actually, you can take an LLM, you can take AI, and you can actually map it to like, like a state machine, and you can do that productively."

Diogo Almeida 40:53 And I hope so. I will not want to overpromise, underdeliver as well. Like I don't know if it's ready for all the applications-

Martin Casado 41:00 Oh, sure

Diogo Almeida 41:00 ... that have been overpromised.

Martin Casado 41:02 Mm-hmm.

Diogo Almeida 41:02 I really, really want it to, and my team will fight for that, obviously. Like we

Diogo Almeida 41:08 really, really care about reliability. We could have released so much sooner. I don't think people realize that, and I don't think that, honestly, I don't think that they will. I, uh, based on what I see of the Twitter discussion-

Martin Casado 41:20 Yeah. [laughs]

Diogo Almeida 41:20 ... I think people will never get it, but like it'll just have like that good vibe of like, "Oh, I can trust this."

Martin Casado 41:26 Yeah.

Diogo Almeida 41:26 So-

Martin Casado 41:26 Well, it's the anti-frustration machine, yeah.

Diogo Almeida 41:28 [laughs]

Martin Casado 41:29 Yeah.

Diogo Almeida 41:30 It's, it's a... I hope so. I hope, do what I mean, right?

Martin Casado 41:34 Yes.

Diogo Almeida 41:34 Like to me, that is about like smoothness in the world, like having everything like just move more smoothly together and interlink like gears.

Martin Casado 41:42 Yeah.

Diogo Almeida 41:42 I actually have my whole like AI utopia on like different axes that I really, really want, and like do what I mean is a huge part of this.

Martin Casado 41:50 Yeah.

Diogo Almeida 41:50 You know, like imagine if all technology just did what you mean.

Martin Casado 41:53 Yeah.

Diogo Almeida 41:54 That is, like that's not sci-fi. Look how smart AI is, right?

Martin Casado 41:57 Yeah. No, it's, uh, it's amazing. And, and maybe that's the thought to close on.

Diogo Almeida 42:03 Oh.

Martin Casado 42:03 Do what I mean.

Diogo Almeida 42:04 [laughs] Yeah, love it.

Martin Casado 42:05 Thank you, Diogo. This has been a great conversation.

Diogo Almeida 42:08 Hell yeah.

Martin Casado 42:08 Really enjoyed it.

Diogo Almeida 42:13 So fun.