Skip to content

By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

The Twenty Minute VCThe Twenty Minute VC

Jev Creator on Will US Open Models Catch Chinese | The Cost of Intelligence & Tokens | Diogo Almeida

Diogo Almeida is the Co-Founder and CEO of TypeSafe AI, the company behind Jev. Jev is an AI model that lets software make decisions automatically and tells developers how confident it is in each call.

Diogo AlmeidaguestHarry Stebbingshost
Nov 16, 20261h 13mWatch on YouTube

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

Great conversations run long. Their best ideas don't have to.

uListen is free on iPhone, iPad, Android, Chrome, and Claude.

See how

Transcript

  1. 0:00 – 1:43

    Intro

    1. DA

      Almost everyone who's paying attention to AI knows reliability is the absolute largest bottleneck. You get Einstein maybe 95% of the time and Mr. Bean the other 5%.

    2. HS

      Now, joining me in the hot seat today is one of the best founders of our time, one of the hottest companies, Diogo Almeida, the founder of TypeSafe, the maker of Jev. Jev is used by a third of the Fortune 500. Pretty incredible. They've been valued at $7.5 billion. Today, we get into the future of models. This is an incredible discussion.

    3. DA

      Data matters a lot more than compute. I don't want to be called a Neo Lab because from my sense, most Neo Labs are incredibly, incredibly wasteful. Anthropic models are just the best, even if the Chinese models match it in benchmarks. It feels like there's no real competition right now, and that surely will change eventually. Everyone has just been in this, like, mono-model horse blinders for so fucking long. I believe that AI has been a bubble for a while, so I would like to keep planning for an eventuality of bubble burst.

    4. HS

      Ready to go? [upbeat music] Diogo, I'm so looking forward to this. We, we had a brilliant exchange following Eric's show from Modal, who-

    5. DA

      Mm-hmm

    6. HS

      ... I actually spoke to last night about you. Um, and so I'm very excited for this. Welcome to the show.

    7. DA

      Thank you. Thank you. Yeah. A, a brilliant-- It's a very small exchange, uh, but I'm, I'm glad you liked it. Uh, yeah, it was, it was, uh, really fun, and I didn't realize it would become this.

    8. HS

      Well, dude, we, we said that it would be a spicy discussion. So before we get into-

    9. DA

      Absolutely

    10. HS

      ... the spice itself, I just wanna lay-

    11. DA

      Mm-hmm

    12. HS

      ... some frameworks and, like, foundations. Sixty seconds, for people who don't know what TypeSafe and Jev is, can you just give us the 60-second intro?

  2. 1:43 – 4:51

    Why AI Reliability Is the Biggest Bottleneck to Automation

    1. DA

      I would argue that Jev is not a specialized model, but my pitch for the people who are not, like, purely software developers is actually more about the state of AI. Like, how can AI be so smart at so many things, you know, like the Millennium Prize problems in math, yet not transform software basically at all? Like, all the SaaS we use is basically the same as it was a decade ago, and we are not automating, like, almost any, like, entire job. You know, like, drive-throughs, we keep failing. Customer service, like, there is AI customer service, but it's actually can't be trusted for the big decisions. How can that be the case? Because it seems so good at some stuff and so bad at other stuff. And to me, what we are doing and what Jev is, is trying to bridge that gap, make AI too good to be true for automation instead of just human-in-the-loop assistance.

    2. HS

      I spoke to Sonali at Sequoia, and speaking of kind of full end-to-end automation, she said, "Why is reliability the biggest obstacle to automation?"

    3. DA

      Well, uh, I will give a caveat first about reliability, which is that I don't mean, like, uptime. I don't mean, like, oh, it's like, you know, a- you know, available and you get lots of throughput. I mean reliability of intelligence, which I'm still trying to find a good word for. But, like, this non-jaggedness of intelligence. And I would argue that almost everyone who's paying attention to AI knows reliability is the absolute largest bottleneck. So here's the example. Like, how good are you at math? How good am I at math? Like, we-- There's not a chance we could even help the Millennium Prize solution. I, I, I believe. I haven't read it, but I would, I would wager that. And yet, the- these same models cannot be even used for accounting, you know, much simpler stuff. And the question is, why? And it's not about, um... It is... These models are extremely high variance. It's that you get, uh... People love this, this saying that I've coined, but-- so, but it's like you get Einstein, like, maybe 95% of the time and Mr. Bean the other 5%.

    4. HS

      [laughs]

    5. DA

      And the problem is that in expected value, it just doesn't work out. Therefore, companies quite wisely learn that you can't actually automate stuff. The number of companies I've seen over, like, the multiple AI hype cycles over the last, like, four years, um, I've seen so many try to automate accounting. They keep failing, and it's because, you know, the math doesn't work out. It's not a diffusion problem in the slightest. And reliability is what gets you... You know, maybe you can trade off, like, some level of Einstein, uh, for just getting it more commonly and more often and getting the same functional intelligence. And that, to me, is Jev's superpower.

    6. HS

      So how do we get there? What is the solution, then, to removing Mr. Bean, as much as I love Mr. Bean?

    7. DA

      I, I, I love Mr. Bean too. I, I actually... You know, I love when agents are, like, insane, you know. Bad vibes forever. It's, it's, it's very fun. It's a very fun story. And also, that is not what I want in my software that runs in the background.

    8. HS

      Yeah.

    9. DA

      Like, as an engineer, I want to, like, write a thing. I want to spend a lot of effort up front. Like, maybe I don't realize it'll be a lot of effort, but it always is. And I want it to run in the background and solve the problem forever. And you just need, like, nines of intelligent reliability to do that. And I would argue that the way to do that, um... Oh, man, I wish I could, like, bring up my slides here, but I

  3. 4:51 – 7:30

    Why Data Matters More Than Compute

    1. DA

      will, I'll, I'll try to use my hands, I guess. Um, so I call it AI's bitterest lesson. Um, Sutton coined a bitter lesson, which is, I, I'll paraphrase for my benefit, is roughly that compute matters more than algorithms. This is all of, like, the scaling build era we had for maybe, like, 2019 to 2022 to 2023 was a scaling build era. Um, you know, like, just scale and you, the models get better. This kind of plateaued at GPT 4.5, and then we kind of stopped scaling as a field because it stopped having payoff. Um, I would argue that there's a bitter lesson, which is that, um, data matters a lot more than compute. This is very obvious to anyone who's done any applied ML. But for, you know, the, the people in the peanut gallery don't seem to realize this, but, like, data is obviously more valuable than compute. And I would argue the most important thing by far is what is your North Star? What do you define as your right task? And the thing is that pre-Jev, the two right tasks that existed in LLM land were RLHF, which is reinforcement learning from human feedback, which means please humans, and RLVR, which means- But, but like score high on benchmarks. And neither of those actually translate to like run in the backward, background software. And I would argue that this is like the root of all of the problems. It is not that the AI is not capable, because the AI is clearly super capable. The problem is that, and, you know, I, you are e-experienced in the real world. Everyone who's experienced the real world knows this. You get what you optimize for, right? And we just happen to optimize for the human in the loop.

    2. HS

      Yeah. Thank you so much for, uh, bluntly giving me so many follow-ups in a short-

    3. DA

      Oh, can I say one thing?

    4. HS

      Yeah.

    5. DA

      Can I say one thing about this too? Um, uh, on the note of reliability, ChatGPT never fails at being ChatGPT. Like, when is the last time you got like random characters, right? Like, it's kind of odd that it... Or, or even if you ask for a polite email, it's not gonna start saying, um, I, I probably shouldn't curse on a podcast, but it's not gonna start saying slurs, right? Um, but humans might make that error sometimes. But the problem is not that it, it's not impolite or sycophantic. The problem is it makes wrong decisions within the text. And it's interesting that it can be reliable at some stuff, but not others. And if you optimize for the important stuff to be reliable on, which is decision-making and judgment, that is how you get real reliability.

    6. HS

      You said there about kind of the plateauing of compute in terms of the benefits that accrue. First, have we got to that stage? Are you serious? I thought the whole point was you-

    7. DA

      On pre-training?

    8. HS

      We, well, purely on pre-training, sure. Uh, but like, like more and more compute still leads to better and better output, no?

  4. 7:30 – 9:05

    Why Scaling Compute Alone Stopped Working

    1. DA

      Yes, but like it's, you need a lot more data than people once upon a time thought. So, um, in, y- you know, like maybe people have like kind of warped this to seem like they were right the whole time. But the narrative back in the day was just scale to the moon, and you will get AGI. And GPT 4.5 is what broke that narrative. T- uh, I, I obviously don't know what's going on behind the labs, but if we believe that the labs were pricing roughly based on like, uh, you know, inference compute, then GPT 4.5 is still the largest model ever by, um, token cost, and it was not, you know, commercially successful, right? Um, and, uh, I think this is obviously the case because you can't just scale compute without really thinking about it. Um, what people are doing-- People are, are throwing a lot of compute at, um, training, but it's a... You need a lot of data with it. This is why you see so many like get rich quick by selling data, RL data to start, to lab type stuff. And you can't really do very much without that data. And if you actually saw, I believe it might have been Dwarkesh. He had a rec-recent Twitter post that showed what matters more, data or comp- or, or algorithms. And data was something like many, many times more effective. It, it was something this week. Um, and this is something that everyone has known for like since the beginning of ML. This is not even specific to, to AI. It's just that the compute scaling era was a, like a weird outlier in this.

  5. 9:05 – 14:00

    What Actually Makes AI Training Data Valuable

    1. HS

      Can I ask, when we say like the value of data, how it's so much more primary over compute, is that synthetic data? Is it like proprietary enterprise data that everyone has in their companies?

    2. DA

      Ooh.

    3. HS

      Is it AI systems generating their own experiences? Is it certain types of data, or is it just broad da- Can you help me understand that?

    4. DA

      I wish I had my visuals here. So, uh, let me work backwards again from my bitterest lesson, which is in order of importance, there's doing the right task, then data, then compute, then algorithms. This is what we like extremely, extremely learned when we made RLHF. Um, you know, like the quality of your a- underlying algorithm matters very little. As long as you do the right task, um, a GPT-2 sized model could beat something like GPT-9, um, which is, you know, and GPT-9 doesn't even exist. Um, so, uh, in order to have data, you actually need to know the right task. And actually, the data for the task is... Or normally, when you create a new task, the data doesn't even exist. Like, when we made RLHF, um, we, like, like human preference data of like comparing LLM outputs did not exist anywhere in the internet. It was not in company database. It was not on the internet. We had to like define it, you know, understand what are the instructions, get human labelers, iterate, and figure out what that data should look like. And that is what unl- like doing the right task is what unlocks the data in the first place. And the problem is bootstrapping a new task is hard because by construction, the data doesn't exist. It's a similar thing for us, but it's also similar for RLVR. Um, before RLVR existed, uh, and I actually want... There, there's actually a tricky thing with what the right task of RLVR is. Um, reinforcement learning from verified rewards, sorry if I'm like going a little bit too advanced here, is that RLVR is not at all about the verifiable rewards. It is, or it is partially that, but the big part of RLVR is this like latent chain of thought, like hidden reasoning, so the models can think in private and then answer. Because even in like 2021, there were verifiable rewards. People tried to do RL, but like PPO without reasoning on unit tests, and it kept failing. So it was actually not at all about the verifiable rewards. It's about the structure of the task, and it just turns out that once you're doing RLVR, the models are a lot better at this. And what even good data means is now a function of how good the models are. This is why people need, uh, you know, the people who make data need access to the cutting-edge models, because what actually makes the data good is whether or not it improves the cutting-edge models. So they need to be at like exactly the right level of difficulty in order to like push the models forward. And, uh, it, there, there's a lot of like nuance and on policy-ness of data. But g- back to the original question, like is data, you know, is, is the right data stored in corporate, uh, you know, whatever data lakes? I would argue no, um, because it's probably not shaped the right way. People are definitely trying to do this, so a lot of data vendors are trying to like yoink this data and then sell it. Maybe they can do some massaging to make it work. But the most important thing is focusing on how do we use this thing to push our task forward. And if you forget that, you are good-hearting a local objective.

    5. HS

      How do we use this thing to push our task forward? Are the data providing companies, Surge, Macaw, are they good businesses? Are they services businesses now which have to go end to end where they really go into implementation and effectiveness of model, not just data serving like they did in the early days? Are they, are they good businesses?

    6. DA

      Th- th- that's more of a you question than a me question. Are, uh, do you think they're good businesses?

    7. HS

      I do personally because I think the market is just so large for the amount of money-

    8. DA

      Mm-hmm

    9. HS

      ... that these LLMs will spend on data. And then I think if you think that we're gonna move into a world where companies own their own intelligence, their data will always be needing supplementary data. And so then actually the TAM becomes just moving from LLMs [laughs] to everyone.

    10. DA

      Interesting. Interesting. That might be a fun debate on whether or not companies will own their own intelligence. Um, that, that might be a fun one. I will actually argue that we are still in, like, hype wave territory, and then there's always, like, a trough, trough of disillusionment, and then, like, the reality thing. I think we've not hidden the reality part of it yet. I believe that in the future, um, both RLHF and RLVR are immensely valuable. I don't know if people will invest as much as they are right now, but I think that each of them result in, like, their own, like... Even if RLVR never existed, ChatGPT would be, like, an incredibly valuable consumer brand. And you will need data for that, but w- you know, data will always be necessary, but how much that company will be willing to spend on data when they actually need to be a real sustainable business, I don't know what that number is. So, uh, it's very hard to ballpark that.

  6. 14:00 – 15:06

    Is Synthetic Data Overhyped?

    1. HS

      What value does synthetic data hold in model improvement?

    2. DA

      The, I don't think that there's, like, one, one form of synthetic data, right? Uh, I think that it, synthetic data is kind of like a tool more than it is like a, like an idol to pray to, and it's dependent on, like, what the, the hands of the team. Like, if you read the research, um, you'd think that synthetic data is mostly over-hyped because it's only used to make small models catch up to large models, but it can't be used to improve large models or, or large state-of-the-art models. So, uh, i- it's really dependent on how you use it. Uh, like everyone-- Well, not everyone. Uh, we use a lot of synthetic data, but, like, we do not use it in the way other people do, and we think that the more interesting part is not that it's synthetic, but, like, how we wield this tool with an extreme laser focus on the right task, which is, like, the reliability of intelligence.

    3. HS

      You said about the hype and the trough, my friend. There is insane-

    4. DA

      Oh, it's pronounced trough. That's good to know. [laughs]

    5. HS

      No, don't... I tried to do it elegantly, okay? I'm British. British charm.

    6. DA

      Oh, yeah.

    7. HS

      Um, are you concerned by the amount of money going in, or do you think it is justified?

  7. 15:06 – 17:18

    Why Diogo Thinks Most Neo Labs Are “Incredibly Wasteful”

    1. DA

      I am concerned for the world, I think. I think that throwing money into the furnace is a bad thing for the world. Oh, man, I really don't want to be, like, too negative, but I don't want to be called a Neo lab because from my sense, most Neo labs are incredibly, incredibly wasteful. Like, it feels like their business model is hire a bunch of, you know, ex-lab people, ba- get a bunch of GPUs, have fun, and then eventually get acquired. Um, and that seems like a big, uh, you know, like, that seems like a waste of resources to me. So I am not a fan of that, um, on one hand. On the other hand, like, people are throwing a lot of money, for example, into RL, and I actually think that this is kind of cool, you know. The, like, there are diminishing returns massively, but the models are getting smarter. It's kind of good for the world. I think it's very cool. Um, would I make that investment with my money? No. Um, uh, obviously not because, you know, we have money to make bets, and we're obviously not gonna do that. Um, so I, I'm not a fan of that in the world either, but maybe, you know, like the VC math works out where it's like a 1% chance of moonshot. It actually, like, has positive expected value. Um, and wait, you had a question. Like, am I concerned and there was a-

    2. HS

      Are you concerned, or is it justified?

    3. DA

      I would guess based on my expected value calculations it is not justified, but would be very happy to be proven wrong, or at least right now.

    4. HS

      Diogo, did you expect it to work as quickly as it did?

    5. DA

      So I would say that for years now, even before TypeSafe existed, I thought that I must be missing something about the field. O- obviously, I, I convinced myself I was not missing something, but it seemed so weird where this direction to me was unbelievably obvious. Like, so freaking obvious that I gave up when I first came to the idea because I thought surely Anthropic is already, like, 80% done. It's, like, the most obvious thing I've ever heard of. So to me, it's really, really obvious that this would be valuable in the world. I still think people don't get it, even though the launch was obviously phenomenal. The part I didn't expect is, um, the, the ChatGPT for developers moment because even non-developers were talking about it. That is, that, that I could not have predicted. That was

  8. 17:18 – 19:46

    What Everyone Gets Wrong About Jev

    1. DA

      super weird.

    2. HS

      What do people not get? What do people get most wrong about Jev and you?

    3. DA

      So my team hates when I talk about this, but, I, the, the, my rule is on a podcast I can be let off the rails. I believe it is a competitive disadvantage for me to talk about what really Jev is doing because people keep missing the mark. And if we just didn't tell anyone, then we would just forever be, like, king of the hill. And I just care too much about the field to not just yell it in, in, like, painful insanity, what the fuck's going on. Sorry for cursing. Um, and the thing that people really don't get is that what we-- is special about the model is not the cost or the speed. It's, again, the reliability of intelligence. And people are still thinking in LLM land, like, uh, like so many people I think are burning their goodwill. There should be copycat models in the future, but they should give a lot of fucks about, uh, reliability in my opinion. Um, you know- And I think that the people who are shipping copycats right now do not. They are trying to, like, do the same thing in LLMs, which is, like, benchmark max, get hype, try to show like, you know, like, the, the, like, the, the little things like, you know, nu- numbers that are not relevant like speed and cost, when the, the real thing they need to do is, like, hunker up in the bunker and make the models really, really smart. And people don't realize that that is what we are all about. Like, when we see benchmarks, even if we are, like, unbelievably in the lead, like, we are still pained when our model is dumb, and that is the thing we focus on. And I think that's what everyone should be focusing on in the field. Like, hey, like, why is it 97% accurate? What was the missing piece? And to be clear, it's not always a model thing. It's very oftentimes a prompting thing, but that is how it is to build a developer tool, right? Like, we need to, like, educate people. We need to figure out, like, how could we prompt this better? We need to figure out, like, is there some nuanced, nuance in this language that, like, could help people make more robust prompts, or do we need to make the model smarter? And I think this is the thing that people really, really miss. And I think developers get it, and they got it way faster than I thought 'cause they already don't give a, a shit about, like, the benchmarks, and they're just like, "Your model is clearly the best at new stuff." But the people... The, it's the same, the same people who are like, "Oh, this is really boring. It's just a classifier," who ended up making the copycats who don't understand that there's a reason why you can, like, put so much effort to make the model even more and more and more, uh, robust.

  9. 19:46 – 20:28

    Why OpenAI Can’t Simply Copy Jev

    1. HS

      You said about copycats there. Can I be blunt? Uh, OpenAI and others-

    2. DA

      Yeah, please do

    3. HS

      ... have moved very quickly in trying to create their own decision model products after you launched. Why don't the big model providers just copy you and you get competed out?

    4. DA

      Um, uh, I don't think that that's a blunt question at all. I think that that's a, like, a totally fair reasonable one. I think that if they one day started caring a lot about reliability, then that would become, like, an interesting battle.

    5. HS

      Diogo, how do they not care about reliability? They want enterprises. They want CI-

    6. DA

      You have no fu- you have no fucking clue how little they care. Um-

    7. HS

      What? How, how? They want enterprises. They want revenue. How are you gonna get a CIO to sign up if you don't have reliability?

  10. 20:28 – 23:02

    Why Benchmarks Don’t Tell You Which Models Are Best

    1. DA

      I think people have a hard time understanding, like, intelligence as a product, which is why people try to use benchmarks as Band-Aids. Um, you know, like, benchmarks are kind of a proxy for understanding the intelligence of the underlying thing. But as far as I can tell, the string field, like, you know, over, you know, like, maybe, like, the last nine months have learned that benchmarks don't mean shit. You know, like, Anthropic models are just the best even if the Chinese models match it in benchmarks. Um, and, like, even when you, like, measure benchmark maxing, you still think, like, the Anthropic models are better than this, uh, better than the competition even though they're not on the Pareto curve of pretty much any benchmark. And the... Basically, it looks like the subfield of, like, system one or decision models learn this in, like, like... I think there was, like, competition maybe, like, starting a week after. It felt like they-- people learned it in, like, two, three weeks, where we're like, "Oh, yeah, the other stuff is just benchmark max. It's total crap." It's very easy to see that it's different. And I think that it, it takes a different type of organization to, again, really, really care about reliability. And it's very hard-

    2. HS

      If they don't care about-

    3. DA

      Go ahead

    4. HS

      ... if they don't care about reliability, what do they care about then, Diogo?

    5. DA

      Well, it really depends on, like, their business objectives, right? Like, uh, back in the ChatGPT is everything days, this is before Claude Code, like, they cared about, like, monthly active users, right? And that's actually a very different metric to optimize for than the best chatbot experience. This is how psychofancy emerged, and it kept going in a cycle of, like, oh, it's more psychofantic. It lies sometimes. But hey, that is what people want, so we just keep doing it in some pockets of space. Um, so you get what you optimize for, and that was, like, what ChatGPT optimized for. I think now people are optimizing for, like, the MAU equivalent of coding agents. And, like, if they're not training a dedicated model for this and really putting, like, a lot of, like, intense thought, I don't think that they will be able to make a comparable model anytime soon. I, I hope that all of them do 'cause I think it'd be very good for the world. Um, it's obviously going to be bad for the business. But this is... The, the, like, you know, right now, we are in the place where it feels like there's no real competition right now, and that surely will change eventually, right? Like, I hope that we get competition and we just show that we are the best 'cause we care the most and we have, like, the smartest people who are working, like, the absolute most laser-focused on the North Star. But ag- again, like, this is not yet competition.

    6. HS

      When you look at the decision model product landscape for mo- for models that you're in-

    7. DA

      Mm-hmm

    8. HS

      ... what does that look like in three years? Is it like an Uber/Lyft market? Is it like a GCP, AWS, Azure market? Like-

    9. DA

      Hmm

    10. HS

      ... what does the market composition look like

  11. 23:02 – 28:12

    Why Decision Models Could Dwarf LLM Token Usage

    1. HS

      in three years' time?

    2. DA

      I'm a little bit of a, a technological idealist, I think. Like, so I actually don't really think of what the compe- competitive landscape is gonna be like. I care more about the technological one. Um-

    3. HS

      Okay. Great

    4. DA

      ... and my, our s- our strength is always going to be technological shock and awe, so we are going to have to keep shipping awesome stuff. And the way I see that landscape is I'm going to ignore competition. Like, I think that in terms of, like, number of token... I, I truly believe that we're gonna, like, mega Jevons paradox. It's not going to be, like, a subtle Jevons. It's not going to be a fake Jevons like LLMs. It's going to be, like, super-duper. There's going to be, like, tons and tons and tons of usage. And I think that it will completely dwarf at least the total token consumption of string LLMs. I think that th- these things are going to be used everywhere, and we're gonna finally have AI run in the background. And I think we're gonna start to even have, like, a shift in how people feel about AI from being, like, this thing that, like, slops crap at them to, like, oh, it actually makes life better 'cause people truly like it when technology improves their lives.

    5. HS

      When you say mega Jevons paradox, do we have the energy and infrastructure to power the demand of a mega Jevons paradox that you foresee?

    6. DA

      Yep.

    7. HS

      And what are the bottlenecks to your, uh- Tech utopia

    8. DA

      So, uh, t- utopia's a strong word. Maybe a dream, uh, could I, is something that could be more, um, in line with reality. I- our models are incredibly efficient in comparison. We're not doing the standard, like, subsidized playbook. We actually can serve these things, um, like incredibly, incredibly efficiently. And I think that there's, like, many boundaries to push on that efficiency angle. So I think, of, of course, we're gonna have to do some sick research, but, like, that's the thing I trust us to do. We can- We'll have to push efficiency quite a bit in order to have that mega Jevonsing occurring. Um, I think that the largest bottlenecks are going to be... It, it, it feels a little bit single player right now, where it's on us for pushing the boundaries of the technology. Um, from a, from an AI point of view, and from a field point of view, we're going to need to interact a lot with developers. We're going to need to help them push the fa- the boundaries. This is what I've been talking about, like a, a role I dream of. Like, um, I'm so sick of, like, the status games, especially in SF, where ML researchers are, like, a higher tier than ML engineers, which are a higher tier than software engineers. I think that that's completely messed up. And what my hope is, is we can create a new type of role that I call... My, my current favorite name is software researcher. Like, someone who actually pushes the boundaries of software. And I think that this is, this is, this, this makes sense, right? Like, if you knew that you were in the Cambrian explosion of the internet or mobile, you'd think that there's, like, so many boundaries to push, and there's so much, like, low-hanging fruit that you need, like, trailblazers to go out there and build, like, you know, like these, like, uh, you know, lighthouses of, like, "Hey, there's value here. Let's, like, have a gold rush there." And, uh, I'm so excited about encouraging people to push those boundaries. I wish I had the time to do it myself, and there's so many crazy exciting things to do. So I think that those two are going to be the, the biggest bottlenecks to get to, like, you know, system one models everywhere, AI actually automating shit. And if I could say one more thing, I, like, if... I don't know if you're a software engineer, um, but I guarantee every single person who's ever been a software engineer has had a conception in their head of, "I can automate this," or, "I can't automate this." Like, you need to know, right? Like, if you wanted to, like, automate, like, something that moves files around, you can automate this. If you want to make, like, a voice interface for your laptop so you don't need to, like, type with your hands, you can't automate this, you know? And my hope is that AI will actually completely remove the can't automate part. It's only going to be a matter of effort. I will cook on this undercooked idea, but this is something that I find to be, like, such a beautiful analogy for AI, where, like, AI is so clearly awesome, yet software itself hasn't changed. Writing software has very much changed, but the software is the same, and that's tragic. And I love the analogy of the blue LED. Um, you know, like, when the red and green LED existed, they were very narrow uses. Like, they could barely be used for anything, just indicator lights because they're very efficient. But when you add the blue LED, it didn't just expand the market cap of LEDs by 50%. It made all sorts of things possible. Like, we couldn't have the screens we have today. We couldn't have, like, white light LEDs that are so much more efficient without this missing composable piece. And my hope is that, um, AI ends up becoming, like, a fourth fundamental logic gate. You know? Uh, like, there's, like, three fundamental logic gates that all software is built on top of. They are so dumb, but it's actually the brilliance of software engineers that enables you to use these bitwise comparisons to create everything today in software. And what if we could add a fourth one? And I think that that should expand what, what we can do by way more than 33%. It should expand what we can do by, like, hundreds of thousands of X, and that is why I'm excited about, like, a Cambrian explosion specifically of software.

  12. 28:12 – 29:32

    Why Enterprise Software Has Barely Changed Despite AI

    1. HS

      Why do you think the current state of software is kind of a tragedy in the state that it's in? And, you know, if you look at enterprise software today, it, it's enterprise software with a sprinkling of AI at best.

    2. DA

      Yep. Yep.

    3. HS

      Can you talk to me, how do you feel about that?

    4. DA

      I think it's actually extremely logical. Um, you know, like, I don't think it's a distribution problem. I think enterprises have so much incentive to adopt AI, and the people who have adopted it the hardest were the ones that were burnt the most. The argument I always make is that, uh, LLMs are unreliable, so they cannot be trusted for decisions with stakes to the company. But you can always make them make decisions with stakes to the user because companies tend to not give a shit about their users. Tragic, but that is what happens. So what ends up happening is you don't have AI deep in the logic of your m- uh, apps. What you end up having is, like, the same B2B SaaS with AI attached to the side where it does not-- it cannot do, like, high-stakes things, but it can, like, shoot slop at the user because people kind of like it because it's responsive, and they make sure that all of the burden of checking, like, is this right or not, is on the user's plate. And this is actually extremely logical for the state of reliability. So I think that it is tragic that people wasted so many resources, like, go- doing the roundabout way to get to where we're at right now, and also, I think that this is the right spot, you know, given how smart and reliable the models are.

  13. 29:32 – 30:57

    Do Incumbent SaaS Companies Have the Advantage in AI?

    1. HS

      Do the biggest benefits accrue to the incumbents who have distribution? When you look at a Salesforce of the world, and then you kinda reflect on what you just said about the tragedy of current state of software, do the benefits fall to those who have distribution, or is there a new generation who benefits accrue to?

    2. DA

      Number one, SaaSpocalypse is obviously not happening, right? Um, Martin Casado coined it as a SaaSapalooza, and I love that. Um, I don't necessarily think it means that big SaaS wins. I think that the people who are best positioned to win are the people who care the most. And SaaS has a massive advantage over AI companies in that they're actually familiar with the workflows. They have, like, the scale in order to, like, spend the e- the, the investment upfront to do the engineering, 'cause it takes real engineering to automate stuff and think about all of the stuff and the, in order to, like, actually solve people's tasks. And that is the real valuable thing here. I don't necessarily think big SaaS will always be the winner, but I think that they are best positioned, not at all due to their distribution, but because they are the ones closest to the problems. They understand, like, hey, if we are at, uh... I don't know anyone about at Intuit. You know, like, if we're in TurboTax, where could we change the interface to make it smoother? And there's not a chance anyone at OpenAI or Anthropic knows how to do that, right? Like, they are still living in their ivory tower. And thinking that, like, software is just not going to exist as a concept. So, uh, I think it, like they could be

  14. 30:57 – 33:34

    Why Diogo Disagrees That Every Company Should Own Its Own Intelligence

    1. DA

      the big winners

    2. HS

      You said there about kind of knowing the processes, knowing the workflows, and the importance of that. I've had Lynn on the show from Fireworks, I had Eric on the show from Modal, and they both said about the importance of owning your own intelligence as a company and the benefits of having this incredible proprietary data-

    3. DA

      Mm-hmm

    4. HS

      ... and these specialized models on a per-company basis. Diogo, if the importance is in the process and the workflow, surely this is the Holy Grail, no?

    5. DA

      Oh, man. How do I say this elegantly, um, and Britishly? Um, so, huh, you know, like, that's a very, that's a, that's a opinion, I think, that goes in the current rails of how the current world works. Uh, the current world thinks about AI, where AI is trying to be posed as a coworker, right? So AI's being, like, presented as a coworker. This is, like, the direction Anthropic is really pushing towards, where it's kind of like the same interface as, like, a human being. And I believe that this is going to be, like, the mechanical horses or the horseless carriages of the AI era. Just like, you know, in the industrial era, people were like, "Let's just make it look like a carriage, but, like, without the horse." I believe that the AI coworkers are a form of that, uh, because it m- just makes sense that if you're doing automation, uh, automation will have a different, um, strength profile than a human would be, and you'd want to lean into the strength of automation. Kind of like in the Industrial Revolution, uh, you don't need to wait for a whole coworker, like a humanoid robot, to do tasks. You just find the extremely narrow thing, like twist bottle cap on bottle, and then you automate it, and it just gets solved forever. So I think that's how work will generally go. I think that if you assume that AI coworkers are the way to go, which I absolutely do not, um, uh, and you assume that you want-- Like, you know, most companies have the willingness, capabilities, and desire to train them because they need to be trained on, uh, like, in distribution for your company, then I think that owning your own intelligence direction can kind of make sense. I do believe that I would think that the labs approach of, like, AI coworker, but general intelligence makes a lot more sense to me because, you know, um, pretty much any company, if you gave them infinite interns that cost, like, 1000x less, you could probably find a lot of productive use for them. Um, and the, like, we as a field should answer the question, why are we not getting that effect right now? And the people who say you need to own your intelligence, their excuse is, "Oh, you need something like continual learning," or, "You need something more narrow." And I just don't think that that jives with reality.

  15. 33:34 – 37:00

    Are Specialized Enterprise Models Actually Worth It?

    1. HS

      So Fireworks have helped several very b- top-tier brand name firms in the Valley. Um, I, I think your Harvey's and your, uh, Ramps, and I think it's Cognition, might have got some wrong there, bu- building their own models. Are you saying that is a mistake to pursue that route and horizontal or alternative models will be as good and it's basically a futile exercise?

    2. DA

      It's complicated, I think. I would 100% believe narrow models would-- So number one, benchmarks are incredibly misleading, and the people who are making these have, like, a lot of perverse incentive to look like their models are better at, i-if, i-if internally and also from an external narrative perspective, to not seem like they will be completely boned by the large models. I don't know how good they are. I would imagine that they could get cheaper, for sure, um, and they could get more narrow. And I think that this is kind of a sign of, like, maturity when, like, we're plateauing at a certain capability level. And I think that AI tends to, to increase in, like, more discrete steps, and these discrete steps occur when you start getting data in a particular distribution. Like, I think it'd be hard to deny that the largest jump in coding agent performance happened around, like, November-ish last year. I can't remember which Opus that was. And that was because that is when they put it in distribution. And everything since then has been, uh, you know, like we could talk about, like, how we love Fable 5.2 or Opus 5.5. It seems like it has been diminishing returns since then, and it's when you're in the diminishing returns that you can start doing, like, the micro-optimizations, like cost per token. Um, my view of the world is that, um, these are all, like, symptoms of not reliably solving problems. Like, software engineers that I know, and I'm a software engineer too, we hate updating dependencies. We hate going into the latest and greatest stuff. The only reason we would do updates is because it didn't work sufficiently before. Otherwise, you would just, like, lock it away in the basement and only update it when it gets, like, way cheaper. And I believe that we have a lot of insufficient automation right now, and that is, like, what as... I, I am biased, of course, but that is what I think the data most suggests. And my view of the world is that because of Jev and Jev-like models, we will start, like, locking in real automation, and people will not want to update that. And the way companies get their own proprietary intelligence is not from having their own trained models, because it's hard to do that, but by having software around them. And the reason why it's so important for the narrative of companies like Harvey and Cognition to say that they own their intelligence is because otherwise they're a very light wrapper on top of the LMS, and it looks like they have no moat. But software and engineering is, like, uh, e- even if it wasn't a moat, it's, like, a phenomenal thing. You know, investing upfront to understand real problems, to automate them so they never have to be solved again, and humans don't have to do this roadwork. I think that that's just so good for the world, and, um, I, I think that's a much more plausible way that the companies of the future progress and, like, their proprietary intelligence happens at the code level. Kind of like how, like, there's no m-- Maybe there's some moat in, like, which database you use, but, like, the real meat of it is how you use those databases.

  16. 37:00 – 39:44

    Is Harvey Too Thin a Wrapper on Frontier Models?

    1. HS

      Diogo, I'm fascinated. I know the Harvey Ligura market pretty well. Do you think they are a thin wrapper? I, I, I-- To me, it is a very ... specific domain that requires deep product functionality that can only be done in a vertical fashion, and that is why they exist and they're very successful. I, I'd push back strongly that they're a thin wrapper.

    2. DA

      So, uh, thin is all relative. Um-

    3. HS

      [laughs]

    4. DA

      ... they are much thinner than I... Uh, uh, oh, that, that, that, that sounds politically correct right now. Um, it, it's like it depends on, like, how deep I would want it to be. And my understanding, last I checked with Harvey, I obviously can't even afford a license, so I don't fully know. Um, back when Harvey was working with OpenAI, they had this dream of like, "Oh, we want to automate larger and larger projects autonomously." And that kind of stuff didn't end up working. Um, and they ended up having to, like, kind of end up pivoting into copilot for lawyers, which my understanding is they still are. And yes, you can do a lot of domain-specific things as a copilot for lawyers, but, like, there's only so much engineering you can do around a copilot, right? Like, is it, like, the domai- domain-specific MCP tools to pull up a rag that is based on, like, real law and all of that? And, you know, like, the reality is people wanna automate, like, large chunks of work and not just make people, like, double-digit percent more efficient, which I think is- tends to be where a lot of AI, uh, lands. So yes, like, I think it's awesome they're doing a lot of engineering around it. I think that there's just going to be a lot, lot more engineering. Like, even system messages are a form of, like, really ghetto engineering, where you're programming the LLM to do behaviors, but you have, like, no guarantees, no confidence, and, like, no real way to test how effective it is other than, like, very weird evals that people do these days. So I would like more engineering absolutely everywhere, especially things that are, like, AI adjacent. And I would actually hope that we can one day move past the copilots and just automate, you know, work that takes weeks to do in the background, and maybe that will be very differently shaped than what work that current, uh, humans would do because automation tends to require, like, an ada- adaptation on what is available technologically.

    5. HS

      You, you said that, uh, earlier about the cost of tokens literally in, uh, at the beginning of kind of a, a two to three-minute talk there. Um-

    6. DA

      Oh, shit. [laughs]

    7. HS

      ... you said about how-

    8. DA

      Am I that bad? Oh, shit. Wow, okay.

    9. HS

      No, no, dude, I, I, I... No, I love it.

    10. DA

      Yeah.

    11. HS

      It's fant-

    12. DA

      Yeah

    13. HS

      ... but the trouble is for me, I have to remember parts of what you say. And my question really was gonna be, when you look at raw intelligence today, is raw intelligence per token actually improving like we're told it is?

  17. 39:44 – 42:19

    Is Intelligence Per Token Really Improving?

    1. DA

      Ooh. Uh, I'm curious on your take on that. I actually want to write a blog post on this.

    2. HS

      No, I don't think anymore. I think it was dramatically. Um, and I think raw int... I, I think it, it is, but, like, m- minutely, like, y- almost to the point where you wouldn't know. And it's kind of like, you know, you meet a 180 IQ person then a 190, you're not gonna tell the difference. So yes, but not significantly.

    3. DA

      Uh, I actually think even the 180 to 190 IQ person is an extremely generous, uh, analogy to what LLMs are doing right now, 'cause I wouldn't call them 180 IQ people or 190. You know, like, I would call them, like, occasionally maybe it's 180, I don't really know, but, like, then sometimes you get, like, an 80, uh, or even worse. Um, and my take is that it's actually not a single dimension. So number one, the benchmarks don't matter, so we should toss them out. If you try to look at bench- like, dollar per benchmark over time, that's complete bullshit. Um, number two, I think that there's, like, a general notion of intelligence, which is kind of like pre-trained intelligence per dollar of compute, which to my understanding is barely moving right now. I do think that as you get more data and as you focus on different subsets of tasks, um, you can make them more in distribution, back to the Bitrus lesson, and you get way better performance within that data distribution. Like, are we from a, a, uh, you know, intelligence per dollar perspective massively moving the needle on coding? Super-duper, but not because the base models intrinsically are more general, but because we're putting coding more in distribution, so it's more reliable on it. But, like, has writing improved in the last, you know, uh, nine months as... I, I don't think so. Maybe, maybe I'm writing with them wrong. Uh, you know, but you, you tell me. Um, but, like, there's many other capabilities where, like, there's absolutely no improvement at all, and it, it becomes... It's, you know... Because if you keep scaling up data in, you know, coding agent land, it's not going to improve lots of the other forms of data, right? It, it, there, there's diminishing returns to it. So I think it's a, it's a complicated and nuanced.

    4. HS

      You said there about, like, the variance in terms of model capabilities, sometimes 180, sometimes 80.

    5. DA

      Mm-hmm.

    6. HS

      Can I ask you, when we look at the different models, we see open source now on many different levels be as close as possible to a lot of frontier in, in a lot of respects. How do you see the open model landscape?

    7. DA

      Oh, boy.

    8. HS

      And what do not many people know that they should

  18. 42:19 – 45:44

    Why AI Benchmarks Are “Total Bullshit”

    1. HS

      know?

    2. DA

      Oh, that last question's a hard one. Um, oh, boy. Uh...

    3. HS

      You can say anything. It's just me and you.

    4. DA

      Yeah. Um, so I, I think the thing that people should know is that benchmarks are total bullshit. All benchmarks are bad. There's perverse incentives, and ironically the people it hurts most are the people who do not have time to actually play with the models, 'cause they're the ones who get manipulated by the benchmarks themselves with higher stakes, including for everything open weight and open research. Uh, I think that there are ways of measuring intelligence. It is very hard. Labs don't publicly talk about it. Um, they have somewhat proxies internally, of course. And it's a very unfortunate public state of affairs that there's so much misinformation out there, and, like, people are not incentivized for actually releasing their own best measures of intelligence, which I think is tragic. And, like-

    5. HS

      Diogo, who, Diogo, who, who bullshits the most?

    6. DA

      Who bullshits the most?

    7. HS

      You said spice on that, so I'll give you spice.

    8. DA

      Oh, okay, okay, okay. The top-tier labs are incentivized to be truthful. So I believe that OpenAI and Anthropic are mostly truthful. The second-tier labs are incentivized to lie to themselves. Like, they want to think that they can become a first-tier lab, so they have lots of internal pressure to benchmark max, but they cannot admit it to themselves. Otherwise, you know, like, their org chart will go ballistic. This is what happened with Llama 4, where they were actually too on the nose about benchmark maxing, even though there was so much internal incentive to do that. Same thing for Google right now, and same thing for a lot of the other Chinese labs. Um, like, I think that they're trying to aim for the top, and therefore, they are not trying to explicitly benchmark max. I think a lot of the people fighting for, like, um, consolation prizes are just doing completely insane levels of benchmark maxing. So one domain, if I were to be, like, a little spicy here, would be, uh, the rough domain of American open source leader. You know, like, it's s- it's enough of a, like, a side, uh, consolation prize that they're n- not aiming for top tier, and that is pure bullshit and optics because no one actual... You know, like, people don't actually use those models, so who's gonna call them out, you know? And the decision model landscape is so much worse, in my opinion, too, because no one even understands what good performance is, so they are all, like, benchmark maxing to the absolute gills. And, um, it's kind of, uh, really good for us because no one will believe those benchmarks. It gives a lot of power to us over the benchmarks itself, which I think is the optimal for the field. Like, in the f- in the ideal world, no one ever cares about benchmarks on any machine learning model, string or system one. But th- th- there, there's so much of that right now.

    9. HS

      I, I, I gotta-- Let's unpack this, dude. Uh-

    10. DA

      Ooh

    11. HS

      ... top tier-one labs have a, like, um, a duty to be truthful, or is in their impact, or that they-

    12. DA

      No, sorry. It's not a duty. They have a-- They do not need to cheat in order to, um, be high up there, and they want to truly believe in... Like, morale is very important. They truly want to actually be top tier, and they can be. So they don't need to cheat, is the

  19. 45:44 – 48:34

    Will US Open-Source Models Catch Chinese Models?

    1. DA

      answer.

    2. HS

      D- I, I think there is this feeling in the community that we need to have an American open source leader, and we need to propel this company forward, whether it's Reflection or, or Poolside or Thinking Machines. Do you feel that we do need to propel an American open source model?

    3. DA

      Wait, wait, what does-- Who is we, and what is propel? Like, is the, is the-

    4. HS

      Well, I think there's a, I think there's a consensus that Chinese open source models are very efficient, and unless we have an alternative to them, Chinese open source models will be considerably used and threaten some elements of American security.

    5. DA

      So it's, it's actually interesting to dig into the threat model. Um, like, th- there, there's definitely a threat if you, like, run them vanilla as how they've been post-trained. But if you're actually using, um, I can't remember what you f- how you described it. Like, you know, like, the private intelligence of companies. In theory, if you're training the models, you can remove a lot of bias to different extents. But, like, in theory, this type of thing is totally doable. So it actually depends on, like, is the threat model, like, literally plug and playing these models.

    6. HS

      So you don't think there's threats from, from Chinese open source models in terms of dangerous prompt injections that you don't know about, back doors?

    7. DA

      So, uh, I would not use a Chinese model in my coding agent vanilla, just to be clear. But, like, that again is one threat model. If you live in a world where coding agent is the only shape of AI, then sure, that is very much a threat. But the, the alternative we need to, like, take into account is, like, how good are the American open source models, and can they catch up to the Chinese models? Because they will-- My understanding is they hold themselves up to a higher ethical and legal baseline, and it might be very hard to catch up to those models because of that. And if the models are actually not useful, like, it really depends on, like, how much of a difference it makes to have a slightly smarter model, which is going to be hard to forecast, especially as things are plateauing. But, um, like, it, it, it might be a fool's errand if they cannot be competitive. So that is going to have to be, like, an open question for the field i- if you just assume the extremely narrow view of plug them into coding agents literally as is. Um, if you imagine a world where there's many other alternatives, like, hey, we can measure bias. We can have threat models for bias. We can train the bias out. We can put guardrails on it. You know, like, this is actually a huge use case for us right now, which is Jev classifying if the tool calls are safe, or are they phoning home to China or whatever else. Like, there's so many other ways to get security that I think that everyone has just been in this, like, mono model, like, horse blinders for so fucking long. And, like, I, I just reject that premise originally.

    8. HS

      [laughs]

  20. 48:34 – 49:52

    Why Diogo Thinks the Answer Is Probably “No”

    1. HS

      Do you think American open source models will catch up?

    2. DA

      In terms of true intelligence, my guess is no, unless China s- decides to stop trying, not due to capabilities, but due to ethics. But it might not make a difference. You know, like, if the field plateaus hard enough, like, maybe, like, yeah, that x percent better doesn't matter.

    3. HS

      Do you think we're right to have the export ban with China?

    4. DA

      My understanding is, like, there's different parts of the, uh, d- I don't know if you mean, like, the GPU expert ban, uh, GPU expert ban-

    5. HS

      Yeah, the GPU export ban specifically. 'Cause th- they're, they're showing-

    6. DA

      Oh

    7. HS

      ... on, on some of the latest, like, DeepSeek models and some of the latest Chinese models, they were in fact actually changed with the latest Blackwells.

    8. DA

      Oh, yes. That was, uh, Will DePue's result. Um, we've been, we've been, uh, ta- I've been talking about that with my friends. Though there's some spicy stuff online. What I heard online was that using the same technique de- You could tell DeepSeek 4.1 Flash wasn't trained in that way, and maybe that is consistent with the field. I don't know. My job is to navigate all of the insane hardware bubble Neo cloud craziness in a way that we end up getting the future we want. And because we are so efficient, like, we can be in, like, the, the sidelines and not be, like, a main player in the GPU stuff, and hopefully ride out whatever crazy stuff is happening right now.

  21. 49:52 – 53:16

    Why Diogo Thinks AI Is Already in a Bubble

    1. HS

      The crazy stuff that's happening, you mentioned that and Neo clouds. Dude, it's a wild space in the Neo cloud layer.

    2. DA

      Mm.

    3. HS

      What happens in the next 12 months in Neo clouds that people are not contemplating or seeing?

    4. DA

      Well, I believe that AI has been a bubble for a while, so I would like to keep planning for an eventuality of bubble burst. And my-

    5. HS

      What, what, what will happen in the bubble burst? Because to me, it's not gonna be a supply side, it's just gonna be a demand side, and we're seeing no demand-side shock. And so what will cause a bubble burst?

    6. DA

      Okay, uh, we, we can get into that. That's a spicy one. Coding agents have definitely changed the game for AI, but this argument has been made for many years now. And actually, I think it's very interesting when you combined many different, like, exponential variance-type things. Um, I believe that some amount of demand is caused by FOMO. FOMO is caused by people thinking that they have good results and thinking that they have to, like, go all in on this type of thing. And I do believe, like, demand can burst quite quickly when people start facing the realities of what AI can do. And my experience of people who, um... My experience of a lot of the world of, like, people spending money on AI is that they do not have a full understanding of AI's capabilities, and they're quite miscalibrated and actually quite overestimating of what AI can do. And I think that's the kind of thing that can cause a demand shock. So I, I don't know. I am not an economist either. I think that there's just been so much overpromise, underdeliver in the AI space.

    7. HS

      That goes against everything that people say, which is that if model progression did not improve for a significant amount of time, there is still so much value left for humans to extract that they haven't extracted yet. That kind of goes against what you're saying.

    8. DA

      Well, uh, I believe that the people who say that are the people who try to extract value of, of out of existing models. So there's a little bit of bias into that. I don't disagree with the spirit of what that said, that says, which is, "Oh my God, the models are so fucking smart." But again, we need to look at... I agree that they're so fucking smart. I just think that, like, their interface and, like, how that smart is unearthed is suboptimal. And again, we need to look at the flip side of the people who went way too all in into AI and got burned by it, right? Like, there's a lot of companies who are like, "Let's fire a lot of our workforce because AI is solving all of these problems," and they need to, like, backtrack on that, and that has, like, a lot of long-term ramifications, uh, to, to, like, the, you know, the future of their companies. And those people tried to not just extract the maximum or overplay their hand, they tried to, like, actually just bet logically with the reality that AI can do a lot more. So I actually think that, uh, that reality is very nuanced, or that story is very nuanced on whether or not, you know, people have the capability of extracting a lot of that value, even if the, you know, capabilities pause.

    9. HS

      I think a lot of people look at the margin structures of a lot of these companies and say that it's bubble-like. Do you have good margins?

    10. DA

      We do, yeah.

    11. HS

      How do you get to scaled revenue, dude? Like, you're, you're so cheap, which is brilliant-

    12. DA

      Mm-hmm

    13. HS

      ... and the cost is, is a huge benefit of using Jev. But the, the scale required to get to a $10 billion revenue business-

    14. DA

      Yep

    15. HS

      ... is just enormous.

    16. DA

      Yep, yep, yep.

    17. HS

      How, how do you help me get there?

  22. 53:16 – 57:06

    How Jevons Paradox Could Create Massive AI Demand

    1. DA

      Uh, Jevons paradox, right? Like, and, and the way Jevons paradox tends to work, um, it's not like just magic, you make it cheaper, and it gets better. That's, like, an insane way the world would work. Um, the way it should work is, like, hey, I have this budget for AI. Oh, Jev makes it that much cheaper. Now I can make this project work, and ideally it works so reliably because Jev's also smarter on the axes that matter for automation, that now I can do these other projects or I can throw intelligence at things I would never throw intelligence at. And even then, if you're starting to throw intelligence at things you haven't throw- thrown intelligence at before, that would not make you increase your budget. And Jevons paradox means increase the budget. Um, what you would need to see is so much value from those projects finally succeeding, which AI tends to fail at, that you're like, "Hey, this actually works. Let's keep betting until the bu- budget matches what it was before and even increases beyond that." So this is why reliability of intelligence is so important. And what I like to say is in the grand metrics that truly matter, which is real automation, Jev, Fable, and Astra are all tied at roughly 0%. Like, yes, there's k- you know, like, a couple of little bits here and there, but, like, we have so freaking much automation to do that I think that, uh, like, li- like, you know, like, we're trying to make an industry that currently doesn't exist, which is run-in-the-background AI. And I think that's gon- that would clearly be huge, and it would be clearly huger than the whole... Huger? Is that a, bigger? Um, than larger. Thicker than, uh, a coding agent, uh, assisting software engineers a la making software, like, 20% more efficient to write or something like that.

    2. HS

      How do you analyze the coding agent market today? I mean, uh, you know, uh, you see Cognition fourth place, uh, so far, be a $1.6 billion revenue business.

    3. DA

      Mm-hmm.

    4. HS

      The depth of this market is so large. How do you analyze this market today and, and what does that look like in three years' time?

    5. DA

      Who's third place?

    6. HS

      Well, you've got Claw Code, you've got Codex, you've got Cursor, you've got-

    7. DA

      Uh, okay, cool, cool

    8. HS

      ... Cognition.

    9. DA

      Yeah.

    10. HS

      You've got Factory.

    11. DA

      Like, I, if I will be totally frank, uh-

    12. HS

      Yeah

    13. DA

      ... I don't analyze the coding agent market. I actually, um, I, I personally want Jev to be used for coding agents, and I think there's very cool sci-fi that could be done. I actually think that the further away we stay from the coding agent market, the more that they keep doing their little fight and we can keep focusing on automation. So from a company perspective, it might even be better in short term if Jev had absolutely no coding agent users in order to make it a little bit easier. It's a little bit too late for that, but I can't tell. Um, but I don't analyze the size of that market. I, I, I, I think that it will be a very large market. Will the market be much larger than it is today? I don't know, because FOMO really moves the needle on these things. Like, I've been around in AI long enough, and surely you have, too. You must have seen, like, uh, l- like, AI looks like this exponential upward, but it's actually been many up, down, up, down, up, down, up, downs. And each FOMO of the previous up adds to the next up thing. But, like, a lot of them have just, like, crashed down. And, like, who's to... I, I, I, I think it would be kind of arrogant to assume this is the one thing that will just be up, up, up, up.

    14. HS

      I so remember, like, 2015, 2016. There was, like, this real, like, hype for about a year, and there was, like, Bloomsbury AI, Message.ai-

    15. DA

      Mm-hmm

    16. HS

      ... and, like, a load of these companies, and it was exactly that. Um, so yes, I completely get you there. We said about the bubble-like moment. With the greatest of respects, you raised a shit ton of money.

    17. DA

      [laughs] Sure. Um-

  23. 57:06 – 1:00:49

    Inside TypeSafe’s $750M Raise at a $7.5BN Valuation

    1. HS

      I mean, tell me about that round. Like, w- was that just like, you know, stop off at one VC, collect term sheet, stop off at another VC, collect term sheet? Yeah. Ah, I've got 28 term sheets. That's, that's... How did that round go?

    2. DA

      It was a very, very confusing time for us. It was kind of an insane time. Oh my God, did we learn so much about VCs themselves. Because-

    3. HS

      What did you learn?

    4. DA

      Well, how technologically advanced the VC's portfolio companies were. Um, like, we, uh, like basically it was all inbound. Um, and we, like, did not have a deck. Um, you know, like, we had to, we, we had to tell people, like, "Hey, we don't have time for this deck, and it doesn't seem anyone else needs this deck. Um, like, you tell us about the market signals you're seeing because, like, we are, everything's on fire right now, and we don't really have the time to pull these things up for you." And it was kind of so sick to see how it evolved for different VCs over the course of the first days and weeks. Like, like, how their excitement changes, how they are telling us how their portfolio companies are using us. We don't snoop on their data, so we actually don't know what they're using it for. And it's so cool to hear them about, like, "Holy shit, you guys are the real deal. You're crucial. This is the future." And that was, like, a really fascinating additional s- I, honestly, I missed that signal on, like, the state of the field. And it was so fucking cool to hear about who's using us and why and how, and, um, that is what enabled the round to work out.

    5. HS

      Did you start with 750 as the ask at 7.5, or did it move up?

    6. DA

      We, it, it, it, it, uh, it moved around a lot. When we even started the conversations, we did not know what we were really looking for in the round. What we ended up deciding on is, like, "Hey, we don't actually..." It, this is maybe, like, kind of crazy because we did raise a lot of money, but we realized that, like, hey, we don't, we don't need to valuation max. We don't need to do, like, a cuck tranche or anything like that. We don't care about that. We don't need, we actually don't need a fuck ton of money either. Um, what we need is allies to fight the largest companies in the world. Let's find the best people to, like, be part of the team, and that's what we really optimized for. The rest to me was, like, more in the micro-optimization territory versus discovering, like, hey, we want, like, the smartest possible money, the best allies, the people who can help us navigate the rocky waters ahead, the, the rough waters ahead. And-

    7. HS

      Yeah

    8. DA

      ... it was, uh, I think we're optimizing from very different things from other companies because, like, we are profitable. We are, you know, money in the bank is going up right now. And qu- quite frankly, a company that's raised as little as we'd raised before should not have a plan to spend, like, you know, whatever we raised right away.

    9. HS

      I can't believe cuck tranche is actually a word these days, but let's roll with it. Um, it, it, it-

    10. DA

      Wait, y- y- you, you know, you know about that, surely.

    11. HS

      I do. It's completely fucked, and it makes me just-

    12. DA

      Yeah.

    13. HS

      Yeah, i- I, absolutely. I, uh, but I have a very big ego.

    14. DA

      Yeah, and like, that, that came up for us. We had so much demand for that. Um, and people were, like, even saying, like, "If, if you're gonna be raising at 3X the valuation, count me in." And, like, the thing is we didn't need the money right now.

    15. HS

      But that, that's, that's a bubble. Like, that's just fucked. That's bullshit. Like, I'm sorry. I lo- I, I do- I really like... Like, uh, to someone who's like, "YOLO 21 billion, I don't care."

    16. DA

      Mm.

    17. HS

      You know what? You just need to get some self-esteem because that is, like, sitting and watching. [laughs]

    18. DA

      Yeah, well, uh, I am, I am, uh, not... Uh, well, well, uh, the thing is we don't need w- we don't need the money or the valuation, right? So, uh, we don't need to do

  24. 1:00:49 – 1:04:43

    Why Diogo Chose Andreessen and Sequoia

    1. DA

      that stuff.

    2. HS

      Why did you choose Sequoia and Andreessen? I love them, but, like, why did you choose them?

    3. DA

      Oh, man, it's gonna s- seem like I'm sucking up, but I love them a lot. Um, Andreessen is... I, I am, like, kind of insane about technology and techno-optimist, and I think a6d is just, like, the embodiment of that. You know, like, they are as philosophically aligned as we can be. I feel like most VCs are, like, they are, from a strategic perspective, lovers of technology, but they are financial institutions that care a lot about how to play their financial cards. While Andreessen, like, deep into the firm, like, they care so, so, so, so much. Like that podcast I did with, like, Martin Casado and Ben Horowitz, I had so much fun. It felt like we were, like, old friends just talking about from our hearts. I mean, maybe they are so good that they can convince me and they've sold me, like, on, like, some fake ass thing, but I don't believe so. I think that they just are so philosophically aligned, and I think, like, what led us to where we're at right now, and I hope we can keep being this way, is, like, wearing our heart on our sleeves about what we care about and just- Blazing our own path in the AI field, and I would love to act in accordance with that and, like, have them on board. Obviously, Martin is brilliant. Um, and actually, to me, Sequoia, if I'm, like, gonna be totally frank, um... Oh, man, how do I say this? Um, I didn't want their money. I wanted Sonya Wang. Um, and, uh, that is why Sequoia's around. I did not get to know Sequoia super well as a fund, but I love Sonya as well, and I just would love to-

    4. HS

      Well, what, help me, help me understand. I, I wanna be the best ambassador. What did you love so much about Sonya that made you choose her over everyone else?

    5. DA

      Oh, man. I would say that she is not just brilliant, but a very unique type of brilliant that I think that we are extremely-- we, we extremely would want on our side. So it's that particular shape of brilliance that I find that, like, no matter how many advisors I would talk to, if I have a conversation with Sonya, like, I would get a different, very interesting perspective that should be part of our strategic cards. I care a lot about strategic thinking for myself. Like, one of my spicy TypeSafe values, you know, is make d- great decisions slowly, very non-startup, but I think that that is just, like, how it should be done. And, like, uh, she is an incredibly valuable thought partner there.

    6. HS

      Make great decisions slowly.

    7. DA

      Absolutely.

    8. HS

      What, what bad decision did you make fast?

    9. DA

      I would say I probably make decisions slower than most other people. I, uh, oh, man. My sense is a lot of the business world is on autopilot. I hate working with traditional businesspeople. Like, they just do things, and I'm okay with doing things if it doesn't matter how well they are done. But, like, for important things, I actually care a lot about the why. What's the philosophy? What's the framework? How is this done? And whenever I work with traditional businesspeople, I feel like I'm being driven crazy because they never do things so well that I don't need to worry about it. And when I do worry about it, I need to understand the why, and I'm like, "Oh, this person's just doing random shit." You know? Like, uh, but, like, I, I think having a plan actually matters quite a bit, and there's so, there's so many things I wish I would spend way more time thinking on. Like, I wish I could spend more time thinking about, like, our data and our data roadmap. I wanna think more about product. I wanna more, think more about, like, you know, how to be more trustworthy. I wanna think about, more about, like, every single role we hire for to make sure that we get, like, the best possible people for these roles. And I think that I should acknowledge that I don't have time for all of these things, and it's so fucking important to, like, think, think, think.

  25. 1:04:43 – 1:09:22

    Why TypeSafe Refuses to Pay $20M AI Compensation Packages

    1. HS

      How hard is the hiring market today? You have to get the best people on your team. How hard is it actually?

    2. DA

      Oh, man. I will say that the hardest part is hiring for, um, our data demon team. It is-- It's-- So, you know, back to Bitress lesson. When you have a new task, you have new type of data. You know, you need a new shape of data in order to make it work. And we basically have our own little subfield of, like, it's, like, kind of like imagine, like, a, you know, the best lab in the world, but only internal to a company to make data the way we make data. And I would think that this should be the easiest thing in the world to hire for because it's so unbelievably cool. And also, we have such a hard time hiring for it because no one knows, uh, how to get into it. You know, we keep getting the wrong type of people applying. And, like, that to me is the thing that keeps me up at night these days of, like, how can we convince, like, these brilliant people who don't even think they can do-- like they can push the boundaries on ML because they're not an ML researcher to work at this role where we literally have a subfield where you'll get tutored by the experts in the world it is. And it's gonna be so valuable because you actually make the model smarter. And, you know, like, I don't know how to communicate that to the world, and that to me sucks.

    3. HS

      Is, is the comp wild? We, we had Eric at Modal say that he has people who he has to fight twenty million dollar packages for some grads who are utterly brilliant. Are the comp packages just unprecedented?

    4. DA

      Not for us. Um, and I'm ha- I'm so happy that we got clarity on this. Like, I actually think that, uh... Like, we got real clarity that we do not want mercenaries. And, like, uh, we, we want to be, like, good, but not like, you know, like the insane lab ones. And I think that that feels great. Like, I feel, I, you know, like, A-Anthropic, I think, is suffering a little bit from their over mercenaries lately because, like, they want missionaries obviously, but also they pay more than anyone else. So, like, there's a lot of incentive for not just mercenaries to join, but, like, fake their culture interviews, which there's, like, an entire market now to prepare people for Anthropic culture interviews. I am so glad that we decided to not to do that. We want to pay people, like, good amounts. That is not the highest thing ever. And we are just super happy where, where we're at algorithm-wise of, like, this is the kind of thing that takes care of people and also filters out the, the, the, the hired guns.

    5. HS

      Final one for you. If you could bake-

    6. DA

      Oh, yeah

    7. HS

      ... one character trait into a potential hire, what would it be? For my company, it would be obsessive. Like-

    8. DA

      Mm-hmm

    9. HS

      ... I need someone to be obsessive about their craft, addicted to making it better. They can't sleep at night. It actually compels them in every way. The ability to become obsessed with something.

    10. DA

      I don't know if that's the same for every role. But if I had to pick, I don't know a better word for this, uh, and it sounds a little sexual, but I-- we don't mean it in a sexual way. Like technophilia. You know? Um, the nonse- not non-sexual. Just to be clear, not sexual. Okay, relax. But, like, we truly, truly love technology. And I think that- Like, that's the kind of thing that's, like, so beautiful in a bubble, and it's the kind of thing I want in every single role, so we can talk about technology at lunchtime or when we're hanging. I want people around me that I like, and I want us to all be, like, this one big team working towards, like, constant, constantly pushing the boundaries of sci-fi because that is how we, we actually ship technological shock and awe. Uh, again, non-sexual, I don't know a better word, technophile, techno- Yeah, yeah, yeah. Just to, just to be clear. Uh, if, if you guys edit this out, that'll be very embarrassing. [laughs] But, uh, yeah.

    11. HS

      Well, don't worry. We- we'll, we'll keep it, and, uh, no, no sexual.

    12. DA

      Oh, yeah.

    13. HS

      We're gonna do a, we're gonna do a quick-fire round now. So I say a short statement.

    14. DA

      Oh.

    15. HS

      You give me your immediate thoughts.

    16. DA

      All right. Okay.

    17. HS

      First-

    18. DA

      I, I tend to not do quick-fire, as a warning. I'll try

    19. HS

      What have you changed your mind on most in the last 12 months?

    20. DA

      How much hype matters.

    21. HS

      How much hype matters?

    22. DA

      Yes. I, I think it matters much less than I thought. When Jev launched, it's like, holy shit, hype matters a lot. And then other stuff launched, and they were, like, benchmark maxing to the gills. And when we talk to real people, everyone is like, "Yeah, we tried them. They suck. No one cares." Like, we can tell already that you are the best and you care the most. And I was like, wow, this is so convenient. We can just do what we love most, which is underpromise, overdeliver. This is the best world ever. So, uh, uh, the... I would not have expected this at launch time, and this is so awesome for us.

    23. HS

      What is the biggest belief you hold that the smartest people at Frontier Labs would disagree with you on?

    24. DA

      How much AI can do today and how re- important reliability

  26. 1:09:22 – 1:10:50

    Why Diogo Doesn’t Hire OpenAI Researchers

    1. DA

      is.

    2. HS

      Why do you not hire from OpenAI?

    3. DA

      My sense is that there's a certain selection bias you need to do to get into OpenAI that is kind of like an anti-signal, um, where I feel like you need to be kind of in a particular Kool-Aid, um, number one. And number two, l- like, I, I... When I say this, I mostly say I don't hire researchers from OpenAI because the researchers at OpenAI, by construction, love to do, like, algorithms and then scaling, and they tend to hate data. Data is a very low-status thing at most labs. And obviously, they don't think about the right task because that is a thing that is, like, quite rare. There are, like... There's maybe, like, three people at OpenAI who think about the right task. And I wouldn't hire them either, not because they aren't fantastic, but because if you're obsessed with the right task, why would you want to do my right task? Like, my right task is machine native AI, right? Like, if you're obsessed with, like, making, like, text-to-image a thing, which Aditya did do at OpenAI. Like, holy shit, he invented an entire field. That is so fucking cool. He is so unbelievably good, and also, like, he... You can't, like, transplant him into another task, right? Like, like, he should just be cooking on his sci-fi too because that is what you do when you're, like, so good at that. Um, so by construction, the... all that's left of the ML research that could be hireable are the ones who are obsessed with algorithms. And that, to me, is the kind of thing to do when you want to throw money in the fur- furnace.

  27. 1:10:50 – 1:11:17

    Why Robotics Is the Most Overhyped AI Category

    1. HS

      What is the most hy- overhyped category today?

    2. DA

      My instincts tell me robotics. Um, like, uh, not because robotics is not cool, but I just think that if you look at the grand scheme of reliability, they're just very far away from it. If you imagine what are the things that need to be nailed first, um, there's, like, tons of prereqs that still haven't been nailed. Um, and I think robotics is the furthest away from that.

    3. HS

      What is the most underhyped?

    4. DA

      Um,

  28. 1:11:17 – 1:13:19

    The Most Underhyped Opportunity in AI

    1. DA

      in my view, this s- direction of software research. Like, people playing with AI plus software to make new things, especially new interfaces. I think that that's so unbelievably underhyped. It's, like, tragic.

    2. HS

      Final one for you. What do you believe today that looks crazy, okay, it looks crazy, but will be incredibly common in five years' time?

    3. DA

      I think that principal agent problems are abound in the world. You know, like, when you have a principal, like an electorate, being represented by an agent that is sometimes misaligned with it, like a representative. And why can't the electorate directly represent their desires and have it behave more like a true democracy? This is just, like, one contrived example, but it happens all over the place. Like, people reporting to you, stockholders to companies. Like, there's so many principal agent problems abound. And my view is that we will find way better solutions to all of these, and it's going to be crazy that we allowed the level of misalignment that we have today. I think that will be such a, such a sweet world where we can start enforcing the letter of the law instead of... Sorry, the spirit of the law instead of the letter. And AI is basically that, right? Like, the, like, im- like, the, the law is kind of like writing software. You know, it's like writing, like, a detailed algorithm to follow in order to make it fair and reproducible. And it was just missing the primitives to specify the spirit. And I think that this will really unite society 'cause I think even for our most controversial topics, I think that a lot of people can unite on the spirit of things. It... The problem is that the spirit can't be enforced, so you need to put the letter, and the letter then has a s- ton of disagreements.

    4. HS

      Do you know my problem with you, Diogo, is that you just don't have much energy. You're just very-

    5. DA

      [laughs]

    6. HS

      You're just monotone, you know?

    7. DA

      Oh. [laughs] Yeah, yeah, yeah.

    8. HS

      D- d-

    9. DA

      M- my, my bad. My bad.

    10. HS

      This is-

    11. DA

      There's a reason that they call me Cocaine Labubu on Discord. [laughs]

    12. HS

      [laughs] Dude, this has been so much fun.

    13. DA

      Hell, yeah.

    14. HS

      Thank you so much for bringing the energy-

    15. DA

      Hell, yeah

    16. HS

      ... and for being so willing to be so open.

    17. DA

      Hell, yeah. It, it was tons and tons of fun.

Episode duration: 1:13:29

In the uListen app, the transcript lights up word by word as you listen. Get uListen

Transcript of episode BE2JqjsN4aY

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.