EVERY SPOKEN WORD
60 min read · 12,081 words- 0:00 – 0:54
Intro
- MCMartin Casado
The world that the AI lives in is all digital, and there's the actual world.
- JNJad Naous
Yeah.
- MCMartin Casado
And, like, those things are not necessarily the same.
- JNJad Naous
When we started doing AI for coding, it was auto-complete. Nobody really knew to what extent you can actually build complex software using AI. And then it basically broke through the expectations because we were able to model more and more complexity in the systems.
- MCMartin Casado
I just don't know if operations is constrained in the same way. Where is even the data to learn how to, like, build a system that understands, like, all of the operations?
- JNJad Naous
My assumption is that we have all the data. The question I think is maybe about complexity and, like, to what extent can you take the things that you've described that are operational-
- MCMartin Casado
Yeah
- JNJad Naous
... and convert them into something that an AI can reason about?
- MCMartin Casado
The dirty secret for the longest time of running, like, a distributed cluster was just human beings managing it. Do you think we're at a spot where you could, like, use AI to run one of these services?
- JNJad Naous
I think that there's enough to be able to-
- 0:54 – 7:23
From Stanford to founding Grepr
- MCMartin Casado
All right, Jad. Thanks for joining me on the podcast. [chuckles] For those of you who don't know, Jad is the CEO of Grepr, but also we, uh, went to grad school in the same group. And when I first met you, I-- we, we had a new student.
- JNJad Naous
No.
- MCMartin Casado
[laughs] I, I walked into the office, and there was a motorcycle helmet and a pack of Marlboros. [both laughing] And I'm like, "This guy must be unbelievably cool." [laughs] So my question is, is what happened to all your coolness? [both laughing] He's like, yeah.
- JNJad Naous
Question number one. [both laughing]
- MCMartin Casado
Maybe, maybe, maybe just for context, maybe be give, give, um, a quick, um, uh, overview of your background. So just, just for those that are listening, you know, Jad is one of the best systems thinkers I know. I love talking to him. We're gonna go kind of do a deep dive of, like, engineering from a systems perspective and how AI is impacting us. We're gonna kinda... And especially when it comes to things like operations. But just to ground it, maybe a bit about your background?
- JNJad Naous
Sure. So, uh, you met me at, uh, one of the peaks in my coolness.
- MCMartin Casado
Yeah, you were pretty cool. [both laughing] Motorcycle and Marlboro cigarettes. [both laughing]
- JNJad Naous
Oh.
- MCMartin Casado
I remember thinking, I'm like, "Wow, are cigarettes even legal?" [both laughing] 'Cause it was, like, so discordant.
- JNJad Naous
I remember Nick seeing me one day smoking outside of, uh, outside of Gates and looking at me and going like...
- MCMartin Casado
[laughs] It was 'cause Ga- N- Nick, Nick Bakayunas was our advisor. Gates was the, the, the building, yeah.
- JNJad Naous
Um-
- MCMartin Casado
Actually, one of my favorite thing, I don't-- do you remember, uh, Balaji Prabhakar, the professor?
- JNJad Naous
Of course.
- MCMartin Casado
My favorite thing about him was that he smoked. [both laughing] He was, like, the only professor. I just thought it was, like, great.
- JNJad Naous
Just to be clear, I don't smoke anymore.
- MCMartin Casado
Yeah. All right.
- JNJad Naous
No. Too old. Too middle-aged for that now.
- MCMartin Casado
[laughs]
- JNJad Naous
Um, but let's see. So, you know, I went to Stanford with, uh, with you. You were always the guy I had to look into, to look, to look up to-
- MCMartin Casado
Yes
- JNJad Naous
... and be like, "Hey, this is the guy I wanna be when I grow up."
- MCMartin Casado
[laughs] That was, that was, I think it was, I, it was, uh, I started four years before you did, so-
- JNJad Naous
Yeah
- MCMartin Casado
... I was ahead of you.
- JNJad Naous
And so you were four years ahead of me. You graduated a while, uh, ahead of me. You went and started Nicira.
- MCMartin Casado
Yeah.
- JNJad Naous
I stayed and stuck around, worked more on OpenFlow, helped, uh, build a bunch of stuff on the hardware side from a security perspective.
- MCMartin Casado
Yeah.
- JNJad Naous
Actually, one of the f- one of the, you know, most fun projects that I worked on there was, uh, I built the first set of, uh, SHA-3 implementations that people were considering using. And, um, I built them in hardware to kind of show whether they work or not and whatever for one of my, for, uh, one of my projects for my thesis. That was fun.
- 7:23 – 10:15
Why observability got so expensive
- JNJad Naous
like what is, uh, why do people think it's too expensive? [clears throat] Turns out there's a few things. First, people were spending somewhere between 5 and 20% of what, uh, they spend on infrastructure, on observability. But ultimately, what they were really trying to spend it on was reliability. They wanted to buy reliability. And, um-
- MCMartin Casado
Of spend, like predictability of spend?
- JNJad Naous
Reliability of their infrastructure and applications.
- MCMartin Casado
I see. I see. I see.
- JNJad Naous
Right? And so-
- MCMartin Casado
Yeah
- JNJad Naous
... they were putting money into observability as a proxy for reliability, and for the longest time, the more money you spend, the more reliability you got out. But as architectures changed and turned into microservices that are much smaller, that are much more vertically integrated, the volume of data that's being collected at every layer expanded. There was much deeper market penetration for observability and kind of the, uh, volume of data that you're collecting and the cost of observability was actually starting to grow faster than the units of function that you're outputting. And, um, and the marginal rate of, or like the marginal increase in reliability for every additional dollar you were spending on observability was just flattening out. [inhales] Um, and so that's why people thought it was expensive, but people were still willing to pay 5 to 20% on reliability. And so the question was like, can we actually solve that problem and bring back the, um, that increase in reliability, that marginal increase in reliability-
- MCMartin Casado
Yeah
- JNJad Naous
... not necessarily by spending more money on observability, but some other way. And so the thinking was like, okay, great, if we can figure out what is the signal, what is the noise, can we actually turn the observability data that you're collecting, that's mostly fluff and mostly insurance at this point, and do that kind of extraction of information upstream, do whatever you need upstream, and be able to ultimately massively reduce the cost of observability downstream. And if you can extract that signal in real time upstream, then you can, one, reduce the cost, but two, you can now feed it into other systems and use it for driving reliability operations afterwards. [clears throat] And in the distance at the time, there was something that was called AI coming, coming into the picture.
- MCMartin Casado
[laughs] Yeah.
- JNJad Naous
And we're like, well, great. If we are in the data stream and we're extracting the signal, then we are in a strategic position to feed that to whatever is gonna come in the future, because that's gonna be necessary.
- MCMartin Casado
Yeah.
- JNJad Naous
And so that's what we built and, um, so Grepr helps companies, you know, if we started with helping them reduce their spend on telemetry massively, and so you put us in the data stream, plug and play, pass the data through, reduce your cost by 90%. And, um, and now we're building out that second layer where we're feeding those signals into the AI layer that's now going and troubleshooting and figuring out when things are going wr- going wrong and fixing them.
- 10:15 – 18:25
How AI is changing engineering
- MCMartin Casado
I'd love to just talk through kind of your experience on how AI is changing dev generally, which is like a lot of people get asked this question, a lot of people answer, but I just feel like, you're kinda like the engineer's engineer. [laughs] Like you've been an architect, you've run product, you've run engineering teams like, you know, like... Actually, how big is your company now? How many people?
- JNJad Naous
We're 10.
- MCMartin Casado
Yeah. So it's, you know-
- JNJad Naous
Still pretty small
- MCMartin Casado
... and, and you've got a, I mean, you're building a lot of software with a relatively efficient team. So m- maybe just a high level sketch on like, you know, if you're a s- senior engineer like you, you know, that has architectural experience like, like what sort of gains do you think people are seeing? Maybe talk through like how you use AI, what gains you're actually seeing, what is the stack, what is your process, how you think about it.
- JNJad Naous
So maybe I can give you like a historical view of like what's been happening-
- MCMartin Casado
Yeah, yeah. That's great
- JNJad Naous
... in terms of like m- running a team and so on.
- MCMartin Casado
That'd be great. Yeah, yeah.
- JNJad Naous
And so what, w- w- you know, when we first started AI, we're still kind of auto-complete.
- MCMartin Casado
Yeah.
- JNJad Naous
Right? I remember like at the time we were using, uh, Copilot and, uh, IntelliJ, and we're like, "Oh my God, look, it auto-completed these tests."
- MCMartin Casado
Yeah.
- JNJad Naous
Um, we're like, "Yeah, but it's just auto-complete." And very rapidly things started evolving and what, what I've noticed is that really good engineers are able to somehow put it into their workflows in a way where they, the AI starts training them.
- MCMartin Casado
Oh, interesting.
- JNJad Naous
In terms of like how to talk to it-
- MCMartin Casado
Yeah, yeah
- JNJad Naous
... like how to get the best out of it. And what I noticed-
- MCMartin Casado
Do, by the way, good, as an aside, do you think that's model specific? Like is it, do you learn how to use AI, you know, writ large, or do you l- get efficient at a given model?
- JNJad Naous
I think there's like some nuances that different models bring. Um, I'm sure looking at your tweets, you've [laughs]
- MCMartin Casado
Yeah
- JNJad Naous
Um, there's like nuances, but there is a, a way to work with AI and like, I think that there's also a way to work diff- with different harnesses. And I think like, you know, we tried, for example, using open code or like different harnesses or like using Codex versus Claude, like Claude Code, like different ones have different ways of managing context or thinking through problems or when to spin up the right sub-agents and-
- MCMartin Casado
Yeah
- JNJad Naous
... what to tell it, like the kind of prompts. Like for example, uh, with Claude Code, I find that if I don't tell it, "Don't spin up, you know, 20 fable agents to do basic research," it'll go and spin up 20 fable agents to do research and burn all your tokens.
- MCMartin Casado
Yeah, yeah.
- JNJad Naous
And so like a lot of these things kind of you start learning about the harness and the, and the models and like how they interact with each other and then how they can interact with the rest of the world. But I digress, like I'm just gonna... I- in my mind, like this is kind of what I started noticing is that like some people were adapting and learning and kind of doing it that way, um, and others weren't as adaptive.
- MCMartin Casado
Yeah.
- JNJad Naous
And then as we started hiring more people into the team, that disparity between, you know, okay engineers and really good engineers just multiplied 10X. And-
- MCMartin Casado
Was it, was it-- Can you predict who will t- could you predict who would be good? Was it like good engineers got better and bad engineers didn't? Or was it-
- JNJad Naous
Yes
- 18:25 – 22:55
Jad's AI stack and workflow
- MCMartin Casado
stack and your process.
- JNJad Naous
So I use, uh, Cloud Code. Um, I'm gonna say something that's actually pretty funny, but I use, uh, Windows. Um-
- MCMartin Casado
What? [laughs]
- JNJad Naous
[laughs]
- MCMartin Casado
Was that created by this company, Microsoft? [laughs]
- JNJad Naous
[laughs] And I use WSL. So I use Linux for everything, like development and so on. There's no, like, X or UI for it. I just use the-
- MCMartin Casado
I see
- JNJad Naous
... command prompt. But, um-
- MCMartin Casado
Why do you use Windows?
- JNJad Naous
'Cause I just never really wanted to learn the keyboard-
- MCMartin Casado
Are you-
- JNJad Naous
... shortcuts on a Mac
- MCMartin Casado
... are you a gamer? Is that [laughs] it? It's like you tab, tab over to Counter-Strike or whatever. [laughs]
- JNJad Naous
[laughs]
- MCMartin Casado
Like, why else? N- I'm not, not hating on Windows. I mean, you know, like, I used it in the '90s. [laughs]
- JNJad Naous
Uh, that's... L- really, like the only reason is I never wanted to learn the, the shortcuts. The... I, I hate the one-button mouse. I hate the one-button stuff. I hate the number of modifiers you have to type for shortcuts and stuff. Like, I, I just, I just never... I'm like, "Why? Why? Why? Why waste time?"
- MCMartin Casado
[laughs] Okay.
- JNJad Naous
So, like, yeah-
- MCMartin Casado
Kind of use Windows-
- JNJad Naous
... I'll use both Linux
- MCMartin Casado
... use Windows, yeah.
- JNJad Naous
Um, and so WSL, I use Cloud Code. Um-
- MCMartin Casado
What is WSL?
- JNJad Naous
That's Windows Subsystem for Linux.
- MCMartin Casado
Great. Great.
- JNJad Naous
So it's, uh, they actually run the Linux, uh, kernel within Windows in a hypervisor and provide you access to it.
- MCMartin Casado
Okay.
- JNJad Naous
It's great. They've, they've done a great job.
- MCMartin Casado
Great. Windows WSL.
- JNJad Naous
And, uh, Cloud Code.
- 22:55 – 24:55
A six-month project in ten days
- MCMartin Casado
Yeah. So how much time... So can you give a sense of, like, a project you do now versus, I don't know, six years ago, like time differences?
- JNJad Naous
So the latest project that I shipped was something like-
- MCMartin Casado
And this is like, by, by the way, this is like, to be clear, this is production code. This isn't-
- JNJad Naous
Yeah. Yeah, this is production code
- MCMartin Casado
... like demo and mock-ups, whatever.
- JNJad Naous
Yeah, yeah.
- MCMartin Casado
So, like, the statement you're about to make is the gains that you get for, like, shipping production.
- JNJad Naous
Yeah. So I'll give you the example of the last... There, there, there's, the last PR that I worked on, or, like, the last set of PRs, was what we call auto, auto promotion in a data lake.
- MCMartin Casado
Yeah.
- JNJad Naous
So in a data lake, you store all the data, and, um, typically the data is, um, column-oriented, parquet files and Iceberg and so on. Um, and when you get data from, um, logs, they're semi-structured, they're arbitrary columns, and so on. In order for it to, to have the best layout of the data and to be able to query it efficiently and fast, you kind of... The ideal situation is that every field in the log event becomes a column that you can filter on-
- MCMartin Casado
Sure
- JNJad Naous
... independently of the rest of the rows.
- MCMartin Casado
Yeah.
- JNJad Naous
And so we call that auto promotion or shredding.
- MCMartin Casado
Yeah, sure.
- JNJad Naous
And doing something like that is incredibly complicated, right? Like, you have to go and figure out how does the, the metadata system work, how do the commits of the metadata system work, how do you coordinate across different, uh, writers, and so on. And, um, that took maybe about 10 days. In previous days, that would've taken maybe about six months-
- MCMartin Casado
Wow
- JNJad Naous
... for a single person.
- MCMartin Casado
Wow. And was this just-
- JNJad Naous
It is-
- MCMartin Casado
... you? Uh-
- JNJad Naous
That was just me.
- MCMartin Casado
Yeah.
- JNJad Naous
It is insane.
- MCMartin Casado
Yeah, so like factor of 10, factor of 20 improvement.
- JNJad Naous
Yeah.
- MCMartin Casado
For production code for new products.
- JNJad Naous
Yes.
- MCMartin Casado
You know, one, you know, one, one, um, question I always have is, like, I can very much understand,
- 24:55 – 30:10
Can AI reason about operations?
- MCMartin Casado
you know, the, these models are trained on all the code ever written, let's call it, right? And so I can understand why they're able to generate code, which is the syntax piece. Um, but then, like, for, you know, you're doing core infrastructure. Let's say you're doing distributed algorithms. Like, like, like, the properties, uh, like, how it handles itself when it's running is very, very different. Like, often you just can't even look at the s- the syntax to know what's gonna happen, right? Like, a lot... Let's say, let's say you're, I don't know, you're, you're, let's say you're writing some distributed system, and then, like, the way that it, it, like, it'll perform, the, the availability, um, the correctness even, is, is gonna be dependent on what the workload looks like. You know, like, like, what are the access patterns, what is the traffic? Uh-
- JNJad Naous
Are you talking about, like, the performance aspects of it, though? Or are you talking about, like, the s- the semantic-
- MCMartin Casado
Well, I'm thinking availa- like, availability.
- JNJad Naous
Yeah.
- MCMartin Casado
Like, you may have to lock things so it's not available for a period of time.
- JNJad Naous
Sure.
- MCMartin Casado
I'm talking about correctness, like maybe you've decided to do eventual consistency, and so, like, if you do a read-
- JNJad Naous
Mm-hmm
- MCMartin Casado
... it's not gonna be consistent because there isn't a lock.
- JNJad Naous
Right.
- MCMartin Casado
Um, and then performance, like, you know, maybe you cache, maybe you don't cache.
- JNJad Naous
Yeah.
- MCMartin Casado
So these are all kind of trade-offs in the design-
- JNJad Naous
Right
- MCMartin Casado
... space, right?
- JNJad Naous
Yeah.
- MCMartin Casado
And, like, m- unless you know, like, the access patterns, you won't know how it's gonna perform in these things, right? And so I just think to myself, okay, I've got an AI, I can understand how it will write, you know, like, like, code for a generic distributed data store, for example. What I don't understand is how it will know to, like, make the trade-offs needed because it doesn't even know what the access patterns are. And it's not like these things can, like... Like, where is even the data to learn how to, like, build a system that understands, like, all of the operations, right? Like, today we rely on human beings to, like, manage clusters and to, like, you know, optimize databases, et cetera. And so can you maybe talk about how you think of, like, I get the writing the code, but, like, you have these deep operational systems. Like, how do you apply AI to that problem? And like, A, have we made any progress at all, or is this still an outstanding problem?
- JNJad Naous
The question I think is maybe about complexity, and, like, to what extent can you take the things that you've described that are operational-
- MCMartin Casado
Yeah
- JNJad Naous
... and, like, that happen in production-
- MCMartin Casado
Yeah
- JNJad Naous
... and convert them into so- something that an AI can reason about? And, uh-
- MCMartin Casado
Exa- exactly.
- JNJad Naous
Right?
- MCMartin Casado
Like, how, how, how can an AI reason about the system? Well, A, first off, like, to what, to what extent do you see that it writing the code that it can actually predict what's gonna happen? Like, how g- g- of a job does it do at that? And the second one is, like, okay, now how do we make it reason about that? So let's ask the first one. Like, like, if I, if I use your method, the, the Jad method of writing code and I build some big distributed system, like-
- JNJad Naous
How often does it actually-
- MCMartin Casado
How often [laughs]
- JNJad Naous
... get it from a performance correctness and stuff like that?
- MCMartin Casado
Yeah, how, how often is it, like, actually operationally, like, pretty sensible? And by the way, I, I work on this dis- this distributed Gaussian splat under, and I would say it's shit. Like, it's horrible. Like, because, like, you know, you've got millions of Gaussian splats that you've got to page in and out and stream, and, like, it just totally depends on, like, the running system. And so-
- JNJad Naous
Yeah
- 30:10 – 42:16
How Grepr catches incidents early
- MCMartin Casado
So have you found-- Or I, I-- Are you doing any of, like, the AI SRE stuff where you actually have a running system that has logs, and you feed it to an AI, and that kind of helps you, like, tweak it?
- JNJad Naous
Yes.
- MCMartin Casado
And how, how, how well does that work? [chuckles]
- JNJad Naous
So we're just starting-
- MCMartin Casado
'Cause an-anecdotally, uh, my experience with other people doing this has not been that great. Like, the reports I've been getting anecdotally have not been great.
- JNJad Naous
So a couple of things. The first one is we're doing that for both operational pieces and, uh, and for security. But the way that we're tackling it is not the way that other people are doing it from an AI SRE perspective. A lot of those systems ba-basically start from a lagging indicator, like, "Hey, this metric spiked," or, like, "This alert happened." Go back through a very complex system and try to determine why that happened.
- MCMartin Casado
Right.
- JNJad Naous
Like-
- MCMartin Casado
Yeah, sure
- JNJad Naous
... it's, uh, you know, that's why those don't work because, like, backtracking into a root cause is actually very problematic. And so instead, what we're doing is doing it the other way around, which is can we actually start by figuring out if there's something different in your environment that has changed? Like, can we do a large scale clustering, a large scale, uh, kind of figure out a needle in a haystack, something that is different? And then-
- MCMartin Casado
Oh, so you're assuming that it w-was working fine before, and now it's not, so something happened? But it, it could be just, like, a change in, like, normal traffic patterns, right?
- JNJad Naous
That's fine too. That's fine too. And so the, the, the thing that we're looking at is, like, when things go wrong, usually the system will tell you something went wrong somehow.
- MCMartin Casado
Yeah.
- JNJad Naous
Right? Um, if it's working fine and scaling as normal, nothing will happen. But sometimes you'll see some, like, message which says, "Error, connections ran out."
- MCMartin Casado
Yeah.
- JNJad Naous
Or, "Memory exceeded," or, "Resource buffer is emptied," or, like-
- MCMartin Casado
Yeah
- JNJad Naous
... some kind of indication, like, some kind of really early warning, like, something happened, and that's the one message that we wanna capture and then go from there and see is that an actual problem that we need to investigate and deal with.
- MCMartin Casado
So how does it do?
- JNJad Naous
So far, it's doing pretty well. I mean, we are running it internally in our systems. We run it, uh, we ran it in, uh, in a couple of e- our customers' environments. And [chuckles] in the cases where... There was a couple of cases where we ignored it-
- MCMartin Casado
Yeah
- JNJad Naous
... and those turned into production incidents.
- MCMartin Casado
Really? Oh, wow.
- JNJad Naous
[clears throat]
- MCMartin Casado
You know, it's kinda-- Uh, just, just from an intuitive perspective, like, I can understand how these models understand syntax because, again, like I said, they're trained on all of the co-code ever written, and it just turns out that covers a lot of useful cases. It's hard for me to actually even think of a bound on the actual operations of software, right? I mean, it feel like, you know, this is a ridic- uh, it's a ridiculous statement, but it's actually kinda true. It's like every time you add a bit to a program, the state space of, like, what it can do operationally increases by a factor of two. So you very quickly get to, like-
- JNJad Naous
[laughs]
- MCMartin Casado
... you know, like more, more, you know, states than atoms in the universe. And arguably, this is why, this is why, like, product approaches to operations have never really worked, right? Like, the chaos engineering stuff never really worked, and we've always kind of relied... You know, actually, the dirty secret for the longest time of running, like, a distributed cluster was just human beings managing it. And so I'm just wondering... And then, you know, and then in order to, like, even train an AI to do it, you have to have, like, enough data and enough configurations. And so, like, I'm just kind of curious, like, do you think we're at a s- at a s- a s- a spot where you could, like, use AI to run a distributed cluster or run one of these services, or do you think that's still on the come?
- JNJad Naous
I think that there's enough to be able to have it start that loop, right? Like, I mean, ultimately, like, I think that, uh, I agree with you. Like, there's infinite space, there's a lot of different things, but ultimately, humans also operate on runbooks. And, like, sure, there's a lot of things that happen in the, um, in production that you ultimately need some human to go and like, "Hey, this is too complex. There's, like, some state that's hidden that I know of because I talked to this other guy, like, at the water fountain the other day-
- MCMartin Casado
Yeah
- JNJad Naous
... and he told me about this thing he deployed." Like, there's a lot of stuff that potentially isn't visible in the state of the system.
- 42:16 – 53:33
Why security is the harder problem
- JNJad Naous
peg.
- MCMartin Casado
And that seems even harder to me, 'cause the operations one, the argument is The system can get in states, in, into broken states that are kind of, like, very rare, and there's a lot of those rare states. [chuckles]
- JNJad Naous
Yeah.
- MCMartin Casado
And so it's just not something that you could, you know, train a model on just because there's so many rare states it can get into that are, that are bad. And so maybe you can reduce that by doing the steps. The security one seems even harder because this isn't just, like, the natural order of rare states. You actually have an adversary that is trying to-
- JNJad Naous
Yes
- MCMartin Casado
... like intentionally find rare states-
- JNJad Naous
Yes
- MCMartin Casado
... and intentionally find stuff-
- JNJad Naous
Yes
- MCMartin Casado
... that you, you, you can't do. And so it may be the case that, like, code becomes solved, it takes a little while, you know, longer to do operations, and it takes a lot longer to-
- JNJad Naous
Yeah, but like also, like, think about... I, I mean, I don't know. To me, like, the whole world is sort of quantized to a certain extent.
- MCMartin Casado
The Planck length. [both laughing] Is that what the, is that what the physicists say? I don't know. I'm not a physicist.
- JNJad Naous
Planck's constant.
- MCMartin Casado
[laughs] I think there's like a actually, uh, I, I literally don't know what I'm talking about right now. But I literally actually think that like, like, like space-time is discretized into a Planck length. But anyway.
- JNJad Naous
But, but yes, I, I mean, I agree. Like, to a certain extent, what you just said, but for operations and systems in general-
- MCMartin Casado
Yeah
- JNJad Naous
... like, there is a discretization of, like, the possible states and the paths through them that are possible.
- MCMartin Casado
Yeah.
- JNJad Naous
Right?
- MCMartin Casado
I, I just think when you're dealing with-
- JNJad Naous
There might be-
- MCMartin Casado
[laughs] There's gonna be a lot
- JNJad Naous
... there might be a ton of them.
- MCMartin Casado
Yeah. [laughs]
- JNJad Naous
There might be a ton of them.
- MCMartin Casado
I'm gonna tell you a story. Remember Sundar who, um, was in our, in our grad school group?
- JNJad Naous
Iyer?
- MCMartin Casado
Sundar.
- JNJad Naous
Sundar Iyer?
- MCMartin Casado
Iyer, yeah, yeah, yeah.
- 53:33 – 58:42
What comes after code
- JNJad Naous
code?
- MCMartin Casado
I mean, my, my sense is, is, you know, AI is gonna continue to take off in a wildfire for a constrained domain, where the, where the solution space is in a constrained domain, right? Where the distribution of the solution space is the same as the distribution of the training set. The distribution of the syntax for code is the same as the distribution of the training set. The distribution of the generated images is the same as the distribution of the training set. In those, you know, areas, I think it's gonna continue to take off as a wildfire. The domains where, like, the distribution of the solution is the natural world, I think it's gonna, you know, have s- you know, it's gonna be much harder, take longer, and I think the economic equivalency is gonna be much more difficult. Self-driving is a great example. I remember when Sebastian Thrun won the DARPA Grand Challenge when I was at Stanford. Remember that? And we were like, "Hooray!"
- JNJad Naous
[laughs]
- MCMartin Casado
You know, self-driving cars is around the corner.
- JNJad Naous
[laughs]
- MCMartin Casado
Seriously.
- JNJad Naous
Yeah.
- MCMartin Casado
And it took 20 years and $170 billion of investment to get to worse unit economics than Uber, right? So that's, you know... So I think that, like, solving, like- Like, solving real problems where you have to deal with the, the distribution of the real universe is just much, much harder. And so I view as AI as this general tool. It's a very effective tool. It's one where, like, you have, you know, a distribution of data, and then hopefully you can apply that to [chuckles] like, you know, some, uh, problem set.
- JNJad Naous
Real world, yeah.
- MCMartin Casado
And we're gonna apply it to a bunch of stuff, and I think economic equilibrium is gonna determine what works and what doesn't work, because I think it's a very, very general tool. It's like a universal function approximator. And wherever that nets out, I mean, again, so-
- JNJad Naous
You know, one of the things that I said earlier was, like, how AI was training humans to, like, use it.
- MCMartin Casado
Yeah.
- JNJad Naous
And I very often think about, like, you know, some of the problems that seem intractable was be- is because the world is kind of that way.
- MCMartin Casado
Yeah, yeah.
- JNJad Naous
And then we're trying to fit AI or the tools-
- MCMartin Casado
Yeah
- JNJad Naous
... that we're building to manage the world as it is.
- MCMartin Casado
Yeah.
- JNJad Naous
But in some situations, we actually tend to change the world to fit to what we can actually do from a-
- MCMartin Casado
No, 100%, of course
- JNJad Naous
... from a, from a digital-
- MCMartin Casado
I think this is the story of robotics.
- JNJad Naous
Yeah, exactly.
- MCMartin Casado
This is literally, this is literally the story of robotics, yeah.
- JNJad Naous
100%. The, the automated data warehouse. And so the, the question, like, that always comes to my mind, like, when we were looking at these data warehouses and whatever.
- MCMartin Casado
Yeah.
- JNJad Naous
And, like, the only one that worked was the one where it was like, "You know what? Just automate the whole fucking thing."
- MCMartin Casado
Yeah, yeah, yeah, yeah.
- JNJad Naous
Um-
- MCMartin Casado
Yeah, yeah
Episode duration: 58:57
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode blkIGLlZePg
