a16zAaron Levie and Steven Sinofsky on the AI-Worker Future
EVERY SPOKEN WORD
65 min read · 13,270 words- 0:00 – 0:36
Introduction: The Evolution of AI Agents
- ETErik Torenberg
We thought that we were looking at the form factor of AI, which is you're talking back and forth to something. The real ultimate end state of AI, and thus AI agents, is these are autonomous things that run in the background on your behalf and executing real work for you. The more work that it's doing without you having to intervene, the more agentic it's becoming.
- MCMartin Casado
Somehow it produces output that it feeds back into itself.
- SSSteven Sinofsky
It's literally just the ampersand in Linux, which is-
- MCMartin Casado
[laughs]
- SSSteven Sinofsky
... it's a background task.
- ETErik Torenberg
[laughs]
- MCMartin Casado
Okay.
- SSSteven Sinofsky
And it's like the worst assistant in the world. And agentification is just hiring a lot of these really bad interns.
- ETErik Torenberg
[laughs]
- 0:36 – 1:39
Defining Agency and Autonomy
- ETErik Torenberg
I thought we'd- I'd start this wide-ranging podcast by asking the very simple but very provocative question, what is an agent?
- MCMartin Casado
Oh, boy. To who?
- ETErik Torenberg
Steven.
- MCMartin Casado
Okay, Steven. That's yeah-
- SSSteven Sinofsky
Oh, to me?
- MCMartin Casado
Yeah, exactly. Steven.
- SSSteven Sinofsky
Oh, oh, I go first. So, so I, I actually have a very old person view of what an agent is, which is it's literally just the ampersand in Linux.
- MCMartin Casado
[laughs]
- SSSteven Sinofsky
Which is, it's a background task.
- ETErik Torenberg
[laughs]
- MCMartin Casado
Okay.
- SSSteven Sinofsky
Because, like, you type something into o3, and then it's like, "Hey, t-t-t-t-t-t- I'm trying this out. Oh wait, I need a password. Can't do that." And it's, like, the worst assistant in the world, and really it's just 'cause they need to entertain you while it's taking a long time-
- ETErik Torenberg
[laughs]
- SSSteven Sinofsky
... to answer your prompt. And so that's my old person view of what an a- And agentification is just hiring a lot of these really bad interns.
- ETErik Torenberg
[laughs]
- MCMartin Casado
[laughs]
- ETErik Torenberg
The intern, they're getting better. The-
- SSSteven Sinofsky
They are getting better.
- ETErik Torenberg
The intern, they're getting better.
- SSSteven Sinofsky
But they still don't remember if I have a password to nature, you know. Like, it's just-
- ETErik Torenberg
Is it possible you guys just had bad interns in, like, the '80s and '90s?
- SSSteven Sinofsky
It, we, we had-
- ETErik Torenberg
Okay
- SSSteven Sinofsky
... we had terrible interns.
- ETErik Torenberg
Okay, okay. I have, like, a very high esteem for interns, so.
- SSSteven Sinofsky
[laughs]
- ETErik Torenberg
Like-
- SSSteven Sinofsky
But now a real answer. [laughs]
- ETErik Torenberg
No, no,
- 1:39 – 4:27
Long-Running Agents and Feedback Loops
- ETErik Torenberg
I mean, uh, I, I think, I, I think collectively we're seeing what, what these are becoming. So if you think about two years ago, the, you know, post-ChatGPT moment, we, we thought that we were looking at the form factor of AI, which is you're talking back and forth to something. And I think to Steven's point, the real, you know, ultimate end state of AI, and thus AI agents, is these are, these are autonomous, you know, uh, uh, things that run in the background on your behalf and executing real work for you. And you're ideally in a, an ideal world interacting with them actually relatively little, m- uh, relative to the amount of value that they're creating. And so, so there's some kind of, you know, metric where the more work that it's doing without you having to intervene, the more agentic it's becoming and, and I think that's, that's sort of the paradigm that we're seeing.
- MCMartin Casado
Yeah, the only addition I'd have in addition to long-running, which I agree, is that somehow it produces output that it feeds back into itself-
- ETErik Torenberg
Mm
- MCMartin Casado
... as input, which you can actually do long-running inference. Like, you can make a video that's really long-running-
- SSSteven Sinofsky
Right
- MCMartin Casado
... but it's just basically a single shot video, and you just throw more compute at it. I, I think there's, like, technical limitations, um, you know, if you start feeding the input back in, 'cause we're not quite sure how to contain that, too. And so, you know, I think you can do a, I think you can measure things based on how long they run, and you could also measure it by how many times it's actually taken its own guidance-
- SSSteven Sinofsky
Mm
- MCMartin Casado
... which would be kind of more of an agency.
- SSSteven Sinofsky
Yeah, 'cause I think-- I do think it's important that in this transition... Look, we are, what Aaron described is where we're gonna be. It, it-
- ETErik Torenberg
Yeah
- SSSteven Sinofsky
... j- it's just that what are the interesting steps that happen along the way? 'Cause we are gonna need it, for the time being, it to stop and say, "Am I heading in the right direction or not?"
- MCMartin Casado
Exactly right. Like it-
- SSSteven Sinofsky
'Cause you, you really, putting aside all the horror stories about, you know, taking, um, action without consent and using accounts and data, whatever, there is this thing where, like, you just don't wanna waste your time-
- ETErik Torenberg
Right
- SSSteven Sinofsky
... on the clock-
- ETErik Torenberg
Right. Yeah
- SSSteven Sinofsky
... while it's churning away way off in the wrong direction.
- MCMartin Casado
Yeah, so the question is to what extent do they have their own agency, which to me means they've spit something out, and they've kind of consumed it back up again, and it's still a sensible thing. Which, by the way, as you start thinking of these things in distribution, it's actually a very difficult thing to do because it doesn't know if it's gonna be spitting something out that's still in distribution when it brings it back in. Like-
- SSSteven Sinofsky
Mm
- MCMartin Casado
... they don't have that self-reflection. So I, I think there's actually a very kind of technical question here of to what extent we can make these things have independent agency. But we can make them long run pretty easily.
- SSSteven Sinofsky
Yeah, yeah. We're good at the long run.
- MCMartin Casado
The long-running thing.
- SSSteven Sinofsky
What, what you get back is, yeah.
- MCMartin Casado
Yeah. [laughs]
- ETErik Torenberg
Yeah, I mean, I think the, um, uh, the, the interesting thing is how the ecosystem is sort of solving, um, or, or mitigating then, you know, the, the issues. Like, you're, you're seeing sort of this logical division of the agents. So they might be long-running, but they're not actually trying to do everything.
- MCMartin Casado
Yeah.
- ETErik Torenberg
And so-
- MCMartin Casado
They're very smart
- ETErik Torenberg
... so the more that you subdivide the tasks out, then actually the more that they, they can go pretty far on, on a single task without, without getting kinda totally lost on what they're, what they're working
- 4:27 – 6:04
Specialization and Task Division in AI
- ETErik Torenberg
on.
- SSSteven Sinofsky
Well, Unix is gonna prove to be right.
- ETErik Torenberg
[laughs]
- SSSteven Sinofsky
Which is, like, you're gonna ha- you're gonna wanna break things up into much smaller granularity and tools. And I think to other points that you've made o- on X, like, you're gonna wanna divide things up so that it's, like, an expert in this thing.
- ETErik Torenberg
Yeah.
- SSSteven Sinofsky
And, and then it might be a different, let's just say body of code, where you go and asks, like, ask, you know, "Are you good at this thing?"
- ETErik Torenberg
Yep.
- SSSteven Sinofsky
"Let me get your answer on, on this part of the problem."
- ETErik Torenberg
Yeah. Um, it's, it's kind of interesting. I, I don't know, um, how much you've plotted this, but, like, the conversation on AGI ha- has sort of evolved, you know, very clearly-
- MCMartin Casado
Oh, yeah
- ETErik Torenberg
... in the past, like, six months. And, and I think that the consensus was, may- maybe not even consensus, what, what some of the view was, let's say two years ago, was is this sort of monolithic system that's just super intelligent, and it solves a- you know, all things. And now if you kinda fast-forward to today, and let's say whatever s- we, we agree kinda state-of-the-art is, it's sort of looking like that's probably not gonna work, and, um, for, for a variety of reasons, at least in, in today's architecture. So then what do you have is maybe a system of many agents, and those agents have to become very, very deep experts in a particular set of tasks, and then somehow you're orchestrating those agents together, and then, you know, now you have two different types of problems. One has to go deep, the other has to be really good at orchestration. Um, and, and that maybe is, is how you end up solving, you know, some of these, some of these issues over the long run.
- MCMartin Casado
I, I just think it's very difficult to think cleanly about this. Like, I've still yet to see a system where you, you, they perform very well, and you don't draw a circle that doesn't have a human being in it somewhere.
- SSSteven Sinofsky
Mm.
- ETErik Torenberg
Oh, yeah. [laughs]
- MCMartin Casado
And so in a sense, like, the G, like, often seems to be coming from... Like, the general seems to be-
- ETErik Torenberg
Yeah.
- MCMartin Casado
So, like, I just...
- 6:04 – 9:10
Anthropomorphizing AI and Economic Impact
- MCMartin Casado
Listen, these things are tremendously good at in-increasing productivity of humans. At some point, maybe they'll increase productivity without humans, but until then, it's just very hard-
- SSSteven Sinofsky
Yeah
- MCMartin Casado
... for me to actually just talk cleanly.
- SSSteven Sinofsky
Well, and it's just, it's, it, it's so important for peopleTo get past sort of the anthropomorphization-
- MCMartin Casado
Yeah, totally
- SSSteven Sinofsky
... of AI, because that's what's holding everybody back. Like AGI is about, about robot-
- MCMartin Casado
Right
- SSSteven Sinofsky
... fantasy land, and it, and that leads to all the nonsense about destroying jobs-
- MCMartin Casado
Yeah, totally
- SSSteven Sinofsky
... and blah, blah, blah. And none of that is helpful because it, it, you have to then you dig yourself out of that hole to just explain, wow-
- MCMartin Casado
Yeah
- SSSteven Sinofsky
... you know, it's really, really good at writing a case study.
- MCMartin Casado
Right. Right, right.
- SSSteven Sinofsky
Like, which, like it writes a better case study than all the people that work for it, but it doesn't know who to write it about.
- MCMartin Casado
Right.
- SSSteven Sinofsky
It doesn't know what necessarily you wanna emphasize. It doesn't know what's, what the budget is, what's needed-
- MCMartin Casado
Well, yeah
- SSSteven Sinofsky
... how many words.
- MCMartin Casado
Right. But and it also turns out like AGI just does an awful lot of work [laughs] in other words.
- SSSteven Sinofsky
Yeah, yeah.
- MCMartin Casado
So for example, someone asked me recently, they say, "Well, um, you know, are you worried that like if we have, uh, AGI, then you'll no longer be investing in software companies?" I, I'm like, "Well, I mean, you're AGI." [laughing] Like, right, you are. I'm still investing in software companies, right?
- SSSteven Sinofsky
Right, right.
- MCMartin Casado
And so like just because you're AGI says nothing about economic equilibrium-
- SSSteven Sinofsky
Right
- MCMartin Casado
... or economic feasibility, et cetera. So like just the term AGI does basically infinite work for every kind of fear we have and maybe every hope that we have. And the moment we tie it down to like not only it solves a class of problems, but the economics pencil out yes or no, we can actually have a more sensible discussion, which I actually I think is finally entering the discourse.
- SSSteven Sinofsky
Yeah.
- MCMartin Casado
I think we're actually talking-
- SSSteven Sinofsky
Yeah
- MCMartin Casado
... a lot more sensibly now than we were a year ago.
- ETErik Torenberg
And so when you hear, when people say things or the, the AI 2027 paper, when they talk about sort of automated research or recursive self-improvement, does that feel like fiction or fa-fantasy, or does it feel like... Or is it thinking that even with those things we're, you know, sort of nowhere near, um, you know, peak software and there would just be unli- unlimited, uh, sort of demand?
- 9:10 – 11:31
Predictions, Progress, and Platform Shifts
- SSSteven Sinofsky
Now, you could do science fiction-
- MCMartin Casado
Yeah
- SSSteven Sinofsky
... and you could say in the future when we all have our personal AI with all this other stuff, and then that's great, but then you say it's gonna happen in 2029-
- MCMartin Casado
Yes
- SSSteven Sinofsky
... you're an idiot.
- MCMartin Casado
Yes.
- SSSteven Sinofsky
A- and so-
- MCMartin Casado
That, that sounds totally correct, right? Because basically, uh, three years ago you would not have been able to conceive of Cloud Code, so, or Cursor or, or, you know, name your, your background agent writing code. So it's like what is the point of having some date at which you're, you're naming something? And, um, and so we've actually seen probably vastly more progress in the past just two years of, of actual applied AI than we would've thought, and yet does it matter that one or two of the predictions didn't play out? Like, no. Um, so, so I think it's probably more interesting to think about like where is the technology from more of a classic Moore's law standpoint and like how much compute do we have, how much data are we working through, um, how powerful are these models?
- SSSteven Sinofsky
I mean, just let me ask you, like as semi-old, like the [laughing] You know, like-
- MCMartin Casado
Guilty
- SSSteven Sinofsky
... well, I mean, like, like nobody after AI collapsed and machine translation and m- and machine vision failed-
- MCMartin Casado
Yeah
- SSSteven Sinofsky
... there, you couldn't find anybody who thought that those would become solved problems.
- MCMartin Casado
Yeah, totally.
- ETErik Torenberg
Hmm.
- SSSteven Sinofsky
Or like, or after neural nets-
- MCMartin Casado
Yeah
- SSSteven Sinofsky
... imploded and like literally you were teaching-
- MCMartin Casado
Or expert, or expert systems or whatever, yeah
- SSSteven Sinofsky
... or expert systems, but you were teaching and like, like if you tried to teach neural nets-
- MCMartin Casado
[laughs]
- ETErik Torenberg
Yeah, yeah
- SSSteven Sinofsky
... like the students would rebel-
- MCMartin Casado
Yeah, yeah, yeah
- SSSteven Sinofsky
... because you were wasting everybody's time. You know, in, in, like in 1989, like Hinton couldn't get funded trying to-
- MCMartin Casado
Oh, yeah, yeah
- SSSteven Sinofsky
... to do neural networks.
- MCMartin Casado
Neural networks, yeah, yeah.
- SSSteven Sinofsky
I, I took, like I, grad school was this three volume history of artificial intelligence thing. Neural nets was like eight pages.
- MCMartin Casado
I, you know, ironically, I remember when ML was the cool thing-
- 11:31 – 16:16
Recursive Self-Improvement and Technical Challenges
- MCMartin Casado
anything, right? So let's take recursive self-improvement. This is one of my favorite ones. So the theory of recursive self-improvement is you have a graph or you have a box which is the thing, and then there's an arrow that goes back to the box which says improve. And then of course you look at that and you're like [laughing]
- SSSteven Sinofsky
Works.
- MCMartin Casado
Right. So I guess, you know, like from an intuitive lay perspective, every time you have a box with an arrow back in it, you're like, "Okay, we're, we're done," right? But like if you know anything about nonlinear control theory, answering that question is one of the most difficult question that we know in all of technical sciences, right? Like does it converge? Does it diverge? Like does it asymptote? Right? So for example, you could recursively self-improve-If you're doing basic search, but you asymptote, right?
- SSSteven Sinofsky
Right.
- MCMartin Casado
And so, like, saying recursive self-improvement from, like, a deeply technical perspective says almost nothing.
- SSSteven Sinofsky
Mm-hmm.
- MCMartin Casado
It says... But, but, but unfortunately, because we tend to anthropomorphize AI, we say recursive self-improvement, and all of a sudden we're like, "And then it, like, overcomes energy boundaries-
- SSSteven Sinofsky
[laughs]
- MCMartin Casado
... and human intelligence," [laughs] and then-
- SSSteven Sinofsky
Well, that's how it goes from being a toddler to being, like, an eight-year-old. It just because it, it figured out how to learn.
- MCMartin Casado
It recursively self-improves, right?
- SSSteven Sinofsky
Yeah, yeah, yeah [laughs] .
- MCMartin Casado
And so, I mean, the reality is, like, nonlinear control systems, which are feedback loops that are adaptive, we don't even have the math for, for, for a relatively simple system to understand what happens. You have to actually know the distributions that come out and go into them. And so these things are gonna improve. They're gonna continue to improve. Maybe they'll improve themselves, but just because they do improve themselves doesn't mean they, they can continue to do it, and this is kinda part of this entire journey is we're learning about these systems.
- SSSteven Sinofsky
Right.
- MCMartin Casado
Again, the good news is I think we're talking a lot more sensibly now than we were a year ago, and hopefully that will continue. I don't... Hopefully, [laughs] hopefully the discourse can recursively self-improve so we're just more sensible.
- SSSteven Sinofsky
Well, the good news is that's involving humans.
- MCMartin Casado
[laughs]
- SSSteven Sinofsky
So we don't actually have to worry.
- MCMartin Casado
[laughs] Oh, that's right.
- SSSteven Sinofsky
But I, I think that, I mean, you, you must be seeing this even with, with customers. I mean, like, take the conversation about, like, hallucinations-
- MCMartin Casado
Yeah
- SSSteven Sinofsky
... and things like that, how, how dramatically that's altered-
- MCMartin Casado
Yeah
- SSSteven Sinofsky
... in just the past two years, say.
- ETErik Torenberg
Yeah. In, in, on two dimensions, actually. So on one dimension, the, the problem of hallucinations has improved, so the, as the models get better, as our understanding of how do you, you know, whether it's RAG or whatever, what, you know, even the, even the, the problem of, uh, of, of actually the efficacy of the context window has, has improved. So you have the technical improvements, um, you know, kind of across the stack, and equally, you have a kind of a cultural understanding to some degree within the enterprise, uh, as to, like, okay, actually, no, these are, these are non-deterministic systems. They're probabilistic. So, so you're starting to see almost a culture shift, which is, okay, uh, you can, you can actually, uh, implement AI in, in es- essentially more and more critical use cases because the employees that are using those systems understand that they do actually have to do the work to verify it. And then the only question is, is what is that ratio of, of time it took to verify versus if I had done it myself and how much efficiency gained for whatever that workflow is? Um, but we are, we're going from probably, like, two and a half years ago where there was, you know, this instant excitement as, as to, "Oh my God, this is going to be the greatest thing of all time," to a reality check within three to six months 'cause everybody was like, "Hallucination is gonna be the, the massive, you know, kinda problem," to now a couple years later after that, which is like, okay, like, we're, we're seeing the hallucination rates shrink. We're seeing the quality of the outputs increase, and we understand that you do have to go and review the work that these AI, you know, agents are doing. And that, that takes on a different form debas- depending on the use case. So in the form of coding, that means just, like, you just have to go review the code in the, in the, uh-
- SSSteven Sinofsky
Which you had to do anyway.
- MCMartin Casado
Which-
- ETErik Torenberg
Yeah.
- SSSteven Sinofsky
People seem to be forgetting.
- ETErik Torenberg
You, you had to do it anyway-
- 16:16 – 22:14
The Role of Experts and Tool Adoption
- MCMartin Casado
better result.
- SSSteven Sinofsky
Well, that, I, I think that this is just an incredibly important point that you're making, and it, it really gets to the heart of what it means to use a tool. Like, you know, you put me-
- ETErik Torenberg
Yeah
- SSSteven Sinofsky
... in front of, like, a 12-inch chop saw and say-
- MCMartin Casado
[laughs]
- SSSteven Sinofsky
... like, "Go fix the fence," really, really bad idea.
- MCMartin Casado
[laughs]
- SSSteven Sinofsky
I mean, I could go buy one.
- MCMartin Casado
Still bad idea.
- SSSteven Sinofsky
I lo- I could cruise the Home Depot and-
- MCMartin Casado
There's a, there's a reason the tools counter [laughs]
- SSSteven Sinofsky
... and, and I'm like, "Ooh, dang, man. I don't have a DeWalt."
- MCMartin Casado
[laughs]
- SSSteven Sinofsky
And I could buy it, but it's really not a, a particularly good idea.
- MCMartin Casado
Right.
- SSSteven Sinofsky
And, and I think that how these platform shifts happen-
- MCMartin Casado
Yes
- SSSteven Sinofsky
... and why there's so much excitement-
- MCMartin Casado
[laughs]
- SSSteven Sinofsky
... over coding is that, well, the best way for a platform shift to take hold is it's the, the experts that are... The, the closest you have to an expert in the new platform is who becomes the most enthusiastic-
- MCMartin Casado
Yeah
- SSSteven Sinofsky
... and the biggest users overall. Like, I, I've been practicing yoga ov- over at, um, the Cubberley Community Center in Palo Alto 'cause the studio's closed for remodel. But, but what's neat is that was, like, the OG place for computer clubs.
- MCMartin Casado
Oh, nice.
- SSSteven Sinofsky
Like, in the early 1990s and the late '80s, like, if you ever wanted to meet the computer club... And you would go, and, like, this is, like, halt and catch fire. Like, you-
- MCMartin Casado
Oh, nice
- SSSteven Sinofsky
... and it's, like, like, a bunch of people with soldering irons and shit, and, like, they're-
- MCMartin Casado
Yeah.
- SSSteven Sinofsky
That's who... And, and, you know, when it, when it didn't work, when something was broken, that wasn't like, "Oh man, these things are terrible. I'm wasting all my time."
- ETErik Torenberg
Right.
- MCMartin Casado
No, yeah.
- 22:14 – 45:55
Changing Workflows: Agents Reshaping Work Patterns
- ETErik Torenberg
work change because of the tool?
- SSSteven Sinofsky
Ah.
- ETErik Torenberg
Versus-
- SSSteven Sinofsky
Yeah, yeah
- ETErik Torenberg
... the tool sort of adapted to the style of work. And so what I'm starting, and we're like only in day one of this, but what I'm starting to see kind of some, some patterns emerge, which is we thought agents would go and learn how we work and then automate that. And then the quest- and so basically agents conform to how we work. The question is when is the moment when we conform to how agents are best used?
- SSSteven Sinofsky
Yep.
- ETErik Torenberg
And you're, you're seeing this in a couple areas, so you're seeing this in engineering to start with, which is like people are saying, "Okay, I'm gonna have agents and then sub-agents for parts of the code base, and then I'm gonna give them kinda read-me files that the agents read, and then, and then I'm gonna actually optimize my code base for the agent as opposed to the other way around in other forms of knowledge work." So within how we use Box, um, with, with our AI product, like you're starting to see people like basically tell the agent like its complete, you know, job, and the, the workflow is now starting to be almost like the agent is almost dictating the workflow in the future as opposed to it's just mapping to the existing workflow. So I don't know like what the history is on this of like when does, when does the work pattern itself shift because of what the technology is capable of, but I think, I think probably where this goes has to be some version of that, which is, which is it's not gonna just be the agents just plop into how we currently do our work and then, and then just automate everything. I do think you start to change what we actu- what the work is itself, and then, and then agents actually go in and accelerate that.
- SSSteven Sinofsky
Well, as important as that is, it's actually more important.
- ETErik Torenberg
Okay.
- SSSteven Sinofsky
Like because what, what happens is where it, there, there's, to reuse the, a word in a different, this anthropomorphization of work, what happens in is that the first tools actually anthropomorphize the work.
- ETErik Torenberg
Uh-huh.
- SSSteven Sinofsky
And so like if you go back, this is every single evolution of computing. I mean, like how long did it take for Steve Jobs to get rid of the number buttons on a smartphone?
- ETErik Torenberg
Right, right, right, right. [laughs]
- SSSteven Sinofsky
Like, like they, they still had number buttons. Or like you look at, at cars, and until Elon got rid of all the controls-
- ETErik Torenberg
Right. Yeah
- SSSteven Sinofsky
... everybody kept all of the controls. I don't wanna get in that fight. But, but like the, what happened with every technology shift is, you know, if you, if you were to look at what accounting software looked like in the '60s before IBM said s-stop. W-we all use double entry, but we need to have people skilled in how computers can do the accounting-
- ETErik Torenberg
Uh-huh
- SSSteven Sinofsky
... not how people can, because we're never gonna figure out how to close the books-
- ETErik Torenberg
Right
- SSSteven Sinofsky
... if we have to automate this whole r-room of people with green eye shades-
- ETErik Torenberg
Right
- SSSteven Sinofsky
... that have a manual process based on how far apart the desks were.
- ETErik Torenberg
Right.
- SSSteven Sinofsky
And, and everything that happened with the rise of, of PCs and personal productivity started off, and I always use this example because I've watched it happen like five times now, which is the, the first PCs that did word processing, the biggest request was how do I fill in like expense reports?And so the whole, this whole world grew up of tractor-fed paper that was pre-printed with the expense report. Right. And so then software, we wrote all of this code, like are you using an Avery 942- [laughs] ... expense report or is it a New England Business Systems A397? And, and like, you know, and then you had like these adjustments in the print dialogue, like .208 inches, and you, you moved little things around, and then you would print out like- [laughs] ... ate dinner, $22, and that was all you printed. Right. And then someone said, "You know, we could use the computer- [laughs] ... to actually print the whole thing." Right. [laughs] And then, like fast-forward, and finally Concur said, "You know, why just take a picture? Why not just- Right ... take a picture of the receipt, and then we could do all of it?" And so then the whole thing gets inverted, and, and every single business process ended up being like that. And, and then there are things that really, really do change the tools. Right. Like when email came along, you know, it used to be to prepare an agenda for a meeting, somebody would open up Word and type in all the things, and then print it out, and everybody would show up to the meeting with this very well format. And now, and then like- Yeah, new case ... email came out, and that whole use case for Word- Right ... just evaporated. Yeah. And, and then an email agenda became no formatting, nothing, just like, "Here are the eight things we're gonna talk about." Yeah. And you show up, and everybody's like, "Did you get the agenda?" You know what, what's interesting about the AI one is it's kind of t- it's like we're seeing the same thing, but vis-a-vis AI. So nobody really predicted the generative stuff, and we've had AI for a very long time. So we had chatbots, we've had, you know... And so you had these kind of like AI shape holes in the enterprise for a long time, and a lot of the mistakes that we see today is people are taking the generative stuff and trying to kind of cram it into the old models- Mm-hmm. Yeah, yeah ... when it's really a new behavior that's emerging, that's very m- much more in... Like, it used to be you'd se- centrally sell, you know, AI to some platform team, and then they would kind of try to get the NLP thing to work or the voice to work for like talking to people on the phone for support, and it was this kind of very central. A lot of the adoption that we see is like much more individual, for example. And so I just think that there is a- Right ... bit of a mismatch, as we're seeing now, that is getting ironed out, too. Well, and, and so I, I think the question is, is, yeah, are we in the phase where we're trying to graft the agents and, and work in basically the, what we've been doing for 30, 40 years- Yeah ... of software, and is this gonna be actually like a, like a, like the first real step function shift we've seen in what the workflow itself should look like? Oh, we- Yeah ... but we are. I mean, like you, if you, you know, remember, people like p- I, I tried to jam the internet into Office. Right. And it, and, and- It was fun to watch ... no, but I mean, you were, you were not watching. Yeah. [laughs] But, but, but like, but, but everybody around was trying to jam the internet- Right ... into their product because that's the only way you could envision it. Right. And it, it- Right ... didn't really... Like, you, you were like, "Well, where else would the internet go?" [laughs] Like, there's no word processor on the internet. Right. Right. Like, there's no spreadsheet on the internet. Yes. And, and then the other people would be like, "Well, let me just try to implement Excel using these seven HTML tags with no script." Uh-huh. That turned out to not be a really good idea either. [laughs] The best was like, "Let's do PowerPoint." Well, how do you do it? You give them five edit controls, tell them their bullet points, and then we'll generate a GIF on the back end and send it back to you as the slide. Yeah. [laughs] Okay, that, that, that was not... And so there was that whole, like that- I think actually maybe the, the main point is just the durability of Office. But- It transcends all- It does ... all disruptions. [laughs] I, I like to think it pretty much rises above everything. Yeah, exactly. [laughs] But, but the thing is, is that that's where we are now- Yeah ... is everybody... A- and you know, like e- Do, but do you think then- No ... I mean, j- just to dig a little bit, so do you think this is similar to the internet in that it's a consumption layer change? 'Cause I always viewed the internet as very much a consumption layer change. Like, I go to a, you know, instead of going to my computer, I go to the internet. But otherwise, things kind of are the same where AI's got this weird quirk which for the first time I can recall, programs are abdicating logic to a third party. Like, we've always abdicated resources, right? Yeah, yeah. Like, so it'd be like, "Okay, I'll use your disks or whatever," but like I'm writing the logic. But this time it feels like we're changing the consumption layer. So like, you know, when my son, you know, talks to an AI character and, you know, he's not going to wellsfargo.com, he's going to an AI character, and so like that's changing kind of how we're interacting with the computer. But also these programs are no longer kind of written by a human- Yeah ... in the same way. So I feel like the change is maybe a bit more sophisticated. Oh, I think that, but this is the, this is why it's a platform shift- Right ... and not just an application shift. Right. Yeah. Like where, where each l- each platform shift changes the abstraction layer with which you interact with computing, but what that also does is it changes th- what you write the programs to. Yeah. Do you, do you remember ever abdicating logic to- Oh, y- here's a great, like here's an example of how disruptive this can be. The, the first word processors in, in, in the DOS era, the character mode era, they all implemented their own print drivers and clipboard. So if you were Lotus and you wanted to put a chart into a memo, you, you, you couldn't 'cause you didn't have a word pro- you didn't sell a word processor, so you actually made a separate program- Uh-huh ... to make something that the leading word processor could consume. Oh. And if you were WordPerfect, your ads said, "We support 1,700 printers." Huh. Like, and you won reviews because you had 1,700 and Microsoft had 1,200. Oh, that's so... [laughs] And so then along comes- That's a great one, actually ... and then so Windows comes along, and, and if you were s- and if you were trying to enter the word processing business, step one, I need to hire a team of 17 people to build device drivers for Epson and Okidata and Canon printers 'cause you can't get them anywhere. Microsoft came along and for Windows built print drivers and a clipboard, and all of a sudden, and also Macintosh did it. Yeah, yeah. All of a sudden, you, there was a way that two applications that had no a priori knowledge of each other- Good deal. Yeah ... but of course, if you were WordPerfect or Lotus, that's a disa- Yeah, yeah. Of course, yeah ... you got creamed by that- Right ... because your ability to control- Right ... your ev- right. And so, and what happened was a bunch of developers were like, "Wow, this is cool," 'cause now I'm just by my... When we did C++ for Windows-Like we were like where w- the demo, in fact, at that Cubberley Community Center, I would go and I would show brand new Windows programmers in 1990 like, "Hey, you don't have to write print drivers and use the clipboard." And like literally standing ovation of, you know-
- SSSteven Sinofsky
Yeah [laughs]
- SSSteven Sinofsky
... all 10 people at the thing.
- SSSteven Sinofsky
[laughs]
- SSSteven Sinofsky
And, and but, but they were like more than happy-
- SSSteven Sinofsky
Yeah
- SSSteven Sinofsky
... to let data interchange between product, 'cause they were like, "That's nothing but opportunity for me."
- 45:55 – 48:47
Division of Labor, Specialization, and New Roles
- ETErik Torenberg
And then is there just a new set of roles? Like, like clearly there's a role in a bunch of organizations emerging, um, which is like, no, I'm just like, my role is like I'm the AI productivity person, and like I just like have a way of, of, you know, creating all new forms of productivity in the organization with AI. So like clearly we'll have a bunch of new roles, but is our current division of labor gonna also collapse in some interesting ways because of AI?
- SSSteven Sinofsky
Well, I, I think that, like if you actually s-stick with the medical example, they're, we're just gonna wake up and there's gonna be way more people with way more specialties.
- ETErik Torenberg
Right.
- SSSteven Sinofsky
And, and AI will have created-
- ETErik Torenberg
So you think it adds
- SSSteven Sinofsky
... more jobs.
- ETErik Torenberg
Yeah.
- SSSteven Sinofsky
And in the interim, the-
- ETErik Torenberg
Do you think AI causes more specialization over time?
- SSSteven Sinofsky
Absolutely.
- ETErik Torenberg
Yeah.
- SSSteven Sinofsky
'Cause everyone's gonna, every human is gonna be way better-
- ETErik Torenberg
Right
- SSSteven Sinofsky
... and, and more knowledgeable about. And I think this is a thing that, that has, has really happened with computing that people forget. Like, there used to just be like this morass of marketing-
- ETErik Torenberg
Right
- SSSteven Sinofsky
... and R&D.
- ETErik Torenberg
Right.
- SSSteven Sinofsky
And all of a sudden, like just, just ... And there used to just be coding, and then there was coding and testing and design and product management and program management and, you know, usability and research and all of these specialties. And all of those had their own tools.
- ETErik Torenberg
Right.
- SSSteven Sinofsky
Go to a construction site. I, I remember growing up, these, our neighbors built a house. We lived in an apartment, and they built a house, and there was Clem, the carpenter.
- MCMartin Casado
Yeah.
- SSSteven Sinofsky
And you built a house with a guy named Clem, who used all the tools and everything. And now, like you build a house, and it's like this 20-person list of sub-subcontractors, all who have whole companies that do nothing but like put in pavers, you know? And, and that's what it's gonna be.
- MCMartin Casado
I mean, there, there's been a, there's been a long disaggregation in the history of IT, right? Like everything in the same sheet metal, then, you know, disaggregate the OS and the hardware, then you disaggregate the apps.
- SSSteven Sinofsky
Right.
- MCMartin Casado
Um, and then it was kind of interesting, like in the last 15 years, we saw the app, and like independent functions got disaggregated, right? It's like almost everything became, like, like an API would become a company, right? You'd have like-
- SSSteven Sinofsky
Yeah
- MCMartin Casado
... Twilio's like Auth became a company, like PubSub became a company, et cetera. And so it may very well be the case that every agent, it becomes like a whole new vertical and a whole new-
- SSSteven Sinofsky
Right
- MCMartin Casado
... specialization.
- SSSteven Sinofsky
Well-
- 48:47 – 54:44
Verticalization, Applied AI, and the Future of Agents
- ETErik Torenberg
Well, I, I think you can kind of underwrite thousands of, of, of these companies emerging. So-
- MCMartin Casado
Yeah
- ETErik Torenberg
... uh, Jared Friedman had a tweet, um, uh, about basically like go deep on a, on a workflow. Um, you know, take, basically do the job of, of some part of the economy, payroll specialist, and then build an agent for that.
- MCMartin Casado
Yeah.
- ETErik Torenberg
And it's not obvious that there's not literally a, like 1,000 of those.
- MCMartin Casado
Yeah, yeah, yeah.
- ETErik Torenberg
So by every vertical-
- MCMartin Casado
That's a whole new segmentation
- ETErik Torenberg
... and every line of department and-
- SSSteven Sinofsky
I just love this because this is like literally the anti-AGI spiel.
- ETErik Torenberg
Yeah.
- SSSteven Sinofsky
It's basically-
- ETErik Torenberg
Right
- SSSteven Sinofsky
... following like the long arc of computer science-
- ETErik Torenberg
Right
- SSSteven Sinofsky
... where as the market grows, the level, the granularity can create a company.
- ETErik Torenberg
Well, it's also economic growth. Like take that-
- SSSteven Sinofsky
Right
- ETErik Torenberg
... payroll example.
- MCMartin Casado
Exactly, exactly.
- ETErik Torenberg
Like today, just like Salesforce, which is always my favorite example, like the idea of having a produc-productive sales force used to just be a consultancy.
- MCMartin Casado
Right.
- SSSteven Sinofsky
And the only way you could ever fix it was hiring a consultancy to show up and analyze what everybody does, and then do a report-
- MCMartin Casado
Right
- SSSteven Sinofsky
... that says, "This is how you need to reorg." And it usually meant go the opposite of whatever you had.
- MCMartin Casado
[laughs]
- SSSteven Sinofsky
A- and then they would leave. And then, you know, people tried, but there was no cloud. So to build-
- MCMartin Casado
Right
- SSSteven Sinofsky
... like CRM, you had to do all that consulting work, and then roll it out. And, and then it was static, and you couldn't maintain it.
- MCMartin Casado
Right.
- 54:44 – 55:54
Platform Competition and the Application Layer
- SSSteven Sinofsky
raises, the, the big company raises the awareness of a whole category.
- ETErik Torenberg
Right. Right.
- SSSteven Sinofsky
And then you just swoop in and you go, w- you, to them you're, I'm just a feature.
- ETErik Torenberg
Right.
- MCMartin Casado
Yeah.
- SSSteven Sinofsky
But to, to you, I'm my, this is my whole life.
- ETErik Torenberg
Right.
- SSSteven Sinofsky
And, and you're gonna win. Look, I just, I always come back. There's a whole company that just signs things.
- ETErik Torenberg
Right.
- MCMartin Casado
[laughs]
- SSSteven Sinofsky
I, I, I like I can't, I cannot believe there's a whole company-
- ETErik Torenberg
Yeah
- SSSteven Sinofsky
... that just signs things.
- MCMartin Casado
I have so much to say about this topic. I mean, even minimally, if you graph like, like the cost to produ- the, the, so the willingness to pay for, um, an inference versus the cost to serve it.
- ETErik Torenberg
Mm.
- MCMartin Casado
Something like for most companies, for most spaces, 20% of the inferences are 80% of the cost. So like actually the problem of the application is just to choose those ones on, which tend to be-
- ETErik Torenberg
Right
- MCMartin Casado
... more domain specific.
- SSSteven Sinofsky
Yeah, yeah.
- MCMartin Casado
This is the problem of inviting the three of us on here-
- ETErik Torenberg
Okay. [laughs]
- MCMartin Casado
... which is like getting us to talk is easy.
- SSSteven Sinofsky
We just overbooked like the next two hours.
- MCMartin Casado
Just getting us to shut up is the trick. [laughs] And so-
- ETErik Torenberg
Yeah. Guys, thank you so much for coming on. This was fantastic. [upbeat music]
Episode duration: 56:04
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode 1z1xZF7X2cQ
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome