Skip to content

By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

a16za16z

Aaron Levie on AI Adoption and Enterprise Workflows | The a16z Show

Steven Sinofsky, board partner at a16z, Aaron Levie, CEO of Box, and Martin Casado, general partner at a16z, discuss the reality of AI inside enterprises.

Aaron Levieguest
Apr 28, 202658mWatch on YouTube

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

Great conversations run long. Their best ideas don't have to.

uListen is free on iPhone, iPad, Android, Chrome, and Claude.

See how

Transcript

0:00 – 1:05

Trailer

Speaker 0:00 So the board goes to the CEO and what does the board say? "We need more AI." [laughs]

Speaker 0:03 Yeah.

Speaker 0:03 And what does the CEO say? "Oh, okay. I'll get like a consultant to do more AI." And then they have some centralized project that nobody knows how it works. They haven't aligned their operations, and those things will fail.

Aaron Levie 0:12 The funniest concept that the more code we write, the less we would need engineers, it'd be the opposite because n- now your systems are even more complex than before, which means that you're gonna be running into even more challenges of when you need to do a system upgrade or when there's downtime and you have to figure out like, what, well, how do I fix that problem, or when there's a security incident. I mean, we're just getting started with the jobs on this front.

Speaker 0:32 They're gonna hit a wall at integration.

Aaron Levie 0:34 Yes.

Speaker 0:35 And, and this, the thing that's not different about AI, and that agents don't fix, that nothing fix, is that any enterprise of a thousand people or more or that's older than 10 years is just a mass of stuff that's sitting there waiting to be integrated. And, and you can't just say it's gonna integrate. AI actually doesn't help to integrate anything.

1:05 – 4:30

Introductions & The Silicon Valley vs Enterprise Gap

Speaker 1:05 Hey, we are here monitoring the situation l- live, and we're very excited to talk about a bunch of AI stuff, and we have, uh, three of us are here today. Uh, there's me, Steven Sinofsky, and M- Martin Casado, who will wave and say hi. I'm Martin. And-

Speaker 1:22 Hello. Hi, Martin

Speaker 1:23 ... and Aaron Levie, who is, is working on the elevation of his hair today, so we're excited about that.

Aaron Levie 1:28 I, it, it just keeps getting more vertical, and, uh, I thought I could kinda tame it, but it didn't work.

Speaker 1:33 A- and is that just a token issue or a parameter, number of parameters issue with the-

Aaron Levie 1:38 [laughs] It is too many parameters. Too many parameters.

Speaker 1:40 Okay. I have the same thing, but in reverse. Okay. So-

Aaron Levie 1:43 [laughs]

Speaker 1:44 [laughs]

Aaron Levie 1:44 You... Hey, listen, you have a distilled model.

Speaker 1:46 There... Yeah. [laughs]

Aaron Levie 1:46 Yeah.

Speaker 1:47 That's my... I run local.

Aaron Levie 1:49 Yeah.

Speaker 1:49 So we had a lot of, uh, there's been a busy week of things, but we're- we wanna bubble it up a little bit and just start talking about where, where things are, are heading. Um, but I'll, I'll let... I- just kick it to you, Aaron, and you start where you are the most excited this moment because you have visited a ton of customers this week and have learned a lot. You've shared a lot on X, but I think you're the most in-the-trenches CEO who is really talking to customers every single day in the enterprise, which is what the three of us tend to look at the most.

Aaron Levie 2:25 Yeah, I think my, uh, it feels like my job these days is just bring reality to the Valley and then bring the Valley to reality as, uh, as much as possible, and it's a, it is a kind of a crazy divide that, that, that exists at the moment. Um, you know, the past couple week-

Speaker 2:39 Well, so let's take a step back.

Aaron Levie 2:40 Yeah.

Speaker 2:40 I actually think it's super interesting. What, what is it, what's the gap caused by? What's, what, what is that?

Aaron Levie 2:47 The gap is caused by... Yeah. Well, I think the gap is, and M- Martin, I'm sure you see this, but I think the gap is caused by the styles of work that exist in Silicon Valley and in engineering roles versus sort of the rest of the world. So, and we've talked about this a couple times in, in different forms, but, but, you know, the, the technical aptitude of an engineer is just like insanely high. The level of wiredness to what's going on in the internet is insanely high. The, the ability to use your own tools and make your own choices is insanely high, and when things go wrong with the systems that you choose, you can just like quickly debug them and then make them sort of work for you. And then obviously you have all the benefits of just the models are really good at code and, and the work is verifiable. So you have like, you know, five or 10 things that make agents work in an enterprise context for engineering, or at least a, even a startup context for engineering, that, that tend to be a, th- there tends to be a gulf between the way you work that way in engineering and the rest of, of sort of knowledge work. And so, and so a lot of what I see is trying to figure out how do we kind of, you know, bottle up all of the greatness that is, you know, what we are seeing from coding agents, what we're seeing from agents that can use computers, um, to how do you bring that into the enterprise where the workflows are, are, you know, quite different. The users are less technical. The data is much more fragmented. The systems are much more legacy. And so that, that tends to be the divide. So it's not even that we're like talking past each other, like in a, in a, one of those kind of classic like government versus industry. It's, it's just literally like there is just a pure workflow and, and, and technology set divide and, and that's why it's gonna be, you know, a number of years for this sort of diffusion to, to roll from what we're seeing in Silicon Valley, what we're seeing as tech startups all around the world, into the rest of knowledge work.

4:30 – 9:16

Why Enterprise AI Efforts Keep Failing

Speaker 4:30 Martin, just to build a... You have a ton of experience in big companies. I- one of the other issues, though, is scale.

Speaker 4:37 Yeah.

Speaker 4:37 And the, the way, the difference in scale that Silicon Valley operates at, at the startup level versus everyone else.

Speaker 4:44 Yeah. I al- I also think that, I mean, these secular trends, like the internet was like this, actually start with individuals, and big companies tend to make decisions centrally, and this is one of the fastest growing secular trends. So like, uh, there's probably a lot of individuals in big companies that are doing it where like-

Speaker 5:05 Yes

Speaker 5:06 ... the big companies themselves don't know even how to think about it. And so when you hear stats like, oh, like AT&T had this stat, like 95% of AI efforts in big companies fail. Like, that's clearly silly because I am sure everybody's using ChatGPT very effectively. What-

Speaker 5:21 Yes

Speaker 5:21 ... what they really should be saying is, you know, whatever. Like I, listen, I sit in these boards too, so the board goes to the CEO and what does the board say? "We need more AI." [laughs]

Speaker 5:28 Yeah.

Speaker 5:28 And what does the CEO say? "Oh, okay. I'll get like a consultant to do more AI." And then they have some centralized project that nobody knows how it works. They haven't aligned their operations, and those things will fail. And so I don't... You know, when we say scale, often we, we think about things like system scale or number of peoples. I think the secular trend is scaling wonderfully, which is being reflected in the numbers of these companies, but organizations don't know how to adjust these kind of, you know, age-old processes that have been, you know, worked on for a decade around, you know, data and governance and operations and compliance, et cetera. That's kind of right now where I think Aar- like Aaron is right between the secular trend and the, the organizational decision body. Uh, and this is something that we actually track very closely because we're starting to see now, I would say in the last few months, finally some real kind of inroads into the enterprise, but it's, it's, it's tepid because... And, and the last thing I'll say of this, one of the reasons is there's a lot of skepticism because the board wants AI, CEO AI failures have created some-... amount of bruising, which is, you know, you know, requiring these companies to get past it in order to do kind of the second go at it.

Aaron Levie 6:38 Well-

Speaker 6:38 And so I think this is exactly where we are.

Aaron Levie 6:40 Yeah, I, I, a hundred percent agree with that, which is that it's good to start with agreements because we, we know how quickly that fades, uh [laughs]

Speaker 6:46 [laughs] Because we'll disagree the rest of the show. Exactly.

Aaron Levie 6:48 Yeah, exactly. Exactly. That's the only time we're gonna agree.

Speaker 6:50 [laughs]

Aaron Levie 6:50 Um, I, I think maybe one more point on the board for agreements, maybe, maybe you guys would, would agree. Um, there, there's also this very interesting, um, dynamic. I, I would say this is a minor one relative to everything else. It's probably 5% of the problem. I, I think it'd be more fun to talk about the, the real problem, but, but there is a fun kind of, as an aside, there's a fun dynamic where, you know, you go to an engineering team classically for the past, you know, and, you know, Steven, you can take us back in, in history on this one, and one of, like, the easiest ways to stall a project was just getting the architecture, you know, kind of the, the fights on, you know, what language to use, what architecture path to go down. That could take months and months to kind of work through as your teams work through that.

Speaker 7:29 Yeah.

Aaron Levie 7:29 Um, because of the pace of change in AI, um, you actually have this incredible dynamic where the, the labs, uh, you know, are, are obviously leapfrogging each other so frequently, but with, with not the exact same paradigm of how you should deploy agents and how they will work, and is the, i- is the, is the agent harness in the computer? Is it outside the computer? Do you run it in your cloud? Is it hosted? What tools does it have access to? Like, we are... Like, this is not a, a, a, a point where these are completely fungible technologies, and so that actually creates a, a, a bit of paralysis because now as an enterprise architecture team in the real world, you're like, "Man, like, what, what horse do I wanna, you know, kinda get behind, and, and which architecture path do I wanna get behind?" Because I've been burned by doing the wrong thing-

Speaker 8:16 Yeah

Aaron Levie 8:16 ... in AI maybe three or four years ago, and I went down some path that now is deprecated or not the right strategy anymore. So, so to some extent, the speed of our change in, in tech actually reduces the ability for the tech to get diffused into the really, really important workflows because now you have a lot of paralysis in, in just making decisions. So, so I actually think it's kind of fine because there's still so much upgrade work people need to do in their infrastructure and their systems and their data, but this is kind of an interesting dynamic where I'll, I'll go have conversations with CIOs and their AI teams, and I'll say, "Hey, what, what are you using for your chat system or your, you know, core agent orchestration?" And they'll say, "Yeah, we're in the middle of a debate between these two or three paradigms."

Speaker 8:55 Right. Right.

Aaron Levie 8:55 And it's... And, and you, and you hear that across almost every single customer because there is a little bit of a nervousness of, like, who do you get in bed with and, and how, how much do you sort of, you know, fully lock yourself into one particular path? And we also know that, that if you don't lock yourself into a path, it's always... Then, then you're building for this sort of duality which is, you know, also takes a lot of work architecturally.

Speaker 9:15 Oh, and actually-

9:16 – 14:38

The Architectural Shift: Treating AI as a User, Not Software

Aaron Levie 9:16 Yeah

Speaker 9:16 ... I, so I, I, I hate to jump in, Steve.

Aaron Levie 9:17 No.

Speaker 9:17 I just like, there's a, like... So Aaron is totally correct, and there's a, there's a very specific instance of this playing out in product companies right now.

Aaron Levie 9:25 Mm-hmm.

Speaker 9:25 And I'll tell you what it is. So, so, so software product companies, um, you know, circa six months ago, they viewed integrating AI was like you're actually integrating it into the product, right? So everybody was, like, adding, like, whatever, this chat feature or, like, the... You know, and so it's kind of like this fusion or this hybrid model. What we're seeing instead is instead of viewing AI as software-

Aaron Levie 9:50 Yes

Speaker 9:50 ... like, just view it as a user. And so [laughs] instead, like, take your product, make it a CLI tool, and then have the AI be an agent that actually uses this. You're not fusing the two. You're just making it more useful for AI. This is a very, very significant architectural and mental shift, right?

Aaron Levie 10:06 Yes.

Speaker 10:06 And so we started as pure product, and then we didn't quite know what the end thing looked like, so we created this h- you know, this, you know, AI software hybrid that hasn't worked, and now we're kind of going to the agentic model, which basically means th- th- the agent is gonna be whatever. It's gonna be cloud code or whatever, and then my product now just should be something that can s- consumed by that, and, like, that's the actual modality. But, you know, within a year now, you've had to rearchitect your software twice. And so I think no matter... But many places that you look in the industry is having this dilemma of actually trying to figure out what the final form looks like. And Steven, you will remember. Remember all the hybrid versions of cloud? [laughs]

Aaron Levie 10:44 [laughs] Yeah.

Speaker 10:45 Remember, like, you know, like, remote desktop and all these things? Like, I think we're kind of, like, speed running that evolution to the final form.

Aaron Levie 10:52 Right, and I, and I think that, that people, uh, in Silicon Valley don't quite l- appreciate when a big company says, "Well, we have to map out our bet that we're gonna make."

Speaker 11:04 Yes.

Aaron Levie 11:04 Because, like, that just seems stupid, and, you know, if you have... If your job history is, you know, five two-year stints at startups that went from seed to series A to acqui-hire or something-

Speaker 11:17 Yeah [laughs] you didn't learn anything

Aaron Levie 11:18 ... you, you, well, you, you never... You, you don't... Your frame of reference is not, you know, picking an accounts payable system-

Speaker 11:25 Right

Aaron Levie 11:25 ... that's gonna last 40 years.

Speaker 11:26 And living with the consequences. Yes. Right.

Aaron Levie 11:27 Yeah, I, I actually, m- I have, like, all these visual aids today. So here's, like, the ultimate, the ultimate engineer if you're in Silicon Valley, is, is-

Speaker 11:36 Lower, lower, lower, lower, lower

Aaron Levie 11:37 ... oh, lower, lower. Where?

Speaker 11:39 Yeah. Yeah, exactly.

Aaron Levie 11:39 Yeah, yeah, is Guilfoyle.

Speaker 11:41 [laughs]

Aaron Levie 11:41 And, and Guilfoyle is like, "I, I don't wanna talk to anyone."

Speaker 11:47 Yes.

Aaron Levie 11:48 "And I, I will just write the code, and you go do your thing." And the, the thing is, is that you, you have people in, in enterprises that are saying, "I'm gonna use the model and do my thing," but they're only... They're gonna hit a wall at integration.

Speaker 12:03 Yes.

Aaron Levie 12:04 And, and this... The thing that's not different about AI, and that agents don't fix, that nothing fix, is that any enterprise of a thousand people or more or that's older than 10 years is just a mass of stuff that's sitting there waiting to be integrated.

Speaker 12:19 Yeah.

Aaron Levie 12:20 And, and you can't just say it's gonna integrate. AI actually doesn't help to integrate anything.

Speaker 12:28 That's right.

Aaron Levie 12:28 And so even if you change everything, the, the people say, "Oh, no, if you make it an agent, then it can just go ahead and, and, and be a user." But if you're a user-

Speaker 12:37 Yeah

Aaron Levie 12:37 ... like, if you've ever called customer service for something, like, literally you get bounced to another human if the system that you're talking to-

Speaker 12:44 Doesn't have access

Speaker 12:45 It doesn't work.

Speaker 12:46 Yeah.

Speaker 12:46 And they're like, "Well, that's a manager," or, "No, you're, you're talking about payment, not reservations." And, and so, like, we're g- we're... What I think is so exciting is that now we have proof of this technology that everybody likes it. I mean, you see all the people who don't like AI are saying, "Look at what's happening in law firms because people are seeing hallucinations and it's ruining legal cases and all this." And, and the reason that's happening is because the 25-year-old associate is the one using AI successfully already, and had been using it for a year. And-

Speaker 13:19 Well, Steve, Steve, it's actually, Steve, it's actually a little worse than that, where it is right now many companies are incentivizing people to use AI by counting tokens.

Speaker 13:29 Oh, yeah, yeah, yeah.

Speaker 13:29 Right? And so I'm not gonna say the name of the g- I, I spoke to someone yesterday who worked for one of these large companies that famously does this, and he's like, "Me and my coworkers have agents do useless tasks-"

Speaker 13:40 They do

Speaker 13:40 ... "just so that we can [laughs] ..." No joke, in, in-

Speaker 13:43 No, no, no, totally. Well, I- you get whatever you measure, so.

Speaker 13:45 Yeah, yeah. [laughs] That's right.

Speaker 13:46 Yeah.

Speaker 13:46 So, like, it's, like, the extreme form of what you're saying, Steven.

Speaker 13:49 Oh, yeah, yeah, yeah.

Speaker 13:50 It's like you have people that are, like, h- being fake productive and producing a lot of, you know, uh, y- y- you could say perhaps problematic artifacts just because-

Speaker 14:01 Well, the, the-

Speaker 14:02 ... they're, they're using these models

Speaker 14:03 ... the, when, when the internet happened, all of a sudden every company needed websites. And so, like, a very famous moment in time was not too long ago when every internal team had, like, a team website, and they went out and they got, like, a vendor to write HTML-

Speaker 14:19 [laughs]

Speaker 14:19 ... and to create their site, and then there was a team. But, like, there's nothing dumber than having a team website at a large company because-

Speaker 14:25 [laughs]

Speaker 14:25 ... a team gets reorganized, like, six months later.

Speaker 14:28 [laughs]

Speaker 14:28 And so companies were just filled-

Speaker 14:31 That's funny

Speaker 14:31 ... with, like, these, with thousands of these dead web, is what, what the expression was. But I, I think-

Aaron Levie 14:37 Oh, but, but-

Speaker 14:37 Go ahead. Go ahead.

14:38 – 20:12

The Integration Wall Agents Can't Climb

Aaron Levie 14:38 No, no, I, well, but we should, we should drill into your integration point, because I do think this is something for, you know, sort of some reality to settle in, in, in the Valley on, on the real world's sort of journey to fully being agentified, and what that's gonna take and what that's gonna look like. And, and your, your point about b- being passed to the different human, you know, based on the role that you need to interact with, you know, agents basically don't have any... There's no real exception yet for the agent having the same problem, because you basically, you know, as you pass through a different human, it's, it's a different set of access controls that, that, that that human has. And if an agent can sort of bypass any of those steps, then, then that's how you instantly get the security risks that, that, like, w- you, you need to kind of pass through those steps so that way you don't accidentally, you know, get to the wrong piece of information-

Speaker 15:26 Right

Aaron Levie 15:26 ... and there's verification. And so there's a lot that, that you need to kind of build out for agents to be able to go and, and work with all these systems. And, and we've talked about this, but, like, most legacy environments don't have the most authoritative, you know, access controls. So you're always as a human going and saying, "Hey, Sally, can you share that thing with me that, that I don't have access to?" Or, "Hey, Bob, what's the number inside your data system for this question?" And so if agents just get the exact same permissions that you had, then they'll just run into these walls everywhere and they won't be able to complete the process. And unlike a human, they're not gonna know to go talk to Sally or ask the question of Bob, so they're gonna just be kind of s- you know, stuck. So what's gonna happen is you're gonna have a lot of agents that don't have access to the right data, um, they're, they're kind of working through systems that, that are, you know, not, not the real sources of truth for the information. They're getting the wrong number. They're getting the wrong document. So this is the real work that enterprises have to go through right now. The good news is that, that it's actually a great time, again, if you're a startup, because you can just, you get to know all the problems, uh, right at the, right out of the gate, so you can design your organizations, you know, to try and avoid this. But for big companies, there's real work that goes into, how do I upgrade my systems? How do I modernize my technology environment? How do I make sure that, you know, agents will have access to the right data, the right documents, the right context to be able to do their work? And that's sort of the, the, the work ahead. And, and there's, you know... I, I, there was this, uh, uh, you know, kind of headline of, of OpenAI working with, um, uh, in Codex, you know, working with Accenture, Deloitte, all, all the major system integrators, and there were some kinda, like, you know, snarky comments online around it, um, that, that I was fascinated by because it sort of sh- showed how, uh, h- how maybe, you know, great that divide is from the rest of the world versus those in tech. Because to me, it was, like, the most obvious announcement of all time-

Speaker 17:14 Right, right

Aaron Levie 17:15 ... which is a large enterprise is gonna have to go through the, the change management, the systems implementation, the integration of technology for these agents to be able to go and work. And so there was this sort of, like, you know, people thought it was somewhat ironic that, oh, we need people to implement the agents that are gonna go automate the people, and it's like, no, that's exactly how it works.

Speaker 17:32 Right, right.

Aaron Levie 17:33 Like, you, you, you actually do need to do lots and lots of work to be able to be in a position where agents can actually go and, and help you do, you know, any of the automation. So, so that is... And that's gonna be, there's gonna be businesses that are doing that for decades. Like, it's gonna be an incredible opportunity for this kind of next generation set of firms, as well as existing ones that, that lean into that.

Speaker 17:53 Let me throw this out there. Well, first, I think the other thing that people shouldn't celebrate when those fail, uh, because they will fail, because they're, as Martin was describing, they're gon- a lot of them are gonna be these sort of top-down mandates where they pick, like, the most acute problem in the company and think, "Oh, AI is gonna go solve that." And the IT people are gonna be like, "Oh, God, that's the worst-"

Aaron Levie 18:12 Yes. Yeah, don't do that. Yeah.

Speaker 18:13 That's, that, that's the worst system to try to do that. But the CEO or CFO or whatever is gonna be obsessed with solving, or the most likely the customer service person will be obsessed. But, but I do think if I were, if I were advising a startup specifically in order to, to, to sort of enter the enterprise space in that way, definitely would be thinking about not just, like, building a company that's step one. I only work with all the headless SaaS software that's out there, because there just won't be any. Like-

Aaron Levie 18:42 No

Speaker 18:42 ... the, but the, the thing you can do is structure the value that you offer, and also this applies to what you go do in a company, is it's really a fork, and the fork is-Is this an agent that is seeking information and presenting it to an, to some human, or is this an agent that's supposed to go act and do something? Like, is this-

Aaron Levie 19:03 Yes

Speaker 19:03 ... is it acquiring or is it doing? Because if, if it's, it turns out that's how, what happened with the internet. The internet got very, very valuable when the first step was just providing access to things to people.

Aaron Levie 19:17 Yes.

Speaker 19:17 And, and like all of a sudden, all the sites that were like that literally did integration. Like, "Hey, I need expense reports, but viewed by department," or, "I need to see our current inventory status across, like, the two companies we've acquired." All of a sudden, the web became the integration point. And so I do think that, that if you just show up first and just say, "Hey, we can actually use agents to learn stuff about what's going on in a company," and in particular, because you're here, Aaron, like, learning across the files becomes way more possible than it ever was before. In fact, AI might be the first time that inside a company's search can actually provide-

Aaron Levie 19:57 Yes, yes

Speaker 19:57 ... immediate value.

Aaron Levie 19:59 Yes.

Speaker 19:59 'Cause the web just wasn't structured to deliver those results. And, and then you start to think, once you can bring them all together, then you can add, like, an Approve button or a Reject button or something like that.

20:12 – 24:40

Should Agents Be Treated Like Humans?

Speaker 20:12 Let, let me, let me just try and provide... Finally, the point where I get to disagree. [laughs] So let-

Speaker 20:17 Uh-oh. We're in trouble now.

Speaker 20:19 No, no, no. I, I just-

Speaker 20:20 Well, you can get invited back. You're invited back, so good.

Speaker 20:22 No, no. I, I think this is a very legit view, but it's not the only view, and in light of AI, it's, I think it's not the only kind of compelling view. So here's the oth- so the, let, let me just try and rephrase. So the, the current view is we've got, like, AI is software. It, it, it works in a different way. Um, we have a current set of systems, and we have to integrate this new type of software with our existing systems so that it can get access to data, it can do things, but in a safe way, right? So here's to the kind of end-to-end argument of why this isn't abov- about evolving software systems. The end-to-end argument is these LLMs are non-deterministic, they are smart, they deal with the long tail of complexity, and it turns out those are all things humans do, too, and we've spent 40 years building interfaces, processes, and design to deal with messy humans. And, you know, we know who to access and we have access control. And so if you have the mindset that an agent is more like a human, and you hire the agent, you give it its own email address, it can access documents like humans can, it can log in, it can request the things that it needs, then it will be drafting on all of the process that, that we've put in place for humans, not for software. And so I would just encourage us as we have this discussion, like listen, I grew up like you guys in software. I always think of every system like software, but these models don't integrate well with software. Actually, I think it turns out, and what we're learning as an industry, is if you view them more like humans and you draft on the, um, mechanisms we've put in place for humans, they're much easier to integrate.

Speaker 21:57 Well, that-

Speaker 21:58 And I think we're seeing-

Aaron Levie 21:58 Yeah, I love that point. I, I think we, I think we agree with that, for sure. I think the issue is humans have a bunch of extra benefits that the agent doesn't have. The human has a lot of context that it gets for, that they get, that we get for free by virtue of we can keep track of the myriad relationships that we've built in our organization and the person to tap on the shoulder when, when we need something done or we need to get information. That's not documented in a company yet, um, in a, in a way that the agent can just sort of draft on. And so, so I, I like, I, I mean, I think we all would agree that, that you ha- you can't treat this like software. You treat these as, as people accessing systems and tools, but they are at a, they're both at an, a massive advantage that they can work in parallel in, at, you know, at infinite scale, and they're at a disadvantage in that they don't know who to tap on the shoulder.

Speaker 22:40 Right. I, let's-

Aaron Levie 22:41 Yeah

Speaker 22:41 ... Aaron, I am all for agent onboarding. Like, you know-

Aaron Levie 22:45 [laughs]

Speaker 22:45 ... the agent comes and it goes to orientation.

Aaron Levie 22:47 Yeah, yeah, yeah.

Speaker 22:48 And then the CEO gives it the culture discussion, and then every... [laughs] I'm not kidding.

Aaron Levie 22:52 No, you're probably right. Every department-

Speaker 22:54 Every department-

Aaron Levie 22:55 100%

Speaker 22:55 ... every department does their pitch.

Aaron Levie 22:56 Yes.

Speaker 22:56 Like, "This is what we do."

Aaron Levie 22:58 Yes.

Speaker 22:58 And like, I mean, I think, I, I actually honestly think given, given the technical nature of these agents-

Aaron Levie 23:03 Yeah

Speaker 23:04 ... and how much entropy they have and kind of how unruly they are, we're gonna have to go through the processes that we've refined around humans.

Aaron Levie 23:13 Yeah, 100%.

Speaker 23:13 Because humans have all of those things.

Aaron Levie 23:14 Yes.

Speaker 23:15 And so I just, you know, it's more about providing schools for them than somehow building some, you know, fancy index database.

Aaron Levie 23:23 No, no, no.

Speaker 23:23 But I think-

Aaron Levie 23:24 Totally, totally agree.

Speaker 23:25 That, I mean, of course, what I love about, about that is you just keep going with the analogy, because what that is, is it's the same argument that humanoid robots will, will be-

Aaron Levie 23:33 Yes. Yeah

Speaker 23:34 ... the best kind of robot.

Aaron Levie 23:35 That's exactly my argument. Right.

Speaker 23:35 Which is we have a whole world designed for humans, and I, like, I, I saw at the Consumer Electronics Show, I saw this robot go into an elevator, and then there was a, a button-pushing robot on the elevator. So-

Aaron Levie 23:49 Yeah

Speaker 23:49 ... because the, the robot was a tiny little thing that, like a Roomba on the floor, it couldn't push the button.

Aaron Levie 23:55 [laughs] Right.

Speaker 23:55 So the same company that invented that robot invented a device you buy for the elevator that pushes the button.

Aaron Levie 24:02 [laughs]

Speaker 24:02 But then I asked, "Why did you need a device to push the button?" And it, and it was very interesting. They said, "Because the elevators don't have systems that they can hook into as a, a robot." Like, there's no Wi-Fi press the button in the elevator capability.

Aaron Levie 24:18 Yeah.

Speaker 24:18 So-

Aaron Levie 24:18 There's no API for that

Speaker 24:19 ... there, there's no, there, there is no headless version of the elevator.

Aaron Levie 24:22 Yeah.

Speaker 24:23 And, and I thought-

Speaker 24:23 That's a, that's actually a great metaphor for, like, the problem that I think that we're actually solving in the enterprise with these agents, which is we just, you know, we have two types of systems, those for humans and those for software, and these tend to be more like humans, so we should draft on those as much as possible-

Aaron Levie 24:38 100%

Speaker 24:38 ... rather than try to retrofit them as-

24:40 – 39:16

Salesforce Goes Headless & What It Means for SaaS

Aaron Levie 24:40 Well, well, and so, so the big news last week was, uh, was I, I, I think, you know, Salesforce, I don't know if they surprised people or not, but, but I mean, based on the reaction, it seemed like it was maybe a surprise. They, they went full headless, and they basically said, you know, like, "We wanna be used everywhere across all of our, uh, all, all the different agents."Um, and I see that as a, a little bit of a bellwether because I think as Salesforce goes, so does a lot of, of, of enterprise software. And I think a lot of people are gonna try and f- you know, have to figure out what is the new business model in this headless world. You know, do you, do you charge a little bit of a, a small just API tax? Is there a seat for the agent? So there's obviously some work to do with that, and Steven, I, I saw one of your tweets on, you know, s- some of the, some of the, you know, complications there. But, but, but, uh, but I think as a, as a moment it's a big deal because, because I think it's a recognition that, that, you know, software will be running in the background. It always has for machine users and applications, and now it is for these sort of probabilistic machine users and, or non-deterministic machine users. And what's cool and, and where I think this gets pretty exciting is, you know, as soon as I saw that announcement, like, I had like five to 10 personal use cases where I would need, you know, the headless version of Salesforce, 'cause I'm always doing just a tremendous amount of, of customer-related intelligence work. I'm going into a meeting-

Speaker 25:57 Yeah

Aaron Levie 25:57 ... I need some information. I need to do... I'm going into a city. Who should I be meeting with? And so if you imagine, you know, being able to run compute in the form of agents across all of your data systems, like, the use cases become, you know, pretty wild around what that opens up. So I think this gives, I think this gives a lot of software platforms all new use cases that they can tap into that where, again, you were normally constrained by the number of people on these platforms, but now the headless user can be, you know, 100 or 1000 X the scale, um, of those human users. So this is a, a, I think an exciting moment because as you have more of these agents running around and the headless software modes, um, you just have, you know, way more use cases for these tools.

Speaker 26:34 I also think-

Aaron Levie 26:35 So-

Speaker 26:36 All right, just, I think on this one-

Aaron Levie 26:37 Please, please. Yeah

Speaker 26:37 ... what's so, what's so super cool is, is that of course the first step is doing exactly what you described, which is just looking stuff up. And so the, the most interesting thing is using this notion that an agent is just a, an entity, it's mo- it's incredibly obvious to me that it's another license. Now, it might have a different-

Aaron Levie 26:57 Exactly

Speaker 26:57 ... license model, but it hac- it has to have an identity. Like when you go look something up in, in the Box, um, CRM system, I, I don't know if it's Salesforce or not.

Aaron Levie 27:07 Yep.

Speaker 27:07 When you use the Box CRM system, it has to be a person, like with a certain amount of access rights.

Aaron Levie 27:14 Yes.

Speaker 27:14 And you presumably, as CEO, you might have access to a bunch of stuff. But also there's a lot of ways that they actually don't want you to have-

Aaron Levie 27:20 Right

Speaker 27:20 ... the rights at the right time. Like, you might be able to look and see who is on the account, but you don't need the up-to-date quota-

Aaron Levie 27:25 Right

Speaker 27:25 ... of those salespeople and stuff, and that might be HR sensitive, and you should probably have some other level to go see that. But as you go down the org, the agent is never gonna have more permissions than the person who's getting it to go do something. And in fact-

Aaron Levie 27:41 Right

Speaker 27:41 ... it's just gonna be like a peer to somebody else in an organization, because otherwise you have all of these issues where the peer, where a human can just say, "Oh, get me the super smart agent that knows everything that I'm not allowed to know."

Aaron Levie 27:54 Right.

Speaker 27:54 And to, to the, in the mo- in the IT architecture sense, what's so fascinating about that is you have to build... You, you can't let the agent get the results and then try to figure out what works or not. But first of all the points that Martin made about, about, um, who, about the, the LLM stochastic model, which is you're not gonna be able to figure out. It's not like a record in a SQL table that you could just apply ACLs to. It's-

Aaron Levie 28:18 Right

Speaker 28:18 ... it's actually like it could be words in a sentence or just the number that shows up.

Aaron Levie 28:23 Yeah.

Speaker 28:23 And so I actually think it, it, that whole discussion about headless for me made the SaaSpocalypse seem even dumber than it was already, and it was already dumb.

Aaron Levie 28:33 [laughs]

Speaker 28:33 So like, it was like at first it was dumb, and then I'm like, oh my God, it's actually much dumber than I thought it was-

Aaron Levie 28:40 [laughs]

Speaker 28:40 ... in the first place.

Aaron Levie 28:41 Totally.

Speaker 28:41 Because you're just gonna have this explosion. Now, someone might-

Aaron Levie 28:44 Yeah

Speaker 28:44 ... come up with a very clever pricing scheme and that-

Aaron Levie 28:46 Yes

Speaker 28:46 ... agents, you know, somehow cost less because maybe for the first five years they're read only or they, they're always tied to a person or something. But the, it is another seat. There is no way-

Aaron Levie 28:57 Yeah

Speaker 28:57 ... around it. And like if you're a SaaS company, you're crazy to try to say, "Oh, just use the credentials of another human." Be- like that's just, that would be like bad security practice from the get-go.

Speaker 29:08 Exactly. So actually in, in fact, um, so this is playing out in many domains. You can even make the argument that like a, a, a headless SaaS doesn't make sense. And, and here's the argument. The argument is... Let, let me give you an example. So, um, if you use OpenClaw, do you know why you use a Mac Mini with OpenClaw? It's number one for iMessage, so-

Speaker 29:28 Right, right

Speaker 29:29 ... so it's-

Speaker 29:29 For the integration

Speaker 29:30 ... so-

Speaker 29:30 Yeah, yeah, yeah

Speaker 29:31 ... because there is no headless version, so you're just gonna like use it.

Speaker 29:34 And it's very easy.

Speaker 29:34 And then the second one is very interesting, which is if you've tried to use headless browsers with agents, the problem is, is all of the websites, um, have anti-scraping measures.

Speaker 29:44 Right, right.

Speaker 29:44 So they don't work.

Speaker 29:45 Mm-hmm.

Speaker 29:46 And so the reason you use a Mac Mini is so it can actually use Safari proper. So to do anything headless kind of assumes that like the entire internet is gonna go headless when I think all of these models, like all of the data is humans working on the actual apps that are not headless. Like that's all of the data anyway. So I think these models are gonna be very good at just using apps like they are today, and we're already seeing this happen, and rather than the headless versions, the non-headless versions are what's actually being used. So you could argue that it's just Salesforce [laughs] like not headless. Like-

Speaker 30:19 Yeah

Speaker 30:19 ... it will go to a-

Aaron Levie 30:20 But wait, but wait-

Speaker 30:21 ... it'll go to a browser-

Aaron Levie 30:21 ... but, but, but wait, actually just to clarify, do you literally mean the agent goes to the browser and-

Speaker 30:26 Yes

Aaron Levie 30:26 ... oh, no, no.

Speaker 30:27 So, so, so-

Aaron Levie 30:27 I'm, I'm taking the other side on that one big time.

Speaker 30:29 Yeah, yeah.

Aaron Levie 30:29 Zero.

Speaker 30:29 No, but, but, but, but-

Aaron Levie 30:30 Yeah

Speaker 30:30 ... let me just, let, let, let me, let me simplify the argument-

Aaron Levie 30:33 Yeah

Speaker 30:33 ... so we can actually have it.

Aaron Levie 30:34 Yeah, yeah.

Speaker 30:34 So, so, so today if you use an agent like NanoClaw or OpenClaw, you could use a headless browser. Let's say I wanted to like look up the value of my house on Zillow. The headless browser simply doesn't work-

Aaron Levie 30:47 Sure

Speaker 30:47 ... because Zillow's so tired of people scraping it, so it will detect headless browsers.

Aaron Levie 30:50 Totally.

Speaker 30:51 So the thing that works is it, you is you pop up Safari and it uses a proper-

Aaron Levie 30:56 Yep

Speaker 30:56 ... Safari directly, right? And so then-

Aaron Levie 30:58 Yeah

Speaker 30:58 ... then all of a sudden it works. And so-

Aaron Levie 30:59 But no, but, but, but, but, uh, I think, I think, um, hmm, I mean, uh, I would just say that, that-That any, any software that has a good API, the agent would absolutely prefer to use the API-

Speaker 31:11 Well-

Aaron Levie 31:11 ... and then you, and then you pop into the browser the moment that you run into some execution problem with, with the system

Speaker 31:16 Well, I mean, and, and, you know, Se- Set- Seth is a, a fantastic long-term computer science software guy. However, these models are trained on data and RL environments from existing software that didn't have those APIs.

Aaron Levie 31:29 Yeah.

Speaker 31:29 And, and, and, and right now, if you actually look at the adoption and the use of these agents, they l- look far more like what a human would do than what, like, a program would do. So maybe you're right, but, A, that's not what we're seeing, and you can-

Aaron Levie 31:46 Yeah

Speaker 31:46 ... honestly make the end-to-end argument when it comes to data [laughs] and-

Aaron Levie 31:49 Well-

Speaker 31:49 ... all of the controls in the internet, to Steven's point, all the existing controls-

Aaron Levie 31:53 Yeah

Speaker 31:53 ... to just be like, "These are gonna actually have the same actions as humans."

Aaron Levie 31:56 Well, the APIs most that, that... I mean, uh, I think the... I mean, the APIs of any software provider will follow the same access controls of, of whatever the, whatever the user is that, that is

Speaker 32:06 Right, but they have to rebu- I mean, they have to re- they have to rebuild it. Uh, I mean, it's like, it's like you've got this existing app, and all the da- the model's trained on-

Aaron Levie 32:13 Yeah

Speaker 32:13 ... all of the people using the app.

Aaron Levie 32:15 Well, well, well, on, on that point, and it's a totally fair point, but, um, I would, I would guess over time you're gonna have really, uh, you, you're gonna have very, um, uh, you know, kind of accurate, rigorous, uh, uh, data sets for, you know, for models to be trained against the MCPs of, of every SaaS platform, the APIs of every SaaS platform. Already, it's in the... You know, they're already training against all of our documentation, um, on our, on our products and our, our APIs. But, but I, I just think, uh, to me it's, it's more just an inefficiency of, of navigating through pixels versus just, you know, you, you can just-

Speaker 32:51 It-

Aaron Levie 32:51 You can just do it quick-

Speaker 32:52 An adage-

Aaron Levie 32:52 Yeah

Speaker 32:52 ... an a- an adage in systems, Aaron, is that, [laughs] is that layers never go away. [laughs]

Aaron Levie 32:57 Yeah, yeah.

Speaker 32:57 They just get layered.

Aaron Levie 32:58 Well, well, so, so-

Speaker 32:59 This model can do the next-

Aaron Levie 32:59 ... I mean, but on that, I'll, I'll support your point 50%, um, if you look at the- Oh, well, there you go. Yeah, yeah, yeah. [laughs] Like, I just, I just think, I, I just think if you need to do a search, uh, for a document, our search API is gonna be a faster way to do it than, you know, you know, clicking through, uh, an interface. Oh, 100%, yeah. Right. But, but, but, but the... but to s- to half support the point, like, the new Codex, the computer use that, on the desktop is, is a- you know, just insane, and I... I mean, Steven obviously knows, you know, everything about how it would work. I... When I saw my ability to move a mouse and then this other sort of mouse moving and clicking things, I was like, "I don't understand computers anymore." Right, right. [laughs] So, so there is a pretty s- and, and, and to your point, Martin, um, my first instinct was to use it for something where I know there's no available API. So I did actually use it right away for a thing that I don't have access to the API. And an agent over time is gonna probably have to figure out, is there an easy MCP or CLI for this action? And if not, then I'm gonna pop into some kind of cloud browser or cloud computer- Yeah ... or maybe a local thing that I can, you know, sort of parallel track and then, and then go and execute that. So that does seem like a reasonable architecture. Um, uh, but, uh, but I, I still think that, like, I'm gonna, I'm gonna pound the Salesforce API massively in, in h- headless mode just 'cause that'll be an efficient way to go look up records. Yeah, yeah, for sure. Yeah, yeah, yeah.

Speaker 34:16 Yeah, I mean, it, I, I think that you're, you're sort of... I think you're both saying the same thing, but there's just a time d-

Aaron Levie 34:23 [laughs]

Speaker 34:23 But no, but there's a time dimension.

Aaron Levie 34:25 Yeah.

Speaker 34:25 And I, I think, like, there was a moment at the internet that, that I really was thinking about when, when you guys were... when I was seeing the timescale difference, whi- which was suddenly the eight million catrillion pages of how to use Word and Excel that we had written over the years, that we posted on the internet, we had used to ship them with the product.

Aaron Levie 34:44 Mm.

Speaker 34:44 And people would have them on their hard drive, not connected to anything, and they would say, like, "How do I make a nice chart?" or whatever, and it never worked. They could never find the thing that they wanted. But what happened with the internet was the net result of everybody finding it caused us to make better documentation, but it also-

Aaron Levie 35:02 Mm-hmm

Speaker 35:02 ... caused Google Search to be better at finding the information that it needed-

Aaron Levie 35:07 Oh, right

Speaker 35:07 ... which then completely-

Aaron Levie 35:08 Right

Speaker 35:08 ... changed the way that we thought about doing documentation.

Aaron Levie 35:12 Oh, yeah.

Speaker 35:12 And I think that with headless, especially for the kind that's just finding things, it's gonna really change the, the way that information is exposed. And so the way that Salesforce sees today of exposing a headless API is... I'm almost certain if I were to go look at it, it's gonna look like the developer API in front of a, behind a CLI, and it's gonna look a lot like that. But in fact, that's not at all how humans using Salesforce interact as a human trying to solve, like, I'm standing in the elevator waiting to go see a customer, what is the stuff I need to know? Like, that, that mapping is completely different, and so the, that API is gonna really change, uh, as a-

Aaron Levie 35:53 Yes

Speaker 35:53 ... result over time. And-

Aaron Levie 35:55 Yes, I, I, I think, I think the API changes for sure. I, I agree with that, but I do think that unlike the humanoid, you know, kinda comparison, where, where sort of the physical world has interesting physics issues, uh, that, that you eventually run into, the digital world doesn't. And so, so at some point, your agent can run in parallel, you know, 500 times, and, it, like, I'm going into, uh... I wanna do a market map of, of customers across the Fortune 500. Um, that agent can, can fan out and do that work in a way that I can't as a person in a browser.

Speaker 36:29 Right.

Aaron Levie 36:30 Um, so, so, so to some extent, agents get to let you, you know, sort of bend the laws of, of normal, you know, human-based workflows. Um, and, uh, and so then just, like, that's why s- that... a- and I think that that means the APIs maybe eventually evolve, but not obviously in the direction of the end user product, but, but maybe more toward an agentic sort of set of workflows of what is that agent looking to do?

Speaker 36:53 Well, but Martin I think would jump in and just say, "Wait, you didn't describe anything new."

Aaron Levie 36:57 Yeah.

Speaker 36:57 "You described an architectural-"

Aaron Levie 36:59 [laughs]

Speaker 36:59 No, but you described an architectural problem with today's software-

Aaron Levie 37:02 Yes, exactly

Speaker 37:03 ... which is, its, its API and performance gate was based on-

Aaron Levie 37:06 So-

Speaker 37:06 ... how much I can typeWhich, which is sort of the point I was making, which was-

Aaron Levie 37:10 Yes

Speaker 37:11 ... our help system was designed on how much we could ship on one CD-

Aaron Levie 37:16 Yes

Speaker 37:16 ... and had no data about what it is that people were trying to do and no context.

Aaron Levie 37:21 Well, well, what, what-

Speaker 37:21 But it didn't change, like, the problem, which is I needed to make a chart.

Aaron Levie 37:25 Yes. Yeah, exactly. The, um, uh... Well, so like one, one real example of this, we, we launched a Box agent that gives, you know, that has, you know, a bunch more capabilities built into it, and one of the capabilities is that it searches across your whole Box environment. But it doesn't, it doesn't have the same limitations of a human-based search where you, you type in one query, you get back a set of results, you look through them. You know, it fans out, does multiple queries. It can look through hundreds of results instantly and do its own re-ranking of that. And so that's just like, you know, again, you wouldn't wanna be rate limited by the same, uh, process that a human went through, which is, which is where the humanoid robot is, you know, you're kinda willing to be like, "Okay, the humanoid is still gonna walk into the elevator. It's still gonna press the button," when actually, you know, in agent, agent world, you're like, "No, no, I just want you to go and instantly press the, the, the floor that I'm going to."

Speaker 38:12 Yeah, yeah. But we should just be very clear, like, um, by the way, I, I very much agree. Uh, but we need to make a distinction between, like, would you ever build an indexing that's only for AI, not for a human? And I think that's less obvious.

Aaron Levie 38:25 Yeah.

Speaker 38:25 So clearly there's-

Aaron Levie 38:27 Yeah

Speaker 38:27 ... like, performance gains based on automation. We've gotta evolve our arch- architectures for those. But if you find a great way to index documents and you don't expose it to a human, I think-

Aaron Levie 38:35 Yeah, you gotta... Yeah, exactly, 100%. Well, well, this is a- I mean, it's actually, and I think this kind of v- probably reinforces some of Steven's, you know, kind of internet analogy on documentation. There is this really interesting thing where, you know, it started out where we, uh, as we've been building our, our next set of agents, um, we first gave it this, the current set of tools. We saw how, how it used those, and then eventually we realized, "Oh, there's actually even, even better way that the agent could do it," so we, we improved the underlying scaffolding.

Speaker 39:04 Right.

Aaron Levie 39:04 And oh, by the way, that will actually help the end user also. So it does let you sort of contribute back into the mothership of, of technolo- of technology improvement that does, you know, sort of lift all the boats of, of your users.

39:16 – 47:53

Scale, Entropy & Why AI Coding Creates as Many Problems as It Solves

Speaker 39:16 Let me, let me ask this. It occurred to me as you were saying it. Like, I, my, I sort of got all tense when y- when the idea became-

Aaron Levie 39:23 [laughs]

Speaker 39:24 No, like, oh, we have 10,000 people hitting our, hitting our SaaS system today, and we've got it all working and it's all great, but, um, now we're gonna have 10,000 new pe- people, which are the agents for each of those 10,000 employees, and they're, they're actually hitting it 500 times as much.

Aaron Levie 39:42 Mm.

Speaker 39:42 Okay, so that SaaS product will collapse.

Aaron Levie 39:45 [laughs]

Speaker 39:46 So, like, that's the, the first order because it wasn't architected for that volume. Like, we saw this with, with all the BI tools. Like, when all the-

Aaron Levie 39:54 Mm

Speaker 39:54 ... BI tools came out, all of a sudden they, they were looking at the SAP data and trying to snapshot it and absorb the whole thing every night for a new kind of set of slices and dice that your view across all 500-

Aaron Levie 40:08 Yes

Speaker 40:08 ... pro... And, and, like, all the people making ERP were like, "Well, we don't do that." And so they had to go build all of this themselves because they had the knowledge of the data. Their API just couldn't, was not designed for, for that kind of lookup. So my, my sort of thing to throw out there and, and fight about is what is it, what does the change management look like in a company? Because you, you can't let loose an agent that hits the system at 500x the humans. And it's not a token thing. It's an actual, like, wow, we don't have the network bandwidth and the, the throughput to handle 500x for any one of our customers. So what happens?

Speaker 40:48 So, so I've, so I've got a provocative adjacency, which you guys can tell me if I'm doing too much on a tangent here, but, but here's my provocative adjacency, which is I don't know if having more agents is that big of an architectural shift. I just feel like we understand, like, whatever. If it's read-only data, you cache it. You know, like, all the state issues are around mutable, mutable globally shared state. We understand the limits of those. We know how to architect around those. We had to tackle all of those things when we went to the internet. And so if you built your system not to handle it, like, you suck at building the system-

Speaker 41:20 [laughs]

Speaker 41:20 ... and you deserve to go down, and just go build a system that doesn't suck. And, like, I just feel like this is kind of standard computer science. However, I do think agents do introduce, um, something that organizations, uh, technically have to deal with, and I'll- l- let me just give the analogy in code, which is in-

Aaron Levie 41:35 I think, uh, Steven, this is what we call mogging on a, on a podcast.

Speaker 41:39 [laughs]

Speaker 41:39 I, I don't know. No, no, I think it's a great question.

Aaron Levie 41:41 You've been question mogged.

Speaker 41:41 I just-

Aaron Levie 41:42 He's been question mogged

Speaker 41:42 I have no idea what he just did, but I-

Speaker 41:44 [laughs]

Speaker 41:44 ... I'm, I'm just looking forward to how he magically made the problem go away. But go ahead. [laughs]

Speaker 41:50 No, no, no, no, no, no, no, no, no. No, the problem is there. I just think, like, we know how to go from 10 users-

Speaker 41:54 Yeah

Speaker 41:54 ... to 1,000 users.

Speaker 41:54 It's there for stupid people.

Speaker 41:55 But-

Speaker 41:55 We just got rid of the stupid people, so now everybody's smart.

Speaker 42:00 No, no.

Speaker 42:01 [laughs]

Speaker 42:01 I just... [laughs] No, no. Okay, so but, so let me give you an example for coding. So this is where I actually think there's a shift in how work gets done. So when you code, um, with AI, your code kinda gets worse over time pretty materially. And so it's almost like you're introducing as many problems as you are solutions, and I don't think we've actually figured out how to manage that. Does this make sense?

Aaron Levie 42:24 Uh, 100%.

Speaker 42:25 In the whole world right now, yeah.

Aaron Levie 42:26 Yeah, yeah, yeah, yeah.

Speaker 42:27 I, I, you know, I mean, like, this, this kind of reasonable question is, you know, if you're using AI, yes, you're productive, but are you creating more problems [laughs] than you've actually solved for solutions? And I do think that there is this actually, you know, open question when it comes to using agents on existing systems for creating things, which is like, like, do we know how to wrap the growing set of entropy around that? And I would say anecdotally, watching companies struggle with AI coding, which of course I'm, I'm, you know, listen, I, I'm very close to many AI coding companies. I'm clearly very bullish on it. I don't think we know how to do that yet. And so the, you know, the agents on, on a, a system, I think we can tackle those with known techniques. Using agents for long-running things organizationally-

Speaker 43:12 Yeah

Speaker 43:13 ... where, like-You know, you know, the, the universe is kind of as clean as it was three days after than you started. I'm not actually quite sure we know how to do that at all.

Speaker 43:22 Well, I, I love that point because that gets back to where we started, which is the difference between scale and not scale.

Speaker 43:28 Yes, yes.

Speaker 43:29 And why it's perfectly rational for big company people to be like, "No fricking way is this coming into our company." Because a big company is about to c- the wheels are gonna come off a big company or a division in a big company or a product in a big company at any minute. Like, if you're a... Martin, we were both giant company executives.

Speaker 43:49 Yes.

Speaker 43:49 Like, literally we woke up every morning thinking, "Oh, the wheels are coming off today. This is the end of it. I'm getting fired by the 5:00."

Speaker 43:57 Yeah.

Speaker 43:57 And w- whatever started, the, what... I left yesterday thinking we were three months late-

Speaker 44:01 [laughs]

Speaker 44:01 ... and it's, we're now nine months late.

Speaker 44:03 Yeah.

Speaker 44:03 And that's a typical day.

Speaker 44:05 Yeah.

Speaker 44:05 And so, but the reason that that doesn't happen is because you put constraints all over the place.

Speaker 44:11 Ex- exactly.

Speaker 44:11 Which is exactly why Gilfoyle can't work at a big company.

Speaker 44:15 [laughs]

Speaker 44:15 Because he, he thinks he knows... And, and it's also why all the, the one-shotting, vibe coding kind of people have no problem saying, "It's fine," because-

Speaker 44:27 Yeah

Speaker 44:27 ... they've never had to live in an environment where the constraint-

Speaker 44:30 Yes

Speaker 44:30 ... was to prevent the whole thing from imploding. And, and-

Speaker 44:33 I, I feel this is so critical, Steve, that you're, like, so, a, and, and again, this is gonna sound like a l- a little tangential, but, um, but it feeds into this, which is I feel like core technologies kind of cater to, like, a, some human need. Like, the internet catered to, like, connectivity, and social networking kind of catered to vanity, and I feel like AI caters to our need to be productive. [laughs]

Speaker 44:55 Mm.

Speaker 44:55 So I feel like we feel like we're being very productive when we do all of these things, but we may actually be creating, like, mounds of extra work to do. And to, to this example, like-

Speaker 45:04 Well, well, Aaron, you're deploying AI right now. Like, Box is all in.

Aaron Levie 45:08 Yeah.

Speaker 45:08 So tell us-

Aaron Levie 45:08 Yeah

Speaker 45:08 ... tell, share a story, like, of the wheels coming off or not coming off.

Speaker 45:13 Yeah.

Aaron Levie 45:13 Well, well, I, I think, um, I think, uh, w- we're probably in the more pragmatic part of the continuum, so, so which is why we don't claim it, it's a 10X productivity gain to our engineering team. It's like, no, no, 'cause we have a lot of guardrails in place that create these constraints automatically in our system. There's, we, we still rely heavily on, on code reviews. We still rely heavily on security reviews before we release anything.

Speaker 45:36 So are you, are you guys coding with, like, a rock and a chisel and stuff?

Aaron Levie 45:39 I know, it, it feels like that sometimes.

Speaker 45:40 That just sounds like [laughs]

Aaron Levie 45:41 We have chalkboards and, like, but, um, uh, no, but, like, I, I, uh, we had this new feature that we, that we launched, and I was like, "Go, go, go, go, go." And, and AI built probably 80 to 90% of the feature, and the, the thing that slowed down the release of it was we have to do a full security review because we can't let there be any, you know, accidental code injection into the thing that we created. So, so there's a lot of stuff where you kind of go super fast, but then you get still rate limited or constrained by some other part of the process. Uh, and I d- I think that's sort of, you know, relatively natural until we figure out then that other part of the process, security review being one, or the actual code review being one, or just even your pipeline for, for, you know, getting things into production being another one. So we, we're doing a, quite a little, you know, quite a, quite a bit of retooling of the whole product development life cycle. But I d- I don't think that it's a 5 to 10X gain. I do think it's a 2 to 3X gain maybe across the board. You are still rate limited by how quickly can you review this stuff and, and check on the work. Um, I do think that, that t- Martin's sort of pointing at, though, a thing that is the, the big open topic across enterprises, uh, and, and s- you know, to some extent, en- engineers will f- will face it first, and, and will find the right equilibrium. The harder part still remains in the rest of knowledge work.

Speaker 46:55 Yeah.

Aaron Levie 46:55 This is why if you're in accounting, you know, we don't quite yet know when you can take your hands off the wheel, you know, doing a full accounting audit, you know, b- because of AI. What, what you can do is have the AI go and, and tr- like w- comb through unlimited amounts of data to find anomalies that maybe are, that would alert your accounting team to, "Oh, we actually have to go dig into this." That's awesome because that's only ne- new level of visibility versus the part of the accounting process where you're doing a fine-tooth comb on making sure every single number is, is accurate. That's probably still humans right now. So I think the key is where do you find the productivity gains, and I do think that, that if you are a CEO or a board of directors or a management team, you're kind of trying to figure out bec- and you're also at, getting confused 'cause Silicon Valley's telling you all the things. And so you have to sort of figure out where is, where is the productivity most potent, where I actually can get the gain, I can get the success with less of the downside. And I think as an industry we're all sort of figuring it out.

47:53 – 58:22

Will AI Kill Jobs or Create More of Them?

Aaron Levie 47:53 S- by, by the way, this is actually why I remain unbelievably optimistic on jobs.

Speaker 47:58 Yeah.

Aaron Levie 47:58 Because I don't think-

Speaker 47:59 Oh.

Aaron Levie 47:59 You, you, like, I just think we've gotten it wrong on, on thinking w- you know, all the places where you're gonna remove humans from this because you still need a human in that, you know, somewhere in the loop. Maybe the abstraction is a little bit higher and you don't need the human in the loop at every, at every single stage that you needed a year ago, but, uh, but you do need a human sort of kicking off the process, reviewing the process, and incorporating whatever the work was. Um, and so that creates just still a tremendous amount of opportunity and jobs across these organizations.

Speaker 48:27 Oh, let me, I have to jump in 'cause I, I have, I have a whole bunch of, like, visual aids I brought today to make it exciting. We got-

Aaron Levie 48:32 Okay.

Speaker 48:32 You got a bunch of comments on, on the MTS Live thing about people agreeing with you, so I don't wanna let that slide because-

Aaron Levie 48:39 Okay

Speaker 48:40 ... you know, we complain about not agreeing with you. But, but, like, here-

Speaker 48:42 Well, totally agree

Speaker 48:44 ... to your point, to your point, this was a book in the '80s called The End of Work.

Aaron Levie 48:47 Yeah.

Speaker 48:48 [laughs]

Speaker 48:48 And, and I... This, so or actually, sorry, it was in the '90s. It, it, it came out, like, six months before the internet hit, and the whole thesis was the technology revolution was a complete bust and we got no gains in productivity, but now there's gonna be no more jobs 'cause the economy is stagnant. Well, and it just w-

Aaron Levie 49:09 Yeah.

Speaker 49:09 And this was a guy, he called himself a futurist.

Aaron Levie 49:12 Yeah, yeah.

Speaker 49:12 And, and, and, like, so the whole-... notion that it, it, that's a, like one of the neat things about this whole AI moment is, like, the number of things that when you hear them the first time you think they're stupid, and then you go back and think about it and you're like, "Oh my God, it's way stupider."

Aaron Levie 49:27 Yeah. [laughs]

Speaker 49:27 And, and this idea that, like, AI just gets rid of jobs, it's as ancient as, like you p- talk about the accountant. Like, one of the things people thought was that computers would get rid of accountants.

Aaron Levie 49:37 Yes.

Speaker 49:37 And, and that was, like, IBM's pitch in, like, 1965, but-

Aaron Levie 49:41 Yeah

Speaker 49:41 ... what it actually did was like, "Oh my God, we could do so much more with accounting-

Aaron Levie 49:45 Yeah

Speaker 49:45 ... now that they're not, like, literally just adding numbers all day."

Aaron Levie 49:49 Yes.

Speaker 49:49 And, and I think-

Aaron Levie 49:50 Well-

Speaker 49:50 ... when you look at, like, just the notion of, like, creating information, synthesizing and all that, like AI is, is an accelerant for that for a person who knows what they're doing, and companies are suddenly gonna want more of those people creating more of that information. Not to mention the fact that if AI is creating valuable information and there's more of it, then more people will need to consume it s- to do something, and the idea, the essence of a company is acting on information. And, and so this idea that information is just gonna get produced easily and be in surplus and not used makes no sense at all. Because, as you know, like, in the unstructured information world-

Aaron Levie 50:28 Yeah

Speaker 50:28 ... the problem is that you can make it, but the consumption of it effectively is the limiting factor.

Aaron Levie 50:33 They've gotta consume it.

Speaker 50:33 That-

Aaron Levie 50:33 Yes

Speaker 50:33 ... and that's the gating factor now.

Aaron Levie 50:36 Um, we, uh, I, I, I think, um, uh, I, we, I had a conversation with one of our board members who's, uh, chair of our audit committee and, um, and so he's a CPA and, and he was telling us, you know, kind of early in his career, I, I can't even retell it 'cause it, it felt so manual. But, so I don't e- I don't even know how the-

Speaker 50:52 [laughs]

Aaron Levie 50:52 ... world worked b- I don't know how the world worked before all of the modern technology, but he was explaining the CPA's process and I was like, you know, it, it seemed like the most manual thing of all time. But, but an- and Steve, I think this is right out of your book, is, like, it was actually quite simple in, in sort of the amount of things you could do-

Speaker 51:09 Yeah

Aaron Levie 51:10 ... because of, of how undigitized and relatively manual the whole thing was. And computers actually only made it more complicated, more comprehensive, and thus created even more jobs because, because of that complexity that, that we introduced. And, and you can just sort of see how easy this is to show up in so many areas of work is, like, we can just now, we can afford to make things more complex. And so if, if you make things more complex, then actually you eventually still run into now new constraints of who can understand that complexity. And, and, and, and so, like, you know, it, it's like to me it's, like, the funniest concept that the more code we write, the less we would need engineers, it would be the opposite. Because, because now your systems are even more complex than before, which means that you're gonna be running into even more challenges of when you need to do a system upgrade or when there's downtime and you have to figure out, like, what, well, how do I fix that problem or when there's a security incident. Uh, it, and so yeah, I mean, this is-

Speaker 52:03 I-

Aaron Levie 52:03 ... this is, uh, this is, like, we're just getting started with the jobs on this run.

Speaker 52:07 Right. And al- listen, w- y- we're, we're a few years actually into this, and you can actually look a bit at the data too, right? Like, what are the companies that are hiring the fastest? Like, the AI native companies, and they're hiring like crazy.

Speaker 52:17 Yeah, yeah.

Speaker 52:18 But not only that, like I remember there was this early prognostication which is AI writing code will get rid of infrastructure. Like, it's gonna commoditize infrastructure-

Speaker 52:26 Mm-hmm

Speaker 52:26 ... and, like, whi- which is this kind of v- very strange, um, prediction given the fact that there's more software than ever before been written.

Speaker 52:33 Right, right.

Speaker 52:34 And sitting on the board of a bunch of infrastructure companies, some that have been flat for a while, they're all doing fantastic [laughs] 'cause there's so-

Speaker 52:40 Right

Speaker 52:40 ... much software. There's so much more software out there now. And so listen, if you look at the data on the ground from the companies, it's more software. The AI native companies are hiring the, the, the, the, the most, and so it's very clear to me that we're in an expansion phase.

Aaron Levie 52:53 A- and, and the, maybe just my, my only final point on this one at least is, um, is I think people, we, we have a little bit of a myopic view in Silicon Valley on, on thinking that, you know, engineering jobs are, you go to work at, at Google or name your, your, you know, tech company and startup, and that, that's an engineering job. And, and we're, we're so wired into that because-

Speaker 53:14 Yeah

Aaron Levie 53:15 ... that's obviously the ecosystem that, that we're all a part of. Um, and then you sort of forget, well, like John Deere is trying to make automated tractors and Caterpillar is trying to have AI systems and Eli Lilly is trying to design even more pharmaceutical, you know, kind of, you know, therapeutics. Um, and, and just you can go through 5,000 other companies. They're gonna now have the next set of engineers that are gonna use Claude Code and Codex and Cursor to be able to automate even more of, of their businesses and be able to d- design and develop even more software for their workflows and their systems. And so it just might n- you know, be that, that you don't go and work on a social network and improve the social network algorithm. You go work at John Deere and you improve the, the, you know, intelligent farming algorithm. And, and that, and, and we just have to, you know, I mean, this is sort of like, like completely, you know, uh, like Marc Andreessen predicted this, you know, 15 years ago, of like software's gonna eat the world. And what that means though is that everybody's gonna have lots of software, and this gives everybody the ability to finally have lots of software, but you still need then an expert or a semi-expert to be actually going and prompting the, the, the, you know, the agent on what to do, reviewing its work, and managing the system that it builds. So, so all of the, you know, predictions on don't go into coding and don't go into software engineering I think will be proven quite, quite wrong.

Speaker 54:31 I think, I mean-

Speaker 54:31 That was so good

Speaker 54:32 ... look, it, it, that was super good, Aaron, and I think that, that the, the, the base case of all of this is just that it, there's too many people out there right now that don't like technology and have a static view of the world. So when they look to whatever it is that they think AI is gonna do, and people hear automation, they just assume it's gonna take things away. Like, here's a-

Aaron Levie 54:53 Well, we have a lot of people who like technology though that are also creating that, uh, that [laughs] so.

Speaker 54:57 Right, right. So here's, like, this is a article, Fighting the Paper Chase.

Speaker 55:01 Lower it, lower it, lower it, lower it.

Aaron Levie 55:01 Lower it, lower it, lower it.

Speaker 55:01 Well, I'm looking at a feed.

Aaron Levie 55:03 Even, even lower.

Speaker 55:04 I'm looking at a feed.

Speaker 55:05 Lower.

Speaker 55:05 What are they doing?

Aaron Levie 55:05 Oh, you're looking at a feed? Okay. Oh, sorry, okay.

Speaker 55:07 Okay.

Aaron Levie 55:07 Okay.

Speaker 55:08 Oh, I see. Oh, you're looking at my Mac camera and-

Aaron Levie 55:11 Yeah, yeah.

Speaker 55:11 Oh, that's why.

Speaker 55:12 Oh.

Aaron Levie 55:12 Oh, you're fancy.

Speaker 55:13 That's what it is.

Aaron Levie 55:13 You're fancy.

Speaker 55:14 Right.

Speaker 55:14 Oh.

Speaker 55:14 So this is, like, Time Magazine, every kid in high school read it, 1981, and but the whole view of what computers would do would be they would automate the paper in a company.And so the idea, like the whole first generation of computing was literally taking paper forms and turning them into something on a screen, then printing them out and-

Aaron Levie 55:35 Yeah

Speaker 55:35 ... then making it all easier.

Aaron Levie 55:37 [laughs]

Speaker 55:37 And you fast-forward and it's all of these things that you just said, Aaron. Like, you know, there was an era when lawyers didn't type, and so what happened was they just, they, they had people who were legal assistants, they called them paralegals, and they did all the typing. And then, like some students at Harvard, they brought a computer into the classroom. And so this is... I'm lowering it so you guys can see.

Aaron Levie 55:56 Yeah, yeah, yeah, yeah.

Speaker 55:57 So they brought, that's a original laptop in there, in the early 1980s. And they brought-

Aaron Levie 56:02 Lovely

Speaker 56:02 ... they brought this computer into the classroom, and then they got thrown out for using it.

Aaron Levie 56:06 [laughs]

Speaker 56:06 And, but they were literally, they used to, used to do law school and you'd write the essays in longhand in a book, and then the professor would have to read them. And now, of course, you just type them, and you have access to the database of all the citations, but that's exactly... Like, nobody deals with a lawyer who isn't in track changes with their, with your contract.

Aaron Levie 56:24 Right.

Speaker 56:24 And, and I, last I checked, there are way more lawyers today than there were 30 years ago.

Aaron Levie 56:30 Yep.

Speaker 56:30 And they all are, every human lawyer you talk to is a computerized lawyer. Their citations come from, from the internet, their, their information and the brief comes-

Aaron Levie 56:40 I-

Speaker 56:40 ... and they type the brief

Aaron Levie 56:41 ... and, and I think, I, I think we, we, the, the, you know, kinda going back to the my- myopic approach, I think we may be over, like, I mean, w- uh, as a big lover of technology, I, I wish this was true, but I think we just over assume that, like, everybody's job is, is just, they're just inside a Microsoft Word, and they're just typing a, a Word document. It's like, like, I mean, most of the time with lawyers I'm like, you know, strategizing something or, or they're like-

Speaker 57:05 Yeah, yeah

Aaron Levie 57:05 ... working through a, a, a complex analysis of a situation. Um, and it's not s- like, I could go to an AI for, for advice, but, and, but that would probably only increase the chance that I go and then call a lawyer to say, "Hey, what do you think about this, this situation that, that, you know, that I'm, that, that we're dealing with?" Um, and so a lot of these jobs just have a lot of context that aren't sitting just, you know, literally on the, on the computer doing all the work. They, they do have to kinda touch grass, um, as a part of the job. And, uh, and so then AI, yeah, AI will help automate the creation and production of the content and the review an- uh, of, of the information, but then it still has to be incorporated into the real world of, of real value production.

Speaker 57:46 I, I feel like we're live, and we're supposed to end at 4:00, so what-

Aaron Levie 57:49 Yeah, I was gonna say [laughs]

Speaker 57:50 ... I'm gonna, I'm gonna do is just say we're live and it's 4:00-

Aaron Levie 57:53 [laughs]

Speaker 57:53 ... and I guess that means we just stop and some lights fade or something. None of us have done this before. We don't know what's supposed to happen. But-

Aaron Levie 57:59 [laughs] That's right

Speaker 58:00 ... someone is waving at me and smiling saying, "Yes, I think you're right." Uh-

Aaron Levie 58:03 The smile is, the smile means stop talking? Okay. All right.

Speaker 58:06 All right. Well, it was great to see everybody.

Aaron Levie 58:07 See ya.

Speaker 58:07 Bye, everyone. [upbeat music]

Episode duration: 58:23

In the uListen app, the transcript lights up word by word as you listen. Get uListen

Transcript of episode dvVbA9OcBqs

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.