a16zDecagon’s Playbook for Building Enterprise AI Applications
EVERY SPOKEN WORD
85 min read · 16,661 words- 0:00 – 1:07
Intro
- JZJesse Zhang
An AI agent should just be the front door of your business, and every interaction, whether it's like reactive or proactive with a customer, should be handled by AI.
- SWSarah Wang
This narrative dominated the first half of 2026, which is that Anthropic, OpenAI, they're the last startups. They're gonna take over everything.
- ASAshwin Sreenivas
Even once you have AGI, agents are going to need somewhere to store work and pull information from and reason about things. I don't think software as a whole in any meaningful way is going away.
- JZJesse Zhang
Unfortunately, the frontier labs, they do have small models, but you can't really control them in the way that you want. So today, 90% of our workflow is on open source.
- ASAshwin Sreenivas
On the specific tasks we want them to do, they actually outperform the large, smart state-of-the-art models.
- SWSarah Wang
Mm.
- ASAshwin Sreenivas
The thing that we built was not an agent that does customer support well, but rather an agent that follows business process well.
- JZJesse Zhang
Instead of us having to write these AOPs, Duet just does all of that.
- SWSarah Wang
Let's say like we hit AGI and the models can do all sorts of things we can't even imagine today. What's Decagon's moat, and like why does Decagon, like 10 years from now, still have a right to exist?
- 1:07 – 5:00
Decagon's Journey from Frontier APIs to 90% Open-Source
- SWSarah Wang
Hey guys, welcome back to the studio.
- ASAshwin Sreenivas
Thanks for having us.
- JZJesse Zhang
Yeah, good to see you.
- SWSarah Wang
Thank you for being here. Before we get into customer support, um, I actually wanted to widen out a bit. Um, and, uh, Jesse, I'm gonna actually, um, mention a piece that you wrote recently that went pretty viral because it's right in the middle of the zeitgeist of conversation right now on open source versus closed source models. And then Thinking Machines, um, Kimi K3, you know, some of the, some very interesting open source models came out sort of right after. Um, and there's this really interesting debate going on around what does it mean to own your destiny when it comes to AI, especially in the enterprise, um, and what does that evolution look like by use case? Um, so actually, since that's a pretty live topic right now-
- JZJesse Zhang
Yeah, yeah
- SWSarah Wang
... why don't we start there?
- JZJesse Zhang
Yeah, sounds good. Um, so I'm gonna talk about our journey first, uh, just to make it very concrete for people. So when we started the company, uh, the goal was just to get something working, right? So if, when you get something, when you're... The goal is to get something working, of course, you're just gonna use the frontier models 'cause you wanna s- get something out there and have it actually deliver value, and so we were using OpenAI Anthropic. At that time, they were kind of like one-upping each other in terms of how the, how the models performed. And then at some point, as we got to larger scale, you know, we started working with larger and larger companies, and they had, you know, millions of, uh, customers, and then we, we also have, uh, we also launched our voice agent, right? So a, a big factor became latency. So it wasn't just like, can you deliver good responses? You have to deliver them really fast. And, uh, the only way to get latency down, um, but also kind of, you know, make our agent operate the way we want it to, is to use smaller models. And [clears throat] when you wanna go to smaller models, unfortunately the, the frontier labs, they do have small models, but you, you can't really control them in the way, the way that you want, and most small models out of the box are not gonna be good enough at the task that we want them to. So you have to fine-tune them, you have to change them, and so that's when we started looking at open source. So this was about a year plus ago. And, um, I mean, it, it worked really well because if you think about it, in the agent, right, so in our agent, our agent's job is to have conversations, so it needs to do a lot of things at once, right? Uh, like one, one, the first step it might do is like, "Hmm, what topic is this person talking about?" Or, and something else it might do is, "Oh, is this person a bad actor that's coming in and trying to mess things up?" There's all these like tasks it has to do. Each individual task doesn't need all of the intelligence of a big model. So, you know, all the frontier models are obviously very smart, but they can do a bunch of different things. Like they can do math, they can do coding. Like, you just need them to be good at that one task, and so that's why you can use a smaller model, and if you fine-tune it to be really good at that task, it can be just as good or better than the big models, right? So that was, that was step one for us. You know, about a year ago, we were like, "Okay, let's start using these open source models." Uh, we, we, we took the small ones, and then, um, you know, that's why we have now a, uh, a research team, and it's a very expensive team. [laughs] But it's... We, we have it because, you know, we need people that are really good at taking these open source models and, and tuning them and, and so on. So today, 90% of our workflow is on open source. And, um, you know, the, again, the main reason was for latency, to really optimize our voice agents. And, um, I think we've just over the last year, we've seen tremendous improvement in like how, how it sounds, how it feels, and, but still also like keeping the accuracy high. Um, and then the remaining 10%, of course, we're still using the, uh, the closed source models and the frontier models for a lot of, you know, new, new projects or new products. And, uh, I think that's just where the industry's moving to. So if you kind of were to generalize this, every model you can kind of evaluate along three dimensions. It's, you know, costs, intelligence, and, uh, latency. And depending on what you need, you wanna kinda be at the limit of those three, and sometimes you can trade off, right? So in our case, we knew that we could actually pull back on intelligence 'cause we, it all had to do with that one task, but now we get these latency advantages.
- ASAshwin Sreenivas
I wanna push
- 5:00 – 9:26
The False Trade-off: Why Fine-Tuned Small Models Win
- ASAshwin Sreenivas
a, a, a tiny bit on that point actually, because, um, oftentimes, you know, when you see these debates being had on Twitter, the, the trade-off tends to be, "Oh, do we want, uh, you know, the smartest model that is very expensive, or can we like dumb it down a little bit and get it cheaper?"
- SWSarah Wang
Hmm.
- ASAshwin Sreenivas
I actually think that is a false trade-off.
- SWSarah Wang
Hmm.
- ASAshwin Sreenivas
Right? Because what we've seen-
- SWSarah Wang
Yeah
- ASAshwin Sreenivas
... in practice is even if you have a, quote, "dumber model," you can get it, and we've seen this in practice, you can get it to higher performance on that specific task. So when we fine-tune smaller, dumber models, it's that they're just not as general purpose, but on the specific tasks we want them to do, they actually outperform the large, smart state-of-the-art models.
- SWSarah Wang
Hmm. Yeah.
- ASAshwin Sreenivas
Right? So we end up getting all three things. It is better at the task, it is cheaper, and it is faster.
- SWSarah Wang
And so do you feel like today, like you need the most frontier models for really anything at Decagon? Because your performance is very good already, so...
- ASAshwin Sreenivas
We, we do, and we often need them, we often end up needing them for auxiliary tasks, right? Where when you have, uh, when you sort of have auxiliary models to our sort of primary conversational flow, right, you have an agent and it's helping a customer with their rebooking, or it's helping them with, uh, a process in healthcare, then these are like well-defined paths, so we have smart, fast models to do that. But we've- For instance, recently launched Duet Autopilot, right? Which is our agent that improves the core conversational agent. Now, for something like Autopilot, it is doing a very complicated job, right? It's saying, "I'm gonna go and review a million conversations that just happened. I'm gonna try and find trends. I'm gonna create variants of the primary model and see which of those variants does better." So now this is a much more broad, open-ended exploratory task. So we think for jobs like that, frontier models that are very smart, that can try out a lot of things make a lot of sense.
- SWSarah Wang
Hmm. Do you think that, um, g- I mean, you guys obviously, and you referenced it, um, Jesse, that you have a research team, right? You guys launched Dec-Decagon Labs, but even before the formal launch, it's always been a part of your culture. Um, do you think that enterprises will get there as well on post-
- JZJesse Zhang
Yeah
- SWSarah Wang
... training open source models? Oh, okay. Interesting.
- JZJesse Zhang
Um-
- SWSarah Wang
What's the timeline?
- JZJesse Zhang
Yeah, I think, I think they'll get there, but it'll probably take longer than people think-
- SWSarah Wang
Hmm
- JZJesse Zhang
... because, you know, even with our team, fine-tuning these models is non-trivial. It's not just like, oh, you can... It's like, all right, we made the decision to use open source, like let's just use open source. Like you have to get the data, and then more importantly, you have to like have good evals. Um, and if you think about our evals, right, our evals are very specific to us. You can't just like use some public eval set and like that, that just does the job. It's like we're testing it on our task, and so we have to generate our own benchmarks and evals. Uh, but I think the point is that, um, at a certain point, it's strictly better to use open source models because when your use case is solidified and you're in production at scale, and you're, you're pretty sure this is the, the sort of shape of the agent, then there's no reason not to use open source, 'cause you, you get these latency benefits, and at the same time, you get the cost benefits. I mean-
- SWSarah Wang
Right
- JZJesse Zhang
... again, we didn't do these for cost benefits, but that's like a nice side effect, right? And once you're there, it's like why, why use frontier for that? But for everything that's new and sort of experimental or you're, as Ashwin was saying, what... Or like kind of these products where you really need the intelligence, you're still gonna use frontier models. And it's just so much easier to use frontier models. You don't have to worry about the infra, you just like, like they just, um, you just use the APIs. And I think that's why in enterprises right now, even though there's l- there's a lot of hype for open source, the sort of share of, of open source inference is actually going down right now because-
- SWSarah Wang
Hmm
- JZJesse Zhang
... people are spinning up all these new use cases, and if you're spinning up new use cases, of course, you're gonna use the frontier models-
- SWSarah Wang
Hmm
- JZJesse Zhang
... until, until they're working.
- SWSarah Wang
Yeah.
- JZJesse Zhang
But of those use cases, you know, some might die off, but like some might, like the enterprises are like, "Okay, great. We wanna keep shipping this and like roll it out." Once it's at that point, they're heavily incentivized to use open source-
- SWSarah Wang
Hmm
- JZJesse Zhang
... 'cause it's way cheaper and faster. At that point, they'll... maybe they can do it in-house or maybe they'll need help, uh, from people to, to help them fine-tune it, but that, that'll eventually happen. I just think it'll be kinda slow.
- SWSarah Wang
Hmm.
- 9:26 – 15:07
Decagon Labs as a Model Factory
- SWSarah Wang
Yeah.
- ASAshwin Sreenivas
The, the, the other reason I think it makes a lot of sense for, uh, enterprises to kind of build their, you know, equivalent of labs, is that the shape of these models is changing constantly.
- SWSarah Wang
Hmm.
- ASAshwin Sreenivas
Right? We don't just build our set of open source models and then, you know, it's done, we can move on to our next thing, and maybe we'll revisit this in two years. You often need to train new models all the time.
- SWSarah Wang
Hmm.
- ASAshwin Sreenivas
Because-
- SWSarah Wang
Yeah
- ASAshwin Sreenivas
... as the frontier changes, as the capability of the models changes, you come up with new use cases for them. You find new places where you're like, "Oh, this task seems to be getting repeated a lot," because now I have this totally new like, you know, frontier model or open source model that now has this capability that they didn't have before, right? So we find ourselves constantly training n- new models and deprecating old ones that are no longer relevant-
- SWSarah Wang
Hmm
- ASAshwin Sreenivas
... because, you know, maybe the frontier has advanced a lot, the open source frontier has advanced a lot, and, you know, the model out of the box can do a lot of things that it couldn't do before. So we've-- Because the model landscape is changing so quickly, uh, Decagon Labs is, in a way, a, a model factory-
- SWSarah Wang
Hmm
- ASAshwin Sreenivas
... of sorts, right?
- SWSarah Wang
Yeah.
- ASAshwin Sreenivas
We've really built it to, um, like, uh, compress the time between new model coming out and, you know, useful fine-tuned to our task model kind of popping out the other end.
- SWSarah Wang
Yeah.
- ASAshwin Sreenivas
Just because-
- SWSarah Wang
Makes sense
- ASAshwin Sreenivas
... this happens all the time.
- SWSarah Wang
How do you guys think about what to in-house from a talent perspective versus... There's also a pretty broad ecosystem right now that is, you know, could be RL as a service, evals, et cetera. Like w- how do you guys... What's your framework for, hey, this is mission critical, and we need to do this best, versus, yes, it'd be great to, you know, outsource this?
- ASAshwin Sreenivas
Um, you know, in practice, we've seen that so many things, uh, relevant to model training are so tightly coupled to the use case that we have.
- SWSarah Wang
Hmm.
- ASAshwin Sreenivas
Right?
- SWSarah Wang
Mm-hmm.
- ASAshwin Sreenivas
That we find that we end up needing to build a lot of tooling internally. W- when we have, um, open source, uh, uh, open source models that we wanna fine-tune, we find that if we can clearly tailor our evals to customer outcomes, it's way better than just looking at like loss curves over time, right? We're not just saying, "Oh, can I do this one specific task?" We're measuring the entire system end to end. We're not just saying, "Is this model good at this task?" We're saying, "Is this model working in concert with all of these other models, delivering the end customer outcome that we care about?" And because that is so unique to our setup, we've found that in practice, we've needed to build a lot of, uh, uh, a lot of the infrastructure that we need to train these models and evaluate them. Now, for other things like getting labeled data and measuring the diversity of our data sets, we're like, "Yep, these are tasks that-
- SWSarah Wang
Yeah
- ASAshwin Sreenivas
... are common across lots of companies," in which case, we want to buy things, uh, from other vendors because that'll just help us get those models to production faster. Ultimately, the only thing that we care about is how can we get the best model to production as quickly as we can.
- SWSarah Wang
And so it sounds like, you know, there's a lot of talk right now about like tokenomics and how expensive it is to actually run a lot of these models. Based on this conversation, it doesn't sound like you guys spend that much time actually thinking about the costs o- of these models. Is that correct? It's really about the performance for you?
- ASAshwin Sreenivas
Performance, uh, latency, and Accuracy is definitely the driving factor for most of this, right? Cost is a nice benefit in that, you know, uh, surprisingly, this is one of the few like tasks where you kinda get all the things for free, right? Like we, we don't actually have to trade off cost and latency and performance, and so just by optimizing for the, the driver's actually latency and performance, and we just get cost as a nice side benefit.
- SWSarah Wang
And do you think that's like something that's unique to the way Decagon is run, or do you think there's something about like this conversation-
- ASAshwin Sreenivas
Mm-hmm
- 15:07 – 21:21
Are Frontier AI Labs the Last Startups?
- ASAshwin Sreenivas
sense.
- SWSarah Wang
I just wanna make this meta observation that a lot of the conversation even just now has been about things like training models, right? Uh, we're talking about reinforcement learning, and, um, it really does blow away what I think is, uh, you know, a lingering misperception about what an AI application is. Um, and just to get into that debate a little bit, because I think it's sort of this narrative that dominated the first half of 2026, which is that Anthropic, OpenAI, they're the last startups. They're gonna take over everything. Applications, they're thin UIs with FDEs, you know, with implementation attached to it. Um, you know, we talked a little bit about Decagon Labs, but can you guys just share what is your... Like how do you think about this potentially false dichotomy of app versus infrastructure company? Um, clearly you guys are so much more than the UI or the implementation. Um, and, uh, you know, does Agent Lab, which, you know, popular, uh, description, um, floating around the last couple weeks, like does that describe it? Like how do you guys think of Decagon?
- JZJesse Zhang
Um, so I'll, I'll give a quick perspective just like from the, from the POV of like an enterprise, and then maybe we can talk about like broader the industry. So let's say I'm like a Fortune 100 company, right? And I'm looking out there on all my use cases, and I have a choice of partnering with an application company or, uh, sort of using the labs and building from scratch. Um, I think there is a lot of merit to partnering with the labs in certain cases. I think if you look at our case, right, like we, we just talked about all this fine-tuning stuff. Uh, I think a common misconception that people have is, you know, uh, fine-tuning is, is a way to like customize it for that customer. In fact, most of the fine-tuning we do is like customizing it for our use case, like the customer service use case.
- SWSarah Wang
Yeah.
- JZJesse Zhang
And it's worth it for us to do it because that's all we do, right? We do these agents across all of these different customers, and so it, it is worth it for us to put in a ton of time and research into like, how do you tune this one model to be good at selecting customer service topics? But if you're the enterprise, is it really worth your valuable research resources to like tune a model for these like customer service behaviors? Probably not, right? So that, that's like, that's one reason why people, um, partner with applications. Another reason is, let's say I do put in the engineering effort to build like a, some agent myself using the frontier models. Um, and, you know, to, to my earlier point, you know, I'm not fine-tuning for behavior, but I'm, I'm sort of teaching the AI my own procedures. And again, that doesn't happen through fine-tuning. That, that happens like in context, because if you were to fine-tune on that, you would have to reverse it every single time, you know, you change your procedures, which doesn't make sense. And so you kind of build out your logic, and you're building your business logic in. Well, then you launch the agent, and then the second day you look at your conversations and you're like, "Oh, well, actually I need to change these three things."
- SWSarah Wang
Mm.
- JZJesse Zhang
And now it's more engineering effort to do that.
- SWSarah Wang
Right.
- JZJesse Zhang
And it's like constantly engineering effort. So I think people will partner with applications when the use case calls for a broader platform, where there's a lot of value in using, you know, the stuff that we've fine-tuned and using the, the software stack we've built on top of the models to capture business logic. And that stuff has nothing to do with the models, right? Like how, how the business logic gets captured by this AI, like how do you handle someone calling in because, you know, their flight was canceled, and they need to rebook three people at once. It's like that is business logic that the AI needs to know, and you're encoding that, but that has nothing to do with the models themselves. And so that, that has to exist in the application layer. I think that's where applications will still shine because you still need the application there, and it's not so much the models. And the labs themselves will have more application capabilities.
- SWSarah Wang
Mm-hmm.
- JZJesse Zhang
But those will be fairly general. Like they're... You can maybe build general agents that can do this thing or that thing, but- For a lot of these like core verticals like ours, our, our, our thesis is that, you know, you're gonna need something that's like very deep and has all the integrations, has all the ability to capture business logic, has the ability to run, you know, tests and experiments, and then review the conversations and run QA and like, you know, have tooling for your compliance team to monitor like what's happening. So that, that's our thesis on it. And so it- it's kinda... it's, it's not black and white. Like there will be some use cases where it does make sense to use the, the frontier models, but there will be these like core verticals where going super deep makes sense.
- KTKimberly Tan
Yeah.
- ASAshwin Sreenivas
I also think, you know, everyone is kind of bleeding into everybody else's space a little bit, right?
- KTKimberly Tan
True. Lots of convergence. [chuckles]
- ASAshwin Sreenivas
All the, all the labs are building applications on top of it because rightly so, they're saying-
- KTKimberly Tan
Mm
- ASAshwin Sreenivas
... this is how the enterprises adopt us more, right? And this is how the enterprises see ROI from using our products. Uh, us on the application layer, we're realizing that, hey, we can squeeze out a lot more performance and latency and cost for the use cases that we care about by building our own models, right? Which... And, and I think this, uh, this split, uh, th- this kind of bleed over makes sense, and I think it'll continue. I'm not as bought into the, the labs are the last startup view of the world, though.
- KTKimberly Tan
Yeah.
- ASAshwin Sreenivas
And I think this is true both for SaaS companies and for the new AI startups.
- KTKimberly Tan
Mm.
- ASAshwin Sreenivas
Because in a way, we, human beings are kind of AGI, right? And human beings have needed to use software for lots of things. You know, you need databases to put stuff in. You need CRMs to track things. And I think even once you have AGI, all our AGI agents are going to need somewhere to store work and pull information from and reason about things. So I think, you know, a, a certain class of SaaS companies that were solely built for people to do work might face a bit of heat.
- KTKimberly Tan
Mm.
- ASAshwin Sreenivas
But I th- I don't think software as a whole in any meaningful way is going away.
- KTKimberly Tan
Mm.
- ASAshwin Sreenivas
And I still think, you know, on the application layer, there's so many different kinds of work that, that need to be done, that can be done faster, more efficiently, more cheaply. Uh, so I think there will always be a, a space for application layer companies. Maybe in the long term, application layer companies just become labs for specific verticals, you know, because your primary product ends up being the models that are just really good at doing those specific tasks. But I think the application layer is probably here to stay. Uh, I also wanna kind of pick on another thing-
- KTKimberly Tan
Mm-hmm
- ASAshwin Sreenivas
... that you kinda said in passing.
- KTKimberly Tan
Yeah, please.
- 21:21 – 28:36
The Forward Deployed Trap: Product vs Consulting Truck
- KTKimberly Tan
[chuckles]
- ASAshwin Sreenivas
Yeah, and, and maybe this is like a, a, a spicier take of, oh, are application layer companies just forward deployed companies that are kinda doing the last mile of work?
- KTKimberly Tan
Yes. Right. Well, well, I was saying that's a misperception.
- ASAshwin Sreenivas
Yeah.
- KTKimberly Tan
But I think it is a common one, yeah.
- ASAshwin Sreenivas
I think it's a really hot thing also, again, not to pick on tech Twitter to be like, oh, like, you know, we need to bring back the forward deployed engineer and, you know-
- KTKimberly Tan
Very hot
- ASAshwin Sreenivas
... every company is hiring tons of forward deployed engineers. Uh, I think this is a trap, actually.
- KTKimberly Tan
Mm. Oh, say more. Okay. [chuckles]
- ASAshwin Sreenivas
My, my view on this is that forward deployed engineers are necessary, uh, are newly necessary for early-stage AI companies because the workflows are new, right?
- KTKimberly Tan
Mm.
- ASAshwin Sreenivas
If you're building a SaaS company five years ago, most SaaS products are pretty well explored, right? Like you roughly know what the user's trying to do, and your job is maybe come with a slightly cleaner workflows. But broadly, you know what the user's trying to do with a design app or a CRM or something like that because those workflows have been explored. With AI products, nobody knows what the workflows are-
- KTKimberly Tan
Hmm
- ASAshwin Sreenivas
... because nobody's used these things before. So a forward deployed engineer in this case is honestly just embedding with the customer to learn the workflow for the first time as the customer learns the workflow for the first time, and they're kind of, you know, kind of b- paving the road, like, you know, they're kinda laying out the track-
- KTKimberly Tan
Right
- ASAshwin Sreenivas
... as they see which way the train is going, in a way. Uh, but long term, I think they should just be building product, right?
- KTKimberly Tan
Mm-hmm.
- ASAshwin Sreenivas
Like once you know what the workflow is-
- KTKimberly Tan
Yeah
- ASAshwin Sreenivas
... you should not be relying on forward deployed engineers anymore, because once you know what the workflow is, if you can productize it, you should productize it and then become, you know, a typical company with the scaling properties of a tech company.
- KTKimberly Tan
Yeah.
- ASAshwin Sreenivas
And if you can't do that, then you're just building a glorified consulting firm.
- KTKimberly Tan
Well, I'd love to dig into this more 'cause I know when we started working together, probably almost exactly three years ago, and you guys landed on this idea, there were two things that were relatively contrarian that now feel kind of standard. The first was when you started an AI customer service, a lot of people were like, "That's a G- GPT wrapper," and we talked a lot about like why that's not the case. And then the second thing you did was say like, "Hey, we actually wanna do the work ourselves. Like, we don't wanna just be a software platform." And now we have the term for that, that's like AI agents and everything, and you've popularized agent PMs and forward deployed as a new type of role or a more common type of role, um, in Silicon Valley. But your like agent PM/forward deployed team has actually evolved a lot since you first got started to now, and so would love to talk about that. Like in the early days, what did it actually look like? And then as you started to learn about these workflows and were able to productize them better, um, how has that function actually evolved for you guys?
- ASAshwin Sreenivas
Yeah. If you look at a lot of our forward deployed teams, they are building product in one way or another, right? So all of our forward deployed engineers, for instance, build core product, right? Their job is to go in and understand, as we're working with an enterprise, what are the things that they need that the product does not do today. But the output of that is not a one-off thing that is just built for that customer. It is something that is contributed to core product in a way that the next 10 customers that ask about the same thing-
- KTKimberly Tan
Mm
- ASAshwin Sreenivas
... get it for free, right? Uh, similarly, our agent PMs are working with our customers to understand, you know- How is the product broken today? How can this actually be deployable within an enterprise? What are the new things that we need to build to our core product to make it deployable within an enterprise? Um, so at the end of the day, all of this boils out, boils down into product improvements, either through actual product improvements or through ca- honestly process improvements, right? Like we work with very, very large enterprises, and we help them through the journey to go from, "Okay, this is what your org looks like today. Here is how we can take you through changing your processes, through implementing new technology into this world where AI agents are doing a lot of work for you." So a lot of their product work is also processizing a lot of, you know, how we make that, how we help a company through that transition.
- SWSarah Wang
And Ashwin, you used to be a deployment strategist at Palantir, so you're very familiar with the forward deployed model that Palantir popularized. How different is that at what you did at Palantir versus the way you conceptualize this role at Decagon? 'Cause I do think people use the term FDE very loosely in Silicon Valley.
- ASAshwin Sreenivas
Yeah. Yeah, and, and I think it's, it's dangerous to mix the two of-
- SWSarah Wang
Hmm
- ASAshwin Sreenivas
... free consulting work versus actually doing product. Uh, you know, Shyam, um, who's the CTO of Palantir today, had, had a phrase, um, internal, I think now it's been written about a ton. He would say, uh, "Forward deployed engineers eat pain and excrete product."
- 28:36 – 37:02
Duet Autopilot: The Agent That Builds the Agent
- SWSarah Wang
lab. And one of my arguments was that there was a good long-term career here, and the response, uh, that this person gave me was, "We'll have AGI. We don't need careers in the long term." Um, it, it really hit me 'cause I thought I was AGI-pilled, but I had really not thought about from that perspective. Um, and I'm curious, you know, we talk about the product improving, right? Can you make that concrete for us? Like h- what are some of the oh shit moments as you improve your product for your customer, either from your end or your customer's end, if you can share, on like, "Oh my God, I didn't realize AI could do that"? And then, of course, I'm going to ask you your thoughts on AGI and what that timeline looks like. 'Cause you're in the nitty-gritty trenches of the enterprises using AI, so you may have a different perspective than, you know, we do.
- JZJesse Zhang
The first thing I wanna say is, uh, I'm, I'm like certain there will be careers after AGI.
- SWSarah Wang
[laughing] Okay.
- JZJesse Zhang
The reason for that is like most of our jobs-
- ASAshwin Sreenivas
We'll see.
- SWSarah Wang
Yeah. [laughing]
- JZJesse Zhang
... for sure are for jobs, are kind of like made up to begin with.
- SWSarah Wang
[laughing]
- JZJesse Zhang
Most jobs are made up.
- SWSarah Wang
Speak for yourself, Jesse.
- JZJesse Zhang
Like unless-
- SWSarah Wang
I have a very real job
- JZJesse Zhang
... unless you're like building infrastructure or like growing food or something, it's like, like most jobs are kind of like layers of abstraction built on top of like other stuff, right? And that, that isn't to say like the jobs aren't valuable. It's just they're kinda made up. So when AGI is here, like it'll change people's jobs, but every- people will still have jobs 'cause you're still gonna do things for other humans and, and whatever. So I, I don't really believe that careers will be gone after AGI. I don't think people are just gonna be sitting around. I'll say one observation, which is like for me, it was definitely, um, Duet. So early days when we were building the product, right, like the, the core problem we're solving, again, being back to sales-led, is like we talked to a bunch of customers and they're like, "Yeah, the value we wanna get out of what you're building for us is that, you know, we can put it in front of customers, they can have conversations, and it's giving them a much better experience, and also it's like, you know, way easier for us operationally, right? It's like you're saving cost and you're making customers happier." So that's, that's like the first agent that we built, and that was like the core agent we, we worked on for the first like year, year to two years. And the agent on our end when we were building it consisted of a ton of stuff. Like we would have to, um, you know, write these procedures, and we kinda created our own format of procedures. We call them agent operating procedures that teach the AI how to do things. We have to write these tools that the procedures can use to access systems and pull APIs and whatever. And then after that, we need to like create all these tests to make sure that this thing is working well and that you can kinda simulate all these different situations. And afterwards, once they're in production, like we would manually be reading conversations, right? There's like a ton of work that goes into it, even though the core product we're building is, is itself an agent. And so what Duet is, is it's kind of a separate agent. It's like a second agent that's much bigger and much slower, but its job is to do all the tasks I just described. So now, instead of us having to write these AOPs and write these integrations and tools into their systems and write these tests and monitor the conversations, Duet just does all of that, right? So it's like a, it's a second agent that is smart enough to do all of these things. And, um, it's, it just feels very magical 'cause you can just literally tell it like, "Hey, I've, I have nothing built yet so far, but here's a bunch of transcripts I have and here's some documentation. Like, you go figure out the best way to like do these, all these procedures I want."
- ASAshwin Sreenivas
Hmm.
- JZJesse Zhang
And it'll go do it, and then of its own accord, it'll also write the tests and simulations that go along with those. And once you're done with that, and you actually put it in front of customers, it'll be the one that's monitoring all the conversations, and it'll flag things where things are going well or poorly, and it'll say like, "Yeah, I read, you know, these 1,000 conversations, and actually there's this one topic that we do really poorly on."
- ASAshwin Sreenivas
Hmm.
- JZJesse Zhang
"And I've noticed that, and I've also drafted these improvements for you," right? So it's like-
- ASAshwin Sreenivas
Wow
- JZJesse Zhang
... it's like one agent that can do all of that. And so that, that's very magical because, well, first of all, it was not possible when we first started the company. It only became possible when all the reasoning models got better. And of course, Anthropic, OpenAI are making these reasoning models mostly for like the Claude Codes of the world. But they are also really good for like Duet, for example. And that was kinda like a moment where it's like, oh wow, like first of all, you can just see the improvement over time of the models.
- ASAshwin Sreenivas
Mm-hmm. Yeah.
- JZJesse Zhang
And two, it can do all these tasks where it just would not be able... You, you would not have expected AI to be able to do all these tasks at once, but it can, it can do them very well. And, um, I think that's, that was like a very visceral moment of like, okay, wow, the models are getting a lot better, and they're becoming like very generalized. You know, like they can do all these things. Like clearly the models were not trained on like our specific task-
- ASAshwin Sreenivas
Right
- JZJesse Zhang
... which is, you know, writing these procedures and, and writing these tests, but they're still good at it.
- ASAshwin Sreenivas
And also, um, to your other question of how, you know, how, how did we come up with all this and how, how did the product improve? Every single thing that Jesse just talked about was the result of four deployed people doing things and us figuring out how to productize it.
- JZJesse Zhang
Mm.
- ASAshwin Sreenivas
Right? So for instance, um, we realized that, hey, when we go into a new customer, we need to spend all this time writing up the AOPs manually, right? And we're like, "Wow, this is quite a lot of time. How do we productize this?" And we built that into Duet. And then the second part, which we call Duet Autopilot, was, uh, once we built Duet, you know, people were using Duet to write things up, and then we're like, "Oh, wow, there's still a lot of time that goes into iterating upon the agent," right?
- JZJesse Zhang
Mm-hmm.
- ASAshwin Sreenivas
Once it goes live, like reviewing conversations, figuring out how to improve it, and we're like, "Great, let's productize that as Duet Autopilot." In fact, AOPs themselves were a result of this exact scenario, because before AOPs, you would have to write all these procedures in code.
- JZJesse Zhang
Hmm.
- ASAshwin Sreenivas
And then we found that, oh, it's taking a lot of forward deployed engineering work to write all these things in code. Wow, wouldn't it be so much easier and efficient if we could productize it by writing it in plain text?
- 37:02 – 47:55
Winning Enterprise: Glass Box vs Black Box (and Beating Sierra)
- ASAshwin Sreenivas
from now.
- SWSarah Wang
By the way, I think just listening to you guys, it's very clear that you guys deeply understand how to sell AI to the enterprise, and I don't just mean, you know, mid-market, newly IPO'd companies. I'm talking about some of the largest companies in the world. Um, and I think this is particularly interesting because you, you both are very, very technical, but also go-to-market animals. Um, so I wanna talk a little bit about that. Um, but you know, it just feels like you've turned what maybe started as feeling more like a David and Goliath with multiple Goliaths, um, uh, market dynamic to really a two-horse race between you and Sierra. Um, so share a little bit more about why some of the largest enterprises in the world are buying from you guys. And I think what's really remarkable to us, as people who have, you know, studied application software for, you know, over a decade, the sales cycles are crazy fast. I mean, I'm sure you guys, with urgency, want them to be faster, but like usually you don't sell contracts that big to enterprises that big, right? That takes two years sometimes. So, um, maybe just share a little bit more on... This is a long question, but like how did you grow that commercial? Did you come with that into founding the business, and what's resonating with these large companies that enables you to get in and move so quickly?
- JZJesse Zhang
Yeah, I mean, we, we have a lot of respect for, for Sierra and also just, you know, other folks in the space, generally the big platforms. Like they all... [clears throat] I think the platforms themselves move a bit slower, but I think that we've, we've met a lot of those teams. They're very competent teams. They're optimizing for a lot of different things at once. Uh, Decagon versus Sierra, I mean, our, our most recent customer actually, uh, turned off of Sierra to come to Decagon. And sort of the reasoning if, when we asked them was, um, it was, it kind of goes back to this like deployment model. It's like, you know, when, when they worked with Sierra, it was mostly FDEs, and it just felt like a black box where, you know, the, the FDEs were good, but they had to go through the FDEs for everything, so to build new journeys or to, uh, even get like a deeper understanding of what was happening in the conversations. And then over time, that, that kinda just created a lot of drag on how quickly they could move, right? So in the initial deployment, that was good, but then over time, maybe the FDEs were staffed with other things, and it just took them a while, uh, to get insight into what was happening and then build out new journeys, right? So over, over the course of the year, they maybe built out, uh, I think they said three.
- SWSarah Wang
Mm.
- JZJesse Zhang
Um, and so the reason they came to us is 'cause they, they had that frustration, and they wanted to kind of have a different model where it was a lot more productized, right? Back to us being like very product-driven in our vision. It's... They should have a, a core product that even if we're there helping them, even if we are forward deployed, it is like a... In, in service of helping them build a product that they can use themselves and iterate really fast and, and kind of sh- have everything in their control, right? So we, we like to call this like a glass box approach instead of a black box. And, uh, and yeah, so within, um, basically a month, they spun up like seven new journeys on Decagon.
- SWSarah Wang
Wow.
- JZJesse Zhang
Yeah, I mean mostly-
- SWSarah Wang
And it'd taken three, a year to get three?
- JZJesse Zhang
Yeah.
- SWSarah Wang
Okay. Interesting with Sierra.
- JZJesse Zhang
And s- uh, again, so it, it's just kind of the speed of iteration. Like some teams will really like this, like, "Hey, we have control of it. We, our teams, especially our non-technical people, can come in and do things, and they understand what's happening in, in the, in the conversations." And maybe there's other teams out there that actually do like the like, "Hey, you, you guys do everything for us," and, uh, that approach. But that, that's kind of the difference in their approaches, and that's why we've been, I would say, having a lot of success there. And then zooming out, just selling to the enterprise. Yeah, I mean, this is something that, uh, you know, Ashwin and I have never sold to an enterprise before, and I think it just so happens that both of us, um, find sales exciting.
- SWSarah Wang
[chuckles]
- JZJesse Zhang
And the enterprise, it was, it was kind of a quick learning curve for us. So I don't think... I think a lot of it honestly came kinda naturally just 'cause our space is so hot, and like w- generally in these conversations, we're not really having to convince people to like invest in this space. It's like more of, "Hey, we're the right approach for you, so like partner with us." And, uh, yeah, with the enterprises, it's, it's really just about like navigating the orgs and really having empathy for what they value and what they're afraid of. Um, so yeah. I mean, we, we just back to being sales led, right? Like we, we always from the beginning were like, "Hey, we're gonna be like extremely strong on the go-to-market side, and that's gonna inform the product, even though we have this product-driven philosophy." Like we don't wanna just be dreaming up random products to build. Um, so we always had that DNA, and then in the early days, we were just, yeah, pushing really hard. You know, I think... I also wanna say, I think we got kind of blessed with a, a really strong early sales team.
- SWSarah Wang
Mm.
- JZJesse Zhang
And we have a lot of really talented people in that group, and that helped us really get leverage as we were talking to, you know, the big enterprises.
- SWSarah Wang
Some of whom cold applied to you guys in the early days, I remember.
- ASAshwin Sreenivas
Yes. Yeah, yeah.
- JZJesse Zhang
[chuckles] Yeah, cold, cold applied, um...
- SWSarah Wang
'Cause I think they were working in the, in the space already, and they, they were seeing from afar like what Decagon was starting to do.
- ASAshwin Sreenivas
Yeah.
- JZJesse Zhang
Yeah, so some of them had like non-traditional sales backgrounds, you know. So they had non-traditional sales backgrounds. They're coming into sales. The other profile we had a lot of in the early days were just like, like Ivy League athletes, I guess.
- SWSarah Wang
Mm-hmm.
- JZJesse Zhang
And those profiles were kind of a good foundation for the group. And, you know, we've had to scale that team really fast, which is never easy. Uh, so there are things that we're still trying to catch up on in terms of enablement and, and org structure. But, um, because we've always had that intensity on the sales side, and the whole company knows that, like, you know, everything starts with sales, and it kinda propagates back. Um, you know, we've always had that focus.
- ASAshwin Sreenivas
The, you know, the other thing I think that helped us a lot, um, you know, to your earlier point about how did you get some of these large deals closed so quickly was, uh, I think we were very curious about how we could productize parts of it-
- SWSarah Wang
Hmm
- ASAshwin Sreenivas
... within the enterprise.
- SWSarah Wang
Yeah.
- ASAshwin Sreenivas
Right? By which I mean, we aren't a company that just says, "Hey, here's a product. We'll throw it over the wall, and, you know, you get it a year later." Because within a lot of these enterprises, the question they have internally, in addition to will this product work for me, is Can I actually get this live, right?
- SWSarah Wang
Right.
- ASAshwin Sreenivas
And with a lot of these enterprises, it's actually complicated, especially if you're in financial services and you're regulated, for instance. So we actually spent a lot of time sort of, uh, mapping out that part of the journey very well, so that when we walk into one of these enterprises, we can walk them through in very granular detail how we go from this first meeting today to going live at 100%.
- 47:55 – 1:14:45
From Customer Support to AI Concierge
- SWSarah Wang
teams?
- ASAshwin Sreenivas
Yeah. So, you know, our, the... When we launched, the original set of use cases that we sold were in customer support. Um, and the reason was because, one, that was one of the biggest challenges that a lot of our early customers were facing, and two, that was where the capabilities of the models ended at the time, right?
- SWSarah Wang
Hmm.
- ASAshwin Sreenivas
Like, that is all, that is the pretty much the limit of what they were capable of doing. Now, however, as models have gotten better and our customers have realized, "Well, why would I have one set of models that just learns about my customers when they have another pro- when they have a problem, and something else when they come to me to buy something," right? And so we had a, a customer that we originally went live with them for customer support. Uh, and then they realized, they were like, "Well, you know a lot about our product now. You know about the capabilities that it has, because you need to do that for customer support. You know how we like talking to our customers and our brand. Um, can you help us with inbound sales?"
- SWSarah Wang
Hmm.
- ASAshwin Sreenivas
Right? When someone comes in, answer questions about us, do some discovery, and then, you know, assign it to the right, uh, uh, enterprise rep if it's, you know, a deal of large enough value. We had another customer that, um- Started using us for a lot of operational workflows, right? So, uh, we are able to now proactively reach out to them once we start seeing, um, any kind of issues on, on, on that, on that customer's account. Um, because ultimately, at the end of the day, the thing that we built, and we kind of built this intentionally from the start, was not an agent that does customer support well, but rather an agent that follows business process well, right? And executing on operational workflows, doing sales lead qualifications, answering customer support questions, at the end of the day, is just an agent following a business process. And we kind of built it flexibly enough to kind of do all these things because we realized that, hey, at a certain point, the models are gonna get better, and they have.
- SWSarah Wang
And what do they specifically get better at that allows you to do that?
- ASAshwin Sreenivas
Uh, it is specifically the ability to follow instructions well, right? So when you had, um, models, you know, let's say a few years ago, you'd have to give it very, very tight guidance-
- SWSarah Wang
Mm-hmm
- ASAshwin Sreenivas
... very specific instructions that you didn't want it to deviate from. And as the models got smarter, you could kind of give it broader and broader guidance, bigger and bigger instructions, and just trust that the models have good enough sense to interpret it like a human would and kind of fill in any missing gaps, right? Because for, again, for customer support, you can have a very tight path that a model should follow, and that's all you really need. Whereas for sales qualifications, you kind of wanna ask open-ended discovery questions. The conversation is gonna kind of bob and weave, and so you need the model to kind of fill in with reasonable things. So that's specifically kind of what the, what the models got better at over the years.
- JZJesse Zhang
Yeah. Um, [clears throat] I think if, if you think about, you know, what is the, like, 12-month product roadmap, because things are moving so fast, like the ans- the real answer to that is like, you know, we kind of like see how it evolves, and we obviously know what we're working on now. But I think realistically in, in today's AI world, it's very difficult to have like a 12-month roadmap to a T. You maybe know how, like some themes of what you wanna build, but ideally, if you have those things, you just build it like right now, 'cause it's so, so fast to build things now. So, um, I would say that, that is, that is one element, but then the sort of long-term vision is still very clear to us now, right? Which is we use the term concierge, but really it just means like, hey, an AI agent should just be the front door of your business, of your brand, and every interaction, whether it's like reactive or proactive with a customer, uh, should be handled by, by AI. And we've already seen that AI is very good at that, right? Customer service is like a q- huge pillar of that, where it's all these inbound interactions. But why, why not have... also be able to do all these other things? So over time, again, we're, we're not trying to figure out on our own what these things are. We kind of have, now have a lot of customers that will give a signal on the things that they care about, and those will be the things that we build.
- SWSarah Wang
Hmm. We've, we've talked a lot about how the models are getting better and, you know, obviously your capabilities are, are moving along even, you know, ahead of that progress. Um, what are some of the bottlenecks right now that you're seeing? Um, whether that's on the capability side, um, I don't know. It could also be, you know, around persistent memory. I don't know if you guys feel like that's kind of up to snuff on where you would like it or, um, could be other bottlenecks. But curious, like where, what would you like to see and what's kind of holding you back?
- ASAshwin Sreenivas
Hiring.
- SWSarah Wang
[laughing]
- JZJesse Zhang
Yeah. We-
- SWSarah Wang
Got it. So it's less on the AI side. It's actually like-
- ASAshwin Sreenivas
That is, that is-
- SWSarah Wang
... people.
- ASAshwin Sreenivas
We are, we are-
- SWSarah Wang
[chuckles]
- ASAshwin Sreenivas
... voracious consumers of tokens, but we would always love more AI people. I think-
- SWSarah Wang
Okay. Yeah
- ASAshwin Sreenivas
... you know, there's so much to build these days-
- SWSarah Wang
Yeah
- ASAshwin Sreenivas
... that it's, uh, you know.
- SWSarah Wang
And why can't you hire-
- ASAshwin Sreenivas
Always need more people
- SWSarah Wang
... AI agents to do the things that you're doing?
- ASAshwin Sreenivas
Yeah. We, we are voracious consumers of, of, of token, really.
- SWSarah Wang
[laughing]
- 1:14:45 – 1:20:00
Will AI Kill Jobs? Jevons Paradox in Customer Support
- SWSarah Wang
Um, I guess a, maybe a last, uh, question would be just since we touched on like hot button X topics. Um, one, and this is k- kind of a serious one actually, but it sort of reentered the narrative, I think, with, um, you know, the Anthropic video, and it's just sort of maybe never left the narrative, but it's on this concept as progress gets better around jobs, um, and the messaging around that. Um, it's a sensitive topic obviously, but it's interesting because I really think customer support was maybe the first end-to-end use case where you could really take an entire job, um... Or sorry, do an entire job, I should say. Take is the wrong word. Um, versus coding was always, you know, pair programming to start with. Um, how has that, y- you know, we, we sort of joked before, but I, you know, we weren't really joking that, um, AI's actually creating jobs. I'm curious, how do you turn that narrative on its head that folks are just losing their jobs, right? Like do you see the upleveling of folks with AI where, you know, maybe they were doing this job and now they're doing something else?
- ASAshwin Sreenivas
Yeah. I mean, we, we, we see this all the time, uh, because, uh, if you recall earlier on when we were talking about, okay, what are we truly doing? Um, we found that in a, for a lot of our customers, there's actually just more demand for things like customer support than there is supply.
- SWSarah Wang
Mm.
- ASAshwin Sreenivas
Right?
- SWSarah Wang
Yeah.
- ASAshwin Sreenivas
Where companies realize that they're like, "Okay, if our cost of doing customer support drops by 30%," most of them are not just immediately saying, "Okay, now what I will do is, you know, let go of 60% of my team." They're saying, "Okay, now that this thing which is clearly valuable for my customers is much cheaper, let me do more of it-
- SWSarah Wang
Hmm
- ASAshwin Sreenivas
... so that my customers retain for longer, so that, you know, they don't churn off as much, so that they activate sooner," things like that.
- SWSarah Wang
Mm. Mm-hmm.
- ASAshwin Sreenivas
Um, you know, we had a customer in the early days, um, they... And this was, you know, probably two and a half years ago at this point, um, where they said, you know, "Our ticket volume," you know, the amount of customer support inquiries that we get, uh, per month was I think, I think it was like 50,000 a month or something based on, uh, you know, the existing services that they had. Once they started using us, they said, "Wow, turns out our customers have a lot of problems." They said, "Let us make support more easily accessible." Right?
- SWSarah Wang
Wow.
- ASAshwin Sreenivas
So instead of it just being in one part, like buried within a support panel, they're like, "Let's put support on every page, and let's make it more prominent in places where people are more likely to get stuck."
- SWSarah Wang
Hmm. Love that.
- ASAshwin Sreenivas
"Let's allow immediate support for even free users rather than-
- SWSarah Wang
Wow
- ASAshwin Sreenivas
... only paying users." Right? So because of this kind of, there's more kind of latent demand for support-
- SWSarah Wang
Yeah
- ASAshwin Sreenivas
... uh, than there is supply. So automating things doesn't necessarily result in just kind of people laying off their entire teams.
- SWSarah Wang
That may be the best example of Jevons Paradox in real life that I've heard.
- JZJesse Zhang
Yeah.
- SWSarah Wang
So it's exciting.
- JZJesse Zhang
Yeah, I think it's like, uh, AI will, uh, kill jobs, but not careers in a way.
- SWSarah Wang
Hmm.
- JZJesse Zhang
Because like those jobs that are being done currently should not be done by humans. Like, they're very mundane and menial. It's like, it's like a super high volume use case, and people are just like kind of picking up the phone and is like, "Okay, let me click here, click here," and like, "Okay, here's the answer," right? And that should be done by AI. But there is actually like a near infinite amount of things that people could be doing to make their customers happier and, you know, take care of them more. And so people will end up doing those things, and more and more of the mundane, repeatable things will get eaten up by AI. So that, that's, that's what we think will happen.
- SWSarah Wang
Jesse, I think on a podcast, maybe Patrick O'Shaughnessy's podcast a couple months ago, you had mentioned that even your customers, when they've been using BPOs for customer support instead, you haven't actually seen like layoffs at the BPO. It just turns out that those employees have gone and done other things instead. Is that still true?
- JZJesse Zhang
Oh, uh, I mean, it's, it's, uh, it really depends on the situation. So there are definitely scenarios where people use their BPOs a lot less or don't need the BPO anymore. There are other situations where they are not in cost cutting mode whatsoever, and [clears throat] their goal is to either their business is growing so quickly they, they don't wanna scale their operations along with their growth, and so with Decagon, they can kinda like keep it flat or, uh, whatnot. Or it's, "Yeah, actually, we still need people, but now the, there's all these other things they could be doing." And it could be more re- revenue generating things. Um, you know, as... That's like a big area for, for us even is like as the AI matures, you first start with these cost cutting use cases because th- those are easy, but then like revenue generating use cases should also be able to be done through this conversational interface. So yeah, there's, it, it really depends on the customer, but, um, you know, we definitely have customers that have made massive changes.
- SWSarah Wang
Actually, I'd like to end it on that uplifting note. Um, and just to repeat what Jesse said, it may kill jobs, but not careers. I love that. Um, thank you so much for joining us, guys. A pleasure to have you.
- ASAshwin Sreenivas
Thanks for having us.
- JZJesse Zhang
Thank you. [upbeat music]
Episode duration: 1:20:15
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode cO1f2wOxSH4
