Why Building an AI Agent Is Easier Than Deploying One

1. 0:00 – 0:58

   Intro

   1. VKVladimir Keil0:00

      If you want to build an aircraft, you need to procure thousands of suppliers. Someone sends a confirmation of like, "Hey, sorry, like this part is going to arrive two weeks later." And if they miss this email, hundreds of millions of damage.

   2. SASeema Amble0:10

      Procurement historically may have been more in a box, and now it's touching legal, it's touching finance, it's touching a bunch of different software systems and people. The opportunity for the AI-native startup is to say, "We're gonna own that entire end-to-end arc."

   3. VKVladimir Keil0:23

      No company and no enterprise starts with fully autonomous negotiation agents from day one. Why? Because they don't trust us, and they don't trust the technology from day one. And by having this human-in-the-loop approach, we are feeding our agent with all the feedback and all the learnings, and then they suddenly trust us for like 10K negotiations, 20K negotiations, 100K negotiations.

   4. EBElena Burger0:41

      When you think about what a durable vertical AI company looks like, what are the qualities that you look for?

   5. SASeema Amble0:47

      It's really, really hard to forecast your moat going forward. If you look back at all the best businesses, at the early stages, they were...

2. 0:58 – 4:19

   Why AI startups still beat incumbents

   1. EBElena Burger0:58

      Welcome back to the a16z podcast. I'm Elena Berger, and today I'm joined by Seema Amble, a partner at a16z, and Vlad Keil, uh, co-founder and CEO of Lio, which builds AI agents for enterprise procurement. Seema, you recently wrote a piece called "The Incumbents Are Coming," and it asks a question facing almost every AI application company: If an established software vendor already has the customer and the data, and the model can work across its tools, where does a startup have an advantage? And, um, we have Vlad here, and Vlad can really help us understand where a startup has an advantage. And so I think a, a good place to start is the cases for and against the incumbents. So if a company can connect a capable AI agent to the software it already uses, where does another application actually, uh, have a use case and an advantage there?

   2. SASeema Amble1:58

      Yeah. Let me back up and frame it up a little bit. So historically, we thought about there's the incumbent, and then there's the startup, and they're, they're fi-- they're, you know, it's a, it's a fight between distribution and, and innovation, which, you know, to, to co- to take my partner Alex Rampell's phrase. Um, however, now there's like this third piece, which you're, you're, you're, you're pointing at, which is, um, the incumbent can layer on, um, one of the models on top or... And, and then there's a much more formidable competitor in the market. And so why do you need an AI-native startup if you've got CloudForce, which is taking Claude plus Salesforce and putting the two together, and then, you know, you've already got all your data and your, your customers, uh, sorry, your employees are all used to using the product, so why another product? Um, I absolutely still think there's obviously still a case for, uh, the AI na-native startup, and it really centers around the fact that, um, the legacy incumbent is limited to their system of record and that record that they have, and they're not completing the end-to-end job. So let me put that, like, more concretely in an example. Um, so say you're, um, a customer, and the customer calls and says they got charged after they got canceled. That, um, resolving that cancellation history isn't just the, uh, you know, customer going into the chat and saying, "Hey, um, I got, uh, I got o-overcharged." And the response there, it has to hit billing, it has to look at all the chat history, it has to look at the contract. That's not one system of record. That's the knowledge around that customer and everything it touched, and that's something that one system of record wouldn't touch. However, the opportunity for the AI-native startup is to say, "We're gonna own that entire, um, end-to-end arc, that end-to-end arc." So that could be in legal, so owning everything from a brief all the way through trial. Um, Vlad can talk more about procurement, but it's, it's really the s- concept of owning the end-to-end, end-to-end work.

   3. EBElena Burger4:03

      Yeah. Uh, Vlad, do you wanna talk about just kind of where, where that does show up in procurement? Like, where a existing incumbent plus a model just is insufficient, and what have you seen just with the companies that you work with?

3. 4:19 – 6:18

   The hidden work behind an $8K line item

   1. VKVladimir Keil4:19

      Sure. So, like, when we think about procurement, I, like, I would assume that most people think about, like, prices, right? So, like, what is the end price that we negotiated on? And surprisingly, the, the record looks always very, very simple and very easy. It's just 8K. That's the result, for example. And, um, I mean, it's the, the same for sales, right? So, um, even if, um, if I come to Seema and tell her, like, "Hey, we, we now partnered up with another enterprise, um, look at the signature here for, like, this contract," it looks, like, very easy. But, like, Seema doesn't see, like, all the work behind it, right? So, like, there are probably, like, 30 stakeholder meetings happened, 500 emails, 20 Excel sheets, and, um, that's the same for the counterpart procurement, right? So you see in your ERP system 8K for aluminum, um, but you don't see that maybe the, um, the supplier did, like, a pushback and asked for, like, um, 10K. Um, you don't see that, like, a cost engineer had run, like, three weeks of Excel sheets and 3D modeling to find out the prices of the, um, of the part and everything else. Um, so that's what we see. So, like, most of the work, um, in procurement actually happens outside of this, um, ERP or any system of record. Yeah.

   2. EBElena Burger5:34

      Yeah. Um, and I'm sure just in, in the workflow that you described, an agent can do, you know, a huge number of things, and different kinds of agents can do a large number of things too. And I actually think that's a good bridge into the next question, which is like, you know, a year ago, I think a lot of the incumbents were tak- ... releasing chatbots, and that was kind of the extent of what you would see. But Seema, in this piece that you wrote, you, you lay out four different kinds of agents: retrieval agents, which are kind of the, the chatbots that we're familiar with, process agents, policy agents, and principal agents. So can you just walk us through all of them and explain kind of what changes in the kind of judgment that's necessary across all

4. 6:18 – 9:27

   Retrieval, process, policy, principal

   1. EBElena Burger6:18

      of them?

   2. SASeema Amble6:18

      Yeah. So a, a year ago, um, I, I, I made this meme, which was the slap on a chatbot strategy, which is essentially, like, all the incumbents effectively had a, a chatbot that sat on top of the system record, which you could chat with to retrieve information, maybe do some light anal-analytics. And that really, um, was in that first bucket of what the retrieval agent is. Um, and maybe w- let me walk you through an example of what each of, like, the retrieval, the process, the policy, and the principal back to the, like, customer, a, um, the, the customer support example just 'cause it's very-- it's easier to understand. So imagine if you're a customer and there's a service outage, and you're calling in, uh, to say, "Hey, I wanna get compensated for this service outage." The retrieval assistant, which the incumbent may have, is going to be able to pull up, "Yes, you know, there-- This was what the contract term said," and, um, "Yes, there was an outage," and just verify that information. It's pulling up information about the customer. It's in the database, and it's just, like, sharing it back and, and maybe synthesizing. The second agent, or the second step in the agent co- um, sequence is the process agent. So that process agent may be able to pull through an approval of credit and say, "Okay, um, based on the-- our policy handbook, um, it was out from these dates, therefore, we o- say you are entitled to, um, money," and it can just apply the bill and just do process. There's no judgment involved. Then as you keep going to the policy agent... So the policy agent isn't just going to a- um, apply the process, but it's gonna say, "Okay, uh, in this situation, it was out for 20 minutes. That's enough to be considered a significant outage." And they're applying that judgment because there isn't really, like, a s- a strict definition around it. And then the last stage, which is the principal agent, you're actually weighing, um, you know, okay, should we offer more compensation? Because that was a pretty terrible outage, and we wanna res-preserve the relationship, and it's worth doing more beyond even what a process or policy says. But the importance of these four things is that most incumbents started out in, if at all, in bucket one. Now, at least they're marketing that they are moving towards process and policy, meaning they would be able to apply more judgment. And, you know, if you look at what they've, they've launched, um, these are workflow agents that will help you get a document signed or, um, input information from a transcription or things like that. They're very s-much still limited to, I would say, retrieval and a little bit of process. They've not gotten into more judgment, and I can get into why. Like, there are all sorts of incentives, uh, that are preventing them from that. But the incumbents are trying, and I think what they're not able to do, they're like... Some of them are saying, "Okay, let me partner with OpenAI, Anthropic, one of the labs, and try to build out, um, take that model capability and complement what they have, um, and superpower it."

   3. EBElena Burger9:21

      Yeah.

   4. SASeema Amble9:21

      Or charge, I should say.

   5. EBElena Burger9:22

      Yeah. Well, do you wanna say why sort of some of the incumbents are holding back?

5. 9:27 – 14:48

   The incumbent's internal conflict

   1. SASeema Amble9:27

      Yeah. Okay. So they're-- I wouldn't say they're holding back, but they're held back. [chuckles]

   2. EBElena Burger9:31

      Yeah. Okay.

   3. SASeema Amble9:31

      Um, yeah. I, I'm sure they, they want to be full force, but there's probably two pieces. One, they have this advantage of distribution, right?

   4. EBElena Burger9:39

      Mm-hmm.

   5. SASeema Amble9:39

      They have the customer trust, which enables them to then sell more products with the customer. So take Salesforce. When Agentforce launched, uh, it was very easy for customers to say, "Yeah, I'm gonna sign up for the Salesforce Agent," especially if it was offered at almost no extra cost. Um, and so they have this trust. They have the distribution. It's often pretty seamless to turn on the product. The flip side is there's all these internal incentive issues, right? Which is if you start getting into more complicated agents, it's, um, it's a c- it's an internal conflict with an existing product that's offering workflow versus you're resolving the work. And those are two products that are, um... Like, if you're resolving a customer support issue c- end to end versus providing, uh, um, a workflow for a support agent, a human agent, those are different buyers. How do you, um... You know, those two teams are in conflict. And then on top of that, I think from a sales perspective, like, what are you selling to the customer? And then oftentimes in a classic, like, incumbent issue, right, is, like, there's two, uh, two VPs, right? And they're different orgs, and they're selling different products, and, like, they will never be able to, uh, figure out what the, the right set of in-incentives and the right person to sell to. But anyway, so there's all these sort of classic incumbent issues I think that the incumbent also runs into.

   6. EBElena Burger10:59

      Makes sense. Um, Vlad, where across this, this kind of spectrum of retrieval agent, process agent, policy agent, principal agent, where, where does Lio sit?

   7. VKVladimir Keil11:10

      Yeah. So, like, we span across, like, I would say, like, all of those categories, and it really depends on the complexity and the risk, um, our agents take, right?

   8. EBElena Burger11:19

      Mm-hmm.

   9. VKVladimir Keil11:19

      So there... Sometimes we, um, can, like, we already have use cases where we run fully autonomously. Um, sometimes you have the human in the loop. It really depends on, like, the budget approval, how, like, how mu- how complex this is, how, how risky this is. Um, but maybe, like, coming back to what Seema said about trust for, for, for the incumbents, like, you talked about, like, internal trust. I think there's, like, also an external trust thing, right? So, like, how do you convince someone to go through from Hey, just like an agent that retrieves some information and maybe runs processes to do, to do like something completely autonomously is like, obviously there's like a product component to it, but mainly it's a people component, so they need to trust you. Um, and like we as a startup, scale-up, we have to earn the trust. Incumbents already have the trust, but this also means they, like, they can destroy the trust if they ship something, like, too early and the product maybe doesn't work, or it's bad, or it's like works, um, decisions, um, in a, in a bad way. So that's, um, that's, that's what we, we can do. Um, and then interestingly, or what, what was like really surprising for us, like those process agents, um, actually became kind of like a side quest, um, for us. Because it... Like when we talk, like when we look at the invoice process, um, or like invoice agents as a process. So you, we, we retrieve some informations, um, you match this across like other documents, and then you push this back to, um, SAP Oracle, like a very clear, um, clear process. Um, and then we figured out, okay, there's like 100% of the software market, right? So that's, that's invoice software. Um, that's how you build it today, but it's actually only like 20% of the work of the job to be done, of the problem, because 80% of the problem is like, um, what if, like what if the invoice is fraudulent? What if there's like a mismatch? Like, what did, what is like if we don't take the happy path? Um, and so we like, um, very fast shifted to, to the next step of agents, like doing those, doing those exception hand- um, um, exception handlings. Um, and we convinced the customers by, um, I think we got like lucky being like always slightly ahead of the curve, so we were able to, to pitch the next generation of agents. So for example, like when we started out three years ago, it was like just retrieving a document. Like nuf- like not impressive at all today, but like three years ago, this was in, like crazy impressive. Um, so we pitched this to customers. We find out, okay, that's a real problem, that's a use, real use case they would pay for, and then we were able to ship this some weeks later. Um, and in the same way, we are doing this now for the next step for process agents and for like fully autonomous agents and for long-running agents. Um, we are pitching this to them, finding out it's a problem, and then we are able to ship this very fast.

   10. EBElena Burger14:02

       I think an interesting point on trust is this internal versus external trust. The other lens on that is, yeah, so you need your customer to buy the procurement software and trust to use it for their internal processes. But one of the really interesting things when we first met Vlad was that they're doing, they're also doing the negotiation. So you have to trust that the Lio agent is going to then interface with a third party and-

   11. VKVladimir Keil14:27

       Mm.

   12. EBElena Burger14:27

       Uh, and there is that piece of trust. Um, and of course, I think a lot of people feel burned by the incumbents and like, you know, they're pretty limited and haven't been able to do what they've marketed they've had in the past, so. But putting that aside, like, I don't know, maybe Vlad, I'd love to hear a little bit how you convinced the customers to s- to trust a, an AI agent to now take on negotiations.

6. 14:48 – 21:13

   What procurement actually looks like

   1. EBElena Burger14:48

      Yeah, and can, uh, when you do that, can you also just like paint the picture of like what's involved in procurement, and who, who are your customers, um, and, and what kind of sort of legacy systems are they used to?

   2. VKVladimir Keil15:02

      Yeah. So like when we, um, when you think about procurement, you like maybe just think about like purchasing or like you, when you look at like B2C world, like just you buy something. Um, but actually it's like a very intense proces- process which like runs the economy, right? And it includes multiple stakeholders and a, a lot of stakeholders and a lot of, um, um, a lot of departments, um, legal, cost engineering, um, obviously procurement, finance, um, all of them have to work, have to work on this, um, um, on, on these decisions. And essentially like when we, um... Like I think, I think the reason is like how, how we convince them is like what I al- um, already said, said earlier on. So like we were like always a little bit ahead of the curve. Okay? So we knew tech- the technology is coming, so we, like even before ChatGPT came out, um, like we, we just started a few weeks before like this ChatGPT breakthrough. Um, so we already heard all of the, all of the problems, then we heard the, the hype and the, let's say, like tech bubble, and we were able to pitch this enterprises, and we quickly figured out, okay, like this is a, this could be like an interesting use case, like just chatbot applications or retrieval agents, um, document processing. Um, and then we, we are able to find out the problem and then ship this quickly to them. And then obviously like this is like the people factor of like trusting, so we, we are telling them something about, and then we're able to really ship something in production. Um, but then there's also like this product perspective where we have, um, a lot of like evals in place, right? So like we, um, the, like no company and no enterprise starts with fully autonomous negotiation agents from day one. No one does that. Um, why? Because they don't trust us, and they don't trust the technology from day one. Um, so we have like a very easy approach of like, um, having a h- like human in the loop, and this is like extremely helpful for us because we see, like we have like, let's say, like the perfect negotiation agent, which is like overall the perfect procurement negotiator, but we don't know exactly how a Fortune 10 enterprise, like the specific Fortune 10 enterprise operates. And by having like this human in the loop approach, we are feeding our la- agent with all the feedback and all the learnings, um, and then they suddenly trust us for like 10K negotiations, 20K negotiations, 100K negotiations. Um, and then you also have like other, um, other agents that are like more long-running. So when we talk about like multimillion-dollar negotiations where you analyze complex 3D models and technical drawings, um, there we on purpose have always experts in the loop, right? So there's an agent running for multiple hours, and then we ask for feedback of the cost, like cost engineer, and then it does the next, um, the next work, um, and, and so on.

   3. SASeema Amble17:46

      Maybe just to double-click on that, how do you-- what-- where do you put the human in the loop on the g- like negotiation side? Uh, you mentioned the cost engineer, but like if I were-- you were going back and forth on a deal, is it mostly around the like, uh, you know, data for, you know, something like cost engineering, or is there anything else where you have humans in the loop there?

   4. VKVladimir Keil18:06

      So a-again, depends on the level of negotiation, right? So like we have to distinguish between like, um, negotiations where you just... Like a negotiation wherein, uh, f- like where enterprises, they never did those because they didn't have the capacity, but by h- by, by deploying agents, they can just, um, capture savings that they weren't aware of, right? So they, they, like before agents or before Lio, they just didn't care about everything which happened below 50K, right? So you can just, like this is maybe a hack for like other startups. You can just send, um, an enterprise an invoice for 40K. They will probably not negotiate because they don't have the capacity to do so. Um, um, except they have Lio agents, then we are going to negotiate, um, against you. Um, but other than that, they are just like, just, um, just paying that. And obviously they're the risk of like, like what is the risk of like you don't, you didn't neg-negotiate at all, so what is the risk now of having a bad negotiation agent? Nearly zero, right? So, um, maybe we miss out on some negotiations, but like it's better than nothing. Um, but still, um, in like in, we're talking about business relationships, and business relationships are not always about the cost and the money, right? So maybe you're not spending a lot on the vendor, um, but maybe you, you're like, you need this business relationship, right? So a good example might be like podcasts or marketing services. Okay, that's like probably like a, a friction of, uh, um, of, of the spend, but you don't want to, like you have a clear business relationship with, with someone like setting up the studio, um, and you don't want like s- like random, some random people doing that because they already know how, how a16z operates and how you want to record all this stuff. Um, so there we have human-in-the-loop, um, approaches, um, where like p-procurement people care about the relationship, so they care about the voice of tone and, and how it works. Um, but mostly autonomously. Um, and then we have the other set of agents where we always have, um, a, a human-in-the-loop approach, and it's like, um, there are like multiple steps in like negotiating. It's also like it's not on-o-only the price, right? It's also like, um, how is the contract designed? So we're talking about legal. How is like, um, collection designed? So we, we talk about finance. Obviously like cost structure, so we're talking about really like cost engineering. Then we talk about commercials, um, that's procurement. Um, and those are not back office people. Those are like highly trained, um, people where they have very specific knowledge of a very specific process of a very specific company and very specific industry. Um, and they feed, uh, those long-running agents of Lio with, with those insights.

   5. SASeema Amble20:45

      That's another example of how procurement his-historically may have been more in a box, and now it's like, okay, it's touching legal, it's touching finance-

   6. EBElena Burger20:53

      Mm-hmm

   7. SASeema Amble20:54

      ...um, and it's touching a bunch of different software systems and people and both specialists and more generalists.

   8. EBElena Burger21:00

      Yeah. Um, can we, can we map this onto a specific, uh, cust- uh, like not a specific customer, but a specific vertical? Like I'm a drone manufacturer. I manufacture humanoid robotics

7. 21:13 – 28:03

   Procurement at Boeing-scale

   1. EBElena Burger21:13

      or something like that. Like how many parts do I have to, you know, order and procure? How many factories am I touching? How many suppliers am I touching? Just all of, all of those things. If you wanna pick maybe, Vlad, a vertical that, um, you know, is, is just managing all of this complexity with Lio and, and just kind of take us through what their experience is. I think that would really help just illustrate exactly, um, just everything that you touch.

   2. VKVladimir Keil21:42

      Again, like when we, like when we as Lio, when we talk about procurement or purchases, like we don't talk about like laptops and pencils, okay? Like we think like that's solved also by Lio agents, but that's easy. We solved this like three years ago. Um, we talk about like when you, like you want to build an aircraft or robots or drones or even like we're now like doing a podcast about, um, again, like AI hype, but even AI needs to be built, right? So you need data centers. Um, and like building means procuring, like someone needs to, like if you build an aircraft, you need to procure thousands of suppliers. You need to build a factory to build this airplane. Um, and like really small frictions can have like a crazy impact, right? So like there's a, like if you're running a very large project of like building a data center or building an aircraft, if there's like one specific part which arrives two weeks later, this can have a, like a damage of like hundreds of millions of dollars, um, and postpone, uh, and postpone the project. Um, so like all of, and all of those, that's why like all of those decisions have to be co-coordinated. Um, and one part is like you, you need to figure out like what you need, with what suppliers you work, um, what are like, what is like the best supplier to, um, to like, to, to get this part. But then once you decided all of the stuff, there's like all this operational, um, back office, um, stuff behind it, which might sound like unnecessary and boring, but again, like operational means someone sends a confirmation of like, "Hey, sorry, like this part is going to arrive two weeks later." And this is like one of 500 emails, um, in the Outlook or Gmail, um, of a procurement manager. And if they miss this email, hundreds of millions of damage done. And this ha- like this happens regularly because the only thing they store in their, um, system of record is then just the date, right? So like it will like, not like not this Wednesday, next Wednesday. That's what you see in the system. But you don't see like, like maybe that's okay, but maybe that's a $100 million damage, and someone has to decide that. And that's, that's also what our agents are doing, right, day to day. They're not only retrieving the information, they are then Making the decisions, um, are like, does this have an impact? What kind of impact, and how can we resolve this?

   3. EBElena Burger23:58

      Yeah. Um, when, when you are sort of so deeply embedded in the physical world, um, what, what kinds of, you know, physical world problems can you intervene with? Like, some things I would think are just, like, unsolvable. You know, like, let's say you have a shipment coming in and a bunch of stuff, like, falls off the ship, or the street is closed [chuckles] or whatever. Like, there, there, there kind-

   4. VKVladimir Keil24:25

      Yeah

   5. EBElena Burger24:25

      ... there's things that, like, you can't do, and obviously there are things that you can do. So, so where, where can you intervene, and where does that really make a difference?

   6. VKVladimir Keil24:35

      Yeah. But, but that's, like, actually it's more like probability, right? So obviously, like, you can't, like, you can't change if, um, if, like, some, like, if there's, like, a damage on the ship. Like, every, every example that you manage, like, you, um, you, you can't, um, change that. But you can, if you have, like, all of the context, you can predict that.

   7. EBElena Burger24:55

      Mm-hmm.

   8. VKVladimir Keil24:56

      Um, because you can predict, like, how, um, how reliable is a supplier, okay? So there are, like, there are ways on, like, protect the goods that you're shipping. Um, and if you have, like, all the context, you have, like, one supplier where, like, 20% of the goods are missing, and then 1% of the good is missing. Um, and maybe, like, this one with 20% is, like, 10X cheaper, but for this use case, it's fine for you to pay 10X the amount, um, because you have, like, a higher probability that this thing actually arrives. Um, so, and this is the powerful thing because you have, like, context not only within one enterprise, um, so we, like, we talked about, like, multiple stakeholders, but there's also context on the outside world, right? So just the agent should, like, have context of all the news out there. Um, maybe even ha-having, like, context of, like, some bets, say, like, on Polymarket of, like, okay, those disruptions are going to happen. Um, then, like, um, information about, like, on the supplier side, on the seller side, on the demand side. And by combining all of those contexts, um, I wouldn't say that there is a limitation, um, in the long, in the long run. Obviously, um, like, today we have, like, different sets of, like, probability, but we can, we can help throughout the process, and this is what we are building, um, building at Lio, right? So it's much bigger than just procurement intercompany. It's more, like, intracompany and, like, how, um, how are, like, businesses, like, how are enterprises doing business with each other, so, like, buyer and supplier side.

   9. EBElena Burger26:26

      You describe Lio as a, as a multi-agent system. So can you describe what the different agents are doing?

   10. VKVladimir Keil26:35

       One level is that we, um, that Lio agents span across, like, all those four categories that, that Seema mentioned, um, in her, in her article. And, um, it again, like, depends on the, um, on the risk and the complexity. Um, so we use, like, all of them, so, like, multiple agents. Um, but the other thing is that, like, in, in order to do a job end to end, those agents need to share information, um, with each other. They need to do this in, like, a very specific order. Um, and when we talk about, like, a multi-agent system, this is essentially what we are doing. We are solving, like, the task end to end. And because the also, like, human-level task involves eight people, e- eight stakeholders, um, and maybe, like, three departments and five different, um, software tools, um, we need to cover, like, all of those to, to do, like, the job end to end, and those agents need to then communicate with each other. And only with a multi-agent system you can do a job end to end. Um, when we started out, we, um, like, started off with, like, an retrieval, like, more like a copilot, obviously, like, three years ago. But then the next step was, like, a single agent. But then we very quickly discovered, okay, like, that's, like, you can't solve an, um, you can't solve an, like, negotiation even, um, without having a contract, um, agent, without, um, maybe, like, having, having an agent looking at the news and everything that I described before. Um, so that's what, um, yeah, that's what we define as a multi-agent

8. 28:03 – 37:57

   A bolt order, end-to-end

   1. VKVladimir Keil28:03

      system.

   2. EBElena Burger28:03

      So if you have, like, a bolt, like an airline company needs to procure a bolt for, let's say, you know, Boeing needs to con- proc-procure a bolt, what exactly is that process for procuring the bolt, and, like, where does Lio step in on that process?

   3. VKVladimir Keil28:19

      Yeah. So, like, this is, like, one of, one of the, um, one of the purchases where w- that we can, like, run fully autonomously, and we can do this because of this multi-agent system. So, like, first of all, someone has a demand, right? So they need to, um, somehow communicate it. And even, like, this part is extremely complicated. Um, so you need to call someone. Maybe you, like, you, like, you open up your laptop because you're, like, a construction worker. You open up your laptop only every second week, um, and now you are required to, like, work with SAP, um, or any other, like, um, ERP system. So you can't even, like, issue the demand. So th- this is, like, how we make it very easy. So you ca- like, you, you, you take a photo, like, you, you, you upload a quote, an Excel sheet, um, and it's e- actually everything that you, that you should know about procurement. Like, no one cares outside of the procurement department about, like, categories, GL accounts, framework contracts. No one cares. Um, we in the procurement world care, but no one outside that cares. And then our agents take off, and they, like, they check the inventory. They find out, okay, they, they ask another plant, "Okay, can we, like, can we source those, um, those bolts internally?" No. Okay, then I'm calling the, um, I'm talking to the sourcing agent, finding out, do we have internal suppliers? Do we have external suppliers? Then some, like, another agent has to draft the RFQ, send out the RFQ over, um, over email. Then bunch of emails arrive. Some of them are, like, completely nonsense. Some of them are, like, just in the email. Some of them are PDFs. Some of them are Excel sheets. We retrieve those informations. Um, then we do, like, the next step, or maybe, like, based on our price benchmarking, there's an opportunity to negotiate. Um, and then, then we have, like- Agents that essentially decide on the next step. So negotiation could mean strategic negotiation with a human in the loop. This could mean autonomous negotiation. This could mean auctions and e-auctions, calling then the specific agent, doing the negotiation. Um, and then, like, doing this, like, end-to-end. Think about, like, confirming the order, shipment tracking, um, invoices. Um, and we, we are able to run this, like, fully autonomously, capture, like, all the context. Um, and then obviously the next powerful thing is do this for, like, more complex, um, parts where we talk about, like, direct procurement, um, where we, um, where we also operate. Yeah.

   4. EBElena Burger30:39

      What, what's, uh, direct procurement?

   5. VKVladimir Keil30:42

      Yeah. So, like, um, essentially, like, everything I just described is, um... The main goal is here automation, right? So you can, like, run this process fully autonomously, and then throughout the process you can, like, generate even more savings, right? So it's not like... So we look at it as like, okay, what is, like, this, like, end-to-end, like, job to be done, how does it look like? So, like, what are they doing, like, 1,000 times a day, but actually they want to do it, like, zero times a day? We've, like, run, like, fully autonomous agents. But there's, like, also opportunities of, like, what are they doing zero times a day, but if a business would do this 1,000 times a day, that would, that would have a crazy P&L impact. Autonomous negotiations on, um, spend they never negotiated before. Um, so, so this is, um, and this is, like, in the indirect procurement part. Like, think about, um, MRO parts, building a factory, the bold example that we did, um, but also laptops and pencils, marketing services, someone who needs to build up this podcast studio. Those are, like, all, all indirect. And then we have, like, direct parts. Um, this is like when you build an airplane. Those are, like, all the suppliers that actually, um, well, that you actually need to build the airplane, um, or to build the drone or to build the robot. Um, and there we don't talk about 50,000 suppliers. We talk about 100 suppliers or 2,000 suppliers maximum. Um, and those are, like, extremely strategically important. Um, and you have maybe on one supplier, like, one billion of spend. So you don't want to run an autonomous negotiation. You want to run a negotiation which, which takes three months and where you're, like, crazy prepared and where you have engineers on your team analyzing, okay, what's the indices for aluminum? What's the indices for oil? Um, how did the price change? Um, so you, like, really take over, like, all of those drawings. You check the quality, um, of this part. And this is what, like, where it gets, like, really exciting deploying, deploying agents.

   6. EBElena Burger32:36

      Yeah. And, and for something like that, presumably, like, you'd have the expert engineers and the other procurement people kind of more as the front of house and, like, the agent is more back of house. Is that the idea? Or is the agent, like, actually it's like you, you sit across the table and you're shaking hands, and it's, like, the robot instead of the human who's, like, negotiating? [chuckles] Like, um, is it... So yeah. Is it, is it more back of house or is it, like, still front of house?

   7. VKVladimir Keil33:02

      It's, it, it's, it's obviously more back of house-

   8. EBElena Burger33:04

      Yeah

   9. VKVladimir Keil33:05

      ... because you, like, because you, like, need, like, these complex multimillion dollar negotiations. And, and that's, that's again, like, a beautiful example. 90% of the work is preparation.

   10. EBElena Burger33:14

       Mm-hmm.

   11. VKVladimir Keil33:15

       Like, the end result that you see in your system of record is like, oh, instead of, like, one billion, I paid $900 million. Okay, but, like, there's, like, three months of preparation and, like, 10 people working full-time on that. Um, and obviously this is, like, hap- happening in the, um, in, in the back. But actually we have some use cases where, um, it's also, like, hap- like, helping in real time. So think about, um, let's assume we would now have a negotiation, and I have, like, a pro- perfect preparation. The same as, like, with, like, those, um, those, those notes, um, those notes that we are having here. Um, imagine, like, while we are negotiating, um, I would have, like, real-time insights on my screen popping up, um, where you tell me the indices for oil changed 10%, so, like, it's increased by 10%, so that's why we need to increase the prices by 10%. And I would have, like, an initial, like, an, an immediate pop-up of, like, that's true, like, oil increased by 10%, but the product, um, has only 30%, um, of oil content, so, like, you, you can, you, you shouldn't increase the price by 10%, but maybe only by 4%. Um, so yeah. There are, like, ex- exciting use cases also, like, in, in the real life, um, part.

   12. EBElena Burger34:26

       Yeah. We, we know that, you know, companies like Harvey and Decagon are really fine-tuning models now. Um, what, what kind of underlying models do you use, and, and how do you approach things like fine-tuning or, or harnessing?

   13. VKVladimir Keil34:41

       Yeah. So, like, we believe, like, you can... So, like, we use, um, multiple models from, like, all, um, all providers, and we really see this as a, like, obviously, like, as a commodity, right? So they, like, have, um, really good, like, general business purpose or, like, reading, creating a PDF and, like, creating, creating Excel sheet, like, all, all of this stuff. Um, but we also believe that, like, for some use cases, you, um, you, you can get extremely far with, like, combining the foundation model with a, with a harness, and you can maybe reach, like, 100% of, like, the job to be done. Um, but there are also some use cases where you can have, like, the best foundation model, the best harness, um, whatever that means, but, like, um, and the best harness, but you still can get only to 80%. Um, and, like, I would say, like, good examples for that is, like, for example, what, um, when we talk about negotiations, what, like, cost engineers doing, right? So they are, like, analyzing drawings, and then they're, like, defining, okay, what should this, um, this part actually cost? Like, and that's why it's called, like, should-cost modeling. Um, and there's, um, there is definitely, like, an opportunity where we're, like, thinking and already started, um, um- F-f-fine-tuning the model to get then to, um, to get then to, to 100% in this part. Another example is like price benchmarking, um, where think about like the, the per-- like you would have a quote, and in the perfect world, you would just drag, drag and drop the quote somewhere, and you would get the perfect price. But it's ex- like, and all those, like all those informations, they are like not publicly available, right? So there's all, like, those are like all proprietary data based on like one, um, one enterprise or like across multiple enterprises. So, so like general purpose models can't train their models on that. Um, so what we are, like what we are thinking about is like maybe like not training just an LLM, but I think like what we see now with models like Jeff, um, also popping up where you have like an, um, you train it on like text data, but the out- like the, the output is actually like an outcome, um, or just like the perfect price. Um, and you can't do this with Harness because, like, if you would give me like an, um, a quote from BCG and a quote from McKinsey, they, they could do like the exact same work, but this could be like a 10X different price. And I would have like no idea like what is better. But if you give this to a procurement manager, he would like initially have a gut feeling on like, okay, like this quote makes sense. Um, or like I think, I think again, like good examples again, like content creation. Like always like I don't know like how much I should pay someone for creating a video. Um, but there's like a gut feeling behind it if I ask like another video creator of how to, um, how, how to do that. But if you ask them to write down the rules, they can't do this because it's like just like gut feeling and instinct. So, and that's where we see a lot of opportunity actually, um, training an agent, but not maybe like a classic LLM, but more exact like again, like what, what companies like, um, or like models like, like Jeff are now doing, um, on the outcome based. So we have like price benchmarking, short cost modeling. Yeah.

9. 37:57 – 44:46

   What a durable vertical AI company looks like

   1. EBElena Burger37:58

      Um, Seema, uh, we've talked a little bit about how labs are really moving into industry-specific work or working with incumbents to do this. Um, when you think about what a durable vertical AI company looks like, what are the qualities that you look for? One is around, you know, owning the end-to-end work that we're talking about, building up this data asset, and being able to do, um, something that, uh, hasn't been done before in many cases. This is all said, I think when you ta-talk a lot about moats, it's really, really hard to forecast your moat going forward. If you look back at all the best businesses, at the, at the early stages, they were, they were just thinking about, "Okay, I'm winning customer trust, I'm selling more to them, and there's a lot of opportunity," versus, "Okay, I'm gonna do these six steps and then get to the seventh step, and then we'll have a moat." And so I think we, we talk a lot about defensibility and durability, and I think part of that is you're locking in the customer. They... There's more dependencies. They find it valuable, and you're doing more of the work. And here it's, it's truly like, okay, you know, old CRM company was a log for all the deals. New sales AI agent is actually owning a lot of the sales prep process and the outbound process, fielding inbound and doing a bunch of the work. Um, the comp- the overall customer is dependent on that product, and that's like a really important signal of getting to the moat. And everything we talk about in terms of, um, in terms of stickiness and network effects and all that is sort of downstream of, um, of, of that initial like customer use and the value of the product. Yeah. Um, Vlad, have, have you had conversations with customers or potential customers who ask you, you know, "Why should I buy your product? Why can't I just, you know, plug into a model and do this myself or like use whatever existing system of record I have plus a model?" Um, like, like what, what do you tell them and, and how do you, how do you convince them to, to use Lio?

   2. VKVladimir Keil40:07

      Yeah, 100%. And, and that's a very fair question, right? And like even if you look like internally at Lio, so like the, the first use case three years ago, which like kind of like went viral in the procurement world, was like just like having a quote and then getting this information into SAP. So like very, like, again, like technically like, hmm, but like, like tremendous business value. Um, and so you, um, you have like this retrieval agent, um, getting like all of the information, putting it into SAP. We-- Like this was our like first product. Um, and we had an engineering team, like obviously small, just like the three of us or maybe like four people building it, um, and then selling it. But this is nowadays a case study, um, if you, if you're applying to work at Lio. So we give this to people to like, to build this, and they have like eight hours to do so. So what I want to say by that is like a product that we, like one of our first use cases, um, can now be somehow built by engineers within eight hours. Um, so because it's very easy to build stuff nowadays. Um, so obviously there's a question, okay, well, so someone can build this within eight hours. Okay, cool, but then couldn't like just procurement departments also just build everything in two months? Um, and the answer is like, yes, you can build this in eight hours, and you can build this, but you will only reach 70%, let's say like the, of the performance. Um, and the problem is 70% of performance or accuracy or however you measured it, it depends really on the task, doesn't mean 70%, um, automation, right? So it's, it's, this can be mean that you like have 70% of the performance, but you still need to do 100% of, uh, 100% of the work, um, because 70% is not, not that much. So again, like all of the people have to, um, have to check the data. Um, so maybe you even created like more work, um, more work than Um, than it did before.

   3. SASeema Amble42:05

      T- two things to, to layer onto what Vlad just said. One, um, overall it's good if there's more adoption of the base models or just, like, GPT products, because it means that people are also willing to trust, um, vertical specific products as well. So I think that r- increased, uh, familiarity, comfort, excitement about AI tools is generally just good for the market. Um, the second thing is I was chatting with the management team of a Fortune 500 company two or three weeks ago, and, um, one of the things they mentioned was they had tried to build out their own cash collection, uh, product. This is a big enterprise business. And they, like, after, I, uh, I don't know, three or four months of work at a minimum, they had found that there wasn't enough context. There was poor quality context. They had a lot of recordings, a lot of screen grabs, um, they tried to pull it all into one system, but they-- there was no, um... There w- it wasn't good enough. Uh, and then there was this giant question around, okay, like, you know, we've got now two different ERPs, and we're about to acquire another one. Um, who's gonna update all the mappings, test the, you know, test out, okay, does it work? Um, and then, like, we keep talking about exception handling. You now need to map that onto a totally different system, a different way of doing things, and I think they quickly realized that the internal build didn't make sense. And so we keep hearing stories of this where people are like, "Okay, I'm gonna do the internal build," and they're like, "Wait a second." It's not different from what DIY's ever been in the past, which corporates have always tried. But, um, I think enterprise companies generally realize that, like, there, there's their core competency, and then there's, um, building internal tools, and, um, they should focus on the, the first camp.

   4. EBElena Burger43:51

      Yeah. Yeah.

   5. SASeema Amble43:51

      Makes sense. Um-

   6. VKVladimir Keil43:53

      Yeah. So that's exactly what I, what I've meant with, like, obviously, like, they can, they can get to 80%, but those last 20% really matter. Um, and you can only get, like, tho- they, they matter to get into production. So that's why you need, like, this harness, right? So you need all those, like, integrations, memory, workflows, and sometimes you need vertical data, um, um, to do that. And like, like, like, the Pareto, Pareto principle, so, like, those 20% can make, like, 80% of the, of the effort. Um, or like they, they are making 80% of the effort. Um, so like, to all, like, those Fortune 500, 150 companies, so you can do this. Um, but then let's say, like, procurement workforce or AI procurement agents should then become, like, one of your core competencies. Um, and you should evaluate whether this makes sense or not for you, um, to, to have this in, in, as like your, your core skill.

10. 44:46 – 58:58

    When both sides deploy agents

    1. EBElena Burger44:47

       Um, Vlad, I'm curious if you're seeing suppliers start to use agents or AI at all, and kind of like what happens when both the buyers and the suppliers are, are fully AI-enabled?

    2. VKVladimir Keil45:02

       Mm-hmm. Yeah. So, like, we 100% believe that, like, in the future there will be, like, agents on both sides. Um, and, um, this, this makes so much sense. Um, but like surprisingly, what we, what we see is like-- So like when we look at the supplier side, this also kind of like equals the seller side, right? So what we see is like that the, that the sales side was like always ahead of the procurement side. Um, but what we're now seeing with those suppliers for those Fortune 500 companies, um, this actually is not true. So they are, like, maybe advanced in like, let's say, um, video recordings and, like, using tools like Granola and all, all, um, on all the stuff, but like, not like really, um, having agents deployed. They're automating the work. Um, and the cool thing is as, like, procurement is like, it's unsexy, right? So sales are sexy, procurement is unsexy, but it's like one process, and procurement is the, the counterpart. But now the good thing is, in those industrial companies, Fortune 5000 companies, procurement has the bigger power to the supplier. Because you, as an, like, cl- typical use case, like automotive supplier, you dictate to your suppliers what they should use, what the quality has to be, um, how they have to, how they have to answer to an specific RFQ. Um, so the opportunity is now if we serve the procurement department and then ca- and they can dictate what a supplier should use, um, why are we not like no- like, why aren't we, like, also pushing them to, like, Lio agents, um, that are also helping them automate the work, um, owning then both sides of, um, of the transaction?

    3. SASeema Amble46:41

       And Elena, I know we were talking earlier today about, you know, how can you have two parties on the same platform? Is-- How does that work? Um, I think it could even extend into, like, legal work, right?

    4. EBElena Burger46:52

       Mm-hmm.

    5. SASeema Amble46:52

       Being... And these are two very adversarial parties, right? But, like, if you have two law firms with clients with different interests, but both benefit from knowing, okay, here's the latest draft, here's where we are with the open issues, here are things that have been agreed upon, and just even tracking that. That doesn't, doesn't really exist right now, right? That's all being human-- that's being created by, um, humans. And so be- that coordination effort, an agent could be doing.

    6. EBElena Burger47:18

       Yeah. Yeah. It's, well, it's super cool to think about how, like, you know, both sides are kind of maybe evolving in tandem. One side might be going a little bit faster, as you're talking about, Vlad. Um, but over time, potentially people are just, like, on the same platform, and actually it's like, it's way better coordinated for, for everyone.

    7. VKVladimir Keil47:38

       Yeah. 100%. Because like, like, also like what we mentioned in the beginning, right? So like, like the obvious, like the obvious question is like, okay, but like we, we also talked a lot about like negotiation agents. What if like both parties have negotiation agents? And price is 100% the point where it's like a zero-sum thing, or like they have like different interests. Um, but we al- what we also discussed is that, like, price is like the outcome of like 5,000 different, like, other tasks that happened. And on those 5,000 other tasks, they have the same incentive. Um, sales wants to have as less, like, little friction as possible. Um, buyers want to have a really fast time to market, right? Again, like coming back to- Building aircrafts, building data centers. You want this data center to be built as fast as possible. You don't want to be building like six months later just because it takes so much time, um, to analyze all of the, um, responses from suppliers. Um, and you want to make the sale also fast. So, like, all of those 500-5,000 other tasks, the incentive is ex-exactly the same, and that's why you can deploy, or Lio can deploy agents also on both sides, um, doing... Like automating the work, um, of all those other tasks. Um, this is the beautiful thing.

    8. EBElena Burger48:55

       I think that there used to be this logic of, like, you shouldn't customize your software too much to, to one end user or one end customer. But I think something about LLMs and AI in general is, like, it might increasingly be possible to customize without slowing yourself down as a business too much. Um, so I'm curious if, if that is something that you're seeing, Vlad or Seema, um, and, and kind of what, what does that mean for, for end buyers of software?

    9. SASeema Amble49:29

       I think the overall principle, there's a lot of forward deployed work happening right now, and part of that is because the state of the customer data and understanding customer N, um, is a lot harder than understanding N plus one. And so we're sort of in the early stages of deployment overall, and that's why there's still a lot of humans as part of this product. Um, and by the way, that is something that is harder for the incumbents to do because they're also, like, not set up in, in a way to have even their, the way their product feedback cycle works, where they have a-- They have implementation teams, but that's very much an afterthought versus something that feeds into the product side. The beauty of AI is, A, it learns over time, so that's when we're talking about learning loops, and you have the right eval process. You can do more and more complicated jobs over time, and part of that is automating the deployment itself. And so you can, uh, and I'd be curious to hear how Vlad is doing it, but a lot of our companies even are doing that at, at, you know, at a rapid clip, where more of the customization, A, is being handled in an automated way, and B, the customer is able to turn the knobs and levers around customization via software, um, versus, okay, I needed to bring in, A, the original, it was like, you know, you brought in Accenture to do your SAP customization, and now it's like, okay, I've got a forward deployed team that's going to build some, you know, help build and spec it out, and ultimately it's going to be completely software driven.

    10. VKVladimir Keil50:58

        Yeah. Um, I mean, that's the reason why, like, when you look at the org structure of Lio, like 85% of the people are, um, engineers. Um, and even if you look at, like, the people where you, like, they don't have an engineering title, they have, like, mostly like an engineering background. And reason for that is, um, we obviously, like, we don't want to be a consulting company, right? So we make sure that we have, like, overall the best, um, agents in indirect and direct and finance, um, in those part. But then, like, as you mentioned, like there is a lot of like forward deployed work, um, to do if you go to enterprises because they have, like, different nuances in their, um, in their processes. Um, but how we work is we, as you mentioned, like building a product in a way where it's like, um, where we're reducing this customization, but also where it's a lot of like self-service. So the job of our, like, FDEs and forward deployed engineers, um, is on the one hand making it like self-service, but like in-internally it's like automating their own job, right? So like their KPI is, like you are seeing this happening like multiple times. Like you, like your, literally your job is like to automate yourself. Um, and then if you automate yourself, you go to the next task. Um, I think it's like also like an, um, approach that Google also is doing. Um, so like, but yeah, 100% agree with you. Um, and that's exactly what we are, um, what we are building, um, and what we, how we're doing it.

    11. EBElena Burger52:22

        There was recently a, a very big system of record event, and, and we're not, we're not talking about Dreamforce. We're talking about the, the Bots and Buyers Summit that, that Lio hosted in, in New York. Um, and, and just wanted to, to kind of hear, you know, stories from the ground and, and what, what you're seeing among buyers. What are people excited about? What are people looking ahead toward? What are people, you know, asking you for? Um, can you just kind of tell us some stories from, from that day and that event?

    12. VKVladimir Keil52:56

        So, I mean, this was the third time that we are doing, that we are doing this event. Now we did it in, um, in New York, just a few blocks from our, um, from our office here, and over 100 procurement leaders, um, arrived, right? And what we, like, we made sure that we, like when we do, when we do such events, that we only invite like high caliber people, right? So like C-level, CPO, vice, um, vice president. Um, and there are like two things very, like, very different, um, of, of, of how we do this or like why, why they are so amazed. So the first thing is, um, when we look at like how procurement used to work in the last 26, 27 years, um, a lot of tools emerged, right? So we can, like, they are like all those technology landscapes, and you can find them on LinkedIn, and you will see, like, there are like 500 procurement tools. But like, if you like, and this is also the reason, like, but when you talk to procurement people, like very painful, and like I challenge someone like to find someone who, like, loves to work with procurement. Like no one does. Like you can really state people hate working with procurement. And I'm talking about the requesters, and I'm talking about the suppliers, and even people in procurement hate working with procurement. Um, so like what's going on if there are like 1,000 tools? Um, and the reason for that is like the, all of the tools, um, um, they just made the, the process more efficient. That's, that's all. But they never changed how tho- how those people actually work. Um, and it's Like crazy to see that they work in emails and on Microsoft Teams, there's, and in Excel sheets and PowerPoints. It's like their main channel where they work on, and it's like zero, like zero AI enabled, um, obviously, um, um, in, in, in, in this part. And the, the other thing is like we, um, we give them like a very cross-department perspective on procurement, right? So we are not talking about like, look at this crazy invoice feature that we developed, but we're more looking like, like someone needs something, um, and in the end you have it on your table, and this can cr- be like across indirect, direct, logistics, finance, and you can see how we are doing it here. And we also putting it into like more into like a physical world because like AI agents, that's very abstract. Um, so what we are doing is we building up booths, and we, we even have this in our offices, um, in, um, also in New York, where you can walk through the booths and experience like, um, all of those agents, like really hands-on. Um, and that's what the, what's, what, what the people love. And it's like, um, I mean, the next event, um, will be with around 700 people. Um, so you can imagine how, how crazy this, this grows. Um-

    13. EBElena Burger55:36

        Procurement people gone wild. Yeah. Yeah. It's gonna be-- It, that one is in, um, Munich?

    14. VKVladimir Keil55:42

        Um, yeah.

    15. EBElena Burger55:43

        Yeah.

    16. VKVladimir Keil55:43

        So we're doing them like in, um, in Europe, Munich, and in New York, um, all the time. Yeah.

    17. EBElena Burger55:50

        Cool. If you're listening and in procurement, you know where to go. [chuckles] Maybe w- I guess one question, uh, one question from me. What do you think it takes to get, uh, to, to get people excited about procurement? Is it, is it the agents? Is it the people? Is it the time save? Like what? Or like something else, like y- you mentioned it, like procurement is one of these things that I, I, I remember, you know, people aren't, they don't like, like it's like a universally kind of disliked low NPS area. I remember talking to a guy who was head of procurement like, I don't know, seven or eight years ago as I was looking at this category, and he was like, "Ugh, I hate talking about this product."

    18. VKVladimir Keil56:27

        [chuckles]

    19. EBElena Burger56:28

        "Like I, I use, you know, this legacy system of record. I'm on Coupa, and I don't, I don't wanna buy anything else. I don't wanna talk about it. It's fine." It was like the most disgruntled customer call I've ever done out of like millions of them. Um, but I'm, I mean, I'm curious, uh, yeah, what it is that you think, you know, really gets people excited about this category.

    20. VKVladimir Keil56:47

        Yeah. So I mean, it's like, it's like, and this is also why, why, why I like procurement is like, it's on one hand like, so like the reason like why we started in procurement are like, it's not essentially like what happened, but like how people react to it, right? So if you talk to the p- to the people, they're like really frustrated. So this means it's an like highly emotional topic, but it's like, like let's be honest, like, um, B2B SaaS, okay? But it's like highly emotional. Um, so that's a good thing. And then when, if you combine this with, um, like something which is boring and niche, um, this is also an advantage because, um, again, like the, it's um, it's like also easy or easy for us for, to amaze those people, right? Because like the, the really last revolution they have seen is like 20 years ago. Um, and then maybe a nicer user interface 10 years ago. Um, but nothing else happened. Um, and so you have like boring, highly emotional, and then plus crazy business impact, right? So like it feels like it's unnecessary, but like, like I, I told you like some examples. So like it has like obviously crazy P&L impact, but it has impact on like the whole economy, right? So we're like, we are talking about like how data centers are built, how aircrafts are built, how cars are built, um, how drones are built. So it's extremely important that, um, you have a fixed procurement process, um, not only to like make it happen and build something, um, but then also when you talk about like when we, when you look at like the competitive landscape. So to get like 1% margin increase, um, you need to make 10% more revenue and 10% more sales. Um, so like if you just manage to get like 1% savings, it's like equals like 10% of sales, um, that you have to do, um, to get the same outcome in your P&L. Um, so like it's tremendously important. And like you combine all of those three things, and then you have like, um, a trillion-dollar business opportunity. That's, that's my opinion. Like for procurement, but there are probably also like other things that are like emotional, boring, and have a crazy business impact.

    21. EBElena Burger58:54

        Well, Vlad, thank you so much for joining us. Uh, this was a ton of fun.

Episode duration: 59:13