No Priors Ep. 25 | With Palantir's CTO Shyam Sankar

1. 0:00 – 10:17

   Palantir's CTO Discusses Company's Background

   1. SGSarah Guo0:00

      (instrumental music) . Palantir has built a data-driven application platform over the past two decades that is used by governments, militaries, and some of the world's largest companies for analytics and operational decisions. Palantir recently announced a new platform, AIP, as part of a push to invest in AI. This week on the podcast, Sara and I talk to Palantir's CTO, Shyam Sankar. He was the company's first business hire, and has led the company for nearly two decades previously as the COO. Shyam, welcome to No Priors.

   2. SSShyam Sankar0:32

      Thanks for having me. Great to be here, Sara and Elad.

   3. SGSarah Guo0:35

      So, I think you have a very unique background. I believe you grew up in Nigeria and then moved to the United States. You got interested in computers reasonably early. It'd be great to just hear your personal story and your background.

   4. SSShyam Sankar0:45

      Y- yeah. I, uh, spent the first three years of my life really in Nigeria. My f- my father had, uh, built the first pharmaceutical manufacturing facility on, on the, on the continent. Uh, until then, all the drugs were really imported. Um, and we fled Nigeria during some violence, and really resettled in the US as, as kind of like refugees. So I, a, a great deal of gratitude, understanding the counterfactual reality of, like, you know, h- how the world could have ended up there. And so I grew up in Florida. Uh, I think relevant to the current age, I grew up in a time where when the space shuttles would launch, we would all file out into, uh, the recess courtyard to actually just watch it. You know? And that seemed...

   5. EGElad Gil1:22

      Cool.

   6. SSShyam Sankar1:22

      ... quite normal. And it also seemed really normal that on Saturday morning at like 6:00 AM you'd be woken up to double sonic booms every now and then. Uh, and so I'm, I'm eagerly awaiting the return of, of the space age that, that commercial space and new space have been bringing back to us here. Uh, but I made my way out to Silicon Valley in, uh, in 2003, and started getting involved with startups. I, my, the first company I worked at was Xoom with an X, uh, that was founded by Kevin Hartz, uh, and it was an international money transfer company. And then after three years at Xoom, I started at Palantir as the 13th employee. Uh, really the first person on the business side, and, and have had the most fantastic ride ever since. But never been more excited about what we're doing than, than what we're doing right now with, with AIP and the opportunities that are in front of us.

   7. SGSarah Guo2:07

      How did you originally find Palantir? 'Cause it, you know, it was a very secretive company very early on. It was sort of a very small community in technology in Silicon Valley that actually heard of it. So I was, I was just sort of curious, like, how you connected with the company and got involved?

   8. SSShyam Sankar2:19

      One of my, uh, friends... Oh, so Peter was also a seed fund investor in, in Xoom, so I got some exposure there, and onene of my friends was actually a freshman year roommates at Stanford with Joe Lonsdale. And so I had heard about this company that was, you know, very small at the time, and it really talked to my heartstrings. My uncle was a victim of, of terrorism, um, I was in New York during 9/11, and it just, like this was gonna be... I, you know, would rather fail on working on a problem this important than, than do anything else.

   9. SGSarah Guo2:50

      Yeah. And then you joined, as you mentioned, as employee number 13, the first business hire, et cetera, and you've had a variety of roles over time. Could you explain a little bit how your role has changed over the lifetime of the company?

   10. SSShyam Sankar3:00

       I think the, the kind of common thread through it all is just doing what you need to do, which I know sounds like a banality, but really when it, when it first started, I like wrote our first kind of candidate management system. I mean, w- what are you doing as a business hire...

   11. SGSarah Guo3:11

       Yeah.

   12. SSShyam Sankar3:11

       ... before you have really a product here? And I was, I was, uh, an aggressive QA tester, you could say, but the, the real initial contribution was what we call forward-deployed engineering. It, it comes from an insight that Alex kind of had around like, well, you know, he, he, he muses, "Why are French restaurants so good?" Well, maybe one theory is that the wait staff is actually part of the kitchen staff there. You know, it's, it's not... The, they're, they have like deep context and understanding of the food. And so the, the forward-deployed engineering idea was that the, the people who are going to be interacting with customers in the field were gonna be computer scientists who could actually understand, uh, what does the product do today? What does it need to do today? How is it... Under what conditions is it gonna work? And, and how do you kind of create this hybrid role that's product management, customer success, and engineering all in one? And that, that's really the team that I first built up. And then as we went from Gotham, and Foundry, and, and now AIP, there, there's a lot to, to do there. There, it's like whether it's interacting more often with the customers, thinking about the technology, uh, or really thinking about, how are you going to get to value? And I think a lot of what forward-deployed engineering really gets you to think about is, how do you do these things backwards? Instead of really going forward from the technology, how do you work backwards from the problems that your customers actually experience, and use that to create an accountability function? Like, is what I'm doing mattering? D- did I make the life of this customer better today? Can I do better tomorrow? And using that absolute standard to judge yourself rather than saying, you know, "Did my software work according to spec?" Like, who cares about the spec? Who cares about what my ambition was yesterday? What's my ambition today, and why shouldn't it be bigger tomorrow?

   13. EGElad Gil4:46

       That framework is super practical. Was it the customers that you were working with early that drove the need for this? Were you trying to solve a particular problem?

   14. SSShyam Sankar4:56

       Well, and I... The customers we're working with in government, it's just motivational, right? Like, you kind of view it and you're, you're thinking about it and like, "How do I walk as many miles in their shoes as possible?" Like, uh, y- how could I possibly just think my job is done 'cause I check the box here? It's like the, the job is defined by, how are they doing, and what more could I be doing for them? Then I think there's a more kind of cynical component of this, which is like, "Okay, well, are you as a company going to succeed in this sort of environment, particularly with early government customers, if you don't have that sort of mentality?" Because i- if you think about the vertical stack you need to deliver your outcome, you're dependent on so many things going right. So if you just want to build the software at this component in the stack, if anything down here isn't working, and y- you know, certainly at, at the time we were doing this, there, there really wasn't AWS yet, but you couldn't depend on any of that stuff working. And so your, your visualization of what you would need to own so that you succeeded needed to be, uh, quite ambitious.

   15. SGSarah Guo5:49

       So before, um, AIP, the company had three key platforms, Gotham, Foundry, and Apollo.Could you tell us about what these different things do for the sake of our audience, since they're less familiar necessarily with the company? And then, how does AIP fit into this?

   16. SSShyam Sankar6:03

       Yeah, Gotham is our flagship government product. It's, it's really focused on intelligence and defense customers, and it helps them integrate and model their data to really drive decisions in the context of their enterprise. So, in the defense community, they'd be thinking about in terms of the kill chain, how do I go from targets to effects on those targets? In intelligence, it's, it's often kind of a different sort of structure, but how do I track and gain context and understanding of the things in the world that I need... I have a responsibility to understand about? You can think of Gotham as kind of conceptually at the highest part of the stack. Much of Gotham then depends on Foundry. Foundry's our general-purpose data integration platform. It allows you to deal with structured and unstructured data, to transform that data, to really treat data like code, and then drive that through to an ontology, a semantic layer that models not only the nouns of your enterprise, like the concepts that you think about, but the, the verbs, the actions. And so I think, you know, the buzzword for this often, digital twin, and that can mean a lot of things to a lot of people. But, you know, how do I have some sort of conceptual understanding and model of what we do as a business and use that to actually affect decisions? And then drive that all the way through to the application and decision-making layer. So, not dashboards that give me visibility, but really pixels that I can make changes, right? So, if I want to allocate inventory, I need a, a platform that's gonna allow me to write back to SAP or read and write from my transactional systems and orchestrate my enterprise. And Foundry, I think, really gives its customers the ability to kind of squint and model their enterprise as a chained series of, of decisions, or kind of like a decision web. And then giving them the modeling and simulation ability to understand and ask counterfactual questions. What happens if I do this? And this is the same platform that was used to build the COVID vaccine response distribution in the U.S. and the UK. Uh, same platform that commercial companies were using to manage the supply chain crises when suddenly steady state kind of equilibrium wasn't really there, and being able to model the counterfactuals became really, really crucial.

   17. SGSarah Guo8:00

       Yeah. It's, it's kind of interesting that you mention those various customers, for example, on the COVID, uh, vaccine distribution side or things like that because, you know, the perception of the company is that very early on, a lot of the earliest customers were intelligence and defense, and then it kind of broadened from there. Is that a correct assessment? And was that intentional at the time, or was it just you found that there's a pocket of customers that really cared about your product and, you know, were a good fit for what you were doing initially?

   18. SSShyam Sankar8:24

       Well, yeah, we founded the company to, to work with intelligence and defense organizations. And, and really, I think, um, we expanded almost reluctantly. You know, I think it was like 2010 or 2011 when we started working, uh, with our first commercial customer. But really what we realized was that it took something as sexy as James Bond to motivate engineers to work on a problem as boring as data integration (laughs) . But this sort of... We had our own ideas of what would be valuable in these spaces, and we built software for it. But all of those ideas kind of presupposed that the data was integrated. And it... and, you know, it kind of... I think the kind of popular view is like, this is a boring and solved problem, but I think it, it might be kind of a boring and highly unsolved problem that people are kind of, like, duct taping together everywhere they go. And so by productizing a solution to that, we r- kind of expanded our market and, and the... what we could... sorts of problems in the world that we could go after. Apollo's a... It's quite an interesting platform as well. So, like, we really originally built Apollo for ourselves. If you think about our customers, the... we're deploying in air-gapped environments. So, how do you deploy modern software when, you know, you can't CI/CD to the target? We had to build this entire infrastructure that allowed us... 'Cause, you know, our software, it's modern software. We have 550 microservices. We're releasing multiple times a day for each one of these services. They need to be able to upgrade independently. But also, our environments are complicated. Like, hey, the submarine only wants to upgrade on these windows, or-

   19. SGSarah Guo9:45

       (laughs)

   20. SSShyam Sankar9:45

       ... the-... and these environments are not connected to the internet. So, how is that gonna happen? We had to build kind of what we think of as kind of a successor to CI/CD, which is like autonomous software delivery and deployment. So, Apollo allows you to think about your software and the environments you're deploying in separately, model the dependencies, and kind of hand that to Apollo to manage and orchestrate the upgrade. It will understand, what's your blue/green update pattern? How do you think about health checks? How do I roll forward? How do I roll backward? How is that integrated with my, uh, understanding of vulnerabilities and CVEs? When do I need to, like, recall

2. 10:17 – 20:25

   Apollo and AIP

   1. SSShyam Sankar10:17

      software or block software? And that software has started to get a lot more traction as people are dealing more and more with complicated environments. Not only air-gapped customer environments in defense and intelligence, but if you go to Europe where people have a strong push towards sovereign SaaS, or a lot of people want you to deploy inside of their VPC, as a SaaS company, how are you gonna manage now having a thousand customer environments to manage? And, and Apollo just makes that really easy.

   2. SGSarah Guo10:44

      And then how does, uh, AIP tie into this? And can you tell us more about AIP and that initiative?

   3. SSShyam Sankar10:50

      Yeah. AIP allows us to... it, it's... it was really a core set of technologies that allow you to bring LLM-powered experiences to your private network on your private data to, to drive the, the decision-making. Everything from, how do I integrate this data? But I think much more interestingly, how do I build these AI-enabled applications? You know, it's like an application forge. And a core part of the proposition here is that these LLMs, they need tools. Like, th- certainly, there's something quite magical about this kind of non-algorithmic compute. You know, it's, it's neither human thought nor kind of traditional computer science. And they're very good at what they're good at, and they're also quite bad at what they're bad at. And, and, and so, like, getting this right is really about providing... sometimes people call it plug-ins, but I, I think tool might be a more appropriate word. But like, how do I give my customers not just a tool bench, but really a tool factory to go make, uh, the tools they need to get the most out of LLMs? Like, an LLM is not gonna know anything about orbital simulation or weaponeering or, um, predicting forward inventory 30 days from now. It's certainly not gonna do that well. But with the right tool, it's gonna do that quite excellently, and it's gonna give you a lot of leverage on the workflows that you already have. Uh, and, and-I think, uh, in some sense, much of the foundational work that we've done with, with Foundry has enabled people to run really quickly with AIP, build copilots that deploy into their existing decision-making surface area. Like, how do I allocate inventory? How do I adjudicate claim, auto claims? And then, um, get that efficiency in, in weeks.

   4. EGElad Gil12:19

      At what, what point did you decide you wanted to make a big investment in LLMs, and, and sort of what did the company do first?

   5. SSShyam Sankar12:26

      Uh, I think it was, it was really around Q4, you know, the la- the last part of the last year there, where it just felt like, obviously, the LLMs are exciting, but what was more exciting to us is it felt like the LLMs were just waiting for something like ontology. You know? It's like, to really get the value out of the LLM, the way that we had modeled the world, it's almost like accidentally, we had spent the last 20 years really thinking hard about dynamic ontologies, how you model them, why they're valuable to humans, and you can kind of think about the ontology as, uh, as having this semantic layer that gives you an incredible amount of compression that you're putting into the context window and allows you to build, um, LLM-backed functions in very reliable ways. And I think part of this is just, like, recognizing the LLMs, they're, it's more like statistics and calculus. And I think this is one of the impedance mismatches for a lot of engineers who are working on them. They're kind of like, they model them a little bit like calculus, a lot like calculus, and then, you know, when it works, it works magically, and when it doesn't work, it falls off a cliff. So, how are you actually gonna get this to work when you kind of have this, like, stochastic genie now? I think you're gonna need a kind of a whole s- tool chain around that, that kind of presupposes it's a stochastic genie. And, uh, I think the ontology's one of these things that massively grounds your LLM in your reality, in your business context, and allows you to manage that without having to, you know, change the model itself.

   6. SGSarah Guo13:47

      What, what are some of those components that you think are the tool chain that you need to sort of model the stochastic genie? And I love that phrase, by the way.

   7. SSShyam Sankar13:53

      (laughs)

   8. SGSarah Guo13:53

      I think that's a really good way to put it.

   9. SSShyam Sankar13:55

      So, you're, you're probably gonna need everything you kind of need with a dev tool chain. But you're gonna have to adjust it for the fact that it's stochastic. So, uh, you even see, like, people call it eval and not unit tests. But you're gonna need, like, how many unit tests do you need? If you're gonna write an LLM-backed function, and it's a stochastic genie, how, how many times does it need to execute before you have confidence that it's gonna do what you want? And then, so then you can think about that, that's like day zero, okay? So I, I build this thing, how do I think about it? But what sort of telemetry and production log data do I need? Uh, and w- and how, how often am I gonna be looking at those traces? And it's like, I might even be writing unit tests against my traces. I guess you could call that, like, a health check, right? And, like, there's gonna be, um, a lot more emphasis that, that you're gonna need there as an engineer as you think about using this. And then there's gonna have to be some calibration on the use case. The best use cases are gonna be ones where, when the LLM gets it right, there's massive upside. And when it doesn't, it's a no op, right? Uh, and, and so picking those ones, I think, are, are gonna be quite important as you build and tune the specific applications of these.

   10. EGElad Gil14:57

       The, going back to this idea of an ontology, I feel like I suddenly understand this much better in that there are a lot of companies right now trying to figure out how to take all of their, uh, messy, less-than-perfectly-integrated and largely unstructured data and create some sort of intermediate representation that the models can handle well. And if you have something like an existing ontology of your business, then leveraging it with LLMs does feel like a, like a really natural, magical fit. That's exciting. Um, maybe to make it a little bit more real, you could walk us through an example of, like, an AI tool using this tool chain ontology that w- you're excited about, that one of your customers is building with you?

   11. SSShyam Sankar15:40

       Yeah, sure. I'll just pick an e- an example that, something we worked on recently in Hawaii, which is, how do I do automated, um, COA generation, co- courses of action generation, from an operational plan? You know, so the Department of Defense has these O plans, they call them, that are kind of like... th- th- that's the other thing. You have these industries where there's just a tremendous amount of doctrine, whether that's pharmaceuticals or defense, where there's so much knowledge in how we want to do things that's been written down. Um, so you have this, this O plan that describes, um, the phases of a potential conflict or the key risks and assumptions. And so you, you might want to do something like a noncombatant evacuation operation. So if, if conflict happens here, how will we get all the civilians out of a, a city? Um, okay. Well, we've thought about that, and we've written that down. It's in this document. And so how do I then just say, like, "Build me, uh, a, a course of action to drive this evacuation?" Well, the plan has specified the resources that you're probably gonna need, the, the types of resources, the phases, the timing of it, uh, the risks and assumptions you need to, to worry about. So then how do I take those words and then hydrate the application state that people use to manage the common operating picture? And that's a big part of what we're really thinking about, which is, you know, I kind of think of, like, chat is a massively limiting interface. You know, at the limit, prompts are for developers. Now, I think that's, it's, it's really hard. Some, you, you know, prompts leak over to users, and, you're, and users sometimes want to chat, and lots of people start with that because of the popularity of ChatGPT. But really, what I want to do i- in this context is b- they're entering a question, like, "Generate a course of action allow- that is this evacuation operation." And what I'm getting back is not words. I'm actually getting a, a map with resourcing, a resource matrix, and the requisition of the necessary resources, and you know, I can hit a button and say, "Yes," and that, that comes to life. Uh, and, and so those are the sorts of experience that we're starting to build with customers now. Uh, and on the commercial end, it's, it's really copilots to help people. We, I, I was just looking at, uh, a demo this morning from my team on helping, uh, a m- a major auto manufacturer adjudicate quality in claims. So how do I manage the cost of quality, cost of warranty on the production line and post-production when these cars are in service? Well, I, I need to be able to cluster these claims and more efficiently understand what, what supplier subcomponents, where in the supply chain, and how do I remediate these issues? How do I drive down my cost of, of warranty and recall? So building copilots that are kind of looking at the text of the claims, u- understanding the components, helping them identify, um-... early indicators and signs of the, kind of, conditions under which these parts need to be recalled and managed. And there, it's really about human agent teaming.

   12. EGElad Gil18:12

       I was listening to the, um... I think you guys hosted, like, an AIP day, like a, a set of, like, demos and presentations recently, and one of the, um, s- points of view you take is that the models themselves are increasingly commoditized, and, and certainly more broadly available and available in open source. How do you think about the value that Palantir is building for its customers?

   13. SSShyam Sankar18:35

       Well, I think it's, uh, how do you, how do you actually use the models to drive these experiences? So, there's kind of two ends of that. One is the existing experiences that people have today. So, h- how do I build a, a copilot that's going to help me adjudicate auto claims, um, or help me understand my production process? On, on the other end of that, it's like, how do I develop trust in the underlying models? You know, if we go back to the stochastic genie here, maybe we should actually think about these as, like, slightly deranged, um, mad geniuses, and then, you know, are you gonna only ask one of, uh, one of these experts to help you solve your problem? Like, h- how do you think about the configuration of, of mad geniuses that you'd actually really want to have? And I think there's-

   14. EGElad Gil19:14

       I want an ensemble of mad geniuses. I'll feel better about that.

   15. SSShyam Sankar19:17

       Yeah, I, I think that's, that's probably the correct direction, and I, I think one that, that, that model companies will have a hard time with, right? Because I think they, they need to have, kind of, one model to rule them all or directionally that's, that's where it needs to go. But if you're an enterprise and you're thinking about I'm, I have high consequence decisions that I'm trying to drive here, how do I do that? And I think, um, certainly if you're living in a chat world where the output's chat, that's valuable, and you want to think about, like, where, where do people agree and disagree? But if you start thinking about actually the output of the model as, uh, a DSL or JSON, that becomes even easier then. Now you, you can actually parse these things in very structured ways to understand not only the consensus view, but maybe divergent views. And then, then, then you're more, kind of, more authentically treating this as, as statistics and not calculus, and then that forces you to flow that through to the UI and, and how you're designing this for actual human users to interact with it in a way that's thoughtfully statistical.

   16. EGElad Gil20:08

       So, you've said that you don't think that, you know, chat is the be all end all, and it's quite limiting, especially when you think about, like, these complex workflows, the outputs that are, um, most efficient for your end users. Uh, what are you imagining in the near term that turns into? Is

3. 20:25 – 28:29

   Future of UI and Application Integration

   1. EGElad Gil20:25

      it assistance, um, overlaid on the existing software that your customers use, whether it's things they've built with you or, uh, software they already have? Is it just automatic workflows and outputs in that software? Um, how... Help us imagine it a bit.

   2. SSShyam Sankar20:41

      Yeah. I, I think the ideal, uh, visualization of it is, is something like I have an application state and I have an intent as a user. Uh, combine those two things to give me a new application state. Now, that can be very hard. I think it, uh, depending on the workflow, it... that might be super obvious. I think if you're thinking about this like, like a copilot for GitHub Copilot, it's more obvious. I have an application state and intent and, you know, you, you generate something for me. Now, it becomes less obvious when some of these things get a little more complex and you may need a little bit of hinting and a little bit of, of user prompting kind of... That's where I think the art is. So, so that, that would be, that would be one piece of it. Then, then another piece of it is like, okay, let's just say that's too hard, like, it... or doesn't fit the use case properly, not just, not just about it being hard. You have the prompt that is then gen-... the return is JSON or DSL that is manipulating the application state, right? The whole point is not to give me answers, but to change my app. And then that starts changing how you think about interacting with these things. It becomes a new UI layer. You know, kind of the, the most extreme version of this is like, why have any UI at all? If you have really beautifully done APIs and you have, let's call it data APIs and ontology, your, your, your... the data that's actually going between these APIs is incredibly well modeled, I think you can actually use the LLM to generate a lot of the experiences that you want as an end user.

   3. SGSarah Guo21:59

      Some people have been talking about this in the context of, um, you know, everything becomes a sort of programmatic agent driven world, where five years from now or 10 years from now, you just have agents that represent you as a user with a specific task interacting with other agents or APIs, and to your point, you really minimize the UI dramatically. Do you think that's, that's the most likely future? Or how do you sort of think about where all this stuff is heading from a UI perspective?

   4. SSShyam Sankar22:21

      It's, it's hard to see so far in the future on this, but what I think it definitely does... When I think about the integration layer, like when I... like, you look at, like, the Gorillas paper, and, and can... you can teach... you know, can you fine tune an LLM to basically tell you what API to call with what parameters? Like, yes, it turns out. And so, okay, so if that's true, what does system integration look like in the future? That's going to be quite different. So then I think it allows you to create more single panes of glass that are actually truly integrated, which is incredibly hard right now. I think there's some subtle and interesting benefits here. Like, one of the consequences of a hack we, we had a number of months ago was that I had an engineer who could build a feature, uh, in a couple hours that we had previously scoped. It was on the road map. It was a feature that was going to take, like, two months and two people, and it's just simply because the amount of UI that was involved was so intensive. You just replace the UI with language, the whole thing changes. So like, that's, that's, that's one way of thinking about it. Okay, well, what sort of UI are you not building today that you actually don't even have to build today? Uh, that you... you probably have the tools, the primitives in the backend or the application that you can now surface. So I think that's... that's an interesting place to go there. Like, I, I, I don't know about the extreme view of like, "Look, there's going to be no UI. Just build every UI custom every time." But I think the ability for a user to get the last mile to be what they need is going to be really powerful. It's a big unlock, and I certainly think in the enterprise context we live in, I see that all the time where there's kind of... there's no perfect solution for all the different, kind of, tugs you're getting from, from the customer to, to generalize that, or the cost of generalizing it is, is so high that you can't actually meet the need. But now they, they can make it specific, actually, to their needs.

   5. SGSarah Guo23:52

      Yeah, the systems integrator point that you made I think is super interesting because I think there's a lot of companies where part of their defensibility is the fact that you basically had to munch specialized data or integrations, you know, that would be the SAPs of the world or different ERP systems, Workday, et cetera. Like, a lot of these things have moats in part because it takes six months to implement them and to integrate against and customize them. And to your point, this can really simplify a lot of the- those things down in a pretty performant way. So it, it's this pretty interesting macro shift that you're undoubtedly seeing firsthand in terms of where your customers are going.

   6. SSShyam Sankar24:24

      I, I think so. I think it's gonna change a lot of things. If you think about, like, how much control it gives the customer in terms of, like, how do you manage your API surface area? Like, people have tried doing this with, with a- like, API buses or bridges, and it's like n- none of that stuff's really kind of working. 'Cause just having all your APIs in a big list, that turns out doesn't help you. But having something that allows you to think about how you string them together is, is, is pretty transformative. And you think about some of these big systems. I was... You know, yesterday, I was seeing a note where... I'm sure it's somewhat dramatized, but Boeing and the Navy are fighting about data rights for, you know, the F-18 Hornet. You know, but that's kind of, like... This whole problem is just a consequence of the fact that you have... Eh, that it's hard to do this with the data rights. And that actually, the, the more that we get to a world where you can fine-tune an LLM, that is the F-18 production design LLM, it, that's, that's a very different world.

   7. EGElad Gil25:12

      What do you imagine doing with that?

   8. SSShyam Sankar25:14

      Well, how you manage... Wha- So, the ability for a third party, let's say the government, it doesn't want to be locked in. It's like, "How c- how can I interrogate the design specs?"

   9. EGElad Gil25:23

      Mm.

   10. SSShyam Sankar25:23

       Or, um, "How do I understand how I'm going to do maintenance here?" So, how, how much of that is just kind of locked up because the expertise is so difficult to acquire? You know, so it's part of the... When I'm, when I'm turning over my first plane, am I also turning over the LLM? Is that part of the value proposition that, that helps you manage this? And that... The kind of heart and pipelines behind that, and the training behind that. So, h- how do I give you more leverage over something that's insanely complicated?

   11. EGElad Gil25:44

       Yeah.

   12. SSShyam Sankar25:45

       And you think about this system, it's, it's assembled. It's a, it's a complex subsystems behind it that are, have all been integrated together. Like, how were they integrated? And what if I want to swap out a component in the future, how do I reintegrate that? And so that's the part that's really hard, and, and it maps to an enterprise. If you think about it and say, "Okay, I need to swap out Workday for something else." Like, what is it gonna break? What does it all touch? How do I, how do I do that? And I think that's gonna get a lot easier.

   13. EGElad Gil26:06

       Shan, can you talk about some of the, uh, technical challenges that you guys feel like you need to solve, or that you're working on now for customers to make this work?

   14. SSShyam Sankar26:16

       Uh, I think the, the, the key one is more conceptual. It's, it's realizing that it is a stochastic genie. So, you know, where I wanna invest the most in the tooling is, is really around, um, kind of, like, a robust eval IDE environment that enables people to unlock the value of this. And, and, like, how do I develop trust and put these, these kind of copilots through probation? How do my users think about that? Uh, another big part of the investment right now is in making these sort of copilot models accessible to everyday users. I th- I think there's a fair amount of, of companies I see going after kind of, let's call it the, the canonical data scientist as an archetype of like, "I wanna fine-tune a model and I'm gonna go do that." I see a smaller number trying to go after devs as an archetype. But I wanna go after the head of maintenance, um, at, at an institution who's like, "Look, I, I know all these things, and I, I don't need all the knobs exposed to me. But I, I, I need you to have an opinion on all those knobs underneath that. And I can get my whole team to help generate the Q&A pairs." And I... Like, there, there's a lot that I'm willing to put into this that revolves around my expertise. Help me get a copilot in production that's affecting the lives of my users. And I, I think that's, like, just a har- like, a lot of hard systems integration. Like, how do I integrate? Like, all those building blocks exist. How do I make that a, a very smooth workflow that you can trust and, and actually use? And I, and I don't want to feel like I've... Like, we- we've solved all the kind of, like, statistical differences here. I think that's... Eh, the more you kind of pull on that thread, the more you realize like, okay, like this, this way of approaching it, that would have... What we would have done with traditional code, you actually have to account for. You need more defensibility in your thought process here, because it's, it's not traditional code.

   15. EGElad Gil27:55

       Do you feel like you are doing a lot of customer education on how to, like, absorb this? Or is it more like you need to do that work in the product so that they don't have to understand, like, the stochastic output?

   16. SSShyam Sankar28:06

       We have to do both. Um, and, and even internally, like, you know, it's like as engineers kind of ramp with it. The more they play with it, you know, it's like the more they kind of, "Okay, I'm starting to get my hands around the genie." Um, so, so I think part of it is like, a lot of customers aren't seeing past chat right now. Everything, everything that's interesting is kind of like a chatbot. And I think some of the most sophisticated customers, it actually scares them. Because they're like, "Well, you know, like, how could I

4. 28:29 – 31:22

   Investment in Co-Pilot Models and Education

   1. SSShyam Sankar28:29

      trust the output, the, the textual output of this thing to make a decision of this sort of consequence?" And, like, getting them to realize that actually we're... You know, you should be thinking about how this, this manipulates your application layer and how you can be using that. And then how do you validate the outputs of that? That, now, that's kind of fitting into the context of your state machine, which is probably my, my biggest comment on agents, is like, I almost can't use that word. Because it, the connotation of it has, has come to be the agent has to come up with his own arbitrary kind of plan. Like, the planning is the exoticism of the agent, as opposed to really the practicalities of an enterprise is there's either an implicit or explicit, and often it's kind of 50/50 or some combination, implicit/explicit state machine that represents that enterprise. So, the idea that you're just gonna have an agent that kind of comes up with a plan and does things is, is, it's not gonna meet reality. But the idea that you're gonna have an agent that has context of, of a part of the, the state machine, understands its authorities, the guardrails, the left and right, of what, what states am I allowed to manipulate? And you probably wanna start pretty small, like one state. You, you have authority over one state transition. That's it. Uh, and then how do I build that up so that I'm linking these together to drive the real automation? And that's gonna map pretty cleanly to the humans you have in the enterprise. Like, there's a human who probably owns that one state transition. And so now you're naturally building these human agent teams, and you're kind of upgrading or promoting your individual contributor to being a manager of agents. Uh, and, and that's, that's a pretty safe way from a change management perspective. It generates the log data I need to have trust that this is actually valuable and helpful and it's assisting the agent. And it, it's probably more akin to how Tesla has tackled, um, self-driving, as opposed to, you know, Cruise. Like, big long shot, we're just gonna go all or nothing. It's like, actually, we're gonna get a little more self-driving every single day.

   2. EGElad Gil30:11

      Yeah. Sort of end-to-end magic planning, we can figure out one state transition at a time with an existing ontology the business understands, and, um, and get feedback on, along the way. That, that feels really promising. Especially with, as you mentioned, like, things like the Gorillas paper. If you're helping people fine-tune to their own data, um, even a small amount of that seems to, like, dramatically increase, um, quality of tool choice and such. So, sounds really exciting.

   3. SGSarah Guo30:37

      One of the things that you've mentioned a few times is really different ways that engineers or people on your team have gotten familiar with the technology and the capabilities of it. And I feel like it's... LLMs are very non-intuitive, relative to both traditional engineering but also traditional ML. You know, I feel like a lot of organizations have kind of had to adapt to thinking differently a little bit in terms of what are the actual capabilities, and how does it work, and where does it fail and, you know, where doesn't it? Were there specific things that Palantir did early on to onboard people to sort of this new way of thinking or have people play around with the models in specific ways? I mean, you mentioned the hackathon. I'm just sort of curious how this all got started and, you know, how you now incorporate it into how people think about these problems.

   4. SSShyam Sankar31:17

      Yeah. We- we've made it a huge organizational focus really to experiment and play with these things. So

5. 31:22 – 38:19

   Exploring AI Implementation in Various Industries

   1. SSShyam Sankar31:22

      like, so how could you bring this to your own, um, area of the product? But as importantly, like, how do we build this into our tool chain? So, hey, we are doing incident response on our stacks. Can... let's- let's have... like, let's build a co-pilot for ourselves to go manage that more efficiently. And so by trying to solve your own problems with it, you get much stronger intuition of, like, where it's amazing and where it falls off a cliff and how you have to think about that as- as you build it. So aggressively adopting it to drive our own productivity has been one dimension of it.

   2. SGSarah Guo31:52

      And who- who came up with that mandate? Like, who was the person who said, "Okay, let's go do this"?

   3. SSShyam Sankar31:57

      That was really my push. You know, it's like, I think that's part of the- the FDE mindset though. You- you know, it's like, it's- it's... no credit to me. It's like it's- it's almost a obvious consequence of our culture of, like, what's the ambition? How do we aggressively dogfood everything? And like, uh, if it doesn't work for us, why would it work for anyone else? And so then I think the- the other part of this is, like, we live in a world where we can't count on GPT-4 everywhere. Like, we don't have that unclassified environment, right? So it- it- it is a beautiful kind of easy button for lots of problems to go after. But then when you start having to use, um, open-source models, you're like, "Oh, this one works in this- in this context for these sorts of problems and th-" So then how do I start exposing engineers to that? 'Cause I think one... maybe one of the easiest ways to understand the models is to use multiple models concurrently and understand the outputs of it. And so kind of like our internal version of ChatGPT isn't one model. It's actually multiple models and you're- you're a bit... you're able to evaluate the output of each of these, how long it took, um, the- the amount of tokens it's putting out. You're able to control and tune it and kind of almost inductively explore the service area of these models.

   4. EGElad Gil32:58

      One thing that was a claim that Alex made that is, um, a wonderful level of ambition is that Palantir as a company is aiming for the entire market share of AI. What- what does that mean? What does that vision look like?

   5. SSShyam Sankar33:10

      I mean, I- I think it's- it's exactly what he said. It's like... we certainly, um, think that we have done the- the necessary pre-work essentially, like the foundational technologies that we've built that allow enterprises to securely up r... to protect their data, to bring these LLMs to their private networks, and then to deploy them operationally, to get beyond kind of the dabbling and the innovation. I- I can harken back to a period when- when data was quite early. Everyone had something like a- a data innovation lab. They're not calling it generative AI innovation lab, but it's kind of structurally similar right now where people are kind of really working hard to think about what use cases and how will it be valuable if you think about this from the customer end. Uh, and actually it's like... I'll tell you what use cases, the same use cases you were working on a year ago. Like, the problems haven't changed. You know, you need to be applying these technologies to the- the problems that are the most important problems in your business, uh, and- and where you have... we've already made those investments and how do I manage and model your digital twin? And I... you know, how do I already connect up the different decisions you're making together? So like if you think about this kind of like connected company to that decision web idea, no decision is truly independent. It's, you know, like the cons... the decisions you made upstream from it affect it. That decision you're about to make is about to have severe consequences for the decisions that can be made downstream. So I... if I can bring that visibility to you, you're actually in some sense simplifying the problem in terms of how much... how hard of a problem am I asking the LLM to solve, and how incrementally can I deploy these things to go faster? And I think that's the compounding loop. So I... we feel like the- the value is really going to accrete to folks who own the application layer in the enterprises, and we're gonna go after that very hard.

   6. SGSarah Guo34:48

      So one- one thing we've talked a little bit about are some of the customer use cases on the DOD side or, you know, more general sort of defense and, um, related areas. And one area that I know that Palantir added quite early on as a vertical was healthcare. And you mentioned some of the work that you did during COVID. I know that last year, uh, Palantir announced, I believe it was a 10-year partnership with Cleveland Clinic to improve patient care. And when I look at the implications of LLMs and generative AI to healthcare, there's so much low-hanging fruit because it's such a big, people-intensive services industry. It'd be great to just hear your viewpoint in terms of how you work with some of these healthcare customers and what you think this coming wave of AI will do. Like, what are the areas that will be most impacted by that?

   7. SSShyam Sankar35:30

      Yeah. Hea- healthcare is roughly a third of our business. It's- it's certainly... I mean, it's pro- probably one of the fastest growing parts of our business as well. Uh, and- and we do that, uh, you know, in- in a number of countries. So the NHS in the UK and- and multiple hospital systems in the US, and across both kind of dimensions of clinical care and operational care, like the opper... hospital operations. And I think that's relevant 'cause the- the pace of adoption for- for these will, uh, vary and- and kind of the challenges you solve for the use cases with LLMs is different between them. I- I think the operational context is- is very obvious in the sense that it's just like operating any institution really. You ha... you have kind of supply, demand. You have labor inputs to that. You're trying to manage that so that you can deliver the product, the care that you actually have. And there it fits very cleanly to how will we help, you know, auto companies get better at what they're doing or how will we help manufacturers or energy companies. Um, and- and there I- I think probably the archetypal pattern that I see across all industries is something like you today have something, if you squint at it, it looks like an alert inbox where, uh, your state machine is essentially saying, here's an exception or something that I need someone to think about. And the human kind of... you... then you have the... so many exceptions, I need some help prioritizing all these alerts, and then you prioritize them and you- and you deal with them.What the promise is of, of, of LLMs and what we're focused on with AIB is turning that from a place where I'm surfacing alerts to a human, to I'm surfacing solutions. Uh, so instead of saying, "Here's an alert, what should we do about it?" It's, "Here's a recommendation. Here's a staged scenario of what you could do about this alert that's happened. Do you approve or reject it?" And that's a concrete manifestation of, of the, uh, like, kind of co-pilot. Uh, and, and what I really like about that is, like, it... To the point of, like, having done the foundational work, for that to really work, you need a primitive that is the scenario, that is this, like, staged edit. You know, like a branch, uh, in, in, in GIT, right? Like, without... That- that's a very powerful primitive. Without it, you, you lose a lot of capability if you have to build that at every customer over and over again. So having, having something that the LLM can say, "Look, here is a branch, and here is a staged set of edits," and then I can have a human evaluate that in the operational context of how they view, uh, h- hospital capacity. That, that's, that's one set of workflows. And then on the clinical side, I think it's really about reducing human toil. Like, I, I don't think you're trying to get the, uh, LLM to decide what, you know, it... What to do for the patient here. That's, it's probably exactly the domain of, of the doctor here. But what's in the clinical records, in the clinical histories? How do I drive the workflows? What is it that the doctor can't get to, or the, or the nurse staff can't get to today because it's, it's just too much toil? And that we can turn that into something that takes, you know, maybe 400 milliseconds. That's gonna improve what's happening at the point of care, so driving completeness in that picture. Uh, and, and I kind of see that as a natural dichotomy between operational and analytical workflows. You know, the other, the other thing I was looking at today is how do you optimize

6. 38:19 – 39:45

   Operational and Analytical Workflows in Context

   1. SSShyam Sankar38:19

      the throughput of a state machine? So, like, I was looking at this claims processing workflow, and then I was looking at this, like, claims optimiza... Like, "What's wrong with my state machine?" is almost the question. And th- this one... The second one is for, like, a manager who's looking down at this and they're saying like, "Oh, there's a cycle here, and what's..." You know? And so this, the sorts of manipulations you're trying to do with the LLM is, it's structurally more analytical. You're not asking it to change the state machine. You're not asking it to... You know, there, there's no magic button there to press. In the operational context, you can get closer to something that's more like, "Give me a recommended action that I can evaluate as a human." And then there's kind of thresholding and learning over where might that be most valuable. And I certainly think one of the things that's promising about that is today we're so constrained by, is it worth solving this alert? You know, because where, where are my human costs to go after solving this alert? In a world where the LLM can process all the alerts and give you a staged set of actions, now you're, now you're prioritizing not on the severity of the alert, but on the possible consequences of the solution. Uh, so that, that's already an improvement in the sort function. And then you're much more likely to be able to get through all of them.

   2. SGSarah Guo39:20

      That's a really useful framing. Yeah. Is... I think that covered all the things we wanted to talk about. I mean, it's a really great overview of what Palantir is doing, and some of the really exciting initiatives and customers that you work with. Thank you so much for joining us at No Priors. It was really a lot of fun and, uh, a lot that we learned. So thank you so much.

   3. SSShyam Sankar39:34

      Sarah, a lot- thanks for having me. It was great.

   4. SGSarah Guo39:36

      Thank you. (instrumental music)

Episode duration: 39:45