AI Agents Are Killing the Engineering Pyramid — Here's What Replaces It

Diana Hu 0:00 [upbeat music]

Diana Hu 0:04 Well, today I'm excited to have Reynold Xin, who's the co-founder and chief architect of Databricks, which is one of the largest AI data infrastructure for enterprises, companies out there. Their last round was at over 130 billion-plus in valuation. Pretty impressive.

Reynold Xin 0:24 Thank you, Diana.

Diana Hu 0:25 And the big shift right now is AI coding agents are working. How is that changing for how Databricks is building products internally?

Reynold Xin 0:37 Yeah, I think one of the things that's super interesting right now is, uh, i- in a way, the AI agents are reshaping the organizational structure because it used to be the case that you have to have humans... You, you build out this pyramid of a engineering team for any mature product. Um, you have, so maybe the manager, the leader, and then the senior engineers, and since they have an army of more junior people, um, they will be contributing code and fixing a lot of bugs. And I think that is changing because the, uh, AI agents are... When designed well, when we have the right harness, it's capable of actually doing a lot of the coding work, um, and in some cases even the design work. So I think it would actually reshape the, uh, or- organizational structure to be a little bit more of a I shape, where, uh, I think teams will become more and more actually in a way top-heavy, um, and have people that really understand what needs to be built, but also how to build them, um, whereas leaving a lot of the grunt work to be done and completely automated by AI agents, and that is a huge implication, I think, to the, uh, both the tech and the, uh, org structure.

Diana Hu 1:42 Interesting. So have you seen then the product velocity of how you guys are shipping to be a lot faster?

Reynold Xin 1:50 I think one of the interesting thing here is that, um, by the way, when, uh... I think it's a good analogy here, which is when steam engines, when the world first discover electric motors or sort of invented electric motors, um, mo- many of the factories were built with steam engines in mind. In the case of steam engine, like people could actually Google this, they had a... I- it's, it's actually very difficult to build a lot of different steam engines. They are pretty big, pretty bulky. So the factories are designed with one gigantic steam engine in mind, and there's a lot of conveyor belts and stuff surrounding the steam engine, so they're very tight, they k- um, pack into a 3D structure. But when electric motors came out, one of the biggest changes, hey, you can build fairly small electric motors. You no longer need a single gigantic electric engine or s- steam engine. But, uh, because the existing factories are configured in a way to fit a single gigantic, um, steam engine, uh, most factory just started by replacing that steam engine with a electric motor, uh, more powerful electric motor. That actually only led to fairly

Reynold Xin 2:50 incremental gains, um, of factory sort of throughput. Um, and over the course of like two or three decades, people start... engineers started thinking about, "Hey, how do we redesign the, um, factory for electric motors?" And that's when really unleashed the productivity gain and throughput from, uh, factories. I think a similar thing is actually happening with the, uh, software, uh, factories, um, as well, and one of the, uh... I- it's actually a lot easier to create a new software factory, um, fully embracing sort of AI tools than just-

Diana Hu 3:23 From scratch

Reynold Xin 3:24 ... from scratch than just taking an giant existing system and slab a bunch of AI in it. You'll still get some incremental gains. There's a lot of tasks that can be automated. Um, but if you just think about without changing the processes and without changing maybe all the tooling and how your CI/CD works, it's actually, it's very, very difficult to get a massive pr- um, productivity gain.

Diana Hu 3:45 That's a very good point. So especially for companies that have been around longer, and with this shift with AI, they have to be very thoughtful on how to do that so they don't end up with this analogy you're saying with a giant-

Reynold Xin 3:56 Yeah

Diana Hu 3:56 ... hole in the middle that used to be the steam engine to then retrofit with AI. What you're saying is almost like to embrace AI for a company that's already further along, you have to almost create new space.

Reynold Xin 4:09 Exactly. Um, it's... One thing is, uh, by the way, don't get me wrong, it is important to replace that giant steam engine with a electric motor also, but that won't give you all the gains.

Diana Hu 4:20 Mm-hmm.

Reynold Xin 4:20 The more important part is to start thinking about how do you reconfigure things. But reconfiguration is slow because you don't want to be too disruptive either. So it's actually a lot easier to create, for example, new teams, new efforts, new product lines, new organizations to be more AI native compared with, uh, maybe the existing bigger machine.

Diana Hu 4:38 Hmm. So tell us a bit about some of the products that Databricks is shipping that is enabling more native AI coding agents to interact with.

Reynold Xin 4:48 Yeah, um-

Diana Hu 4:48 Which might not be some of the products that people are know you as much for.

Reynold Xin 4:51 Yeah, exactly. Um, actually one of our fastest growing product is something that doesn't even have a Databricks brand on it. Um, it's, it come from the Neon acquisition, and the, uh, the whole point of the Neon product is that it's a PLG-driven motion. It's super easy to sign up, so very different from the traditional Databricks enterprise motion.

Diana Hu 5:08 What is Neon actually?

Reynold Xin 5:09 Uh, yeah, I was gonna... And Neon give you a serverless Postgres, um, that auto scales, um, super, super rapidly and also allows you to take a snapshot of the database and restore and branch off the database just like you can do with code. One of the thing with AI agents is that, um, AI agents move incredibly fast, and you can use AI agents to run a lot of experiments in parallel. Many of these experiments might not work out. Some of them might. And for the ones that don't work out, you want it to be super, super cheap. For the ones that might work out, you actually want to be able to run on the infrastructure that can scale you to the point of going to production, um, at scale. And, uh, I think historically, sort of infrastructure, especially databases, were designed to be heavyweight. Uh, they were think of to support, hey, the most mission critical applications. Um, but Neon's approach is, hey, let's design something that is super, super cheap, um, because you can start very, very small just getting a Postgres database. But, um, and if you want to branch off, run a lot

Reynold Xin 6:09 of experiments, you can do that instantly. But if any one of the experiments actually start taking off and becomes maybe what you want to go into production, you just use the same environment and actually go, uh, autoscale to whatever you need. Um, so that's like... Actually, Neon's been growing, um, like crazy. Um, it's... When we acquired Neon just about, uh, actually less than a year ago, the revenue has gone up more than 10x, um, just in less than a year, and, uh, we're seeing massive adoption, I think mostly because of the agentic workloads. They are very, very different from the, uh, past workloads.

Diana Hu 6:42 Is it because a lot of, um, sort- AI coding agents, or if you go on ChatGPT or Claude and you ask for, "Help me build this with a Postgres database," Neon becomes the recommended product?

Reynold Xin 6:54 Yeah, that's, uh, one of the key reasons. Another one is, um, it's also powering, um, many sort of agentic coding platforms like Replit, Vercel, and many others that are coming in the pipeline. Um, for many of this platform, they suffer from exactly the same issue I talked about earlier, which is they want each individual experiment or each app to be super cheap.

Diana Hu 7:15 Mm-hmm.

Reynold Xin 7:15 Um, but then if they do take off, they want to be able to take it to a pretty far scale. And that, um, it's just a difficult problem from a sort of conventional infrastructure point of view.

Diana Hu 7:25 This is fascinating because you guys, Databricks, have been really the hardcore infrastructure company, and infrastructure historically is really heavyweight. It's meant to be done for really production grade and hardcore engineering systems and studio systems. And in this new world, in this shift when AI coding agents started to work, there's this new thing with lightweight infra that's becoming a thing, and sounds like Neon is one of them.

Reynold Xin 7:49 Yeah. I think more generally and broadly than Neon, I do think infrastructure needs to evolve in the agentic era, which is, uh, it needs to be able to start super lightweight. It can't be this sort of a delicate thing that requires an army of people to babysit and costs, like, millions of dollars-

Diana Hu 8:05 Mm

Reynold Xin 8:05 ... um, for every little thing. Like, it, it needs to be able to support even at approximately zero cost to begin with, and when it does, whatever that's being built on top of it, um, gen- like, actually demonstrates value, um, then it can actually start scaling up the cost.

Diana Hu 8:22 What advice would you have for founders that are s- getting started and building for this new world and having this concept of lightweight infra rather than the old school heavyweight?

Reynold Xin 8:31 Yeah, I think it's actually a great time right now for in- sort of disruption infrastructure, honestly, um, because pretty much every piece of infrastructure was designed to be super heavyweight. Even the word infrastructure sounded super heavyweight, right?

Diana Hu 8:44 Mm.

Reynold Xin 8:44 It reminds you of PG&E and-

Diana Hu 8:46 [laughs] Yeah

Reynold Xin 8:46 ... maybe oil pipelines. Um, the... A- and I think there's a lot of architectural evolution with the cloud and with agents that you can now actually start thinking about it. Many of infrastructure became super heavyweight, not necessarily because there's a fundamental sort of technical limitation. It's mostly because it was designed initially just for high value sort of services. But now I think, um, with agentic coding, there's gon- and maybe the individual value of this, each service or each experiment is very low, but in aggregate they can be very large. So there's sort of a, um, opportunity to target the very long tail, um, to be building something pretty valuable, and that's just something most incumbents have never even thought of, so v- have a very difficult time transitioning to.

Diana Hu 9:28 Awesome. I mean, this, this sounds exciting future for everyone building. I think that's-

Reynold Xin 9:34 Definitely.

Diana Hu 9:34 Thank you so much for coming and chatting with us, Reynold.

Reynold Xin 9:36 All right. Thank you, Diana. [outro music]