AI Agents Are Killing the Engineering Pyramid — Here's What Replaces It

1. DHDiana Hu0:00

   [upbeat music] Well, today I'm excited to have Reynold Xin, who's the co-founder and chief architect of Databricks, which is one of the largest AI data infrastructure for enterprises, companies out there. Their last round was at over 130 billion-plus in valuation. Pretty impressive.

2. RXReynold Xin0:24

   Thank you, Diana.

3. DHDiana Hu0:25

   And the big shift right now is AI coding agents are working. How is that changing for how Databricks is building products internally?

4. RXReynold Xin0:37

   Yeah, I think one of the things that's super interesting right now is, uh, i- in a way, the AI agents are reshaping the organizational structure because it used to be the case that you have to have humans... You, you build out this pyramid of a engineering team for any mature product. Um, you have, so maybe the manager, the leader, and then the senior engineers, and since they have an army of more junior people, um, they will be contributing code and fixing a lot of bugs. And I think that is changing because the, uh, AI agents are... When designed well, when we have the right harness, it's capable of actually doing a lot of the coding work, um, and in some cases even the design work. So I think it would actually reshape the, uh, or- organizational structure to be a little bit more of a I shape, where, uh, I think teams will become more and more actually in a way top-heavy, um, and have people that really understand what needs to be built, but also how to build them, um, whereas leaving a lot of the grunt work to be done and completely automated by AI agents, and that is a huge implication, I think, to the, uh, both the tech and the, uh, org structure.

5. DHDiana Hu1:42

   Interesting. So have you seen then the product velocity of how you guys are shipping to be a lot faster?

6. RXReynold Xin1:50

   I think one of the interesting thing here is that, um, by the way, when, uh... I think it's a good analogy here, which is when steam engines, when the world first discover electric motors or sort of invented electric motors, um, mo- many of the factories were built with steam engines in mind. In the case of steam engine, like people could actually Google this, they had a... I- it's, it's actually very difficult to build a lot of different steam engines. They are pretty big, pretty bulky. So the factories are designed with one gigantic steam engine in mind, and there's a lot of conveyor belts and stuff surrounding the steam engine, so they're very tight, they k- um, pack into a 3D structure. But when electric motors came out, one of the biggest changes, hey, you can build fairly small electric motors. You no longer need a single gigantic electric engine or s- steam engine. But, uh, because the existing factories are configured in a way to fit a single gigantic, um, steam engine, uh, most factory just started by replacing that steam engine with a electric motor, uh, more powerful electric motor. That actually only led to fairly incremental gains, um, of factory sort of throughput. Um, and over the course of like two or three decades, people start... engineers started thinking about, "Hey, how do we redesign the, um, factory for electric motors?" And that's when really unleashed the productivity gain and throughput from, uh, factories. I think a similar thing is actually happening with the, uh, software, uh, factories, um, as well, and one of the, uh... I- it's actually a lot easier to create a new software factory, um, fully embracing sort of AI tools than just-

7. DHDiana Hu3:23

   From scratch

8. RXReynold Xin3:24

   ... from scratch than just taking an giant existing system and slab a bunch of AI in it. You'll still get some incremental gains. There's a lot of tasks that can be automated. Um, but if you just think about without changing the processes and without changing maybe all the tooling and how your CI/CD works, it's actually, it's very, very difficult to get a massive pr- um, productivity gain.

9. DHDiana Hu3:45

   That's a very good point. So especially for companies that have been around longer, and with this shift with AI, they have to be very thoughtful on how to do that so they don't end up with this analogy you're saying with a giant-

10. RXReynold Xin3:56

    Yeah

11. DHDiana Hu3:56

    ... hole in the middle that used to be the steam engine to then retrofit with AI. What you're saying is almost like to embrace AI for a company that's already further along, you have to almost create new space.

12. RXReynold Xin4:09

    Exactly. Um, it's... One thing is, uh, by the way, don't get me wrong, it is important to replace that giant steam engine with a electric motor also, but that won't give you all the gains.

13. DHDiana Hu4:20

    Mm-hmm.

14. RXReynold Xin4:20

    The more important part is to start thinking about how do you reconfigure things. But reconfiguration is slow because you don't want to be too disruptive either. So it's actually a lot easier to create, for example, new teams, new efforts, new product lines, new organizations to be more AI native compared with, uh, maybe the existing bigger machine.

15. DHDiana Hu4:38

    Hmm. So tell us a bit about some of the products that Databricks is shipping that is enabling more native AI coding agents to interact with.

16. RXReynold Xin4:48

    Yeah, um-

17. DHDiana Hu4:48

    Which might not be some of the products that people are know you as much for.

18. RXReynold Xin4:51

    Yeah, exactly. Um, actually one of our fastest growing product is something that doesn't even have a Databricks brand on it. Um, it's, it come from the Neon acquisition, and the, uh, the whole point of the Neon product is that it's a PLG-driven motion. It's super easy to sign up, so very different from the traditional Databricks enterprise motion.

19. DHDiana Hu5:08

    What is Neon actually?

20. RXReynold Xin5:09

    Uh, yeah, I was gonna... And Neon give you a serverless Postgres, um, that auto scales, um, super, super rapidly and also allows you to take a snapshot of the database and restore and branch off the database just like you can do with code. One of the thing with AI agents is that, um, AI agents move incredibly fast, and you can use AI agents to run a lot of experiments in parallel. Many of these experiments might not work out. Some of them might. And for the ones that don't work out, you want it to be super, super cheap. For the ones that might work out, you actually want to be able to run on the infrastructure that can scale you to the point of going to production, um, at scale. And, uh, I think historically, sort of infrastructure, especially databases, were designed to be heavyweight. Uh, they were think of to support, hey, the most mission critical applications. Um, but Neon's approach is, hey, let's design something that is super, super cheap, um, because you can start very, very small just getting a Postgres database. But, um, and if you want to branch off, run a lot of experiments, you can do that instantly. But if any one of the experiments actually start taking off and becomes maybe what you want to go into production, you just use the same environment and actually go, uh, autoscale to whatever you need. Um, so that's like... Actually, Neon's been growing, um, like crazy. Um, it's... When we acquired Neon just about, uh, actually less than a year ago, the revenue has gone up more than 10x, um, just in less than a year, and, uh, we're seeing massive adoption, I think mostly because of the agentic workloads. They are very, very different from the, uh, past workloads.

21. DHDiana Hu6:42

    Is it because a lot of, um, sort- AI coding agents, or if you go on ChatGPT or Claude and you ask for, "Help me build this with a Postgres database," Neon becomes the recommended product?

22. RXReynold Xin6:54

    Yeah, that's, uh, one of the key reasons. Another one is, um, it's also powering, um, many sort of agentic coding platforms like Replit, Vercel, and many others that are coming in the pipeline. Um, for many of this platform, they suffer from exactly the same issue I talked about earlier, which is they want each individual experiment or each app to be super cheap.

23. DHDiana Hu7:15

    Mm-hmm.

24. RXReynold Xin7:15

    Um, but then if they do take off, they want to be able to take it to a pretty far scale. And that, um, it's just a difficult problem from a sort of conventional infrastructure point of view.

25. DHDiana Hu7:25

    This is fascinating because you guys, Databricks, have been really the hardcore infrastructure company, and infrastructure historically is really heavyweight. It's meant to be done for really production grade and hardcore engineering systems and studio systems. And in this new world, in this shift when AI coding agents started to work, there's this new thing with lightweight infra that's becoming a thing, and sounds like Neon is one of them.

26. RXReynold Xin7:49

    Yeah. I think more generally and broadly than Neon, I do think infrastructure needs to evolve in the agentic era, which is, uh, it needs to be able to start super lightweight. It can't be this sort of a delicate thing that requires an army of people to babysit and costs, like, millions of dollars-

27. DHDiana Hu8:05

    Mm

28. RXReynold Xin8:05

    ... um, for every little thing. Like, it, it needs to be able to support even at approximately zero cost to begin with, and when it does, whatever that's being built on top of it, um, gen- like, actually demonstrates value, um, then it can actually start scaling up the cost.

29. DHDiana Hu8:22

    What advice would you have for founders that are s- getting started and building for this new world and having this concept of lightweight infra rather than the old school heavyweight?

30. RXReynold Xin8:31

    Yeah, I think it's actually a great time right now for in- sort of disruption infrastructure, honestly, um, because pretty much every piece of infrastructure was designed to be super heavyweight. Even the word infrastructure sounded super heavyweight, right?

Episode duration: 9:45