Skip to content
No PriorsNo Priors

No Priors Ep 106 | With GitHub CEO Thomas Dohmke

This week on No Priors, Sarah and Elad talk with GitHub CEO Thomas Dohmke about the rise of AI-powered software development and the success of Copilot. They discuss how Copilot is reshaping the developer workflow, GitHub’s new Agent Mode, and competition in the developer tooling market. They also explore how AI-driven coding impacts software pricing, the future of open source vs. proprietary APIs, and what Copilot’s success means for Microsoft. Plus, Thomas shares insights from his journey growing up in East Berlin and navigating rapidly changing worlds. Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @ThomasDohmke Show Notes: 0:00 Introduction 0:37 GitHub Copilot’s capabilities 4:12 Will agents replace developers? 6:04 Copilot’s development cycle 8:34 Winning the developer market 10:40 Agent mode 13:25 Where GitHub is headed 16:45 Building for the new challenges of AI 21:50 Dev tools market formation 29:56 Copilot’s broader impact 32:17 How AI changes software pricing 39:16 Open source vs. proprietary APIs 48:01 Growing up in East Berlin

Sarah GuohostThomas DohmkeguestElad Gilhost
Mar 13, 202550mWatch on YouTube ↗

EVERY SPOKEN WORD

  1. 0:000:37

    Introduction

    1. SG

      (instrumental music) . Hi, listeners, and welcome back to No Priors. Today, we're joined by Thomas Dunk, the CEO of GitHub, a platform used by over 150 million developers worldwide to collaborate and build software. As CEO, Thomas has overseen the development of tools like GitHub Copilot. Before becoming CEO, he helped shape GitHub's product strategy and power its global expansion, and previously worked at Microsoft. In this episode, we'll talk about the future of software development, the role of AI in coding, open source, and product plans for Copilot. Thomas, welcome to No Priors. Maybe we can start with the meat of

  2. 0:374:12

    GitHub Copilot’s capabilities

    1. SG

      it. What is happening with, um, Copilot and, uh, the new releases at GitHub recently?

    2. TD

      You're heading straight, straight into it. Um, we're really excited about, you know, making Copilot more agentic. A few days ago, we announced the agent mode, uh, in, in Copilot and VS Code. Um, so instead of just, you know, chatting with Copilot and getting responses and then copy and pasting the code e- into the editor, or, or using autocompletion, the original Copilot feature, you can now, uh, work with an agent and it helps you, you know, to implement a feature. And when it needs to install, like, a package, it, it shows you the command line, terminal, commando, and you can just say, "Okay, run this." Um, you're still in charge, right? So that's the k- the crucial part of, um, uh, these agents that we have available today. That as the human you're still, uh, as the human developer, you still need to be in the loop. Uh, but we also showed, you know, uh, a teaser of what's about to come in, in 2025. Um, we call this, uh, Project Padawan, you know, because it's like a Jedi in a Padawan. You, you gotta have patience and you gotta, you know, learn how to use The Force. Um, but we think, you know-

    3. SG

      (laughs)

    4. TD

      ... uh, in 2025, we get into a place where you can assign a GitHub issue, a well-defined GitHub issue, um, to Copilot, and then it starts creating a draft pull request and it outlines the plan and then it works through its plan. And you can, similar to how you observe, uh, a coworker, like you can see how it commits changes into the pull request and you r- can review this and, uh, and provide feedback t- to Copilot. And so, it, uh, Copilot b- basically graduates from a pair programmer to a peer programmer that, that becomes a member of your team.

    5. SG

      The obstacles to that right now are some new model advancements. Is it just building out some other core technology? Is it just the UI? Like what, what is keeping that from happening right now?

    6. TD

      Yeah, I think the first thing is the model, uh, the full o3 model that's not available yet, but OpenAI showed, um, uh, as part of the Shipmas, uh, uh, right before the holidays. We're going to see, you know, improved reasoning. And I think it's as the models get better in reasoning, um, we're going to get closer to 100% of this V-Bench, which is that benchmark, uh, out of 12 repos, um, uh, open source Python repos. Um, a team in Princeton identified, uh, 2,200 or so, uh, issue pull request pairs. Effectively all the models and agents are measured against. And so that's number one, you know, the, the model and the agent combination. I think the second piece is just the figuring out what's the right user interface flow. Um, if you think about the workflow of a developer, right? You f- you have an issue that somebody else filed for you, you know, user, quarker, product manager, or something that you filed yourself. Now how do you know whether you should assign Copilot to this, um, the agent, uh, to it, um, or, or whether you sh- need to refine the issue to be more specific, right? It's, it's crucial that the agent is predictable. That you know that this is a, a task the, the agent can solve. If not, then you need to steer it. So steerability is the next thing, and to either, you know, extend the, the definition, um, uh, or the agent needs to come back to you and, and ask you additional questions. And then at the end of the process, you wanna verify the outcome and, and so in our demo, that's where we're thinking the right flow here is actually that the agent works in a pull request, like similar to a human developer with loads of commits and then you can roll back those commits or, or check them out in, in, in your VS Code. We saw that with, with some of the agents that are available is that, do I, as a developer actually tolerate (laughs) the agent? Like, is it actually saving my time or is it wasting my time? And the, the more often you see it wasting your time and just, um, uh, burning compute cycles, the less likely you're going to use it again. And so if you are predictable, steerable, you know, verifiable and tolerable. If we get to that f- for all four criterias to a certain level, I think we're going to see a, a wide, um, adoption of, of agents.

    7. SG

      How

  3. 4:126:04

    Will agents replace developers?

    1. SG

      far away do you think, um, these agents are from being sort of the median programmer equivalent? And then how much longer do you think it takes to get to sort of superhuman?

    2. TD

      You know, I thought about this this morning, right? Like what, um, if regardless of what agent, uh, you're thinking of, a travel agent or a coding agent, or maybe it's an agent that designs your house, the, the fundamental challenge is actually the same as, as you as, have as a human, uh, developer, right? Like, you have this big idea in your head, uh, and you can sketch it on a whiteboard, but then you wanna start coding and you have to take this big idea and break it down into small chunks of work. I think that's the part where we are far away from, from agents actually being good enough to take a very rough idea and, and break it down into small pieces without you as, as, um, as developer or as architect or, or even in plan- when planning your travel, constantly getting questions back of what decisions you wanna make. You know, what database for cloud. Like imagine you give the agent a task saying, you know, build GitHub or, um, build a mobile app or something. Like it will just be not specific enough, right? So that's the systems thinking that I think the median developer will not, uh, be replaced by agent. And the flip side of that is a lot of, you know, what developers do is just picking up issues and fixing bugs and, and finding where to fix the bugs, adding a feature that comes from a, from a customer. And then you have to navigate the code base and figure out what files you have to modify. And I think there we are going to see dramatic, uh, uh, progress over the year. We actually, you know, when we record the demo for the Padawan project, um, uh, we actually had one of our product managers use an issue and, and the agent create the pull request, uh, uh, themselves, right? And so a PM that usually doesn't code and doesn't, uh, write code in the code base, um, was able to use the agent to create a real pull request that was then reviewed by the developer and, and merged into the code base. So in some ways we're already there. In other ways, we, um, need to get to the point where you trust it enough, um, that, that you're using it day in, day out.

    3. SG

      I'm sure you guys

  4. 6:048:34

    Copilot’s development cycle

    1. SG

      were doing a bunch of dogfooding before releasing agent mode.

    2. TD

      Yeah.

    3. SG

      And how to run as well. Maybe if we just, uh, sort of zoom out from that, from the eval phase. Like can you describe what the overall like development cycle is for Copilot today? Like how you do planning and make decisions about what to try and, um, how you improve it?

    4. TD

      The, uh, industry calls now AI engineering, um, which is, you know, we have extended the full stack, um, of backend and, and frontend development with AI development. And, and so how do we, uh, use a new version of a model or new model as, as we have now the model picker in, in Copilot. We are constantly dealing with multiple models from multiple vendors. How do we integrate that into, into our stack? We have, you know, an applied science team that runs, uh, evaluations. Um, uh, we have a team that builds out these, these benchmarks, um, that the applied science team t- uses to, um, compare models with each other. But also the teams that build new features like code review agents or the SWE agents or agent mode users to, to validate their work, um, as, as part of their test suite, right? So it's, it's no longer just the, the data scientists and the engineer that those roles have, you know, uh, uh, more and more overlap and, and they're collaborating, uh, day in, day out. Um, we do a lot of experimentation, uh, with A/B testing, um, where we flight, uh, new versions, uh, or new fine-tuned versions of a model. Um, uh, after the offline testing or an online test first with, uh, GitHub and Microsoft employees and, and then with sets of the population.And then overall, you know, obviously we have a roadmap, um, of features, um, you know, that we wanna build, um, in a long backlog, um, not just for Copilot but all up for GitHub, right? Like, um, GitHub is, um, you know, turning 18 this year. I think the- it's 18 years that since the founders, uh, in late 2007 started working on it and then it launched, um, uh, in early t- 2008 and, and Microsoft turns 50 actually, uh, uh, April 4th. Um, and so we have a long backlog of, of, of customer feedback where we're using Copilot to build those features, you know, in, in agent mode now, um, you know, to, to accelerate our feature delivery. But at the same time, you know, the market is moving so fast, uh, whether we're m- meeting with OpenAI or with Entropic or with, with Google, we learn about new, new model versions and then our roadmap changes, um, from one day to another, you know? And, and, you know, I'm sure you guys are seeing that as well, like the market is moving so fast where we're literally sitting on an exponential curve of, uh, of, of innovation, uh, that is hard to keep up and you can't really plan more than, than a month or two a- ahead of time.

    5. SG

      What do you think about competition being on that exponential curve? I think, like, it is wild to think that,

  5. 8:3410:40

    Winning the developer market

    1. SG

      uh, you know-

    2. TD

      (laughs)

    3. SG

      ... SWE agents as you describe them, like, didn't exist as an idea a year ago. You know, we now have a market full of folks experimenting with these products. Um, how, how do you think about winning the... like GitHub is obviously a very dominant force overall as is Copilot, but how do you think about, you know, winning the developer over and what they care about in that, um, you know, changing and competitive market?

    4. TD

      The way we think about winning is that, you know, we care deeply about developers, um, and that's always been, you know, the heart of GitHub is that we put developers first and that we are developers that are building products, uh, for developers. You know, we have the saying at GitHub is that, uh, we're building GitHub, uh, with GitHub, uh, on GitHub using GitHub, right? And so everything that, that we do in the company including, you know, the- our legal terms and our HR policies and, um, our product management sales, um, sales enablement, all these functions are in GitHub issues and GitHub discussions and GitHub repos. So I think that's number one that we deeply care about our own product and we're using it for everything day in day out. You know, the first thing I do in the morning is open the GitHub app on, on my mobile phone and, and then Slack, um, uh... as a lot of our, you know, operations company chat runs through Slack. Number two is, you know, y- you mentioned competition. I mean, it's obviously like they've never seen anything like that in the developer space. It's, it's the most exciting time I think for developer tools. And you know I've been a developer for, uh, over 30 years. It- it's amazing to see, um, uh, the innovation, you know, um, the, the news, uh, that is coming out every day and I think that energy, you know, is in the market and i- innovation-driven both on the open source side and on the closed source side, right? Let's not forget, you know, that it's not one-sided. Uh, as much as there's innovation on proprietary models and, and software, there is as m- as equal innova- amount of innovation in open source and, uh, on GitHub. And, and that energy obviously gravitates i- into, into us. I'm a big Formula 1 fan and, like, it's go- it's good when there's competition because the races are so much more fun to watch if there's multiple teams that can-

    5. SG

      (laughs)

    6. TD

      ... that can win the championship. And I think the, the same, we feel about the competition. It, it gives us motivation every single day when we wake up to do better, uh, to move faster and to, to ultimately win with the best product in the

  6. 10:4013:25

    Agent mode

    1. TD

      market.

    2. SG

      You have such, like, rich data about how people are actually using Copilot. What is surprising you even from the last, um, week or so since, uh, agent mode was released?

    3. TD

      The thing that always surprised us, um, from, from the early days was how much code Copilot is writing. You know, some of the folks, uh, from Microsoft and GitHub on, on your podcast, uh, in, in the past and, uh, in the early days, you know, s- soon after we launched Copilot preview, uh, it already wrote like 25% of the code. And I remember that meeting where we looked at this in a product review and I said, "That must be a mistake in the telemetry. Go, go back and validate that it can't be true that it's writing 25% of the code." Because it was just autocompletion and, you know, as, as cool as that was, at the same time, you know, you... it still ma- made a lot of mistakes i- in the early days. Um, but it, it quickly dawned on us that A, the number is true, and B, that's just the learned behavior of software developers, right? Like, you're typing something, you're always reaching the point where, uh, you, you need to look something up so you go to your browser and, and you find code on, on Stack Overflow or on Reddit or blogs or, or on GitHub and then you're copy and pasting that and you're modifying it anyway afterwards, right? Like, that's the, the inner loop is always this kind of like you, you write something, you try it out with the, um, with the compiler and debugger and then you keep modifying until you make it work. Uh, and that number, you know, then quickly rose to, to around 50% depending on the, on the programming language. Um, if you look now, um, uh, you know, um, with, with these agents it's, it's hard to measure that because, you know, when you, you can literally go into agent mode and say I wanna, uh, you know, build a, a snake game i- in Python and it writes all the code for you, right? Like it writes multiple files so the denominator becomes zero, right? Like (laughs) -

    4. SG

      (laughs)

    5. TD

      ... it's like infinite percentage because you ne- you... the only thing you wrote was, was a prompt and the 15-minute demo from, from two years ago is a one-minute demo now. And I think that's, you know, uh, still surprising in many ways that we, we are al- already so far ahead on, on that curve. Um, and then the, the opposite is also true, right? You can, you can get it into a place where, you know, it just keeps rewriting the same file or deletes the whole file because it gets stuck somehow in, in the logic and, and so it grounds us also in the reality. It's like we are not close to an agent just autonomously parsing through all my GitHub issues and, and fixing all my backlog for me. The only thing I'm really doing is just validating, um, and, and becoming the code review human, uh, uh, for, for this software development agent, right? Like so we are in this, um... you know, we're swinging between the excitement of how much it can already do and the reality where it gets stuck in very simple scenarios where it's like you're trying to kind of like figure out the prompt of telling it, uh, just do the one thing and then you just go into the file and change the, whatever the background color yourself.

    6. EG

      That makes sense. Outside of, um,

  7. 13:2516:45

    Where GitHub is headed

    1. EG

      a lot of the-... agentic efforts that you all are doing. And obviously, I think that's amongst the most interesting stuff that's happening right now. What are other big areas you want, um, GitHub to evolve over the coming few quarters? Like, what, a- are there other big thrusts or is it all kind of, it's all in on AI and that should be the, the focus on the company?

    2. TD

      Well, so far we only talked about, you know, the, the generics we agent where you can assign an issue and, um, and it generates a pull request. But if you actually look in the developer life, uh, the day-to-day, you know, in, in, in most companies, um, that's maybe two or three hours of your day that you're actually writing code. Um, and then you're spending an equal amount of time of reviewing code, um, of, of your coworkers. And while we don't believe that goes away from a pure security and trust perspective, you always want to have, you know, another human in the loop before you merge code into production. At the same time, we believe code review agents, um, and, and code review is, is, is a big topic where AI can help you, especially when you work, you know, with a distributor team in different time zones, uh, where you don't wanna wait, uh, uh, for, you know, the folks on the West Coast, uh, to wake up, uh, to, to get a, an initial loop of feedback, so I think code review is a big topic for us. And, and again, you know, the AI part is one piece to that, but the user interface is equally important, like, right? Like, if, uh, ideally you get feedback and then you can work with the code review agent, um, on that feedback to, to loop because you know, won't always get exactly the right feedback to just click accept, accept, accept.

    3. EG

      Yeah.

    4. TD

      You have to have a user interface, um, you know, cloud environment where you can just open this. If you always have to, you know, clone the repo on your local machine and install the dependencies, switch to a different branch, you're still, you know, having way too much boilerplate work, right? So moving to a, a cloud environment where you can just, you know, try out the, the changes that, um, came from code review and, and, and can modify them, uh, to make them work and have that, you know-

    5. EG

      Mm-hmm.

    6. TD

      ... fast outer loop. In that same realm of security, uh, vulnerabilities, um, which is A, you know, we want, um, you know, your code scanning not only find vulnerabilities, but also fix them. You know, an even simpler version of that is linter errors, you know, like code formatting and, and those kinda things hopefully all go away and just the AI fixes all that instead of you going through 100 linter warnings telling you where to put the spaces in the, in the parentheses. But also if you look in, in any decent-sized pro- software project, it has outdated dependencies, it has lots of, you know, known software vulnerabilities, hopefully non-high risk, um, and, and a lot of them low risk or where somebody decided that's not actually, uh, crucial to fix right now because the code is not reachable or, you know, we have other priorities. Having AI to burn down that security backlog, um, will make, you know, both the open source ecosystem and a, a lot of commercial software projects so much better because it brings that, you know, effort down f- that every engineering manager swings back and forth between, you know, the tech debt, the legacy code, you know, the security, accessibility, European regulation, whatever, right? And the innovation backlog. And there isn't really, like, a balance between the tools, just like what is the most urgent issue, the most, uh, the biggest fire drill? Is it your sales team telling you, "If we don't get that one feature, we can't sell the product?" Or is it the security team telling you, "You gotta fix that one issue, um, otherwise we are going to flag you up to the management chain," right? And so we... That's I think is the AI, the AI side of things. But e- similarly, GitHub as a platform needs to evolve to support or, and have all the primitives for these agents and the AI to work in, in tandem with the human.

    7. SG

      Do you think there

  8. 16:4521:50

    Building for the new challenges of AI

    1. SG

      are, uh, problems that people are not addressing yet e- that emerge from this transition in how software development is done, right? Like, so for example, you know, you feel like we're somewhere between crossing the tipping point of the majority-

    2. TD

      Mm-hmm.

    3. SG

      ... of code being generated this year to maybe, like, all of the code in, like, s- some cases or some, some tasks. How does that change, like, testing or, or, you know, the way we should look at technical debt or any of that?

    4. TD

      To be clear, I don't think all of the code is written by AI. I think the way, uh, this will work is that we have two layers. We have the machine language layer, uh, you know, which is Python or Ruby or Rust, right? Those are effectively abstractions of the, of the chipset, um, uh, the machine instruction set. Um, and that's the last layer that's deterministic, right? Like, programming language inherently when I, uh, it does exactly what I want it to do. And then human language is inherently non-deterministic, right? We can all, the three of us can say the same sentence and, and mean a different thing. And so while we will use human language to describe a lot of the, you know, features and, and behaviors that w- we're gonna build, we will still have the programming language layer below that and that we are going back and forth, um, as engineers to, to figure out, uh, uh, is the code that was written by AI actually the correct one? Is it the one that, you know, aligns with my cost profile, if you will, as an ex- as an example, right? Like, at the end of the day, we're still all running businesses that have to have, uh, uh, positive profit margins. I think we're going to, as engineers, have both of these layers. Um, and, and we're heading into a world of more human language and, and, and less, and less programming language. Um, but at the same time, you know, we are in the world where lots of financial services institutions still run COBOL code on mainframes, and we are very far away of just taking that, you know, uh, uh, code that's 30, 40 years old and just having an agent that transforms that magically into a cloud application, right? Like, I think that's coming, but it's like self-driving cars, um, uh, are coming as well, uh, but we don't know when that p- c- c- cut over point actually happens where you can have a car without a steering wheel and it drives you everywhere, you know, uh, within, within the country you, you live in, right? Like, the, the, it works for Waymo in, in San Francisco and it doesn't work for Waymo, uh, all the way down to SFO to San Jose yet, right? And so the, the scope will increase, but we are, we are far away from, I think, uh, solving all the tech debt and all the legacy code that exists. And so we are still, uh, I think for, like, a decade or so, uh, at least going to have software developers that work in, in lots of old school, you know, PHP code and, and COBOL code and, and, and all that stuff. While at the extreme other end of the spectrum with web development and, and AI, you're going to be able... And you're st- we are already there. Like, you know, just look at a, a 10-year-old, give them, you know, um, a tool like, like Copilot or, you know, um, Repollet, Bolt, um, you name it, and, and have them type a couple of prompts and let them explore how that works and how they can similar to Stable Diffusion mid-journey render software themselves and, and iterate on that.

    5. SG

      You yourself lead, you know, a large team of software engineers. As you said, you know, you have more human language and instruction versus machine language. Does it change what you look for or what you want to, like, develop in your own team?

    6. TD

      Well, what we're looking at right now is, I think this, how do you describe actually a problem, uh, specific enough that an agent can pick it up, right? Like, basically the planning and tracking side of software development, the issue, right? That's, that's often the biggest, uh, challenge that you have as soon as you have a decent team size. Right? Like, ten-person startup has no problem then... Most of ten-person startups don't have a product manager. Uh, the-

    7. SG

      Mm-hmm.

    8. TD

      ... founder is the product manager and the rest is just building the stuff, and if you have a problem to solve, you have very short communication paths. If you have a thousand engineers, the biggest problem is, uh, what do you wanna build? How do you build it? What did you actually mean when you wrote up this thing? And if you look into that space, there isn't much AI helping you yet. Um, we have, we have been early phases, um, ourselves with that with, with copilot workspace, uh, where we have, like, a spec and a brainstorming agent that, um, basically looks at what you wrote in the GitHub issue, uh, compares it with the code base and describes you the before and after in human language. And then you can, similar to a Notion doc, just modify the act, um, and basically add stuff to the, to the specification. So I think that's going to be a whole set of agentic behavior that we're going to bring into the product management, uh, uh, space. Um, similar for, for designers, right? Like today, uh, uh, a lot of designs are, you know, hand drawn, in, in Figma. Uh, I think tomorrow you're going to, as a designer, type effectively the same specification as a product manager and you have, you know, an AI to render the code for the wireframes and then apply, you know, grounding out of your design system, uh, to make it look like your product, right? And so those disciplines get closer to each other and the product manager will be able to, if they're good in writing a specification, create the whole change set, and the designer will be able to take over part of the product management role and the, and the engineer, like, gets closer to these other roles as, if you know if they're good in describing the feature can, can like take over that part as well. So I think that's where a lot of innovation is going to happen in, in rethinking how the, you know, traditional disciplines in a software engineering team are

  9. 21:5029:56

    Dev tools market formation

    1. TD

      evolving in, in the, in the coming years as we have more and more of these agents available and they're actually good at what they do.

    2. EG

      As you think about these different agents and these different use cases, do you think it's gonna be the same company or product that provides all three? Do you think it's gonna be one interface? Is it gonna be a different interfa- I'm just sort of curious how you think about the actual flow in terms of very different users in some sense, although with some overlapping either responsibilities or goals. And what, what are the set of tools that they interact with and is it, is it a singular tool? Is it many? Is it one company? Is it many? Where does it launch out of? Like, how do you think about all that stuff?

    3. TD

      One of our strongest belief at GitHub is developer choice. Um, and you know, imagine a GitHub, um, as a platform where you had only JavaScript libraries available or only, you know, React, um, uh, available to you and we would tell you that's the only open source library you need, uh, to put an application, right? Like, there would be a, a set of users, uh, using React, uh, uh... using GitHub, um, b- because they love React and the rest would go somewhere else because some other platform would offer them all these other, uh, open source components. Right?

    4. EG

      Mm-hmm.

    5. TD

      In AI I think we're going to see the same thing. We're going to see a stack or universe of, um, um, companies that offer different paths of the software development life cycle and, and developers pick the one that, um, you know, they like the most, that they have experience with and, you know, that convicted are the future, you know. A lot of that is, um, part of a belief system. You know, programming languages in, in many ways are, uh, uh, very similar. Um, and then if you look at, uh, the discussion between developers you, you ha- get the feeling they're very different, uh, to each other-

    6. EG

      Mm-hmm.

    7. TD

      ... right? At the end of the day they're all, all compiling, uh, down to, uh, uh, an instruction set that runs on your, on your Apple M4 chip or, or your, your Intel CPU or AMD or NVIDIA or whatever. Right?

    8. EG

      Mm-hmm.

    9. TD

      So I think we are going to have a stack of different tools and there's going to be, um, um, companies that offer, you know, uh, all the tools... Well, not all of them because you're never going to have all of the developer tools out of one hand anyway, right? Like, think about GitHub. We are big platform but then you still have an editor and an operating system and a container solution and a cloud that doesn't come from GitLab, right? Like, um, you know, we at HashiCorp, Terraform or Vault as, as example or, or Vercel and Next.js as another example, right? There's... Like, go into any random company in the Bay Area and they're all going to have a different stack of tools that they have combined because they believe that's the best stack for them at, at this point. So I think in this AI world we're going to see the same thing. You're going to have, um, choice of different agents. You... yeah, we are already there where you have choice of different models and um-

    10. EG

      Mm-hmm.

    11. TD

      ... uh, some believe the cloud model is better, others believe the OpenAI's model is better. You know, uh, the tr- reality is somewhere in the middle and different scenarios are bet- better with different models. And I think the same will be true, um, uh, in, in this agentic future that we're heading into.

    12. EG

      Is that true given, um, the generalizability that we're seeing? In other words, if you were to remove X percent of the models and you just got stuck with one of the ones you mentioned, up to a point you'd still be extremely happy given the relative capabilities we had four or five years ago, right? In other words, it's a little bit of like, we have so many great options and some things are better than others but fundamentally anoth- any one of these things would be spectacular, um, by any sort of baseline metric.

    13. TD

      It depends on what end state, you know, we're talking about, right? Like, uh, if the, if the singularity is coming then none of that matters.

    14. EG

      Five years from now. Five years.

    15. TD

      You know we started copilot almost five years ago, um, June 2020.

    16. EG

      And that was, what? GPT-3 at that point?

    17. TD

      GP-3 was really the early experiments and then, uh, we got this model that then eventually became Codex, um, which was the code specific, you know-

    18. EG

      That's fair. Yeah.

    19. TD

      ... version of the model. And, and today that, you know, uh, no longer really exists, right? Like today everybody sits on top of, of one of these more powerful base models.

    20. EG

      Mm-hmm. Yeah, yeah, and that's kind of my point is to some extent, um, the, the generalizability started to take over. And so I'm just a little bit curious how you think about generalizability versus specialization in a five-year time horizon for agents.

    21. TD

      I can see that happening at the model layout. Um-... but it's again, like, predicting a little bit of, you know, when do we truly have self-driving cars? Um- And you know, I had a Tesla for ten years, uh, with, with self-driving and, uh, autopilot in, in one form or another, and it kee- still cannot make the left turn into my, (laughs) into my neighborhood. I can see that future happening, but I don't know when that is, and when the models are basically just all, you know, about equal. Um, but I think for software developers, um, the, the lowest level only matters until there's differentiation at the higher le- layer of the stack, right? Like, think programming language or open-source libraries are great examples for that because, you know, if you zoom out enough, they're all the same, right? Like, at the end of the day, you know, whether you're building an app with, you know, Swift or, um, Kotlin or R- React Native-

    22. EG

      Mm-hmm. Mm-hmm.

    23. TD

      ... um, what does it matter?

    24. EG

      Yeah.

    25. TD

      And like, that's just like, the intricacies of software development and the belief system that we have. Um, and so I think the differentiation is going to come from, uh, both, you know, where the developer gets the most, um, the best experience in, in doing their day-to-day, right? Like, where can I, you know, start my morning, uh, pick up, you know, something I wanna work on, uh, explore my creativity, and, and get the job done with, with the least amount of frustration-

    26. EG

      Mm-hmm.

    27. TD

      ... and, and the highest, uh, amount of, um, ROI in terms of what can I ship in... Like, software development, you know, over the last 30 years has always... Or actually, the last 50 years if you go back, you know, all the ways to the 1970s when, you know, microcomputers c- came and, and, and all of sudden, you no longer had to share a mainframe with others. It was always about, how can I take all my grand ideas that are v- bigger than what I can actually achieve as an individual, uh, how can I, you know, get that done faster? I don't think we are at the, at the top of that exponential curve. I think there's still a lot to come. The other question you could ask is, when do the CEO of GitHub get to the point where my backlog is empty?

    28. NA

      (laughs)

    29. TD

      And I just don't believe that that point is ever coming.

    30. EG

      Yeah, there's a super related interesting question of, uh, to what you're saying which is, for how long are humans making decisions on what agents to use? Because if you look at it, uh, there's certain roles, a lot of the ones that you mentioned, developers, designers, et cetera, that have traditionally trend- tended to be a little bit trend-based.

  10. 29:5632:17

    Copilot’s broader impact

    1. EG

      business success of all of it, right? And I think it's been quite striking on the earnings calls, um, more recently that have been done. Uh, what can you share in terms of, uh, business and financial metrics and the impact that Copilot and GitHub more generally are having for Microsoft?

    2. TD

      Not a lot beyond (laughs) what's in the earnings call.

    3. EG

      Sure.

    4. TD

      I'm trying to remember. I think the last number we shared was, um, uh, a few quarters ago, 77,000, uh, organizations, uh, using Copilot. Uh, and back then, um, uh, the number of paid users, uh, was 1.8 mil- million paid users. Um, we haven't shared an updated, uh, number since, so I c- can't share that latest number. But I think what's really interesting from these earnings calls, if you look at, you know, the number of, um, logos that, uh, Satya has called out, it's across the whole spectrum, uh, of, of industries. It's not just, you know, uh, cool startups. Uh, it's, it's not, uh, just financial services institution. It's really every industry, uh, that has adopted, uh, Copilot. And I don't think there has been a developer tool that has been adopted, uh-

    5. EG

      Mm-hmm.

    6. TD

      ... with such a velocity across the whole spectrum of software development in, in any, in any company size and in any industry. You know, if you think about it, $20, um, on, uh, compared to the salary of an average software developer in the United States is like, what, 0.1% if, if at all. Uh, and then we're talking about, you know, 25, 28% productivity gains on, on the end-to-end, um, 55% or higher on, on the coding task. But, but as we said earlier, right, developers do more than just coding.Um, that's an incredible ROI on the- the dollar spent. Uh, and I think that's what is- is driving this adoption curve. And then any company that's now a software company, and they all have the same problem described earlier, they have long backlogs and- and way too much work and- and every time, you know, one of the managers goes to their team and- and asks them, "How long does it take to open a feature?" It becomes the Jim Kirk, uh, Scotty, uh, joke, uh, that, uh, you know, "How long does it take to repair the Warp Drive?" And you get an estimate that's o- outrageously long, and then it becomes a negotiation, uh, where the captain sets the deadline instead of the- the engineer actually, uh- uh, estimating, well, what's possible. And I think that's where, you know, a lot of the, uh, business success of- of Copilot is coming from, all the people writing software are frustrated how long it takes, um, not because they don't, uh- uh, think the engineers are good, but because the complexity of building software.

    7. EG

      How much do you think this pricing

  11. 32:1739:16

    How AI changes software pricing

    1. EG

      change is? And I know that it's just speculation at this point. When you're actually replacing people, and I know in a lot of industries, it could be legal, it could be accounting, it could be coding, people say, "Well, eventually this will shift to value-based pricing." Because eventually, instead of just paying 20 bucks a month to make a person more productive, you're actually replacing a person who costs 50 or 100 or $200,000 a year or whatever it is, depending on what their role is. Process is just in different disciplines. So, I'm just sort of curious how you think about... Uh, is this eventually a rent-a-programmer and it's priced like a programmer? Does it all get commoditized and eventually something that would normally cost 100,000, 200,000, $300,000 a year costs you $1,000 a year? Like how do you think about where this market goes?

    2. TD

      I think it's going to be compute-based or- or some unit that's, you know, a derivative of- of compute, um, uh, as a metric.

    3. EG

      So it's gonna be cheap?

    4. TD

      It's going to be cheap in the same way that your dishwasher in your kitchen is not a derivative of, um, what a person, uh-

    5. EG

      Mm-hmm.

    6. TD

      ... would cost you when- when doing your dishes every single day. Uh, but I think the buyer persona is- is not going to be willing, um, to- to pay for a machine, you know, whether it's a dishwasher or an agent, a price that's an equivalent of human developer. And- and I think that's actually, uh, the correct, um, uh, mindset because I- I don't believe that the AR agent is actually replacing the developer. The creative part is still coming from the software developer, the systems thinking. Predicting the future always has the- the fun part of that and coming back on the podcast (laughs) in a year or two, and you're telling me how wrong I was about my predictions. But I think there's a lot of decisions, uh, that are made in software development that a human has to make. Um, what database, you know, what cloud, um, a lot of that are a function of the business and how it operates, um, you know, which cloud you're using is- is not necessarily a question of how much the cloud costs. It's a question... it is a- a strategic decision of, you know, the CTO or- or the leadership, engineering leadership team. Um, and more and more we see, you know, companies using more than one cloud because they want, don't want to have a dependency on- on just one single supplier in the same way that, you know, uh, uh, any random car manufacturer has multiple suppliers for airbags because they don't want to be stuck with, you know, their factory line when- when airbags are not deliverable-

    7. EG

      Sure.

    8. TD

      ... from that one supplier. And so I think, you know, the agents, uh, the price, the price points will certainly go up as- as these agents become more powerful, you know. We see, we see that with OpenAI, where the highest tier now- now costs $200 for deep research and the o1-pro model and including, uh, people see the value, the value in that. And, um, I think, you know, two years ago if- if we had, uh, predicted that you, you, we wouldn't have believed it. You're willing to pay $200 a month, uh, for a chat agent because the- the flip side of that often is in- in software that- that people, uh, feel like a $5 subscription for a mobile app is a lot of money. And you can just see that when you look into the reviews of- of apps that move from a one-time payment, you know, to a subscription model of how- how many people, uh, uh, don't like that model because they feel like software is something that you buy once, like a CD, and then- and then you own it. That definitely there are going to, uh, price increases, um, that will be based on the value that you're getting out of it. Because, you know, the- the other side of that is that, um, uh, human developers are expensive because, uh, there's limited supply. Uh, agents, uh, will have infinite supply that- that will only be limited by the amount of, uh, compute capacity, uh, GPUs available in data centers.

    9. SG

      Speaking of the unlock of supply, like we've been talking about like what is the... what is the pricing of the code generation. I think there's also a question of just like what happens to the value of software at all? Like, everybody's been talking about Jevons paradox for a while. I don't want to ask about that.

    10. TD

      (laughs)

    11. SG

      But maybe something more specific. You're from East Germany.

    12. TD

      Mm-hmm.

    13. SG

      You remember, uh, the Trabant car?

    14. TD

      I do. (laughs) I had one. No, well, my parents had one.

    15. SG

      Oh, okay. Right. So you can- you can tell me what it was actually like. But, um, good for- good for you guys because it was this... it was the okay car but it was the default car that ended up having this like 10-year waiting list because of the supply constraint with the rest of the world and then as soon as the wall came down you, you know, the demand completely collapses.

    16. TD

      Yeah.

    17. SG

      Right? Because you have access to the world of cars, and- and pricing at least did. I guess one question I have for you is... I'm- I'm generally such an optimist about like the demand for software being very elastic, but I think of that as volume and quality and variation. Are there types of software that you think collapses in value when AI takes away like some of the scarcity of engineering?

    18. TD

      You know, the Trabant, um, the wait list was actually I think 17 years, um, in the late '80s.

    19. SG

      Okay. 17, not 10. Yeah.

    20. TD

      That road, by the way, still exists. Um, it... today it exists in, uh, supercars, right? Like often you can buy a supercar like, um, you know, the top end, uh, Porsche 911, um, S3 or whatever. And then, uh, the resale price is higher than the- the new price because you can't get one to go to a dealer because as a... at the dealer you have to buy like 100 Porsches first before you get, uh, a slot for- for that exclusive top-of-the-line Porsche, or Ferrari is the same thing. And so the Trabant actually, um, the one that my- my- my dad owned, um, he sold I think in '84, '85, uh, to a neighbor at a higher price than- than we bought it because you could shortcut the- the 17-year wait, uh, to get a car, and often parents had a subscription, quote-unquote "subscription" like, uh, signed up their kids already for a car when- when the kids were still young. Uh, uh, so you could actually get one by the time y- you've reached, you know, uh, uh, adulthood and- and could do a driver's license. And so, you know, I think we're going to see, uh, uh, coming to your- to your software question, right? Like, uh, we're going to see it going both ways, right? Like if you're thinking about Copilot, Copilot, um, you know, costs, um, for businesses $20, uh, per user per month. That's actually almost exactly the same price as you pay for GitHub Enterprise, uh, which is $21, uh, per user per month. Right? And so for storing all your repositories, managing all your- all your issues, um, your whole software development life cycle was $21 per user per month and- and many used to perceive that as a lot of... a lot of money for, you know, DevOps. Um, and then we came with Copilot auto-completion and that was $20 a month, right? And so all of a sudden that- that sub-feature of the software development life cycle, auto-completion, uh, cost $20 and- and that goes back to Elade's question, right? Like if there's the value, uh, where you get the ROI and you- you get 25% productivity increases, yeah, I mean, you're- you're willing to pay more, uh, for something that, you know, probably five years ago if I told you auto-completion is going to be that standalone feature driven by AI that- that costs-... uh, more than the average selling price for all of GitHub, you would have said, "Oh, that's ... that sounds unlikely. I think we're going to see deflation of, of software prices." Um, and so I think it's, it's, it's a mix of both, you know. Some things we won't pay for it anymore. Um, you know, nobody pays for the operating system, uh, uh, uh, anymore, and then at the same time you pay, uh, uh, way more than ever for your Netflix subscription and, uh, for your Office subscription and, and all those kinda things. So I think both of these things will be, will be true at the same time and it's all about how much value do you get for your business, uh, paying for that solution whether, whether it's doing it yourself or using something that you manage yourself or install on, on your own server.

    21. SG

      GitHub

  12. 39:1648:01

    Open source vs. proprietary APIs

    1. SG

      is foundational infrastructure for open source, so I'm sure you have, like, general opinions about what's happening in the, the open-source ecosystem. Today, you can use Claude and OpenAI in Copilot, and Gemini-

    2. TD

      Yeah.

    3. SG

      ... but not necessarily open-source models right now.

    4. TD

      Correct. So in Copilot, we have, uh, Claude, Gemini, and then OpenAI, and O- OpenAI has different models. That I was ... I, I was just processing this in my head, "Wait, there's more than three models," but it's the 4o model and the, um, o1 and the o3 mini model. In GitHub models, um, which is our model catalog, um, we have, uh, open source or open base model like, like LLaMA, um, a- as an example, and then all kinds of other models like Mistral, uh, Cohere, uh, Microsoft's Phi, Phi 4 model. And the model catalog, you know, while it's a separate, uh, feature within GitHub, um, you can add model, add models, uh, in Copilot because Copilot has extensions and so you can actually reach from Copilot into the model catalog. And so if you wanna just-

    5. SG

      Oh, cool.

    6. TD

      ... ????? run quickly inference against Phi 4, you can do that by, by using the add models extension in Copilot. So that way we have more models than just the, the ones that are packaged into Copilot.

    7. SG

      I didn't know that. Um, what do you think is the relevance of open source versus the proprietary model APIs for developers in the future?

    8. TD

      The biggest thing, I think, is that open source is going to drive, uh, innovation, and, and we saw that, you know, with DeepSeek, uh, earlier this year, um, or actually, you know, couple weeks ago. It's not that long ago even. Um-

    9. SG

      Long year, yeah.

    10. TD

      (laughs) It's been ... It feels like already half a year has been passed instead of just a month and a half. Um, but I think open source is, is going to drive innovation. Um, you know, we saw that with image models like Stable Diffusion, and now there's the FLUX model from, uh, a startup actually not too far from, from my home base in Germany in, in the Black Forest in Freiburg. Uh, uh, Black Forest Labs is actually the company behind FLUX and, and so we're going to see innovation, I, I think, you know, in... on open source models that, that drive the other vendors. And this back and forth between the open source ecosystem and the propriety, um, you know, closed source, um, uh, companies, um, will, I think, accelerate, uh, the whole space. Um, you know, DeepSeek is, is the most prominent example right now, where you can, you know, look into this. The, the paper is open, uh, the, the models are open. Some of them are like, you know, uh, uh, fully open source under the MIT license. Others are, like, uh, open weights and so you can look at the weights and then the code to run it is open source, but the weights itself are under, under some of ?????? license and, and, and governed by, uh, Chinese law, uh, and whatnot. And I think that, uh, is going to drive innovation and, um, it's going to open up that space and it democratizes access, um, because if you just wanna play y- uh, with a model, um, you don't have to, to run, uh, uh, inference against a commercial API. You can just, you know, try it out yourself on y- on your local machine and, and play with this. And if you think about kids and students and, and research, that, that opens up a huge space and that's ultimately, you know, what, uh, has always been, um, part of our DNA at GitHub. Was that a satisfying answer, Sara? (laughs)

    11. SG

      Yeah. Yeah. I, I think, um, the most-

    12. TD

      Yeah.

    13. SG

      ... satisfying answer is like, somebody wins, right? But I, I, I think that's a very hard s- uh, thing to predict right now.

    14. TD

      What has won, uh, by phone or Android? Uh, has ... Wha- which ... Windows or La- Linux, um, or macOS, uh, for that matter? I, I'm ... Like, I, I think we like to think about these, uh, uh, you know, binary battles in the tech industry, and the reality is, um, uh, that's not actually how that works and certainly not, uh, in, in the developer space, right? Like, React hasn't won. Um, and there's always going to be the next thing, you know? And before React, there was a jQuery or, or whatever, you know, li- uh, library you preferred. I think there's going to be a next programming language after Python and, and TypeScript and, and Rust. And, and Rust in itself, you know, uh, wasn't really a thing, uh, uh, five years ago. And, and so there's going to be more languages that are probably closer to, to human language and to, to be more specific about the natural language layer in AI and, and the programming language layer, uh, that, that converts down to the, to the CPU or GPU. And that's ... So I think there is no, no winning. There's always just the ... It's ... You're playing the infinite game. It's like Minecraft. Software-

    15. SG

      (laughs)

    16. TD

      Software is like Minecraft and there is no winning in Minecraft. You can win little battles and, and they're isolated to a certain, um, sub challenge or whatever, quest, but ultimately, uh, we're building a bigger and bigger world of software and there's always going to be a next big thing.

    17. SG

      That's a, it's a funny analogy. If I think about any individual developer, uh, like, there's something people have been saying to me. Developers of a particular ilk, right? Um, really strong technical people who are more exer- exper- not all of them, but like more experienced Gremlin systems developers, often people very attached to Rust, um, and, and they'll say basically, like, they're worried about the next generation of developers building the taste and understanding of architectural choices and the trade-offs and corner cases of how a particular implementation can fail given some shape of data, given their experiences of the actual implementation, right? And so they're, they're worried, you know... Obviously the right thing to do for anybody who wants win that next, you know, level of Minecraft in 2025 is like, use AI aggressively, learn to use it. But, like, does that concern from this segment of d- I'm, I'm sure you've heard it. Does that concern, like, resonate with you at all? Like, can you foster the requisite depth of understanding of engineering at an abstract level when we're not writing the code or is it like a silly, silly concern?

    18. TD

      I wouldn't call it silly, um, because obviously, you know, there, there's some truth, truth to that, right? It's easy...... uh, you know, to cheat at a programming exercise or advent of code and, and those kind of things. Um, as these AI models get better, these competitions of who's the best hacker or coder are going to have to move to a whole different level, um, where you assume that the developer is using AI to, to solve the challenges because otherwise, um, uh, it's going to be way too easy. If you're thinking about the next generation of developers, you know, maybe not 2025 but 2035, like, look, you know, and you mentioned, you know, me growing up in East Germany and then the wall fell and I bought a Commodore 64 but they had no internet and so I bought books and, and magazines and that was it, right? Like, there, there was no forum I could go to and, and ask questions. Um, I could go to... I went to computer club, uh, uh, every, every Wednesday or so until nobody there had, had anything to say anymore that I didn't know already, right? Um, if you, if you take that and compare it to today, um, the kids of today and those that want to learn coding have an infinite amount of knowledge available to them. Um, and you know what? Also an infinite amount of patience, um, because, you know, Copilot doesn't run out of patience. Parents do. I, I am one. And so it's incredibly, um, democratizing, uh, to have AI available if you want to learn coding. Your parents don't have to have any technical background. Um, all you really need is an internet connection, um, o- on your mobile phone, uh, and, and, and one of these Copilots or, or ChatGPTs or what- whatever you prefer, and you can start asking coding questions and you can ask about Boolean logic and about systems thinking and you can go infinitely deep on, on any of those questions and, and, and, and, you know, traverse to, to other topics as you like, right? And so I think, you know, we are going to see, you know, a new generation of, um, uh, humans that, you know, grow up with the technology and for them it's just natural to, to leverage their, their personal assistant, their personal set of agents. Um, you know, I listen... they call it the orchestra of agents, uh, and you're the conductor of that orchestra of agents. And, um, they know how to do that and so they can achieve in the same amount of time so much more than, uh, we could, you know, in the last, in the last 30 years. And I think that's incredibly exciting because like again, like, find me a developer that doesn't have this big idea of that computer game or software system or feature that they always wanted to build and, and don't have the time for. Like, my engineers talk much more about being over-committed and, and burned out and, and not having enough time for all the things I'm asking for and the customers are asking for and the security team is asking for. Um, and so I think that's, that's just where, where we're heading and where... how this is going to be, uh, super exciting, um, both actually in open source as well, right? Because open source's sustainability is, is another big topic that we could probably spend another hour on, uh, and in, in any kind of software that people want to build.

    19. SG

      I definitely agree with that, um, you know, excitement and optimism. I think about my three kids and like, uh, like what they would be able to learn, at what pace, um, with the, you know, the AI resources that people will have, and I'm incredibly jealous. I'm like, "I could be-"

    20. TD

      Yeah.

    21. SG

      ... much better as an engineer so much faster with, as you said, the infinite patience and understanding of today's models. And by the way, I was very lucky. My parents are both engineers, right? But, uh, y- you know, it's a very human dynamic where, uh, I'd ask a question and my dad would be like, "Uh, it's logic, Sarah."

    22. TD

      (laughs)

    23. SG

      And I'm like, "Oh, no." (laughs) Um, can I ask you, uh, you know, maybe a, a, a more personal question to close?

  13. 48:0150:34

    Growing up in East Berlin

    1. SG

      Like, East Berlin, uh, you know, you have this unique experience of this really rapid technological change after reunification. Do you think that informs at all how you think about, like, the speed of the current AI transition and how, like, users and human beings will react to it?

    2. TD

      I always wanted to believe that a lot of my life has been, you know, um, defined by that one moment of change, um, in, in 1989 and, you know, I remember, you know, the night when the wall fell or when it was announced that the, uh, wall would be opened and it was a Thursday night and then Friday was normal school. Saturday was still school as well, a half day in school, and I think I was one of four kids that, that showed up in, in my class and then they sent us home and, and, and, and, and we actually crossed over to, to West Berlin. The thing they... the thing that is important for that generation of kids that lived through that change is that they can no longer return to that childhood, you know? Home, home is gone. Like, you know, there isn't, like, that store in the corner that's the same as it was like, uh, 40 years ago and the schools are all gone. The system is gone. The traditions, a- all that resolved in, in, into, you know, that new world. And, and so it's a bit like when you're moving from one country to another, which then I, uh, you know, did 10 years ago as well to move, uh, when Microsoft bought my company. Once you have done that step in your life, um, you, you gain a whole new perspective, um, o- on things and I think that's... You know, unification in 1990 and then, you know, um, through the steps of my life including, you know, to become the GitHub CEO through, you know, random decisions, or at the time they felt random, is how I got here and this is how, um, I look forward and I'm optimistic, uh, about the future, um, uh, while, while recognizing my past and, and taking in all... uh, some of those experiences, um, (laughs) when I talk with you guys and, and, uh, and reflect on what it was like in the '90s, '80s, '90s to, to program on a Commodore 64 before and after the internet, right? Before and after open source, before and after the cloud, before and after mobile, and now we have before and after AI and there's no looking back. Like, the, the future will be that we have AI for almost everything we do in our lives if we want to. You know, you can still always throw your, your cell phone, uh, uh, into the corner and, and enjoy your day, uh, without, without the internet.

    3. SG

      This has been great, Thomas. Thanks so much for having a conversation.

    4. TD

      Thank you so much for having me.

    5. EG

      It was great to connect, sir. I appreciate the time and everything else. (instrumental music plays)

    6. SG

      Find us on Twitter @nopriorspod. Subscribe to our YouTube channel if you wanna see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way, you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.

Episode duration: 50:34

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode XqvNnl2AJko

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

Add to Chrome