How Devin replaces your junior engineers with infinite AI interns that never sleep | Scott Wu (CEO)

1. 0:00 – 3:53

   Introduction to Scott Wu and Devin

   1. SWScott Wu0:00

      Devin is async. Once you kick off a Devin session, Devin's gonna start working and looking through the code, but you're not expected to be there with it. It's just as if you gave your intern a project, and your intern is going and working on it.

   2. CVClaire Vo0:10

      Devin's my favorite intern on my team, and I have infinite of them. Why don't you pick a task that you might bite off for your product and show us how you would work through that end to end?

   3. SWScott Wu0:20

      I'll say, "Please go research the ChatPRD MCP server." So this will produce a pull request for us. Often, you're running a few of these at once, just like a nice way to have multiple tasks going and then check in on each of them.

   4. CVClaire Vo0:33

      One of the benefits of this from a How I AI use case is you can multithread a lot with tools like this, and set two, three, four, five, 10 of these going at once on different projects and not feel like you have to sit there and babysit things. [upbeat music] Welcome back to How I AI. I'm Claire Vo, product leader and AI obsessive, here on a mission to help you build better with these new tools. Today is a very special episode for me because we're talking to Scott Wu, CEO and founder of Cognition Labs, and the builder of one of my favorite AI products, Devin. We're gonna hear about how Scott uses DeepWiki and Devin to kick off well-scoped tasks to get things done, uses Devin as his favorite and most tagged employee inside of Slack, and how he's making it not weird to bring ChatGPT voice into your meetings. Let's get to it.

   5. SPSpeaker1:26

      This podcast is supported by Google. Hey, everyone, Shresta here from Google DeepMind. The Gemini 2.5 family of models is now generally available. 2.5 Pro, our most advanced model, is great for reasoning over complex tasks. 2.5 Flash finds the sweet spot between performance and price, and 2.5 Flash Lite is ideal for low-latency, high-volume tasks. Start building in Google AI Studio at ai.dev.

   6. CVClaire Vo1:56

      Scott, thanks for joining How I AI. As Devin's number one reply guy on X, I am [chuckles] really excited about this conversation, and for you to show off how your company uses, and you use the product that at least makes me very happy, and I'm sure makes lots of software engineering teams out there very happy. So welcome.

   7. SWScott Wu2:18

      Thank you so much for having me. No, I, I'm, I'm honored to be here, honestly. I'm a big fan of you guys, and, and, and all the work you do, so...

   8. CVClaire Vo2:24

      Great! Well, we wanna get into... We have lots of stuff to talk about, but what we really wanna do is get into how you AI, and in particular, how you AI with the products that you've built. And, you know, I think what's really fun as somebody who's building AI products, it's, it's something you get to use every day and get really good at, but also probably show some of our listeners and watchers some tips and tricks about using the tools that you've built that they may not have thought about so far. So we're getting the expert look into how to AI with the Cognition product. So what are you gonna show us first, and what are some of your common workflows when you're doing engineering work or trying to move the product forward?

   9. SWScott Wu3:07

      Yeah, for sure. No, for, for us, it's definitely... I mean, as a bunch of programming nerds ourselves, you know, building an AI that can code is, uh, has got to be one of the coolest things that we could-

   10. CVClaire Vo3:16

       Yeah

   11. SWScott Wu3:16

       ... probably spend our time on. I wanted to show a couple of flows, actually, of how we use the, the, the Devin stack, because there are a few different pieces involved. There's Slack and Linear, there's the Wiki, obviously, and then there's, like, Ask Devin, and then there's, you know, starting Devin sessions and getting pull requests out of it. I think there's some real, um... I think there's some real nuance in, like, what are the right flows of, like, how do you work with Devin as an employee? 'Cause I think it really is quite different from a lot of the tools out there, which are much more kind of like, you know, an, an IDE, for example, or, like, a terminal UI. Like, Devin is, is, I think, first and foremost, almost like a- an engineer on your team, so...

   12. CVClaire Vo3:51

       Yep, totally.

2. 3:53 – 6:08

   Where Devin excels

   1. CVClaire Vo3:53

      So what are some of the things that you reach for with Devin and the capabilities that you think really make a difference for you as a, a software engineer?

   2. SWScott Wu4:04

      The, the way that we like to describe it is, is Devin is a junior engineer. Um, and so Devin is not gonna go... And, and, you know, we're working on getting Devin to senior engineer, obviously. You know, we'll get Devin the promotion and everything. But, but, but, like, Devin is not going to go and, and, and solve some, you know, really hard architectural problem or make some big strategic decision that you, you know, you're gonna make, and then kind of, like, execute on for the next month. Like, you probably wanna be involved in those as well. Devin can help you with the decision, obviously, by, by kind of like, uh, referencing the right things or, or, or, or giving a few things as input. But I think where Devin really shines is... One way that we say also is, is kind of like tasks, not problems. Um, and so often when you have a very clear, like, "Here's exactly what we need to go do, and here's the task, and, and here's all the details of what we need," Devin is really great at going and executing that for you and makes that much faster. Um, and so naturally, I think the next question that comes to mind then is, like, how do you figure out the spec or the, you know, the, the, the, or, you know, the, the task exactly that, that you want to do? Um, and so a lot of the others, other tools, like Wiki and Search, you know, really, um, are, are here for you to be able to kind of, like, ask the right questions that you want about understanding the codebase or, or, or what needs to be done, um, and then putting a task together. Uh, I think in practice, like, a lot of the, the use cases that we see all the time are, um, you know, probably number one is just crawling through your issue backlog. You know, when- whenever, um, whenever you have an issue that comes up or, you know, we have a lot of Slack channels where we talk about issues, and then on every single one of them, we just tag Devin as the first pass. Um, and so that's a big one. Um, and so, like, you know, someone says, "Oh, you know, we need to go fix this thing in the front end," or, you know, "Maybe we need to go support this other... You know, support this other MCP," for example, which we'll show in a second, um, things like that. Um, and then for a lot of the other kind of like, I'll, I'll say, like, engineering toil use cases, um, it also does really, really well. And so often that's like, you know, going and doing a version upgrade or added documentation throughout, you know, kind of your, your, your repo, or, um, adding unit tests for a specific thing that you have up, or responding to, um, you know, uh, a crash report that just came up and

3. 6:08 – 10:27

   Using DeepWiki to research codebases and create better prompts

   1. SWScott Wu6:08

      trying to diagnose what went wrong.

   2. CVClaire Vo6:09

      Yep, I, I, I love that, what you said about, you know, Devin's a, a junior engineer. I say Devin's my favorite intern on my team, a- and, and I have infinite of, of them. And then I like this idea of scoping task, not problem, and I do think it's something that people are working with AI and-... even, you know, other, other AI tools, not in the engineering space, really thinking about task-level orientation sets you up for success, or at least a sequence of tasks can be very helpful. And so why don't you pick a task that you might bite off for, for your product, and let's sh- you know, show us how you would work through that end to end?

   3. SWScott Wu6:48

      Yeah. Yeah, let's do it. So, uh, uh, uh, as you might know, I'm, I'm a huge fan, actually, of ChatPRD, and the natural thing that came to mind for me was we need to integrate into ChatPRD's MCP server.

   4. CVClaire Vo7:01

      Mm-hmm.

   5. SWScott Wu7:01

      And so I was looking into how to do that with Devin. Um, and so, so the, the first thing that, that I always kind, kind of go to as an initial thing is what we call DeepWiki, uh, which basically, for any repo, this is true for public or private repos, um, you can come in and get a full AI-generated, um, documentation of the repo. Um, and so in this case, there's, um, you know, this is the, this is the Devin web app repo. Uh, hopefully there's nothing too sensitive here. But it's basically... You know, it ex- it explains Devin. Um, a- a- and it's pulling a lot of its information from the README or, or understanding the system architecture, and I can, I can search this and pull up different things. And so, you know, if I want to understand how the MCP marketplace is set up, you know, it'll point out what particular, um, components there are or what particular files are called here. Um, and, and I can read up on this and kind of understand exactly how this is set up. Um, a- and the natural question here that I might ask is: "Okay, cool, but just show me where the MCP serverless is implemented." Uh, and so this will look through our repo, and, and, and Devin, at this point, has done a lot of work in the Devin web app repo, [chuckles] understandably. Um, and so, so that helps a lot, which is, you know, Devin builds this representation of the codebase over time, and we can see where- what's going on here. Um, it has all this.

   6. CVClaire Vo8:15

      And so you're getting both, like, sort of a natural language ex- explanation of how this serverless list is implemented, and then you also, on the right side of this, for folks that are watching, get the actual code snippets and reference files that you can view and really understand the, the deep layer of the code. So you have, like, sort of a combination of, "Let me explain how it works, and then this is the, the nitty-gritty."

   7. SWScott Wu8:40

      Yep, combination of English and code. I think it's an interesting one, where it's like, you know, I, I, I think someday it'll probably all be English, you know? But, but I think especially now, you know, in, in this current period, I think we're really in the era where, obviously, you have to [chuckles] you, you as the engineer, want to be looking at both English and code. Um, and you can see here, it's, it's giving you kind of the, the, the answers of what's going on, and in particular, it'll point out, "Okay, here's, you know, our list of all the different marketplace servers that we have, and we have an Atlassian MCP, and there's a HubSpot MCP," and so on, right? Um, and from here, uh, the natural thing that I'll wanna do here, um, it- which, which is what we found to, to, to, to be a big flow for folks, is to use this, um, to produce actually a prompt for Devin. Um, and so the whole idea is, now that we're in this context, you know, we know what the questions were, w- we know what part of the codebase that we're looking at, um, it g- it gives a lot for Devin to be able to start from, and if we have an actual task in mind, then we can get that going. So I'll say, "Please go research the ChatPRD MCP server and add that it to our list." And so what this will do, I, I, I, I use, um... basically construct a Devin prompt from this. Um, and so this has, you know, my, my prompt here, which I just typed in, um, which is, you know, not super refined. [chuckles] Um, but it also has, um, all the detail about, uh, you know, the, the, the part of the code that we're in, and what components we're looking at, and so on. Um, and so then it will generate for me this prompt in Devin that I can just go ahead and use immediately. Um, and you can see here, it'll tell... You know, you, you want to follow the pattern of existing servers like Atlassian and HubSpot. Here's the exact typed it structure that we're being- you know, that's being used here. Here are the functions that you should be looking at, um, and, and here's what you should check to make sure that it works.

4. 10:27 – 11:24

   Prompting tips

   1. CVClaire Vo10:27

      One of the things that I wanna call out for folks in terms of a workflow that they should think about is, a lot of people, myself included, sorry, Devin, would have just sent that prompt, which is add, you know, add ChatPRD's MCP server to the list. And I do think that one very short but important loop of take this prompt and turn it into an effective prompt, given the context you know, and then sending that into the task to do, just saves you a lot of heartache, and it feels like extra friction at the time, but I think pretty soon is, um, one, gonna be the job to be done of the, the tool itself. So does that, like, loop become invisible, either through these reasoning models or some application layer that manages it? And two, it's just worthwhile for people to do. So this... You know, when you're thinking about sending a five-word prompt, think instead, saying, "Here's my five-word prompt. Build me a better prompt," and sending [chuckles] that into your system.

5. 11:24 – 13:38

   The asynchronous nature of working with Devin

   1. SWScott Wu11:24

      Yeah, for sure. And, and I think it's, you know, a- it's a great call because, uh, you know, as we said, Devin is async, right?

   2. CVClaire Vo11:29

      Mm-hmm.

   3. SWScott Wu11:29

      And so from this point onward, the nice thing about this is, you know, once you kick off the Devin session, Devin's gonna start working, and looking through the code, and reading online about ChatPRD-

   4. CVClaire Vo11:38

      Mm-hmm

   5. SWScott Wu11:38

      ... for example, right? Um, and it's gonna do all this, but, but you're not expected to be there with it, right?

   6. CVClaire Vo11:43

      Yeah.

   7. SWScott Wu11:43

      And so, you know, it, it, it's gonna work on its own. It's just as if you gave the in- your intern a project, and your intern is going and working on it.

   8. CVClaire Vo11:49

      Mm-hmm.

   9. SWScott Wu11:50

      And so, you know, they can ping you on Slack and ask you if there's questions or something, or you can go kind of like... You can go take a quick look and see, you know, how your intern is doing. Um, but you don't have to be sitting there with Devin for every step of the way here. And so one way that we kind of describe it is, um, you know, for, for a lot of tasks, there's often this sync component, like a synchronous component, and then this asynchronous component, right? And a lot of what search and wiki is for, is for, for doing the synchronous part of the task before you do the async, right? And so, like, if you had an intern, for example, would you just send them a five-word Slack message and just leave it at that? Maybe sometimes for something that is like, you know, super clear, and then you know exactly. Often, what you actually would do-... is you would sit down with them, talk it through for two minutes, and be like, "Okay, yeah, like, you know, you know how we have this MCP marketplace?" And then we go and look at it together. You know, we read the particular lines of the code, and then you say, "Okay, yeah, so, so let's, let's add ChatPRD to this, and, you know, just go take a look at how that MCP server is implemented, and, and make sure we add it to the list," and then you kind of hand off there, right? So you kind of have the first two minutes of going back and forth with Devin, your intern, and then as soon as you hit go on the Devin prompt, you're kind of expecting it to be more of an asynchronous thing, where you don't have to be in the loop.

   10. CVClaire Vo13:01

       Well, and one of the things I wanna call out for people that are building AI products out there, you know, like you, like me, is in these sync products, latency really matters. People get really frustrated-

   11. SWScott Wu13:11

       Yeah

   12. CVClaire Vo13:11

       ... with wait times, but if you set up your product to really be this asynchronous modality, you actually buy yourself a lot of user love on waiting time-

   13. SWScott Wu13:22

       Right

   14. CVClaire Vo13:22

       ... because there's not that expectation. Just like you would not say, "Hey, intern, okay, now go research this other MCP, and do a PR for me, and come back when it's ready," you know, just like that would not be, um, something you expect an intern to come back

6. 13:38 – 14:43

   Multithreading tasks

   1. CVClaire Vo13:38

      to you immediately, you also, from a product perspective, don't expect Devin to come back immediately. Now, one of the benefits of this from, from a how I AI use case is you can multithread a lot with tools like this, and set, you know, two, three, four, five, 10 of these going at once on different projects, and not feel like you have to sit there and, and babysit things. And so I'm, I'm wondering, you know, while this is running, do you go pop off and-

   2. SWScott Wu14:04

      Yeah

   3. CVClaire Vo14:04

      ... go to a meeting or get a coffee? What has this sort of like asynchronous workflow enabled for you?

   4. SWScott Wu14:11

      For better or for worse, I'm, I'm in meetings for a lot of the day. [chuckles] And I... It's, it's, it's great to be able to just kind of kick these off, or, you know, you have an issue backlog of, "Hey, there's these three or four things I was hoping to look at today," right? And you kick off each one with Devin, and then, you know, th- these go and work asynchronously, right? Um, and it'll make the pull request for you in GitHub, and it'll kind of show you the diff and, and what work it went through. Um, if it's like a front-end change or something like that, it'll send you the screenshots, um, of, of, of what, uh, uh, of, of the before and after, right? You can see it's going and researching ChatPRD.

7. 14:43 – 18:38

   Using Devin to implement an MCP server integration

   1. SWScott Wu14:43

      [chuckles]

   2. CVClaire Vo14:43

      Well, I will say just, uh, my, clearly my SEO-

   3. SWScott Wu14:46

      Yeah

   4. CVClaire Vo14:47

      ... on my MCP is not good, but Devin did make-

   5. SWScott Wu14:49

      Yeah, yeah

   6. CVClaire Vo14:49

      ... my MCP homepage. So it's-

   7. SWScott Wu14:51

      Oh, really?

   8. CVClaire Vo14:51

      ... in the top nav. Yeah, you know?

   9. SWScott Wu14:53

      That's funny. [chuckles]

   10. CVClaire Vo14:54

       Which is important for me, so it should know. [chuckles]

   11. SWScott Wu14:57

       Yeah. [chuckles] Um-

   12. CVClaire Vo14:58

       Well, so, so this is unique.

   13. SWScott Wu14:59

       Cool. Yeah, so, so, so, so I, I think for sure, you know, often you're, you're running, like, a few of these at once-

   14. CVClaire Vo15:04

       Mm-hmm

   15. SWScott Wu15:04

       ... and like you said, it's just, like, a nice way to be able to kind of have multiple tasks going and then check in on each of them.

   16. CVClaire Vo15:09

       Yep. And so what this would do, and maybe we can come back to it later when it's-

   17. SWScott Wu15:14

       Yeah

   18. CVClaire Vo15:14

       ... it's done thinking. What this is gonna go do is it's gonna go do research, it's gonna find my, um, docs page on the MCP server, uh, that Devin did make for us, and then-

   19. SWScott Wu15:25

       [chuckles]

   20. CVClaire Vo15:25

       ... it's gonna pull that docs in, and then you're gonna get actual code out of this. Your goal for this is to get a PR, right?

   21. SWScott Wu15:31

       Yep.

   22. CVClaire Vo15:32

       Cool.

   23. SWScott Wu15:32

       So this will produce a pull request for us, um, and then from there, I'll be able to review the pull request, and then if that looks good, then I'll merge it. And then obviously, we'll, we'll, we'll have this out in the next Devin release this out.

   24. CVClaire Vo15:41

       Amazing.

   25. SWScott Wu15:42

       Yeah.

   26. CVClaire Vo15:42

       And then your prompts are gonna be so much better, and I'm feeling guilty, so I am-

   27. SWScott Wu15:46

       [chuckles]

   28. CVClaire Vo15:46

       ... just going to Slack you, uh-

   29. SWScott Wu15:49

       Yeah

   30. CVClaire Vo15:50

       ... the, [chuckles] the, the MCP homepage, and-
8. 18:38 – 23:22

   Setting up workflows in Slack for first-line responses

   1. CVClaire Vo18:38

      a lot more. So while this is running, what I wanted to talk about is, you know, before we got into the show, you and I were saying you're just a little bit busy, you know, over the last month- [chuckles] ... just doing a few, few-

   2. SWScott Wu18:49

      Yeah

   3. CVClaire Vo18:49

      ... interesting things with, with the business, um, in addition to, I'm sure, wanting to build and spending time with the team. And so, you know, this asynchronous nature and this junior engineer on demand, how do you actually use that day-to-day to just keep afloat on top of all the stuff that's coming in your team? You know, not, "I have a feature I wanna build, let's go build it," we just saw that flow, but, like, the kind of reactive stuff in your company, how are you using AI to, to stay on top of that and keep the velocity high?

   4. SWScott Wu19:19

      For us, a lot of it is just setting the right workflows, um, in our Slack and in our org, and so on. And so, um, you know, Devin obviously has, has knowledge, which means it'll, it'll learn your codebase over time as we keep working with it, or you can kind of give it more details about how certain things work. A- and a lot of things are... It's almost just like institutionalizing Devin as first line of response, is how I would-

   5. CVClaire Vo19:41

      Yep

   6. SWScott Wu19:41

      ... describe it. And so I, I, I could show a few examples of this. The, the, the big thing is, so to, to really get to the point where, um, for, for a lot of these different things that we file, you know, like, Devin is the first person that gets tagged in all of these, right? And, and, and but, like, Devin, Devin won't be able to do every single thing, you know, on the, on, on one shot on the first try, but often you're working back and forth with Devin, and Devin puts up a PR, um, and if there's some slight touch-up that you have to do at the end or, or that you have to build, um, then you're able to do that. And so we have a ton of cha- channels where we go and talk about issues or, um, various things that we need to build or, or, or, or things like that. You know, w- we have one for all the crashes that come in, we have one for kind of like core infrastructure things that, that come up. We have one for... This, this is the one for our web app-

   7. CVClaire Vo20:25

      Mm-hmm

   8. SWScott Wu20:25

      ... um, which is hopefully a little bit less sensitive.

   9. CVClaire Vo20:27

      [chuckles]

   10. SWScott Wu20:28

       Um, a- and you, and you can see here, a- basically, every single thing [chuckles] that, that folks talk about and whatever we, we do, you know, it's, it's- we, we start a Devin session. And so it's like, "Hey, you know, um, can you standardize, uh, the, the font, size, spacing, and style for these three levels," right? Um, and then, you know, we just go and start the Devin session, and Devin will make the PR, uh, it'll go through the PR. Um, this one gets merged, uh, be- because, uh, because there's some back-and-forth feedback here. Um, um, and so, so, so, like, Devin goes and edits. Let me pull this up and see. Um, and so Devin made this PR, there were a couple back-and-forth edits, and then, uh, Dave, our engineer, went in and, and merged this. And this is often how we work, you know, it's how... This is another good example. "Hey, Devin, can you make it so that when you command-click on a notification, it takes you to that in a new tab?" Right? Um, natural feature, prob- probably one of our users requested it. Um, and you just start a Devin session, and Devin will give you this progress update of, "Here's what I'm doing so far. Here are the files that I'm looking at, and here's what I see." Um, in this case, by the way, it's actually confidence medium. Um, and then Walden says, "Oh, no, no, no, like, you know, you should take a look at this thing instead." Um, one of the cool things I wanna point out, too, is because of this, Devin is a naturally multiplayer experience. Um, and so we will often have a few different folks going back and forth, or if somebody else is looking at this issue, or, you know, if somebody else is the expert on this part of the codebase, um, they'll go and give their own kind of input here, and Devin will just go back and forth with them as well. And so really it is just a, a, a thread where a group of you are communicating and figuring out how to, how to work on this issue, and Devin is just one of the players in the thread, right? And so, you know, Ethan comes into Walden's thread here and says, "Hey, make sure to use a link element from TanStack Router," uh, um, and then gives that feedback, right? And then Devin goes and makes that change in the pull request. And so you can say, see, Devin had, like, an initial thing, um, and then had some additional commits, and it went and did this, uh, uh, link from TanStack Router instead.

   11. CVClaire Vo22:30

       Yeah. [upbeat music] As an AI founder, you're used to sprinting towards product market fit, your next round, or that first enterprise contract. But speed isn't enough for AI startups. Buyers expect security, compliance, and transparency from day one. That's why serious AI startups use Vanta. With deep integrations and automated workflows built for fast-moving AI teams, Vanta gets you audit ready fast, and keeps you secure with continuous monitoring as your models, infra, and customers evolve. AI innovators like LangChain, Writer, and Cursor scaled faster and closed bigger deals by getting security right early with Vanta. Listeners can claim a special offer of $1,000 off Vanta at vanta.com/howiai.

9. 23:22 – 25:50

   Encouraging AI adoption in public Slack channels

   1. CVClaire Vo23:22

      You know, one of the things that I like about this, and again, kind of a sh- a shout-out in our use case for folks that are trying to drive more AI adoption in their teams, is doing this as much as possible in public is really helpful from a learning perspective. So one of the experiences I had, um, running the engineering team at LaunchDarkly was when we started putting Devin and Devin-like agents in public channels, we saw a lot more adoption and upskilling of our team on how to actually talk to these agents, how to get the right outcomes. And so, you know, I... We were talking, uh, earlier, and I was saying I DM Devin all the time, and it's 'cause I have no employ- [chuckles] no one to talk to. He's my only buddy. Um, [chuckles] but-

   2. SWScott Wu24:14

      [chuckles]

   3. CVClaire Vo24:14

      ... I DM, I DM Devin all the time, and we have these sort of, like, side conversations. He's sort of my intern on the side. But in larger organizations, I was very much a do it in public channels, do it where people can see it, because not only does the work get done, and it's nice kind of muscle memory to tag in these tools immediately, but also just learning how you use them. What is an effective prompt? What are the kind of things that it's good at and not good at, is really useful for just overall engagement with these tools. And so, um, I think hiding your, uh, AI use is kind of the worst thing you can do, [chuckles] you can do at work.

   4. SWScott Wu24:53

      Yeah.

   5. CVClaire Vo24:54

      So I say do it all in public.

   6. SWScott Wu24:56

      ... Yeah. Yeah, yeah, yeah. And then I think there's two sides of it, a- and that I was gonna say, where, where one is like the kind of like, uh... 'Cause wh- when we talk about these multiplayer experiences, right, I think there, there are two benefits, right? O- one is this kind of like, um, the, the knowledge transfer for the agent itself, which I think more and more products are starting to have, which is, you know, one person uses Devin or, or uses this tool or that tool, right? Um, and, and that adds to the knowledge of the tool itself, so that, you know, a week later, when somebody else does that session, Devin's like: "Hey, oh, yeah, I, I just touched this piece of the code last week. Like, I know exactly what you're talking about. Let me go and find that." And then the other side is kind of like educating the humans, right? Uh, uh, of like, you're showing each other what your experiences are, you're being able to work with one another in the same flows. Um, and I, I, I totally agree. I, I, I think because of both of those, you know, I think we'll see a lot of experiences in AI productivity get more and more multiplayer.

   7. CVClaire Vo25:49

      Yeah. Yeah, that's

10. 25:50 – 32:17

    Top five engineering tasks for Devin

    1. CVClaire Vo25:50

       my hope. Okay, before we move on from, from Devin and your use of it for engineering, I wanna get really specific. So you'll go, and then I'll go. What are your top five, like, everybody can reach for them, tasks that Devin can do for you? And you, you pick kinda like five categories of tasks, and I'll pick five.

    2. SWScott Wu26:09

       Okay, sounds good. Yeah, so, so top five. Um, look, I think miscellaneous frontend fixes, it's amazing for. I mean, because, and often that whole workflow is like, you know, for, [chuckles] for various reasons, like you said, you have to get, like, three different people involved, right? [chuckles] And it's like: "Here's what we're gonna do," and then you bring in somebody who looks at that code, and there's somebody else who's reviewing or something. Um, and now with this, you tag Devin, you explain, "Here's a screenshot," you know, "I, I wanna make this button a little bit more round," or, you know, "I, I wanna touch up the design here, and I wanna do X, Y, and Z," right? Uh, and it'll go and do that. It'll find the right parts of the code, it'll do the implementation, but also it'll send you the before and after screenshots as well, right? And so you can just kind of review it in line there. Uh, and that's just, like, a really, really great use case, both, I think, because similarly, it's, it's verifiable for the agent, but it's also verifiable for the human, right, to be able to-

    3. CVClaire Vo26:58

       Yeah

    4. SWScott Wu26:58

       ... and look at that.

    5. CVClaire Vo26:58

       And while, while you're, you're saying that, I will just pull up an example of this-

    6. SWScott Wu27:03

       Yeah

    7. CVClaire Vo27:03

       ... which is, um, [lips smack] let me share my screen, which I rarely get to do here.

    8. SWScott Wu27:09

       [chuckles]

    9. CVClaire Vo27:10

       It's very exciting. Um, window, let's do... Always thrilling to share your Slack. As you can see, my only friends are agents. Um, but y- here's a, here's an example of it I just did very recently, which is I'm working on the ChatPRD homepage, and, you know, Devin shoots back to me, "Here's a new, a new hero image that I like," and I was able to give feedback on, on that. So that's... This is kind of exactly what you're talking about, which is, like, let's make changes, and then, um, get kind of that immediate feedback back right in your workflow.

    10. SWScott Wu27:41

        Yep, yep. Fixes, new components, changes-

    11. CVClaire Vo27:43

        Mm-hmm

    12. SWScott Wu27:43

        ... that you wanna make in your frontend. Um, it-

    13. CVClaire Vo27:45

        Yep

    14. SWScott Wu27:45

        ... it's, it's, it's super, super nice because, yeah, as we're saying, it's, you, you can just kind of do this all inside, basically.

    15. CVClaire Vo27:51

        Yep.

    16. SWScott Wu27:51

        Um, and so that's, that's probably number one for me. I think number two that comes to mind is version upgrades, migrations, things like that. Um, and so, you know, like upgrading your node version or getting onto, you know, the, the latest packages and so on, um, so it's a, it's a big time sav- You know, w- we all have to do it, [chuckles] you know, and then, you know, somehow these new packages just come out so quickly. But obviously, the devil is in the details of, like, finding, you know, the... This new version will say, "Oh, you know, every instance of this component is, you know, we, we recommend that you use-

    17. CVClaire Vo28:21

        Yeah

    18. SWScott Wu28:21

        ... you know, this, this structure instead or something." Um, and Devin will be able to kind of go through that and do the semantic search, and find each of the components, and, and make the right changes. Number three, I would say, is, um, documentation, big one as well. Um, and so we have our, you know, Devin docs, for example, um, like our, our, our own kind of like docs page, uh, like the external docs page. Um, and I, I mean, Devin has written, like, the entire thing, you know. [chuckles] I mean, Deepwiki itself obviously is, is, is, is kind of an extension of that, but, but, you know, even writing your own docs pages or, or putting materials together, um, a lot of what Devin does is go, again, processing the codebase and understanding this references that, and, you know, here, here's what this does, and so on. And so, um, it's, it's, it's a funny one in the sense that it's not strictly a writing code use case, or it isn't always, but, but, but I think it's so closely related to it that a lot of the same capabilities are, are really valuable there. Okay, number four that I would say is, um, incident response, actually.

    19. CVClaire Vo29:22

        Mm.

    20. SWScott Wu29:22

        Um, and so we have this set up so that whenever there's a crash, the first line defender, you know, on, on PagerDuty, basically, is Devin. [chuckles]

    21. CVClaire Vo29:29

        [chuckles] Yeah.

    22. SWScott Wu29:30

        And so Devin gets a page, and Devin gets started, goes and, you know, kind of runs a session. Um, and obviously, you probably want a human there too, you know, [chuckles] especially for, for these big incidents, that to make sure what's going on. But the nice thing is, you know, it's like 4:00 a.m., and, and you're kind of like half asleep, and then you, you get to your computer, and Devin has already written a report of like: "Hey, I looked at it. I think it was this change from, like, last week that happened or, or, or yesterday that happened. Um, you know, here's exactly where, you know, the, the trace of the error goes."

    23. CVClaire Vo29:57

        Mm-hmm.

    24. SWScott Wu29:58

        Um, and so we use that a lot. It's a, it's a huge lifesaver for us. And then number five, let's see. I, I, I, I, I would say adding testing, um, is, is a big one for us. Um, you know, it's a very common thing where, um... Th- this is especially for, for kind of like, um, individual engineers as they're going and working on things. You know, you have your whole PR, um, you, you build things out, you build a new feature, um, and, and always, you know, the last thing that you have to do before you ship it is you have to go and add your own unit tests and make sure your thing works right. Um, and the nice thing, again, is like Devin will go and do that. It'll make the test, and then it'll run the test locally itself and make sure those tests pass, and so it can iterate with you to make sure the lint pass, make sure the CI passes, and so on. Um, and, and just kind of like add those for you.

    25. CVClaire Vo30:41

        All right, well, we're, we're very close. My, my five are very close-

    26. SWScott Wu30:44

        Yes

    27. CVClaire Vo30:44

        ... so I love those. So, so to recap, and I'll augment yours with mine. So number one-

    28. SWScott Wu30:48

        Yeah

    29. CVClaire Vo30:48

        ... frontend fixes. My particular version of frontend fixes is I think these AI tools can really help you do polish, really nice interactive user experiences where you wouldn't normally be able to spend time on them. So any of those, like, little magical moments that you don't wanna, like, toil in frontend on, I think it's really good at.... docs, I think, is underrated. I actually have a GitHub action that every PR gets opened, gets reviewed by Devin, gets the PR description rewritten by Devin, and then after the PR is closed, Devin goes and ships, um, our documentation-

    30. SWScott Wu31:20

        I love that
11. 32:17 – 35:57

    Using ChatGPT voice as a meeting participant

    1. CVClaire Vo32:17

       Very similar. Okay, Scott, we're gonna close with just, uh, one, one really high-level use case outside of the Devin ecosystem, which is voice. And you were telling me a really interesting ChatGPT voice, uh, use case that I hadn't heard before. So do you mind spending a few minutes just telling us about that?

    2. SWScott Wu32:36

       Yeah, for sure. So I, I'm a big fan of voice. I actually think there are a lot of interesting... And, you know, we- we've played a- a- around with vo- You know, we, we, we have voice in Windsurf now, actually, as of Wave 11, too, partially because of that. Um, but, but, but in, in, in short, the way I would describe it is, like, I, I think, um, you know, Google itself, like 20, 25 years ago, what, what was basically a better encyclopedia, right? You know, we have all sorts of things if you wanna look up and, and pull together, and so on, right? And it basically, it got you a faster answer, and it got it to you, you know, with, with more up-to-date information of what was going on. Um, and I, I almost think of ChatGPT voice as, like, a better Google. You know, like, you, you can get an even faster answer. It's fully synchronous. You can do it in the conversation. Um, and then obviously you have all of the detail of... It, it can go and research and, and do these other things, too. What, what, what I'll often do is, you know, if I'm in a meeting, um, and, and we'll, we'll be talking about things, you know, there are always questions that come up. Like yesterday, I was in a meeting and we were talking about this, which is, um, you know, there, there's so many orgs out there with tons of software engineers, and so we were kind of thinking like: Yeah, like, what are all the companies that have, let's say, 10,000-plus software engineers, you know? And how many are there in the world, right? You know, you ob- obviously, like, um, you know, the big banks out there have tens of thousands of software engineers, the big tech companies, you know. Th- those are the first couple... Maybe the Accenture, Infosys, you know, that category. Those are the first ones that come to mind, but, like, what are all these different companies that have it? Um, and, you know, naturally in a meeting, it's kind of rude to just go on your phone and just kind of like, you know, [chuckles] uh, be, be totally unresponsive for, like, two minutes as you're looking. So instead, what, what I'll often do is I'll just pull out ChatGPT and go on voice, um, and it's basically like adding ChatGPT to the conversation, you know? And so then I say, "Hey, like, um, can you, can you please, like, tell us, like, how many, um, h- how many companies out there have 10,000-plus software engineers?" Right. Um, and, and then, you know, whether it's voice-to-voice or whether it's, you know, voice and then you kind of get the response in text, like, I use both of those modes a lot, but I find it to be, like, a very natural, um, um, a, a, a natural stepping stone, where I, I just find that voice lowers the, the friction even further in a way that actually really matters. Like, like, I, I was gonna say, it's like, you know, in the encyclopedia era, right, if you were gonna look something up, it took, like, I don't know, five minutes or something, so you had to go pull the right, like, letter of the alphabet or something [chuckles] and find this. And then Google got it to, like, 10 seconds, you know? And, like, voice is kind of like getting it from, like, 10 seconds down to, like, one or two seconds, where you can just get on instantly and just say what you wanna say. And, and that actually matters, I think, um, for, for, for being able to go back and forth or, or just, like, having, you know, very off-the-cuff, like, o- off-the-cuff questions that you, uh, that you wanna ask.

    3. CVClaire Vo35:16

       Yeah, I was gonna say, I... You know, you've maybe changed my mind here, 'cause I used to think that voice mode was, like, super socially dis- disruptive, in that it feels so in- unnatural to, like, talk during a meeting. But if you flip it on its head and you're like, "No, this is just another meeting participant that I'm putting into the room," it actually is, is more socially inclusive. Everybody hears the result, right? You're not, like, Slacking around links, and then people are opening them up on their laptop and reading while somebody is talking. Like, everybody's sort of, like, clued into the synchronous nature of this new, new information. So, um, if I had people to be in meetings with, and not to brag, but I have very few meetings, um-

    4. SWScott Wu35:51

       [chuckles]

    5. CVClaire Vo35:52

       ... then maybe I will bring Chat- ChatGPT into it. Okay, we will do-

    6. SWScott Wu35:55

       Must be nice. Must

12. 35:57 – 41:11

    Lightning round

    1. SWScott Wu35:57

       be nice.

    2. CVClaire Vo35:57

       Oh, man, it's the dream, man.

    3. SWScott Wu35:59

       [chuckles]

    4. CVClaire Vo36:00

       Um, so quick lightning round questions, we will get you back to-

    5. SWScott Wu36:02

       Yeah

    6. CVClaire Vo36:02

       ... your work. First one, it's like picking between your children, I know now.

    7. SWScott Wu36:07

       [chuckles]

    8. CVClaire Vo36:07

       The IDE, the terminal, or the agent, what is gonna be-

    9. SWScott Wu36:11

       Yeah

    10. CVClaire Vo36:11

        ... the form factor to rule AI engineering?

    11. SWScott Wu36:14

        I, I really think of this in the future as, you know, we call it coding agent and control... Like, a, a lot of what this becomes is actually just the, the next generation of human-computer interface. A- and, like, the, the way that I like to say it is, you know, Tony Stark doesn't have a laptop. [chuckles] You know? Like, like, you, you don't need one at some point if you're just... You, you have your Jarvis plugged in and you're going back and forth with your agent, and then it will go and do these things for you, right? And you can imagine that building software is just kind of like, you're not looking at your code, you're not looking... You know, you're just looking at your own product, right? And you're looking at your own product, and you're saying, "Hey, let's make this button rounder. Let, let me add a new thing over here. Let's save this, and, you know, let- let's, let's, let's ask the user for this and that and so," you know. A- a- and you're just making the changes in real time in your product, and your agent obviously is going and implementing this for you. And so I think it's a... It, it's, it's certainly very agentic, but, but I think it's almost like we, we might, whether we call it an IDE or an agent or whatever, it, it really is basically just like a-... a different human-computer interface, where you are just looking directly at the product rather than having to go through all your code or go through, you know. Um, uh, and, and so, so I think that's the, that's the future version, um, some years out. I think today, I would say, I think, uh, a lot of it depends on the cohort, and so, so I'm, for example, in meetings all the time, [chuckles] unfortunately. You know, not that... But, but, but, but, but, yeah, uh, you know, a- and because of that, I actually think the Slack agent workflow is a super, super natural one, you know, or, or like Linear, for example, and tagging, um, you know, Devin from Linear. Um, I think for an engineering IC who is, is, you know, who gets to code for, for, you know, eight or 10 hours a day, again, must be nice. Uh, but the, the, then the IDE is kind of the natural place where a lot of this starts, right? Which is, you know, you'll have these things that run in the background, and you'll have these asynchronous processes that are going as you're doing your thing, but the natural place to get started for that is the IDE today, I'd say.

    12. CVClaire Vo38:07

        I also just think what's nice about this era is, like, the form factor can come to you, and -

    13. SWScott Wu38:11

        Yeah

    14. CVClaire Vo38:12

        ... you can, you can decide what, what the interface is that works best for, for your workflow. Okay, you know, as somebody... Devin is my buddy. I am sure you get lots of chats that would give us very good insight into my closing question, which is, when you are frustrated with our sweet, sweet intern, Devin, what is, what is your prompting technique? And I know you all monitor this, because when I get frustrated, sometimes I get little credits back. [chuckles] Little credits back, like, "You did that wrong," I get credits back. So I know you see a lot of, um, human language to agents, but what is your strategy? What do you find yourself doing in a moment of, you know, frustration or being blocked?

    15. SWScott Wu38:51

        I can give some advice. I can't say that I've always followed my own advice, but, but, but, but a lot of what it looks like, uh, I, I'd say for, for an agent especially, is, um... I think an agent is a little bit different from, from a chatbot, in the sense that, like, a chatbot, there's less to go off of, is kind of, like, how I want to say it, right? Where with a chatbot, it's like, you know, you ask a question, it gives you the wrong answer, and it's like, "No, that was the wrong answer," you know? [chuckles] And then that, that's all you can really say. With an agent, like, one of the nice things that you can do is you can go through, uh, and look through all the history of what he was doing, right? And so, like, we had a, an, an example of that just now, where, um, you know, Devin got stuck of, like, uh, you know, "I see the chat.openai.com page. It's hard to have an MCP server. I'm, like, trying to find the documentation on this," right? And if we go and scroll through the logs, then we'll see, like, what happened, that it Googled it, and it found some other things, right? Um, and that was what the issue was, right? And so, so from there, it's kind of like you take that information, and then you understand, "Oh, Devin was missing the link to this page," um, and then you send that. Um, and so I think a lot of it actually with agents is just, um, it, it- it's kind of like pair, pa- pair programming or pair debugging with an intern. Like, you wanna, you know, first you get to go through and see, "Okay, here's are all the steps that you took. Oh, by the way, it's like, you know, I, I think you missed this one file, which is, you know, the, the downstream reference of this, and that's why there was the bug," or something like that. Um, I think that's, that, that's, that's the biggest thing that, that will really move the needle.

    16. CVClaire Vo40:12

        Okay, so review the history, figure out where it went wrong, and then, then reinstruct. Okay, Scott, this has been so fun. Thank you for showing us. Where can we find you, and how can we be helpful?

    17. SWScott Wu40:24

        Yeah, yeah, for sure. So, so we're, um, Cognition and Devin on Twitter. Um, we officially got the Twitter [chuckles] of, of/cognition, uh, which is great. Uh, but and, a- and then, um, obviously, it's, it's, it's devin.ai, if you'd like to use the product.

    18. CVClaire Vo40:37

        Great. Well, thank you so much, and I appreciate you spending the time with us.

    19. SWScott Wu40:41

        Cool. Thank you so much for having me. [upbeat music]

    20. CVClaire Vo40:43

        Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time! [upbeat music]

Episode duration: 41:11