Live coding session with Boris Cherny and Jarred Sumner

1. 0:00 – 1:02

   Intro

   1. SPSpeaker0:00

      [upbeat music]

2. 1:02 – 2:08

   Stage setup: a live, developer-first coding session

   1. SPSpeaker1:02

      Please welcome to the stage Head of Claude Code of Anthropic, Boris Cherny, and creator of Bun at Anthropic, Jarred Sumner. [upbeat music]

   2. JSJarred Sumner1:22

      [applause] Hello. All right, so this is a developer conference. We're gonna be doing a little bit of talking, but mostly we're just gonna be, like, coding.

   3. BCBoris Cherny1:34

      Woo! [applause]

   4. JSJarred Sumner1:34

      So this is for the developers in the room.

   5. BCBoris Cherny1:37

      [applause]

   6. JSJarred Sumner1:38

      I'm gonna start by talking a little bit about how, uh, how Bun uses Claude Code to, to build and, and maintain Bun, and also kind of how our setup, uh, works and s- 'cause it's kind of a, a slightly more advanced setup than, uh, what's common today. Um, but first I'm gonna get a few agents running to just fix some GitHub issues. Uh...

   7. BCBoris Cherny2:03

      This is classic Jarred doing work during a talk. [laughs]

3. 2:08 – 3:11

   RoboBun workflow: auto-reproduce GitHub issues and open PRs with tests

   1. JSJarred Sumner2:08

      Um, so, so in Bun's repo, uh, every time somebody submits an issue, uh, we have a Claude bot automatically run and try to reproduce the issue. So you can see this person has, uh, this, this side effects, and this is like one of the most recent issues. Um, and we, and we can see that Robobun, which is our, our, our bot, uh, uh, went and managed to reproduce the issue, uh, and submitted a PR automatically. Um, and s- and this PR is like, it has, uh, it, it... all these PRs always have tests. Um, it's one of the actual hard requirements before it can submit a PR. Um, uh, and so, so the challenge here is, like, does this code look correct? Um, and one of the things we do to, to check that is does the test fail in the previous version of Bun and pass, uh, in the, in this debug branch? And it, and the bot actually can't submit a PR without that, uh, being the case.

4. 3:11 – 4:10

   From debugging to merging: shifting the bottleneck to review confidence

   1. BCBoris Cherny3:11

      And so this is... Just to make sure I understand, so this is, like, every single issue that goes up in the Bun issue tracker, you have Robobun automatically try to reproduce it before anyone looks at it?

   2. JSJarred Sumner3:22

      Yeah. And this saves a lot of time because we have so many open GitHub issues.

   3. BCBoris Cherny3:26

      Mm.

   4. JSJarred Sumner3:26

      Um, it, it really moves the challenge from is, uh, from just fixing and debugging the issue to is this the right thing to merge? Like, is this the right fix?

   5. BCBoris Cherny3:36

      Mm.

   6. JSJarred Sumner3:36

      Um-

   7. BCBoris Cherny3:37

      How, how, how good is it? Is it, is this doing like 100% of PRs? Is it, like, 10%?

   8. JSJarred Sumner3:41

      We can go to the Insights, uh, uh, and go to Contributors, and then if we go, uh, last three months, uh, uh, and this is specifically to main, we can see that Robobun is now a bigger contributor to Bun than I am. [laughs]

   9. BCBoris Cherny3:57

      [laughs] [clapping]

   10. JSJarred Sumner4:00

       And that's with merging not all of its PRs for sure. You can see we have a lot of PRs open right now. Um, the, the challenge is really how do we know can we merge the PR?

   11. BCBoris Cherny4:09

       Mm.

5. 4:10 – 6:19

   Multi-agent code review: RoboBun + CodeRabbit back-and-forth

   1. JSJarred Sumner4:10

      Um, and that's the test. Um, and then the other thing that's really interesting about this is, uh, uh, we have automatic code review bots that, that run, um, and then they're going back and forth. So, like, CodeRabbit leaves a comment, and then Robobun leaves a comment. And then they go back and forth, and they, and Ro- CodeRabbit did the-

   2. BCBoris Cherny4:36

      [laughs]

   3. JSJarred Sumner4:39

      [laughs]

   4. BCBoris Cherny4:39

      I love this.

   5. JSJarred Sumner4:42

      Yeah. And, and it also marks the comments as resolved when it's done. Um, and you can see they actually went a lot. Like, there's a lot of back and forth here. There was, like, 30 comments or something.

   6. BCBoris Cherny4:52

      And so you're using, like, a combination of agents. So this is, like, code review. This is, like, Claude code review, and then also CodeRabbit, and, like, you're using them together.

   7. JSJarred Sumner5:00

      Yeah. And I, I think basically, like, CodeRabbit is, is good for, like, kind of stylistic issues and things that are like make sure that it follows the Claude MD. And then the, the Claude code review is really good at here's this really subtle edge case that would've taken me, like, 30 minutes of reading all the code and having all the context to, to, like, figure out.

   8. BCBoris Cherny5:21

      Mm.

   9. JSJarred Sumner5:21

      And, and so, so it's really good at surfacing bugs that you, you need the full context to really understand. Um, and I think basically it's, it's really hard to, to actually have all this automation without having code review that i- is in the loop with the, with Claude. They're replying, or replying is, like, very performative, but, like, at, like, fixing.

   10. BCBoris Cherny5:48

       Mm.

   11. JSJarred Sumner5:48

       Um, and that's also a big part of, like, what used to take so much time, uh, w-when, uh, w- like, why PRs would take so much time to merge is because you'd have to, like, like, check out the branch locally, fix a lint error, then run the linter locally, then push it back up, and there's all this switching cost that's constantly there.And so when you-- so I think this is, like, an especially good use case for, for LLMs because otherwise, like, it just takes up so much time to, to ship.

6. 6:19 – 8:00

   Why this works well for Bun (and how it generalizes to support tickets)

   1. BCBoris Cherny6:19

      Mm. And, and I guess, like, especially for the Bun codebase because it's like it's systems code, it's, you know, it's very easy to repro an issue and then see if the issue is fixed. Because this is kinda back to what we were talking about before with, like, this kind of verification loop. It's all systems code, so it's really like a test case on a particular architecture, and you can essentially, like, repro or verify anything.

   2. JSJarred Sumner6:41

      Yeah. That's one of the-- one thing that makes it easier in Bun's codebase is because it's a, a CLI tool, um, because, uh, we don't need to run a browser to test things. But you can also just, like, have something set up to like take a screenshot or record a video or those sorts of things.

   3. BCBoris Cherny6:57

      Mm.

   4. JSJarred Sumner6:57

      Um, in Bun's case, we don't need that, at least not yet. Uh, uh, there's a couple of things we could do that for. Like, we have some front-end stuff that would be nice.

   5. BCBoris Cherny7:06

      Mm-hmm.

   6. JSJarred Sumner7:06

      Um, but yeah, I think this is, like, the, the direction that I think is really interesting is because it saves so much time. Um, and this is not-- this is something that, like, this is specifically for Bun, but the more generalizable thing when, because most products are not open source, um, uh, is like instead of an issue, maybe the starting point is, like, a, a customer support ticket. So, like, you could ha- you could imagine an automatically, uh, h- passing customer support tickets to a Claude bot to then go and try to reproduce the issue and then submit a PR and then, uh, having code review go back and forth. Um, and that's where I think for a lot of companies it becomes a lot more impactful.

   7. BCBoris Cherny7:49

      Mm.

   8. JSJarred Sumner7:50

      Because it just saves so much developer time.

   9. BCBoris Cherny7:53

      We should think of some kinda name for this pattern. Maybe it's like adversarial code review or, or something like that.

   10. JSJarred Sumner7:58

       Yeah, I don't know.

   11. BCBoris Cherny7:59

       Mm.

7. 8:00 – 10:05

   Claude.md and ‘compound engineering’: documenting every repeated lesson

   1. JSJarred Sumner8:00

      Um, but I do think, like, uh, th- there's also a few other things about this that's, uh, like, it-- the-- if you just do this, then it doesn't quite work. The very first step you need is to, like, make sure the development environment is set up. Um, like I, I think this has been talked about before, but like, uh, like Claude MD is, uh, very important, um, because otherwise it's gonna just submit PRs that don't quite make sense for you to merge. Um, so like we very much emphasize in Bun's codebase that it runs this special command to do the build. Um, and this both builds and runs the command, so it, like, forwards the arguments.

   2. BCBoris Cherny8:38

      Mm-hmm.

   3. JSJarred Sumner8:38

      Um, 'cause that's also one confusing thing is like, uh, w- because Bun has to be compiled, uh, you want to make sure that it's running the actual changes and not like a debug build that's like stale. Um, we also go into a lot of detail about how to run tests, how to write tests, um, where to put the test. Um, uh, and a lot of like here's how-- here's all the issues that we've run into previously. Like, basically, the, the pattern here is like every time that you find yourself repeating something, it, it should probably go in Claude MD.

   4. BCBoris Cherny9:14

      Mm.

   5. JSJarred Sumner9:14

      Because the, the question now is like, h-how do you make it maintainable to have lots of Claudes running all the time? And, uh, and to do that, it needs to be written down, it needs to be documented. Um, so like a really small detail is like we, we check the-- we have it, like, to make sure that Claude sees the error message, uh, we make it print the error message before, uh, the, like, less informative, uh, conditions.

   6. BCBoris Cherny9:43

      So th- so this is sorta like you have Claude write a test, and then the test is bad or something about it doesn't work, and then you see this kind of repeated like once or twice, and then you just tell Claude, add it to the Claude MD so that every time in the future when you write a test, you do it correctly the first time.

   7. JSJarred Sumner9:59

      Yeah.

   8. BCBoris Cherny9:59

      So this is like, uh, like compound engineering. It's kinda this. Yeah.

8. 10:05 – 12:10

   Closing the loop: agents that run tests, read CI, and arrive “merge-ready”

   1. JSJarred Sumner10:05

      Um, and then also it's helpful to like give it an overview of where all the folders are, like what all, like how the code is laid out, um, and like about dependencies. Um, uh, uh, another thing I think is interesting is like making sure that it can read your CI errors and like build logs.

   2. BCBoris Cherny10:27

      Mm.

   3. JSJarred Sumner10:27

      Like, you wanna, you wanna set up the agent to be able to, be able to read the co-- like to do the full loop of ma- of like writing the code, testing the code works, checking CI, monitoring CI, um, and, uh, uh, reading all the errors so that way by the time it gets to a person, everything is like set up. Like, the ideal is that you read the code and you, you have very clear indications that you can be high confidence to merge it. And the only way for that to be true is if it is set up for success.

   4. BCBoris Cherny11:01

      It's interesting. I remember when we, when we first met, you were talking about like your vision of like everyone being able to run hundreds of agents in parallel and how that would work. And I feel like I didn't really get it at the time, although now, like every night I, I'm, I'm running like hundreds of agents every single night. And I feel like now I'm finally there. But this is a thing that you've been thinking about for a long time, so it feels like this is sort of like the setup in order to be able to kinda scale up agents way more. Like, you need the self-verification so that agents can run autonomously.

   5. JSJarred Sumner11:29

      Yeah. Like, th-this has gone through many iterations in Bun's codebase. Um, we previously just had a Discord bot where I could just @ mention the bot, and it would spin up a container. Um, uh, it didn't have like the CI stuff, it didn't have, uh, the code review stuff. Um, and it's so much better now, especially with like Opus 4.7. Um, uh, uh, it, it's-- all this stuff is getting so much better. Um, oh yeah, we can also check on how, uh, how it's going. It looks like it created a PR, uh, the first one. Um, uh, and it wrote tests.

9. 12:10 – 12:56

   Adoption curve and model inflection point: ‘47’ makes it practical

   1. BCBoris Cherny12:10

      Um, so maybe while we look at this, I'm curious for, like, just to get a show of hands for people in the room. Like, as people think about their development process, raise your hand if it looks something like this, where you have, like, a bunch of terminal windows or desktop tabs, and you're kinda pasting in issues. Okay. So that's, like, maybe half of people. And then what if it looks like RoboBun or something more like that, where it's, like, closing the loop a little more? It's like the next level of abstraction. Starting to get there. Yeah. I think it's, like... It's not surprising 'cause I think model capabilities are just getting there. Like, I think 47 is the first model where it's really felt like it's able to do this. And in the past, maybe you could do it with, like, a bunch of scaffolding. Like, you just throw a bunch of tokens at it, and it can kinda work. But now it's, like, efficient enough. You can actually do this day-to-day.

10. 12:56 – 15:40

    Live PR triage: judging plausibility, style alignment, and deeper control-flow risks

    1. JSJarred Sumner12:56

       Yeah. Um, let's see. So the first PR is there. Uh, let- let's see if it did any others. Uh, okay, it did two PRs. Um, uh, this looks very plausible.

    2. BCBoris Cherny13:14

       That's cool. And this, like, this before or after, is this, like a... Uh, do you tell it to do that or it-

    3. JSJarred Sumner13:18

       Sometimes it does it.

    4. BCBoris Cherny13:19

       Mm.

    5. JSJarred Sumner13:19

       Um, uh, it's, it's pretty good about knowing, like, when it should do that.

    6. BCBoris Cherny13:24

       Mm.

    7. JSJarred Sumner13:24

       Like, when it's, like, a string formatting thing.

    8. BCBoris Cherny13:26

       Mm-hmm.

    9. JSJarred Sumner13:27

       Um, uh, it's also-- it also kept, like, bun style of the label, which is good because Node style is slightly different.

    10. BCBoris Cherny13:35

        Mm-hmm.

    11. JSJarred Sumner13:36

        Um, let's see. Does this change look good? Um, yeah. Uh, mostly what I'm thinking right now is, uh, it, it, it's, it did this. This is good 'cause, like, you don't wanna write one byte at a time. You wanna write-

    12. BCBoris Cherny13:57

        Mm

    13. JSJarred Sumner13:57

        ... uh, uh, in, in chunks. Um, and then it used, uh, saturation, uh, to, to... But I don't like that. That's... Or, like, you shouldn't have to do that.

    14. BCBoris Cherny14:07

        Does anyone here actually know Zig?

    15. JSJarred Sumner14:10

        Some sort of. [laughs]

    16. BCBoris Cherny14:12

        [laughs]

    17. JSJarred Sumner14:13

        But yeah.

    18. BCBoris Cherny14:14

        This is how I feel looking at this code. [laughs]

    19. JSJarred Sumner14:16

        And, and you can see it followed the patterns from the, from the Cloud MD await using, and then that, that pattern of, of, uh, reading all the prom-- like resolving all the promises at the same time. Um-

    20. BCBoris Cherny14:28

        And, and, and so, like, what's your workflow? Like, when you, when you see something like this, are you usually going in and, like, commenting? Or are you just gonna wait for, like, code review to come in and, and drop a comment?

    21. JSJarred Sumner14:37

        Um, usually it, it, it depends on the-- like how complicated it is. Um, in this case, I'll, usually I'll wait for... Like, this one is actually pretty simple. I feel pretty high confidence that if the tests pass, then, uh, I would probably merge this. Um, uh, but I still would wait for, uh, the code review for, for at least for the cloud code review one to run, um, uh, just in case. 'Cause what I really like about that is it will find things that, like, from, from-- that aren't in the diff that are, like, from tracing the control flow, um, which is what you want when it's, like, a human reviewing it. It's, like, somebody who has a lot of context who, who can think, "What are all the edge cases that this might run into?" And the, and the signal-to-noise ratio is pretty good. It's something maybe, like, ten percent of the time it's wrong. Like, that's-- And, and for, like, how that used to be, like, with other code review products that we've tried, it was like, basically, you had to ignore most of what it said.

    22. BCBoris Cherny15:38

        Mm.

    23. JSJarred Sumner15:38

        Um...

11. 15:40 – 17:53

    How long has this been possible? Scaling commit volume and shifting bottlenecks

    1. BCBoris Cherny15:40

       It's pretty cool. How, how long, how long has something like this worked? Is it, like, a latest model thing? Or, like, have you had, like, like, RoboBun or this kinda, like, automated repro, automated fixing, like, this whole pipeline? Like, how, how long has that been actually possible?

    2. JSJarred Sumner15:52

       We can probably, like, see this in a chart somewhere. Um, that's kind of a lot of commits, but I think that's, that might not be on main. That might be that, the, the Rust thing. Um, uh, uh-

    3. BCBoris Cherny16:08

       Yeah. I heard bun is gonna be written in Rust soon. Is that-

    4. JSJarred Sumner16:10

       I don't know.

    5. BCBoris Cherny16:11

       [laughs]

    6. JSJarred Sumner16:12

       Or there's a... I just have a cloud running, and we'll see what happens. [laughs]

    7. BCBoris Cherny16:18

       [laughs]

    8. JSJarred Sumner16:18

       Um, uh, but you can see, like, the, the, the volume of commits there, it's, like, kinda lower, and then it-

    9. BCBoris Cherny16:25

       Mm

    10. JSJarred Sumner16:26

        ... it's, it's definitely gotten up a lot, and then it's gone up a lot. Uh, n- now, really, the bottleneck is, like, do I feel good about merging this?

    11. BCBoris Cherny16:35

        Mm.

    12. JSJarred Sumner16:35

        Am I confident that its changes are correct? And that's new, 'cause it used to be, like, the code wasn't good enough.

    13. BCBoris Cherny16:41

        What, what do you think, what do you think is left? Like, uh, like, w- what's it gonna take before... Well, like, is there, like, a missing tool or, like, a m- missing model capability or model version or something for you to kinda feel like RoboBun can fully close the loop? Issue comes in, and then, like, fix goes out automatically.

    14. JSJarred Sumner16:58

        I think it needs a little bit more, uh... It needs to-- It takes a lot of time to verify the changes are correct.

    15. BCBoris Cherny17:05

        Mm.

    16. JSJarred Sumner17:06

        Um, this was kind of already true, like, when it's a person pushing up a PR. Um, uh, but I think the, the challenge is, like, how do we make it... How do we make sure to communicate sufficient proof that the changes are correct? Um, or making it easier to, like, roll back things. Um, that's... I think those are kinda the two directions. But I think, like, for the majority of, like, simple issues, we should probably be pressing merge a lot more, and the bottleneck now is actually, like, CI and, like, making sure that, uh, like, like, the... In, in having, like, fully running the code, like, the, the-- making sure all the test stuff works.

    17. BCBoris Cherny17:53

        Mm.

12. 17:53 – 20:22

    Large-feature development with Claude: HTTP/3, image processing, and benchmarks

    1. JSJarred Sumner17:53

       Um, but I think it's, like, basically there for, like... And it, it-- The large projects are still non-trivial, but also I've been doing some pretty large PRs lately with, with Claude, mostly with, in the... Like, not as much with RoboBun, but with, like, Claude Code.

    2. BCBoris Cherny18:09

       Mm.

    3. JSJarred Sumner18:10

       Um, like, uh, we recently added support for, uh, uh, a built-in image processing library to Bun. I could probably pull up the PR. Um-Uh, and that was Claude. Um, uh, and also we did a bunch of follow-up PRs too. Uh-

    4. BCBoris Cherny18:27

       Yeah. It's like, it, it, it's interesting 'cause I think like when I look at different people using Claude Code, everyone is like at a kinda different level of like sophistication or kind of like adoption of this. And I think for me, the hardest thing is the model changes very often, so I have to like constantly retune and kinda recalibrate to what it can do. And like as an engineer, it's hard 'cause it's like a very weird technology. It's like the first technology I've used that's like that. And I, I sort of feel like this is actually, the way that you do it is ahead of how the Claude Code team does it for Claude Code itself.

    5. JSJarred Sumner18:55

       [chuckles]

    6. BCBoris Cherny18:55

       And to me, like that, that, like the, the way the Claude Code team does it is actually very automated, but this is like even further ahead. This is like almost like full liftoff, like full- fully closed loop.

    7. JSJarred Sumner19:04

       Like in the last two weeks, we've added an HTTP/3 server to Bun. Uh, there's a PR for an HTTP/2 server. There's, uh, fetch support for HTTP/3 and HTTP/2. Um, there's this image processing API. There's the ongoing Rust rewrite, which may not ship. Uh, uh, uh, that, that's like the most ambitious one I've done so far. Uh, and I-- "done" is too strong a word 'cause it's very much not done. Uh, uh, uh-

    8. BCBoris Cherny19:32

       And so, and so even something like this, like th- this is like a benchmark. So Cl- Claude ran this benchmark for you?

    9. JSJarred Sumner19:37

       Yeah, Claude ran this benchmark. Uh, I gave it like a... It, this ran in like a, a separate like, uh, uh, on like a Linux box. Um, uh, yeah, and it, and I was like, "Make it faster than Sharp." And [chuckles] that's basically what it did. Um-

    10. BCBoris Cherny19:53

        [laughs]

    11. JSJarred Sumner19:54

        I gave it like a few ideas like, "Oh, you could try like read this code in JavaScript core to like figure out the, how to like avoid cloning the typed array when it's not strictly necessary." Um, but like, it, it then went and did it and figured it out. Um-

    12. BCBoris Cherny20:10

        Mm.

    13. JSJarred Sumner20:11

        Uh, and like, uh, yeah, it's pretty crazy honestly, 'cause it was, these, th- this wouldn't, none of this would've worked, uh, several months ago.

13. 20:22 – 22:14

    Hill climbing explained: metrics + verification enable autonomous iteration

    1. BCBoris Cherny20:22

       Hmm. Yeah. I f- I feel like, like within, like within Anthropic, like within an AI lab, you call this kind of thing hill climbing, and th- this is this idea that like if you give the model some sort of metric, and then you give it a way to verify its result, you can just make it iterate and keep going and keep going until it hits that metric. Um, and th- this is something like 47 I think is uniquely good at, and I think it's something really underutilized because I think it's the first model that's actually very good at that. And if you just give it a target, you give it a way to like improve the performance, and you give it a way to measure, it'll just like keep going until, until it's done if you let it go in auto mode.

    2. JSJarred Sumner20:57

       Yeah. And you can also see like, uh, this is another case where like the code review comments was really, really helpful because like there was, like in this PR, there was like 100 comments or something. Uh, and it's just going and fixing everything. Um, uh, uh, like it goes on for a while.

    3. BCBoris Cherny21:19

       And in the me- in the meantime, you're just working on something else.

    4. JSJarred Sumner21:21

       Yeah. I was-- This was not like my, the thing I was 100% focused on. I was maybe like 10% focused on this. I was doing like five things at once. Um, uh, uh, and this definitely wasn't possible like six months ago, um, three months ago. Like th- this is like very recent that-

    5. BCBoris Cherny21:41

       Yeah

    6. JSJarred Sumner21:41

       ... this is doable.

    7. BCBoris Cherny21:42

       Okay. So how, how are sessions doing? Are you-

    8. JSJarred Sumner21:46

       Yeah. So we have one PR there. There's almost another PR coming up, it looks like pretty soon. Um, uh, this one is, should be the trickiest one. Uh, um, it mostly looks good though. Um, like the, or like it looks plausible based on these, these changes. I wouldn't exactly do it this way, but I think that's, we need like a better, op- more optimized way to do this-

    9. BCBoris Cherny22:11

       Mm-hmm

    10. JSJarred Sumner22:11

        ... 'cause that's a lot of checks. Um-

14. 22:14 – 25:03

    Claude Code UX and autonomy: auto mode, permissions, no-flicker CLI, and monitoring loops

    1. BCBoris Cherny22:14

       And, and looking, looking at your setup here, so you're using, you mostly use CLI?

    2. JSJarred Sumner22:19

       Yeah.

    3. BCBoris Cherny22:20

       And, um, do you always use auto mode for permissions?

    4. JSJarred Sumner22:23

       Auto-- Yeah. And then before that, I used dangerouslySkipPermissions. Um-

    5. BCBoris Cherny22:29

       [laughs] [audience applauding]

    6. JSJarred Sumner22:30

       [laughs]

    7. BCBoris Cherny22:31

       No, you guys, it can delete stuff if you do that.

    8. JSJarred Sumner22:33

       [chuckles] I don't-

    9. BCBoris Cherny22:34

       Auto mode is better.

    10. JSJarred Sumner22:34

        I think I'm not supposed to recommend that. Um, but I think it's just not fun to wait for Claude to like press Approve 'cause then you just like go off and do something else, and then it's just been sitting there. So that's why auto mode is really good, 'cause it's actually like a real way to fix that instead of just like trusting.

    11. BCBoris Cherny22:53

        Mm-hmm. And I, I also noticed like the, the input, like the little composer, it's tucked to the bottom of the screen, so you're using no flicker mode.

    12. JSJarred Sumner23:01

        Yeah. I'm using no flicker. Honestly, I think we should just like make that the default 'cause it's so much better. Um, like you can see, I can scroll really fast, and like, like you could scroll fast before, but like sometimes there would be a flicker, and now there's not. Um-

    13. BCBoris Cherny23:16

        Ha- have folks tried no flicker mode for CLI? Yeah, a few people? Yeah. So it's like we launched it on April Fools', so-

    14. SPSpeaker23:26

        [laughing]

    15. BCBoris Cherny23:27

        ... you can think i- [laughs] in hindsight, it came across as a joke a little bit. But if you actually do Claude code no flicker equals one Claude, so just like set that environment variable, we totally rewrote the renderer that's running in the CLI. So it's using virtualized scrolling, virtualized selection, and so what this means is like constant memory usage, constant CPU usage, and also some nice stuff like, uh, like if Jarred types, he can actually like click around the composer, and so you can actually click and mouse events work, which is pretty crazy for a terminal.

    16. JSJarred Sumner23:59

        So I'm just also having it monitor the PR. Um, and you can see it's, uh, it ran some commands, and then it's gonna go to sleep for twenty minutes and wake back up. Um, twenty minutes is probably a little bit too long, but it's okay. Um, uh, oops.

    17. BCBoris Cherny24:14

        And what's that? Is that like using like a loop or something?

    18. JSJarred Sumner24:17

        I think so. Uh, uh, uh-Yeah, and then it's... Uh, let's see, how else is it doing? And then the other ones are still, uh, apparently it fixed an extra bug as well. Uh, uh-

    19. BCBoris Cherny24:36

        Okay, so we got-- Okay, so it's been, what? It's been like twenty minutes or so, twenty-five minutes, and we got one... How many PRs have we gotten? Two?

    20. JSJarred Sumner24:43

        Uh, three PRs.

    21. BCBoris Cherny24:46

        Three PRs. That's not bad.

    22. JSJarred Sumner24:48

        Yeah, and I think we'll get a fourth one once it finishes running the tests.

    23. BCBoris Cherny24:53

        Hmm. And then in the meantime, RoboBun is still running and kinda like generating even, even more PRs.

    24. JSJarred Sumner24:58

        Yeah. Every time somebody submits an issue, it, it tries to reproduce it.

15. 25:03 – 28:45

    What’s next: deeper verification, planning/taste, and when to trust auto-merge

    1. BCBoris Cherny25:03

       Yeah. I k- I kinda feel like every-- Like, the, the way Claude Code makes you think is every time there's a new bottleneck, you have to kinda automate that bottleneck, and then there's always some other bottleneck after, and you kinda move on to that. And, like, it started, like, writing code was the bottleneck, and now it's no longer the bottleneck. And then, like, v-verification and running tests, that was, like, the bottleneck, and that's no longer the bottleneck. And now there's, like, a deeper layer of verification. Maybe that, maybe that's it. What do, what do you think are the bottlenecks remaining?

    2. JSJarred Sumner25:31

       It's definitely this deeper layer of verification. Um, I feel like the bottleneck after that's gonna be, like, planning.

    3. BCBoris Cherny25:38

       Mm-hmm.

    4. JSJarred Sumner25:39

       Like, what should we do, and what should we not do? And what is the right way to fix this? Um, and, like, ideally, Claude would be smart enough, or, like, we could trust Claude enough to, to merge the PRs by itself.

    5. BCBoris Cherny25:52

       Mm-hmm.

    6. JSJarred Sumner25:52

       Um, and I think, like, in a, in certain projects, you could pro- probably do that, um, and just have that be automatic completely. Um, I think Bun is not yet, or, like, it's not yet there for Claude. Or sorry, for, for Bun. Um, but, uh, I think it'd be really cool if it, if, like, we had the tooling, uh, for us to feel confident enough to do that.

    7. BCBoris Cherny26:14

       Mm-hmm. So, so, like, right now, RoboBun, it doesn't, like, build features. It, it doesn't-

    8. JSJarred Sumner26:18

       Um-

    9. BCBoris Cherny26:18

       ... do, like, feature requests yet.

    10. JSJarred Sumner26:19

        It, it-- That's true, yeah. It doesn't do feature requests, but it, we do also use it sometimes. We, we-- So we can also, uh, uh, @mention it in either Discord or Slack, and it will, like, try to implement the feature. Um, uh, so sometimes when people are like, "Hey, Bun is missing this thing," um, then I just @mention the bot, and maybe, like, an hour later, there's a PR.

    11. BCBoris Cherny26:44

        Mm-hmm.

    12. JSJarred Sumner26:44

        Um, a b- a bunch of times I've-- somebody's, like, tweeted at me something like, "This-- Can you fix this bug?" Or whatever, and that's basically what I do.

    13. BCBoris Cherny26:52

        [laughs]

    14. JSJarred Sumner26:53

        Um, and then I reply with a link to the PR. Um-

    15. BCBoris Cherny26:56

        Should we add a RoboBun account on Twitter-

    16. JSJarred Sumner26:58

        [laughs]

    17. BCBoris Cherny26:58

        ... in a second?

    18. JSJarred Sumner27:00

        Um, uh, and I think, like... So, like, it, it can do feature requests, but I'm, I'm hesitant for it to implement literally everything anybody asks for in a GitHub issue, 'cause that's kind of a lot.

    19. BCBoris Cherny27:13

        'Cause in some ways it's k- it's kinda crazy to put something like a image processing library inside of Bun. But, like, we talk about engineering taste, and there's, like, an element of taste that goes into that.

    20. JSJarred Sumner27:22

        Yeah.

    21. BCBoris Cherny27:22

        Like, you felt like that's a good idea. And, like, we're not sure yet if Claude is at the point where it would also think this is a good idea. But, you know, at some point in the future it'll get there.

    22. JSJarred Sumner27:31

        Yeah.

    23. BCBoris Cherny27:31

        Maybe it's starting to get there.

    24. JSJarred Sumner27:32

        Totally. And I do think that, like, PRs become suggestions. Like, having, uh... Like, not merging PRs used to be... Y- like, you feel bad if you don't merge, like, a coworker's PR, uh, because, like, they put work into that. But you don't have to feel bad when it's, like, Claude. Um, uh, so, like, if the PR is wrong or, or, or, or, or whatever reason, then you can just not merge it. And, uh, but it does mean that, like, the, the, like, the, the bar for, uh, w-what you, you merge is, like, should, should it be there? 'Cause I think there's also a difference with, with when it's, like, people. Because with people you want, you don't want people to feel bad about their, like, lo-lost work. Um-

    25. BCBoris Cherny28:21

        Mm-hmm.

    26. JSJarred Sumner28:23

        So sort of in some ways it does actually end up raising the bar for what you decide to merge.

    27. BCBoris Cherny28:28

        Hmm. Yeah, it's, it's interesting how, like, as the bottlenecks move, the dynamics change a little bit. It's sort of like having to trust each other, having to trust people on the team. This kinda changes a little bit. Now it's a little bit more about, like, do we have the right automation, and, like, do we trust the automation, like, as, as a group.

    28. JSJarred Sumner28:44

        Yeah.

16. 28:45 – 31:59

    Wrap-up: watching the fourth PR land and the vision of iterative bottleneck automation

    1. BCBoris Cherny28:45

       So I think we're almost at time. Is there any, like, maybe one last thing we wanna, we wanna show people, or we can kinda check in on kinda, like, the, the progress that we've made?

    2. JSJarred Sumner28:55

       Um, I don't think so. Uh, uh, yeah, it's, it's still going one last o-onto that fourth PR. It's going back and forth, like, found a bug and then fixed a bug.

    3. BCBoris Cherny29:07

       Mm-hmm.

    4. JSJarred Sumner29:08

       Uh, looks like it's about to submit the PR now.

    5. BCBoris Cherny29:12

       Okay. Let's wait for this one.

    6. JSJarred Sumner29:14

       [laughs]

    7. BCBoris Cherny29:16

       Just love to have one more. This is, like, the cool thing about auto mode. It's, like, in auto mode I can let... Like, Claude runs, like, for hours and hours at a time. Like, I run it almost every night. I'll just have a bunch of Claudes running in auto mode. And it's like, before this, it just didn't work 'cause it always got stuck at some kind of permission request. And th-that was crazy. Like, this entire thing was one prompt, and, and that just ran for thirty minutes.

    8. JSJarred Sumner29:38

       Yeah. This is all I said.

    9. BCBoris Cherny29:47

       Okay. So it's pushing it.

    10. JSJarred Sumner29:50

        It's about to submit the PR. That sounds like the right fix too. This has been an issue that's been open for a long time.

    11. BCBoris Cherny30:01

        Mm-hmm. Okay. And we got a PR.

    12. JSJarred Sumner30:14

        Yeah, we can go in this issue and we can see how many outputs it has. Twenty.

    13. BCBoris Cherny30:19

        Yeah.

    14. JSJarred Sumner30:19

        It's kind of a lot.

    15. BCBoris Cherny30:23

        Cool. Maybe we can pause there. Um, but to me, this is just, like, such a cool vision of where engineering is going, I think, for everyone in this room. And, you know, we're gonna see this first. We're gonna have to figure it out first, and then everyone else is gonna have to figure this out. So, you know, like we were talking about this morning, just excited to be on this journey together. And like you can see, we haven't figured everything out yet. Um, but I think, like, the mode that we're in is just constantly experimenting, constantly trying to, to see what the next bottleneck is so we can solve it.

    16. JSJarred Sumner30:57

        It's very exciting. This is so cool. [audience applauding] I like this stuff more.

    17. BCBoris Cherny31:07

        Yeah. Cool. All right. Should I, like-

    18. JSJarred Sumner31:11

        Yeah. You can leave it. [outro music]

Episode duration: 32:00