Live coding session with Boris Cherny and Jarred Sumner

1. SPSpeaker0:00

   [upbeat music] Please welcome to the stage Head of Claude Code of Anthropic, Boris Cherny, and creator of Bun at Anthropic, Jarred Sumner. [upbeat music]

2. JSJarred Sumner1:22

   [applause] Hello. All right, so this is a developer conference. We're gonna be doing a little bit of talking, but mostly we're just gonna be, like, coding.

3. BCBoris Cherny1:34

   Woo! [applause]

4. JSJarred Sumner1:34

   So this is for the developers in the room.

5. BCBoris Cherny1:37

   [applause]

6. JSJarred Sumner1:38

   I'm gonna start by talking a little bit about how, uh, how Bun uses Claude Code to, to build and, and maintain Bun, and also kind of how our setup, uh, works and s- 'cause it's kind of a, a slightly more advanced setup than, uh, what's common today. Um, but first I'm gonna get a few agents running to just fix some GitHub issues. Uh...

7. BCBoris Cherny2:03

   This is classic Jarred doing work during a talk. [laughs]

8. JSJarred Sumner2:08

   Um, so, so in Bun's repo, uh, every time somebody submits an issue, uh, we have a Claude bot automatically run and try to reproduce the issue. So you can see this person has, uh, this, this side effects, and this is like one of the most recent issues. Um, and we, and we can see that Robobun, which is our, our, our bot, uh, uh, went and managed to reproduce the issue, uh, and submitted a PR automatically. Um, and s- and this PR is like, it has, uh, it, it... all these PRs always have tests. Um, it's one of the actual hard requirements before it can submit a PR. Um, uh, and so, so the challenge here is, like, does this code look correct? Um, and one of the things we do to, to check that is does the test fail in the previous version of Bun and pass, uh, in the, in this debug branch? And it, and the bot actually can't submit a PR without that, uh, being the case.

9. BCBoris Cherny3:11

   And so this is... Just to make sure I understand, so this is, like, every single issue that goes up in the Bun issue tracker, you have Robobun automatically try to reproduce it before anyone looks at it?

10. JSJarred Sumner3:22

    Yeah. And this saves a lot of time because we have so many open GitHub issues.

11. BCBoris Cherny3:26

    Mm.

12. JSJarred Sumner3:26

    Um, it, it really moves the challenge from is, uh, from just fixing and debugging the issue to is this the right thing to merge? Like, is this the right fix?

13. BCBoris Cherny3:36

    Mm.

14. JSJarred Sumner3:36

    Um-

15. BCBoris Cherny3:37

    How, how, how good is it? Is it, is this doing like 100% of PRs? Is it, like, 10%?

16. JSJarred Sumner3:41

    We can go to the Insights, uh, uh, and go to Contributors, and then if we go, uh, last three months, uh, uh, and this is specifically to main, we can see that Robobun is now a bigger contributor to Bun than I am. [laughs]

17. BCBoris Cherny3:57

    [laughs] [clapping]

18. JSJarred Sumner4:00

    And that's with merging not all of its PRs for sure. You can see we have a lot of PRs open right now. Um, the, the challenge is really how do we know can we merge the PR?

19. BCBoris Cherny4:09

    Mm.

20. JSJarred Sumner4:10

    Um, and that's the test. Um, and then the other thing that's really interesting about this is, uh, uh, we have automatic code review bots that, that run, um, and then they're going back and forth. So, like, CodeRabbit leaves a comment, and then Robobun leaves a comment. And then they go back and forth, and they, and Ro- CodeRabbit did the-

21. BCBoris Cherny4:36

    [laughs]

22. JSJarred Sumner4:39

    [laughs]

23. BCBoris Cherny4:39

    I love this.

24. JSJarred Sumner4:42

    Yeah. And, and it also marks the comments as resolved when it's done. Um, and you can see they actually went a lot. Like, there's a lot of back and forth here. There was, like, 30 comments or something.

25. BCBoris Cherny4:52

    And so you're using, like, a combination of agents. So this is, like, code review. This is, like, Claude code review, and then also CodeRabbit, and, like, you're using them together.

26. JSJarred Sumner5:00

    Yeah. And I, I think basically, like, CodeRabbit is, is good for, like, kind of stylistic issues and things that are like make sure that it follows the Claude MD. And then the, the Claude code review is really good at here's this really subtle edge case that would've taken me, like, 30 minutes of reading all the code and having all the context to, to, like, figure out.

27. BCBoris Cherny5:21

    Mm.

28. JSJarred Sumner5:21

    And, and so, so it's really good at surfacing bugs that you, you need the full context to really understand. Um, and I think basically it's, it's really hard to, to actually have all this automation without having code review that i- is in the loop with the, with Claude. They're replying, or replying is, like, very performative, but, like, at, like, fixing.

29. BCBoris Cherny5:48

    Mm.

30. JSJarred Sumner5:48

    Um, and that's also a big part of, like, what used to take so much time, uh, w-when, uh, w- like, why PRs would take so much time to merge is because you'd have to, like, like, check out the branch locally, fix a lint error, then run the linter locally, then push it back up, and there's all this switching cost that's constantly there.And so when you-- so I think this is, like, an especially good use case for, for LLMs because otherwise, like, it just takes up so much time to, to ship.

Episode duration: 32:00