Cursor Team: Future of Programming with AI | Lex Fridman Podcast #447

Lex Fridman 0:00 The following is a conversation with the founding members of the Cursor team, Michael Truhl, Swale Asif, Arvid Lundmark, and Aman Singer. Cursor is a code editor based on VS Code that adds a lot of powerful features for AI-assisted coding. It has captivated the attention and excitement of the programming and AI communities, so I thought this is an excellent opportunity to dive deep into the role of AI in programming. This is a super technical conversation that is bigger than just about one code editor. It's about the future of programming, and in general, the future of human AI collaboration in designing and engineering complicated and powerful systems. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description. And now, dear friends, here's Michael, Swale, Arvid, and Aman.

Lex Fridman 0:59 All right, this is awesome. We have Michael, Aman, Swale, Arvid here from the Cursor team. First up, big ridiculous question, what's the point of a code editor?

Michael Truell 1:10 So the- the code editor is largely the place where you build software, and today, or for a long time, that's meant the place where you text edit, uh, a formal programming language. And for people who aren't programmers, the way to think of a code editor is, like, a really souped-up word processor for programmers, where the reason it's- it's souped up is code has a lot of structure. And so the- the, quote unquote, "word processor," the code editor can actually do a lot for you that word processors, you know, sort of in the writing space haven't been able to do for- for people editing text there. And so, you know, that's everything from giving you visual differentiation of, like, the actual tokens in the code to, so you can, like, scan it quickly, to letting you navigate around the code base, sort of like you're navigating around the internet with, like, hyperlinks. You're going to sort of definitions of things you're using, to error checking, um, to, you know, to catch rudimentary bugs. Um, and so traditionally, that's what a code editor has meant, and I think that what a code editor is, is going to change a lot over the next 10 years, um, as what it means to build software maybe starts to look a bit different.

Lex Fridman 2:16 I th- I think also a code editor should just be fun.

Aman Sanger 2:19 Yes. That is very important. That is very important, and it's actually sort of an underrated aspect of how we decide what to build. Like, a lot of the things that we build, and then we- we try them out, we do an experiment, and then we actually throw them out because they're not fun. And- and so a big part of being fun is, like, being fast a lot of the time. Fast is fun.

Lex Fridman 2:42 Yeah, fast is... (laughs)

Aman Sanger 2:44 Yeah. (laughs)

Lex Fridman 2:45 Oh yeah, that should be a T-shirt.

Aman Sanger 2:47 (laughs)

Michael Truell 2:48 But, like, fundamentally, I think one of the things that draws a lot of people to- to building stuff on computers is this, like, insane integration speed, where, you know, in other disciplines, you might be sort of gatekept by resources or the ability, even the ability, you know, to get a large group together, and coding is this, like, amazing thing where it's you and the computer, and, uh, that alone, you can- you can build really cool stuff really quickly.

Lex Fridman 3:09 So for people who don't know, Cursor is this super cool new editor that's a fork of VS Code. It would be interesting to get your kind of explanation of your own journey of editors. How did you... I think all of you are f- were big fans of VS Code with Copilot. How did you arrive to VS Code, and how did that lead to your journey with Cursor?

Michael Truell 3:33 Yeah, um, so I think a lot of us, well a- all of us were originally Vim users.

Aman Sanger 3:40 Pure- Pure Vim.

Michael Truell 3:40 Pure Vim, yeah.

Aman Sanger 3:41 (laughs)

Michael Truell 3:41 No Neo Vim, just pure Vim in a terminal. And at le- at least for myself, it was around the time that Copilot came out, so 2021, that I really wanted to try it. So I went into VS Code, the only platform, uh, the only code editor in which it was available, and even though I, you know, really enjoyed using Vim, just the experience of Copilot with- with VS Code was more than good enough to convince me to switch. And so that kind of was the default until we started working on Cursor.

Lex Fridman 4:14 And, uh, maybe we should explain what Copilot does. It's like a really nice autocomplete. It suggests, as you start writing a thing, it suggests one or two or three lines, how to complete the thing. And there's a fun experience in that, y- you know like when you have a close friendship and your friend completes your sentences?

Aman Sanger 4:33 (laughs)

Michael Truell 4:34 (laughs) Yeah.

Lex Fridman 4:34 Like, when it's done well, there's an intimate feeling. Uh, there's probably a better word than intimate, but there's a, there's a cool feeling of like, "Holy shit, it- it gets me."

Michael Truell 4:43 Yeah. (laughs)

Aman Sanger 4:43 For real. Yeah.

Lex Fridman 4:44 You know? Now, and then there's an unpleasant feeling when it doesn't get you. Uh, and so there's that- that kind of friction, but, uh, I would say for a lot of people, the feeling that it gets me overpowers the it doesn't.

Aman Sanger 4:55 And I think actually one of the underrated aspects of GitHub Copilot is that even when it's wrong, it's- it's, like, a little bit annoying, but it's not that bad, because you just type another character, and then maybe then it gets you, or you type another character and then- then it gets you. So even when it's wrong, it's not that bad.

Michael Truell 5:09 Yeah, you- you can sort of iterate- iterate and fix it.

Aman Sanger 5:11 Yeah.

Michael Truell 5:11 I mean, the other underrated part of Y- uh, Copilot for me sort of was just the first real- real AI product. So the first language model consumer product.

Lex Fridman 5:21 So Copilot was kind of like the first killer app for, uh, LLMs.

Michael Truell 5:26 Yeah. Yeah.

Aman Sanger 5:27 Yeah.

Michael Truell 5:27 And, like, the beta was out in 2021.

Lex Fridman 5:29 Right. Okay.

Michael Truell 5:31 (laughs)

Lex Fridman 5:31 Uh, so what's the- the origin story of Cursor?

Michael Truell 5:34 So around 2020, the scaling loss papers came out from- from OpenAI, and that was a moment where this looked like clear, predictable progress for the field, where even if we didn't have any more ideas, looked like you could make these models a lot better if you had more compute and more data.

Lex Fridman 5:49 Uh, by the way, we'll probably talk, uh, for three to four hours on- on the topic of scaling laws.

Michael Truell 5:55 (laughs)

Aman Sanger 5:55 (laughs) Yes. Yes.

Michael Truell 5:55 (laughs)

Lex Fridman 5:55 But just... (laughs) Just to summarize, it's a paper and a set of papers and a set of ideas that say bigger might be better for model size and data size in the- in the realm of machine learning.

Aman Sanger 6:05 It's bigger and better, but predictively better.

Lex Fridman 6:08 Okay, that's another topic of conversation.

Michael Truell 6:10 Yes, yeah.

Lex Fridman 6:10 But anyway, so-

Michael Truell 6:11 So around that time, for some of us, there were, like, a lot of conceptual conversations about, "What's this gonna look like? What's the- the story gonna be for all these different knowledge worker fields, about how they're gonna be, um, made better, uh, by this technology getting better?" And then, um, I think there were a couple of moments where, like, the theoretical gains predicted in that paper, uh, started to feel really concrete, and it started to feel like a moment where you could actually go and not, you know, do a PhD if you wanted to work on, uh, do useful work in AI. It actually felt like now there was this- this whole set of systems one could build that were really useful. And I think that the first moment, we already talked about a little bit, which was playing with the early beta of Copilot. Like, that was awesome and magical. Um, I think that the next big moment where everything kind of clicked together was actually getting early access to GPT-4. So it was sort of end of 2022 was when we were, um, tinkering with that model. And the step-up in capabilities felt enormous. And previous to that, we had been working on a couple of different projects. We had been, um... Because of Copilot, because of Scaling Laws, because of our prior interest in the technology, we had been, uh, tinkering around with tools for programmers, but things that are, like, very specific. So, you know, we were building tools for, uh, financial professionals who have to work within a Jupyter Notebook or, like, you know, playing around with, can you do static analysis with these models? And then the step-up in GPT-4 felt like, look, that really made... can create the theoretical gains that, um, we had predicted before. It felt like you could build a lot more just immediately at that point in time. And also, if we were being consistent, it really felt like, um, this wasn't just gonna be a point solution thing. This was gonna be all of programming was gonna flow through these models. It felt like that demanded a different type of programming environment, a different type of programming. And so we set off to build that- that sort of larger vision, uh, around then.

Arvid Lunnemark 7:59 There's one that I distinctly remember. So my roommate is an, an IMO Gold winner, and, uh, there's a competition in the US called the Putnam, which is sort of the IMO for college people, and it's- it's this math competition. H- he's exceptionally good. So Shengtan and Aman, I remember it sort of... So June, June of 2022, had this bet on whether the mo- like, 2024, June or July, you were going to win a gold medal in the IMO with the- with, like, models.

Lex Fridman 8:31 IMO is the-

Arvid Lunnemark 8:31 And-

Lex Fridman 8:32 ... International Math Olympiad.

Arvid Lunnemark 8:33 Uh, yeah, IMO is International Math Olympiad. And so Arvid and I are both sort of... you know, also competed in it. So it was sort of personal and-

Lex Fridman 8:42 (laughs)

Michael Truell 8:43 (laughs)

Arvid Lunnemark 8:43 ... and I- I remember thinking, "Man, this is just... This is not gonna happen." This was like... It was un- like, even though I- I sort of believed in progress, I thought, you know, IMO Gold just... like, Aman is just delusional.

Michael Truell 8:55 Yeah.

Arvid Lunnemark 8:56 That was the, that was the... And to be honest, it, I mean, I- I was, to be clear, very wrong, but that was maybe the most prescient bet in the group.

Lex Fridman 9:05 So the- the- the new results from DeepMind, it turned out that you were correct. That's what the-

Michael Truell 9:10 I was-

Arvid Lunnemark 9:10 Well, it was-

Michael Truell 9:11 ... close.

Arvid Lunnemark 9:11 ... technically not.

Michael Truell 9:12 Technically incorrect, but one point away. Aman was very enthusiastic about this stuff.

Arvid Lunnemark 9:17 Yeah.

Michael Truell 9:17 Uh, and before, Aman had this, like, Scaling Laws t-shirt that he would walk around with-

Lex Fridman 9:22 (laughs)

Michael Truell 9:22 ... where it had the, like, charts and, like, the formulas on it.

Lex Fridman 9:25 Oh, so you, like, felt the AGI or you felt the, uh, Scaling Laws?

Michael Truell 9:28 (laughs) Yeah, I- I distinctly remember there was this one conversation, uh, I had with- with Michael where, before I hadn't thought super deeply and critically about Scaling Laws, and he kind of posed the question, "Why isn't scaling all you need?" Or, "Why isn't scaling going to result in massive gains in progress?" And I think I went through like the- like the stages of grief. There is anger, denial and then finally, at the end, just thinking about it, uh, acceptance. Um, and I think I've been quite hopeful and, uh, optimistic about progress since. I think one thing I'll caveat is I think it also depends on, like, which domains you're gonna see progress. Like, math is a great domain because... especially, like formal theorem proving, because you get this fantastic signal of actually verifying if the thing was correct. And so this means something like RL can work really, really well. And I think, like, you could have systems that are perhaps very superhuman at math and still not technically have AGI.

Lex Fridman 10:27 Okay, so can we take it all the way to Cursor?

Michael Truell 10:30 Mm-hmm.

Lex Fridman 10:30 And what is Cursor? It's a fork of VSCode, and VSCode is one of the most popular editors for a long time. Like, everybody fell in love with it. Everybody left Vim. I left Emacs for it. Sorry.

Michael Truell 10:44 (laughs)

Arvid Lunnemark 10:45 (laughs)

Lex Fridman 10:46 Uh, so it unified, in some fun- fundamental way, the, uh, the- the developer community. And then, the... you look at the space of things, you look at the Scaling Laws, AI is becoming amazing, and you decided, "Okay, it's not enough to just write an extension for your VSCode because there's a lot of limitations to that. We're... we need... if AI is going to keep getting better, better and better, we need to really, like, rethink how the- the AI is going to be part of the editing process." And so you decide to fork VSCode-

Michael Truell 11:16 Yeah.

Lex Fridman 11:16 ... and start to build a lot of the amazing features we'll be able to- to, uh, to talk about. But what was that decision like? Because there's a lot of extensions-

Michael Truell 11:25 Mm-hmm.

Lex Fridman 11:25 ... including Copilot of VSCode that are doing sort of AI type stuff. What was the decision like to just fork VSCode?

Michael Truell 11:33 So the decision to do an editor seemed kind of self-evident to us for at least what we wanted to do and achieve. Because when we started working on the editor, the idea was, "These models are going to get much better. Their capabilities are gonna improve, and it's going to entirely change how you build software," both in a, "You will have big productivity gains," but also radical in how, like, the act of building software is going to change a lot. And so you're very limited in the control you have over a code editor if you're a plug into an existing coding environment. Um, and we didn't want to get locked in by those limitations. We wanted to be able to, um, just build the most useful stuff.

Lex Fridman 12:08 Okay, well then the natural question is...You know, VS Code is kind of, with Copilot, a competitor. So how do you win? Is it- is it basically just the speed and the quality of the features?

Michael Truell 12:20 Yeah, I mean, I think this is a space that is quite interesting, perhaps quite unique, where if you look at previous tech waves, maybe there's kind of one major thing that happened and it unlocked a new wave of companies. But every single year, every single model capability, uh, or jump you get in model capabilities, you now unlock this new wave of features, things that are possible, especially in programming. And so, I think in AI programming, being even just a few months ahead, let alone a year ahead, makes your product much, much, much more useful. I think Cursor, a year from now, will need to make the Cursor of- of today look obsolete. And I think, you know, Microsoft has done a number of, like, fantastic things, but I don't think they're in a great place to really keep innovating and pushing on this in the way that a startup can.

Lex Fridman 13:13 Just rapidly implementing features.

Michael Truell 13:15 A- and push- Yeah, like, and- and kind of doing the research experimentation necessary, um, to really push the ceiling.

Arvid Lunnemark 13:24 I don't- I don't know if I think of it in terms of features as I think of it-

Michael Truell 13:26 Mm-hmm.

Arvid Lunnemark 13:26 ... in terms of, like, capabilities for- for programmers. It's that, like, you know, as, you know, the new o- one model came out, and I'm sure there are gonna be more- more models of different types, like longer context and maybe faster. Like, there's all these crazy ideas that you can try, and hopefully 10% of the crazy ideas will make it into something kinda cool and useful. And, uh, we want people to have that sooner. To rephrase, it's like, an underrated fact is we're making it for ourself. When we started Cursor, you really felt this frustration that, you know, models, you could see models getting better, uh, but the Copilot experience had not changed. It was just like, "Man, these- these guys, like, the ceiling is getting higher. Like, why are they not making new things? Like, th- they should be making new things. They should be like," You know, like- like, "Where's- where's- where's all the alpha features?" There- there were no alpha features. It was like, uh, I- I'm sure it w- it was selling well. I'm- I'm sure it was a great business, but it didn't feel... I- I'm- I'm one of these people that really want to try and use new things, and it was like just there was no new thing for, like, a very w- long while.

Lex Fridman 14:35 Yeah, it's interesting. Uh, I don't know how you put that into words, but when you compare Cursor with Copilot, Copilot pretty quickly became, started to feel stale for some reason.

Aman Sanger 14:45 Yeah, I- I think one thing that I think, uh, helps us is that we're sort of doing it all in one.

Lex Fridman 14:52 Mm-hmm.

Aman Sanger 14:52 We're- we're developing the- the UX and the way you interact with the model, um, at the same time as we're developing, like, how we actually make the model give better answers. So, we're like, how you build up the- the prompt or- or, like, how do you find the context, and for a Cursor tab, like, how do you train the model? Um, so I think that helps us to have all of it, like, sort of like the same people working on the entire experience end-to-end.

Arvid Lunnemark 15:17 Yeah, it's like the- the person making the UI and the person training the model, like, sit to, like, 18 feet away. So-

Aman Sanger 15:23 Mm-hmm. Often the same person, even.

Arvid Lunnemark 15:25 Yeah, often- often even the same person. So, you- you can- you can create things that are- that are sort of not possible if you're not- you're not talking, you're not experimenting.

Lex Fridman 15:34 And you're using, like you said, Cursor to write Cursor.

Arvid Lunnemark 15:37 Of course.

Aman Sanger 15:37 Oh, yeah.

Arvid Lunnemark 15:38 Yeah.

Lex Fridman 15:39 Well, let's talk about some of these features. Let's talk about the all-knowing, the all-powerful, praise be to the tab.

Aman Sanger 15:46 (laughs)

Michael Truell 15:46 (laughs)

Lex Fridman 15:47 The, you know, auto-complete on steroids, basically. So what, how does tab work? What is tab?

Michael Truell 15:53 To highlight and summarize at a high level, I'd say that there are two things that Cursor is pretty good at right now. There- there are other things that it does, um, but two things that it helps programmers with. One is this idea of looking over your shoulder and being like a really fast colleague who can kind of jump ahead of you and type and figure out what you're- what you're gonna do next. And that was the original idea behind... That was ki- kind of the kernel of the idea behind, uh, good auto-complete, was predicting what you're gonna do next. But you can make that concept even more ambitious by not just predicting the characters after your cursor, but actually predicting the next entire change you're gonna make, the next diff, the next place you're gonna jump to. Um, and the second thing Cursor is- is pretty good at right now, too, is helping you sometimes jump ahead of the AI and tell it what to do and go from instructions to code. And on both of those, we've done a lot of work on making the editing experience for those things ergonomic, um, and also making those things smart and fast.

Arvid Lunnemark 16:54 One of the things we really wanted was we wanted the model to be able to edit code for us. Uh, that was kind of a wish, and we had m- multiple attempts at it before- before we had a sort of a good model that could edit code for you. Um, then after- after we had a good model, I think there- there have been a lot of effort to, you know, make the inference fast for, you know, uh, having- having a good- good experience. And, uh, we've been starting to incorporate... I mean, Michael sort of mentioned this, like, ability to jump to different places, and that jump to different places, I think, came from a feeling of, you know, once you- once you accept an edit, um, it's like, "Man, it should be just really obvious where to go next." It's like- it's like I- I'd made this change, the model should just know that, like, the next place to go to is, like, 18 lines down. Like, uh, if you're- if you're a WIM user, you could press 18JJ or whatever. But, like, why- why even- why am I doing this? Like, the model- the model should just know it. And then so- so the idea was, yo, you just press tab, it would go 18 lines down and then make... It show you- show you the next edit and you would press tab. So it was just you, as long as you could keep pressing tab. And so the internal competition was, how many tabs can we make someone press? Once you have, like, the idea, uh, more- more, uh, sort of...... abstractly, the, the thing to think about is sort of like once... How m- how, how are the edits sort of zero, zero entropy? So once you've sort of expressed your intent and the edit is... there's no, like, new bits of information to finish your thought, but you still have to type some characters to, like, make the computer understand what you're actually thinking. Then maybe the model should just sort of read your mind and, and all the zero entropy bits should just be, like, tabbed away.

Sualeh Asif 18:40 Yeah. There's-

Arvid Lunnemark 18:41 That, that was, that was sort of the abstract version.

Sualeh Asif 18:42 There, there's this interesting thing where if you look at language model loss on, on different domains, um, I believe the bits per byte, which is kind of character normalized loss for code, is lower than language, which means in general, there are a lot of tokens in code that are super predictable, a lot of characters that are super predictable. Um, and this is, I think even magnified when you're not just trying to auto-complete code, but predicting what the user's going to do next in their editing of existing code. And so, you know, the gold Cursor tabs, let's eliminate all the low entropy actions you take inside of the editor. When the intent is effectively determined, let's just jump you forward in time. Skip you forward.

Lex Fridman 19:22 Well, well g- what's the intuition and what's the technical details of how to do next Cursor prediction? The, the, that jump, that's not, that's not so intuitive, I think, to people.

Sualeh Asif 19:31 Yeah. I think I can speak to a few of the details on how, how to make these things work. They're incredibly low latency, so you need to train small models on this, on this task. Um, in particular, they're incredibly pre-filled token hungry. What that means is they have these really, really long prompts where they see a lot of your code and they're not actually generating that many tokens. And so the perfect fit for that is using a sparse model, meaning an MoE model. Um, so that was kind of one, one breakthrough, one, one breakthrough we made that substantially improved performance at longer context. The other being, um, a variant of speculative decoding that we, we kind of built out called speculative edits. These are two, I think, important pieces of what make it quite high quality, um, and very fast.

Lex Fridman 20:20 Okay. So MoE Mixture of Experts, the input is huge, the output is small.

Sualeh Asif 20:25 Yeah.

Lex Fridman 20:25 Okay. So, like, what, what, what else can you say about how to make... Is, like, is caching play a role in this particular-

Sualeh Asif 20:30 Oh, caching. Caching plays a huge role.

Arvid Lunnemark 20:32 Mm-hmm.

Sualeh Asif 20:33 Um, because you're dealing with this many input tokens, if every single keystroke that you're typing in a given line you had to rerun the model on all of those tokens passed in, you're just going to, one, significantly degrade latency, two, you're gonna kill your GPUs with load. So you need to, you, you need to design the actual prompts used for the model such that they're cache, caching aware, and then, yeah, you need to, you need to reuse the KV cache across requests just so that you're suspending less work, less compute.

Lex Fridman 21:04 Uh, again, what are the things that tab is supposed to be able to do kinda in the near term? Just to, like, sort of linger on that. Generate code, like fill empty space, also edit code across multiple lines?

Sualeh Asif 21:21 Mm-hmm.

Arvid Lunnemark 21:21 Yeah.

Lex Fridman 21:21 And then jump to different locations inside the-

Arvid Lunnemark 21:23 Mm-hmm.

Lex Fridman 21:23 ... same file?

Sualeh Asif 21:24 Yeah.

Lex Fridman 21:24 And then, like, laun-

Arvid Lunnemark 21:25 Hopefully jump to different files also. So if you make an edit in one file and maybe, maybe you have to go, maybe you have to go to another file to finish your thought, it should, it should go to the second file also.

Sualeh Asif 21:35 Yeah.

Arvid Lunnemark 21:36 And then in the-

Sualeh Asif 21:36 The full, the full generalization is, like, next, next action prediction. Like, sometimes you need to run a command in the terminal and it should be able to suggest the command based on the code that you wrote too. Um, or sometimes you actually need to... Like, it suggests something, but you, you... It's hard for you to know if it's correct because you nee- actually need some more information to learn. Like, you need to know the type to be able to verify that it's correct. And so maybe it should actually take you to a place that's like the definition of something and then take you back so that you have all the requisite knowledge to be able to accept the next completion.

Lex Fridman 22:13 So providing the human the knowledge.

Sualeh Asif 22:15 Yes.

Lex Fridman 22:17 Right.

Arvid Lunnemark 22:17 Mm-hmm. Yeah.

Lex Fridman 22:18 Can you integrate, like... I just, uh, gotten to know a guy named PrimeGen who I believe has an SS... You can or- order coffee via SSH?

Sualeh Asif 22:28 (laughs) Oh, yeah.

Arvid Lunnemark 22:29 Oh, we did that.

Sualeh Asif 22:30 We did that. Uh-

Lex Fridman 22:31 So can that... Also the model do that?

Sualeh Asif 22:32 (laughs)

Lex Fridman 22:33 Like, feed you and like-

Sualeh Asif 22:35 Yeah.

Lex Fridman 22:35 ... and provide you with caffeine? Okay, so that's the general framework then.

Sualeh Asif 22:39 Yeah, yeah. And the, the magic moment would be if it is... Programming is this weird discipline where, um, sometimes the next five minutes, not always, but sometimes the next five minutes, what you're gonna do is actually predictable from the stuff you've done recently. And so can you get to a world where that next five minutes either happens by you disengaging and it taking you through, or maybe a little bit more of just you seeing next step what it's gonna do and you're like, "Okay, that's good, that's good, that's good, that's good." And you can just sort of tab, tab, tab through these

Sualeh Asif 23:08 big changes.

Lex Fridman 23:09 As we're talking about this, I should mention, like, one of the really cool and noticeable things about Cursor is that there's this whole diff interface situation going on. So, like, the model suggests with, uh, with the red and the green of like, "Here's how we're gonna modify the code." And in the chat window you can apply and it shows you the diff and you can accept the diff. So maybe can you speak to whatever direction of that?

Arvid Lunnemark 23:32 We'll probably have like four or five different kinds of diffs. Uh, so we, we have optimized the diff for, for the auto-complete. So that has a different diff interface than, uh, than when you're reviewing larger blocks of code. And then ho- we're trying to optimize, uh, another diff thing for when you're doing multiple different files, uh, and, and sort of at a high level, the difference is for when you're doing auto-complete, it should be really, really fast to read. Uh, actually, it should be really fast to read in all situations. Um, but in auto-complete it's sort of you're, you're really... Like, your eye is focused in one area and you, you can't be in too many... You... The humans can't look in too many different places.

Lex Fridman 24:15 So you're talking about on the interface side, like the-

Arvid Lunnemark 24:17 On the interface side. So it currently has this box on the side. So we have-

Lex Fridman 24:21 Mm-hmm.

Arvid Lunnemark 24:21 ... the current box, and if it tries to delete code in some place and tries to add other code, it tries to show you a box on the side.

Sualeh Asif 24:28 You could maybe show it if we pull it up in Cursor.com.This is what we're talking about.

Aman Sanger 24:32 Go on.

Sualeh Asif 24:33 Exactly, yeah.

Aman Sanger 24:33 So that- that box, it was, like, three or four different attempts at trying to make this- this thing work, where first, the attempt was, like, these blue crossed-out lines. So before, it was a box on the side that used to show you the code to delete by showing you, like, uh, like Google Doc style, you would see, like, a line through it.

Sualeh Asif 24:55 Oh, yeah.

Aman Sanger 24:55 And then you would see the- the new code. That was super distracting. And then we tried many different var- uh, you know, there was, there was sort of deletions, there was trying to do red highlight. Then the next, uh, iteration of it, which is sort of funny, would y- you would hold the, on Mac, the option button, so it would, it would sort of highlight a region of code to show you that there might be something coming. Uh, so maybe in this an- example, like, the input and the value, uh, would get, would all get blue, and the blue would highlight that the AI had a suggestion for you. Uh, so instead of directly showing you the thing, it would show you that the AI, it would just hint that the AI had a suggestion, and if you really wanted to see it, you would hold the option button, and then you would see the new suggestion.

Sualeh Asif 25:42 Mm-hmm.

Aman Sanger 25:42 Then if you release the option button, you would then see original code.

Sualeh Asif 25:47 Mm-hmm. So that's, by the way, that's pretty nice, but you have to know to hold the option button. Yeah.

Aman Sanger 25:51 Uh, so it wa- it was-

Sualeh Asif 25:52 And by the way, I'm not a Mac user but I got it.

Aman Sanger 25:54 (laughs)

Sualeh Asif 25:55 Option, op- it's a button, I guess- (laughs) ... you people have.

Aman Sanger 25:59 It's the, you know, it's, again, it's just, it's just non-intuitive. I think that's the, that's the key thing.

Sualeh Asif 26:03 And- and there's a chance this- this is also not the final version of it.

Aman Sanger 26:07 I am personally very excited for, um, making a lot of improvements in this ar- area. Like, uh, we- we often talk about it as the verification problem, where, um, these diffs are great for small edits. Uh, for large edits, w- uh, even, or, like, when it's multiple files or something, it's, um, actually a little bit prohibitive to- to review these diffs. And, uh, uh, so there are, like, a couple of different ideas here. Like, one idea that we have is, okay, you know, like, parts of the diffs are important. They have a lot of information. And then parts of the diff, um, are just very low entropy. They're like exam- like, the same thing over and over again. And so maybe you can highlight the important pieces and then gray out the- the not-so-important pieces. Or maybe you can have a model that, uh, looks at the diff and- and sees, "Oh, there is a likely bug here. I will, like, mark this with a little red squiggly and say, like, 'You should probably, like, review this part of the diff.'" Um, and ideas in- in that vein, I think, are exciting.

Sualeh Asif 27:11 Yeah, that's a really fascinating space of, like, UX design engineering.

Aman Sanger 27:16 Yeah.

Sualeh Asif 27:16 So you're basically trying to guide the human programmer through all the things they need to read and nothing more.

Aman Sanger 27:24 Yeah.

Sualeh Asif 27:24 Like, optimally.

Aman Sanger 27:25 Yeah, and you want a- an intelligent model to do it. Like, currently, diffs algor- diff algorithms are, they're like alg- like, they're just like normal algorithms. Uh-

Sualeh Asif 27:36 (laughs)

Aman Sanger 27:36 ... there's no intelligence.

Sualeh Asif 27:37 Yeah.

Aman Sanger 27:37 Uh, there's, like, intelligence that went into designing the algorithm, but then th- there, there's no, like, you don't care if the, if it's about this thing or this thing, uh, and so you want a model to- to do this.

Sualeh Asif 27:47 So I think the- the- the general question is like, man, these models are going to get much smarter. As the models get much smarter, uh, the- the changes they will be able to propose are much bigger. So as the changes gets bigger and bigger and bigger, the humans have to do more and more and more verification work. It gets more and more and more hard, like, it's just, you need, you need to help them out. It- it sort of... I don't, I don't want to spend all my time reviewing code. (laughs) Uh, can you say a little more across multiple files diff? Yeah, I mean, so GitHub tries to solve this, right? With code review. When you're doing code review, you're reviewing multiple diffs across multiple files. But like Arvid had said earlier, I- I think you can do much better than code review. You know, code review kind of sucks. Like, you spend a lot of time trying to grok this code that's often quite unfamiliar to you, and it often, like, doesn't even actually catch that many bugs. And I think you can signific- significantly improve that review experience using language models. For example, using the kinds of tricks that Arvid had described of maybe, uh, pointing you towards the regions that actually matter. Um, I think, also, if the code is produced by these language models, uh, and it's not produced by someone else, like, the code review experience is designed for both the reviewer and the person that produced the code. In the case where the person that produced the code is a language model, you don't have to care that much about their experience, and you can design the entire thing around the reviewer such that the reviewer's job is as fun, as easy, as productive as possible. Um, and I think that- that feels like the issue with just kind of naively trying to make these things look like code review. I think you can be a lot more creative and- and push the boundary on what's possible.

Aman Sanger 29:42 And just one- one idea there is, I think ordering matters. Generally, when you review a PR, you- you have this list of files and you're reviewing them from top to bottom. But actually, like, you actually want to understand this part first because that came, like, logically first, and then you want to understand the next part, and, um, you don't want to have to figure out that yourself. You want a model to guide you through the thing.

Sualeh Asif 30:06 And is the step of curation going to be more and more in natural language, is the goal, versus with actuar-

Aman Sanger 30:11 Uh, I think sometimes. I don't think it's going to be the case that all of programming will be natural language. And the reason for that is, you know, if I'm pa- programming with Swalla, and Swalla's at the computer and the keyboard, uh, and s- sometimes if I'm, like, driving, I want to say to Swalla, "Hey," like, "implement this function," and that- that works. And then, sometimes, it's just so annoying to explain to Swalla what I want him to do, and so I actually take over the com- com- keyboard and I show him, I- I write, like, part of the example, and then...... it makes sense. And that's the easiest way to communicate. And so I think that's also the case for AI. Like, sometimes the easiest way to communicate with the AI will be to show an example, and then it goes and does the thing everywhere else. Or sometimes, if you're making a website, for example, the easiest way to show to the AI what you want is not to tell it what to do but, you know, drag things around or draw things. Um, and, yeah. And, and, like, maybe eventually we will get to, like, brain-machine interfaces or whatever and it can, like, understand what you're thinking. And so I think natural language will have a place. I think it will not, definitely not be the way most people program most of the time.

Lex Fridman 31:20 I'm really feeling the AGI with this editor.

Aman Sanger 31:23 (laughs)

Narrator 31:23 (laughs)

Lex Fridman 31:23 Uh, it feels like there's a lot of machine learning going on underneath. Tell me about some of the ML stuff that makes it all work.

Sualeh Asif 31:31 Well, Cursor really works via this ensemble of custom models that, that we've trained alongside, you know, the frontier models that are fantastic at the reasoning intense things. And so Cursor Tab, for example, is a, is a great example of where you can specialize this model to be even better than even frontier models if you look at evals on, on the, on the task we set it at. The other domain, which it's kind of surprising that it requires custom models, but, but it's kind of necessary and works quite well, is in Apply. Um, so I think these models are, like the frontier models are quite good at sketching out plans for code and generating, like, rough sketches of, like, the change. But actually creating diffs is quite hard, um, for frontier models, for in your training models. Um, like, you try to do this with Sonnet, with o1, any frontier model, and it, it really messes up stupid things like counting line numbers, um, especially in super, super large files. Um, and so what we've done to alleviate this is we let the model kind of sketch out this rough code block that indicates what the change will be, and we train a model to then apply that change to the file.

Lex Fridman 32:42 And we should say that Apply is, the model looks at your code. It gives you a really damn good suggestion of what new things to do, and the seemingly, for humans, trivial step of combining the two, you're saying is not so trivial.

Aman Sanger 32:59 Contrary to popular perception, it is not a deterministic algorithm.

Sualeh Asif 33:03 Yeah. I, I, I think, like, you see shallow copies of Apply, um, elsewhere, and it just breaks, like, most of the time because you think you can kind of try to do some deterministic matching, and then it fails, you know, at least 40% of the time. And that just results in a terrible product experience. Um, I think in general, this, this regime of you are going to get smarter and smarter models, and like, so one other thing that Apply lets you do is it lets you use fewer tokens with the most intelligent models. Uh, this is both expensive in terms of latency for generating all these tokens, um, and cost. So you can give this very, very rough sketch and then have your smaller models go and implement it because it's a much easier task to implement this very, very sketched out code. And I think that this, this regime will continue where you can use smarter and smarter models to do the planning, and then maybe the implementation details, uh, can be handled by the less intelligent ones. Perhaps you'll have, you know, maybe o1, maybe it'll be even more ca- capable models given an e- an even higher level plan that is kind of recursively, uh, applied by Sonnet and then the Apply model.

Aman Sanger 34:16 Maybe we should, we should talk about how to, how to make it fast.

Sualeh Asif 34:19 Yeah.

Aman Sanger 34:19 I feel like-

Sualeh Asif 34:20 Yeah.

Aman Sanger 34:20 ... fast is always an interesting

Narrator 34:21 (laughs)

Aman Sanger 34:21 ... detail. Fast is good.

Narrator 34:22 Yeah.

Lex Fridman 34:23 How do you make it fast?

Sualeh Asif 34:25 Yeah. So one big component of making it fast is speculative edits. So speculative edits are a variant of speculative decoding. And maybe it'd be helpful to briefly describe speculative decoding. Um, with speculative decoding, what you do is you, you can kind of take advantage of the fact that, you know, most of the time, and I'll, I'll add the caveat that it would be when you're memory bound in, in language model generation, um, if you process multiple tokens at once, um, it is faster than generating one token at a time. So this is like the same reason why if you look at tokens per second, uh, with prompt tokens versus generated tokens, it's much, much faster for prompt tokens. Um, so what we do is instead of using what speculative decoding normally does, which is using a really small model to predict these draft tokens that your larger model will then go in and, and verify, um, with code edits, we have a very strong prior of what the existing code will look like. And that prior is literally the same exact code. So what you can do is you can just feed chunks of the original code back into the, into the model, um, and then the model will just pretty much agree most of the time that, "Okay, I'm just gonna spit this code back out." And so you can process all of those lines in parallel. And you just do this with sufficiently many chunks, and then eventually you'll reach a point of disagreement where the model will now predict text that is different from the ground truth original code. It'll generate those tokens, and then we kind of will decide after enough tokens match, uh, the original code to restart speculating in chunks of code. What this actually ends up looking like is just a much faster version of normal editing code. So it's just like, it looks like a much faster version of the model rewriting all the code. So just, we, we can use the same exact interface that we use for, for diffs, but it will just stream down a lot faster.

Aman Sanger 36:21 And then, and then the advantage is that wir- wireless streaming, you can just also be reviewing, start reviewing the code-

Sualeh Asif 36:27 Exactly.

Aman Sanger 36:27 ... before, before it's done so there's no, no big loading screen. Uh, so may- maybe that, that is part of the, part of the advantage.

Lex Fridman 36:36 So the human can start reading-

Sualeh Asif 36:38 Yeah.

Lex Fridman 36:38 ... before the thing is done.

Aman Sanger 36:39 I think the interesting rift here is something like, like speculation is a fairly common idea nowadays. It's like not only in language models. I mean, there's, there's obviously speculation in CPUs, and there's p- like speculation for databases and-

Arvid Lunnemark 36:52 ... speculation all over the place.

Lex Fridman 36:54 Let me ask the sort of, the ridiculous question of, uh, which LLM is better at coding? GPT, Claude? Who wins in the context of programming? And I'm sure the answer is much more nuanced because it sounds like every single part of this involves a different model.

Sualeh Asif 37:12 Yeah. I think there, there's no cl- model that pre-dominates, uh, others, meaning it is better in all categories that we think matter. The categories being speed, um, ability to edit code, ability to process lots of code, long context, you know, a couple of other things and, and, kind of, coding capabilities. The one that I'd say right now is just kind of net best is Sonnet. I think this is a consensus opinion. R1's really interesting and it's really good at reasoning. So if you give it really hard, uh, programming interview style problems or lead code problems, it can do quite, quite well on them. Um, but it doesn't feel like it kind of understands your rough intent as well as Sonnet does. Like, if you look at a lot of the other frontier models, um, one qualm I have is it feels like they're not necessarily over fi-... I'm not saying they, they train in benchmarks. Um, but they perform really well in benchmarks relative to, kind of, everything that's kind of in the middle. So if you tried in all these benchmarks and things that are in the distribution of the benchmarks they're evaluated on, you know, they'll do really well, but when you push them a little bit outside of that, Sonnet's, I think, the one that, that kind of does best at, at, at kind of maintaining that same capability. Like you kind of have the same capability in the benchmark as when you try to instruct it to do anything with coding.

Lex Fridman 38:37 What... Another ridiculous question is the difference between the normal programming experience versus what benchmarks represent. Like where do benchmarks fall short, do you think, when we're evaluating these models?

Arvid Lunnemark 38:49 By the way, that's like a really, really hard... It's like, like critically important detail of like how, how different, like, benchmarks are versus, versus like real coding. With real coding, it's not interview style coding. It's you're, you're doing these... You know, humans are saying like half broken English sometimes and sometimes you're saying like, "Oh, do what I did before." Sometimes you're saying, uh, you know, "Go add this thing and then do this other thing for me and then make this UI element." And then, you know, it's, it's just a, like a lot of things are sort of context dependent. You really want to, like, understand the human and then do, do what the human wants as opposed to sort of this... Maybe the, the way to put it is sort of abstractly is, uh, the interview problems are very well specified. They f- lean a lot on specification while the human stuff is less specified.

Michael Truell 39:49 Yeah. No, I think that this, this benchmark question is both complicated by what, um, Sohail just mentioned and then also to, uh... What Aman was getting into is that even if you, like, you know, there's this problem of, like, the skew between what can you actually model in a benchmark versus, uh, real programming and that can be sometimes hard to encapsulate because it's like real programming's, like, very messy and sometimes things aren't super well specified what's correct or what isn't. But then, uh, it's also doubly hard because of this public benchmark problem. And that's both because public benchmarks are sometimes kind of hill climbed on, but then it's like really, really hard to also get the data from the public benchmarks out of the models. And so for instance, like one of the most popular, like, agent benchmarks, Sweet Bench, um, is really, really contaminated in the training data of, uh, these foundation models. And so if you ask these foundation models to do a Sweet Bench problem, but you actually don't give them the context of a code base, they can like hallucinate the right file pass, they can hallucinate the right function names. Um, and so the, the... It's, it's also just the public aspect of these things is tricky.

Sualeh Asif 40:53 Yeah. Like in that case it could be trained on the literal issues or pull requests themselves and, and maybe the labs will start to do a better job, um, or they've already done a, a good job at decontaminating those things. But they're not going to omit the actual training data of the repository itself. Like these are all, like some of the most popular Python repositories, like SymPy is one example. I don't think they're going to handicap their models on SymPy and all these popular Py- Python repositories in order to get, uh, true evaluation scores in these benchmarks.

Michael Truell 41:23 Yeah. I think that given the dearths in benchmarks, um, there have been like a few interesting crutches that, uh, places that build systems with these models or build these models actually use to get a sense of are they going in the right direction or not? And, uh, in a lot of places, uh, people will actually just have humans play with the things and give qualitative feedback on these. Um, like one or two of the foundation model companies, they, they have people who that's, that's a big part of their role and you know, internally we also, uh, you know, qualitatively assess these models and actually lean on that a lot in addition to like private evals that we have.

Arvid Lunnemark 41:56 It's like the VIBE. (laughs)

Lex Fridman 41:58 The VIBE, yeah. The VIBE.

Michael Truell 41:59 (laughs)

Arvid Lunnemark 41:59 It's like a VIBE.

Lex Fridman 41:59 (laughs) The VIBE benchmark, human benchmark.

Michael Truell 42:02 Yeah.

Lex Fridman 42:02 The humans, you pull in the humans to do a VIBE check.

Michael Truell 42:05 Yeah.

Lex Fridman 42:06 Okay. I mean that's, that's kind of what I do, like just, like reading online forums and Reddit and X just like... Well, I don't know how to properly load in people's opinions 'cause they'll say things like, "I feel like Claude or GPT has gotten dumber or something." They'll say, "I feel like..." And then I sometimes feel like that too, but I wonder if it's the model's problem or mine.

Sualeh Asif 42:34 Yeah. With Claude there is an interesting take I heard where I think AWS has different chips, um, and I, I suspect they have slightly different numerics than, uh, NVIDIA GPUs and-Someone speculated that Claude's degre- de- degraded performance had to do with maybe using the quantized version that existed on AWS Bedrock versus, uh, whatever was running on, on Anthropic's GPUs.

Lex Fridman 43:00 I interview a bunch of people that have conspiracy theories, so I'm glad-

Aman Sanger 43:03 (laughs)

Sualeh Asif 43:04 (laughs)

Lex Fridman 43:04 ... you spoke, you spoke to this conspiracy theory.

Aman Sanger 43:05 Well, it's-

Sualeh Asif 43:06 (laughs)

Aman Sanger 43:06 ... it's not, not, like, a conspiracy theory as much as-

Sualeh Asif 43:09 (laughs)

Lex Fridman 43:10 (laughs)

Aman Sanger 43:10 ... it is they're just, they're, like, they're, you know, humans, humans are humans and there's, there's these details.

Lex Fridman 43:14 Yes.

Aman Sanger 43:14 And, you know, you're doing, like, this queasy amount of flops. And, you know, chips are messy and, man, you can just have bugs. Like, bugs are... It's- it's hard to overstate how, how hard bugs are to avoid.

Lex Fridman 43:28 What's, uh, the role of, uh, a good prompt in all this? So you s- you, you mentioned that benchmarks have really, uh, structured, well-formulated prompts. Wh- what, what should a human being doing to maximize success? And what's the importance of what the humans... You wrote a blog post on, you called it, uh, Prompt Design.

Aman Sanger 43:50 Yeah. Uh, I think it depends on which model you're using, and all of them are slightly different and they respond differently to different prompts. But, um, I think the original GPT-4, uh, and the original sort of pre-dable models last, last year, they were quite sensitive to the prompts. And they also had a very small context window. And so we have all of these pieces of information around the code base that would maybe be relevant in the prompt. Like, you have the docs, you have the files that you add, you have the conversation history. And then there's a problem, like, how do you decide what you actually put in the prompt and when you have a, a limited space? And even for today's models, even when you have long context, filling out the entire context window means that it's slower. It means that sometimes the model actually gets confused, and some models get more confused than others. And we have this one system internally that we call Preempt, which helps us with that a little bit. Um, and I think it was built for the era before where we had 8,000, uh, token context windows. Uh, and it's a little bit similar to when you're making a website. You, you sort of, you, you want it to work on mobile, you want it to work on a desktop screen, and you have this, uh, dynamic information which you don't have, for example, if you're making, like, designing a print magazine. You have, like, you know exactly where you can put stuff. But when you have a website or when you have a prompt, you have these inputs, and then you need to format them to always work. Even if the input is really big, then you might have to cut something down. Uh, and, and, and so the idea was, okay, like, let's take some inspiration. What's the best way to design websites? Well, um, the thing that we really like is, is React and the declarative approach where you, um, you use JSX in, in, in JavaScript. Uh, and then you declare this is what I want, and I think this has higher priority or, like, this has higher Z index than something else. Um, and then you have this rendering engine. Uh, in web design it's, it's, like, Chrome, and, uh, in our case it's the Preempt Renderer, uh, which then fits everything onto the page. And as you declaratively decide what you want and then it figures out what you want. Um, and, and so we have found that to be, uh, quite helpful. And I think the role of it has, has sort of shifted over time, um, where it initially was to fit to these small context windows. Now it's really useful because, you know, it helps us with splitting up the data that goes into the prompt and the actual rendering of it. And so, um, it's easier to debug because you can change the rendering of the prompt and then try it on old prompts because you have the raw data that went into the prompt. And then you can see did my change actually improve it for, for, like, this entire eval set.

Lex Fridman 46:45 So do you literally prompt with JSX?

Aman Sanger 46:48 Yes.

Sualeh Asif 46:48 Yeah.

Aman Sanger 46:48 Yes. So it kind of looks like React. There are components. Like, we have one component that's a file component and it takes in, like, the cursor. Like, usually there's, like, one line where the cursor is in your file, and that's, like, probably the most important line because that's the one you're looking at. And so then you can give priorities. So, like, that line has the highest priority, and then you subtract one for every line that, uh, is farther away. And then eventually when it's rendered, it figures out how many lines can actually fit in its centers around that thing.

Lex Fridman 47:17 That's amazing.

Aman Sanger 47:17 Yeah.

Sualeh Asif 47:18 And you can do, like, other fancy things where if you have lots of code blocks from the entire code base, you could use, uh, retrieval, um, and things like embedding and re-ranking scores to add priorities for each of these components.

Lex Fridman 47:31 So should humans, when they ask questions, also use, try to use something like that? Like, would it be beneficial to write JSX in the, in the prompt? Or the whole idea is it should be loose and messy?

Aman Sanger 47:43 I, I think our goal is kind of that you should just, uh, do whatever is the most natural thing for you.

Sualeh Asif 47:49 Yeah.

Lex Fridman 47:49 Well-

Aman Sanger 47:49 And then we, our job is to figure out-

Sualeh Asif 47:52 Mm-hmm.

Lex Fridman 47:52 Mm-hmm.

Aman Sanger 47:52 ... how do we actually, like, retrieve the relevant things so that your thing actually makes sense.

Lex Fridman 47:56 Well, this is, uh, sort of the discussion I had with, uh, Arvin of Perplexity is, like, his whole idea is, like, you should let the person be as lazy-

Aman Sanger 48:05 Yes.

Sualeh Asif 48:05 Mm-hmm.

Lex Fridman 48:06 ... as he wants to be.

Sualeh Asif 48:06 Mm-hmm.

Aman Sanger 48:06 Mm-hmm. Mm-hmm.

Lex Fridman 48:06 But, like, yeah, that's a beautiful thing, but I feel like you're allowed to ask more of programmers, right?

Sualeh Asif 48:14 Yes. Yes.

Lex Fridman 48:14 So, like, if you say, "Just do what you want," I mean, humans are lazy. There's a kind of tension between-

Aman Sanger 48:20 Yes.

Lex Fridman 48:20 ... just being lazy-

Aman Sanger 48:21 Yeah.

Lex Fridman 48:21 ... versus, like, provide more as, uh, be prompted almost like the system pressuring you or inspiring you to be articulate.

Sualeh Asif 48:32 Uh-huh.

Aman Sanger 48:32 Yeah.

Lex Fridman 48:32 Not in terms of the grammar of the sentences, but in terms of the depth of thoughts that you convey inside the, uh, the prompts.

Sualeh Asif 48:39 I think even as a system gets closer to some level of perfection, often when you ask the model for something, you just are not, uh, not enough intent is conveyed to know what to do.

Aman Sanger 48:51 ... and there are, like, a few ways to resolve that intent. One is the simple thing of having the model just ask you, "I'm not sure how to do these parts based on your query. Could you clarify that?" Um... I think the other could be maybe if you... There are five or six possible generations, given the uncertainty present in your query so far. Why don't we just actually show you all of those and let you pick them?

Lex Fridman 49:19 H- how hard is to... for the model to choose to speak... talk back? Sort of versus... (laughs) generative. That's, uh, that's hard to sort of, like-

Aman Sanger 49:27 Yeah.

Lex Fridman 49:27 ... how to deal with the uncertainty.

Aman Sanger 49:29 Mm-hmm.

Lex Fridman 49:30 Do I, do I choose to ask for more information to, uh, reduce the ambiguity?

Arvid Lunnemark 49:36 So, I mean, one of the things we, we do is, um... It's like a recent addition, is try to suggest files that you can add. So... and while you're typing, uh, one can guess what the uncertainty is and maybe suggest that, like, you know, maybe, maybe you're writing your API and, uh, we can guess using the commits, uh, that you've made previously in the same file that the client and the server is super useful. And, uh, there's, like, a hard technical problem of how do you resolve it across all commits, which files are the most important given your current prompt. And we're still sort of... Uh, initial version is rolled out, and I'm sure we can make it much more accurate. Uh, it's, it's, it's very experimental. But then the idea is we show you, like, do you just want to add this file, this file, this file also, to tell, you know, the model to edit those files for you. Uh, because if, if you're... maybe you're making the API, like, you should also edit the client and the server that is using the API and the other one resolving the API, and so that would be kind of cool as both there's the phase where you're writing the prompt and there's... before you even click enter, maybe we could help resolve some of the uncertainty.

Lex Fridman 50:54 To what degree do you use, uh, agentic approaches? How useful are agents?

Aman Sanger 50:59 We think agents are really, really cool.

Lex Fridman 51:02 (laughs)

Aman Sanger 51:02 Like, I, I, I think-

Lex Fridman 51:04 Okay.

Aman Sanger 51:04 ... agents is like, um... It's like it resembles sort of like a human. It's, it's sort of like the things... Like, you can kind of feel that it... like, you're getting closer to AGI because you see a demo where, um, it acts as, as a human would. And, and it's really, really cool. I think, um, agents are not yet super useful for many things. They... I think we're, we're getting close to where they will actually be useful. And, and so I think, uh, there are certain types of tasks where having an agent would be really nice. Like, I would love to have an agent. For example, if... Like, we have a bug where you, you sometimes can't Command+C and Command+V, uh, inside our chat input box, and that's a task that's super well specified. I just want to say, like, in two sentences, "This does not work. Please fix it." And then I would love to have an agent that just goes off, does it, and then, uh, a day later, I, I come back and I review the, the thing.

Lex Fridman 52:02 You mean, it goes, finds the right file and-

Aman Sanger 52:05 Yeah. It finds the right files. It, like, tries to reproduce the bug. It, like, fixes the bug, and then it verifies that it's correct. And this is... could be a process that takes a long time. Um, and so I think I would love to have that. Uh, and then I think a lot of programming... Like, there is often this belief that agents will take o- over all of programming. Um, I don't think we think that that's the case because a lot of programming, a lot of the value is in iterating or you don't actually want to specify something upfront because you don't really know what you want until you've seen an initial version and then you want to iterate on that, and then you provide more information. And so for a lot of programming, I think what you actually want is a system that's instant that gives you an initial version instantly back, and then you can iterate super, super quickly.

Lex Fridman 52:52 Uh, what about something like that we're thinking about, Replit Agent, that does also, like, setting up the development environment, installing software packages, configuring everything, configuring the databases and actually deploying the app?

Aman Sanger 53:05 Yeah.

Lex Fridman 53:05 Is that also, in, in the set of th- things you dream about?

Aman Sanger 53:10 I think so. I think that would be really cool. I... For, for certain types of programming, uh, it, it would be really cool.

Lex Fridman 53:15 Is that within scope of Cursor?

Aman Sanger 53:17 Yeah. We aren't actively working on it right now, um, but it's definitely, like, we want to make the programmer's life easier and more fun. And some things are just really tedious and you need to go through a bunch of steps and you want to delegate that to an agent. Um, and then some things, you can actually have an agent in the background while you're working. Like, let's say, you have a PR that's both backend and frontend and you're working on the frontend and then you can have a background agent that does some work and figure out kind of what you're doing and then when you get to the backend part of your PR, then you have some, like, initial piece of code that you can then iterate on. Um, and, and so that, that would also be really cool.

Lex Fridman 53:58 One of the things we already talked about is speed. But I (laughs) I wonder if we can just, uh, linger on that some more and the, the various places that, uh... the technical details involved in making this thing really fast. So every single aspect of, uh, Cursor, most aspects of Cursor feel really fast. Like I mentioned, the apply is probably the slowest thing and for me personally.

Aman Sanger 54:19 Yeah.

Lex Fridman 54:19 I'm sorry. The pain on Harvey's (laughs) face as I say that.

Aman Sanger 54:21 (laughs) I know. It's a, it's a pain. It's a pain that we're feeling and-

Lex Fridman 54:24 Well...

Aman Sanger 54:24 ... we're working on fixing it. (laughs)

Lex Fridman 54:27 Uh... (laughs) Yeah. I mean, it says something that something that feels... I don't know what it is, like one second or two seconds, that feels slow, that means... that's actually, uh, shows that everything else is just really, really fast. Um, so is there some technical details about how to make some of these models, how to make the chat fast, how to make the diffs fast? Is there something that just jumps to mind?

Aman Sanger 54:49 Yeah. I mean, so we can go over a lot of the strategies that we use. One interesting thing is cache warming. Um, and so what you can do is if... as the user is typing, you can have...

Sualeh Asif 55:01 Yeah. You're, you're probably going to use, uh, some piece of context, and you can know that before the user's done typing. So, you know, a- as we discussed before, reusing the KV cache results in lower latency, lower cost, uh, cross-requests. So as the user starts typing, you can immediately warm the cache with, like, let's say, the current file contents. And then when they press Enter, uh, there's very few tokens it actually has to pre-fill and compute before starting the generation. This will significantly lower TTFT.

Lex Fridman 55:30 Can you explain how KV cache works?

Sualeh Asif 55:33 Yeah. So the way transformers work, um-

Lex Fridman 55:37 (laughs)

Arvid Lunnemark 55:37 (laughs)

Sualeh Asif 55:38 ... uh-

Lex Fridman 55:38 I like it.

Sualeh Asif 55:38 ... I mean, what... (laughs)

Arvid Lunnemark 55:40 (laughs)

Sualeh Asif 55:40 Like one, one of the mechanisms that allow transformers to not just independently... Like the mechanism that allows transformers to not just independently look at each token but see previous tokens are the keys and values, it's attention. And generally, the way attention works is you have at your current token some query, and then you've all the keys and values of all your previous tokens, which are some kind of representation that the model stores internally of all the previous tokens in the prompt. And like by default, when you're doing a chat, the model has to, for every single token, do this forward pass through the entire, uh, model. That's a lot of matrix multiplies that happen, and that is really, really slow. Instead, if you have already done that, and you store the keys and values, and you keep that in the GPU, then when I'm... Let's say I have stored it for the last n tokens. If I now wanna compute the, the output token for the n+1 token, I don't need to pass those first n tokens through the entire model because I already have all those keys and values. And so you just need to do the forward pass through that last token, and then when you're doing attention, uh, you're reusing those keys and values that have been computed, which is the only kind of sequential part, um, or sequentially dependent part of the transformer.

Lex Fridman 57:00 Is there, like, higher level caching of, like, caching of the prompts or that kind of stuff that could help?

Sualeh Asif 57:06 I see. Yeah. That, that... There, there's other types of caching you can kind of do. Um, one interesting thing that you can do for CursorTab is you can basically predict ahead as if the user would've accepted the suggestion and then trigger another, uh, request. And so then you cache. You've done the speculative. It's, it's a mix of speculation and caching, right? Because you're speculating what would happen if they accepted it, and then you have this value that is cached, this, this, uh, suggestion. And then when they press Tab, the next one would be waiting for them immediately. It's a, it's a kind of clever heuristic/trick, uh, that uses a higher level caching and, and can give, uh, the... It, it, it feels fast despite there not actually being any changes in the, in the model.

Arvid Lunnemark 57:54 And, and if you can make the KV cache smaller, one of the advantages you get is, like, maybe, maybe you can speculate even more. Maybe you can guess here's the 10 things that, you know, could be useful, uh, like, uh, like predict the next 10. And, and then, like, it's possible the user hits the, the one of the 10. It's, like, much higher chance than the user hits, like, the exact one that you show them. Uh, may- maybe they type another character in, and we sort of hits, hit something else in the cache.

Sualeh Asif 58:19 Yeah.

Arvid Lunnemark 58:19 So there's, there's all these tricks where, um, uh, the, the general phenomena here is, uh, I think it's, it's also super useful for our RL is, you know, may- maybe a single sample from the model isn't very good, but if you predict, mm, like, 10 different things, uh, turns out that one of the 10, uh, that's right is, the probability is much higher. So there's these passed K curves. And, you know, part of RL try... Like, what, what RL does is, you know, you can, you can exploit this passed K phenomena to, to make many different predictions. And, and, uh, one, one way to think about this, the model sort of knows internally, has, like, has some uncertainty over, like, which of the K things is correct or, like, which of the K things does the human want. So when we RL our, uh, you know, CursorTab model, one of the things we're doing is we're predicting w- which, like, which of the hundred different suggestions the model produces is more amenable for humans. Like, wha- which of them do humans more like than other things? Uh, maybe, maybe, like, there's something where the model can predict very far ahead versus, like, a little bit and maybe somewhere in the middle. And, and you just... And then you can give a reward to the things that humans would like more and, and sort of punish the things that it wouldn't like and sort of then train the model to output the suggestions that humans would like more. You, you have these, like, RL loops that are very useful that exploit these passed K curves. Um, Emad maybe can, can go into even more detail.

Sualeh Asif 59:48 Yeah. It's a little... It is a little different than speed. Um, but I mean, like, technically, you tie it back in because you can get away with the smaller model if you RL your smaller model, and it gets the same performance as, as the bigger one. Um, that's like... A- a- and so while I was mentioning stuff about KV, uh, about reducing the size of your KV cache, there are, there are other techniques there as well that are really helpful for speed. Um, so kind of back in the day, like, all the way two years ago, uh, people mainly used multi-head attention. Um, and I think there's been a migration towards more, uh, efficient attention schemes, like group query, um, or multi-query attention. And this is really helpful for then, uh, with larger batch sizes being able to generate the tokens much faster. The interesting thing here is, um, this now has no effect on that, uh, time to first token pre-fill speed. Uh, the thing this matters for is, uh, now generating tokens. And, and why is that? Because when you're generating tokens, instead of, uh, being bottlenecked by doing these super parallelizable matrix multiplies across all your tokens, you're bottlenecked by how quickly, uh, it's for long context, um, with large batch sizes, by how quickly you can read those cache keys and values. Um-Um, and so then how, uh, that's memory bandwidth and how can we make this faster? We can try to compress the size of these keys and values. So multi-query attention is the most aggressive of these. Um, where normally with multi-head attention, you have some number of, quote-unquote, "Attention heads," um, and some number of, kind of, query, query heads. Uh, multi-query just preserves the query heads, gets rid of all the key value heads. Um, so there's only one kind of key value head and there's all the remaining, uh, query heads. With group query, um, you instead, you know, preserve all the query heads and then your keys and values are kind of in, uh, th- there are fewer heads for the keys and values, but it's, you're not reducing it to just one. Um, but anyways, like, the whole point here is you're just reducing the size of your KV cache.

Aman Sanger 1:02:00 And then there is MLA.

Sualeh Asif 1:02:02 Yeah, multi-latent. Um, that's a little more complicated. And the way that this works is it kind of turns the entirety of your keys and values across all your heads into this kind of one latent vector that is then kind of expanded in for its time.

Aman Sanger 1:02:19 But MLA is from this company, uh, called DeepSeq. Um, it's, it's quite an interesting algorithm. Uh, maybe the key idea is sort of, uh, in both MQA, uh, and in other places, what you're doing is sort of reducing the, uh, num- like the number of KV heads. And the advantage you get from that is, is, you know, there's less of them, but, uh, maybe the theory is that you actually want a lot of different, uh ... Like you want each of the, the keys and values to actually be different. So one way to reduce the size is you keep, uh, one big shared vector for all the keys and values, and then you have smaller vectors for every single token. So that when you mul- you can, you can store the only the smaller thing as some sort of like low rank reduction, and the low rank reduction would that ... And at the end of the time when you, when you eventually want to compute the final thing, uh, remember that like your memory bound, which means that like you still have some, some compute left that you can use for these things. And so if you can expand the, um, the latent vector back out and, and somehow like this is far more efficient because just like you're reducing, like for example, may- maybe, you know, like reducing by 32 or something, like the size of the vector that you're keeping.

Sualeh Asif 1:03:38 Yeah. There's perhaps some richness in having a separate, uh, set of keys and values and query that kind of pairwise match up versus compressing that all into one in that interaction at least.

Lex Fridman 1:03:51 Okay. And all of that is dealing with, um, being memory bound.

Sualeh Asif 1:03:55 Yeah.

Lex Fridman 1:03:56 And what ... I mean, ultimately, how does that map to the user experience, trying to get the thing?

Sualeh Asif 1:04:02 Yeah. The t- the two things that it maps to is you can now make your cache a lot larger because you've less space allocated for the KV cache. You can maybe cache a lot more aggressively and a lot more things so you get more cache hits, which are helpful for reducing the time to first token for the reasons that were kind of described earlier. And then the second being when you start doing inference with more and more requests and larger and larger batch sizes, you don't see much of a slowdown in, as it's generating the tokens, the speed of that.

Aman Sanger 1:04:31 Well, it also allows you to make your prompt bigger for certain things.

Sualeh Asif 1:04:34 Yeah, yeah. So, uh, like the basic, the size of your KV cache is, uh, both the size of all your prompts multiplied by the number of prompts being pro- processed in parallel. So you could increase either of those dimensions, right? The batch size or the size of your prompts without degrading the latency of generating

Sualeh Asif 1:04:51 tokens.

Lex Fridman 1:04:51 Arvid, you wrote a blog post, Shadow Workspace.

Aman Sanger 1:04:54 Yes.

Lex Fridman 1:04:54 Iterating on code in the background.

Aman Sanger 1:04:56 Yeah.

Lex Fridman 1:04:56 So what's going on behind the scenes?

Aman Sanger 1:04:58 Uh, so to be clear, we want there to be a lot of stuff ha- stuff happening in the background, and we're experimenting with a lot of things. Uh, right now, uh, we don't have much of that happening other than like the, the cache warming or like, you know, uh, figuring out the right context too that goes into your command queue prompts, for example. Uh, but the idea is if you can actually spend computation in the background, then you can help, um, help the user maybe like add a slightly longer time horizon than just predicting the next few lines that you're gonna make. But actually like in the next 10 minutes, what are you going to make? And by doing it in the background, you can spend more com- computation doing that. And so the idea of the shadow workspace that, that we implemented, and we use it internally for like experiments, um, is that to actually get advantage of doing stuff in the background, you want some kind of feedback signal to gi- give back to the model. Because otherwise, like you can get higher performance by just letting the model think for longer. Um, and, and so like 01 is a good example of that. But another way you can improve performance is by letting the model iterate and get feedback. And, and, and so one very important piece of feedback when you're a programmer is, um, the language server, which is, uh, this thing, it exists, uh, for most different languages and there's like a separate language server per language. And it can tell you, you know, you're using the wrong type here, and then gives you an error. Or it can allow you to go to definition and sort of understands the structure of, of your code. So language servers are extensions developed by, like there is a TypeScript language server developed by the TypeScript people, a Rust language server developed by the Rust people, and then they all int- interface over the language server protocol to a VS code. So that VS code doesn't need to have all of the different languages built into VS code, but rather, uh, you can use the existing compiler infras- infrastructure.

Lex Fridman 1:06:50 For linting purposes? What, what-

Aman Sanger 1:06:52 It's for, it's for linting, it's for going to definition, uh, and for like seeing the, the right types that you're using. Um-

Lex Fridman 1:06:59 So it's doing like type checking also?

Aman Sanger 1:07:01 Yes. Type checking and, and going to references. Um, and that's like when you're working in a big project, you, you kind of need that. If you, if you don't have that, it's like really hard to, to code in a big project.

Lex Fridman 1:07:12 Can you say again how that's being used inside Cursor, the-... the language s-server protocol communication thing?

Aman Sanger 1:07:20 Yes. So it's being used in Cursor to show to the programmer, just like in VS code. But then the idea is you want to show that same information to the models, the IO models. Um, and you want to do that in a way that doesn't affect the user, because you want to do it in background. And so, uh, the idea behind the shadow workspace was, okay, like, one way we can do this is, um, we spawn a separate window of Cursor that's hidden, and so you, you can set this flag in Electron as hidden. There is a window, but you don't actually see it. And inside of this window, uh, the AI agents can modify code however they want, um, as long as they don't save it, because it's still the same folder. Um, and then can get feedback from, from the linters and go to definition and, and iterate on their code.

Lex Fridman 1:08:04 So, like, literally run everything in the background. Like, as if... Right.

Aman Sanger 1:08:08 Yeah.

Lex Fridman 1:08:09 May- maybe even run the code? Is that-

Aman Sanger 1:08:10 Uh, so that's the, uh, eventual version.

Lex Fridman 1:08:13 Okay.

Aman Sanger 1:08:13 That's what you want. And a lot of the blog post is actually about, how do you make that happen? Because it's a little bit tricky. You want it to be on the user's machine so that it exactly, uh, mirrors the user's environment. And then on Linux, you can do this cool thing where you can actually mirror the file system and have the AI make changes to the files and, and it thinks that it's operating on the file level, but actually that's stored in, in memory and you, you can, uh, uh, create this kernel extension to, to make it work. Um, whereas on Mac and Windows, it's a little bit m- more difficult, uh, and, and, uh, but it's, it's a fun technical problem, so that's why - Yeah.

Lex Fridman 1:08:57 ... right at the end.

Sualeh Asif 1:08:57 One, one maybe hacky but interesting idea that I like is holding a lock on saving. And so basically, you can then have the language model kind of hold the lock on, on saving to disc, and then instead of you operating in the ground truth version of the files, uh, that are saved to disc, you, you actually are operating on what was the shadow workspace before and these unsaved things that only exist in memory that you still get linter errors for and you can code in. And then when you try to maybe run code, it's just, like, there's a small warning that, that there's a lock, and then you kind of will take back the lock from the language server if you're trying to do things concurrently. Or from the, the shadow workspace if you're trying to do things concurrently.

Lex Fridman 1:09:31 That's such an exciting future, by the way. It's a bit of a tangent, but, like, to allow a model to change files. It's scary for people, but, like, it's really cool. To be able to just, like, let the agent do a s- a set of tasks and you come back the next day and kind of observe, like it's a colleague or something like that. Yeah.

Sualeh Asif 1:09:52 Yeah. And I think there may be different versions of, like, runability, where for the simple things, where you're doing things in the span of a few minutes on behalf of the user as they're programming, it makes sense to make something work locally on their machine. I think for the more aggressive things, where you're making larger changes that take longer periods of time, you'll probably want to do this in some sandbox remote environment. And that's another incredibly tricky problem of how do you exactly reproduce or mostly reproduce to the point of it being effectively equivalent for running code the user's environment with this remote, r- remote sandbox?

Aman Sanger 1:10:27 I'm curious what kind of agents you want for, for coding.

Lex Fridman 1:10:30 (laughs)

Sualeh Asif 1:10:31 Oh, for-

Aman Sanger 1:10:31 Do you, do you want them to find bugs? Do you want them to, like, implement new features? Like, w- what, what agents do you want?

Lex Fridman 1:10:36 So by the way, when I think about agents, I don't think just about coding. Uh, I think... So for t- pr- this, th- this particular podcast, there's video editing, and a lot of... If you look in Adobe, a lot of... There's code behind. Uh, it's very poorly documented code. But you could interact with Premiere, for example, using code, and, uh, basically all the uploading, everything I do on YouTube, everything, as you could probably imagine, I do all of that through code. And s- and including translation and overdubbing, all of this. So I envision th- all of those kinds of tasks, so automating many of the tasks that don't have to do directly with the editing. So th- that. Okay. That's what I was thinking about. But in terms of coding, I would be s- fundamentally thinking about bug finding. Like, many levels of kind of b- bug finding, and also bug finding, like, logical bugs... Not logical, like spiritual bugs or something.

Aman Sanger 1:11:33 (laughs)

Sualeh Asif 1:11:33 (laughs)

Lex Fridman 1:11:34 Ones, like, sort of big directions of implementation, that kind of stuff.

Aman Sanger 1:11:38 What's your opinion on bug finding?

Sualeh Asif 1:11:40 Yeah. I mean, it's really interesting that these models are so bad at bug finding, uh, when just naively prompted to find a bug. They're incredibly poorly calibrated.

Aman Sanger 1:11:51 Even the, the smartest models.

Lex Fridman 1:11:52 Mm-hmm.

Sualeh Asif 1:11:52 Exactly. Even o- even o1.

Lex Fridman 1:11:54 How do you explain that? Is there a good intuition?

Sualeh Asif 1:11:58 I think these models are a really strong reflection of the pre-training distribution. And, you know, I do think they, they generalize as the loss gets lower and lower, but I don't think the, the, the loss in the scale is quite... Or the loss is low enough such that they're, like, really fully generalizing on code. Like, the things that we use these things for, uh, the frontier models, that, that they're quite good at are really code generation and question answering. And these things exist in massive quantities in pre-training with all of the code on GitHub on the scale of many, many trillions of tokens and questions and answers on things like Stack Overflow and maybe GitHub Issues. And so when you try to push one of these things that really don't exist, uh, very much online, like, for example, the CursorTab objective of predicting the next edit given the edits done so far, uh, the brittleness kind of shows. And then bug detection is another great example where there aren't really that many examples of, like, actually detecting real bugs and then proposing fixes. Um, and the models just kind of, like, s- really struggle at it. But I think i- i- it's a question of transferring the model. Like, in the same way that you get this fantastic transfer, um, from pretrained models, uh, just on code in general to the CursorTab objective, uh, you'll see a very, very similar thing with generalized models that are really good at code to bug detection. It just takes, like, a little bit of kind of nudging in that direction.

Aman Sanger 1:13:25 Like, to be clear, I think they sort of understand code really well. Like, while they're being pre-trained, like, the representation that's being built up, like, almost certainly, like, you know, somewhere in the stream there's...... the model knows that maybe there's- there's some sket- something sketchy going on, right? It sort of has some sketchiness. But actually eliciting this, the sketchiness to, uh... Like actually, like, part- part of it is that humans are really calibrated on which bugs are really important.

Lex Fridman 1:13:55 Mm-hmm. Yeah. Yeah.

Aman Sanger 1:13:55 It's not just actually, it's not just actually saying like, "There's something sketchy." It's like, it's just, it's just a sketchy trivial? It's this sketchy, like, "You're gonna take the server down."

Lex Fridman 1:14:03 Yeah. Yeah.

Aman Sanger 1:14:03 It's like, like, part of it is maybe the cultural knowledge of, uh... Like, why is the staff engineer a staff engineer? A staff engineer is, is good because they know that three years ago, like, someone wrote a really s- you know, sketchy piece of code that took- took the server down. And as opposed to, like-

Lex Fridman 1:14:19 (laughs)

Aman Sanger 1:14:20 ... as opposed to maybe there's like, you know, you just, this thing is like an experiment, so like a few bugs are fine. Like, you're just trying to experiment and get the feel of the thing. And so if the model gets really annoying when you're writing an experiment, that's really bad. But if you're writing something for s- super production, you're like writing a database, right? You're, you're writing code in Postgres or Linux or whatever, like your line is 12 volts. You're, you're, it's- it's sort of unacceptable to have even an edge case. And just, just having the calibration of like, how paranoid is the user? Like...

Lex Fridman 1:14:51 Yeah. But even then, like if you're putting in a maximum paranoia, it still just like doesn't quite get it.

Aman Sanger 1:14:57 Yeah,

Aman Sanger 1:14:58 yeah, yeah.

Lex Fridman 1:14:58 I mean, uh, but this is hard for humans too, to understand what- which line of code is important and which is not. Like, you, I think one of your principles on the website says if, if, if a code can do a lot of damage, one should add a comment that say, "This- this- this line of code is, is dangerous."

Aman Sanger 1:15:17 And, uh, all caps.

Lex Fridman 1:15:18 All caps. (laughs)

Aman Sanger 1:15:18 And repeat it 10 times.

Lex Fridman 1:15:20 Read it 10... No, you say like, "For every single line-"

Aman Sanger 1:15:24 Yes.

Lex Fridman 1:15:24 ... of code inside the function, you have to a- and that- that's quite profound. That says something about human beings, because the, uh, the engineers move on. Even the same person might just forget how it can sink the Titanic, a single function. Like you don't-

Aman Sanger 1:15:38 Yeah, it-

Lex Fridman 1:15:39 You might not intuit that quite clearly by looking at the single piece of code.

Aman Sanger 1:15:43 Yeah, and I think that, that one is also, uh, partially also for e- today's AI models, where, uh, if you actually write "dangerous, dangerous, dangerous" in every single line, like, uh, the models will pay more attention to that and will-

Lex Fridman 1:15:58 (laughs)

Aman Sanger 1:15:58 ... be more likely to find bugs in that region.

Lex Fridman 1:16:00 That's, uh, actually just straight up a really good practice of labeling code, of how much damage this can do.

Aman Sanger 1:16:08 Yeah, I mean, it's controversial.

Lex Fridman 1:16:09 (laughs)

Aman Sanger 1:16:09 Like, it, it's, some people think it's ugly. Uh, Swalek does not like it. Well, I- I actually think it's, it's, uh, like-

Lex Fridman 1:16:14 (laughs)

Aman Sanger 1:16:14 In fact, I- I actually think this is one of the things I learned from Arvid is, you know, like, I, uh, sort of aesthetically, I don't like it.

Lex Fridman 1:16:22 Mm-hmm.

Aman Sanger 1:16:22 But I think there's certainly something where like, it's- it's useful for the models and- and humans just forget a lot, and it's really easy to make a small mistake and cause like, bring down the... You know, like, just bring down the server and like, y- like, uh, of course we t- we like test a lot and whatever, but there- there's always these things that you have to be very careful.

Lex Fridman 1:16:42 Yeah, like with just normal doc strings, I think people will often just skim it when making a change and think, "Oh, this, I- I know how to do this." Um, and you kind of really need to point it out to them so that that doesn't slip through. Yeah, you have to be reminded that you could do a lot of damage. That's like, we don't really think about that. Like...

Aman Sanger 1:17:01 Yeah.

Lex Fridman 1:17:02 You think about, okay, m- how do I figure out how this works so I can improve it? You don't think about the other direction that it could

Michael Truell 1:17:08 Yeah.

Lex Fridman 1:17:08 ... le- so...

Aman Sanger 1:17:08 Until, until we have formal verification for everything. Then you can do whatever you want and you, you know for certain that you have not introduced a bug if the proof pass.

Michael Truell 1:17:18 But concretely, what do you think that future would look like?

Aman Sanger 1:17:22 I think, um, people will just not write tests anymore, and, um, the model will suggest, like you write a function, the model will suggest a spec, and you review the spec. And, uh, in the meantime, a smart reasoning model