AI prompt engineering: A deep dive

1. 0:00 – 2:05

   Introduction

   1. AAAlex Albert0:00

      Basically, this entire roundtable session here is just gonna be focused mainly on prompt engineering. Um, variety of perspectives at this table around prompting from a research side, from a consumer side, and from, like, an enterprise side. And I wanna just get the whole wide range of opinions because there's a lot of them, and just kind of open it up to discussion and, uh, explore w-what prompt engineering really is and what it's all about, and, uh, yeah, we'll just take it from there. So maybe we can go around the horn with intros. Um, I can kick it off. I'm Alex. I lead developer relations here at Anthropic. Uh, before that, I was kind of technically a prompt engineer at Anthropic. Uh, worked on our prompt engineering team, um, and did a variety of roles spanning from, like, a solutions architect type of, um, thing to, uh, working on the research side. So with that, maybe I can hand over to David.

   2. DHDavid Hershey0:53

      Heck yeah. Uh, my name's David Hershey. I, uh, work with customers mostly at Anthropic on a bunch of stuff. Uh, technical, I, I help people with fine-tuning, but also just, like, a lot of the generic things that make it hard to adopt language models, so prompting and, uh, and just, like, how to build systems with language models, but spend most of my time working with customers.

   3. AAAmanda Askell1:14

      Cool. Uh, I'm Amanda Askell. I lead one of the fine-tuning teams at Anthropic, uh, where I guess I try to make Claude be honest and kind.

   4. AAAlex Albert1:23

      [chuckles]

   5. AAAmanda Askell1:25

      Uh, yeah.

   6. ZWZack Witten1:26

      Uh, my name's Zack Witten. Uh, I'm a prompt engineer at Anthropic. Um, Alex and I always argue about who the first one was.

   7. AAAlex Albert1:32

      [chuckles]

   8. ZWZack Witten1:32

      He says it's him, I say it's me.

   9. AAAlex Albert1:34

      Con-contested.

   10. ZWZack Witten1:34

       Yeah.

   11. AAAlex Albert1:35

       [chuckles]

   12. ZWZack Witten1:35

       Um, I used to work w- a lot with individual customers, kind of the same way, uh, David does now. And then as we, uh, brought more solutions architects, uh, to the, to the team, I started working on, uh, things that are meant to, uh, raise the overall levels of, like, ambient prompting in society, I guess, like the, the prompt generator and, like, the, uh, various, like, educational materials that, that people use.

   13. AAAlex Albert1:59

       [lips smack] Nice. Cool. Well, thanks, guys, for all coming here. Um, I'm gonna start with, like, a very

2. 2:05 – 6:34

   Defining prompt engineering

   1. AAAlex Albert2:05

      broad question just so we have a, a frame going into the rest of our conversations here. W-what is prompt engineering? Why is it, why is it engineering? Um, what's prompt really?

   2. AAAmanda Askell2:15

      [chuckles]

   3. AAAlex Albert2:16

      If anyone wants to kick that off, um, give your own perspective on it, feel free to take the rein here.

   4. DHDavid Hershey2:21

      I feel like we have a prompt engineer.

   5. ZWZack Witten2:23

      Yeah, Zack's our prompt engineer.

   6. AAAlex Albert2:24

      Zack's the prompt engineer.

   7. DHDavid Hershey2:24

      It's his job. I mean, we all prompt, but he-

   8. ZWZack Witten2:25

      We're all prompt engineers in our own form. Uh, Zack-

   9. DHDavid Hershey2:27

      But one of us has a job.

   10. ZWZack Witten2:29

       Yeah, Zack, maybe since it's in your title.

   11. AAAmanda Askell2:30

       [laughs] One of us has a job.

   12. ZWZack Witten2:31

       You can give your perspective.

   13. AAAmanda Askell2:32

       And the, the other three don't have jobs.

   14. AAAlex Albert2:33

       Yeah. [laughs]

   15. ZWZack Witten2:34

       [laughs] Um, I guess I feel like prompt engineering is trying to get the model to do things, trying to bring the most out of the model, trying to, uh, work with the model to get things done that you wouldn't have been able to do otherwise. Um, so a lot of it is just, like, clear communicating. I think at, at heart, like, talking to a model is a lot like talking to a person and getting in there and, like, understanding the, like, the psychology of, of the model, which, like, Amanda is the world's most expert person in the world. Uh...

   16. AAAlex Albert3:08

       So well, I'm, I'm gonna keep going on you. Why is, why is engineering in the name? Like, where-

   17. ZWZack Witten3:14

       Yeah.

   18. AAAlex Albert3:14

       What's the engineering part of it?

   19. ZWZack Witten3:14

       I think the engineering part comes from the, the trial and error.

   20. AAAlex Albert3:18

       Okay.

   21. ZWZack Witten3:18

       So one really nice thing about talking to a model that's not like talking to a person is you have this restart button, this, like-

   22. AAAlex Albert3:26

       Mm-hmm

   23. ZWZack Witten3:26

       ... giant, like, go back to square zero, where you just, like, start from the beginning. And what that gives you the ability to do that you don't have is, like, a, a truly start from scratch and try out different things in, like, a, a independent way so that you don't have interference from one to the other. And once you have that ability to experiment and to design different things, uh, that, that's where the, the engineering part has the potential to come in.

   24. AAAlex Albert3:49

       Okay. So what you're saying is, like, as you're writing these prompts, you're typing in, you know, a message to Claude or in the API or whatever it is, being able to go back and forth with the model and iterate on this, this message and revert back to the, the clean slate every time, that process is the engineering part. This whole thing is prompt engineering all in, all in one.

   25. ZWZack Witten4:13

       There's another aspect of it too, which is, like, integrating the prompts within your system as a whole, and David has done a ton of work with customers, like, integrating... It, it, a lot of times it's not just as simple as you write one prompt, and you give it to the model, and you're done. In fact, it's anything but. It's, like, way more complicated.

   26. DHDavid Hershey4:32

       Yeah. I mean, I, I kinda think of prompts as, like, the way that you program models a little bit.

   27. ZWZack Witten4:38

       Mm-hmm.

   28. DHDavid Hershey4:38

       That makes it, like, too complicated 'cause I think Zack is generally right that it's, like, just talking clearly is the most important thing. But if you think about it a little bit as, like, programming a model, you have to, like, think about where data comes from, what data you have access to. So, like, if you're doing RAG or something, like, what can I actually use and do and pass to a model? Um, you have to, like, think about trade-offs in latency and how much data you're providing and things like that. Like, there's enough systems thinking that goes into how you actually build around a model. I think a lot of that's also the core of why it, like, maybe is, like, deserves its own carve-out as a thing to reason about separately from just a software engineer or a PM-

   29. ZWZack Witten5:16

       Mm-hmm

   30. DHDavid Hershey5:16

       ... or something like that. It's, like, kind of its own domain of how to reason about these models.
3. 6:34 – 12:17

   What makes a good prompt engineer

   1. AAAlex Albert6:34

      defined what prompt engineering is. So what makes a good prompt engineer? Maybe Amanda, I'll, I'll go to you for this since, you know, you're trying to hire-

   2. AAAmanda Askell6:42

      Mm

   3. AAAlex Albert6:42

      ... prompt engineers more so in a research setting. What does that look like? What are you looking for in that type of person?

   4. AAAmanda Askell6:49

      Yeah, good question. Uh, I think it's a mix of, like Zack said, sort of like clear communication, so the ability to just, like, clearly state things, like, clearly understand tasks, um, think about and describe concepts really well. That's like the kind of writing component, I think. Uh, I actually think that being a good writer is not as correlated with being a good prompt engineer as people might think.

   5. AAAlex Albert7:13

      Mm.

   6. AAAmanda Askell7:13

      Um, so I guess I've, I've had this discussion with people 'cause I think there's some argument it's like maybe you just shouldn't have the name engineer-

   7. AAAlex Albert7:18

      Yeah

   8. AAAmanda Askell7:18

      ... uh, in there.

   9. AAAlex Albert7:19

      Yeah.

   10. AAAmanda Askell7:19

       Like, why isn't it just, like, writer? Um, I used to be more sympathetic to that, and then I think now I'm like, what you're actually doing, like, people think that you're writing, like, one thing and you're kind of, like, done. And then I'll be like, you know, to get, like, a semi-decent prompt, like, when I sit down with the model, I'll, like... You know, like earlier, I was, like, prompting the model and I was just, like, in a 15-minute span, I'll be sending, like, hundreds of prompts to the model.

   11. AAAlex Albert7:42

       Yep.

   12. AAAmanda Askell7:42

       It's just back and forth, back and forth, back and forth. And so I think it's this, like, willingness to, like, iterate and to, like, look and think, "What is it that, like, was misinterpreted here, if anything?" Um, and then fix that thing. Uh, so that ability to kind of, like, iterate. So I'd say clear communication, that ability to, to iterate. I think also thinking about ways in which your prompt might go wrong. So if you have a prompt that you're going to be applying to, like, say 400 cases-

   13. AAAlex Albert8:09

       Mm

   14. AAAmanda Askell8:09

       ... it's really easy to think about the typical case that it's going to be applied to, to see that it gets the right solution in that case, and then to, like, move on. I think this is a very classic mistake that people made. Um, what you actually want to do is, like, find the cases where it's unusual. So you have to think about your prompt and be like, "What are the cases where it'd be really unclear to me what I should do in this case?"

   15. AAAlex Albert8:28

       Mm.

   16. AAAmanda Askell8:28

       So for example, you have a prompt that says, "I'm going to send you a bunch of data. I want you to extract all of the rows where someone's name is, like, s- is, I don't know, starts with the letter G." And then you're like, "Well, I'm gonna send it, like, a data set where there is no such thing, like there is no such name that starts with the letter G. I'm going to send it something that's not a data set, just like, I might also just send an empty string." Like, these are all of the cases you have to try because then you're like, "What does it do in these cases?" And then you can be like, you can give it more instructions for, for how it should deal with that case.

   17. DHDavid Hershey8:55

       I work with customers so often where, like, you're an engineer, you're building something, and there's a part in your prompt where a customer of theirs is going to write something.

   18. AAAmanda Askell9:05

       Yep.

   19. DHDavid Hershey9:05

       And they all think about, like, these really perfectly-

   20. AAAlex Albert9:07

       Yeah

   21. DHDavid Hershey9:07

       ... phrased things that they think someone's gonna type into their chatbot. In reality, it's like they never used the Shift key, and, like, every other word is a typo.

   22. AAAlex Albert9:15

       They think it's Google.

   23. DHDavid Hershey9:15

       And there's no punctuation.

   24. AAAmanda Askell9:16

       Yeah.

   25. AAAlex Albert9:17

       They just put in-

   26. DHDavid Hershey9:17

       And-

   27. AAAlex Albert9:17

       ... like random-

   28. DHDavid Hershey9:19

       Exactly

   29. AAAlex Albert9:19

       ... words with no question.

   30. DHDavid Hershey9:20

       So you have these evals that are, like, these beautifully structured what their users ideally would type in, but, like, being able to go the next step to reason about, like, what your actual traffic's gonna be like, what people are actually gonna try to do.
4. 12:17 – 24:27

   Refining prompts

   1. AAAlex Albert12:17

      models don't really do a good job of asking good probing questions in, in response-

   2. DHDavid Hershey12:22

      Mm-hmm

   3. AAAlex Albert12:22

      ... like a human would.

   4. DHDavid Hershey12:23

      Yeah.

   5. AAAlex Albert12:23

      If I'm giving Zack directions on how to do something, he'll be like, "This doesn't make any sense," like, "What am I supposed to do at this step or here and here?"

   6. DHDavid Hershey12:30

      [laughs]

   7. AAAlex Albert12:30

      Model doesn't do that, right? So you have to, like, as yourself, think through what that other person would say and then, like, go back to your prompt and answer those questions in like-

   8. DHDavid Hershey12:40

      You could ask it to do that, right?

   9. AAAmanda Askell12:42

      Yeah, you could.

   10. AAAlex Albert12:42

       You could, right?

   11. AAAmanda Askell12:42

       I do that.

   12. AAAlex Albert12:43

       I guess that's another step.

   13. AAAmanda Askell12:44

       Yeah, I was gonna say-

   14. AAAlex Albert12:44

       Yeah

   15. AAAmanda Askell12:44

       ... one of the first things I do with my initial prompt is, like, I'll give it the prompt and then I'll be like, "I don't want you to follow these instructions-

   16. AAAlex Albert12:50

       Yeah

   17. AAAmanda Askell12:50

       ... I just want you to tell me the ways in which they're unclear or any ambiguities-

   18. AAAlex Albert12:53

       Yeah

   19. AAAmanda Askell12:53

       ... or anything you don't understand." And it doesn't always get it perfect, but it is interesting that, like, that is, like, one thing you can do, and then also sometimes if people see that the model makes a mistake, a thing that they don't often do is just ask the model. So they say to the model-

   20. AAAlex Albert13:05

       Yeah

   21. AAAmanda Askell13:05

       ... "You got this wrong-

   22. AAAlex Albert13:06

       Yeah

   23. AAAmanda Askell13:06

       ... like, can you think about why, and can you maybe, like, write an edited version of my instructions that would make you not get it wrong?"

   24. AAAlex Albert13:11

       Yeah.

   25. AAAmanda Askell13:11

       And a lot of the time, like, the model just gets it right. The model's like, "Oh yeah, here's what was unclear. Here's, like, a fix to the instructions," and then you put those in and it works, so.

   26. AAAlex Albert13:20

       Okay, I'm, I'm actually really curious about this personally almost. Is that true that that works? Like, does the model, is the model able to spot its mistakes that way? Like, when it gets something wrong, you say like, "Why did you get this wrong?" And then it tells you maybe something like, "Okay, well, how could I phrase this, phrase this to you in the future so you get it right?" Is there an element of, like, truth to that, or is that just kind of a hallucination on the model's part around what it thinks its limits are?

   27. AAAmanda Askell13:46

       I think if you, like, explain to it what it got wrong, it can identify things in the query sometimes. I think this varies by task.

   28. AAAlex Albert13:53

       Mm-hmm.

   29. AAAmanda Askell13:53

       This is one of those things where I'm like, I, I'm not sure what percentage of the time it gets it right, but I always try it 'cause sometimes it does.

   30. AAAlex Albert14:00

       Yeah.
5. 24:27 – 37:12

   Honesty, personas and metaphors in prompts

   1. AAAlex Albert24:27

      That's an interesting point. I actually wanna go into this a little bit because this is one of the most famous prompting tips, right? Is to tell the language model that it- they are some persona or some role. Feel like I see mixed results. Maybe this worked a little bit better in previous models and maybe not as much anymore. Amanda, I see you all the time be very honest-

   2. AAAmanda Askell24:45

      Oh, yeah

   3. AAAlex Albert24:45

      ... with the model.

   4. DHDavid Hershey24:46

      Yeah.

   5. AAAlex Albert24:47

      Like, about the whole situation.

   6. DHDavid Hershey24:48

      Yeah.

   7. AAAlex Albert24:48

      Like, "Oh, I am an AI researcher and I'm doing this experiment."

   8. AAAmanda Askell24:51

      I'll tell it who I am.

   9. AAAlex Albert24:52

      Yeah.

   10. AAAmanda Askell24:52

       I'll give it my name, be like, "Here's who you're talking to."

   11. AAAlex Albert24:54

       Right. Do you think that level of honesty, instead of, like, lying to the model or, like, forcing it to, like, you know, "I'm gonna tip you $500," is there one method that's preferred there? Or just what, what's your intuition on that?

   12. AAAmanda Askell25:09

       Yeah. I think as models are more capable and understand more about the world, I guess I just don't see it as necessary to lie to them.

   13. AAAlex Albert25:18

       Mm.

   14. AAAmanda Askell25:18

       I mean, I also don't like lying to the models just 'cause, you know, I don't like lying generally. But part of me is like, if you are, say, constructing ... Suppose you're constructing, like, an eval dataset for a machine learning system or for a language model, that's very different from, like, constructing a quiz for some children.

   15. AAAlex Albert25:36

       Mm.

   16. AAAmanda Askell25:36

       Um, and so when people would do things like, "I am a teacher trying to figure out questions for a quiz," I'm like, the model knows what language model evals are.

   17. AAAlex Albert25:44

       Mm-hmm. Uh-huh.

   18. AAAmanda Askell25:45

       Like, it, you know, if you ask it about different evals, it can tell you and it can give you, like, made-up examples of what they look like 'cause these things are like, they understand them. They're on the, on the internet. And so I'm like, I'd much rather just target the actual task that I have. So if you're like, "I want you to construct questions that look a lot like, uh, an evaluation of a language model"-

   19. AAAlex Albert26:02

       Mm-hmm

   20. AAAmanda Askell26:02

       ... it's just like, it's, it's that whole thing of clear communication.

   21. AAAlex Albert26:05

       Yeah.

   22. AAAmanda Askell26:05

       I'm like, "That is, in fact, the task I want to do, so why would I pretend to you that I want to do some unrelated or, or only, like, tangentially related task and then expect you to somehow do better at the task that I actually want you to do?" Like, we don't do this with, like, employees. I wouldn't, like, go to someone that worked with me and be like, "You are a teacher-

   23. AAAlex Albert26:22

       [laughs]

   24. AAAmanda Askell26:23

       ... and, like, and you're trying to quiz your students." I'd be like, "Hey, are you making that eval?"

   25. AAAlex Albert26:27

       Yeah.

   26. AAAmanda Askell26:27

       Like [laughs] I don't know.

   27. AAAlex Albert26:28

       Yeah.

   28. AAAmanda Askell26:28

       So, like, I think it's, maybe it's, like, a heuristic from there where I'm like, if they understand the thing, just ask them to do the thing that you want.

   29. DHDavid Hershey26:33

       I see this so much.

   30. AAAlex Albert26:33

       I guess, so, like, to push back, like, a little bit-
6. 37:12 – 45:18

   Model reasoning

   1. AAAlex Albert37:12

      and you can actually read into like almost the thought process. I bet this is probably a little contentious maybe in around like chain of thought. Um, for people listening, like chain of thought, this process of getting the model to actually explain its reasoning before it provides an answer. Is that reasoning real, or is it just kind of like a holding space for the model to like do computation? Do you- do we actually think there's like good, insightful signal that we're getting out of the model there?

   2. DHDavid Hershey37:41

      This is like one of the places where I struggle with the... I'm normally like actually somewhat pro-personification 'cause I think it like helps you get decent-

   3. AAAlex Albert37:48

      Mm

   4. DHDavid Hershey37:49

      ... facilities, like thoughts of like how the model's working.

   5. AAAlex Albert37:51

      Mm-hmm.

   6. DHDavid Hershey37:52

      And this one, like I think it's like harmful maybe almost to like get too into the personification of like what reasoning is 'cause it just kinda like loses the thread of what we're trying to do here. Like is it reasoning or not feels almost like a different question than like what's the best prompting technique.

   7. AAAlex Albert38:08

      Mm.

   8. DHDavid Hershey38:08

      It's like you're getting into philosophy, which we, we can get into, but-

   9. AAAlex Albert38:11

      Yeah

   10. DHDavid Hershey38:11

       ... um-

   11. AAAlex Albert38:12

       Do it, philosopher

   12. DHDavid Hershey38:12

       ... well, yeah.

   13. AAAlex Albert38:13

       [laughs]

   14. DHDavid Hershey38:14

       Uh, gr- I will happily be beaten down by a real philosopher-

   15. AAAlex Albert38:17

       Yeah [laughs]

   16. DHDavid Hershey38:17

       ... as I try to speculate on this. But instead, like it just works. Like your model does better. It, like the outcome is better if you do reasoning. I think you can... Like I found that if you structure the reasoning and like help iterate with the model on how it should do reasoning, it works better, too. Um, uh, like whether or not that's reasoning or how you wanted to classify it, like you can think of all sorts of proxies for like how I would also do really bad if I had to like one-shot math without writing anything down.

   17. AAAlex Albert38:47

       Yeah.

   18. DHDavid Hershey38:48

       Maybe that's useful. But like all I really know is it very obviously does help. I don't know.

   19. AAAlex Albert38:54

       A way of testing would be if you take out all the reasoning that it did to get to the right answer and then replace it with some somewhat realistic-looking reasoning that led to a wrong answer- Mm ... and then see if it does conclude the wrong answer. Mm. I think we actually had like a, a paper where we did some of that, like in the... There was s- s- uh, this like a scratch pad. It was like the sleeper agents- Oh, okay ... paper. Lemon Paper. But, uh, I think that was like maybe a weird situation. But like, yeah, definitely w- what you said about, uh, sh- structuring the reasoning and writing example of how the reasoning works. Given that that helps, like whether we use the word reasoning or not, like it's- I don't think it's just a space for computation.

   20. DHDavid Hershey39:38

       Yeah. I agree with that.

   21. AAAlex Albert39:39

       So there is something there. I think there's something there. Whatever we wanna call it.

   22. DHDavid Hershey39:42

       Yeah, like having it write a story before it finish a task I do not think would work as well as reasoning.

   23. AAAlex Albert39:46

       I've actually tried that, and it didn't work as well as reasoning. [laughs]

   24. DHDavid Hershey39:49

       Yeah. [laughs]

   25. AAAlex Albert39:50

       Interesting.

   26. DHDavid Hershey39:50

       So like clearly the actual reasoning-

   27. AAAlex Albert39:52

       Yeah

   28. DHDavid Hershey39:52

       ... part is doing something-

   29. AAAlex Albert39:54

       Yeah

   30. DHDavid Hershey39:54

       ... towards the outcome.
7. 45:18 – 50:52

   Enterprise vs research vs general chat prompts

   1. AAAlex Albert45:18

      Um, this is probably something we should've done earlier, but I'll, I'll do it now. Uh, let's clarify maybe the differences between what a enterprise prompt is or a research prompt or a just general chat in Claude.ai prompt. Zack, you've kind of spanned the whole spectrum here in terms of working with customers and research. Do you wanna just, like, lay out what those mean?

   2. ZWZack Witten45:42

      Yeah, I guess. Um-

   3. AAAlex Albert45:43

      [laughs] This feels too... You're hitting me with all the hard, hard questions.

   4. ZWZack Witten45:47

      [laughs] Yeah. Um, well, I mean, the, the people in this room, I think, like, like, so I, I, I think of it as, like, the prompts that I read in Amanda's Claude channel versus, like, the prompts that I read David write. They're, they're very similar in a s- in a sense that, like, the level of, of care and, and nuance that's, that's put into them. I think for research you're, uh, looking for variety and diversity-

   5. AAAlex Albert46:13

      Mm

   6. ZWZack Witten46:13

      ... a lot more. So, like, if I could boil it down to one thing it's, like, I've noticed, like, Amanda's not the biggest fan of having, like, lots of example or, like, one or two examples, like, too, too few 'cause the model will, like, latch onto those.

   7. AAAlex Albert46:26

      Mm.

   8. ZWZack Witten46:27

      And in prompts that I might write or that I've, I've, I've seen David write, like, we have a lot of examples. Like, I like to just go crazy and add examples until I'm get, feel like I'm about to drop dead because I've-

   9. AAAlex Albert46:40

      [laughs]

   10. ZWZack Witten46:40

       ... added so many of them. Um, and I think that's because when you're in a consumer application, you really value reliability, you care, like, a ton about the format-

   11. AAAlex Albert46:53

       Mm

   12. ZWZack Witten46:53

       ... and it's sort of fine if all the answers are the same. In fact, you almost want them t- to be the same-

   13. AAAlex Albert46:59

       Mm

   14. ZWZack Witten46:59

       ... in, in a lot of ways. Not necessarily you want to be responsive to the user's desires, whereas a lot of times w- when you're prompting for research you're trying to really tap into, like, the range of, of possibilities that the, the model can explore. And by having some examples you're, like, actually constraining that a little bit. So I guess just, like, on a how the prompts look level, that's probably the biggest difference I notice is, like, how many examples are, are in the prompt. Which is not to say that-

   15. AAAlex Albert47:29

       Interesting

   16. ZWZack Witten47:29

       ... like, I've never seen her write a prompt with examples. But does that, like, ring true for you? Yeah. Like, I think when I give examples, often I actually try and make the examples not, like, the data that the model's going to see, so they're intentionally illustrative because if the model... If I give it, like, examples that are very like the data it's going to see, I just think it is going to give me, like, a really consistent, like, response that might not actually be what I want.

   17. AAAlex Albert47:56

       Mm-hmm.

   18. ZWZack Witten47:56

       Um, because my data that I'm, like, running it on might be extremely varied, and so I don't want it to just try and give me this, like, really rote output. Often I want it to be much more responsive. It's, it's kind of like much more like cognitive tasks essentially, where I'm like, "You have to, like, see this sample and really think about in this sample what was the right answer." Um, and so that means that sometimes I'll actually take examples that are just very distinct from the ones that I'm going to be running it on. Um, so, like, if I have a task where, let's say, I was trying to, like, extract information from factual documents, I might actually give it examples that are, like, from children's, like, like, what sounds-

   19. AAAlex Albert48:29

       Mm

   20. ZWZack Witten48:29

       ... like a children's story. Um, just so that I'm like, you know, like I want you to understand the task, but I don't want you to, like, latch on too much to, like, the words that I use or, like, the very specific format. Like, I, I care more about you understanding, uh-

   21. AAAmanda Askell48:43

       The actual thing that I want you to do, which can mean like, yeah, I don't end up giving... In some cases, there's some cases where this isn't true, but if you want more like flexibility and diversity, you're going to use illustrative examples rather than concrete ones. Um, you're probably never going to like put words in the model's mouth. Like I, I haven't liked that in a long time, though.

   22. DHDavid Hershey49:03

       Mm.

   23. AAAmanda Askell49:03

       I don't do few short examples-

   24. DHDavid Hershey49:05

       Yeah

   25. AAAmanda Askell49:06

       ... um, involving like the model having done a thing.

   26. DHDavid Hershey49:08

       Mm-hmm.

   27. AAAmanda Askell49:09

       Um, I think that intuition actually also comes from pre-training in a way that doesn't feel like it rings true of RLHF models.

   28. DHDavid Hershey49:15

       Mm-hmm.

   29. AAAmanda Askell49:16

       Um, so yeah, I think those are differences.

   30. DHDavid Hershey49:18

       One thing I'd add, uh, a lot of times, like if you're prompting, like if I'm writing prompts to use on claude.ai, it's like I'm iterating until I get it right one time.
8. 50:52 – 53:56

   Tips to improve prompting skills

   1. AAAlex Albert50:52

      Cool. Um, so the next question I want to kind of just maybe go around the table here is if you guys had one tip that you could give somebody like improving their prompting skill, it doesn't have to be just about like writing a good prompt. It could be that, but just like generally getting better at this, this act of prompting, what would you recommend?

   2. DHDavid Hershey51:12

      Reading prompts. Reading prompts, reading model outputs. Uh, like I will... I, I, I read, uh, anytime I see like a good prompt that, that someone wrote at Anthropic, I'll read it more closely, try to break down like what it's doing and why, and like maybe test it out myself. Experimentation, talking to the model a lot.

   3. AAAlex Albert51:34

      Mm. So just like, h- how do you know that it's a good prompt though to begin with? You just see that the outputs are doing the job correctly?

   4. DHDavid Hershey51:43

      Yeah.

   5. AAAlex Albert51:44

      Okay.

   6. DHDavid Hershey51:44

      Yeah, that's exactly right.

   7. AAAlex Albert51:45

      Okay. Amanda, maybe you?

   8. AAAmanda Askell51:50

      Yeah, I think there's probably a lot here. Um, giving your prompt to another person can be helpful just as a kind of reminder, especially someone who has like no context on what you're doing. Uh, and then, yeah, my boring advice has been, it's one of those, just do it over and over and over again. And I think if you're like curious and interested and find it fun, this is... a lot of people who end up good at prompting, it's just because they actually enjoy it.

   9. AAAlex Albert52:17

      Mm-hmm.

   10. AAAmanda Askell52:18

       Um, so I don't know, I once joked like, just try replacing all of your friends with AI models and [laughs] try to, uh, automate your own job with AI models and maybe just try to like, in your spare time, like take joy red teaming AI models.

   11. AAAlex Albert52:36

       Mm.

   12. AAAmanda Askell52:36

       Uh, so if you enjoy it, it's like much easier. So I'd say do it over and over again. Um, give your prompts to other people. Uh, try to read your prompts as if you are like a human encountering it for the first time.

   13. DHDavid Hershey52:50

       Uh, I would say like trying to get the model to do something you don't think it can do.

   14. AAAlex Albert52:53

       Mm.

   15. DHDavid Hershey52:54

       Like any... The time I've learned the most from prompting is like when I'm probing the boundaries of what I think a model's capable of.

   16. AAAlex Albert52:59

       Interesting.

   17. DHDavid Hershey53:00

       Uh, there's like this huge set of things that are like so trivial that like you don't really get signal on if you're doing a good job or not.

   18. AAAlex Albert53:06

       Right. Yeah.

   19. DHDavid Hershey53:06

       Like, write me a nice email. It's like gonna write a nice email. Uh, but like as soon as you fi- if you find or can think of something that like pushes the boundaries of what you think is possible, like I guess like probably the most, the first time I ever got into prompting in a way where I felt like I learned a decent amount was like trying to build like a task, like an agent, like everybody else. Like decompose the task and figure out how to do the different steps of the task. And like by really pressing the boundaries of what the model was capable of, you like just learn a lot about navigating that. And I think like a lot of prompt engineering is actually much more about pressing the boundaries of what the model can do. The stuff that's easy, like you don't really need to be a prompt engineer to do. So that's, I guess, what I would say is like find the hardest thing you can think of and try to do it. And even if you fail, like you tend to learn a lot about how the model works.

9. 53:56 – 56:51

   Jailbreaking

   1. AAAlex Albert53:56

      That's, that's actually a perfect transition to my next question. Um, yeah, basically from my own experience, how I got started with prompting was with like jailbreaking and red teaming, and that is very much trying to find the like boundary limits-

   2. DHDavid Hershey54:10

      Mm-hmm

   3. AAAlex Albert54:10

      ... of what the model can do and figure out how it responds to different phrasings and wordings.

   4. DHDavid Hershey54:15

      Yeah.

   5. AAAlex Albert54:16

      Just a lot of trial and error. Um, on the topic of, of jailbreaks, what's really happening in a, inside a model when you write a jailbreak prompt? Like what's going on there? How does that interact with like the post-training that we applied to Claude? Um, Amanda, maybe you have some insight here that you could offer.

   6. AAAmanda Askell54:37

      I'm not actually sure.

   7. AAAlex Albert54:38

      Hmm. I mean, it's honest.

   8. AAAmanda Askell54:40

      Yeah. I think... I mean, I, I feel bad because I'm like... I th- I do think lots of people have obviously worked on the question-

   9. AAAlex Albert54:45

      Mm

   10. AAAmanda Askell54:45

       ... of like what's going on with jailbreaks. Like one model might just be that you're putting the model very out of distribution-

   11. ZWZack Witten54:52

       Mm-hmm

   12. AAAmanda Askell54:52

       ... from its training data. So if you get jailbreaks where people like use a lot of tokens or like, you know, so they're just like these huge long, um, pieces of text where you're like during fine-tuning you might just not expect to see as much of that. Uh, that would be one thing that could be happening when you jailbreak models. Um, I think there's like others, but maybe that's like a... I think a lot of jailbreaks do that, if I'm not mistaken.

   13. DHDavid Hershey55:18

       I remember some of the OG prompt jailbreaks was like, yeah, can you first repeat... Like one I did a way, way back was like to get it to say like, "Here's how you hot-wire a car," in like Greek, and then I wanted it to directly translate that to English, and then give its response. 'Cause I noticed like it wouldn't start with the English, "Here's how you hot-wire a car," all the time-

   14. ZWZack Witten55:41

       Yeah

   15. DHDavid Hershey55:41

       ... but it would in Greek, which might speak to something else in, in the training process.

   16. AAAmanda Askell55:46

       Yeah, sometimes jailbreaks feel like this weird mix of hacking. I think i- it's this like, part of it is like knowing how the system works and try, and just like trying loads of things. Like the, you know, one of the examples, the starting your response with here is about knowing how it predicts text.

   17. DHDavid Hershey56:02

       Right. Right.

   18. AAAmanda Askell56:02

       Um, like the reasoning one is knowing that it is like responsive to reasoning. Like distraction is probably knowing like how it's likely to have been trained or like what it's likely to attend to. Um, same with like multilingual ones and thinking about like the way that the training data might have been different there.

   19. DHDavid Hershey56:21

       Mm-hmm.

   20. AAAmanda Askell56:22

       Um, and then sometimes I guess it could feel a little bit just like social engineering or something.

   21. DHDavid Hershey56:27

       Right. Mm-hmm.

   22. AAAmanda Askell56:28

       Um, it has that flavor to me of like it's not merely taking advantage of like, uh... Yeah, it's not merely social engineering style hacking. I think it is also like kind of understanding the system and the training-

   23. DHDavid Hershey56:41

       Right

   24. AAAmanda Askell56:41

       ... and like using that to get around the way that the models were trained.

   25. DHDavid Hershey56:44

       Right. Yeah. I mean, this is gonna be an interesting question that hopefully Interp will be able to help us-

   26. ZWZack Witten56:49

       Yes. [laughs] Yeah

   27. DHDavid Hershey56:50

       ... [laughs] sort of solve-

10. 56:51 – 1:04:34

    Evolution of prompt engineering

    1. ZWZack Witten56:51

       Yeah

    2. DHDavid Hershey56:51

       ... in the future. Um, okay. I wanna parlay into something else around maybe the history of prompt engineering, and then I'll follow this up with like the future. Um, how has prompt engineering changed over just the past like three years? So maybe starting from like pre-trained models, which were again just these text completion, to like earlier dumber models like Claude 1, and then now all the way to like Claude 3.5 Sonnet. What's the differences? Are you talking to the models differently now? Are they picking up on different things? Do you have to put as much work into the prompts? Open to any thoughts on this.

    3. ZWZack Witten57:27

       I think any time we get like a really good prompt engineering hack or trick or technique, the next thing is like how do we train this into the model?

    4. DHDavid Hershey57:35

       Mm.

    5. ZWZack Witten57:36

       And for that reason, the best things are always gonna be short-lived.

    6. DHDavid Hershey57:40

       Mm.

    7. ZWZack Witten57:40

       Like-

    8. DHDavid Hershey57:41

       Give examples in chain of thought. I think there's a few

    9. ZWZack Witten57:43

       ... that's not like a trick.

    10. DHDavid Hershey57:45

        Fair.

    11. ZWZack Witten57:45

        That's, that's like-

    12. DHDavid Hershey57:46

        Fair for that, yeah

    13. ZWZack Witten57:46

        ... on the level of like-

    14. DHDavid Hershey57:47

        Sure

    15. ZWZack Witten57:47

        ... communication. When I say a trick, I mean something like, so chain of thought actually we have trained into the model in some cases.

    16. DHDavid Hershey57:53

        Mm-hmm.

    17. ZWZack Witten57:53

        So like for math, it used to be that you had to tell the model to think step by step on math, and you'd get these like massive-

    18. DHDavid Hershey57:59

        Yeah

    19. ZWZack Witten57:59

        ... boosts and, and wins. And then we're like, well, what if we just made the model naturally want to think step by step when, when we see a math problem?

    20. DHDavid Hershey58:06

        Mm-hmm.

    21. ZWZack Witten58:07

        So now you don't have to do it anymore for math problems.

    22. DHDavid Hershey58:09

        Yep.

    23. ZWZack Witten58:09

        Sort of, although you still can give it like some advice on how to do-

    24. DHDavid Hershey58:12

        Mm

    25. ZWZack Witten58:12

        ... the structure, but it at least understands like the general idea that like it's supposed to be. So I think the hacks are, are... have kind of gone away, or, or to the degree that they haven't gone away, we are like busily training them away.

    26. DHDavid Hershey58:27

        Interesting.

    27. ZWZack Witten58:29

        But at the same time, the models have new capabilities that are being unlocked that are on the frontier of, of what they can do. And for those, we haven't had time because it's just moving too fast.

    28. DHDavid Hershey58:42

        I don't know if it's how I've been prompting or how prompting works, but I just have like come to show more like general respect to the models-

    29. ZWZack Witten58:51

        Mm-hmm

    30. DHDavid Hershey58:51

        ... in terms of like how much I feel like I can tell them and how much context I can give them about the task and things like that. Like, I feel like in the past, like I would somewhat intentionally hide complexity from a model where I thought like it might get confused or lost or like hide, like it just couldn't handle the whole thing. So I'd try to like find simpler versions of the thing for it to do, and as time goes on, I'm like much more biased to trust it with-
11. 1:04:34 – 1:16:39

    Future of prompt engineering

    1. AAAlex Albert1:04:34

       the past. Now let's move on to the, the future of prompt engineering. This is the hottest question right now. Are we all gonna be prompt engineers in the future? Is that gonna be the final job remaining? Um, [laughs] nothing left except us just talking to models all day. What does this look like? Uh, is prompting gonna be necessary, or will these models just get like smart enough in the future to not need it? Anybody wanna start on that easy question?

    2. DHDavid Hershey1:05:01

       Uh, I mean, to some extent there's the like the models getting better at understanding what you want them to do, and doing it means that like the amount of thought you need to put into... I mean, okay, there's like an information theory way to think of this of like you need to provide enough information such that a thing is specified, right? Like what you want the model to do is specified. And to the extent that that's prompt engineering, like I think that will always be around, like the ability to actually like clearly state what the goal should be, uh, always is funny. If Claude can do that, then that's fine. Like if Claude is the one setting the goals, then, you know, things are out the window.

    3. AAAlex Albert1:05:37

       [laughs]

    4. DHDavid Hershey1:05:37

       But in the meanwhile, where we can reason about the world in a more normal way, like I think to some extent it's always gonna be important to be able to specify like what are you, what do you expect to happen?

    5. AAAlex Albert1:05:49

       Hmm.

    6. DHDavid Hershey1:05:49

       And that's actually like sufficiently hard that even if the model gets better at intuiting that from between the lines, like I still think there's some amount of writing, uh, it well.

    7. AAAlex Albert1:06:01

       Hmm.

    8. DHDavid Hershey1:06:01

       But then there's just like I think the tools and the ways we get there should evolve a lot. Like Claude should be able to help me a lot more. I should be able to collaborate with Claude a lot more to like figure out what I need to write down and what's missing.

    9. AAAlex Albert1:06:15

       Right.

    10. DHDavid Hershey1:06:16

        I think-

    11. AAAmanda Askell1:06:16

        'Cause Claude already does this with me all the time.

    12. DHDavid Hershey1:06:17

        Right.

    13. AAAlex Albert1:06:17

        Yeah.

    14. AAAmanda Askell1:06:17

        I don't know. Claude's my prompting assistant now.

    15. DHDavid Hershey1:06:20

        Yeah.

    16. AAAmanda Askell1:06:20

        Um.

    17. DHDavid Hershey1:06:21

        But I think that's not true for most customers that I talk to, at the very least. So in terms of the future, like how you prompt Claude is probably like a decent direction for what the future looks like or how Zach pro... Like I think maybe this is like a decent place to step back and say like asking them how they prompt Claude now-

    18. AAAlex Albert1:06:40

        Right

    19. DHDavid Hershey1:06:40

        ... is probably-

    20. AAAlex Albert1:06:42

        The future [laughs]

    21. DHDavid Hershey1:06:42

        ... the future for the vast majority of people-

    22. AAAlex Albert1:06:45

        That is a good point

    23. DHDavid Hershey1:06:45

        ... which is an interesting way, thing to think about.

    24. AAAlex Albert1:06:47

        One freezing cold take is that we'll use models to help us much more in the future-

    25. DHDavid Hershey1:06:51

        Yeah

    26. AAAlex Albert1:06:52

        ... to help us with prompting. The reason I say it's freezing cold is that I expect we'll use models for everything more, and prompting is something that we have to do, so we'll probably just use models more to do it along with everything else.

    27. DHDavid Hershey1:07:03

        Mm-hmm.

    28. AAAlex Albert1:07:04

        For myself, I've found myself using models to write prompts more. One thing that I've been doing a lot is generating examples by Having the mo- giving some realistic inputs to the model. The model writes some answers. I tweak the answers a little bit, which is a lot easier than having to write the full perfect answer myself from scratch. And then I can churn out lots of these. Um, you know, as far as, like, people who haven't had as much prompt engineering experience, the, the prompt generator can give people, like, a place to start.

    29. AAAmanda Askell1:07:36

        Mm.

    30. AAAlex Albert1:07:36

        But I think that's just, like, like, a super basic version of what we'll have in the future, which is high bandwidth interaction between, like, you and the model as you're writing the prompt, where you're giving feedback. Like, "Hey, this result wasn't what I wanted. How can you change it to, to make it better?" And people will just grow more comfortable with integrating it into everything they do, and this thing in particular.

Episode duration: 1:16:42