Claude Engineering Lead: "Stop Using Claude Like It’s 2024"

1. 0:00 – 3:04

   You are asking too little of AI

   1. FRFelix Rieseberg0:00

      If all we're doing is, like, summarize this email for me, you're not really benefiting from the new intelligence.

   2. MMMarina Mogilko0:04

      This is Felix Rieseberg. His team built Claude Cowork in about a week and a half. Now he leads engineering for Claude at Anthropic, and he doesn't write code anymore.

   3. FRFelix Rieseberg0:14

      I'm just a conveyor belt for Claude. Claude needs to have access to all the crash data, and then needs to, like, figure out itself what it needs to fix, what it doesn't need to fix.

   4. MMMarina Mogilko0:21

      Is there still anything you would not trust an A- agent with?

   5. FRFelix Rieseberg0:24

      I rarely want the current session to judge itself. That I think is still really iffy.

   6. MMMarina Mogilko0:30

      How do I protect myself from AI just confidently presenting mistakes to me?

   7. FRFelix Rieseberg0:35

      It happens to a lot of us. I have, like, a very practical, easy answer.

   8. MMMarina Mogilko0:40

      Felix, thank you so much.

   9. FRFelix Rieseberg0:42

      Thank you.

   10. MMMarina Mogilko0:43

       I am very excited to have you as a user of Claude, like an avid user. We're using Claude for all the operations on this channel. Every team member has their own project for the things they're responsible for, and as someone who runs it, l- let's... It's a pretty big team for a creator. I have thirty-five people, and I want them to be as efficient as possible. I want to produce as much as possible. But also something that I've noticed this year, we were able to launch, I think, three new social media channels. So we launched a newsletter. We launched LinkedIn a year and a half ago because, you know, it's text-based.

   11. FRFelix Rieseberg1:18

       Yeah.

   12. MMMarina Mogilko1:18

       So AI has enabled me in many, many ways, and the goal of this conversation is to make me even more advanced and all of the viewers and all the listeners.

   13. FRFelix Rieseberg1:27

       I mean, from what I've, what I've seen you do, it seems like you're pretty advanced as it is.

   14. MMMarina Mogilko1:30

       But there is no borderline, right, to, to being as ambitious as possible. [laughs]

   15. FRFelix Rieseberg1:34

       That's true. That's true. Yeah. I think I'm discovering that about myself every single day. Like, I'm sort of... One of the most fun things of working on Claude by far is this notion of I built this product, and in comparison to my other products, with my other products it was pretty clear to me what you're gonna use this for, right? Like, uh, the example I always go with is I worked on Slack, and, like, I kind of have a feeling of, like, what are you gonna do with this? The thing that's magical about working on Claude is that I see people really fly and do amazing things that I didn't consider possible.

   16. MMMarina Mogilko1:59

       Mm.

   17. FRFelix Rieseberg1:59

       And that's like a, that's like s- for a product builder, that's such a treat, you know, to see people fly with this thing that you made.

   18. MMMarina Mogilko2:05

       Is there something you've seen that's super ambitious and you're like, "Oh my God, I want to copy this"?

   19. FRFelix Rieseberg2:09

       I'm not sure about copying, but two days ago, um, there's, there's... I think he's actually a product manager by trade. His name is Pavel, and he pointed Claude at a large amount of NASA data-

   20. MMMarina Mogilko2:21

       Mm.

   21. FRFelix Rieseberg2:21

       -and asked Claude to, like, find him a new planet, and it did. So apparently-

   22. MMMarina Mogilko2:25

       What?

   23. FRFelix Rieseberg2:25

       -it is not confirmed yet. NASA is working on actually confirming it. I'm a l- on a little shaky ground here because I'm not an astrophysicist of any kind. But apparently Claude may have found a new planet we didn't know of before.

   24. MMMarina Mogilko2:36

       This is fascinating. Okay. [laughs]

   25. FRFelix Rieseberg2:38

       And it's the kind of thing that I would have never considered, but if you hear the story of, I think for us who work maybe on more knowledge worky things, right, the idea of, like, point AI at a large amount of data and you're gonna find interesting things, that in, like, a, in, like, in, in isolation seems pretty reasonable, right? Like something we do every single day. But the idea of, like, looking at all of the public data we have about space [laughs] and considering it a similar problem of, like, go and see if you find anything in here we haven't noticed before, so

2. 3:04 – 5:44

   What Felix’s workday looks like now

   1. FRFelix Rieseberg3:04

      beautiful.

   2. MMMarina Mogilko3:04

      Can you talk to me about your day? You helped build Claude Cowork, but you said you're not coding anymore. So what happens in the morning? What do you do? Is it, like, your agents are briefing you on what's coming or what is it?

   3. FRFelix Rieseberg3:17

      That is true. My agents do brief me a little bit on what's coming for the day. I've really leveled up in sort of altitude on the problems that I'm thinking about, right? Like, and sometimes, sometimes I feel like these days I'm, I'm much more concerned with human emotion, empathy, trying to understand how people work in a way that I think I was less so as a, as a classic engineering manager a couple of years ago. So I think for me, what I'm really discovering is that the amount of times I have to reach out to other professions to get their help has really dwindled down. I'm now this weird mix between, like, a product manager and engineering manager, just a traditional people manager, a data scientist, right? To answer your question very specifically, most days I'm, I'm busy with a lot of prototyping, right? We just try a lot of things. We sort of, like, think about stuff. We sleep over it for a couple of nights, try to figure out if it's a good idea or a bad idea. And then we just go tell Claude to, like, build all the various variations and ideas we had, and then we need to do critique, try to figure out which ones of these have good ideas, which ones are bad. But I think prototyping is really what I spend most of my day on.

   4. MMMarina Mogilko4:16

      How do you decide what is a good idea? Like, what are the signals you're looking at?

   5. FRFelix Rieseberg4:20

      This has been difficult for people for a long time now, and I'm not sure that I have all the answers to, like, what is a good idea, what is a bad idea. I think generally speaking, the way we look at good or bad ideas, the calculus has changed, right? Like, if you think about products we've built in the past for the last ten, fifteen years, we, we always made implicit cost decisions, right? We, like, we, we sort of had to figure out, okay, especially for software, can I build this? What is the maintenance cost? What is the ongoing cost? Um, how much, how much time do I invest in this? What is the opportunity cost? All of these decisions that engineering managers make to figure out do I go and build this, yes or no? And I think now we can fast-forward to the end state much quicker and be like, "All right, just build me a prototype. Let me play with this. Let's try it out." I think human attention is still rather limited. Let's say you, for instance, you have a lot of taste in AI, right? Like, I can probably show you maybe one thing a day. I can easily build a thousand. I can't show you a thousand. Even if you're willing to sit down with me for all day-

   6. MMMarina Mogilko5:12

      Mm.

   7. FRFelix Rieseberg5:12

      -like, at some point, your eyes are gonna glaze over and you're like, "Okay, I'm kind of done with this." So what, what makes a good product? I think especially now that we live in this AI world, there's this old saying that, like, perfect is sort of when you can't take anything away anymore, and less so about, like, building a product that is every single feature under the sun. Like, we can make as many buttons as you want us to, but I think good products actually have fewer buttons. I kind of know whether or not a product really works if we put it in front of a few people and you see things click for them rather quickly, and they can do things that they couldn't do before. That, that's really the signal I'm looking

3. 5:44 – 7:13

   Using AI to do things you couldn’t do before

   1. FRFelix Rieseberg5:44

      for.

   2. MMMarina Mogilko5:44

      And how do you think about the underlying problem that you're solving? So for example, if you came to my company now-

   3. FRFelix Rieseberg5:49

      Yeah.

   4. MMMarina Mogilko5:49

      -how would you think about the tasks that you would fully automate with AI versus just leave it to humans versus maybe just automating parts of it?

   5. FRFelix Rieseberg5:56

      The first thing I wanna say is that I think about automation and augmentation in slightly different ways. Automation can be really empowering if you automate away tasks that sort of keep you from doing work that, that you enjoy, or if it's, like, expensive and automating it away, like, saves you a lot of, saves you a lot of money or time. In those cases, automation is, is useful, and I think it's really where AI currently probably provides most of the economic value, right? Like, when we evaluate is this worth it or not, we usually look at automation. The area I find most interesting though, and I think where people Really do things that they weren't possible with AI before is augmentation. It's this idea of AI is helping you do things that you couldn't do by hand because maybe you're lacking a skill or techniques or, um, just the raw knowledge to do something. And when I go to a company, the thing that is often most impressive to me is less so what the tool does and more so how you change your workflow.

   6. MMMarina Mogilko6:50

      Mm-hmm.

   7. FRFelix Rieseberg6:50

      Does that make any sense?

   8. MMMarina Mogilko6:51

      Yeah.

   9. FRFelix Rieseberg6:52

      AI is an interesting product because to really harness it, you kind of have to rethink how you do things. You have to, like, think about your outcomes. You have to figure out what does good mean, what does bad mean. Like, how do I evaluate whether or not this thing that I'm asking for, like, whether the result is good? And the people who impress me the most are usually the ones who are very good at coming up with... like, very good at reflecting on their own workflows.

4. 7:13 – 12:57

   How do you judge AI’s creative work?

   1. MMMarina Mogilko7:13

      Yeah, but it's so hard. It... When I'm thinking about my work, for example, and a lot of people use AI for drafting texts.

   2. FRFelix Rieseberg7:19

      Yeah.

   3. MMMarina Mogilko7:19

      Like, how do you define a good text? Like, is it still possible for, like, knowledge work? 'Cause for coding, I guess-

   4. FRFelix Rieseberg7:24

      Mm-hmm

   5. MMMarina Mogilko7:25

      ... it's possible, right? 'Cause you know what you want. You don't want any bugs. But when it comes to knowledge work and, like, the result of human creativity, how do you even define what's good for the agent?

   6. FRFelix Rieseberg7:35

      Yeah, it's such a tricky problem. Like, the, the example I often go with is, let's say your job is to, like, report on the weather, right? And you're like, "Agent, please figure out what the temperature is outside." There's, like, two ways to look at that because you have to rethink what good means, right? Like, if the model just guesses the temperature and is correct, that should not be a passing grade. That should be bad.

   7. MMMarina Mogilko7:53

      Mm-hmm.

   8. FRFelix Rieseberg7:53

      We should, like, rethink what we're doing here. But if the model is actually off by, like, half a degree or something, but it, like, um, really figured out the right location, figured out some reputable data sources and, like, sort of the, the, the way it got there was reputable and good, that should actually be better than, like, a guessed good result-

   9. MMMarina Mogilko8:09

      Mm-hmm

   10. FRFelix Rieseberg8:09

       ... if that makes any sense. But for writing and, like, all the other outputs where you feel like a lot of taste is involved, and it's like a little-

   11. MMMarina Mogilko8:16

       Yeah

   12. FRFelix Rieseberg8:16

       ... hard to, like, prove or verify whether or not the result is, like, better than another, it is tricky, but I think... I really enjoy talking to very good writers who are also a little AI-friendly.

   13. MMMarina Mogilko8:27

       What do they say?

   14. FRFelix Rieseberg8:28

       It is, it is fascinating to watch them work. My friend Adam Davidson, who was a staff writer at a number of, like, pretty good publications, he has all of these various prompts for Claude where he's like, "Okay," like, "Take my argument and put it to, like, magnify it by 100. Um, just mess with my text a little bit."

   15. MMMarina Mogilko8:44

       Mm-hmm.

   16. FRFelix Rieseberg8:45

       Like, "What are the things you disagree with the most?" Like, "What is the strongest counterargument?" It's interesting because I think, to me, as someone who's not a writer, I think the naive approach would be given a text, Claude, make this good. And I think what I'm discovering more and more is that for knowledge work, the secret sauce is, like, less in asking for a good output and sort of doing one step in between, which is the best in the field, how do they work, and can we automate those steps?

   17. MMMarina Mogilko9:10

       This would kind of mimic their workflow, but still, the taste isn't there. That's why I think you still need a lot of-

   18. FRFelix Rieseberg9:17

       Yeah, I think that's true

   19. MMMarina Mogilko9:19

       ... feedback.

   20. FRFelix Rieseberg9:19

       I think that's true.

   21. MMMarina Mogilko9:19

       Yeah.

   22. FRFelix Rieseberg9:19

       But the interesting thing is that if you talk to really good writers, most of them would disagree that the text is really good just because they have, like, the Midas touch. They would often say, "No, no, this is the result of hard work."

   23. MMMarina Mogilko9:30

       Interesting.

   24. FRFelix Rieseberg9:31

       Right? Like, it's this idea-

   25. MMMarina Mogilko9:32

       Yeah

   26. FRFelix Rieseberg9:32

       ... of, like, most artists have a process.

   27. MMMarina Mogilko9:34

       Mm-hmm.

   28. FRFelix Rieseberg9:35

       And many of them would actually probably find it, I don't want to say insulting, but, like, if you... The, the more I talk to artists and the more I explore this idea of, like, is your stuff just good because it comes natural to you and it just, like, happens to be good, or do you have a process that you, like, refined over the years, a lot of them are quite proud of their process and like, "No, this is very, very hard work." Like, "There's a lot, like, a lot of, like, m-manual individual steps go into making this good. It's not like I sit down and suddenly there's a book that's great." And if you go to, like, writer studios, right, there's, like, a lot of whiteboards and, like, a lot of information. And I'm, I'm noticing this across a number of different domains where Claude is quite receptive to being told not, "I want a result that looks like ha ha ha," but, "Claude, I want you to take the same steps that a professional in this field would take." This is, like, very high-minded, so let me just bring this down a little bit. Like, I think we're currently in the public seeing this quite a bit with, like, motion video and 3D, where I think Opus 5.5 has made a pretty big jump. But the big jump that the public has made is to just tell Claude to use Blender, right?

   29. MMMarina Mogilko10:35

       Yeah.

   30. FRFelix Rieseberg10:35

       Just like, "Claude, please use the same tools that professional animators use," and now the results are quite good.
5. 12:57 – 17:36

   How to catch AI’s confident mistakes

   1. MMMarina Mogilko12:57

      me?

   2. FRFelix Rieseberg12:57

      Yeah, it's a good point. I think it still happens to a lot of us. Um, I, I have, I have, like, a very practical, easy answer, which is adversarial reviews, um, which has proven quite effective in coding, math, and, like, various other areas where we can give you numerical results on, like, how effective it is. Adversarial reviews basically boil down to the idea of you've given, given one Claude a task- Spin off another 10 that try to, like, break this down. Their whole task is to attack this and prove to you why it's wrong. It's essentially the same concept as red teaming-

   3. MMMarina Mogilko13:26

      Mm-hmm

   4. FRFelix Rieseberg13:26

      ... or short selling.

   5. MMMarina Mogilko13:27

      Are you using the same model for that, or you want to use another model?

   6. FRFelix Rieseberg13:30

      Kinda depends on the task. This is a little context-dependent, um, but usually the adversarial review, it is, it is sometimes easier to find flaws than to create new things, much like as it is with humans, right? It's much easier for me to say whether or not the new phone is good.

   7. MMMarina Mogilko13:44

      Mm-hmm.

   8. FRFelix Rieseberg13:44

      It's much easier for me to do that than to design a new phone from scratch. So usually you can, like, take a step down in intelligence and still get very good results. That's the very practical advice output.

   9. MMMarina Mogilko13:52

      How would you prompt that? Like, if there is a process already, what's the additional prompt that you add?

   10. FRFelix Rieseberg13:56

       So you have some kind of output, right? Like, let's say it's a slide deck or a document, um, or in my case, it's ... You talked about this blog post that I wrote this morning. I just tell Claude very-- I'm, I'm using Claude Projects, which is this new product that we have that allows you to do work in multiple threads with a little bit more ease, and I basically tell Claude, "Please fire off five threads-

   11. MMMarina Mogilko14:16

       Mm

   12. FRFelix Rieseberg14:16

       ... and task each of them with, like, proving that this, this post is wrong, that I'm thinking about it the wrong way, that I have my facts wrong, that I misunderstood something." And then they go off with a given goal of, like, break this down. Just, you know, prove that I'm wrong.

   13. MMMarina Mogilko14:30

       How do you then deal with all the output? 'Cause I, I do the same, but I use, I just use different models when I'm-

   14. FRFelix Rieseberg14:36

       Mm-hmm

   15. MMMarina Mogilko14:36

       ... prepping for the interview. Like, tell me if I'm digging too deep into this or that.

   16. FRFelix Rieseberg14:40

       Yeah.

   17. MMMarina Mogilko14:40

       But then I'm giving so much different feedback, and I'm like, "Okay, now I have to decide," 'cause this model gave me this, this gave me something else, and gave, this gave me something completely different.

   18. FRFelix Rieseberg14:49

       Yeah.

   19. MMMarina Mogilko14:49

       How do you then decide [laughs] what to use?

   20. FRFelix Rieseberg14:52

       [laughs] I, I think you're gonna enjoy this because you can see where this is going. The answer is once again Claude.

   21. MMMarina Mogilko14:57

       Mm.

   22. FRFelix Rieseberg14:57

       What we use in a coding context is something we call a judge.

   23. MMMarina Mogilko14:59

       Mm-hmm.

   24. FRFelix Rieseberg15:00

       Where you get the output from the people who are, like, the, the agents whose job it was to attack your output, and they're like, "All right. Here's our main points of criticisms," and then you use-

   25. MMMarina Mogilko15:09

       You decide [laughs]

   26. FRFelix Rieseberg15:09

       ... yet another Claude-

   27. MMMarina Mogilko15:11

       Yeah

   28. FRFelix Rieseberg15:11

       ... and their job is... Your, your job is to validate and verify and, like, opinionate whether or not this is valid feedback.

   29. MMMarina Mogilko15:17

       Mm.

   30. FRFelix Rieseberg15:17

       Um, and then what you can sometimes do is that ultimately, um, depending on how you set up your workflow, th-the feedback, if it is valid, should probably go back to the Claude that actually made the thing, and that Claude too, that agent too, can be discerning in whether or not to act on feedback, right? Because sometimes, um, and we see this in a coding context fairly often. I also see it in, like, the knowledge work output that I make. Sometimes the feedback is, like, maybe reasonable but, um, has missed a point or is actually not that actionable or is not that useful, and that's usually when I know that my output is pretty good. That's one way. That's when you are really quite uncertain whether or not the output is good yourself, and you can't easily check. But for a lot of, a lot of things that I create with AI on the other hand, I actually don't need to fully understand the entire flow. Because as I mentioned, like, to me as a person, it is ... I do have opinions about almost anything I use every single day. I've remade my website with all of these 3D assets. I have no idea how to use Blender. I've never used it. But I can sort of judge whether or not this looks good, feels good, and that might be enough, right? Like, you sort of put yourself much more in a client position. For so many other things in our life, we don't really have a good way of verifying whether or not we get a good output other than, like, giving feedback to the person we're using, right?
6. 17:36 – 22:47

   Are we overcomplicating AI agents?

   1. FRFelix Rieseberg17:36

      trust it 100%.

   2. MMMarina Mogilko17:37

      When you say agent, do you have a special agent for fact-checking that you've trained, or it's just, uh, one agent that you're using for everything?

   3. FRFelix Rieseberg17:43

      It's just, just the Claude thread.

   4. MMMarina Mogilko17:45

      Okay. Okay.

   5. FRFelix Rieseberg17:45

      I feel like this is going to be a spicy take, which I think you'll like. I think we're kind of over-rotating on this agents thing, and I can back this up with a little bit. Like, I think models are really good generalists, and we've seen this now time and time again. Like, when we first started looking at math as an industry, I think the, the first company to really look at math, like, with, with a very serious lens was Google DeepMind, and they trained a custom model and a custom harness, and they unleashed it on the Math Olympiad, and they got a silver medal, which at the time was state-of-the-art. And then four or five months later, all the major generalist models come along and win a gold medal. I understand why people use agents. I have empathy. Because it's easier for us humans to rationalize what AI can do if we sort of, like, put on a little hat and, like, you're a data scientist.

   6. MMMarina Mogilko18:29

      Mm-hmm.

   7. FRFelix Rieseberg18:29

      Right? But that same data scientist has all the intelligence as, like, a field-leading biologist or mathematician or what have you, and I think we're, we're sometimes limiting ourselves in our use of AI if we sort of force ourselves to first c- like, make up this cast of characters. So for a lot of these things that I just me- I didn't really have any agents who are being told who they are.

   8. MMMarina Mogilko18:50

      But does the data matter? 'Cause for every agent, you upload certain dataset. Or just because your Claude has access to all your tools, it doesn't really matter what you feed to each particular agent.

   9. FRFelix Rieseberg19:00

      Exactly.

   10. MMMarina Mogilko19:01

       But then isn't it a lot of data for it to consider when answering each project- particular question?

   11. FRFelix Rieseberg19:05

       It is true, but it's quite discerning in what kind of data it wants to use, right?

   12. MMMarina Mogilko19:08

       Mm-hmm.

   13. FRFelix Rieseberg19:08

       Like, I think the right approach here is not to make these slices of data and then give every single agent the slice of data that it wants. Because many of us, if we wanna be quite effective at our job, we do actually need the full view on the entire business, right? We're not quite effective if we only get a small slice of data.

   14. MMMarina Mogilko19:23

       Yeah.

   15. FRFelix Rieseberg19:23

       Like, the, as an engineering manager, if you only give me code access, I'm probably gonna be much less useful to you as opposed to if I also have customer data, access to customer calls. Sometimes I do wanna sit in, like, a, like, a customer call, like, when we're trying to sell something. I wanna have deep usage information. Ideally, I get business data about what is working, what isn't, and you want your agents to have the same range of data. Now, how do you deal with the fact that it's probably a lot of data?

   16. MMMarina Mogilko19:48

       Mm.

   17. FRFelix Rieseberg19:48

       Tools. You give them tools, and you document really well what kind of data is behind this tool so that they don't actually have to ingest the whole thing, right? You're just like, "If you do want business data-" Here's the tool to get business data

   18. MMMarina Mogilko20:00

       So when you're setting this up, 'cause I'm thinking, like, when you mention this, should I move away from all my projects that have [laughs] access to all the, like, specific data? For example, I don't know, there's a project for ranking a guest, and we have the whole, uh, description of the process and then all the old scripts there-

   19. FRFelix Rieseberg20:15

       Mm-hmm

   20. MMMarina Mogilko20:15

       ... so that we know how each guest performed. But now that you're saying this, does that mean that I should move away from that project, tell my Claude, "Okay, we have... This is a folder with all the scripts if you want to look at them. In Notion, we have all the intelligence on how-

   21. FRFelix Rieseberg20:28

       Mm-hmm

   22. MMMarina Mogilko20:28

       ... every podcast guest performed, and, uh, this is how we're ranking guests," and just leave it there. And whenever I want to rank a guest, I just start a new chat without telling it anything, and just tell it, "Rank the guest," and it's gonna-

   23. FRFelix Rieseberg20:40

       Yeah, I'm careful to be prescriptive in workflows.

   24. MMMarina Mogilko20:43

       Mm-hmm.

   25. FRFelix Rieseberg20:43

       Like, I always say, if it works for you, it works for you, and I don't think you should change it.

   26. MMMarina Mogilko20:47

       The problem is the models are getting better, and I want to get better. So this's been working for this year, but a year-

   27. FRFelix Rieseberg20:52

       Yeah

   28. MMMarina Mogilko20:52

       ... is a lot. [laughs]

   29. FRFelix Rieseberg20:53

       Yeah, okay. Well, I mean, if, uh, if you're looking for the recommendation that I give for, like, what do I do-

   30. MMMarina Mogilko20:58

       Mm-hmm
7. 22:47 – 23:27

   “I’m just a conveyor belt for Claude”

   1. MMMarina Mogilko22:47

      my own decisions.

   2. FRFelix Rieseberg22:48

      Like, I think the upfront cost is higher. Like, if you... Like, even a year and a half ago, I was in this world of, like, telling Claude, "I found another crash. Please go and fix the crash," right? But it very quickly dawned on me, I'm just a conveyor belt for Claude, right? Like, I go look at our crashes, pick one up, move it over, drop it off in front of Claude, and I'm like, "Please fix this." I'm like, okay, this is-

   3. MMMarina Mogilko23:08

      Mm-hmm

   4. FRFelix Rieseberg23:08

      ... this is... No. I'm just gonna... Claude needs to have access to all the crash data, and then needs to, like, figure out itself what it needs to fix, what it doesn't need to fix. The upfront cost is really worth it, but it's also, like, important to, I think, to consider that there is upfront cost. But once you set this up, um, it, it opens up so many doors for you, and that's where I find a lot of, like, real hard value coming out

8. 23:27 – 24:06

   Your first AI agent in 20 minutes

   1. FRFelix Rieseberg23:27

      of AI.

   2. MMMarina Mogilko23:27

      If you've been hearing about AI agents and wondering how you'd actually set up one, I put together a guide that walks you through your first one in about 20 minutes. It helps you choose what to automate and get it working with no coding required. You can get it through my free newsletter. The link is below. For someone who just- who's listening to this conversation and who thought they would need, like, a major setup to start using AI correctly, would you say, uh, if somebody's starting just today, they're using it for knowledge work. They don't need all these tone of voice things like my business strategy. You just give it access to your email. It learns from it, from your tone of voice. You give access to your newsletters or whatever.

   3. FRFelix Rieseberg24:05

      Yeah.

   4. MMMarina Mogilko24:05

      So you don't need any

9. 24:06 – 25:08

   Felix’s advice for getting started with Claude

   1. MMMarina Mogilko24:06

      of that setup anymore.

   2. FRFelix Rieseberg24:07

      The best way to get started with Claude today, I think, is step one, download the desktop app. It has a built-in browser where you can log Claude into some system you're using, and forget about connectors. Forget about, like, downloading all the data. Forget about, like, all kinds of things. Just let Claude use your computer a little bit, sort of give away the keyboard. That gets you so far and really shows you what works and what doesn't work, and then at some point you might discover, oh, Claude is really helping me with whatever tool you're using, whatever internal or external tool you're using. And once you see that a workflow really works for you, you might consider, okay, can we give Claude something better than, like, having to manually click in this dashboard around? Can we give it some kind of connector?

   3. MMMarina Mogilko24:43

      Mm-hmm.

   4. FRFelix Rieseberg24:44

      But just starting with a browser gets you so far.

   5. MMMarina Mogilko24:46

      Okay, and when it-- Just to wrap up with the agents, when you say agents, that on- that means you have just basically chats, right? And that's it. There, there's no, like, specific agent who does research, no specific agent who does your tax returns, I don't know.

   6. FRFelix Rieseberg25:00

      No, no. I, I mean, I do have projects, so I do think about, I do think about different work items, but I don't think about different roles.

   7. MMMarina Mogilko25:07

      Mm-hmm.

10. 25:08 – 29:39

    Why “you’re an expert” isn’t enough

    1. FRFelix Rieseberg25:08

       Right? So I don't... I, I never tell Claude, "You're an excellent whatever."

    2. MMMarina Mogilko25:13

       Yeah.

    3. FRFelix Rieseberg25:14

       Um, I think this was, this was very en vogue sort of, like, 2024. We were like, "Claude, you're an excellent engineer. Please go fix whatever problems-

    4. MMMarina Mogilko25:21

       Mm-hmm

    5. FRFelix Rieseberg25:21

       ... are here." Um-

    6. MMMarina Mogilko25:22

       But you still use that in, in prompting now. Act like something that we started with. Act like a top whatever.

    7. FRFelix Rieseberg25:28

       I specifically implore Claude to, like, tell me what tools someone top of the field in this field would use.

    8. MMMarina Mogilko25:34

       Mm.

    9. FRFelix Rieseberg25:34

       And then I'm like, "I want you to use-

    10. MMMarina Mogilko25:36

        Mm-hmm

    11. FRFelix Rieseberg25:36

        ... the same fields." Um, a problem most models still have today is that if you simply tell it, data science is maybe a good example or, like, accounting, a lot of the models still have the problem that if you tell them to act like a very good accountant, that they engage in a little bit of role play.

    12. MMMarina Mogilko25:49

        Mm.

    13. FRFelix Rieseberg25:50

        And role play is not really what we're going for. We want the techniques, the tools, the knowledge that a professional would use. We don't really want just the vocabulary, right? Like, I don't, I don't need Claude to sound like- I don't need to put on a costume

    14. MMMarina Mogilko26:01

        You just need to act like, yeah

    15. FRFelix Rieseberg26:02

        I actually want it to talk to me like I'm five, but I want it to use all the top tools in the field.

    16. MMMarina Mogilko26:07

        Is there still anything you would not trust an AI agent with?

    17. FRFelix Rieseberg26:10

        That's a tricky question. We can just keep layering more validation on top. So I think the question for me is much more... It, it's less about, it's less about what do I not trust Claude with and much more about, uh, what things do I not trust one single session with.

    18. MMMarina Mogilko26:24

        Mm.

    19. FRFelix Rieseberg26:24

        And I think the, the list of things where I don't trust one single session is actually much larger than the list of, like, things I wouldn't trust Claude with. I rarely want the current session to judge itself. That I think is still really iffy.

    20. MMMarina Mogilko26:38

        Mm.

    21. FRFelix Rieseberg26:39

        Like, the current Claude session that is running, wherever you have it, in Claude Code, in Claude Cowork, in Chat, in even in a project, the current session tends to be quite bad at being a, being a judge of whether or not the work it did was quite good. So as much as I can, I try to find that outside signal, and the outside signal in a pinch can be another Claude session. That has proven to be quite effective. But my, my preference is to find some hard outside evidence for what works and what doesn't work. And sometimes you need to, like, trust the process a little bit and give Claude a little bit of time. Like, for instance, in your field, I'm kind of making this up, but I assume that maybe, like, the number of views or subscribers is, like, an important metric that you would-

    22. MMMarina Mogilko27:15

        Oh, the number of views, that's, that's what we're... Whenever I give a task to Claude, I tell it, like, "We need to hit this number of views."

    23. FRFelix Rieseberg27:22

        Yeah.

    24. MMMarina Mogilko27:22

        Go and try. But you... That, and that's, that's something that we're still, like, struggling with is this feedback loop, right?

    25. FRFelix Rieseberg27:30

        Yeah.

    26. MMMarina Mogilko27:30

        'Cause it's me giving feedback to my team, and then my team actually going and posting something else, not what Claude gave us. It's, like, maybe 70% the same, but then the 30% that I've added is actually my taste.

    27. FRFelix Rieseberg27:43

        Yeah.

    28. MMMarina Mogilko27:43

        How do you close that loop?

    29. FRFelix Rieseberg27:45

        A thing that has worked for us is there's probably some kind of place where this feedback usually happens. Is it usually in person, or does it usually happen-

    30. MMMarina Mogilko27:50

        It's, for all of us-
11. 29:39 – 37:34

    How to stop being your company’s bottleneck

    1. FRFelix Rieseberg29:39

       interesting.

    2. MMMarina Mogilko29:39

       But then it's teaching me. I want it to teach itself. [laughs] And I think a lot of people who start using AI experience this. You want to do more. You see how-

    3. FRFelix Rieseberg29:47

       Yeah

    4. MMMarina Mogilko29:47

       ... productive you are. But you... I don't... There is no agent that I trust 100%.

    5. FRFelix Rieseberg29:51

       Mm.

    6. MMMarina Mogilko29:51

       Every output goes through my team-

    7. FRFelix Rieseberg29:53

       Mm-hmm

    8. MMMarina Mogilko29:53

       ... and then eventually through me. So I become the biggest bottleneck in the company.

    9. FRFelix Rieseberg29:57

       Honestly, I feel like our products will do that for you pretty well today. I'm actually quite confident. Like, if Claude has the ability to watch you give this feedback for a while, like, the first thing you would probably do is you would sort of run it side by side.

    10. MMMarina Mogilko30:07

        Mm.

    11. FRFelix Rieseberg30:07

        This is an experiment I recommend to almost everyone. It's, like, really fun to do. If you currently have a trust problem, th- th- you will probably not be super excited to immediately give over the reins-

    12. MMMarina Mogilko30:16

        Oh, yeah

    13. FRFelix Rieseberg30:17

        ... right, like, from one day to the next.

    14. MMMarina Mogilko30:18

        Mm-hmm.

    15. FRFelix Rieseberg30:18

        But sort of run it side by side. Like, you read something. You write down your feedback, and, um, you do that for a few rounds and then just to check what kind of feedback Claude would've given.

    16. MMMarina Mogilko30:27

        Mm-hmm.

    17. FRFelix Rieseberg30:27

        And, like, where do they differ? Where are they the same? That also gives you something where you can actually sharpen the workflow a little bit because I think models will still be imperfect, right? There's still gonna be things that w- where, where the feedback is, like, a little different. And, like, even for most humans, if we take the same human and we ask them to review the same document, like, on a Monday versus a Friday, the feedback will be different. But this, like, side-by-side process is, is rather powerful because it shows you so much.

    18. MMMarina Mogilko30:51

        And how do I build this agent that talks to my team? Is it just something that I... like a website where they upload stuff and it gives feedback?

    19. FRFelix Rieseberg30:59

        I would recommend Claude Tag, which is Claude inside Slack.

    20. MMMarina Mogilko31:02

        Uh-huh. And you build... You give it access to my taste and-

    21. FRFelix Rieseberg31:05

        Yeah. It's like, it's, like, currently our core multiplayer experience, right, where-

    22. MMMarina Mogilko31:10

        Fun, yeah

    23. FRFelix Rieseberg31:10

        ... multiple people can talk to Claude at the same time.

    24. MMMarina Mogilko31:12

        Yeah. Yeah. Yeah.

    25. FRFelix Rieseberg31:13

        Yeah.

    26. MMMarina Mogilko31:13

        This is all getting very interesting when it comes to building things, right?

    27. FRFelix Rieseberg31:17

        Mm-hmm.

    28. MMMarina Mogilko31:17

        Because you can build so many things easy. What becomes valuable when you're building software? And I'm talking about, uh, people who are beginner entrepreneurs. I'm thinking about myself. T- what is it? T- 2015 we came here with a booking platform for study broad experiences. Amazing. Like, our mode was building this whole platform-

    29. FRFelix Rieseberg31:37

        Yeah

    30. MMMarina Mogilko31:37

        ... because we hired a lot of engineers, was do- hundreds of thousands of dollars. We automated stuff. Now I can do this in two days by myself.
12. 37:34 – 38:37

    The gap between AI’s capabilities and how we use it

    1. FRFelix Rieseberg37:34

       be quite successful.

    2. MMMarina Mogilko37:34

       I was just talking to a Nobel Prize laureate in economics, and he was telling me that we have such powerful models, but we really lack the application layer. Do you agree with him that there is this huge gap between what models are capable of and what average user is using them for?

    3. FRFelix Rieseberg37:49

       Oh, yeah, for sure. Yeah, we definitely have, like, a capability overhang, right? Like, the idea of you have an amazing model in front of you, and if you're essentially using it to, to ask, "Hey, what's the capital of France again?" You're probably not really tapping into what is possible there. Our founder and CEO, Dario, talks about this concept of, like, a data center of geniuses. My job really is to figure out, okay, like, let's assume we have a data center of geniuses. How do I empower you, the user, to actually make use of that? And it requires creativity. It requires, like, a rethinking of, like, your needs and, like, what you think is possible. Um, it's a, it's a really hard problem that I don't actually think is new. Right? It's this notion of people have always been held back by, by what they thought is possible rather than what is actually possible. Yeah, I do agree that we have like, w- we've just scratched the surface

13. 38:37 – 39:51

    Give Claude the goal beyond the task

    1. FRFelix Rieseberg38:37

       of what's possible.

    2. MMMarina Mogilko38:38

       How can one close that on an individual level?

    3. FRFelix Rieseberg38:40

       On an individual level?

    4. MMMarina Mogilko38:41

       Yeah.

    5. FRFelix Rieseberg38:42

       I thought about this, like, all weekend. I think on an individual level, the thing that you see on X a lot is this, this rallying cry of be more ambitious. We ought to be more ambitious. Like, what being more ambitious really means is, I think, so context-dependent. But I would like people to shed all the shackles of, like, what they believe is possible, and rethink why are they doing things, right? Like what-- Like, I'm doing a certain task in my business. What I, what am I hoping for is going to happen next? And you just keep giving the model that task. Like, whatever outcome you're hoping for, give the model that task. You can't play this forever. You will eventually find the limits of the current top intelligence. Like a good counterargument here is okay, like the purpose for most businesses is to make money, and if I just tell Claude, "Go make money," it's probably going to, like, not make it very far.

    6. MMMarina Mogilko39:31

       Mm.

    7. FRFelix Rieseberg39:31

       So we, we haven't reached like the top level of altitude yet. It's somewhere in the middle. But once you bump against that ceiling, you do know that you're at the forefront, you're at the frontier, and you're gonna really benefit.

    8. MMMarina Mogilko39:41

       Oh, that's a good feeling. Yeah.

    9. FRFelix Rieseberg39:41

       You're gonna benefit from the next model generations.

    10. MMMarina Mogilko39:43

        Yeah.

    11. FRFelix Rieseberg39:43

        Because if, if all we're doing is, like, summarize this email for me, or like, make this email sound less angry, you're not really, like, benefiting from the new intelligence.

14. 39:51 – 43:24

    What to build and learn to stay valuable

    1. MMMarina Mogilko39:51

       What would be your advice to someone who's just starting out now as a software engineer in this ever-evolving...

    2. FRFelix Rieseberg39:57

       In this ever-evolving like, landscape-

    3. MMMarina Mogilko39:58

       Yeah

    4. FRFelix Rieseberg39:58

       ... of, like, how to do software engineering?

    5. MMMarina Mogilko40:00

       Yeah.

    6. FRFelix Rieseberg40:00

       I think it's, I think it really depends on what you want as a software engineer, right? Like, what does it, what does it mean to be a software engineer? I think we're like, as engineers, we're current- we're currently in this like, moment of crisis where we're like, what does it mean to be a software engineer? Um, does it mean that I'm- I've completely memorized the entire API service in React? Is that what I'm valuable for? Is that what I do? Um, and I think that's probably less the case now. To me, this is a pretty welcome change. I've always been of the opinion that a good engineer versus a bad engineer, I measure-- I've always measured my engineers on their impact, and the impact is like, does the user like this enough to buy the software? There's, there's always for me been relatively little value in like really elegant, beautiful mastery solutions where we can be really proud of the engineering craft here, but the user doesn't feel any benefit.

    7. MMMarina Mogilko40:46

       How do you develop that? What do you double down on now in order to be relevant in three years if, like, every position is suddenly a senior position? What would be your advice there?

    8. FRFelix Rieseberg40:56

       The way I got my start in this career is by, like, building a strong portfolio. I think that would be the same advice today. I think building a portfolio that is not just like a mock is much easier. So my recommendation for anyone young would be go and build some real things. Prove that you can do it, prove that you have the empathy, prove that you have the chops, the ability to talk to customers. Like, figure out what is necessary versus not necessary. Um, I do think there's like a slightly new definition of what software engineering is. I personally think that's good. I do see the young people adapt to this new style of thinking much quicker than s- some of the older engineers who maybe enjoyed the luxury of primarily working on, on tickets without having to ask why too many times.

    9. MMMarina Mogilko41:36

       Mm-hmm.

    10. FRFelix Rieseberg41:36

        I've seen so many people be quite successful in that. Um, the things that I would maybe, to contrast, not recommend anymore, it is true that I think like completely memorizing individual API services I think has become less valuable. What is still valuable is being able to combine different systems in interesting ways. I think the value of that is amplified now because you can so quickly combine things, right? You can just like point AI-

    11. MMMarina Mogilko42:00

        And explore

    12. FRFelix Rieseberg42:00

        ... at like three different systems and be like, "I want these three-- Like, I want these five ingredients mashed together. Go and do it."

    13. MMMarina Mogilko42:05

        Yeah. Is that something that you look for when you hire people?

    14. FRFelix Rieseberg42:08

        Yeah.

    15. MMMarina Mogilko42:08

        This ability to-

    16. FRFelix Rieseberg42:09

        Yeah.

    17. MMMarina Mogilko42:10

        Is there something else that you're particularly looking for?

    18. FRFelix Rieseberg42:12

        I mean, I don't know how you hire. Do you still, do you, do you care a lot about the resume?

    19. MMMarina Mogilko42:15

        I do.

    20. FRFelix Rieseberg42:16

        Yeah?

    21. MMMarina Mogilko42:16

        When, 'cause, 'cause when you mention relationships, they're very important for my industry.

    22. FRFelix Rieseberg42:20

        Yeah.

    23. MMMarina Mogilko42:20

        I am looking for someone who knows people. I am looking for someone who knows how other businesses operate, 'cause this is not-

    24. FRFelix Rieseberg42:25

        Yeah

    25. MMMarina Mogilko42:25

        ... public information. Like, how do I learn how to, what, what do I charge as a podcaster, right?

    26. FRFelix Rieseberg42:30

        Yeah.

    27. MMMarina Mogilko42:30

        What do I charge brands? I want someone who knows exactly what's going on in the market. So we do care about relationships, but when it comes to the creative part, actually with creative part, we still look at how many views their videos have produced, 'cause-

    28. FRFelix Rieseberg42:42

        Exactly, yeah

    29. MMMarina Mogilko42:43

        ... that means they know the art. I don't look at their degree, of course.

    30. FRFelix Rieseberg42:46

        Yeah, yeah, exactly.
15. 43:24 – 49:23

    What children should learn in the AI age

    1. FRFelix Rieseberg43:24

       an amazing degree.

    2. MMMarina Mogilko43:25

       What would you tell your kid when she grows up and says like, "Dad, I want to do coding." What would be your advice?

    3. FRFelix Rieseberg43:30

       I mean, she's nine months old right now, so I think-

    4. MMMarina Mogilko43:33

       Oh, yeah, but I-

    5. FRFelix Rieseberg43:33

       ... things will change a little bit.

    6. MMMarina Mogilko43:34

       I think you have to be in line for your daycare already. [laughing]

    7. FRFelix Rieseberg43:37

       [laughing]

    8. MMMarina Mogilko43:37

       Just, just saying.

    9. FRFelix Rieseberg43:38

       It all happens quicker than you think.

    10. MMMarina Mogilko43:40

        Yeah.

    11. FRFelix Rieseberg43:40

        What would I tell my daughter? Um, I think I would tell my daughter to very early on try to understand how businesses operate, how people collaborate, um, how to really find value. I have not gone to the University of Waterloo, but I'm such a fan of the University of Waterloo's process.

    12. MMMarina Mogilko43:53

        Oh, yeah. All the amazing minds.

    13. FRFelix Rieseberg43:54

        All the amazing minds. But I think the University of Waterloo has always had this approach which was quite unique within, within computer science engineering, which was, this is an outside perspective, they might disagree with me on this. But my outside perspective was always coming from Germany, where computer science is very much computer science. It's informatics, it's a lot of math, and very little, like only a sliver of how do we actually build a good product, um, let alone computer engineering. JavaScript was, in Germany for a long time, universally considered to be like not what real computer scientists use.

    14. MMMarina Mogilko44:24

        Mm.

    15. FRFelix Rieseberg44:25

        And the University of Waterloo's approach has always fascinated me so much because I would get these interns or junior engineers who've worked at five different companies before in their internships. It was pretty common, especially in San Francisco, that you get an intern, and it's their like fourth internship, and they previously interned at Tesla, Google, and Apple. And they have so many interesting things they learned about how companies operate. What are the upsides and downsides? How do you actually advocate for an idea? How do you get resources? How do you convince legal to get something done? I think those are the skills that really, like, for me, made University of Waterloo's engineers almost immediately effective. It's their first job maybe with me, but they're in a completely different category than someone who has maybe even gotten a PhD in computer science but has never worked in a company. They might be very smart at what they do- But as we all know, like, building a good product and getting it out there is about much more than just knowing what code to write. You need to be able to talk to people. You need to be able to, like, work together with other functions. You need to have, like, a little bit of a, of a sense for how does this company run. So I would tell my daughter to probably, like, really invest in that.

    16. MMMarina Mogilko45:26

        Love it.

    17. FRFelix Rieseberg45:27

        What do you tell your kids? What do you think your kids should do in the ever-evolving economy?

    18. MMMarina Mogilko45:31

        I tell them to try all the things, and I tell them, "You can be anything," 'cause sometimes they're like, "Oh, robots." No, you can be the queen of... Like, if I tell her, like, "You can build a robot," she's, like, not very excited. I tell her, "You can be the queen of robots."

    19. FRFelix Rieseberg45:45

        Yeah.

    20. MMMarina Mogilko45:45

        [laughs] And she's like, "Oh, yeah, I like queen of robots." And then she says, and we, uh, we bought this, uh, self-driving car that you can program yourself.

    21. FRFelix Rieseberg45:52

        Oh, that is fun.

    22. MMMarina Mogilko45:53

        And she was building it with her dad.

    23. FRFelix Rieseberg45:54

        That's really fun.

    24. MMMarina Mogilko45:55

        And, like, just telling them they can be anything-

    25. FRFelix Rieseberg45:57

        Yeah

    26. MMMarina Mogilko45:57

        ... and there is no saying that they cannot touch because it's, I don't know, out of this world, whatever. 'Cause I came here with a notion that everyone here is from another planet.

    27. FRFelix Rieseberg46:06

        Yeah.

    28. MMMarina Mogilko46:07

        If you went to Stanford, for me, that was, like, something that I couldn't even fathom.

    29. FRFelix Rieseberg46:11

        Yeah.

    30. MMMarina Mogilko46:12

        But seeing that these are all normal people-

Episode duration: 49:23