Claude Code Secrets for PMs: The Operating System, Skills, and Data Viz

1. 0:00 – 1:40

   Intro

   1. AGAakash Gupta0:00

      Claude Code just hit $2.5 billion in annualized revenue in nine months. It is the fastest B2B software product ramp in history. So why are most people still using it like a chatbot?

   2. CVCarl Vellotti0:12

      Everyone says it's not about prompt engineering anymore, it's about context engineering.

   3. AGAakash Gupta0:16

      Why are we so obsessed with managing the context? What happens if you hit the context limit?

   4. CVCarl Vellotti0:20

      Where does that data actually come from, and how is it calculated is a, a major area where people just do not trust AI.

   5. AGAakash Gupta0:25

      Carl Vellotti has turned Claude Code into a operating system: native skills, data visualization, and sub-agents managing context.

   6. CVCarl Vellotti0:34

      When you have a sub-agent like this, what's awesome is that instead of doing that in its main session, it has spun up basically a clone of itself that is now doing all that work. So what I think is so amazing about this skill is that it actually doesn't give Claude Code specific new abilities. All it is, is just a really good prompt.

   7. AGAakash Gupta0:51

      This is how most people use Claude Code: type a prompt and get output. Carl Vellotti flipped it. He built skills that interview through a framework before building anything.

   8. CVCarl Vellotti0:59

      When you understand how Claude works, it makes it feel less like you're just prompting Claude and watching it, and more like you understand how it works and you're working together.

   9. AGAakash Gupta1:09

      Before we go any further, do me a favor and check that you are subscribed on YouTube and following on Apple and Spotify podcasts. And if you wanna get access to amazing AI tools, check out my bundle, where if you become an annual subscriber to my newsletter, you get a full year free of the paid plans of Mobbin, Arise, Relay app, Dovetail, Linear, Magic Patterns, DeepSky, Reforge Build, Descript, and Speechify. So be sure to check that out at bundle.aakashg.com. And now into today's episode.

2. 1:40 – 6:51

   Does Claude Code still matter vs Cowork and OpenClaw

   1. AGAakash Gupta1:41

      OpenAI just acquired OpenClaw, Anthropic released Cowork, and the question on the top of everybody's minds is, does Claude Code still matter? Carl Vellotti has been knee-deep in these tools, and he says the answer is yes. His last two episodes crossed over a million views across platforms, and if the first episode was the beginner version of Claude Code and the second episode was the advanced masterclass, today is the operating system layer. If you're already an eighty out of one hundred on Claude Code, we are gonna bring you to a ninety-five out of one hundred, and if you stay till the end, you will understand context management, creating sub-agents to manage your context for you, auto-triggering skills, and building an operating system around it all. So let's get right into it. The question on the top of everybody's minds is, should they still be using Claude Code in a world of Cowork and OpenClaw? What do you think?

   2. CVCarl Vellotti2:29

      Yeah. So it's a lot of these new tools. They're amazing. Claude Code was the first, kind of the most powerful, where all of the everything else is built on top of. You have tools like Cowork, which is, basically it's a, it's a UI that g- builds on top of Claude Code that tries to make it easier to use. Um, and it, it's good. For the, for the use cases that it does have of, like, managing files and things like that, it is a good tool. But ultimately, it is built on top of Claude Code. So all of those features, they exist in Claude Code first in a more powerful way, and then they kind of get constrained by Cowork. So if you want the most power just to be able to do things and do work, Claude Code is still, it's still the number one tool. And I would say with OpenClaw, it has, uh, it's, it's a lot like Claude Code, where essentially it is a way that you can connect to Claude Code or a Claude Code-like service, uh, from other channels. You can have it running all the time. It's really good for monitoring and, and tasks where you really can let it run autonomously. But if you need to be in the loop and you need to be in the driver's seat and you really want the most power in the tool, then you don't really do that in OpenClaw. You do that with Claude Code, and then you can set up OpenClaw to do some of that monitoring type of work. But if, if you need the person doing the most powerful work that you can, you really, you have to be in Claude Code.

   3. AGAakash Gupta3:48

      Yeah. For product managers, where you need to be the driver's seat, AI still can't do the thinking for you, Claude Code is still gonna be the tool of choice. OpenClaw is really cool for living and having the heartbeat feature and being able to chat with it, but Claude Code is still the most powerful. Both you and I have been living in Claude Code pretty much eight hours, 10 hours a day for the last six to eight months, and we've encountered four major friction areas that we're gonna help people solve in today's episode. If they listen to the whole episode, what are they gonna learn?

   4. CVCarl Vellotti4:17

      Yeah. So when I kind of look at our first two episodes we've done, we did a b- we did kind of an intro, we did advanced, and I would consider this one the mastery course, where you're really good at using Claude Code, but there's a still another level deeper that you can go to solve the biggest problems that you might have with Claude Code. So the first is just context management. How can you make it so that your context window doesn't fill up so fast and then you have to wait and compact, and then you do a couple more messages and it compacts again? So what are some strategies that you could really manage context to use Claude Code just way more effectively and not have to wait as long? The next one is just figuring out: How can you use skills to shore up other major weaknesses of Claude Code? People will say Claude Code's not very good at research. They'll say it's really not that good at UA, UI design. They'll say that, uh, it, it really can't design slides and things like that very well on its own. Most of the time, what you need to do is you just need to figure out: How can you provide the tool that Claude Code can use to do those things better? So just shoring up all of those weaknesses with skills. And then we'll answer one of the biggest questions that people have when they're working with AI, which is: How can you trust the results? For a lot of people, it, they don't really need AI to just be able to do tons of mass-produced work. You know, for product managers or, or people where they're really accountable for a very small number of documents, they don't need lots of work to be done. They just need the work that they do do to be very high quality and very trustworthy. And so today, we'll look at an example of how you can do that with data, uh, using Claude Code, where you can know exactly how it got its answers and that you can reproduce them deterministically every time. And then we'll put all of this together, and that's kind of like the last major question that people have about Claude Code is, "Okay, it can do all these things, but how do you actually put it together in a cohesive system so that when you're using it, you're not just kind of having files go all over the place?" So really today, I would say, is taking all of that-... Claude Code knowledge and Claude Code usage that you already have and just taking it to the absolute next level with giving Claude Code its own skills inside of a really strong operating system.

   5. AGAakash Gupta6:19

      I'm so excited for this. [laughs] It's gonna improve my own Claude Code usage. So show us the sauce. How do we handle context better?

   6. CVCarl Vellotti6:26

      Yeah. So the first thing to do, and this is, this is kind of like a fun little trick that I just recently learned, is you can actually... There's a lot of different elements of the Claude Code UI that you can actually customize. And so right here, uh, and just kind of a quick note on the setup, we're in Cursor here, but we're not using any Cursor features. We have Claude Code open here on the side, uh, and we're using the Claude Code CLI, and, uh, we'll just use that for this entire podcast.

3. 6:51 – 10:41

   Setting up the context status line

   1. CVCarl Vellotti6:51

      So the first thing I really wanna show is, uh, there's something that you can customize called the status line. And so you can actually have Claude Code show you how much context you've used at any given point throughout a conversation. And what's nice is that you really get a much better sense of how is your context filling up, what types of requests are filling up the context, where, where is it getting eaten into if you just set this up. So the first thing to do is just set up a nice status line, and it's, it's incredibly easy to set up a status line. There is a slash command called status line and, uh, I'm just gonna go ahead... I'm gonna go ahead and just put in a really nice one. So we're gonna... There's a bunch of different things you can include in this status line. So here we're gonna have it show us what model we're using right now, what folder are we in, and then make a little UI for the amount of context that we've used so that we can visualize it as we go. And it's almost like a tiny little piece of software or a tiny little UI where you can have it, like, when you're, when you have a lot of context left, it's green. When you have medium left, it's orange. And then when you're almost filled with your context window and Claude Code's about to compact, it turns red. And, um, this is something I discovered from another plugin we'll look at later, but it's incredibly easy and you can just literally do /status line, explain what you want, and Claude will build it into its UI right there. So we're, we're about to see that. We, um, we see here it spun up its own custom sub-agent called status line setup and is now going, going ahead and building that. And, uh, if you see just a second ago, we had nothing under here, and now we have this status line that we just described. So we have... It tells us that we're using Opus 4.6. Um, we're in this demo file that will be available to anybody that watches this video. And then this is how much context we've used. So we're only at 20% and we're in the green. So number one thing, set up, set up a nice status line and then that will help you really kind of learn about how to use context, uh, as you put in different prompts.

   2. AGAakash Gupta8:44

      And for people who maybe haven't encountered this issue, right, why are we so obsessed with managing the context? What happens if you hit the context limit? Doesn't it just auto compact? Isn't, isn't there no problem really?

   3. CVCarl Vellotti8:55

      Yeah, that, that's a good question. Um, first of all, I would say [laughs] the compacting is kind of one of the worst things about using Claude Code, um, especially if, if you give it, like, a long task. Like, "Hey, can you go research this thing for me?" And then it runs a bunch of web searches, and every single time it uses a tool call and every time it reads a webpage, it's, it's eating into that context window. And so if, if you have a bigger project where you have- you're working with a lot of files at once, it's easy for this to get to 100% with just, like, five messages and then you're waiting, like, you're waiting a long time. You're waiting, you know, two or three minutes for it to compact, and it can just really get in the way of, of your flow of using the tool. So avoiding that is-

   4. AGAakash Gupta9:31

      Mm.

   5. CVCarl Vellotti9:31

      For, for me personally, [laughs] that's why it's helpful to, um, kind of have the context get used up slower. But the real, kind of the real, uh, even, like, bigger and more, more important lesson is that as I've been using AI more and using Claude Code more, I knew this before, that everyone says it's not about prompt engineering anymore, it's about context engineering. But that's just absolutely, it's kind of the golden rule of using these AI and using these really powerful tools. They are able to work with the information that they have, and so you wanna make sure that they're only having the information that you actually need them to have, but that when they have, need that information, it's kind of everything that they could possibly use. And so you can, if you're limiting the things that they don't need, then you provides a lot more room for you to include the things that you do need, and the overall quality just really dramatically goes up.

   6. AGAakash Gupta10:18

      Exactly, and I think that there's still this phenomena of context rot which has been measured, which is that as you get longer into the conversation, the quality degrades. And as you use up your context window, you're gonna see that happen. So you actually want to give AI minimal amount of context to actually have it understand what you want it to do.

   7. CVCarl Vellotti10:40

      Exactly, exactly.

4. 10:41 – 12:03

   Ads

   1. AGAakash Gupta10:41

      Here's the dirty secret about prototyping. You spend two weeks building a prototype. You validate your assumptions. Engineering loves the direction. Then what happens? You throw the whole thing away. Bolt changes this completely. When you prototype in Bolt, you're not building throwaway mockup. You're building real front-end code that integrates with your existing design system. So when you hand it to engineering, they don't throw it away. They ship on top of what you've built. I use Bolt every single day. I host my LAN PM job cohort on it, and honestly, I'm up till 2:00 AM some days just vibing in the tool, having fun, and building. That's when you know a product is good, when you're using it past midnight, not because you need to, but because you want to. Check out Bolt at bolt.new/aakash. That's B-O-L-T.N-E-W/A-A-K-A-S-H. Link in the show notes. Today's episode is brought to you by Amplitude. Replays of mobile user engagement are critical to building better products and experiences, but many session replay tools don't capture the full picture. Some tools take screenshots every second, leading to choppy replays and high storage costs from enormous capture sizes. Others use wireframes, but key moments go missing, creating gaps in your understanding. Neither approach gives you a truly mobile experience. Amplitude does things differently. Their mobile replays capture the full experience, every tap, every scroll, and every gesture with no lag and no performance hit. It's the most accurate way to understand

5. 12:03 – 17:49

   Sub-agents for context preservation

   1. AGAakash Gupta12:03

      mobile behavior. See the full story with Amplitude.

   2. CVCarl Vellotti12:06

      And so one of my favorite ways to do that, so again, this was just setting up... Okay, now we, we have an idea of-... where our context is. So of course, the, the, you can always just do the nuclear option of clear, so that basically starts a new session. So the most effective way, and, and I have been using Claude Code for months and months, but I honestly only just realized in the last few weeks how amazing sub-agents can be for controlling context. So as an example, so right now, uh, one thing you can always do is you can run, um... Let me get rid of this. Ah, okay. One thing you can always do is you can always run /context, and that will show you where is the context of Claude getting eaten up. And so right now we see I have a couple things already enabled. So I have, um, right, if you go to the top it gives you, like, your overall summary. So here we have 16% of our context is used, which is crazy. That's just, I haven't done anything. We just cleared. So 16% is just eaten up, um, for doing... Well, for me, let's see what is taking it up. So the system prompt, which you can't really get rid of, that's just, like, part of Claude Code, so that's taking up 2%. And then all of my tools that I have enabled right now, I have some MCPs, some custom agents, other, all of this together is taking up another 8%. And then I have a bunch of these, like, I, right now I have Claude and Chrome enabled. So this is one thing just right off the bat. Do you have a bunch of stuff that's turned on that you don't actually need? And then you could go through and, and turn it off. So that is, like, the quickest way to just make sure that you're not eating up all your context.

   3. AGAakash Gupta13:30

      Yeah.

   4. CVCarl Vellotti13:30

      Um, but let's, let's go through an example here. So let's, uh, let's just give Claude a, a task where we're gonna see it eat up a lot of its own context. So we just saw right now to start this off, we're at 16%. So I'm just gonna run a pretty normal command here, which is research the top five Claude Code tips from this week. So when we ask it to do that, uh, actually one, I do have, uh, one tool turned on for re- for search, which we'll use later, but I'll turn that off for now. Um, okay, so let's go ahead and run that again. So research the top five Claude Code tips from this week. And so what is Claude gonna do here? Well, it's going, it has a web search tool, and it has a web fetch tool. Um, oh, okay, it's getting ahead of me here. It's, uh, what it's doing here and what it's decided to do is it's decided instead of running that search on its own, which is oftentimes what you do, or if you, if you ask it to do a task without telling it specifically to use agents, then it will just start running that prompt, and then it will kind of do all of that in your chat session, and it will eat up a lot of that information or a lot of that, uh, context window that it has just doing the searches and then just getting the results and then summarizing them.

   5. AGAakash Gupta14:40

      Yeah.

   6. CVCarl Vellotti14:41

      But when you have a, when you have a sub-agent like this, what's awesome is that instead of doing that in its main session, it has spun up a, like, basically a clone of itself that is now doing all that work. And what we're gonna get back here in our main session is we will just get the summary of it. So you save your main session from having to do all those tool calls and read all that information and then just get the summary. And so that right off the bat, if you can remember, uh, to use Claude or tell Claude to use sub-agents to do its work, uh, instead of having it do it in its main thread, for many things that will automatically just save a lot of its context because it's just getting the result instead of all of the work to get that result.

   7. AGAakash Gupta15:23

      And to implement this agent, I believe it's two parts, right? In your CLAUDE.md, you tell it what you just told it, "Use sub-agents to use your work," and then you create the sub-agent, the research agent, correct?

   8. CVCarl Vellotti15:33

      So yeah, great question. So Claude can do two things. One, it can either just make a, on the fly, make a new instance of itself and give it a task, which I think is what it did here, versus earlier when we saw it do that, um, the status line, that's like a dedicated sub-agent that is already sort of defined of how it works. Um, and sometimes it can also use those. So those, those both work. M- un- if you don't wanna take the time to, like, actually define a sub-agent, this will work for even just, like, on-the-fly tasks as well.

   9. AGAakash Gupta16:01

      Awesome.

   10. CVCarl Vellotti16:01

       Okay, so here what's cool is we have the result here, and let's go ahead and look at our context from when we started. We had 16% to begin with, and, uh, now we're only at 16.5%. So if we did this type of, uh, if we did this type of thing without doing that, uh, and when I was testing this on my own, it would fill it up to 25%, which is a lot for just a basic web search, right? Um, and the other thing that's really cool is that we saw it run this search, and, um, it kind of reports back on exactly what it did. It ran 10 tool uses, and it used, uh, almost 30,000 tokens. So all of that information is, is saved from our main session because we had it use this sub-agent.

   11. AGAakash Gupta16:43

       Nice.

   12. CVCarl Vellotti16:44

       Yeah. So that's a good one. Um, and this is where I would say it, uh, it really starts to feel when you can kind of understand how does Claude Code work and where are its weaknesses. It should dynamically create these agents, and we actually did see it do it here. Oftentimes, it will not do it when it would be a good idea. And the kind of, like, mental framework to have in your mind is, do you need this main session to do all the work? And so sometimes if you're, like, coding something or you have, like, a specific question, it's helpful for it to have all the context of, like, all those files so that you can ask it something directly and it doesn't have to see what its agents did and then pull that context in. Um, but here, when you understand how Claude works, you can kind of tell it how to operate itself, which really just levels up and it makes it feel less like you're just prompting Claude and watching it, and more like you understand how it works and you're working together.

   13. AGAakash Gupta17:32

       Love it. So I assume you could just say something like, "Use a research sub-agent" in the prompt if you wanna force it or it hasn't done it and it's doing the context filling up way. Is there a skill or something we should be using to further encourage Claude to use sub-agents?

   14. CVCarl Vellotti17:48

       Yeah, great

6. 17:49 – 23:58

   Creating skills live

   1. CVCarl Vellotti17:49

      question. So in fact, I do have in this, uh, in this file, I have a skill called Spin Up, which, uh, well, it's kind of the, the, the lingo is that it spins up a sub-agent. So sometimes when I'll be giving it a long task, I can say, you know, "ResearchAll the top Claude Code creators in the world, and I'm sure you would come up. And then we can, you just add to the end of this spin up. So instead of saying, "Please make sure you'd use dedicated sub-agents," we have this rule here where, um, basically it'll run the skill of spin up, and then it knows that it should ru- uh, parallelize that and, and use a sub-agent instead of doing it itself. So that's just kind of a shortcut to make this quicker.

   2. AGAakash Gupta18:28

      Nice. Can you just show us the rest of the spin up skill once we execute that?

   3. CVCarl Vellotti18:32

      Yeah. So let's go ahead and run it here, and then I'll make this bigger. Um, so spin up, so set up a new product. Uh, sorry, what this says, "When the user runs spin up, you help them configure..." Oh, sorry, this is actually not quite what we would want. Um, okay, so this is a good opportunity. So let's, let's stop this. So this spin up command, I think, uh, has some stuff that we don't want. Um-

   4. AGAakash Gupta18:53

      And for people who are wondering how you stopped it, you hit Control-

   5. CVCarl Vellotti18:56

      Yeah, great question

   6. AGAakash Gupta18:56

      ... right?

   7. CVCarl Vellotti18:57

      So, and yeah, and this is where you just, when you really get familiar with the UI. So Escape will stop it, and then one thing you can always do is you can always hit Escape two times, and then that will give you all of the messages that you've done. And this is also actually another way to, like, kind of save context. If you, if you ask a question and you realize it was the wrong question, instead of having that in the memory or if it did a bunch of stuff wrong, you can always just roll back to earlier, and then that will just get rid of all of that, and it just puts you right back in there with that exact same, uh, sort of wherever the context window was at that. So anything that you roll back and you erase is literally completely gone. So you have to be a bit careful, um, make sure that you do that and it wasn't something you maybe wanted, but it's a good way to kind of continue, conserve your, uh, your context window.

   8. AGAakash Gupta19:39

      So I think we're gonna fix the spin up age skill now and show people how to edit skills, yeah?

   9. CVCarl Vellotti19:44

      Yeah, let's do it. So you know what we can do? We'll just go ahead and delete this skill 'cause it actually has some stuff that's a bit wrong.

   10. AGAakash Gupta19:49

       Awesome.

   11. CVCarl Vellotti19:49

       So we'll just delete it, and we'll show how can you make a skill with Claude Code. So please make a new skill that helps me tell you when you should use, um, dedicated sub-agents instead of the main session to preserve your context window.

   12. AGAakash Gupta20:12

       Nice. And you guys see how easy and natural language it is? Skills is the unlock, [laughs] like Carl said. Whenever you feel like there's a weakness, in this case there's a context management weakness, spin up a skill, and now it's gonna write its skill for itself.

   13. CVCarl Vellotti20:26

       Yeah, exactly. And so, uh, now it's looking at the existing skill structure. It'll make sure it matches the format. And, um, I-- Anthropic recently made a nice change where you used to have to restart Claude Code for it to pick up skills, uh, that you made, but now it'll actually, it should just pick it up directly in here. And I didn't tell it what to call it, so it's gonna come up with its own name.

   14. AGAakash Gupta20:45

       Oh, yeah. It won't be slash spin up anymore. [laughs]

   15. CVCarl Vellotti20:47

       Yeah.

   16. AGAakash Gupta20:48

       I think this is now a good time to warn people, don't download too many skills from random skill marketplaces, um, especially if you're probably running Claude dangerously skip permissions. There have been a lot of malware and prompt injection attacks in those skills that people have been analyzing online. So it's generally best if you're gonna get a skill, get it from me or Carl, somebody you super trust, or just create them yourself.

   17. CVCarl Vellotti21:14

       Yeah, exactly. Your own skills are good, and then we'll look at one other marketplace that Vercel recently came out with, um, that's a, it's maybe not perfectly safe, but it is a pretty well-validated marketplace. But yeah, if you make them, I think most of the benefit from skills is, like, you know your own workflow. Once you've seen that you're repeating something multiple times, then it's time to make a skill, and it will be customized to your exact project.

   18. AGAakash Gupta21:35

       Mm-hmm.

   19. CVCarl Vellotti21:36

       So let's see. It came up with one called context guard here. Before starting any non-trivial task, evaluate whether it should run in the main session or be delegated to a sub-agent. So let's try this again. So we had our question from before, which was research the top five Claude Code tips from this week, and then let's ask it to use context guard. Context guard. There we go. And then you always-

   20. AGAakash Gupta21:59

       In the last episode, we sometimes had trouble triggering skills. It's best to just have this invocation with a slash command or use context guard skill in your prompt so that you don't run into the issue of it sometimes not using it.

   21. CVCarl Vellotti22:12

       Yeah, exactly. No, I did figure out how to fix that, so we'll, we'll cover that, we'll cover that quickly. But the, the most surefire way is, you know, know your workflows, know the skills that you have, and then, um, yeah, just use them when you need them. Um, okay, so it looks like it went ahead and did it. It said, "Applying the context guard. This is a web research task, high context, multiple searches, page reads, delegating to a sub-agent." So now anytime that we need to do this type of thing, um, you can just run this, uh, run this command, context guard, and that will preserve your main session a lot. And again, the rule is do you need the main session to know it? No, we don't need it to, like, see what all of the web results, like, web search results were. We just need to know what the final output is.

   22. AGAakash Gupta22:51

       This is fire. So how do we fix the skill invocation issue?

   23. CVCarl Vellotti22:55

       We'll get to that in one second. Um, the other things to show that are good is, so while this is running, one thing that's cool is when you have a sub-agent running is here it says, "Use Control + B to run in the background." So I'm gonna do Control + B, and what that will do is it actually now, uh, if we see down here it says, um, one, one local agent, and so we can go and see what that's doing. But now it's kinda nice because we can actually just continue our main chat while that's running so that we don't have to wait for it. And so this is really nice if you have, like, a bunch of stuff running in the background that you don't, you're not, like, immediately dependent on the results. You can have that agent run in the background, and then you continue your conversation, and then as soon as this local agent finishes, it will kind of inject the stuff into the chat, and then Claude Code will read what it, what it came up with.

   24. AGAakash Gupta23:40

       Nice. And you just clicked on the local agent to see that, right?

   25. CVCarl Vellotti23:43

       Uh, so I just navigated down and then hit Enter.

   26. AGAakash Gupta23:45

       Ah, down and then, yeah, cool.

   27. CVCarl Vellotti23:47

       And so here we see what it... It's kind of cool too. You can see, like, what was the initial prompt that Claude gave to it, and then we can see it's fetching and searching. When it's done, the next thing I wanna show is my f- absolute

7. 23:58 – 31:13

   The ask user questions tool

   1. CVCarl Vellotti23:58

      favorite Claude Code feature. It's my, it's, like, my absolute favorite. I like sub-agents. I think they're probably my second favorite feature. My absolute favorite feature of Claude Code is the ask user questions tool.Have, uh, do you, have you used it? Do you find yourself using it?

   2. AGAakash Gupta24:11

      Actually, no [laughs]

   3. CVCarl Vellotti24:13

      Oh, really? Okay. This is a-

   4. AGAakash Gupta24:14

      I feel like I'm missing out. I feel like... This is why I love recording these episodes, so I can learn the latest alpha [laughs] .

   5. CVCarl Vellotti24:20

      Yeah. The ask user questions tool is, is so awesome. So what we're gonna get back here, whenever this finishes, it's being very thorough, um, is we'll have this big list of Claude Code tips that it found from the internet. And then in nu- in, in, you know, let's say that we wanted to implement some of those into our, uh, system, then normally we'd have to read them, and we'd have to say, "Okay, can you tell me about this one? Can you tell me about this one?" Um, or, "How we, how can we implement this?" So we got these, like, five top Claude Code tips. So now what we're gonna do is we're gonna say, "Please use your..." And then the magic phrase here is just ask user questions. I don't think it has to be in one word, but that's kind of like the official way that it is. So ask user questions tool to help me see if we should implement any of these.

   6. AGAakash Gupta25:06

      Mm.

   7. CVCarl Vellotti25:07

      And what's so cool here is that Claude will now go through that, and it will dynamically come up with a list of questions to ask us, but it will be in, like, a new custom UI within Claude Code. Um, so what we see here is it says, "Which of these Claude Code tips would you like to implement?" And so it's giving us these in, like, a UI. So we can say, "Okay, I want this one and I want this one," and then when we hit submit, it will now have that in its, uh, in its memory, and it will be able to go through it directly. This tool, I just use it all the time. Like, if you are coming up with a list of requirements for a feature, then you can say, "Please use your ask user questions tool to grill me on all the possible requirements for this, uh, spec so that we can fill all that in." And what's so awesome is that this will kind of take it from where Claude Code is... The main reason that people don't like the output from AI is because there's just so many assumptions and so many things that it gets wrong because it really didn't have that in its context. But if you have it ask you, it will predict, like, where are my weaknesses that I have or, or where are the assumptions that I'm making that I need from the user, and it will ask you questions. And if you, if you really make it strong, like a strong prompt, like, "Please ask me as many questions as you possibly can," it will just keep asking you, and sometimes it'll be, like, 10 or, or 12 questions or something. Uh, I know one person on Twitter said that it asked them 67 questions, so they probably had a very vague starting place. But it's just, uh, it's just this really nice way to be able to work with Claude directly in this session rather than, uh... Uh, it's, it's, otherwise, it's actually pretty hard to a- answer questions. It might do one at a time, or it might ask you a bunch, and then you can number them in the chat. But this is just this beautiful UI that it will do right here in, in Claude Code, which I, I love this feature.

   8. AGAakash Gupta26:43

      Mm. I like it. So f- it's sort of like that idea that you're not engineering this massive prompt or building this massive skill to solve the problems for you. You're saying, "Please ask the user a question if you need clarification."

   9. CVCarl Vellotti26:57

      Yeah, exactly. Um, and yeah, and you can use it in all different kinds of way. You can say, um, "Hey, I have this spec. Use your ask user questions tool to, um, like, shore up any areas where you think might be, uh, unclear," or, "We're about to start this project. Use your ask user questions tool to ask some clarifying questions." Thing, things like that. So really just, like, any time it would be helpful for the AI to get more context to help you have a better output, use that tool, and then you can just do it right there. And it's, it's just, it's just so nice. It's just right there in the chat, um, better than anything else I've seen. I, I would say it's one of Claude Code's, like, most probably creative features.

   10. AGAakash Gupta27:31

       Amazing. So what other skills should people be creating?

   11. CVCarl Vellotti27:35

       Yeah. So, uh, I would say in general, whenever you're kind of, like, going through your workflow and you realize you're doing the same thing multiple times, that is a great time to add a skill. The other main time I would say it's really good to figure out how can you build a skill or how can you find a skill is areas where you think Claude Code is really weak. Then you might be able to just come up with, like, a prompt or skill that gives it the tools or just the knowledge it needs to do much better work. So an example here, um, if you type /plugins, it'll show you kind of like the overall Claude Code marketplace. The, the kind of most, the most safest possible skills that you can have are, of course, Claude's own official plugins. One of the most impressive of all of the skills I think that people have talked about is, um, so Claude has, they have 56 of their own plugins, is this front-end design skill. So you know, uh, it's very commonly talked about that, uh, AI always makes this sort of purple gradient, generic kind of, uh, UI, right? If you, if you just say, "Please make me a landing page like this." And I actually have a really good example. Um, when we, uh, let me, let me open the browser here. When we, uh, did our first episode, I kind of did, like, a small lead magnet, um, so that people could get the, the co- the, like, code that we did from that, and then I, I, I now have a Claude Code for everyone as my main thing, but I had Claude Code School. So this is a, a real example. I actually shipped this. This, this is a real live page right now. Um, this is the kind of output you get if you say, "Hey, just make me this." And this is just incredibly AI, right?

   12. AGAakash Gupta29:07

       Yep. Screens written by Claude [laughs] .

   13. CVCarl Vellotti29:10

       Yeah. You, you just know. But I also, I, for our next, uh, kind of like for our next version or our more advanced one, I had it, I had it use its front-end design skill, and this is what it came up with, where it's just, it's just much cleaner and nicer and feels, you know, it's, it, it's, it's still, uh, you know, still a classic kind of webpage, but now it just has so much more personality in this than the first one. So I would say this is just a perfect example where, okay, I think AI is bad at this. Well, maybe all you actually need is a skill. And what's cool about this skill in particular, so let's go ahead and look at it. Um, we'll have Claude Code open it up for us. Um, there's a couple different ways that you can approach skills. So what I think is so amazing about this skill is that it actually doesn't give Claude Code any, like, specific new, um, abilities. All it is is it's just a really good prompt. Um, I'm gonna have it open it for us here in Cursor. One thing that's interesting is when you install a plugin or you install something from the marketplace, it, uh, depending on how you have it configured, um-It's not actually in this repo. It's kind of like at the computer level, so it's, we're not seeing front-end design over here in skills. It's like a couple levels back. Um, but this is, this is what we have here. So it just shows like, uh, th- this is what all skills have, which is their name, and then their description, and, and when they're supposed to be used. Um, but this is just the prompt that Anthropic has come up with to tell the AI just better rules for how to do front-end design. So, like, think about the purpose of what you're building. Um, make sure that you have a clear conceptual direction. Um, here's the aesthetics, and it basically tells it to, "Don't be AI. Instead, follow these rules." And so I think it's, it's good just to show, you can have an incredibly powerful skill that doesn't have any MCPs or APIs or any code with it. Just write a really good prompt that you perfect over time can make a really, really good skill.

   14. AGAakash Gupta31:05

       Although skills with tools are pretty powerful, right?

   15. CVCarl Vellotti31:08

       Yes. Skills with tools are definitely more powerful. So that's what we'll look at, um, next.

8. 31:13 – 33:33

   Ads

   1. AGAakash Gupta31:13

      Today's podcast is brought to you by Pendo. Welcome to the SaaS plus AI agent era, where every product team, no matter the size, can build world-class experiences. Meet Pendo AI agents, virtual teammates that read your product data, chat with users, and take smart in-app actions so you don't have to. With agent analytics, Pendo shows exactly how those agents drive adoption, complete tasks, and even cut churn. No extra engineering lift, fully SOC 2 and HIPAA ready. And because Pendo's behavioral insights sync straight back into your BI stack, you get AI-ready data to fine-tune models and prove ROI in one place. Smaller team, bigger ambitions, you no longer need an army to deliver software your customers love. Grab early access at pendo.com/aakash. That's P-E-N-D-O.com/A-A-K-A-S-H. Today's episode is brought to you by NayaOne. In tech buying, speed is survival. How fast you can get a product in front of customers decides if you will win. If it takes you nine months to buy one piece of tech, you're dead in the water. Right now, financial services are under pressure to get AI live. But in a regulated industry, the roadblocks are real. NayaOne changes that. Their air-gapped, cloud-agnostic sandbox lets you find, test, and validate new AI tools much faster, from months to weeks, from stuck to shipped. If you're ready to accelerate AI adoption, check out NayaOne at nayaone.com/aakash. That's N-A-Y-A-O-N-E.com/A-A-K-A-S-H. I hope you're enjoying today's episode. Are you interested in becoming an AI product manager, making hundreds of thousands of dollars more, joining OpenAI and Anthropic? Then you might wanna do a course that I've taken myself, the AI PM certificate ran by OpenAI product leader Miqdad Jaffer. If you use my code and my link, you get a special discount on this course. It is a course that I highly recommend. We have done a lot of collaborations together on things like AI product strategy, so check out our newsletter articles if you want to see the quality of the type of thinking you'll get. One of my frequent collaborators, Pavel Hern, is the Build Labs leader, so you're gonna live build an AI product with Pavel's feedback if you take this AI PM certificate. So be sure to check that out. Be sure to use my code and my link in order to get a special discount.

9. 33:33 – 36:57

   Tool-powered skills with Tavily and Firecrawl

   1. AGAakash Gupta33:33

      And now back into today's episode.

   2. CVCarl Vellotti33:35

      Is, uh, there's kind of two ways that you can power up your skills. The first one is just you make a skill, and in that skill, it tells Claude to use, like, official tools. So what we're, we, uh, in this, uh, repo, I have, uh, one, a skill called web research. And so Claude, if you just have it do a general, uh, web search, t- like we did earlier in this, uh, demo, where we had it look up the top five, um, Claude Code tips from the week, those results that Claude Code gives are just notoriously unreliable. Um, one, it's basically just running Google searches, and it's just grabbing stuff from the first page. And I think you kind of know if you're just searching stuff on Google and you're clicking the first links, you have absolutely no idea if those are actually good links. I think, uh, pre-AI, w- uh, the internet was really starting to feel just, like, so much SEO-generated slop, even before there was AI slop. And, uh, if you-- do you really want that to be in the context window for Claude? So I would say this is another example of where people say, "Oh, you can't really trust Claude to do reliable research." Well, again, this is an area where if you just give it better tools, it can actually do better research.

   3. AGAakash Gupta34:42

      Mm-hmm.

   4. CVCarl Vellotti34:43

      And so in this, uh, skill, basically, we have it connected to two tools. One is a, an MCP called, um, Tavily. I'll zoom in here. Um, MCP, uh, called Tavily. So this is a-- it's, it's really good. It basically is just, like, a much more powerful, almost Perplexity type of search-

   5. AGAakash Gupta35:02

      Yeah

   6. CVCarl Vellotti35:03

      ... um, that gets, uh, much higher quality results than just a classic web search. And so here we're turning it on. It's a very generous free tier if anyone wants to try it. And then what we also have in this, um, skill is we have it, uh, c- connected to something called Firecrawl, which Firecrawl basically is once you, once you have a, a URL, that will scrape it and get really clean markdown. And so it's a nice way where normally when you go to a website and Claude looks at it, there's a bunch of extra HTML and, like, maybe JavaScript and things like that. Firecrawl will just get just the sort of, like, raw information that the AI actually needs without all of that other stuff. And, um, one thing that's cool about this is that, um, Firecrawl is actually, the way I have it set up, is actually not an MCP. So if we look at the MCPs I have set up here, um, MCPs are, are good, but as we saw earlier when we were just looking at our context window, they eat up a lot of context just right off the bat. Like, if we look at our context, we see that, um, Tavily is taking up, like, a couple thousand tokens just by existing.

   7. AGAakash Gupta36:05

      Yeah.

   8. CVCarl Vellotti36:05

      Versus what you can also, uh, install are CLIs. So a CLI is kind of, it's interesting. Uh, it's kind of like an older technology. It, it, it used to be the way that you would operate a computer in the beginning with a command line interface.And they kind of fell away, of course, with people wanting, like, better UIs for most things. But AIs are really, really good at using command line interfaces. And, um, essentially what they are is it's like you have a tool, and then you download it onto your computer, and so it's just sitting on your machine. And then the AI can use the command line directly, uh, just through because we're basically in the CLI here. And so you have all the power of that tool just installed on the, on your actual machine, so that when the AI wants to use it, it doesn't actually have to have context just holding it in there. It can just use that CLI directly from your machine. And so what we're seeing is, you know, originally there was MCPs, but I actually think MCPs

10. 36:57 – 39:30

    CLI vs MCP vs API hierarchy

    1. CVCarl Vellotti36:57

       are kind of starting to be the less preferred way over CLIs. And most tools already had CLIs, and there's becoming a lot more. So for example, if you've ever tried to connect the Google Workspace MCP, um, I actually tried to do that on an- another, uh, podcast live, and it's just, it's such a hassle. But there's actually a Google Workspace CLI, and that's what, for example, OpenClaw uses. So it's just way more reliable than using an MCP because it's just on the machine, and it's easy for the claw to figure out how to use it. And I think what we're gonna see in the next, over the next year is just a lot more CLI tools because AI are so good at using them.

    2. AGAakash Gupta37:33

       Yeah. Andrej Karpathy said it himself, like, you should prefer a CLI connection. And I think the hierarchy is basically MCP at the bottom because it eats up so much context, then API, CLI at the top. And funny enough, CLI is basically just usually a wrapper on top of the API, but agents like to work with CLIs.

    3. CVCarl Vellotti37:52

       Yeah, exactly. The other CLIs I would say are just awesome. Um, GitHub CLI, I would say basically mandatory. If you're, like, doing anything with GitHub, the CLI Claude is just aggressively competent at, at the GitHub CLI. It's, it's amazing. And, um, the Vercel CLI is another really good one, where, um, if you want to deploy your app, then you can-- it can check the logs, it can configure your environment variables. Um, those two, if you're, like, doing any vibe coding, if you have the GitHub CLI and the Vercel CLI, so much easier and so much more powerful.

    4. AGAakash Gupta38:24

       I think this is one of the most underrated hacks. If you take anything away from this episode, stop using MCPs, start using CLIs.

    5. CVCarl Vellotti38:31

       Yeah. And, uh, just literally, if you just, uh, if you just say, "Hey, Claude, does a CLI for this exist?" More often than you'd probably guess, the, the answer is, is yes. And, um, here what... And, and, and just download that and use that because it's just, it's just so powerful. So what I did is I went ahead and ran our web research, which uses Tavily to find stuff and then Firecrawl to kind of scrape the pages. And, um, that's just a... This is, again, a really good way to shore up that web search is not that good for AI. But if you give it access to these tools, it can be way more powerful. But then I have those wrapped into a skill.

    6. AGAakash Gupta39:02

       Amazing. So we covered front-end design, web search. I think the other thing people talk about is making slides.

    7. CVCarl Vellotti39:08

       Yeah. So another area where if you just say, if you just tell Claude, um, "Hey, can you..." Like, if it's something-- Web pages are, I think, easier for Claude to understand. There's tons of web pages in its data, so it kind of knows what they're supposed to look like. Um, but if you ask it to do something like, "Make a graphic for this," or, "Make a slide for this," then, and you look at its result, they're usually

11. 39:30 – 43:32

    Make slides skill with Puppeteer

    1. CVCarl Vellotti39:30

       not that great. Like, they're usually not laid out the best. There's usually, like, alignment issues that could be better or, like, the space isn't filled very well. So one of the best things that you can do, and this is, again, all across the theme of just, okay, well, I'm saying that Claude Code is bad at, at bad at these things. But for example, what a lot of people will do, and you'll might find yourself doing this, is you say, "Okay. Please make a slide in HTML that covers, you know, this thing." And then you m- it makes it, and then you screenshot it, and then you say, "Hey, can you fix this, this, and this?" And then it does it again, and you say, "Hey, can you fix this, this, and this?" Instead of, uh, you doing that, think, "Can I give Claude a way, can I give Claude Code a way to check its own work?"

    2. AGAakash Gupta40:07

       Mm-hmm.

    3. CVCarl Vellotti40:07

       And that is just a huge unlock for anything that you're doing with, like, visual design with AI, is give it a way to see its own work. And so what we have here is we have a skill called Make slides, and this is a bit of, a bit more advanced. So, so far we've had just words, and then we had words telling it that it has access to a couple tools. This one actually has, uh, code in kind of telling it how to use code in the skill.

    4. AGAakash Gupta40:28

       Oh.

    5. CVCarl Vellotti40:28

       So that when it runs this skill, it will now know that it can run that code to help it check its own work.

    6. AGAakash Gupta40:34

       Mm-hmm.

    7. CVCarl Vellotti40:35

       So here we have our Make slides skill. Um, this, uh, it tells it, like, okay, I want it to be these dimensions. I want it to have these layouts. So this is similar, you know, to the front-end design skill we saw. But what's new is that there's an actual build process. So it's step one, and it's telling it that, um, like, kind of the different types of code that it can use.

    8. AGAakash Gupta40:53

       Mm-hmm.

    9. CVCarl Vellotti40:54

       And it's saying, "Hey, you can measure overflow." So it's using a tool called Puppeteer, which is, like, a very classic front-end development tool, so you can take screenshots of HTML pages just exactly as they look when they're-

    10. AGAakash Gupta41:05

        Mm-hmm

    11. CVCarl Vellotti41:05

        ... rendered. And we're saying, "Hey, you have access to Puppeteer. Use it in this particular way." So it's telling it, like, the dimensions that it should grab. Um, and then it's doing a calculation here to see does anyth- does any of this text fall off the page? And that's a, that's a thing that if Claude doesn't have a tool to check it, to check that, it can't really do it on its own. But with a tool, it can do it basically perfectly. And so that's what this, that's what this skill does, is just embeds that code-based tool directly in the skill so that when we say, "Hey, make a slide," it will run these tools that it knows it has access to.

    12. AGAakash Gupta41:38

        So this is the implementation of the advice when I asked Carl the question, "What skills should you make?" And he said, "Well, anything that's coming up in your workflow." If it's coming up that you're iterating on these slides by having to look at it, build a skill. And now you guys are seeing the most advanced way to build a skill with code in it.

    13. CVCarl Vellotti41:52

        Yep. And so here I said, "Use your Make slides skill to make a single detailed slide for a presentation called From Watching to Collaborating," kind of the theme of this. And so now what it's gonna do is it's gonna go ahead and, um, create that, screenshot itself, calculate. And then, uh, one thing I found is if you just say, like, "You must iterate three times before you show me," it'll be much better than if you say, "Oh, just go until you're done." Um, I would say this is an area where you really see AI just-... clearly have, like, great taste. You know, if, if you give it an infographic that just has tons of information on it, they're like, "Wow, this is so amazing." Uh, but the actual content or the layout might be terrible. So it's, it's helpful to just say, "Hey, you have to improve it three times," and then it will find some ways to, uh, to try doing that.

    14. AGAakash Gupta42:34

        Yes. And there's also this crazy phenomena where if you repeat yourself with your skills, the AI does better. There was just a Google paper released where they showed that literally just pasting in a prompt twice helps. And this is, uh, I've been using this in almost all my skills these days. I'll say, after I've executed it once, I'll say, "Go double-check against this skill again," and it really improves it. And so Carl's technique is a way to sort of encode that so that it builds it into itself.

    15. CVCarl Vellotti43:01

        Yeah, that's super interesting. I think, uh, I've heard that, that-- I've heard that called the builder validator pattern, where, like, when a, when a AI just runs its own work, it doesn't really have that, like, self-reflection to, like, really know if it did it. Like, it tries to follow the rules. So just having it... I, I, I've never heard of just running the prompt the same time. That's very interesting. But I've, I have had it just like, "Okay, do this, and then here's the rules that that was supposed to do." Did it follow those rules in, like, the second pass? And it, the quality just goes up so much.

    16. AGAakash Gupta43:30

        So while this is building, can you show us how to

12. 43:32 – 46:49

    Auto-invoking skills with hooks

    1. AGAakash Gupta43:32

       auto-invoke skills? 'Cause I'm just on the edge of my seat about this. [chuckles]

    2. CVCarl Vellotti43:35

       Yes. Okay. So this is the riskiest part of this podcast. So last time, uh, I said that there was... Okay, well, actually, taking a step back, the way that skills are supposed to work is if when we look at this skill, for example, we have this skill and it says, okay, use when the user mentions make slides, make a slide, create slides, presentation. So it's supposed to just know from when you message it that, okay, great, the person just said, uh, make slides, so now I know I have this skill and I should use it. It, it doesn't really work that way. Uh, and it is kind of just, like, a funny thing about Claude Code is it, this, that auto in-invocation of skills is really finicky, and, and I think most people have seen that it doesn't work. One thing you can do, like, the most dead simple way is add to your CLAUDE.md the skills that your AI has access to.

    3. AGAakash Gupta44:22

       Yep.

    4. CVCarl Vellotti44:23

       That does, it actually does make a, a difference, but then you're adding to your CLAUDE.md, and it's kind of hard to, like, if you add a new skill, you have to go back and update the CLAUDE.md, plus then you're also, like, eating up context for no reason. Um, so the way to do it is with hooks, which is another thing that we covered last time, um, but then the demo didn't really work. But I think it's gonna work this time. I, I, uh, really heavily tested it. So what we have, um, a hook is basically some sort of, like, thing that you can have happen around the whole Claude Code workflow. So it can be when you send a message, it can be when a message comes back, it can be before compaction, it can be after compaction. So if you just think about, like, all of the things that Claude Code can do, like, just the whole life cycle of, like, everything that's possible within what Cl- like, runs a tool, runs a web search, all of that, you can add a hook before or after, and that basically says, "Okay, this type of thing has to happen." An example would be maybe every time a new dot, uh, a new markdown file is created, it has to have, you know, some formatting at the top, for example. So here, what we're gonna do is we're gonna use a hook, um, that... Okay. So we're gonna use a hook. It's a user prompt submit hook. So basically, every time I run a message, then that's when this hook will fire. And what we already have configured before I, uh, type this is we have a script that runs, and it just says, "Okay, are any of the words in the thing that the user just typed in, are those in the keywords of the skills that exist, uh, in the skills?"

    5. AGAakash Gupta45:55

       Nice.

    6. CVCarl Vellotti45:55

       And, um, this is nice 'cause it's just a script, so it's not, it's not like a... There are, like, there, there would be a way to do it where you would have the LLM, like, actually look, but that's very slow, whereas this is a script that just says, "If there is a match, then, uh, I will give it to the LLM," and then the LLM can decide, "Oh, okay, now is the right time to use that," or, "Now is not the right time to use that."

    7. AGAakash Gupta46:18

       Nice.

    8. CVCarl Vellotti46:18

       So what we did is we just added this hook, and then what, what we see happens is, uh, the way hooks work is, so we had the hook here, but it wasn't activated, 'cause you have to put it into your settings. And so that was how I, that was how I set this up. So now with, there is a hook on user prompt submit, says whenever this command is, whenever this, you know, command is run, user prompt submit, then run this code. And that code sees was one of the words that I just put in, does it match a keyword of one of my skills?

    9. AGAakash Gupta46:45

       Mm-hmm. The biggest question that PMs have about AI outputs is,

13. 46:49 – 55:09

    Jupyter notebooks for data trust

    1. AGAakash Gupta46:49

       "How can I trust this?" And you've built some workflows that people can use to enhance that. What are they?

    2. CVCarl Vellotti46:56

       Yeah. So one thing I've realized is that for, for product managers and, and people who, you know, they have a few things that have to be right, and they have to really be able to rely on it be to, to be right, one of the biggest areas that people are just really concerned about is using AI to do data analysis. 'Cause on one hand, it should, it should be really good at that, right? Like, it, it can run code, and y- you see it do all this other type of code stuff. But for data analysis, you really need the result to be correct, and I think it's an area where if you're not a data analyst and you don't really know exactly how the LLM is, is coming up with its results, then you're gonna be relying on these. You're gonna be presenting these to people. You're gonna be showing your boss this data. How, where does that data actually come from and how is it calculated is a, a major area where people just do not trust AI. But it turns out they're actually, like the other things we've been talking about, this is a weakness, but there is a, a sort of a tool or workflow that we can use to, um, give Claude Code mu- we can get much more visibility into Claude Code's process so that we can trust its results more.

    3. AGAakash Gupta47:54

       Mm-hmm.

    4. CVCarl Vellotti47:55

       And so the, the best way to do this is, um, this is something I, I've kind of play- been playing around with more over the last few months is, uh, is Jupyter Notebooks. So this is the kind of, like, standard artifact that analysts will use, and it's actually a really good thing to use within Claude Code. And the very cool thing is that we can basically create, uh, a, a Jupyter Notebook, which is just, it shows what were the exact queries used for this-What were the results? And it can visualize that all for you. And the really cool thing about Jupyter notebooks is that a Jupyter file will render natively in Cursor or VS Code. So you can actually just stay in the tool without having to look at it somewhere else. And it's, it's honestly one of the, like, nicest types of things that you can have be displayed here, like right in your, in your terminal or your IDE.

    5. AGAakash Gupta48:42

       How-

    6. CVCarl Vellotti48:42

       So here what we, what we have is we have, um, in our data folder here, we have survey responses. So we have 213 survey responses from all different types of people in this sort of, like, fictional company around how they're using this to-do tool. And so what we're gonna say is, "Please, uh, analyze this data in a Jupyter notebook and show me what we're working with." So just to start, you can... And this is kind of like a very small, quick example of how can you work with data in Claude Code. And so here it's gonna go ahead and it's gonna look at this, and it will just kind of show us what we're working with. It's pretty hard to see CSVs. There are some extensions that make it easier to see, but they're not great. So it's kind of nice to just have Claude Code, like, show you what you're working with. And right off the bat, we have here, we see this new file was created, this Jupyter notebook, and it's just showing us a very simple... It's actually still working here. Um-

    7. AGAakash Gupta49:33

       Showing-

    8. CVCarl Vellotti49:33

       So this is the code.

    9. AGAakash Gupta49:35

       Yeah.

    10. CVCarl Vellotti49:35

        Yeah, so it's running Python, and then what it will do is it's, uh, I, it looks like it has to install a few things to make this, like, actually render. But what we'll see is we'll see, like, an actual UI example of, um, of what this code is running for us. So we see that it's kind of just getting the, the columns and data, and it will present that to us. We're seeing Claude do some live debugging here.

    11. AGAakash Gupta49:56

        Yeah. If you ever go look over your data scientist's shoulder, they live in Jupyter notebooks. And so how do we add proof of work to our analysis? That's really what we wanna do. We wanna create traceable analysis. The Jupyter notebook is solving that problem for you.

    12. CVCarl Vellotti50:12

        Exactly. And, um, yeah, so now we see that there's this notebook. It, it just got the columns. It's showing us, like, what is the overall shape, so it has this many rows with this many columns, and here's an example of what some of those look like. So what we're kind of just seeing here is what is a Jupyter, how does it work? It's the code. It's kind of like some example output, and then it's rendering that nicely for us. But of course, you can do much more sophisticated things. You can... And it's honestly amazing that once you get more familiar with the types of data questions that Claude can answer, it's like you're working with a data analyst that will just answer any questions you want about your data. It will segment it in any way. You can do pretty sophisticated statistical types of analyses, all types of things that if you're working at a company, getting access to an analyst's time to ask questions that sometimes you're not even sure, like, exactly what you wanna learn or what you're hoping to learn, um, or, or if it would be worth it to do an, an analysis. That used to be something that had your whole process. You probably don't have that many analysts at your company. Long time for them to look at it, and you have to get it on their, their own research or data roadmap. Now, you can just do a lot of that exploration right here on your own, and you can actually trust the results. So for the next thing, what we'll ask here is, um, "Just show me the distribution of, uh, enterprise interest scores as a chart." So now we're moving from just looking at the raw data to now actually creating, like, a nice visual that shows a bar chart of the distribution, uh, between these different groups.

    13. AGAakash Gupta51:34

        So step one, it'll help you visualize a CSV. Step two, it'll help you create charts.

    14. CVCarl Vellotti51:39

        And we see it running its code over here, and then it's gonna-

    15. AGAakash Gupta51:43

        The really powerful mental model for Claude Code in general is take the roles that you work with. You almost wanna create versions of Claude Code for those various roles. That's kind of what the front-end design skill is doing. It's taking the designer role. It's trying to, like, distill what some of that person is doing. Here, we're trying to distill sort of what an analyst might be doing.

    16. CVCarl Vellotti52:04

        Yeah, I love that. That's a really good framing. It's like, what are the tools or thought processes that these roles have? And then how can we h- give those types of things, um, to Claude Code so that it is able to do stuff like that? And of course, it won't always be the exact same, but it, it, it approximates it much more, and that's usually good enough for what you need to do as, as a PM, at least to start. So we see now there's a whole nother set of code here, and now we have this nice distribution.

    17. AGAakash Gupta52:30

        Wow.

    18. CVCarl Vellotti52:31

        And so just to kind of round this out, um, as one example, so we're gonna have it do, like, a really crazy analysis. We're gonna say, "Show me a correlation heat map of all the numeric columns, satisfaction, feature importance scores, enterprise interest, and NPS. What patterns jump out?" So now we're going from just, you know, we know what we are hoping to get, like, out of the data, um, and now we're having it actually do a much more sophisticated analysis and then also look at it and tell us what it's noticing. And this is where you can really become, like, a collaborative partner with the AI. Um, of course, you're gonna see the things that it's saying, but oftentimes with this type of stuff, it's, it's pretty good, like, off the bat, um, especially if you give it some of those, like, more ways to think about what would be interesting in the data.

    19. AGAakash Gupta53:11

        And if you're feeling like, "I couldn't craft a prompt as good as Carl did," think about creating an analytics manager sub-agent and this as your analytics sub-agent, and say, "Analytics manager agent really good at formulating prompts for the analyst," and then the analyst goes execute. If you create this sort of sub-agent structure, again, replicating what it would be like in a real company, you can outsource a lot of the thinking to Claude Code.

    20. CVCarl Vellotti53:37

        Yeah, 100%. That, that is a good way to do it. You can have it, like, kind of know what are a bunch of different types of analyses that it could do. I think this is a really good, uh, example of where, like, it's much more fun to learn these things 'cause now there's an actual way that you could immediately apply them. So here we have this, like, awesome correlation roadmap. I, I don't think we need to go, like, too deep into it, but this is... It looks cool, and if you, if you knew that this was the right type of analysis that you wanted to do, then, uh, now we, we have it here. And, and what this type of analysis does is it just shows correlation between things. So, like, in the survey results, how often do people talk about enterprise interest and security? Okay, well, that kind of gives us a sense here, um, that, like, enterprise cares about security, enterprise cares about team management, enterpriseDoesn't care as much about pricing, right? They're a little bit more price flexible.

    21. AGAakash Gupta54:24

        Mm-hmm.

    22. CVCarl Vellotti54:25

        And so this is just one example of, like, a really cool analysis that, like, y- you know, the analyst at your company, I'm sure they could do it, but it would take them some time. Um, you're not even sure if it would be the right thing to ask. But now we have this, like, awesome analysis, and there's all different types of regression testing and all that kind of stuff, where if you, if you learn what those things are, now it's really easy to execute them. And you can have a lot more confidence in that you-- when you show your analyst or you show your manager, "Here is where we got that data from." And then they can look at this and tell you if there's anything wrong with that, rather than, you know, just having no transparency into how the AI actually got to that end result.

    23. AGAakash Gupta54:59

        So we've put together all these features for people, context management, skills, sub-agents, hooks, even Jupyter Notebook

14. 55:09 – 1:05:58

    The operating system file structure

    1. AGAakash Gupta55:09

       Python analysis. I think the biggest roadblock for people is, like, putting this all together into a system. So how do people build the right operating system for their Claude Code setup?

    2. CVCarl Vellotti55:18

       Yeah, I would say this is something that, uh, it's... The, the main, the main, main way is you kind of use it, and then you, you probably, like, even here, we haven't really been doing stuff in the system. So every file that Claude has been making has been keep getting kind of, like, dumped out here, and then now we have to figure out, you know, where should it go in the overall system. So just figuring out, like, where should the context layer live? Like, where are, where is, are the types of information that you wanna easily be able to provide to Claude Code? Where does that live? What are the different types? And then the other thing is thinking about, okay, well, I'm not just gonna have this be a static workplace where I'm just gonna only reference, say, "Oh, hey, in my knowledge folder, like, I have these different people I work with. I have, like, stuff about my company." You're actually gonna be creating new work as well and figuring out where that should go. So I'll just do kind of a quick kind of rundown of the system that I have found w- works really well in talking to other PMs. Um, I think it's a good starting place, and of course, you'll customize it yourself. Um, so for things where you just wanna be able to reference that don't change very often, having just a Knowledge folder with some of, like, the key things that you wanna be able to say, "Hey, Claude, um, I have a meeting with David Chen," who in this case is, uh, we're, we'll say is the director of Core. It's your direct manager. Um, these are the things that they care about. Um, how should I structure, you know, this piece of writing, or things like that. So just keeping in mind, like, who are the people you work with, what are, like, different reference material, research, having that in, like, kind of a static Knowledge folder, somewhere for meetings to go, um-

    3. AGAakash Gupta56:45

       And Carl's being pretty humble about this, but actually putting in the time to build this, guys, I'm not kidding, it will 10X the outputs. So if you spend the hour to add in this really important context and then structure it in this way he said, the outputs you get will look nothing like the outputs somebody else using Claude Code is building. And that's how you w- we talked about AI slop for a second, right, earlier. That's how you avoid that AI slop label.

    4. CVCarl Vellotti57:09

       Yeah, 100%. And there's so much cool stuff you can do with, with this type of knowledge in here. So one, people love this people folder, which is, like, kind of like a, a little dossier about, like, different people at your company. It-- no one even ever has to see this, right? This can be your way that you communicate with those people. But there's, um, or what you've learned about those people. But what's so cool is, like, now that Claude Code is becoming more central to all these, uh, like, you can connect to everything. You could, um, you know, Granola, which is the meeting transcriber, I think, uh, sponsor of this show, they, uh, they have an MCP that connects, um, you can pull in your meeting notes. So you could literally have a workflow. Like, you have a skill called, like, update, uh, people, that would look at what did people say in the meeting that you just had with them, and then add notes around, like, what are the types of things they care about or don't care about into their file. And then when you go to draft a message to them or you're thinking about something, um, related to them, you have this, like, really golden type of context from real meetings in here and that, that, that just provides such a totally different level of information and specificity that you can give to these models than just saying, "Oh yeah, I have my manager. He likes, you know, curt communication," or, or something like that.

    5. AGAakash Gupta58:15

       Exactly.

    6. CVCarl Vellotti58:16

       Yeah. So just having a somewhere for it to live so that then you can build on top of it is really the most important thing. Um, I will say the, the, so having your knowledge folders, like, and meetings and stuff like that is helpful. I would say the other main one that I personally find is the, like, absolute most helpful thing for me is I have a Projects folder. And the way I use a Projects folder is anytime I'm starting any kind of new task, like I'm preparing for this podcast, for example, I'll create a new folder called, you know, product growth Claude Code ep three. And then now anytime that I'm doing any work, like for example, um, like a real example for when I was preparing for this podcast is this is our third episode on Claude Code. And so I said, uh, I, I have a, I have my own, um, like transcription. There's actually a YouTube transcribing CLI. So I said, "Hey, please grab the transcripts from our first two episodes. What did we cover? What did we not cover?" And then it drops those into this folder, and then I say, "Okay, let's draft, uh, what could we do, look at in this next episode?" That goes in this folder. And so you kind of have a place for all that stuff to live. And what's so cool is that the next time, like let's say I leave Claude Code or, um, I'm on a different machine, or it's just another day, instead of having to sort of figure out, like, okay, what was all the information related to that project? What you can do is you can just literally take this folder, and then I'm gonna hold Shift, and you can just drop it in here and say, "Hey, we need to work on this board deck. Here's all the research I've done for it before. Get up to speed, and then let's work on it." And so the Projects folder is just, it's just such a nice way to have everything be couched, you know, somewhere in your system. And then, uh, next time you're coming back to it, it's all there. And then you archive it, but anytime you wanna go back to it, you have all of the initial artifacts and anything else that you created with Claude Code all right there, right in that Project folder.

    7. AGAakash Gupta1:00:00

       Nice. So I think it might be worth people doing a one-time download to build a Projects folder, right? So like, "Hey, here's our linear and our notion. Please build out my last five Projects folders. Pull the initial PRDs, pull the features results write-ups." And that's how you can practically implement this pretty fast.

    8. CVCarl Vellotti1:00:16

       Yep, exactly. And then just kind of like the very last thing to show, like once you do have this all set up, like what are some things that you can do? One, I'll just show quickly that, that people like a lot, is you can have like a kind of start standup command. So what this does, uh, or, uh-Yeah, start standup skill. So when we go to our skills and we look at, um, standup, what this does is it will pull in information from GitHub, it'll pull it in from our tasks folder, it'll pull it in from our calendar if we had our calendar connected. It'll check Linear, it'll... And then it'll put it all together for us. I don't have all of that hooked up right now, um, but it will look, it'll use the information that it does have available locally. And so it'll go through and it will pull in all that information, and all of those tools and all of those skills and all of those things that you have set up from this system, you can just start to com- you know, mix and match them, and having it all in one place lets you just use all this stuff in, like, a completely different way so that it just, right there, it has all the information it needs without you having to go and check all those places, you know, one by one. And so when I say I'm living in Claude Code all day, it, once you get to a certain point, it's like, oh my gosh, I do not wanna have to leave Claude Code. Like, I do not wanna have to go to check Google Analytics for how my website's doing. I really wanna just be able to ask Claude, because then I can say, "Hey, how is my, uh, website traffic, and can you compare that to the posts I've made on LinkedIn this week?" And that would be almost an impossible analysis to do if I was just clicking into those individual UIs. But when you have Claude Code here set up and connected to everything, and you have a place for that information to go, then it just becomes more and more compounding and more and more powerful.

    9. AGAakash Gupta1:01:49

       And that's something, to be honest, guys, I haven't done, so I need to do that right now. [laughs]

    10. CVCarl Vellotti1:01:55

        [laughs]

    11. AGAakash Gupta1:01:55

        And what about your CLAUDE.md file, your goals MD? What else? Just walk us through the rest of this operating system, how you set it up so that people can kind of get to your expert level fast.

    12. CVCarl Vellotti1:02:04

        Yeah, sounds good. So, um, the other ones that, I would say the other ones that are good to have, yeah, so data, we, like, we were kind of dropping all of our data analysis into here. Um, tasks, so just some... And I would f- I would say that, you know, you can, you can connect, um, Claude Code to Apple Reminders or Todoist or things like that. Oftentimes, you kind of, as you use Claude Code more, you just kind of start to become, like, a little bit more markdown native, I would almost say. And so, uh, I like just having a, a file that just says, okay, what are the things that I'm currently working on, uh, and then what are, like, other things that if I don't have time to work on them now, just drop them into this, uh, this other folder. And so this is a good way, when I just ran that standup command that says, okay, these are all the things you're working on, there's, these are the things that are not done yet. So having somewhere to manage, like, the work, and then also a place to h- h- house, um, ideas that you don't have time to get to is really helpful. Um, and then the other really, really, uh, the other important ones, tools. So this is where, um, for example, here we, uh, we have our meeting prep, uh, as kind of a tool. So this is something where, um, what it will do is this one will actually go through and it knows how to run the code in order to put together, uh, a meeting preparation for me. And so if I'm, for example, building my standup skill, I can say, "You have access to this tool, um, that lets you prep meetings," or, "You have access to this tool that, uh, will pull metrics." So for example, I have a big script that will pull in, like, my Beehiiv, um, newsletter information. It'll check my analytics and, and that's much better to have as, like, a pre-built script that you can then test. And what's, what's nice about having it in a tools folder is that anytime you build a new tool or, like, a new capability for Claude, it goes here and then you can, in the future, uh, like, include in your CLAUDE.md, "Hey, you have a tools folder. Um, use that to figure out, like, what access to other things that you actually have." Uh, and then the last thing that's just really good to set up and keep updated is your overall Claude file. Um, if, if you're not familiar, the, the Claude file has, like, a really kind of, like, special use in Claude Code, which is that it's automatically in context all the time. Um, so for example, when we run context over here in Claude Code, and we see, like, what are all the things taking it up, um, the CLAUDE.md will always be there. And what's cool is it's not just there in the first message, it's there in the background for every message. So this is the type of context you want it to just always be aware of. And, um, so here I have, like, a fictional company set up called Gradeflow. Um, and this is just, like, good to include things about how you like Claude to work with you, who you are, what are you working on, so that every prompt is kind of through a, a little bit of a, um, like, it's through a little bit of a lens of what you're actually working on rather than just being generic. And this is the type of thing where people love OpenClaw because it is so, has so much personality and memory about you. This is, like, the very li- very lightweight version that exists in Claude Code. Um-

    13. AGAakash Gupta1:04:54

        Yeah, it's not quite a soul MD file, but I would say, guys, there's a lot of alpha in continuing to iterate on your CLAUDE.md file every week, every month. Every time you run into an issue with Claude Code, pull it into your CLAUDE.md file and update that CLAUDE.md file. We have covered a lot of stuff, Carl. You have a lot more, too, at cc4pms.com, at The Full-Stack PM Newsletter, on your hilarious Instagram, your awesome LinkedIn and Twitter. Is there anywhere else people should be going to finding you and learning from you?

    14. CVCarl Vellotti1:05:21

        No, I think you hit the main ones. Um, CC4PMs is my free Claude Code for product managers course. I have a, another version, cc4everyone.com, that is, uh, just a more general version but s- covers the same thing. The cool thing about those courses is that they are Claude Code taught in Claude Code, so you'll say, um, "Hey, start lesson," and then Claude Code will actually guide you through a bunch of the stuff, for example, that I kind of showed you today. And then my newsletter, The Full-Stack PM. Those are the places you can find me.

    15. AGAakash Gupta1:05:47

        Amazing. Do check him out, do subscribe, and like and comment if we should have him back another time.

    16. CVCarl Vellotti1:05:53

        I, I, I hope, I hope we see some likes and comments. I, I love being on this show.

    17. AGAakash Gupta1:05:56

        Amazing. We love having

15. 1:05:58 – 1:06:48

    Outro

    1. AGAakash Gupta1:05:58

       you, Carl. Thank you so, so much.

    2. CVCarl Vellotti1:06:00

       All right. Thank you.

    3. AGAakash Gupta1:06:01

       Bye, everyone. I hope you enjoyed that episode. If you could take a moment to double-check that you have followed on Apple and Spotify Podcasts, subscribed on YouTube, left a rating or review on Apple or Spotify, and commented on YouTube, all these things will help the algorithm distribute the show to more and more people. As we distribute the show to more people, we can grow the show, improve the quality of the content and the production to get you better insights to stay ahead in your career. Finally, do check out my bundle at bundle.aakashg.com to get access to nine AI products for an entire year for free. This includes Dovetail, Mobbin, Linear, Reforge Build, Descript, and many other amazing tools that will help you as an AI product manager or builder succeed. I'll see you in the next episode

Episode duration: 1:06:48