EVERY SPOKEN WORD
80 min read · 15,738 words- 0:00 – 1:54
Intro
- AGAakash Gupta
I just saw the future of AI agents, and I'm not even overselling it. I had the founder of Relay App show us new stuff around how to use plain English to do vibe experimentation, to build out AI agents that make you more productive, that help you save money on employees. What are the limitations today of AI agents?
- JBJacob Bank
AI agents are still, in my experience, not ready to take on very complex tasks in a totally autonomous way without a human in the loop.
- AGAakash Gupta
'Cause I was just mind-blown by-
- JBJacob Bank
Yeah
- AGAakash Gupta
... how you've put 12 agents together to replace an employee. What are some of the other ones in here? What do they look like?
- JBJacob Bank
My Follow-Up Drafter. This was something that I wish I had discipline around my whole career, but I always forgot to do. I really think it's, like, a critical skill for all of us PMs for the future, that we can manipulate and construct our own AI agents to match our particular needs. What I wanna show you here is this prompt is ridiculously simple. And I'm sure people will argue with me on this, but, like, in my personal experience, a very simple prompt with one to three good examples at the quality of models that we have today is plenty good. Our entire webinar process is, like, I come up with a topic, I write it in a Google calendar event, I show up for the meeting, and then my AI agents do [laughs] all of the, all of the other marketing around it. It's, it's pretty awesome. [laughs]
- AGAakash Gupta
Whoa. That's a lot of marketing channels. People say I'm productive, but you are really productive. [laughs] That's insane. What's the next AI agent people need to build? Really quickly, I think a crazy stat is that more than 50% of you listening are not subscribed. If you can subscribe on YouTube, follow on Apple or Spotify Podcasts, my commitment to you is that we'll continue to make this content better and better. And now on to today's episode. Jacob, thanks so much for being on the podcast.
- JBJacob Bank
Thanks for having me. I'm excited.
- 1:54 – 2:14
Meet Jacob: The One-Man Marketing Army
- AGAakash Gupta
What are we gonna do today?
- JBJacob Bank
We're gonna see three real AI agents that I actually use and are super valuable and that every single product manager can and should build for themselves.
- AGAakash Gupta
Amazing. Let's get right into it.
- JBJacob Bank
The first thing I wanna show you is my executive assistant. I'm the founder of a small startup.
- 2:14 – 8:27
Live Demo: 12-Agent Executive Assistant
- JBJacob Bank
I don't have a human executive assistant, but I've built my own with a combination of the 12 AI agents that I'm showing on screen right now. You can see that I have them, them divided into calendar management, email management, and task management. And I've built these up over time as, as use cases have emerged where I wish I had an executive assistant to help me. So let me show you just a couple of the most valuable ones, and I wanna communicate how easy these are to build, but how valuable they, they actually are. So the first one I sh- I wanna show you-
- AGAakash Gupta
And before you even get there-
- JBJacob Bank
Yeah
- AGAakash Gupta
... no, this is like most people are paying Athena or some-
- JBJacob Bank
Right, thousands a month. Yeah
- AGAakash Gupta
... AI company, like, 2,000 a month or something at least. So we're saving you, at the price of a Relay subscription, what's the regular pro price?
- JBJacob Bank
Yeah. Our cheapest tier is $20 per month. I would say all of this executive assistant functionality can fit on the $40 a month plan. [laughs]
- AGAakash Gupta
Okay. So basically $960 a month of savings if you're able to tune your agent to your liking. So let's see if we like Jacob's.
- JBJacob Bank
Exactly. So let me show you, um, the first one I created, which is the Meeting Briefing Generator. I have a ton of external meetings as part of being founder/CEO. Sometimes it's with a customer or a prospective partner. Sometimes it's a podcast [laughs] like this one. And so I created a Meeting Briefing Generator that does the following for me. Before I have an event upcoming on my primary calendar, I like to get my meeting briefing 30 minutes before. And you can see in this flowchart view all of the things my AI agent is going to do for me in preparation for that meeting. First, it's gonna look at all the guests in the guest list, and then it's gonna do research on each guest individually. It's gonna look at all the past emails I've exchanged with that guest. It's gonna look at any meetings we've shared and find the notes of any meetings. It's gonna go find their LinkedIn profile, and then it's gonna combine that into a dossier on that individual. And if y- I can show you actually the prompt for the dossier because I think it's actually pretty cool. What I've told my assistant is, if I'm meeting someone for the first time, focus on the public information, like the information in their LinkedIn profile, because I've never met them before, so I wanna know the basics of who they are and what they're up to. If it's someone I've met several times before, I probably don't need their LinkedIn profile information as w- as much, but I really need the context from our previous meetings and emails. Then once the AI has done the research on each of these individual guests, it will then send me a notification to my meeting preparation channel in Slack with a combined notification. And just for kicks, I wanna show everyone the meeting prep notification that I got for this recording [laughs] that we're doing now.
- AGAakash Gupta
[laughs]
- JBJacob Bank
Um, that's always a fun, a fun behind-the-scenes. So I got this, I got this about 30 minutes ago. It says I'm gonna have a conversation with Aakash. It's gonna be hosted at this Riverside link. It gives me some more information about you, your background, et cetera. It includes the context of the emails we've exchanged about this. It includes a link to the guide you helpfully sent me to help me do a good job in this recording. And it told me s- you know, I need to start, be there in the Riverside five to 10 minutes early. And all of this was produced automatically 30 minutes before the meeting by my Meeting Briefing Generator agent. [laughs]
- AGAakash Gupta
Love it. And this all is getting sent to you in the place where you work, in Slack.
- JBJacob Bank
Exactly. I choose to have it in Slack. Other people-- And this is... Let me pause here 'cause I think this is actually super important. This is why the skill of building your own agents is so valuable. I might like my meeting briefings in Slack. You might like yours over WhatsApp. Someone else might like theirs over email. I like to get mine 30 minutes before. You might like to have yours at the beginning of the day or four hours before. I like to cover the previous meetings and notes with guests. You may wanna just focus on their information in your CRM, like HubSpot. And so in addition to having a pre-built agent like this that you can just plug and play, I really think it's, like, a critical skill for all of us PMs for the future, that we can manipulate and construct our own AI agents to match our particular needs. This is like, this is gonna be just as an important skill for the next 10 years of our careers as knowing how to make a spreadsheet was for the last 10 years of our careers.
- AGAakash Gupta
Yeah. Everybody got promoted to manager
- JBJacob Bank
Exactly. Yeah. Th- and I think it's, um, I think we're all TLMs. That's more of an engineering title, but I, I think it applies to PM too, where we still need to have our IC core intact because we all need to be doers, and we all need to know what goods look, good looks like. We need to be able to write good documents, communicate effectively, do research on our own. But then we also need to be able to manage [laughs] this, this team of AI agents that works on our behalf. I think there's gonna be sort of a, a squeezing that happens, where there's gonna be less need for purely bureaucratic people management, 'cause we're gonna have smaller teams. There's gonna be less need for super junior execution work. We're all gonna be these super ICs, super delegators that are, that are in the middle. And that's cool, 'cause that's, like, the most fun type of job you can have, where you're close enough to the work to really feel it, but also have some leverage in the form of help.
- AGAakash Gupta
So if we look at this, let's walk through some of the key steps here so that people understand them. First of all, you're using multiple AI models, I noticed in there. Using ChatGPT and Claude. Why'd you make that decision?
- JBJacob Bank
Yeah. So when deciding on which AI model to use, there's a cost-to-quality trade-off, and then there are also certain models that are better and worse at specific things. So you can see that I chose to write the individual dossier in GPT-4.1, which is a generally pretty high-quality model, but also very reasonable. And then when I run and write the final notification that's gonna go to me, back when I built this workflow, Claud- Claude 3.5 Sonnet was the state-of-the-art, and it's still good enough. I might change it to Claude 4 Sonnet now. And now that I have this model picker on screen, I think it's worth it to linger here, because what we've tried to do as part of the product is help give people intuition about which models are a good fit for which tasks. For example, when I'm parsing information out of a very large PDF, I'll typically choose a Gemini model. They have the longest-
- AGAakash Gupta
Yeah
- JBJacob Bank
... context window, and they're very cost-effective. When I have a writing
- 8:27 – 15:05
Meeting Briefing Generator Walkthrough
- JBJacob Bank
task, I'll typically pick Claude 4 s- Claude Sonnet 4. And when I have-
- AGAakash Gupta
Beautiful emails
- JBJacob Bank
... like, a spreadsheet analysis task, I'll typically pick, uh, OpenAI's o3.
- AGAakash Gupta
[lip smack] Okay. Very interesting. So it's really about kind of developing your preference-
- JBJacob Bank
Yep
- AGAakash Gupta
... for models for specific tasks, and there are some basic guidelines out there, like you just mentioned, like those models tend to be the best at those specific things. But that's also gonna change, perhaps by the time this episode is published or even changing-
- JBJacob Bank
Exactly. That's why I'm-
- AGAakash Gupta
So-
- JBJacob Bank
... I'm very hesitant to give specific model advice, 'cause anything I tell you will certainly be wrong three to four months from now. [laughs] I think the more-
- AGAakash Gupta
Yeah
- JBJacob Bank
... the more important thing is that we all develop a skill set of when a new model comes out, run it through a few test cases or go through a couple of our existing agents. It's very easy to swap out the model, like I just showed you. Do a couple. Like, they don't have to be super disciplined, quantitative evals. Just, like, run the meeting briefing generator with a few different models and see which one, uh, see which one performs best. Because these, y- as you can see, I haven't tweaked this workflow in a while, but I'm sure now that I've switched from 3.5 Sonnet to 4 Sonnet, they're gonna be a little bit better. [laughs]
- AGAakash Gupta
Amazing. So last question about this is the sequence and LinkedIn icons that we see. What are going on with those tools?
- JBJacob Bank
Yeah. So this sequence here is a sub-workflow, and that sub-workflow is a little helper I've created that automatically looks up LinkedIn profiles based on email addresses. And, uh, it only works for work email addresses, but the technique is, uh, let's say you're searching for my LinkedIn profile, and you know my email address is jacob@relay.app. The way you would probably do that is you'd go to Google, [laughs] you'd search jacob@relay.app LinkedIn profile, and my profile will be the first one there. So this little sequence or sub-workflow generates a couple Google Search queries that are likely to yield the right LinkedIn profile. It executes those Google Search queries programmatically, and then it gets the most likely LinkedIn profile URL, which then gets passed into a data fetching step that takes that likeliest LinkedIn profile URL and fetches the actual profile data from LinkedIn, which will include my title, my experience, my education and everything. So that's a little, a little set of building blocks that all combine to fetching the personal dossier from LinkedIn on the person.
- AGAakash Gupta
Love it. So if we go back to the executive assistant, because I was just mind-blown by-
- JBJacob Bank
Yeah
- AGAakash Gupta
... how you've put 12 agents together to replace an employee. What are some of the other ones in here? What do they look like?
- JBJacob Bank
Yeah. Uh, another good one that's also in the, the meeting space is my follow-up drafter. This was something that I wish I had discipline around my whole career, but I always forgot to do. What I wanna do is, after every meeting, within certainly 24 hours, but ideally more like two hours, everyone should get a follow-up email from me. And that follow email, uh, follow-up email should summarize anything that we talked about, definitely summarize any commitments that I made or they made, and include any notes, recording, or transcript that's valuable. And I would say before AI, my follow-through on this [laughs] was, like, I don't know, 25%. Because you're, you're busy.
- AGAakash Gupta
Yeah.
- JBJacob Bank
You're in back-to-back meetings. Like, you just, you just don't think of it. And so I built an AI agent to do it for me, and now my follow-through is, like, 95 or 100%, and I can tell you why it's not always 100%, for good reason. And so here's how this AI agent works. And again, it's super simple. A simple step-by-step workflow with two choices that the AI has to make inside of it. I use Fireflies as my AI meeting notetaker, and so the trigger for this agent to wake up is when a new transcript is created in Fireflies. Now, when I first built this workflow, I jumped right to step five. I said, "Once the transcript is created, write a follow-up email and then stick it in my Gmail drafts folder." And then I realized I needed a couple of these intermediate steps. I needed a step to figure out if we should follow up at all, because there's a bunch of reasons where a follow-up isn't actually appropriate. It could've been a no-show. It could've been just, like, a coffee chat catch-up that would be kind of weird to follow up with a recording and transcript. It could've been an internal team meeting that is ca- that the notes are captured elsewhere. And so the first thing I added was an AI step that says-Review this transcript, AI, and decide if a follow-up is even appropriate in this case. And I gave it some guidance that said, "In general, we should follow up whenever we meet a user, customer, or prospect, and we should not follow up if it's a no-show, uh, or an internal team meeting." If no follow-up is required, we're done. End the run, stop doing work. No need to do anything further. If there is follow-up required, we actually need to go to Google Calendar and look up the event that matches this title, 'cause it turns out that from Fireflies, we get the title of the event and the content of the transcript, but we don't actually get the email addresses of all the attendees. So we need to-
- AGAakash Gupta
Mm
- JBJacob Bank
... fetch those separately from the Google Calendar event. That's what this find event step is for. And then, once we have the transcript, once we know we should follow up, once we have the attendee details from the event itself, then we can ask AI to write the follow-up draft. And what I wanna show you here is this prompt is ridiculously simple. And, uh, and I'm sure people will argue with me on this, but, like, in my personal experience, a very simple prompt with one to three good examples at the quality of models that we have today is plenty good. The longer-
- AGAakash Gupta
Mm
- JBJacob Bank
... the prompts I make [laughs] and the more convoluted they become, somehow the worse my, my output is anecdotally. And so I wrote a very simple prompt. "Given the attached transcript and notes, please write a po- polite follow-up message and keep it concise." And then I gave it a few examples of previous follow-up notes I've written that I thought were good and matched my voice. And then it sticks a draft in my Gmail. Now, I chose for this to be a draft in my Gmail rather than an email that gets sent automatically, 'cause I still wanna have that final human in the loop opportunity to edit the message before it goes out. There may have been some additional crosstalk before the transcript started. There may be some other point that I wanna emphasize. And there's a general principle for me that, like, whenever I'm sending an email to a customer or a partner or something that's a high-stakes communication, I always review the result of an AI step before I actually send it.
- AGAakash Gupta
So I think this is actually really important for people to just realize is n- let's not use a note-taker that just automatically sends 80% done summaries to everybody. Add in the human in the loop step so that you can make that final 20% upgrade as needed. And I'm actually curious, you mentioned the crosstalk. Would it be possible for us to go into that and modify that workflow to potentially fetch, like, did we have a conversation in Slack since then or something like that?
- JBJacob Bank
Oh, and wait. Tell, tell me more. So, like, uh, you wanna add to the follow-up any emails that, that happened as well?
- 15:05 – 18:52
Ads
- JBJacob Bank
Today's episode is brought to you by Miro. Let me ask you something. How many tools are you juggling just to get a single project across the finish line? One for brainstorming, another for planning, something else for tracking tickets. That's where Miro comes in. It becomes an all-in-one collaboration workspace. Whether you're consolidating user research from several interviews, developing and synthesizing product briefs or a wireframe, or project managing development, Miro brings everyone into the same space. It's fast, intuitive, and fully loaded with features like project templates, two-way Jira sync, and integration with software like draw.io and PlantUML. Miro's AI features can be used to synthesize elements in a board to develop a ready-to-review product requirements document in seconds. If you're tired of tab overload and scattered workflows, try Miro. Head to miro.com and see why over 90 million users choose Miro to guide from idea to outcome. Today's episode is brought to you by Jira Product Discovery. If you're like most product managers, you're probably in Jira tracking tickets and managing the backlog. But what about everything that happens before delivery? Jira Product Discovery helps you move your discovery, prioritization, and even roadmapping work out of spreadsheets and into a purpose-built tool designed for product teams. Capture insights, prioritize what matters, and create roadmaps you can easily tailor for any audience. And because it's built to work with Jira, everything stays connected from idea to delivery. Used by product teams at Canva, Deliveroo, and even The Economist. Check out why and try it for free today at atlassian.com/product-discovery. That's A-T-L-A-S-S-I-A-N.com/product-discovery. Jira Product Discovery. Build the right thing.
- AGAakash Gupta
Yeah, something like that. Like-
- JBJacob Bank
Yeah
- AGAakash Gupta
... since the meeting happened, just double-check if there were any emails sent between each other.
- JBJacob Bank
Yeah, exactly. We would use the same exact technique that we used for this find event step. We said, "Hey, look up, look up any events that, um, look up the event that matched this title." Now let's say we also wanted to find emails exchanged between these participants. I would add another step. I would type find email, and now I'm searching in my Gmail, and I'm looking for any emails that match the criteria, and the criteria I wanna look for are, um, did the from address contain any of the event guest emails? If no-
- AGAakash Gupta
Nice. That was easy
- JBJacob Bank
... if no emails are found, continue. If more than 50 emails are found, just continue with the first 50. I don't think that's gonna... Oh, and I actually should, um, I need to add one more thing because you, you mentioned a time-bound. You want things that-
- AGAakash Gupta
Yeah
- JBJacob Bank
... have happened recently, right? Not emails that ex-
- AGAakash Gupta
Yeah
- JBJacob Bank
... that happened a l- a long time ago. So I wa- wanna also say, and the send date time comes after-
- AGAakash Gupta
The event time
- JBJacob Bank
... the current date minus, let's say, like, I don't know, two hours. I want, I only wanna check for, like, super, super recent emails. And then I can add that, those emails as context to my AI step, and I can say something like, "If there was already a follow-up email from the other person, uh, take that account, uh, take that into account in my message."Because if they've already followed up and said, "Thanks so much for the meeting," I don't wanna send another, "Hey, thanks so much for the meeting," without referencing that they already said, "Thanks so much for the meeting," if you know what I mean. And so that was the process of modifying that, that workflow to pull in some additional emails and get a new draft,
- 18:52 – 25:19
Building Follow-Up Drafter Agent
- JBJacob Bank
and then I have to reconnect the email addresses, and I have to-
- AGAakash Gupta
And the way Relay works, do I need to put in my API keys for Anthropic and OpenAI, or is those API calls part of the PM paying Relay?
- JBJacob Bank
You have two options. By default, we serve a non-technical audience who doesn't know what an API key is or what an OpenAI platform account is. So we maintain accounts with all the major models, certainly OpenAI, Anthropic, uh, Gemini, but also, uh, kind of purpose-specific models like ElevenLabs or Assembly for speech, et cetera, et cetera. And so the default usage pattern is that you'll use our AI credits. You pay us, we pay the model provider. If you want to use your own credentials for some reason, like you have a fine-tuned model, or you have a particular developer tier level, or you have some custom agreement with OpenAI that governs your data use, then you could, as an advanced user, connect your own API key.
- AGAakash Gupta
Nice. So that's the EA. I think we got people really good insight into how that works. What's the next agent that we would wanna build?
- JBJacob Bank
So the next one I wanna show everyone is a competitor pricing tracker, and the reason I think this one is really cool is the EA use cases are things that you kinda have to do anyway, and the AI is saving you time. With competitive intelligence, these are things that all PMs, at least in my career, you, like, wish you had time to do but never actually have time to do. [chuckles] And so the AI is providing you, like, a net new capability that you didn't have before. And in this case, we have a competitor pricing tracker because our competitors are changing their prices all the time. They're changing the amount of included credits or tokens, or they're changing their tier structure, or they're adding free tiers or removing free tiers, and it's really hard in a hot space to stay on the pulse of that. So here's how the competitor AI, the competitor pricing tracker works. I choose to run this monthly, 'cause monthly is the right cadence for me. You could run it, if you have a super fast-moving market, you could run it weekly or even daily. But for our market, monthly is sufficient. And then, uh-
- AGAakash Gupta
Mm-hmm
- JBJacob Bank
... I have a Google sheet, and the Google sheet includes, um... Can you see the Google sheet I'm presenting now?
- AGAakash Gupta
Yep.
- JBJacob Bank
Yeah. It includes all of our main competitors and a summary of their current pricing. Can you see as I'm scrolling, scrolling through this?
- AGAakash Gupta
Yep.
- JBJacob Bank
Then I'll go back to the agent. So what the agent does is every month it wakes up, it looks at the row in that sheet, and then it goes to that pricing page and scrapes the current information from it, and then has an AI step that, given the raw text of that pricing page, provides a detailed human-readable summary like the ones I had in that spreadsheet. So that's the basics. Go to the website automatically every month and scrape down their pricing. Then this is, this is where it's cool. If we haven't found the pricing information for that competitor in the past, like let's say they just launched or something, or we only started adding them to the tracker now, we're gonna add that pricing summary, um, from scratch. If we have already, uh, documented pricing for that particular competitor, we're gonna ask AI, we're gonna tell AI, you can see the prompt here, "You will be given two summaries of the pricing of this company. First will be the previous summary from last month, and second will be the new summary from this month. Please determine whether they're the same, [chuckles] and if something material has changed, like, tell me and why." And so then, uh, in addition to updating the spreadsheet with the latest pricing, if they're materially different, we get an update in Slack. So for example, one of our competitors or alternatives, Gumloop, recently introduced, uh, a lower price individual user plan. And so we got an immediate notification for, from this in Slack, like, "Hey, Gumloop introduced this new plan. Here's the pricing details of it. Here's how it co- it compares to their previous plans." And so this is just, like, one more thing you can take off of your plate and delegate to an AI to, to do for you.
- AGAakash Gupta
Wow. And for any pricing PM, this would be a gold s- send. But if you're not a pricing PM, I imagine you could just change the prompt to whatever element it is. So, like, for instance, when I was at a firm, I really cared about, well, what are the loan terms that-
- JBJacob Bank
Total-
- AGAakash Gupta
... Klarna and Affirm-
- JBJacob Bank
Totally
- AGAakash Gupta
... are giving off.
- JBJacob Bank
Exactly.
- AGAakash Gupta
So I can monitor that.
- JBJacob Bank
Maybe you wanna care about recent customer testimonials or recent product features that were launched, or maybe instead of scraping their pricing page, you wanna scrape their YouTube channel, or you wanna scrape the founder's LinkedIn posts. Those all are all in the theme of competitive research that you can do. Whatever element of your competitor's strategy you care about, like, you can set up this exact structure of AI agent to keep yourself up to date on it.
- AGAakash Gupta
Nice. And can we see what a final summary looks like for the pricing?
- JBJacob Bank
Yeah. So let me go back to the sheet, and let me go back. Let me look at, like, Lindy, for example. Um, so this is the pricing plan summary. They've got a free plan, they've got a pro plan, they've got a business plan, and they have custom plan pricing, and then they have some additional services. And so these are live update every month. Let's see. I know Make just did a br- a, a big pricing change. Let me see if I can capture that here. They basically changed their pricing structure from operations to credits. Let's see if we've picked that up yet. One sec. Okay. Um, oh, no. So this one is still on operations 'cause they just rolled out the credit change today. But the next time, [chuckles] the next time this agent runs, it'll h- it'll, it'll tell me, "Make switched their model from an operations-based model to a credit-based model. Here's what you need to know about it."
- AGAakash Gupta
Amazing. I think this will be really, really useful for all sorts of people. What's the next AI agent people need to build?
- JBJacob Bank
Okay. The next one I wanna show you we're gonna build from scratch, and that is tracking your brand mentions on Reddit. I don't know if you've talked about Reddit much previously, but I think Reddit is the most misunderstood and underappreciated place on the internet. [laughs] Everyone has this image of Reddit that it's just, like, 14-year-old boys sharing memes, and of course, there, there are com- communities on Reddit for whom that's true. Um, but Reddit is, like, the definitive source of human information in interest-based communities, and it makes up, like, a huge set of the sources in LLM calls. So if you want your product to be positively mentioned by ChatGPT, you need to know how you're showing up on Reddit. And
- 25:19 – 30:57
Live Build: Reddit Brand Tracker
- JBJacob Bank
so what I wanna show you here is how to build, uh, a weekly brand tracker that's gonna show all the places where you show up on Reddit, give you a summary of how, of, of how your brand is appearing, and then it's gonna give you links, uh, for each of the individual posts in case you wanna upvote a post that's positive or if you wanna react to a post that's negative. So let me show you how to build that. First, like with the competitor pricing, uh, uh, tracker that I showed, we're gonna put this agent on a schedule. It's gonna wake up every week, and it's gonna check Reddit for brand mentions.
- AGAakash Gupta
What's the third agent people need to build?
- JBJacob Bank
So the third agent we're gonna build is a Reddit brand tracker, and this is super important because Reddit is one of the main sources of information for all LLMs. So if you want your product to show up favorably in ChatGPT, you better have good organic content talking about you on Reddit. So what we're gonna build here is an AI agent that runs every week, and it's gonna look up all the pa- the places our brand is mentioned on Reddit. It's gonna send us a summary report along with the links to all the specific posts and comments where we're mentioned. So if it's a positive post or comment, you can upvote it, and if it's a negative post or comment, you might wanna react to it and help correct any misconceptions about your product. So let's, let's just dive in and build it together quickly. It'll take about 10 minutes. So-
- AGAakash Gupta
Okay
- JBJacob Bank
... because this is gonna run weekly, we're gonna start with what's called a scheduled trigger. A scheduled trigger is very much like a recurring calendar event. So I'm gonna click Add trigger, and I'm gonna click Scheduled, and what you'll see on the right is an interface that looks just like a recurrence picker in a digital calendar. It says when is the first instance, and then how frequently does it repeat. And I'm gonna change daily to weekly, 'cause I wanna do this weekly. But if you wanna do daily brand checks on Reddit, you know, be my guest. You have this flexibility to, to run it whenever, whenever you want. Then the next thing we wanna do is actually pull the data down from Reddit. So I'm gonna add a step. I'm gonna type Reddit. And I'm gonna, uh, say Get posts by search query. And can you give me, uh, a brand that we wanna search? Maybe one of the products you've, you've worked on in the past.
- AGAakash Gupta
Oh. Yeah, let's do that. We were just talking about Affirm.
- JBJacob Bank
Yeah.
- AGAakash Gupta
A-F-F-I-R-M. Let's try that.
- JBJacob Bank
Affirm. Now, that might be tricky 'cause affirm is a general word, so should, should we, can we add, like, one other word to the search query?
- AGAakash Gupta
Oh, let's do Fortnite maybe.
- JBJacob Bank
Fortnite?
- AGAakash Gupta
Fortnite. Yes. Although it, it's gonna have so much content.
- JBJacob Bank
Yeah, yeah. Let's, l-let me, l-let me actually pick n8n. Um, I like n8n.
- AGAakash Gupta
Okay.
- JBJacob Bank
It's another alternative to Relay.app that your audience should totally check out. It's another agent builder that's more optimized for technical users. And this is one that I wanna monitor 'cause they're one of our main competitors, so I wanna see what people are talking about. And so in Reddit-
- AGAakash Gupta
Yeah
- JBJacob Bank
... you can sort content, um, in your search results, and if you're a Reddit user, these will look familiar to you, by relevance, by how hot something is, new, top, et cetera. Top is the most viewed overall. And so I'm gonna pick Top, 'cause I wanna see the, the posts that have gotten the most traction. And I'm gonna look for all the, the posts that mention n8n in the last week. If no posts are found, which is not gonna be the case for this query, continue without a result. And then I can do a quick test just to make sure some posts come back. And you can see here it found 25 posts, uh, an AI agent army, 12 AI tools that I use, this n8n workflow. So I can already tell, like, this is good content here about n8n. So now I have, I have the raw clay [laughs] with which my AI can mold something useful. So now, next I'm gonna create an AI step that analyzes those Reddit posts and writes a summary for me. So I'm gonna press the plus button. I'm gonna type AI, and I'm gonna type Write. And in this case, I wanna have this report sent to me as an email, for example. And I'm gonna say, uh, "Given a list of Reddit posts from the past week about our brand, n8n, please write a detailed report of our brand sentiment. Uh, include any key quotes, uh, top use cases, uh, or other notable posts." This is not the best prompt in the world, but this'll be enough to, to demonstrate what the, what the system can do. And now I wanna just, uh, pause here for a moment because I wanna show how creating a prompt is different in the context of an agent or a workflow than it is in the context of ChatGPT. There's two major differences. Number one, in the context of ChatGPT, we're having a live, interactive back-and-forth conversation, so if I don't get everything right in the prompt the first time, I have plenty of opportunities to refine and converse. When this is running in the agent or the workflow behind the scenes, it's gonna be one shot. [laughs] So anything you want included in the prompt, you better include here. The second is this, because this is one AI step that has steps that precede it and steps that follow it, you need to think really hard about the inputs and outputs. So now I've written a prompt, but I need to give this prompt the information it needs, the Reddit posts from the previous step, to actually produce the report I've asked for. So I click Data from previous steps, and I select Reddit posts. And then I can select the model. Um, and like I mentioned, I like Claude, uh, Sonnet for writing, so I'm gonna pl- uh, pick Claude, Claude 4 Sonnet. And you can see here that this is using, uh, our, our default built-in AI credits rather than my own connection.And I'm gonna hit-
- AGAakash Gupta
Yeah
- JBJacob Bank
... use this model. Now, the output here is going to be rich text, because I wanna have this in a nicely formatted email report. Now let me test-
- AGAakash Gupta
Mm
- JBJacob Bank
... this step to, to see what the AI's gonna write. And this'll take about 30 seconds, so we can, we can talk while it's going. [laughs]
- 30:57 – 35:49
Ads
- AGAakash Gupta
Before we dive deeper, let's talk about something every PM faces: getting alignment on product decisions. You know that feeling when you're trying to explain a user flow to engineering or justify a design choice to leadership, and you're just describing it with your hands? That's where Mobbin comes in. Mobbin is the world's largest library of real-world mobile and web app designs from industry-leading apps like Airbnb, Uber, and Pinterest. Instead of spending hours taking screenshots or hunting for inspiration, you can instantly find exactly how successful products handle onboarding, paywalls, checkout flows, whatever you're facing. Over 1.7 million product builders use Mobbin to benchmark against best-in-class products and show their teams proven solutions. Whether you need to convince stakeholders there's a better way to handle user activation or research how top apps approach feature discovery, Mobbin gives you the visual proof to back up your product decisions. Check out mobbin.com/aakash. That's M-O-B-B-I-N.com/A-A-K-A-S-H, and get 20% off your first year. Today's episode is brought to you by the AI PM certification on Maven. Run by Miqdad Jaffer, who is a product leader at OpenAI, this is not your typical course. It's eight weeks of live cohort-based learning with a leader at one of the top companies in tech. OpenAI just doesn't stop shipping, and this is your chance to learn how. Run along with product faculty and Mo Ali, the course has a 4.9 rating with 133 reviews. Former students come from companies like OpenAI, Shopify, Stripe, Google, and Meta. The best part? Your company can probably cover the cost. So if you want to get $500 off, use my code AAKASH25 and head to maven.com/product-faculty. That's M-A-V-E-N.com/P-R-O-D-U-C-T-F-A-C-U-L-T-Y. [laughs] So when it comes to creating this prompt and getting a really good one-shot prompt, what are the personal things you're doing to harden it besides testing and iterating?
- JBJacob Bank
For me, testing and iterating is 90% of the battle. [laughs] I tend not to agonize-
- AGAakash Gupta
Okay
- JBJacob Bank
... too much about prompt frameworks, um, or anything like that. But my general technique, which I didn't, I didn't fully showcase here, is I always give, like to give one sentence of context on what role the LLM is playing for me and what its goal is. So I should've included something like, "You are a competitive analyst," or a product marketer, or a product manager at blah company trying to do blah. That's super helpful. Then second, I describe the task and any particular guidance or parameters of the task. For example, the nec- the necessity of including key quotes. And then what I would typically do is I would give one or two examples of output that I thought were really good. That's my general prompt framework. One sentence of context, however many sentences are required, typically between two and six sentences of what the actual task is, and then, uh, an example or, or two. So let's see what the AI produced here. N8N brand sentiment report. Past week analysis. Overall brand sentiment, strongly positive. Positive sentiment, 88% of posts. High engagement. The top post reached 17, 18 votes, which is, like, crazy traction on Reddit. You can see that the main, uh, uh, detraction from N8N is that it has heavy setup and a high learning curve, which is, like, the main thing we counterposition against from Relay. We try to be much, much simpler. Uh, and users seeing help with implementation. It shows me about the top use cases. The AI-powered personal assistant was the most popular use case, and actually seeing this from my AI Reddit listener last week is what inspired me to make that post about my executive assistant that you saw a couple days ago. [laughs] So it was actually my, my competitive content research that gave me the idea to even package up my ex- executive assistant in that way and, and post about it. You can see some other information here. Here are some cool notable, notable quotes. Um, there's a cost consciousness theme, 'cause N8N is very good value, especially if you self-host. There are some technical developments. Like, this is, this is crazy good for writing a two-sentence prompt that we just kinda hacked together quickly, and I think this'll be really valuable for PMs. So now the last thing we need to do is we, we set up the trigger, which is gonna run every week. We fetch the posts from Reddit. We had the AI write the summary. And then the last thing I wanna do is send this to myself as an email.
- AGAakash Gupta
Yep.
- JBJacob Bank
And so I'm gonna say, uh, subject, N8N report. And then in the body of the email, all I have to do is pull the output of step three into the body of step four. Now this workflow is done. I turn it on, and every week I'm gonna get a report like that in my inbox of the most relevant posts about N8N in the past week.
- AGAakash Gupta
It's a thing of beauty. And the thing is, like, if you don't want a weekly, like you said, you can get a monthly or whatever, and so it's-
- JBJacob Bank
Totally. Like, let's say-
- AGAakash Gupta
Yeah, monthly
- JBJacob Bank
... oh, man, it, like, there's not enough motion happening weekly. Like, let's just switch it to monthly.
- AGAakash Gupta
Yep.
- 35:49 – 37:57
Managing AI Agent Notification Overload
- AGAakash Gupta
So I think that there's two sort of interesting implications with these AI agents. The first is that you're gonna be getting a lot more pings, whether those are-
- JBJacob Bank
Yep
- AGAakash Gupta
... Slack pings or email pings. And I think most people are gonna have to go from a little bit of pure maker day, uh, to use the Paul, um, the YC Combinator founder's framing, to somewhat of a manager day, right? 'Cause you're gonna be getting all these pings.
- JBJacob Bank
Yeah.
- AGAakash Gupta
So how do you handle all those pings and all this human-in-the-loop review that ends up coming up because of these agents?
- JBJacob Bank
Yeah, so two points there. One is for these scheduled tasks, I try to keep them on a really organized cadence. So I always have my competitive research land in my inbox on Fridays. I have, excuse me, my content ideas for the next week land in my inbox on Saturdays, 'cause I often like to make a post on Sunday. Then I have our support summary from the week land in my inbox on Monday, because on Tuesday [laughs] we have our team meeting where we talk about support. So I've kind of, for all the ones that run on a scheduled basis, those are actually pretty easy to deal with, because I kind of set them on a cadence that matches the rest of my work.For the AI agents that are operating ad hoc, for example, we have agents that every time we publish a YouTube video automatically generates a LinkedIn post. Those are natural to do kind of a human in the loop in the flow of work, 'cause typically I've just posted the YouTube video, and then I get a ping five minutes later saying, "Now do you wanna [laughs] a- approve the LinkedIn post as well?" Then the-
- AGAakash Gupta
Nice
- JBJacob Bank
... the second technique I wanted to mention is [smacks lips] I've also built an agent or set of agents that then synthesize and aggregate [laughs] all the other stuff. So one of my examples is, like, I'm subscribed to many newsletters, including yours. [laughs] Um, and there's a lot of content hitting my inbox every day. And so I have a newsletter summarizer agent that every day summarizes all the newsletters that have come in that day. It sends me a digest at 5:00 PM, which is, I know, a good time for me at the end of my workday. It includes, like, the three-sentence summary of each with a link to drill in if, if I want to. So I've a- in addition to my AI agents creating information overload, I also use them to help me, help me manage external information overload. [laughs]
- 37:57 – 41:00
Current AI Agent Limitations Explained
- AGAakash Gupta
Love it. And then there's been insane hype around agents. We hyped agents a little bit throughout this episode. I wanna end on the limitations-
- JBJacob Bank
Yep
- AGAakash Gupta
... because there are a lot of limitations. Obviously, you know, your job hasn't been replaced.
- JBJacob Bank
No.
- AGAakash Gupta
My job hasn't been replaced quite yet. Who knows about content writer? Might be a little bit sooner than CEO.
- JBJacob Bank
Yeah.
- AGAakash Gupta
But what are the limitations today of AI agents?
- JBJacob Bank
So AI agents are still, in my experience, not ready to take on very complex tasks in a totally autonomous way without a human in the loop. Um, and, and there's two components to that that are important. One is there's a spectrum between a workflowy experience and an agenty experience. Now, a workflow, the technical definition of a workflow is that the flowchart of the task is predefined by the workflow creator. You'll notice that everything I showed you here was technically a workflow. We say step one, then step two, then AI step three, then step four, then step five. Whereas in an agent, you would just say, "Hey, agent, your goal is to monitor my brand mentioned on Reddit. Your tools are you can fetch Reddit posts from time to time and write summaries from time to time. Good luck. Go." I personally have not seen how our customers have success with that level of breadth and autonomy for agents unless they are very, very good at prompting.
- AGAakash Gupta
Mm-hmm.
- JBJacob Bank
So my recommendation to everyone just getting into it is don't just write -- Don't, don't just try to jump right from being a ChatGPT beginner to being, like, a master agent prompter. Go via, [laughs] go via AI-powered automations and, and, and workflows on, on the journey there. And then second, for anything that is high stakes, keep a human in the loop. I never put anything on my LinkedIn profile that I haven't reviewed first. I never send anything to a customer that I haven't reviewed first. The, the way I would think about human in the loop, the necessity of human in the loop is think about two axes. One is how good is the AI likely to be at that task, and the second is how high stakes is the task? So if the AI is very good and the stakes are low, do it fully autonomously. That competitor pricing tracker example is a perfect example. AI is very, very good at it, and the stakes are low. Like, if I get that competitor's price in the report wrong one month, it's not the end of the world. So for that, I had no h- human in the loop. Fully autonomous. For the meeting follow-up, the AI is pretty good at it, but not perfect, but the stakes are high. Like, if I have a customer meeting or a partner meeting, I really wanna make sure that that follow-up email is high quality, so I always have a human in the loop for those. So those are the two things I would think about for people who are just getting into agents. Number one, lean towards workflows more than truly agentic experiences to give yourself the control of the flow and the steps. And then number two, for anything high stakes or that you're not confident AI can do well, leave yourself in there as a, as a human in the loop until you build confidence in the system.
- AGAakash Gupta
Amazing. For people who haven't been tracking your journey along the way, how -- Can you share some stats? How big are you guys? Who is using you?
- JBJacob Bank
Yeah. So we have low thousands of customers,
- 41:00 – 49:11
Relay's $2M ARR with 10-Person Team
- JBJacob Bank
so we're not a, you know, we're not like a Lovable or a, a, a Bolt or a V0, but we have, we have material traction. Our lane in the market is we are trying to ser- be the most intuitive, easiest-to-use product for less technical audiences. So that means less technical folks at technology companies. And by less technical, I mean people who aren't comfortable writing a ton of code. [laughs] So that's, um, marketers, PMs, support people, salespeople, HR operations. Roughly half of our customers are non-technical roles at technology companies, and then roughly half of our customers are real-world non-technical [laughs] businesses. Real estate, lawyers. We have contractors. We have dry ice salesmen. We have professional basketball teams. We have carbonated beverage manufacturers. Like, there's quite a long tail of, of real-world small and medium businesses. So we lean small and medium business. We lean towards a less technical audience, whereas N8N, for example, really emphasizes it's the tool for technical people who know their way around a JSON object or an HTTP request. And so if that's you, you should totally pick N8N. [laughs] But if you want a simple, easy-to-use, intuitive way to get started and build the workflows that we showed, that's what we aim for Relay.app to be.
- AGAakash Gupta
Okay. And are you fundraising? How are you building this thing?
- JBJacob Bank
So we raised a seed round, um, of $8.8 million that was co-led by Coastal Ventures and Andreessen Horowitz. Coastal Ventures was the VC that backed my, my previous company, and then I met Andreessen Horowitz, uh, along the way. And our current plan is to become profitable and just run a great business. Um, and, uh, uh, the reason I say that is I currently don't feel like we're capital constrained with anything that we're trying to do. We have a, we have a very small team. We're 10 people, six engineers, two designers, a product manager, myself, and even though my background is in product, I'm doing essentially everything but product at the company. Marketing, support, sales, HR, finance, operations, et cetera. And here's, here's my claim.Um, we're gonna see way smaller teams in the future [laughs] than we saw in the past. We're already seeing this trend right now, and, you know, we're at thousands of customers now with 10 people. I can totally see us getting to tens of thousands or even hundreds of thousands of customers with the exact same headcount. Maybe one-
- AGAakash Gupta
Mm
- JBJacob Bank
... more engineer, maybe one person to coordinate all of our support agents, but I see no reason why we need to be a 30-person team or 100-person team. Because, you know, my experience, I worked at Google for many years, and my experience of working at Google was things are hard, not because there's a lot of absolute work to be done, but because there's a lot of people that need to agree to get anything done. [laughs]
- AGAakash Gupta
Yeah.
- JBJacob Bank
And so, so our current plan is to just ru- And, and when I say we wanna beco- be profitable and not raise future funding, that doesn't mean that we have a lack of ambition. Like, I wanna build a ubiquitous product that's u- b- used by millions and millions of people. I just don't feel like capital is the constraint that's stopping us from, from doing that our own. We just need to do hardcore product work for the next several years with a small, really motivated team, and then figure how to speak about it effectively and market it to the, to the world.
- AGAakash Gupta
Because some of your competitors, they are fundraising a lot more. How do you think about that? Does that add pressure to it?
- JBJacob Bank
Um, it's something I go back and forth on in my mind. I'm always happy when I see a competitor fundraise. [laughs] It's like, uh... And I'm, I'm happy for two reasons. One is because it's validation that the space is important and that we're comp- like, we're competing in an area that matters, 'cause it means that... So, so for example, N8N just raised a huge round at a $2.3 billion valuation. Lindy has raised at a big valuation. Gumloop recently raised a Series A. Relevance AI d- recently raised a Series A. Casty r- recently raised a Series A. Wordware raised, you know, a huge C round coming out of YC. I see all of that as, as... The only thing I, I read from that is there's evidence that this is a really compelling and exciting space, which is the place that I wanna be. I don't wanna be competing or I don't wanna be around the fringes [laughs] where there's, there's no one else competing. Like, I wanna compete with the best for a market that matters. So that's one interpretation. And the second interpretation is it also helps me reinforce the lane that makes sense for us. So we are not gonna win today by hiring the biggest enterprise sales force. That's not our strategy. That's not what we're trying to do. We are trying to win by building the most intuitive, simplest, and easy-to-use product. And I'm curious to hear your experience. In my experience, I've never seen a bigger team build a simpler or more intuitive product. I've seen bigger teams build more features and more go-to-market motions and spend more money on ads. And so for me, our difference in fundraising strategy is also informed by our difference in product strategy, which is, for me, you know, I saw this tweet that really bugged me. It was from a founder who was like, "I wanna prove I can build a great company without VC money, just like people prove that they could climb Everest without having oxygen." I don't think that's the right mindset. [laughs] Like, the... It's, building a startup is hard enough. Like, we don't need to make it any harder just to, like, show how tough we are. That's not the reason we don't wanna raise money. It's not because I wanna tr- prove that I can climb Everest without oxygen. I just think for our particular strategy to maintain the most intuitive, easiest-to-use product, keeping a small product team [laughs] is, like, the best way to achieve that goal. It's not... We're not depriving ourselves of oxygen. We're giving ourselves the right kind of oxygen, if that makes sense. [laughs]
- AGAakash Gupta
And you mentioned it is such a red-hot space. It is on the fringes. For people who haven't really grokked it, and actually, I polled my community-
- JBJacob Bank
[laughs]
- AGAakash Gupta
... uh, two weeks ago now.
- JBJacob Bank
Yeah.
- AGAakash Gupta
And I asked them, "How many of you have built an AI agent?" You know, 2% of them had. [laughs]
- JBJacob Bank
Yeah.
- AGAakash Gupta
So even amongst PMs who are reading AI PM certificate-
- JBJacob Bank
I, I know. It, it, it's, it's so early. But let me make, let me make my, my best bull case for this market. I've worked on productivity tools my whole career. Productivity tools my whole career were a place where you go to click buttons to do your day job. You go to Gmail, you click, you click some buttons, [laughs] and you send some emails. You go to Salesforce, and you click some buttons, and you update some opportunities. You go to Google Docs, and you, you type some buttons, and then, uh, a PR, a PRD comes out. A productivity tool was always just, like, an interface in which you clicked buttons to do things. And yes, AI-powered productivity tools are really cool. ChatGPT is a place where you go and click buttons, and oh my God, it's AI-powered, so the results that come back are, like, pretty phenomenal and amazing. And Copilot experiences, whether it's Lovable or the Cursor autocomplete or Grammarly are amazing because you're in this canvas of your IDE, and now you have this coding assistant that'll help you write faster. But fundamentally, you are still going to a place and clicking those buttons. An AI agent is working for you automatically behind the scenes. This is g- this is the first time we've achieved a technology that is, like, actually equivalent to hiring someone. Like, I am, I am our entire marketing team, and y- you, you see me on LinkedIn, you see me running webinars, you see me sending emails to our newsletter, you see me doing customer support. Like, there's no way I could do all of this if I didn't have my AI team behind me. We would probably need a four or five or maybe even 10, f- 10, uh, person, full person, uh, full, full-time team to do this. And so I don't think that has quite clicked in people's mind how big the opportunity is. Like, we are finally taking the leap from being a nicer environment to go click buttons to something that actually does work on your behalf. And it's hard. Like, it's a ment- it's a mental leap to figure out, like, what is the job description for this AI agent? How do I give the AI agent the training that it needs? How do I give it the instructions that I need? And I think there's a lot of people that are professionally overcomplicating building AI agents. And so one of my goals in showing you the, the, the Reddit listener that we built in, what, five minutes, is like, it's not as hard as people think. Like, if you have some basic skills with ChatGPT and you've built some basic workflows, uh, in Zapier at some point in your career, you have all the tools you need to build your first one, two, five, 10, 100. I have about 250 AI agents working on my behalf right now. Um, and so that's why I think, like, we're still very much in, in the visionary phase of the market from a customer adoption perspective, but, like, as soon as we hit the, the early majority, it's gonna start to, it's gonna start to explode. M- my hope and prediction. [laughs]
- 49:11 – 54:52
Why PMs Are Losing to Sales/Marketing Teams
- AGAakash Gupta
What are the pockets? Who are the people? What are the industries? You know, you mentioned things like real estate lawyers. What are those places that they are actually uptaking AI agents faster? I was surprised PM was so low.
- JBJacob Bank
[sighs] I think the fastest by far is in go-to-market functions. That's certainly customer support and customer service. I mean, look at Intercom Fin and Decagon and Sierra. Like, that is already a red-hot market with, you know, many, many millions of, of, of, of ARR going around. So, I think support teams are very ahead of the game because they already have a mental model for how, how a human gets work done within a support team and what the metrics are, and those are places where an AI can often provide faster, higher quality service at, at a lower cost. And then I think, uh, sales teams and marketing teams are also adopting AI tools way faster than PMs are. Sales teams espec- uh, especially, whether it's for prospecting or lead qualification or lead routing, or in marketing, competitive research, content research, content generation. I think from what we see in our data, like those three functions are the furthest ahead. And I hope this lights a fire under your audience, 'cause I know PMs, like, we always think of ourselves as the innovators and the technology people, and, and every company I've always worked at, there's been this sort of like friendly, I don't know about rivalry or like, you know, joshing each other between the go-to-market side of the business and the product engineering side of the business. And the product engineering side of the business is always like, "We're the technologists. We're the visionaries. We're so far ahead." [laughs] "And like you salespeople, you don't get it." And I think in, when it comes to adoption of AI tools, I think it's the opposite right now. Like, I think the customer-facing teams are leading the charge, and PMs need to get our act together. Um, now part of that is because AI agents, like the ones I showed you, are best fit, are best fit for like frequently recurring tasks that share a very common pattern, and I think those are more common in sales marketing support than they are in PM. But I think that also is just a lack of imagination, that we just haven't tried hard enough to think about the elements of the PM job. And, and the way I always think about it is like we're all gonna use AI in three modalities. We're all gonna use chatbots, ChatGPT, Claude, Gemini, Grok, pick your favorite. We're all gonna use copilots. Every single one of us should be living, using Lovable. Every single one of us should be bus- be using Descript for video editing or, you know, pick your equivalent in, in category. And we're all gonna be using agents. And the challenge for all of us is to figure out like what is the right tool for the job. So, if I'm writing a PRD, I don't want an agent to write a PRD for me. It's not that frequent of a use case. I don't need that level of autonomy. I wanna use something more like a Claude project or a custom GPT, and I wanna have an interactive, uh, e- experience. On the other hand, if I wanna get a weekly report of all the open P1 bugs affecting a certain class of VIP customers, like have an agent do that. Like, there's no reason for you to be manually doing that same SQL query and writing that same email to your leadership every single week. Like, make an agent do that. You review it. [laughs]
- AGAakash Gupta
Yes. There were so many jobs I had early on in my PM career, which were literally like take emailed scheduled SQL query-
- JBJacob Bank
Yeah
- AGAakash Gupta
... put it into Google Sheet, update PowerPoint slide. [laughs]
- JBJacob Bank
I mean, every single PM-
- AGAakash Gupta
Like all of that
- JBJacob Bank
... we all have 10 of these, right? Like whether it's what have competitors launched recently, what pricing changes have happened recently, what are all the features we've launched recently, put it in our release notes. What are all the bugs that we fixed recently? What are all the bugs that have come in? Who are all the new customers this week? Who are all the upgrades this week? Who are all the churns this week? Like, anytime you find yourself writing weekly in the [laughs] in like an email subject, like that should be a mental, like that should be an immediate signal, like I should have an agent writing this for me. [laughs]
- AGAakash Gupta
Love it. So, it's all of the recurring tasks. So-
- JBJacob Bank
Yeah
- AGAakash Gupta
... I guess if we had to summarize like the top, I don't know, 10 for PMs, I'm just thinking off the top of my head, and then let, help me finesse this, right? There's probably, like we just said, like the, your executive assistant, right? So, that's your meeting prep-
- JBJacob Bank
Yeah. And that's, and that's a whole bucket of managing your meetings, managing your calendar, managing your tasks, finding time for your tasks, notifying you about tasks that have been done, managing your emails, summarizing your newsletters, filtering out cold spam email. Like, I would say, you know, I have my executive assistant of 12. I think that's about right because I don't think anything I do in email, calendar, or tasks is unique to me. Like, those are things that we pretty much all do in our work. So, that's one. Executive assistant is one big bucket. Then competitive understanding is another big bucket where, like you said, you might wanna be tracking competitor interest rates or competitor pricing or competitor feature launches or, you know, uh, because content marketing is so important for us, whenever a competitor has a LinkedIn post or a Twitter post or a Reddit post or a YouTube video that hits, I wanna know about that. So, there's a whole cluster around competitive tracking. Then there's a whole cluster around, um, support and user feedback. So, for example, synthesis from user interviews, synthesis from user feedback in product, synthesis from support tickets, synthesis from organic brand mentions on Reddit, like I showed you. Like customer, customer understanding is, is, uh, another, another, um, big bucket. And I think the third big bucket is, um, project management and the cadence of, of the PM, the, the, the cadence of the PM job. Metrics reports, monthly product newsletter, weekly, you know, uh, release notes for the iPhone app, quarterly update for VPs or other stakeholders. The sort of, I guess you would call it like product rhythm of business, rhythm of business type use cases. Those are the four big buckets I see PMs, um, benefiting most from with agents. And within each bucket, you could have anywhere from one to 20 agents within each bucket, depending on how deep down the rabbit hole you wanna go.
- 54:52 – 1:00:40
Choosing Between Agent Platforms
- AGAakash Gupta
So, PMs often, they don't have the ability to choose which tool. So, let's address this specifically for the product leader.
- JBJacob Bank
Yeah.
- AGAakash Gupta
If I'm the product leader, I'm figuring out, should I get my 20 PMs a Relay license or a Lindy license or a Make license or a N8N license?What's the difference between these tools?
- JBJacob Bank
So first, I think you should offer people all of the above. Because the way these tools charge is it's not per seat. It's not per seat. It's based on usage. And so the base subscription for any of these plans is super cheap, like 20 to $100 a month for a, for a team plan [laughs] where, like, anyone can use it, have a login. It only gets expensive when you start using it a lot, but that's when it starts accruing a lot of value. So if I were a PM manager, I would not say, "We all use Zapier," or, "We all use Relay," or, "We all use n8n." I would say like, "Hey, swipe your credit card, pay 20 bucks for, for whichever one you want." And then yeah, if there becomes, like a, a, a winner emerges, then you can convert it to a team plan. But I, but I-- But this is not the kind of case where, like, you have to buy one 50K-a-year subscription and then get the whole team on that. So my, my quick answer is, like, buy all of them, let people try all of them, and experiment. And then when you're looking at the space, I think there's, um... The main axis of decision-making, I would say, is how technical is the team that is gonna be using this tool. If you have a more technical team, I think you wanna lean towards the n8n, Makes, and Gumroads of the world. If you have a less technical team, you wanna lean towards the Zapiers, Relays, and Lindys of the world. That's like the first highest order split. Then let me take our not, our less, our less technical world of Zapier, Lindy, Relay, and I'll tr- I'm gonna try to be really even-handed for everyone. Zapier, it's the no one got fired for i- buying IBM product in this space. And I don't mean that in a bad way. I mean, they have the most integrations. They've been around a long time. They're executing really well. They have a strong team. But the product's 14 years old [laughs] and so th- you're just kind of, like, limited in how much UI flexibility you can have to innovate on top of, like, their core mechanics. So I don't... I, I mean that back when IBM, maybe 20 years ago, like when that was still, like, a really, really good [laughs] uh, default choice.
- AGAakash Gupta
Yeah.
- JBJacob Bank
Um, Lindy and Relay I think are both, like, more intuitive, nicer UI, more modern versions. Lindy leans a little bit more towards agentic ma- agentic magic, and Relay leans a little bit more towards define each of the steps of your workflow. Of course, you can also make workflows in Lindy, and you can also make full-scale agents in Relay, but that's the main difference. And so, um, when you're thinking about your, the ca- the technical capability of your audience and the tools you need to integrate with, that's the way I would kind of make the choice. And like, but, but, but again, the highest order bit is, like, let people choose their own tools, let people experiment with all of them, and then see which winner emerges.
- AGAakash Gupta
Mm. And if you had to make the pitch, Relay versus Lindy, what are the things you're getting in Relay?
- JBJacob Bank
So Relay has fewer features than Lindy. We don't have a built-in meeting recorder. We don't have voice agents. We don't have, uh, browser-based actions. Relay has a more limited product scope, but if you wanna build a basic agent that triggers based on your email or sends you meeting briefings or track your competitors or does the read a listener, we've built, I think, the simplest and most intuitive experience to get you started. Um, it's the easiest to create, easiest to test, and, and easiest to run. And so, so again, if you're beginning your AI agent journey, and you're looking for simplicity, ease of use, and intuitiveness, like, that's where we see people succeed with, with Relay.
- AGAakash Gupta
How long does it take to build, like, an executive assistant like that 12 agents we saw there?
- JBJacob Bank
Yeah. So for me, if we were doing, like, a timed challenge and I had to build those 12 from scratch, I could probably do all 12 in an hour. But that's not really a fair, a fair test, because I've already built them all myself, and I'm an expert in the tool 'cause I use it all day, every day. I think the, the better way to think about it is it's gonna depend on how experienced you are in ChatGPT and how experienced you are with workflow tools like Zapier. Let's say you have a moderate level experience with ChatGPT, but you have very little experience with a workflow tool. I would say building a high-quality meeting follow-up tool would take, a, a follow-up agent, would take you about an hour, and then subsequent agents will get faster. But the key point here, I think, is I don't think you should think about how long would it take to build one agent as sort of like a fixed quantity. I would really think about, how can I invest in agent building as a skill so that I can get really good at building these things quickly? Um, so for example, like if I wanted to build a quick meeting briefing generator for myself or a competitive researcher for myself right now, I've now built up the skills that I could do that in five minutes and, like, blow the mind of my peers and my managers, and I want everyone listening [laughs] to, to, to build up the skills that enables them to get to that point.
- AGAakash Gupta
Yeah. It's like a skill set. Like anything, I think you need to put in, I would say probably, I've probably put in like 30, 40 hours now across these various tools, so now I can build an agent in like 10 to 15 minutes, and I feel pretty happy about it. So it, it doesn't take as much time as you think.
- JBJacob Bank
Yeah, it's, we're not talking like a 10,000 hou- hours, like, being a performance violinist. [laughs] It's way easier than that. Like, if you spend, like, I, I run live Build With Me sessions every week. If you come to like three of my Build With Me sessions, you're gonna be top 10% at using AI agents, like, among PMs already.
- AGAakash Gupta
So you mentioned that you are Relay's
- 1:00:40 – 1:06:40
55-Agent Marketing Team Breakdown
- AGAakash Gupta
marketing team. That's actually mind-blowing. [laughs] So you need to break down for us what are the agents that are helping you. How are you a one-man marketing team for this 10-person team?
- JBJacob Bank
Okay. I got, I gotta show you this, because this was also my most viral LinkedIn post ever. It got 33,000 comments, where I showed an org chart of my, of my AI agent marketing team. Um, uh, basically it's a team of about 55 agents now that's organized by marketing channel.So our marketing strategy is an organic content community education-based strategy. So I have a set of agents that help me on LinkedIn, a set of agents that help me on X, a set of agents that help me on Reddit, a set of agents that help me on YouTube, a set of agents that help me with our email marketing and our newsletter, a set of agents that help me with our webinars, a set of agents that help me with competitive research, a set of agents that help me with our partner, our partner program and our affiliate program, and a set of agents that help me with our, our cohort-based learning. Those are sort of our main marketing channels. And in each of those buckets, there's-
- AGAakash Gupta
Wow
- JBJacob Bank
... anywhere between three and 10 agents. So let me talk about webinars specifically, 'cause, um, I'll tell you the process of a webinar at a typical company, and then I'll tell you the process of a webinar at Relay.app. In a typical company, there's like a product marketer and then an email newsletter marketer and there's like a webinar marketer and then there's the PM, and then there's the whole, this whole process of setting up the calendar many months in advance. And, um, then a month before, you do a pre-meeting and a pre-recording and all that. Here's the process that I use with, with my AI agents. I create an event on our relay.app events Google Calendar, and I put in the title of the event, which is the topic of the webinar. Like, uh, trigger, trigger, Triggers 101 is the one I did yesterday. Then here's what happens. One of my AI agents automatically adds that to our event signup page, which was vibe coded with lovable so that people can sign up for, for that event. Another agent automatically writes a LinkedIn post for my personal account four days before with a poster that's, again, AI-created announcing the webinar. Another one of my agents writes an email to our newsletter saying, "Here's this upcoming a- uh, session. Here's the topic." When someone signs up on that, uh, lovable vibe-coded signup page, another agent automatically adds them to the calendar event, which is like a standard Google Calendar event with Google Meet. It automatically sends them a confirmation email that they've been enrolled. It automatically sends them a reminder three hours before the meeting. It automatically sends them the recording after the meeting. We have another agent that automatically tracks all the signups and publishes them into Slack so we can see the LinkedIn profiles of everyone who's attending. We have another agent that tracks the attendance afterwards and lets us know how many people are showing up to know how many people from our team should come to ask questions. We have another agent that automatically turns the transcript into a YouTube description and then a follow-up social post of the recording. So those are like the eight or nine agents. And so our entire webinar process is like, I come up with a topic, I write it in a Google Calendar event, I show up for the meeting, and then my AI agents do all [laughs] of the, all of the other marketing around it. It's, it's pretty awesome. [laughs]
- AGAakash Gupta
Whoa. That's a lot of marketing channels.
- JBJacob Bank
Yeah.
- AGAakash Gupta
People say I'm productive, but you are really productive. [laughs] That's insane.
- JBJacob Bank
Yeah, and it took a while to set up this whole system, and it didn't, it didn't happen overnight. But I was like, "Oh, it would be cool if we automatically promoted these to our newsletter." Build that agent. "Oh, I should also promote them on LinkedIn." Build that agent. "Ugh, writing these YouTube descriptions is super annoying 'cause I have to like re- listen to the whole thing and make the chapters. Like, let me just tr- transcribe the audio with Assembly AI and then automatically write the chapters and descriptions." So I built up this motion over time, but now it's truly a joy to do webinars 'cause all I have to do is put an event on the calendar, show up, and like the rest of the value just, just happens. [laughs] And that's, that's why I was saying, you know, earlier that I- my mind has been blown by AI agents already, and yes, there are places where they're over-hyped and there's places where they overpromise, but like already I feel like I'm living in the future relative to the old world of like going to Google Docs and typing up a, a script and I'm just, I'm excited for everyone to get there 'cause it's so much fun. [laughs]
- AGAakash Gupta
It is so much fun. That's insane. So I wanna shift, because we've been talking a lot about how PMs use AI agents for productivity. But you also have a lot of expertise to help people with on building agents into their products. And I would assert that if you wanna build your product for the future that we're moving into, you need to be thinking about how to build AI agents into your products.
- JBJacob Bank
Yep.
- AGAakash Gupta
Do you agree?
- JBJacob Bank
Yes. I think, um-
- AGAakash Gupta
And so if that's true... Oh, go ahead.
- JBJacob Bank
No, no, no. I think it's true, and we'll talk about why. [laughs]
- AGAakash Gupta
[laughs] Why? Why is it true?
- JBJacob Bank
So the nature of everyone's work is gonna change, and our expert- our expectations of product tops- our, our expectations of productivity tools are gonna go way, way up. A, a place where you go to just click boxes will no longer be a sufficient product in the market. You will be expected to have some sort of chatbot experience, some sort of copilot experience, or some sort of agent experience. Now, I don't think, maybe I should caveat this and say, I don't think every product necessarily needs an agent experience in the product. I think every product needs a first-class AI experience that makes it dramatically easier to complete the user journeys within that product. So for example, Figma may not be as well suited to an AI agent, but it's very well suited to a copilot that automatically fixes the spacing on your mocks, or automatically adopts the right color palette, or automatically creates a nice prototype for it, or automatically produces the right component. Like, Figma doesn't necessarily need to have an agent that goes and works for you automatically in the background, but they definitely need some sort of AI copilot, and they may also
- 1:06:40 – 1:11:41
Building AI Into Your Product Strategy
- JBJacob Bank
need some sort of AI chatbot. So the way I would think about it if I were a product builder is if your product already has a very significant natural language interface, for example, it's a search, research, or discovery product, like almost certainly you should be trying to build a ChatGPT-style chatbot to go from like a static search box with results to an interactive, uh, text-based sparring partner. If you have a product, uh, upon which there is a canvas of creation, a document is being created, a mock is being created, some co- code is being created, you better have a really good copilot. [laughs] And that copilot better have really, really good like in-canvas auto-complete, and it probably should also have some side panel natural language copilot experience.And then if you are a product that does workflows or automations, um, for example, like email marketing sequences or project management automations of marking things as done, like you better be adding [laughs] some more AI and agentic capabilities into those. So, so, so maybe I was wrong to say that every product should necessarily have agents. Every product should be rethought through the lens of AI, but chatbot, copilot, or agent could be the right modality depending on the product and the interaction pattern.
- AGAakash Gupta
Hmm. Really helpful nuance there. And I think one other element of building an AI agent that people need to understand is you don't need to just build an agent into your product, actually. I think that maybe can you help explain MCP to folks? Because I think instead of building integrations to other platforms, what if you became the platform that AI agents integrate with?
- JBJacob Bank
Yeah. So let me touch on MCP briefly. I mentioned earlier that the definition of an AI agent is, uh, an AI system that is given a goal and a set of tools, and then figures out [laughs] how to combine those tools to achieve its goal. What is an example of a tool? An example of a tool could be asking a question of ChatGPT. An example of a tool could be running a web search. An example of a tool could be sending an email. An example of a tool could be looking up a record in Salesforce. And so the question became quickly is like, I wanna enable my AI agents with tools that touch lots of my SaaS apps, and there's basically two ways to do that. One way to do that is what we do in Relay.app primarily is you have API-based integrations, where every primary action that's exposed in, like, for example, the Gmail API, you can encode as a workflow action in your flowchart. MCP is a slightly different technological approach of achieving the same thing, of exposing the actions that can be taken in a product as tools for an AI agent. So if you connect to, for example, a monday.com MCP server, you could then write a prompt to your agent that's like, "Look up all the tasks that affect, that were involved in this project on monday.com, and update all their status to done." And then the AI agent would have a tool that can then make these MCP calls to Monday for it to take action on its behalf. So that's, that's the promise of MCP, is that you can go to ChatGPT, or you can go to Claude, or you can go to any agent-building tool like Relay.app, and you can, like, magically give prompt-based instructions to have real-world actions taken in all of your tools. In practice, I have not seen many people succeed with MCP in real-life workflows. I'm cur- maybe you've seen, maybe you've seen something, maybe you've seen something different. Um, I think the promise is very real and very exciting. It is still something that only, I think, the technical visionaries have turned into, like, day-to-day value. Like, I'm curious, what percentage of your audience would you say has used-
- AGAakash Gupta
Very low, very low.
- JBJacob Bank
Yeah.
- AGAakash Gupta
Again, that's where, like, that's why I wanna build this content, is to increase the percentage.
- JBJacob Bank
Yeah.
- AGAakash Gupta
I just feel like more people should become tools that these AI agent workflows are calling, you know?
- JBJacob Bank
Totally.
- AGAakash Gupta
Like, so many people, these SaaS apps, they're often, the way they end up building lock-in is they become, like, a data source of some sort. Like, you know, like a-
- JBJacob Bank
Ex- ex- exactly. And so-
- AGAakash Gupta
... a common, an, an AI company out there.
- JBJacob Bank
Yeah, exactly. And so for all you, all the product builders out there who have, who are working on a system of record, like a CRM or a work management tool, any kind of system of record or data store, you probably think a lot about your user interface for manipulating that data store and occasionally about your API. I would encourage you to think very hard about your API and very hard about your MCP server, because in two years, I bet many, many more actions will happen on your data via AI agents and workflows than via humans coming into your UI. [laughs] And so I expect we're gonna see APIs get way better and MCP servers get way better, and we're very quickly gonna see a usage pattern shift from primarily humans clicking buttons to primarily agents making a API or MCP calls.
- 1:11:41 – 1:15:36
Why Jacob Left Google Director Role
- AGAakash Gupta
100%. The last thing I wanna cover is a little bit about you, because you have a fascinating story. First of all, you left your job as a director of product management at Google. That has some pretty heavy golden handcuffs associated with it. Why did you leave your job as a director of PM at Google?
- JBJacob Bank
I left my job at Google for two reasons. So at the time, um, uh, I was the product lead, uh, for Gmail, for Google Calendar, and for several of other, um, productivity and collaboration tools under the Google Workspace umbrella. I left for two reasons. One is I felt like I wanted to personally learn and grow in a different direction. I'd been at Google for six years. I was managing a team of team of, of, of teams of PMs. I knew how to do performance reviews. I knew how to recruit. I knew how to manage people out. I knew how to run OKR reviews. I knew how to do product strategy presentations. Like, I felt like I knew that version of the job, and while it would've been... I could, I could kinda see what my life would be like five years from now, and I'm like, "Oh, shoot, it's gonna be the same job. [laughs] I'm gonna have, like, the same f- the same fundamental skills." So one big reason was I wanted to, to learn and grow, and I knew from my previous experience of starting a company that nothing compares. Nothing compares [laughs] to, to, to, to being a founder. And then second was I knew I wanted to build something that was a cross-product AI-powered workflow tool, and I did not feel like that vision could be successfully built at Google or at any other incumbent. Because at any incumbent, you always through, look through the lens of, like, "Oh, well, everyone's using our button-clicking interfaces, Gmail and Calendar and Docs and Drive. Like, let's make them click buttons in our product better." Um, I remember we'd have customer meetings where people would say, like, we'd say, like, "Hey-"Look at this awesome new reply button we built in Gmail. Look, it does three AI replies. Isn't that so cool?" And customer would be like, "Yeah, yeah, whatever. Like, your reply button is fine. My real problem is I have this data in Salesforce, and this data in HubSpot, and this data in, in Zendesk, and I wanna connect it all." And we'd be like, "Ooh, sorry, that sounds hard. Good luck. Use the API." [laughs] And so, um, uh, for... Those were the, the two motivators for me. One is I wanted to learn and grow, and only being a founder can really push you to learn and grow to, to the max. And then second, this cross-product, AI-powered workflow opportunity, I think, had to be tackled by, by a new startup rather than by any individual market leader in the productivity space.
- AGAakash Gupta
Mm. So you had that calling for this specific solution, which is interesting. I think a lot of people just have that, especially PMs, especially PMs who've reached kind of the level you've reached, which is, like, very senior in your career, they have that founder calling. And these days, I talk to a lot of PMs. I'd say probably, like, more than 40% of the PMs [laughs] I talk to these days, they're like... They're thinking, "Should I start a lean AI company like Jacob? Should I start a 10-person AI company that could potentially have a huge outcome?" What's your recommendation for PMs who are considering becoming an AI founder?
- JBJacob Bank
Yes. Do it. [laughs] Uh, no. Um, that's, that's, that's the fa- the facetious answer. The facetious, facetious answer is, like, yes, obviously, do it. There's more nuance to it. Um, you will learn and grow faster than in any previous role you've had in your career. You will build more skills faster than you have in every previous, any previous role in your career. You will do your best work in a way that you haven't been able to do anywhere. What will come with that is an intense feeling of being lost, and confused, and demoralized at least 75% of the time. [laughs] And so th- that's basically the trade-off you need to, you need to go into. Like, you, you're gonna have this amazing experience, and you're gonna come out really strong on the other side. But don't let any con- anyone convince you, like, it's all glamour and fun the whole time. [laughs] I remember, like, I'll give you a concrete example of that. Like,
- 1:15:36 – 1:19:09
Brutal Truth About PM-to-Founder Transition
- JBJacob Bank
when I was running Gmail, I could basically get a meeting with anyone I wanted for whatever reason, because everyone wants to appear or integrate with Gmail or have their emails delivered in Gmail. It's like, it's a position of, of, of authority in the technology world to be, like, the product lead for a top 10, a top 10 app. The very next day, when I was startup founder of unknown product, like, I could not beg a line marketer at a startup to spend half an hour giving me feedback on something. Like, this, like... You ha- you just have to be ready to, like, completely lose any ego that you have. You are going back to the beginning. Like, no, like, no, no one cares about the title of your previous job. No one cares about how many users your previous product has. Like, you are back at the zeroth, [laughs] the zeroth rung. And so that's, like, a humbling and, and, and fun experience. The other big thing... H- here, here's my other kind of hot take. I think being a PM at a large software company is the worst possible preparation for being a founder. I don't think there's any job that is worse preparation for being a founder. I think running a hotdog stand is being, is better preparation for being a founder. I think, you know, being a dog walker is better preparation for being a founder. Being a contractor is definitely better preparation for being a founder. The reason being a PM at a large company is so dangerous is, like, you think it's the same, 'cause you're like, "Oh, I'm the CEO of the product. That's what the PM job description says. It says I'm the CEO of the product." And you think you have these similar experiences of crafting product and doing user interviews, but it is completely different to manage the inertia of an already successful product than it is to create something from nothing. And so that's the other, uh, I hope people, I hope people see that I'm saying this in a way, like, an uplifting way, where it's, like, an opportunity to learn and grow in new ways. But I think that it's, like, it's gonna... It's such a reality check for people, where they think, like, "Oh, I trained in this great PM culture at Facebook, or at Google, or at Amazon, or at Stripe," or, like, pick your favorite. Like, "I, I, I trained at this great PM factory. Surely, I am now prepared to be an amazing founder." Like, immediately unlearn everything you ever thought you knew about product management or your career in general. And like, go st- go talk to a bunch of local small business owners, and you'll learn a lot more that way about, like, how do you get your first few customers? How do you set up basic marketing? How do you set up payroll? Like, [laughs] um, I'm, I'm not saying to set up payroll is the hard part of a, a, of, of, of starting a company. The hard part about starting a company is, like, there is literally no inertia. You have nothing. You have no product, no customers, no messaging, no marketing, no positioning. No one cares at all about you. Everyone already has way too many tools. Or everyone already has way too many salespeople banging down their door. Like, how to turn nothing into something is just so different. When I, like, I can pat myself on the back for doing a god- good job on Gmail all I want. When I inherited Gmail, we already had 800 million users. And yeah, we grew from 800 million to a couple billion, and I feel good about that. But, like, we already had the 800 million. [laughs] So it is a totally different experience to go from, like, your zeroth customer to your first customer. [laughs]
- AGAakash Gupta
Yes. That's a bold take, but I think a reality a lot of Big Tech PMs should hear, so that they go do talk to those small business owners early on when they're starting, because so many of them do. We could talk for another 90 minutes. Perhaps we'll have to in the future. Jacob, thank you so much for doing the podcast.
- JBJacob Bank
Thanks for having me. This was super fun.
- AGAakash Gupta
All right,
- 1:19:09 – 1:19:36
Outro
- AGAakash Gupta
everybody, we'll see you in the next one. I really hope you guys enjoyed that episode. It would mean a ton to me and the team if you could please subscribe on YouTube, follow on Apple and Spotify podcasts, and leave a rating and review. Those ratings and reviews really help grow the show and help other people discover the show, and they help fund the production so that we can do bigger and better productions. Can't wait to share the next episode with you. Until then, see you later.
Episode duration: 1:19:50
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode SGUXMugyo6g
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome