AI Is the Biggest Cyber Threat — Only Okta’s AI Security Playbook can safe you

Jack Hirsch, Head of AI Products at Okta ($15B market cap), reveals the wildest AI cybersecurity threats. He shares why AI agents are the biggest security blindspot, and explains his controversial take on why traditional PM experience is bad founder preparation. Transcript: https://www.news.aakashg.com/p/jack-hirsch-podcast Timestamps: 00:00 Intro 02:02: Wildest AI Cybersecurity Threats 04:27 Moment AI Changed Security 06:32 How AI Agents Change Security Equation 11:14 Most Dangerous AI Threats 14:16 Okta's AI Threat Detection System 18:10 Ads 19:50 Okta's AI Security Playbook Revealed 22:15 T-Shaped Identity Strategy Breakdown 26:42 One Thing Every Company Must Do This Year 31:37 Ads 33:16: How to Handle AI Security Threats 40:09 Building AI-Secure Products at Okta 46:47 Fundamental AI Product Development Principles 01:06:29 Butter.ai Startup Story (2015) 01:19:18 Why Evernote Failed Despite Early Success 01:20:48 Hustling Into PM at Evernote 01:26:51 Personal Identity Protection Guide 1:30:36 Outro Thanks to our sponsors: 1. Amplitude: The market-leader in product analytics: https://amplitude.com/session-replay?utm_campaign=session-replay-launch-2025&utm_source=linkedin&utm_medium=organic-social&utm_content=productgrowthpodcast 2. The AI Evals Course for PMs: Get $1155 off with code ‘ag-evals’: https://maven.com/parlance-labs/evals?promoCode=ag-evlas 3. The AI PM Certificate: The #1 AI PM certificate: https://maven.com/product-faculty/ai-product-management-certification?promoCode=AAKASH550C7 4. Kameleoon: Leading AI experimentation platform: http://www.kameleoon.com/ ---- Key takeaways: 1. Identity is Everything: Over 80% of breaches stem from identity attacks, not device or network vulnerabilities. You cannot get security right without getting identity right - this is the new reality. 2. DPRK Infiltration Operations: North Korean agents are passing full interview processes, getting hired, having laptops shipped to device farms, and operating as inside threats within major organizations. 3. AI Agents = Security Blindspot: Companies deploy AI agents en masse without treating them as identities requiring access management. JP Morgan's CISO called this out as the biggest current threat vector. 4. Help Desk Social Engineering: Attackers use AI voice cloning and deepfakes to impersonate employees calling help desk for password resets, MFA bypasses, and account access - often successfully. 5. Session Security Over Time: Authentication degrades after login. Okta focuses on continuous session monitoring and risk signal sharing between security vendors rather than constant MFA prompts. 6. T-Shaped Identity Strategy: Deep identity security (phishing-resistant auth, lifecycle management, risk sharing) plus broad integration across all enterprise systems - not just SSO and MFA. 7. Cross-App Access Standard: New OAuth standard allows AI agents to inherit user permissions across enterprise apps without individual OAuth dances for thousands of employees. 8. Essential vs Discretionary AI: Essential AI (bot detection, fraud prevention) stays always-on. Discretionary AI (log summaries, access reviews) gives customers opt-out control for compliance. 9. AI Product Principles: Accelerate don't abdicate, solve real problems before prototyping, ignore AI hype cycle. Use AI as thought partner, not replacement for product judgment and domain expertise. 10. Personal Security Stack: Lock credit reports immediately, use password manager with unique passwords, enable passkeys everywhere, lock phone number with carrier PIN to prevent SIM swapping attacks. ---- Where to find Jack: LinkedIn: https://www.linkedin.com/in/jackhirsch/ Okta: https://www.okta.com Where to find Aakash: Twitter: twitter.com/aakashg0 LinkedIn: linkedin.com/in/aagupta/ Newsletter: news.aakashg.com #cybersecurity #ai #productmanagement About Product Growth: The world's largest podcast focused solely on product + growth, with over 187K listeners. Hosted by Aakash Gupta, who spent 16 years in PM, rising to VP of product, this 2x/week show covers product and growth topics in depth. Subscribe and turn on notifications to get more videos like this.

Aakash GuptahostJack Hirschguest

Sep 22, 20251h 31mWatch on YouTube ↗

EVERY SPOKEN WORD

85 min read · 16,783 words

0:00 – 2:02
Intro
1. AGAakash Gupta
  Okta has been making huge waves. You guys probably get one of the best views into what is happening in cybersecurity out there. What are the wildest AI cybersecurity threats you have seen?
2. JHJack Hirsch
  The DPRK is basically planting workers into many of the organizations that you might be familiar with, going through full interview loops, virtual backgrounds, ship laptops, and then there are inside threats. Yeah, there's another one around help desk scenarios, people calling in saying, "I need to reset my password. I no longer have access to my corporate email." And then suddenly you've got someone that's come in through a help desk. But ultimately, it all comes down to the human element, which is always the weakest link.
3. AGAakash Gupta
  Some reports are saying cybersecurity threats are up 40% compared to last year. AI is really enabling these hackers to do more.
4. JHJack Hirsch
  Yes, I think 40% is probably a low estimate. Identity actually has become the primary threat vector. Before it was devices, networks. Now they're going after the identity.
5. AGAakash Gupta
  So what was the first moment you realized AI is gonna change security forever?
6. JHJack Hirsch
  I wrote myself a little phishing kit based on the Okta SDKs. That's when I realized if we're not careful, the wheels are gonna come off the bus.
7. AGAakash Gupta
  Really quickly, I think a crazy stat is that more than 50% of you listening are not subscribed. If you can subscribe on YouTube, follow on Apple or Spotify podcasts, my commitment to you is that we'll continue to make this content better and better. And now on to today's episode. Okta is one of the world's largest cybersecurity companies, with a market cap of $15 billion. Jack Hirsch is the product leader behind it. He leads all of Okta's core products, including their AI products. Today, we're gonna break down what you need to know about AI cybersecurity threats, how to secure yourself against them, and how to build products in this AI age. If you're somebody who's building AI products or building products, there's gonna be something for you. Jack, welcome to the podcast.
8. JHJack Hirsch
  Thanks for having me. Great to
2:02 – 4:27
Wildest AI Cybersecurity Threats
1. JHJack Hirsch
  be here.
2. AGAakash Gupta
  Okta has been making huge waves. You guys probably get one of the best views into what is happening in cybersecurity out there. What are the wildest AI cybersecurity threats you have seen?
3. JHJack Hirsch
  That's a great place to start. Uh, I will say that the threat landscape has changed quite a bit in the past few years. Um, I think just jumping straight to the wildest, uh, I think the, the DPRK is basically planting workers, uh, into many of the organizations that you might be familiar with. Uh, and so what they're doing is going through full interview loops. They will have, you know, virtual backgrounds. They will be using, uh, AI interview tools. They will get onboarded. They will mail their devices to, uh, device farms somewhere, you know, in whatever country, whether it be the United States, places in Europe. They have places where they receive laptops, and they'll ship them over to DPRK, and then there are inside threats, uh, inside. So I think that's one, uh, very, very scary threat vector we're seeing across the industry. Um, yeah, there's another one that's also, also just as, uh, fun, not quite as nefarious, but really needs-- Uh, or I guess not quite as creative, but really needs focus, is around, uh, help desk scenarios, so things like MFA resets, right? So people calling in saying, you know, "Hi, I'm Aakash. Um, uh, I need to reset my password. Can you just go ahead and reset my password? Oh, I don't have a-- I no longer have access to my corporate email. Sign me in." You know, like, uh, "Reset my MFA, reset my password." And then suddenly you've got someone that's come in through a help desk. Oftentimes, those get really sophisticated. They will impersonate voices. Uh, they will go in via Slack. Uh, you know, there'll be threat actors that move laterally and again, use all the tools in the, like, [chuckles] we're all talking about VO for, for, uh, generating small clips of video. Uh, there are some really sophisticated threat vectors when you're combining all these, like, audio and video generation tools, voice changers, et cetera. So yeah, but ultimately, it all comes down to, uh, often it comes down to the human element, which is always the weakest link. So yeah, uh, exciting times in the world of cybersecurity.
4. AGAakash Gupta
  Yeah.
4:27 – 6:32
Moment AI Changed Security
1. AGAakash Gupta
  I think some reports are saying that this year, cybersecurity threats are up 40% compared to last year. So it seems like AI is really enabling these hackers to do more.
2. JHJack Hirsch
  Yes, I think 40% is probably a low estimate by what we're seeing at Okta. Um, but I will say that, uh, one of the things that is most interesting for us at Okta is that, uh, identity is actually becoming, uh, has become over the past few years, the primary threat vector. So before it was devices, networks, and people who are attacking systems, right? Poor coding practices. Now they're going after the identity, whether that be, uh, the physical identity in a DPRK, uh, DPRK cases or digital identities. Um, so yeah, uh, identity really is the, the center of modern cybersecurity. Um, so very interesting times, exciting times to be at Okta, exciting times to be working in identity security.
3. AGAakash Gupta
  So what was the first moment you realized AI is gonna change security forever?
4. JHJack Hirsch
  Um, I, uh, I pulled up, uh, a vibe coding, a vibe coding tool. It's not one of the big ones. It was one of the, uh, you know, one of the sort of early movers and, uh, I wrote myself a little phishing kit based on the Okta SDKs. So looked like Okta, felt like Okta, and since then, uh, we actually just published a report, um, that sure enough, threat actors have figured this out themselves. Um, but yeah, that's when I realized, uh, if we're not careful, the wheels are gonna come off the bus.
5. AGAakash Gupta
  Because the vibe coding tools are crazy good at replicating any design.
6. JHJack Hirsch
  Absolutely. I mean, especially when the designs themselves, they're SDKs, right? Like, they're not, they don't necessarily have to be that good. Um, you just pick a domain name that's one or two letters off of what your, your target is. You know, you assume they're using, uh, insert your identity provider, insert your SSO provider, grab those SDKs, and start building. Um, and then it's all about finding your targets and, you know, doing the last mile social engineering to get there. But yeah, it's, uh, it's pretty wild out there.
6:32 – 11:14
How AI Agents Change Security Equation
1. AGAakash Gupta
  Being an employee is, like, scarier than ever. Like [chuckles] those little phishing tests that companies send you, they could just easily catch you now. Why can't traditional cybersecurity stop AI attacks?
2. JHJack Hirsch
  [sighs] Yeah. Um, I... The reality is, uh, traditional cybersecurity... And I think, [sighs] well, traditional cybersecurity typically focuses on a defense in depth strategy, right? There's some combination of device security, network security, and of course, identity security. Um, identity used to be considered, like, an IT thing. Like, oh, it's SSO, it's MFA, the IT folks will take care of that. It is very much a security game, uh, as of the past five, 10 years. And so, uh, the thing that needs to change is that we need to start thinking about identity as the, the final frontier and the first frontier in terms of stopping cyber threats. Uh, so I think that's probably the biggest, uh, the biggest thing to keep in mind.
3. AGAakash Gupta
  How do AI agents change the security equation?
4. JHJack Hirsch
  Uh, AI agents change the security equation in a couple different ways. Um, one is, uh, they present a threat vector in and of themselves in their capabilities, but I think more interestingly and sort of more nefariously and kind of flying under the radar is that we're deploying AI agents en masse in every single modern organization.
5. AGAakash Gupta
  Yeah.
6. JHJack Hirsch
  And we're not thinking about them as identities that we need to manage that are being granted access to critical resources inside of the organization. And so it was just, uh, just a, uh, a couple months ago, the, uh, CISO, chief information security officer, of JPMorgan Chase put out an open letter to the SaaS ecosystem, basically calling out SaaS vendors for not taking this seriously. And there was a h- there was a hint that AI agents was, were just making this worse.
7. AGAakash Gupta
  Mm.
8. JHJack Hirsch
  Uh, this idea that we're collapsing authentication and authorization. We're taking these AI agents, we're bringing them into our corporate boundaries, we're giving them unfettered access to our customer data, our user data, uh, and frankly, with, with little control, little visibility, little oversight, uh, little security. And so I think that is probably the most clear and present danger today. Um, there's obviously, you know, external AI agent threats to modern organizations, but I think the, the real, the real scariest thing is the fact that security teams and IT teams don't wanna be hindering their organization's adoption of AI tools, so they have to let them in, right? The business always wins, and security is always playing catch-up, frankly.
9. AGAakash Gupta
  [laughs]
10. JHJack Hirsch
  And so the unfortunate thing there is that, uh, oftentimes those things are, uh, potentially doing nefarious things, whether the users like them to or not. So that's, uh, that's probably the biggest issue.
11. AGAakash Gupta
  AI agents and MCP, I feel like people have been at least talking about security in the circles I come to. What are the underrated things? What are the security threats that nobody is talking about within AI that they should be?
12. JHJack Hirsch
  AI, MCP, normal security threats, what else is there outside but still AI-adjacent security threats?
13. AGAakash Gupta
  Exactly.
14. JHJack Hirsch
  Okay.
15. AGAakash Gupta
  That nobody's talking about, but they should be.
16. JHJack Hirsch
  Great. Um, well, I think as context windows are getting bigger and bigger, you can just slurp in an entire, uh, API surface and look for various combinent- combinations of access patterns and look to go in, you know, what used to be the hard way, used to be someone had to be, uh, very, very smart, keep a lot of context in their head, understand how, uh, different access functions at an API surface, uh, uh, uh, fundamentally function.
17. AGAakash Gupta
  Yeah.
18. JHJack Hirsch
  And then now you can just jam that all in an LLM, tell it that you want in, tell the agent to go, and they will go. Um, and so I think, uh, uh, if you, if you think about how APIs get built at large organizations in API security, there are dozens, hundreds of teams that are building various bits and pieces of API surface. How often are they actually cross-checking different, uh, uh, different access patterns across those teams, right? Everyone is so siloed. AI is not siloed. AI will take it all in. AI will find the vulnerabilities and will go after them very, very easily.
19. AGAakash Gupta
  Mm.
20. JHJack Hirsch
  So, uh, and that you do not have to be a sophisticated actor to go after those. So I think, uh, especially a lot of the vibe coded tools that are out there, uh, on the market, this is, uh... I, I would recommend you build some sort of, um, uh, you know, red teaming concept LLM and turn it loose on whatever application you're vibe coding, especially if you're bringing any sort of sensitive information into it.
21. AGAakash Gupta
  Mm.
11:14 – 14:16
Most Dangerous AI Threats
1. AGAakash Gupta
  I like that. So is the state-of-the-art then to basically ha- train an LLM to red team against you and act as that agent so that you can follow those threat vectors?
2. JHJack Hirsch
  I mean, I don't know about state-of-the-art necessarily, but I feel like w- I would be remiss if I didn't recommend people do that, right?
3. AGAakash Gupta
  Yeah.
4. JHJack Hirsch
  I mean, you think about, uh, a really m- robust software development life cycle, you're definitely having security review things before they go out the door.
5. AGAakash Gupta
  Yeah.
6. JHJack Hirsch
  If you don't have a security team, what are you gonna do, right? Uh, we all talk about having these AI board of advisors, right? Well, think about your software development life cycle. Who are the, what are the personas that you want involved in that life cycle? You might have a legal review, I don't know. Uh, but you definitely want a security review, and so training an LLM to review the code that you're publishing. There's so many, uh, like every second day, every second hour, I should say, there's a story about some vibe coded app getting hacked.
7. AGAakash Gupta
  Yeah.
8. JHJack Hirsch
  Um, andFrankly, all of that me- much of that could have been, uh, dealt with if you just would've, you know, thought adversarially for a beat before you pushed it out.
9. AGAakash Gupta
  Nice. I like that. So as you're pushing these things out, add that adversarial beat in, just like you would've if you weren't vibe coding it. So there's so many other AI threat vectors we haven't even talked about yet, right? AI-generated identities, deepfakes, synthetic fraud. What of these are the most dangerous, and why?
10. JHJack Hirsch
  Um, deepfakes are very, very dangerous, I think, on a human level. I think we're seeing, um, I saw Pennsylvania state senator, I think, speaking about how his physical identity was compromised, um, because of a deepfake, right?
11. AGAakash Gupta
  Wow.
12. JHJack Hirsch
  Um, people are... And this is, this is pretty com- I mean, not pretty common, but I think this is, this is made at and out of the sort of technosphere and into the general news cycle, where you'll have, uh, I think, I think the, the person in question I'm thinking about, uh, their son had some, uh, YouTube recordings up. And so they scraped the son's YouTube, uh, grabbed a bunch of their audio, faked their audio, called their dad, who happened to be a Pennsylvania s- I believe a Pennsylvania state senator, we should check this afterwards, um, and basically said, "Hey, I'm in jail. Uh, [laughs] help me get out. I need, you know, need money." And it was a real-time conversation that they had with their son. Uh, and so, uh, I think those things, when you extrapolate them out into the software world, like, yeah, I might, if I were a help desk representative and I got a call from the CEO, I might reset the CEO's password if the CEO called me and said, "Hey, I need you to reset my password. I'm locked out."
13. AGAakash Gupta
  Yeah.
14. JHJack Hirsch
  Uh, that's a really easy way into an organization, unfortunately. Uh, and so in the security, in security circles, um, locking down those critical access flows from support desks, that's a, a also a major area of focus.
15. AGAakash Gupta
  Crazy. So hopefully people got the picture, right?
14:16 – 18:10
Okta's AI Threat Detection System
1. AGAakash Gupta
  AI cybersecurity, if you haven't been hearing about it on the news yet, you should be, because it's a serious, serious issue. You guys are at the center of this all. You need to secure your own systems from these types of crazy attacks we've just been talking about. What is your own AI threat detection system? Walk us through it.
2. JHJack Hirsch
  So, uh, actually I'll, I'll maybe take this in two angles. One is the same, uh, threat vectors, uh, of, you know, credential stuffing, M- like push bombing MFAs. So like basically sending MFA pushes through some phishing site over and over and over again until a user relents and just says, "Okay." Like, we need to guard against that. And so at Okta, we have bot detection, fraud detection built into the standard auth flows.
3. AGAakash Gupta
  Mm-hmm.
4. JHJack Hirsch
  And that's evolved quite a bit. It's taking in a bunch of new signal types. Uh, but that is sort of business as usual, I would say. It's just evolving with the times. The second part, which I think is actually more interesting, frankly, is the assumption now that identities are effectively compromised. Everyone's identity is compromised.
5. AGAakash Gupta
  Mm.
6. JHJack Hirsch
  Um, I think the stat that I last saw was somewhere between two and four billion credentials and session tokens and cookies were stolen just in the past year.
7. AGAakash Gupta
  Wow.
8. JHJack Hirsch
  And I think that was just from the Fortune 1000 alone, which is where they were able to get the data. So if you think about this, the front door is wide open, right? So you need to ready your defenses. So we used to say, you know, in an enterprise context, "Oh, it's all about SSO and MFA and securing the point of access." Well, we've all had that experience where you're MFA-ing over and over and over again, and it's exhausting, right? You're getting like, oh, if you're at work, you're, okay, single sign-on into Slack, single sign-on to Teams, single sign-on to this, single sign-on to that. Um, that's exhausting. And then over time, you know, those, like, mission critical systems, if you're in a finance system, they might force you to reauth every few hours or once a day. It's exhausting. So how do you maintain that session security over time? Because that session security naturally degrades over time, right? You create the session, you cut the cookie, create the token. Just by the very fact that that thing exists out in the wild means that there's more likelihood that someone's gonna compromise that, right? And so a huge area of focus for us at Okta has been, how do we maintain session security over time while keeping the user experience, you know, as good or better, uh, than, you know, pushing MFA all the time?
9. AGAakash Gupta
  Yeah.
10. JHJack Hirsch
  Uh, and so we've taken an approach of, uh, so Okta, you know, this is not about Okta, but I think in general, the open ecosystem approach, which we're huge fans of, uh, we have an open ecosystem approach using open standards like Shared Signals Framework, uh, which is effectively, uh, security vendors will share risk signals with each other. So for example, CrowdStrike or Zscaler or Palo Alto Networks will share a risk signal on behalf of Akashi and say like, "Hey, your identity may have been compromised, but CrowdStrike may have detected it."
11. AGAakash Gupta
  Mm.
12. JHJack Hirsch
  So what remediations can we take on the identity side, uh, to be complementary to device or network security, right? Uh, and so I think how we do that in a seamless way, that's been really, really exciting. So maintaining that session security over time. And as an end user, I think we don't, we don't think about it because ideally, the, the, uh, the ideal experience is you're not MFA-ing all the time. You're just seeing continuous access.And what's happening in the background is we're continuously reverifying all of the signal that we have about you. Has your device posture changed? Has your network environment changed? Have you immediately magically traveled around the world to Azerbaijan, you know, in one step? Like, and did we recognize that? We would, we have first-party detections. Did a third party recognize that? And how we orchestrate remediation around the security ecosystem, that's been, uh, probably the second area of focus, and I think that's new as of the past
18:10 – 19:50
Ads
1. JHJack Hirsch
  couple of years.
2. AGAakash Gupta
  Today's episode is brought to you by Amplitude. Replays of mobile user engagement are critical to building better products and experiences. But many session replay tools don't capture the full picture. Some tools take screenshots every second, leading to choppy replays and high storage costs from enormous capture sizes. Others use wireframes, but key moments go missing, creating gaps in your understanding. Neither approach gives you a truly mobile experience. Amplitude does things differently. Their mobile replays capture the full experience, every tap, every scroll, and every gesture, with no lag and no performance hit. It's the most accurate way to understand mobile behavior. See the full story with Amplitude. AI evals are one of the most important skills for PMs, and I know you know they matter. The question is, are you doing them right? Most teams are winging it with basic metrics and hoping for the best. Meanwhile, the teams that actually ship reliable AI, they've cracked the code on systematic evaluation. Today's episode is brought to you by the AI Evals for Engineers and PMs course by Hamel Hussain and Shreya Shankar. This live Maven course will teach you the battle-tested frameworks from Hamel and Shreya, who are the engineers behind GitHub Copilot's evaluation system and 25-plus production AI implementations. Four weeks, live instruction. Next cohort starts July 21st. Start shipping AI that actually works. Enroll at maven.com with my code AG-PRODUCT-GROWTH for over $800 off. That's AG-PRODUCT-GROWTH.
19:50 – 22:15
Okta's AI Security Playbook Revealed
1. AGAakash Gupta
  So for customers, what's in Okta's AI Security Playbook? Give us the secret sauce.
2. JHJack Hirsch
  Oh, uh, that is actually really exciting. So we've got some, some new things in the works, uh, that we have not yet officially unveiled, but I would love to share with you.
3. AGAakash Gupta
  Ooh.
4. JHJack Hirsch
  Um, so, uh, we are in the month, uh, so early in August, uh, we announced a, uh, a new technology with the OpenID, uh, basically as part of our, the, the OAuth standard that we call cross-app access. So what this is, is a way to allow an AI agent to request access to a resource inside the organization through the central identity provider.
5. AGAakash Gupta
  Mm.
6. JHJack Hirsch
  So what this means is practically imagine, like, the, uh, the Claude 4.1 launch video had, uh, someone asking a question of Claude and it reaching out to Gmail, Google Calendar, Asana, all of those things. Imagine starting your workday, uh, as an employee of Acme Corp, and you're day one, right, employee of Acme Corp, and you sign into Claude, your AI agent companion, and it immediately has access to all of your, you know, uh, Google Workspace, your Asana, your Slack, your, you know, Salesforce, all of the things that you need to be productive, it's all wired in immediately. That should be the end user experience. Now, the difference is today, if you were to do that, you're doing an OAuth dance. Every user is doing an OAuth dance. You say, "Oh, you wanna connect Claude to Gmail? Sure, do an OAuth dance. Oh, you wanna connect Claude to Asana? Do an OAuth dance." Imagine spraying that out to thousands of employees or hun- you know, tens of thousands of employees at the largest organizations in the world. So we don't want our customers to go through that. We want our customers to have that elegant experience where all their users just sign in and they start using their AI agents, and they're augmented immediately. So that's the, that's what we're, we're going for. And so I think from, you know, our, our customers at Okta, generally security and IT practitioners. So for them, the benefit they get is they get visibility into all the agents that are deployed in their enterprise.
7. AGAakash Gupta
  Mm.
8. JHJack Hirsch
  They get the ability to control with a very granular access. These users might be able to connect Claude just to these applications
22:15 – 26:42
T-Shaped Identity Strategy Breakdown
1. JHJack Hirsch
  over here, but these other users, they might be privileged. They can connect Claude to a bunch of other services over here. So they get that granular level of control. They get to manage the lifecycle of those AI agents. So I think that's a huge area of investment for us at Okta right now.
2. AGAakash Gupta
  Mm. [lips smack] So in June of this year, you spoke with Gartner. You broke down T-shaped identity strategies, which I think are deep AI for threats and broad for security. Can you break this down?
3. JHJack Hirsch
  Yeah. Uh, so this is part of one of the sort of mega trends in identity security these days. Uh, we used to think, again, about identity as being the point of a- auth, right? Secure auth. Well, it turns out that there's a whole user lifecycle, so that the top of the T is before auth, what do we do to make sure that you're gonna have the right level of access to the right things at the right time, no more, no less, and that's it. And that, all that happens before you ever hit a sign-in box. And then when you go to auth, how can we make it phishing-resistant? How can we guarantee that phishing resistance regardless of what device you're using, whether it be a Mac or a PC, iOS, Android, whether you're on a managed device, unmanaged device, all of that needs to be phishing-resistant, um, and controlled by the organization. And then a- again, the, the last part of the T is how you maintain that session security over time.
4. AGAakash Gupta
  Mm.
5. JHJack Hirsch
  So again, as we talked about, session security degrades over time, so how do we maintain that session security without, uh, wi- without dropping the, you know, the dreaded MFA, MFA, MFA-
6. AGAakash Gupta
  [chuckles]
7. JHJack Hirsch
  ... um, you know, continuously. And then the, the sort of base of the T... is about not just having, uh, so identity used to be thought of as, again, SSO.
8. AGAakash Gupta
  Yeah.
9. JHJack Hirsch
  Now it's so much more. It's managing the user's life cycle, so joiners and movers and leavers. Like, you know, Jack changed teams. Does he have access to the right tools when he changed teams, right? All of those, all of those types of integrations, those are lifecycle management integrations. There's an open standard called SCIM that manages those lifecycle changes. Then it's how do you share risk signals, for example, with, uh, security, with other security vendors or tooling. There's actually, uh, tooling on both of our devices right now that will share risk signals back with Okta using an open standard.
10. AGAakash Gupta
  Mm.
11. JHJack Hirsch
  Right? So, uh, not just SSO and MFA and lifecycle management and risk signal sharing, but then session termination is another one. So it's how deep you go in those integrations. So sess- session termination would be like, uh, let's say we discover that your identity's been compromised. Oh, no. Uh, someone stole a credential off of your device and is replaying it. I don't know why I wanna pick on Azerbaijan, but, uh-
12. AGAakash Gupta
  [laughs]
13. JHJack Hirsch
  ... it's fun to say, and so I'll pick on them. But say, like, there's someone else, there's a threat actor replaying your session elsewhere. How do we terminate your session, not just at the IDP, not just at Okta or on Entra or on Ping or wherever, but in all the downstream applications so that you get signed out of Slack and Zoom and, uh, Salesforce and, and then your device logs you out as well. So how do we shut down access all at once? So that's way more than just the moment of auth. It's maintaining security through that entire end user's lifecycle and having a depth of integration, uh, that goes way beyond SSO and MFA.
14. AGAakash Gupta
  Mm-hmm.
15. JHJack Hirsch
  Uh, and so that's, that's fundamentally what's needed, and that means that you can detect risks, you can respond to them in an automated way, you can remediate them. Uh, and so again, you know, the moment we discover that I'm not actually sitting across from you and it's actually a DPRK threat actor-
16. AGAakash Gupta
  [laughs]
17. JHJack Hirsch
  ... uh, you know, we can terminate this, uh, this interview immediately.
18. AGAakash Gupta
  Oh, very interesting.
19. JHJack Hirsch
  So that's, that's the, yeah, maybe the analogy.
20. AGAakash Gupta
  So before we close this Okta section here, how many AI-powered attacks do you reckon Okta is stopping per day, and your success rate?
21. JHJack Hirsch
  Oh, goodness. Um, this, I'd actually wanna check with our security team on this. I, I'm worried about drastically undershooting-
22. AGAakash Gupta
  [laughs]
23. JHJack Hirsch
  ... because I'm gonna use an M in front of the illion and not a B.
24. AGAakash Gupta
  Oh.
25. JHJack Hirsch
  Um, but it's, it's of that scale.
26. AGAakash Gupta
  Wow.
27. JHJack Hirsch
  Um, and so when you think about it, Okta's doing tens of millions of logins a month, right? We are, Okta is so big, and this is actually one of the things that, you know, we can talk, talk about if we get into product management as a discipline and sort of the ecosystem. We see macroeconomic trends in our data, right? We are so big that that's,
26:42 – 31:37
One Thing Every Company Must Do This Year
1. JHJack Hirsch
  we see that sort, those sort of moves.
2. AGAakash Gupta
  Yeah.
3. JHJack Hirsch
  Right? And so yes, absolutely, we're seeing a significant amount of AI-powered attacks, uh, across the Okta product suite.
4. AGAakash Gupta
  Crazy. So let's move into the playbook for any company.
5. JHJack Hirsch
  Sure.
6. AGAakash Gupta
  What's one thing every company should do this year to protect themselves from AI-enabled threats?
7. JHJack Hirsch
  Number one, you cannot get security right without getting identity right. Um, this is not about, this is not a pitch or a plea for Okta. This is, uh, I guess a plea for every organization to make sure that their users are protected. Um, again, threat actors today, the vast majority, over 80% of threat of, uh, breaches stem from some form of attack on identity.
8. AGAakash Gupta
  Yeah.
9. JHJack Hirsch
  And so what that means is every organization needs to get their identity security in order. Um, it's no longer just enough to have SSO. It's definitely not. It's not enough to just have MFA. It's not just enough to have phishing-resistant MFA. It's not enough to have... You know, you need to have that continuous session security, or else you're gonna continue to have those, uh, a- you know, long-lived, uh, session threats.
10. AGAakash Gupta
  Mm.
11. JHJack Hirsch
  We need, you know, you need to close down support desk and make sure that all of those acts are, are the, those actions that the s- those support folks can take is logged and controlled. So all of that is to say is you can't get security right if, uh, if you don't get identity right.
12. AGAakash Gupta
  Mm.
13. JHJack Hirsch
  Um, that's, that's the number one thing.
14. AGAakash Gupta
  What are the biggest mistakes companies are making when trying to AI-proof their security?
15. JHJack Hirsch
  I think assuming that there's gonna be a, a singular solution, frankly. Um, and I think that we've been, you know, we're trained to think about... And when I say singular solution, there are these, like, monolithic, or let's call it microlithic platforms, uh, up, up in, uh, Washington, based in Washington, that, uh, will say, oh, they'll, they'll keep you secure.
16. AGAakash Gupta
  Mm.
17. JHJack Hirsch
  But they'll keep you secure when you're using products in their ecosystem, right? That's their, that's their commercial incentive, right?
18. AGAakash Gupta
  Yeah.
19. JHJack Hirsch
  Is to keep, keep their, their customers safe. And they wanna say, "Oh, if you use something outside of our ecosystem, that's scary. You don't wanna do that." Uh, and then at the same time, you know, the best-of-breed world has had a bit of a reckoning too, right? So if you say, oh, well, I'll, I'll take, again, the identity space. Identity and access management used to be thought of as a totally different market than identity governance, totally different market than privileged access management. All of those markets are converging, and so there's the platformification around all these security vendors. So I think a, another big area of focus here needs to be on making sure that you're investing in your primary security, uh, primary security clouds wisely, and I think identity definitely deserves to be in there. I also think, uh, frankly, network, you know, you need a defense in depth, device strategy. All of these things need to be thought through. Um, yeah.
20. AGAakash Gupta
  What's the right stage to really just take... Like, when do you need to take a really close look at how AI-proofed your security is? Is that any stage company? Is there a certain stage you need to reach?
21. JHJack Hirsch
  I think anytime you have something to lose.
22. AGAakash Gupta
  Hmm.
23. JHJack Hirsch
  So I'll, uh, this was not... So, uh, many, many years ago, I worked for a company called Evernote.
24. AGAakash Gupta
  Hmm.
25. JHJack Hirsch
  Evernote famously had a breach where a bunch of, uh, usernames and password, uh, hash, salted hash passwords, uh, were released. Um, and that really, that was a huge catalyst for the, uh, I wouldn't say demise, but definitely, like they're not considered a winner by any, you know, by Silicon Valley standards at this point.
26. AGAakash Gupta
  Yeah.
27. JHJack Hirsch
  And so they had a lot to lose. Um, and the threat vector there that I, I don't know if it's been disclosed, but years later, I'm pretty sure statute of limitations has expired, but, like that was related to very similar to a help desk attack, right? That was an identity-based attack that eventually-
28. AGAakash Gupta
  Oh
29. JHJack Hirsch
  ... got a threat actor in, [lips smack] uh, and was able to push code to production to exfiltrate that data. And so, uh, yeah, I mean, I think, you know, in terms of to answer your question very directly, as soon as you have something to lose, um, if users are using normal credentials to get access to production systems, um, and they're just using a username and password, that should be a huge red flag. They need to be using phishing-resistant authentication. Every single system that that user uses, they are now a privileged user. Every single system that that user use- uses needs to be phishing resistant, needs to have risk signal sharing on, needs to have continuous, uh, continuous session monitoring. Um, yeah, because, uh, it is, it is way too easy to hit one of those missteps and lose
31:37 – 33:16
Ads
1. JHJack Hirsch
  it all.
2. AGAakash Gupta
  Today's episode is brought to you by the AI PM Certification on Maven. Run by Miqdad Jaffer, who is a product leader at OpenAI, this is not your typical course. It's eight weeks of live cohort-based learning with a leader at one of the top companies in tech. OpenAI just doesn't stop shipping, and this is your chance to learn how. Run along with product faculty and Mo Ali, the course has a 4.9 rating with 133 reviews. Former students come from companies like OpenAI, Shopify, Stripe, Google, and Meta. The best part? Your company can probably cover the cost. So if you want to get $500 off, use my code AAKASH25 and head to maven.com/product-faculty. That's M-A-V-E-N.com/P-R-O-D-U-C-T-F-A-C-U-L-T-Y. Today's episode is brought to you by the experimentation platform Kameleoon. Nine out of 10 companies that see themselves as industry leaders and expect to grow this year say experimentation is critical to their business. But most companies still fail at it. Why? Because most experiments require too much developer involvement. Kameleoon handles experimentation differently. It enables product and growth teams to create and test prototypes in minutes with prompt-based experimentation. You describe what you want, Kameleoon builds a variation of your webpage, lets you target a cohort of users, choose KPIs, and runs the experiment for you. Prompt-based experimentation makes what used to take days of developer time turn into minutes. Try prompt-based experimentation on your own web apps. Visit kameleoon.com/prompt to join the waitlist. That's K-A-M-E-L-E-O-O-N.com/prompt.
33:16 – 40:09
How to Handle AI Security Threats
1. AGAakash Gupta
  So everybody needs to do it. So then walk us through your playbook that, like for those companies that don't have security professionals, how should they be evaluating if their current security can handle AI threats?
2. JHJack Hirsch
  I think f- uh, so first off, um, AI threats in general, it's, uh, there's any number of different market segments and categories, right? Are you trying to secure your software, the software that you're actually building? Are you trying to secure your network perimeter, your boundary, if you're running a physical operation? Uh, are you trying to, uh, secure your devices? So really it, it's gonna depend on the type of organization that you're, you're running.
3. AGAakash Gupta
  Yeah.
4. JHJack Hirsch
  But I think all of that diversity of, of types of organizations means that there's only one constant. It's identity. And I hate to say this coming from Okta, a- as the world's leading independent neutral identity vendor, but like fundamentally, identity is central to security. And so you can't get it right. Uh, you cannot get security right without getting identity right. So the, the way that I would start, frankly, is making sure that identity security is locked down.
5. AGAakash Gupta
  Hmm.
6. JHJack Hirsch
  Um, I think it's, at this point, we no longer... Most organizations, most startups, I would imagine most of your viewers, they're not gonna be working from a SKIF, you know, for the DoD, right? They're gonna be sitting in a room like this, maybe not with the microphones and lights, but they're gonna be working together on something that they think is gonna be cool and change the world.
7. AGAakash Gupta
  Yeah.
8. JHJack Hirsch
  And that's awesome, and they should do that, but as soon as you have something to lose, get that identity security down right, because then you're gonna have other people. You might have contractors. You might be using, you know, uh, you might be using contracting platforms to give other people access to various parts of your ecosystem. How do you secure their access? How do you make sure that, uh, they have access to the right things only, right? So maybe it's really easy if you have Google Workspace and you're just giving them access to one or two docs, and they have their own domain, but as soon as you need to bring them into your trusted boundary, say they're labeling your A, you know, they're labeling your data for AI, they're working in your code base, they're working in your design base, they're, uh, a guest member in your Google Workspace, um, that's when it starts to get very, very real, and those things happen very quickly as you're scaling, right? You always, you're gonna end up bringing contractors, consultants, business partners. They come in very, very quickly, and making sure that they have the right access, uh, fundamentally is an identity challenge.
9. AGAakash Gupta
  And for people who don't truly grok what's going, what you're saying here, basically you're saying that you should have the login for these systems go through an identity portal like Okta, which is identifying do they actually have access to this? Are they getting the right access? And did I understand that correctly?
10. JHJack Hirsch
  That's exactly right. So, you know, fundamentally when people think about Okta, it's like, oh, it's, it's a single sign-on, right? You sign on once with one set of credentials, and I get access to all my work tools. That's, that's sort of where Okta started 10, 15 years ago, and now, if you're, sorry, 15, 17 years ago. But now-Identity is so much more than that. Uh, it can't just be that. It can't just be that login box. So that's a simple enabler of IT productivity. But then it's, it's all the security stuff that we were talking about, that's, that's the, the key driver right now. So as soon as you start exposing any part of your ecosystem and bringing users into your safe boundary-
11. AGAakash Gupta
  Mm
12. JHJack Hirsch
  ... that's when identity becomes critical. Because ultimately, I might be using a devi- you know, if, if you're starting a company and I'm a contractor, I might be using a device that you give me. Chances are I'm using my device. Do you know if I have malware on my device? Do you know if I'm a malicious actor? How are you logging what's happening, you know, in your ecosystem? Identity is that, the first and last fence when you don't have network control. If you go work at, you know, any of the largest organizations in your, in the world, you're gonna be using a VPN on your device. So they know that you're gonna be inside a trusted network boundary. You can't do that as a startup, right? You're not gonna do that as a startup. Same thing like, ooh, there's gonna be device management on your device. That means they're gonna put a binary on your device, and they're gonna be monitoring everything that you're doing. They're gonna be controlling what websites you go to so that you're not gonna download that malware that's gonna expose all of their corporate systems. As you're building, you don't have that.
13. AGAakash Gupta
  Yeah.
14. JHJack Hirsch
  Right? And so fundamentally, all you have is making sure the right users get access to the right thing at the right time, and that's it.
15. AGAakash Gupta
  Mm-hmm.
16. JHJack Hirsch
  So, uh, yeah, if I had one easy button for someone starting up or thinking about their security posture and their step one, it's identity, identity, identity. That's it.
17. AGAakash Gupta
  What AI security threat is coming in the next few years that companies aren't preparing for?
18. JHJack Hirsch
  Pass. [both laughing] Um, no, legitimately, I, I honestly... There is so much in flux right now that it is, it is hard to see around the bend.
19. AGAakash Gupta
  Yeah.
20. JHJack Hirsch
  I think we are-- So that message of, uh, you know, and you choose to use this or not for the-
21. AGAakash Gupta
  Mm-hmm
22. JHJack Hirsch
  ... the podcast, but fundamentally, most organizations have not done the basics, and that's not to, like, shame most of- Like, every, every organization of any consequence has breaches. Spoiler alert, everyone gets breached.
23. AGAakash Gupta
  Mm.
24. JHJack Hirsch
  Now, it's how you detect it, how fast you respond, what your internal and public responses are to that, that, that separate. Like, you know, again, I, I don't wanna pick on competitors here, but like, uh, or pick on anyone in the security space 'cause everyone gets punched in the nose. Heck, Okta got punched in the nose a couple years ago. Um, and so you just have to assume that, that that is going to happen, and there are gonna be new vectors that come up. And so securing thing, like discovering your service accounts, so if you have, like, standing credentials, you know, and you're building an app, you're using N8N internally, and you're orchestrating a bunch of things, but you, you grab those credentials to your Salesforce, and that's where all your customer data or your CRM, where all your user data is, and you hand that off to N8N, and it's just sitting there as a static credential, that's a breach vector.
25. AGAakash Gupta
  Mm.
26. JHJack Hirsch
  That means that there's a static credential that probably no one is watching. And so you're not putting that in a system where it's locked, where it's being rotated automatically, um, so that if it does get leaked, that it won't be actually be used for, uh, nefarious purposes. So there's all these, like, basics that need to be done in most organizations where like, yeah, the threat vectors will continue to evolve. But if we're as, you know, if we are not evolving our security posture as modern organizations, the basics are gonna be the thing that trip us up, not the novel AI-driven, you know, threat vector. It's all identity, identity, identity. Again, 80% today, it was 80% last year, it was 80% the year before.
27. AGAakash Gupta
  Wow.
40:09 – 46:47
Building AI-Secure Products at Okta
1. JHJack Hirsch
  Um, so again, security identity, everything else, everything can be secondary after that, and I think that will be a, a really good way to future-proof yourself.
2. AGAakash Gupta
  Okay. So we talked a lot about security. We have one last angle on security, which is building AI secure products. How has AI changed how you build products at Okta, and what are your AI security principles?
3. JHJack Hirsch
  There's sort of two ways to answer this. One is the products that we're, we're building and delivering for our customers, and the other one is how we build products in the age of AI. And I will start with how we're delivering AI products to our customers.
4. AGAakash Gupta
  Mm.
5. JHJack Hirsch
  I think, uh, we basically made the ter- determination that there are two categories of AI products for our customers. Uh, the first are essential AI services. This is what we were talking about earlier, blocking bot attacks, right? Catching credential stuffing, catching MFA bombing, those sorts of things, we don't wanna turn that off. We can't turn that off and keep the system up. It's like, it's like, uh, turning off DDoS protection. Like, we're not gonna do it, right? And so that is just part of keeping our service healthy. We run multi-tenant SaaS. If you get, uh, if you get taken down, there might be collateral damage. If someone is DDoSing you, they might, you know, have collateral damage on other tenants in that environment. Um, so we don't wanna shut that down, and that keeps us safe, that keeps all of our customers safe, and those are essential AI services. Now, there's the other tier, which are discretionary. Those are things like, "Hey, we're using an LLM to summarize how did you get access to the thing you had access to when your identity was compromised?" Or when we're doing a, an access review to make sure that, "Oh, you, you, you have access to the, uh, special, super secret Asana finance, uh, uh, Asana finance, uh, uh, workflow. Y- how did you get access to that?" Well, we probably want an LLM to summarize that because there's probably umpteen checkpoints that you went through, and I don't know. I'm, I'm just making, making the, the access mechanism up. But we wanna summarize how you got access with an LLM. That's discretionary. We can give you the raw log output, and you can have at it.Uh, if you don't want any part of your data processed by an LLM. So we give our customers the control to either opt in or opt out. So if they're buy- and, and it can be either opt in or opt out, frankly. So, like, if you're buying a, a product or a SKU from us, we assume you're opting in. If you don't want that functionality, you don't have to buy the product. Um, but if it comes by default out of the, you know, it comes by default out of the, uh, out of the box, we're gonna let our customers turn that off. So for those discretionary services, we wanna make sure we give ultimate control to our customers. Uh, we wanna make sure... I mean, we work in, you know, [chuckles] global supply chain logistics trust us with FedEx.
6. AGAakash Gupta
  Mm-hmm.
7. JHJack Hirsch
  The Department of Defense trusts Okta. So we are working... There are healthcare systems that are running on Okta, and so we need to make sure that in whatever regulatory environment, whatever security environment, we're doing the best we can to protect our customers and make sure that they're operating like they want to, so we gotta give our con- uh, that control to our customers.
8. AGAakash Gupta
  Hmm. I just wanted to cover, like, if some people wanna build, like, we just hyped up AI so much, AI threats, so as a PM or as a product builder, what are the things you need to be thinking about to make sure that your product's AI secure?
9. JHJack Hirsch
  Oh. Uh, well then, uh, I think the first thing I would do, uh, is look into the new OAuth standard that's coming out, which is Cross-App Access.
10. AGAakash Gupta
  Yeah.
11. JHJack Hirsch
  And so I would imagine that a significant portion of your audience is building B2B SaaS, is that right?
12. AGAakash Gupta
  Yeah, 100%.
13. JHJack Hirsch
  Okay.
14. AGAakash Gupta
  More than 50%, actually. [chuckles]
15. JHJack Hirsch
  Great. Uh, okay. 150 or above 50%, either way.
16. AGAakash Gupta
  Yeah, above 50 is what I meant.
17. JHJack Hirsch
  Excellent, excellent. So, uh, if you are trying to deploy software inside of a modern enterprise, use the security capabilities that are available to you to become enterprise-ready.
18. AGAakash Gupta
  Yep.
19. JHJack Hirsch
  So, uh, a year ago we announced a, a standard with the OpenID Foundation we called IPSI. Uh, the Interoperability Profile for Secure Identity in the Enterprise, but no o- no one cares about the, the, the acronym is what matters. And it's literally just make sure your app supports SSO. Make sure your app supports lifecycle management so users can be deprovision, you know, provisioned on day one, deprovisioned on their last day, so you don't have random user accounts floating around in your critical systems. It's very ba- very, very basic, like, things like that.
20. AGAakash Gupta
  Yeah.
21. JHJack Hirsch
  And then with Cross-App Access, make sure that when you're deploying into an enterprise, especially if you're building an AI agent, you should not be tapping all of those users for OAuth grants to every single downstream app they want.
22. AGAakash Gupta
  Okay.
23. JHJack Hirsch
  Right? So I'll, uh... Actually, let's, let's use n8n as an example.
24. AGAakash Gupta
  Yeah.
25. JHJack Hirsch
  So let's say that you are deploying n8n into an enterprise, right? There are two different ways you could go about doing that. One is, like, min bar access, where you say n8n has access to just, like, the corporate help desk content, right? So I wanna use this as, like, a help desk, uh, deflection tool. Well, that's not super secure content. Everyone in the organization is gonna have the same level of access. There's nothing novel that's gonna differ between your access and my access. But that's not gonna change the world. That's not gonna change how people work. What you really want is something, especially when you think about MCP and agent-to-agent access and agent-to-resource access, you want something that is gonna know, like, if you're deploying an AI agent, that is gonna know that you're you and I'm me, and we each have different levels of access. And so that's what Cross-App Access does, is it gives the enterprise the ability to say, "Hey, the, uh, security admins get access to all of these applications here," and your AI agent or your application that you're building gets access to those. So this works for AI agents, it works for to-do apps, wiki apps, knowledge management apps. Uh, and so the same way that you wanna just, like, basically, uh, provision access by using that OAuth standard-
26. AGAakash Gupta
  Yeah
27. JHJack Hirsch
  ... uh, of Cross-App Access, that is going to give the enterprise visibility and control over those OAuth grants so that your users don't have to do it, and it's gonna smooth out the delivery into any modern enterprise, uh, going forward. So there's an adoption game there. Don't get me wrong, like, it's gonna take a year or two for, for that to get adopted. But if you go to websites like Enterprise Ready and you're looking at, like, the spec sheet of stuff they have where it's like SOC 2 and, you know, oh, you must support SSO, obviously you wanna support SSO. You definitely wanna support provisioning. But I think
46:47 – 1:06:29
Fundamental AI Product Development Principles
1. JHJack Hirsch
  getting the security story straight, it will be a, an incredible unblocker to get you into any business, small or big.
2. AGAakash Gupta
  Security, we talked about it. It was the whole first half of the podcast. Now I wanna move into building products.
3. JHJack Hirsch
  Sure.
4. AGAakash Gupta
  So you lead the AI products that are being built at Okta. What are the fundamental principles you guys are following? How are you guys building AI product at Okta?
5. JHJack Hirsch
  Maybe I'll just talk about how we build using AI-
6. AGAakash Gupta
  Mm
7. JHJack Hirsch
  ... and some principles there. Um, 'cause I think, I think that's actually almost more interesting, the AI products. We talked a lot about identity. I think it's sort of how we build, which is interesting-
8. AGAakash Gupta
  Yeah
9. JHJack Hirsch
  ... and kind of the secret sauce. I think there are three principles or a few principles, and I'll just name three of them. Like, the first one is just use AI to accelerate, not abdicate responsibility.
10. AGAakash Gupta
  Mm.
11. JHJack Hirsch
  So acceleration over abdication. I think that's, that's one. Um, two is don't forget your PM fundamentals. Jumping to prototypes oftentimes leads us to solution... like jumping to solutions without really deeply understanding the problem.
12. AGAakash Gupta
  Yeah.
13. JHJack Hirsch
  And so I think that's two. And then three is, like, be aware that we're in an AI hype cycle and don't fall victim to it, right? Really focus on problems. Really focus on customer problems, on user pain points, and that will always lead the way to great products. So, uh, well, we can, we can go through this. So, like, on the acceleration, not abdication, everyone should be using AI to be better at their job, regardless of their function. Today, if you're not using AI, you are being left behind, full stop.But that doesn't mean that you wanna abdicate 100% of your responsibility. Like, I imagine that many of the people that are watching, watching this or listening to this, they are looking for ways to be better, and there's a temptation to effectively outsource your entire role to the best prompt you've ever created and the best conversation in your million token context window. Don't do that. That's terrible. Like, that's not why, that's not why you were hired, right?
14. AGAakash Gupta
  Yeah.
15. JHJack Hirsch
  You were hired because... You know, maybe use AI to get you 80% of the way there.
16. AGAakash Gupta
  Yep.
17. JHJack Hirsch
  But use your own intellect and your own creativity to hone the last 20%, because that's the difference between, like, you know, original LLM output that's super anodyne and magical content that really helps you, you know, visualize the outcome you want, that feels like you've already had the sparring partner. Um, I've had really wonderful examples of when we didn't do that, uh, we, we, we didn't- we abdicated too much responsibility. And I, I won't call those PMs out here, obviously.
18. AGAakash Gupta
  [laughs]
19. JHJack Hirsch
  But, uh, I was reading a spec, uh, once that had a bunch of competitive intelligence in it, and it looked, it looked bland in the way that if you look at a bunch of AI-written text-
20. AGAakash Gupta
  [laughs]
21. JHJack Hirsch
  ... you sort of know. You're like, you don't know, but you know without, you know, without... You don't-
22. AGAakash Gupta
  Almost sniffing it, right?
23. JHJack Hirsch
  Yeah. Yeah. You smell it, you know.
24. AGAakash Gupta
  [laughs]
25. JHJack Hirsch
  And I saw it, and I was like, just out of curiosity, I'm gonna put an AI checker, and like sure enough, 100% AI-generated. And I was like, damn it, okay, I'll just... Uh, we'll, we'll, we'll go with it. Because I assumed that they must have checked it and just used that to sort of rewrite it. This particular spec had a bunch of competitive positioning in it, and when I brought that to the field, when we brought that to the field, our, our, our sales, there was someone who had worked at that competitor, and they were like, "That is factually incorrect, and all of the assumptions that you've been using to build that product are wrong." [laughs] And so that was horrible, like, change costs for us. We had to, like, effectively go back to the drawing board.
26. AGAakash Gupta
  Oh, man.
27. JHJack Hirsch
  So please use it to accelerate, not abdicate your responsibility. We hired you because you're brilliant and you're passionate. Use that brilliance and that passion to deliver the, the last bit. Um-
28. AGAakash Gupta
  Yeah. It hallucinates a lot, though, so you really have to check the facts.
29. JHJack Hirsch
  Absolutely. And even if you're using, I mean, it doesn't matter which, you know, whether it be, you know, ChatGPT or Grok or Gemini, whatever, like, doesn't matter how good you think you've gotten your prompt, there's always gonna be hallucination. There's always gonna be... I mean, it's like, uh, using Wikipedia for your reports, you know, way back when.
30. AGAakash Gupta
  Yeah.
1:06:29 – 1:19:18
Butter.ai Startup Story (2015)
1. JHJack Hirsch
  super valuable.
2. AGAakash Gupta
  Mm. And then it sounds like engineering that board of directors with the right context.
3. JHJack Hirsch
  Absolutely.
4. AGAakash Gupta
  So you just mentioned your startup, and that's exactly where I wanted to go next, was a little bit about your background. Um, you created an AI startup well before it was cool. So tell us the story of Butter.ai. How did it come about?
5. JHJack Hirsch
  So first off, uh, the name Butter is really, really important to it. I don't know if you know why it was called Butter. You familiar, uh, you know the show Rick and Morty?
6. AGAakash Gupta
  Yes.
7. JHJack Hirsch
  Okay. So you know the Butter Bot in Rick and Morty?
8. AGAakash Gupta
  Oh, I don't, but I know-
9. JHJack Hirsch
  So, okay. So, so there was a scene in Rick and Morty where they're sitting around, they're sitting around the dining room table, they're having a conversation. Uh, and Rick, who's the, you know, sort of mad scientist, uh, mad scientist character, fiddling with a robot and it comes to life on the table. And it rolls over to him and says, "What is my purpose?" And he says, "Oh, uh, pass the butter." And the robot comes, you know, passes butter, and he says, "Thank you," and swipes a piece of butter, you know, on the toast. Conversation keeps going, the butter- the Butter Bot interrupts, and he says, "What is my purpose?" And he says, "Oh, you pass butter." And the robot looks down at his little robot hand and says, "Oh my God." And Rick's like, "Yeah, welcome to the club, pal." And so it's this like whole moment of like you realize that your purpose is just to pass butter and, uh, you know, you're sort of this bot. This agent, this AI thing is feigning self-awareness, but is it self-aware? Is it just feigning self-awareness? We don't know. It's never really discussed in the show. And so we created Butter.ai, uh, as a way of just passing the information. It was, the whole idea is like you're just gonna say, "Hey, where are the Q3 financials? How did we do last quarter?" It's gonna find the information. It's gonna find the right document. If you don't have the right access to the right document, it'll way, find its way through the organization, ask permission on your behalf if you want it to, grab that information, and surface it up to you. So that was the whole... So initially we wanted to call it DocuBot. We thought that was lame, and so we went with Butter.
10. AGAakash Gupta
  [laughs] And why did you build an AI startup? What was this, 2017?
11. JHJack Hirsch
  This was 2015.
12. AGAakash Gupta
  Wow. This is-
13. JHJack Hirsch
  2015. So timing is everything in startup, folks. Don't for- don't forget that. Timing is everything. Uh, being too early doesn't always pay.
14. AGAakash Gupta
  Yeah.
15. JHJack Hirsch
  Um, so yeah. I think we, we, uh, overshot it a little bit in terms of timing. This was like just pre-LLM-
16. AGAakash Gupta
  Mm
17. JHJack Hirsch
  ... um, LLM world. The value of it, I mean, so frankly, we were at Evernote, and we had this, we kept having this problem of like, where's that document? Where is that document? We need, we need like a docu bot. We need, and enterprise search sucks. It's terrible.
18. AGAakash Gupta
  Yeah.
19. JHJack Hirsch
  Um, and so we need something that's smarter. Search has gotten better. Let's just build that. And so this was one of like a handful of, of, uh, product ideas we were batting around back in 2015. Um, and yeah, we, we test- we did a bunch of, you know, we again, tested a bunch of prototypes. Turns out search is a Sisyphean challenge. Um, you're never gonna get it exactly quite right, and the cost of missing is really hard in enterprise search. So enterprise search is a really hard market to, to play in. It's very, very tempting, especially with LLMs, but it's a difficult market to be in.
20. AGAakash Gupta
  I think Glean is kind of in that market now.
21. JHJack Hirsch
  Absol- oh, absolutely. And Glean, if you're out there, a huge fan. Uh, they actually, I, I think, um, I think the, the enterprise, I, I don't even know if I would call them enterprise search anymore, but it's sort of where they, where they started. Um, but yeah, there's, th- this has been going for a long time. Like, you know, we keep trying to solve very similar problems. There was Coveo way back in the day. So like enterprise search is a, is a well-established place, and what's great is we have these technology leaps that create new opportunities to remake whole industries. And Glean is taking a really clever approach now, where obviously it's not just about, you know, click-through rang- click-through rate and mean reciprocal rank, and where in the search results those things are that you're finding, and making sure that it's the top one, two, or three. It's about grabbing that information, surfacing the right context out of those documents with LLMs, um, creating agentic workflows that are sort of like content aware. Really cool space to be in. Really, really hard market.
22. AGAakash Gupta
  Mm-hmm. It sounds like you didn't build, you didn't succeed as much as you wanted to with Butter. Did LLM unlock something specifically for Glean that wasn't there when you were working on Butter?
23. JHJack Hirsch
  Oh, a- absolutely. Um, absolutely. I think we spent a, we spent a lot of sort of organizational calories on things like text summarization, NLP. Remember that?
24. AGAakash Gupta
  Yeah.
25. JHJack Hirsch
  Remember natural language processing, understanding. So this is, you know, the first, the first iteration of this was a Slack bot where you could just, you could add it to a bunch of channels, you can @mention it, you do whatever, and it would just go find you the bot. And so Slack was the original interface. We also did, um, we built a Chrome extension, so if you're searching like Google, but your result happened to be in Dropbox, it would surface, say, "Actually, the thing I think you're looking for is in Dropbox," or, "The thing that you're looking for is in Notion," or the, you know, somewhere else. So the interface, you know, there were a few interfaces to it. Um, but yeah, I, I wish we had the technology of LLMs. I think it might have changed some things, but fundamentally it's a knowledge management challenge. It's really, really hard, uh, to get right, and I think that, uh, from a business perspective, it's, it's a difficult place to be in. I'd much rather be in the path of revenue. I'd much rather be helping customers or users make money, save money, be more secure. There are some like core value props that I think are much more attractive from a business standpoint. Um, and I think that, you know, enterprise, like productivity on a personal level, so, so anchored to it, so gravitated. Like I love the idea of augmenting, you know, myself as a human. I felt like Evernote, when I was using it in grad school, gave me superpowers, 'cause I could dump everything in there. I could search through case studies and documentation and notes and everything, and I just like had it all at my fingertips, and I felt superhuman as I was doing that. And that, that I think is, you know, enhancing personal productivity is a virtuous cause. I love it.But it's always considered discretionary spending.
26. AGAakash Gupta
  Yeah.
27. JHJack Hirsch
  And so it was very, very subject to, you know, uh, economic dips. Um, and, uh, you know, in a business context, especially in the past few years with everything been, being so topsy-turvy, that discretionary spending is the first thing to get cut. Um, and so I'd much rather be in those mission critical industries, um, and delivering mission critical outcomes to customers or users.
28. AGAakash Gupta
  So you're living the dream. You're a founder. You get acquired, and now for the last, I guess, seven or eight years, you've been a product leader. What has that transition been like? What's kept you in product leadership?
29. JHJack Hirsch
  The-- Well, I will say this, and this is, uh, I love Box, uh, love Box, love all my former colleagues at Box, love my current, uh, people that are still there. Uh, still a huge fan of the people there. But I described it, uh, to the now CTO at Box that I felt like I was a bullet hitting jello.
30. AGAakash Gupta
  [chuckles]
1:19:18 – 1:20:48
Why Evernote Failed Despite Early Success
1. AGAakash Gupta
  and we have to talk about it because it was like the king of the world. Where I operate in the productivity space, building a second brain, that was like everything. And like you said, that aha moment when you start searching your Evernote database, your notes, you're making connections you didn't think about. It was wonderful back in 2017. Why did it all fall apart?
2. JHJack Hirsch
  Oh my gosh, it was still wonderful in 2017?
3. AGAakash Gupta
  [laughs] I think so.
4. JHJack Hirsch
  Yeah, I do. [laughs] Uh, yeah. So I guess where, where do you wanna start?
5. AGAakash Gupta
  Well, why did it-
6. JHJack Hirsch
  Why did-
7. AGAakash Gupta
  Why is it, you know, not one of the big Silicon Valley success stories?
8. JHJack Hirsch
  Um, that's... Okay, that's a good question. Um, I'd say probably... Let's see. Let- I'll give it three reasons. Um, one is personal productivity just has a limited TAM, okay? So that's, that's one. I'd say two is Evernote was inherently single player. When Evernote was born, I think it was 2008, 2007, 2008, it was the era of social media when everything belonged to the world, right? And Evernote was yours. It was your private thoughts. And so everything from the ground up was architected to be single player. And to graduate to the next level, uh, Evernote needed to go multiplayer.
9. AGAakash Gupta
  Mm.
10. JHJack Hirsch
  And so when some... Uh, so that, that was probably two. And then three is I think we raised too much money-
11. AGAakash Gupta
  Mm
12. JHJack Hirsch
  ... frankly, and we got out over our skis. And so I think there are other, like, compounding factors there. I think, you know, there was a breach that really... That, like, data-wise, that set us back between
1:20:48 – 1:26:51
Hustling Into PM at Evernote
1. JHJack Hirsch
  6 and 12 months-
2. AGAakash Gupta
  Wow
3. JHJack Hirsch
  ... which in VC-funded world, especially late stage, that is an absolute killer, right? So I think there's, there's other, other factors there, but I'd say those three are probably the biggest ones. So yeah, TAM, uh, inability for, to transition from single to multiplayer, um, and just raising too much money.
4. AGAakash Gupta
  So you had a really interesting story about how you hustled your way into PM at Evernote. How did that happen?
5. JHJack Hirsch
  That was actually really funny. So it started here in San Francisco. I was sitting in Dolores, uh, with a good friend of mine, and he asked me, "Where, where do you wanna go? What do you wanna work on after you graduate from your MBA? Do you wanna work at a place like Google?" And I said, "Oh, you know, as much as I love Google, I wanna work for something I'm, like, deeply passionate about, something I love. Like, I wanna work on a product that I love, something like Evernote, you know?" And he was like, "Whoa, my dad works at Evernote."
6. AGAakash Gupta
  Oh.
7. JHJack Hirsch
  And I was like, "Oh, interesting." Okay. So I went home. I was going to school in Baltimore at the time, so I went all the way back across the country. Uh, on Evernote's website at the time, they had a bunch of coding challenges. So they didn't have any PM roles posted. They had a bunch of coding challenges for engineers, and I solved all of the coding challenges. I printed them out on paper, I mailed them to my friend, and I said, "Hey, put these on your dad's desk." And on the, the cover sheet was, "Put on Phil Constantino," the then VP of Product's desk.
8. AGAakash Gupta
  Mm.
9. JHJack Hirsch
  And so I got a call a few days later from the then VP of Product, and he basically asked, "Why am I talking to you?"
10. AGAakash Gupta
  [laughs]
11. JHJack Hirsch
  And this was, like, you know, uh, second year. This is post summer internship of an, a two-year MBA program, and this is, like, maybe November or December of that year. And he was like, "Why am I talking to you?" And I was like, "Uh, uh, uh, a, a winter internship."
12. AGAakash Gupta
  Oh.
13. JHJack Hirsch
  And he said, "Oh. How much time do you get off your winter internship?" And I'm like, "I don't know, three, four weeks." He's like, "Can you do something in three, four weeks?" I was like, "Absolutely, I can do a lot in three to four weeks. Put me in, coach."
14. AGAakash Gupta
  [laughs]
15. JHJack Hirsch
  And he's like, "How often are you in California?" And I was like, "Oh, I'm in California all the time. I'm actually gonna be there next week." I literally opened up southwest.com-
16. AGAakash Gupta
  [laughs]
17. JHJack Hirsch
  ... as we're on that call. I flew myself, like, the next day across the country, and I showed up at their office like everything was cool and I was meant to be there.
18. AGAakash Gupta
  Wow.
19. JHJack Hirsch
  Um, and you know, I had, uh, basically three weeks to prove myself. Um, and then the, uh, CEO, uh, Phil Libin was like, "Hey, you wanna join, you gotta join now. We're not gonna wait six months."
20. AGAakash Gupta
  Oh.
21. JHJack Hirsch
  And I called my school and I said, "Hey, how many classes can I miss?" [laughs]
22. AGAakash Gupta
  Uh-huh.
23. JHJack Hirsch
  They gave me the exact number, and that is exactly the number that I missed, and I was basically flying my way back and forth across the country. And I mean, at the time, like, the team that I, I joined, like, they had just been on the cover of Inc. magazine. Like, they were flying high. This was pre-billion dollar, you know, billion-plus valuation, but it was, like, the start of this idea of, like, you know, bettering yourself, right? It was... It really was that iconic personal productivity brand that was blooming-
24. AGAakash Gupta
  Mm
25. JHJack Hirsch
  ... and just, like, crossing the chasm into mainstream, and I just, I wanted to find my seat on that rocket ship, and I wouldn't trade it for the world. But yeah, that was my, my intro to PM, um, was starting, starting effectively as an intern.
26. AGAakash Gupta
  That's insane hustle, though. So how can people reverse engineer that? What... I think, I guess, some of the lessons, right, network and location. You being in SF and talking to a friend in SF helped because his dad was at Evernote. Number two, like, thinking super creatively, I guess, about how to get in, right? Doing coding challenges for a PM interview, very rare. And then I think third thing, taking a lot of initiative to just, "Yeah, I'll do an internship for [laughs] three to four weeks," right? Nobody does that. Are there any other elements people should focus on?
27. JHJack Hirsch
  I mean, I think... So when hunting for roles, I think it's really natural for people to treat it like-I don't know, online dating.
28. AGAakash Gupta
  [laughs]
29. JHJack Hirsch
  Just, like, swipe left, swipe... I mean, there are tools now, right? Like, oh, you swipe right on a job and an LLM will apply for you.
30. AGAakash Gupta
  Yep.
1:26:51 – 1:30:36
Personal Identity Protection Guide
1. AGAakash Gupta
  people, AI cybersecurity attacks, they're crazy, and they're gonna continue to get more and more sophisticated. We walked through the playbook companies must have. Let's end on the personal angle. Let's say my identity gets breached tomorrow by some ibers- AI cybersecurity threat. What's the one thing I can do to save myself?
2. JHJack Hirsch
  Number one, so your physical identity, you, your Social Security number, your credit score, all of that is very, very easy to breach.
3. AGAakash Gupta
  Mm.
4. JHJack Hirsch
  Like trivial, trivial to breach. Lock your security reports. Like, it's so easy. Just go to the major credit bureaus, TransUnion, Equi- like, just go to them. There's online flows. They'll try to get you to pay for them. You don't have to pay. Go lock your credit reports.
5. AGAakash Gupta
  Oh.
6. JHJack Hirsch
  That means that any time someone tries to open a checking account in your name, a credit card, sign up for a phone, file an insurance claim, or sign up for an insurance, uh, uh, sign up for insurance, file an insurance claim against your current insurance, all of that stuff effectively gets blocked-
7. AGAakash Gupta
  Mm
8. JHJack Hirsch
  ... while your credit report is frozen. So I think your physical identity is this horrible vestige [laughs] that we need to be securing. So I think that is, that is number one, and it can be absolutely ruinous if you have had this happen to you. I have actually had my physical identity compromised when I was buying a house.
9. AGAakash Gupta
  [lips smack] Oh, man.
10. JHJack Hirsch
  So I had to, I unlocked my credit report. There was paper, that I got mail stolen. I hadn't locked... I didn't know that I had mail stolen. Um, and then I found out that people had used my, someone had used my Social Security number. Um, so locking your credit report is number one. Then there's, like, digital security. So I think physical security, so easy to breach. Lock it down. Digital security, goodness gracious, please, please, please, please use a password manager. Use strong passwords. Use different passwords for every site.
11. AGAakash Gupta
  Mm-hmm.
12. JHJack Hirsch
  Uh, use passkeys. If you haven't used a passkey, it's magical. Uh, passkeys, uh, actually, we can get into why they're so magical and why they're so awesome and why they're so secure. You're not using a password. You are just using a biometric, some sort of biometric marker for yourself, and that's a multi-factor in and of itself-
13. AGAakash Gupta
  Yeah
14. JHJack Hirsch
  ... because you're using a possession factor and an inherence factor, your possession of your device and something that is inherent to you, like your fingerprint or your face ID.
15. AGAakash Gupta
  Yep.
16. JHJack Hirsch
  So password manager, passkeys, please, and then last one, last bonus, lock your phone number.
17. AGAakash Gupta
  Oh.
18. JHJack Hirsch
  So, uh, oftentimes you will get, you know, f- probably if you don't use a super modern bank, you'll get MFA codes via SMS. SMS is really, really easy to, to breach. Um, and so, uh, make sure that your phone number is locked with your provider. So I, I can speak confidently in the United States, every single phone provider gives you the option to give you some sort of a PIN code or secondary security lock that makes it so that you can't steal the phone number and transfer it to another third-party SIM. So stealing someone's, effectively, their phone number means that if they do happen to get your password, your MFA is compromised, right?
19. AGAakash Gupta
  Yeah.
20. JHJack Hirsch
  Um, so SMS is a really bad second factor, it turns out.
21. AGAakash Gupta
  Mm.
22. JHJack Hirsch
  So it's not phishing-resistant. Um, and so yeah, I would say that's probably good. So physical identity with, uh, your credit reports, use a password manager, use passkeys. Um, yeah, that's probably a, a great start.
23. AGAakash Gupta
  This was a masterclass not just in security, but building AI products, using AI to build products. Thank
1:30:36 – 1:31:18
Outro
1. AGAakash Gupta
  you so much, Jack.
2. JHJack Hirsch
  Thanks for having me.
3. AGAakash Gupta
  All right. Bye, everybody. So if you wanna learn more about how to shift to this way of working, check out our full conversation on Apple or Spotify Podcasts. And if you want the actual documents that we showed, the tools and frameworks and public links, be sure to check out my newsletter post with all of the details. Finally, thank you so much for watching. It would really mean a lot if you could make sure you are subscribed on YouTube, following on Apple or Spotify Podcasts, and leave us a review on those platforms. That really helps grow the podcast and support our work so that we can do bigger and better productions. I'll see you in the next one.

Episode duration: 1:31:27

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode 4dWbwGCPjUY

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

Add to Chrome