Skip to content
Aakash GuptaAakash Gupta

What AI PMs REALLY Need to KNOW in 2026 (Agents, Discovery, EVERYTHING)

Todd Olson spent 28 years in product management and built Pendo to $2.5B. He reveals why AI PM jobs doubled to 20% of all postings (and pay 30-40% more), the exact 5-layer technical pyramid to upskill from core PM to AI PM, and how to ship AI features at scale with proper evals, cost optimization, and the right product strategy. Full Writeup: https://www.news.aakashg.com/p/todd-olson-podcast Transcript: https://www.aakashg.com/the-complete-ai-pm-roadmap-how-to-upskill-from-core-pm-to-ai-pm-with-pendo-ceo-todd-olson/ ---- Timestamps: 0:00 - Intro 1:29 - Episode Begins 3:24 - Why AI PMs Get Paid 30-40% More 6:07 - How to Upskill into AI PM 11:50 - Ad 12:54 - The 5-Layer Technical Pyramid 16:30 - AI/ML Fundamentals 23:00 - Data Pipelines & RAG 33:02 - Trace Analysis & PM-Eng Tension 40:44 - Cost & Performance Optimization 48:56 - Evals Are Your Domain 56:03 - AI Product Roadmap 1:04:16 - Live Demo: Pendo's AI Features 1:13:07 - Ad 1:14:12 - Stakeholder & Board Management 1:22:03 - Outro ---- 🏆 Thanks to our sponsor: Reforge: Get 1 month free of Reforge Build with code BUILD - https://reforge.com/aakash ---- Key Takeaways: 1. AI PM market exploded - Last year 10% of PM jobs were AI PM jobs. This year it's 20%. They pay 30-40% more because of scarcity and skill level. But Todd warns: "You better damn well be good and know what you're talking about if you're gonna call yourself an AI PM because we are going to interrogate the hell out of it." 2. Real requirement is production at scale - Not "I built prototype at 1-person startup." Hiring managers want 20,000 paying B2B customers experiencing your AI feature successfully. To get there: upskill internally at current company by shipping AI features on your roadmap. 3. The 5-layer technical pyramid - Foundation: AI/ML fundamentals, data pipelines, prompt engineering. Middle: Observability (trace analysis), cost optimization, evals. Top: Product strategy, stakeholder management, leadership. You need to climb all 5 layers. Most PMs stop at layer 1. 4. RAG is table stakes - "RAG is the de facto way to build." You ingest data, create embeddings, feed into vector database, look up relevant context, pass to LLM. Todd: "If you put too much in context window, just like a human, you get confused. You want to give the right context." 5. PM-engineering tension is real - At startups, PMs do trace analysis. At large companies, engineering managers push back: "This is my world. I don't want some PM shadowing me." Similar to Data Dog—most PMs don't have login. Know the line. Be fluent but respect boundaries. 6. But evals are YOUR domain - Unlike trace analysis, evals are where PMs are the expert. "The PM is probably the best-suited human being to author and manage eval sets." You understand user and business needs. Engineers don't have that context. This is must-have competency now. 7. Cost optimization will matter - Some AI companies have sub-15% gross margins. Traditional software is 70-80%. Todd: "It's not a business at sub-15%." Eventually you'll rearchitect systems because infrastructure is too costly. Rule: when something's faster, it's cheaper (both buying compute). 8. Solve hard problems, not shiny objects - Todd's test: "Are we gonna do much better job than ChatGPT out of box? Why would we just wrap that and slap Pendo logo on it?" His discovery agent example: hard part isn't interviewing customers—it's finding which to interview, prioritizing, scheduling. Automate that workflow. 9. Kill bad features ruthlessly - Todd shipped features couple years ago that weren't great and turned them off. "Too often we hold on to something. Turn them off. Be unafraid. The more stuff in your product, the worse the experience is by default." 10. Control the narrative with boards - Don't show up with no story and get crushed with random requests. Todd: "Show them how you actually run your business. I want to see what you're looking at, not something just made for me." Think deeply about how each bet drives shareholder value. ---- 👨‍💻 Where to find Todd Olson: LinkedIn: https://www.linkedin.com/in/toddaolson/ Twitter/X: https://x.com/tolson Company: https://www.pendo.io 👨‍💻 Where to find Aakash: Twitter: https://www.x.com/aakashg0 LinkedIn: https://www.linkedin.com/in/aagupta/ Newsletter: https://www.news.aakashg.com #aipm #productmanagement #pendo ---- 🧠 About Product Growth: The world's largest podcast focused solely on product + growth, with over 200K+ listeners. 🔔 Subscribe and turn on notifications to get more videos like this.

Aakash GuptahostTodd Olsonguest
Dec 3, 20251h 21mWatch on YouTube ↗

CHAPTERS

  1. 0:00 – 5:04

    AI PM hiring surge: what job-posting data does (and doesn’t) mean

    Aakash opens with a striking stat: AI PM job postings doubled year over year. Todd adds nuance: job postings are marketing signals as much as they are real capability needs, and the line between “AI PM” and “regular PM” is blurring fast as AI becomes embedded everywhere.

    • AI PM postings grew from ~10% to ~20% of PM listings year over year
    • Job listings reflect employer branding and candidate signaling, not just true scope
    • AI fluency is becoming table-stakes across PM roles
    • AI tech changes weekly; recency of experience can beat “years of experience”
    • Most companies will use AI broadly, not only through dedicated AI PM roles
  2. 5:04 – 8:15

    AI as a PM productivity multiplier—and as an invisible product ingredient

    Todd separates two AI realities: using AI to do your job faster and using AI inside customer-facing products. The takeaway is that AI often improves features even when it’s not marketed as “AI,” and PMs who don’t adopt AI tools will be outpaced.

    • Two AI uses: PM workflow acceleration vs. AI-powered product experiences
    • Prototype faster with tools like Replit/Bolt/V0; don’t wait on others to validate ideas
    • Use LLMs for research (e.g., Deep Research for landscape scans)
    • If your product touches text, calling LLM APIs is often a default option
    • Many AI-powered implementations won’t be branded as AI features (e.g., AI-assisted CSS generation)
  3. 8:15 – 13:45

    Why AI PMs command 30–40% higher pay (and the ‘AI washing’ trap)

    Todd explains compensation premiums as a mix of market heat and skill scarcity, similar to other highly technical PM specializations. He warns that companies will rigorously test claimed AI expertise because resume “AI washing” is rampant.

    • Pay premiums come from hot demand and scarce proven experience
    • AI may enable fewer people to do more, temporarily widening productivity gaps
    • Technical specialties historically pay more (analytics PMs, technical PMs)
    • Hiring processes will aggressively validate AI claims
    • Avoid superficial certificates and inflated branding; credibility is critical
  4. 13:45 – 15:30

    Upskilling roadmap: the 5-layer pyramid for becoming effective in AI products

    Aakash introduces a structured progression: foundations (ML basics, data pipelines, prompts), then observability/monitoring/cost, then evals, then strategy/roadmap/stakeholders, and finally leadership topics like ethics and culture. This frames the rest of the episode as a step-by-step guide.

    • Bottom layer: AI/ML fundamentals, data pipelines/RAG, prompt engineering
    • Next: observability, trace analysis, production monitoring, cost optimization
    • Then: evals and QA become core PM work for AI systems
    • Higher layers: AI product strategy, roadmaps, and stakeholder management
    • Top: leadership topics—ethics, safety, privacy, team building
  5. 15:30 – 19:06

    AI/ML fundamentals: model tradeoffs, privacy/security, and staying current

    Todd emphasizes that fundamentals are less about theory and more about hands-on familiarity with models, capabilities, and constraints. The conversation highlights model selection tradeoffs, multimodality, open-source options, and how enterprise privacy and vendor constraints shape decisions.

    • Build firsthand intuition by using models and tools continuously
    • Understand tradeoffs: quality vs. speed vs. cost across models/providers
    • Enterprise constraints matter: privacy, security, residency, DPAs, country availability
    • Open-source/self-hosted models can simplify compliance and customer adoption
    • Model landscape changes constantly; testing new models becomes ongoing work
  6. 19:06 – 22:00

    Data pipelines & RAG: context windows, embeddings, vector DBs, and scale reality

    RAG is positioned as the default pattern for many AI product features because it’s how teams provide the right context to models. Todd breaks down embeddings/vector search and explains why PMs must understand performance, governance, and scaling implications even if they don’t implement the pipeline themselves.

    • RAG supplies relevant context without overloading the context window
    • Core mechanics: ingest data → create embeddings → store in vector DB → retrieve context
    • Too much context can degrade outputs (confusion similar to humans)
    • Scale and performance constraints emerge quickly in real-world deployments
    • PMs benefit from fluency to make good tradeoffs and partner with engineering
  7. 22:00 – 24:22

    Prompt engineering: real skill, plus the rise of AI platform PM specialization

    Prompting is reframed as structured instruction and context design, akin to being good at search. Todd predicts increasing role specialization, with platform-oriented AI PMs enabling domain PMs, while everyone still needs baseline prompting competence.

    • Better instruction/context yields better model behavior and reliability
    • Prompting is a practical skill, not just social-media “prompt hacks”
    • Domain experts will shape prompts to encode business/user intent
    • Emerging role split: AI platform PMs vs. domain/product PMs
    • Even with specialization, prompt literacy becomes universal for PMs
  8. 24:22 – 29:51

    Trace analysis & observability: debugging agents—and navigating PM–engineering boundaries

    As systems become more agentic (agents calling tools and other agents), understanding traces helps diagnose errors and performance bottlenecks. Todd notes real ownership tension: PMs should be conversant and helpful during incidents but may not own tooling or day-to-day tracing in larger orgs.

    • Agent orchestration increases complexity; traces show what happened across chains
    • Tracing helps pinpoint errors, retries, recoveries, and latency hotspots
    • PM–engineering tension often centers on “who owns how it’s built”
    • Recommended stance: PMs learn enough to ask smart questions and assist in P0s
    • At scale, dedicated teams may own observability; PMs contribute selectively
  9. 29:51 – 32:00

    Production monitoring realities: ops/SRE ownership, access constraints, and company context

    Todd draws a clear line between understanding and ownership: in many companies, ops/SRE teams handle production monitoring and incident response. He also highlights practical access controls—contracts, data restrictions, and background checks—that limit who can touch customer systems.

    • In many orgs, ops/SRE teams—not PMs—run monitoring and incident workflows
    • PM role varies dramatically by company size, maturity, and structure
    • Customer contracts and security policies restrict system/data access
    • Background checks and limited-access roles create enforced separation of duties
    • PMs should adapt expectations to their environment while staying informed
  10. 32:00 – 37:06

    Cost & performance optimization: gross margins, caching, and the ‘vibe-coded’ risk

    AI features introduce real COGS and gross margin pressure, making cost optimization a product concern, not just an engineering detail. Todd explains why the industry will likely move toward smaller/tuned models and why performance improvements often reduce cost; Aakash adds the pattern of optimizing quality first, then cost.

    • AI systems directly impact COGS and long-term business viability
    • Many AI companies run unattractive margins today; pressure to rationalize will grow
    • Future trend: smaller/tuned models once quality gains hit diminishing returns
    • Performance and cost are tightly linked; faster often equals cheaper compute
    • Caching and system design matter; auto-generated/vibe-coded apps ignore cost by default
  11. 37:06 – 46:54

    Evals & experimentation: why evaluation is now a core PM domain

    Todd argues evals are more PM-owned than other technical layers because PMs best understand user intent and business outcomes. They discuss how AI lowers the cost of generating variants, making experimentation more central, and how metrics may shift from activity to outcome completion in agentic products.

    • Evals are ‘AI grading AI’; quality of eval sets strongly drives product quality
    • PMs are best positioned to author/manage eval criteria and datasets
    • Engineers provide harness/tooling, but PMs define what ‘good’ looks like
    • AI reduces the cost of variants, increasing cadence and importance of experiments
    • Metrics shift toward outcomes and sentiment/frustration signals (e.g., rage prompts)
  12. 46:54 – 54:01

    AI product roadmapping & discovery: solve hard problems, ship a POV, and kill weak features

    Todd advises against ‘wrapping ChatGPT’ and instead focusing on hard, tedious workflows where proprietary context creates real differentiation. He emphasizes ruthless pruning of low-retention AI features and articulates a strategic POV: workflows over roles, with AI as a cross-cutting modality across products.

    • Start with hard problems and unique assets/data—not AI for AI’s sake
    • Discovery pain points (e.g., finding the right users to interview) are prime targets
    • Be willing to throw away/turn off weak AI features quickly
    • Develop a clear strategic POV; don’t copy competitors’ ‘agent with a title’ approach
    • Future orientation: workflows matter more than org charts; humans + agents collaborate end-to-end
  13. 54:01 – 59:05

    Stakeholder & board management in the AI era: control the narrative and show roll-up to value

    Todd explains boards respond to the narrative you present; show up without one and you’ll get dragged by shiny-object requests. He recommends using the board as a working partner, bringing specific questions for input, and consistently tying AI bets to business objectives and shareholder value.

    • Boards mirror what you present—control the narrative with a clear story
    • Treat board members as aligned partners with broader market exposure
    • Respond to AI feature pressure with evidence: research findings and tradeoffs
    • Don’t create ‘board-only’ artifacts; show how you truly run the business
    • Translate AI roadmap into growth, outcomes, and shareholder value impact
  14. 59:05 – 1:03:17

    Leadership layer: privacy, ethics, regional constraints—and building with ‘toggles’ for uncertainty

    At the highest level, Todd focuses on aligning AI work with existing compliance posture, anticipating bias/safety risks, and accounting for regional and industry constraints (e.g., Germany’s works councils, HIPAA). A key execution principle is adaptability: build systems with switches so you can change behavior quickly without pulling the product.

    • AI must comply with existing privacy/compliance commitments
    • Bias/fairness and guardrails matter; prevent unsafe or off-domain responses
    • Regional/industry constraints can block features (data residency, works councils, regulated sectors)
    • Regulation uncertainty is real; pendulum swings can force rapid changes
    • Architect with toggles/switches to disable or modify AI behavior per segment/region
  15. 1:03:17 – 1:21:53

    Live demo: Pendo’s agent analytics, discovery automation, and cross-source insight synthesis

    Todd demonstrates Pendo’s AI capabilities: agent analytics (conversations, retention, rage prompts), journey/path analysis around agent use, and interactive ‘agent mode’ for analyzing segments. He also shows discovery automation (finding interview candidates and triggering guides) and synthesis across enterprise sources (Gong, support, Salesforce) to surface top feature requests and create linked ideas.

    • Agent analytics: prompts/conversations, retention by use case, rage prompts, replay-backed debugging
    • Hybrid UX analysis: paths and journeys before/after interacting with an agent
    • Dashboards as self-serve PM artifacts tying AI investment to outcome improvements
    • Agent mode: guided questions to avoid hallucination; interactive, multimodal outputs
    • Discovery and insights: auto-identify interview users, launch guides/Calendly flows, synthesize feedback across systems into actionable ideas

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.