Skip to content
Aakash GuptaAakash Gupta

Masterclass: How to Turn an AI Agent into a Real Product (No Code)

Tyler Fisk reveals how to build AI agents that actually work in production. He breaks down the meta-prompting system, shows how to build multi-agent architectures live, and demonstrates why most teams fail at production agents. --- Full Writeup: https://www.news.aakashg.com/p/tyler-fisk-podcast Transcript: https://www.aakashg.com/how-to-build-production-ai-agents-complete-masterclass-with-tyler-fisk/ --- Timestamps: 00:00:00 - Intro 00:01:34 - Tyler's Background: AI Agent Expert 00:03:00 - Live Demo: Building Apple Customer Service 00:04:12 - Gigawatt: The Agent That Builds Agents 00:16:08 - Ads 00:17:00 - Deep Research & Knowledge Base Setup 00:26:02 - Why Multi-Agent Systems Beat Single Agents 00:30:00 - Temperature Settings Explained (Ice Peak) 00:34:05 - Ads 00:39:47 - System Instructions & Meta-Prompting 00:51:26 - Testing Core & Echo Agents Together 01:05:50 - RAG Databases & Enterprise Documents 01:24:11 - Production Workflow with Human-in-Loop 01:35:10 - $1.6M Course Business Results 01:39:53 - Outro --- 🏆 Thanks to our sponsors: 1. Maven: Get $135 off Tyler’s course with my code AAKASHxMAVEN - https://maven.com/sara-davison/scale-with-aiworkflows-foundations?promoCode=AAKASHxMAVEN 2. Vanta: Get $1,000 off AI security & compliance at vanta.com/acos - http://vanta.com/aakash 3. Testkube: Leading test orchestration platform - http://testkube.io/ 4. Kameleoon: Leading AI experimentation platform - http://www.kameleoon.com/ 5. The AI Evals Course for PMs: Get $1155 off with code ‘ag-evals’ - https://maven.com/parlance-labs/evals?promoCode=ag-evlas --- Key takeaways: 1. Stop Vibe Coding: Most teams write one prompt, test twice, ship to production, and hope for the best. Tyler's rule: "We would never put it into production without a human-in-the-loop checkpoint. 2. Use Meta-Prompting to Build Agents: Tyler built Gigawatt—an agent with 72,000 characters of system instructions that builds other agents. It researches the domain, writes V1 instructions, evaluates itself (scores out of 100), identifies gaps, and rewrites to V2. Goes from 77% to 86%+ quality. 3. Build Multi-Agent Architectures: Don't build one agent that does everything. Separate concerns like you'd separate teams. For Apple: Core (expert agent, temp=0, finds facts) + Echo (email agent, temp=0.7, writes responses). 4. System Instructions Need 7K-9K Tokens: Structure includes Role (job description), Context (business details), Instructions (step-by-step process), Criteria (guardrails), Examples (meta reasoning). 5. Temperature Is Your Secret Weapon: Tyler's Toy Story analogy: Imagine an icy peak in a claw machine. Temp=0 (frozen): claw picks from top only—deterministic, precise. Temp=1 (melted): claw grabs anywhere—creative, varied. 6. Information Hierarchy Prevents Hallucinations: Priority order: RAG database first (scraped company docs), System instructions second (built-in expertise), Web search third (with chain-of-verification). When agents search without verification, they hallucinate. 7. Build Complete Workflows: Tyler's 9-step production workflow with 5+ agents: Email arrives → Sentiment analysis (Cinnamon) → Expert research (Core) → Email writing (Echo) → QA loop → Human checkpoint (Slack) → Generative filter → Send → Log to memory. 8. Observational Evals Come First: Test 20+ different scenarios manually. Include edge cases and adversarial inputs. Document every failure. Save golden examples. Only after building confidence do you add systematic evals in production. 9. Calculate ROI as Labor Cost Reduction: Traditional cost: $460/day (expert time + customer service rep + manager review) = $138K/year. AI cost: $153/day (platform fees + API credits + human review) = $45.9K/year. Savings: $92K annual (67% reduction). 10. Emotion Prompting Actually Works: Tyler ends every prompt with "Go get 'em slugger." Based on research: positive reinforcement improves LLM outputs by ~15%. The same psychology that works on humans works on LLMs. "Be nice to your AI. They're gonna have robot bodies soon." --- 👨‍💻 Where to find Tyler: Instagram: https://www.instagram.com/tyfisk/ LinkedIn: https://www.linkedin.com/in/tyfisk/ AI Build Labs: https://www.linkedin.com/company/ai-build-lab/ --- 👨‍💻 Where to find Aakash: Twitter: twitter.com/aakashg0 LinkedIn: linkedin.com/in/aagupta/ Newsletter: news.aakashg.com #aiagents #productmanagement 🧠 About Product Growth: The world's largest podcast focused solely on product + growth, with over 187K listeners. Hosted by Aakash Gupta, who spent 16 years in PM, rising to VP of product, this 2x/week show covers product and growth topics in depth. 🔔 Subscribe and turn on notifications to get more vidoes like this.

Aakash GuptahostTyler Fiskguest
Oct 16, 20251h 40mWatch on YouTube ↗

CHAPTERS

  1. 0:00 – 2:59

    Production AI agents without a CS degree: what you’ll build today

    Aakash introduces Tyler Fisk and frames the goal: build a production-style, multi-agent workflow live—without code. Tyler sets expectations for going from idea to coordinated agents that work together in real time.

    • Goal: build multiple agents that orchestrate as a workflow
    • No CS degree required; practical agent-building is accessible
    • Focus on productionization vs. “vibe coding” demos
    • Live build constraints: speed, clarity, and iteration
  2. 2:59 – 4:08

    Tooling setup: TypingMind + “Gigawatt” agent + MCP tools

    Tyler opens in TypingMind and introduces Gigawatt, his prompt/AI engineering agent. He explains why tool access (Exa, Perplexity, sequential thinking) matters and how his process starts with practitioner-style discovery.

    • TypingMind as an LLM playground with API-layer control
    • Gigawatt as a dedicated prompt engineering / AI engineering assistant
    • MCP/tools enabled: Exa, Perplexity, Sequential Thinking
    • Discovery mindset: understand business/problem before building
  3. 4:08 – 10:01

    Defining the Apple customer-service workflow and agent roles (Core + “You’ve Got Mail”)

    They decide to build for Apple: one expert agent that researches and supplies verified answers, and a separate email-writing agent that responds to customers. Tyler gathers clarifying requirements to prevent role confusion and ensure clean handoffs.

    • Use case: inbound Apple customer-service emails (broad coverage)
    • Two-agent architecture: expert researcher (Core) + email responder
    • Clarifying questions to constrain scope and expectations
    • Information hierarchy: RAG first, then system knowledge, then web with verification
  4. 10:01 – 14:16

    How to recreate Gigawatt: meta-prompting, research frameworks, and verification

    Aakash asks how someone could replicate Gigawatt without Tyler’s course/product. Tyler explains his approach: collect best-practice prompting techniques, incorporate research (e.g., chain-of-verification), and let LLMs help write strong prompts.

    • Gigawatt productization plans vs. course distribution
    • Rebuilding approach: codify your own “what good looks like”
    • Chain-of-verification to reduce hallucinations (confidence + fact-checking)
    • Meta-prompting: agents helping build/improve other agents
  5. 14:16 – 22:51

    RAG knowledge base setup: domain scrape + deep research parallelization

    Tyler sets up a RAG knowledge base by scraping Apple’s domain in Cassidy AI, then spins up deep research tasks in parallel. He introduces a second agent (“Clear”) specialized in writing deep research prompts for Perplexity/Claude.

    • Cassidy AI used for no-code scraping and knowledge base creation
    • Domain-level scrape (1,000 pages) to seed RAG quickly
    • Clear agent generates deep research prompts for multiple tools
    • Parallel execution: multiple agents + multiple research engines
  6. 22:51 – 26:30

    Building the PRD for the expert agent: alignment before prompting

    With research running, Tyler has Gigawatt produce a PRD for the expert agent (Core). They emphasize why PRDs prevent agents from mixing roles (e.g., expert accidentally drafting customer-facing emails).

    • PRD as structured alignment: what it does/doesn’t do, outputs, audience
    • Core’s audience is another agent, not the end customer
    • Avoid role mix-ups by explicit inputs/outputs
    • Planning before building increases reliability and speed
  7. 26:30 – 32:25

    Why multi-agent beats single-agent + temperature explained (Ice Peak analogy)

    Tyler explains multi-agent design using a real business analogy: experts aren’t always good customer communicators. He then explains temperature as controlling randomness/creativity using the “claw machine / icy peak” probability distribution analogy.

    • Separate domain expertise from customer-facing writing for better outputs
    • Different agents can use different temperatures/settings
    • Temperature controls probability distribution of next-token selection
    • Low temp = deterministic; high temp = more creative but riskier
  8. 32:25 – 38:12

    System-instruction engineering for Core: XML structure, examples, and emotion prompting

    Tyler switches to prompt-engineering mode to generate Core’s system instructions in XML (role/context/instructions/criteria/examples). He explains why he avoids premature few-shot examples, and introduces “emotion prompting” as a research-backed technique.

    • System prompt sections: role, context, instructions, criteria, examples
    • Meta-style scenarios instead of strict input/output few-shot early on
    • Emotion prompting: positive reinforcement can improve model performance
    • XML chosen due to Claude’s strengths and structured parsing benefits
  9. 38:12 – 55:21

    Self-review loop: scoring the prompt, iterating to A-level, then deploying Core

    Gigawatt critiques its own system prompt section-by-section with quantitative scores and improvement suggestions. Tyler uses saved prompts to generate an improved version, then deploys Core into Cassidy with tools (RAG, web search, data analysis).

    • Meta-eval prompt: score + rationale + targeted improvements
    • Saved prompts accelerate repeatable iteration workflows
    • Cassidy deployment settings: model choice, context window, tools toggles
    • RAG configured to always search a curated folder
  10. 55:21 – 56:07

    Testing Core: JSON inter-agent outputs, citations, and human-readable conversion

    They test Core with a real buying question (iPhone Air vs Pro) and see Core return structured JSON suitable for agent-to-agent transfer. Tyler highlights citations for auditability and then converts output to markdown for easier human review.

    • Inter-agent JSON improves downstream parsing and reliability
    • Citations enable verification during evals
    • RAG + web research combine for up-to-date comparisons
    • Switching formats (JSON → markdown) for different audiences
  11. 56:07 – 1:18:00

    Designing the email agent (Echo): brand voice translation and system prompt creation

    Tyler uses an existing “You’ve Got Mail” agent (built for a Nashville hot chicken brand) as inspiration, then adapts the concept to Apple’s tone. Gigawatt proposes the name “Echo” and generates XML system instructions with escalation criteria and examples.

    • Reuse proven patterns: start from an existing email-agent template
    • Brand voice shift: playful “Southern” → clean, Apple-like clarity
    • Name selection matters for mental model consistency (Core + Echo)
    • System instructions include escalation triggers and formatting conventions
  12. 1:18:00 – 1:24:09

    Running Core + Echo together: observational evals and XML-tagged drafting stages

    Tyler routes Core’s research output into Echo to generate the customer email. He explains how tags like <think>, <scratchpad>, and <answer> support internal drafting and production parsing, while noting brand/tone improvements to iterate on.

    • Manual orchestration mimics production message passing
    • Drafting pipeline: reasoning → scratchpad → finalized answer
    • Tags enable extraction of subject/body for workflow automation
    • Client review checkpoint: tone, verbosity, bullet use, brand fit
  13. 1:24:09 – 1:32:22

    From demo to production: human-in-the-loop workflows, Slack approvals, and evals

    Tyler walks through a real Cassidy workflow: email ingestion, sentiment analysis, expert + research agents, drafting, QA iteration, and Slack-based approval before sending. He stresses human-in-the-loop safety, then highlights the need for systematic evals and audit trails.

    • Production requires human-in-the-loop gating and gradual autonomy
    • Workflow example: sentiment agent, expert agent, web research agent, email agent
    • Slack approval pattern: “ship it / revise / confirmed” generative filter
    • Evals + observability: audit trail, goldens database, continuous improvement
  14. 1:32:22 – 1:40:43

    Scaling costs, enterprise RAG, and results: course business growth + what’s next

    They discuss tool costs in context of labor replacement and improving model economics, then touch on handling large enterprise document corpora (OCR, vectorization, GraphRAG). Tyler closes with AI Build Lab’s growth metrics and future plans: scaling education and productizing Gigawatt.

    • Cost framing: compare agent spend to human time and approval overhead
    • Enterprise docs: OCR (e.g., Gemini Flash), unstructured pipelines, embeddings
    • Limitations of plain RAG and move toward agentic/GraphRAG approaches
    • Business outcomes: ~$1.6M run rate; future: scale courses + build Gigawatt product

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.