Lenny's PodcastThe role of AI in new product development | Ryan J. Salva (VP of Product at GitHub)
CHAPTERS
- 0:00 – 4:48
From Arctic Code Vault to AI code generation: the origin story hook
Ryan opens with the Arctic Code Vault project—archiving a snapshot of public GitHub code in a Finnish vault—and explains how that same dataset became a catalyst for experimenting with large language models. This sets up the central theme: applying AI to software creation at massive scale.
- •Arctic Code Vault as a long-term preservation of open-source code
- •Public GitHub code as a uniquely valuable dataset for AI
- •Partnership with OpenAI to explore what LLMs can do with code
- •Early realization: LLMs can power novel developer experiences
- 4:48 – 7:22
Ryan’s unconventional path into product and developer tools
Ryan shares his academic background in philosophy/English and frames software as a new creative medium. He connects his motivation—helping people create—to his career across startups, Microsoft, and GitHub.
- •Studied philosophy of aesthetics and critical theory, not computer science
- •Software as a modern creative medium akin to paint/brushes in past eras
- •20+ years in engineering/product across startups and Microsoft
- •Motivation: enable creativity and creation through tools
- 7:22 – 11:20
Why GitHub (post-acquisition) became the center of gravity for developer innovation
Ryan explains why he moved from Microsoft to GitHub after the acquisition. He describes GitHub’s expanding product surface beyond repos into the full developer lifecycle, and why that made it the best place to have impact.
- •Microsoft role: internal dev infrastructure + Azure DevOps
- •GitHub as where developer community, learning, and mindshare live
- •GitHub’s broadened platform: Codespaces, Actions, Advanced Security
- •Appeal of building multiple V1 products and iterating with community feedback
- 11:20 – 14:21
What GitHub Copilot is: multi-line AI autocomplete for staying in flow
Ryan defines Copilot as a step-change from traditional IntelliSense: multi-line code suggestions driven by OpenAI’s Codex model (derived from GPT-3). The focus is reducing context switching and keeping developers in a creative flow state.
- •Copilot = multi-line autocomplete inside the IDE (VS Code, IntelliJ, etc.)
- •Powered by Codex, a GPT-3 derivative specialized for code
- •Infers intent from surrounding context: variables, methods, comments
- •Reduces drudgery (syntax, boilerplate, dummy data) and doc/StackOverflow trips
- 14:21 – 18:21
Unexpected use cases: education and learning through building real tools
They discuss standout stories of Copilot use, especially in teaching. Ryan highlights an educator pairing students with small businesses to build internal tools—Copilot acts like a guide that accelerates learning and real-world output.
- •Copilot helps learners by giving immediate, contextual guidance
- •Students build real internal tools for businesses and gain resume-worthy experience
- •AI support can speed up onboarding into unfamiliar codebases
- •Copilot assists even when the language is known by building context quickly
- 18:21 – 22:28
How Copilot began: LLMs as programming languages + the OpenAI ‘clone storm’ incident
Ryan recounts the early spark: recognizing code as another ‘language’ well-suited to LLMs. A memorable moment came when GitHub infrastructure saw massive cloning traffic—OpenAI harvesting repos—leading to a more responsible data-sharing approach and deeper collaboration.
- •Key insight: Python/JS/Java are languages with constrained semantics for AI
- •OpenAI’s heavy cloning activity triggered reliability alarms at GitHub
- •GitHub responded by packaging a dataset more responsibly
- •Arctic Code Vault snapshot became a practical dataset for experiments
- •Early capabilities: translation between natural language and code, and between languages
- 22:28 – 27:45
Finding the right UX: from side panels to inline suggestions (and performance constraints)
Ryan details how they iterated on product experience, eventually landing on inline gray-text suggestions that minimized distraction. He emphasizes that making the model useful required parameter tuning, latency targets, and prompt-crafting experimentation—not just “having a model.”
- •Explored different interfaces (e.g., side panels) before inline autocomplete
- •Partnered with VS Code team to enable multiline autocomplete extensibility
- •Latency is critical for flow; sweet spot ~200ms for suggestions
- •Significant work on model parameters, prompting, and what context to feed the model
- •Remarkable speed: core product experience matured in ~14–16 months
- 27:45 – 31:04
GitHub Next and the ‘three horizons’: turning moonshots into products
Ryan explains GitHub Next as a dedicated Horizon 2/3 team focused on ambiguous, longer-term bets. He describes how horizons are less about calendar dates and more about confidence/ambiguity, and why ring-fencing R&D from operational product teams can work.
- •GitHub Next pursues second/third-horizon projects (moonshots)
- •Horizon definitions are approximate; ambiguity/confidence matters more than time
- •GitHub intentionally separates R&D from EPD (engineering/product/design) delivery
- •Early phase goal: give researchers freedom without full production constraints
- 31:04 – 38:45
The handoff: when to transition from R&D to EPD (and how to do it without breaking momentum)
Ryan shares the signals that triggered the shift from research to productization: real customer pull and “magical” feedback at meaningful scale. He describes moving researchers temporarily to seed an EPD squad, scaling a technical preview, and then gradually rotating researchers back once replacements are fully competent.
- •Transition starts when there’s customer signal with medium confidence and clear problem-solution fit
- •Researchers moved temporarily to seed product squads and enable knowledge transfer
- •Technical preview scaled from tens of thousands to hundreds of thousands
- •Researchers return to Next only when there’s true replacement-in-seat continuity
- •Operational fundamentals (privacy, security, uptime, accessibility) mark the graduation from experiment to product
- 38:45 – 45:00
Ethics and responsible AI in Copilot: persona framing, filters, and cross-functional governance
The conversation turns to the unique ethical and legal challenges of AI that generates text/code. Ryan explains how framing Copilot as an ‘AI pair programmer’ helped define acceptable behavior, and how the team worked with legal, privacy/security, developer feedback, and Responsible AI models to reduce harmful outputs beyond crude blocklists.
- •AI products require heavy legal, privacy, security, and community engagement
- •‘AI pair programmer’ persona guides expectations for appropriate behavior
- •Early versions had minimal filtering; moved to blocklists, which created new issues
- •Shift toward AI-based sentiment/toxicity detection models (Azure Responsible AI)
- •Goal: avoid offensive/disruptive suggestions while preserving developer utility
- 45:00 – 48:47
Where AI goes next for developers: beyond autocomplete to the whole toolchain
Ryan forecasts AI permeating the entire development lifecycle, not just writing code. He highlights opportunities like summarizing pull requests, improving collaboration, and removing drudgery so developers can focus on design patterns and outcomes.
- •Copilot is the ‘tip of the spear’ for AI across developer workflows
- •Potential applications: PR/commit summarization, build and workflow optimization
- •AI shifts attention from syntax/parameters to architecture and outcomes
- •AI should augment—not replace—developers and existing quality systems
- •Lower barriers could invite more people into software creation
- 48:47 – 54:50
Scaling Copilot: GPU scarcity and the challenge of community trust
Ryan outlines two major scaling constraints: limited global GPU supply for training/serving and the need to earn legitimacy with developers. Because society is still forming expectations around AI trained on public code, he notes the product org had to scale community dialogue and trust-building as much as engineering.
- •Hardware constraint: rare GPUs needed for training and inference amid disrupted supply chains
- •Social/product constraint: building a model the community feels ownership over
- •Ongoing concerns: training on public code, security/bugginess, new attack vectors (e.g., poisoning)
- •GitHub messaging: Copilot is not a replacement for developers
- •Maintain guardrails: keep unit tests, static analysis, and review processes in place
- 54:50 – 57:07
Portfolio management at scale: allocating effort across moonshots, operations, and iteration
Ryan shares how he thinks about distributing resources across bold bets, operational maintenance, and incremental improvements. He emphasizes that the right split depends on context, but offers a practical baseline for larger teams.
- •Typical allocation: 5–10% bold experiments, ~25–30% operations, ~60% incremental progress
- •Allocation shifts based on external circumstances and team maturity
- •Startups often invert the model—everything is a single big bet
- •Portfolio framing helps sustain innovation while meeting reliability expectations
- 57:07 – 1:03:18
Lightning round: books, podcasts, media, interview questions, and a Copilot shout-out
In a quick Q&A, Ryan shares recommendations and personal practices, including a favorite interview prompt. He closes by crediting a key Copilot researcher and reinforcing admiration for the work behind the product.
- •Book recs: ‘Make It So’ (UX via sci-fi) and David Foster Wallace stories
- •Podcast rec: ‘The Memory Palace’ for historical storytelling
- •Recent watch: ‘Arrival’ (language and memory themes)
- •Interview question: teach something new in one minute (completeness, complexity, clarity)
- •Shout-out to a key Copilot innovator/research leader
- 1:03:18 – 1:04:59
Closing: where to find Ryan and how listeners can help improve Copilot
Ryan shares where he’s reachable online and invites developers to try Copilot and share honest feedback publicly. The episode ends with Lenny’s wrap-up and show credits.
- •Ryan’s handle: ryanjsalva across platforms
- •Encourages trying Copilot via free trial
- •Requests both positive and negative feedback in public forums
- •Feedback helps identify novel use cases and rough edges
- •Podcast outro and subscription/review prompt
