Lenny's PodcastCareer frameworks, A/B testing, onboarding tips, selling to engineers | Laura Schaffer (Amplitude)
CHAPTERS
- 0:00 – 0:49
Cold open: Adding sign-up questions unexpectedly boosts conversion
Laura recounts a Twilio experiment where adding extra questions to the sign-up flow was expected to add friction and reduce conversions—but instead improved sign-ups by ~5%. The surprising outcome sets up a broader theme: user psychology and “good friction” can outperform simplistic assumptions about minimizing steps.
- •Ran an A/B test by adding questions to the sign-up form
- •Expected conversion to drop due to added friction
- •Observed ~5% conversion lift despite no personalization
- •Surprise result tees up deeper thinking about user behavior
- 0:49 – 5:51
Show setup: Laura’s new Amplitude role and what the conversation will cover
Lenny introduces Laura Schaffer, noting her first week as Head of Growth at Amplitude and her prior leadership roles at Rapid and Twilio. He previews the episode’s focus: career frameworks, experimentation, growth strategy, and selling to engineers/developers.
- •Laura begins as Head of Growth at Amplitude (following Elena Verna)
- •Background: VP Product & Growth at Rapid; growth leadership at Twilio
- •Episode themes: career growth, experimentation, growth strategy, selling to developers
- •Framing the conversation around practical, experience-backed lessons
- 5:51 – 7:11
From ‘manager-led’ careers to carving your own path
Laura explains why the default approach to career growth—doing your job well and relying on your manager to advocate—is limiting. She shares a more proactive approach: create visibility and opportunity by bringing high-leverage insights and solving real company problems.
- •Traditional career growth depends heavily on manager advocacy and org constraints
- •Role trajectories can be rigid; manager changes can reset progress
- •Proactive path: identify valuable problems and propose solutions
- •Build personal brand through demonstrable impact beyond your job description
- 7:11 – 9:12
Bandwidth story: moving from sales into a new role by spotting repeatable customer needs
Laura shares an early-career example at Bandwidth where she noticed repeated customer questions and inefficiencies in the sales motion. By proposing self-serve information and checkout improvements, she effectively created a new role for herself—an early form of “growth” work.
- •Joined Bandwidth (early-stage) in sales; noticed repeated customer questions
- •Recognized inefficiency: customers waited; company paid commission repeatedly
- •Proposed putting answers and flow online for self-serve purchasing
- •Executive support turned her idea into a new role (early “e-commerce/growth” work)
- 9:12 – 12:25
Customer insights as a career accelerant: why exec teams need your ‘on-the-ground’ signal
Laura frames customer proximity as a superpower: executives are sharp but often disconnected from daily customer pain. Bringing crisp, metric-relevant insights helps leaders see what they’re missing—and creates opportunities and trust for the person supplying that insight.
- •Execs lose direct customer contact as organizations scale
- •Customer insights tied to North Star metrics are especially powerful
- •Becoming “the person who knows the customer” builds internal leverage
- •Insights can open doors beyond your immediate team and manager
- 12:25 – 16:14
Twilio example: the ‘voice of the customer’ report that built credibility and created a growth team
At Twilio, Laura began in product marketing but systematically collected and shared customer insights showing onboarding wasn’t as “easy” as leadership believed. Her ongoing voice-of-customer digest grew into company-wide sessions attended by senior leaders, enabling her to pitch and form Twilio’s growth team.
- •Joined Twilio when no growth team existed; started in product marketing
- •Found a gap: leadership narrative (“easy to use”) vs customer reality (struggling to get started)
- •Built a recurring voice-of-customer report and quarterly sessions
- •Earned trust and allies; pitched and helped create a dedicated growth org
- 16:14 – 20:20
Practical tactics: ‘ungate your knowledge’ and build an internal subject-matter reputation
Laura and Lenny discuss actionable ways to create opportunity without waiting for formal permission: share what you know, publish insights, and become known for a distinctive strength. She stresses this complements (not undermines) your manager by making it easier for others to recognize your contributions.
- •Share insights proactively rather than waiting for approval
- •Build a visible internal brand as a subject-matter expert (SME)
- •This accelerates promotions by increasing exposure in calibration rooms
- •Customer-facing teams can leverage their insight to break into product roles
- 20:20 – 23:53
Experiment #1: Turning ‘friction’ into confidence by asking better sign-up questions
Laura describes her first major growth experiment at Twilio: adding segmentation questions to the sign-up flow despite strong internal belief that developers would abandon. The result surprised everyone—conversion improved—because the questions reassured users they were in the right place.
- •Twilio sign-up originally asked only email/username/password
- •Internal conviction: developers hate extra steps; keep the gate wide open
- •Added questions (language, use case, product interest, developer vs non-developer)
- •Lift came from reduced anxiety and increased confidence—not personalization
- 23:53 – 29:53
User psyche as a growth lever: ‘good friction’ vs ‘bad friction’
Laura generalizes the lesson: new users are often anxious and searching for signs they’ll fail or the product won’t fit. Thoughtful friction that reduces uncertainty can increase activation and conversion by making users feel understood and capable.
- •First-time experiences are inherently scary; users anticipate difficulty
- •Questions can signal compatibility (e.g., language support, use-case fit)
- •“Good friction” reassures; “bad friction” blocks without benefit
- •Design onboarding with psychological state in mind, not just step count
- 29:53 – 33:59
Experiment #2: Fixing onboarding by burying the ‘scary stuff’ (hot dog / peanut butter analogy)
A later onboarding change underperformed until Laura’s team identified the real blocker: the fear of telecom/phone-number configuration. By leading with familiar code/docs and embedding the intimidating step later (“pill in the hot dog”), conversion improved.
- •Prescriptive onboarding checklist didn’t yield expected lift
- •Developers found telecom concepts unfamiliar and intimidating
- •Step 1 ‘get a phone number’ triggered early fear and drop-off
- •Moved users into a docs-first flow, then embedded phone number setup later
- 33:59 – 38:38
Why iteration beats big redesigns: experiments fail often—use failure as a compass
Laura argues teams should default to iterative testing rather than rebuilding whole experiences. Given 80–90% of experiments fail, shipping smaller tests helps you learn faster, reduce wasted effort, and use failures to steer toward what works.
- •Industry data suggests most experiments fail (often 80–90%)
- •Large redesigns risk months of work on the wrong approach
- •Iterative tests catch wrong assumptions earlier and cheaper
- •Aim for early versions that are ‘embarrassing’ but informative
- 38:38 – 41:59
Validate faster and cheaper than A/B tests: painted doors, mocks, and staged evidence
Laura explains that A/B testing is often the most expensive validation method due to engineering/design time and long run times. She recommends earlier-stage techniques—painted doors and prototype tests—to filter ideas before investing in full experiments.
- •A/B tests are costly in time, people, and opportunity cost
- •Use painted-door tests to validate interest before building
- •Use mocks/prototypes with users to invalidate weak ideas early
- •Reserve deeper experiments for ideas already vetted by cheaper signals
- 41:59 – 49:51
Experimentation decision-making: mixing qualitative + quantitative and rethinking 95% confidence
Laura describes a pragmatic approach to making calls under uncertainty: combine qualitative evidence with quantitative results, and sometimes accept lower confidence thresholds to increase experimentation throughput. She emphasizes pre-commitment: define thresholds and hypotheses upfront to avoid post-hoc rationalization.
- •Uses qualitative feedback to strengthen decisions beyond pure metrics
- •95% confidence is critical in high-stakes domains, less so in many SaaS contexts
- •Lower confidence can increase velocity and total annual wins
- •Must set hypotheses and decision rules before running the experiment to prevent bias
- 49:51 – 54:43
How long growth needs: set annual expectations, avoid weekly ‘wins,’ and communicate ranges
Laura warns that pressure for short-term wins incentivizes vanity metrics and data-massaging. She recommends setting longer time horizons (often a year) and using low/medium/high outcome ranges to communicate uncertainty and avoid unhealthy operating rhythms.
- •Short reporting cycles can distort behavior and encourage p-hacking-style narratives
- •Healthy growth orgs evaluate progress over longer windows
- •Use low/medium/high forecasts tied to hypotheses and bet types
- •Growth outcomes are often ‘lumpy’—a few wins can drive outsized results
- 54:43 – 58:40
Big Twilio win: Quick Deploy/Code Exchange to help non-developers reach ‘$1 of spend’
Laura shares a major initiative that unlocked significant pipeline by serving a growing segment of non-developer users who still wanted to build with Twilio. By enabling a “create your own demo” / low-code deployment path, the team helped buyers get to early value and handed sales a stronger starting point.
- •Noticed growth in “non-developer” sign-ups; many still wanted to build
- •Sales insight: if users reach minimal spend, sales can scale them to larger contracts
- •Identified blockers: writing code from scratch and standing up servers
- •Built Quick Deploy on Code Exchange to deliver an aha moment without heavy engineering effort
- 58:40 – 1:05:42
PLG + enterprise together: how to approach moving from sales-led to product-led
Laura frames PLG and enterprise sales as complementary “selling through product” vs “selling through people.” She cautions against simply chopping an enterprise product into a cheaper plan; instead, start from the self-serve user’s distinct problems and design the PLG motion around that reality.
- •PLG and sales-led growth are two sides of the same coin
- •Common mistake: copy/paste enterprise offering into a PLG plan
- •Start with self-serve customer problems and context, not product packaging debates
- •Example for Amplitude: startups often lack analysts; PLG can help with templates/benchmarks and guided outcomes
- 1:05:42 – 1:11:22
Serving developers: why they skip marketing, avoid sales, and must ‘prove it’ themselves
Laura explains why developers behave differently from other audiences: they often bypass the marketing site and resist sales outreach because the risk of adopting the wrong tool falls directly on them. Winning developers requires excellent self-serve experiences that let them validate reliability and fit through hands-on building.
- •Developers often go straight to sign-up and docs, skipping marketing pages
- •They strongly avoid sales contact—sometimes even using personal emails to evade it
- •Core driver: accountability—if the tool fails, the developer bears the consequences
- •Treat self-serve onboarding as the primary ‘sales force’ for developer products
- 1:11:22 – 1:21:12
Lightning round and wrap: books, interview question, tools, and operating principles
In the lightning round, Laura shares favorite books, a standout interview technique, and tools she relies on for growth work. She closes with a key operating mantra—ship early enough to be ‘embarrassing’—and offers ways listeners can connect and get help, including job leads.
- •Book picks: The Simple Path to Wealth; Atomic Habits
- •Favorite interview question: ask about a non-cherry-picked, recent ship
- •Tools: Hotjar/Qualaroo, Slack, Builder, ChatGPT (and Amplitude with an asterisk)
- •Process principle: if you’re not embarrassed by v1, you waited too long
