How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, Github)

How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, Github)

Lenny's PodcastJul 30, 20231h 16m

Nicole Forsgren (guest), Lenny Rachitsky (host)

Definitions and relationships between developer productivity, developer experience (DevEx), and DevOpsThe DORA research program and the four key software delivery performance metricsThe SPACE framework for selecting balanced, meaningful productivity metricsHow to practically measure and improve developer productivity and experienceUsing qualitative (people) data and quantitative (system) data togetherImpact of AI tools (e.g., GitHub Copilot) on developer workflows and productivityCommon pitfalls and best practices in rolling out DevEx and productivity initiatives

In this episode of Lenny's Podcast, featuring Nicole Forsgren and Lenny Rachitsky, How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, Github) explores measuring Developer Productivity: DORA, SPACE, and DevEx That Works Nicole Forsgren, co-author of Accelerate and the DORA reports, breaks down how to rigorously measure and improve developer productivity and experience. She explains the DORA metrics (speed and stability) and the SPACE framework (a holistic way to pick balanced metrics), and how they fit together in practice.

Measuring Developer Productivity: DORA, SPACE, and DevEx That Works

Nicole Forsgren, co-author of Accelerate and the DORA reports, breaks down how to rigorously measure and improve developer productivity and experience. She explains the DORA metrics (speed and stability) and the SPACE framework (a holistic way to pick balanced metrics), and how they fit together in practice.

A central theme is that shipping smaller, more frequent changes makes systems both faster and more stable, debunking the belief that speed harms reliability. Nicole also stresses starting with a clear problem definition (“what exactly are we trying to improve?”) before measuring anything.

She covers how to blend qualitative (surveys, interviews) and quantitative (system telemetry) data, common pitfalls in DevEx initiatives, and why AI tools like GitHub Copilot change workflows but don’t simply translate to “fewer engineers needed.”

Finally, she previews her upcoming book on measuring DevEx, shares practical frameworks (SPACE, DORA, a four-box hypothesis framework, and a decision spreadsheet), and emphasizes communication: tailoring insights to leaders and developers in language that resonates.

Key Takeaways

Start with a precise problem, not with metrics or tools.

Most DevEx efforts stumble because leaders say vague things like “improve developer experience” without specifying whether they mean culture, tooling friction, inner/outer loop, or something else. ...

Get the full analysis with uListen AI

Use DORA’s four metrics to track software delivery performance holistically.

DORA focuses on two speed metrics (deployment frequency, lead time for changes) and two stability metrics (MTTR, change fail rate). ...

Get the full analysis with uListen AI

Ship smaller, more frequent changes to improve both speed and quality.

Batching work into large, infrequent releases increases blast radius, complexity, and debugging time, making systems less stable. ...

Get the full analysis with uListen AI

Use the SPACE framework to choose balanced, non-gamed metrics.

SPACE (Satisfaction & well-being, Performance, Activity, Communication & collaboration, Efficiency & flow) is a lens for picking at least three complementary metrics instead of over-optimizing on easy counts like lines of code or PRs. ...

Get the full analysis with uListen AI

Combine survey data with system telemetry; don’t dismiss subjective feedback.

Developer surveys reveal friction, heroics, and missing data that instrumentation can’t see (e. ...

Get the full analysis with uListen AI

Measurement is a journey: start scrappy, then automate over time.

Early on, you’ll rely more on interviews and surveys because they’re fast to spin up; as you mature, you invest in robust instrumentation and pipelines. ...

Get the full analysis with uListen AI

AI coding assistants change how we work, not just how fast we type.

Tools like GitHub Copilot shift time from writing to reviewing code and free cognitive capacity for harder problems, but they also introduce new issues around trust, overreliance, learning, and onboarding. ...

Get the full analysis with uListen AI

Notable Quotes

If you're talking about culture, this is totally different than if you're talking about friction in tool chains. If you're on different pages, you're heading in completely different directions.

Nicole Forsgren

When you move faster, you are more stable. When you push less frequently, you will have more unstable systems.

Nicole Forsgren

DevOps is not a toolchain you buy. Marketing teams labeled toolchains DevOps because they wanted your money.

Nicole Forsgren

You will never find yourself in a situation when you don’t want both types of data—data from people and data from systems.

Nicole Forsgren

Anyone who’s looking at AI tools and thinking, ‘Now I can lay off half my workforce,’ that’s not what this is about.

Nicole Forsgren

Questions Answered in This Episode

How can we concretely define “developer experience” in our organization so that leaders and ICs are aligned on what we’re improving?

Nicole Forsgren, co-author of Accelerate and the DORA reports, breaks down how to rigorously measure and improve developer productivity and experience. ...

Get the full analysis with uListen AI

Where do our current delivery metrics (deployment frequency, lead time, MTTR, change fail rate) sit relative to DORA’s elite benchmarks, and what would it take to move one category up?

A central theme is that shipping smaller, more frequent changes makes systems both faster and more stable, debunking the belief that speed harms reliability. ...

Get the full analysis with uListen AI

Which three SPACE dimensions should we prioritize first, and what specific metrics (both survey and telemetry-based) can we use for each?

She covers how to blend qualitative (surveys, interviews) and quantitative (system telemetry) data, common pitfalls in DevEx initiatives, and why AI tools like GitHub Copilot change workflows but don’t simply translate to “fewer engineers needed.”

Get the full analysis with uListen AI

What are the biggest sources of friction and heroics our developers face today that don’t show up in our tools or dashboards?

Finally, she previews her upcoming book on measuring DevEx, shares practical frameworks (SPACE, DORA, a four-box hypothesis framework, and a decision spreadsheet), and emphasizes communication: tailoring insights to leaders and developers in language that resonates.

Get the full analysis with uListen AI

How should we adapt our productivity measurement and workflows as we adopt AI coding assistants, particularly around review practices, learning, and trust in generated code?

Get the full analysis with uListen AI

Transcript Preview

Nicole Forsgren

... starting with what is your problem or what is your goal? I would say this is a bigger challenge than most people recognize or realize. 80% of the folks that I work with, this is their biggest problem, even at, like, executive levels. Teams will have gone off for several months and they're tackling something and they'll come back with uncertainty. And they'll say, like, "Well, you told me to improve developer experience." And I'm like, "Okay, what do you mean by this? Are you talking about inner and outer loop? Are you talking about friction? Are you talking about culture?" But if you're talking about culture, this is totally different than if you're talking about friction in tool chains. If you're on different pages, you're heading in completely different directions.

Lenny Rachitsky

(intro music plays) Welcome to Lenny's Podcast, where I interview world-class product leaders and growth experts to learn from their hard-won experiences building and growing today's most successful products. Today my guest is Nicole Forsgren. This is actually my first recording back since going on pat leave for the past couple of months, and what an awesome episode to get back into the swing of things. Nicole is the developer productivity expert, having written the award-winning book Accelerate, and she's been the co-author of the State of DevOps Report year after year. She's currently a partner at Microsoft Research, leading developer productivity, research, and strategy, and she's helped some of the biggest companies in the world move faster, improve product quality, and transform their cultures. In our conversation, we get into the weeds of how to go about measuring and improving your engineering team's productivity and experience. We talk about the DORA framework and the SPACE framework and how to actually implement them to understand how your engineering team is doing. Nicole also shares benchmarks for what elite companies are at. We talk about why moving faster turns out to be one of the best ways to improve quality and stability, plus pitfalls you want to avoid, and also a preview of a new book that she's working on and so much more. Enjoy this episode with Nicole Forsgren after a short word from our sponsors. Today's entire episode is brought to you by DX, a platform for measuring and improving developer productivity. DX is designed by the researchers behind frameworks such as DORA, SPACE, and DevEx, including Nicole Forsgren, who is my guest for this very episode. If you've tried measuring developer productivity, you know that there are a lot of basic metrics out there and a lot of ways to do this wrong, and getting that full view of productivity is still really hard. DX tackles this problem by combining qualitative and quantitative insights based on the very research Nicole and her team have done, giving you full clarity into how your developers are doing. DX is used by both startups and Fortune 500 companies, including companies like Twilio, Amplitude, eBay, Brex, Toast, Pfizer, and Procter & Gamble. To learn more about DX and get a demo of their product, visit their website at getdx.com/lenny. That's getdx.com/lenny. Nicole, welcome to the podcast.

Install uListen to search the full transcript and get AI-powered insights

Get Full Transcript

Get more from every podcast

AI summaries, searchable transcripts, and fact-checking. Free forever.

Add to Chrome