Skip to content
Lenny's PodcastLenny's Podcast

How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, Github)

Dr. Nicole Forsgren is a developer productivity and DevOps expert who works with engineering organizations to make work better. Best known as co-author of the Shingo Publication Award-winning book Accelerate and the DevOps Handbook, 2nd edition and author of the State of DevOps Reports, she has helped some of the biggest companies in the world transform their culture, processes, tech, and architecture. Nicole is currently a Partner at Microsoft Research, leading developer productivity research and strategy, and a technical founder/CEO with a successful exit to Google. In a previous life, she was a software engineer, sysadmin, hardware performance engineer, and professor. She has published several peer-reviewed journal papers, has been awarded public and private research grants (funders include NASA and the NSF), and has been featured in the Wall Street Journal, Forbes, Computerworld, and InformationWeek. In today’s podcast, we discuss: • Two frameworks for measuring developer productivity: DORA and SPACE • Benchmarks for what good and great look like • Common mistakes to avoid when measuring developer productivity • Resources and tools for improving your metrics • Signs your developer experience needs attention • How to improve your developer experience • Nicole’s Four-Box framework for thinking about data and relationships — This episode is brought to you by DX—a platform for measuring and improving developer productivity: https://getdx.com/lenny Find the full transcript at: https://www.lennysnewsletter.com/p/how-to-measure-and-improve-developer Where to find Nicole Forsgren: • Twitter: https://twitter.com/nicolefv • LinkedIn: https://www.linkedin.com/in/nicolefv/ • Website: https://nicolefv.com/ Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • Twitter: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ In this episode, we cover: (00:00) Nicole’s background (07:55) Unpacking the terms “developer productivity,” “developer experience,” and “DevOps” (10:06) How to move faster and improve practices across the board (13:43) The DORA framework (18:54) Benchmarks for success (22:33) Why company size doesn’t matter  (24:54) How to improve DevOps capabilities by working backward (29:23) The SPACE framework and choosing metrics (32:51) How SPACE and DORA work together (35:39) Measuring satisfaction (37:52) Resources and tools for optimizing metrics (41:29) Nicole’s current book project (45:43) Common pitfalls companies run into when rolling out developer productivity/optimizations (47:42) How the DevOps space has progressed (50:07) The impact of AI on the developer experience and productivity (54:04) First steps to take if you’re trying to improve the developer experience (55:15) Why Google is an example of a company implementing DevOps solutions well (56:11) The importance of clear communication (57:32) Nicole’s Four-Box framework (1:05:15) Advice on making decisions  (1:08:56) Lightning round Referenced: • Chef: https://www.chef.io/ • DORA: https://dora.dev/ • GitHub: https://github.com/ • Microsoft Research: https://www.microsoft.com/en-us/research/ • What is DORA?: https://devops.com/what-is-dora-and-why-you-should-care/ • Dustin Smith on LinkedIn: https://www.linkedin.com/in/dustin-smith-b0525458/ • Nathen Harvey on LinkedIn: https://www.linkedin.com/in/nathen/ • What is CI/CD?: https://about.gitlab.com/topics/ci-cd/ • Trunk-based development: https://cloud.google.com/architecture/devops/devops-tech-trunk-based-development • DORA DevOps Quick Check: https://dora.dev/quickcheck/ • Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations: https://www.amazon.com/Accelerate-Software-Performing-Technology-Organizations/dp/1942788339 • The SPACE of Developer Productivity: https://queue.acm.org/detail.cfm?id=3454124 • DevOps Metrics: Nicole Forsgren and Mik Kersten: https://queue.acm.org/detail.cfm?id=3182626 • How to Measure Anything: Finding the Value of Intangibles in Business: https://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273/ • GitHub Copilot: https://github.com/features/copilot • Tabnine: https://www.tabnine.com/the-leading-ai-assistant-for-software-development • Nicole’s Decision-Making Spreadsheet: https://docs.google.com/spreadsheets/d/1wItAODkhZ-zKnnFbyDERCd8Hq2NQ03WPvCfigBQ5vpc/edit?usp=sharing • How to do linear regression and correlation analysis: https://www.lennysnewsletter.com/p/linear-regression-and-correlation-analysis Find all lightning round references at: https://www.lennyspodcast.com/ Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com. Lenny may be an investor in the companies discussed.

Nicole ForsgrenguestLenny Rachitskyhost
Jul 29, 20231h 16mWatch on YouTube ↗

At a glance

WHAT IT’S REALLY ABOUT

Measuring Developer Productivity: DORA, SPACE, and DevEx That Works

  1. Nicole Forsgren, co-author of Accelerate and the DORA reports, breaks down how to rigorously measure and improve developer productivity and experience. She explains the DORA metrics (speed and stability) and the SPACE framework (a holistic way to pick balanced metrics), and how they fit together in practice.
  2. A central theme is that shipping smaller, more frequent changes makes systems both faster and more stable, debunking the belief that speed harms reliability. Nicole also stresses starting with a clear problem definition (“what exactly are we trying to improve?”) before measuring anything.
  3. She covers how to blend qualitative (surveys, interviews) and quantitative (system telemetry) data, common pitfalls in DevEx initiatives, and why AI tools like GitHub Copilot change workflows but don’t simply translate to “fewer engineers needed.”
  4. Finally, she previews her upcoming book on measuring DevEx, shares practical frameworks (SPACE, DORA, a four-box hypothesis framework, and a decision spreadsheet), and emphasizes communication: tailoring insights to leaders and developers in language that resonates.

IDEAS WORTH REMEMBERING

5 ideas

Start with a precise problem, not with metrics or tools.

Most DevEx efforts stumble because leaders say vague things like “improve developer experience” without specifying whether they mean culture, tooling friction, inner/outer loop, or something else. Alignment on the exact goal prevents teams from working for months on the wrong problem.

Use DORA’s four metrics to track software delivery performance holistically.

DORA focuses on two speed metrics (deployment frequency, lead time for changes) and two stability metrics (MTTR, change fail rate). High performers show that speed and stability move together when you adopt good technical and architectural practices, overturning the myth that you must trade one for the other.

Ship smaller, more frequent changes to improve both speed and quality.

Batching work into large, infrequent releases increases blast radius, complexity, and debugging time, making systems less stable. Moving to small, frequent deployments reduces merge conflicts, shrinks the blast radius, and shortens the time and cognitive load needed to restore from failures.

Use the SPACE framework to choose balanced, non-gamed metrics.

SPACE (Satisfaction & well-being, Performance, Activity, Communication & collaboration, Efficiency & flow) is a lens for picking at least three complementary metrics instead of over-optimizing on easy counts like lines of code or PRs. This avoids perverse incentives and captures the full complexity of knowledge work.

Combine survey data with system telemetry; don’t dismiss subjective feedback.

Developer surveys reveal friction, heroics, and missing data that instrumentation can’t see (e.g., work outside version control, Rube Goldberg pipelines). In practice, when surveys and telemetry disagree, surveys are often right—so treat people-data as a first-class signal, not a “nice-to-have.”

WORDS WORTH SAVING

5 quotes

If you're talking about culture, this is totally different than if you're talking about friction in tool chains. If you're on different pages, you're heading in completely different directions.

— Nicole Forsgren

When you move faster, you are more stable. When you push less frequently, you will have more unstable systems.

— Nicole Forsgren

DevOps is not a toolchain you buy. Marketing teams labeled toolchains DevOps because they wanted your money.

— Nicole Forsgren

You will never find yourself in a situation when you don’t want both types of data—data from people and data from systems.

— Nicole Forsgren

Anyone who’s looking at AI tools and thinking, ‘Now I can lay off half my workforce,’ that’s not what this is about.

— Nicole Forsgren

Definitions and relationships between developer productivity, developer experience (DevEx), and DevOpsThe DORA research program and the four key software delivery performance metricsThe SPACE framework for selecting balanced, meaningful productivity metricsHow to practically measure and improve developer productivity and experienceUsing qualitative (people) data and quantitative (system) data togetherImpact of AI tools (e.g., GitHub Copilot) on developer workflows and productivityCommon pitfalls and best practices in rolling out DevEx and productivity initiatives

High quality AI-generated summary created from speaker-labeled transcript.

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

Add to Chrome