Building a culture of excellence | David Singleton (CTO of Stripe)

1. 0:00 – 4:22

   David’s background

   1. DSDavid Singleton0:00

      The way we think about product development at Stripe, it, it really is to find the correct set of early users to kind of co-create the product with. Maybe, uh, the best example of that is Stripe Billing. When we got to starting the Stripe Billing product, we realized that there were a number of our existing users, these were companies like Figma and Slack, um, who were already using Stripe for payments, but had these subscriptions business models. And we figured that there were gonna be many more of these kind of companies into the future and we, we could see that they, they were really kind of pushing the boundaries of what was possible here. So we decided to co-create the product with them. So we had shared Slack channels. We'd actually show them product on a very regular basis, get their feedback on it. And only when that original kind of alpha group was super, super happy with the product did we then think it might be ready to go to, to a broader audience. So that is just how we build product at Stripe. And that means that every engineer building product at Stripe really has many of the kind of attributes and will exercise many of the attributes that you'll often find in, in PMs in, in other companies.

   2. LRLenny Rachitsky1:02

      (instrumental music) Welcome to Lenny's Podcast, where I interview world-class product leaders and growth experts to learn from their hard-won experiences building and growing today's most successful products. Today my guest is David Singleton. David is chief technology officer at Stripe, where he's responsible for guiding its engineering and design teams, a role he's had for over five years. Prior to Stripe, David was VP of engineering at Google, where he spent over a decade. And in hearing from David, you'll quickly be able to tell how passionate he is about the craft of building great products and building great teams. We dig into Stripe's unique approach to hiring, how they built a very product-oriented engineering team which allowed them to hold off on hiring their first product manager for many, many years, how they operationalize their operating principle of be meticulous in your craft, including a number of fascinating internal processes like engineer occasions, walking the store, and something called friction logging. David also shares what it takes to run an engineering culture with the uptime and scale requirements of a Stripe, plus how AI is already impacting how they work and lessons around leadership, management, and planning. I am so thankful David carved out time to come on the podcast, and I know that you'll learn something valuable from it. With that, I bring you David Singleton after a short word from our sponsors. This episode is brought to you by Mixpanel, offering powerful self-serve product analytics. If you listen to this podcast, you know that it's really hard to build great product without making compromises. And when it comes to using data, a lot of teams think that they only have two choices: make quick decisions based on gut feelings or make data-driven decisions at a snail's pace. But that's a false choice. You shouldn't have to compromise on speed to get product answers that you can trust. With Mixpanel, there are no trade-offs. Get deep insights at the speed of thought at a fair price that scales as you grow. Mixpanel builds powerful and intuitive product analytics that everyone can trust, use, and afford. Explore plans for teams of every size and see what Mixpanel can do for you at mixpanel.com. And while you're at it, they're hiring. Check out mixpanel.com to learn more. This episode is brought to you by Eppo. Eppo is a next-generation A/B testing platform built by Airbnb alums for modern growth teams. Companies like DraftKings, Zapier, ClickUp, Twitch, and Cameo rely on Eppo to power their experiments. Wherever you work, running experiments is increasingly essential, but there are no commercial tools that integrate with a modern growth team stack. This leads to wasted time building internal tools or trying to run your own experiments through a clunky marketing tool. When I was at Airbnb, one of the things that I loved most about working there was our experimentation platform where I was able to slice and dice data by device types, country, user stage. Eppo does all that and more, delivering results quickly, avoiding annoying prolonged analytic cycles, and helping you easily get to the root cause of any issue you discover. Eppo lets you go beyond basic click-through metrics and instead use your North Star metrics like activation, retention, subscription, and payments. Eppo supports tests on the front end, on the back end, email marketing, even machine learning claims. Check out Eppo at GetEppo.com. That's GetEppo.com and 10X your experiment velocity.

2. 4:22 – 12:27

   How Stripe’s unique hiring process has helped them build an incredible team

   1. LRLenny Rachitsky4:22

      David, welcome to the podcast.

   2. DSDavid Singleton4:25

      Thanks for having me, Lenny. It's great to be here.

   3. LRLenny Rachitsky4:27

      I know Stripe Sessions is coming around the corner in, uh, later next week, and I imagine it's a pretty hectic time for you. And so I just wanna extra appreciate you making time for this in the middle of all that.

   4. DSDavid Singleton4:37

      Thank you. Well, it's, it's a pleasure. It's a lot of fun. Stripe Sessions is our annual user conference so we get together a bunch of folks from businesses building on Stripe and it's a great opportunity both to share what we've been up to over the last year but also just learn a lot from them about, you know, what, what they would like to see us doing, and so I find it very energizing. It is, of course, a lot of work to put it together, and in particular I'm spending a bunch of time getting some demos together today, which I love building demos. So, but it's a real pleasure to, to join you today.

   5. LRLenny Rachitsky5:05

      So as I was preparing for this conversation, I kind of realized that the Stripe alumni are, are just killing it on this podcast, both in terms of the number of people that have come from Stripe and also just in terms of the quality and the popularity of some of the episodes. So folks like Clara Hughes Johnson and Shreyas Doshi and Eka, and my first question is essentially, what is it that y'all do that allows you to attract, hire, close, keep people of that caliber? And I'm not gonna accept just we keep a high bar or we just spend a lot of time on hiring. I'm curious just, like, what is it that y'all do that you think that other companies don't do that's maybe unique to the way Stripe finds and hires?

   6. DSDavid Singleton5:44

      Well, yeah. I'd love to tell you more about how we find and hire people. I, I think it makes sense to kind of zoom out for a second 'cause it, it's material here. So think about what Stripe's doing. Our core thesis is that the internet economy is going to be much bigger and more important in the future than it is today, and that's certainly played out over the last 10 years. And so we are really a, a business full of...... product minded builders who are seeking to make it easier for businesses to get started and to operate against that backdrop. And so, you know, we got started building a credit card payments API. Before Stripe came along, accepting credit card payments online was way too hard. You had to go talk to your bank. You had to maybe sign a contract with Visa and Mastercard. You then had to kind of string together all this very complicated infrastructure and then put a bunch of kind of restrictions and requirements on the business. And John and Patrick who are co-founders had a previous business and they, they tell this funny story actually, which is while they were building that company, they assumed... It was an internet company. They assumed the hardest thing was going to be, like, putting together hardware in a data center. But it turned out because there were services like AWS around by then that that was actually quite easy. However, when it came to accepting payments in that business, it was just as hard as they had expected building hardware to be.

   7. NANarrator7:00

      Hmm.

   8. DSDavid Singleton7:00

      They had to actually deploy some physical hardware, they had to do a whole bunch of these partnerships. And it was very difficult especially as an upstart, you know, folks that didn't have any reputation yet, to kind of be taken seriously and be able to, to crack into that. And so that's kind of the inspiration behind Stripe. So we started out solving a really kind of important and profound problem for these businesses getting started and scaling, and obviously taking a, a very developer-centric approach to that in the early days. I think what's exciting to know about and important to know about next is, as we've scaled, w- we really kind of figure out what to do based on paying really close attention to our users, those are the businesses building on Stripe. What are they getting out of the products as they are today? And what are the problems adjacent to the ones that we're already solving that they, you know, h- have to put a lot of their own energy into, but really feel like they shouldn't and they'd like us to help? And when we find that confluence of kind of need and, and then adjacency to what we're already doing, that's usually our invitation to go and build the next layer of our economic infrastructure. So we think about what we do today as building economic infrastructure for the internet and beyond. So to bring this back to like how do we attract the, the right talent and, and, and why does this matter? First of all, a lot of people, most people I think come to Stripe because they see that there is this great opportunity to help businesses at scale. So it really is about taking part in a mission that matters, and are really motivated by that idea that we spend a lot of time with our users, the folks that actually, you know, need our help, and are guided by them. And that mindset, I think, attracts a certain kind of person who wants to take a lot of agency, like wants to understand the problems deeply and come up with their own ideas about how to, to solve for them. Because we're building infrastructure, it's also our approach that we really do think that we can do the best work kind of behind the scenes. We don't have to be, uh, you know, out having the, the splashy launch, but hopefully all of our users can and we are quietly working behind the scenes to make their businesses better and more effective over time. And I think that attracts folks with a certain kind of mindset, which is the interest and kind of tenacity to dig into kind of difficult problems that may not be obvious and obviously kind of attractive to everyone in the world, then really plug away at them until we've made a tremendous amount of progress. And so it attracts folks that are longterm thinkers, that care about building, and also actually wanna work very collaboratively together. So when I think about what makes Stripe different to other companies, it really is this like singularity of purpose. Our mission is increasing the GDP of the internet and building this financial infrastructure to do that, and a tremendous amount of teamwork and collaboration to get that done. And so that attracts a certain kind of person. W- when we think about finding those people, um, we don't hide any of that. Like, y- you have, you have to care about that in order to, I think, be excited about working at Stripe. And so we work very hard to make it very clear as we're, as we're bringing people in that that's, that's what this is about. And to, to that end, we're often very patient when we're hiring. So especially for critical roles, you know, leadership hiring in particular, we're happy to take our time and, you know, meet tens or hundreds of people. And then in particular, sometimes we'll meet someone we think is a really good fit for a given role, but they're not available right now. (laughs) Uh, and then we'll be patient. We'll, we'll keep building the, the personal relationship with them and oftentimes, uh, the, the moment of time comes when it makes sense for them to join. And in order to pull that off, it's also the case that hiring at Stripe is a very personal activity. So look, there are lots of other companies where recruiting is kind of like this machine. And by the way, we have an awesome recruiting team, but folks across the rest of the company partner very closely with our recruiting team. So it's not, it's not like it's a machine that delivers new hires and you're like, "Oh, well maybe this person is here and I can put them to use on that thing." Um, our managers spend a lot of time identifying exactly what they need for the various roles that we have, and then getting to know the, the, the potential candidates, the people out there in the world that can do these jobs. Actually like really go quite deep with them on why it might make sense for them to feel like, number one, they can like feel very personally fulfilled against that mission here, and then also why it represents like a bigger opportunity for them than maybe anything else they can spend their, their time and, and attention on. And to that end, another thing that I think Stripe has definitely delivered for, for many, many people who, who join here is just... And I care about preserving this and the culture, is just tremendous learning opportunities. Um, very, very many people at Stripe today are doing a quite different job to the one that they joined for. In order to do that, they've often had to learn a lot and, and stretch themselves into new areas. It's also the case that when you think about that financial infrastructure we're building, the space really rewards people who want to go and learn a ton of detail in how these systems that are like not that, you know, popular or well understood in the broader world, how they actually work, and, and assimilate just a lot of context. And then in particular, you can build better stuff at Stripe. You're...... sharing that context with each other and helping other people learn. Um, so I think that all ladders up to this kind of, um, group of folks that you mentioned, um, all, all the folks you mentioned that are, are wonderful Stripes or former Stripes that come together, uh, in service of our users.

   9. LRLenny Rachitsky12:22

      I always love to hear what people call their people internally, so Stripes, good to know.

   10. DSDavid Singleton12:26

       That's right.

3. 12:27 – 14:11

   An example of a relentlessly curious and passionate employee

   1. DSDavid Singleton12:27

   2. LRLenny Rachitsky12:27

      Um, so just to summarize what you shared, which I, I was just writing as you were talking about, the things that essentially help Stripe build this incredible team. A strong mission and a mission that draws people in that are incredible. And people may be hearing them be like, "Yeah, yeah, like sure." But it, in my experience having invested in a bunch of companies and working with a lot of founders, there's such a dichotomy between how hard it is for companies to hire when they don't have a mission people really care about versus a mission that's like really meaningful. So there's something so powerful there, and so it's 100% I can see how that works so well. And then the other things you mentioned, patience, personal connections between the hiring manager and the people they're hiring. And I see that on Twitter, a lot of PMs are just like doing things, like basically it's like they're running their own little startup on Twitter.

   3. DSDavid Singleton13:09

      Mm-hmm.

   4. LRLenny Rachitsky13:10

      Like, uh, Jeff Weinstein, uh, at Atlas.

   5. DSDavid Singleton13:12

      Yeah, so J- Jeff runs... Jeff is the PM for our Atlas product, and he's a great example of someone who cares a lot about learning exactly hi- how the world works. To me, for instance, one of the things that that team, and Jeff really has driven this himself, have done recently is recognizing that when you're starting a new business in the US, one of the most frustrating steps is that you have to apply for this employer identification number from the IRS, and it used to have like an extremely long backlog. And it's really hard, you can't do... you can't op- you can't like start running your business until you have it. And Jeff dug into the details, the team dug into the details, and we're just like kept asking, "Does it have to be this way? Does it have to be this way?" Eventually we worked with the IRS to make it possible for, for us to issue those, those numbers much, much more quickly, like instantly as you sign up. And I think that's exactly the, a great example of how being curious, having this relentless passion to solve problems for users and then just keep plugging away at a problem delivers this great result, and, you know, an environment where you can learn a lot from each

4. 14:11 – 16:39

   Structured hiring loops at Stripe

   1. DSDavid Singleton14:11

      other.

   2. LRLenny Rachitsky14:11

      I hope he got a, a big promotion for getting the IRS to do something that he needed. That's incredible. I'm gonna as- try to go one layer deeper on the hiring piece and then move on to a different topic.

   3. DSDavid Singleton14:22

      Sure.

   4. LRLenny Rachitsky14:22

      But what is just the interview process like? I don't know how involved you are with the product management hiring process, but just I'm curious what that is like high level and, and engineering hiring process, however you can share.

   5. DSDavid Singleton14:32

      Yeah. There are a couple of things that are very common across hiring for all roles at Stripe and in particular engineers and PMs. One of the things is, we do have structured hiring loops, so we put everyone through a very consistent process. So very personal as we help folks figure out why they might want to be here, but then we have a consistent way of evaluating talent. And I think that's important because it means that you can, uh, as an interviewer, as a hiring manager, you can actually really start to understand what is the broad spectrum of responses you might get here and what do I actually learn about individual candidates when I'm asking like similar questions? The other thing that all these loops have in common is, nothing is a trick question. Like, we try to put people into an environment which is as close to doing the actual work together as we possibly can. So, I mean, for engineers for instance, we've got several different coding exercises that they do. They share their screen with the interviewer, it's actually kind of pair programming, you're welcome to, like, use Google to search for, or Stack Overflow to search for, for answers. We don't, we don't care, like we just want to see the outcome. Uh, you're happy to... uh, we're, we're happy for you to ask your interviewer questions as you go along and, and to, you know, we've created these exercises that are as close to real problems as we need to solve at Stripe as possible. The same's true on, on the PM side, uh, one of the things for instance we do with PMs is we have a written exercise. We've got obviously a variety of different problems, um, and we find that very valuable for actually just seeing how does someone attack a, a real problem that they would be solving at work, and will they want to do that in a way that is curious, digging into the details, collaborative with their interviewer and so forth. So, that's the point. Structured loops that are consistent and then questions that are as close to the real job as we possibly can.

   6. LRLenny Rachitsky16:13

      And for the questions on the PM side, do you have them do it at home or do you have them do it in the office?

   7. DSDavid Singleton16:17

      It's a variety. Uh, right now I've, at Stripe we're very much in a hybrid mode so a lot of our interviews are still happening over Zoom, although we have plenty of, uh, great activity going on in our offices as well. The written exercises tend to be something that you do in your own time. We, we suggest a time bound so it's not an unreasonable demand on your, uh, the rest of your life, and then the, the other exercises are, you know, with an interviewer

5. 16:39 – 21:56

   How Stripe built a product-minded engineering culture

   1. DSDavid Singleton16:39

      on Zoom.

   2. LRLenny Rachitsky16:40

      Okay. So speaking of product managers at Stripe, Stripe is kind of famous/infamous for waiting a long time to hire your first product manager. From what I understand it was like around the 200th employee and maybe five years in. Which to me tells me that you're really good at building product minded engineers, uh, or hiring and building a product minded engineering culture. How do you do that? How do you go about doing that? And then I'm also just curious what PMs ended up bringing to the table and how they kind of collaborate now.

   3. DSDavid Singleton17:06

      Yeah. So I mean at Stripe today we have lots of product managers, lots of engineers. Product management is an extremely valuable and important job family. I'll talk more about what folks do today. It is true that our, our original team, I think we hired our first PM a little bit earlier than you said, but our, our original team was, was really an engineering team. However, they were extremely product minded and I would honestly say that of all of the, the very early team that I've worked closely with and, and know well over time, every single one of them could also have been and, and essentially also was acting as a, as a PM. If you think about the nature of the f- of the first Stripe product, and again today we solve problems for businesses from the very small to the very large across a whole range of financial infrastructure. But the, the initial product was very developer focused and we still have a developer focus at the core of many things that we do. The best PMs for developer focused products, I think, are usually very technical, they're mostly former engineers themselves or maybe kind of current engineers still tinkering on the side, but bringing, you know, a lot of, a lot of user insight and strategy to, to the problem. And every single early member of the team at Stripe had to act like that in order to, to, to work the way that we do and that we want to. So remember again that is to get to know our users very well and then bring those kind of personal insights into our product development loop. So, I mean in fact the way we think about product development at Stripe, it, it really is to find the correct set of early users to kind of co-create the product with. So maybe, uh, the best example of that or at least a, an example of that is Stripe Billing, so it's our solution for subscriptions and invoicing. And when we got to starting the Stripe Billing, um-... product, we realized that there were a number of our existing users, these were companies like Figma and Slack, who were already using Stripe for payments, but had these subscriptions business models. And we figured that there were gonna be many more of these kind of companies into the future and we, we could see that they, they were really kind of pushing the boundaries of what was possible here. So we decided to co-create the product with them. Um, the, the early team, uh, working on billing got a bunch of those companies, i- included, uh, Slack and Figma, but a bunch more as well, got to know them, but the individuals who were operating those systems at those companies personally understood exactly what their challenges and hopes and dreams for the future were, and then brought them into like a kind of shared product development loop. So we had shared Slack channels, we'd actually show them product on a very regular basis, get their feedback on it. And only when that original kind of alpha group was super, super happy with the product did we then think it might be ready to go to, to a broader audience. So that is just how we build product at Stripe. And that means that every engineer building product at Stripe really has many of the kind of attributes and will exercise many of the attributes that you'll often find in, in PMs in, in other companies. But that doesn't mean that there isn't a really important job for PMs to do here as well. So something else that I think is worth knowing about Stripe and is somewhat unique is building product is also a much more kind of team sport at Stripe across functions. So there are the obvious functions that most companies have, engineers, engineering managers, product managers, designers, and those folks all get involved in, uh, you know, throughout the entire, uh, life cycle of product development at Stripe. But also if you think about the nature of our products, it turns out that a lot of our products are about abstracting over what the, uh, the financial system does at large. So sometimes there's a capability in a certain country and there's a different one in another country and we wanna make it all work consistently. So our financial partnerships matter. So sometimes folks from our partnerships team are part of the kind of early development, uh, phase of, of, of a product. Similarly, when you think about product legal and many of the other functions around risk and compliance and so forth, we actually need their creative thinking in order to build the right stuff for our users. So the point is, it's a way more cross functionally collaborative process than I've seen in a lot of other companies. And PMs play just this absolute linchpin role across a lot of that. So PMs are very frequently the ones that are providing the kind of locomotion to bring all of that, that partnership together. They're very frequently the folks that are synthesizing what we're learning from our users. Now everyone is talking to users, uh, across all those job families, but the PM is probably the person talking to folks the very most, and probably the one who's, who's synthesizing the very best. They often and usually also bring a lot of the product strategy. "So okay, this is what we're doing today, but what does it ladder up to? And therefore how do we make the right choices to make sure that we're taking the right longer term path?" And so it's a super important and, and valuable role. And uh, I, uh, I mean you, you've mentioned a couple of great ones, uh, on the call here already, but I think that the PMs at Stripe, uh, embodying a lot of, uh, a lot of the culture I talked about in our operating principles, uh, just contribute a, a tremendous amount to what we get done.

6. 21:56 – 25:39

   Stripe’s operating principles

   1. DSDavid Singleton21:56

   2. LRLenny Rachitsky21:56

      Perfect segue to where I was gonna go with the next question, which is one of your operating principles is to be meticulous with your craft. And I know this is something that you focus on a lot.

   3. DSDavid Singleton22:06

      Yeah.

   4. LRLenny Rachitsky22:06

      What I'm curious about is h- that sounds great. Everyone, you know, would love for that to be a, a focus of theirs and a core value in the way they operate, but there's always this trade-off of like, "We just gotta ship stuff." Like it doesn't ne-

   5. DSDavid Singleton22:17

      Mm-hmm.

   6. LRLenny Rachitsky22:17

      How do we wait until it's perfect? So what I wanna ask is just how do you operationalize this principle, first of all? And then I'm gonna have a few follow-up questions.

   7. DSDavid Singleton22:25

      Yeah, g- great question. Before I talk about this one operating principle, let me just set a little bit of context on-

   8. LRLenny Rachitsky22:29

      Yeah.

   9. DSDavid Singleton22:30

      ... what are our operating principles at all. So at Stripe we have operating principles and they are, they're kind of like corporate values, but they're not just values. They, they're much less abstract than that. So actually we were just talking about the early Stripe team. I think one of the best things that they did, I mean they did a bunch of great stuff, finding early product market fit, but one of the, the most, uh, foresighted things, uh, that they did was to kind of take a little detour and think about what is it about how we've been working so far that we think has made us successful and is going to be important as we continue to scale? Um, and, and we capture those in a set of operating principles. So their behavior is distilled from the most effective Stripes. Um, and we, we really care about them, um, so we, we talk about our operating principles a lot internally. I'll often see individual Stripes kind of celebrating each other's work in terms of which operating principle this kind of best represented. We frame a lot of our feedback processes and so forth around them, although not exclusively. The very first operating principle we have, by the way, is users first. So we've already on this call touched a bunch on how do we put our users at the center of understanding what we should be doing at all. And that idea that we form these deep personal relationships with our users to guide everything really comes, uh, from and is, is captured by that operating principle. We also have operating principles th- they're kind of segmented into how we work, who we are, and then a bunch that the leaders must execute.

   10. LRLenny Rachitsky23:52

       Mm-hmm.

   11. DSDavid Singleton23:52

       So I mean, a- another example of, of one of the how we work ones is move with urgency and focus. Like we really believe that even though we're building infrastructure that will persist for decades, it is important that we solve the needs our users have right now, you know, relatively rapidly so that it's, it, it actually makes sense and is, it, it is delivering value for them. So anyway, there, there's a range of them. You asked about being meticulous in our craft. So this is another how we work principle, and it is, it, it is very important at Stripe. And y- you also call out, Lenny, a kind of a challenge of being meticulous, which is, uh, well, there's so much to get done, so where can I be meticulous? Uh, because uh, well if you do it across everything, maybe e- a lot of progress would grind to a halt. Right? So well first of all, let me tell a couple stories. Um, so before I joined Stripe, I spent some time, uh, and this was a big part of deciding to join, playing with the product. And something that, that really delighted me was that when I started integrating the product, uh, there were parts of the experience where...I was stuck, right? So I- I- I made a mistake with how I'd integrated the API, and I was getting an error message back. But I wasn't stuck for very long because something that the team at Stripe had done was we- we made the error messages coming out of our API link to the piece of the docs that told you how to solve the problem. Genius. Um, that is a really, um, uh, good example of being meticulous in the craft. So it is- it- it's pretty uncommon and certainly very uncommon back then for companies to do this. Like, error messages on your API? Who's ever gonna look at them? Well, it turns out developers while they're building and if they run into problems, they will look at them. It's actually a really high stakes moment because it's a time that matters. So that is an example of where we've been meticulous in the product. And

7. 25:39 – 32:22

   How Stripe uses “friction logging” to build a meticulous product culture

   1. DSDavid Singleton25:39

      the- the way that we kind of figure out where to be meticulous, like where to really sweat the details and go above and beyond, is again we try to be as user first as we possibly can. So we have a process, uh, that is widely used across product teams and even, like, folks building stuff for- internally for Stripes to use, which we call friction logging. The- the way this works is you- you need to kind of put yourself in a particular user's shoes. Like, it's actually important to have a very clear idea of who is the person I am kind of modeling the friction for right now. So for instance, I might be, uh, integrating the Stripe Billing API, um, and I might put myself in the shoes of an engineer at, you know, let's say, uh, Atlassian, which is a- uh, one of the world's largest SaaS platforms and using actively Stripe Billing, uh, for- for, uh, automating all- all of their revenue today. I- I might put myself in the mindset of that person. I might have met them recently and kind of understood them a- a little bit, uh, better than, uh, than otherwise doing this particular thing. And then I will go through the process of actually using the product, starting off in the dashboard, hopping over to the particular page in the docs that I need to follow, starting to write code and so forth in order to do my integration, and make really careful and meticulous notes of what the- the experience is. So stream of consciousness notes, but paying particular attention to the places where I ran into friction that I think the- that particular user I- I'm mental modeling would find, uh, difficult. And then those are the places where we will tend to go and invest a tremendous amount in really thinking how could we solve this problem, and then putting a lot of attention into detail into making it right. And by the way, that- that- uh, the- the example I gave you around the error messages, there's actually more code in the jobs that serve the Stripe API to handle those edge cases than in the actual, uh, main flow. And I think that's quite remarkable. Most people wouldn't do that. But it turns out not only was it something I was impressed with, but when I talk to Stripe users, this is very frequently something they tell me, um, delights them about the product. It's a reason that they can adopt it more quickly and it's a reason that they keep adopting other Stripe products because they know that it's gonna be easier than- than doing anything else. So it really matters. So the point is, like, we are intentional about where the places where, uh, being particularly meticulous matters. And then when we are, we try to do it in quite a systematic way. It- it's great business as well. So for instance, um, we actually just earlier today, um, put a blog post up that explains we- we've been tuning our, uh, what- what we call our payment element, which is our kind of more f- kind of batteries included UI for putting payment integrations directly on your website. You can style it to fit with the rest of your page, but it has a lot of powerful features and we also have a set of hosted services called Stripe Checkout. And the point here is we've gone and scoured a whole bunch of, like, checkout flows on the internet over the course of many years and we look out for all of the little broken edges, and we've been working hard on these products to- to tune those. And then not- we don't just stop there. We've also been really obsessing about what are the things that we can do in the flows that we've already built that will, you know, remove latency or remove a click or whatever. And what we have found, we've recently measured the difference between... So with- with a bunch of actual users who'd migrated from a fairly vanilla integration where they, you know, built their own checkout flow to one of these surfaces, to our payment element or to Stripe Checkout. It turns out that it increases the average user's revenue by 10.5%. And that is- that's huge, right? So this is an industry where little changes that you- you might, uh, go about making, or even, like, big changes that you pour blood, sweat, and tears into will typically deliver uplifts in- that you measure in basis points. That's hundredths of a percentage point, and this, you know, th- this set of small changes compounding over time ladders up to 10.5%. It's huge. And the point is we only get there by knowing that this is a thing that might ultimately matter and being meticulous in every single step along the way and that it compounds to a dramatic impact in the end. So I think that- that particular, uh, set of experiences is- is one where this operating principle has really mattered. Thinking about those kind of flows is what led us to build Link, which is our product for, uh, making one-click checkout available. And that has had a tremendous impact, uh, both on convenience for customers of our users and also on their conversion rate and so forth. So it- it really matters. And then one final, uh, uh, other area if I may. I think Stripe is relatively well known for putting a lot of care and attention into our- our website, a lot of our, you know, marketing or product explanation pages. And I think for a business like ours, it's not immediately obvious that it makes a tremendous amount of sense to be super meticulous in building those experiences, but we are. And again, it- I think it- it really matters, uh, for us and for our users. So the- the way that we- we got into this mode was we recognized that we are building a lot of infrastructure. So we built what we call our global payments and treasury network. It is literally our product infrastructure for moving money into Stripe, holding balances on behalf of our users, moving money out, all of the automation, uh, between, uh, moving money between different businesses and all of the kind of regulatory stuff we have to handle, all handled in this- this big payments engine. But as a user...... it's very hard for you to kind of come in and inspect that and say, "Was it good?" Um, I mean hopefully you can go and talk to a bunch of other Stripe users, but it's very hard to know from the outside. So if we put a lot of care to, care and, uh, to, to care for detail and attention into the experiences that we used to explain the value and the power of that, it actually helps our users understand exactly how it is that we are trying to operate for them. And that's where all the meticulousness that we put into our site have come from. There are lots of features like, you know, we have a spinning globe on our, uh, page that shows you what, what kind of methods you can use in different countries.

   2. LRLenny Rachitsky31:36

      Mm-hmm.

   3. DSDavid Singleton31:36

      Uh, we have this kind of big animated wave on our home page. And, you know, in general, Stripe is kind of known for these, you know, great internet moments when we launch products. Um, and so we did it because we wanted our users to be able to see the care and attention that we were taking. It's turned out to have this, you know, wonderful kind of side benefit to some extent because those experiences tend to get shared a lot (laughs) like on Twitter and LinkedIn and elsewhere because they're worth looking at just 'cause they're beautiful and they, you know, often push the web platform forward. And that then means that folks are likely, folks who themselves want to push internet business forward are likely to know about Stripe and know about our capabilities. So meticulousness plays out with impact in so many different ways at Stripe, um, and, uh, I- I- I enjoy, uh, you know, working that way in, in many different areas.

8. 32:22 – 35:02

   How to operationalize friction logging

   1. DSDavid Singleton32:22

   2. LRLenny Rachitsky32:22

      I have, uh, a few follow-up questions along those lines. But before we do that, I wanna double click on this friction logging process.

   3. DSDavid Singleton32:28

      Sure.

   4. LRLenny Rachitsky32:28

      Just to understand how you actually do that. So maybe two questions there. Is this like a regular task people have on their plate, like say as ex- executives or just PMs in general? And is there like a template that you share of just like you talked about who we are you, what company you're working for, what problem are you trying to solve? How do you actually kind of tactically operationally use that?

   5. DSDavid Singleton32:50

      Yeah. Great question. So it's a, it's a practice that is quite valuable generically. There are many, there are many places where you can apply friction logging in order to, uh, really kind of shine a light on what is the most e- effective place to invest time and effort. So we use the practice across lots of different functions and, uh, in lots of different ways at Stripe. But it is the case that almost every product team has, uh, someone, it's often the PM, sometimes it's the engineering manager, who has a regular repeating loop of going through the, you know, end to end flow of the product and writing a friction log. For years and years, I have gone through the process of onboarding as a user to Stripe once a month, writing a friction log, and then tagging in the right people, uh, across the company, uh, that might need to take action on some of the things that we're observing. And one of the reasons that that kind of process of taking a big step back, um, is useful is at this stage we have many people, we have, uh, thousands of engineers who are working in parallel. And while everyone cares a lot about being meticulous, paying attention to detail, getting the details right, it can often end up with experiences that diverge. And going through this process of looking at it all together on a regular rhythm with friction logging really helps us maintain a cohesive whole while we all operate in parallel. And so therefore, you know, senior leaders, executives, I guess, of, uh, our larger areas will often do this as a practice themselves in order to kind of make sure that everything that they're responsible for is coming together well. So it happens kind of recursively at different layers. To your question about is there a template, it's a relatively simple process. So yes, there is a template, and in fact there's a stripe.dev article that maybe we can put into the show notes that has the template, but it literally say what you want to do, be very explicit about the user that you are, uh, trying to have a mental model for, 'cause that helps you make the right choices as you go through flow. And then really just keep a very clear stream of consciousness log of, uh, of what you're finding as you go. By the way, also really important, like praise the stuff that's good.

   6. LRLenny Rachitsky34:49

      Mm-hmm.

   7. DSDavid Singleton34:49

      So I, I'll often send these, these docs around to, you know, a lot of folks inside of the company and it's a great opportunity to recognize great work done. So if I ran into that error message that links the documentation, that's a great example where I'd be like, "This is awesome. Nice work."

9. 35:02 – 36:53

   How to set PMs up for success

   1. DSDavid Singleton35:02

   2. LRLenny Rachitsky35:02

      Being a PM that has received emails from the CEOs of companies with all the problems they ran into and they're trying to book, uh, it's often like, "Oh God, I have like so many goals I gotta hit and uh, I have this timeline, I have these things I'm working on." What is the culture like/how do you give teams space to do these things that are just like, we're just gonna make the experience better? How do you actually do that so that PMs aren't stressed out when they get these things?

   3. DSDavid Singleton35:26

      It, it starts with our operating principles. So because we have operating principles of users first and being meticulous in our craft, we actually tend... All of our kind of ways of building plans tend to be wired quite well around this idea that we're going to reserve enough time-

   4. LRLenny Rachitsky35:40

      Mm-hmm.

   5. DSDavid Singleton35:41

      ... uh, to really make the experience good. Maybe, you know, the most pronounced version of that is not even the issues we identify in friction logs. Um, maybe, maybe in a bit we can talk about it, how we invest in reliability and so forth.

   6. LRLenny Rachitsky35:53

      Mm-hmm.

   7. DSDavid Singleton35:53

      But things go wrong. And one of the things that's very important to me is that we are a learning organization. We need to understand why they went wrong and then take action to stop the same thing going wrong in future. And so that means that we identify incident remediations, that's what we call those. And we prioritize those carefully. And most of them, the ones that matter, actually get prioritized before other work on the roadmap, which means that as a PM and when you're building your roadmap and your plans, you do have to think about, well, approximately how much bandwidth do I need to reserve in this area in order to address, um, polish stuff that might come up through friction logging and also to take care of, you know, tho- those kind of operational things as well. It varies by team. It varies by stage of, of product. So there is no Stripe kind of like, "We will, uh, reserve 50% of our time to do this stuff." That, that wouldn't make any sense. But we do encourage and, and ask every team to think hard about how much should they be setting aside for those kind of activities. Does that make sense?

   8. LRLenny Rachitsky36:47

      Absolutely. I was gonna ask you if there's a percentage and, uh, basically you trust the teams to set aside the right amount of time.

   9. DSDavid Singleton36:52

      That's right.

10. 36:53 – 41:17

    Stripe’s collaborative approach to product evaluation

    1. DSDavid Singleton36:53

    2. LRLenny Rachitsky36:53

       Awesome. Okay. So sort of along the same lines, I actually asked Shreyas Doshi what I should ask you.

    3. DSDavid Singleton37:00

       Oh, cool. (laughs)

    4. LRLenny Rachitsky37:01

       (laughs) You had no idea I was gonna do this. And, uh, what he said is I should ask you how you did UX reviews pre-shipping-... right after you joined Stripe and you just said you were very good at this. And, uh, so how do you, what is- what is that like and, and how should people, uh, learn from your experience?

    5. DSDavid Singleton37:19

       The way that I did UX reviews very early on in my time at Stripe and like the way that every group at Stripe does these reviews is, uh, employing some of the techniques we talked about already. So that- that active friction logging, it- it's very often done asynchronously. So someone will sit down and reserve, you know, an afternoon going through the products and make meticulous notes, and that's super useful. We also find it very valuable to go through that process together so we will bring together, uh, both the team that built the product that we're looking at, but also a lot of their cross-functional partners. So things like this, the support folks that are going to be handling or are kind of building the process to handle, uh, questions from users about it. Uh, executives in the org that they're a part of, um, to do these, uh, these walkthroughs where we're taking exactly the same approach as I described with friction logging. So imagining we're a particular user and experiencing the product together and then just discussing very openly, we actually, the way that we do this today is we have a- a log of issues that we want to talk about that- that anyone can type into while, uh, the PM usually is walking us through the flow and then we'll get together at the end and actually talk about each issue and, you know, does this need to be addressed? What do we really think about it? So that- that's the approach both that- that I took early on and that we take in a general, at scale today. There's a lot of benefit to- to doing this together. So someone once described this to me as if you were a chef, you're gonna- you're gonna taste the soup and you're gonna talk to your kitchen staff about, like, what went into the soup, when it was good and wasn't so good. If you run a movie studio, you're gonna sit down and, like, watch a lot of movies together. And so actually kind of looking at product together I find extremely energizing and also really valuable in actually teaching some of the kind of bar for craft and- and values around how we want the product to inter-operate for our users at scale across the company. And building on that, um, there's something that- that- that we do occasionally and- and- and I've done occa- occasionally at Stripe called Walk the Store, um, where we'll actually do that process of, like, looking at the product together with the whole company. So, uh, we- we have a weekly meeting called Friday Fireside where, you know, not, you don't have to attend, but the majority of the company does attend. And we've done a few series where we'll actually go through some, you know, really interesting and critical product flows together and talk about, you know, how this is reflecting various, uh, you know, priorities that we have and- and shifts that we're trying to make, but in- in particular with the user experience and a particular user at the core. And that turns out to have a tremendous amount of benefit and- and value in helping just everyone have a s- a shared language for talking about things and get on the same page about everything. So yeah, those are some of the techniques that- that- that Shreyas might've been thinking of.

    6. LRLenny Rachitsky40:02

       It's amazing how many directions you all come at to create this high-quality end product. Walking the Store, friction logging, these, like, entire company meetings looking at the product, UX reviews. Like, this is how it's- it's possible to build something so high-quality, right? It's not just we have this value and we're gonna be building great stuff. It's like you have to come at it from so many places.

    7. DSDavid Singleton40:24

       Yeah, that's an interesting observation. I think almost anything that you talk about as a value, and being a value it matters, but you need to have a practice behind that means that the value becomes real for everyone. So we very frequently find that for any product development team, so long as they are, they identify the right metrics that actually represent the experience that they want to deliver for their users and then get together frequently, like predictably to look at how the team is doing against those metrics, but then everything else just happens naturally because you- you understand what you're trying to move and you, every little micro decision you're making puts in your own time prioritization and then in terms of the actual work you're doing will ladder up to that. And so why do I say that? It- it is like you have to have the predictable and regular thing that ladders up to the value you're trying to deliver in order to- to make it happen.

11. 41:17 – 42:58

    Advice for presenting to CTOs

    1. DSDavid Singleton41:17

    2. LRLenny Rachitsky41:17

       Coming back to the UX review, just a question there. Presenting to the CTO is often very stressful in a review like that. What advice would you give PMs and designers and just leads of a team for how to prepare for a meeting like that, whether it's Stripe or anywhere?

    3. DSDavid Singleton41:33

       I personally try to be as- as friendly and un-scary as possible, but I know that no matter how much I do that, that it, these can be high-stakes meetings. At Stripe, and I think this probably goes for most companies, but at Stripe the- the- the main answer here is put your user's hat on and if you understand your user well and what they're seeking to get out of the experience and anchor any, you know, questions you get asked or, uh, wobbles you may feel of like, "Oh, that was, like, out of- out of the blue question," back to, "Well, here's remember what we're trying to do for our users." That- that usually makes any meeting like that go really well. And so that would be my- my main piece of advice. So I mean, a risk that exists in any company as you get to, you know, run a bigger team or whatever is that individual contributors, individuals will have, you know, very small amount of time with you in general. So there's always a risk that you might make a kind of, like, fairly unimportant remark and it will be taken out of context to be something very important. So I- I personally also try very hard to anchor feedback I'm giving in what is the, what's the user experience we're trying to deliver? Does this actually matter? Um, recognizing that- that- that is a risk. Um, and, uh, yeah, it definitely takes constant practice.

    4. LRLenny Rachitsky42:51

       You've touched on a couple of these things, but just if someone's trying to get better at building product, either a PM or designer engineer,

12. 42:58 – 45:28

    How to get better at building products

    1. LRLenny Rachitsky42:58

       what kind of advice do you often give for helping people just build better product?

    2. DSDavid Singleton43:02

       Almost always goes back to things we've already talked about here. So remember at the very top I talked about iterating very closely with users? If you have a mechanism to listen to users, get something in their hands...... quite quickly and then get their feedback on it to run it back through a feedback loop, you're very unlikely to go wrong, especially if you put a lot of thought into these being the right users to whose needs you want to focus on because they ladder up to your strategy. It's actually very hard to go wrong if you, uh, if you have that feedback loop working really well. So if you don't have that feedback loop, and actually it's remarkable as I talk to folks, you know, across the industry and so forth, it's remarkable how frequently that feedback loop doesn't exist in product development cycles. So if you don't have that, go figure out how to create it. And if you do have it but you can't get something into users' hands very rapidly from you having the idea that this might matter to getting their feedback on it, figure out how to make that go faster as well. And at Stripe, we focus a lot on all of our developer tooling and infrastructure on making it possible for changes to get delivered continuously to production so that we can get them in front of users very rapidly, and you know, that- that's really important.

    3. LRLenny Rachitsky44:08

       (instrumental music) This episode is brought to you by Braintrust, where the world's most innovative companies go to find talent fast so that they could innovate faster. Let's be honest, it's a lot of work to build a company, and if you want to stay ahead of the game, you need to be able to hire the right talent quickly and confidently. Braintrust is the first decentralized talent network where you can find, hire and manage high-quality contractors in engineering, design and product for a fraction of the cost of agencies. Braintrust charges a flat rate of only 10%, unlike agency fees of up to 70%, so you can make your budget go four times further. Plus, they're the only network that takes 0% of what the talent makes, so they're able to attract and retain the world's best tech talent. Take it from DoorDash, Airbnb, Plaid, and hundreds of other high-growth startups that have shaved their hiring process for months to weeks at less than a quarter of the cost by hiring through Braintrust's network of 20,000 high-quality vetted candidates ready to work. Whether you're looking to fill in gaps, upskill your staff, or build a team for that dream project that finally got funded, contact Braintrust and you'll get matched with three candidates in just 48 hours. Visit usebraintrust.com/lenny or find them in my show notes for today's episode. That's usebraintrust.com/lenny for when you need talent yesterday.

13. 45:28 – 52:03

    Stripe’s “engineerications” and the importance of getting into the weeds as a leader

    1. LRLenny Rachitsky45:28

       Something that I've heard about you is that you get into the weeds, you get really deep as a CTO of a company at the scale of Stripe, and I came across this term engineer vacations, which I think connects to this. Can you talk about that and how you think about just getting in the weeds?

    2. DSDavid Singleton45:44

       First of all, in terms of how- how much getting in the weeds matters, at Stripe, we find that it's very important for engineering managers in particular, but really all managers, to- to really have a very, uh, detailed understanding of whether their teams are kind of on the right track and where they're getting stuck in order to make sure that we make the most progress in unit time for our users. So we find that, again, that the nature of the problems we're solving really reward domain expertise, like assimilating a deep understanding, and we ask all of our managers to be kind of the editors-in-chief of their team's plans. And I think the only way to do that is to get the right loop for yourself to understand what is actually going on on the ground. Now, it would be really damaging if I was only doing work, you know, at the IC engineering level e- every day. Like, that- that doesn't ladder up to someone who's able to, like, help guide the team or whatever. So it's important to do this judiciously, but done, uh, on the right cadence, I think is very, very valuable. So engineer vacation is something that I- I personally do. So it's obviously a portmanteau of engineer and vacation, but it's not a vacation.

    3. LRLenny Rachitsky46:46

       Okay. How do you get there? (laughs)

    4. DSDavid Singleton46:47

       The way... The- the way we get to the name of the term, or at least I- the way I think about it, is what you do on engineer vacation is I'll clear, and other people try to do this as well, but clear several days in a row, three or four days, actually join a team, pick up a small task, hopefully a small feature that we can get all the way from start to finish, like in production, um, and do that, going through the exact experience the team has. So you get to understand the developer tools, the build infrastructure, how you get stuff reviewed, how good is the documentation, how long did I have to wait in order to actually, you know, see my thing live in- in order to start that feedback loop that I talked about with users? So you're actually going and joining a team and doing some work. While one does that, uh, it's really valuable and important to keep a friction log, because then what you want to do is write up the experience both as an aide-mémoire for yourself, because it then helps you go and, um, you know, for all of the- the next, like, year's worth of conversations I might be involved in around, uh, making trade-offs and helping set priorities, that context really helps. So I- I actually reread these reports periodically, and it's also really valuable to actually share them with the team, which then demonstrates, well, I understand what it's actually like and here's how we're going to kind of, you know, make sure that w- we're carrying that information through into prioritization. So that's what you do. The- the name is, uh, it's often the hardest thing about these is finding the time to clear your calendar for that many days. So the reason I think about it as like a vacation is, of course, people go on vacation. By the way, I work very hard to use all my, uh, vacation days every year. I think that taking a break and- and recharging your creative juices is valuable. So the point is like when you're on vacation, the world goes on and it's all fine. So I treat it like I'll decline every meeting in order to- to clear my- my- my schedule and have a maker schedule in order to do that. And so I- I've done this, you know, many times at Stripe, I- I advise engineering managers at Stripe that they should do an engineer vacation sometime in their first quarter to six months at the company. Uh, it gives you a tremendous amount of context for- for what your teams are actually experiencing and then for folks to do it once a year on an ongoing basis, and it- it provides a tremendous amount of context.

    5. LRLenny Rachitsky48:49

       That is insane. I've never heard of that process. How do you (laughs) stay knowledgeable and up to date on the- the infrastructure and the code that you have to code in all of a sudden?

    6. DSDavid Singleton48:59

       Well, I mean-

    7. LRLenny Rachitsky48:59

       Things move so fast.

    8. DSDavid Singleton49:00

       The- the process is literally to help you do that. Uh, I- I guess maybe something I left out in explaining it is, um...... will often identify a buddy on the team who's gonna, like, help you through.

    9. LRLenny Rachitsky49:11

       (laughs)

    10. DSDavid Singleton49:11

        So, um, for what it's worth, at Stripe we write, um, a lot of code in Ruby. We write them, w- we write a lot of code in other languages like Java and TypeScript as well, but, uh, our kind of core infrastructure i- is mostly implemented in, co- core product infrastructure's mostly imple- implemented in Ruby.

    11. LRLenny Rachitsky49:24

        Mm-hmm.

    12. DSDavid Singleton49:24

        When I started, I had never written a line of Ruby in my life, and I knew I wanted to do this thing, I'd actually done something like it before, uh, joining, and I was really scared. Like, what kind of, like, w- how much credibility might I lose if I show up and I can't write a line of Ruby? But my, my first buddy, uh, was, uh, a guy called Akshay, who's still at Stripe doing great stuff, um, and he helped me learn Ruby during these three days. And yeah, they ribbed me a little bit, but, but ultimately everyone really appreciated that I'd done it.

    13. LRLenny Rachitsky49:51

        Imagine being the engineer that has to review your PR.

    14. DSDavid Singleton49:54

        I tell them this is an important, uh, part of the process, "Please don't treat me any different to anyone else." And, you know, obviously, uh, to make this work well, you have to set the expectation that you're not gonna, like, (laughs) take immediate action if something isn't exactly the way you want it to be, and it's a more kind of longitudinal process.

    15. LRLenny Rachitsky50:11

        Is there something that you found in this process that comes to mind that was really surprising or interesting or just kind of lasting in the past s- in a couple of years?

    16. DSDavid Singleton50:18

        One of the most interesting things that I think I've run into is, there are a lot of places that, as we scale, we know that it makes a lot of sense to invest in automation, and there were a bunch of places in our development process where while we've done that, um, we were working hard to help each other out and actually saying, "Hey, please come talk to this other team if you want to use this thing." And if you're working, for instance, in a different time zone from someone else, it's often the case that actually it would be much better just to document it well and automate it and then maybe make the, uh, the consultation available async. And so, like, running into those kind of pieces of friction and then, you know, having a good conversation about it has, has, has really helped. But one good thing about Stripe, I think, is that, that kind of change doesn't depend on, on me doing anything. So we really care about making Stripe a place where engineers can do some of the best work of their careers, and that means being enabled with good tools and good developer productivity, so we invest a lot in developer productivity. We have a team, um, who, uh, work on our dev tools and they actually run exactly the same product development process that I just described for our external users internally. So, understand your users really well, talk to them a lot, um, as, so for instance, we do a monthly survey, uh, we operate at enough scale now where we can ping enough people once a month that we get a statistically, uh, significant sample of the entire organization without having t- to, to get everyone to respond, every individual who responds but once every six months, and we, we ask f- you know, very directly, "What's the experience you're having?" And then we use that to prioritize the roadmap of our developer productivity team. We also measure everything, um, so we know where people are getting stuck and, uh, and when we compare both the kind of free responses and, and the data, it helps us invest in the best places to make everyone else more productive.

14. 52:03 – 59:29

    Auto-testing and other strategies to improve shipping speeds

    1. DSDavid Singleton52:03

    2. LRLenny Rachitsky52:03

       Okay. That's exactly where I was hoping to go next. And let me set this question up. So I saw a tweet of yours recently where, um, I have some notes here I'm gonna read, that you deploy changes to your core API 16.4 times a day on average, your uptime has been 99.999%, and one in 10 people in the world have transacted with a business powered by Stripe at this point. And so my question is just, what does it take to achieve something like this, especially in terms of tooling and culture and, and everything else you were just talking about?

    3. DSDavid Singleton52:33

       Well, I think it's worth saying, at Stripe we are trying to do something, I think, relatively unique, which is the, what we, what we power for the businesses that build on Stripe, our users, it's, it's really business critical to them. Like, we're literally talking about money coming into your business to help you, uh, run your operations or make it e- e- even possible for you to run your operations and, you know, maybe pay your employees and so forth. Um, all the revenue automation that we do around billing, subscriptions, our financial automation products helping you close your books, like, this stuff matters. You cannot get it wrong. And we also operate... Stripe today, like, we, we move as much money a- as all of e-commerce was when Stripe got started.

    4. LRLenny Rachitsky53:13

       Mm-hmm.

    5. DSDavid Singleton53:13

       So we operate at very significant scale. And so business critical, significant scale, we just have a tremendous duty to our users t- to get this right and be extremely reliable and available for them. So we do think about that a lot. Now, there's one way to be very reliable, which is to try to change things as infrequently as possible. Like, never change anything, then you have many fewer opportunities for things to break. But we don't take that approach. Um, the needs of our users are, uh, evolving so rapidly, the number of ways that we want to serve them and can serve them is evolving rapidly enough that it m- matters a lot that we can operate, uh, in a kind of constantly changing environment. Hopefully I've explained, like, why that matters, like, that type feedback about the user is only possible if you can actually get something in their hands.

    6. LRLenny Rachitsky53:57

       Mm-hmm.

    7. DSDavid Singleton53:58

       So we choose to design the way we work to hold those two things true at the same time. And it does take a lot of care and attention and it takes a lot of systems. So, um, maybe just to kind of build this up, uh, first of all, there's a lot in our development process and the ways that we take, uh, changes into our product, uh, that, that makes this possible. One of those is, we really care a lot about having really good test suites. So we believe in automated testing, we don't have manual testers because manual testers couldn't possibly cover the, the vast array of API endpoints and configurations that we offer. But automated tests can. So we work hard to have a lot of automated test coverage, and then every single change that an engineer produces gets run through this battery of tests before it can even possibly go towards production. And then we work very hard as changes end up in production to put them through increasingly, uh, realistic and then more broadly exposed environments. So we have a bunch of staging environments where we'll, like, send a, a battery of, like, more realistic end-to-end tests. Then finally, when something actually goes to production, it starts at a very small percentage of the traffic and then ramps up to the whole, so we can detect problems before they become huge problems.There's a number of things that we had to do to make that all possible. So for instance, um, every change that a Stripe engineer submits, once it's passed those tests, it actually ends up in production automatically, um, uh, e- over the course of the next 45 minutes or so. Um, and that th- the ... I don't think there are a lot of financial services companies, at least not, uh, until maybe the last few years, that have operated that way. And so that takes a certain mind shift. You have to actually, actually have to assume that that's going to happen and put the right systems and processes in place. And then the other thing that's important is to recognize that we just, we, we, we have to obsess about getting, like, very systematically good at addressing anything that can go wrong. So I mentioned earlier, it's really important to me that we are a continuously learning organization. And I mean, s- something else that matters of course, is things will go wrong. You know, sometimes there is a downstream partner where something breaks. Other times there is, you know, uh, a particular kind of network outage out of our control or whatever. So things will go wrong. So it's important that we have the right systems in place that minimize the damage that any individual thing going wrong, uh, can cause. And we work hard on that. We have redundant systems in a bunch of places. Uh, we think hard about how something that breaks for one user wouldn't kind of carry over into, into other users. And then we actually very actively work hard to put things right when they are wrong. So that's called incident response at most companies, including Stripe. Um, I actually think Stripe is very, very good at incident response. We've got very good tools for both declaring incidents and then pulling the right people together to, to put them right. But we don't stop there. We really obsess about reviewing them carefully and identifying not only, like, what would have stopped this thing happening, but how could we prevent this whole class of issues in future? And as I said earlier, we prioritize working on that stuff ahead of anything else on the roadmap. That's because of, remember what it is we're trying to do, how, how important it is, and how if we don't do that well, we can't move quickly for our users. So that's how we do it. By the way, I don't want to, I don't want to kind of come across here and sound like we've got it all figured out. Of course, all of this is always entirely, you know, in flux. We're always anxious to figure out how we can make it go better. For instance, in recent years, we realized that we could get a lot of benefit from what we call chaos testing, that's like injecting errors and making sure that the systems respond in such a way that it doesn't cause any impact on users. So it's constantly evolving and we, you know, we re- really enjoy learning from other companies and learning from our users as well. But it's something we care a tremendous amount about and think very rigorously and systematically about it.

    8. LRLenny Rachitsky57:40

       Did you say that it takes only 45 minutes from pushing code to it going into production?

    9. DSDavid Singleton57:45

       Typically, yes. So we, we ... That battery of tests that I described that gets run on every change, um, that gets run in parallel to when you send it for review by another human.

    10. LRLenny Rachitsky57:56

        Mm-hmm.

    11. DSDavid Singleton57:56

        So it's run once, typically takes about 15 minutes. And then, uh, once the change is merged into our code base, we run that same test suite one more time, so another 15 minutes, and typically it takes about 30 minutes for the systems to automatically deploy it in production. That's, that's how we run. Um, and, you know, think about establishing that tight feedback loop with users. That means you can get feedback from a user in the morning. You can figure out how to address it, and you can actually put something back in their hands by the end of the day. And that loop running inside of 24 hours, I think, is pretty important.

    12. LRLenny Rachitsky58:25

        That's incredible. I remember times at Airbnb where it took hours for tests to finish because they just, there was a lot of them. And they fixed that over time, but I'm used to more on, on that scale.

    13. DSDavid Singleton58:37

        It takes constant effort to, to hold those. Those, those numbers, by the way, are important. Like, those are about the right numbers, I think, for a company operating our kind of scale. Of course, as you add more stuff, you add more tests. So we've had to work on, we've had to invent mechanisms to make it run more in parallel. We now have a thing called selective test execution where we figure out for the, not the battery that runs before it goes into production. That we run everything and just run it in more machines to make it parallelized. But for the changes, the, uh, the tests that we run for individual changes, we actually have, uh, systems that figure out, well, which tests are the ones that might be material for this particular thing? Um, and that's been, like, a real source of innovation. In fact, our, our distributed, uh, change and test environment at Stripe is the biggest distributed system that we actually run. Um, and so running that is, you know, requires just as much energy and, and effort and rigor and, and craft as everything else.

15. 59:29 – 1:00:54

    Improving developer productivity

    1. DSDavid Singleton59:29

    2. LRLenny Rachitsky59:29

       What are some of the things you've done that have had the most impact on developer productivity?

    3. DSDavid Singleton59:34

       We've touched on a couple of these already. That, that auto deploy mechanism definitely had a very significant individual impact. So before we introduced that, so th- that sometime within the last five years, each deploy to production required an engineer to actually babysit it, to, like, watch it as it went live, make sure all the charts looked good. And it was a bit of, it was a bit of a kind of game of roulette because you always wished that your change would get bundled in with someone else's so that they were paying attention to it for you. So literally being able to be, like, fingers on keyboard on something else while your change is going live and it's being automatically monitored, that, that was a big, uh, improvement in our productivity. Also, like, very small things. Th- this is just like optimizing checkout. Very small changes really compound in a big way. So for instance, about eight months ago we made it possible, um ... We've got a, we use a tool for code review. Um, uh, it's, it's quite popular. It's GitHub Enterprise. But we've built a bunch of our own, uh, experiences and controls around that. We have a little tick box that you can tick on your, uh, on your pull request, that's what you call an individual change that an engineer produces, that is auto merge, which means once the reviewer says this is good, I don't need to come back and look at it again. The system should just take over.

    4. LRLenny Rachitsky1:00:43

       Mm-hmm.

    5. DSDavid Singleton1:00:43

       And that just cuts out one whole kind of human distraction step and that laddered up to a big change in productivity as well. So these very small changes can, if you're deliberate with them, ladder up to

16. 1:00:54 – 1:07:03

    How AI has impacted the way Stripe builds product

    1. DSDavid Singleton1:00:54

       a big impact.

    2. LRLenny Rachitsky1:00:55

       I'm gonna go in a different direction now. AI, very hot right now.

    3. DSDavid Singleton1:00:58

       Yeah.

    4. LRLenny Rachitsky1:00:59

       Has AI impacted the way you all build product yet? And if not, where do you think it starts to go in the next few years?

    5. DSDavid Singleton1:01:06

       Yeah. We're very excited about AI. Uh, now to just take one step back, Stripe has been using machine learning and, you know, advanced machine learning techniques at the heart of our product since the very early days. Um, Radar is our solutions for payment fraud. ... and it has used, uh, I mean, this is the very core of our product and has used machine learning since we introduced it. We also, if you think about the nature of Stripe, we operate in an environment where we have to do a lot of work to kind of catch bad actors in the system, fraudsters or fraudulent businesses, so we've been employing a lot of machine learning techniques there, again, for years, and, you know, we couldn't operate the company without them. So, you know, the difference between ML and AI, I think when we think about AI today, we're mostly talking about the applications of large language models, but those are really based on this new technology called Transformers. That was a paper that came out of Google a while back, uh, about four years ago, five years ago, uh, and we put Transformer technology into those systems, uh, at some point more than a year ago, and it- it- it's- it's proved to have a very profound impact on our ability to solve those problems for our users, which- which is great. But we are also excited about, uh, the applications of large language models. I mean, there- there's really two dimensions to this. Number one, we feel privileged to be able to help serve and power a lot of businesses getting, uh, started in this space. One big difference between AI startups and- and others is it's actually quite expensive to operate an AI startup in terms of the compute resources, like running this stuff in these GPU machines is- is quite expensive. So typically, um, these companies need to have a monetization model from day one, and I'm really excited to say, like, most of the AI economy is- is running on Stripe, and we're working very hard to make sure we serve all their needs. OpenAI, for instance, is using us for, uh, monetizing, uh, ChatGPT Plus-

    6. LRLenny Rachitsky1:02:56

       Mm-hmm.

    7. DSDavid Singleton1:02:56

       ... and all their other products, but they don't just stop there. We're actually helping power all of their subscriptions and, uh, revenue tracking and- and financial operations, uh, around, uh, the business. And the point about that is, we've been putting very large engineering teams on this stuff for years, and that means, you know, anyone who wants to build a- a- a reconciliation product and a subscriptions business model, if you want to do that yourself, you have to put, you know, big engineering teams, uh, on them. But if you can use Stripe instead, you can actually focus those really precious engineering resources on keeping up with the rate of, you know, breakneck innovation in this space. So we're excited about that, but we're also really excited about the applications of these AI techniques on our own ability to serve our users better. And so for instance, at sessions we'll be talking about a few of these. We wa- w- we started working with OpenAI when they had the beta, it was kind of private beta of GPT-4 available at the beginning of this year. The first thing we thought of was we have a lot of documentation on Stripe and we put a lot of care and attention into making it good, but if you want to achieve something with your Stripe integration, you're typically gonna spend quite a lot of time, uh, reading docs and kind of synthesizing them as- as an end user. We realized we can have GPT-4 read all of our docs, store them as embeddings, and then answer questions for developers, um, and so we now have that, um, available in kind of early stage release, uh, inside of our documentation, it's turning out to be pretty valuable. We've also been able to apply these techniques to other areas of our products. So something that we're gonna show an early version of at sessions is, we have, uh, a really powerful product as part of our revenue and financial automation suite, which we call Sigma. It allows you to write SQL queries across all of your Stripe data so you can like interrogate your Stripe data to understand your business in really fine grained detail, you know, which country has the fastest growing sales, which segment in- in which country has, you know, uh, particular attributes to it. Very powerful product, but it's one that is, uh, potentially challenging for non-developers to adopt. Like you do have to know how to write SQL queries, unless you just use the ones in the menu, in order to get the full value out of the product. Well, it turns out with large language models, we can apply this technology where you can ask questions in natural language and our engine will actually write the SQL query for you. And we've had to do a lot of work to kind of tune that to make sure that it's reliable and understandable and it's going really well. And we also see great applications in actually applying these technologies to make us more efficient internally, answer users' questions faster, help each other out, uh, more quickly, and so we're doing those things as well. One concrete thing that we did, which I'm- I'm pretty excited about, is not long after we saw ChatGPT come out, we realized it would be really awesome if we could apply that to many use cases inside of Stripe, but as you can imagine, a lot of the data that we, uh, are dealing with on a daily basis is, uh, very sensitive to our business, to our users, um, and we care a lot and we have a lot of governance around this, but we wanted to make that technology bill. We couldn't say, "Hey Stripe, go and use ChatGPT." So we stood up an integration to GPT-4 and an internal UI to- to use models like that, but here's the thing that I- I want to kind of relate to all of your viewers. We find that we built presets into that. So when you- when you work with large language models, you wanna write a prompt that then helps get the model into a state where it's going to be able to- to solve the particular problem at hand. And we find that writing prompts is something that's accessible to folks across lots of different job families, not just engineers. Um, folks in our, uh, marketing team, folks in our user support team have been able to use this as well. But you put- you can put a lot of care and attention into getting a prompt that works really well and we know how to make it possible for those to be shared across the company.

    8. LRLenny Rachitsky1:06:38

       Mm-hmm.

    9. DSDavid Singleton1:06:38

       Um, so I can grab the preset to help me, um, write, you know, a blog post with the- the right kind of tone and I find it extremely valuable in- in- in helping me and- and many other people across the company. So I think there's a lot, uh, a lot of innovation in how companies, uh, serve their users and operate is- is possible right now, and we are working hard to employ all of that on behalf of our users.

17. 1:07:03 – 1:09:24

    Why David is excited about Copilot

    1. DSDavid Singleton1:07:03

    2. LRLenny Rachitsky1:07:03

       What about as engineers? Do you, are you pro Copilot? Is there something along those lines you find useful, weary of? How do you think about that?

    3. DSDavid Singleton1:07:12

       We're excited about Copilot. We- we've made Copilot available to our, uh, engineers internally. Um, we ran a, like, fairly rigorous trial, uh, to make sure that we felt good about, uh...... about its actual, you know, impact on, uh, our ability to write code and the, the, the code it was actually generating, and find that it was very effective for us. Both in productivity, but also just in helping our engineers feel, feel like they could direct their energy towards some of the bigger problems, like how should this stuff fit together, rather than some of the micro-problems. And so because we had such great feedback on that, we've, we've made it available, uh, very broadly internally and, uh, we'll be very excited to see where that, that space develops over time.

    4. LRLenny Rachitsky1:07:56

       Do you have a sense of the productivity gain that, say... I imagine you have a lot of 10X engineers. The gains that an amazing engineer gets out of a co-pilot versus, you know, a newer engineer. What impact do you think-

    5. DSDavid Singleton1:08:07

       It's honestly too early to tell-

    6. LRLenny Rachitsky1:08:09

       Yeah.

    7. DSDavid Singleton1:08:09

       ... because we've only recently made it available at scale. What we're typically seeing is that the kind of tasks that it accelerates are, um... Remember I mentioned how much e- energy we put into having comprehensive test suites? It's actually using co-pilot to generate your test cases turns out to be extremely valuable.

    8. LRLenny Rachitsky1:08:25

       Mm-hmm.

    9. DSDavid Singleton1:08:26

       Uh, there's, there's a reasonable amount of boilerplate that is kind of like repeated code and concepts that goes into writing good tests, but you also have to reason very carefully about, "Is this actually testing exactly what I want?"

    10. LRLenny Rachitsky1:08:37

        Mm-hmm.

    11. DSDavid Singleton1:08:37

        So copilot takes a lot of that boilerplate generation out of the way, so that you're actually thinking about the stuff that really matters and I think that is going to have a profound impact on quality of code that we write as well. But it's a little too early to say, 'cause we literally just rolled it out, what kind of actual measurable impact it's having.

    12. LRLenny Rachitsky1:08:54

        I was talking to an engineer and he's just like, "I like the act of writing code and it makes me sad that this is doing this for me." So he, he turns it off, but I think over time people will be like, "Yeah, that's true. I never really enjoyed this part of it."

    13. DSDavid Singleton1:09:07

        Yeah. I, I mean I, I'll speak from personal experience. I love writing code. Like I really love writing code. It's pretty much what I do for fun. I also love writing code with Copilot because as I said, it, it means that I can focus on the parts that really matter and I think that's been a fairly consistent experience across Stripe as well.

    14. LRLenny Rachitsky1:09:23

        Awesome.

18. 1:09:24 – 1:14:30

    Lessons from managing people

    1. LRLenny Rachitsky1:09:24

       So you run a massive engineering team. I don't know if you shared the numbers of engineers, but I know there are many. What are, and this is a broad question, just what are some lessons you've learned about managing people and/or managing engineers, whichever direction you want to go?

    2. DSDavid Singleton1:09:40

       Wow, big question. (laughs) There, there are so many different directions we could take that. So I'll, I'll share a few, um, observations. They don't necessarily all have a particular theme to them. I think, uh, as I've been responsible for bigger and bigger teams over time, um, one of the things that, you know, just I repeatedly learn is I, I personally will not be involved in really any of the decisions that, that matter and happen. Like there are, there are thousands of decisions that an organization of any skill is making every single day. You know, every individual engineer or PM is making hundreds of small decisions and some big ones every day that affect the kind of general trajectory. So the most important thing is to really focus on hiring the right people and that means like hiring people that you can trust with a tremendous amount of autonomy. Like if I try to get involved in lots of decisions, everything will grind to a halt. So how do you do that? It is important, obviously, to be rigorous. We've described already what our hiring process is. I really lean into getting to know folks in, uh, you know, very, very well as, as we're hiring them. By the way, something I didn't mention earlier that is, I think, important here is we pay a lot of attention at Stripe during the hiring process to references. Now this is typically later in the process, but, you know, if you put someone through an interview process, you've probably spent what, like eight hours total with them? If you talk to people that actually have worked with them before, you're probably benefiting from thousands of hours of direct experience. So like, we take this very, um, seriously and, and we, we try to ask, you know, smart questions that will help get real signal out of that. Um, I certainly find when it comes to hiring folks that have then gone on to really, you know, make a transformative impact, that references are often the time that you get the, the, the best conviction on, on the hire or not. So hire the right people and then you have to trust them. Now trust is interesting because it's actually quite challenging to trust people that you've just hired, right? Well, we've got no stake really on you yet. Um, so what I find is you do actually just have to be generous in giving trust and assuming trust by default at the beginning, but then you do need to, you know, hold people accountable enough that they are proven that they, uh, you know, they, they can handle that trust. So, you know, sometimes that means hiring someone who you think quite plausibly can end up in a very big role into a smaller role to begin with. Other times you just have the big role that you need to, to fill, in which case, um, it's important to like be quite conscious about really trusting and delegating, uh, to them with, you know, a lot of support. So I'm always working hard to delegate sometimes like a little bit more than I'm comfortable with because that's the only way to like really operate at significant scale. I think something else, uh... And sorry, this is really a smorgasbord, it's what are the differences-

Episode duration: 1:29:59