Archie Abrams: Why Shopify bans KPIs and optimizes for churn

1. 0:00 – 2:30

   Archie’s background

   1. AAArchie Abrams0:00

      (instrumental music) When you have... Teams naturally break up the world into different funnel stages or different points in the journey. It gets very seductive to just look at my part of the funnel and what's my conversion rate through that part of the funnel, right? And then the team starts to optimize for that conversion rate as their North star. But in practice, it's actually almost always easier to just make it harder to do the thing right before your step in the funnel to increase your conversion rate. Instead of, "I'm trying to convert a bunch of people, I just want more people to get activated." And then once you start thinking that way, you realize, actually, the best way to get more people to get to a step is just get more people in the door in the first place. That will always hurt your conversion rate, but it may actually give you more people on the outside.

   2. LRLenny Rachitsky0:46

      (instrumental music) Today my guest is Archie Abrams. Archie is VP of Product and Head of Growth at Shopify, where he leads an org of over 600 people across product, design, engineering, data, ops, and growth marketing. Shopify is both an incredibly unique and also an incredibly successful business. And they do things very differently. And as a result, there's a lot that we can learn from how they approach building product and driving growth. Some examples include their priorities and product roadmap are driven by 100-year vision that comes from Tobi, the CEO, and the core product teams don't have metrics or KPIs. They're essentially banned. And instead, decisions are made based on taste and intuition and building towards this long term vision. Also the growth team optimizes for churn, which is unlike any other company I've ever come across. And once you hear why, this will make a lot of sense. Also, they keep long term holdouts for every experiment they run, and they automatically look at the impact these experiments have had on the business a year later, two years later, and three years later, and then revisit these decisions down the road. And in our conversation, we dig into all this, plus how Shopify organizes their growth team, how they run experiments, how the growth team collaborates with the product team, how they measure impact, plus Archie shares a bunch of very specific and interesting examples of changes that have driven growth for the business, and so much more. This is such a fascinating conversation, and I know this will give you a lot to think about in terms of how you run and organize your own product and growth teams. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. It's the best way to avoid missing future episodes, and it helps the podcast tremendously. With that, I bring you Archie

2. 2:30 – 6:17

   Shopify’s impressive growth

   1. LRLenny Rachitsky2:30

      Abrams. (instrumental music) Archie, thank you so much for being here, and welcome to the podcast.

   2. AAArchie Abrams2:38

      Thanks, Lenny. Excited to be here.

   3. LRLenny Rachitsky2:40

      Okay, so what I want to do with our time together is to basically do kind of a, a living archeology of how Shopify grows and what you specifically have learned about growing a company like Shopify into this just juggernaut of a business that it's turned into. To give people a little bit of a sense of just like how large Shopify has gotten, so ser- maybe surprising them about the scale of this company at this point, could you share some stats that, uh, about the scale of the business at this point?

   4. AAArchie Abrams3:09

      Yeah, absolutely. So overall, we're about 10% of, of e-commerce in the United States.

   5. LRLenny Rachitsky3:14

      Wow.

   6. AAArchie Abrams3:14

      So basically, if you're not buying, you know, on Amazon or Walmart, you're probably buying on a, a Shopify powered store. And behind the scenes globally, we did about 235 billion in GMV in 2023, which is roughly the size of the economy of Finland. So we got a big, big economy and big impact happening from Shopify.

   7. LRLenny Rachitsky3:37

      Wow. Uh, I think interestingly with Shopify, it's kind of this behind the scenes tool, and so I imagine many people have no idea they're using Shopify a lot of time when they're buying stuff online. And I think some of these numbers kind of creep up on people just how large a company like Shopify has gotten.

   8. AAArchie Abrams3:53

      100%.

   9. LRLenny Rachitsky3:56

      (instrumental music) This episode is brought to you by Explo, a game changer for customer facing analytics and data reporting. Are your users craving more dashboards, reports, and analytics within your product? Are you tired of trying to build it yourself? As a product leader, you probably have these requests in your roadmap, but the struggle to prioritize them is real. Building analytics from scratch can be time-consuming, expensive, and a really challenging process. Enter Explo. Explo is a fully white labeled embedded analytics solution designed entirely with your user in mind. Getting started is easy. Explo connects to any relational database or warehouse, and with its low code functionality, you can build and style dashboards in minutes. Once you're ready, simply embed the dashboard or report into your application with a tiny code snippet. The best part? Your end users can use Explo's AI features for their own report and dashboard generation, eliminating customer data requests for your support team. Build and embed a fully white labeled analytics experience in days. Try it for free at explo.co/lenny. That's E-X-P-L-O.co/lenny. This episode is brought to you by Dovetail, the AI first customer insights hub for all teams. Dovetail has always been the go-to tool for teams who want to find insights in customer calls, user interviews, or documents. Now they've stepped it up with the release of Dovetail 3.0. That's three new products and a ton of AI features that make it faster and easier than ever before to truly get at the heart of what your customers want. You can get a real time pulse on what your customers are thinking with Dovetail's automated feedback analysis platform, Channels, or full summaries and insights from every customer interaction your team has ever had with their AI chatbot, Ask Dovetail. You can even recruit from over three million participants directly in Dovetail. All this is just the tip of the iceberg. Dovetail wants to give everyone in the organization instant access to their customers at any time. From roadmaps, discovery, to strategy sessions and more, it's never been easier to make customer centric decisions. The good news is, Lenny's listeners can try Dovetail's Pro plan along with their newest products and features free for 30 days. Just go to dovetail.com/lenny.I

3. 6:17 – 8:43

   Shopify’s unique approach to churn and retention

   1. LRLenny Rachitsky6:18

      want to start with something that I think is most unique from what I've heard, and I think there's going to be a lot of really unique approaches to how you all think about growth. Uh, one of the most interesting things I've heard is that how you think about churn and retention. To most companies the most important thing is to increase retention, reduce churn. My sense, or my understanding is you guys are kind of the opposite. You, one, don't think tons about churn, you almost optimize for churn. Talk about that. How does that work? (laughs)

   2. AAArchie Abrams6:48

      The way we think about churn is really going back to Shopify as a kind of our mission and what we want to do, which is to increase the amount of entrepreneurship on the, on the internet. And so as a business we want to make it as easy as possible to get started with, with your online store, with your business. But most businesses do, do ultimately fail, and so the way we, we look at it is can we lower the barriers to getting started and get as many people in the door trying their hand at entrepreneurship? If we do that, again many of those businesses, many of those folks will maybe on their first attempt not be as successful, but we're going to have a set of merchants who go on to become extremely big businesses, the Allbirds of the world, FIGS, et cetera. And the way the Shopify business model works is we do charge a subscription, but most of our revenue comes from payments, which is tied to directly to a merchant's success. So in a given cohort of merchants, a lot of people will start. Some of those people on their first attempt as entrepreneurship might not succeed, but the folks who do go on to be successful will kind of make that entire cohort of merchants who started something that makes Shopify as a business extremely successful. That's why we lower the barriers to get started and help folks grow, and those winners make the whole thing work.

   3. LRLenny Rachitsky8:12

      I love that. So kind of what I'm hearing is it's, it's not that you don't want people to stick around, you don't want people... It's not that you don't want people to succeed, it's that you're not optimizing every new shop for sticking around long term. It's basically make it as easy as possible for people to try it, and all you need is a few big wins for it to all work out.

   4. AAArchie Abrams8:33

      Correct. And, and that's really kind of a different insight than most SaaS companies that, you know, they get a comp- customer, they really never want that person to leave, and we, we want to lower that barriers to get started and

4. 8:43 – 11:08

   Monetization model and success metrics

   1. AAArchie Abrams8:43

      be successful.

   2. LRLenny Rachitsky8:44

      One of the main reasons companies focus so much on churn and retention is because it costs them a lot of money to drive new customers and users. I imagine there's an almost an implied it's really cheap for you all to find new customers because of maybe the brand, the word of mouth. Is that, is that true?

   3. AAArchie Abrams8:59

      I think that definitely has some dynamics. I think the bigger factor is the monetization model. For most SaaS companies, they're making from a subscription, right? They're pay- you know, 29 bucks a month is the only way that they're going to really monetize. Whereas our business works you have a, a folks who are paying a subscription, but as folks get bigger, because we're monetizing on that GMV that that merchant is producing or the revenue the merchant's producing in the form of payments and other services, it allows us to grow with the merchant. And those really successful merchants make the whole system work well.

   4. LRLenny Rachitsky9:34

      Got it. So basically your net dollar retention or net revenue retention is just absurd for the winners and it makes up for all the losers, slash not losers-

   5. AAArchie Abrams9:43

      It is.

   6. LRLenny Rachitsky9:43

      ... people that have tried to build an online-

   7. AAArchie Abrams9:44

      Yes. Tried, tried and haven't been first successful. And you can think of it the, the other parallel is, you know, an angel investing, right? Most of angel investments are not gonna work out, but the couple that do make that entire investment, uh, kind of portfolio successful.

   8. LRLenny Rachitsky10:00

      With retention, uh, not being the primary goal and the metric you guys focus on optimizing, how do you know if you're doing well? Is it some number of these winners have to come out every quarter, every year? How do you think about progress and, and achieving and, and success basically for, for growth?

   9. AAArchie Abrams10:18

      This way is thinking about a cohort of users we, we, we acquire in a given time period, say a quarter. And then over the next year, two years, three years, four years, five years, how much GMV have those merchants produced in total? Not on a per merchant basis, that's... But, uh, in total did that cohort generate GMV? And if they generate GMV, that will translate into revenue and gross profit, and all of those things that we can then use to reinvest in growing the business. So it's really looking at the total value, but on that GMV basis. And GMV is a power law based metric, and so it's really that power law that drives the success of each cohort. Again, going back to investing, same thing there. Each vintage from a fund, how much did that return kind of as a fund? And it's really driven by the few, the few really, uh, successful

5. 11:08 – 23:00

   Long-term experimentation and metrics

   1. AAArchie Abrams11:08

      outliers.

   2. LRLenny Rachitsky11:09

      So this begs the question, that sounds like a very long feedback loop, and I don't know what I'd do with that information if five years from now, "Oh, okay, that was a really good idea we did five years ago."

   3. AAArchie Abrams11:18

      Correct.

   4. LRLenny Rachitsky11:19

      Comment on that 'cause it touches on something you said about how metrics aren't actually a driver of how y'all think at Shopify. So take that wherever you want to go.

   5. AAArchie Abrams11:26

      Yeah. So it's, it's, it's interesting on I think with Shopify we got very purposely set up different parts of the org to think on very different time horizons and with very different ways of thinking about how to build product and the like. Very different than a lot of companies that typically have maybe one kind of unified, there's one North Star that the entire company is rallying around. And so there's three major product groups at Shopify. There's core product, which is basically building the 100 year, the right things for commerce 100 years from now. There is merchant services, which is building things like payments, shipping, kind of the tools that entrepreneurs need to be successful with a more kind of shorter or medium term horizon. And then growth is really thinking about kind of that end-to-end customer journey. How can we bring folks on and make sure they're successful?And then from a metric standpoint, we do have some, obviously some leading indicators in growth that we're looking at on a given experiment or, or what have you. But the key and what we try to instrument in our experimentation is the ability to really look at long-term effects of, of experiments. So we have constantly will re-look at an experiment a year later, see that the way the GMV curve or the distribution was different than we might have originally thought, and that'll actually change what we do with, from that previous experiment. And so there's a lot of long-term monitoring of experiments over these very long time horizons to both inform what those input metrics are, but more importantly hold ourselves accountable to, did we actually move what we cared about, which is that long-term GMV, in the right way?

   6. LRLenny Rachitsky13:12

      Wow. Okay, I wanna spend more time here. So you, the way you're describing it is the way the business operates is you think, uh, "What is our 100-year plan? How do we think, where does this need to be in 100 years?" And with that, it allows you to run these long holdout, kind of holdout experiments to see, is something we're doing impacting the business broadly? And, and because you think so long term, you can take a year or two or three to see if there's an impact and then make adjustments versus, you know, him having to drive a certain metric every quarter or every year.

   7. AAArchie Abrams13:47

      Correct. And I mean, on, on growth, we're definitely in the, you know, we wanna drive metrics on a short-term basis and we can do that obviously.

   8. LRLenny Rachitsky13:53

      Yeah.

   9. AAArchie Abrams13:53

      Um, but we- we- we have the- the luxury and just the way kind of Tobi kind of thinks about the world and the way we operate to really think about these long-term effects and make sure that we're holding ourselves accountable with these long-term holdouts, and then constantly refining the input metrics that we're using and getting a lot smarter, uh, about that. But because we take that long horizon, it allows us to be better in the short term and- and just get a lot smarter. And a lot of counterintuitive things. And I would encourage everyone, if you can, look at some of the experiments that you thought were your biggest winners. Look at the downstream metrics for a year or two years on that experiment, and I bet you'd be surprised how many times the metric is different than what you thought it would be after a year.

   10. LRLenny Rachitsky14:36

       Because, uh, where people just make a call at a certain point in time, here's the lift-

   11. AAArchie Abrams14:39

       Done.

   12. LRLenny Rachitsky14:39

       ... and it's-

   13. AAArchie Abrams14:40

       Here's the lift.

   14. LRLenny Rachitsky14:40

       ... from your experience. I love this, 'cause very few people have experiences running a ho- a long-term experiment. And so this is a really interesting insight that you're sharing that, I guess, uh, how often do you find this to be true in your long-term holdouts where things end up being very different down- down- downstream?

   15. AAArchie Abrams14:59

       Uh, I think there's, there's probably two things that have been very common. And I would say in, in quite a few cases. You get a short, you get a lift on the, uh, a metric up front, a more short term metric, number of people who become a paying shop or number of people who make their first sale in Shopify. And then you look a year later, and there's actually no incremental lift on GMV from that cohort. And so I think it like actually trains a lot of us in, in growth that are looking at these short term metrics. Like, a lot of the time, it is actually more pull forward effect than you fully, fully realize, or an incremental user that's just really not worth that much. So that's one. And then two, the, so it's just effect size goes away. There are cases where the experiment has flipped the other way, and then there are cases, and these are the most interesting ones, where you realize that you uncovered a pocket of merchants that are actually extremely valuable entrepreneurs who go on to be successful that you missed in your, your kind of normal short term, uh, measurement techniques. Um, and so kind of all across the board we see that, but actually the most common is, there actually isn't a long term lift from a lot of things that you might think in the short term are.

   16. LRLenny Rachitsky16:16

       Is there an example in that second bucket of what you mean when you say there's like a pocket of valuable merchants?

   17. AAArchie Abrams16:22

       Yeah, I think a lot of this has to do with, um, we call monetary friction, right? So one, one of the, the hardest things to do with a business is when you're getting started is, you might not have any revenue coming in, right? And, and you're kind of bootstrapping, which, in Shopify's case might be 39 bucks a month, but it's still, it's a real expense. And so typically when you can lower the barriers to monetary friction in some form, that could be all sorts of monetary friction early, the common belief is that will usually get lower quality folks coming in the door. Because usually discounts are associated with lower quality. If you think about in a business case, if I give you a little less, a little monetary boost and reduce that monetary friction, I can actually causally change your ability to become successful, 'cause I've given you a little bit more time to try that idea a little bit longer. I've given you that opportunity to move your business over to Shopify. And so often in those types of experiments, you see that you've basically unlocked a class of people who might have given up without that monetary, reducing that monetary friction.

   18. LRLenny Rachitsky17:35

       Interesting. And giving them time to actually make it work.

   19. AAArchie Abrams17:39

       To make it work.

   20. LRLenny Rachitsky17:39

       Okay. So, uh, just roughly, do you, do you have a sense of how often you find no effect after a year that you saw early impact just to ballpark it?

   21. AAArchie Abrams17:49

       Yeah, it's probably in the, it's- it's in the 30 to 40% range.

   22. LRLenny Rachitsky17:52

       Okay. Like I think you're tearing the heart out of so many growth people right now, and nobody wants to hear this that works on growth, where you're saying potentially a third (laughs) of the experiments they're running today that are showing lift probably don't have that same, don't have any impact down the road.

   23. AAArchie Abrams18:09

       Yes. Unfortunately, I think that's-

   24. LRLenny Rachitsky18:11

       Brutal.

   25. AAArchie Abrams18:12

       ... probably more common than we like to, we like to believe.

   26. LRLenny Rachitsky18:15

       Yes. And that's gonna, nobody wants to hear this, uh, except-

   27. AAArchie Abrams18:19

       Yeah.

   28. LRLenny Rachitsky18:19

       ... ex- except people that, you know, you should want to hear this because if you want to build a business that grows and-

   29. AAArchie Abrams18:24

       Correct.

   30. LRLenny Rachitsky18:25

       ... continues to grow, it's better to know-
6. 23:00 – 26:42

   Examples of big wins that Archie’s team has shipped

   1. LRLenny Rachitsky23:00

      on and on. (laughs) While we're in the topic of just experiments and what you've done, I'm curious if there's just any examples of big wins that your team has shipped that might inspire people as they're thinking about launching experiments? I know there's probably some trade secret stuff you don't want competitors to know, and I know this is particular to Shopify and the platform eCommerce, but I guess is there anything that would be worth sharing of like, oh, here's a huge win that maybe we didn't expect?

   2. AAArchie Abrams23:25

      Going back, there's, there's always a lot of value in thinking through, um, kind of monetary friction, as I mentioned. Like that's always gonna be something to, to explore, trial dynamics, different types of incentives. All of those things are, are very kind of impactful. I would say on things that are maybe more like practical and, and for everyone, there's an enormous amount, and we do see these with long term effects, but just the nuts and bolts of sign-up, collecting the right information, and you usually want to collect more information than most people think you do in your sign-up flow, if you can then leverage that to personalize the guidance, and this is for a SASS product, the guidance that someone can get when they onboard into Shopify. So whether you're coming on ... Shopify is a very diverse product, in-person selling, online selling, different channels. There's the nuts and bolts of get more information from folks, build trust in there, give them right amount of guidance when they come on in a personalized way. And that may sound like, okay, that's kind of obvious, but the amount of impact by just nailing...... those flows has never ceased to, to amaze me, and setting up that person for long term success. Um, so kind of monetary friction, and then just really good onboarding, personalization. A well of, of opportunities there.

   3. LRLenny Rachitsky24:49

      I love that onboarding comes up every time I ask anyone where they've seen ongoing success and opportunities, particularly in actually surprisingly driving retention. Like, it's interesting-

   4. AAArchie Abrams25:00

      Yeah.

   5. LRLenny Rachitsky25:00

      ... that that's not what you look at, but turns out that's one of the biggest levers for increasing retention. Uh, interesting that even for a company that doesn't look at retention, that's a big opportunity.

   6. AAArchie Abrams25:07

      Yes. And it's really because it's setting people up for, in Shopify's case, I think the big thing about, um, all of our metrics is what we get very nervous about is the easiest way to increase retention is always to constrict the funnel stage one above the retention metric you're trying to optimize for. The simplest way to increase my signup to activated thing is just make it harder to sign up, right? Like, nuts and bolts, that will always happen. And so when you have teams on that kind of like local conversion rates, you kind of get all these weird team incentives, because they're optimizing to basically implicitly make it harder to do the step before them. And because we focus on that long term GMV number of merchants who are successful, orienting every team to think about the total number of people, not the rate, but the total number of people who got to the end of their kind of part of the journey is a very powerful way to incentivize people to do the, the right thing in terms of getting people set up versus do the, "I'm going to constrict the top, the funnel step right before me to make my local conversion rate look better," which is the bane of my existence, but something I, I see a lot of teams like implicitly or, or explicitly do, uh, when they, when they get too focused on rates as a way to think about the world.

   7. LRLenny Rachitsky26:32

      Incentives, what a power, what a lever.

   8. AAArchie Abrams26:33

      Incentives, what a power, what a lever.

   9. LRLenny Rachitsky26:35

      (laughs) I definitely want to chat a little bit more about metrics. I know you have a really interesting take that's kind of built on what you're just talking about. But first of all,

7. 26:42 – 27:14

   Monetary friction

   1. LRLenny Rachitsky26:42

      you mentioned this term monetary friction as one of the levers that you've seen success with. Can you just describe what that actually means?

   2. AAArchie Abrams26:48

      Totally. So things like trial, trial dynamics, trial length, trial amount, it means, you know, incentives. So what is in your product? What do people, uh, value and need in order to be successful? So in Shopify's case, that might be like app score credits or things like that. Um, but those are the two forms of monetary friction we, we talk about, and then of course, actual price point, but that's what the kind of the larger

8. 27:14 – 29:47

   Metrics

   1. AAArchie Abrams27:14

      bucket of monetary friction is.

   2. LRLenny Rachitsky27:16

      So let's follow this thread of metrics. You're big on absolute numbers. You've... And you've been talking about this already versus like percentages and ratios. Talk about that and how you encourage your teams to think about metrics.

   3. AAArchie Abrams27:29

      Yeah, I think one of the things that I think happens particularly in a large, and Shopify is about 600 folks. Like, when you have teams naturally break up the world into different funnel stages or different points in the journey, it gets very seductive to just look at my part of the funnel and what's my conversion rate through that part of the funnel, right? And then the team starts to optimize for that conversion rate as their North Star over a longer time period. "I'm going to try to move my conversion rate from 10 to 12%," or, or what have you. But in practice, you, you know, talked about like, it's actually almost always easier to just make it harder to do the thing right before your step in the funnel to increase your conversion rate. If I make it harder to sign up, it's going to be very easy to increase signup to activated rate because I just have fewer people and the people who made it through are higher intent. And so I see teams get really stuck when they, like, are trying to optimize conversion rate, but they just make it harder to do the previous thing, versus everyone is thinking about absolute number of people who made it through their, quote, "stage of the funnel." So instead of, "I'm trying to convert a bunch of people, uh, a conversion rate, I just want more people to get activated." And then once you start thinking that way, you realize actually the best way to get more people to get to a step, sometimes and often, is just get more people in the door in the first place. So make it easier to sign up or reduce friction. It's the opposite, right? And so, like, because that will, that will always hurt your conversion rate, but it may actually give you more people on the outside, and a lot of teams get very nervous, their retention rate went down, their LTV went down. "Oh my goodness, this is going to affect our ability to pay." No, your CAC also went down by probably more. And so now you have the ability to likely spend more and you have more people through the door, um, getting to each point in the activation or the, the merchant journeys.

   4. LRLenny Rachitsky29:33

      What I'm hearing is essentially teams are goaled not on increase, lift, lift this conversion step by some percentage, it's drive some incremental absolute number of new, uh, merchants potentially.

   5. AAArchie Abrams29:45

      Merchants, exactly.

   6. LRLenny Rachitsky29:46

      Yeah. Um,

9. 29:47 – 33:03

   Shopify’s growth team structure

   1. LRLenny Rachitsky29:47

      this is a good segue to I want to hear how you structure your growth team at Shopify. Like, essentially, what's the raw structure? What are the different teams and what do they focus on? And then what are like the functions within each teams?

   2. AAArchie Abrams29:59

      We have two big, uh, groups within, within growth. So one is what we call growth R&D. So this might be what you traditionally consider like product design, engineering, data, your traditional product teams. Then we have growth marketing, which in Shopify's case is paid acquisition, kind of media buying, affiliate marketing, email content, and SEO. So that's kind of like growth R&D, growth marketing. Within growth R&D, three, uh, three pillars. One is what we call growth products. And so this is basically everything from kind of landing pages, signup, onboarding, monetization, so trial incentives, the like, all the way through to...... home- our, what we call our home feed, our engagement to basically get more merchants, again, not only to retain, but to gi- keep giving entrepreneurship a try to become bigger and bigger businesses. So that's growth product, kind of the full life cycle there. Second is what we call our enable pillar. And this t- pillar's building tools for both growth and the rest of Shopify. So things like experimentation platform, our communication platform. We have our business intelligence tooling that, that powers a lot of what we're doing. Our MarTech work to support our growth marketing team. And then our third bucket, which is maybe a little different from most growth teams is actually our customer support group that's within growth, because we want to think about customer support as part of this merchant journey of coming on, giving entrepreneurship a try all the way through to here's the support I need as I'm becoming a, you know, multi-billion dollar business on Shopify. So those are the, like the three big growth product buckets. And then within growth marketing, it's a more traditional channel set up, paid, all the different channels, online, offline, SEO, email and affiliates.

   3. LRLenny Rachitsky31:49

      Super cool. Okay. So within growth R&D, I just took notes. I'm gonna summarize what you just shared-

   4. AAArchie Abrams31:54

      Yeah.

   5. LRLenny Rachitsky31:54

      ... which is awesome. S- so there's kind of three big buckets. One is growth product, which essentially is like onboarding. It feels like it's like the top of funnel, get people, uh, in. Well, okay, so growth marketing feels like that's super top funnel, bring people to the site.

   6. AAArchie Abrams32:07

      That's super top funnel.

   7. LRLenny Rachitsky32:08

      Yeah.

   8. AAArchie Abrams32:08

      Correct.

   9. LRLenny Rachitsky32:08

      Okay, got it. So growth marketing drive people to Shopify.com.

   10. AAArchie Abrams32:13

       Correct.

   11. LRLenny Rachitsky32:13

       Then growth within R&D team growth product takes that user and tries to get them to activate it. Enable helps... It feels like that's like internal tooling and ways to make the teams-

   12. AAArchie Abrams32:25

       All-

   13. LRLenny Rachitsky32:25

       ... internally more efficient.

   14. AAArchie Abrams32:26

       Correct.

   15. LRLenny Rachitsky32:27

       And...

   16. AAArchie Abrams32:27

       Both growth and outside growth.

   17. LRLenny Rachitsky32:29

       Awesome. Okay. And then the, uh, customer support team. That's really interesting. So there's a customer support product team that helps new sh- merchants be s-successful. And does that include like actual customer support agents? Is that like within that team?

   18. AAArchie Abrams32:46

       That's not. We build the tooling to make those-

   19. LRLenny Rachitsky32:48

       Got it. Okay, cool.

   20. AAArchie Abrams32:48

       ... support advisors kind of superheroes, and then on the- the help center, all of our AI stuff, uh, to make kind of a, a great customer experience for people who are-

   21. LRLenny Rachitsky32:58

       Got it.

   22. AAArchie Abrams32:58

       ... just engaging in a self-serve. Um, so it's the tooling-

   23. LRLenny Rachitsky33:01

       Got it.

   24. AAArchie Abrams33:01

       ... and the experience for, for merchants.

10. 33:03 – 37:10

    Goal setting and forecasting

    1. AAArchie Abrams33:03

    2. LRLenny Rachitsky33:03

       Okay. So with these teams, is there anything you can share about just like how you think about metrics/goals for these different buckets? We don't need to get too deeply, but just does everyone basically have like an absolute new merchants goal or is it a little different?

    3. AAArchie Abrams33:18

       So yeah, so, you know, at the, the highest level, we think about that, that total cohort value, right? We bring in a set of merchants in a given year, how much GMV, how much m- is that set of merchants worth, um, over the next three, four years to Shopify, right? And that's, that's the most important thing that we wanna focus on. And then that of course the, from efficiency standpoint that of course meeting our, our payback guard rails and, and all of that. So that's kind of like the, the macro growth perspective. Cohort value over kind of cost and, and payback. So that, that's the macro point of view. And then within growth marketing, you know, each channel operates with certain guard rails around their LTV to CAC. Same thing for content and SEO, that operates with kind of a guardrail model for each piece of content. How much is that gonna come back and, and, and down the line? For growth products, it's a, also a combination of total GP or incremental, uh, cohort value that's produced from those, those teams, right? So everything is basically gonna be measured on from, from an experiment, ideally measured over a very long time period. What was the incremental cohort value lift that this, this generated and, and that, that's how we think about it, kind of measure the impact of each of those, those sub-teams along the way. Each of those have a specific part of the funnel they play with, but because they're measured on absolutes and they really think about that absolute value, we don't get caught into like, did your conversion rate over the course of this year go up or down? It's kind of irrelevant. What was the sum of the impact over a long period on that total cohort value that we're trying to produce from, for four merchants?

    4. LRLenny Rachitsky35:01

       And the way you come up with this goal, I imagine, is you have a forecast of where things would go organically and then like here's the lift we want to see from the work this team does this quarter this year.

    5. AAArchie Abrams35:12

       Correct. Um, and, and then we're gonna measure against for each experiment, did it actually get to where we-

    6. LRLenny Rachitsky35:19

       Yeah.

    7. AAArchie Abrams35:20

       ... we expect that lift to be?

    8. LRLenny Rachitsky35:21

       And those experiments, again, are those all long-term hold out experiments where you look, wait a year or some you-

    9. AAArchie Abrams35:27

       We call.

    10. LRLenny Rachitsky35:28

        You get-

    11. AAArchie Abrams35:29

        We call, we call the experiment after three weeks, but in all cases-

    12. LRLenny Rachitsky35:32

        Got it. You still watch them.

    13. AAArchie Abrams35:32

        ... the group is held. We watch them.

    14. LRLenny Rachitsky35:34

        I see.

    15. AAArchie Abrams35:34

        And that's where that ping comes back.

    16. LRLenny Rachitsky35:36

        I see.

    17. AAArchie Abrams35:36

        Every experiment is watched, and that-

    18. LRLenny Rachitsky35:37

        Got it.

    19. AAArchie Abrams35:37

        ... ping comes back 3, 6 months, 12 months to re-look at, was this actually successful?

    20. LRLenny Rachitsky35:42

        Okay, cool.

    21. AAArchie Abrams35:44

        Um, so that creates the loop of shipping value quickly, but making sure we're holding ourselves accountable to did this actually produce results over a long period or did it actually just have this neutral effect? It's like, oh, then we can learn from that and get better.

    22. LRLenny Rachitsky36:01

        This episode is brought to you by Dovetail, the AI first customer insights hub for all teams. Dovetail has always been the go-to tool for teams who want to find insights in customer calls, user interviews or documents. Now they've stepped it up with the release of Dovetail 3.0. That's three new products and a ton of AI features that make it faster and easier than ever before to truly get at the heart of what your customers want. You can get a real time pulse on what your customers are thinking with Dovetail's automated feedback analysis platform, Channels, or pull summaries and insights from every customer interaction your team has ever had with their AI chatbot, Ask Dovetail. You can even recruit from over three million participants directly in Dovetail. All this is just the tip of the iceberg.Dovetail wants to give everyone in their organization instant access to their customers at any time. From roadmaps, discovery, to strategy sessions and more. It's never been easier to make customer-centric decisions. The good news is, Lenny's listeners can try Dovetail's Pro Plan along with their newest products and features free for 30 days. Just go to dovetail.com/lenny.

11. 37:10 – 41:36

    Examples of long-term results within Shopify

    1. LRLenny Rachitsky37:10

       So maybe just to dig into this again 'cause it's so interesting, basically product team ship stuff, they run an experiment, they see impact, say it's 5% lift on something. Ha ha, huzzah, you did it, great work, performance review, up, your exceeds, you're doing great, this team's killing it. And then a year later, you realize, oh, that didn't, that didn't last. How often do you find, like, a team that is shipping wins looks back and ends up seeing, like, "Oh, that wasn't actually as successful as it..." I know you said, like, maybe it's like a third of the time.

    2. AAArchie Abrams37:38

       Two-third, yeah.

    3. LRLenny Rachitsky37:39

       Yeah. Okay. So it's still like roughly... Yeah.

    4. AAArchie Abrams37:43

       And, and it's great learning, and that's the way we take it is, like, wow, okay, now we, we really uncovered something and it's, like, such a successful discovery.

    5. LRLenny Rachitsky37:49

       Mm-hmm.

    6. AAArchie Abrams37:49

       Wow, okay, we thought this thing, but now we, now we learned it actually wasn't as true as we thought. Cool. What, what can we take from that and be smarter, uh, next time so we don't just double down on the wrong, the wrong things?

    7. LRLenny Rachitsky38:02

       That's so interesting. And aga- and you mentioned most of the reason this is the case when something doesn't show lift down the road is it's pulling forward success that would have been seen later on its own if you had not even shipped this thing.

    8. AAArchie Abrams38:14

       Correct.

    9. LRLenny Rachitsky38:14

       Awesome. Is there an example by any chance that comes to mind of something like that that's just like, wow, that was a big win and then like, oh, I see, we just pulled forward some, some revenue from the future?

    10. AAArchie Abrams38:24

        Yeah, so I think one, one good example is, is something around payment failure notification. So one of the things that a lot of teams have or, or see is we call it Dunning effects, where somebody might have a payment, uh, not go through, a credit card that doesn't go through. So we, we did a bunch of experimentation around, hey, how can we alert people that their credit card is failed, their payment, uh, attempt failed? And that's a typical kind of growth win, usually produces a lot of short term impact, and that's what we saw here. We were doing much better alerting, reminding people, sending them a million emails about it. Cool, we got some pretty major lift. You look back six, 12 months, there was really no long term lift. And, and why is that? 'Cause there's really a s- a s- a little bit of a selection bias there. The people who were letting that, that payment fail probably weren't actually that dedicated to this entrepreneurship craft. They may have updated their credit card, but they, they still really weren't in it. And so that was a good example of, you know, a bunch of this stuff around kind of payments, even quote preventing churn when you look, like, at 6, 12, 18 months, on a GMV metric, not a lot of lift, um, over that long term horizon.

    11. LRLenny Rachitsky39:37

        I love this example. Like, I could see so many people having run experiments like this and like, "Oh, we found such a huge win. Woo hoo! This team's killing it, what a great idea. Of course this makes sense. Duh." (laughs) And then turns out-

    12. AAArchie Abrams39:51

        It-

    13. LRLenny Rachitsky39:52

        ... it's nothing long term.

    14. AAArchie Abrams39:53

        Yeah.

    15. LRLenny Rachitsky39:54

        Um, yeah.

    16. AAArchie Abrams39:54

        Which is great because we were-

    17. LRLenny Rachitsky39:55

        I love that. Yeah.

    18. AAArchie Abrams39:55

        'Cause we were spending, you know, we were gonna spend a lot of time kind of, okay, what else can we do here? Spend... It's like, no, actually, bigger fish to fry in some-

    19. LRLenny Rachitsky40:04

        Mm-hmm.

    20. AAArchie Abrams40:04

        ... in a lot of other areas. So just it helps the team just feel really good that their, their work is really the things that, um, that, uh, that were good. Uh, you know, another one that, that went the other way, um, which was really interesting was, in our online store, and this might be if you use Shopify, we have sections and blocks that come, come preconfigured. Um, and so we tested, okay, if we, if we give you a preconfigured block of like, you should have an image up top, then a text banner, and then a collage with your pho- um, products, that should help folks, um, understand what to do when they're building their online store. It actually had no lift in people converting to, uh, a paying merchant. However, when we looked longer term on that six months later, it had a pretty massive impact on the number of people who were selling and producing GMV. And why is that? Because it didn't likely really influence anyone to, like, buy Shopify or pay for Shopify, but the people who, who used it created better stores that were higher converting.

    21. LRLenny Rachitsky41:10

        Wow.

    22. AAArchie Abrams41:11

        And so they got early sales but they actually converted one of their, their visitors and they got momentum and they stuck with entrepreneurship a little bit longer. And we saw that in that opposite way. And so this is an example of that neutral, so we tend to ship neutral. That's like it could be positive, and so let's, like, let it go if we have good intuition about it and it will turn. So we've seen a bunch of these things go in very different directions.

12. 41:36 – 42:05

    Shipping neutral experiments

    1. AAArchie Abrams41:36

    2. LRLenny Rachitsky41:36

       This is so fascinating. I didn't realize that you ship neutral experiments. That's an interesting insight. So it's like, if you feel good about it and it's neutral, you ship it.

    3. AAArchie Abrams41:45

       In our, in our culture of kind of like aim, kind of aim heavy, if the intuition is right that this probably is helping merchants, why do we start with that the original control is better if it's neutral? Let's start with, like, what would we have shipped if we were a blank slate? And if it's neutral, actually neither is better. So let's just pick the one we feel better about and ship that.

    4. LRLenny Rachitsky42:02

       Mm-hmm. Makes so much sense. Oh man,

13. 42:05 – 48:04

    Building a hundred-year company

    1. LRLenny Rachitsky42:05

       okay. So let's talk about this a little bit more, so this aim heavy concept, this idea of thinking 100 years out. Can you just share more about just that insight and that philosophy? I know it sounds like it comes from Tobi of how he likes to think about the business.

    2. AAArchie Abrams42:18

       Totally. This is all come, you know, is all Tobi, of really making sure Shopify is so oriented around we are here to build a 100 year company, and so the decisions we're gonna make are really oriented towards the long term success of, of merchants, of Shopify. You know, embedded in all of our principles are make the best product in the world, make money to do more of one, never reverse principles two and three. In every kind of executive meeting, every town hall-... that slide comes up. It's like, you've been at Shopify, you've probably seen that slide 10,000 times. But it's an important reminder. Like, job is to build the best product for merchants over the long period of time, and then all kind of the metrics and kind of the make money part of it, secondary to that. That's all we care about is that, that long term piece. It, it ties a little bit to that, that original conversation about kind of entrepreneurs and being the core of like why we just want more people to start businesses and go. It's very, um, seductive, I think, in kind of most companies, including in Shopify because we wanna... We can support large enterprise businesses today, right? Big brands who wanna get off a outdated solution and come over to Shopify. It's very easy to just say, "Oh, that's very concrete." As an existing business, we wanna have them come join Shopify. And in the short term, it feels really good because it brings a lot of revenue right away. But if you're thinking about the long term, 100 years from now, guess what? All of the big brands of today will be out of business, in... Many of them will be out of business in 30, 40, 50 years. The real success of Shopify is getting every business to start with us and go. But, but making that type of investment and being so focused on that entrepreneur segment and making it easier is how we build kind of a very, very long term oriented company. So just even how we do capital investment, how we do product decision-making comes back to, hey, we can't chase kind of the short term, um, even more concrete things.

    3. LRLenny Rachitsky44:24

       Is there an example that comes to mind where you did that, where something short term looked like, "Oh, we should definitely do this, but we're thinking long term. We're thinking 100 years out, so we're gonna approach it this way"?

    4. AAArchie Abrams44:35

       It's kind of very much just imbued in, in the culture. It's almost even, almost everything kind of feels that way. And I'll give like, practically speaking, every six weeks, we... All the kind of R&D group leads, we get together and we sit with Tobi and, and each other and review every single project across the company. Every six weeks, every single R&D pull up the dashboard and look at it. And in that conversation, so much of the conversation is about both the technical how. How are we building this in a way that allows for Shopify to have optionality and the technical decisions that we are making? And I think for Tobi, one of the things I've, I've learned and it's so... Is that the how, the technical architecture determines strategy in a technology company even more than the kind of what and who we're building for. And if you build the right, kind of technical how and set yourself up to have a platform that can be adaptable, flexible, that is incredibly valuable over the long term. It means we will sometimes take longer to ship a feature. It means we will not chase kind of certain deals or what have you. But we're gonna kind of make that an investment. And it comes through in all of our reviews and just how we've got to do our work together.

    5. LRLenny Rachitsky45:53

       Wow. That is really unique. I have not heard of that, where the how... Usually, it's the opposite.

    6. AAArchie Abrams45:59

       Yes.

    7. LRLenny Rachitsky45:59

       Let's not worry about how we're gonna build this thing. It's why are we building this thing? And then when are we building it? And not just like the architecture is the k-... Is the most important thing.

    8. AAArchie Abrams46:09

       Yeah. No, I mean, it's like in the last one, we had a... It was great. We had a 30-minute discussion about we are... How to build CSV importers for people coming over from different platforms. And it was all about we're using open source library, doing it internally or doing it in the core codebase. Are we building a separate first-party app to do it? It was incredible detail. This is what's amazing about Tobi. The technical detail of how we're gonna do this was incredibly important to get right, to kind of set up this type of infrastructure. In most companies, it'd be, "Okay, what? You're gonna make it easier for people to migrate their data over? Cool."

    9. LRLenny Rachitsky46:42

       (laughs) Right.

    10. AAArchie Abrams46:43

        Team go figure out the how. And if team does figure out the how, then we work on it with kind of Tobi in the details because the how is so important to how we build for the future.

    11. LRLenny Rachitsky46:54

        That's fascinating. And usually, it's how do we do this as quick as possible?

    12. AAArchie Abrams46:57

        Yes.

    13. LRLenny Rachitsky46:57

        Because CVS importing is not a core differentiator. Let's just build something good enough. We'll ship it. We'll move on.

    14. AAArchie Abrams47:02

        Totally.

    15. LRLenny Rachitsky47:02

        This is the opposite.

    16. AAArchie Abrams47:04

        Yes.

    17. LRLenny Rachitsky47:04

        That is fascinating. What's also really interesting about this is I think about Brian Chesky at Airbnb, where I worked for a while, and his... So one, he also had this idea of the 100-year vision and thinking for the future way out in 100 years. But interestingly, since he's a designer, he had a very different focus. So Tobi, he was an engineer. He still codes from what I can see on Twitter. He's still building things.

    18. AAArchie Abrams47:24

        Absolutely.

    19. LRLenny Rachitsky47:26

        Uh, so I could see why his brain goes there and why he's really strong in the how. Brian, on the other hand, is very focused on experience and making sure the design is amazing and, uh, the app is exactly what he wants it to feel like. You know, it's very like experience oriented. So it's interesting that founders, these founders lean into the thing that they're strong at and understand deeply. And that ideally connects with the way this business specifically wins and grows. And it makes sense. A platform, I could see why engineering would be so essential to get right. Travel, hospitality, consumer app, I can see why design is so important.

    20. AAArchie Abrams48:03

        100%.

14. 48:04 – 51:30

    Why Shopify doesn’t use KPIs

    1. AAArchie Abrams48:04

    2. LRLenny Rachitsky48:04

       Fascinating. S- One more tidbit that I've heard about how y'all think about this is, is metrics. And you, you mentioned before we started recording that a lot of the company doesn't actually have metrics that drive what they build, especially within the core business, which I think would surprise a lot of people. Most people are like, "Every team needs a metric and a KPI and this is how we measure progress and this is how we know if they're doing well." Talk about just how that works, how companies... How most of the company doesn't have a metric ??????

    3. AAArchie Abrams48:31

       Yeah. It's funny, we, we rant against KPIs are basically banned as a... And OKRs are banned and, and all that. So you know, we... And so certainly like in, you know, in growth, we have the metrics but they, they take a different form. And then in core, it, it truly is do we have conviction...... that this is the right technical foundation to build the future of commerce. And that is built through certainly looking at data. So it's not that teams are not looking at data and using it as a piece of their puzzle, but it's not the overriding. And when we sh- go to ship a feature in core, it's not like a team is held accountable for this metric over this six months. It's much more did we ship the right thing? And we're gonna kind of get at that through a variety of lenses. Uh, could be, some of that could be data, qualitative, just our own kind of product sense, uh, of what's good or not. And so that, you know, I think the, the, the upside of that is I think we tend to ship things in core and that are incredibly, um, forward facing and maybe we take more risks. I think the, to acknowledge some of the, the downside of it though is sometimes conversations get extremely subjective, uh, about what is the right thing to do. And so that, that requires kind of a, um, the right way of, of having kind of good discussions, kind of openness from all leaders and from teams to debate those things. But it does result in some squishiness which, again, has its pros and cons, but kind of taste is kind of what drives a lot of what we're shipping in core.

    4. LRLenny Rachitsky50:06

       Yeah. I'm glad you touched on that. I was gonna say, okay, everyone would love this idea. Just build things that we think are awesome, it's gonna be great. But then you build a whole org with teams and people building stuff. How does one know if they're building things that are good and helping versus not? And you're pointing out there are pros and cons to that. The pros is we're not optimizing for some short term wins and driving some poor metric. The con is you might ship stuff that... Like, there's a lot of subjectivity and people may not agree and it's a lot of kind of squishy stuff.

    5. AAArchie Abrams50:39

       Yeah, totally. Uh, Glen, who's, you know, heads up core product, I mean, one of the things that's so impressive about Glen and kind of that core team is they go incredibly deep into every single release that are shipped. And so you have, you have a central kind of eye on the quality and how it all fits together. And so that, I think, helps make sure there's a consistent kind of bar for taste that, that Glenn, a bunch of folks, Tobi obviously, that kind of can, can enforce that. So it creates obje- it's subjective, but it's objective in the sense that it's kind of a small number of people who really hold what that, that bar is and needs to be. I think if it's just subjective, let's just ship what we want without kind of a couple people really holding that, that quality and that taste bar, that's where things go really

15. 51:30 – 54:30

    Shopify’s “Get shit done” framework

    1. AAArchie Abrams51:30

       sideways.

    2. LRLenny Rachitsky51:31

       Awesome. That's exactly what I was gonna ask is who, who's the ultimate decider of taste and, and, uh, what is good? And so it sounds like basically Tobi above and then he's kind of deputized Glen and relies on him to make a lot of these final calls.

    3. AAArchie Abrams51:44

       Yes.

    4. LRLenny Rachitsky51:45

       And that... Yeah, okay. And then I imagine Glen has some folks that he kind of deputizes to make smaller decisions along the way. Or not.

    5. AAArchie Abrams51:53

       Yes.

    6. LRLenny Rachitsky51:53

       Or he's very involved in everything.

    7. AAArchie Abrams51:54

       I, I mean, yeah. And I think, I think this is the fun thing about Shopify. Literally, like, we have a, our own internal project management system that's been kind of crafted just for Shopify. And every ship-

    8. LRLenny Rachitsky52:03

       What is that called, by the way? It's got like a cool name, right?

    9. AAArchie Abrams52:06

       GSD.

    10. LRLenny Rachitsky52:07

        GSD. Yeah.

    11. AAArchie Abrams52:08

        Get shit done.

    12. LRLenny Rachitsky52:09

        Stands for something. Yeah, okay, sure.

    13. AAArchie Abrams52:10

        Get shit done. Yeah, yeah, yeah.

    14. LRLenny Rachitsky52:11

        That's what I wrote.

    15. AAArchie Abrams52:12

        So get shit done. And every project, so you got a core project, you got merchant service, you got growth project, and the expectation is that the group leads, every single project that goes out has a few minute video with Figmas and everything. And everything that's shipped needs to be okay-toed. So approved by the group lead. There's nothing that can ship without that okay-to approval. And that okay-to approval has to be Glen, Karl, myself, different groups. And so that is kind of how the, everything is reviewed. Now, of course, there's great, amazing teams that do amazing work. Um, but it is kind of that, that's how the, the system, system works.

    16. LRLenny Rachitsky52:50

        And okay-to specifically means someone above re- reviews it or all you, this whole team, everyone looks at it?

    17. AAArchie Abrams52:56

        No, just w- so Glen reviews the core stuff.

    18. LRLenny Rachitsky52:58

        Got it. Just Glen is the okay-to.

    19. AAArchie Abrams52:58

        Karl reviews the MS okay-to. Um, so it's, it's interesting.

    20. LRLenny Rachitsky53:05

        It's basically Glen as founder mode and not as a founder where he's involved in all the details.

    21. AAArchie Abrams53:10

        Yes. Yes.

    22. LRLenny Rachitsky53:10

        Has final say. So this is a really cool example of founder mode, but not as a founder and-

    23. AAArchie Abrams53:15

        Correct.

    24. LRLenny Rachitsky53:16

        ... in the way you guys operate. And I imagine sometimes Tobi disagrees with Glen and then they talk about it and things get ironed out.

    25. AAArchie Abrams53:22

        Totally.

    26. LRLenny Rachitsky53:23

        Great.

    27. AAArchie Abrams53:23

        And that's why we come together every six weeks.

    28. LRLenny Rachitsky53:25

        Hmm.

    29. AAArchie Abrams53:25

        Kind of everyone in person to review every project so we can hash out those disagreements. We go through all the core projects, all the merch service projects, all the growth projects. And it's a great forum to say, "Hey, here's where we disagree really on the how and the, the tactics of what's happening." It's... And we can flag those things, have good debates about where there might be misalignment.

    30. LRLenny Rachitsky53:43

        Amazing. What a, what a unique way of working. I'm so fascinated by all this. So what I'm hearing essentially within core, Glen and his team come up with here's what we're gonna build the next quarter. You guys have, uh, twice-a-year releases. Is that right? Or is it every season?
16. 54:30 – 58:48

    Cross-team collaboration

    1. LRLenny Rachitsky54:30

       drive growth, hit these goals. How do you collaborate ac- across these two teams? What... Do you have a model for how you work together? Because these feel like very different ways of working.

    2. AAArchie Abrams54:41

       Yeah. H- honestly, it's been one of the things I'm very proud of. Like, we've built a really great partnership for the last three and a half years because it, it's intentionally meant to be almost at odds. And, and that's like part of the structure o- of how we want to work. Um, but, but it comes from, I think, a place of...... of respect on- on both sides. And it's safe for anyone. It's, okay, here's what growth is gonna do, but we're gonna do it in a way that's- that is high quality, that is shipping really good stuff for merchants. We're probably gonna approach it in a- in a faster way. We might disagree on things. But we're gonna have reasonable paths to kind of handle that conflict. And so while there's no magic bullet, it wasn't like, "These are the surfaces that growth can touch, these are not," it was like, "You can go anywhere in the product." But let's go figure out how to work together to figure out that quality bar, to understand when you're gonna be different on it, on the quality bar, to get something out to learn. And just building trust along the way, that we're actually gonna ship high quality things when we ship it to 100% and move. And so a lot of great work on the team to make that- those relationships really strong.

    3. LRLenny Rachitsky55:47

       Got it. So basically you guys are like, "Uh, moving this button over here is gonna drive so much growth." And then Glenn's like, "No. This is not acceptable. We don't want a button here. This looks terrible, everyone's gonna hate it." So it's- so that's a healthy tension. Like I'm describing in like a compatible way.

    4. AAArchie Abrams56:01

       That's a healthy tension. Yeah, yeah.

    5. LRLenny Rachitsky56:01

       Yeah.

    6. AAArchie Abrams56:01

       Totally. And it's like-

    7. LRLenny Rachitsky56:02

       Yep.

    8. AAArchie Abrams56:02

       ... okay, so how we gonna- how we gonna work to figure this out? And it might be, "Hey, we're gonna move the button. Hey, let's run the- let's run the test. Let's see the short term lift. You know we're gonna monitor it long term.

    9. LRLenny Rachitsky56:10

       Mm-hmm.

    10. AAArchie Abrams56:10

        You know when we ship it, it's gonna be high quality, like high quality polished." Okay, and you trust us to like make those- those trade-offs. And I- I wish I had a better answer of like, it's very human, right? It's very that trust that's very, uh, that's very important to any of these. I think growth with other teams is like, there's no replacement for just the human... trust, and then following through on commitments of, oh, we're actually gonna make this thing really good.

    11. LRLenny Rachitsky56:38

        Is there an example of that that comes to mind? Where you had something that was- you thought was gonna drive meaningful growth. You showed it to Glenn, he's like, "No. I don't know about this." And then either you iterated, or you just like forget it, this isn't right for the platform, even though it's gonna sh- drive some meaningful growth.

    12. AAArchie Abrams56:54

        The place that we- we often come back to is, and this is with, you know, I think Tobi is great, Tobi and Glenn, is on wizards. So wizards of-

    13. LRLenny Rachitsky57:07

        Like onboarding carousels? Yeah.

    14. AAArchie Abrams57:08

        ... onboarding carousels, some way that basically has folks get set up by not using the actual product. And so we've- oh, we've always kind of danced around, and we- we have a very specific no wizard principle. But I think the- sometimes the- the tension is wizards do serve a- can serve a purpose in certain circumstances. And so we've- but we've avoided doing that. But we've always worked to try to make the principles of what a wizard does really well, which is it- it simplifies the product into something that allows people to have a lower bar, to try to work with core to bring that into the actual experience itself. So the example of that experiment I mentioned to you, of giving pre-filled sections in the online store editor, you could have solved that in a wizardy way of like enter a few things and we're gonna generate these sections for you. Instead, we actually took those pre-generated things based on what we know about you, and put it into the actual project experience itself. So it tried to get at some of the principles of what a wizard can do well, without avoiding the- the wizard principle, without creating an actual wizard. So that- that's been some of the like, how do we work together to get the intent of what the growth idea is, but in a way that's consistent with the- the way we want to build in core?

    15. LRLenny Rachitsky58:34

        Got it. And I get why that you think about this a lot, 'cause you talked about one of the biggest levers is onboarding and helping more people get, uh, activated. And so I could see why you spent a lot of time thinking about how do we help pe- more people succeed there.

    16. AAArchie Abrams58:46

        Yes.

17. 58:48 – 1:01:12

    The importance of an opinionated founder

    1. AAArchie Abrams58:48

    2. LRLenny Rachitsky58:48

       I want to ask your insight on this idea that people might be listening to this and feeling like, oh, we need to build a team that just builds great product, and is not constrained by metrics and driving growth short term. Thinking long term, thinking 100 years, like this is inspiring I think to a lot of companies, because this sounds great. What do you think it takes to make something like that work? Because in a bad case, this team just sits around and builds whatever they want, and the rest of the company's like, "Goddamn, this sucks. I have to show success, and metrics, and moving a metric, and this team over there just build beautiful things." Is it like you need a founder like Tobi that prioritizes this and values it and has really good taste and intuition? Like what do you think are important elements of something like that, of- of this approach working at a company, based on what you've seen?

    3. AAArchie Abrams59:37

       I- I- yeah. I think it- it needs to have a very opinionated founder or set of people who are driving what good looks like. I think if it is... and I think Shopify, you know, a few years ago, before maybe sometimes drifted into the mode of we are just gonna build stuff, and each kind of team is just gonna build stuff, not really accountable for it. And that is a very, very bad state to end up. So I think you either have to use, my sense is, metrics as accountability, which is the most common kind of way to drive accountability and focus, or extremely strong founder, or set of folks who have extremely strong opinions on what good is and what taste is. If you're one of those two, you can make it work. But the worst case is, let's just go build a bunch of cool stuff in kind of a haphazard way that I don't think would work.

    4. LRLenny Rachitsky1:00:30

       Yeah. This is great. So either you need metrics to tell you if you're doing the right thing, or really correct and good taste in- in your founder.

    5. AAArchie Abrams1:00:42

       Correct.

    6. LRLenny Rachitsky1:00:42

       Cool. I think that's a really good way of sum-... and like I imagine every founder is gonna think, oh, that's me. I have this. I can do this. I think it's rare in real life, like it's rare that you're like a Tobi or Brian Chesky-

    7. AAArchie Abrams1:00:54

       Yes.

    8. LRLenny Rachitsky1:00:54

       ... or- or Elon.

    9. AAArchie Abrams1:00:55

       Yes.

    10. LRLenny Rachitsky1:00:55

        Yeah. These are s-

    11. AAArchie Abrams1:00:56

        100%.

    12. LRLenny Rachitsky1:00:56

        You know, it's hard... yeah. It's hard to internalize that, but I think that's the reality.

    13. AAArchie Abrams1:01:01

        Yes.

    14. LRLenny Rachitsky1:01:01

        So most people will be more successful building things that are driving metrics they can track in an experiment.

    15. AAArchie Abrams1:01:07

        Yes.

    16. LRLenny Rachitsky1:01:08

        Awesome. This is very fascinating. I am so happy we're spending so much time on this.

18. 1:01:12 – 1:06:42

    Growth and sales integration

    1. LRLenny Rachitsky1:01:12

       Okay. There's a few other random things I'm gonna touch on. One is sales. So historically Shopify's been very product like growth, very organic, go check it out, sign up, shop a store, start a store, grow. And you guys have layered on sales. In a sales motion, that's an increasing part of your business. What have you learned about your team, the growth team working with sales and making that a successful relationship?

    2. AAArchie Abrams1:01:36

       Yeah. No. I- i- it has been great over the last couple years as built out the sales org because it's, it's added a whole new kind of motion to Shopify as Shopify's product got better. It wasn't... It can a- it can serve the biggest companies in the world. It was like the natural evolution, well, let's... We're gonna do that for people to grow up on Shopify and to be the biggest companies, but we're also gonna take folks another platform, bring them over. For growth in sales, I think the biggest learning from the, kind of the R&D side at least, has been the scale is very different w- with sales. And so it's really hard to use as much kind of quantitative data to make some growth, to make some of those decisions. And so a lot of it has been building, um, much more qualitative insights working with merchant success, sales about the face, the challenges they're facing and onboarding, uh, a large customer. So how do we build import tools that work for them? How do we make sure they have the right guidance in the product for a very different set of use cases? So a lot of it has just been like very much empathy building with sales about what that merchant journey looks like. And quite frankly, challenging ourselves to think differently. That's been one thing. And then second, we kind of built two very distinct funnels for a, for a little bit. There's like a sales funnel. You come in, you contact us. That's it. There's no mention of self-service. There's no this. There's just like drive MQLs, boom. Then there's the self-service thing. There's no mention of sales anywhere. And so one of the last things the year... Last year we've been really doing is how do we create these hybrid journeys where there is... We shouldn't force the merchant to choose, "Do you wanna talk to sales or do you wanna do self-service?" We should give them the options, whatever path that they want to go on. And so a lot of that has been building into the self-service journey over to sales and then from sales into self-service. That's broken a lot of metrics in the business, that's broken a lot of ways people have thought about their jobs. And so there's been a lot of kind of cultural resetting and just getting smarter from a metric standpoint about how do we measure this thing, go hybrid journey. They came in via self-service, they went over to sales. How do we value each of those components, um, in the process? And transparently, it's something we're still getting better at. Um, but it's really important to, to get there.

    3. LRLenny Rachitsky1:03:55

       Is there an example of something that broke that would be illustrative of what you're describing?

    4. AAArchie Abrams1:04:00

       Yeah. Yeah. So thing that breaks is you drive someone to, uh, from s- an ad over to self-service. We typically look at only the self-service LTV of that person. But what happens is they come in, they sign up via self-service, and then they go talk to sales. They get changed to a sales driven merchant, which means that that value of that merchant, which is usually actually quite large, does not get associated back to that ad campaign. Oh, guess what that makes? That means you would probably reduce investment on that ad campaign because you weren't valuing that because our system had two different models for calculating LTV. Sales driven one and a self-service one. Uh oh, we're gonna make suboptimal investment decisions now by kind of moving things around, even though it's the right thing to do. So a lot of it is in rebuilding a lot of the instrumentation, how we do LTV modeling, how we do attribution, how we do incrementality testing across each of those different types of outcomes. Because it was not kind of an intuitive thing for us, you know, originally because we had built all of these systems with a much more-

    5. LRLenny Rachitsky1:05:14

       Yeah, makes absolute sense.

    6. AAArchie Abrams1:05:16

       ... siloed view.

    7. LRLenny Rachitsky1:05:16

       Yeah. Basically, attribution gets a lot more complicated. Uh, are you going in like a multi-touch attribution direction or is there something even more clever?

    8. AAArchie Abrams1:05:25

       You know, my, my rant is I am a, uh, well, I won't touch attribution as it's, has its place. I think ideally what we want to get through is what we really care about is incrementality. And so incrementality is kind of the, the gold standard for people who are less like, you know, attribution measures like how do you assign value to a given touchpoint, right? Click, a view, et cetera. But it doesn't tell you causally what drove something, right? That's where incrementality tells you. Incrementality test is basically don't show ads on Meta for certain number of people, show it to the other set, see what the lift is in the outcome. And so a lot of what we're doing is trying to get a lot, is continuing to get even more sophisticated and incrementality measurement for not just self-serve outcomes, but for self-service outcomes that then drive to sales, for sale specific outcomes. And as soon as we have that kind of incrementality at the channel level, we can get a lot more sophisticated in terms of our, our bidding, budgeting and all that. But that's really the key, the key thing we want to get to.

    9. LRLenny Rachitsky1:06:35

       There's certain topics that alone can be their own podcast conversation to just dive deep into this stuff. But I'm gonna stop myself and not go further down that track.

19. 1:06:42 – 1:08:49

    Shopify’s marketing structure

    1. LRLenny Rachitsky1:06:42

       Let me touch on a couple more things before we, before I let you go. Uh, one is marketing. So we talked about sales. Marketing, you guys don't have a CMO. There's no Shopify CMO. Instead you embed marketing leads within the org. For folks that are trying to grapple with that, should we hire a CMO? Should we do something else? What have you learned about maybe the benefits and also maybe some downsides of approaching it the way you guys have approached it?

    2. AAArchie Abrams1:07:07

       The, the benefit is, so there's, there's growth marketing who sits in...... and growth. There's revenue marketing that sits over closer to sales. There's a, a brand team under Harley, who does, who does amazing work, our president. Um, there's marketing embedded in core and PMM sit with product managers there. There's shop marketing on a consumer side. So it's, marketing is truly everywhere in the org. And I think the benefit of it is it's closest to the primary kind of goal that those marketers are trying to do, right? They sit with growth, so we can focus on kind of that self-service motion. Harley's an amazing communicator, so brand sits with, with him so he can be... He can have a lot of influence over that. And so I think it sits with the people what's most relevant outcomes are driving, which is great because it allows us to move faster with less kind of coordination. I think it only works because Tobi, Harley have such amazing intuition on what the brand is, needs to be, and all of that, that some of what the CMO does of kind of creating the cohesive story of Shopify is kind of held in, in their, their heads and kind of th- they, they have the pin on that. And so that allows then that piece, that CMO's job, to not be as important at Shopify, but the other pieces are obviously critical, but they can be now closer to the, the action in where they're kind of going to drive the most, most impact. The downside is things are sometimes very messy, right? So that's the... (laughs)

    3. LRLenny Rachitsky1:08:40

       Yeah, it's another example where the founder can... Their background and interest and skills can impact significantly the way the work is structured and who you hire and don't hire.

20. 1:08:49 – 1:11:09

    Insights on discounting from Udemy

    1. LRLenny Rachitsky1:08:49

       Okay, uh, one last question, totally different topic, discounting. So you worked at Udemy for a long time, and from what I understand, discounting was one of the key reasons Udemy succeeded and one of the big differentiators. Um, I'm curious what you learned about discounting, the power of discounting as a growth lever.

    2. AAArchie Abrams1:09:09

       Yeah. So d- so Udemy is a very, uh, as we know, Udemy is an online marketplace for, for online courses. So come on course. And I think what was happening in, you know, I started, we were there 2012-ish, was people were, "What is this online course thing? I don't really understand what it is. I don't understand what the value is and what I'm willing to pay." And so what discounting has a really powerful effect on is, it can signal value with a high list price, but then bring something down to an affordable price. And that may seem like, of course that's- that's obvious, but in online courses what was important is the list price would be high, at 100 bucks, so it's associated with like a college course, but what people really value this thing as was a book. And so you could signal very high, signal quality through price, which was very murky at that point in online learning. Signal value through price, discount it to 10 bucks, or that was the typical Udemy deal, and then... So 99% off, 90% off. We might see like fire sales, but- but it- it changed the value and willingness to pay, and then it tapped into the fact that, and still is, education is very aspirational. And so what a lot of people missed in that, in education is, yes, we want people to actually take the course, but that's actually in many cases not the job to be done, that there's an emotional job that's even more important, which is I'm feeling like I'm making progress in my educational journey, and just the act of purchasing a course or the act of buying a book is progress. And so if you can make it very enticing, very high value thing, cheap, urgency, you can let people make that emotional journey by the act of purchasing, which then allowed us to actually have very good retention, because you could keep coming back to that emotional job over and over again, which discounting with urgency allowed us to

21. 1:11:09 – 1:17:45

    Lightning round

    1. AAArchie Abrams1:11:09

       do.

    2. LRLenny Rachitsky1:11:10

       Amazing. Well, with that, we reached our very exciting lightning round. Archie, are you ready?

    3. AAArchie Abrams1:11:15

       I'm ready.

    4. LRLenny Rachitsky1:11:17

       First question, what are two or three books that you recommended most to other people?

    5. AAArchie Abrams1:11:21

       So one, I- I love to go back to, like, marketers who wrote in like the 1920s. And so one that I- I love is Scientific Advertising by Claude Hopkins. So he's actually one of the first kind of direct marketers that came out. He kind of innovated on some of the concepts of copywriting and just how you, like, sell a product around, can't make this product, can't sell the product, and you tell the product will help the customer achieve their goals. And so it's really fun, I find it really fun to go back in time because there's a lot of really good first principles thinking that I think we've actually lost in more modern stuff where it's like personalization bandits, optimization, all this stuff, where it's like, no, like what... How do you actually write and sell things really effectively? So Scientific Advertising is, is a great book.

    6. LRLenny Rachitsky1:12:09

       Just like the name alone, it sounds really cool, especially for someone in your, in your (laughs) shoes. That feels like the perfect book for your, your role. And I think there's so much wisdom in just, like, the things someone figured out many years ago about what convinces people to buy something is still true and people over-complicate it, and just coming back to the original is often really useful.

    7. AAArchie Abrams1:12:28

       Totally. Um-

    8. LRLenny Rachitsky1:12:29

       Awesome.

    9. AAArchie Abrams1:12:29

       And then The Perfect Mile about, um-

    10. LRLenny Rachitsky1:12:31

        Oh. Mm-hmm.

    11. AAArchie Abrams1:12:32

        ... the, uh, the chase for a sub four-minute mile by Roger Bannister and a few other, um, folks is, uh, just a wonderful... As a runner, it's a really fun book to read about kind of perseverance, how these folks really kind of all competed to get to that, that really amazing goal of, of under four minutes in a mile.

    12. LRLenny Rachitsky1:12:52

        Awesome. Do you have a favorite recent movie or TV show you really enjoyed?

    13. AAArchie Abrams1:12:58

        You know, I- I went back in time and I watched, for the first time actually, the entire seas- or all the episodes of The Sopranos.

    14. LRLenny Rachitsky1:13:05

        Mm-hmm.

    15. AAArchie Abrams1:13:06

        Which was quite, quite fun. Highly recommend. I did that and The Wire in the last, like, six months.

    16. LRLenny Rachitsky1:13:13

        It's a lot of watching.

    17. AAArchie Abrams1:13:14

        It's a lot of... So it's a lot of watching. I, you know, work out in the morning on my, my elliptical or bike, so it's a nice way to...... that's a nice, uh, workout s- show.

    18. LRLenny Rachitsky1:13:22

        That's so smart. That's a good motivator to just work out to something. I gotta watch the next episode. The Wire is like-

    19. AAArchie Abrams1:13:26

        Exactly.

    20. LRLenny Rachitsky1:13:27

        ... it's like hour-long episodes and five seasons times 20... I think it's 22 episodes per season, right?

    21. AAArchie Abrams1:13:32

        Yeah.

    22. LRLenny Rachitsky1:13:32

        Oh, jeez. It's a lot of, a lot of watching but, and I, I did that once and it was like, I've got a lot, I've got a lot of episodes to watch. But incredible. Okay. It's funny I sh- you should say the Sopranos, I feel like a number of people recently told me they're watching the full Sopranos again. It's like a trend recently for some reason.

    23. AAArchie Abrams1:13:49

        Oh, interesting.

    24. LRLenny Rachitsky1:13:51

        Anyway, do you have a favorite product you recently discovered that you really love?

    25. AAArchie Abrams1:13:54

        So the AI music creator. Um, my kids and I ... I'm, I'm the least musical person in the world. It's been amazing. My kids and I will create songs together about our days, about what's going on. So it's just been, been really fun to be able to have a musical experience for a non-musical person and have that creative experience for them and it's been, it's been really awesome. Do you-

    26. LRLenny Rachitsky1:14:15

        Suno is insane. I think it's Suno.AI-

    27. AAArchie Abrams1:14:17

        Yes.

    28. LRLenny Rachitsky1:14:17

        ... if folks wanna check it out. It's just-

    29. AAArchie Abrams1:14:19

        Yes.

    30. LRLenny Rachitsky1:14:19

        It'll, it's just, like, such a fun party trick too, just to, like, write a song on the spot about something that you're thinking about. Awesome. Two more questions. Do you have a favorite life motto that you often come back to, find helpful in work or in life?

Episode duration: 1:17:45