No Priors Ep. 50 | With Stripe Head of Information Emily Glassberg Sands

1. 0:00 – 0:38

   Background

   1. EGElad Gil0:00

      (instrumental music) . Today, Sara and I are joined by Emily Glassberg Sands, who's the head of information at Stripe, which includes data science, growth, machine learning infra, business applications, and corporate technology. Emily was previously the VP of data science at Coursera, where she led development of AI-powered products to have personalized learning, scalable teaching, skill measurement, and more. We're excited to talk with Emily today about Stripe, AI, fintech, and education. Emily, welcome to New Priors.

   2. ESEmily Glassberg Sands0:32

      Thanks so much for having me.

   3. EGElad Gil0:34

      Oh, yeah. Thanks so much for joining. So you now lead the information org at Stripe. Can you tell us a

2. 0:38 – 2:31

   Emily’s role at Stripe

   1. EGElad Gil0:38

      little bit more about what the organization does, how it's evolved under your tenure, and what are some of the span of responsibilities that you're focused on?

   2. ESEmily Glassberg Sands0:45

      Yeah, so I joined Stripe, uh, back in 2021, originally actually to lead data science. And David Singleton, Stripe's CTO, reached out. I didn't know a ton about Stripe, but I knew millions of businesses were using it to collect payments, which had to mean really interesting data on those businesses and on a large swath of the economy. Stripe's clearly helping companies run more effectively and also in a position to learn from its data what kind of interventions significantly improve companies' long-term success, uh, and in some cases, to actually action those. Today, I wear two hats. So, uh, the first is I support a bunch of different teams that are together tasked with enabling the effective use of data across Stripe, and this includes, you know, from decision-making internally to, uh, building data-powered products. We've been investing a bunch in foundations, which includes building out our ML infrastructure and better organizing our data, you know, the really sexy stuff. Um, but also in applications like seeding a bunch of new gen AI bets and, and getting them out to our users. So that's kind hat one, and then second, I'm accountable for our self-serve business. So a huge number of SMBs and startups come to Stripe directly. Um, to get started, they self-serve through the website, and we're really focused on understanding who those users are, um, getting them the right shape of integration efficiently, um, building product experiences, um, that meet their needs, including as they grow, um, and growing the portfolio of products they use. So, um, for many of our users, it's not just payments, but invoicing or subscriptions or billing or tax or rev rec, um, depending on, on what their business model demands.

   3. EGElad Gil2:21

      Yeah, and I guess Stripe for a long time has been doing different things in ML in terms of traditional ML. You know-

   4. ESEmily Glassberg Sands2:26

      Yeah.

   5. EGElad Gil2:26

      ... I think fraud detection and the fraud detection API that you all have is one example of that. But you were

3. 2:31 – 4:44

   Adopting early gen AI models

   1. EGElad Gil2:31

      actually quite early in terms of adopting LLMs and sort of early generative AI models. Could you tell us a little bit more about how that came about, how the interest was sparked, and how adoption really took off?

   2. ESEmily Glassberg Sands2:41

      I mean, I think it's fair to say that Stripe is, first and foremost, an AI company. As you know, fintechs, including Stripe, have long used traditional ML in many contexts, including sort of fraud and risk. But first and foremost, we're building financial infrastructure for the internet. So Stripe got started by enabling first really digitally native startups to accept online payments, and then over time, millions of companies started relying on Stripe's financial infrastructure for a bunch of different needs, whether that's reducing fraud or managing money flows or unifying online or offline commerce all the way to launching embedded financial offerings. Um, and so as not a kind of first and foremost AI company, uh, we, probably like a lot of people listening to this podcast, had kind of our, like, "Hey, what the heck are we going to do?" moment, uh, a year or so ago when LLMs really broke through the zeitgeist. And we were looking at the technical breakthroughs and the product launches all over the ecosystem, um, with awe, but also honestly, a little bit of overwhelm. The sense of, well, there's very clearly a real opportunity here to better serve our users, but what is it exactly and how do we get it off the ground quickly and safely? So it starts with a story of three engineers who hacked together in three weeks an internal beta for an LLM explorer. And the basic idea of LLM Explorer was, "Hey, let's get a ChatGPT-like interface in the hands of the 7,000 talented Stripe employees and really let them figure out how to apply it to their work." Our leaders all the way up to John and Patrick have intentionally crafted this strong culture of kind of bottoms-up experimentation, and we think a lot about sustaining it internally as we grow, and with LLMs, it was no different, right? And so where we started was, "Let's quickly unlock internal experimentation. Let's get LLMs safely in the hands of all employees at Stripe." The enthusiasm was palpable, you know, at Stripe, as it was across industry. We knew the experimentation, um, was going to happen, and so we really wanted to make sure that we enabled it to happen well and safely.

   3. EGElad Gil4:42

      It feels to me like when people start adopting

4. 4:44 – 8:17

   Promoting internal usage of AI

   1. EGElad Gil4:44

      LLMs, they tend to do it in sort of three areas as an enterprise. There's what are you doing in terms of external products and how do you incorporate it? There's how do you use it for internal tools or use cases? And then there's what are your vendors doing? You know, if you're using Intercom or Zendesk, are they adding it, and if so, how do you think about that as a company? The third one seems to be each team kind of deals with it as our vendors bring it up. Um, I found in, in general, people have tended to follow the pattern that you mentioned, which is they start off kind of thinking, "Hey, what should we do externally?" And then they immediately collapse into doing something internally just so that people get their hands on it, they try it out, they kind of see what it does and how it works and they get some internal efficiencies, and then they start thinking about the external product side of it. Um, you know, was there anything you did specifically to start to promote that internal usage? Did you do a big internal hackathon? Did you try other ways beyond sort of the, the, the s- some of the things that you mentioned in terms of adoption internally just so that you start spreading the thinking and knowledge about it?

   2. ESEmily Glassberg Sands5:42

      I think you're spot on that a lot of companies, you know, the first place they go in their mind is, "How can this manifest in our product? How can we help our users?" And then they realize, "Hey, any one person can't actually answer that question." We need to be putting this in the hands of folks with a range of different backgrounds and expertise thinking about different parts of our product and business to really apply it. And that's exactly what kind of this built-in-three-weeks beta did. Within days, a third of Stripe's...... were using it, and you can think of LLM Explorer basically as a front end that supports multiple models in the back end. So we started just with GPT-3.5 and GPT-4, but today, we serve over a half dozen models through the tool. We knew we needed to have certain security features, stripping PII and rehydrating, et cetera, straight from the start, and we spun up, yes, a Slack channel and a hackathon and more to help kind of build momentum. We didn't actually need to do much to build, um, momentum. Within days, a third of Stripes were using it, and so from there, we started to look at, okay, what are they using it for? What do we see in the logs? And the answer was, Stripes were using it for all sorts of things, honestly, um, but there was this opportunity cr- to create more community and sharing in the tool directly so that they could build on each other's work and weren't sort of doing that in- in- inside Slack. So, um, a simple example, but shortly after we launched the original tool, we set up this little functionality called presets, and it basically just lets you save and share your prompt engineering. Um, maybe this exists. If not, some startup should go build it for everybody, and then everyone else at Stripe can, like, search and upvote, and you see what bubbles to the top, and basically, overnight, we had, like, 300 of these reusable LLM interaction patterns, and they ran the gamut, but, you know, um, just an example, like, thousands of Stripes still today use the Stripe style guide, which basically, you know, I don't care if you're a product marketer writing copy for the website or a sales development rep writing a cold email or, like, you're an exec who's preparing for a meeting, you run your copy or talk tracker whatever through this style guide, and it returns back to you the same content in Stripe tone. The enthusiasm was palpable, and we had to figure out ways to harness it and build more of a community around it. Um, the weekly active user count of this LLM Explorer is still at almost 3,000, which is just shy of half the company using it every single week, um, and yeah, for sure engineers, but also a ton of salespeople and marketers and ops folks. So I think there's a lot that the technology can actually do to create the community, and then the next step is, okay, how do you get it from this, like, internal prototyping to actually, like, enabling also more production-grade solutions?

   3. SGSarah Guo8:16

      How did you begin

5. 8:17 – 10:36

   Applied ML accelerator teams

   1. SGSarah Guo8:17

      to look at that data or go from explore to exploit here, right? Uh, so one of the, uh, um, Stripes I talked to said that I should ask you about applied ML accelerator teams. I don't know if that's, like, here or further along in the funnel of, like, you know, your- your plan to get, um, these new AI capabilities distributed across the company in- in real ways.

   2. ESEmily Glassberg Sands8:39

      We can talk about it here, and we can talk about it later. So the idea of accelerators is basically, um, ring-fencing one to two-pizza teams, and multiple of them, to get new AI bets seated, and one of the accelerators was actually what produced this LLM Explorer. So it's very hard to just pull three engineers off of, you know, their work building Radar to build an LLM Explorer, so we have this sort of, um, experimental bats funding internally. It's run out of, um, you know, by David Singleton, so out of our CTO's office, and it'll basically be like, "Hey we'd like to build a one-pizza team, and we- we want to fund it for six months, so relatively durable, and here's roughly the charter, and here's roughly the milestones, but we're gonna learn and iterate as we go," and so actually, this infrastructure is an example of an output from the accelerator. Um, we have other accelerators that are working on the applied side. So, hey, you know, we know that given the advances in LLMs in particular, there's way more we can do for our support experience, both user-facing and also internally for our ops agents, um, and to your comment earlier, Elade, for sure, there's third-party solutions we can buy, but is there some, you know, homegrown solution that can actually be used across a variety of internal applications, um, and can we just go build that? So that's- that's another example of the kind of thing, um, that our applied accelerators build, and I think, you know, the applied accelerators aren't... You staff a one, you know, you fund a one-pizza team and you go hire these people. There are actually opportunities for growth and development for internal talent. So the vast, vast majority of folks who join the accelerator have been at Stripe for many years. Um, they're doing a rotation onto the accelerator. It's likely to become their permanent home, but that's up to them.

   3. SGSarah Guo10:33

      You mentioned you have some favorite applications that have already,

6. 10:36 – 13:30

   Radar fraud assistant

   1. SGSarah Guo10:36

      like, these sort of assistant capabilities. Can you talk about some of them?

   2. ESEmily Glassberg Sands10:39

      The primary ways we're finding, um, LLMs useful today at Stripe in user-facing applications is first automating the writing of code, um, and then second, accelerating information retrieval, and both are proving really powerful for our users. So on automating code, um, Radar Assistant and Sigma Assistant are two new products that are in beta and rolling out to all users soon. Radar Assistant is really about generating custom fraud rules from natural language. So most folks listening probably have heard of Stripe Radar. Uh, it was one of our first non-payments products. It's an ML-powered product. It helps identify and block fraudulent transactions, um, but then in addition to the core Radar product, which works generally under the hood without any user-provided direction, we have Radar for fraud teams, which is about letting users write custom rules. So maybe you know you don't have any customers in a given country and you want to block any transactions, um, from IP addresses in that geo. To generate these rules, employees that are users used to have to code up the rules themselves, but Radar Assistant lets them use natural language to write those rules. So, um, it's a little thing, but speed matters a bunch in fighting fraud. You have to work faster than the fraudsters, and with Radar Assistant, a whole range of people in an organization, from fraud analysts all the way to less technical folks, can implement rules quickly and directly without having to work through a developer.

   3. SGSarah Guo12:03

      I actually think that's, um... And I do want to hear about Sigma Assistant. I think that's a really interesting pattern that applies beyond perhaps the fraud world because there are so many...Um, let's say, like just decision engines today that are some combination of, uh, heuristics and then machine learning together. Um, and, uh, I-I think that will continue and the ability to, um, take natural language, explicitly describe policy and have that work really well with less engineering assistance, I think is gonna be useful in like other domains. Like, you know, it could be underwriting, could be fraud, could be other choices.

   4. ESEmily Glassberg Sands12:39

      Totally. And, you know, I think for some of our customers, um, this is opening the aperture in terms of which employees can use solutions like custom radar rules. But for a lot of our customers, it's allowing them to use these solutions for the first time, right? So think about the non-technical small businesses on Stripe, a bunch of them. You don't have to be technical to get started on Stripe. You can use our no-code integrations. You can use payment links, you can use hosted invoices. These are companies who, like wouldn't dream of coding up custom fraud rules. And so not having to have... They just don't have the developer skills on hand. And so just not needing to, being able to use these tools, um, with just plain English, I think is, is really powerful. And more broadly, I really love that democratizing power of generative AI, and it's very much aligned with, with our founding ethos.

   5. SGSarah Guo13:27

      You're also gonna talk about Sigma Assistant.

   6. ESEmily Glassberg Sands13:29

      Sigma Assistant

7. 13:30 – 14:32

   Sigma assistant

   1. ESEmily Glassberg Sands13:30

      is similar in that it generates code from natural language, but it's in a pretty different context. It's actually applied to generating business insights. So Sigma is our SQL-based reporting product. Um, it lets businesses analyze and get insights directly from their Stripe data. And Stripe data is, as we've talked about, pretty interesting for most of our users, it's all of their revenue data. So which customers where are buying what, for how much, who's retaining, who's churning? Um, pretty central to a bunch of different decisions the firm has to make. And Sigma Assistant is all about making sure our customers' employees don't have to speak SQL to get access to those business insights. They can just use natural language to ask questions, um, of the Stripe data. Um, some of the folks in the beta are asking, you know, really interesting questions and, and getting them answered, you know, from the very basic, "How much revenue did we generate in December?" Um, to, you know, "What types of customers tend to be most delayed with their payments?" So we're excited to be rolling that out broadly later this year.

   2. EGElad Gil14:29

      Where do you hope all this technology to be in one or two years?

8. 14:32 – 17:00

   How will AI affect Stripe in 3 years

   1. EGElad Gil14:32

      Like how do you think generative AI will impact your business, your customers, the way you do things?

   2. ESEmily Glassberg Sands14:37

      When I step back and ask where should we be in kind of three years, five years, I think the vision, the opportunity is, is much bigger than what we could do in, in a year. With a fintech lens in general, I think the current sort of gen AI advances beg the question of what does it actually mean to apply generative AI to the economy at large? You could start with payments optimization. I think folks know that we do a bunch of backend and front end optimizations for payments. Is there some actually new foundation model built on financial data that would blow the existing conversion and auth and fraud and cost optimization models out of the water? Um, you know, we can do incremental model improvements today and quarter over quarter they drive meaningful bips of uplift. But it doesn't feel crazy to think that a good foundation model could outperform more traditional approaches by, I don't know, (laughs) 100 bips, 200 bips. So I think just in payments optimization alone, we can ask the question of what might foundation model look like in that context? And then I think where it gets really interesting, um, with generative AI on all this payments data is can we actually become more of the economic operating system for our users? And you can imagine all sorts of ways this could be productized. Everything from a dashboard of insights, then recommendations to like an API you hit to get customer level predictions to like, you know, turning important business model and personalization decisions. So pricing, recommendations, discounting, kind of on autopilot with Stripe. We know we can abstract away a bunch of the need for our users to worry about payments and refunds and disputes. But you could imagine also starting to tackle those sort of higher order tasks, understanding the value of users and setting the right price and determining the geo strategy. And then this is more on a macro level, but businesses rely on all sorts of economic signals, CPI for tracking inflation or small business index for tracking the health of the sector. And those are very useful for steering business decisions based on macro trends, but they tend to be quite lagging. And so this question of can real-time data speed the time to insight and thus response I think is interesting. So, you know, those are all more sort of future-looking, but I'm very bullish on a world where we're able to really holistically help users grow their businesses, um, well beyond payments, but built on payments data.

   3. SGSarah Guo16:58

      Rewinding all the way back to now, you

9. 17:00 – 18:28

   Knowing when it’s time to invest more fully in AI

   1. SGSarah Guo17:00

      are a year into the exploration. How do you decide to invest beyond a one-pizza team? Like w- does that happen organically in all of the product engineering teams you have? Does it happen where you like at some cadence look at the usage date and say like, "Oh, these like, you know, top-down bottoms up are things we care about." Like, do you need to restructure the org to make that happen?

   2. ESEmily Glassberg Sands17:21

      Yeah, it's a great question and I don't think we 100% have the answer of what's the right operating model, but we've been very conscious and iterative, um, as we're learning. And so, so far the answer is both. Like, you know, it's not one, one-pizza team or two-pizza teams. It's four of them today, and should it be six or should it be eight or should it be ten? And then in parallel, where can we really support the vertical teams or the core product organization in adopting LLMs or generative AI more broadly directly? There are a couple of examples, but at Stripe we're very focused on leveraging AI so that non-technical folks that are users can do things that they couldn't do before. And then also, so that technical folks can move an order of magnitude faster. And there are some pretty obvious industry standard ways that we're finding LLMs can automate the writ- writing of code and accelerate information retrieval.... and we're building those both out of the existing vertical teams and out of the accelerators.

   3. EGElad Gil18:26

      That makes sense. You mentioned, um,

10. 18:28 – 22:04

    Deciding how to proliferate models

    1. EGElad Gil18:28

       that you have s- you know, I think it was at least six different models you're using internally. How do you think about what models to use for what, and do you focus on RAG, fine-tuning, open source, closed source, time to first token, inference? I'm sort of curious, like, what that matrix of decisions is relative to specific use cases, and how you ended up with this, this sort of proliferation of models, because I feel like the more sophisticated people get, the more they tend to have this proliferation happen internally.

    2. ESEmily Glassberg Sands18:54

       So we do have a proliferation of models, but we are not centrally, for example, like within our ML infrastructure group, super prescriptive about what model individual applications need to use. So I talked about LLM Explorer and the presets and sort of that was back in March, and we very quickly turned that into building an internal API for more programmatic use of LLMs, right? We want it to be equally easy and safe for Stripes to build production grade systems and services. There are 60 applications built on that now. A bunch internal, but also several external, and I'm happy to talk about a couple of them. That's what planted the seeds for a lot of the product initiatives we're now investing in more heavily. Um, we have default models, uh, based on the use cases, but we also give individual teams agency to choose based on cost considerations, latency considerations, um, there's obviously, like, depending on the application, different performance requirements, um, and then, you know, there also is this very real question of cost. So, you know, we're running this infrastructure centrally, um, but we found that for, um, the most expensive applications, you know, we do bill them to the local teams. And so, we work with them very closely to understand what makes sense given the ... you know, economics of the product, um, the importance of quality at this stage, how they're thinking about scaling, what the latency requirements are, and so on. Um, we have heard that previously, um, it was a little bit overwhelming for individual teams to figure out what model to use, but also to go through the enterprise agreement and get the infrastructure up and running, and so centralizing a lot of that I do think has, um, sort of economies of scale, um, but again, we're not prescriptive and we, we do leave agency, um, to the individual applications to make those trade-offs.

    3. SGSarah Guo20:45

       Well, what other infrastructure do you s- decide to build centrally, right? So, another Stripe told me that I should ask you about your internal experimentation and, um, sort of testing infrastructure, and so love to hear about anything new you've built in order to, um, like enable teams from, you know, your org or a central org.

    4. ESEmily Glassberg Sands21:03

       Yeah, so, you know, I think it's always a combination of buy and build, and, you know, we recognize that, um, there are a lot of great companies building a lot of great ML infrastructure and experimentation solutions, and some of them are very point and some of them are very general and, um, you know, we stitch together where there's a clear external solution and we build internally where we feel our need is more unique or somehow very important and not currently satisfied by the market. Our experimentation platform is one that we've built internally. We run a lot of charge level experiments and latency and reliability requirements (laughs) for charge level experiments are very, very high. And so building and running that internally has been worthwhile, but there are lots of cases, flight, weights and biases, there's lots of third party solutions that we, that we lean on as well.

    5. EGElad Gil22:01

       When you think forward on the w- the

11. 22:04 – 25:41

    Whitespace for fintechs employing AI

    1. EGElad Gil22:04

       directions that the overall financial services industry is going, and let's put Stripe aside for a second, because I think Stripe, um, is obviously a core company to sort of the internet economy and it touches so many different pieces of fintech and things like that, but where do you think outside of Stripe the biggest white space for fintechs employing AI is? Like, from a startup perspective or even an incumbent perspective, like where do you think this sort of technology will have the biggest impact?

    2. ESEmily Glassberg Sands22:30

       It's a great question and I don't know exactly what others will do. I think, um, having a really robust understanding of identity, who businesses are, what they're selling, um, has always been important, and, you know, I think often in industry we think it's important for marketing or sales or sort of go to market motions, but it's also super important in fintech. Um, yeah, it's important for credit lending decisions, but it's also important for supportability, um, decisions and understanding, uh, where, you know, the business does or does not meet the requirements of, uh, a given card network, uh, or a given bin sponsor. And so I think that that identity piece, like who is this merchant? Are they who they say they are? Um, but also what are they... what's their business? What are they selling, and how does that map to this pretty complicated regulatory environment, um, is a really interesting and hard problem that lots of folks are solving in their own ways, but, uh, is, is likely, um, an opportunity. I think there's almost certainly an opportunity to, you know, whether Stripe does it or somebody else does it, to make, um, sort of financial integrations way more seamless. Um, Stripe has a whole suite of no code products so you can use, uh, you know, payment links or no code invoicing. But how does one actually build, uh, a really robust, um, specific to the user integration without needing, um, you know, a, a substantial number of payments engineers or, um, any complicated developer work? LLMs are proving that they can be very good at writing code. Um, we have a couple cases actually where we're already seeing it work, but...... as the, as the decisions get more and more complicated, I think there's still a lot of work to do, um, to build the right integration and to build it well, um, in an automated way. And then I think, as I mentioned before, some of this layer on top of the payments data of like, okay, you could build solutions that make payments work better, but payments actually allows you to really deeply understand and improve the business is- is pretty fascinating. And you'd have to think about like, is it a startup that does that or is it an incumbent that does that, and what's the, what's the business model? Um, what's the business model there? But, you know, if I think about the case of Stripe, um, you know, sort of Stripe has the opportunity to be beneficent, right? Incentives are super aligned. The more Stripe can help its users businesses grow, the more Stripe grows, and the more the economy grows. And so, whether it's Stripe or someone else using financial data to help businesses be more successful, to grow the pie, to grow the GDP, um, I think is- is really powerful.

    3. SGSarah Guo25:38

       It's a really unique dataset. Is there something

12. 25:41 – 27:51

    Leveraging payments data for customers

    1. SGSarah Guo25:41

       in that data... Um, the obvious example to me that comes up is Radar, but, uh, otherwise, like, leveraging that data and giving it back to merchants in some useful way already?

    2. ESEmily Glassberg Sands25:51

       Yeah. So Radar's a great example. I think you also see it throughout our payments products. So maybe the most salient to a consumer, like an end user, not our customer but our customer's customer, would be something like the Optimized Checkout Suite. So it's this bundle of front-end payments optimizations, and it's a lot of little things, honestly, like dynamically presenting payment methods in the order that are most relevant for the customer that really add up in terms of driving efficient checkout experiences for end users, and in turn, driving up revenue for our customers and- and growing the internet economy. Um, less salient to- to the end user is this whole host of back-end payments optimizations. So, um, for example, we use ML to optimize authorization requests for issuers, basically identifying the optimized retry messaging and routing combinations, uh, to recover a big chunk of false declines, about 10%, so billions of- of dollars globally. Um, and there are very similar applications, um, across a range of our products. So for example, for recurring charges in our billing product, we use smart dunning to reduce declines. Um, it actually reduces declines by about 30%. You basically identify the optimal day and time to retry a payment, um, for transactions that are declined, for example, due to insufficient funds. It's really easy to know at what day and time sufficient funds will pop in. And the list goes on. You know, Stripe Radar, which you mentioned, you know, considers a thousand characteristics of a transaction and figures out in less than 100 milliseconds if- if each of the, you know, billions of legitimate payments, um, made on Stripe, um, can go, can go through. And so-

    3. SGSarah Guo27:28

       Mm-hmm.

    4. ESEmily Glassberg Sands27:28

       ... you know, those are all payments or payments-adjacent optimizations, but conversion, auth, fraud, we didn't really talk about cost optimization, that's another one, are all places where having that scale of data allows us to create a better experience for the end user, create more revenue for the business, and grow the economy.

    5. SGSarah Guo27:48

       So I know that your background in, um, like

13. 27:51 – 30:10

    Labor economics and data

    1. SGSarah Guo27:51

       labor economics has influenced both your career decisions, like joining Coursera and Stripe, and your approach to data science. Can you, like talk a little bit more about like how you think that shapes you as a leader or even Stripe's approach to like understanding like macro trends and macro data?

    2. ESEmily Glassberg Sands28:06

       The through line in my career, both in academia and then in industry, has been using data to understand how individuals and firms make decisions, and in particular, to help those decisions be higher quality. And so, you know, you mentioned labor economics. I've long been fascinated by who gets access to opportunity and why. So it started all the way back in college, I met this playwright in New York, she told me less than a fifth of productions on US stages were written by women, and asked if I could help figure out why. And as part of that, I read an audit study, so, you know, some excellent playwrights donated four never-before-seen scripts. I sent them out to hundreds of theaters and asked them, um, whether they wanted to put it on stage, why or why not, and I just varied the pen name. Like, is this written by Mary Walker or Michael Walker? Um, and briefly, you know, basically I found that when purportedly written by a woman, the exact same script was less likely to be produced, but more importantly, the theater community cared. Like, they wanted the best plays in production, and so the study spurred awareness and over time, change, and today half the productions on US stages are written by women. And I think that early experience showed me how powerful data, especially when you use kind of robust econometrics and causal inference and actually are getting to the root of the drivers, um, can be in understanding and improving decision-making, and it's why I pursued a PhD in economics. It's what took me out of academia, um, to Coursera. Coursera at the time, uh, had only 40 people, but it already showed the potential to dramatically expand access to world-class learning, and done right, also downstream labor market opportunities. Um, and that's also a lot of what led me to Stripe. You know, well before me, Stripe was operating as kind of a beneficent player in the ecosystem and has been very interested in genuinely helping businesses on Stripe grow and using data to do that. And sometimes we, um, help by guiding them and sometimes we help by actually just building the product for them, but that's been kind of a through line in- in my journey and a lot of what I- I love about Stripe.

    3. EGElad Gil30:08

       How much does Stripe think about macroeconomics?

14. 30:10 – 32:54

    Macro economic trends for strategic decisions

    1. EGElad Gil30:10

       So you have this amazing view into the global economy through all the commerce transactions that are happening on your platform across so many different industries. How does that data inform how Stripe thinks about different aspects of its business? So for example, my sense is Google, through AdWords and other ad-related products, similarly has a pulse on where spend happening or not happening. Does it look like we're tipping into a recession?

    2. ESEmily Glassberg Sands30:34

       Yeah.

    3. EGElad Gil30:34

       ... that impacts hiring decisions or other things for them. I'm just sort of curious if similar things translate for Stripe over time.

    4. ESEmily Glassberg Sands30:39

       Yes, certainly, um, we do get rich insight into where the economy is headed, and we use it to guide our internal decision-making. I think there's an interesting question we're exploring on, is there a version of this that we can actually be providing to our users to help them make decisions and help them grow? The example of, like, the CPI or small business index and, like, can we get that in users' hands six months earlier (laughs) so that it's way more actionable is a really interesting question. And honestly, we're early days there, but I think as part of thinking about how might we become more of the economic operating system for our users, it's not just the micro-components of, you know, how do you price or how do you personalize, it is also the macro-components of how do you think about the ecosystem that you're operating in and how can we help you operate more effectively given the macro trends that you're operating in?

    5. EGElad Gil31:41

       That makes sense. So you don't look at the one pizza team and say, "No pepperoni this month." You know, "Cheese only."

    6. ESEmily Glassberg Sands31:45

       No, no, no, no, no.

    7. EGElad Gil31:46

       (laughs)

    8. ESEmily Glassberg Sands31:46

       I mean, I think you know John and Patrick pretty well. They as... Yeah. No. That would be whiplash. Stripe is in a, um, very fortunate position to really be in charge of our own destiny and be able to take a very long-sighted view in choosing where and how to invest in the business. And so, no, from the, from the perspective of, like, do we add-

    9. EGElad Gil32:07

       Yeah.

    10. ESEmily Glassberg Sands32:07

        ... 10 people or 100 people or 1,000 people, we're not micromanaging at that level. That's much less driven, honestly, by the macro on average, and much more driven by where do we see opportunities to serve users given what's happening, and, I mean, we can even talk about AI users, right? AI users actually have... There's this whole wave of AI startups, and they have fundamentally different needs than a bunch of the waves of startups before them. And so, that actually begs the question of where do we invest more now to get ahead of those needs, um, 'cause we know, 'cause we know there's demand.

    11. EGElad Gil32:41

        Yeah, makes a lot of sense. I guess, uh, the last area that we had sort of questions about given your, um, background and all the amazing things you've worked on over time is, you know, you spent a lot of time at Coursera, which is really focused on how do you bring different forms

15. 32:54 – 35:36

    How will AI impact education

    1. EGElad Gil32:54

       of online learning and knowledge to the world. And one of the areas that a lot of people have talked about from a global equity perspective in AI and its impact is education.

    2. ESEmily Glassberg Sands33:02

       Yeah.

    3. EGElad Gil33:03

       And so we're really curious to get your thoughts on how you view AI impacting education, but also importantly, where will that first substantiate? Is that a US-based thing? Is it certain countries or markets? Is it K- through tel- 12? Is it college? Is it post-college learning? Like, we're just a little bit curious how you think about, you know, AI in education, and where is it going to be most important in the short run versus long run?

    4. ESEmily Glassberg Sands33:25

       I was at Coursera for about eight years. I grew from an IC to leading the end-to-end data team. And through that journey, I was increasingly motivated by building products that were only possible because of the data. And the first places we started were in the obvious places. Oh, you personalize discovery of content, you personalize the learning experience, you do more to scale the teaching experience. But where we moved to relatively quickly was what you would think of as less education and more labor market, which is how can we use education data to help learners and companies measure and close skills gaps and get folks into the jobs that best fit their skill profiles? And so, you know, that's not at all to downplay the opportunity, uh, that we have in AI to make meaningful advances in how, you know, elementary school students learn and make that learning really customized-

    5. EGElad Gil34:25

       Right.

    6. ESEmily Glassberg Sands34:25

       ... to them and make sure that there is high-quality instruction in lots of pockets of the world that wouldn't otherwise have it. But I also think there's this important pull through to the labor market. And, you know, I'm a labor economist by training. You know, people get education for two reasons. They get it to develop, um, skills, but they also get it to be rewarded for those skills in the labor market. And that first piece is like how you develop skills, the learning, and that's really important, and AI can definitely help. But the second piece is like how you signal that learning out in the market. How you build a credential. And, you know, I- I was on, um, uh, some world economic forums, and we were working a bunch on, "Hey, could we make with data, skills more the currency of the labor market?" And Coursera has substantially moved that direction, including in their enterprise product, and I hope many others will move that direction too. Um, not instead of or as a substitute for using it in a learning experience, but just recognizing that so much of what individuals need from education is that signaling, is that credential. I think the best way, most equitable, fairest way to do that, uh, is through skill measurement.

    7. EGElad Gil35:33

       Okay. That makes a ton of sense. As discussed earlier, Stripe has this

16. 35:36 – 39:24

    Unique needs of AI startups

    1. EGElad Gil35:36

       amazing vantage point into all sorts of different online businesses and how they're evolving over time. What are the differences between some of these AI-centric, sort of next-gen companies that Stripe serves as customers, versus what you've seen traditionally in e-commerce or SaaS or other areas?

    2. ESEmily Glassberg Sands35:51

       It's a great question. I mean, we've worked hand in hand with the builders of a bunch of different technology waves to make sure they have the financial infrastructure they need. Some of the earliest waves were marketplaces, infra platforms, social media. Um, think kind of the young DoorDash or Instacart or Postmates or Twilio and, you know, those grew up to become some of the, the largest companies today. We've, we've grown up with them. There's also, as you noted, kind of the SaaS wave, and the current wave is AI. And in terms of the unique needs of AI startups, um, probably four notable differences versus the prior waves. You know, the first is just at a basic level, unlike a bunch of the past generations of software startups, we're seeing AI startups have substantial compute costs right out of the gate, and that that's putting a bunch of pressure to build monetization engines faster.Um, the second thing we're seeing is a lot of these startups are seeing global demand for their products straight out of the gate, right? They're making digital art or music or all sorts of borderless things, and they want to get that across borders from day one. Third, I would say, is a lot of subscription businesses, and obviously we see subscription businesses in a bunch of different contexts, but especially sort of the AI startups that are consumer-facing, heavily skewed towards subscription business models. And then I'd just say fourth, as a corollary of the first, may be obvious, because these startups are generally monetizing at a much earlier stage, they're at an interesting spot where, you know, with very lean teams, they need to operate financially, uh, like very real businesses, right? They need to grow up, uh, a little faster than they're, than they're sometimes ready. And so, uh, we're seeing, you know, a bunch of, uh, adoption of our revenue and financial automation suite, uh, to deal with, to deal with those differences.

    3. EGElad Gil37:38

       That's pretty amazing. It actually reminds me a little bit of the '70s. Uh, original, uh, vesting schedules were four years because companies would go public within four years, and so that's where the four-year vest comes from for stock, and so I think historically, companies used to grow up really fast, and then you look at the initial internet wave and Yahoo! and eBay and a variety of companies became profitable within a few years. And so it feels like this AI wave is exhibiting a lot of the same characteristics, and that may just be reflective or indicative of, um, real product-market fit and enormous user demand that's almost pent up. I feel like whenever you have one of those waves, that's when you see this rapid monetization, so.

    4. ESEmily Glassberg Sands38:13

       It's happening so fast. I mean, we saw a massive spike in the number of generative AI companies on Stripe over the last year, you know, and a bunch of them were two-person teams you've likely never heard of all the way... well, maybe you've heard of, but most people haven't, all the way to kind of hyper-scaling startups with millions of users like Otter.ai and, and Midjourney. You know, we were looking at the list of top 50 AI companies put out by Forbes last year and noticed over half were using Stripe, and, you know, oftentimes, you have top startups and a bunch of them aren't even really monetizing yet, and it's striking what share of these AI companies are monetizing and monetizing early, um, and monetizing fast. At the foundation layer, yes, OpenAI and Mistral, um, but also a bunch of companies at the application layer. Um, Moonbeam for writing assistant or Runway for video editing, which is pretty remarkable.

    5. NANarrator38:57

       Emily, this is a great conversation. Thanks for doing it with us.

    6. ESEmily Glassberg Sands39:00

       Thank you so much for having me.

    7. EGElad Gil39:01

       Yeah, thanks for joining. Very good to see you, and, uh, amazing as usual.

    8. NANarrator39:06

       Find us on Twitter @nopriorspod. Subscribe to our YouTube channel if you wanna see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way, you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.

Episode duration: 39:24