Roundtable #7: Spotify, Adobe and Linkedin on How AI Changes The Future of Product & Design | E1097

1. 0:00 – 0:50

   Intro

   1. HSHarry Stebbings0:00

      What do the best product leaders think about AI? I talked with the heads of product from Adobe, LinkedIn, and Spotify. And we covered everything from LLM predictions in 2024 to how product teams should adopt AI.

   2. SBScott Belsky0:12

      I think it's actually common belief that there are gonna be a few mega models in the future that are gonna do everything for every company in the cloud. What we're saying is, actually, it will probably be the opposite.

   3. TCTomer Cohen0:22

      When you're thinking in an AI first principle kind of way, you're really unleashing the idea of control. What happens basically in AI is you don't control the experience anymore.

   4. GSGustav Söderström0:30

      Something that designers need to get good at in this world, they need to understand GPT-4 as well as they understand the user. So an example that I think is a good example is ...

   5. HSHarry Stebbings0:42

      Alrighty. I am so excited for this. I've wanted to do this one for a while. So we're gonna do a couple of intros first so everyone can get familiar with each other's voices. And so

2. 0:50 – 2:20

   Panel Introductions

   1. HSHarry Stebbings0:50

      we're going to start with you, Scott, then Gustav, and then Tomer. Who are you and what are you most well known for?

   2. SBScott Belsky0:56

      I was the founder of a company called Behance back in the day, and, uh, we were acquired by Adobe. Um, for the last, uh, six years or so, I, I served as chief product officer for about five years overseeing the creative cloud business and then, um, and then have, uh, recently taken over strategy corporate development, uh, design and emerging products for the company in this kind of chief strategy emerging products officer type role, uh, over the last year.

   3. HSHarry Stebbings1:20

      (laughs) It's quite an all encompassing role. Um, Gustav, your turn.

   4. GSGustav Söderström1:23

      So I'm Gustav Sederström and, uh, similar to Scott, I started and sold a few companies and then, um, I worked at Spotify for almost 15 years now as a CPO and then CTO and now recently as, uh, co-president together with, uh, Alex Nordstrom. So we run the company together.

   5. HSHarry Stebbings1:41

      Very exciting transition there. Tomer, your turn.

   6. TCTomer Cohen1:44

      Hey, everyone. Uh, chief product officer for LinkedIn. Uh, responsible for basically what we build from the company's strategy to overseeing the teams building it. Been at LinkedIn for 12 years now. Joined, uh, right at the cuff of shifting from desktop to mobile as a company, so that was fun. I started my career as an engineer, so I was doing anything from semiconductors, chip on design algorithms, to embedded systems, all the way to the internet right now in AI.

   7. HSHarry Stebbings2:09

      Okay, so now everyone knows individual voices. We're actually also very lucky because there are three very distinct voices (laughs) . I didn't realize quite how, how apt the selection of this roundtable was in that perspective.

3. 2:20 – 4:52

   AI's Role in Evolving Product Development

   1. HSHarry Stebbings2:20

      I want to start with probably kind of the most broad but also kind of important, which is fundamentally we're all told that AI changes everything that we do. When we apply that to product, what do we think are the most significant ways that AI will change the product development process, specifically on the product development side? What will change specifically with AI? And I just want to throw that out there so anyone can jump on it.

   2. SBScott Belsky2:41

      Well, I mean, I'm sure we're all experimenting with, um, with GitHub Copilot in our engineering teams and there will certainly be, you know, a, an improvement in both the productivity of developing code and products and also testing and, um, you know, and, and identifying and reconciling bugs and that sort of thing. But the only other thing I would throw in there is that when you're developing the interface of a product, you are oftentimes trying a few different approaches to find the one that works best. And that just takes a lot of time, uh, to have three or four different options and, you know, and explore, but then, you know, stop going down that path and go down another path. And then fast forward now into, you know, today into the future, we're in a world where AI will suggest alternative scenarios. You can try variations and just look and, you know, with the mistake of the eye, find a better solution that, or at least something you want to A-B test. And so I think that the product development process, you know, will, w- the products will become much better frankly, you know, just with, with fewer cycles, we'll find better solutions.

   3. GSGustav Söderström3:47

      I have some thoughts there on, um, I mean, one way to maybe think about it holistically is that back, uh, when I started and, and Tomer just referenced this when, when it was the transition to, to mobile and even before then, at least I kind of thought of the UI as the product. And then machine learning came along and the machine learning was there to solve, help the UI and that shifted completely. Now AI is the product and the UI is there to, to help the AI to capture better signal. Yeah, I think most people agree that the real transition there was TikTok which is like almost no UI. It's just the video and the UI is just trying to make the export export algorithm be more efficient than anything before it. And, and, you know, most people in tech agree that it wasn't really the algorithm of TikTok, it was the UI that maximized what was, you know, pretty traditional export export algorithm like since the days of Hot or Not. So I think that's the big change, like the AI is the product and the UI is there to help the, the AI these days and, and maybe it always was. I mean, users never came to Spotify to click buttons. It was always about new music or something. It's just more clear now, I think.

4. 4:52 – 7:51

   Future of UI in the Age of AI

   1. GSGustav Söderström4:52

   2. HSHarry Stebbings4:52

      I've had so many guests on the show before say about the stupidity of UI and how AI will make UIs redundant.

   3. SBScott Belsky4:58

      I mean, UI is here to help us understand the object model of a pr- like, you know, find what we want, you know, and, and, and with as little friction as possible engage and so, you know, UI in some cases should disappear but there's new UI these days in the form of the tone of a product and the way that something is, um, is the inflections that are used, the persona, you know. I'm, I'm like thinking about UI design in the future as in some ways like persona design. When I'm interacting with, uh, with Spotify's DJ, you know, and the DJ is like cracking jokes and like being casual with me, that's a UI decision to some degree of a product that's driven by AI as an example.

   4. GSGustav Söderström5:39

      Yeah, absolutely.

   5. TCTomer Cohen5:41

      I, I, I would emphasize one thing here because I think we're, we're jumping into the UI and, um, we talked about the desktop to mobile, uh, transition. I think at LinkedIn I've been holding that flag around like there's the mobile first movement and there's, there's the AI first movement. It didn't start last year. It started for me probably 2016 already.One of the biggest implications for me when building product is the understanding that, I think the Gustaf spawn AI is the product. But for many product builders, AI is something they delegate. And I think that's, like, really been delegated to the AI team or the engineering team. I think AI strategy starts from the CEO and makes its way down. And, uh, with many companies right now, AI, that's something that the team is doing. Reminds me of, like, early on with the mobile team, there was a mobile team building it and it was a whole different... The rest of the company was building something else. Like, the analogy I give usually is, like, imagine, like, a river rafting, a river rafting boat and you have the guy at the back with the two massive pedals and that's AI today in product. Then everybody else on the sides, they add accuracy, they add some power, but they're not as significant as the guy on the back with the pedals. And one of the biggest changes I've seen with, uh, AIs, you know, when we moved from desktop to mobile, not everybody made it through the transition because it required you to unlearn how to build a little bit. It wasn't about hedging, I'm going to put all these things there and watch the kind of heatmap. It was about making a decision about how you build a relevant experience. When you're thinking in AI first principled kind of way, you're really, uh, unleashing the idea of control, in my opinion. So what happens basically in AI is you don't control the experience anymore. It's not deterministic anymore. For many product leaders I know, it's really hard to let go. It's like you're a chef at the restaurant and all you dictate is the ingredients, maybe the knobs a little bit, but you don't control the output. I can't tell you how many people just flop on that. They just cannot un- comprehend the fact that they don't control the experience.

   6. GSGustav Söderström7:41

      I like that.

   7. TCTomer Cohen7:41

      And that's the massive af- affirmation of AI first mentality.

   8. GSGustav Söderström7:44

      It's like probabilistic, yeah, experiences rather than deterministic, right?

   9. TCTomer Cohen7:48

      100%. It's not deterministic.

   10. HSHarry Stebbings7:50

       Do you guys

5. 7:51 – 8:55

   Managing Uncontrolled AI Outputs in Business

   1. HSHarry Stebbings7:51

      not find that inherently concerning? Like, I think every company on this call is a public company. When you don't have the control of the outcome and the outcome could be a hallucination, d- is that not concerning to you when you don't control the outcomes?

   2. SBScott Belsky8:07

      Well, it's funny. I mean, you know, in, in some ways, listen, for some, for some practices, hallucina- hallucination is a bug but in some areas it's also a feature, right? So if I'm trying to discover some cool new music, I mean, I can imagine that might be a feature as opposed to a bug. You know when you're trying to-

   3. HSHarry Stebbings8:26

      Exactly.

   4. SBScott Belsky8:26

      ... you know, when you're trying to do generative fill in Photoshop and imagine what's behind an object or, uh, you know, extending the frame of a photo, hallucination is actually a feature. Uh, and if you get the wrong answer or something you don't like, you can just, you know, run it again. So, um, I think it is a bigger problem though in, in applications that really, you know, are mission critical around, you know, writing NDAs, you know, (laughs) for legal purposes or, you know, where, where hallucination actually could get you in trouble.

6. 8:55 – 9:58

   Innovating UI Through AI Advancements

   1. SBScott Belsky8:55

   2. HSHarry Stebbings8:55

      How radical do you think you get with UIs? Everyone here wants to be at the forefront of design, of kind of interfaces with consumers. How radical can one be in how far you push it quickly?

   3. GSGustav Söderström9:08

      I think, uh, I want to build on something Scott said there because I think it's the definition of what is UI changes. So y- you took the example of the, of the AI DJ we did and you're exactly right. It was the design team, that user tested w- what is this? Is this, uh, a utility like Google? Is it a, an AI person like Alexa or, or do we actually, um... is the UI that we digitize to real person, with a real personality that exists? And, and we chose the latter. So that is, that is design and, and it is brand. So I think designers need to, need to think holistically about the experience in that sense and that, that is still design. It is the user experience. And I also think something that, that designers, um, need to get good at in this world is to understand the, the capabilities they have.

7. 9:58 – 14:05

   Importance of AI Literacy for Designers

   1. GSGustav Söderström9:58

      To u- they need to understand basically GPT-4 as well as they understand a user. What, what, what, uh, what it needs and how it works. So an example that I think is a good example is Midjourney. So Midjourney, the designers there clearly understood the, the performance of the model when they built the Midjourney experience. Right? It took like two minutes to generate an image. It was wrong one out of four times. They could have built a horrible experience where you waited for two, two minutes and got disappointed 75% of the time. Instead, because the designers there and product people understood the capabilities, they built a full

   2. GOGuest (brief laughter only)10:30

      (laughs)

   3. GSGustav Söderström10:30

      ... tolerant UI that gave you, because they also understood how, how diffusion works, so you can do it in stages, they gave you four low res shots at the same time in 20 seconds. And said, "Is any of these good enough?" Right? So that's what I think design needs to do. Designers and product people need to understand the models and the performance very, very deeply so that they can make sure the experience matches the current level of performance. It's the same for us. If, if we have a one in 10 shot in recommending a good song, we probably need to recommend 10 on the screen at the same time to get like a good chance of, of, of a hit, right?

   4. HSHarry Stebbings11:02

      Okay. Gustaf, you said there about kind of understanding the models very deeply, but again, I am basically an absorber of knowledge from smart people and then I try and kind of amalgamate it together. Everyone tells me that actually every company or the best companies will use multiple models at the same time and transition between them. How will product people be able to have eight different models in use at the same time, know all of them really well to utilize them effectively? How does that look?

   5. TCTomer Cohen11:28

      I, I don't think it's the product people. I think this is where you build, like, a platform that basically enables you to understand the task you're trying to accomplish. Cost, we should talk about cost at one point because this is a very costly software and costly technology. And then that's actually what you're bringing mass people with, uh, you know, deciding what kind of model you use for what purpose really starts at the application layer, but the decision is not made at the application layer. Like, that's actually... If I think of, like, why I would expect to see some massive innovation and startups to show up, it's in this tier.... to really allow people to leverage multiple models at multiple cost centers and resources and-

   6. HSHarry Stebbings12:07

      Yeah.

   7. TCTomer Cohen12:07

      ... efficiencies, and then completely mask it from the developer from kind of the front, um, like from the designers or the product folks.

   8. SBScott Belsky12:15

      Just to, just to, uh, I completely ag- agree with Tomer on this. Like, I think it's actually sort of commonplace belief that there are gonna be a few mega models in the future that are gonna do everything for every company in the cloud. And I think what we're saying is actually it will probably be the opposite, right? There will be many, many, many long tail models, some learning locally on people's machines or in applications that they install, some open source, some not open source. And there needs to be, like, logic around the routing, you know, not only to the model that's the best, um, prepared or, you know, the most, uh, specialized model for a given, uh, query, but also the most cost efficient. And, um, and a lot of companies like ours, you know, in market with AI products today are actually, you know, counting on the reduced costs over time, you know, from some of those technologies. So I think that's a great point and I actually, you know, there are some startups now that are kind of defining themselves as, like, router, uh, you know, startup.

   9. TCTomer Cohen13:13

      There, there's, like, two levels I think which are kind of fun to think about. One is at the application layer and one is at the kind of more of the kind of meteor layer, which is, like, the dispatcher analogy.

   10. SBScott Belsky13:23

       Mm-hmm.

   11. TCTomer Cohen13:23

       And there is a dispatcher at the kind of application layer from the application. Like, imagine the idea of agents, right? You come in... We all know by now building one agent to rule them all is not the right design. You want to have multiple types of agents. Like, I take, uh, a complication like LinkedIn, there's, like, a seller agent, um, there's a, a knowledge agent, a job seeker. There's so many agents you want to build, but you want to build as a dispatcher around that, that know almost like a, a team coach that knows, like, how to get that team to work together. Same at the meteor level when you want to start building some kind of dispatcher or router, to Scott's point, around which technology to best use. That, that's where I, I could imagine tremendous innovation happening.

   12. SBScott Belsky14:04

       Mm-hmm.

8. 14:05 – 16:41

   Analyzing Costs in AI Model Adoption

   1. SBScott Belsky14:05

   2. HSHarry Stebbings14:05

      Can I ask, guys, you know, w- when we think about the multiple models that we'll use, you mentioned there that we'll have actually kind of, uh, open/close, very specialized. When we think about, like, cost, the more models we have, the more costly it is also and margins are impacted. How do we feel about cost and implementation when thinking about model adoption?

   3. GSGustav Söderström14:25

      From Spotify's point of view, um, most of the stuff we built in-house, to Tomer's point, is actually mostly about cost, right? So if you, if you look at, uh, for example, as we talked about generating a lot of voice, you want to generate like two minutes of voice per day for half a billion people. It's a billion minutes that can ruin you. Like, cost is actually already, like, the biggest factor in if you can deliver products. Even though the models are there to generate great voice, the problem is actually doing it, uh, cost-efficiently. So, so I think that's completely right. A lot of the innovation and, and technology has to go towards routing, often actually for cost purposes. I, I do think I have a little bit of a different view on there is some chance that per company, I think you will want to embed the entire user history in one model, all right? So today most companies, including Spotify, many separate systems that are optimized for different purposes and they sort of collaborate in semi-predictable ways. There's not one system, but if you look at some of the papers coming out of Amazon and so forth, they are starting to look at you, you re- you literally tokenize all of it basically. The log, the how you scroll, how you click, what you listen to, what was in the content that you listened to, and then you just, you just do a prediction, token prediction based on that, based on all of this, what is going to happen next. So we may still see, like, one large model. I think the world is going to go towards you embed your entire user history, everything they did into one space. I still think to, to your point, Scott and Tomer, you will have specialized models for, for example, for voice, for video, for di- for different purposes. So it depends on if you're talking about the user data or about generic capabilities like rendering and voice and so forth.

   4. SBScott Belsky16:08

      Yeah. And in some cases I guess you could take the user data at, at the router level and kind of, you know, go to different models for different purposes.

   5. GSGustav Söderström16:15

      Exactly. You can embed that-

   6. SBScott Belsky16:17

      Right.

   7. GSGustav Söderström16:17

      ... you know, as a user history.

   8. SBScott Belsky16:19

      But I think, I think Gustaf's point about the cost efficiency, the good news is that the desire to have more, um, performative models, you know, and the desire to have more cost-efficient models are often like parallel efforts, you know, towards the same outcome. And so, you know, this will all get... you know, the margins will get better as these models get better also.

   9. HSHarry Stebbings16:40

      Do you think

9. 16:41 – 18:04

   Moore's Law's Impact on AI Development

   1. HSHarry Stebbings16:41

      this follows standard Moore's Law Theory in terms of development of technology and the cost reduction that we see there? Like, timeline wise. Is it years and years?

   2. GSGustav Söderström16:51

      I, I think it will. Y- you know, some people say Moore's Law is, is, uh, starting to come to an end in terms of just more transistors, but we barely started on neural hardware. So I'm sure we're going to see the same effect even if it's not more transistors per square inch necessarily for a, for a good while. Uh, I also think we'll start... we'll continue to see bigger and bigger models, but there is the opposite pressure as well. Every month, there is a much smaller model that did what the bigger model did a month ago better. So bo- both things are happening at the same time. So I do think we'll see that progress for some time.

   3. TCTomer Cohen17:22

      No, coming from embedded systems, you can already see verticalization of software with chips dedicated where, you know, Microsoft announced they're gonna start building their ƒ. You can actually see, like, the idea of how do I get that efficiency and gains in resources and power and cost. You know, when I was building semiconductors a long time ago, it was like every chip was about, "How do I get, you know, in half the power, double the capacity and, like, lower the cost?" So that was always the kind of mindset, and I think you can start, start seeing that verticalization. This goes to open versus closed. There's so many implications of that. But if you're ƒ to that-... closed, highly resourced, optimized system, you'll start seeing that from companies and Apple and Microsoft and so on.

10. 18:04 – 19:36

    Balancing Data Size and Model Efficiency

    1. TCTomer Cohen18:04

    2. HSHarry Stebbings18:04

       I was gonna ask you, uh, each of you have incredible amounts of data, you know, LinkedIn, Adobe, Spotify. Insane amounts of user data. Everyone kind of puts the question forward of what comes first. Is it the size of the model or the quality and size of the data? How do you think about that and the importance of data versus model?

    3. TCTomer Cohen18:22

       Yeah. I, I think the size of the model matters, but it really (laughs) depends on what you're trying to do. If you wanna have this, like, uh, amazing, uh, kind of personal assistant like in the movie Her, then, like, you wanna build, like, a massive model and... By the way, the model size is really the number of parameters you have. Uh, that's really pretty much it. But, uh, on, on the flip side, and this is where, like, uh, this is where, like, it depends on what you're trying to solve and this is why I'm not a big fan of broad questions because, like, I usually ask, "What are you trying to do?" It's like, the volume of your training data also matters. If you have a large model that is under-trained, it will underperform a small model which is really well, like, well, well-trained. So you're just wasting resources and you're gonna get, like, uh, less efficient, uh, results. And then there's already examples right now that, like, when you build specialized model for media analogies, like an athlete that you transform into a weightlifter or, or like a long distance runner, they perform much better. So, like, you know, uh, I, I can't talk on behalf of Scott and Gustav, but if we're trying to build, if I'm trying to build a go- job seeker coach guide, I, I, I would better build that agent specifically for that role versus some kind of, like, lifelong coach that will be very hard for me to build. Those, those will perform much, much better.

11. 19:36 – 26:43

    Overcoming Challenges in AI Model Implementation

    1. TCTomer Cohen19:36

    2. HSHarry Stebbings19:36

       Can I ask for you guys, what was the hardest thing about model implementation? When you look at current product and current tech stack, when thinking through model selection and then model implementation, what's the hardest thing?

    3. SBScott Belsky19:47

       The little secret is that there's a lot of, um, final mile tuning and work that probably we're all doing under the hood, that is work that only we could do 'cause we know our customer really well, we know our product and experience really well and, you know, it's those little finesse moments. I mean, it's the same playbook all along, right? What makes a customer love using a product? Because they, you know, it's... If the product is empathetic to their problems and if the interface, you know, meets them where they are and, you know, I think we get lost sometimes in the technology and forget that it's not the technology that makes us successful, it's the user's experience of the technology that makes us successful. And that's why I think the role of designers is more important, not less important, in this modern world. And, you know, I, I've been thinking a lot about, uh, just the, the pace of change that we're all dealing with these days in our, you know, in our respective companies. I'm sure like you, uh, li- uh, li- like me, you're waking up and every morning you're like, "Oh my gosh," like, "What, (laughs) what new breakthrough do I have to figure out today, you know, in terms of how it, how it im- impacts our business?" You know, and in this, like, newsletter exercise I force myself to do every month called Implications, like, I, I'm, I'm calling it the, uh, surfing waves and the Cambrian explosion. This notion of, like, you pick a wave and then you're like, "Oh, wait, I'm on the wrong wave." Like, how do you transition to the next, that wave over there, but then wa- this wave over there and it's almost like this frenetic, you know, uh, uh, motion that we're all in. And the only, like, res- solution I can come to is just, like, doubling down on empathy with the customer. You know, it's like we're so... There's so many new technologies and possibilities being thrown at us, but if we just kind of, like, take the pulse of where the customer is and where they're likely going even more so, like, maybe we can help make the right decisions. Otherwise, it's, like, wild.

    4. GSGustav Söderström21:36

       Yeah. I, I, I agree with, with, uh, Scott. And I, I think on your question on what was the hardest thing, that depends, as Tomer said, (laughs) sort of like, what are you asking about and, and when? Initially, it was very hard to find talent, uh, on the technology side. That's now getting easier. And so the problem keeps moving up the stack. And, and as we talked about, I actually think the hardest thing for us now has been to sort of retool the company to think about what the model needs-

    5. SBScott Belsky22:02

       Mm-hmm.

    6. GSGustav Söderström22:03

       ... uh, to serve the user. You know, to rethink design, to re- rethink, uh, product. For a while, there was a lot of work around, uh, making sure data was useful. So it keeps changing, what the problem is.

    7. HSHarry Stebbings22:14

       What does retooling the company mean, Gustav? Is it... Like, again, this is kind of my question, which

    8. GSGustav Söderström22:18

       It's, it's what I spoke about. We, we literally had, have had... We have a bets board where we kind of stack rank what we're doing. We literally had a bet called, uh, AI is the product, for like two years, (laughs) to really emphasize the shift. Getting people in that mindset, um, educating them to understand how this works, to start to use these things and get empathy, not just for users, but for the models. Because people first underestimate the technology and then they're excited and they overestimate what it can do, so they, they create products that are not mature yet. And so there's a lot of work in getting people realistic about where the, where these things are right now. That, that was probably the hardest thing, to get everyone sort of on the same level of, of knowledge and understanding.

    9. TCTomer Cohen23:02

       Yeah. One- w- wait. O- one fun one, and I'm curious if Scott and Gustav are doing this as well, but I remember when we shifted from desktop to mobile, we literally had to force people to bring mobile designs, because again, they were like... We had to force them. Literally said, like, "There's not gonna be a j- a, a jam session unless you bring a mobile design." Now I wa- now I wanna see your prompt.

    10. GSGustav Söderström23:21

        Yeah.

    11. TCTomer Cohen23:21

        'Cause ultimately that, that's the interface that people are using to generate their results. So I, I wanna, want to learn how you're building your prompt and I wanna critique your prompt. I wanna understand your prompt really, really well. And, uh, I, I can't underestimate how much intelligence and even art goes into building the, the right prompts.

    12. GSGustav Söderström23:39

        For sure.

    13. TCTomer Cohen23:39

        But I never imagined doing product jams and asking to see people's prompts going into the, into the session, and now we're doing this on a regular basis.

    14. GSGustav Söderström23:47

        I think this is, like, a very democratic thing, 'cause it used to be if you were, like, a product person or a designer and you had a, an idea, you kind of needed to convince engineering to build a prototype. In this space, what is happening more and more is like-

    15. TCTomer Cohen23:59

        I like that.

    16. GSGustav Söderström23:59

        ... the designer or the product person come...... with the prompt and the idea of edit. It just generates something and then you can, you know, that's very empowering, I think.

    17. TCTomer Cohen24:06

        Hundred percent.

    18. SBScott Belsky24:07

        I- I- Listen, this is, it's- it's- it's so true and, you know, we've all been talking about design-driven, you know, innovation and product and whatever for- for a decade or more. Um, but I- I think this point about how you have these core teams that are building APIs that are ultimately representing the capabilities of the model and you can actually directly empower designers to start to explore interfaces and- and, you know, and to the point about the intonations and the persona and all those other decisions that these designers are now making. They thought they went to school for graphic design or product design, but now they're actually, you know, thinking about the conversation inflections and all these other nuances of design. But that's what we did too, you know. We had our design team actually build a team within that, um, did all of the early developments at Firefly and it was a very, like, design-driven exercise to figure out the interfaces and how they integrated into the products.

    19. GSGustav Söderström24:59

        I wanted to just quickly go back to one question you asked there, um, about sort of size of models and- and data, 'cause I- I think it's interesting to sort of think about. My bet is that size of models will keep being very important for some time. Actually now, you know, the Chinchilla paper, it's very predictable how it's going to scale and it's unclear why it wouldn't scale even if slightly diminishing for- for a while more. We're not even at the level of the brain yet in terms of- of- of, uh, of connections. So- so that seems, uh, likely. My hunch would be though that in the longer term you're going to be able to do most of what you want with reasonable size models and, like, having a much bigger model doesn't help that much. So I would still bet that having lots of user data and lots of high fidelity user data, basically great user understanding, is going to be important, um, in the long term. To- to your question on, like, does- does data really matter? One argument is it doesn't matter, it's all going to be in the model. The other argument is like, no, the models will be very powerful but be able to, to be able to ask them good questions, you need a lot of data about that user. And that's kind of the bet that I would make. The- there was this, um, this, um, paper called MedPrompt that came out recently where, you know, GPT-4, like a general model, if you prompt it the right way, it actually beats, um, a- a fine-tuned, um, model that was fine-tuned on medi- medical data through this very, very clever prompting. But it sort of still proves that in order to do this prompt, you needed to embed a lot of user data. So I still think long term, data is going to matter to have proprietary or a great user understanding of the- of the specific user.

12. 26:43 – 27:34

    Spotify's Approach to Implementing AI

    1. GSGustav Söderström26:43

    2. TCTomer Cohen26:43

       Why doesn't Spotify create its own models, Gustav?

    3. GSGustav Söderström26:46

       Well, partially for that... so we have our own models as well. We d- we do both. We have lots of our own models, but we don't have a GPT-4 that we haven't told the world about. I'm sorry to reveal that. But-

    4. TCTomer Cohen26:56

       Spotify is (shouts) , just... (laughs)

    5. GSGustav Söderström26:58

       (laughs)

    6. SBScott Belsky26:58

       (laughs)

    7. GSGustav Söderström26:58

       But it's also what I said before, like, our goals are different. OpenAI's stated goal is AGI. That will drive different incentives. It will not optimize for delivering two minutes of audio to- to half a billion users per day, right? It's not necessarily going to optimize for- for cost efficiency in building pragmatic products. So that's- that's why, you know, there's no point in trying to compete with them for us because we don't even have the same goal. So we're trying to build exactly these things that we don't think they will build. And we would buy the things that w- they will build or someone else will build. Obviously we're working with- with Google and GCP, so that's where we do most of our work.

    8. TCTomer Cohen27:33

       Guys,

13. 27:34 – 28:31

    Adobe's AI Development Strategy

    1. TCTomer Cohen27:34

       how do you think about the decision to build your own models? I'm just intrigued on, like, you know, I asked Scott at Atlassian that question and he was like, "No, it's not our call. Like, we don't need to."

    2. SBScott Belsky27:43

       Actually, I think that- that's sort of the right answer, is you should build models if you're the best company in the world to build them. And so, um, you know, for us it was like, you know, who- who's going to build a better imaging model, you know? We- we've got the- the advantage in terms of understanding the customer's use cases, we have the data. You know, no one should build a better imaging model than Adobe. And so we're like, "Okay, we need to set out and make sure we hold ourselves to that level." But when it comes to, like, building a massive LLM, to Gustav's point, like, let's partner with the LLMs of the world that- that, you know, do this, you know, as a business. And- and I think it's as simple as that. So what- when we're looking through all the ver- different vectors, you know, we want to build the models that we believe our- our- our competitive advantage is to- is to be able to build and be best at, and then there's others we should partner

14. 28:31 – 35:22

    LinkedIn and AI Model Integration

    1. SBScott Belsky28:31

       with.

    2. TCTomer Cohen28:31

       Yeah. There- there's a- there's a junction there, at least for us, giving just kind of the amounts of data and nodes in the graph is that what we're building is an integration model with the models to be really powerful. So today, many companies will just push all the information to do the prompt, right? And that's more of, like, I would say, like, a interim solution until you have your own integration with the model on the first (inaudible) . So imagine the- the prompt already knows to reach the dispatcher or the router and then it will basically grab the- both of the re- there will be conflation at the tier level that the prompt can also be masked from. And I think that's actually really powerful. I don't know if I want to compete with the nev- you know, in the next dev- uh, developer conference from OpenAI despite us being some, you know, uh, kind of siblings to an extent, uh, from a Microsoft standpoint because they're- that's what they're trying... That's- that's what they'll do best. But what we do best is understanding how it works for LinkedIn. What Gustav does amazingly well is understanding the idea of- of audio and- and media. What Scott will do really well is the idea of design and- and visuals. Building that asset for me is a massive differentiation. Competing for... which I think would be also super hot to compete and you're competing for technology, I don't think you're competing for really solving problems, I don't think is the right solution. I- I- I don't see anybody winning there, really. Does it change how you do anything with regards to data collection, data treatment, data cleansing?

    3. GSGustav Söderström29:55

       I- I- I love that.

    4. TCTomer Cohen29:56

       Or data (inaudible) .

    5. GSGustav Söderström29:56

       Okay, this is... I have to take this one, okay? I'm gonna start.

    6. SBScott Belsky29:59

       (laughs)

    7. TCTomer Cohen29:59

       (laughs) You was keen for that one, wasn't you?

    8. GSGustav Söderström30:00

       Um, 'cause like this is- this is a big one for me. It's been a pet peeve of mine for a long time. I've been preaching, "preaching", like AI first mentality for a long time and really trying to push both design and product folks and-

    9. TCTomer Cohen30:12

       ... even engineers to think about data collection and quality data for a long time. And again, this has been, like, a thought I've been, like ... It's almost like, uh, somebody else will do it. Data science will figure out how to just collect it from the signals. And like, you don't understand, this is, this is the, literally, this is oxygen. This is what feeds ... If AI is at the center, what basically brings it to life is data. I remember, like, early on at LinkedIn, I used to literally, like, spend so much time filtering by myself, data, so I can basically say, "This is good food. This is good diet. This is what you'd eat for, like ... This is what the algorithm should, should eat." But somehow people gravitate towards the UI or they gravitate towards the specific i- the visual elements of it, not understanding that what actually feeds the algorithm is the data. My opinion, as a product leader, whether you're a designer or an engineer or a product manager, like, data is literally at, um, according to your second most important job, uh, in your role. It's understanding how do you br- how do you basically start to infuse all of that collection and make sure it's high quality. The first more important job is understanding the objective of the algorithm, which most people also ... People outsource this stuff. It drives me insane. They outsource what is the objective of the algorithm to do. They kind of create some kind of spec, not- nowhere in the spec is it ever written what is the algorithm supposed to do in a nuanced way, and they outsource this stuff. But data is a massive, massive pet peeve of mine because most people just disregard it, or they just assume it will be there, somebody's gonna create it and collect it.

    10. HSHarry Stebbings31:38

        Scott, Gustav?

    11. SBScott Belsky31:39

        We, we just always have to be very careful because we're so focused on, uh, for the purposes of generating content, for- the, the commercially viable and safe nature of the model matters a lot. We're always trying to, um, learn how customers use our tools. And so we have, like, ways of, you know, having the customer allow us to get data on how they're using the tool, but we don't ever train our models off of the customer's creations. So like, what the customer makes does not train our models, unless they submit it to stock and want it to train our models, um, in which ca- in which case they get compensated for it. You know, it's interesting, like when we first launched our models, like there were, there were some people who went in and, you know, typed in Spider-Man doing something and said, "Oh my God, this sucks." Like, you know, "Firefly doesn't even know who Spider-Man is." And we're, we had to explain, like, that's actually ... that's a feature, you know? It shows that we didn't train on copyrighted content, and you can trust the way that these models are trained as opposed to some other models. So the, the data policies matter, matter a lot. But one thing I should say is that, you know, Adobe's always been two businesses really, it's like the digital media business and the digital experience business, which is a te- set of tools that big companies use for their marketing and their customer data profiles and all that sort of stuff. Whether you go to Nike or any of these other brands, like a lot of them will use our tools to manage their customer experiences at scale. And I've always felt like in some ways Adobe was two different companies sometimes. And in the last year, the- suddenly it makes so much sense because the digital experiences you're delivering, like the marketing, the content, you know, your experience on the Nike website or whatever, can be informed by the company's relationship with you, and the sizes of shoes that you've purchased before and, you know, and where you live and things like that. So, you know, the data around- for marketing purposes, you know, companies are gonna leverage their own data to dramatically personalize digital experiences in ways we can't even fathom right now. And to me that's really, that's really cool, you know, for the future of digital.

    12. GSGustav Söderström33:39

        From our point of view, we had the same journey that Tomer had about data. Uh, we did realize the value of data early on with all the playlists we had, which was the basis for our first sort of, uh, simple embeddings. And so we also- you know, f- for us obviously the, the, the usage data, how you playlist, how you listen, i- is very important. So we had that journey as well. I think the best example is actually, uh, Scott and Firefly, where you're also solving, you're solving an IP problem, you're solving a use case. Like, uh, it's not actually- you're not- you're also solving a technical problem of I want painting and so forth, but you're also solving, um, a business problem by the licenses and the rights you have to the data. I think that's, that's a very important, um, uh, angle that you may forget. You can solve many problems for, for, um, a consumer or bi- or a user, and I actually think that i- in general, you know, we've seen the first wave of AI now where the technology is very impressive, it's starting to do useful things, but we still ... Usually the big changes come when that technology enables a new business model somehow.

    13. HSHarry Stebbings34:46

        Mm.

    14. GSGustav Söderström34:46

        And that hasn't really happened yet. I'm absolutely certain it's going to happen, but so far it's, like, mostly sustaining innovation.

    15. HSHarry Stebbings34:52

        What do you think that business model looks like, Gustav?

    16. GSGustav Söderström34:54

        I don't know yet. I think, I think one, uh, example is, is, uh, Firefly, who, who's trying to, uh, to solve a business problem as well. And most of these things are not yet trying to think about if b- business models can change completely, and, and they probably could. But I think before that happens, this will be sustaining innovation that largely accrues to bigger players. Usually when the business model is challenged and someone counterpositions like a, you know, like a big business model, that's when things really change. And that hasn't really happened yet.

    17. HSHarry Stebbings35:21

        You said

15. 35:22 – 49:17

    Strategies for Rapid Scaling Using AI

    1. HSHarry Stebbings35:22

       innovation and the value accrue to bigger players there. I think that, that's kind of what everyone's asking, which is like who wins? Is it incumbents or startups? And all startups say, "Well, you know, the incumbents, they can't move very fast." And I mean, respectfully, all three of you have moved really fast. My question is like, is it kind of old BS that incumbents can't move fast? How have you guys been able to move so fast to embrace it? Scott, you talked about kind of the, you know, Cambrian explosions or Cambrian waves and jumping to ... How do you move so fast at scale, and is it BS that incumbents can't?

    2. SBScott Belsky35:55

       Well, here's the thing. I mean, every platform shift is not the same, right? They're all different. And if you're going from on-premise software to delivering in the cloud, that is a major, major transition that I can imagine many incumbents would struggle to do in a way as fast as a startup could do. And maybe that's why startups totally led the pack in terms of cloud alternatives to on-premise software in the past.And then you... Mobile was probably the same sort of thing. Different stack of developers, you know, totally different go to market in some cases, different relationships with Apple app stores. And like, you know, there were so many differences. But with AI, I'm not actually sure that as a platform shift, which it certainly is, there are as many, um, differences in how a company facilitates, you know, and, and leverages that platform shift. Because, you know, when someone comes into Photoshop and they have an image that they want to change, right? If I can just give them a default bar that just shows up where they can just articulate what they want instead of using all the knobs and bells and whistles of Photoshop that have accumulated over the last 40 years, right? That is like a very incredible way of disrupting Photoshop to some degree. I don't have to build a net new company and a net new product from zero to one in order to reimagine, um, the capabilities of Photoshop using AI in this instance, right? People might not want to even download Photoshop, in which case that's why we brought these capabilities to Firefly as a standalone app as well. I'd love to take, you know, on behalf of Adobe, I'd love to take the credit of like moving super fast, you know, and I do think we executed well, but I also think that there are some nuances of this faci- this platform shift in particular that do favor those companies that know their customer well and already reach them.

    3. GSGustav Söderström37:44

       It's always easier to analyze other people's companies. (laughs) I think like the risk for Adobe was before they moved to a subscription model. Like the subscription model works with this as well. So that's why it's not disruptive. I think what has to happen somehow, someone has to figure out a business model that is disruptive to someone. Some people think that it will be disruptive to, to, to Google's business model, you know, that the ads model won't hold up in a conversational world. I'm not so sure that's actually what's gonna happen, but something like that would need to happen. I think for Spotify, for example, when we started, it was the business model of streaming that was the disruptive thing. The technology was, was, was, uh, you know, a bit of peer-to-peer and stuff, but it was the business model that took Apple eight years of collateral damage to follow. That, that's kind of what needs to happen, I think. And we haven't really seen that yet. I, I think it might happen. It could also be that this is just more productivity for the existing players. It... I don't know yet.

    4. TCTomer Cohen38:40

       Yeah, I, I, I appreciate the push on the business model because I, I don't think we often talk about it, um, at least when you rebuild, but you can play the, the regular like SaaS model based on seats. And if your technology really makes the organization much more successful, then your seats volume model does not work anymore.

    5. GSGustav Söderström38:58

       Yeah, that's a good point.

    6. TCTomer Cohen38:59

       So then you just... So is it really just price that you increase? Like that, that becomes like a really big conversation to have around what's the business model when your product makes the organization more efficient, you're cannibalizing your own seats model.

    7. HSHarry Stebbings39:11

       There's a brilliant piece by Sarah Tamble at Benchmark actually, which is, talks about selling the work and not the seats.

    8. TCTomer Cohen39:16

       Exactly. You actually still need to sell because the idea of like we talk about hiring and we immediately imagine hiring people, but you're really hiring task completion.

    9. SBScott Belsky39:24

       By the way, this is a... This question about what business models will be disrupted by AI is something we probably don't talk about enough. I mean, think about how many people in the world get paid on an hourly basis for what can be achieved in an hour. I mean, even lawyers charge by the hour still, designers charge by the hour. And now you see these new technologies that are readily available, it's like how does that even... how does it even work? You know, what you're really paying for, of course, right, is like the, the depth of experience and judgment that one applies to their work. You know, your lawyer's ability to know based on their, like, you know, 30 years in practice like what's likely to happen to advise you in the right decision. If that's made in three minutes, like how do you... how do you compensate them? And I think the same goes for the seats part as well. You know, the... if you can... I mean, the collap- the functions of so many businesses are collapsing now. And, uh, and so the idea of selling seats function by function, like how many people in procurement, how many people in financial planning, how many... I mean, it's such an old antiquated way of building a business to some degree in the age of AI. So I'm excited about this new value-based, you know, uh, innovation vector for business modeling.

    10. HSHarry Stebbings40:34

        Is that challenging for you buying software then? When you think about buying new tools today, does it change how you think about buying tools? We're all huge buyers of tools internally as well. Like you say antiquated, Scott, I mean it nicely, but I'm sure like we all buy a huge amount of seat-based tools, like...

    11. SBScott Belsky40:49

        Well, we do and... but I think it's just-

    12. HSHarry Stebbings40:50

        You gonna change anything?

    13. SBScott Belsky40:52

        No. At so- at some point, probably sooner than later. Again, we're early in the days of AI, but at some point, um, when, when j- when there are more tools in the organization that do many different things, you know, I think that the idea of buying seats function by function will just evolve. You know, I, I... But you're right, it's a good question, Harry, like when is that really going to be material and how is it going to manifest?

    14. TCTomer Cohen41:16

        Yeah, Harry, I think, I think, I think your next podcast is with a bunch of CFOs and how they're making decisions-

    15. HSHarry Stebbings41:22

        (laughs) Exactly.

    16. TCTomer Cohen41:22

        ... because this is actually... this is where everybody's-

    17. HSHarry Stebbings41:25

        Come on, give me a break, okay. Like... (laughs)

    18. TCTomer Cohen41:27

        Yeah, this is gonna be the next productivity layer. And I think like for many companies, I, I don't think coming and saying I want to deploy this pro- you know, this program for 20,000 employees is going to work anymore. People are going to try and understand like, wait, wait, like what's more productive? How much output are we going to get? And what... Those are phenomenal questions we never used to ask a year and a half ago. But that's innovation for me. That's like an innovation catalyst.

    19. HSHarry Stebbings41:48

        Again, speak to many people and everyone says that, you know, AI development progression technology, stellar, 10 out of 10, we're doing a great job. Enterprise adoption, about a negative 4 out of 10, no good. Like, you know, I think there's a stat, 32% of European corporates do not know what Slack is. So that's concerning. How do you think about enterprise adoption keeping pace in any way with AI development?

    20. SBScott Belsky42:16

        Well, it's... I think it's probably good news that we're so early in the adoption curve because that means that there's a lot of, you know, a lot of potential.... ahead, ahead, ahead for, for, for these companies. It doesn't happen linearly. You know, that's the thing. I think that the adoption happens in moments where there's like a step function, you know, breakthrough or people just suddenly realize that, I mean, a lot of enterprise selling also happens because of who else is buying it. When I'm trying to work with customers who are thinking about, "Are we re- are we ready to change the way we do this?" It's much easier when they hear about that company that has already changed the way they do this and how, you know, they're saving money and a- a- and they're- and they're producing better output. It's like, "Oh, okay." Um, but, you know, I think that the part o- part of this is always about getting small teams in big companies to start to play with something, you know, and that's where I think enterprise sales can fall flat sometimes and you have to have design partnerships with early customers as opposed to traditional sells. You know, to go into early customers and say, "Hey, I'm not even trying to sell you on this. I just want a small team that can start to, you know, play with some of our technology." We have this, uh, tune your own model capabilities now for Firefly where brands can use some of their own IP and make a version of the model for them, and we've been going to some specific companies and saying, "Hey, like, be our partner and let's play with it." And then, of course, like that play becomes utility and then that utility becomes a reference point, um, and eventually, you know, when we're ready to sell it more broadly.

    21. GSGustav Söderström43:46

        So I agree with Scott both that it is early and, yeah, maybe it's our job to start using these things ourselves internally first and get them to work. But I also agree with Scott that I think it's gonna go much faster than moving to cloud. It wasn't really possible. I- I know we actually were on prem, companies that old. It wasn't possible to just also be on the cloud and try a little bit, it was incredibly hard. I don't think this will be that hard. I think you can try these things in parallel. So I think it's gonna be an S-curve, but I think it's gonna be a sharper S-curve once it happens.

    22. HSHarry Stebbings44:18

        Can I ask, we've spoken about kind of the seismic change needed in product and designer minds. If you were to sit down with young designers, young product people, you know, we have like 700,000 that listen to 20 Product, what would you say to them in terms of what they can/should do to equip themselves for the changes that we're seeing so they're best placed in their careers? Should they learn prompting world class (laughs) ? Should they learn... wh- what is that?

    23. TCTomer Cohen44:45

        Yeah, well, I- I think, oh, a- again to j- I'll start with a quick, um, uh, just a stat from this. We're basically tracking very closely at LinkedIn, like just looking at talent overall for the world. Like, the job is changing on you whether you like it or not. So, like, I think just one mindset for- for us, I think e- everybody here on the call and for every generation to come, like your job is gonna change much faster than you think. By the way, it might even change titles, the tasks will change, so it's really about how you morph for that. We looked at, like, the last five years. Obviously, AI has been p- playing a big role, but the skillset necessary to do a job changed by 25% in the last five to six years. By 2030, they're gonna change by at least 65%. So like, whether you want it or not, your job is changing, so the nece- the necessity to learn and to really start shaping your understanding of, uh, of the task you're trying to do, how the task is morphing is critical. For me, I would put it under the- the umbrella of growth mindset, just the ability to learn. Then there's- becomes like more of a question around, okay, so what's- how should I think about beyond being AI literate and be- be- beyond being, uh, focused in an AI first mindset, what should I do? I think there's two aspects that come to mind to me. One is like a more of a T-shaped perspective where I would assume you should have more broad skills than you had before. The ability to activate or to kind of bring to life broad skills, I would still, to that angle, I would still build expertise around a specific domain or specialty. We talked about disruption before between incumbents and startups. Usually we talk about tech. There's so many industries yet to be disrupted that, uh, nobody talks about, but those could be remarkable areas to focus on. And lastly, I think it's innately what makes us human. Things like soft skills, interpersonal dynamics, um, and imagination. Like just focusing on those 'cause that- those will stay. Those are not going anywhere.

    24. HSHarry Stebbings46:38

        Guys, hit me. Go granular. What do I actually do? Like, my job's changing.

    25. SBScott Belsky46:43

        I mean, listen, e- treating yourself as a business is something also everyone should always do, right? It's, um, you know, how you organize your, uh, your ideas and information, you know, how you track your own spreadsheet for your own budget. Like we're all, we all have our own business called ourselves that we- we also manage, you know, as- as part of- part of being a human being. So embrace these tools. You know, a lot of- lot of folks used, uh, you know, Google apps for themselves before they introduced them to their companies. You know, a lot of us used Evernote or Notion or other things and then we said, "Wait, I want to use this with my team." Um, so I- we have to have some, um, embracing. I, you know, novelty precedes utility. Um, and also, by the way, having, uh, playing with something gives you ideas of how you could maybe pilot it with something you do in your day job and then if you have a pilot, that's how you learn as to whether or not it's, you know, relevant for actual practice. So that's how we stay on the edge, and it's interesting. Like we all know people who are those early adopters who love playing with new tools and- and then we all know people who are like the total pragmatists who are just like, "I don't use it until I'm told I have to change my tool and you can rip this old tool out of my hands, you know, at the last minute that it's, uh, supported within the company." If you're listening and you're early in your career, you can be the person on your team that introduces new practices. Um, that's your advantage in a company, especially working with a lot of people that have been around a lot longer, is you can be the one who, uh, who pioneers new practices.

    26. GSGustav Söderström48:17

        Yeah, and I think in general, like the, when we think about big shifts is that's actually when the opportunity arises. To Scott's point, a lot of people are slow to change. So the- it usually is the case that one of the advantages of coming straight out of school is what the disadvantage is you have no experience, but the advantage is usually you have the latest and greatest knowledge. Whereas someone who worked in a company for 10 years does- they're using the old tools. So I- I think that's still true.... uh, you can have that advantage. But even if you're in a company, the only advice is to educate yourself. And as we said before, in this instance, it's actually simpler than ever. You don't have to ask for permission to learn how to prompt and how to use these things. Uh, you know, education is... you know, on, on the internet is basically free now. It's very cheap. I, I think it's easier than ever, more democratic than ever. Um, and to, to, to Tomer's point as well, it's just gonna keep changing. There, there's actually... the risk is if you're in a company that you don't develop fast enough. I think if you're, if you're fresh, um, you're gonna have those later skills.

    27. HSHarry Stebbings49:16

        Perfect. Listen,

16. 49:17 – 55:54

    Quick-Fire Round

    1. HSHarry Stebbings49:17

       I want to dive into a quickfire. So I say a short statement, you give me your immediate thoughts. Scott, I like your newsletter. Uh, what's your biggest lesson from running applied to product?

    2. SBScott Belsky49:26

       Oh, from running? Um, wow, I-

    3. HSHarry Stebbings49:29

       Did you see his li- eyes light up there? That was like, ooh.

    4. SBScott Belsky49:32

       I love running. No, I think that-

    5. GSGustav Söderström49:33

       I was running.

    6. HSHarry Stebbings49:33

       Otherwise, it's whoop. I saw the whoop as well, Scott.

    7. SBScott Belsky49:36

       I mean, for me, there are a lot of lessons that I've extracted from running. I'll tell you one of them, which is when I'm running, I often have ideas and I'm obsessed with capturing ideas and I'm always worried about forgetting ideas. But when you're running, I just, like, force myself to keep running because I don't want to give myself an excuse to stop running to capture an idea. And that period that I am forcing myself to keep thinking about this idea as opposed to writing it down, it always gets better. And I think about that now in my, like, everyday work. I'm so impulsive, I'm always, like, trying to be decisive. I feel like that's one of the best practices of being an executive is decisiveness. But if you can, like, sit with something a little longer than comfortable, oftentimes you end up with a better approach to how to say it, you know, to how to give the feedback to the person, you know, how to, like, solve that problem with the product. And so I have this, like, mantra of wait for it that, uh, I've learned from running.

    8. HSHarry Stebbings50:31

       I have intense paranoia when I have that idea and I just have to remember it for like a three-hour fucking run, right?

    9. SBScott Belsky50:37

       (laughs)

    10. HSHarry Stebbings50:37

        Jesus Christ.

    11. SBScott Belsky50:38

        Well, I'm not running that long, so that's okay. (laughs)

    12. GSGustav Söderström50:40

        I totally agree that the cadence of running actually, I think is almost like a process. You're forced to iterate the idea. It's, it's almost like in a, uh, uh, you know, in a trance. I, I totally agree. Having to be forced to be s- thinking of something and you have this clock going tick, tick, tick for an hour, it certainly helps the idea.

    13. HSHarry Stebbings50:58

        Yeah. No, I, I agree with you. Tomer, uh, tell me, if you could change one thing about the LinkedIn product today, what would you change that you haven't already?

    14. SBScott Belsky51:06

        (laughs)

    15. TCTomer Cohen51:06

        Yeah, I think, you know... Well, uh, what, what's, what's interesting right now, we talked about the, the idea of UI and complexity. There's this great law that I'm... k- uh, I, I really like it was really helpful early in my career to think about how to build products, which is the conservation of co- complexity. The idea that every product has an inherent amount of complexity and the i- and the idea is can you solve it in the product development side or do you actually put it on the user to solve by themselves? And I think we're in this phase right now where there's so many use cases and audiences that people come to the LinkedIn kind of main app for, and that just had inherent complexity because there was so many... you know, you come in the morning, you're trying to see what's happening in the world, later in the afternoon you interview somebody, you want to check out their profile, then, you know, your boss, uh, upset at you later in the day and you want to see what's out there for you. This is all three use cases in the same, like, 24 hours. I think now with AI, and I would claim probably this would be more generalized statement, the more complex your product, the more impact AI could have in terms of simplifying it for your user base. So one thing that we're already underway for us is truly take away the complexity and solve it with the idea of bringing in more of this new, uh, large language models, and in general, this new brain that can now sit on top of it. So instead of adding more features, it's the same experience, you just more stores your need. So more TBD on that one.

    16. HSHarry Stebbings52:28

        I think you should call it a name like Einstein. I love this. Don't you love Einstein as a name for Salesforce?

    17. GSGustav Söderström52:34

        (laughs)

    18. HSHarry Stebbings52:34

        I think this is brilliant. Uh-

    19. GSGustav Söderström52:35

        We ca- we call it, uh, Fighting the Entropy in Spotify. (laughs)

    20. HSHarry Stebbings52:41

        Love it. Uh-

    21. SBScott Belsky52:41

        Love it.

    22. HSHarry Stebbings52:41

        Uh, tell me, uh, Scott, what have you changed your mind on in the last 12 months?

    23. SBScott Belsky52:46

        The, the need for centralization, you know, and where things should and shouldn't be centralized. Like, I always go back and forth honestly on this. You know, there have been periods of time where I felt like design needed to be in different organizations because people had to prioritize and properly resource design for their business. And then recently, I re-centralized design in one organization because our strategy, you know, required it. And I guess one of the things I'm learning is that sometimes, you know, you make a completely opposite decision at different times in the same business because the playbook is different. And we typically don't think that way. We like... when we make a huge decision and it's like, this is the way it's gonna be, we just stick to it because we almost think like it's first principles, but you just have to be willing to discard the playbook consistently.

    24. HSHarry Stebbings53:34

        Can product leaders... uh, okay, let's go for you, Tomer. Can product leaders not be trained in AI today?

    25. SBScott Belsky53:40

        (laughs)

    26. HSHarry Stebbings53:40

        Can you be a CPO and not have really spent hard yards in AI?

    27. TCTomer Cohen53:45

        I would not hire you to my org if you're...

    28. SBScott Belsky53:48

        (laughs)

    29. TCTomer Cohen53:48

        ... if you don't have the willingness and the aptitude, uh, to go deep. Again, you don't ha- uh, I don't need you to be the engineer working on the model, but I need you to understand, one, the objective you're trying to build and how to build it. Otherwise, what are you here h- for? And two, all the underlying principles that come with it, the idea of it's not being deterministic, so how do you build the knobs so elegantly that the experience is, you know, great in any shape or form? Two, how do you think of data collection in a way that's responsible but really fuels what you're trying to build? And three, it's that velocity we just talked about. If you don't have the velocity of learning, I, I don't think you'll last.

    30. HSHarry Stebbings54:27

        Gustav, final one for you. What are you most excited about when you look at AI in the coming years and what it can do for everyone, but also for Spotify? Just what excites you most?

Episode duration: 55:54