Skip to content
Lex Fridman PodcastLex Fridman Podcast

Gustav Soderstrom: Spotify | Lex Fridman Podcast #29

Lex Fridman and Gustav Soderstrom on spotify’s Gustav Soderström on Music, AI, Creation, and the Future.

Lex FridmanhostGustav Soderstromguest
Jul 29, 20191h 47mWatch on YouTube ↗

EVERY SPOKEN WORD

  1. 0:00 – 3:00

    Favorite song debate & True Romance as a lifelong musical anchor

    1. LF

      The following is a conversation with Gustav Soderström. He's the chief research and development officer at Spotify, leading their product, design, data, technology, and engineering teams. As I've said before in my research and in life in general, I love music, listening to it and creating it, and using technology, especially personalization through machine learning to enrich the music discovery and listening experience. That is what Spotify has been doing for years, continually innovating, defining how we experience music as a society in the digital age. That's what Gustav and I talk about among many other topics, including our shared appreciation of the movie True Romance, in my view, one of the great movies of all time. This is the Artificial Intelligence podcast. If you enjoy it, subscribe on YouTube, give it five stars on iTunes, support it on Patreon, or simply connect with me on Twitter @lexfridman, spelled F-R-I-D-M-A-N. And now here's my conversation with Gustav Soderström. Spotify has over 50 million songs in its catalog, so let me ask the all-important question, I feel like you're the right person to ask, what is the definitive greatest song of all time?

    2. GS

      (laughs) It varies for me, personally.

    3. LF

      So you can't speak definitively for everyone?

    4. GS

      (laughs) I wouldn't believe very much in machine learning if I did, right?

    5. LF

      Okay.

    6. GS

      Because it meant everyone had the same taste.

    7. LF

      So for you, what is... you have to pick, what is the song?

    8. GS

      All right, so it's, it's pretty easy for me. There is this song called "You're So Cool" by Hans Zimmer, uh, soundtrack to True Romance.

    9. LF

      Ah.

    10. GS

      It was a movie that made a big impression on me and it's kind of been following me through my life. Actually had it play at my wedding. I sat with the organist and helped him play it on an organ, which was a pretty, pretty interesting experience. It's, uh...

    11. LF

      That is probably my, uh, I would say top three movie of all time. Yeah, it's just an incredible movie.

    12. GS

      Yeah, and, and it came out during my formative years and, uh, as I've discovered in music, you, you shape your music taste during those years, so it definitely affected me quite a bit.

    13. LF

      Did it affect you in any other kind of way?

    14. GS

      Well, the movie itself affected me back then, it was a big part of culture. I didn't really adopt any characters from the movie-

    15. LF

      (laughs)

    16. GS

      ... but it was a, it was a great story of love, some fantastic actors and, and, you know, really I didn't even know who Hans Zimmer was at the time, but fantastic music. And so, um, that song has followed me, and the movie actually has followed me throughout my life.

    17. LF

      That was Quentin Tarantino actually, I think, uh, director, directed or produced that, or... So it's not Stairway to Heaven or Bohemian Rhapsody, it's, uh...

    18. GS

      Tho- those are, those are great. They're not my personal favorites, but, uh-

    19. LF

      But they're out there.

    20. GS

      I've realized that people have different tastes and that's, uh, that's a big part of, of what we do.

  2. 3:00 – 4:28

    Why music exists: escapism, focus, and ‘tuning’ the brain

    1. LF

      Well, for me, I would have to stick with Stairway to Heaven. So 35,000 years ago, I looked this up on Wikipedia, flute-like instruments started being used in caves as part of hunting rituals, in primitive cultural gatherings, things like that. This is the birth of music. Since then, we had a, a few folks, Beethoven, Elvis, Beatles, Justin Bieber of course, Drake. So in your view, let's start like high-level philosophical, what is the purpose of music on this planet of ours?

    2. GS

      I think music has many different purposes. I think there is, um, there is certainly a big purpose which is the same as much of entertainment, which is escapism and to be able to live in some sort of other mental state for a while. But I also think you have the, the opposite of escaping, which is to help you focus on something you are actually doing. And so I think people use music as a tool to, to tune the brain to the activities that they are actually doing and, um, it's kind of like, in one sense maybe it's the roost signal. If you, if you think about the brain as neural networks, it's maybe the most efficient hack we can do to actually actively tune it into some state that you want to be. And you can do it in other ways, you can tell stories to put people in a certain mood, but music is probably very effective to get you through a certain mood very fast, I think.

  3. 4:28 – 7:06

    Private vs social listening: intimacy, guilty pleasures, and limited sharing

    1. LF

      You know, there's a, there's a social component historically to music, where people listen to music together. I was just thinking about this, that to me, and you mentioned machine learning, but to me personally, music is a really private thing. Like, I'm speaking for myself, I listen to music... like almost nobody knows the kind of things I have in my library except people who are really close to me, and they really only know a certain percentage. There's like some weird stuff that I'm almost probably embarrassed b- by, right?

    2. GS

      It's called the guilty pleasures, right? Everyone has them.

    3. LF

      (laughs) The guilty pleasures, yeah. Hopefully they're not too bad. But, uh, so it's a, I- for me it's personal. Do you think of music as something that's social or as something that's personal? Or does it vary?

    4. GS

      So I think it's the same, it's the same answer, that you use it for, for both. We- we've thought a lot about this during these 10 years at Spotify obviously. In one sense, as you said, music is incredibly social. You go to concerts and so forth. On the other hand, it is your, your escape, and everyone has these things that are very personal to them. So what we've found is that when it comes to, to, um... M- most people claim that they have a friend or two that they are heavily inspired by and that they listen to, so I actually think music is very social, but in a smaller group setting. It's an, it's an intimate form of...... of, um, it's an intimate relationship. It's not something that you necessarily share broadly. Now at concerts, you can argue you do. But then you've gathered a lot of people that you have something in common with. I think this broadcast sharing of music, um, is something we tried, uh, on social networks and so forth. But it turns out that people aren't super interested in just what their friends listen to.

    5. LF

      Mm-hmm.

    6. GS

      Um, they're interested in understanding if they have something in common, perhaps, with a friend, but not, you know, not just as, as information.

    7. LF

      Right. That's, that, that's really interesting. I, I was just thinking of it this morning listening to Spotify. I really have a pretty intimate relationship with Spotify, with my playlists, right? I've had them for many years now and they've grown with me together. There's, there's an intimate, uh, relationship you have with a library of music that you've developed. And we'll talk about different ways we can play with that. Can you, uh, do the impossible task and try to

  4. 7:06 – 10:00

    From live performance to recorded constraints: how formats shape music

    1. LF

      give a history of music listening from your perspective, from before the internet and after the internet. And just kinda everything leading up to streaming on Spotify and so on.

    2. GS

      I'll try. It could be a 100-year podcast, but... (laughs)

    3. LF

      Yeah.

    4. GS

      I'll, I'll try to do a brief version. There are some things that, that I think are very interesting during the history of music, which is that before recorded music, you, to be able to enjoy music, you actually had to be where the music was produced because you couldn't, you couldn't record it and time shift it, right? Cre- creation and consumption had to happen at the same time, basically concerts. And so you either had to get to the nearest village to listen to music. And, uh, while that was cumbersome and it severely limited the distribution of music, it also had some different qualities, which was that the creator could always interact with the audience. It was always live. And also there was no time cap on the music. So I think it's not a coincidence that these early classical works, they're much longer than the three minutes. The three minutes came in as a restriction of the first wax disc that could only contain a three-minute sing- song on one side, right? So actually, the re- recorded music severely limited the, or put constraints... I won't say limit. I mean, constraints are often good, but it put very hard constraints on the music format. So you kind of said like instead of doing these, this opus like many, you know, tens of minutes or something, now you get three and a half minutes because then you're out of wax on this disc.

    5. LF

      Right.

    6. GS

      But in return, you get am- amazing distribution. Your reach will, will widen, right?

    7. LF

      Just on that point real quick. Without the mass-scale distribution, there's a scarcity component where you kind of look forward to it. We had that, uh, it's like the Netflix versus HBO Game of Thrones. You, like, wait for the event because you can't really listen to it. So you, like, look forward to it and then it's, uh, you derive perhaps more pleasure because it's more rare for you to listen to a particular piece. Do you think there's value to that scarcity?

    8. GS

      Yeah, I think that, that is definitely a thing. And there's always this component of if you have something in infinite amounts, will you value it as much? Probably not. Humanity is always seeking some... It's, it's relative, so you're always seeking something you didn't have. And when you have it, you don't appreciate it as much. So I think that's probably true. But I think that's why concerts exist, so you can actually have both.

    9. LF

      Right.

    10. GS

      But I think net, if you couldn't listen to music in your car driving, that, that, that'd be worse. That cost would be bigger than the benefit of, of the anticipation I think that you would have.

    11. LF

      So, uh, yeah, it started with live concerts, then, uh, it's being able to, you know, the, the phonograph invented, right? You started to be able to, uh, record music.

  5. 10:00 – 11:37

    Radio, shared culture, and the tradeoff with personalization

    1. GS

      Exactly. So then, then you got this massive distribution that, that made it possible to create two things. I think first of all, cultural phenomenons, they probably need distribution to be able to happen. But it also opened access to, you know, for, for a new kind of artist. So you started to have these phenomenons like Beatles and Elvis and so forth that were really a function of distribution, I think. Obviously of, of talent and innovation, but there was also technical component. And of course, the next big innovation to come along was, was radio, broadcast radio. And I think, uh, radio is interesting because it started not as a music medium, it started as, as an information medium for, for news. And then radio needed to find something to fill the time with so that they could honestly play more ads and make more money.

    2. LF

      Mm-hmm.

    3. GS

      And music was free, so, so then you had this massive distribution where you could program to people. I think those things, that ecosystem is what, what created, um, the ability for, for, for hits. But it was also a very broadcast medium. So you would tend to get these massive, massive hits, but maybe not such a long tail.

    4. LF

      In terms of choice of, uh, everybody listens to the same stuff?

    5. GS

      Yeah. And, and as you said, I think there are some social benefits to that. Uh, I think, for example, there is, there's a high statistical chance that if I talk about the latest episode of Game of Thrones, we have something to talk about.

    6. LF

      Yeah.

    7. GS

      Just statistically. In the age of individual choice, maybe some of that goes away. So I, I, uh, I do see the value of, like, you know, shared cultural components, but I also obviously love personalization.

  6. 11:37 – 19:26

    Digitization, piracy, and Spotify’s origin story: access beats ownership

    1. LF

      And, uh, so l- let's catch this up to the internet. So maybe Napster... Well, first of all, there's, like, MP3s.

    2. GS

      Exactly.

    3. LF

      There was, like, tapes, CDs.

    4. GS

      There was a digitalization of music with a CD really. It was physical distribution, but the music became digital.

    5. LF

      Yeah.

    6. GS

      And so they were files, but basically boxed software. It's an use a software analogy. And then you could start downloading these files. And I think there are two interesting things that happened back to music used to be longer before it was constrained by the distribution medium.I don't think that was a coincidence. And then, really the only music genre to have developed mostly after music was a file again on the internet is EDM. And EDM is often much longer than the traditional music. I think, I think it's interesting to think about the fact that music is no longer constrained in minutes per song or something. It's, it's a, it's a legacy of, of an old distribution technology. And you see some of this new music that, that breaks the format. Not so much as I would have expected actually by now, but, but it still happens.

    7. LF

      So, first of all, I don't really know what EDM is.

    8. GS

      Yeah. Electronic dance music.

    9. LF

      Yeah.

    10. GS

      Right? You could say Avicii was one of the biggest in this genre.

    11. LF

      So, the main constraint is of time?

    12. GS

      Of time, right.

    13. LF

      Something that, like, three, four, five minute song-

    14. GS

      So, you could have songs that were eight minutes, 10 minutes, and so forth, because the, you know, it, it started as a digital product that you downloaded, so you didn't have this, this constraint anymore. Uh, so I, I think it's something really interesting that I don't think has fully happened yet. K- we're kind of jumping ahead a little bit to where we are, but...

    15. LF

      Mm-hmm.

    16. GS

      ... I think there is, there is tons of form and innovation in music that should happen now, uh, that couldn't happen when you needed to really adhere to the distribution constraints. If you didn't adhere to that, you would get no distribution. So, so Bjork, for example, Icelandic artist, she made a full iPad app as an album. That's very expensive, you know, uh, even though the app store has great distribution, she gets nowhere near the distribution versus staying within the three-minute format. Uh, so I think now that music is fully digital inside these streaming services, there is, there is the opportunity to change the format again, and allow creators to be much more creative without limiting their, their distribution ability.

    17. LF

      That's interesting that... You're right, it's surprising that we don't see that taken advantage more often. It's almost like the constraints of the distribution from the '50s and '60s have molded the culture to where we want the five, three to five minute song than anything else. Not just... So we want the song as consumers and as artists. Like I, 'cause I write a lot of music, and I never even thought about writing something longer than 10 minutes. That's, it's, uh, it's really interesting that those constraints put you-

    18. GS

      Because all your training data has been three-and-a-half minute songs, right? (laughs)

    19. LF

      Right. (laughs) It's right. Okay, so yeah, digitization of data led to then MP3s.

    20. GS

      Yeah, so I think you had this file then that was distributed physically, but then you had the components of digital distribution, and then the internet happened, and there was this vacuum where you had a format that could be digitally shipped, but there was no business model. And then, all these pirate networks happened. Uh, Napster and, and then Pirate, in Sweden, Pirate Bay, which was one of the biggest. And it, you know, I think from a consumer point of view, which, which, which kind of leads up to the inception of, of Spotify, from a consuming point of view, consumers for the first time had this access model to music where they could, without kind of any marginal cost, they could, they could, um, try different tracks. You could use music in, in new ways. There was no marginal cost. And that was a fantastic consumer experience. To have access to all the music ever made, I think was fantastic, but it was also horrible for artists because there was no business model around it, so they didn't make any money. So, the, the user need, uh, almost drove the user interface bef- before there was a business model. And then there were these download stores, uh, that allowed you to download files, uh, which was a solution, but it didn't solve the access problem. There was still a marginal cost of 99 cents to try one more track.

    21. LF

      Mm-hmm.

    22. GS

      And I think that that heavily limits how you listen to music. The, the example I always give is, uh, you know, in Spotify, a huge amount of people listen to music while they sleep, while they go to sleep and while they sleep. If that costed you 99 cents per three minutes, you probably wouldn't do that. (laughs)

    23. LF

      (laughs)

    24. GS

      And you would be much less adventurous if there was a real dollar cost to exploring music. So, the access model is interesting in that it changes your music behavior. You can be... You can take much more risk 'cause there's no marginal cost to it.

    25. LF

      Maybe, let me linger on piracy for a second 'cause I, I find, uh, especially coming from Russia, piracy is something that's very interesting. To me, uh, not me of course ever, but, uh-

    26. GS

      (laughs) Of course.

    27. LF

      ... I have, I have friends who have partook in, uh, piracy of music, software, TV shows, sporting events. And usually, to me, what that shows is not that they're... They can actually pay the money and they're not trying to save money. They're choosing the best experience. So, what to me piracy shows is a business opportunity in all these domains, and, uh, that's where I, I think you're right Spotify stepped in, is basically piracy was, is an experience. You can explore, w- find music you like, and actually the interface of piracy isn't, is horrible because it's f- I mean, it's-

    28. GS

      Bad metadata.

    29. LF

      Yeah. Bad metadata, there's a lot-

    30. GS

      You know, long download times, all kinds of stuff.

  7. 19:26 – 25:55

    Competing with ‘free’: latency as the killer feature and the early tech stack

    1. LF

      So how do you compete with free? 'Cause that's an interesting world of the internet, where most people don't like to pay for things. So Spotify steps in and tries to, yes, compete with free. How do you do it?

    2. GS

      So I think two things. One is people are starting to pay for things on the internet. I think one way to think about it was that advertising was the first business model, because no one would put their credit card on internet. Transactional with Amazon was the second, and maybe subscription is the third. And if you look offline, subscription is the biggest of those. So that may still happen. I think people are starting to pay, but definitely back then, we needed to compete with free, and the first thing you need to do is obviously to lower the price to free. And then you need to be better somehow, and the way that Spotify was better was on the user experience, on the, on the actual performance, the latency of, uh, you know, even if, even if you had high bandwidth broadband, it would still take you 30 seconds to a minute to download one of these tracks. So the Spotify experience of starting within the perceptual limit of immediacy, about 250 milliseconds, meant that the, the, the whole trick was it felt as if you had downloaded all of Pirate Bay. It was on your hard drive. It was that fast, even though it wasn't, and it was still free. But somehow, you were actually still being a legal citizen. That, that was the trick that Spotify managed to, to pull off.

    3. LF

      So yeah, I've actually heard you, uh, say this or write this, and I was surprised that I wasn't aware of it because I just took it for granted. You know, whenever an awesome thing comes along, you're just like, "Oh, of course it has to be this way." That- that's exactly right, that it felt like the entire world's library is at my fingertips because of that, of that latency being reduced. What, what was the technical challenge in reducing the latency?

    4. GS

      So there was a, a group of really, really talented engineers, uh, one of them called Ludvig Strigeus. He wrote the... Actually from Gothenburg.

    5. LF

      Mm-hmm.

    6. GS

      He wrote the initial, um, uh, the uTorrent client, which is kind of an interesting backstory to Spotify, you know, that, uh, we have one of the top developers from, uh, from BitTorrent clients as well. So he wrote uTorrent, the world's smallest Bit Torrent client, and then, um, he, um, he was acquired very early by Daniel and Martin, who founded Spotify. And they actually sold the uTorrent client to Bit Torrent, but kept Ludvig. So Spotify had a lot of, uh, experience within peer-to-peer networking. So the original innovation was an, was a distribution innovation, where Spotify built an end-to-end media distribution system. Up until only a few years ago, we actually hosted all the music ourselves. So we had both the server side and the client, and that meant that we could do things such as having a peer-to-peer solution to use local caching, uh, on the client side, because back then, the world was mostly desktop.

    7. LF

      Mm-hmm.

    8. GS

      But we could also do things like, um, hack the TCP protocols, things like Nagle's algorithm for kind of exponential back-off or ramp-up, and just go full throttle and optimize for latency at the cost of bandwidth. And, uh, all, all of this end-to-end control meant that we could do an experience that felt like a, a step change. These days, we actually are on, on, um, GCP. We don't host our own stuff and, and everyone is really fast these days. So that was the initial competitive advantage, but then obviously you have to move on over time.

    9. LF

      And that was, uh, that was over 10 years ago, right?

    10. GS

      That was in 2008, the product was launched in Sweden. It was in a beta, I think 2007.

    11. LF

      And it was on the desktop, right? So-

    12. GS

      It was desktop only.

    13. LF

      There's no phone then.

    14. GS

      There was no phone. The iPhone came out in 2008, but the App Store came out one year later, I think. So the writing was on the wall, but there was no phone yet.

    15. LF

      You've mentioned that people would use Spotify to discover the songs they like and then they would torrent those songs to- so they can copy it to their phone. Just hilarious.

    16. GS

      Exactly.

    17. LF

      Uh, uh, uh, not torrent, pirate. Uh, it- seriously, piracy does seem to be (laughs) a- a- like a, a good guide for business models. Video content, as far as I know, Spotify doesn't have video content.

    18. GS

      Well, we do have music videos-

    19. LF

      Mm-hmm.

    20. GS

      ... and we do have videos on the, on the service, but the way we think about ourselves is that we're an, we're an audio service. And, uh, we think that i- if you look at the amount of time that people spend on audio, it's actually very similar to the amount of time that s- people spend on video. So the opportunity should be equally big, but today it's not at all valued. Video is valued much higher. So we, we think it's basically completely undervalued. So we think of ourselves as an audio service, but within that audio service, I think video can make a lot of sense. I think for-... when you're, when you're discovering an artist, you probably do want to see them and understand who they are, to understand their identity. You won't see the video every time, no. 90% of the time, the phone is going to be in your pocket. For podcasters, you use video. I think that can make a ton of sense. So we do have video, but we're an audio service, where ... Think of it as, we call it internally background-able video, video that is helpful, but isn't, isn't the driver of the narrative.

    21. LF

      I think, also, if we look at, uh, YouTube, the way people ... There's quite a few folks who listen to music on YouTube.

    22. GS

      For sure.

    23. LF

      So in some sense, YouTube is a bit of a competitor, uh, to, uh, to Spotify, which is very strange to me that people use YouTube to listen to music. They play essentially the music videos, right? But don't watch the videos, and put it in their pocket.

    24. GS

      Well, I think, I think it's similar to, to, uh, uh, what ... Strangely, maybe it's similar to what we were for the piracy networks.

    25. LF

      Right.

    26. GS

      Where YouTube, for hi- historical reasons, have a lot of music videos. So you use ... People use YouTube for a lot of the discovery part of the process, I think.

    27. LF

      Right.

    28. GS

      But then it's not a really good sort of, quote-unquote, "MP3 player" because it doesn't even background. Then you have to keep the app in the foreground, so, so the consum- ... It's not a good consumption tool, but it's a decently good discovery too-. I mean, I think YouTube is a fantastic product and I use it for all kinds of purposes. Educational, so-

    29. LF

      That's true. If I were to admit something, I do use YouTube a little bit for the discov- to assist in the discovery process of songs, and then if I like it, I'll, I'll add it to Spotify. (laughs) I'll add it to-

    30. GS

      But that's okay. That's okay with us.

  8. 25:55 – 31:07

    Scaling adoption: invites, ‘legal fast piracy,’ and the psychology of ownership

    1. GS

      No, that's fine.

    2. LF

      So, uh, the- this kind of incredible ... You look at Napster, you look the early days of Spotify, how do you ... One fascinating point is, how do you grow a user base? So you're there in S- in Sweden, you have an idea. I saw the initial sketches, they look terrible.

    3. GS

      (laughs)

    4. LF

      Uh, well, how do you grow user base from, uh, from a few folks to millions?

    5. GS

      I think, uh, there are a bunch of tactical answers. So first of all, I think you need a great product. I don't think you take a bad product and, and, uh, market it to be successful. So you need a great product.

    6. LF

      But, so- sorry to interrupt, but it's a totally new way to listen to music too, so it's not just ... Did people realize immediately that Spotify is a great product?

    7. GS

      I think they did. So back to the point of piracy, it was a totally new way to listen to music illegally, but people had been used to the access model in Sweden-

    8. LF

      Right.

    9. GS

      ... and the rest of the world for a long time through piracy. So one way to think about Spotify, it was just legal and fast piracy.

    10. LF

      Yeah.

    11. GS

      And so people have been using it for a long time. So they weren't alien to it. They didn't really understand how it could be illegal, because it w- seemed too fast and too good to be true.

    12. LF

      Yeah.

    13. GS

      Which I think is a great product proposition, if you can be too good to be true. Uh, but wha- what I saw again and again was people showing each other, clicking the song, showing how fast it started and saying like, "Can you believe this?" You know. So I really think it was about speed. Then we also had, um, an invite produ- program that was, that was really meant for scaling, because we hosted our own service. We needed to control scaling. But that built a lot of expectation and, uh, I don't want to say hype, because I ... hype implies that it was, that it wasn't true.

    14. LF

      No.

    15. GS

      Uh, expectations.

    16. LF

      Excitement.

    17. GS

      Excitement around-

    18. LF

      Yeah.

    19. GS

      ... the product, and we've replicated that when we launched in the, in the US. We also built up an invite-only program first. There are lots of tactics, but I think you need, um, you need a great product that solves some problem, and, and ba- basically, the key innovation, there was technology, but on a meta level, the innovation was really the access model versus the ownership model, and that was tricky. A lot of people said that they, I mean, they, they wanted to own their music. They would never kind of rent it or borrow it. But I think the fact that we had a free tier, which meant that you get to keep this music for life as well, helped quite a lot.

    20. LF

      So this is an interesting psychological point that maybe you can speak to. It was a big shift for me. Like, uh, like I had to ... It's, it's almost like, uh, I had to go to therapy for this. Is, uh ... I think I would describe my early listening experience, and I think a lot of my friends do, as basically hoarding music. As you're like slowly, one song by one song, or maybe albums, gathering a collection of music that you love.

    21. GS

      Mm-hmm.

    22. LF

      And you own it. It's like of- especially with CDs or tape, you like physically had it.

    23. GS

      Exactly.

    24. LF

      And, and with Spotify, what I had to come to grips with, and it was kind of liberating actually, is to throw away all the music.

    25. GS

      I've had this therapy session-

    26. LF

      Yeah. (laughs)

    27. GS

      ... with lots of, (laughs) with lots of people. And I think the mental trick is ... So actually we s- we've seen the user data when Spotify started, a lot of people did the exact same thing. They started hoarding as if the music would disappear.

    28. LF

      Right.

    29. GS

      Right? Almost the equivalent of downloading. And so, you know, we, we had these playlists that had limits of like a few 100,000 tracks and we figured no one will ever. Like, well they do. (laughs)

    30. LF

      (laughs)

  9. 31:07 – 34:05

    Playlists as a ‘programming language’: 3B playlists, retention, and semantics

    1. LF

      Whereas Spotify, the more liberating kind of approach is I was just enjoying... Of course I listened to Stairway to Heaven over and over, but I, because of the extra variety, I don't get as sick of them. There's an interesting statistic I saw th- so Spotify has, maybe you can correct me, but over 50 million songs, tracks, and over three billion playlists. So...

    2. GS

      Yes.

    3. LF

      Fifty million songs and three billion playlists. 60 times more playlists than songs. (laughs) Wh- what do you make of that?

    4. GS

      Yeah, so the way I think about it is that from a, um, from a statistician or machine learning point of view, you have all these, um, if you want to think about reinforcement learning, but you have this state space of all the tracks and you can take different journeys through this, through this world. And, um, these, I think of these as like people helping themselves and each other, creating interesting vectors through this space of tracks.

    5. LF

      Mm-hmm.

    6. GS

      And then it's not so surprising that across, you know, many tens of millions of kind of atomic units, there will be billions of paths that make sense, and we're probably pretty quite far away from having found all of them. So kind of our job now is users... When, when Spotify started, it was really a search box that was for the time pretty powerful, and then, uh, I like to refer to it as this programming language called playlisting-

    7. LF

      (laughs)

    8. GS

      ... where if you, as you probably were pretty good at music, you knew your new releases, you knew your back catalog, you knew your Stairway to Heaven, you could create a soundtrack for yourself using this playlisting tool that's like meta programming language for music to soundtrack your life. And people who are good at music, it's back to how do you scale the product. For people who are good at music, that was ac- actually enough. If you had the catalog and a good search tool and you can create your own sessions, you could create really good... A soundtrack for your entire life. Probably perfectly personalized because you did it yourself. But the problem was most people, many people aren't that good at music, they just can't spend the time. Even if you're very good at music, it's going to be hard to, to keep up. So what we did to try to scale this was to essentially try to build, you can think of them as agents that this, this friend that some people had that helped them navigate this music catalog, that's what we're trying to do for you.

    9. LF

      But also there is something like 200 million active users on Spotify.

    10. GS

      Yes.

    11. LF

      So there... Okay, so from the machine learning perspective, you have these 200 million people plus, uh, that are creating... It's, it's really interesting to think of, uh, playlists as, um... I mean, I don't know if you meant it that way, but it's almost like a programming language. It's, um, or at least a trace of, um, exploration of those individual agents, uh, th- the listeners, and you

  10. 34:05 – 41:01

    Recommender systems: collaborative filtering, embeddings, and Echo Nest fusion

    1. LF

      have all this new tracks coming in. So it's a fascinating space that, uh, is ripe for machine learning. So th- is there mo- is there, is it poss- how can playlists be used as data in terms of, uh, machine learning and, and to s- to help Spotify organize the music?

    2. GS

      So we found in our data, not surprising that people who playlisted lots, they retained much better. They had a great experience. And so our first attempt was to playlist for users. And so we acquired this company called Tunigo of editors and professional playlisters and kind of leveraged the maximum of, of, um, human intelligence to help, to help, uh, build li- li- kind of these vectors through the track space-

    3. LF

      Mm-hmm.

    4. GS

      ... for, for people. Uh, and that, that broadened the product. Then the, the obvious next... And, and we, you know, we used statistical means where they could see wha- when they created a playlist, how did that playlist perform, you know? They could see skips of the songs, they could see how the songs perform, and they manually iterated the playlist to maximize performance for a large group of people. But there were never enough editors to playlist for you personally. So the promise of machine learning was to go from kind of group personalization using editors and, and tools and ta- statistics to individualization. And then what's so interesting about the, the three billion playlists we have is we... And the, the truth is we lucked out. This was not a priority strategy as is often the case. It looks really smart in hindsight, but it's, it was dumb luck. Uh, we looked at these playlists and we had some people in the company, um, a person named Erik Bernhardsson, who was really good at machine learning already back in, in, back then in like 2007, 2008. Uh, back then it was mostly collaborative filtering and so forth. But we realized that what, what this is, is people are grouping tracks for themselves that have some semantic meaning to them, and then they actually label it with a playlist name as well. So in a sense, people were grouping tracks along semantic dimensions and labeling them. And so could you, could you, uh, use that information to find that, that latent embedding? And so we started playing around with collaborative filtering.And, uh, we saw tremendous success with it, basically trying to extract some of these, uh, some of these dimensions. And, and if you think about it, it's not surprising at all. It'd be quite surprising if playlists were actually random, if they had no semantic meaning.

    5. LF

      Okay.

    6. GS

      For, for most people, they group these tracks for some reason. So we just happened to cross this incredible data set where people are taking- taken these tens of millions of tracks and grouped them along different semantic vectors.

    7. LF

      Uh, and the semantics being outside the individual users, so it's some kind of universal... there's a universal embedding that holds across people on this earth.

    8. GS

      Yes. I, I do think that, um, the embeddings you find are gonna be reflective of the people who playlisted. So if, if you have a lot of indie lovers who playlist, your, your embedding is gonna perform better there. But what we found was that, yes, uh, there were these, these, uh, latent similarities. They were very powerful. And we, we had the... it was interesting because I think that the people who playlisted the most initially were the so-called music aficionados who, who were really into music, and they often had a certain... their taste was of- of- of often cert- geared towards a certain type of music. And so what surprised us, if, if you look at the problem from the outside, you might expect that the algorithms would start performing best with mainstreamers first, because it somehow feels like an easier problem to solve mainstream taste than really particular taste. It was the complete opposite for us.

    9. LF

      Hmm.

    10. GS

      The recommendations performed fantastically for people who saw themselves as having very unique taste. That's probably because all of them playlisted (laughs) and they didn't perform so well for mainstreamers. They actually thought they were a bit too particular and unorthodox. So we had the complete opposite of what we expected, success within the hardest problem first, and then had to try to scale to more mainstream recommendations.

    11. LF

      So, uh, you've also acquired Echo Nest, that analyzes song data. So, in your view, maybe you can talk about, so what kind of data is there from a machine learning perspective? There's a, like a huge amount w- we're talking about playlisting and just user data of, of what people are listening to, the playlist they're constructing and so on. Uh, and then there's the, the actual data within a song, what makes a song, I don't know, the, the actual waveforms, right? Is there any... th- how do you mix the two? How much value is there in each? To me, it seems like user data is, uh... well, it's a romantic notion that the song itself would contain useful information. But I, if I were to guess, user data would be much more powerful. Like playlists would be much more powerful.

    12. GS

      Yeah. So we use both. Uh, our biggest success initially w- was with playlist data without understanding anything about the structure of the song. But when we acquired the Echo Nest, they had the inverse problem. They actually didn't have any play data. They were just... they were a provider of recommendations, but they didn't actually have any play data. So they, they looked at the structure of songs sonically, and they looked at Wikipedia for cultural references and so forth, right?

    13. LF

      Oh, cool. Cool.

    14. GS

      And did a lot of NLU and so forth. So we got that skill into the company and combined kind of our user data with their, with their kind of, uh, uh, content-based. So y- you can think of it as we were user-based and they were content-based in their recommendations. And we combined those two. And for some cases, where you have a new song that has no, no play data, obviously you have to try to go by either, you know, who the artist is or, or the sonic information in the song or what it's similar to. So, so there's definitely value in, in both. And we do a lot in both. But I would say yes, the user data captures things that, that have to do with culture in the greater society that you would never see in the, in the content itself. But that said, we have seen... uh, we have a research lab in, in Paris when, you know, we can talk about more about that on kind of machine learning on the creator side, what it can do for creators, not just for the consumers. But what, where we looked at how does the structure of a song actually affect the listening behavior? And it turns out that there is a lot of... we can, we can predict things like skips based on this, you know, based on, on the song itself. We could say that maybe you should move that chorus a bit 'cause your skip is gonna go up here. There, there is a lot of latent structure in the music, which is not surprising 'cause it is some sort of mind hack. So there should be structure, that's probably what we respond to.

  11. 41:01 – 48:07

    Creator tools & feedback loops: bringing ‘GitHub + analytics’ to music and podcasts

    1. LF

      You just blew my mind actually for, uh, from the creator perspective. Um, so that's really interesting topic, uh, that probably most creators aren't taking advantage of, right? So there's... I've recently got to interact with a few folks, YouTubers, who are, uh, like obsessed with this idea of what do I do to make sure people keep watching the video? And they like look at the analytics of, of which point do people turn it off and so on. First of all, I don't think that's healthy, but, uh, it's, it's... 'cause you can do it a little too much. But it is a really powerful tool for helping the creative process. You just made me realize you could do the same thing for creation of music. And so is that something you've looked into of, uh, how-

    2. GS

      Yeah.

    3. LF

      ... and is it... can you speak to how much opportunity there is for that kind of thing?

    4. GS

      Yeah. I listened to, to, uh, the podcast with Zerosh.

    5. LF

      Yeah.

    6. GS

      And I thought it was fantastic and I reacted to the same thing where he said, I think he said he posted something in the morning-

    7. LF

      Yeah.

    8. GS

      ... immediately watched the feedback where the drop-off was and then responded to that in the afternoon.

    9. LF

      Yeah.

    10. GS

      Which, which is quite different from how people make podcasts, for example. (laughs)

    11. LF

      Yes, exactly.

    12. GS

      I mean, the feedback loop is almost non-existent.

    13. LF

      That's right.

    14. GS

      So if we back out, um, one level, I think actually both for music and podcasts, which we also, uh, do as at Spotify, I think there's a tremendous opportunity just for the creation workflow. And, um...I think it's really interesting speaking to you who, because you're a musician, a developer, and a podcaster.

    15. LF

      Mm-hmm.

    16. GS

      If you think about those three different roles, if you, if you make the leap as a musician, if you, if you think of it, about it as a software tool chain, really, your DAW with the stamps, that's the IDE, right? That's where you work in source code format-

    17. LF

      Mm-hmm.

    18. GS

      ... with your, with, with what you're creating. And you sit around and you play with that. And when you're happy, you compile that thing into some sort of, you know, AAC or MP3 or something.

    19. LF

      Mm-hmm.

    20. GS

      You do that because you get distribution. There are so many run times for that MP3 across the world and car servers and stuff. So, so you kind of compile this executable and you ship it out in kind of an old-fashioned boxed software analogy, and then you hope for the best.

    21. LF

      Right.

    22. GS

      Right? But as a, as an, as a software developer, you would never do that. First, you go on GitHub and you collaborate with other creators.

    23. LF

      Yeah.

    24. GS

      And then, you know, you'd think it'd be crazy to just ship one version of your software without doing an A/B test, without any feedback loop.

    25. LF

      Right.

    26. GS

      And then-

    27. LF

      Issue tracking. (laughs)

    28. GS

      Exactly. And then you would, you would look at the feedback loops and try to optimize that thing, right? So, I think if you think of it as a, as a very specific software tool chain, it, it looks quite arcane. Uh, wha- you know, the tools that a music creator has versus what a software developer has. So, tha- that's kind of how we think about it. And why wouldn't a, why wouldn't a music creator have something like GitHub where you could collaborate much more easily? So, we have, we, we bought this company called Soundtrap which has a kind of, um, Google Docs for music approach where you can collaborate with other people on the kind of source code format-

    29. LF

      Mm-hmm.

    30. GS

      ... with stamps. And I think introducing things like AI tools there to help you as you're creating music, both in, uh, in, uh, helping you, um, you know, put accompaniment to your music like drums or something, um, help you master and mix automatically, help you understand how this track will perform, exactly what you would expect as a software developer-

  12. 48:07 – 1:00:13

    Podcasting strategy: one audio app, discovery challenges, and preserving the ecosystem

    1. GS

      Exactly.

    2. LF

      And if Spotify is creating those tools out, that's a, it's a really exciting actually world. But le- let's talk a little about podcasts. It's ... So I have trouble talking to one person.

    3. GS

      (laughs)

    4. LF

      (laughs) So it's a bit terrifying and, uh, kind of hard to fathom, but on average, 60 to 100,000 people will listen to this episode. Okay? So, uh-

    5. GS

      That's intimidating.

    6. LF

      It's intimidating. Uh, so I host it on Blubrry.... I don't know if I'm pronouncing that correctly, actually. It looks like most people listen to it on Apple Podcasts, Castbox, and Pocket Casts, and only about 1,000, uh, listen on Spotify, in th- just my podcast, right? So, h- how, wh- where ... (laughs) Do you see a time when Spotify will dominate this? So, Spotify is relatively new, uh, in, into this-

    7. GS

      In podcasting definitely.

    8. LF

      ... in podcasting, sorry, yeah, in podcasting. What's the deal with podcasting and Spotify? Uh, how serious is Spotify about podcasting? Do you see a time where everybody would listen to ... You know, probably a huge amount of people, majority perhaps, listen to music on Spotify. Do you see a time when the same is true for podcasting?

    9. GS

      Well, I certainly hope so. That is our mission. Our mission as a company is actually to enable a million creators to live off of their art, and a billion people be inspired by it. And what I think it- is interesting about that mission is, it actually puts the creators first, even though it started as a consumer-focused company, and it says to be able to live off of their art, not just make some money off of their art as well. So, it's a qu- it's quite, um, an ambitious project. And, um, so we think about creators of all kinds, and, uh, we kind of expanded our mission from being music to being audio a while back, and, uh, that's not so much because we think we made that decision. We think that m- decision was, was made for us; we think the world made that decision. Whether we like it or not, when you put in your headphones, you're going to make a choice between music and a new episode of, of, uh, of y- of your podcast or something else, right?

    10. LF

      Yeah.

    11. GS

      We're, we're in that world whether we like it or not.

    12. LF

      Yeah.

    13. GS

      And that, you know, that's how radio worked.

    14. LF

      Yes.

    15. GS

      So, we decided that, um, we think it's about audio. You, you can see the rise of audiobooks and so forth. We think audio is this great opportunity, so we decided to enter it, and, and obviously, uh, uh, Apple and Apple Podcasts is, is absolutely dominating in, in, um, podcasting and we didn't have a single podcast only, like, two years ago. What we did, though, was we, we, we looked at this and said, "You know, can we bring something to this?" Uh, you know, we, we want to do this, but the ... back to the original Spotify, we have to do something that consumers actually value-

    16. LF

      Mm-hmm.

    17. GS

      ... uh, to be able to do this. And the reason we've gone from not existing at all to being the, the, uh, by quite a wi- quite a wide margin the second-largest podcast-

    18. LF

      Mm.

    19. GS

      ... consumption, still, still wide gap to, to iTunes but we're growing quite fast, I think it's because when we, when we looked at the consumer problem, um, people said surprisingly that they wanted their podcasts and music in the same, in the same application. So, what we did was we took a little bit of a different approach where we said, "Instead of building a separate podcast app," uh, "we thought, is there a consumer problem to solve here? Because the others are very successful already." And we thought there was in making a more seamless experience where you can have your podcasts and your music, uh, in the same application.

    20. LF

      Okay.

    21. GS

      Uh, because we d- we think it's audio to you. And that, that has been successful, and that meant that we actually had 200 million people to offer this to instead of starting from zero. So, I think we have a good, uh, chance because we're taking a different approach than the competition, and back to the other thing I mentioned about, um, creators, um, because we're looking at the end-to-end flow, I think there's a tremendous amount of innovation to do around podcasts as a format. When we have creation tools and consumption, I think we, we could, uh, start improving what podcasting is. I mean, podcast is this, this opaque, big, like, one, two-hour file that you're streaming, which it really doesn't make that much sense in 2019 that ... it's not interactive, there's no feedback loops, nothing like that. So, I think if we're gonna win, it's gonna have to be because we build a better product, for creators and for, for consumers. So, we'll see, but it's certainly our goal. We have a long way to go.

    22. LF

      Well, the creators part is really exciting. You already ... you got me hooked there, 'cause the only stats I have, uh, Blubrry just recently added the stats of whether it's f- listened to the end or not.

    23. GS

      Yeah.

    24. LF

      And that's like a huge improvement, but that's still nowhere to where you could possibly go in terms of statistics.

    25. GS

      Yeah, just download the Spotify Podcasters app and verify, and then, then you'll know where people dropped out in this episode.

    26. LF

      Oh, wow. Okay. (laughs) The moment I started talking. Okay. I might be depressed by this. But, okay, so one, um, one other question is, uh, the or- original Spotify for music, and I have a question about podcasting in this line, is the idea of albums. I have, um, what did you ... uh, music aficionados, friends who are really, uh, big fans of music often, uh, really enjoy albums, listening to entire albums of, of an artist. Correct me if I'm wrong, but I feel like Spotify has helped replace the idea of an album with playlists, so you create your own albums. That's, that's kind of the way at least I've experienced music, and I have really enjoyed it that way. One of the things that was missing in podcasting for me, I don't know if it's missing, I don't know, it's an open question for me, but the way I listen to podcasts is the way I would listen to albums. So, I take the Joe Rogan Experience and that's an album, and I listen ... You know, I like, I t- I, I put that on and I listen one episode after the next, then there's a sequence and so on. Is there room for doing what you did for music, or doing what Spotify did for music but, uh, creating playlists? Sort of, uh, this kind of playlisting idea of breaking apart from podcasting, uh, from individual podcasts and creating kind of, uh, this interplay, or, or have you thought about that space?

    27. GS

      It's a, it's a great question. So, I think in, um-In music, you're right, the, basically you bought an album, so it was like you bought a small catalog of like 10 tracks, right? It was, it was, again, it was actually a lot of, a lot of consumption. You think it's about what you like, but it's based on the business model.

    28. LF

      Right. (laughs)

    29. GS

      So you paid for this 10 track-

    30. LF

      Yeah.

  13. 1:00:13 – 1:19:24

    Product philosophy for ML: expectations, ‘algotorial’ curation, and user signals

    1. LF

      So jumping back into terms of this fascinating world of, uh, recommender system and listening to music and using machine learning to analyze things, do you think it, it's better to what currently, correct me if I'm wrong, but currently Spotify lets people pick what they listen to for the most part? There's a discovery process, but you kind of organize playlists. Uh, is it better to let people pick what they listen to or recommend what they should listen to?... something like Stations by Spotify-

    2. GS

      Yeah.

    3. LF

      ... that I saw that you're playing around with. Maybe you can tell me what's the status of that? This is, uh, Pandora style app that just kind of, as opposed to you select the music you listen to, it kind of feeds you the music you listen to. What's the status of Stations by Spotify? What's its future?

    4. GS

      The story of Spotify, as we have grown, has been that we made it more accessible to different, different audiences.

    5. LF

      Mm-hmm.

    6. GS

      And, um, Stations is another one of those where the question is some people want to be very specific. They actually want to hear Stairway to Heaven right now. That needs to be very easy to do. And some people, or even the same person, at some point might say, "I want to feel upbeat," or, "I want to feel happy," or, "I want songs to sing in the car."

    7. LF

      Right.

    8. GS

      Right? So they put in, they put in the information at a very different level, and then we need to translate that into that, what that means musically. So Stations is a test to, to create like a consumption input vector that is much simpler, where you can just tune it a little bit and, and see if that increases the overall reach. But we're trying to kind of serve the entire gamut of super advanced, so-called afic- music aficionados, all the way to, to people who they love listening to music, but it's, it's not their number one priority in life, right? They're not gonna sit and follow every new release from every new artist. They need to be able to, to influence music at a, at a, at a different level. So we're trying... You can think of it as different products. And I think when one of the, one of the interesting things, uh, to answer your question on if it's better to let the user choose or to play, I think the answer is the, the challenge when you, um, when, when machine learning kind of came along, there was a lot of thinking about wha- what does product development mean-

    9. LF

      Mm-hmm.

    10. GS

      ... in a, in a machine learning context? People like Andrew Ng, for example, when he went to Baidu, he started doing a lot of practical machine learning, went from academia and, and, you know, he thought a lot about this. And, and he, he, he had this notion that, you know, a product manager, designer, and engi- they used to work around this wireframe, kind of describe what the product should look like or some talk about when you're doing like a chatbot or a playlist, how do you... What are you gonna say? Like it should be good?

    11. LF

      (laughs)

    12. GS

      That's not a good product description. So how do you, how do you do that? And he came up with this notion that, um, the test set is the new wireframe. The, the job of the product manager is to source a good test set that is representative of what... Like if you say like, "I want to play the status songs to sing in the car," the job of the product manager is to go and source like a good test set of what that means.

    13. LF

      Mm-hmm.

    14. GS

      So then you can work with engineering to have algorithms to try to produce that, right? So we, we try to think a lot about how to structure product development for, for a machine learning age. And, and what we discovered was that a lot of it is actually in the expectation. And you can go, you can go two ways. So let's say that if you, if you set the expectation with the user that this is a discovery product, like Discover Weekly-

    15. LF

      Mm-hmm.

    16. GS

      ... you're actually setting the expectation that most of what we show you will not be relevant. When you're in the discovery process, you're gonna accept that, actually, if you find one gem every Monday that you totally love, you're probably gonna be happy. Even though the statistical meaning one out of 10 is terrible or one out of 20 is terrible from a user point of view, because the setting was Discovery, it's fine, but-

    17. LF

      Can I... Yeah, sorry to interrupt real quick.

    18. GS

      Yeah.

    19. LF

      I just actually learned about Discover Weekly, which is a Spotify, I don't know, it's, it's a feature of Spotify that shows you cool songs to listen to. I, uh, maybe I can do issue tracking. I couldn't find it on my Spotify app.

    20. GS

      It's, it's in your library.

    21. LF

      It's in the library. It's in the list of live.

    22. GS

      Yeah.

    23. LF

      'Cause I was like, "Whoa, this is cool. I didn't know this existed," and I tried to find it, but, uh, okay. (laughs)

    24. GS

      I, I will show it to you and feedback to our product teams.

    25. LF

      (laughs) Yeah.

    26. GS

      Maybe you can find it.

    27. LF

      There you go. But yeah, it's a... So yeah, sorry. J- just to, uh, j- just to mention the expectation there is basically they, you're going to discover new songs.

    28. GS

      Yeah. So, so then you can be quite adventurous in, in the recommendations you do. But, but if you're... But we have another product called, uh, Daily Mix, which kind of implies that these are only gonna be your favorites.

    29. LF

      Mm-hmm.

    30. GS

      So if, if you have one out of 10 that is good and nine out of 10 that doesn't work for you, you're gonna think it's a horrible product. So actually a lot of the product development we learned over the years is about setting the right expectations. So for Daily Mix, you know, algorithmically, we would pick among things that feel very safe in your taste space.

Episode duration: 1:47:03

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode v-9Mpe7NhkM

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

iOSAndroidClaudeChrome