Chandra Narayanan: Top 5 Lessons from Leading Analytics at Facebook | E1126

1. 0:00 – 0:44

   Intro

   1. CNChandra Narayanan0:00

      You never want to be a quitter. Set things right, fix things. If you think about why anyone comes to work, four different reasons. One is, they love what they do. Number two, they love the people they work with. Number three, they feel like they can learn from the people that they work with. And four, the company is going up and to the right. If one of these four does not work, they will leave.

   2. HSHarry Stebbings0:20

      (instrumental music) Chandra, I am so excited for this. I heard many good things from Alex Schultz before the show, but thank you so much for joining me today.

   3. CNChandra Narayanan0:27

      Thank you so much, Harry. Uh, it's really my honor. I, I love this show. I would say this is my favorite podcast show, and I have listened to so many amazing people saying amazing things. And I, I've learned so much from this show, so it's truly an honor.

   4. HSHarry Stebbings0:40

      Uh, listen, I've, I've learned just as much, so, uh, (laughs) I, I think it's a pretty cool

2. 0:44 – 2:35

   Seminal Advice from Rohan at PayPal

   1. HSHarry Stebbings0:44

      job that I have.

   2. CNChandra Narayanan0:44

      Mm-hmm.

   3. HSHarry Stebbings0:44

      I, I would love to start, though, with some chronology. I heard you got some seminal advice, uh, early on from one of your managers at PayPal, uh, Rohan. So, I'd love to start with this. What was the advice from Rohan at PayPal and how did it change your mindset?

   4. CNChandra Narayanan1:00

      Yeah. So, I was actually, uh, Rohan was my manager, and, uh, I, I think he went on to lead, uh, a bunch of different teams at, at, at PayPal. And, uh, I was, uh, uh, working in, at, at, at PayPal and was primarily on risk management and doing a bunch of different analysis. I actually got caught in a crossfire between two senior leaders, and what ended up happening was it became really, really frustrating for me, and, uh, I couldn't actually do my work. And, uh, basically, what I did at that point was I was, uh, so frustrated that I wanted to quit. And, uh, rather than address the problem, I wanted to quit, and I went and told Rohan, and said, "You know what? I don't want to stay here. I actually want t- to quit." And for which he basically said, "You know what? You never want to be a quitter." And he said, "Set things right, fix things, and then when you're in a better state of mind, come back to me, and I'll see what I can do." So, I stayed on for another six to nine months, and at that point, I got into a much better state of mind, fixed all the issues I had with the senior management, and then fixed the problems there was, and tried to get our, uh... and actually got, uh, the, the PayPal trajectory back on track. And then I went back to Rohan six to nine months later, and then basically told him, "Okay, man, I fixed a bunch of different stuff, and I still feel like PayPal is not the right place for me for several reasons." And, uh, and I told him, "I want to go to a different company." And he actually reached out to Facebook and literally got me a job there.

3. 2:35 – 4:09

   Importance of Fixing What’s Broken

   1. CNChandra Narayanan2:35

   2. HSHarry Stebbings2:35

      (laughs) I, I do just wanna stay on that because that's a really interesting thought of, like, fix what's broken before moving on. I always think about the opportunity cost of time. Sometimes it can take nine months, 18 months, two years to fix something that's broken. Do you still think that it's worth spending the time to fix it if, bluntly, a lot of it's out of your control, and the end state isn't even one that you would want to be in?

   3. CNChandra Narayanan3:00

      You know, I mentor a lot of, uh, uh, uh, uh, I mean, basically, uh, high school kids and undergraduates. And I tell them the number one most important thing that you need to learn as a, as a kid, especially at their ages, 'cause they're extremely talented, where they're very smart, they can do things, is character building. To me, I think it was a character-building exercise. The fact is that you need to build, uh, y- uh, not quitting is a character-building exercise. I mean, as we are a child, we're being told, "Don't lie," which I think is a character-building exercise. Uh, just crossing any line that you want to, that, that you think you shouldn't, is basically a character-building exercise. And I, I think it was a very strong character-building exercise for me. So yes, specifically, how long it takes, I, I think it's, uh, it's gonna hit you regardless. Uh, at some point or other, it's gonna hit you if you don't build character. So I think, uh, two years would've been a very long time, but I think six to nine months, I think it is worthwhile because otherwise, I might never have done it. And that has helped me so much later because when it became very tough situations at Facebook, I knew how to handle it. I didn't back away. I didn't say I would run away from a problem. I stayed on with it, and it was a very, very good six to nine months, uh, uh,

4. 4:09 – 5:26

   Toughest Situation at Facebook

   1. CNChandra Narayanan4:09

      time spent.

   2. HSHarry Stebbings4:09

      Can I ask, what was the toughest situation at Facebook, and how did the preparation that you had before, in terms of the character-building, help you be ready for it?

   3. CNChandra Narayanan4:19

      I think the hardest part for me was, uh, was, uh, when I was, uh, working on a team. And it turned out that, uh, that the kind of data that we were actually, uh, providing was people in the organization didn't want, didn't want to see the truth. And I was trying to be a truth speak- truth seeker and say the truth there. And it ended up being that, uh, the senior leadership there did not like what I was saying, and I almost got fired for it. And, uh, but that's when I think people like Javi and Alex at, uh, at Facebook actually backed me up. And, uh, and the tru- the trust they, they had in me was, was amazing. And the fact that I didn't actually run away from it and stayed on helped me. And the fact that there were people who were willing to back me up to the hilt helped me a lot.

   4. HSHarry Stebbings5:08

      I mean, that, that's incredible to hear on Alex and Javi, you know, backing you in a challenging situation for sure. When you reflect back on that incredible journey, 'cause it was an immense journey that you had with Facebook, what are one or two of your biggest lessons or takeaways do you think?

   5. CNChandra Narayanan5:22

      Couple of biggest le- lessons is, I think, one is, uh, focus on impact.

5. 5:26 – 5:54

   Lessons Learned from Facebook

   1. CNChandra Narayanan5:26

      Uh, especially after we went to Sequoia Capital and saw so many companies, and then I found out what was the thing that was so different in Facebook than any other company, it ended up being that it was really the fact that Facebook focus on impact, and many other companies did not. And so, that made a material difference, in my opinion, to how-

   2. HSHarry Stebbings5:46

      What does it, what does it mean to focus on impact versus not?

   3. CNChandra Narayanan5:51

      Yeah.

   4. HSHarry Stebbings5:51

      'Cause everyone says they have a big vision, and they have ambitious and grand

6. 5:54 – 7:29

   Importance of Focusing on Impact

   1. HSHarry Stebbings5:54

      goals. What does it actually mean?

   2. CNChandra Narayanan5:56

      Impact is, I mean, uh, is basically, uh, essentially, w- the thing that you're doing, is it a needle mover? That's basically what it is. Is, does it move the move needle? The thing, uh, are you always prioritizing on the most important things all the time?And if you are able to do that, you're actually focused on impact. Now, how do you measure that and how do you do that? So I think of it as a, uh, m- in my own, uh, framework, and I, I talk to companies about it, I also did that internally at Facebook is that I think of impact measured in three different ways. One is you, uh, move a metric. If you take a me- specific metric that you have in mind, which is, uh, maybe your North Star Metric, and you wanna move that, that essentially is one way to create impact. The second is influence, uh, a product. Decision that could be, like, you're doing things to identify an opportunity to help set a roadmap or a strategy, those would be another way of, uh, creating impact. And the third was influencing and changing a process, which would be like, "Hey, it's something that they do, being done manual and I can now automate it," and that would also relate to create impact. And I tell, I used to tell people at Facebook that if you're not doing one of this, if something you're doing, an activity doesn't come into one of these things, you're probably not having impact. Then what ends up happening is that you confuse motion with progress. And so there's a lot of motion, you do a lot of different work, and you find out that it's not prioritizing on the right thing. And so, uh, essentially, impact is saying you need, you shouldn't be confusing motion with progress.

   3. HSHarry Stebbings7:22

      Uh, can you just, for those listening who don't know the difference between motion and progress, how do you define the difference between motion and progress? I'm loving this.

   4. CNChandra Narayanan7:28

      Yeah. Uh, motion

7. 7:29 – 8:37

   Difference in Motion & Progress

   1. CNChandra Narayanan7:29

      is about doing tons of work, and I re- recall when I used to be younger, I would say I worked 24 hours, I didn't sleep three nights in a row, and that's just basically saying you did a bunch of different things, lots of activity. But if you ask those three days of activity, what was the impact, what was the value of the, uh, the value of it, uh, activity, and you can't actually say much. "Oh, I did 2,000 things." And I think what ends up happening is that we, we, uh, I, I, I think it's a, it's a rite of passage. I think at a, at an earlier phase of life you actually work hard and you feel good about it, and working hard is the most important thing. I still think it's the most important thing, by, by the way. But if you work hard and don't work on the right things and you don't prioritize it, that doesn't lead to impact. And so the idea is that the motion is all about activity, lots of different activity, and the moment you think it's, it's kind of like a vanity metric. I mean, it's like saying I got, I mean, uh, I, I have these billions of installs where I, the, the r- retention is like 1%, right? It's not something that you really want to... it's sort of like a vanity metric. I, I think, uh, that's like a vanity metric, in my opinion.

   2. HSHarry Stebbings8:31

      Does activity not lead to progress though? And what I mean by that is, by doing lots of things, even

8. 8:37 – 17:36

   Data-Driven Decision Making

   1. HSHarry Stebbings8:37

      if you are directionless-

   2. CNChandra Narayanan8:39

      Yeah.

   3. HSHarry Stebbings8:39

      ... you get data that will direct you in certain progressive directions.

   4. CNChandra Narayanan8:45

      I have a framework for this. So if you think about your activity, and if you think about it, everything feels like, uh, uh, in terms of if you take a completely new activity, right, you're actually learning up in the, in a, in a curve, and then you start a flat note, right? I mean, if you look at it, you go in a straight line, you keep growing as you go along, and then after a point of time, y- you, you asymptote. You, you don't grow anymore. Just take for example basically this thing brushing your teeth. I mean, brushing your teeth is now second nature. You can't say that if I spend two hours on it, you're gonna get that much better. Probably not incremental. There's opportunity cost to it. So as you, in terms of activity, if every single activity that you do were on the straight line, that every single hour that you spend, every single day you spend improves or increases or you're better each day, then I think you should be spending time on all kinds of different types of activity and you'd be okay. But I think if you spend 80% of your time actually saying that you're, all you're t- trying to do is, you know, provide a, a, I mean, a, basically you're traffic ticket at the New Jersey turnpike and you're handing it out, that can't be. If n- n- 80% of your time is just doing that you can't say, "I'm getting better and better at it." So I think it's a kind of activity to do whether you are on the growth curve of the activity or in the, or you're in the asymptote of the activity. And I think if you maximize that, then you're probably in a much better shape. Mark once said in a Q&A in, internal to Facebook, that he said that he... and a lot of the reasons why we do that is because of how secure we are as people. And he would basically say that, something like, I, I don't remember the exact number, but he said like 80% of his time he is, he does activity that's outside his comfort zone, which basically means that he's trying to be on this sloping growth curve rather than the flat curve. I think most people can't do that because they're not secure enough to do it. But if you are able to do that, I think you'll have the greatest impact you can in the shortest amount of time. And impact also is, if you think about the total amount of impact is the impact over time, right? The total amount of impact you can o- o- divided by the time that you spent.

   5. HSHarry Stebbings10:43

      I, I totally agree with you and get you. Can I ask, when you think about that activity leading to impact, how did that drive influence your decision-making advice when you think about Sequoia? Because venture is a weird business in the way that it's serendipitous. There is some elements of luck that is quite unexplainable. "I ran into Chandra after years and decided to do the deal." A- how did you think about that when you think about your time with Sequoia and driving the data effort there?

   6. CNChandra Narayanan11:12

      Uh, I think at the highest level, uh, you know you have so many things in terms of the data team that you had at Sequoia that you can actually do. For example, you could be spending time on, on sourcing. There are three types of activities that we did. Basically, we, I would spend time on sourcing, which is trying to identify companies, uh, that you, that investors can go talk to. Second is due diligence. Once the company comes in and they give, give, give, give us the data room, we basically need to make a recommendation on whether this company is doing well or not. And the third is company portfolio building, which is like going to, go to the companies that are trending well and basically try to spend your time and try to see how you can make a good company great. And that's the third thing. Now, if you were to spend the time, it's again come, come back to what, how you spend your time. If you were to spend the time that is l- a company that is likely not going to go up and to the right and you spend all your time on that, you can imagine how much of impact the company would have. The same thing would be true with f- uh, the sourcing. If you give, uh, an investor 200 different leads rather than the top five leads that matter...... I think it makes a material difference to how it is. It's also the d- due diligence. Eh, in the due diligence, I, I think we made a lot of great progress on that, is basically coming up with frameworks so that we can actually make very good decisions on it. What we realized on the due diligence part in particular, what we realized even through the way we prioritized was that how quickly can we say no? And that was primarily what we arrived to, because that was the fastest way. What is the one signal in every company that would say no rather than say yes?

   7. HSHarry Stebbings12:43

      Talk to me. What progress did you make there, and how did you come to answer that question of how fast can we say no and focus our attentions more effectively?

   8. CNChandra Narayanan12:51

      There was a company where we would actually find out that the- they were growing really well, everything was going right, uh, to the right, and then we would find out... Uh, we asked them for their marketing spends, and we would find the marketing spends, the reach of the marketing, when we looked at the marketing reach, it had, uh, it, uh- it was very close to the addressable market. So basically, they already talked to everyone, in which case we realized that, that, that one thing is not. Another company we found out that, uh, the reason why we did not invest was we realized that their, um, their all the older cohorts were doing really, really well, retaining well, uh, and engaging well. But we found that the more recent cohorts were actually starting to decline, and we could see it in the data, and that is the one reason that we did not invest in them. So, these are just two examples, but every time that we would look at, we were stressed in- in saying, "How can I say no to the company?"

   9. HSHarry Stebbings13:45

      Can I ask, it's e- I'm, you know, I'm a venture investor as well. Um, it's easy to say no. You know, there's always an, "I don't like the sector. I don't like the size of the market at the time," whatever reason we give. It's harder to say yes and see beauty where others don't. Do you feel that was the right approach?

   10. CNChandra Narayanan14:03

       So, two things.

   11. NANarrator14:04

       (music)

   12. CNChandra Narayanan14:04

       I think if you look at it from a funnel perspective and from the time perspective, if you t- think about it, if you think about the funnel and saying the investors talk to, you know, uh, m- let's say 2,000 companies in all per year, uh, um, probably it's more, but let's say it's 2,000 to 5,000 companies in all. And then you have this bucket of, uh, l- let's say there are only 50 great companies in all every year that you can even invest into. Then you want to catch those 50. And the thing is that if you make wrong investments, and many of this, by the way, that the- that we stopped, uh, Sequoia from investing where I think they would have otherwise invested. It was so close to investment. So then, you're going to spend so much more companies. So, think about every investor. If you have 10 great- 10 companies in your portfolio, every investor has 10 c- 10 companies in their portfolio. If eight of them are not so good and two are great, they spend all their time on that two, but they can't do anything about the eight, they still need to talk to them. If you can make it like out of the 10 you have five good and five bad, you're in so much better place in terms of where you are. So, I actually think that it really helps the investors themselves because you do- you do want fewer and fewer bad investments. I actually think that good investments are relatively... I mean, there may just be this one great investment that nobody knows about, and those are hard, and I agree with that. But if you think about the good investment, very quickly everyone knows about it. It's not that hard for good investments. It's only the bad investment that you may end up investing, I believe, is where most of the- most of the opportunity lies.

   13. HSHarry Stebbings15:29

       That's so interesting in terms of like, actually where you said no being the most valuable. Not being to save them the time upfront, but it's to save them investing money that would otherwise have been misinvested.

   14. CNChandra Narayanan15:40

       And time.

   15. HSHarry Stebbings15:41

       And time.

   16. CNChandra Narayanan15:42

       Investing money and time. Time is a big one for, uh, Sequoia because e- each investor, they'd rather be spending, uh, on five good investments out of eight than two good investments out of 10.

   17. HSHarry Stebbings15:54

       What do you think makes Sequoia so good, Chandra, having seen it internally?

   18. CNChandra Narayanan15:57

       So first, I think the quality of the found- founders. They're exce- I mean, of the investors themselves are exceptional. I actually think their brand for several, several years. I mean, nobody doesn't talk to them, right? I mean, you have everyone talking to Sequoia, which actually gets them all the leads that they want to, and that, that's pretty exceptional. The third is a diversity of the people that are in the group, is like you have all the way from, you know, I mean, during those times is M- Mike Moritz and Jim Goetz and, and Roelof and, and Pat, and of course, so many other people. And they have this diversity of people with different types of skills, and I think their collaborative decision-making is also excellent in the way that it's, uh, the collaborative decision-making. I think the process that they take in terms of how they go through the, you know, the- from the one pages, from the time that they actually, uh, source the deal to going through, uh, the due diligence process to ge- getting to the one pages and, you know, to actually talk about it and then going through the decision-making process. I love the way that they do their process. They keep talking about one thing. Everyone going into the room, they talk about this prepared mind. I keep t- talking about it since to my team and so on, is like, eh, people who go into the room, there's already, if you have a Mon- Monday morning meeting, they circulate the entire memo on a Friday, and when people go into that room and, and make a collective d- decision, they go with a prepared mind, which basically means you better have read your memo. You come in knowing everything. We're not gonna talk to you about the basics. We'll go deep into the i- investment itself. And I, I actually think it's a- it's a- a- it's a- it's a phenomenal firm.

   19. HSHarry Stebbings17:29

       I totally agree with you. Um, I do want to go back to the core question of like, the impact versus activity, 'cause I can- I can go

9. 17:36 – 21:10

   Building World-Class Teams

   1. HSHarry Stebbings17:36

      off on many tangents, Chandra, so don't worry about that. Was there any other takeaway that you wanna highlight from Facebook?

   2. CNChandra Narayanan17:41

      I think the second one I would say is the- I- is the, uh, I learned what it meant to build a world-class organization or a world-class team that was happy. And again, this came a lot from, uh, from the growth team at Facebook. I- I think both, uh, I would attribute Alex Schultz and, uh, Harvey to it. They knew they had a very, very high bar. Uh, their first, I think, high bar in terms of the people that they had, the high bar in expectations about them. But...... and in the way that they cared. So, they found a way that w- in terms of... I mean, we talk about impact earlier. I think it's the impact per capita. So, the way I think about it is, like, impact is equal to impact per capita or impact divided by the number of people times the number of people. They always cared about impact per capita. How much can each person do and then multiply it by the number of people? So as a result, what they would have is the growth marketing team which, uh, which, which Alex Ran had Brian Hale, which who was on this show. Uh, and there are so many other people who are on this show. I think Ryan was in the show. So, there are multiple people on the show, and this is all Alex. And it was like a seven people team or a six people team. And I would think like how can six or seven or eight people have such enormous output? And it's just the comradery, the way he built teams, the way that he would n- make sure that every single person who came in would be incredible and never try to grow very, very fast. And so it is impact per capita. I think what we get confused a lot of time is total impact. There's like... Total impact can be gained by lots of people.

   3. HSHarry Stebbings19:09

      How do you calculate impact per capita? If I'm like a founder listening and I'm like, okay, I've got seven people in the marketing team or a growth team. H- how do I actually do that, Chandra?

   4. CNChandra Narayanan19:19

      It's kinda hard but, uh, I would say when I first joined Facebook and I remember, uh, uh, an engineer, Harry, uh, who worked on the payments team told me once, "The way we think about the impact here is basically, uh, take our market cap," which I think was 10 billion then, and they were probably, let's say, and, and for the math, for the sake of math, I'm just going to say half, half. Thousand engineers, I think it was far fewer. But just, just for math, uh, I'm just saying it's, uh, it's thousand engineers, so that would be like 10 million per engineer if I'm doing my math right. So essentially, he's saying every single engineer contributes 10 million to this. Uh, I think it was more like 20 or 50 million. But... So every kind- so every new engineer that c- comes in, and, uh, will have to contribute so much. Otherwise, if you can't find something that they can do that can be of that type of impact, don't hire. And I think it's the same sort of mindset that Alex and Harvey and everyone else had which is like, do not add more people. One thing if you add more people, what happens it's, uh, you, the A+ players becomes A and so on, and, and very fast. You need to get to grow very slowly so you can reach s- uh, equilibriums very slowly, and then keep the value of the entire team high. So, do not hire very fast. And so that- that was one thing. I will also say that the way you mea- measure it, yes, it's harder in, in many things, but you kinda know. In ma- many ways, you, you can think about is like if you basically say I can only have X number of engineers, and then essentially peg your engineers to every other part of the team. So, it's 20:1, let's say, engineer to PM ratio or from PM to data scientist is 1:1 ratio. So, but essentially if you're saying engineers are building stuff and you have a market cap or some value there, you can actually say what the value of everyone should be.

   5. HSHarry Stebbings20:58

      I love that. No, you absolutely can. I, I just think so few people probably approach it in that way. You, you, you spoke about Ryan, you spoke about Alex. These are some of the best growth minds that we have. Um, growth is quite a

10. 21:10 – 21:50

    Defining Growth & Importance of Hypotheses

    1. HSHarry Stebbings21:10

       widely used term, Chandra. Um, how do you define growth today? What is it? What's it not also?

    2. CNChandra Narayanan21:16

       I just think of growth as, uh, is basically about identifying methods, approaches that scale product market fit in a scalable way. The way you do that is by identifying a North Star metric and moving that metric. And in order to move the metric, you need to basically identify and prioritize the most important opportunities that move the metric.

    3. HSHarry Stebbings21:43

       How do you select North Star metric? How do you advise founders on choosing the right one?

    4. CNChandra Narayanan21:48

       The North Star metric itself,

11. 21:50 – 32:47

    Selecting & Changing North Star Metrics

    1. CNChandra Narayanan21:50

       a lot of... I mean, there, there's a lot of thought that is obviously... Uh, a lot of people have, um, multiple thoughts on how do you actually select the North Star metric. I, I, I, I actually think that for your specific company, there's l- there is a mission and it needs to tie to something in your mission. For example, at Facebook, it- at that time that we were there, it was like making the world open and connected. So it naturally was you wanted to get everyone in the world on it, so it just naturally meant that you wanted to get the largest number of people to use a product, so MAU made a ton of sense in terms of what you're trying to do. So, that's one way to think about it is like try to get all of that.

    2. HSHarry Stebbings22:25

       Can I just understand? If we think about like connected world, number of users, is that not an output metric which is kind of tough for you to work towards then because you really wanna work on obviously inputs that lead to outputs?

    3. CNChandra Narayanan22:37

       Hundred percent. So, uh, in many cases, uh, I'll give you the example of, uh, I, I actually think that, uh, at Facebook, you could actually move the DAU metric too by mu- multiple ways or DAU, MAU metrics. But for example, if you're trying to move, let's say, the number of friends, for example, that could be more of an input metric. So, I actually think that the goal itself that you have, the goal that you have, uh, needs to be movable. And you could move it through multiple, multiple different ways in terms of what you can do. But I actually think that, uh, you're not trying to, uh... If you choose a metric that can't be moved, that's not a good metric. For example, I'll take the example of advertiser growth. In advertiser growth at Facebook, uh, I, I... In the advertisers, we decided not to move the revenue metric but decided to move the advertiser growth metric because that's a metric that we actually could tangibly move. But as... We didn't have the levers to move the revenue metric. But I, I do believe from an active user perspective or DAU or MAU perspective, we actually could move the metric in terms of what those, uh, I mean, in terms of what those metrics were.

    4. HSHarry Stebbings23:44

       How often do you change your North Star metric, Chandra?

    5. CNChandra Narayanan23:47

       I think a lot of your North Star metrics changes, uh, you change it largely because... I mean, for example, when Instagram came along, uh, to Facebook, eh, we still had the MAU as our goal.And so, when we had the MAU as a goal, uh, I mean, uh, basically Kevin's system was like, "What? We're now in the mobile world. People use the phones all the time. We should be doing, uh, we do DAUs, we don't do MAU- MAUs." It makes no ce- sense because essentially people are using the product every single day. Why are you still caught up in the MAU?" And that made us start to think because we were still in the web world, which is fine, where people weren't using the product every day. So, in that sense, as the company starts transitioning, let's say, from a web world to a mobile world, you need to change. And that is one reason why you would actually change your metric, is, like, move from a MAU goaling metric to a DAU kind of a goaling metric. Th- that's just an example, but I- I- I think the market is one reason. The second reason is actually, uh, the world changing on you itself, and, uh, and- and so on. And third, I mean, you may have just picked a bad metric. So- sometimes in companies it's just hard. It takes a lot of iteration to get to the right metric.

    6. HSHarry Stebbings24:57

       Do you think pattern recognition and playbooks are good or do you think they're misleading? When we look at a pre-AI world versus a post-AI world, a pre-COVID versus a post-COVID, the world changes so much.

    7. CNChandra Narayanan25:09

       I know, man. I know, I know. I think the- the- the problem, as you point out, the biggest problem for anything with- with your i- intuition led is bias, is bias and how the world change around you. And you- those are the ones that you need to guard against. But honestly, that's the best you got at that point. So if you actually have to make a decision... And I actually think that the cost of not making d- decision, in my opinion, is far worse than making a wrong one, as long as you can iterate fast. But you're right, you're not gonna get it. The intuition is not fail-safe, for sure, but even data is not fail-safe. Data also works under certain assumptions that you have and those assumptions can just get ripped off, uh, can change. Yeah.

    8. HSHarry Stebbings25:51

       Oh, they can. And also, yeah, I mean, data can be misleading in many respects too. Um, I- I do have to ask, I spoke to Julie on your team before, and she told me that I had to ask, kind of, while we're still in the formulation stage of strategy, there's North star but there's also hypothesis. And you've said never forget the hypothesis in your approach. So, how do you define really what is a good hypothesis and do you always need one?

    9. CNChandra Narayanan26:15

       Any problem that you're trying to solve, um, you do... If you want to deeply understand... If you wanna make great decisions, you need to understand a phenomena very deeply. That- that's my thesis. If you wanna understand S- ... If you wanna make a d- great decision, you need to understand that phenomena that you're trying to make a decision very well. If you want to understand that, you basically need to, uh, a- analyze that problem very well and- and, uh, and using data to- to come up with that. In order for you to construct the story around the phenomena, it turns out the data has a lot of things. I think of data as a manifestation of you have a story that you care about, and then it's manifested in the data that you have. So the data itself is manifested in a ton of different metrics, for example, and you understand and say, "Okay, active users went up," or, "New users went up," or you find that ad spend went up and so on and so forth. And what ends up happening is that you need to come up with a bunch of hypothesis of why they went up or down. And generally these types of hypothesis would be, for example, if you look at active users going up or something, the- primarily the reasons are either- either it's seasonality or it's a product change or it's a sales change or the, uh, there is a marketing change or, uh, competition or things like that. Or- or... And it's manifested as behavioral change in customers, and that's how it is manifested. And then ultimately you see it in the data. So to me, if you do... If you wanna deeply understand a phenomena and make great decisions, you need to understand the story behind it really well. If you want to understand the story behind it very well, you have to go into metrics and see how they changed, uh, identify them, and then be able to come up the right hypothesis of why they changed, and then if you do that, you can make great decisions.

    10. HSHarry Stebbings27:59

        I don't know if you know Annie Duke- Annie Duke, but she is a writer and a poker player who wrote a book called Thinking in Bets, which basically talks about, kind of, decision-making. And she has, uh, talked about this theory, and I can't remember what it's called, but it's essentially when good process leads to not successful outcome or when successful process leads to bad outcome. Uh, how do you think about that and does that go against the importance of hypothesis?

    11. CNChandra Narayanan28:27

        In terms of how you do the hypothesis, you need to stress test it, which means you come up with hypothesis, and then you need to look at data to validate the hypothesis. So you just can't do a hypothesis and say, "This is it," and so on. So the way that you- you do this... And I'll tell you a- a quick example that we did, for example, at- at- at- at PayPal in fraud. For example, the way we would do that, we found out that, for example, there are bu- bunch of different... Uh, I mean, we wanted to reduce fraud, which is the most important metric for, uh, risk manage and probably the most important for even PayPal in terms of the me- metric that they cared about, one of the most important metrics that they cared about. So what we did was we said, okay, uh, we made a hypothesis and we basically said, "Hey, people, if they use the same cookie, uh, whi- which is a web cookie, and if they... if more than five people use the same cookie on the same day, there's something wrong with it." Which ba- basically we are saying that people were using a single computer, they were taking other peop- taking over other people's accounts and doing a bunch of different activity on it, and then there is something bad about it. So we basically start with the hypothesis and say, "Hey, this is wrong." So we go... So then you go into the data and try to validate it and saying, "Okay, is it really true?" And when you go into the data, you say, "Yeah, many of them actually fraud." So you look at good use cases and the bad use case. The good... The bad use cases are, yeah, the- many of them are fraud, so. Then you look at the good use case. For the same person using the same computer-It turned out that a bunch of them were family members, and that's something you can actually find out through deeper investigations. Or they were using an internet café. They were the same people using the same internet café, logging in, uh, o- o- out of the product. So, what do you end up doing is that you need to go back from data to hypothesis and back to data, and basically solve the problem that way. And so if you actually do that and you're rigorous about it, and you really want to tell the story, and you want to make sure that the, the story is right, you need to get that now. There may be still things that you don't know that you don't know, and your story still may be wrong, but, uh, you still want to get to the point of getting from data to hypothesis and back, and then be able to tell what the story around it, s- and then hypothesis, what story it is. If you can do that, you can make great decisions.

    12. HSHarry Stebbings30:37

        Is it always obvious being able to tie data to hypothesis? Like, where does the challenge come in that transition between the two?

    13. CNChandra Narayanan30:45

        There are multiple problems here. One is that I think when you have the data, um, it is possible... I mean, you have a hypothesis, you can't even check it. For example, at Facebook, we'd have saying that, "Hey, by the way, we would, uh, we would want to know why advertiser's churning," and there's no easy way to find it. I mean, you actually need to do a survey, it takes a long time, and then they will basically say, "ROI." When they say, "ROI," we still won't understand what ROI meant to them because we don't actually know the data behind it. So, a lot of this time, I mean, there are things that you can actually find out through the product, and so it's already in the product and it's in the data, and you can actually provide that type of hypothesis. To validate the hypothesis many, many times, you can't do it through just data. You need to do it through, essentially, user experience research and, and so on. So, there is a, a problem of just being able to validate your hypothesis because typically speaking, uh, there's a lot of time you don't have the data. The flip side is also true. You may not even be able to come up with the right hypothesis, and I've seen that happen too. It's like you don't know what went on, and suddenly you realize, uh, what happened. For example, what happened at, uh, uh, Facebook, and it happened, we came up with the hypothesis very, very, much, much later is like, we found that there's certain parts of America which had a, uh, uh, an increase in time spent, and we didn't know what happened. And we kept looking and said, "What is all this?" And we kept digging deep, and it ended up being winter. Very cold winters, people staying at home, and they use the product a lot more, and the time spent went up. And if you look at just time spent across US on extremely cold days, you see an increase in time spent, not...

    14. HSHarry Stebbings32:19

        You mentioned that kind of incredible team members, uh, are helping contribute to the decision-making and the discovery. I spoke to so many people around you, Chandra, and they said that one of your great skills is in building teams, and in inspiring and encouraging great talent to be its best. I do want to touch on this, and we mentioned a lot there in terms of operations and the granularity of it. When's the right time, do you think, to add a first growth hire? When you're advising companies and founders today, how do you advise them?

    15. CNChandra Narayanan32:46

        You don't

12. 32:47 – 36:28

    First Growth Hire

    1. CNChandra Narayanan32:47

       want to hire anyone before product-market fit because at the end of it, I think we- as I mentioned to you, I think growth is about scaling that product-market fit in a sustainable way. It basically means that if you're not able to grow, i- if you, if you've not even reached the point where you can actually scale it, no point getting your first hire.

    2. HSHarry Stebbings33:05

       I was just with a, a billion-dollar founder the other day and they said, "To me, product-market fit is when literally monkeys could run the business, and people want it so much that they would still sell really big product volumes." (laughs) What would it be to you? Other people are like, "Oh, your 10 first customers." There's many different variants and they're not right, it's just different. What- what's your, like, ugh, they have it or they don't?

    3. CNChandra Narayanan33:29

       I've now f- because I worked, especially being at Sequoia and worked with enterprise and e-commerce and small companies and large companies and all kinds of different companies, I don't have a single answer for this. Uh, it's a single simple answer that will satisfy everyone, but I would basically say it's essentially, uh, if it's a consumer product, people want to come back. They want to keep using the product. They seem to love the product, and you're not throwing more money at it, which means, that's why I mean by sustainable. It's not like you're throwing huge amount of marketing dollars to just keep them, uh, on, and there is some, they're actually g- being able to provide, uh, a value that other companies don't. Uh, and I think with the product-market fit, it doesn't mean the product-market fit you can actually scale it because the moment you start charging, for example, it's a free product, and maybe just the people are only using it because of the free product. The moment you charge it and charge the same amount as a competitor, it may just go away. So, it's not that it's guaranteed that you're going to go from... I mean, product-market fit, scaling product-market fit, unit economics, scaling unit economics, I think those are all four different steps in my opinion. So, I actually think that it is, uh, you can get to product-market fit, but you may not be able to scale it for several reasons. And if you get to scaling product-market fit, you may not get to unit economics. And if you do, and you get to unit economics, you'll never be able to scale it.

    4. HSHarry Stebbings34:48

       I listen, I completely agree with you. I think product-market fit has many chapters. You- you've worked with many SaaS companies, you start in PLG, you scale into mid-market, you move into enterprise. At each tangent, you need to regain product-market fit in each customer segment. So, I couldn't be more aligned to you. What a terrible question, Chandra. I mean, God, I apologize. Uh, (laughs) listen.

    5. CNChandra Narayanan35:08

       (laughs) .

    6. HSHarry Stebbings35:08

       So, we have that realization that it's post-product-market fit. Okay. I'm a founder, you're an investor in my business and you're advising me. What's the right profile for that first growth hire?

    7. CNChandra Narayanan35:19

       I'm not sure if you just hire one single person, it's gonna be that valuable to you, meaning that what will that one single person do? And it probably is a team that you need to hire. Uh, and if you think about the one single person that you're trying to hire, i- if you are thinking about the one single person you're trying to hire is probably the leader of the team who can then get you everyone else, such as getting you the... I mean, a- as you know, I mean, you need to get the designer and the growth marketing person and the analytics person, the product manager, right? And, uh, engineer, all of them on board. So, if you don't have that, I am not sure how you can actually...... actually, uh, function because you can have someone, they won't have... and of course the marketing spends the dollars too for it. And, uh, so if you don't have that, I don't know h- what they would do if you had just one person. But I would basically say that you need to hire... if you were to hire only one person, it'd probably be someone who is relatively senior who can hire everyone else or be part of the team and so that you can actually get a team that can actually work and, and move a metric.

    8. HSHarry Stebbings36:21

       Okay. And so, it's really interesting. Your answer there jumped straight to a growth team being independent, a standalone growth team.

13. 36:28 – 39:30

    Centralized vs. Decentralized Growth Teams

    1. HSHarry Stebbings36:28

       W- the alternative could be that you have designers, you have marketers with a growth slant towards the more analytical, more rigorous in that way. Um, you have PMs in a similar vein, and they work as an integrated part of the existing org. Do you think growth teams need to stand independently or do you think they can function within the org?

    2. CNChandra Narayanan36:50

       I think there are, uh, two answers to this. I think the answer at an earlier stage of your growth, I think they should be a standalone team. I think it's for two reasons. One is I think it's the- it's for the best practices, meaning that they could learn from each other and then they can go solve problems across a company. Suddenly, you separate this team and break, put them into the different paths, I don't know if they'll be able to build the same culture of the know-how. A lot of it is knowing how to do this really, really well. And all of this transfer, if you went from, let's say I- I worked on many parts of this from pages growth, to games growth, to advertiser growth, to... you know? And then when you have all of these types of growth, what ends up happening is that these sort of knowledge does translate, and imagine that y- all this was not part of one single team and they were separated all over. I actually think th- don't think the learnings will be there, so my feeling at the earlier stage, you would- should all, I mean, it should be centralized. Now, once you get a large enough team, if you get a large enough team, I actually think that it can start to be decentralized and work, go into product teams, but I actually think that early on, it should be centralized. So, the more important thing is that somewhere in the interim, you probably have a, a role where you embed people into teams, you embed people into team but you still have a centralized organization, and then at some point, you move completely decentralized.

    3. HSHarry Stebbings38:08

       My question to you there was, that's wonderful but it almost feels inverted when you look at the budget of a scaling company. You know, this centralized standalone is when you have the least money and then when you're scaled and you do have the money, that you could afford that centralized, y- y- you... Do you see what I mean? So, is it possible to do that early and how do you think about the need for that centralized team with the budget of a, a, a young company that's maybe just hit series A?

    4. CNChandra Narayanan38:37

       Yeah. At the end of it, t- think about it, you're at a series A. I mean, are you gonna do 20 things? Are you gonna do two things, three things, four things for the f- from the growth team? So, there are fewer things that you're actually gonna do, and for that, a small team will just suffice. You're doing very, very few things and for that, those five people that I talked about just do, and you'll work on one, and you'll work on the second, you'll work on the third, and you'll actually work on multiple types of problems. At any time, you probably don't work on more than two or three because the surface area can't be that large. If you're going to spend that much time on everything, you're probably gonna do nothing right. So, I think at a smaller phase of your company, you don't need... it's... the surface area narrows so you can actually do it with just a centralized team. But actually, when the surface area grows, you can actually start to spread your wings out. So, I actually think it works out.

    5. HSHarry Stebbings39:19

       That's really interesting to hear in terms of that transition from centralized to decentralized. Founders are always told, "Hey, you need to hire for 18 months ahead of time, uh, for the future company that you

14. 39:30 – 42:34

    Hiring for the Future Company

    1. HSHarry Stebbings39:30

       will be." Do you agree with that given the many different hiring experiences and scaling journeys you've seen?

    2. CNChandra Narayanan39:35

       This comes back to the product growth, right? I think that if you think about it from a... I mean, I've... while at Facebook, uh, s- so there's... I'll give you an example. At Facebook, uh, when I joined Facebook, uh, analytics was all about counting numbers, and then it was creating dashboards, and then it was doing AB testing, and then it was going towards goals, roadmap, and strategy. So initially, the types of person that we needed to hire were basically people who can just count numbers. And secondly, so it was bunch of people who can create infra to create dashboards. The third was people who actually were statisticians who could do AB testing. And then finally, it was people who could influence and f- even now I think it's Facebook is like, uh, the analytics team is largely about influence, even inside the growth team and so on, throughout, it's about influence. So, the point is that if I'd actually hired three years earlier and said I'm gonna hire people for influence, what would ended up happening was people wouldn't have been able to do that. Same people who do the AB testing who needs stats background are now the ones who can actually influence, and what would happened is that you wouldn't have actually taken the journey in the right way. So, what ended up happening was that we, we built the dashboards, we automated it. AB testing, we built something called Delta, it automated it. And then when we could automate all of this stuff, it will become easier for us to move towards what we think was the most valuable thing that analytics teams could do, is influence. So again, we would, even in the growth team, we would actually understand, identify opportunities, influence the roadmap of products, of what n- of the roadmap of what we need to build. So, my point is that I think it depends on the growth of your company and where you are at, I think there are... the... at a later stage, I started to build for much, much longer, 18 months and so on, but the earlier stages, I actually, uh, hired for what I needed right now because there's no point in hiring someone who will be so valuable for you in 18 months and who co- can influence everything but can't build your dashboard. But I, I think... And I actually, um, advise a bu- bunch of different companies now. I... depends, I, uh, change my answer based on who I'm talking to. If I think this company is in a rocket ship-... for example, a company like OpenAI, I would be like-- in fact, I did that. And when I talked to OpenAI, they asked me who should be hiring for the head of analytics and head of growth and so on. I said, "Oh, someone who's l- looking at a rocket ship. Don't worry whether they are statisticians or data. Influence is the most important thing for them." But if I were to be at an early stage series A company, I would say, "Hire for someone now because I don't know how your... How your GSS or early part of your product market fit." If you reach certain levels of scale, I think it's a very different thing. So, I think it's a lot to do with the stage at which you are at, and I think your answer is either very short-term or long-term. But keep in mind that it's going to become long-term if the c- company goes into a rocket ship, you do need to hire for the long-term.

    3. HSHarry Stebbings42:28

       I'm fascinated. You mentioned the word influence there. And obviously, I spoke to Alex before the show, and he mentioned that you did

15. 42:34 – 47:04

    Challenges of Influencing

    1. HSHarry Stebbings42:34

       face some, uh, pushback against some of the data that you presented at Facebook and some of the ideas that you kind of outlined. Why do you think that was the case? Just help me understand that.

    2. CNChandra Narayanan42:46

       I think influence is a hard problem. So I think, uh, influence is obviously an art, not a science. And so, um, if, uh, people are f- very receptive about what you're gonna say, it's very w- it's actually very easy. So, people who are very data-informed, who actually want to l- to- to, uh, to embrace data, actually it's, it's actually very easy for you to do that. So, uh, uh, there's also what and how there, which means, like, uh, what do you want to say and how you say it. And so if you actually are not doing that very well... So, one thing is, like, I may just not do a good job of influencing. So once... Say, if you have the right data and being able to write and be able to influence, I think there are multiple reasons. One is the, the person that, that you're talking to. If they're not receptive, it's very hard for you to influence. Second, if you're not doing a good job, you may not be able to influence. Third is that you m- the answer may be right, but if you don't have the right data to convince people, you may not be able to. And the fourth, I think, is just, um, biases all around. I think if people have strong biases about what they want to see and don't want to see, I think it's very hard. So, for me, I think, in particular, uh, I, uh... In my earlier, earlier time I had Facebook, I think I struggled with influence largely because of the fact that I ha- I actually think that there was, um... There were specific leaders, and this probably what Alex is talking about, where who were not receptive to what I was saying. And as a result, uh, it became very hard for me to influence them, and it might have, uh... It both hurt me and, I think, the company a bit.

    3. HSHarry Stebbings44:26

       Has the way that you influence changed over time? Often, say when you're younger, you may take a more dogmatic, binary approach with passion and "Chandra, you've got to," and over... Has- has your approach changed?

    4. CNChandra Narayanan44:38

       Yeah, it's pattern recognition, uh, Harry.

    5. HSHarry Stebbings44:41

       Mm-hmm.

    6. CNChandra Narayanan44:41

       I, I actually think that over a period of time, you do so much of it because that's all I practiced. At Facebook, if you think about, uh, what I did the most was influence over a period of time. And then you have so many different characters, and each of them just becomes a cast of characters, and you start to understand what ticks with them and what does not. And once you do that, there is a way of approaching influence. So, I actually think it's, uh... I, I, I, I... It's an art. You have an idea and say, "Oh, this type of person, they can be influenced through data." You need to be really good at storytelling. For example, Harvey, you go t- tr- try to influence Harvey, you just throw the data, "Don't give me anything other than just data. Just tell me the data. Don't tell me the story." You go to Chris Cox, "Don't just throw data at me, tell me the story, please." So there is these types of... And both are extremely well-meaning, but there are multiple different spectrum of people that you work with, and you have to understand who they are. I also found that when I worked with Sequoia, I think I found four different types of founders. People who love data and knew data and wanted to embrace, that's one. There are people who love data, thought they knew a lot, but would not listen to you because they kn- thought they knew more than you. And then there was a third who did not know data, but said, "There is something out there, come help me." And that was fine too. And there are people like, "This is crazy. This is so stupid. I don't believe in data. It's all should be d- design. This is crazy." So, you have all of these types of founders, and some of them, you just can't. The people who are like, "I'm going to be closed, have a wall in front of me," you just can't influence. There's no way to influence. But the people who are like, "I don't know much, but I am hungry," you can. The people that are, uh, that are... That know data but they say, "I want to know more because I want to learn from people around me," those you can. The people who are adamant and saying, "I know more than you," very difficult. And I've had all these four. So you got to judge that of who are you talking to and then, then how you tell the story. Y- you tell a story? Do you just show data? How much data do you show? Do you go deep into storytelling and tell them every small thing or just peel layer after layer after layer? And you need to learn that while you're talking to them. And then on the fly, be ready to influence. And that's a very hard thing.

    7. HSHarry Stebbings46:57

       What are the biggest mistakes people make when trying to influence people, have you found?

    8. CNChandra Narayanan47:01

       The biggest one I used to make, and I think

16. 47:04 – 48:27

    Mistakes in Influencing

    1. CNChandra Narayanan47:04

       most people make, is the, um... Is, uh, is confusing the what and the how. Is like, what message do you want to deliver and how do you deliver the message? And so when you deliver the message, we are all human beings. If you can do it in a way that is easier for that person to handle, I think, or they... It can resonate with them, uh, or all the influencing skills I talked to you about, then I think it makes your job much more easier. I've seen many, many people try to influence but they, they fail. I mean, for example, I knew a, a very senior leader at, at Facebook who would be like, "I'm just..." They'd be so blunt and say, "This is it. Take it or leave it."... or, and it will never resonate because they didn't try hard to influence. They just wanted to say the facts but they didn't try to say, "Is that person actually..." I mean, at the end of it, influence is not about just saying, it's about having the impact, making sure you're able to influence into making a change in the direction that you think should be the right way to do it. It's not just about saying things. It's actually making the change. For that, you need to go far more than just saying things in any way you want. So, you've gotta start to understand people and psychologies and all these things.

    2. HSHarry Stebbings48:20

       I said to someone the other day, you know, "No, I'm very direct and clear in communication." They said, "Uh, doesn't matter. It's not what you

17. 48:27 – 51:54

    Hiring Exceptional People

    1. HSHarry Stebbings48:27

       say, it's how it's heard."

    2. CNChandra Narayanan48:28

       Exactly.

    3. HSHarry Stebbings48:29

       And I was like, "Huh. That's a very annoying answer." (laughs)

    4. CNChandra Narayanan48:34

       (laughs)

    5. HSHarry Stebbings48:36

       Um, but I totally agree with you and I love that separation between the what and how. I do want to touch on the hiring process. You've hired many incredible teams. I spoke to Alex about your hiring process and the incredible people you brought. How do you think about the hiring process for how you add great people to teams?

    6. CNChandra Narayanan48:53

       Yeah. I think very dimensionally, which basically means that I don't think about a person as a person. I think of them as a body of skills. So, what that basically means that are they peaking in one or two or three different things and not a liability in the others. That's kinda how I think about every person. So, the point is that, so if that's what it is, I would think like, oh, this person is ex- exceptional in A, B, C. They can bring a lot to the table. At the end of it, what you're trying to do is hire a bunch of different people who are excellent at what they do, that they can learn from each other, and become better each day. So, what you're really trying to assemble, overall, is, if you think about why anyone comes to work, it's for four, four different reasons. One is they love what they do. Number two, they love the people they work with. Number three, they feel like they can learn from the people that they work with. And four, the company is going up and to the right. If one of these four does not work, they will leave. So, that's what they're trying to do. So, essentially what we mean is if you have the people with the same skills, all of them being exactly the same skills, they're not gonna learn from each other. If you're gonna have people that you feel like you don't love working with and they're all jerks, how are you gonna stay? If the company is not doing very well, it's actually a company that was great 50 years back and doing nothing now, are you gonna stay? No. The company has got to go up and do right. Finally, do you love what you do? You come in to do something. You came to do design, or you came to do growth marketing, or you came to do analysis, or you do something. Do you actually enjoy your job? And if all four of them fit, you do a great job. I mean, you love it. So essentially what happens is when you are trying to hire-

    7. HSHarry Stebbings50:30

       I mean this in the nicest way. Do you actually believe that though? I mean, that's why Morgan Stanley and Goldman Sachs exist (laughs) . They do not love what they do in a lot of cases. They do not love the people. They think their boss is an asshole. Uh, but they get a massive comp package and the year-round bonus is 200% base, and they live a nice li- nice life in New York or London or wherever, and they stay for years. I've got many friends who are gr- like, great talents and they're at shit companies but they love their team.

    8. CNChandra Narayanan51:04

       I agree with you that there are reasons why people may stay but you ask me for the kind of person that I want to hire and what I want on a team and what, what really makes them come every day. The thing about the Morgan Stanley guy who you're talking about, they're probably working out of fear or they're working because of some boss. If you ask the same question of, is, are you getting the best out of the person, him or her, I don't think you are. It's hard for me to think over a long period of time. Maybe over a week or two, you can. But a period of a year, I don't think you're gonna get the best out of the person. So, if you want to get the best out of the person, they really need to do- enjoy what they do. It's about making sure that they come in, they're not, they don't come to work with fear. They come with, with loving what they do and building a bottom-up culture that they can be happy and wonderful at.

    9. HSHarry Stebbings51:47

       You said about, kind of, the body of skills that you want to really see them spike on. When you think about, kind of, growth teams and analytics teams in particular,

18. 51:54 – 55:05

    Skills for Growth & Analytics Teams

    1. HSHarry Stebbings51:54

       are the, is that body of skills a finite, small, limited number which you want to see or is it largely, large and expansive?

    2. CNChandra Narayanan52:03

       I think there are a few things. Uh, one is I think, uh, from an analytical perspective, um, uh, I- I'll talk about the body as skills and then I'll talk about the outcomes. Uh, the body of the skills would be, like, are they able to break a problem down? That's becomes an important thing which basically, uh, is, like, if the, if you take, if you gave them a complex problem, are they able to simplify it and make it essentially converted into a, uh, uh, uh, a business question into a technical question? And then from the technical question, can they convert into a data question? And then from a data question, are they able to do the right type of analysis? And then from the analysis, can they design the right type of insights? And then from those insights, can they convert into the actionable insights? From the actionable insights, can you convert it to opportunities and decisions? So, there's a mindset of how, and the first few skills that you have is breaking it down is more like a consulting skill. It's like a mindset that you need to have. The second skill is a technical skill which requires some amount of coding, et cetera. The third is analysis skill, and the fourth is basically, uh, a lot of it w- has to do with, um, uh, with the fact that you, you, I mean, in terms of synthesis skills and influencing skills that you need to have. And if you did all of that, you can actually do what you can actually, uh, do what you can do great at, uh, at, uh, in terms of analyzing a funnel, analyzing retention, or analyzing any sort of problem that you want and identify opportunities, set roadmaps for that, and trying to, uh, move, move the thing forward. I would also say that in terms of analytics itself, there are literally only two things you need to do. One is indexing.... which is, you wanna see if things are unindexed or overindexed. In anything, you're comparing and benchmarking. And the second thing is asking the so what question. There are only two things. Underindex, I mean, so for example, u- overindexing and underindexing means that if you have a certain, uh, n- if New York is going up, you're asking if Chicago also is going up. You need to benchmark to something. And if s- if New York is going much more up, it's overindexed, and so then you want to pry there's an opportunity of some kind. I'm just simplifying it. And then, if New York went up by .001%, is it material? And that's the so what question. And that helps your priorities. So essentially, analytics, literally you need to know only how to index and how to, how to ask a so what question. If you do that really well, that's really the two skills that you need. But in order for you to even get there, you need technical skills. You need all the other kinds of skills because you need to munch data. And then, being able to tell the story, put it together and influence, right? That's the other part that he said. You can actually do the so what, do that, but if you don't know how to influence it, it's a problem. That's a slightly different dimension. The influencing is a different dimensional. Technicals is, technical is a different dimension. But the analysis itself, the core to it, is basically only two things.

    3. HSHarry Stebbings54:58

       Can I ask you, I'm, I'm sorry. I'm very naive and ignorant, so forgive me for my stupid questions. But on the indexing side, does that not

19. 55:05 – 58:55

    Importance of Indexing

    1. HSHarry Stebbings55:05

       just relate everything to the average? As we said that the index, and we wanna be exceptional. Think about LPs in funds. The index in venture is shit. We wanna be in the top decile, which is you name your fund.

    2. CNChandra Narayanan55:19

       Let me give you an example, right? I, I worked with MongoDB while I was at Sequoia.

    3. HSHarry Stebbings55:23

       Yeah.

    4. CNChandra Narayanan55:24

       So, at Mongo, what we basically did was we had a, uh, we basically were looking at the pay- payment conversion rates. And, uh, I went in there, and they had a data science team and they had an analytics team, and they'd worked on it for s- I, for, I don't know how long. But I went in there, t- took a quick look and found out that Germany had a very low conversion rates compared to its neighbors like France or, or, uh-

    5. HSHarry Stebbings55:48

       Mm-hmm.

    6. CNChandra Narayanan55:48

       ... the rest of Western Europe. It had had a far lower payment conversion rate and I was like, "It shouldn't be. It should be similar to the other European countries. Why is it so low?" And that's the benchmark. It doesn't have to be the average. It has to be what you think is a similar country or similar something. And so, in this case, it turned out that Germany is very different in that's it, it's, it's a very bank country. It's an ACH country, so they use banks a lot. Whereas the r- rest of the country used credit cards a lot. And it turned out that MongoDB did not have a way for people to pay using, uh, financial instruments using banks. They only had a credit card. So, people would go try look at bank. There was no bank, uh, bank available, so they would just not convert. And so, what I did was I told them, "Hey, guys. Go and, and, uh, change that." And I think it materially changed the topline of, of the companies, uh, in terms of what it was. I think, I, I think they told me, I don't, I don't know whether it's true, but it's, uh, it's a few percentage increase. Overall increase, uh, in terms of, i- in terms of conversions because Germany was a big country and it was converting at abysmally low rates.

    7. HSHarry Stebbings56:52

       What does it take to do so what well? Like, what does a good so what lead to?

    8. CNChandra Narayanan56:57

       The so what is all asking the question of, okay, this thing increased by .1% or .5% or 1%, so what? That's all it's asking. And if you ask the so what question, you are basically saying so what? Yes. Oh, if I do that it'll increase my wow, my North Star Metric, or MAU, my North Star Metric by 100,000 users, or by 200 users. So, if it's 200 users, you see like, duh, it's no value to me. But if it's 100,000 users, wow, that's super cool. That's what I wanna move. So, when you look at the so what, you have to have a sense if something is material or not. And material basically means you have to have a sense of what your overall goal is. If you have an overall goal, is this the highest opportunity you can have or one of the high opportunities that you can? So you've got to tie whatever you're trying to do back to something into a North Star Metric. Even if it's not, this goes to the art and science part, even if it's not an exact, you should have a sense of how much you're gonna drive. Without asking the so what question, that's a problem. And what I've realized was the difference between good analytical people and the, who can be good at insights, but the ones that can, that are good at actionable insight, the muscle is really the so what question. It's not the analysis. They can be great at coming up with lots of indexing and analysis, but coming up with, they don't ask so what questions enough. And that's what I've seen differentiate very, very good analytical leaders and not, and who can take basically data into action better because they can, they know how to go from insights to actionable insights.

    9. HSHarry Stebbings58:31

       I think it's very similar to, like, growth investing though, actually, and the difference between a junior and a senior, which is someone who can, you know, just look at data in isolation and that's fine, versus one who ties it to a core decision because of the data that they've seen. That's the difference. So, I totally agree. Are there questions in the interview process that you will most frequently revert to to understand their abilities,

20. 58:55 – 1:02:05

    Identifying Performance Issues

    1. HSHarry Stebbings58:55

       their spikes, their skills, as you described them, a body of skills?

    2. CNChandra Narayanan58:59

       The more, more and more senior you get, the number one, and probably the only thing I care about is their ability to simplify. And, uh, I remember Chris Cox once told me that, I, I asked him, "What is the, what is the one single thing that senior people can do?" He said, "Simplify." That's the same thing I have is like, can you actually simplify? When you look at a very senior person, I look at it from s- if they can simplify. Simplify shows clarity of thought. Clarity of thought shows your first principles thinking. So, if you have great first principles thinking, you have higher degrees of clarity of thought, which leads to simplification. And so, the more senior you are, the kind of skills that I look for is can you actually simplify, which basically means that you can take very complex problems, break it down, and you can actually take on harder and harder problems because you're thinking first principles.

    3. HSHarry Stebbings59:51

       How do you test if someone's a good simplifier?

    4. CNChandra Narayanan59:54

       So, what I do is I actually have a blog.I have a blog that I have written, I don't know whether you guys have come across it, but-

    5. HSHarry Stebbings1:00:01

       I saw it in the prep, yeah.

    6. CNChandra Narayanan1:00:02

       ... please... Okay.

    7. HSHarry Stebbings1:00:03

       That's great.

    8. CNChandra Narayanan1:00:03

       I- I- I wrote a series of blogs while I was at Sequoia. And when I did the series of blogs, I wrote one on- on, um, on sustainable growth. And on the sustainable growth, I have them read it. I send them and say, "Go read it." And there are things in there which is not quite right. There are things in there that make sense. And I basically, first question I generally ask them is, synthesize. Just tell me, what are the primary takeaways? So those are the kinds of things. So, I go through every single thing. And it's not just that I go deep into retention, I go deep into growth itself. I ask them, uh, things that you ask me. It's like, "What is product market fit?" And, uh, I see how they think. Uh, can I have a dialogue with them? Because it's- when you are interviewing for these people, I mean, I know what's on their resume. I mean, t- it tells you- me a bunch of different things about their resume, like, oh, they can code, or they can do this, or they can do that. I mean, that's- sure, that's- I just take that for granted. On top of that, I'm trying to think, in a working environment when I'm talking to you and I'm discussing with you, can I actually have a conversation with you? And then, in the moment, can you think deeply? And that's what I'm trying to look for.

    9. HSHarry Stebbings1:01:08

       It's funny, I say the best interviewers are able to have a very defined schedule but in real time move with conversation flows to make it-

    10. CNChandra Narayanan1:01:14

        Yeah.

    11. HSHarry Stebbings1:01:14

        ... a very natural conversation. And then you have to bring it back. That's very hard. But it's to your point there, about like real-time conversation and being able-

    12. CNChandra Narayanan1:01:22

        Yeah.

    13. HSHarry Stebbings1:01:22

        ... to move with it.

    14. CNChandra Narayanan1:01:23

        Exactly. And that's what I do. I tell them also upfront in the interview, I tell them, "Look, no answer is wrong- wrong answer."

    15. HSHarry Stebbings1:01:30

        Huh.

    16. CNChandra Narayanan1:01:30

        "I only care about how you think. If you are on the wrong track, I'll immediately tell you." Because some of them are objective. So I say, "I'll immediately t- tell you what is, but I won't judge you on it. It's fine." I- all I want to do is that I- I want you to have... if you're giving- g- giving me ten idea- ten ideas and eight of them are good and two- two are absurd, I won't judge you on it. Because inside a great company, those two will be weeded out anyway. But I don't want those eight not to be there, so say whatever you want.

    17. HSHarry Stebbings1:01:58

        (laughs) What are the biggest hiring mistakes you've made?

    18. CNChandra Narayanan1:02:01

        Early on, the big mistakes that I- I used to make, and I make less of it

21. 1:02:05 – 1:09:08

    Hiring Mistakes

    1. CNChandra Narayanan1:02:05

       now, I think I probably still make them, is, um, I think about slope and asymptote. Asymptote is how good are you, slope is how fast are you growing. And so what I would hire more for is people with asymptote, which basically means that, oh, they have such a good profile, et cetera, they've done so much, et cetera, but they won't look at the growth and they may actually flatten out on the growth part, and that is probably... Much of my mistakes, you go back to it, one big one would be the hiring for asymptote rather than slope. Now, I don't mind. You can imagine whatever, it's just a matter of catch-up, right? People with the slower asymptote or the faster growth, yeah, is- are going to overtake. And I generally tend to invest in people over a very long period of time. But within reason, within two people, within slope, and this- I know that basically the- the effect of compounding will just overtake the person and if you're willing to invest over time, you could.

    2. HSHarry Stebbings1:02:55

       I- I- I totally agree with you. Do you think people are destined for a certain stage of a company's life cycle? You've seen many different stages and I'm just thinking about the slope plateauing. Do- are certain people destined for certain stages?

    3. CNChandra Narayanan1:03:08

       If you're below a certain age, you do- you're not institutionalized. Let's say you're below 27, 30, whatever those ages are, you're probably not institutionalized. Un- until that, I think it's very easy to mold anyone. But beyond a certain age, I think it's so much harder. If you're actually in a certain type of company, you're set in your ways, you have certain ways of thinking, it's that much harder for you to change. I don't think it's impossible, that's not hard. But I would say that if you've developed the growth mindset very early on, which means that even if you're 31, 35, and you've developed a growth mindset very early on and your mind is nimble and flexible, you can take on almost anything. It's like, how early did you get on your- did you develop a growth mindset? And if you did, I think you can continue to challenge yourself on anything. But if you didn't, I think you need to catch them young.

    4. HSHarry Stebbings1:03:57

       What was the last thing you challenged yourself on?

    5. CNChandra Narayanan1:04:00

       I think if you look at my career, I am an oceanographer by training, and then I went- did- I did- I went to- w- I- I was a professor and went into high-performance computing, and then I did weather forecasting, then I went to climate change. I was- uh, is in risk management at Facebook, then I move to analytics at Facebook and led ma- many of the teams there, went to venture capital and did a bunch of different things, and now I'm doing a startup. So, I think my journey has been one which has always been like trying to disrupt myself, to be honest with you. And I think the h- the biggest challenge now I have is, uh, is, uh, essentially, I- I think two things. One is growing the company itself, and I think growing a remote team in India. And, uh, the growing- the remote team in India had its own challenges, uh, partly because of cultural reasons of India and, uh, the- and the fact that I think, uh, not being able to role model, uh, easily because we were in the US and they were in India, the rest of the team was in India. And so I think that has been a- uh, that has been a challenge in itself. And so on, but I think-

    6. HSHarry Stebbings1:05:08

       Do- do you find it tough? Because at Facebook you are inundated with data. You know, you put something live and there's 100 million people on it, if you want to, by the end of the day. With- with any new product it's like you fight and claw for every new customer. Is that tough to transition to?

    7. CNChandra Narayanan1:05:23

       Yeah. So, I think what I've started to develop while at Facebook was creating frameworks to think. What it is basically, when you start to develop frameworks, like a formula is a framework, right? And at Facebook we used to de- develop form- formulas, like, you have a formula for the entire company, which is if you look at revenue as the driving metric that you care about, you have number of users and then you have time spent per user and then you have ads per time spent is formula. So those are ways actually to think in terms of frameworks. So if you think about things in frameworks, you don't need that much of data. What you really need to think about is, in- in terms of that, uh...... and which is why when I went to Sequoia, there's so little data. Compared to Facebook, there's so little data. And then I needed to kind of think about abstractions. How do you create abstractions and frameworks? So then what we realized what all the companies at Sequoia, for example, we classify for the types of company phase that Sequoia had was eight different types of companies; e-commerce, two-sided marketplaces, uh, you know, consumer subscription, consumer ads, and, and, and, and, and, and SaaS obviously, and so on. So, there are these seven or eight different types of companies, eight formulas. And then what tends to happen is that then you don't need to have a lot of data. You, all you really needed to have is a way of thinking. And for that, you need to have a way of thinking from first principles and if you can do that really well, you don't need a huge amount of data. I always have believed every large data problem can be... can actually be brought down to a small data problem. Every large data problem can be brought down to a sm- I'm being extreme here. Obviously, nothing in gen AI is like that but, um, in terms of analytics, I think it's actually true. It's like most large data problem can be reduced to small data problems and then you're solving small data problems. So, uh, essentially if you think of it that way, you don't need large data to solve problems. You need the right data for solving the right problem and you need to piece it out. That's the art.

    8. HSHarry Stebbings1:07:14

       I love that, how this conversation goes. Uh, can I ask you, uh, we... going back to the actual talent, h- how fast do you know when you've made a mis-hire? Again, you've hired so many people.

    9. CNChandra Narayanan1:07:24

       Three things I look for. One is I think you look at skill gap-

    10. HSHarry Stebbings1:07:28

        Mm-hmm.

    11. CNChandra Narayanan1:07:29

        ... knowledge gap, and value/culture gap. Those are the three things I expect. If somebody is not doing well, performance not doing well, because somebody's not performing, that can be easily found out. More easily found out if someone is not performing in terms of, uh, how they're delivering, you can find out. But not all people who are not performing, uh, will do badly. It may be that they're in the wrong role, or it may just be that they're deep thinkers. I've seen that too. Or they may just not be good at what they're trying to do, and so on. So, for that you need to analyze three things. One is, is the problem that they're trying to do is a skill problem? This will be... their general na- their, their graded Python, but this is C++ problem, they can't do it. Okay, that's why they're slowing down. Second is like knowledge problem. It's a Python problem, they know Python really well, but this space is security and they don't understand security, so they can't do it really well. Values is like are they not working hard or any other cultural things that may be. You have to first identify what the problem is and as soon as you identify it, if you think it is fixable, then you need to put them on the right track and see what happens. So, I think identifying whether or not they're performing is an easier problem which should... you can actually do, talk to peop- uh, not just talk to, but just look at the pr- productivity and see what's going on. But to diagnose what the reason is, is, is the one that you need to go after. And I've done... often, many, many times, I've found that the reason why they're not doing well is because they're not in the right, uh, they're not on the right team or not in the right role or in the right thing, and you just need to shift them and then suddenly they become a force multiplier.

    12. HSHarry Stebbings1:09:01

        That's, uh... That's so interesting. I interviewed someone the other day, a, a founder. I can't remember who. I think it was Brian at Hou- at, um, HubSpot. And

22. 1:09:08 – 1:11:48

    Timing of Performance Improvement Plans

    1. HSHarry Stebbings1:09:08

       he said that when you put someone on like an improvement plan, it never works. Just get rid. Do you agree?

    2. CNChandra Narayanan1:09:16

       I think when you put up- p- someone on a performance plan, it's three months too late. It shou- three months before you should have intervened and tried to do the right thing. What happens most of the times is that people put them on the improvement plan when they've already decided there's no chance of success. And that is why it doesn't work. If you think there's a 20%... if, if you think there's an, uh, 70% chance of their or 60% chance of them succeeding and you put them on a performance plan, then it might work. But what ends up happening is that we time it so badly, we time it at 10% or 5% or not even 1%, and I think it's the timing which is the problem. And most managers take too late to intervene and so... and they're not willing to have honest, direct conversations. And because they don't have direct conversations, it becomes that much harder largely because most of us are conflict-averse and we won't have those conversation and it becomes too late. And then what do you do? Try to go into a m- m- performance plan. I think it's less about the performance plan is when you time it. But regardless, I don't think you need to call it that. Just why do you call it? The, the label is not great anyway.

Episode duration: 1:22:56