An inside look at Mixpanel’s product journey | Vijay Iyengar

1. 0:00 – 4:12

   Vijay’s background

   1. VIVijay Iyengar0:00

      The issue for us at the time was that we took people away from the investment in our core product to go do those other things. Like, we, we moved people, right? And so the, the trap there is that you leave yourself ripe for disruption in your core because someone else can out-invest you in, in that core. And so if you're the leader in some core product, our takeaway here is you should continue to out-invest everyone else in that core and then invest, you know, the profits that come out of that core into the next venture. Like, invest profits and not people. Or, or venture capital, which is maybe, like, net present value of profits or something to that effect. But don't take people away from the core to go do those other things, 'cause then, then you end up distracted.

   2. LRLenny Rachitsky0:37

      (instrumental music) Welcome to Lenny's Podcast, where I interview world class product leaders and growth experts to help you get better at the craft of building and growing products. Today my guest is Vijay Iyengar. Vijay is currently head of product at Mixpanel. He actually has a very similar career trajectory to myself, where he started as an eng intern at Amazon, then he was an engineer for a while at Uber, then he became an eng manager at Mixpanel, but then he shifted from an eng manager to director of product, and now head of product at Mixpanel. You don't often see people moving from an eng leadership role straight to director of product, so it was really interesting to hear what he took from his eng experience and brought into his approach to product leadership. But we spend the bulk of our time talking about what he's learned from the journey that Mixpanel has been on, where they started with a simple product and scaled to a number of different products, solving many problems for customers, and then made the hard decision to scale back to just a single core focused analytics product. We talk about why they made that choice, what they learned about when it makes sense to expand to new products and when you probably shouldn't, and how they approached that organizationally. We also talk about how Mixpanel builds product, how they think about product philosophy, how they prioritize, and also what you're probably doing wrong in how you set up your analytics for your own product. With that, I bring you Vijay Iyengar after a short word from our wonderful sponsors. This episode is brought to you by Pando, the always-on employee performance platform. How much do you love the performance review process? Mm, yeah. It's time-consuming, subjective, biased, and there's rarely any transparency. With the rapid shift to distributed work, it's a struggle to create the structure and transparency that you want to help your employees have the highest impact and growth in their careers. Pando is disrupting the old paradigm of performance management, including a continuous employee-centric approach so employees stay engaged, see their progression in real time, and know exactly when and how they can level up. With Pando, managers can leverage competency-based frameworks to effectively coach and develop your teams and align on consistent growth standards, resulting in higher quality feedback and higher performing teams. Visit pando.com/lenny for more info, and get a special discount when you sign up and reference this podcast. That's pando.com/lenny. This episode is brought to you by Notion. If you haven't heard of Notion, where have you been? I use Notion to coordinate this very podcast, including my content calendar, my sponsors, and prepping guests for launch of each episode. Notion is an all-in-one team collaboration tool that combines note taking, document sharing, wikis, project management, and much more into one space that's simple, powerful, and beautifully designed. And not only does it allow you to be more efficient in your work life, but you can easily transition to using it in your personal life, which is another feature that truly sets Notion apart. The other day, I started a home project and immediately opened up Notion to help me organize it all. Learn more and get started for free at notion.com/lennyspod. Take the first step towards an organized, happy team today, again, at notion.com/lennyspod. Vijay, welcome to the podcast.

   3. VIVijay Iyengar3:52

      Thanks, Lenny. Great to be here. Huge fan of the, the pod, so glad I can contribute.

   4. LRLenny Rachitsky3:58

      I definitely want to talk about Mixpanel's journey, both as a product team and a product. But before we get there, as an engineer, you were a longtime engineer and then you became a product leader. Is there anything you had to unlearn as an engineer in the way you thought about leadership and product and,

2. 4:12 – 7:15

   How Vijay learned to be more open-minded to new ideas

   1. LRLenny Rachitsky4:12

      and business?

   2. VIVijay Iyengar4:13

      One of the things, if, if you've been in engineering for a while, is that you develop this, um, tendency to immediately respond with no to new ideas. And I think from the engineering perspective, this is because you spend a lot of your time building and maintaining ideas that maybe are half thought out or didn't really go anywhere, and you just feel like the full brunt of the, the maintenance cost of that. And so you build up this, this scar tissue and this immune response to say no to new ideas. And, and you... It's a hard no, like, "No, we're definitely not gonna do it." And I think I had to unlearn that moving into product, because, you know, you get a lot of ideas coming from a lot more places in the organization and ideas are, are fragile in their infancy. And it's... You know, a hard no can really kill a whole direction that you could potentially go that could be very high reach and high impact. So one thing that I found is the best way to get to a no, if, if you ultimately need to get there, is, is to try to make it work. Like start tr- try to make yes work, and, and document how you've tried to make yes work. And do that earnestly, not as a, you know, as an exercise of just an alternative that you're considering. Try to do it sincerely, and get to no after trying to make yes work. Um, and so that's, that's one thing I've been trying to just apply my engineer problem-solving brain to, to do that instead of thinking about how it might not work and, and saying no.

   3. LRLenny Rachitsky5:25

      Is that something that you recommend engineers work on? Like, looking back. I know as, as a PM it's like, "Oh, I love when engineers say yes. This is awesome. I'm gonna help everyone learn to say yes." But as an engineer, obviously that's a challenge often. What do you recommend to folks that are engineers currently that maybe want to improve on this or, or shift in how they think about saying yes, saying no when they're asked about something new?

   4. VIVijay Iyengar5:48

      I think some of the best engineers that I've worked with actually a- already do this. By, by default they're able to balance both in their head. 'Cause ultimately it's this balancing act of you just went on call, you were woken up three times at 3:00 AM due to var- various bad ideas, and the next morning you wake up and then stand up. It's like, "Hey, can we do this new thing?" You're, you, you kind of have to have that empathy and, and do that.So I think the exercise is just, like, just take 10 minutes to consider the idea and just sincerely consider how might we make it work? And if at the end of those 10 minutes it's, like, futile and, and there's no path, it's fine to say no. And it's a good, good instinct to say no a- actually in a lot of cases. But yeah, I would recommend that to engineers. I think it would have been better in my career for sure if I had learned that sooner.

   5. LRLenny Rachitsky6:26

      I want to spend some time on the Mixpanel product journey. It's been an interesting, uh, roller coaster. I think the company's been around for how many years? Since 2010, 2009?

   6. VIVijay Iyengar6:34

      2009, yeah.

   7. LRLenny Rachitsky6:35

      So I know it started as kind of a very simple product analytics product back in the day. And then as you do with ambitious companies, you look for more problems to solve. You look for more problems to solve from your customers so as I understand it, you guys added a lot more products-

   8. VIVijay Iyengar6:49

      Yeah.

   9. LRLenny Rachitsky6:49

      ... to the suite of Mixpanel products, and then I know that there were some challenges with scaling that and maybe the products didn't stick as much as you were hoping. And what I understand is recently you moved back to just a single core analytics product. And so I'd love to just hear that journey of what that process was like, what you learned as a product leader and as a company about kind of scaling, expanding, trying to solve a lot of problems and then coming back to one core straightforward problem.

3. 7:15 – 12:53

   Mixpanel’s journey

   1. LRLenny Rachitsky7:15

   2. VIVijay Iyengar7:15

      Mixpanel started in 2009, uh, as provide product analytics to EPD teams and I think early on it saw a lot of success because it built this in-house database called ARB which stands for Arbitrary Segmentation. And that was necessary because events data, which is the fuel for product analytics, is few orders of magnitude larger than, than most other types of data that people collect and so you need a specialized approach to deal with it. And so that, I think, spurred the first wave of explosive growth because product analytics was a really burning problem at the time. People were shipping mobile apps like crazy and they needed a solution that could scale and that was kind of a durable mode for Mixpanel for a while. And I think because we had this SDK that was installed in so many apps and we had this really scalable event collection and analytic interface, it was just natural to expand into a few adjacencies that would leverage those same technologies. The first one was messaging, being able to send targeted messages to, to users which is, you know, something that's fairly natural you might want to do it, especially if you have an SDK already installed. Yeah. The other aspects that we've added into was data infrastructure and trying to be sort of the single source of truth of data in companies. And what ended up happening was that by 2018 we, we had this big churn problem. We had something like 40% churn, revenue churn, our core product and when we dug into it, it wasn't that people were churning because they didn't need product analytics anymore. They had the need. They were just churning to competition because we were just not up to the market in terms of the features we had in our core. And when we dug into why that was, it was just that we had a 50% engineering team that was building products across three domains, right? Product analytics, messaging and data infrastructure stuff. Our engineering team was just spread too thin to address all of those core gaps in, in functionality and so we made a, a really hard call at the time. We said the hard no to those two other categories and decided to focus our entire engineering team on closing the gap on product analytics and, and innovating there. And from a process standpoint, how we operationalized this was we threw away all our planning and all the execution and the work that we, we planned to do so far and we did something very simple. We took all the churn reasons that our customer success and sales teams had been painstakingly collecting for years, grouped them by category which was like roughly product features we needed to build. Sorted descending by ARR, took the top 10 things and made that our roadmap. I just gave eng- every engineer, you know, direct access to customers and gave them a bucket to go work on which I think goes against about a million product best practices out there of just doing that but I think given the context at the time, we needed to optimize for speed and speed comes when you have extreme clarity on what you want to do and, and focus. And so we really just optimized for speed in that time. And so in that first year we, we moved really quickly and we shipped something like 100 features in that year and closed a lot of gaps. Again, not... These are all vanity metrics, like measuring number of features doesn't mean anything.

   3. LRLenny Rachitsky10:04

      And what, what year was this by the way?

   4. VIVijay Iyengar10:06

      Uh, this is 2018 to 2019.

   5. LRLenny Rachitsky10:08

      Mm-hmm. Got it.

   6. VIVijay Iyengar10:09

      Um, yeah, so we moved really fast shipping all these features and instantly saw the improvements to, to, to win rate and, and to retention but, you know, one of the, the cracks that started to emerge was we neglected the holistic design of our, our product at the time, right? And if you're, if you're shipping features that quickly it's, you don't have time to stop and think like where does this go and how does this fit into our overall system architecture? And what started to happen was that we were hitting diminishing returns with some of these features and, like, not considering the holistic design and consistency meant the reach of every feature was low, right? Like, you had to rebuild it for every part of the product that, that we were in. So at the time we made, we spun off to SecondStream that was very design led and I think this was also coincided around the time we adopted Figma and it's a really broad design at the seat of the table of the company and, and we just set out this goal to make design one of our, our key differentiators. So this, this design driven initiative was really about how can we think about the system architecture of our product? What are the key building blocks of Mixpanel? Where do they need to fit? How few of them can we have which is a really important step and then how will users discover them and how do they relate to each other? And I think this realization was born out of the, the fact that like so many great products win or lose based on their architecture. I think Notion for example, like that pages and blocks architecture is, is so strong and you can hang so many features off of those core building blocks in a way that has such high impact on, on reach and, and discovery of those features. So anyway, we, we did that in parallel with the... And continued that grind on, on core gaps and so the end result of that phase which is from, you know, 2018 to maybe late 2021, 2022 was our retention went from about 60% to 90% and our NPS went from 16 to 50. So I think yeah, I mean, th- there's a lot in there to, to unpack but refocusing on the core really helped us, uh, achieve those results.

   7. LRLenny Rachitsky11:52

      Got it. Yeah, I have a lot of questions about this. So interesting. So that phase that you went through where you sorted things by potential ARR, was that the phase of expanding to multiple products or that was post we're going to focus on analytics and go all in there?

   8. VIVijay Iyengar12:06

      Oh, that was post focusing on analytics.

   9. LRLenny Rachitsky12:08

      Okay.

   10. VIVijay Iyengar12:08

       Yeah.

   11. LRLenny Rachitsky12:08

       Yeah. And you're saying that you had a stream of just, build all the features that were lacking that are causing customers to churn. And in parallel there was a track of, let's build this product such that it all connects and works together well and it's really well thought through long term.

   12. VIVijay Iyengar12:24

       The first thing, I might have made it seem like it was just the buckets were features. We, we did take the step of turning them into problems and being clear, like exposing engineers directly to the customers that had those problems, and then invent a solution to solve them. So I mean, it, you know, loosely there were features involved, but a lot of them are, were kind of core problems we needed to solve.

   13. LRLenny Rachitsky12:40

       That first approach is so interesting. Like, it's kind of like, yes, we will make more money if we focus on these features. To your point, it ends up being just a bunch of features and products that kind of maybe don't synergize. Looking back, was that a good idea to approach

4. 12:53 – 14:03

   When to optimize for speed

   1. LRLenny Rachitsky12:53

      it that way, at least for a while?

   2. VIVijay Iyengar12:54

      It highly depends on your context. In a very competitive context where there are just table stakes features that customers need and that it's been validated by the market, you need to optimize for speed more so than anything else. But it is an approach that outlives its usefulness pretty fast.

   3. LRLenny Rachitsky13:10

      Mm-hmm.

   4. VIVijay Iyengar13:10

      And, we, we put that approach behind us relatively quickly after that phase. Um, and I, I would actually say, like, that design-driven phase was the next phase where it was, okay, like, we're not bleeding on the table stakes anymore, but we wanna make a holistic product that has high reach, high impact on the features, and, and is actually usable. And so that was, I think, a, a, like a follow-on phase that's necessary. Obviously depending on your particular circumstances and competitive dynamics, you can sequence them differently, but I think it was the right call to just... Sort of like that on-call thing again where, you know, when you're in trouble, you gotta get out of trouble. You can't mow your lawn while your house is on fire. You gotta put out the fire and then deal with everything else.

   5. LRLenny Rachitsky13:47

      Yeah.

   6. VIVijay Iyengar13:47

      So that, that's kind of the approach we took.

   7. LRLenny Rachitsky13:49

      What's an example of a feature or product that you launched within that first track, and then what's an example of something that came out of the designer-led approach, if anything comes to mind?

   8. VIVijay Iyengar14:00

      I think out of the first track ... Oh man, there's so many that were

5. 14:03 – 17:26

   The feature phase vs. the design phase

   1. VIVijay Iyengar14:03

      just core. Like, we didn't have a good cohorts product at the time, like just being able to create behavioral cohorts of users and create them from any, any report that we built, right? And I think that, I mean, it's just table stakes and- and analytics to- to be able to do that. So that was one of the first things we built, and it was fairly obvious. There's a lot of other things in like more advanced types of funnel analytics and, and flows, uh, visualization that was, you know, really interactive. I think on the design-led phase, the biggest thing I think was visualization consistency and making our charts interactive in a consistent way across a- all, across our entire product. Um, and so there's two things that enabled. One was that every time you added a new visualization or a new enhancement to a visualization or like how something is sorted in one report, it just instantly applied everywhere. So just the reach was multiplied for everything to be added. And then the other thing is it just made the product more accessible, let us add dark mode. It made, made our visualizations really stunning and really easy to see what the takeaways were. And then every new visualization we added inherited all those benefits.

   2. LRLenny Rachitsky14:59

      I'm trying to think about, like, being at a company that goes through this phase of, hey, we're just gonna build a bunch of stuff that we know we need. And it feels l- like, hearing it, it's like, oh yeah, and then we're just gonna make it all look great and connect and work well. I imagine that wasn't planned, and I imagine that wasn't easy to get people to-

   3. VIVijay Iyengar15:17

      Yeah.

   4. LRLenny Rachitsky15:17

      ... maybe slow down on just building more products and features or push it in a direction where it's all gonna make sense. Can you talk at all about what that process was like, like how hard it was to shift from, "We're just gonna knock through all this checklist of things," to like, "Let's just-"

   5. VIVijay Iyengar15:31

      Yeah.

   6. LRLenny Rachitsky15:31

      "... figure it, let's slow it down, let's spend a lot of time designing"?

   7. VIVijay Iyengar15:35

      It was definitely more messy, uh, internally than I described it. One of the key junctures was when we had this really talented design team and we were putting them on these very tactical projects that, that was, like frankly like that was very engineering-led, right? And design would often come in at the end and be asked like, "Hey, can you just make this look nice and put some pixels on it?" And it's just such a waste of your design team to, to have them do that. But at the same time, the pace was so high that they didn't have time to come up for air and, and do anything else. Um, and so there's actually this moment where I was an engineering manager, a- a- and part of this, and, you know, had a meeting with our BM and our head of design at the time, and he said, "Hey, we can actually do the next three months of projects without any design." Which was a kind of controversial thing to say, but we're doing this so that you can take three months with a set of designers and go think about the system architecture of the product and we'll be, you know, we'll wait for that to be done before we do any architectural, things that might impact the architecture. And I think that gave designers, like, a bit of breathing room to go do that, just like separating them for a bit from, from the tactical fire. Because what was happening instead was, we would get towards the end of the project, bring design in, and they would use each project as an opportunity to squeeze in like, "Oh, and we can simplify here," and that's just a classic way to blow up scope at the end of the project because there wasn't a dedicated space for design-led projects. And I think that, that was kind of a key friction point that we ultimately had to decouple for a bit and then regroup and say, "Okay, now what? Like, what's our strategy?" And, and just take on projects purely for the sake of improving consistency, reach, depth of, of our UX.

   8. LRLenny Rachitsky17:02

      Also looking back, the process you went through of adding a bunch of products to solve more customer problems, something every founder and product team thinks about. "When should we add new product lines? When should we expand beyond the core?" I'm curious what you take away as a lesson and what you'd advise other founders and companies when it comes to, when is it time to expand and think about a new product and a third

6. 17:26 – 20:03

   The importance of not losing focus on your core product

   1. LRLenny Rachitsky17:26

      product and a fourth product?

   2. VIVijay Iyengar17:28

      I don't know if there's a hard and fast rule here. I can just maybe say what made sense and didn't make sense in our context. The issue for us at the time was that we took people away from the investment in our core product to go do those other things. Like, we, we moved people, right? And so the, the trap there is that you leave yourself ripe for disruption in your core because someone else can out-invest you in, in that core. And so if you're the leader in some core product, our takeaway here is you should continue to out-invest everyone else in that core and then invest, you know, the profits that come out of that core into the next venture. Like, invest profits and not people or, or venture capital, which is maybe like net present value of profits or something to that effect. But don't take people away from the core to go do these other things 'cause then, then you end up distracted. And the other thing, aspect of that is that...... those secondary products we took on were in categories of their own. And it's really tempting and you'll often get dragged into building, you know, a- a- ent- accidentally entering another category and then you'll end up building these bolt-on products that are the nth best in their category, right? And, and, like, the adjacent categories for analytics are, like, CDPs or message targeting or feature flagging or something. But, you know, there's not that many people th- that need the sixth best CDP or the eighth best feature flagging or the tenth best, um, message targeting tool. And it ends up being, you know, in aggregate will contribute 5 to 10% to your revenue, won't seriously accelerate your growth rate, and then takes engineers away from the, the core product. And so those were the circumstances that we were in. And I think if you're seeing churn to your competitor on your core product and you, you're not best in class on any of the other ones, then maybe it's time to reevaluate. And then the last thing I'll say there is that it's also 10X more painful than you think to cut mild successes (laughs) than, than anything else. And just organizationally it's painful and there's teams that have whole roadmaps and, you know, it's, it's, it's a really painful experience. So, you know, you have to think really hard before you, you kick those off.

   3. LRLenny Rachitsky19:18

      That is really, really insightful advice. Makes me think about if you bundle good enough solutions, there needs to be kind of this anchored tenet that, like, I will not give this thing up and I'll use, like, the third best version of something else if you have it.

   4. VIVijay Iyengar19:30

      Right.

   5. LRLenny Rachitsky19:31

      But if you're not that valuable and important, you're not gonna convince people to use something 'cause you, they're, you're competing against the best in every category.

   6. VIVijay Iyengar19:39

      Exactly, yeah.

   7. LRLenny Rachitsky19:40

      That is really interesting. I've been doing this kinda series on how different companies approach building product and I have a few questions I'd love to ask around the product development process in Mixpanel.

   8. VIVijay Iyengar19:50

      Sure.

   9. LRLenny Rachitsky19:51

      The first is just, like, how do you plan? How do you plan? I know it evolves, but just how do you plan currently? Like, how long are your planning cycles? How far ahead do you plan in detail? Do you use OKRs? Maybe, maybe let's start there, those

7. 20:03 – 20:53

   How Mixpanel organizes teams around buckets of problems

   1. LRLenny Rachitsky20:03

      three kind of sub-questions.

   2. VIVijay Iyengar20:04

      We have these kind of unsolved problems in analytics that, that we're going after. For us, that's, like, people always want more power, more simplicity, better data trust, faster onboarding, better collaboration, better price/performance. And so we largely organize our teams around those problems and, and those missions. One quick aside there is that, you know, some of those problems have a tension with each other. Like, power and simplicity, there's, there's a trade-off there, right? And w- we want one team to own both so that they can th- they're kind of forced to, to confront that tension and beat that trade off. And so that's kind of how we think about generally our, our product team is these cross-functional EPD teams, each of which that's focused on solving these long-lived paired problems for customers, or, like, our core analysis team focuses on that power/simplicity trade-off problem. In terms of planning,

8. 20:53 – 23:09

   Mixpanel’s most recent six-month time horizon planning cycle

   1. VIVijay Iyengar20:53

      the way it works is that we, we plan on a six-month time horizon, um, and I can talk about our most recent planning cycle actually 'cause we're, we're just completing it.

   2. LRLenny Rachitsky21:01

      Yeah, let's do it.

   3. VIVijay Iyengar21:02

      Yeah. Basically it, it started out with the, this strategy memo that our, our leadership team wrote that basically just conveys, "This is where we want to go as a company in the next year and here's how the product team can contribute most of that," and just established these key pillars. We shared that with the teams and they took that and also combined that with all the quantitative and qualitative context they're constantly consuming about the problem they're working on and then, and our customers and ideated and developed this series of bets for the next six months, which I think are to some extent similar to OKRs where a bet, the anatomy of a bet is that it's a problem we want to solve, our hypothesis on the solution, and then some plan to win, like, some plan to actually get there and, uh, a way to measure that you got there. And I think one of the unique things that, that we did relative to other companies that do planning is I think it usually is sort of this W process of there's the strategy memo and then teams ge- generate bets and then there's a review and then they go back and i- iterate and then they finalize. And we kind of collapsed the middle part of the W where myself and our head of design actually spend time with each of the teams actually ideating on the bets and participating in the solution discovery process, going into the, the jam sessions and adding Figma stickies ourselves with ideas and, and thoughts on things. Which we did because we aren't a huge product team and we, we're not gonna do, like, 50 things in a HAP. We're gonna do maybe 10 to 12 things. And so that's enough that we don't, we can do something that doesn't scale, uh, if that enables high bandwidth communication between us and the teams. And it ends up being more messy and unstructured I bet in that, in that phase 'cause we're just, we're in there contributing ideas as well, but by the end of it I think the team leaves feeling both more confident in their bets 'cause there's more thought that's gone into it and then more aligned top to bottom on, you know, why we made certain decisions. And so I think that, that's one thing that's different, and then we conclude that process with a road show where we present to the rest of the company and then get their feedback as well.

   4. LRLenny Rachitsky22:49

      How long is this process generally?

   5. VIVijay Iyengar22:50

      The teams did, uh, pre-work for a couple of weeks, like, two weeks in December, and then we did a two-week sort of sprint on, on solutioning and ideation in, in January. Like, the first two weeks of, of January.

   6. LRLenny Rachitsky23:01

      Awesome. And what's the end result of planning for each team? Do they deliver a document with like, "Here's our strategy, here's

9. 23:09 – 25:16

   Mixpanel’s favorite tools

   1. LRLenny Rachitsky23:09

      the big bets, here's a roadmap"? Is there a template you pass around? How do you kind of get to a thing that people share and present and comment on?

   2. VIVijay Iyengar23:17

      Yeah. I think there's basically three artifacts that are kind of linked to each other. So the first is, uh, we use Notion, um, and so we have a, a database in Notion called Bets which is where e- you know, each page in the database is a bet. And that has a template, yeah. So it's, it's, like, kind of roughly what I described with a few more sections of, like, what problem are we solving? What's the evidence of demand? Like, what's the reach and impact of this problem? How will we know we're successful? Um, and then what's the key driving hypothesis behind the solution? Um, and then a, a rough plan. And then that's tied with the presentation that's kind of like a, a tight summary of that that has, like, one slide per bet and then is also tied with more of an execution focus. You know, how do we sequence and staff this thing and eliminate dependencies which the engineering team contributes to? So I think those are the three artifacts that are, are linked together.

   3. LRLenny Rachitsky24:01

      This episode is brought to you by Lemon.io. You've achieved product-market fit. You're able to activate, engage, and retain your customers, but you don't have the engineers that you need to move as fast as you want to because it's hard to find great engineers quickly, especially if you're trying to protect your burn rate. Meet Lemon.io.Lemon.io will quickly match you with skilled senior developers who are all vetted, results-oriented, and ready to help you grow. And all that at competitive rates. Startups choose Lemon.io because they offer only handpicked developers with three or more years of experience and strong, proven portfolios. Only 1% of candidates who apply get in, so you can be sure that they offer you only high-quality talent. And if something ever goes wrong, Lemon.io offers you a swift replacement so that you're kind of hiring with a warranty. To learn more, just go to lemon.io/lenny and find your perfect developer or tech team in 48 hours or less. And if you start the process now, you can claim a special discount exclusively for Lenny's podcast listeners, 15% off your first four weeks of working with your new software developer. Grow faster with an extra pair of hands. Visit lemon.io/lenny. I know you have some

10. 25:16 – 26:57

    The RICE framework for prioritization (and when to ignore the C and E)

    1. LRLenny Rachitsky25:16

       insights on prioritization and some strong opinions on how to prioritize. Can you talk a bit about that and how you advise your product teams to prioritize?

    2. VIVijay Iyengar25:24

       One really common framework in prioritization is RICE, reach, impact, confidence, effort. And I think it's simple and fairly robust, which is, I think, generally good qualities of a framework. But one of the traps with RICE that we observed is that the C and E, the confidence and effort, tend to cause you to prematurely deprioritize potentially high-reach, high-impact bets, really innovative things. Uh, and we encountered this on one of our teams early last year where we just RICEd everything, uh, all the ideas, and a lot of the high-reach, high-impact things ended up at the bottom because confidence and effort were just s- so murky for them, as they should be typically for, for high-reach, high-impact ideas. And so one exercise that we push our teams on is just ignore the C and E for a little longer than is comfortable (laughs) and, and just sit with those high-reach, high-impact ideas, with, like, engineers and designers in the room committed to actually trying to solve them. Like give it a, a fair shot. And you'll often find, like, if you spend a week on that set of ideas, you can get pretty far in understanding the confidence and, and the effort. You can probably find a higher confidence, lower effort way to do them. Then add the C and E back in, and then RICE as usual. And the goal is to end up with a, a reasonable mix of innovative bets, incremental bets, and then ones that are, you know, technical debt or, or product debt that you need to address.

    3. LRLenny Rachitsky26:38

       I usually just cut out the C myself. I find that it's not that powerful. Do you do this in a, like, a Google Sheet? Do you use this in Notion? How do you actually recommend teams do this prioritization or just eyeball it?

    4. VIVijay Iyengar26:50

       I'm not super opinionated on, on the exact tools that teams use. I think this is, like, a team local exercise typically. But most teams use Notion and just

11. 26:57 – 30:17

    The problem with estimations, and why Basecamp suggests using a six-week time box

    1. VIVijay Iyengar26:57

       simple tables or, or databases in Notion-

    2. LRLenny Rachitsky26:59

       Sweet.

    3. VIVijay Iyengar26:59

       ... work fine, um, for this. I think the other thing on prioritization that's always tricky is estimation. And, you know, every engineer will tell you estimates are all lies, and if you say it'll take eight weeks, it'll take eight months. But one, I think the core problem with estimation is you're asked to estimate things before you know what the thing is, and, and it's just a, a strange output to be expected to produce. And one approach that I found really interesting is from this book called Shape Up by Basecamp, which is this idea of appetites over estimates, where instead of making the estimate an output of planning, you make the time box or an appetite the input, and you say, like, "We want to solve X problem, and we're willing to invest six weeks solving that problem." Obvious question there is like, you know, how do you pick that time window? It just seems arbitrary. And so the Basecamp people suggest just pick six weeks for everything. And they're really austere about, like, if you can't scope hammer something down to six weeks, you're doing it wrong. Which I think is, uh, it, it can work and has a lot of benefits if you... It, like, creates a rhythm in your company. But one approach I found that works better is pick a reasonable sounding appetite and just explore the two to three options around it. Pick six weeks and then say, "What would we do differently if we only had four weeks or eight weeks?" And you'll kind of naturally find the efficient frontier of, you know, cost and, and, and impact, and then align on that. And the important thing is that you check in after that time period and say, "Is there any new information that suggests we should continue? Did we uncover the biggest risks and are, are we just on the long tail of things?" And, and actually be honest with yourself about, about that. I think that's important regardless of what framework you use.

    4. LRLenny Rachitsky28:29

       I really like that. So does that how you actually operate? You create a time box, so we have four weeks for this project. Whatever we get done, we ship. Whatever we don't, we push out.

    5. VIVijay Iyengar28:37

       We operated that way in engineering, like, particularly on the infrastructure side, 'cause we, we had this series of projects that would just take forever, and the longer it takes, the longer it's going to take. And so we- we've done that exercise quite a bit. I, I'd say more on the, the, more engineering heavy projects than, than others. But we're starting to adopt it more in the product side as well. The main exercise we've taken on the product side is more the consider what would y- what would you do differently with different time boxes a- approach.

    6. LRLenny Rachitsky29:04

       Just a thought exercise.

    7. VIVijay Iyengar29:05

       Yeah, it's a good thought exercise and it just forces everyone to truly score the requirements, right? Like, critical nice to have is nice, but really if, you know, in two weeks you're gonna get pulled off to do something completely different, what would be a complete solution that addresses the core problem? And it forces you to pull, like, the meat of the problem in first as opposed to just doing the things that are surrounding it.

    8. LRLenny Rachitsky29:24

       That's cool. I really like that. I've done that myself. I'm curious if anyone ever does the Shape Up process for real, where it's just like we will ship anything that is ready within six weeks and not actually have, like, specific deadlines or kind of, like, concrete goals of products they need to ship in specific ways.

    9. VIVijay Iyengar29:40

       Yeah, well, I think the Shape Up process, if you run it all the way, they do th- Their idea is that you can actually predict on a six-week time horizon. So you, you can just hammer down scope to something that is complete. Like, it needs to be complete. It can't be milestone one that's like a half-baked thing in six weeks. Which I think that rigor, like the rigor they applied to that across the board, you, you need to do it all the way. You can't ad- adopt the process halfway, I think is the, is the challenge. Otherwise you end up with people shipping, like, milestone one and then moving on, which is not the complete product.

    10. LRLenny Rachitsky30:08

        Makes sense. Uh, a couple more questions around how you build product. You mentioned that you have a unique approach to keeping product teams close to customers.

12. 30:17 – 33:31

    How Mixpanel keeps product teams and engineers connected to customers via Slack

    1. LRLenny Rachitsky30:17

       And I'm curious what you've learned there, what you've found to be helpful in just kind of, yeah, keeping product teams close to your customers.

    2. VIVijay Iyengar30:23

       I think this is one thing that is something we invested in pretty early on at Mixpanel, uh, actually around that time in 2018 when we...... refocus on our core product. One of our sales engineers, Aaron, built this automation where he piped all these customer gaps that we got, that were reported by our customer success and sales teams, piped that into Slack in just a feed. And what this created was this culture where all engineers and designers could consume that raw feed of direct points of customer with no gatekeeper, no process to access it, no pre-aggregation, right? And I think this scales pretty far. Like at, at a product team of our scale and with our reach of customers, we don't get so much feedback that someone couldn't read it in 20 minutes every day. And for, like four or five years in engineering, every day I would read all the gaps that we got and many engineers would do that. And one of the, the rituals that that has enabled is we'll find that engineers will go into that channel and react with a message with an email emoji, which means, "I'm gonna email this customer and find out more," right? And they'll, they'll just email the customer and say, "Hey, I'm the engineer that built this feature. I saw you said this specific thing. Can you tell me more? I'd love to understand." They ask the five whys and then they improve the product on their own. And I think that culture is just so important and it just, it, it just empowers all engineers and designers to think like, think like a PM a little bit, which I think takes a little bit of the load off on the PM to be the gatekeeper of all that information. And then over time we've evolved it quite a bit, uh, as our, our data stack has involved. So we now not just take customer requests, but we take things that are posted on Twitter and, and NPS survey feedback and, you know, win-loss notes from our competitive deals and pipe them both into Slack and into Notion so that we can both get the real-time feed and then we can sort and aggregate and, and tag things accordingly. But the key artifact of this is that it's, it's all open. Uh, there's no gatekeeper behind that process.

    3. LRLenny Rachitsky32:11

       That sounds both amazing and wild. Do you still allow engineers just to email customers and ask them questions about this stuff, or is that harder to do as you've grown?

    4. VIVijay Iyengar32:19

       Oh, no, we still allow that. Yeah. (laughs)

    5. LRLenny Rachitsky32:21

       Wow.

    6. VIVijay Iyengar32:21

       Um-

    7. LRLenny Rachitsky32:21

       That's awesome.

    8. VIVijay Iyengar32:22

       One nice thing about the stack actually, the data stack, is that, uh, so all, basically all these feeds of information land in our data warehouse, which is BigQuery. And, and from there they're pushed out via a reverse ETL tool we use called Census to, to Slack and Notion. So that makes it no code. But one of the benefits of that is that we can enrich all of these feeds with, you know, who's the account, what's their ARR, who's the CSM, and like all that other contact information. So it's usually not like an engineer is blindly reaching out to a cus- customer without letting a CSM know if it's like a million dollar account or something. It's, the idea is just like trust them with that context and they can tag the right people and make the right call from that.

    9. LRLenny Rachitsky32:57

       I'm so curious how that gets prioritized and how PMs are looped into all that, but we don't have to get too deep into that, but that's a, that's a really cool process. I haven't seen that before, where engineers are just emailing customers digging into questions and problems.

    10. VIVijay Iyengar33:09

        The trap, of course, is what you just called out, is like you can't be reacting to everything all the time. And, and certainly if you ship a redesign, right, like the first two weeks of that, there's gonna be a bunch of feedback that's like, "I hate this. Go back." And I think that's sort of an organizational muscle we've had to build to balance the reaction, and that's just a thing we've had to practice doing. But I think the trade-offs are worth it.

    11. LRLenny Rachitsky33:28

        Awesome. One last question along these lines. Can you just talk

13. 33:31 – 35:15

    SaaS tools Mixpanel’s teams use

    1. LRLenny Rachitsky33:31

       about the tools you use, like the SaaS products you use to run your product team, for collaboration, communication, notes, docs? You mentioned Notion as an example.

    2. VIVijay Iyengar33:38

       I think our stack is actually fairly standard, uh, these days. So we have Slack, Zoom for communication, Notion for our docs and any long form writing and as a wiki and database, and Figma and Figjam for, for design and whiteboarding. I think what's actually more interesting is our data stack and the, the productivity we get out of that. Like I briefly touched on this, where basically all of our data gets EPLed out of all the systems we have, lands in BigQuery, gets joined and modeled, and then pushed out via Census to all the other tools in our stack. And I think that's been a huge productivity unlo- unlock, because you can build internal tools with very little code. If you can write SQL, you can build an internal tool basically and that pushes information to the teams that need it. And so that, I think, just has unlocked a lot of these types of things, like automating qualitative signals with no code in a reliable way. And then if someone was like, "Oh, could I get ARR on this?" Yeah, sure. It takes two seconds to, to do that. So I think the, that data stack has been a huge productivity unlock for us.

    3. LRLenny Rachitsky34:37

       Awesome. Have you guys shared that anywhere online, just to show kind of the stack that you guys have built?

    4. VIVijay Iyengar34:41

       We have a couple blog posts that talks about our, our stack for sort of like, we use this both for kind of our PLG infrastructure and like our product-led sales, you know, like defining a PQL and alerting a new user if it's that criteria, but then we also use it for internal tools. Uh, yeah, we have a few blog posts on that topic.

    5. LRLenny Rachitsky34:55

       Sweet. Uh, we'll follow up and include some of that in the show notes.

    6. VIVijay Iyengar34:59

       Yeah, definitely.

    7. LRLenny Rachitsky35:00

       Final line of questioning. You're one of the smartest people in the world on product analytics, heading product for Mixpanel. I'm curious what you think most people get wrong when they're setting up product analytics for their site, their product, their company.

14. 35:15 – 37:43

    The biggest product analytics mistakes

    1. LRLenny Rachitsky35:15

    2. VIVijay Iyengar35:15

       It's maybe a bit of a hot take (laughs) because I think so many people-

    3. LRLenny Rachitsky35:18

       Ooh. There we go.

    4. VIVijay Iyengar35:19

       (laughs) Yeah. So many people still do this, but I think the biggest mistake is, is setting up analytics using client side SDKs, client side tracking. So, um, like web and mobile SDKs, like putting a Mixpanel.track or a Segment.track in your web app or your, your mobile apps. And the reason it's a hot take is that for many people that's, product analytics and SDK tracking are synonymous. They're like, "All right, Mixpanel means SDK. I have to put an SDK in my web and, and mobile app." But that's a mistake because it, we've just seen time and time again it leads to poor data quality and difficulty to maintain that, that, that data. So the problem on web is just due to ad blockers and other unreliable things in, in the JavaScript world, you end up dropping 20 to 30% of your events, and so it just doesn't match your internal databases. And then on mobile, there's two problems. The first is that you have to reinvent tracking for both iOS and Android 'cause it's two different languages and two different platforms, generally speaking. And so you end up with, you know, many duplicate events that semantically mean the same thing but are just different because of the two platforms, and you might have two teams owning that. And the second issue, which is I think even worse, is that you are kind of beholden to clients updating their mobile app to get the latest version that has your latest tracking. So if you wanna add new tracking, it'll only apply to people who have the latest version and beyond. Worse yet, all of your old tracking, whether it's broken or you made a mistake, is still out there in the wild, and so you're constantly getting events that are, that are old and, and broken.And so what we recommend instead, and that we've seen a lot more customers, uh, adopt recently, is just track events from your servers, right, instead of your clients. And that has three benefits. One, it's instantly cross-platform, web and mobile and, and TV and whatever other platform, they're all gonna go through your servers so you instantly get 100% reach. The second is it's in an environment you control. So if you wanna update tracking, you can update it and it updates for 100% of, of your users. And then the third thing is that, and this is I think maybe unintuitive but it's, it's true, is that engineers have, have been tracking events from servers forever. It's called logs, right? And e-events are just logs with a user ID in them, and so they don't need to deal with learning a new SDK and dealing with all that. They just, just have to track logs that have s-some structure and a user ID in them, and they're tracking events. And so i- if it's easier for the developer, it'll, it'll get done in a, in a higher quality way. So I think the really simple advice there is just, just start tracking events from your servers instead of from your clients, and, and if you need to supplement it later on with context that's only on the client, you can add that on later, but server side should be the default.

    5. LRLenny Rachitsky37:41

       Maybe I'll ask question just what's changed

15. 37:43 – 41:09

    The present and future of analytics

    1. LRLenny Rachitsky37:43

       most in how companies work with analytics in the past few years, and then just where do you think things are going in the space of analytics?

    2. VIVijay Iyengar37:51

       Yeah. So I think one huge trend is the rise of the data warehouse. So these are, you know, Snowflake, BigQuery, Redshift, um, and, you know, they're really scalable and they speak the SQL standard, which has led to this explosion of tools that have emerged around them and make it really easy and cheap to load data into the data warehouse and then also easy to push data out of the data warehouse by tools like Do that. And this has a few implications. So the first is that the data warehouse becomes the center of gravity for all data in your company, whether it's product marketing and sales data, um, they all land there. And I think that that's really valuable today in this product-led growth world, and a lot of ink has been spilled about that. But, like, from a data standpoint, that means, you know, all these teams need to be operating off of the same version of the truth, and that version of the truth is sitting in the warehouse and it just needs to be joined correctly. The second thing in terms of where things are going is that events, um, like a, like a time series of users did this action at this time are the universal data model for analytics. And the reason for that is every action, every interaction a customer has, whether it's with your sales team, like a Gong call, or with your marketing team, like they clicked on an ad or viewed a, a marketing article, or with your product, which is more well known, those are all events. They, they, they can all be modeled as events. And it, it's super granular, super intuitive as a way to understand what, what people are doing, and it's really powerful because oftentimes you wanna ask questions about sequences of events, right? Like, which user spent this much time on my pricing page and then looked at three developer docs? Like, that's probably a, a user I wanna reach out to. Or, you know, a- a- a... So many things can be modeled off of that. And so I think data warehouse is becoming the, the loading dock for all of this data, which can be very easily modeled as events, but it's not a very great analytical tool for events because SQL is optimized for rows and tables and joins and not events and sequences of events and segmentations of events. And so one of the things that we're spending a lot of time thinking about is, how do we get that really rich, trusted, comprehensive dataset from the data warehouse into a tool that's optimized from the UI down to the, the data model for events? Because that unlocks, like, really fast, intuitive exploration of data on a dataset that people already have and trust. So that's, I think one of the big trends we're excited about and, and what we see as the future.

    3. LRLenny Rachitsky40:03

       Interesting. And you think Mixpanel is in a good place to help people do that? Is that how you see this? Like, this is something that companies like yours will help people solve, or this is something everyone's gonna have to figure out for themselves, or there's, like, a whole new class of startups launching to help them make a mess out of data warehouses?

    4. VIVijay Iyengar40:18

       No, there's always a new class of startups joining-

    5. LRLenny Rachitsky40:20

       (laughs)

    6. VIVijay Iyengar40:20

       ... in, in the analytics space. It's never, never a dull moment. Yeah, I think this is something that we, we're looking to solve because, I mean, analytics is only as good as the data, and if people are already collecting great data in the warehouse, that means that we integrate with the warehouse really well, then, then we get access to that good data. Increasing what we've been seeing is that companies like in the reverse ETL space, like Census and Hightouch, are effectively reinventing the CDP, reinventing data movement tool like Segment on top of the data warehouse. And so really our strategy there is just tightening our integration with those tools. And we've seen just huge growth in people using their data warehouse as the source for events. Like, not adding SDK tracking anywhere, and just saying, "I already have events sitting here. I trust all of them. They're from all parts of my business. Why can't I do analytics on my support tickets and my Gong calls just as easily as I can do it, um, about my user behavior?" And so I think that's,

16. 41:09 – 41:58

    How adopting a product mindset has helped Vijay grow his career

    1. VIVijay Iyengar41:09

       that's something we're seeing and, and we're investing in.

    2. LRLenny Rachitsky41:11

       Awesome. Anything else you'd like to share before we get to our very exciting lightning round?

    3. VIVijay Iyengar41:16

       Yeah. We opened talking about the, the transition from engineering to product, and I think one of the things that's just been really fruitful in my career, both on the engineering side and, and product side, is just adopting that product mindset and, and getting closer to customers, consuming the raw feed of customer context, taking every opportunity to talk to them. And I, I'm really excited to see, you know, things like this podcast and, and your newsletter and, and other forums for engineers to also develop that product mindset and, and get closer to customers. Because I think long term that just means better products and services at lower and lower prices, which is just innovation, right? So I'm, I'm really excited to see, see more of that in the world.

    4. LRLenny Rachitsky41:53

       Hear, hear. With that, we've reached our very, very exciting lightning round. I've got

17. 41:58 – 46:00

    Lightning round

    1. LRLenny Rachitsky41:58

       six quick questions for you. I'm just gonna go through 'em pretty fast. Whatever comes to mind, just share, and we'll see how it all goes.

    2. VIVijay Iyengar42:03

       Sure.

    3. LRLenny Rachitsky42:03

       Sound good?

    4. VIVijay Iyengar42:04

       Let's do it.

    5. LRLenny Rachitsky42:05

       Okay. Uh, what are a couple books that you recommend most to other people?

    6. VIVijay Iyengar42:09

       On the business book standpoint, uh, there's this book called The Goal by Eliyahu Goldratt. And it's kind of an old book, but I like it because it's sort of written in this, like, fast-paced thriller (laughs) you know, model. It's like m- like a fiction book, but it's about this idea of the theory of constraints, like finding constraints in a system and h- how you can remove them to improve productivity. So I, I found it just, like, a fun read and, and also really insightful. Non-technical books that I, I, I recommend to folks, uh, particularly those that live in SF, which is Cool Gray City of Love by Gary Kamiya, who is a longtime SF resident. And it just goes into, you know, the...... like, the history and the communities and the, even the geography of San Francisco. And I've just discovered so many little pockets in the city from, from reading, uh, reading that book. So it's, um, something I recommend to people who live in San Francisco.

    7. LRLenny Rachitsky42:53

       What's a favorite other podcast that you enjoy, other than this podcast?

    8. VIVijay Iyengar42:57

       I'll do a non-tech one, uh, for this. So I'm, I'm a huge fan of the show The West Wing, and so there's this podcast called The West Wing Weekly that goes into each episode of The West Wing and brings in actors from the show, a-as well as folks from, from government, um, to talk about each episode. And it's just a delight to listen to, if you, if you love The West Wing.

    9. LRLenny Rachitsky43:13

       Wait, so they go back to the old show, The West Wing, and talk about old episodes with politicians?

    10. VIVijay Iyengar43:19

        Yeah. Uh-

    11. LRLenny Rachitsky43:19

        That's cool.

    12. VIVijay Iyengar43:20

        Yeah.

    13. LRLenny Rachitsky43:20

        Wow.

    14. VIVijay Iyengar43:20

        Yeah, exactly. I, uh, so the, the show's over. Uh, the, or the podcast is over, so-

    15. LRLenny Rachitsky43:24

        Oh, okay.

    16. VIVijay Iyengar43:25

        ... uh, you, you have, like, all seven seasons. But yeah, I think it started in 2016 or 2015 or something, so yeah.

    17. LRLenny Rachitsky43:29

        Got it. So cool. Favorite recent movie or TV show that you really enjoyed?

    18. VIVijay Iyengar43:34

        Pretty mainstream TV tastes. Um, so I really enjoyed, um, uh, WeCrashed and, and Severance. That's, uh, two good shows I really enjoyed last year.

    19. LRLenny Rachitsky43:42

        Awesome. Favorite interview question that you like to ask people that you're interviewing?

    20. VIVijay Iyengar43:46

        I'm a big fan of open-ended questions, and so one of the questions I ask in the, in the behavioral interview at the start is, "Walk me through the story of you from college to now, or high school to now," if they're a more junior candidate. And couple interesting things here. Like, interesting to see where people spend most of their time talking and, and where they don't, um, and also, you know, how they describe the other people in, on that journey and, and do they use, like, a standard framework to describe everyone or do they, you know, go into each person uniquely? It's just tons of follow-up questions from, from that.

    21. LRLenny Rachitsky44:16

        Final question is this, who else in the industry do you most respect as a thought leader?

    22. VIVijay Iyengar44:22

        Got a lot of inspiration from, from Gibson Biddle and his, his product strategy, um, Medium thing, and in particular the, there's a piece on, on proxy metrics and, like, the shape of metrics you should use, uh, which I found is like a really, like, the way he frames it is a really elegant way to measure reach and impact at the same time, of your metrics. And then also a big fan of Shishir from Coda, specifically his essays on, on Eigenquestions, framing problems, and, uh, the one on marginal return on contribution is really interesting.

    23. LRLenny Rachitsky44:49

        Amazing. Both, uh, guests of this podcast and people I love.

    24. VIVijay Iyengar44:53

        Yeah.

    25. LRLenny Rachitsky44:53

        Great choices. Vijay, this was awesome. Thank you so much for joining me. Two final questions. Where can folks find you online if they wanna reach out, learn more about what you're up to, and then how can listeners be useful to you?

    26. VIVijay Iyengar45:04

        I'm on, uh, on, on Twitter and, and LinkedIn. I think my, my handles will be in the show notes. Uh, not super active there, but I'll, I definitely check DMs and would love to connect on either of those. And then how can listeners be useful to me? Yeah, I mean, ultimately at Mixpanel we're, we're building a product for product teams. So two things. If you haven't used Mixpanel in the last four years, we've changed a lot as I've described on the pod, um, so, you know, check it out and, uh, happy to take any feedback to help us improve the product.

    27. LRLenny Rachitsky45:31

        Awesome. Vijay, thank you so much.

    28. VIVijay Iyengar45:33

        Thanks, Lenny. It's been great.

    29. NANarrator45:35

        (instrumental music)

    30. LRLenny Rachitsky45:36

        Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

Episode duration: 46:50