The Twenty Minute VCBrian Tolkin, Head of Product @Opendoor: How to Hire the Best Product Teams | E1257
EVERY SPOKEN WORD
90 min read · 17,917 words- 0:00 – 0:44
Intro
- HSHarry Stebbings
What was the single best product decision you made?
- BTBrian Tolkin
Upfront pricing for UberPool. When we first launched, it was variable pricing depending on whether you matched or not. You opened the Uber app, you say, "This is where I'm going," and, and it was just like, "Okay." And then based on minutes and miles, it, the cost would post how to calculate, and so we built upfront pricing where you'd see that beforehand.
- HSHarry Stebbings
Ready to go? (instrumental music plays) Brian, dude, I am so excited for this. I've wanted to make this happen for a while, so thank you so much for joining me today.
- BTBrian Tolkin
Thank you for having me. I'm, I'm super excited as well. This'll be great.
- HSHarry Stebbings
Dude, I spoke to so many people that worked with you at Uber,
- 0:44 – 4:18
Lessons from Launching Uber China Pool
- HSHarry Stebbings
and I heard that you were instrumental in the China Pool Uber launch, which allowed Uber to compete directly with Didi in major cities. This is what everyone told me.
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
What are your biggest lessons from that time and launching China Pool so efficiently?
- BTBrian Tolkin
We were launching in China. At the same time, we were standing up a Chinese, uh, uh, a data center in China, and there was all sorts of technical issues with the day before launch getting everything to work properly. And we were launching for, uh, in Chengdu, um, uh, which is a city in China, um, that... By the way, a city of, like, 20 million people that most people at Uber had never heard of. And we were launching for, for rush hour 'cause the product re- relies heavily on liquidity to, to make efficient matches and, um, all that stuff. And so, uh, it, it wasn't working and it was, you know, 9:00 PM, 10:00 PM, 11:00 PM, uh, midnight, 1:00 AM. I think I s- slept 30 minutes on the floor of the Chengdu Uber office. Launched at, like, 5:30 or maybe 6:00 AM, um, and, uh, knock on wood, we were able to figure it out and, and it, and it sort of worked.
- HSHarry Stebbings
What are your biggest product lessons-
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
... from doing that launch-
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
... and from that time with Uber in China?
- BTBrian Tolkin
My biggest product lessons are (laughs) , uh, one, um, underlying data, like understanding the components that make your work, that make a product work really, really matter. So in our case, um, we are trying to do matches, um, between two riders for, for UberPool. The thing that works, the thing that customers care about is, uh, the quality of the match and the price of the ride. The thing that drivers care about is also somewhat, uh, the quality of the match. Um, and those things depend on having really good mapping and routing data. And the reality is, in China, there's no Google Maps. It's much more difficult to have, have, uh, routing and mapping data. And so we had to work really, really hard, uh, to try and figure out how we can make good matches work, and the reality is when we first launched, there, there weren't that good, that good of matches, and the road infrastructure in, in China is very, very challenging. You have massive highways and overpasses and, like, all this complexity that, that, um, is just a little bit simpler in, in, in places like the US. And I think we underappreciated the complexity of, um, how you would make good matches without awesome underlying road data.
- HSHarry Stebbings
What do you know now that you wish you'd known then when you think back to that time?
- BTBrian Tolkin
We probably could have done a better job at the start, um, acknowledging the, like, cultural differences of building apps in China versus, versus the US, right? And the reality is you go to China and there's, it's, it's different, right? It's not as, like, the sleek, elegant design isn't, um, where design sort of falls into the background, isn't as prominent, and it's a lot more, you know, color and red and big buttons and that type of stuff.
- HSHarry Stebbings
Do you think we're seeing the globalization of product design? When you look at TikTok, when you look at-
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
... Redmode, when you look at the-
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
... intrusion of kind of Asian influence into actually Western product design, are we becoming more like them or are they becoming more like us?
- BTBrian Tolkin
I think there's absolutely a convergence. I think probably the influence is a loo- a little bit more being pulled, uh, towards us, but the reality is, like, apps like, you know, TikTok, there's a lot of things going on, things flying around, scrolling, all that stuff, like, um, as our attention spans shrink. Uh, yeah, I think we're, we're certainly meeting in the middle.
- 4:18 – 5:55
Worst Product Decision at Uber
- BTBrian Tolkin
- HSHarry Stebbings
What was the worst product decision you made at Uber, and how did that impact your mindset going forward?
- BTBrian Tolkin
Uh, back in the early days of UberPool, um, uh, the way you accessed the product was a sort of this, um, subset of UberX. So it wasn't all on the slider as it is today, um, it was sort of you go to UberX and then above it you see this little, um, toggle where you can pick UberPool or you can pick UberX and you, um, uh, see some differences around the price or the time that you would get there. And, um, uh, for a little bit, uh, the default was UberPool, and I think that was, uh, a poor product decision, um, because, uh, uh, even if you had chosen UberX on your last ride, it defaulted back to UberPool, and so the UI wasn't clear enough what was happening, and so people were accidentally choosing UberPool. You know, they thought they were getting an UberX, that's maybe what they're accustomed to or whatever, um, and then someone else would show up in the car (laughs) , um, and they'd be like, "Oh, no."
- HSHarry Stebbings
How did that impact your product decision making?
- BTBrian Tolkin
Um, I think we prioritized the need and the desire to test out the bounds of liquidity, i.e. we prioritized, uh, the needs in, in, in some ways of the, of the business above the needs of the user in that case, or respect for the user's choices and, and wishes, and I think that's a lesson I've, I've deeply internalized. I think, you know, good PMs, you have to do both, right? You have to understand what works for the user and works for the, the, uh, the business. I think in this case, we prioritized, uh, too aggressively to one direction.
- 5:55 – 8:15
How AI is Changing the Role of Product Managers
- BTBrian Tolkin
- HSHarry Stebbings
Oof, that's a tough one. You said there, kind of about good BA- good PMs being able to balance between the kind of needs of the business but also the needs of the user. The role of the PM itself is changing so much, it would seem, especially in a world of AI.
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
Can you talk to me about how the PM role changes in a pre versus a post-AI world most significantly?
- BTBrian Tolkin
The tools and means and mechanisms, I think, will change, right? And so your, your primary tool or artifact of, like, writing a PRD versus building a quick demo may change as AI makes it easier and cheaper and, and, um-... to, to, to build. And I think we will see that. We'll see a collapsing or converging of the end product design triad. I think they will never be the s- the same, but I think everything will be pulled in tighter. So, I think all of that will change. I think what won't change is actually the core of the PM job, which is you gotta go talk to users, you gotta go figure out what people want, you gotta go build it, and you gotta go, uh, build it in a way that makes sense for the business, right? And I think trying to actually figure out the creative part of like, "Okay, I've got c- some customers telling me this, I've got some customers telling me that, I've got my CX team telling me this, I've got my user interviews telling me this, I've got my data telling me that. What do I actually, like, build? What do I do? How do I make a good decision that, that works and how do I do it with velocity? How do I know type one from type two decisions?" Like, those core components I actually don't think change in a pre- or post-AI world. I think what changes is your ability to communicate those ideas, your, your, your ability to do more upfront, your ability to do better user research because you can show prototypes easier. So, I think the tool chest gets bigger, but the core skillset of, like, actually figuring out what people want and does it work for the business, um, is the same.
- HSHarry Stebbings
Does Figma get replaced by Replit in this world? I'm, I'm seeing more and more people jump that.
- BTBrian Tolkin
I, I, I don't know on the company level. I think, uh, maybe, maybe the generalized version of that is do more PMs start in prototyping land than they do in strictly design land? And I think the answer to that is yes. Um, but at the company level, like, you know, I think Figma continues to, uh, build for, uh, designers, PMs, and engineers, and make the product closer and closer to being able to be implemented in production code, so...
- 8:15 – 11:55
How AI is Transforming Product Development
- BTBrian Tolkin
- HSHarry Stebbings
How does the product development process change in a world of AI? I had from many people that worked with you that you're like the master of the actual process of product development. How does that change in a world of AI?
- BTBrian Tolkin
Um, that's super kind of people to say. I d- I don't know if I would consider my- my- myself a master. If you go back 20 years ago, it's like pretty waterfall-y, and I think we've- we've, as a- as an industry moved away from that. But you still have sort of like the PM owns this artifact of the initial PRD or, or, or, or one-pager, and design owns the- the- the- the Figma file or- or the design prototype, and engineering owns the actual implementation and the code. And I think that creates a process in and of itself, right? Where, like, the PM is- is sort of at- at the top of the funnel and developing the idea. Then the designer comes in, you work with the designer, and then, um, and it sort of moves down funnel. And I think AI completely collapses that cycle and accelerates a lot of the upfront, up-funnel work where- where, again, you can just do- you can build a prototype and you can show that to customers, and like the PM and the designer might just work collaboratively to say like, "Let's just build a prototype. Let's skip the documents stage, and let's just, uh, uh, do that instead of sort of, um, maybe talking as a first step and then showing some flat files, and then maybe building one or two prototypes to- to sort of refine your hypotheses." And so I think, um, the development process just accelerates a bunch.
- HSHarry Stebbings
You mentioned the one-pager there.
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
Say we keep it just for now.
- BTBrian Tolkin
Sure.
- HSHarry Stebbings
You've seen many different types of one-pagers. What is a great one-pager, and where do most people go most wrong?
- BTBrian Tolkin
Yeah, so I- I think if you're early in the process, um, which is probably where you're writing a- a one-pager, um, the im- the important thing to get right is the problem definition, the why, (laughs) right? Like, what are- what are we a- what is the core insight that is the reason that we're even talking about this project? Um, and that's usually a user insight or a business insight, um, and/or both. Um, and- and then anybody should be able to pick up that one-pager and say, "Okay, I get where we're- I get why we're doing this, and I get the problem." And it's- it's totally fine/expected that the- the solution is- is probably pretty underexplored in those one-pagers. I think people get wrong where the one-pager is the solution description, not the problem statement.
- HSHarry Stebbings
How do you think about prioritization of problems to go after? There are so many different problems that you could choose. A lot of teams have a lot of different opinions. How do you determine we are gonna commit resources to this and not this?
- BTBrian Tolkin
The standard way would be like impact, confidence, and effort, right? And I think that's a- a reasonable good framework, and there's a reason it's been around forever and i- it's the reason that- that- that people use it. I think the- the part that has to match up is something about time horizons, um, and the priorities of- of the company at- at any given time. And so what I mean by that is like one tension that often arises is, um, how much do we, uh, build new features now versus maybe stabilize the existing features versus pay down, you know, tech debt, right? And I think these can become, you know, spiritual debates or whatever, uh, or religious debates. But-
- HSHarry Stebbings
Does that become much more challenging when you're public?
- BTBrian Tolkin
It d- it is more challenging when you're public, but it doesn't change I think the core principle, which is you still need to decide at- at the highest level what timeframe you are optimizing for. And so I think things like experience debt and- and- and, uh, uh, tech debt, the debt part is actually like a really apt description where you trade paying it down now for paying later, or you trade taking it on now to pay
- 11:55 – 13:23
Balancing Growth vs. Technical Debt at Series A
- BTBrian Tolkin
later.
- HSHarry Stebbings
You're an angel investor in my company, in a hypothetical situation. I've got invest- and I'm Series A style.
- BTBrian Tolkin
Okay.
- HSHarry Stebbings
So, I've got investors, and they want me to go from two million to eight million in a year ARR.
- BTBrian Tolkin
Mm-hmm. Yeah.
- HSHarry Stebbings
But I'm mindful that I've got growing technical debt.
- BTBrian Tolkin
Sure.
- HSHarry Stebbings
Do I focus on new product expansion and just let the technical debt ride it out-
- BTBrian Tolkin
Mm-hmm.
- HSHarry Stebbings
... or do I solve technical debt at the kind of, um, at the cost of maybe lower growth and lower new products?
- BTBrian Tolkin
So, I- I think when you're early-ish stage like that, um, at, you know, seed, Series A, doing two million, um, trying to get to eight million, like...You have to earn the right to exist in the future, and paying down tech debt doesn't pay the bills. And so I, I, I think at that stage, it's probably a little bit early to start paying down technical debt. Now, it, it, it a little bit depends. Are you in, like, a land grab? Uber was in a land grab for many, many years, right? It had to be first. It had to, uh, get riders, it had to get drivers, right? And so there's a very much a land grab competitive dynamic that pushes a, you know, a certain philosophy and velocity. I think most early stage companies tend to be that, where, um, you have to figure out if the thing that you're building and that 2 million ARR you may not know yet actually has value, and people want it, and people will pay for it. And so I think, um, early stage, slowing down to, to, to pay down that debt is oftentimes a, a bit challenging.
- HSHarry Stebbings
Should the CEO always
- 13:23 – 13:51
Should the CEO Also Be the CPO?
- HSHarry Stebbings
be the CPO?
- BTBrian Tolkin
Always, no, but in the early stages, yes. Um, so, you know, there are certain scales of companies where that, you know, may- maybe doesn't make sense anymore, um, but in the, in the early days, yeah, I, I do think they should. Um, I think, uh, outsourcing your... If you think that, um, the most important asset your company has is the product that it's delivering, I th- I think at the early stages, outsourcing that to someone else can be really, really
- 13:51 – 17:16
Lessons in Expanding from Single to Multi-Product
- BTBrian Tolkin
challenging.
- HSHarry Stebbings
We mentioned, like, the focus on new products or expansion of products-
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
... versus, um, technical debt.
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
You have gone from single to multi-product. What's your biggest lessons on how to go from single to multi-product well? It's a challenge so many face.
- BTBrian Tolkin
There's a product challenge and then there's, like, a cultural challenge. Um, the, the, the product challenge is, um, you have this chasm to cross, which is you can't degrade your core product, right? It's a, it's a classic innovator's dilemma. You can't degrade your core product to build a new product, uh, on top of it, um, un- until you know that that new product is working, right? Um, you have a bunch of users who, who care about your core product. And so, you know, I, I think the, the best way that, um, I've seen this approached is by giving new products a little bit of a sandbox, a contained sandbox, that, uh, relaxes a bunch of the constraints of the company, um, but does it in a way that if it's completely fails or whatever, it actually doesn't harm the overall core product experience. So Uber and, and Opendoor have an advantage where you can do this geographically by city, right? And so... Or in, you know, Uber's case, even more extreme is you do what Uber Eats was, which was, like, a totally separate app, a totally separate org, and totally separate, separate everything, um... Go ahead.
- HSHarry Stebbings
It doesn't harm it. Does it have to benefit it? A lot of people say, "If you're gonna do a second product, it has to benefit the first."
- BTBrian Tolkin
I don't think it has to benefit the first, but it has to take advantage of some competitive advantage that your company has, right? And so if you think about sort of a, a, a classic two-by-two matrix, um, where you have, you know, your customer set, and your, uh, uh, competitive advantages, um, as a company, your core product is your existing customers with your existing, uh, product, uh, and competitive advantages. Uh, if you have a new customer set and a new set of competitive advantages or capabilities, that's probably just a new company. And so you're either playing in, do we take our core capabilities and that a- a- att- attach to a new customer set, or do we take a new customer set and attach to our core capabilities? And I think you can play in either one of those, but in, in, in both cases, it, it doesn't have to make the core product better, but it does have to make the core experience for your customers better, and it does have to make the business, obviously, better.
- HSHarry Stebbings
What fuck-ups did you make in going from single to multi-product?
- BTBrian Tolkin
I think the biggest challenge with going from (laughs) from single product to, to multi-product, I mean, I, I actually described, probably the biggest fuck-up was, was probably the one that I described, uh, previously, was, like, th- that was Uber going from UberX to Uber Pool in, in, in one, and that was actively, I think, harming the overall user experience. So that was probably the single biggest.
- HSHarry Stebbings
What about at Opendoor?
- BTBrian Tolkin
Yeah, I, I think at, at, at Opendoor, we had initially started building, um, uh, heavily on, um, uh, the buyer side of the business, and I think, um, we... And we knew we wanted to be multi-product. We had a, a, a bunch of initiatives over the year building a retail buyer business, building a mortgage business, and I think we, we could have been better about, um, focusing on what are, what are our core strengths, and how do we not, um, be in sort of that new customer set, new capability quadrant, but, uh, new capability set, existing customer. And I think that's, um, where the company is heading now, um, with a, a lot of its new products focused on, um, helping sellers and, and, and how sellers can, uh, sell their home in, in a variety of ways.
- 17:16 – 21:35
The Power of Simplification: Finding the Kernel of Truth
- BTBrian Tolkin
- HSHarry Stebbings
You said to me before, the most important product skill set is simplification, (laughs) speaking of, uh, product expansion, uh, is simplification, and finding the kernel of truth, this is a brilliant one, is a sea of cacophony.
- BTBrian Tolkin
Mm-hmm.
- HSHarry Stebbings
I read this and I was like, "My God, it's like Ernest Hemingway's fallen-"
- BTBrian Tolkin
(laughs)
- HSHarry Stebbings
... "into my email." Um, so c- can you explain that to me, of why a kernel of truth is a sea of cacophony?
- BTBrian Tolkin
The product job is challenging, right? You have executive pressure. You have your th- uh, uh, independent thoughts. You have what customers are telling you. You have, uh, uh, maybe scaled customer feedback from your CX team. You have maybe your sales team yelling at you with some other feature requests. It's like kinda, it's the prioritization exercise we talked about. And so you have all of these different pressures, and they often come in, in a variety of different forms, but oftentimes, they come in the form of solutions. "Hey, we need this feature." "Hey, it would be great if the product did this." "Hey, we lost a deal because of that," whatever, and I think the core of that product job is to say, "Okay, there's all of this feedback, all of this noise." Like, what actually matters? What actually matters to, to, to a user? In the, um, uh, Uber example f- f- for, for a second, there's a ton of things that make that product work or in the early days that made that product work. but at the end of the day, is there a car available? Can it get to me quickly, you know, within, within five minutes at a price that's reasonable? And if you can do that, all the other stuff, all, all the other product.... nuances, like, kind of fade away, right? And so that's the simplicity that matters for that product. And so trying to align and say like, "Okay, how do we prioritize everything that moves one of those particular dimensions?" Um, but actually, like, trying to understand that when, you know, maybe someone's saying like, "Hey, cancellations are a big problem, and our CAC is too high," and, like, all these other things that are problems, real problems or potential solutions, and find like, okay, that's what I really need to focus on. Uh, that's what matters.
- HSHarry Stebbings
Who is the one to set that OKR of that is what you need to focus on? Is that the CEO? Is that the CPO? Is that the head of product?
- BTBrian Tolkin
That level, where you're talking about that, needs to come tops down, um, and say this is the, the success metric for the company, and then that cascades to the rest of the, the org. So m- for example, in the Uber case, just to extend, you know, trips may just be the, the metric, right? Like, that's the OKR (laughs) that, that, that, that matters, um, but I'm doing UberPool, right? And so, like, okay, I can see that as a, as a product leader for my area, right, which is not the whole company, obviously, for my area and say, "Okay, how do I ladder to that," right? And so what ladder- what matters to me is, okay, in this case it's easy, UberPool trip count, right? But I can match my OKRs and everyone else can ladder their OKRs to, to the top level one. So OKRs to me are like cascading trees, right? Where w- how- wherever you are in the organization, like, it should be layering up to the one or two above that.
- HSHarry Stebbings
What are the big mistakes people make with OKRs in product, do you think?
- BTBrian Tolkin
I think the biggest one is having too many. Um, I think the b-
- HSHarry Stebbings
How many can you have?
- BTBrian Tolkin
I think teams can generally have a few, like three, (laughs) at any given time that really matter. Maybe three to five, depending on the size of the team, but if you're focused on everything, uh, you're not focused on anything, right? And so I think in the OKR process, it's important to, A, have a manageable number, but then, B, be able to articulate what important things you're not doing, um, and the tough trade-offs that you had to make, 'cause if y- if you're doing everything and you're doing it all and you're doing it all at once, it wasn't necessarily a rigorous prioritization exercise.
- HSHarry Stebbings
Where did you focus at Opendoor where, with the benefit of hindsight, you shouldn't have? And how does that impact your mindset?
- BTBrian Tolkin
Opendoor's had, uh, different variations of, you know, a mortgage product that, that didn't succeed, and I think those were, like, well-articulated, well-reasoned decisions, so I think the outcome isn't necessarily what, you know, the company wanted at the time, um, but I think it's, it's hard to divorce, like, bad decisions from bad outcomes. So the hindsight becomes very easy to say, you know, "We should've focused elsewhere," but I think they were well-reasoned decisions at the time, um, but I think, uh, you know, focusing on, um, sort of the, the, the, the right to win for sellers is, is, is where a lot of the magic is, is, uh, today, and I think that's, that's the right focus area.
- 21:35 – 23:34
Are People Naturally Suited for Certain Company Stages?
- BTBrian Tolkin
- HSHarry Stebbings
Okay. Can I ask you, when you think about... We've spoken about kind of single to multi-product, we've spoken about technical debt, sk- do you think people are destined for certain stages of company development? It's often said. Do you agree with that?
- BTBrian Tolkin
No, because I believe in human agency and the ability for people to adapt their skill set. I think bet- I think certain skill sets are better adapted at certain stages of company, um, and, uh, so at any given time, you're s-
- HSHarry Stebbings
But you agree that people can adapt skill sets?
- BTBrian Tolkin
I do think people can adapt skill sets. I think people can grow with companies. I think people can grow, uh, in their career. I do think people can adapt their skill set. I think people want to have to do that, and, uh, that may be the, the, the hardest challenge because people can be very good at a thing, and it's easy to stay good at that thing and do that thing.
- HSHarry Stebbings
How often should you change OKRs? You, we were talking about having kind of three to five per team, how do we think about how often we should change them, and how, how should they be communicated to the team?
- BTBrian Tolkin
Teams should be developing their own OKRs. They should be defining the work that matters the most to the strategy and the top level OKRs that are outlined. A couple of, uh, philosophies here. One, generally, you know, the quarterly planning process or whatever, I, I, I think that makes sense. I don't think you should be changing your objectives that frequently, right? Otherwise it's kind of an unfocused strategy. It's unlikely you, uh, officially completed your objective in a quarter, right? And, um, maybe e- e- you know, every s- uh, year, um, or six months, but I think, um, the, the more nuanced answer is you earn the right to, to set OKRs on longer time horizons by proving you can execute on shorter time horizons. So, super early stage companies, like, uh, having an annual, year-long OKR process when, like, it's, uh, not clear what you're gonna build in three weeks, that doesn't feel like a super worthwhile exercise, and so I think you earn the right to plan on longer time horizons by proving that you can, uh, set and execute, um, over
- 23:34 – 28:07
Speed vs. Taste: What Matters More in Product Development?
- BTBrian Tolkin
shorter ones.
- HSHarry Stebbings
How do you think about the importance of speed of execution in product versus the importance of taste, of refinement, of beauty, of kind of human preference? How do you think about that balance and what's ultimately more important?
- BTBrian Tolkin
I probably personally fall, uh, a, a little bit more on the velocity side, um, and, uh, I think that's, uh, you know, a- not an extreme, extreme version of that. I think the reality is, as good as your product intuition is, as... The reality is, you throw something out into the world, you ship something, and you figure out if people like it or not, and, uh, the more shots on goal you take with that feedback loop, the more likely you are to succeed. And so I, I don't think you can throw out crap, right? Because the risk is then you have, like, a negative signal where actually your execution was just bad, right? And that's the reason it didn't work. So you have to have a minimum bar, right? And that's what viable and, and minimum viable MVP is, but, like...... um, velocity matters a lot.
- HSHarry Stebbings
Dude, how do you determine whether your new product change, your product update, your redesign is shit versus users are just not used to the transition or the change? I remember the iPhone losing its home button. Me and all of the people around me were like, "This is terrible. Swipe up." Now it's preposterous to think of that. How do you think about whether it's a bad decision or it's just user preferences changing?
- BTBrian Tolkin
The iPhone one is particularly challenging because it's hardware. Um, and so hardware's hard. (laughs) Um, uh, in software though, you know, I think you, you tend to have the ability to give people time. I think what, what you need to do is be a little bit more rigorous in how you think about, um, your metrics and your definition of success, where, you know, the classic, "Okay, we're gonna make a change. We're gonna roll it out until we get stat sig, and then we're gonna pick the winner, and then that'll move forward." Like, that may, uh, lead you in the wrong direction here, right? Because it may be, yeah, you have a novelty effect, and it... and, and the numbers decrease. Or frankly, you have a novelty effect, and the numbers increase, but it's temporary. And so if you think there's the risk of that change being just a- sort of like a, um, a change to, to, to customers' behavior, you may just set up your experiment differently and you say, "Okay. We're gonna look at all the data, but we're not gonna make a decision in the first two weeks." We have to have the, the fortitude, the guts, the, the, the gumption to say, "We might see patterns in the behavior where behavior is slowly changing over time, and so we actually need to evaluate this experiment on month, w- you know, or week five through eight data, not week one through four data."
- HSHarry Stebbings
To what extent are you a gut-driven product leader or a data-driven product leader? If I were to put you in one camp.
- BTBrian Tolkin
So if you were p- to put me on a continuum where zero is 100% gut, zero's gut only, um, uh, 100 is data only, um, you'd probably put me at, like, 65? Um, with one, with one caveat, which is data is not just in this, you know... Opendoor has this lesson for sure. Um, not just, uh, you know, quantitative A/B tests, right? Uh, talking to users is data. Like, that is real. If you talk to 10 customers and they tell you something, um, that is just as much data as "I looked at 150 data points and, you know, here was the insight." So 65, um, is, is the answer because true gut is just like, "I just went with what I thought."
- HSHarry Stebbings
Is simple always better in product?
- BTBrian Tolkin
For a given, uh, necessity of complexity, simple is better. And so what I mean by that, let's take the Uber example. Push button, get ride. That's the original, you know, Uber thesis, right? And it was Uber Black, right? Push button, get awesome ride. Um, that's way simpler than open app, see slider, pick car type, push button, get ride, right? But the reality is, if you want options to serve different customers, to serve different use cases, the slider's a pretty simple UI to do it, where you can articulate everything in, in, in one sort of paradigm. So for a given sort of necessity of complexity or necessity of feature set, yes, simple is always better. But you can be very reductionist and say, "Well, if simple's always better, Uber moving to a second car type in one app was a bad choice," and I don't think
- 28:07 – 32:32
Dictatorial Product Leadership vs. Collaborative Decision-Making
- BTBrian Tolkin
that's true.
- HSHarry Stebbings
One thing I think about often is Gustav at Spotify, who's a dear friend and one of the best product leaders in the world, I think. Um, and he always says that talk is cheap, and so we should do more of it, in terms of the internal discussions being so valuable and actually encouraging much of them. I, on this hand, think he's wrong. (laughs)
- BTBrian Tolkin
(laughs)
- HSHarry Stebbings
Which is incredibly bold and arrogant given he's CEO of like a $100 billion company and I have a podcast. Um-
- BTBrian Tolkin
(laughs) You are, you are underselling yourself.
- HSHarry Stebbings
I, I, I'm precauti... But I think actually dictatorial product leadership is underrated. How do you think about that balance between strong leadership and product direction versus a real emphasis on discussion, debate and internal dialogue?
- BTBrian Tolkin
So I think consensus product decision-making, um, is challenging and, and, and wrong. So you think A, I think B, let's meet in the middle is, is challenging. But I think A, and I'm not going to ask you because I think you think B and I don't wanna hear a s- a difference of opinion is not good. Right? And so I think, um, talk is cheap, therefore we should do more of it. Uh, I would interpret that and, and agree with the interpretation of get all the opinions on the table, make sure to gather everyone's expertise, and then make the best decision given the information. So I think slow decision-making is expensive, is for, for a vast majority of decisions, but so is not listening to a- a- anybody else and gathering opinions. So, uh, you know, I, I think the, the... if, if dictatorial in this de- definition is defined as like, "I have all the right answers and my opinion always wins." I don't think that... I do- I don't agree with that. Um, but, uh, that's different than consensus.
- HSHarry Stebbings
I find very rarely does a slower decision lead to a better outcome.
- BTBrian Tolkin
I agree. I agree with that one.
- HSHarry Stebbings
Yeah?
- BTBrian Tolkin
I agree with that one.
- HSHarry Stebbings
Can I ask you, when it comes to hiring for product teams, I, I do wanna discuss this 'cause you said about hiring for true product teams, and I thought that was just interesting. What do you mean when you say true product teams versus just good PMs?
- BTBrian Tolkin
Earlier in my career, I was like, "Okay, if you, you know, find someone who's smart and, and hardworking and can do that distillation, and there's a good PM, you can give them any problem and they'll figure it out." Right? And I think for some generalist problems and generalist people, that's true, but not all PMs are created equal, right? There are PMs who grew up ... as, as designers and then became a PM, or engineers, or have a technical background, or a data background, or a business background, or an ops background, right? And, like, I think it's a... We can be more nuanced in thinking and saying, "What does this, this team need, and therefore, is this PM the right PM fit for the team?" So it's sort of the same way you have founder-market fit, right? You have, like, PM-team fit.
- HSHarry Stebbings
How do you know the PM type that you need? You look across the team and go, "Oh, we're missing a consult- a former consultant analytical brain who's n- wants to be CEO of the product"?
- BTBrian Tolkin
In a pithy way, someone at, uh, Opendoor, who I work closely with, um, had this phrase, which is, "You hire your strategy," right? And so the person you pick to play that role will dictate how that person defines, you know, the success of, of that product. And so, I think you need to have a perspective on that when you're hiring. So for example, if you look around and, and there are two dimensions. There's what is the, um, sort of team, uh, as in maybe the product, uh, that they're working on, that team need, i.e., "Hey, this is a backend infrastructure thing where the key to success is the algorithm that we put forth, therefore, the PM that we need to have and, and needs to be able to deeply understand whatever the case is. You know, optimization or have a more mathy background." So there's that sort of type of team fit. That, I think, is most important. Then the second is, like, your product team, your functional product team, and you can say, "Okay, our functional product team has, like, a lot of people that fit this type of mold. We need somebody who's going to push us in the direction of being sort of that, like, crazy, out there tinkerer, always trying the new, you know, n- newest tech tool, and they're gonna ingest that energy into our product team because we are also a team." So I think both of those are important.
- 32:32 – 36:54
Lessons from Hiring Mistakes
- BTBrian Tolkin
- HSHarry Stebbings
When you look back on your hiring decisions, when they've gone wrong, what did you not see that you should have seen?
- BTBrian Tolkin
So I believe that, um, poor hiring decisions are, uh, almo- are, are never just the per- part of ne- never just the responsibility of the person being hired, the, the, the new person who it didn't work out with. It's almost always, uh, the company or the hiring person's fault, right? And so I think if... In, in that context, um, what I've seen most effectively is either that person wasn't set up for success, and so they didn't have enough clear direction, um, or definition of success or clear outcomes. They had the wrong skill set to sort of what we were just talking about. "Hey, their interest is on the design side, but what the team really needed was, like, a more engineering technical leader 'cause that's, that's some of the challenges that the team was facing." Um, or, uh, and this is sort of a corollary to the first one, the ambiguity of the role and the ambiguity of the challenge was above that person's ability to distill the am- that ambiguity and find what really matters.
- HSHarry Stebbings
Do you give them take-home assignments in the interview process?
- BTBrian Tolkin
Yes, most of the time, uh, I do believe some type of work product is very helpful.
- HSHarry Stebbings
What is the best work product? Again, I'm a founder. You're helping me. How do I test them?
- BTBrian Tolkin
(laughs) The best best is people you've worked with before, because that's a heck of a lot better than a one-hour interview. Um, but I think, you know, I, I, I think it can either be, "Show me a, a, a previous work product that you've worked on," or some type of consistent case study of like, "Hey, here's a somewhat..." Depending on the, the, the, the role, uh, the seniority of the role. "Here's a somewhat ambiguous problem. Let's talk about how you'd handle it or put together a, a, a doc, or a deck, or whatever on how you would handle it." And I think the, the, um, important part here is it can't be so narrowly scoped, because the, the tactics of the job you can learn, how to run a sprint, how to do prioritization. Like, that, that stuff you can learn. That's not what the test is. The test is the clarity of thought and, um, uh, getting to that outcome.
- HSHarry Stebbings
Totally get you and agree. You said there about how to run a sprint. Sprints are incredibly common across companies. What are the biggest tips on how to run the most effective sprints?
- BTBrian Tolkin
Just be clear. Uh, the, the, and the, the biggest tip I have is, like, be clear on what you're trying to accomplish in that sprint, and make sure that everybody is aligned on it. I'm a big-
- HSHarry Stebbings
Ideal-
- BTBrian Tolkin
... I'm a big fan of alignment.
- HSHarry Stebbings
Ideal timeline for a sprint?
- BTBrian Tolkin
Probably two weeks. For most teams, two weeks. Growth teams often can run it one week, and that's okay.
- HSHarry Stebbings
Bryan, I'm enjoying this a lot. I find alignment one of those BS management terms-
- BTBrian Tolkin
(laughs) Yeah.
- HSHarry Stebbings
... which is kind of overused and kind of put in the same kind of culture bucket, which means nothing-
- BTBrian Tolkin
Mm-hmm.
- HSHarry Stebbings
... but meant a lot in kind of the last five years. Am I unfair?
- BTBrian Tolkin
Let me give you what I mean when I said it. Everybody, at any given time, should ideally understand the most important thing that they can be working on, why that matters, and how that ladders up to customer or company goals, and should be able to, uh, fluently discuss those things. If they can do that, and it aligns to the right company goals that have been set, um, then you're aligned. I'm not talking about necessarily agreement, but I can d- I can, I can understand why I'm doing it.
- HSHarry Stebbings
Do you believe in disagree and commit?
- BTBrian Tolkin
Yes, I do.
- HSHarry Stebbings
You don't think that... I, I don't understand how people will work weekends, stay up late at night, give everything when they fundamentally disagree. You'll get the mediocre. You'll get a- you'll get participation and commit.
- BTBrian Tolkin
(laughs) Uh, you... It's a good, it's a good push. It's a good question. I think this, like, goes back to sort of the, the Steve Jobs, uh, Stanford commencement address, which is, "If you look in the mirror too many days in, in a row and you realize that you don't like what you're doing, you should probably do something else." I think that applies to disagree and commit, which is for a sprint, for a month, whatever, can I disagree and commit? For sure, right? Because that, that is, like, how we make velocity-based decisions. If I'm disagreeing and committing too much-... then yeah, I agree with you. Then like I'm in the wrong place because I'm not aligned to where we're going fundamentally, and it's probably time to start thinking about something else.
- 36:54 – 40:24
The Best Backgrounds for Hiring Great PMs
- BTBrian Tolkin
- HSHarry Stebbings
If you could choose the ideal background for the one that's worked best for you personally when hiring PMs, what background type has been most effective coming into a PM role?
- BTBrian Tolkin
I think there is a tremendous amount of undervalued appreciation for internal transfers from other functions into product at the same company.
- HSHarry Stebbings
Engineering, product, design, which becomes less important and which becomes more important in a world of AI?
- BTBrian Tolkin
Engineering becomes more important.
- HSHarry Stebbings
I would say less. That would be my-
- BTBrian Tolkin
Okay. Why would you say less?
- HSHarry Stebbings
Because I think unless you're talking about top 1% performance-based-
- BTBrian Tolkin
Mm-hmm.
- HSHarry Stebbings
... model optimization, like the truly difficult technical challenges-
- BTBrian Tolkin
Mm-hmm.
- HSHarry Stebbings
... you're gonna see a kind of commoditization of engineering challenges because you're gonna have so many tools that help good engineers be in the same ballpark.
- BTBrian Tolkin
Mm-hmm.
- HSHarry Stebbings
I think strategic product mastery will still have a place, and I think actually design will have an even bigger place because I think you'll see the de- the sh- actually design become way shitter with AI doing good enough-
- BTBrian Tolkin
(laughs) Mm-hmm.
- HSHarry Stebbings
... and a lot of people being like, "Oh, happy with good enough," but then the design pros will be up here.
- BTBrian Tolkin
Fair. I think though what you just articulated though is the top 1% of designers, the top 1% of, of PMs, and compare that to the average engineer, the ability to still architect a system that thinks ahead, thinks around corners, I think that will still matter. But I think to, to answer the question maybe how, how you were particularly answering it, the top 1%, the top 5% of all of the skill sets get a lot more valuable and the median gets a lot less valuable.
- HSHarry Stebbings
At Opendoor, what is the best team, product and or design?
- BTBrian Tolkin
At Opendoor, and I think this is true at a lot of, um, operationally intense companies, um, we actually consider sort of the, the classic triumvirate of eng product and design, eng product ops and design because it's so difficult to, uh, divorce your digital product from, you know, the physical embodiment of, of whatever you're doing. And so... Oh, by the way, this was also true in a slightly different format at Uber.
- HSHarry Stebbings
What I, what I find really challenging with you as a product leader loving hard physical product businesses is you could have the most beautiful consumer software experience, say in Uber or in Opendoor, but something completely out of your control, a third-party externality, could destroy the customer experience, making them think you're shit. When you've done everything you can-
- BTBrian Tolkin
Mm-hmm.
- HSHarry Stebbings
... that's very different to another business which is a pure play software business.
- BTBrian Tolkin
Mm-hmm. Yeah, I mean, the, the, the wa- (laughs) The reason potentially I enjoy thinking about those problems and the way I've articulated this is computers are deterministic. Humans in the real world are not, right? And so you have real world en- entropy and you have to think about how your product adapts to that wor- real world entropy. And yeah, that, that's hard for sure. I don't wanna pretend that that's easy, right? Like it, it, it totally is challenging. But that's kind of why.
- HSHarry Stebbings
But you're comfortable with the customer experience being out of your hands?
- BTBrian Tolkin
Am I comfortable with it? I don't know about that, but like I'm, I acknowledge some of the most important impactful products exist in the real world, not just the digital realm. And therefore, you have to grapple with the realities of the real world.
- 40:24 – 41:39
Managing Momentum as a Product Leader
- BTBrian Tolkin
- HSHarry Stebbings
You said about being biased there because I said ne- "Which is your favorite?" How do you think about managing momentum as a product leader? It's very difficult to do well.
- BTBrian Tolkin
One of the things that I try to think about is you might be a natural if you feel like the team needs a momentum boost. That might be the team is newly formed. That might be you're coming off an extended break. Obviously some, some people, uh, take, take a break around the holidays. That could be you just had a difficult business outcome. Um, that could be a, a, a reduction in force or s- something like that. And so you're like, okay, we need to inject and infuse some positive energy and momentum here. Let's maybe unnaturally ship something that, yeah, maybe is a little bit lower impact but is higher confidence, lower effort, so we can prioritize speed and confidence of impact so the team gets a little bit of, uh, um, excitement moving forward and that compounds. And so you may have a slight unnatural prioritization that says, "Okay, we just came back from the holidays, everyone's pumped about their plans. Yes, we just thought about the next year, but like let's ship something that matters in the next week."
- HSHarry Stebbings
I agree. I think it applies actually just to broader teams. I always say you have to manufacture hype and management and goodwill and good things.
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
Like when someone does something that's like normally eh, like make it a much bigger thing because everyone wants-
- BTBrian Tolkin
Right.
- HSHarry Stebbings
... to feel, especially in harder times, "Yeah, we
- 41:39 – 43:13
The Case for Staying Longer at One Company
- HSHarry Stebbings
did something great." Final one before we do a quick fire. You said before about the benefits of staying for longer periods of time at one company for individuals. That is different to how most people operate in the Valley. You are incredibly promiscuous as a group, uh, and jump from one AI company to another right now it would seem. Um, why do you believe in the benefits of staying at one?
- BTBrian Tolkin
So as someone who's biased because I've spent, you know, two, two periods of longer stretches, uh, at, at, at companies, I think the reality is, uh, you just become more effective and therefore you can do more, um, more eff- more, more quickly. You have deeper relationships, you have deeper context on the company, you have deeper context on the customer base, and therefore you can be more effective. So if you're, if you're hopping every 18 months or two years and you assume, you know, it takes six months to ramp, uh, you know, obviously that's very long to, to ramp to be any effective, but like to, to really deeply understand and gain context like, and then at some point you're interviewing and thinking about your next thing, like you just don't have that much time to be effective. Obviously if it's the wrong fit-You should, you should move on, but I don't think a goal should be to move on every two years as, you know, you can canonically read about, 'cause I think you will just get more effective with time.
- HSHarry Stebbings
You're advising a student coming out of university today. Do you start a company, do you join a startup, or do you join a big company?
- BTBrian Tolkin
Uh, join a relatively early-stage company.
- HSHarry Stebbings
Why?
- BTBrian Tolkin
The right balance between seeing some of what works, but allowing you to make your own mistakes.
- 43:13 – 46:42
Quick-Fire Round
- BTBrian Tolkin
- HSHarry Stebbings
Okay, listen, I wanna do a quick-fire. So, I say a short statement, you give me your immediate thoughts.
- BTBrian Tolkin
Okay.
- HSHarry Stebbings
You mentioned before-
- BTBrian Tolkin
No nuance, I got it. (laughs)
- HSHarry Stebbings
No, no nuance, uh, a la poubelle. Uh, you mentioned before you have a six-month-old. My brother's having a baby literally now.
- BTBrian Tolkin
Oh, m- congratulations. Mazel tov.
- HSHarry Stebbings
Like, like, uh, yeah, she's in hospital now.
- BTBrian Tolkin
Amazing.
- HSHarry Stebbings
What would you advise him, it's his first child, on parenting?
- BTBrian Tolkin
Uh, say yes to all of the help that anyone offers or provides, uh, and actually living close to, um, parents, friends, family who can help is a superpower and a cheat code.
- HSHarry Stebbings
Yeah. Don't worry, Mum's there. (laughs)
- BTBrian Tolkin
Yeah. Exactly.
- HSHarry Stebbings
Um, what's, what's the most common reason founders don't get product market fit?
- BTBrian Tolkin
Not deeply understanding the problem and falling in love with the solution of the problem.
- HSHarry Stebbings
What do you know now that you wish you'd known when you entered product?
- BTBrian Tolkin
The hiring lesson, um, that, that I gave earlier of making sure to really understand what the problem needs, and then finding the person who's really good at that type of problem.
- HSHarry Stebbings
What was the single best product decision you made?
- BTBrian Tolkin
Upfront pricing for UberPool.
- HSHarry Stebbings
Meaning, like, this is the price and they know it definitively?
- BTBrian Tolkin
Yeah. When we first launched, uh, it was, um, variable pricing depending on whether you matched or not, and the quality of that match.
- HSHarry Stebbings
What? Oh, okay.
- BTBrian Tolkin
Back, back, back at the time, I don't know if you remember this, like, you opened the Uber app, you say, "This is where I'm going," and, and it was just like, "Okay." And then based on minutes and miles, it, the cost would post-ho- post-hoc calculate. And so we built upfront pricing where you'd see that, uh, uh, beforehand.
- HSHarry Stebbings
What other function has the most tension with product?
- BTBrian Tolkin
Operations. The speed and cadence and types of problem-solving is, is, is very, very different. And so, uh, it can be very challenging, uh, to all speak the same language, um, on things that have a product or technical solution versus an ops-based solution.
- HSHarry Stebbings
What was the most recent consumer wow moment you had with a product?
- BTBrian Tolkin
It's cheating, but the honest answer is ChatGPT.
- HSHarry Stebbings
Really?
- BTBrian Tolkin
Yeah.
- HSHarry Stebbings
Huh. Okay.
Episode duration: 46:52
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode XKVXrQchWIE
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome