Skip to content

By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

a16za16z

The $700 Billion AI Productivity Problem No One's Talking About

Russ Fradin sold his first company for $300M. He’s back in the arena with Larridin, helping companies measure just how successful their AI actually is.

Russ FradinguestAlex Rampellhost
Dec 1, 202557mWatch on YouTube

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

Transcript

0:00 – 1:07

Introduction

Russ Fradin 0:00 85% of the companies we talked to said they really believe they only have the next 18 months to either become a leader or fall behind.

Alex Rampell 0:07 You know, we have our little group chat where we have another friend who's like, "Oh, all this stuff is overhyped and it's going to zero."

Russ Fradin 0:11 Totally wrong.

Alex Rampell 0:11 But you'll never know.

Russ Fradin 0:12 Every time I use AI, it's amazing.

Alex Rampell 0:14 There's somebody at every big company who has figured out, "I could do something in one minute that used to take eight hours."

Russ Fradin 0:20 28-year-old guy who was using ChatGPT really, really well, and they had him create a 30-slide deck, and they did a global call for everyone in the investment bank for this guy to spend an hour walking people through how to use ChatGPT. But that's absurd. That's an absurd way to hope people adopt world-changing technology. Cursor has taken mediocre engineers and made them good, but it's taking amazing engineers and made them gods. Every board meeting I go in for my other four metrics, I have some report of how are we doing against those reports, and on AI, all I have is the amount of stuff we bought.

Alex Rampell 0:49 When a measure becomes a target, it is no longer accurate as a measure.

Russ Fradin 0:54 Even though we thought we had our quota set and we thought everyone was productive, it turned out we thought we were productive, and actually it turned out we could be much more productive.

Alex Rampell 1:00 But compared to what? [upbeat music] I'm excited to be here with my friend Russ

1:07 – 2:09

Early Career, Ad Tech, and Web 1.0

Alex Rampell 1:07 Fradin, um.

Russ Fradin 1:08 Yeah, good to see you.

Alex Rampell 1:09 Uh, I've known you for a long time.

Russ Fradin 1:11 That's right.

Alex Rampell 1:11 And I think when I first met you, I still actually remember meeting you the first time. It was from, uh, I think Josh McFarland. Um, and Josh was at Google, and he was like, "Yeah, there's this guy, Russ Fradin, and he started this company, Adify, and he sold it to Cox for all this money."

Russ Fradin 1:24 [laughs]

Alex Rampell 1:24 You know, back then, $300 million was a lot of money.

Russ Fradin 1:26 I know. It's- it is amazing. I, I, I-

Alex Rampell 1:28 Now it's like a B round. But, like, back then, like, that was a huge acquisition. And, uh, the- there, there was like, "Oh, like, Russ, like, I ha- like, amazing person that pulled this off." Um, and, uh, I think we met in Florida-

Russ Fradin 1:41 That's right

Alex Rampell 1:42 ... uh, on a Silicon Valley Bank trip, [mouth clicks] and kind of all things, uh, e- everything comes full circle in the end. But, um, you know, AI is probably the hottest thing in the history of the world. Um, but you also worked in what was the hottest thing in the history of the world in Web 1.0.

Russ Fradin 1:56 Yeah.

Alex Rampell 1:57 But now there's this big question. Actually, it reminds me of Ad Tech. I think it's kind of a, a nice little segue because, like, Ad Tech, you're trying to figure out, does the advertising work?

Russ Fradin 2:04 Right.

Alex Rampell 2:05 Like, a lot of Ad Tech is, here's an advertisement, and there's this attribution

2:09 – 3:30

Attribution Problems in Ad Tech & AI

Alex Rampell 2:09 problem-

Russ Fradin 2:09 Yep

Alex Rampell 2:10 ... of the sale happened. Who is responsible for that sale? Was it the banner ad on Yahoo? Was it the last click that happened on Google? Was it the coupon site that, you know, stuffed a cookie on your machine? So part of Ad Tech is just how do I fi... Like, I'm buying ads, that's part of it, but part of it is also did it work? And AI, there's all sorts of stuff around making AI work, which is, like, technically very, very challenging. But then there's the question of did it actually yield a benefit?

Russ Fradin 2:38 Yep.

Alex Rampell 2:39 Which is probably the biggest question for... I mean, there, there's, there's a lot of, like, myths on this on both sides, but would love to kinda hear about the origins of Larridin and how you think about even some similarities between the two.

Russ Fradin 2:49 Sure. Yeah, there, there's a lot of parallels really to what happened in the '90s with advertising and the growth of the internet and what we're seeing with AI. I mean, forget the capital markets perspective. It is, it is funny to think about, uh, what, what is defined as big from an exit these days versus five years ago, 10 years ago, 20 years ago, w- and that's, that's kind of its own topic. But just when I moved out here, I moved out to Silicon Valley in 1996, and I was the first guy at the first online ad network. And in the early days, it was just there are websites, we should put ads on them. Great, how do we do that at scale? Great, what are the metrics we should capture? Then you saw the growth of things like comScore and Nielsen as they moved into television

3:30 – 5:49

Building Measurement Infrastructure

Russ Fradin 3:30 to figure out, like, how do I actually plan this? How do I spend this? How do I give tools? Right. All of the money lived in TV or in radio, and there were these tools like Nielsen, Arbitron, IMS Health on the pharmaceutical side. There were all these tools to help people understand what they were getting when they advertised on television. You had to build that entire stack for the internet. You had companies like DoubleClick or Flycast where I was, or, uh, companies like Omniture building a different part of the stack, companies like comScore building a different part of the stack. And those companies, obviously Google and Facebook are two of the most amazing companies ever built, but if it wasn't for all of that infrastructure, their revenue just wouldn't have grown as quickly. And I, I really do think we'll see the same thing in AI now, the technology is unbelievable. And, you know, my core thesis when I was thinking about starting Larridin after having been, you know, the first guy at the first online ad network and having been maybe the first, one of the first two executives at comScore way, way 25 years ago back in the day, um, my, my partner Jim and I sat down and we said, "Look, you know, every time there's a tremendous shift in budget, and especially when it happens at a great pace, like what happened from TV to digital advertising, which happened in a lot of categories from client server to cloud, anytime that happens, people need to rebuild all of the infrastructure. There's a great opportunity to build all of these tools around measurement, around governance, not with the goal of stopping anything, frankly with the goal of accelerating it. Because if I am a large company, yes, I'm going to experiment a ton with AI today. It's the most exciting thing that's happened in the last 20 years from the technology standpoint. It's amazing. It's wonderful. But also there are very boring but important questions. I have 35,000 people in my workforce. They can't all get retrained all at once with perfect knowledge and perfect security. How does it affect my D&O insurance? Was the project ultimately valuable? And so we really wanted to start a company about how would you build kind of the measurement and governance set of tools, not to be a gatekeeper, but to empower more of this spending. I, I think as we grow, we will be the best friend to all of the AI companies.

Alex Rampell 5:34 Yeah, and, and, um, I mean, maybe we can get into how you're doing this.

Russ Fradin 5:38 Mm-hmm.

Alex Rampell 5:38 But, um, just to kind of level set a little bit, and I love this framing that you gave me. I've, I've stolen it. Um, wh- when I steal a phrase, it's the most sincere form of flattery-

Russ Fradin 5:47 Of course

Alex Rampell 5:47 ... of course. But, um, I, I

5:49 – 7:51

Software Eating Labor: Productivity Shifts

Alex Rampell 5:49 just released a, a little video about kind of, uh, how software is eating labor.

Russ Fradin 5:52 Mm-hmm.

Alex Rampell 5:52 So, you know, software eats the world. This was a thesis that our firm is founded on, but it's eating labor, but it doesn't actually mean that, like, jobs are gonna go away.

Russ Fradin 5:59 For sure.

Alex Rampell 5:59 Largely, what it means is that people are going to be, like, 10 times more productive, where I can't hire anybody to do this job, but I can hire AI to do it.

Russ Fradin 6:07 Mm-hmm.

Alex Rampell 6:07 So you have companies where, uh, their software budget is very, very smallBut their labor budget is enormous. And step one of, like, the mega opportunity that excites us as a firm is that people say, "Oh, I'm gonna start hiring software." But now that means that your software budget is enormous.

Russ Fradin 6:25 Yes.

Alex Rampell 6:25 Because right now, if you have, like, a $10 billion labor market-- labor budget and, like, a $1 software budget, you're not gonna try to cut, you know, optimize your $1 software budget, but you're really gonna say, "Okay, do I need to hire more people? Can I make people more productive?" You know, all these things that are going through people's minds right now, and this is yielding a lot of the mega growth curves of the AI software companies. But now this chart is gonna be a little bit more balanced-

Russ Fradin 6:48 Sure

Alex Rampell 6:48 ... of, like, the $10 billion of labor, you know, maybe that goes to eight, and now you spend two bill- or $1 billion on software. So, like, the, the net spending for the company is actually lower. The company is more profitable, productivity gains galore. But then is this productive?

Russ Fradin 7:03 Right.

Alex Rampell 7:03 Like, I, I always wanna know if the humans are productive, but then, like, is the software yielding more-- me more productivity, and how do I measure that? So everybody's excited about this gold rush. I'm gonna use these t- these tools, but, like, do they work, and how well do they work, and what's the baseline?

Russ Fradin 7:18 Yes.

Alex Rampell 7:18 So, so I, I've stolen your framing of it's like, you know, if Chase spends $18 billion on software or what- whatever-

Russ Fradin 7:24 Right

Alex Rampell 7:24 ... and now they can double that-

Russ Fradin 7:25 Two hundred billion. Right

Alex Rampell 7:26 ... but, like, they need to know if they're getting their money's worth. They need to figure out if this is actually efficient spend.

Russ Fradin 7:32 Yes. Look, a funny thing, uh, uh, not funny, a thing you will hear said frequently by the people running the largest AI companies in the world, the people running the largest firms investing in AI in the world is they'll say something along the lines of, you know, today global IT spend is $1 trillion, and we think because of AI and agents, that

7:51 – 13:54

The Challenge of Measuring AI ROI

Russ Fradin 7:51 could go to $10 trillion. And, you know, let's ignore whether that's true or false. It's certainly the bull case for NVIDIA, for OpenAI, and for all of the other things that we spend all of our time doing. And so when you think about it, I think we said, um, I think if I remember correctly, JPMorgan Chase's global IT spend is on the order of $18 or $19 billion, and they spend a couple hundred billion dollars a year on people. So if you really think about that, well, is their IT spend gonna go from $18 billion to $180 billion? Uh, seems unlikely in the next couple months, but it's certainly gonna go up. And if it's gonna go up, what does the CFO need to understand? And at the same time, because of the pace, you know, a, a way I like to frame this that I think everyone knows but I think it's important to say out loud is yes, there have been tons of shifts, right? We've shifted society a million times. We've shifted from farms to cities, right? We, we all know all of these examples. But we ca- we've never had a time where we've expected the entire global workforce of knowledge workers to be retrained immediately on a new set of tools that didn't exist six months ago, right? And so there is an element where everyone needs to figure this out as we go along. So what did we start with as a company, right? Our first set of tools is just what do you have in your company, and are people flat out using it? You've spent all of this money. Are people using it? And what you find is, you know, 80-something percent of our customers find far more tools being used by their employees than they know about and they've licensed. That doesn't mean it was bad, by the way. Some of those tools are dangerous, and they should worry about that. Some of those tools are-- might be very popular, and they need to bring them into the fold and understand what's happening. But from an IT standpoint, you normally don't allow software to just be used across your organization with access to your organization's data and have no idea what's happening. We're letting that happen in AI all the time, and I don't really say that to our customers as a fear sell. It's to be expected. Things are moving quickly. You have to know what's going on. So we start with the baseline of just flat out what's happening. The second set of things we try and solve is how do we get people using this stuff more in a productive way on the AI side, on the agent side? How do we get people using this in their workflow? I'm a marketer working at General Mills or something like that, right? I'm a marketer working at General Mills. How is General Mills going to help me use these tools? And what I've generally found with employees, if you really wanna drive employee usage of tools, you have to make them feel safe so they won't look dumb, and you have to make them understand that they can use this safely without getting fired. Because again, it's one thing if you're 22 years old and you've been using these tools effectively your entire life since, you know, high school. But if you're a 42-year-old person who's been, you know, had a 20-something-year career and you're working in your job every day, and by the way, you also have things you do at home, and you have business travel. You have all of these things you have to do. Also, you have to become an AI expert. You really would like to not look dumb, and you'd like to accidentally not upload the wrong data and get yourself fired. This is actually a bigger issue in some countries where there's a bunch of EU regulations around AI that do matter, and if I'm an employee at a company, I don't want to look dumb. So if I'm a CFO, we bought all these tools. What did we actually buy, number one?

Alex Rampell 11:02 Right.

Russ Fradin 11:03 Number two, how do we get people actually using these tools? Because the usage on these tools in the enterprise is less than people would think today-

Alex Rampell 11:11 Mm-hmm

Russ Fradin 11:11 ... which makes sense, by the way. I'm gonna get to the productivity thing in a second. But, you know, anyone listening to this, if you've ever been a part of any software rollout at any enterprise ever, a very boring but very important question is how do we drive actual usage? And sure, everybody uses email. People use Workday, 'cause if you don't use Workday, you're gonna get fired, right? You're not gonna get your paycheck. But most enterprise software, your intranet software from SharePoint, things like that, are used by a relatively small set of the population that you wish were using it. And so if the goal is to get people more productive using AI tools, you want to drive actual employee engagement.

Alex Rampell 11:46 Right.

Russ Fradin 11:46 So we built a suite of tools around that. And then you have to get into productivity, which is did this get people actually more productive? Is my organization actually more productive? So today I, I know where I wanna go with Larridin. What I like to think about today is what we're doing today on the productivity side is not as far as I'd like it to go, but it's certainly better than anything that exists in the market. And so what we're doing today is we're marrying the behavioral data that no one else has, which is, is Alex a heavy user of ChatGPT or not? Just flat out. We're, we're not doing it at the individual level, but we'll use that example for the podcast 'cause we have to worry about the employee privacy concerns that companies have for their own employees. But at the end of the day, I wanna understandDid my users in the legal department that were using this expensive legal tool I bought-

Alex Rampell 12:33 Right

Russ Fradin 12:33 ... are they more productive than my users in the legal department that are not? Because what I've definitely done is I've driven up my OpEx.

Alex Rampell 12:39 Right.

Russ Fradin 12:39 I've bought this software, I've driven up my OpEx, but are they more productive? Are my marketers that are using Claude or ChatGPT actually more productive?

Alex Rampell 12:46 And how do you measure that?

Russ Fradin 12:47 So today we do it the only way productivity research has ever existed so far, which is we take the normal productivity survey market research that people have done for 50 years, not ideal, but it is the gold standard. It's McKinsey, it's Towers Watson, it's Accenture, and we lay on top of it proprietary data that other folks don't have, which is actual usage. So the way I think of it is, the worst way to measure productivity is I'm gonna send a survey to my employees and say, "Do you feel more productive today from using ChatGPT?" First of all, there's a definition issue. Second of all, people are gonna answer the way you hope they'll answer. But third, you have no idea if they're actually using the tools.

Alex Rampell 13:21 Right.

Russ Fradin 13:22 So a better way to do that, I learned this years ago at comScore. I-- one of the things, one of the many things I did at comScore is I ran our survey market research group, and one of the reasons the comScore surveys were great is we had the behavioral data married with the actual survey responses. We're doing the same thing here. Where I ultimately would like to get to is full passive measurement on productivity. The truth with that is for enterprises, that's gonna require a level of additional data sharing that we're not getting yet from customers. We, we will eventually get there.

Alex Rampell 13:49 But to kind of, like, put a finer point on this, um, so I'm a lawyer.

Russ Fradin 13:53 Mm-hmm.

13:54 – 17:46

The Productivity Baseline Problem

Alex Rampell 13:54 I work at, you know, at, at some big company. Um, you know, productivity to a certain extent, like, if I only have to work four hours a day versus eight hours a day, like, that's great for me.

Russ Fradin 14:05 Mm-hmm.

Alex Rampell 14:05 Like, I kinda feel like it's a win because I, I often think about, like, the principal-agent problem, right?

Russ Fradin 14:09 Sure.

Alex Rampell 14:09 So, uh, everybody is an agent, and then there's the ethereal being of the corporation, which is the principal, and it's like, you know, yeah, I guess if I own stock in my corporation, I want it to be more profitable.

Russ Fradin 14:22 Sure.

Alex Rampell 14:22 But really I wanna work as little as possible and get paid as much as possible. Like, that's kind of every individual agent's job.

Russ Fradin 14:28 Right.

Alex Rampell 14:29 And then you have these tools, so, like, theoretically it's like everybody's gonna adopt these things if they get to be lazier.

Russ Fradin 14:35 Yep, yep.

Alex Rampell 14:35 Like, everybody wants to be lazier.

Russ Fradin 14:37 Sure.

Alex Rampell 14:37 Right? They wanna be lazier and richer.

Russ Fradin 14:39 Yes.

Alex Rampell 14:39 I feel like these are, like, the universal, like, kind of human conditions. Like-

Russ Fradin 14:41 There's some small set that want promotions, but I agree for 90%

Alex Rampell 14:44 But it's richer, right?

Russ Fradin 14:45 Yeah. That's true. That's true.

Alex Rampell 14:46 So, like, if you can get the promotion by doing less work-

Russ Fradin 14:48 Sure

Alex Rampell 14:48 ... I'm sure people would opt for that. But I guess, like, how... Like, think about everybody will use these tools or, like, I think I was telling you this sad story because our kids go to the same school.

Russ Fradin 14:57 Yeah.

Alex Rampell 14:57 It's like younger kid gets busted cheating-

Russ Fradin 15:00 [chuckles]

Alex Rampell 15:00 ... right, with ChatGPT. Like, clearly productivity gain for him.

Russ Fradin 15:04 Right.

Alex Rampell 15:04 Right? Because it allowed him to be lazier and, you know, r-richer with his video game time-

Russ Fradin 15:08 Sure

Alex Rampell 15:08 ... until we confiscated his phone. Um, but-

Russ Fradin 15:11 But also a set of rules that you can get in trouble for.

Alex Rampell 15:13 Set, set of rules. But, but, but, like, take that example. Like, so you can imagine the individual agent, you know, the human being, the, the lawyer in this example is benefiting, you know, but then does the company benefit?

Russ Fradin 15:23 Right.

Alex Rampell 15:23 Because to a certain extent, like, I'm paying you the same amount of money, uh, I want you to work for eight hours a day.

Russ Fradin 15:28 Right.

Alex Rampell 15:29 So actually my expectation should be that if you're, whatever, I don't know what the lawyer does-

Russ Fradin 15:33 Sure

Alex Rampell 15:33 ... but, like, drafting legal drafts, like, if you can now do it in four hours versus eight, you know, and spend four hours, uh, you know, playing golf, like, you're thrilled. You got a productivity gain. The company didn't actually benefit.

Russ Fradin 15:45 Right.

Alex Rampell 15:45 Um, so what you kinda want is you want both parties to benefit, which is always tough because sometimes it's very, very hard to sell products to people that eliminate their jobs.

Russ Fradin 15:54 Sure.

Alex Rampell 15:55 That's probably the hardest-

Russ Fradin 15:55 It's basically impossible

Alex Rampell 15:56 ... thing to sell. But, like, I mean, m-maybe kind of taking this example and, and riffing on it, like, uh, now I can do in eight hou- like, in four hours I can do what used to take me eight.

Russ Fradin 16:04 Sure.

Alex Rampell 16:04 But how does the comp- a company's like, "Oh, wow, you're still, you're, you're, you're operating at your baseline."

Russ Fradin 16:09 Above that. Right.

Alex Rampell 16:09 But actually you should be able to do twice as much with this tool. So I guess, like, how do you define the baseline? How do you address that problem?

Russ Fradin 16:15 Sure.

Alex Rampell 16:15 How do you think... Am I, am I framing it the right way?

Russ Fradin 16:17 Uh, I, I think you're, I think you're framing it... I, I think you're certainly framing it a right way for certain size of companies, right? We all know for Silicon Valley what you're gonna have becau- just because of the competition and the equity form of compensation. What you'll have is if I can get done in four hours what I could've done in eight, I'm just gonna work four more hours and then another four.

Alex Rampell 16:34 Right. Right.

Russ Fradin 16:34 And that, that's very different. That, that's... There's some subset of workers at all size companies, probably a larger percent in Silicon Valley, but s-smaller percent in, at GE, right? There are people at GE who wanna one day become the CEO of GE, and those people will work as much as they possibly can. So there's some subset of workers there. For the rest, look, there's an interesting question about how is management gonna evolve overall, right? I think behind all this, the first question i- the first question we're trying to solve is, like I said, do people use these? And a- from a corporation standpoint, for our measures of productivity, which is we're defining it with each of our customers, right? For our measures of productivity as we ping folks, is there a difference in productivity between the heavy users and the lighter users? What we want to measure with that, we w- we're not doing this today, what we want to measure with that is then some concept of raw tonnage of work, right? The ultimate ... There's this lingua franca when we talk about kind of employees of FTE, right? And we all know that you work different than I work and than, you know, various people work, and we all know that, yet if I'm the CFO of, I don't know, let's pick on JPMorgan again. If I'm the CFO of JPMorgan, I have a fundamental, you know, horse sense for what do 1,000 FTE do versus 500

17:46 – 20:27

Defining and Measuring Productivity

Russ Fradin 17:46 FTE versus 2,000 FTE, and AI's gonna break all of that for, for sure. And so our main goal today is just to build the baseline for our customers, which is at the end of the day, are the people using these tools fundamentally more productive than the folks that aren't? Layer on top of that tonnage of amount of time worked.

Alex Rampell 18:03 Mm.

Russ Fradin 18:03 You can get a pretty good... It's never perfect. People are on vacation. You have to measure this as groups, right? Any given person was out sick one day or was on a flight one day or was at a training one day that it's impossible to measure, and you, you... It seems from a systems standpoint they weren't working. They actually were working. They were doing a training, right? So think of this as at the aggregate data. It's never useful. None of this data is ever useful at the Russ Fradin level.

Alex Rampell 18:26 Right.

Russ Fradin 18:26 I mean, to get existential, was I productive yesterday? It's, is unknowable. I, I can't know if I was productive yesterday.

Alex Rampell 18:31 I, I think you were. I, I-

Russ Fradin 18:32 Let... I, I was all for it. But what we're trying to do at the systems level for companies is understand is there some correlation between specific use of these tools on an advanced side, light side, heavy sideHe-heavy user of the tool-

Alex Rampell 18:46 Yeah

Russ Fradin 18:46 ... lighter user of the tool. Were the u- were the users more productive in their job? Were the employees more productive in their job? And then measure on top of that amount of time those segments of workers were actually working. 'Cause the goal if I'm a CFO today is not to understand did Ben do a good job and did Tina do a good job. The goal is to understand I have definitely been asked to spend 50% more on OpEx.

Alex Rampell 19:10 Right.

Russ Fradin 19:12 Did I drive something?

Alex Rampell 19:13 Yeah.

Russ Fradin 19:13 Right? And, and then by the way, there are interesting questions. We know this around staffing size and will companies get more done because people will actually work eight hours?

Alex Rampell 19:21 Yep.

Russ Fradin 19:22 Look, it will turn out it-- I suspect it is true that this is one of the things managers do. I suspect it is true over time if it becomes clear that all of your employees are now working four hours a day instead of eight, you will probably decide to have fewer employees and-

Alex Rampell 19:40 Right

Russ Fradin 19:40 ... they will work, the remaining employees will work six hours a day. So I'm not sure I really buy in the next couple of years you will see people in large companies actually just working half as much. You know, sole proprietors i-is what it is. Like I, if I were a sole proprietor lawyer, kind of my only measure of productivity today is to myself anyway, right?

Alex Rampell 20:00 Right.

Russ Fradin 20:00 It's how hard do I wanna work for and how much money do I wanna make?

Alex Rampell 20:02 Well, there the principle is the agent.

Russ Fradin 20:03 Right. So that's fine.

Alex Rampell 20:04 This is why it's like yeah... But this is, this is why it's so important, and this is why, I mean, candidly, I love what you do.

Russ Fradin 20:09 Sure.

Alex Rampell 20:09 Obviously, I love what you do. That's why you're here. Um, but, uh, you-- there is no baseline. Like it's like-

Russ Fradin 20:15 Right

Alex Rampell 20:15 ... did this work? Well, first you have to know how, like, you have to define the outputs. You have the inputs-

Russ Fradin 20:20 Right

Alex Rampell 20:20 ... which are largely just like time and money.

Russ Fradin 20:22 Right.

Alex Rampell 20:23 Um, and then you have the outputs. And part of it is actually it is

20:27 – 21:41

Goodhart’s Law & the Pitfalls of Metrics

Alex Rampell 20:27 kinda complicated to come up with an output. Like, uh-

Russ Fradin 20:28 For sure

Alex Rampell 20:29 ... do you know, uh, Goodhart's law?

Russ Fradin 20:31 Uh, go ahead.

Alex Rampell 20:32 So Goodhart's law, I lo- I love this one. It's like when a target becomes a measure... Sorry, when a measure becomes a target-

Russ Fradin 20:38 Yes

Alex Rampell 20:38 ... it is no longer accurate as a measure.

Russ Fradin 20:41 Yes.

Alex Rampell 20:41 Right? So if I say, "Okay, I'm gonna judge you based on... I'm gonna..." Like, how many emails are sent every day? Well, that's a measure. But once it becomes a target, it's like, "I want you to send more emails," well, you're no longer m- Like, the measurement gets-

Russ Fradin 20:53 Sure

Alex Rampell 20:53 ... corrupted-

Russ Fradin 20:54 Th-this is p-p-

Alex Rampell 20:54 ... because, because now, now people decide to do more things to hit this target, and it's no longer an objective measure. So, you know, part of it is if I'm trying to figure out like, okay, there's a product called Harvey.

Russ Fradin 21:05 Right.

Alex Rampell 21:05 A lot of people love Harvey, and it seems to make people a lot more productive, but compared to what?

Russ Fradin 21:11 Right. And so to me, the only way you answer that... Let, we'll talk about Harvey. Nothing against Harvey. I'm sure Harvey's amazing. Uh, to me, the only way to really understand this, and that's why I think the traditional way companies are doing this just doesn't work at all, which is, hey, let's survey the people that use Harvey and ask them if they were productive. And by the way, they will all say yes because no one ever answers they weren't, number one. And number two, uh, my boss paid for the product. I'm-

Alex Rampell 21:33 Right

Russ Fradin 21:33 ... I'm gonna say it was a good product, right? Unless we all universally hate it, which I assume is not true of Harvey 'cause everyone seems to like Harvey, so that's wonderful. So I think all you can

21:41 – 24:18

The Harvey Example: Usage vs. Value

Russ Fradin 21:41 actually do is... It's why I think the traditional way of measuring this is broken. Look, it's why we started LearnIt. All you can really do is understand without asking people how much usage of Harvey are these people actually doing, right? We have five people, we have six people, whatever they'd say on a survey. Two have never logged in, right? We've all seen the joke about, you know, your project is due, it's due in an hour, you said you were caught up, and then you, "Oh, shit, I have to ask permission for this Google Doc," right? So there's two of the six, I'm making these numbers up of course, two of the six people actually signed up for Harvey the day they were told and then never went back to it at all. They're very happy with the way they work. They work that way all day, every day. Two of the six log in and use it a little bit, and two of the six use it all the time. The only way I can even begin to understand if that software is valuable is by knowing that data passively-

Alex Rampell 22:28 Yeah

Russ Fradin 22:28 ... without asking those folks a question.

Alex Rampell 22:30 Yeah.

Russ Fradin 22:31 And then asking everyone the same questions about productivity and measure it with amount of work actually output. And if I take those three things together, then I can begin to form an understanding of was Harvey useful.

Alex Rampell 22:42 Right.

Russ Fradin 22:43 You and I had a discussion with someone, uh, where they were talking about one of the ways they m- uh, incent their engineers at their company is they have a leaderboard of the amount of money each engineer spends on Claude Code. And the founder was talking about how, uh, he went to one of his best engineers and said, "I don't understand what's happening. You're one of our best engineers. Why aren't you spending m- any money with, uh, Cursor?" I'm sorry, not Claude Code. "Why aren't you spending m- any money with Cursor? I, I really don't get what's going on." And so that, that was an example of for these companies where they're very developer-heavy-

Alex Rampell 23:12 Right

Russ Fradin 23:13 ... you probably don't need us. If you're a very developer-heavy company, probably measuring amount of money spent on Cursor plus your kind of normal management understanding of is this person actually working? If they come in for two hours a day, you may be happy with that, you may not. That's gonna be company specific and lifestyle specific and culture specific. But you're in the office, I see you're there. You're not spending any amount cur- money on Cursor. What's up, right?

Alex Rampell 23:38 Right.

Russ Fradin 23:38 We have these metrics. The issue, though, is we see this explosion of there's hundreds of AI tools, and companies have hundreds of roles. And so that's why we wanna try and replace, you know, the McKinsey Corporate Health Index with Tows Watson or the Accenture surveys with some real useful data around AI. But I think that kind of Cursor example really crystallized in my mind what you'd wanna be able to do for a whole company, which is how much did this person work? So I have that quali-- you know, I have that quantitative judgment. Qualitatively as a manager, right, we're not replacing this, did they do a good job? And then fundamentally, did they use the tools? And when you take those three things together, that's the only way you're gonna have measurement. And like I said, when you think about my, you know,

24:18 – 27:38

Surveys vs. Behavioral Data

Russ Fradin 24:18 micro world of if you really think JP Morgan's gonna go from spending 18 billion a year in IT to 30 billion or 40 billion, the CFO's not just gonna say, "No problem," right? Today, our customer is the CIO. I think over time our customer becomes a partnership with the CIO and the CFO. The numbers are just big.

Alex Rampell 24:34 Yeah.

Russ Fradin 24:35 It's like cloud spend. They, the numbers are just so big people are going to pay attention. It's going way beyond experimental.

Alex Rampell 24:41 But, and obviously the companies themselves, like if you ask any company that is trying to sell you anything and you ask them-

Russ Fradin 24:48 Sure

Alex Rampell 24:48 ... "Does your product work?"They will probably 99 times out of 99-

Russ Fradin 24:52 Sure

Alex Rampell 24:52 ... say like, "Of course it does."

Russ Fradin 24:53 Sure.

Alex Rampell 24:53 Right? That, like, it's the best, it's the best. You need to have an independent arbiter. Um, and that, that's where you guys come in. But kind of like double-clicking on this, this point before, it's almost like reinforcement learning at a company-wide level-

Russ Fradin 25:05 Sure

Alex Rampell 25:05 ... right, of what is the outcome that I'm looking for? And sometimes it's clear-

Russ Fradin 25:09 Right

Alex Rampell 25:09 ... right? So, um, and th- this is where the, the measurement and target thing is also-

Russ Fradin 25:13 Sure

Alex Rampell 25:14 ... relevant because it's like I want you to write more lines of code. Well, if that's... There's a measurement of, like, how many lines of code were written-

Russ Fradin 25:20 Sure

Alex Rampell 25:20 ... but if it becomes the target, then you're just, like, writing gobbledygook code and, like, you're... For sales, it's very easy.

Russ Fradin 25:25 Right.

Alex Rampell 25:26 I want you to sell more stuff, but there's a lot of latency between, like, you go talk to a customer, and then you go collect money. So you might have targets in between. You might have measurements in between.

Russ Fradin 25:36 Sure.

Alex Rampell 25:36 If you're a lawyer, draft more contracts. So I guess, how do you try to define the goals? Because some of them are just, like... It's kind of, I think of it as, like, background information that's going through. It's like emails that are being sent-

Russ Fradin 25:48 Right

Alex Rampell 25:48 ... or, you know, Slacks that were sent or Google Docs that were edited. Like, there are these key-- Like, there are these very, very clear measurements, but those aren't necessarily outputs.

Russ Fradin 25:59 So first, your point on measurement, this is why I said earlier, if you think about any time true third-party measurement exists, there's this interesting dynamic, and we saw this at comScore, but everyone has seen this anytime they tried to build a third-party measurement company. Omniture saw this in the early days. Google at some point fought it and then actually bought Urchin and built Google Analytics, right?

Alex Rampell 26:20 Right.

Russ Fradin 26:20 Because it turned out it's actually good when your customers can track value if what you do is actually valuable.

Alex Rampell 26:25 Right.

Russ Fradin 26:26 And so my general perspective is I think today a lot of the AI companies probably look askance at us, but I think over time, certainly the AI tools that actually provide value are gonna love us, right? The way you will ultimately unlock real enterprise budget is because people believe these tools are actually valuable. So w- what we do today, and this is a journey, right? The company's about a year old. So what we do today is we work with all of our customers, say, "Look, here are the baseline productivity questions that are gold standard that people have asked for seventy years." You know, there's pros and cons to them, but this is-- You have to start somewhere. This is where we start. And let's define a set of metrics for each of your, uh, departments. One of the things we've found that actually seems to matter, not as a metric that companies share with their employees, 'cause then you have the Goodhart's law problem, but as an actual reality on the ground-

Alex Rampell 27:14 Right

Russ Fradin 27:15 ... is fundamental responsiveness. There is an element of, I spend some amount of money on my legal department, and I am happy with the amount of productivity they do today. So there is an element of, unless I'm trying to fire lawyers, which I'm not, you can argue I-- how would I measure the value of software? I guess my lawyers might be happier, but I don't have a churn problem there, so frankly, why should I do this?

27:38 – 30:00

Interdepartmental Responsiveness & Real-World Metrics

Russ Fradin 27:38 And so what we found is, one of the things we found is just almost an interdepartmental SLA, which is it turns out if I roll out these tools and I'm not firing employees, 'cause one way to look at this is could I fire half my lawyers? Turns out companies don't really like firing people. Companies do fire people if they have to, but I've actually never met a CFO that got excited about firing thirty percent of the workforce. Outside of call centers, that's a different issue we could talk about. Like, companies treat their call center employees different from the rest of their employees. But outside of call centers, I've never met a CFO who is... If you went to a CFO and said, "You can fire half your FP&A people," he doesn't wanna fire Tina. He knows Tina. He's met Tina's husband and children. Like, he doesn't wanna fire Tina. He'd like Tina to be happier and more productive, and actually, he'd like her to do a great job and never quit, right? Companies don't really like churn. So one of the metrics we found that people seem quite excited about is just did this raise or lower the interdepartmental responsiveness? So our measure would be, am I now comfortable sending more things to legal, right? If I'm gonna keep my legal department the same size, I'm not gonna start suing more people. We're talking about companies here, not law firms, where it's a different measure of productivity, right? They're cost centers, not profit centers. So one thing to do it is did over time, because my lawyers are now more productive, are other departments asking them more questions? Are they getting their responses faster? When I'm in product and I'm asking for input from engineers, are they responding more quickly, right? That is a, that is a good way for me to see behaviorally we become more productive that's not lines of code. Now, by the way, I agree, if you expose the metric and say, "Hey, you better be responsive," people can lie. They can-

Alex Rampell 29:10 Yeah, yeah

Russ Fradin 29:10 ... send Slack messages back and forth. But what I'd really like to understand is, as a map, which of my departments use these tools more, and do they become more responsive to my other departments? 'Cause there's an element of when you're at a big company, people know this. It's one of the reasons small companies do so well in innovation. There's just a giant coordination problem for all of these companies, and we know this. And, you know, in Silicon Valley, it's fun to make fun of these companies, but actually every pr- entrepreneur's secret dream is to become so large that they have a giant-

Alex Rampell 29:36 Right

Russ Fradin 29:36 ... bureaucratic company.

Alex Rampell 29:37 Right.

Russ Fradin 29:37 Of course. Google did not plan to have a giant bureaucracy thirty years ago. They just became so successful, they now do have a giant bureaucracy.

Alex Rampell 29:43 Right. And, uh, you know, kinda that, that's kind of a good segue into perhaps the, the state of A-AI enterprise, right? So you went on this whole, like, list... You talked about three hundred and fifty people?

Russ Fradin 29:52 [clears throat] Yeah. We, we interviewed-

Alex Rampell 29:54 That's a, that's a large number of people

Russ Fradin 29:54 ... three hundred and fifty heads of IT at major companies.

Alex Rampell 29:57 Um, and a-across the whole, the whole gamut,

30:00 – 32:59

Enterprise AI Adoption: What the Data Shows

Alex Rampell 30:00 right?

Russ Fradin 30:00 Yeah. Yeah, yeah.

Alex Rampell 30:00 It was- it wasn't just, uh, you know, kind of Silicon Valley companies that-

Russ Fradin 30:03 No, not a, not a... I mean, in all honesty, my whole career, basically, I spent a couple years helping my friend at Carbon, and I spent a year trying to fix Wine.com. But other than that, my whole career has been selling software to large companies, mostly large companies or older companies.

Alex Rampell 30:18 Right.

Russ Fradin 30:19 Yes, there's the occasional Silicon Valley company that grows very quickly, but if you're in the Fortune 500, you are going to be twenty-plus years old ninety-nine percent of the time, right? And so if you're gonna sell to someone with more than a thousand employees, they're almost by definition an older company.

Alex Rampell 30:32 Yeah. So what-- May-maybe give us the highlights of, of what you learned.

Russ Fradin 30:36 Sure. We, we saw a bunch of different things, and, you know, people have seen this before. I, I actually don't think of this... You'll see people turn this into kind of clickbaity fear-mongering things. I don't, I don't really think of it that way. So first of all, we saw, like, this-- we know this from Gartner, this, like, seven hundred billion dollars being spent in enterprise AI. It's growing very, very quickly. It's gonna keep growing quickly. And one of the things we found is something like seventy percent of leaders we talked to saidWe are sure we are wasting money here. It's being spent so quickly. And by the way, shame on us, we had no system to measure this in the first place. I'll get back to the report in a second, but I was talking to a customer today. Uh, why did we sign them as a customer? They're a very profitable business owned by a PE firm, and their bosses, the p- their PE owners, gave them five things they had to do this year, and one of the five was adopt AI across the organization. And he said, "Every board meeting I go in for my other four metrics, I have some report of how are we doing against those reports. And on AI, all I have is the amount of stuff we bought."

Alex Rampell 31:34 Right.

Russ Fradin 31:34 It, it's, it's not-

Alex Rampell 31:35 So yes.

Russ Fradin 31:36 Yeah, yes.

Alex Rampell 31:36 We adopted AI.

Russ Fradin 31:37 Yes, I'm doing great, but it turns out-

Alex Rampell 31:38 We have, we have a large family of AI. We adopted all these AI children.

Russ Fradin 31:40 Yeah, we bought all of it.

Alex Rampell 31:40 Yes.

Russ Fradin 31:40 It's all great, but it turns out we wanna actually do it. And so what we found is these leaders like maybe they are right that 70% of their projects are failing. Regardless if they're right, it's a giant problem they feel that way because they have no system to figure it out in the first place. No one believes 75% of their ad spend is failing. It's not because their ad planners are smarter than their AI buyers. It's because there are 20 years of systems in place to help me understand when I buy this ad campaign, when I spend this money, when I do this app install, whatever it is, did it actually drive value for me? And we just don't really have that in AI, like I said, outside of some very, very specific verticals. And so really the biggest thing we found was... It, it was kind of three things. One, you saw the AI spend. Two, something like, as I said, they believe 70-something percent of AI projects are wasted. But the other thing we found is basically 80, 85%, I can't remember, 80, 85% of the companies we talked to said they really believe they only have the next 18 months to either become a leader or fall behind. So I think one of the things, one of the reasons you've seen this giant unlock in budget is there's tremendous anxiety at these enterprises going like, "We're going to lose if we don't adopt this stuff," yet so we're adopting it quickly. We have no particular idea if it's succeeding. Our employees aren't really using it. By the way, a forgotten group in the company for all of this AI, and, uh,

32:59 – 37:31

Employee Anxiety & Training Gaps

Russ Fradin 32:59 for my last company, we built a very, very large HR, uh, technology company. Uh, we sold into heads of HR, touched all the employees in the company, but we sold into heads of HR. And so as we've talked to a lot of our old customers who aren't really our customers today, but they're influencers, what they will all say at all of these large companies is, "Hey, our employees are really worried." It's not even they're worried they're gonna lose their job. Ev-there's a base level of worry about AI and the economy, all that stuff. It's not even they're worried they're gonna lose their job. It's just they're getting told to use a new system all day, every day, right? Generally, if you work in a large company, there's one or two new systems initiatives a year. Now there's 20 new tools.

Alex Rampell 33:32 Yeah.

Russ Fradin 33:32 They don't know, they don't know what they're allowed to do, and they have no training. How do they actually... How do I get people using these tools? And so you have this weird w- you have this weird, almost perfect storm. It's why we're excited about Larridin. It's why you're excited about Larridin. You have this perfect storm of tremendous growth in budget, tremendous anxiety that none of it is working, tremendous anxiety from their employees about what they're even allowed to do. And so what we're trying to do is, I don't think we solve all of that, that would be an absurd thing to say, but I think we really help with all of that about, like, what is your plan to measure this in the first place? Did anyone use it? Did they become more productive when they did? How do you give them the tools to use it more?

Alex Rampell 34:06 Yeah. Well, and that, that kind of last point is super interesting as well because it's like there's the did it work, how well did it work, you know, what are the measurements, make sure that the measurements don't become targets-

Russ Fradin 34:16 Right

Alex Rampell 34:16 ... like all the stuff that we just talked about. And then to use the metaphor of my, my son who cheated on his math homework, um, there are people that are just like, wow, like they're, they're the go-getters in the company.

Russ Fradin 34:27 Sure.

Alex Rampell 34:27 This is actually why I am convinced that AI is under-hyped.

Russ Fradin 34:31 Yes.

Alex Rampell 34:31 Uh, you know, we have our little group chat where we have another friend who's like, "Oh, all this stuff is over-hyped and it's going to zero." No, no, no.

Russ Fradin 34:35 Totally wrong. Every time I use AI-

Alex Rampell 34:37 Totally wrong

Russ Fradin 34:37 ... it's amazing.

Alex Rampell 34:38 Because you go... It does, it has not diffused. You, you have like the 19-year-old kid-

Russ Fradin 34:44 Yes

Alex Rampell 34:44 ... or, you know, my, my 13-year-old son who's like, "Wow, normally homework would take me two hours."

Russ Fradin 34:48 Right.

Alex Rampell 34:48 "Now it takes me one second."

Russ Fradin 34:50 Right. It's amazing.

Alex Rampell 34:50 Right? And obviously that's bad, right?

Russ Fradin 34:52 Sure.

Alex Rampell 34:52 Like I'm not using him as the, uh-

Russ Fradin 34:54 Sure

Alex Rampell 34:54 ... that's why we confiscated his iPhone, right?

Russ Fradin 34:55 Sure.

Alex Rampell 34:55 But there are these productivity unlocks where it's probably not gonna happen top-down.

Russ Fradin 35:01 Sure.

Alex Rampell 35:01 It's like somebody in the company, and sometimes again, I, I, not to like oversimplify human behavior, but it's like I wanna be lazy and I wanna be rich.

Russ Fradin 35:09 Yes.

Alex Rampell 35:09 Right? Like these are the two things that are motivating people, and like I found this tool that allows me to be lazier and richer that actually helps the company.

Russ Fradin 35:18 Yes.

Alex Rampell 35:18 So not the math, not the cheating, right? It's like you're getting like I can now... I know that my boss was thinking this would take eight hours. I've figured out a way to do it in five seconds.

Russ Fradin 35:28 And it's really good, by the way.

Alex Rampell 35:29 And it's really-

Russ Fradin 35:30 It's amazing

Alex Rampell 35:30 ... it's really good. And the worst thing that can happen, this is like the inverse of everything that we just talked about.

Russ Fradin 35:34 Sure.

Alex Rampell 35:34 The worst thing that can happen is that guy keeps it a secret.

Russ Fradin 35:37 Right.

Alex Rampell 35:37 Right? Because like... And, and he might be afraid. It's like, oh, am I allowed to use this? But what you should do, this is how AI will go from under-hyped to like correctly hyped, um, and correctly diffused, is it's like there's somebody at every big company who has figured out this like I could do something in one minute that used to take eight hours. We need to like make this person a hero-

Russ Fradin 35:59 Absolutely

Alex Rampell 35:59 ... like memorialize this and push it out through the entire company. So like how do you do that?

Russ Fradin 36:03 Th-this was my point on the, uh, what, what we're doing on the AI engagement side, and th-that's a, that's a great question. This is my point on what we're doing on the AI engagement side. This is one of these areas where everyone's interests are aligned. The employee that's working very hard loves recognition, and by the way, he'd like his coworkers to come up to speed. Um, the employee that's scared wants support and wants training, and by the way, companies actually want their employees to be more productive. I know it's a fun thing for a subset of people to tweet about, but like I said, I have yet to find the CEO in all of... I've spent 30 years selling things to CEOs. I've yet to find the CEO who wakes up in the morning and wants to run a smaller company. He wants more employees, and he wants more profit. He wants more revenue. But contrary to popular belief, they want more employees. They like running big companies. They do. You can find old interviews of Larry Page talking about his plan for how Google was gonna have a million employees one day, and he was spending a lot of time thinking about self-driving cars to move the cars around the parking lot 'cause this is before remote work.

Alex Rampell 37:01 Yeah. [clears throat]

Russ Fradin 37:02 And literally where were the million employees gonna park all the cars? And I remember re- I read that15 years ago. Like, that stuck in the back of my mind of I have never met a CEO who wants to run a smaller company. It's one of the reasons, by the way, totally unrelated point in the capital markets, one of the reasons if you ever meet a CEO of a conglomerate, they never wanna break up the conglomerate.

Alex Rampell 37:18 Right. Right.

Russ Fradin 37:18 'Cause they like running bigger companies. It's more fun, right? I've run-- I've had my companies grow. They're fun when they're bigger. It is. It's super cool. And so from an employee standpoint, what we built

37:31 – 41:08

The Nexus Product & Safe AI Usage

Russ Fradin 37:31 with this Nexus product is effectively a product where we said... Or I'll, I'll, I'll use an anecdote. I was in the UK in July, uh, and, uh, I went on a bunch of sales calls, and I was talking to someone at a bank, very large, very reg- European bank is among the most regulated folks in the world, least, you know, fast to adopt new technology for, for good reasons, honestly. And they were telling me a story about how they had a, it was a 28-year-old guy, I don't remember what level that makes you in investment banks, so let's say a director, but I don't remember. 28-year-old guy who was using ChatGPT really, really well in the investment banking side of the business. And they had him create a 30-slide deck, and they did a global call for everyone in the investment bank for this guy to spend an hour walking people through how to use ChatGPT.

Alex Rampell 38:15 Yeah.

Russ Fradin 38:15 I'm sure that was very cool for him, but that's absurd. That's an absurd way to hope people adopt world-changing technology. Another absurd thing to do is to go out and buy some LMS course that HR is gonna buy that, you know, the secret to a lot of LMS is other than things you must do or you will lose your job, like sexual harassment training, HIPAA training in certain orders, no one, no one does it. They, they just don't go. And so how do I actually get people using these tools? And this was my point from earlier, is you wanna help them, A, not look dumb, and B, know they won't get fired. And so what we effectively did is built these wrappers that exist around the models, and so we don't tell people to use Claude or to use Gemini or to use ChatGPT.

Alex Rampell 38:52 Right.

Russ Fradin 38:53 Then the other thing we built, 'cause again, people were also worried about getting fired. They're worried about getting fired 'cause of the economy, 'cause of AI, 'cause whatever, it's a new tool. I would like to think I could get fired. By the way, when you're talking about European banks, there's a lot of regulation that is a legitimate concern of, if our employees do the wrong thing, we will get fined.

Alex Rampell 39:09 Right.

Russ Fradin 39:09 Forget whether you fire them. These companies don't wanna get fined. And so the other thing we did is we, we basically trained our own kind of customized Llama model to block people from asking questions that are illegal or the company doesn't want you to. And so we're not talking about hackers here. True bad actors in the company has plenty of security solutions. What we're really talking about is the X percent of the people who I'm in people ops in HR at a large company, I'm supposed to do a workforce analysis. Am I allowed to go into ChatGPT and load in our full employee database with race and gender? I don't know. I would like to not get fired. Maybe I'm allowed to and maybe I'm not. And by the way, I think it's incumbent on the company to say to our employees, "Here is a safe space. Nothing you can do here is going to get you fired." So we-- Oh, Alex, you're not allowed to up-- That has Social Security data. Don't, don't share that. You're not allowed to ask that prompt because in Europe, we're not allowed to use HR-- we're not allowed to use AI to write employee reviews. I don't know if that's a good law or a bad law. I, I didn't write the law. But there are companies that look at the EU AI regulations and say to themselves, "Our read of the regulation," I'm not gonna, you know, prosecute that, "our read of the regulation is we believe it's illegal, and we will get fined-

Alex Rampell 40:20 Yeah

Russ Fradin 40:20 ... if our employees use AI tools to do employee reviews." So great, if I am a European-based company and I want my employees using AI, I have to block them from using it for those use cases. And so what we've tried to build is this almost like harness to say, "You can be more productive. You're not gonna look dumb. You're gonna be more productive, and you're not gonna make any mistakes that get you fired." And so what we found is that actually drives more AI usage, surprising literally no one. And from a company standpoint, what do you want? A, I want the usage, and B, I wanna build up that IP of what really works for my company. It's a total unlock. And same thing on the coding side, right? Uh, you know, Cursor has taken mediocre engineers and made them good, but it's taking amazing engineers and made them gods, right? And so our goal should be how do we help people get much more productive with all of this? How do we help them use Cursor more effectively, Harvey more effectively? We started with

41:08 – 43:40

The Future of Work: Job Loss or Job Creation?

Russ Fradin 41:08 all of the LLMs more effectively.

Alex Rampell 41:09 Yep. Um, so maybe we could talk, I mean, this is a little bit philosophical, but future of work. Um, because to a certain extent, like, all right, you're, you're the me- like, if you're the measurement-

Russ Fradin 41:22 Sure

Alex Rampell 41:22 ... like, the measurement inevitably will become a little bit more of a target.

Russ Fradin 41:26 Mm-hmm.

Alex Rampell 41:26 Um, and I, I always like to kinda remind people that I think it was 97, 98% of Americans when the Constitution was ratified were farmers.

Russ Fradin 41:34 Right.

Alex Rampell 41:34 And they all lost their jobs, um, due to these pesky things-

Russ Fradin 41:37 And all worked out okay

Alex Rampell 41:38 ... like the tractor and-

Russ Fradin 41:39 Right

Alex Rampell 41:39 ... fertilizer and all these things, and l- I think the, the average, uh, life expectancy was, like, 35, and-

Russ Fradin 41:44 Right

Alex Rampell 41:44 ... most children died in childbirth or shortly thereafter. It's like, you know, we, w- things have changed-

Russ Fradin 41:49 Right

Alex Rampell 41:49 ... but this is what, uh, this is what technology brings you. Um, I mean, nobody knows the answer to this, but given that you're, you're in char-

Russ Fradin 41:56 Sure

Alex Rampell 41:56 ... you're in charge of a company that's, that's measuring AI productivity and human productivity and kind of AI and, and humans working together, I mean, what, what's your timetable for, like, how fast things change? Are we going to see, you know, net new jobs created? Like every-- And by the way, like-

Russ Fradin 42:09 Sure

Alex Rampell 42:10 ... behind every one of these, there are all sorts of jobs that start becoming around that didn't exist before. So maybe that's kind of part two of the question, because, like, the job that we have right now, like, you know, filming a podcast-

Russ Fradin 42:20 Sure

Alex Rampell 42:20 ... like, that wasn't a job like 200 years... Like, there's so many jobs that just, like, nobody could even think of.

Russ Fradin 42:25 Of course.

Alex Rampell 42:25 So, so I guess, like, where do you think things are going, and, like, what types of, maybe to put a nuance on it, like, what types of, like, future jobs do you see in and around this, like, new stuff?

Russ Fradin 42:34 So I don't buy for a second there's gonna be large-scale job loss because of AI, f- frankly, because of what we've seen through all of history, which is just flat-out capitalism. If my two choices are I can maintain my base level of productivity, but fire a bunch of my employees and be more profitable, that is a fine idea in the short term. If I'm-- That's probably a good idea for a PE firm to go along and go around and buy a bunch of m- minorly profitable companies, fire half their employees, and make them more profitable, but that's what PE firms have done for a long time for non-competitive companies anyway, yet employment has still increased, right? So you can argue we've always had a func- or for the last fif-40 years you can argue we've had a function whose goal is to take underperforming companies and fire a bunch of employees, right? And let's, let's say that that's what PE firms have done and that's, you know, what AI could theoretically do, yet employment has increased. So I, I don't buy an AI... And look, it's philosophical and, you know, I don't have any special expertise 'cause I am building a measurement company. But I don't buy it because your competitor across the street is not gonna fire all those employees. He's just gonna do more with those employees and he's gonna

43:40 – 52:45

The Competitive Advantage of AI

Russ Fradin 43:40 kill your business, right? I mean, this is the, um, Jeff Bezos "your margin is my opportunity" line. To the extent that AI is going to drive up your margin, that will be all of your competitor's opportunity to be less profitable and compete with you. So i- other than some very niche, monopolistic, I can fire everybody, you know-

Alex Rampell 43:58 Right

Russ Fradin 43:58 ... one-man firm, like, will we have one-woman firms that do a billion in revenue? Probably. But today we have very profitable, you know, one-man, one-woman operations. Not many people work at the Joe Rogan podcast. I, I don't think that many people work for Ben Thompson Incorporated, and yet I imagine those are quite profitable businesses-

Alex Rampell 44:15 Right

Russ Fradin 44:15 ... the best I can tell. So that, that, that's amazing, and there'll be a ton of opportunity to be more successful solo entrepreneur, so I absolutely believe there'll be even more entrepreneurs. But at a very high level, I just don't believe the Fortune 500 will employ fewer people in 30 years than they do today, because the ones that try and cut all the people will no longer be in the Fortune 500. So I just flatly... Because we live in a competitive world, it's w- we haven't seen any proof yet-

Alex Rampell 44:41 Yeah

Russ Fradin 44:42 ... that the economy is zero-sum, right? W- maybe, right, you can argue that, but we haven't seen any proof yet. GDP keeps increasing. It increases slower in some places and faster in other places, but it's generally grown. Employment has generally grown. I just don't know why you'd believe that this time is different, because the competitive point of view. I-- The tech is different. The tech is amazing, but fundamentally what will almost definitively happen, there is an interesting theoretical question that, you know, is more like a, you know, I don't know, a Ivy League grad school, you know, discussion about wouldn't it be more fun as a society, wouldn't we all be happier if everyone agreed we'd work half as much and be just as productive as we are today? I don't know, maybe, but that's not human nature, right? And so I'm not even sure that's true. I, I tend to believe in the Tyler Cowen point that all that really matters is growth. And so my general perspective is y- you as a VC would just never get excited if one of your companies came in here and said, "Hey, we got to 100 million in revenue." And you know why? Because AI tools are so good, we're gonna fire 90% of our employees and we're gonna make 90 million in profit. You would not be excited with that entrepreneur because you know that Sequoia is gonna fund a direct competitor to that company who's gonna keep hiring-

Alex Rampell 45:56 Yeah

Russ Fradin 45:56 ... who's gonna be happy with 10% margins, and is gonna destroy your company. And we, we all know this, so I-

Alex Rampell 46:00 Right

Russ Fradin 46:00 ... I don't... It's one of these things of, like, you know, there's a lot of fun headlines about, oh, AI, and then you have to have the counterpoint of, oh, it's gonna take the jobs, and oh, kids these days. I don't know, like, when you... We, we all know this. We've all seen this. You can find articles about when the TV came out, it was the end of reading. When newspapers came out, it was the end of conversation. So I do think it is scary that new tools are coming out and it is impacting the entire globe of all knowledge workers everywhere. So what? So I think there'll be opportunities to be podcasters. There probably will be more plumbers. There will be a lot more employment around building data centers, right? There's going to be a whole set of engineers. Maybe we will ha- need a lot more astronauts. Elon says we're going to Mars. Like, someone is going to have to scrub the toilets in the space station, and someone is going to have to pilot the plane, to pilot the ship to the space station.

Alex Rampell 46:50 Well, it's the self-driving-

Russ Fradin 46:51 Yeah

Alex Rampell 46:51 ... s- s- self, self-driving spaceships. Um-

Russ Fradin 46:54 So, uh, by the way, there is a chance I will turn out to be wrong, and in that case, I don't know, uh, maybe I'll spend more time on vacation.

Alex Rampell 47:00 Well, it's interesting. Uh, I, I talked to this economist, Ed Glaeser, um, uh, I think he's at Harvard, and he was saying... Uh, I, I asked him this question.

Russ Fradin 47:09 Sure.

Alex Rampell 47:09 It's like, you know, "What, what's gonna happen with jobs, and, you know, how do you compare this to everything else?" He's like, "Well, what's really interesting is that this is arguably the first time that the job losses might be borne by, like, white collar-

Russ Fradin 47:23 Sure

Alex Rampell 47:23 ... super educated people."

Russ Fradin 47:23 That's why everybody gets scared.

Alex Rampell 47:25 Well, but, but so he actually had a different framing on it. So, so yes, agreed, but, uh, almost tautologically, hyper-educated people are hyper-educated-

Russ Fradin 47:34 Sure

Alex Rampell 47:34 ... so should be able to rejigger themselves and do something else, versus in all of these previous revolutions where it's like you have somebody that really has no skills-

Russ Fradin 47:45 Right

Alex Rampell 47:45 ... and just, like, showed up at work with no skills and got paid.

Russ Fradin 47:49 Yes.

Alex Rampell 47:50 And there are a lot of jobs that look like that when there's tremendous labor shortages.

Russ Fradin 47:54 Yep.

Alex Rampell 47:54 So if you were doing anything in 1849-

Russ Fradin 47:57 Right

Alex Rampell 47:57 ... apparently-

Russ Fradin 47:58 Right

Alex Rampell 47:58 ... in California, like, boom, right? It's like there's just like, "Oh, you're a human-

Russ Fradin 48:02 Right

Alex Rampell 48:02 ... you have a pulse-

Russ Fradin 48:03 We need someone with a pick. Go

Alex Rampell 48:04 ... like, like, g- go do this."

Russ Fradin 48:05 Right.

Alex Rampell 48:05 Or just like, you know, uh, "Do you see that line over there? Like, uh, yeah, like straighten it out."

Russ Fradin 48:09 Right.

Alex Rampell 48:10 So, um, but what's different is that, and yes, it's scary for some people, but, like, everything right now, and maybe robots will work better in the future, but, like, everything right now is bit manipulation going after or augmenting, I would argue more augmenting, white collar hyper-educated people by virtue of the fact that they're hyper-educated.

Russ Fradin 48:30 Sure.

Alex Rampell 48:31 Um, it might not... Like, this does not mean that, like, you know... What, what happened to Detroit?

Russ Fradin 48:35 Sure.

Alex Rampell 48:35 Right? It's like that w- that actually wasn't about automation. That was about, like, the Japanese built better cars.

Russ Fradin 48:40 Right.

Alex Rampell 48:41 There, there are a lot of reasons why that happened, but what do you do with somebody who had, like, a very, very high-paying job but actually didn't have that many skills-

Russ Fradin 48:49 Right

Alex Rampell 48:49 ... and now they lost that job? Well, because they don't have any skills, they can't find another job.

Russ Fradin 48:52 Right.

Alex Rampell 48:53 Whereas if you are highly skilled, you will find something else to do.

Russ Fradin 48:56 And I do think there's an element of, look, there are certainly a set of people who were pretty highly educated. They were in good classes. They got into a good school, whatever that meant. They got a good job. They worked pretty hard in their 20s, a little less hard in their 30s, and a little less hard in their 40s.But they're paid pretty well, and those people probably are a little uncomfortable today because their career-- Fra-frankly, there's, there's some professions that just require continuing edu-education. By the way, if you're an electrician or a plumber or a doctor or a lawyer, some of these professions just require constant upkeep and constant education. That's not true in a lot of professions. There are a lot of jobs where you get to 40 or 50, and you can keep doing a good job, but you don't really have to learn much new. You, you can just keep doing a good job doing what you're doing, and you don't have to learn much new, and that's probably quite uncomfortable.

Alex Rampell 49:43 Right.

Russ Fradin 49:43 It, it-- I, I acknowledge it is quite uncomfortable for those people, but to your point, they're educated, they have skills. We have much more of a knowledge economy, so I don't necessarily have the issue of I literally have this house in Detroit, the jobs are now in Knoxville, right? Forget Japan. The jobs are now in Knoxville. I don't wanna move to Knoxville, right? We all know the data on mobility and housing costs and all that.

Alex Rampell 50:04 Yeah.

Russ Fradin 50:04 So sure, but at the end of the day, to your point, yes, there's a set of people who've probably been slowly working less and pushing themselves less, and now they have to push themselves more, and that just is. Uh, my joke all the time, I, I won't say the company I use, but part of my sales pitch for Larridin when I'm talking about this, I say, "Look, when we talk to employees, you know, look, your average employee is a 42-year-old associate brand manager, and if you ask them what they want out of AI, w-they do want it to go away. Their number one risk w-wish would be that it would just go away. I'd like that. I'd li-- 'cause I liked yesterday." But, uh, we'd make a lot of money if we had the to-power to make AI go away, right? Super big blackmail business, but we don't have that power. So all we can do is give you the tools to use these things better-

Alex Rampell 50:44 Right

Russ Fradin 50:44 ... and help you be more productive and help you as a manager understand if your team is using these tools better. And so, you know, m-my macro point is I just don't really believe there will be wide-scale mass unemployment. Might an individual have to push themselves more? Yeah, for, for sure, and some of those will be sad about that, and like, um, that same thing is true in the entertainment industry, right? The, the jobs have moved, and jobs have shift, and people don't watch movies the way they used to, and, you know, TV seasons used to be 22 episodes, and now they're eight 'cause consumer preferences have changed. And so that is probably lousy if you were a guy who played a part on "Law & Order," right? I mean, that probably is uncomfortable.

Alex Rampell 51:23 Right.

Russ Fradin 51:23 I'm not being callous. I'm sure there's many ways it will impact my life negatively, but I, I just don't buy that they won't be more educated.

Alex Rampell 51:29 Yeah, and a, a lot of this actually predates AI.

Russ Fradin 51:31 Sure.

Alex Rampell 51:31 Uh, there, there's this great, uh, article or interview with the CEO of Waste Management-

Russ Fradin 51:36 Okay

Alex Rampell 51:36 ... before ChatGPT came out. Um, and he was saying, "I have unlimited... Like, I get resumes every day from somebody who has an MBA and wants to work, like, in our office for..." And, like, the k- they're, like, negotiating against themselves.

Russ Fradin 51:50 Right.

Alex Rampell 51:50 Like, the price keeps going down there. Um, so 100 jobs for... 100 applications for-

Russ Fradin 51:55 Right

Alex Rampell 51:55 ... every opening, or I forgot what he said. "I need to hire truck drivers."

Russ Fradin 51:59 Right.

Alex Rampell 51:59 Uh, somebody who... And not just, like, self-driving.

Russ Fradin 52:02 Right, right.

Alex Rampell 52:02 Like, I need somebody who actually is collecting the trash.

Russ Fradin 52:05 Right.

Alex Rampell 52:05 Like, that's what Waste Management does. For $150,000 a year, I can't find them.

Russ Fradin 52:09 Right.

Alex Rampell 52:09 So it is kinda interesting how, how things have flipped. But I, I guess the other thing that I would say is, uh, I would, I would almost argue that a lot of AI's problem right now in terms of diffusing into the workplace, it's almost a product marketing problem.

Russ Fradin 52:24 Sure.

Alex Rampell 52:24 Right? Where it's like, okay, AI can do anything.

Russ Fradin 52:27 Right.

Alex Rampell 52:27 Right? But I'm not looking for anything. Like, if I say, like, "Hey, I can do anything," and you're like, you're like, "Oh, I don't, I don't need you."

Russ Fradin 52:34 Right, right, right.

Alex Rampell 52:34 It's like, no, I can do this one thing very, very... Oh, you do that? And, and, like, that, that needs... Like, th-this is kind of... I think once you have more of these articulations of

52:45 – 54:00

The Product Marketing Problem in AI

Alex Rampell 52:45 what can be done, and, like, the things that have really kind of gone hypergrowth, it's like, oh, like, I have AI, it does everything.

Russ Fradin 52:50 Right.

Alex Rampell 52:51 Oh, I will help you code a hundred times better.

Russ Fradin 52:53 I will help you code better.

Alex Rampell 52:53 Oh, I want that.

Russ Fradin 52:54 I have this chatbot for... Yeah, look, a long, long time ago, but it's, it's funny when you get old. Uh, a long, long time ago, I was, like I said, I was the first guy at comScore, and comScore's sales pitch in the early days, comScore, for those that don't know, right, basically had all the data for everything that was happening on the internet. And so the founders were true geniuses, and they basically knew everything that was happening everywhere on the internet. And our sales pitch in the early days would be, "We know everything." I mean, obviously not, but would basically be like, "We know everything. What would you like to know?" And it turned out that wasn't a really good sales pitch. You would sometimes accidentally run into someone who would go, "Oh, my God, I need to know this. Could you do this?" And we'd say, "Yes, we could," and there you go. But it turned out we only had a couple sellers that could figure that out in real time. And then it turned out if we said, "We can tell you the market share for Visa versus MasterCard versus others in Japan," turns out Visa really wants to know that.

Alex Rampell 53:45 Yeah, yeah.

Russ Fradin 53:46 Uh, but I can also tell you the share for your pharmaceutical drug versus others in research online. Turns out they also want to know that.

Alex Rampell 53:52 Right.

Russ Fradin 53:52 Uh, after, I'm gonna get it wrong, but Ford had some giant issue with Bridgestone tires setting on fire. It turns out they

54:00 – 56:58

The Importance of Specific Use Cases

Russ Fradin 54:00 really do wanna know that did employees... did search for Ford get worse because of Bridgestone, right? So people do wanna know these specifics, and like I said, I, I think that's exactly the right way to think about it is you need this product marketing problem. Look, it's why we're so focused on what's happening, are they more productive-

Alex Rampell 54:14 Right

Russ Fradin 54:14 ... how do you get them to use it more? We can actually do a lot of things, but you can't sell things that way. I mean, that's, that's more, like, general entrepreneurially advice.

Alex Rampell 54:21 Right.

Russ Fradin 54:21 But it turns out building something amazing that people don't know how to use mostly doesn't work, unless it does, which is ChatGPT, right? So one in a million times it does work out, Facebook, ChatGPT, on the consumer side.

Alex Rampell 54:33 Well, just like there, it's just like it's magic.

Russ Fradin 54:35 Yeah, it's magic.

Alex Rampell 54:35 If you, if you show somebody a magic trick or you get somebody addicted-

Russ Fradin 54:39 Right

Alex Rampell 54:39 ... um, and y-you'll guess-

Russ Fradin 54:41 Sure

Alex Rampell 54:41 ... which one I'm referring to for, for which. But, um, the... If you ever watch Seinfeld-

Russ Fradin 54:45 Yep

Alex Rampell 54:46 ... there's this great episode where Jerry buys his father a Sharp Wizard, which was, like, an early PalmPilot ki-

Russ Fradin 54:52 Oh

Alex Rampell 54:52 ... It was, like, this early smart computer, like, in the 1990s. Never went on to, to great things, but it did everything.

Russ Fradin 54:57 Right.

Alex Rampell 54:57 It was like, you know, you could run these applications, and Jerry's trying to explain it to his dad, and he's like, "Well, I don't, I don't get it. What does it do?" He's like, "Well, like, look here. It has a tip calculator." He's like, "Oh, my God, a tip calculator?"

Russ Fradin 55:06 Right.

Alex Rampell 55:06 And then he explains it to all of his friends. He's like, "Look at this."

Russ Fradin 55:08 This tip-

Alex Rampell 55:08 "My, my son, he's a comedian. He's doing great. He got me a tip calculator." And Jerry's like, "No, it does other things." And it often ends up being frustrating for the company that does the other things-

Russ Fradin 55:18 Sure

Alex Rampell 55:18 ... because they aspire to have this, like, more broad horizontal platform. But what we kinda need is more of these, uh, these tip calculator things. Um, so I know, I know we're, we're-

Russ Fradin 55:27 Yes, thank you. This has been a lot of fun.

Alex Rampell 55:28 But actually, w-why don't we just, uh, is there anything that we haven't talked about that you wanna get in, like a little sol-soliloquy that you can, uh-

Russ Fradin 55:36 No, I... Look, I think, Pete, I'll leave you with two thoughts, the one, one, one related to something you brought up in the conversation and the professor, but one related to Larridin. So look, my general perspective is anytime you see some giant shift in budget, you're going to build a set of very important but very boring tools. What's actually happening? Are people more productive? How do I get them to use it more? And, you know, there's, there's a ton of business there, and then I'll leave you with a... w-leave you with an unrelated thought to your, to your Harvard professor. So as you said, uh, uh, our kids go to the same school, and, uh, my oldest kid is in 12th grade and just got into college, and he got into his top choice, and he's very proud of himself, and it's a very highly rated school, and I'm very happy for him. And he came home and he said, "Dad, look at this ranking." And he showed me the new US News World Reports ranking that showed the different rankings. And, um, and I was like, uh, his name's Henry. I said, "Henry, I'm very proud of you. There's gonna be a lot of different rankings over a lot of different years, and there's only one thing you have to know for sure. Whatever the ranking says, everyone knows number one is Harvard."

Alex Rampell 56:34 [laughs]

Russ Fradin 56:34 "And so it doesn't matter. Don't get excited. Whatever it says, wherever it puts your school, everyone always knows whatever the ranking is, Harvard's number one." I did not go to Harvard. [upbeat music]

Episode duration: 57:07

In the uListen app, the transcript lights up word by word as you listen. Get uListen

Transcript of episode VMv00WR8EaA

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.