Skip to content

By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

Silicon Valley GirlSilicon Valley Girl

Wharton’s Top AI Professor: Stop Starting Software Companies — The Real AI Money Is Here

Ethan Mollick explains how to work with AI agents without giving up your own thinking.

Ethan MollickguestMarina Mogilkohost
Nov 3, 202653mWatch on YouTube ↗

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

Transcript

  1. 0:00 – 0:50

    Intro

    1. EM

      We can easily give up skills to the AI, and we may never get those skills back.

    2. MM

      This is Ethan Mollick, one of TIME's 100 Most Influential People in AI. He leads the Generative AI Labs at Wharton. When everybody has Claude advising on the strategy, how do you avoid sounding just like everyone else?

    3. EM

      You don't want the AI making those choices, even if they're good ones.

    4. MM

      No.

    5. EM

      You want to be inserted into that process. I'm deciding when to insert my own agency into the system rather than just sending off agents to do work.

    6. MM

      The adaption piece has to happen from scratch- [laughs]

    7. EM

      Yes

    8. MM

      ... every, every two weeks. That's, that's what I'm feeling.

    9. EM

      So we have to purposely add friction to our lives in certain key areas so we maintain an advantage.

    10. MM

      So you will never delegate generating ideas to AI?

    11. EM

      I wouldn't say never delegate, but you-

    12. MM

      Ethan, welcome. You teach entrepreneurship and innovation at Wharton. TIME named you one of

  2. 0:50 – 1:57

    How Do You Teach Ambition?

    1. MM

      100 Most Influential People in AI. About half a million people read your Substack, One Useful Thing, and your new book, Coexistence, just came out. Welcome to the show. When you teach your students, you also ask them to start companies while they're in class, and you tell them to work on the most ambitious ideas. How do you teach ambition?

    2. EM

      That is always a hard question, right? Ambition looks like, we call this entrepreneurial efficacy in the entrepreneurship world, right? Same kind of thing, agency. Um, and it's actually quite tough to sort of do. We don't have like a playbook for it. But I actually find that people create barriers that you can kind of help them overcome by getting them to... by just asking them to be more ambitious and being in a pe- room where peers are more ambitious. So, uh, a lot of what I just asked, it's really easy is, now is like, "What could you do more than that?"

    3. MM

      Hmm.

    4. EM

      And that was a hard question before AI because you stopped because you had the ability to get, only get so much done. And now, as the frontier moves out, pushing for ambitions is, is a much bigger deal.

    5. MM

      What are the coolest ideas that you've seen in your class?

    6. EM

      So it's funny 'cause like the last version I taught was six months

  3. 1:57 – 3:31

    What Surgeons and Soldiers Built in His Wharton Class

    1. EM

      ago, and so the, already that was like the beginning of agentic, agentic works. My next semester starts in January. So I, I actually, when, when, uh, after ChatGPT came out, I had been following, you know, obviously, uh, models for a while. But after DaVinci, which, which came out right, a couple days before ChatGPT, and ChatGPT came out, and then a couple days later I had a class and I basically just dropped the entire class and said, "I'm gonna show you how to use ChatGPT." And by the end of the class, one of my students had actually stopped paying attention and had coded up his entire project for the semester during the class period. So-

    2. MM

      Hmm

    3. EM

      ... and showed me the results at the end. And so every year since then, the ambition level has gone up further and further. What was most interesting this time was all of these people who are subject matter experts. I've got surgeons and I have biomedical device people, and I've got finance people, and I have, uh, you know, uh, NGO and humanitarian people, soldiers, all in this class, and suddenly they were all building real stuff, right? Like, you know, and so there were really interesting ideas. How do we track our entire supply chain? 'Cause a lot of companies don't actually know their entire supply chain, and how do we figure out where's the financial weakness in that chain that we might have to watch out for? Really interesting set of ideas. Uh, there was a bunch of really interesting stuff around, um, like specific problems in medicine about how do you actually give information to patients in ways they understand as opposed to just forms. So I mean, just really interesting, but very narrow, right? Focused on something people know. And part of what is interesting is, like, software can be many things, and if you're trying to think about solving a problem that you have deep experience in, suddenly there's a lot of capabilities there that, that are really exciting.

    4. MM

      Are there any problems that you're telling them to avoid solving?

    5. EM

      There- among those of us who teach entrepreneurship at, at some of the leading business

  4. 3:31 – 7:26

    Startup Ideas That Fail Every Year

    1. EM

      schools, we, uh, you know, we used to exchange notes about the topics we see people address over and over again that we know will never work out, but like, we're always conflicted. Should we tell them it's not gonna work out or let them try and figure it out for themselves? So there's a few that, that are sort of standard problems that are actually kind of a good example of where s- technology hits its limit. One of the most common is we are going to replace brokers for houses, right? It seems like a really bi- like you're paying a couple percent to these people who, like, are doing increasingly less work for you. This is obviously a case of disintermediation, and every time the idea fails because it turns out there's a deep social integration of political integration and legal integration of brokers into the process that you just can't dislodge. So sometimes the reality of the world is quite different than what you, people think. And then the other is, um, students thinking too narrowly about their student problems. So everyone always wants, you know, better ways to, like, get homemade food to them-

    2. MM

      Mm

    3. EM

      ... uh, or better ways to, you know, hold stuff w- between semesters that probably aren't as universal once you're out of, uh, out of college.

    4. MM

      This is so interesting. You're mentioning these ideas, and then I can draw parallels to businesses that actually worked, like DoorDash was solving the problem of delivering food on campus. And then when you talked about real estate, like Airbnb, right? We, we would've thought like, "Oh, who would have rent, um, a, an apartment from strangers?" And like, yes, there's social element of trust, but then Airbnb took over, and now we trust Airbnb. How do you know your idea is not gonna survive real world?

    5. EM

      Well, I don't know, right?

    6. MM

      You just try, right?

    7. EM

      That's why I don't tell people, right?

    8. MM

      Mm-hmm.

    9. EM

      Like these are the highest failure rate ideas, but obviously because they wanna do it, right, because they keep coming up, they're not bad ideas. The whole idea is you can't filter them out initially. Like, there should be a better way to buy and sell a house, right? And, you know, people do want home-cooked meals more easily than a meal delivery service, which is sort of what they're asking for now is like, "How do I have my mother cook a meal for me?" is essentially, or father, is essentially what they're asking. And so there should be a way to crack this. There's demand for it. But they're harder problems than people think. And in some ways, AI makes them more ambitious, right? 'Cause it's no longer the technical piece. But the numbers look really good, and then they just hit a point where it's like, "Oh, there is a challenge here." So whoever cracks it makes a lot of money, but it is also an idea that everyone is- we've seen people throw themselves against over and over again. So maybe some people who are watching or listening will be the ones to, to break the, uh, real estate broker experience.

    10. MM

      Yeah.

    11. EM

      But it's not as easy as it appears, I guess.

    12. MM

      Exactly. It's like when do you give up? 'Cause there are so many shiny things around you, and you can build things in a day, right? It's so hard to focus on a problem when you hit the wall. Do you agree with this phrase: It is as easy to work on a big problem as on small problems? Basically, the amount of effort that you put into building a small business versus large business is the same 'cause you're gonna have the same headache, hiring, firing people. Do you agree with that?

    13. EM

      In general Like, it, I mean, I think if you're trying to build a business from scratch, right, aim big because you may not get there anyway, right? And also ambition helps motivate people. It gets you the best people to work for you. It creates a real self-fulfilling prophecy, right? But I also think that it is... There are phases to a business, right? So that initial phase might be difficult no matter what. Scaling presents its own challenges versus running a small business, right? So they're, they're not quite equivalent, right?

    14. MM

      Stages.

    15. EM

      Like trying to get to the next level. You- if you're gonna raise a series B and a series C, that's very different than you're running, you know, a, uh, like a, you know, a local business of some sort, right? So I, I think that-

    16. MM

      Different skill set.

    17. EM

      Yeah, yeah. I mean, if you start from a blank slate, right, and you want to take over the world, go fully ambitious. And, but I also, by the way, I think that, you know, we keep talking about ideas. I think people aren't being ambitious enough about the kinds of ideas they pursue, partially because of there's sort of been a Silicon Valley mindset about software, right? And I, I think a lot about... I, I, I studied the history of technology, among other things, and done work on it. And if you look at who got re- like, really rich during the Industrial Revolution, right? It wasn't people selling steam engines, even though this keeps being the analogy in Silicon Valley as like, "I want to sell picks and shovels." You don't want to sell picks and shovels. You want to dig a gold mine with new gold mine technology.

    18. MM

      Mm.

    19. EM

      And I feel like we're kind of missing that point here. If, if you can make a business more efficient, just do it yourself right now, right? Overcome those barriers directly.

    20. MM

      What is gonna happen to software companies when you think about entrepreneurship? Is it really worth starting

  5. 7:26 – 13:24

    Is It Still Worth Starting a Software Company?

    1. MM

      a software company these days?

    2. EM

      So I think, you know, it's good news for entrepreneurship overall. Like, the early data looks really good, and you can understand why it's a really good idea. I think everyone's a little too, like, SaaS-brained at this point, right? Which is like, what do you do to make money? Software as a service.

    3. MM

      Mm-hmm.

    4. EM

      And I think that that is under threat, right? Like, it's a different sort of environment than it was before. I spend a lot of time with big companies now, as well as entrepreneurs, and there are people getting it. There, there's a lot of conflict over what the future of software looks like. On the other hand, I think software is increasingly becoming something that's table stakes for doing a lot of other interesting work. So I kind of would love to see more entrepreneurs branch out from software being the thing they sell to actually making or doing something, because there's never been a better time to do that.

    5. MM

      What do you think is the future of SaaS? 'Cause we're seeing all of these companies like Copy AI, right? That has started a few years ago, but now I think their revenue is going down significantly because of AI. 'Cause it's basically the new models are replacing what they were doing.

    6. EM

      So it's kind of interesting, right? You have a bunch of stuff happening at once. On one hand, everybody's a coder now. I mean, and that's not just a theoretical thing, right? So there's this great set of studies out of Anthropic suggesting that actually managers now write better code than coders, at least in first passes, right? 'Cause it's about giving clear instructions more than being great at code. Um, and things are transforming quite quickly, right? You've got 100 times more perfor- productive people. So companies are also completely in a mess as a result. But they all have their targets on, you know, SaaS vendors, right? There's not a large company out there that doesn't want to think about how do we become more independent, how do we build our own tools. And so it'll be a while before they get that all right. Uh, and a lot of them are still very happy. I mean, s- you know, as far as I know, you know, both OpenAI and Anthropic still use Slack. They, you know, from Salesforce-

    7. MM

      Mm-hmm

    8. EM

      ... they still probably use Workday. So, like, there'll be more choices being made about what kind of software vendor to go with. But I think the more interesting thing is that companies are realizing that a lot of these startups that were selling them stuff are just selling them the things they could build already, right?

    9. MM

      Yeah.

    10. EM

      They, they don't have their own LLMs, so they're selling LLM capabilities to them that they already have, and that's creating a really interesting dynamic.

    11. MM

      So what do you think is gonna happen in a couple years? 'Cause I already see myself. Like, for example, we're using this software, but because we're getting to the limit of the number of subscribers and the price is gonna raise, I'm like, "Can we just try and vibe code this and, like, connect Amazon services for, like, the newsletter stuff," right? I don't know. Like, what do you think is gonna happen in a couple of years?

    12. EM

      I think that software will be democratized, right? So it, like, the whole thing... I, I teach at Wharton. I teach MBA students, right? I teach undergrads. And a lot of the MBA students aren't technical. So, like, the key to success for, you know, the last 10 years has been find a technical person who can code for you. I think it's incredibly freeing to view this like other kinds of fields where you can get- approach it in different kinds of way, right? There'll be software-heavy companies where you really need top engineers working with AI to succeed, and then there's gonna be, like, software as a toolkit that we get things done. And I think democratizing that's incredibly helpful, right? I mean, you don't want to be a software company. You want to make your newsletter. You want to make your videos. And, you know, there were things standing in your way. And software, by the way, was one of them, right? It was also you needed to be able to do your own finance. I mean, when I did my first startup company, I did not realize... It was the late '90s, first internet bubble. Um, but I did not realize that, um, you could pay somebody a couple cents a pay period to handle taxes for your employees.

    13. MM

      Mm-hmm.

    14. EM

      So I would calculate taxes in Excel every week for the first few weeks, not realizing you could just do that. So even just advice, all of the kind of pieces you get, like, make this much easier now than it ever was.

    15. MM

      This part of the episode is sponsored by MongoDB. Companies want AI agents to do real work, but a lot of them get stuck right before launch. The agent forgets important details, and giving it access to all your company's data, like customer records and internal emails, feels risky. Nobody knows what the agent will see or who's responsible if it makes a mistake. MongoDB just launched Atlas Agent Engine to help with this. It's one place to build, deploy, and govern your agents. Before the agent does anything, the system checks if it's allowed to do it. Each agent runs in its own separate space. Rules set by the company can't be changed by an employee or by another agent, and every action is saved together with the name of the person or agent who allowed it. The agent also has memory. It remembers what it learned last time, so it doesn't start from zero and doesn't spend tokens rereading the same things. That means it gets more accurate over time and you pay less. And it doesn't need you to move your data. It can reach into Salesforce, Snowflake, S3, or your own internal systems and use the data where it already lives. You can use AI models from OpenAI, Anthropic, or Google and change them later without starting over. It runs on Atlas, the same platform that more than 75% of Fortune 100 companies already trust in 130 regions. The link is in the description. [swoosh] Using AI and the process of it, I really like the study that you mentioned, the, the Kenny entrepreneurs, where people who had experience in business, they used AI and it helped improve their strategy versus people who did not have experience. Can you talk about that, when everyone is using AI to strategize a lot?

    16. EM

      Yeah. So it was an interesting study. It was GPT-4, right, long obsolete system. There's now 180 models better than GPT-4, 19 of which can run on your phone. Um, so, you know-

    17. MM

      Mm

    18. EM

      ... big difference, and it was only giving advice. So the fact that I think the number was 18% profit increase for the best entrepreneurs, like, that's a crazy number. Like, we do studies all the time. We try and move the needle in entrepreneurship. That is a crazy number. But it only helped entrepreneurs who are already being successful because it helped them expand in the sort of way that you're talking about. Entrepreneurs who are already failing couldn't implement the advice. They were already going down the hole, and there was no way to kind of pull back. So it was-- So we know this stuff can be effective, and that was before coding, right? That was literally, "What should I do now?" And entrepreneurship, like, it turns out, by the way, about 50% of Americans have had a good idea of, or they think they have had a good idea for business in the last five years, and almost nobody pursues it because of all these barriers. So even just the advice to overcome those barriers is a real golden age for entrepreneurship compared to what we've had before.

    19. MM

      How do you know when to trust it? You mentioned newer models. Do you think the study will no longer apply because newer models are so much better,

  6. 13:24 – 16:10

    4 Skills You Need to Use AI Well

    1. MM

      or you still need to have this judgment?

    2. EM

      I talk about this in the new book, but I think th- there, there are sort of four things you need to use AI successfully, and that's deep knowledge, right? Some expertise in something. And that's, by the way, like, entrepreneurship is all about getting disciplined expertise, right? You go to an industry or you interview people. You know, you learn about the industry. You test ideas out. I mean, that is building expertise, and I think that's every bit as important as it was, right? Otherwise, you're just sharing Claude ideas, and Claude ideas are, look all very similar to each other, right? Second is wide knowledge. Increasingly important for entrepreneurship is knowing a little bit about the fields you're not in, right? So if you're in software, you should know a little bit design and a little bit about, uh, accounting because that helps you apply the AI more widely. And then agency, which we've been discussing a little bit here. And then finally, some sense of taste-

    3. MM

      Mm-hmm

    4. EM

      ... like selecting good ideas out out of the, the slop pile, uh, is also really helpful. So I think that what we're seeing with the more agentic systems is they can run longer periods of time, right? GPT-4 can do four minutes of work independently with 50% success rate. Now we're, uh, well over 17 hours of work. But you still need to be able to have some guidance in these systems. And even as they get really good, it turns out the human touch actually does matter 'cause variation is what makes entrepreneurs successful.

    5. MM

      Yeah, and that's the whole challenge, right? When everybody has Claude and Claude is advising or ChatGPT, whatever they're using, advising on the strategy, how do you avoid sounding just like everyone else?

    6. EM

      Yeah, I mean, it, it's kind of funny because you, you realize that people are more important than we thought, uh, or that some people thought, right? So everybody who was early on to using Claude is a good writer back, you know, when f- f- you know, Opus 4.5 or whatever. They write, wrote really well, and for a while nobody knew what Claude writing sounded like. And now if you l- open like LinkedIn or X, it's just like nauseating, right?

    7. MM

      Yeah.

    8. EM

      It's like what's carrying the weight?

    9. MM

      Oh, yes.

    10. EM

      What is it, right? It's just all the same stuff.

    11. MM

      Mm-hmm.

    12. EM

      And even not just the little phrases, like the way it approaches things, the kind of language it uses. And you h- like, well, they should-- If you didn't hire a marketer, you're probably making a mistake. Now, the marketer can do a lot more work than they could do before, but their human variation actually matters quite a bit. And I think we're seeing that in a lot of other cases, which is the AI pushes people towards a sort of a consensus strategy or view. We have some experiments where we found that we can make the ideas almost as diverse as a group of people. But if you map the ideas that, uh, that the AIs come up with and map the ideas that a group of humans come up with, the AI still doesn't cover the full range of human ideas-

    13. MM

      Mm

    14. EM

      ... even if you're pushing it really hard. So this variation piece is gonna become increasingly important. Like, what are you adding to the system to guide or steer it so you're not just Clauding about everything? And that might be okay. Like, it's okay if you're using Claude for finance and that's not your competitive advantage. But in the area where you are narrowly aiming, you need to stand out, you need a differentiator, you need a moat that has to be human to some extent, even as the models get incredibly capable.

    15. MM

      Yeah. You mentioned taste, and, uh, this is a question I ask a lot of my guests. How do you work on taste? What do you tell your students?

    16. EM

      We

  7. 16:10 – 18:37

    How to Develop Taste

    1. EM

      know a little about teaching people taste, right? Uh, like you need to see good and bad examples of things. So like- You want people to explain to you how to pull something apart. So I, I, one of the, uh, o- w- one of... I, I bring into a, a- an art professor actually, uh, in my classes now who talks about sort of thinking about, like, how do you, how does a designer see the world, right? So you get this kind of perspective, like what makes a font good or bad? What makes an image interesting or not? And so I think we need language to discuss things for taste, right? If you talk to somebody who really understands movies, they have a taste for movies that's individual, but they can probably talk widely about that. In the same way, like venture capitalism, like venture capitalists are all about taste, right? They develop a sense of taste for a particular kind of problem space or not. So I, I think that seeing a lot of variation is really important. Learning to be disciplined about this is another example of why I think education has some value here. Um, being exposed to lots of options, I mean, those help give you a sense of taste that you need.

    2. MM

      How is it different from judgment? 'Cause AI is doing a lot of judgment calls. Like, you present the data, and analyzes it, tells you, like, "This is a better decision." How is it different when you approach it from the taste perspective?

    3. EM

      I don't like the idea of judgment per se, because AI's obviously doing judgment. You can't do a 17-hour task without the AI making judgment calls. Just like we, we don't have creativity in the list 'cause the AI's obviously making creative decisions. The question is whether they match the kind of judgment and creative decisions you would make.

    4. MM

      Mm-hmm.

    5. EM

      Right? So we can call that under taste or knowledge, but it's, it's a v- kind of a different question, right? So judgment is this hard thing we don't really know how to teach, and we don't really have a good example of it, but some people... You know, what context is that judgment happening in becomes the question.

    6. MM

      Yeah.

    7. EM

      So it's, it's a hard, it's a hard thing to ask, right? Is like what is judgment? And so the AI's exercising some kind... There's obviously a human judgment element that's different, but we haven't had to build the language around differentiating these things. I, I kinda think about h- what's happening in math right now, right, which is, um, this sort of explosion of proofs generated by AI, and one of the things mathematicians will tell you when you talk to them is a bunch of revelations about their field or maybe clarifications about their field that outsiders didn't know, right? There's an aesthetic element to math that actually matters, right, to, to a mathematician, that if you, if the proof doesn't help teach you something beautiful or interesting, there's no value in it for the field of math. Even if we've solved this problem, who cares? And like, so we're starting to differentiate things that we didn't have to differentiate before because-

    8. MM

      Yeah

    9. EM

      ... the o- only kind of judgment and intelligence we have is human, so we didn't have to come with a different category.

    10. MM

      And it's so interesting when I think about elegance and taste and judgment. It feels like people who have experience in the field are better

  8. 18:37 – 20:42

    Why Junior Jobs Stopped Training People

    1. MM

      positioned to learn that versus someone who's just starting out. Instead of getting time to learn basic things, they need to go straight into judgment, straight into taste. Is that how the market is progressing with AI?

    2. EM

      There's a couple things happening at one time, sort of when you think about markets and education, right? So first of all, we are temporarily in a, uh, crisis in universities. I mean, we're always in a crisis in universities, so this is not new.

    3. MM

      Mm.

    4. EM

      Uh, anyone who's been in education knows this. But we are in a temporary AI crisis, right? And we already kind of know the outlines of that solution, which is AI's an incredibly good tutor when used as a tutor, and it's very bad if you're just cheating and getting answers from it. Even if you think you're learning, you're not, right? So eventually we'll have AI tutors outside of class. Inside of class we'll do more action learning. I mean, if you've been to a business school, law school, you've already done these kind of case studies and discussions and activities, and, you know, we'll do in-class tests. We, we'll figure it out. Um, the real problem is where people actually have de- have developed taste and judgment is in the world, right? And I... We had this great deal for the last few thousand years, which I would send you, you know, a newly trained recruit who didn't know anything, um, and we had apprenticeship, right? That's how the last 4,000 years of white collar training have operated. So you'd get a really junior person who was desperate to show you they were good. They'd work for cheap because they wanted to move up the, the realm, the, the rungs, and they would do the work that a middle manager didn't wanna do.

    5. MM

      Mm-hmm.

    6. EM

      And in return, they would get a little bit of pay, they'd get to show off their stuff, and they'd learn the ropes. And the middle manager would get their work passed off to somebody who could at least do some of the work for them, uh, and make their lives easier, right? And that's how they moved up the management chain, and that all broke, right? Because every... You know, your incentives as an intern or as a first-year associate or whatever you are, your incentives are 100% deliver the best product you can. Who's better than you? Claude or ChatGPT or Gemini. So you will deliver Claude or ChatGPT or Gemini answers. And this is not like being digitally native because you don't know what you're delivering. You're just handing stuff over. And every middle manager has realized, "Why would I give feedback to a human who takes a lot of time to do the work? I could just have Claude do it myself." So we've just broken the entire system that forms professional taste and judgment, and we haven't figured out a full replacement yet.

    7. MM

      What are your thoughts? What are you telling your students?

    8. EM

      So I think that there are a couple things. One is at the individual level, we now have

  9. 20:42 – 22:12

    "Cognitive Surrender": What to Keep Doing Yourself

    1. EM

      to start being really disciplined about when we're going through cognitive surrender, when we're giving stuff up to AI, and that's really hard to do. We're built... Like, first of all, entrepreneurship, uh, like, which is why I teach, right, and what we've been talking about here, like, that's about trying to make your life as easy as possible so you can focus on the hard stuff, right? Um, and you know, but then you learn nothing, right-

    2. MM

      Mm

    3. EM

      ... as a result of that. But it's also true in every other aspect of life. Like, we can easily give up skills to the AI, and I think we all feel tempted every day to do this, and we may never get those skills back. So we have to purposely add friction to our lives in certain key areas so we maintain an advantage, but we're not incentivized to do so. So at the individual level it's gonna be making choices. At the organizational level- Companies are gonna do one of two things. Um, they're gonna eith- either follow the easy path or hard path. And I worry about the easy path, which is only hire senior people, and we're seeing some signs of that. I know you've had some economists on talk about this. There's some early signals that junior jobs might be threatened. We're not 100% sure yet. Um, as you, um, uh... You know, or the other option, the harder option, is that we actually make training a serious part of jobs, right? So you actually have to give people time to be inefficient and learn on the job. And we, rather than learning be sort of an accidental thing, we'll have to teach them. Maybe you have to actually add an extra year to universities that we teach you to be a, you know, a Google person or a McKinsey person or, you know, whatever it is, and there's a year of customized training for you to teach you because we know how to teach and assess people in classrooms that we don't know how to do in the real world anymore.

    4. MM

      This is such a tough problem, and then I'm thinking about a professional who even completed, uh, that step, and they

  10. 22:12 – 23:43

    What Happens to "Good Enough" Professionals

    1. MM

      graduate being good enough, but not exceptional, and the models keep improving. What is happening to these good enough people?

    2. EM

      So everything is too early to tell anything yet, right?

    3. MM

      I know, but do you ever... Like, when I think about, like, where are we headed? [laughs]

    4. EM

      Yeah, I mean, I... But also, how long is good enough, good enough, right? Like, the issue is-

    5. MM

      That's also progressing

    6. EM

      ... right? Like, I mean, that's progressing, too. So, you know, this concept that we developed in our first paper where we examined this, we called the jagged frontier, that AI is good at some stuff you wouldn't expect and bad at some stuff you wouldn't expect, and it's hard to know in advance what that is. As long as the frontier remains pretty jagged, um, there are always bottlenecks that humans kind of need to fill in, and those can be really weird ones. Like, if you think about, like, AI replaces a lawyer. Well, it turns out a lawyer does lots of things with friction in the world that the AI can't necessarily do, even if you had a robot doing the work, right? 'Cause there's human systems they interact with. There's legal requirements they interact with. Some people want to talk to a human. So that creates these bottlenecks that, that, that actually reward different talents than before. And you see this happen in coding, too, right? Like, what makes a good coder now is completely different than three years ago 'cause you're not writing code anymore. So I think one thing I would say is good enough is a bundle of skills, and it may very well be the people who are, like, mediocre coders, but really good at the vision about building software-

    7. MM

      Or communication

    8. EM

      ... right? Yeah, or communication. And, and those will change, too, over time, right?

    9. MM

      Yeah.

    10. EM

      I'm not gonna say the AI will never do that task. It probably will.

    11. MM

      Mm-hmm.

    12. EM

      So I think we're going to see ourselves ping-ponging across what skills are valuable in a really rapid way without an easy answer.

    13. MM

      Yeah. And I really like the jagged frontier study, and I was getting thoughts, maybe it's us who don't understand [laughs] the models good enough-

  11. 23:43 – 27:44

    Why Experts Get the Most Out of AI

    1. EM

      It is

    2. MM

      ... because sometimes you re-prompt, and you get the result that you wanted.

    3. EM

      Well, my fa- one of my favorite phenomena is watching people who know nothing about a field getting the AI to do, like, impossible things, and you're like, "Wait, how'd you do that?" You're like, "I just asked." You're like, "You can't ask the AI for that," and they did, and it, like, works. And you're like, "Oh, great," you know?

    4. MM

      Mm-hmm.

    5. EM

      "What are we doing here?" So there, I mean, there is this sort of weird phenomenon. There... I mean, there's two things that we know make you better at AI. It used to be AI was a level- mostly a leveler, right? If you're a bad performer, you got a boost. That still is that way, but what we're increasingly seeing- and there's some nice results of Anthropic as well as some other studies that suggest that the people who are, uh, mo- have the most expertise get the most gains from AI at this stage.

    6. MM

      Mm-hmm.

    7. EM

      And that wasn't true until agents kind of came around in the last year or so. Um, but the other thing we know is that there's some AI whispering ability that is like knowing how the AI thinks that might be sort of theory of mind. Like, if I know people really well and I get to know the mind of Claude or ChatGPT or Gemini or whatever I'm using, I become more effective in working with it. Um, and so those are sort of two different skill sets. And then there's, like, a mysterious third category, which is like some people seem to be good at this in ways we don't know. There's this great study by Alex Imas and a few others. He's now at, um, he was at Chicago. Now, he's at DeepMind. Um, but they measured people creating agents for negotiation, and there are a couple interesting things. One thing they found is your skill multiplier becomes much higher because the agent engages in many negotiations, so a small difference in skills in creating good agents has a huge impact on bottom line.

    8. MM

      Mm-hmm.

    9. EM

      The second thing they found was that the people who are good at creating agents are not necessarily people who would be the best negotiators in real life. I mean, even, there was even a gender divide. So one of the, uh, things that we find repeatedly in studies, um, for a wide variety of reasons is that, um, women tend to outperform in negotịa- underperform in negotiation, and there's been a lot of work on this. It's been a big topic of concern, but women made much better negotiation agents than men did.

    10. MM

      Ah.

    11. EM

      And, you know, and again, we don't actually know... Like, I don't even know how to teach you how to do that necessarily at this stage.

    12. MM

      Is that a management skill?

    13. EM

      I, I mean, there is a management skill. There's a theory of mind skill. There's an expertise piece.

    14. MM

      Interesting.

    15. EM

      But, like, w- things are happening so quickly that-

    16. MM

      Mm-hmm

    17. EM

      ... you know, we, like, again, we can't pull them apart that easily.

    18. MM

      This is... And I, I just had Felix from Anthropic in the studio, and he was talking about how it's very important to try and understand your workflow in... like, what's going on in your head when you create a perfect text, or when you go through a perfect negotiation, like, what was your thinking like? So you can translate that onto AI, and this is how you build a workflow.

    19. EM

      I feel like that's kind of made up. I mean, it might be true. To me, the analogy that makes the most sense as somebody who's run organizations and managed people is, like, thinking of AI like a team I'm delegating work to.

    20. MM

      Mm-hmm.

    21. EM

      And if I can do a good job delegating to a team and realizing where a team goes wrong, which is very different than knowing my own process, right? The AI will not replicate my process, and part of what I try and do is, like, preserve stuff for me that way also, right? Like, I write all... You know, I m- it might seem crazy since I do Substack and X every... I mean, I write all my own stuff, right? I write first drafts of everything-

    22. MM

      Right

    23. EM

      ... on my own. I mean, yeah, I type it all out, right? Because if I don't do that, um, the AI is very seductive and will absolutely, like, dominate what I'm thinking in any direction, and then you're just-

    24. MM

      Uh-huh

    25. EM

      ... gonna read, like, Claude results filtered through me. And we know-

    26. MM

      Do you, like, do you use voice at all or?

    27. EM

      I, I use voice sometimes for this. Um, I find, like, t- just typing, uh, on a computer is easier, um-

    28. MM

      Wow

    29. EM

      ... for a lot of this stuff.

    30. MM

      Mm-hmm.

  12. 27:44 – 33:23

    Should You Let AI Generate Your Ideas?

    1. EM

      we actually have old evidence on this. Brainstorming was invented, I think, 1955, and by 1957 we knew it was a bad idea. Because basically, anytime you do a group ideation process, your, your ideas become fixated on whoever says the first idea out the gate, and then there's a judgment process.

    2. MM

      Mm-hmm.

    3. EM

      And so anchor on other people's ideas. So you lose natural variation. So the way you should do brainstorming before any group process is write- everyone writes down ideas individually, and then only start generating ideas as a group once you've done individual idea generation, and you end up with much better ideas with higher diversity, right? Same thing with AI. You should absolutely use AI for idea generation, but only after you've generated ideas for yourself.

    4. MM

      So here's what happens after a single podcast. When I learn something new about AI, I go ahead and change some processes in my company, see if it works, see if the models are really powerful enough to handle that thing that I just delegated, and then I write about it. I don't type my emails like Ethan does. I actually talk to my AI and then edit them, and I also have a person helping me with that. But it's basically a summary of what we tried with AI, what worked and what didn't, and of course, a lot of strategic stuff, how I navigate this media career and building businesses as a mom, as a woman, as an immigrant. So if you want to get a closer look at my process, subscribe to my free newsletter, Future Proof, and let's grow together. And you talked about this effortful thinking and having AI as a companion. Can you describe that process?

    5. EM

      Yeah, I mean, I think that what you wanna do is you- you're kind of trying to mentally sweat while doing this. Like, if you're not kind of p- if it's not uncomfortable, you're probably not doing enough learning or thinking. Like, it works just like exercise or anything else.

    6. MM

      Mm-hmm.

    7. EM

      Right? And so in learning, we call this desirable difficulties. Like, if you're not encountering difficulties, you're not learning. And so there's certain mental processes like ideation that can't be streamlined that much, right? And it's very tempting to do that, and we don't recogn- we learn to recognize the cost of, like, we never walk around or, you know, we never, you know, we never exercise. We haven't learned the same kind of thing with our brains 'cause there was no alternative, right? You know, you could Google something, but that was not the same kind of thing as AI do my thinking for me. So I think that we have to be recognizing that we're doing that kind of effortful work.

    8. MM

      So interesting 'cause I got to thinking that maybe, like, the way I get ideas could be a workflow. Like, maybe there is something that's happening in my life that leads me towards a certain idea, and maybe AI can help me pinpoint.

    9. EM

      It can. I mean, I just think you have to be care- and by the way, there is a, uh, in ideation, there is something called the equal odds rule that turns out to be true, which is that some people, uh, generate both more ideas, and those people who generate more ideas generate better ideas than most other people, and it's not necessarily deeply correlated with intelligence. There's, uh, creativity is a trait. Some people are just more creative. So if you're less creative, I think we don't need to hold creativity on a pedestal, right? Then maybe AI is helping you out more, right, generating ideas.

    10. MM

      Yeah.

    11. EM

      If it's an area that you have competitive edge, kind of going back to where we were before or writing or whatever it is or podcasting, right? We can generate an AI podcast. It will not be as good as an expert having done this a lot and thinking through this. So at least for now, that human edge, right, is, is a real one, and that's where you wanna make sure you're not really giving stuff up, right? I worry less. I mean, ideation is just one of many traits. Like, most people don't have good ideas. That is okay. We lionize ideas too much, right? Execution, uh, you know, emotional connection. There's lots of other great things people could do, and so you wanna be strategic about when you're using these systems.

    12. MM

      I really like what you said about people who generate more ideas tend to generate more good ideas, and I think we just need to be learning how to constantly generate things and, like, maybe write them down or test them out 'cause it's so easy to test out ideas.

    13. EM

      Y- yeah, and I think that that, um, ideas that don't do anything aren't that useful. The whole discipline of entrepreneurship l- before AI, right, was built around this idea, especially Silicon Valley entrepreneurship and the lean startup and all of the other dominant methods, strategies that we teach people to become good entrepreneurs. All of those strategies come from this idea of learning at the cheapest cost you can, right? So it was all about, like, don't just come with a beautiful idea in the building, right? As the cost of software development drops, w- the goal was get out there and talk to people. Show them your product, right? Do customer interviews. Create minimum viable products and learn quickly, and then all the VC sort of reorient around this too, right? It was originally built around giving you a giant check to scale your idea, and it became how do we give you seed rounds and pre-seed rounds and Y Combinator. All of, basically the entire evolution of entrepreneurship was based around the idea that it got cheaper and cheaper to test ideas over time. And so now we've really hit rock-bottom cheapness, right? Like, you can, you can, uh, don't trade... The AI is not a customer when you simulate customers, but it's a pretty good first pass-

    14. MM

      Mm-hmm

    15. EM

      ... at being a simulated customer. There's now no longer s- an excuse to not at least push your ideas a little bit.

    16. MM

      Yeah. Do you think a good idea is worth more now versus five years ago?

    17. EM

      I think a good idea is worth more when combined with execution, right? I think that there is, that you, that an idea alone has never been worth that much. I mean, it's great, but, you know, one of the conflicts that typically happens is in entrepreneurship classes all everywhere is the person who came up with an idea wants all the credit for the idea. And one of the things you see as a, you know, as a sort of a semester goes on and then a lot of the startups out of Wharton turn into real businesses and raise lots of money and, and end up being very successful. But you kind of see the process of like, oh, everyone trace- the person who came up with the idea is like, that was the amazing thing. And then by the time we're, you know, six months in, the idea has changed. It's rarely the idea that has made everything successful, right? It's, it's a set of explorations-

    18. MM

      It's the execution

    19. EM

      ... and everything else. So I think ideas have always been sort of a dime a dozen, and the AI can generate those ideas now better than anyone else. But if you actually execute on them, there's real value there.

    20. MM

      I agree 100%. And when executing ideas, you become a manager either of people or agents, and there are so many

  13. 33:23 – 35:14

    How to Manage AI Agents

    1. MM

      people who are now, not forced, but they have to manage agents because of their work. What are the tips that you will give them, the management tips that they should be deploying?

    2. EM

      It, it's the revenge of the MBAs, right? Like, I teach at-

    3. MM

      [laughs]

    4. EM

      ... uh, uh, you know, I, I, I, I, uh, uh, you know, I teach at a business school, so it's, like, very, it's gratifying to be like, "Ha, ha, everyone has to be a manager now."

    5. MM

      Right. [laughs]

    6. EM

      All, all, all my business people were looking for technical people.

    7. MM

      [laughs]

    8. EM

      Now all the technical people need management skills. But actually, those management skills do apply, right? So what you need to... It turns out, I mean, uh, we've been doing a lot of experiments with that at the Generative AI Lab at Wharton, which is what I, I lead, and a lot of the experiments basically show, you know, look, techniques that work on people work on AI. And so I think what you need to be thinking about is how do you manage a process because one of the dangers of an AI, of, of, uh, the cost of an AI system is have assigning a lot of work, not even the token cost, just the effort cost of monitoring work is quite high. So we built all these systems for doing that, right? So if I'm specifying things for an agent, I will specify things like here are the points I want you to check in with me on-

    9. MM

      Mm-hmm

    10. EM

      ... about the work. Here's where I want you to give me diverse examples that I can select from so I make a choice, right? I'm deciding when to insert my own agency into the system rather than just sending off agents to do work. I am making sure to specify what good and bad looks like in really kind of clear ways. And I'm watching really early on in the process to see whether or not there's a failure in my ability to communicate to the agent, right?

    11. MM

      Mm-hmm.

    12. EM

      Oftentimes they'll head off in a direction that sort of so- like, you know, Claude has its own obsessions, right? They'll head off in those directions unless you intervene quite early on-

    13. MM

      Mm

    14. EM

      ... and often. So, you know, those management skills about thinking about am I clearly specifying what I need? Am I g- you know, am I establishing a feedback loop, right? Am I dealing with, you know, reducing uncertainty over the course of a project? Those things end up really applying here.

    15. MM

      Interesting. What, what is the biggest mistake that you think people are making when they're managing those agents?

    16. EM

      I think they're trusting the agents

  14. 35:14 – 36:03

    The Biggest Mistake: Trusting Agents Too Much

    1. EM

      too much.

    2. MM

      Mm-hmm.

    3. EM

      W- You'll get an amazing result. Like I, I... everyone loves sort of s- I, I do a lot of single prompt, like this is awesome kind of stuff. But the guidance process is incredibly important, and I think that it can be very disempowering, um, both because the agent knows things you don't, but also because the interfaces we have to work with AI now are increasingly hostile to augmentation and embracing automation, right? They hide the chain of thought, right? And, and even the chain of thought th- summaries no longer appear. It, it's supposed to feel more like magic, and that's disastrous for being a human in the loop. I don't think we can be in the loop anymore. Managers are not really in the loop. They assign work and they have people check in on them. So we have to now create forced positions for the AI to report back to us, otherwise they will try and do magic for you, and you don't want that.

    4. MM

      Can you describe a process in your own workflow where you can show us how you create

  15. 36:03 – 39:47

    Ethan's Real Agent Workflow, Step by Step

    1. MM

      this result and how we create the feedback loop so that people can learn how to do that?

    2. EM

      Sure. Like I was exploring a sort of random topic that interests me in the humanities, which is... and I've done this for entrepreneurial pitching as well, which is there's a lot of research that stories have shapes, right? And so, um, there i- and so, like there's rise and fall stories-

    3. MM

      Mm

    4. EM

      ... and, you know, and like the Cinderella stories, and this is like a long-standing question. And I actually did some research, um, before L- with, with primitive LMS, uh, about whether or not we could take entrepreneurial pitches and find the shape of a good pitch. So it's a topic that's really interested me for a long time, and as agents have gotten better, one of the things I've had them do is assign them to explore this topic, right? And so there's a few things there. One of them is, um, you know, I was assigning them to kind of generate a bunch of ideas on how to do this, and then I... but I didn't have it then execute its favorite idea, 'cause that would've been bad. So I had to give it feedback partway through. So give me a bunch of ways that you might approach this, figuring out what the shape of story research is real. Yeah, and then I, you know, it's like, "Now speculate for me," just like I would for a student. "Give me some thoughts about how you might approach this," and give it feedback again. "Great, why don't you execute one, two, three, and four, a first draft of all of them, show me the results, and then let's pick which example works better from this. Great, I really like what you're doing with three here, but number two has this really nice feature. Let's talk about how we combine those together."

    5. MM

      Mm-hmm.

    6. EM

      Right? "Why d- You know, let's... You know, can you show me where the t- you know, this looks wrong. Can you show me why you think this went wrong?" So it's almost like a Socratic method kind of thing of me guiding the AI and giving myself lots of choice points where the really tedious, annoying work, get... you know, figure out where 5,000 story of 5,000 stories are, build the semantic coding tools, I let the AI do, so I get the interesting work for myself. And like that's... if there's one message of sort of the latest book and where I'm most desperate to try and communicate with corporate America, it's like we have an automation world that will be bad for everybody in a lot of ways. But an augmentation world is like the dream of work, right? You do the interesting stuff and the AI does the crap stuff, and you get to just direct an organization and it's really fun.

    7. MM

      Before we keep going, I'd love to get to know you a little better. I share a lot about my work here, and I'm curious about yours. What do you do and what you'd like more help with? We've put together a little survey. Your answers will help me decide what to cover in future conversations and what resources to put together for you. I'll leave the link in the description. Please, please, please take a couple minutes and tell me about you, about your business, about your ambition, so I can make better content for you. I really, really appreciate you. Thank you so much for doing this. The problem with this that I see is that the models progress so fast, so you build a workflow, you describe things, and then, I don't know, the new model comes out and they tell you it's better to not give it a workflow because it's so good at understanding what data to use for a particular workflow, and sometimes it thinks about things you didn't think about and it can make a better workflow. How often do you have to rewrite the whole thing?

    8. EM

      So that's not a workflow I was giving you, right? That wasn't step one, I do this, and step two, I do this. This was the way I would manage a researcher team-

    9. MM

      Mm

    10. EM

      ... or, you know, or a startup team.

    11. MM

      So it's a one-off thing where-

    12. EM

      Everything, everything's an interaction where-

    13. MM

      Mm-hmm

    14. EM

      ... I'm learning from what the system does. I don't know in advance what's gonna be good or bad. If you're trying to launch a startup company, have a deeper conversation, reach some sort of conclusion, come with a marketing plan, you know, do financial analysis, all of these involve lots of choices. You don't want the AI making those choices, even if they're good ones.

    15. MM

      No.

    16. EM

      You wanna be in, you know, inserting into that process. So it's sort of a meta process, right? The idea is like assume you have a really smart person or a really incredible team doing work, but you still want it to be your work. When should you talk to the AI about this, right? You still want that visibility into the system, otherwise you're just... it's just Claude. Even if it's brighter than you, you're inserting nothing into the process anymore.

    17. MM

      I agree.

    18. EM

      And so anyone could type the same sentence and get the same result.

    19. MM

      What are the skills, I know you wrote about it, what are the skills that you're just delegating to AI and you c- you said you can't wait to

  16. 39:47 – 48:32

    What He Can't Wait to Hand Off to AI

    1. MM

      delegate a, a few skills.

    2. EM

      Basically everything having to do with, um, so fact-checking is great. The-

    3. MM

      Mm.

    4. EM

      The, now-

    5. MM

      And it's really good at it now.

    6. EM

      It's really good. Actually, it's kind of, um, it's creating a kind of crisis at how good it is. I, so with my latest book, when I delegated to the, uh, to the AI, um, to, to fact-check it, right? And I had human fact-checkers. It's, it's a big release, right? But not only did the AI spot errors that I made, but even more often it spotted errors in the stuff I was citing. So it's like table two in this really complicated academic m- you know, paper has an error. And I'm like, "What do I do with this?" Like, do I e- like wh- you know, we have a whole method of corre- like correction in science that doesn't make any sense anymore because it's sp- it, it found an error in an AP story I was citing that it was, it cited the wrong city an event took place in. Like, it's like meta fact correction, right? So like superhuman fact-checking, and I've been fact-checked by The New Yorker and other thi- like this was pretty amazing. Um, and then there's just like a lot of like busy work, right-

    7. MM

      Mm-hmm

    8. EM

      ... that the AI does really well. Like, uh, never fill out another student form for your child if you can avoid it, right?

    9. MM

      Mm.

    10. EM

      Like i- immunization forms it does really well.

    11. MM

      Yeah. [laughs]

    12. EM

      Um, so, you know, um, and, you know, then there's the tempting line of like, "Oh, should I have it do all my mandatory training for me?" And the answer is like, "Not yet," I think. But, um-

    13. MM

      Mm

    14. EM

      ... so I think that there, when I talk about de-skilling myself, like I cannot wait to de-skill myself on a whole bunch of useless things I had to be good at doing. Filling out forms, you know, particular kinds of legal documents, signing NDAs. Like I wanted to be able to handle those things 'cause I think it's well within the range of what the AI can do.

    15. MM

      That worries me a little because, uh, I had this thing, uh, a couple days ago. I was on an airplane, so the Wi-Fi was really shitty, and I had to fill in the form. So I asked my dot to fill it in for me, but I just said, "Fill it in." But what it did, it sent it.

    16. EM

      Yes.

    17. MM

      And I only realized in the morning, and I'm like, "Oh my God." And my team was like texting me, "This doesn't sound like you." And I'm like, "Oops"

    18. EM

      Yes

    19. MM

      But then I realized how convenient that was. Like, because it completely took it off my to-do list.

    20. EM

      Yes

    21. MM

      And it's... Kind of scares me, 'cause Dot drafts all the replies, and I just correct maybe 10% of it, but it's so good, and I'm like, "What- what's, what's gonna happen?"

    22. EM

      So I- I had to set a line where I have a Dot and I also have, uh, Codex scheduled tasks going a couple times a day, summarizing stuff and, and giving me what's happening in my world. And, um, and I won't send drafts to anyone who I... Like, I have-- I write stuff to myself- to people who I have any kind of connection with or I feel bad doing it, right?

    23. MM

      Mm-hmm.

    24. EM

      So that's another line I'm trying to do. It drafts a lot of nice rejections for people who aren't students yet, questions about my class. But I, I mean, I had the same thing where the AI emailed my dean about something, you know, when I was like, "Don't do this." And so, but this is part of why also the sort of YOLO no tr- you know, trusting everything. Like, you do have to build a, a set of trust with this, just like with people. And like once you build that, you're like, "Okay, I know why it fired off that message," right?

    25. MM

      Mm-hmm.

    26. EM

      Dot is a little too proactive. I mean, again, though, hiding stuff, right? Like, when I have a scheduled task running in Codex, I have a system prompt, right? And I have... I understand the context it's in, and I've been tweaking that, that prompt for it over time to make it good and, you know, uh, and using it for low stakes before higher stakes. And Dot just encourages you, like, "Oh yeah, just ask it questions." But we don't know its system prompts.

    27. MM

      Exactly.

    28. EM

      We don't know. There's, there's no place to change the instructions about how proactive it should be.

    29. MM

      You-

    30. EM

      And that's a little more ominous

  17. 48:32 – 49:55

    The Mindset That Helps When Everything Keeps Changing

    1. MM

      What is the mindset hack that they should be adopting to just go through it and thrive with it?

    2. EM

      I think you have to have a little bit of excitement and curiosity. I think there's a mindset of like, I think we all are nervous and a little resentful and like no... And I'm a very, like, enthusiastic person who uses AI for all sorts of stuff, and like, there is no doubt that that is real, right?

    3. MM

      Mm-hmm.

    4. EM

      And I think it's okay to acknowledge that.

    5. MM

      Mm.

    6. EM

      It's also okay to acknowledge that you have imposter syndrome 'cause we all do. Like, I'm as up to speed as anyone. You are, too. Everyone's making this up. You had Anthropic and OpenAI people in this chair. They're making it up, right? We're all figuring this out as we go along. So I think it's okay to feel nervous. Like leaving aside existential anxiety, I think it's okay to feel nervous. It's okay to feel a little resentful. I think that there's also a positive side, and I think that's kind of how I view the... Like, but suddenly you're able to do so much more.

    7. MM

      Mm.

    8. EM

      So part of what I do is focus on the more you can do, and with a sense of play and experimentation, and that can help you get into better mindset. I... and just acknowledge the resentment's real. Like, it is annoying. And like the labs are not doing us any favors. Like, they have an unreasonably good technology, and then they just keep doing new stuff with it, and sometimes it works, sometimes it doesn't. Sometimes they'll just cancel a whole thing that you ca- depended on for a while. Like, when you interview professors, like half of them want to quit, and the other half think it's the most exciting time they've ever worked. And I think you see the same thing with executives, even entrepreneurs, right? There's like half that's like, "I didn't sign up for this."

    9. MM

      Yeah.

    10. EM

      Right? Like, "I didn't want to do it this way."

    11. MM

      But it is what it is. Uh, I had a question about local models. When do you think they're applicable for

  18. 49:55 – 51:08

    Local Models: When Will They Matter?

    1. MM

      individual users? 'Cause I tried a few of them, and I don't think I'm ready to delegate stuff to local models 'cause they're a few steps behind the frontier. But at the same time, with this all progressing so fast, we're gonna have, you know, Argon or Astra on our phones in a few years. What happens to an average user? Do you think we're gonna use the local models more, and what happens to those large companies? What do you think is gonna happen in like, say, five years? 'Cause I think there's a research that says by 2030, 50% of enterprise tasks are gonna be l- run locally. Do you think-

    2. EM

      I think that's a made-up number that has-

    3. MM

      Yeah

    4. EM

      ... no basis in reality.

    5. MM

      [laughs]

    6. EM

      I mean, I, I mean, first of all, when I talk to enterprises, right, a lot of them don't wanna run local inference.

    7. MM

      Mm.

    8. EM

      And they certainly don't want you having a local, like, they don't want every machine running a model, right?

    9. MM

      Yeah.

    10. EM

      Like, they, they've been using SaaS vendors for years. It's not that weird. You know, the token costs are an issue, and they're thinking about that. But like, I think a lot of people... So this is a meta point about AI, which is people keep fixating on the problems of today and assuming they'll last forever.

    11. MM

      Exactly.

    12. EM

      You know, all the legal stuff that we had to do to be like is it, is it, you know... Can this do NDAs? We need specialized legal AIs. Then nope, it's just better models. So I think it's, it's a little easy to get fixated and not ride the exponential.

    13. MM

      Is there an overarching thing that we should be thinking about when thinking about the progress with AI?

    14. EM

      I think continuing being more ambitious

  19. 51:08 – 52:05

    Set Goals AI Can't Hit Yet

    1. EM

      and trying to aim to have at least a couple things that don't work yet, unsaturated personal benchmarks that you're kind of aiming for-

    2. MM

      What are those for you?

    3. EM

      Or business ideas. So I can still not get a high-taste academic paper from scratch out of these systems.

    4. MM

      Mm.

    5. EM

      So I can get academic papers that are true and correctly written, but they're not interesting, and they're not telling us anything fundamentally new. So that's a big one for me as an academic who studies this stuff. I mean, you can't vibe code a true startup from scratch at this stage. Like, you can't take an idea and say execute. I mean, that's a big gap, right, between that and reality. You know, I do a lot of work in game development. That's been a topic I wrote about-

    6. MM

      Mm. Yeah

    7. EM

      ... I've done stuff in, and we're getting closer, but like the game development from scratch, there's still lots of interactions. Writing a good story that I actually wanna read, right, that's coherent. So there's a lot of them out there when you're kind of... I mean, all of these would have seemed ridiculous as, as even possibilities, you know, two years ago, but we're close.

    8. MM

      That's exactly what can be either fascinating and frightening because o- the overarching thing

  20. 52:05 – 53:40

    What Happens When AI Learns Taste?

    1. MM

      of, you know, a good game, good research is taste again, right? What if in two years AI learns taste? I know we talked about not projecting the problems in too far into the future, but what do you think the future gonna look like when AI learns the taste?

    2. EM

      I still th- so I mean, we can p- speculate at any point that AI just does all human stuff better, right? And at that point, I don't have anything useful to say, right, in some ways 'cause it's like it just does stuff. I think that the jaggedness is underestimated continuously.

    3. MM

      Mm.

    4. EM

      And I think the nature of what taste is gets underestimated as well, which is that when we talk about taste, right, we mean your taste. And that universal like, like there's still going to be some similar-ness to the AI systems in the, for the foreseeable future-

    5. MM

      Mm-hmm

    6. EM

      ... that will become tiring for most people under most circumstances. Claude Ideas, you know, Claude Websites, Cla- like, and you can get it to do incredible stuff with your taste added in. So there still remains jaggedness at the outside edges. When that goes away, I mean, what do we do? We shrug and say, "Then we're," you know, w- I, I don't have an answer at that stage.

    7. MM

      I don't know. Buy, buy stocks of these corporations-

    8. EM

      [laughs] Right

    9. MM

      ... that are about to IPO. [laughs] I guess that's a pillar, safety pillar. Thank you so much, Ethan. That was very interesting, very entertaining, and thank you for sharing your workflows. And please read Ethan's new book, and I really, really like your Substack.

    10. EM

      Thank you so much, and this was a real pleasure.

    11. MM

      Yeah.

    12. EM

      Awesome.

    13. MM

      If AI makes it easier to build things, how do you decide what's actually worth building? Watch my conversation, by the way, one of my most favorite conversations on this channel, with Brian Chesky. We talk about how he comes up with ideas and thinks about what people need. Click here to watch.

Episode duration: 53:40

In the uListen app, the transcript lights up word by word as you listen. Get uListen

Transcript of episode sHHYVvG_C4Q

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.

iOSAndroidClaudeChatGPTChrome