How to Stay Great When AI Is Good Enough | Matt Beane, UC Santa BarBara

1. 0:00 – 1:17

   Intro

   1. SPSpeaker0:00

      2025 was the year of try everything with AI, get results with AI, do your very best, experiment, learn, licenses for everyone. 2026 is the year that the board of directors is coming to the CTO and saying, "Where is our return on our investment?" Jensen Huang from NVIDIA is very famously on interviews saying, "If I'm paying five hundred thousand dollars for an engineer, they need to be burning at least two hundred and fifty thousand dollars worth of tokens." You can burn many, many, many tokens very inefficiently. All AI can do most of the time is create B+ content. It will just... lots of free B+ content, and you will forget what an A+ looks like. A wise leader in an organization will reward people with cash or promotion or visible recognition for stopping a B+ idea. I'm Matt Beane. I'm an Associate Professor at UC Santa Barbara in the Technology Management Department and also CEO and Co-founder of Skillbench, a startup in the AI enablement space. [gentle music] [keyboard clacking]

2. 1:17 – 1:33

   Don't Let AI Dumb You Down

   1. SPSpeaker1:17

      I wrote an article about this called Don't Let AI Dumb You Down, and I put that on my Substack. That was two and a half years ago. This-- Immediately I saw this technology, I went, "This is a major risk." I can produce many, many... lots of lines of code. I can produce documents for free or cheap. Should

3. 1:33 – 2:31

   The B+ Trap

   1. SPSpeaker1:33

      I have done that? Uh, some experts, uh, say that the most valuable function of expertise is to say, "We will not do that thing." That is a B+ idea. That is not an A+ idea. We will skip the B+ ideas. Unhealthy organizations, unhealthy, uh, you know, people who create problems for themselves will see lots of ideas and go do them, and it turns out they produce lots of B+ ideas. That's not too good. You have to have restraint. You have to only do the maximum quality thing. Um, AI will make that harder. So then even when you're in a task, you produce some content and some output. In general, it could be good, but there are quality problems inside. If you haven't made yourself learn to think, write, and do whatever this expert process is you're trying to do very well, you will miss these quality problems. And then you slowly and subtly prevent yourself from learning, and you might even lose

4. 2:31 – 2:50

   The Trillion-Dollar Deskilling

   1. SPSpeaker2:31

      skill. I said in the book that this is a trillion-dollar problem, and I think it's really true. It is actually true that if we don't take active steps to protect ourselves from this, we will de-skill ourselves, other people, the next generation. That will harm the economy, not right now, but

5. 2:50 – 3:30

   Fighters vs. Coasters

   1. SPSpeaker2:50

      a few years from now. The difference between, uh, those who care and are trying to protect their learning and those who are just pushing through and using AI, one is to your disposition. Are you just driven to make sure that you can do your own independent work and get your own independent results? This is some-sometimes just something about a given individual that makes them fight and protect their learning. But a lot of it also has to do with incentives and the s-social structure you're embedded in. If the nature of my job is to produce more lines of code as a software engineer, I'm focused a lot on software engineering today, or burn lots of tokens.

6. 3:30 – 4:00

   Burning Tokens, Running in Circles

   1. SPSpeaker3:30

      This is another one that is very common now. Jensen Huang from NVIDIA is very famously on interviews saying, "If I'm paying five hundred thousand dollars for an engineer, they need to be burning at least two hundred and fifty thousand dollars worth of tokens." You can burn many, many, many tokens very inefficiently. It's like saying, "I burned a lot of, uh, calories today." That-- This could be okay, but did you just run in circles? The immediate first question is: Is that a B+ idea or an A- idea? You have to

7. 4:00 – 4:37

   Reward the B+ Killers

   1. SPSpeaker4:00

      set some threshold. A wise leader in an organization will reward people with cash or promotion or visible recognition for stopping a B+ idea from making it forward. You have to do more to create incentives for people to insist on superb ideas only. And then someone who does pick an A+ idea will win. And you'll go, "Wait, I didn't know you were allowed to do this." And it's like, yeah, this is how innovation works. You have to be very patient. You have to be persistent. You have to focus on, "Nope, that's not good enough."

8. 4:37 – 7:03

   What Shadow Learners Reveal About How We Build Skill

   1. SPSpeaker4:37

      [keyboard clacking] In twenty eighteen, I published the first paper out of my dissertation, which was on robotic surgery, uh, which generated this idea of the shadow learning process. And shadow learning is building skill through non-approved means, inappropriate means. Why would people do this? The reason is that in current circumstances in surgery, but I have since checked this in over thirty-five occupations, when a new technology arrives that disrupts the way that you do work, it also disrupts the way that you're supposed to learn on the job. What most of our skill comes from doing the work, and most of that comes with as we interact with experts on the job. The new technology allows the expert to do more work independently, which means that there's less room for the novice to participate. So you show up to the work, you hope to learn and build skill just by doing, and the expert is on Zoom, never comes to work. I don't get to interact with them. What am I learning after a year, two years? Not as much. Most people will continue to try to learn the old-fashioned way. You know, I will try to find ways to, uh, get involved in projects or something like this. Um, and they will struggle. But a very rare few, uh, set of individuals will instead of trying to participate normally, they will find norm and rule-bending, culturally inappropriate ways to build skill. So, for example, in s- robotic surgery, medical residents who were effective at building skill, given this barrier to participation, they, uh, found ways to operate on patients without the senior surgeon in the room. It's not illegal, but that is not appropriate for a junior resident to be operating on a patient alone. But this is what the residents who were very good at building skill quickly did. They also, for instance, spent a lot of time on YouTube watching recorded video of surgery, like, a hundred times as much as a normal resident. So shadow learning is, "I can't build skill the normal way, so now I have to invent deviant or rule-breaking, bending ways to build my skill." I've found this in every other occupation I've looked at. These are not practices we should copy, obviously. It's not appropriate. However, these people are fighting to protect some things in their work experience that will allow them to

9. 7:03 – 7:59

   The Skill Code - Three C's

   1. SPSpeaker7:03

      build skill. [keyboard typing] Figuring out what that is, what are they fighting to protect, was the purpose of writing my book. The first third of that book, I looked across all the shadow learners that I found across many contexts to figure out what do they have in common. In fact, none of them explicitly identified these characteristics. Like, "I have a plan. I'm going to protect these things in my work." But if you look at their behavior, you can see what they were fighting to protect, and that's the first third of my book. Challenge, complexity, and connection. The three C's in The Skill Code. So the point is the shadow learner is a critical source to go to to figure out what are they fighting to protect? What can we do to keep working conditions healthy for productivity, but also for skill development? Not copy them, but they're a diagnostic inf- They, they give you critical diagnostic information.

10. 7:59 – 9:16

    Challenge - Learn at the Edge

    1. SPSpeaker8:00

       So challenge is the one that is perhaps most intuitive for folks, which is you need to be close to but not at the edge of your capability to learn, to build skill while you're working. So it must be very difficult, require a lot of focus. You're not happy and relaxed. It's very intense experience, somewhat stressful, and you're not performing at your best. A little bit less than your best because you're straining. It's ch- It's a challenge. You must have a healthy amount of this challenge in order to really build skill well. Another, though, is that an expert is there to help you manage the frustration associated with that challenge. It's natural when you try to do something that is at the edge of your capability that you will fail in small ways. This is required. When you fail, you get frustrated. If an expert is present and doing a good job, they can help you put this failure into context to say, "Yeah, you tried to do this, and you didn't quite succeed, but you're now capable of attempting this task. Last week, you couldn't even attempt." So they help you sort of understand that your frustration is natural and that, in fact, you have already progressed quite a bit. The expert has an important role to play in helping you process the frustration of challenge enough so that you can continue to push

11. 9:16 – 11:29

    Complexity - See the Whole System

    1. SPSpeaker9:16

       yourself further. The next of those is complexity, which is about how you digest and engage with the broader work experience that you're embedded in, not just the focal task. So the focal task for a surgeon might be making a suture well and then tying a knot. These are very focal skills that I have to get very good at. But a good surgeon that's learning well is also engaging with the nurse, the scrub tech, the supply in the hospital, the finances for the hospital, the computer, and the IT system. They're trying to understand the entire system of work that they're embedded in, and they give themselves time to reflect on the whole system so that they can process and understand it. Engaging with complexity b- makes you better able to handle surprise and discover new ideas because I'm more attuned to the system. The individual, you must always preserve space and time to reflect on the circumstance and look left, look right, try to understand. You yourself can do a great deal to be curious and try to engage with the broader system. At the same time, performance pressures in jobs are very high, and it's very hard for individuals to devote the time and effort to do this kind of engagement. And leaders in organizations can do a lot to enable this kind of success. So, for instance, you can have job rotation program. I have seen two warehouses where, in warehouse one, people just go in, s- get on the line, and do their job. And they get paid the same wage as someone over here who, um, th- will learn more in, uh, warehouse B because they come in, and for two to three days, they work on the line, they put their item in the bag, and then they are rotated to a different part of the line. And these rotations give you an exposure to different parts of the experience. Same wage, same job, same job title, but that person is going to be more adaptive. Interestingly, the leaders in those organizations are not doing it to try to develop employees. When I ask them, it's very fascinating. They're doing it to create resiliency in the line. What matters to them is that the building is an effective processing unit. That means it needs to be able to handle surprise. It needs to be able to notice problems and catch quality

12. 11:29 – 12:58

    Connection - Trust Is a Skill

    1. SPSpeaker11:29

       issues. And then the third C, which is connection, which is a bond of trust between human beings. Trust and respect, actually, I should say. We don't often think of bonds of trust and respect as critical for learning, except actually if you reflect on your own work experience, anybody, and you think about when did I learn the most and why, often it's a who. There's a person. The person who trusted you, who gave you an opportunity, who gave you some healthy feedback, that you wanted to earn their trust and respect by getting better. I can speak for myself. This was part of my career for sure. This kind of bond of trust and respect for the novice, it provides them with motivation to do better because you-- it's just important to you intrinsically. This is how humans are wired. We want to earn the trust and respect of mentors, senior people. And it goes for the other direction as well. Senior people, it's meaningful to realize that you can help a junior person develop and give them challenges to help them do this. You want to earn their trust and respect in response. But for the junior person, it also means that that senior person, if you do this, will give you your next opportunity. They'll help you. So it's a functional thing as well. It's not just relational. Challenge, complexity, and connection were consistently evident across all of the shadow learning that I looked at. All these people who were breaking and bending rules, they were finding ways to protect challenge, complexity, and connection. Unconventional ways, inappropriate ways, but they were grabbing for those three

13. 12:58 – 13:09

    What Leaders Must Do Now

    1. SPSpeaker12:58

       things. [keyboard typing] [gentle music] What's the leader's role in protecting healthy, good quality

14. 13:09 – 13:56

    Learn in Public

    1. SPSpeaker13:09

       target selection? Individual leadership action is and has always been important. It will remain important. You have to lead by example and get primary data, personal experience with how people are doing the job in the organization. Most effective leaders now that I see spend personal significant time with the most advanced AI building things and showing how quickly they can do that in a way that's valuable to the organization, and including displaying failure and waste when they use. Reporting out to the organization as the leader or a leader of a function and saying, "You know, I tried to do this thing with AI, and it was very bad. I decided it's a bad idea." But still, revealing this to people shows no one knows how to use this technology now. So modeling

15. 13:56 – 14:56

    Hire Juniors, Learn Both Ways - Inverted Apprenticeship

    1. SPSpeaker13:56

       this is, is very important. The general trend, uh, that seems to have shown up in the economy in the last year or year and a half is that firms have slowed down hiring at the junior level and tried to retain and grow talent at the more senior expertise level. It is very shortsighted. You need to be much more assertive about hiring junior people who are AI native and can do astounding things with AI, even if they're professionally less experienced or have no experience, and set up learning dynamics that are bidirectional so that the senior person can learn from the junior person and the junior person can learn from the senior person. I called this in my book and in my research inverted apprenticeship. Especially when new technologies arise, you need a bidirectional learning. New employees have fresh perspective on the organization and can help improve things. So healthy organizations are willing to take a short-run, uh, hit to the productivity in order to ensure that they're ready

16. 14:56 – 16:38

    When AI Beats Us at Everything

    1. SPSpeaker14:56

       for tomorrow. [keyboard typing] It's very rare in human history that we get a general-purpose technology that really changes the way we do things. It is going to be messy in the beginning. No one will get it right. We must all learn from each other, and so it's important to have, have high standards but also be forgiving with, uh, yourself and others and understand that everybody's going to be wasteful and make mistakes. I think it's important to be open to the possibility that AI in between three and forty years will be better at everything than humans are. Literally everything, all tasks, including empathy, including judgment, including creativity. This is not a crazy view to hold anymore. It is not obvious to me that our governmental, educational, institutional systems will adapt fast enough, even given a thirty-year timeline, even given fifty, and that means change will be more disruptive and difficult. We can either make AI part of the solution and make for a better future and a better path to the future that is less painful, more beneficial for more people, or change will come in more painful ways. I want to live in the future where we are grateful that AI has arrived, that we can look around and see that everybody is better off, not just a few, because of AI, and I think we must act immediately and together to create that kind of future. [gentle music]

Episode duration: 16:38