No Priors Ep. 28 | With Khan Academy’s Creator Sal Khan

1. 0:00 – 8:41

   Sal Khan's Journey

   1. SGSarah Guo0:00

      (instrumental music plays) Sal Khan democratized learning with his educational YouTube videos that turned into Khan Academy, which has 150 million learners from all over the world today. 15 years later, he's as energized as ever about making learning personalized with artificial intelligence. This week on the podcast, Elad and I talk with Sal about the impact of AI on education. He says, surprisingly, we're on the cusp of the biggest positive transformation that education has ever seen. Khan Academy has recently created Khanmigo, a chatbot tutor that can nudge learners in the right direction if they get stuck. Sal, welcome to No Priors.

   2. SKSal Khan0:38

      Thanks for having me.

   3. SGSarah Guo0:39

      Sal, can you start by giving us some background on yourself and how you ended up starting Khan Academy?

   4. SKSal Khan0:44

      Sure. (laughs) Y- you know, you go way back, my original background was in tech. I go to business school, I end up being an analyst at a small hedge fund, and it was shortly into that, it was 2004, I was a year out of business school, I had just gotten married, I was based in Boston at the time. I was born and raised in New Orleans. My family was visiting me up in, up in Boston after the wedding, and it just came out of conversation, my 12-year-old cousin, Nadia, was having trouble with school, math in particular. She was being placed into a slower math track. Her parents, my aunt and uncle, hadn't gone to school in this country, so I don't think they understood the implications that, on that track, she wouldn't end up taking calculus in high school, et cetera, et cetera. So I took it pretty seriously. I said, "Hey, I'm up for tutoring you, Nadia, if you're up for receiving it." And she agreed, so she goes back to New Orleans, I start tutoring her remotely. She actually gets caught up with her class. She was actually having trouble with unit conversion. She gets a little ahead of her class. I call up her school. I said, "I really think Nadia Rahman should be able to retake that placement exam." They say, "Who are you?" (laughs) I say, "I'm her cousin."

   5. SGSarah Guo1:40

      (laughs)

   6. SKSal Khan1:40

      And, and they let her. And that same Nadia that started off as, for lack of a better word, a remedial student, was then put into an advanced math track. So I was, I was hooked. I, it was fun to s- stay connected with family. I really enjoyed geeking out on the math and it was, it seemed to be really helping my family. And so, I started tutoring her younger brothers. Word spreads in my family free tutoring is going on.

   7. SGSarah Guo2:02

      Mm-hmm.

   8. SKSal Khan2:03

      Before I know it, I'm tutoring 10, 15 cousins. And, you know, my, m- I still had the day job. I was working as an analyst at a hedge fund. And I saw a common pattern with my cousins. Uh, they just, the main reason they were struggling was they had gaps in their knowledge. Oftentimes, the reason they were having trouble with that algebra equation was because they weren't fluent in dividing decimals, or negative numbers, or exponents. And so, I started making software for them, which I did for fun, 'cause I, that, that part of my brain wasn't being used fully in my hedge fund job. (laughs) I started making software that would generate problems for them, give them immediate feedback, provide hints if they needed them, allowed me to keep track. I put a little database behind it and allowed me to keep track of what they were doing and when they were doing it, and what they were getting wrong and right. And I, I called that Khan Academy. That was the domain name, it was available! It was kind of a fun family project. And a- about a year later now, we're about 2006, and I was showing this off at a dinner party. A- and by this point, my family had moved out here to Silicon Valley.

   9. SGSarah Guo3:00

      Did you still have the hedge fund job?

   10. SKSal Khan3:01

       The same hedge fund job, but actually, it's interesting. I have a whole theory, uh, that we can keep referring to that benevolent aliens are, uh, using Khan Academy to prepare humanity for first contact.

   11. SGSarah Guo3:12

       Okay.

   12. SKSal Khan3:12

       And, because there's a series of things (laughs) in, in, in my journey that seem like it was almost like, you know, people were moving me around (laughs) in a way that, that benefitted this. But one of them was my boss, I mean, it was a two-person hedge fund at the time, his wife decided to become a law professor.

   13. SGSarah Guo3:31

       Hmm.

   14. SKSal Khan3:31

       And it was interesting, 'cause at first she thought she was going to go to UVA and in case, you know, we would've ended up living on a farm in Charlottesville or something. (laughs) But at the last minute, she decided that-

   15. SGSarah Guo3:40

       Okay.

   16. SKSal Khan3:40

       ... she wanted to take the job at Stanford. And so, uh, my wife and I came out here and my wife ended, who- who's a physician, uh, she ended up doing her fellowship at Stanford because of my boss's wife. Not wise choice, but that, that landed us here in Silicon Valley which I think ended up becoming very important for the future of Khan Academy. And then, I was showing this off at a dinner party in, in Silicon Valley to a friend, and he said, "Well, you know, this is cool, but how are you scaling your lessons?" And I'm like, "Well, I'm not. Like, it's hard to do with 15 cousins-

   17. SGSarah Guo4:07

       Yeah. (laughs)

   18. SKSal Khan4:07

       ... what I was doing with one." He said, "You know, there's this YouTube thing. Why don't you record some, your lessons as that?" And I immediately d- thought that was a horrible idea. I thought YouTube was somewhat frivolous for dogs on skateboards, cats playing piano, et cetera.

   19. SGSarah Guo4:21

       Mm-hmm.

   20. SKSal Khan4:21

       Uh, and, you know, I was like, "You know, what, what, how do I even record that?" This was before phones had cameras, et cetera, but I decided, "Oh, th- there's got to be a way to record your screen while you're drawing something," and it's called screen capture. (laughs)

   21. NANarrator4:33

       (laughs)

   22. SKSal Khan4:33

       And then I just, uh, started using it, and I started just covering things that I found myself repeating a lot from my cousins. And, uh, my cousins in- famously or infamously (laughs) told me that they liked me better on YouTube than in person. I think what they were saying is-

   23. SGSarah Guo4:48

       (laughs)

   24. SKSal Khan4:49

       ... they, they, they liked having an on-demand version. There was no judgment, no shame if they had to review things. It was available all the time. Uh, but they really still appreciated having me in their life, I think. Uh, but then, you know, YouTube, just a lot of people started discovering it. A lot of people were also discovering the, the web app. I actually had to shut down registrations because it was crashing my $30 a month Java Servlet web hosting (laughs) um, th- uh, th- that I was using at the time.

   25. SGSarah Guo5:14

       At what point did you open it up to anybody that wasn't a cousin?

   26. SKSal Khan5:18

       Well, the YouTube videos by defi- you know, uh, I, I didn't, I, I saw no reason to make them private.

   27. SGSarah Guo5:22

       Right, I meant the, like, Java Servlet app.

   28. SKSal Khan5:25

       The Java Servlet app, initially I had it opened up to anyone who wanted to register and I was actually, you know, my day job, I was an analyst at a hedge fund, but I was emailing local schools and seeing if they wanted to use it. And some of them w- were interested in using it. Famously, the first school that used Khan Academy, this was in 2007, was Sidwell Friends, which is, I didn't even know. They're, they're a very-

   29. SGSarah Guo5:45

       Mm-hmm.

   30. SKSal Khan5:45

       ... elite school in DC. It's where all of the, you know, senators and presidents kids go. But they were actually, a fifth grade classroom, this teacher was John Mormino, was one of the first, and he was just using this home brew software that I (laughs) that I was, I was creating. But I actually stopped registrations on that at 10,000 because, once again, it was crashing my $30 a month web hosting (laughs) and my cousins weren't able to use it.
2. 8:41 – 19:53

   Mastery Learning and AI in Education

   1. SKSal Khan8:41

      we didn't allow those gaps to fill. Whenever we felt a little bit uneasy about how fluent we were with something, we d- we gave ourselves a little bit of extra practice. And we weren't just memorizing things. We always wanted to understand why, you know, why does the long division algorithm work, you know? Let me think about that. Does it... Okay, that makes intuitive-

   2. SGSarah Guo8:57

      Mm-hmm.

   3. SKSal Khan8:57

      ... sense, why, um, the place values work, et cetera, et cetera. So I think when you do that, then later and later, math frankly gets easier and easier, not harder and harder. And if you do the opposite-

   4. SGSarah Guo9:06

      Mm-hmm.

   5. SKSal Khan9:06

      ... uh, where you, w- where you don't understand the c- the conceptual ideas behind the algorithms and the equations, and then you also have fluency gaps. I mean, the number of classrooms I now visit where seventh graders, they understand exponents, but they literally, you know, three times seven, they're grabber- grabbing their calculator. I'm not exaggerating. I literally saw a seventh grader in the Bronx-

   6. SGSarah Guo9:26

      What? (laughs)

   7. SKSal Khan9:26

      ... for three times seven, grabbing their calculator, and I literally slapped their hand. I don't know if that can get in trouble for that anymore, but like-

   8. SGSarah Guo9:32

      (laughs)

   9. SKSal Khan9:32

      ... I literally slapped their hand. I was like, "No, you figure out three times seven." And the kid, he started to use his fingers. And I'm like, this is, on some level, malpractice that no one sat down with this kid for, like, an evening and said, "You have got..." Like, it has to be automatic. It's gotta be in milliseconds that you know what three times seven is. 'Cause h- he, he understood all the concepts of exponents and algebra and all that, but the cognitive load of having to do that. So anyway, I was experiencing that from a very intuitive point of view. Obviously, once Khan Academy got more traction, started working with more educators, people would tell me, "Oh, you're just talking about mastery learning." I was like, "Yes, that's exactly what I'm talking about," which is just-

   10. SGSarah Guo10:10

       (laughs)

   11. SKSal Khan10:10

       ... that you always have the opportunity and incentive to fill in your gaps. "Oh, you're talking about differentiation, that everyone is learning at different paces and you have..." J- I was like, "That's exactly what I'm talking about." And then people are talking about, "Well, yeah, mastery learning, Benjamin Bloom," even though it's arguably the oldest way of learning, that you keep working on something if y- (laughs) if you haven't mastered it yet. He coined it as mastery learning. He also wrote this whole, you know, two sigma problem that if someone gets one-on-one tutoring, personalization, mastery learning, can they get, can get accelerated dramatically. So yes, it was intuition for me, uh, that seemed to work, but now I'm very familiar with a large body of literature that actually backs it up from a, from pretty, pretty rigorously.

   12. EGElad Gil10:48

       Could you talk a little bit about the range of things that Khan Academy does now? Because you have everything from Khan Labs to Khanmigo to a variety of other efforts. It'd be great to just kinda get a view of, you know, broadly what you focus on currently and what are the areas you're most excited about.

   13. SKSal Khan11:02

       Yeah, and the answer I'm gonna give you will probably make any, like, textbook strategy person cringe. (laughs) Uh, because (laughs) , you know, oftentimes people say, "Just focus on one thing," and maybe it's coming from me that I feel like the, the opportunity to transform education's not going to happen if you just do it narrowly. So I, I view our charter, uh, you know, the mission, free world class education for anyone anywhere, which by itself is a very big statement, uh, and almost delusional when I was one guy operating out of a closet, you know, a little over a decade ago. But we are, we are covering all of the core academic material from pre-K through the core of college. That's the goal. Uh, we wanna do it so it's personalized. We actually even wanna do it so it can r- eventually result in credentials. And so what we are doing, already on Khan Academy... So we have Khan Academy Kids, which is used by nearly two million kids, primarily in America. It's, it's actually kind of turning into, like, the Sesame Street of this, (laughs) of this generation. Really great efficacy studies. Kids are spending an average of 90 minutes a month on it, which is pretty high for an average. That's a large chunk of all of the kids in America between the ages of three and seven who are using that. That's all subjects. That's math, reading, writing, character development, social/emotional, whatever you wanna call it. Then as you get into older grade levels, it's called Big Khan. You know, that's where we just keep going from just the basics of math all the way through calculus and statistics and multivariable and, and, and on and on. Science, we have science, a pretty strong progression from, I would say, late elementary school through early college as well, going all the way to biology, chemistry, physics. And this is not just videos. This is... Most of our resources are actually behind the exercise platform where you can get as much practice as you need, deep item banks, immediate feedback. Teachers can keep track of what they're, what's going on and, and, and assign through the platform. We've added humanities, so, you know, American history, civics and government, world history. We have art history. So we're, we're filling out that grid of all of the core academic material. Financial literacy, computer science, so, so all, all of that, economics is, is happening. Early on an- in 2012, a-2011, I wrote a book, The One World Schoolhouse, and the first third of the book was kinda like the history of education. The middle third of the book was my journey getting here. The last third of the book was in a world where things are changing and tools like Khan Academy exist, what should education look like? Just, it's almost like a first principles exercise. Like, could we have full-year schooling, mixed-age classrooms, personalization mastery throughout, more time for hands-on if you can do the other stuff more efficiently? One thing to write about it, a whole other thing to implement it. On top of that, my oldest child was about to enter kindergarten, and I felt like a hypocrite if (laughs) if I was, uh, telling everyone to do mastery learning and if my kid just went to the local school that was not doing mastery learning. So we started Khan Lab School literally underneath, on the first floor of the offices of Khan Academy. That was back in 2014. That is now a K through 12 program. We've had three graduating classes already. Uh, so I'm happy to talk more about that. We started a- another online school with Arizona State called Khan World School. We just started last year with that, but that's now 6th through 12th grade, and there's some very exciting things happening there. And then most ... Well, I- I'm glazing over a lot. We are working on credentials, so we have a pilot with Howard University where kids in Title I high schools are getting mastery on Khan Academy, and that's resulting in college algebra credit, which is, solves a major need. Happy to talk more about that. (laughs) And then, uh, most recently, we're doing a lot with generative AI. We started partnering with OpenAI about a year ago, so well before it was really a big thing, and we had to keep it quiet for a while. Uh, but Khanmigo now, you know, it's a, in our minds, it's a tutor for every student. It's a teaching assistant for every teacher. It can support students while they do traditional work on Khan Academy, answering questions, making it relevant, motivating them, even advising them. It, we have functionality where it can be a guidance counselor, it can be a academic coach. But they can also debate the AI and practice their fine, you know, fine-tune their argumentative skills. They can talk to simulations of historical or fictional characters. And, um, teachers, they can use it to create lesson plans, grade, create rubrics, refresh, refresh their own knowledge.

   14. EGElad Gil15:06

       Mm-hmm. You've said that, um, we're on the cusp of the biggest positive transformation that education has ever seen. How important do you think AI is relative to the broader set of access that you have through things for YouTube and online coursework and all the rest of it? Is this a complete game changer? Is it an add-on? Like, what's the relative degree of importance of this shift right now?

   15. SKSal Khan15:25

       I think in the very short term, it is going to be a meaningful add-on. I think if you go three to five years in the future, it will be a game changer. And the reason I say that is, going back to Benjamin Bloom, but I think this even predates Benjamin B- the gold standard was always to have a personal tutor. Uh, you go back to if you were a prince in most of ... If you were Alexander the Great 2,300 years ago, you had Aristotle as your personal tutor. And Aristotle would speed up, slow down, motivate you when you're feeling down, like do all of these things with you, and that was always the gold standard. Two, 300 years ago, utopian idea of mass public education, but in order to do that economically, we had to make compromises, one of which is n- you don't get a personal tutor. You don't get a one-on-one to- teacher. We're gonna batch you into groups of 30. We're gonna move you at a set time or pace. We're gonna apply some lectures and standards and homework, et cetera. On the test, some of you are gonna get a, uh, 100%. Some of you are gonna get 80. Some of you are gonna flunk it. Too bad. The, the batch needs to move forward, and somehow we expect those of you who didn't know 20 or 30 or 40% of the material on the simpler stuff to understand the more advanced stuff. And what a- what, what happens, those gaps accumulate and kids start falling off, and then you eventually put kids on different assembly lines, uh, when you start tracking them. And that kind of worked during the Industrial Revolution where you didn't need a lot of people in the knowledge economy. You needed kind of basically educated people to work in factories, et cetera, et cetera. You know, not, not, not so acceptable anymore, but that was a compromise that we had to make. Benjamin Bloom, 1984, comes up with, uh, writes this two-sigma problem where he showed one-on-one tutoring, two standard deviation improvement. Two standard deviations takes someone from the 50th percentile to the 96th percentile. So an average student then becomes a very strong, becomes an exceptional student. He calls it a two-sigma problem because there's no way you're gonna be able to do that in a real classroom. Like, like, we don't have the resources to give every- everyone a tutor. And then he even, back in 1984, tries to see, well, what, what could you potentially do if you approximate a tutor using technology? And he, he kind of throws out the conjecture that you could get one standard deviation of improvement. And to a large degree, everything that we've been doing at Khan Academy was how can we approximate, start to scale pieces of what a tutor could do? So a micro-explanation. That's what a video does, and it's on demand whenever you want it. Exercises with immediate feedback, that also starts to approximate. And then give teachers and give tutors and give parents information so that they can a- also act more like tutors. As opposed to just giving a lecture to everybody, they can look at the data, where kids are, and then do more focused interventions. Have groups of three or four. The kids that need help with the negative numbers while the other kids keep working. Now work with the five kids who have h- need help with decimals while the others ... So it was all about, okay, in a class of 30, can we help approximate tutoring? What generative AI ... And I didn't think this was gonna happen in my lifetime. So when we started p- we were playing with GPT-4 about a year ago, it blew my mind that you could actually get it not only to pretend to be a tutor, it actually had good, quote, "tutor moves". It was, it was being truly Socratic. It had some big issues with it around the math and the hallucinations. We've been working pretty feverishly to mitigate those pretty significantly. The- it's pretty darn good now. And so it's already ... And we've, we've already started putting it out in real schools out there. We're already seeing it's increasing engagement. More kids are not only engaged, but they're getting ... They're, they're not getting as blocked. Teachers are definitely ... You know, we haven't done rigorous studies on it yet. It's very early. But anecdotal, teachers are like, "That's answering questions that the students either were afraid to ask or that I as a teacher would not have been able to get to." Teachers are already really enjoying the teaching assistant functionality. It's saving them a ton of time writing lesson plans, ru- rubrics. They kind of feel less alone as a teacher. And, and now we're looking at ways that it can even help address some of the problems that generative AI introduced, where, uh, we actually just made an announcement today with Instructure, the folks who make Canvas, the learning management system that, you know, goes out to, it's most of higher ed and a large chunk of K-12. We're like, "Well, if, if you use the AI to create rubrics, lesson plans, and then" ...... the AI essentially administers the assignment with the student and doesn't do it for them, but does it alongside them. Then the AI can report back, not just on the outcome of the assignment, but actually the process. Like, "Hey, I worked with a student. We brainstormed thesis statements and then they were having a little bit of trouble backing it up, but that we eventually got there and it took us about four hours." And so you can be pretty confident in that situation that the kid didn't just copy and paste from ChatGPT. So I think in the next year or two, it's going to be a really useful tool that's going to increase teacher productivity and support a lot of students.

3. 19:53 – 23:10

   Future of AI Tutors in Education

   1. SKSal Khan19:53

      I think when you start going about three years out, we're already working on the notion of, of memory for the tutor. So the tutor, if you tell it, uh, if you ask the, the AI, "Why should I learn this?" And it says, "Well, what do you care about?" And you say, "Well, I, I love women's soccer." The World Cup's going on, so that's front of mind. (laughs) "I love women's soccer." It'll say, "Okay," and it'll, it'll make a, a connection to that. But the next time, it should remember that you like women's soccer, so that it doesn't have to ask you again. And so it's creating... But it could, you know, you like language in a certain... You like more casual language or more formal language or your reading level, whatever it might be. So n- we are actually already prototyping that it remembers these things about you. We're making that very transparent to the user so that they can say, "No, I'm not into women's soccer anymore. I'm into gymnastics." (laughs) Or whatever it might be. But we think that type of memory and the type of memory where it can refer to previous conversations will allow it to be more... Allow it to, to graduate from being just a on-demand help from, for certain tasks, to being something that could have a long journey with you, have a narrative. I know it sounds a little bit wild, but I could imagine en five or 10 years, you're gonna get your best college recommendation from Khanmigo. 'Cause Khanmigo's like, "I've been working with this kid for hours for the last 12 years." (laughs) And let me tell you about this. (laughs) And, and by the way, they love gymnastics. (laughs) Like, and so... And you have a great gymnastics-

   2. EGElad Gil21:10

      Yeah.

   3. SKSal Khan21:10

      ... program at whatever. Uh, so, so I think that's... And, and you're gonna have, you know, we're already playing with the text-to-speech. I know people have heard text-to-speech before. What's about to come will blow your mind. It's hard to differentiate from a real human being, which is scary on oth- other dimensions of life, but it's good for tutors. And then I think in three to five years, yeah, you're gonna be able to video conference with, with, with your tutor, which then makes it like-

   4. EGElad Gil21:32

      Mm-hmm.

   5. SKSal Khan21:32

      ... a real thing that you can have a long, a long journey with.

   6. SGSarah Guo21:36

      Have you ever read the, the Neal Stephenson book, The Diamond Age?

   7. SKSal Khan21:39

      Not only have I read it, I used to give it away to people. Every employee at Khan Academy, uh, used to get The Diamond Age, and, you know, you're referring to it because, you know, this, it takes place in this, like, neo-Victorian, not too far off future in, like, China, and this member of this, like, neo-nobility gets this AI ta- tablet app for his granddaughter to educate her, The Young Lady's Illustrated Primer, and it gets bootlegged and it gets in the hands (laughs) of 200,000 orphan girls who live in barges, and then they essentially just take over. And so I've always used Young Lady ill- Young Lady's Illustrated Primer to our team at Khan Academy as like, "This is what we're hoping to build in the long run." And I never thought we were gonna fully build it. I thought we were gonna be able to approximate it, but already Khanmigo can do some things that are maybe even beyond what The Young Lady's Illustrated Primer did, where you can talk to Don Quixote in any language that you want. Uh, you can ha- get into debates with it, et cetera, et cetera, and I think in the next three to five years, almost everything that Neal Stephenson imagined, I think he wrote the book in 1994, um, I think is actually going to be a reality, which I didn't think was going to happen in my lifetime.

   8. EGElad Gil22:52

      How does that impact the structure of the actual school? So say that you think ahead five or 10 years, and you're in the context of an advanced K through 12. Maybe it's Khan Labs, maybe it's something else. Wh- what happens or what changes or how do you interact with the tutor versus the classroom environment, and I know it's hard to predict the future, but I'm sure you've thought a lot about this stuff.

4. 23:10 – 29:35

   Education's Future With Generative AI

   1. EGElad Gil23:10

   2. SKSal Khan23:10

      I, I think big picture, at a place like Khan Lab School where y- you know, even before generative AI, we've been... And at Khan World School actually, we just documented, we, I, we were almost embarrassed to say the results, um, you know, Steve Levitt of Freakonomics fame, (laughs) he actually looked at it himself 'cause he was like, "This, is this real?" And he's like, "Oh, it's real." I mean, these kids are learning about three to four times faster. Not 3% faster or 30% faster. Three to four times faster.

   3. SGSarah Guo23:34

      Wow.

   4. SKSal Khan23:35

      And, and at Khan Lab School, we've consistently seen it, you know, at least 1.5 to two grade levels in a year, and this is before, before generative AI. So I think in some of these settings, we haven't had the same issues that the... You know, in the traditional school system, kids are getting about 0.7 grade levels per year. Now, I think if you go to more fluent areas, fl- uh, areas where parents are college educated and they can afford more supports for their kids, y- you're probably seeing at least a grade level or maybe closer to what we're seeing at Khan Lab School, or like 1.5, two grade levels a year. Uh, but I think in, in environments where basic fluency has been less of an issue, you're, you're now going to free up more time for teachers so that they're gonna have to spend less time doing things like grading and lesson pla- uh, most teachers spend about half their time doing that kind of stuff. So that, that gives them-

   5. SGSarah Guo24:25

      Mm-hmm.

   6. SKSal Khan24:25

      ... more energy for themselves, but also more energy to just be with the students, which I think is really powerful. Uh, students are gonna feel a lot more supported, and I think if you, if you fast-forward, mm, three to five years, kids are gonna feel very supported in their, that kind of, that basic fluency and that, that core academic, but I think it's gonna be like the Magic School Bus. Like, you're gonna be able to put on your, like, Apple visors or whatever that, that, you know, (laughs) and you're gonna be able to, like, go on with the, with the teacher and, like, you know, jump into the, the circulatory system or time machine yourself to ancient Greece and, and have a debate with Socrates, and you're gonna be... Like, I, I think we're gonna have... A- and you're gonna have, um, uh, you know, one of the things we do at Khan Lab School is just try to create more time and space for students' passions, and so you're gonna have a world where a middle school student is going to be able to create, like, studio quality techno music or, l- you know, a Lord of the Rings quality, (laughs) you know, epic, epic science fiction movie or fantasy movie. So, so I, I actually think it's gonna be very exciting, and I think if you go into, let's call it, more mainstream classrooms-... I think we're gonna go a lot further towards solving the, just the core fluency issue. And not just in math, but in writing as well because that's the other thing. The main problem with writing, it's very resource-intensive. It's hard to get feedback. It's very expensive for the teacher to give feedback for 40 kids. It's mind-numbing, frankly. Kids have to wait a week to get that feedback. Then they may or may not get a chance to iterate on it, and so most people aren't getting much practice, and that's why you're seeing a big problem with writing on top of the problem with math. So, I feel confident. I mean, already this year in, in Newark, New Jersey, which, you know, is not a special case school district, the North Ward ha- just started using Khan Academy pretty intensively in November. This is pre-generative AI. As of June, 70% of those 6,000 kids used Khan Academy at a level that's associated with pretty profound efficacy gains. We're still waiting on those test scores, but we're, I think we're gonna show for the first time in, like, US history, or recent history, a pretty large acceleration of a, of a, a large urban school district in math. And that's pre-generative AI. So, I'm feeling actually very optimistic about that. And then yes, especially for those students whose families might not get an opportunity to, to, you know, put them in AI summer camps like we do with our kids, or, you know, take trips or, you know... I think AI's gonna bring the world to them, um, in, in really, in really powerful ways. They're gonna be able to practice their debating skills. They're gonna be able to connect things to other ideas. They're gonna be able to create pretty profound things. They're gonna be ab- be able to also go on that Magic School Bus, and it won't s- gonna, it's not gonna be something that requires a, a ton of resources. So, I'm, I'm overall hopeful. You know, there's always forces of friction and cynicism and et cetera, et cetera, but because teachers have a lot to gain here, it's gonna save them a lot of time and energy. I think you're gonna see faster uptake of this than you see of other, other solutions.

   7. EGElad Gil27:13

      There's the old saying that the future is here, it's just not equally distributed. And with things where you have sort of strong structure in place, like the education system, is this a five-year transformation, a 20-year transformation, a 40... Like, how long will it take for schools to adopt this despite all the incentives to do so?

   8. SKSal Khan27:30

      Yeah, I've learned not to be too aggressive in my predictions, but I think it will be safe to say that in five years, almost all teachers will be using generative AI for lesson planning, grading, progress reports, rubric creation. I think that is alread- I, I, that might happen by, by, like, late next year, honestly. So that's-

   9. EGElad Gil27:51

      Mm-hmm.

   10. SKSal Khan27:51

       ... that's gonna happen fast. So, I think in a world where almost all teachers are already using the tool, I think having a generative AI tutor supporting the student, I think that's for sure going to be mainstream in three to five years. I think, you know, because ChatGPT introduced the emergency around cheating, like, it's not like it's a nice-to-have, it's, like, a problem (laughs) that's, like, broke the education system. People need a solution. And so I think that has to be fixed in the... and I think it's going to, going to be fixed with generative AI as well. That's, that has to come into play in the next year or two. So, you know, f- five years in the future, you might, when you just visit a classroom, you might not superficially see a lot of differences. But I think when you start double-clicking on it and you realize that, wow, teachers have a lot more time now for student-facing things, a lot fewer students are stuck, there's a lot more space for differentiation now, I think it, that that will be transformative, and I think you're gonna see it in student engagement, motivation, and, and in things like test scores.

   11. SGSarah Guo28:49

       So, in this novel, The Diamond Age, like, one of the things that happens is because these tutors are so powerful and they have memory, right, some of the things you're working on, and they're engaging because they have narratives and the dragons that's, like, teaching you calculus with, and physics with, like, your environment. There's a, there's a lot of engagement with that, and much less interest, in some cases, in sort of, like, group schooling settings. Like, can you talk a little bit about how you think about that? 'Cause you also started Schoolhouse, schoolhouse.world, with our, our mutual friend, Shishir, founder of Coda, and, and, you know, this is, like, small groups online, and believing that there's still the importance of this. Can you, can you talk about sort of how those pieces fit together and what the motivation was there?

   12. SKSal Khan29:32

       Yeah, absolutely. And, and you know, just a little bit more detail on

5. 29:35 – 33:22

   Connecting Learning Through Tutoring and Collaboration

   1. SKSal Khan29:35

      Schoolhouse. We started during the pandemic. Th- you know, that was a moment where t- like, Khan Academy's usage went up 3X, and it was clear that, you know, it's great if people are using Khan Academy, but they, they needed... everyone was isolated, and they needed more human support as well.

   2. SGSarah Guo29:48

      Mm-hmm.

   3. SKSal Khan29:48

      And I'm always thinking about, okay, how can you give something at scale that used to feel really expensive? And tutoring, live human tutoring, for the most part, is expensive. Uh, but I was like, "Well, what if you could leverage volunteerism?" And so it was a bit of a, you know, optimistic idea that there would probably be a lot of people who wanna do, you know, what I did with Nadia, that a lot of people would actually love to do that. And so we set up schoolhouse.world as a way, you know, it's, it's, it's another nonprofit, a sister nonprofit to Khan Academy. Its mission is to connect the world through learning. And the whole idea is, is that you get free tutoring, and who gives the tutoring? It's, it's volunteers. And a lot of those volunteers could be near peers. They could be high school students, college students, but some of them also are teachers who teach at a fancy private school and wanna give back. They might be a retired professor, et cetera, et cetera. It's, it's gone well. I mean, it's def- definitely at a smaller scale than Khan Academy. It's on the order of, uh, Schoolhouse is on the order of about 10,000 folks per month. But it's doing some pretty powerful things. Already a large fraction of the kids who take any given SAT are already using Schoolhouse to get live human tutoring. Th- you know, Khan Academy had been the official, uh, has been the official practice for kind of the, the asynchronous tutoring aspect of the SAT or the, the individual practice. But now you can get, you can join cohorts and be- be- become part of that. And, you know, I think the power of Schoolhouse is, uh, as powerful as it is to get help o- academically, I think that social connection, especially social connection across borders, et cetera, et cetera, hugely powerful. Same thing with Khan Lab School, Khan World School, we think it's all about the human-to-human connection. Khan World School, which is an online high school, but its anchor is a Socratic seminar where kids are getting together almost every day and they're debating things. Now, it's not...... five hours on Zoom, we think that's wrong. That's mind-numbing. That was what was wrong with pandemic schooling. Like, th- that's- that's literally gonna ruin kids' eyes and, like, ruin them.

   4. EGElad Gil31:37

      Mm-hmm.

   5. SKSal Khan31:37

      Um, but you know, an hour, two hours, but you also don't want completely asynchronous. You want some interaction. And at Khan Lab School, even though people think it's, Khan Academy is downstairs from that, of course, yes, we do use Khan Academy, but if anyone visits, you're going to see far more collaboration and interaction amongst these students, and far more talking (laughs) amongst the students than you will see pretty much at any other school. And I've become a believer more and more and more, yes, it is important to improve test scores, et cetera, et cetera, and we are doing that, but that social/emotional, that character development, that ability to collaborate, uh, with other folks, that's arguably the more, the more important thing. And if I, I almost didn't do it by design out the gate, but if I'm, one of the things I'm proudest about at- at all of these projects we're doing is that element, is that- that the- the collaborative element, the connected element. You know, whenever we bring in new faculty to Khan Lab School, they're- they're- they're always like, "It's weird how collaborative these kids are." Even when you get into high school, even in Silicon Valley in high school where a lot of other schools get quite competitive, secretly quite competitive, but everyone knows it, that's not happening at KLS. The kids are genuinely collaborative. I think part of that is more time and space to pursue your passions. I think a lot of, part of it is mastery learning. It isn't about, you know, n- elbowing out your peer. It's about, "Hey, everyone here should learn it." And I think a lot of it's about the peer support, which we s- we're trying to scale at Schoolhouse, but has always been part of Khan Lab School.

   6. EGElad Gil32:59

      How did the relationship with OpenAI and GPT-4 start, and the- the origins of Khanmigo? Was that something you came up with and you went to OpenAI? Did they come to you? Like, I'm a little bit curious about the origins of both the product and the relationship.

   7. SKSal Khan33:11

      Yeah, you know, this story is gonna be just another one of the- the data points behind the benevolent alien theory. Uh, (laughs) they, th- it was, uh, about a year ago, end of summer of 2022, uh,

6. 33:22 – 40:42

   Implications of GPT 4 on Education

   1. SKSal Khan33:22

      I got an email from Greg Brockman and Sam Altman, um, who I was familiar with, I didn't- I didn't know super well, saying, "Hey, we're working on our newest model. We'd love to- to talk to you about it." And I was skeptical whether it would have any real strategic bearing on what we do at Khan Academy, but just as someone who's followed the industry and has always been fascinated by this, you know, I- I knew, I had been following GPT-2, GPT-3, DALL-E, so I was like, "Oh, are they talking about GPT-4? That's kinda cool, if I get to be on the early (laughs) you know, get to see what's-

   2. EGElad Gil33:57

      Yeah.

   3. SKSal Khan33:57

      ... happening there." So, uh, we had that meeting, and- and they hadn't even finished the first training run of GPT-4, but they said, "You know, we think it's gonna be done in about two weeks, and we think this is gonna be the model that really wakes up people to the power of generative AI. We want..." There's- there's two reasons why they were talking to us. The first was, "We want to- to launch with some social positive use cases. To us, the most obvious ones are education and healthcare, and y'all are the first call we're making (laughs) ." Like, "And we wanna do it with organizations that people trust and is, are going to, like, do it in a thoughtful way." And so li- a- as far as I know, we were literally the first people they called. A- and- and- and I said, "Well, you know, I'm- I- I'm up for looking at it." Oh, and the second reason why they wanted to chat with us is, it turns out that Bill Gates, when he saw GPT-3, he's like, "Oh, this is cool," but, uh, but we all know GPT-3 d- really didn't have a good handle on knowledge. And he- he told the OpenAI team, he's like, "I'll be impressed if this could pass the AP biology exam." He literally said that. And so, I think part of the reason they also reached out to us is like, "Y'all have a lot of AP biology items. Could we test this (laughs) on- on-" And we're like, "Yeah, sure." I was a little bit of, like, "What's in it for us?" But like, "Yeah, sure, that's kind of..." I- I mean, I knew what was in it for me. I just wanted to see what this was. So about two or three weeks later, they said, "Oh, you know, could we give you a demo?" I was like, "Yeah, sure." So it was me and my chief learning officer. We get in, and they had a, uh, AP bio question, and they said, "Sal, what's the answer to this?" And I was like, "Uh, okay," it was about osmosis. I'm like, "It's C." And they said, "All right, let's see what it says." And- and you know, it said, "It's C." I'm like, "Oh, yeah. That's- that's, maybe it's luck." I was like, "Ask it to explain why it's C." And I know everyone's used to ChatGPT and even GPT-4 now, but this was before anyone had seen any of this stuff. And I asked it to explain, and it explained it, explained it quite well. And then I said, "Explain why the other choices aren't right." It explained that quite well. I said... And that's when I started to get the goosebumps. And I- I said, "Create five more questions like this one." It did it. And that's when I'm like, "Okay, this- this is- this is a game changer." And then they're like, "Would you like access?" I'm like, "Yep. (laughs) Please give us access." So that-

   4. EGElad Gil35:59

      (laughs)

   5. SKSal Khan35:59

      ... that weekend, I literally couldn't sleep. And, you know, my- myself, our chief technology, our chief learning officer, we were literally Slacking each other at, like, 4:00 in the morning with, like, things that we had gotten it to do that, like, seemed like science fiction. The most notable was a- taking on personas, like acting as a tutor-

   6. EGElad Gil36:15

      Mm.

   7. SKSal Khan36:15

      ... and being able to engage in a Socratic dialogue. And we also were discovering some math errors and stuff like that. We were surfacing that to Open AI. It turns out that they did have some mistakes in their training data, which we, you know, so that was our service to humanity above and beyond the core (laughs) Khan Academy work. They realized like, "Oh, yeah, there's some math..." But then we started to say like, "Look, if we can- if we can figure out ways to put right guardrails around this, make sure there's safety, security, we can mitigate the math errors, we can mitigate the hallucinations, this is a game changer." So that's when we started working. ChatGPT comes out a few months later. I Slack Greg Brockman, I'm like, "Wait, hey, I thought we were in under NDA." (laughs) Like, "What are y'all launching?" And he's like, "No, this is just on GPT-3.5, but everyone's, you know, seems to be taken by this, even though what we're working on for was so much better." I- I was bummed at first, but I was like, "Oh," i- in hindsight, it was a good thing, 'cause it threw out this very imperfect thing that was not designed for education. The education world took it the hardest, 'cause kids are gonna cheat now, et cetera, et cetera. And then when we launched with GPT-4 in, on March 15th of 2023, it almost allowed us to directly address all of the concerns that people were already having around ChatGPT. And so that's, yeah, that- that's how we got to where we got to.

   8. EGElad Gil37:22

      That's really cool. H- how did you end up thinking about safety? 'Cause you mentioned, for example, you wanted to ensure there was safety around the tool, and probably certain answers, or certain questions couldn't necessarily be asked. Like, how did you end up approaching that?

   9. SKSal Khan37:33

      So step one, we-... uh, as soon as we saw it, we're like, "Oh, this could be used to cheat." And so we, like, okay, how do you help without cheating? And so that's when we said, we started putting guardrails in the, the system prompts, so to speak, where it says, "Don't give the answer. Make sure the student does the bulk of the work. It's okay to help them, but primarily ask leading questions." You know, we have, we have a lot. We've tested a lot of prompts and to just get to that sweet spot, and we're constantly tweaking it. So that's the first guardrail. Second guardrail was, you know, this is a, a new environment, and we don't know where kids are gonna take this, so let's make it transparent to other stakeholders what's going on. So we said, "Okay, we're gonna record every interaction with the AI and make it accessible f- to parents and teachers, uh, if the student is under 18." And, you know, we can debate that for, you know, there's good arguments why older students, there might be some constructive stuff that they could use the AI for that's probably not good to share with parents. But anyway, that's a... We took the most conservative stance out the gate, especially for under 18 there. And then the, the next one is, you know, we have a second AI that's moderating the conversations, and if it looks like it's going into a unhealthy place, not only will it kinda shut down that, that, that thread of the conversation, but it will actively notify parents and teachers. And so that's, those are... And then we, we, we are trying our best to do, to do digital literacy as well to students to recognize, like, look, here's what you could use generative AI for. Here's where you should be skeptical of generative AI, et cetera. And I, we, we, we are feeling increasingly confident that that combina-... And, and also, none of the interactions, and this is to credit to OpenAI, none of the interactions are being used to train the AI. And, uh, and, uh, and arguably, there could be benefit in the future for that to happen. But because this is all new frontier and it's all very sensitive, we and OpenAI just said, "Yeah, let's just keep this as kosher as possible. Let's just make this, like, you know, as safe as possible."

   10. SGSarah Guo39:28

       If, if you think about, you know, going beyond Khan Academy, you've also spent 15 years generally thinking about education. Do you have a prediction for what happens to, like, university with all of these technologies?

   11. SKSal Khan39:39

       Yeah. I, I think... I, I don't think generative AI dramatically changes what was already going to happen to universities. I, I think it's not news to anyone that the return on investment for, for university, especially if you have to take on debt, is, is, is mixed.

   12. SGSarah Guo39:54

       Mm-hmm.

   13. SKSal Khan39:54

       Depending on, on who you are and what you do with it. Obviously, we have a student debt crisis, and even if we were able to forgive some, it's not gonna solve it for the ne- for the next generation of students. And so universities are bloated with cost. They're not particularly good at, you know, either informing or preparing students for... You know, in certain cases, they are, and in s- other cases, they're not, of, of preparing students for what they need. I think you're going to have a world where the very elite universities, pe- people join those, yes, i- i- i- it's, it's almost like you're joining a cast, you're joining a, a club, so I think they're going to be fine. I think they're, you know, whether or not they can justify their ROI, they're gonna be fine. I think, now, there's some interesting things that are happening around, you know, legacy admissions, et cetera, et cetera, so (laughs) that'll, that'll be interesting. I

7. 40:42 – 46:47

   Future of Education and Job Skills

   1. SKSal Khan40:42

      think the community colleges at the other end of the spectrum, they're very flexible, and they are very good at thinking about, what are the needs? Like, okay, are there, are there emp- are there industry certifications that could be useful? Okay, we can support students for that. Are there other types of, you know, mini badges that could help someone get a job? We can do that. So I think the community colleges that are low-cost and nimble and are able to adapt to the changing environment, I actually think they're gonna do fine too. I think that vast middle of universities, especially the ones that are charging, you know, $60,000, $70,000 when you include room and board per year, and a lot of the kids are graduating with debt that is very hard to even cancel with bankruptcy, and they're not, they're not getting the, the pay to be able to... Uh, you know, I, I, uh, w- I tell everyone, you know, all this student debt stuff, they should be holding the universities accountable. Like, where did that money go? Like, who got that money, right? Someone lent it to the student, and then that went someplace. It went to the university. They took the money, thank you very much. And now they're just kinda hiding when you have trillion dollars or $2 trillion of debt and it's ruining these kids' lives. I think if you, if you, if you just said, "You know what? The s- the, the universities are accountable for 10% of that," their, (laughs) their tune will change overnight. Overnight, they will start informing students much more about the career outlook. If you were to, like, yeah, not everyone's gonna become a, a museum curator, let me just tell you that right now. (laughs)

   2. SGSarah Guo42:04

      (laughs)

   3. SKSal Khan42:04

      And by the way, the kid who you heard about who, who just became a muse- museum curator majoring in this major, it turns out that their grandmother, like, built the museum, right? Like, (laughs) it's a, it's a hard job to get. So don't think that this is gonna lead there. Just be informed. And I think the universities, if they were o- on the hook for even a small fraction of that debt, would almost, like, not let you major in something, unless (laughs) they felt like they were, you were gonna pay back the debt. So I think you're gonna, uh, you know, there's a reckoning there. University costs can't get bloated forever. I think some of what's going on around admissions, the scrutiny, I think it's healthy. And, you know, I'm a big advocate of a competency-based world where it's not how long you sat in a chair or, you know, how long you threw Frisbees and went to frat parties. (laughs) It's, it's, it's what do you know and what do you not know? And so we are working, and I know others are working, to create competency signals, credentials that, if you do it, employers will recognize it. And it doesn't matter if you did it when you're 16 or when you were 36, you're now qualified for that job. I think that's where employers wanna be, and I think it's not gonna, quote, "disrupt college," but it's gonna give college an alternative, uh, which I think is very healthy.

   4. EGElad Gil43:12

      I think one of the big shifts that's coming as well is just the type of skills that are gonna be relevant in a more and more AI-related world. So if you think ahead 10 years, 20 years, 30 years, et cetera, are there specific things that, for example, you're encouraging your kids to learn or that you think that people should really be focused on as they think ahead for their own children, you know, in terms of the skill sets that will be relevant in a generative AI world?

   5. Allah and I both have, like, preschool, kindergarten-aged kids, so these are active decisions.

   6. SKSal Khan43:40

      No. It's a good, good, good question. It is something I think a lot about. I, you know, I think there's two, there's at least two metaphors that, that can help us think about that problem. One metaphor w- could be, you know, we can go back to the 19th century, and I'm not just gonna talk... You know, a lot of people made the metaphor of like steam engines and horses and all of that, but I actually think the camera is the best metaphor because being a- an artist was a real thing. It was really a technical field. You were a portrait painter, and like the best artists, you know, you would study for years to be as accurate to reality as you could and then you could like, can look at how the light moves and all that, and all of a sudden a camera comes out. And all of a sudden artists started saying, "Okay. That's the end of art." (laughs) Like, this thing can capture reality better than, you know, anyone can, but then very quickly people realized that no, in some ways this liberates the artist, that this, this allows us to think about... I mean, it's not a coincidence that the impressionist movement kind of coincided with the advent of the camera. People started saying, "No. It's not about capturing reality. That's... Art should be about the impression. Art should be about the feeling it conjures." And it actually all of a sudden led to an explosion of, like, what art is. It allowed art to come out of that trap of just painting nobility and like these grand scenes and start getting into things that evoke and, and really challenge us. I think the same ana- you know, people are saying, "Oh, crap. This thing can write pretty well. (laughs) This thing can code pretty well. This thing can create movies, create images pretty well." So what that tells me is it, it, it kind of liberates the creator to move beyond that. Someone who can elevate and integrate and manage these tools... You know, the other metaphor I would say is imagine if... Just as we, we're seeing with Khanmigo, every teacher now has a teaching assistant. So now every coder is now going to have an army of coding assistants. Every writer has, has a writing assistant. So people are gonna have to move into the managerial ranks quite quickly and figure out, okay, instead of being a coder, I'm now an architect. Instead of being an, a, a, a writer, I'm now an editor, and in order to do that you have to know those skills really, really, really well, arguably better than previous generations. So I think it's even more important that people get not just adequate but excellent at writing, communicating, coding. I think creativity, because there's so many outlets, s- someone who's creative who leverages these tools is going to be like unstoppable. They're gonna have like godlike power, and that's good. I know there's, you know, there's these debates with the Screen Actors Guild and (laughs) it's a very sensitive issue, but the way I think, just as YouTube, there's a lot of talent that would not have been discovered without YouTube. You would not know about Justin Bieber. You would not know about me. Yes, I just put myself in the same category, very self-aggrandizing.

   7. EGElad Gil46:14

      (laughs)

   8. SKSal Khan46:15

      But, you know, think about how many movies we s- we've, we've all seen that had like $100 million budget and were horrible, like horrible waste-of-time movies for $100 million. You're now going to have a ton of people be able to create movies with similar special effects and, and acting and, and, and screenplays and everything, and a lot of them are gonna be horrible. In fact, even more of them are gonna be horrible, but every now and then we're gonna discover the Justin Bieber of movie-making and, and like that's an amazing movie and that some kid, he or she did it on a budget of $1,000 not $100 million. So how do we foster that? How do we give more time and space for that?

8. 46:47 – 47:41

   Importance of Traditional Skills in Education

   1. SKSal Khan46:47

      But I think if you have kids, it's kind of very traditional advice. They should just get really good at reading, writing, math. I, I've never been a subscriber of like, oh, there's Google, you can search stuff, you don't have to learn anything anymore. Nope. The more that you have rapid access to knowledge in your head, the more that you are fluent with your mathematics, that you have information and, and concepts at the tip of your fingers, these tools will accelerate you more than anyone else, and the kid that has to get the calculator for three times seven or has to go to Google to figure out when World War II was, the world is, is going to pass them by.

   2. EGElad Gil47:20

      Amazing. I think that's a really great note in terms of an optimistic view of, you know, it's the same basic skills that'll be important 20 years from now and kids really just need to learn the basics, and you've provided such a great platform to achieve that by. So thank you for all the work you've been doing.

   3. SKSal Khan47:33

      Oh, thanks for having me. (music)

Episode duration: 47:41