Classroom 2050: Unleashing AI, XR, Gaming

1. 0:00 – 4:00

   The classroom of 2023

   1. SPSpeaker0:00

      This is a moment for reimagining learning

   2. SKSal Khan0:02

      Generative AI kind of jumped us forward 20 years faster

   3. SSSteph Smith0:06

      There is this assumption that every student is gonna use it to cheat

   4. SKSal Khan0:09

      The most power happens when the AI works with the student

   5. SSSteph Smith0:12

      Can teachers handle VR? Can teachers handle AI?

   6. SPSpeaker0:15

      Yeah.

   7. SSSteph Smith0:15

      Like, absolutely.

   8. SPSpeaker0:16

      I have 100% of teachers opting in to our program

   9. SKSal Khan0:19

      You can go to Ancient Rome, you could talk to Caesar, you could talk to Cleopatra

   10. AMAllison Matthews0:24

       The kids in school today are growing up into a world that's incredibly fast-paced and it's complex

   11. SPSpeaker0:31

       The technology's here, the, the pedagogy's here, the teacher training is here. We just now need to put our heads down and execute

   12. SSSteph Smith0:37

       The year is 2023, and so much has changed in the last 50 when it comes to technology. We have smartphones that fit in our pockets. We have the world's knowledge accessible to anyone with an internet connection. And we even have tools now that summarize incredibly complex concepts in the voices of our favorite fictional characters. But now, reflect on the state of the classroom, the very forum where we raise the next generation. How has that evolved in the same time period? Despite so many children already using these cutting-edge technologies like AI, XR, and gaming outside the classroom, interestingly enough, they haven't really made their way in. And in this episode, we explore why that is, especially given the learning loss that we saw prior to and during COVID, and how these technologies might actually help us radically upgrade the classroom of the future. This episode was actually recorded live at the Aspen Ideas Festival, which I got to attend for the first time ever recently, and I was joined by four unique guests. We had Sal Khan, founder of Khan Academy, which recently unveiled Khanmigo, an AI-powered tutor. We also had Anurupa Ganguly. She is the founder of PrismsVR, which is using VR to better teach our children math and science concepts through spatial reasoning and abstraction. We also had Allison Matthews, head of Minecraft Education, who's tailoring the most popular game in the world to empower and teach young people skills like coding or even active citizenship. And finally, we had Romy Drucker, director of the education program at the Walton Family Foundation, who has a bird's eye view into the challenges and opportunities in the space. Honestly, I wish I grew up in an era where these technologies were at my fingertips. So let's not squander this opportunity, and let's dive in. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any a16z fund. Please note that a16z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com/disclosures. [upbeat music] All right. Welcome, everyone. Welcome to Classroom 2050. Welcome to our classroom for the next 50 minutes. I'd love to just start by getting a pulse from the room.

   13. SPSpeaker3:19

       Mm.

   14. SSSteph Smith3:19

       How many of you are excited about new technologies like AI, XR, extended reality, gaming? Let's put your hands up. Who's excited? Great. Now, how many of you... Keep your hands up and then put them down. How many of you feel like these technologies have successfully made it into the classroom for the better? Put your hand down if you don't think they've made it there. Keep it up. I only see a few hands-

   15. SPSpeaker3:43

       [laughs]

   16. SSSteph Smith3:43

       ... still up there. But luckily, we have some wonderful founders today who are working on exactly that problem, figuring out how we elevate the classroom after it's been relatively stagnant, quite frankly, for many years. And to s- kick things off, let's really ground ourselves on where

2. 4:00 – 5:15

   Data on the current classroom

   1. SSSteph Smith4:00

      we are. Romy, maybe you can help us here. What is the data saying in terms of the student experience? What are kids really facing? How can we really get a better understanding of that?

   2. SPSpeaker4:10

      Thank you so much, Steph, for being here and moderating this discussion. As many of you know, the pandemic was a huge setback in education, and it exacerbated and widened already deep inequities. We know from data that came out last week that math achievement levels are the lowest they've been in this country since 1990, and literacy achievement levels are the lowest they've been since 2004. And the pandemic erased two decades of progress that we had made in education through a variety of different change initiatives. So we have a lot of work to do. The good news is that students, families, and educators agree that this is a moment for reimagining learning, both embracing technology, but also thinking about how we strengthen relationships in classrooms, how we better support student wellbeing to address the mental health crisis.

   3. SSSteph Smith5:11

      Thank you, Romy. And Anurupa, I wanna move to you.

   4. SPSpeaker5:13

      Mm-hmm.

   5. SSSteph Smith5:14

      What will it take specifically

3. 5:15 – 7:23

   Developing the next gen of STEM talent

   1. SSSteph Smith5:15

      to develop the next generation of STEM talent? I know you have some data there. Give us a, a grounding there in terms of what that will really take and what we're seeing up until now.

   2. SPSpeaker5:26

      Absolutely. I discovered that the top indicators of success in post-secondary STEM is, number one, your ability to think spatially. The Soviets had first discovered this in 1970s. So if you kind of like, uh, think about a rough definition, it's your ability to rotate 3D objects in your mind and maintain perspective of that object. The second top indicator of success in post-secondary STEM is your ability to abstract. And when I say abstract, I don't mean looking at word problems, talking, drawing diagrams and visualizations, creating charts and tables, and creating equations, because those are all 2D representations of thought. Abstraction is Isaac Newton. He was sitting under a tree, an apple fell. He's like, "Whoa, how do I ascribe language notation and build mathematical models of this physical experience?" So I started Prisms three years ago to scale these best practice methodologies. How do you immerse children in problems in the first person, create the intellectual need for why they have to learn something, and then through building a solution to that problem, learn the math and science that you need to? So how does Prisms work? Um- You, we, we, uh, uh, utilize virtual reality, so students put their VR headsets on, their mobile VR headsets, and they're transported to a world. It is my mission and commitment that when kids see math problems in, in math class, they should not be like, "That's not a real problem. No one really important's actually solving it." They should go, "I heard that in the news. I know absolutely that very smart people are working on this," building mathematical identities from a very young age, and this belief that I can contribute to the mathematical sciences. So that through line of going from physicality, simulations, data visualizations, charts, tables, equations, is really scalable. It's most of secondary math and science. So we've built out all of algebra one, geometry, middle school math, algebra two, pre-calculus. We'll be pushing into all of higher ed just very soon.

   3. SSSteph Smith7:03

      I mean, we, we have so much to dig into there.

   4. SPSpeaker7:05

      [laughs]

   5. SSSteph Smith7:05

      And by the way, we didn't have a chance to call this out, but-

   6. SKSal Khan7:08

      Yeah

   7. SSSteph Smith7:08

      ... we wanted, you know, there's many ways that people learn, and instead of talking about the technologies that these wonderful founders are building, we wanted to make sure you had a glimpse into really what we're talking about here. And so we've talked about different technologies, so this is a glimpse into VR. One of

4. 7:23 – 12:42

   AI in the classroom

   1. SSSteph Smith7:23

      the other technologies entering the classroom is AI. You know? One of the topics of the year. So Sal, why don't you talk to us about what you're building there, and of course we'll also pull, pull up, you know, some of the technology that you're building.

   2. SKSal Khan7:34

      Yeah. So, uh, y- you know, Khan Academy's been around a little while now, and most folks associate us with videos, but most of our resources have been around personalized practice. I, I'm kind of a, a traditionalist and a progressive, where, where I'm, I say, "Well, but there's also just gaps in students' foundations." The number of times that I've gone to like a seventh grade math classroom, this was recent, I was in the Bronx. They're working on Khan Academy. The kids were doing, uh, exponents, and it was like something to the 3rd to the 7th power, so it's to the 21st power, 3 times 7. I saw an 8th grader grab a calculator for that. I literally slapped their hand. I'm not sure if you're allowed to do that.

   3. SSSteph Smith8:04

      [laughs]

   4. SKSal Khan8:04

      But I slapped their hand. I'm like, "What's 3 times 7?" And I saw the 8th grader start to do this, and that's just a very basic example-

   5. SSSteph Smith8:09

      Mm-hmm

   6. SKSal Khan8:09

      ... of like there's massive fluency gaps in students in just some-

   7. SSSteph Smith8:13

      Sure

   8. SKSal Khan8:13

      ... very basic, and it's causing too much cognitive load. So that's where Khan Academy started, is, hey, can we give people as much practice, learn at their own time and pace? You're a teacher with 30 kids, but there is a way that all 30 kids can go at different paces. You as a teacher can understand where the kids are. Ha- and the students have multiple shots at goal in order to understand those, those underlying skills. That's what we've been working on for, for 10 plus years. Um, and happy to talk more about it, but we're seeing some very promising results pre-AI. Uh, but then, um, last summer, OpenAI reaches out to us. Uh, this is obviously months before ChatGPT and all of that, and they said, "Hey, we're training our next generation model." It was GPT-4. Uh, for those, y- when, when ChatGPT came out, it was based on GPT-3.5. Um, we were, I was skeptical 'cause I had been paying attention to previous generations of these models. But when we started playing with GPT-4, we said, "This one's different. This one you can actually get it to act as a tutor." It still had issues around hallucinations, which is when the AI just makes up facts.

   9. SSSteph Smith9:15

      [laughs]

   10. SKSal Khan9:15

       It still has... That, that is the, the technical term for it. Um, it, it has, it had issues when it came to, um, uh, math, which obviously, you know, we don't only do math, but math is very close, close to our heart and it's gotta be accurate. Uh, but we kept wor- and we, we were worried about guardrails and what if students use it for negative things or they go into some type of weird rabbit hole. So we said, "Oh, could we put guardrails around it?" Uh, so we started working pretty aggressively. We were under NDA with OpenAI since, uh, last summer. Then ChatGPT came out, um, and we weren't going to launch our thing until the GPT-4 launch in, in, in, in March. And I, and I remember te- telling Greg Brockman, like, "Hey, I thought we were waiting." He's like, "No, ChatGPT is nothing new. It's just a chat interface on top of GPT-3.5." And that captured everyone's imagination, and it, it kind of, um, uh, immediately put a bit of a bomb into the education system, where everyone was like, "Oh, kids are gonna use this to cheat, write their essays, and by the way, it's bad at math and it makes up things," et cetera, et cetera.

   11. SSSteph Smith10:15

       [clears throat]

   12. SKSal Khan10:16

       I, I was initially bummed 'cause like we're working on what I think is a reasonably thoughtful solution here. Now this thing just gets thrown out into the universe-

   13. SSSteph Smith10:23

       Mm-hmm

   14. SKSal Khan10:23

       ... and it's just a, everyone's first impression is going to be AI plus education is really bad. Um, in hindsight, I actually think that was a blessing because what it did is it forced people to grapple with some real things. When we first saw GPT-4, months before ChatGPT, we were like, "Oh yeah, people are gonna be able to use this to write essays. People are gonna be able to use this to do college es- you know, do their homework, et cetera. How do we mitigate these, these risks?" But the fact that ChatGPT, within a month or two, 40% of teachers were using this very imperfect tool for themselves for things like lesson planning. And students were already using it, but it was, it was cau- it was forcing the issue on the education. Most of ed tech is, things are okay, maybe we could use technology to make it better. But this was like, wow, things just got broken. Maybe there's, we need to do something to make it better. Um, so we launched w- our A- our implementation, which we call Khanmigo. We've always tried to approximate personal tutoring, uh, at Khan Academy through videos and exercises, but I think, uh, generative AI kind of jumped us forward 20 years faster than I thought, um, if you had asked me a year ago, uh, where it can act as a tutor, and, and maybe just as interestingly act as a teaching assistant to help generate lesson plans, uh, help refresh a teacher's knowledge, create rubrics, et cetera, et cetera. And that last screen you just saw, it was actually in Spanish as well. It works in every major language, which by itself already solves a major issue where you can be doing your work in English, but get help in Brazilian Portuguese, in Spanish, in Vietnamese, whatever language you need that help in. Um, it does not cheat. If you ask the stude- if you ask Khanmigo, um, "Hey, tell me the answer," it will not tell you the answer, but say, "I'm your tutor," it'll act Socratic and say, "Well, what do you think is the next step?" We've done a bunch. I, I, unless y'all want to, I won't go into all the details, but it's substantially better, I won't say perfect, but substantially better at math than what you would get just out of the box, uh, with, with GPT-4. We have a second artificial intelligence. Well, all conversations are monitorable by parents and teachers, which we thought was an important guardrail. And then we have a second AI that monitors the conversations. Yeah.

   15. SSSteph Smith12:28

       Founders so far have talked about maybe how we're early on, there are misconceptions, misperceptions from whether it's students, teachers, people in school boards about welcoming this technology. So I wanna get into those conversations too. But Allison, please

5. 12:42 – 14:12

   The role of gaming in the classroom

   1. SSSteph Smith12:43

      Give us a glimpse in the, into the world of gaming because that is a form of technology that has immense growth as well, just like AI and extended reality, um, but maybe isn't met with the same level of open arms. So would love to hear what you're building.

   2. AMAllison Matthews12:58

      Yeah, for sure. Minecraft is an immersive 3D virtual space that's ful- filled with beautiful biomes and all kinds of animals. So Minecraft Education, we built on top of that wildly pl- wildly popular video game a powerful teaching and learning platform by putting safety, security, privacy features that educators need and parents expect on top, as well as an in-game library that's full of learning experiences from learning to code and math, history, literacy, but all the way to learning how to be a more active citizen, um, or learning about climate change. We know, and maybe you do too, that video games are one of the most powerful ways to engage a person in a deeply engaging, deeply immersive experience, and it lights up intrinsic motivation. Intrinsic motivation is just that desire to, to keep trying to go deeper, to learn more, um, because you love it. You're having fun. You ha- you're passionate about it.

   3. SSSteph Smith14:10

      Amazing. Anurupa, I'm

6. 14:12 – 16:38

   Misconceptions of technology in classroom

   1. SSSteph Smith14:12

      gonna go to you next with these misconceptions, partially because Anurupa is actually a previous educator herself. And so tell us a little bit more about maybe the way that people view the introduction of these technologies, whether it be VR, but also AI, also gaming, and where maybe some of those assumptions may be misplaced.

   2. SPSpeaker14:31

      Yeah. I think a common misconception is that technology's gonna replace the pedagogy, and it's not. Pedagogy is pedagogy, and so what the tech will allow you to do is it will scale that. So in the case of Prisms, you are building an elementary school building in India that creates a shaded region. You're building a satellite that does XYZ, and to do that thing you have to learn ratios. You have to apply, um, linear functions. And so what, what, what, what I've learned is that instead of talking about the technology, talk about the learning design. And when you're focused on the learning design, the- whether it's a, a, a powerful, um, AGI tool or a g- or game design, or in my case, spatial computing, that is just fueling a way of teaching and learning. And when you, when you project it in that manner, nobody pushes back. Like, I have 100% of teachers opting in to our program, and that would be, th- that, that might seem like, "Are you sure, Anurupa? Like, everyone's a little bit afraid of VR and..." No, because I, I... In my deck, you don't see VR anywhere. So that was, that's the first thing that, that I would say is a key misconception, is it's not about the technology. The t- technology powers the learning design. And the second thing that I would say was, um, it is then therefore incumbent upon those who are building learning solutions to over-index on training teachers in the pedagogy and not the tech. We focus too much time on, "Let me teach you on, about, like, how to utilize this hardware." No, we're g- we're gonna spend 30 seconds on that because you're actually very smart, and you know how to do it. You just, you just need to kind of have that confidence, and we're gonna spend the market share of the time on lesson study, intellectual preparation, um, job-embedded coaching. How do you get learning outcomes? So I, I would say kind of that's where my mind to in terms of the main-

   3. SSSteph Smith16:02

      Mm-hmm

   4. SPSpeaker16:02

      ... misconceptions. And the, and the bow that I would tie on that is, um, there's been this feeling and perception that school systems are regressive, and they're somehow contra to, to progress. When you show them how it's standards aligned, how you teach bell to bell in a 15-minute instructional episode, how you, uh, prepare for it, you're not gonna get any pushback. It's just kind of showing them how you go from point A to point B.

   5. SSSteph Smith16:22

      Sal, maybe you can comment on that. You already touched on some ideas where, you know, there's this assumption that with AI in the classroom, you know, every student is gonna use it to cheat. It- they're never gonna learn again 'cause why would they, right? They've got this, this assistant who can do it for them. So maybe just speak to that directly.

7. 16:38 – 20:00

   The ways teachers and students use AI

   1. SSSteph Smith16:38

      What are you seeing in terms of maybe the reality or what can be designed to maybe construe that reality?

   2. SKSal Khan16:44

      Yeah. I, I think, uh, ChatGPT has forced folks into one of two modes, either have some assignments where you just say, "Use, use whatever help you want," and now I have a higher ex- And that's reasonable 'cause we now use it in the workplace, et cetera. There's another mode even, you know, without technology. Everyone said, "Okay. Well, I guess we're gonna have to do more in-class proctoring now," things like that. Um, I think p- uh, both of those are probably part of the mix. We, we think actually the most power happens when the AI works with the student. Um, so, like, in, in a couple of months we're gonna be launching something and we're gonna initially do it for college essays, which is a whole other... This is the first-

   3. SSSteph Smith17:21

      Mm-hmm

   4. SKSal Khan17:21

      ... college admission cycle in the ChatGPT era.

   5. SSSteph Smith17:23

      That's true.

   6. SKSal Khan17:24

      So it's gonna be very interesting.

   7. SSSteph Smith17:26

      Mm-hmm.

   8. SKSal Khan17:26

      Um, but we've created activities where it can interview the student and help surface things that would make for good college essays, so brainstorm with the student, not for the student.

   9. SSSteph Smith17:35

      Mm-hmm.

   10. SKSal Khan17:35

       Um, a modality where you can write an essay of any kind, um, and the, and then ask for feedback from the AI, and not just on... You know, we've all seen, uh, technologies that can help with your grammar and things like that. Um, but once again, not just tell you, "This is how to write it grammatically correct," but say, "Hey, look at your verb tense again." Like, once again, act like a, like a tutor. But now with generative AI, especially with GPT-4, you can, it can really go into the argument that you're making and how do you make it more emotive, and how do you really make it an excellent, uh, paper. And what's interesting about that is when you work with the AI and all of the interactions are logged, um, a teacher can actually not just introspect on the final outcome, they can introspect on the process. So- We're building a mechanism right now where the teacher can a- can assign through the AI, "Hey, I want everyone in my 11th grade AP American History course to write a paper on how the world would've been different if the American Revolution didn't happen," or I don't know, something. And then every student will be able to work on it with Khanmigo, and Khanmigo will be like, "Okay, do you think that really backs up your argument? How about this?" Like, they're, they're riffing together, which I think it'll be a better experience for a lot of students, and then even while that's happening, the teacher will be able to talk to Khanmigo, and is like, "Okay, what, what are, what's everyone up to?" And says, "Okay, I al- I've already started to, you know, this student is doing great. This student seems kind of stuck. This student just copied and pasted something in there. I don't know where it came from. We might wanna double-click on that. Maybe they're using ChatGPT."

   11. SSSteph Smith19:00

       [laughs]

   12. SKSal Khan19:01

       Um, and, and I think then all of a sudden, um, you, you've, you've kind of addressed this issue, where now the artificial intelligence not only do you not have to worry about policing it in the same way, it can help students write better, it can give teachers better feedback. You know, a lot of mastery learning is multiple shots at goal. Um, if-

   13. SSSteph Smith19:17

       Mm-hmm

   14. SKSal Khan19:17

       ... if you're at a certain level, 80%, you should try again. You should be able to do it writing, but writing, it's very hard to get feedback. Uh, but now the AI can give multiple rounds, and then also give the teacher first pass of like, "You know, based on the rubric we created together, I'd give the student a four out of five on that dimension, a five out of five on that dimension." Uh, so I think that's probably where we're gonna go.

   15. SSSteph Smith19:36

       By the way, this isn't crazy. I did a year of exchange in Sweden, and they have a totally different grading system, including you can submit an assignment, and you can pass or fail, and if you fail, they tell you what to change. They tell you how to get better. So this is not, like, completely, you know, grown out of AI and, and, and totally new. These concepts do exist elsewhere.

   16. SKSal Khan19:56

       We've been doing that in our lab school with humans, but it's more resour- resource-

   17. SSSteph Smith19:59

       Right

   18. SKSal Khan19:59

       ... intensive, yeah.

8. 20:00 – 23:25

   The future role of a teacher

   1. SSSteph Smith20:00

      Exactly. Allison, actually, that dovetails perfectly into this idea that if we are introducing these technologies, what is the future of a teacher, or teachers in general? How does that role evolve? And then also, if you have time, I'd love to just hear your, your take on how people view gaming, 'cause that's a very interesting topic on its own, so.

   2. AMAllison Matthews20:18

      Yeah, so, um, so Minecraft Education is now in 119 countries, and we've got millions of players every month, and a big focus of the team is on connecting with educators so we can understand what are the problems that they're trying to solve. And one of them is similar to what Sal was talking about in, in that in any given classroom, you're gonna have a full range of readiness for whatever the course material is that's being covered. And so what teachers really need is a tool that enables differentiated learning. So a way that, um, someone who s- totally understands the content can s- can take it further, and someone who's, who's it's new to them, they haven't understood this before, just start to get familiar with the, with the material. And s- to your question of the role of the teacher, because the game supports, it's, we've got scaffolded learning experiences in the game, which means if you're stuck, if you don't know Minecraft or if you don't really understand the lesson, th- we've got little things built into the lessons that can help give you tips or clues on how to move forward. Um, so students can be self-directed in that way, which gives, frees the teacher up to actually work with the students who need more attention, or to challenge the students who are racing ahead. So, um, so that's one. I o- I wanted to point out one other thing for teachers, 'cause we've had a lot of teachers, especially early on, tell us, "Okay, this is the... Like, the kids love this game. Why would I bring this into the classroom? I'm gonna lose control immediately. They all understand it better than I do." So yes-

   3. SSSteph Smith21:56

      [laughs]

   4. AMAllison Matthews21:56

      ... this is true. And none of us as adults will ever understand Minecraft as well as the average eight-year-old.

   5. SSSteph Smith22:02

      Yeah. [laughs]

   6. AMAllison Matthews22:03

      That's just a fact. Um, and so what that requires is a mindset shift, right?

   7. SSSteph Smith22:08

      Mm.

   8. AMAllison Matthews22:08

      The kids in school today are growing up into a world that's incredibly fast-paced, and it's complex, so they need mindset that's gonna b- enable them to be successful in a world that's changing rapidly. So they need to build a growth mindset. They need to build leadership skills, collaboration, communication skills, and these are sort of those higher order things that can be, um, achieved through playing a game. And the, the teacher has the ability to generate some of these conversations in the classroom about, "Tell me, tell me why you did that in the game. Help me understand your thought process that made you go destroy that other student's, like, work in the game. Let's, let's have a conversation about digital citizenship and the fact that, you know what, what we do in a virtual world creates feelings in the real world." And this is something that, you know, it's really important for all young people, and adults frankly, to learn. Like, how can we be better, better citizens in the world? So just sort of an example of-

   9. SSSteph Smith23:13

      Definitely

   10. AMAllison Matthews23:14

       ... taking things to the next level.

   11. SSSteph Smith23:15

       I love that. All right, so Romy, how can the many folks from different backgrounds who have different associations with the classroom,

9. 23:25 – 25:55

   Personalization and ownership in education

   1. SSSteph Smith23:25

      how can they get involved? How can they really help us bring these technologies or this innovation into the classroom?

   2. RDRomy Drucker23:32

      Yeah, I love this conversation, and I think you're hearing a couple of consistent themes about learning in the future. One is the importance of agency and personalization. When we think about, and you heard this from all of our speakers, the kinds of experiences that are gonna help young people find their purpose, help them navigate this increasingly complex world, they need to feel a sense of ownership over their education, and they need to feel excited about learning. W- We released new research last week with Gallup where... So students are telling us that they want an experience that reflects the kind of personalization, the kind of ownership, this sense that they are not just sort of- ... in a transaction, um, but really building the skill sets and mindsets, as Allison said, that are gonna help them be successful. The other thing that's coming up is that, you know, the pandemic really kind of changed the conversation about where learning happens. And what we experienced is that learning can happen anywhere. Obviously, it needs to happen in school because that's where students are for most of the day. And also, many parents are thinking differently about what learning looks like, learning on a continuum, different experiences that provide students with those opportunities to engage and explore. And I think Minecraft is a great example, whether it's being used in a classroom or outside of a classroom, to build on top of the skill building and the mindset building that is happening. Um, and we know that educators and families play a huge role in that, and that, you know, the point of adoption for these technologies is too late. So one of the things we need to be doing is incorporating early on in the R&D process, and I know this matters to everyone on the stage, educator and family perspectives when it, when it comes to the development of these products. Um, and I know, you know, the speakers will have great examples of how you're facilitating that kind of inclusive R&D. But as we think about really deploying these solutions across districts, across schools, that feedback loop and strengthening that feedback loop is gonna be more important than ever before.

   3. SPSpeaker25:52

      That's right.

   4. SSSteph Smith25:52

      Absolutely. And Anurupa, maybe you can speak to that because

10. 25:55 – 28:40

    Getting hardware into the classroom

    1. SSSteph Smith25:55

       maybe the, the group has already picked up on this, but there's a level of hardware as well with VR, right? And so you not only have to get into these classrooms, but you have to get educators, school boards to opt in to this new cost. And so how are you doing that? How are you working with these folks to not only open their aperture for the technology, but also the cost and, you know, figuring out how you can roll this out really broadly so that if it is as game changing as we think it can be, that everyone has access.

    2. SPSpeaker26:24

       Yeah. And I wanted to kinda touch, uh, piggyback on the, the role of the teacher tr-

    3. SSSteph Smith26:28

       Please, yeah

    4. SPSpeaker26:29

       ... before coming to this, is that, um, I was a math director for many years. And w- at NCTM, we've been talking about student-based learning, problem driven learning, discourse for decades. So what's been really fun is that the role of the teacher isn't changing at all. She's just now able to do it. What was happening before is we were telling them to do discourse, we were telling them to do PBL, and the tools that she had were not amenable to that. So what's so great about going back to the question of adoption is when we go in, we're like, "Nothing is changing. You as the great teacher, you are launching an activation conversation." You put your headset on, and they start working. Teacher has a, a web-based dashboard, so she's monitoring p- and, and providing feedback. So when we share this pedagogy, school boards are like, "That's exactly how we've wanted to teach. How do we do that?" Oh, by the way, there's a hardware investment. Well, how much? Well, VR isn't for every day. VR has a very specific role in the curriculum. Um, all the other resources that we have, the, the, the wonderful computer software that, that's been built and evolved over the past couple decades, like, that's great. It just needs to be founded in something that's really real world and visceral for kids. So VR is only the first two days of a unit of study. Tells you why you're learning what you're learning. It helps you derive a mathematical construct through a series of physical and tactile experiences. And then once that's done, those card of headsets can go to the next classroom. So we're, we're deploying at about a 15 to one ratio at present. As we d- uh, d- d- create more and more content, that might need to come down to 10 and potentially five over time. But that's really been the sell. Um, and, and so I, I don't know if that kinda gets to the heart of the question.

    5. SSSteph Smith27:56

       It does.

    6. SPSpeaker27:56

       But we're selling a vision that is not new. People have always wanted to teach this way. We are now providing the hardware, and that, that, the hardware wasn't available five years ago. So for those of you, I've, I've been, I had a DK1. I'm a big VR en- a VR enthusiast. If you guys would have been following, a few years ago it was not standalone. There was a bulky umbilical cord with these, like, big pieces of hardware. Movement tracking was bad. Hand tracking wasn't there. Eye tracking, it was uncomfortable at best, right? That's not the case anymore. There's a psychological safety and a comfort that kids have. Kids, most of them have the Quest at home. So when they come in and see the, they see the Quest and the Picos in schools, I cannot tell you if I had a nickel for the number of times kids have said, "Finally, you guys are coming into the 21st century."

    7. SSSteph Smith28:35

       [laughs]

    8. SPSpeaker28:35

       'Cause they get to see the tech that they have at home in their classrooms.

    9. SSSteph Smith28:39

       Definitely.

11. 28:40 – 30:15

    Misconceptions of teacher’s abilities and appetite for tech

    1. SSSteph Smith28:40

       And I wanna just touch on something that you've ta- told me directly, and you kinda touched on there. But, like, when you are going to these schools, maybe a misconception is that, oh, teachers don't want this. But they're responding, like, "I've been wanting this forever," right?

    2. SPSpeaker28:53

       Yeah.

    3. SSSteph Smith28:53

       Like, this is, and, and they're also-

    4. SPSpeaker28:54

       Yeah

    5. SSSteph Smith28:54

       ... something that I thought was interesting that you've mentioned is they are smart, and there's this, this maybe misconception around, like, oh, can teachers handle VR? Can teachers handle AI?

    6. SPSpeaker29:04

       Yeah.

    7. SSSteph Smith29:04

       Like, absolutely.

    8. SPSpeaker29:05

       Well, like, it, it, it's what happened with, like, many teachers who'd held lower expectations for their kids of like, "Oh, well, like we don't know if the kids can do it." Same thing, same thing with district leadership. The number of times the superintendents have told me, like, "My teachers can't do this," I'm like, "Put your teachers in front of me." And I'm gonna tell you, say one example and then, um, uh, tie a bow. But Humble ISD, small 45,000 student district in Texas, um, they got, they were like, "You know, only a small group of my teachers are gonna wanna use this." I said, "Okay." Uh, you know, 'cause I've seen this a million times happen now. So they got 90 teachers in a room. They'd only purchased for about 40 teachers. Every single teacher put their hand up and said, "We wanna do it," and they were texting their friends. So 170 teachers opted into a program they'd only paid for, for 40 teachers. And again, it's underestimating what our teachers are capable, w- cap- capable of and what they want. And as I mentioned, like, kids are digital natives, so when the VR headsets come out, they're off to the races. They know the UI. So the teachers can focus on pedagogy. She doesn't have to worry about teaching them the controllers and teaching them all the, the, the kids have that. She gets to focus on the questioning, the conversation, the convergence on key mathematical notation conventions and outcomes. And we must build with that in mind, that if we make teachers the conduit for the tech, we are setting ourselves up for failure.

    9. SSSteph Smith30:14

       Mm-hmm.

12. 30:15 – 36:26

    Why implement these changes now?

    1. SSSteph Smith30:15

       Something you also spoke to was the why now. So Sal, I'd love to get your take on this. I mean, maybe it's a little obvious with AI because the last year has been so crazy. But at the same time Tell me a little bit more about the why now, and specifically, what might this unlock? Like, let- let's think a little more broadly about the next couple years and think past just like, okay, the, our kids will be using, you know, a, a chatbot.

    2. SKSal Khan30:35

       Yeah, I mean, to Romy's, uh, opening remarks, the situation is not good. And a lot of people blame the pandemic, but it was pretty bad before the pandemic. Uh, you know, people have looked at the NAEP results, and I think it was Detroit had 3% of kids in ei- I think they were eighth graders who were proficient in math. 3%. And it's like, wow, the pandemic was pretty bad. Well, before the pandemic, it was 6%. Uh, so it was still pretty bad. Um, and this is why you see things like a majority of kids in the US who graduate from high school, so you're already talking about, you know, the top, I don't know, 80%, and then the subset of them that wanna go to college, then now you're talking about the top 40 or 50%. A majority of them, when they get to their colleges, they don't even place into college algebra.

    3. SSSteph Smith31:17

       Right.

    4. SKSal Khan31:18

       This is the first time that mastery learning is being enforced, and their colleges say, "Wait, you're not even ready to learn high school math yet? You're gonna take remedial math," and that has all sorts of predictive, you know, that the kids aren't gonna graduate, they're gonna have debt, no degree, et cetera, et cetera. And there's even been people saying, "Well, maybe we shouldn't make algebra a requirement." I'm like, this is like, it's really, like, eighth-grade level algebra that we're-

    5. SSSteph Smith31:36

       Yeah

    6. SKSal Khan31:36

       ... that, that is considered a weeder class in our colleges. Uh, and that's because the average American student learns about .6 to .7 grade levels per year, um, which, yeah, after 12 years, you're at a seventh-grade level. It, it, uh, it all... Uh, so, so that's, that's the, the baseline. And so this is something that obviously we've, pre-AI, we're, have been very focused on, and, you know, we, we, we have a lot of... We have about 50-plus efficacy studies. I think we're the most studied platform because we're just out there. Uh, but they all say the same thing. People put in, if students put in 30 to 60 minutes a week, so not, like, a huge amount, they're accelerating 40, 50%. In some of our more, let's call it lab environments, we kind of run some schools. They're more like sister organizations, um, where the students are putting closer to 30 minutes a day. Because they're learning at their zone of proximal development, all of these ideals that ed schools have always been teaching, the differentiated instruction, mastery learning, zone of proximal development, but teachers had no way of implementing it, uh, we, we actually haven't been sharing this because we almost think that the results are outlandish. The kids are getting about two grade levels a year, like, when you're, when you do it, when you do it consistently. And so it's not just genius kids who are doing calculus in middle school. It can actually be maybe all kids. And that's at the same time that there's kids down the street who, by the time they're 19, they can't do algebra, much less calculus. And it has nothing to do with, I think, any kind of innate ability. Uh, so that's the baseline. Uh, generative AI, uh, you know, there's always been this dream of, uh, you know, the... I always cite these science fiction books, uh, Diamond Age, where they-

    7. SSSteph Smith33:07

       Favorite book ever.

    8. SKSal Khan33:07

       Oh, that's your favorite? Very good.

    9. SSSteph Smith33:08

       [laughs] Yeah.

    10. SKSal Khan33:09

        Well, you're kind of working on it, too.

    11. SSSteph Smith33:10

        I am, yeah.

    12. SKSal Khan33:10

        We're working, taking different... Where it's, you know, a future where it's a neo-Victorian China where there's an app for, for educating a, a, a member of nobility's granddaughter. It, it's an AI-based virtual reality app, um, that can educate a, a child, and it gets pirated. It gets in the hands of these 200,000 orphan girls living in, in barges, and they just take over, uh, because they get such a good education. And a lot of folks in the education world have always used the name of that app in the book, which is The Young Lady's Illustrated Primer. That's kind of like the code word for the true north that I think a lot of us-

    13. SSSteph Smith33:43

        Mm-hmm

    14. SKSal Khan33:43

        ... I mean, mine, c- I mean, well, everything-

    15. SSSteph Smith33:45

        Mm

    16. SKSal Khan33:45

        ... that all of us are doing are kind of somewhat inspired by things like that. And when, you know, the generative AI capabilities, not only did it throw a bomb into how our kids are gonna cheat, et cetera, et cetera, so that created a, an urgency to figure out solutions. Um, but it also cr- I think laid out part of the solution-

    17. SSSteph Smith34:04

        Yeah

    18. SKSal Khan34:04

        ... which we can now get much closer to the Young Lady's Illustrated Primer.

    19. SSSteph Smith34:06

        Yeah. By the way, for the educators in the audience, I would love if you could challenge your kids to come up with some positive sci-fi. We need more of that.

    20. SPSpeaker34:13

        Yeah.

    21. SSSteph Smith34:14

        Technology is-

    22. SPSpeaker34:15

        Yeah

    23. SSSteph Smith34:15

        ... you know, it's, it's inevitable-

    24. SPSpeaker34:17

        Yeah

    25. SSSteph Smith34:17

        ... and it's natural for us to-

    26. SPSpeaker34:18

        Yeah

    27. SSSteph Smith34:18

        ... you know, fix the world.

    28. SPSpeaker34:19

        I wanna say one thing to that. It's one of my favorite books because, um, uh, the au- the author's making a case for, like, one of the most important qualities that young girls should, should, should have, and if you sh- were to take a guess, what would that be? If you could teach your young girls anything, what would that be? Take risks.

    29. SKSal Khan34:35

        Yes.

    30. SPSpeaker34:35

        Courageous. Yeah. The book makes the case for subversion. Mm. And it's, it's, it's incredible. It changed my life. I read it, um, when I was, when I was a teenager before I went to, went to college. So thank you for referencing that. That, you're right. That's exactly what we're all kind of building towards.
13. 36:26 – 39:28

    Call to action for the classroom of 2050

    1. RDRomy Drucker36:26

       this. So I would just come back to what I said at the beginning, which is the importance of collaboration in tackling this kind of educal- educational change is just, we need it more than ever before. We need to be working across lines of difference. We need to be coming together around a common vision for what is possible for young people. Um, and that's just gonna take a different kind of conversation and dialogue. And as we do that, as we talk about ed tech, continuing to come back to the cultural and structural changes in education, which means just, again, continuing to think about how we organize schools, think about the choices we're giving to families, and making the job of teaching easier and more sustainable for teachers.

    2. SSSteph Smith37:12

       Thank you. Allison?

    3. AMAllison Matthews37:13

       Okay, so from a gaming perspective, I talk to a lot of parents who are like, "Yeah, my kids love Minecraft. I don't know what they do in there, but I know it's really, really lots of creativity." So my call to action would be to play the game with your kids, or if you can't play the game with your kids, sit beside them while they play because in Minecraft, they're, like, figuring out who they are as a human. Like, if they're the person playing with their friends who's organizing the mission, um, maybe they're going to grow up to be a leader or a manager. If they're the person who's designing, like, this incredible castle, maybe they'll grow up to have a world in design or urban planning. Or maybe they're the person that builds all the crazy machines with redstone, so maybe they're gonna be an electrical engineer or they're gonna, you know, go into science in some way. And so it's a great way for you to get to know your kids on sort of a deeper level, and then help them see the things that maybe they don't see about themselves.

    4. SSSteph Smith38:10

       Amazing. We've-

    5. SKSal Khan38:11

       Play with all of the above. That's the best way to learn it.

    6. SSSteph Smith38:13

       Yeah.

    7. SKSal Khan38:14

       And don't let fear get in the way of trying something out because if we do that, all that's gonna happen is all of our kids are gonna benefit from all of this. I'm not worried about them, but then the other kids are not, and then that's-

    8. SSSteph Smith38:23

       Mm-hmm

    9. SKSal Khan38:23

       ... just gonna create a bigger divide.

    10. SSSteph Smith38:24

        Thank you. Anurupa?

    11. SPSpeaker38:26

        I'll, I'll be brief. Um, I know this panel is called the Classroom of 2050, but I actually wanna be very clear it's the Classroom of 2023. It's happening right now. Like, it's, go to a school near you. All of us have deployed to hundreds of school systems. I mean, people who have been here for way longer than I have, millions and millions of kids are using. So, um, just wanted to, like, end with that, that the technology's here, the, the pedagogy's here, the teacher training is here. We just now need to put our heads down and execute. And if I ha- were to have a last word, the, the thing that I've seen missing in school districts, I've worked across many, many of them, I have very large teams, we mi- we were missing execution, and that's the ... Everything is now there. Pieces are there. We just now need to, like, collaborate and execute together. Thank you.

    12. SSSteph Smith39:05

        Thank you. And thank you so much, everyone, for listening. [applause] [upbeat music] Thanks for listening to the a16z podcast. If you like this episode, don't forget to subscribe here on YouTube to get our exclusive video content. We'll see you next time. [upbeat music]

Episode duration: 39:28