Transcript of Veritasium: What Everyone Gets Wrong About AI and Learning – Derek Muller

You could just have like babysitters in the classroom, and then you’d have the expert radio in, and they could radio in to a thousand classrooms.

So immediately, you have this economy of scale. And people see the same thing today. Elon Musk says AI education will be like having Einstein as a teacher for every child. The same idea. We’re going to scale up the best and the brightest. You know, we’re going to have this most intelligent entity teaching everyone instead of, I don’t know, what we have today.

Okay. So 1950s, it was TV. Academics actually conducted studies where they would have a lecture going on in one room like this one, then they’d have it videoed and closed circuit TV in the lecture room next door. And they would test the students to see who learned more or less. The answer, no significant difference. No big surprise because they’re getting basically the same experience, assuming the TV system works reasonably well.

Then in the 1980s, I think people at MIT anyway thought that they had it. They thought they had the revolution in education because the thing we had been missing was interactivity. And now you could actually interact with the computer, you could go back and forth with it, you could program.

So there was this idea that if we taught kids how to program, say how to program a turtle, then they would get better reasoning skills overall. What happened? The kids got proficient at programming the turtle, but those skills did not transfer to other sorts of reasoning. We didn’t get better thinkers, and that was the concept. That was why computers and interactive computers and doing this whole turtle programming was meant to revolutionize education once more.

In the 1990s, you get this, “the use of video discs in classroom instruction is increasing every year and promises to revolutionize what will happen in the classroom of tomorrow.” I don’t know. Many of you here may not know what a video disc is. It was a huge CD like thing, like a DVD. I used to use these, when I was in high school. So, you know, they were there. Did they revolutionize education? No.

There’s one more I’ll talk about. MOOCs, massive open online courses. About thirteen years ago, this was all the rage. “The big idea that can revolutionize higher education. It’s the MOOC.” “Will MOOCs revolutionize higher education?” At least they’re asking a question. “Revolution hits The universities know it’s hit.” “The phenomenon of MOOCs revolutionizing education in the digital age.” “The online education revolution drifts off course,” gets sad. “The MOOC revolution may not be as disruptive as some had imagined.” “Why MOOCs won’t revolutionize higher ed?” “What if MOOCs revolutionize education after all?” A story in ten headlines.

Anyway, I hope you take my point. People are all too excited, all too ready to put the word revolutionize next to education. This is old school, new school, it’s the same school.

What is going on? We shouldn’t be using this word revolutionize. “You keep using that word. I do not think it means what you think it means.” You know it’s a good talk if there’s a Princess Bride reference.

Why Revolutions Fail to Materialize

So why didn’t these revolutions materialize or why didn’t they eventuate? You could say educational institutions are just they have a lot of inertia. They’re not willing to change. We’re all just set in our ways. You could say it was just technological overhype. Or you could say we didn’t have the capabilities of AI yet. This time this time is different. Or maybe it’s something else.

So I’m going to explore this concept a little bit. I think one hint comes from this clip, a question that I asked to some people in Los Angeles. So let’s take a look.

[VIDEO CLIP STARTS:] DEREK MULLER: You go into a toy store and there’s a toy bat and a toy ball. Together they cost a dollar 10. And the bat costs a dollar more than the ball. How much does the ball cost?

PEOPLE: [In unison] 10 cents!

DEREK MULLER: We’re all wrong, aren’t we? Is the answer? Think about what is the answer. What is the answer?

PERSON: 5 cents.

DEREK MULLER: Yes. Very good. [VIDEO CLIP ENDS:]

Now, like, I don’t think these people are silly. I don’t think they’re stupid. I think they could work it out. You know, if the ball was 10¢ and the bat was a dollar more than the ball, it would be a dollar 10. And so together, they would be a dollar 20. Right? So the correct answer, obviously, you guys know it, 5¢.

So why do they all say 10¢? It’s because that number came to their head as they listened to the question. They don’t know where that number came from, but it was in their head, and it sounded right, so they blurted it out all at the same time. Really quite amazing. Right?

System 1 and System 2 Thinking

Something I love about that clip. So why do they do that? Well, the insights are in this book, which is one of my favorite books of all time. “Thinking Fast and Slow” by Daniel Kahneman. So in the book, he’s talking about our two systems of thought.

We’ve got two things, two kinds of processes going on in our brains at one time. The fast processes, system one, and slow processes, system two. And at the start of one of his chapters, he imagines, what if there was a Hollywood movie where these systems are turned into characters. And so I decided to take that inspiration and run with it. So I created these two characters inside my head.

System one is on the left and system two is on the right. Now system two is kind of the person who you think you are. That’s the voice in your head. It’s a slow, effortful system. You know, thinking takes some effort.

If you ask him to multiply, like 13 times 17, he’s going to say, I don’t want to do it. But you could force him to do it, and he can go through a series of steps. So that’s the thing about system two is it’s slow, it’s methodical, it can catch mistakes, it can work through processes, it can think about thinking.

Now system one on the other hand is rapid fire. This guy is doing things fast in the background. You have no idea that he’s doing it. He’s collecting all the information from your senses. He’s pulling out the relevant information, and he is getting rid of everything else because you’re just getting bombarded with stimuli. So he’s pulling out those key pieces of information that you’re going to work with. And he does all of this without you, the system two thinking conscious thing, really being aware that he’s doing it.

And you noticed also that he was, like, in a library setting. So he’s sort of associated with all of your long term memory. So he’s got all of that, and that’s what allows him to work so quickly and so effectively. And when those people said 10¢ to me, he served up the answer, 10¢. And this guy, system two, was just like, yeah, 10¢. Like, I’m not going to think about it. I’m not going to check that answer. He could check the answer, but he doesn’t want to because he’s lazy and fair enough.

The goal for us should not be always to use system two, but it should be for us to know when we need system two versus when system one will be fine. So the idea is, hopefully, you want to optimize, allow system one to handle everything that they can handle and only hand off to system two things that you need to.

Now in my travels, I came across these two characters. And something that I thought about later was that they almost embody system one and system two together. So the question that I asked them was like, how long does it take for the earth to go around the sun?

[VIDEO CLIP STARTS:] DEREK MULLER: How long does it take for the earth to go around the sun? What do you reckon, Charles?

CHARLES: It was over twenty four hours? Obviously, day. Yes. [VIDEO CLIP ENDS:]

I asked the question to system two. He was like, I don’t want to do it. Like, what do you think? You know? System one over here. What do you think? He’s like, obviously, a day. It sounds good. Yeah. A day. Sun comes up, sun goes down. Yeah. And then what did he do? He just spat it out. He didn’t stop to think about it. He was like, obviously, yeah. It’s a day.

What was amazing about this conversation was this conversation moved on, and I asked I had a whole set of questions that I asked them. I think at this point, I was asking them, did dinosaurs live at the same time as humans or something silly like that. Anyway, that’s what I was doing. And then I noticed a pause, and I was looking at his face, and you could see that he was thinking. You could see that system two had snapped into action. So I just didn’t interrupt.

The Limitations of System Two Thinking

Hey. The earth doesn’t take one day to get around the sun. It takes like a year. You see the shock on his face. The whole time, he was just running system one.

And then at the very end, it was like, woah. Hang on. Wait a minute. All right. The thing about System two is that it is very limited.

So this paper came out in ’55. It’s a classic. The magical number seven plus or minus two, the limits on our capacity for processing information. So the idea here was really that you can only handle seven new things in your working memory that system two can deal with at one time, plus or minus two. It turns out over the years, that estimate of seven has been revised down to about four.

And there’s a test you can do to kind of assess this and figure out, okay, what is my span? How many pieces of information can I just work with at one time? So the way this test goes is I show you some numbers that you can look at briefly, and then I take them away. And what you try to do is say each number on the beat. There should be like a beat going.

I know how we’re going to do that, but you say each number, but you add one to it. So increment each digit by one after the numbers have disappeared. I think I’ll do it for you so you guys don’t feel weird about this. But I just wanted to demonstrate how we can show that this span is quite limited.

So hopefully I get this right because I know what the numbers are. Okay. So here we go. There’s some numbers. Six, five, zero, two.

Right? Okay. That wasn’t too hard. You can make the task harder by instead of adding one, you add three. And instead of having four numbers, can go to five or to six to really push yourself and see what happens.

Cognitive Load and Pupil Dilation

What’s amazing about this task is that while you’re doing it, while you’re really activating system two, there’s physiological responses that take place. Your heart beats a bit faster, your skin gets a little sweatier, and interestingly, your pupils dilate. So when you’re really thinking hard, when you’re using that system two thinking, your pupils are actually expanding just a little bit.

So I got a super macro lens and some friends, and I showed them numbers and then got them to do the add one task.

[VIDEO CLIP STARTS:] DEREK: And so we can try to see if you can spot the growing in the pupil of this eye while he’s doing the task here we go 43972 SUBJECT: Okay it goes a little bit five four zero three [VIDEO CLIP ENDS:]

I feel like in the middle, when he’s like in the middle of the numbers and he’s still got to remember the remaining numbers, then it’s like, you know, like it’s just and then at the end, it sort of starts collapsing back. He’s like, okay, now I can relax a little bit again.

So it’s interesting to see just how effortful this is. This measure is known as cognitive load, or another way of thinking of how much mental effort you are investing in something. And we can break down how the cognitive load is being expended into three categories.

So there’s intrinsic cognitive load, which is just the amount of cognitive load you need for a particular task. So in that case, you know, if I’m giving you six numbers, it’s going to require a large intrinsic cognitive load. Or sometimes when we’re doing something in physics, even something simple like f equals m a, which to, you know, physicists and stuff kind of is not that hard, but for a new student, each of those things is new and sophisticated. It’s a complex concept. And so in fact, even that, just teaching that has a large intrinsic cognitive load.

There’s also things that are totally extraneous. These could be things like, I don’t know, maybe there’s someone chewing next to you or your seat’s uncomfortable. There’s just distractions and things around you that are taking away from your ability to focus. So extraneous cognitive load is not great.

And then finally, have germane cognitive load, is can you actually use your attention, use system two to do some maybe thinking about thinking, like observing kind of what you’re through as you’re thinking through a problem, or noticing things that might be helpful patterns for you to use later. So germane load is great if you can get it.

Chunking and Expert Performance

Now when it comes to this is another really classic study because one question you might have is if we have such a limited system too, how do we accomplish anything? And if you go back several decades, people were wondering what makes a chess master a chess master? Do they have really high IQ? Do they have great spatial reasoning? Do they have a really large working memory span, which is the kind of thing that we’ve been looking at?

You know, what makes them unique? And these researchers got together and they did a study where they showed some different level chess players from beginner, intermediate, and master. I don’t know if they were grandmaster, maybe just a master, but someone who’s very experienced in chess. They showed these different people a chessboard like this that was basically stopped in the middle of a game.

They showed them this chessboard for five seconds. They had it in front of them. Chessboard’s there. And then they covered up the chessboard, and they had another blank chessboard beside it. And they said to these chess players, okay, you’ve seen it for five seconds. Now put the pieces on the board exactly as they were in that chessboard that you just observed.

When you ask novices to do this, they’ve only seen it for five seconds, when you ask the novice to do this, they will get about four pieces correct after looking at the board just once. In further iterations of the study, they were allowed to look at it for another five seconds and then put some more down. And so they were sort of counting up how many it took. But just on the first glance, a beginner could do four pieces.

The grandmaster could do 16 pieces after one glance, after just five seconds of looking at it. Why were the grandmasters so good at remembering where all those pieces were on the board? It’s a lot. It’s because of a phenomenon called chunking, which is where you can see things that look like disparate bits of information, separate pieces of information, but you can actually see them as one thing.

So in the case of the example I gave you earlier, if you reverse the order of these numbers, you get 1945, which is no longer four numbers, it’s one thing, the year that the Second World War ended. Right? And so that makes it much easier to remember. And this is true for basically everything we come into contact with. We’re not normally dealing with it in its most basic chunks.

So words, you know, you don’t have to parse out every letter. Even phrases, you know, they have meaning and you can think of them as one thing. Or equations like this one for a physicist, those become one thing. Even the Schrodinger equation, yes, for a lot of people in the room, I imagine that’s just a single entity in working memory and you don’t have to, you know, you could write that close your eyes and write it down again.

So the point is that the more we get experience with things, the more we practice, the more we interact, the more we use system two to work through problems, the more we develop this long term memory. And that long term memory allows us to chunk the things in our world, and that allows us to deal with much more complex situations.

So that’s exactly what’s happening with a chess master. I have heard a lot of people say this. That class really taught me how to think or that teacher taught me how to think. And I think that’s more of a question than than it might seem at first. I mean, at first, it might seem like, yeah, that that feels reasonable. Like, I think a lot of us have this experience where we feel like, oh, yeah. After that, I really learned how to think. I learned how to reason.

But if you go back to this chessboard study, the next thing they did as part of the study was they rearranged the pieces, same number of pieces as before, but now they positioned them as though they would never occur in a regular chess match. They just put the pieces all over the place at random. You would never ever see this in regular chess match. And they tested again the novices and the masters. And now, the masters did no better than the novices.

The thing that made them so good at remembering where the pieces should go was that they had seen a lot of chess boards. They’d seen a lot of configurations just like that. And if the pattern wasn’t there, then they were no better off than the novices. The exact same situation.

There Is No General Thinking Skill

My argument here is that there is no general thinking skill. There is no general problem solving skill. What there is are these complex webs of long term memory.

This is what we build up over our lifetimes. So a physicist is not necessarily going be a good chess player. And a chess player is not going to be a good physicist. This applies to all domains, I would say. I mean, you can make arguments for specifics where you could say, such and such skill will transfer. But I think in general, the idea is an expert in one field is not an expert in another.

And it’s just because of this. It’s because that long term memory that your system one is working with is really specialized to reflect the experiences you’ve had and to have recognized all the patterns in those experiences to allow you to chunk new situations when they come to you.

It’s so powerful when your system one is this really complex web that you don’t even feel like you’re thinking. In the same way that those people said 10¢ to me, if you ask Magnus Carlsen, what is he doing when he plays chess? Most of the time, I know what to do. I don’t have to figure it out. Right? He’s saying, don’t need system two. System one is so well developed that it can solve any problem at a glance.

For him, chess is a game of recognition. He recognizes the board in the same way that we recognize faces or physicists recognize a physics problem. Right? That’s the way we get to be really good at things. So really what we want to be doing in education is using system two’s resources very carefully and repeatedly such that we store information in long term memory to allow our system one to do things that effectively feel automatic.

Implications for Education

So what are the implications of this for education? The first one is that we should eliminate extraneous cognitive load, which is fairly obvious. So you should have a comfortable seat. You should be able to see the board. Everything should be legible. The sound from my microphone should be pristine and pure. I shouldn’t have an accent. If I have an accent, it makes it harder for you. That’s extraneous. Just trying to think, what was that word? Stuff like that. Having subtitles sometimes helps. My wife loves to watch all TV shows with subtitles. So now that’s what I do too.

How can we limit intrinsic cognitive load? This is point number two. We don’t want to overfill what you can handle. And my guess for you is that the reason why a lot of physics lectures fail, sometimes if they do, is because the professor doesn’t limit intrinsic cognitive load enough. They go through too much novel material in the one lesson and it’s just too much for anyone to handle in terms of their working memory. Once you’re overloaded, there’s basically nothing you can do. Your system two doesn’t know how to deal with that.

So one thing I would say, you know, when you’re teaching is you have to try to obviously start where your students are, and you have to keep the work kind of bite sized because as soon as you start introducing four, five, six novel concepts in one lesson, you’re going to lose people.

Another way to think about limiting intrinsic cognitive load when it comes to music is get people to play songs that they already know. This is why we do that, I think. Right? Learning to read music takes time and effort. So the first things you’re going to do with students is get them to play songs they already know. Then they know what the rhythm should be. And so even though, like, yes, in theory, they should know how to read these notes and they should know what a quarter note is and understand the beats.

I can tell you as someone who played French horn for several years, I didn’t really know how to read music for at least a couple years after I started playing. Like, the music told me what fingers to push down because I could have understood the notes. But in terms of rhythms and things like that and really being able to sight read music, I think it took a couple years. So I think this is one of those things where we limit intrinsic cognitive load by getting students to play songs they already know.

Another way to limit intrinsic cognitive load is to slow things down. So anyone who’s practicing music, for example, like this harpist, who’s a friend of mine, she describes her practice like this. You can practice everything exactly as it is and exactly as it’s written, but at just such a speed that you have to think about and know exactly where you are and what your fingers are doing and what it feels like.

So she’s really talking about using system two there very slowly, deliberately, effortfully thinking through every little thing that she’s doing in those moments. And then when she does that enough times, enough repetitions of this, you get something that looks like this. Yeah. So good.

# What Everyone Gets Wrong About AI and Learning

The Illusion of Effortless Expertise

Right? Every time I see superhuman performance, the thing that I’m thinking about is how they have used system two very slowly and effortfully to build up that structure in long term memory that allows them to do things that look so amazing and effortless.

It’s also at this point that I bring up this idea that discovery learning can be dangerous. When I was going through school, the dominant paradigm in education was something called constructivism. The basic tenet of constructivism is that students are active constructors of their own knowledge, something I do not disagree with.

It’s basically what I’m saying up here that system two is about being active and effortful in how we engage with material, how we practice. The problem is, I think some people didn’t know how to implement teaching for this constructivist paradigm. Their thinking was if the students need to be active, then telling won’t work. But that’s not true. I’m telling you stuff right now, but I bet you’re still being active and effortful in your brains, is the place where you need to be active and effortful.

So that was the problem with constructivism. And it led to some classrooms where people really pulled the scaffolding away from students, I would argue too early. So it was “we want to teach problem solving. Here’s a problem. Solve it.”

You don’t know how to do that? Figure it out. Construct. This is just my personal pet peeve. I have some hang ups about constructivism.

The Balance Between Guidance and Discovery

But this is where I say discovery learning can be dangerous. I can see flip sides of it. For example, if you go to a new city and you don’t use a GPS, it’s really hard to figure out where you have to go. But if you do it and you use your system two to figure out and follow every street and look at all the signs around and read these things, you’re looking really carefully at everything.

At the end of that, you’ll probably be able to retrace your steps. If you use the GPS, it’ll be much easier for you. You’re offloading processing from system two. You’re offloading it to the GPS. Can you find your way back? No. You’re going to need the GPS to find your way back.

I think there has to be some balance in education between the kind of GPS guidance and the internal guidance. Ideally, I would love to see a gradual phasing out of the support to allow the person to figure out the directions on their own. This has been I think well backed up in research of things like the worked example effect.

And this is even a further idea of sort of fading out the assistance as you go. But the idea of first giving someone, “Hey, here’s a problem and here’s how you solve it. Here’s a problem that is not quite at the end. Here’s a problem partially done and here’s one for you to do from start to finish.” Right? But it is this idea of like, I’m going to reduce your intrinsic cognitive load.

I’m not just going to give you this problem at the end and say, figure it out. I’m going to give you this scaffolding because I know that what your working memory is is very limited. That system two has very limited resources, and I’m not going to tax them with you trying to think about all sorts of things at the same time. I’m going to give you this assistance. So to me, this is the scaffolding that helps system one move through and learn new problems.

I think again, this is kind of a problem we have in complex domains like physics where to the physics professor, everything’s perfectly clear because their system one is so fully developed. But to a student, it’s not. So this is the expert-novice divide. The professor can’t see with the student eyes what that problem looks like. Right? Only the student can see it that way.

The Importance of Mastery

Okay. So what else do we need to do in education? We need to repeat that effortful practice until we achieve mastery. Why is mastery so important?

Because when you show a skill with mastery, that means it’s now a system one domain. For example, for all of us here in the room, our basic times tables, I imagine, are mastered. They’re in system one. You don’t have to think about it. What’s so useful about that?

When you come to do other problems which require that information, you’re not going to overload system two when you do it because all that stuff is automatic. So that’s the key. That’s how we’re going to get better and better performance is by building up more and more capabilities which are automatic.

If you never get to the level of mastery, then moving on is going to always cause you problems because you’re going to be having to use extra areas in your working memory to deal with thinking about that first before you can think about the other thing.

Increasing Germane Load

Next we have increased germane load. So how do you get people to engage in really active thinking, extra level active thinking? One thing that was tried, for example, with this problem. Okay. So the bat and ball problem is actually from a test. I think it’s called the cognitive reflection test.

And I think there’s actually three questions on this test. The bat and ball problem is only one. But this test has been given out to thousands and thousands of people around the world, including incoming students at very elite colleges. And when you give this test, you typically find that about ninety percent of people get at least one wrong. At least 90% get at least one of the three questions wrong, and they’re all kind of like the bat and ball problem in a certain way.

So then what some researchers tried was they thought, instead of printing out this test in like really clear font on nice white paper, we are going to print it out in like a terrible, very hard to read font, maybe photocopy it a few extra times so it gets really difficult to read, and crumple up the paper or something, and then we’ll hand it out. And what happens? The error rate drops to 35%. Making the test harder to read made students more likely to answer it correctly. Why?

Because they didn’t have that easy like, okay, I can read it and an answer immediately comes to mind. They didn’t have that experience. Instead what they had was, oh, this is challenging to read, which kicked their system two into action. And so it was activated and was able to find the correct answer. So I’m not suggesting this is a good strategy, just making things more confusing. But I will say it can be effective.

And I honestly, I feel like I’m seeing advertisers apply this sort of principle sometimes. In the old days of advertising, the 50s, 60s, I feel like it was all about having a jingle, showing what your product did, why it was better than your competitors, and then having this great jingle that would just get stuck in your head and you’d all remember the name of the brand and why it’s better. And then these days, I think we’ve got so bombarded by advertising that system one has got great at tuning it out. And so now a new strategy I see some advertisers employ is let’s confuse people.

This was a billboard that I saw in Sydney near the beach, and I was definitely intrigued. The power of un, unexplained, television, uncost the earth, unpay more, unspend more. It didn’t really tell you anything. It’s like, how is this good advertising? A few days later at a bus stop, I saw this.

Unexplained. With un, there is no stress, just unstress. No hassle, just unhassle. With un, you can undo what you did. You can un drive through the car wash with the window down or un break dance in front of your teenage son and his mates or un flood the downstairs laundry that now doubles as the pool room.

Un makes life relaxing and unreal. Un your life. Be happy and live for now. Don’t worry, un worry. What is this an ad for?

Insurance. Yes, this is an insurance ad. But how lame are insurance ads? And this is pretty great. They really got me.

The Role of AI in Education

So what role is AI going to play if you believe in this picture of education that I’ve painted? I think the positive role that I see for AI is that it can provide timely feedback and that’s essential. When you are learning any skill, you’re going to go through the repeated rounds of practice and you’re going to need immediate feedback in order to learn. When you’re playing the harp, if you miss a note, you know immediately. That’s great immediate feedback.

If you’re practicing tennis and your shot’s in or out, you probably know, if it hits the net or goes over, you know right away. So these are really helpful in terms of training your brain. There are some different industries and some different professions where people seem like experts, but they’re actually not. Stock pickers is one example because you may get some feedback that like maybe this was a good decision or a bad decision. But honestly, the stock market is, at least in the short term, incredibly random.

And so that feedback is never very informative for what you should do the next time. This has also been shown with political pundits and some economists, anyone who’s looking to predict these future trends which are much less reliable in terms of the validity of the environment. So one thing I see AI doing is providing this timely feedback which will really help as we saw in the first case with Sal Khan and his son.

The Danger of Reduced Effortful Practice

But the thing that I’m really worried about is how AI has this opportunity to reduce effortful practice. I have four kids who are eight, six, four, and zero. And I worry about them that, if they’re going to be – will they write an essay? Will they write a hundred essays? If there is a generative AI that can write for them, what forces them to practice crafting those sentences? And if they don’t craft those sentences, what happens to their brains?

The argument here is that you get good at your command of the English language. You get good at being able to speak in front of people, at being able to express your thoughts in writing by doing it again and again and again and again. And you should suck at the beginning. And you shouldn’t let that stop you. And you should keep going and going and making slight tweaks and improving and getting feedback and getting going.

If they never do that, I really worry. What gets into system one? Do they have an amazing network of connected knowledge that they can draw on? Do they have things that are automated? I fear that they won’t.

How do we force people to have to do that painful effortful work when there’s a magic machine that’ll do it for you? That’s a big concern. What about drawing? You know, if you can just ask it to make a picture of whatever you like. The bat and the ball was AI by the way. I can’t draw. But again, like, what will happen to people’s artistic abilities?

So this is I think my biggest concern is if it prevents us from going through this painful, effortful process, which is the core process of learning using your limited system two resources to engage with things and practice again and again and again even when it’s hard, even when it doesn’t feel good, even when you’re not great at it. That is my big concern.

Revisiting the Big Questions

I want to come back to these two big questions. Why aren’t people learning and why haven’t the education revolutions materialized?

To the first question, I want to be a little bit more generous to the people we saw at the beginning of this talk. Our brains are designed to help us be effective in this world, which means finding food and shelter, finding a mate, integrating socially so that we’re not ostracized, being able just to hang out and have fun. Like, all of those things are what we should be doing. And maybe it shouldn’t be such a surprise that people don’t know what elements it takes to make water or even that CO2 could pose an existential threat.

I know everyone in this room will agree like that is important. We should all know that and we should figure out how to work with that. I guess what I’m saying is I think it’s understandable that a lot of people don’t focus on that. Don’t know that. Don’t think about it. It’s not part of the world that they exist in just because they’re so busy with social media and Instagram or whatever because that’s about connecting with other people. So I want to be a little bit generous there.

And when it comes to this question of why haven’t education revolutions materialized with film and TV and radio and computers and MOOCs and now AI, Part of me wants to say, I think we might have already found the best thing. Being in a room with other people, other learners, a teacher, and some time to talk. Education is a social activity.

You know, people care about other people. I think that the tech hype comes from a place of believing that the problem of education is not being able to get the information to the student. That’s not the problem. It’s not the problem now, and it wasn’t the problem a hundred years ago. When you have books, I mean, the information is all there assuming people have access the students had access to those books.

And yet, they’re probably not going to learn very much unless they have a great teacher, unless they have a group of like minded peers to go through that with them, unless they have a reason to do it. An analogy for you. You know, the world is full of heavy objects, and yet most people are not ripped. Do you see where I’m going with this? The world is full of fields.

# What Everyone Gets Wrong About AI and Learning

Not many people running on them. There’s plenty of ways to get exercise, but obesity is a huge problem. I hope the analogy I’ll get there. I think about teachers because I was one. I kind of am one.

I think about teachers a little bit like personal trainers. Like, the gyms are there, but unless there’s someone who you’re meeting there and you’re held accountable to and someone to say, “Another one.” And then they go, “Another. Give me another rep. Keep going. It’s burning. Yeah. Keep going.” You know? Someone to tell you the homework and someone to hold you accountable and someone to really energize you and maybe a group of other people who are doing it at the same time and we’re all going to this together. That’s when you see results.

This doesn’t just happen in a vacuum. So I guess that’s my big thought that teachers are some of the greatest people in the world doing an incredible job of connecting with students and creating communities of learners, and that’s what it’s about. It’s about that social experience. It’s about getting excited and holding people accountable and forcing them to put their reps in.

So for me, that’s why none of these technologies are ever going to revolutionize education. Thank you.

All right. Thank you, Derek. That was a fantastic talk.

Q&A Session

All right. So now we have some time for questions. For those in the theater who would like to ask a question for the speaker, we have a microphone set up there on the stairs. So please make your way there if you have a question. We do particularly encourage young members of the audience who have a question to feel free to come up and ask.

And just a reminder for anyone and all folks either online or who are in the theater asking questions, please keep your questions short and sweet to give us enough time to answer everyone’s questions.

AUDIENCE QUESTION: Hi, Derek. Thank you for coming today. You kind of touched on it a little bit when you talked about your kids, but the question that I have is what do you think about using AI to answer your question when you’re working on a problem as opposed to looking up for the answers yourself in a maybe more complicated way, like in the books or whatever?

DEREK MULLER: I think that as long as the AI is sound and a lot of it is reasonably sound at this point, right? Then I would say I don’t think there’s a difference really. I think you may as well just talk to the AI, assuming the AI is reasonably knowledgeable and it’s not going to hallucinate things like that. But in terms of just “is it better if it takes me longer?” Like, no, I think that’s strictly a question of efficiency. And at this point, I wouldn’t worry about being more efficient. I think it’s fine.

AUDIENCE QUESTION: Okay. Thank you.

AUDIENCE QUESTION: Do you have any suggestions for how to get these insights about education enacted into policy? Is that something that should come from teachers or from the government or from public? What are your thoughts?

DEREK MULLER: My thoughts are that policy is very challenging. In Australia, recently, they’ve implemented what they call a direct instruction policy, which is kind of what I’m talking about, which is that telling is not wrong, which was kind of taboo for a few decades.

I think education is a really hard field. It’s a difficult field to research in because there’s so many variables and also because the researchers are not disinterested in the outcomes. Typically, researchers want to create something good that improves people’s schooling, but that also encourages them to p-hack and do all sorts of other things and probably without even being conscious of what they’re doing. And it just leads to a level of research that I feel like does not hold up to scrutiny.

Having said that, I feel like there are some strong leadings from the research like the things I was showing tonight that I think do point us in this direction of things we clearly need to do. We need to build up that long-term memory. We need to have this effortful practice. We need to push people beyond their comfort zone.

So I don’t know exactly where the movement comes from, but there are education researchers. So hopefully it comes from there. As I say, it’s happening in some places, it’s happening in Australia. Hopefully it’ll spread throughout the world.

AUDIENCE QUESTION: Hello. I’m curious because you mentioned art, and I found that many people that choose to go into the arts aren’t necessarily doing it for money or for a reward at the end. So do you think that—and I’m reluctant to use the word fear—but do think that our fear should be that people start chasing art or the people who observe the art no longer care of the origin of where it was made?

DEREK MULLER: I’m not entirely sure I understand your question. When you mentioned generative AI, how people still do things when AI can do it for them, people who chase art already are doing it without a necessary promise of a reward.

I think that’s great. And probably that’s really encouraging that people are just into making art for the sake of art. And so maybe I should be less worried about that. I mean, like I personally for my kids, I’m just worried like, are they going to learn how to write?

If you look at what’s happened to our handwriting, all of us, because we don’t really handwrite. So what do you expect to happen? Well, it’s going to get a lot worse. We’re better typists than we are handwriters. And what if we become not that great at writing at all?

I think there’s one thing that I didn’t quite mention in the talk, this idea that it is that vast structure that I think allows experts to have great insights. I was thinking about this yesterday, almost something I put in the talk. It’s about how you see a lot of great advancements in physics coming from young physicists. And part of the reason for that is I think when they build their networks, the teachers who teach them have slightly different networks that follow the historical progress of the science.

And at some point, you know, there were some dead ends that didn’t work out in physics. And those are still in the minds of the teachers. But they’re not in the minds of the students because they don’t get taught to the students. And I think that frees up those structures to find new possibilities, which is kind of why I think Einstein maybe was open to this idea of, well, maybe space and time are malleable. But for the people that came before that, their structure just wouldn’t have permitted that kind of flexing that new idea to come around.

Anyway, I think it’s really important that you have a very well-developed structure of writing in order to be able to express yourself and express ideas. If there’s a machine that does it for you, maybe we lose all kinds of important insights.

AUDIENCE QUESTION: Hi, Derek, huge fan. So you kind of talked a lot about System One and System Two and how System One is quick and it doesn’t make much effort to think. And I have observed that with AI coming over, we are using System One quite more often and keeping System Two more on rest. And I feel like System Two is designed to be more curious, it takes the process slowly and understands the problem to solve it. With System Two being on rest and it not being able to be much curious, and I think like curiosity and learning goes hand in hand, how do you feel about the learning evolving? Or let’s say, my question is like if you had AI at the age of 20, do you think Veritasium would exist?

DEREK MULLER: These are good questions. If AI was around when I was 20, would Veritasium exist? Don’t know. That is a crazy counterfactual. I don’t know. Hopefully, we’re still all encouraged to follow the things that really intrigue us and engage our interests.

I think as the first gentleman who talked about, can I use AI to look up or aim to look it up in a book? I mean, it doesn’t matter. So in some ways, I think AI provides a speed up that way. And even these days, I haven’t really explored that area very much in terms of the things I create. But I’m intrigued to think about what could I create in future using AI.

And maybe it’s possible that I can create some things that are really incredible and powerful. I don’t know. I feel like AI is just tainted a little bit because there’s so much like AI sloth out there. But I try to imagine a world in which you have really high quality AI and maybe it’s amazing. So I don’t know.

It’s a short answer to your question, but I will experiment and maybe you’ll see some Veritasium that sort of has AI in it and you can tell me whether you think it’s better or worse.

AUDIENCE QUESTION: I look forward to that.

AUDIENCE QUESTION: Hi. I was wondering how you’d solve the problem of scaling because like you mentioned the best way for teaching would be establishing a personal connection, right? Like that’s easy to do when I’m teaching like 10 people. But if I scale that up to 100, it’s harder for me to build a personal connection with 100 people and having them be accountable and be encouraging to each one of them individually. How would you approach that problem? And I think that’s somewhere where AI can be helpful.

DEREK MULLER: Yeah. Or how do you scale a personal trainer? How do you scale a plumber? How do you scale an electrician? You don’t. You just have lots of them. And I think that’s the answer. I think that’s the solution.

I just think people are always going to complain about education. People are always going to be like, “They don’t know anything.” Whatever. I feel like it might be close to the optimal. You know what I mean? And that’s why it’s so hard. I am more optimistic about us disrupting say, health care in the US than I am about us disrupting education. It’s just so hard. I don’t know.

AUDIENCE QUESTION: I guess that just comes down to resource management, I guess, because you can’t have enough teachers.

DEREK MULLER: Yeah. I mean, we do have enough teachers in a way. And the goal should always be to get more of them and to make them better. And I think that’s a system problem, but we can do that. I think that’s probably the way to go.

AUDIENCE QUESTION: Hi, Derek. Earlier you touched on the way that people learn like an effective way that people learn in the classroom where it’s like you have a framework and you slowly take away line by line of the equation. Kind of within this context, how do you think or where do you think the place of AI would be within this way of learning, if at all?

DEREK MULLER: Yes. It’s a great question. I think AI can offer lots of scaffolding. You can say, “Hey, can I get a hint?” So there’s a remarkable capability for it to sort of fill all kinds of gaps in learning. Or like you could say to the AI, “Hey, I need to learn about this period of Canadian history, ask me 50 questions about this.” Like, what a great tool to be able to have if you can use it effectively, right? So all of those ways that you can use it as an educational aid, I think will be great.

I think the place where it won’t be great is where it allows you to do the work without doing the work. And it’s that work which is essential for learning. So that’s my big concern.

AUDIENCE QUESTION: Hello. I’m just going to say, the unadvertised, I still think it’s one of the most effective ones. I’ve never forgotten it. My question was, it’s like with the onset of chatbots and things, it’s become so easy to get assignments just done without work, right? But it’s like, I’ve personally never used them because I feel it’s against my pride to, you know, but I mean other people in post-secondary, right? It’s like people just say, “Yeah, no, I use Reclid” and it’s like, well, I mean, do you think it would be better to just outright remove it from the early stages of education and for only to come back for when you’re older, when you’ve developed your effectiveness at work and stuff?

DEREK MULLER: I absolutely think there’s going to have to be some of that. I think it’s a little bit like, when I was at school, we had TI-82 calculators, maybe TI-83s. And we were allowed to use them on a portion of the exam and then there was a portion of the exam where we weren’t. And I think it’s the same thing. I think when it comes to writing, there will probably be some writing where you can have an AI aid and somewhere it has to be no AI.

# What Everyone Gets Wrong About AI and Learning

And I totally think that’s essential, especially at the lower levels of schooling. It’s like we’re going to have to have writing sessions for hours in class just to get people that experience of going through this challenging process. Yes. All right. Thank you.

Q&A Session

AUDIENCE QUESTION: Hi, Derek. Thank you for being here. I just wanted to ask to touch on the previous questions, I think that you were going to make Veritasium either way, to be honest. And I fear that many of the people in this room know how to break their brains and get dopamine out of it. So how do you think we can effectively get people excited about breaking their brains on their own? Because I think that’s the key thing, if I can share that.

DEREK MULLER: What do you mean breaking their brains?

AUDIENCE QUESTION: Well, I mean, I feel like everybody here might understand, like when you don’t understand something, it’s exciting. A lot of people when they don’t understand something, it’s not exciting. So how do you think we change that?

DEREK MULLER: That is hard. I feel like that’s the existential question. And this comes back to me asking random people on the street, you know, what elements does it take to make water? What’s happening to CO2 in the atmosphere right now? It is not hard for me to find people who don’t know the answers to those questions.

Like most people don’t know the answers to like most of these simple questions. And so, I don’t know that you can ever take someone who is just not interested and make them interested. Kind of like people are going to be into what they’re into. And this is where I just have compassion for, like we’re kind of not as a species, we’re not evolved to seek out theoretical physics. And yet a small tiny minority of us do, but then we shouldn’t look around at everyone else and be like, what are you guys doing?

I get it for this room, but I don’t get it for the public. So I don’t know. That’s my thought. Like maybe there isn’t a way.

AUDIENCE QUESTION: Tough question.

DEREK MULLER: Yes, it’s a tough one. Thank you.

AUDIENCE QUESTION: So you’ve migrated from short form video to longer form. And so my question is, is there a sweet spot where you get content overload? You know how you’re saying there’s only so much content you can pack into a lesson for example. So I teach. So I’m just curious, is there like a minute or content limit? I love how deep you are able to go into stuff.

DEREK MULLER: Yes. I mean maybe sometimes we do overload people with stuff honestly. Like YouTube has algorithmically pushed us. I don’t know if it’s apparent or not, but YouTube has algorithmically pushed us toward longer videos. So in the early days, the videos were three minutes, two, three minutes and now they’re 30.

And why? Because if we make a good thirty minute video, it gets shown a lot more. And that’s just the name of the game. So I know that sometimes as we approach sort of fifteen minutes into a video, twenty minutes into video and we’ll start putting integrals and derivatives and stuff into the video. And I’ll recognize that like at this point, we may be losing some people and maybe this is too much.

But then it’s there for the people who want to keep watching. And I feel like if you’ve watched twenty minutes, why not twenty five, let’s keep going. So I don’t know that this is also like kind of a business question for us, which is that like, what do you put in the first minute? That’s pretty essential. You don’t want to start with crazy math.

What do you put in the first five minutes? If the writer that I’m working with comes to me and says, yes, I want to go through this whole derivation at minute twenty-five, I’m like, let’s do it. By that point, the only people around are still the true believers or another thing that’s great about videos, if someone is feeling like they don’t get it, can always go back, rewatch, they could pause it, they could go take out a textbook, they could write some things down. So hopefully the videos can still be effective for a certain population.

But I do recognize that we may lose people sometimes later on. Thank you.

AUDIENCE QUESTION: What’s your favorite video you’ve ever made in your life?

DEREK MULLER: It’s such a hard question. I do feel like every video is like one of my children. Like it’s very hard to love some more than others. The one that’s done the best for us is on Gödel’s incompleteness theorem. So I wonder if I would say that one. Also, there’s the shade balls one on the reservoir. Have you seen the shade balls one?

AUDIENCE QUESTION: Yes. I was going to say the radioactivity one is my go to.

DEREK MULLER: Which one?

AUDIENCE QUESTION: Radioactivity, like the most radioactive place on earth. It’s an oldie but goodie.

DEREK MULLER: An oldie but a goodie. Yeah. There’s one that I really like, which is about – it’s only like four minutes long, but it’s about the most common cognitive bias out there. And I just feel like it does such a good job of hitting on what science is all about without really talking about it in a sort of boring methodological way. Like it’s a really concrete way to think about it. So I was excited by that video. Thank you.

AUDIENCE QUESTION: So if you find that in-person education succeeds because people are forced to engage with the content, they’re forced to put a conscious effort in. Why do you find that online edutainment succeeds when people can check out so easily?

DEREK MULLER: Yes. So I kind of think maybe they’re doing different things. Like if I was your teacher or lecturer, then there would be stuff to slog through, as well as stuff to enjoy. I think being an online educator, I get to just pick some highlights and make some exciting stuff that hopefully energizes people, shows them what’s possible. I feel like my job is sort of a combination of educator, also magician, also comedian or something, not that I’m particularly funny, but I think it’s all those things.

So I think there’s a place for that. And then there’s a separate place for like, now you have to sit down and practice and that can be, you know, tedious. So hopefully, one leads to the other, at least in some cases.

AUDIENCE QUESTION: So do you find that people tend to self-select for edutainment?

DEREK MULLER: 100%, yes. I mean, one of the reasons we could do what we do is because we aggregate from across the world. If I was ever on just a broadcast channel somewhere like the CBC here or ABC in Australia, like I could never go into the depth. As soon as I would put down a derivative on screen or something, they’d be like, no, no, no, no, no, you cannot. No, don’t do that. Let’s do an analogy to like skiing or something. You just lose all the rigor.

So the thing I’m so grateful for is the Internet came along at a time when I wanted to do that and there were people in the world who were hungry for it. So I just feel so incredibly lucky to be able to do this and go into depth and, yeah, reach a lot of people with it. Thank you.

AUDIENCE QUESTION: Hi. My question is, if you start talking to AI or, like, if AI was in a human body, talk like a human, acts like a human, and you become friends with it, would it count as social interacting?

DEREK MULLER: I think that is an amazing question. That’s a real question. Let me say this, that one of the best ways to learn that’s been shown has been if you have like a one-on-one tutor. There’s what’s called the two sigma effect that the performance of those students is two standard deviations higher than everyone else. So it’s a huge effect. And if we could replicate that effect by having something that is so natural and so human seeming that it would just feel like you’re there with another human, that might be really, really, really powerful.

I wonder. I feel like this is such a big question. Like how much do we need to know that it’s a real human there or how much could a fake human actually make us feel in the exact same way? And I think that’s what it comes down to. That’s such an amazing question. Thank you.

AUDIENCE QUESTION: Great talk. I’ll start here. So I definitely agree that revolutionized isn’t the right word. But I do think that all of those technologies have changed education in some way. Right? So like for me personally, I know that the accessibility that was gained by so much content being on the Internet has completely changed my educational journey. So my question is, if revolutionized isn’t the right word, what would you say is?

DEREK MULLER: Yes. I feel like a lot of technology just becomes a tool. You know, I think of them as tools. I just think the contrast is so weird when you think about how has the Internet changed our lives. It’s really profoundly changed our lives. And like it’s touched on education and yes, like independent learning journeys and so on, yes, the Internet is huge for that.

But in terms of like what has it done in the classroom, what’s it done to pedagogy curriculum, all that sort of stuff taking people up? I don’t think it’s done that much. And I just think it’s so remarkable when I think about the way all these technologies really have revolutionized things like smartphones, right? Again, totally changed our lives. Now we’re all on social media and we’re all depressed. But like, it’s done that. It’s had a huge impact.

So, I just come back to like what happens in education? Well, it just seems like these things become tools in the hands of educators. They’re tools to be used and exploited and so on. So I mean that’s the word that comes to mind for me. Thank you.

AUDIENCE QUESTION: If we didn’t completely remove AI from education, what tools do you think we would add and what tools do you think we would remove?

DEREK MULLER: I don’t know if I fully grasp it. Maybe what parts of AI would you keep in the classroom?

AUDIENCE QUESTION: Yes, what kind of tools are we going to add and what are we going to take away?

DEREK MULLER: I mean, for me, the big role of AI is that kind of like drill and practice type stuff, really getting people to do the reps, giving immediate feedback with whatever someone’s working on. I think there can be some powerful use cases. Yes. The real risky use case is the one where people use it to do their work for them, which is I think the one where it’s being used most unfortunately. So that’s the one that’s concerning. Yes, that’s the one we’ve got to get rid of. Thank you.

AUDIENCE QUESTION: Thank you for your wonderful talk. My question was hinted at prior and I’d just like to make it a little bit more precise. So I’m a grader for this quantum mechanics course and I graded my students’ midterms recently and the performance was extremely poor. It was abysmal. And this in no way was indicated by their performance on assignments.

And of course, I think it’s very reasonable to conjecture that this is due to the advent of ChatGPT, as students can just copy the solutions from ChatGPT, right? Whereas on the midterm, they don’t have that access. And I think it’s very reasonable to argue, oh, perhaps we should change the evaluation methods for classes to be more exam heavy, less assignment heavy. But at the same time, you can argue that exams aren’t the best methods of evaluation either due to performance anxiety and all these other things. So I’m wondering, what do you think is a preferred rubric or evaluation method for courses nowadays with the advent of AI?

DEREK MULLER: Yes. I mean, the first thing that came to mind is maybe the way that assignments are done now shouldn’t be the way they’re done in future. Like maybe assignments need to be done in the same way that midterms are done to remove that risk. Because I really worry that that is how you build that long term memory, that huge structure, that’s what we’re after. And if we’re not building that, then yes, like by the time you get to a midterm or the final, like it’s too late. Like the practice has to come in the lesson.

I can tell you that when I was lecturing first year physics at UTS University of Technology in Sydney, I had like stacks of cards at the front of my lecture that said in huge letters A and B on one and C and D on the other. And I would force the students to come and get these cards at the beginning of every lecture. It was a 400 seat lecture theater. And then as I would go through, I would ask a question maybe once a minute or once every two minutes and they’d have to hold up these cards.

And that’s really helpful for me as a lecturer to make sure that they are where I think they are. It’s also really helpful for them in that it keeps system two engaged. It keeps them like having to make that decision every few minutes, which stops them from, for example, falling asleep, which deactivates both System one and two. But it’s also really easy to tune out. And that’s also not helping.

Challenges in Higher Education Assessment

So the whole point of getting this back and forth even in a large lecture theater just with these cards, I mean, my attempt at making this interaction and getting people to be thinking all the time being really mentally active. So I’d say we have to do the same thing with this work that they’re doing, with the assignments. Like it has to be done. We’ve got to find some way to make an AI isolation booth or something. I will just say one more thing.

AUDIENCE QUESTION: I think though when you get into higher education, especially in Physics, the more the higher up you are, the longer it takes to complete a question. For instance, on a three hour exam, there are probably only like three questions because it takes an hour to do each. And so reconciling that in a university setting is much more difficult because you can’t just complete a question within the span of a few minutes and then give an answer, right? So could you comment?

DEREK MULLER: No, I mean, it’s a really good point. I just don’t see any way of getting away from the idea of there’s going to have to be assessments at some point.

It could be the midterms and the finals and the people are failing those because they’re not doing the assignments properly. It’s just something that people have got to come to terms with. And I know that sucks from like a structural and logistic perspective, but like there’s no other way. If you’re not learning the material, you’re not graduating – it’s just the way it is.

I thought that I was an amazing lecturer for my time. And I also thought that I gave very fair and very reasonable questions. And on the midterms, my students were scoring 40% on average. I feel that pain.

AUDIENCE MEMBER: Yes. It’s excruciating.

MODERATOR: All right. Well, that’s a good place to leave it. Thank you so much, everyone here and online. Let’s thank Derek again.

Related Posts

• Transcript: Maya Angelou’s Greatest Life Lessons on Oprah Podcast
• Transcript: The Next 50 Years: Humanity, AI, Power w/ Yuval Noah Harari
• How to Stand Out in an AI World by Thinking Better, Not Faster: Sol Rashidi (Transcript)
• Modern Wisdom: w/ Alex Petkas on True Story Of The Odyssey (Transcript)
• Reconstruction: Malcolm Gladwell Interviews Barack Obama (Transcript)