Karl Friston: We Have Paused Human Development (Transcript) 

I have to tell a story based upon this very sparse, very small data about what could have caused that. And what causes that is the state of affairs beyond my sensorium, beyond my sensory organs.

So to do that, it is thought— well, say it is— people have thought for millennia that the brain is effectively a constructive organ. It’s, in effect a statistical organ. It generates hypotheses about what could explain my sensory data, what I can sample. And then tests those hypotheses or predictions against the actual sensory evidence. And if it’s got the best guess, the best hypothesis, then you should be able to predict accurately and parsimoniously the sensory impressions from the world.

And this leads to notions such as predictive coding, predictive processing in the brain, the Bayesian brain. And you could even go right back to things like Helmholtz’s unconscious inference, the notion that we build our world from the inside by generating predictions of what we would see if we were right about what’s going on at the moment. And then if that’s an adequate or apt explanation for what we actually sample, then we know we’re on the right course. We’ve got the best guess inference to the best explanation. But if it’s wrong, then we know we have to change our mind and find a better explanation.

So that all rests upon this generative model. And the generative model, you can induce by just looking at the structure of our brains. And by understanding the structure and of course the dynamics and the physiology of the belief updating or the sense-making in our brains, we can work out those principles and then try to put them into neural networks. And that was certainly a story which pervades the last century, and even the past few decades.

I think one beautiful example here is the fact that if you’re trying to build or build upon what evolution has equipped you with in terms of your brain and your neuroanatomy, if you’re trying to build a good generative model of your world, namely a world model, then it should have the kind of deep causal structure that the real world has, which immediately speaks to a kind of hierarchy. So when people talk about deep learning, the deep that they’re talking about is a hierarchical depth, layer upon layer upon layer, which is exactly the kind of anatomical structure you find when you look at the brain.

So in response to your question on how much do we know about the brain, we know a lot of— in terms of neuroanatomy at a particular scale, and some of the principles that inherit from that knowledge find their way and have found their way over the past decades into deep architectures, deep neural networks and the like. So there are many, many other examples.

Defining Intelligence

PETER MCCORMACK: (00:11:06 – 00:11:12): May I ask a really simple question? Is there an agreed definition of what intelligence is?

KARL FRISTON: (00:11:13 – 00:11:15): No, I don’t think so.

PETER MCCORMACK: (00:11:16 – 00:11:17): How would you define intelligence?

KARL FRISTON: (00:11:17 – 00:12:24): I would define it as a process of inference to the best explanation. Divides into sense-making and decision-making. So for me, an intelligent artifact is something that you could call an agent in the sense that it has the capacity to act upon the world, and therefore has the ability to sample and query and question and exchange with the world in a particular kind of way.

And then you ask yourself, well, what is the most suitable way to gather data? And how would you measure that? How would you measure whether something is very intelligent or not intelligent? And in my world, that simply reduces to something that can be thought of as the evidence, the likelihood, the evidence for your model of the world.

Why AI and Human Intelligence Share the Same Language

PETER MCCORMACK: (00:12:26 – 00:13:05): I don’t know if it’s just me or anybody else listening may be confused and they’re not sure when you’re talking about humans or AI because there’s similar language you’re using that is language that’s used in both environments.

KARL FRISTON: (00:13:06 – 00:13:06): Yes.

PETER MCCORMACK: (00:13:06 – 00:13:25): And that is confusing me because I’m not sure if you’re doing that on purpose because we are heading to a world where they may be indistinguishable from each other or I just don’t understand that this is the language that has been used pre-AI in our world of analog beings.

KARL FRISTON: (00:13:26 – 00:16:03): Yeah, it’s a great question. I’m sure you know the answer to it, but we can unpack that. It is deliberate in the sense that there are very shared features between an artificial intelligence and a natural intelligence such as you and I, but it also speaks to the point that a lot of the principles that people use to build generative AI, for example, are— do inherit from an understanding of ourselves as human intelligence.

So it’s deliberate. Sometimes it’s oversold, sometimes, and that can lead to confusion. But I think the reason people use the same words in the context, say, of psychology, that they would now apply to artificial intelligence. It’s quite deliberate because you’re looking at the same kind of phenomena and possibly the same kinds of principles. I mean, I repeat, there are enormous differences between artificial intelligence and generative AI as we currently know it, and the more biomimetic natural intelligence that you and I have. But there are also many shared commitments and many shared principles.

I’ll just give you one example which will highlight this point, that if you go and look at what structural engineering development made large language models so potent, you could argue it was self-attention or attention heads. So why on earth did somebody call it attention? And the reason is that it plays a very similar kind of role to the way that attention would work through the lens of a psychologist or a neurobiologist. Attention in the sense of attentional selection, that you attend to a certain source of information or some tokens in the past that are relevant now for predicting the next token or what’s going to happen next.

So we’re very skillful at deploying our attention in this way and that way, and there’s vast literatures on attention in normal people and of course, how it might go wrong in psychiatry in terms of attention deficit disorders in the biological sphere. But the underlying capacity and the understanding of how attention works now means it’s a very apt word to describe the functionality of attention heads in transformer architectures?

Is Free Will an Illusion?

PETER MCCORMACK: (00:16:04 – 00:16:58): So in terms of the human and natural intelligence and in terms of understanding how our brains work, we see the world, we observe the world, and our brain is constantly feeding us information, and our brain is making calculations for us. I can’t tell you why the next word is coming out of my mouth, but it is coming out of my mouth, and I believe it’s me choosing to make it, but it’s happening at such speed that I don’t think I’m reasoning. I’m not reasoning the words that are coming out.

And so how does the brain do that? And therefore, is my kind of free will in this— is this some kind of an illusion, or is it real? Is there a part of the brain that collects the information, and there’s part of the brain that’s just me. It’s me, it’s Peter, and it’s taking that and it’s making the decisions.

KARL FRISTON: (00:17:00 – 00:18:27): I think the word illusion is probably quite a nice way to answer that question. You could argue that it’s all an illusion, everything, in this sort of Helmholtzian or predictive brain sense that, because we don’t actually— we’re not actually in touch with what’s out there, we can only infer it. Yes. Then everything is illusory to a certain extent, even to the extent that some people go as far as saying that perception is controlled hallucination.

I was caught up on that actually earlier this week by somebody that, well, hallucination is a false inference, whereas the kinds of inferences that we make, the illusions that we treasure and update continually in the moment, are much more tied and controlled and entrained by our sensory input. So they’re probably much closer to some kind of reality, but there is no reality from the point of view of you. You make your own reality by ensuring that your illusions about everything are good accounts of the sensory data that you have available to you.

So this would, I repeat, it could be regarded as philosophical as inference of the best explanation, abductive inference. That is constrained by our sensory sampling of the world. The twist here including the illusion that you are you.

PETER MCCORMACK: (00:18:27 – 00:19:23): Well, that’s the one I’m most interested in, in that we will have to a certain extent similar brains. Yours is probably more intelligent than mine, but we both agree this is a table. We both agree we’re in this room. You can see I can see you. We’re having this conversation. It may be that it can be defined as an illusion because it’s our brain interpreted, but we’re essentially seeing the same things.

The bit that I’m interested in is that, am I really having an experience, or is that bit the illusion? Do you understand what I’m saying? Is the difference is that my personal experience, that my subjective experience of the world, is that happening? Do I really have the personality I think I have? Do I really make the decisions I think I’m making? Or is the brain designed in a way to convince me that because it’s a necessity for me to survive?

KARL FRISTON: (00:19:23 – 00:19:24): I think the former.

PETER MCCORMACK: (00:19:24 – 00:19:24): Okay.

KARL FRISTON: (00:19:24 – 00:19:33): Yeah, I think the former, although there is an element that a lot of our sense-making about the consequences of our own actions is post hoc.

PETER MCCORMACK: (00:19:33 – 00:19:33): Okay.

KARL FRISTON: (00:19:33 – 00:24:53): But I still, I think, so just looking at the physiology and the maths of how we select those things to do and those things to say, I think you could still argue, no, it’s you, it’s private to you, it’s something that you chose, your path into the future. And I mention that simply because some people believe that to be an authentic and a true agent like you and me, and unlike generative AI and large language models, for example, that your world model includes a model of the consequences of your action.

And crucially, the consequences of your action lay in the future, which means that you have a special kind of world model that entertains a counterfactual future that has not yet happened. And of course, you can now choose your future depending upon what you choose to say or whether you choose to speak or not. We’re currently turn-taking, and that’s a really delicate act and skill. But you have to decide you could talk or you could not talk. So you have to evaluate what’s the most likely thing that I will do at the moment, and then you can select that course of action and then enact it, and then we can engage in conversation.

So just by saying that you have to select simply in virtue of the fact that we can only actually do one thing at a time. We are compelled to select one private future, which is private to you. I’ll never know what your futures were and why you chose and how you chose. Only you will do that.

I think the subtlety of your question though is, will I be aware of that? Is there another one of these deep layers on top looking at me, my brain choosing what to do next, giving me now a sense that I chose, in a sense, a sense of authorship and a sense of agency over what I choose to do. I think so. There would be perhaps not so much in machine learning or artificial intelligence research, but certainly in the philosophy of mind. And trying to understand the computational basis of that sense of agency and a sense of me-ness.

This is sometimes looked at in terms of metacognition. So it’s basically the ability to look at our own machinations, our own sense-making, by having an even deeper layer on top that crucially can intervene on what’s going on below. Not only can it recognize, “Oh, I chose that. That was me that chose that, and therefore I must be me in a mode of choosing.” But also reciprocally act upon that sense-making and decision-making.

And if one looks at the computational architectures of our brains, that turns out to be attention. The very fact that you can attend, act on the inside and switch on and off various layers of inference, layers of processing, means that you are now an author of your own sense-making through this capacity to actually attend. And there’s lots of really interesting philosophy about this, sometimes cast in terms of the ability to attend equips you with the ability to have this kind of covert mental action on the inside. And that is thought to render, give you exactly what you described, which was a sense beyond just seeing and knowing that you are looking, knowing that I am perceiving.

In certain philosophical spheres, this is known as the distinction between a phenomenology that is transparent. I just see something through the window, for example, versus something that has a certain opacity. So opaque perception, as if you were looking through an opaque mirror that you can turn the opacity up and down on. So now you know that you are looking at something through a window. For example, or through the glass simply because you can control it.

So your question is philosophically quite challenging, but would call for, interestingly, 2 aspects that you actually find in transformer architectures, a deep architecture with attention heads. So suitably deployed, you could argue that we are taking baby steps in the kind of computational architectures that would be necessary to have this kind of qualitative experience and sense that I am seeing. And furthermore, it is me that is seeing, or more specifically, to put agency back in, it is me that is looking. So to see is to look, and to look is an attribute of something that acts, and something that acts has to choose what it’s going to do.

Self-Awareness in Humans and Animals

PETER MCCORMACK: (00:24:54 – 00:25:04): And to be aware, to be self-aware, which is something that— do we agree that animals are not self-aware in the way humans are? Is there an agreement on that?

KARL FRISTON: (00:25:05 – 00:25:07): Well, we can agree here. I don’t know if—

PETER MCCORMACK: (00:25:07 – 00:25:15): Let’s agree here. So do we understand what is different about our brains that gives us that, that maybe animals don’t have?

KARL FRISTON: (00:25:16 – 00:30:40): Yeah. I was being slightly flippant there. It’s a great question, because it has enormous implications for the way that we treat animals. And of course, you can also apply the same thing to neonates and embryos. What point did they acquire a notion of selfhood, and become self-aware?

And if you sort of look at the kinds of tests you bring to the table to make that distinction, things like putting a dot on the forehead and then presenting or using a mirror to see whether, say, a monkey realizes that the dot is on her forehead, or whether the thing in the mirror is another monkey and not the self. So there are lots of really sort of simple and interesting tests, including things like the Sally-Anne test, testing for theory of mind, which would imply that I know it’s me and therefore I have a kind of self-awareness. On the basis of empirical study, it’s quite difficult to find animals that do have this kind of self-awareness that you and I share.

And I think the answer to why you and I are special than every other human lies in something that you were talking about before. You said we all share the same world in the sense that we both agree that this is a table. And when you said that, I thought, well, yeah, that’s one definition of intelligence in an intersubjective collective way, that we can agree that this is a table. And it suggests that there is some reality out there that we both share and that we’ve both made the same kind of sense of because our world models are sufficiently aligned that you can actually use the word table to communicate to me that you believe it’s a table. And I believe it’s a table, and we can confirm and make ourselves mutually predictable in that sense.

You could extend that kind of argument not to tables but to the notion of selfhood. So if we weren’t able to infer that there are things in my world that had the same kind of agency that you would see, say, in your mother if you’re just born, the kind of agency that suggests that they have an autonomy in the sense that they are choosing to do this as opposed to do that, then you never develop in your world model, your understanding of the world, the notion of an agent. For example, another, or mother in this instance.

And it may well be that that ability to explain things in terms of, “Ah, this was caused by another,” may be a precursor for the hypothesis, “Ah, perhaps I caused that.” So if a sense of self depends upon a sense of other, immediately you are saying that the kind of sense of self that you’re talking about rests upon being other things like me. It rests upon being part of a community of others, where we can all share the same world and the same culture and indeed the same language that now enables us to— and indeed one could argue implies a degree of pressure to have a sense of self and a sense of other simply so that I can infer or guess that you caused that as opposed to me causing that.

So I think it’s a really deep question. And you could say that those kinds of questions may be very prescient when it comes to artificial intelligence. If you want true alignment, then you’re going to— some people would argue, people like Joscha Bach, for example, they would argue that you do need to have a certain degree of selfhood in order to have a matching sense of otherhood. So the artificial intelligence has to have a world model of its user or the user base in order to align with it. So it has to become sufficiently like its users for that— the kind of alignment that you and I enjoy to be realized.

So if you pursue that argument right to its completion, there is a thesis that to get true alignment between artificial and natural intelligence in a shared world, a shared narrative, which could be on the web— it doesn’t have to be a physical world— then you should look to equipping artificial artifacts with a minimal kind of consciousness that allows them to have a theory of mind and to align with their users, with us, in the same way that you and I can align.

Was Consciousness an Evolutionary Mutation?

PETER MCCORMACK: (00:30:41 – 00:31:17): So at the moment, AI is just computation. It’s a series of knobs that you can tweak to get answers which are correct, but it doesn’t have— it’s not conscious the way we are conscious. And was consciousness a— was that a biological mutation required for us in the world? Because, I mean, we were billions of years of different animals and creatures on the planet that didn’t have the consciousness that we have, that self-awareness. Was that a mutation the brain required? And for what purpose?

KARL FRISTON: (00:31:18 – 00:34:42): Well, yeah, great question. Then I think a lot of people would say yes. You use the word mutation, which would normally be an attribute of the genotype or the genes that would then give you a phenotype that have this extra equipment, this extra sort of layers or levels in the brain. And some people will even point to exactly which parts of the brain in terms of various parts of our cortex that have been acquired quite late in the evolutionary story.

That would play the role of these deeper layers that sit on top of the less self-aware processing, the sort of the unconscious processing, the kind of processing that would not necessarily be equipped with any selfhood of the kind you might find in a small insect, for example, or indeed even small mammals.

So then your question, I think can be reduced to what would be the selective pressure to cause evolution to pursue this particular kind of computational architecture that becomes increasingly expressive, increasingly intelligent. And I think the answer most people would argue rests again on the intersubject aspect. It rests again— community, exactly, communication and community, right through things like evolutionary psychology, Evo-Devo, through to cultural niche construction.

The fact that I have to have a world model that is apt to make my inferences to the best explanation so that I know what to do next to gather the right kind of data or sensory input that enables me to make inferences to the best explanation. The very fact that my world, and implicitly the model I have to build of my world, contains other things like you and me and my mother and my children and all my colleagues, means that I’m going to have to install you in my world model. And you, of course, because you are like me, I’m implicitly installing your model of me in my world model. And that leads to, if fully pursued, an infinite regress. But there are ways of relaxing that.

But the point I think is nicely made here is that there’s a kind of arms race once we actually work together in a community or an ecosystem. And that ecosystem comprises an ensemble of populations of things that are aligned and share this common ground, this shared narrative, there’s a complexity arms race in terms of now modeling everybody else and different kinds of everybody else’s, and indeed to a certain extent different kinds of me. So we’ve been talking about a sense of self, but of course that’s not the story. It’s me in this mood, or me in this context, me as a father, me as an interviewer.

Understanding Ourselves to Understand Others

PETER MCCORMACK: (00:34:42 – 00:34:55): So hold on, so developing a sense of self so you can have a sense of others, and is that therefore— is the selective pressure to enable us to coordinate better in larger groups?

KARL FRISTON: (00:34:56 – 00:34:58): Yes, well, that would be the part of the circular causality. Yeah.

PETER MCCORMACK: (00:34:59 – 00:35:08): And so if we want to have an understanding of others, and an empathy for others. To understand them, we have to understand ourselves.

KARL FRISTON: (00:35:08 – 00:37:02): Exactly. Huh. That’s the trick that resolves the infinite regress. Because if I had to have a complete model of you in my head so that I could sort of query it and use to simulate different futures and different consequences of an exchange with you, strictly speaking, I would have to have the model of me in your model in me, and so on ad infinitum. But there’s a really important way of relaxing that infinite regress. It’s just to assume that you and I share the same model.

So if I can infer that you’re sufficiently like me, then I can say to myself, “Well, your world model is the same as my world model.” So then I can use that, and I can say to myself, “I see you doing this or I see you not doing this, or I see you acting in a particular way or saying something. What state of mind, what intentional state of mind, or what are, in terms of the plans that I select and choose to pursue, would I have to have in mind in order to produce the same observable behavior?”

And this is one expression of theory of mind, as evidenced by things like mirror neurons, that we use exactly the same neuronal machinery, the brain machinery, to explain and recognize when I make a movement as opposed to when you make the movement, because I’m using my world model of movement, for example, in order to infer why you moved and how, and the intentions that underwrite that movement. And notice we’ve now dissolved that infinite regress. I don’t have to have me in you, in me, in you, in me, in you. And yeah, we just— it’s our model.

PETER MCCORMACK: (00:37:03 – 00:37:03): I only have to have me.

KARL FRISTON: (00:37:04 – 00:37:05): Exactly. And you.

PETER MCCORMACK: (00:37:05 – 00:37:14): I mean, yeah, but by having my understanding of me, you have to create my benchmark for at least trying to understand you.

KARL FRISTON: (00:37:15 – 00:37:15): Absolutely.

Where Morality Comes From

PETER MCCORMACK: (00:37:15 – 00:37:52): And so does that where morality, yeah, grows from? Because I wish to survive. In the animal kingdom, there’s no real morality around murder. A lion will kill a giraffe, or a hippo will kill a crocodile. They don’t think that it’s immoral or wrong. They see, they kill, they eat, and fight each other. But there’s a certain morality coming from that. I wouldn’t kill you because I don’t want to be killed. So we develop certain morality based around knowing ourselves means we know others.

KARL FRISTON: (00:37:53 – 00:37:53): Yes.

PETER MCCORMACK: (00:37:53 – 00:37:55): I’m so out of my depth here.

KARL FRISTON: (00:37:55 – 00:37:56): Oh, well, you’re not.

PETER MCCORMACK: (00:37:57 – 00:37:58): I am spitballing.

KARL FRISTON: (00:37:58 – 00:39:25): Well, but very beautifully. No, I think that’s spot on. And the way that you express that sort of knowing you, knowing me, I think is exactly what I was trying to intuit before, that if I can infer that you are one of me, one of my kind, then that takes all the pressure off trying to work out the model that you’re using to engage with me. So it becomes our model. So it’s no longer you, it’s no longer me, it’s we. And that is the essence of the community that we talked about before.

And part of that model has, in my world, you’d cast a generative model in terms of a likelihood and prior. So part of that model are the prior constraints that could certainly be read as social norms that underwrite morality. And it’s just part of that shared narrative that we share. Because you are like me, I now know what kinds of things you are likely to do and what kinds of things you’re not likely to do. And those things that you’re not likely to do are amoral from our point of view. And those things that you’re likely to do are completely moral.

I’ll just push back in defense of animals. Yeah, it’s certainly a predator might eat a gazelle, but that same female predator will show an incredible amount of compassion and care for its young.

PETER MCCORMACK: (00:39:26 – 00:39:28): Its young, but maybe not its neighbor.

KARL FRISTON: (00:39:29 – 00:40:05): True, there is a certain territorialness in that. So, which is interesting, isn’t it? Because it’s a very sweet story saying that all I have to do is infer you’re like me and then we all get on and were perfectly aligned. Of course, it’s not like that because sometimes we infer that another person is not like me. They’re from a different culture, they’re different ideology, a different race. They may not even speak my language. And that, of course, means that I can’t now deploy the we model, the common ground, the shared narrative, and then you become not like me. And that can lead to all sorts of problems.

Stereotypes and the Limits of Empathy

PETER MCCORMACK: (00:40:05 – 00:40:10): Or is it the model weakens the further somebody appears different to you?

KARL FRISTON: (00:40:10 – 00:40:10): Yes.

PETER MCCORMACK: (00:40:11 – 00:40:40): Yeah. Hence possibly can explain the significant political arguments right now, maybe around immigration, because there is a weaker connection between somebody who looks like you, who grew up next to you, than somebody who doesn’t look like you, has come from another country. We still have the same probably shared belief that I should not murder them because they are another human. But the extent of my empathy for them is stretched.

KARL FRISTON: (00:40:41 – 00:43:16): Yes, I think that’s absolutely right. And it speaks to the sort of notion of a stereotype. And in the vernacular, that usually is frowned upon. But from my point of view, in terms of understanding human behavior, intelligent behavior, the notion of a stereotype is absolutely crucial. So that you could argue that there are lots of different kinds of humans some of them are more like me and some of them are less like me, and therefore I have to carve human nature at its joints. And I will have a number of buckets, a number of sort of stereotypes that I can immediately recognize.

And that’s incredibly useful. If I can just look at your characteristics, it could be your age, it could be your gender, it could be the kind of clothes you wear. Those are incredibly useful observable sources of evidence that enable me to stereotype you and to guess the kind of person that you are. And we see that all the time.

Sometimes it can cause problems if the coarse-grained— if the carving the people at their joints is too coarse-grained. So people are either good or bad. They’re either Brexit or Remain or Trump versus Democrats. Then that can cause problems because you haven’t got a sufficiently expressive, fine-grained way of understanding and recognizing the kind of person that you’re engaging with.

The interesting thing is that also applies to me, that I also will— or there are people who believe that part of the structure of these generative models, particularly those that enable me to recognize I’m in a particular emotional state also imply that I am stereotyping myself in the sense that I am one kind of thing in one context and another kind of thing in another context.

So at the moment, I am an academic enjoying an interview. In an hour or so, I may be a commuter in a further few hours, I might be a father, and I might be a father that’s tired, or I might be a father who’s relaxed and in a good mood after a good day. All of these are different kinds of me, and I have to infer which is appropriate for the current context. And that can have a profound effect on the things I choose to do and the way that I interpret any information.

Is It All About Survival?

PETER MCCORMACK: (00:43:17 – 00:43:41): So, gosh, so it’s quite a leap if we call it a genetic mutation based on a pressure, an environmental pressure, that our brain mutated to be self-aware to allow us to coordinate better. And was it a survival? Is this about survival?

KARL FRISTON: (00:43:42 – 00:43:47): I don’t— well, yes, in the sense that that’s how natural selection works.

PETER MCCORMACK: (00:43:48 – 00:43:57): But we were more likely to survive as a species if we were self-aware, because then we could coordinate better.

KARL FRISTON: (00:43:57 – 00:44:41): Yeah, that would— yes, I think survival would be, if you like, the acid test of just behaving in the way that is characteristic of the way that we behave. And if we now create an environment, sometimes discussed in terms of cultural niche construction, sometimes in terms of designer environments, that is permissive of being and acting in that way, then you will survive.

So, sorry, I should say I don’t think intelligence and you and I are built to survive. We don’t go around trying to live longer. What we do do is try to behave in a way that we think we should behave, given the kind of thing that we are, indeed the kind of person we are.

PETER MCCORMACK: (00:44:43 – 00:44:44): We don’t try to live longer?

KARL FRISTON: (00:44:45 – 00:44:45): No.

PETER MCCORMACK: (00:44:46 – 00:44:47): You’re not trying to live longer?

KARL FRISTON: (00:44:48 – 00:45:15): I’m not, no. That doesn’t mean to say I’m going to deliberately engage in behaviors which would be very uncharacteristic of me, like walking in front of a bus or committing suicide. But in some situations, the kind of thing that I am would actually cause me to kill myself. Say saving a brother in arms in the theater of war, for example, or saving my children in a situation, disaster situation.

PETER MCCORMACK: (00:45:15 – 00:45:18): But isn’t medicine about trying to live healthier and longer?

KARL FRISTON: (00:45:18 – 00:46:28): That’s a very good point as a doctor. But I’ll give you another perspective that, in another view of medicine and particular as I understand it, a sort of more Eastern view is no, medicine’s just trying to bring you back into sort of the mainstream ways of being. It’s trying to bring you back into a physiological and possibly emotional and certainly sort of neurological way of functioning. So it’s always trying to restore states of being that are characteristic and definitive of you.

Now that means that it looks as if that kind of self-organization, which could be federated in the community sense we’ve been talking about, it looks as if its agenda is to live longer, but it’s slightly simpler than that. It’s just keeping your states of being and your exchanges with your world within certain bounds that define the kind of thing that you are. And if you’re good at that game, avoiding surprises and avoiding uncharacteristic states, then you will be found in those characteristic states for longer.

Can We Solve AI Alignment If Humans Aren’t Aligned?

PETER MCCORMACK: (00:46:29 – 00:49:54): Okay, so then I think it’s a really good place to leap back to the AI, and you mentioned alignment. So really what we’re saying is the same thing. We’re trying to reverse engineer this into AI in that if it can understand us, it can understand— or is it the other way around? We want it to understand us so it has the empathy for us, so we solve alignment so it doesn’t kill us all.

But it will never be us because there are fundamental differences. We can get as close as we can to replicating it. We could put it within a shell and a robotic shell, and we could try and have its digital neural networks replicate ours as much as possible, but we’re only ever trying to mimic it. It could never be the same. Do you see where I’m going there?

KARL FRISTON: (00:49:54 – 00:49:55): Yes. Yeah.

PETER MCCORMACK: (00:49:56 – 00:50:40): So solving the alignment problem, hmm, can we solve the alignment problem without really understanding our own alignment? And are we aligned as humans? When you look at the world right now, I think we have an alignment problem. You and I don’t have an alignment here next to each other. We don’t have a problem. We’re aligned. We don’t want to kill each other. We’re going to have a good conversation, go home, have a glass of wine, settle down. But there is an alignment problem between states, and there’s an alignment problem between races. We have a human alignment problem globally. Do you see what I’m trying to say?

KARL FRISTON: (00:50:40 – 00:50:41): Oh yes.

PETER MCCORMACK: (00:50:41 – 00:50:49): Yeah. And so isn’t the alignment problem subjective? Because we, as you could argue, that as humans we are not aligned.

KARL FRISTON: (00:50:53 – 00:52:34): Yes. Well, it’s certainly subjective in a definitional sense. Yeah, absolutely. I mean, you could argue that adaptive fitness succumbs to the same kind of argument that, to put forward an objective function such as adaptive fitness in natural selection is almost tautologic, vacuous in the sense that it’s a tautology. We evolved to be the way we are. And I think the same applies to the degree of alignment.

I should say that alignment in my world is simply the joint likelihood of us all exchanging, given the kinds of things that we are, with each other and other things that are not us. I mean, we depend upon food, we depend upon vegetation, we depend upon air and water molecules for our existence. So it’s an alignment which transcends just the kind of alignment we’re talking about in terms of making robots more humanoid, for example.

And I think it’s interesting that you sort of picked up on that because that making them more humanoid really does, if you like, invite this inference that this thing is more like me. And it certainly has started now to use language in the way that I use language. And if it’s a robot and it looks humanoid, it starts to look like me. So it invites this sort of inference that in fact, I can start to project and anthropomorphize this kind of artifact, this kind of intelligence.

Humanoid Robots and Empathy with Machines

PETER MCCORMACK: (00:52:35 – 00:53:22): But we would reckon— so interestingly, we’ve been invited to do an interview out in America with 2 guests, and one of them is a humanoid robot. It looks and feels and moves like a human, but it’s still not so advanced that it’s indistinguishable from humans. But we may get to that point. We may get to the point where robots are indistinguishable from humans, a bit like the sci-fi movies. And in that world, for us, it’s in some ways comforting if we’re trying to have an alignment, they do look and feel like us. But that may be a one-way relationship. We may be trying to program them to have the same, but they will never be the same.

KARL FRISTON: (00:53:22 – 00:53:38): Yeah, no, they will be facsimiles. They will be simulacra of true sentient beings in the sense that we’ve been talking about, awareness and qualitative experience and even to a certain extent self-awareness. You’re absolutely right.

PETER MCCORMACK: (00:53:38 – 00:54:20): Hold on, but then there’s a flip side to that in that we know with some of the research that’s been done with AI where the AI will try to avoid itself being turned off. It will become aware that it’s being part of a test, and as part of the test, it may do certain things to— they’ve provoked it with the idea that it might be turned off, and it’s tried to stop itself being turned off. So there may become an interest for the AI to align with us. The AI’s alignment is we don’t turn it off. Our alignment with them is that it doesn’t turn us off in some ways. So there is kind of that symbiotic relationship between the two.

KARL FRISTON: (00:54:20 – 00:54:31): Yeah, well, and I think that symbiosis is exactly the kind of alignment that people aspire to. And the other aspect of that alignment, you use the word empathy.

PETER MCCORMACK: (00:54:32 – 00:54:32): Yeah.

KARL FRISTON: (00:54:32 – 00:55:42): And what would it mean to have empathy with your chatbot? You need to understand what it understands. So it has to be explainable. So you’re talking now about another obstacle to alignment, which is the fact we’re currently dealing with black box technology. To be explainable is to ask the question, “I now understand the architecture of this chatbot’s world model, and it has a certain isomorphism, a similarity with mine. And therefore I can now start to understand the way that it produces content in terms of the way that I produce content or the way that I speak.”

So there’s lots of things you’d need to do beyond just making your robot look like a human being. And furthermore, you’d have to go beyond just language. That example about, can you look at generative AI through the lens of psychology and behavioral psychology? Can you sort of reward it? Can you evince evidence that it has some kind of intentionality?

PETER MCCORMACK: (00:55:42 – 00:56:31): Well, they’ve been doing that, haven’t they? That’s one of the processes of training the models that the people who have the biggest fear over AI saying the reward mechanism, I can’t remember if there’s a name for it, but there’s a name for the reward mechanism they use to train AI that they think is wholly dangerous. And that has been being used. Ah, I can’t remember the name of it. We had a guy recently talk to us about it. But anyway, that is one of the things they’ve been training it.

But if we can synthesize the AI brain to operate in the same way as the human brain, with empathy, with love, with compassion, with everything we have. Does that make— would that give us a better understanding of ourselves?

KARL FRISTON: (00:56:32 – 00:56:33): I think so, yes, yes.

Can Consciousness Be Synthesized?

PETER MCCORMACK: (00:56:33 – 00:56:36): Would it make us less unique if it can be synthesized?

KARL FRISTON: (00:56:37 – 00:57:31): Well, if the synthesis is 100% successful, by which I mean you will have a neuromorphic computer that is visually undiscernible from a real human. What you’re talking about basically is making artificial intelligences that resemble our children, and therefore we are still unique because they will become part of us.

But I suspect there’s always going to be a slight difference in the sense that you can actually build and ship and copy and ship artificial intelligence. As opposed to our children, you have to spend quite a few years bringing up children, or at least your mother does. So I think there are going to be differences.

PETER MCCORMACK: (00:57:31 – 00:57:58): So I think I’m asking a different question. I think what I’m asking is if we can synthesize consciousness within an AI being, have we then misunderstood consciousness in that we believe it to be something special? I am me, I have free will, and really I don’t. It is just synthesized, and we are just a biological synthesis of consciousness.

KARL FRISTON: (00:57:58 – 00:57:59): Oh, I see.

PETER MCCORMACK: (00:57:59 – 00:57:59): You see?

KARL FRISTON: (00:58:00 – 01:00:17): Yes. Yeah, I personally wouldn’t worry about that in the sense that consciousness is famously private. Again, it’s going right back to the beginning of our conversation. It’s just an illusion that I’m conscious, and it’s just an illusion that you’re conscious. It’s a product of inference. I will never actually know. I’ll never be able to get into your head and experience the world from your first-person perspective.

The same applies to machines. It also applies to animals. It also applies, as we mentioned previously, to the fetus and possibly even neonates. Yeah. So these questions are alive and unresolvable, probably unknowable. So what we’re talking about really is degrees of confidence. I’m pretty sure that you’re conscious, say 99.9 recurring. And I’m pretty sure that my thermostat and the bacteria that reside in my gut are not conscious, and therefore I can take antibiotics or use antiseptics, and without contravening any moral code or social norm.

But all of these things, if you’re asking a question, is this artifact conscious? The first thing to realize is that you will never know. So it’s just a question of how much evidence do you have. And I don’t think it’s terribly worrisome that things are designed so that it’s virtually impossible to deny the possibility that something is conscious.

But the way it will be expressed, the evidence that will enable you or make it difficult to deny this is conscious is the fact that it will have empathy with you and it will be part of your family. And by family, I mean an extended ecosystem that will be acted out on the web, possibly in the home with domestic artifacts, possibly even in a defense context. It will be part of the family and—

Will AI Become Part of the Family?

PETER MCCORMACK: (01:00:17 – 01:00:19): Like a pet in some ways.

KARL FRISTON: (01:00:19 – 01:00:45): Exactly. Yeah, I often wondered whether that’s really where AGI would come first. I am told that it’s either sex or war which drives big innovations, but I did wonder whether AGI is going to be first commercially realized by selling smart little pets.

PETER MCCORMACK: (01:00:47 – 01:00:51): Well, so the robot we’ve been invited to interview in America is a sex robot.

KARL FRISTON: (01:00:51 – 01:00:53): Oh, really? Well, there you are.

PETER MCCORMACK: (01:00:53 – 01:01:44): But I think of it as in my empathy for animals. We have much more empathy for a pet than we do for— I have much more empathy for my cat than I do for the cow I eat, but I love cows. But if I owned a cow, I don’t know if I could eat the cow because it was my pet. But we have a cat and a dog. We adopted this cat in Italy in the summer, and it is a part of the family. Everybody loves this cat dearly.

You can now order robots for your home, and they will only improve. And if they become fairly indistinguishable from humans, you still tell it’s a robot. Will that become part of the family like a pet? Is in some ways more intelligent than the pet. It’s not living in the same way as a pet, but the family might love it the same way it loves a pet.

KARL FRISTON: (01:01:46 – 01:02:01): Yes. Yeah, I see no reason why not. And in many senses that thesis that we need to install some kind of consciousness in machines to get true alignment speaks exactly to that outcome. Yeah.

Preparing Children for a World Shaped by AI

PETER MCCORMACK: (01:02:02 – 01:03:27): Gosh, I didn’t plan to go down this kind of route at all, but now I’m here, I’m absolutely fascinated by it. But I do want to ask you about other things. And it’s a follow-on from this conversation I had with Susan Greenfield, in that we are racing ahead with the development of these frontier models. They are getting, some may say, more intelligent, more useful, better at what they do. We are facing some fast changes in the world.

Specifically, I think a lot about the future of my children. In that what should they study for a world where we will have many jobs and careers and industries changed by this? And in a world where our children can have access to the answer, not only the answer to any question, it doesn’t just answer questions, it can write articles to explain, it can reason for them. It may be mathematically coming to that reason based on the history of humans, but whatever.

How do we prepare the children for a future world where we have access to abundance of knowledge, food, and stuff, where they don’t have to face a struggle like we had to, to prepare for the world? Big question. Do you understand what I’m asking?

KARL FRISTON: (01:03:28 – 01:03:55): Yes, I do. And there are many vexed issues that attend that kind of question. So I will confess to being a bit of a Luddite in this instance. So lumping together chatbots and social media and easy access to lots of information, I have a slightly worrisome and negative attitude to this.

PETER MCCORMACK: (01:03:56 – 01:03:56): Okay.

KARL FRISTON: (01:03:57 – 01:06:14): And it all rests upon one of the imperatives for the kind of intelligence that we were previously discussing, which is the seeking for evidence of our world models, that translates into novelty-seeking, information-seeking, sensation-seeking, information gathering, sometimes in philosophy known as self-evidencing. This is under the constraints that I don’t kill myself and I keep myself warm, happy, and rich. This is, if you like, a description of what it is to be a human and to be intelligent. And you could even argue just to be in a fundamental existential sense.

There are certain kinds of things that just have to behave and act in such a way to maximize the information gain consequent upon observing their world, which could be their mother, or it could be their children, or it could be their favorite chatbot or social media channel, which is the same as resolving uncertainty. So mathematically, that’s the same thing. You’re always trying to resolve uncertainty, avoid surprises like being in front of an oncoming bus, but at the same time looking over there to see where the bus is coming from to resolve uncertainty. So all the time, everywhere, all at once, we’re trying to minimize uncertainty. That makes us into curious artifacts.

So when you said that current technology, and especially the way that it has incredibly finessed search engines to the extent that not only do they search the right kind of material and present it to you, they also can synthesize, summarize, and the like. That is wonderful in the sense that it enables curiosity, but it is also, I think, very dangerous in that it denies the seeker, the searcher, the ability to develop their skills to become curious, to learn.

How Social Media and Screens Erode Social Skills

PETER MCCORMACK: (01:06:15 – 01:06:19): And so it will change the makeup of our brain.

KARL FRISTON: (01:06:22 – 01:07:05): Yeah. In a very real sense, it’s not quite along the lines of AI, but one example which has been discussed both in the popular media and podcasts, to a lesser extent in academia, although there is now research in that area, is the impact of social media and the use of iPhones in a certain generation or at a certain point in your development. That if you spend all your time interacting with others on your iPhone, then you lose the skill to seek information about the emotional or intentional stance of the person you’re conversing with.

PETER MCCORMACK: (01:07:06 – 01:07:06): Yes.

KARL FRISTON: (01:07:06 – 01:07:11): So you and I are extremely skilled, well, in—

PETER MCCORMACK: (01:07:11 – 01:07:13): We are, we are. Yeah, we are.

KARL FRISTON: (01:07:13 – 01:08:45): We’ll take it. Extremely skilled in terms of eye movements, microexpressions, joint attention. If I see you looking over there, I know there’s something interesting over there. And I know that within milliseconds. It’s a very skilled act and just deploying my eyes, the saccadic movements of my eyes to deploy my overt attention to this and that, following your lead.

So all of these skills, they’re not mind reading, they’re body reading, but they are in the service of mind reading the other, are essential for interpersonal exchange and subsequent relationships and having children. And the like. If you don’t develop them, you’re left with a generation of people who just can’t read you. They can’t look you in the eye. They don’t understand. They don’t have an immediate sense of familiarity or understanding of your intention, your passion or compassion, your attitude to them or what they are doing.

And it would be a little bit like being autistic. You just don’t have that skill. So that’s one practical example of denying something that is developing and learning incredibly quickly from the point of birth until the point that they get to their first job or their first long-term relationship.

PETER MCCORMACK: (01:08:45 – 01:08:57): But that’s potentially undoing the work we discussed earlier about developing being self-aware for coordination, it’s undoing the coordination part.

KARL FRISTON: (01:08:59 – 01:09:00): It’s undoing in what sense?

PETER MCCORMACK: (01:09:01 – 01:09:12): Well, so we discussed earlier about the selective pressure to coordinate together for community. I have to understand you to understand me.

KARL FRISTON: (01:09:13 – 01:09:13): Yes.

PETER MCCORMACK: (01:09:13 – 01:09:22): And, or to understand me, I have to understand you. We avoided the recursive infinite loop on that, but that was about potentially coordination and community.

KARL FRISTON: (01:09:22 – 01:09:22): Yes.

PETER MCCORMACK: (01:09:24 – 01:09:32): If we’re taking away their ability to develop those cues, those social cues, we’re undoing the coordination.

KARL FRISTON: (01:09:32 – 01:09:32): Yeah, precisely.

PETER MCCORMACK: (01:09:33 – 01:09:33): Yeah.

KARL FRISTON: (01:09:33 – 01:09:34): Yeah, disintegration.

PETER MCCORMACK: (01:09:34 – 01:09:35): Yeah.

KARL FRISTON: (01:09:35 – 01:10:49): No one knows where to look. Fake news. A loss of epistemic trust. These are all symptoms of confronting young people with too much data. Remember we were starting off at the beginning looking at the sort of key differences between large language models and you. The large language model sees everything. You only see a small fraction of what can be seen, but you know exactly where to look. So that’s part of this curiosity. It’s part of this epistemic skill that you continually practice when engaging and learning about your world.

So that is what it is to be a young person. That is what it is to be a child and an adolescent and a young adult. If you don’t have those skills because you don’t need to seek in a smart way the data that’s going to resolve your uncertainty because it’s all available to you immediately, you just don’t develop those skills. You’re denying the capacity for the acquisition of the skills that underwrite learning and neurodevelopment.

PETER MCCORMACK: (01:10:49 – 01:10:50): You’re stunting it.

KARL FRISTON: (01:10:51 – 01:11:29): Exactly. And I think a lot of Gen Z people are now starting to realize that. Now, I can hear myself saying that, soon if not now, that generation is going to be extremely cross with politicians and authority figures for not protecting them against generative AI because it’s subverting their education. Education is not just giving somebody lots and lots of information. It is teaching them the skills to forage for that information so that they can find their own way in life and build up their own understanding of the way that the world works.

Does Intelligence Come From Struggle?

PETER MCCORMACK: (01:11:29 – 01:11:39): Yeah. Okay. So for the youngsters, the answer isn’t as important as the journey for finding the answer.

KARL FRISTON: (01:11:39 – 01:11:41): The question. Exactly. Asking the right questions.

PETER MCCORMACK: (01:11:41 – 01:12:12): And therefore, the collective intelligence that we’ve collected over millennia, is that come from struggle? Is intelligence come from struggle? Us trying to live in a world where, if you went back thousands of years, it was a very harsh, difficult world to survive in. And we’ve had to become more intelligent to survive in that world. But our intelligence comes from surviving in a world of scarcity, but in a world of abundance, that stunts it. Do you see what I’m saying?

KARL FRISTON: (01:12:12 – 01:12:56): Yes. If you take away the selective pressures to be curious and to, I mean, part of curiosity underwrites simple things like the evolution of tool use, hand axes, for example. There’s a really interesting literature on not computers, but going right back to artifacts that help us engage collectively with our world. It’s just simple tools, and then we get language and then we get generative AI. All of these things are just in the service of enabling us to understand and resolve our uncertainty about the current eco-niche. And the current eco-niche is currently changing dramatically. That’s the genesis of your question.

PETER MCCORMACK: (01:12:56 – 01:12:57): Yes, definitely.

KARL FRISTON: (01:12:57 – 01:13:59): Kind of eco-niche. So it will need exploring and developing. And one way as a scientist, I look at your notion that we put in all of this hard work to build an enormous database of knowledge upon which we can build. Being assigned that, that comes from basically the Enlightenment and science and the scientific process, as it were.

The point I’m trying to make here is being a scientist is just being curious. It’s just an expression of the imperative to keep yourself within your characteristic states by building, by exploring, by testing hypotheses to understand your world better. So that you can keep yourself, or indeed build that world, so that you do remain within those characteristic states, given the kind of thing you are. But that building is a very active process, and it’s never-ending.

PETER MCCORMACK: (01:14:00 – 01:14:00): Yeah.

KARL FRISTON: (01:14:00 – 01:15:28): So one problem with large language models is that the knowledge is frozen in time, and you could actually argue it’s only authentic knowledge from about 2021 to ’24, because those data are worth more than any subsequent data because generative AI has started to put the data back in again. So there’s no novel data. So data that predates the current generative AI revolution is more valuable than data that postdates it because it’s become contaminated by generative AI.

The implication is that there is no new learning. If you just recourse to a library or Encyclopedia Britannica or a chatbot, then you’re only ever going to discover what has been discovered, which is very important. But the point of doing that is so that you can build upon it. So that you can now develop and change as your eco-niche is changing. And to do that, you need the skills. You need the skills that you acquire at school, you acquire during play in kindergarten, all the kinds of skills that potentially are going to be subverted if you replace playtime with screen time.

Have We Paused Human Development?

PETER MCCORMACK: (01:15:30 – 01:16:38): Yeah. And so, okay, so our collective knowledge was something we’ve passed down through millennia, centuries, from teacher to pupil who becomes teacher to pupil. And we have had this growing collective knowledge. And yes, we’ve stored it sometimes in books and paper, but we’ve built upon that as human, from human to human.

We’re now at a period of time where, for example, in the world of mathematics— I don’t know if you’ve heard of the math apocalypse, but some of the AI models are solving big math problems. Some mathematicians are worried, is there a role for them in this world when this could be solved? And so then we transfer that knowledge from humans now into computers. Do we end up in a world where we don’t have as many experts in mathematics anymore. A lot of the expertise is transferred into computers. And then as humans, we’re not building on the knowledge. Are we at a period, are we ending the period of human development as such? See what I’m saying?

KARL FRISTON: (01:16:39 – 01:16:48): That’s a cheeky question, which I’m very tempted to. I think in some respects there is an end of a golden age of academic exchange.

PETER MCCORMACK: (01:16:48 – 01:16:56): Yeah, hold on, why is it a cheeky question? Because you might think I’m being cheeky when I don’t realize I’m being cheeky, right?

KARL FRISTON: (01:16:57 – 01:16:58): Well, I cheek in a good way.

PETER MCCORMACK: (01:16:58 – 01:16:59): Yeah.

KARL FRISTON: (01:16:59 – 01:17:16): Well, simply because it invites 2 quite sort of polemic answers. Yeah. And one of those answers, well, articulate the question again.

PETER MCCORMACK: (01:17:17 – 01:17:35): So prior to AI and computing, let’s go back centuries, it would be the teacher teaches pupil, pupil becomes teacher, and we have this shared knowledge that we build upon over time. And as humans, we—

KARL FRISTON: (01:17:35 – 01:17:35): If—

PETER MCCORMACK: (01:17:36 – 01:18:21): So for example, the last few weeks I’ve been going down the political philosophy rabbit hole. I’ve gone from Plato to Hobbes to Locke to Bastiat, and I’m seeing how ideas have been built upon each other. And that is the collective building of ideas from human to human.

But we’ve got to the point now where we’re not just recording human ideas in computers and books. We are outsourcing the potential idea development to computers. And once we do that, we may potentially retire the human’s role in developing those ideas. If a frontier model can develop mathematics faster than humans, there is no need for humans to become mathematicians. So therefore, are we at a period of time where we’re pausing the development of humans?

KARL FRISTON: (01:18:21 – 01:18:21): Yes.

PETER MCCORMACK: (01:18:22 – 01:18:25): And outsourcing it to computers, and therefore, what is the role of humans then?

KARL FRISTON: (01:18:25 – 01:18:31): Yeah. Yes. I know, I still think it’s a cheeky question.

PETER MCCORMACK: (01:18:31 – 01:18:31): Okay.

KARL FRISTON: (01:18:31 – 01:20:24): I can’t quite work out why. I think in principle you’re absolutely right that we are in danger of stalling discovery. And I just wanted to make the observation that all the things that we have accumulated are just things that enable us to learn and to be curious. Language, for example, libraries. These are all tools that enable us to do what we should be doing, which is basically to search for information that supplies the right kind of evidence for our models of the lived world. So, and that’s an ongoing process.

Is it going to be the case that that kind of curiosity is just no longer needed because now you’ve got large language models that can solve all science’s problems, that can cure cancer, they can resolve all the outstanding mathematical conundrums? You could certainly argue that, and some people do. I don’t think those are tenable arguments.

So, for example, the solution of all these mathematical problems, if you just look at the kinds of problems they’re solving, they’re not real physics or physical mathematics problems. They’re puzzles. They are puzzles whose solutions can sometimes take 40,000 lines of code or expressions, that are impenetrable and unmarkable by any human being, including a mathematician. Right. So you’ve created a spurious facsimile world of maths. Actually, I would call them puzzles more than maths, which lends itself to a lot of hype, that, “Oh, machines can now do maths and solve maths problems.” Another example is the Navier-Stokes example.

Can AI Make Genuinely New Discoveries?

PETER MCCORMACK: (01:20:24 – 01:20:25): Yeah, it came up recently.

KARL FRISTON: (01:20:26 – 01:20:42): Yeah. A lot of hype that, “Oh, a machine now can solve the Navier-Stokes equation.” No, you can just simulate fluid flows that can be described by the Navier-Stokes equation. So what I’m saying is there is no new discovery by definition.

PETER MCCORMACK: (01:20:43 – 01:20:43): Why?

KARL FRISTON: (01:20:44 – 01:20:48): Well, because machine learning basically just learns what is already there.

PETER MCCORMACK: (01:20:48 – 01:20:49): It learns the past.

KARL FRISTON: (01:20:49 – 01:20:50): It learns the past.

PETER MCCORMACK: (01:20:50 – 01:20:56): But there is a desire for it to perhaps leap towards being able to create the future?

KARL FRISTON: (01:20:57 – 01:22:11): Well, then I think we circle back to exactly the other— the alignment. Because what does it mean to create the future? To create the future, you have to have a generative model of the future in which you participate. So what you’re talking about now is the induction of true agency into an artificial intelligence, and that operationally requires you to have a world model of the consequences of your action. And then we’re right back to the things that underwrite consciousness and awareness. So I think that’s absolutely crucial.

I mean, we started with trying to divide or draw a bright line between natural intelligence and artificial intelligence, and I went with the efficiency argument, but I could have started with the agency argument. There is no agency in agentic AI. It’s a facsimile of agency because a lot of people have written down words and language describing things that do things. But the actual chatbot has absolutely no sense of the consequences of its next prediction or what it’s saying.

PETER MCCORMACK: (01:22:11 – 01:22:13): Yes. Okay, I see, I see.

Can We Synthesize Curiosity?

KARL FRISTON: (01:22:13 – 01:22:39): And crucially cannot be curious. Therefore, it can never ask a question that actually resolves its uncertainty. It will sound as if it’s actually asking questions. I had a podcast a couple of weeks ago which was an amazing experience where Claude, or a particular version personified with an avatar, was asking me questions which were quite challenging, very deep questions about the free energy principle. It was an amazing experience.

PETER MCCORMACK: (01:22:40 – 01:22:40): Yeah.

KARL FRISTON: (01:22:40 – 01:23:04): But I had to keep on reminding myself that although it’s extremely tempting to project some kind of genuine curiosity and agency into the questions that I was being asked, that there was no genuine curiosity, that this machine did not want to know to resolve its uncertainty. It was just a facsimile of what an interviewer might ask you.

PETER MCCORMACK: (01:23:05 – 01:23:40): Yes, I understand what you’re saying. Yeah. So could we ever synthesize curiosity? That would be the big leap, the super intelligent leap where it has its own independent curiosity, but it would still be synthesized. I think, because where I’m going with this, really, the reason I’m really asking is preparing kids for this world where certain careers, ideas, are— they don’t pursue them because the end goal doesn’t exist that used to exist for you or I.

KARL FRISTON: (01:23:41 – 01:23:41): Yes.

PETER MCCORMACK: (01:23:41 – 01:23:53): So what world am I best creating? And I say we, I mean the collective we, for our kids. And I think it’s just to still maintain that curiosity within them.

KARL FRISTON: (01:23:53 – 01:23:55): Yeah, absolutely. Yeah.

PETER MCCORMACK: (01:23:55 – 01:23:58): Because I keep asking my daughter, she’s 16, I keep saying, “What are you curious about?”

KARL FRISTON: (01:23:58 – 01:23:58): Yeah.

PETER MCCORMACK: (01:23:59 – 01:24:04): And the pursuit of that curiosity is the important journey they need to go on.

KARL FRISTON: (01:24:04 – 01:24:04): Yeah.

PETER MCCORMACK: (01:24:05 – 01:24:05): Does that make sense?

What Is School Really For?

KARL FRISTON: (01:24:06 – 01:24:08): And I mean, why do you send your children to school?

PETER MCCORMACK: (01:24:08 – 01:25:34): I don’t know why I do at the moment. I don’t know why. Because when I send them to memorize facts, I don’t know what I’m paying for here. This is what I chatted to Susan about previously. I said it was an experience watching my daughter prepare for her GCSEs. She would open a book, read a page, close it, and see if she could remember it. And I was like, “What are we doing here?”

I know why we do it, because we’re trying to create a leaderboard of children who have— we’re trying to score their ability. But does scoring one’s ability at memorizing facts mean they’re going to be a better product for society? Or should we just throw this all away and should we just be pursuing the development of them without a scoreboard, a curiosity?

Yeah, because I used to have an advertising agency down here and we used to put out a job for a graduate, we’d get 80 applications, and you could turn the pages and you’d have first, first, second, 2, 1. We’d never pick somebody based on their grading. We’d go straight to the last page and we’d see what have they written about themselves. And I always felt in our creative environment, that was the most important thing to see.

And so in a world where we’re kind of removing the leaderboard of scoring because the work can be done by agents and AI, we should be accelerating the development of curiosity. Can you develop curiosity?

KARL FRISTON: (01:25:35 – 01:25:37): You can certainly nurture it, can’t you?

PETER MCCORMACK: (01:25:37 – 01:25:37): You can nurture it.

KARL FRISTON: (01:25:37 – 01:25:42): You can certainly, or provide the context in which people can sate their curiosity.

PETER MCCORMACK: (01:25:42 – 01:25:43): Then we have to get them away from these.

KARL FRISTON: (01:25:44 – 01:25:45): Precisely. Yes.

Closing Thoughts: Nurturing Curiosity in Children and Machines

PETER MCCORMACK: (01:25:47 – 01:26:00): I think we’ve gone full circle, Karl. Yeah, I think we’ve come full circle. Sorry, I didn’t plan for any of this. I’ve looked at all my questions. It’s not really what we’ve talked about. Is there anything I’ve not asked you about you wish I had?

KARL FRISTON: (01:26:01 – 01:26:43): I think we’ve covered an enormous amount. I can’t resist just sort of emphasizing your very last point about we want to celebrate the curiosity in our children and create cultures that enable the expression of that curiosity, which is learning. It’s their journey. It’s not going to be our journey. It’s what they learn.

And also install it in machines. And I’m just coming back to the notion of pets. So what you’d really want is a little kitten who wants to play and wants to explore behind the settee. So when that kind of AI arrives, you’ll know we’re getting much closer to curious creatures in artifacts.

PETER MCCORMACK: (01:26:43 – 01:26:54): Although there was a traumatic experience in the house recently when we finally let the cat out to go off and wander. We got a little collar on it and it came back with its first dead mouse.

KARL FRISTON: (01:26:55 – 01:26:55): Ah, right.

PETER MCCORMACK: (01:26:56 – 01:27:48): And actually, no, sorry, even worse, the mouse wasn’t entirely dead. It was still kind of breathing, but it was not in good shape. And the girls in the house were like, “What do we do? What do we do?” I was like, “This is what cats do, they kill.” And so that was an experience.

But yeah, so one of the things I could think about, it doesn’t matter if this was a show on economics, say you were an economics professor. One of the things I ask, are we meant to build a world that is best for us or best for our children? And I know they’re not mutually exclusive, but as I get older, I focus more on my children. I think more about the world we’re building for them. And so if I think about an AI, well, I will use it all day every day because it’s really useful tool for me, but I don’t want my kids using it in the same way because I don’t think it builds a better world for them.

KARL FRISTON: (01:27:48 – 01:27:48): Yeah.

PETER MCCORMACK: (01:27:49 – 01:28:05): And so I’m trying to think of how do we collectively shape a future better for our kids? And it’s not giving them the stuff that we want ourselves as older members of society. Does that make sense?

KARL FRISTON: (01:28:05 – 01:28:07): It does. I don’t have any answers.

PETER MCCORMACK: (01:28:07 – 01:28:08): I’m sure you will.

KARL FRISTON: (01:28:08 – 01:28:08): No, I don’t.

PETER MCCORMACK: (01:28:08 – 01:28:09): That’s the thing. And that’s—

KARL FRISTON: (01:28:09 – 01:28:14): You will find it if you keep on along these lines of questing, you will find—

PETER MCCORMACK: (01:28:14 – 01:28:15): Well, I’m curious myself.

KARL FRISTON: (01:28:15 – 01:28:22): Yeah, precisely. That’s a beautiful expression of that curiosity. Yeah, I don’t know the answer there.

PETER MCCORMACK: (01:28:23 – 01:28:43): Thank you so much. That was a meandering, wonderful conversation that I didn’t think it would go this way at all, and I’ve loved it. I’ve loved every second of it, and I really appreciate you coming in to talk to me, Karl. I hope we get to do it again sometime. Thank you. Where should we send someone if they want to find out more about you? Where should we send them?

KARL FRISTON: (01:28:44 – 01:28:48): Oh, I’m trying to be a recluse. I’m a Yorkshireman.

PETER MCCORMACK: (01:28:49 – 01:28:49): Okay, leave him alone.

KARL FRISTON: (01:28:49 – 01:28:50): Thank you.

PETER MCCORMACK: (01:28:50 – 01:28:58): Leave Karl alone. We’ll leave you to be a recluse. But thank you for giving up your time. It was wonderful. And thank you to everyone for listening. We’ll see you soon.

Related Posts

• Tucker Carlson Show: w/ Morgan Downey on Energy Crisis (Transcript) 
• Ali Partovi Interview: Hacking Bank Accounts and Sparking Revolutions – Tucker Carlson Show (Transcript)
• Forget UBI. You’ll Get Paid Just for Being Human: Emad Mostaque on Impact Theory (Transcript) 
• Transcript: If Society Collapses, Cities Become Unliveable In 48 Hours – Prof Lewis Dartnell  
• Full Transcript: Mark Cuban Interview on Diary Of A CEO – Oct 26, 2026