Read the full transcript of Google DeepMind’s VP of Research Zoubin Ghahramani’s interview on Google DeepMind: The Podcast with host Hannah Fry, August 26, 2026.
EDITOR’S NOTE: In this insightful interview, host Hannah Fry talks with Zoubin Ghahramani—Cambridge professor and VP of Research at Google DeepMind—about a problem he has been working on for three decades: teaching machines to know what they don’t know. Drawing on Bayesian thinking and real-world systems from weather models to AlphaFold, he argues that uncertainty isn’t a flaw in AI so much as a missing ingredient for more reliable, humble, and useful intelligence.
Introduction
HANNAH FRY (00:00:00 – 00:01:06): Welcome to Google DeepMind, the podcast. Now, if you ask an AI a question, then it will usually give you an absolute answer with unwavering authority, even if that answer turns out to be wrong. In fact, today’s AI seems to be missing a fundamental human trait, self-doubt.
But long before the current wave of large language models, one academic researcher was trying to give machines a sense of their own limitations. Zoubin Ghahramani has spent the last 30 years pioneering a type of intelligence built on the mathematics of uncertainty. Today, as a professor at Cambridge and co-lead of Frontier AI at Google DeepMind, Zoubin finds himself at the heart of another interesting debate.
On the one side are those who are hoping that pure scale will be the answer to ever-improving AI, and on the other are those like Zoubin who believe that true intelligence requires innovations in architecture. And that improving machine uncertainty may be one of the missing pieces. Zoubin, welcome to the podcast.
ZOUBIN GHAHRAMANI (00:01:06 – 00:01:06): Thank you.
Why Uncertainty Matters in AI
HANNAH FRY (00:01:07 – 00:01:16): If you were to distill it all down, I mean, your central thesis is that we need to have uncertainty in AI. Just, I mean, give me the top line of it.
ZOUBIN GHAHRAMANI (00:01:16 – 00:01:17): The why? Yeah.
HANNAH FRY (00:01:17 – 00:01:17): Why?
ZOUBIN GHAHRAMANI (00:01:18 – 00:02:09): Well, if you think about intelligence, one of the most important parts of intelligence is decision-making. You can’t have an intelligent system that doesn’t make decisions, from bacteria to animals to humans to robots. Decision-making is really important. And if you want to make decisions in the real world, our perception is limited. So we are always uncertain about the state of the real world. And we need to make decisions under uncertainty. We can’t know everything. We don’t know everything from our senses. We can’t predict the future. And so fundamentally, to build an intelligent system, you need a system that can represent uncertainty, that can update its uncertainty, and then can use that to make good decisions under uncertainty.
Types of Uncertainty
HANNAH FRY (00:02:10 – 00:02:21): Because actually, I mean, there’s 2 different types of uncertainty. I guess, right? There’s the uncertainty of just the inherent randomness of the world. Yeah. There’s a pedestrian in a normal typical street.
ZOUBIN GHAHRAMANI (00:02:22 – 00:02:22): Yeah.
HANNAH FRY (00:02:22 – 00:02:25): And you just don’t know which way they’re gonna turn.
ZOUBIN GHAHRAMANI (00:02:25 – 00:02:25): Yeah.
HANNAH FRY (00:02:25 – 00:02:29): But then there’s the uncertainty of like a scenario that you’ve never encountered before.
ZOUBIN GHAHRAMANI (00:02:29 – 00:04:04): Let me give an example from something that is becoming more and more of a reality in all our lives, which is self-driving cars. So when you’re in a self-driving car, the self-driving car has been trained on lots and lots of data. It’s seen many, many scenarios. But you can imagine that there is what’s called the long tail of things that could happen. Like, for example, the car may have not been trained in many instances of hailstorms, and it may not have been trained with, horses suddenly jumping in front of the car in a hailstorm.
And so essentially what you really want from an intelligent system is a certain self-awareness, if we can use those terms, a self-awareness about its uncertainty. So it needs to be able to know the situation that it’s in is something that is unusual or it hasn’t seen before. And in the case of the self-driving car, for example, if it were to have a sense of its uncertainty, it would basically decide to slow down because it hasn’t encountered that situation before. It’s not confident that the horse isn’t a bicycle or whatever it is.
There are many different kinds of uncertainty. But the beauty of it is that from a mathematical point of view, we can boil it all down to probabilities. So we can map all these different forms of uncertainty onto probabilities and then use the rules of probability theory to manipulate uncertainty, update your state of uncertainty, etc.
HANNAH FRY (00:04:05 – 00:04:09): But how important is it that a machine can tell the different types of uncertainty apart?
ZOUBIN GHAHRAMANI (00:04:10 – 00:04:25): Yeah, I think it’s important insofar that the different types of uncertainty may mean different decisions. So, for example, if you have what’s called aleatoric uncertainty, which is the sort of randomness of a coin flip—
HANNAH FRY (00:04:26 – 00:04:27): which way is the pedestrian going to turn?
ZOUBIN GHAHRAMANI (00:04:27 – 00:05:00): Or which way is the pedestrian going to turn? Might want to decide that you’re going to give up on trying to predict because it is just random. Whereas in other cases, your state of belief is uncertain and you would want to collect more information to sort of— and in fact, that’s the definition of information, right? So information, a bit of information that we use in computer science, is the reduction of your uncertainty by a factor of 2. That’s what a bit is. And so collecting information is the way we reduce our uncertainty.
Uncertainty in Human Cognition
HANNAH FRY (00:05:00 – 00:05:04): How do these ideas of uncertainty map onto what humans are doing?
ZOUBIN GHAHRAMANI (00:05:05 – 00:05:47): So I actually studied cognitive science.
So we go through our lives perceiving things. Perceiving the world is fundamentally an act of sensing something that is uncertain. I can’t see the back of your head, but I can sort of infer what the back of your head might look like from sort of—
HANNAH FRY (00:05:47 – 00:05:48): You’d hope it’s there.
ZOUBIN GHAHRAMANI (00:05:48 – 00:07:03): I would hope it’s there. And, similarly, let’s say I’m hiking in Costa Rica. And suddenly there’s a rustling of the leaves. It may be a jaguar, right? And so my senses, through evolution, my senses have developed to take in perceptual information, take in my prior beliefs, because there are jaguars in Costa Rica, or there wouldn’t be jaguars in the middle of London. And that process of combining information has been modeled by cognitive scientists and psychologists and neuroscientists through the language of probabilistic inference, basically.
And also, actually, one of the interesting things is that we as humans are actually quite bad at estimating probabilities explicitly. Like, if you ask somebody what is the probability of a certain event, they might get it wrong by orders of magnitude, for example. And we have these fallacies of probabilistic inference and belief. Kahneman and Tversky showed that humans are actually quite bad at representations of uncertainty at a conscious level. But in our perceptual systems, unconsciously, we tend to be quite good about these things because our survival depends on them.
HANNAH FRY (00:07:03 – 00:07:13): I’m thinking also about babies here, or toddlers, and sort of the way that they learn is a lot about their belief in what action will drive a particular outcome.
ZOUBIN GHAHRAMANI (00:07:13 – 00:07:35): Yeah, I mean, it’s all implicit. Obviously, babies don’t know what probabilities are, and they won’t be able to write down any equations for you. But there are many cognitive scientists and psychologists who try to understand human learning, human perception, human decision-making using these same formalisms that we’re using for AI systems.
Correctness vs. Confidence
HANNAH FRY (00:07:35 – 00:07:46): I think there are a few things here we should probably tease apart before we get really into it, because there’s a difference between correctness and confidence, which both sometimes come under the umbrella of uncertainty.
ZOUBIN GHAHRAMANI (00:07:46 – 00:08:46): Yeah, absolutely. A great example of this is we use AI systems for image classification all the time. So you give it an image and it gives you an answer of what’s in the image. And we can measure correctness, but we also want it to tell us how confident it is.
And more than a decade ago, people discovered that you can take an image, for example, of a school bus, modify just a few pixels in that image in an imperceptible way. So a human being would look at it and say, well, that’s an image of a school bus. You give it to the neural network and it confidently says, “that’s a cheetah. 99%, that’s a cheetah.” And so, in fact, you could do it for any category. You could turn the school bus into a monkey or whatever in the eyes of the neural network.
And so that sort of adversarial example shows us that it’s not correctness that we care about alone. It’s actually correctness and confidence. We don’t want systems that can be overconfidently wrong.
HANNAH FRY (00:08:46 – 00:08:48): Yeah. Or can be fooled, I guess.
ZOUBIN GHAHRAMANI (00:08:49 – 00:08:52): Or can be fooled. And I think our systems can still be fooled in that way.
Overconfidence and the Path to AGI
HANNAH FRY (00:08:52 – 00:08:59): These ideas of overconfidence and humility, are these going to end up being important as we start to build AGI?
ZOUBIN GHAHRAMANI (00:09:00 – 00:09:44): Yeah, absolutely. I think we’ve advanced a great deal with systems that can acquire a lot of knowledge. But when we deploy them, for example, large language models, we’ve all encountered this. No matter what large language model you interact with, it will confidently give you an answer, and then you question it, and maybe it’ll then flip-flop and give you a different answer.
And so it’s hard to trust the system that is overconfident. It’s also hard to trust people that are overconfident, right? I think we would like all intelligent systems to have a sense of their limits, their knowledge, and we need to bake that into our AI systems for sure.
Early Days in AI: The 1980s
HANNAH FRY (00:09:44 – 00:09:52): Absolutely. I know that you’ve been working in AI for a really long time, right? I mean, since the late ’80s, basically.
ZOUBIN GHAHRAMANI (00:09:52 – 00:09:52): Yeah.
HANNAH FRY (00:09:52 – 00:09:53): What was the field like then?
ZOUBIN GHAHRAMANI (00:09:54 – 00:11:47): I’m pretty boring in that I was interested in AI as a teenager. So when I was 14 or 15, I already wanted to work in AI. When I went to university, I needed a summer job. So I went to the head of the computer science department, this very famous computational linguist, Aravind Joshi, at University of Pennsylvania. And I said, “I’m here for the summer. I’m a first-year student. I need a summer job.”
And what he said was, there are these 2 books that have come out. These were called Parallel Distributed Processing. This was in 1986, the same year that the backpropagation paper had come out that launched the whole neural network, kind of modern neural network revolution. And he gave me a summer job reading these books and explaining them to him, which was the most wonderful summer job one could possibly have if you’re interested and curious about things. So I learned about neural networks back then.
And you asked me, what was it like in the mid to late 1980s? Well, the dominant paradigm in AI was expert systems. So people were looking at rule-based systems that would make decisions, and they were quite brittle. And neural networks, which were sort of modeled after the human brain, were actually much more flexible. And fundamentally, they could learn from data in a way that previous methods were not very good at learning from data. So that was really a revolution back in the 1980s.
We talk about the transformer revolution, all that, but back in the mid-1980s, that was a real revolution. And it attracted people from cognitive science, from computer science, psychology, neuroscience, economics. And then I went on to write an undergraduate thesis on learning how to parse human natural language using neural networks. So it was one of the very earliest. I didn’t publish it, but if I’d published it, it would have been a very early paper on small language. Let’s call them small language models because my models were very small.
HANNAH FRY (00:11:47 – 00:11:48): Wait, what year was this?
ZOUBIN GHAHRAMANI (00:11:48 – 00:12:31): This is 1989, and you’re very much ahead of the game. Yeah. And we had a parallel computer. So now people think about data centers and all sorts of GPUs and parallel computers. And we had the most sci-fi, beautiful, iconic parallel computer of the time, which was this Connection Machine, which was this cube with 65,000 little red lights blinking inside this, kind of opaque cube. Because it had 65,000 processors. So I would sit there coding up in this parallel language neural networks for natural language. And it was pretty revolutionary at the time because neural networks were the counterculture. People were very enamored of the old school of AI.
A Pivot Toward Probability
HANNAH FRY (00:12:32 – 00:12:38): But then, okay, despite being ahead of the game by approximately, what, 40 years? Yeah.
ZOUBIN GHAHRAMANI (00:12:39 – 00:12:39): Then I messed it up.
HANNAH FRY (00:12:40 – 00:12:42): Yeah. Well, what happened?
ZOUBIN GHAHRAMANI (00:12:42 – 00:13:27): No, no. What happened is actually really fascinating. So I was working in neural networks, and at the time— this is now the early 1990s— we felt like we understood how they worked. So neural networks are these amazing function approximators. We can feed them data, they can map from inputs to outputs, from X to Y, from images to labels of images, and so on. And remember, the datasets were very small at the time. So when I was writing my undergraduate thesis, the World Wide Web didn’t even exist. Compute was also very limited. So we felt like we understood neural networks. We felt like they’re nice function approximators. But people started to uncover these beautiful relationships between neural networks and other ideas from probability theory, statistics, stochastic processes.
HANNAH FRY (00:13:27 – 00:13:32): I mean, you actually gave up working on neural networks in favor of these probability questions.
ZOUBIN GHAHRAMANI (00:13:32 – 00:13:33): Yeah, yeah, absolutely.
Neural Networks, Bayesian Thinking, and the Problem of Uncertainty
Hannah Fry (00:13:34 – 00:13:40): Was it annoying, though, that in the end neural networks were the thing and you were there so much earlier than anyone else?
Zoubin Ghahramani (00:13:40 – 00:14:28): Yeah, well, I didn’t— I did— redeem myself. I spent a number of years working with Geoff Hinton. He went on to win the Nobel Prize for his work on neural networks. So we were attempting to do deep learning, but we were using probabilistic models rather than simple neural networks. We were overcomplicating things because we hadn’t seen what happens with large-scale data. And then I went off and worked on Bayesian machine learning and probabilistic models and all these things for many years. But I never totally dismissed neural networks, actually, because I knew that they work. It’s just that they weren’t sort of, from a research point of view, they weren’t as interesting to me because the mathematics was sort of, at the time, we felt well understood.
The 2015 Nature Paper on Probabilistic Uncertainty
Hannah Fry (00:14:29 – 00:14:47): In 2015, though, you wrote this seminal Nature paper which appeared in the same issue as all of the foundational deep learning and reinforcement learning papers by Hinton. And in that paper, you argue many aspects of intelligence depend crucially on the careful probabilistic representation of uncertainty.
Zoubin Ghahramani (00:14:48 – 00:14:48): Yes.
Hannah Fry (00:14:48 – 00:14:50): I mean, has everyone heard you?
Zoubin Ghahramani (00:14:50 – 00:15:33): Have they heeded your warning? No, no. I think there are many people who understand that and believe that. I think that it depends on the level that you think about things. So actually, if you look at large language models, they are probabilistic models. They predict the probability of the next token or word given a sequence of previous tokens. So probabilities are at the heart of everything we do in machine learning. But what’s missing is we’re not really doing what I said, which is the careful representation of probabilities. We’re actually sort of hoping that the models represent probabilities okay because we’ve trained them on enough data.
Hannah Fry (00:15:33 – 00:15:35): But not thinking about it in an explicit way.
Zoubin Ghahramani (00:15:35 – 00:16:00): Yeah. If you look in a giant neural network, you can’t really find the explicit representation of, say, the probability that it thinks something or other. It’s sort of spread out somehow over all the activations of the billions of units in the neural network.
What Is Bayesian Thinking?
Hannah Fry (00:16:01 – 00:16:10): And by contrast, I mean, you’re more of a, I guess, a Bayesian thinker. Yeah. Just explain for anybody who hasn’t come across this before, just explain to us what that actually means.
Zoubin Ghahramani (00:16:10 – 00:16:37): Yeah. So Bayes’ rule is this fascinating and very simple concept from probability theory. So before you observe something, before you get some evidence or data, you have what’s called prior beliefs. You represent those with a probability distribution. So, for example, think of a detective story, like a whodunit. There is a number of suspects, and you may have some prior beliefs about, like, it’s the butler that did it or whatever, right?
Hannah Fry (00:16:38 – 00:16:39): He’s looking suspicious.
Zoubin Ghahramani (00:16:40 – 00:18:28): Yeah, somebody’s looking suspicious or something, right? So you have some beliefs, you represent those with a probability distribution, and there are many, many reasons why probability theory is the right way of representing beliefs. Whole, like, branches of mathematics that have proven that.
And now you observe some evidence, like, the murder weapon is found in the pantry or something like that, right? And so you take your prior beliefs, multiply them by what’s called the likelihood, the probability under each possible culprit, and then you renormalize because probabilities have to sum to 1. And from that, you get your posterior beliefs, your new state of knowledge. And through that evidence, by the way, you’ve gained information literally measured in bits, how much your uncertainty has decreased.
And now, if you get more evidence, you just take your current posterior probability distribution, which is now your new prior, and you repeat and rinse. You do it again. You get the new evidence, you update the probabilities, and so on and so forth.
And through that application of Bayes’ rule, we can model both perception— like, I open my eyes, I see something, then I see more things, and I know it’s not a jaguar that’s following me in Costa Rica— but you can also model what learning is. So learning is you have a model, the model has parameters. At the beginning, you don’t know what the parameters should be. You get some data, and you update the model parameters. Sequentially through that data, applying Bayes’ rule in theory. That’s the sort of beautiful model that I had been pushing forward as a model of learning.
Hannah Fry (00:18:28 – 00:18:53): Because I guess, I mean, on the one hand, this is a very elegant mathematical way to link together evidence and unknowns and knowns. Yeah. But I guess actually on an intuitive level, I mean, this sort of is the way that our brains work. I mean, going back to your example of the detective, you know, it’s like, oh, I did think it was that person, but now this new evidence has come in and I’ve changed my mind. Exactly. And that’s essentially a one-sentence description of what Bayes’ rule is doing, right?
Zoubin Ghahramani (00:18:53 – 00:18:54): Exactly. Yeah, yeah.
Hannah Fry (00:18:54 – 00:18:57): But this is a formal way to get AI to be able to do it.
Zoubin Ghahramani (00:18:57 – 00:19:39): Exactly. So we would like to build AI systems that accumulate knowledge and information over time, and we would like them to be rational, sort of like Data from Star Trek is a very rational being. We don’t want them to flip-flop around unpredictably based on no evidence. And I would argue that ideally we would like them to be even more rational than humans. I want my AI, just like I want my calculator to be really good at multiplying large numbers. I would actually like our AI systems to be more rational, better at representing and manipulating probabilities than humans are.
How Good Are Today’s AI Systems at Representing Uncertainty?
Hannah Fry (00:19:39 – 00:19:43): Okay. If we fast forward to today, I mean, the paper you wrote was a decade ago.
Zoubin Ghahramani (00:19:43 – 00:19:44): Decade, yeah.
Hannah Fry (00:19:44 – 00:19:51): How good really are the AI systems that we’re all used to playing around with, how good are they at representing uncertainty?
Zoubin Ghahramani (00:19:51 – 00:19:54): Yeah, they’ve been getting better, but they’re not very good.
Hannah Fry (00:19:54 – 00:19:55): Right.
Zoubin Ghahramani (00:19:57 – 00:20:13): And you can tell that because if you interact with a large language model and it asserts something, you can ask it, how confident are you? And it might say something back, but it’s really doing next token prediction. It doesn’t have an explicit representation of how confident it is in that statement.
Hannah Fry (00:20:13 – 00:20:14): So it’s not calculating Bayes’ rule.
Zoubin Ghahramani (00:20:15 – 00:21:02): It’s not calculating Bayes’ rule, at least not explicitly. It may be because you’ve trained it on trillions of tokens of stuff on the web. It’s mimicked a lot of other kinds of reasoning traces and so on. It’s sort of faking it, right? And you can tell it’s faking it because then if you push back and you say something silly like, “no, I think you’re wrong,” then it might respond, “oh, sorry, yes, I was wrong.” So it’s not really coherent. It’s not going to stand its ground. And our systems are getting better at factual grounding and things like that. But we haven’t really nailed the idea of how one of these models should be able to represent probabilities over its beliefs.
Hannah Fry (00:21:02 – 00:21:09): Because that, I mean, I feel like that would be a very useful feature for a large language model to be able to say it’s not sure. Why do they struggle with that so much?
Zoubin Ghahramani (00:21:10 – 00:21:52): They struggle because the paradigm for training them hasn’t prioritized that. We train them to be just really good at modeling the data. If the data involves a lot of human reasoning by many different humans with many different beliefs, then what you get is a soup. You get sort of a mishmash of everything. But if we want to build, like I said, self-driving cars, or robots, they need to be able to reason about the real world. They need to have an understanding of cause and effect. They need to have an understanding of their own uncertainty. And they need to use that to be able to act in the real world in a safe way.
Hallucinations and the Problem of Factual Grounding
Hannah Fry (00:21:54 – 00:21:57): But then, I mean, I’m just thinking about hallucinations here.
Zoubin Ghahramani (00:21:57 – 00:21:58): Yeah.
Hannah Fry (00:21:58 – 00:22:03): Because that’s part of this as well, right? There are sometimes where you just want— there’s a verifiable fact that it’s getting wrong.
Zoubin Ghahramani (00:22:04 – 00:22:04): Yeah.
Hannah Fry (00:22:04 – 00:22:07): And that all plays into this too.
Zoubin Ghahramani (00:22:07 – 00:23:08): Yeah. I mean, hallucinations are a symptom, right? Of course, sometimes we want our models to hallucinate. So there is a tension between not hallucinating at all and not being creative, right? For example, if I want my large language model to write me a short story of that time that Albert Einstein went to the moon in a rocket, that’s clearly a hallucination, but it’s an act of creative writing. So it should be able to do that. It should be able to infer that that’s what you want, infer your intent.
If I ask it a factual question, it should try to be grounded. And of course, at Google, we think a lot about grounding our models in what’s available. And even information on the web is often contradictory, right? And so you want to hedge your bets. So actually, what you would like is a system that’s able to tell you for any statement some estimate of its belief or probability. I think that’s what we want. We don’t have it yet.
Semantic Entropy and Building Uncertainty Into AI
Hannah Fry (00:23:08 – 00:23:22): No. How might you build it if you wanted large language models to have uncertainty in there? How do you do it? Because there are some quite good ideas. I mean, I’m thinking about semantic entropy here, right? Which is one of your students came up with. Tell me a little bit about that. How might that work?
Zoubin Ghahramani (00:23:22 – 00:24:02): Yeah, I mean, you can take the internals of a particular model and try to infer from that its degree of belief. So before it answers, before it produces a token in a large language model, you actually have a probability distribution over all possible next tokens. And that probability distribution, the entropy of that distribution tells you something about the uncertainty. A low entropy distribution is very spiky. Is very certain. A high entropy distribution is very spread out. It’s very uncertain. And so there are ways of sort of teasing from the internals of a model how certain it might be.
Hannah Fry (00:24:03 – 00:24:12): Yeah, I guess I’m thinking here about an example. Let’s say the Eiffel Tower in Paris, right? Like, those 2 things would be an example of something that’s quite spiky.
Zoubin Ghahramani (00:24:13 – 00:24:27): Yeah, sort of like if you ask it, where is the Eiffel Tower? It should have a high probability over Paris, although I believe there’s one in New York as well, isn’t there? There’s a little Eiffel Tower.
Hannah Fry (00:24:28 – 00:24:29): Vegas as well, I think they’ve got one.
Zoubin Ghahramani (00:24:29 – 00:24:40): Maybe there’s one in Vegas as well. So basically, depending on the context, you might have little bits of entropy on these other Vegas and New York as options, but Paris would have a big spike on it.
Hannah Fry (00:24:40 – 00:24:53): Is that sort of the idea then, that like in the dataset, Paris and the Eiffel Tower appear near each other a lot? You have a lot of like a really big signal there, whereas, I don’t know, the Eiffel Tower and sort of Marrakesh.
Zoubin Ghahramani (00:24:53 – 00:25:28): Yeah, that might have very low probability. The problem with that is that’s sort of faking it in the sense that you’re relying on the data. I’ll give you the analogy of a calculator. It’s like trying to build the calculator just by showing it examples of addition and multiplication. But imagine you never show it a particular number. Then it might not generalize to that particular number. And you don’t want to fake a calculator. You want a calculator that actually calculates. You want it to actually reason about the world that it’s in. That’s what we want from our AI systems.
Why Is Building True Uncertainty So Hard?
Hannah Fry (00:25:29 – 00:25:34): So why hasn’t this been done yet? I mean, what is it that makes it so hard to do?
Zoubin Ghahramani (00:25:34 – 00:26:23): It is genuinely hard because it’s computationally hard. So essentially, I would argue we had all the ingredients of AI maybe 15 or 20 years ago. Like, we kind of know how to build rational systems. We just thought, well, representing probability distributions over every possible thing is computationally intractable. It would take giant supercomputers that we don’t have, and it would be incredibly slow. You’d be waiting for millions of years before you get the answer. So people abandoned that for, let’s go with just training from data. And I think we can revisit this idea. And I have ideas for how to do this, and I’m exploring them now, but it’s not sort of super well-formed yet.
Uncertainty in Weather Forecasting
Hannah Fry (00:26:23 – 00:26:41): Okay, so moving away from the large language model, the transformer-type system, then, for a moment, because, I mean, there are other examples of really cutting-edge artificial intelligence which does handle uncertainty in more of this Bayesian way that you’re describing? I’m thinking about weather forecasting here.
Zoubin Ghahramani (00:26:41 – 00:26:41): Ah, yes.
Hannah Fry (00:26:42 – 00
Weather Forecasting and Uncertainty
Zoubin Ghahramani (00:26:46 – 00:28:07): Yeah. So we’ve developed a whole series of state-of-the-art weather forecasting models at Google DeepMind. And if you look at the GenCast model, sort of very recent model, one of the key features it has, it can predict sort of weather over 15 days. And it can do it very fast, much faster. It can do it sort of like in 8 minutes rather than on a giant supercomputer for hours. And it’s obviously using neural networks and things like that.
But a key ingredient for getting this to work is that it uses a diffusion model. So that already is like the image generation models that already is manipulating probability distributions over time. But then it generates an ensemble of forecasts. So if you’re trying to track a tropical storm like Hurricane Melissa, let’s say, which we did with this model, you have the data so far, and then you want to be able to forecast the track of this into the future because your decisions depend on that, whether you evacuate a city or whether you, you know, call in emergency services and so on. And so it represents that with an ensemble of forecasts. It has a whole probability distribution over the possible tracks.
Hannah Fry (00:28:07 – 00:28:09): You rerun the model over and over and over and over again.
Zoubin Ghahramani (00:28:09 – 00:28:23): And then as you get more data, so, you know, a few hours later you get more observations, you update that ensemble. And that is essentially applying these basic ideas from Bayesian updating to this sort of problem.
Hannah Fry (00:28:23 – 00:28:27): Because, I mean, the original weather forecasting models that came out of here did not have this.
Zoubin Ghahramani (00:28:27 – 00:28:35): Yeah, that was sort of added on. And every time we add on, you know, these features, it makes the model better.
Hannah Fry (00:28:36 – 00:28:41): Because you’re essentially saying there is inherent uncertainty in the way that weather works.
Zoubin Ghahramani (00:28:41 – 00:28:42): Absolutely.
Hannah Fry (00:28:42 – 00:28:48): You can’t just sort of run the model once and be like, oh, well, that’s going to be the weather tomorrow. Yeah. You have to do it lots of times and then work out a probability.
Zoubin Ghahramani (00:28:49 – 00:29:19): Yeah, yeah. It’s a combination of both that inherent randomness a sort of classic butterfly effect in weather in that it’s chaotic, it’s very hard to predict. But also there is an uncertainty just because we have limited numbers of sensors, right? So it’s a system that represents its beliefs about the weather, whether it’s the trajectory of this hurricane and its intensity. And then those beliefs get updated as you take in the sensor measurements and as time progresses.
Hannah Fry (00:29:20 – 00:29:26): I think there’s something quite delightfully counterintuitive about that, that you add in uncertainty to the system.
Zoubin Ghahramani (00:29:27 – 00:29:27): Yeah.
Hannah Fry (00:29:27 – 00:29:30): And it makes the predictions more accurate.
Zoubin Ghahramani (00:29:30 – 00:30:14): Exactly. Yeah. I think it is definitely a key insight in AI. I mean, it’s not that you’re sort of making the system noisy in an arbitrary way. That’s not going to help you. But what you’re doing is you’re being honest about the fact that your sensors are inaccurate. Sometimes you get faulty sensors, just like sometimes my ears are blocked and I can’t hear very well. And sometimes your model assumptions are wrong, right? That is also a form of uncertainty. And so all of those different forms of uncertainty have to be represented somehow, approximately. We can’t do it all exactly. So that we can get better calibrated forecasts.
AlphaFold and Representing Uncertainty
Hannah Fry (00:30:15 – 00:30:29): I think the other example that really manages to get this uncertainty idea right is AlphaFold, where the protein prediction, I mean, is color-coded by how sure the model is that that’s the correct folding, right?
Zoubin Ghahramani (00:30:29 – 00:31:00): Yeah, absolutely. And essentially, you’re fundamentally trying to solve an uncertain problem. You’re going from a sequence to the folded structure. And the physics itself means that some parts of the protein are going to wiggle around more. So you don’t know exactly where they are. And you also have just uncertainty because you’ve used a model to predict that. It’s not like experimental data. And so you need to be able to represent that uncertainty in the sort of cloud of forecasts of where the molecules are.
The Danger of Over-Hedging
Hannah Fry (00:31:01 – 00:31:12): There is a danger here that you could go too far the other way. You could end up with a model that was so honest about his uncertainty that it just sort of didn’t ever really give you an answer.
Zoubin Ghahramani (00:31:12 – 00:31:13): It just always said, “I don’t know.”
Hannah Fry (00:31:13 – 00:31:15): Just hedging. Hedging all the time.
Zoubin Ghahramani (00:31:15 – 00:31:19): That would be pretty funny. But, you know, I think there’s different degrees of “I don’t know.”
Hannah Fry (00:31:19 – 00:31:21): How do you get the balance right then?
Zoubin Ghahramani (00:31:21 – 00:32:21): Yeah, I think the balance is if there is a repeatable event in the real world, then you want to be calibrated in that if I say the chance of rain is 0.7 or 70%, then for all days that I’ve said that, if I sum up over all those days, if I’m calibrated, then on 70% of those days it actually rained, on 30% it didn’t rain. So that’s a calibrated probability.
Now, if you ask me a statement of uncertainty about something that is not a repeatable event, so for example, we may be uncertain about the first day a human being reaches Mars. So that’s a date. It’s either going to happen or it’s not going to happen. And that date, when it happens, you’ll be certain of it. That’s the sort of event that, you know, you can have probabilities over, you can have beliefs over, and it will resolve itself when it happens, if it happens.
Hannah Fry (00:32:22 – 00:32:28): And Bayes also has a way of handling that where it’s not just 50%, right? It’s not just half a dozen.
Zoubin Ghahramani (00:32:28 – 00:32:42): Yeah, absolutely. So what Bayesian statistics tells you is that it’s perfectly valid, and in fact the right thing to do, to use probabilities to represent your degree of uncertainty about things that only happen once.
Communicating Uncertainty to Human Users
Hannah Fry (00:32:42 – 00:32:55): How do you communicate that uncertainty in a way that actually means it adds value rather than just has a human sort of nodding along or kind of taking cognitive shortcuts when the machine says it’s really confident?
Zoubin Ghahramani (00:32:55 – 00:34:43): Yeah, I think people have different reactions to AI systems. Some people are just very skeptical and will kind of not believe anything the AI system tells it. Other people are going to end up being overreliant, let’s say.
Let’s imagine a future where we’ve got not a very distant future, but imagine a future where we’ve got AI systems in a medical domain, for example. And you have doctors aided by an AI system looking at patients and symptoms and test results and so on. It’s a great example of why we need uncertainty. If the AI system says something, you really want it to convey its uncertainty because that’s literally what is going to determine your treatment plan or whether you take one decision or another. These could be life and death decisions, right?
So it’s absolutely essential that, first of all, we don’t become over-reliant on overconfident AI systems. But to be able to do that, we need our AI systems to be honest about their uncertainty and bring that uncertainty in a visible form to the human users so we can understand it. And I mean, my colleague at Cambridge, David Spiegelhalter, has done a lot of amazing work on how to convey uncertainties and probabilities in elegant ways. To the general public. And I think there are ways you can do that. You can visualize the answer and so on.
So, I mean, I really believe that it’s important to have AI systems that are doing complementary things, that are additive to humans, that are helping people solve problems that we care about. And in order to do that, you want them to be honest about what they know and what they don’t know.
Hannah Fry (00:34:44 – 00:34:47): It feels like this is really very critical that we get this right. You know?
Zoubin Ghahramani (00:34:47 – 00:34:47): Yeah.
Hannah Fry (00:34:47 – 00:34:51): It’s like such an important important part of designing our collective future with AI?
Zoubin Ghahramani (00:34:51 – 00:35:12): Yeah, I think it is really important. I mean, I don’t want to take away from the fact that AI systems have been incredibly useful already. You know, I think they’re pretty good at some of these things, and we can do better. And there are open problems along the way. And, you know, that’s one of the reasons we actually need more research, actually.
The Bayesian Debate in AI
Hannah Fry (00:35:12 – 00:35:22): The thing is, I mean, I’m sort of sitting here agreeing with you. You’re sort of also a Bayesian thinker. So, sort of, yeah, this is very much my philosophy. But not everybody does.
Zoubin Ghahramani (00:35:22 – 00:35:24): No, no, that’s right.
Hannah Fry (00:35:24 – 00:35:32): Not everyone agrees with you. I mean, there are some people who sort of say, look, you just put in more data, and then you don’t need to worry about uncertainty because the model will know everything.
Zoubin Ghahramani (00:35:32 – 00:36:21): Yeah, I think that is a view that a lot of people have in the field. And they’re not completely wrong, just like I’m not completely right in that the models are actually pretty good at a lot of useful things. The problem is that when you stretch them in the long tail of sort of unusual things, then you can uncover some gaps. And also, I think when the decisions— if you’re just interacting with a chatbot, the decisions may not be so consequential. But if we’re trying to build self-driving cars that are reliable, or medical AI systems that are helping us make diagnosis decisions, then we really do care about getting those probabilities right.
Hannah Fry (00:36:21 – 00:36:42): Because I guess when it comes to building AGI, there’s sort of, to oversimplify it, 2 camps, really. One which says you just need scale, you just need more data, more compute, off you go. And then the other that’s sort of known, we actually need a new architecture to be able to do things better that we can’t do at the moment. It sounds like you’re very much in the second camp.
Zoubin Ghahramani (00:36:42 – 00:37:42): Yeah, I think we’ve made a lot of progress in the first camp. So of course our systems are incredibly useful. They’re used by billions of people every day. But that doesn’t mean that we’ve run out of interesting things to discover. So I’ll give you a few examples of things that I think are important areas of research.
One example is continual learning. So the way we currently train our models, and we means everybody in the field, we train a giant model and then we use it in products or we release it in various forms. And then a few months later, we train another giant model and so on. If you compare that to how humans and animals learn, we learn continuously. We’re basically constantly getting a stream of data and constantly adapting our connections between our neurons and so on. And our AI systems are not really able to do that very well. They suffer from things like catastrophic forgetting and so on and so forth.
Hannah Fry (00:37:42 – 00:37:57): Which absolutely links back to the idea of Bayesian thinking that we were talking about earlier, because the reason why humans, animals are able to do that is because we have this updating system in our minds of incorporating evidence with existing knowledge.
Continual Learning and Energy Efficiency
Zoubin Ghahramani (00:37:57 – 00:38:46): Yeah. So it turns out if you think of learning from a strictly Bayesian point of view. You have your prior beliefs, you get a data point, you update them, you get a posterior, and you get another data point, you update them. It turns out that that sort of Bayesian updating can do continual learning and does not suffer from catastrophic forgetting and all these things, in theory. Actually, many of our attempts at doing continual learning in large neural networks are approximations of that Bayesian updating. So that’s one area of research is continual learning.
Another area where I think we may need breakthroughs is energy efficiency. So if you look at the power consumption of a human brain, it’s about 20 watts. I don’t want to make an equivalence.
Hannah Fry (00:38:46 – 00:38:47): It’s a light bulb.
Zoubin Ghahramani (00:38:47 – 00:38:48): Yeah, it’s a light bulb.
Hannah Fry (00:38:48 – 00:38:49): It’s a rubbish light bulb.
Zoubin Ghahramani (00:38:50 – 00:39:14): Yeah, it’s a nice, very energy-efficient light bulb, let’s say. If you compare that to training a large language model in a big data center, it’s orders of magnitude off. I don’t want to compare a single brain to a large language model because single brains involve kind of the single lived experience of a human being. Large language models are basically giant soups of all of world knowledge of some kind.
Hannah Fry (00:39:14 – 00:39:16): But they’re definitely less efficient.
Zoubin Ghahramani (00:39:16 – 00:40:03): But they’re way less efficient. Right. So we can certainly afford to do more research in more energy-efficient learning. I also think that there may be new architectures that we need to discover. Basically, the 2 workhorses of modern AI are transformers and diffusion models. They’re great, but there may be completely other co-evolutions of software and hardware, like novel hardware architectures that may involve very sparse neural networks of various kinds. And, oh, I’ll give you another one. Our learning systems are incredibly data inefficient compared to human and animal learning. So again, I think Bayesian ideas can help us there.
Hannah Fry (00:40:04 – 00:40:15): Sort of sounds a bit like you’ve discovered a magic trick, you know, like this magic trick which, like, embeds humility and uncertainty, offers the opportunity for continual learning, and makes it data more efficient.
Zoubin Ghahramani (00:40:15 – 00:40:15): Yeah.
The Computational Cost of Ideal AI Systems
Hannah Fry (00:40:15 – 00:40:20): And you can write it in a single line of Bayes’ rule. So sort of like, is this a little bit too good to be true?
Zoubin Ghahramani (00:40:20 – 00:41:26): It’s too good to be true, right? Obviously, we’ve known this for a long time. I think it’s important to understand these ideas. So I think all students of machine learning should at least understand that it’s possible to do these things. The magic trick comes with a big curse. The curse here is that to do all of this is computationally very slow.
So essentially, if you look at textbooks in AI, they will explain to you how to do some of these things, but they will say we can’t do these things exactly because they’re computationally hard problems. They’re like, kind of NP-complete or NP-hard. Problems to solve, and so we need to approximate them somehow.
And you could argue, well, we know how to build ideal AI systems using these concepts. Our modern AI systems are approximations to that. Can we have our cake and eat it too? Can we have the best of both worlds here?
Hannah Fry (00:41:26 – 00:41:34): Because the thing is, what you just said there about, well, it’s very computationally expensive, it’s very slow. They were saying that about neural networks in the ’80s.
Zoubin Ghahramani (00:41:34 – 00:41:35): Yeah.
Hannah Fry (00:41:35 – 00:41:36): You’re not going to make that mistake twice.
Revisiting Old Ideas with New Compute Power
Zoubin Ghahramani (00:41:37 – 00:42:16): Yeah, I think we can revisit some of these ideas with the compute power that we have now. The giant state-of-the-art supercomputer, parallel connection machine computer that I used in my undergraduate years is actually slower than the Pixel phone that I have in my pocket. Computation is getting better, faster. And also, we have decades of ideas on how to approximate these methods really efficiently, right? And so we have the tools, we just need to put them together and maybe come up with a few new ideas.
Hannah Fry (00:42:16 – 00:42:42): What really strikes me is everything we’ve discussed, they feel like very human like qualities, right? These sort of ineffable characteristics of humility and honesty and uncertainty and doubt. I mean, did you imagine when you started all these years ago that this would be the thing that our systems are lacking rather than just computational power or power for analysis?
Intelligence That Transcends Human Flaws
Zoubin Ghahramani (00:42:42 – 00:43:52): I mean, certainly I didn’t really imagine we would be where we are now, right? Because I think if you talk to any AI researcher, they will be saying that they’re stunned by the rate of progress. But I also think that although these are human qualities we want to add, they’re also kind of fundamental qualities of intelligence systems.
So I think we need to have a concept of intelligence that transcends humans, because, as I mentioned before, we have flaws in our reasoning. We’re actually quite bad at making good rational decisions under uncertainty in the real world. We will miscalculate probabilities or misestimate probabilities and so on.
So I think if we build human-centric AI systems, we work backwards from, well, what do humans need? What are sort of society’s biggest problems? What are humanity’s biggest problems? And what are the AI systems that we need to solve those things? And for, I think, all problems that matter, I would rather have an AI system that knows when it doesn’t know than an AI system that is arrogant and overconfident.
Hannah Fry (00:43:53 – 00:43:57): What an amazing point to end on. Zoubin, thank you so much. That was brilliant.
Zoubin Ghahramani (00:43:57 – 00:43:58): Thank you, Hannah.
Closing Thoughts: Embracing Uncertainty
Hannah Fry (00:43:58 – 00:44:36): For years, we tried to build AI that was focused on accuracy, right? Building systems that crunch through enormous amounts of data to land an answer. And these things, they’re astonishingly capable, of course, but they’re also quite brittle in some ways. When they fail, they fail with total confidence.
But by teaching AI to embrace uncertainty, it gives it something more human— humility and honesty and the wisdom to doubt. And this isn’t a vulnerability. It doesn’t make AI weaker. It grounds it in reality. Making it a collaborator we can actually rely on.
Related Posts
- Transcript: Howard Lutnick Interviews Anthropic’s Tom Brown at G20 Innovation Summit
- Transcript: Sam Altman Interview on OpenAI, New AI Model and Global Growth – G20 Innovation Summit
- Transcript: The OpenAI–Hugging Face Incident – Black Hat USA 2026
- Transcript: Emad Mostaque Interview on Peter McCormack Show
- Andrew Ng: The Biggest Opportunities in AI Aren’t Where You Think (Transcript)