François Chollet: How We Get To AGI (Transcript)

It’s the ability to blaze fresh trails and build new roads. Attributing intelligence to what’s really a crystallized behavior program, a skill program, that’s a category error. You are confusing the process and its output. So don’t confuse the roads and the process that created the roads.

Formalizing Intelligence

To formalize this a bit, I see intelligence as the conversion ratio between the information you have, mostly your past experience, but also any developer-imparted priors that the system might have, and your operational area over the space of potential future situations that you might encounter. And that’s going to feature high novelty and uncertainty. So intelligence is the efficiency with which you operationalize past information in order to deal with the future. It’s an efficiency ratio.

And that’s the reason why using exam-like benchmarks to evaluate AI models is a bad idea. They’re not going to tell you how close we are to AI, because human exams weren’t designed to measure intelligence. They were designed to measure task-specific skill and knowledge. They were designed according to assumptions that are sensible for humans but not for machines. For instance, most exams assume that you haven’t read and memorized all the exam questions and the answers beforehand.

Key Concepts for Measuring Intelligence

If you want to rigorously define and measure intelligence, here are some key concepts that you have to take into account.

The first is the distinction between static skills and fluid intelligence. So between having access to a collection of static programs to solve known problems versus being able to synthesize brand-new programs on the fly to face a problem you’ve never seen before. And of course, it’s not a binary. It’s not one or the other. There’s a spectrum between the two.

The second concept is operational area. For a given skill, there’s a big difference between being skilled only in situations that are very close to what you’ve seen before and being skilled for any situation within a very broad scope. For instance, if you know how to drive, you should be able to drive in any city, not just in a specific geofenced area. I can learn to drive in San Jose and then move to Sacramento and you can still drive, right? Again, there’s a spectrum there. It’s not binary.

And lastly, you should look at information efficiency. For a given skill, how much information, how much data, how much practice did you need to acquire that skill? And of course, higher information efficiency means higher intelligence.

The reason these definitions matter a lot is that as engineers, we can only build what we measure. So the way we define and measure intelligence is not a technical detail. It really reflects our understanding of the problem of cognition. It scopes out the questions we are going to be asking, and so it determines the answers that we are going to be getting. It’s the feedback signal that drives us towards our goals.

And a phenomenon you see constantly in engineering is the shortcut rule. So it’s the fact that when you focus on achieving a single measure…

Measuring Success and the Paradox of Targeting

of success, you may succeed, but you will do that at the expense of everything else that was not captured by your measure. So you hit the target, but you miss the points. And you see this all the time on Kaggle, for instance. We saw it with the Netflix prize, where the winning system was extremely accurate, but it was way too complex to ever be used in production. So it ended up never being used. It was effectively pointless.

We also saw it in AI with chess playing. The reason the AI community set out to create programs that could play chess back in the 70s was because people expected it to teach us about human intelligence. And then a couple of decades later, we achieved a goal when Deep Blue beat Kasparov, the world champion. And in the process, we had really learned nothing about intelligence. So you hit the targets, but you miss the points.

For decades, AI has chased task-specific skill, because that was our definition of intelligence. But this definition only leads to automation, which is exactly the kind of system that we have today. But we actually want AI that’s capable of autonomous invention. We don’t want to stop at automating known tasks. We want AI that could tackle humanity’s most difficult challenges and accelerate scientific progress. That’s what AGI is meant to be. And to achieve that, we need a new target. We need to stop targeting fluid intelligence itself, the ability to adapt and invent.

So one definition of AGI only unlocks automation. So it increases economic productivity. Obviously, it’s extremely valuable. Maybe it also increases unemployment. But the other definition unlocks invention and the acceleration of the timeline of science. And it’s by measuring what you really care about that we’ll be able to make progress. So we need a better target. We need a better feedback signal.

The ARCGI Benchmark: A New Measure of Intelligence

What does that look like? My first attempt at creating a way to measure intelligence in AI systems was the ARCGI benchmark. So I released ARC1 back in 2019. It’s like an IQ test for machines and also humans. ARC1 contains 1,000 tasks like this one here. And each task is unique. So that means that you cannot cram for ARC. You have to figure out each task on the fly by using your general intelligence rather than your memorized knowledge.

And of course, solving any problem always requires some knowledge. And in the case of most benchmarks, the knowledge barriers that you need are typically left implicit. In the case of ARC, we made them explicit. So all ARC tasks are built entirely on top of core knowledge barriers, which are things like objectness, elementary physics, basic geometry, topology, counting. So concepts that any 4-year-old child has already mastered. And solving ARC requires very little knowledge, and it’s knowledge that is very much not specialized. So you don’t need to prepare for ARC in order to solve it.

And what makes ARC unique is that you cannot solve it purely by memorizing patterns. It really requires you to demonstrate through the intelligence. And meanwhile, pretty much every other benchmark out there is targeting fixed, known tasks. So they can’t actually be solved or hacked via memorization alone. That’s what makes ARC fairly easy for humans but very challenging for AI. And when you see a problem like this, where a human child can perform really well, but the most advanced, the most sophisticated AI models out there struggle, that’s like a big red flashing light telling you that we’re missing something, that new ideas are needed.

One thing I want you to keep in mind is that ARC is not going to tell you whether a system is already AGI or not. That’s not its purpose. ARC is really a tool to direct the attention of the research community towards what we see as the most important unsolved bottlenecks on the way to AGI. So ARC is not the destination, and solving ARC is not the goal. ARC is really just an arrow pointing in the right direction.

And ARC has completely resisted the pre-training scaling paradigm. Even after a 50,000x scale-up of pre-trained base alarms, the performance in ARC stayed near zero. So we can decisively conclude that fluid intelligence does not emerge from scaling up pre-training. You absolutely need test and adaptation in order to demonstrate genuine fluid intelligence. And importantly, when the arrival of test and adaptation happened last year, ARC was really the only benchmark at the time that provided a clear signal about the profound shift that was happening. Other benchmarks were saturated, so they could not distinguish between a true IQ increase and just brute-force scaling.

Evolution from ARC1 to ARC2

So now you see this graph and you’re probably asking, well, clearly at this point ARC1 is also saturating. So does that mean we have human-level AI now? Well, not yet. What you see on this graph is that ARC1 was a binary test. It was a minimal reproduction of fluid intelligence. So it only really gives you two possible modes. Either you have no fluid intelligence, in which case you will score near zero, like Bayes’ algorithms, or you have non-zero fluid intelligence, in which case you will instantly score very high, like the O3 model from OpenAI, for instance. And of course, every one of you in this room would score within noise distance of 100%. So ARC1 saturates well below human-level fluid intelligence.

And so now we are in need of a better tool, a more sensitive tool, that would provide more useful bandwidth and better comparison with human intelligence. And that tool is ARCGI2, which released in March this year. So back in 2019, ARC1 was meant to challenge the deep learning pattern, where models are big parametric curves used for static inference. And today, ARC2 challenges reasoning systems. It challenges the test adaptation pattern.

The benchmark format is still the same. There’s a much better focus on probing compositional generalization. So the tasks are still very feasible for humans, but they’re much more sophisticated. And as a result, ARC2 is not easily brute-forceable. In practice, what this means is that in ARC1, for many tasks, you could just look at it and instantly see the solution without having to think too much about it. With ARC2, all tasks require some level of deliberate thinking, but they still remain very feasible for humans.

And we know this because we tested 400 people firsthand, in person, in San Diego, over several days. And we are not talking about people who have physics PhDs here. We recruited random folks, Uber drivers, UCDS students, people who are unemployed, so basically anyone trying to make some money on the side. And all tasks in ARC2 were solved by at least two other people that saw it. And each task was seen on average by about seven people. And so what that tells you is that a group of 10 random people with majority voting would score 100% on ARC2. So we know these tasks are completely doable by regular folks with no prior training.

Current AI Performance on ARC2

So how well do AI models do? Well, if you take Bayesian alarms, models like GPT-4.5, LAMP4, it’s simple, they get 0%. There is simply no way to do these tasks simply via memorization. Next, if you look at static reasoning systems, so systems that use a single chain of tasks that they generate for the task, they don’t do much better. They do on the order of one to 2%. So very much within noise distance of zero. So what it tells you is that to solve ARC2, you really need test and adaptation. All systems that do meaningfully above zero are using TTL. But even then, they’re still far below human level.

So compared to ARC1, ARC2 enables much more granular evaluation of TTF systems, systems like O3, for instance. And that’s where you see that O3 and other systems like this are still not yet quite human level. And in my view, as long as it’s easy to come up with tasks that any one of you can do that are easy for humans, but that AI cannot figure out, no matter how much compute you throw at it, we don’t have AGI yet. And you will know that you are close to having AGI when it becomes increasingly difficult to come up with such tasks. We’re still not there yet.

Looking Ahead to ARC3

And to be clear, I don’t think ARC2 is the final test. We’re not going to stop at ARC2. We’ve started development on ARCGI3. And ARC3 is a significant departure from the input-output pair formats of ARC1 and 2. We are assessing agency, the ability to explore, to learn interactively, to set goals, achieve goals autonomously. So your AI is dropped into a brand new environment where it doesn’t know what the controls do. It doesn’t know

What the goal is. It doesn’t know what the gameplay mechanics are. Just to figure out everything on the fly, starting with what it’s even supposed to do in the game. And every single game is entirely unique. They’re all built on top of core knowledge pairs only, just like in ARC1 and 2.

So we’ll have hundreds of interactive reasoning tasks like this one. And efficiency is central to the design of ARC3. So models won’t just be graded on whether they can solve a task, but on how efficiently they solve it. And we are establishing a strict limit of the number of actions that the model can take. And we are targeting the same level of action efficiency as we observe in humans.

So we’re going to launch this in early 2026. And next month in July, we’re going to release a developer preview so you can start playing with it. What’s it going to take to solve ARC2? And we’re still very far from it today. Then solve ARC3. And we’re even further away from that. Maybe in the future, solve ARC4, eventually get to AGI.

What Are We Still Missing?

So I’ve said that intelligence is the efficiency with which you operationalize the past to face a constantly changing future. But of course, if the future you face has really nothing in common with the past, no common ground with anything you’ve seen before, you could not make sense of it no matter how intelligent you were.

But here’s the thing. Nothing is ever truly novel. The universe around you is made of many different things that are all similar to each other, like one tree is similar to another tree is also similar to a neuron, or electromagnetism is similar to hydrodynamics, is also similar to gravity. So we are surrounded by isomorphisms.

The Kaleidoscope Hypothesis

I call this the kaleidoscope hypothesis. Our experience of the world seems to feature a never-ending novelty and complexity, but the number of unique atoms of meaning that you need to describe it is actually very small. And everything around you is a recombination of these atoms.

And intelligence is the ability to mine your experience to identify the atoms of meaning that can be reused across many different situations, across many different tasks. And this involves identifying invariants, structure, things that seem to be repeated principles. And these building blocks, these atoms, are called abstractions.

And whenever you encounter a new situation, you’re going to make sense of it by recombining on the fly abstractions from your collection to create a brand new model that’s adapted to the situation.

Key Components of Intelligence

So implementing intelligence is going to have two key parts. First, there’s abstraction acquisition. You want to be able to efficiently extract reusable abstractions from your past experience, from a feed of data, for instance. And then there’s on-the-fly recombination. You want to be able to efficiently select and recombine these building blocks into models that are fit for the current situation.

And the emphasis on efficiency here is crucial. How intelligent you are is not just determined by whether you can do something, it’s determined by how efficiently you can acquire good abstractions from your experience, how efficiently you can recombine them to navigate novelty.

So if you need hundreds of thousands of hours to acquire a simple skill, you’re not very intelligent. Or if you need to enumerate every single move on the chessboard to find the best move, you’re not very intelligent. So intelligence is not just demonstrating high skill, it’s really the efficiency with which you acquire and deploy these skills. It’s both data efficiency and compute efficiency.

Limitations of Current AI Models

And at this point, you start to see why simply making our AI models bigger and training them on more data didn’t automatically lead to AGI. We are missing a couple of things.

First, these models lack the ability to do on-the-fly recombination. So at training time, they were learning a lot. They were acquiring many useful abstractions, but then at test time, they were completely static. You could only use them to fetch and apply a prerecorded template. And that is a critical problem that test and adaptation is addressing. CTA adds on-the-fly recombination capabilities to our AI. And that’s actually a huge step forward that gets us much, much closer to AGI.

That’s not the only problem. Recombination is not the only thing missing. The other problem is that these models are still incredibly inefficient. If you take gradient descent, for instance, gradient descent requires vast amounts of data to distill simple abstractions, many orders of magnitude more data than what humans need, roughly three to four orders of magnitude more.

And if you look at recombination efficiency, even the latest state-of-the-art CTA techniques, they still need thousands of dollars of compute to solve Arc1 at human level. And that doesn’t even scale to Arc2.

Compositional Generalization

And the fundamental issue here is that deep learning models are missing compositional generalization. And that’s the thing that Arc2 is trying to measure. And the reason why is that there’s more than one kind of abstraction. And this is really important.

I said that intelligence is about mining abstractions from data and then recombining them. There’s really two kinds of abstraction. There’s type one and type two. They’re pretty similar to each other. They mirror each other.

So both are about comparing things, comparing instances and merging individual instances into common templates by eliminating certain details about the instances. So basically, you take a bunch of things, you compare them, you draw the details that don’t matter, and what you’re left with is an abstraction.

Two Types of Abstraction

And the key difference between the two is that one operates over a continuous domain and the other operates over a discrete domain. So type one, or value-centric abstraction, is about comparing things via a continuous distance function. And that’s the kind of abstraction that’s behind perception, pattern recognition, intuition, and also, of course, modern machine learning.

And type two, or program-centric abstraction, is about comparing discrete programs, which is to say graphs. And instead of trying to compute distances between them, you’re going to be looking for exact structure matching. You’re going to be looking for exact isomorphisms, subgraph isomorphisms. And this is what underlies much of human reasoning. It’s also what software engineers do when they’re refactoring some code. So if you hear a software engineer talk about abstraction, they mean this kind of abstraction.

So two kinds of abstraction, both driven by analogy-making, either value analogy or program analogy, and all cognition arises from a combination of these two forms of abstraction. You can remember then the left brain versus right brain metaphor, one half for perception, intuition, and the other half for reasoning, planning, rigor.

And transformers are great at type one abstraction. They can do everything that type one is effective for. Perception, intuition, pattern recognition, they all work well. So in that sense, transformers are a major breakthrough in AI, but they’re still not a good fit for type two. And this is why you will struggle to train one of these models to do very simple type two things, like sorting a list, or adding digits provided as a sequence of tokens.

Discrete Program Search

So how are we going to get to type two? You have to leverage discrete program search, as opposed to purely manipulating continuous interpolated spaces learned with gradient descent. Search is what unlocks invention beyond just automation.

All known AI systems today that are capable of some kind of invention, some kind of creativity, they rely on discrete search. Even back in the 90s, we were already using genetic search to come up with new internet designs. Or you can take AlphaGo with Move37. That was discrete search. Or more recently, the AlphaEvolve system from DeepMind, all discrete search systems. So deep learning doesn’t invent, but search does.

So what’s discrete program search? It’s basically commutative search over graphs of operators taken from some language, some DSM. And to better understand it, you can try to draw an analogy between program synthesis and the machine learning techniques you already know about.

In machine learning, your model is a differentiable parametric function. So it’s a curve. In program synthesis, it’s going to be a discrete graph, a graph of ops, symbolic ops from some language. In ML, your learning engine, the way you create models is gradient descent, which is very computation, by the way. Gradient descent will let you find a model that fits the data very quickly, very efficiently.

In program synthesis, the learning engine is search, it’s commutative search, which is extremely computation efficient, obviously. In machine learning, the key obstacle that you run into is data density. In order to fit a model, you need a dense sampling of the data manifolds. You need a lot of data. And program synthesis is the exact reverse. Program synthesis is extremely

Data efficient. You can fit a program using only two or three examples. But in order to find that program, you have to sift through a vast space of potential programs. And the size of that space grows combinatorially with problem complexities. You run into this combinatorial explosion wall.

I said earlier that intelligence is a combination of two forms of abstraction. Type one and type two. And I really don’t think that you’re going to go very far if you go all in on just one of them, like all in on type one or all in on type two. I think that if you want to really unlock their potential, you have to combine them together. And that’s what human intelligence is really good at. That’s really what makes us special. We combine perception and intuition together with explicit step-by-step reasoning. We combine both forms of abstraction in all our thoughts, all our actions, everywhere.

For instance, when you’re playing chess, you’re using type two when you calculate, when you unfold some potential moves step-by-step in your mind, but you’re not going to do this for every possible move, of course, because there are too many of them. You’re only going to be doing it for a couple of different options. Like here, you’re going to look at the knight, the queen, and the way you narrow down these options is via intuition, via pattern recognition on the board, and you build that up very much through experience. You’ve mined your past experience unconsciously to extract these patterns, and that’s very much type one. So you’re using type one intuition to make type two calculation tractable.

Merging Type One and Type Two Systems

So how is the merger between type one and type two going to work? Well, the key system two technique is discrete search over a space of programs, and the blocker that you run into is combinatorial explosion. And meanwhile, the key system one technique is curve fitting and interpolation on the curve. So you take a lot of data, you embed it on some kind of interpolating manifold that enables fast but approximate judgment calls about the target space. And the big idea is going to be to leverage these fast but approximate judgment calls to fight combinatorial explosion and make program search tractable.

A simple analogy to understand this would be drawing a map. So you take a space of discrete objects with discrete relationships that would normally require combinatorial search, like pathfinding on a subway system, for instance, and you embed these objects into a latent space where you can use a continuous distance function to make fast but approximate guesses about discrete relationships. And this enables you to keep combinatorial explosion in check while doing search.

A New AI Architecture

And this is what the full picture looks like. This is the system that we are currently working on. AI is going to move towards systems that are more like programmers that approach a new task by writing software for it. And when faced with a new task, your programmer-like meta-learner will synthesize on the fly a program or model that is adapted to the task. And this program will blend deep learning sub-modules for type one sub-problems, like perception, for instance, and algorithmic modules for type two sub-problems.

And these models are going to be assembled by a discrete program search system that is guided by deep learning-based intuition about the structure of program space. And this search process isn’t done from scratch. It’s going to leverage a global library of reusable building blocks of abstractions. And that library is constantly evolving as it’s learning from incoming tasks.

So when a new problem appears, the system is going to search through this library for relevant building blocks. And whenever in the course of solving a new problem, you’re synthesizing a new building block, you’re going to be uploading it back to the library, much like as a software engineer, if you develop a useful library for your own work, you’re going to put it on GitHub so that other people can reuse it.

The Future of AI and Scientific Discovery

And the ultimate goal here is to have an AI that can face a completely new situation and is going to use its rich abstraction library to quickly assemble a working model, much like a human software engineer can quickly create a piece of software to solve a new problem by leveraging existing tools, existing libraries. And this AI is going to keep improving itself over time, both by expanding its library of abstractions and also by refining its intuition about the structure of program space.

This system is what we are building at NDI, our new research lab. We started NDI because we believe that in order to dramatically accelerate scientific progress, we need AI that’s capable of independent invention and discovery. We need AI that could expand the frontiers of knowledge, not just operate within them. And we really believe that a new form of AI is going to be key to this acceleration.

Deep learning is great at automation, it’s incredibly powerful for automation, but scientific discovery requires something more. And our approach at NDI is to leverage deep learning-guided program search to build this programmer-like meta-learner and to test our progress. Our first milestone is going to be to solve RKGI using a system that starts out knowing nothing at all about RKGI. And we ultimately want to leverage our system for science to empower human researchers and help accelerate the timeline of science.

Related Posts

• Full Transcript: Zanny Minton Beddoes Interviews Elon Musk
• Transcript of Daniel Kokotajlo Interview: Diary Of A CEO Podcast
• Transcript: Why a Social Media Ban Won’t Save Teens – Candice Odgers  
• Transcript of Elon Musk Interview: SXSW Live – March 9, 2013
• Digital Afterlife: Who Owns Your Data When You’re Gone? – Suresh Sankaran Srinivasan (Transcript)