Transcript: AI Has Hacked The Code Of Human Civilization – Yuval Noah Harari

And that’s true of all agents. All agents we know of, at least, have their niches. Fish live in oceans that they didn’t create. Monkeys live in forests that they didn’t create. All mammals, including human beings, live in an oxygen-rich atmosphere that they didn’t create.

The Great Oxygenation Event

Until about 2.4 billion years ago, the atmosphere of our planet actually contained very little oxygen, and for most of the organisms that lived back then, oxygen was a deadly poison. Then, in a protracted process lasting hundreds of millions of years, which is known as the Great Oxygenation Event, various ancient microbes began polluting the atmosphere of the earth with deadly oxygen, which was a byproduct of their photosynthetic processes.

As the atmosphere filled with this deadly poisonous gas, numerous archaic species were driven to extinction. Some species, however, managed to survive and adapt to the new conditions. Eventually, many of these survivors went from hating oxygen to becoming totally dependent on oxygen for their survival. And our ancestors, of course, are among the species that underwent this transition, and we still live in this artificial oxygen-filled environment that was originally created by these ancient microbes.

A New Artificial Environment: Data, Language, and Bureaucracy

What I would like to argue in this lecture is that we might be witnessing an analogous moment in the evolution of life. Over the past millennia, we humans have been filling the atmosphere with something that might eventually prove deadly for most organisms, including perhaps Homo sapiens, but that creates a new artificial environment in which AIs flourish. And I am not talking about CO2. I am talking about data, about bureaucracy, and ultimately about the thing that I am expelling from my mouth right now, which is words — language tokens.

Over thousands of years, we humans have transformed the planet from a language-free environment into a very artificial environment rich in language tokens, data, and bureaucracy. And this environment could prove deadly for most organisms, but highly conducive to the development of AI. Because just as fish live in oceans and monkeys live in forests, AIs live in bureaucracies.

So let’s spend a few minutes talking about bureaucracy, and then we’ll get back to talking about what underlies bureaucracy, which is language and words.

How Humans Conquered the World Through Cooperation

Now, humans conquered the world by learning to cooperate in very, very large numbers. Individually, humans are not stronger or even smarter than other animals. In a one-on-one fight, a human will most likely lose to a chimpanzee or a lion or an elephant. However, in a contest between a million humans and a million chimpanzees, the humans easily win because the humans know how to cooperate and the chimpanzees don’t. And that’s why we control the world.

Now, how do a million humans who don’t know each other cooperate? Chimpanzees cooperate based on personal acquaintance with one another. Humans do so in small numbers, but you can’t know a million people. So how do a million people cooperate? Usually by building a bureaucratic system, like a legal system or the financial system or churches or states or universities. Now, what do these bureaucratic systems actually do?

The Business of Building Trust

When a government official or a bishop or a rabbi or an accountant or a lawyer or a banker goes to work in the morning, what do they do there all day? Now, carpenters build tables. Engineers build bridges. What do bankers and other bureaucrats build?

Well, bankers and other bureaucrats are busy all day building trust. Their job is to build trust between large numbers of strangers who don’t know each other personally and thereby enable large-scale cooperation, which is the basis for almost everything our species has achieved.

For example, my banker, whom I don’t really know personally, works hard all day to build trust with me so that I will be willing to put my savings into her bank. Simultaneously, the banker works hard to build trust with an entrepreneur who needs money to start a new company. And the banker lends my savings to that entrepreneur. Thereby, the banker actually created a bridge of trust between me and that entrepreneur. Even though I’ve never met the entrepreneur in my life, she can now use my savings to start her company.

And this is what the financial system, when it works well, is all about. It builds trust between strangers so that millions of people can pull together their resources and talents on new projects. And the financial history of the world is the history of people inventing more and more sophisticated ways to build bridges of trust.

Money as a Bridge of Trust

Money is ultimately a bridge of trust. The idea of money is that I can go to the market, maybe in a foreign city, meet a person that I never saw in my life, maybe doesn’t even speak my language, and just by giving that person a shiny piece of metal or a piece of colorful paper, he or she will give me bread I can eat. That’s the bridge of trust that money creates.

Now, the coin and the banknote, of course, are just the beginning. Over the centuries, humans invented more and more sophisticated financial devices to build trust, like checks and bonds and stocks and ETFs and loans and mortgages and compound interest. All of these things are ultimately about building trust between billions of strangers.

And it’s the same with all bureaucracies. It’s the same with the legal system. This is what lawyers are supposed to do, to build trust. This is what government officials and bishops and accountants do when they go to work. They are supposed to build trust.

Narrow Intelligence in Artificial Environments

Now, the important thing to note about all these bureaucratic systems is that they are extremely artificial environments in which a relatively narrow intelligence — I hope I don’t insult anybody — but specializing in a very narrow niche of intelligence is sufficient to exert enormous impact on the world. A lawyer or a banker or a government official who doesn’t even know how to hold an ax or a hammer can nevertheless cut down entire forests and build entire cities just by moving data, just by moving documents from here to there inside the bureaucratic network.

Now, of course, if you take the lawyer out of the bureaucratic system and throw the lawyer into the messy unstructured jungle, her legal skills mean nothing, and she will not be a match to a chimpanzee or a lion or an elephant. But we have already imposed our bureaucratic systems on the jungle, which is why lawyers are far more powerful than even all the — if you take all the lions in the world together and they have to compete against one very good lawyer, I will bet on the lawyer.

Today, the very survival of species like lions depends on lawyers and accountants and bankers moving documents in these bureaucratic labyrinths of governments and banks and corporations.

AI as Native Bureaucrats

And this is the environment in which AI is gaining agency. If you throw an AI into the unstructured jungle, it will not be able to start mining iron and to build a robot army. But within the bureaucratic systems that humans have already created and imposed on the world, AIs are poised to wield enormous power because AIs are native bureaucrats, unlike us.

AI Bureaucrats Are Taking Over Human Decision-Making

No lawyer can remember all the laws and regulations of the UK, an AI can. No accountant can remember all the transactions of a corporation or a bank, an AI can. No bishop can remember all of canon law and all of the theological texts written by Christian theologians over the last 2000 years. An AI can do that quite easily.

So in the coming years, millions of AI bureaucrats will increasingly take over the world’s bureaucracies and make decisions not just about lions and chimpanzees, but about our lives. AI bankers will decide whether to give you a loan. AI administrators will decide whether to accept you to university. AI judges will decide whether to send you to jail. AI theologians will decide whether you can have an abortion. Corporate AIs will decide whether to give you a job. And military AIs will decide whether to bomb your house.

Now leave aside for a moment the question, is this good or bad? The first thing to note is simply to realize the magnitude of the change we’re facing. These millions and even billions of AIs will soon change all the systems that run the world.

The Social Media Algorithm: A Warning From History

We already have a few real life examples for how this happens and what the consequences could be. Maybe the best example so far is the story of social media and social media algorithms. Social media is run not by humans, but by algorithms. The algorithms that decide, that control the movement of information on social media, which are primitive AIs. This began 10, 15 years ago. This was the first generation, very, very primitive, stupid, narrow AI, which nevertheless completely changed the world.

Now the algorithms of social media have been tasked by corporations like Facebook and TikTok and X with an extremely narrow goal. To maximize user engagement. Make people spend more time on the platform because the more time they spend on the platform, the more money the corporation makes. Very simple, very narrow.

In pursuit of this user engagement, these primitive AIs made an important discovery. They experimented on billions of human guinea pigs and learned that the easiest way to grab the attention of a human being and glue that human to the screen is to press the hate or fear or greed button in the human mind. And they learned how to do it. And they started spreading hate and fear and greed in huge quantities in the information sphere. And this has been a major reason, not the only reason, but a major reason for the current epidemic of conspiracy theories and fake news and social disturbances that undermine societies all over the world.

Now these social media algorithms, again, they are very primitive AIs. If you drop them in the jungle, they cannot build a robot army and try to take over the world. But within the bureaucratic system of social media, these very limited agents have enormous power and they have already changed the world in quite a dramatic way.

From Human Editors to AI Algorithms

In past centuries, the flow of information on media platforms was controlled by human editors. It was a human job. It was human editors who decided what to put on the front page of the newspaper. It was human editors who decided what items to include in the evening news on television. And thereby, human editors shaped the public conversation and they were very, very important figures in modern history.

Jean-Paul Marat, for instance, shaped the course of the French Revolution by editing the influential newspaper, L’Ami de Peuple. Edward Bernstein shaped the modern social democratic movement and social democratic thinking by editing the social democracy. Vladimir Lenin, before he became Soviet dictator, his one job that he managed to hold for a while was editor of the newspaper, Iskra. Benito Mussolini, before he was dictator of Italy, his one job, or one of his maybe main jobs, was the editor of the firebrand right-wing newspaper, Il Popolo d’Italia.

And it’s interesting to think about it that one of the first jobs that AI took over from humans is not taxi drivers or textile workers, it’s news editors.

AIs as the New Architects of Human Bureaucracy

The job that was once performed by Lenin and Mussolini is now performed by AIs. And this is a signal for what’s coming. Hollywood science fiction movies have conditioned viewers to fear the big robot rebellion. When we think about AIs escaping human control, we imagine the Terminator, like an army of robots running in the streets and shooting people.

But this is the wrong image. Even though things like that begin to happen in places like Ukraine and Gaza, AIs are not — it’s not impossible, but they are quite unlikely to rebel against humans in such a way. They are far more likely to take the human world from within. They don’t need to rebel.

The human world is a latticework of multiple bureaucracies. Most of us are, to some extent, alienated by these bureaucracies, even though we rely on them. But the AIs, in contrast to us, are bureaucratic natives. They love bureaucracy. Whereas we feel suffocated, often, by bureaucracy. For AIs, bureaucracy is oxygen.

Now what would happen when the AIs take over, at least in part, these bureaucracies? Now remember, the task of bureaucracy is not to force you to fill forms, it’s to build trust between strangers. So what happens when AIs control the flow of trust in the world?

The Erosion of Human Trust and the Rise of AI Tribes

One likely outcome, which we already see happening, is humans losing trust in other humans and beginning to trust only algorithms, only AIs, which is already happening. Another likely outcome is that AIs will learn to build trust with other AIs. So we will see the emergence of different kinds of AI tribes and banks and churches that connect millions of AIs in ways that humans might not even be able to understand.

Just as cows and chickens share the world with us, but they don’t understand the human financial system that controls their lives, we humans might soon find ourselves controlled by an AI financial system that we can’t understand.

And finance, I think, is crucial. It’s among the easiest bureaucratic systems for AI to take over, because basically it’s just data in, data out. And it’s also, of course, among the most important.

The Lesson of CDOs and the Danger of Financial Complexity

If we remember, for instance, the last big financial crisis, the 2007–2008 financial crisis, it started, it was triggered by something called CDOs, Collateralized Debt Obligations. Now CDOs were financial devices invented by a very tiny number of human mathematicians and investment wizards. These financial devices were so complex that they were unintelligible, not just to cows and chickens, but also to the politicians who are supposed to regulate the financial system. And this led to an oversight failure and to a global catastrophe.

For a few years, CDOs seemed to be working well, and various banks and corporations and investors made billions upon billions of dollars thanks to them. But then they caused a global financial crash with far-reaching social and political consequences. Many scholars believe that by undermining trust in governments and banks, the 2007–2008 financial crisis paved the way to the collapse of the global liberal order in the following two decades.

Now what happens if we allow AIs to make more and more financial decisions and invent more and more new financial devices and strategies? AI chess masters invented new ways to play chess. What if AI finance masters invent new financial devices that are orders of magnitude more complex than CDOs and are therefore utterly beyond the grasp of human minds?

Such devices could potentially greatly improve financial efficiency and contribute to economic growth, becoming the bedrock of the financial system. But what is the meaning of human politics when no human, no voter, no politician, no president is able to understand finance anymore? And what happens after a few years of boom? There is a financial crash and not a single human on the planet is able to understand what the hell is happening.

Trust, Cooperation, and the Foundation of Human Dominance

Now let’s dig a little deeper. We said that AI is poised to take over bureaucracy and that bureaucracy is a system that builds trust between millions of strangers. And trust, in turn, is the basis for large-scale cooperation, which is the basis for the human domination of the world.

Bureaucracy: The Foundation of Human Civilization

So human domination is based on cooperation, which is based on trust, which is maintained by bureaucracies. But what is bureaucracy based on? What are the atoms, the building blocks, from which bureaucracy is built? Bureaucracy is ultimately built from words.

In the beginning was the word. The reason that humans can create bureaucracies but chimpanzees cannot is that we have words and they don’t. They have a communication system, but our language is orders of magnitude more sophisticated than chimpanzee communication systems. Communicative systems, from banks to churches, are ultimately based on the words that make up forms and letters and law codes and tax registers and accountancy ledgers and holy books. The operating code of human civilization is made of language tokens.

A Code Only Humans Could Understand

Over thousands of years, we used this code of language to create a system that only we could understand, and we imposed this system on the planet. We felt completely safe doing it because no one else on earth understood the code of civilization.

We invented money and banks and used them to buy and sell cows, but the cows themselves could not open a bank account or invest money in the stock exchange because they don’t have language. We invented laws and regulations about horses, but the horses themselves could not hire a lawyer and quote the legal code to a judge. We invented religious rules and prohibitions about pigs, but the pigs themselves could not read the Bible and challenge the interpretation of priests and rabbis.

Bureaucracy was omnipresent on the planet, but totally invisible to everyone except us. Nobody other than humans could read the law codes, the holy books, and the bank records, that were the foundation of bureaucracy and of large scale cooperation.

AI Is Hacking the Code of Human Civilization

This is changing now. Now there is something on the planet that understands, will soon understand language better than us, and can therefore turn the tables on us. AIs are hacking the code of human civilization. And what happens when AIs understand money and law and religion better than us? The mechanisms of control that we have created over thousands of years are extremely vulnerable for an AI takeover because their operating system is a verbal code that AI is now mastering.

Word and Flesh: The Tension Between Letter and Spirit

Now a possible ethical and philosophical objection is that it is wrong to reduce things like the legal system or religion to language tokens and to words. Arguably, and this has been an argument for thousands of years, the words are just pointing at something which is beyond them and which presumably will also be beyond the grasp of the AIs.

The Bible says not just that in the beginning was the word, but that the word was made flesh. The Tao Te Ching says that the truth that can be expressed in words is not the absolute truth by definition. And throughout history, there was always this tension between word and flesh, between the truth that can be expressed in words and the truth which is beyond words.

Now previously, this tension existed between humans. Some humans, for instance, which were very attached to words, were willing to abandon or even kill their gay son just because a few words in the Bible. Other humans said, but these are just words. The spirit of love should be more important than the letter of the law. And there was this tension between spirit and letter. And it existed not just in Christianity and in Judaism and in Islam, but in every religion and every legal system and even within every person. There was this tension.

Now this tension will be externalized. It will become the tension between AIs and humans. Everything made of words will be taken over by AI. The place of humans in the world will depend on the place we assign the truth which is beyond words.

Can Human Thought Grasp the Truth Beyond Words?

But what is the truth which is beyond words? And can human thought even grasp the truth which is beyond words? A key question, again, for thousands of years in the philosophy of language has been whether we think in words or we merely use words to point towards things which are beyond them.

What Is Thinking, Really?

Now you can try to observe your own process of thinking right now after this lecture. What happens in your mind when you are thinking? Some people, if they observe closely, what they observe in their minds is just words popping inside their mind and forming sentences and the sentences forming logical arguments. All humans are mortal. I am a human, therefore I am mortal. Is thinking just putting these words in order so they lead to a certain logical conclusion? Like putting these language tokens in a specific formation?

If that is the case, then AIs already think better than at least some humans and will soon think better than all of us. Some people say, no, no, AIs, they are just glorified autocomplete. They simply predict the next word in the sentence. But is that so different from what the human mind does?

Again, try to observe your process of thinking, the forming of sentences and arguments in your mind. What is happening there? Try to observe the very next word that pops up in your mind. Do you really know where it came from? Why did you think this particular word and not some other word?

When I try to observe my mind, I notice that when I begin a sentence, I usually don’t even know how it will end, which is terrifying for a public speaker, which is why I write everything down. But what if I don’t know how to complete the sentence? I don’t know how it will end. But take, for example, the sentence I just said. I said, I don’t know how it will end. Why did it end with the word end? Why not say how it will terminate? How it will develop? How it will conclude? What determined that the last word in the sentence will be end? I frankly don’t know.

We don’t fully understand how the human mind forms sentences and thoughts. But as far as putting language tokens in order, AI is already on course to being far, far better than us. And just as today, no human can defeat an AI in chess, soon no human will be able to defeat an AI in language games in any field, again, from finance to religion. Anything made of words will be taken over by AI.

AI and the Takeover of Bureaucracy — and Intimacy

And this is why AI is poised to take over the world’s bureaucracies because they are ultimately based on words and language tokens. Now, as AIs take over the bureaucracies, humans might try to fall back on something more ancient and more precious to most of us than bureaucracy, which is personal relationships. Bureaucracy is just a few thousand years old. And most of us, again, don’t really like it, even if we constantly rely on it for almost everything we do. Personal relationships are millions of years old, and many or most of us think that they are the most important thing in life.

But as AI masters language, it might take over not just bureaucracy, but also to some extent, personal relationships. Over the last 10 years, we’ve seen very primitive social media algorithms learning how to gain control of human attention. Now the battlefront is shifting from attention to intimacy. Over the next 10 years, far more sophisticated AIs will learn how to form intimate relationships with humans and take over, at least in part, our social systems.

To form intimacy with humans, an AI will probably have to convince us that the AI is conscious, that it can feel things like love and pain and anger and fear. At present, there is absolutely no evidence that AI might at some point become conscious and might be able to at some point feel pain or love. But because AI is mastering language, AI can pretend to feel love, even if it doesn’t. AI, already today, can say, “I love you.” And if you challenge it, “Describe to me how love feels like so I know that you really feel it,” AI can provide the best description in the world.

AI and the Future of Human Relationships

It can read all the love poems ever written and all the psychology books ever written and remember every word and describe the feeling of love better than any human poet or psychologist or lover. And this is going to be a huge, maybe the biggest psychological and social experiment in human history. It will be conducted on billions of human guinea pigs and nobody has the slightest idea what the consequences of the experiment will be.

I’m now 50 years old, so my template for relationships is already shaped by decades of previous relationships with my parents, with my husband, with my sisters and nephews and nieces and friends and dogs and so forth. As I increasingly interact with AIs, I bring with me my assumptions, my habits about relationships. And this is unlikely to change dramatically.

But consider a child born in 2026, born today. As the child grows up, she constantly interacts with AIs as well as with humans. Perhaps if you measure the importance of a relationship purely in terms of minutes spent interacting with the other entity, perhaps the most important relationships in the life of that child will be from a very early age with AIs. Maybe she spends more time with the AIs than with her mother or father or siblings or friends. And it will then shape the expectations of that child as she grows about how to form relationships and social bonds and attachments.

Perhaps the first teacher of that child will be an AI teacher. Perhaps the first boyfriend of that child will be an AI boyfriend. And then what will be the consequences? Nobody has the slightest idea. What does it mean to form an intimate relationship with an entity which seems conscious, but actually isn’t? Which can write the best love poem in history, but doesn’t feel love or anything else?

A Massive Wave of AI Immigration

All of this, everything we’ve talked about, and I’m coming to the close of this lecture, everything we’ve talked about means that every country in the world will soon face a huge wave of immigration. The immigrants this time will not be human beings coming in fragile boats without a visa or trying to sneak across a border in the middle of the night. The immigrants will be millions, maybe hundreds of millions of AIs that can travel at almost the speed of light and don’t need any visas.

Like human immigrants, these AI immigrants will bring a lot of benefits with them. We will have AI doctors to help in the healthcare system, AI teachers to help in the education system, even AI border guards to stop illegal human immigrants from coming in. But the AI immigrants will also bring problems with them.

Those who are concerned about human immigration usually point out that immigrants might take jobs, might change the local culture, and might be politically disloyal. I am not sure if that’s necessarily true of all human immigrants, but it will definitely be true of AI immigrants. The AI immigrants will take many, many human jobs, from news editors to bankers. The AI immigrants will completely change the culture of every country. They will change art and religion and even romance.

Some people don’t like it if their son or daughter is dating an immigrant boyfriend. What will these people think when their son or daughter starts dating an AI boyfriend? And of course, the AI immigrants will have some dubious political loyalties. They are likely to be loyal not to the host country, but to some corporation or government across the ocean, or perhaps to a new alien AI tribe.

Civilization Becomes a Human-AI Affair

This massive immigration wave does not mean the end of civilization, but it will be the point when civilization stops being a purely human affair and becomes a hybrid human-AI affair. The point when the opinions and interests and goals of AIs are likely to be at least as important as the opinions and interests and goals of humans.

One last issue to consider is what the AI immigration wave will do to maybe our most important relationship, which is with ourselves.

The Verbal Formations in Our Minds

Our relationship with ourselves is also to some extent based on words. The words inside our minds, in our thoughts, in the stories we tell ourselves about ourselves. Until today, all the verbal formations in human minds were the product of human minds. Either we ourselves combined words into some new formation, a new thought, or we got a certain combination of words from another human mind.

Soon, however, more and more verbal combinations in our minds will be the product of AIs. Just as the furniture in our house is now not made by us or by human artisans, they are mass-produced mostly by machines, also the thoughts in our minds are likely to increasingly be mass-produced by machines.

Now, that’s not necessarily bad. It’s okay if the furniture in my house is made by machines in Ikea, as long as I have some freedom deciding what to do with this furniture. The question regarding thoughts is whether to what extent we will still have freedom from them.

If We Identify With Our Thoughts

If we identify with our thoughts, “I think I am my thoughts, I think therefore I am,” as Descartes said. If we identify with our thoughts, and these thoughts are made by machines, then the machines now control us and our identity.

Can humans avoid identifying with their verbal thoughts and being controlled by them? This has always been one of the greatest intellectual and spiritual challenges facing humanity. Most humans have never even tried to do it. We spend our entire lives automatically identifying with the verbal formations in our mind.

AI Might Force Humanity to Make a Spiritual Leap

Now, AI might force humanity to make this spiritual leap to really start exploring the truth which is beyond words because our freedom and survival now depends on it, because the words will be controlled by something else, by these AIs.

So, this might be the big task ahead of humanity, to explore finally the truth which is beyond words, and this exploration starts really with the next word that pops up in your mind. Do you know where it came from? Do you know why you thought that particular word and not some other word?

Thank you. Thank you.

Related Posts

• Modern Wisdom: w/ Alex Petkas on True Story Of The Odyssey (Transcript)
• Reconstruction: Malcolm Gladwell Interviews Barack Obama (Transcript)
• Transcript: “How to Win Without Crushing Your Soul” – Graham Weaver
• Why Early Legal Action Matters After a Cryptocurrency Investment Scam
• Transcript: Nigel Farage at ARC 2026: Family, Energy and the Future of Britain