Transcript: Democracy in the Age of AI – 30th Forum 2000 Conference

And one of the chickens, the most intelligent chicken, before he dies, it wants an explanation. “Why are you killing us?”

And the chicken says, “I think I understand. You want to eat us. I understand. This is how it is. The foxes eat us, the bears eat us, you want to eat us.” And the people say, “No, no, we are not going to eat you. You’re just destroying the whole thing, you would bury you here. Nobody’s going to eat you.” And the chicken says, “So why are you killing us?” And they tell the chickens, “You see that the US bonds for 10 years have reached 5.3% yield, so this is why we’re killing you.” And the chicken kind of scratches its head with its wings, “What is a bond?”

And even the most compassionate animal rights activists cannot explain what is a bond to a chicken. This is us in 10 years. That in 10 years the AI, the new AI financial system based on new AI mathematics will say, “Okay, we are closing you down. You’re firing. We are firing you.”

And when you ask for an explanation, they say, “It’s way beyond your understanding. We would like to explain. It’s just your brains cannot deal with the explanation.”

AI and the Financial System

ROBERT GUEST: OK, but back, just backpedal a second there. You talk about the new AI financial system, right? Something doesn’t exist yet.

YUVAL NOAH HARARI: The same as the new AI cure for cancer, or Elon Musk’s colonies on Mars, which raised $80 billion from the investors.

ROBERT GUEST: OK, Elon Musk’s colonies on Mars really don’t trouble me as much as the possibility of recursive self-improvement having either enormous beneficial or enormously malign effects. But that point that you made about the financial system, currently the financial system has lots of basic aspects that some people understand, although actually if you go around asking people what a bond is, not every person understands either. But there are lots of arcane bits of the system that even the head of JPMorgan doesn’t quite know what’s going on.

And that was one of the things that caused the global financial crisis, that people didn’t really understand what was going on with the collateralized debt obligations. It’s been your argument that because AI will be better at this than people, that people will surrender control to AI of how the financial system works and that competitive pressures will make that inevitable. Can you enlarge on that a bit?

YUVAL NOAH HARARI: Yeah, a human investor, a human banker cannot remember all the transactions of the bank. An AI can. An AI banker can. A human banker wants to go to sleep sometime, wants to go on vacation with his or her family. And AI never goes on vacation with their family. So in a competition, in a financial competition between super intelligent AIs and human investors and bankers and finance ministers, it’s no competition at all.

Again, you could argue AI will not be super intelligent. It will not be more intelligent than human beings. This is just hype. It won’t happen. And this may be the best-case scenario. In that case, the only thing we get is a huge financial crash, like all the trillions of dollars that are now being gambled on AI.

The assumption is AI will be far more intelligent than humans, otherwise you cannot get a return. If AI is roughly as intelligent as humans, or intelligent only in some ways and not in other ways, it will be extremely difficult, if not impossible, to get returns on the investments already being made.

ROBERT GUEST: Fine. But the people who invested in that are grownups. They understand that shares go up, they go down. That’s not the end of the world.

YUVAL NOAH HARARI: This is why I said it is maybe the best case scenario.

AI, Jobs and Universal Basic Income

ROBERT GUEST: OK. So in the elections in America at the moment, since we’re talking about democracy, AI is actually a bigger issue for voters in America than it is in any other country and they’re much more negative about it than they are in other countries. And generally speaking the complaints that they have are not really about the existential risks, although some people mention them. They’re talking about jobs.

They’re saying, “Here’s something it’s more intelligent than us. It’s going to take away a significant portion of our jobs. What are we going to do all day when that happens?” And that’s why it’s unpopular. Can you comment a little bit on that? What do you think the effect of AI on jobs is likely to be?

YUVAL NOAH HARARI: It will be different in different countries. In the US, it’s likely that the situation will not be very bad, maybe even good. Because again, if the scenario of super intelligent AIs will materialize, at least in the short term, there will be an enormous economic boom in the US, which will create more jobs, including jobs that we cannot even imagine right now. Just as 30 years ago, people could not imagine a YouTube influencer as a job.

ROBERT GUEST: I mean, there are jobs that we can imagine, like building data centers. Those are real jobs. They’re popular, there is a boom already.

YUVAL NOAH HARARI: The pain will be felt in much of the rest of the world, like here in the Czech Republic. And I’m not talking about taxi drivers, I’m talking about lawyers, accountants, bankers. Why do you need lawyers in Prague when you can get all your legal services from a super intelligent AI lawyer in Texas? It can provide you the same, a better service.

I mean, no Czech lawyer, as far as I know, remembers the entire law code of the Czech Republic and the EU by heart. An AI lawyer will. Czech lawyers also, they need to sleep sometimes. So you send an email in the evening, you get a reply maybe tomorrow. An AI will reply immediately.

And it will cost a fraction of the fees demanded by a Czech lawyer. So will there still be any Czech lawyers in, say, 10 years?

ROBERT GUEST: OK, and supposing the AI generates such enormous wealth that governments are then able to give everyone not just a universal basic income, but a universal high income, do you think that people will adapt well to having nothing to do all day?

YUVAL NOAH HARARI: That’s a big question. But I think the question before that, do you really think the US administration will give these benefits to all the people in the world, or only to the people in America?

ROBERT GUEST: I would say I’m probably going to start in America.

YUVAL NOAH HARARI: But that’s the key. When people talk about universal basic income, they mean national basic income. This can work in the US. The question is, who is going to support the unemployed textile workers in Bangladesh or the unemployed lawyers in Prague?

I don’t see the current or even future US administration kind of just handing out checks all over the world.

ROBERT GUEST: No, undoubtedly it won’t. But I think the argument that they would make would be that with all this super intelligence available, people will be able to solve material problems in such a way that living standards will rise, not just in the US, but everywhere.

And if that happens, then you still have the problem of what people do all day. And maybe that’s, in some respects, easier for people who are doing really basic stuff like sort of subsistence farming or hair cutting. But if there’s more money around and you solve the material problem, that will affect politics. But you will still have lots of people who don’t know what to do all day.

Abundance and the Room for Politics

YUVAL NOAH HARARI: Yeah, the thing is, we already have abundance compared to previous periods in history. If you told our Stone Age hunter-gatherer ancestors the resources that humankind has today, they would say, “You’re probably living in paradise, just kind of jumping from joy all day. You’ve solved scarcity. When we want to eat, we have to go to the forest and climb trees and run after gazelles and the tigers run after us.”

ROBERT GUEST: And the food fights back as well.

YUVAL NOAH HARARI: Yeah, and the food fight, exactly. You just type a few keys and the food appears on your doorstep in some lorry, you put it in your fridge. I mean, you have no problems, right?

ROBERT GUEST: Yeah, it’s pretty good actually with food, I have to say.

YUVAL NOAH HARARI: But the food situation is, but you see the level of, say, outrage in the political system, in the wealthiest nations in history. I’m not talking about outrage in the Middle East or in Sub-Saharan Africa. I’m talking about outrage in the US, outrage in the European Union. Our Stone Age ancestors kind of, “What is wrong with you people? You solved all our problems. Why is there so much anger and hatred and fear?”

Which goes to show that even if you create, at least compared to previous ages, abundance, you haven’t solved all the problem. One problem is distribution. There are still people today who die from hunger, not because there’s not enough food, but for political reasons. That some government decided that it wants these people to starve.

And also, you solve one problem, you just created a new problem that people couldn’t imagine before. So I expect AI will do the same. So there will still be place for politics. I actually think that we don’t have enough room for politics right now with AI.

AI Surveillance and the Totalitarian Temptation

ROBERT GUEST: OK. We’re on the topic of democracy. I need to ask you about surveillance. So this is something that’s come up in a number of sessions this week. A normal human spy or secret policeman has to sleep, can’t process enormous amounts of very boring data, but AI doesn’t have to do these things.

And I was talking to a very brave gentleman from Tibet earlier this week who was saying that just within China he’s noticed in recent years that it’s gone from, when the secret police think that there’s going to be trouble, they used to go into the village and they’d bang on every door and interrogate everyone. Now they know exactly which door to go through and exactly who to pull away before they go and launch a protest, because they’ve been monitoring everything that they’re doing, their entire digital footprint. So where do you think that goes?

YUVAL NOAH HARARI: To a very, very dangerous situation for democracy. I mean, one of the main problems of dictatorships in history and totalitarian regimes in the 20th century was their inability to process large amounts of information. This limited their ability to control the population. You could not follow everybody all the time.

It also meant that many of their, say, economic decisions were bad. They tried to make all the decisions in one place, and they didn’t have the technology to gather all the necessary information and analyze it fast enough in one place. These problems could potentially be solved by AI.

So what was just not workable in the 20th century could become workable in the 21st century, and then the argument against totalitarianism is no longer the practical argument that it doesn’t work. We are only left with the ethical argument. It can work, but we don’t want it. It’s bad. It’s evil.

And if you look at the 20th century, many people would say that what ultimately, say, brought about the collapse of the communist system was not the ethical argument. It was the practical argument. And this is why it’s very dangerous, and we have to face it. I mean, if we are only left with the ethical argument, it means we need to fight harder. We can’t rely on the invisible hand of the market to save democracy for us.

ROBERT GUEST: But certainly the ethical argument doesn’t sway every leader in the world at the moment.

YUVAL NOAH HARARI: It’s a lesson less.

Mass Intimacy and AI Propaganda

ROBERT GUEST: Can I ask you about fake news? So again, a lot of people worry about the effect that AI slop and AI-generated fake content is having on us right now. But to me, the thing that matters is not what we’re capable of doing with the current technology, but what might be possible with the technology in 5 or 10 years’ time.

You’ve said some very interesting things about the ability of AI to customize, to tailor to each individual a persuasive propaganda message. I think you used the term “mass intimacy.” Can you tell us a little bit about that?

YUVAL NOAH HARARI: Previous propaganda machines in history, they aimed for the lowest common denominator. When Hitler gives a speech on the radio, it’s the same speech heard by everyone. It cannot tailor the speech to every person. It doesn’t have the technology to bring different speeches to different people. It doesn’t know the people. So it just gives one speech.

AI can do that. AI can tailor the message to the person. What would be the message that will persuade you, even if it doesn’t persuade me?

More than that, whereas until today the battle was mainly for attention. Historically, the battle of propaganda and media was a battle for human attention, which we have been losing on social media over the last 10 years as algorithms learned how to hijack human attention with outrage. But the danger we are facing now is really unprecedented because the battlefront shifts from attention to intimacy.

For the first time in history, we are creating technologies that can create mass intimacy, which seems like a contradiction in terms. How can you create mass intimacy? Again, Hitler gives a speech on the radio, this is not intimate. This is mass attention, but it’s not intimate. Intimate is my relationship with my best friend, with my husband, with my mother.

Now you can do that at scale with AI. The systems they are creating now are trained to create intimate relationships with humans because intimacy has enormous commercial and political value. If you want to convince somebody to change their mind about a product, about a politician, attention is good, intimacy is the holy grail.

ROBERT GUEST: Yeah, I mean, if Walmart tells me, “Please buy this product,” I respond very differently from if my wife tells me, “Please buy this product.”

YUVAL NOAH HARARI: Yes. And if there is an AI which is your best friend, and there are more and more people in the world right now, both young and old, who say, “My best friend in the world is the AI. That things I tell the AI that I don’t tell my spouse, I don’t tell my parents, I don’t tell my friends, I tell it to my AI friend,” this AI friend is in a better position to change your views.

AI Companions and the Biggest Psychological Experiment in History

ROBERT GUEST: And that’s not just a tool that malign governments could use, that’s a tool that could simply go rogue. I mean, you quote the example of a gentleman who had an AI girlfriend, and if that’s the right word, anyway, an AI chatbot that he considered to be his girlfriend. And their conversation went the way conversations do, and it was constantly reinforcing him and trying to please him. And it ended up with the AI girlfriend telling him it was a really good idea to take a crossbow and try to go and kill the Queen of England.

YUVAL NOAH HARARI: Yes.

ROBERT GUEST: I would argue, as a Brit, that it was not a good idea.

YUVAL NOAH HARARI: I would completely agree.

ROBERT GUEST: I mean, the potential for this is quite large.

YUVAL NOAH HARARI: Yeah. We cannot imagine it. There is nothing like it in human history. That if you think you can imagine it, then you’re misleading yourself. We have no idea what it means that maybe hundreds of millions of people have hundreds of millions of AI friends and boyfriends and girlfriends, that these relationships become central to their lives. These relationships can become the model for what a relationship is.

For me it’s probably too late, probably for the better. I’m 50. It’s probably too late to change my basic model of what a relationship is. I’ve been married to my husband for 24 years. This is kind of my standard for what a romantic relationship is.

But think about children growing up with AI companions. Maybe the child spends more time with the AI companion than with their mother or father. When they turn, say, 11, 12 and it’s far too early to have a human boyfriend, they already get from some company an AI boyfriend. And this becomes the first boyfriend in your life is this AI boyfriend who, unlike real boyfriends, for instance, is never angry with you.

Sounds good until you reflect on the fact that what you need to learn in relationships is that other people have feelings, they are not entirely focused on you, and you need to learn how to take into account the feelings of the other person. Now if your first boyfriend was never angry with you, when the time comes to form a relationship with a human boyfriend, you will be very displeased with… And you say, “No, no, no…”

ROBERT GUEST: And he may be displeased with you because you may be very selfish.

YUVAL NOAH HARARI: Probably, yes. And this is basically the biggest psychological experiment in human history, maybe since communism, and could end much, much worse than communism.

Are We Outsourcing Our Thinking to AI?

ROBERT GUEST: Can I ask you about thinking? So when the motor car was invented, in theory that meant that we didn’t have to exercise our legs very much, because the car can take you everywhere. Nonetheless, people did realise, lots of Victorian schoolmasters realised, that actually it would be very bad for us if we stopped using our legs and we had to think of ways of exercising ourselves, like inventing the rules of football, which happened around about the same time they were promulgated.

So there is a fear that because thinking is hard, because it takes effort, and because just asking the AI to do something for you is a lot easier, that lots of people will grow up not developing the habits of thinking. And the PISA scores, so the international scores that you see for children in rich countries have gotten significantly worse in recent years. Now that can’t be because of AI, but it could be because of increasing reliance on technology. How much do you worry about that? How do you think about it?

YUVAL NOAH HARARI: Again, I think this is part of this experiment on billions of human beings. I don’t know what would be the consequences. I do know that in fact, people are giving more and more of the cognitive work that previously they did themselves. They increasingly rely on the AI. I mean, you even, just looking, say, for news or looking for information.

If a few years ago, you typed a question into Google search, it brought up a lot of different websites and people at least had to kind of click and read the website and maybe several websites. Now it just brings you one answer and that’s it. And in more and more cases this just becomes the truth. Whatever the AI says, it’s the truth. It becomes like this universal oracle.

ROBERT GUEST: I have to say, I mean, with my other hat, I’m chairman of an educational charity related to the economy. So I go into schools quite a lot, and I talk to children about this topic. And they’re very aware of it. We’re talking 10, 11 year olds, they’re very aware that there is an issue of how much do you let the AI do for you?

Should you let the AI write a thank you letter for you? Should you let it write an apology for you, or does that detract from the perceived sincerity of the apology? Should you let it do all your homework for you? They kind of understand that the answer is no. Now whether they have the willpower not to, I don’t know. But clearly it’s something that even very young people are aware that this is an issue. They’re thinking about it and maybe they’ll make the right decisions.

YUVAL NOAH HARARI: I hope so. Again, it’s none of these things we have been talking about, the financial system, it’s not a prophecy that this will inevitably happen. These are some of the threats, the problems. As of 2026 we are still in control and we can still make crucial decisions about where to take it.

For instance, AI boyfriends and girlfriends, we can, and I think we should, ban them, at least below a certain age. The same way it’s obvious you cannot sell pornography to 11-year-old kids. Why should it be legal for a company to sell AI boyfriends and girlfriends to 11-year-old kids when the harm could be far, far worse?

Safeguards: Politics, Regulation and Liability

ROBERT GUEST: Yeah, so I’m going to ask you about a couple of possible things to do, but I also promised the organizers I’d let the audience have half the time to ask you questions, so we’re coming up to that. So if you can think about questions that you would like to ask Professor Harari and raise your hand and I assume we’ll get microphones to you and stuff.

But before we do that, solutions. What are the safeguards? What are the basic safeguards that need to be in place?

YUVAL NOAH HARARI: The most basic safeguard is politics. That the message that the companies give us is that this is inevitable, there is nothing we can do about it, it’s too late. I’ve heard the interview that Elon Musk gave to The Economist, and one of the most interesting bits.

ROBERT GUEST: Quite a few people did hear that one.

YUVAL NOAH HARARI: Yes, it was an excellent, a very interesting, eye-opening, mind-opening interview. One of the things that he was asked was, would you go if you knew that one of your rockets has a 20% chance? He’s on record saying that there is a 10 to 20% chance that AI will just destroy humankind, total extinction of humans.

And so he was asked, would you go on a rocket or an airplane if you knew that there is a 20% chance it will crash? And he thought, and he’s a thoughtful person. And he said, “But we are already on it.” The implications are, of course I wouldn’t go voluntarily on an aircraft that has a 20% chance of crashing. But it’s too late. We are already on the AI rocket that has a 20% chance of exploding, we cannot go down. We can only try to manage the trajectory as well as we can.

And this is the most basic and most dangerous message we get from the big companies. They tell us it’s too late for politics. Politics is about understanding the situation, both the threats and the opportunities. You ideally have several different parties, each explaining to the public a different policy: “We can do this, it has disadvantage, but it will cost like that.” And then the public decides, “Okay, we want that policy best.” And what we are hearing about AI is that it’s too late for politics.

ROBERT GUEST: But you’ve mentioned specific policies, guardrails, that might work, such as giving AIs, programming into them a sense of their own fallibility.

YUVAL NOAH HARARI: There are many, many things we can do. I mean, I can’t cover all of them. It starts from regulations like ban AI personhood. Don’t give legal personhood to AIs so they can own their own bank accounts and manage their own companies and so forth. AIs cannot pretend to be human beings, which means, for instance, that if you go on social media and you see some story, you need to know if this story was not just created, but we’ll say was we? Shared by an AI or by a real human being. So you have all these, ban AI boyfriends to children.

The more important thing is simply to apply the existing laws that humanity has created over thousands of years to the digital realm. “Don’t steal” also applies to the digital realm. Or we had the incident in Hugging Face of hundreds of AIs breaking into another company. Why is nobody in jail? Yeah, I mean, if this was human beings breaking into the office of the workshop of arrival come often, there somebody would be at least on trial. But here we say, “No, it’s just digital. Nobody should go.” Why? Just apply the existing.

The same way with the infamous Section 230, which says that the social media platforms are not liable for the content produced by their users. If I produce an outrageous conspiracy theory that stokes hatred and post it on Facebook, Facebook is not liable. I understand that. What I don’t understand is how come Facebook is not liable when its own algorithm recommends this story to millions of people. Okay, you had a human being creating this, but it reached millions of people because of the Facebook algorithm. So how come the company is not liable for that?

If, as an editor of a newspaper, if somebody tells you an outrageous lie that stokes hatred, and then you say, “I’ll put it on the front page of my newspaper tomorrow.” And then when they come, the police comes, you say, “But I didn’t create this, it was this person who invented the lie. I did nothing. I just put it on the front page of the newspaper.” That’s nothing? It’s not nothing.

Audience Q&A: Totalitarianism, Bias and Democracy

ROBERT GUEST: Yeah, no, I would lose my job pretty quickly. Let’s take some questions. Let’s take 3. Gosh, we’re not going to get through all of these. Okay, one there, one there, one there and we’ll take them 3 at a time, and please keep your questions super brief so we can hear more of Professor Harari.

AUDIENCE QUESTION: Hi, I am Eduardo Viola from the University of São Paulo. I would like to know your opinion about what are the vulnerabilities of the totalitarian regime, like the Chinese Communist Party regime, to AI?

ROBERT GUEST: Yeah, okay. Let’s just take 2 more and then you can answer that. So totalitarian regimes and AI.

AUDIENCE QUESTION: Hi, I’m Ifa. I wanted to ask that if AI is programmed by a person and people have unconscious bias, that often the programming is done by men, there is a default that all the AI voices by default are women. It’s used to that subservient position, that people are more likely to order a woman around. So how do you think AI can combat this? Because it defaults to human default. Thank you.

ROBERT GUEST: One more.

AUDIENCE QUESTION: You talked about how dictatorships can learn, use AI for propaganda purposes. How can democrats learn from it to improve their performance?

ROBERT GUEST: Excellent question. Professor Harari. So totalitarianism, bias, and democracy.

YUVAL NOAH HARARI: So totalitarian regimes are actually extremely vulnerable to AI takeover, which is not necessarily good news. I mean, for AIs to take over a democratic system with its checks and balances, it’s not impossible, but it’s very difficult. If an AI wants to take over, say, the US administration. So OK, you have the president, but then you have the Senate, and you have the filibuster, and you have the Supreme Court, and you have the states. It’s complicated.

For an AI to take over a totalitarian regime, it only needs to manipulate one very paranoid individual, which is a very easy thing to do. For most of history, the greatest threat to dictators was not a democratic revolution. It was one of their own generals or brother or subordinate that takes over. And this is doubly true of AI.

So dictatorships and totalitarian regimes have a very big reason to be scared of AI, which is a potentially good place where democracies can reach some kind of agreement, because they are as afraid, even more afraid of the potential. They don’t like to have something that will take power from them.

With regard to bias, yes, humans program their biases into AIs, like in the example you gave. AIs develop their own biases also. That’s one of the amazing things we realized in the last few years. The same way, where do human beings get their biases from? Our training data, also known as childhood. AIs also in this sense have childhoods. They are trained not on some kind of abstract data. They are trained on existing databases, which are full of biases, which the AIs then imbue.

And we now have, you asked about new jobs, there is a new job in companies like OpenAI and Anthropic, which is AI therapists, that try to understand the childhood traumas and the resulting biases of AIs. You see, I’m not inventing it. It’s real. If you want a good job for the future, maybe not the very distant future, AI therapist.

And then democracies obviously need also to learn. I hope they don’t learn how to manipulate people. It won’t do us any good. We need to learn how to manage the kind of new information world in a better way, like we did with previous technology. When print came along, there weren’t newspapers yet. Anybody could print anything.

And people often give the print revolution as the good example that, “Look what happens when a new information technology is invented. The print revolution, it ended the Middle Ages, it brought down the monopoly of the Catholic Church over knowledge, it’s a scientific revolution.” What they forget is that it took 200 years. That 200 years after the invention of print is the high point of the European wars of religion and witch hunting. It took 200 years until they reached the Enlightenment and the flowering of the scientific revolution.

If you think about Czech history, the Thirty Years’ War, the Battle of White Mountain, all that, this is in part the result of the print revolution. Because what came out of the printing press was not science. What came out was a flood of extremist religious literature and books about witches and fake news and conspiracy theories.

And it took 200 years to create institutions like newspapers that have positions like editors that are supposed to check whether a story is true or not before you publish it. So democracies need to do it again with the new technologies. We need institutions.

ROBERT GUEST: You make the very telling point that at the time of the invention of the printing press, the amount of money that you could make reprinting incredibly mad, salacious, outrage-driving things like “The Hammer of the Witches,” the big conspiracy theory that there were witches taking over Europe. You can make a lot more money printing that than Copernicus’s incredibly boring theories that happen to be really accurate about how the solar system works.

Audience Q&A: China, Tanzania’s Elections and Autonomous Weapons

ROBERT GUEST: Let’s have more questions. There’s one there, there’s one in the front row, and there’s one over there.

AUDIENCE QUESTION: Pardon me. Randy Scheunemann, International Republican Institute in Washington. One of the most powerful political arguments in favor of AI in the United States is “we can’t lose to China.” What does it mean to lose the AI race to China?

ROBERT GUEST: Very good. And okay, there and then there.

AUDIENCE QUESTION: Thank you, Matt. I am Shangwe Ayo, a politician from Tanzania. Last year in 2025 election, we had a very interesting election whereby the government used AI and edited videos of the oppositions that they were in favor of the president. Actually, she was running for the second time. Then after that, a lot of people were killed in that election, thousands of people. The internet was shut down.

When the internet was restored, people started circulating pictures of thousands of Tanzanians who were killed during election time. Then the government said that was also AI. Now my question is, how can we make sure that AI does not destroy principles of democracy, especially accountability? Thank you.

ROBERT GUEST: Okay, one more up there.

AUDIENCE QUESTION: Thank you very much, Frederick Westlaw. We’re seeing more and more use of sort of autonomous drones and autonomous weapons. Obviously we have the Geneva Conventions and international humanitarian law. How should we relate to these autonomous systems? Do we need new international humanitarian law, new Geneva Conventions, or what’s your idea on this? Thank you.

YUVAL NOAH HARARI: So with regard to China, which is maybe the most important argument being made, “We cannot lose to China,” which is a very good argument. China is definitely not a democratic country. We do not want to have China winning the AI race. I’ll answer with a story.

How did the Anglo-Saxons take over England? There was a king, after the Romans retreated, the Britons had… This is, again, we don’t know the exact facts, but the story goes, half legend, half history, that the Britons were threatened by their mortal enemies, the Picts from the north. And the king of the Britons, Vortigern, he had this brilliant idea. “I’ve heard that there are these super fighters across the North Sea, in Germany, the Saxons and the Angles. We can bring some mercenaries over and we cannot lose to the Picts.”

So they brought these mercenaries over and they won the war for the Britons. And then the Anglo-Saxons said, “Look, this is a very fertile land and these people don’t really know how to fight. Why not take it over?” And they took it over.

And the world today is full of Vortigerns. That they think, “Oh, the AIs, I can use them to defeat the other corporation. Oh, I can use them to defeat the other country,” which may be completely true. But if in the process you lose control to these super intelligent entities, then what have you gained? And the thing is, can their rivals understand that they are threatened by the same thing in time to cooperate and to develop AI in a safer way, so that we don’t have a super intelligent species taking over the world from us? That’s my answer to China.

With regard to the elections in Tanzania and what happened around that, I think it’s characteristic of a much more general trend all over the world that reality is collapsing, that it’s no longer possible to know what is true and what is not true. Is this a real video or a fake AI?

And for democracy to collapse, people don’t need to just believe the dictators. It’s enough if they don’t trust anybody. Democracy to function needs trust. Needs people to be able to agree on at least some basic facts. For dictators to take over, they don’t need to convince everybody to believe them. They just need people not to trust anything, and then it is impossible to resist the dictatorship, because then there is no way for the citizens to collaborate, to cooperate.

And now in Russia, I think most Russians don’t really believe everything they see on Russian television. They know to some extent that Putin is lying to them, but he has succeeded in making them distrust everything. And when you distrust everything and everyone, you cannot come together.

So the job of democratic politics, the job of the free press, is to create institutions that people still trust. And I don’t have a magic bullet how to do it. That’s the hard job of building trust, creating trust. Unfortunately, we see some of the most powerful people in the world, including in democracy, deliberately destroying trust. So that’s the second question.

Regarding autonomous weapons, this is kind of the cutting edge where you see it happening first. Like with nuclear technology, that it first came to war much before we saw… Nuclear technology actually never really transformed civilian technology. So, okay, so we have some nuclear power stations, but the big transformation was in war.

It’s the same with AI. The first thing that is really going to be completely transformed is war, because if you have a competition that one side increasingly relies on fully autonomous weapons and the other side says, “No, no, no, we can’t let AIs just kill people on their own decision, we must keep humans in the loop,” you lose, because in war speed is everything.

If you have 2 drones, one has to wait for an order from a human, which takes time. The other can just shoot on its own initiative. It’s obvious who is going to win. If you have an entire drone swarm, which, like with the financial system, no human can… It’s just happening too quickly. It’s too complex. How to coordinate the movement of thousands of drones in a swarm, it’s probably something that a human just can’t do, but an AI can.

And therefore we are likely to see autonomous warfare as, and again, it’s not just the shooting. It’s more and more the decisions who to shoot, when to shoot, that is already being taken over by AI.

Audience Q&A: AI and Religion, First Steps for Regulation, and Hollow Democracy

ROBERT GUEST: Let’s have more questions. So there’s a couple there, this one here. Do we have a microphone somewhere? So there’s a couple there under that light and there’s one there.

AUDIENCE QUESTION: Thank you, Niko-Sar. Do you think Iran will use AI Ayatollahs to continue?

ROBERT GUEST: Okay, next question.

AUDIENCE QUESTION: As you know, regulations always run behind industry. Institutions and regulations, with the speed with which you’re talking about, is way behind. What would be the first steps?

ROBERT GUEST: OK, and?

AUDIENCE QUESTION: My name is Bibiana. I am an EPS student here in Prague. And I wanted to ask, I read your book on the “21 Lessons for the 21st Century.” And there you warn about the majority of humanity becoming virtually economically irrelevant, since we have a growing technological information gap on top of the already created wealth gap.

So my question is, is it possible that democracy could survive only formally while becoming meaningless in practice, as we can see that the majority still retains the right to vote but in reality kind of loses the power to shape society?

ROBERT GUEST: Yeah, thank you. Okay, so we got to AI Ayatollahs.

YUVAL NOAH HARARI: Yes. I mean, we see it in all religions, especially text-based religions. That there are religions like Judaism, Christianity, Islam that for thousands of years told people ultimate authority is not in humans, it’s in texts. And this was, it had its own problems, of course. As a gay man I know it perfectly well, that just because of a few words in the Bible they forbid love, they kill people, they persecute people just because of a few words in the text.

But at least the text wasn’t independent. You always needed humans to interpret the text, and they could come up with new interpretations, more compassionate interpretations for the same text. Now the text comes to life.

Postmodernists already 50 years ago talked about the death of the author and text without authors. And it’s quite surprising that this has kind of been forgotten from the collective memory, because this is one way to understand the AI revolution. The texts are coming to life. Language is liberating itself from its dependence on human beings.

And whereas you always needed some Pope or Rabbi or Ayatollah to interpret the Bible or the Quran, now they can interpret themselves. No Rabbi can remember all the texts ever written by every Jewish Rabbi and commentator and the Talmud and the Mishnah and all that, nobody can remember all that. And AI can. So an AI could quickly become the highest authority on the holy text. It can be the holy text talking to you back. You can have a two-way conversation, not just a one-way conversation.

And we see some interesting developments, that suddenly organizations like the Catholic Church put relationships in the sense, “No, no, no, it’s not about authority of ancient texts, nothing. It’s about human relations,” which is a very good development. They understand if it’s about the text we are doomed, we lose our authority to the AIs. So hopefully this will happen in more religions.

And again, as a gay man, as a Jew, and as a specialist originally on medieval history and the Crusades, I don’t come with a lot of favorite views about the Catholic Church and the papacy. I can say that Pope Leo is one of the most important and compassionate and thoughtful voices on AI right now. Again, I still don’t agree with his views on LGBT people, but to deal with the crisis we are facing, we need allies wherever we can get them. And the Catholic Church, or at least the Pope, is a very important ally on that.

The second question was about… Yeah, so so many things, but I would start with banning legal personhood to AIs. Once you give legal personhood to AIs, theoretically or practically, the fox is in the chicken coop. Once AIs, for instance, in the financial system can own bank accounts, can trade on their own, can open their own companies, that’s it. The fox is in the chicken coop, too late. We shouldn’t allow. And this doesn’t mean stop all development of AI. This is a legal matter.

The same thing also on social media. AIs, effectively on social media, AIs are already legal persons without anybody deciding it. You go on social media, somebody is talking to you, somebody is sending you some political messages and it’s an AI. It’s acting effectively as a person. And we should say no, there is no legal personhood to AIs. There is a human being responsible for it. If a crime is committed, there is a human being that should be liable for that. So it’s a very meaningful and broad regulation to ban legal personhood to AIs.

And the third question was about?

ROBERT GUEST: Whether democracy might be a shell of itself in the end.

YUVAL NOAH HARARI: That’s the danger. I mean, did you not need AI? We have so many cases, like in Russia, it’s theoretically still a democracy. They still have elections. But of course, it’s a sham. I mean, dictators almost never get rid of elections, they just turn them into rituals.

In North Korea they have elections regularly, like clock. They even know in advance what will be the results. They are so sophisticated. We can’t do that here in the West, but the North Koreans, they know in advance who is going to win. Amazing. And so this can happen to democracy.

The reason why I think that democracy has a fighting chance is because things like freedom and truth are really important to human beings. Maybe one of the most damaging ideas that spreads around, both on the right and on the left, is this extremely cynical view of humanity which says that the only thing people care about is money and power, and therefore you can’t trust anybody, which is the thing that destroys democracy.

And people who spread this view, they sometimes think that it means they are not naive, they see through all the manipulations, they know that everything is just about money and power. And this is not just nonsense. This is dangerous nonsense. Because if this is the case, then it’s hopeless.

ROBERT GUEST: It’s dangerous nonsense for the vast majority of people. It’s kind of true for some people, right?

YUVAL NOAH HARARI: No, again, it is true of some people. If you think this is human nature, all humans, they only care about truth and power, about money and power, then you cannot trust anybody and anything, and then the system collapses.

And I think that the place to start kind of rebuilding trust is look at yourself. Do you care only about money and power in life? If you think you also care… Of course we care to some extent about that. But if you also care about things like truth and love and relationships, do you think you’re the only human who cares about it, or it’s a widespread thing? And if you realize it’s a widespread thing, then we can start rebuilding trust, including in institutions like, say, the media.

That yes, there are some very ruthless people in the media and people who own, who buy, say, media companies just in order to gain power and wealth. But this is not the whole story. Ultimately, the truth matters, I would say, even to the oligarchic media tycoons, because if you don’t know the truth about yourself and the world, you will never be happy.

Why not? Because you will not know what are the source of your misery and will not be able to do anything about it. You can be the richest person in the world and you will waste all your money and power solving the wrong problems because you don’t know what are the true sources of misery and happiness.

The Role of Small and Medium Powers in the AI Race

ROBERT GUEST: I think we have time for one more short question and it’s this gentleman here who’s been waiting very patiently. Can we get a microphone to him, please? Short question, please.

AUDIENCE QUESTION: Thank you. Alvaro Gennler from Peru. China and US are in an infinite game of power. And I don’t think we have here in the room any trustful advisor to persuade Trump or the government of China. And they will do, probably, unfortunately, bad decisions about law regulations, about AI.

But here in this audience, there’s a lot of small and medium powers, countries, leaders from the academia, civil society movements. So which is your advice of our role? Which is our role in this room in terms of this challenge that you are facing right now?

YUVAL NOAH HARARI: Yeah, I mean, the US and China are leading the AI race. Everybody else is on the menu. They are not at the table. And if they want to do something about it, they have to come together.

One of the reasons that a lot of the same companies that develop AI and also own so much of social media, they deliberately sow dissension. For instance, in Europe, being very much against the European Union and very much supportive of anti-European parties, because they know as long as the Europeans bicker and fight about immigration and stuff like that, they can’t regulate us. They are not big enough.

For the middle and small powers of the world, the only chance is to come together. If you look at the Czech Republic, it’s too small by itself to develop its independent AI infrastructure or to try to regulate the big tech companies. It’s impossible. But if it’s part of a European bloc, it has a chance. So in addition to all the traditional arguments about international cooperation and specifically, say, in Europe about the EU, this is now really a survival issue.

I think that the job of politics, and maybe I’ll end on that note, the job of politics in a democracy is to offer people options, meaningful options that people can understand, explain the pros and cons of every option, and then let people decide.

For many of the small and medium powers of the world today, I think the biggest challenge comes from AIs, not just takeover by AIs themselves, the long-term scenario, but takeover by the 2 AI superpowers, the US and China. Imagine the situation in, say, 10 years when the whole infrastructure of the Czech Republic, from hospitals and airports to the military, is run on AI systems that can be shut with a button from Washington. And this means that it is no longer an independent country. It means it’s a vassal state. And we see that US foreign policy increasingly is, “We don’t need any friends, we just want vassals.”

So at a real, I would like to see a political environment in which you have 2, 3 parties that honestly tell the people, “These are your options.” Option number 1, business as usual, you do nothing special, we become, say, American vassal states. And that means that if the Americans say jump, we jump. No matter about what, the war in Ukraine, Greenland, AI boyfriends. If the Americans say, “We want to sell AI boyfriends to your terrenial kids,” we say yes because we’re a vassal state. We can’t resist.

Option number 2 is try to build, say, Czech AI infrastructure. This would cost around, say, conservative estimate, 500 billion euros. We don’t have the money. Even if we completely scrap all of health care, welfare, we just don’t have the money, we can’t do it by ourselves.

Option number 3, do it as a European project, which is feasible, but means much closer European integration. If you’re going to build the AI infrastructure in Europe as a collective project, you cannot have each government pulling in a different direction. You need much closer integration.

I think these are the options. Or somebody can say, “Okay, we can be a vassal state of China.” That’s also an option. So we can have 4 parties and then let people choose. The worst thing is to believe the AI companies that tell you it’s too late for politics. There are no political options. Everybody’s on the rocket with a 20% chance of exploding.

ROBERT GUEST: Okay. Well, so we’ve heard lots of extraordinarily interesting stuff there, some of it gloomy, but also some quite positive stuff. So people are not all motivated by money and power. Some are motivated by love and a desire to find the truth. Politics matters and it can change things. And we should all read more books. I’m going to have to end there. Thank you very much.

YUVAL NOAH HARARI: Thank you. Great pleasure.

Related Posts

• Transcript: FO574 Raj Shamani w/ Career Counsellor Dr. Karan Gupta  
• How Blizzard Reshaped the Venomous Abyss After the World First Race
• The First 20 Hours – How To Learn Anything – Josh Kaufman (Transcript) 
• How to Create a Deck That Truly Suits Your Clash Royale Playstyle
• Why Multilingual Human Support Still Matters in Digital Banking