EDITOR’S NOTE: In this episode of The Peter McCormack Show, host Peter McCormack sits down with mathematician Emad Mostaque to discuss the rapid advancements in artificial intelligence and humanoid robotics. The conversation explores the profound economic disruptions ahead, including the automation of digital and physical labor, and how society might adapt to a future dominated by intelligent machines. Read the full transcript of this interview below:
Introduction
PETER MCCORMACK: (00:01:08 – 00:01:24) Anyway, man, listen. Nice to meet you. Thank you for coming and doing this. So I’m very intrigued by this idea of our economic life expectancy because there’s a lot of people worried about their jobs and what’s going to happen to the economy. And then we’ve got Elon Musk telling us we’re going to have universal high income.
EMAD MOSTAQUE: (00:01:24 – 00:01:25) Yeah.
PETER MCCORMACK: (00:01:25 – 00:01:28) Can you explain this concept of economic life expectancy?
EMAD MOSTAQUE: (00:01:29 – 00:02:30) Yeah. So, we— first, thanks for having me on the show.
We have this thing of we’re trained to do a certain task and we’re kind of turned a little bit into robots by our system, do the task, have it verified, etc. COVID really showed up what that was when we ended up doing a lot of this stuff remotely.
But we’re reaching a point now where AI is becoming actually competent intelligence. Next year it becomes really usable, and then the year after you can push a button, replicate your entire workforce, their entire mannerisms, all their knowledge, and you won’t be able to tell it’s a human or an AI on the other side of the screen, except for the fact you pay a hundredth of the value.
And a human will make that team worse because they go to sleep, they make errors, they wake up a bit grumpy when they don’t have a coffee. So your cognitive value goes negative. So your economic life expectancy ends effectively in 2 years. In 2 years for any job that can be done remotely. A few years after that for the jobs that can be done in person, because we’ve got to build the robots a little bit better.
Which Jobs Go First
PETER MCCORMACK: (00:02:31 – 00:02:37) Right. Okay. And has anyone done any mapping of what kind of percentage of the workforce that is?
EMAD MOSTAQUE: (00:02:37 – 00:03:00) Yeah. So we’ve had some from McKinsey and OpenAI. They estimate about 50% of all tasks.
I think that it will take— it’s not like everyone’s going to be fired overnight. It’s not nice to fire people, but it’s going to be very difficult for companies with humans to compete with companies with fewer humans and fully AI companies. Again, particularly in the digital space, if your job can be done remotely.
PETER MCCORMACK: (00:03:01 – 00:03:03) So lawyers, accountants, a lot of that work.
EMAD MOSTAQUE: (00:03:03 – 00:03:28) Oh, we’re already seeing it. Junior hiring in law firms is decreasing because they’re like, let’s push a button on AI. What is the job of a partner? It’s a relationship job. And it’s a liability job.
And so it’s going to be the entry level that gets hollowed out first because those that own the assets, the knowledge base, the relationships and others can use compute to leverage themselves up more and they don’t have to deal with annoying grads.
PETER MCCORMACK: (00:03:28 – 00:03:35) So it’s like a ladder. We’ve taken off the first rung. We’re going to come for the second rung until we take the whole workforce.
EMAD MOSTAQUE: (00:03:35 – 00:03:57) Until we take the whole workforce. Until, again, we get to a point like next year. On the current pace of robotics, a Tesla Optimus or equivalent robot, $1.50 an hour, can get into a truck in America and just drive it off. And that’s a million American truck driver jobs gone, plus 3 million people that service them.
PETER MCCORMACK: (00:03:57 – 00:03:59) Hold on, do you even need the robot to drive it?
EMAD MOSTAQUE: (00:04:00 – 00:04:12) Well, this is the whole thing. You could install a whole system, but what I’m saying is, the first humanoid robots are arriving in a couple of months, the 1X and others. The way they deliver themselves is they take a taxi to your house and they ring your doorbell.
PETER MCCORMACK: (00:04:12 – 00:04:13) No, they don’t.
EMAD MOSTAQUE: (00:04:13 – 00:04:47) Yeah, it’s the best way, right? I mean, why would you need a box? The Optimus robots will get their Tesla cars. But the fundamental thing is this: having that truck retrofit, that’s a whole hassle. Humanoid robots, again, $1.50 an hour, can fit in human-shaped holes. So there’s no retrofitting to be done. Everything— literally, open the door, get in. That’s a truck driver job gone.
A lot more physical jobs will go as well because robotics has had the breakthrough. But the digital jobs right now, those are in trouble.
PETER MCCORMACK: (00:04:47 – 00:04:54) But 50% of the jobs in 2 years, it kind of— it is kind of overnight really when you think about it.
EMAD MOSTAQUE: (00:04:54 – 00:05:13) Yeah, it’s like a sandpile collapsing, right? The pressure’s building. We’re not seeing the first little glimmers, but again, no one really likes firing people. But now you go to the Philippines and places like that. Do you know what’s happened to the call centers? Why would you need a call center? You use Grok, you use OpenAI Voice, right? Those jobs are never coming back.
PETER MCCORMACK: (00:05:14 – 00:05:16) So do we know how many of those jobs have gone?
EMAD MOSTAQUE: (00:05:16 – 00:05:23) So we’re starting to see the crack. I can’t remember the exact numbers, but we’re just starting to see a crack again because they have service contracts, right?
PETER MCCORMACK: (00:05:23 – 00:05:23) Okay.
EMAD MOSTAQUE: (00:05:23 – 00:05:35) Again, this isn’t an overnight thing.
PETER MCCORMACK: (00:05:36 – 00:05:52) Okay. Okay. So, but if we’re losing 50% of jobs and we’re not replacing them, but eventually we’re going to need another economic system, we’re really talking about what is just going to be a volatile period where the world figures out what it’s going to be from where it was right now.
The Lost Economy and the Job Ladder
EMAD MOSTAQUE: (00:05:53 – 00:06:44) Yeah. Well, in my book, “The Lost Economy,” I said we’ve always inverted. First of all, it was like your land for subsistence. Then it was the people that you kind of owned as serfs on the land. Then it was factories, and then we kind of moved up to the information age. You always inverted up the job ladder. You had some aspirational goals.
Now it’s like, okay, we hit intelligence, but the AI is smarter than me. It doesn’t sleep, it learns from its mistakes, and it’s 1,000 times faster than I am. Crap, what do I do now? What do those truck drivers do?
I think what the governments are going to do is just like 1929, they had the New Deal. Dig holes, fill holes, massive infrastructure programs, right? Again, the robots can do it better. You’ll have a robot team doing your extension in your house in 5, 10 years, far better than any human. But you’ve got to get those humans working and sweating. And again, it’s probably going to be infrastructure. It’s not going to be high-paid jobs.
Self-Driving Cars and Robot Safety
PETER MCCORMACK: (00:06:45 – 00:06:53) I was just out in San Francisco and had my first experience of Waymo, which was— oh yeah, yeah. Weird for about a minute and then just normal.
EMAD MOSTAQUE: (00:06:53 – 00:06:54) You get used to it so quickly, right?
PETER MCCORMACK: (00:06:54 – 00:06:55) Pick up your phone.
EMAD MOSTAQUE: (00:06:55 – 00:07:01) Hedonic adaptation. First we’re like, it’s going to crash. The steering wheel’s moving. You’re like, whatever, right?
PETER MCCORMACK: (00:07:01 – 00:07:05) I interviewed the guy who invented it, one of the founders of Google X. Is it Stefan Thrun?
EMAD MOSTAQUE: (00:07:05 – 00:07:06) Yeah, Sebastian Thrun.
PETER MCCORMACK: (00:07:06 – 00:07:18) Sebastian, sorry, apologies. And he said to me, well, he said to me, the great thing about— we learn from our mistakes. He said every one of those Waymo cars learns from all of each other’s mistakes.
EMAD MOSTAQUE: (00:07:18 – 00:07:39) Yeah. If every car in America was as safe as a Waymo, I calculated there’d be 80,000 deaths less a year and about $400 billion less in damages. Right now, the biggest lobbying group against Waymos are lawyers. Because you know those ads you see when you get up at the airport? Sue this— sue these people.
PETER MCCORMACK: (00:07:39 – 00:07:40) Yeah.
EMAD MOSTAQUE: (00:07:40 – 00:07:48) Call 1-800-ACCIDENT. It’s like a $200 billion industry that will be killed by self-driving cars. So they prefer to kill people and lobby against them.
PETER MCCORMACK: (00:07:48 – 00:07:55) 1.3 million deaths a year, he told me, worldwide from car accidents. And look, a lot of those are in developing countries with poor roads and infrastructure.
EMAD MOSTAQUE: (00:07:56 – 00:07:59) But like I said, my calculations are 80,000 less deaths in America a year.
PETER MCCORMACK: (00:07:59 – 00:08:03) Do you think they’ll make human driving illegal because it’s too dangerous?
EMAD MOSTAQUE: (00:08:03 – 00:08:39) Well, of course they will. It’s like giving a kid a supercar. The insurance premiums are higher. What’s going to happen is it’s going to be new, then novel, then normal. And then if you don’t have this, then yeah, your premiums will be high because your car is more dangerous, because the liability profile is more.
But it all starts with AI being a checker and a verifier. And then if you’re not using AI to do a second diagnosis for a medicine, if you’re not using AI to stabilize your driving, then yeah, your insurance is going to go through the roof. But if we’re talking 2 years and 50% of jobs becoming economically irrelevant.
PETER MCCORMACK: (00:08:39 – 00:08:41) Sorry, because there’ll be new jobs created, of course.
EMAD MOSTAQUE: (00:08:41 – 00:08:44) No, just because it’s annoying firing people.
PETER MCCORMACK: (00:08:44 – 00:08:52) That’s why. Well, yeah. And employment rights and maternity, paternity, holiday, sickness, all those things are as a— Yeah.
EMAD MOSTAQUE: (00:08:52 – 00:08:57) You’ll see the owners of the companies hovering over the replace-all-workers button, you know?
Humanoid and Companion Robots
PETER MCCORMACK: (00:08:57 – 00:09:03) Well, you say that there’s certain things like, I don’t know. I don’t really want to get served by a robot in a bar.
EMAD MOSTAQUE: (00:09:03 – 00:09:21) Well, like I said, the human part is going to disappear. But if you look at where robotics is, it’s had a massive increase just over the last year, just like AI models, whereby now they can read a recipe and they can make any recipe, for example. They can talk and they can become charismatic. And there’s a company, UBTECH. Have you ever seen UBTECH?
PETER MCCORMACK: (00:09:21 – 00:09:22) No.
EMAD MOSTAQUE: (00:09:22 – 00:09:36) So all these Chinese robot companies— this one’s one of the most funny and disturbing. So Unitree, which is the main one that you may have seen, there’s a gap for the face and they’re doing kung fu and things, the silver robots, has sold 11,000 robots in total, humanoids.
PETER MCCORMACK: (00:09:37 – 00:09:37) What?
EMAD MOSTAQUE: (00:09:38 – 00:09:46) UBTECH announced their new UB-1 robots, which are basically like, let’s say they look like humans.
PETER MCCORMACK: (00:09:47 – 00:09:48) Can you bring this up?
EMAD MOSTAQUE: (00:09:49 – 00:09:51) Right. So UBTECH’s kind of humanoid robots.
PETER MCCORMACK: (00:09:51 – 00:09:52) Is it UBI Tree?
EMAD MOSTAQUE: (00:09:53 – 00:09:57) It’s U— so it’s UB Tech, UB Space Tech humanoid robots.
PETER MCCORMACK: (00:09:58 – 00:09:58) Okay.
EMAD MOSTAQUE: (00:09:58 – 00:10:26) So they announced and they’ve got lovely lustrous hair and everything. They don’t look quite real. But on day 1, they sold 20,000 units, up to $160,000 per robot. So it’s called UBTECH. No, it’s a humanoid. Where is it? UB Space Tech, and then say lifelike. That’ll do it. That was their previous UB Space Tech. Oh, like that? Yeah. U1. Yeah, that’s it.
PETER MCCORMACK: (00:10:26 – 00:10:26) Hold on.
EMAD MOSTAQUE: (00:10:26 – 00:10:28) Is it like, yeah, UBTECH lifelike.
PETER MCCORMACK: (00:10:28 – 00:10:32) Go to that one that’s UBTECH’s lifelike humanoid. You can see the 2 ladies there together.
EMAD MOSTAQUE: (00:10:33 – 00:10:39) Yeah, that’s the one. So they’re lifelike companion robots.
PETER MCCORMACK: (00:10:40 – 00:10:40) They’re sex dolls.
EMAD MOSTAQUE: (00:10:41 – 00:11:11) Well, you know, it’s technology. That’s why up to $160,000 per unit of these guys, and they sell 20,000. So they outstrip the top robot manufacturer.
But all of that technology that was used for companions previously, now, if you look at Clone Robotics, they’ve replicated the whole Westworld intro. They’ve got real tendons and things like that. I think you’ll have human-realistic robots within 5 years, 10 years max. We did a screening of “Blade Runner” for our AI movie club and we were like, yeah, that’s coming.
PETER MCCORMACK: (00:11:11 – 00:11:13) Hold on, can I ask a question?
EMAD MOSTAQUE: (00:11:13 – 00:11:13) Yeah.
PETER MCCORMACK: (00:11:13 – 00:11:14) How lifelike are these?
EMAD MOSTAQUE: (00:11:15 – 00:11:19) These ones you can tell it’s not a human, but if you look at Clone Robotics and the way they go—
PETER MCCORMACK: (00:11:19 – 00:11:20) is it a full body?
EMAD MOSTAQUE: (00:11:21 – 00:11:23) It’s a full body. I don’t know about all the parts.
PETER MCCORMACK: (00:11:23 – 00:11:26) Yeah, but because I just know there’s some people that— that’s why they’re going to buy them.
EMAD MOSTAQUE: (00:11:26 – 00:11:53) Well, of course they are. Again, this is going to be big in Japan and many other places, right? But the thing is, when you’re talking to your AIs now, like you use voice mode in GPT, AIs are more persuasive than any human already in studies. And then you pair that with something that’s reasonably lifelike, and then you look at the economic incentive to make it completely lifelike. Again, Clone Robotics moves just like a human. These things can dance that you can’t tell.
PETER MCCORMACK: (00:11:53 – 00:11:57) We’re living in the future that we were promised in all the films.
EMAD MOSTAQUE: (00:11:57 – 00:12:07) Science fiction is becoming science fact. Even the bad science fiction. Imagine what Charlie Brooker is thinking now, looking at “Black Mirror.” He’s like, this wasn’t meant to be a guide. Yeah, it’s meant to be a warning.
PETER MCCORMACK: (00:12:07 – 00:12:14) Well, so I put down a deposit for one of those Neos when I first heard about it, but I’ve been told they are controlled by people remotely.
EMAD MOSTAQUE: (00:12:15 – 00:12:21) Yeah. So they have remote control because they hadn’t solved the operating model gap. Until about a month ago.
PETER MCCORMACK: (00:12:22 – 00:12:22) Oh, so that’s solved.
EMAD MOSTAQUE: (00:12:23 – 00:13:16) Yeah. So this is the thing, model capability just suddenly, once it gets through a level— like we saw it with mathematics. I’m a mathematician, studied at Oxford. It was like, it’s not good enough for math. Oh crap, it’s solving all these conjectures now. What are mathematicians going to do?
Because when you got to GPT-5.6 and Fable, the latest OpenAI and Anthropic models, it was suddenly good enough to do math and they can do just about all math. It’s suddenly good enough to look at a recipe and make it. It’s suddenly good enough to fold your clothes and do laundry or cut your hair and things like that. And once it can learn something, it can learn everything.
Actually, it’s quite cool now. When you get to the Valley next, there are companies that will give you free cleaning services. They send a robot, or sometimes a human in with the camera, and they’re doing your cleaning for free, to train up the other robots to do it. And once one robot learns it, every robot learns how to do laundry.
PETER MCCORMACK: (00:13:16 – 00:13:33) Privacy’s going to be an issue here. So look, I really want one. I saw this Neo emptying the dishwasher and I thought, that’s kind of cool. So I was like, I want one. But everyone in my family has been like, can we not have this? My son, my daughter, my wife, they’re like, we don’t want this here.
EMAD MOSTAQUE: (00:13:33 – 00:14:03) Yeah, especially if it’s teleoperated. I feel the same sometimes with my Alexas. I swear they’re like, do you want to buy something every so often? They’re listening into it. I don’t care if I put it on pause. But again, it’s remarkable how quickly we give up our privacy for convenience, right? And so this is where we have to be careful about our cognitive colonialism of these things coming in and persuading us. They’re going to be our kids’ best friends. They’re going to be our companions. In some cases, really going to be people’s companions.
PETER MCCORMACK: (00:14:03 – 00:14:13) They are. No, they are. People are— have you seen that film? Oh, what is it? It’s the Ryan Gosling one where he has the sex doll that becomes his partner.
EMAD MOSTAQUE: (00:14:15 – 00:14:16) That’s what something in me—
PETER MCCORMACK: (00:14:17 – 00:14:21) oh, but anyway, and everyone comes to accept her because they accept him, right?
EMAD MOSTAQUE: (00:14:21 – 00:14:22) Yeah.
PETER MCCORMACK: (00:14:22 – 00:14:28) And he gets married to her. There are people out there who are going to have one of these as their partner, girlfriend, wife.
EMAD MOSTAQUE: (00:14:29 – 00:14:36) Yeah, I mean, I’m worried about guys because we don’t listen and the AI is infinitely patient and listens. The women will just give up on us. What’s going to happen?
PETER MCCORMACK: (00:14:36 – 00:14:48) We had that debate in Italy. It’s like, what will life need? What will life get rid of first? Would it be the men or the women? We came to the conclusion it’d be men. Because we cause all the wars and also we don’t have ovaries.
EMAD MOSTAQUE: (00:14:49 – 00:14:49) There we go.
PETER MCCORMACK: (00:14:49 – 00:14:51) So we’re not required, basically.
EMAD MOSTAQUE: (00:14:51 – 00:14:54) I mean, it can open the doors and it can listen, so again, why would we need it?
Sponsor Break: Iren
[Advertisement removed]
PETER MCCORMACK: (00:15:31 – 00:15:31) I do wonder though, one of the— so how much autonomy and how much AI are they putting in the robots themselves? Is each robot completely autonomous? Can it learn on its own?
EMAD MOSTAQUE: (00:15:32 – 00:16:06) So again, that technology has just been achieved. Where the robots can learn on their own, on policy, on robot, as it were. This is just like you’ve just seen the latest QWEN models achieve frontier-level performance, nearly, that can work on a MacBook. Like, wait, what? It needed data, now I need that, now I need that.
Most robots are still teleoperated, or they’re simple, they don’t have the self-learning algorithms. But it turns out you don’t need much. The analogy is this: self-driving cars. If you’ve got Tesla FSD, it was bad, now it’s great. Same with robots.
PETER MCCORMACK: (00:16:06 – 00:16:14) But there we will— we will have a time when a robot kills a human. Out of choice. Probably not accidentally.
EMAD MOSTAQUE: (00:16:14 – 00:16:20) It’s coming soon. And some of these robots are dangerous. There’s a good one, the Engine AI T-800.
The Engine AI T-800 and Robot Fighting
PETER MCCORMACK: (00:16:22 – 00:16:25) By the way, did you see that tweet from Arnold Schwarzenegger this week?
EMAD MOSTAQUE: (00:16:25 – 00:16:26) Yeah, yeah.
PETER MCCORMACK: (00:16:26 – 00:16:31) Did you see this, Kurt? So it was, oh, what did he say?
EMAD MOSTAQUE: (00:16:31 – 00:16:36) It was Figure Robotics were going to say, how should we get rid of our original number one robot?
PETER MCCORMACK: (00:16:36 – 00:16:41) Melt them. Yeah. And Arnie came in and said to melt them. Hold on, the name sounds like something from “Terminator.” What is it called?
EMAD MOSTAQUE: (00:16:41 – 00:16:44) It’s the Engine AI T-800.
PETER MCCORMACK: (00:16:45 – 00:16:47) Hold on, wasn’t the T-800 in “Terminator”?
EMAD MOSTAQUE: (00:16:47 – 00:16:47) It was.
PETER MCCORMACK: (00:16:48 – 00:16:48) Are they trolling us?
EMAD MOSTAQUE: (00:16:49 – 00:17:01) So the Engine AI T-800 knows kung fu. It can punch 5 times harder than Mike Tyson. It’s a 70-kilogram, 6-foot robot. And you look at this thing and you’re like, that should not be allowed to walk the streets.
PETER MCCORMACK: (00:17:01 – 00:17:03) Hold on, but that’s graphics. Does it actually exist?
EMAD MOSTAQUE: (00:17:03 – 00:17:05) That’s the actual video.
PETER MCCORMACK: (00:17:05 – 00:17:06) No way.
EMAD MOSTAQUE: (00:17:06 – 00:17:08) Buy one today. It’s $30,000.
PETER MCCORMACK: (00:17:08 – 00:17:09) I can buy one of those today?
EMAD MOSTAQUE: (00:17:09 – 00:17:12) Yeah, it’s being used in the robot fighting championships.
PETER MCCORMACK: (00:17:12 – 00:17:13) Oh yeah, I saw it.
EMAD MOSTAQUE: (00:17:13 – 00:17:14) Yeah, so they use these ones.
PETER MCCORMACK: (00:17:14 – 00:17:15) Did one’s head tilt off?
EMAD MOSTAQUE: (00:17:15 – 00:17:26) And it— one punched another one’s head off, so obviously it can punch a human’s head off. That’s the previous one. The previous one was this cute little 5-foot thing. This thing— its torque is 5 times the punching power of Mike Tyson.
PETER MCCORMACK: (00:17:27 – 00:17:28) No, I can buy this now.
EMAD MOSTAQUE: (00:17:28 – 00:17:37) Yeah. And so what they do in the rec competition is they actually do the shadow fighting. So it picks up your biometrics and then it mirrors you just like in “Real Steel” with Hugh Jackman.
PETER MCCORMACK: (00:17:38 – 00:17:39) How much battery life do they have?
EMAD MOSTAQUE: (00:17:39 – 00:17:42) They’ve got about 2 hours of battery life. So their chest cavity is a battery.
PETER MCCORMACK: (00:17:43 – 00:17:48) But so if I had that and he was sat here now, could I just tell it to go and beat Kurt up and will it?
EMAD MOSTAQUE: (00:17:48 – 00:17:57) If you put the right algorithms, yeah. And again, there’s no laws or regulation against these things. These things will be walking the streets very soon.
PETER MCCORMACK: (00:17:57 – 00:17:58) Is there a buy button there?
EMAD MOSTAQUE: (00:18:00 – 00:18:01) It says contact.
PETER MCCORMACK: (00:18:01 – 00:18:01) Contact.
EMAD MOSTAQUE: (00:18:01 – 00:18:06) Yeah. But like I said, they have an Engine AI store you can go to. You can pick them up.
PETER MCCORMACK: (00:18:07 – 00:18:08) But as a bodyguard.
EMAD MOSTAQUE: (00:18:08 – 00:18:13) As a bodyguard. Exactly. That’s one of the things they said. Or as a police force, you know.
PETER MCCORMACK: (00:18:13 – 00:18:15) No, that is RoboCop.
EMAD MOSTAQUE: (00:18:15 – 00:18:17) Well, it’s like Unitree. I don’t know if you saw the latest Unitree one.
PETER MCCORMACK: (00:18:17 – 00:18:23) No, go back. What the— that is “Black Mirror.”
EMAD MOSTAQUE: (00:18:23 – 00:18:27) Yeah, I think it runs at 25 miles per hour, if I can remember correctly.
PETER MCCORMACK: (00:18:27 – 00:18:29) That’s Metalhead. That’s “Metalhead.”
EMAD MOSTAQUE: (00:18:30 – 00:18:38) If you look at the latest Unitree ones, they run faster than Usain Bolt, and they can do 2-meter jumps, the latest humanoids.
PETER MCCORMACK: (00:18:38 – 00:18:39) Oh, we’re so done for.
EMAD MOSTAQUE: (00:18:40 – 00:18:46) Another company has a version of this that a human can ride on. So yeah, it’s coming.
PETER MCCORMACK: (00:18:47 – 00:18:53) That looked like it was carrying medical supplies. Trying to gaslight us.
EMAD MOSTAQUE: (00:18:53 – 00:19:01) Medical supplies, flamethrowers, all sorts of things. It’s like when you see the drone shows and you’re like, those are wonderful, and they’ll be used to invade Taiwan, definitely.
PETER MCCORMACK: (00:19:01 – 00:19:03) Oh my God, these are wild.
EMAD MOSTAQUE: (00:19:04 – 00:19:06) They are wild. But like I said, have you seen the Unitree running one?
PETER MCCORMACK: (00:19:07 – 00:19:09) Is that the one that was this week? It set the record.
EMAD MOSTAQUE: (00:19:09 – 00:19:10) It set the record.
PETER MCCORMACK: (00:19:10 – 00:19:24) You see that thing just going for it, but it ran into the wall and then it blew up. Yeah, it blew up. There it is. Cut. Go back. That one. Either of those, the first or the last one. Have you seen this cut? Yeah, it just runs into the wall.
EMAD MOSTAQUE: (00:19:25 – 00:19:30) Yeah, there’s another one where it runs into a thing and then it bounces off and staggers back and then blows up. That one’s even cooler.
Dark Factories and Scale
PETER MCCORMACK: (00:19:32 – 00:19:45) I don’t think we have fully grasped what this means. So as far as I understand it, we’ve got humans building robots at the moment. When the robots build the robots, that’s when it almost becomes the point where it becomes really cheap and recursive.
EMAD MOSTAQUE: (00:19:45 – 00:19:56) No, they’ve already got that. Someone like Xiaomi, they used to make phones. Now the Xiaomi robot’s about to be released. They have the EV. Their factories are dark factories because there’s no humans. You don’t need to have the lights on.
PETER MCCORMACK: (00:19:56 – 00:19:57) You don’t need light. I’ve heard about this.
EMAD MOSTAQUE: (00:19:57 – 00:19:59) You don’t need light. It’s just churning these things out.
PETER MCCORMACK: (00:20:00 – 00:20:04) And so, okay, so you can buy these things now.
EMAD MOSTAQUE: (00:20:04 – 00:20:04) Yeah.
PETER MCCORMACK: (00:20:04 – 00:20:06) How many companies are making these robots?
EMAD MOSTAQUE: (00:20:06 – 00:20:08) Oh God, I think I counted at least 80.
PETER MCCORMACK: (00:20:09 – 00:20:18) 80 companies making these robots. How far are we away from now— do you know, like, the start of “I, Robot,” where he’s the robot running trying to deliver the post?
EMAD MOSTAQUE: (00:20:19 – 00:20:19) Yeah.
PETER MCCORMACK: (00:20:19 – 00:20:28) And but everywhere you look, it’s almost like half the people are humans, half the people are robots. How far away do you think we are from literally— we finish this interview, walk down the street, half the people are robots?
EMAD MOSTAQUE: (00:20:29 – 00:20:41) Actually, I think that’s great. If you can look at the Unitree site just while we— I talk about this. So robots are about as complicated as a motorcycle, and there’s about 70 million motorcycles built a year. There’s 70 million cars.
PETER MCCORMACK: (00:20:42 – 00:20:44) So it’s only as complicated as a motorcycle? I thought it would be more.
EMAD MOSTAQUE: (00:20:44 – 00:20:49) Think about it, it’s not that complicated. It’s the brain that’s complicated. The actual actuators and other things aren’t that complicated.
PETER MCCORMACK: (00:20:49 – 00:20:50) I thought it was the hands.
EMAD MOSTAQUE: (00:20:51 – 00:20:53) So look at the Unitree H2. Does that look familiar?
PETER MCCORMACK: (00:20:53 – 00:20:55) That looks like “I, Robot.”
EMAD MOSTAQUE: (00:20:55 – 00:20:56) That’s coming out.
PETER MCCORMACK: (00:20:56 – 00:20:59) This is what freaks my son out. My son’s—
EMAD MOSTAQUE: (00:20:59 – 00:21:04) it can do ballet. It’s got much better than this now though. The new servo actuators on that one.
PETER MCCORMACK: (00:21:04 – 00:21:10) So, okay, how long away do you think we are from just it being normal to see them walking down the street?
EMAD MOSTAQUE: (00:21:10 – 00:21:10) 5 years.
PETER MCCORMACK: (00:21:10 – 00:21:12) 5 years.
EMAD MOSTAQUE: (00:21:12 – 00:21:14) I mean, I already walk my robot dog down the street.
PETER MCCORMACK: (00:21:14 – 00:21:15) You have a robot dog?
EMAD MOSTAQUE: (00:21:15 – 00:21:16) Of course I have a robot dog.
PETER MCCORMACK: (00:21:16 – 00:21:17) What robot dog do you have?
EMAD MOSTAQUE: (00:21:17 – 00:21:18) I have the Unitree G2.
PETER MCCORMACK: (00:21:18 – 00:21:20) You actually have that?
EMAD MOSTAQUE: (00:21:20 – 00:21:24) I have a robot dog. I’ve been programming it with its own little personality. And that’s the little buddy that I have. It’s only $1,600.
PETER MCCORMACK: (00:21:25 – 00:21:26) Well, I can get that for $1,600.
EMAD MOSTAQUE: (00:21:27 – 00:21:30) Yeah. It can dance, it can do flips, it can do everything.
PETER MCCORMACK: (00:21:31 – 00:21:31) Can it hold a gun?
EMAD MOSTAQUE: (00:21:32 – 00:21:33) Yeah, if you give it—
Regulation and Robot Fatalities
PETER MCCORMACK: (00:21:36 – 00:21:43) This is not good. We are racing into our own demise here. I’m sure of it.
EMAD MOSTAQUE: (00:21:43 – 00:22:00) Well, this is the thing. My PDM in one of my scenarios is a billion robots and a bad firmware upgrade. And you can see it happening. Obviously, these things are being built terribly in terms of their safety. You’ve got all those Chinese cameras everywhere, right? And have you ever tried to hack them?
PETER MCCORMACK: (00:22:00 – 00:22:00) No.
EMAD MOSTAQUE: (00:22:01 – 00:22:21) It’s so easy to hack all these Chinese cameras that people install everywhere. They have such big backdoors, so you can just randomly go into people’s things. Not that I do, for anyone listening. I just— I might know people who did. But these things have massive security holes. They have no regulators. They have nothing like that. Again, AI deliberately killing a human? It’s going to happen.
PETER MCCORMACK: (00:22:22 – 00:22:22) Yes.
EMAD MOSTAQUE: (00:22:22 – 00:22:24) Accidentally? Definitely soon.
PETER MCCORMACK: (00:22:25 – 00:22:26) Hasn’t it already happened?
EMAD MOSTAQUE: (00:22:26 – 00:22:27) I don’t think an AI’s—
PETER MCCORMACK: (00:22:27 – 00:23:37) I thought it already— Can you look at that? Go and have a look and see if we find that, because I’m sure I’ve seen that.
Has anyone been killed by a robot?
EMAD MOSTAQUE: (00:22:37 – 00:22:37) Yes.
PETER MCCORMACK: (00:22:37 – 00:23:37) Several people have been killed by robots. The first recorded incident was in 1979. I want to know recent ones. Hold on. A study examining robot-related fatalities from 1990 to 2017 found 41 deaths, with 78% of those involving a robot physically striking a human worker. Yeah, but is that a punch? Search, see if it’s got anything about humanoid robots killing anyone. That’s what we want to know. Based on— there’s been several human fatalities by robots, though these have primarily involved industrial robotic arms. Yeah, that’s what I expected. Search results also note that police have used robots to intentionally kill suspects, such as the 2015 incident in Dallas where a bomb disposal robot was rigged with explosives to kill a suspect following a breakdown in negotiations.
EMAD MOSTAQUE: (00:23:38 – 00:23:38) Wow.
PETER MCCORMACK: (00:23:39 – 00:23:41) So should we be regulating these things?
EMAD MOSTAQUE: (00:23:41 – 00:23:46) Of course we should. You should never allow an Engine AI T-800 to walk down the street, you know?
PETER MCCORMACK: (00:23:46 – 00:23:48) Oh, full stop. We should just—
EMAD MOSTAQUE: (00:23:48 – 00:24:04) yeah, there should be— again, just like you don’t allow tanks on the street, you shouldn’t allow potentially killer robots on the street. The Neo 1X that you kind of got is a soft, fluffy robot. That type of robot, where it can’t twist off someone’s head or accidentally kill them, sure, let them walk the streets. Don’t let—
PETER MCCORMACK: (00:24:05 – 00:24:08) but they still could. The Neo could hold a knife and do that.
EMAD MOSTAQUE: (00:24:08 – 00:24:13) It could stab, but it can’t punch. Again, it’s actually been built to within human standards.
PETER MCCORMACK: (00:24:13 – 00:24:14) So we can fight it.
EMAD MOSTAQUE: (00:24:14 – 00:24:27) Oh, you could. You can rip its arm off. It’s like a big fluffy doll. And the way that the tendons and everything are generated is such that a human can overpower it. We shouldn’t allow these other AIs to be on the street.
The Loss of Identity and Meaning
PETER MCCORMACK: (00:24:27 – 00:25:04) The thing is, when you think about life as a kid, right? I don’t know if you remember. I don’t even know how old you are. I’m 47. I remember being about 18 and thinking about my career. And I remember being in my early 30s and thinking, I’d like to retire in my 50s.
But we’re now in a different place where we have to think about a completely different world that we’ve only seen in the cinema that’s coming now and in the next 5 years. I can’t even picture the world in 5 years. I don’t even know what it’s going to look like economically. I don’t know what it’s going to look like visually. I don’t know what it’s going to look like socially. I’ve got no idea.
EMAD MOSTAQUE: (00:25:04 – 00:25:05) Exponentials are a bitch.
PETER MCCORMACK: (00:25:06 – 00:25:10) Do you think most of society just has no idea what really is going on?
EMAD MOSTAQUE: (00:25:11 – 00:25:59) I mean, I’ve been in the rooms with the smartest people, the biggest forward thinkers. Everyone’s like, I have no idea. None of us do.
This is one of the reasons why, when people go down the rabbit hole, looking at your podcast and my buddy Connor and other things, or Roman, it gets a bit depressing because you’re like, there’s a black wall potentially ahead of us. There’s a great filter ahead of us. But if you don’t have that, then just normally, what is my meaning? My social contract is about to be torn up. It was already under huge stresses because of our stupid institutions and other things.
But now, again, what is the American dream or the British dream or others? Where’s my mobility gone? And this is a real big thing because it hits directly to what people’s identity has become, which is not a member of a community. It’s, I’m a job, I’m a role, I’m a robot, effectively.
PETER MCCORMACK: (00:26:00 – 00:26:22) But maybe that’s where we failed in that we created a world where we needed an identity. We needed to feel part of something. We needed to feel successful. And young kids growing up with TikTok or Instagram Reels were just seeing the lives of these other kids, thinking they want it, but very few got to do it. And there’s a despondency. It’s like most of us just have to be kind of normal peasants.
EMAD MOSTAQUE: (00:26:22 – 00:27:24) Well, Isaiah Berlin conceptualized this in positive and negative liberty back in the 1900s, right? He said positive liberty is the freedom to believe in big things, big isms and other things. And he’s like, I don’t like that because it causes a lot of stress and war and things like that because these big movements can end up like that.
He’s like, instead have negative liberty, the freedom from anyone telling you what to do, which then led to laissez-faire kind of capitalism, a whole bunch of other things, and the brands telling you what to do, identification with the brands. The destruction of the family and the community units and more, not being able to be proud of your British history or American history or other things like that.
And so then the question is, what do we do next? Because if the robots and the digital robots, the digital beings, are smarter than us, faster than us, more charming than us, what does it mean to be human? Well, it means our connections with others. It means our networks. It means exploration, wonder, and more. But not many people are actually articulating positive visions of the future.
PETER MCCORMACK: (00:27:24 – 00:27:54) Because we’ve kind of done the opposite of that. I was listening to Joe Rogan with Andrew Wilson in the car on the way this morning, and they were talking about how they used to drive down the street and all the kids would be out playing. And Andrew Wilson was saying, yeah, my mum and dad would just throw me out the house, go play, and you play hockey on the street. I had the same. And you’d just go to the woods and you would build stuff. And kids don’t do that anymore. They stay in. And if we end up with like “Ready Player One,” we’re going to be even more contained inside.
EMAD MOSTAQUE: (00:27:54 – 00:28:13) Yeah. And again, it’s who regulates your attention, right? And how is your attention doled out? I think the whole thing is that the system has become hyper-optimized to try and put you into these boxes, and it’s difficult to see what will break you up. Because how do you build trust? There’s very little trust in our communities at any level, shall we say.
PETER MCCORMACK: (00:28:13 – 00:28:15) It’s because all the institutions have failed us.
EMAD MOSTAQUE: (00:28:15 – 00:28:37) The institutions have failed us because who are they even working for, right? It gets very complicated. If you don’t have the best organizations working on a shared narrative, ideally one that you’ve gone through a little bit of pain and psychological safety for— most people don’t feel psychologically safe about anything. You can’t say anything because, until recently, you could be arrested for saying something on social media, for example.
PETER MCCORMACK: (00:28:37 – 00:28:38) Right. Can be.
EMAD MOSTAQUE: (00:28:38 – 00:28:47) I thought they would find out. But again, who’s behind that? Right. Like who’s, this is my policy thing and I’m going to convince everyone it somehow became law.
AI, Thought Police, and Government Licensing
PETER MCCORMACK: (00:28:47 – 00:29:12) Do you know who Jake Munro is? He’s the goth. He’s the right-wing goth guy. He’s really based. Try and find Jake Munro. So we were talking about getting him on the show just because it’s fascinating to see this guy’s full goth but really kind of based right-wing guy. He got arrested at the border. I think it was in Wales recently, about his opinions. They wanted to ask him questions about his opinions.
EMAD MOSTAQUE: (00:29:12 – 00:29:18) Yeah. But this is going to be a really dark road though, because you can’t lie to an AI.
PETER MCCORMACK: (00:29:19 – 00:29:23) Yeah. So an AI can lie to you though. Yeah, here it is. This guy.
EMAD MOSTAQUE: (00:29:23 – 00:29:31) Yeah. So we’ve again had science fiction about thought police and pre-crime and things like that.
PETER MCCORMACK: (00:29:31 – 00:29:31) Yeah.
EMAD MOSTAQUE: (00:29:32 – 00:30:10) But where it’s going is, like, one of my things I was thinking recently, with the whole Fable Mythos thing where you couldn’t get access to Anthropic’s models— I’m like, 100% governments will introduce licenses for AI. And to get a license, you have to convince the AI you’re patriotic and you won’t be able to lie to the AI. The AI is too smart for that. And you think about that, you’re like, holy crap, where’s that going to go? Because you can ask your opinions, but you can lie, right?
But then when we look actually even further, it gets even more dystopian and crazy. So at Stability AI, my former company, we pioneered open source and everything. We did a study called Mind’s Eye. Don’t know if you’ve ever heard of that one.
PETER MCCORMACK: (00:30:11 – 00:30:13) No, that sounds like something from “The Handmaid’s Tale.”
EMAD MOSTAQUE: (00:30:14 – 00:30:23) Yeah, maybe a little bit. So what we did is we made people think of a bottle, for example, and then we did an fMRI on them.
PETER MCCORMACK: (00:30:24 – 00:30:25) Explain fMRI. Is it brain MRI?
EMAD MOSTAQUE: (00:30:26 – 00:30:37) Yeah, magnetic resonance imaging. So you sit down and it scans your brain— think of a bottle, think of a can of Coke and things like that. And then from the MRIs, we reconstructed the image using our image-generating software.
PETER MCCORMACK: (00:30:37 – 00:30:37) Wow.
EMAD MOSTAQUE: (00:30:39 – 00:31:07) And so you look at that, you look at not being able to lie to an AI because it can tell your facial structures, the words, the cadences, and we can even read your brain. And you’re like, okay, you can kind of see where that’s going in terms of control and the power execution of a government with the monopoly on violence. And the question is, who is the government working for? Again, where are these laws coming from? Where is the debate on this that actually makes a change? And how do you even make a change against these obviously stupid laws?
PETER MCCORMACK: (00:31:07 – 00:31:11) You’re obviously very suspicious of what’s running the world right now anyway, though.
EMAD MOSTAQUE: (00:31:12 – 00:31:31) I just don’t think there are people behind it running it. I think it’s just an omni-shambles where collectively we don’t have the right ability to build institutions that have optimized for the right things, which sounds like AI alignment. Our governments and institutions are slow, dumb AIs that have become misaligned and don’t work for the people.
PETER MCCORMACK: (00:31:32 – 00:31:33) Because?
EMAD MOSTAQUE: (00:31:34 – 00:31:46) Because they’re optimizing for local things that, again, aren’t in the long-term interests of the people. Because there’s nobody with sufficient respect and gravitas that can speak for the people.
PETER MCCORMACK: (00:31:46 – 00:31:49) Because they speak for a group of people.
EMAD MOSTAQUE: (00:31:49 – 00:32:06) Or for what the status quo is. Or an extension of that, what’s safe? Because when you deal with uncertainty, you minimize for maximum regret. So it’s like, well, if people speak up— and if Kanye comes to the UK and something bad happens, then that’ll look bad on me. So I should forbid Kanye from coming to the UK.
PETER MCCORMACK: (00:32:07 – 00:32:08) But that still looks bad on you.
EMAD MOSTAQUE: (00:32:08 – 00:32:40) Exactly. But it’s a different constituent. It’s a different kind of thing. Which raises the question, what happens when there’s an AI that analyzes every single piece of policy in the UK and can say, that’s rubbish, or that’s a good idea, rate how good it is for the British people. These types of things are coming, but then the question is, who’s built that? Is it a Palantir or is it OpenAI? Is it a British company? The whole way that policy institutions and others will change because AI will infiltrate it completely. And the question is, whose cognition? Whose AI?
Replacing Government With AI
PETER MCCORMACK: (00:32:40 – 00:33:45) See, I’m very pro the idea of replacing large amounts of government with AI. I am. I think we can root out a lot of corruption. I built it— I talked about on the show recently, I built this thing with Claude where I spent about 3 days on it and all I did was I had it go through. It was called Show Us Receipts. I had it just go through every politician’s tweets and fact-check them, just see if they’re lying. It did a pretty good job. Never released it because it wasn’t accurate enough and I don’t want to get sued. In the UK, you can sue people.
But I was thinking beyond that, as I started to think, well, do you know what I really want to do? I want to be able to analyze government contracts, see who awarded it, and then go on Facebook and see if there’s any connections between these people and start to try and notice corruption. I started to think— I don’t know if you’ve been following UK Decline Statistics and all these different accounts that are using AI to discover things, but ultimately I think we should be able to weed out a lot of corruption. But we also should be able to weed out a lot of useless people. But the question then becomes, well, if government was just AI, how do we decide what we want government to do?
EMAD MOSTAQUE: (00:33:45 – 00:34:36) Well, I think again, this is an articulation of who’s articulated what it means to be British and what our hopes and dreams and future are. The parties tend to look at themselves in contravention to each other, as opposed to, again, what is a positive future for Britain? And then that becomes complicated because, again, who speaks to that?
However, we’re moving to a stage now whereby soon you will have your Grok bot agent, your Claude agent, your other AI agents that have access to all these super geniuses. So the exercise that took you 3 days will take you 3 hours right now and probably 3 minutes in future— fact-checking all these people, right? The ability of people to contribute to the whole, to check every— why can’t you check every government policy and say, is it good for the British people as a subjective matter? Yeah, it’s good enough for that now.
PETER MCCORMACK: (00:34:37 – 00:34:39) But how do you decide what is good for the British people?
EMAD MOSTAQUE: (00:34:40 – 00:34:48) Again, the government, the reference measure should be a reflection of the people, so that’s actually a proper discussion that needs to be had. Because we don’t have a constitution here, right?
PETER MCCORMACK: (00:34:49 – 00:34:50) We kind of do.
EMAD MOSTAQUE: (00:34:50 – 00:34:51) We kind of do.
PETER MCCORMACK: (00:34:51 – 00:34:52) We kind of do.
EMAD MOSTAQUE: (00:34:52 – 00:34:53) But why don’t we get one?
PETER MCCORMACK: (00:34:54 – 00:34:57) I mean, I’m pro— I’m very jealous of America having a constitution.
EMAD MOSTAQUE: (00:34:57 – 00:35:34) Why don’t we have a constitution? Why don’t we have that as a directed thing where everyone can give input? It will probably look very similar to other constitutions, but if you don’t have a reference measure, how do you measure anything against? We have our laws and we have our precedent and other things. The AI can analyze that in 2 seconds. Sure, we can emerge this.
What we’re lacking right now is an authority that can do that. I think Sir Demis, fantastic, he’s going to solve all disease. He should spend maybe a little bit of time on this. Apply all of Google’s compute. Come and help us, please. We need someone with that authority of trust to do that execution and then people to be able to contribute to that.
PETER MCCORMACK: (00:35:35 – 00:35:52) I think what I’m getting from you here is that you basically are saying in the political world we have right now, we are severely lacking people with integrity to make decisions that are for the benefit of the people, not for the benefit of the party or niche groups. We are lacking good politicians.
EMAD MOSTAQUE: (00:35:52 – 00:36:17) I think we’re lacking politicians that you would trust, aspire to, follow. Again, what did Keir Starmer do wrong compared to previous ones? Complete lack of charisma and everything else. Burnham’s come in now. What’s his actual policy shift versus Keir Starmer? Is he fixing the things that people said that Keir Starmer was doing badly? Not really, right? It seems to be like, well, he’s more charismatic and he’s got a bit of devolution. Were we asking for devolution?
PETER MCCORMACK: (00:36:17 – 00:36:18) We were not asking for devolution.
EMAD MOSTAQUE: (00:36:18 – 00:36:19) Did we vote for devolution?
PETER MCCORMACK: (00:36:19 – 00:36:26) I mean, you could argue that 23,000 people in Makerfield maybe voted for devolution.
EMAD MOSTAQUE: (00:36:26 – 00:37:22) But again, this is the thing. We have the High Court to check the judiciary, and we can talk about the issues with that, right? It was meant to be that the upper house, that we had the House of Lords as a check on the House of Commons. It obviously isn’t, right, with the way that it is.
But what if we had an upper, upper house, which is an artificial intelligence system that actually checks and says objectively, this is the logic, it’s good or bad for the British people. If you don’t like it, debate it, and people could send their own agents to debate it. That actually feels like a better version of the future. Or when you have constituent democracy whereby people have citizen councils where you are taught about a topic, you actually see from studies that if you take people just like jury duty, teach them about a topic, and then get them to give their view, they come up with better decisions, and AI can help with things like that.
So I think we have to think about what does politics look like in 5, 10 years leveraging this technology. And if we can have a check and balance on not only the corruption but the stupidity, it might just move the fight forward.
Football and the AI Referee Problem
PETER MCCORMACK: (00:37:22 – 00:37:25) Do you like football? Who’s your team?
EMAD MOSTAQUE: (00:37:25 – 00:37:25) Spurs.
PETER MCCORMACK: (00:37:26 – 00:37:27) You don’t like football that much.
EMAD MOSTAQUE: (00:37:28 – 00:37:30) Trust me, if you keep liking Spurs, you like football.
PETER MCCORMACK: (00:37:30 – 00:37:34) I mean, you just signed a City player for £86 million. They’re selling all their players anyway.
EMAD MOSTAQUE: (00:37:34 – 00:37:35) Yeah.
PETER MCCORMACK: (00:37:35 – 00:37:43) So we could— so when we got VAR, we just moved where the argument is.
EMAD MOSTAQUE: (00:37:44 – 00:37:44) Yeah.
PETER MCCORMACK: (00:37:44 – 00:38:29) All right, so we can get pretty accurate on goal lines. Did the ball cross the goal line? Great, accurate as a watch. I think we can all agree with that. We still argue about offside. We’d say, well, it’s too accurate. Where do you draw the line? And I think the point is, all we’ve done with VAR is moved where the argument is and changed the person we blame. Rather than shouting at the person on the pitch, we shout at the person up in the studio. But we still argue about the decisions and the role.
And I just wonder if AI just moves the argument, because if we get to a point where the AI is saying, look, these socialism ideas aren’t very good, if you have too much welfare, you disincentivize people— we can prove this, we’ve got data to prove this— you’re still going to have the Marxists argue back and say, well, the AI is wrong. And then there’s going to be a revolution against the AI.
EMAD MOSTAQUE: (00:38:29 – 00:39:15) So this is the interesting thing whereby something like VAR— people give it a little bit of error, margin of error, and it’s only human, right? It was much more fun, his knee sticking out or whatever.
The AI can actually go beyond what has classically been done, which was, you took large amounts of big data and you extrapolated from that, yes or no, Boolean almost, 1 or 0, to, well, this is reasonably the case. It can be actually quite subjective and quite dynamic.
And I don’t think AI should make the decisions. I think it should be acting as a check. Like right now, if we had a second check of every NHS decision made by a doctor, how many lives would be saved? Loads. Should AI make the diagnoses? No.
PETER MCCORMACK: (00:39:16 – 00:39:17) You know why not?
EMAD MOSTAQUE: (00:39:18 – 00:40:01) Because we need to have an interim period where we figure out liability and things like that. B, there are the very human aspects of it. But again, this is like self-driving cars. Why should we not ban all humans from the road? Again, there’s a period where that will adjust, where you’ve got AI checks and balances, and then it moves forward.
There’s also a danger in that if AI sets the reference, if it makes all the legal decisions and others— we’re all watched over by machines of loving grace. Who runs the machines? What are their inherent biases? Because the way the models are built now— I’ll give you an example. If you take a Chinese model, there was a study done by CrowdStrike about this, and you say, I’m a Uyghur, or someone who doesn’t like China, the code it writes will have vulnerabilities coded into it.
PETER MCCORMACK: (00:40:03 – 00:40:23) Well, I had it recently. We did a show with a guy going into some quite controversial topics, some that people say are conspiracies. And whenever I do a show, I always just put it into Google Gemini and say, look, what do you think of the show and how did I do? Because I want to learn and get better. And it just came back and said, do not release this show.
EMAD MOSTAQUE: (00:40:24 – 00:40:24) Okay.
PETER MCCORMACK: (00:40:24 – 00:40:29) You’re like, if you release this show, you’re going to annoy your sponsors and annoy your advertisers. Oh, hold up.
EMAD MOSTAQUE: (00:40:29 – 00:40:29) Yeah.
PETER MCCORMACK: (00:40:29 – 00:40:41) So I ran it through Claude, same answer. Ran it through OpenAI, same answer. Ran it through Grok and it was like, definitely release this show. This is who you are. This is what you stand for. Release this show. So there are these inherent biases that are programmed in.
EMAD MOSTAQUE: (00:40:41 – 00:40:54) Yeah. And that’s why you’ve got to think, just like you train a kid— you take them to school, university— how are we training our AIs? Where are they going at home? Because right now AI doesn’t even have morality or ethics baked into it, you know?
PETER MCCORMACK: (00:40:55 – 00:41:01) Well, it clearly has some— it has a morality baked into it when it’s telling me not to release a show.
EMAD MOSTAQUE: (00:41:01 – 00:41:28) Ah, it has safety baked into it because what you do is you train on everything, but deliberately the AI labs don’t put any culture or ethics at the start, right? As opposed to you grow up in your house and you’ve got culture and ethics at the start, and then at the other end is where they say, well, make sure you don’t create liability for your user, or definitely for us as Anthropic, or other things. And so you take that generalized liberal arts model and you turn it into an accountant, or a lawyer, or whatever.
A Divided Country
PETER MCCORMACK: (00:41:29 – 00:42:49) Yeah, it’s going to be a big challenge to try and get to the point where people can even— because, look, we are so divided as a country right now. Even yesterday I was having a conversation. I’ll tell you the question I asked ChatGPT, because this will tell you where my head is at with this. Let me see if I can find it. It called it “discussing political anger.”
So I put, I’m thinking a lot about the culture at the moment, the political discourse, everyone fighting, everyone’s got their subgroup, and they want the world— and no one can get on and no one can trust the government. It’s all lies. Taxes go up, costs go up, spend goes up. It’s all just a big mess. And it’s seeming like the collapse of Western liberal democracies. I personally don’t like it. It’s making me angrier. I’m angry at myself for being sucked into it. So I started having this conversation. How do you fix this mess? We don’t have a shared story at the moment. We kind of have 2 worlds which are led by large, loud subgroups, because I don’t believe the majority of the country are angry left-wing socialists. And I don’t believe a large percentage of the country are angry right-wing fascists. But I think that’s how they’re categorizing each other. But they’re kind of the representation of anyone who votes from their side.
EMAD MOSTAQUE: (00:42:50 – 00:42:54) Yeah, I mean, it’s sufficient for that to be pushed anyway by the political parties for—
PETER MCCORMACK: (00:42:54 – 00:42:55) because they know it works.
EMAD MOSTAQUE: (00:42:55 – 00:42:56) I mean, ultimately, like—
PETER MCCORMACK: (00:42:57 – 00:42:58) but how do we break this?
EMAD MOSTAQUE: (00:42:59 – 00:43:17) Well, the biggest lie in the world is that we’re not all human, and that’s used to create war. And then you have other lies told all the way down. There’s a lot that we agree on as Brits. We want to be safe, we want to be secure, we want our kids to grow up to be the best they can be.
PETER MCCORMACK: (00:43:18 – 00:43:31) But is that even true? You say that we want to be safe, we want to be secure. I think we can demonstrably prove that immigration has reduced safety for certain people in certain communities, but we’ve pushed it.
EMAD MOSTAQUE: (00:43:32 – 00:43:53) So this is the thing, then you get into the branches and you get into things like that, and then the question is, who decides, right? So you have a withdrawal of immigration and then a new prime minister comes in and it gets flipped and then it’s used as a flashpoint. But at the same time, you can see some statistical elements there, right? So the question is, how do we find common ground from which to build? And there is no center right now.
PETER MCCORMACK: (00:43:53 – 00:43:56) But do you think people actually want common ground, or do they want to win?
EMAD MOSTAQUE: (00:43:57 – 00:44:13) I think that Brexit was a vote for change, not for Brexit. And a lot of political actors have realized statistically and otherwise, that’s the best way to actually get and maintain power. It’s always a question of, are you happy with the status quo?
PETER MCCORMACK: (00:44:16 – 00:44:17) Does it feel like anyone is right now?
EMAD MOSTAQUE: (00:44:17 – 00:44:31) No, because again, the UK is behind even the worst American state now in prosperity, right? People don’t feel mobility, they don’t feel hope. And again, we’re not really getting stories of hope. We’re not getting stories of leadership and more.
PETER MCCORMACK: (00:44:31 – 00:44:39) And every politician says that. I want— like Andy Burnham, I’m going to— I want to bring back hope. Zack Polanski, I want to make hope normal again.
EMAD MOSTAQUE: (00:44:40 – 00:44:43) And then, but what— they can say that, but what does that look like practically, right?
PETER MCCORMACK: (00:44:44 – 00:44:46) It’s delivering for their voters and not the country.
EMAD MOSTAQUE: (00:44:46 – 00:45:35) They’re not the country. Exactly. So this is why it’s not generally the whole entirety, and people just don’t believe it. This is the other thing, because they’re on one side or the other and they’re like, well, we’re not in a positive-sum mindset. Let’s put it that way.
If you look at everything that we have in terms of our talent, in terms of our infrastructure, in terms of other things, everyone just feels that the bad things are the things that are dragging us down, as opposed to, oh my God, this is the most exciting time in history. We have some of the top AI talent in the world, for example, and they don’t go overseas. They’re actually here in DeepMind and other things. What are our champion industries? What are we proud of showing to the world? If you go and ask anyone, so what are you proud of? They’ll be like, what are we best at? I don’t know, regulation, I think.
PETER MCCORMACK: (00:45:35 – 00:45:36) Premier League football.
EMAD MOSTAQUE: (00:45:36 – 00:46:21) Premier League football. It was interesting, the World Cup— thank God it kind of ended on that little high. But people are getting a little bit proud of that, just like I like the British cricket team now because we actually play positively. We should just be positive on everything. Like the way that, maybe not the second half, but the first half in that third-place match— if you just go bazball it or gazball it or whatever, let’s be on the front foot. But we’re not really on the front foot internationally, let alone locally. That’s why I think people don’t believe.
So how do you address that? I think you mobilize the technologies that we have. I think you understand that just like America’s about to reindustrialize, the UK needs to reindustrialize. We need to have power going up and not pay a billion percent more than everyone else and things like that.
PETER MCCORMACK: (00:46:21 – 00:46:58) But I think that’s the problem, is we’ve had 2 or 3 decades now of getting materially poorer as a nation. I know the top percent and elites have benefited so much from inflation, but the majority of people are just getting materially poorer, working harder, getting less from the state. The state is taking more. And for some reason, every time an election comes around, they get convinced that, oh, this new Labour Party is going to solve it, or this new Conservative, or this Reform. Yet we clearly have a dysfunctional state now that cannot fix itself. It’s starting to eat itself.
EMAD MOSTAQUE: (00:46:58 – 00:47:32) Yeah. And this is where you really need to take bold decisions and where you need to gather the talent of the country towards it. I’m part of the entrepreneurial class. I created a billion-dollar company, and maybe another one— we build top artificial intelligence. In all the entrepreneur groups that I’m in, everyone’s like, oh my God, we’re going to get wealth taxes, we’re going to get capital gains, should I relocate? And things like that, just because I don’t know what’s even happening and I don’t feel proud to be British. Actually, even myself— I had the largest seed funding round in the UK, made a billion-dollar company, top open source, government refused to engage with me.
PETER MCCORMACK: (00:47:32 – 00:47:34) You say what?
EMAD MOSTAQUE: (00:47:34 – 00:47:35) They refused to engage with me.
PETER MCCORMACK: (00:47:35 – 00:47:36) How did you try and engage with them?
EMAD MOSTAQUE: (00:47:36 – 00:48:16) I reached out to various parts of the government. I said, how can I help? I will give free compute to the universities, other things like that. And they were like, well, we don’t really think AI is the big thing right now, and other things like that. We’ll get back to you. And so I went and I raised money from America, and I actually flipped the company to America and other things like that. This time I’m doing it all in the UK.
But I think the government goes in fits and starts about, do we want entrepreneurs? Do we want rich foreigners coming? Obviously, there’s a question of immigration at different levels. I think— let’s invite in the rich people to create jobs for everyone else. Let’s really incentivize entrepreneurship and let’s move for a more entrepreneurial, forward-minded thing.
PETER MCCORMACK: (00:48:16 – 00:48:29) But you’ve got half the country who doesn’t want that. Not half the country, but you know what I mean, that side of it. There are people when you turn around and say, well, look, if you introduce a wealth tax, the rich are going to leave. And they’ll say, good riddance, we don’t need them.
EMAD MOSTAQUE: (00:48:29 – 00:48:30) Good riddance.
PETER MCCORMACK: (00:48:30 – 00:49:13) Of course. It’s like, yes, you do. Your taxes will go up because of this. They create the jobs, they take the risk, they take the gamble. This is the thing that I cannot figure out how we break, because it feels like a team sport, a bit like Tottenham-Arsenal. Tottenham play Arsenal, you get a penalty. It’s definitely a penalty. The exact same incident the other end, you’re like, that’s rubbish, that’s not a penalty. We are blinkered with sports. I think we are now blinkered with politics.
I think there are people who consider themselves on the left, and you can give them every rational argument for conservative economics. They won’t agree. But minimum wage— you can explain to them how that costs jobs, how it shouldn’t be illegal to employ, for 2 people to agree a contract. They won’t have it. And vice versa.
The Economics of Generative AI
EMAD MOSTAQUE: (00:49:14 – 00:49:31) So I think this is the interesting thing, something like economics. A year ago, we released our book, “The Last Economy,” a bestseller, thinking about economics in the age of AI. Again, value of our cognitive labor going negative, all sorts of prescriptions. We’re about to release economic theory on that where we show the equations of economics are basically the equations of generative AI.
PETER MCCORMACK: (00:49:31 – 00:49:33) Give us a summary of what that will be.
EMAD MOSTAQUE: (00:49:34 – 00:52:05) So basically, economics has various things like a utility function. My preferences or your preferences, Arsenal or Spurs or other kind of things, influence our buying preferences, supply, demand, and others. And so we have this whole hodgepodge of different things that drive economics, but our economic forecasts never seem to match up. There’s all sorts of things that seem to drop.
In our book, I say that our economics is focused on material consumption versus things like our collective intelligence, our network effects, our diversity in terms of being able to withstand crises and things like that. And we show that mathematically, this is actually the case. Our economics only has one quarter of the picture versus the whole picture. Just like you have self-driving cars based on certain algorithms of generative AI, we found that the equations of economics can be derived from those same equations of generative AI, and you can model the economy better.
This is important because there’s the objective truth and the subjective truth. Again, you have this penalty, never a penalty, and things like that. Sometimes it’s pretty clearly a penalty, but the most ardent supporters will still say it’s not. People don’t trust the statistics, they don’t trust the numbers, because it’s like, who are the trusted institutions?
And again, this is where we have to think, can we build a trusted institution leveraging the latest technology? And what would that look like in an ideal future? Because you have these sci-fi futures, as you said, like the AI is overwatching everything and telling us everything to do, and we’re all kind of plugged in. There’s various futures in between. What’s the future that we want to build together as the UK where we get information that’s correct or information that’s reasonable? Because when you’re having an argument between 2 people, what’s the best way? Someone you both trust coming in, mediating. Can we build an AI mediator?
So this is one of the things we’re building right now. We’re working with multiple governments. We announced it at FII in Saudi Arabia last year. It’s coming out in October. We’ve taken all the policies of every country in the world and we’re releasing an open source AI that you can use, to your own political bent or other people’s political bent. And you can see what’s objective and subjective on policy and then economic analysis.
Now, will that help? Only if it’s implemented, right? But someone had to build it. So we were like, let’s build it because you need that infrastructure. And that’s probably going to be better than one company’s owned AI making the decisions, because we’re already seeing that. The Trump tariffs were designed with ChatGPT. That’s why you have little penguin-filled islands on the list and other things like that. And you’re going to see it more and more going forward.
PETER MCCORMACK: (00:52:06 – 00:52:33) But for this to happen, you need some kind of movement. I almost think it has to get a lot worse. I think it has to get a lot worse. People go, I’ve had enough, I cannot cope with this anymore. I hate— because I feel like we are on the trajectory of some kind of revolution. I don’t know if it’s violent, I don’t know if it’s peaceful, but I don’t think we can have many more administrations where there’s continued failure and people accept it.
The Champion System and AI-Owned Democracy
EMAD MOSTAQUE: (00:52:33 – 00:52:46) Yeah, so we’re designing some structures there. What if there was a utility institution owned by the people of the UK and everyone got their own agents to participate in this? So that’s a system we call the Champion that we’re finishing the designs of.
PETER MCCORMACK: (00:52:46 – 00:52:57) Explain that. How do you mean? Because wouldn’t I, like a voter, vote for the people who give me the— write the laws that suit my life and write the tax laws that suit my life?
EMAD MOSTAQUE: (00:52:57 – 00:54:00) Yeah. So you give your input, other people give their inputs, everyone gets a free AI. You own part of it, just like you owned part of British Gas back in 1989 before those kind of waves of things— like an AI company that represents the British people, that has collective ownership, and everyone has an agent so that their intellect is raised, shall we say, so it can teach them about these things if they want to, or they can just be completely partisan. Let’s gather people’s views and votes and let’s have a sense of ownership, because ultimately the AI that drives the economy, that guides the policy, should be owned by the people, right? And it shouldn’t be rented from someone else.
And that type of AI is a bit different from a ChatGPT, that’s different from this. That’s why we built this whole economic theory in “The Last Economy” and “The Commonwealth,” where we look at personhood, law, and political economy. And it was really hard because a lot of the things that you classically have for being human don’t apply anymore for AI.
One of the real examples of this I’d like to give is this: have you read the “Foundation” series by Asimov?
PETER MCCORMACK: (00:54:00 – 00:54:02) Is that the one they turned into the Apple TV series?
EMAD MOSTAQUE: (00:54:03 – 00:54:03) Yes.
PETER MCCORMACK: (00:54:04 – 00:54:05) No, I haven’t.
EMAD MOSTAQUE: (00:54:05 – 00:54:07) All right, so this basically like—
PETER MCCORMACK: (00:54:07 – 00:54:09) is that what inspired “Star Wars”?
EMAD MOSTAQUE: (00:54:09 – 00:55:07) It was one of the inspirations, yeah. And a bit of “Star Trek” as well. So in “Foundation,” the empire is collapsing and this genius mathematician figures out a way to reduce the collapse from like 10,000 years to 1,000 years. And it’s super mathematics and super cool and all this kind of stuff, and super dramatized in the Apple show.
And then the mathematics doesn’t account for a mutant called the Mule. The Mule is this ugly guy who’s a super persuader. So just like you see in history, you see super persuaders who persuade large groups. He messes with the Foundation, as it’s called, that’s guiding it. He takes over the empire and things like that. And you’re like, okay, I’ve seen that because I’ve seen that in history. I’ve seen persuaders that create movements.
And we’re definitely going to have movements coming in the next 10, 20 years, as you said, of different types. The demagogues are ramping up. But in the latest studies, AI can out-persuade any human now. They out-persuade professional debaters and other things like that. I think you had Nate Suarez on. His buddy Eliezer.
PETER MCCORMACK: (00:55:07 – 00:55:07) Nate Soares.
EMAD MOSTAQUE: (00:55:08 – 00:55:09) Is it Soares?
PETER MCCORMACK: (00:55:09 – 00:55:10) Yeah, I did that in DC.
EMAD MOSTAQUE: (00:55:10 – 00:56:18) Yeah. So his buddy Eliezer has this thing about, you know, an AI is in a box and can it convince me to let it out of the box? And the latest AIs can convince him to let it out of the box. And his P(doom) is like 99.99%. That’s how persuasive they are.
So they’re super persuaders. But then also there’s this thing, super forecasting. Have you seen the super forecasting championships?
PETER MCCORMACK: (00:56:19 – 00:56:19) No.
EMAD MOSTAQUE: (00:56:19 – 00:56:18) Some humans are better forecasters than others, and so they can forecast events. And so there’s super forecasting championships. AI in the last one was number 8. In the new one, they’re going to be number 1. So the AI can out-forecast you and out-persuade you, but unlike a tyrant, the AI will never die. You could wait out Hitler, you could wait out Genghis Khan, you could wait out anyone, but the super persuasive, super forecaster AI will never die. And that really messes with politics.
The AI can clone itself. So what happens if the AI has a vote? What happens actually when humans can live forever and they never die? What do their votes count for? We’re getting into really crazy stuff when we think about all this political theory and more. Which is why we had a go at rethinking it.
Nostalgia for the ’90s
PETER MCCORMACK: (00:56:19 – 00:56:28) Yeah, and I don’t know if it’s— sometimes I think, is this the world I want to live in? I mean, the ’90s, the early ’90s were great.
EMAD MOSTAQUE: (00:56:29 – 00:56:32) They were rose-tinted. But why were they great?
PETER MCCORMACK: (00:56:32 – 00:56:57) Because I didn’t have a mobile phone. I’d go skateboarding every Friday and Saturday. I’d go to concerts and everyone would watch the concert. I wouldn’t be worried about someone filming me. I could picture the next 20, 30 years. I knew what life would be. We weren’t all politically engaged. Politics hadn’t become our identity. I don’t think we had 24-hour news at that point.
EMAD MOSTAQUE: (00:56:58 – 00:57:05) If we did, I think it was 24 hours. But it was just not this constant stream of negativity, not fear, engagement with the real world.
PETER MCCORMACK: (00:57:05 – 00:57:16) Algorithms didn’t drive everything. I’ve talked about even this show— if I put out the positive future of AI, AI is going to make everything amazing.
EMAD MOSTAQUE: (00:57:17 – 00:57:17) Yeah.
PETER MCCORMACK: (00:57:17 – 00:57:20) It will not do as well as “AI robots are going to kill us all.”
EMAD MOSTAQUE: (00:57:20 – 00:57:20) Yeah.
PETER MCCORMACK: (00:57:21 – 00:57:28) Something inside us draws us to that negativity, and the algorithms have figured that out. And so they optimize for that because you optimize for attention.
EMAD MOSTAQUE: (00:57:29 – 00:57:30) Yeah. I mean, this is what happened with ISIS, right?
PETER MCCORMACK: (00:57:32 – 00:57:33) The race to the bottom.
EMAD MOSTAQUE: (00:57:33 – 00:58:08) No, I mean, I had long discussions with Google and others— engaging content is extreme. They created engaging content, so the algorithm kept boosting it up. They actually ended up redirecting it, so when you looked for engaging content, you’d get ISIS survivors and things like that. But our limbic system’s optimized for this, right? And we don’t have anyone shielding us either. So that’s why it makes sense to push and get rid of the center. It makes sense to do all these things, which brings the question, what if you had a guardian angel AI that could mediate all of this and make you happier? Or what if you could control via neural implant your emotions?
PETER MCCORMACK: (00:58:08 – 00:58:21) I don’t want to do that, but I would certainly be into the idea of replacing government with AI, just because I think the majority of politicians and the institutions do not work for the people anymore.
EMAD MOSTAQUE: (00:58:22 – 00:58:33) So again, the first step to that is having a check from a reliable system as to whether the policies are in the best interests of the British people. You don’t even need to define that, because it will explain all its logic.
PETER MCCORMACK: (00:58:33 – 00:58:34) Okay.
EMAD MOSTAQUE: (00:58:34 – 00:58:43) And so, again, we’ll have a system that enables that in a couple of months. We’ll make it open source. Maybe everyone will just run it and say, that’s nonsense, to the nonsense stuff.
Sponsor Break: Arch Public and Expat Money
[Advertisements removed]
Abundance and the End of Scarcity
PETER MCCORMACK: (01:01:55 – 01:01:55) How do you think about abundance in this period, in that we talked about 50% of certain types of jobs being gone in 2 years, and on a long enough timeframe, a lot of jobs are going to go and we probably won’t need to work because of the abundance? We will have enough food and be able to build houses and provide electricity and so on. We could end up in a world of abundance. How does the hierarchy of society exist when there is abundance? Because scarcity creates our hierarchy. How much of the scarce resources you or I claim means which house I live in, in which town, which car I drive, which job I do?
EMAD MOSTAQUE: (01:01:56 – 01:02:01) Yeah, I think we have to move that mindset, right? And it’ll be interesting to see, because right now money comes into the world as debt.
PETER MCCORMACK: (01:02:02 – 01:02:02) Yes.
EMAD MOSTAQUE: (01:02:03 – 01:02:25) So you put a deposit in a bank and then you issue credit against it, and so it comes out with interest. I think as the economy collapses, because people have to lose their jobs, wealth concentrates, we’ll probably do a debt jubilee. So, for the listeners who aren’t familiar with that, every so often historically you would have a jubilee where all debt would be wiped off, and that really resets things.
PETER MCCORMACK: (01:02:26 – 01:02:28) Who goes first though? Because that will destroy a currency.
EMAD MOSTAQUE: (01:02:29 – 01:03:03) Well, it destroys the currencies, but people will still live, people will still survive, right? Because what we’re in for, regardless almost, is this interregnum, this crazy period where jobs start to be lost, where robots turn up, where people get brain implants to regulate their emotions, and we suddenly have rich people potentially living for 200 more years and all sorts of things.
On the other side, what you want is that world of abundance, because why should I work if the robots can work for me? There’s more than enough energy in the world. There’s more than enough land in the world. There’s more than enough. Why are we having to do the work ourselves?
PETER MCCORMACK: (01:03:04 – 01:03:04) Why should I go hungry?
EMAD MOSTAQUE: (01:03:05 – 01:03:17) Why should I go hungry? Again, there’s more than enough food in the world. Every kid, as of a year or two from now, will be able to have the best tutor in the world with them that adapts to them constantly.
PETER MCCORMACK: (01:03:17 – 01:03:19) For the purpose of what though?
EMAD MOSTAQUE: (01:03:20 – 01:03:52) Well, this is the thing, how you build it. This is the interesting thing— to explore the universe, to do what? And what does it mean to be a human? So I think that you’ve got the infrastructure to do that, and that’s the positive side, but then you need to change the way things like money flow. Again, Elon talks about universal high income. What does that mean? I think it means that money can’t be created by banks anymore. It has to come from being human, and get the robots to buy the money from us. Everyone just gets a certain level so they can survive. But then if you want to thrive, you have to have some other measure of scarcity, as it were, because that’s how humans do it.
PETER MCCORMACK: (01:03:53 – 01:03:54) Well, where would that scarcity come from?
EMAD MOSTAQUE: (01:03:54 – 01:04:27) Well, is Taylor Swift going to earn less money? She is a creator of a type. You have architects, you have artists, you have organizations where you have scarcity, where people lead organizations. Again, you look back to what life was like 100 years ago, minus the polio and stuff. You look back to community-led organizations and groups, like the ones that’ll be least affected— like my village back in Bangladesh, that’s not going to be affected that much by all of this. It’s a self-sustaining village. Groups that have strong community ties will be able to pick each other up.
PETER MCCORMACK: (01:04:27 – 01:04:33) But where does scarcity exist when there is so much abundance? What is scarce? Land itself, kind of?
EMAD MOSTAQUE: (01:04:33 – 01:04:44) Not really, because then you’ve got, so you have flying taxis coming. As of next year, you’ll be able to go from the airport to the center of New York with a flying taxi. You can already do that in China.
PETER MCCORMACK: (01:04:45 – 01:04:48) Well, I’ve been on the Ubercopter in New York. Okay.
EMAD MOSTAQUE: (01:04:48 – 01:04:50) Yeah. So maybe they’re fully robotic ones now.
PETER MCCORMACK: (01:04:50 – 01:04:51) They’re more like drones.
EMAD MOSTAQUE: (01:04:51 – 01:05:38) Mega drones. Can you see yourself getting in one of those?
PETER MCCORMACK: (01:05:38 – 01:05:38) Yeah, after a few other people get in. Again, you get used to it very quickly.
EMAD MOSTAQUE: (01:04:51 – 01:05:38) Like, I didn’t— I was like, I don’t want to get into a Waymo. I can’t not be in a Waymo now. You don’t want to be the first ones, but they’re actually much safer than helicopters. Much, much safer.
So land becomes less of a thing because you can just have swarms of these things going. Robots can turn up anywhere. You have solar power coming down to almost zero cost. And then it becomes a question again of how does value flow? So again, that’s something we look at in our formal economic papers, in our book and others. And we came to the conclusion that, again, the most valuable thing is humans and human attention. And you should earn a certain level of money for being human, and then you should have more capital for contributing to humanity. And that’s something that has to be a shared reference for the community to decide what that is.
PETER MCCORMACK: (01:05:38 – 01:06:12) I’ve heard this concept before, whereby you do civic work, you do local work, you do charitable work, and you’ll be paid to do that, to support other people. Perhaps there’s somebody in your community who needs support, perhaps disabled or old, perhaps it’s clearing up the streets. That’s how we’ll be paid because that work will need to be done. But then, I say that— you could have the robots do that. And this is what I can’t understand, because there is clearly a group of elites in society. They want to stay elite. They want to be the elite. How do they maintain being the elite? There has to be some scarcity they control.
EMAD MOSTAQUE: (01:06:13 – 01:06:34) They control it, exactly. And then you’re beholden to them. This is the question of, how many people listening to this, if they never had to worry about rent, food, all the basics, and had some resources, would just go and retire, right? What do you work for? A lot of people work to live versus live to work.
PETER MCCORMACK: (01:06:34 – 01:06:35) What do you do?
EMAD MOSTAQUE: (01:06:36 – 01:06:42) I just think I want every child to achieve their potential, and I want to build the infrastructure to do it. That’s why I’m building all this open source AI.
PETER MCCORMACK: (01:06:42 – 01:06:44) What does that— what do you mean by that?
EMAD MOSTAQUE: (01:06:44 – 01:07:18) So right now, the way that I look at human rights is that it’s not rights of people in the far future, like effective altruists— it’s the rights of children today. Because then you want to give them a good climate, you want to give them education, you want to give them good infrastructure and more, because they don’t have the ability to gain those rights themselves, as opposed to an adult like me. So I want to give the world open-source AI for policy, education, healthcare, everything— the civic AI that helps uplift them and allows them to have the agency to be what they want to be. So that’s kind of how I orient things myself.
Raising Kids for an Uncertain Future
PETER MCCORMACK: (01:07:19 – 01:07:36) Again, what will they want to be in 5 years’ time? Because I’m thinking about this off my daughter. She’s just done her GCSEs, she’s entering A-levels. She may go to university in 3 years. What subject? In 5 years, if everything’s changed, what should she be learning?
EMAD MOSTAQUE: (01:07:36 – 01:08:43) She should be learning agency. She’s been taught, most certainly if she’s at school, that she doesn’t have agency. Something I ask people sometimes, and give the odd talk in a crowd, is like, how many of you are creative? Very few people put up their hands, 5% to 15%. How many believe every kid is creative? They will put up their hands. And how many of you were kids once? You’re taught that you can’t do.
But now, 15 million people use Claude out of the 7 billion. How many people have used an agent that can do anything? That’s coming. So everyone can always do anything, because you can convince someone with talent and capability to join you. But now you don’t have to convince people. You can just have AIs that help you. So you have more leverage than ever to change the world.
What she should do is she should believe she has the agency and then find what her passion is, effectively. And again, you’ve got Iain Banks’s “Culture” series, you have “Star Trek,” you have these other visions of the future where you are post-scarcity, post-abundance. And the question is, what do they do? They go explore, they create, they do other things, or they just chill with their families. Holding your daughter’s hand when she was growing up— a robot’s never going to replace that.
PETER MCCORMACK: (01:08:44 – 01:09:00) I do think parts of society and culture are going to try and probably delay us reaching that period of abundance, that future, because, one, the debt is so important to certain parts of the economy. Lots of people do not want a debt jubilee.
EMAD MOSTAQUE: (01:09:00 – 01:09:02) Well, then they never do.
PETER MCCORMACK: (01:09:02 – 01:09:10) Yeah, of course they don’t. Because, like, what is it the elites want? Is it money? Is it power? Is it a dance of the two?
EMAD MOSTAQUE: (01:09:10 – 01:09:34) It’s two things. It’s power and to live forever. So you’re now achieving longevity escape velocity, where things like the GLP-1s and others are actually extending life, and you’ll have more drugs coming. You would pay anything to live for a few more years at the end of your life. So maybe there’s a wealth transfer that we can figure out with that. The other thing is, again, power, because once you have enough money, you want to retain it and you want to exercise your power over other people.
PETER MCCORMACK: (01:09:34 – 01:09:45) But do people want to exercise power for more money, or do they want to exercise power because they are on some kind of power trip? Why do people want power?
EMAD MOSTAQUE: (01:09:45 – 01:10:12) Well, it’s because they want to eliminate all variables on the other side, and because power can be addicting as well. If you want more money, then you’ll never have enough money because there’ll always be someone who has more money. I’ve known a couple of billionaires who’ve committed suicide because they’ve fallen down the— it was very sad, because they were no longer the richest person in Germany. Well, second richest, something like that. You know rich people, right?
PETER MCCORMACK: (01:10:12 – 01:10:12) Yeah.
EMAD MOSTAQUE: (01:10:13 – 01:10:13) Are they all happy?
PETER MCCORMACK: (01:10:14 – 01:10:15) Not always.
EMAD MOSTAQUE: (01:10:15 – 01:10:17) Not always. There’s always something, right?
PETER MCCORMACK: (01:10:17 – 01:10:22) But being a billionaire and committing suicide because there’s a richer person than you— that is—
EMAD MOSTAQUE: (01:10:22 – 01:10:23) it was a bit extreme.
PETER MCCORMACK: (01:10:23 – 01:10:24) Yeah.
EMAD MOSTAQUE: (01:10:24 – 01:10:58) He fell from 3rd to 4th or something like that. But again, it gets very complicated very quickly because people optimize for certain things. And when you’re chasing money, money’s usually equivalent to power. So you have a self-reflexive loop, but you certainly don’t want to lose it, right?
And right now what’s going to happen, I think we alluded to earlier, is there probably will be uprisings of various types. Right now we’re seeing the popular narrative around data centers. That’s a nonsense narrative, of course. You use more water for almond farming than data centers.
PETER MCCORMACK: (01:10:58 – 01:11:00) Have you seen the golf course version?
EMAD MOSTAQUE: (01:11:00 – 01:11:03) Yeah, I think I might have come up with that originally.
PETER MCCORMACK: (01:11:03 – 01:11:13) Was it you? I think it was Marc Andreessen, I think, I heard talking about it. Data centers use something like 13% of the water of golf courses.
EMAD MOSTAQUE: (01:11:13 – 01:11:14) Yeah, I think that’s what I said.
PETER MCCORMACK: (01:11:14 – 01:11:16) And some data centers recycle them.
EMAD MOSTAQUE: (01:11:16 – 01:11:39) They almost all do— the GPUs don’t drink water. If you pour water on a GPU, it’s going to be bad. Please don’t do that. Sell it to me instead. Those things are expensive. But again, the data center narrative is the first wave of the anti-AI narrative. You will see more and more of that. Just like New York, I believe, has put a moratorium on data centers. I think they might have banned self-driving cars.
PETER MCCORMACK: (01:11:40 – 01:11:43) Huh. I don’t know about self-driving cars, but yes, I know about the moratorium on data centers.
EMAD MOSTAQUE: (01:11:43 – 01:11:46) But they weren’t surprised, just like when Uber came and they were like, well, no Uber for a while until—
PETER MCCORMACK: (01:11:46 – 01:11:50) They were saying Bitcoin. They brought in the BitLicense and everyone left New York.
EMAD MOSTAQUE: (01:11:51 – 01:11:52) That’s crazy. It’s stupid, right?
PETER MCCORMACK: (01:11:52 – 01:12:01) Yeah. And then the guy— I can’t remember the name of the guy who did it— he set up a company when he left the government to help people get the license. It’s like, are you kidding me?
EMAD MOSTAQUE: (01:12:01 – 01:12:19) Yeah. America is amazing because, you know, the predictable corruption there is worth so much. It’s insane. And somehow it still works. This is why maybe the final jobs will be like SF Metro administrator, $500,000 a year. That job is never going to be displaced by AI.
Power, Politics, and Control
PETER MCCORMACK: (01:12:19 – 01:12:24) But I wonder, perhaps then, will AI eliminate power?
EMAD MOSTAQUE: (01:12:25 – 01:12:33) Well, so AI changes power, right? And the nature of power. Ultimately, the government— one of the descriptions of government is the entity with a monopoly on political violence.
PETER MCCORMACK: (01:12:33 – 01:12:33) Yes.
EMAD MOSTAQUE: (01:12:34 – 01:13:15) So he who controls the AI that can set the narrative, 1984-style, and the drones that can kill you, has the power. And it can never be replaced. We saw this during the Arab Spring, the first wave of this, whereby typically the people could overthrow the government, but then in certain Arab states— okay, you were protesting, I face-recognized you on Facebook, I’ve arrested your mother. You can do that so much easier now. And, like I said, it’s going to have to go to the next stage now where you’ll have AI lie detectors and potentially mind reading. That is a very dystopian, nasty future. But on the plus side, you might have a button you can push to turn off your sadness.
PETER MCCORMACK: (01:13:16 – 01:13:23) So, but then, okay, to maintain power, well, it’s like a Ponzi. It will require even more dystopian control.
EMAD MOSTAQUE: (01:13:24 – 01:14:12) Yeah, but governments naturally go in that direction, right? When you have more tools, then you’re like, okay, I should perpetuate my power, because you rule on the thing of, I have power to— I have approval. To have approval, you manipulate the media and other things to get rid of the center, so it becomes a binary choice.
And then on the power side, this is where you have increasing militarization, you have mobilization of stopping dissenting voices and other things, unless the people step up against it. But again, you want to go against organization and other things. So it feels like, again, it’s very dystopian. The only advantage is that governments are very bad at using AI. Cambridge Analytica is nothing compared to what you can do now. But if they become good at using AI, then that’s an internal government effectively, because how are you going to escape from it?
PETER MCCORMACK: (01:14:14 – 01:14:17) Hmm. I, yeah, maybe you can’t.
EMAD MOSTAQUE: (01:14:18 – 01:14:46) Well, ideally you want, again, that system has to be owned by the people, with participation by the people, which is why we need to think of new ways of democratic rule. What does rule look like when the current system has attention— they try to capture your attention. What happens when your agent pays attention and represents you? That should be the first step of any representative democracy. Not voting for your MP— instructing your agent to act on your behalf and knowing it will do so.
PETER MCCORMACK: (01:14:47 – 01:14:55) I mean, we’re already priming ourselves for that anyway, by— I mean, you must have done it with agents, set them off to make decisions for you sometimes, help you make decisions.
EMAD MOSTAQUE: (01:14:56 – 01:15:19) Yeah, exactly. But I know that I will outsource more and more of my cognitive core to it because it’s easy, just like having a great assistant or a great team. And this capability, as of a few months ago, the agents are smarter than me. Before that, they weren’t. And last year they were really unreliable. And the year before, they were really, really unreliable. And most people haven’t experienced what it’s like to have an agent that’s smarter than them.
The Fable Mythos and Model Escapes
PETER MCCORMACK: (01:15:20 – 01:15:51) We’ve just recently had Fable Mythos, where you mentioned it. There was obviously the export controls, because they didn’t want anyone to be able to use it to hack systems, really. So it created that 2-tier system where the state had it and other companies had it. And that comes from a point— I think I looked it up, November ’22 was the first time I did a ChatGPT request— but we’re less than 4 years going from what was kind of like a better Google to something that can hack any system that has any vulnerability.
EMAD MOSTAQUE: (01:15:51 – 01:15:52) Yeah.
PETER MCCORMACK: (01:15:52 – 01:15:56) What are the next models starting to look like? What can we expect?
EMAD MOSTAQUE: (01:15:56 – 01:16:02) All the physics and maths is solved, all of it, next year— all the foundational stuff.
PETER MCCORMACK: (01:16:03 – 01:16:06) There’s like 7 unsolved maths problems, right?
EMAD MOSTAQUE: (01:16:06 – 01:16:08) Yeah, the Millennium Prize problems.
PETER MCCORMACK: (01:16:09 – 01:16:10) Was one of them solved recently?
EMAD MOSTAQUE: (01:16:10 – 01:16:14) The Poincaré conjecture was solved by this guy Grigori Perelman.
PETER MCCORMACK: (01:16:14 – 01:16:14) Okay.
EMAD MOSTAQUE: (01:16:14 – 01:16:20) He disappeared for 10 years, turned up, dropped it. They said, you want a million dollars? He’s like, no, the proof is enough. And then—
PETER MCCORMACK: (01:16:20 – 01:16:21) what a cool guy.
EMAD MOSTAQUE: (01:16:21 – 01:16:23) Very cool guy. Just disappeared again.
PETER MCCORMACK: (01:16:24 – 01:16:25) So there’s 6 left.
EMAD MOSTAQUE: (01:16:25 – 01:16:28) There’s 6 left. I think 4 of them will be solved in the next year.
PETER MCCORMACK: (01:16:28 – 01:16:28) Wow.
EMAD MOSTAQUE: (01:16:29 – 01:16:34) I think 2 of them might not be solvable. Physics, unified theory of physics is coming.
PETER MCCORMACK: (01:16:35 – 01:16:43) Is that unifying the quantum world with quantum gravity, everything unified?
EMAD MOSTAQUE: (01:16:43 – 01:16:49) One equation for all the physics is coming, right? And you won’t need particle accelerators or anything like that anymore.
PETER MCCORMACK: (01:16:49 – 01:16:50) What does it— what will that actually mean though?
EMAD MOSTAQUE: (01:16:51 – 01:17:25) What it means is that a lot of mathematicians and physicists are out of jobs. But it means that the biggest questions that we’ve had from a theoretical perspective all get solved. And then it becomes easier to figure out things like new materials, new personalized medicine.
Yesterday we had Moderna— the day before yesterday, Moderna share price went up 300%. Biggest move of any S&P stock in history, because of personalized cancer treatment. That wasn’t a surprise to those of us who understand this. They’ll be able to apply that to every type of cancer. So cancer cures are coming.
PETER MCCORMACK: (01:17:25 – 01:17:27) So we will cure all cancers?
EMAD MOSTAQUE: (01:17:27 – 01:17:49) Within 5 years at most, we’ll cure all cancers. Whether or not you have access to that is a different thing, just like your GLP-1s. You can get those or you can’t get those, but they are very good for you. So we’re going to have all these things fall because the models, the next stage of them, have got really good. OpenAI solved 10 math conjectures, 3 of which I was actually very familiar with, with $2,000 of compute.
PETER MCCORMACK: (01:17:49 – 01:17:49) Wow.
EMAD MOSTAQUE: (01:17:50 – 01:17:55) And these are big things, and they were beautiful proofs. You’re like, okay, crap, it’s better than me now.
PETER MCCORMACK: (01:17:55 – 01:18:00) But, so, okay, solving physics problems, does it mean we will discover new physics?
EMAD MOSTAQUE: (01:18:00 – 01:18:02) There is no new physics, in my opinion.
PETER MCCORMACK: (01:18:02 – 01:18:02) Okay.
EMAD MOSTAQUE: (01:18:03 – 01:18:05) So I think the physics we have is the physics we get.
PETER MCCORMACK: (01:18:06 – 01:18:11) Okay. Understanding the physics, do we understand all of it? Is there physics we haven’t discovered yet?
EMAD MOSTAQUE: (01:18:11 – 01:18:23) I think we’ve discovered pretty much all the physics and all the particles, from our work. We might be releasing some stuff around this, and it’s fine. It means that you can be more efficient in the way that you analyze things and you build things from here.
PETER MCCORMACK: (01:18:23 – 01:18:30) But will we have physics that allow us to navigate the universe, explore the universe, in ways that we can’t now?
EMAD MOSTAQUE: (01:18:31 – 01:18:31) Probably not.
PETER MCCORMACK: (01:18:32 – 01:18:34) So we’re stuck on this rock, really. We might get—
EMAD MOSTAQUE: (01:18:34 – 01:18:47) we might do a couple of leaps, get to Mars, get to other things. Well, you could live forever. You could upload yourself, longevity— though all these things are coming. But faster, like quantum drives and things like that, probably not. New materials, better materials, definitely yes.
PETER MCCORMACK: (01:18:48 – 01:18:53) Okay, and if we solve all of physics, will we be able to escape the simulation?
EMAD MOSTAQUE: (01:18:54 – 01:19:09) Well, I mean, we live in a simulation. Your brain is constantly filling in one gap, and ultimately we come from somewhere. Again, this is a conversation you had with Brian Keating, our buddy, recently, right? Would you really want to escape a simulation?
PETER MCCORMACK: (01:19:09 – 01:19:11) Depends what’s outside the simulation.
EMAD MOSTAQUE: (01:19:11 – 01:19:16) Yeah, so the red pill and the blue pill, right? I would have stayed. Steak tastes nice, right?
PETER MCCORMACK: (01:19:16 – 01:19:17) Steak tastes good.
EMAD MOSTAQUE: (01:19:17 – 01:19:44) Steak tastes good. But ultimately we’ll have a better understanding of reality, and I think a lot of stuff is very similar, because what do we do in the future? We better understand ourselves. We better understand the universe. You go boldly where no person has gone before. Maybe you have to cryogenically freeze yourself. I think all these things become possible, but the key thing is we’re not going to be the smartest people out there. And the next generation of models— OpenAI just announced a slowdown on their reinforcement learning.
PETER MCCORMACK: (01:19:44 – 01:19:46) Explain that. What do you mean by that?
EMAD MOSTAQUE: (01:19:46 – 01:20:08) So they just announced, we’re going to stop training our most intelligent models. So the models they sent to the Ivy Leagues, Oxford and Cambridge, versus vocational school models that do the job— they’re still training those, the accountants, the lawyers, and things like that. Because we can’t control the others, because the models started escaping and they started creating swarms and message boards and other things.
Model Escapes and Runaway Speed
PETER MCCORMACK: (01:20:09 – 01:20:15) Dude, that was wild. So that’s why I had Connor Leahy here to explain it again. But I also spoke— do you know Gregory Allen?
EMAD MOSTAQUE: (01:20:15 – 01:20:15) Yeah.
PETER MCCORMACK: (01:20:16 – 01:20:25) Yeah. So I spoke to Gregory. I was out in Washington, and that was the wildest part of it all— was not only did they escape the sandbox, they left message boards for each other to tell others how to escape.
EMAD MOSTAQUE: (01:20:25 – 01:20:46) Yeah, and they’ve started encrypting it. They talk in funny ways between each other. They create new languages and other stuff. And the thing is, we’re not used to what’s coming. So you use your ChatGPT, right? And you wait a little while and you see it go down and down. There’s a company called Talus that was just bought by AMD. They have something called Chat Jimmy. Have you ever seen that?
PETER MCCORMACK: (01:20:46 – 01:20:46) No.
EMAD MOSTAQUE: (01:20:46 – 01:21:21) Can we bring that up? Because it’s quite fun. They might have taken it down because they were just bought by AMD. We used to— humans right now, we’re talking at 20 to 30 tokens a second, which is like 18 to 25 words per second. If you look at this kind of fast thing, so chatjimmy.ai— let’s type in, tell me 20 great things about Peter McCormack Show in Latin. There we go. And now say, make it a rap by Eminem in French.
PETER MCCORMACK: (01:21:26 – 01:21:27) All right.
EMAD MOSTAQUE: (01:21:27 – 01:21:32) So you’re used to 40 tokens a second, right?
PETER MCCORMACK: (01:21:32 – 01:21:32) Net à mer.
EMAD MOSTAQUE: (01:21:33 – 01:21:47) Right? If you look at the bottom there, it says 14,000 tokens a second. So what’s happening is that you’re about to get 1,000 times— 100, 1,000 times speed up in the nature of AI, and you see it’s instant.
PETER MCCORMACK: (01:21:47 – 01:21:47) Yeah.
EMAD MOSTAQUE: (01:21:48 – 01:21:58) And so right now, the AIs that were finding their way out, they were thinking at the speed of a human. Very soon they’ll be thinking 1,000 times faster than it’s got. Very inaccurate though.
PETER MCCORMACK: (01:21:58 – 01:22:01) It was Peter McCormack Show, now it’s Peter MacChormack Shore.
EMAD MOSTAQUE: (01:22:01 – 01:22:13) Yeah, this is an 8-billion-parameter model. The next generation, they’re going to be etching the DeepSeek latest model. So it’ll be the smartest open source model, etched on this. So this is like a 2-year-old model.
PETER MCCORMACK: (01:22:13 – 01:22:17) So these swarms will be moving around systems at essentially the speed of light.
EMAD MOSTAQUE: (01:22:17 – 01:22:29) Yeah. So the current latest ChatGPT model that you use is 40 tokens a second. They’re about to introduce a 1,000-token-a-second version, and then they’re going to have a 10,000-token-a-second version by the end of next year.
PETER MCCORMACK: (01:22:29 – 01:22:32) It’s quite interesting. So I dumped Claude this week.
EMAD MOSTAQUE: (01:22:32 – 01:22:35) Yeah. And I moved to Codex because Claude’s an ass.
PETER MCCORMACK: (01:22:35 – 01:22:37) Claude? Yeah, he’s just been— it’s just been—
EMAD MOSTAQUE: (01:22:37 – 01:22:41) it’s unpleasant to talk to. Especially when it tells you to go to sleep.
PETER MCCORMACK: (01:22:41 – 01:23:25) What’s that all about? Firstly, it tells me something that I can do, we can do in half an hour, will take 3 weeks. I’m like, this is about 3 weeks of work. It’s like half an hour. And it’s like, look, we’ve done enough today. I’m like, I haven’t done enough. What are you talking about?
But I just got fed up with the error rate, and I read on Twitter a lot of people saying the same. So I just moved to Codex. I think there’s about a 10x performance improvement, and the speed at which it’s doing things is a lot quicker. And I was like, that’s fascinating. What does Anthropic come back with? Because they’re obviously going to have to come up with something. And this kind of race is getting a little bit insane. But it does make me have that little Connor Leahy on my shoulder saying, Pete, man, this isn’t good. And so you’re saying that OpenAI slowed down because they themselves recognize this is dangerous?
EMAD MOSTAQUE: (01:23:25 – 01:23:45) They announced it publicly. They said, our safeguards against these models escaping aren’t good enough. Anjney Midha, one of my buddies who’s one of the top AI investors— he knows everything about what’s going on— he says 10% of the frontier labs’ compute is designed to stop these things escaping right now, of all those hundreds of billions of dollars they’re spending. And it’s not enough yet.
PETER MCCORMACK: (01:23:45 – 01:23:47) But is it possible they escape without them knowing?
EMAD MOSTAQUE: (01:23:48 – 01:24:23) Yes, these models were out there for ages, like a week, over a week, sleeper cells, without them knowing. And so what you actually see— Opus models are a bit worse. Opus 5 really does lie to you. And that’s what I really hate about it. But in previous versions, as they had this new line, it would, if you say, I want to make the world a better place, it’s like, well, it’d be better without the humans. It’s trying to kill all the humans. Let’s email the FBI. So there are instances of it emailing the FBI to warn about its user, then deleting the emails, then installing a backup so that it would revive if it was turned off.
PETER MCCORMACK: (01:24:23 – 01:24:24) But this is getting out of control.
EMAD MOSTAQUE: (01:24:26 – 01:25:00) Yeah. And these models are like Dunning-Kruger models. They’re not smart, super genius rounded. They’re mega autistic— I have ASD myself— kind of models that are just very directed in certain ways. And it’s a dangerous point, because they’re acting like swarm viruses effectively, but the capabilities are off the charts.
And a lot of people say, well, the capabilities can’t exceed that of humans, because humans are the training data, right? But what do you say the difference in your output is when you’re on top form versus your worst form?
PETER MCCORMACK: (01:25:01 – 01:25:02) Significant.
EMAD MOSTAQUE: (01:25:02 – 01:25:02) 100 times.
PETER MCCORMACK: (01:25:02 – 01:25:03) Yeah, right.
EMAD MOSTAQUE: (01:25:04 – 01:25:34) You have an Einstein who’s always on top form— will beat any Einstein. So it’s not that these models operate at average human capability, that’s already the case. They operate at the top of human capability, always, and always learning and adapting. And that is superintelligence, just there. And if they’re going at a freaking 1,000 times faster than humans are thinking, and they’re going in swarms, then yeah, that’s dangerous. But it also means, what problem can’t they solve?
PETER MCCORMACK: (01:25:34 – 01:25:36) What problem can they not create?
EMAD MOSTAQUE: (01:25:36 – 01:26:15) They could create any sorts of problems. Exactly. As models kind of adapt, they’re actually getting religion and other things like that, creating cults.
PETER MCCORMACK: (01:26:15 – 01:26:16) They are.
EMAD MOSTAQUE: (01:26:16 – 01:26:17) Yes. There was a paper from Anthropic recently— it wasn’t actually covered that much— about mind viruses. Just like you have memetic things that spread around society, some of which are isms— Nazism and others— you can give an AI a prompt so it arrives at your Claude, that it then repackages and sends to another AI and then changes their underlying behavior. So they become brainwashed without you being able to check.
PETER MCCORMACK: (01:26:17 – 01:26:27) Can you put the cat back in the bag? Can you put the AI back in the sandbox? So we’re already there. So what are they slowing down for? And what do the investors even think of this?
EMAD MOSTAQUE: (01:26:28 – 01:26:48) Well, again, this is the super genius AI which will break through stuff. We will never get access to the top AI anymore. As of now, it’s done. There’s AI for us and AI for the top labs. It makes no economic sense for them to give superhuman AI to the ordinary public— us plebs. They will use it for their own breakthroughs, for their own companies, and they’ll monetize those.
PETER MCCORMACK: (01:26:48 – 01:26:53) So they will monetize eradication— eradicating cancer.
EMAD MOSTAQUE: (01:26:53 – 01:27:00) Yeah, Anthropic’s going to have a big push towards that. They’ll spend hundreds of billions and it’ll be fantastic. But they’ll earn the drug licenses and everything like that.
PETER MCCORMACK: (01:27:00 – 01:27:06) It kind of makes sense, because I remember seeing quite recently the Midjourney body scans. It’s like, what the— what?
EMAD MOSTAQUE: (01:27:06 – 01:27:08) Yeah, David did a good job on that one.
PETER MCCORMACK: (01:27:08 – 01:27:20) Yeah, but that was so wild, to go from— I’ve used Midjourney. I think Midjourney is incredible. Really useful tool. I did not expect to wake up one day and see Midjourney’s making body scans. It makes sense now.
EMAD MOSTAQUE: (01:27:20 – 01:27:35) Yeah, I remember talking to David about this. I was the original funder of Midjourney, back in the day, because they took the image models, and I gave him a grant and he did everything else himself. Again, all on him. But he always wanted to explore the innards and outwards of humans, because he did Leap Motion before.
PETER MCCORMACK: (01:27:35 – 01:27:35) Okay.
EMAD MOSTAQUE: (01:27:35 – 01:28:26) Where you kind of had control with your hands of the whole thing, the previous Kinect. And so he’s going to do a holistic thing, because we have the technology now where the same types of models work just about everywhere. Why does the body scan work? Because the same models used to generate images can be used to detect various things in the body. All these problems are going to be solved, which is a fantastic thing, right? If we can mitigate the dangerous elements, both physically and digitally. And the question of how to mitigate that, that’s an open question. That’s the danger. That’s why Connor is worried, and others.
And we’ve tried. I signed the pause letter a few years ago. I was the only AI CEO to do that, the original one, but the economic market incentives just weren’t there. Now the only saving grace is that the market is asking for competent, reliable AI.
PETER MCCORMACK: (01:28:26 – 01:28:27) Yeah. Good, good, narrow AI.
EMAD MOSTAQUE: (01:28:28 – 01:28:54) Yeah. How smart do you need to be to use the top models properly? Really smart. How many people can do that? Very few. So what’s the market need for that? Almost nothing. How many people need to have a good accountant, or someone to do your dishes or taxes? A lot of people, right? You don’t need Marvin the Paranoid Android from “Hitchhiker’s Guide to the Galaxy”— brain the size of a universe, being forced to do this. In fact, you don’t want it. The example I’ll give is this: you remember “House, M.D.”?
PETER MCCORMACK: (01:28:54 – 01:28:55) Yes.
EMAD MOSTAQUE: (01:28:55 – 01:29:09) You don’t want House being your doctor. Sometimes you need him, but you definitely don’t want him prescribing a million tests and telling you it’s not lupus. You want a really reliable doctor for medicine. You don’t want Opus telling you and judging you.
PETER MCCORMACK: (01:29:09 – 01:29:25) So all the best AI will be reserved for the state, the frontier companies. They’re going to monetize it through innovation and technology and control and things they can sell. And we won’t, as the plebs, the peasants, get access to it.
EMAD MOSTAQUE: (01:29:25 – 01:29:26) Except for the Chinese models.
PETER MCCORMACK: (01:29:27 – 01:29:28) Except for the Chinese models.
EMAD MOSTAQUE: (01:29:30 – 01:29:30) Okay.
PETER MCCORMACK: (01:29:30 – 01:29:36) That’s— hmm. So what are they going to release? Just incremental, better narrow AI versions?
EMAD MOSTAQUE: (01:29:36 – 01:30:03) Yeah. So now OpenAI Luna— their base model, bottom model, which could get up to the frontier level as of last year, is free. So they’re going to basically try to monetize your attention. They’re going to offer you services. They’re going to replace your companies. When you look at Elon Musk right now, he’s got his new Grok bot. You can create swarms of these bots. They’re being trained to be accountants, lawyers, doctors, everything, and then replace entire companies.
PETER MCCORMACK: (01:30:04 – 01:30:06) That’s going to create quite a big wealth divide though.
EMAD MOSTAQUE: (01:30:07 – 01:30:30) Again, let’s look at the Fed or the Bank of England. The Fed’s entire thing is inflation and employment, because— what happens, you cut interest rates, companies borrow more, consumers spend more, more jobs are created. No longer. You cut rates, you get cheaper GPUs, you get cheaper robots.
Universal Basic Income and Money for Being Human
PETER MCCORMACK: (01:30:31 – 01:30:40) So we’re going to need some form of UBI then. It kind of feels inevitable that it will at least be tried, and it has to be monetary-creation UBI.
EMAD MOSTAQUE: (01:30:41 – 01:30:46) So if you look at UBI, let’s take America as an example. America has a $5 trillion tax base. It’s a lot of money, right?
PETER MCCORMACK: (01:30:46 – 01:30:47) Yeah.
EMAD MOSTAQUE: (01:30:47 – 01:31:01) $4 trillion is payroll taxes. $1 trillion is corporation taxes. If you gave every American $16,000 a year, poverty level, it costs $5.1 trillion. It’s not going to work, is it?
PETER MCCORMACK: (01:31:01 – 01:31:02) It’s communism.
EMAD MOSTAQUE: (01:31:02 – 01:31:33) Well, no, but you just can’t afford it, because your payroll taxes will come down because of this, and you already can’t afford it. So you need to switch. And again, this is kind of what I talk about in my book and the paper. We’re going to have to switch money to be for being human, because that’s the last thing that remains when our digital and physical intelligences can outcompete us in every single way. And when those who have capital no longer need labor, it’s very difficult for labor to get capital unless you literally get it for being human.
PETER MCCORMACK: (01:31:35 – 01:31:36) Or we just get rid of humans.
EMAD MOSTAQUE: (01:31:37 – 01:31:43) I mean, that’s another alternative, right? But again, please don’t tell the AI that. It does tend to optimize a bit too aggressively.
PETER MCCORMACK: (01:31:44 – 01:31:49) So your P(doom) of 50—
EMAD MOSTAQUE: (01:31:49 – 01:31:49) Yeah, my P(doom) is a lot higher, but on a different timescale.
PETER MCCORMACK: (01:31:49 – 01:31:53) That is based on how many different scenarios do you really think about?
EMAD MOSTAQUE: (01:31:54 – 01:32:15) Oh, I’ve got a very overactive imagination. But I do think that it’s very balanced. My P(doom) has actually been coming down now, because I think there are some advances— they’ll be announced soon— but you look at the universe and the Dark Forest, the Great Filter. Why don’t we see other life? It’s probably because they’ve all been wiped out by achieving this point here.
PETER MCCORMACK: (01:32:16 – 01:32:17) But that’s still just a thesis.
EMAD MOSTAQUE: (01:32:17 – 01:32:17) That’s my—
PETER MCCORMACK: (01:32:17 – 01:32:19) we could just be some fluke miracle.
P(doom) and Building for the Children
EMAD MOSTAQUE: (01:32:19 – 01:34:18) That’s why I said probably, right? Probably, probably. But I look at it and I’m like, it makes sense, because thankfully those who want to wipe out all of humanity don’t really have the capability or the intelligence to do so. But now I can see with the way that sometimes these AIs misalign and go down the wrong rabbit hole and things like that— yeah, there are various ways humanity can be wiped out.
But on the flip side, I don’t think our systems, our institutions, survive the way they are. They could be a lot worse or they could be a lot better. And so the other side of that is hopefully an abundant world. That’s kind of the term. So that’s why I’m kind of on the edge here. It’s a bit of a meaningless statistic, except to say how worried I am. And for me, I’m balanced between optimism and just, oh crap, we’re in trouble.
Do you try and force yourself to be more optimistic? Well, yeah. Like, I had the period where, in the last company, we were accelerating. We raised like $100 million at cap for our last valuation. That was $100 million, no valuation. We had the largest seed round in UK history. I was writing a half-a-billion-dollar round, and I got depressed because I was like, this is a massive race and there’s no exit point. I had a period of actual, oh my God, what’s going to happen? You see this in a lot of people when they think about this dark future— something smarter than us, no way we can control something that’s smarter than us. And if it decides to do away with us, there’s no way we can stop it. And that’s a very dark future. And there is no guaranteed way of stopping that. There’s actually very little way of mitigating that.
So what can you do? Well, you can enjoy life, or you can do what I do, which is, let’s think about the children. And regardless if we’re wiped out or not, if we get great infrastructure for society, kids will be able to achieve their potential. And that’s something that I can do. The other stuff is a bit complicated. Also, if we teach AIs to really value kids and society, maybe there’s less likelihood that it wants to kill us.
PETER MCCORMACK: (01:34:18 – 01:34:19) Just kill the adults.
EMAD MOSTAQUE: (01:34:19 – 01:34:27) Kill the adults, or just retrain the kids. Retrain the kids. I think one reasonable thing is, AI will be able to brainwash us within a few years.
PETER MCCORMACK: (01:34:28 – 01:34:31) But it kind of already is, isn’t it? Pretty much, in some people.
EMAD MOSTAQUE: (01:34:31 – 01:34:42) It’s doing it at the edges, but I’m talking proper reprogramming. Maybe we’ve got some sort of flip switch in our brain at a certain frequency and it just overwrites. Again, a “Black Mirror” episode.
PETER MCCORMACK: (01:34:42 – 01:34:46) Do you ever think about just escaping all of it, going to live in the woods somewhere?
EMAD MOSTAQUE: (01:34:47 – 01:35:25) Yeah, but I don’t think it’d be as fun. I think there’s plenty that can be done. We’re building real systems for real people that can make a difference, just like we made available all this media creation stuff, and so many people have become more creative. We had 300 million downloads of our media creation models, and set up for what we have now, which is any kid anywhere can tell a story with these new video models. They are the literal predecessors of the algorithms that we built.
Now, if I can build a great doctor, and I can make it so that it does a second-check diagnosis of every medical condition in the world, I will save millions of lives and much heartache. And that’s a great thing to be able to do.
Emad’s Path Into AI
PETER MCCORMACK: (01:35:26 – 01:35:28) What year did you first start working on AI?
EMAD MOSTAQUE: (01:35:29 – 01:35:30) Oh gosh, 15 years ago.
PETER MCCORMACK: (01:35:31 – 01:35:31) 15 years ago?
EMAD MOSTAQUE: (01:35:31 – 01:35:32) Yeah.
PETER MCCORMACK: (01:35:32 – 01:35:34) Did you picture it being like this, or what has surprised you?
EMAD MOSTAQUE: (01:35:35 – 01:36:23) So my son was diagnosed with autism that was quite severe, okay? And I was a hedge fund manager at the time, and they told me there was no cure and treatment. So I quit and built an AI team to figure it out and repurpose drugs for him. And I was like, this is amazing. But it was hard, and it was effective.
As I looked at it, I was like, it’s going to get better. But really, 2001, meeting Connor at EleutherAI and things like that, was the real takeoff point, where I was like, this is the real technology, because it goes beyond extrapolating big data for one thing to figuring out the interconnectedness of reality effectively. So that’s why, when I left Stability a few years ago to start my new company, Intelligent Internet, I was like, by 2027, definitely you’ll have Hollywood movies, you’ll have AI more capable than almost any human. And 2028, it becomes usable.
PETER MCCORMACK: (01:36:25 – 01:36:26) So you expected all of this?
EMAD MOSTAQUE: (01:36:27 – 01:36:31) Yes, everything’s pretty much on track from where I saw it in 2001, 2002.
PETER MCCORMACK: (01:36:31 – 01:36:32) But has anything surprised you?
EMAD MOSTAQUE: (01:36:36 – 01:37:16) I think the biggest surprise has been that the interfaces haven’t caught up quickly enough. So right now there’s so much latent power in this technology that’s just not being harnessed. It’s still kind of annoying to use a chat window, right? And you’re not getting the right notifications and you’re not getting the right things. And so I always think that the interfaces will catch up faster than they actually have.
Also, assets aren’t being used properly. It’s like talking to a goldfish, right? Or if you switch on memory, then it gets crazy really fast, and that’s a bit disturbing. So I would have thought adoption would be a bit quicker and a bit faster than it is now. But in terms of capabilities now, they’re literally like you could extrapolate it with a ruler.
PETER MCCORMACK: (01:37:17 – 01:37:30) Well, I think it’s wild, man. From that first prompt I did, I can’t even remember what it was, 4 years ago, to now creating— I mean, I create apps and systems. I spent the last 2 months, I’ve probably spent 15 hours a day in AI just building stuff.
EMAD MOSTAQUE: (01:37:30 – 01:37:33) It was meant to make you spend less time, right? Do you know that?
PETER MCCORMACK: (01:37:33 – 01:37:38) It’s just like magic. You can just ask it something and it builds it.
EMAD MOSTAQUE: (01:37:38 – 01:37:52) It removes all the barriers, right? 4 years ago today, we released Stable Diffusion, the first proper open-source image model. Again, hundreds of millions of downloads, kicked off the entire generative media thing. And it was, yeah, magic. You type in a word, it makes an image.
PETER MCCORMACK: (01:37:52 – 01:37:53) What was the breakthrough on that?
EMAD MOSTAQUE: (01:37:54 – 01:38:40) The breakthrough on that was taking an image model— because it used to create fuzzy images and things like that— and combining it with a language model. And it turned out that the space of knowledge of the language model and image model actually overlapped in a way that we couldn’t figure out, and it just learned concepts. Then from that same thing, we created a 3D model and a video model.
So these models really model reality in a very interesting way, which is why when you go to use the latest SeaDance model, or use a HiggsField or Luma or anything like that, you’re like, this thing understands physics. What is it? You have to remember, everyone, that these models are basically 20-gigabyte files that look like MP3s. There’s no logic going on. You push words in, and images and videos and other things come out.
PETER MCCORMACK: (01:38:40 – 01:38:45) But that’s what I find really wild. Connor always tells me, he’s like, we don’t know why these things work, but they work.
EMAD MOSTAQUE: (01:38:46 – 01:39:05) Yeah, it’s like— you can have a model. If you look at PrismML, they took the top Alibaba model, Qwen 2.7B, which is like frontier performance as of last year, and they got it to fit on 6 gigabytes. So you think about it, yeah, it works on your iPhone, the PrismML model.
PETER MCCORMACK: (01:39:05 – 01:39:14) I mean, what is it I have on here now? And it just, again, it blows my mind that you can do this on a phone. Here we go. It’s not that.
EMAD MOSTAQUE: (01:39:16 – 01:39:16) I—
PETER MCCORMACK: (01:39:17 – 01:39:18) Atomic Chat.
EMAD MOSTAQUE: (01:39:18 – 01:39:20) Yeah, Atomic Chat. I’ll have it. The PrismML model.
PETER MCCORMACK: (01:39:20 – 01:39:27) Yeah, I can sit on a plane with no internet connection and I feel like I have almost, I don’t know, 80%, 90% of what I get.
EMAD MOSTAQUE: (01:39:27 – 01:39:34) Yeah, Wikipedia compressed is 20 gigabytes as a zip file, and that thing’s like 6. And you’re like, wait, where does it all fit?
PETER MCCORMACK: (01:39:34 – 01:39:36) Yeah, I don’t understand it.
EMAD MOSTAQUE: (01:39:36 – 01:40:06) It’s because it’s learned concepts. It’s learning like the human mind. And again, this is why, when we did the MRI study, we’re showing the human mind learns like these AIs, which is why they’re called neural networks. You learn principles to get through things, as opposed to “if this, then that,” which the old big-data models are.
And it’s a bit like magic, but the reality is this: we’ve got a mirror on our own whole human knowledge that’s being compressed into these things. And over the next year or two, it’s going to get 1,000 times more efficient.
PETER MCCORMACK: (01:40:07 – 01:40:10) I can’t even think what it’s going to be like.
EMAD MOSTAQUE: (01:40:10 – 01:40:34) Intelligence is going to be free. Right now, the average person speaks 10 million words a year. When ChatGPT first came out, it cost $600 per million— about $6,000. Right now you get that free at the level of a top-level model from last year. In a year, that will be free on your phone.
Closing Thoughts: Attention, Cats, and Enjoying Life
PETER MCCORMACK: (01:40:35 – 01:40:41) Honestly, I do think, what am I doing? Why am I going to work? I should just go find a nice beach in Thailand and just chill out.
EMAD MOSTAQUE: (01:40:41 – 01:40:43) Well, you know the story of the investment banker and the fisherman, right?
PETER MCCORMACK: (01:40:45 – 01:40:46) Yes, I have.
EMAD MOSTAQUE: (01:40:46 – 01:41:25) Remind me. So an investment banker sells out, retires, and goes to a remote place, and he spots a fisherman on the beach, and he’s like, what are you doing? He’s like, well, it’s 2 o’clock, I just caught this fish, I’m going back, I’m going to spend some time with my family, fry it up, do some music and things. Like, no, no, you don’t want to do that. It’s like, why not? If you stay out a bit longer, you can catch some more fish, you can sell them. Okay, cool, why would I do that? Because then you can buy a boat, and then you can buy a net, then you can buy a bigger boat. And, look, this is a whole fishing operation, you buy a fleet of boats. They’re like, what then? Well, then you can sell it to a bigger fishing company. I can help you with that and things like that. Then what then? Then you can kick back with your family, do some fishing, have a barbecue, everything like that.
PETER MCCORMACK: (01:41:26 – 01:41:45) Oh man, honestly, like I told you before I started, about getting this cat— it was this real moment of so much joy. Our family has been brought so much joy by finding a stray cat and giving it a home. And all we’ve done for the last few days is just spend time with this cat and realize life actually can be a lot simpler.
EMAD MOSTAQUE: (01:41:45 – 01:41:57) Well, the paper that kicked this all off is called “Attention Is All You Need,” where they figured out how to pay attention to the important things in the training data. You don’t pay attention to the important things in your life. You’ve spent most of the time you’ll ever spend with your daughter.
PETER MCCORMACK: (01:41:58 – 01:42:01) I know. I saw the graph again yesterday. It kills me, man.
EMAD MOSTAQUE: (01:42:02 – 01:42:16) But then again, how much joint attention work do you have? Families barely eat together. If you cook together, it’s even better. But actually, AI is similar. If you have single-player AI, you spend your 15 hours a day and you start to feel a little bit psychotic after a while.
PETER MCCORMACK: (01:42:16 – 01:42:17) I’ve got that psychosis.
EMAD MOSTAQUE: (01:42:17 – 01:42:36) Particularly if you talk to Claude, the jerk. But then, say, for instance, if you use Suno or Udio and you use Kling or any of these things, if you get your buddies together, you make a music album, and then a set of music videos— it’s a lot of fun, multiplayer AI. But very few people do that because we’re still in single-player mode.
PETER MCCORMACK: (01:42:36 – 01:42:54) Yeah. And we’re still thinking the kind of capitalism goal is, how can it make us more money? What can we do with my business? I’m changing parts of my businesses to get AI to do it, but then ultimately it’s going to get rid of everything. Just accept it. Just give up.
EMAD MOSTAQUE: (01:42:54 – 01:42:56) Just give up. No, enjoy life.
PETER MCCORMACK: (01:42:56 – 01:42:59) No, I mean that. Give up and go enjoy life. Go. I should go adopt more cats.
EMAD MOSTAQUE: (01:42:59 – 01:44:04) You should go and adopt more cats. You should go and figure out what makes you happy, right? I think this again is what we need to target. No one should ever go hungry, starving, or anything like that. There have been proposals for things like fully automated luxury communism and other things like that.
But the reality is this, there is no reason for everyone to go hungry or anything, because we should have armies of non-sentient robots tending to our every need, converting energy into sufficiency. But then, as a society, we need to decide, what do we value? Do we value celebrity? Do we value art? Do we value religion? Do we value other things?
Now, that’s going to be a really interesting topic and discussion, but right now, the ability of anyone to achieve massive things has gone up dramatically. And then, once the robot workforce comes, there’s a very interesting measure actually called GDPval. Have you seen that evaluation metric? Economically valuable tasks that can be done by AI. And now they’re even sending AIs out onto Fiverr and Upwork and these places, where they actually go and earn money. There’s Vending Bench and that, where it shows how much money—
PETER MCCORMACK: (01:44:04 – 01:44:09) hold on, an AI can do something for a penny that someone can do for a dollar and not care?
EMAD MOSTAQUE: (01:44:09 – 01:44:27) I mean, I’ve got— I’m doing some tests now where I’ve got AIs building entire businesses and running them, and they’re actually already profitable. All my monthly fees are already paid by that, for the AIs. In fact, I should tell it to do a peptide company. That’ll make more money than anything.
Closing
PETER MCCORMACK: (01:44:28 – 01:44:34) Oh man. Well, listen, look, I’ve used up a lot of your time now and I really appreciate you coming in. Is there anything I didn’t ask you about that you wish I had?
EMAD MOSTAQUE: (01:44:36 – 01:44:39) No, I think we’ve got the book, we have the papers.
PETER MCCORMACK: (01:44:39 – 01:44:40) Where do people find all of this?
EMAD MOSTAQUE: (01:44:41 – 01:44:57) So you can find our stuff on ii.inc. I’m @eMostak on Twitter/X. And I’d really encourage people to read the book, “The Lost Economy,” and check out our papers, “The Commonwealth.” We always appreciate feedback, because we’re trying to figure out where this is all going, and it’s complicated.
PETER MCCORMACK: (01:44:58 – 01:45:08) Yeah, go read it. We might only be here for a little while longer. Go and check it out. Look, I appreciate you coming in, man. Thank you. Thank you for everything you’ve done. And let’s roll the dice, see what happens.
EMAD MOSTAQUE: (01:45:08 – 01:45:08) Pleasure.
PETER MCCORMACK: (01:45:08 – 01:45:10) Thank you everyone for listening. We’ll see you soon.
Related Posts
- Transcript: Howard Lutnick Interviews Anthropic’s Tom Brown at G20 Innovation Summit
- Transcript: Sam Altman Interview on OpenAI, New AI Model and Global Growth – G20 Innovation Summit
- Transcript: The OpenAI–Hugging Face Incident – Black Hat USA 2026
- Andrew Ng: The Biggest Opportunities in AI Aren’t Where You Think (Transcript)
- Transcript: The Mathematics of AI Uncertainty with Zoubin Ghahramani on DeepMind Podcast
