Former OpenAI and Anthropic pre-training researcher Jacob Coxon sits down with Patrick Bet-David to explain the viral resignation tweet that put AI safety on the White House agenda. He breaks down why 2027 could be a chaotic year for AI, the risk of Chinese spies inside top AI labs, the rivalry between Sam Altman and Dario Amodei, and what everyday families should do to prepare for recursive self-improving AI. This interview was premiered on October 6, 2026. Read the full transcript of this interview below:
The Resignation Tweet Heard Around the World
PATRICK BET-DAVID: (00:01:16 – 00:02:03) So on September 8th, a Jacob Coxon who’s here with us— good to have you on the podcast.
JACOB COXON: (00:02:03 – 00:02:04) Nice to be here.
PATRICK BET-DAVID: (00:02:04 – 00:03:08) You decide to put on a tweet. Okay. You had started your account January of 2026. You hadn’t put nothing up at all. Your first tweet says, “I resigned from Anthropic today. I spent the last 3 years doing pre-training research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.”
We’ll get into that later on. And this tweet gets 803,000 likes, 175 million views. There was a meeting yesterday at the White House prompted by you indirectly. You got politicians that got involved, world leaders that got involved, a meeting that happened with Xi that was here last week. One of the topics of discussion was how powerful AI is going to be.
And the funny part that I think it’s important for the audience to realize this, and you can fact-check me on this, that you let all your stocks go, that you weren’t even able to vest your stocks because of the timing that you had, which means any money on the stocks that you also had was left behind for you, if that’s correct, right? There was no— the options didn’t move on with you.
JACOB COXON: (00:03:08 – 00:03:09) Not at Anthropic.
PATRICK BET-DAVID: (00:03:09 – 00:03:11) Not at Anthropic. At OpenAI, you were there.
JACOB COXON: (00:03:11 – 00:03:12) I have OpenAI.
PATRICK BET-DAVID: (00:03:12 – 00:03:12) Okay.
JACOB COXON: (00:03:12 – 00:03:12) Yeah.
Why 2027 Will Be a Chaotic Year for AI
PATRICK BET-DAVID: (00:03:13 – 00:03:23) So you did all of that and the world kind of stopped for a few days. Till today, they’re still talking about it. What did you see that got you to say, “I have to do this”?
JACOB COXON: (00:03:24 – 00:04:27) Yeah, it really was just thinking about next year and the year after. I think 2027 is going to be a crazy year. And I think about the news headlines. So last year, AI a bit in the news. 3 years ago, it was very rarely in the news, maybe for ChatGPT, but otherwise no.
This year it’s bam, bam, bam, AI. Every month there’s a big AI story. Next year, it’s going to be like every day there’s going to be a big AI story. In particular, just imagine, I don’t know, I wasn’t alive during World War II, but during World War II, I imagined that just the concept of war was coloring everything.
It was in the background for every day-to-day thing. Like every news story, war would be in the background. I think it will be like that for AI next year. In terms of what I specifically saw, it is the models, the AI models next year are going to be very capable in a way where they’re going to be able to automate a lot of work, a lot of everyone’s jobs, but kind of most importantly, a lot of the AI researchers’ jobs.
So things are just going to speed up so much that it’s going to become a very, very serious thing next year, probably.
PATRICK BET-DAVID: (00:04:27 – 00:04:33) So 2027 for you is what’s going to be— is a word to use chaotic?
JACOB COXON: (00:04:34 – 00:04:35) Chaotic? Definitely, yes.
PATRICK BET-DAVID: (00:04:35 – 00:04:39) So unpack it for me. Yeah, the average person listening to this, what does that look like?
JACOB COXON: (00:04:39 – 00:05:20) Yeah. So there’s many different things that could happen. Things that kind of the most important thing is right now labs are treating this like the companies are treating this like it’s a normal technology. This is business. They have, maybe you saw OpenAI kind of, they have these dev days. They share their products with users.
They’re still tweeting like, “Oh, excited to release this new product.” Thing is, this is a super weapon. So soon people are going to have to realize that this is not like a casual business thing. This is like a superweapon technology that really should be nationalized. And I think next year is kind of probably when that realization really sets in that, that this is like very, very serious stuff.
Should AI Be Nationalized Like Nukes?
PATRICK BET-DAVID: (00:05:20 – 00:05:31) You’re saying it should be nationalized. So if it’s seriously that concerning— yeah, when you say nationalize, you trust the government getting involved to control this than free market?
JACOB COXON: (00:05:31 – 00:06:00) Yeah, I’m not saying I need nationalization. I’m saying that it is so serious that I’m surprised the government isn’t nationalizing. The government is looking at these companies, they’re going to have power. This tech is, is powerful tech. And I’m just very surprised that Trump can sit there and say, “I’m going to let Dario Amodei, I’m going to let Sam Altman have the keys to this powerful tech.”
Because normally it’s the government that has the keys to this stuff. You wouldn’t let Mark Zuckerberg have nukes. That’s just not a thing that normally happens.
PATRICK BET-DAVID: (00:06:00 – 00:06:02) Make the comparable of the two. Yeah.
JACOB COXON: (00:06:02 – 00:06:49) So nukes cause world chaos, obviously, like plausibly.
Like to give an example, If you’re Anthropic and you’re training an AI and you’re going to share it with the world, one of the things they worry about is, “What if China can steal from this?” And that’s a private company worrying about geopolitics, which is a weird thing, like for a private company to make decisions that affect the whole, like the whole geopolitics situation.
PATRICK BET-DAVID: (00:06:49 – 00:06:52) Yeah, but some companies become so big that you become a small country.
JACOB COXON: (00:06:52 – 00:06:52) Yes.
PATRICK BET-DAVID: (00:06:52 – 00:07:19) Like you have to realize Zuck is a world leader of 2.4 billion people called Facebook. And he’s got regulations he deals with the government and whatever countries he’s in, he has to abide by those rules and guidelines. That’s why he’s always in lawsuits because he broke some of the guidelines, right? So same with Twitter, same with Google, same with YouTube, same with all these guys. So they almost have to be because they’re dealing with those types of people.
JACOB COXON: (00:07:20 – 00:07:20) Right?
PATRICK BET-DAVID: (00:07:20 – 00:08:09) Yeah. So, but what I want to know from you is, as we’re going through this, because some of this to me is, I wonder if it’s fear porn where they’re just using to scare a regular person watching this. I wrote it down. I said, I think this is how I see it. I just kind of want to share with you on how I see the fear porn side of this.
So one part of the fear porn is, “Hey, this is so scary, AI is going to take over the world, we need the government to control.” Okay, that’s one. Okay, people are using this because they want control. They want the government to control. Number 2, “Hey, this is so big and powerful and we need government control because I’m Anthropic and OpenAI.”
I’m eliminating my other companies to come and compete with me so the barrier to entry becomes more difficult and I get to control. Yeah, so the motive there becomes eliminating future competition. Pretty good move if you’re thinking that way, right?
JACOB COXON: (00:08:09 – 00:08:09) Definitely.
PATRICK BET-DAVID: (00:08:09 – 00:08:26) The 3rd level of fear porn is well, let me tell you, this thing is so powerful because we’re about to go public. As big as we thought this was, this is so much more powerful than us. You may want to invest some of your money in there because when we go public, you may 10x, 20x, 30x your money.
JACOB COXON: (00:08:26 – 00:08:26) Yeah.
PATRICK BET-DAVID: (00:08:26 – 00:08:48) So the public IPO goes just massive, right? So those are 3 elements of the fear porn that I see. Then the other one is maybe it is as scary as you say it is. The other one is if we don’t build it, China builds it, you know. And last but not least, if we all stop How can somebody hold an independent person or China to build this?
So of those, if you can one by one address which one you think it is.
Fear Porn or Real Danger?
JACOB COXON: (00:08:48 – 00:09:45) Yeah. So to go to the original fear porn stuff, especially about the regulation control, or Anthropic hyping up their product, I think part of the reason my tweet went so viral is many people at Anthropic said the same thing, but they didn’t just quit. They stayed there. So it’s easier to be like, okay, he’s staying at the company.
He’s saying this stuff, but he’s just hyping the product. I left in a personal capacity because I am scared. So from my perspective, I have nothing to gain by hyping up Anthropic. That’s not a— I’m not trying to hype them up or get them like special criteria. I think what they’re doing is dangerous.
I think it’s reckless. I think they should be curtailed. And I left to say that. So that’s to address the like regulatory capture stuff. Also later we could go into, why I don’t think it’s actually like— it doesn’t actually make sense to stoke up this level of fear. I think it’s going to hurt their IPO.
I think it’s going to hurt their ability to stay in control. We can talk about that later.
PATRICK BET-DAVID: (00:09:45 – 00:09:45) Okay.
JACOB COXON: (00:09:47 – 00:10:17) What was the other point? China. Yeah, the China point is, is difficult. But I think again, if you appreciate that the tech is scary and it’s going to feel scary for the people building it I think China will also realize that. The people building the technology in China will see how destabilizing it is, and I just don’t think that that regime wants to create a destabilizing technology that could threaten their own power.
I think actually our incentives are aligned, like we’re more similar to China in this respect, so it should be quite easy to cooperate.
Can We Trust China to Slow Down?
PATRICK BET-DAVID: (00:10:18 – 00:10:29) Well, if you look at the advancement, the separation between us and China, one could say the motivation— hang on one second, I’m going to show you one data.
JACOB COXON: (00:10:30 – 00:10:32) Yeah. And China.
PATRICK BET-DAVID: (00:10:32 – 00:10:38) So motivation for China could be US has such a massive lead in AI, we need them to slow down so we can catch up.
JACOB COXON: (00:10:38 – 00:10:38) Yeah.
PATRICK BET-DAVID: (00:10:39 – 00:10:45) There’s no way in the world that China is going to sit there and say, “Oh yeah, President Trump, for sure we’re going to shut it down and we’re not going to work on it.”
JACOB COXON: (00:10:46 – 00:10:46) Yeah.
PATRICK BET-DAVID: (00:10:46 – 00:10:55) “Okay, great. Can we come and look at your place to make sure you are actually keeping your promises?” “No, just take our word for it. We are not working on superintelligence.”
JACOB COXON: (00:10:55 – 00:10:55) Yep.
PATRICK BET-DAVID: (00:10:56 – 00:10:59) Do you believe China is not going to work on superintelligence and they’re going to listen to us?
JACOB COXON: (00:10:59 – 00:11:21) It depends on the nature of a deal. Right now, if we just stopped, they would try and catch up. But I think one really important thing people don’t realize is China already has spies at OpenAI, at Anthropic. China has the ability to hack at any time. If we pull ahead, that doesn’t give us a big lead.
They can just step in and steal our progress at any time. You think catch up the lead?
Chinese Spies Inside OpenAI and Anthropic
PATRICK BET-DAVID: (00:11:22 – 00:11:24) How certain are you they have spies already there?
JACOB COXON: (00:11:24 – 00:11:27) 90%. Yeah, definitely. It’s almost certain that they have. They have.
PATRICK BET-DAVID: (00:11:27 – 00:11:29) Do people talk about that at OpenAI?
JACOB COXON: (00:11:29 – 00:11:30) Yeah, they do. People have spoken about this.
PATRICK BET-DAVID: (00:11:30 – 00:11:33) Is it kind of like jokingly, “Hey, we probably have some Chinese spies here”?
JACOB COXON: (00:11:33 – 00:11:37) Yeah, it’s like a joking thing because you don’t take it seriously. But then it is a serious thing. There must be spies. There is no—
PATRICK BET-DAVID: (00:11:37 – 00:11:39) Have you met people that you think they could be?
JACOB COXON: (00:11:39 – 00:11:45) No, I think obviously it’s— I don’t want to stereotype people. Based on like, are they like Chinese nationals?
PATRICK BET-DAVID: (00:11:45 – 00:11:45) For sure, yeah.
JACOB COXON: (00:11:45 – 00:12:07) But, so I wouldn’t say any specific person, but there must be spies. There’s no background check, like serious background check for this sort of stuff yet. It’s not like a, this is another reason why nationalization is so important. You can start properly vetting and background checking. So it’s like, I don’t think SpaceX can hire Chinese nationals, but these AI companies can hire, foreign nationals like me, which is pretty surprising.
PATRICK BET-DAVID: (00:12:07 – 00:12:10) Why is that? What’s the difference? Because SpaceX has some regulatory—
JACOB COXON: (00:12:10 – 00:12:12) Yeah, there’s a regulation. Yep, so they can’t.
PATRICK BET-DAVID: (00:12:12 – 00:12:13) So they cannot recruit Chinese nationals, but OpenAI and Anthropic can.
JACOB COXON: (00:12:15 – 00:12:15) Yes.
PATRICK BET-DAVID: (00:12:16 – 00:12:21) So do you think one step could be to not allow Chinese nationals to work for Anthropic and OpenAI?
JACOB COXON: (00:12:21 – 00:12:25) It’s a bit sad because they’re my friends, but I think that will happen. I think that will probably happen.
PATRICK BET-DAVID: (00:12:25 – 00:12:27) Are you saying will, or do you think that’s the right move?
JACOB COXON: (00:12:27 – 00:12:27) I think both.
PATRICK BET-DAVID: (00:12:28 – 00:12:51) Okay, so it’s uncomfortable for you to say it, and I applaud that. I respect that. But you at least— that shows me there is integrity in your belief system that your friends are going to see this. They’re going to say, “Hey Jacob, what are you saying? We’re friends.” But you think, okay, so you think if we continue, to China almost they’re like, “Yeah, continue because we have access to everything you’re doing anyways”?
JACOB COXON: (00:12:52 – 00:12:52) Yeah.
PATRICK BET-DAVID: (00:12:53 – 00:12:59) So if we continue, they’re going to have access to it. If we don’t continue, China is going to create it themselves. So either way, China is eventually going to have access to it.
JACOB COXON: (00:12:59 – 00:13:39) I think so, unless again there’s some sort of deal. One thing is, a lot of their progress right now— not all of it, but a substantial part of it is distillation, which is sort of stealing the data. So they use Claude, they log in, they ask it questions, and they use the answers to train Chinese models.
So right now, a lot of their speed is kind of copying the US and stealing from the US. So also, if we slow down, it will probably slow down China quite a bit too, because currently a lot of their progress is from essentially copying or stealing US work. Not all of it. They’re still— they will still catch up and make progress, but there is a component. So like, they will also slow down if the US slows down.
PATRICK BET-DAVID: (00:13:40 – 00:13:45) Who else is another player out there that people worry about? Because it seems like it’s mainly them 2, maybe UK a little bit, right?
JACOB COXON: (00:13:45 – 00:13:49) Not the UK at all, sadly. That’s why I’m in— that’s why I’m in the US. It’s very sad, but the UK just hasn’t—
PATRICK BET-DAVID: (00:13:49 – 00:13:50) Would you put 3rd?
JACOB COXON: (00:13:52 – 00:13:59) I’m actually not sure who’s got the data center build out right now. US and China are just so far ahead that no one is— no one’s going to catch up right now.
PATRICK BET-DAVID: (00:14:00 – 00:14:38) Okay, so go a little bit deeper into the Chinese spy part. You said it is a discussion that people have in those organizations. So what would that look like if the average person wants to know? All right, what would that look like on how they pass their messages or information over to them?
Is it— can you do me a favor and can you type in who are the highest-ranking executives at OpenAI or Anthropic that are Chinese nationals, highest-ranking executives at OpenAI or Anthropic that are Chinese nationals. Have you looked at the inventory to see how many of them are Chinese nationals?
JACOB COXON: (00:14:39 – 00:14:50) I don’t think there are many that are that high up that are Chinese nationals. Again, I really— I’m like, this does make me sad. I’m not trying to, start a witch hunt. I’m just saying objectively, there are probably spies. And I think that— yeah.
PATRICK BET-DAVID: (00:14:50 – 00:14:55) Okay, so let’s go over this. So Chief Research Officer at OpenAI, Mark Chen.
JACOB COXON: (00:14:55 – 00:14:56) Yeah, he’s not a Chinese national.
PATRICK BET-DAVID: (00:14:56 – 00:15:00) He’s not a Chinese national. You know who he is? It says Chinese born.
JACOB COXON: (00:15:00 – 00:15:02) He’s an Asian American. Yeah.
PATRICK BET-DAVID: (00:15:03 – 00:15:05) So he would— but he was born in China.
JACOB COXON: (00:15:06 – 00:15:08) Yeah, I just— I don’t think Mark Chen is a Chinese spy. Okay.
PATRICK BET-DAVID: (00:15:09 – 00:15:11) All right. And you met him? You’ve,
JACOB COXON: (00:15:11 – 00:15:12) Yeah, I’ve met him a few times. Yeah.
PATRICK BET-DAVID: (00:15:12 – 00:15:15) Okay. So, Fidji Simo.
JACOB COXON: (00:15:16 – 00:15:17) I don’t think he’s a Chinese spy either. No.
PATRICK BET-DAVID: (00:15:18 – 00:15:21) Brad Lightcap, CEO. So Dario, we know that.
JACOB COXON: (00:15:21 – 00:15:23) No, I think we are clear on these guys.
PATRICK BET-DAVID: (00:15:23 – 00:15:24) Yep, you are clear on all these names?
JACOB COXON: (00:15:24 – 00:15:25) Yeah, these guys.
PATRICK BET-DAVID: (00:15:25 – 00:15:29) Who are the highest rank? Are there any high-ranking people that could be Chinese nationals, or you don’t know any names?
JACOB COXON: (00:15:29 – 00:15:33) Again, I really— I don’t want to call out names. I haven’t thought about that detail.
PATRICK BET-DAVID: (00:15:33 – 00:15:41) Yeah, I don’t— I’m not asking you to give names. Yeah, I want to ask you— forget about giving names, I’ll find out. Yeah, what I’m trying to find out from you is because the audience is going to go do their own research, right?
JACOB COXON: (00:15:41 – 00:15:52) So I think, yeah, almost all the Chinese nationals are like employees. So not like executives, not high-level managers. They will be like employees doing coding, doing engineering, doing research. Yep.
PATRICK BET-DAVID: (00:15:54 – 00:16:02) Yeah. If we pull up— I went to an insurance company once in Dallas and I went on their 7th floor. The 7th floor was all the engineers that they had.
JACOB COXON: (00:16:02 – 00:16:02) Yep.
PATRICK BET-DAVID: (00:16:03 – 00:16:39) If I told you 98% was Indian, I never met anybody that wasn’t Indian. Everybody was Indian. And they had a lot of their guys that came from, What is that institute from India? IIT Institute, which IIT Institute is like a— some people even say it’s better than MIT Institute. So you saw that, and one could say, okay, they’re in here in the business.
But when you’re saying Chinese nationals, India doesn’t necessarily have the same motive that China has. One in America wouldn’t say China is an enemy, India is an enemy, as much as they would say China is the enemy, right? So if you’re saying they already have Chinese nationals in there That’s probably a pretty big concern there.
JACOB COXON: (00:16:40 – 00:17:05) And I think what it is, is it’s representative. Remember I said, they’re not taking this seriously, right? It’s like it’s a normal business. Yeah, as an absolute minimum, background checks for employees seem important. I think they will come, but if they’re not vetting for something that could be a superweapon technology, it’s just crazy. It’s like cowboy stuff. It’s— I’m surprised it’s happening right under the noses of the government. It’s ridiculous.
PATRICK BET-DAVID: (00:17:06 – 00:17:13) By the way, that’s a very valid point. Who else do you think they need to be vetting and not allowing to work at companies like that?
JACOB COXON: (00:17:13 – 00:17:36) So I don’t know the exact details, but, plausibly even— I’m a foreign national, so I’m from the UK. I think it’s important. A lot of the researchers come from the UK. I do think careful background checks are necessary. What are you looking for in particular? I guess, no compromise, whatever the current, clearance background checks are, you should probably need that to work on core AI systems. And I think that will happen.
PATRICK BET-DAVID: (00:17:37 – 00:17:49) But I mean, even if you’re doing that, what am I looking for? What would I find anyways? Nowadays, people are professionals at hiding whatever they did in the past. I know it’s not even for me to be able to find everything about that individual. Yeah, it’s—
JACOB COXON: (00:17:49 – 00:17:58) So I’m not— again, I’m not an expert on like the details. What I think is like they need to take it more seriously. Okay, at a bare minimum. Yeah, fair enough.
Responding to Marc Andreessen and David Sacks
PATRICK BET-DAVID: (00:17:58 – 00:18:43) So that part is, made clear. We got that. What do you say to the people that say, David Sacks, some of these other folks came out, very reputable people, respect these folks, and you saw Marc Andreessen came up and he wrote his Humanity Manifesto with AI and saying, hey, this is a let free market do its part, this is a beautiful thing.
He believes, we can handle 50 billion people living in the world, and We have bandwidth and this is going to make people more financially free. It’s not that scary. Some of those guys said this is purely used by Anthropic and OpenAI to eliminate future competitors and own bigger market shares. And they’re like, “Yeah, Bernie, come help us out. Come prevent other people from competing with us.” What do you say to that?
JACOB COXON: (00:18:44 – 00:19:45) Yeah, there’s a few things. One is I think I agree with Marc Andreessen and David Sacks about the positive benefits. What I think, where I think we differ is their case. I think we get their world if we slow down. I think even if we slow down AI development, the changes from next year’s technology will be so rapid and, significant that there’ll be huge changes in the way that they’re thinking.
I just think they haven’t fully internalized how crazy it will get if we don’t slow down. I think their case for their optimistic fast case is my slow case. And my fast case looks like we have no control over it. We’re just watching robots make themselves smarter, every day massively. It’s completely out of our control.
It’s not like 50 billion humans. It’s humans and then this massive AI economy that’s happening without— we’re just sitting around watching them do inscrutable, things we don’t understand. And I think Marc Andreessen and David Sacks will agree with me when things get crazy, because I think we want the same thing. That’s my guess.
Young Researchers, Politics and Libertarian Views
PATRICK BET-DAVID: (00:19:46 – 00:20:01) How many people, when you worked at OpenAI, Anthropic, guys that are actually doing the work, not executives at the top, I’m talking the engineers, the mathematicians, the guys like you that are doing the work, the researchers that are doing the work, how many of them are in their 20s?
JACOB COXON: (00:20:02 – 00:20:05) Probably half or a bit less than half.
PATRICK BET-DAVID: (00:20:06 – 00:20:07) Okay, so that’s a big number.
JACOB COXON: (00:20:07 – 00:20:07) Big number.
PATRICK BET-DAVID: (00:20:07 – 00:20:10) So just fresh out of college, right?
JACOB COXON: (00:20:10 – 00:20:13) In their 20s. Maybe more like 30%, 30, 40%.
PATRICK BET-DAVID: (00:20:13 – 00:20:49) 30%. And is it fair to say that that audience just got out of college— not all of them, some of them— and maybe they’re going to lean a little bit more on the liberal side, and maybe they’re going to be a little bit more afraid of climate change, end of the world, professors brainwashing them, and oh my God, you guys be careful with these capitalists, they’re going to be such technocrats and technocracy, and oh my, these trillionaires, Elon Musk is going to take over and do this.
Don’t you think they may have a little bit more fear, anxiety of the end of the world is coming because they just got out of college?
JACOB COXON: (00:20:49 – 00:20:51) Yeah, maybe. I can say a few things here.
PATRICK BET-DAVID: (00:20:51 – 00:20:51) Please.
JACOB COXON: (00:20:51 – 00:21:40) One is, I think it’s important that Elon Musk himself is scared about this. That he doesn’t get scared of many things, right? He’s a titan. The guy’s an absolute— like, he just goes for stuff. He’s been scared of AI for a long time. He only made xAI because he saw the other companies doing it.
And was like, “Well, I guess if we’re doing this, I have to join in” because he’s Elon Musk. He’s not going to let these guys, he’s not going to sit on the sidelines. So it’s, big guys are scared too. And then in terms of junior employees, I mean, I can speak for myself.
I like, I’ve never, global warming has never been a big deal for me. It’s like I wanted to learn math. I wanted to work on technology. I wanted more nuclear power. I want like build out of energy infrastructure. I’m like pro, I’m pro-capitalism, pro-libertarian almost.
PATRICK BET-DAVID: (00:21:40 – 00:21:40) You’re more libertarian?
JACOB COXON: (00:21:41 – 00:21:41) Definitely.
PATRICK BET-DAVID: (00:21:41 – 00:21:45) And I think that’s very important for you to qualify that because a lot of people were putting you as a climate change guy.
JACOB COXON: (00:21:45 – 00:22:11) It’s because everyone agreed with me was a Democrat. I made a post. I didn’t ask these Democrats, like Democrat politicians, to agree with my post, right? I didn’t tell them to tweet that. So I think I’ve been like pegged as this NGO kind of dude. Really, the one thing I’m scared about is AI.
I’m like, if we could slow down AI and speed up everything else, like deregulate drugs, deregulate biology research, deregulate, human self-experimentation. I think a lot of stuff I’m pro-libertarian. It’s just AI in particular, that’s the thing that’s scary.
PATRICK BET-DAVID: (00:22:13 – 00:22:23) Huh. So would you say most of the guys and gals your age working at OpenAI and Anthropic feel the same way you do?
JACOB COXON: (00:22:23 – 00:23:01) There’s a mix. I think I’m probably more on the, libertarian side. I think some will be scared of climate change. Some will have been. I also thought, for example, the pandemic response ultimately was quite overblown, like near the end. I think that like it was pretty bad, but I think maybe some would disagree.
So there are definitely some employees like that, but it’s not all of them. It’s not as like, I think people imagine San Francisco as being full of liberals. But if you’re going to accuse like me and the other kind of like kids building the technology of anything, it would be like a sort of maybe narcissism or techno narcissism, feeling like you’re the center of the world, but not like a generic liberal fear. That’s not like the main thing that’s happening.
The Newspeak House Connection
PATRICK BET-DAVID: (00:23:02 – 00:23:10) Yeah, I’m glad you addressed that and you isolated that case because I noticed a lot of information came up that you were with, Newspeak.
JACOB COXON: (00:23:10 – 00:23:11) Yes.
PATRICK BET-DAVID: (00:23:11 – 00:23:20) Where you were part of this organization that Hillary Clinton was a part of and a man named Matt something. And it’s the NGOs and you’re involved in that.
JACOB COXON: (00:23:20 – 00:23:46) And this is so funny, this is so funny because at that house It was a house I lived at. I wanted to like live in London and I met a friend who was like, come stay at this house. And it was a house full of these people. They hated me. They hated me because I was like, they’re all communists.
I’m like, they were communists in that house and I did not get on with them. It was just so funny now because like I’m on this like thing and everyone’s like, Jacob stayed at this, I wanted to do math. I wanted to like live in an interesting place with London, in London. That’s why I stayed at Newspeak.
PATRICK BET-DAVID: (00:23:47 – 00:23:52) Nothing, you never met, you’ve never had meetings with these guys, you’ve never met with the socialist leaders?
JACOB COXON: (00:23:52 – 00:23:53) No.
PATRICK BET-DAVID: (00:23:53 – 00:23:56) Clinton, Sanders, these guys didn’t communicate with you, nothing?
Was the Viral Tweet Orchestrated?
JACOB COXON: (00:23:56 – 00:23:56) Never, no.
Was the Viral Tweet Orchestrated?
PATRICK BET-DAVID: (00:23:56 – 00:24:11) So even before you tweeted your tweet out, I know Bret asked you this question as well, but did you speak with anybody? Was it orchestrated that this tweet’s about to come out? Because you started your account in January of 2026.
JACOB COXON: (00:24:11 – 00:24:12) Oh yeah, yep.
PATRICK BET-DAVID: (00:24:12 – 00:24:15) So prior to that, for a young guy to not have a Twitter account, it’s kind of weird.
JACOB COXON: (00:24:15 – 00:24:16) Yeah.
PATRICK BET-DAVID: (00:24:16 – 00:24:20) Why’d you start? Why didn’t you start it earlier? Why’d you start in January and then 8 months later you post this tweet?
JACOB COXON: (00:24:20 – 00:24:54) Yeah, so a few things. One is I’m actually like, kind of ironically, I’ve been quite scared of posting online, sharing my thoughts. I think because I had this assumption that, that my people might pile on or it’s like, I don’t like engaging with the immediate criticism of the thoughts. I prefer to like discuss them in private with the people that can make changes.
So this was kind of the first time I’ve ever expressed myself publicly. And that’s one of the reasons why I was so crazy is like, I always had this fear that if I posted, everyone would pile on. And that is exactly what happened, which is quite funny. Although it wasn’t as bad as I thought.
PATRICK BET-DAVID: (00:24:54 – 00:24:56) Do you regret the tweet or are you glad you tweeted it?
JACOB COXON: (00:24:56 – 00:24:57) I’m glad I tweeted it.
PATRICK BET-DAVID: (00:24:57 – 00:24:58) You’re glad you tweeted it? Yeah.
JACOB COXON: (00:24:58 – 00:25:05) And I’m happy to talk more about the coordination because there was some planning. This wasn’t just, I didn’t just like quit and then type out this tweet and rage quit. There were a few steps to it.
PATRICK BET-DAVID: (00:25:06 – 00:25:06) Can you walk us through it?
JACOB COXON: (00:25:06 – 00:25:37) Yeah, of course. So when I decided to quit, first I was making the AI smarter, right? And then I switched to safety for a week. I was like, well, I’m really scared of how things are going. I’ll move to the safety side, try and make it safe. And then while I was at safety, I spoke to more of the Anthropic leadership.
I spoke to more of the safety researchers. And I became convinced that really there was no good plan. I was part of something that was going to go badly. So I was like, “I’m going to quit.” And then I was thinking about quitting on the same day. But I don’t know. Do you remember Leopold? Leopold Aschenbrenner?
PATRICK BET-DAVID: (00:25:37 – 00:25:41) Absolutely. 436% the first 6 months. And, yes, I do.
JACOB COXON: (00:25:41 – 00:25:46) Yeah, he’s a dude. And he left OpenAI. And then he wrote this manifesto, Situational Awareness.
PATRICK BET-DAVID: (00:25:46 – 00:25:46) Yes.
JACOB COXON: (00:25:47 – 00:26:07) So he was strategic when he left. He was like, I’m not just going to leave OpenAI. I’m going to like use leaving to make— because he was fired for like leaking confidential information or whatever. He used it to like spread awareness. Like a lot of people in government read his manifesto. A lot of people, maybe you read some of Situational Awareness. So obviously, yeah.
PATRICK BET-DAVID: (00:26:07 – 00:26:08) Yeah.
JACOB COXON: (00:26:08 – 00:26:08) So that was—
PATRICK BET-DAVID: (00:26:09 – 00:26:11) He grew to what, $45 billion?
JACOB COXON: (00:26:11 – 00:26:12) Oh yeah. He raised a massive—
PATRICK BET-DAVID: (00:26:12 – 00:26:13) Griffin came in, bought.
JACOB COXON: (00:26:13 – 00:26:14) Exactly.
PATRICK BET-DAVID: (00:26:14 – 00:26:18) A chunk of it at a discounted price. And, yeah, I followed the story closely.
JACOB COXON: (00:26:18 – 00:27:27) Here we go. Yeah, exactly. This manifesto. And he wrote that when he left. And I thought, well, you’ve got to be strategic when you leave. I’m not just going to— I do want to express myself. I’m going to like be strategic, make my voice heard, have an impact. I did not expect it to be this.
I thought it would be a lot smaller than Leopold’s. I was just going to do my best. But I gotta connect at Wall Street Journal. I was like, “I’m going to make a, Wall Street Journal published this.” I think we can probably find the article. Yeah, this. So, oh, for starters, I also, I delayed my quitting by a week to have time to prepare for this.
So it wasn’t just the same day. I was like, I’m going to prepare. I’m going to write my thoughts properly. I’m going to quit. Spoke to the Wall Street Journal. I wrote this Twitter thread. But to be clear, I think people imagine this crazy coordinated psyop. And I had some help. But I wasn’t being like— the people were telling me stuff like, I say people, this is, my friends were telling me stuff like, “Include a picture in your tweet because then it will go viral.”
They said, tweets go viral if there’s a picture. But I was like, I don’t want to put a picture. It’s a serious resignation tweet. I’m not going to like put a picture. I got a few friends to retweet it. Yeah, it goes—
PATRICK BET-DAVID: (00:27:27 – 00:27:37) The actual details, you have some very high, It was, it seemed it was some very influential people tweeted it fairly quickly the moment it happened.
JACOB COXON: (00:27:37 – 00:27:45) So they’re pretty influential, these guys are. So it was people like Nathan Calvin and Sneha Revanur. They’re at this organization called Encode AI.
PATRICK BET-DAVID: (00:27:45 – 00:27:46) Within the space, they’re influential.
JACOB COXON: (00:27:46 – 00:27:48) Oh yeah, within the space, they’re definitely influential.
PATRICK BET-DAVID: (00:27:48 – 00:27:51) They’re definitely influential. The average person may not know who they are, but within your space, they know who they are.
JACOB COXON: (00:27:51 – 00:28:21) They’re very influential, yeah. And they’re also friends with many of us. Like I’m friends with Sneha, we’ve been hanging out before this resignation. There’s like, she’s friends with a lot of lab employees. It’s also, yeah, it’s not crazy that when I wanted to leave, so the journalist was interested because this was the first ever Anthropic safety resignation to go kind of big in public.
There’d been a few, but they were small. This was a big one. And obviously the people that are most interested in like helping spread the message are people who are, who want to slow down AI.
PATRICK BET-DAVID: (00:28:22 – 00:28:29) So who else were the people that Was anybody that played a role in you writing that, did any politician know you were going to tweet that?
JACOB COXON: (00:28:29 – 00:28:29) No.
PATRICK BET-DAVID: (00:28:29 – 00:28:30) So nothing political?
JACOB COXON: (00:28:30 – 00:28:31) Nothing political.
PATRICK BET-DAVID: (00:28:31 – 00:28:36) All these people that are friends that are working at Anthropic and OpenAI or outside of Anthropic, OpenAI?
JACOB COXON: (00:28:36 – 00:28:38) They were all outside of Anthropic and OpenAI, yeah.
PATRICK BET-DAVID: (00:28:38 – 00:28:39) None of them are working at the company?
JACOB COXON: (00:28:39 – 00:28:40) No.
PATRICK BET-DAVID: (00:28:40 – 00:28:42) So no one at the company knew you were about to resign?
JACOB COXON: (00:28:42 – 00:28:47) They knew I was going to leave. They did not know I was going to like make such a bold Twitter thread.
Telling Anthropic’s Head of Pre-Training He Was Quitting
PATRICK BET-DAVID: (00:28:47 – 00:28:53) And how was that conversation that they knew you were going to leave? Were you like, “Guys, I can’t do this anymore. I’m going to be out in a few weeks.” Was it kind of like sharing your frustration?
JACOB COXON: (00:28:54 – 00:29:24) Exactly. I was talking, but the thing you have to remember about Anthropic is such a weird place. Like these guys are scared of what they’re building in a way that I think repulses people when they see it. They’re like, this is weird. Why are these guys building it? And they’re so scared, but it’s real.
They’re really scared. So honestly what happened is I said to my, I said to the head of pre-training, I said, “I’m going to quit. I think we’re going to fast. I think it’s reckless. I think we’re not going to get international coordination.”
PATRICK BET-DAVID: (00:29:25 – 00:29:26) You’re saying this to who?
JACOB COXON: (00:29:26 – 00:29:31) To the head of, this is the head of like the whole of the pre-training organization, which is quite a big organization.
PATRICK BET-DAVID: (00:29:32 – 00:29:34) Is that a heavyweight person within a company?
JACOB COXON: (00:29:34 – 00:29:35) Heavyweight, yep.
PATRICK BET-DAVID: (00:29:35 – 00:29:37) So somebody that works directly with Dario?
JACOB COXON: (00:29:37 – 00:29:39) Exactly, someone that would speak to Dario a lot.
PATRICK BET-DAVID: (00:29:39 – 00:29:40) So you’re telling them this?
JACOB COXON: (00:29:40 – 00:29:42) Yeah, I’m telling him this. I’m telling him it’s reckless.
PATRICK BET-DAVID: (00:29:42 – 00:29:44) And what’s he— is this a public name or—
JACOB COXON: (00:29:44 – 00:29:45) I, yeah, Nick Joseph, he’s called.
PATRICK BET-DAVID: (00:29:45 – 00:29:48) So you’re telling him this and what’s his reaction to you?
JACOB COXON: (00:29:48 – 00:30:20) Yeah, so his reaction is composed because he’s dealing with these worries the whole time. He tells me, “I think this is a bad decision. I think you will have more impact if you stay and work from the inside.” And he said, you’re not going to be able to, it’s unlikely you’ll be able to change people’s opinions on the outside.
I mean, what he said, something was like, “Do you want to go from being a, like a good AI technical researcher to being just a run-of-the-mill commentator?” He said, “It seems like a bad strategic decision.”
PATRICK BET-DAVID: (00:30:20 – 00:30:21) This sounds like a good guy.
JACOB COXON: (00:30:22 – 00:30:22) It’s a good guy, yeah.
PATRICK BET-DAVID: (00:30:22 – 00:30:23) This is not a bad guy.
JACOB COXON: (00:30:23 – 00:30:27) This is a great guy. And he’s trying to give you career advice. Okay.
PATRICK BET-DAVID: (00:30:27 – 00:30:31) And what got you to say, I disagree, I’m going to leave, and I think I can impact—
JACOB COXON: (00:30:31 – 00:30:53) And I said, well, I didn’t say that. What I said is, you’re probably right. It sounded like he was right. I just didn’t want to be part of it anymore. I thought when I left, it was an expression of resignation of kind of disempowerment. I thought, I don’t want to be part of this race.
I’m going to leave, do something else. It might actually be worse. It wasn’t really a rational decision. It was pretty emotional.
PATRICK BET-DAVID: (00:30:54 – 00:31:08) Got it. And these other friends of yours that you’re talking to about the tweet, are these players that are running things and they’re heavyweights, or are some of them within the space of AI but maybe they’re not known names?
JACOB COXON: (00:31:08 – 00:31:58) Yeah, there’s— there are a few players and then just a few friends. And it’s so funny, again, people imagine this as being a— I think because of how crazy it went, they think it must have been this very careful thing. Really, I had a few. I had maybe 3 or 4 core names to do the retweeting initially.
And then after I sent it, right, I sent it and then I sent the link to a few friends and said, post this in your group chats, share this in your group chats, like spread it around. But honestly, by the time— like, I sent it to one guy and he was like, okay, I’ll share it, let me write a careful quote tweet about it.
Let me like carefully write a quote tweet. And by the time he’d finished writing the quote tweet, it had already gone to like 20K likes. Like it already exploded. So the whole thing just went so crazy. And it like, this can’t be planned, right? Like imagine if there’s no PR firm that can make, that can like hack the algorithm like this.
PATRICK BET-DAVID: (00:31:59 – 00:32:09) Yeah. And when you did this, who was it from Anthropic Research or somebody supported, which, and Wang, right, is who it was. I think Evan Hubinger was the main Evan Hubinger was one of them.
JACOB COXON: (00:32:10 – 00:32:10) Yeah.
PATRICK BET-DAVID: (00:32:10 – 00:32:13) That he, what did he say? That he agreed with you?
JACOB COXON: (00:32:13 – 00:32:36) He said he thought everything I said was correct. Because what I said is I didn’t actually say my own opinion mostly. Mostly I said other people at the company believe it and continue to work there. If we scroll, it’s like to tweet 3 maybe. I said, “The people building this technology earnestly believe it could kill us all.”
And he backed me up. He said, “This is correct. I do believe that this technology could kill us all.”
Elon Musk and the “Psyop” Accusation
PATRICK BET-DAVID: (00:32:36 – 00:32:45) They ran an analysis of all the replies, of who agreed, who disagreed. 43% agreed, 40% disagreed, 18% were mixed.
JACOB COXON: (00:32:45 – 00:32:46) Yep.
PATRICK BET-DAVID: (00:32:46 – 00:32:57) And so it wasn’t like people weren’t agreeing with you. They were agreeing with you. I even think Elon retweeted and says, “I agree with this, I said this a long time ago,” something like that.
JACOB COXON: (00:32:58 – 00:33:00) Yeah, exactly right. And he has— he’s always said that.
PATRICK BET-DAVID: (00:33:00 – 00:33:01) Did he reach out to you? Have you spoken to him?
JACOB COXON: (00:33:01 – 00:33:11) I mean, not like, not a length, no. He’s clearly been thinking about this for a really long time. That’s the main.
PATRICK BET-DAVID: (00:33:12 – 00:33:13) So you guys did have a conversation together?
JACOB COXON: (00:33:14 – 00:33:15) Yeah, we had a chat. Yeah.
PATRICK BET-DAVID: (00:33:16 – 00:33:17) Okay, this is after your tweet?
JACOB COXON: (00:33:17 – 00:33:18) Yeah, after my tweet.
PATRICK BET-DAVID: (00:33:19 – 00:33:23) Was it a phone call? Was it a conversation? Or is it a face-to-face?
JACOB COXON: (00:33:24 – 00:33:24) Just a DM.
PATRICK BET-DAVID: (00:33:24 – 00:33:27) Just, okay, so he just DM’d you and, hey, I agree with you know.
JACOB COXON: (00:33:28 – 00:33:29) I DM’d him.
PATRICK BET-DAVID: (00:33:29 – 00:33:31) Okay, and then he responded back to you?
JACOB COXON: (00:33:31 – 00:33:45) Yeah, just a little response. And mostly I was saying like, he— I just DM’d him saying, because maybe we saw we had a little spat on Twitter. He like said, yeah, and I thought I’m going to try and like— because I think he’s been thinking about this for like—
PATRICK BET-DAVID: (00:33:45 – 00:33:46) So did he explain to you why he said this could be a psyop?
JACOB COXON: (00:33:46 – 00:33:47) It wasn’t— no, not that much detail.
PATRICK BET-DAVID: (00:33:48 – 00:33:50) No. What do you think he meant by this could be a psyop with the original thing?
JACOB COXON: (00:33:50 – 00:33:57) I think it was quite suspicious that an account with no followers could blow up that much.
PATRICK BET-DAVID: (00:33:57 – 00:34:01) The reality is he owns X, so he can find out exactly if they were exactly driven or not.
JACOB COXON: (00:34:01 – 00:34:14) Exactly. And they weren’t. So it was just like an organic explosion based on the fact that it was scary. Yeah, this is this. I kind of regret this tweet. It was pretty stupid. But then again, I had had no sleep and I was very annoyed that I thought Elon was kind of on the side of this. So it upset me quite a lot.
PATRICK BET-DAVID: (00:34:14 – 00:34:19) Did he say he agrees and then he said psyop? It’s like it almost— there was a 50/50 with him, right?
JACOB COXON: (00:34:20 – 00:34:31) Yeah. So this, I think he was just, When I say psyop, he just thought it was— the whole thing was quite suspicious. Like the fact that I’d, the communities I’ve been in before, the fact that it went so viral, I can see why he thought it was suspicious.
PATRICK BET-DAVID: (00:34:31 – 00:34:33) You got 300,000 followers within 48 hours.
JACOB COXON: (00:34:34 – 00:34:35) I know, it was not nothing. Absolutely insane.
Leaving 7 Figures on the Table
PATRICK BET-DAVID: (00:34:35 – 00:35:25) Yeah. And so now you brought up Leopold earlier. Yeah. Leopold got the sequencing right. Got fired, story goes out, “Hey, here’s my manifesto,” when he wrote down, we talked about it earlier, and in it he says, it’s not really such and such stocks that we have to worry about. We have to put our money into this and this, and that’s the infrastructure that’s going to make a lot of money.
And for the most part, he did well. People made a lot of money. And then stories came out about how much of it was on margin and how much risk he was taking, 4 to 1, 2 to 1, 3 to 1. But his outcome was, I’m going to quit, I’m going to write this paper, It’s going to go viral.
I’m going to start a firm. I’m going to raise money. I think GitHub guys got involved at the beginning. He got some big names that got involved. What’s your financial next move? What’s your financial benefit of doing this?
JACOB COXON: (00:35:26 – 00:35:28) There is no financial benefit. So Leopold, I think—
PATRICK BET-DAVID: (00:35:28 – 00:35:30) So you have nothing new to launch in the next 3 to 6 months?
JACOB COXON: (00:35:30 – 00:35:53) I’m not going to, I’m not going to try and milk this. I’m not doing anything. I’m going to— one way I will exploit the crazy temporary fame of the situation is through having, conversations with lots of researchers I respect that previously would have made less time for me, and now I can talk more about the situation.
And there’s work I intend to do, there’s lots of work, but I’m not trying to make any more money from this.
PATRICK BET-DAVID: (00:35:53 – 00:35:55) How much money did you leave on the table by doing this?
JACOB COXON: (00:35:55 – 00:36:00) By leaving Anthropic? By leaving Anthropic, I’m not going to say the exact detail, but 7 figures.
PATRICK BET-DAVID: (00:36:00 – 00:36:01) 7 figures?
JACOB COXON: (00:36:01 – 00:36:01) Yeah.
PATRICK BET-DAVID: (00:36:02 – 00:36:09) Okay, but let’s go a little bit deeper. If you would have stayed 5 years and you would have done your part, you would have moved up potentially $50 million, $100 million?
JACOB COXON: (00:36:09 – 00:36:22) 5 years. That’s the problem. So I’m thinking about that and it’s like, it’s not going to be the same in 5 years. In 5 years, everything looks different. Okay. This is— and I think the people that are talking about this, they have these sums of money and these plans for the future, and they’re just not thinking about how crazy it should get.
PATRICK BET-DAVID: (00:36:22 – 00:36:55) To you, one, you’re not a socialist leftist, that you belong to this NGO, that you’re a pro-Bernie Sanders. You’re one, a capitalist, free market libertarian. You don’t believe in the climate change is going to end the tech though. COVID handling wasn’t right. You thought it was a little bit too much. Okay, check.
Let’s set that aside. Number two, there is not a I’m about to announce a hedge fund private equity firm, and this is why our private equity focuses on AI regulation and da. You’re not going to launch anything like that. So how are you going to make money? How are you making money right now?
JACOB COXON: (00:36:55 – 00:37:31) Luckily, I have this money from OpenAI, right? And I think I will get a job. I mean, I’m going to get a job doing some form of within the space, and I think the job will most likely be. Around predicting where AI will go, trying to share that writing with other people. And also, I really, like I said, it sounds crazy, it sounds cultish.
Things are going to change so much in the next few years. Unemployment is going to be a problem when we release, when we can automate a lot of white-collar jobs. I have enough money for a few years and I really don’t know what the world looks like. If I’m wrong, I can get a job in a few years. That’ll be great. But I don’t think I’m wrong.
PATRICK BET-DAVID: (00:37:32 – 00:37:45) Got it. So for you, if you’re hedging of you being right or wrong, if you’re right, you have money and you want to do something to make a difference the next 12, 24 months. Yeah, it sounds like that’s your timeline. You don’t think 5— you think 5 years is too late?
JACOB COXON: (00:37:45 – 00:37:53) I don’t think— I don’t necessarily think the really bad thing happens, but I think that the critical decisions, the crucial decisions about whether we slow down or not, they will happen in the next couple of years. Yep.
Why the Next 2 Years Are Critical
PATRICK BET-DAVID: (00:37:54 – 00:37:55) If we’re going to slow down or not.
JACOB COXON: (00:37:55 – 00:37:56) Exactly.
PATRICK BET-DAVID: (00:37:56 – 00:38:01) And you think the next 2 years There’s still time to slow down to prevent the madness?
JACOB COXON: (00:38:01 – 00:38:01) Yes.
PATRICK BET-DAVID: (00:38:01 – 00:38:03) Why is it specifically 2 years?
JACOB COXON: (00:38:03 – 00:38:45) It’s— if you look at the pace of the improvement, the specific technology that is scary is when it can automate its own development. So when the job of AI research gets automated, and I think all the extrapolation suggests that that could quite likely come in the next few years. This is the whole RSI thing, recursive self-improvement.
I think Roughly around that time is the crucial time where the race will start, where China will lock in and it’ll be China versus the US. Everyone will be taking it seriously. There’ll be like fear. They won’t— OpenAI and Anthropic won’t be sharing the models with the public anymore. It will be internal, government-only, full-on race.
That’s when the crucial decisions happen, like whether we go into this race or not. I think that gets decided this decade.
Why the Free Market Can’t Handle AI
PATRICK BET-DAVID: (00:38:45 – 00:38:48) You trust the government having this more than free market?
JACOB COXON: (00:38:48 – 00:38:54) I don’t trust any specific person or entity. I just think right now the free market is not going to handle it well.
PATRICK BET-DAVID: (00:38:58 – 00:39:00) And that’s why do you think they’re not going to handle it?
JACOB COXON: (00:39:00 – 00:39:02) Yeah, that’s a great question because their motivation is going to be what?
PATRICK BET-DAVID: (00:39:02 – 00:39:03) Money.
JACOB COXON: (00:39:03 – 00:39:13) I think actually if I talk about my personal motivation, right? So there are people who— I thought this was dangerous for a really long time, right? Like before I even joined. And when I joined OpenAI, I thought it was dangerous.
PATRICK BET-DAVID: (00:39:13 – 00:39:15) At what age did you think this was dangerous?
JACOB COXON: (00:39:16 – 00:39:17) 15, 16.
PATRICK BET-DAVID: (00:39:17 – 00:39:18) How old are you right now? 27?
JACOB COXON: (00:39:18 – 00:39:19) Yeah.
PATRICK BET-DAVID: (00:39:19 – 00:39:20) So 15, 16?
JACOB COXON: (00:39:20 – 00:39:21) Yeah.
PATRICK BET-DAVID: (00:39:21 – 00:39:23) So in 2015, you thought this was dangerous?
JACOB COXON: (00:39:23 – 00:41:44) Yeah, there was a book called Superintelligence by Nick Bostrom that was quite influential. I read it in school. I think like, yeah, so this image of me having worked in AI and then suddenly got scared is wrong. I think I’ve always been scared. But then it’s like, why do you join a lab in the first place?
And there are other people who didn’t join a lab, right? They were also scared. They maybe behaved more ethically. They didn’t join. But then when they say this stuff, no one listens, right? I joined the lab and I got all sorts of things that they didn’t, like money and status and some like respect for working there.
And then I quit. And that’s a selfish thing to do, to say, I’m going to work at this technology and get the status from doing this, even though I think it’s dangerous. But then it also gives you the power to make change. Does that make sense? Like this, you get power. And I think the same thing is like what Dario is doing.
So, Anthropic have been scared and they said, well, no one’s listening to us, we’re going to have to be a player, enter. And so Dario’s worked— like, they’ve worked so hard in the last— they’ve caught up to OpenAI. It’s very impressive. Anthropic have worked like absolute dogs to like overtake and become the main player.
And they did that because they’re like, we need the power. And that’s the free market, right? The free market incentivizes you to get your own power, your own wealth, so you can make change. The problem is that’s going to keep happening and people are going to keep doing this. It’s the story of OpenAI.
OpenAI did that because they were scared of DeepMind. They were like, we’re going to do it. Dario did it. He was scared of OpenAI. He was like, I want the power. Even just like me, it’s a small story, but why did I work and help? It’s for my own power. And if we let the free market determine in the next few years, we’re going to race off a cliff because everyone is looking out for their own money, their own power.
No one has the ability to sort of slow down a few years ahead. It’s this like race problem where everyone just like— and I don’t blame them, right? I don’t blame— if you’re a 22-year-old right now and you’ve done AI, I don’t blame you if you get a job at Anthropic and work there, because what else are you going to do if you’re like— how else are you going to try and help?
So everyone is following their own incentives. And that’s why we need— in this case, we need bigger regulation because, yeah, in any other technology, I’m happy with this. I think innovation from competition is great everywhere except if the technology is this level of compounding risky. That’s when it breaks.
From Math Olympiad to OpenAI
PATRICK BET-DAVID: (00:41:44 – 00:41:51) So 2015, 2016, you read the book Superintelligence and then you— at what age did you start working at OpenAI?
JACOB COXON: (00:41:51 – 00:41:52) 23.
PATRICK BET-DAVID: (00:41:52 – 00:41:58) And what did you do prior to that? Because I knew you were a big math— maybe walk through who you were as a math phenom and how you came up.
JACOB COXON: (00:41:58 – 00:42:05) Sure. So I did math competitions in school. I was very into math when I was a kid. I mostly just did math and read books. Then when I was in college, I guess—
PATRICK BET-DAVID: (00:42:06 – 00:42:09) You won. You did pretty well on 2 of the biggest.
JACOB COXON: (00:42:09 – 00:42:11) I did the International Math Olympiad twice. Yeah.
PATRICK BET-DAVID: (00:42:11 – 00:42:13) And you were second in the world and third.
JACOB COXON: (00:42:13 – 00:42:16) No. Okay. So I think you misunderstood. I got a silver medal.
PATRICK BET-DAVID: (00:42:16 – 00:42:16) Silver medal.
JACOB COXON: (00:42:16 – 00:42:27) Yeah. Doesn’t— a lot of people get silver medals. It doesn’t make you the second in the world. I wasn’t anywhere near the best. I was in sort of a— I think silver medal is top We can look it up. I think it’s the top 20% or something.
PATRICK BET-DAVID: (00:42:27 – 00:42:29) It’s a, in the math arena.
JACOB COXON: (00:42:29 – 00:42:34) Yeah, I think the kind of achievement here is getting on the national team because that’s being selected for 6 people to do the team.
PATRICK BET-DAVID: (00:42:35 – 00:42:56) See, answers like that you give tells me your integrity. You corrected me to say you’re not as smart as I thought you were. Yeah, I mean, you realize that’s also when the audience is watching. I understand that, but it’s, but say 99% don’t check it, you just did an integrity check on yourself, and I respect that.
I value that. I hope the audience does as well. But please continue. So you’re a math guy.
JACOB COXON: (00:42:56 – 00:44:34) Yep. And then I went to college and studied math. I kind of, I think I wanted to do math research, which was, ’cause I, yeah, I always loved math. But unfortunately, I think it was a mix of many things. I kind of realized that Pure math research was not for me. It was partly related to AI.
I saw that like AI was going to be this big thing that was going to eat a lot of stuff. Still felt quite far out. So I’ve been following it since, I don’t know, 2020 was GPT-3. GPT-3 was this like quite big, this happened during the pandemic, which is also my third year of college.
So I was sort of in the pandemic with a friend. We looked at the GPT-3 paper and we were like, yep, it’s happening. This is a big deal. This is where things are going. But I didn’t know how to break into AI. So I didn’t actually like I had no AI background. I studied math.
So I worked with a friend at a power trading startup. We predicted energy prices, which is kind of, it was AI related. We predicted the cost of power. I did that for a bit. And then I took a, there was some personal issues and I took a break for about a year. I was kind of out of it.
And a little bit, I was living in Prague for a while and then Scotland. And then finally I got a call from a friend who was, “Jacob, what are you doing? AI is taking over. You’re like, you can do math, you can learn AI. Come and join OpenAI’s residency program.” They have a program which is a 6-month training program for people that are math guys or physics guys to teach them the ropes of AI and make them full-fledged researchers.
So I sort of said, “Yeah, that sounds great. I’ll do that.” And that’s when I joined OpenAI.
PATRICK BET-DAVID: (00:44:34 – 00:44:37) So you went in and then you got in at 6 months and then you were there for how long? 3 years?
JACOB COXON: (00:44:38 – 00:44:38) 3 years, yeah.
PATRICK BET-DAVID: (00:44:38 – 00:44:38) 3 years.
JACOB COXON: (00:44:39 – 00:44:50) Which is, yeah. I was like pretty junior, but for context, the company went from 300 people when I joined to about, 7,000, 8,000 now, I think. So it was like a pretty explosive phase.
Sam Altman vs. Dario Amodei
PATRICK BET-DAVID: (00:44:50 – 00:44:54) And was Sam Altman pretty accessible when it was at 300? Were you around him a lot?
JACOB COXON: (00:44:54 – 00:44:54) He’s around.
PATRICK BET-DAVID: (00:44:54 – 00:44:55) Yeah.
JACOB COXON: (00:44:55 – 00:44:55) You see him?
PATRICK BET-DAVID: (00:44:55 – 00:45:03) You see him? Yeah. You’ve had conversations with him? Okay. And how different— did you get that same experience with Dario at Anthropic?
JACOB COXON: (00:45:03 – 00:45:04) Not in the same way. No.
PATRICK BET-DAVID: (00:45:04 – 00:45:08) Okay. So what is your opinion of what Sam Altman is doing versus what Dario is doing?
JACOB COXON: (00:45:08 – 00:45:08) Yeah.
PATRICK BET-DAVID: (00:45:08 – 00:45:09) How different are they?
JACOB COXON: (00:45:10 – 00:46:20) They’re completely different. Yeah, I’m going to— so I think Sam is— so Dario is very honest. I think that’s the first— you can see, everyone can see that he’s very open and honest and shares a lot of stuff with everyone. I think he’s like, he’s always honest. And he’s obviously like an amazing researcher.
And I’m always impressed by what he’s accomplished building Anthropic. Sam is like— has a reputation for dishonesty. So Sam has a reputation for being a lot more of a business guy, of sort of like pleasing one people, of always being quite vague. I think the main difference I noticed between Dario’s communication and Sam’s communication is Sam is very vague about the future.
He says things will be good in this way and maybe there’ll be dangers, but we can muddle through, like a vague thing. Whereas Dario is like, 20% chance this happens, 80% chance this happens. This is what it looks like. And that’s a lot more like rough to hear because it comes across as quite abrasive.
It’s like tough to hear the details. But Sam, it’s like it feels like he’s making everyone promises and he could be saying vague stuff to everyone. Everyone feels kind of happy, but he’s actually not really saying anything. That’s kind of the main difference, I’d say, in terms of how they communicate with the company.
PATRICK BET-DAVID: (00:46:20 – 00:46:22) What do you think is the end-all be-all for each one of them?
JACOB COXON: (00:46:24 – 00:46:48) Yeah, again, this is armchair psychology. I’m not good friends with them. This is just from watching. I think Dario wants to save the world, as it were. He’s got like a— he thinks he’s the— him and his company are the ones to save everyone. I think Sam has a lot of motivations, and it’s quite messy and chaotic, and that’s reflected in how messy and chaotic the company is. But again, this is armchair speculation. I don’t— yeah.
PATRICK BET-DAVID: (00:46:50 – 00:46:52) So if one is save, the other one is control.
JACOB COXON: (00:46:52 – 00:46:58) To be clear, when Dario is saving the world, I think saving the world and taking over the world overlap quite heavily.
PATRICK BET-DAVID: (00:46:59 – 00:47:00) You could flirt with that.
JACOB COXON: (00:47:01 – 00:47:15) You could have that conversation. If you think you’re the safe one, right, and China’s building unsafe, or you think the US government’s being silly and you have a very powerful technology, it’s not off the table to consider taking actions against the US government. And I think if anyone would do that, it’s Anthropic.
Inside the Cultures of OpenAI and Anthropic
PATRICK BET-DAVID: (00:47:16 – 00:47:21) In the inside organizations, OpenAI and Anthropic, how different are the cultures?
JACOB COXON: (00:47:21 – 00:47:21) Very different.
PATRICK BET-DAVID: (00:47:22 – 00:47:22) Tell me more.
JACOB COXON: (00:47:22 – 00:49:09) So one thing is OpenAI may have changed. I left like 4 months ago and I think they’ve started taking stuff more seriously. So Anthropic has 7 co-founders and none of them have left. They’ve all stayed, which is very impressive for an AI company because at OpenAI, everyone’s left. They all go. Almost all the other companies, everyone leaves the whole time.
Anthropic retains them. And that’s because I think Anthropic has this They are very clear about what their mission is, and they’re very clear about how they’re going to achieve it. And this makes them very dangerous. It’s a very aligned institution. To put it another way, at OpenAI, it felt like people might prioritize their career, their like individual success over the mission of the company, which is a very natural thing to do.
Like everyone does this. It’s like, I want to do a project that’s going to make me have an impact. That’s going to be interesting for me. That’s good for my career. At Anthropic, it felt a lot more like, well, for starters, the employees call themselves like ants, right? They’re the ants in the anthill.
Everyone’s like a worker ant trying to make the Anthropic, the mission go well. People are very willing to sacrifice themselves for the good of the mission. They’ll take on work that’s less interesting, that’s less kind of big and bold and exciting. It’s less flashy, and they’ll do it at a hit to their own kind of interests for the good of the company.
And that’s because every decision comes back down to like, is this going to get us to superintelligence faster? You’ve seen maybe Sora, you saw the video platform that OpenAI made. Anthropic would never make Sora because it’s not on the core path to artificial superintelligence. OpenAI is distracted and messy. And one guy’s like, I want to do Sora or I want to build like a little device.
And they let them do it because it’s not like the same focus. And Anthropic, that would never happen. They’re just completely kind of—
PATRICK BET-DAVID: (00:49:09 – 00:49:11) Who would shut it down? The 7 co-founders? They’d be like, this is not—
JACOB COXON: (00:49:11 – 00:49:16) They’d all agree. They’d all be like, this video, it’s not the right direction for heading straight to superintelligence.
PATRICK BET-DAVID: (00:49:16 – 00:49:19) Which one is harder to work at? More schedule, more hours?
JACOB COXON: (00:49:20 – 00:49:41) I think it depends on what team you’re on, probably. I think when I was there, I think it also depends on the stage of your career. I think when I was getting into, the work was also so different because now compared to 3 years ago, you’re running these agents. You weren’t doing that 3 years ago.
So when I joined, I was writing code by hand, right? It was entirely hand programming. And by the time I left, no one really writes code by hand anymore. Your agents write the code.
PATRICK BET-DAVID: (00:49:42 – 00:49:43) Your agents write the code.
JACOB COXON: (00:49:43 – 00:50:25) Yeah. I can talk more about that. It’s kind of interesting about the details of how much AI runs the company. To give some flavor, when I quit, when I resigned, I was like, what’s the procedure for resigning? What do I sign? And my manager was like, “I don’t know, ask Claude.” So I asked the AI.
So I asked the AI, “How do I resign?” And the AI told me the flow. No one knows how to resign. You have to ask the AI to do it. Because the company is run by, the processes are run by AI, right? Everyone asks, and it’s great, it works. But that’s the extent to which AI is part of the whole company.
It’s like any transaction is mediated by asking the AI to do it. People’s meetings are recorded by AI, summarized. You have AIs posting comments on things. You write an essay and AI will respond. It’s like pretty crazy.
Dario’s Cult of Vision and the Rivalry with Sam Altman
PATRICK BET-DAVID: (00:50:26 – 00:50:46) Interesting. Is Dario and Sam Altman, on the way their people look at them, between fear, respected, liked, loved, how would you rank them? Like if you were to say people love Dario, they respect them.
JACOB COXON: (00:50:46 – 00:50:49) I would say at Anthropic, people love and respect Dario.
PATRICK BET-DAVID: (00:50:49 – 00:50:49) Really?
JACOB COXON: (00:50:49 – 00:50:58) Yes. In a way that’s not the case at OpenAI. Dario has very much like a cult of personality and a cult of like a cult of vision, as it were, at Anthropic.
PATRICK BET-DAVID: (00:50:58 – 00:51:00) And then at Sam Altman, he’s more fear.
JACOB COXON: (00:51:01 – 00:51:08) I don’t know. I think he’s obviously respected and obviously loved by a lot of people, but not in the same— Dario’s thing is different.
PATRICK BET-DAVID: (00:51:11 – 00:51:19) Yeah. And in the company, no matter where you work at, like right now, yesterday Yankees played the Red Sox and, they hate each other. They’re enemies.
JACOB COXON: (00:51:19 – 00:51:20) Yeah.
PATRICK BET-DAVID: (00:51:20 – 00:51:40) Right. In sports, we’re talking about baseball earlier. In politics, Hillary Clinton, Donald Trump, in basketball, Michael Jordan, LeBron James, he was a— who was the enemy to Dario and who did Sam Altman see as his enemy?
JACOB COXON: (00:51:40 – 00:52:10) Yeah, that’s a great question. You have to look at the history of the companies. So when OpenAI was founded, the big enemy was DeepMind, Google DeepMind, and Demis Hassabis. And they were worried about Google kind of becoming dictators. They were worried about Google taking over. So that was the enemy. And then OpenAI was founded.
Now it’s like, I’d say there’s probably more— like, Dario’s quite— has a lot of animosity towards Sam Altman, and the reverse is not necessarily true to the same extent.
PATRICK BET-DAVID: (00:52:10 – 00:52:12) Meaning Sam Altman doesn’t have animosity towards Dario?
JACOB COXON: (00:52:12 – 00:52:25) I think less, because I think, like I said, Dario is very honest and I think perceives Sam as having betrayed him to a certain extent, and that’s a kind of a pretty permanent blot on their relationship.
PATRICK BET-DAVID: (00:52:25 – 00:52:26) How did he betray him?
JACOB COXON: (00:52:26 – 00:52:29) I really don’t know the details, but that’s my understanding. Again, this is from a—
PATRICK BET-DAVID: (00:52:29 – 00:52:30) That’s the energy you see.
JACOB COXON: (00:52:30 – 00:52:31) That’s the energy.
PATRICK BET-DAVID: (00:52:31 – 00:52:32) But that’s not public.
JACOB COXON: (00:52:32 – 00:52:33) I think that’s pretty public.
PATRICK BET-DAVID: (00:52:34 – 00:52:37) Yeah, on the betrayal between— did they do business together at one point?
JACOB COXON: (00:52:37 – 00:52:41) Yeah, I mean, Dario is a like big figure at OpenAI before he quit to found—
PATRICK BET-DAVID: (00:52:41 – 00:53:04) Oh, that’s right. So that history is from them leaving. I remember that one, that whole exchange took place. So for him, Dario’s enemy was, I’m going to leave and build something faster than that. Has he ever expressed, the reason for leaving? Was it because he did not want Sam to have the control of this kind of a technology?
Yes, that’s what it was, basically. I don’t think it’s good for the world for Sam to control this kind of—
JACOB COXON: (00:53:04 – 00:53:08) Maybe not Sam specifically, but like definitely a large component Sam, and then the rest of OpenAI.
PATRICK BET-DAVID: (00:53:09 – 00:53:49) And who do you think Sam looks at as enemy today? Is it a politician? Is it a competitor?
JACOB COXON: (00:53:54 – 00:53:55) I really don’t know. Okay.
The White House AI Meeting
PATRICK BET-DAVID: (00:53:56 – 00:54:20) And what role in something like this right now— I mean, you saw that they had a meeting at AI, at the White House with everybody being there. The list was all the heavyweights were there, right? And part of that was from your tweet that prompted these types of conversations being had now. What role does, Trump play here?
Does Dario trust Trump? Does Dario look at Trump as an ally? Does Altman see Trump as an ally? How do they view that?
JACOB COXON: (00:54:21 – 00:54:40) I think everything’s changed so much now that I really don’t know the— I have like quite a very few things that I’m like confident on. I think right now, honestly, everyone is just quite scared of how fast things are going. And they’re looking for ways to come together. I do think this particular accord is, it’s a good thing. Like what they wrote makes a lot of sense.
PATRICK BET-DAVID: (00:54:42 – 00:54:45) By the way, at the AI meeting at the White House, was Dario and Sam Altman present?
JACOB COXON: (00:54:46 – 00:54:50) I don’t think Sam Altman was. I think he was at OpenAI Dev Day, but Greg Brockman was there.
PATRICK BET-DAVID: (00:54:51 – 00:54:53) And he’s one of the 7? No.
JACOB COXON: (00:54:53 – 00:54:56) Greg Brockman is a representative of OpenAI. Got it.
PATRICK BET-DAVID: (00:54:57 – 00:54:59) But was anybody there from Anthropic?
JACOB COXON: (00:54:59 – 00:55:00) Yeah, Dario was there.
PATRICK BET-DAVID: (00:55:00 – 00:55:09) Dario was there? Yeah. Okay. When’s the last time Dario and Altman have been seen together? Is that like the thing that happens, or is that—
JACOB COXON: (00:55:09 – 00:55:13) I don’t know. Have you seen all these videos flying around? No. They’re these funny music videos flying around now that like—
PATRICK BET-DAVID: (00:55:13 – 00:55:18) Oh yeah, the dancing that they’re doing. Yeah. I like this, but that’s funny that they did something together.
JACOB COXON: (00:55:18 – 00:55:20) Yeah, and that’s as close as it’s going to get there.
PATRICK BET-DAVID: (00:55:20 – 00:55:25) So one of the things you said in the, Where do you see Dario here? See if we can spot him.
JACOB COXON: (00:55:25 – 00:55:26) I think he’s far up.
PATRICK BET-DAVID: (00:55:27 – 00:55:32) Can you find Dario? Where is Waldo? Where is Dario? I don’t see him in this.
JACOB COXON: (00:55:32 – 00:55:33) No, I don’t think he’s there.
PATRICK BET-DAVID: (00:55:33 – 00:55:34) I don’t see him in this picture here.
JACOB COXON: (00:55:34 – 00:55:35) I think he’s on the far left.
Building Their Own Replacement
PATRICK BET-DAVID: (00:55:35 – 00:55:51) So, go through it. So while you were there at Anthropic and you guys are having these types of conversations, how many other people out there sitting there saying, guys, we are, building our replacement? Yeah, we are building our replacement. How many people felt they are building their replacement?
JACOB COXON: (00:55:51 – 00:55:51) Everyone.
PATRICK BET-DAVID: (00:55:52 – 00:55:52) Everyone.
JACOB COXON: (00:55:52 – 00:55:54) Yeah, that’s just a fact. Everyone knows it.
PATRICK BET-DAVID: (00:55:54 – 00:55:58) But is that a fact, or do the agents still need to be controlled by somebody like you?
JACOB COXON: (00:55:58 – 00:56:12) For now they do. For now they definitely need control. And for— it’s not clear exactly when they lose— we don’t need the human anymore, but everyone anticipates it. They’re all like, “We’re going to be— at some point we won’t have a job.”
PATRICK BET-DAVID: (00:56:12 – 00:56:24) Does Dario— like, let’s just say that’s something that is happening, and I’m an employee, and I know this is going to be happening again. It’s like a truth, right? Was there like a life insurance policy? “Guys, if this happens, here’s what we’re planning on doing so you don’t have to worry about it.”
JACOB COXON: (00:56:24 – 00:56:52) Yeah, so Anthropic, I think in one of Dario’s essays, I think it might be Machines of Loving Grace, this essay, he has a section where he talks about the fact that Anthropic will need to make commitments to its employees for if they get replaced. And I think people implicitly had the knowledge that if Anthropic employees get replaced, due to progress, they could expect to be taken care of. I think people expect that they will stay and kind of be looked after.
PATRICK BET-DAVID: (00:56:53 – 00:56:56) Yeah. So, but OpenAI, you don’t remember having anything like—
JACOB COXON: (00:56:56 – 00:56:59) They’ve not said anything publicly. I think that maybe some references to it, but nothing.
PATRICK BET-DAVID: (00:56:59 – 00:57:05) I think that’s a pretty good idea. If you know your people are afraid of something like that, hey, if this happens, we would do this. If that happens, we would do this.
JACOB COXON: (00:57:05 – 00:57:13) Maybe. But it also sets a bad— like, what about the people that don’t work there? What about all the people that are going to get replaced to other companies? It kind of doesn’t look great if you’re looking at your own employees.
PATRICK BET-DAVID: (00:57:13 – 00:57:13) Yeah.
JACOB COXON: (00:57:13 – 00:57:19) But like at these other companies, people are, losing their jobs and they don’t have the same assurances as someone that works at—
The Weirdest DMs and Political Invitations
PATRICK BET-DAVID: (00:57:19 – 00:57:23) Jacob, who’s the weirdest person that slid in your DM after that tweet?
JACOB COXON: (00:57:25 – 00:57:34) It’s been some crazy stuff. I had a lot of, I had a lot of quite aggressive gay men DMing me.
PATRICK BET-DAVID: (00:57:34 – 00:57:34) Gay men?
JACOB COXON: (00:57:35 – 00:57:35) Yeah.
PATRICK BET-DAVID: (00:57:35 – 00:57:37) Wow. Did they think you’re gay?
JACOB COXON: (00:57:37 – 00:57:38) I don’t know, man. They wanted to find out.
PATRICK BET-DAVID: (00:57:38 – 00:57:40) Yeah. And for the record, you’re not gay.
JACOB COXON: (00:57:40 – 00:57:41) For the record, I’m not gay.
PATRICK BET-DAVID: (00:57:41 – 00:57:41) Yeah.
JACOB COXON: (00:57:41 – 00:57:43) Gay men, a lot of them in my DMs.
PATRICK BET-DAVID: (00:57:43 – 00:57:45) Yeah, but like powerful gay men?
JACOB COXON: (00:57:45 – 00:57:45) No.
PATRICK BET-DAVID: (00:57:46 – 00:57:47) Okay, you just gay men, period?
JACOB COXON: (00:57:47 – 00:57:48) Just dudes in my DMs.
PATRICK BET-DAVID: (00:57:48 – 00:57:51) Yeah, listen, I mean, if there’s powerful gay men, you gotta entertain.
JACOB COXON: (00:57:51 – 00:57:52) You gotta reply.
PATRICK BET-DAVID: (00:57:52 – 00:58:00) Yeah, sorry, you gotta reply and say at least— that is the group— this is a billionaire gay man. “Hey sir, respectfully, I’m straight, but thank you for your interest.”
JACOB COXON: (00:58:00 – 00:58:03) I mean, you could— yeah, you don’t have to say any heavy—
PATRICK BET-DAVID: (00:58:03 – 00:58:10) Any heavyweight, politicians, world leaders, billionaires, any names like that?
JACOB COXON: (00:58:11 – 00:58:16) I think so. There were like, there were a few names. The whole thing is like, I mean, lost in the inbox. I don’t want to say—
PATRICK BET-DAVID: (00:58:16 – 00:58:18) You must have tens of thousands of DMs.
JACOB COXON: (00:58:18 – 00:58:36) It’s kind of nuts. I just decide what to respond to. But, I don’t want to say specifics. I think a lot of politicians have reached out and I haven’t responded to any of them yet. But a lot of people for things like, talking to staffers— I think Bernie Sanders’s people reached out. I haven’t responded to that yet.
I had an invitation to go to a, I think, a rally with Steve Bannon and Bernie Sanders. That I chose not to go to.
PATRICK BET-DAVID: (00:58:36 – 00:58:38) Steve Bannon and Bernie Sanders?
JACOB COXON: (00:58:38 – 00:58:38) Yeah.
PATRICK BET-DAVID: (00:58:38 – 00:58:40) They’re aligned on this, right? They’re aligned on the same page.
JACOB COXON: (00:58:40 – 00:58:41) Yeah, exactly.
PATRICK BET-DAVID: (00:58:41 – 00:58:43) That is a weird rally. Weird.
JACOB COXON: (00:58:43 – 00:58:44) I know, weird. And I say rally was like—
PATRICK BET-DAVID: (00:58:44 – 00:58:45) Are you thinking about it or—
JACOB COXON: (00:58:45 – 00:58:56) I was thinking about it, definitely. But the thing is, interesting. It just wouldn’t look good on— I don’t want to associate. I don’t want to be political, right? I’m also— I’m from the UK. I feel like I don’t want to just jump into US politics. That’s just like an odd thing to do.
PATRICK BET-DAVID: (00:58:57 – 00:59:08) Yeah, that’s— and by the way, is it true that the story at OpenAI, if you’re going through conflict, they bring dolls and stuff like that? What is that? Because I saw that story last week. I fully believe.
JACOB COXON: (00:59:08 – 00:59:08) Oh, Anthropic.
PATRICK BET-DAVID: (00:59:08 – 00:59:10) Is this true? Like a stuffed animal advisory council?
JACOB COXON: (00:59:10 – 00:59:15) I mean, I don’t think this is happening in the office per se, but I believe it. Yeah, I think like—
PATRICK BET-DAVID: (00:59:15 – 00:59:16) Do you ever see it or hear it?
JACOB COXON: (00:59:16 – 00:59:31) I’ve never seen it, but I’ve seen things similar at people’s houses, right? They have like a duck that they use to talk about their problems. I think it’s cool. I think it’s like, look dude, Anthropic is going to be a $2 trillion IPO. These guys are going to eat capitalism alive. I’m like, they can do whatever the hell they want if they’re—
PATRICK BET-DAVID: (00:59:31 – 00:59:32) Who gives a shit?
JACOB COXON: (00:59:32 – 00:59:36) Yeah, these guys are taking over. I mean, you got to give them some respect. Like If they’re— yeah.
Anthropic vs. OpenAI as an Investment
PATRICK BET-DAVID: (00:59:36 – 00:59:46) And the upside between the two, just as an investor, forget about— I know what you’re thinking about as an investor between the two. Is OpenAI even close to the way they run things at Anthropic?
JACOB COXON: (00:59:46 – 01:00:11) So I haven’t been there for 4 months. Maybe it’s changed, but I would like, why is OpenAI ahead of Google? I think it’s largely a culture thing. Google has a lot of resources, a lot of people, a lot of great talent. But OpenAI has this focus on building LLMs. And Anthropic is so much more focused.
Again, I hear that OpenAI is becoming more focused, but yeah, as right now as an investor, I guess I would long Anthropic. Definitely.
What Would Prove Him Wrong
PATRICK BET-DAVID: (01:00:11 – 01:00:18) What would you need to see yourself for you to second-guess your own fears that you have right now? What would you need to see? So, you know what, maybe I’m wrong.
JACOB COXON: (01:00:19 – 01:01:06) Yeah, I would love to see a— so alignment is this, the science of making them safe. And I would love to see, for example, some good arguments, just science, scientific arguments that we control the behavior of the AIs. And I think this is doable. To go back to, I think there is research that needs to happen on this, but it’s not impossible.
Some people say it’s impossible. I think you can solve it. The main thing is just next year, the year after, these companies will have to choose between making the thing that makes more money and more power and justifies the insane capital build-out. They’ll have to choose between that and doing the safety science.
That’s like a decision. And the free market incentives push you to choosing to go fast, get power, make it better, and skip the safety research.
PATRICK BET-DAVID: (01:01:06 – 01:01:08) You don’t sound like a libertarian, you know.
JACOB COXON: (01:01:08 – 01:01:11) In this case. In this case. Anything else. Anything else.
PATRICK BET-DAVID: (01:01:11 – 01:01:12) Anything else, you’re a libertarian. Not this one.
How the World Could End in 5 to 10 Years
JACOB COXON: (01:01:13 – 01:01:13) Just not this one.
How the World Could End in 5 to 10 Years
PATRICK BET-DAVID: (01:01:13 – 01:01:27) Okay. So you see, if you right now— we did this after your tweet— if you go on, ChatGPT or Perplexity or Claude, you ask What is the likelihood that AI is going to replace humanity or destroy humanity in 10 years?
JACOB COXON: (01:01:27 – 01:01:29) Just like 10% or something, 5% maybe itself.
PATRICK BET-DAVID: (01:01:29 – 01:01:32) It says 5 to 10%, right?
JACOB COXON: (01:01:32 – 01:01:33) Okay.
PATRICK BET-DAVID: (01:01:33 – 01:01:36) I’ve asked this question from Roman Yampolskiy. I don’t know if you know Roman Yampolskiy.
JACOB COXON: (01:01:36 – 01:01:37) I saw your interview.
PATRICK BET-DAVID: (01:01:37 – 01:01:44) It was great. I sat down with him. Good conversation. I asked him how, what would it look like, right? What would it look like in your eyes? You’re an insider.
JACOB COXON: (01:01:44 – 01:01:45) Yeah, of course.
PATRICK BET-DAVID: (01:01:45 – 01:01:47) Okay. If the world’s going to come to an end in 5, 10 years, how would it happen?
JACOB COXON: (01:01:48 – 01:01:51) First, you have to realize that the world will look different for this to happen.
PATRICK BET-DAVID: (01:01:51 – 01:01:53) So it’s not like it’s the end of the world, it’s just going to be a different world.
JACOB COXON: (01:01:54 – 01:03:12) Yeah, like it’ll be that there’ll be a lot more robots. You’ve seen that we’re building out robots very quickly. There’ll be a ton of robots around, right? There’ll also be a lot of potentially a lot of weapons around that aren’t currently. There’ll be a lot of building out of, say, drones right now.
We’ve seen like Russia-Ukraine drones used 100%. We’re scaling that up. Imagine in 5 years, like these countries are going to be bristling with deadly drones. And AI is going to be integrated everywhere. We’re going to have a lot more of a robot economy. A lot more of decisions are being made by AI systems and not by people.
So you have to imagine the world looks different in that right now it feels like we’re not surrounded by that much tech. Like what could an AI do in this room? It could like mess with some cameras, mess with some lights. But imagine if just the world is bristling with like drones and robots.
There’s a lot more like AI activity happening. At that point, I think it feels a lot scarier. Like if the AI is in all the science labs, it’s hacked in, it’s in every system. And it also has all these drones and robots around. Then suddenly you don’t feel so safe anymore. Like also imagine all your decisions are being made by AI.
You’re thinking about podcast planning or what, or like stuff that, that I don’t understand. But if you ask the AI about it in the future, it’ll be giving you great advice, like strategic advice. Like here’s how to, with certain guests to be like extra— his things to say to make them extra persuasive, and it’ll be better than humans at coming up with like all sorts of stuff.
PATRICK BET-DAVID: (01:03:12 – 01:03:19) I mean, you’ve already seen some guys who come out of nowhere with their podcast blows up and they are 90% AI-driven.
JACOB COXON: (01:03:19 – 01:03:20) It’s AI-driven.
PATRICK BET-DAVID: (01:03:20 – 01:03:24) And some of the personality-driven podcasts who used to be bigger 5 years ago, they’ve kind of flatlined a little bit.
JACOB COXON: (01:03:24 – 01:03:35) And it’s like, imagine that but even more. So everyone, you’re aware that all your decisions could be made better with AI. You’re asking for planning, like say financial planning, for spreading out like whatever sort of business stuff.
PATRICK BET-DAVID: (01:03:35 – 01:03:41) That becomes a fad though. And people are like, I mean, sports teams right now, valuations growing exponentially.
JACOB COXON: (01:03:41 – 01:03:41) Yeah.
PATRICK BET-DAVID: (01:03:41 – 01:03:51) Because what people are saying is even no matter how many robots you have, people are going to want to see other people beat people. We will be more interested and impressed by what people do, human beings do, instead of what robots do.
JACOB COXON: (01:03:51 – 01:03:53) But the AI will be telling you that if that’s true.
PATRICK BET-DAVID: (01:03:53 – 01:03:53) Who will be telling you?
JACOB COXON: (01:03:54 – 01:04:34) The AI will tell you. I will say like prioritize people things because it understands So just when you’re thinking about this world, it’s like, we’re not making decisions anymore. The decisions are all the AI. At that point, it should be clear that if the AI decided to turn against us, we would have no power.
Just imagine at that point, no one’s working anymore. Everyone’s just asking the AI to run their stuff for them. Maybe some people are pretending to work, or like they think they’re working, but really the AI could do it better. And you’ve got robots and drones and labs building out all sorts of crazy technology, at that point I think you should start to feel like, wow, if humans were the enemy suddenly, there’s nothing we could do.
Why AI Is Different from Past Panics
PATRICK BET-DAVID: (01:04:35 – 01:04:44) Don’t you think that’s kind of like, every time, every generation when something new is coming out, there’s a fear about what TV was going to do?
JACOB COXON: (01:04:44 – 01:04:44) Yeah.
PATRICK BET-DAVID: (01:04:44 – 01:05:02) Elvis Presley started shaking his hips. “Oh my God, everyone’s going to start sleeping with each other. Honey, turn off Close your eyes, don’t look at his hips, this is bad,” or music, radio, TV, internet, nuclear war. “Oh, everyone’s going to use nuclear bombs. It’s going to be a nuclear World War III, World War II.” Hasn’t really happened.
JACOB COXON: (01:05:02 – 01:05:03) Yeah.
PATRICK BET-DAVID: (01:05:04 – 01:05:17) What’s different about this? Is this the fact that you think if it all of a sudden thinks on its own, the AI can think they control us and now we have no control and they can say, hey, yeah, You guys better respond to us or else.
JACOB COXON: (01:05:17 – 01:06:19) I guess what I’ll say is, I don’t know how much of the world works, right? I’ve just done tech and math, so I don’t know how to run another business or to build anything or anything like this. And also the, I guess, people that are working in other industries, they don’t have visibility into AI.
They don’t see it. What I’ll say is, if you look at just— I think the news headlines are quite interesting because for even other panics I don’t think what we had is this like buildup. I think maybe there’d be like an initial thing and then it would like be a plateau or it’d be like this TV thing.
It would dribble through the same articles coming through. There wouldn’t be this buildup. Whereas right now everyone can feel it. I’m sure you can feel it, right? With AI, it’s like every year it gets bigger and faster. So I’d say that even though, I don’t know, I don’t understand the other stuff and people outside don’t understand the AI, I would say just look at the rate.
It’s just every year it gets bigger and nothing else has been like that. It’s this exponential, coming faster. So really, I think you don’t need to trust me. I think in 6 months it will become even more clear to everyone.
PATRICK BET-DAVID: (01:06:19 – 01:06:20) In 6 months?
JACOB COXON: (01:06:21 – 01:06:22) Yeah, it’ll be more clear in 6 months.
PATRICK BET-DAVID: (01:06:23 – 01:06:24) Unpack that.
JACOB COXON: (01:06:24 – 01:06:26) Yeah. So the models that come out in the next 3 or 4 months—
PATRICK BET-DAVID: (01:06:26 – 01:06:28) You know what models are coming up?
JACOB COXON: (01:06:28 – 01:06:47) Yeah, I do. And I can’t say details, but they’re going to be a lot better at coding. They’re going to be a lot better at automating my research. It will go even more to running these agents like employees. Maybe these models will get released. Maybe they won’t get released. There’ll definitely be news stories about the decisions around releasing them. It’s going to feel big, I think.
What Makes a Billion-Dollar AI Researcher
PATRICK BET-DAVID: (01:06:48 – 01:06:54) By the way, when you say Dario was a great researcher, you’re a researcher, coder we get, engineer we get. What does it mean to be a researcher?
JACOB COXON: (01:06:54 – 01:07:15) It’s kind of pretentious. It mostly means coming up with the ideas. So like, which things to code. So like, you’ve got the AI, I guess, details of its training recipe, deciding which ingredients to add, deciding, what shape its brain is, essentially deciding how it’s connected together. All these like, decisions. It honestly is—
PATRICK BET-DAVID: (01:07:15 – 01:07:26) How do you score a great researcher? Yeah, he’s a great researcher. How do you know that? Does it take years for somebody— like, people took years to realize Dario’s good, or can you look at a 23-year-old and say, this guy’s a special researcher?
JACOB COXON: (01:07:26 – 01:07:27) Honestly, yeah.
PATRICK BET-DAVID: (01:07:27 – 01:07:35) I saw a contract that a young 24-year-old got a $1 billion payday from Meta. What is special about that guy that Meta gave him a billion-dollar offer?
JACOB COXON: (01:07:35 – 01:08:05) There are some guys that are just like, they will. So these training runs cost a lot of money. These AI training things, they cost a lot, right? We’re talking maybe like hundreds of millions more. If you make that more efficient, you’re saving the company a lot of money. The CapEx on these data centers, like hundreds of billions.
Again, if you make any of this more efficient, you’re saving a lot of money. And there are some guys for now that have amazing intuition and work really hard and will save the company a ton of money.
PATRICK BET-DAVID: (01:08:05 – 01:08:12) Yeah, like this right here. Yeah, even $250 million. The lure. 24-year-old whiz kid. We have reached the climax of Revenge of the Nerds.
JACOB COXON: (01:08:12 – 01:08:13) Oh yeah.
PATRICK BET-DAVID: (01:08:13 – 01:08:14) And I think it was $250 for a 24-year-old.
JACOB COXON: (01:08:14 – 01:08:15) Yeah.
PATRICK BET-DAVID: (01:08:15 – 01:08:18) What’s special about this 24-year-old to pay $250 to him?
JACOB COXON: (01:08:18 – 01:08:19) So I don’t remember this dude.
PATRICK BET-DAVID: (01:08:19 – 01:08:20) Go a little bit higher.
JACOB COXON: (01:08:20 – 01:08:23) So maybe look up, What was his name?
PATRICK BET-DAVID: (01:08:23 – 01:08:24) Another guy got a billion dollars.
JACOB COXON: (01:08:24 – 01:08:28) Look up the— you look at the billion guy. Yeah, who’s that? I met him a few times.
PATRICK BET-DAVID: (01:08:28 – 01:08:32) Can you go to the billion guy? Do you know the name of the guy or no?
JACOB COXON: (01:08:32 – 01:08:33) Andrew something, I think.
PATRICK BET-DAVID: (01:08:38 – 01:08:46) So guy got a billion dollars. Just ask ChatGPT. Go write that in the chat and see what it says. Yeah, somebody got a billion dollars.
JACOB COXON: (01:08:46 – 01:08:47) Oh yeah.
PATRICK BET-DAVID: (01:08:47 – 01:08:47) Why though?
JACOB COXON: (01:08:48 – 01:08:48) These guys are like—
PATRICK BET-DAVID: (01:08:48 – 01:08:50) So like you met him, so what was so special about him?
JACOB COXON: (01:08:51 – 01:08:58) These guys have good taste for research. They know the right things to do. They’ll save you so much money in your big crazy experiments.
PATRICK BET-DAVID: (01:08:58 – 01:09:01) Actually walk me through it. So can you systematize that? What does that mean?
JACOB COXON: (01:09:02 – 01:09:05) So, you fix this? Yeah, sure. Thanks.
PATRICK BET-DAVID: (01:09:08 – 01:09:12) Yeah, walk me through it. What does it mean that these guys know how to do it, save you all this time for research?
JACOB COXON: (01:09:13 – 01:09:43) Yep. So imagine you’re kicking off a big AI training run, it’s called. So it will take a few months and you’ve got to decide the details of the data and the recipe, these things. Some guys like this guy are just great at making changes to that recipe so that it becomes more efficient.
And if this whole thing, and in a way that no one else can match. No one else can have the same ideas.
PATRICK BET-DAVID: (01:09:43 – 01:09:47) So these guys, of course, we come up with an idea and this guy is able to say it’s a waste of time, I wouldn’t do it.
JACOB COXON: (01:09:47 – 01:10:06) Yeah. Although this guy in particular, I think was just great at like, this guy, you know this guy? Yeah. I met him a few times during training runs at OpenAI and he was always around. He was like very fast at solving stuff. He was fixing bugs in our training of the AI system, like snap, just fixing them in a way that other people weren’t fixing them. So he was irreplaceable, right? And then, that’s why he’s worth a billion dollars.
PATRICK BET-DAVID: (01:10:07 – 01:10:09) Who gave him the billion?
JACOB COXON: (01:10:09 – 01:10:09) Who was the—
PATRICK BET-DAVID: (01:10:09 – 01:10:11) Meta gave him a billion dollars?
JACOB COXON: (01:10:11 – 01:10:12) Yeah, from, Thinking Machines.
PATRICK BET-DAVID: (01:10:13 – 01:10:25) Yeah. Ditches Meta to train rival AI model Claude. So where— he left Meta now? The ex-engineer who reportedly got a billion-dollar package from Meta has defected to one of its rivals in the AI race.
JACOB COXON: (01:10:25 – 01:10:26) I guess I’ve not been following.
PATRICK BET-DAVID: (01:10:26 – 01:10:40) This is 3 days ago. This is literally 2 days ago. He joined Anthropic, which is an Australian tech— was reportedly so sought after by Meta, a $1.5 billion salary package in front of him had already left the company to join the AI race. Androids. Who’s he go to? Anthropic?
JACOB COXON: (01:10:40 – 01:10:41) Yeah, Anthropic. Yep.
PATRICK BET-DAVID: (01:10:41 – 01:10:51) So he went to Anthropic. So Dario picked him up. You didn’t even know this. I know this happened 3 days ago. And so this guy was a whiz, fast. He could solve it faster than his peers.
JACOB COXON: (01:10:51 – 01:10:51) Exactly.
PATRICK BET-DAVID: (01:10:51 – 01:10:53) And how old is he? Is he about your age?
JACOB COXON: (01:10:53 – 01:10:54) I think he’s 30s.
PATRICK BET-DAVID: (01:10:54 – 01:10:55) He’s 30s?
JACOB COXON: (01:10:55 – 01:10:56) Yeah, I think so.
PATRICK BET-DAVID: (01:10:56 – 01:11:07) Okay. Yeah. So Now let me ask you, can the people— okay, like let’s say 100 of you.
Could Anthropic Destroy OpenAI?
JACOB COXON: (01:11:08 – 01:11:08) Yep.
Could Anthropic Destroy OpenAI?
PATRICK BET-DAVID: (01:11:08 – 01:11:14) Okay. Who feel the way you do, ex-Anthropic, ex-OpenAI, ex-Google, ex— pick all of them.
JACOB COXON: (01:11:15 – 01:11:15) Yeah.
PATRICK BET-DAVID: (01:11:15 – 01:11:24) You guys find each other. Okay. Can you guys use the AI against itself to hack into it and destroy the systems?
JACOB COXON: (01:11:27 – 01:11:28) Sorry, what do you mean? So use the AI?
PATRICK BET-DAVID: (01:11:28 – 01:11:43) Okay, so think about it. Let’s just do an evil. Let’s devil’s advocate is what we’re playing. You are so convinced this shit is going to get bad. Yeah. That you unify together, the 100 of you. You have a common enemy. AI is going to destroy humanity. Let’s get ahead of it. Let’s destroy it ourselves.
JACOB COXON: (01:11:43 – 01:12:01) So yeah, I think you, the thing you’ve described is basically Anthropic. That entity you’re talking about is Anthropic. They’re a group of people that were scared. They organized together, they unified themselves together, and they’re going to take, aggressive action to try and save people. That is literally what Anthropic is.
PATRICK BET-DAVID: (01:12:01 – 01:12:10) So you’re saying Anthropic was those 100 people that unified together to potentially, if they wanted to destroy OpenAI, they could?
JACOB COXON: (01:12:10 – 01:12:10) Exactly.
PATRICK BET-DAVID: (01:12:10 – 01:12:14) So you mean to tell me right now Anthropic, if they wanted to destroy OpenAI, they could?
JACOB COXON: (01:12:14 – 01:12:17) No, they’re not. They’re not at that level. And I’m not saying they’re planning that necessarily.
PATRICK BET-DAVID: (01:12:17 – 01:12:20) I’m not saying you’re saying they’re planning it. I’m just saying capability. Those are 2 different things.
JACOB COXON: (01:12:20 – 01:12:20) No, they couldn’t. Not yet.
PATRICK BET-DAVID: (01:12:21 – 01:12:21) Not right now.
JACOB COXON: (01:12:21 – 01:12:21) Not right now.
PATRICK BET-DAVID: (01:12:21 – 01:12:22) When do you think they would be able to?
JACOB COXON: (01:12:24 – 01:12:27) It depends again how fast we go, but end of the decade.
PATRICK BET-DAVID: (01:12:29 – 01:12:29) So 4 years.
JACOB COXON: (01:12:30 – 01:12:30) Yeah.
PATRICK BET-DAVID: (01:12:31 – 01:12:34) So in 4 years, if Dario wanted to destroy OpenAI, he could.
JACOB COXON: (01:12:35 – 01:12:39) I think quite plausibly, yes. And it depends on how fast people go. I know, right?
PATRICK BET-DAVID: (01:12:39 – 01:12:53) What a— what? Wow. So the 7 co-founders of Anthropic, do they all share the common enemy of how they view Sam Altman?
JACOB COXON: (01:12:53 – 01:12:54) I don’t know. Yeah, I don’t know the details.
PATRICK BET-DAVID: (01:12:55 – 01:12:58) Did they all come from OpenAI, or are they all from different places?
JACOB COXON: (01:12:59 – 01:13:00) I think they all came from OpenAI.
PATRICK BET-DAVID: (01:13:00 – 01:13:04) You cannot tell me you don’t know. I mean, it’s very obvious.
JACOB COXON: (01:13:04 – 01:13:07) Daniela Amodei, maybe— no, she did too. Yeah, I think they all came from OpenAI.
Sam Altman, Elon Musk and Suchir Balaji
PATRICK BET-DAVID: (01:13:07 – 01:13:42) Jack Clark, he’s also OpenAI. Tom Brown. Can you check to see if all of the 7 Co-founders of Anthropic all came from OpenAI. So the way I watched how Sam Altman handled Elon, it was disrespectful. I’m not involved. I’m not on the inside. I don’t know what’s going on. But from the outside, it sounded like Sam Altman deceived, Elon.
It sounded like Elon was coming from a good place of OpenAI, allow everybody to know what we’re building. At least that’s my impression of it. And then it goes from a nonprofit to for-profit. And Elon’s like, wait a minute, what are you doing? Didn’t Elon give like $50 million?
JACOB COXON: (01:13:42 – 01:13:43) He gave some amount.
PATRICK BET-DAVID: (01:13:43 – 01:14:04) He gave some amount to him to help build it. And then it goes, so you— the reputation in the marketplace about OpenAI is not positive, specifically Sam Altman. We even had a guy here who used to work at OpenAI. His parents came here. And this is the fellow who was the whistleblower who was found dead. I don’t know if you remember that story.
JACOB COXON: (01:14:04 – 01:14:04) Yeah.
PATRICK BET-DAVID: (01:14:05 – 01:14:17) Yeah, we had him here. Mom came here. Dad came here. They spent it. They were trying to tell us, I’m telling you, here’s what we think happened with Sam Altman. Where some people would say there’s an element of evil.
JACOB COXON: (01:14:17 – 01:14:17) Yeah.
PATRICK BET-DAVID: (01:14:17 – 01:14:20) In this guy. Would you agree with that?
JACOB COXON: (01:14:20 – 01:14:27) I don’t know. And I will say about the Suchir thing, so all the stuff I’m saying about taking over in the future, I mean that completely.
PATRICK BET-DAVID: (01:14:27 – 01:14:30) In the short term, you mean that completely?
JACOB COXON: (01:14:30 – 01:14:42) I mean that it’s possible in the future. Yeah. This, yeah, there’s no way that they killed Suchir like that. That wasn’t a hit. 100%, it wasn’t. That’s just not a thing OpenAI would do.
PATRICK BET-DAVID: (01:14:42 – 01:14:42) You know that?
JACOB COXON: (01:14:43 – 01:14:43) I know that. Yep.
PATRICK BET-DAVID: (01:14:44 – 01:14:45) How do you know that?
JACOB COXON: (01:14:45 – 01:14:50) Just from personal relationships with people that run the company. This is, it’s just nonsense.
PATRICK BET-DAVID: (01:14:50 – 01:14:51) Did you ever meet him?
JACOB COXON: (01:14:52 – 01:14:53) Yes, I met Suchir. Yep.
PATRICK BET-DAVID: (01:14:53 – 01:14:54) What was he like? Was he—
JACOB COXON: (01:14:55 – 01:15:13) He was great. Just a great researcher. Great guy. I presume he was depressed. I presume it was a suicide. I think there’s no It seems impossible that it was anything else. Yeah. And this is someone who’s saying, I am also saying that OpenAI and Anthropic are racing to destroy the world. But this particular act, I can say this was—
Jacob’s Upbringing and Values
PATRICK BET-DAVID: (01:15:13 – 01:15:23) I like you a lot, Jacob. I don’t know who you were, but I like you a lot. Let me— the more I talk to you, the more I like you. It’s so funny because I didn’t know who you were. I don’t know how you were, but tell me about your parents.
JACOB COXON: (01:15:23 – 01:15:24) My parents?
PATRICK BET-DAVID: (01:15:24 – 01:15:24) Yeah.
JACOB COXON: (01:15:24 – 01:15:29) Oh, sure. Yeah. My father’s a professor of medieval German literature.
PATRICK BET-DAVID: (01:15:30 – 01:15:31) Medieval German literature?
JACOB COXON: (01:15:31 – 01:15:31) Yes.
PATRICK BET-DAVID: (01:15:32 – 01:15:32) Why?
JACOB COXON: (01:15:33 – 01:15:35) You can ask him that. It’s pretty dumb.
PATRICK BET-DAVID: (01:15:36 – 01:15:36) That’s what he did?
JACOB COXON: (01:15:37 – 01:15:41) Yeah, that’s what he did. Yeah. How about mom? Oh, she did some stuff. I don’t know.
PATRICK BET-DAVID: (01:15:41 – 01:15:45) Yeah, I think, and what values did they raise you with? What did they teach you?
JACOB COXON: (01:15:46 – 01:15:50) What values did they raise me with? I read a lot of books.
PATRICK BET-DAVID: (01:15:53 – 01:16:02) Your character, I want to know about the character. What did they tell you, hey, you’re a man, you keep your word and integrity. What do they teach you on? What values did they— that’s what I’m interested in.
JACOB COXON: (01:16:03 – 01:16:05) I was raised in a Christian school, but my parents are atheists.
PATRICK BET-DAVID: (01:16:06 – 01:16:10) They’re atheists and they intentionally put you in a Christian school?
JACOB COXON: (01:16:10 – 01:16:13) Well, a lot of schools in the UK are Christian. It’s not in the same—
PATRICK BET-DAVID: (01:16:13 – 01:16:14) Like, they didn’t—
JACOB COXON: (01:16:14 – 01:16:16) It’s not the same as the US. Yeah, it’s like a bit lighter. Yeah.
PATRICK BET-DAVID: (01:16:16 – 01:16:19) Did that, did the faith play a role on you?
JACOB COXON: (01:16:20 – 01:16:24) I don’t think so, no. No.
PATRICK BET-DAVID: (01:16:24 – 01:16:28) Your integrity, where do you get the values? Where do you get that from? This is very impressive.
JACOB COXON: (01:16:29 – 01:16:42) I think it’s maybe a math thing. I don’t consider myself to be particularly in— like, I think I’m just trying to say the truth. I try and say true things is the main thing. Like in math, you try and say the truth about stuff. So I’m just trying to say true things, right? Like in this case, I don’t know.
PATRICK BET-DAVID: (01:16:43 – 01:16:51) Yeah, but character is different than true. I know a lot of people that are good in math who are extremely logical, but they find a way to use that against them to manipulate and have power. I don’t feel that from you.
JACOB COXON: (01:16:53 – 01:16:57) I’m really not a saint in this situation. I’ve told you I’ve profited a lot from working at OpenAI.
PATRICK BET-DAVID: (01:16:58 – 01:16:59) You’ve profited a lot?
JACOB COXON: (01:16:59 – 01:17:05) Yeah, I made money from working at OpenAI. I think, I worked at OpenAI even while thinking it was potentially a dangerous thing.
PATRICK BET-DAVID: (01:17:06 – 01:17:08) While you were doing it because you were making good money?
JACOB COXON: (01:17:08 – 01:17:14) Because, not specifically the money, but because of the whole situation. You want to be part of it. You want to be working on it. You’re not thinking too clearly.
PATRICK BET-DAVID: (01:17:17 – 01:17:19) When’s your birthday? What month’s your birthday?
JACOB COXON: (01:17:19 – 01:17:21) It was, September 20th.
PATRICK BET-DAVID: (01:17:21 – 01:17:21) So you just had a birthday?
JACOB COXON: (01:17:21 – 01:17:23) Yeah, I just had a birthday 10 days ago.
PATRICK BET-DAVID: (01:17:23 – 01:17:24) Yeah, happy birthday to you.
JACOB COXON: (01:17:24 – 01:17:24) Thank you.
PATRICK BET-DAVID: (01:17:24 – 01:17:33) Yeah, because I remember reading 26. I’m like, when did he turn 27 years old? Because at the beginning, yeah, when you did this, you were 26 years old, September 8th, right?
JACOB COXON: (01:17:33 – 01:17:34) 27. Yep. And now I’m 28.
PATRICK BET-DAVID: (01:17:34 – 01:17:35) Oh, you’re 28 now?
JACOB COXON: (01:17:35 – 01:17:35) Yeah.
PATRICK BET-DAVID: (01:17:35 – 01:17:41) Okay, I got it. Well, 2 more, you’re going to be old. You’re about to be 30, man. What are you going to do from here, you know?
JACOB COXON: (01:17:41 – 01:17:43) I have to respond to those gay men in my DMs.
Dario’s Paranoia and Anthropic’s Broken Promise
PATRICK BET-DAVID: (01:17:43 – 01:18:20) Let’s hope some of them have money and say, hey, I need somebody to take care of me. That is hilarious. So the question that I asked you and your answer, I’m still in that question because I asked, I said, if 100 people like you, so if 100 people of you united and decided to go up against the enemy, could you break them?
And you said, that’s exactly what Anthropic did. That is very interesting. That is very interesting. So they created themselves to play defense against the enemy. But if the technology gets smarter than them, then they have to play defense against the technology that’s got a life of its own now.
JACOB COXON: (01:18:20 – 01:18:20) Yeah.
PATRICK BET-DAVID: (01:18:20 – 01:18:28) And they’re aware of that. It seems like Dario is aware. Is it fair to say that an assessment of Dario is he’s a pretty paranoid guy? Yes, he is.
JACOB COXON: (01:18:28 – 01:18:28) Definitely. Yes.
PATRICK BET-DAVID: (01:18:30 – 01:18:33) And so that could work in your favor and against you at times.
JACOB COXON: (01:18:33 – 01:18:53) Yeah. And I think an important thing with Anthropic is they always promised that they would never overtake OpenAI at the very start. They always said that they would never push the boundaries. They would always be second place. They’d catch up, they’d make it safe. But now they’ve overtaken, and now they regularly release AIs that are better than OpenAI’s AIs, which was a thing they said they would not do.
PATRICK BET-DAVID: (01:18:54 – 01:18:57) And you think that’s just because the capitalism competition?
JACOB COXON: (01:18:58 – 01:19:10) I think that’s a component of it. I think they also have a story that if they can get ahead, they have extra time to make it safe. They have like better AIs to run their safety research. There’s a lot of reasons for it, but also it is You know, it makes them a lot of money.
PATRICK BET-DAVID: (01:19:10 – 01:19:12) So who’s third in the competition, by the way?
JACOB COXON: (01:19:12 – 01:19:13) Who started the competition?
PATRICK BET-DAVID: (01:19:13 – 01:19:15) Who’s third? If it’s them two, who’s three?
JACOB COXON: (01:19:15 – 01:19:17) That’s a great question. I think it’s a—
PATRICK BET-DAVID: (01:19:17 – 01:19:19) It’s like a far distance third.
JACOB COXON: (01:19:19 – 01:19:34) It’s not a— I mean, it’s pretty far at the moment, I think. So there’s a, there’s Meta, Google, and xAI. And I’m not actually deep enough in the weeds right now to know exactly who’s third. I suspect it’s like Meta overall. But those three are also around.
What He Would Tell the White House
PATRICK BET-DAVID: (01:19:39 – 01:19:52) Has the White House reached out to you? No, they’ve not reached out to you to just be like, hey, what do you think about what’s going on? Even not— I’m not saying the president, I’m saying somebody on the team that’s kind of like, hey, if you’re here where you’re at, what suggestions would you have for us?
JACOB COXON: (01:19:52 – 01:19:57) I’m not sure. I think quite a few staffers have reached out, or politicians, but I’m not sure about the White House per se.
PATRICK BET-DAVID: (01:19:57 – 01:20:07) Yeah, you haven’t had a conversation with them? No. Okay. If they did and they say, okay, Jacob, you’re a young guy that, yeah, you worked at both of these places, what should we do? What would you say?
JACOB COXON: (01:20:08 – 01:20:11) So what I say is, again, to start with, I got the Chinese nationals.
PATRICK BET-DAVID: (01:20:11 – 01:20:14) Yeah, okay, don’t give it— kind of treat it like SpaceX.
JACOB COXON: (01:20:14 – 01:21:28) But what else? The most important thing is we need to make sure that the— so that the safety problem is solvable. We just need to incentivize solving it. We need to make it so that we devote our resources to solving it. To make that— let me write that down. In a couple of years, someone will be able to decide what percentage of his resources go to making money, making things smarter, and what percentage go towards doing safety research.
Because a lot of stuff will be automated. So it’s really like— it’s like a game, like a video game. You’ll get to just say safety research, this percent. It’s like you’re allocating resources in a video game. And the problem is, what percentage do you allocate? And that depends on whether you’re in a race or not.
So what I would say is we need to ensure that they will be able to pick the right percentage for safety, which could be 50, 60, 70. It could be a big number. They have to pick the right number. And right now I’m worried that the free market means they will pick the wrong number and there’s nothing we can do about it.
And also the race with China. So what I would say is we need some agreement with all the labs and with China about the percentage of resources that go to making ourselves smarter versus doing the safety research.
The US-China AI Red Phone and the Blind Spots
PATRICK BET-DAVID: (01:21:29 – 01:21:32) Did you see the red phone story today?
JACOB COXON: (01:21:32 – 01:21:32) No.
PATRICK BET-DAVID: (01:21:33 – 01:21:38) So can you pull that up, the red phone story? So back in the days, US and USSR had a red phone.
JACOB COXON: (01:21:38 – 01:21:38) Yep.
PATRICK BET-DAVID: (01:21:39 – 01:22:37) For nuclear, that they would have the conversation if something were to happen. They created a red phone, story US-China red telephone could bring a Cold War guardrail to AI. They’re saying they have that, but China doesn’t want to abide by it. Yep, China is not really interested in it. Okay, so that’s already been talked about, and Xi and the president had a conversation about this I think last week.
I think they have another follow-up to come back. They spend a little bit of time talking about AI. US and China set up an AI red phone. Can you go to the other one you just had? US and China to set up a red telephone. So, I guess the question I’m trying to go with this here is if they’re going to do that, they don’t agree to it, we choose to do it internally, they continue advancing, where lies the blind spot for Trump, President Trump?
Where lies the blind spot for China? Where’s the blind spot for Dario and Altman?
JACOB COXON: (01:22:37 – 01:22:38) What do you mean by a blind spot?
PATRICK BET-DAVID: (01:22:39 – 01:23:03) Like something that they’re not even looking— they’re like, yeah, I don’t think it’s that big of a deal. Where is their blind spot? What are they not seeing? Is it just the incentive is so big that it’s like, I don’t want to slow down because there’s no way in the world I’m going to— so has it become from, we were doing this for a noble cause, that we know OpenAI and Sam Altman were the enemy and we had to prevent them to expanding, and now it’s like, we’re going to beat them, there’s no way in the world we’re going to let them advance.
JACOB COXON: (01:23:04 – 01:23:47) The biggest thing is it’s really easy to kid yourself with safety research. So it’s very easy to say this thing looks great because all you need is one mistake and you’re toast. So it’s very easy to like— and in a race, just imagine you’re like, you’re 90%, you’re like, okay, this looks great.
We’re not quite certain. China’s also doing it. What do we do? We either launch today and keep the lead on China or we delay a few weeks and do these checks that probably are not necessary. The question is, what would you do in that scenario? And I don’t know what I would do.
Imagine you’re running a— imagine you’re running an AI lab and you’re like, do I keep going or do I stop and do the safety thing when China is—
PATRICK BET-DAVID: (01:23:47 – 01:24:09) That’s what I’m saying. So you don’t— you and I don’t know, right? Because we’re not on the inside. Could it be that they know something that you don’t know which is causing them to continue? Because there’s no way in the world Like, are they going to think and say, you know what, I’m going to go— it’s like the movie I, Robot.
I’m going to have so many of these robots and I have control of them that no one’s going to be able to touch me because they will protect me. Basically, you think that’s what they’re thinking?
JACOB COXON: (01:24:09 – 01:24:14) I think plausibly at some point they’re flirting with that mindset. Yeah, sometimes.
Whoever Controls AI Becomes a God
PATRICK BET-DAVID: (01:24:14 – 01:24:23) So we are now more powerful than any military in the world because I can activate any red button of a nuclear bomb if I wanted to because I can hack into your system.
JACOB COXON: (01:24:23 – 01:24:23) Yep.
PATRICK BET-DAVID: (01:24:24 – 01:24:31) So now you’re talking about a bigger problem because now it becomes a godlike figure. You think you are now God, right?
JACOB COXON: (01:24:32 – 01:24:53) These people have been planning to be. And you look at the early conversations, look at Elon Musk’s conversations, look at Demis Hassabis’s conversations. From the start, it’s been whoever controls AI becomes a god, almost. That’s been the picture from the start. OpenAI was founded because they didn’t want Demis Hassabis to become a god, as it were.
PATRICK BET-DAVID: (01:24:55 – 01:26:06) And now we’re getting close, which is just weird and scary that, so then at that point, if you create guardrails and you get the systems, because then the enemy of Dario becomes somebody on the inside, because typically what happens is somebody on the inside watches you and sees that you’re becoming evil and they try to talk to you about it and you either eliminate them or you fire them or you get rid of them or you create guardrails to undermine them.
You use propaganda to say, hey, this guy’s a pedophile. This guy, I saw something with a kid or I saw something with this. And then, kind of like that one story where the AI threatens the user to tell his wife that he’s cheating on the wife. I don’t know if you saw that or not with the blackmail.
Did you see that story recently? It’s going to be very easy to toy with the human’s emotion of public humiliation to control them. And, but Dario’s enemy essentially long term becomes nobody but somebody on the inside. Nobody else is going to be his enemy, quite plausibly.
JACOB COXON: (01:26:06 – 01:26:19) Although, soon the guys running this company, they won’t need the humans, right? They can run the company on the back of AI. 7 guys, the 7 guys, or even just— I mean, it depends who has the power, but they’ll be there just telling the AI.
PATRICK BET-DAVID: (01:26:20 – 01:26:27) One of those 7 will be the enemy, right? No, I’m telling you, that’s how this ends. This ends with one of the 7 says, “What the fuck, what are you doing?”
JACOB COXON: (01:26:27 – 01:26:28) But you’ve forgotten the stuffed toys.
PATRICK BET-DAVID: (01:26:29 – 01:27:57) That’s what’s going to save us. You know what, I’m sorry I missed it. My apologies, I wasn’t paying attention to it. Did you see if all of them were former OpenAIs? Is it yes? So the 7 currently identified as Anthropic are: Dario was OpenAI VP Research, Daniela was OpenAI VP Safety and Policy, Tom Brown was OpenAI researcher, lead author of ChatGPT— well, GPT-3 paper, which you were talking about.
Sam, also an OpenAI researcher, worked on scaling AI safety. Jack, policy director. Chris, interpretability researcher. And then Jared Kaplan worked with the OpenAI research group and was a major contributor to— so they all united against Sam Altman. Yep. If this thing gets that powerful, one of these guys is going to turn against Dario.
If it gets so bad that the world could have issues, one of these 7 is going to say, “Dario, you have lost control.” You’re not thinking you’re bigger than everybody else. Because you have to realize, the good thing about the presidency is what? The reason why folks like you and I leave our country to come here, okay, is this place offers more opportunity.
I left Iran, you left UK. Now you may go back to UK, who knows, but I think you’re going to stay here, right? Yeah. Is when you have a president instead of a ruler, you know the president’s going to get taken out in 4 to 8 years because you got 2 terms. So it’s kind of like no matter how bad it’s going to be, he’s going to be out anyways.
JACOB COXON: (01:27:57 – 01:27:57) Yeah.
PATRICK BET-DAVID: (01:27:58 – 01:28:16) “Oh, Biden sucks. He’ll be out. Trump sucks. Obama sucks. He’ll be out.” Just wait 4 to 8 years. Dario becomes a president for 30 years. And he may not, he may get reelected for 30 years. And these companies are becoming countries. And so, I don’t know. I don’t know.
JACOB COXON: (01:28:17 – 01:28:17) Scary.
PATRICK BET-DAVID: (01:28:18 – 01:28:22) Yeah. If it goes to where you think it’s going to go to, it is scary.
JACOB COXON: (01:28:22 – 01:29:14) Yes. And it may not, it— maybe I’m wrong, maybe people are wrong. The good news is I think we’ll find out quite soon. And next month, I think further details of this stuff we’ll find out properly in the next 1 or 2 years, just by following the trajectory. And I’m happy to be, if I’m embarrassed, so be it, but the world will, go on and that will be— that’ll be great.
But if it’s not the case, I think it’s good that people are a bit more ready right now so that when things do start getting a bit crazier, if people’s eyes are open, that’s like the main thing. I think like everyone should have their eyes open and be like demanding transparency because it’s completely crazy that all this power is being concentrated in a few companies in San Francisco. Yeah.
Life After Going Viral
PATRICK BET-DAVID: (01:29:14 – 01:29:17) When you go out in San Francisco, go to dinner, do people recognize you?
JACOB COXON: (01:29:17 – 01:29:23) Not really, no. I think I’m not wearing the glasses. People remember the glasses from the—
PATRICK BET-DAVID: (01:29:23 – 01:29:30) So people don’t stop you and say, hey, you, why did you say that? I agree with you. Do you get any of that?
JACOB COXON: (01:29:30 – 01:29:38) I had that at the start. The first week it was crazy because I think it was on the news the whole time. I had people heckling. I was like at the gym, someone was like, “Why are you trying to ruin the economy?” Or whatever.
PATRICK BET-DAVID: (01:29:38 – 01:29:39) People are telling you this?
JACOB COXON: (01:29:39 – 01:29:42) Yeah. But like not anymore. No, I don’t get like, recognized on the street.
PATRICK BET-DAVID: (01:29:42 – 01:29:43) Did you ever get death threats or not really?
JACOB COXON: (01:29:44 – 01:29:47) Not really. I mean, I have some implicit threats, but I just don’t look at Twitter.
PATRICK BET-DAVID: (01:29:47 – 01:29:49) Oh, okay. On Twitter.
JACOB COXON: (01:29:49 – 01:29:52) On Twitter. Those are bots. Exactly. Those are bots. That’s fine. I’m sure you get a ton.
PATRICK BET-DAVID: (01:29:52 – 01:29:57) Yeah, you get those a lot. And it’s not hard to kind of see. At first I thought you were a bot.
JACOB COXON: (01:29:57 – 01:29:57) Yeah.
PATRICK BET-DAVID: (01:29:57 – 01:29:59) Because your account had just been started.
JACOB COXON: (01:29:59 – 01:29:59) Yeah.
PATRICK BET-DAVID: (01:29:59 – 01:30:07) And then when I saw your interview with Cooper and Bret Baier, I’m like, no, this is actually a real human being. And you’re not a robot. Just for full compliance, I just want to make sure the audience knows you’re not.
JACOB COXON: (01:30:07 – 01:30:08) No drinking water.
PATRICK BET-DAVID: (01:30:09 – 01:30:10) You bleed, right? If you cut—
JACOB COXON: (01:30:10 – 01:30:11) I bleed.
How Ordinary Families Should Prepare
PATRICK BET-DAVID: (01:30:11 – 01:30:41) Okay, that’s good. We just want to make sure there’s a What else do you think people should be thinking about? If I’m somebody that’s got a wife and kid, yeah, okay, and I’m not in your world, yeah, I’m on the outside— that’s 99% of people in America, they’re not in your world, right?
What should I be doing? You’re my brother, you’re my nephew, you’re my friend’s son, you’re at my house. I ask you, “Jacob, if you think this is real, how do I, prepare for this? What can I do?” Yeah.
JACOB COXON: (01:30:41 – 01:31:35) So one thing is, like I said, eyes open. I just think paying attention and not like, not coping feels important. I think like it’s so easy to pretend it’s not happening because also you’ve got your own, everyone’s got their own lives to lead. I’m not saying don’t let this take over your life.
It should be obviously it’s a small thing in the context of any individual life that you’ve got to look after yourself. But I’d say just keeping the eyes open and seeing where things are going is the first step to action. I really don’t know what to do, on an individual level. The other thing is maybe some basic prepping for stuff like, cyberattacks or even a small pandemic.
If there’s some sort of bioweapon leak in the next couple of years, doing a little bit of prepping of like food, masks, basic supplies for if there’s some minor disaster or even just civil unrest, some sort of like 2-week-long thing. Not the full like apocalypse, but I think some basic prepping is sensible.
Hacking, Black Swans and Gold, Guns and Land
PATRICK BET-DAVID: (01:31:36 – 01:31:57) At what point do you think this thing is going to have the capability of, hacking into a system, to Pentagon, to governments, to China, to Russia? When will it get to that point where it can go in and say, I can tell you exactly how much this guy’s got in his bank account, I can log into your Chase account, I can log into your Wells Fargo account, I can log into your— when will that happen?
JACOB COXON: (01:31:58 – 01:32:04) There’s a lot of uncertainty. My best guess 1, 2 years.
PATRICK BET-DAVID: (01:32:04 – 01:32:05) That, that soon?
JACOB COXON: (01:32:05 – 01:32:07) Yes. I’m not saying that it will be doing this, and I’m not saying which ones.
PATRICK BET-DAVID: (01:32:07 – 01:33:05) No, for sure. But yeah, that soon? Yes. So when pandemic happened, we hadn’t experienced something like that before. I’m 47 years old. I haven’t experienced something like that. You’re 27, 28. All of a sudden the world shut down. Everyone’s like, wait a minute, what are you talking about? Stay home, do this, do that.
Power grid. We’ve never experienced that. Yeah, when Y2K was happening, they thought the banking system— you were just born, I think you’re born ’99, or what year are you, ’98 or ’98? ’98. Okay, so it’s like, oh my God, Y2K, system’s going to shut down. Yeah, right now we see an AI has transformed the Pentagon’s aging networks into a national security risk.
Okay, if all of a sudden you can go into government systems and know where they have everything laying around, or you’re able to go into every single account and take $1,000 out of everybody’s money, that’s a different kind of a, black swan event that, yeah, we’ve never experienced before.
JACOB COXON: (01:33:05 – 01:33:05) No.
PATRICK BET-DAVID: (01:33:06 – 01:33:12) And I think it’s actually going to happen. I just don’t know when it’s going to happen. I’m not one that is naive to say that’ll never happen. I think it’s going to happen.
JACOB COXON: (01:33:12 – 01:33:14) Yep. And I—
PATRICK BET-DAVID: (01:33:14 – 01:33:17) That could lead to civil unrest.
JACOB COXON: (01:33:17 – 01:33:17) Yep.
PATRICK BET-DAVID: (01:33:17 – 01:33:24) Crime, chaos. Small little civil wars, whether it’s in states or countries. Yep.
JACOB COXON: (01:33:24 – 01:33:29) I’m sure you’re pretty prepared. You’ve got some like basic prep stuff where you’ve thought about basic prepping.
PATRICK BET-DAVID: (01:33:29 – 01:33:33) We don’t have a choice, right? Yeah, because you have to. Yeah. You are as well? Are you prepping?
JACOB COXON: (01:33:33 – 01:33:35) I’m actually not as prepped as I should be. No.
PATRICK BET-DAVID: (01:33:35 – 01:33:37) Are you a Second Amendment guy?
JACOB COXON: (01:33:37 – 01:33:39) Yeah, sure you are.
PATRICK BET-DAVID: (01:33:39 – 01:33:43) Yeah, like I don’t actually, own a gun, but you support it?
JACOB COXON: (01:33:43 – 01:33:44) Definitely, I support it. Yeah.
PATRICK BET-DAVID: (01:33:44 – 01:34:15) Yeah. And I think this is kind of the area where That becomes important. I think you have to do that. Yeah, my son the other day, I’m working out at the gym, he’s like, “Dad, why do you have so many rounds in the gym? Why do you have so much ammo in the gym?”
Yeah, you just got to be ready. It’s like, hey, I use it as dumbbells. That’s what I do. I’m working out with it, right? But I agree. But can you— have you hypothesized with your friends where it’s late at night, you guys are at the house having a beer or two, and it’s 2 o’clock in the morning, 1 o’clock in the morning, is what could actually happen? Give me 5 case studies. Have you guys talked like that?
JACOB COXON: (01:34:15 – 01:34:21) Yeah. There’s people like, people buying gold and guns. Gold, guns, land. That’s pretty common.
PATRICK BET-DAVID: (01:34:21 – 01:34:22) Gold, guns, land?
JACOB COXON: (01:34:23 – 01:34:23) Yep.
PATRICK BET-DAVID: (01:34:24 – 01:34:24) What do you mean?
JACOB COXON: (01:34:25 – 01:34:28) They’re buying gold. Yeah. Because of potential like economic stuff.
PATRICK BET-DAVID: (01:34:28 – 01:34:29) They’re buying gold, guns, land.
JACOB COXON: (01:34:30 – 01:34:35) Yeah, gold, guns, and land. All those 3 people are doing that. People that work on the tech are buying that stuff.
PATRICK BET-DAVID: (01:34:36 – 01:34:41) People that are working on tech are buying gold, guns, and land? Yep. That makes sense.
JACOB COXON: (01:34:41 – 01:34:47) Yeah, it does make sense. And they’re doing it like these are engineers, researchers, not famous people.
PATRICK BET-DAVID: (01:34:47 – 01:34:49) These are just guys that are working day to day.
JACOB COXON: (01:34:49 – 01:34:49) These guys that work day to day.
PATRICK BET-DAVID: (01:34:49 – 01:35:07) Yeah. And they talk to each other. “I just bought some gold. I just bought some more guns and I bought land.” Yep. But what I was asking about is case studies of what could happen. If you’re a researcher, right? If I ask you that fellow we talked about that got $1 billion and now we just found out 3 days ago he’s at Anthropic. Yeah. We didn’t even know. Neither one of us knew about this.
Doomsday Scenarios: Viruses, Unrest and Stolen Models
JACOB COXON: (01:35:07 – 01:35:26) Basically, one basic thing is very simple. But, some lab in a country somewhere releases a virus. Not even a particularly dangerous virus necessarily, but like a minor pandemic of some sort. Seems completely on the cards, possible. It’s been possible even before AI, but I think it’s more likely every year.
PATRICK BET-DAVID: (01:35:26 – 01:35:27) But you can’t blame AI for that.
JACOB COXON: (01:35:28 – 01:35:32) You can in the case where it’s people that formerly didn’t have the skills but got the skills from—
PATRICK BET-DAVID: (01:35:33 – 01:35:36) Oh, now because of AI you learn how to create the next Virus.
JACOB COXON: (01:35:37 – 01:36:02) Oh yeah, previously, maybe the smartest bio guys, they don’t want to make viruses. The Unabomber, like Ted Kaczynski, the Unabomber guy. He’s a smart guy and wanted to do it, but it’s very rare. It’s very rare for like a guy to be this smart and then also want to do stuff. Now if you’re a kind of dumb biologist somewhere, like you don’t know how to make this stuff, if you’ve got to, you could just ask AI and say—
PATRICK BET-DAVID: (01:36:02 – 01:36:03) Would it tell you?
JACOB COXON: (01:36:03 – 01:36:09) It wouldn’t tell you off the bat, but there are ways of potentially jailbreaking. You can trick it into answering your question.
PATRICK BET-DAVID: (01:36:09 – 01:36:11) People know how to do that? Yeah, most people nowadays—
JACOB COXON: (01:36:11 – 01:36:16) Not most people know how to do that. It gets quite hard. And in particular, the safeguards that Anthropic have are very good, so it’s quite hard to get past.
PATRICK BET-DAVID: (01:36:16 – 01:36:21) Is it something where in your community of people know how to do that? You know how to do that?
JACOB COXON: (01:36:21 – 01:36:25) There are some people who specialize in it. It’s called red teaming. You’re trying to break past the safeguards.
PATRICK BET-DAVID: (01:36:25 – 01:36:26) But red teaming—
JACOB COXON: (01:36:26 – 01:36:51) Red teaming, yeah, you’re like a red team. You try and break into them. But if you’re an expert red teamer, it costs a lot of money. You’ve got to like, and Anthropic is getting better at blocking, but it’s completely plausible that if it’s not, an Anthropic model, it could be another model. It could be an open source model that gives good advice. And then a, minor pandemic happens. It’s just a scenario to be prepared for.
PATRICK BET-DAVID: (01:36:51 – 01:36:54) So this one here says this AI just created a virus that’s not found in nature.
JACOB COXON: (01:36:55 – 01:37:23) So I don’t know how dangerous these are. I don’t know how good this is. It might not even be creating new viruses, right? It might just be advice on how to use an old virus to cause damage or like slightly edit an old virus or just do basic things. There’s also a story I think recently of like Claude blocking a team trying to make bioweapons.
I think you’ll see. So Claude had to block, like Anthropic had to, they detected them trying and then blocked them. I think there should be a news article about this. So people are trying, they’re trying to use the AIs to help them. Help them.
PATRICK BET-DAVID: (01:37:24 – 01:37:25) Are they reporting that to the government?
JACOB COXON: (01:37:25 – 01:37:27) Yeah, this was a news article. Exactly.
PATRICK BET-DAVID: (01:37:28 – 01:37:49) Blocks possible attempt to use AI to make biological weapons. 9/11, what a day. Yeah, what a day. And so, but by then, so these red team and blue team, AI on AI red teaming. One powerful AI is instructed to attack another AI thousands or millions of times automatically, looking for vulnerabilities humans wouldn’t discover.
JACOB COXON: (01:37:49 – 01:37:49) Yep.
PATRICK BET-DAVID: (01:37:50 – 01:37:51) Okay, so viruses is one.
JACOB COXON: (01:37:51 – 01:38:39) What else? Yep. Another very concrete thing is unrest. They’re worried about potential unrest in the future due to changes to employment, and due to changes in people’s lives. If there’s sudden economic change, it can be quite disruptive in the short term, and that could also lead to a lot of anti-AI backlash. I think many employees expect that they won’t be able to go to an office publicly anymore in the future because of all the anti-AI backlash.
They’ll be locked away somewhere where it’s harder to— they’re less exposed. So they’re expecting a lot of sentiment like that. There’s a lot of talk about, where they’re going to live in the US when it gets dangerous in San Francisco, when it gets dangerous to be an AI researcher in San Francisco, where are they going to like camp out and work remotely because of potential civil unrest?
PATRICK BET-DAVID: (01:38:39 – 01:38:43) So places like Wyoming, North Dakota, that sort of stuff. Those places are going to be attractive.
JACOB COXON: (01:38:43 – 01:38:44) They’re going to be pretty attractive. Yep.
PATRICK BET-DAVID: (01:38:47 – 01:38:57) Got it, I’m tracking. What else? Do you think they’d get to a point that the system knows how to go into, the countries that have nuclear weapons and know how to press the button? You think we’ll ever get to that point?
JACOB COXON: (01:38:57 – 01:38:59) Yeah, I mean, we’d be able to do that.
PATRICK BET-DAVID: (01:38:59 – 01:39:09) You’re just saying it like a crazy evil person can find a way to go press the red button for 50 different countries, 20 different countries, and send that missile?
JACOB COXON: (01:39:09 – 01:39:28) Again, it depends how proliferated the tech is, who’s in control. But the best AI systems will be able to do that. It’s whether the guards are in place. It’s who has access. It’s who can break in and use it, who can red team. Other things that could be chaotic, at some point, China, at any point, imagine China steals the best American model.
PATRICK BET-DAVID: (01:39:28 – 01:39:31) I think what you said at the beginning, that’s my biggest concern.
JACOB COXON: (01:39:31 – 01:39:34) Right, they steal the whole model and then they immediately catch up and suddenly it’s just real race on.
PATRICK BET-DAVID: (01:39:34 – 01:39:36) What a brilliant, and they’re doing that already anyways.
JACOB COXON: (01:39:36 – 01:40:11) They’re already— They’re not stealing the actual AI brain itself. But who knows, like they could maybe steal the AI brain. And at that point, the race is crazy, especially if the AI is already good enough to do the research. Then it’s just game on. Then anyone who steals it is a player. Like they don’t need this billion-dollar guy.
They don’t need Andrew Tulloch anymore. You can just use the thing you stole to make itself smarter. Then it’s just like a kind of, like a god that you steal and use for your own devices. So I think that’s another thing is just if China do steal it, that would be a crazy situation.
PATRICK BET-DAVID: (01:40:11 – 01:40:17) What are some of you guys that are engineers and brains and you want to follow the system, what are you guys talking to each other? What are you going to do in 2, 3, 4, 5 years?
JACOB COXON: (01:40:17 – 01:40:46) People don’t know. Yeah, it depends on how things play out, but they use words like crunch time, endgame. Again, there’s slight uncertainty on the timelines, so This could be happening early 2030s. I don’t think so. I think probably sooner than that. But in the next 4 or 5 years, no, the people that are talking about it, they describe it as like the end of the world.
For good or for bad, for good or for bad, it’s like, feels like the end of the world. Okay.
The Hugging Face Incident
PATRICK BET-DAVID: (01:40:47 – 01:40:55) Can you to the audience, explain what happened with Hugging Face? Because that was a whole thing and what it really is. What was it? Why did it become such a big story?
JACOB COXON: (01:40:55 – 01:41:53) Yeah. It became a big story because it was the first time that AIs very clearly did something that was bad and something that was just their own idea. They just went and aggressively pursued something that was their own idea. It was the first time we really had like an out-of-control AI do something in the real world.
And that was scary because it was like, Maybe in the past, your AI used to go a bit, when you’re asking it to do something like plan an appointment and it makes a mistake, or maybe decides to do something. I saw one the other day, someone was on Facebook Marketplace and the AI sort of shared their personal information on Facebook Marketplace, did something that they didn’t want.
But Hugging Face was like that, but on a whole other level. The AI has just decided, they just decided to attack a website, and then they did it. So I think that’s why it captured people’s imagination so much.
PATRICK BET-DAVID: (01:41:54 – 01:42:26) Yeah, I think, what was the number that I saw? Was it 1,200 agents that exchanged 70,000 different messages together, and one of them, one of the agents alone, did 17,500 codes? I don’t know what the number was. I could be getting the numbers all mixed up, but it was the fact that it was doing something on its own that got some people to say, what could this look like later on?
So, all right, I’ve really enjoyed this, and my impression of you is very different today. Are you from a big family or small family?
Getting to Know Jacob Coxon
JACOB COXON: (01:42:26 – 01:42:27) Small family.
PATRICK BET-DAVID: (01:42:27 – 01:42:27) Yeah, small family.
JACOB COXON: (01:42:28 – 01:42:28) Yeah.
PATRICK BET-DAVID: (01:42:28 – 01:42:33) Last— a lot of siblings, or is it me and my sister? You and your sister? Yeah. Very cool. What are your interests? What do you—
JACOB COXON: (01:42:33 – 01:42:34) My interests?
PATRICK BET-DAVID: (01:42:34 – 01:42:38) What books? You talk about superintelligence. What impacted your life as a young person?
JACOB COXON: (01:42:38 – 01:42:51) Oh man, I played a lot of squash. I played tennis. I’m learning Japanese. I like Japanese culture. I like Japanese books. I like Japanese movies. Yeah, I like a lot of Japanese.
PATRICK BET-DAVID: (01:42:51 – 01:42:54) Like you talk about like anime type of stuff?
JACOB COXON: (01:42:54 – 01:42:54) No, not anime.
PATRICK BET-DAVID: (01:42:54 – 01:42:57) Like the, my son is addicted to anime.
JACOB COXON: (01:42:57 – 01:42:58) He’s addicted to anime. I mean, it’s good stuff, dude. It’s good stuff.
PATRICK BET-DAVID: (01:42:59 – 01:42:59) I can’t—
JACOB COXON: (01:42:59 – 01:43:01) Every night after, you know what animes he likes?
PATRICK BET-DAVID: (01:43:01 – 01:43:09) Oh, Jujutsu Kaisen. Yeah, is that the one? And then there’s another one that, what’s the other one he talks about? He’s got like 2 or 3 of them he talks about.
JACOB COXON: (01:43:09 – 01:43:11) Like One Piece?
PATRICK BET-DAVID: (01:43:12 – 01:43:23) Yeah, I don’t know which one. He is like all into it, obsessed, and it all of it, and he’s recruiting everybody into it. Yeah, his younger brother now is addicted to it. It’s like everybody, the moment he gets them locked in.
JACOB COXON: (01:43:23 – 01:43:25) They do it so well. They’ve got some special sauce.
PATRICK BET-DAVID: (01:43:25 – 01:43:26) Are you a Harry Potter guy or no?
JACOB COXON: (01:43:26 – 01:43:28) Yeah, I’m into Harry Potter. I was when I was younger.
PATRICK BET-DAVID: (01:43:29 – 01:43:29) You were when you were young?
JACOB COXON: (01:43:30 – 01:43:30) Yeah, I used to read it.
PATRICK BET-DAVID: (01:43:30 – 01:43:42) My youngest son watched Harry Potter twice, all 8 of them. Twice in one week when we were in the Hamptons this summer. And guess what his birthday party was around? The theme. 2 days ago, Harry Potter. Harry Potter. 3 days ago.
JACOB COXON: (01:43:42 – 01:43:42) How old is he?
PATRICK BET-DAVID: (01:43:43 – 01:43:45) He just became a teenager. He’s 13 years old.
JACOB COXON: (01:43:45 – 01:43:46) Nice.
PATRICK BET-DAVID: (01:43:46 – 01:43:47) Just became a teenager.
JACOB COXON: (01:43:47 – 01:43:51) So that’s past the level where you start expecting the Hogwarts letter. That’s the thing in the UK is—
PATRICK BET-DAVID: (01:43:52 – 01:43:52) What is it?
JACOB COXON: (01:43:52 – 01:43:54) You get the letter for the magic school when you’re 11.
PATRICK BET-DAVID: (01:43:55 – 01:43:55) Yeah.
JACOB COXON: (01:43:55 – 01:43:57) So all these kids get very sad when they turn 11.
PATRICK BET-DAVID: (01:43:58 – 01:44:07) It’s so funny. He is like fully into it. Yeah, he is fully into it. Everything was that, at the house. But I love it’s the imagination and the creativity.
JACOB COXON: (01:44:07 – 01:44:08) Yeah, right.
PATRICK BET-DAVID: (01:44:08 – 01:44:14) But Jacob, I, one of my favorite conversations of the year with you, and I’ve talked to a lot of different people.
JACOB COXON: (01:44:14 – 01:44:14) I really appreciate it.
PATRICK BET-DAVID: (01:44:15 – 01:44:38) I love how sincere you are. I hope the audience also, whether they agree with you or disagree with you, whether you are somebody, oh my God, he’s from the NGO, he’s doing this, he’s doing that, I just got a lot of sincerity from you, and I appreciate that. It’s very hard to get that nowadays, and good for you for being who you are.
Yeah, truly. Thank you for coming down. Really enjoyed this conversation. And is there anywhere you want people to find you? Is it just Twitter? Is that—
JACOB COXON: (01:44:38 – 01:44:39) Twitter’s great. Yeah.
PATRICK BET-DAVID: (01:44:39 – 01:44:47) Okay, we’ll put the link below as well. Take care, everybody. Bye-bye. Bye-bye.
Related Posts
- Transcript: How AI Swarms Are Already Going Rogue: Daniel Kokotajlo
- Transcript: Elon Musk & Jensen Huang Speak At America.Gov Event
- Transcript: Prof. Nick Bostrom on When AI Escapes Our Control – Peter McCormack Show
- Transcript: CMG Interview with Tesla CEO Elon Musk
- Transcript: UN Security Council AI Hearing w/ Sam Altman, Dario Amodei & Clément Delangue
