Transcript: Aaron Bastani Interviews Karen Hao on Downstream IRL

Because that is such an expansive term. It can’t really… It’s political convenience is precisely. It doesn’t mean anything.

KAREN HAO: It does frustrate me a little bit. So I often use the analogy that AI is like the word “transportation.” I mean if transportation can refer to bicycles or rockets or self-driving cars or gas guzzling trucks, they’re all different modes of transportation, serve different purposes, different cost benefit analyses. And you would never have a politician say “we need more transportation to mitigate climate change.” You would be like, but what kind of transport? What are you talking about?

Well, yeah, “we need more transportation to stimulate the economy.” I mean, maybe in that case it’s like, there is a vagueness around the AI discussion that is really unproductive. And I think a lot of that leads to confusion where people think AI equals one thing and AI equals progress and so we should just have all of it.

But actually if we were to use the transportation analogy, having more bicycles, having more public transit sounds great. But if someone were actually referring to just using rockets to commute from Dublin to London and we were like, “everyone should get a rocket now, that’s going to bring us more progress.” You would be like, what are you talking about? And that’s effectively what these companies are doing with artificial general intelligence.

AARON BASTANI: When you’re giving people tools for free with regards to generative AI to just generate stupid images of nonsense. That’s kind of what we’re doing, right? I presume you would take that analogy to that level. It’s like saying let’s use a rocket to get from Dublin to London to Paris.

KAREN HAO: Yeah, exactly. It’s not fit for the task and the extraordinary amount of environmental costs for flying that rocket when you could have flown a much more efficient plane to do the same thing is like, what are you doing?

And that’s one of the things that people don’t really realize about generative AI, is that the resource consumption required to develop these models and also use these models is quite extraordinary. And oftentimes people are using them for tasks that could be achieved with highly efficient, different AI techniques. But because we use the sweeping term AI to mean anything, then people just think, “oh, yeah, I’m just going to use ChatGPT for my one stop shop solution for anything AI related.”

The Environmental Cost of AI

AARON BASTANI: So right now, data centers globally I think are about 3, 3.5% of CO2 emissions. I think the data centers for AI are a tiny fraction of that, but obviously they’re growing at an extraordinary pace.

KAREN HAO: Yeah.

AARON BASTANI: Are there any numbers out there with regards to projected CO2 emissions of data centers globally 5, 10, 15 years from now, or is that also… It’s so recent that we can’t really speculate about the numbers involved.

KAREN HAO: There are numbers around the energy consumption which you could then use to kind of try and project carbon emissions. So there was a McKinsey report that recently projected that based on the current pace of data center and supercomputer expansion for the development and deployment of AI technologies, we would need to add around half to 1.2 times the amount of energy consumed in the UK annually to the global grid in the next five years.

AARON BASTANI: Wow.

KAREN HAO: Yeah. And most of that will be serviced by fossil fuels. This is something that Sam Altman actually even said in front of the Senate a couple weeks ago. He said it will most probably be natural gas. So he actually picked the nicest fossil fuel. But we are already seeing reports of coal plants having their lives extended. They were meant to be retired, but they’re no longer being retired explicitly to power data center development.

We’re seeing reports of Elon Musk’s xAI, the giant supercomputer that he built called Colossus in Memphis, Tennessee. It is being powered with around 35 unlicensed methane gas turbines that are pumping thousands of toxic air pollutants into the air, into that community. So this data center acceleration is not just accelerating the climate crisis, it also is accelerating the public health crisis of people’s ability to access clean air as well as clean water.

The Water Crisis: AI’s Hidden Environmental Impact

So one of the aspects that’s really under talked about with this kind of AI development, the OpenAI’s version of AI development, is that these data centers need fresh water to cool because if they use any other kind of water, it would erode, corrode the equipment, it would lead to bacterial growth. And so most often these data centers actually use public drinking water because when they enter into a community, that is the infrastructure that’s already laid to deliver the fresh water to companies, to businesses, to residents.

And so one of the things that I highlight in my book is there are many, many communities that are already, they do not have sufficient drinking water even for people. And I went to Montevideo, Uruguay to speak with people about historic level of drought that they were experiencing, where the Montevideo government literally did not have enough water to put into the public drinking water supply. So they were mixing toxic wastewater in just so people could have something come out of their taps when they opened them.

And for people that were too poor to buy bottled water, that is what they were drinking. And women were having higher rates of miscarriages, elderly were having an exacerbation or an inflammation of their chronic diseases. And in the middle of that, Google proposed to build a data center that would use more drinking water.

AARON BASTANI: This is called potable water, right? This is potable water.

KAREN HAO: Yeah, exactly.

AARON BASTANI: You can’t use sea water because of the saline aspect.

KAREN HAO: Exactly, exactly. And Bloomberg recently had a story that said two-thirds of the data centers now being built for AI development are in fact going into water scarce areas.

Corporate Lawlessness and Democratic Erosion

AARON BASTANI: You said a moment ago about xAI, unlicensed energy generation using methane gas. When you say unlicensed, what do you mean?

KAREN HAO: As in the company just decided to completely ignore existing environmental regulations when they installed those methane gas turbines. And this is actually one of the things that I concluded by the end of my reporting was not only are these companies really corporate empires, but also that if we allow them to be unfettered in their access to resources and unfettered in their expansion, they will ultimately erode democracy. That is the greatest threat of their behaviors.

And what xAI is doing is a perfect example of at the smallest level, these companies are entering into communities and completely hijacking existing laws, existing regulations, existing democratic processes to build the infrastructure for their expansion. And we’re seeing this hijacking of the democratic process at every level, the smallest local levels, all the way to the international national level.

AARON BASTANI: It’s kind of that orthodoxy of seek permission after you do something is now, I mean, when you start applying…

KAREN HAO: This is business as usual for those companies. That’s part of their expansion strategy, which we’ll talk about.

The Major Players in the AI Race

AARON BASTANI: And we’re going to talk about the sort of global colonial aspect as well regards to resource consumption, resource use, just bring it back to the US Again, because at the top of this conversation I want to offer a bit of a primer to people out there who they maybe know what AI is. They maybe have used ChatGPT.

What are the major companies we’re now talking about in this space, particularly in the United States of America over the last five years? Who are the people in this race to AGI? Allegedly Artificial General Intelligence. Something which either might be sentient, probably not, or capable of augmenting its own intelligence. More plausible. Who are the major players in that field right now?

KAREN HAO: One caveat on AGI is that it’s as ill defined as the term AI. So I like to think of it as just a rebranding. The entire history of AI has just been about rebranding and the term deep learning was also a rebranding.

So anyway, but the players, OpenAI, of course, they were the ones that fired the first shot with ChatGPT. Anthropic major competitor, Google, Meta, Microsoft, they’re the older Internet giants that are now also racing to deploy these technologies. Safe Superintelligence, which spun out of also an OpenAI splinter. There are many OpenAI splinters.

So this was founded very recently by the former Chief Scientist of OpenAI and thinking machines Lab, founded very recently by a former chief technology officer of OpenAI. And Amazon is now trying to get into the game as well, and Apple is also trying to get in the game. So basically all of the older generation tech giants, as well as a new crop of AI players are all jostling in this space.

AARON BASTANI: And that’s just the US, right?

KAREN HAO: And that’s just the US right. So the Chinese ecosystem is interesting because they’re not. So they don’t really use the term AGI. This is like a very kind of unique thing about the US ecosystem is that there’s a quasi religious fervor around that underpins the construction of AI products and services.

Whereas in China it’s much more like these are businesses we’re building products that users are going to use. So if you’re just looking at companies that are building chatbots that are sort of akin to ChatGPT, then we’re talking about ByteDance, owner of TikTok, Alibaba, the equivalent of Amazon, Baidu, the equivalent of Google, Huawei, the equivalent of Apple, and Tencent. The what is the equivalent of Tencent? I guess Meta is the equivalent of Tencent. So they’re also building on these things. And there’s similarly a crop of startups that are moving into the generative AI space.

The Business Case for AI Investment

AARON BASTANI: And in Europe, we’ve got the little tiddlers like Mistral in France, you know, really not at the races because we’re Europe. What’s the business case for all this? Because obviously you’ve got massive companies, often driven by maximizing shareholder value, multitrillion dollar valuations. You do these things, you invest money to make money as a capitalist society.

So what is the business case made by, say, Microsoft when they have their shareholder meetings and they say, “we’re going to allocate 40, 50 billion dollars towards building data centers and so on”?

KAREN HAO: So it’s really, it’s interesting that you mentioned Microsoft because Microsoft has recently been pulling back their investments in data centers. They went all in, and now they’re really rapidly starting to abandon data center projects.

So to answer your question, it is really unclear what the business case is. And Microsoft has been one of the first companies to start acknowledging that. And Satya Nadella has come onto some podcasts recently where he actually stunned some people in the industry by being quite skeptical of whether or not this race to AGI was productive.

But one of the things that I really felt after reporting what is driving the fervor is you can’t actually fully understand it as just a story about money. It has to also be understood as a story of ideology. And because when in the absence of a business case, then you ask, why are people still doing this?

And the answer is there are people who genuinely, fervently believe, and they talk about it as a belief in this idea that we can fundamentally recreate human intelligence. And that if we can do that, there is no other more important thing in the world. Because what else? How else should you be dedicating your time other than to bring about this civilizationally transformative technology?

And so that’s part of why. What drives OpenAI? What drives anthropic? What drives safe superintelligence? These other smaller startups, and then the bigger giants, which are more business focused and more classic companies that actually care about their bottom lines, they end up getting pressured because shareholders are seeing the enormous amounts of investment by these startups and they’re seeing users start shifting from Google search to using ChatGPT as search.

ChatGPT should not be used as search, but consumers think that it is. And then shareholders ask in Google’s shareholder meetings, “what are you doing with AI? What is your AI strategy? Why aren’t you investing in this technology?” And so then all of the other giants end up racing in the same direction.

AARON BASTANI: What does Warren Buffet make of it? That’s what I want to know. Is he sort of like, if he’s like, “you guys are all wasting your money,” he’s probably right. I have no idea.

KAREN HAO: Has he invested in AI?

AARON BASTANI: I don’t think so. He just sticks to coke and these sorts of things, doesn’t he? I mean, there’s two rationales. So I think one is, like you say, a quasi religious fervor has inflected the investment decisions of some of the world’s most valuable companies, which is just an extraordinary thing to even think about, I suppose.

The other one is that a lot of people in this space, as we’ll talk about in a moment, are heavily influenced by people like Peter Thiel. And Peter Thiel’s orthodoxy is that competition is for idiots. Right. If you’re going to start a business, it has to be a monopoly.

And I can only presume that companies like Microsoft, etcetera, although maybe that’s not the best example now given recent events. But Xai OpenAI matter. The only reason you would invest ultimately hundreds of billions, trillions of dollars into this is because First Mover Advantage gives you monopoly on the most transformational technology since the Steam engine. I mean, that’s the only way I can make sense of it, right?

Has anybody in that space kind of said that “we want the monopoly on AGI. We want to be the Facebook of AGI”?

KAREN HAO: Well, what OpenAI often says to investors is if you make this seemingly fantastical bid into our technology, you could get the biggest returns you’ve ever seen in your life, because we will then be able to use your funding to get to AGI first. So it’s still riding on this concept of the fact that there might be an AGI, which is high, it’s not rooted in scientific evidence.

And even if we fail, we will successfully be able to automate a lot of human tasks to the point where we can convince a lot of executives to hire our software instead of a labor force so that in and of itself could potentially end up generating enough returns for you more than you’ve ever seen before. So that’s usually the pitch that they make.

But, you know, it is a huge risky bargain that these investors are actually pitching into. And, you know, a lot of investors, they have a bandwagon mentality like they aren’t necessarily doing their own analysis to say, “let me do this investment.” They’re just seeing everyone glom onto this thing and they’re like, “well, I don’t want to miss out. Why don’t we glom on as well.”

But there are some investors that have actually recently reached out to me to be like. One of the most underreported stories right now is the amount of risk that is not just being taken on by these VCs is actually being taken on by the entire economy. Because the money that these investors are investing comes from university endowments and things like that. So if the bubble pops, it doesn’t just pop for Silicon Valley, it actually will have ripple effects across the global economy.

The Origins of OpenAI

AARON BASTANI: I mean, when you look at the sort of e commerce bubble in the late 90s, okay, it was a bubble, you know, pets.com or whatever had these crazy valuations, but buying and selling goods and services offline and then taking that online, I mean that makes sense. That’s a plausible sort of commercial model.

But like you say, nobody’s really done that with artificial intelligence. It does kind of feel like you read these stories about tulip mania in 17th century Holland and it does kind of feel very similar. You mentioned OpenAI and we’ve talked about it many times. And of course OpenAI is the central organization in this book. What’s the big idea behind OpenAI? When it starts and when does it start?

KAREN HAO: Let’s end of 2015.

AARON BASTANI: 2015. So it’s 10 years old. What are the animating values that give birth to OpenAI?

KAREN HAO: So OpenAI started as a nonprofit, which many people don’t realize based on the fact that it’s one of the most capitalistic, if not the most capitalistic organization in Silicon Valley today. But it was co founded by Elon Musk and Sam Altman as a bid to try and create a fundamental AI research lab that could develop this transformative technology without any kind of commercial pressures.

So they positioned themselves as the anti Silicon Valley, the anti Google. Because Google at the time was the main driver of AI development. They had developed a monopoly on some top AI research scientists. And Musk in particular had this really great fear of not just Google, but Google’s acquisition of DeepMind, where he was very worried that this consolidation of some of the brightest minds would lead to the development of AI that would go very badly wrong.

And what he meant by very badly wrong was was it could one day develop sentience consciousness, go rogue and kill all humans on the planet. And because of that fear, Altman and Musk then thought, “we need to do a nonprofit, not have these profit driven incentives. We’re going to focus on being completely open, transparent and also collaborative to the point of self sacrificing if necessary. If another lab starts making faster progress than us on AI and on the quest to AGI, we will actually just join up with them. We will dissolve our own organization and join up with them.”

And that didn’t hold for very long.

AARON BASTANI: So what’s the theory behind that? Because, you know, at that point, Google is Now Alphabet, maybe 2015 is maybe the world’s most valuable company. I don’t know. It’s certainly up there. And this is a nonprofit. So how are they going to achieve AGI before Google?

KAREN HAO: So initially, the bottleneck that they saw was talent. Right, Google has this monopoly in talent. We need to chip away at that monopoly and get some of those Google researchers to come to us and also start acquiring PhD students that are just coming out of uni.

And because of that, I have come to speculate, this is not based on any documents that I read or anything. I’ve come to speculate that part of the reason why they started as a nonprofit in the first place, it was a great recruitment tool for getting at that bottleneck. They could not compete on salaries with Google, but they could compete on a sense of mission.

And in fact, when Altman was recruiting the chief scientist, Ilya Sutskever, who was the critical first acquisition of talent that then led to many other scientists being really interested in working for OpenAI, he appealed to Sutskever’s sense of purpose. Like, “do you want a big salary and just to work for a for profit company, or do you want to take a pay cut and do something big with your life?”

And it was actually that reason that Sutskever said, “you know what, you’re right, I do want to work for a nonprofit.” And so that’s how they initially conceived of competing with Google, was, we’re starting a little bit late to the game. How do we first get a bunch of really, really smart people to join us? Let’s create this really big sense of mission.

And I open the book with two quotes in the epigraph, and one of them is from Sam Altman writing a blog post in 2013. And he quotes someone else that says, “successful people build companies, more successful people build countries. The most successful people build religions.”

And then he reflects on this and says, “it seems to me that the most successful founders in the world don’t actually set off to build a company, they off to build a religion. And it turns out building a company is the easiest way to do so.” And so it’s not like 2013 and then 2015, he creates OpenAI as a nonprofit.

The Nonprofit Facade and Capital Reality

AARON BASTANI: It’s important to say as well, Sam Altman is not some sort of idealistic pauper. He’s working at Y Combinator. He is very much enmeshed within the Silicon Valley elite. I suppose also there’s tax as well, right? If you’re a nonprofit, you’ve got the mission. You’ve also got a bunch of tax breaks which you don’t have as a for profit. So maybe there’s a very cynical genesis there.

But I suppose just reading your book and becoming more familiar with the arguments over time, clearly the amount of compute you have was always going to be critical. If you believe on the neural network model, the deep learning model, the amount of compute you have is always going to be critical. And it just seems implausible that a nonprofit could ever have been able to compete with Google, for instance, ever. Like it seems implausible because you have to spend, as we now see, tens of billions, hundreds of billions of dollars on compute. Did nobody say that? Did nobody say, “Hey, you know, like the bottleneck isn’t just talent action, it’s being able to spend hundreds of billions of dollars on these Nvidia GPUs.”

KAREN HAO: It’s so interesting because at the time the idea that you needed a lot of compute was actually neither very popular nor one that was seen as that scientifically rigorous. So there were many different ideas of how to advance AI. One was we already actually have all the techniques that we need and we just need to scale them. But that was considered a very extreme opinion.

And then on the other extreme it was we don’t even have the techniques yet. And interestingly, recently there’s a New York Times story that says “Why We Likely Won’t Get to AGI Anytime Soon” by Kate Metz. And he cites this stat that 75% of the longest standing, most respected AI researchers actually still think to this day we don’t actually have the techniques to get to AGI if we will ever. So we’re kind of coming full circle now and it is starting to become unpopular again, this idea that you can just scale your way to so called intelligence.

But that was the research vibe when OpenAI started, was we can actually maybe just innovate on techniques, right? And then very quickly, because Ilya Sutskever in particular was a scientist who anomalously did think that scaling was possible and because Altman loved the idea of adding zeros to things from his career in Silicon Valley and because Greg Brockman, the Chief Technology officer, also very Silicon Valley entrepreneur, liked that idea as well. Then they identified, “Why don’t we go for scale? Because that is going to be the fastest way to see whether we can beat Google.”

The For-Profit Conversion and Musk’s Exit

And once they made that decision, about less than a year in roughly, is when they started actually talking about that. That’s when they decided we actually need to convert into a for profit. Because the bottleneck has shifted now from acquiring talent to acquiring capital.

And that is also why Elon Musk and Sam Altman ended up having a falling out. Because when they started discussing a for profit conversion, both Elon Musk and Sam Altman each wanted to be the CEO of that for profit. And so they couldn’t agree. And originally Ilya Sutskever and Greg Brockman chose Musk. They thought that Musk would be the better leader of OpenAI.

But then Altman essentially, and this is something that is very classic, a very classic pattern in his career, became very persuasive to Brockman, who he had had a long term relationship with, about why it could actually be dangerous to go with Musk and like, “I would definitely be the more responsible leader,” so on and so forth. And then Brockman convinced Sutskever and the two chief scientist, chief Technology officer pivot their decision and they go with Altman. And then Musk leaves in a huff and says, “I don’t want to be part of this anymore.”

AARON BASTANI: Which has become rather typical of the man, hasn’t it, subsequently? But that is incredible, really. So by 2016, there’s a recognition that in terms of capital investment, they’re going to have to go toe to toe with maybe at that point the world’s biggest company. And they’re a nonprofit. I just find it weird that. But lots of people bought the propaganda that OpenAI was in some way open.

KAREN HAO: Yeah.

AARON BASTANI: What did the “Open” stand for, by the way?

KAREN HAO: The “Open” originally stood for Open Source, which in the first year of OpenAI, they really did open source things. They did research and then they would put all their code online. So it really was like they did what they said. And then the moment that they realized we got to go for scale, then everything shifted.

From Activism to $300 Billion Valuation

AARON BASTANI: It’s such an amazing story and so emblematic of the 2010s that you have this organization which presents itself as effectively an extension of activism, ends up becoming. Today, some people value OpenAI at $300 billion and it’s doing all these terrible things which we’re going to talk about.

Sam Altman, specifically, who is he? What’s his background. How does this guy who nobody’s heard of become the CEO of a company which today is, it’s almost more valuable than any company in Europe, for instance.

KAREN HAO: Yeah. Altman is. He’s spent his entire career in Silicon Valley. He was at first a founder, a startup founder himself. And he was part of the first batch of companies that joined Y Combinator, one of now today one of the most prestigious startup accelerators in Silicon Valley. But at the time he was the very first class and no one really knew what YC was.

He did that for seven years. He was running a company called Looped, which was a mobile based social media platform, effectively a Foursquare competitor, but which actually started earlier than Foursquare. It didn’t do very well, it was sold off for parts.

But what he did do very well during that time was ingratiate himself with very powerful networks in Silicon Valley. So one of the first and longest mentors that he ended up having throughout his career is Paul Graham, the founder of Y Combinator, who then plucked Sam Altman to be his successor. And Sam Altman then at a very young age became president of YC and then he ended up doing that for around five years.

Altman’s Strategic Positioning

And during his tenure at YC, he dramatically expanded YC’s portfolio of companies. He started investing not just in software companies, but also pushing into quantum, into self driving cars, into fusion, and really going for those hard tech engineering challenges.

And if you look at how he ended up then as a CEO of OpenAI, I think that he basically was trying to figure out what is going to be the next big technology wave. “Let me test out all of these different things, position myself as involved in all of these different things.” So in addition to all his investments, he started cultivating this idea of AI also. “Seems like maybe it’ll be big. Let me start working on an idea for a fundamental AI research lab” that becomes OpenAI. And once OpenAI started being the fastest one taking off, then Altman hops over and becomes CEO.

AARON BASTANI: He hops over. So how does that happen? Where does he come from? Because like you say originally, it’s got people like, he’s there. Who’s there first? Him or Ilya Sutskever.

KAREN HAO: Technically, Altman recruited Sutskever, but Altman was only a chairman. He didn’t take an executive role at OpenAI, even though he founded the company. And similarly with Musk, Musk didn’t have an executive role, he was just a co chairman. So it was just the two of them that were chairman of the Board and Ilya Sutskever and Greg Brockman were the main people, the main executives that were actually running the company day to day in the beginning.

The Master Manipulator

AARON BASTANI: I mean, I have to say, reading the book, Sam Altman, he comes across as a master manipulator, like masterful manipulator and understander of human psychology. There’s this great quote, let me get it up, which you have. I think it’s from Paul Graham. Sam Altman has it. “You could parachute him into an island full of cannibals and come back in five years and he’d be the king. If you’re Sam Altman, you don’t have to be profitable to convey to investors that you’ll succeed with or without them.”

I mean, he just sounds, he’s also described, by the way, as a once in a generation fundraising talent. I think that’s by you.

KAREN HAO: Yeah.

AARON BASTANI: How is he able to just basically come out of nowhere and compete with people like Elon Musk Zuckerberg as this kind of intellectual heavyweight in Silicon Valley in regards to one of the major growth technologies of our decade?

KAREN HAO: So from the public’s perspective, he came out of nowhere. But within the tech industry, everyone knew Sam Altman. You know, like I, as someone who worked in tech, like I knew Sam Altman ages ago because Y Combinator was just so important.

AARON BASTANI: It was as a CEO of central company that valuable. Was it always something that he might be in?

The Valley’s Gatekeeper

KAREN HAO: No, I don’t think people ever thought that he would jump to become the CEO of a company because he has such an investor mindset. And his approach has always been to be involved in many, many companies. I mean, he invested in hundreds of startups as both the president of YC and running some personal investment funds as well.

But people, he was well respected within the Valley. He was seen as a critical linchpin of the entire startup ecosystem, and not just by people within the industry, but by policymakers, which is key. He started cultivating relationships with politicians very, very early on in his tenure as the president of YC.

And for example, I talk in my book about how Ash Carter, the head of the Department of Defense under the Obama administration, came to Altman asking, “How can we get more young tech entrepreneurs to partner with the U.S. government?” So he was seen as a gateway into the Valley.

And obviously the Valley isn’t just made up of startups. There’s also the tech giants. But back then, starting a startup was way cooler than working at a tech giant because Google, Microsoft, they were considered the older, safer options if you really wanted job security. But if you wanted to be an innovator, if you wanted to do breathtaking things, you would build a startup and then that start. Your number one goal as a startup founder was to get into YC. So Altman was the pinnacle. He was emblematic of the pinnacle of success in the Valley.

AARON BASTANI: And he. Even if his net worth wasn’t the same as other people in terms of his social capital, his networking, he understood early on that’s where the real value lies.

KAREN HAO: Exactly.

The Uncomfortable Truth About Leadership

AARON BASTANI: So interesting. I mean, some notes that I wrote down, because there are points where I’m thinking, why on earth is this gentleman the CEO of such a valuable company? Seems kind of useless. And the notes I had down were people pleaser.

KAREN HAO: Yes.

AARON BASTANI: Liar. Conflict averse. How do you become the CEO of such a successful company? Maybe you think that or don’t think that. I don’t know. I mean, at points it comes across as almost psychotic, the capacity to lie.

Here’s an interesting question for me, and I don’t know how comfortable you are with answering it. In writing this book, there’s another alternative timeline where you basically write a hagiography of Samuel and you leave all of that out, right? There are other writers out there. I won’t name them. They sell a ton of books and they write very positive, affirming biographies of these visionary leaders, whether it’s Elon Musk or Steve Jobs, et cetera.

Why didn’t you just write that book about Sam Altman? You know, you would have made a ton more money, right? But I’m reading this stuff and I’m thinking, why couldn’t this. And it’s so deft and nuanced, your portrait of Sam Altman. I just think the guy. I mean, this is going to really hurt him when he reads this stuff, I imagine. Why didn’t you do that? Take the easy route?

The Challenge of Documenting Sam Altman

KAREN HAO: I don’t know that that would have been the easy route. I mean, I just wrote the facts and the facts come out that way. You know, I interviewed over 260 people across 300 different interviews. And over 150 of those interviews were with people who either worked at the company or were close to Sam Altman. And that’s just what they presented, was all of the details that I ended up putting in.

And one of the things that just came through again and again. Two things that came through again and again. No matter how long someone worked with him or how closely they worked with him, they would always say to me, “At the end of the day, I don’t know what Sam believes.” So that’s interesting.

And then the other thing that came through was I would ask them, “Well, what did he say to you? He believed in this meeting, at this point in time for why the company needs to do this XYZ thing.” And the answer was, he always said he believed what that person believed, except because I interviewed so many people who have very divergent beliefs, and I was like, wait a minute, he’s saying that he believes what this person believes and then what that person believes, and they’re literally diametrically opposite.

So I just ended up documenting all of those different details to illustrate how people feel about him. I mean, he’s a polarizing figure, both extreme in the positive and negative direction. Some people feel he is the greatest tech leader of our generation. And they don’t say that he is honest when they say that. They just say that he’s one of the most phenomenal assets for achieving a vision of the future that they really agree with.

And then there are other people who hate his guts and say that he’s the greatest threat ever. And it really also comes down to whether or not they agree with his vision and they don’t. And so then his persuasive powers suddenly become manipulative tactics.

Comparing Leadership Styles

AARON BASTANI: I mean, if you compare them to somebody like Elon Musk as a CEO who is obviously far from perfect. But Elon Musk makes big bets. He has gut instincts. He’s very happy to alienate people if he thinks he’s right about something. And obviously I don’t agree with him on many, many things, but that’s quite a sort of. There’s an archetype with regards to a business leader that looks like that.

And then you’ve got somebody like Sam Altman. He’s doing all of these things, the people pleasing, the conflict aversion. And yet he’s managed to lead this company to essentially a third of a trillion valuation. He must obviously be doing something right as well. So what are his sort of comparative advantages as a business leader? Because on paper, I read all that stuff, and I think the guy wouldn’t be able to get up in the morning and make breakfast. And yet he’s accomplished some extraordinary things.

KAREN HAO: Yeah, I think it really comes down to he really does understand human psychology very well, which not only is helpful in getting people to join in on his quest. So he’s great at acquiring talent. And then he’s said himself, “I’m a visionary leader, I’m not an operational leader and my best skill is to acquire the best people that then operationalize the thing.”

So he’s good at persuading people into joining his quest. He’s good at persuading whoever has access to whatever resource he needs to then give him that resource, whether it’s capital, land, energy, water, laws. And then he is. People have said that he instills a very powerful sense of belief in his vision and in their ability to then do it.

AARON BASTANI: He’s good. We say in English soccer, we would say good man manager. He can inspire people.

KAREN HAO: He inspires people to do things that they didn’t think that they would be able to do. Yeah, but I mean, this is why there’s so much controversy. He is such a polarizing figure because everyone has a very personalized encounter relationship with him because he often, he does his best work in one on one meetings when he can say whatever he needs to say to get you to do, believe, achieve whatever it is that he needs you to do.

And that’s also part of the reason why there’s so many diverging, people that are like, “Oh, I think he believes this, I think he believes that.” And they’re totally divergent. It’s because he’s having these very personalized conversations with people.

And so some people end up coming out of those personalized meetings feeling totally transformed in the positive direction, being like, “I feel superhuman. I can now do all these things and it’s in the direction that I want to go. I’m building the future that he sees and I see and we’re aligned.”

And then other people end up coming out of these meetings feeling like, “Was I played? Was he just telling me all these things to try and get me to do something that’s actually fundamentally against my values?”

The Scale of Research

AARON BASTANI: You said you spoke to 150 people who were connected with OpenAI?

KAREN HAO: Over 200 or 150 interviews.

AARON BASTANI: Yeah, sorry, 150 interviews. 250 interviews altogether. 270.

KAREN HAO: 270 people altogether.

AARON BASTANI: The numbers are all nobody. It’s absolutely incredible. I should have said this right at the start, really. What’s your personal sort of bio on all this stuff? Because of course, when people out of journalism, media cover technology, the intersection of that with politics, we’ve gone, well, they don’t really know what they’re talking about. They’re generalists because they come out of journalism. What’s your background? Because it’s quite particular.

From Engineering to Journalism

KAREN HAO: I studied mechanical engineering at MIT for undergrad, and I went and worked in Silicon Valley because that’s what I thought I wanted to do. I lasted a year before I realized it was absolutely not what I wanted to do. And then I went into journalism.

And the reason why I had such a visceral reaction against Silicon Valley is because I was quite interested in sustainability and how to mitigate climate change. And why I went to study engineering in the first place was I thought that technology could be a great tool for social change and shaping consumer behaviors to prevent us from planetary disaster.

And I realized that Silicon Valley’s technology incentive structures for producing technology were not actually leading us to develop technologies in the public interest. And in fact, most often it was leading to technologies that were eroding the public interest. And the problems like mitigation of climate change that I was interested in were not profitable problems. But that is ultimately what Silicon Valley builds. They want to build profitable technologies.

And so it just seemed to me that it didn’t really make sense to try and continue doing what I wanted to do within a structure that didn’t reward that. And then I thought, well, I’ve always liked writing. Maybe I can use writing as a tool for social change. So I switched to journalism.

AARON BASTANI: You went to MIT Review. Right.

KAREN HAO: And then I went to a few publications, and then eventually MIT Technology Review to cover AI, and then Wall Street Journal.

AARON BASTANI: I mean, these are big. Just so people know there’s real credibility behind this, all these interviews, this CV. And it’s interesting as well, you say, I wouldn’t write a hagiography. I just wrote what was there. I mean, maybe that’s partly an extension of your sort of STEM background, right? Rather than writing propaganda on a puff piece, which, let’s be honest, is most coverage of this sector.

The MIT Advantage

KAREN HAO: Well, you know, people often ask me, how much did my engineering degree help me in reporting on this? And I think it helps me in ways that are not what people would typically assume. I went to school with a lot of the people that now build these technologies. I went to school with some of the executives at OpenAI.

And so for me, I do not find there to be magic. I don’t find these figures to be towering or magical. I remember when we were walking around dorm rooms together in our pajamas. And it instills in me this understanding that technology is always a product of human choices and different humans will have different blind spots.

And if you give a small group of those people too much power to develop technologies that will affect billions of people’s lives, inevitably that is structurally unsound. We should not be allowing small groups of individuals to concentrate such profound influence on society when it is not. You cannot expect any individual to have such great visibility into everything that’s happening in the world and perfectly understand how to craft a one size fits all technology that ends up being profoundly beneficial for everyone. That just doesn’t make sense at all.

And I think the other thing that it really helps me with is Silicon Valley is an extremely elitist place and it allows me to have an honest conversation with people faster. Because if they start stonewalling me or trying to pretend that there are certain things that these technologies are capable of that they’re not actually capable of, I will just slap my MIT degree down and be like, “Cut the bull crap, tell me what’s actually happening.”

And it is a shortcut to getting them to just speak more honestly to me. But it’s not actually because of what I studied. It’s more just that it signals to them that they need to speed up their throat clearing.

AARON BASTANI: That’s really interesting though.

KAREN HAO: Yeah.

The Threat to Democracy

AARON BASTANI: Because I do feel like lots of coverage of this sector. I mean, again, I can only speak in regards to the UK and we’re a tiddler compared to you guys. But at the intersection of particularly politics and technology, the coverage by political journalists at Westminster, Keir Starmer and Rachel Reeves say we’re going to build more data centers. Isn’t that fantastic? Actually, not necessarily. They’re not going to create that many jobs once they’re built. They can use a ton of energy, ton of water. What’s the upside for the UK taxpayer? There is very little interrogation of just the press releases.

And it’s really interesting to me that you’ve come out of MIT and then you’ve taken this trajectory. Is this stuff you just talked about, knowing these people, this tiny group of people whose decisions now affect billions already, is this stuff on the present trajectory? Is it an existential challenge to democracy? Challenge is speculative. Is it going to end democracy?

KAREN HAO: I think it is greatly threatening and increasing the likelihood of democracy’s demise. But I never make predictions of, “This outcome will happen” because it makes it sound inevitable. And one of the reasons why I wrote the book is because I very much believe that we can change that and people can act now to shape the future so that we don’t lose democracy.

AARON BASTANI: But on this trajectory. Right. If the next 20 years are like.

KAREN HAO: The last 10 years on this trajectory, for sure, I think it will end democracy. Yeah.

AARON BASTANI: How quickly we’ve really screwed up in the last 20 years.

KAREN HAO: Right.

AARON BASTANI: I wonder, you know, it’s kind of.

KAREN HAO: Yeah, I’ll give it maybe 20 years.

The Environmental Cost

AARON BASTANI: 20 years, yeah. We used to have this thing called privacy. High streets, childhood, all gone. You’ve said that what OpenAI did in the last few years is they started blowing up the amount of data and the size of the computers that need to do this training in regards to the deep learning. Give me a sense of the scale. We’ve talked a little bit about the data centers, but how much energy, land, water, is being used to power OpenAI, just specifically as one company.

KAREN HAO: Yeah. To power OpenAI. That’s really hard because they don’t actually tell us this. So we only have figures for the industry at large and the amount of data centers.

AARON BASTANI: So it’s not in their annual reports, for instance?

KAREN HAO: No, well, they don’t have annual reports because they’re not a public company.

AARON BASTANI: Of course.

KAREN HAO: Yeah. So that’s one of the ways that. And actually it doesn’t matter if they’re a public company because Google and Microsoft, they do have annual reports where they say how much capital they’ve spent on data center construction. They do not break down how much of those data centers are being used for AI.

They also have sustainability reports where they talk about the water and carbon and things like that, but they do not break down how much of that is coming from AI either. And they also massage that data a lot to make it seem better than it actually is.

But even with the massaging, there was that story two years ago, or sorry, last year, 2024, where both Google and Microsoft reported, I think it was a 30% and 50% jump in their carbon emissions because largely driven by this data center development.

AARON BASTANI: Yeah. And also the context here is over the last. It was one of the good news stories of the last sort of 10 to 15 years is that CO2 emissions per capita in the US has kind of plateaued. Right. Across the west had kind of plateaued.

KAREN HAO: And actually in the UK, energy consumption drops.

AARON BASTANI: Yeah. I mean, we stopped making things, but still. You know, everything’s made in East Asia now. But no, but it was kind of a good story and I kind of bought it. Right. I thought that, you know, we’d have. We’d kind of plateaued. Obviously the Global south would consume more energy, but we are as well, should we look at these companies as kind of analogous to the East India company of the 19th century?

The East India Company Analogy

KAREN HAO: That is the analogy that I have increasingly started using, especially with the Trump administration in power, because the British East India Company very much was a corporate empire and started off not very imperial. They just started off as a company, very small company based in London, and of course, through economic trade agreements with India, gained significant economic power, political power, and eventually became the apex predator in that ecosystem.

And that’s when they started being very imperial in nature. And they were the entire time abetted by the British Empire, the nation state empire. So you have a corporate empire, you have a nation state empire.

And I literally see that dynamic playing out now where the US Government is also in its empire era. The Trump administration has quite literally used words to suggest that he wants to expand and fortify the American empire. And he sees these corporate empires like OpenAI as his empire building assets.

And so I think he is probably seeing it in the same way that the British Crown saw the British East India Company of “let’s just let this company acquire all these resources, do all these things, and then eventually we’ll nationalize the company and then India formally becomes a colony of the British Empire.”

So Trump, whatever the equivalent, modern day equivalent would be of nationalizing these companies, is his end game. Like he is helping them strike all these deals and installing all this American hardware and software all around the world with the hope that then those become national assets.

And then, you know, there was actually just a recent op ed in Financial Times from Maria Shake, one of the former EU parliamentarians, who pointed out, like, isn’t it so convenient for the US to get all of this American infrastructure installed everywhere around the world so that the US Government could literally turn it off at any time? I mean, if you want to talk about empire building, there’s that.

But at the same time, these corporate empires are also trying to use the American empire as an asset to their empire building ambitions. So there’s a very tenuous alliance between Silicon Valley and Washington right now in that each one is trying to use the other and ultimately trying to dominate the other.

And there’s a growing popularity in Silicon Valley of this idea of a politics of exit, this idea that democracy doesn’t work anymore. We need to find other ways of organizing ourselves in society. And maybe the best way of organizing ourselves is actually a series of networked companies with CEOs at the top.

So I don’t ultimately know who’s going to win, like the nation state empire or the corporate empire. But either version is bad because all of the people in power now, both the business executives and the politicians, do not actually care at all about preserving democracy.

AARON BASTANI: I mean, the analogy of India is really interesting. So I think I might have my dates wrong. East India Company is running things until 1857. You have the Indian Mutiny, basically an uprising against East India Company. And then of course that commercial endeavor has to be underpinned by the organized violence of the British imperial state.

And it does feel like that could be the next step of what happens with regards to US interests overseas. I suppose one retort would be, well, hold on, it sounds kind of good. I’m a socialist. I kind of like the idea of SpaceX being nationalized. I kind of like the idea of the federal government having a 51% stake in OpenAI and Tesla and Meta. What would you say to that?

KAREN HAO: I don’t necessarily know if my critique is of the nationalization of the company more as like, why are they nationalizing these companies and what are they, what you know, like the, because of this end game mentality of “let’s just let these companies run rampant around the world so that ultimately whatever their assets are, become our assets,” is leading the Trump administration to have a completely hands off approach to AI regulation.

They’re quite literally, they proposed the big beautiful bill which passed the House and is now going up to the Senate with a clause that would, if implemented, put a 10 year moratorium on AI regulation at the state level, which is usually. The state level is usually where regulation, sensible regulation happens in the US.

So they’re doing all of these actions now with wide ranging repercussions that will be very difficult to unwind in the name of this idea that maybe if they just allow these companies to act with total impunity, that it will ultimately benefit the nation state.

How Tech Leaders View the World

AARON BASTANI: How do people like Sam Altman look at the rest of the world outside the US? These kind of tech leaders? And how do they look at Little Britain and Italy? And how do they look at us? What do they think about us? You know, you’ve been inside their minds.

KAREN HAO: Yeah. I mean they see them as resources. They see different territories as different types of resources, which, I mean, is what older empires did. You know, they would look at a map and just draw out the resources that they could acquire in each geography. “We’re going to go here and acquire the labor, we’re going to go here and acquire the lands, we’re going to go here and acquire the minerals.”

I mean that’s literally how they talk. When I was talking with some OpenAI researchers about their data center expansion, there was one OpenAI employee who said, “we’re running out of land and water.” And he was just saying, yeah, we’re just trying to, we’re just trying to look at the whole world and see where else we can place these things.

What other geographies can we find all the conditions that we need to build more data centers. Land without earthquakes, without floods, without tornadoes, hurricanes, all these natural disasters and can deliver massive amounts of energy to a single point and can cool the systems. And they’re looking at that level of abstraction to what are the different pieces of territory and resources that we need to acquire.

AARON BASTANI: And that includes other parts of the West. Yeah, that’s not just the Global South.

KAREN HAO: No, it includes other parts of the west as well. Yeah. So there have been rapid data center expansion in rural communities in both the US and the UK and it always ends up in economically vulnerable communities because those are the communities that often actually opt in to the data center development initially because they are not informed about what it will ultimately cost them and for how long.

And so I spoke with this one Arizona legislator who said, “I didn’t know it had to use fresh water.” And for the UK audience, AZ is a desert territory. There is no, there’s a very, very stringent budget on fresh water. And after that legislator found out, she was like, “I would have never voted for having this data center in.”

But the problem is that there are so few independent experts for the, these legislators, city council members to consult that they’re the only people that they rely on for the information about what the impact of this is going to be are the companies. And all the companies ever say is “we’re going to invest millions of dollars, we’re going to create a bunch of construction jobs up front and it’s going to be great for your economy.”

The Hidden Costs of Data Centers

AARON BASTANI: Yeah. I mean, that’s all we hear about data centers in this country. And it’s a great, it’s a great top line for the Chancellor and the Prime Minister because they can say, say tens of billions of pounds worth of investment. Okay. But in terms of long term jobs, how many. And also, by the way, for that rural community and God knows where, you know, the northeast of England or whatever.

KAREN HAO: Yeah.

AARON BASTANI: You’re not telling them that actually they can’t use their hose pipes for three months a year because all the water’s going to that local data center.

KAREN HAO: Exactly.

AARON BASTANI: And it’s quite extraordinary. And the most scary thing about all of it is in the UK at least, the politicians don’t know any of that. I sincerely don’t think the Chancellor knows any of that. That, and there’s no real. I mean, even if you use the prism of colonialism, imperialism with regards to exploitative economic relations between the United States and other parts of the world, they think you’re a Trotskyist. Yeah, that’s, that’s the crazy things. They can’t even look after their own people. Because if looking after your own people boils down to being too left wing.

KAREN HAO: Well, I think part of it is also that they don’t really realize that it’s literally happening in the UK. So that. So to connect it to the UK data center development along the M4 corridor has literally already led to a ban in construction of new housing in certain communities that desperately need more affordable housing.

And it’s because you cannot build new housing when you cannot guarantee deliveries of fresh water or electricity to that housing. And it was due to the massive electricity consumption of the data centers being built in that corridor that led to that ban.

AARON BASTANI: That’s nuts. That’s the most valuable real estate for housing in the country. Them four.

KAREN HAO: Yeah.

AARON BASTANI: And do you think UK politicians are aware of that contradiction or is that just. That’s.

The False Trade-Off of AI Development

KAREN HAO: I mean, you know, I don’t know if they are aware, if maybe they don’t have awareness or maybe they are aware and they’re also thinking of other trade offs. I mean, now in the UK and in the EU at large, there’s just this huge conversation around data sovereignty. And of course technology sovereignty is this whole concept of developing the EU stack.

And why is it that we don’t have any of our tech giants? Why don’t we have any of this infrastructure? And here, Starmer just said this week during London Tech Week, “we want to be AI creators, not AI consumers.”

So I think in their minds maybe this is a viable trade off. We, we skimp a little bit on housing for the ability to have more indigenous innovation. But I think the thing that is often left out of that conversation is this is a false trade off.

People think that you need colossal data centers to build AI systems. You actually do not. This is specifically the approach that OpenAI decided to take. But actually before OpenAI started building large language models and generative AI systems at the these colossal skills, the trend within the AI research community was going the opposite direction towards tiny AI systems.

And there was all this really interesting research looking into how small your datasets could be. To create powerful AI models and how little computational resources you needed to create powerful AI models. So there were interesting papers that I wrote about where you could have a couple hundred images to create highly performant AI systems or you could have AI systems trained on your mobile device that’s not even one computer chip running on your mobile device.

And OpenAI took an approach that is now using hundreds of thousands of computer chips to train a single system. And those hundreds of thousands of computer chips now are consuming city loads of energy.

And so if we divorced the concept of AI progress with this scaling paradigm, you would realize then you can have housing and you can have AI innovation. But once again, there’s not a lot of independent experts that are actually saying these things. Most AI experts today are employed by these companies.

And this is basically the equivalent of if most climate scientists were being bankrolled by oil and gas companies. They would tell you things that are not in any sense of the word scientifically grounded, but just good for the company.

The Colonial Relationship Between AI and Global Labor

AARON BASTANI: I interviewed a great guy twice actually now, a guy called Angus Hansen who’s really just on it with regards to the exploitative nature of the increasingly exploitative nature of the United States economic relations with the UK. Just fascinating, fascinating book. And man, and I just don’t think it’s cut through to our politicians here how bad it’s getting.

And you’re saying about AI consumers or creators, I mean ultimately you’re talking about Meta, you’re talking about Alphabet, you’re talking about Xai, you’re talking about OpenAI. We are consumers, we are dependent. It’s a colonial exploitative relationship with regards to big tech has been for a really long time. Our smartest people which the taxpayer trains here go to the US. I think one of the top people at Slack is a UK national, Demis Hassabis, you know, DeepMind now working under the umbrella of Alphabet.

And yeah, it just doesn’t make sense for me with regards to that formulation. They simply don’t get it. You know, I came here using my MasterCard. Millions of Brits use Apple Pay and Google Pay and MasterCard and Visa. And every time we do it, 0.1, 0.2% crosses the Atlantic and it just goes over the heads of our political class, which is very unnerving in regards to the efficiency of these smaller systems.

Where does DeepSeek fit in all of this? Because of course the scaling laws at the heart of OpenAI, which is you get to AGI by more compute, more parameters, more data is kind of untethered a bit by the arrival of DeepSeek.

DeepSeek: Challenging the Scaling Paradigm

KAREN HAO: Yes, DeepSeek is such an interesting and complicated case because they basically it’s a Chinese AI model that was created by this company, High Flyer, and they were able to create a model that essentially matched and even exceeded some performance metrics of American models being developed by OpenAI and Anthropic with orders of magnitude less computational resources, less money.

That said, it’s not necessarily the perfect. I don’t think the world should suddenly start using DeepSeek and saying DeepSeek solves all these problems because it’s still engaged in a lot of data privacy problems, copyright exploitation, things like that. And some people argue that ultimately they were distilling the models that were first developed through the scaling paradigm. So you first develop some of these colossal scaling models and then you end up making them smaller and more efficient. So some people argue that you actually have to first do that scaling before you get the efficiency.

But anyway, what it did show is you can get these capabilities with significantly less compute. And it also showed a complete unwillingness of American companies now that they know that they can use these techniques to make their models more efficient. They’re still not really doing it.

AARON BASTANI: Why do they like giving their money to Nvidia? What’s the…

KAREN HAO: Because if you continue to pursue a scaling approach and you’re the only one with all of the AI experts in the world, you persuade people into believing this is the only path and therefore you continue to monopolize this technology because it locks out anyone else from playing that game. And also because path dependence, these companies are actually not that nimble. They end up the way that they organize themselves. It’s not so easy for them to just immediately swap to a different approach. They end up putting in motion all the resources, all of the training runs, so on and so forth over the course of months, and then they just have to run with it.

So DeepSeek actually wasn’t the first time that this happened. The first time that this happened was with image generators and stable diffusion. And stable diffusion was specifically developed by an academic in Europe who was really pissed that the AI companies like OpenAI were taking a scaling approach to image generation. He was like, “This is literally wholly unnecessary and they’re spending thousands of chips, all of this energy to produce DALL-E.”

And ultimately he ended up producing stable diffusion with a couple hundred chips using a new technique called latent diffusion. Hence the name Stable diffusion. And, you know, arguably it was actually an even better model than DALL-E, because users were saying that stable diffusion had even better image quality, better image generation, better ability to actually control the images than DALL-E.

But even knowing that latent diffusion existed, OpenAI continued to develop DALL-E with these massive scaling approaches. And it wasn’t until later that they then adopted the cheaper version, but it was just significantly delayed. And I was asking OpenAI researchers why that doesn’t make any sense. Why did you do that? And they were like, “Well, once you set off on a path, it’s kind of hard to pivot.”

AARON BASTANI: Also, Jensen Huang, the CEO of Nvidia, is really charismatic. Right? I mean, it’s quite funny because I’m a Marxist. Just, I’m going to make that confession. You have these big sort of structural understandings of how history happens, and then you sort of realize, actually, this guy’s really charismatic and this person’s really manipulative, and all of a sudden the world’s hyperpower is making these technological decisions, okay, quite strange.

We talked about data centers, we talked about Earth, water, energy. I want to talk also about some of the more exploitative practices with regards to workers in the global South. You use one really galling example, actually, in Kenya. Can you talk about some of the research around that? Some of the people you met?

The Hidden Human Cost: Content Moderation in Kenya

KAREN HAO: Yeah. So I ended up interviewing workers in Kenya who were contracted by OpenAI to build a content moderation filter for the company. And at that point in the company’s history, it was starting to think about commercialization after coming from its nonprofit fundamental AI research roots. And they realized, if we’re going to put a text generation model in the hands of millions of users, it is going to be a PR crisis if it starts spewing racist, toxic, basic, hateful speech.

In fact, in 2016, Microsoft infamously did exactly this. They developed a chatbot named Tay. They put it online without any content moderation, and then within hours, it started saying awful things. And then they had to take it offline. And to this day, as evidenced by me bringing it up, it’s still brought up as a horrible case study in corporate mismanagement.

And so OpenAI thought, “We don’t want to do that. We’re going to create a filter that wraps around our models so that even if the models are generating this stuff, it never reaches the user because the filter then blocks it.”

In order to build that filter, what the Kenyan workers had to do was wade through reams of the worst text on the Internet, as well as AI generated text on the Internet, where OpenAI was prompting its models to imagine the worst text on the Internet. And the workers then had to go through all of this and put into a detailed taxonomy. Is this hate speech? Is this harassment? Is this violent content? Is this sexual content? And the degree of hate speech, of violence, of sexual content.

So it was, they were asking workers to say, does it involve sexual abuse? Does it involve sexual abuse of children? So on and so forth. And to this day, I believe if you look at OpenAI’s content moderation filter documentation, it actually lists all of those categories. And this is one of the things that it offers to clients of their models, business clients of their models, that you can toggle on and off each of these filters. So that’s why they had to put this into that taxonomy.

The workers ended up suffering very many of the same symptoms of content moderators of the social media era, absolutely traumatized by the work, completely changed their personalities, left them with PTSD. And I highlight the story of this man, Moffat Okinyi, who’s one of the workers that I interviewed who showed to me that it’s not just individuals that break down, it’s their families and communities, because there are people who rely on these individuals.

And so Moffat was on the sexual content team. His personality totally changed as he was reading child sexual abuse every day. And when he came home, he stopped playing with his stepdaughter, he stopped being intimate with his wife. And he also couldn’t explain to them why he was changing because he didn’t know how to say to them, “I read sex content all day.” That doesn’t sound like a real job. That sounds like a very shameful job. ChatGPT hadn’t come out yet, so there was no conception of what does that even mean.

And so one day his wife asks him for fish for dinner. He goes out, buys three fish, one for him, one for her, one for the stepdaughter, and by the time he comes home, all their bags are packed and they’re completely gone. And she texts him, “I don’t know the man you’ve become anymore and I’m never coming back.”

AARON BASTANI: You say that’s the case with regards to text. Are people also having to engage with images as well? I mean, that was more of a social media thing, is that… Yeah, too.

KAREN HAO: Yeah. There were workers that they then. So after this, they contracted these Kenyan workers. That contract actually was canceled because there was a bunch of scrutiny on that company and the third party company that they were contracting the workers through and huge scandal.

AARON BASTANI: It’s Sama, right?

KAREN HAO: Sama. Yeah. And there was a huge scandal around Sama. And then OpenAI ended up shifting to other contractors who were then involved in moderating images.

AARON BASTANI: And were they remunerated for the kind of work they were doing quite well or…

KAREN HAO: For the Kenyan workers. They were paid a few dollars an hour, right? Yeah.

The Crisis Playbook: Exploiting Venezuelan Refugees

AARON BASTANI: And then on the other side of the Atlantic, you talk about people in South America doing effectively mechanical Turk piece work for these companies as well. Can you talk about that a little bit?

KAREN HAO: Yeah. So generative AI is not the only thing that leads to data annotation. This has actually been part of the AI industry for a very long time. And so I ended up years ago interviewing this woman in Colombia who was a Venezuelan refugee about the specific thing that happened to her country in the global AI supply chain.

So when in 2016, when the AI industry first started actually looking into the development of self driving cars, there was a surge in demand for highly educated workers to do data annotation labeling for helping self driving cars navigate the road. You have to show self driving cars. This is a car, this is a tree, this is a bike, this is a pedestrian. This is how you avoid all of them. These are the lane markings. This is what the lane markings mean. And they’re humans that do that.

And it just so happened in 2016 when this demand was rising, that Venezuela as a country was dealing with the worst peacetime economic crisis in 50 years. So the economy bottomed out. A huge population of highly educated workers with great access to Internet suddenly were desperate to work at any price. And these became the three conditions that I call the crisis playbook in my book that companies started using to then scout out more workers that were extremely cheap for working for the AI industry.

And so the woman that I met in Colombia, she was not just. She was working in a level of exploitation that was not based on the content that she was looking at. She was labeling self driving cars and labeling, you know, retail platforms and things like that. The exploitation was structural to her job and that she was logging into a platform every day and looking at a queue that automatically populated with tasks that were being sent to her from Global north company.

And most of the time the tasks didn’t appear. And when they did, she had to compete with other workers to claim the task first in order to do it at all. And because there were so many Venezuelans in crisis and so many of them were finding out about data annotation platforms in the end, there were more and more and more workers competing for smaller and smaller volumes of tasks.

And so these tasks would come online and then disappear within seconds. And so one day she was out on a walk when a task appeared in her queue and she sprinted to her apartment to try and claim the task before it went away. But by the time she got back, it was too late. And after that she was like, “I never went on a walk during the weekday again.” And on the weekends, which she discovered is less often, less likely for companies to post tasks, she would only allow herself a 30 minute walk break because she was too afraid of that happening again.

AARON BASTANI: And did she detail about how that gave her sort of anxiety or insomnia or mental health kind of overheads? That sounds insane. Sounds like an insane way to live.

The Human Cost of AI Development

KAREN HAO: It completely controlled her life. She didn’t tell me about whether or not it gave her insomnia, but it completely controlled the rhythms of her life in that she had this plugin that she downloaded that would sound an alarm every time a task appeared so that she could cook or clean or whatever without literally just looking at the laptop the whole day.

And she would turn it on to max volume in the middle of the night, because sometimes tasks would arrive in the middle of the night. And if the alarm rang, she would wake up, sprint to her computer, claim the task, and then start tasking at like 3am in the morning.

And she had chronic illness. One of the reasons why she was tethered to her apartment, doing this online work in the first place was not just because she was a refugee, but also because she had severe diabetes. And it got to the point where she ended up in the hospital and was completely blind for a period of time. And the doctor said that if you had not come to the hospital when you did, you would have died.

And so she was tethered to her home because she had to inject herself with insulin like five times a day. And it was this really complicated regime that didn’t allow her to commute to a regular office, have a regular job. So she was doing all of this extremely disruptive dysregulating work on top of just trying to manage extreme severe diabetes.

AARON BASTANI: I mean, it’s extraordinary you’ve managed to unveil those stories, I think. I mean, that’s why the book is so interesting. Fascinating for me, that’s why it’s got the plaudits, is that you’re speaking to people who are on first name terms with Sam Altman. Then you’re talking to Venezuelan refugees in Colombia.

And it’s really important to say that this work is being done for multi-trillion dollar companies. That’s the other side of it, right? You’re saying Elon Musk worth 300 billion plus dollars and then there are people. That’s where the value is being generated.

The Logic of Empire in AI

KAREN HAO: Yeah, exactly. And that’s the reason why I really wanted to highlight those stories is because that’s where you really see the logic of empire. There is no moral justification for why those workers whose contribution is critical to the functioning of these technologies and critical to the popularity of products like ChatGPT are paid pennies when the people working within the companies can easily get million dollar compensation packages.

The only justification is an ideological one, which is that there are some people born into this world superior and others who are inferior. And the superior people have a right to subjugate the inferior ones.

What Can the Public Do About Big Tech?

AARON BASTANI: My last question, what does the US public do about big Tech if it wants to take on some of these issues? Income inequality, regional inequality, global imperial overreach, et cetera. A few proposals and which somebody can execute on. What would you suggest?

KAREN HAO: Yeah, I wouldn’t even say it’s just the US public. I mean anyone in the world can do something about it. And one of the remarkable things for me in reporting stories is people who felt like they had the least amount of agency in the world were actually the ones that put up the most aggressive fights and actually started gaining ground on these companies in taking resources from them.

So I talk about Chilean water activists who pushed back against a Google data center project for so long that they’ve stalled that project now for five years. And they forced Google to come to the table and the Chilean government to come to the table. And now these residents are invited to comment every time there’s a data center development proposal, which they then said, it’s not the end of the fight. They still have to be vigilant and at any moment, if they blink, something could happen.

But anyone in the world I think, has an active role to play in shaping the AI development trajectory. And the way that I think about it is the full supply chain of AI development. You have a bunch of resources that these companies need to develop their technologies. Data, land, energy, water. And then you have a bunch of spaces that these companies need access to deploy their technologies. Schools, hospitals, offices, government agencies, all these resources.

And all of these spaces are actually places of democratic contestation. They’re collectively owned, they’re publicly owned. So we’re already seeing artists and writers that are suing these companies saying, “No, you cannot take our intellectual property.” And that is them reclaiming ownership over a critical resource that these companies need.

We’re seeing people start exercising their data privacy rights. I mean, one of my favorite things about visiting the UK and EU as an American that has no federal data privacy law to protect me, is to reject those cookies. Every single webpage that I encounter, that is me reclaiming ownership over my data and not allowing those companies to then feed that into their models.

We’re seeing, just like the Chilean water activists, hundreds of communities now rising up and pushing back against data center development. We’re seeing teachers and students escalate the public debate around do we actually want AI in our schools? And if so, under what terms? And many schools are now setting up governance committees to determine what their AI policy is so that ultimately AI can facilitate more curiosity and more critical thinking instead of just eroding it all away.

The same thing, I’m sure wherever your audience is sitting right now, if they work for a company, that company is for sure discussing their AI policy. Put yourself on that committee for drafting that policy. Make sure that all the stakeholders in that office are at that table actively discussing when and under what conditions you would accept AI and from which vendors as well. Because again, not all AI models are created equal. So do your research on which AI technologies you want to use and which companies are providing them.

And I think if everyone can actually actively play a role in every single part of the supply chain that they interface with, which is quite a lot, most people interface with the data part. Many people will now have data centers popping up in a community near them. And everyone goes to school at some point. Everyone works in some kind of office or community at some point.

If we do all of this pushback 100,000 times fold and democratically contest every stage of this AI development and deployment pipeline, I am very optimistic that we will reverse the imperial conquest of these companies and move towards a much more broadly beneficial trajectory for AI development.

AARON BASTANI: Yeah, we’ve had big tech social media for the last 15, 20 years. And I suppose the question is, is the same set of patterns going to apply to this stuff? And I think when you speak to someone like Jonathan Haidt, when he talks about young people and their consumption now of social media and mobile telephones, et cetera, his real worry is AI.

And if there is this laissez faire attitude from policymakers and also, let’s be honest, from civil society, that there was over the last 15, 20 years. I mean, he’s terrified about the implications. So it’s interesting to see there’s congruence between what you’re saying, what Jonathan Haidt’s saying. Can I ask you one more question?

KAREN HAO: Yeah.

AARON BASTANI: Have you ever read Dune by Frank Herbert?

The Dune Analogy: AI as Religious Mythology

KAREN HAO: I’ve watched the movie and it’s sitting on my bedside table to actually read the original. And I’m so glad that you asked me this, because this is an analogy that I use all the time now to describe the AI world.

AARON BASTANI: Yeah, the Butlerian Jihad.

KAREN HAO: So one of the things that was so shocking to me because we already talked about this, like, quasi religious fervor within the AI community. And I was interviewing people who, one of the people that their voice was quivering when they were telling me about the profound cataclysmic changes on the horizon. Like, these are very visceral reactions. These are true believers.

And Dune strikes me as a really good analogy for understanding this ecosystem because Paul Atreides’ mom, in the story, she creates this myth to help position Paul as a supreme leader and to ultimately control the population. And the people who encounter this myth, they don’t know that it’s a creation, so they’re just true believers. And at some point, Paul gets so wrapped up in this own mythology that he starts to forget that it was originally a creation.

And this is essentially what I felt like I was seeing with my interviews of people in the AI world, because I had the opportunity to start interviewing people starting all the way back in 2019. I interviewed some people back then and for the book to just map out their trajectory. And there were non believers back then that are true believers now. Like, if they were able to stay long enough at that company, they all, in the end, become true believers in this AGI religion.

And so there’s this vortex of, it’s like a black hole, ideological black hole. I don’t know how to explain it, but people, when they swim too long in the water, it just becomes them.

AARON BASTANI: So what you’re saying is Sam Altman is the Lisan Al Gaib. That’s the character. And Paul Graham maybe was the mother.

KAREN HAO: It would seem like that would be the most appropriate character to assign to him. Yeah.

AARON BASTANI: Wow, this has been fabulous. And I have to say, honestly, the book is really, really exceptional. Empire of AI. I read it so much, that dust jacket. I think my daughter actually ripped it off. But anyway, it is a sensational book, sensational journalism, fantastic journalist. We don’t have enough of those in the world.

KAREN HAO: Thank you.

AARON BASTANI: Real pleasure to meet you, Karen. Thanks so much for joining us.

KAREN HAO: It was great to meet you.

Related Posts

• Full Transcript: Zanny Minton Beddoes Interviews Elon Musk
• Transcript of Daniel Kokotajlo Interview: Diary Of A CEO Podcast
• Transcript: Why a Social Media Ban Won’t Save Teens – Candice Odgers  
• Transcript of Elon Musk Interview: SXSW Live – March 9, 2013
• Digital Afterlife: Who Owns Your Data When You’re Gone? – Suresh Sankaran Srinivasan (Transcript)