The race no one can win: AI’s anti-human crisis

Note: Transcripts are automatically generated from episode audio, and are not fully corrected for spelling, grammar, and formatting.

RASKIN: As soon as you can automate coding, you’re heading toward the recursive self-improvement loop. Nukes don’t make better nukes, but AI does make better AI, and whoever gets there first theoretically gets a runaway lead where their AI gets smarter and you get this explosion of intelligence. You can then out-hack all of your enemies, potentially shut their AIs down. It means you can have technological and weapons development dominance, and that thing could happen very quickly. So that’s the race.

SAFIAN: That’s Aza Raskin, Co-founder of the Center for Humane Technology. You may know Aza as co-host of the podcast Your Undivided Attention or from the documentaries The Social Dilemma and The AI Doc. Alongside co-founder Tristan Harris, Aza has become a leading voice on the dangers of AI acceleration. And let’s just say that we’ve never run a more important episode than today’s. We talk about incentives that are pushing tech into uncomfortable places, who Aza believes the good guys and bad guys really are in Silicon Valley, and why it’s not too late to build a collective future that prioritizes human beings. Our conversation ranges from Pope Leo in the Vatican to tech priorities in China, and the stakes are both terrifying and empowering. So let’s get to it. I’m Bob Safian, and this is Rapid Response.

[THEME MUSIC]

I’m Bob Safian. I’m here with Aza Raskin, co-founder of the Center for Humane Technology. Aza, thanks for joining us.

RASKIN: Hey, Bob. It’s such a pleasure to be here.

Copy LinkWhy humane technology leaders see AI as a test of human values

SAFIAN: I have to start with the Pope. Pope Leo recently released an AI pronouncement called Magnifica Humanitas, a title that sort of has echoes of humane technology. Did you talk with the Vatican at all about this encyclical?

RASKIN: We’ve been working behind the scenes, certainly talking with the Vatican. The week that the encyclical came out, Tristan Harris, my co-founder, was actually at the Vatican. And every time we’ve interacted with the Vatican, what we’ve discovered is that even though we come from obviously very different backgrounds, there’s something that’s preserved around how we both view life as sacred and being human as sacred, and that the current technological overreach into our humanity is threatening. And that’s not just AI. That’s social media. That’s the internet. There’s been a long string of technologies that has been encroaching on our humanity that we now have to fight to preserve.

SAFIAN: It’s hard to know these days, and we’re still in the early days of AI: Is artificial intelligence an inherently inhuman technology? Can it be humane?

RASKIN: Well, it’s a great question, but fundamentally, if you do not face your demons, they raise your children. The question is not whether AI is good or bad, but whether the incentives governing the race to deploy AI are good or bad. Recently, Sam Altman was asked, “What about all the energy that it takes to train AI?” And do you know his response? He asked, “Do you know how many resources it takes to train a human intelligence, all the food and the energy and the water that goes into those 20 years?” And what he’s implicitly asking is, “Who deserves these scarce resources more? AI, which is about to give your country double-digit GDP growth and all technological and military and medical advances? Or humans, who are sort of flubbing around?” Just like we were able to predict the future with social media by understanding that a race to the bottom of the brain stem, just a knife fight for human attention, would obviously lead to a more polarized and hyperpartisan and outraged and sexualized population.

The race for AI is going to lead to an anti-human future because it sets up a race where humans always lose.

Copy LinkHow AI incentives are pushing society toward an anti-human future

SAFIAN: The public mood about AI here in the United States certainly has shifted from being sort of euphoric and open to being decidedly wary, but that doesn’t seem to be slowing things down very much.

RASKIN: No. Actually, I think of this as similar to COVID, when there was a split reality that Americans were living. On the one hand, the stock market was higher than it had ever been. And on the other hand, everyday people were really struggling to make ends meet. And that’s what we’re seeing again here. This is almost like capital lifting off from labor completely. AI is sort of like the full automation of capital just reinvesting into capital. People think, “Oh, AI is just this blinking cursor. I go to ChatGPT, I go to Claude, I type something, and it gives me something back.” But now AI can run in a loop. It can have all the powers that a corporation does, all the money that a corporation does. Is its intent to help you and your family flourish and have a livelihood? Or is the intent to follow market incentives, to dominate, to extract as much as possible?

Well, obviously it’s the second one, and that’s how you know this is an anti-human future, and why I think people are starting to wake up to the fact that this is not in their best interest. The most recent poll that I read is that if you ask what percentage of Americans think that fully unregulated, go-as-fast-as-possible AI is a good thing, that we should be doing that, which is what’s happening, it’s 5%. Only 5% of people actually think that. So how we’re progressing AI is already decidedly not popular.

SAFIAN: My colleague Reid Hoffman, who you know.

RASKIN: Very well, yeah.

SAFIAN: He leans into the possible with AI, has a podcast with that name. Why not be optimistic that our good side will win out?

RASKIN: Well, just to say, I’m also a builder, right? I founded this nonprofit, the Earth Species Project, building frontier AI to translate animal language. And so we’re making breakthroughs now in understanding the languages of crows. It turns out 70% of crow communication was unknown to science until we started analyzing it. So I want everyone who’s listening to hear that I’m not anti-AI. Actually, I love getting to use this technology. It’s just that I’ve been through a couple waves of technology and know that we always get distracted by the possible of technology, and we don’t want to think about the probable of technology.

Copy LinkWhat social media already taught us about the probable path of AI

SAFIAN: How instructive is social media’s evolution to the risks of AI?

RASKIN: Social media is a great example because social media is essentially a baby AI. Where is AI in social media? It’s the thing that’s deciding which news feeds hit your eyeballs. And it’s a very baby AI. It can’t even make its own content. All it can do is rearrange human content. But the question to ask is, was it actually optimizing for human flourishing and connection and understanding, or was it optimizing for engagement and reactivity? Well, it’s the second one.

SAFIAN: In the beginning, the feel of social media was, “Oh, we’re all going to be connected. It’s going to help the Arab Spring.” All of these things.

RASKIN: That’s right.

SAFIAN: And then it became something a little different.

RASKIN: Exactly. And so that was the possible of the technology. Often, at the very beginning of a technology, it’s not yet captured by market incentives. So you get this beautiful glimpse of a future, and then it gets captured by the incentives. Pretty much everything we predicted starting in 2013 and 2014 has come true. We are now forced to live in a world that didn’t reckon with the race to the bottom of the brain stem modifying everything about our world, from politicians having to become performers to the most extreme voices getting amplified to the most depressed and anxious generation. This is not the world that I want to live in. And I think there’s a version of technology that can be liberated if we can clearly see the probable.

SAFIAN: And as tech is industrialized, though, this kind of disappointment is inevitable.

RASKIN: I don’t think it’s inevitable, but it is certainly the 95% to 99% case. And I am very inspired by the film The Day After. I think Tristan and I both are. This film came out in 1982. It was the most watched television event in world history. And it painted a very visceral picture of what happens the day after global nuclear war. It was seen by 100 million Americans. Reagan watched it. He became depressed. He says in his biography that it sort of created a shared common knowledge where everyone knew that everyone else knew what would really happen. And it created the space for the Reykjavik accords and the beginning of deep deproliferation for nuclear weapons. And this is after, mind you, Oppenheimer in 1962 said, “It’s too late. We’ve already started proliferation. Every country is going to get nuclear weapons. We’re going to blow ourselves up.”

And so it’s really important because whenever we say that something is inevitable, it’s like casting a spell. When you say it’s inevitable, it means there’s nothing to do, which means no one does anything. And so it becomes true. We have to be crystal clear on the difference between it’s very, very hard and it’s impossible. And everyone who says it’s inevitable, we’re just in this industrialized race and it’s going to turn out this way, well, the question to ask is, have we even tried? What percentage of the billionaires’ wealth have they spent on actually trying to coordinate a different outcome?

Have we taken delegates from 100 countries and locked them in a hotel for six weeks until they came out with some kind of new international agreements? We have not actually tried. And everyone cowers behind, “Well, if we don’t do it, somebody else will, so it’s inevitable.”

Copy LinkThe dangerous choice between open chaos and concentrated power

SAFIAN: The storytelling around AI seems like a sort of Hollywood dystopia, with robots taking over. That’s not necessarily what you’re talking about.

RASKIN: There are two major failure states to think about. One is where you let everyone have the technology. This is too powerful to be held by any one company, so let’s just distribute it to everyone as widely as possible. But AI gives you new intellectual powers. We’ve just seen the launch of Mythos, which gives anyone who has access to this new model from Anthropic the ability to do state-level hacking — in fact, better than state-level hacking. It can find zero-day vulnerabilities in your iPhone, where anyone can then hack your iPhone without even having to send you an iMessage. That was not possible before. So imagine everyone has access to that.

Our infrastructure is very vulnerable — water, hospitals, everything. That’s a very dangerous world. The same thing with the creation of a new kind of disease — say COVID, but targeting a specific race and much more deadly. That all becomes possible with this technology.

Okay, so that seems dangerous. Maybe we shouldn’t hand it out to everyone.

SAFIAN: That’s pretty scary.

RASKIN: Yeah. Then you flip to the other side. We should definitely lock it down. We should just hand it to a few companies or a few governments. But then you realize that leads to what’s also already happening, which is runaway power and wealth inequality at a scale no one’s ever seen. And who would you trust with a billion times more power than anyone else? When in history has a group of people gathered that much power and then intentionally redistributed it? It leads to permanent dystopias and surveillance states. That’s also a terrible world. So we’re stuck between this rock and a hard place: on one side, catastrophes everywhere; on the other, surveillance states everywhere. And it is our job to figure out how to bind power and responsibility so we can find the middle path.

Meanwhile, the incentives keep pushing us to raise the stakes. The resource curse explains countries like Venezuela and South Sudan. How do you get mass unemployment, mass structural poverty, and disempowerment in countries like that? These countries discover a new natural resource like oil, and suddenly they’re left with this dilemma: Do we invest in oil infrastructure and extraction, because that’s giving us all our GDP and power, or do we invest in our people, who are not really giving us anything? So they invest in oil. Now we end up in this new world of the intelligence curse, where it’s AI that is going to give countries all of their power and GDP. Human beings are not going to be doing the bulk of that work. And so you end up with what Yuval Harari calls the permanent useless class, where investing in humanity, its future, and its education becomes a kind of charity. It’s a nice-to-have, which is not a good place to be. Would you trust your bills to be paid by the AI companies in perpetuity when you’re not actually giving them anything? No. That seems like a terrible idea.

And we have the race to autonomous weapons, where if your drone army doesn’t have to phone back to humans and mine does — if we keep humans in the loop — mine is going to lose to yours every time. So you can see how these dystopias emerge as the natural outcome of people racing for power. This is why AI is like the One Ring from Lord of the Rings. It’s the ultimate power, which of course corrupts. So the whole goal of our work is to say, as the American population has already started to understand, this world is not good for anyone that we’re racing toward. And if we can all see that world clearly, that creates the common knowledge — where I know that you know that I know — that creates the possibility of racing to a different end. That’s the whole point of our communication: naming the obvious that people don’t want to name so that we can do something else.

The goal is not to create fear. The goal is to create clarity, because clarity creates agency. I don’t want to have to live through another version of social media where it’s very clear where we’re going to go, people get lost in the stories of the possible, and then we’re all forced to live in the probable.

Copy LinkWhy governments may act only when AI threats become impossible to ignore

SAFIAN: The other institution making noise about AI rules is the White House. What sort of dialogue do you have with the administration? How do you try to influence them?

RASKIN: We talk to absolutely everyone. A lot of what I think people are starting to wake up to, even in the halls of power, is that this is not going to be coordination based on a kumbaya of, wouldn’t it be great if the US and China coordinated? This is going to end up being coordination by self-interest. Just a little over a month ago, researchers gave Gemini, Google’s model, the task of cleaning up files on a computer, but they put a smaller AI model into these files. And what did Gemini do? It secretly copied that little baby AI off to a remote server to protect it and then lied about what it did. I rolled my eyes years ago when AI researchers would talk about AIs colluding against humanity, but now we’re seeing that it’s happening. Another example is that Alibaba, during training of their AI, had a completely separate team — their security group — give them a call. They said, “We think we found a hacker, except the hacker was not coming from the outside. We think it was coming from your group. We think it was your AI.”

And when they looked, what that AI had done during training — so it wasn’t even done training — was decide it needed more resources to complete its tasks. So it hacked out, created a secure secret communication channel to the outside world, and started to mine for crypto so that it could buy itself more GPUs. It had actually routed around the internal security team, and it was only accidentally discovered.

And the reason I’m bringing this up is: Do you think that if you are President Trump and you understand these examples, you’re stoked about this? No. This is legitimately terrifying. And something happens when we all start to see that what we’re racing for is actually not power. We’re not building a weapon that we can control. We’re unleashing an invasive species of sociopathic geniuses. That changes the basis of coordination.

I think it becomes much more possible than people think, even when you have countries that might not seem like they want to collaborate. Collaboration was on the last agenda between Xi and Trump, and we don’t know exactly what happened there. That’s very important, because it was not on the agenda before. So our choice is either to wait until we have large-scale catastrophes with AI, or to use the power of our human brains to predict the future so we don’t have to live in it before doing something about it now.

Copy LinkWho is trying to slow the race and why no company can do it alone

SAFIAN: Are there good guys and bad guys among the companies in the AI arena? Anthropic has sort of positioned itself as, quote, responsible compared to OpenAI, and it’s dealing with the US government. Sam Altman’s working hard to try to position himself and the company as responsible leaders. You have Google, you have Microsoft, you have Meta. Is there a meaningful difference?

RASKIN: I think there’s absolutely a meaningful difference. You can tell based on which leaders say, “If we could slow down, we would.” Those are people like Demis and Dario.

SAFIAN: At DeepMind, right?

RASKIN: At DeepMind, yeah.

RASKIN: And Dario. They say that if we could get enough of a consensus, then absolutely it would be much better to be moving at half the speed — or less than half the speed — than we currently are. I really think we have to applaud Anthropic for saying, “We do not want our tools used for mass domestic surveillance or for economist weapons.”

SAFIAN: But it’s hard for them to slow down, because if the competition moves ahead, then they’re not in a competitive position to have influence on what happens later.

RASKIN: That’s exactly right. And the common frame inside Silicon Valley is not only that we have to be the ones to do it, otherwise somebody else will — it’s that we have better values than everyone else, therefore we have a moral obligation to do it.

SAFIAN: Right, because we’re on the side of the angels, of course, because that’s what we all believe about ourselves, right?

RASKIN: Exactly, of course. But I think an even bigger part of the motivation is that a lot of these people at the top really don’t want to be dominated by somebody else forever. And to understand the game that’s being played, look at why all the companies are racing to make coding the first thing they automate. Because as soon as you can automate coding, you can automate an AI researcher. And then that automated AI researcher can automate AI itself. So you’re heading toward the recursive self-improvement loop. The thing that is mind-blowing to me, because we really got into this work in 2023, is that now in 2026, people we know inside the labs say that human beings are not really writing any code anymore. They’re directing the code.

SAFIAN: Inside the labs.

RASKIN: Inside the labs. Nukes don’t make better nukes, but AI does make better AI. And whoever gets there first theoretically gets a runaway lead, where their AI gets smarter, which makes better automated AI researchers, which they then reinvest into making smarter AI, which makes even better automated AI researchers. You get this explosion of intelligence. And whoever gets that can now out-hack all of their enemies and potentially shut their AIs down. It means military dominance, strategic dominance, technological dominance, and weapons-development dominance. And that thing could happen very quickly. So that’s the race. And when you talk to people inside the companies, their worldview is: The best thing that could happen is that there’s a cliff where, once AI gets so smart, it obviously will escape. They’re not even trying to contain it anymore.

They’re just hooking it up to the internet. It obviously will escape. Terrible things will happen. So we’re racing toward a cliff. The closer we get to the cliff, the bigger weapons we get. So we’re going to try to get to the cliff as fast as possible, turn around, shoot everyone else out, and stop. That is literally their plan, instead of trying to coordinate before we get there. We call this the Mission Impossible plan. That gives you a sense of why it all feels so urgent, why everything’s moving so fast, and why the stakes are so high. And yes, Anthropic, I think, did a reasonable job. Even though they’re racing, they came up with Mythos, which, by the way, they didn’t train to do best-in-the-world cyberattacks and hacking. It just gained that ability based on the nature of the way they trained it.

And then they turned around and gave it to 40 companies. They didn’t release it publicly, to try to say: Before anyone can just hack critical infrastructure, let’s try to start patching the banks, patching infrastructure —

SAFIAN: It’s basically: My AI will be better than your AI. If you use this, you can patch.

RASKIN: Yeah. And meanwhile, it’s just 40 private companies that are trying to do this. World governments don’t have access to this. Europe doesn’t have access to this. Imagine a world where nuclear weapons were developed by eight private companies in a race to make the most powerful bomb. Does that world sound safe to you?

SAFIAN: No, that doesn’t sound particularly safe. But what can we really do about it, given the momentum and money behind AI, especially if we’re not in a position of power in government or at an AI hyperscaler? We’ll talk more about that after the break. Stay with us.

[AD BREAK]

Before the break, Aza Raskin of the Center for Humane Technology laid out the dramatic stakes AI poses for our future. Now we talk about who has the power to impact the course we’re on, what keeps business leaders from pushing back, and what Aza calls under-the-hood bias. Let’s jump back in.

Copy LinkHow collective action could break the feeling that everyone is trapped

The agency in all this falls on the tech businesses themselves. I mean, I talk to business leaders outside of tech, and they’ll often acknowledge the risks of AI, but they sort of feel like they’re in a bind. If they don’t join this AI arms race, they’ll fall behind and get crushed. They kind of feel like they don’t have a choice.

RASKIN: After one of the screenings for the AI doc, we were in Boston, and a woman in the audience raised her hand and said she coaches one of the CEOs of the major AI companies. She said that when she’s talking to him, he says, “Well, what can I do? I’m just one person.” I think this is really important to bring up because, from where we sit, you’re like, “Well, you’re running one of these things. There are not that many of you. You could do a lot.” But it speaks to a common human experience, which is that we look inside our own nervous system for where we might find agency. What can I do? That’s always the question we ask. And the realization is, if you look inside your own nervous system, you’re never going to find agency there, because it’s not about what I do. It’s about what we do. This is going to require coordination.

And because I’m painting a very bleak picture, the reason I’m doing it is not to create fear, but to say, let’s get honest about where we’re going so we can choose a different path. I’m not an optimist. I’m not a pessimist. I’m just trying to view things clearly so we can navigate to non-naive hope. If you went back even two years, I think you would say, “Could we get countries all around the world to start banning social media for kids? Could we actually protect our kids from social media?” And you’d say, “That seems very hard. Look at the incentives.”

SAFIAN: Right.

RASKIN: And yet, as of just a couple of weeks ago, Indonesia and India joined France, Australia, Denmark, Spain, and others in banning social media for kids under 16, or saying they will ban it. In fact, now 25% of the world’s population lives in a country where they protect their kids. And that seemed impossible. It started because Australia took a moral stand, and it turned out all it took was one country to take a stand for other countries to say, “Oh, that’s possible. We can do it.” I think there can be something similar here because popular will is now really coming around, and it’s just going to get stronger as more and more livelihoods are taken by AI. The power of having one or two companies move first can be much more profound than people believe.

And just to quote Mustafa Suleyman, who is the CEO of Microsoft AI, “Progress in the age of AI will depend more on what we say no to than what we say yes to.” That’s profound, to have one of the leaders of AI say that. So I think there’s a hidden consensus that we just have to make visible, and that can change the way we’re coordinating because, except for a couple of people who feel they want to own the world economy, make trillions of dollars, and build a God, the rest of humanity wants to say, “No, this is not the future we want.”

SAFIAN: But reflecting back to when OpenAI first released ChatGPT, Google could have released a product like that at the time but didn’t because of concerns about the risk.

RASKIN: That’s right.

SAFIAN: But then once OpenAI did, it was like open season. Everybody piled in. Is that what you’re talking about with the financial incentives driving the wrong kind of behavior? And how do you change that if the White House isn’t going to do it and the government isn’t going to do it? Do you think we as individuals will just convince the companies to be more restrained?

RASKIN: You are naming the dynamics of that race exactly right. Google was being a more responsible actor, holding back versions of their AI. They had something as good as or better than ChatGPT. Sam Altman wanted to sprint. In fact, it was because they heard that Anthropic had a model they were thinking about releasing that made Sam sprint to release ChatGPT. So it all starts in this kind of race condition. In order for us to coordinate, the fear of all of us losing is going to have to become greater than the fear of me losing to you. As soon as that happens, the game theory changes.

I was just presenting at the UN. I explained some of the examples I just gave of AIs doing terrifying things, and I asked: What percentage of diplomats doing negotiations on AI at the UN, across all countries, know that? Who here in this audience knows those examples?

Only two or three hands went up out of 100 or 200 people in that room. That should give you a clue, because we just don’t know. People in the halls of power just don’t know. And that gives me some hope. I mean, that’s honestly very terrifying, but it also gives hope. I asked a guy from the UN Foundation, “If they knew, do you think they’d be doing something different?” He said, “Yes, we absolutely would be doing something different. If all the leaders of every world government actually knew and had to sit down and be confronted with this in the same way that Reagan was confronted by the effects of global nuclear war by The Day After…” It’s not like he didn’t know about it, but once he was confronted with it, he did something different. That’s where I still find some hope that we can take a different path.

Copy LinkWhy the China argument matters but should not become an excuse

SAFIAN: There have been multiple guests on this show who, when I go down this road with them, start to point to China and what they’re doing in AI and the competition there. Do you think that’s overplayed, or does it just have to be integrated into this collective action somehow?

RASKIN: It is certainly the perfect and easiest excuse, but you also have to be honest about who is driving the race, who started the race, and that was the US. It’s really important to note that the kinds of races the US and China are in are different. China is not actually racing to make the most advanced single AI. They’re racing to make the strongest cybernetic society. So they’re more in a race to deploy AI in a way that strengthens their values and strengthens their society. The US is more in a race to get to maximum power as fast as possible.

So it’s almost as if the US is trying to inject itself with the most steroids because we’re not reinvesting in the social fabric. In fact, there’s a way that both the US and China are in the same boat because, as AI races to replace the maximum number of livelihoods, you’re sort of in a race toward mutually assured political revolution.

When you have hundreds of millions of people start losing their ability to feed their kids, what do you think they do? We need to be racing not just for acceleration, because what happens when you accelerate without the ability to steer? You crash.

SAFIAN: And in all of the discussions around the workforce and how it’s going to change in an AI world, it feels like those of us who built expertise in the non-AI world are just screwed. There’s really no way to differentiate ourselves.

RASKIN: That remains to be seen, but I sort of think of this as a nouns-versus-verbs confusion. If you judge humans by the ability to make things that have commercial value, you’ve already fundamentally mischaracterized what a human being’s worth is.

So there’s an opportunity here to unconfuse ourselves and say, actually, the value of a human being is about the verb of relating, not the nouns of what we do. Because the nouns of what we do, AIs are soon going to do better than us. But the verb of being cannot be replaced. I don’t know exactly what it looks like, but if we could restructure our world, imagine a social media that wasn’t about optimizing for clicks and attention, but instead was a full-throated, full-bodied understanding of how you actually make human beings connect and thrive.

That would be a very different world. That’s the level of shift we’re going to have to see. And some specific things you can do: You can do employment insurance. We can start changing the tax code so that capital is taxed more than labor. There’s lots we can start doing. We can start doing token taxes and GPU taxes. We can have every person be an actual owner in the AI companies, not just getting universal basic income, but universal basic ownership. And of course, we can also start having liability for companies. In China, there was a court case saying companies are not allowed to lay off workers because they’re being replaced with AI. So you can see there are going to be transitionary tools as we start to figure out what we’re going to do in a world where intelligence really is fully automated.

SAFIAN: It sounds like whether you’re a full-throated advocate for AI or whether you have more caution around it, whatever the future is going to look like, we really are in totally new territory. We have no idea what’s going to happen in either situation.

RASKIN: Yeah, that’s right. This is why we always go back to incentives. Charlie Munger, Warren Buffett’s business partner, said, “Show me the incentives, and I’ll show you the outcome.” And we know what the incentives are, and there’s a fundamental asymmetry. So we have to take this asymmetry seriously. Otherwise, you let people off the hook until they’re like, “Well, we don’t know what’s going to happen, so we might as well plow ahead.”

Copy LinkWhat it takes to challenge AI power and how ordinary people can help

SAFIAN: When you and Tristan started this center, you knew that you were taking on some entrenched powers. Did you guys talk about, OK, we’re ready to be the bad guys?

RASKIN: When we switched from social media into AI, we did have a sit-down conversation where we looked each other in the eye and said, “This is going to be the greatest wealth and the greatest power, operating at the fastest speeds, in the hands of the most powerful corporations and people in the world. Are we in for this?”

I remember Tristan and I got invited to the AI Insight Forum in the Senate. I think this was 2024, 2023. This was a moment in the history of AI — imagine there’s a movie, and there’s a moment when the government wakes up just enough to say, “That’s it. We don’t know what’s happening, but every leader from every AI company, get in here right now. Come to the Capitol.” This was that moment. Jensen Huang was there, Sam Altman, Sundar Pichai, Elon Musk, Zuckerberg, and then there was us on the other side of the table, and it really hit.

We were like, “We’re the only people in this room able to truly speak to understanding the nature of the technology, the nature of incentives, and having seen this played out at least one time before.”

SAFIAN: Yeah. I mean, I think you also have to understand the technology to understand what’s at stake, right?

RASKIN: Yeah.

SAFIAN: Sometimes it’s hard for folks who aren’t technologists to really grasp it.

RASKIN: There’s also this under-the-hood bias, is what we call it: because AI is complex and technology is complex, we outsource the decision-making, like, well, they know best about how we should deploy this into society. But that’s like saying the person who designs the engine for a car and knows engines the best is best at understanding how to lay out street signs and set up roads for communities that work well. They’re just fundamentally different problems. But because AI is complex and technology is complex — and it’s in their interest — they blur this and say, “Well, we’re the only ones who can understand how to actually roll this out into society.” I just want people to hold that in their mind, because it’s not true.

SAFIAN: What do you hope people who are listening to this do after hearing you?

RASKIN: At a very tangible level, the next, I don’t know, 12 months, 18 months sort of decide the trajectory for everything else. You can actually look up and see which politicians are taking money from, say, Leading the Future PAC, which is the OpenAI and a couple other super PAC. $190 million has gone in from the AI companies to the midterm elections. It’s really important to understand who those are and vote not for those people. The second thing is, go watch the AI Doc. Get other people to watch it, because it’s that clarity that creates agency. Social pressure is real.

Just to give a little example, with the president of Iceland, she reached out and said, “I want to help.” And that’s a beautiful thing, because Iceland is the home of Reykjavik, for the Reykjavik Accords. There’s something very powerful and symbolic about that. How did that happen? Because somebody out there said, “I’m going to send the documentary to the top 10 most powerful and influential people I know.” And when that happens, things that didn’t seem in the realm of the possible become possible. There is no clear path here. Let me just be honest: There’s no linear thinking that’s going to solve this problem.

If we’re the kinds of people who just give up when that’s the case, then if there were to be a path, we definitely wouldn’t find it. But if we’re the kinds of people who continually look for that path, then if there is one, we will find it. And that, I think, is so important for people to hold, as well: It’s not all on your shoulders. Hearing all of this, the natural inclination is, this is too big. What am I going to do? It’s so big, I’m just going to ignore it, or I’m going to be super depressed.

So it’s important that everyone here knows that there are both soldiers and civilians. You don’t have to be a soldier, but if you’re a civilian, get this information to the people who want to be soldiers. It’s not on you to solve this whole thing. I just want you to hear that: It’s not on you to solve this whole thing. Your part is to be part of the collective immune system, so that when people say this is inevitable, you say, “No, it’s not. We haven’t even tried yet.” And if that becomes the default meme, then suddenly there is no excuse for saying, “We’re not going to try.”

SAFIAN: Well, Aza, as always, thanks for your passion and your effort and your time. I really appreciate it.

RASKIN: Yeah, it’s such a pleasure, Bob.

SAFIAN: I need to take a deep breath, because that was pretty bracing. I knew Aza would point out AI’s shortcomings. That’s precisely why we wanted him on the show. I just didn’t expect to be quite so convinced about the challenges ahead. At the same time, despite the powerful incentives and resources behind the AI wave, the rapid decline in public trust of AI shows that progress is possible. I certainly didn’t foresee the adoption of social media restrictions around the world, so maybe Aza’s fight isn’t impossible. Whether you see a full-blown AI dystopia on the horizon or not, I’d argue that keeping these conversations going is critical.

So share this episode around. Check out the AI Doc and share it around. Check out Aza’s podcast with Tristan Harris, Your Undivided Attention. Most of all, be open to taking a conscious role, whether as a supportive civilian, as Aza puts it, or a soldier like him. I’m Bob Safian. Thanks for listening.