Demis Hassabis: Society is “not ready” for the AI future

Note: Transcripts are automatically generated from episode audio, and are not fully corrected for spelling, grammar, and formatting.

DEMIS HASSABIS: A lot of the cyber issues that we’ve seen in the last six months or so could’ve been addressed with better engineering, more robust engineering.

The human’s very much still in the loop, and the next phase, which I think carries the most risk, is closed RSI, closed loop, without the human reviewer process. That will obviously be faster, but I think it’s also a very dangerous way to get to full AGI.

I do think there are some things that generally all the leading labs agree on, like the idea of third-party auditors or evaluators coming in. Obviously, there’s a question about which third parties, and I think we need a healthy ecosystem for that. You could have good start-ups and companies doing that. There’s enough revenue there for them, given the amount of investment going into AI as a whole.

JEFF BERMAN: This is Masters of Scale. I’m Jeff Berman, your host. In today’s episode, our very own Reid Hoffman sits down with the incredible Demis Hassabis. Demis is the co-founder and chair of Google DeepMind and chief scientist of Alphabet. Reid and Demis discuss the race to AGI, the future of AI-fueled scientific discovery, the need for more cooperation on safety efforts, and much more. I hope you enjoy it.

REID HOFFMAN: Demis, thank you for joining us.

HASSABIS: Great to be here. Thanks for having me.

HOFFMAN: We’ve known each other for, I think, well over 10 years now, maybe approaching 12 or something like that.

HASSABIS: Probably, yeah.

HOFFMAN: It was an honor when you asked me to be on the AI ethics board for Google, for our various funny meetings.

HASSABIS: Short-lived, yes.

HOFFMAN: Yes.

Copy LinkHow AI can transform medicine and human health

HOFFMAN: Let’s start, because one of the things we’ve been trying to do at Summit is envision the world as we’d want to make it. The theme has been agency. As you’ve been a leader in this and knew well before most people were thinking about AI, one of the key things for thinking about AI for human benefit is medicine and medical care.

Say a little bit about how you envision this, and how this helps people say, “If I could be getting that, I understand why you AI guys are actually trying to build something that creates a longer human experience, a higher-quality human experience.” Call it anti-x-risk.

HASSABIS: Yeah, for sure. If we take a step back for a moment and look at the big picture, and you know this from knowing us quite early in our DeepMind journey, the whole point of me working on AI for my whole career was to use it as the ultimate tool to advance science and medicine and our understanding of the world around us, while also improving human health. I’ve always said that the number one thing we should use these amazing tools for, which we were dreaming about 20 years ago and have now built, is to improve the human condition and improve human health. I’ve tried to do that, certainly with my own more personal projects, obviously things like AlphaFold, but also the spinout drug discovery company, Isomorphic. I’ve tried to push it really hard in that direction, as I know you have more recently, and I think it’s exactly the sort of thing we need to be showing the world that AI can do. What better thing than to use AI to help us cure terrible diseases?

HOFFMAN: So protein folding was this kind of intersection of the idea of, like, we have really good data. We could use scale learning, because the whole cognitive revolution from learning, starting with even Pong, and self-play and all the rest, made it seem like this would be a really good way of doing it. How does AlphaFold lead to medicines?

One of the questions I get frequently is, “Yeah, you’re talking about that, but is it true? Is it going to happen anytime soon?” So I’m passing along the question I get to you.

HASSABIS: Yes, sure. Well, look, biology is the most complex system in the universe, right?

HOFFMAN: Yeah.

HASSABIS: That’s why it’s so hard, because of the emergent behavior and the interactions. Proteins are the building blocks of life. That’s how I’ve always seen them. The protein folding problem, which is what we tried to solve with AlphaFold, was considered a 50-year grand challenge in biology. It’s sometimes described to me as Fermat’s Last Theorem, but for biology.

HOFFMAN: Yeah.

HASSABIS: It’s critical to understand the structure of a protein from its genetic sequence, its amino acid sequence, to understand its function. That’s really important for understanding basic biological mechanisms, but also for understanding what goes wrong in disease and what things to target. If you’re designing a drug, you need to know what the 3D surface of the protein looks like so you know which part of that protein to bind to. So it was always critical, but still only a small part of the entire drug discovery process. It’s an absolutely critical part of that. With AlphaFold, we were able to accelerate it by orders of magnitude. And interestingly, if you think about it, we finished AlphaFold 2 back in 2020, which is six-plus years ago now, and it’s still probably the best concrete example of AI applied to a really hard scientific problem that was thought to be sort of uncrackable.

HOFFMAN: Yeah.

HASSABIS: That shows you how far ahead of its time it was.

HOFFMAN: Yeah.

HASSABIS: Now we’re really proud of the impact it’s made. Four million researchers around the world use it, pretty much every biology and medical researcher in the world. Almost every drug in the future will have made use of AlphaFold or the structures. But it’s not enough.

HOFFMAN: Yeah.

HASSABIS: We want to expand it to biochemistry and also predict things like toxicity or ADME properties, all these other things that you need to know before you can have a drug that’s safe to use and that’s efficacious and efficient in whatever it is you’re trying to tackle.

Copy LinkHow recursive self-improvement accelerates AI progress

HOFFMAN: Yep. So, speaking of acceleration, let’s go to one of the hot topics of the day: recursive self-improvement.

What is your kind of theory of the game? And I say this because you are a master games person, so I use that deliberately. If you don’t know what I’m referring to, either look at the documentary or look it up on Wikipedia. So, RSI: opportunities, concerns.

HASSABIS: Yeah. Look, RSI is obviously an extremely useful technique to make progress. It mostly works best in what are called verifiable domains. Math is one, which is why we’re seeing so much advance there. Coding’s another. But originally it was games. One of the first famous recursive self-improving systems was one we invented quite a lot of years ago now called AlphaZero, built off the back of AlphaGo, which was the famous program that beat the world champion at Go back in 2016 and was the beginning, I think, of the modern AI era. Then we built a second version that was in some ways more consequential, AlphaZero, that didn’t need to use any human data anymore. It basically played against itself. It started at random, and then after around a dozen to two dozen different epochs, it would play like a million games against itself. It would then train a new version of itself that was slightly better than the old version, and then that would play a million games against itself, and so on.

It could play any game, not just Go. It could play any two-player perfect-information game. For me, it was especially visceral as a chess player because I remember watching it play chess, and within less than 12 hours, it would go from random play in the morning to where, at 11 a.m., I could still just about beat it. Then by lunchtime it was grandmaster level, and by teatime it was better than any chess-playing entity, human or computer.

HOFFMAN: Teatime is dinnertime for the U.S. folks.

HASSABIS: Kind of handcrafted chess computer, it was better than all of them. So we saw the power of that recursive self-improvement. Obviously, the thing all these domains have in common is you can verify the answer. Is it a correct answer? Is it an optimal answer? Is the equation correct? Does the code run? Did you win the game?

HOFFMAN: Yeah.

HASSABIS: Those domains are important because you can verify the answer, so you know when you’ve hit the right point. But it also means you can create almost infinite synthetic data, because you can use the current best versions you have to set new problems or make new moves and then verify whether the synthetic data was correct. The problem with synthetic data a lot of the time is you can’t verify that it’s right or from the right distribution, but in these cases you can. So we’ve seen enormous progress with math and with coding, and coding’s obviously become enormously valuable, and everyone is having to do that now.

But I do worry about it, because there are different forms of RSI. Right now we’re in the early days of RSI, I would say, where what’s happening, and you’re probably all experiencing this as well with your teams, is it’s making a human engineer maybe three, four, or five times more productive. But the human is still very much in the loop, reviewing code, architecting things, and still writing code. It’s all just enhanced. The next phase, which I think carries the most risk, is closed-loop RSI, where you basically manage to figure out how to do it without the human review process in the procedure. That will obviously be faster, but I think it’s also a very dangerous way to get to full AGI. For two reasons, it was sometimes referred to 20 years ago as an intelligence explosion, because you don’t quite know how to keep control of such a system or such a process. And the end result will likely be quite alien. It’s going to be like an alien intelligence. There’s no reason to think it would be in a similar kind of mind space that humans are in. So that will make it very hard for us to understand it, and maybe for it to understand us. I think we should be very cautious as we get closer to that closed-loop form of RSI, and I think all the AI researchers know that. But there is an intense competitive race to improve your coding systems.

HOFFMAN: So, presume that the incentives to be in a race continue. You and I, as some people already know, have tried to figure out whether there are various ways you can get global collaboration to lessen that. You’ve dedicated a huge amount of energy to that, so thank you.

It’s a hard game, as you know. But presume that the competitive pressure to continue doing RSI and coding continues. In that environment, what are the best ways that we can potentially navigate that? Because we have to see where RSI is going and what it’s pointed at. What are the kinds of things the leading labs, such as your own and the others, should be doing to try to navigate this environment?

HASSABIS: Well, what we’re doing, and what I think other leading labs are trying to do, is make sure there are more prosaic protections in place, like ensuring cyber defense is accelerating faster than cyberattack, and making sure you give access to trusted cyber defenders first on maybe slightly fine-tuned versions of your general models that are fine-tuned for cyber defense. That way, you keep the defenders ahead of the attackers. It’s a pretty tricky problem to solve in general.

HOFFMAN: Yeah.

HASSABIS: We’re also building much more hardened sandboxes. I think a lot of the cyber issues we’ve seen in the last six months or so could have been addressed with better engineering, more robust engineering, and better monitoring systems. It shouldn’t be possible for agents to be running for weeks without anyone realizing it. These are the kinds of things we’ve seen from some of our competitors and others. Maybe this is just the beginning of it happening, so I think everyone will get rapidly better. But they have to, because otherwise something bad is going to happen inadvertently. Some kind of incident, I sometimes call it like an oil spill, is going to happen on the current trajectory if we don’t collectively pay a lot more attention and put a lot more resources into these types of systems.

Eventually, you want formal guarantees, maybe some bounds around what these systems can do, and tests for how robust things like the guardrails are. This is actually something I encourage a lot of top mathematicians I know well to think about. They may be pretty sad about the state of pure math, but their skills would be incredible if they pivoted to these types of domains.

More in areas like complexity theory and game theory. Of course, we’re now going to have to start dealing with swarms of these agents, perhaps potentially collaborating in some way. How does one keep track of all of that?

HOFFMAN: Should there be ways that the labs are encouraged, required, or permitted to collaborate on this? And if so, what should some of those ways be? Because it obviously gets stronger if there is sharing of intelligence, methodology, and mutual accountability.

HASSABIS: Yeah, there’s been lots of proposals, and I’ve been working on these things a lot. You saw my essay over the summer.

HOFFMAN: Yeah.

HASSABIS: Maybe some of you read it, about the potential for an SRO, an industry-run standards body with some kind of government oversight over it. I think something like that is going to be needed. Ideally, it becomes international. It’s U.S.-led, and obviously the state of geopolitics today makes it difficult. Our international institutions are very weak, as we all know, just at the time when we need them to be strongest, in a way. So that’s a bit unfortunate, but we’ve got to start somewhere.

HOFFMAN: By the way, British “a bit unfortunate” is holy ****, just to translate.

HASSABIS: Just to translate.

HOFFMAN: To translate.

HASSABIS: Another translation, yes, like teatime and dinnertime. Yes, exactly. “A bit unfortunate.” So we’re trying to do things about that. Various of us are trying to do things about that, and that essay did have an impact in the White House, in the administration, and with our colleagues. Eventually, I think some of that energy became the accord, the famous letter everyone signed.

HOFFMAN: Yeah.

HASSABIS: Which is better than nothing.

HOFFMAN: Yes.

HASSABIS: I don’t think it goes far enough, but it’s a start, and at least it acknowledges from all of the main players the issues that are there and that they need to be addressed. It doesn’t say too much about what to do about them, but that has to come next. At least it gives air cover, I think, for the teams to work together on what sorts of benchmarks matter, what safe best practices are on safety, and what you’re seeing in terms of analysis of what could happen with these types of systems.

HASSABIS: I do think there are some things that generally all the leading labs agree on, like the idea of third-party auditors or evaluators coming in. Obviously, there’s a question about which third parties.

HOFFMAN: Yeah.

HASSABIS: I think we need a healthy ecosystem for that. You could have good start-ups and companies doing it, and there should be enough revenue there for them, given the amount of investment going into AI as a whole. It’s a bit like there are auditors in the finance industry or in any of these industries. So I could see that happening, and some of the big players are moving into this, like Accenture. We’re working with them, and other leaders are as well. I think they’re a little ahead of the game because they’ve acquired a good startup, a good friend of ours, Mark Warner’s startup, so they have some expertise internally. But I think we need more of that. And then there are other things, like the leading labs maybe red-teaming each other before a general release of these systems.

Copy LinkWhy intelligence does not require consciousness

HOFFMAN: So let’s move from that to a little bit more of the fantastical.

It started a few years back. I think it was a Google engineer who said, “I asked Gemini if it was conscious, and it said yes, so therefore it’s conscious,” which seemed a little bit like —

HASSABIS: Yes, premature.

HOFFMAN: A weak inference.

HASSABIS: Yes.

HOFFMAN: But that’s now gaining some momentum with some of the things especially Anthropic is doing, and the way it strategizes thinking that, at some point, it’s almost like a Pascal’s wager.

HASSABIS: Mm.

HOFFMAN: At some point, there will be consciousness in these systems, and we want those conscious systems to look back and say that we were treating them the same way that we treat a 2-year-old.

Which is to say, “Hey, look, we were good to you as a 2-year-old.”

So it kind of continues through. Obviously, to many people, that looks kind of wacky, right?

Part of what I think maybe many people don’t realize is that you also come to this from a depth of expertise in brain science.

One of the things is, it’s not just, “Oh, did the chatbot say it was conscious or not?” That’s not the way to make a decision on this kind of thing. What do you think some of the ways to think about this are, and also how we should be deliberately trying to shape it?

HASSABIS: Yeah, well, look, I think there are quite a few things to say about this. First of all, despite thousands of years of thinking about this, there isn’t an agreed-upon definition of consciousness, so that’s one issue. I think that’s one issue with labeling things conscious.

HOFFMAN: Yeah.

HASSABIS: Especially with the hard problem of consciousness: Is it really feeling this internally, like the way we seem to?

HOFFMAN: Well, I don’t know about you, but yes.

HASSABIS: Yes, exactly. I used to argue with Daniel Dennett about this: that there’s an additional thing. We’re not just behaving in ways we’d imagine conscious beings behave.

HOFFMAN: Yeah.

HASSABIS: But we also share the same substrate.

HOFFMAN: Yes.

HASSABIS: We have the same squishy brains, and so it’s most parsimonious to think you’re feeling what I’m feeling. But of course, with an AI, that’s not the case. You don’t have that second piece, the substrate equivalence.

HOFFMAN: Yeah.

HASSABIS: I think it’s going to be very hard to conclude that. There are dangers. There are two reasons why I think we should be focusing on building intelligent tools today, not conscious entities. One is a safety argument, a technical safety and risk argument, which is that there’s a direct trade-off between creating something with self-volition or self-awareness, even prior to consciousness. These are the prerequisites to a conscious entity, and that conflicts with instruction-following, or what is sometimes called corrigibility.

HASSABIS: This idea of a system actually following instructions, and if it’s going wrong, you as the user, or the human user, telling it, “Stop doing that,” or “Switch yourself off,” or whatever it is, to change course. If you are training your system to be able to reject commands based on its own internal workings, then you’re setting up a situation where, if you’ve got your constitution wrong or your rules wrong, and this is what Asimov was always warning about —

HOFFMAN: Yes.

HASSABIS: With his Three Laws, that was a very simple form of a constitution, and his books are all warnings that you’re never going to get that right in every corner case, in every edge case. I think his books show that very well. Yet we’re a bit hubristically thinking that some people think that what is sometimes called the misalignment problem is under control, just because today’s systems — the good news is they seem more malleable than we might have imagined 15 or 20 years ago.

HOFFMAN: Yeah.

HASSABIS: Which is great news in terms of their steerability.

HOFFMAN: Yeah.

HASSABIS: But that doesn’t mean we’ve solved the misalignment problem in total as we get closer to AGI and superintelligence and so on. I think it’s dangerous to think we have. That’s the first thing. There’s a technical problem, especially if you combine it with something like an RSI.

HOFFMAN: Yes, exactly. It’s the reason why we’re going in this direction.

HASSABIS: We were talking earlier, right? RSI with self-volition is doubly dangerous. That’s one thing. The second thing is more of a societal issue, which is that I don’t think we are ready yet as a society. We haven’t had the debate with wider society — philosophers, theologians, artists, the humanities, not just technologists, and people from many different cultures — about whether we want to cross that second Rubicon.

Are we ready to have the discussion about whether these systems have moral standing? Do we need to worry about what we are doing with them in this box? Are they slaves of some sort? All these questions come up, and they are complex questions. I don’t think we’re equipped to even begin dealing with them. My view is that we should cross the first Rubicon on its own, which is to build these intelligent tools, and then actually use these intelligent tools to do more neuroscience. I’d go back and do more neuroscience to understand what the neural correlates of consciousness really are.

HOFFMAN: Yes.

HASSABIS: Finally understand that.

HOFFMAN: Yes.

HASSABIS: Then that’ll probably take five or 10 years, and then we can cross the second Rubicon—

HOFFMAN: Yes.

HASSABIS: Well, have a discussion about whether we should cross the second Rubicon, and if so, how.

HOFFMAN: Yes.

HASSABIS: Right? Of the conscious-entity type.

HOFFMAN: How would you, because you engage in hours of these discussions, answer the folks who say, “Look, the only way we get the intelligent correlate to be powerful enough is that it has, as it were, the ephemeral accidental?” HOFFMAN: Because it actually has this cognitive state and so on.

HASSABIS: Right. We should talk about this a lot. We discussed this — even 10-plus years ago — and I used to discuss it in my Ph.D. more than 20 years ago — I felt, and I still feel, that intelligence and consciousness are double dissociable.

Which means you can have an intelligent system that’s very smart on IQ and so on, kind of like the systems we have today, which I personally don’t feel are at all conscious yet.

HOFFMAN: Yeah.

HASSABIS: Right? Everyone probably has their own view, but I don’t feel they’re there yet. And then you can have conscious entities that are not that smart. For example, my pet dog — he’s lovely, and he’s pretty conscious, I think. He dreams and everything, but he’s not that smart.

HOFFMAN: Yes. And that’s provable.

HASSABIS: So I think you can have double dissociation here.

HOFFMAN: Yes.

HASSABIS: Many of the higher animals — dolphins, monkeys, elephants, and so on — I think most of us would view them as conscious beings.

They exhibit all of that sort of behavior. And of course, they’re also made of carbon, with the same neurons and types of things we’re made of. So I feel like it’s pretty obvious they have some of those qualities, but they’re not necessarily that smart. It feels like you can dissociate these things, and I think, to the extent that you can, we should.

Copy LinkWhat AI sovereignty means for countries and companies

HOFFMAN: We’re coming close to the end of our session and the end of the Summit. Sad. But what should we think about in terms of the whole notion of AI sovereignty?

Because obviously one of the things going around in a lot of countries right now is: Is this really important? And one of our earliest conversations was with you trying to solve this problem.

HASSABIS: Yeah.

HOFFMAN: So I wanted to share some thinking, and obviously it’s updated from whatever that was, 13 or 14 years ago, but—

HASSABIS: Yeah. Well, look, I think the first thing to say is that it looks to me like there’s going to be half a dozen — six to eight if you include Chinese companies — offerings at the frontier or near the frontier, right? And it feels like there’s going to be leapfrogging, and maybe something is three to six months ahead, but it’s going to be relatively close.

So there are a lot of options, depending on whether you’re talking about an enterprise or a country. I think, as an enterprise, you have a lot of options to arbitrage between these different things. Some of them are better at certain things — multimodal understanding versus coding. It’s very hard to be better on all vectors of importance at once. So I think that’s going to happen.

And then open source also has its risks, because, as we’ve discussed for many years, how do you restrict bad actors from using that while enabling as many good actors, scientists, and others to use these fantastic tools? So that’s an unresolved problem. But open source is maybe only lagging six months to a year behind. Obviously, we need Western open-source solutions.

But I think at this point, if we’re talking about countries as well — and companies that need their data to be private, or want to fine-tune further on their own sensitive data, whether that’s enterprise data or sovereign data — that’s going to be one solution.

There’s also looking at what downstream things can be done with these tools in big companies and big sectors that can benefit from them, like healthcare, where you can use these almost like commodities and then build the value on top of it. I wouldn’t recommend every country, for example, trying to build a frontier model. That just doesn’t make sense to me.

HOFFMAN: Yeah, yeah.

HASSABIS: So they’re going to have to find some other way of shortcutting that.

HOFFMAN: Yes. And also collaborating and working out arrangements with multinational companies like your own.

HASSABIS: Yeah.

HOFFMAN: I think one of the reasons I wanted to end on that question is that part of the theme of Summit is agency. And agency is not just a question of individual agency — like, how do you take agency?

But it’s also a collective question: How do we take agency?

And I think part of what’s actually important in the AI age is that we take agency not just as any one country, but as countries together. It’s part of how we recognize that we are in this together and that we are building for the benefit of humanity. I think that’s the arc we started with, Sarah Friar, and thank you for joining us. Ending on Demis has had such intensity that I was so grateful you set the agenda for how I think about helping in this from our very first discussion at King’s Cross. So let’s give Demis a hand.

HASSABIS: Thank you.

HOFFMAN: Thank you.