AI Roundup: Anthropic, OpenClaw, layoffs, and more

Note: Transcripts are automatically generated from episode audio, and are not fully corrected for spelling, grammar, and formatting.

JEREMY KAHN: One reason the Department of War wanted to take this action the way it did is specifically to intimidate other companies into compliance. I think it really wanted to send a message here: You either get on board with doing things the way we want, or the consequences are going to be very severe.

RANA EL KALIOUBY: Are companies really cutting jobs because of the productivity gains of AI?

KAHN: In some cases, companies are saying that these cuts are due to AI when actually they’re using that as an excuse to cover up poor business decisions.

EL KALIOUBY: A lot of these Gulf countries have big ambitions to be AI hubs. How does this instability threaten that?

KAHN: Iran did launch some drone attacks and missile attacks against data centers in the UAE and in Bahrain. It was really the first time that we saw deliberate targeting of data centers.

EL KALIOUBY: Jeremy Kahn is the AI editor and my co-chair at Fortune’s Brainstorm AI Conference, and my go-to person to unpack what is happening in AI.

There’s a lot happening right now: there’s a standoff between Anthropic and the Pentagon, the rise of AI agents and, of course, the instability in the Middle East and how that affects AI.

Even for someone in the AI space day in and day out, it feels like a lot. It’s so hard to keep up. There’s a lot of uncertainty and just a lot of confusion. So let’s try to make sense of it.

I’m Rana El Kaliouby, and this is Pioneers of AI, a podcast taking AI apart.

Hi, Jeremy. So good to see you again. Thank you for joining us again on Pioneers of AI.

KAHN: Rana, it’s great to see you, and thank you for having me back again.

Copy LinkWhy the Anthropic Pentagon clash could reshape AI governance

EL KALIOUBY: All right, so let’s dig into it. Let’s start with Anthropic. This all started to unfold over the last few weeks.

KAHN: It’s definitely still important, Rana, and it is an ongoing situation because Anthropic has now sued the Department of War to try to overturn this decision to label them a supply chain risk. They have filed for an injunction, actually, to stop the ruling from going into effect.

EL KALIOUBY: Anthropic filed two lawsuits, actually, one in California and one in the DC area. There have been some initial hearings in both of those cases, and we will see what happens there. A lot of legal experts I’ve talked to think Anthropic has a very good case. Whether they will get an injunction is another matter.

I want to rewind a little bit. This has been brewing for many months, really since the U.S. government has been working more closely with the big AI companies. But it really escalated when Anthropic demanded that the Department of War not use their technology for domestic surveillance or for fully autonomous weapons, which then the Trump administration deemed made Anthropic a supply chain risk. I guess this is the first time in U.S. history that an American company has been deemed a supply chain risk. Why is this such a big deal?

KAHN: Yeah, so this is really an unprecedented decision by the U.S. government. The supply chain risk designation was actually developed in order to prevent foreign suppliers that might pose a sabotage risk from entering the supply chain of defense products.

A lot of legal experts say it’s really inappropriate for the Department of War to have used the supply chain risk designation essentially in a retaliatory way. They were quite happy to buy Anthropic technology other than these two red lines, and it was really a contract dispute that went awry.

They couldn’t reach contract terms that they could agree on. So a lot of people said, OK, fine, you cancel the contract. But to go further and try to label them a supply chain risk, which means that any Defense Department contractor cannot use Anthropic’s Claude models in the fulfillment of a Department of Defense contract, a lot of people feel like that’s going too far.

Actually, the Department of War, Secretary of War Pete Hegseth, had tried to interpret this supply chain risk ruling even more broadly than that. He had tweeted out on X that the designation meant that anybody who had a Defense Department contract had to cease all commercial relationships with Anthropic, which legal experts say there’s just no basis for in the statute.

Even if the supply chain risk designation holds, it should apply only to the work that companies do in fulfilling Defense Department contracts. It should not mean that they have to sever all commercial relationships with Anthropic and not use Anthropic’s models for any purpose.

Copy LinkHow one government label could become an existential threat

EL KALIOUBY: Yeah, because that would also mean, for example, major providers like AWS or Google Cloud. Both use Claude, right? Technically they would have to stop that.

KAHN: Yes. Not only do AWS and Google have this close relationship with Anthropic, they’re also investors in Anthropic. Some people said if you really take the Hegseth interpretation, they’d have to divest from those investments. It really would be, as some people said, if you take that very broad interpretation, an existential threat to Anthropic. It probably would be the end of them as a company.

EL KALIOUBY: I’m personally really fascinated by this because, as you know, when I had my company Affectiva, we had very similar language in our terms and conditions when we were licensing our technology. Obviously this is at a very different scale, but it’s perhaps the most prominent example where AI ethics and alignment are coming head to head with government use. What do you make of that?

KAHN: Yeah, it’s fascinating. It’s one of the reasons this is such an important issue. It speaks to this wider question about who is going to control advanced AI and who’s going to decide what guardrails should be in place around this technology’s use.

A lot of AI policy people I talk to say we really need Congress to act here. There should be laws around this. It should really be up to democratically elected representatives to make these decisions and set these high-level rules. How do we want AI used in war within the U.S. military? What do we think about the use of AI for the surveillance of U.S. citizens? Those sorts of decisions need to be made by Congress.

But Congress hasn’t acted. As you know, Congress has yet to pass any real federal AI regulation. In the absence of that, we’re falling back on things like a contractual negotiation between one part of the government and Anthropic.

There are also questions about what the whole industry does. Another reason the Anthropic case is so important is that if you look at the number of amicus briefs other people have filed in support of Anthropic, it includes a number of very prominent AI researchers and computer scientists who work at other AI companies. It also includes Microsoft itself as a corporate entity. They filed an amicus brief supporting Anthropic’s position. So I think it’s very interesting. This is a case that’s going to have really big ramifications for these big questions about how we govern AI.

Copy LinkWhy penalizing companies for saying no creates a chilling effect

EL KALIOUBY: What does it mean for the AI industry, but also industry at large, if a company can be penalized for saying no to the government?

KAHN: Yeah, that’s a great question.

A lot of people feel that if you can’t reach a contractual agreement, is it fair that the punishment for that is that the government can essentially try to destroy your business? A lot of people are saying no, that that’s very un-American.

It’s going to have this chilling effect, I think. One reason the Department of War wanted to take this action in the way that it did is specifically to intimidate other companies into compliance. I think it really wanted to send a message here: You either get on board with doing things the way we want, or the consequences are going to be very severe.

We’ve seen this play out several times with the Trump administration, where they’ve taken very punitive action against people who have disagreed with them on some particular policy decision or who they perceive as politically not aligned with them. They have sought to punish those businesses. You saw it with the law firms they felt had done work for the Biden administration or had opposed them in the past. You’ve seen it with the universities, where they’ve stripped funding from universities they feel are not complying with the policies they want. In some cases those actions have been illegal, but they’ve had this tremendous chilling effect, and that seems to be the playbook they’re using here with the Anthropic case.

Copy LinkWhat OpenAI dealmaking reveals about AI red lines in defense

EL KALIOUBY: OK, so as all of this was unfolding, the Department of War worked out a deal with OpenAI that also has a lot of confusion around it. So where do we stand with that?

KAHN: Right. It was a pretty amazing story. Right in the middle of this very public dispute, while the negotiations were still ongoing, you had OpenAI CEO Sam Altman first come out and say that he agreed with the red lines that his rival Anthropic was trying to impose, that basically OpenAI had the same red lines.

Then, just hours later, he announced, oh, by the way, we are talking to the Department of Defense too, and we have reached a deal. According to Sam Altman and according to the national security policy executives at OpenAI, their deal included language that was attempting to establish the same red lines, but with one big exception.

So the exception was that OpenAI agreed to contract language that says it’s OK for the Department of War to use our technology for, quote, all lawful purposes, and then it tries to define that further in the contract.

Critics have pointed out that there are a lot of loopholes here that the Department of War could step through to essentially do the things it wants to do, and could essentially do mass surveillance, could use autonomous weapons and still comply with the contract.

In particular on the surveillance issue, what Anthropic was most worried about was the idea that you could buy a lot of commercially available data. This is data that’s available for sale to anyone. Then you could stitch that together using AI to create quite an accurate picture of the activities of a lot of specific American individuals. The question is, is that mass surveillance? The government is trying to argue it isn’t, and a lot of other people say, well, it feels a lot like mass surveillance.

EL KALIOUBY: We’re going to take a short break. More in a minute.

Copy LinkWhat Nvidia and Bezos signal about the next AI buildout

EL KALIOUBY: So Nvidia just wrapped up GTC, which is their annual global AI conference. What are some of the headlines that came out of that?

KAHN: The biggest headline that came out of the event was Jensen Huang, Nvidia’s CEO, saying that he saw visibility to $1 trillion of GPU and Nvidia product sales through the end of 2027. So that was the big headline number.

That was twice the amount he had predicted Nvidia would sell through 2026. So he’s saying, basically, a year later, I can see that we’re going to double that number, which is already massive — $500 billion — and we’re going to have $1 trillion worth of revenue. It’s a pretty astounding number, although it’s interesting that Nvidia’s stock is priced so highly that the market didn’t respond all that much. It kind of shrugged it off. In fact, there was a little bit of a bump up when he first came out with the number, but by the next day the stock was actually a bit down from where it had been.

So that was the big headline, but under that there were some other very interesting announcements. One of the most interesting, I thought, is that they said their next-generation server rack that’s going to come out by 2027 is going to include both Nvidia’s latest-generation Vera Rubin GPUs, but also these things called LPUs, or language processing units. Those are being made by a company called Groq, which was an AI chip startup.

Nvidia did this partnership deal with them earlier in the year, one of these weird arrangements often called reverse acquihires, where they licensed Groq’s technology and then hired into Nvidia Groq’s co-founder and CEO, Jonathan Ross, and some other members of the founding team.

I think what’s interesting here is it’s an acknowledgment that you might need other chip architectures, particularly when it comes to AI inference. This was really an acknowledgment, for the first time by Nvidia, that maybe the GPU is not the ideal architecture for inference, and maybe we’re going to want to use some other kinds of chips.

EL KALIOUBY: All right. This is also kind of hot off the press. Jeff Bezos is apparently raising a $100 billion fund. It’s an AI manufacturing fund to basically buy manufacturing companies and AI-fy them, implement AI to speed up their automation. He’s targeting aerospace, chipmaking and defense. What do we know about this?

KAHN: Yeah, so this is a brand-new announcement from Bezos. They haven’t started to make any investments yet, but it is very interesting, and it follows a playbook you’ve seen a lot of other private equity players thinking about in the past two years. You’ve actually seen a number of new PE funds launched with a similar concept or thesis.

We’re going to buy up a lot of old-line companies in existing industries, add a bunch of AI to them, they’re going to be so much more productive and then we’re going to sell them back off. We’re going to make a lot of money.

We’ll see what happens here. It probably is true that some manufacturing and some industries are really backward or are using very legacy processes. But it’s also true that a lot of manufacturers have invested a good amount over the past several decades in automation and robotics. I wonder where the gains still are and what companies he is going to buy.

Especially with a $100 billion fund, you’re going to be making some pretty big investments. So these have to be fairly large manufacturers, and yet you have to find large manufacturers that have not already implemented a good amount of automation. So I think it’s going to be interesting to see what he invests in.

EL KALIOUBY: Yeah. One of our investment theses at my fund, Blip Ventures, is that there’s a huge opportunity in these antiquated industries that are maybe not AI-forward, and coming in and reimagining the workflows. So I do believe that this is a huge business opportunity. But I also wonder if we’re underestimating the cultural and actual workforce challenges of bringing people on board and what that looks like.

KAHN: Yeah, I think it’s a fascinating area. It is going to be one of the big differentiators: Can you actually get your workforce to think differently?

Copy LinkWhy AI neo labs are attracting big money despite unclear business models

EL KALIOUBY: OK, so next I want to ask you about these AI neo-labs. These are not companies in the traditional sense. They operate more like private research institutions, and they’re often founded by former OpenAI, DeepMind, Anthropic and Google Brain researchers. The idea is that they’re not really fixated on building a product or shipping a product, but more on exploring what is possible.

KAHN: I’ll just share that there’s a fairly new one. I just saw news about a company called Mere, founded by former Anthropic researchers. They’re raising $175 million at a $1 billion valuation, specifically doing AI R&D for biology and material science.

EL KALIOUBY: Again, as an investor, I am really curious about these companies because, A, they’re raising massive initial rounds of funding at crazy valuations, pre-product and pre-revenue. What is the commercial viability of some of these labs?

KAHN: That’s a really good question. I think the hope is that they all become like OpenAI or Anthropic, or really maybe OpenAI is the best example, because OpenAI started out doing blue-sky research. It was playing around with lots of different ideas around how you build more intelligent AI models.

It was originally doing a lot of work in reinforcement learning, and then it started playing around with these transformer-based language models kind of on the side. Then suddenly those took off, it hit it big, and it reoriented the whole company around that and came out with a way to productize it almost by accident.

It was sort of an accidental company in some ways. I think investors may be hoping that lightning strikes twice and you can do the same thing with another lab — that you’ll fund all this blue-sky research, these people are really smart, they’re going to find something, and then we’ll figure out how to productize it and the money will flow in. So I think that’s kind of the bet.

But it is a high-risk bet, and it’s quite possible that it won’t work out. One of these neo-labs, called Thinking Machines Lab, which Mira Murati, the former CTO of OpenAI, founded with a bunch of other ex-OpenAI alums, has really struggled.

They have put out a small product that helps other AI researchers optimize training. That’s the product they’ve put out so far. But other than that, it’s not really clear what they’re working on. They’ve started to suffer a lot of staff defections to other AI labs.

They lost some people to Meta. They’ve lost some people who’ve gone back to OpenAI. They lost some people who went to do their own startup. They lost someone to Google. You wonder what’s going on there. From the reporting we’ve done on it so far, it does seem like part of the issue has been the lack of a product strategy.

There’s been some disagreement among the co-founders about which way they want to build and what their product is going to be. I think that kind of lack of direction, or lack of agreement, has caused problems. You could see maybe that some of the other neo-labs may have some of the same issues.

But I am interested to see what happens because they are all working on interesting, outstanding problems in AI. A lot of them are trying to solve issues around continuous learning, or they’re trying to build world models that better understand cause and effect, or they’re trying to work on problems around common-sense reasoning. There are all these sort of outstanding issues in the field.

Copy LinkHow to tell real AI productivity gains from AI washing

EL KALIOUBY: So Jeremy, there have been massive layoffs at companies like Amazon, Oracle and Meta. Are companies really cutting jobs because of the productivity gains of AI, or is AI just becoming a scapegoat for a volatile market? And how can we know?

KAHN: Yeah, that’s a really good question, and I think the answer is it’s hard to tell what’s really happening here. In some cases, there’s clearly a little bit of what’s being called AI-washing going on, where companies are saying that these cuts are due to AI and productivity gains from AI because it makes the company sound very cutting-edge, like they’re doing something really smart and they’ve really figured it out, when actually they’re using that as an excuse to cover up poor business decisions.

In a lot of cases they’re still trying to unwind tremendous amounts of overhiring that took place during the pandemic years. Some of these companies tripled their workforces in just three years, and they’re still trying to slim down.

But in some cases I think we really are seeing big productivity gains from AI, particularly around software development. Some of these companies are saying, we just don’t need as many developers.

The other thing I think you’re starting to see, which is AI-related but is not quite about the productivity gains of AI, has to do with the cost of AI and the cost of AI capital spending. Particularly in the case of companies like Oracle and Meta, which are among the companies spending tens of billions of dollars to build out AI infrastructure and data centers, I think there’s some sense that we need to find savings somewhere else, at least some savings, and one place to find it is labor.

I think in some cases companies are betting not on present productivity gains, but on hoped-for future productivity gains. They’re saying, well, we have to find cuts to compensate for this spending somewhere. We’re going to assume that AI is going to enable us to increase productivity in the future and save on labor costs, so we’re going to make some of those cuts now.

One other technical thing that’s going on: some of these software companies have a tremendous amount of stock-based compensation and also have been trading at really high multiples. I think there’s a sense that as those multiples come down, if they keep the same number of employees, they’re going to have to issue even more stock to compensate them, because now each share is worth less. That is going to have this huge dilution effect on existing investors, and I think they’re worried about that. One way to avoid that dilution effect is actually to cut staff.

EL KALIOUBY: More with Jeremy after a short break.

Copy LinkHow Middle East conflict is putting regional AI ambitions at risk

So I want to shift topics now and talk about AI as it relates to home. As you know, I’m Egyptian, and I’ve been checking on family and friends there. There is, of course, so much suffering with the war in Iran and the ripple effects for the Gulf and Lebanon. But I want to talk about the implications of this war for AI.

A lot of these Gulf countries have big ambitions to be AI hubs. How does this instability threaten that?

KAHN: Yeah, it’s very interesting. I think it’s a big outstanding question as we go forward.

We did see in the first week or week and a half of the war that Iran did launch some drone attacks and missile attacks against data centers in the UAE and in Bahrain. It was really the first time that we saw deliberate targeting of data centers. Iran even said that they were targeting these data centers because they knew that AWS did work for the U.S. government. Some of the U.S. military applications are run on AWS, including the instances of Claude that were so controversial.

It’s unclear whether the attacks actually affected any U.S. military operations. But what they did do is take out a lot of other services for people in the Gulf region. I was talking to somebody who said that online banking for his bank in Dubai was down for days after this. It really caused quite a lot of disruption.

I think it’s making people question whether this is the safe region they really want to be investing in when it comes to AI. Do we want to be using these data centers for training and inference to cover large swaths of the world? The countries in the region, Saudi Arabia and the UAE, were trying to sell themselves as, look, our cost of energy is so cheap here that we can produce intelligence at a much better price point than anywhere else.

Therefore, it makes sense to do all your training here. And at least for serving geographies near the Middle East — maybe into Asia, down into Africa, maybe even into Europe — it would make sense to run your inference loads out of the Middle East as well.

I think that’s all being questioned now because of this. I was talking again to a friend who works in the AI sector in the Gulf. He thought it might be even more of a problem trying to convince Western AI researchers to come and work in the region.

The Gulf nations had been relatively successful, particularly the UAE, at doing that. He thinks that’s going to be really challenged, at least for the next few years, and that it’s going to be very hard to lure high-priced and rare AI talent that’s not native to the region to come and relocate there.

EL KALIOUBY: I have a lot of American friends who live in Dubai, and I think the UAE in general has done a great job rallying people and giving people a sense of safety. I don’t know any of my friends who are based in the UAE who are thinking about coming back, but it’s complicated.

Copy LinkWhy AI agents are spreading fast despite major security concerns

It’s also unfolding in real time, so we’ll see where it all lands. All right, let’s talk about AI agents.

Open Claw made headlines a few weeks ago and kind of went viral. It had its viral moment. I didn’t personally try it out because I felt like there were too many security risks. But it is still continuing to grow, apparently especially in China. Tell us more about that, because that surprises me.

KAHN: People in China have gone crazy for Open Claw. I was talking to my colleague who’s based in the region, and he says it’s just nuts. Everybody is using it. Everybody wants to use it. You had the major Chinese internet companies setting up these kiosks where they would help people install Open Claw on their laptops, and people were lining up around the block to get this done.

There were all these stories about little grandmas and aunties who were going to get Open Claw on their laptops. I don’t know what they were using it for, but they were very enthusiastic to have it. People have kind of gone wild for it.

I’m actually quite surprised myself because of the security risks you were talking about. We did a lot of reporting around this, and Open Claw is not the safest thing. It could potentially, depending on what you give it access to, cause all kinds of problems around data security and privacy.

I’m really surprised that in the Chinese context there aren’t a lot of stories about people being scammed or losing vital financial information due to how vulnerable Open Claw is to these prompt injection attacks. It’s pretty vulnerable to potentially importing malware onto your machine, depending on where it goes and explores on the internet.

So, yeah, I’m surprised. But people have apparently gone wild for it and just love it. One of the great things about Open Claw is that it’s persistent, it’s always on, so it can do stuff for you 24/7. I think people love that idea, and people are kind of eager for that.

It’s interesting that Open Claw also spawned all of these copycat products that you’re starting to see come out from the likes of Perplexity, which created this thing called the Perplexity computer, or Microsoft, which created this thing called Microsoft Tasks. Anthropic created a version of Claude that functions a bit like Open Claw now.

The idea of these other versions was to try to give an Open Claw-like experience but solve some of those security challenges. With some of the other solutions now being rolled out by the likes of Microsoft, it is easier to configure and set up. I think it’s going to have to get really easy for people to get mass adoption.

EL KALIOUBY: We use a lot of Anthropic’s Claude Code, and we initially started implementing a chief-of-staff AI agent in the terminal, which took a little bit of finagling. But once you got through the first instantiation of it, it was fine. Now they just have it in their app, and it’s a lot easier to connect Claude Code to your email and your other systems and calendaring and whatnot, and it’s amazing.

I love it. So I’m curious to see what the next iteration of these things is. I think the interfaces are becoming more and more accessible by the day.

KAHN: Yeah, absolutely.

EL KALIOUBY: Yeah, that’s exciting.

So Jeremy, we covered a lot of ground. One last question: What is one topic you’re watching that we haven’t talked about?

KAHN: Oh gosh. One thing I’m watching is what’s going on at Meta. Meta famously spent all this money to create this new superintelligence lab, and they’ve spent tens of billions of dollars building out AI infrastructure, and billions and billions of dollars hiring top AI researchers.

Now we’re hearing reports that they’ve delayed again the release of their latest model, presumably because the training hasn’t gone as well as they wanted. It’s apparently maybe not up to snuff yet. There are rumors about what’s going on there. They also just reorganized how they run AI across the company, creating a new AI applications group that reports to their CTO, Bosworth. So I think that’s one to watch. What’s actually happening there? That’s a story we’re watching.

EL KALIOUBY: Jeremy, that was fascinating. So much to unpack here.

KAHN: Thanks so much for having me.