We have updated our Privacy Policy and Terms of Use for Eurasia Group and its affiliates, including GZERO Media, to clarify the types of data we collect, how we collect it, how we use data and with whom we share data. By using our website you consent to our Terms and Conditions and Privacy Policy, including the transfer of your personal data to the United States from your country of residence, and our use of cookies described in our Cookie Policy.
{{ subpage.title }}
How the Department of Homeland Security’s WMD office sees the AI threat
The US Department of Homeland Security is preparing for the worst possible outcomes from the rapid progression of artificial intelligence technology technology. What if powerful AI models are used to help foreign adversaries or terror groups build chemical, biological, radiological, or nuclear weapons?
The department’s Countering Weapons of Mass Destruction office, led by Assistant Secretary Mary Ellen Callahan, issued a report to President Joe Biden that was released to the public in June, with recommendations about how to rein in the worst threats from AI. Among other things, the report recommends building consensus across agencies, developing safe harbor measures to incentivize reporting vulnerabilities to the government without fear of prosecution, and developing new guidelines for handling sensitive scientific data.
We spoke to Callahan about the report, how concerned she actually is, and how her office is using AI to further its own goals while trying to outline the risks of the technology.
This interview has been edited for clarity and length.
GZERO: We profile a lot of AI tools – some benign, some very scary from a privacy or disinformation perspective. But when it comes to chemical, biological, radiological, and nuclear weapons, what do you see as the main threats?
Mary Ellen Callahan: AI is going to lower barriers to entry for all actors, including malign actors. The crux of this report is to look for ways to increase the promise of artificial intelligence, particularly with chemical and biological innovation, while limiting the perils, finding that kind of right balance between the containment of risk and fostering innovation.
We’re talking in one breath about chemical, biological, radiological, and nuclear threats — they’re all very different. Is there one that you’re most concerned about or see as most urgent?
I don’t want to give away too many secrets in terms of where the threats are. Although the task from the president was chemical, biological, radiological, nuclear threats, we focus primarily on chemical and biological threats for two reasons: One, chemical and biological innovation that is fostered by artificial intelligence is further along, and two, chemical and biological formulas and opportunities have already been included in some AI models.
And also because relatedly, the Department of Energy, which has a specialization in radiological and nuclear threats, is doing a separate classified report.
So, that’s less about the severity of the problem and more about what we’ll face soonest, right?
Well, anything that’s a WMD threat is low probability, but high impact. So we’re concerned about these at all times, but in terms of the AI implementation, the chemical and biological are more mature, I’d say.
How has the rise of AI changed the focus of your job? And is there anything about AI that keeps you up at night?
I would actually say that I am more sanguine now, having done a deeper dive into AI. One, we’re early in the stages of artificial intelligence development, and so we can catch this wave earlier. Two, there is a lot of interest and encouragement with regard to the model developers working with us proactively. There are chokepoints: The physical creation of these threats remains hard. How do you take it from ideation to execution? And there are a lot of steps between now and then.
And so what we’re trying to build into this guidance for AI model developers and others is pathway defeat — to try to develop off-ramps where we can defeat the adversaries, maybe early in their stage, maybe early as they are dealing with the ideation, [so they’re] not even able to get a new formula, or maybe at different stages of the development of a threat.
How are you thinking about the threat of open-source AI models that are published online for anyone to access?
We talked a little bit about open-source, but that wasn’t the focus of the report. But, I think that the more important thing to focus on is the sources of the ingestion of the data – as I mentioned, there is already public source data related to biology and to chemistry. And so whether or not it is an open-source model or not, it's the content of the models that I'm more focused on.
How do you feel about the pace of regulation in this country versus the pace of innovation?
We’re not looking at regulations to be a panacea here. What we’re trying to do right now is to make sure that everyone understands they have a stake in making artificial intelligence as safe as possible, and really to develop a culture of responsibility throughout this whole process — using a bunch of different levers. One lever is the voluntary commitments.
Another lever is the current laws. The current US regime between export controls, privacy, technology transfer, intellectual property, all of those can be levers and can be used in different ways. Obviously, we need to work with our international allies and make sure that we are working together on this. I don’t want to reveal too much, but there is interest that there can be some allied response in terms of establishing best practices.
Secretary Alejandro Mayorkas has noted that regulation can be backward-looking and reactive and might not keep up with the pace of technology. So, therefore, we’re not suggesting or asking for any new authorities or regulations in the first instance. But if we identify gaps, we may revisit whether new authorities or laws are needed.
In terms of legislation, do you think you have what you need to do your job? Or are there clear gaps in what’s on the books?
I actually think that the diverse nature of our laws is actually a benefit and we can really leverage and make a lot of progress with even what we have on the books now — export controls, technology transfers, intellectual property, criminal behavior, and obviously if we have CFATS on the books, that would be great — the Chemical Facility Anti-Terrorism Standards from my friends at CISA. But we do have a lot of robust levers that we can use now. And even those voluntary commitments with the model developers saying they want to do it — if they don’t comply with that, there could even be civil penalties related to that.
Can you tell me about the safe harbor measure that your report recommends and how you want that to work?
There are two aspects to the safe harbor. One is having an “if you see something, say something” aspect. So that means people in labs, people who are selling products, people who say stuff like, “that doesn’t ring true.” This standard can be used as a culture of responsibility.
And if somebody does report, then there could be a safe harbor reporting element — whether they’ve done something inadvertently to create a new novel threat, or they’ve noticed something in the pipeline. The safe harbor for abstaining from civil or criminal prosecution — that may need regulation.
Are you using AI at all in your office?
Yep. Actually, we are using AI on a couple of different detection platforms. The Countering Weapons of Mass Destruction Office has the subject matter expertise for CBRN threats here in the department and we provide training, technology, equipment, and detection capability. So we’ve been using algorithms and AI to help refine our algorithms with regard to identifying radiological, nuclear, chemical, and biological threats. And we’re going to continue to use that. We also are using AI as part of our biosurveillance program as well, both in trying to identify if there is a biodetection threat out there, but also if there is information that would indicate a biological threat out there in the country, and we’re trying to use AI to look for that in content.
Let’s end on an optimistic note. Is there anything else that gives you hope about AI factoring into your work?
The promise of AI is extraordinary. It really is going to be a watershed moment for us, and I'm really excited about this. I think thinking about the safety and security of the chemical and biological threats at this moment is exactly the right time. We’ve got to get in there early enough to establish these standards, these protocols, to share guidance, to fold in risk assessments into these calculations for the model developers, but also for the public at large. So I’m fairly bullish on this now.
Graphic Truth: Big bombs get big budgets in 2023
The world’s nuclear powers increased their spending on these apocalyptic weapons by a record 13% between 2022 and 2023, according to the International Campaign to Abolish Nuclear Weapons. Cumulatively, they spent a cool $91.4 billion on building, maintaining, and researching nuclear weapons.
Well over half of that spending came from the United States, to the tune of $51 billion. The next highest spenders were China and Russia, with comparatively frugal expenditures of $11 billion and $8 billion, respectively. The increases were not driven by building new weapons — arsenal levels remained fairly stable, according to a different study by the Stockholm International Peace Research Institute — but instead by developing new technology to target and launch the weapons.
The US and UK, which saw the largest increases in nuclear spending, are developing new rockets and submarines that they hope will help deter attacks. The US, UK, Russia, China, France, India, and North Korea are also reportedly developing so-called hypersonic missiles, which can travel over five times the speed of sound to avoid interception.
That amount of spending comes to $2,898 every second — roughly what the average global household makes in three months. As if spending vast amounts on weapons that could effectively end the world in about two hours wasn't tragic enough, in countries like North Korea and Pakistan, endemic poverty and economic stagnation mean every dollar spent on nukes is one less spent on food, fuel, and medicine.
Residential and office buildings are seen in Beijing, China, January 10, 2017.
Hard Numbers: Chinese house prices drop, Maryland governor pardons cannabis convicts, Nuclear spending soars, Putin visits Kim, Record migration through Mexico
3.9: China reported Monday that home prices across the country fell at a faster rate in May than at any time since last summer. They’ve dropped3.9% since last May, and they’ve now reached their lowest level since 2014. Housing prices are especially sensitive in China because property was once a primary engine of high growth, but the sector is now deeply in debt.
175,000: Maryland Gov. Wes Moore signed an executive order on Monday to pardonmore than 175,000 cannabis-related convictions. The use of marijuana remains a crime at the federal level, but 24 states have legalized it and another 14 allow marijuana use for medical purposes.
3,000: A nuclear watchdog reports that the world’s nine nuclear-armed states together spent $91.4 billion in 2023. That’s nearly$3,000 per second. The report says the United States spent $51.5 billion, which is “more than all the other nuclear-armed countries put together.” China spent $11.8 billion. Russia spent $8.3 billion.
24: Russia’s Vladimir Putin arrived in Pyongyang today for a two-day visit with North Korean leader Kim Jong Un. It’s Putin’sfirst trip to the DPRK in 24 years, and he and Kim are expected to reaffirm the friendship between their countries. Putin is likely to want ammunition (and maybe some soldiers) for his war in Ukraine. Kim would like to have Russian technologies that can boost his country’s missile program.
177: The Mexican government reported Sunday that some1.39 million people from 177 countries traveled through Mexico so far this year trying to reach the United States without entry papers. For reference, the United Nations has 193 member states.
Nuclear-capable missiles are displayed during a massive parade to mark the 60th anniversary of the founding of the People's Republic of China in Beijing October 1, 2009.
US, China talk tough on nukes and banks
National Security Council arms control official Pranay Vaddiraised a lot of eyebrows recently by saying the US may need to expand its nuclear arsenal. Citing the expansion and diversification of nuclear arsenals by Russia, China, and North Korea, Vaddi toldthe annual meeting of the Arms Control Association that "more nuclear weapons are required to deter our adversaries.”
In response, an unnamed Chinese embassy representative told Russia’s state-affiliated Tass news agency that Washington is "undermining nuclear disarmament and non-proliferation regimes and should stop doing it.” The representative criticized the U.S. for “clinging” to a first-use nuclear policy, withdrawing from arms control treaties and enhancing NATO's nuclear capabilities.
Get ‘em back at the bank. But while China is worried about a potential future war, Western countries are trying to curb Beijing’s support for Russia’s invasion of Ukraine. Sources say the US expects G7 nationsto deliver a stern warning to small Chinese banks to stop providing financial assistance to Russia to wage war on Ukraine. “Our concern is that China is increasingly the factory of the Russian war machine,” said Daleep Singh, US deputy national security adviser for international economics, dubbing Beijing “the arsenal of autocracy.”
Observers don’t expect immediate punitive actions, such as restricting access to the SWIFT messaging system or cutting off access to the dollar. And by targeting smaller institutions instead of larger ones, the G7 seeks to curb support for Russia without causing major disruptions to the global economy. We’ll be watching for the statement – and the fallout - at the upcoming G7 summit in Italy June 13-15.“Keep the nuclear codes away from that robot”
The United States has issued a warning to two fellow nuclear powers, in so many words telling China and Russia, “Keep your nuclear weapons firmly in human control.”
In a May 2 press briefing, US State Department official Paul Dean said that the government has explicitly told France and the United Kingdom that the decision to deploy nuclear weapons must stay out of reach of autonomous artificial intelligence systems — and said it welcomes China and Russia to make the same pronouncement.
Global powers are racing to level-up their military capabilities with cutting-edge artificial intelligence. The US military recently tested an autonomously controlled X-62A jet in a dogfight simulation, which it called a success; and AI has been used on both sides of the Russia-Ukraine war. Meanwhile, the US is trying to cut off China from powerful computer chips needed to run AI systems using stringent export controls, while giving grants to chipmakers willing to expand operations in America.
The State Department’s pronouncement sounds alarming, but the nuclear powers may, in fact, be on the same page, even if their diplomatic interests are more entrenched and complicated.
Alex Brideau, Eurasia Group’s practice head for Eurasia, says he doesn’t believe the US was accusing China or Russia of pursuing AI use in their nuclear command and control controls; rather, Washington is seeking public assurance on the matter. Still, since the US-Russia diplomatic relationship has been strained by the war in Ukraine, Russia might revel in the ambiguity.
“That’s not necessarily because Russia intends to explore the use of AI this way,” Brideau says. “Instead, Moscow might want to add it to the broader set of security issues, nuclear and non-nuclear, that it wants Washington to negotiate over.”
Rick Waters and Jeremy Chan, from Eurasia Group’s China practice, said they think China is on the same page as the US regarding this norm. Chan pointed out that Zhang Jun, who until recently served as China’s permanent representative to the UN, made two important points in a UN speech in March: (1) “nuclear weapons must never be used and a nuclear war never fought,” and (2) “countries should continue to enhance the safety, reliability, and controllability of AI technology and ensure that relevant weapon systems are under human control at all times.”
China may issue an explicit statement after the upcoming US-China dialogue, expected in the coming weeks. That said, Chan thinks Beijing may be reluctant to do so given other unresolved disagreements with the US over nuclear doctrine — “namely the US refusal to commit to a no-first-use policy and reduce its nuclear stockpile.”
The US is simply trying to avoid a classic sci-fi scenario: What algorithm can doom civilization without humanity’s involvement? Surely, that’s the quickest path to annihilation. Luckily, it sounds like its adversaries are already on the same page.
A SpaceX Falcon 9 rocket lifts off from the launch pad at Launch Complex 39-A at the Kennedy Space Center in Cape Canaveral, Fl. in April 2022.
The next frontier of warfare: Russian space-based nukes
Maybe Russia should’ve been invited to Munich after all … News dropped on Thursday that Moscow is developing new space-based nuclear weapons.
Could these new nukes hit American cities? No, according to the White House. But they could hit satellites, wreaking havoc on terrestrial communications, transportation systems, and even financial transactions. In other words, Russia could take cyberattacks to a higher level, literally.
While China and the US also have the ability to attack satellites, neither has gone nuclear with it. The Outer Space Treaty of 1967 explicitly bans the use of nuclear weapons in space. Russia seems not to be paying much heed to that old scrap of paper.
But more dangerous still, the rupture between Moscow and Washington over Ukraine has left the world’s two leading nuclear superpowers with almost no dialogue or treaty limitations on nuclear weapons at all.
That vacuum is now about to extend into space itself.
FILE PHOTO: Military vehicles carrying DF-5B liquid-fuel intercontinental ballistic missiles (ICBM) march past the Tiananmen Rostrum during the military parade to commemorate the 70th anniversary of the victory in the Chinese People's War of Resistance Against Japanese Aggression in Beijing, China, 3 September 2015.
To get closer, the US and China talk nukes
The National Zoo may have lost its giant pandas, the most iconic symbol of US-China friendship, but breakthroughs in military discussions may help stabilize the relationship. Just ahead of the expected Biden-Xi summit in San Francisco, Beijing and Washington held their first talks on nuclear arms since 2019 and are reportedly planning to announce the resumption of formal military-to-military communications.
The State Department describes the nuclear talks as “constructive,” and China’s Foreign Ministry said they broached the subject of nonproliferation. It’s a subtle but important shift: China has rarely factored into arms control discussions in the past, as Russia and the United States possess nearly 90% of the world’s nuclear weapons. Their bilateral relationship over these apocalyptic arsenals far outweighed any other nuclear state’s position, China included.
Until, that is, Russia decided it wasn’t so interested in arms control anymore. In February, Moscow pulled out of the New START treaty, the last remaining arms control deal with the United States. A dangerous development, given the possibility of escalation in Ukraine, but an intriguing political opportunity for Beijing to augment its international status. After all, if Russia doesn’t want to help set the global rules of the road for nuclear weapons, why shouldn’t China?
From the White House’s perspective, sitting down to talk nukes with Beijing can act as a useful confidence-building measure while also addressing growing security concerns. The Pentagon reports that Beijing has expanded its nuclear arsenal to about 500 warheads, with a target of 1,500 by 2035. That would put China about on par with the US in terms of deployed weapons (counting mothballed warheads, the US arsenal still dwarfs all others except Russia’s) and is reason enough to open dialogue. Axios reports both sides are preparing to announce a formal resumption of military contacts, which were suspended last year after former House Speaker Nancy Pelosi visited Taiwan, infuriating Beijing.
Russia, for its part, seems to have woken up and smelled the plutonium. After news of the US-China nuclear talks broke, the Kremlin said it remained willing to talk nukes with Washington — but would not tolerate “lecturing.” If the US and China are talking about setting up nuclear non-proliferation agreements between the two of them, Russia has no say over how or where they are applied. In short, Beijing winds up looking like a responsible global leader while Russia looks like a rogue state.
Israel control in Gaza: No end in sight
Ian Bremmer shares his insights on global politics this week on World In :60.
How long will Israel's indefinite security control in Gaza last?
I think a long, long time. Was it a Colin Powell that said, you know, you break it, you own it in terms of Iraq and Americans were there for a long, long time. I can't see anyone willing to come in and play a security role that will work after the Israelis have wrought absolute destruction on the people and infrastructure of Gaza, which is clearly what is required if you want to destroy Hamas. And indeed Netanyahu has said that they're going to stay and as long as it takes, essentially in terms of security. By the way, the Israeli people strongly don't want that. They don't want an occupation, but they're kind of stuck in one. And that's one of the reasons why a ground war was something to think more carefully about. And look, there are no easy answers for anyone here. This is going to get a lot worse before it gets better.
What do you expect from the Biden-Xi meeting at APEC?
Well, it's definitely happening. Big summit next week in San Francisco. I'll be there, should be a lot of fun. And look, the Chinese, they don't agree with the United States on a lot of policies. There's not a lot of trust in the relationship, but they are adults. And in that regard, they do want ultimately a level of stability. They don't want the world to burn. They'd like, for example, the Israel-Hamas conflict to be over soon. They'd like Russia-Ukraine to be over soon. They're not taking a leadership role on any of these things. And the US meeting, the Biden meeting with Xi, I suspect, is going to be reasonably strong because there's been so much prep for it on a bunch of issues that both countries are trying to build some stability. Fentanyl, for example, artificial intelligence, for example, climate, for example. Having said all of that, Taiwan and technology are the big bugbears in the relationship. And right now they're both heading in a more problematic direction, not in a better one.
Finally, is a nuclear Israel concerning during a time of war?
I'm actually going to say no. A nuclear Israel is actually less concerning in a time of war because it is yet one more reason why the Israelis don't need to feel that they face an existential threat from external enemies. There are a lot of people out there, certainly Hamas, certainly Hezbollah, certainly the Islamic Republic of Iran, whose stated goals are the end of Israel. But they can't come close to bringing it about. And indeed it was the Israeli security failure to under Netanyahu that allowed the October 7th terror attacks to pose such a threat to Israel that it did. So on balance, I'd say it's stabilizing, but I also should say, let's keep in mind that there are some insane people that are on the far right that are still in government in Israel, including the heritage minister who was openly musing about the possibility of using a nuke on Gaza, which is something that should get you fired and Netanyahu hasn't yet, to my knowledge, fired him, which is insane. So, I mean, let's also recognize that there are crazy people in the government, like the Proud Boys of Israel, essentially. And really, they cannot be pushed out fast enough.
- Is Israel ready for the nightmare waiting in Gaza? ›
- Israel orders 'complete siege' of Gaza ›
- Why Israel now supports an Iran nuclear deal ›
- Ian Explains: How Israel & Iran went from friends to enemies ›
- China hawks’ Beijing trip makes a Biden-Xi summit more likely ›
- Can Biden-Xi meeting ease tensions? ›