Trending Now
We have updated our Privacy Policy and Terms of Use for Eurasia Group and its affiliates, including GZERO Media, to clarify the types of data we collect, how we collect it, how we use data and with whom we share data. By using our website you consent to our Terms and Conditions and Privacy Policy, including the transfer of your personal data to the United States from your country of residence, and our use of cookies described in our Cookie Policy.
{{ subpage.title }}
AI superintelligence is coming. Should we be worried?
Are AI companies recklessly racing toward artificial superintelligence or can we avoid a worst case scenario? On GZERO World, Ian Bremmer sits down with Daniel Kokotajlo, co-author of AI 2027, a new report that forecasts how artificial intelligence might progress over the next few years. As AI approaches human-level intelligence, AI 2027 predicts its impact will “exceed that of the Industrial Revolution,” but it warns of a future where tech firms race to develop superintelligence, safety rails are ignored, and AI systems go rogue, wreaking havoc on the global order. Kokotajlo, a former OpenAI researcher, left the company last year warning the company was ignoring safety concerns and avoiding oversight in its race to develop more and more powerful AI. Kokotajlo joins Bremmer to talk about the race to superhuman AI, the existential risk, and what policymakers and tech firms should be doing right now to prepare for an AI future experts warn is only a few short years away.
“One of the unfortunate situations that we're in as a species right now is that humanity in general mostly fixes problems after they happen,” Kokotajlo says, “Unfortunately, the problem of losing control of your army of super intelligences is a problem that we can't afford to wait and see how it goes and then fix it afterwards.”
GZERO World with Ian Bremmer, the award-winning weekly global affairs series, airs nationwide on US public television stations (check local listings).
New digital episodes of GZERO World are released every Monday on YouTube. Don't miss an episode: subscribe to GZERO's YouTube channel and turn on notifications (🔔).GZERO World with Ian Bremmer airs on US public television weekly - check local listings.
What is artificial general intelligence?
Artificial General Intelligence (AGI) is the holy grail of AI research and development. What exactly does AGI mean, and how will we know when we’ve achieved it? On Ian Explains, Ian Bremmer breaks down one of the most exciting (and terrifying) discussions happening in artificial intelligence right now: the race to build AGI, machines that don’t just mimic human thinking but match and then far surpass it. The idea of AGI is still a little hard to define. Some say it’s when a computer can accomplish any cognitive task a human can, others say it’s about transfer learning. Researchers have been predicting AGI’s arrival for decades, but lately, as new AI tools like ChatGPT and DeepSeek become more and more powerful, there is a consensus that achieving true general intelligence in computers isn’t a matter of if, but when. And when it does arrive, they say it will transform almost everything about the way humans live their lives. But is society ready for the huge changes experts warn are only a few years away? What happens when the line between man and machine disappears altogether?
GZERO World with Ian Bremmer, the award-winning weekly global affairs series, airs nationwide on US public television stations (check local listings).
New digital episodes of GZERO World are released every Monday on YouTube. Don't miss an episode: subscribe to GZERO's YouTube channel and turn on notifications (🔔).GZERO World with Ian Bremmer airs on US public television weekly - check local listings.
How DeepSeek changed China’s AI ambitions
But when the Chinese startup DeepSeek released its AI models in January, claiming they matched American ones in performance at much cheaper prices to develop, the US lead was suddenly called into question. If DeepSeek can be believed, they achieved a huge technological advance without unfettered chip access — an affront to the US government’s export controls that, it thought, were keeping China at bay.
After DeepSeek, China is emboldened
Now, the Chinese tech industry seems emboldened, with a slew of new releases from startups and incumbents alike. This breakthrough has jumpstarted AI development across China that has, in an instant, changed global tech competition and reshaped Beijing’s tech strategy.
Alibaba, Tencent, and Baidu, along with newcomers like Manus AI, have since released their own advanced models. Many of these are available for free as open-source software, unlike the subscription models of OpenAI and others.
“DeepSeek shifts the narrative — not by immediately putting China ahead, but by undermining America's AI dominance and forcing Silicon Valley giants onto the defensive much sooner than anticipated,” said Tinglong Dai, professor at Johns Hopkins Carey Business School.
“DeepSeek did two things: increase confidence in China's ability to innovate and convince policymakers to push hard on tech advancement now,” said Kenton Thibaut, senior resident China fellow at the Atlantic Council's Digital Forensic Research Lab.
At a press conference earlier this month, Chinese Foreign Minister Wang Yi wrote off America’s strict export controls. “Where there is blockade, there is breakthrough,” he said. “Where there is suppression, there is innovation; where there is the fiercest storm, there is the platform launching China’s science and technology skyward like the Chinese mythological hero Nezha soaring into the heavens.”
Beijing’s shifting focus
After DeepSeek, Thibaut notes, the Chinese government has signaled it will expand support to finance technological innovation — increasing its relending program budget, establishing a new national venture capital fund, allowing unprofitable firms to go public, and increasing mergers and acquisitions in the Chinese tech sector.
This is a major shift from just a few years ago when Beijing sought to put the explosive domestic tech sector in its place — infamously sinking the IPO of the rideshare giant Didi and closing a key loophole for companies going public on foreign markets in 2021.
Beijing’s incentives are now “aligned” with developing the domestic tech sector, Thibaut said, “Both are aligned on the understanding that companies have major incentives to localize — i.e. using domestically produced chips, even if they aren’t as good as NVIDIA’s — in the long term because of just how uncertain and unpredictable chip availability is and will be.”
And China's embrace of open-source AI models, which are freely available for the public to download and modify, has also raised eyebrows because it stands in contrast with the mostly closed Western models, with Meta’s Llama as a notable exception. If China can get its open-source models to be commonly used by Western developers, it could make an important stake in the global AI space. That said, the open-source model could hinder the economic benefits of AI in China — at least, in terms of making money directly off of these advancements.
For now, we’re witnessing a moment of confidence for China — one shared by both its government and tech sector. “Xi Jinping surely feels emboldened,” Dai said, “viewing this as tangible evidence of Western vulnerability and China’s rising trajectory.”
A computer generated image of the letters AI.
The new AI threats from China
A flurry of impressive new artificial intelligence models is coming online in China. DeepSeek grabbed the world’s attention in January with its powerful and allegedly low-cost R1 model, then Alibaba followed it up with a new model called Qwen 2.5-Max, before Tencent released the model Hunyuan Turbo S that it claimed was faster than DeepSeek.
But there’s even more competition now. On Saturday, Baidu announced two new versions of its AI model, Ernie, that are adept at more complex “reasoning” tasks, a major point of emphasis right now for the top American AI companies like OpenAI and Anthropic. And a Chinese newcomer called Manus recently launched an AI “agent” that can complete multi-step tasks for users — say, order and pay for a pizza — that’s receiving lots of hype though it’s currently invite-only.
The dam has burst in the Chinese tech industry and now every player is racing to release software that can beat DeepSeek but can also compete with the top AI countries in the world.
Liang Wenfeng, founder of startup DeepSeek, delivers a speech at the 10th China Private Equity Golden Bull Awards in 2019 in Shanghai, China.
DeepSeek says no to outside investment — for now
DeepSeek founder Liang Wenfeng has shrugged off hungry requests to invest in the Chinese artificial intelligence startup, according to a Monday report in the Wall Street Journal.
The company’s AI models are open-source and free to use, and Liang is reportedly hesitant to take any outside investment that could change that.
The chatbot company has amassed millions of users while drawing scrutiny from global regulators concerned about a Chinese company handling sensitive user data. Chinese tech giants Tencent and Alibaba are among those that have tried to invest in DeepSeek. The company is effectively controlled by Liang through his hedge fund High-Flyer, though the hedge fund spun off DeepSeek as an independent company when it was created in 2023.
For now, DeepSeek will continue to fly solo. But with pressure from investors, who knows how long Liang will be willing to hold on to the enterprise by himself?
In this photo illustration, a DeepSeek logo is seen displayed on a smartphone with a South Korea Flag in the background.
South Korea halts downloads of DeepSeek
South Korea’s Personal Information Protection Commission, the data privacy authority in the country, said the app “lacked transparency about third-party data transfers and potentially collected excessive personal information.” It’s unclear whether the third party in question is the Chinese government.
The government said the ban will only be lifted after DeepSeek adheres to the country’s privacy and security laws, which are considered among the world’s most stringent. While DeepSeek’s R1 model has quickly become one of the foremost large language models, it’s the first such one to hail from China — and thus, privacy advocates and global regulators have criticized its privacy policy, which states that the company will share data with China. “We store the information we collect in secure servers located in the People's Republic of China,” the policy says. The Chinese Foreign Ministry urged South Korea not to “politicize” trade issues and said Chinese companies comply with local laws where they operate.
Italy already banned DeepSeek nationwide over privacy concerns, while Taiwan and Australia each banned the app on government devices. In the United States, there’s no federal prohibition on government devices, but legislation was introduced earlier this month to do just that. Meanwhile, New York, North Carolina, Texas, and Virginia have all banned DeepSeek on state-owned devices.
Software is difficult to ban — especially since virtual private networks can mask one’s location — but countries concerned by Chinese access to their citizens’ data are trying their best.
DeepSeek logo seen on a cell phone.
First US DeepSeek ban could be on the horizon
Lawmakers in the US House of Representatives want to ban DeepSeek’s AI models from federal devices.
Reps. Josh Gottheimer and Darin LaHood, a Democrat from New Jersey and a Republican from Illinois, respectively, introduced a bill on Thursday called the “No DeepSeek on Government Devices Act.” It would work similarly to the ban of TikTok on federal devices, which was signed into law by President Joe Biden in December 2022. Both bans apply to all government-owned electronics, including phones and computers.
DeepSeek’s R1 large language model is a powerful alternative to the top models from Anthropic, Google, Meta, and OpenAI — the first Chinese model to take the AI world by storm. But its privacy policy indicates that it can send user data to China Mobile, a Chinese state-owned telecom company that’s sanctioned in the US.
Since DeepSeek shot to fame in January, Australia and Taiwan have blocked access on government devices; Italy has banned it nationwide for citizens on privacy grounds. Congress may go further and try to ban DeepSeek in the United States, but so far no members have proposed doing that.
Is DeepSeek the next US national security threat?
Before DeepSeek released its R1 model last month, America’s long-term AI dominance felt like a sure thing.
DeepSeek is a Chinese startup, born from a hedge fund, that claims to have used a fraction of the computing power of US competitors while making an artificial intelligence model that rivals the best that Northern California’s labs have to offer. Critics have alleged that the company has been dishonest about claims it only spent $6 million training the model. But for anyone taking DeepSeek at face value, it has been a revelation that sent shockwaves not only through Silicon Valley but also through Wall Street and Washington.
The Biden administration spent the past few years clamping down on powerful US-made chips flowing into China, but evidently, DeepSeek figured out how to build a great model with a dearth of high-tech resources.
“It shines a spotlight on the limits of the US export control system,” said Xiaomeng Lu, geo-technology director of Eurasia Group. “Technology has evolved in a way that regulators failed to anticipate.”
“Necessity is the mother of invention” – that’s how Jack Corrigan of Georgetown’s Center for Security and Emerging Technology put it. “US efforts to hobble China’s AI sector created a need for Chinese developers to innovate a more efficient approach to AI.”
But DeepSeek’s impact goes beyond its own efficiency. It’s an open-source model, meaning its code is available for anyone to use and modify. “Due to the open-source nature of their model, it will be much harder to restrict access to it entirely,” said Valerie Wirtschafter, a fellow at the Brookings Institution. “The other more pragmatic question is whether Congress has the appetite for more whack-a-mole-style tech regulations, given the chaos that has unfolded since the passage of the TikTok ban.”
US government agencies such as NASA and the Navy have banned DeepSeek models on their devices, as did Congress, but there’s been no US effort to try and ban it more widely among the public, as Italy did on Thursday, citing unresolved data privacy concerns. And America’s top cloud providers, including Microsoft Azure and Amazon Web Services, have already added access to R1.
Justin Sherman, founder of Global Cyber Strategies, says that the Trump administration has a toolbox to “screen, restrict, and even expel non-US tech from the US tech supply chain on national security grounds,” particularly through the Commerce Department’s Information and Communications Technology and Services. Still, he cautions against letting “stock market temperaments, reductive China panic in Washington, and media overinflation of industry AI claims” steer nuanced policy decisions.
DeepSeek’s true threat is likely strategic rather than technical. “DeepSeek’s latest model raises the question of what happens if China becomes the leader in providing publicly available, freely downloadable AI models,” said Sam Winter-Levy, a fellow at the Carnegie Endowment for International Peace. “While the US is obsessed with the race to see who can build the single biggest and most powerful model, perhaps even artificial general intelligence, the Chinese might win the race to see who can build really useful and cost-effective models that will be used by people and companies around the world.” At a minimum, China’s overnight success has quickly leveled the playing field for US-China competition over technology.
Perhaps then the answer to DeepSeek requires a rethinking of what American dominance in AI really means. Banning any specific app or model would just be a Band-Aid on a bullet wound.