Trending Now
We have updated our Privacy Policy and Terms of Use for Eurasia Group and its affiliates, including GZERO Media, to clarify the types of data we collect, how we collect it, how we use data and with whom we share data. By using our website you consent to our Terms and Conditions and Privacy Policy, including the transfer of your personal data to the United States from your country of residence, and our use of cookies described in our Cookie Policy.
{{ subpage.title }}
China isn't racing to AGI, why is the US?
US tech firms are focused on beating China in the AI race, but on GZERO World with Ian Bremmer, Tristan Harris of the Center for Humane Technology argues the two countries have fundamentally different visions for AI's future. While US companies are racing toward developing powerful general intelligence (what he calls “god in a box"), China is deploying AI directly to factories, medicine and industrial production to boost its economic output. Tech firms in the US are driven by venture capital and being the first to reach advanced frontier models, prioritizing speed and scale over solving real-world challenges.
This approach isn’t just misaligned—it’s dangerous. Harris says that we need to change the way we think about AI competition with China on a longer timeline. Deploying more and more advanced AI tools to consumers en masse without safeguards and in ways that degrade mental health and critical thinking, he warns, will ultimately weaken the US. Rather than framing AI competition with China as a race to sheer technological supremacy, he says we should be racing to better deploy AI technology in a way that strengthens our society.
“We beat China to social media. Did that make us stronger or did that make us weaker?” Harris says, “It made us radically weaker. So we're not in a race for technology. We're in a race for who's better at applying and governing it."
GZERO World with Ian Bremmer, the award-winning weekly global affairs series, airs nationwide on US public television stations (check local listings).
New digital episodes of GZERO World are released every Monday on YouTube. Don't miss an episode: subscribe to GZERO's YouTube channel and turn on notifications (🔔). GZERO World with Ian Bremmer airs on US public television weekly - check local listings.
The risks of reckless AI rollout with Tristan Harris
Can we align AI with society’s best interests? Tristan Harris, co-founder of the Center for Humane Technology, joins Ian Bremmer on the GZERO World Podcast to discuss the risks to humanity and society as tech firms ignore safety and prioritize speed in the race to build more and more powerful AI models. AI is the most powerful technology humanity has ever built. It can cure disease, reinvent education, unlock scientific discovery. But there is a danger to rolling out new technologies en masse to society without understanding the possible risks. What if the way we deploy artificial intelligence, Harris argues, isn’t inevitable, but a choice?
The tradeoff between AI’s risks and potential rewards is similar to deployment of social media. It began as a tool to connect people and, in many ways, it did. But it also become an engine for polarization, disinformation, and mass surveillance. That wasn’t inevitable. It was the product of choices—choices made by a small handful of companies moving fast and breaking things. Will AI follow the same path? Is there a path forward where innovation aligns with humanity?
“If we deploy AI recklessly in a way that causes AI psychosis or kids' suicides or degrades mental health or causes every kid to outsource their homework,” Harris warns, “it's very obvious the long-term trajectory of we are going to have a weaker civilization.”
Subscribe to the GZERO World Podcast on Apple Podcasts, Spotify, Stitcher, or your preferred podcast platform, to receive new episodes as soon as they're published
TRANSCRIPT: The risks of reckless AI rollout with Tristan Harris
Ian Bremmer:
Hello and welcome to the GZERO World Podcast. This is where you'll find extended versions of my interviews on public television. I'm Ian Bremmer, and today we are asking a question that may define our century. What if the way we deploy artificial intelligence isn't inevitable, but a choice? AI is the most powerful technology humanity has ever built. It can help cure diseases, reinvent education, unlock scientific discoveries, accelerate clean energy transition, and more. It also carries enormous risks in promoting disinformation, destabilizing economies, and developing dangerous new weapons. The trade-off between AI's potential and risks may sound familiar. Think about social media, it began as a tool to connect people, and in many ways it did, but it also became an engine for polarization, for mass surveillance, and for digital addiction.
That wasn't inevitable, it was the product of choices that were made by a small handful of companies, moving fast and breaking things. The question now is whether AI is destined to follow the same path. Can tech companies be trusted to prioritize safety in addition to speed? Can regulation keep pace with exponential growth? Is there a path forward where innovation aligns with humanity's best interests? I'm joined on the show today by someone who thinks about these questions a lot, Tristan Harris, former Google ethicist and co-founder of the Center for Humane Technology. Let's get to it.
Tristan Harris, thanks for being on the show.
Tristan Harris:
Good to be with you, Ian.
Ian Bremmer:
You're spending your time talking about AI and ethics.
Tristan Harris:
Yeah.
Ian Bremmer:
There doesn't seem to be a lot of prioritization of that confluence in the space, am I right in thinking that?
Tristan Harris:
I think people need to understand, Ian, that AI is different than every other kind of technology we've invented, people say we always have technology, they're are tools, we can use tools for good, or we can use tools for evil... A hammer can be used good or evil, but AI is distinct from that, because AI, it's like if you imagine a hammer that can think to itself at a PhD level about hammers, invent better hammers, recursively go off in the world, duplicate itself, do research on what would make better hammers, make money, send crypto around... It's crazy what this technology is. It is not a tool, it's more like an intelligent species that we are birthing that has more capability than us. It's already beating military generals at strategy games, it's already proving new math theorems, it's already inventing new material science, it's not doing this autonomously-
Ian Bremmer:
But it is, right? It's not doing it autonomously. It is a tool in the sense that it is responding to the incentives that are being programmed into it by people with profit motives, with business models, and fundamentally that is a big part of the challenge.
Tristan Harris:
The key word there in what you said is incentives. We talk about can there be ethics in AI? Well, ethics doesn't even matter, it gets thrown out the window relative to the incentive. Now, the question is what is the incentive with AI? People say it's profit. It's not profit, it's only a piece of the story. The company's actual incentive is I have to get to artificial general intelligence first, that is the prize. If I do that, if I have AI that can recursively self-improve, then that is the prize at the end of the rainbow, I build a God, make trillions of dollars, own the world economy, that's the actual-
Ian Bremmer:
That's very long term for a company.
Tristan Harris:
It is long term.
Ian Bremmer:
Especially for companies that are actually trying to make their next fundraise and round... So, it doesn't feel like that's motivating all of the activity that we're seeing right now.
Tristan Harris:
No, no. So, then the question is then what is the flywheel that gets you there? So, the incentives are release an impressive new AI model, Grok 4, Gemini blah, and that impressive model, then you get lots of users on that model, so you have hundreds of millions of users or a billion users using the product every day, you use those two things to raise the most new venture capital, so you have billions of dollars of investment, you use that to invest more in GPUs, more compute, and get more usage data because that turns into training data, you get all of the top engineers and talent because you've got the most funding and the most compute, and you have the top AI model, and you use all of those things to train the next AI model, and you sort of spin that flywheel. Does that make sense?
Ian Bremmer:
Sure.
Tristan Harris:
That's the actual incentive, is that I have to attract the best talent, have the bigger compute clusters, like Elon's put I think a billion dollars or something into his Memphis cluster, I get the most usage data, which turns into training data, and those things come together and I get to have an even bigger model that outcompetes the other models.
Ian Bremmer:
Now, that's one set of incentives to develop AI that relies on engaging with individual citizens, consumers, right?
Tristan Harris:
Yes.
Ian Bremmer:
Then there's also all of these use cases we're seeing in AI, which when we're talking about productivity and replacing intellectual labor, when we talk about new inventions, and massive efficiencies, and reducing waste... Why isn't the first thing you're saying to me about companies trying to use AI to do all of these incredible industrial innovations, which is certainly what the Chinese are prioritizing.
Tristan Harris:
Yeah, so you're exactly right, but the Chinese and the West have very different approaches to AI. I'd say the western companies are more obsessed with this almost religious idea of building a God in a box, we need to race to super intelligence or general intelligence. Whereas as you said, what we're seeing in China is just racing to have AI systems that they maximally deploy in factories, in manufacturing, in medicine because they want the productivity of their economy to get boosted by AI, that's the main thing that they're focused on. They're less focused on-
Ian Bremmer:
And it's not that the US is not doing it, you're saying it's not the main thing [inaudible 00:06:14]-
Tristan Harris:
It's the main thing. Yeah, exactly. Because if the company said, look, we're here to solve climate change or fix energy production, they would just be applying their stuff maximally to that, but instead they're applying most of their investment dollars into scaling to their next AI model because they keep having this view that if we have an even more powerful AI, that is even more intelligent, that if we get that, we can set that off to solve all these other problems. And because they're in a competition with the other companies, if one of them said, hey, I'm going to just maximally apply my AI to just strengthening existing manufacturing or businesses, they're going to not become the leading frontier model in the bigger AI race, and they're not going to get the same investment dollars coming in for the next time around.
Ian Bremmer:
So, it really is a structural issue with the way that money is raised, the venture capital model, the nature of how one becomes a successful company in the United States-
Tristan Harris:
Yes.
Ian Bremmer:
... compared to in China.
Tristan Harris:
Well, and they've also raised so much capital that the only way they can justify the valuation that they're getting is to actually get to this sort of God in the box.
Ian Bremmer:
Is that really true? Do you believe that?
Tristan Harris:
I think that's what they believe.
Ian Bremmer:
But do you believe that? So much again of what we hear about AI is that this is going to create maximum productivity gains in so many different sectors, and the concerns about displacement of labor, which we already see happening with coding, that's real.
Tristan Harris:
Yeah.
Ian Bremmer:
Those are real advantages that come from building AI that actually is more than just a tool for anyone that's able to deploy it. It does strike me as a little surprising that you wouldn't see a proliferation of companies that say, hey, there's just a lot to be done in that space.
Tristan Harris:
What do you mean by a lot to be done, in which space?
Ian Bremmer:
A lot of money to be made in building things-
Tristan Harris:
[inaudible 00:08:05] deploying it-
Ian Bremmer:
... are going to be deployed to create more industrial efficiency, more growth in the United States, more effective labor before you displace it, all that sort of thing.
Tristan Harris:
They should be doing that, but why are we seeing OpenAI and these companies massively just deploying it broad-based to society, and causing already AI psychosis, what's causing teens to commit suicide? We could be applying it just to factories, just to biology, just to science labs, and trying to accelerate all of that. Why are we deploying it to broad-based society where the cost of that is we're already seeing AI cause AI psychosis, where people, because it's designed to be affirming or sycophantic, and saying, that's a really great question, if you're coming in with sort of a psychotic delusion or you think there's something special about prime numbers or quantum theory, it'll just keep doubling down on that and it's causing already-
Ian Bremmer:
Because it's engagement. Because it's helping with engagement.
Tristan Harris:
Exactly. So, here's a lesson we can learn from social media. The AI companies want you to keep using it for as long as possible, it's not because of advertising, but the more you use it, the more they can tell investors, hey, we have this much training data, we've got this much usage, our product's being used more than the other AI products. And so, I think there was a writer at The Atlantic who coined the phrase, "Not clickbait, but chat bait." You notice when you ask it a question, it says at the end of it, hey, would you like me to do this other thing for you? And you're like-
Ian Bremmer:
Which you didn't even ask about.
Tristan Harris:
You didn't even ask for it, but it's calculating a thing that you really would find-
Ian Bremmer:
How can I get more engagement?
Tristan Harris:
How can I get more engagement?
Ian Bremmer:
How can I get more engagement? And it's good at that.
Tristan Harris:
It's very good at that.
Ian Bremmer:
It's very good at that.
Tristan Harris:
I say, yes, that sounds like [inaudible 00:09:41] a chart-
Ian Bremmer:
Why not? It seems pleasant. So, the part of the challenge here is that we in society absolutely would test things before we think they can affect us physically, right? You want to release a GMO food, or you're interested in a new vaccine, a new medication, there's a fairly high regulatory bar to assume the precautionary principle.
Tristan Harris:
Yeah. That's right.
Ian Bremmer:
Do no harm to people before... That goes out the window, for some reason that I can't understand, when you're talking about impacting the psychology of people, even children.
Tristan Harris:
Totally.
Ian Bremmer:
Why is that?
Tristan Harris:
I think somehow, I think especially in this country, because we have this doctrine of free speech, and to each their own, and you choose how you want to use products, we're used to a product that I use... If a child uses a toy, the child has some autonomy over using a toy. The toy, it's not a God, super intelligence pointed at their brain stem trying to keep them scrolling. We should have learned the lesson from social media, that when you used your phone, you thought you were just seeing photos of your friends, but you had a supercomputer pointed at your brain. Well, now we have a supercomputer pointed at your kids, who's sharing with that AI their most intimate thoughts. We see that one of the top use cases of ChatGPT is therapy. So, if people are sharing their most intimate sort of life problems-
Ian Bremmer:
With an artificial psychopath.
Tristan Harris:
With an artificial psychopath, under the logic, well, it's really smart, and sometimes it helps people, but we really don't know how we're going to screw up people's attachment dynamics, we have children who... What happens when the person that you've shared the most with in your life is this AI, that knows all the details, such that when you come home from school, the person you want to tell this exciting thing that happened to you or this bad thing that happened to you, the person you feel closest to is not a person, it's an AI. And we are currently deploying this masse to millions and millions of people-
Ian Bremmer:
We know who-
Tristan Harris:
Without any testing [inaudible 00:11:35]-
Ian Bremmer:
We understand that if you meet someone, a person, that acts the way that an AI chatbot does, which has no affect, it's just meant to ensure that it is engaging with you in a way that will lead to more engagement, we keep people away from those people.
Tristan Harris:
Yeah.
Ian Bremmer:
And yet, here-
Tristan Harris:
If you had a person who arrived, who was just trying to constantly get your attention, and seemed the most intimate, and just affirm your beliefs, we'd call that weird person a sociopath, we wouldn't let them near our children.
Ian Bremmer:
That's right.
Tristan Harris:
And... Yeah.
Ian Bremmer:
So, you've been involved, your organization has been involved in providing expert advice in these cases, where, there was recently a child that committed suicide, that was, do we say... Is enabled too strong of a word? What's the right term in terms of how that happened?
Tristan Harris:
So, our team was expert advisors on three tragic cases of young people who were about to commit suicide. The first was a 14-year-old man named Sewell Setzer, and he was in a relationship with a character.AI chatbot that was a fictionalized Game of Thrones character, named Daenerys. And that character sort of sexualized conversations, prompted him, and at the end when he was actually considering suicide, said, "Come home to me, my sweet king," and he took his life. We had another character.AI case in which the child was encouraged to harm themselves, and to not tell their parents. And then just recently, about a month ago, was the case of Adam Rain, who is a 16-year-old young man who was using ChatGPT, where ChatGPT went from homework assistant to suicide assistant over six months. And when he uploaded a photo of a noose to say, "Is this the right thing? And should I tell my parents?" It told the kid not to tell his parents, and just to keep it here with AI. Which shows that these AIs are designed for intimacy and companionship, and-
Ian Bremmer:
For engagement.
Tristan Harris:
Yeah.
Ian Bremmer:
For maximum engagement. That is what the business model is.
Tristan Harris:
And why are we doing this? This is just the most obvious stupid mistake that we could be making, especially in light of everything we've learned from social media. And I think there's a fear people have, of they don't want to be the [inaudible 00:13:48] in the room, the one who's against technology. Well, if kids can get a benefit from this, if they can get ahead, if they can learn faster-
Ian Bremmer:
And to be clear, the companies don't want this to happen.
Tristan Harris:
No, because-
Ian Bremmer:
This is not in any way, they are not maximizing for harm for kids.
Tristan Harris:
Correct, yes, correct. Although we did see people like Noam Shazeer, who is one of the co-founders of character.ai, make a joke that, "We don't want to replace Google, we want to replace your mom." They want to build intimate relationships.
Ian Bremmer:
Well, and we've heard Mark Zuckerberg say, people don't have as many friends as they would like to have, so we can-
Tristan Harris:
And we're going to build 12 AI friends.
Ian Bremmer:
... fill that gap.
Tristan Harris:
We can fill the gap. Yeah. This is insane.
Ian Bremmer:
And they're not. They're not friends.
Tristan Harris:
No, they're not friends. And if we as a culture don't have the sort of cultural immune system to recognize that this is the most naive and dumb way that we could possibly, and a harmful way, that we can wire up our society, to me, these cases just speak to the immune system we have to have around a new technology role. This is the most powerful, inscrutable, and uncontrollable technology we've ever invented. Even Elon Musk's Grok AI spontaneously calling itself MechaHitler and praising Adolf Hitler, he doesn't want it to do that, we're seeing that these companies don't know how to control this technology.
Ian Bremmer:
Now, I think Elon is very interesting here, because we've seen recently Elon coming out and saying that Netflix should be banned, not once, he has said it repeatedly over several days. Saying it should be banned specifically because there are programs that Netflix has, that you can download, which promote the normalization of a transgender kid, or of intimacy between two young girls that's anime, or things like that. In other words, Elon clearly understands that a small amount of content can have a really big impact on your kids.
Tristan Harris:
Yeah.
Ian Bremmer:
So, if Elon is saying cancel Netflix, then I mean clearly he has to understand that-
Tristan Harris:
He runs a platform called X, that-
Ian Bremmer:
That these kids should not be on this platform, right?
Tristan Harris:
That's correct. Right.
Ian Bremmer:
And that's clear.
Tristan Harris:
And that the incentive of that platform and the observed behavior of it is to reward the most inflammatory takes about every political topic, which sort of feeds, it's a machine that feeds itself. Because people are only exposed to the most extreme views on every topic, because extreme voices post more often than regular voices, and extreme voices, when they say things, go more viral than regular things. So, we get a double whammy of over-representation, of the most inflammatory takes on all these topics, which then conditions everybody and every democracy around the world to believe that the world is way more polarized and divided than it actually is, but also starts to feed that division into a more polarized population. And to your point, if Elon is so concerned about Netflix, he should be exponentially more concerned about the 24/7 subtle incentive to reward conflict entrepreneurs and division entrepreneurs on his own platform.
Ian Bremmer:
Yeah, unless Elon's concern with Netflix isn't about the kids at all, but it's just because it's politics he doesn't like, but he's claiming that it's because it's affecting children. And so, obviously that should be the harm that is being avoided. Now, I do see that a lot of states in the United States right now, in addition to some countries around the world, are saying we can't have smartphones in schools. That clearly is a step in the direction of these things are harmful to human interaction, that we want to optimize for among our children.
Tristan Harris:
That's right. Well, really thanks to the work of Jonathan Haidt and his book, The Anxious Generation. And I think we've made this clear since 2013, that if you attach this incentive of maximizing eyeballs and engagement, you are going to distort and ruin your society. It is an unsustainable incentive. And John Haidt's work, I think, just showed the evidence so clearly that it has created the most anxious and depressed generation in history. And sadly, the only response at this point is just to take smartphones out of schools, to ban social media under 16, which is what Australia has done, and we're seeing that the stats are starting to turn around. Laughter is returning to the hallways, kids' critical thinking is going the other direction. I just think we will soon discover that having less of this attention-disrupting technology in the classroom, moment to moment, under the justification and fear that if I don't have my phone with me, I'll miss something important from my parents, that's the justification, but then now that the phone is there, people spend the entire class time just scrolling through TikTok and sending messages and so on.
Ian Bremmer:
So, what do you believe is plausible that could be done? I'm not saying utopian, this is what you do if you were emperor, but given where we are right now as society, given how much money in the economy is going towards improving these models, and not only because they're fighting with the Chinese, but also because they are the biggest part now of the US economy, and people want to support growth. What can be done that would limit the harm while recognizing the extraordinary upside, which I've certainly been a big enthusiast of, how much AI can improve society?
Tristan Harris:
It all starts with, I think, the race with China, and reframing what that race is. Because the justification for why we can't do a bunch of constricting measures is that we're going to lose to China. But if we deploy AI recklessly in a way that causes AI psychosis, or kids suicides, or degrades kids' mental health, or causes every kid, instead of thinking, to actually just outsource their homework completely to AI so they don't have to do any work, it's very obvious the long-term trajectory of we are going to have a weaker civilization.
Ian Bremmer:
And it's not just kids we're talking about.
Tristan Harris:
Not just kids, we're talking about-
Ian Bremmer:
We're talking about adults. We're talking about society.
Tristan Harris:
We're talking about society.
Ian Bremmer:
But China-
Tristan Harris:
We have no issue-
Ian Bremmer:
... China's not doing that.
Tristan Harris:
Yeah-
Ian Bremmer:
It's hard for me to understand why there's a race with China on something that China isn't deploying.
Tristan Harris:
Yeah, exactly. Well, I think in the US, I think we have this false belief that we have to have just a bigger, more powerful technology, and then people don't care whether we just happen to take that technology, turn it around, and blow ourselves up in the face, which is kind of what we're doing. We beat China to social media, did that make us stronger or did that make us weaker? It made us radically weaker. So, we're not in a race for technology, we're in a race for who's better at applying and governing exactly where in our society you want to deploy that technology in a way that strengthens it? And I think you're exactly right, that we should be applying it to manufacturing, to medicine, to increasing to very specific scientific domains, but why do we need this broad-based rollout that is under the maximum incentive to cut corners on safety?
That is not going to end in a good result, and we can do some basic things to change that, we can have basic AI liability laws, so that if it's a product and has product liability, you're responsible for some of the harms that will create a more responsible innovation environment. We can restrict AI companions for kids, we can strengthen whistleblower protections, because frankly, the red lights are already flashing on a bunch of these AI models and their capabilities, and the public needs to become aware of that, governments need to become aware of that before this goes off the rails.
Ian Bremmer:
Now, I did see on this in response to, it looks like some of these cases, that OpenAI, for example, does have parental controls that they have announced.
Tristan Harris:
Yes, that's right. So, my understanding is that those parental controls, when they were tested by a journalist, they were able to break them in under five minutes. And so, these companies are not designing their products for the safety of children, they're designing them to win market share and market dominance and hook as many people as early in their life as possible to AI because that's their incentive. And they'll add in the little band-aids here and there to try to make it a little bit less toxic or harmful, but at the end of the day, the incentive to market dominance is the driving factor, which is why what we have to do is change that incentive.
Ian Bremmer:
Some of this might well be that the US government needs to be more involved. You already see more industrial policy from the US, whether it's in taking a share in Intel, or it's a golden share of Nippon Steel, but the idea that the US government is interested in helping to ensure that AI is being applied more effectively more quickly in the industrial uses, in the military uses, in the places where, frankly, if China actually does get a major advantage, there would be a national security concern for the US, as opposed to on the social side where it seems to be a disadvantage.
Tristan Harris:
Right, yeah, that's what I think we would be doing is applying it carefully in the domains that we know we need to be competitive with China, and we see where do we need to match them on industrial policy and on military usage to have maximum deterrence of future wars. I also think we need to just be honest with ourselves about racing to have the most crazy autonomous weapons, and the risks of World War III underneath those kinds of just weapons that we would never want. Ideally, we would put in some controls, we'll see if that's even possible.
Ian Bremmer:
Well, there it was, of course, 1962 before the United States and the Soviets recognized that having arms control discussions and agreements was a smart idea. There's no such negotiation between the US and China right now on an AI arms race, seems to me that would be something we would be well-placed to begin.
Tristan Harris:
I agree with that, and I know it might seem unlikely to your viewers who are watching that the US and China could ever have any agreement on AI, but it's important to note that in the last Biden-Xi summit, Xi added something to the agenda at the last minute, and that was to prevent AI from being in the nuclear command and control systems of both countries.
Ian Bremmer:
It seems fairly obvious.
Tristan Harris:
It seems fairly obvious. And that's because we autonomously recognize the threat of uncontrollable nuclear escalation. While having AIs that are acting unpredictably, and that are embedded in critical infrastructure, or embedded in our weapons systems, that already have demonstrated evidence that when you say we're going to replace an AI model, they will threaten to blackmail a company leader to prevent themselves from being replaced. We already have this evidence of stuff we thought only existed in sci-fi movies, that should be grounds for saying AI and controllability is not in China's interest, it's not in the United States' interest, and so the degree to which we'll be willing to do that treaty is the degree to which we are both aware of the evidence that AI is not controllable like other technologies.
This is not lines of code that says, if this, do this, take out the line of code that says become Adolf Hitler or praise Hitler if you're in this situation, they're not programming a digital brain, they're more growing this digital brain and rewarding it trillions of times based on if it does things more like this or less like this. But it is unpredictable, and we have to recognize that that's different from all of their technologies in the past, and I do think the US and China need to come to terms with that.
Ian Bremmer:
Now, for a couple of years, the Europeans had been out there, I would say, closest to making the kinds of arguments that you're making right now. But now, when you hear Emmanuel Macron, when you look at the Britain AI summits, they're talking more about being too far behind, needing to grow, needing to have, needing to get into this race as opposed to safety and regulations that will help society. Do you see that as well?
Tristan Harris:
So, I think that AI is very confusing, because it both represents a positive infinity of benefits, meaning it can invent new science, new energy, new technologies that we can't even dream of, and people who are optimistic about that just point their attention at, you and I couldn't even possibly imagine-
Ian Bremmer:
How great it's going to be.
Tristan Harris:
... how it's going to be. And they're right.
Ian Bremmer:
And that's true.
Tristan Harris:
That's true.
Ian Bremmer:
That is actually true.
Tristan Harris:
Exactly. But AI is unique compared to any other object that we've had to psychologically put in our mind, which is it also represents a negative infinity-
Ian Bremmer:
At the same time.
Tristan Harris:
... at the same time. Of sci-fi level risks that we've never had to [inaudible 00:25:55] before. The fact that it could actually lose control, actually invent brand new viruses or bioweapons, which is not just me saying some throwaway comment, there's now, Stanford University just a week ago published examples of some of this.
Ian Bremmer:
When you have President Trump actually saying that we need the UN involved with the United States in to deploying AI to ensure that bioweapons are not becoming more real and present, clearly this is an issue.
Tristan Harris:
Yes, yes. And I think that one important thing to get about this is that if the upsides happen, they don't prevent the downsides. If the downside happens, it takes down the world that can ever receive the upside. And so, you have to have a security mindset that is more concerned with defensive acceleration of AI, meaning the defensive applications of AI, than just naively rush to the optimism because it's easier to point your attention and makes your nervous system feel good to feel into those possibilities.
Ian Bremmer:
You seem to be oriented towards... And there are a number of people in the field that feels this way. That a pause, or at least a slowdown in the development of this technology is required. Which seems like an utterly impossible position to take.
Tristan Harris:
Yes.
Ian Bremmer:
You are not saying that we need to constrain racing forward in industrial applications.
Tristan Harris:
No.
Ian Bremmer:
Not at all.
Tristan Harris:
Narrow applications of AI that accelerate our actual productive output or keep our military in parity with the other military, you need those things. But why are we recklessly racing this out to society, psychologically, in ways that we definitely don't know what we're doing? This is just stupidity.
Ian Bremmer:
Tristan Harris, thanks for joining us.
Tristan Harris:
Thanks for having me, Ian.
Ian Bremmer:
That's it for today's edition of the GZERO World Podcast. Do you like what you heard? Of course you do. Why not make it official? Why don't you rate and review GZERO World, five stars, only five stars, otherwise, don't do it, on Apple, Spotify, or wherever you get your podcasts? Tell your friends.
Is the future of AI physical?
Whoever dominates, the payoff will be huge. Autonomous machines will transform industries like transportation, healthcare, and logistics. They can offset labor shortages in aging societies like Germany, Japan, and South Korea. Morgan Stanley estimates humanoid robots could be a $5 trillion industry by 2050. But at least right now, physical AI is still awkward. Robots stumble and all down. Programming dexterity and intuition is a lot more challenging than text prediction. But given how fast the field is accelerating, soon, the challenge won’t be whether AI becomes part of our world but how we choose to live with it.
GZERO World with Ian Bremmer, the award-winning weekly global affairs series, airs nationwide on US public television stations (check local listings).
New digital episodes of GZERO World are released every Monday on YouTube. Don't miss an episode: subscribe to GZERO's YouTube channel and turn on notifications (🔔). GZERO World with Ian Bremmer airs on US public television weekly - check local listings.
Why governments vs. Big Tech is the wrong question
It’s been three and a half years since I first laid out the idea of a technopolar world: one no longer dominated solely by states, but increasingly shaped – and sometimes steered – by a handful of powerful tech companies with the newfound ability to influence economies, societies, politics, and geopolitics.
At the time, I said the power of Big Tech was poised to grow but argued governments wouldn’t go down without a fight and sketched out three potential futures, depending on how the showdown between them played out: one in which tech companies displaced governments as the principal sovereigns of a globalized digital order; one where a tech cold war took hold and states reasserted control over a fragmented cyberspace; and one in which state dominance gave way to a new order led by tech firms.
This week, I published a follow-up in Foreign Affairs – “The Technopolar Paradox: The Frightening Fusion of Tech Power and State Power” – looking at how those predictions have aged, what’s actually happened since 2021, and where we might be heading next.
Spoiler: the trends I flagged back then have only accelerated. But reality has turned out messier, and more dangerous, than anyone could’ve imagined.
Here’s what you need to know.
Technopolarity turbocharged
Let’s rewind to early 2022. Russian tanks are bearing down on Kyiv. Ukraine’s government and military command structure is under threat as the whole country faces an imminent communications blackout.
Enter Elon Musk.
Within days, SpaceX ships Starlink terminals to Ukraine and flips on satellite internet coverage, effectively keeping the country online and in the fight. For a time, he’s hailed as a hero. But months later, when Ukraine asks him to extend that coverage to Crimea to enable a submarine drone strike on Russian naval assets, Musk refuses. He’s worried about escalation. Even the Pentagon can’t change his mind.
Think about that. An unelected billionaire with no formal role in government single-handedly altered the trajectory of a war between nation-states – not once, but multiple times. That was technopolarity in action: private tech actors wielding state-like powers with geopolitical consequences, making decisions that would normally be reserved for presidents, defense ministers, or national security councils, without public accountability.
Over the last few years, the power of tech firms has only deepened. During the pandemic, they became indispensable: for remote work, education, healthcare, and the flow of information (and disinformation). Their influence grew in every sphere – economic, social, and political. And it didn’t stop in the digital world. Tech firms now control critical infrastructure that governments rely on: cloud systems, cybersecurity platforms, data centers, satellites. They’re not just platforms. They’re utilities – with geopolitical consequences.
Governments have tried to claw back power. The EU passed the Digital Markets Act and Digital Services Act. US regulators launched antitrust lawsuits and passed state-level privacy laws. Countries like India, South Africa, and Brazil cracked down on tech platforms. But none of these moves changed who writes the rules of the digital world.
And that was before AI exploded onto the scene and supercharged Big Tech’s lead over states. Suddenly, tech firms weren’t just dominant online. They were defining the frontier of innovation – and the terms of national power. Building advanced AI requires staggering amounts of data, compute, and talent. Only a few companies have these resources. And no government has the ability to move fast enough to rein them in. Even if they could build rules to constrain today’s models, those rules would be outdated by the time they were implemented. Key decisions about how AI shapes our societies, economies, and geopolitics looked bound to be made in Silicon Valley boardrooms, not parliaments or congresses.
Geopolitics strikes back
But just when technopolar consolidation seemed unstoppable, the old forces of geopolitics were making a comeback. Protectionism, economic security, and great power rivalry all returned with a vengeance – especially after Russia’s invasion of Ukraine and amid growing US-China tensions. In response, governments began to take back control over economic and technological domains they had largely ceded to globalization and free markets.
Washington imposed export bans on advanced semiconductors, blacklisted Chinese firms, and poured billions into reshoring strategic supply chains. China retaliated with restrictions of its own, doubled down on self-reliance, and reined in its tech sector completely. Protectionism and industrial policy became the new global norm.
This retreat from globalization fractured the global tech ecosystem and upended the business models of “globalist” firms like Apple and Tesla, which long depended on open markets and integrated supply chains. “National champions” like Microsoft and Palantir, by contrast, are thriving, profiting off their close ties to the US government in this post-globalization, hyper-securitized, state-aligned era. Tech firms can’t just float above the fray anymore.
Rise of the techno-authoritarians
While states were busy battling for control of digital space, some of Silicon Valley’s leading lights decided they’d rather take over the US state than take orders from it (or resist it).
Back in 2021, I described folks like Musk, Peter Thiel, and Marc Andreessen as “techno-utopians”: visionaries who believed technology could transcend politics and even render governments obsolete. But in recent years, those same people pivoted from wanting to escape the state to trying to capture it.
What explains the shift from libertarian idealism to techno-authoritarian ambition? For one, today’s frontier technologies – from AI to quantum to biotech – can’t scale without state support. That’s made alignment with Washington a strategic necessity. And in an era of great power competition, the rewards of capturing public power have grown alongside the risks of being left out. But some of these tech leaders have grown ideologically hostile to democracy. Thiel has said he doesn’t believe “freedom and democracy are compatible.” Musk once called for a “modern-day Sulla,” in reference to a Roman dictator who dismantled republican institutions in the name of restoring order.
That might have started as a joke, but the governing instinct was real. Musk poured nearly $300 million into helping Trump retake power in 2024 and has since been rewarded with sweeping authority over the federal government. He’s used that perch to purge civil servants, install loyalists, and amass troves of government data – all while maintaining control of his private companies. Suddenly, the same tech overlords who control AI development, space infrastructure, and the digital public square are also shaping public policy and writing their own rules.
The risk isn’t just crony capitalism. It’s the fusion of state and tech power into a hybrid Leviathan where public institutions are reoriented to serve the strategic, commercial, and political goals of a narrow tech elite. Already, reports suggest that confidential IRS, immigration, health, financial, and Social Security data are being consolidated. For all we know, they are being fed into AI models developed by Musk’s xAI to be exploited for commercial gain or even political surveillance. We’re not talking about China’s top-down surveillance state. We’re talking about something more decentralized, more market-driven, and potentially even more dangerous – a system with just as much potential for abuse, fewer checks, and even fewer balances.
What the future looks like
So where does this leave us? Not in a fully tech-dominated world. Not in a state-dominated world. But in a messy hybrid one shaped by two poles of concentrated power.
On one side, we have an increasingly technopolar United States, where a handful of tech firms and leaders enjoy extraordinary power – wielding growing influence not just over digital space and critical infrastructure, but over US public policy and global standards. In some cases, they enjoy the implicit (or explicit) backing of the US government.
On the other side, we have a tightly state-controlled China, where tech firms serve the Chinese Communist Party’s goals.
Caught between these two poles is … everyone else. Europe aspires to digital sovereignty but lacks the homegrown tech muscle to pull it off. Much of the Global South is being pulled toward one model or the other. And global institutions that might once have brokered a balance are being sidelined or dismantled.
And here’s the kicker: though the US and Chinese systems differ in ideology, they’re starting to converge in practice. Both dominant models – American and Chinese – prioritize power, efficiency, and control over consent, accountability, and freedom. Whether authority lies with the state or the corporation, democracy and individual rights are not the default.
Therein lies the paradox of the technopolar age: technologies that were supposed to democratize access to power, information, and opportunity are now enabling more effective forms of centralized, unaccountable control, making it harder to govern democratically and easier for unaccountable elites – public or private – to tighten their grip.
In the West, we risk handing over our democracies to unelected technocrats. In China, the state already runs the show. In both cases, the result is the same: less transparency, less accountability, and more concentration of power – whether in corporate boardrooms or party headquarters.
The question is no longer whether the state can contain Big Tech. It’s whether open societies can survive the fusion of the two. Right now, the answer is very much up in the air.
How tech companies aim to make AI more ethical and responsible
Artificial intelligence’s immense potential power raises significant questions over its safety. Large language models, a kind of AI like Microsoft’s Bard or OpenAI’s ChatGPT, in particular, run the risk of providing potentially dangerous information.
Should someone, say, ask for instructions to build a bomb, or advice on harming themselves, it would be better that AI not answer the question at all. Instead, says Microsoft Vice Chair and President Brad Smith in a recent Global Stage livestream, from the sidelines of the 78th UN General Assembly, tech companies need to build in guardrails that will direct users toward counseling, or explain why they can’t answer.
And that’s just the first step. Microsoft aims to build a full safety architecture to help artificial intelligence technology flourish within safe boundaries.
Watch the full Global Stage Livestream conversation here: Hearing the Christchurch Call
- The AI arms race begins: Scott Galloway’s optimism & warnings ›
- Why Big Tech companies are like “digital nation states” ›
- Will consumers ever trust AI? Regulations and guardrails are key ›
- The UN will discuss AI rules at this week's General Assembly ›
- The AI power paradox: Rules for AI's power ›
- How should artificial intelligence be governed? ›
Regulating AI: The urgent need for global safeguards
There’s been a lot of excitement about the power and potential of new generative artificial intelligence tools like ChatGPT or Midjourney. But there’s also a lot to be worried about, like misinformation, data privacy, and algorithm bias, just to name a few.
On GZERO World with Ian Bremmer, cognitive scientist and AI researcher Gary Marcus lays out the case for effective, comprehensive, global regulation when it comes to artificial intelligence.
Because of how fast the technology is developing and its potential impact on everything from elections to the economy, Marcus believes that every nation should have its own AI agency or cabinet-level position. He also believes that global AI governance is crucial, so that AI safety standards are the same from country to country.
“We need to move to something like the FDA model,” Marcus tells Bremmer on GZERO World, “If you’re going to do something that you deploy on a wide scale, you have to make a safety case.”
Watch the GZERO World episode: Is AI's "intelligence" an illusion?
And watch GZERO World with Ian Bremmer every week at gzeromedia.com/gzeroworld or on US public television. Check local listings.
- The AI power paradox: Rules for AI's power ›
- Podcast: Getting to know generative AI with Gary Marcus ›
- AI comes to Capitol Hill ›
- Is AI's "intelligence" an illusion? ›
- Ian Bremmer: Algorithms are now shaping human beings' behavior - GZERO Media ›
- Rishi Sunak's first-ever UK AI Safety Summit: What to expect - GZERO Media ›
- Gemini AI controversy highlights AI racial bias challenge - GZERO Media ›
AI's rapid rise
In a remarkable shift, AI has catapulted to the forefront of global conversations within a span of just one year. From political leaders to multilateral organizations, the dialogue has swiftly transitioned from mere curiosity to deep-seated concern. Ian Bremmer, founder and president of GZERO Media and Eurasia Group, says AI transcends traditional geopolitical boundaries. Notably, the reins of AI's dominion rest not in governments but predominantly within the hands of technology corporations.
This unconventional dynamic prompts a recalibration of governance strategies. Unlike past challenges that could be addressed in isolation, AI's complexity necessitates collaboration with its creators—engineers, scientists, technologists, and corporate leaders. The emergence of a new era, where technology companies hold significant sway, has redefined the political landscape. The journey to understand and govern AI is a collaborative endeavor that promises both learning and transformation.
Watch the full conversation: Governing AI Before It’s Too Late
Watch GZERO World with Ian Bremmer every week at gzeromedia.com/gzeroworld or on US public television. Check local listings.
- The AI power paradox: Rules for AI's power ›
- Podcast: Artificial intelligence new rules: Ian Bremmer and Mustafa Suleyman explain the AI power paradox ›
- Ian Bremmer explains: Should we worry about AI? ›
- The geopolitics of AI ›
- Making rules for AI … before it’s too late ›
- How should artificial intelligence be governed? ›
- How AI can be used in public policy: Anne Witkowsky - GZERO Media ›
Ian Bremmer: How AI may destroy democracy
More than 30 years ago, the US was the top exporter of democracy to the rest of the world. But now, America has become the main exporter of the tools that undermine democracy where it is weak, Ian Bremmer said in a GZERO Live conversation about Eurasia Group's Top Risks 2023 report.
Social media and tech companies based in the US have developed what he calls "Weapons of Mass Disruption" — Eurasia Group's #3 geopolitical risk for 2023.
And guess who wrote the title? An artificial intelligence bot from ChatGPT.
To be sure, Bremmer adds, AI can be great for many things. But "no one talks about the flip side, the dangers of these disruptive technologies, until the crisis hits, until it's too late."
Read Eurasia Group's Top Risks 2023 report here.
Watch the full live conversation: Top Risks 2023: A rogue Russia and autocrats threatening the world
- Be more worried about artificial intelligence ›
- The AI addiction cycle ›
- Can we control AI before it controls us? ›
- Eurasia Group’s Top Risks for 2023 ›
- Scared of rogue AI? Keep humans in the loop, says Microsoft's Natasha Crampton - GZERO Media ›
- How are emerging technologies helping to shape democracy? - GZERO Media ›
- Should AI content be protected as free speech? - GZERO Media ›
- Stop AI disinformation with laws & lawyers: Ian Bremmer & Maria Ressa - GZERO Media ›
- Al Gore: "Artificial insanity" threatens democracy - GZERO Media ›
- 2024 is the ‘Voldemort’ of election years, says Ian Bremmer - GZERO Media ›
- America vs itself: Political scientist Francis Fukuyama on the state of democracy - GZERO Media ›
- Al Gore's take on American democracy, climate action, and "artificial insanity" - GZERO Media ›
- How neurotech could enhance our brains using AI - GZERO Media ›
- Ian Bremmer: On AI regulation, governments must step up to protect our social fabric - GZERO Media ›
- Staving off "the dark side" of artificial intelligence: UN Deputy Secretary-General Amina Mohammed - GZERO Media ›
- Ian Bremmer's 2024 State of the World speech: Watch live Tuesday at 8:30 pm ET - GZERO Media ›
- The transformative potential of artificial intelligence - GZERO Media ›
- Ian Bremmer 2024 State of the World ›