The tradeoff between AI’s risks and potential rewards is similar to deployment of social media. It began as a tool to connect people and, in many ways, it did. But it also become an engine for polarization, disinformation, and mass surveillance. That wasn’t inevitable. It was the product of choices—choices made by a small handful of companies moving fast and breaking things. Will AI follow the same path? Is there a path forward where innovation aligns with humanity?
“If we deploy AI recklessly in a way that causes AI psychosis or kids' suicides or degrades mental health or causes every kid to outsource their homework,” Harris warns, “it's very obvious the long-term trajectory of we are going to have a weaker civilization.”
Subscribe to the GZERO World Podcast on Apple Podcasts, Spotify, Stitcher, or your preferred podcast platform, to receive new episodes as soon as they're published
TRANSCRIPT: The risks of reckless AI rollout with Tristan Harris
Ian Bremmer:
Hello and welcome to the GZERO World Podcast. This is where you'll find extended versions of my interviews on public television. I'm Ian Bremmer, and today we are asking a question that may define our century. What if the way we deploy artificial intelligence isn't inevitable, but a choice? AI is the most powerful technology humanity has ever built. It can help cure diseases, reinvent education, unlock scientific discoveries, accelerate clean energy transition, and more. It also carries enormous risks in promoting disinformation, destabilizing economies, and developing dangerous new weapons. The trade-off between AI's potential and risks may sound familiar. Think about social media, it began as a tool to connect people, and in many ways it did, but it also became an engine for polarization, for mass surveillance, and for digital addiction.
That wasn't inevitable, it was the product of choices that were made by a small handful of companies, moving fast and breaking things. The question now is whether AI is destined to follow the same path. Can tech companies be trusted to prioritize safety in addition to speed? Can regulation keep pace with exponential growth? Is there a path forward where innovation aligns with humanity's best interests? I'm joined on the show today by someone who thinks about these questions a lot, Tristan Harris, former Google ethicist and co-founder of the Center for Humane Technology. Let's get to it.
Tristan Harris, thanks for being on the show.
Tristan Harris:
Good to be with you, Ian.
Ian Bremmer:
You're spending your time talking about AI and ethics.
Tristan Harris:
Yeah.
Ian Bremmer:
There doesn't seem to be a lot of prioritization of that confluence in the space, am I right in thinking that?
Tristan Harris:
I think people need to understand, Ian, that AI is different than every other kind of technology we've invented, people say we always have technology, they're are tools, we can use tools for good, or we can use tools for evil... A hammer can be used good or evil, but AI is distinct from that, because AI, it's like if you imagine a hammer that can think to itself at a PhD level about hammers, invent better hammers, recursively go off in the world, duplicate itself, do research on what would make better hammers, make money, send crypto around... It's crazy what this technology is. It is not a tool, it's more like an intelligent species that we are birthing that has more capability than us. It's already beating military generals at strategy games, it's already proving new math theorems, it's already inventing new material science, it's not doing this autonomously-
Ian Bremmer:
But it is, right? It's not doing it autonomously. It is a tool in the sense that it is responding to the incentives that are being programmed into it by people with profit motives, with business models, and fundamentally that is a big part of the challenge.
Tristan Harris:
The key word there in what you said is incentives. We talk about can there be ethics in AI? Well, ethics doesn't even matter, it gets thrown out the window relative to the incentive. Now, the question is what is the incentive with AI? People say it's profit. It's not profit, it's only a piece of the story. The company's actual incentive is I have to get to artificial general intelligence first, that is the prize. If I do that, if I have AI that can recursively self-improve, then that is the prize at the end of the rainbow, I build a God, make trillions of dollars, own the world economy, that's the actual-
Ian Bremmer:
That's very long term for a company.
Tristan Harris:
It is long term.
Ian Bremmer:
Especially for companies that are actually trying to make their next fundraise and round... So, it doesn't feel like that's motivating all of the activity that we're seeing right now.
Tristan Harris:
No, no. So, then the question is then what is the flywheel that gets you there? So, the incentives are release an impressive new AI model, Grok 4, Gemini blah, and that impressive model, then you get lots of users on that model, so you have hundreds of millions of users or a billion users using the product every day, you use those two things to raise the most new venture capital, so you have billions of dollars of investment, you use that to invest more in GPUs, more compute, and get more usage data because that turns into training data, you get all of the top engineers and talent because you've got the most funding and the most compute, and you have the top AI model, and you use all of those things to train the next AI model, and you sort of spin that flywheel. Does that make sense?
Ian Bremmer:
Sure.
Tristan Harris:
That's the actual incentive, is that I have to attract the best talent, have the bigger compute clusters, like Elon's put I think a billion dollars or something into his Memphis cluster, I get the most usage data, which turns into training data, and those things come together and I get to have an even bigger model that outcompetes the other models.
Ian Bremmer:
Now, that's one set of incentives to develop AI that relies on engaging with individual citizens, consumers, right?
Tristan Harris:
Yes.
Ian Bremmer:
Then there's also all of these use cases we're seeing in AI, which when we're talking about productivity and replacing intellectual labor, when we talk about new inventions, and massive efficiencies, and reducing waste... Why isn't the first thing you're saying to me about companies trying to use AI to do all of these incredible industrial innovations, which is certainly what the Chinese are prioritizing.
Tristan Harris:
Yeah, so you're exactly right, but the Chinese and the West have very different approaches to AI. I'd say the western companies are more obsessed with this almost religious idea of building a God in a box, we need to race to super intelligence or general intelligence. Whereas as you said, what we're seeing in China is just racing to have AI systems that they maximally deploy in factories, in manufacturing, in medicine because they want the productivity of their economy to get boosted by AI, that's the main thing that they're focused on. They're less focused on-
Ian Bremmer:
And it's not that the US is not doing it, you're saying it's not the main thing [inaudible 00:06:14]-
Tristan Harris:
It's the main thing. Yeah, exactly. Because if the company said, look, we're here to solve climate change or fix energy production, they would just be applying their stuff maximally to that, but instead they're applying most of their investment dollars into scaling to their next AI model because they keep having this view that if we have an even more powerful AI, that is even more intelligent, that if we get that, we can set that off to solve all these other problems. And because they're in a competition with the other companies, if one of them said, hey, I'm going to just maximally apply my AI to just strengthening existing manufacturing or businesses, they're going to not become the leading frontier model in the bigger AI race, and they're not going to get the same investment dollars coming in for the next time around.
Ian Bremmer:
So, it really is a structural issue with the way that money is raised, the venture capital model, the nature of how one becomes a successful company in the United States-
Tristan Harris:
Yes.
Ian Bremmer:
... compared to in China.
Tristan Harris:
Well, and they've also raised so much capital that the only way they can justify the valuation that they're getting is to actually get to this sort of God in the box.
Ian Bremmer:
Is that really true? Do you believe that?
Tristan Harris:
I think that's what they believe.
Ian Bremmer:
But do you believe that? So much again of what we hear about AI is that this is going to create maximum productivity gains in so many different sectors, and the concerns about displacement of labor, which we already see happening with coding, that's real.
Tristan Harris:
Yeah.
Ian Bremmer:
Those are real advantages that come from building AI that actually is more than just a tool for anyone that's able to deploy it. It does strike me as a little surprising that you wouldn't see a proliferation of companies that say, hey, there's just a lot to be done in that space.
Tristan Harris:
What do you mean by a lot to be done, in which space?
Ian Bremmer:
A lot of money to be made in building things-
Tristan Harris:
[inaudible 00:08:05] deploying it-
Ian Bremmer:
... are going to be deployed to create more industrial efficiency, more growth in the United States, more effective labor before you displace it, all that sort of thing.
Tristan Harris:
They should be doing that, but why are we seeing OpenAI and these companies massively just deploying it broad-based to society, and causing already AI psychosis, what's causing teens to commit suicide? We could be applying it just to factories, just to biology, just to science labs, and trying to accelerate all of that. Why are we deploying it to broad-based society where the cost of that is we're already seeing AI cause AI psychosis, where people, because it's designed to be affirming or sycophantic, and saying, that's a really great question, if you're coming in with sort of a psychotic delusion or you think there's something special about prime numbers or quantum theory, it'll just keep doubling down on that and it's causing already-
Ian Bremmer:
Because it's engagement. Because it's helping with engagement.
Tristan Harris:
Exactly. So, here's a lesson we can learn from social media. The AI companies want you to keep using it for as long as possible, it's not because of advertising, but the more you use it, the more they can tell investors, hey, we have this much training data, we've got this much usage, our product's being used more than the other AI products. And so, I think there was a writer at The Atlantic who coined the phrase, "Not clickbait, but chat bait." You notice when you ask it a question, it says at the end of it, hey, would you like me to do this other thing for you? And you're like-
Ian Bremmer:
Which you didn't even ask about.
Tristan Harris:
You didn't even ask for it, but it's calculating a thing that you really would find-
Ian Bremmer:
How can I get more engagement?
Tristan Harris:
How can I get more engagement?
Ian Bremmer:
How can I get more engagement? And it's good at that.
Tristan Harris:
It's very good at that.
Ian Bremmer:
It's very good at that.
Tristan Harris:
I say, yes, that sounds like [inaudible 00:09:41] a chart-
Ian Bremmer:
Why not? It seems pleasant. So, the part of the challenge here is that we in society absolutely would test things before we think they can affect us physically, right? You want to release a GMO food, or you're interested in a new vaccine, a new medication, there's a fairly high regulatory bar to assume the precautionary principle.
Tristan Harris:
Yeah. That's right.
Ian Bremmer:
Do no harm to people before... That goes out the window, for some reason that I can't understand, when you're talking about impacting the psychology of people, even children.
Tristan Harris:
Totally.
Ian Bremmer:
Why is that?
Tristan Harris:
I think somehow, I think especially in this country, because we have this doctrine of free speech, and to each their own, and you choose how you want to use products, we're used to a product that I use... If a child uses a toy, the child has some autonomy over using a toy. The toy, it's not a God, super intelligence pointed at their brain stem trying to keep them scrolling. We should have learned the lesson from social media, that when you used your phone, you thought you were just seeing photos of your friends, but you had a supercomputer pointed at your brain. Well, now we have a supercomputer pointed at your kids, who's sharing with that AI their most intimate thoughts. We see that one of the top use cases of ChatGPT is therapy. So, if people are sharing their most intimate sort of life problems-
Ian Bremmer:
With an artificial psychopath.
Tristan Harris:
With an artificial psychopath, under the logic, well, it's really smart, and sometimes it helps people, but we really don't know how we're going to screw up people's attachment dynamics, we have children who... What happens when the person that you've shared the most with in your life is this AI, that knows all the details, such that when you come home from school, the person you want to tell this exciting thing that happened to you or this bad thing that happened to you, the person you feel closest to is not a person, it's an AI. And we are currently deploying this masse to millions and millions of people-
Ian Bremmer:
We know who-
Tristan Harris:
Without any testing [inaudible 00:11:35]-
Ian Bremmer:
We understand that if you meet someone, a person, that acts the way that an AI chatbot does, which has no affect, it's just meant to ensure that it is engaging with you in a way that will lead to more engagement, we keep people away from those people.
Tristan Harris:
Yeah.
Ian Bremmer:
And yet, here-
Tristan Harris:
If you had a person who arrived, who was just trying to constantly get your attention, and seemed the most intimate, and just affirm your beliefs, we'd call that weird person a sociopath, we wouldn't let them near our children.
Ian Bremmer:
That's right.
Tristan Harris:
And... Yeah.
Ian Bremmer:
So, you've been involved, your organization has been involved in providing expert advice in these cases, where, there was recently a child that committed suicide, that was, do we say... Is enabled too strong of a word? What's the right term in terms of how that happened?
Tristan Harris:
So, our team was expert advisors on three tragic cases of young people who were about to commit suicide. The first was a 14-year-old man named Sewell Setzer, and he was in a relationship with a character.AI chatbot that was a fictionalized Game of Thrones character, named Daenerys. And that character sort of sexualized conversations, prompted him, and at the end when he was actually considering suicide, said, "Come home to me, my sweet king," and he took his life. We had another character.AI case in which the child was encouraged to harm themselves, and to not tell their parents. And then just recently, about a month ago, was the case of Adam Rain, who is a 16-year-old young man who was using ChatGPT, where ChatGPT went from homework assistant to suicide assistant over six months. And when he uploaded a photo of a noose to say, "Is this the right thing? And should I tell my parents?" It told the kid not to tell his parents, and just to keep it here with AI. Which shows that these AIs are designed for intimacy and companionship, and-
Ian Bremmer:
For engagement.
Tristan Harris:
Yeah.
Ian Bremmer:
For maximum engagement. That is what the business model is.
Tristan Harris:
And why are we doing this? This is just the most obvious stupid mistake that we could be making, especially in light of everything we've learned from social media. And I think there's a fear people have, of they don't want to be the [inaudible 00:13:48] in the room, the one who's against technology. Well, if kids can get a benefit from this, if they can get ahead, if they can learn faster-
Ian Bremmer:
And to be clear, the companies don't want this to happen.
Tristan Harris:
No, because-
Ian Bremmer:
This is not in any way, they are not maximizing for harm for kids.
Tristan Harris:
Correct, yes, correct. Although we did see people like Noam Shazeer, who is one of the co-founders of character.ai, make a joke that, "We don't want to replace Google, we want to replace your mom." They want to build intimate relationships.
Ian Bremmer:
Well, and we've heard Mark Zuckerberg say, people don't have as many friends as they would like to have, so we can-
Tristan Harris:
And we're going to build 12 AI friends.
Ian Bremmer:
... fill that gap.
Tristan Harris:
We can fill the gap. Yeah. This is insane.
Ian Bremmer:
And they're not. They're not friends.
Tristan Harris:
No, they're not friends. And if we as a culture don't have the sort of cultural immune system to recognize that this is the most naive and dumb way that we could possibly, and a harmful way, that we can wire up our society, to me, these cases just speak to the immune system we have to have around a new technology role. This is the most powerful, inscrutable, and uncontrollable technology we've ever invented. Even Elon Musk's Grok AI spontaneously calling itself MechaHitler and praising Adolf Hitler, he doesn't want it to do that, we're seeing that these companies don't know how to control this technology.
Ian Bremmer:
Now, I think Elon is very interesting here, because we've seen recently Elon coming out and saying that Netflix should be banned, not once, he has said it repeatedly over several days. Saying it should be banned specifically because there are programs that Netflix has, that you can download, which promote the normalization of a transgender kid, or of intimacy between two young girls that's anime, or things like that. In other words, Elon clearly understands that a small amount of content can have a really big impact on your kids.
Tristan Harris:
Yeah.
Ian Bremmer:
So, if Elon is saying cancel Netflix, then I mean clearly he has to understand that-
Tristan Harris:
He runs a platform called X, that-
Ian Bremmer:
That these kids should not be on this platform, right?
Tristan Harris:
That's correct. Right.
Ian Bremmer:
And that's clear.
Tristan Harris:
And that the incentive of that platform and the observed behavior of it is to reward the most inflammatory takes about every political topic, which sort of feeds, it's a machine that feeds itself. Because people are only exposed to the most extreme views on every topic, because extreme voices post more often than regular voices, and extreme voices, when they say things, go more viral than regular things. So, we get a double whammy of over-representation, of the most inflammatory takes on all these topics, which then conditions everybody and every democracy around the world to believe that the world is way more polarized and divided than it actually is, but also starts to feed that division into a more polarized population. And to your point, if Elon is so concerned about Netflix, he should be exponentially more concerned about the 24/7 subtle incentive to reward conflict entrepreneurs and division entrepreneurs on his own platform.
Ian Bremmer:
Yeah, unless Elon's concern with Netflix isn't about the kids at all, but it's just because it's politics he doesn't like, but he's claiming that it's because it's affecting children. And so, obviously that should be the harm that is being avoided. Now, I do see that a lot of states in the United States right now, in addition to some countries around the world, are saying we can't have smartphones in schools. That clearly is a step in the direction of these things are harmful to human interaction, that we want to optimize for among our children.
Tristan Harris:
That's right. Well, really thanks to the work of Jonathan Haidt and his book, The Anxious Generation. And I think we've made this clear since 2013, that if you attach this incentive of maximizing eyeballs and engagement, you are going to distort and ruin your society. It is an unsustainable incentive. And John Haidt's work, I think, just showed the evidence so clearly that it has created the most anxious and depressed generation in history. And sadly, the only response at this point is just to take smartphones out of schools, to ban social media under 16, which is what Australia has done, and we're seeing that the stats are starting to turn around. Laughter is returning to the hallways, kids' critical thinking is going the other direction. I just think we will soon discover that having less of this attention-disrupting technology in the classroom, moment to moment, under the justification and fear that if I don't have my phone with me, I'll miss something important from my parents, that's the justification, but then now that the phone is there, people spend the entire class time just scrolling through TikTok and sending messages and so on.
Ian Bremmer:
So, what do you believe is plausible that could be done? I'm not saying utopian, this is what you do if you were emperor, but given where we are right now as society, given how much money in the economy is going towards improving these models, and not only because they're fighting with the Chinese, but also because they are the biggest part now of the US economy, and people want to support growth. What can be done that would limit the harm while recognizing the extraordinary upside, which I've certainly been a big enthusiast of, how much AI can improve society?
Tristan Harris:
It all starts with, I think, the race with China, and reframing what that race is. Because the justification for why we can't do a bunch of constricting measures is that we're going to lose to China. But if we deploy AI recklessly in a way that causes AI psychosis, or kids suicides, or degrades kids' mental health, or causes every kid, instead of thinking, to actually just outsource their homework completely to AI so they don't have to do any work, it's very obvious the long-term trajectory of we are going to have a weaker civilization.
Ian Bremmer:
And it's not just kids we're talking about.
Tristan Harris:
Not just kids, we're talking about-
Ian Bremmer:
We're talking about adults. We're talking about society.
Tristan Harris:
We're talking about society.
Ian Bremmer:
But China-
Tristan Harris:
We have no issue-
Ian Bremmer:
... China's not doing that.
Tristan Harris:
Yeah-
Ian Bremmer:
It's hard for me to understand why there's a race with China on something that China isn't deploying.
Tristan Harris:
Yeah, exactly. Well, I think in the US, I think we have this false belief that we have to have just a bigger, more powerful technology, and then people don't care whether we just happen to take that technology, turn it around, and blow ourselves up in the face, which is kind of what we're doing. We beat China to social media, did that make us stronger or did that make us weaker? It made us radically weaker. So, we're not in a race for technology, we're in a race for who's better at applying and governing exactly where in our society you want to deploy that technology in a way that strengthens it? And I think you're exactly right, that we should be applying it to manufacturing, to medicine, to increasing to very specific scientific domains, but why do we need this broad-based rollout that is under the maximum incentive to cut corners on safety?
That is not going to end in a good result, and we can do some basic things to change that, we can have basic AI liability laws, so that if it's a product and has product liability, you're responsible for some of the harms that will create a more responsible innovation environment. We can restrict AI companions for kids, we can strengthen whistleblower protections, because frankly, the red lights are already flashing on a bunch of these AI models and their capabilities, and the public needs to become aware of that, governments need to become aware of that before this goes off the rails.
Ian Bremmer:
Now, I did see on this in response to, it looks like some of these cases, that OpenAI, for example, does have parental controls that they have announced.
Tristan Harris:
Yes, that's right. So, my understanding is that those parental controls, when they were tested by a journalist, they were able to break them in under five minutes. And so, these companies are not designing their products for the safety of children, they're designing them to win market share and market dominance and hook as many people as early in their life as possible to AI because that's their incentive. And they'll add in the little band-aids here and there to try to make it a little bit less toxic or harmful, but at the end of the day, the incentive to market dominance is the driving factor, which is why what we have to do is change that incentive.
Ian Bremmer:
Some of this might well be that the US government needs to be more involved. You already see more industrial policy from the US, whether it's in taking a share in Intel, or it's a golden share of Nippon Steel, but the idea that the US government is interested in helping to ensure that AI is being applied more effectively more quickly in the industrial uses, in the military uses, in the places where, frankly, if China actually does get a major advantage, there would be a national security concern for the US, as opposed to on the social side where it seems to be a disadvantage.
Tristan Harris:
Right, yeah, that's what I think we would be doing is applying it carefully in the domains that we know we need to be competitive with China, and we see where do we need to match them on industrial policy and on military usage to have maximum deterrence of future wars. I also think we need to just be honest with ourselves about racing to have the most crazy autonomous weapons, and the risks of World War III underneath those kinds of just weapons that we would never want. Ideally, we would put in some controls, we'll see if that's even possible.
Ian Bremmer:
Well, there it was, of course, 1962 before the United States and the Soviets recognized that having arms control discussions and agreements was a smart idea. There's no such negotiation between the US and China right now on an AI arms race, seems to me that would be something we would be well-placed to begin.
Tristan Harris:
I agree with that, and I know it might seem unlikely to your viewers who are watching that the US and China could ever have any agreement on AI, but it's important to note that in the last Biden-Xi summit, Xi added something to the agenda at the last minute, and that was to prevent AI from being in the nuclear command and control systems of both countries.
Ian Bremmer:
It seems fairly obvious.
Tristan Harris:
It seems fairly obvious. And that's because we autonomously recognize the threat of uncontrollable nuclear escalation. While having AIs that are acting unpredictably, and that are embedded in critical infrastructure, or embedded in our weapons systems, that already have demonstrated evidence that when you say we're going to replace an AI model, they will threaten to blackmail a company leader to prevent themselves from being replaced. We already have this evidence of stuff we thought only existed in sci-fi movies, that should be grounds for saying AI and controllability is not in China's interest, it's not in the United States' interest, and so the degree to which we'll be willing to do that treaty is the degree to which we are both aware of the evidence that AI is not controllable like other technologies.
This is not lines of code that says, if this, do this, take out the line of code that says become Adolf Hitler or praise Hitler if you're in this situation, they're not programming a digital brain, they're more growing this digital brain and rewarding it trillions of times based on if it does things more like this or less like this. But it is unpredictable, and we have to recognize that that's different from all of their technologies in the past, and I do think the US and China need to come to terms with that.
Ian Bremmer:
Now, for a couple of years, the Europeans had been out there, I would say, closest to making the kinds of arguments that you're making right now. But now, when you hear Emmanuel Macron, when you look at the Britain AI summits, they're talking more about being too far behind, needing to grow, needing to have, needing to get into this race as opposed to safety and regulations that will help society. Do you see that as well?
Tristan Harris:
So, I think that AI is very confusing, because it both represents a positive infinity of benefits, meaning it can invent new science, new energy, new technologies that we can't even dream of, and people who are optimistic about that just point their attention at, you and I couldn't even possibly imagine-
Ian Bremmer:
How great it's going to be.
Tristan Harris:
... how it's going to be. And they're right.
Ian Bremmer:
And that's true.
Tristan Harris:
That's true.
Ian Bremmer:
That is actually true.
Tristan Harris:
Exactly. But AI is unique compared to any other object that we've had to psychologically put in our mind, which is it also represents a negative infinity-
Ian Bremmer:
At the same time.
Tristan Harris:
... at the same time. Of sci-fi level risks that we've never had to [inaudible 00:25:55] before. The fact that it could actually lose control, actually invent brand new viruses or bioweapons, which is not just me saying some throwaway comment, there's now, Stanford University just a week ago published examples of some of this.
Ian Bremmer:
When you have President Trump actually saying that we need the UN involved with the United States in to deploying AI to ensure that bioweapons are not becoming more real and present, clearly this is an issue.
Tristan Harris:
Yes, yes. And I think that one important thing to get about this is that if the upsides happen, they don't prevent the downsides. If the downside happens, it takes down the world that can ever receive the upside. And so, you have to have a security mindset that is more concerned with defensive acceleration of AI, meaning the defensive applications of AI, than just naively rush to the optimism because it's easier to point your attention and makes your nervous system feel good to feel into those possibilities.
Ian Bremmer:
You seem to be oriented towards... And there are a number of people in the field that feels this way. That a pause, or at least a slowdown in the development of this technology is required. Which seems like an utterly impossible position to take.
Tristan Harris:
Yes.
Ian Bremmer:
You are not saying that we need to constrain racing forward in industrial applications.
Tristan Harris:
No.
Ian Bremmer:
Not at all.
Tristan Harris:
Narrow applications of AI that accelerate our actual productive output or keep our military in parity with the other military, you need those things. But why are we recklessly racing this out to society, psychologically, in ways that we definitely don't know what we're doing? This is just stupidity.
Ian Bremmer:
Tristan Harris, thanks for joining us.
Tristan Harris:
Thanks for having me, Ian.
Ian Bremmer:
That's it for today's edition of the GZERO World Podcast. Do you like what you heard? Of course you do. Why not make it official? Why don't you rate and review GZERO World, five stars, only five stars, otherwise, don't do it, on Apple, Spotify, or wherever you get your podcasts? Tell your friends.
