Emotional AI: More harm than good?

February 28, 2023

Emotional AI: More harm than good? | GZERO World

Generative AI mimics human-generated text, images, and video, and it's got huge implications for geopolitics, economics, and security. But that's not all - emotionally intelligent AI is on the rise.

And sometimes the results are ugly. Take the mental health nonprofit, KOKO, which used an AI chatbot to support counselors advising 4,000 people who were seeking counseling. The catch: The patients didn't know that a bot was generating the advice they were receiving. While users initially rated the bot-generated responses highly, the therapy lost its effectiveness once the patients were informed that they'd be talking to a fancy calculator.

The real question is: When does emotionally intelligent AI cross the line into emotionally manipulative territory?

This is not just a concern for virtual therapists -- politics could be impacted. And who knows, maybe even your favorite TV host will use generative AI to convince you to keep watching. Now there's an idea.

From Your Site Articles

What We’re Watching: Gaza ceasefire begins, Venezuela’s opposition wins Nobel, Peru’s president ousted

Palestinian children look at rubble following Israeli forces' withdrawal from the area, after Israel and Hamas agreed on the Gaza ceasefire, in Khan Younis, in the southern Gaza Strip, October 10, 2025.

REUTERS/Ramadan Abed

Israel approved the Gaza ceasefire deal on Friday morning, bringing the ceasefire officially into effect. The Israeli military must withdraw its forces to an agreed perimeter inside Gaza within 24 hours, and Hamas has 72 hours to return the hostages.

Hard Numbers: Russia hits the heaters in Ukraine, EU Commish survives no-confidence vote, Cameroon’s president runs it back again, scientists find “rogue star”

In these photos, emergency units carry out rescue work after a Russian attack in Ternopil and Prikarpattia oblasts on December 13, 2024. A large-scale Russian missile attack on Ukraine's energy infrastructure left half of the consumers in the Ternopil region without electricity, the Ternopil Regional State Administration reported.

Latin America News Agency

What We’re Watching: China increases its trade leverage, Modi and Starmer meet, US undercuts Canada’s auto industry

U.S. President Donald Trump takes part in a welcoming ceremony with China's President Xi Jinping at the Great Hall of the People in Beijing, China, November 9, 2017.

REUTERS/Damir Sagolj

China has implemented broad new restrictions on exports of rare earth and other critical minerals vital for semiconductors, the auto industry, and military technology, of which it controls 70% of the global supply.

Emotional AI: More harm than good?

GZEROMEDIA

More from GZERO Media

Hard Numbers: Europe to target small packages, US hits Chinese oil refiners, free speech win in UK, Taliban blames Pakistan for Kabul explosion

What We’re Watching: Gaza ceasefire begins, Venezuela’s opposition wins Nobel, Peru’s president ousted

Israel approved the Gaza ceasefire deal on Friday morning, bringing the ceasefire officially into effect. The Israeli military must withdraw its forces to an agreed perimeter inside Gaza within 24 hours, and Hamas has 72 hours to return the hostages.

The unseen wounds of war

We’re living in a time of record-high conflict, a level of violence not seen since World War II. The past four years have been marred by deadly battles in Ukraine, Gaza, Sudan, Myanmar, and beyond, with approximately 14% of the planet’s population now impacted by war.

You vs. the News: A Weekly News Quiz - October 10, 2025

Think you know what's going on around the world? Here's your chance to prove it.

President Macron is down but not out

French President Emmanuel Macron is scrambling to pull France out of a deepening political free fall that’s already toppled five prime ministers in two years. Tomorrow he’ll try again—and this time, says Eurasia Group’s Mujtaba Rahman, the fifth pick might finally stick.

GZEROMEDIA

Trump's role in brokering Israel-Hamas deal

After two years of war in Gaza, a ceasefire and peace deal may finally be within reach, and Ian Bremmer says former President Trump deserves credit for.

Gaza got a peace deal, but Ukraine shouldn’t get its hopes up

Hard Numbers: Russia hits the heaters in Ukraine, EU Commish survives no-confidence vote, Cameroon’s president runs it back again, scientists find “rogue star”

Graphic Truth: Canada's auto industry under attack

Washington and Toronto buzzed this week with talk of the Canada-US relationship. On Tuesday Prime Minister Mark Carney joined President Donald Trump in the Oval Office for a warm but inconclusive meeting.

What We’re Watching: China increases its trade leverage, Modi and Starmer meet, US undercuts Canada’s auto industry

China has implemented broad new restrictions on exports of rare earth and other critical minerals vital for semiconductors, the auto industry, and military technology, of which it controls 70% of the global supply.

Emotional AI: More harm than good?

GZEROMEDIA

Subscribe to GZERO Media's newsletter: Signal

More from GZERO Media

Israel approved the Gaza ceasefire deal on Friday morning, bringing the ceasefire officially into effect. The Israeli military must withdraw its forces to an agreed perimeter inside Gaza within 24 hours, and Hamas has 72 hours to return the hostages.

We’re living in a time of record-high conflict, a level of violence not seen since World War II. The past four years have been marred by deadly battles in Ukraine, Gaza, Sudan, Myanmar, and beyond, with approximately 14% of the planet’s population now impacted by war.

Think you know what's going on around the world? Here's your chance to prove it.

French President Emmanuel Macron is scrambling to pull France out of a deepening political free fall that’s already toppled five prime ministers in two years. Tomorrow he’ll try again—and this time, says Eurasia Group’s Mujtaba Rahman, the fifth pick might finally stick.

GZEROMEDIA

Subscribe to GZERO Media's newsletter: Signal

After two years of war in Gaza, a ceasefire and peace deal may finally be within reach, and Ian Bremmer says former President Trump deserves credit for.

Washington and Toronto buzzed this week with talk of the Canada-US relationship. On Tuesday Prime Minister Mark Carney joined President Donald Trump in the Oval Office for a warm but inconclusive meeting.

China has implemented broad new restrictions on exports of rare earth and other critical minerals vital for semiconductors, the auto industry, and military technology, of which it controls 70% of the global supply.