Emotional AI: More harm than good?

Emotional AI: More harm than good? | GZERO World

Generative AI mimics human-generated text, images, and video, and it's got huge implications for geopolitics, economics, and security. But that's not all - emotionally intelligent AI is on the rise.

And sometimes the results are ugly. Take the mental health nonprofit, KOKO, which used an AI chatbot to support counselors advising 4,000 people who were seeking counseling. The catch: The patients didn't know that a bot was generating the advice they were receiving. While users initially rated the bot-generated responses highly, the therapy lost its effectiveness once the patients were informed that they'd be talking to a fancy calculator.

The real question is: When does emotionally intelligent AI cross the line into emotionally manipulative territory?

This is not just a concern for virtual therapists -- politics could be impacted. And who knows, maybe even your favorite TV host will use generative AI to convince you to keep watching. Now there's an idea.

More from GZERO Media

- YouTube

America’s retreat from global aid is leaving a massive funding gap that no other country is stepping in to fill, leaving the world’s poorest to pay the price, warns IRC president & CEO David Miliband on GZERO World with Ian Bremmer.

College graduate unemployment rate.
Eileen Zhang

“Pain and agony and suffering,” wrote Sam Angel, about his job hunt. He recently graduated with a masters in Cold War military history from Columbia University in New York, having decided to go right into a masters program after finishing undergrad.

Protesters line the street outside Alligator Alcatraz in Ochopee, Florida, holding signs during a vigil on Aug. 10, 2025.

60: A federal judge gave the White House and the Florida state government 60 days to shut down “Alligator Alcatraz,” a controversial immigration detention center in the Florida Everglades that has become a symbol of US President Donald Trump’s severe immigration policies.