Mind your clicks — AI is watching

Privacy experts warn that new computers with built-in artificial intelligence technology mean Big Tech is increasingly watching what you do.

On May 21, Microsoft announced that it’s building AI directly into Windows operating systems that power PCs. The company announced a new line of what they’re calling Copilot+ PCs, available on June 18, built onto Microsoft’s own Surface devices, as well as on Windows devices from manufacturers such as Acer, Dell, HP, Lenovo, and Samsung.

Microsoft’s Copilot chatbot, which runs on OpenAI’s GPT-4o AI model, will factor into many aspects of a user’s computing experience, whether they’re typing on Microsoft Word or browsing the web.

But the star of the show is something called Recall, which watches your activity in a way that can be summoned on-demand when you’re frustrated and looking for something you once saw. “With Recall, you can access virtually what you have seen or done on your PC in a way that feels like having photographic memory,” Microsoft wrote in a blog post.

Cooper Quintin, senior public interest technologist at the Electronic Frontier Foundation, said he is concerned about the issue of data collection when it comes to new AI technology.

“This will create a treasure trove of sensitive information for law enforcement, spies, or hackers,” he said.

Microsoft has said Recall processes and stores content locally on the device, rather than sending it to the cloud, to boost privacy of users.

Calli Schroeder, senior counsel and global privacy counsel at the Electronic Privacy Information Center, said that only ameliorates some concerns. “Constant screenshots of user actions for Recall and other AI programs is a form of surveillance,” she said. “Keeping the data on-device addresses some of the risks of that surveillance but doesn’t change the fact that user activities are being monitored and recorded in great detail and volume.” She noted that data collection is nothing new, but the increased scale presents fresh potential for “misuse, breach, and interference.”

Microsoft’s website says that users can “limit which snapshots Recall collects,” but Quintin and Schroeder both said they hope that the default experience is that users have to opt into using the features rather than opt out.The United Kingdom’s Information Commissioner's Office, the country’s data watchdog, told the BBC that it has contacted Microsoft to request additional information about how Recall works. “We are making enquiries with Microsoft to understand the safeguards in place to protect user privacy," a spokesperson said.

Schroeder said she hopes that technology companies that build AI systems will allow adequate outside monitoring from civil society groups, technologists, researchers, and regulators. “Governments are some of the only bodies with power to force enough disclosure of system structure to confirm whether that is accurate and whether those measures adequately protect people,” she said.

Manufacturers are racing to get their own devices up to speed too. Ahead of the Microsoft announcement, Dell announced its Latitude AI PCs in February (and has seen its stock double in the past six months on AI hype) while HP released its suite of AI PCs in March. And it won’t just be PCs: Google announced it’s adding AI features, powered by its Gemini model, to Chromebooks. Meanwhile, Apple has suggested that its iOS 18 operating system, which powers iPhones, will come with a suite of generative AI features when it’s released in the coming months. And its next chips for Macs and iPads, called M4, should provide the computing power needed in the likely case the company wants to follow its rival’s lead.

Microsoft’s announcement also signals that AI could quickly infiltrate all of our computing experiences — whether we want it or not, and whether we’re aware of it or not. While users still have to search out AI chatbots, they may soon be impossible to avoid.