Meta to start capturing employee mouse movements, keystrokes for AI training data

Meta is installing new tracking software on US-based employees' computers to capture mouse movements, clicks and keystrokes for use in training its artificial-intelligence models, part of a broad initiative to build AI agents that can perform work ‌tasks autonomously, ⁠the ⁠company told staffers in internal memos seen by Reuters.

The tool will run on a list of work-related apps and websites and will also take occasional snapshots of the content on employees' screens for context, according to one memo, posted by a staff AI research scientist on Tuesday in a dedicated ⁠internal channel ‌for the company's model-building Meta SuperIntelligence Labs team.


The ​purpose ​of the exercise, according to the memo, was ⁠to improve the company's models in areas where ​they still struggle, like choosing from dropdown menus ​and using keyboard shortcuts.

"This is where all Meta employees can help our models get better simply by doing their daily work," it said.

Meta spokesperson Andy Stone said the data collected would not be used for performance assessments or ‌any other purpose besides model training and that safeguards were in place to protect sensitive content.

"If we're ​building agents ​to help people ⁠complete everyday tasks using computers, our models need real examples of how people actually use them - things like mouse movements, clicking buttons, ​and navigating dropdown menus. To help, we're launching an internal tool that will capture these kinds of inputs on certain applications to help us train our models," said Stone.