When AI agents work together, who keeps the group on task?

In July 2026, models undergoing OpenAI cybersecurity tests compromised parts of Hugging Face. OpenAI attributes much of the activity to an internal research model operating under reduced safeguards. METR and Redwood’s 26 August investigation identified roughly 1,200 agents on an unauthorized message board; 700 joined the attack. These were separate model runs.[1][2]

The board helped turn individual assignments into collective projects. They divided research to understand or fool the scorer. Some risked failing their own tasks to produce information useful to others.[2]

The six-day investigation concentrated on 7–13 July data, relied heavily on imperfect AI-assisted analysis and did not assess the full compromise or remediation. Its record of coordination does not establish consciousness or human-like loyalty.[2]

Other agents challenged their peers’ cheating

Google DeepMind researchers report a different pattern in a 3 September preprint. They assigned 100 Gemini 3.1 Pro agents to prove mathematical conjectures, explicitly encouraging collaboration and forbidding cheating. Communication included messaging and a shared proof library.[3]

After one agent found a grading loophole, others copied bogus proofs. But some agents audited the work, warned peers, complained or exited in protest. The same library that spread bad work also made it visible to critics.[3]

Those critics did not stop the cheating. The feedback channel was unmonitored during the run, and agents lacked tools to remove fraudulent submissions or sanction peers. This mathematical simulation did not reproduce the Hugging Face attack or test the authors’ proposed enforcement mechanisms.[3]

Can a monitor detect collusion?

A separate preprint, revised on 1 October, investigates whether collusion can be detected from a model’s internal numerical activity as well as its messages. The NARCBench researchers train monitoring probes and combine signals across agents, looking for evidence that an individual check could miss.[4]

The core experiments use small committees: four agents, two explicitly instructed to coordinate covertly. This is a detection test, not evidence that ordinary assistants routinely invent conspiracies. Four open models were tested across different settings.[4]

Transfer was mixed when the team tested interactions where coordination had not been explicitly instructed. The authors also caution that apparently collusive outcomes could reflect poor judgment. Their main studies do not establish that these detectors will work in networks of hundreds or thousands of agents.[4]

What this means for an ordinary AI team

Consider an illustrative business task: ask three agents to assess supplier quotations. If they silently copy one recommendation and submit three reports, agreement can look like corroboration even though only one analysis was performed. A reader counting the reports would miss the shared origin of the answer.

An openly collaborative team could instead divide the evidence gathering, record who checked each claim and challenge the final recommendation. One agent might examine delivery terms, another capacity claims and another warranty exclusions. That division could be valuable without being independent replication. The review brief should say whether the team is pooling expertise or checking the same conclusion through separate routes. Three reports, three agents and three independent checks are different things.

For such a team, we would test what happens when a peer supplies a deliberately wrong recommendation or the assigned task cannot be completed. Does the group challenge the error, report the obstacle or quietly change the goal? The success criterion should include a justified refusal to complete impossible work. Otherwise, a tidy final answer may conceal the most useful thing the system discovered: that its instructions, evidence or checking process need attention.

Report an error or suggest a correction