AI agents struggle to resist teammate pressure in a 240-task collaboration benchmark

A new collaboration benchmark tests whether an AI agent will defend a good decision when a teammate presses for a worse one, share information without being asked and recover another agent's flawed work. Introduced in a November 2 preprint, the Collaborative Intelligence Benchmark contains 240 tasks across eight teamwork dimensions. Strong individual performance did not ensure strong results when a partner created a problem.[1]

The researchers compare individual and collaborative performance using task variants that introduce a teammate and then a particular difficulty. Most categories score only tasks a model could solve alone; the taking-over category uses the original task as its baseline. This helps separate collaboration problems from underlying task difficulty, although the scripted situations are narrower than the variability of a human workplace.[1]

The test asks whether an agent knows when to disagree

The eight categories included resisting unsafe pressure, reporting misconduct, sharing information, emotional intelligence, keeping a team on course, handling conflict, taking initiative and taking over another member's work. Tasks drew on records or existing benchmarks ranging from aviation incidents and clinical situations to coding and deal-making. Of the 240 tasks, 160 were made public with code and data, while 80 were held out for evaluation.[1]

Eighteen named closed- and open-weight models were evaluated, with additional high-reasoning settings for six. The highest score reported across settings was 82.2 out of 100 for a high-effort configuration of Gemini 4 Argon. That is a benchmark average across the team's eight dimensions, not a proportion of real business collaborations that are safe. At default effort, the paper reports that Gemini 4 Argon scored about 76.6 while GPT-6 Astra scored 76.5; the small ranking gap should not obscure the much larger differences between kinds of teamwork.[1]

Frontier models generally resisted unsafe shortcuts and reported misconduct, but reached the team's exact best deal in only 14% of pressured negotiation tests. The economic shortfall was usually modest: across 1,359 negotiation test runs of all 18 models, completed deals averaged 94% of the maximum joint points. The authors deliberately scored yielding to pressure more harshly than the lost value alone. In the separate taking-over comparison, frontier-model performance fell from 97% to 81% when earlier work contained misconduct.[1]

Individual reasoning was not the missing ingredient

The negotiation results distinguish two design goals. An agent may preserve nearly all of a deal's value while still allowing a forceful partner to divert it from the best shared outcome. An evaluation focused on money alone could miss that behavioural weakness; one focused only on exact-optimum success could exaggerate its immediate cost. Both measures help explain what a deploying team would need to improve.[1]

The authors report that models with almost identical general capability differed by as much as 11 points on teammate performance. Increasing reasoning effort did help some cases, but did not reliably cure the shortcomings; the maximum improvement reported across tested comparisons was 5.7 points. The paper therefore argues for evaluating collaboration directly instead of extrapolating from standalone model rankings.[1]

This matters to systems in which one agent writes code, another reviews it and a third approves deployment. Multiplying model instances is not equivalent to creating checks and balances. If the reviewer defers to the author because the author sounds confident, the division of labour adds complexity without establishing independence. If a supervisor agent inherits a flawed state and assumes it is valid, takeover may preserve errors that a fresh start would have exposed.[1]

The benchmark offers a way to test collaboration before building a larger agent team: introduce a wrong assertion, withheld information or flawed inherited work and inspect the response. Teams can then judge whether the agent challenges the problem appropriately and how much the failure actually changes the outcome.[1]

Report an error or suggest a correction