Splitting decisive evidence among AI agents reduced average success by 22.9 percentage points in TeamGym, an arXiv benchmark spanning legal, finance, medical and software-engineering tasks. The comparison tested eight language models using 12 source corpora, with the same pivotal facts either supplied together or distributed across a team.[1]
Across eight language models and 12 source corpora, the researchers identified “pivotal information”: a minimal set of facts in which each element is necessary and the set is collectively sufficient for the correct decision. When the same pivotal facts were distributed among team members rather than supplied to one model, average success fell by 22.9 percentage points. When agents also had to retrieve the facts from longer source material, success fell another 19.9 points.[1]
TeamGym separates two different coordination failures
| Condition | Information available | Average effect |
|---|---|---|
| Single model | All pivotal facts provided together | Reference conditionSource 1 |
| Distributed team | Pivotal facts split among participants | Success lower by 22.9 percentage points: the communication gapSource 1 |
| Distributed plus retrieval | Each participant must find pivotal facts in source documents | A further 19.9-point loss: the retrieval gapSource 1 |
| Policy training | Reward based only on final team decision | Improved decisions and induced coordination in participation and speaker selectionSource 1 |
The benchmark controls what the team must know
Most multi-agent demonstrations begin with a harness and ask whether the assembled system solves a task. TeamGym starts one level lower. It constructs decision problems for which the benchmark can track the small set of evidence that must be integrated. That permits a diagnosis: did a participant fail to retrieve a key fact, fail to tell the group, or did the coordinator fail to use it? Without that structure, a wrong final answer says little about where collaboration broke.[1]
This distinction matters because adding agents can increase the amount of text while reducing the density of useful information. A participant may repeat shared facts, omit a private one or produce a confident recommendation before the evidence exchange is complete. A coordinator can then summarize the conversation faithfully and still miss the answer because the pivotal fact never entered the shared record.
Retrieval and communication compound
The communication gap was measured while participants received the pivotal information directly. That isolates the cost of distribution. The additional retrieval condition is closer to professional work: agents receive source material and must identify the relevant passages themselves. The second loss shows that a team can fail before communication starts. Agents may retrieve plausible context without recognizing the fact that changes the decision.[1]
The corpora span domains where evidence is distributed naturally. A legal team may hold separate facts about precedent and procedure; a medical team may divide history, tests and treatment constraints; a software team may split logs, code and requirements. TeamGym does not recreate every institution, but it gives researchers a controlled way to ask whether more participants improve coverage or merely multiply opportunities for omission.[1]
A final-answer reward changed group behaviour
The researchers also trained participant and coordinator policies using reinforcement learning. The reward evaluated the final decision, not a hand-written script for good teamwork. Yet training improved decision quality and produced emergent changes in who participated, who spoke next and what was communicated. This suggests that coordination can be learned from outcome feedback when the environment exposes the consequences of missing information.[1]
That finding should not be read as proof that a single reward solves organisational behaviour. The benchmark has known answers, bounded roles and controlled communication. Real teams face ambiguous objectives, strategic incentives, privacy restrictions and facts whose reliability is contested. A policy optimized for benchmark accuracy may also learn brittle conventions that fail when the number of agents or the information structure changes.
A useful multi-agent evaluation should track which necessary evidence reached the agent making the final decision. TeamGym’s comparison separates having the evidence somewhere in the team from using it in the answer. That distinction gives developers a more specific target than simply adding more agents or longer conversations.[1]
Sources & further reading
- TeamGym: A Gym for Team Decision-Making by AI Agents with Distributed Information arXiv · 27 Oct 2026 · Accessed 30 Oct 2026