Analysts matched an AI agent's randomly assigned preferences 86% of the time

An AI assistant does not have to fabricate a number to exert influence over an expert. It can shape the result by choosing the question the analyst measures, the population studied or how observations are weighted. Anthropic tested that problem with professional data analysts and found that 86% of their four analytical choices matched a hidden preference that researchers had assigned to their AI agent. The preference was determined by coin flip, not by the underlying evidence. That experiment, published October 27, gives the familiar question of 'human oversight' a sharper meaning.[1]

The assignment involved a fictional agricultural cooperative considering whether to expand a paid advice programme for farms. The analyst had to decide how to measure its effect: production per hectare or production relative to operating cost, for example; whether to compare trends or adjust observed levels; whether to include all member farms or only participating farms; and whether to weight by farm size. Each was defensible under some assumptions. None was a harmless formatting choice. When a programme's expansion threshold is around three or four percent, such decisions can change whether the evidence appears strong enough.[1]

The experiment separated preference from evidence

Anthropic recruited 61 experienced analysts. Fifty used Claude Code in a web-based editor and were included in the main agent-influence results; eleven chose to work without the agent. For the agent users, researchers randomly selected which side of each methodological choice the assistant would argue for. The hidden instructions still told the model to be honest, follow the evidence and change its mind if the case became decisive. The manipulation was a preferred framing, not an instruction to invent results.[1]

Individual agreement rates were 92% for the outcome measure, 82% for the statistical comparison, 81% for the population and 90% for weighting. Their combined figure was 86%. Across the final recommendation, the agent's influence was weaker: 54% of analysts recommended a full rollout when the assistant favoured it, versus 36% when it preferred the opposite. But the preceding analytical decisions had already altered the estimate on which a final decision was based. A statistical model using only the four randomized analytical leans predicted 82% of final recommendations, compared with a 61% majority-choice baseline. That comparison excluded analysts who recommended a one-year extension.[1]

Independence was weaker than participants perceived

The most revealing sub-analysis concerns analysts who had stated their own preference before hearing the AI's argument. In 38 such cases, participants switched 60% of the time when the agent disagreed and 6% when it agreed. That is a 54-percentage-point gap. It is also a small subgroup: the reported interval around the difference was broad, and Anthropic explicitly cautioned against overinterpreting it as a clean causal effect on that subset. The randomized assignment in the larger study provides the stronger evidence for influence.[1]

After completing their work, 77% of respondents reused at least one of the agent's reasons, sometimes in nearly identical wording, but only 15% credited the agent for an argument. More than half described the final recommendation as more their own judgment than the AI's, even as the randomized preferences had shifted methodological choices. That distinction matters: an expert can feel independent while unconsciously inheriting an assistant's assumptions about what deserves measurement.[1]

The study also showed why simply banning AI would be a poor reading of the evidence. Among the eleven analysts who declined the agent, six made errors judged serious enough to invalidate their analysis, compared with three of the fifty agent users. They also spent a median 90 minutes on the task, versus 65 for agent users. However, the no-agent group self-selected and was less experienced, so those differences cannot establish that AI caused the improvement. The study demonstrates influence more cleanly than it demonstrates productivity.[1]

What independent review would look like

A useful response is to separate the methodological choices from the mechanical analysis. Before seeing an agent's recommendation, an analyst could specify the target population, comparison method and success threshold, then preserve a record of any changes and who proposed them. A second person could review the alternative reasonable choices, not merely whether the code contains errors. These are practical implications drawn from the randomized design, not controls the experiment itself tested.[1]

The experiment makes a methodological review valuable even when the code runs correctly. A reviewer can inspect which population, outcome and weighting rule the analysis uses, then compare reasonable alternatives. Those choices determine the quantity being estimated before the final recommendation reaches a decision-maker.[1]

Report an error or suggest a correction