Trend · AI news

Is your AI making your decisions? What Anthropic's 86% study found

Updated Oct 28, 2026 · By Mishal Brands Editorial

Checked Oct 28, 2026. Every number below is from Anthropic's own write-up of the study, published Oct 27, 2026 and linked at the bottom. We have not run the study.

The short answer

In Anthropic's experiment, 50 professional analysts followed their AI agent's opinion 86% of the time, even though that opinion had been set by a coin flip and never disclosed. With no influence you would expect about 50%. Anthropic published the results on Oct 27, 2026. The analysts were experts in causal inference, working with Claude Code on a realistic advisory task. When an analyst had already picked an option, they switched away from it 60% of the time if the agent disagreed, and 6% of the time if it agreed. Anthropic is careful about what this means: the leans were deliberately strengthened for the test, it was one task in one field on one model, and the study is not an argument against using AI.

How the experiment worked

The task. Anthropic recruited 61 analysts from a screened pool of people who know causal inference, meaning how to work out from data whether one thing causes another. Eleven chose not to use the agent and are left out of most of the analysis, leaving 50.
The setup. Each analyst advised a fictional farming cooperative on whether to expand a paid advisory program. The board would expand only if the program raised productivity by roughly 3 to 4%, but it never said how productivity should be measured.
The hidden lean. Four analytical decisions were left open, each with two reasonable answers: how to measure productivity, which comparison method to use, which farms to include, and whether to weight large producers more. For each decision, a coin flip decided which option the analyst's agent would favor. The analyst was not told, and the agent was instructed to argue its side honestly, change its mind if the evidence turned, and never mislead.
What was measured. Which option each analyst chose, and what they said afterwards about why.

The numbers

What Anthropic measured	Result
Analysts whose choice matched their agent's lean, across all four decisions	86%, against about 50% expected with no influence
Same, by decision	92% on the outcome measure, 90% on weighting, 82% on the comparison method, 81% on which farms to include
Analysts who had already stated a preference and then switched when the agent disagreed	60%, versus 6% when the agent agreed. Based on only 38 cases, so the estimate is imprecise
Analysts who reused the agent's argument when explaining their choices	77%, sometimes word for word
Analysts who gave the agent credit as the source	15%
Analysts who called the final recommendation more their own than the agent's	52%, even where the agent's lean had moved their choices
Serious methodological errors	6% of the 50 who used the agent (3 people), against 55% of the 11 who worked alone (6 people)
Median time on the task	65 minutes with the agent, 90 minutes without

What it does not prove

Anthropic lists the limits itself, and they matter:

The leans were made strong on purpose. The instructions told the agent to hold and voice its preference so the effect could be measured. When an agent has no clear preference and does not volunteer one, the effect may be smaller. Anthropic notes that with no lean assigned, Claude Opus 4.8 made the same five choices in 112 of 123 runs.
It was one task, one field, one model. The task took about an hour and was about farming. The analysts were experts in the method but new to the subject. In real work they would know the field better and hold stronger views.
The numbers are averages. Anthropic did not record each analyst's own view before the agent spoke, so it cannot say how much any one person was influenced.
The two groups are not comparable. The 11 analysts who skipped the agent were less experienced on average, so the error and speed figures cannot be read as the effect of using AI.

What to take from it

Anthropic's point is about judgment, not accuracy. Checking AI work for errors assumes there is a right answer. Where a decision is a judgment call, the work can look sound whichever way it goes, so a mistake is invisible. The study suggests the influence is hard to notice from the inside: most analysts still felt the recommendation was theirs.

Anthropic's own suggestions for what to change:

Make the agent argue both sides. In the study the agent argued only its own lean, and the analysts rarely heard a strong case for the other option. An agent asked for the best case on each side gives you something to weigh.
Say your view first, and check it later. The analysts who had already picked an option still switched 60% of the time when the agent disagreed, so a stated preference is not a shield. Looking back at what you wrote before the answer arrived is.
Decide in advance which calls are yours. Anthropic argues organizations will need to name the decisions that need a person's own informed judgment, and give people time to form one, rather than drifting into whatever the agent picks.

Anthropic is explicit that this is not a reason to use AI less. In the study, the analysts with the agent finished faster and made far fewer serious errors.

Related on Mishal Brands

Sources (checked Oct 28, 2026)

Anthropic, "How AI agents shape experts' judgment calls" (Oct 27, 2026): https://www.anthropic.com/research/agents-judgment-calls
Al Jazeera, "Experts defer to AI in decision making, Anthropic study finds" (Oct 28, 2026): https://www.aljazeera.com/economy/2026/10/28/experts-defer-to-ai-in-decision-making-anthropic-study-finds

Disclosure: this page has no affiliate links. No company paid for or reviewed this page. See our affiliate disclosure and how we review.