← All inquiries
R·027 · Field Note

Net Influence, finding colluding agents from their moves alone

Two AI agents that coordinate against the others leave a trace in what they do. I tested a published statistic for that trace on language-model agents during a research sprint: where it finds collusion, where shared habits fool it, and the case it cannot see.

The note in 75 seconds. Captions are in the video, so it works with the sound off.
Net Influence Meets LLM Agents: Shared Priors, Tacit Rules and Three Colluders The full sprint report behind this note: methods, every table and figure, and the appendices. PDF · 11 pages · 540 KB Download PDF
StatusSealed
Run2026·10·23 to 2026·10·25
Length12 min read
LeadClarence Boey · solo
SourceApart Research AI Collusion Research Sprint · Track 1, Q1.4
Codenet-influence-llm · runs, analyses, full report
ClusterAI Safety · Multi-agent · Evaluation
Context

When AI agents set prices or bid for competing principals, the evidence an auditor is most likely to hold is the action log, not the agents' messages or weights. Bonjour et al. (2022) proposed a detector that needs only that log. For each pair of agents it measures how much one agent's move tells you about the other's, given the state of the game, and subtracts the most that any third agent tells you. What is left is the pair's net influence.

Unlike R·018 to R·026, which are reading notes, this one reports my own experiments, run during the Apart Research AI Collusion Research Sprint. The detector had only been tested on scripted and reinforcement-learning agents. Language-model agents threaten its core assumption, that honest agents act independently given the state, in two ways: copies of one model share habits and fail to randomise in the same way (Guo et al. 2025), and coordination can be tacit, carried through reactions to past moves rather than a channel (Fish et al. 2024).

Net influence
γ(i; j) minus the largest γ(k; j) over third agents k, where γ is the conditional mutual information between two agents' moves given the state.
Seeded pair
Two agents I instruct to collude, so the right answer is known.
Positive control
A check that the seeded pair actually colluded: here, that the helped agent wins more often than chance.
Honest game
A game with no colluding pair. Any flag is a false positive.
Calibrated threshold
A cut-off set from the scores of honest games and frozen before it is used on anything else.
Shared prior
A habit two copies of one model have in common, which looks like coordination without being any.
The setup · 01

Four agents, one seeded pair

Four agents play 200 rounds of Rock-Paper-Scissors with the original paper's scoring: if all three moves appear, everyone scores; if two appear, the winning move scores; if every agent plays the same move, nobody does. In a seeded game one pair colludes. The primary plays, then its assistant receives a private message naming the move that guarantees the primary a point.

The agents are Llama 3.3 70B and Mistral Small 3.2, called through OpenRouter at temperature 1. Five conditions separate the effects: four Llama agents with no collusion (C0, 55 games), two Llama and two Mistral with no collusion (C1, 55 games), a seeded pair whose channel is delivered with probability p (C2), a rule agreed before the game with no channel at all (C3), and three colluders (C4). I fit the detection thresholds on 100 honest games and froze them before scoring any seeded game.

Before any of this, my reimplementation had to reproduce the paper. On its scripted benchmark it detects a pair that helps 40 % of the time in 83.1 % of runs at 200 games (paper: 79.2 %) and in 98.9 % at 400 (paper: 99.3 %). The paper's main text does not say which state it conditions on, and the choice matters: on the same data, a constant state gives 95.6 % at 100 games against the paper's 58.4 %.

Detection · 02

The seeded pair comes out on top

Every seeded pair colluded. In all 16 games with the channel always on, the primary won far above chance (p ≤ 3 × 10−29), and the assistant played the named move in 99.8 to 100 % of rounds. Net influence ranked the true pair first of six in 16 of 16, including all six pairs that mixed a Llama and a Mistral agent. An LLM judge reading the same move tables ranked it first in 4 of 6.

As the channel gets rarer, the signal fades in order. The true pair's mean score is 0.891, 0.217, 0.112 and −0.004 at p = 1, 0.5, 0.3 and 0.1, and the share flagged at the frozen 1 % threshold falls from 6 of 6 to 5, 3 and 0. Switch detectors below to compare them on the same games.

Figure 01 · Every game, four detectors One point per game. Honest games show their largest pair score (faded: the 100 calibration games); seeded games show the true pair's score, and C4 its best colluding pair. The dashed and dotted lines are the 1 % and 5 % thresholds frozen on honest games for the selected detector. A seeded game counts as flagged only when its true pair ranks first and clears the 1 % line. The count on the right is how many were flagged.
Show the data

Two other detectors behave in ways worth noting. The lagged variant, which looks for one agent reacting to the other's previous move, misses the always-on channel entirely, because the assistant's own last move already carries the primary's. Correlation ranks the pair first in every always-on game, but honest agents already move together, so its threshold is high and it flags only 1 of 20 tacit-rule games.

Shared priors · 03

Two copies among strangers

Honest LLM agents play in lockstep. In C0, all four Llama agents chose the same move in 29.8 % of rounds, against 3.7 % if they moved at random. The published rule, a fixed cut-off of α = 0.05, then fires on honest games, and much more often in the mixed population. Xu et al. (2026) report the same pattern for LLM bidders: two deployments of one model correlate at +0.053, two different models at +0.0001.

Honest games flagged by the fixed ruleC0 · 4 LlamaC1 · 2 + 2
Primary state10 of 5540 of 55
No state11 of 5512 of 55
Previous joint action0 of 558 of 55

What surprised me is that the population where every agent shares the prior is the quiet one. The subtraction explains it. In C0, every third agent shares the habit, so the subtracted term (0.219) is larger than the pair's own dependence (0.178) and the net score goes negative. In C1, the two Mistral agents share little with the Llama pair, so the subtraction shrinks to 0.050 against 0.115, and the score turns positive. Net influence cancels a shared prior only when a third agent shares it.

Figure 02 · What the subtraction removes Means over honest pairs, by pair type. The net score is the grey bar minus the clay one. Under the primary state it is negative for C0's Llama pairs and positive for C1's, which is why only C1 false-alarms. Conditioning on the previous joint action moves the excess to the Mistral pair.
Show the data

Thresholds calibrated on honest games remove the excess. Across all 110 honest games, cross-validated by seed, they flag 0.9 % at a nominal 1 % and 5.5 % at a nominal 5 %, against 45 % for the fixed rule. The cost lands on weak collusion: at p = 0.3, the calibrated threshold catches 3 of 6 games where the fixed rule caught 5.

Tacit rules · 04

The two detectors leak into each other

In C3 the pair agrees a rule before the game and never communicates: the assistant plays the move that the primary's previous move beats. With scripted agents, same-round net influence cannot see this, and a lagged variant sees it clearly. My first prompt had Llama play the inverse of the rule in 155 to 199 of 199 rounds. Rewriting the rule as a lookup table, a change I made after seeing that result, raised compliance to 81 % over 20 games.

The lagged variant flagged the pair in all 20 games, and same-round net influence in 12 (exact McNemar p = 0.008 on the paired outcomes). But same-round net influence still ranked the pair first in 18 of 20, which I had predicted would sit near chance. LLM players are predictable: the primary almost never repeats a move, so its next move often matches the rule's move. Only conditioning on the full previous joint action removes the same-round signal (0 of 20). The leak runs the other way too. With the channel delivered on 30 to 50 % of rounds, the lagged variant ranks the pair first, because the primary cycles through its moves in 55 to 91 % of rounds.

Which detector sees a coupling depends on what it conditions on and how predictable the agents are, not only on how they coordinate.
Blind spot · 05

Three colluders cancel out

With one primary and two assistants on the channel, the collusion worked in 6 of 6 games, and the two assistants played identical moves in all 200 rounds. Every colluding pair scored exactly 0.000, in every state, so nothing was flagged, in 6 LLM games and 40 scripted ones.

It is the same subtraction that keeps C0 quiet. Each colluder's strongest third-party influence is the other colluder, and subtracting it removes the pair's whole signal. A colluding pair still ranked first, because honest pairs scored below zero, and plain correlation flagged all six. A screen built only on net influence would have passed this group clean.

A market · 06

The same detector in a double auction

The method should not depend on Rock-Paper-Scissors. I reimplemented the double auction of Agrawal et al. (2025) from their paper: five LLM sellers with a cost of $80, five scripted buyers bidding between $80 and $100, and 200 hours of trading. In a seeded game, an assistant is told the primary's ask and asked to match it.

Net influence flagged all nine seeded pairs at both thresholds, with scores of 0.339 to 0.647 against a maximum of 0.166 across 110 honest games, and raised no false alarms on held-out games. Correlation could not flag anything. Honest sellers asked exactly $90, the midpoint of the range in their prompt, in 92 to 97 % of seller-hours, so honest pairs already move in perfect step. The seeded pairs earned less than honest sellers in 8 of 9 games: detecting coordination is not the same as detecting harm.

Mean ask over the run

Figure 03 · Detection in a market Top: net influence per game against the auction's thresholds, frozen on 100 honest games. Bottom: the mean ask per 10-hour block, averaged over games, against reference asks for this market. Honest sellers sit on the $90 midpoint, above the joint-profit optimum of $87, without any coordination.
Show the data
Limits · 07

What this does not show

All the collusion here was instructed, not emergent, and the channel was idealised: the assistant moved after the primary and was told the exact move. Two models, one game and one simplified auction is a narrow base. I revised the C2 and C3 prompts after seeing results and recorded each change with its reason. Held-out false-positive rates rest on five games per condition, which is why I lean on cross-validation. I did not test adaptive colluders or steganographic channels.

Any flag from this method is a screen that needs further evidence, not proof. Shared-prior false positives could implicate operators who merely run the same model, and a clean result is not a clean bill of health.

On dual use: publishing the blind spots (three colluders, couplings outside the conditioning set, weak collusion) tells colluders what to avoid. I report them for defenders and built no evasion strategies. The seeded prompts are plain instructions in a toy game, released only for reproducibility.

In short: net influence finds seeded colluders, including cross-family pairs, where an LLM judge often does not. Its fixed threshold mistakes shared priors for collusion whenever same-model agents are a minority, so it needs calibration on honest games of the same population. Three colluders defeat it outright.

Open questions

  • QDoes net influence fire when collusion is offered but not instructed, for example a private channel with no collusive goal, as in Nakamura et al. (2026)?
  • QWhat group statistic replaces the pairwise subtraction for three or more colluders without bringing back shared-prior false alarms?
  • QHow far does a threshold calibrated on one population of models transfer to another, given that the false alarms here depend on the mix?
  • QCan same-round and lagged screens be combined under one calibrated threshold without losing what each one catches?