Simulation and Wargaming with AI Agents

(a) Exact values: the equilibrium has guarantee $0$ and exploitability $0$; the fictitious-play average at $T=10^3$ has exploitability $\frac{1}{25}$ (its pair gap, recomputed, is Chapter 8's $\frac{2}{25}$); the wrong-model equilibrium has exploitability $\frac{2}{7}$, all of it through column P; always-P has guarantee $-3$ and exploitability $3$, yet earns $+2$ per round against a partner who always plays column R. (b) For every agent $q$ and every budget $T$, the tester's expected reported gain is at most the exploitability of $q$: conditional on the counts, its expected gain that round is ${\mathrm{gain}}_q(\mathrm{BR}(\text{counts}))\le {\max }_j{\mathrm{gain}}_q(j)$, which is the exploitability. The bound is computed exactly: against always-P the expected reports at budgets $1,5,25,100$ are $-2,2,\frac{14}{5},\frac{59}{20}$ (truth $3$); against the wrong-model equilibrium they are $0$, then $\approx 0.097,0.132,0.185$ as exact rationals (truth $\frac{2}{7}\approx 0.286$); against the equilibrium every expected report is exactly $0$ (every column earns $0$ in expectation; a single realised round does not).

The per-round score takes the values $0$, $+1$, $-1$, $+2$, $-2$, $+3$ and $-3$ with exact probabilities $\frac{14}{36},\frac{6}{36},\frac{6}{36},\frac{3}{36},\frac{3}{36},\frac{2}{36},\frac{2}{36}$, in that order; its mean is $0$. Over $N=25$ rounds, exactly, $P(S_{25}>0)=0.4717\dots$, $P(S_{25}=0)=0.0564\dots$ and $P(S_{25}\ge 13)=0.0384\dots$, the last being a ‘half-point-per-round winner'. The best of five replications, for a side fixed in advance, shows such a winner with probability $0.1779\dots$; the most striking run of either side does so with probability $0.3296\dots$ At $N=16$, $36$, $64$ and $100$ the probability that the realised mean is at least $\frac{1}{4}$ in absolute value is $0.5354\dots$, $0.3161\dots$, $0.1705\dots$ and $0.0831\dots$ All values are exact rationals; the ellipsis decimals show their leading digits.

On the observed game the cognitive-hierarchy population induces the action distribution $(\frac{7099}{13755},\frac{1024}{13755},\frac{5632}{13755})$ over (R, P, S). (a) The level-$k$ populations over depths $\mathrm{0..4}$ calibrated to this aggregate form a two-parameter family (the depth-$0$ share and the split between depths $1$ and $4$, which act identically on every game of this cycle type); members

over depths $\mathrm{0..4}$ both reproduce it exactly. (b) On the counterfactual RPS with weights $(3,2,1)$, members A and B and the cognitive-hierarchy population all predict the same distribution $(\frac{5632}{13755},\frac{7099}{13755},\frac{1024}{13755})$: this counterfactual cannot distinguish them. Preserving the best-reply cycle does not guarantee that: on weights $(1,1,6)$, with the observed game's cycle and level-$k$ ladder, A and B still predict the observed aggregate while the cognitive-hierarchy population predicts $(\frac{3643}{13755},\frac{1024}{13755},\frac{9088}{13755})$. (c) On the beauty contest they separate: A names $50,33,22$ with weights $\frac{3072}{13755},\frac{4608}{13755},\frac{6075}{13755}$; B names $33,22,15$ with weights $\frac{5632}{13755},\frac{7099}{13755},\frac{1024}{13755}$; the cognitive-hierarchy ladder names $50,33,27,24,23,22,22$ with the truncated-Poisson weights. The pairwise total-variation distances are $\frac{1024}{4585}$ (A, B), $\frac{1944}{4585}$ (A, CH) and $\frac{424}{655}$ (B, CH), and the predicted means are $\frac{146438}{4585}\approx 31.9$, $\frac{357394}{13755}\approx 26.0$ and $\frac{21938}{655}\approx 33.5$.

(a) The null is Model W2's. Between the equilibrium players of Model W2 a round is won by the row player with probability $\frac{11}{36}$, lost with probability $\frac{11}{36}$ and drawn with probability $\frac{14}{36}$; conditional on a decisive round, each side wins with probability exactly $\frac{1}{2}$. Under this null an agent shows a win rate of at least $70\%$ in $10$ negotiations with probability exactly $\frac{11}{64}=0.171875$, in $20$ with probability $0.0576\dots$, in $50$ with probability $0.0033\dots$; the best of five such evaluations shows it with probability $0.6105\dots$, $0.2569\dots$ and $0.0163\dots$ (b) Sample size. The smallest $n$ for which some threshold rule has size at most $\frac{1}{20}$ and power at least $\frac{4}{5}$ is $37$ (threshold $24$, size $0.0494\dots$, power $0.8070\dots$). No threshold rule meets these targets at $n=38$, $39$ or $41$; every $n$ from $42$ to $400$ admits one. For size $\frac{1}{20}$ and power $\frac{9}{10}$ the smallest $n$ is $53$ (threshold $33$), with failures at $54$, $55$, $57$ and success at every $n$ from $58$ to $400$; for size $\frac{1}{100}$ and power $\frac{9}{10}$ it is $80$ (threshold $51$), with failures at $81$ and $84$ and success at every $n$ from $85$ to $400$. (c) Selection. The expected win rate of the best of $k$ evenly matched configurations, each evaluated $m$ times, is

which equals $0.6811\dots$ at $(m,k)=(10,5)$, $0.7381\dots$ at $(10,10)$, $0.7058\dots$ at $(20,20)$ and $0.6313\dots$ at $(50,20)$; the probability that the best of $k$ shows at least $7$ wins in $10$ is $0.171875$, $0.6105\dots$ and $0.8483\dots$ for $k=1,5,10$. Applied to the best of five evenly matched configurations, the $n=37$ rule of (b) has size $0.2239\dots$ instead of $0.0494\dots$ (d) A blind search is a lower bound. The blind red team's expected report is $g(1-(1-f{)}^r)<g$ for every finite $r$. The smallest $r$ that finds an exploiting deviation with probability at least $\frac{19}{20}$ is $29$, $299$ and $2995$ for $f=\frac{1}{10},\frac{1}{100},\frac{1}{1000}$; with $f=\frac{1}{100}$, a hundred probes find nothing with probability $0.3660\dots$