RL never reads the reasoning it rewards. Agon makes a rival model the grader: both solve the same problem, read each other's reasoning, and you score by solving what your rival couldn't. ~2x GRPO's accuracy at half the trace length.
two traces with the same answer get the same reward, no matter how they reasoned. every restart and "let me reconsider" is another lottery ticket, so padding gets reinforced through any lucky rollout that contains it. no label can fix this. a rival can.
a drafter solves from scratch. a challenger reads the drafter's solution summary, final answer hidden, and wins by being correct where the rival failed. roles rotate every step. your grader is also your competitor, and it's learning too.
show the challenger only the rival's answer and the gain collapses. what helps is reading the reasoning, not copying the result.
same two models, three rungs: let them talk, train them to help each other, train them to beat each other. exchange is the lever, competition compounds it.
| method | pass@1 | final trace |
|---|---|---|
| zero-shot | 23 | 6.1k |
| vanilla GRPO | 30 | 8.1k |
| just let them talk (MoA) | 34 | 6.9k |
| trained to help each other | 46 | 5.1k |
| trained to beat each other (Agon) | 61 | 3.5k |
Qwen3-0.6B, DeepMath hard split, matched budget. the 0.6B pair beats GRPO-trained 4B, a model 7x larger. no length penalty anywhere: short traces are what winning looks like.
we started with text. next is reasoning inside latent space.
full paper →