AdaPrefix-GRPO

the dead zone

take the hardest slice of any dataset and the model stops landing rollouts: zero correct, zero gradient, zero learning. and these are exactly the problems with the most to teach. on our split, GRPO throws away two thirds of the data:

the dial

prepend a slice of a correct solution and let the model finish. solve rate becomes a smooth function of slice length: difficulty stops being a property of the problem and becomes a knob.

the controller

the right slice length is a moving target: as the model learns, yesterday's prefix becomes too easy and the problem drifts out of the sweet spot. so we run a thermostat: measure the solve rate as training moves, steer every problem back to 0.5, where GRPO's signal peaks, and anneal the prefixes to zero, so the trained model never needs the crutch.

results

Qwen3-1.7B, DeepMath hard split, matched compute. evaluation always without a prefix.

a 0.6B on this recipe beats a 3x bigger model on vanilla GRPO: 41.8 vs 30.1.

same answers in half the tokens: density is trainable. full paper →