Same Answer, New Box: How Much of RLVR's Out-of-Domain Gain Is Answer Format?
Author1, Author2, Author3, Author4, Author5
Abstract
The paper investigates the extent to which gains in reinforcement learning with verifiable rewards (RLVR) are attributable to answer format rather than actual answer quality.
Reality Card
82% of the reported out-of-domain multiple-choice gain is due to answer format rather than improved answers, as shown by a new extractor.
On four released 7B SimpleRL base-RL pairs, 82% of the out-of-domain gain reported by extractors disappears when using a new extractor that reads answers regardless of format.
The results depend on the specific extractor used, which may limit generalizability to other contexts or models.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.