Benchmarks Don't Talk Back: Learning Through Self-Disagreement
Not provided
Abstract
The paper introduces the concept of dialectical competence in language models, emphasizing the importance of knowing when to hold or yield a position in response to counter-reasoning.
Reality Card
The authors demonstrate that language models can be trained to improve their dialectical competence through self-generated opposition, enhancing their ability to contest answers without external models or human annotations.
The proposed method allows models to learn through self-disagreement, achieving reinforcement learning from verifiable rewards without requiring preference data or human annotation.
The method's effectiveness may vary based on the initial model's capabilities and the complexity of the generated antitheses.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.