Objective-Conditioned Active Preference Learning for Multi-Objective Alignment
Not provided in the abstract
Abstract
This paper presents a novel active learning framework for aligning large language models with multiple objectives, reducing annotation costs while effectively utilizing informative conflicts.
Reality Card
The OGC-DPO framework effectively reduces annotation costs for multi-objective alignment by selecting the most informative queries based on the model's implicit gradient.
Empirical experiments demonstrate OGC-DPO's effectiveness over existing methods under the same annotation cost.
The paper does not provide specific details on the reproducibility of the empirical results or the generalizability of the method across different models.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.