Depth-Aware Activation Steering via Attention Residuals
Not specified in the provided content
Abstract
This paper presents a depth-aware activation steering method that enhances LLM behavior modulation by distributing persona vectors across multiple layers, leading to stronger internal alignment and behavioral shifts compared to single-layer injection.
Reality Card
The proposed method achieves a 10x increase in final alignment on held-out prompts by training depth weights on a steering objective.
Stronger internal alignment with p<0.0001 and behavioral shifts compared to single-layer injection.
The method's effectiveness may be limited by the need for careful tuning of intensity windows and the potential for random directions to degrade alignment.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.