🧪 Test?View on arXiv
Where Grokking Happens: Distributed Utility and Fourier Recoding Without a Module Switch
Not specified in the provided content
Transformergeneralizationactivation patternsgrokking
2609.17571
Builder Relevance
1h ago70%
Abstract
The paper investigates the transition from memorization to generalization in Transformers, introducing Transition Games to analyze activation patterns.
Reality Card
Core Claim
The study identifies that grokking occurs as spectral recoding of an existing distributed circuit rather than through a module switch.
Method / Result
Selected degree-two modes account for 67–92% of addition contrast across replacement games.
Limitations
The findings may be limited by the specific conditions of the Transition Games and the reliance on exact activation patterns.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.
No verified implementation link has been attached yet. AIBuzzHub will keep this panel separate from unverified search results.