Where Global Context and Weight Sharing Work in Compact Vision Backbones
Not provided in the abstract
Abstract
This paper investigates the effectiveness of global self-attention and weight sharing in enhancing the performance of compact convolutional backbones like EfficientNet.
Reality Card
Inserting lightweight zero-gated residual self-attention adapters into the MBConv trunk improves accuracy to 74.02%, outperforming a plain B0 control by +0.954 percentage points.
Achieved 74.02% accuracy with self-attention adapters, a +10.82 points improvement in a single epoch with weight sharing.
The study's reliance on specific configurations and the potential degradation of accuracy with post-hoc kernel averaging may limit reproducibility.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.