🧪 Test?View on arXiv
Does Marginal Coverage Guarantee Class-Conditional Safety for Zero-Shot VLMs Under Shift?
Author1, Author2, Author3, Author4, Author5
multimodalconformal predictionzero-shot learningmodel evaluation
2608.19376
Builder Relevance
2h ago70%
Abstract
The paper investigates the reliability of marginal coverage in zero-shot vision-language models under deployment shift, revealing significant discrepancies in class-conditional safety.
Reality Card
Core Claim
Marginal coverage should be viewed as an average reliability statistic rather than a safety guarantee for class tails in zero-shot vision-language models.
Method / Result
On ImageNet-Sketch, worst-class coverage falls to approximately 0%, despite marginal coverage being around 0.86.
Limitations
The need for labels for every class in target-side class calibration poses a significant limitation for practical deployment.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.
No verified implementation link has been attached yet. AIBuzzHub will keep this panel separate from unverified search results.