🧪 Test?View on arXiv
Star Light, Star Bright: Can Multimodal Models Parse an Astronomy Visual Question Answering Dataset Right?
J. Naiman, A. Author, B. Author, C. Author, D. Author
multimodalvisual question answeringastronomydataset creation
2610.22474
Builder Relevance
1h ago70%
Abstract
The paper investigates the performance of low-cost large multimodal models on a domain-specific visual question answering dataset for astronomical images.
Reality Card
Core Claim
The study reveals that low-cost LMMs frequently fail to accurately answer domain-specific questions related to astronomical images.
Method / Result
The dataset comprises 2001 figures and 47357 questions, with models showing high error rates in answering domain-specific queries.
Limitations
The models tested showed similar median errors to human-verified figures, indicating potential limitations in their accuracy and reliability.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.
No verified implementation link has been attached yet. AIBuzzHub will keep this panel separate from unverified search results.