The Silent Failures in Agentic Data Science
Author1, Author2, Author3, Author4, Author5
Abstract
This paper critiques existing evaluations of agentic data science, highlighting the need for a principled failure mode analysis to develop trustworthy agentic data science pipelines.
Reality Card
The paper demonstrates that existing evaluations of agentic data science overlook critical methodological failures, emphasizing the need for a comprehensive analysis of failure modes.
The analysis identified that the majority of agentic solutions contain at least one failure mode across 220 data science tasks.
The evaluations primarily focus on predictive performance, which may mask underlying methodological issues.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.