Tracing Knowledge and Reasoning in LLM
Not provided in the content
Abstract
The paper proposes a task-centric framework to analyze the internal processing of Large Language Models (LLMs) across various benchmarks, revealing insights into their knowledge and reasoning capabilities.
Reality Card
The study shifts evaluation from mere performance scores to characterizing the internal processing patterns of LLMs, enabling better benchmark selection and model choice.
Evaluated 16 LLMs across 24 diverse benchmarks, revealing systematic differences in processing between knowledge-centric and reasoning-centric tasks.
The reproducibility of the results may be limited by the specific benchmarks and models chosen for analysis.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.