Beyond Right and Wrong: Evaluating Second-order Social Reasoning in Large Language Models
Not provided in the abstract
Abstract
This paper introduces a framework for evaluating second-order social reasoning in LLMs, highlighting the disparity between AI predictions and human expectations regarding social norm enforcement.
Reality Card
The study reveals that current LLMs overpredict negative sanctions in social norm violations compared to human judgments, particularly as social distance increases.
The release of the NormReact dataset, which includes 450 norm violation scenarios annotated for emotions and behavioral responses.
The findings suggest that LLMs may misrepresent social regulation, indicating a need for further evaluation in norm-sensitive applications.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.