CrisisBench: an evaluation suite for crisis counseling AI
Not specified in the provided content
Abstract
CrisisBench is a privacy-preserving evaluation suite for crisis counseling AI that addresses the limitations of existing benchmarks in capturing essential aspects of crisis conversations.
Reality Card
CrisisBench enables scalable, privacy-preserving evaluation of crisis counseling AI, revealing that current systems perform strongly on structural competencies but diverge from expert counselors on response-level failures.
In multi-turn evaluation, retrieval-augmented generation scored highest overall with a score of 0.961.
The evaluation systems underperformed human counselors on excessive empathy and questioning, indicating a gap in response-level performance.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.