Papers/2611.21641
🧪 Test?View on arXiv

RAW: Refusal Aversion in the Wild, A Causal Measurement Method for Deployed LLMs

Not provided

user engagementLLM evaluationcausal inference
2611.21641
Builder Relevance
80%
15h ago

Abstract

The paper introduces RAW, a method to measure the user-side cost of refusal in deployed LLMs, revealing significant impacts on user re-engagement.

Reality Card

Core Claim

The RAW method allows for the causal measurement of user disengagement due to refusal in deployed LLMs, costing 3 to 17 percentage points of subsequent re-engagement for about half of the models tested.

Method / Result

The method runs on any deployed model from logs alone for about $0.30 per 1,000 conversations.

Limitations

The method's applicability is limited to existing logs and may not be generalizable across all types of LLMs or user interactions.

Paper to code

Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.

No verified implementation link has been attached yet. AIBuzzHub will keep this panel separate from unverified search results.
← Back to all papers