🧪 Test?View on arXiv
Qistas: A Pre-Registered, Portable Protocol for Fairness Evaluation of Large Language Models
fairnessevaluationlanguage modelsprotocols
2610.40893
Builder Relevance
1h ago70%
Abstract
The paper presents Qistas, a protocol for evaluating the fairness of large language models across different social contexts, particularly focusing on Iraq.
Reality Card
Core Claim
Qistas provides a portable and pre-registered protocol for evaluating fairness in LLMs, addressing the limitations of existing benchmarks that are predominantly English-centric.
Method / Result
In its first field test, 24 out of 25 template groups failed at least one fairness check.
Limitations
The protocol is still in pilot form and has not yet evaluated any models, which may limit immediate applicability.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.
No verified implementation link has been attached yet. AIBuzzHub will keep this panel separate from unverified search results.