AIBuzzHub
Signal Brief
Home/Events/CWE-bench v2: More coverage, quality, and transparency

CWE-bench v2: More coverage, quality, and transparency

Confirmed
Confidence
90%
Impact: 80%
Updated 2m ago

Consensus Brief

CWE-bench v2 introduces a public set of 100 tasks designed to evaluate models on their ability to identify and fix security vulnerabilities. The tasks cover 91 different CWEs and are intended to improve the quality and transparency of model evaluations in cybersecurity. The results are reproducible and tracked on leaderboards for both open-weight and API-only models.

Sourced from
Hugging FaceT1Hugging FaceT1Hugging FaceT1Hugging FaceT1+8 more
Primary: Hugging Face

What Changed Since Last Update

2m ago

New official source added: Hugging Face published an update on Tue, 03 No ("CWE-bench v2: More coverage, quality, and transparency").

Claim Ledger

4 claims tracked across sources

Confirmed Fact

CWE-bench v2 opens up a public set of 100 tasks that anyone can run and evaluate their models on.

Confirmed Fact

Gemini 4 Argon leads with 67.8% pass@1 and 81.0% pass@4.

Confirmed Fact

95 tasks are solved by at least one model, indicating that the tasks are solvable.

Confirmed Fact

DeepSeek V4.1 Flash, Claude Opus 5.5, and Gemini 4 Argon form the Pareto frontier in terms of cost and performance.

Role-Based Impact Analysis

Source Timeline

9 sources corroborating

T1
CWE-bench v2: More coverage, quality, and transparency
Hugging Face·19h ago
T1
CWE-bench v2: More coverage, quality, and transparency
Hugging Face·19h ago
T1
CWE-bench v2: More coverage, quality, and transparency
Hugging Face·19h ago
T1
CWE-bench v2: More coverage, quality, and transparency
Hugging Face·19h ago
CWE-bench v2: More coverage, quality, and transparency
Hugging Face·19h ago
T1
CWE-bench v2: More coverage, quality, and transparency
Hugging Face·19h ago
T1
CWE-bench v2: More coverage, quality, and transparency
Hugging Face·19h ago
T1
CWE-bench v2: More coverage, quality, and transparency
Hugging Face·19h ago
T1
CWE-bench v2: More coverage, quality, and transparency
Hugging Face·19h ago