Index version update

Artificial Analysis updated its model evaluation index on October 28, 2026, introducing changes to how models are assessed. The new version replaces public evaluations with private tests that laboratories cannot access, increasing the share of private tests to 60%.

Model scores decline

In the updated index, eleven open models listed on the open models page lost between 12 and 16 points compared to the previous version. Mistral Large 4, which is not on the open models page, lost 13 points, while Xiaomi's MiMo-V2.6-Pro and DeepSeek V4.1 Flash each lost 13 points.

Closed models maintain lead

The best closed models in version 5 of the index scored between 50 and 58 points. Claude Opus 5.5 scored 58 points, the highest, followed by GPT-6 Astra with 57 points and GPT-6.1 Sol with 55 points. These models retained their positions from the previous version.

Index composition changes

The new index includes four categories: intellectual work, code, sciences, and general knowledge, each contributing 25% to the overall score. Three tests were removed, including GDPval-AA and SciCode, while two new tests were added: FrontierCode 2.0 and Terminal-Bench-Science. These tests focus on programming and scientific tasks conducted in a terminal environment.

Contamination and performance

Artificial Analysis cites contamination as a reason for the drop in open models, where test questions might appear in training data. On Terminal-Bench-Science, GPT-6 Astra achieved 63% success, while DeepSeek V4.1 Flash managed only 13%. The exact impact of each change on the overall score remains unclear without detailed per-test results.

Published weights and gaps

Artificial Analysis highlights a growing gap between open and closed models. The best open model, Kimi K3.1 Pro, scored 46 points, down from 55 in the previous version. With published weights, the gap between DeepSeek V4.1 Flash and Claude Opus 5.5 increased from 12 to 26 points. Mistral Large 4, which is not yet published, is classified as proprietary and is not listed on the open models page.

Effort and cost

The index also measures the reasoning effort setting of each model: max, xhigh, or high. Claude Opus 5.5 scored 35 points at low effort, costing $0.98 per task, which is 3 points higher than the best published-weight model. At maximum effort, the same task costs $10.04, highlighting the trade-off between performance and cost.