Aider PolyglotBenchmark scores and sources

Published result; benchmark version and evaluation conditions remain in the id and result note.

pillar: coding · weight 1 within pillar · unit: % (higher is better) · official board ↗

Row	Model	Result	Normalized (0–100)	Confidence
1	o3-pro OpenAI	84.9%	84.9	22% confidence 22 percent, Low
2	GPT-5 OpenAI	81.3%	81.3	100% confidence 100 percent, Full
3	o3 OpenAI	76.9%	76.9	87% confidence 87 percent, High
4	o4-mini OpenAI	72%	72.0	87% confidence 87 percent, High
5	DeepSeek-R1 DeepSeek	71.4%	71.4	88% confidence 88 percent, High
6	Claude Opus 4 Anthropic	70.7%	70.7	100% confidence 100 percent, Full
7	Claude Sonnet 3.7 Anthropic	60.4%	60.4	100% confidence 100 percent, Full
8	Claude Sonnet 4 Anthropic	56.4%	56.4	100% confidence 100 percent, Full
9	DeepSeek V3 0324 DeepSeek	55.1%	55.1	74% confidence 74 percent, Medium
10	o3-mini OpenAI	53.8%	53.8	96% confidence 96 percent, High
11	GPT-4.1 OpenAI	52.4%	52.4	100% confidence 100 percent, Full
12	Claude Sonnet 3.5 v2 Anthropic	51.6%	51.6	100% confidence 100 percent, Full
13	GPT OSS 120B OpenAI	41.8%	41.8	79% confidence 79 percent, Medium
14	Qwen3 32B Alibaba / Qwen	40%	40.0	67% confidence 67 percent, Medium
15	GPT-4.1 mini OpenAI	32.4%	32.4	87% confidence 87 percent, High
16	Claude Haiku 3.5 Anthropic	28%	28.0	100% confidence 100 percent, Full
17	GPT-4o (2024-08-06) OpenAI	23.1%	23.1	100% confidence 100 percent, Full
18	QwQ 32B Alibaba / Qwen	20.9%	20.9	67% confidence 67 percent, Medium
19	GPT-4o (2024-11-20) OpenAI	18.2%	18.2	61% confidence 61 percent, Medium
20	Qwen2.5-Coder-32B-Instruct Alibaba / Qwen	16.4%	16.4	90% confidence 90 percent, High
21	Llama 4 Maverick 17B Instruct Meta	15.6%	15.6	78% confidence 78 percent, Medium
22	GPT-4.1 nano OpenAI	8.9%	8.9	82% confidence 82 percent, High
23	Gemma 3 27B IT Google	4.9%	4.9	74% confidence 74 percent, Medium
24	GPT-4o mini OpenAI	3.6%	3.6	100% confidence 100 percent, Full

Results are as published by the source behind each value (hover or tap the number). Benchmark scores are shown individually for every source/variant (a model may have multiple rows), and vary by version, harness and date; the normalized column uses the method’s fixed 0–100 scales and feeds the coding pillar of the SI Score.

Aider PolyglotBenchmark scores and sources

What changed

Alerts on this device