SWE-bench VerifiedBenchmark scores and sources
Published result; benchmark version and evaluation conditions remain in the id and result note.
| Row | Model | Result | Normalized (0–100) | Confidence |
|---|---|---|---|---|
| 1 | Claude Opus 5 Anthropic | 96%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jul 24, 2026
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 96.0 | 100% confidence 100 percent, Full |
| 2 | Claude Mythos 5 Anthropic | 95.5%Anthropic Mythos 5 system-card claimsLab claim, not an independent evaluation. Anthropic internal harness, thinking enabled; average over 5 trials; June 9 release card, section 8; immutable edition [variant] Mythos 5 system cardPublished Jun 9, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 95.5 | 5% confidence 5 percent, Low |
| 3 | Claude Fable 5 Anthropic | 95%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 95.0 | 100% confidence 100 percent, Full |
| 4 | Claude Opus 4.8 Anthropic | 88.6%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 88.6 | 100% confidence 100 percent, Full |
| 5 | Claude Sonnet 5 Anthropic | 85.2%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jun 30, 2026
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 85.2 | 93% confidence 93 percent, High |
| 6 | Claude Opus 4.7 Anthropic | 83.5%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] maxPublished Apr 20, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 83.5 | 100% confidence 100 percent, Full |
| 7 | Ornith 1.0 397B DeepReinforce | 82.4%Official model cards via models.devLab-reported; metric percent resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 82.4 | 100% confidence 100 percent, Full |
| 8 | DeepSeek V4 Pro DeepSeek | 80.6%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 80.6 | 85% confidence 85 percent, High |
| 9 | MiniMax-M3 MiniMax | 80.5%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Claude CodePublished Jun 1, 2026
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 80.5 | 100% confidence 100 percent, Full |
| 10 | Qwen3.7 Max Alibaba / Qwen | 80.4%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published May 19, 2026
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 80.4 | 53% confidence 53 percent, Medium |
| 11 | Kimi K2.6 Moonshot AI | 80.2%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 80.2 | 85% confidence 85 percent, High |
| 12 | MiniMax-M2.7 MiniMax | 79.9%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Claude CodePublished Jun 1, 2026
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 79.9 | 88% confidence 88 percent, High |
| 13 | Gemini 3.5 Flash Google | 79.3%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] highPublished Jun 1, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 79.3 | 100% confidence 100 percent, Full |
| 14 | Claude Opus 4.5 (latest) Anthropic | 79.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] live-SWE-agent; mediumPublished Dec 15, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 79.2 | 14% confidence 14 percent, Low |
| 15 | Claude Opus 4.5 (latest) Anthropic | 79.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Sonar Foundation AgentPublished Dec 5, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 79.2 | 14% confidence 14 percent, Low |
| 16 | DeepSeek V4 Flash DeepSeek | 79%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 79.0 | 53% confidence 53 percent, Medium |
| 17 | MiMo-V2.5-Pro Xiaomi | 78.9%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 78.9 | 88% confidence 88 percent, High |
| 18 | Claude Opus 4.6 Anthropic | 78.7%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 18, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 78.7 | 100% confidence 100 percent, Full |
| 19 | GLM-5.2 Z.ai | 78.7%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] maxPublished Jun 25, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 78.7 | 100% confidence 100 percent, Full |
| 20 | Hy3 Tencent | 78%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 78.0 | 88% confidence 88 percent, High |
| 21 | DeepSeek V4 Pro DeepSeek | 77.6%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] maxPublished Jun 18, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 77.6 | 85% confidence 85 percent, High |
| 22 | Mistral Medium 3.5 Mistral AI | 77.6%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 77.6 | 37% confidence 37 percent, Low |
| 23 | Mistral Medium (latest) Mistral AI | 77.6%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 77.6 | 6% confidence 6 percent, Low |
| 24 | Gemini 3 Pro Preview Google | 77.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] live-SWE-agentPublished Nov 20, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 77.4 | 68% confidence 68 percent, Medium |
| 25 | Qwen3.7 Max Alibaba / Qwen | 77.3%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Jun 18, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 77.3 | 53% confidence 53 percent, Medium |
| 26 | Qwen3.6 27B Alibaba / Qwen | 77.2%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 77.2 | 53% confidence 53 percent, Medium |
| 27 | GPT-5.4 OpenAI | 76.9%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] highPublished Mar 6, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 76.9 | 100% confidence 100 percent, Full |
| 28 | Claude Opus 4.5 (latest) Anthropic | 76.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; high; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 76.8 | 14% confidence 14 percent, Low |
| 29 | Claude Sonnet 4 Anthropic | 76.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] EPAM AI/Run Developer AgentPublished Aug 4, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 76.8 | 100% confidence 100 percent, Full |
| 30 | Qwen3.6 Max Preview Alibaba / Qwen | 76.7%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published May 28, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 76.7 | 64% confidence 64 percent, Medium |
| 31 | Claude Opus 4.5 Anthropic | 76.7%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 5, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 76.7 | 100% confidence 100 percent, Full |
| 32 | Kimi K2.6 Moonshot AI | 76.7%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published May 8, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 76.7 | 85% confidence 85 percent, High |
| 33 | Step 3.7 Flash StepFun | 76.5%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published May 29, 2026
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 76.5 | 13% confidence 13 percent, Low |
| 34 | Qwen3.5 397B-A17B Alibaba / Qwen | 76.4%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 76.4 | 69% confidence 69 percent, Medium |
| 35 | Muse Glimmer 30B Meta | 76%Official model cards via models.devLab-reported; metric resolve rate; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Aug 10, 2026
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 76.0 | 6% confidence 6 percent, Low |
| 36 | Gemini 3 Flash Preview Google | 75.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; high; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 75.8 | 93% confidence 93 percent, High |
| 37 | MiniMax-M2.5 MiniMax | 75.8%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 75.8 | 100% confidence 100 percent, Full |
| 38 | MiniMax-M2.5 MiniMax | 75.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; high; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 75.8 | 100% confidence 100 percent, Full |
| 39 | Gemini 3.1 Pro Preview Custom Tools Google | 75.6%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 24, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 75.6 | 32% confidence 32 percent, Low |
| 40 | Claude Opus 4.6 Anthropic | 75.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 75.6 | 100% confidence 100 percent, Full |
| 41 | Ornith 1.0 35B DeepReinforce | 75.6%Official model cards via models.devLab-reported; metric percent resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 75.6 | 100% confidence 100 percent, Full |
| 42 | Gemini 3 Flash Preview Google | 75.4%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 18, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 75.4 | 93% confidence 93 percent, High |
| 43 | Claude Sonnet 4.6 Anthropic | 75.2%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 21, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 75.2 | 100% confidence 100 percent, Full |
| 44 | Claude Sonnet 4 Anthropic | 74.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Harness AIPublished Jul 31, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 74.8 | 100% confidence 100 percent, Full |
| 45 | Claude Sonnet 4.5 (latest) Anthropic | 74.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Sonar Foundation AgentPublished Nov 3, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 74.8 | 43% confidence 43 percent, Low |
| 46 | GPT-5.3 Codex OpenAI | 74.8%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] highPublished Feb 25, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 74.8 | 32% confidence 32 percent, Low |
| 47 | Claude Sonnet 4 Anthropic | 74.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Lingxi-v1.5Published Jul 20, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 74.6 | 100% confidence 100 percent, Full |
| 48 | Claude Opus 4.5 (latest) Anthropic | 74.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; medium; 1.16.0Published Nov 24, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 74.4 | 14% confidence 14 percent, Low |
| 49 | GPT-5 OpenAI | 74.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Prometheus-v1.2.1Published Oct 15, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 74.4 | 100% confidence 100 percent, Full |
| 50 | Step 3.5 Flash StepFun | 74.4%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 74.4 | 88% confidence 88 percent, High |
| 51 | Hy3 preview Tencent | 74.4%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 74.4 | 13% confidence 13 percent, Low |
| 52 | Gemini 3 Pro Preview Google | 74.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.15.0Published Nov 18, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 74.2 | 68% confidence 68 percent, Medium |
| 53 | GLM-5.1 Z.ai | 74.2%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published May 15, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 74.2 | 64% confidence 64 percent, Medium |
| 54 | MiniMax-M2.1 MiniMax | 74%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 74.0 | 13% confidence 13 percent, Low |
| 55 | GLM-4.7 Z.ai | 73.8%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 73.8 | 69% confidence 69 percent, Medium |
| 56 | Kimi K2.5 Moonshot AI | 73.8%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 17, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 73.8 | 100% confidence 100 percent, Full |
| 57 | GPT-5.2 OpenAI | 73.8%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] highPublished Feb 12, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 73.8 | 100% confidence 100 percent, Full |
| 58 | GPT-5 OpenAI | 73.6%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] highPublished Feb 6, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 73.6 | 100% confidence 100 percent, Full |
| 59 | Qwen3.6 35B-A3B Alibaba / Qwen | 73.4%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 73.4 | 37% confidence 37 percent, Low |
| 60 | Claude Opus 4.1 Anthropic | 73.3%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 11, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 73.3 | 80% confidence 80 percent, High |
| 61 | Claude Opus 4 Anthropic | 73.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] ToolsPublished May 22, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 73.2 | 100% confidence 100 percent, Full |
| 62 | Gemini 3 Pro Preview Google | 72.9%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 13, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 72.9 | 68% confidence 68 percent, Medium |
| 63 | GPT-5.2 OpenAI | 72.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; high; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 72.8 | 100% confidence 100 percent, Full |
| 64 | GPT-5.2 Codex OpenAI | 72.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 2.0.0Published Feb 19, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 72.8 | 43% confidence 43 percent, Low |
| 65 | GLM-5 Z.ai | 72.8%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 72.8 | 90% confidence 90 percent, High |
| 66 | GLM-5 Z.ai | 72.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; high; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 72.8 | 90% confidence 90 percent, High |
| 67 | Qwen3.5 27B Alibaba / Qwen | 72.4%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 72.4 | 37% confidence 37 percent, Low |
| 68 | Claude Sonnet 4 Anthropic | 72.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] ToolsPublished May 22, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 72.4 | 100% confidence 100 percent, Full |
| 69 | GLM-5 Z.ai | 72.1%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 15, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 72.1 | 90% confidence 90 percent, High |
| 70 | Qwen3.5 122B-A10B Alibaba / Qwen | 72%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 72.0 | 37% confidence 37 percent, Low |
| 71 | GPT-5 OpenAI | 71.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] OpenHandsPublished Aug 7, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 71.8 | 100% confidence 100 percent, Full |
| 72 | GPT-5.2 OpenAI | 71.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; high; 1.17.2Published Dec 11, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 71.8 | 100% confidence 100 percent, Full |
| 73 | GPT-5 OpenAI | 71.5%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] mediumPublished Feb 5, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 71.5 | 100% confidence 100 percent, Full |
| 74 | Claude Sonnet 4.5 (latest) Anthropic | 71.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; high; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 71.4 | 43% confidence 43 percent, Low |
| 75 | Kimi K2 Thinking Moonshot AI | 71.3%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 71.3 | 19% confidence 19 percent, Low |
| 76 | Claude Sonnet 4.5 Anthropic | 71.3%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 5, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 71.3 | 80% confidence 80 percent, High |
| 77 | Claude Sonnet 4 Anthropic | 71.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] BloopPublished Jul 10, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 71.2 | 100% confidence 100 percent, Full |
| 78 | GPT-5 OpenAI | 71.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Prometheus-v1.2Published Sep 29, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 71.2 | 100% confidence 100 percent, Full |
| 79 | Laguna XS 2.1 Poolside | 70.9%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] HarborPublished Jul 2, 2026
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 70.9 | 100% confidence 100 percent, Full |
| 80 | Claude Sonnet 4 Anthropic | 70.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Moatless ToolsPublished Jun 11, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 70.8 | 100% confidence 100 percent, Full |
| 81 | Kimi K2.5 Moonshot AI | 70.8%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 70.8 | 100% confidence 100 percent, Full |
| 82 | Kimi K2.5 Moonshot AI | 70.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; high; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 70.8 | 100% confidence 100 percent, Full |
| 83 | Nemotron 3 Ultra 550B A55B NVIDIA | 70.7%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jun 4, 2026
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 70.7 | 100% confidence 100 percent, Full |
| 84 | Claude Opus 4 Anthropic | 70.7%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 6, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 70.7 | 100% confidence 100 percent, Full |
| 85 | Claude Sonnet 4.5 (latest) Anthropic | 70.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.13.3Published Sep 29, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 70.6 | 43% confidence 43 percent, Low |
| 86 | Claude Sonnet 4 Anthropic | 70.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] OpenHandsPublished May 24, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 70.4 | 100% confidence 100 percent, Full |
| 87 | DeepSeek V3.2 DeepSeek | 70%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; high; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 70.0 | 56% confidence 56 percent, Medium |
| 88 | Qwen3-Coder 480B-A35B Instruct Alibaba / Qwen | 69.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] OpenHandsPublished Aug 5, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 69.6 | 47% confidence 47 percent, Low |
| 89 | Ornith 1.0 9B DeepReinforce | 69.4%Official model cards via models.devLab-reported; metric percent resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 69.4 | 100% confidence 100 percent, Full |
| 90 | MiniMax-M2 MiniMax | 69.4%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 69.4 | 96% confidence 96 percent, High |
| 91 | GPT-5.2 OpenAI | 69%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.17.2Published Dec 11, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 69.0 | 100% confidence 100 percent, Full |
| 92 | GLM-4.6 Z.ai | 68.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] UndisclosedPublished Sep 30, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 68.2 | 55% confidence 55 percent, Medium |
| 93 | GPT-5.1 OpenAI | 68.0%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] highPublished Feb 18, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 68.0 | 85% confidence 85 percent, High |
| 94 | Claude Opus 4 Anthropic | 67.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.0.0Published Aug 2, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 67.6 | 100% confidence 100 percent, Full |
| 95 | North Mini Code Cohere | 67.6%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] SWE-agentPublished Jun 9, 2026
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 67.6 | 13% confidence 13 percent, Low |
| 96 | Claude Haiku 4.5 (latest) Anthropic | 66.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; high; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 66.6 | 33% confidence 33 percent, Low |
| 97 | Claude Sonnet 4 Anthropic | 66.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] SWE-agentPublished May 22, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 66.6 | 100% confidence 100 percent, Full |
| 98 | Claude Sonnet 3.7 Anthropic | 66.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Aime-coder v1Published May 14, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 66.4 | 100% confidence 100 percent, Full |
| 99 | GPT-5.1 OpenAI | 66%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; medium; 1.15.0Published Nov 20, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 66.0 | 85% confidence 85 percent, High |
| 100 | GPT-5.1 Codex OpenAI | 66%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; medium; 1.16.0Published Nov 24, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 66.0 | 14% confidence 14 percent, Low |
| 101 | GPT-5 OpenAI | 65%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; medium; 1.7.0Published Aug 7, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 65.0 | 100% confidence 100 percent, Full |
| 102 | Claude Sonnet 4 Anthropic | 64.9%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.0.0Published Jul 26, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 64.9 | 100% confidence 100 percent, Full |
| 103 | GPT-5 Mini OpenAI | 64.7%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] mediumPublished Feb 1, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 64.7 | 99% confidence 99 percent, High |
| 104 | o4-mini OpenAI | 64.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] PatchPilot-v1.1Published May 3, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 64.6 | 87% confidence 87 percent, High |
| 105 | GLM-4.5 Z.ai | 64.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] UndisclosedPublished Jul 28, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 64.2 | 47% confidence 47 percent, Low |
| 106 | Kimi K2 Thinking Moonshot AI | 63.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.17.2Published Dec 10, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 63.4 | 19% confidence 19 percent, Low |
| 107 | Claude Sonnet 3.7 Anthropic | 63.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] ToolsPublished Feb 24, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 63.2 | 100% confidence 100 percent, Full |
| 108 | o3 OpenAI | 62.3%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] mediumPublished Feb 12, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 62.3 | 87% confidence 87 percent, High |
| 109 | Devstral Medium Mistral AI | 61.6%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jul 10, 2025
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 61.6 | 6% confidence 6 percent, Low |
| 110 | MiniMax-M2 MiniMax | 61%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.17.0Published Nov 24, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 61.0 | 96% confidence 96 percent, High |
| 111 | Claude Sonnet 3.7 Anthropic | 61.0%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 4, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 61.0 | 100% confidence 100 percent, Full |
| 112 | GPT-5 Mini OpenAI | 59.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; medium; 1.7.0Published Aug 7, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 59.8 | 99% confidence 99 percent, High |
| 113 | GLM-4.7-Flash Z.ai | 59.2%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant]
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 59.2 | 69% confidence 69 percent, Medium |
| 114 | o3 OpenAI | 58.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.0.0Published Jul 26, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 58.4 | 87% confidence 87 percent, High |
| 115 | Qwen3.6 Plus Alibaba / Qwen | 57.9%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published May 14, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 57.9 | 80% confidence 80 percent, High |
| 116 | Gemini 2.5 Pro Google | 57.6%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 13, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 57.6 | 90% confidence 90 percent, High |
| 117 | Claude Sonnet 4 Anthropic | 57%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Artemis Agent v2Published Sep 24, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 57.0 | 100% confidence 100 percent, Full |
| 118 | GPT-5 Mini OpenAI | 56.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 2.0.0Published Feb 17, 2026
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 56.2 | 99% confidence 99 percent, High |
| 119 | Qwen3-Coder 480B-A35B Instruct Alibaba / Qwen | 55.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.0.0Published Aug 2, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 55.4 | 47% confidence 47 percent, Low |
| 120 | GLM-4.6 Z.ai | 55.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.17.1Published Dec 1, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 55.4 | 55% confidence 55 percent, Medium |
| 121 | GLM-4.5 Z.ai | 54.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.9.1Published Aug 22, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 54.2 | 47% confidence 47 percent, Low |
| 122 | Devstral 2 Mistral AI | 53.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.17.2Published Dec 9, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 53.8 | 19% confidence 19 percent, Low |
| 123 | Gemini 2.5 Pro Google | 53.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.0.0Published Jul 26, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 53.6 | 90% confidence 90 percent, High |
| 124 | Devstral Small Mistral AI | 53.6%Official model cards via models.devLab-reported; metric resolved; transcribed by MIT models.dev catalog; not independently evaluated [variant] Published Jul 10, 2025
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 53.6 | 19% confidence 19 percent, Low |
| 125 | Claude Sonnet 3.5 v2 Anthropic | 53%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] OpenHandsPublished Oct 29, 2024
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 53.0 | 100% confidence 100 percent, Full |
| 126 | Claude Sonnet 3.7 Anthropic | 52.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 0.0.0Published Jul 20, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 52.8 | 100% confidence 100 percent, Full |
| 127 | Qwen3-Coder 30B-A3B Instruct Alibaba / Qwen | 52.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] EntroPO + R2EPublished Sep 1, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 52.2 | 19% confidence 19 percent, Low |
| 128 | Qwen3-Coder 30B-A3B Instruct Alibaba / Qwen | 51.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] OpenHandsPublished Aug 5, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 51.6 | 19% confidence 19 percent, Low |
| 129 | Claude Sonnet 3.5 v2 Anthropic | 51.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] AutoCodeRover-v2.1Published Jan 22, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 51.6 | 100% confidence 100 percent, Full |
| 130 | Nemotron 3.5 Lightning 30B A3B NVIDIA | 51.6%Official model cards via models.devLab-reported; metric accuracy; transcribed by MIT models.dev catalog; not independently evaluated [variant] BF16; reasoning; NeMo Evaluator
Retrieved Oct 9, 2026 · factual citation; MIT transcription Open source ↗ | 51.6 | 100% confidence 100 percent, Full |
| 131 | GPT-4.1 OpenAI | 48.5%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 8, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 48.5 | 100% confidence 100 percent, Full |
| 132 | Claude Sonnet 3.5 v2 Anthropic | 46.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] AutoCodeRover-v2.0Published Nov 8, 2024
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 46.2 | 100% confidence 100 percent, Full |
| 133 | o4-mini OpenAI | 45%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.0.0Published Jul 26, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 45.0 | 87% confidence 87 percent, High |
| 134 | o3-mini OpenAI | 42.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Agentless LitePublished Feb 14, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 42.4 | 96% confidence 96 percent, High |
| 135 | DeepSeek V3 0324 DeepSeek | 42%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] SWE-ExpPublished Aug 6, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 42.0 | 74% confidence 74 percent, Medium |
| 136 | Claude Haiku 3.5 Anthropic | 40.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] ToolsPublished Oct 22, 2024
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 40.6 | 100% confidence 100 percent, Full |
| 137 | GPT-4.1 OpenAI | 39.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.0.0Published Jul 26, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 39.6 | 100% confidence 100 percent, Full |
| 138 | GPT-4o OpenAI | 38.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Agentless-1.5Published Oct 28, 2024
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 38.8 | 59% confidence 59 percent, Medium |
| 139 | GPT-4o OpenAI | 38.4%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] AutoCodeRoverPublished Jun 28, 2024
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 38.4 | 59% confidence 59 percent, Medium |
| 140 | Qwen2.5-Coder-32B-Instruct Alibaba / Qwen | 38%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] Skywork-SWE-32BPublished Jun 16, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 38.0 | 90% confidence 90 percent, High |
| 141 | Devstral Small Mistral AI | 38%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] SWE-agentPublished Jul 25, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 38.0 | 19% confidence 19 percent, Low |
| 142 | GPT-5 Nano OpenAI | 34.8%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; medium; 1.7.0Published Aug 7, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 34.8 | 85% confidence 85 percent, High |
| 143 | GPT-4o OpenAI | 32.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] MASAIPublished Jun 12, 2024
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 32.6 | 59% confidence 59 percent, Medium |
| 144 | GPT-4o (2024-11-20) OpenAI | 31.0%Epoch AI BenchmarkingEpoch-owned evaluation mean score; scores only, no benchmark questions [variant] Published Feb 11, 2026
Retrieved Oct 9, 2026 · CC-BY Open source ↗ | 31.0 | 61% confidence 61 percent, Medium |
| 145 | Gemini 2.5 Flash Google | 28.7%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.0.0Published Jul 26, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 28.7 | 47% confidence 47 percent, Low |
| 146 | GPT-4o OpenAI | 27%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] EPAM AI/Run Developer AgentPublished Oct 16, 2024
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 27.0 | 59% confidence 59 percent, Medium |
| 147 | GPT-4o OpenAI | 26.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] AppMap NaviePublished Jun 15, 2024
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 26.2 | 59% confidence 59 percent, Medium |
| 148 | GPT OSS 120B OpenAI | 26%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.7.0Published Aug 7, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 26.0 | 79% confidence 79 percent, Medium |
| 149 | GPT-4.1 mini OpenAI | 23.9%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 0.0.0Published Jul 20, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 23.9 | 87% confidence 87 percent, High |
| 150 | GPT-4o OpenAI | 23.2%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] SWE-agentPublished Jul 28, 2024
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 23.2 | 59% confidence 59 percent, Medium |
| 151 | GPT-4o OpenAI | 21.6%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 0.0.0Published Jul 20, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 21.6 | 59% confidence 59 percent, Medium |
| 152 | Gemini 2.0 Flash Google | 13.5%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 0.0.0Published Jul 26, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 13.5 | 28% confidence 28 percent, Low |
| 153 | Qwen2.5-Coder-32B-Instruct Alibaba / Qwen | 9%SWE-bench VerifiedSWE-bench published model plus agent result; harness retained, not a base model evaluation [variant] mini-SWE-agent; 1.0.0Published Aug 3, 2025
Retrieved Oct 9, 2026 · factual citation Open source ↗ | 9.0 | 90% confidence 90 percent, High |
Results are as published by the source behind each value (hover or tap the number). Benchmark scores are shown individually for every source/variant (a model may have multiple rows), and vary by version, harness and date; the normalized column uses the method’s fixed 0–100 scales and feeds the coding pillar of the SI Score.