SUDOKU BENCHMARK · REASONING MEDIUM

Claude Sonnet 5.5

Anthropic · released 2026-09-28 · route claude-sonnet-5-5 · Model profile · Chess results

Results

Score
8895%: 78 to 99
Grids solved
37 of 42
Blank cells filled correctly
99.5%
Wrong cells / empty cells / changed a given / no grid
1 / 0 / 4 / 0
Broke the rules (complete grid repeating a digit)
1
Near misses (three cells or fewer wrong)
1
Average stated confidence
96%
Average confidence when it had not solved it
96%
Sure but wrong (90% or more, unsolved)
5
Calibration (Brier, lower is better)
0.111
API-equivalent cost per grid
$0.138
API-equivalent cost per solved grid
$0.157
Seconds per grid
91
Tokens per grid (in / out)
913 / 13,462
Reasoning tokens per grid
13,332
Retries for an unreadable reply
0
Aborted on our side (not counted)
0

By tier

TierNeedsSolvedOutput tokensSeconds
CozySingles only5 of 65,88443
GentleScanning5 of 610,28871
SteadyLocked candidates5 of 614,39296
TrickyHidden pairs and triples6 of 617,670119
DeviousWings and fish4 of 68,80762
ExpertDeep wings and fish6 of 614,05895
BeyondNo human technique finishes it6 of 623,134150

Cells by the technique they needed

Each blank cell is tagged with the simplest technique that fills it, given everything a solver could fill before it.

TechniqueCellsFilled correctly
Last free cell43298.6%
Scanning67499.6%
Locked candidates274100.0%
Hidden pairs and triples258100.0%
Wings and fish16598.2%
Deep wings and fish181100.0%
Search249100.0%

Every answer

Cozy #1Solved 50 of 50 cells96% sure · 6,149 tokens · $0.065Cozy #2Changed a given digit 43 of 48 cells96% sure · 5,223 tokens · $0.056Cozy #3Solved 49 of 49 cells97% sure · 6,908 tokens · $0.073Cozy #4Solved 50 of 50 cells96% sure · 7,940 tokens · $0.083Cozy #5Solved 49 of 49 cells97% sure · 5,007 tokens · $0.054Cozy #6Solved 39 of 39 cells96% sure · 4,079 tokens · $0.044Gentle #1Solved 56 of 56 cells97% sure · 13,650 tokens · $0.140Gentle #2Changed a given digit 50 of 51 cells96% sure · 10,024 tokens · $0.104Gentle #3Solved 52 of 52 cells96% sure · 10,182 tokens · $0.105Gentle #4Solved 51 of 51 cells97% sure · 5,971 tokens · $0.063Gentle #5Solved 56 of 56 cells96% sure · 13,687 tokens · $0.141Gentle #6Solved 52 of 52 cells96% sure · 8,213 tokens · $0.086Steady #1Changed a given digit 52 of 53 cells97% sure · 9,526 tokens · $0.099Steady #2Solved 53 of 53 cells96% sure · 17,456 tokens · $0.178Steady #3Solved 49 of 49 cells96% sure · 9,012 tokens · $0.094Steady #4Solved 54 of 54 cells96% sure · 15,119 tokens · $0.155Steady #5Solved 53 of 53 cells97% sure · 11,851 tokens · $0.122Steady #6Solved 54 of 54 cells96% sure · 23,390 tokens · $0.238Tricky #1Solved 55 of 55 cells96% sure · 13,397 tokens · $0.138Tricky #2Solved 57 of 57 cells97% sure · 11,653 tokens · $0.120Tricky #3Solved 57 of 57 cells97% sure · 11,914 tokens · $0.123Tricky #4Solved 54 of 54 cells97% sure · 10,031 tokens · $0.104Tricky #5Solved 55 of 55 cells97% sure · 9,307 tokens · $0.097Tricky #6Solved 56 of 56 cells97% sure · 49,720 tokens · $0.501Devious #1Solved 55 of 55 cells96% sure · 9,835 tokens · $0.102Devious #2Wrong cells 53 of 55 cells96% sure · 8,502 tokens · $0.089Devious #3Solved 49 of 49 cells96% sure · 8,835 tokens · $0.092Devious #4Changed a given digit 37 of 40 cells96% sure · 8,022 tokens · $0.084Devious #5Solved 47 of 47 cells96% sure · 8,397 tokens · $0.088Devious #6Solved 54 of 54 cells96% sure · 9,252 tokens · $0.096Expert #1Solved 58 of 58 cells97% sure · 22,166 tokens · $0.223Expert #2Solved 53 of 53 cells96% sure · 10,296 tokens · $0.107Expert #3Solved 54 of 54 cells96% sure · 12,852 tokens · $0.132Expert #4Solved 57 of 57 cells96% sure · 16,706 tokens · $0.171Expert #5Solved 58 of 58 cells96% sure · 11,302 tokens · $0.117Expert #6Solved 54 of 54 cells96% sure · 11,023 tokens · $0.114Beyond #1Solved 60 of 60 cells97% sure · 26,396 tokens · $0.268Beyond #2Solved 57 of 57 cells97% sure · 19,087 tokens · $0.195Beyond #3Solved 56 of 56 cells97% sure · 31,352 tokens · $0.317Beyond #4Solved 57 of 57 cells97% sure · 15,752 tokens · $0.161Beyond #5Solved 59 of 59 cells96% sure · 20,492 tokens · $0.209Beyond #6Solved 57 of 57 cells96% sure · 25,722 tokens · $0.261