SUDOKU BENCHMARK · REASONING MEDIUM

Claude Haiku 5.5

Anthropic · released 2026-10-07 · route claude-haiku-5-5 · Model profile · Chess results

Results

Score
3195%: 19 to 43
Grids solved
13 of 42
Blank cells filled correctly
93.1%
Wrong cells / empty cells / changed a given / no grid
8 / 0 / 21 / 0
Broke the rules (complete grid repeating a digit)
8
Near misses (three cells or fewer wrong)
7
Average stated confidence
96%
Average confidence when it had not solved it
96%
Sure but wrong (90% or more, unsolved)
29
Calibration (Brier, lower is better)
0.643
API-equivalent cost per grid
$0.015
API-equivalent cost per solved grid
$0.049
Seconds per grid
111
Tokens per grid (in / out)
912 / 30,157
Reasoning tokens per grid
30,053
Retries for an unreadable reply
0
Aborted on our side (not counted)
0

By tier

TierNeedsSolvedOutput tokensSeconds
CozySingles only3 of 615,85360
GentleScanning1 of 622,69083
SteadyLocked candidates1 of 634,434127
TrickyHidden pairs and triples3 of 633,600126
DeviousWings and fish2 of 620,78477
ExpertDeep wings and fish2 of 638,906146
BeyondNo human technique finishes it1 of 644,833162

Cells by the technique they needed

Each blank cell is tagged with the simplest technique that fills it, given everything a solver could fill before it.

TechniqueCellsFilled correctly
Last free cell43296.3%
Scanning67493.9%
Locked candidates27493.1%
Hidden pairs and triples25891.5%
Wings and fish16592.1%
Deep wings and fish18190.1%
Search24990.0%

Every answer

Cozy #1Solved 50 of 50 cells97% sure · 16,880 tokens · $0.0085Cozy #2Changed a given digit 45 of 48 cells97% sure · 12,834 tokens · $0.0066Cozy #3Solved 49 of 49 cells97% sure · 14,523 tokens · $0.0074Cozy #4Solved 50 of 50 cells97% sure · 22,664 tokens · $0.012Cozy #5Changed a given digit 47 of 49 cells96% sure · 14,485 tokens · $0.0074Cozy #6Changed a given digit 37 of 39 cells96% sure · 13,730 tokens · $0.0070Gentle #1Changed a given digit 50 of 56 cells97% sure · 24,768 tokens · $0.013Gentle #2Solved 51 of 51 cells97% sure · 19,228 tokens · $0.0098Gentle #3Wrong cells 50 of 52 cells97% sure · 17,928 tokens · $0.0091Gentle #4Wrong cells 49 of 51 cells96% sure · 19,744 tokens · $0.010Gentle #5Wrong cells 55 of 56 cells97% sure · 37,740 tokens · $0.019Gentle #6Changed a given digit 45 of 52 cells95% sure · 16,733 tokens · $0.0085Steady #1Wrong cells 47 of 53 cells96% sure · 47,932 tokens · $0.024Steady #2Changed a given digit 45 of 53 cells97% sure · 26,957 tokens · $0.014Steady #3Wrong cells 48 of 49 cells97% sure · 20,374 tokens · $0.010Steady #4Solved 54 of 54 cells96% sure · 40,368 tokens · $0.020Steady #5Changed a given digit 53 of 53 cells97% sure · 36,937 tokens · $0.019Steady #6Changed a given digit 45 of 54 cells97% sure · 34,036 tokens · $0.017Tricky #1Changed a given digit 42 of 55 cells96% sure · 34,408 tokens · $0.017Tricky #2Changed a given digit 51 of 57 cells97% sure · 45,939 tokens · $0.023Tricky #3Solved 57 of 57 cells96% sure · 40,672 tokens · $0.021Tricky #4Solved 54 of 54 cells97% sure · 27,394 tokens · $0.014Tricky #5Changed a given digit 46 of 55 cells97% sure · 18,264 tokens · $0.0093Tricky #6Solved 56 of 56 cells96% sure · 34,925 tokens · $0.018Devious #1Changed a given digit 48 of 55 cells96% sure · 20,177 tokens · $0.010Devious #2Changed a given digit 46 of 55 cells95% sure · 30,128 tokens · $0.015Devious #3Changed a given digit 44 of 49 cells96% sure · 20,278 tokens · $0.010Devious #4Solved 40 of 40 cells97% sure · 14,358 tokens · $0.0074Devious #5Solved 47 of 47 cells97% sure · 18,428 tokens · $0.0094Devious #6Wrong cells 52 of 54 cells97% sure · 21,335 tokens · $0.011Expert #1Changed a given digit 52 of 58 cells97% sure · 83,330 tokens · $0.042Expert #2Changed a given digit 36 of 53 cells97% sure · 19,760 tokens · $0.010Expert #3Changed a given digit 54 of 54 cells97% sure · 26,482 tokens · $0.013Expert #4Solved 57 of 57 cells97% sure · 43,745 tokens · $0.022Expert #5Changed a given digit 56 of 58 cells96% sure · 27,271 tokens · $0.014Expert #6Solved 54 of 54 cells96% sure · 32,847 tokens · $0.017Beyond #1Changed a given digit 52 of 60 cells97% sure · 51,581 tokens · $0.026Beyond #2Changed a given digit 48 of 57 cells95% sure · 44,178 tokens · $0.022Beyond #3Wrong cells 54 of 56 cells97% sure · 55,398 tokens · $0.028Beyond #4Wrong cells 55 of 57 cells96% sure · 37,645 tokens · $0.019Beyond #5Solved 59 of 59 cells96% sure · 38,412 tokens · $0.019Beyond #6Changed a given digit 49 of 57 cells96% sure · 41,784 tokens · $0.021