SUDOKU BENCHMARK · REASONING MEDIUM

GPT-6 Astra

Openai · released 2026-09-03 · route gpt-6-astra · Model profile · Chess results

Results

Score
7695%: 64 to 88
Grids solved
32 of 42
Blank cells filled correctly
94.0%
Wrong cells / empty cells / changed a given / no grid
9 / 0 / 1 / 0
Broke the rules (complete grid repeating a digit)
9
Near misses (three cells or fewer wrong)
0
Average stated confidence
100%
Average confidence when it had not solved it
100%
Sure but wrong (90% or more, unsolved)
10
Calibration (Brier, lower is better)
0.238
API-equivalent cost per grid
$0.453
API-equivalent cost per solved grid
$0.595
Seconds per grid
226
Tokens per grid (in / out)
27,660 / 6,987
Reasoning tokens per grid
6,614
Retries for an unreadable reply
0
Aborted on our side (not counted)
0

By tier

TierNeedsSolvedOutput tokensSeconds
CozySingles only5 of 62,76896
GentleScanning4 of 64,133130
SteadyLocked candidates5 of 610,358325
TrickyHidden pairs and triples4 of 65,546183
DeviousWings and fish4 of 68,886296
ExpertDeep wings and fish6 of 65,917207
BeyondNo human technique finishes it4 of 611,303344

Cells by the technique they needed

Each blank cell is tagged with the simplest technique that fills it, given everything a solver could fill before it.

TechniqueCellsFilled correctly
Last free cell43298.2%
Scanning67495.7%
Locked candidates27498.2%
Hidden pairs and triples25884.1%
Wings and fish16590.9%
Deep wings and fish181100.0%
Search24985.9%

Every answer

Cozy #1Solved 50 of 50 cells100% sure · 1,797 tokens · $0.151Cozy #2Solved 48 of 48 cells100% sure · 3,141 tokens · $0.304Cozy #3Wrong cells 41 of 49 cells100% sure · 4,855 tokens · $0.358Cozy #4Solved 50 of 50 cells100% sure · 2,923 tokens · $0.262Cozy #5Solved 49 of 49 cells100% sure · 1,845 tokens · $0.208Cozy #6Solved 39 of 39 cells100% sure · 2,048 tokens · $0.138Gentle #1Wrong cells 48 of 56 cells100% sure · 3,896 tokens · $0.311Gentle #2Solved 51 of 51 cells100% sure · 5,579 tokens · $0.353Gentle #3Solved 52 of 52 cells100% sure · 3,848 tokens · $0.342Gentle #4Wrong cells 33 of 51 cells100% sure · 2,681 tokens · $0.250Gentle #5Solved 56 of 56 cells100% sure · 5,908 tokens · $0.500Gentle #6Solved 52 of 52 cells100% sure · 2,887 tokens · $0.260Steady #1Solved 53 of 53 cells100% sure · 3,710 tokens · $0.348Steady #2Solved 53 of 53 cells100% sure · 3,227 tokens · $0.237Steady #3Wrong cells 43 of 49 cells100% sure · 2,359 tokens · $0.179Steady #4Solved 54 of 54 cells100% sure · 13,044 tokens · $0.700Steady #5Solved 53 of 53 cells100% sure · 11,426 tokens · $0.606Steady #6Solved 54 of 54 cells100% sure · 28,379 tokens · $1.49Tricky #1Solved 55 of 55 cells100% sure · 5,754 tokens · $0.473Tricky #2Wrong cells 46 of 57 cells100% sure · 5,801 tokens · $0.380Tricky #3Wrong cells 26 of 57 cells100% sure · 5,110 tokens · $0.341Tricky #4Solved 54 of 54 cells100% sure · 6,032 tokens · $0.390Tricky #5Solved 55 of 55 cells100% sure · 6,078 tokens · $0.525Tricky #6Solved 56 of 56 cells100% sure · 4,500 tokens · $0.282Devious #1Solved 55 of 55 cells100% sure · 13,165 tokens · $0.786Devious #2Wrong cells 40 of 55 cells100% sure · 11,490 tokens · $0.658Devious #3Solved 49 of 49 cells100% sure · 15,585 tokens · $0.965Devious #4Solved 40 of 40 cells100% sure · 3,833 tokens · $0.356Devious #5Solved 47 of 47 cells100% sure · 4,585 tokens · $0.264Devious #6Changed a given digit 53 of 54 cells100% sure · 4,657 tokens · $0.325Expert #1Solved 58 of 58 cells100% sure · 11,374 tokens · $0.699Expert #2Solved 53 of 53 cells100% sure · 2,708 tokens · $0.170Expert #3Solved 54 of 54 cells100% sure · 4,814 tokens · $0.380Expert #4Solved 57 of 57 cells100% sure · 4,921 tokens · $0.326Expert #5Solved 58 of 58 cells100% sure · 8,503 tokens · $0.460Expert #6Solved 54 of 54 cells100% sure · 3,180 tokens · $0.274Beyond #1Wrong cells 48 of 60 cells100% sure · 6,586 tokens · $0.377Beyond #2Wrong cells 34 of 57 cells100% sure · 7,619 tokens · $0.466Beyond #3Solved 56 of 56 cells100% sure · 20,556 tokens · $1.09Beyond #4Solved 57 of 57 cells100% sure · 12,496 tokens · $0.823Beyond #5Solved 59 of 59 cells100% sure · 5,950 tokens · $0.359Beyond #6Solved 57 of 57 cells100% sure · 14,610 tokens · $0.858