Skip to main content
Scores land on the console’s Benchmarks page, in the same matrix as every other run, comparable across runs, methods and base models.
TunePlane benchmarks page

The benchmark matrix: runs down, benchmarks across, best score highlighted.

​
What you can score

Scoring a run needs it to have been exported. --then export at submission saves you remembering.

​
Pick benchmarks

Built in: gsm8k, math, humaneval, mmlu, ceval, ifeval, plus the RTL packs. Anything published as a benchmark pack appears here too. Run several across different categories. Post-training reliably improves one axis and quietly damages another, and that only becomes visible if you measure more than the one you were optimising.

​
Options worth knowing

A --limit score is not comparable with a full one, and nothing downstream marks it as truncated. Use it to check the pipeline works, then run without it.
Both runners produce the same score contract, so the same benchmark is comparable whichever ran it.

​
Score every run automatically

Running it by hand is fine once. Making it happen on every run is what stops a regression reaching production. An evaluation gate in the experiment config goes further: a run that falls below a threshold is marked as failing its gate. See the eval: block in your method’s template.

​
Report scores from a harness that cannot run here

A licensed tool, or a physical test rig:
create returns a run id, and submit reports the score file against it. --train-run is what lets the evaluation gate and the model version treat these scores as authoritative; without it they are recorded but not. They enter the same matrix, labelled as externally produced.

​
Check it worked

Then open the Benchmarks page. A row for your run with a score in each requested column means it worked. An empty column means that suite failed, and the job log names which. Scores are ingested idempotently: re-reporting the same run and benchmark updates the row rather than adding a second one.

​
Reading the matrix

Each metric carries a declared direction, so the comparison knows which way is better without guessing. Where a benchmark reports more than one metric, the gap between them is often the useful part: GSM8K’s strict and flexible matches diverging means the model can do the arithmetic but is not formatting the answer as asked.

​
Next

Write a benchmark pack to declare your own evaluation set · Rubrics to score what no rule can decide · Comparing runs