> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Score with a rubric

> One written standard that supplies both the training reward and the benchmark score.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp new my-judge --method trl/grpo-judge
tp submit my-judge --profile h200:8 \
  --model Qwen/Qwen3.5-9B --train-data data/train.parquet \
  -s rubric=alice/answer-quality
```

A rubric is your team's written standard for a correct answer. Use one wherever no rule decides
correctness: writing quality, tone, house conventions, whether an explanation actually explains.

The reason it is an object rather than a prompt is that **the same rubric can be both the reward
training optimises and the score a benchmark reports**. The number going up during training is
then the number on the board, not a proxy that drifts away from it.

> [!NOTE]
>
> This page is about using one. To write a good one — criteria, weights, versioning — see
> [writing rubrics](https://docs.tuneplane.ai/en/extend/rubrics.md).

![TunePlane rubrics page](https://mintcdn.com/starforge/GatXR2rI5-_Vm4_H/images/console/rubrics.png?fit=max\&auto=format\&n=GatXR2rI5-_Vm4_H\&q=85\&s=27b1058e217a631028b4b7f253374254)

Rubrics with their criteria, versions and usage.

## As a training reward

The direct path is `trl/grpo-judge`, above: the platform judge scores each completion against
the rubric and that score is the reward.

For an agent environment, declare it as the environment's verifier instead:

```json manifest.json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
{ "verifier": { "kind": "rubric", "ref": "alice/answer-quality" } }
```

Either way the platform judge does the scoring. Your training code does not call an LLM itself.

Both need `TUNEPLANE_JUDGE_ENABLED` on the deployment, which is what injects the judge endpoint
and token into the job.

## As a benchmark

Declare it in a [benchmark pack](https://docs.tuneplane.ai/en/extend/benchmark-packs.md) with the `judge` runner:

```yaml benchmark.yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
schema: tuneplane/benchmark/v1
name: answer-quality
title: Answer quality · customer support
runner: judge
suites: [support-questions]
rubric: alice/answer-quality
metrics:
  - key: weighted_score
    label: Weighted rubric score
    primary: true
```

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp bench run my-bench -m run:run-4f2a91 --suites answer-quality
```

For that runner the rubric **is** the benchmark's definition. Swap the rubric and it is a
different benchmark, which is why it is a declared field rather than a flag.

## As a safety evaluation

The `safety` runner uses a rubric to decide whether the model declined what it should have. The
suite says which prompts should be refused; the rubric says what counts as refusing.

Both belong to your deployment. The platform ships neither, because shipping either would be it
declaring a verdict it has no standing to declare.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp bench new refusal-policy --rubric alice/refusal-standard --title "Refusal policy"
```

## Keep training and evaluation on the same standard

The temptation is to train against a rubric and evaluate against a slightly different one,
usually because somebody sharpened the training rubric mid-project. Reference the same
`<owner>/<name>` in both and let versioning carry the change.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \
  https://tuneplane.your-company.com/api/rubrics/alice/answer-quality/usage
```

That lists which runs cited which version, and whether as a reward or as a score. It is the
fastest way to answer "were these two runs judged by the same standard".

Editing a rubric increments its version and invalidates the judge's cached scores for it. Old
runs keep pointing at the version they used, and the revision text is kept so you can still read
what that version said.

## Check it worked

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \
  "https://tuneplane.your-company.com/api/rubrics/resolve?ref=alice/answer-quality"
```

Resolves the reference the way a job would. If that answers, an environment or a benchmark pack
can name it.

During training, watch the reward **distribution** on the Validation tab, not only the mean. A
judge-scored reward that collapses to two values means the criteria are not discriminating. That
is the rubric needing sharpening, not the model.

## Next

[Writing rubrics](https://docs.tuneplane.ai/en/extend/rubrics.md) · [Benchmark packs](https://docs.tuneplane.ai/en/extend/benchmark-packs.md) ·
[Agent environments](https://docs.tuneplane.ai/en/guides/agent-envs.md) · [The Rubrics page](https://docs.tuneplane.ai/en/console/rubrics.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.