> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# tp annotate

> Send two runs' answers out for human judgement and bring the preferences back.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp annotate status
tp annotate push run-4f2a91 run-8b1c02 --limit 200
tp annotate pull tuneplane-run-4f2a91-vs-run-8b1c02 --into qa-preference
```

Two run ids, because preference annotation asks which of A and B is better. The walkthrough is
[collecting preference annotations](https://docs.tuneplane.ai/en/guides/annotation.md).

## Worked examples

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp annotate status                                        # is Argilla configured at all
tp annotate push run-4f2a91 run-8b1c02 --limit 200
tp annotate push run-4f2a91 run-8b1c02 -d q3-writing      # name the Argilla dataset yourself

tp annotate pull tuneplane-run-4f2a91-vs-run-8b1c02                          # statistics only
tp annotate pull tuneplane-run-4f2a91-vs-run-8b1c02 --into qa-preference
tp annotate pull tuneplane-run-4f2a91-vs-run-8b1c02 --into qa-preference -v 20260912
```

`pull` takes the **Argilla** dataset name as its argument; `--into` names the platform dataset the
preferences land in, as a new version. Without `--into` it only prints the statistics.

Start with `tp annotate status`. Everything else fails confusingly on a deployment with no Argilla
instance wired up.

## `tp annotate`

Manage preference annotation

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp annotate <subcommand>
```

| Command | Description |
| - | - |
| `tp annotate pull` | Pull annotations back as a dataset |
| `tp annotate push` | Send answers out for annotation |
| `tp annotate status` | Check the Argilla integration |

### `tp annotate pull`

Pull annotations back as a dataset

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp annotate pull <DATASET> [options]
```

| Argument | Description |
| - | - |
| `DATASET` | Argilla dataset name (**required**) |

| Option | Description |
| - | - |
| `--into` | Land them as a new version of this platform dataset; omit to only print the statistics |
| `--version`, `-v` | Dataset version; defaults to the date |

### `tp annotate push`

Send answers out for annotation

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp annotate push <RUN_A> <RUN_B> [options]
```

| Argument | Description |
| - | - |
| `RUN_A` | Run id of the first training run (**required**) |
| `RUN_B` | Run id of the second training run (**required**) |

| Option | Description |
| - | - |
| `--dataset`, `-d` | Argilla dataset name; synthesised when empty |
| `--limit` `<int range>` | How many pairs to push at most (default `200`) |

### `tp annotate status`

Check the Argilla integration

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp annotate status
```

No options beyond `--help`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.