> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Run a hyperparameter sweep

> Expand a grid into one submission per variant, and cut the hopeless ones early.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp sweep my-grpo --profile h200:4 \
  -g policy.optimizer.kwargs.lr=1e-6,2e-6,5e-6 \
  -g loss_fn.reference_policy_kl_penalty=0.01,0.05 \
  -s grpo.num_generations_per_prompt=8
```

Six variants, submitted one by one. `-g` declares a grid axis and is repeatable; several axes
take the Cartesian product. `-s` is a fixed override every variant shares.

Each variant goes through the ordinary submit path: local validation, quota, queue. There is no
batch channel that skips a gate.

## Preview the grid first

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp sweep my-grpo --profile h200:4 \
  -g policy.optimizer.kwargs.lr=1e-6,2e-6 --dry-run
```

Prints the variant list with each one's full override set, and submits nothing. Do this before
a grid you have not counted.

## Track the group

Every variant carries one `sweep_id`, formed as `sweep-<experiment>-<timestamp>`. The console's
Jobs page badges sweep members, and [Projects](https://docs.tuneplane.ai/en/console/projects.md) puts their curves on one
chart.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp job stop-sweep <SWEEP_ID>    # stop every active job in the group
```

Quota counts per variant. Six variants at four GPUs is 24 concurrent GPUs of demand, so
variants over quota wait in `QUEUED` rather than failing — which also means they do not all
start at the same moment.

## Cut the hopeless variants early

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp sweep my-grpo --profile h200:4 \
  -g policy.optimizer.kwargs.lr=1e-6,1e-5,1e-4 \
  -g loss_fn.reference_policy_kl_penalty=0.01,0.05 \
  --early-stop
```

The platform compares variants at step **rungs**, stops the ones at the bottom, and lets the
rest continue.

| Flag | Default | Meaning |
| - | - | - |
| `--stop-metric` | the method's primary metric | Which metric to judge on. Higher- or lower-is-better comes from the method's metric contract |
| `--rungs` | `100,300,900` | The rungs, in steps |
| `--keep` | `3` | Keep 1/3 of the variants at each rung |

A stopped variant's job page says at which step, against how many variants, and at what rank.

Four rules the comparison follows, which are worth knowing before you read a result:

* Judging happens only at a rung, never continuously, so a variant in a normal dip survives.
* Only variants that reached the same rung are compared.
* Fewer than four variants at a rung means no decision.
* At least one variant always survives, whatever `--keep` says.

An administrator can turn the whole feature off with
`TUNEPLANE_SWEEP_EARLY_STOP_ENABLED=false`, without editing any sweep.

## How to spend less

Sweep the learning rate across orders of magnitude first — `1e-6`, `1e-5`, `1e-4` — then run a
finer sweep inside the magnitude that won. Lower `grpo.max_num_steps` to the smallest number
that shows a trend, pick the top two, and train those at full length.

## Check it worked

`tp sweep` prints the `sweep_id` and how many variants were submitted or queued. `tp job ls`
shows them, and the console's Jobs page badges them as one group.

## Next

[Submitting a job](https://docs.tuneplane.ai/en/guides/submit.md) · [Comparing runs](https://docs.tuneplane.ai/en/console/review.md) ·
[Pipelines](https://docs.tuneplane.ai/en/guides/pipelines.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.