> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Train with NeMo-RL

> Create, tune and submit a NeMo-RL run, and what the adapter overwrites for you.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp new my-grpo --method nemo-rl/grpo
tp validate my-grpo
tp submit my-grpo --profile h200:8
```

NeMo-RL is the default catalogue for a full post-training stack, at `0.7.0`. Methods:
`nemo-rl/sft`, `dpo`, `grpo`, `grpo-lora`, `ppo`, `rm`, `distillation`, `maxrl`, `opsd`.

Two things you do not have to do: pass `--observability-url`, and call `tuneplane.report`. This
framework's observability is `platform`: the integration enables native TensorBoard output
(`logger.tensorboard_enabled`). An isolated collector reads the event files without modifying
NeMo-RL logger classes. Explicitly configured external trackers remain available.

## Create and tune

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp new my-grpo --method nemo-rl/grpo
tp methods nemo-rl/grpo          # every key you can set, with ranges
tp validate my-grpo
```

Model and datasets come from `configs/base/` and a `configs/models/` fragment, so a submission
usually needs no `--model` or `--train-data`. See [config.yaml](https://docs.tuneplane.ai/en/guides/config.md) for how the
layering resolves.

## Submit

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
git add -A && git commit -m "grpo lr 2e-6"
tp submit my-grpo --profile h200:8
```

The adapter overwrites four keys from the environment the server injects. Do not fight them in
your config:

| Key | Comes from |
| - | - |
| `cluster.num_nodes` | `TUNEPLANE_CLUSTER_NUM_NODES` |
| `cluster.gpus_per_node` | `TUNEPLANE_CLUSTER_GPUS_PER_NODE` |
| `checkpointing.checkpoint_dir` | `TUNEPLANE_OUT_DIR` |
| `logger.log_dir` | `TUNEPLANE_OUT_DIR` |

## Put rollout on different cards

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp submit my-grpo --profile train=h200:8 --profile rollout=h100:2
```

With a separate rollout pool the adapter sets `policy.generation.colocated.enabled=false` and
fills the colocated resource counts from `TUNEPLANE_POOL_TOPOLOGY`. You do not maintain a
non-colocated overlay YAML for this.

Multiple pools need a `kubernetes` or `slurm` Fleet. Node Fleets support one homogeneous pool across at most 8 machines.

## Bring your own run.py

The recipe's entrypoint is the versioned NeMo-RL file. If the recipe declares
`experiment_override` and that file exists in your experiment directory, yours wins — which is
how an experiment with a custom dataset or a tool environment ships its own `run.py`.

`spec.source.entrypoint` in the JobSpec is refused. You cannot override the recipe from
submission JSON.

The container needs `NEMO_RL_DIR`, which the deployment sets as `TUNEPLANE_NEMO_RL_DIR` and is
typically `/opt/nemo-rl` in the NGC image. Without it the submission fails naming the missing
configuration.

## Check it worked

`tp job logs` shows Ray starting and then the first training step. The Charts tab fills with
`train/reward`, `validation/accuracy`, `train/entropy`, `train/kl_penalty` and
`train/token_mult_prob_error`.

Watch that last one. Above one or two percent, weight sync between the trainer and the rollout
engine is broken, and a reward curve that looks fine cannot be trusted.

## After training

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp export my-grpo --checkpoint checkpoints/step_500 --checkpoint-format nemo-dcp
tp eval my-grpo --model /shared/hf_export/my-model --eval-config configs/eval/gsm8k.yaml
tp bench run bench-my-grpo -m run:run-4f2a91 --suites gsm8k,mmlu
```

Formats here are `nemo-dcp` and `nemo-megatron`. `--then export --then eval` on submit runs the
first two automatically once the job succeeds.

## Next

[Pipelines](https://docs.tuneplane.ai/en/guides/pipelines.md) · [Benchmarks](https://docs.tuneplane.ai/en/guides/benchmarks.md) ·
[Agent environments](https://docs.tuneplane.ai/en/guides/agent-envs.md) for GRPO and PPO against tool environments


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.