> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Adopt a new upstream version

> What to do when NeMo-RL, verl or TRL ships a release you want.

Every published version is one entry in a recipe's version matrix plus one artifact in the
runtime registry. Adopting a new upstream release is declarative: in most cases nothing in the
platform's own code changes.

## The steps

**1. Build the execution artifact**

Build the training image for the new version (K8s/Docker: OCI digest-pinned) or SIF/SQSH (Slurm), publish it to the runtime registry, and assign a `runtime_id`.

**2. Declare the version variant**

Add a new version entry under `runtime.versions` in the recipe YAML: `runtime_id`, dependencies, and the **version-bound differences** — entrypoint override (`entrypoint`), parameter path overrides (`path_overrides`), and certified observer patch versions (`observer_versions`).

**3. Run adaptation tests**

SDK unit tests cover the version matrix (entrypoint / parameter paths / observers asserted per version); full regression via `tests/`.

**4. Canary release**

Do not make the new version `default_version` at first: early adopters opt in explicitly with `tp new --framework-version X` / `tp recipe upgrade --framework-version X`. Switch the default once it is stable.

**5. User upgrade**

After the catalog is published, existing experiment locks show drift hints; users upgrade explicitly with `tp recipe upgrade` — there is no silent switching.

## Framework-specific notes

| Framework | Common differences |
| - | - |
| **NeMo-RL** | Official YAML base updates (sync `configs/base/`), Megatron/DTensor config key migrations |
| **verl** | Training entrypoint consolidation (e.g. the unified main\_ppo in 0.9), Ray version requirements |
| **TRL** | Trainer API signature changes, Accelerate config default changes |

## What not to do

Three things that look like shortcuts and are not:

| Do not | Because |
| - | - |
| Write `if version >= X` branches in adapter code | Version differences belong in the recipe declaration, where a reader can see them |
| Use `latest` or a branch name as a version | Only exact semver or a full commit is accepted. A moving version makes two runs of "the same" recipe incomparable |
| Point an existing `runtime_id` at a new image | Artifacts are immutable. New image, new id — otherwise a job that pinned the old digest cannot be reproduced |

## Check it worked

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp methods verl/grpo
```

The new framework version appears in the method's version list. Then create an experiment
against it explicitly and submit once:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp new canary-verl --method verl/grpo --framework-version 0.9.1
tp submit canary-verl --profile h200:8
```

A job that reaches `RUNNING` and reports its first metric means the image, the entrypoint and the
observer all line up.

## Next

[Method catalogue](https://docs.tuneplane.ai/en/guides/methods.md) · [Recipe packs](https://docs.tuneplane.ai/en/extend/recipe-packs.md) ·
[Maintenance mode](https://docs.tuneplane.ai/en/ops/maintenance.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.