Choose a method

tp methods                  # everything this deployment offers
tp methods nemo-rl/grpo     # one method's tunable parameters, with ranges

tp methods is authoritative for what your server has published. The tables below are a map, not a version pin. The console’s Methods page shows the same catalogue with the parameters laid out, which is easier to read before a first submission. A method id is <framework>/<method>.

What a recipe pins

tp new writes recipe.lock.json beside your config:

Field	Meaning
`recipe.name`, `recipe.version`	The method and the recipe version
`recipe.digest`	A digest of the manifest and its templates
`framework.kind`, `framework.version`	The exact framework build, such as `verl@0.9.0`
`framework.runtime_id`	Which deployment artifact runs it: an OCI image, a SIF, a SQSH
`requires.core`	The `tuneplane` versions this recipe works with

At submit the CLI handshakes with the server’s catalogue, and the lock has to match what the server published. When the platform publishes a newer recipe, an old lock is refused rather than silently upgraded:

tp recipe status my-grpo                        # what changed
tp recipe upgrade my-grpo                       # move to the current catalogue
tp recipe upgrade my-grpo --framework-version 0.9.0
tp submit my-grpo --profile h200:8 --upgrade-recipe   # both at once

One recipe can publish several framework versions. Entrypoint changes and moved parameter paths are declared on the version, not branched in code, so adopting a new upstream release is usually a manifest and an image:

tp new my-verl --method verl/grpo --framework-version 0.9.0

NeMo-RL

The default framework for a full post-training pipeline. Catalogue release nemo-rl@0.7.0.

Method	What it is	When
`nemo-rl/sft`	Supervised instruction fine-tuning	First stage; its output feeds DPO or GRPO
`nemo-rl/dpo`	Direct preference optimisation	Pairwise labels; cheaper than GRPO
`nemo-rl/grpo`	Group relative policy optimisation	Math and reasoning RL, no critic
`nemo-rl/grpo-lora`	GRPO with LoRA	Tight GPU memory, or a larger base on one node. LoRA learning rate runs one to two orders of magnitude higher than full fine-tuning
`nemo-rl/ppo`	PPO with a critic and GAE	Dense rewards, or a distorted in-group baseline
`nemo-rl/rm`	Bradley-Terry reward model	An RL reward source, or calibrating an LLM judge
`nemo-rl/distillation`	On-policy distillation	The student samples, the teacher supplies full-distribution KL
`nemo-rl/maxrl`	Maximum-likelihood RL	Puts weight on low-pass-rate hard problems
`nemo-rl/opsd`	Self-distillation	Teacher is the same model plus a reference solution

verl

ByteDance verl, synchronous. Catalogue releases verl@0.9.0 (the default) and verl@0.9.1. Every RL method here runs the same upstream entry point, verl.trainer.main_ppo. What separates them is the advantage estimator the recipe pins. The estimator is not a hyperparameter: setting it in config.yaml is refused at submission, because a config that disagrees with the method it claims to be is worse than an error.

Method	What it is
`verl/grpo`	Synchronous GRPO
`verl/dapo`	GRPO plus the four DAPO techniques, on by default: clip-higher, dynamic sampling, token-level loss, overlong reward shaping
`verl/ppo`	GAE PPO with a critic. For dense or process rewards
`verl/rloo`	Leave-one-out baseline: unbiased, and no division by the in-group standard deviation
`verl/reinforce-baseline`	REINFORCE++-baseline: group mean, then global-batch whitening. The same estimator as `openrlhf/reinforce-baseline`
`verl/remax`	Baseline from the model’s own greedy answer. One sample per prompt, at the cost of an extra greedy pass
`verl/distillation`	On-policy distillation. Needs a second Resource Pool for the teacher
`verl/sft`	The upstream SFT trainer, launched with explicit torchrun argv

Loss variants are parameters, not methods: GSPO, CISPO, GMPO, clip-cov, kl-cov and GPG are all policy_loss_mode on any RL recipe above. Rollout correction is the same — every verl RL method carries the rollout_is* parameters, because the rollout engine and the trainer are never quite the same policy.

TRL

Hugging Face TRL with Accelerate. Catalogue release trl@1.10.0.

Method	What it is
`trl/sft`	`SFTTrainer`
`trl/dpo`	`DPOTrainer`
`trl/grpo`	`GRPOTrainer`, online RL
`trl/grpo-judge`	GRPO whose reward is the platform judge scoring completions against a versioned rubric. For tasks with no rule that decides correctness
`trl/rm`	`RewardTrainer` on a sequence-classification head
`trl/kto`	Binary good/bad labels, no pairs
`trl/rloo`	Leave-one-out in-group baseline; same family as GRPO
`trl/opsd`	`DistillationTrainer`, on-policy. Needs a same-vocabulary teacher

trl/grpo-judge is the one place where training and evaluation share an asset: the same rubric scores a benchmark, so what you optimise and what you report cannot drift apart.

OpenRLHF

DeepSpeed ZeRO with a colocated vLLM hybrid engine. Catalogue releases openrlhf@0.11.0 (the default) and openrlhf@0.11.2. Images are built by the deployment from deploy/docker/Dockerfile.openrlhf, because upstream publishes no image with the package in it. An administrator sets TUNEPLANE_IMAGE_OPENRLHF or adds it to the runtime registry.

Method	What it is
`openrlhf/sft`	Upstream SFT: packing, multi-turn loss, continued pre-training
`openrlhf/dpo`	Upstream DPO, with IPO, cDPO and an NLL regulariser
`openrlhf/rm`	Bradley-Terry reward model, usable as `reward_pretrain` for its RL methods
`openrlhf/grpo`	Group-norm GRPO, no critic. `no_std_norm` is the Dr.GRPO setting
`openrlhf/ppo`	GAE PPO with a critic; actor, critic, reference and reward colocated
`openrlhf/reinforce-baseline`	REINFORCE++-baseline. The same estimator as `verl/reinforce-baseline`, so a run moves between stacks unchanged

PyTorch

A native PyTorch loop under torchrun. Catalogue release pytorch@2.14.0. Images are built by the deployment from deploy/docker/Dockerfile.pytorch.

Method	What it is
`pytorch/image-classification`	The only `supervised` method: labelled pairs trained from initialisation, with no base model. The model and input pipeline live in the experiment’s `train.py`

Evaluation and custom

Method	What it is	Where
`evalkit/benchmark`	lm-eval or evalscope; scores land on the dashboard	Benchmarks
`custom/custom`	Runs `train.sh`. You bring the image and call `tuneplane.report`	Custom training

custom/custom requires --image. The platform does not guess an entrypoint, and it never falls back to custom after another adapter fails.

Pick a starting point from the data you have

Per-framework walkthroughs: NeMo-RL · verl · TRL · OpenRLHF · PyTorch · your own trainer A common chain is SFT → DPO or GRPO → benchmark. See pipelines.

Get started

Core concepts

Train a model

Evaluate and serve

Agent environments

Extend the platform

Collaborate

Integrations

Reference

Help

What a recipe pins

NeMo-RL

verl

TRL

OpenRLHF

PyTorch

Evaluation and custom

Pick a starting point from the data you have

Next

​What a recipe pins

​NeMo-RL

​verl

​TRL

​OpenRLHF

​PyTorch

​Evaluation and custom

​Pick a starting point from the data you have

​Next

What a recipe pins

NeMo-RL

verl

TRL

OpenRLHF

PyTorch

Evaluation and custom

Pick a starting point from the data you have

Next