Bring your own trainer

tp new my-custom --method custom/custom
# write experiments/my-custom/train.sh
tp submit my-custom --profile h200:8 \
  --image registry.example.com/my-trainer:v1 \
  --observability-url https://wandb.example/my-proj

custom/custom runs exactly one file: experiments/<name>/train.sh. The platform does not read a FRAMEWORK variable, does not look for a run.py, and never falls back to custom because another adapter failed. What that file does is yours.

Do not go custom to change a learning rate. The catalogue methods already wire metrics, checkpoints and images, and custom hands all three back to you.For a supervised PyTorch loop over labelled data, pytorch/image-classification gives you the same freedom inside train.py plus multi-node torchrun, checked hyperparameters and a platform image.

Scaffold

tp new my-custom --method custom/custom

You get train.sh, plus config.yaml, recipe.lock.json and README.md. The recipe’s entrypoint is train.sh by name, so renaming it fails submission with custom entrypoint is missing or escapes the experiment directory. The adapter runs bash <absolute path to train.sh>. The working directory is the job package root, not the experiment folder, so point Python at "${TUNEPLANE_EXP_DIR}/train.py" rather than a relative path. The custom adapter only compiles the training operation. tp export and tp eval against a custom experiment are refused.

The environment contract

The scaffold dies with :? if any of these is missing. Do not rename them.

Variable	What it is
`TUNEPLANE_WORK_DIR`	The unpacked job package root
`TUNEPLANE_EXP_DIR`	This experiment’s directory
`TUNEPLANE_OUT_DIR`	The output directory, already created and isolated per run. The control plane resolves it; never compose it yourself
`TUNEPLANE_FRAMEWORK`, `TUNEPLANE_RECIPE`	Both `custom`
`TUNEPLANE_CLUSTER_NUM_NODES`	Nodes you actually have
`TUNEPLANE_CLUSTER_GPUS_PER_NODE`	GPUs per node you actually have
`TUNEPLANE_JOB_ENDPOINT`, `_RUN_ID`, `_TOKEN`	Ingest credentials. Absent outside a job
`TUNEPLANE_JOB_ENABLED`	`1` when ingest is bound, else `0`

HF_HOME, HF_TOKEN and the *_DATA_DIR variables arrive too, when the deployment has them. The full list is in the job environment reference. Quota and the watchdog are measured against TUNEPLANE_CLUSTER_*. Occupying more GPUs than that gets the job warned and can get it stopped, so pass the same numbers into accelerate launch --num_processes or torchrun --nproc_per_node.

Three things your script must do

Write everything you want to keep under $TUNEPLANE_OUT_DIR. The scratch tree disappears when the container exits.
Report scalars with tuneplane.report, or declare native TensorBoard/JSONL outputs for the independent collector. Stdout alone does not produce metric points.
Respect TUNEPLANE_CLUSTER_*.

The platform looks for these paths under the output directory. They are realpath-checked and must stay inside it:

Path	For
`checkpoints/*`	Weights
`logs`	Text logs you write yourself. Optional — stdout already streams
`hf_export`	A HuggingFace export directory
`eval/report.json`	An evaluation report you write by hand

Write train.sh

set -euo pipefail is already there. Replace the scaffold’s exit 1 with a launch.

exec python "${TUNEPLANE_EXP_DIR}/train.py" \
  --output-dir "${TUNEPLANE_OUT_DIR}" \
  --config "${TUNEPLANE_EXP_DIR}/config.yaml"

exec accelerate launch \
  --num_processes "${TUNEPLANE_CLUSTER_GPUS_PER_NODE}" \
  "${TUNEPLANE_EXP_DIR}/train.py" \
  --output_dir "${TUNEPLANE_OUT_DIR}" \
  --config "${TUNEPLANE_EXP_DIR}/config.yaml"

exec torchrun \
  --nproc_per_node "${TUNEPLANE_CLUSTER_GPUS_PER_NODE}" \
  "${TUNEPLANE_EXP_DIR}/train.py" \
  --output-dir "${TUNEPLANE_OUT_DIR}"

Do not cd somewhere and write ./checkpoints. Use the variables.

Report metrics

The PyPI name is tuneplane and the import is tuneplane. The reporting module pulls in neither transformers nor Ray, and it never raises into your training loop. With no TUNEPLANE_JOB_TOKEN it does nothing, which is what you want outside a job.

from tuneplane.report import init, log, finish

init(hparams={"lr": 1e-5, "batch_size": 8})

for step, batch in enumerate(loader):
    loss = train_one(batch)
    log({"train/loss": float(loss)}, step=step)

finish()

from tuneplane.report import TunePlaneCallback
from trl import SFTTrainer

trainer = SFTTrainer(..., callbacks=[TunePlaneCallback()])
trainer.train()

from tuneplane.report import log

log({"env/tool_success_rate": 0.83, "env/avg_turns": 3.2}, step=step)

init() is idempotent. Nested dicts are flattened, and a non-scalar that cannot be reduced to a mean is dropped. prefix= prepends a namespace when the key does not already carry one. TunePlaneCallback is duck-typed rather than a TrainerCallback subclass. It calls init on train begin, log on on_log, log(..., prefix="validation") on evaluate, and finish at the end. Hardware sampling starts with init(monitor_hardware=True), on rank zero only — one sampler per job, not one per GPU. The interval is TUNEPLANE_JOB_MONITOR_INTERVAL, in seconds, default 10. Two things not to do: do not POST to the ingest API yourself, because stdout is already forwarded and a second path duplicates lines; and do not hard-code the console URL, because the container already has TUNEPLANE_JOB_ENDPOINT.

Why —observability-url is required

The published custom/custom recipe declares observability: external. Two consequences:

Submission requires --observability-url. Without it, compile fails with custom external observability requires spec.framework.observability_url. Any URL your team uses works; the platform does not start wandb for you. It reaches the job as TUNEPLANE_EXTERNAL_OBSERVABILITY_URL.
The adapter does not rewrite PYTHONPATH. import tuneplane works only if the wheel is in your image, or you put it on PYTHONPATH yourself.

A recipe declaring observability: platform behaves the other way round: --observability-url is refused, and the runner prepends the capsule roots to PYTHONPATH so from tuneplane.report import log works with no wheel in the image. You cannot flip that from your experiment directory — it is a property of the published recipe. For SDK-based charts with this recipe, install tuneplane in your image and still pass --observability-url. Native collection is a separate output declaration and deployment path. TUNEPLANE_JOB_ENABLED=1 is separate. That is the ingest binding the server always sets when the job has an ingest token.

Submit

git add -A && git commit -m "custom trainer"
tp submit my-custom --profile h200:8 \
  --image registry.example.com/my-trainer:v1 \
  --observability-url https://wandb.example/my-proj

--image is the image: custom has no catalogue pin, since its runtime.default_version is user-managed. Tags work; pin @sha256:… in production so a moving tag cannot change the job after admission. An empty TUNEPLANE_ALLOWED_IMAGE_REGISTRIES refuses custom user images, while catalogue frameworks can still use the deployment’s defaults. Ask an administrator to add your registry hostname. See custom images.

Check it worked

tp job logs shows your own launch line, then your trainer’s output. The Charts tab gets its first point after your first log() call. Inside the container:

python -c "import tuneplane.report; print('ok')"
echo "$TUNEPLANE_OUT_DIR" "$TUNEPLANE_CLUSTER_GPUS_PER_NODE"
env | grep TUNEPLANE_JOB

When it does not work

Symptom	Cause
Fails in seconds, log says `no training command yet`	The scaffold `train.sh` was never edited
Logs flow, charts stay empty	No `log()` or callback, or `import tuneplane` failed, or weights are still downloading
`ModuleNotFoundError: tuneplane`	No wheel in the image, and the recipe is `external` so `PYTHONPATH` was not patched
Imports work locally, not in the job	`train.sh` execs a different Python than you tested. Align `PATH` with the image’s venv
Charts and logs both empty	The container never started. Check the state and the Events tab
Checkpoints vanished	Written to the working directory or `/tmp`. Use `$TUNEPLANE_OUT_DIR/checkpoints`
An allowlist error	The registry host of `--image` is not in `TUNEPLANE_ALLOWED_IMAGE_REGISTRIES`
`custom adapter` refuses `export`	Custom has no export or eval compile path

config.yaml is yours

Nothing in the custom adapter reads it. The custom recipe declares no parameters, so tp validate has nothing to check and --set values land in spec.hyperparams without reaching your argv. Parse the file in train.py. --model and --train-data are the verl and TRL bindings. Custom ignores them: use the environment and the files you packed.

Custom images · Job environment · Reporting API

Get started

Core concepts

Train a model

Evaluate and serve

Agent environments

Extend the platform

Collaborate

Integrations

Reference

Help

Bring your own trainer

Scaffold

The environment contract

Three things your script must do

Write train.sh

Report metrics

Why —observability-url is required

Submit

Check it worked

When it does not work

config.yaml is yours

Next

​Scaffold

​The environment contract

​Three things your script must do

​Write train.sh

​Report metrics

​Why —observability-url is required

​Submit

​Check it worked

​When it does not work

​config.yaml is yours

​Next

Scaffold

The environment contract

Three things your script must do

Write train.sh

Report metrics

Why —observability-url is required

Submit

Check it worked

When it does not work

config.yaml is yours

Next