A values file worth starting from
values.yaml
Three values decide whether it works
ReadWriteMany
required
An RWO claim does not fail. Kubernetes gives each pod its own volume, so a multi-node job’s
checkpoint shards split across pods and the run cannot be resumed — with nothing in the logs to
say why. The chart defaults to RWX; the StorageClass has to actually support it.
url
required
Training pods POST their metrics to it. Use the in-cluster service name. An external ingress
address leaves the cluster and comes back in;
127.0.0.1 is the worker’s own loopback.string
required
Without
TUNEPLANE_WEB_JWT_SECRET each process signs with its own key: every restart logs everyone
out, and more than one replica cannot work at all. The chart’s install notes warn when neither
this nor secrets.create is set.Secrets
secrets.create: true renders a Secret from secrets.data — convenient on a test cluster, wrong in
production, because the values land in your release history.
Point secrets.existingSecret at a Secret from kubectl create secret, Sealed Secrets, or the
External Secrets Operator:
One thing the chart does for you
TUNEPLANE_K8S_STORAGE_PVC is filled in from the claim the chart creates, so the executor is always
told about the claim that actually exists. That pairing is easy to break by hand — the executor
checks a claim that is not the one mounted, reports healthy, and the mismatch only surfaces when a
multi-node run fails to resume.
Override it only when reusing a claim the chart does not own:
Mounting Volumes
TUNEPLANE_VOLUME_ROOT from mountPath.
The second half is the one to know about. Mounting the claim on its own does nothing: without
TUNEPLANE_VOLUME_ROOT the platform delivers a Volume by copying its contents into the working
tier on every run, and the claim you mounted is never read. Nothing fails, which is why that
configuration can sit there for months.
The mount path must not sit under storage.mountPath. The console refuses to start if it does,
because the storage root is bind-mounted writable into jobs and a Volume beneath it would be
reachable by a second, writable path.
corpus.* is the deprecated spelling of the same block, kept so a values file written before the
rename keeps working. It mounts the claim as before and deliberately does not fill
TUNEPLANE_VOLUME_ROOT, so upgrading the chart does not change how an existing deployment delivers
Volumes. Setting both blocks is refused rather than resolved.
Verify before you apply
Check it worked
TUNEPLANE_K8S_STORAGE_SHARED declaration
against the claim’s real access modes, which is the check that catches an RWO claim before a job
does.
The chart is verified with
helm lint and helm template on every change. Installing it against a
live cluster is not something CI here can do — no runner has one — so the first install on your
cluster is the first real test. Run the dry-run above first.Native collector gateway
Setgateway.enabled=true. The same gateway serves Node control on 8443 and
collector telemetry on 8444. Working storage must be enabled.
Set config.TUNEPLANE_TELEMETRY_GATEWAY_TARGET,
config.TUNEPLANE_TELEMETRY_COLLECTOR_ARTIFACT and
config.TUNEPLANE_TELEMETRY_SPOOL_ROOT using the
native telemetry profile.
Place the pinned artifact on the working PVC where console and worker processes
can read it. Automatic TLS supplies the public CA without a manual trust path.
For externally managed TLS, set gateway.tlsSecret to an existing Secret
(tls.crt, tls.key) and config.TUNEPLANE_TELEMETRY_TRUST_BUNDLE to a public CA
file readable by console and worker processes. No workload receives a gateway private key.
Training pods receive only their launch’s private grant directory as a read-only
PVC subpath and their isolated host-local spool directory as a hostPath mount.
Cluster policy must explicitly permit that configured host path. Use persistent
local disks and compatible workload UIDs. Shared training output remains on its
existing PVC; it is not the SQLite spool. A spool belongs to its original host:
rescheduling to another host does not recover its pending records. Preserve that
host’s directory for recovery and do not treat a newly created directory as replay
of the old one. This profile does not promise recovery after permanent node loss.
A bounded initialization container sets the attempt directory to mode 0700
and the platform grant file owner’s UID, so a matching non-root training image
can write it. It receives only that spool mount, runs as UID 0 with CHOWN and
FOWNER, and receives neither the grant PVC nor gateway credentials. Cluster
policy must permit this initialization step. Root training images remain compatible.