Skip to main content
The Rust collector reads declared TensorBoard or JSONL sources without importing the training framework. It sends bounded batches to the telemetry listener of the gateway, a Python process that also serves Node control on a second listener. The two listeners have separate RPC capacity and separate database pools. It is not an LLM inference proxy.

​
Deploy the listener

The gateway needs the migrated PostgreSQL database, the same signing configuration as the control plane, and the object-store configuration used for rich records and artifacts. With the telemetry target configured below, startup creates or restores the shared TLS identity from PostgreSQL. All replicas need the same TUNEPLANE_SECRET_ENC_KEY and TUNEPLANE_WEB_JWT_SECRET. The certificate covers the configured gateway hostname.
For an externally managed certificate, pass --certificate and --private-key and configure TUNEPLANE_TELEMETRY_TRUST_BUNDLE with its public CA chain. Compose supplies the gateway profile. Helm supplies gateway.enabled, gateway.tlsSecret and ports 8444 (telemetry) and 8443 (Node control). Expose gRPC with a TLS-preserving TCP route and verify reachability from execution hosts. Collectors use dedicated workload tokens bound to run, attempt and member, over server-authenticated TLS. Node control tokens and user login tokens cannot substitute for them. The platform provisions and renews these credentials; do not paste them into training code or experiment configuration. No collector client certificate is required by the normal deployment path.

​
Install the collector artifact

Build for the execution hosts’ operating system and architecture with the pinned Rust 1.91.0 toolchain and installed target/native toolchain. Linux production artifacts use musl. For example:
The command prints a content-addressed directory containing manifest.json and tuneplane-collector. Deploy that complete directory where the control plane and launch workers can read it. Configure the managed telemetry profile together:
With automatic gateway TLS enabled and neither gateway trust-bundle override set, startup supplies the public CA and each launch distributes it to the collector. Set TUNEPLANE_TELEMETRY_TRUST_BUNDLE only for an externally managed certificate. TUNEPLANE_TELEMETRY_SPOOL_BYTES defaults to 256 MiB and TUNEPLANE_TELEMETRY_DRAIN_SECONDS to 30 seconds; set them only to override these limits. Node enrollment configures the Node control connection; it does not replace the three telemetry settings above. Use the corresponding ARM target for ARM hosts. The launch path verifies the artifact manifest and digest; a binary for the console host is not necessarily suitable for the training host. The official TensorBoard Python library is used for conformance testing, not installed into the training environment by the collector.

​
Provision local durable storage

The spool root must be private, durable host-local storage on every execution machine, not NFS or another shared network filesystem. Source logs and output artifacts can still use shared storage. Configure backend mounts explicitly; a missing or unsuitable spool grant must not silently fall back to shared output storage. For Node Fleets, provision the telemetry spool root on each compute machine. The control plane does not mount compute-host spool directories. Kubernetes and Slurm also require their backend-specific local-storage grants. Object delivery does not make a shared filesystem suitable for a SQLite spool. See storage and Node execution.

​
Select sources and inspect results

The integration artifact declares sources, ownership and metric mappings. Assign one reader to each shared source. Do not assume every framework writes on every rank, discard nonzero-rank sources blindly, or deduplicate equal values from different members. Preserve their source identities; publish an aggregate from its designated owner. TensorBoard scalars and structured JSONL records are distinct inputs. Large TensorBoard images and histograms are not implicitly streamed as scalar data. Rich JSONL channels have their own schemas and budgets. W&B can coexist as an experiment tool; its proprietary local history is not a supported source merely because TensorBoard reading is enabled. Open the Job’s Charts tab. Canonical curves use the frozen mapping; the raw scalar inspector lets you choose attempt and source. Queries bound returned points and series. A reduced plot is not a claim that all raw records were returned. Unknown metric semantics stay explicit rather than being guessed from a familiar tag name.

​
Failure and recovery checks

A lost ACK replays the same durable batch identity. Disk loss, exhausted source retention and a destroyed spool are different failures from a process restart and do not inherit its recovery guarantee. Verify the actual storage and backend profile before claiming multi-machine durability. The existing HTTP/SDK reporting path still serves supported lifecycle, log and reporting use cases. Native telemetry is not configured merely by setting TUNEPLANE_INGEST_URL.