> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat with a checkpoint

> Start a vLLM service from a training artifact and talk to it in the browser.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp serve start run:run-4f2a91 --gpus 1
tp serve ls
```

Starts an OpenAI-compatible vLLM server from that run's export and prints its endpoint and
token. Then open the console's Playground page and chat.

A Playground Session is for a person to try a model. For application traffic, use a
[Model Deployment](https://docs.tuneplane.ai/en/guides/deployments.md) instead.

## Start one

**CLI**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp serve start run:run-4f2a91 --gpus 1        # a training artifact
tp serve start Qwen/Qwen3.5-9B --gpus 2       # any hub model
tp serve start /shared/models/my-model        # a path on shared storage
```

`--gpus` is the tensor-parallel size.

**Console**

Go to **Playground**, pick a model from the artifact dropdown or type a hub id, choose a GPU
count, and start. The page shows status, remaining TTL and an extend button.

## Manage it

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp serve ls                              # your services, with endpoints and remaining TTL
tp serve extend run-4f2a91 --hours 4     # keep it alive longer
tp serve stop run-4f2a91                 # stop it and release the GPUs
```

> [!WARNING]
>
> A session holds real GPUs and counts against your quota. It has an idle TTL, set by the
> deployment, and stops on its own when that expires. That is the protection against starting one
> and forgetting it, not a fault. Extend it if you still need it.

## Compare two models side by side

The Playground sends one prompt to two services at once, which is the fastest way to see what an
alignment stage actually changed. Start two sessions — the SFT artifact and the GRPO artifact —
and pick both in the A/B selector.

Chat controls are temperature, max tokens and a system prompt. The endpoints are
OpenAI-compatible, so anything that speaks that API can use them:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl https://tuneplane.your-company.com/inference/<id>/v1/chat/completions \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"model": "default", "messages": [{"role": "user", "content": "Hello"}]}'
```

## Which backends can serve

| Fleet kind | Session form |
| - | - |
| `kubernetes` | A standalone vLLM Pod, not a RayJob, reached through the Kubernetes API proxy |
| `node` | A vLLM container on a registered Node, with a reserved port reachable from the control plane |
| `slurm` | Not supported: compute nodes usually have no inbound path from the console |

## Check it worked

`tp serve ls` shows the session with an endpoint and a TTL. If it stays pending, the weights are
still loading — a 9B model takes a minute or two on a cold cache.

The Playground is a premium feature an administrator enables per role or user. When it is not
enabled, the page explains that rather than erroring.

## Next

[Model deployments](https://docs.tuneplane.ai/en/guides/deployments.md) for application traffic ·
[Model registry](https://docs.tuneplane.ai/en/guides/model-registry.md) · [The Playground page](https://docs.tuneplane.ai/en/console/playground.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.