tp serve

tp serve start run:run-4f2a91 --gpus 1
tp serve ls

Starts an OpenAI-compatible vLLM server and prints its endpoint and token. The walkthrough is chatting with a checkpoint.

Worked examples

tp serve start run:run-4f2a91 --gpus 1          # a training artifact's export
tp serve start Qwen/Qwen3.5-9B --gpus 2         # a hub model; --gpus is the tensor-parallel size
tp serve start /shared/models/my-model          # a path on shared storage
tp serve start run:run-4f2a91 --ttl-hours 8     # a longer idle TTL than the server default

tp serve ls                                     # status, endpoint, remaining TTL
tp serve extend run-4f2a91 --hours 4            # keep it alive longer
tp serve stop run-4f2a91                        # stop it and release the cards

extend and stop take the run id, which is what tp serve ls prints. --ttl-hours is on start; extend takes --hours.

A session holds real GPUs and counts against your quota. It stops on its own when its idle TTL expires. Extend it if you need longer, and stop it as soon as you are done.

Check it worked

tp serve ls shows an endpoint and a TTL. That address is OpenAI-compatible, so it goes straight into any OpenAI SDK client with the token beside it.

`tp serve`

Manage inference services

tp serve <subcommand>

Command	Description
`tp serve extend`	Extend an inference service’s TTL
`tp serve ls`	List my inference services
`tp serve start`	Start an inference service
`tp serve stop`	Stop an inference service

`tp serve extend`

Extend an inference service’s TTL

tp serve extend <RUN_ID> [options]

Argument	Description
`RUN_ID`	required

Option	Description
`--hours` `<float>`	How much longer, in hours (default `1.0`)

`tp serve ls`

List my inference services

tp serve ls

No options beyond --help.

`tp serve start`

Start an inference service

tp serve start <MODEL> [options]

Argument	Description
`MODEL`	An HF id / an absolute path on shared storage / run:<run_id> (required)

Option	Description
`--gpus`, `-g` `<int>`	How many cards (the tensor-parallel size) (default `1`)
`--ttl-hours` `<float>`	Idle TTL in hours (default: the server setting)

`tp serve stop`

Stop an inference service

tp serve stop <RUN_ID>

Argument	Description
`RUN_ID`	required

No options beyond --help.

CLI reference

Set up

Experiments

Train

Artifacts and evaluation

Resources

Administration

Worked examples

Check it worked

`tp serve`

`tp serve extend`

`tp serve ls`

`tp serve start`

`tp serve stop`

​Worked examples

​Check it worked

​tp serve

​tp serve extend

​tp serve ls

​tp serve start

​tp serve stop

Worked examples

Check it worked

`tp serve`

`tp serve extend`

`tp serve ls`

`tp serve start`

`tp serve stop`