> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy a model version

> Put a model behind a stable internal address, with revisions you can promote and roll back.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl -X POST https://tuneplane.your-company.com/api/model-deployments \
  -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "support-agent",
    "model_source": {"kind": "model", "model": "alice/support-agent"},
    "config": {"engine": "vllm", "gpus": 2, "max_model_len": 8192},
    "fleet_id": "fleet-gpu-a",
    "serve_mode": "proxied"
  }'
```

You get a Model Deployment: a stable address that application traffic can call, serving whatever
revision is currently promoted.

Most people create one from the console's [Deployments page](https://docs.tuneplane.ai/en/console/deployments.md) instead.
The API is here because a deployment is the thing you most often want in a script.

## A Deployment is not a Playground Session

| | Playground Session | Model Deployment |
| - | - | - |
| For | A person trying a checkpoint | Application traffic |
| Lifetime | An idle TTL, then it stops | Until you suspend or delete it |
| Address | Issued per session | Stable, and outlives every revision |
| Auth | Your own login | A revocable Deployment Token |
| Changing the model | Start another session | Create a revision and promote it |

## Where the model comes from

`model_source.kind` is one of:

| Kind | Fields | Use it for |
| - | - | - |
| `model` | `model` as `<owner>/<name>`, optional `version` | The normal case. Omit the version to follow whatever is promoted |
| `artifact` | `run_id`, optional `step` | One run's export directly |
| `huggingface` | `repository`, `revision`, `use_linked_credential` | A hub model |
| `modelscope` | `repository`, `revision` | A public ModelScope repository |
| `shared_path` | a path | Weights already on shared storage |

`model` with no version is what makes `tp model promote` the only step needed to change what
is being served. See [the model registry](https://docs.tuneplane.ai/en/guides/model-registry.md).

Both are pinned on the way in. An `artifact` source records which exported step it took, and a
`model` source with no version resolves production once and stores the number it got. A revision
that loads different weights after a restart is not a revision, and re-following a moving
pointer on every launch is how that happened.

## Serving configuration

```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
{
  "engine": "vllm",
  "gpus": 4,
  "dtype": "bfloat16",
  "quantization": "none",
  "max_model_len": 8192,
  "gpu_memory_utilization": 0.9,
  "served_model_name": "support-agent",
  "tensor_parallel_size": 2,
  "data_parallel_size": 2
}
```

| Field | Default | Notes |
| - | - | - |
| `engine` | `vllm` | Or `sglang`. Different engines, not a backend choice |
| `gpus` | `1` | How many cards the deployment holds |
| `dtype` | `auto` | `float16`, `bfloat16` |
| `quantization` | `none` | `awq`, `gptq`, `bitsandbytes`, `fp8` |
| `max_model_len` | engine default | Context length |
| `gpu_memory_utilization` | `0.9` | |
| `max_concurrency` | unset | |
| `served_model_name` | `model` | The `model` field callers send |
| `tensor_parallel_size` | all cards | |
| `pipeline_parallel_size` | `1` | For a model that does not shard cleanly |
| `data_parallel_size` | `1` | For replicas of a small model across cards |

`tensor_parallel_size × pipeline_parallel_size × data_parallel_size` must equal `gpus`. Anything
else is a container that either cannot start or quietly uses fewer cards than it was charged
for, and the second is worse because nothing looks wrong.

For a knob the schema does not model, `extra_args` is passed to the engine verbatim after the
platform's own flags, and `env` sets the engine process environment — `VLLM_*`, `NCCL_*`,
`HF_HOME`. Flags the platform has to be right about, such as `--port` and `--model`, are
reserved and refused.

Credentials do not belong in `env`. A private model source takes a governed Model Credential,
which is never echoed back on a read.

## Proxied or direct

`serve_mode` decides whether the console is in the data path.

| Mode | Traffic goes | Reflow capture |
| - | - | - |
| `proxied` (default) | Through the console to the engine | Yes |
| `direct` | Straight to the engine | **No** |

A `direct` deployment captures nothing, so there is no [Reflow](https://docs.tuneplane.ai/en/guides/reflow.md) buffer to mine
later. Pick it for latency, knowing that.

## Serving you already run elsewhere

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl -X POST https://tuneplane.your-company.com/api/model-deployments/external \
  -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \
  -d '{"name": "newapi-prod", "project_key": "@alice/rtl-assistant"}'
```

An **external** deployment is an engine somebody else runs: a gateway such as New API in front
of your own serving stack. Nothing is launched, so there is no model, Fleet, cards or release
check to give. What it creates is the return path — an identity to own a Reflow buffer and a
Deployment Token — so that traffic the platform never sees can still come back as training
data. It has no revisions, playground, logs or metrics; its status is `External`, never `Ready`,
because the platform checked nothing. See [reporting traffic](https://docs.tuneplane.ai/en/guides/reflow.md).

## Which Fleet serves it

`fleet_id` is yours to choose and the platform never picks for you. A deployment is long-lived
and its address outlives the decision, and Fleets differ on things the platform cannot rank:
which department owns the machines, which data may touch them.

## Call it

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl -X POST https://tuneplane.your-company.com/api/model-deployments/<id>/tokens \
  -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \
  -d '{"name": "support-service"}'
```

That returns the token once. Then application traffic uses the OpenAI-compatible endpoint:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl https://tuneplane.your-company.com/inference/<id>/v1/chat/completions \
  -H "Authorization: Bearer <deployment token>" \
  -H "Content-Type: application/json" \
  -d '{"model": "support-agent", "messages": [{"role": "user", "content": "Hello"}]}'
```

A Deployment Token authorises traffic to one deployment and is revocable on its own, independent
of anyone's login session. Revoke one with
`DELETE /api/model-deployments/<id>/tokens/<token_id>`.

## Change what it serves

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
# a new revision, promoted when it reports ready
curl -X POST .../api/model-deployments/<id>/revisions \
  -d '{"model_source": {...}, "config": {...}, "allow_downtime": false}'

# back to a known-good one
curl -X POST .../api/model-deployments/<id>/rollback \
  -d '{"revision_id": "rev-7", "allow_downtime": false}'
```

A revision is an immutable snapshot of the model source and the serving configuration together.
Promotion switches the endpoint to a revision that is already ready, so the address never points
at something still starting. `allow_downtime: true` skips waiting, for a deployment whose cards
cannot hold both revisions at once.

`suspend` releases the serving capacity and keeps the identity, the revisions, the address and
the tokens. It is not deletion.

## Check it worked

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \
  https://tuneplane.your-company.com/api/model-deployments/<id>
```

`status` reads ready and `current_revision.internal_endpoint` is filled. The console's
Deployments page shows the same thing, plus `logs` and `metrics` for the engine process.

`desired_state` is what you asked for and `status` is what is observed. They differ while a
revision starts, and a difference that persists has its reason in `last_error`.

## Next

[Model registry](https://docs.tuneplane.ai/en/guides/model-registry.md) · [Reflow](https://docs.tuneplane.ai/en/guides/reflow.md) ·
[The Deployments page](https://docs.tuneplane.ai/en/console/deployments.md) · [Inference API](https://docs.tuneplane.ai/en/api-reference/inference.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.