> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Backup and restore

> What actually has to survive, and what you can afford to lose.

Three tiers, and knowing which tier a thing is in decides how much effort it deserves.

## Tier 1 — irreplaceable

| What | Where | Notes |
| - | - | - |
| The database | PostgreSQL | Every job record, quota, rubric, model version, audit entry. Losing it loses the platform's memory, not its files |
| `<root>/runs/**/out` | Storage root | Checkpoints, exports, evaluation reports. Nothing reconstructs these |
| `<root>/state` | Storage root | Control-plane bookkeeping |
| Object storage | Your S3 bucket | Datasets, job packages, archives. Back it up as a bucket, not through the platform |
| `TUNEPLANE_WEB_JWT_SECRET` and `TUNEPLANE_SECRET_ENC_KEY` | Your secret store | Losing the encryption key makes every stored integration credential unreadable |

## Tier 2 — expensive to lose, reconstructible

| What | Cost of losing it |
| - | - |
| `<root>/cache/hf` | Every job re-downloads its weights. On an internal link that can cost more than the training |
| `<root>/cache/datasets` | Re-pulled from object storage on next use |
| `<root>/runs/**/logs` | Gone for good once a job is terminal, but rarely load-bearing after a run is understood |

## Tier 3 — do not bother

`<root>/cache/volumes`, `<root>/packages` (when object storage is on), and `<root>/runs/**/work`
for finished runs. All reconstructible, all reclaimed under disk pressure anyway.

## Backing up the database

**Postgres**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pg_dump --format=custom --file=tuneplane-$(date +%F).dump "${TUNEPLANE_DB_URL/postgresql+psycopg:/postgresql:}"
```

Schema migrations run automatically at startup, so a dump restored into a newer build will be
migrated forward. Restoring a dump into an *older* build is not supported — keep a note of the
version each dump came from.

## Restoring

**1. Stop the console**

Or put it in maintenance mode first and let jobs drain — see [maintenance](https://docs.tuneplane.ai/en/ops/maintenance.md).

**2. Restore the database and the storage root**

They have to be consistent with each other. A database that references a run directory which was
restored from an older snapshot will show artifacts that are not there.

**3. Restore the secrets**

Especially `TUNEPLANE_SECRET_ENC_KEY`. Without the original, stored integration credentials decrypt to
nothing and every user has to re-link.

**4. Start, and check**

`GET /api/version` answers, the job list renders, and one finished run still shows its artifacts.

> [!WARNING]
>
> Test the restore. A backup nobody has restored is a hypothesis, and the failure mode — a missing
> encryption key discovered during an incident — is the expensive kind.

## What is not in scope

Training data that lives outside the platform, and container images. Both are somebody else's
backup: the source of truth for a dataset you pushed is the object store, and for an image it is your
registry.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.