> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Mount a governed folder

> Give a job read-only access to material nobody is allowed to download.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp volume create acme/support-transcripts -d "Tier-2 tickets, redacted"
tp volume push ./transcripts --name acme/support-transcripts
tp submit my-sft --profile h200:8 --volume acme/support-transcripts
```

The job reads the files at `$VOLUMES_DIR/support-transcripts`, read-only. Nobody can download
them unless the owner turns that on.

## A Volume is not a dataset

| | Volume | Dataset |
| - | - | - |
| Versions | None. Its current contents are its contents | Immutable versions |
| Reference | `<owner>/<name>` | `<owner>/<name>[@version]` |
| Changes | Files added and removed in place | A new version each time |
| Use it for | Internal documents, reference PDFs, material that keeps growing | Training data a finished run has to be able to name |
| Download | Off until the owner turns it on | Available to anyone who can reference it |

A version suffix on a volume reference is refused when the spec is parsed. So is a malformed
id, and two volumes whose leaf names would collide at the same mount path.

## Create and fill one

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp volume create acme/support-transcripts -d "Tier-2 tickets, redacted"
tp volume create acme/refs --public                    # any signed-in user may reference it
tp volume push ./transcripts --name acme/support-transcripts
tp volume ls
```

`tp volume push` uploads everything under the directory, minus OS and Office noise. Raise
`--concurrency` on a fast link.

Removing files, or the whole volume:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp volume rm acme/support-transcripts -p old/2024 -p drafts
tp volume rm acme/support-transcripts --yes
```

## Point at storage you already have

A volume does not have to hold bytes the platform put there. If the material is already on
storage your organisation runs — a MinIO bucket, a NAS export — an administrator can register
it instead of copying it.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp volume ls
```

Registering is seconds; copying five terabytes is hours to days. The permission model is the
same either way: visibility, download and preview still belong to the owner, previews are
still watermarked and audited, and a job still mounts it read-only.

Three things work differently, and each is deliberate:

* **The platform never writes to it.** `tp volume push`, deleting files and emptying the
  volume are all refused. The material belongs to the storage it lives on; change it there.
* **Deleting the volume removes the registration**, not the files. The count that comes back
  says one object.
* **An administrator registers it.** Deciding what the platform reaches into is an
  administrative decision, and the connection it reads through names which paths may be
  referenced and which projects may reference them.

If the connection is deleted or narrowed later, the volume stops resolving and jobs that
reference it are refused at submit — not hours later, and never by reading an empty
directory.

Volumes can be read by preprocessing, cleaning, conversion, evaluation and training
jobs. The job-use permission applies to all of these workflows.

## Open a directory to users in the console

1. An administrator opens **System management → Storage connections** and adds
   an S3 connection or a filesystem directory already mounted on the host.
2. Set the allowed directories and projects, then test the connection. Connecting
   storage does not publish its contents.
3. On the connection row, choose **New volume**. Choose an allowed directory or
   enter a subdirectory, give the volume a name, and choose its visibility.
4. Enable previews if needed. Downloads remain off. A private volume is visible
   only to its owner and administrators; public visibility is still intersected
   with the connection's project restrictions.
5. Optionally set **Allowed projects** on the volume to narrow the connection scope.
   **Allow job use** is independent of previews and downloads. Turning it off
   blocks new submissions, but does not stop running jobs or revoke existing copies.
6. Users open **Data → Volumes** to browse and preview permitted material, then
   reference the volume in a job. **View volumes** on a connection filters this list.

Filesystem volumes omit symlinks and special files. The console needs filesystem
access to browse and preview them; the process preparing a job also needs access
when copying material to its working directory. A successful console probe does
not prove a compute node can reach the same directory.

Read-only access prevents changes to the source. It does not prevent job code
from copying data into logs, outputs or network requests. Previewed bytes can also
be saved. Stronger containment requires an execution environment that restricts
these exits; these volume settings alone do not provide it. Filesystem downloads,
when explicitly enabled, use an authenticated stream; the console buffers the file
in the browser before saving it, so use the API for large downloads.

## Reference it from a job

Declare it in `experiments/<experiment>/config.yaml`, which is the durable form.
Merge `volumes` into an existing `data` section instead of adding a duplicate YAML key.
Your script must also accept or remove this platform metadata before passing config
to a strict trainer constructor. For the TRL SFT recipe, use `--volume` below
instead of adding `data` to the file passed directly to `SFTConfig`:

```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
data:
  volumes:
    - acme/support-transcripts
    - bob/reference-pdfs
```

Or override for one submission:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp submit my-sft --profile h200:8 \
  --volume acme/support-transcripts --volume bob/reference-pdfs
```

Each one appears at `$VOLUMES_DIR/<leaf name>`. Read the path from the environment; never
hard-code it.

```python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os, pathlib

root = pathlib.Path(os.environ["VOLUMES_DIR"]) / "support-transcripts"
for path in root.rglob("*.txt"):
    ...
```

A job that cannot see a referenced volume is refused at submission. That is deliberate: an
empty directory would make a permission problem look like a modelling problem.

## Read files in a preprocessing script

For a volume named `acme/models`, the root inside the job is `$VOLUMES_DIR/models`,
**not** `$VOLUMES_DIR/acme/models` and not the administrator's NAS path. The platform
sets `VOLUMES_DIR` in the job environment; it is usually absent on your laptop.

In the console, open a subdirectory and choose **Use this directory**. The usage
panel generates configuration and Python examples for the directory you opened.
Declaring a volume makes its files available; your script decides which ones to read.

```python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os
from pathlib import Path

source = Path(os.environ["VOLUMES_DIR"]) / "documents" / "incoming"
output = Path(os.environ["TUNEPLANE_OUT_DIR"]) / "processed"
output.mkdir(parents=True, exist_ok=True)

for path in source.glob("*.txt"):
    text = path.read_text(encoding="utf-8").strip()
    (output / path.name).write_text(text, encoding="utf-8")
```

This example needs `acme/documents` attached to the job, with text files under
`incoming/`. Write results outside the read-only source. For a job submitted with
`--output-dataset`, write dataset files to `$TUNEPLANE_DATASET_OUT_DIR` instead.

## Train with a model stored in a volume

Suppose `acme/models` contains `my-model/config.json`, tokenizer files and all the
model weight shards. Attach `acme/models`, then resolve the model directory
**inside your training script**:

```python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os
from pathlib import Path
from transformers import AutoModelForCausalLM, AutoTokenizer

model_dir = Path(os.environ["VOLUMES_DIR"]) / "models" / "my-model"
tokenizer = AutoTokenizer.from_pretrained(str(model_dir), local_files_only=True)
model = AutoModelForCausalLM.from_pretrained(str(model_dir), local_files_only=True)
```

Pass the resulting model and tokenizer to your trainer. This example is for a
Transformers-compatible text-generation model. Install compatible libraries in
the job environment and use the appropriate loader for multimodal or other
architectures. A directory listing alone does not prove the checkpoint is complete
or that the runtime supports it. Save checkpoints to `$TUNEPLANE_OUT_DIR`, not into
the model directory.

### Use the TRL SFT recipe

The generated `experiments/<experiment>/train.py` passes `args.model` to
`SFTTrainer`. To accept a job-side environment path, add this after
`args = parser.parse_args()` in **your copy of that script**:

```python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os

args.model = os.path.expandvars(args.model)
```

Then submit with the volume and a single-quoted model path:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp submit my-sft --profile h200:1 \
  --volume acme/models \
  --model '${VOLUMES_DIR}/models/my-model' \
  --train-data data/train.jsonl \
  --validation-data data/validation.jsonl
```

Replace the experiment, hardware profile and data paths with yours. The single
quotes preserve the variable on your laptop; the added line expands it inside
the job. Without that script change, do not assume that `--model` or a framework's
YAML loader expands environment variables. Different recipes use different
configuration loaders. Attaching the volume alone never chooses a model or
configures a training dataset.

## Keep large model libraries scoped

Referencing a volume attaches its **whole directory**. Reading `models/my-model`
in a script does not select a partial mount. With a pull connection, the platform
synchronizes the volume to job storage before startup; an existing filesystem
view can avoid this copy when configured by the administrator.

For a library containing hundreds of gigabytes of unrelated models, register the
required model subdirectory as its own external volume. For example, a volume
`acme/my-model` rooted at `my-model/` is read at `$VOLUMES_DIR/my-model`. This also
lets the administrator grant access to that model without exposing its siblings.

## Which backends can mount one

| Fleet kind | Mounts a volume | How |
| - | - | - |
| `node` | Yes | One filesystem at the same path on every node |
| `kubernetes` | Only with `TUNEPLANE_K8S_VOLUME_PVC` | A read-only claim at `/volumes` |
| `slurm` | No | The platform does not control the image's bind list |

On a backend that cannot, submission is refused naming the setting, rather than starting a job
whose mount would be empty.

## Turn one file into something people can read

Two permissions, both the owner's, both off by default:

| Permission | What it allows |
| - | - |
| `allow_download` | Fetching the bytes back out. The only way material leaves by design |
| `allow_preview` | The console rendering one file at a time, inline. Every read is audited |

They are independent. Listing the structure — file names, sizes — is always available to anyone
who can reference the volume; content is not.

## Produce a dataset from a volume

Reading a volume and publishing the result is a **Processing Run**: your code, on the platform,
with provenance.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp submit my-prep --profile h200:1 \
  --volume acme/support-transcripts \
  --output-dataset acme/support-sft@v1
```

The job writes into `$TUNEPLANE_DATASET_OUT_DIR` and the platform publishes when the run
succeeds. The job never holds a registry credential, and the published dataset inherits the
classification of the material it read.

This is also how you clean a volume: read, treat, publish. The platform never rewrites what you
uploaded.

## Check it worked

`tp volume ls` shows the volume with its file count. In a running job:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
ls -la "$VOLUMES_DIR"
```

The mount is read-only, so a write there fails — that is the mount working, not a bug.

> **What a Volume does not protect against**
>
> If your code can read the plaintext, no storage control stops exfiltration. A script that can
> open a PDF can base64 it into its own stdout.
>
> What a Volume does enforce: no download route while `allow_download` is off, no preview route
> while `allow_preview` is off, one audited read per file when it is on, a read-only mount, an
> audited reference, and no bulk log download. Every way out is a decision somebody made and a
> record somebody can find.
>
> Keeping material off the cluster entirely is the deployment's egress policy, not this feature.

## Next

[Datasets](https://docs.tuneplane.ai/en/guides/datasets.md) · [Submitting a job](https://docs.tuneplane.ai/en/guides/submit.md) ·
[The Data page](https://docs.tuneplane.ai/en/console/data.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.