tp dataset

tp dataset push gsm8k-zh v2 ./out/gsm8k-zh

Uploads a directory as an immutable version. The walkthrough is publishing and using a dataset.

Worked examples

# preprocess locally
tp dataset prepare                          # the scripts this project has
tp dataset prepare gsm8k-zh                 # runs common/data/prepare_gsm8k_zh.py

# push a version
tp dataset push gsm8k-zh v2 ./out/gsm8k-zh
tp dataset push gsm8k-zh v2 ./out/gsm8k-zh --public
tp dataset push alice/gsm8k-zh v2 ./out --readme CARD.md

# describe it
tp dataset card alice/gsm8k-zh              # print the current card
tp dataset card alice/gsm8k-zh -f README.md # update it without pushing data

# inspect before pushing
tp dataset check ./out/my-train --against ./eval/gsm8k-test.jsonl
tp dataset check ./out/my-train --against ./eval/test.jsonl --fields question
tp dataset quality ./out/my-train
tp dataset fingerprint ./gsm8k-test.jsonl -o gsm8k.fingerprint.json

# list and share
tp dataset ls
tp dataset ls alice/gsm8k-zh -v v2          # one version's file manifest
tp dataset visibility alice/gsm8k-zh --public
tp dataset visibility alice/gsm8k-zh --private

Notes the flag table cannot carry

DATASET without an owner lands in your own namespace. Only an administrator can push across namespaces.
VERSION is immutable. A version of the same name cannot be overwritten; push a new one.
--readme defaults to README.md inside PATH when there is one. It becomes the dataset card, which belongs to the dataset rather than to one version.
tp dataset check exits 1 when there is overlap, so it drops into CI as a gate.
Upload is chunked, checksummed per file, and resumable: re-running after an interruption skips what already landed.

Check it worked

tp dataset ls alice/gsm8k-zh

The new version appears with its file count and size. Then reference it:

tp submit my-grpo --profile h200:8 \
  --train-dataset alice/gsm8k-zh@v2 --train-data train.parquet

`tp dataset`

Manage datasets

tp dataset <subcommand>

Command	Description
`tp dataset card`	Show/update a dataset description (README.md)
`tp dataset check`	Check training data for evaluation overlap
`tp dataset fingerprint`	Compute a dataset fingerprint
`tp dataset ls`	List the visible datasets
`tp dataset prepare`	Preprocess a dataset locally
`tp dataset push`	Upload one dataset version
`tp dataset quality`	Produce a dataset quality report
`tp dataset visibility`	Change a dataset’s visibility (owner or admin)

`tp dataset card`

Show/update a dataset description (README.md)

tp dataset card <DATASET> [options]

Argument	Description
`DATASET`	Dataset id (<owner>/<name>) (required)

Option	Description
`--file`, `-f`	Update the description from this markdown file; omit to print the current one

`tp dataset check`

Check training data for evaluation overlap

tp dataset check <PATH> [options]

Argument	Description
`PATH`	Training data directory or file (jsonl / json / csv / parquet) (required)

Option	Description
`--against`, `-a`	Evaluation set directory or file; the question is whether its questions leaked into training (required)
`--fields`	Only these fields (comma separated), e.g. question,prompt. Defaults to every string field

`tp dataset fingerprint`

Compute a dataset fingerprint

tp dataset fingerprint <PATH> [options]

Argument	Description
`PATH`	Data directory or file (required)

Option	Description
`--out`, `-o`	Write it to this file; omit to print a summary
`--fields`	Only these fields (comma separated)

`tp dataset ls`

List the visible datasets

tp dataset ls [DATASET] [options]

Argument	Description
`DATASET`	Dataset id (<owner>/<name>); omit to list everything visible

Option	Description
`--version`, `-v`	Show one version’s file list; defaults to the latest

`tp dataset prepare`

Preprocess a dataset locally

tp dataset prepare [DATASET]

Argument	Description
`DATASET`	Dataset name; leave empty to list the options

No options beyond --help.

`tp dataset push`

Upload one dataset version

tp dataset push <DATASET> <VERSION> <PATH> [options]

Argument	Description
`DATASET`	Dataset name (lands in your own namespace), or a full <owner>/<name> (an admin can cross namespaces) (required)
`VERSION`	Version, e.g. v1 / 20260812 (required)
`PATH`	Local directory (required)

Option	Description
`--public`	Make it public on first creation (anyone can reference it in training)
`--readme`	Dataset description (markdown); defaults to the README.md in the directory
`--concurrency` `<int>`	Uploads in flight. Left alone, it is chosen from the size of what you are sending; the right number depends on the link, which nothing here can see (default `0`)

`tp dataset quality`

Produce a dataset quality report

tp dataset quality <PATH> [options]

Argument	Description
`PATH`	Data directory or file (required)

Option	Description
`--fields`	Only these fields (comma separated)

`tp dataset visibility`

Change a dataset’s visibility (owner or admin)

tp dataset visibility <DATASET> [options]

Argument	Description
`DATASET`	Dataset id (<owner>/<name>) (required)

Option	Description
`--public`	Make it public
`--private`	Make it private

CLI reference

Set up

Experiments

Train

Artifacts and evaluation

Resources

Administration

Worked examples

Notes the flag table cannot carry

Check it worked

`tp dataset`

`tp dataset card`

`tp dataset check`

`tp dataset fingerprint`

`tp dataset ls`

`tp dataset prepare`

`tp dataset push`

`tp dataset quality`

`tp dataset visibility`

​Worked examples

​Notes the flag table cannot carry

​Check it worked

​tp dataset

​tp dataset card

​tp dataset check

​tp dataset fingerprint

​tp dataset ls

​tp dataset prepare

​tp dataset push

​tp dataset quality

​tp dataset visibility

Worked examples

Notes the flag table cannot carry

Check it worked

`tp dataset`

`tp dataset card`

`tp dataset check`

`tp dataset fingerprint`

`tp dataset ls`

`tp dataset prepare`

`tp dataset push`

`tp dataset quality`

`tp dataset visibility`