Skip to main content
The last command lands the preferences as a new version of the qa-preference dataset, through the same path as tp dataset push: immutable versions, a fingerprint, a quality report. An LLM judge scores automatically. Some judgements only a person can make, and this is the chain that turns those back into training data.

​
Before you start

  • Two finished runs with validation samples, on overlapping prompts.
  • An Argilla deployment your administrator has configured. tp annotate status says so.

​
Pair two runs

Two runs are required: preference annotation asks which of A and B is better, and a single run’s samples can only be rated, not compared. Pairs are aligned by prompt, not by sample index. Two validation passes do not guarantee the same sampling order, so index alignment would pair A’s third question with B’s third and the preferences would mean nothing. Pairs whose answers are identical are dropped before they reach anybody.

​
Pull the result back

You get prompt / chosen / rejected rows. Without --version the version is today’s date. Then point the next DPO run at it:

​
Check it worked

tp annotate pull prints how many pairs it read, how many produced a preference, and how many were skipped. tp dataset ls alice/qa-preference shows the new version. A “can’t tell” answer produces no training data: DPO needs an ordering, and forcing ties into it teaches the model a signal that was never there. The skipped count is printed so it does not look like lost data. When several people annotate one pair, they are resolved by majority into one row. A pair they split evenly has no ordering anybody agreed on, so it produces no row and is counted separately. Pulling needs the same access as pushing: you can read a dataset back only if you can reach both runs its answers came from, and a dataset this platform did not push is not readable through it.

​
Next

Datasets · Rubrics for the automated version of the same judgement · Pipelines
The annotation interface, permissions, progress tracking and inter-annotator agreement are Argilla’s. The platform does the two ends Argilla cannot: pairing, which needs both runs’ validation samples, and landing the result as a versioned dataset.A/B order is derived from the prompt rather than from which run is which. It is stable across re-opens, so an annotator returning to a record sees the same layout, but an annotator cannot learn that “A is always the new model” and develop a systematic bias.
This address is deployment configuration, at the same trust level as the database and object-storage addresses, so the “reject private addresses” rule that applies to user-supplied webhook URLs is not applied here. On an internal deployment Argilla is on a private network by definition.