sf bench 不同:后者通过外部 harness 跑学术标准基准,
结果能跨 run、跨方法、跨别人的模型比较。在一个方法内部迭代时用 sf eval,要做比较时用 sf bench。
它也能在训练后自动执行:
Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
Run a recipe’s native evaluation
sf eval my-grpo --model Qwen/Qwen2.5-7B
sf bench 不同:后者通过外部 harness 跑学术标准基准,
结果能跨 run、跨方法、跨别人的模型比较。在一个方法内部迭代时用 sf eval,要做比较时用 sf bench。
它也能在训练后自动执行:
sf submit my-grpo --profile h200:8 --then eval
sf evalsf eval <EXP> [options]
| 参数 | 说明 |
|---|---|
EXP | Experiment name or path (必填) |
| 选项 | 说明 |
|---|---|
--run-id | VeRL SFT: the training run id that receives the validation samples |
--model | NeMo-RL or VeRL SFT: HF model path / Hub id |
--eval-config | NeMo-RL: explicit evaluation config path |
--data | verl: explicit evaluation data path |
--step <int range> | VeRL SFT: the training step the exported checkpoint belongs to |
--profile | Hardware profile (managed by the server registry; sf status lists the available values) |
--allow-dirty | Allow uncommitted changes in the working tree |
--dry-run | Only print the command that would be submitted |