Model Registry

v0.26.0 ships a local model registry at ~/.soup/registry.db. Every fine-tune can be pushed, tagged, searched, diffed, and walked as a lineage DAG — entirely offline, no server, no cloud.

Why a registry

Most people end up with 17 checkpoint directories called output-3-final and no idea which one won. The registry fixes that: one row per fine-tune, carrying config, eval baseline, and parent lineage.

Push a run

# After training a model
soup registry push \
  --run-id run_20260420_143052_a1b2 \
  --name chat-llama \
  --tag v1 \
  --notes "Alpaca 15k, 2 epochs, lr=2e-4"

The registry derives lineage from the run's config (same base model, different run id); there is no --parent flag.

List, show, search

# List everything
soup registry list

# Filter
soup registry list --name chat-llama
soup registry list --tag prod
soup registry list --base Qwen/Qwen3-8B --task sft

# Show details + lineage
soup registry show chat-llama:v1

# Search across name, base, task, notes
soup registry search "medical"

--limit (-l) caps the rows: soup registry list shows 100 by default and soup registry search 50. A value outside the allowed range is clamped, not rejected: 1 to 1000 for list, 1 to 500 for search.

Referring to an entry

Every command that takes a reference accepts an entry id, a unique id prefix, name:tag (the form soup registry push prints, for example chat-llama:v1) or registry://<id>. The separator is a colon: chat-llama@v1 is read as an id prefix, matches nothing and exits 1 with "Registry entry not found". If a name and tag match several entries, the most recent one wins.

Diff two versions

soup registry diff chat-llama:v1 chat-llama:v2

soup registry diff prints a table of config differences between the two entries, and an eval-delta table when either entry has eval results attached:

     Config diff: reg_20261006_6a0a45ef1ac4 -> reg_20261006_10a81a6c72ae
┌─────────────────┬─────────┬────────┬────────┐
│ Path            │ Kind    │ Left   │ Right  │
├─────────────────┼─────────┼────────┼────────┤
│ training.lora.r │ changed │ 16     │ 32     │
│ training.lr     │ changed │ 0.0002 │ 0.0003 │
└─────────────────┴─────────┴────────┴────────┘

When a benchmark was measured more than once, the newest score is the one compared. Since v0.76.0 soup registry diff, soup eval compare and soup eval leaderboard pick one row per benchmark through a shared helper (newest created_at first, then insertion order for equal timestamps; a row with no score is skipped). Through v0.75.2 each of them let the oldest score win and ignored a re-measurement: upstream's example is run A logging 0.30 then 0.45 and run B logging 0.50 then 0.40, where the diff reported +0.20 and no regression instead of the real -0.05 regression. If you compared entries that had been measured more than once on an earlier release, re-run the comparison; nothing in the stored rows changed. The same rule now applies to soup card's scorecard and to a registry://<id> baseline in the eval gate.

Attach things to an entry

An entry can carry artifact files. Each attached file is stored as one record with its path (which must sit under the working directory), SHA-256 and size; a directory is refused, so a command that produces several files attaches each one. The kind is a closed list: adapter, merged, gguf, awq, gptq, onnx, tensorrt, torchao, dataset, config, eval_results, eval_suite, canaries, diagnose_report, steering_vector, judge_calibration, branch_ref, edited_model, grace_codebook, shrink_report, bom and attestation. There is no soup registry attach command; the attach happens through a flag on the command that produces the file:

What a failed attach does depends on the command. Since v0.76.0 soup diagnose, soup eval unlearning, soup steer train and soup edit set treat a requested attach as part of success: a failed attach (unknown or ambiguous entry, unreadable registry, no weight file) prints an error and exits 1 with the file left on disk, and, for soup diagnose, soup eval unlearning and soup edit set, the flag without --output is refused before anything runs (exit 3 for soup diagnose, 2 for the other two); soup steer train has no such refusal, because its --output is optional. Through v0.75.2 those four passed the registry helper arguments it does not take, or a directory, so they printed a warning, recorded nothing and exited 0. soup export --registry-id is the exception: a failed attach there is a yellow warning and the command still exits 0, because the export itself succeeded, and when no --registry-id is given and no entry matches the model's output directory it records nothing and prints nothing.

Promote to prod

soup registry promote chat-llama:v2 --tag prod

Tagging is how you mark stable versions. Any number of tags can point at the same entry.

Lineage DAG

soup history chat-llama

soup history takes an entry name and prints one node per entry with that name: its id, its tags in parentheses and its creation time. Where they exist, ancestors, descendants and branches appear as child nodes. It prints no eval scores.

chat-llama
├── reg_20261006_6a0a45ef1ac4 (v1) 2026-10-06T01:14
└── reg_20261006_10a81a6c72ae (prod, v2) 2026-10-06T01:14

Cycle detection is enforced on every walk — forking a descendant back onto an ancestor raises an error.

Delete

soup registry delete chat-llama:v1 --yes

soup registry delete <ref> --yes deletes that one entry together with the database rows that hang off it (its tags, artifact records and lineage links). Entries that descended from it are not deleted; they lose the link. It removes registry records only, not the files they point at. Without --yes it only prints About to delete <id> (<name>). Pass --yes to confirm. and exits 0, so there is no interactive confirmation and no subtree preview.

Use as an eval-gate baseline

The eval gate accepts registry://<id> as the baseline, so quality regressions are diffed against any historical run rather than a static file.

Storage

• SQLite at ~/.soup/registry.db — human-readable, portable
• Single writer, no daemon
• Name (up to 128 characters) and tag (up to 64) are validated: letters, digits, _, - and . only, starting with an alphanumeric, so neither can contain :, @ or a path separator
• SQL LIKE-wildcard escaping on every search term

See also

• Eval gate — baselines can point at registry entries
• Soup Cans — export a registry entry as a portable .can
• CLI reference