Configuration

Soup uses a single YAML config file for all settings. Run soup init to generate one.

Reproducibility and full fine-tuning (v0.73.0)

Two training keys landed in v0.73.0. Both are config keys, not CLI flags.

training:
  seed: 1234           # weight init of new params, data order, dropout
  data_seed: 7         # optional: vary data order while holding init fixed
  quantization: none   # required with r: 0, because the default (4bit) is refused
  lora:
    r: 0               # 0 means no adapter: full fine-tuning, dense checkpoint

seed and data_seed are the first seed knobs in the project's history: before this every run trained at seed 42 with no way to change it, so "run this twice with a different seed" was impossible. Both default to unset rather than to 42, deliberately, because an unset seed has to keep reproducing two different historical defaults (the trainer's 42 and the multipack sampler's 0) and a plain 42 would have silently re-ordered every existing multipack run. A boolean is refused by name, since seed: true would otherwise become seed 1.

The honest scope, and it widened in v0.73.1: the seed reaches the trainer arguments in every wrapper now, applied before the model loads, so setting it on a DPO run is real. It reached one wrapper of eighteen when it shipped in v0.73.0, which is what the original note here described. On a streamed run it additionally seeds the adapter initialisation for all five streaming tasks, which is why a streamed run is bit-reproducible from a seed while a resident 4-bit one still is not.

lora.r: 0 selects full fine-tuning on task: sft and, since v0.75.0, task: embedding: no adapter is built, and the run writes a dense checkpoint rather than a LoRA. On any other task the gate does not apply at all, and that is worth knowing before you rely on it: classifier, reranker, cross_encoder and asr gate LoRA behind their own flag and ignore the rank entirely, while the remaining tasks hand the 0 straight to PEFT, which rejects it with its own message about a positive integer rather than Soup's named one. Rank 0 was chosen because three parts of the codebase already read it as "no adapter" and because r: 0 used to crash, so no config that worked before changes meaning. It is refused with a named message alongside a non-transformers backend, a non-text modality, any quantisation (the default counts: quantization is 4bit unless you set it, which is why the block above sets none), Spectrum, LISA, layer streaming, or any LoRA-shaped option, and it refuses rather than running a no-op if your freeze settings leave nothing trainable.

Config Structure

base: meta-llama/Llama-3.1-8B-Instruct   # HuggingFace model ID (required)
task: sft                                   # Training task
# backend: unsloth                          # 2-5x faster (pip install "soup-cli[fast]")
# modality: text                            # text, vision, or audio

data:
  train: ./data/train.jsonl                 # Path to training data
  format: alpaca                            # Data format (auto-detected if omitted)
  val_split: 0.1                            # Validation split ratio (evaluated each epoch since v0.76.0, except on grpo, ppo and online_dpo)
  max_length: 2048                          # Max sequence length (64-1048576)
  # image_dir: ./data/images               # For vision modality
  # audio_dir: ./data/audio                # For audio modality

training:
  epochs: 3
  lr: 2e-5
  batch_size: auto                          # auto or integer
  quantization: 4bit                        # the default; also none, 8bit, gptq, awq, hqq:Nbit, ...
  # weight_decay: 0.01                      # default 0.01
  # max_grad_norm: 1.0                      # gradient clipping, default 1.0
  # logging_steps: 10                       # log metrics every N steps
  # save_steps: 100                         # checkpoint every N steps
  # warmup_ratio: 0.03                      # 0.0-0.5, default 0.03
  # scheduler: cosine                       # LR scheduler
  # quantization_aware: fp8               # fp8 or quest; true (int8 QAT) is refused since v0.76.0
  # optimizer: adamw_8bit
  # gradient_checkpointing: true
  # stream_layers: false                   # Train or align a model bigger than VRAM (v0.72, BETA)
  # stream_source: auto                    # auto | ram | disk
  # stream_buffers: 2                      # 2-8
  # stream_pin: false                      # v0.74.0: force the pageable store
  # stream_read_ahead: 2                   # v0.76.0, disk tier only: 1-8
  lora:
    r: 64
    alpha: 16
    dropout: 0.05
    # target_modules: auto                 # Auto-detected per model
    # use_dora: false                      # Weight-decomposed LoRA

output: ./output

Defaults worth knowing

These are the values a key takes when you leave it out. The quantization default is the one that surprises people: quantization is 4bit unless you set it, so a config that does not mention it trains a QLoRA. The exception is eight tasks whose trainer never quantizes the base: on those an unset value resolves to none (see Keys a task ignores).

What batch_size: auto does

auto starts from a static estimate built from the model size, sequence length, quantization and LoRA rank. On a CUDA card the SFT trainer alone then refines it with a real probe, one forward and backward at each candidate size, starting at the estimate, halving when it runs out of memory and doubling while it fits, never above four times the estimate (training.auto_batch_size_strategy chooses auto, the default, probe or static; see OOM-probe auto batch size). The answer is cached per base model, sequence length, quantization, LoRA rank and GPU. Every other task, every CPU run and static use the static estimate unchanged. task: kto floors the estimate at 2, because a KTO config with batch_size: 1 is refused (below).

It errs high when the whole model trains. The estimate charges trainable-parameter memory as a small share of the model, at most 5% and derived from the LoRA rank, and charges none for lora.r: 0. The probe never takes an optimizer step, so it does not see the optimizer state either, which for AdamW is two extra values for every trainable parameter. With lora.r: 0 or with unfrozen_parameters (Spectrum), set batch_size explicitly, or start low and raise it. LISA (lisa_enabled) is a separate case: the probe also runs while every layer is still trainable, so its gradient figure is not LISA's either, and which way it errs was not measured. Set batch_size explicitly there as well.

Layer streaming keys

These turn on layer streaming, which trains a model larger than your VRAM by keeping the frozen base out of the card entirely. They are config keys, not CLI flags: there is deliberately no --stream-layers.

Setting any of these while stream_layers is false is refused, because the knobs would silently do nothing. Streaming also reads one environment variable for the disk tier, SOUP_LAYER_STREAM_STRIPE_DIRS (see Environment variables).

A key the schema does not declare is no longer silently dropped (v0.74.0). Until then none of the config models overrode the library default of ignoring undeclared keys, so a typo validated clean and was discarded: training.quantizaton: none trained 4-bit, training.gradient_checkpoint: true did no checkpointing, data.max_len: 512 truncated at 2048, and soup train --dry-run printed "Config valid. Ready to train!" Loading a config now walks the whole model tree and reports every key it cannot place in one report, naming the field you probably meant. Since v0.75.0 the load is refused. soup train exits 1 before the training stack is imported, and the API and Web UI loader raises ValueError with the same text. Nothing is defaulted and nothing is substituted; the message says Refused. rather than Not applied., which would have read as if the run went ahead without the key, and the scan stops at 100 findings and says so. Two things are worth knowing. A config that must stay loadable on v0.74 as well needs the key removed, not renamed, because v0.74 warns and ignores it while v0.75 refuses it, and neither applies it. And soup plan and soup apply read the YAML as a plain mapping, so they do not run this check at all. soup sweep is stricter, because a swept parameter that matches nothing produces a grid of identical arms and a meaningless winner: it exits 1 before the first arm.

That refusal covers keys the schema does not know. A key the schema declares and no code reads is a different case, and v0.76.0 split it three ways.

1. Refused at load (new in v0.76.0). Dozens of combinations that loaded on v0.75 are now refused with a message that names the field and what to change. On v0.75 some of them were ignored and some failed later. Examples: use_liger or use_flash_attn on a task that is not sft or tts, rope_scaling_type on a task that does not rescale RoPE, freeze_layers on dpo, quantization_aware: true, optimizer: muon on the transformers backend, and so on. They are listed in Settings refused at load. A config that loaded on v0.75.2 can stop loading, so run soup doctor --config soup.yaml first: it exits 2 with the error and Config could not be loaded (see above).. Unknown keys are reported together, but a cross-field refusal prints only the first one it reaches, so fix that setting and run it again until it prints no error.
2. Warned at load, refused from v0.77 (upstream's stated deadline). Twenty-nine fields are accepted by the schema but read by nothing. Set to anything other than the schema default, each prints Warning: <section>.<name> is accepted but read by nothing; v0.77 will refuse it, and the load continues with exit 0. The 18 under training are long_context_grpo, vision_grpo, quantize_ref_model, grace_codebook, grace_codebook_size, grace_codebook_dim, lr_groups, early_stop_patience, citation_recall_threshold, forgetting_eval_steps, forgetting_benchmark, forgetting_stop, checkpoint_eval_steps, checkpoint_eval_metric, checkpoint_eval_tasks, checkpoint_keep_top, convergence_window and convergence_rel_tol; the 11 under data are video_dir, video_fps, video_maxlen, image_min_pixels, image_max_pixels, image_resize_algorithm, split_thinking, eval_on_each_dataset, resize_vocab, extend_conversation and skip_prepare_dataset. Delete the key to keep the config loading after that. Three deprecated values also warn and are ignored: data.remove_unused_columns: true, an explicit quantization: 4bit or 8bit on one of the eight tasks below, and soup ab --beta.
3. Still accepted in silence. A few keys are inert and carry no load warning and no announced deadline: the four parent switches forgetting_detection, checkpoint_intelligence, early_stop_on_regression and convergence_detection, plus forgetting_threshold (soup train prints a yellow Note: that they are set but not enforced) and warmup_auto. The pages that document such a key say so, as "accepted by the schema, but nothing in v0.76.0 reads it". An unread key still takes part in the config hash that soup ship --config compares, so editing one makes earlier --evidence read as stale.

soup doctor --config also prints a small backend-support table: 18 settings known to be unread on task: sft with backend: mlx (that table), 3 on backend: unsloth (quantization_aware, fp8_attention, nvfp4, all rejected), and 2 each for vision and audio SFT on the transformers backend (a few of the MLX entries are refused at load before the table is reached: see the MLX page). For every other task and backend it has no entry, so it prints None of the N setting(s) known to be unread on task=<t> backend=<b> is switched on in this config. whether or not the config has a problem. Read that line as "not reviewed", never as an all-clear.

stream_vram_probe and stream_vram_override are not interchangeable, and the difference is the point. stream_vram_probe is a measurement: it runs the step and reads the peak. stream_vram_override is an assertion you are making about how much VRAM is really available, which is what you need when the pre-flight cannot see the truth, as under a per-process memory cap in a hosted notebook.

Streaming also constrains keys you already set: quantization must be none or 4bit, batch_size must be a concrete number rather than auto, and task must be one of sft, dpo, orpo, simpo and kto. Streaming also refuses use_liger, use_flash_attn, quantization_aware, fp8_attention, nvfp4, rope_scaling_type, freeze_layers, freeze_ratio, relora_steps and a LoRA target on lm_head or embed_tokens; the full table is on Layer streaming. (kto needs batch_size of 2 or more; since v0.76.0 that holds for every KTO run, not only streamed ones.) Scaling a streaming run covers sizing, the VRAM pre-flight and the disk tier; Preference losses over streaming covers the four alignment tasks.

Keys a task ignores

Eight tasks load the base unquantized because their trainers never quantize it: distill, classifier, reranker, cross_encoder, prm, moe_lora_routing, unlearn and asr. On them:

• An unset training.quantization resolves to none (it is 4bit everywhere else).
• An explicit 4bit or 8bit (or load_in_8bit: true) loads with Warning: training.quantization: 4bit has no effect on task='distill' and is ignored: its trainer never quantises the base, so the run trains unquantised. Set quantization: none. Soup v0.77 will refuse it. and is treated as none.
• Any Quant Menu value such as gptq, and bnb_4bit_quant_storage, is refused at load.
• task: prm also refuses a non-default training.lora block, because it fine-tunes every base parameter (lora.r: 0 loads).

expand_layers is refused unless the resolved quantization is none.

How target_modules: auto is resolved

lora.target_modules: auto (the default) resolves in this order:

1. Architectures that PEFT maps itself (llama, mistral, qwen2 and so on) use PEFT's own default.
2. Architectures in Soup's table: the Qwen3.5 family and a set of Mixture-of-Experts models (qwen3_moe, deepseek_v3, deepseek_v4, glm4_moe, glm_moe_dsa, granitemoehybrid, kimi_k2, kimi_k25, gpt_oss, minimax_m2, minimax_m3_vl, mistral3). These target the attention projections; MiniMax-M3 and mistral3 use language-tower patterns, so the vision tower and projector stay unadapted.
3. Anything else is refused with training.lora.target_modules='auto' has no mapping for model_type=..., which asks for an explicit training.lora.target_modules list or, for a Mixture-of-Experts model, training.moe_lora: true. training.lora.target_parameters alone also suffices.

On granitemoehybrid only a few layers carry attention (4 of 40 on granite-4.0-h-tiny-base), so training prints a yellow Partial LoRA coverage: line. The notice ends by telling you to set training.lora.target_modules explicitly to include the other layers; Soup has not measured whether adapting a state-space projection trains well.

Templates

Soup includes 21 built-in templates:

soup init --template chat          # Conversational fine-tune
soup init --template code          # Code generation
soup init --template medical       # Domain expert
soup init --template reasoning     # GRPO reasoning (DeepSeek-R1 style)
soup init --template vision        # Vision/multimodal fine-tune
soup init --template audio         # Audio/speech fine-tune
soup init --template kto           # KTO unpaired preference
soup init --template orpo          # ORPO (no reference model)
soup init --template simpo         # SimPO length-normalized preference
soup init --template ipo           # IPO regularized preference
soup init --template bco           # BCO binary classifier preference (v0.40)
soup init --template rlhf          # Full RLHF pipeline (SFT -> RM -> PPO)
soup init --template pretrain      # Continued pre-training on raw text
soup init --template moe           # MoE fine-tuning (ScatterMoE LoRA)
soup init --template longcontext   # 128k+ context fine-tuning
soup init --template embedding     # Sentence embedding fine-tuning
soup init --template tool-calling  # Function / tool-calling fine-tune (v0.25)

Four more are regulation-shaped (compliance pack, v0.71.35). Each is a valid training config on a license-clean Apache-2.0 base plus header comments naming that regime's exact commands, because Soup's compliance controls are CLI flags and commands rather than config keys:

soup init --template hipaa         # Protected Health Information
soup init --template soc2          # SOC 2 Trust Services Criteria
soup init --template eu-ai-act     # EU AI Act Annex XI/XII
soup init --template sr-11-7       # SR 11-7 Model Risk Management

Task-Specific Config Keys

Since v0.76.0 a key set on a task, backend or path that never reads it is, for many keys, refused when the config loads rather than ignored. The refusal names the key, the task or backend, and what to do (Use task: sft, or remove the key); soup train, soup train --dry-run, soup sweep and soup doctor --config all refuse it, and deleting the key is the fix. For most of these keys a run that finished on v0.75 ignored the key and trained as if it were not there, and some of the rest stopped after the model had loaded. Three are different: for use_longlora, ebft_variant and quantization_aware: true the key was applied, wrongly, so a run that finished on v0.75 did not do what its config said (see Re-run this). The rows above say which keys: use_liger, use_flash_attn, auto_mixed_precision, freeze_layers, freeze_ratio, rope_scaling_type, moe_aux_loss_coeff, loss_watchdog and its two siblings, loraplus_lr_ratio, moe_lora, plus data.chat_template (below), expand_layers, backend: unsloth on the ten tasks with no unsloth setup, and data.add_new_tokens / new_special_tokens outside the eleven tasks that add tokens. A key not named there is still accepted and ignored by a task that does not read it, so check this column. use_ring_attention and use_galore still go through their pre-flight check first, on any task. That check can refuse a machine that lacks the package or, for use_galore, a config with quantization: 4bit or 8bit (the default is 4bit) or backend: unsloth, so use_galore: true on a DPO run with the default quantization stops the run before it starts. The curriculum_metric and curriculum_buckets rows are the other exception: with curriculum_dynamic: true (which requires curriculum: true) the dynamic-curriculum callback reads them on the 13 tasks that allow it (sft, pretrain, dpo, grpo, kto, orpo, simpo, ipo, bco, reward_model, embedding, ppo, preference). Check this column before assuming a key took effect on a DPO or GRPO run.

data.chat_template is applied since v0.76.0 by SFT and by the DPO, ORPO, SimPO, KTO, BCO, IPO, PPO, reward-model, GRPO and Online DPO trainers, for conversational data; through v0.75.2 only SFT applied it, so on the others the tokenizer's own template rendered the prompts. Tasks that never render chat (pretrain, embedding, classifier, reranker, cross_encoder, prm, asr, moe_lora_routing, unlearn) refuse the key at load.

More keys, and where each is documented

These keys are all read by the code, and the defaults and refusals in this table are what the config loader does, checked against the schema at v0.76.0.

Two declared settings are the opposite case. data.remove_unused_columns now defaults to false; an explicit true loads with a warning and is ignored, and upstream says v0.77 will refuse it. The advise: block is accepted by the schema, but nothing in v0.76.0 reads it: its three keys goal, probe and record are used by nothing (soup advise takes them as flags only, see soup advise).