Trace-to-Preference

v0.26.0 turns traffic logs into DPO / KTO training data. Point soup data from-traces at LangChain, OpenAI, or Soup-serve logs. Soup extracts preference signals — thumbs, regenerations, user edits — and writes clean pairs.

Ingest a log

soup data from-traces \
  --logs chat_logs.jsonl \
  --format langchain \
  --signal thumbs_up \
  --output prefs.jsonl

Flags

Signals

• thumbs_up — a prompt with both a thumbs-up and a thumbs-down response becomes a pair: the thumbed-up output is chosen, the thumbed-down one rejected. A prompt with only thumbs-up feedback yields no pair
• regenerations — if the user hit regenerate, the later response is preferred
• user_edit — if the user rewrote the response, the edit becomes chosen

What a trace record has to look like

The soup-serve parser reads a top-level signal in this vocabulary and falls back to a nested feedback.rating (up or down) for older logs:

{"prompt": "Q", "output": "Good", "signal": "thumbs_up"}
{"prompt": "Q", "output": "Bad",  "signal": "thumbs_down"}
{"prompt": "Q", "output": "Raw",  "signal": "user_edit", "edited_output": "Polished"}

On a three-row directory like the one above, --signal thumbs_up wrote {"prompt": "Q", "chosen": "Good", "rejected": "Bad", "source": "thumbs_up"} and --signal user_edit wrote the edited text as chosen and the original as rejected. Who writes what:

• soup ingest is a producer. It writes one record per trace with trace_id, prompt, output, source, metadata and a top-level signal of thumbs_up, thumbs_down or none, so its output feeds this command directly.
• user_edit and regenerated rows come from your own pipeline. signal: user_edit has to accompany the edit field: a record with edited_output and no signal reads as "no trace carried a signal". The edit is read from edited_output, edited_response or a nested feedback.edited_output.
• soup serve --trace-log records carry no signal yet (ts, prompt, response, latency_ms, tokens), so harvesting that directory builds nothing.

Through v0.75.2 the soup-serve parser read the signal only from a nested feedback.rating that no Soup producer writes, so soup ingest output and --trace-log records alike read as none, and --signal thumbs_up wrote an empty file and exited 0. Since v0.76.0 the top-level signal is read, and a harvest that reads traces but matches none says so. On a directory of --trace-log records it printed Read 1 trace(s) but built no pairs for --signal thumbs_up. No trace carried a signal. Check the record shape: ... and then Wrote 0 preference pair(s). That is still exit 0, so check the count line in a script rather than the exit code. A source that reads zero traces says so before the count line. Trace files are capped at 100,000 lines, and a PII reminder panel prints on every run.

If you built a preference dataset with from-traces on an older release from soup ingest output, it is worth checking that it is not an empty file.

Review before training

soup data review prefs.jsonl --sample 10

Previews the first N pairs in the file, in file order (--sample / -s, 1 to 100, default 10), each with its prompt, chosen and rejected text cut to 400 characters. It is not a random sample, so it will not show you problems that only occur later in the file. Use it as a sanity check before burning a GPU on junk data.

Train on them

DPO or KTO — the output format of from-traces matches both:

task: dpo
data:
  train: prefs.jsonl
  format: dpo

Safety

• Logs never execute model code — parsers only read JSON.
• Traces may contain PII or secrets; Soup warns on ingest and leaves scrubbing to you before sharing pairs externally.
• A judge that stops answering stops the run. With --judge, each request is retried twice on a 429, a 5xx or a transport error. A pair whose judge call keeps failing is skipped and counted, but once 3 requests in a row have spent their retries unanswered the command exits 1, names the judge URL and how many pairs were not judged, and writes no output file (v0.76.0, #1522). Before, from-traces --judge waited out the retries for every pair against a judge that never answered, and then wrote an empty file with exit 0. This comes from the code and the changelog; it was not run here, because no judge endpoint was available.
• Output JSONL can be validated with the same soup data validate pipeline as any other dataset.

See also

• Data formats
• DPO training guide