Written to runs/_unseen_per_frame.json (not tracked): overall and crystal
tp/fp/fn and matched IoU per unseen gold frame and model, which lets the
monitor page bootstrap the difference between any pair.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The cluster's first run was gated against a baseline measured on the Spark.
The same weights score lower there (F1 0.840 vs 0.861, mask score 0.706 vs
0.757 on the 203 gold frames), so the gate failed a model that beats the
active one on F1 in 98.7 % of bootstrap draws when both are measured on one
machine.
Gold reports now record the environment (ultralytics, torch, host). When the
active model's report comes from other versions, train_run.py measures its
weights again in the job and gates against that; the predictions feed the
paired bootstrap. Without the weights there is no verdict and no promotion.
The SLURM scripts pull the active model's weights.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A paired bootstrap over the 149 unseen gold frames (2026-10-02) showed that a
difference between two models carries about +-0.023 of noise in F1, +-0.018 in
the mask score and +-0.10 in crystal F1, while the gate's tolerance was 0.005.
- tools/gate_stats.py: per-frame counts and a paired bootstrap (pure Python).
- train_run.py: when the baseline's gold predictions exist, the gate requires
the candidate to beat it on F1 or the mask score in 90 % of the draws and
not to lose the other in more than 90 %; the point rule is the fallback and
is always reported (pass_point_rule).
- tools/compare_unseen.py replaces the Spark's ~/fair_gate_all.py: all models
on the frames none trained on, on the frozen core (data/gold/unseen_core.json,
149 frames) and on all unseen frames once the gold set grows, with the
interval and win share against the active model.
- measure_weights.py keeps a version's gate verdict when it re-measures it.
- tools/spark_cycle.sh: gold growth, export, re-measure, train, gate, compare,
push, with what cycles 3 and 4 taught built in.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The exit code 2 of a failed gate made the cycle script retry and train
2026-10-01_v4-recipeA twice.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Three ledger runs for the cluster, exported on dataset/2026-09-29_v3 with the
promoted recipe: nano at 1024 px, small at 640 px, small at 1024 px. A run can
now carry its own training arguments (train_args in run.json, set with
ledger new --kv), which train_run.py applies under any --set; train.sbatch
forwards extra arguments after the run id (e.g. --batch 8). yolo26s-seg.pt
joins the base weights under LFS. slurm/README lists the runs, the questions
they answer and the commands.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
