mx/AareLC_train
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity
Files
main
AareLC_train/runs
T
History
jungfrauandClaude Fable 5.1 ddaa5a99e6
CI / ledger-and-tools (push) Successful in 20s
Details
dedup study: suppressing nested duplicate outlines on the unseen gold frames 
Containment (intersection over the smaller area) between human-drawn crystal pairs never
exceeds 0.42, while the model-drawn duplicates sit at 0.96-1.0, so a rule at 0.9 cannot
lose a real pair. On 231 unseen frames, v4: precision 0.831 -> 0.876, recall 0.881 -> 0.869,
F1 0.855 -> 0.873, 46 outlines removed, 8 true positives lost where the less confident of
two outlines was the one matching the label.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 22:26:18 +02:00
..
2026-09-28_alc-v1
run 2026-09-28_alc-v1: promoted by the gate after the mask-term fix (F1 +0.045, mask score 0.678 -> 0.710)
2026-09-28 19:18:27 +02:00
2026-09-28_alc-v1-800px
run 2026-09-28_alc-v1-800px: gate_failed
2026-09-29 00:31:02 +02:00
2026-09-28_alc-v1-humanonly
run 2026-09-28_alc-v1-humanonly: gate passed, not promoted (A/B on clean gold: masks no tighter without accepted-unchanged labels, crystal worse)
2026-09-28 20:44:27 +02:00
2026-09-28_alc-v1-mask1
run 2026-09-28_alc-v1-mask1: gate_failed
2026-09-28 23:54:53 +02:00
2026-09-28_alc-v1-noflip
run 2026-09-28_alc-v1-noflip: gate_passed
2026-09-28 21:30:31 +02:00
2026-09-28_alc-v1-overlap
run 2026-09-28_alc-v1-overlap: aborted on purpose (nested classes need overlap_mask off)
2026-09-28 23:55:33 +02:00
2026-09-29_alc-v1-noflip-mask1
ledger: 2026-09-29_alc-v1-noflip-mask1 deployed at the beamlines (weights only)
2026-09-29 20:47:44 +02:00
2026-09-29_v2-noflip-mask1
ledger: v3-noflip-mask1 training on the Spark (restarted 07:13 after a memory-starved hang); v2-noflip-mask1 aborted, superseded by v3
2026-09-30 07:16:27 +02:00
2026-09-29_v2-recipeA
run 2026-09-29_v2-recipeA: gate_failed; comparison updated
2026-09-29 17:58:39 +02:00
2026-09-29_v3-noflip-mask1
ledger: 2026-09-29_v3-noflip-mask1 deployed at the beamlines
2026-10-02 13:25:45 +02:00
2026-09-29_v3-recipeA
run 2026-09-29_v3-recipeA: gate_passed; comparison updated
2026-09-29 23:42:41 +02:00
2026-09-30_v3-n-1024px
cluster runs on dataset v3: resolution and model-size levers
2026-09-30 07:18:06 +02:00
2026-09-30_v3-s-640px
small cluster model re-gated on the Spark in one environment: gate_passed against v3; comparison after cycle 4
2026-10-02 13:30:10 +02:00
2026-09-30_v3-s-1024px
cluster runs on dataset v3: resolution and model-size levers
2026-09-30 07:18:06 +02:00
2026-10-01_v4-noflip-mask1
run 2026-10-01_v4-noflip-mask1: promoted; comparison updated
2026-10-02 13:25:46 +02:00
2026-10-01_v4-recipeA
run 2026-10-01_v4-recipeA: gate_failed; comparison updated
2026-10-02 13:25:45 +02:00
clean_gold_comparison.json
gold: 285 frames (batch 8 added by mount: 83 corrected frames, 2-per-mount cap, 20 siblings held out); every model re-measured
2026-10-05 15:49:30 +02:00
clean_gold_comparison.txt
all models re-measured on the 203-frame gold set; comparison on the unseen frames updated
2026-09-29 18:00:50 +02:00
dedup_study.json
dedup study: suppressing nested duplicate outlines on the unseen gold frames
2026-10-05 22:26:18 +02:00
README.md
small cluster model re-gated on the Spark in one environment: gate_passed against v3; comparison after cycle 4
2026-10-02 13:30:10 +02:00

README.md

Training runs

Generated by python tools/ledger.py render; do not edit by hand.

Run Status Dataset Base Version F1 Mask IoU Gate Last change
2026-09-28_alc-v1 promoted dataset/2026-09-28_alc-v1 /home/jungfrau/AareLC/models/yolo26n-seg.pt yolo26n-seg-alc-v1-2026-09-28 0.821 0.881 yes 2026-09-28T19:17
2026-09-28_alc-v1-800px gate_failed dataset/2026-09-28_alc-v1 /home/jungfrau/AareLC/models/yolo26n-seg.pt yolo26n-seg-alc-v1-800px-2026-09-29 0.808 0.895 no 2026-09-29T00:31
2026-09-28_alc-v1-humanonly gate_passed dataset/2026-09-28_alc-v1-humanonly /home/jungfrau/AareLC/models/yolo26n-seg.pt yolo26n-seg-alc-v1-humanonly-2026-09-28 0.820 0.877 yes 2026-09-28T20:44
2026-09-28_alc-v1-mask1 gate_failed dataset/2026-09-28_alc-v1 /home/jungfrau/AareLC/models/yolo26n-seg.pt yolo26n-seg-alc-v1-mask1-2026-09-28 0.820 0.888 no 2026-09-28T23:54
2026-09-28_alc-v1-noflip gate_passed dataset/2026-09-28_alc-v1 /home/jungfrau/AareLC/models/yolo26n-seg.pt yolo26n-seg-alc-v1-noflip-2026-09-28 0.835 0.881 yes 2026-09-28T21:30
2026-09-28_alc-v1-overlap aborted dataset/2026-09-28_alc-v1 /home/jungfrau/AareLC/models/yolo26n-seg.pt - - - - 2026-09-28T23:55
2026-09-29_alc-v1-noflip-mask1 deployed dataset/2026-09-28_alc-v1 /home/jungfrau/AareLC/models/yolo26n-seg.pt yolo26n-seg-alc-v1-noflip-mask1-2026-09-29 0.827 0.894 yes 2026-09-29T20:47
2026-09-29_v2-noflip-mask1 aborted dataset/2026-09-29_v2 models/base/yolo26n-seg.pt - - - - 2026-09-30T07:16
2026-09-29_v2-recipeA gate_failed dataset/2026-09-29_v2 models/base/yolo26n-seg.pt yolo26n-seg-v2-recipeA-2026-09-29 0.797 0.874 no 2026-09-29T17:56
2026-09-29_v3-noflip-mask1 deployed dataset/2026-09-29_v3 models/base/yolo26n-seg.pt yolo26n-seg-v3-noflip-mask1-2026-09-30 0.837 0.887 yes 2026-10-02T11:30
2026-09-29_v3-recipeA gate_passed dataset/2026-09-29_v3 models/base/yolo26n-seg.pt yolo26n-seg-v3-recipeA-2026-09-29 0.831 0.881 yes 2026-09-29T23:42
2026-09-30_v3-n-1024px exported dataset/2026-09-29_v3 models/base/yolo26n-seg.pt - - - - 2026-09-30T07:17
2026-09-30_v3-s-1024px exported dataset/2026-09-29_v3 models/base/yolo26s-seg.pt - - - - 2026-09-30T07:17
2026-09-30_v3-s-640px gate_passed dataset/2026-09-29_v3 models/base/yolo26s-seg.pt yolo26s-seg-v3-s-640px-2026-10-01 0.861 0.886 yes 2026-10-02T13:29
2026-10-01_v4-noflip-mask1 promoted dataset/2026-10-01_v4 models/base/yolo26n-seg.pt yolo26n-seg-v4-noflip-mask1-2026-10-02 0.852 0.891 yes 2026-10-02T13:25
2026-10-01_v4-recipeA gate_failed dataset/2026-10-01_v4 models/base/yolo26n-seg.pt yolo26n-seg-v4-recipeA-2026-10-02 0.832 0.884 no 2026-10-02T04:12

States: exported → queued → training → measured → gate_passed → gate_failed → promoted → deployed (or aborted).

2026-09-28_alc-v1

  • 2026-09-28T16:51:45 · exported · jungfrau@mx-spark-01 — first loop run, on the Spark
  • 2026-09-28T16:51:45 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-28T17:40:30 · measured · jungfrau@mx-spark-01
  • 2026-09-28T17:40:30 · gate_failed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-overlap-false-2026-04-12/gold_report.json", "f1_delta": 0.0452, "mask_iou_delta": -0.0217, "class_f1_delta": {"loop_all": 0.0786, "pin": 0.0339}, "pass": false}
  • 2026-09-28T19:17:44 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-28T19:17:49 · measured · jungfrau@mx-spark-01
  • 2026-09-28T19:17:49 · gate_passed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-overlap-false-2026-04-12/gold_report.json", "f1_delta": 0.0452, "mask_iou_delta": 0.0327, "mask_score": 0.7103, "mask_score_baseline": 0.6775, "matched_only_iou_delta": -0.0217, "class_f1_delta": {"loop_all": 0.0786, "pin": 0.0339}, "pass": true}
  • 2026-09-28T19:17:49 · promoted · jungfrau@mx-spark-01 — by the gate

2026-09-28_alc-v1-800px

  • 2026-09-28T23:55:28 · exported · jungfrau@mx-spark-01 — training size 800 px; same dataset and gold as 2026-09-28_alc-v1
  • 2026-09-28T23:55:28 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-29T00:31:00 · measured · jungfrau@mx-spark-01
  • 2026-09-29T00:31:00 · gate_failed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-2026-09-28/gold_report.json", "f1_delta": -0.0123, "mask_iou_delta": -0.0099, "mask_score": 0.7004, "mask_score_baseline": 0.7103, "matched_only_iou_delta": 0.0143, "class_f1_delta": {"loop_all": -0.0063, "pin": 0.0151}, "pass": false}

2026-09-28_alc-v1-humanonly

  • 2026-09-28T20:01:45 · exported · jungfrau@mx-spark-01 — twin of 2026-09-28_alc-v1 without the 163 accepted-unchanged frames; to be trained on the cluster
  • 2026-09-28T20:07:22 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-28T20:42:32 · measured · jungfrau@mx-spark-01
  • 2026-09-28T20:42:32 · gate_passed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-2026-09-28/gold_report.json", "f1_delta": -0.0004, "mask_iou_delta": 0.0113, "mask_score": 0.7215, "mask_score_baseline": 0.7103, "matched_only_iou_delta": -0.0033, "class_f1_delta": {"loop_all": 0.0292, "pin": 0.0042}, "pass": true}
  • 2026-09-28T20:44:27 · gate_passed · jungfrau@mx-spark-01 — gate passed vs the beamline baseline; not promoted: on the 66 clean frames B ties A on F1 (0.851 vs 0.853), mask IoU no tighter (0.878 vs 0.883), crystal F1 0.40 vs 0.62; A stays the candidate

2026-09-28_alc-v1-mask1

  • 2026-09-28T21:30:31 · exported · jungfrau@mx-spark-01 — full-resolution masks (mask_ratio=1); same dataset and gold as 2026-09-28_alc-v1
  • 2026-09-28T21:30:31 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-28T23:54:44 · measured · jungfrau@mx-spark-01
  • 2026-09-28T23:54:44 · gate_failed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-2026-09-28/gold_report.json", "f1_delta": -0.0003, "mask_iou_delta": -0.0013, "mask_score": 0.709, "mask_score_baseline": 0.7103, "matched_only_iou_delta": 0.0072, "class_f1_delta": {"loop_all": -0.0063, "pin": 0.0196}, "pass": false}

2026-09-28_alc-v1-noflip

  • 2026-09-28T21:03:25 · exported · jungfrau@mx-spark-01 — no horizontal flip (fliplr=0): the camera never shows the pin on the left; same dataset and gold as 2026-09-28_alc-v1
  • 2026-09-28T21:03:25 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-28T21:30:30 · measured · jungfrau@mx-spark-01
  • 2026-09-28T21:30:30 · gate_passed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-2026-09-28/gold_report.json", "f1_delta": 0.0142, "mask_iou_delta": 0.0093, "mask_score": 0.7196, "mask_score_baseline": 0.7103, "matched_only_iou_delta": -0.0001, "class_f1_delta": {"loop_all": 0.0168, "pin": 0.0144}, "pass": true}

2026-09-28_alc-v1-overlap

  • 2026-09-28T23:54:53 · exported · jungfrau@mx-spark-01 — overlap_mask on; same dataset and gold as 2026-09-28_alc-v1
  • 2026-09-28T23:54:53 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-28T23:55:32 · aborted · jungfrau@mx-spark-01 — skipped on purpose: overlap_mask merges nested masks (crystal inside loop_face inside loop_all), so the setting cannot be right for these labels

2026-09-29_alc-v1-noflip-mask1

  • 2026-09-29T07:31:18 · exported · jungfrau@mx-spark-01 — no horizontal flip + full-resolution masks (fliplr=0, mask_ratio=1); same dataset and gold as 2026-09-28_alc-v1
  • 2026-09-29T07:31:18 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-29T10:36:12 · measured · jungfrau@mx-spark-01
  • 2026-09-29T10:36:12 · gate_passed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-2026-09-28/gold_report.json", "f1_delta": 0.0062, "mask_iou_delta": 0.0249, "mask_score": 0.7352, "mask_score_baseline": 0.7103, "matched_only_iou_delta": 0.0133, "class_f1_delta": {"loop_all": 0.0084, "pin": -0.0167}, "pass": true}
  • 2026-09-29T11:13:44 · promoted · jungfrau@mx-spark-01 — by decision after the A/B comparison on the unseen frames (gate had passed vs A)
  • 2026-09-29T20:47:16 · deployed · jungfrau@mx-spark-01

2026-09-29_v2-noflip-mask1

  • 2026-09-29T17:06:19 · exported · jungfrau@mx-spark-01 — default recipe (no flip, full-resolution masks) on dataset v2: to be trained on the cluster with sbatch slurm/train.sbatch
  • 2026-09-30T07:16:27 · aborted · jungfrau@mx-spark-01 — superseded: dataset v3 exists, cluster runs registered on it instead

2026-09-29_v2-recipeA

  • 2026-09-29T17:06:19 · exported · jungfrau@mx-spark-01 — settings A (fliplr 0.5, mask_ratio 4) on dataset v2, Spark
  • 2026-09-29T17:06:19 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-29T17:56:27 · measured · jungfrau@mx-spark-01
  • 2026-09-29T17:56:27 · gate_failed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-noflip-mask1-2026-09-29/gold_report.json", "f1_delta": -0.0123, "mask_iou_delta": -0.0039, "mask_score": 0.7298, "mask_score_baseline": 0.7337, "matched_only_iou_delta": -0.0168, "class_f1_delta": {"loop_all": 0.0089, "pin": 0.0021}, "pass": false}

2026-09-29_v3-noflip-mask1

  • 2026-09-29T22:41:13 · exported · jungfrau@mx-spark-01 — promoted recipe (fliplr 0, mask_ratio 1) on dataset v3: candidate for the beamlines, Spark
  • 2026-09-29T23:42:41 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-30T07:12:16 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-30T13:55:59 · measured · jungfrau@mx-spark-01
  • 2026-09-30T13:55:59 · gate_passed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-noflip-mask1-2026-09-29/gold_report.json", "f1_delta": 0.0284, "mask_iou_delta": 0.0211, "mask_score": 0.7547, "mask_score_baseline": 0.7337, "matched_only_iou_delta": -0.0041, "class_f1_delta": {"loop_all": 0.0228, "pin": 0.0301}, "pass": true}
  • 2026-09-30T13:55:59 · promoted · jungfrau@mx-spark-01 — by the gate
  • 2026-10-02T11:30:41 · deployed · jungfrau@mx-spark-01 — deployed at the beamlines by the user (weights only), reported 2026-10-02; replaces yolo26n-seg-alc-v1-noflip-mask1-2026-09-29

2026-09-29_v3-recipeA

  • 2026-09-29T22:41:12 · exported · jungfrau@mx-spark-01 — settings A (fliplr 0.5, mask_ratio 4) on dataset v3: quick signal, gate only, Spark
  • 2026-09-29T22:41:13 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-29T22:46:24 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-29T22:48:37 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-09-29T23:42:37 · measured · jungfrau@mx-spark-01
  • 2026-09-29T23:42:37 · gate_passed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-noflip-mask1-2026-09-29/gold_report.json", "f1_delta": 0.0221, "mask_iou_delta": 0.0074, "mask_score": 0.741, "mask_score_baseline": 0.7337, "matched_only_iou_delta": -0.0101, "class_f1_delta": {"loop_all": 0.0446, "pin": 0.0057}, "pass": true}

2026-09-30_v3-n-1024px

  • 2026-09-30T07:17:45 · exported · jungfrau@mx-spark-01 — cluster: resolution lever, nano at 1024 px, default recipe (no flip, full-resolution masks)

2026-09-30_v3-s-1024px

  • 2026-09-30T07:17:45 · exported · jungfrau@mx-spark-01 — cluster: both levers, small model at 1024 px, default recipe

2026-09-30_v3-s-640px

  • 2026-09-30T07:17:45 · exported · jungfrau@mx-spark-01 — cluster: capacity lever, small model at 640 px, default recipe
  • 2026-10-01T01:30:13 · queued · duan_j@gpu102
  • 2026-10-01T01:30:13 · training · duan_j@gpu102 — train_run.py
  • 2026-10-01T04:18:50 · measured · duan_j@gpu102
  • 2026-10-01T04:18:50 · gate_failed · duan_j@gpu102 — {"baseline": "/data/user/duan_j/AareLC_train/models/registry/yolo26n-seg-v3-noflip-mask1-2026-09-30/gold_report.json", "f1_delta": 0.0024, "mask_iou_delta": -0.0492, "mask_score": 0.7056, "mask_score_baseline": 0.7547, "matched_only_iou_delta": -0.0215, "class_f1_delta": {"loop_all": -0.003, "pin": -0.0104}, "pass": false}
  • 2026-10-02T13:29:54 · gate_passed · jungfrau@mx-spark-01 — re-gated on the Spark in one environment against yolo26n-seg-v3-noflip-mask1-2026-09-30 (the cluster verdict compared two environments); not promoted: four times the parameters, needs a latency check at a beamline

2026-10-01_v4-noflip-mask1

  • 2026-10-01T23:52:09 · exported · jungfrau@mx-spark-01 — promoted recipe (fliplr 0, mask_ratio 1) on dataset v4: candidate for the beamlines, Spark
  • 2026-10-02T04:13:26 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-10-02T13:25:34 · measured · jungfrau@mx-spark-01
  • 2026-10-02T13:25:34 · gate_passed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-v3-noflip-mask1-2026-09-30/gold_report.json", "f1_delta": 0.015, "mask_iou_delta": 0.0122, "mask_score": 0.767, "mask_score_baseline": 0.7547, "matched_only_iou_delta": 0.0042, "class_f1_delta": {"loop_all": 0.024, "pin": 0.0068}, "pass": true}
  • 2026-10-02T13:25:34 · promoted · jungfrau@mx-spark-01 — by the gate

2026-10-01_v4-recipeA

  • 2026-10-01T23:52:09 · exported · jungfrau@mx-spark-01 — settings A (fliplr 0.5, mask_ratio 4) on dataset v4: quick signal, gate only, Spark
  • 2026-10-01T23:59:18 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-10-02T02:04:53 · measured · jungfrau@mx-spark-01
  • 2026-10-02T02:04:53 · gate_failed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-v3-noflip-mask1-2026-09-30/gold_report.json", "f1_delta": -0.0057, "mask_iou_delta": -0.0083, "mask_score": 0.7464, "mask_score_baseline": 0.7547, "matched_only_iou_delta": -0.0031, "class_f1_delta": {"loop_all": 0.0095, "pin": -0.0036}, "pass": false}
  • 2026-10-02T02:05:55 · training · jungfrau@mx-spark-01 — train_run.py
  • 2026-10-02T04:12:20 · measured · jungfrau@mx-spark-01
  • 2026-10-02T04:12:20 · gate_failed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-v3-noflip-mask1-2026-09-30/gold_report.json", "f1_delta": -0.0057, "mask_iou_delta": -0.0083, "mask_score": 0.7464, "mask_score_baseline": 0.7547, "matched_only_iou_delta": -0.0031, "class_f1_delta": {"loop_all": 0.0095, "pin": -0.0036}, "pass": false}
Reference in new issue View Git Blame Copy Permalink