experiments 2026-09-28: comparison on the unseen gold frames
CI / ledger-and-tools (push) Successful in 20s
CI / ledger-and-tools (push) Successful in 20s
This commit is contained in:
jungfrau committed
1 parent
40dc9e54e3
commit
d2d525487c
3 files changed
+179
No files matched your search
@@ -5,7 +5,11 @@ Generated by `python tools/ledger.py render`; do not edit by hand.
|
||||
| Run | Status | Dataset | Base | Version | F1 | Mask IoU | Gate | Last change |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| 2026-09-28_alc-v1 | promoted | dataset/2026-09-28_alc-v1 | /home/jungfrau/AareLC/models/yolo26n-seg.pt | yolo26n-seg-alc-v1-2026-09-28 | 0.821 | 0.881 | yes | 2026-09-28T19:17 |
|
||||
| 2026-09-28_alc-v1-800px | gate_failed | dataset/2026-09-28_alc-v1 | /home/jungfrau/AareLC/models/yolo26n-seg.pt | yolo26n-seg-alc-v1-800px-2026-09-29 | 0.808 | 0.895 | no | 2026-09-29T00:31 |
|
||||
| 2026-09-28_alc-v1-humanonly | gate_passed | dataset/2026-09-28_alc-v1-humanonly | /home/jungfrau/AareLC/models/yolo26n-seg.pt | yolo26n-seg-alc-v1-humanonly-2026-09-28 | 0.820 | 0.877 | yes | 2026-09-28T20:44 |
|
||||
| 2026-09-28_alc-v1-mask1 | gate_failed | dataset/2026-09-28_alc-v1 | /home/jungfrau/AareLC/models/yolo26n-seg.pt | yolo26n-seg-alc-v1-mask1-2026-09-28 | 0.820 | 0.888 | no | 2026-09-28T23:54 |
|
||||
| 2026-09-28_alc-v1-noflip | gate_passed | dataset/2026-09-28_alc-v1 | /home/jungfrau/AareLC/models/yolo26n-seg.pt | yolo26n-seg-alc-v1-noflip-2026-09-28 | 0.835 | 0.881 | yes | 2026-09-28T21:30 |
|
||||
| 2026-09-28_alc-v1-overlap | aborted | dataset/2026-09-28_alc-v1 | /home/jungfrau/AareLC/models/yolo26n-seg.pt | - | - | - | - | 2026-09-28T23:55 |
|
||||
|
||||
States: exported → queued → training → measured → gate_passed → gate_failed → promoted → deployed (or aborted).
|
||||
|
||||
@@ -20,6 +24,13 @@ States: exported → queued → training → measured → gate_passed → gate_f
|
||||
- 2026-09-28T19:17:49 · gate_passed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-overlap-false-2026-04-12/gold_report.json", "f1_delta": 0.0452, "mask_iou_delta": 0.0327, "mask_score": 0.7103, "mask_score_baseline": 0.6775, "matched_only_iou_delta": -0.0217, "class_f1_delta": {"loop_all": 0.0786, "pin": 0.0339}, "pass": true}
|
||||
- 2026-09-28T19:17:49 · promoted · jungfrau@mx-spark-01 — by the gate
|
||||
|
||||
## 2026-09-28_alc-v1-800px
|
||||
|
||||
- 2026-09-28T23:55:28 · exported · jungfrau@mx-spark-01 — training size 800 px; same dataset and gold as 2026-09-28_alc-v1
|
||||
- 2026-09-28T23:55:28 · training · jungfrau@mx-spark-01 — train_run.py
|
||||
- 2026-09-29T00:31:00 · measured · jungfrau@mx-spark-01
|
||||
- 2026-09-29T00:31:00 · gate_failed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-2026-09-28/gold_report.json", "f1_delta": -0.0123, "mask_iou_delta": -0.0099, "mask_score": 0.7004, "mask_score_baseline": 0.7103, "matched_only_iou_delta": 0.0143, "class_f1_delta": {"loop_all": -0.0063, "pin": 0.0151}, "pass": false}
|
||||
|
||||
## 2026-09-28_alc-v1-humanonly
|
||||
|
||||
- 2026-09-28T20:01:45 · exported · jungfrau@mx-spark-01 — twin of 2026-09-28_alc-v1 without the 163 accepted-unchanged frames; to be trained on the cluster
|
||||
@@ -27,3 +38,23 @@ States: exported → queued → training → measured → gate_passed → gate_f
|
||||
- 2026-09-28T20:42:32 · measured · jungfrau@mx-spark-01
|
||||
- 2026-09-28T20:42:32 · gate_passed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-2026-09-28/gold_report.json", "f1_delta": -0.0004, "mask_iou_delta": 0.0113, "mask_score": 0.7215, "mask_score_baseline": 0.7103, "matched_only_iou_delta": -0.0033, "class_f1_delta": {"loop_all": 0.0292, "pin": 0.0042}, "pass": true}
|
||||
- 2026-09-28T20:44:27 · gate_passed · jungfrau@mx-spark-01 — gate passed vs the beamline baseline; not promoted: on the 66 clean frames B ties A on F1 (0.851 vs 0.853), mask IoU no tighter (0.878 vs 0.883), crystal F1 0.40 vs 0.62; A stays the candidate
|
||||
|
||||
## 2026-09-28_alc-v1-mask1
|
||||
|
||||
- 2026-09-28T21:30:31 · exported · jungfrau@mx-spark-01 — full-resolution masks (mask_ratio=1); same dataset and gold as 2026-09-28_alc-v1
|
||||
- 2026-09-28T21:30:31 · training · jungfrau@mx-spark-01 — train_run.py
|
||||
- 2026-09-28T23:54:44 · measured · jungfrau@mx-spark-01
|
||||
- 2026-09-28T23:54:44 · gate_failed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-2026-09-28/gold_report.json", "f1_delta": -0.0003, "mask_iou_delta": -0.0013, "mask_score": 0.709, "mask_score_baseline": 0.7103, "matched_only_iou_delta": 0.0072, "class_f1_delta": {"loop_all": -0.0063, "pin": 0.0196}, "pass": false}
|
||||
|
||||
## 2026-09-28_alc-v1-noflip
|
||||
|
||||
- 2026-09-28T21:03:25 · exported · jungfrau@mx-spark-01 — no horizontal flip (fliplr=0): the camera never shows the pin on the left; same dataset and gold as 2026-09-28_alc-v1
|
||||
- 2026-09-28T21:03:25 · training · jungfrau@mx-spark-01 — train_run.py
|
||||
- 2026-09-28T21:30:30 · measured · jungfrau@mx-spark-01
|
||||
- 2026-09-28T21:30:30 · gate_passed · jungfrau@mx-spark-01 — {"baseline": "/home/jungfrau/AareLC_train/models/registry/yolo26n-seg-alc-v1-2026-09-28/gold_report.json", "f1_delta": 0.0142, "mask_iou_delta": 0.0093, "mask_score": 0.7196, "mask_score_baseline": 0.7103, "matched_only_iou_delta": -0.0001, "class_f1_delta": {"loop_all": 0.0168, "pin": 0.0144}, "pass": true}
|
||||
|
||||
## 2026-09-28_alc-v1-overlap
|
||||
|
||||
- 2026-09-28T23:54:53 · exported · jungfrau@mx-spark-01 — overlap_mask on; same dataset and gold as 2026-09-28_alc-v1
|
||||
- 2026-09-28T23:54:53 · training · jungfrau@mx-spark-01 — train_run.py
|
||||
- 2026-09-28T23:55:32 · aborted · jungfrau@mx-spark-01 — skipped on purpose: overlap_mask merges nested masks (crystal inside loop_face inside loop_all), so the setting cannot be right for these labels
|
||||
@@ -71,6 +71,76 @@
|
||||
"iou_threshold": 0.5
|
||||
}
|
||||
},
|
||||
"2026-09-28_alc-v1-800px": {
|
||||
"per_class": {
|
||||
"loop_all": {
|
||||
"tp": 59,
|
||||
"fp": 10,
|
||||
"fn": 5,
|
||||
"precision": 0.855072463768116,
|
||||
"recall": 0.921875,
|
||||
"f1": 0.887218045112782,
|
||||
"mean_mask_iou": 0.8815339593847193,
|
||||
"n_matched": 59
|
||||
},
|
||||
"pin": {
|
||||
"tp": 56,
|
||||
"fp": 5,
|
||||
"fn": 3,
|
||||
"precision": 0.9180327868852459,
|
||||
"recall": 0.9491525423728814,
|
||||
"f1": 0.9333333333333333,
|
||||
"mean_mask_iou": 0.9197473537826396,
|
||||
"n_matched": 56
|
||||
},
|
||||
"crystal": {
|
||||
"tp": 4,
|
||||
"fp": 2,
|
||||
"fn": 3,
|
||||
"precision": 0.6666666666666666,
|
||||
"recall": 0.5714285714285714,
|
||||
"f1": 0.6153846153846153,
|
||||
"mean_mask_iou": 0.8425290211607122,
|
||||
"n_matched": 4
|
||||
},
|
||||
"loop_face": {
|
||||
"tp": 30,
|
||||
"fp": 11,
|
||||
"fn": 23,
|
||||
"precision": 0.7317073170731707,
|
||||
"recall": 0.5660377358490566,
|
||||
"f1": 0.6382978723404255,
|
||||
"mean_mask_iou": 0.8851212986608521,
|
||||
"n_matched": 30
|
||||
},
|
||||
"ice": {
|
||||
"tp": 0,
|
||||
"fp": 0,
|
||||
"fn": 2,
|
||||
"precision": 0.0,
|
||||
"recall": 0.0,
|
||||
"f1": 0.0,
|
||||
"mean_mask_iou": 0.0,
|
||||
"n_matched": 0
|
||||
}
|
||||
},
|
||||
"overall": {
|
||||
"tp": 149,
|
||||
"fp": 28,
|
||||
"fn": 36,
|
||||
"precision": 0.8418079096045198,
|
||||
"recall": 0.8054054054054054,
|
||||
"f1": 0.8232044198895028,
|
||||
"mean_mask_iou": 0.8955712111409039,
|
||||
"n_matched": 149
|
||||
},
|
||||
"dataset": {
|
||||
"gold_images": 66,
|
||||
"pred_images": 120,
|
||||
"missing_predictions": 0,
|
||||
"iou_threshold": 0.5
|
||||
}
|
||||
},
|
||||
"2026-09-28_alc-v1-humanonly": {
|
||||
"per_class": {
|
||||
"loop_all": {
|
||||
@@ -141,6 +211,76 @@
|
||||
"iou_threshold": 0.5
|
||||
}
|
||||
},
|
||||
"2026-09-28_alc-v1-mask1": {
|
||||
"per_class": {
|
||||
"loop_all": {
|
||||
"tp": 59,
|
||||
"fp": 12,
|
||||
"fn": 5,
|
||||
"precision": 0.8309859154929577,
|
||||
"recall": 0.921875,
|
||||
"f1": 0.874074074074074,
|
||||
"mean_mask_iou": 0.8836420109433821,
|
||||
"n_matched": 59
|
||||
},
|
||||
"pin": {
|
||||
"tp": 55,
|
||||
"fp": 2,
|
||||
"fn": 4,
|
||||
"precision": 0.9649122807017544,
|
||||
"recall": 0.9322033898305084,
|
||||
"f1": 0.9482758620689654,
|
||||
"mean_mask_iou": 0.9211536151590175,
|
||||
"n_matched": 55
|
||||
},
|
||||
"crystal": {
|
||||
"tp": 3,
|
||||
"fp": 3,
|
||||
"fn": 4,
|
||||
"precision": 0.5,
|
||||
"recall": 0.42857142857142855,
|
||||
"f1": 0.4615384615384615,
|
||||
"mean_mask_iou": 0.8327909254868483,
|
||||
"n_matched": 3
|
||||
},
|
||||
"loop_face": {
|
||||
"tp": 39,
|
||||
"fp": 7,
|
||||
"fn": 14,
|
||||
"precision": 0.8478260869565217,
|
||||
"recall": 0.7358490566037735,
|
||||
"f1": 0.7878787878787878,
|
||||
"mean_mask_iou": 0.8596259543418816,
|
||||
"n_matched": 39
|
||||
},
|
||||
"ice": {
|
||||
"tp": 1,
|
||||
"fp": 0,
|
||||
"fn": 1,
|
||||
"precision": 1.0,
|
||||
"recall": 0.5,
|
||||
"f1": 0.6666666666666666,
|
||||
"mean_mask_iou": 0.8958333333333334,
|
||||
"n_matched": 1
|
||||
}
|
||||
},
|
||||
"overall": {
|
||||
"tp": 157,
|
||||
"fp": 24,
|
||||
"fn": 28,
|
||||
"precision": 0.8674033149171271,
|
||||
"recall": 0.8486486486486486,
|
||||
"f1": 0.8579234972677595,
|
||||
"mean_mask_iou": 0.8899232217103997,
|
||||
"n_matched": 157
|
||||
},
|
||||
"dataset": {
|
||||
"gold_images": 66,
|
||||
"pred_images": 120,
|
||||
"missing_predictions": 0,
|
||||
"iou_threshold": 0.5
|
||||
}
|
||||
},
|
||||
"2026-09-28_alc-v1-noflip": {
|
||||
"per_class": {
|
||||
"loop_all": {
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
clean gold frames: 66
|
||||
run F1 recall IoU matched mask score loop/pin/crystal/face/ice F1
|
||||
beamline yolo26n-seg-overlap-false-2026-04-12 0.687 0.67 0.903 0.606 0.71/0.84/0.17/0.63/0.29
|
||||
2026-09-28_alc-v1-800px 0.823 0.81 0.896 0.721 0.89/0.93/0.62/0.64/0.00
|
||||
2026-09-28_alc-v1-humanonly 0.851 0.86 0.878 0.759 0.93/0.94/0.40/0.75/0.67
|
||||
2026-09-28_alc-v1-mask1 0.858 0.85 0.890 0.755 0.87/0.95/0.46/0.79/0.67
|
||||
2026-09-28_alc-v1-noflip 0.846 0.86 0.879 0.755 0.90/0.93/0.47/0.75/0.67
|
||||
2026-09-28_alc-v1 0.853 0.85 0.883 0.749 0.90/0.92/0.62/0.76/0.50
|
||||
Reference in new issue
Block a user