GotthardGandClaude Fable 5.1
831367d35d
A paired bootstrap over the 149 unseen gold frames (2026-10-02) showed that a difference between two models carries about +-0.023 of noise in F1, +-0.018 in the mask score and +-0.10 in crystal F1, while the gate's tolerance was 0.005. - tools/gate_stats.py: per-frame counts and a paired bootstrap (pure Python). - train_run.py: when the baseline's gold predictions exist, the gate requires the candidate to beat it on F1 or the mask score in 90 % of the draws and not to lose the other in more than 90 %; the point rule is the fallback and is always reported (pass_point_rule). - tools/compare_unseen.py replaces the Spark's ~/fair_gate_all.py: all models on the frames none trained on, on the frozen core (data/gold/unseen_core.json, 149 frames) and on all unseen frames once the gold set grows, with the interval and win share against the active model. - measure_weights.py keeps a version's gate verdict when it re-measures it. - tools/spark_cycle.sh: gold growth, export, re-measure, train, gate, compare, push, with what cycles 3 and 4 taught built in. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
211 lines
3.1 KiB
JSON
211 lines
3.1 KiB
JSON
{
|
|
"frozen": "2026-10-02",
|
|
"note": "core = gold frames no model had trained on when the set was frozen; seen = gold frames the April beamline model trained on (or a dataset used)",
|
|
"core": [
|
|
"tpi_10130",
|
|
"tpi_10697",
|
|
"tpi_1119",
|
|
"tpi_14517",
|
|
"tpi_14688",
|
|
"tpi_14689",
|
|
"tpi_14698",
|
|
"tpi_14709",
|
|
"tpi_14711",
|
|
"tpi_14715",
|
|
"tpi_14718",
|
|
"tpi_15087",
|
|
"tpi_16223",
|
|
"tpi_16235",
|
|
"tpi_16237",
|
|
"tpi_16252",
|
|
"tpi_16278",
|
|
"tpi_16292",
|
|
"tpi_16319",
|
|
"tpi_16321",
|
|
"tpi_16325",
|
|
"tpi_16357",
|
|
"tpi_16361",
|
|
"tpi_16747",
|
|
"tpi_16753",
|
|
"tpi_16766",
|
|
"tpi_16770",
|
|
"tpi_16956",
|
|
"tpi_16963",
|
|
"tpi_17889",
|
|
"tpi_20221",
|
|
"tpi_21693",
|
|
"tpi_31669",
|
|
"tpi_31928",
|
|
"tpi_31929",
|
|
"tpi_32762",
|
|
"tpi_33849",
|
|
"tpi_34038",
|
|
"tpi_34389",
|
|
"tpi_36563",
|
|
"tpi_4053",
|
|
"tpi_45649",
|
|
"tpi_52402",
|
|
"tpi_55303",
|
|
"tpi_55371",
|
|
"tpi_55383",
|
|
"tpi_55408",
|
|
"tpi_55416",
|
|
"tpi_55565",
|
|
"tpi_55896",
|
|
"tpi_55912",
|
|
"tpi_55941",
|
|
"tpi_55958",
|
|
"tpi_56012",
|
|
"tpi_56096",
|
|
"tpi_56104",
|
|
"tpi_56715",
|
|
"tpi_56717",
|
|
"tpi_56840",
|
|
"tpi_57070",
|
|
"tpi_57242",
|
|
"tpi_57300",
|
|
"tpi_57342",
|
|
"tpi_57654",
|
|
"tpi_57684",
|
|
"tpi_57702",
|
|
"tpi_57991",
|
|
"tpi_58097",
|
|
"tpi_58229",
|
|
"tpi_58417",
|
|
"tpi_58588",
|
|
"tpi_58595",
|
|
"tpi_58842",
|
|
"tpi_59004",
|
|
"tpi_59011",
|
|
"tpi_59120",
|
|
"tpi_59182",
|
|
"tpi_59312",
|
|
"tpi_59412",
|
|
"tpi_59496",
|
|
"tpi_59564",
|
|
"tpi_59570",
|
|
"tpi_59911",
|
|
"tpi_59988",
|
|
"tpi_60052",
|
|
"tpi_60625",
|
|
"tpi_61066",
|
|
"tpi_61069",
|
|
"tpi_61087",
|
|
"tpi_61317",
|
|
"tpi_61327",
|
|
"tpi_61334",
|
|
"tpi_61340",
|
|
"tpi_61343",
|
|
"tpi_61344",
|
|
"tpi_61350",
|
|
"tpi_61358",
|
|
"tpi_61361",
|
|
"tpi_61364",
|
|
"tpi_61372",
|
|
"tpi_61374",
|
|
"tpi_61379",
|
|
"tpi_61380",
|
|
"tpi_61381",
|
|
"tpi_61385",
|
|
"tpi_61392",
|
|
"tpi_61411",
|
|
"tpi_61419",
|
|
"tpi_61428",
|
|
"tpi_61430",
|
|
"tpi_61436",
|
|
"tpi_61437",
|
|
"tpi_61439",
|
|
"tpi_61441",
|
|
"tpi_61442",
|
|
"tpi_61443",
|
|
"tpi_61445",
|
|
"tpi_61447",
|
|
"tpi_61449",
|
|
"tpi_61450",
|
|
"tpi_61452",
|
|
"tpi_61457",
|
|
"tpi_61565",
|
|
"tpi_61572",
|
|
"tpi_61579",
|
|
"tpi_61581",
|
|
"tpi_61594",
|
|
"tpi_61610",
|
|
"tpi_61613",
|
|
"tpi_61618",
|
|
"tpi_61627",
|
|
"tpi_61629",
|
|
"tpi_61633",
|
|
"tpi_61635",
|
|
"tpi_61651",
|
|
"tpi_61653",
|
|
"tpi_61658",
|
|
"tpi_61670",
|
|
"tpi_61686",
|
|
"tpi_61702",
|
|
"tpi_61769",
|
|
"tpi_61834",
|
|
"tpi_61839",
|
|
"tpi_61865",
|
|
"tpi_6203",
|
|
"tpi_7667",
|
|
"tpi_8507",
|
|
"tpi_8744",
|
|
"tpi_9752"
|
|
],
|
|
"seen": [
|
|
"tpi_10036",
|
|
"tpi_10948",
|
|
"tpi_11106",
|
|
"tpi_11133",
|
|
"tpi_11523",
|
|
"tpi_12381",
|
|
"tpi_12548",
|
|
"tpi_12567",
|
|
"tpi_12580",
|
|
"tpi_1265",
|
|
"tpi_12804",
|
|
"tpi_14167",
|
|
"tpi_14524",
|
|
"tpi_14530",
|
|
"tpi_14552",
|
|
"tpi_14642",
|
|
"tpi_14863",
|
|
"tpi_2014",
|
|
"tpi_2640",
|
|
"tpi_3077",
|
|
"tpi_3466",
|
|
"tpi_3550",
|
|
"tpi_4296",
|
|
"tpi_4313",
|
|
"tpi_4528",
|
|
"tpi_5066",
|
|
"tpi_5178",
|
|
"tpi_5225",
|
|
"tpi_5387",
|
|
"tpi_5425",
|
|
"tpi_5725",
|
|
"tpi_5956",
|
|
"tpi_6202",
|
|
"tpi_6209",
|
|
"tpi_6730",
|
|
"tpi_6737",
|
|
"tpi_6787",
|
|
"tpi_7072",
|
|
"tpi_7302",
|
|
"tpi_7348",
|
|
"tpi_7532",
|
|
"tpi_7658",
|
|
"tpi_7683",
|
|
"tpi_7751",
|
|
"tpi_7810",
|
|
"tpi_7836",
|
|
"tpi_7879",
|
|
"tpi_8142",
|
|
"tpi_8196",
|
|
"tpi_8348",
|
|
"tpi_8400",
|
|
"tpi_8715",
|
|
"tpi_8855",
|
|
"tpi_9805"
|
|
]
|
|
} |