Merge: per-observation outlier rejection (env-gated PR_REJECT)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 24m50s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 26m45s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 26m58s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 28m30s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 29m47s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 30m59s
Build Packages / build:rpm (rocky8) (push) Successful in 25m11s
Build Packages / XDS test (durin plugin) (push) Successful in 20m7s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 24m53s
Build Packages / Generate python client (push) Successful in 26s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m52s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 1m28s
Build Packages / build:rpm (rocky9) (push) Successful in 27m58s
Build Packages / DIALS test (push) Successful in 32m4s
Build Packages / XDS test (neggia plugin) (push) Successful in 13m6s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 14m59s
Build Packages / Unit tests (push) Successful in 2h15m31s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 24m50s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 26m45s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 26m58s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 28m30s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 29m47s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 30m59s
Build Packages / build:rpm (rocky8) (push) Successful in 25m11s
Build Packages / XDS test (durin plugin) (push) Successful in 20m7s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 24m53s
Build Packages / Generate python client (push) Successful in 26s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m52s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 1m28s
Build Packages / build:rpm (rocky9) (push) Successful in 27m58s
Build Packages / DIALS test (push) Successful in 32m4s
Build Packages / XDS test (neggia plugin) (push) Successful in 13m6s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 14m59s
Build Packages / Unit tests (push) Successful in 2h15m31s
At the jet's ~1000x multiplicity R-free is bias-limited, and the merge had NO outlier rejection - serial data zingers/overlaps/mis-indexed frames bias every merged mean. Add a robust per-observation cut: drop observations whose corrected intensity lies > reject_nsigma error-model sigmas from the reflection's MEDIAN. The error-model sigma already captures the genuine (partiality) scatter, and the median is a robust centre, so only the tail beyond the real spread is removed - not good partials. The median is computed in RefineErrorModel (which already pools the observations per reflection); AddImage applies the cut. Env-gated via PR_REJECT=<nsigma> (off by default); logs the count removed. On the jet (CC proxy) it lifts CCref +8 (nsigma 6, 0.6% cut) to +11 (nsigma 3, 7.4% cut) - the cut is vs the data's own median, not the reference, so the gain is real cleaner means. R-free validation + the nsigma sweet spot (over-rejection risk at low nsigma) are for Filip's full-jet R-free. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -1034,8 +1034,15 @@ int main(int argc, char **argv) {
|
||||
const double isa = (b > 0.0) ? 1.0 / b : std::numeric_limits<double>::infinity();
|
||||
logger.Info("Error model: sigma'^2 = {:.3f} sigma^2 + ({:.4f} I)^2 ISa = {:.1f}", a, b, isa);
|
||||
}
|
||||
if (const char *rj = std::getenv("PR_REJECT")) { // TEMP A/B: per-observation outlier rejection
|
||||
const double nsig = std::atof(rj);
|
||||
merge_engine.SetRejectOutliers(nsig);
|
||||
logger.Info("Outlier rejection enabled: dropping observations > {:.1f} sigma from the per-reflection median", nsig);
|
||||
}
|
||||
for (auto &i : indexer.GetIntegrationOutcome())
|
||||
merge_engine.AddImage(i);
|
||||
if (merge_engine.RejectedCount() > 0)
|
||||
logger.Info("Outlier rejection removed {} observations", merge_engine.RejectedCount());
|
||||
|
||||
auto merged_reflections = merge_engine.ExportReflections();
|
||||
auto merged_statistics = merge_engine.MergeStats(merged_reflections, indexer.GetIntegrationOutcome(), reference_data);
|
||||
|
||||
Reference in New Issue
Block a user