From db9cc9106fb871f8e120274de58b569765aeb2fe Mon Sep 17 00:00:00 2001 From: Filip Leonarski Date: Sat, 5 Sep 2026 13:41:16 +0200 Subject: [PATCH] rugnux: the per-reflection correction factor is named for what it is, not for what it once held The factor multiplied into each integrated intensity was called rlp, for reciprocal Lorentz-polarization, and until this week that is all it held. It now also carries the sensor efficiency at the angle the beam arrives, and on the stills path it holds that efficiency and the polarization with no Lorentz term at all - correctly, since the Lorentz factor of a still is one. Three different products under one name that promises exactly one of them, in code where the neighbouring member is the total correction. Rename it prescaling_corr: multiplicative, applied before scaling, therefore not a scale, and silent about its contents - which is the point, since the contents have now grown twice. It is also what DIALS calls the same product. The stills refinement member spelled "1 / rlp" becomes inv_corr, and the comments and usage text that promised "the Lorentz-polarization factor and nothing else" now say what is actually there. The Lorentz term keeps its own name where it is computed, because that name is correct. The two external spellings are untouched: the CBOR key and the reflection dataset are a published format, and a reader that meets an unknown key would take the factor as zero, which both the merge key and the ingest treat as a reflection to drop - so every reflection would vanish and the run would still exit zero. No output changes: the merged and unmerged files of two full runs are byte for byte what the previous binary wrote, four stored files from before the efficiency correction still re-scale identically, and the reflection datasets of the process file are unchanged. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N --- common/Reflection.h | 11 ++++- docs/CBOR.md | 2 +- docs/CPU_DATA_ANALYSIS_INTEGRATION.md | 6 +-- docs/HDF5.md | 2 +- docs/RUGNUX_ADVANCED.md | 2 +- docs/RUGNUX_INTEGRATION.md | 8 ++-- frame_serialize/CBORStream2Deserializer.cpp | 2 +- frame_serialize/CBORStream2Serializer.cpp | 2 +- image_analysis/UpdateReflectionResolution.cpp | 2 +- image_analysis/WriteReflections.cpp | 48 +++++++++++-------- image_analysis/WriteReflections.h | 6 ++- .../BraggIntegrationEngine.cpp | 4 +- .../bragg_prediction/BraggPrediction.cpp | 2 +- .../bragg_prediction/BraggPredictionGPU.cu | 2 +- .../bragg_prediction/BraggPredictionRot.cpp | 2 +- .../bragg_prediction/BraggPredictionRotGPU.cu | 2 +- .../scale_merge/AnisotropyAnalysis.cpp | 8 ++-- image_analysis/scale_merge/HKLKey.cpp | 4 +- .../scale_merge/ReindexAmbiguity.cpp | 2 +- .../scale_merge/RotationScaleMerge.cpp | 31 ++++++------ .../scale_merge/RotationScaleMerge.h | 8 ++-- .../scale_merge/RotationScaleMergeGPU.cu | 28 +++++------ .../scale_merge/RotationScaleMergeGPU.h | 2 +- image_analysis/scale_merge/ScaleOnTheFly.cpp | 6 +-- .../scale_merge/StillsPartialityRefine.cpp | 23 ++++----- reader/HDF5MetadataSource.cpp | 2 +- rugnux/rugnux_cli.cpp | 2 +- tests/AnisotropyAnalysisTest.cpp | 2 +- ...ggIntegrationEngineCompressedImageTest.cpp | 2 +- tests/BraggIntegrationEngineGPUTest.cpp | 2 +- tests/CBORTest.cpp | 10 ++-- tests/JFJochReaderTest.cpp | 4 +- tests/MergeScaleTest.cpp | 2 +- tools/jfjoch_extract_hkl.cpp | 2 +- .../JFJochViewerReflectionListWindow.cpp | 2 +- writer/HDF5DataFilePluginReflection.cpp | 2 +- 36 files changed, 134 insertions(+), 113 deletions(-) diff --git a/common/Reflection.h b/common/Reflection.h index 1bba51809..e3e86ec17 100644 --- a/common/Reflection.h +++ b/common/Reflection.h @@ -25,10 +25,17 @@ struct Reflection { float var_bkg; // non-signal (background) part of sigma^2, carried to the merge float sigma; float dist_ewald; - float rlp; + // Everything known a priori that multiplies the raw integrated count on its way to a quantity + // proportional to |F|^2: the reciprocal Lorentz factor, the reciprocal polarization factor and + // the sensor's angle-dependent efficiency, whichever of them the path that filled it applies. + // It is deliberately NOT an enumeration of those terms - it is the product of every deterministic + // per-reflection correction, and that set has grown before and will grow again. What it is not is + // a scale: the fitted per-image scale and the partiality stay out of it and are divided in + // separately below. (Named after DIALS's prescaling_correction, which holds the same product.) + float prescaling_corr; float partiality; // fraction of the reflection recorded in the sampled (rocking) slice float zeta; - float image_scale_corr; // I_true = scaling_correction * I; scaling_correction = rlp / (partiality * image_scale) + float image_scale_corr; // I_true = image_scale_corr * I; = prescaling_corr / (partiality * image_scale) bool observed = false; bool on_ice_ring = false; // sits on a hexagonal-ice powder ring: excluded from scaling, kept for merging }; diff --git a/docs/CBOR.md b/docs/CBOR.md index 536a11803..8be543079 100644 --- a/docs/CBOR.md +++ b/docs/CBOR.md @@ -167,7 +167,7 @@ See [DECTRIS documentation](https://github.com/dectris/documentation/tree/main/s | - sigma | float | standard deviation, estimated from counting statistics (photons) | | | | - image | float | image number (present for each spot) | | | | - rp | float | Distance to Ewald sphere \[Angstrom^-1\] | | | -| - rlp | float | Reciprocal Lorentz and polarization corrections | | | +| - rlp | float | Prescaling correction: the product of every deterministic per-reflection correction (reciprocal Lorentz, reciprocal polarization, sensor efficiency). The key name is historical and kept for compatibility | | | | - partiality | float | Partiality of the reflection | | | | - phi | float | phi angle from XDS: difference from middle of current frame, not absolute \[deg\] | | | | - zeta | float | Lorentz zeta factor (reciprocal-space geometry term) | | | diff --git a/docs/CPU_DATA_ANALYSIS_INTEGRATION.md b/docs/CPU_DATA_ANALYSIS_INTEGRATION.md index 5c3d0ca14..70a16cee8 100644 --- a/docs/CPU_DATA_ANALYSIS_INTEGRATION.md +++ b/docs/CPU_DATA_ANALYSIS_INTEGRATION.md @@ -159,9 +159,9 @@ where $c$ is the pixel value and the de-biased variance $v$ (background plus mod The integrator is selected by `--integrator boxsum|gaussian|empirical` (default `gaussian`). -### 9.4 Lorentz–polarization factor handling +### 9.4 The prescaling correction -For integrated reflections, polarization correction can be applied as a multiplicative correction to the reflection scale via the geometry-based polarization term (§2.2). A Lorentz-like factor is carried as `rlp` in predictions, and used during scaling/merging (§10). +Every deterministic per-reflection correction is carried as one multiplicative factor, `prescaling_corr`, which the prediction fills and the integrator adds to. It currently holds the reciprocal Lorentz factor (rotation only), the reciprocal polarization factor from the geometry-based term (§2.2), and the sensor's angle-dependent efficiency (§9.6); it is a product, not a fixed list, and terms have been added to it before. Scaling and merging (§10) use it as the numerator of the reflection's total correction. It is not a scale — the fitted per-image scale and the partiality are separate. --- @@ -204,7 +204,7 @@ $ where: - $G_i$ is the image scale factor, -- $L_{ij}$ is a Lorentz-like / geometry factor; predictions carry its **reciprocal** as `rlp`, so $L = 1/\texttt{rlp}$ and the correction below is applied as a multiplication by `rlp`, +- $L_{ij}$ is the prescaling correction of §9.4; predictions carry its **reciprocal** as `prescaling_corr`, so $L = 1/\texttt{prescaling\_corr}$ and the correction below is applied as a multiplication by `prescaling_corr`, - $P_{ij}$ is a partiality term (model-dependent), - $I_h$ is the merged (true) intensity parameter for that unique reflection. diff --git a/docs/HDF5.md b/docs/HDF5.md index 52a46f29c..2b5c4c7ea 100644 --- a/docs/HDF5.md +++ b/docs/HDF5.md @@ -435,7 +435,7 @@ mostly onto the standard | `predicted_x`, `predicted_y` | pixel | name standard, units differ | predicted position. NXreflections `predicted_x/_y` are *physical* lengths; the pixel datasets are `predicted_px_x/_y` | | `observed_x`, `observed_y` | pixel | name standard, units differ | observed centroid (pixels; standard pixel form is `observed_px_x/_y`) | | `observed_frame` | | standard | image number of the reflection | -| `lp` | | standard | Lorentz–polarization factor (stored as `1/rlp`) | +| `lp` | | standard | reciprocal of the prescaling correction (stored as `1/prescaling_corr`). The NXreflections name is `lp`, but the stored product also carries the sensor efficiency term | | `partiality` | | standard | recorded fraction of the reflection | | `delta_phi` | deg | **extension** | XDS Δφ: offset from the centre of the current frame | | `zeta` | | **extension** | Lorentz ζ factor (reciprocal-space geometry term) | diff --git a/docs/RUGNUX_ADVANCED.md b/docs/RUGNUX_ADVANCED.md index 24daa9691..8649a450e 100644 --- a/docs/RUGNUX_ADVANCED.md +++ b/docs/RUGNUX_ADVANCED.md @@ -335,7 +335,7 @@ Scaling and merging: | `--reference-column