rugnux: the per-reflection correction factor is named for what it is, not for what it once held
The factor multiplied into each integrated intensity was called rlp, for reciprocal Lorentz-polarization, and until this week that is all it held. It now also carries the sensor efficiency at the angle the beam arrives, and on the stills path it holds that efficiency and the polarization with no Lorentz term at all - correctly, since the Lorentz factor of a still is one. Three different products under one name that promises exactly one of them, in code where the neighbouring member is the total correction. Rename it prescaling_corr: multiplicative, applied before scaling, therefore not a scale, and silent about its contents - which is the point, since the contents have now grown twice. It is also what DIALS calls the same product. The stills refinement member spelled "1 / rlp" becomes inv_corr, and the comments and usage text that promised "the Lorentz-polarization factor and nothing else" now say what is actually there. The Lorentz term keeps its own name where it is computed, because that name is correct. The two external spellings are untouched: the CBOR key and the reflection dataset are a published format, and a reader that meets an unknown key would take the factor as zero, which both the merge key and the ingest treat as a reflection to drop - so every reflection would vanish and the run would still exit zero. No output changes: the merged and unmerged files of two full runs are byte for byte what the previous binary wrote, four stored files from before the efficiency correction still re-scale identically, and the reflection datasets of the process file are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
This commit is contained in:
1 parent
d6ccdbdb61
commit
db9cc9106f
36 files changed
+134
-113
No files matched your search
+9
-2
@@ -25,10 +25,17 @@ struct Reflection {
|
||||
float var_bkg; // non-signal (background) part of sigma^2, carried to the merge
|
||||
float sigma;
|
||||
float dist_ewald;
|
||||
float rlp;
|
||||
// Everything known a priori that multiplies the raw integrated count on its way to a quantity
|
||||
// proportional to |F|^2: the reciprocal Lorentz factor, the reciprocal polarization factor and
|
||||
// the sensor's angle-dependent efficiency, whichever of them the path that filled it applies.
|
||||
// It is deliberately NOT an enumeration of those terms - it is the product of every deterministic
|
||||
// per-reflection correction, and that set has grown before and will grow again. What it is not is
|
||||
// a scale: the fitted per-image scale and the partiality stay out of it and are divided in
|
||||
// separately below. (Named after DIALS's prescaling_correction, which holds the same product.)
|
||||
float prescaling_corr;
|
||||
float partiality; // fraction of the reflection recorded in the sampled (rocking) slice
|
||||
float zeta;
|
||||
float image_scale_corr; // I_true = scaling_correction * I; scaling_correction = rlp / (partiality * image_scale)
|
||||
float image_scale_corr; // I_true = image_scale_corr * I; = prescaling_corr / (partiality * image_scale)
|
||||
bool observed = false;
|
||||
bool on_ice_ring = false; // sits on a hexagonal-ice powder ring: excluded from scaling, kept for merging
|
||||
};
|
||||
|
||||
+1
-1
@@ -167,7 +167,7 @@ See [DECTRIS documentation](https://github.com/dectris/documentation/tree/main/s
|
||||
| - sigma | float | standard deviation, estimated from counting statistics (photons) | | |
|
||||
| - image | float | image number (present for each spot) | | |
|
||||
| - rp | float | Distance to Ewald sphere \[Angstrom^-1\] | | |
|
||||
| - rlp | float | Reciprocal Lorentz and polarization corrections | | |
|
||||
| - rlp | float | Prescaling correction: the product of every deterministic per-reflection correction (reciprocal Lorentz, reciprocal polarization, sensor efficiency). The key name is historical and kept for compatibility | | |
|
||||
| - partiality | float | Partiality of the reflection | | |
|
||||
| - phi | float | phi angle from XDS: difference from middle of current frame, not absolute \[deg\] | | |
|
||||
| - zeta | float | Lorentz zeta factor (reciprocal-space geometry term) | | |
|
||||
|
||||
@@ -159,9 +159,9 @@ where $c$ is the pixel value and the de-biased variance $v$ (background plus mod
|
||||
|
||||
The integrator is selected by `--integrator boxsum|gaussian|empirical` (default `gaussian`).
|
||||
|
||||
### 9.4 Lorentz–polarization factor handling
|
||||
### 9.4 The prescaling correction
|
||||
|
||||
For integrated reflections, polarization correction can be applied as a multiplicative correction to the reflection scale via the geometry-based polarization term (§2.2). A Lorentz-like factor is carried as `rlp` in predictions, and used during scaling/merging (§10).
|
||||
Every deterministic per-reflection correction is carried as one multiplicative factor, `prescaling_corr`, which the prediction fills and the integrator adds to. It currently holds the reciprocal Lorentz factor (rotation only), the reciprocal polarization factor from the geometry-based term (§2.2), and the sensor's angle-dependent efficiency (§9.6); it is a product, not a fixed list, and terms have been added to it before. Scaling and merging (§10) use it as the numerator of the reflection's total correction. It is not a scale — the fitted per-image scale and the partiality are separate.
|
||||
|
||||
---
|
||||
|
||||
@@ -204,7 +204,7 @@ $
|
||||
where:
|
||||
|
||||
- $G_i$ is the image scale factor,
|
||||
- $L_{ij}$ is a Lorentz-like / geometry factor; predictions carry its **reciprocal** as `rlp`, so $L = 1/\texttt{rlp}$ and the correction below is applied as a multiplication by `rlp`,
|
||||
- $L_{ij}$ is the prescaling correction of §9.4; predictions carry its **reciprocal** as `prescaling_corr`, so $L = 1/\texttt{prescaling\_corr}$ and the correction below is applied as a multiplication by `prescaling_corr`,
|
||||
- $P_{ij}$ is a partiality term (model-dependent),
|
||||
- $I_h$ is the merged (true) intensity parameter for that unique reflection.
|
||||
|
||||
|
||||
+1
-1
@@ -435,7 +435,7 @@ mostly onto the standard
|
||||
| `predicted_x`, `predicted_y` | pixel | name standard, units differ | predicted position. NXreflections `predicted_x/_y` are *physical* lengths; the pixel datasets are `predicted_px_x/_y` |
|
||||
| `observed_x`, `observed_y` | pixel | name standard, units differ | observed centroid (pixels; standard pixel form is `observed_px_x/_y`) |
|
||||
| `observed_frame` | | standard | image number of the reflection |
|
||||
| `lp` | | standard | Lorentz–polarization factor (stored as `1/rlp`) |
|
||||
| `lp` | | standard | reciprocal of the prescaling correction (stored as `1/prescaling_corr`). The NXreflections name is `lp`, but the stored product also carries the sensor efficiency term |
|
||||
| `partiality` | | standard | recorded fraction of the reflection |
|
||||
| `delta_phi` | deg | **extension** | XDS Δφ: offset from the centre of the current frame |
|
||||
| `zeta` | | **extension** | Lorentz ζ factor (reciprocal-space geometry term) |
|
||||
|
||||
@@ -335,7 +335,7 @@ Scaling and merging:
|
||||
| `--reference-column <label>` | Reference MTZ column to use (default: auto — F-model, else IMEAN/I/…) |
|
||||
| `--model <file>` | Validate the merged intensities against this atomic model (PDB or mmCIF, gzipped or not; the format is taken from the file's content) — R-work / R-free and maps (see [Validating against a model](#validating-against-a-model-rugnux-model)). It also settles the frame the reflections are written in: the enantiomorph, and the [indexing ambiguity](#the-indexing-ambiguity) where no `-z` did. For serial stills given `-C` / `-S`, the model's structure factors become the per-image reference |
|
||||
| `--write-process-h5` | Also write the (large) `_process.h5` when merging (default: only `.mtz`/`.cif`) |
|
||||
| `--export-unmerged` | Write `<prefix>_unmerged.mtz`, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. **On by default** whenever there is an output prefix. Rotation partials are summed into one full per reflection. Intensities carry the Lorentz-polarization factor and nothing else — the partiality is not divided out and the per-image scale is not applied. Written in `--mode mx` and `--mode scale`, and with `--no-merge`. See [The unmerged export](RUGNUX_INTEGRATION.md#the-unmerged-export) |
|
||||
| `--export-unmerged` | Write `<prefix>_unmerged.mtz`, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. **On by default** whenever there is an output prefix. Rotation partials are summed into one full per reflection. Intensities carry the Lorentz, polarization and sensor-efficiency corrections and nothing else, with the applied factor in the `LP` column — the partiality is not divided out and the per-image scale is not applied. Written in `--mode mx` and `--mode scale`, and with `--no-merge`. See [The unmerged export](RUGNUX_INTEGRATION.md#the-unmerged-export) |
|
||||
| `--no-export-unmerged` | Do not write `<prefix>_unmerged.mtz`. It is the largest file a run produces, so a run whose observations are not going to another scaling program can skip it |
|
||||
| `--no-p1-crosscheck` | Do not write `<prefix>_P1.mtz`, the P1 cross-check merge described under [Output files](RUGNUX_TUTORIAL.md#output-files). Every rotation run that determines its own space group writes it; a run given `-S` writes none either way |
|
||||
| `--export-unmerged-partials` | Write `<prefix>_unmerged_partials.mtz`, the same observations with each partial as its own row (one batch per image) for the reading program to sum. Off by default, and independent of `--export-unmerged` |
|
||||
|
||||
@@ -119,9 +119,11 @@ leaves an unmerged file in, and the order AIMLESS requires of its input — so b
|
||||
file directly.
|
||||
|
||||
**What has been applied to the intensities, and what has not.** `I` and `SIGI` carry the
|
||||
**Lorentz-polarization factor and nothing else**; the factor itself is in the `LP` column, so raw
|
||||
counts are `I/LP`. LP is applied because it is per-observation geometry that varies by more than two
|
||||
orders of magnitude across a sweep and no reader can reconstruct it. Deliberately *not* applied:
|
||||
**prescaling correction and nothing else** — the reciprocal Lorentz factor, the reciprocal
|
||||
polarization factor and the sensor's angle-dependent efficiency. The factor itself is in the `LP`
|
||||
column, so raw counts are `I/LP`. It is applied because it is per-observation geometry and instrument
|
||||
response that varies by more than two orders of magnitude across a sweep and no reader can
|
||||
reconstruct it. Deliberately *not* applied:
|
||||
the **partiality is not divided out** (it is reported in `FRACTIONCALC`), and the **per-image scale is
|
||||
not applied at all** — those programs fit their own scale model, and handing them pre-scaled data
|
||||
would have them fit a correction to a correction. No resolution cut, outlier rejection or ice-ring
|
||||
|
||||
@@ -516,7 +516,7 @@ namespace {
|
||||
else if (key == "image")
|
||||
r.image_number = GetCBORFloat(map_value);
|
||||
else if (key == "rlp")
|
||||
r.rlp = GetCBORFloat(map_value);
|
||||
r.prescaling_corr = GetCBORFloat(map_value);
|
||||
else if (key == "rp")
|
||||
r.dist_ewald = GetCBORFloat(map_value);
|
||||
else if (key == "partiality")
|
||||
|
||||
@@ -271,7 +271,7 @@ inline void CBOR_ENC(CborEncoder &encoder, const Reflection& r) {
|
||||
CBOR_ENC(mapEncoder, "sigma", r.sigma);
|
||||
CBOR_ENC(mapEncoder, "image", r.image_number);
|
||||
CBOR_ENC(mapEncoder, "rp", r.dist_ewald);
|
||||
CBOR_ENC(mapEncoder, "rlp", r.rlp);
|
||||
CBOR_ENC(mapEncoder, "rlp", r.prescaling_corr);
|
||||
CBOR_ENC(mapEncoder, "partiality", r.partiality);
|
||||
CBOR_ENC(mapEncoder, "zeta", r.zeta);
|
||||
CBOR_ENC(mapEncoder, "image_scale_corr", r.image_scale_corr);
|
||||
|
||||
@@ -30,7 +30,7 @@ ResolutionStats UpdateReflectionResolution(const UnitCell &cell, std::vector<Int
|
||||
if (ret.d_low < d)
|
||||
ret.d_low = d;
|
||||
r.d = d;
|
||||
r.image_scale_corr = r.rlp / r.partiality;
|
||||
r.image_scale_corr = r.prescaling_corr / r.partiality;
|
||||
} else
|
||||
r.d = NAN;
|
||||
}
|
||||
|
||||
@@ -666,16 +666,16 @@ std::vector<Reflection> SumRockingEvents(const std::vector<IntegrationOutcome> &
|
||||
++j;
|
||||
|
||||
double sum_p = 0.0, sum_I = 0.0, sum_var = 0.0, sum_var_bkg = 0.0;
|
||||
double p_rlp = 0.0, p_frame = 0.0, p_x = 0.0, p_y = 0.0, p_delta_phi = 0.0, p_zeta = 0.0,
|
||||
double p_corr = 0.0, p_frame = 0.0, p_x = 0.0, p_y = 0.0, p_delta_phi = 0.0, p_zeta = 0.0,
|
||||
p_bkg = 0.0;
|
||||
for (size_t m = i; m < j; ++m) {
|
||||
const Reflection &r = *parts[m].r;
|
||||
const double p = r.partiality;
|
||||
sum_p += p;
|
||||
sum_I += static_cast<double>(r.I) * r.rlp;
|
||||
sum_var += static_cast<double>(r.sigma) * r.sigma * r.rlp * r.rlp;
|
||||
sum_var_bkg += static_cast<double>(r.var_bkg) * r.rlp * r.rlp;
|
||||
p_rlp += p * r.rlp;
|
||||
sum_I += static_cast<double>(r.I) * r.prescaling_corr;
|
||||
sum_var += static_cast<double>(r.sigma) * r.sigma * r.prescaling_corr * r.prescaling_corr;
|
||||
sum_var_bkg += static_cast<double>(r.var_bkg) * r.prescaling_corr * r.prescaling_corr;
|
||||
p_corr += p * r.prescaling_corr;
|
||||
p_frame += p * r.image_number;
|
||||
p_x += p * DetectorX(r);
|
||||
p_y += p * DetectorY(r);
|
||||
@@ -688,13 +688,16 @@ std::vector<Reflection> SumRockingEvents(const std::vector<IntegrationOutcome> &
|
||||
if (sum_p < min_partiality || sum_p < min_captured_fraction)
|
||||
continue;
|
||||
|
||||
// The Lorentz-polarization factor is applied by the writer, which multiplies I by rlp, so
|
||||
// divide the event's own factor back out of the sums here. LP is the same geometry for every
|
||||
// part of one event to a median 2e-4, so the file's I/LP is still the raw count sum.
|
||||
full.rlp = static_cast<float>(p_rlp / sum_p);
|
||||
full.I = static_cast<float>(sum_I / full.rlp);
|
||||
full.sigma = static_cast<float>(std::sqrt(sum_var) / full.rlp);
|
||||
full.var_bkg = static_cast<float>(sum_var_bkg / (static_cast<double>(full.rlp) * full.rlp));
|
||||
// The prescaling correction is applied by the writer, which multiplies I by prescaling_corr,
|
||||
// so divide the event's own factor back out of the sums here. Every term in it - Lorentz,
|
||||
// polarization, sensor efficiency - is a function of the reflection's position and rocking
|
||||
// geometry, which move by almost nothing between the parts of one event (median 2e-4 on the
|
||||
// Lorentz term), so the file's I/LP is still the raw count sum.
|
||||
full.prescaling_corr = static_cast<float>(p_corr / sum_p);
|
||||
full.I = static_cast<float>(sum_I / full.prescaling_corr);
|
||||
full.sigma = static_cast<float>(std::sqrt(sum_var) / full.prescaling_corr);
|
||||
full.var_bkg = static_cast<float>(sum_var_bkg
|
||||
/ (static_cast<double>(full.prescaling_corr) * full.prescaling_corr));
|
||||
full.partiality = static_cast<float>(sum_p);
|
||||
full.image_number = static_cast<float>(p_frame / sum_p);
|
||||
full.observed_x = static_cast<float>(p_x / sum_p);
|
||||
@@ -762,11 +765,16 @@ void WriteUnmergedMtzReflections(const std::vector<IntegrationOutcome> &outcomes
|
||||
// them there, so the index the reflection was actually measured at is recoverable - that is the
|
||||
// crystal-frame information careless scales on, and what makes the file unmerged rather than a
|
||||
// merge waiting to happen.
|
||||
// I and SIGI are the integrated intensity with the Lorentz-polarization factor applied and
|
||||
// nothing else, which is what IOBS means in every unmerged format (LP records the factor, so the
|
||||
// raw counts are I/LP). LP is geometry, not a scale, and a program that reads this file has no
|
||||
// way to recover it. The partiality is NOT divided out - that is a scale, FRACTIONCALC carries
|
||||
// it, and every program this file is for wants to handle it its own way.
|
||||
// I and SIGI are the integrated intensity with the prescaling correction applied and nothing
|
||||
// else, which is what IOBS means in every unmerged format, and LP records the factor that was
|
||||
// applied, so the raw counts are I/LP. That factor is geometry and instrument response, not a
|
||||
// scale, and a program reading this file has no way to recompute it. The partiality is NOT
|
||||
// divided out - that is a scale, FRACTIONCALC carries it, and every program this file is for
|
||||
// wants to handle it its own way.
|
||||
// Note that LP therefore carries slightly more than its name says: the prescaling correction now
|
||||
// also holds the sensor's angle-dependent efficiency, for which DIALS writes a separate QE column
|
||||
// and AIMLESS calls LP "the Lorentz/polarization correction (already applied)". A consumer that
|
||||
// divides LP out to recover raw counts removes the efficiency term with it.
|
||||
gemmi::UnmergedHklMover hkl_mover(mtz.spacegroup);
|
||||
// The batch headers are written for every image the observations span, not only for the images
|
||||
// that produced one. AIMLESS starts a new run wherever the phi series jumps, so an image that
|
||||
@@ -800,13 +808,13 @@ void WriteUnmergedMtzReflections(const std::vector<IntegrationOutcome> &outcomes
|
||||
mtz.data.push_back(static_cast<float>(hkl[2]));
|
||||
mtz.data.push_back(static_cast<float>((partials ? 256 : 0) + isym));
|
||||
mtz.data.push_back(static_cast<float>(batch));
|
||||
mtz.data.push_back(r.I * r.rlp);
|
||||
mtz.data.push_back(r.sigma * r.rlp);
|
||||
mtz.data.push_back(r.I * r.prescaling_corr);
|
||||
mtz.data.push_back(r.sigma * r.prescaling_corr);
|
||||
mtz.data.push_back(r.partiality);
|
||||
mtz.data.push_back(DetectorX(r));
|
||||
mtz.data.push_back(DetectorY(r));
|
||||
mtz.data.push_back(phi_start_deg(r.image_number) + wedge_deg / 2.0f);
|
||||
mtz.data.push_back(r.rlp);
|
||||
mtz.data.push_back(r.prescaling_corr);
|
||||
mtz.data.push_back(0.0f); // FLAG: nothing here is a rejected observation
|
||||
mtz.data.push_back(r.delta_phi_deg);
|
||||
mtz.data.push_back(r.zeta);
|
||||
|
||||
@@ -56,9 +56,11 @@ void WriteShelxHklReflections(const std::vector<MergedReflection> &reflections,
|
||||
// rotation program writes, over the events the combine would assemble (--min-partiality). False
|
||||
// writes one row per integrated box instead, flagged as partials for the reading program to sum.
|
||||
// Stills have no rocking events and are unaffected either way.
|
||||
// The intensities carry the Lorentz-polarization factor and nothing else - the partiality and the
|
||||
// The intensities carry the prescaling correction and nothing else - the partiality and the
|
||||
// per-image scale are left for the reading program to fit, since every program this file is for fits
|
||||
// a scale model of its own.
|
||||
// a scale model of its own. The factor that was applied is in the LP column, so raw counts are I/LP;
|
||||
// note that the prescaling correction now also carries the sensor efficiency term, which XDS keeps
|
||||
// out of RLP and DIALS writes in a separate QE column.
|
||||
void WriteUnmergedMtzReflections(const std::vector<IntegrationOutcome> &outcomes,
|
||||
const UnitCell &unitCell,
|
||||
const DiffractionExperiment &experiment,
|
||||
|
||||
@@ -232,8 +232,8 @@ std::vector<Reflection> BraggIntegrationEngine::Finalize(const std::vector<Refle
|
||||
}
|
||||
refl.observed = true;
|
||||
if (polarization)
|
||||
refl.rlp /= geom.CalcAzIntPolarizationCorr(refl.predicted_x, refl.predicted_y, polarization.value());
|
||||
refl.image_scale_corr = refl.rlp / refl.partiality;
|
||||
refl.prescaling_corr /= geom.CalcAzIntPolarizationCorr(refl.predicted_x, refl.predicted_y, polarization.value());
|
||||
refl.image_scale_corr = refl.prescaling_corr / refl.partiality;
|
||||
refl.image_number = static_cast<float>(image_number);
|
||||
out.push_back(refl);
|
||||
}
|
||||
|
||||
@@ -210,7 +210,7 @@ int BraggPrediction::Calc(const DiffractionExperiment &experiment, const Crystal
|
||||
.observed_y = NAN,
|
||||
.d = d,
|
||||
.dist_ewald = dist_ewald_sphere,
|
||||
.rlp = qe_corr,
|
||||
.prescaling_corr = qe_corr,
|
||||
.partiality = 1.0f,
|
||||
.zeta = 1.0,
|
||||
.image_scale_corr = qe_corr
|
||||
|
||||
@@ -127,7 +127,7 @@ namespace {
|
||||
out.observed_y = NAN;
|
||||
out.d = 1.0f / sqrtf(recip_sq);
|
||||
out.dist_ewald = dist_ewald;
|
||||
out.rlp = 1.0f;
|
||||
out.prescaling_corr = 1.0f;
|
||||
out.partiality = 1.0f;
|
||||
out.zeta = 1.0f;
|
||||
out.image_scale_corr = 1.0f;
|
||||
|
||||
@@ -195,7 +195,7 @@ int BraggPredictionRot::Calc(const DiffractionExperiment &experiment, const Crys
|
||||
.observed_y = NAN,
|
||||
.d = d,
|
||||
.dist_ewald = dist_ewald_sphere,
|
||||
.rlp = lorentz_reciprocal * qe_corr,
|
||||
.prescaling_corr = lorentz_reciprocal * qe_corr,
|
||||
.partiality = partiality,
|
||||
.zeta = zeta_abs,
|
||||
.image_scale_corr = lorentz_reciprocal * qe_corr / partiality,
|
||||
|
||||
@@ -190,7 +190,7 @@ namespace {
|
||||
out[count].observed_y = NAN;
|
||||
out[count].d = 1.0f / sqrtf(p0_sq);
|
||||
out[count].dist_ewald = dist_ewald;
|
||||
out[count].rlp = lorentz * qe_corr;
|
||||
out[count].prescaling_corr = lorentz * qe_corr;
|
||||
out[count].partiality = partiality;
|
||||
out[count].zeta = zeta_abs;
|
||||
out[count].image_scale_corr = lorentz * qe_corr / partiality;
|
||||
|
||||
@@ -1043,7 +1043,7 @@ std::vector<AnisotropyObservation> ScaledObservations(const std::vector<Integrat
|
||||
// it would silently empty this on the --mode scale path.
|
||||
auto usable = [](const Reflection &r) {
|
||||
return std::isfinite(r.I) && std::isfinite(r.sigma) && r.sigma > 0.0f && r.d > 0.0f
|
||||
&& std::isfinite(r.rlp) && r.rlp > 0.0f && std::isfinite(r.partiality);
|
||||
&& std::isfinite(r.prescaling_corr) && r.prescaling_corr > 0.0f && std::isfinite(r.partiality);
|
||||
};
|
||||
|
||||
if (!rotation) {
|
||||
@@ -1055,7 +1055,7 @@ std::vector<AnisotropyObservation> ScaledObservations(const std::vector<Integrat
|
||||
for (const auto &r : outcomes[i].reflections) {
|
||||
if (!usable(r) || !(r.partiality >= min_partiality))
|
||||
continue;
|
||||
const double corr = r.rlp / (r.partiality * g[i]);
|
||||
const double corr = r.prescaling_corr / (r.partiality * g[i]);
|
||||
out.push_back({r.h, r.k, r.l, static_cast<float>(r.I * corr),
|
||||
static_cast<float>(r.sigma * corr), r.d});
|
||||
}
|
||||
@@ -1132,8 +1132,8 @@ std::vector<AnisotropyObservation> ScaledObservations(const std::vector<Integrat
|
||||
for (size_t m = i; m < j; ++m) {
|
||||
const Reflection &r = *parts[m].r;
|
||||
sum_p += r.partiality;
|
||||
sum_I += static_cast<double>(r.I) * r.rlp;
|
||||
sum_var += static_cast<double>(r.sigma) * r.sigma * r.rlp * r.rlp;
|
||||
sum_I += static_cast<double>(r.I) * r.prescaling_corr;
|
||||
sum_var += static_cast<double>(r.sigma) * r.sigma * r.prescaling_corr * r.prescaling_corr;
|
||||
p_g += r.partiality * g[parts[m].image];
|
||||
}
|
||||
const Reflection &first = *parts[i].r;
|
||||
|
||||
@@ -81,7 +81,7 @@ bool AcceptReflection(const Reflection &r, std::optional<double> d_min_limit, st
|
||||
return false;
|
||||
if (d_max_limit && r.d > d_max_limit)
|
||||
return false;
|
||||
if (!std::isfinite(r.rlp) || r.rlp == 0.0f)
|
||||
if (!std::isfinite(r.prescaling_corr) || r.prescaling_corr == 0.0f)
|
||||
return false;
|
||||
if (!std::isfinite(r.sigma) || r.sigma <= 0.0)
|
||||
return false;
|
||||
@@ -97,7 +97,7 @@ bool AcceptReflection(const Reflection &r, double d_min_limit, double d_max_limi
|
||||
return false;
|
||||
if (d_max_limit > 0.0 && r.d > d_max_limit)
|
||||
return false;
|
||||
if (!std::isfinite(r.rlp) || r.rlp == 0.0f)
|
||||
if (!std::isfinite(r.prescaling_corr) || r.prescaling_corr == 0.0f)
|
||||
return false;
|
||||
if (!std::isfinite(r.sigma) || r.sigma <= 0.0)
|
||||
return false;
|
||||
|
||||
@@ -173,7 +173,7 @@ double ReindexAmbiguityResolver::ReferenceCC(const std::vector<Reflection> &refl
|
||||
const auto it = reference_data.find(hkl_key_generator(h[0], h[1], h[2]));
|
||||
if (it == reference_data.end())
|
||||
continue;
|
||||
const double x = static_cast<double>(r.I) * r.rlp / r.partiality;
|
||||
const double x = static_cast<double>(r.I) * r.prescaling_corr / r.partiality;
|
||||
const double y = it->second;
|
||||
if (!std::isfinite(x) || !std::isfinite(y))
|
||||
continue;
|
||||
|
||||
@@ -261,7 +261,7 @@ RotationScaleMerge::RotationScaleMerge(const DiffractionExperiment &experiment,
|
||||
// Background under a reflection is proportional to the incident flux, so the frame's mean background
|
||||
// measures the flux directly, from data that is already here.
|
||||
//
|
||||
// It enters through rlp, the numerator of corr = rlp / (partiality * G), which is exactly the scale
|
||||
// It enters through prescaling_corr, the numerator of corr = prescaling_corr / (partiality * G), which is exactly the scale
|
||||
// factor - so it applies on the host and on the GPU alike, and the IRLS fits only what is left. The
|
||||
// proxy is not flux alone: a growing ice ring, a shadow drifting across the detector or absorption
|
||||
// changing with the goniometer angle also move the background. Those are all SLOW, and what they put in
|
||||
@@ -269,7 +269,7 @@ RotationScaleMerge::RotationScaleMerge(const DiffractionExperiment &experiment,
|
||||
// the part no smoothed per-frame scale can represent anyway - is taken on trust.
|
||||
//
|
||||
// Neither half of this costs a pass over the observations of its own: the per-frame mean background is
|
||||
// accumulated by the ingest fill loop, which already reads every reflection, and the division into rlp
|
||||
// accumulated by the ingest fill loop, which already reads every reflection, and the division into prescaling_corr
|
||||
// rides on the finiteness pass that follows it.
|
||||
void RotationScaleMerge::MeasureIncidentFlux(const std::vector<double> &mean_bkg) {
|
||||
frame_flux.assign(n_frames, 1.0);
|
||||
@@ -507,7 +507,7 @@ void RotationScaleMerge::Ingest() {
|
||||
// value-initialisation a sized vector performs is half a gigabyte of zeroes faulted in on one
|
||||
// thread and overwritten on the next line.
|
||||
const auto floats = [n] { return std::unique_ptr<float[]>(new float[n]); };
|
||||
auto I = floats(), sigma = floats(), rlp = floats(), part = floats(), zeta = floats(),
|
||||
auto I = floats(), sigma = floats(), prescaling_corr = floats(), part = floats(), zeta = floats(),
|
||||
corr = floats(), bkg = floats(), img = floats(), dd = floats(), px = floats(),
|
||||
py = floats(), vbkg = floats();
|
||||
std::unique_ptr<uint8_t[]> onice(new uint8_t[n]);
|
||||
@@ -518,13 +518,13 @@ void RotationScaleMerge::Ingest() {
|
||||
ParallelChunks(n, nthreads, [&](int lo, int hi) {
|
||||
for (int i = lo; i < hi; ++i) {
|
||||
const auto &o = partials[i];
|
||||
I[i] = o.I; sigma[i] = o.sigma; rlp[i] = o.rlp; part[i] = o.partiality;
|
||||
I[i] = o.I; sigma[i] = o.sigma; prescaling_corr[i] = o.prescaling_corr; part[i] = o.partiality;
|
||||
zeta[i] = o.zeta; onice[i] = o.on_ice; frm[i] = o.frame; corr[i] = o.corr;
|
||||
bkg[i] = o.bkg; vbkg[i] = o.var_bkg; img[i] = o.image_number; dd[i] = o.d;
|
||||
px[i] = o.px; py[i] = o.py;
|
||||
}
|
||||
});
|
||||
gpu_->SetPartials(n, n_frames, I.get(), sigma.get(), rlp.get(), part.get(), zeta.get(),
|
||||
gpu_->SetPartials(n, n_frames, I.get(), sigma.get(), prescaling_corr.get(), part.get(), zeta.get(),
|
||||
onice.get(), frm.get(), corr.get(), frame_start.data(), frame_count.data());
|
||||
gpu_->SetCombineInputs(bkg.get(), vbkg.get(), img.get(), dd.get(), px.get(), py.get());
|
||||
gpu_->SetRawRuns(static_cast<int>(rawrun_start.size()), static_cast<int>(perm.size()), perm.data(),
|
||||
@@ -608,9 +608,10 @@ void RotationScaleMerge::BuildInRangeObservations(const std::vector<SortKey> &ke
|
||||
Obs &obs = partials[at];
|
||||
obs.h = r.h; obs.k = r.k; obs.l = r.l;
|
||||
obs.I = r.I; obs.sigma = r.sigma; obs.d = r.d;
|
||||
// Divide the incident flux out of rlp here: rlp is the numerator of corr, so the fits
|
||||
// below see only what is left.
|
||||
obs.rlp = static_cast<float>(r.rlp / frame_flux[o]);
|
||||
// Divide the incident flux out of prescaling_corr here: it is the numerator of corr, so
|
||||
// the fits below see only what is left. The flux is the one term in the product that is
|
||||
// measured from the data rather than computed from the geometry.
|
||||
obs.prescaling_corr = static_cast<float>(r.prescaling_corr / frame_flux[o]);
|
||||
obs.partiality = r.partiality; obs.zeta = r.zeta; obs.delta_phi = r.delta_phi_deg; obs.bkg = r.bkg; obs.var_bkg = r.var_bkg;
|
||||
obs.px = r.predicted_x; obs.py = r.predicted_y;
|
||||
obs.image_number = r.image_number;
|
||||
@@ -618,7 +619,7 @@ void RotationScaleMerge::BuildInRangeObservations(const std::vector<SortKey> &ke
|
||||
obs.on_ice = r.on_ice_ring ? 1 : 0;
|
||||
obs.corr = r.image_scale_corr;
|
||||
obs.group = -1;
|
||||
finite_ok[at] = (std::isfinite(obs.I) && std::isfinite(obs.rlp) && obs.rlp != 0.0f
|
||||
finite_ok[at] = (std::isfinite(obs.I) && std::isfinite(obs.prescaling_corr) && obs.prescaling_corr != 0.0f
|
||||
&& std::isfinite(obs.sigma) && obs.sigma > 0.0f) ? 1 : 0;
|
||||
++at;
|
||||
}
|
||||
@@ -1182,10 +1183,10 @@ void RotationScaleMerge::FitPerFrameG(std::vector<Obs> &obs, const std::vector<i
|
||||
if (!std::isfinite(mean)) continue;
|
||||
double coeff;
|
||||
if (unity) {
|
||||
coeff = mean; // partiality already folded into the full, rlp = 1
|
||||
coeff = mean; // partiality already folded into the full, prescaling_corr = 1
|
||||
} else {
|
||||
if (!(std::isfinite(o.zeta) && o.zeta > 0.0f)) continue; // Rotation model needs zeta > 0
|
||||
coeff = o.partiality * SafeInv(o.rlp, 1.0) * mean;
|
||||
coeff = o.partiality * SafeInv(o.prescaling_corr, 1.0) * mean;
|
||||
}
|
||||
so.push_back({coeff, static_cast<double>(o.I), SafeInv(o.sigma, 1.0)});
|
||||
}
|
||||
@@ -1204,8 +1205,8 @@ void RotationScaleMerge::UpdateCorr(std::vector<Obs> &obs, const std::vector<dou
|
||||
auto &o = obs[i];
|
||||
if (!frame_scaled[o.frame]) continue;
|
||||
const double denom = static_cast<double>(o.partiality) * g[o.frame]; // B_term = 1 (no B refine)
|
||||
if (std::isfinite(o.rlp) && std::isfinite(denom) && denom > 0.0)
|
||||
o.corr = static_cast<float>(o.rlp / denom);
|
||||
if (std::isfinite(o.prescaling_corr) && std::isfinite(denom) && denom > 0.0)
|
||||
o.corr = static_cast<float>(o.prescaling_corr / denom);
|
||||
else
|
||||
o.corr = NAN;
|
||||
}
|
||||
@@ -2547,7 +2548,7 @@ void RotationScaleMerge::Combine() {
|
||||
full.var_bkg = static_cast<float>(var_bkg_full);
|
||||
full.var_per_I = static_cast<float>(var_bkg_full * var_bkg_full * sum_cwb);
|
||||
full.d = d;
|
||||
full.rlp = 1.0f;
|
||||
full.prescaling_corr = 1.0f;
|
||||
full.partiality = 1.0f;
|
||||
full.corr = 1.0f;
|
||||
full.image_number = peak_frame;
|
||||
@@ -3894,7 +3895,7 @@ RotationScaleMerge::Result RotationScaleMerge::Run(bool for_search, bool full_st
|
||||
o.h = st.h[i]; o.k = st.k[i]; o.l = st.l[i];
|
||||
o.I = st.I[i]; o.sigma = st.sigma[i]; o.d = st.d[i];
|
||||
o.var_bkg = st.var_bkg[i]; o.var_per_I = st.var_per_I[i];
|
||||
o.rlp = 1.0f; o.partiality = 1.0f;
|
||||
o.prescaling_corr = 1.0f; o.partiality = 1.0f;
|
||||
o.corr = scaled_fulls_on_gpu ? st.corr[i] : 1.0f;
|
||||
o.image_number = st.image_number[i]; o.frame = st.frame[i];
|
||||
o.px = st.px[i]; o.py = st.py[i];
|
||||
|
||||
@@ -111,7 +111,7 @@ private:
|
||||
// scale-fulls/merge. Flat (not nested per image); a POD so the arrays translate straight to CUDA.
|
||||
struct Obs {
|
||||
int32_t h, k, l;
|
||||
float I, sigma, d, rlp, partiality, zeta, delta_phi, bkg, var_bkg;
|
||||
float I, sigma, d, prescaling_corr, partiality, zeta, delta_phi, bkg, var_bkg;
|
||||
// Fulls only, written by the combine: the full's variance as a function of intensity,
|
||||
// var(I) = var_bkg + var_per_I * I. The merge rebuilds it at the reflection's mean.
|
||||
float var_per_I = 0.0f;
|
||||
@@ -320,8 +320,8 @@ private:
|
||||
|
||||
// --- helpers (each a flat pass; see the .cpp) ---
|
||||
// Turn the per-frame mean background under the reflections (accumulated by the ingest fill loop) into
|
||||
// the per-frame incident flux, which the finiteness pass then folds into rlp so that
|
||||
// corr = rlp / (partiality * G) divides it out and G fits only the residual. See the .cpp for why a
|
||||
// the per-frame incident flux, which the finiteness pass then folds into prescaling_corr so that
|
||||
// corr = prescaling_corr / (partiality * G) divides it out and G fits only the residual. See the .cpp for why a
|
||||
// background is a usable flux meter and what it costs when it is not.
|
||||
void MeasureIncidentFlux(const std::vector<double> &mean_bkg);
|
||||
|
||||
@@ -351,7 +351,7 @@ private:
|
||||
const std::vector<int32_t> &fcount, const std::vector<double> &group_mean_in,
|
||||
bool unity, std::vector<double> &g);
|
||||
|
||||
// corr = rlp / (partiality * G[frame]); leaves corr unchanged for frames that could not be fit.
|
||||
// corr = prescaling_corr / (partiality * G[frame]); leaves corr unchanged for frames that could not be fit.
|
||||
void UpdateCorr(std::vector<Obs> &obs, const std::vector<double> &g,
|
||||
const std::vector<uint8_t> &frame_scaled) const;
|
||||
|
||||
|
||||
@@ -65,9 +65,9 @@ namespace {
|
||||
}
|
||||
|
||||
// Per-observation scale-fit coefficient (rotation model) and accept flag, recomputed each scaling
|
||||
// iteration once the group means are known. coeff = partiality * (1/rlp) * mean[group].
|
||||
// iteration once the group means are known. coeff = partiality * (1/prescaling_corr) * mean[group].
|
||||
__global__ void PrepScaleObsKernel(int n_obs, const int32_t *__restrict__ group,
|
||||
const float *__restrict__ partiality, const float *__restrict__ rlp,
|
||||
const float *__restrict__ partiality, const float *__restrict__ prescaling_corr,
|
||||
const float *__restrict__ zeta, const uint8_t *__restrict__ on_ice,
|
||||
const double *__restrict__ group_mean,
|
||||
const float *__restrict__ sigma, double *__restrict__ inv_sigma,
|
||||
@@ -88,7 +88,7 @@ namespace {
|
||||
ok = isfinite(mean);
|
||||
}
|
||||
sco_ok[i] = ok ? 1 : 0;
|
||||
sco_coeff[i] = ok ? float(double(partiality[i]) * SafeInvD(rlp[i], 1.0) * mean) : 0.0f;
|
||||
sco_coeff[i] = ok ? float(double(partiality[i]) * SafeInvD(prescaling_corr[i], 1.0) * mean) : 0.0f;
|
||||
}
|
||||
|
||||
// One block per frame: robust per-frame scale G by IRLS (Cauchy), over the frame's contiguous obs.
|
||||
@@ -255,17 +255,17 @@ namespace {
|
||||
if (reject[frame[i]]) corr[i] = 0.0f;
|
||||
}
|
||||
|
||||
// corr = rlp / (partiality * G[frame]) for fitted frames; unchanged otherwise (grid-stride).
|
||||
// corr = prescaling_corr / (partiality * G[frame]) for fitted frames; unchanged otherwise (grid-stride).
|
||||
__global__ void UpdateCorrKernel(int n_obs, const int32_t *__restrict__ frame,
|
||||
const float *__restrict__ rlp, const float *__restrict__ partiality,
|
||||
const float *__restrict__ prescaling_corr, const float *__restrict__ partiality,
|
||||
const double *__restrict__ g, const uint8_t *__restrict__ scaled,
|
||||
float *__restrict__ corr) {
|
||||
for (int i = blockIdx.x * blockDim.x + threadIdx.x; i < n_obs; i += gridDim.x * blockDim.x) {
|
||||
const int f = frame[i];
|
||||
if (!scaled[f]) continue;
|
||||
const double denom = double(partiality[i]) * g[f];
|
||||
corr[i] = (isfinite(double(rlp[i])) && isfinite(denom) && denom > 0.0)
|
||||
? float(rlp[i] / denom) : NAN;
|
||||
corr[i] = (isfinite(double(prescaling_corr[i])) && isfinite(denom) && denom > 0.0)
|
||||
? float(prescaling_corr[i] / denom) : NAN;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -629,7 +629,7 @@ struct RotationScaleMergeGPU::Impl {
|
||||
int n_obs = 0, n_frames = 0, n_groups = 0;
|
||||
|
||||
// immutable per-obs
|
||||
CudaDevicePtr<float> I, sigma, rlp, partiality, zeta;
|
||||
CudaDevicePtr<float> I, sigma, prescaling_corr, partiality, zeta;
|
||||
CudaDevicePtr<uint8_t> on_ice;
|
||||
CudaDevicePtr<int32_t> frame;
|
||||
CudaDevicePtr<float> corr; // mutable, resident across iterations
|
||||
@@ -673,7 +673,7 @@ struct RotationScaleMergeGPU::Impl {
|
||||
CudaDevicePtr<int32_t> f_h, f_k, f_l, f_frame, f_group;
|
||||
CudaDevicePtr<float> f_I, f_sigma, f_d, f_img, f_px, f_py, f_var_bkg, f_var_per_I;
|
||||
CudaDevicePtr<uint8_t> f_on_ice;
|
||||
// scale-fulls (Unity model, kept resident): all-ones partiality/rlp/zeta so the shared scaling kernels
|
||||
// scale-fulls (Unity model, kept resident): all-ones partiality/prescaling_corr/zeta so the shared scaling kernels
|
||||
// yield coeff=mean, plus the working corr, the per-obs scale scratch, and the fulls frame/group CSRs
|
||||
// (built on the host from the small f_frame/f_group key arrays, over the emit-ordered fulls).
|
||||
CudaDevicePtr<float> f_corr, f_partiality, f_rlp, f_zeta, f_sco_coeff;
|
||||
@@ -725,7 +725,7 @@ RotationScaleMergeGPU::~RotationScaleMergeGPU() {
|
||||
bool RotationScaleMergeGPU::Available() const { return impl_->available; }
|
||||
|
||||
void RotationScaleMergeGPU::SetPartials(int n_obs, int n_frames,
|
||||
const float *I, const float *sigma, const float *rlp,
|
||||
const float *I, const float *sigma, const float *prescaling_corr,
|
||||
const float *partiality, const float *zeta, const uint8_t *on_ice,
|
||||
const int32_t *frame, const float *corr0,
|
||||
const int32_t *frame_start, const int32_t *frame_count) {
|
||||
@@ -733,7 +733,7 @@ void RotationScaleMergeGPU::SetPartials(int n_obs, int n_frames,
|
||||
auto &d = *impl_;
|
||||
d.n_obs = n_obs;
|
||||
d.n_frames = n_frames;
|
||||
Upload(d.I, I, n_obs); Upload(d.sigma, sigma, n_obs); Upload(d.rlp, rlp, n_obs);
|
||||
Upload(d.I, I, n_obs); Upload(d.sigma, sigma, n_obs); Upload(d.prescaling_corr, prescaling_corr, n_obs);
|
||||
Upload(d.partiality, partiality, n_obs); Upload(d.zeta, zeta, n_obs); Upload(d.on_ice, on_ice, n_obs);
|
||||
Upload(d.frame, frame, n_obs); Upload(d.corr, corr0, n_obs);
|
||||
Upload(d.frame_start, frame_start, n_frames); Upload(d.frame_count, frame_count, n_frames);
|
||||
@@ -778,12 +778,12 @@ void RotationScaleMergeGPU::ScalePartials(int iters, double robust_k, double min
|
||||
ReduceGroupMeansKernel<<<grp_blocks, BLK>>>(d.n_groups, min_partiality,
|
||||
d.group_perm.get(), d.group_start.get(), d.group_count.get(),
|
||||
d.I.get(), d.sigma.get(), d.partiality.get(), d.corr.get(), d.group_mean.get());
|
||||
PrepScaleObsKernel<<<obs_blocks, BLK>>>(d.n_obs, d.group.get(), d.partiality.get(), d.rlp.get(),
|
||||
PrepScaleObsKernel<<<obs_blocks, BLK>>>(d.n_obs, d.group.get(), d.partiality.get(), d.prescaling_corr.get(),
|
||||
d.zeta.get(), d.on_ice.get(), d.group_mean.get(), d.sigma.get(), d.inv_sigma.get(),
|
||||
d.sco_coeff.get(), d.sco_ok.get());
|
||||
FitPerFrameGKernel<<<d.n_frames, BLK>>>(d.n_frames, robust_k, d.frame_start.get(), d.frame_count.get(),
|
||||
d.I.get(), d.inv_sigma.get(), d.sco_coeff.get(), d.sco_ok.get(), nullptr, d.g.get(), d.scaled.get());
|
||||
UpdateCorrKernel<<<upd_blocks, BLK>>>(d.n_obs, d.frame.get(), d.rlp.get(), d.partiality.get(),
|
||||
UpdateCorrKernel<<<upd_blocks, BLK>>>(d.n_obs, d.frame.get(), d.prescaling_corr.get(), d.partiality.get(),
|
||||
d.g.get(), d.scaled.get(), d.corr.get());
|
||||
}
|
||||
CudaCheck(cudaGetLastError(), "kernel launch");
|
||||
@@ -1145,7 +1145,7 @@ void RotationScaleMergeGPU::ScaleFulls(int iters, double robust_k, double min_pa
|
||||
const int obs_blocks = std::min(65535, (nf + BLK - 1) / BLK);
|
||||
const int grp_blocks = std::min(65535, (d.n_groups + BLK - 1) / BLK);
|
||||
|
||||
// Unity model: partiality/rlp/zeta = 1 so coeff = mean; corr starts at 1.
|
||||
// Unity model: partiality/prescaling_corr/zeta = 1 so coeff = mean; corr starts at 1.
|
||||
FillKernel<<<obs_blocks, BLK>>>(d.f_corr.get(), nf, 1.0f);
|
||||
FillKernel<<<obs_blocks, BLK>>>(d.f_partiality.get(), nf, 1.0f);
|
||||
FillKernel<<<obs_blocks, BLK>>>(d.f_rlp.get(), nf, 1.0f);
|
||||
|
||||
@@ -29,7 +29,7 @@ public:
|
||||
// Upload the immutable per-observation fields (once). Arrays are length n_obs unless noted; the
|
||||
// frame CSR (frame_start/frame_count) is length n_frames and indexes the obs arrays in frame order.
|
||||
void SetPartials(int n_obs, int n_frames,
|
||||
const float *I, const float *sigma, const float *rlp, const float *partiality,
|
||||
const float *I, const float *sigma, const float *prescaling_corr, const float *partiality,
|
||||
const float *zeta, const uint8_t *on_ice, const int32_t *frame,
|
||||
const float *corr0,
|
||||
const int32_t *frame_start, const int32_t *frame_count);
|
||||
|
||||
@@ -114,7 +114,7 @@ void ScaleOnTheFly::Scale(IntegrationOutcome &integration_outcome) const {
|
||||
const auto it = reference_data.find(hkl_key_generator(r));
|
||||
if (it == reference_data.end())
|
||||
continue;
|
||||
const double coeff = r.partiality * SafeInv(r.rlp, 1.0) * it->second;
|
||||
const double coeff = r.partiality * SafeInv(r.prescaling_corr, 1.0) * it->second;
|
||||
obs.push_back({coeff, static_cast<double>(r.I), SafeInv(r.sigma, 1.0)});
|
||||
}
|
||||
|
||||
@@ -128,8 +128,8 @@ void ScaleOnTheFly::Scale(IntegrationOutcome &integration_outcome) const {
|
||||
|
||||
for (auto &r: integration_outcome.reflections) {
|
||||
const double denom = r.partiality * result.G;
|
||||
r.image_scale_corr = (std::isfinite(r.rlp) && std::isfinite(denom) && denom > 0.0)
|
||||
? static_cast<float>(r.rlp / denom)
|
||||
r.image_scale_corr = (std::isfinite(r.prescaling_corr) && std::isfinite(denom) && denom > 0.0)
|
||||
? static_cast<float>(r.prescaling_corr / denom)
|
||||
: NAN;
|
||||
}
|
||||
|
||||
|
||||
@@ -26,12 +26,13 @@ namespace {
|
||||
|
||||
// One accepted reflection reduced to the physical partiality fit: the base reciprocal vector q (crystal
|
||||
// frame at the stored per-image orientation), the reference full intensity, the measured intensity, the
|
||||
// Lorentz factor and the weight. dist_ewald / partiality are recomputed from q under the refined tilt.
|
||||
// reciprocal of the prescaling correction (the factor that turns a true intensity back into counts)
|
||||
// and the weight. dist_ewald / partiality are recomputed from q under the refined tilt.
|
||||
struct FitObs {
|
||||
double qx, qy, qz;
|
||||
double Iref;
|
||||
double Iobs;
|
||||
double lp; // 1 / rlp
|
||||
double inv_corr; // 1 / prescaling_corr
|
||||
double weight; // 1 / sigma
|
||||
};
|
||||
|
||||
@@ -106,7 +107,7 @@ namespace {
|
||||
double qx, qy, qz;
|
||||
double s0x, s0y, s0z, inv_lambda;
|
||||
double gamma0, gamma_e, bw;
|
||||
double G, lp, Iref, Iobs, weight;
|
||||
double G, inv_corr, Iref, Iobs, weight;
|
||||
|
||||
template<typename T>
|
||||
bool operator()(const T *const psi, T *residual) const {
|
||||
@@ -121,7 +122,7 @@ namespace {
|
||||
const T sbw = T(bw) * ceres::abs(qr[2]);
|
||||
const T sig2 = T(gamma0) * T(gamma0) + sig_ang * sig_ang + sbw * sbw;
|
||||
const T p = ceres::exp(T(-0.5) * de * de / sig2);
|
||||
residual[0] = T(weight) * (T(G) * p * T(lp) * T(Iref) - T(Iobs));
|
||||
residual[0] = T(weight) * (T(G) * p * T(inv_corr) * T(Iref) - T(Iobs));
|
||||
return true;
|
||||
}
|
||||
};
|
||||
@@ -185,7 +186,7 @@ double StillsPartialityRefine::RefineOne(IntegrationOutcome &outcome,
|
||||
.qx = q.x, .qy = q.y, .qz = q.z,
|
||||
.Iref = it->second,
|
||||
.Iobs = static_cast<double>(r.I),
|
||||
.lp = SafeInv(r.rlp, 1.0),
|
||||
.inv_corr = SafeInv(r.prescaling_corr, 1.0),
|
||||
.weight = SafeInv(r.sigma, 1.0),
|
||||
});
|
||||
|
||||
@@ -245,7 +246,7 @@ double StillsPartialityRefine::RefineOne(IntegrationOutcome &outcome,
|
||||
for (size_t j = 0; j < obs.size(); ++j) {
|
||||
const double p = ComputeP(obs[j].qx, obs[j].qy, obs[j].qz, psi[0], psi[1],
|
||||
S0.x, S0.y, S0.z, inv_lambda, gamma0, gamma_e, bw);
|
||||
coeff[j] = p * obs[j].lp * obs[j].Iref;
|
||||
coeff[j] = p * obs[j].inv_corr * obs[j].Iref;
|
||||
Iobs[j] = obs[j].Iobs;
|
||||
weight[j] = obs[j].weight;
|
||||
}
|
||||
@@ -263,7 +264,7 @@ double StillsPartialityRefine::RefineOne(IntegrationOutcome &outcome,
|
||||
.qx = o.qx, .qy = o.qy, .qz = o.qz,
|
||||
.s0x = S0.x, .s0y = S0.y, .s0z = S0.z, .inv_lambda = inv_lambda,
|
||||
.gamma0 = gamma0, .gamma_e = gamma_e, .bw = bw,
|
||||
.G = G, .lp = o.lp, .Iref = o.Iref, .Iobs = o.Iobs, .weight = o.weight});
|
||||
.G = G, .inv_corr = o.inv_corr, .Iref = o.Iref, .Iobs = o.Iobs, .weight = o.weight});
|
||||
problem.AddResidualBlock(cost, new ceres::CauchyLoss(settings_.robust_k), psi);
|
||||
}
|
||||
if (settings_.prior_sigma_deg > 0.0) {
|
||||
@@ -305,16 +306,16 @@ double StillsPartialityRefine::RefineOne(IntegrationOutcome &outcome,
|
||||
before.emplace_back(r.partiality, r.image_scale_corr);
|
||||
|
||||
// Write the refined partiality + scale correction onto every reflection of the crystal (not only the
|
||||
// fit subset), so the merge sees a consistent model. image_scale_corr = rlp / (partiality * G), the
|
||||
// same composition ScaleOnTheFly writes.
|
||||
// fit subset), so the merge sees a consistent model. image_scale_corr = prescaling_corr /
|
||||
// (partiality * G), the same composition ScaleOnTheFly writes.
|
||||
for (auto &r: outcome.reflections) {
|
||||
const Coord q = base_q(r);
|
||||
const double p = ComputeP(q.x, q.y, q.z, psi[0], psi[1], S0.x, S0.y, S0.z, inv_lambda,
|
||||
gamma0, gamma_e, bw);
|
||||
r.partiality = static_cast<float>(p);
|
||||
const double denom = p * G;
|
||||
r.image_scale_corr = (std::isfinite(r.rlp) && std::isfinite(denom) && denom > 0.0)
|
||||
? static_cast<float>(r.rlp / denom)
|
||||
r.image_scale_corr = (std::isfinite(r.prescaling_corr) && std::isfinite(denom) && denom > 0.0)
|
||||
? static_cast<float>(r.prescaling_corr / denom)
|
||||
: NAN;
|
||||
}
|
||||
outcome.image_scale_g = static_cast<float>(G);
|
||||
|
||||
@@ -396,7 +396,7 @@ bool ReadReflectionsFromGroup(HDF5Object &file,
|
||||
.bkg = bkg.at(i),
|
||||
.var_bkg = var_bkg_val,
|
||||
.sigma = int_err.at(i),
|
||||
.rlp = lp_val,
|
||||
.prescaling_corr = lp_val,
|
||||
.partiality = partiality_val,
|
||||
.zeta = zeta_val,
|
||||
.image_scale_corr = image_scale_corr_val
|
||||
|
||||
@@ -174,7 +174,7 @@ void print_usage() {
|
||||
std::cout << " -z, --reference-mtz <file> Reference MTZ file" << std::endl;
|
||||
std::cout << " --reference-column <label> Reference MTZ column to use (default: auto - F-model, else IMEAN/I, else FP/FOBS/F)" << std::endl;
|
||||
std::cout << " --model <file> After merging, validate vs this atomic model: R-free, sigma_A-weighted 2mFo-DFc/mFo-DFc maps, the model as placed (written beside them), and - where the merge kept the Bijvoet split - an anomalous difference map with the strongest anomalous scatterers named by the atom they sit on. PDB or mmCIF, gzipped or not, recognised by content rather than by extension" << std::endl;
|
||||
std::cout << " --export-unmerged Write <prefix>_unmerged.mtz, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. On by default, whenever there is an output prefix. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-captured-fraction (or --min-partiality) is not written, exactly as in the merge, since a full is what the reading program is told these rows are. Intensities carry the Lorentz-polarization factor and nothing else: the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Written in --mode mx and --mode scale, and with --no-merge" << std::endl;
|
||||
std::cout << " --export-unmerged Write <prefix>_unmerged.mtz, an unmerged MTZ (POINTLESS column layout) of the integrated observations, for aimless / pointless / careless. On by default, whenever there is an output prefix. On a rotation run the partials of each reflection are summed into one full, written at the batch its rocking curve is centred on, with the summed rocking-curve fraction in FRACTIONCALC; an event that caught less of its rocking curve than --min-captured-fraction (or --min-partiality) is not written, exactly as in the merge, since a full is what the reading program is told these rows are. Intensities carry the Lorentz, polarization and sensor-efficiency corrections and nothing else, with the applied factor in the LP column: the partiality is not divided out and the per-image scale is not applied at all, since those programs scale the data themselves. Written in --mode mx and --mode scale, and with --no-merge" << std::endl;
|
||||
std::cout << " --no-export-unmerged Do not write <prefix>_unmerged.mtz. It is the largest file a run produces, so a run whose observations are not going to another scaling program - a regression battery, say - can skip writing it" << std::endl;
|
||||
std::cout << " --export-unmerged-partials Write <prefix>_unmerged_partials.mtz, the same observations with each partial as its own row (one batch per image) flagged for the reading program to sum, instead of summed here. Off by default and independent of --export-unmerged; both can be written in one run" << std::endl;
|
||||
std::cout << " --no-p1-crosscheck Do not write <prefix>_P1.mtz. That file is the same observations merged in P1 instead of the space group the run determined, so a wrong space group can be re-merged, re-solved or re-refined without processing the images again. Every rotation run that determines its own space group writes it - including one that determined P1, where it repeats the merged output - so it is there to be harvested whatever the run decided. Not written when -S fixed the space group: that run never integrated the group's centring absences, so its P1 merge would be missing whole classes of reflections. It is a full merge, not the degraded one the search runs on, and it costs one extra merge" << std::endl;
|
||||
|
||||
@@ -161,7 +161,7 @@ namespace {
|
||||
r.d = 3.0f;
|
||||
r.I = I;
|
||||
r.sigma = 10.0f;
|
||||
r.rlp = 1.0f;
|
||||
r.prescaling_corr = 1.0f;
|
||||
r.partiality = partiality;
|
||||
return r;
|
||||
}
|
||||
|
||||
@@ -33,7 +33,7 @@ Reflection MakeReflection(float x, float y, float d, int hkl) {
|
||||
r.predicted_x = x;
|
||||
r.predicted_y = y;
|
||||
r.d = d;
|
||||
r.rlp = 1.0f;
|
||||
r.prescaling_corr = 1.0f;
|
||||
r.partiality = 1.0f;
|
||||
return r;
|
||||
}
|
||||
|
||||
@@ -33,7 +33,7 @@ Reflection MakeReflection(float x, float y, float d, int hkl) {
|
||||
r.predicted_x = x;
|
||||
r.predicted_y = y;
|
||||
r.d = d;
|
||||
r.rlp = 1.0f;
|
||||
r.prescaling_corr = 1.0f;
|
||||
r.partiality = 1.0f;
|
||||
return r;
|
||||
}
|
||||
|
||||
+5
-5
@@ -1078,7 +1078,7 @@ TEST_CASE("CBORSerialize_Image_Reflections") {
|
||||
.predicted_x = 1024.5f, .predicted_y = 768.25f,
|
||||
.observed_x = 1024.75f, .observed_y = 768.5f,
|
||||
.d = 2.345f, .I = 1234.5f, .bkg = 3.25f, .var_bkg = 41.5f, .sigma = 37.125f,
|
||||
.dist_ewald = 0.0025f, .rlp = 1.75f, .partiality = 0.875f, .zeta = 0.625f,
|
||||
.dist_ewald = 0.0025f, .prescaling_corr = 1.75f, .partiality = 0.875f, .zeta = 0.625f,
|
||||
.image_scale_corr = 1.125f
|
||||
});
|
||||
refs_in.push_back(Reflection{
|
||||
@@ -1087,7 +1087,7 @@ TEST_CASE("CBORSerialize_Image_Reflections") {
|
||||
.predicted_x = 5.0f, .predicted_y = 10.0f,
|
||||
.observed_x = 5.5f, .observed_y = 10.5f,
|
||||
.d = 4.0f, .I = 12.0f, .bkg = 0.5f, .var_bkg = 6.75f, .sigma = 4.25f,
|
||||
.dist_ewald = -0.001f, .rlp = 2.5f, .partiality = 0.5f, .zeta = 0.25f,
|
||||
.dist_ewald = -0.001f, .prescaling_corr = 2.5f, .partiality = 0.5f, .zeta = 0.25f,
|
||||
.image_scale_corr = 0.875f
|
||||
});
|
||||
refs_in.push_back(Reflection{
|
||||
@@ -1096,7 +1096,7 @@ TEST_CASE("CBORSerialize_Image_Reflections") {
|
||||
.predicted_x = 0.0f, .predicted_y = 0.0f,
|
||||
.observed_x = 0.25f, .observed_y = 0.5f,
|
||||
.d = 10.0f, .I = -1.5f, .bkg = 0.125f, .var_bkg = 0.875f, .sigma = 1.5f,
|
||||
.dist_ewald = 0.0f, .rlp = 1.0f, .partiality = 1.0f, .zeta = 1.0f,
|
||||
.dist_ewald = 0.0f, .prescaling_corr = 1.0f, .partiality = 1.0f, .zeta = 1.0f,
|
||||
.image_scale_corr = 1.0f
|
||||
});
|
||||
|
||||
@@ -1146,7 +1146,7 @@ TEST_CASE("CBORSerialize_Image_Reflections") {
|
||||
CHECK(b.var_bkg == Catch::Approx(a.var_bkg).margin(1e-6f));
|
||||
CHECK(b.sigma == Catch::Approx(a.sigma).margin(1e-6f));
|
||||
CHECK(b.dist_ewald == Catch::Approx(a.dist_ewald).margin(1e-6f));
|
||||
CHECK(b.rlp == Catch::Approx(a.rlp).margin(1e-6f));
|
||||
CHECK(b.prescaling_corr == Catch::Approx(a.prescaling_corr).margin(1e-6f));
|
||||
CHECK(b.partiality == Catch::Approx(a.partiality).margin(1e-6f));
|
||||
CHECK(b.zeta == Catch::Approx(a.zeta).margin(1e-6f));
|
||||
CHECK(b.image_scale_corr == Catch::Approx(a.image_scale_corr).margin(1e-6f));
|
||||
@@ -1180,7 +1180,7 @@ TEST_CASE("CBORSerialize_Image_MetadataHeadroom", "[CBOR]") {
|
||||
.predicted_x = 1234.5f, .predicted_y = 678.9f,
|
||||
.observed_x = 1234.0f, .observed_y = 678.0f,
|
||||
.d = 2.345f, .I = 1234.5f, .bkg = 12.3f, .sigma = 35.6f,
|
||||
.dist_ewald = 0.001f, .rlp = 0.5f, .partiality = 0.9f,
|
||||
.dist_ewald = 0.001f, .prescaling_corr = 0.5f, .partiality = 0.9f,
|
||||
.zeta = 0.3f, .image_scale_corr = 1.1f};
|
||||
|
||||
std::vector<SpotToSave> spots(max_spots);
|
||||
|
||||
@@ -2431,7 +2431,7 @@ static Reflection MakeTestReflection(int i, int j) {
|
||||
.I = 1000.0f + f,
|
||||
.bkg = 10.0f + f,
|
||||
.sigma = 2.0f + 0.5f * f,
|
||||
.rlp = 1.0f + 0.125f * f,
|
||||
.prescaling_corr = 1.0f + 0.125f * f,
|
||||
.partiality = 0.5f + 0.01f * f,
|
||||
.zeta = 0.01f + 0.001f * f,
|
||||
.image_scale_corr = 1.0f + 0.25f * f
|
||||
@@ -2458,7 +2458,7 @@ static void CheckReflectionRoundTrip(const Reflection &got, int i, int j) {
|
||||
CHECK(got.I == Catch::Approx(want.I));
|
||||
CHECK(got.bkg == Catch::Approx(want.bkg));
|
||||
CHECK(got.sigma == Catch::Approx(want.sigma));
|
||||
CHECK(got.rlp == Catch::Approx(want.rlp)); // stored as 1/rlp, inverted again on read
|
||||
CHECK(got.prescaling_corr == Catch::Approx(want.prescaling_corr)); // stored as 1/prescaling_corr, inverted again on read
|
||||
CHECK(got.partiality == Catch::Approx(want.partiality));
|
||||
CHECK(got.zeta == Catch::Approx(want.zeta));
|
||||
CHECK(got.image_scale_corr == Catch::Approx(want.image_scale_corr));
|
||||
|
||||
@@ -94,7 +94,7 @@ TEST_CASE("AcceptReflection_ResolutionLimits") {
|
||||
Reflection r{};
|
||||
r.I = 100.0f;
|
||||
r.sigma = 5.0f;
|
||||
r.rlp = 1.0f;
|
||||
r.prescaling_corr = 1.0f;
|
||||
r.d = 20.0f;
|
||||
|
||||
// No limits: only the finiteness checks apply.
|
||||
|
||||
@@ -90,7 +90,7 @@ int main(int argc, char **argv) {
|
||||
.I = r.I,
|
||||
.last_image = i,
|
||||
.count = 1,
|
||||
.rlp = r.rlp,
|
||||
.rlp = r.prescaling_corr,
|
||||
.image_number = r.image_number
|
||||
};
|
||||
bool found = false;
|
||||
|
||||
@@ -91,7 +91,7 @@ void JFJochViewerReflectionListWindow::addReflectionRow(int index, const Reflect
|
||||
QString::number(r.bkg, 'f', 1)));
|
||||
|
||||
row.append(mkNumItem(r.partiality, QString::number(r.partiality, 'f', 2)));
|
||||
row.append(mkNumItem(r.rlp, QString::number(r.rlp, 'f', 3)));
|
||||
row.append(mkNumItem(r.prescaling_corr, QString::number(r.prescaling_corr, 'f', 3)));
|
||||
tableModel->appendRow(row);
|
||||
}
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ void HDF5DataFilePluginReflection::Write(const DataMessage &msg, uint64_t image_
|
||||
obs_y.emplace_back(refl.observed_y);
|
||||
bkg.emplace_back(refl.bkg);
|
||||
var_bkg.emplace_back(refl.var_bkg);
|
||||
lp.emplace_back(1.0/refl.rlp);
|
||||
lp.emplace_back(1.0/refl.prescaling_corr);
|
||||
partiality.emplace_back(refl.partiality);
|
||||
phi.emplace_back(refl.delta_phi_deg);
|
||||
zeta.emplace_back(refl.zeta);
|
||||
|
||||
Reference in new issue
Block a user