diff --git a/docs/ACKNOWLEDGEMENT.md b/docs/ACKNOWLEDGEMENT.md index f50c63d6c..ceef72480 100644 --- a/docs/ACKNOWLEDGEMENT.md +++ b/docs/ACKNOWLEDGEMENT.md @@ -281,3 +281,12 @@ Report of the IUCr Subcommittee on Statistical Descriptors" (1989), Acta Cryst. S. C. Abrahams, H. D. Flack, E. Prince and A. J. C. Wilson, "Statistical descriptors in crystallography. II. Report of a Working Group on Expression of Uncertainty in Measurement" (1995), Acta Cryst. A51, 565-569 [doi:10.1107/S0108767395002340](https://doi.org/10.1107/S0108767395002340). + +**Sensor absorption at oblique incidence** — the angle-dependent quantum efficiency of a flat +sensor, and the radial parallax variance that comes from the same integral, are the Beer-Lambert law +taken along a ray that crosses t/cos(alpha) of sensor and converts at a random depth. The +attenuation coefficients are the NIST tabulation: J. H. Hubbell and S. M. Seltzer, "Tables of X-Ray +Mass Attenuation Coefficients and Mass Energy-Absorption Coefficients from 1 keV to 20 MeV for +Elements Z = 1 to 92 and 48 Additional Substances of Dosimetric Interest" (1995, data updated 2004), +NIST Standard Reference Database 126 +[doi:10.18434/T4D01F](https://doi.org/10.18434/T4D01F). diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index d4665f8ed..cdfdf9501 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -3,6 +3,7 @@ ### 1.0.0-rc.166 +* `rugnux` corrects each reflection for the sensor's quantum efficiency at the angle the diffracted beam meets the detector. * `rugnux --model` treats the model as a hypothesis: it decides the enantiomorph and the indexing only where its R-work beats that of the same model in random orientations, and a model the data reject is still scored, placed and mapped, but leaves the reflection files byte for byte what a run with no model writes. * `rugnux --model` places the model against the data as a rigid body before scoring it, writes sigma_A-weighted 2mFo-DFc and mFo-DFc maps in place of the unweighted 2Fo-Fc and Fo-Fc, and writes the model as it was placed - `_model.cif`, and `_model.pdb` where the PDB format can express the cell - in the cell and space group of the reflection files beside it. * `rugnux` and `jfjoch_viewer` read PILATUS miniCBF sweeps natively, and open masters written at other facilities, including Eiger 1.x and third-party NXmx. diff --git a/image_analysis/SensorAbsorption.h b/image_analysis/SensorAbsorption.h new file mode 100644 index 000000000..6b72f6c51 --- /dev/null +++ b/image_analysis/SensorAbsorption.h @@ -0,0 +1,143 @@ +// SPDX-FileCopyrightText: 2026 Filip Leonarski, Paul Scherrer Institute +// SPDX-License-Identifier: GPL-3.0-only + +#ifndef JUNGFRAUJOCH_SENSORABSORPTION_H +#define JUNGFRAUJOCH_SENSORABSORPTION_H + +#include +#include + +// How much of the beam a flat sensor stops, as a function of the angle at which the beam enters it. +// +// A photon arriving at incidence angle alpha to the sensor normal crosses t/cos(alpha) of sensor +// instead of t, so more of it is absorbed: the detector is MORE efficient at high angle than at +// normal incidence, and an uncorrected reflection out at the detector edge reads high. Dividing by +// that ratio is the whole correction. +// +// a(alpha) = t / (L cos alpha) crossing length in attenuation lengths +// QE(alpha) = 1 - exp(-a) absorbed fraction +// correction to I = QE(0) / QE(alpha) normalised so normal incidence is untouched +// +// Two limits, both physical and both reached in this corpus: +// - L << t (long wavelength): every photon stops in the entrance skin, QE = 1 at every angle and +// the correction is exactly 1. This is why the low-energy case is inert rather than gated off. +// - L >> t (thin sensor, hard X-rays): absorption is proportional to path length and the +// correction tends to cos(alpha) - a factor of 2 at 2theta = 60 degrees. This is the regime +// small-molecule work at 18-25 keV lives in. +// +// alpha is the angle to the DETECTOR NORMAL, not the scattering angle. The two coincide only on an +// untilted detector. On a tilted one the correction acquires an azimuthal dependence at fixed +// resolution, which is the only part of it that is not degenerate with an overall Wilson B. +// +// Attenuation coefficients: NIST X-Ray Mass Attenuation Coefficients (Hubbell & Seltzer), total +// mu/rho with coherent scattering. Photoelectric absorption dominates over the whole range used +// here, and because the correction is a RATIO of two absorbed fractions the photoelectric branching +// ratio cancels exactly; only the attenuation length enters. Coherent scattering, which attenuates +// without converting, is a few per cent of mu/rho in Si below 20 keV and is kept in the total - it +// biases L slightly short, and the correction depends on L only through t/L. +// +// NOT modelled, and negligible where this is used: K-fluorescence escape from the sensor (in CdTe +// above the Cd K edge at 26.71 keV a fluorescence photon can leave the sensor, so the charge is +// recorded in the wrong pixel or not at all); charge sharing between pixels; and the obliquity of +// the entrance window. Above the Cd/Te K edges the escape term is large and this model should not +// be trusted as it stands. +namespace sensor_absorption { + +// NIST mass attenuation coefficient mu/rho [cm^2/g] against photon energy [keV]. Absorption edges +// are the repeated energies - the table is walked in order and interpolated log-log, which is how +// these tables are meant to be read. Si has only the K edge at 1.839 keV, far below anything used. +struct MuRow { double E_keV, mu_rho; }; + +inline constexpr MuRow SI_TABLE[] = { + {1.0, 1570.0}, {1.5, 535.5}, {1.8389, 309.2}, {1.8389, 3192.0}, {2.0, 2777.0}, + {3.0, 978.4}, {4.0, 452.9}, {5.0, 245.0}, {6.0, 147.0}, {8.0, 64.68}, {10.0, 33.89}, + {15.0, 10.34}, {20.0, 4.464}, {30.0, 1.436}, {40.0, 0.7012}, {50.0, 0.4385}}; + +inline constexpr MuRow CD_TABLE[] = { + {1.0, 7350.0}, {1.5, 2931.0}, {2.0, 1473.0}, {3.0, 541.4}, {3.5375, 357.5}, + {3.5375, 1152.0}, {3.63101, 1083.0}, {3.727, 1013.0}, {3.727, 1389.0}, {4.0, 1170.0}, + {4.018, 1157.0}, {4.018, 1324.0}, {5.0, 768.5}, {6.0, 479.3}, {8.0, 225.4}, {10.0, 124.4}, + {15.0, 41.78}, {20.0, 19.20}, {26.7112, 8.809}, {26.7112, 50.65}, {30.0, 37.65}, + {40.0, 17.78}, {50.0, 9.779}}; + +inline constexpr MuRow TE_TABLE[] = { + {1.0, 8434.0}, {1.5, 3608.0}, {2.0, 1832.0}, {3.0, 679.2}, {4.0, 329.7}, {4.3414, 267.8}, + {4.3414, 788.2}, {4.47465, 750.4}, {4.612, 699.5}, {4.612, 944.5}, {4.7728, 878.2}, + {4.9392, 806.2}, {4.9392, 929.2}, {5.0, 901.4}, {6.0, 572.1}, {8.0, 270.2}, {10.0, 150.1}, + {15.0, 50.78}, {20.0, 23.41}, {30.0, 7.878}, {31.8138, 6.738}, {31.8138, 37.19}, + {40.0, 20.64}, {50.0, 11.45}}; + +// Log-log interpolation, clamped at both ends of the table. +template +double InterpMuRho(const MuRow (&tab)[N], double E_keV) { + if (E_keV <= tab[0].E_keV) + return tab[0].mu_rho; + if (E_keV >= tab[N - 1].E_keV) + return tab[N - 1].mu_rho; + std::size_t i = 1; + while (i < N - 1 && tab[i].E_keV < E_keV) + i++; + const double e0 = tab[i - 1].E_keV, e1 = tab[i].E_keV; + if (!(e1 > e0)) // the two rows of an absorption edge: take the upper side + return tab[i].mu_rho; + const double f = (std::log(E_keV) - std::log(e0)) / (std::log(e1) - std::log(e0)); + return std::exp(std::log(tab[i - 1].mu_rho) + f * (std::log(tab[i].mu_rho) - std::log(tab[i - 1].mu_rho))); +} + +// Attenuation length 1/mu [um] of a sensor material at a given wavelength. Si and CdTe are the +// sensors in use; an unrecognised material is treated as silicon, which is what DetectorSetup +// defaults to anyway. +inline double AttenuationLength_um(const std::string &material, double lambda_A) { + if (!(lambda_A > 0.0)) + return 0.0; + const double E_keV = 12.39842 / lambda_A; + double mu_rho_cm2_g, rho_g_cm3; + if (material == "CdTe") { + // Mass fractions from the atomic weights, Cd 112.414 and Te 127.60. + constexpr double w_cd = 112.414 / 240.014, w_te = 127.60 / 240.014; + mu_rho_cm2_g = w_cd * InterpMuRho(CD_TABLE, E_keV) + w_te * InterpMuRho(TE_TABLE, E_keV); + rho_g_cm3 = 5.85; + } else { + mu_rho_cm2_g = InterpMuRho(SI_TABLE, E_keV); + rho_g_cm3 = 2.3290; + } + const double mu_cm = mu_rho_cm2_g * rho_g_cm3; + return mu_cm > 0.0 ? 1e4 / mu_cm : 0.0; +} + +// Everything the per-reflection correction needs, reduced to the two numbers that are constant for +// a dataset. Built once on the host; the GPU predictor takes the two floats. +struct SensorQE { + float a0 = 0.0f; // t / L, the optical thickness at normal incidence + float qe0 = 1.0f; // absorbed fraction at normal incidence + bool active = false; // false where the sensor is opaque and the correction is exactly 1 + + // Opaque beyond this: exp(-20) = 2e-9, so QE(0)/QE(alpha) rounds to exactly 1.0f for every + // incidence angle and the correction is bit-identical to not applying it. This is what makes + // the long-wavelength case inert without a flag or a threshold anyone has to choose. + static constexpr float OPAQUE_A0 = 20.0f; + + static SensorQE Build(const std::string &material, double thickness_um, double lambda_A) { + SensorQE q; + const double L = AttenuationLength_um(material, lambda_A); + if (!(thickness_um > 0.0) || !(L > 0.0)) + return q; + q.a0 = static_cast(thickness_um / L); + q.qe0 = static_cast(1.0 - std::exp(-q.a0)); + q.active = q.a0 < OPAQUE_A0; + return q; + } + + // The multiplicative correction to an intensity recorded at incidence angle alpha: QE(0)/QE(alpha). + // Always <= 1, because a sensor is more efficient off-normal than head-on. + [[nodiscard]] float Factor(float cos_alpha) const { + if (!active || !(cos_alpha > 1e-3f)) + return 1.0f; + const float qe = 1.0f - std::exp(-a0 / cos_alpha); + return qe > 0.0f ? qe0 / qe : 1.0f; + } +}; + +} // namespace sensor_absorption + +#endif // JUNGFRAUJOCH_SENSORABSORPTION_H diff --git a/image_analysis/bragg_prediction/BraggPrediction.cpp b/image_analysis/bragg_prediction/BraggPrediction.cpp index de48b63a3..a43e1cc05 100644 --- a/image_analysis/bragg_prediction/BraggPrediction.cpp +++ b/image_analysis/bragg_prediction/BraggPrediction.cpp @@ -6,6 +6,7 @@ #include "../../common/JFJochMath.h" #include "../../common/Logger.h" #include "BraggPrediction.h" +#include "../SensorAbsorption.h" #include "../bragg_integration/SystematicAbsence.h" void BraggPrediction::GrowCapacity(int count) { @@ -92,6 +93,12 @@ int BraggPrediction::Calc(const DiffractionExperiment &experiment, const Crystal float pixel_size = geom.GetPixelSize_mm(); float F = det_distance / pixel_size; + // Angle-dependent sensor efficiency; per-dataset constants collapsed to two numbers. Inert + // wherever the sensor is opaque, which is every long wavelength. + const auto &det_setup = experiment.GetDetectorSetup(); + const auto sensor_qe = sensor_absorption::SensorQE::Build( + det_setup.GetSensorMaterial(), det_setup.GetSensorThickness_um(), geom.GetWavelength_A()); + const float epsilon = 1e-5f; const float s0_sq = S0 * S0; const float rad_to_deg = 180.0f / static_cast(PI); @@ -185,6 +192,12 @@ int BraggPrediction::Calc(const DiffractionExperiment &experiment, const Crystal if ((x < 0) || (x >= det_width_pxl) || (y < 0) || (y >= det_height_pxl)) continue; + // Sensor quantum efficiency at this reflection's angle of incidence on the + // detector - the diffracted direction against the detector normal, so the + // detector tilt is carried for free. See sensor_absorption::SensorQE. + const float qe_corr = sensor_qe.Factor( + S_rot_z / std::sqrt(S_x * S_x + S_y * S_y + S_z * S_z)); + float d = 1.0f / sqrtf(recip_sq); reflections[i] = Reflection{ .h = h, @@ -197,10 +210,10 @@ int BraggPrediction::Calc(const DiffractionExperiment &experiment, const Crystal .observed_y = NAN, .d = d, .dist_ewald = dist_ewald_sphere, - .rlp = 1.0, + .rlp = qe_corr, .partiality = 1.0f, .zeta = 1.0, - .image_scale_corr = 1.0 + .image_scale_corr = qe_corr }; ++i; } diff --git a/image_analysis/bragg_prediction/BraggPredictionRot.cpp b/image_analysis/bragg_prediction/BraggPredictionRot.cpp index 59fb41c81..b660ac1fa 100644 --- a/image_analysis/bragg_prediction/BraggPredictionRot.cpp +++ b/image_analysis/bragg_prediction/BraggPredictionRot.cpp @@ -3,6 +3,7 @@ #include "../../common/JFJochMath.h" #include "BraggPredictionRot.h" +#include "../SensorAbsorption.h" #include "../bragg_integration/SystematicAbsence.h" @@ -60,6 +61,14 @@ int BraggPredictionRot::Calc(const DiffractionExperiment &experiment, const Crys const float bandwidth_sigma = settings.bandwidth_sigma; const float half_wavelength_A = geom.GetWavelength_A() / 2.0f; + // Angle-dependent sensor efficiency. Per-dataset constants (thickness, material, wavelength) + // collapse to two numbers here; the per-reflection part is one exponential below. Inert - and + // bit-identical to not applying it - wherever the sensor is opaque, which is every long + // wavelength, so it needs no flag and no threshold anyone has to choose. + const auto &det = experiment.GetDetectorSetup(); + const auto sensor_qe = sensor_absorption::SensorQE::Build( + det.GetSensorMaterial(), det.GetSensorThickness_um(), geom.GetWavelength_A()); + for (int h = -settings.max_h; h <= settings.max_h; h++) { // Precompute A* h contribution @@ -165,6 +174,15 @@ int BraggPredictionRot::Calc(const DiffractionExperiment &experiment, const Crys float dist_ewald_sphere = std::fabs(S.Length() - one_over_wavelength); + // Sensor quantum efficiency at this reflection's own angle of incidence on the + // detector. The angle is taken against the DETECTOR NORMAL - S_rot is the + // diffracted direction in the detector's own frame, so its z component over its + // length is that cosine already, at no cost. Taking it here rather than from the + // resolution is what makes it right on a tilted detector, where the incidence + // angle stops being a function of resolution and the correction stops + // cancelling within a resolution shell. + const float qe_corr = sensor_qe.Factor(S_rot_z / S.Length()); + float d = 1.0f / sqrtf(p0_sq); reflections[i] = Reflection{ .h = h, @@ -177,10 +195,10 @@ int BraggPredictionRot::Calc(const DiffractionExperiment &experiment, const Crys .observed_y = NAN, .d = d, .dist_ewald = dist_ewald_sphere, - .rlp = lorentz_reciprocal, + .rlp = lorentz_reciprocal * qe_corr, .partiality = partiality, .zeta = zeta_abs, - .image_scale_corr = lorentz_reciprocal / partiality, + .image_scale_corr = lorentz_reciprocal * qe_corr / partiality, }; i++; } diff --git a/image_analysis/bragg_prediction/BraggPredictionRotGPU.cu b/image_analysis/bragg_prediction/BraggPredictionRotGPU.cu index 63126406b..b114fee5b 100644 --- a/image_analysis/bragg_prediction/BraggPredictionRotGPU.cu +++ b/image_analysis/bragg_prediction/BraggPredictionRotGPU.cu @@ -4,6 +4,7 @@ #include #include "../../common/JFJochMath.h" #include "BraggPredictionRotGPU.h" +#include "../SensorAbsorption.h" #ifdef JFJOCH_USE_CUDA #include "../indexing/CUDAMemHelpers.h" @@ -166,6 +167,19 @@ namespace { float dist_ewald = fabsf(sqrtf(Sx * Sx + Sy * Sy + Sz * Sz) - C.one_over_wavelength); + // Sensor quantum efficiency at this reflection's angle of incidence on the detector. + // Sr is the diffracted direction in the detector's own frame, so Srz over its length is + // the cosine to the detector NORMAL - which carries detector tilt for free. Mirrors + // sensor_absorption::SensorQE::Factor on the CPU side. + float qe_corr = 1.0f; + if (C.qe_a0 > 0.0f) { + float cos_alpha = Srz / sqrtf(Sx * Sx + Sy * Sy + Sz * Sz); + if (cos_alpha > 1e-3f) { + float qe = 1.0f - expf(-C.qe_a0 / cos_alpha); + if (qe > 0.0f) qe_corr = C.qe_qe0 / qe; + } + } + out[count].h = h; out[count].k = k; out[count].l = l; @@ -176,10 +190,10 @@ namespace { out[count].observed_y = NAN; out[count].d = 1.0f / sqrtf(p0_sq); out[count].dist_ewald = dist_ewald; - out[count].rlp = lorentz; + out[count].rlp = lorentz * qe_corr; out[count].partiality = partiality; out[count].zeta = zeta_abs; - out[count].image_scale_corr = lorentz / partiality; + out[count].image_scale_corr = lorentz * qe_corr / partiality; count++; } return count; @@ -234,6 +248,12 @@ namespace { kc.bandwidth_sigma = settings.bandwidth_sigma; kc.half_wavelength_A = geom.GetWavelength_A() / 2.0f; + const auto &det = experiment.GetDetectorSetup(); + const auto sensor_qe = sensor_absorption::SensorQE::Build( + det.GetSensorMaterial(), det.GetSensorThickness_um(), geom.GetWavelength_A()); + kc.qe_a0 = sensor_qe.active ? sensor_qe.a0 : 0.0f; + kc.qe_qe0 = sensor_qe.qe0; + kc.Astar = lattice.Astar(); kc.Bstar = lattice.Bstar(); kc.Cstar = lattice.Cstar(); diff --git a/image_analysis/bragg_prediction/BraggPredictionRotGPU.h b/image_analysis/bragg_prediction/BraggPredictionRotGPU.h index f58ec0abb..7e06a1e58 100644 --- a/image_analysis/bragg_prediction/BraggPredictionRotGPU.h +++ b/image_analysis/bragg_prediction/BraggPredictionRotGPU.h @@ -22,6 +22,11 @@ struct KernelConstsRot { float mosaicity_multiplier; float bandwidth_sigma; // relative dlambda/lambda as a sigma; 0 = monochromatic float half_wavelength_A; + // Angle-dependent sensor efficiency, reduced to the two per-dataset numbers the kernel needs: + // the optical thickness at normal incidence and the absorbed fraction there. qe_a0 = 0 leaves + // every reflection untouched (see sensor_absorption::SensorQE). + float qe_a0; + float qe_qe0; Coord Astar, Bstar, Cstar, S0; Coord m1, m2, m3; float m2_S0; diff --git a/reader/HDF5MetadataSource.cpp b/reader/HDF5MetadataSource.cpp index fa86cf8d6..ef24274ec 100644 --- a/reader/HDF5MetadataSource.cpp +++ b/reader/HDF5MetadataSource.cpp @@ -1006,11 +1006,25 @@ HDF5MetadataSource::OpenResult HDF5MetadataSource::Open(const std::string &filen .value_or(0)))); // Sensor thickness/material drive the parallax/absorption model, so take them from the file // rather than the DetectorSetup default. + // Say so when they are absent rather than let the DetectorSetup default stand in silently. + // The default (320 um Si) is right for every JUNGFRAU and wrong for the 450 um Si and 750 um + // CdTe sensors this code also reads, and both the angle-dependent efficiency correction and + // the parallax variance term are computed from these two numbers - so a file that omits them + // gets a correction derived from an assumption, which the user should be told about. The + // correction is still applied: a 320 um Si assumption is closer to every real sensor than + // switching the physics off, and declining would silently disagree with the same data read + // from a file that does state its sensor. if (master_file->Exists("/entry/instrument/detector/sensor_thickness")) detector.SensorThickness_um( ReadLength_m(*master_file, "/entry/instrument/detector/sensor_thickness") * 1e6); + else + Logger("HDF5Reader").Warning("No sensor_thickness in the file; assuming {:.0f} um for the " + "sensor absorption model", detector.GetSensorThickness_um()); if (master_file->Exists("/entry/instrument/detector/sensor_material")) detector.SensorMaterial(master_file->GetString("/entry/instrument/detector/sensor_material")); + else + Logger("HDF5Reader").Warning("No sensor_material in the file; assuming {} for the sensor " + "absorption model", detector.GetSensorMaterial()); // Optional, because a file that states no saturation value anywhere is a real and common // thing: an Eiger master links saturation_value into a companion _meta.h5, and a deposited // dataset frequently does not include that file, leaving neither the NXmx name nor the