Files
Jungfraujoch/image_analysis/WriteReflections.h
T
leonarski_fandFilip Leonarski 19b135186d
Build Packages / Create release (push) Successful in 7s
Build Packages / build:portable:macos-arm64:nocuda (push) Successful in 4m56s
Build Packages / build:rugnux:linux-aarch64:cuda (push) Successful in 9m10s
Build Packages / build:portable:linux-x86_64:nocuda (push) Successful in 13m51s
Build Packages / build:portable:linux-x86_64:cuda (push) Successful in 16m56s
Build Packages / build:jfjoch:rocky8:nocuda (push) Successful in 20m59s
Build Packages / build:portable:windows-x86_64:nocuda (push) Successful in 21m25s
Build Packages / build:jfjoch:rocky9:nocuda (push) Successful in 21m17s
Build Packages / build:jfjoch:ubuntu2204:nocuda (push) Successful in 14m14s
Build Packages / HDF5 consumer tests (DIALS, XDS) (push) Successful in 25m59s
Build Packages / build:portable:windows-x86_64:cuda (push) Successful in 25m48s
Build Packages / build:jfjoch:ubuntu2404:nocuda (push) Successful in 15m52s
Build Packages / build:jfjoch:rocky8:cuda-sls9 (push) Successful in 19m33s
Build Packages / Generate python client (push) Successful in 29s
Build Packages / Build documentation (push) Successful in 1m19s
Build Packages / build:jfjoch:ubuntu2204:cuda (push) Successful in 20m49s
Build Packages / build:jfjoch:rocky9:cuda-sls9 (push) Successful in 25m31s
Build Packages / build:jfjoch:rocky8:cuda (push) Successful in 25m24s
Build Packages / build:jfjoch:rocky9:cuda (push) Successful in 26m5s
Build Packages / build:jfjoch:ubuntu2404:cuda (push) Successful in 22m19s
Build Packages / Unit tests (push) Failing after 2h28m24s
Build Packages / Validation Battery (push) Successful in 40m33s
v1.0.0-rc.175 (#85)
* Rugnux: Processes a collection live while jfjoch_broker writes it (`rugnux <master.h5> <broker URL>`).
* Rugnux: Processing starts on the first images and confirms its choices once all are in, so analysis can begin during collection.
* Rugnux: Faster processing, especially on servers with several GPUs and CPU sockets and on computers without a GPU (`--no-gpu`, macOS).
* Rugnux: Uses much less GPU memory, so large data sets stay on the GPU of smaller cards, and a data set too large for the card is processed on the CPU instead.
* Rugnux: On macOS, Bragg integration runs on the Apple GPU.
* Rugnux: Better handling of twinning and pseudo-symmetry, with the twin hypothesis written as `<prefix>_twin_<SG>.mtz`, and a wrong `--model` no longer confirms the data or decides the hand.
* Rugnux: `--model` accepts predicted (AlphaFold) models and falls back to molecular replacement when the model is not placed as a rigid body (experimental `--mr` forces it), settling space group, hand and twinning and writing a different answer as `<prefix>_mr.mtz`.
* Rugnux: Molecular replacement is faster and settles the space group in more cases with several copies in the asymmetric unit.
* Rugnux: Deposition-ready mmCIF, a richer report, and numbered steps on the terminal with the full log in `<prefix>_log.txt`; `_P1.mtz`, `--estimate-beam-center` and `--beam-center-search` are removed.
* Rugnux: The beam-stop shadow is found correctly on tilted detectors.
* Rugnux: Stills files that record a stationary goniometer axis get the default stills geometry refinement.
* jfjoch_viewer: "Analyze dataset" uses the same defaults as a bare `rugnux <file>`, and shows the results of processed CBF, marCCD and SMV sweeps.
* jfjoch_viewer: The Files panel refreshes on F5 and every few seconds, and the plot type and other selections stay put during live collection.
* jfjoch_writer: Memory stays bounded and old data files are closed efficiently on long runs, any failure ends the run in an error state the broker is told about, and the master file appears only after its data files; the ZeroMQ pusher logs when a stalled writer makes it drop images.
* Fixes: The TCP image pusher no longer blocks the broker's receiver, GPU out-of-memory failures stop with a clear message, and the XDS plugin masks module and chip edges.

---------

Co-authored-by: Filip Leonarski <jungfrau@filips-mac-mini.home>
Reviewed-on: #85
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-10-16 15:34:43 +02:00

114 lines
6.2 KiB
C++

// SPDX-FileCopyrightText: 2025 Paul Scherrer Institute
// SPDX-License-Identifier: GPL-3.0-only
#pragma once
#include <string>
#include <vector>
#include "../common/Reflection.h"
#include "../common/UnitCell.h"
#include "../common/DiffractionExperiment.h"
#include "IntegrationOutcome.h"
#include <gemmi/mtz.hpp>
struct MergeStatistics;
struct TwinningAnalysisResult;
// The error model as it is reported, already formatted. `isa` is the whole-range 1/sqrt(a*b), the
// same quantity XDS's ISa denotes, so a file written here is directly comparable with a CORRECT.LP;
// `isa_asymptotic` is the strong-reflection tier, which only the rotation path has. `a` and `b` are
// in XDS's convention, sigma^2 = a*(sigma0^2 + b*I^2). Empty strings are written as unknown.
struct ErrorModelReport {
std::string isa;
std::string isa_asymptotic;
std::string a;
std::string b;
};
// What the reflection files record about where they came from, beyond the experiment itself and the
// Rugnux version and commit: the MTZ HISTORY records and the mmCIF's _software and
// _jfjoch_provenance. Empty strings are left out.
struct OutputProvenance {
std::string command_line;
std::string input_file; // the input as found on disk, absolute
std::string input_identity; // a cheap identifier of it: size and modification time
std::string collection_date; // as the input states it (NXmx /entry/start_time), ISO-8601
};
// The merged reflections, their statistics and the experiment, as one PDBx/mmCIF data block.
// nthreads: workers for the per-reflection row formatting, which is the bulk of the file.
void WriteMmcifReflections(const std::vector<MergedReflection> &reflections,
const UnitCell &unitCell,
const DiffractionExperiment &experiment,
const MergeStatistics &statistics,
const ErrorModelReport &error_model,
const TwinningAnalysisResult &twinning,
const OutputProvenance &provenance,
const std::string &filename,
size_t nthreads);
void WriteMtzReflections(const std::vector<MergedReflection> &reflections,
const UnitCell &unitCell,
const DiffractionExperiment &experiment,
const OutputProvenance &provenance,
const std::string &filename);
// SHELX HKLF-4 text file, FORMAT(3I4,2F8.2) h k l I sigma(I), scaled so the largest value fits the
// field, ended by a 0 0 0 record. nthreads: workers for the row formatting, as for the mmCIF.
// Merged reflections, Bijvoet mates as separate records (the stills output).
void WriteShelxHklReflections(const std::vector<MergedReflection> &reflections,
const DiffractionExperiment &experiment,
const std::string &filename,
size_t nthreads);
// Unmerged scaled fulls, one record each at its measured index (the rotation output).
void WriteShelxHklReflections(const std::vector<ScaledFull> &fulls,
const std::string &filename,
size_t nthreads);
// Unmerged observations in the column and batch-header layout POINTLESS writes: aimless, pointless,
// careless and iotbx.merging_statistics all read that layout. H K L are the ASU indices and M/ISYM
// recovers the index the reflection was measured at (which is what careless needs to see the crystal
// frame) and says whether the observation is a partial.
// sum_partials: add the partials of each rocking event into one full, written at the batch of the
// event's centroid with the summed rocking-curve fraction in FRACTIONCALC - the plain sum every
// rotation program writes, over the events the combine would assemble (--min-partiality). False
// writes one row per integrated box instead, flagged as partials for the reading program to sum.
// Stills have no rocking events and are unaffected either way.
// The intensities carry the deterministic per-reflection corrections and nothing else - Lorentz
// and polarization in LP, the sensor's angle-dependent efficiency in QE, and the attenuation of the
// flight-path medium in FLIGHT - so raw counts are I / LP * QE * FLIGHT. The partiality and the
// per-image scale are left for the reading program to fit, since every program this file is for
// fits a scale model of its own.
void WriteUnmergedMtzReflections(const std::vector<IntegrationOutcome> &outcomes,
const UnitCell &unitCell,
const DiffractionExperiment &experiment,
bool sum_partials,
const OutputProvenance &provenance,
const std::string &filename,
size_t nthreads = 1);
// The same file in two steps, for a caller that builds it while the outcomes' per-image scale fields
// are still being written: UnmergedMtz reads everything but mosaicity_deg, which
// SetUnmergedMtzMosaicity then copies into the batch headers. The Mtz points at the experiment's space
// group, so the experiment has to outlive it.
gemmi::Mtz UnmergedMtz(const std::vector<IntegrationOutcome> &outcomes,
const UnitCell &unitCell,
const DiffractionExperiment &experiment,
bool sum_partials,
const OutputProvenance &provenance,
size_t nthreads = 1);
void SetUnmergedMtzMosaicity(gemmi::Mtz &mtz, const std::vector<IntegrationOutcome> &outcomes);
void WriteReflections(const std::vector<MergedReflection> &reflections,
const UnitCell &unitCell,
const DiffractionExperiment &experiment,
const MergeStatistics &statistics,
const ErrorModelReport &error_model,
const TwinningAnalysisResult &twinning,
const OutputProvenance &provenance,
const std::string &filename,
size_t nthreads,
const std::vector<ScaledFull> *scaled_fulls = nullptr);