Jungfraujoch/docs/RUGNUX_REPORT.md

The results report

:local:
:depth: 2

<prefix>_report.txt records what the run determined, next to the reflection files. It is written on every --mode mx and --mode scale run that has an output prefix — there is no option to enable or disable it. Two cases follow from that:

• An empty output prefix (-o "", the "compute the statistics, persist nothing" mode) writes nothing, the report included.
• --no-merge still writes a report. It determined an indexing and a geometry result, and those are recorded; the merging section then says MERGE= NOT_PERFORMED rather than being omitted, so the absence is a statement and not something a reader has to infer.

The report is never allowed to fail a run: if it cannot be written (unwritable path, full disk) the failure is logged as a warning and the run finishes normally.

Exit status. rugnux exits 0 when the run completed — everything it determined, declined to determine (CANNOT_DETERMINE) or warned about is in the report — and non-zero when the run stopped: unreadable input, no usable lattice, a -S group the indexed lattice cannot host, an indexer that could not run. The reason goes to the terminal, and the report may not exist in that case — so a script branches on the exit status first and greps the report second.

Format

The model is XDS's CORRECT.LP: prose and tables a crystallographer reads top to bottom, with a structure a script can consume without parsing prose.

• KEY= value assignment lines. Every number worth extracting is one, so a consumer gets it with a single grep '^ISA= ' and never has to read a sentence. Key names are stable.
• Fixed-width tables with a stable header row for anything that is genuinely tabular — the resolution shells, the space-group candidates, the sweep-quality ranges.
• WARNING: lines, one per finding, in plain English: WARNING: Frames 500-600 out of beam (10.1 deg, scale 0.12 and CC 0.30 of the run, 2% scaled). grep '^WARNING:' finds every one.
• Section banners (***…*** around a numbered title) delimiting the blocks.

REPORT_VERSION= is the format's own version. Key names, table columns and the reason vocabulary below are an interface other software may depend on: they do not change without that number moving. Adding a key does not move it — a consumer that greps for what it needs is unaffected by one more line.

The header block above section 1 records how the result was produced: RUGNUX_VERSION= and RUGNUX_GIT=, DATE=, INPUT_FILE= and OUTPUT_PREFIX=, plus

• COMMAND_LINE= — the invocation as one shell-ready line, arguments containing spaces quoted.
• WALL_TIME= — the whole invocation in seconds. It covers everything the process did, opening the file and setting up included, so it is a little larger than the Processing time printed on stdout, which starts once the analysis does.
• GPU_COUNT= and GPU= — how many GPUs were visible and what they are, e.g. GPU= 4x NVIDIA A100-SXM4-80GB; several models on one machine are listed as separate groups. GPU_COUNT= 0 appears on its own, with no GPU= line, when nothing was visible — which is the first thing to check when a run took far longer than expected. rugnux prints the same line at startup, before the run, so a missing GPU can be caught while there is still time to stop.

Rates, per-image costs and progress remain on stdout only.

Sections, in order: 1. DATA SET, 2. INDEXING, 3. GEOMETRY POST-REFINEMENT (rotation only), 4. SPACE GROUP DETERMINATION, 5. SCALING AND MERGING, 6. TWINNING, 7. RADIATION DAMAGE, 8. SWEEP QUALITY, 9. DIFFRACTION ANISOTROPY, 10. MODEL VALIDATION (only with --model), 11. WARNINGS. Numbers are fixed: a section that does not apply is left out and the ones after it keep their numbers.

SPOT_RESOLUTION_ESTIMATE= in section 1 is how far the merged data are expected to reach, read off the found spots alone — no lattice, no integration, no merge — so it is there on a run that never merges, and on a run that does it can be read against INCLUDE_RESOLUTION_RANGE in section 5. It is a prediction, good to about 0.2 Å on rotation data; nothing is cut on it. It is not limited to what the detector records: where it reads finer than the high-resolution end of INCLUDE_RESOLUTION_RANGE, the crystal diffracts past the corner and the run was detector-limited.

JFJOCH_DATASET_SETTINGS= in section 1 is the geometry the run integrated at — on a rotation run the post-refined one — written as the object jfjoch_broker takes it in: the four required properties of dataset_settings in broker/jfjoch_api.yaml, on one line of valid JSON, so a refined beam centre and distance can go back to the instrument for the next collection without anyone retyping them.

JFJOCH_DATASET_SETTINGS= {"beam_x_pxl": 2078.24, "beam_y_pxl": 2233.92, "detector_distance_mm": 190.311, "incident_energy_keV": 12.4000}

grep '^JFJOCH_DATASET_SETTINGS=' out_report.txt | cut -d' ' -f2- > geometry.json

Which pass. A rotation run integrates twice — once at the geometry in the input file, then again at the post-refined geometry — and can integrate a third time if a guard rejects the second pass. There is one report, for the pass that became the canonical output, and PASS= / PASS_DECISION= in section 1 say which pass that is and on what evidence, so no number in the file is ambiguous about which geometry produced it.

Not in the report: timing, frame rates, thread counts, per-image progress and library banners. Those are process, not result, and stay on stdout.

Sweep quality and the reason vocabulary

Section 8 lists the stretches of the sweep over which the crystal delivered much less than the rest of the run — the feedback a beamline control system needs to tell an operator that a crystal should be recentred or recollected. Nothing is excluded on the strength of it; the frames still carry signal, and this is a message for the beamline, not a filter.

SWEEP_QUALITY_STATUS= COMPUTED
SWEEP_QUALITY_COUNT= 1
SWEEP_QUALITY_REASONS= no_diffraction crystal_out_of_beam weak_diffraction loss_of_centring radiation_damage
SWEEP_ROTATION= 360.0
FLUX_PEAK_TO_TROUGH= 1.03
SCALE_MODULATION_PEAK_TO_TROUGH= 1.00

  FIRST_IMAGE   LAST_IMAGE   N_IMAGES  ROTATION  REASON                SEVERITY   SCALE      CC   INDEXED
  -----------  -----------  ---------  --------  --------------------  --------  ------  ------  --------
          500          600        101      10.1  crystal_out_of_beam       0.83    0.12    0.30      0.02
  -----------  -----------  ---------  --------  --------------------  --------  ------  ------  --------

SWEEP_QUALITY_STATUS distinguishes COMPUTED (the diagnostic ran; a count of 0 means the sweep was clean throughout) from NOT_COMPUTED (it did not run — no scaling and merging, or stills data). A consumer must not read a missing table or a zero count as "clean" without checking it. SWEEP_QUALITY_REASONS lists the whole vocabulary this version can emit, so an unknown code is distinguishable from a missing one.

Reason code	Meaning
no_diffraction	The range recorded essentially no diffraction from the indexed lattice.
crystal_out_of_beam	Frames were lost: over the range a per-image scale could be fitted far less often than over the run.
weak_diffraction	The frames all still index, but with much less intensity — the cause was not determined.
loss_of_centring	One cycle of modulation per revolution: the crystal is off the rotation axis.
radiation_damage	The range runs to the end of a sweep whose quality was already decaying.

The vocabulary is closed and stable: a code is never renamed, and never reused for a different meaning. New codes are only ever added, and adding one moves REPORT_VERSION.

The columns are: FIRST_IMAGE/LAST_IMAGE — inclusive, in processed-image ordinals (the numbering of <prefix>_image.dat and of every other per-image array rugnux writes; with -s/--stride the source image is start + ordinal * stride); ROTATION — the width of the range in degrees; SEVERITY — the fraction of the run's typical diffracting power missing over the range, 0 (as good as the run) to 1 (nothing at all); SCALE and CC — the range's mean per-image scale and CC-to-merge relative to the run median; INDEXED — the fraction of the range's frames that were scaled at all. Every range also appears as a WARNING: sentence in section 9.

The same finding is written per image into the _process.h5 as /entry/MX/sweepQuality, when one is written — see HDF5.

Diffraction anisotropy

Section 9 reports how much the fall-off with resolution depends on direction, and whether that is established above the data set's own systematic error. It runs automatically on every merging run — there is no flag — and it is a description only: no intensity is corrected, no reflection is removed on a directional criterion, and the merged data and the written reflection files do not depend on direction at all. The algorithm is in CPU/GPU data analysis ▸ Diffraction anisotropy.

Two different quantities are reported and they are not interchangeable. ANISOTROPY_DELTA_B is a rate — the range of the principal components of the anisotropy tensor, on the ordinary crystallographic B scale, so it is directly comparable with phenix.xtriage's B_cart, ctruncate's anisotropic B and AIMLESS's anisotropic ΔB. ANISOTROPY_D_MIN_PRINCIPAL is where the signal actually runs out along each principal direction. A crystal can have a large ΔB and almost no spread in directional limit, or the reverse.

key	meaning
ANISOTROPY_VERDICT	DETECTED | NOT_DETECTED | CANNOT_DETERMINE
ANISOTROPY_FREE_DIRECTIONS	Deviatoric directions the Laue class allows — 5 triclinic, 3 monoclinic, 2 orthorhombic, 1 tetragonal/trigonal/hexagonal, 0 cubic
ANISOTROPY_DELTA_B	The anisotropic ΔB (Ų), fitted on intensities with nothing dropped
ANISOTROPY_DELTA_B_LINEAR	The part of it that follows exp(−½ sᵀBs). This is the number the verdict is gated on, and the report says which of the two it is quoting
ANISOTROPY_PRINCIPAL_B	The three principal components, relative to the weakest
ANISOTROPY_D_MIN_PRINCIPAL	Diffraction limit (Å) along each principal direction — where ⟨I/σ(I)⟩ in a 20° cone about it falls through 2
ANISOTROPY_D_MIN_CENSORED	One flag per direction. 1 means ⟨I/σ(I)⟩ never fell through 2, so the limit is the edge of the measured data, a bound and not a measurement. The prose marks it with a <
ANISOTROPY_D_MIN_SPREAD	Range of the three limits — itself a lower bound if any is censored
ANISOTROPY_SHAPE	LINEAR (a real Debye–Waller B) | FLAT (the deficit does not follow a B at all, so ΔB may be an under-estimate) | CONVEX (grows faster than a B can) | UNDETERMINED (the verdict moved on rebinning)
ANISOTROPY_FLOOR, ANISOTROPY_SIGNIFICANCE	The data set's own systematic-error floor (Ų) and ΔBlinear over it. Banded: below 2 not established, 2–3.5 marginal, above 3.5 established, above 5 strong
ANISOTROPY_DETECTION_LIMIT	The smallest ΔB that could have been established on these data. It is set by systematic error, not by counting, so it does not improve with more reflections or a longer exposure
ANISOTROPY_N_OBSERVATIONS, ANISOTROPY_FORBIDDEN_Z, ANISOTROPY_SIGMA_SYSTEMATIC	The unmerged observations the floor was measured on, that measurement against its own counting noise, and the floor before the counting part is added back

CANNOT_DETERMINE is a real answer, not an evasion. The verdict is not measured against counting statistics — real data carry systematic error far larger than that, and gating on counting error reports anisotropy on data sets that have none. Instead the data set measures its own systematic error in the tensor directions its Laue class forbids, where the true value is exactly zero whatever the crystal is. Where that measurement cannot be made, the run says so and gives the reason: a triclinic Laue class (no forbidden direction exists), an observed rotation under about 90°, merged data at the noise floor, a scale model carrying no dose term (--no-scaling-corrections), or no unmerged observations. A cubic Laue class is different again — symmetry forces ΔB to be exactly zero, and the run says that rather than reporting a measurement.

Where anisotropy is detected and the directional limits differ by more than 0.5 Å, a WARNING: line says so, since refinement and map interpretation should allow for it.

Model validation

Section 10 appears only with --model. It reports the supplied model against the merged data — R-factors, maps, anomalous sites — and, separately, whether the data accepted the model at all.

The two are not the same question, and the report keeps them apart. The R-factors, the maps and the rigid-body placement describe the model: they are computed and reported whatever the answer, because a model that does not belong to this crystal still has an R against it, and that is the negative result. MODEL_FIT= is the answer, and it is what governs whether the model was allowed to change anything about the written reflections.

There is no threshold on R behind it. What a model that explains nothing reaches against a given data set depends on its atom count and B-factors as much as on the data, so the same model is refitted — and re-placed as a rigid body, exactly as the real one is — from MODEL_FIT_NULL_REPLICATES random orientations about its own centroid, and MODEL_FIT_SIGMA is how far the real fit sits above that distribution. The statistic is R-work, not R-free - not because nothing is refined against the working set (the placement's six parameters are), but because every null replicate is placed the same way, so what they buy is bought on both sides and cancels; and it is decided on an order of magnitude more reflections than R-free.

That null is only built where the model claims one of the two things it could change — the enantiomorph, or an indexing other than the one the data were merged in. A model already in the data's space group on a crystal with no merohedral ambiguity, which is the isomorphous case a screening campaign is made of, claims neither: MODEL_FIT= NOT_TESTED, MODEL_DECISIONS_TAKEN= NONE, and the run does not pay for a null that would gate nothing. NOT_TESTED is not REJECTED — it says the question was never put, not that the data answered it badly — and the three values are distinguishable by grepping the one key. The MODEL_FIT_NULL_* and MODEL_FIT_SIGMA keys are absent in that case, since there is no null to report; R_WORK, R_FREE, the maps and the rigid-body shift are all there as usual.

key	meaning
MODEL_VALIDATION	PERFORMED | NOT_PERFORMED (with MODEL_VALIDATION_REASON, and no R-factors)
MODEL_FIT	ACCEPTED | REJECTED | NOT_TESTED — whether the model may decide anything, or had nothing to decide
MODEL_FIT_STATISTIC	What the verdict was taken on; R_WORK
MODEL_FIT_VALUE, MODEL_FIT_NULL_MEAN, MODEL_FIT_NULL_SD, MODEL_FIT_NULL_REPLICATES	The real fit, and the null of the same model in random orientations. Absent when NOT_TESTED
MODEL_FIT_SIGMA	The real fit above that null, in its standard deviations. Signed. Absent when NOT_TESTED
MODEL_DECISIONS_TAKEN	NONE | ENANTIOMORPH | INDEXING | ENANTIOMORPH+INDEXING
MODEL_ENANTIOMORPH_ADOPTED, MODEL_INDEXING_OPERATOR	The two decisions individually; x,y,z is no reindexing
MODEL_INDEXING_MARGIN, MODEL_INDEXING_MARGIN_NULL, MODEL_INDEXING_MARGIN_SIGMA	Present only where a merohedral ambiguity was probed. The winner's lead over the runner-up in R-free, against the lead a random placement of the same model produces

MODEL_DECISIONS_TAKEN= NONE — whether the model was rejected or never tested — means the reflection files are byte for byte what a run with no model would have written — same space group, same indexing, same .mtz, .cif, .hkl and _unmerged.mtz. A rejected model is therefore safe to try: it costs the null's compute and changes nothing else.

SPACE_GROUP_ENANTIOMORPH= in section 4 reads ASSUMED_FROM_MODEL when the hand written in the files is the model's. Assumed, not determined: merged intensities cannot see the hand at all — |F| is invariant under the change of hand — so an accepted model asserts it out of prior chemical knowledge. It is only ever written where MODEL_FIT= ACCEPTED, and the anomalous difference map vetoes it outright where the map says the model and the data are in opposite hands. (Before REPORT_VERSION= 6 this value was spelled DETERMINED_FROM_MODEL and was emitted whenever a model file merely parsed.)