diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 3365fc28..af7c8ec5 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -3,51 +3,52 @@ ### 1.0.0-rc.165 This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. -* `rugnux --model` no longer reindexes the reflections when the data were merged in the enantiomorph of the model's space group; it adopts the model's group as a label and leaves every reflection where it is. The two groups of an enantiomorphic pair have the same rotation operations, so they index and split into Bijvoet hands identically - the label carries no handedness - while the change-of-hand operator is the inversion, so reindexing by it swapped I(+) with I(-) and inverted every anomalous difference. On a tetragonal reference dataset that turned a correct anomalous map (the sulfurs and the bound chlorides at 6 to 16 sigma) into its negative. -* `rugnux --model` warns when the anomalous density at the model's atoms comes out inverted - deeper troughs than peaks - which means the data and the model are in opposite hands, and names the atom that shows it. The reflections are not reindexed to make the two agree: that would hide whether it is the model or the data that is in the wrong hand. R-free cannot see this at all, being blind to the hand. -* `MEAN_ATOM_DENSITY_SIGMA`, the mean 2Fo-Fc density at the model's atom centres that `rugnux --model` reports, is read from the map by cubic rather than linear interpolation and comes out around a tenth higher as a result. The map, the R-factors and every other number are unchanged; only this reading of the map moves, so it is no longer comparable with the figure earlier versions printed. -* `rugnux --model` writes an anomalous difference map (`_anom.ccp4`) when the merge kept the Bijvoet split, and names the ten model atoms it peaks highest on - `ANOMALOUS_SITE_01`..`_10` in the results report, and one line in the log - so a run says which atoms carry the anomalous signal and how many sigma each one is. -* The rugnux results report records how the run was invoked, what it cost and what it ran on: `COMMAND_LINE=` is the command line as one shell-ready line, `WALL_TIME=` the whole invocation in seconds, and `GPU_COUNT=` / `GPU=` how many GPUs were visible and which ones. The total wall time is also printed on stdout, next to the processing time it is slightly larger than. -* The rugnux results report has a `10. MODEL VALIDATION` section when `--model` was given: R-work / R-free with their reflection counts, the bulk-solvent and scale parameters, the mean 2Fo-Fc density at the atom centres, `MODEL_INDEXING_OPERATOR` and `MODEL_ENANTIOMORPH_ADOPTED` for what was applied to the written reflections, and where the maps were written - or `MODEL_VALIDATION= NOT_PERFORMED` and the reason, when the model could not be used. `REPORT_VERSION` is 4; `WARNINGS` moves from section 10 to 11 and no existing key changed. -* `rugnux --model` reads an mmCIF coordinate file as well as a PDB one, gzipped or not, taking the format from the file's content rather than its name. A `.cif` model was rejected before, and the run then finished successfully with no R-free and no maps. -* A model `rugnux --model` cannot use - unreadable, or without atoms, a unit cell or a usable space group - is reported as a `WARNING:` line in the results report instead of only in the log, so a run that was asked to validate and could not is no longer indistinguishable from one that was never asked. -* rugnux says which GPUs it can see before it starts processing, so a machine that turns out to have none - a driver mismatch, a leftover `CUDA_VISIBLE_DEVICES` - is noticed while there is still time to stop rather than after a run that took far longer than it should have. -* rugnux writes `_unmerged.mtz` on every run that produces an output prefix, instead of only when asked for it with `--export-unmerged`. It is written in `--mode mx` and `--mode scale` and with `--no-merge`, alongside the merged files and replacing none of them. `--no-export-unmerged` turns it off. -* `/start` asks the writer whether the run can be written before the detector is armed, so a run whose output file already exists, or whose output directory cannot be created, is refused up front with the writer's own message instead of failing once the detector is running. This needs the TCP image stream or the built-in HDF5 writer; the ZeroMQ stream has no way to answer and is unchanged. -* An output data file already in the way is refused when the collection starts, not when the file is renamed into place at the end of it. -* `/wait_till_done` answers 500 with the message when a collection ended in an error - packets missing, or a writer that could not write - instead of 200. A cancelled collection and a collection that only triggered a warning still answer 200. +* `rugnux --model` adopts the model's space group as a label where the data were merged in its enantiomorph, instead of reindexing the reflections - which swapped I(+) with I(-). +* `rugnux --model` warns, naming the atom, when the anomalous density at the model's atoms comes out inverted, which means the data and the model are in opposite hands. +* `rugnux --model` writes an anomalous difference map (`_anom.ccp4`) when the merge kept the Bijvoet split, and names the ten model atoms it peaks highest on as `ANOMALOUS_SITE_01`..`_10`. +* `MEAN_ATOM_DENSITY_SIGMA` is read from the map by cubic rather than linear interpolation and comes out around a tenth higher; it is no longer comparable with the figure earlier versions printed. +* `rugnux --model` reads an mmCIF coordinate file as well as a PDB one, gzipped or not, taking the format from the file's content rather than its name. +* A model `rugnux --model` cannot use is reported as a `WARNING:` line in the results report instead of only in the log. +* The rugnux results report has a `10. MODEL VALIDATION` section when `--model` was given; `REPORT_VERSION` is 4, `WARNINGS` moves to section 11 and no existing key changed. +* The rugnux results report records how the run was invoked, what it cost and what it ran on: `COMMAND_LINE=`, `WALL_TIME=` and `GPU_COUNT=` / `GPU=`. +* rugnux says which GPUs it can see before it starts processing. +* `rugnux --export-unmerged` also writes `_unmerged.mtz` on a `--no-merge` run, and is ignored on a run with no output prefix instead of writing a file called `_unmerged.mtz`. +* `/start` asks the writer whether the run can be written before the detector is armed, so a run whose master file already exists, or whose output directory cannot be created, is refused up front with the writer's own message. This needs the TCP image stream or the built-in HDF5 writer; the ZeroMQ stream is unchanged. +* A calibration that fails goes to `Error` carrying the reason instead of `Inactive`, so `/wait_till_done` and `/wait_until_running` report it; a cancelled calibration still goes to `Inactive`. +* `/wait_till_done` answers 500 with the message when a collection ended in an error. A cancelled collection and a collection that only triggered a warning still answer 200. +* A pending start failure is discarded by `/cancel` and `/deactivate`, as it already was by `/start` and `/initialize`. +* `/scan_result` no longer reports the previous run's images after a collection that failed to start, or after `/deactivate`. * The TCP image stream protocol version is 4. `jfjoch_writer` and `jfjoch_broker` have to be of the same release, as before. ### 1.0.0-rc.164 This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. -* rugnux now tells you whether a crystal diffracts anisotropically and how far it reaches in each direction, without a second program: a new `9. DIFFRACTION ANISOTROPY` section in `_report.txt` and matching `_reflns.pdbx_aniso_B_tensor_*` / `_reflns.jfjoch_aniso_*` items in the merged mmCIF report the anisotropic deltaB, the diffraction limit along each principal direction, and a `NOT DETECTED` / `DETECTED` / `CANNOT DETERMINE` verdict measured against the data set's own systematic error. It is a description only - no intensity is corrected, no reflection is removed, and the merged data do not depend on direction. -* rugnux can hand its integrated observations to another scaling program: `--export-unmerged` writes `_unmerged.mtz`, an unmerged MTZ readable by aimless, pointless, careless and `iotbx.merging_statistics`, in `--mode mx` and `--mode scale` alike. Each rotation reflection's partials are summed into one full; `--export-unmerged-partials` writes one row per image instead. Intensities carry the Lorentz-polarization factor and nothing else, since those programs scale the data themselves. Lattice-centring absences are not written; screw and glide absences are. -* rugnux integrates crystals with broad spots better - where it changes anything, per-shell mean I/sigma improves by up to 31% and R_meas by up to 24% - because on rotation data the integration signal radius is now taken from the crystal's own measured spot width instead of a fixed 4 px. `--adaptive-integration-radius=off` restores the fixed radius and an explicit `--integration-radius` still overrides both. The widened radius applies to the final integration pass only, and a pattern too dense for it is re-integrated at 4 px with a note in the log. -* rugnux discards fewer stills reflections for want of a background ring, improving per-shell R_meas over most of the signal-bearing range: the stills background ring now runs to 14 px instead of 12. The gain reverses in shells below a mean I/sigma of about 4. -* rugnux determines the space group with thresholds that mean the same thing on a weak crystal as on a strong one: symmetry operators are scored on resolution-normalised intensities (E squared) instead of raw merged intensities, and a reflection counts as genuinely present on its counting significance instead of on the merged I/sigma, which saturates at the merge's own ISa. The search resolution cut is no longer able to move the answer, and the twin-law H bound moves from 1.70 to 1.85, which stops one class of correct high-symmetry assignment being refused as twinning. -* rugnux says what the space-group search tested and what it could not: the twin-law disagreement H is printed for every operator together with the adopted point group's H ratio and its bound; alternatives that are not on the reported lattice are named with how their cell differs; and a lattice centring the data could not test - the crystal having been integrated on the primitive sub-cell, so the reflections it extinguishes were never measured - is marked `UNTESTED` and warned about where it is adopted, as coming from the lattice metric rather than from the intensities. -* rugnux `--mode scale` re-merges a `_process.h5` in the right symmetry without being told it: the file now records the space group on every run - a two-pass rotation run wrote none before, so re-merging defaulted to P1 - together with the change of basis under `/entry/MX/reindexMatrix` where the lattice was re-seated, and `--mode scale` also reports the Wilson B-factor estimate instead of `WILSON_B= nan`. A file written before this stops with a message naming the two cells and the override to use, instead of failing inside the merge. A third-party reader of a `_process.h5` must apply `reindexMatrix` where it is present. -* rugnux installs on its own, as a package called `rugnux` - `dnf install rugnux` or `apt install rugnux` - instead of arriving inside `jfjoch-viewer`. It pulls in none of the acquisition stack, so a machine that only processes data no longer has to carry the broker, the detector libraries or Qt to get it. Installing it over a `jfjoch-viewer` from rc.163 or earlier, which still owns `/usr/bin/rugnux`, upgrades cleanly rather than failing on the duplicate file. -* rugnux is also a standalone download, built for arm64 as well as x86_64: `rugnux--linux-{x86_64|aarch64}-cuda.tgz` and `rugnux--win64-cuda.zip` on the release page, for machines that are not managed by a package manager. The aarch64 build targets GH200 and DGX Spark, and is untested on hardware. -* Every portable Linux binary is now a single self-contained file: cuFFT is linked statically instead of being shipped beside the executable and found through an rpath, so `rugnux` and `jfjoch_viewer` need nothing but an NVIDIA driver, and only to use the GPU. The `.rpm`/`.deb` continue to take cuFFT from the distribution. The developer utilities `jfjoch_extract_hkl` and `jfjoch_recompress` are no longer packaged anywhere. -* Jungfraujoch needs six fewer shared libraries on the machine - libopenblas and libmetis, and libgfortran, libquadmath, libgomp and libz behind them - because the Ceres LAPACK, METIS and SuiteSparse back-ends are no longer built. Nothing in the code ever selected them, and results are unchanged. -* The PCIe driver DKMS package builds for the kernel it is being installed for instead of the running one, so a module built while a kernel update is being applied loads after the reboot. -* The PCIe driver builds on RHEL 9.5 and later, and on their CentOS Stream, Rocky and AlmaLinux equivalents, where the `vm_flags` kernel interface was backported into the 5.14 kernel. -* A data collection started with `async_start` that fails to start - a writer refusing to overwrite an existing file, for instance - is reported as an error by `/wait_until_running` and `/wait_till_done` instead of as a timeout and a successful collection respectively. The error message is the one the writer gave. -* A calibration that is cancelled or that fails to collect its pedestals is no longer reported as a successful one. The broker goes to `Inactive` with an error message and has to be initialized again, instead of sitting in `Idle` looking ready to measure while holding partial pedestals - data collected in that state was silently mis-converted. +* rugnux reports diffraction anisotropy in a `9. DIFFRACTION ANISOTROPY` section of `_report.txt` and in `_reflns.pdbx_aniso_B_tensor_*` / `_reflns.jfjoch_aniso_*` in the merged mmCIF. It is a description only - no intensity is corrected and no reflection is removed. +* `rugnux --export-unmerged` writes `_unmerged.mtz` for aimless, pointless, careless and `iotbx.merging_statistics`, with each rotation reflection's partials summed into one full and the Lorentz-polarization factor applied and nothing else; `--export-unmerged-partials` writes one row per image instead. +* Lattice-centring absences are not written to the unmerged MTZ; screw and glide absences are. +* On rotation data the integration signal radius is taken from the crystal's own measured spot width instead of a fixed 4 px; `--adaptive-integration-radius=off` restores the fixed radius and `--integration-radius` still overrides both. +* The stills background ring runs to 14 px instead of 12. +* The space-group search scores symmetry operators on resolution-normalised intensities (E squared) instead of raw merged intensities, counts a reflection as present on its counting significance instead of on the merged I/sigma, and raises the twin-law H bound from 1.70 to 1.85. +* The space-group search reports the twin-law disagreement H for every operator with the adopted point group's H ratio and bound, names alternatives that are not on the reported lattice, and marks a lattice centring the data could not test as `UNTESTED`. +* `_process.h5` records the space group on every run, and the change of basis under `/entry/MX/reindexMatrix` where the lattice was re-seated, so `--mode scale` re-merges in the right symmetry instead of defaulting to P1. A third-party reader of a `_process.h5` must apply `reindexMatrix` where it is present. +* A `_process.h5` written before that stops with a message naming the two cells and the override to use, instead of failing inside the merge. +* `rugnux --mode scale` reports the Wilson B-factor estimate instead of `WILSON_B= nan`. +* rugnux installs on its own, as a package called `rugnux` - `dnf install rugnux` or `apt install rugnux` - instead of arriving inside `jfjoch-viewer`, and upgrades cleanly over a `jfjoch-viewer` from rc.163 or earlier. +* rugnux is also a standalone download, built for arm64 as well as x86_64: `rugnux--linux-{x86_64|aarch64}-cuda.tgz` and `rugnux--win64-cuda.zip`. The aarch64 build is untested on hardware. +* Every portable Linux binary is a single self-contained file: cuFFT is linked statically instead of being shipped beside the executable. The `.rpm`/`.deb` continue to take cuFFT from the distribution. +* The developer utilities `jfjoch_extract_hkl` and `jfjoch_recompress` are no longer packaged anywhere. +* Jungfraujoch needs six fewer shared libraries on the machine, the Ceres LAPACK, METIS and SuiteSparse back-ends no longer being built. Results are unchanged. +* The PCIe driver DKMS package builds for the kernel it is being installed for instead of the running one. +* The PCIe driver builds on RHEL 9.5 and later, and on their CentOS Stream, Rocky and AlmaLinux equivalents. +* A data collection started with `async_start` that fails to start is reported as an error by `/wait_until_running` and `/wait_till_done`, instead of as a timeout and a successful collection respectively. +* A calibration that is cancelled, or that fails to collect its pedestals, is no longer reported as a successful one. * A failed `/initialize` is reported to `/wait_until_running` and `/wait_till_done` as soon as it happens, instead of when their timeout expires. -* `space_group_number` accepts space groups up to 230 in the API schema, so cubic space groups can be recorded. The broker always accepted them; the generated clients rejected them before the request was sent. -* The results report's `REPORT_VERSION` is 3, two sections having been added. Existing key names and table columns are unchanged. -* The merged statistics table has **9** resolution shells instead of 10, which is what XDS reports. The bins were already XDS's - equal steps in 1/d^2 between the lowest- and the highest-resolution reflection the merge kept - so at the same resolution limits the two tables now have the same shell boundaries and can be read row for row. `--resolution-shells` sets a different count. -* `rugnux --model` now settles the frame the merged reflections are written in, not only the frame the R-factors and the maps are computed in: the `.mtz`/`.cif`/`.hkl` come out in the model's indexing, and where the data were merged in the model's enantiomorph they take the model's hand and space group - which on anomalous data puts I(+) and I(-) the right way round. The indexing choice is logged with the winning R-free and the runner-up, so a decision made within noise is visible. -* `rugnux --model` can resolve the indexing ambiguity of a **serial stills** run, which a model could not do before: structure factors computed from the model become the per-image reference, the same role a reference MTZ plays. It needs the cell and space group up front (`-C` / `-S`). Without one or the other, a merohedral serial run still merges both hands together and says so. -* The rugnux documentation opens with a quick start - the default run, and runs with a reference MTZ, with a model, or with the space group and cell pinned - and explains the indexing ambiguity: what it costs on rotation and on serial data, and which of `-z` / `--model` resolves it in each case. The long reference pages now carry a table of contents. - -### 1.0.0-rc.163 -This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. - -* Packaging: each package installs its license notices under a directory of its own - `share/doc/jfjoch_broker`, `share/doc/jfjoch_writer`, `share/doc/jfjoch_viewer`, `share/doc/jfjoch_driver_dkms` - instead of all of them into the shared `share/doc/jfjoch`, so `jfjoch` and `jfjoch-writer` can be upgraded one at a time instead of only together. +* `space_group_number` accepts space groups up to 230 in the API schema, so cubic space groups can be recorded. +* The results report's `REPORT_VERSION` is 3. Existing key names and table columns are unchanged. +* The merged statistics table has 9 resolution shells instead of 10, which is what XDS reports, so the two tables can be read row for row; `--resolution-shells` sets a different count. +* `rugnux --model` writes the merged `.mtz`/`.cif`/`.hkl` in the model's indexing, and logs the choice with the winning R-free and the runner-up. +* `rugnux --model` can resolve the indexing ambiguity of a serial stills run, given the cell and space group up front (`-C` / `-S`); without one or the other a merohedral serial run still merges both hands together and says so. +* The rugnux documentation opens with a quick start and explains the indexing ambiguity; the long reference pages carry a table of contents. ### 1.0.0-rc.162 This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.