5 Commits

Include renames

Author	SHA1	Message	Date
leonarski_fandClaude Opus 5.5	aa3fa6b9c7	Reader: stream the frame data into the page cache ahead of the image loops ... A cold run on a spinning disk waited on the disk twice over. The CBF header scan read 256 kB from every frame on eight threads that each strode through their own share of the sweep, so they drifted apart and the scan became a seek storm (34 s for 2400 frames here); and after it, the pre-scan and the first-pass indexing touch a few hundred frames and leave the disk idle until the first image loop reads everything at seek-bound rates. - ReadAhead (reader/): once the dataset is open, rugnux starts eight threads that read the data files - HDF5 data files (legacy, VDS or the integrated master) or the per-frame CBF/marCCD/SMV files - in 4 MB pieces taken strictly in order, into a throwaway buffer. One stream reads this disk at 125 MB/s, eight in-order streams at 190 MB/s, 32 at 157 MB/s. It never gets more than a quarter of MemAvailable (GlobalMemoryStatusEx on Windows, 4 GiB where there is no figure) ahead of what ReadRawImage has handed out, so a dataset bigger than the cache does not evict its own start, and it stops with the reader. Plain ifstream reads: portable, no POSIX calls. - Header scans (CBF, marCCD, SMV) hand the files out in order from an atomic counter (sweep::ForEachInOrder) instead of striding: 18 s -> 12 s for 2400 cold CBF headers. The CBF header is first read with a 16 kB probe and again with the old 256 kB one only when the separator is not in it, so the parsed header is exactly what it was: 12 s -> 6 s. Output unchanged: p.hkl, p.mtz and p_unmerged.mtz md5-identical to the rc173 baseline on 6toc (CBF, 2400 frames, 6.0 GB) and 9q41 (HDF5 VDS, 900 frames, 5.1 GB), and on 6z9g (HDF5, 12.8 GB) to the unmodified branch; myob (p.hkl p.mtz p_P1.mtz p_unmerged.mtz) md5-identical to the reference. Measured cold (files evicted with POSIX_FADV_DONTNEED before every run), same code without this commit vs with it, on a shared box (load 20-70, other agents reading the same disk, so single runs scatter by +-20 s): 6toc wall 61.7/62.7 -> 49.3/49.8 s (clean pairs); all data resident after 62/51/50 -> 45/41/42 s 9q41 wall 67.3 -> 57.8 s (clean pair); resident after 58/46/43 -> 48/37/35 s 6z9g resident after 81 -> 69 s The first image loop can look slower with this in CBF runs: the old 256 kB header probes pulled ~70% of the data in as kernel readahead, so the old loop started warm - after a 34 s header scan instead of 10 s. Warm (myob, NVMe, cached): 19.35/20.00 s without, 19.76-20.16 s with; the read-ahead then only copies 9.3 GB out of the page cache, 0.44 s wall and 3.4 CPU-s measured standalone. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1G8gJVAy6gp1K5Dz3NE5C	2026-09-27 10:32:58 +02:00
leonarski_f	30b6800289	v1.0.0-rc.171 (#81) ... * Rugnux: basic support for CCD images (marCCD, SMV) and for gzipped miniCBF. * `jfjoch_viewer`: opens the CCD formats, and fixes to the dataset plots. * Documentation updates. Reviewed-on: #81 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-17 14:42:52 +02:00
leonarski_f	9aae0c2ba7	v1.0.0-rc.169 (#79) ... * Building Jungfraujoch no longer needs zlib or Eigen installed on the machine, and the dependencies the build fetches are pinned and updated to current releases. * rugnux: improvements in indexing, lattice selection and geometry post-refinement, which index crystals that previously returned no lattice and keep the better of the two geometries a run measures. * rugnux: improvements in beam-centre measurement, beam-stop detection and space-group determination. * rugnux: the unit cell reported with a determined space group now obeys that group - a cell whose symmetry was confirmed from the intensities is re-refined under it, and a cell the group cannot describe is reported with a warning rather than as it stands. * rugnux drops the stretches of a rotation sweep whose removal measurably improves the merged intensities and reports what became of every frame, and decides the resolution cut on the crystal's own diffraction rather than on its ice rings. * The rugnux results report is machine-readable - every line that is not `KEY= value` data starts with `#` - and states the build it was written by, its authorship and its terms of use (`REPORT_VERSION= 8`). * `jfjoch_viewer`: improvements in the file manager (CBF frames beside HDF5 datasets, a remembered root), the dataset plots, the inspector and the image statistics, plus a settable font size, a view of the rugnux results report, usable performance over a remote display (`ssh -X`) and a reset of all settings to defaults; the reciprocal-space window is removed. * Broker fixes around DECTRIS collections and dark-mask calibration: re-initialising after a run that never started no longer freezes the broker, a cancelled calibration is abandoned instead of reported as done, and a collection whose start message never arrives ends by itself. Reviewed-on: #79 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-15 17:09:31 +02:00
leonarski_f	77bc0cfe52	v1.0.0-rc.168 (#78) ... * rugnux is substantially faster - a corpus of 145 rotation datasets processes in about two thirds of the time - with identical results. * A crystal whose lattice looks more symmetric than it is because the beam centre is off is no longer processed on the wrong cell. * rugnux prints at startup, and writes at the foot of every results report, a short acknowledgement of the X-ray research community whose methods it implements and of the open-source projects it builds on; `ACKNOWLEDGEMENT.md` now ships in every package beside `LICENSE` and `THIRD_PARTY_NOTICES.md`. Reviewed-on: #78 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-10 13:51:16 +02:00
leonarski_f	680c36c20d	v1.0.0-rc.166 (#76) ... * `rugnux --mode calibration` writes `<prefix>.json` beside the `.poni`, whose `dataset_settings` member is a `jfjoch_broker` `dataset_settings` body as it stands. * `rugnux` and `jfjoch_viewer` read PILATUS miniCBF sweeps natively, without conversion. * Masters written by other facilities open, including Eiger 1.x and third-party NXmx variants. * `rugnux` measures the beam centre on every run, and indexes with it when the file's value indexes nothing. * A detector swung out on a 2theta arm is placed where the file says it stands, and the calibration can hold the tilt fixed. * `rugnux` writes the unmerged MTZ by default, and a P1 merge beside it, so a wrong space group can be re-merged without reprocessing. * Significant improvements to symmetry handling in `rugnux`: the lattice, the point group, the setting and the systematic absences. * The `rugnux` report gives the resolution the CC1/2 fit reached, beside the range the reflections were written to. * The `rugnux` report gives the twinning statistics measured before the space group was decided, beside the ones measured after. * The `rugnux` report gives the strong-direction diffraction limit, and warns when CC1/2 is not monotone with resolution. * `rugnux` ranks screw axes on the evidence their absences carry, rather than on how many control reflections a candidate happens to have. * Twinning is no longer reported when the L-test contradicts it. * The `rugnux` report gives the detector tilt, the measured tilt and the direct beam beside the beam centre, and a post-refined beam centre is judged against the run's own measurement rather than the file's. * `--no-refine-tilt` holds the detector tilt at the value in the file, instead of zeroing it, when the calibration starts from the spots. * The `jfjoch_viewer` grid scan view draws the cells in the proportion of the scan steps, so the map has the shape of the scanned area. Reviewed-on: #76 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-09-02 21:17:31 +02:00