1132 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
leonarski_fandClaude Opus 4.8	fd7e2ae6f9	image_preprocessing: treat the type-max marker as invalid for unsigned pixels ... The error/invalid pixel marker (the pixel type's extreme value, e.g. 0xFFFFFFFF for EIGER uint32) was only mapped to the internal error value for SIGNED types - the check was gated on std::is_signed<T>. For unsigned EIGER data a 0xFFFFFFFF pixel therefore fell through to the saturation test and, exceeding the (HDF5) saturation_value, was silently classified as saturated rather than invalid. Both are excluded from integration, so results are unchanged where such pixels are also in the static pixel mask (as in the JUNGFRAU test data - Thau_9 and lyso are bit-for-bit the same), but the classification/statistics were wrong and a stray 0xFFFFFFFF outside the static mask would be mis-accounted. Test the error marker before saturation and drop the is_signed gate in both the CPU and GPU preprocessors: the HDF5 saturation_value stays the saturation threshold, the pixel type's maximum is the invalid marker (priority masked > error > saturated). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-10 14:17:56 +02:00
leonarski_fandClaude Opus 4.8	f0dc61ef58	rugnux: write refined geometry (beam, tilt, axis) to _process.h5 ... The master-file geometry is written from the StartMessage at file open, before indexing/refinement - the file is streamed per image, so it cannot wait for the refined values. rugnux therefore stored the nominal beam centre / detector tilt / rotation axis, not the geometry its own refinement found (jfjoch_broker, by contrast, deliberately keeps the user-provided values). Add optional refined geometry to EndMessage; at finalization the writer overwrites the beam_center_x/y, PONI rot1/2/3 and rotation-axis datasets in place with these. The broker leaves them empty, so its output is unchanged. For EcwtCQ244-E3 the _process.h5 now reports the refined beam 761.6/847.7 (nominal 764/850, matching XDS's 760.2) plus the refined tilt and rotation axis. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-10 14:17:56 +02:00
leonarski_fandClaude Opus 4.8	7d240a120c	rugnux: recover a centred Bravais lattice from intensities (de-novo C2) ... When de-novo rotation indexing demotes a pseudo-symmetric metric to a primitive/triclinic cell, the space-group search never tested the candidate symmetry operators - its operator set is bounded by the indexer's lattice, whose holohedry is trivial for P1 - so a genuinely centred lattice (C-centred monoclinic, EcwtCQ244) was locked to P1. A merely pseudo-symmetric triclinic cell (EcwtCQ066) is metrically indistinguishable from a real one (same 2x C-lattice, same absences), so the decision cannot be made from spot positions: the 2-fold is an intensity property and must be decided after P1 integration, as XDS does in CORRECT. After the P1 space-group search, if the metric is pseudo-symmetric, reindex the merged reflections to the candidate's conventional setting and re-run the search there. The I(h) vs I(Rh) correlation confirms a real centred lattice and rejects a pseudo-symmetric one. The centring itself cannot be read from absences here (integrated in the primitive cell, its centring-absent reflections do not exist), so among the point-group-equivalent candidates the intensities cannot separate, the one matching the metric candidate's centring is chosen. On confirmation the integrated reflections, lattice and cell are reindexed, the rotation merge is re-ingested, and the data are re-merged in the adopted space group. Scaling-only - the indexer is untouched. EcwtCQ244-B1-1 and -E3 -> C2 (2-fold CC 0.99); EcwtCQ066 stays P1 (CC 0.31); the rest of the 24-crystal de-novo battery is bit-identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-10 14:17:56 +02:00
leonarski_fandClaude Opus 4.8	12efbc1beb	rugnux: emit non-standard mmCIF items under a "jfjoch" reserved prefix, not pdbx_ ... ISa and the twinning indicators (L-test means, second moment of I) have no standard mmCIF item. They were written as _reflns.pdbx_* names, borrowing the wwPDB PDBx/mmCIF dictionary's reserved prefix and so masquerading as standard items the dictionary does not actually define. Move them to the IUCr-sanctioned local-data-name extension: a private reserved prefix on the attribute of the existing _reflns category (_reflns.jfjoch_*), matching the DDL2 pattern the spec gives for private items. Write-only tags with no reader, so this is a self-contained, back-compatible rename. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-10 13:01:05 +02:00
leonarski_fandClaude Opus 4.8	09c1a04786	rugnux: measure total processing time incl. scaling, not just the per-image pass ... The reported "Processing time" / "Frame rate" stopped the clock right after the per-image analysis loop, excluding scaling, merging, resolution cutoff, twinning and file writing. Move the timing to the end of Rugnux::Run() so processing_time_s (and the derived frame_rate_hz / throughput_MBs) span the whole run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-10 12:27:50 +02:00
leonarski_fandClaude Opus 4.8	ca5ac9f838	Name the Windows NSIS installer jfjoch-viewer-*, not jfjoch-* ... Match the Linux viewer TGZ naming; update the Gitea upload glob to suit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-10 07:26:10 +02:00
leonarski_fandClaude Opus 4.8	a4cc908ff1	rugnux: run the L-test on rhombohedral data + write twinning stats to CIF ... The Padilla-Yeates L-test paired each reflection with a neighbour two steps along an axis ((h+2,k,l) etc.). A step of 2 preserves the reflection condition of P/I/C/F/A/B lattices but violates R-centring (-h+k+l = 0 mod 3, since 2 != 0 mod 3), so every neighbour was systematically absent -> zero pairs, and the L-test silently did not run on ANY rhombohedral crystal (twinning there was left to the weaker second-moment test alone). Add diagonal (1,1,0)/(1,1,3) fallback steps that preserve the mod-3 condition in both obverse and reverse settings, tried only when the axis steps find no present partner (P/I/C/F behaviour unchanged). On the twinned R3 insulin the L-test now runs: 25k pairs, <L^2> = 0.224 -> twinning detected. Also write the twinning indicators to the output mmCIF next to ISa: _reflns.pdbx_L_test_mean_abs_L / _L_test_mean_L_squared / _second_moment_I, as free-text pdbx values, guarded on pair/reflection count. Threaded the TwinningAnalysisResult into WriteReflections (both the full-pipeline and --scale callers). Note: on rhombohedral lattices the L-test reads ~0.03 lower than on P/I/C/F (its allowed step is slightly more local), so it separates the insulin twin pair only weakly (twinned 0.398 vs untwinned 0.427) and the second moment remains the cleaner discriminator there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-09 21:59:37 +02:00
leonarski_fandClaude Opus 4.8	8c288c8697	rugnux: recover genuine high symmetry in de-novo space-group search ... The POINTLESS-style symmetry search discarded a real point group on two hard rotation-test crystals: - Ins_I_3 (I23): one of the eight cubic 3-fold operators scored CC 0.483, just under the 0.5 per-operator floor, so I23 fell to I222 and merging collapsed (3-fold orientation ambiguity; CC1/2 1%, R-meas 558%). Gate A now scores each CONJUGACY CLASS of a point group's rotations by its mean CC - conjugate operators are symmetry-equivalent and must stand or fall together - so the cubic 3-fold class (mean 0.57) is confirmed. -> I23, CC1/2 99.9%. Degenerate (few-pair / NaN) class members are skipped rather than vetoing the class. - Benas_3/7 (F432): the chi^2 self-consistency gate rejected the true cubic group because the weak-data error model is badly miscalibrated (reduced chi^2 ~150), which inflates the chi^2 ratio with point-group order for genuine high symmetry too. The ratio bound is now widened by log10(chi2_ref) so a broken error model does not veto real symmetry, while a false operator whose ratio is far worse than its subgroup's is still caught. The bound stays a per-candidate chi^2 test - chi^2 still arbitrates and is never bypassed. Verified: F432 (chi2_ref 151, ratio 3.2 < bound 4.2) accepted; a calibrated pseudo/twin R32 (Ins_H_3: chi2_ref 2.6, ratio 4.2 > bound 2.4) still rejected, stays R3. SearchSpaceGroup.{cpp,h} only. Validated on the 24-crystal rotation-test battery: exactly these two space groups change (both now match the reference); zero regressions - every pseudo-symmetry P1 case and every correct assignment unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-09 21:14:18 +02:00
leonarski_fandClaude Opus 4.8	4f3153d6c7	rugnux: automatic CC1/2-logistic high-resolution cutoff ... Merge/report/write to the CC1/2 fall-off instead of the detector edge. After the final merge, fit a logistic to a fine-binned CC1/2 profile and cut one shell past cc_target (default 0.30) - generous, DIALS-style. Only the written MTZ/CIF and the reported shell table are trimmed; scaling, the 3D combine and the error model still run over the full range, and the per-image _process.h5 keeps full data for re-scaling. The de-novo P1 merge that feeds the space-group search is never cut (gated on the final merge). - ScalingSettings: resolution_cutoff {Off,CCHalfLogistic}, resolution_cc_target, report_shell_count (report shell count is now configurable). - New ResolutionCutoff.{h,cpp}: ComputeCCHalfLogisticCutoff (fine-bin CC1/2 + grid/refine logistic fit + one-shell extension + degeneracy/low-res guards). - Wired into RotationScaleMerge (rotation) and MergeOnTheFly/Rugnux (stills), plus the rugnux --scale path. - CLI: --resolution-cutoff cc-logistic|off, --resolution-cc-target, --resolution-shells. --scaling-high-resolution stays the manual override (wins and disables the auto-cut). Validated on the 24-crystal battery: no crashes (incl. Benas/cytC_10/Thau_9); --resolution-cutoff off is bit-identical to baseline; cutoffs reproduce the design table within ~0.05 A (lysoC 1.30, Ins_I 1.43, cytC_2 1.68, EcwtCQ244-E3 1.54); good-to-edge datasets (lyso_ref, EP_cs_02-10) correctly keep the full range. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-09 17:34:49 +02:00
leonarski_f	4df811bf73	Remove rugnux work-in-progress documents from Git	2026-07-09 16:18:08 +02:00
leonarski_fandClaude Fable 5	be937d0163	docs: rugnux automatic resolution-cutoff design (CC1/2 logistic, one shell too far) ... CC1/2 fall-off fit as a logistic in 1/d^2, cut one shell past CC1/2~0.3 - generous for ML, robust to the broken error model (works on 4 A Benas where I/sigma cannot), and avoids defining 'the' resolution. Includes experiments vs XDS and an implementation plan. Not yet implemented. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 15:44:40 +02:00
leonarski_fandClaude Fable 5	c0f6522778	docs: rugnux de-novo symmetry/indexing investigation notes ... Records the C2-missed-as-P1 root cause + the (stashed) RotationOnly recovery fix and why it is a re-orientation step, the Benas F432 point-group/error-model finding, and the CC1/2-logistic resolution-cutoff design. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 15:42:28 +02:00
leonarski_fandClaude Fable 5	ad73b4af99	writer: detect overwrite conflict at start for back-channel transports ... Previously an existing output file was only detected at the final rename, after the whole collection had run. Add an up-front check in the FileWriter constructor that fails before acquisition arms. Only the master file is checked, and only by the single writer that owns it (write_master_file). In a multi-writer TCP/ZMQ setup the per-image data files are staggered across writers by file number, and a writer does not even know the total writer count, so it cannot enumerate its own subset; checking all data files in every writer would make each writer stat files it never writes and race the writers already creating them. Data-file conflicts stay caught by their owning writer at the final rename. Gate the check on whether the transport can report the failure to the broker: the direct HDF5 pusher throws in-process and the TCP writer returns a START-failure ACK, so both abort the collection cleanly (verified: an asymmetric rejection cancels the already-started sibling). The ZeroMQ path is fire-and-forget with no back-channel, so StreamWriter opts out via image_puller.SupportsAck() and keeps writing .tmp files, failing only at the final rename (leaving the images on disk). Documented in docs/JFJOCH_WRITER.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 15:34:37 +02:00
leonarski_fandClaude Fable 5	36be60774f	image_pusher: reset writer when EndDataCollection finalization fails ... HDF5FilePusher::EndDataCollection released the FileWriter only after writer->WriteHDF5()/Finalize(), both of which throw at rename time when a target file already exists and overwrite is off. The throw skipped writer.reset(), leaving 'writer' non-null so the next StartDataCollection died with "Image pusher is already writing images". Tear the writer down on every exit path (try/catch + rethrow) so a failed collection no longer blocks the next one. The error still propagates and the orphaned temp files are intentionally left on disk. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 14:43:42 +02:00
leonarski_fandClaude Fable 5	ec05a6c4d6	Viewer: coral secondary-lattice spots, ring arcs, legend, ROI swatches, save JPEG ... - Colour indexed spots from a secondary (or further) lattice in coral (#FA7268), the viewer's finishing accent, in the diffraction-image overlay. - Draw resolution rings by sweeping ResPhiToPxl into a QPainterPath (like the azimuthal ROI arcs), replacing the 4-point bounding-box ellipse and the poni_rot_3=0 hack, so rings follow the geometry on tilted detectors. - Add a spot/prediction colour legend to the Image-features side panel; the two user-adjustable swatches track the colour picker. - Show a colour swatch matching the on-image annotation on each ROI-list item, via a shared ROIAnnotationColor palette used by both the overlay and the list. - Add "Save image as JPEG..." / "...with overlay as JPEG..." to the image right-click menu, reusing a factored renderToImage() helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 12:04:12 +02:00
leonarski_fandClaude Fable 5	a4559e576d	Preview: geometry-following arcs, true beam center, unified colours, predictions ... Draw the resolution ring and azimuthal-ROI outlines by sweeping ResPhiToPxl through the geometry (a shared DrawArc/DrawThickLine helper), so they trace the true conic on a tilted detector instead of PONI-centred circles. Draw the beam crosshair at GetDirectBeam_pxl() (the real beam/detector intersection) rather than the stored PONI. Unify the spot overlay colours with the viewer: green = not indexed, magenta = indexed (primary lattice), cyan = ice ring, coral = secondary/further lattice. Add Bragg-prediction overlay (dark-red circles, skipping systematic absences) gated on a new show_predictions preview setting, wired through the OpenAPI spec, the image.jpeg HTTP handler, and the frontend toggle (+regenerated TS client). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 12:04:00 +02:00
leonarski_fandClaude Fable 5	03dad9fe93	image_analysis: remove dead RotationParameters; receiver opts out of retain_outcomes ... RotationParameters was a moving-average store (profile_radius, beam_x, beam_y, mosaicity) held by IndexAndRefine but never read or written - none of its four accessors had any call site. Remove the class, its member, the include, and the CMake entries. Also set retain_outcomes=false for the online receiver's IndexAndRefine: online scaling/merge at the end of a run is not enabled yet, so the whole-run integration_outcome is unused there. Revert to true when online scaling lands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 11:28:16 +02:00
leonarski_fandClaude Fable 5	b2f0af12db	image_analysis: add retain_outcomes flag to IndexAndRefine; viewer opts out ... IndexAndRefine::integration_outcome is a whole-run vector (resize(GetImageNum())) holding up to ~0.8 MB of reflections per indexed image. It is consumed only by the offline scaling/merge pass in rugnux (ScaleAllImages / GetIntegrationOutcome). In the viewer's interactive/live analysis it is written but never read - dead storage that, with auto-reanalyze on during live follow, grew ~0.8 MB per frame for the whole run. Add a retain_outcomes constructor flag (default true). When false, skip the whole-run resize and the per-image store; the current image's reflections are still returned via the outgoing message. The viewer's live IndexAndRefine passes false. rugnux (including GUI processing jobs, which build their own Rugnux -> IndexAndRefine) and the online receiver keep the default true, so scaling/merge still has the accumulated data. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 11:02:10 +02:00
leonarski_fandClaude Fable 5	dc8a80f3e0	viewer: fix HTTP live-follow OOM via datasetLoaded backpressure + shared pixel mask ... The viewer could grow to ~100 GB RAM when live-following an HTTP broker. The rc.153 images_in_flight backpressure only throttled imageLoaded; the heavy per-frame payload rides datasetLoaded, fanned out over ~10 queued cross-thread connections with no cap. In HTTPSyncDataset follow mode (entered when an operator clicks an image while following live) RefreshDatasetOnly_i emits a fresh full dataset every autoload tick with no imageLoaded, so the gate never engaged and the queued events - each pinning a full JFJochReaderDataset (full-detector PixelMask + per-image plots) - accumulated without bound. Backpressure datasetLoaded the same way as imageLoaded: a datasets_in_flight counter (cap 2), all emits routed through EmitDatasetLoaded_i, and AutoLoadTimerExpired gated on it (covers HTTPSyncDataset). The window routes the worker's datasetLoaded through a single OnDatasetReady sink that fans out synchronously via datasetReady and acks with datasetConsumed. Under load stale datasets are dropped; the next tick sends the latest. Share the pixel mask instead of deep-copying it: JFJochReaderDataset::pixel_mask is now shared_ptr<const PixelMask>, so per-frame dataset copies share the ~72 MB mask. UpdateUserMask does copy-on-write; JFJochHttpReader caches the mask by arm_date so a live refresh reuses one shared mask per acquisition. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 10:58:10 +02:00
leonarski_fandClaude Fable 5	b4a29ebb32	Update grid scan_result test for dropped placeholder rows ... Commit 023a30cb0 intentionally filters uncollected placeholder rows (number = -1) out of ScanResultGenerator::GetResult() but left this test asserting the old full-grid vector size. The 3x3 grid collects only 7 of 9 cells, so scan_result now has 7 rows; the two uncollected placeholders sit at grid indices 5 and 8, after the checked positions, so the number assertions still hold. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-09 09:44:21 +02:00
leonarski_fandClaude Fable 5	c6af375dc0	CHANGELOG: move branch changes from rc.156 to a new rc.157 section ... The rugnux consolidation bullet had been attributed to rc.156, which is the release this branch is based on. Restore rc.156 to match main and collect all of this branch's changes under a new rc.157 entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 20:43:44 +02:00
leonarski_f	1b5ffb4337	VERSION: 1.0.0-rc.157	2026-07-08 20:29:08 +02:00
leonarski_fandClaude Opus 4.8	023a30cb08	Drop uncollected placeholder rows from scan_result ... ScanResultGenerator pre-sizes its vector to the full expected image count, so GetResult() returned placeholder entries (number = -1, no angle) for images not yet collected. These leaked to the scan_result REST endpoint, where clients saw useless number=-1 rows and absent angle. Filter them out, matching the number<0 skip already used in FillEndMessage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-08 20:25:20 +02:00
leonarski_fandClaude Fable 5	4055e94b7d	Fix inverted per-image compression ratio in Lite receiver ... JFJochReceiverLite computed the per-image compression ratio as compressed/uncompressed (~0.15x) instead of uncompressed/compressed (~7x), disagreeing with the FPGA receiver and the aggregate statistic. Swap the ratio and guard against a zero compressed size. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 15:48:01 +02:00
leonarski_fandClaude Fable 5	51bfdd2b35	Fix acquisition error-handling and rugnux CLI issues from branch review ... Broker acquisition failure paths: - Report a truncated writer output / missing packets as errors ahead of the queue-full warning, so an incomplete dataset is no longer masked as a "reduce frame rate" warning. - Route PCIe/FPGA hardware faults to the Error state: make PCIeDeviceException a JFJochCriticalException, and stop JFJochServices::Stop() from slicing the caught exception (capture/rethrow via exception_ptr) so the critical type survives to the state machine. - Stop the (possibly partially-armed) detector, not just the receiver, when a detector arm fails, so the next start does not see it BUSY. - Return structured Error_message JSON for bind_json operation failures (e.g. a failed synchronous /start), matching the no-arg endpoints. rugnux CLI: - Default -N to all hardware threads, resolved once after parsing so full analysis, --azint-only and --scale behave the same. - Parse numeric option arguments strictly (reject non-numeric input and trailing garbage) instead of atof/stod, which silently yielded 0 or aborted the process; require --wavelength > 0. - Emit -R attached (-R100) in the reproduced command line so it re-parses. - Compare the reference MTZ against the cell that actually drives the merge in the --scale path. - Correct the --scaling-high-resolution help text and de-duplicate the offline-output experiment setup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 15:48:01 +02:00
leonarski_fandClaude Fable 5	77e53ac714	Fix exception handling in acquisition failure paths ... Follow-ups to the Idle-vs-Error acquisition failure split: - JFJochServices::Stop: raise the critical detector fault BEFORE the ordinary receiver/writer exception. Both can be set at once (detector not idle, then receiver->Stop() also throws); throwing the ordinary one first masked the detector fault and returned the broker to Idle instead of Error, skipping the required re-initialisation. - SLSDetectorWrapper::Stop/Deactivate: rethrow a JFJochException as-is (as Start already does) so InternalStop's JFJochCriticalException is not downgraded to an ordinary JFJochException by the generic catch. - JFJochServices::Start: wrap the whole best-effort receiver cleanup (Cancel + Stop) in the catch so a throwing Cancel cannot replace the original detector exception before it is re-raised. Also fix the "cannot create indexing pool" failure seen when retrying initialize after a failed acquisition start: a failed start leaves the receiver (and its GPU resources) alive until the next Start, but the retried initialize rebuilds the indexer pool first, so the fresh GPU indexer had to coexist with the stale receiver and its init failed. JFJochReceiverService::Indexing now releases the previous receiver before (re)building the pool (safe: only runs when the receiver is idle), which also removes a dangling pool pointer held by that receiver. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 14:36:25 +02:00
leonarski_fandClaude Fable 5	01e4f81e44	Consolidate offline tools into rugnux; merge on by default ... Fold jfjoch_azint and rugnux_scale into the single rugnux CLI, and flip scaling/merging from opt-in to on-by-default. - rugnux --azint-only replaces jfjoch_azint (reuses ProcessMode:: AzimuthalIntegration; adds the correction toggles and geometry overrides the old tool carried). - rugnux --scale replaces rugnux_scale (re-scale/merge stored reflections, same simpler scale/merge as before: no space-group search or rot3d defaults). Its workflow is folded into the CLI verbatim. - Merging is now on by default for rotation and stills; --no-merge disables it, replacing -M/--scale-merge. Also fix the batch ReadReflections reader so it works on rugnux's own NXmxIntegrated _process.h5: read per-image reflections/mosaicity/lattice from the master (global index) first, falling back to the source-data locator for legacy/VDS datasets. Without this, both --scale and the former rugnux_scale read zero reflections from an integrated snapshot. RugnuxCommandLine now emits `rugnux --azint-only` and `--no-merge`; docs and tests updated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-08 14:09:15 +02:00
leonarski_fandClaude Opus 4.8	1caa57744a	Rebrand offline processing as rugnux (jfjoch_process -> rugnux) ... Split the naming: rugnux = data-processing subsystem, Jungfraujoch = streaming/acquisition. Executables jfjoch_process -> rugnux (source tools/rugnux_cli.cpp) and jfjoch_scale -> rugnux_scale; the processing library process/ -> rugnux/ with class/target JFJochProcess -> Rugnux (JFJochProcessObserver -> RugnuxObserver, JFJochProcessCommandLine -> RugnuxCommandLine). Doc JFJOCH_PROCESS.md -> RUGNUX.md, reconciled with the live usage message (drop dead -P/--partiality, -w/--wedge; --process-as-stills -> --force-still; add the real rot3d scaling knobs). New docs/NAMING.md explains both names, with pronunciation and a note on Romansh. rugnux now scales and merges rotation data automatically (implicit -M); stills still require an explicit -M. jfjoch_viewer and its classes keep their names (rename deferred); only their references to the renamed library are updated. The _process.h5 output suffix and ProcessConfig/Mode/Result are kept. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-07 22:46:58 +02:00
leonarski_fandClaude Opus 4.8	0a28da7c2e	Acquisition errors return to Idle instead of Error state ... Going to the Error state after any acquisition failure forced an expensive re-initialisation (reconnect + full detector config + pedestals), even when the detector was still perfectly configured and only a transient problem occurred. Split failures into two classes: - ordinary acquisition failures (receiver/writer/analysis problems, missed packets, writer disconnect) return to Idle with an Error-severity message, so the run can be retried without re-initialising; - failures that leave the detector in an undefined state throw the new JFJochCriticalException and still go to Error, forcing re-initialisation. The detector layer is the single source of truth for "detector faulted": SLSDetectorWrapper::InternalStop and DectrisDetectorWrapper::CheckBusyOrError raise JFJochCriticalException on the ERROR state, and JFJochServices::Stop raises it when the detector does not return to idle at the end of a run. MeasurementThread dispatches purely on exception type - no detector-status field checks. Also: - JFJochServices::Start now stops the receiver it launched if the detector fails to start, so the service returns to idle cleanly and the run can be retried. - A synchronous Start rethrows the failure to the caller (HTTP layer) via a stored exception_ptr, so a failed start is no longer reported as HTTP 200. - JFJochServices::Stop no longer throws on a writer error; it is reported through receiver_output.writer_err and surfaced as an Idle + Error-severity message, which makes the previously-dead writer_err branch in MeasurementThread live. Initialisation, calibration/pedestal, SelectDetector and Deactivate keep the Error-state behaviour, as those genuinely leave the system in an undefined state. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-07 14:10:48 +02:00
leonarski_f	d6389e12da	v1.0.0-rc.156 (#66) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_process: Major rotation (rot3d) data processing overhaul - robust profile-fit integration, Cauchy-loss scaling with optional absorption surface, de-novo indexing and space-group/centering determination fixes, and merging statistics + ISa in the mmCIF output. * jfjoch_process: Add EXPERIMENTAL ice-ring detection (--detect-ice-rings) that excludes ice reflections from scaling. * Compression: Add BSHUF_ZSTD_RLE_HUFF, make compression size-aware (drop frames that don't fit rather than aborting), and add the jfjoch_recompress tool. * jfjoch_viewer: Report "Multiple lattices detected" and grey out "Analyze dataset" on a live connection. * jfjoch_broker: Write smargon chi/phi goniometer positions to NXmx; read sensor thickness/material from HDF5 metadata. * CI: Build Windows (CUDA and non-CUDA) installers.Reviewed-on: #66 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.156	2026-07-03 19:18:56 +02:00
leonarski_f	54c667190f	v1.0.0-rc.155 (#65) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_process: Remove pixelrefine option (replaced with ProfileIntegrate2D) * jfjoch_viewer: Some graphical improvements. * jfjoch_viewer: Simplify und unify data analysis settings. * jfjoch_writer: Add TCP keepalive to increase robustness if jfjoch_broker "dies" in the middle of data acquisition. Reviewed-on: #65 1.0.0-rc.155	2026-06-25 22:01:48 +02:00
leonarski_f	6136f858af	v1.0.0-rc.154 (#64) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_broker: Fix to TCP file pusher (remove kernel zero copy to improve reliability) Reviewed-on: #64 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.154	2026-06-25 18:12:00 +02:00
leonarski_f	75e401f0e5	v1.0.0-rc.153 (#63) ... This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use. * jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing * jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files * jfjoch_broker: Add ROI calculation in non-FPGA workflow * jfjoch_broker: Fixes to TCP image pusher * jfjoch_broker: Remove NUMA bindings * jfjoch_broker: Improvements to indexing * jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values * jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors * jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs * jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results Reviewed-on: #63 1.0.0-rc.153	2026-06-23 20:29:49 +02:00
leonarski_f	c49bd2ac3b	v1.0.0-rc.152 (#62) ... * jfjoch_broker: Fix bounds for azimuthal integration for Q spacing (allow Q of 1e-5) * jfjoch_viewer: Adjust Q bounds for azimuthal integration * jfjoch_azint: Add tool to do quick azimuthal integration Reviewed-on: #62 1.0.0-rc.152	2026-06-17 20:36:24 +02:00
leonarski_f	ef52dac2ee	v1.0.0-rc.151 (#61) ... * jfjoch_broker: For PSI EIGER detector allow to disable individual half-modules by putting empty hostname Reviewed-on: #61 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.151	2026-06-16 14:13:29 +02:00
leonarski_f	90e804acd7	v1.0.0-rc.150 (#60) ... * jfjoch_broker: When in FPGA workflow (with PSI detectors) azimuthal integration might be forced to CPU - this will require more computational power, but it enables more integration bins and reports standard deviation of each bin. * jfjoch_broker: Raise error if one is in FPGA flow and there are too many azimuthal integration bins. Reviewed-on: #60 1.0.0-rc.150	2026-06-15 20:24:15 +02:00
leonarski_f	ea575f790a	v1.0.0-rc.149 (#59) ... * XDS plugin: Fix HDF5 mutex to run on multiple processors Reviewed-on: #59 1.0.0-rc.149	2026-06-13 21:27:41 +02:00
leonarski_f	cc3eb8352c	v1.0.0-rc.148 (#58) ... This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144. * jfjoch_broker: Improve azimuthal integration (add <I^2> calculation) * jfjoch_broker: Fixes around indexing, aiming to handle multi-lattice crystals (work in progress, it is not fully integrated) * jfjoch_writer: Save mean(I), stddev(I), and count(I) for each azimuthal bin Reviewed-on: #58 1.0.0-rc.148	2026-06-08 08:30:35 +02:00
leonarski_f	75de40f52b	v1.0.0-rc.147 (#57) ... This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144. * jfjoch_viewer: Add reciprocal space viewer * jfjoch_process: Two pass algorithm that does spot finding/indexing + integration of full dataset * jfjoch_process: Improve logic for rotation indexer, to make execution more deterministic (still work in progress) Reviewed-on: #57 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-06-02 11:49:24 +02:00
leonarski_f	fc68a9baed	v1.0.0-rc.146 (#56) ... This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144. jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset. jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling. jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file. jfjoch_scale: Dedicated tool for rescaling/merging existing data. jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel. WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being. Reviewed-on: #56 1.0.0-rc.146	2026-05-28 18:48:35 +02:00
leonarski_f	75f1c5f954	SHIM library improvements from the HDF Group	2026-05-08 11:39:51 +02:00
leonarski_f	caef26873e	v1.0.0-rc.145 (#55) ... This is an UNSTABLE release. The release has significant modifications for HDF5 writing logic - in case of troubles go back to 1.0.0-rc.144. * **Default HDF5 writing mode is with VDS, not soft-links** - this improves DIALS compatibility and makes format more future-proof, NXmx legacy format might be phased-out in the future. * XDS plugin: Improve performance of VDS reading. * jfjoch_writer: Significant improvement on how file systems I/O are handled through a dedicated pass-through VFD. * jfjoch_writer: Clean-up of HDF5 routines to better handle issues. Reviewed-on: #55 1.0.0-rc.145	2026-05-06 21:50:02 +02:00
leonarski_f	7d34e8a049	v1.0.0-rc.144 (#54) ... This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132. * jfjoch_broker: Improve performance of preview JPEG image generator at receiver startup (saving about 150 ms on measurement start for 16M) Reviewed-on: #54 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.144	2026-05-01 17:06:36 +02:00
leonarski_f	6e0bb971ac	v1.0.0-rc.143 (#53) ... This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132. jfjoch_broker: Avoid copying gain calibration together with DiffractionExperiment Reviewed-on: #53 1.0.0-rc.143	2026-04-30 22:16:50 +02:00
leonarski_f	87fde1b32e	v1.0.0-rc.142 (#52) ... This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132. * Support for newer CUDA architectures (notably Blackwell); minimum CUDA version 12.8 * Minor changes to jfjoch_process, jfjoch_fpga_test and jfjoch_lite_perf_test to make them more consistent Reviewed-on: #52 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-04-30 16:47:53 +02:00
leonarski_f	d760b12a18	v1.0.0-rc.141 (#51) ... This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132. * jfjoch_broker: Azimuthal integration mapping is generated with parallel computations, significantly reducing setup times * frontend: Fix selection of FFTW in indexing settings Reviewed-on: #51 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>	2026-04-30 13:04:54 +02:00
leonarski_f	239a441ee6	v1.0.0-rc.140 (#50) ... This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132. * jfjoch_broker: For DECTRIS detectors, ZeroMQ link is persistent, to save time for establishing new connection * jfjoch_broker: Minor bug fixes for rare conditions Reviewed-on: #50	2026-04-29 21:40:22 +02:00
leonarski_f	4878318c27	v1.0.0-rc.139 (#49) ... This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132. * jfjoch_broker: Further reduce startup time for DECTRIS detectors by selectively modifying SIMPLON parameters on `/start` * jfjoch_broker: Further reduce startup time for DECTRIS detectors by not setting beam center and detector distance via SIMPLON API on '/start' * jfjoch_broker: Add an extra message to ZeroMQ puller ready to monitor Lite worklow preparation time * jfjoch_broker: Image buffer configuration is postponed for Lite receiver flow till start message is received * jfjoch_broker: Use nanoseconds internally for frame/image/readout time * jfjoch_broker: Extra messages added for receiver operation (to be removed after debugging finished) * jfojch_broker: Improve profiling of different data analysis steps * jfjoch_broker: Record integration reflection count * jfjoch_broker: Fix bug where ZeroMQ preview frequency was confusing time units (micro vs. milliseconds) * jfjoch_broker: Fix bug where '/wait_till_done' got deadlocked * jfjoch_writer: Fix confusion between NaN and zero in floating-point datasets **Breaking changes**: detector definition is now using nanoseconds to define minimum frame time, minimum count time and readout time. Reviewed-on: #49 1.0.0-rc.139	2026-04-29 09:50:50 +02:00
leonarski_fandtakaba_k	230480e390	v1.0.0-rc.138 (#48) ... This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132. jfjoch_broker: Cleanup DECTRIS start-up code to enable a shorter start time jfjoch_broker: Allow for asynchronous start to allow overlapping detector configuration with other beamline preparations jfjoch_broker: Goniometer axis name is converted to lowercase jfjoch_broker: Fix bug, where wrong HTTP error codes were returned jfjoch_broker: Improve sigma estimation during merging (K. Takaba) --------- Co-authored-by: takaba_k <kiyofumi.takaba@psi.ch> Reviewed-on: #48 Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch> Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch> 1.0.0-rc.138	2026-04-27 19:56:14 +02:00
leonarski_f	c981e1b91c	v1.0.0-rc.137 (#46) ... This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132. * jfjoch_broker: Better track time for each operation in the processing stack * jfjoch_broker: Rewrite preprocessing of diffraction images in the non-FPGA workflow to better use GPUs (work in progress) * jfjoch_broker: Remove ROI calculation in the non-FPGA workflow (work in progress) * jfjoch_viewer: Toolbar displays image number starting from 1 (instead of 0) Reviewed-on: #46 1.0.0-rc.137	2026-04-25 19:59:21 +02:00