77 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 5 7bb74a9dc8 review fixes: two silent wrong outputs, and one rule for a quantity nobody measured
From four whole-branch code reviews of rc166. Reviewing the net diff rather than
the commits found what per-commit review structurally cannot: a later commit
leaving an earlier one's claim standing, and two cases of a later commit quietly
undoing an earlier one.

THE FITTED RESOLUTION WAS THE P1 CROSS-CHECK'S. The block that merges in P1 to
write <prefix>_P1.mtz saves and restores the error model around itself, because
that merge is not the run's answer. A later commit taught the same function to
report the CC1/2 resolution fit and did not extend the list, so every de-novo
rotation run in a non-P1 group has been quoting FITTED_RESOLUTION - the number
the report itself calls the one to quote - from a merge with n_ops times the
unique reflections at a fraction of the multiplicity.

AND IT ROUND-TRIPPED THE SPACE GROUP THROUGH ITS NUMBER. A number names only the
reference setting, which stopped being enough when the search learned to adopt
P 1 1 2(1) or I 1 2 1. Everything written after that block - the unmerged MTZ
included - therefore carried the reference setting while the merged file carried
the adopted one: two files describing one dataset in two different settings. It
now carries the group.

A PONI CANNOT STATE A MIRRORED OR QUARTER-TURNED DETECTOR, and the fits became
orientation-aware on this branch while the writer did not. It wrote five numbers
that silently described a different geometry from the one measured; it now
refuses, and says the JSON beside it has the full one.

A REFUSED FIT'S ERROR BARS COULD BE HANDED BACK AS AN ACCEPTED GEOMETRY'S.
RingOptimizer::Run writes its uncertainty only where the solve is usable, and
CalibrateFromSpots runs it twice - tilt free, then tilt pinned. A failed second
fit kept the first's sigmas, valid flag and all. It is cleared on the way in.

ONE RULE FOR NOT MEASURED. The report had four conventions for it and printed the
same missing quantity two ways on adjacent lines: SIGANO as the literal "nan" and
CC_ANOM by absence, for a Friedel-merged run that split no Bijvoet pair - which is
the default. A quantity a run did not measure now writes no key, and the shell
table's dash follows the same rule rather than a 0.0% that reads as a measured
total failure. ANISOTROPY_D_MIN_BEST also stops printing nan when only its first
principal direction is unmeasured.

Four claims that a later commit made false are corrected where they stand: the
merge header promising an order-independence the balancing rule gave up, the
reference page arguing against CCanom 28 minutes before it shipped, the screw
threshold whose "three dead reflections clear it" the evidence floor caps at 19.2
nats, and a shell comment calling equal width in 1/d^2 equal volume.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 17:27:56 +02:00
leonarski_fandClaude Opus 5 8a04773d1d api: record the beam size at the sample, and write it where NXmx puts it
dataset_settings gains beam_size_x_um and beam_size_y_um, the horizontal and
vertical size of the X-ray beam where it meets the sample. They follow the same
route total_flux takes - OpenAPI, DatasetSettings, the CBOR start message, the
HDF5 master, and back out of a stored file - and nothing consumes them; this is
metadata a beamline can state and a downstream program can read.

NXmx puts this in the application definition rather than the base class: not
NXbeam's extent (rank 2, nP x 2, per scan point, always FWHM of a rectangular
aperture) but NXmx's own incident_beam_size, a recommended rank-1 two-element
array in the order x, y. Both are live and neither is deprecated, so the choice
matters; the MX definition wins in an MX file. Written as one array with a
units attribute of "m", like every other length in the master, so the settings
hold micrometres and FillMessage converts once.

The unit table of ReadLength_m becomes LengthUnitFactor so the array read can
share it: a master written elsewhere may state this in millimetres.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-09-02 09:59:34 +02:00
leonarski_fandClaude Opus 5 7c3ab72fed reader: refuse a CBF that is not a detector image instead of misreading it
Three ways a miniCBF opened silently wrong.

A byte-offset CBF with no PILATUS header at all was accepted: Count_cutoff then
defaulted to 0, SaturationLimitFromValue(0) is 1, and every pixel at or above one
count was flagged saturated - the integration accept gate drops the whole
reflection, so the run comes out empty for a reason nothing reports. The pixel
size defaulted to 0 with no validation anywhere downstream, which collapses every
resolution, every scattering vector and the beam centre in millimetres. XDS
writes its correction files in exactly this shape, so this is not hypothetical.
A pixel size is now required to claim the file at all, and a missing Count_cutoff
leaves the saturation limit unset - falling back to the container's own overflow,
which can only fail to call a pixel saturated - with a warning saying so.

And the header captures are character classes, not number grammars: "[\d.eE+-]+"
matches a bare "." and "(\d+)" matches a digit string too long for int64.
std::stod and std::stoll answer both with a raw std:: exception, which escaped
the format probe - CanRead catches JFJochException only - so merely LOOKING at a
corrupt file threw out of the viewer's open path. A header field that does not
parse is now reported as a malformed header.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-08-31 07:17:40 +02:00
leonarski_fandClaude Opus 5 26fc4b02b3 symmetry: carry the space group as the group, not as its number
The adopted space group travelled the pipeline as a bare int and was rebuilt
downstream with find_spacegroup_by_number, which returns the reference setting.
So every setting a number cannot name was destroyed one line after it was
determined: P 1 1 2 came back as P 1 2 1, I 1 1 2 as C 1 2 1, R 3:R as R 3:H.

DatasetSettings now holds the gemmi::SpaceGroup itself, DiffractionExperiment
exposes it as GetGemmiSpaceGroup() / GetSpaceGroupOrP1(), and everything that
used to take an int - HKLKeyGenerator (its int constructor is gone, so the
compiler finds the callers), the merge, the R-free flags, French-Wilson, the
reindexing ambiguity, the completeness enumeration, the MTZ and mmCIF exports,
the model validation - takes the group. -S keeps the setting the symbol names
rather than reducing it to a number.

The end message carries both spellings and a reader prefers the name, since
only the name keeps the setting while the number is what a reader written
before the name understands. It carries them over CBOR too: the determined
group was never serialised at all, so a group rugnux chose reached the master
file only when the same process wrote it, and an online writer fell back to
whatever the user had supplied at the start. Both keys are optional additions,
so an older reader skips them and a newer one reads an older sender.

On disk the master's /entry/sample/space_group carries the extended
Hermann-Mauguin name and is what the reader takes the group from, so a setting
survives a _process.h5 and the --mode scale that re-reads it; the number stays
beside it and is the fallback for files written before. Every one of the 230
reference settings the old writer could produce reads back as itself, so older
files are unaffected.

Stage A and Stage B of the search still enumerate reference settings only, so
this determines no group differently today - it is what the enumeration needs
before it can be widened.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
2026-08-31 07:16:43 +02:00
leonarski_f 6ed4ea541e reader: a link to a file that is not there is not a dataset that exists
Every DECTRIS Eiger master links saturation_value, pixel_mask,
bit_depth_readout and serial_number into a companion <prefix>_meta.h5, and
that file is routinely not kept when a dataset is archived or deposited. The
existence test asked only whether the LINK was written, which it is, so every
optional-field guard in the reader answered yes and the read that followed
threw. A deposited Eiger 16M set could not be opened at all, over values the
reader was perfectly prepared to do without.

Exists() now asks the second question too - whether the object the link names
can be reached - so an orphaned link reads as absent and the fallbacks behind
it do their job.

The saturation value is then allowed to be missing outright, because on such a
file it is: neither the NXmx name nor the DECTRIS one is readable, and there is
no third place to look. Left unset, GetSaturationLimit() falls back to the
container's own overflow. That is the safe direction - it can only fail to call
a pixel saturated, where too LOW a value drops the whole reflection and
silently removes the strongest data - and the run says out loud that nothing
will be called saturated.

The set that could not be opened now processes to 2.17 A against a deposited
2.40 A, in the deposited space group, with a cell agreeing to 0.08%. Output is
byte-identical on datasets that already opened.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EFEJG6WBQv8th4UJFNe53N
(cherry picked from commit 0637979f6d95b446406ab70d1f3982841d195b36)
2026-08-30 21:43:54 +02:00
leonarski_fandClaude Opus 5 fb18457e6f docs: bring the changelog and the method notes up to rc.166
The rc.166 changelog was missing fourteen user-visible changes and carried
rationale and measurements that belong here instead. Added: the native miniCBF
sweep reader, the third-party and firmware-1.x NXmx masters, the plain LZ4
filter, the image orientation taken from the file's module direction vectors,
the two beam-centre flags and their rescues, and the FFT reach past 500 A.
Trimmed the rest to one line each, moving the numbers out of the user-facing
file.

RUGNUX.md described the input as a single Jungfraujoch master file, which it
has not been since this branch; it now covers the foreign and legacy masters,
the accepted compression filters and the miniCBF sweep, including how a sweep
is collected from one named frame. Six options existed with no entry in the
table - --beam-center-check, --beam-center-search, --fft-min-unit-cell,
--min-indexed-spots, --rot3 and --no-p1-crosscheck - and -C now moves both FFT
cell bounds, which was not written down anywhere.

CPU_DATA_ANALYSIS.md carried two statements this branch made false: 7.5 still
said pass 2 reuses pass 1's space group, and 13.1 still said centrings are
ranked by net absence count. Both now describe what the code does - the group
is determined after pass 2, and centrings are ranked by the same Beta-tail
likelihood the screw test uses. Also documents the per-zone screw scoring, the
coplanarity volume-fraction guard, the plane-normal transform, the FFT cell
bounds and the twelve refined candidates.

The miniCBF reader implements the x-CBF_BYTE_OFFSET scheme and reads the imgCIF
axis table from the specification alone. No CBF code is vendored or linked, so
there is no licence obligation, but reimplementing a published specification
carries one of credit: ACKNOWLEDGEMENT.md gains a section and the two
algorithms carry a one-line reference each. Both DOIs were resolved before
being written.

Rottger's initial was wrong where this branch first cited it - K, not A.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MxrrPcxodNiXzhNiECCVp5
2026-08-30 21:15:31 +02:00
leonarski_fandClaude Opus 5 a27c4cf26f reader: take a miniCBF's mounting from the imgCIF axis table its header states
A miniCBF header states three things about how the instrument is put together
that the reader was assuming instead: which laboratory direction the image's
columns run along, which its rows run along, and which the spindle turns about.
Some beamlines append a CBF template block holding the full imgCIF axis table,
which says all three outright.

Two instruments in the corpus are not what was assumed, in two different ways.
One mounts its detector a quarter turn round, so the image's columns run
vertically. Another turns its spindle about the VERTICAL, with the image mounted
the usual way; its table says so, and its "# Oscillation_axis" line says so a
second way, by naming the image direction the spindle runs along rather than a
vector. Either error leaves the spindle 90 degrees from the image. That is not a
sign, so the run's axis-sign rescue cannot reach it, and no refinement recovers
it: all three affected sweeps indexed nothing usable.

So the table is read. The element axes give the image orientation, matched against
the eight discrete mountings exactly as the NXmx module directions already are -
the match itself moves to DetectorOrientation, so both readers share one
definition rather than two copies. The goniometer axis with no parent gives the
spindle DIRECTION; its sign stays the rescue's business, which is the part a
convention can legitimately differ on. The detector axis with no parent gives the
2theta arm, replacing the assumption that the arm shares the spindle's axis - the
one header stating both states them with the same vector, so this changes no
answer, only what it rests on. imgCIF's frame differs from the internal one by a
half turn about x, a rotation and not a mirror, as writer/HDF5NXmx.cpp already
records from the other side.

Where a header carries no table, a "+SLOW" on the Oscillation_axis line still
says the spindle runs along the image's slow direction. That is the only thing one
of the three affected sets says about it. The axis NAME on that line stays
unusable - the header that carries both says "X.CW" where its own table says Y -
but the direction token is not: where both are present they agree, which is what
makes reading it evidence rather than a guess.

Also: naming a frame with no directory at all now finds its sweep. parent_path()
of a bare filename is empty and iterating an empty path finds nothing, so running
from inside the data directory reported that no images were found.

Measured, with nothing on the command line. The vertical-spindle protein set goes
from no usable lattice to 100% indexed, P 6(3) 2 2 with a cell 0.43% from
deposited, 87846 reflections at 86.3% completeness and CC(1/2) 0.995. Its
companion from the same detector, which has no table and only the +SLOW token,
goes from a spurious monoclinic cell at 2.3% completeness and I/sigma 0.21 to the
right orthorhombic lattice, 97.7% indexed, 59.7% complete, CC(1/2) 0.996. The
quarter-turned set's three sweeps, at three arm positions, now all index without
the hand-passed quarter turn they needed and agree on one cell to 0.03 A. Six
miniCBF sets that state no table and no +SLOW - including one whose
Oscillation_axis line names an axis in a third dialect - are byte-identical in
.hkl, .mtz, .cif and the image statistics, as are two NXmx sets, which is the
shared orientation matcher moving nothing on that path either.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f
2026-08-30 14:03:32 +02:00
leonarski_fandClaude Opus 5 5c44e544dc reader: place a detector swung out on a 2theta arm where the file says it stands
Chemical crystallography reaches high angle by swinging the detector out on a
2theta arm. Both readers had the number and neither used it: the miniCBF header's
Detector_2theta was parsed into a struct member nothing ever read, and on the NXmx
side the rotation was in the depends_on chain, which was not followed at all. A
sweep taken at 30 degrees was therefore processed with its detector plane 30
degrees from where it stood, and nothing indexed.

The geometry could already express it, and needed no change: the arm turns the
detector about the sample, so the distance is still measured along the detector
normal and the beam centre is still the point of normal incidence - which is
exactly the PONI convention, and a swung detector is one PONI rotation. What moves
is the direct beam, by distance*tan(2theta), off the beam centre and often off the
detector.

NXmx is the harder half, because the swing has no field of its own: it is one
rotation in the chain the detector's position depends on, and "two_theta" is only
one beamline's name for that dataset. So the chain is followed and its rotations
composed, rather than a field of one name being looked for - each transformation
states its vector in the frame of the one it depends on, which is why the product
is the whole placement. Translations are skipped; they are the distance and the
beam centre, which the file states separately in the square-on frame. Vectors come
from McStas through the same 180-degree turn about z the module directions already
use, a proper rotation, so an axis carried through it turns the same way.

The three rotations a file this system writes ARE that chain, and are also read as
the PONI angles - so those three paths are skipped, or every tilted file we have
ever written would come back tilted twice. That is the one way this change could
have broken existing data, and the test for it writes a tilted file and reads it
back.

For miniCBF the arm turns about the base spindle axis: on the four-circle geometry
those headers describe the two are one axis, and the imgCIF axis table such a
header carries states them with the same vector. Both now come from one constant,
so a later correction to the frame moves them together.

Measured. On a swung NXmx sweep the chain gives rot2 = -0.34907 rad for the 20
degrees it states, and the sweep goes from "nothing was integrated" to 25000
reflections at 82.2% completeness and CC(1/2) 0.9993, in the same space group and
the same cell to 0.03 A as the square-on sweep of that crystal; the opposite sign
indexes nothing. A miniCBF sweep at 30 degrees goes the same way, to 0.585 A, and
a second sweep of that crystal at 55 degrees reaches 0.476 A and reproduces the
cell again - with a low-resolution limit of 2.36 A rather than 13 A, which is what
a detector swung that far records. On all of them post-refinement recovers the
header's own beam centre and distance, and the beam stop shadow sits within four
pixels of where the swung geometry puts the direct beam, 417 and 537 pixels from
where the unswung one does. Seven sets whose detector is square to the beam, three
of them carrying a chain whose 2theta is zero, are byte-identical in .hkl, .mtz,
.cif and the image statistics.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f
2026-08-30 12:36:46 +02:00
leonarski_fandClaude Opus 5 c9ca7e424a reader: read a third-party NXmx master, and stop a broken one reading as empty
A valid NXmx master written outside the DECTRIS toolchain could not be opened. Measured on a
Diamond-written master of a 360 deg EIGER 16M sweep, where the images and the meta file are pure
DECTRIS and only the master is third-party - which is why the two sides disagree on units at all.
Five independent things, of which two were silent:

* The image size came from detectorSpecific/x_pixels_in_detector, a DECTRIS extension rather than
  NXmx, so a third-party writer has no reason to emit it. It now comes from the image array's own
  shape, as it already did for a VDS master.
* Lengths were assumed to be metres and the units attribute was never read. A pixel size, sensor
  thickness or distance stated in millimetres - correct NXmx - was silently a factor of a thousand
  out. The unit is now read; an undeclared one still means metres, an unknown one is refused.
* The detector distance can sit in NXinstrument rather than in NXdetector; that is now the last
  fallback after the NXmx and the firmware-1.x spellings.
* A pixel mask that is an external link into a file not holding it passed the Exists() check and
  then threw on the open. Whether the array is there is now decided by opening it.
* Each data file was re-opened and searched for /entry/data/data, ignoring the path the master's
  own link names. A master linking to a plain /data therefore found no images at all - and that
  was a warning and exit code 0 over a sweep sitting right there, not an error. The link is now
  taken at its word, and a master that links to data files but yields no images is an error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3yNBXk4wKdMZy1ak2NY7f
2026-08-30 08:27:53 +02:00
leonarski_fandClaude Opus 5 08f57235d6 reader: take the image orientation from the module directions the NXmx file states
NXmx says how the stored image sits in the detector plane, in the NXdetector_module
fast_pixel_direction and slow_pixel_direction vectors. rugnux read neither, and
assumed every detector was mounted the way this system mounts its own. A facility
that bolts its detector a quarter turn round therefore produced a geometry that
was wrong by 90 degrees, which no amount of refinement recovers: the run indexed
nothing and reported that it could find no lattice.

The two vectors are read, taken from McStas into the internal frame, and matched
against the eight discrete image orientations. An exact match is adopted; anything
else is left alone, because an orientation that is not discrete is a continuous
rotation of the detector in its own plane and cannot be told apart from the tilt by
looking at the module. Every file examined here is exactly discrete.

Measured over the 94-dataset battery: 78 masters state no module vectors at all and
12 state the standard ones, so the change can reach exactly 4. Three of those go
from "Nothing was integrated" to a complete merged result with no flags -
43/93/92% indexed, CC(1/2) 0.957/0.984/0.998 - and are the sets that until now
needed a hand-passed --rot3 pi/2. The fourth states a half turn, which the existing
rotation-axis sign rescue already absorbed, and is unchanged to every digit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW
2026-08-29 23:00:35 +02:00
leonarski_fandClaude Opus 5 fed077e683 geometry: hold the detector plane as axis vectors, and give the discrete part its own home
The detector plane was three PONI angles and nothing else, so the two things it
cannot express - an image mirrored in Y, and one mounted at a multiple of 90
degrees - had no home at all. They are now the DetectorOrientation carried by the
detector setup, composed with the PONI rotation into one orthogonal matrix whose
columns ARE the fast axis, the slow axis and the sample->PONI normal:

    lab = R(rot1, rot2, rot3) * Delta * ( (x-bx)*p , (y-by)*p , distance )

GetFastAxis/GetSlowAxis/GetNormalAxis read those columns and DetectorAxes() sets
the plane from them, decomposing back to the angles; PoniRotMatrix and
PoniAnglesFromMatrix are the conversion in both directions, exact on the canonical
branch (rot2 in [-pi/2, pi/2]) and with a stated convention at gimbal lock. The
angles stay stored rather than re-derived, so a geometry given as angles is
written back as the same angles, to the bit.

Delta is never inferred. In particular an arbitrary rot3 is NOT decomposed into a
quarter turn plus a residual: rot3 is a fitted quantity, and a least-squares step
must not be able to turn the stored image. It is set only where something states
it - the detector setup, --detector-mirror-y / --detector-quarter-turns, or the
value a file this system wrote records - and defaults to the identity, which makes
the whole change a no-op for every existing detector and every existing file.

It is a different setting from DetectorSetup::mirror_y, which flips the MODULE
LAYOUT while an image is assembled and so decides what the stored pixels are.
Merging the two would apply the mirror twice for every modular detector, or change
the pixel content of every file written; both are ruled out. The new one earns its
keep exactly where the old one is a no-op: a detector whose image arrives already
assembled has no layout to flip.

Both generators are signed permutations of the in-plane offset, so they preserve
the distance from the PONI. That is why almost nothing downstream changes:
everything needing an azimuth already goes through LabCoord, and everything that
does not needs only a radius. The two hand-written copies of the rotation -
XtalResidual and RingOptimizer - take the discrete part as four constants next to
cos_rot3/sin_rot3, since it acts in the detector frame where rot3 acts in the
laboratory and cannot be folded into it. RingOptimizer needs it despite being a
radial fit: it fits the tilt, and the discrete part changes which way the tilt
tips a ring.

Carried as two optional CBOR keys and two detectorSpecific datasets, both
back-compatible; the NXmx module axis vectors and the translation direction stop
being hardcoded and are computed from it, reproducing today's values exactly at
the identity. GetPoniRotMatrix is renamed GetDetectorMatrix, because it is no
longer only the PONI rotation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW
2026-08-29 23:00:35 +02:00
leonarski_fandClaude Opus 5 367eba55b1 reader: take a miniCBF rotation axis from the goniometer the header states
Every miniCBF sweep was handed the same hardcoded axis regardless of what its header said, and
Chi/Kappa/Phi/Omega were not parsed at all - there were no members for them. A sweep collected on a
tilted chi cradle therefore ran with an axis that is 54.7 degrees wrong.

The header angles and their increments are now read, the scanned axis is identified from the
non-zero increment (the name is consulted only when no increment is stated, which is what absorbs the
five different spellings the corpus contains, including one file that states no axis name at all),
and a phi scan composes the head chain. An omega scan returns the base axis untouched, because a
fixed chi cannot tilt the axis it hangs from.

-9999 is a sentinel meaning "not set", not an angle. It is treated as absent, so it can never reach
the geometry.

The direction and sense are not invented: these files append an imgCIF _axis loop stating their own
vectors, and SOURCE with GRAVITY fix the imgCIF-to-internal transform, which independently reproduces
the transform this repository already documents for NXmx. Under it the file's own stated phi axis is
exactly the composed one, to four decimals.

Driving the real reader over all 39 corpus sweeps, 37 return the previous axis bit-identically -
including every sweep carrying a large fixed chi, every sentinel header and every axis-name spelling.
Only the two genuine phi scans move, and an unrelated rotation dataset is unchanged end to end.

This is necessary but not sufficient for the one dataset that motivates it: with the axis corrected
it still does not index, because that detector is also mounted rotated 90 degrees in its own plane,
which the reader does not yet read. Compensating both takes its phi sweep from no indexed validation
frames to 90.89% indexed and a complete merge, which is what shows this half is load-bearing. The
detector mount and the two-theta swing belong to the detector-frame work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW
2026-08-29 19:58:50 +02:00
leonarski_fandClaude Opus 5 60c15d6931 reader: open a master whose beamline writes standards-correct NXmx differently
Four defects, hit in sequence, that between them stopped eight masters from one beamline before any
geometry question was reached. The files are correct NeXus; the reader was assuming one writer's
conventions.

ReadScalar demanded rank 0. NXmx puts no rank on distance, saturation_value, two_theta or det_z, and
these files write them as shape (1,). Any dataspace holding exactly one element is now accepted; a
genuine vector is still refused.

frame_time was read unconditionally and NXmx does not require it.

A virtual-dataset source filename of "." was resolved as a relative path, giving <dir>/. - but "."
is HDF5's spelling for THIS file, and these masters compose /entry/data/data as a virtual dataset
over datasets in themselves that are external links to the data files.

The virtual source's DATASET PATH was parsed and then ignored in favour of a hardcoded
/entry/data/data, while these data files keep their images at the root. Fixing that exposed a fifth:
the positional-read fast path used the master's own path, but a dataset reached through an external
link lives in another file and a chunk address is an offset into THAT file.

An audit of all 38 masters in the non-SLS corpus finds exactly these eight need the change and the
other 30 need nothing. Corpus A/B over 73 dataset pairs, base and patched back to back: 64
byte-identical results, 9 identical failures, none differing. The blast radius is bounded by
construction - JFJochReader is linked only by rugnux and jfjoch_viewer, and the one shared header
this touches only ever accepts more, so nothing that opened before can read differently.

With it the eight files read completely: 12850 images, no decode errors, seven of them spanning two
source datasets through the master's own external links.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW
2026-08-29 19:58:33 +02:00
leonarski_fandClaude Opus 5 1ea6ba25be reader: open a master that states its goniometer axes in one place and their angles in another
A hybrid master carries an NXmx /entry/sample/transformations group holding one EMPTY SUBGROUP per
axis - the direction as a vector attribute, no NX_class, no units, no angles - beside a legacy
/entry/sample/goniometer group holding all the angles. Opening it failed outright with "Cannot open
HDF5 dataset /entry/sample/transformations/omega": the existing legacy fallback keys off
Exists("/entry/sample/transformations"), which is true here, so it never fired, and the axis stub
was then opened as if it were the angle dataset.

Present is not the same as usable. GoniometerGroup now takes transformations only if it holds at
least one DATASET, and ReadAxis asks IsDataSet rather than Exists, so a member that is not a dataset
can no longer be read as one. HDF5Object gains that predicate, in the style of the neighbouring
Exists.

The stub is the load-bearing half, not merely the thing that crashed. In the legacy branch, before
falling back to the assumed (-1,0,0), the reader now looks for the NXmx stub and takes its stated
vector. With it the axis is (0,-1,0) and the run indexes 60/60 validation frames; with the stub
deleted the assumption applies and the same file indexes 0/60 on both schemes and both signs, and
the run stops with no lattice. So without this half the fix would have turned "cannot open" into
"found no lattice" - a differently shaped failure, not a success. The direction stated here is 90
degrees from the assumption, not merely its negation, which the rotation first pass could have
recovered on its own.

The vector size check moved out of the first branch so it now covers every path that produces one.

Verified: the file processes end to end, 100% of frames indexed, cubic cell 105.87 against a
deposited 105.88 (0.009%), space group reported as I23 or I213 - correctly refusing to choose, since
the reflections that separate them are extinguished by the I-centring and were never measured. A
both-layout master reprocesses unchanged (100% indexed, P212121). [HDF5] passes: 2194 assertions in
91 cases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lc5JG6kJqZoCWaoZ43JGTW
2026-08-29 15:47:41 +02:00
leonarski_fandClaude Opus 5 fa4aa30a1e reader: match the legacy companion suffixes exactly, not any underscore
The legacy DECTRIS layout stores five companion scalars beside each goniometer
axis - AXIS_end, _start, _increment, _range_average, _range_total - and only the
bare name is the axis itself. The first cut rejected _end and _range, which left
_start and _increment to be offered to ReadAxis as axes in their own right.

Matching the full suffix set, anchored at the end of the name, also keeps a
genuine two_theta axis readable, which a "contains an underscore" test would not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 21:56:12 +02:00
leonarski_fandClaude Opus 5 df3696e6e2 reader: fall back to the pre-NXmx key names, so an Eiger 1.x master opens
Firmware 1.x writes the same three values under different names. Try the NXmx spelling first and
the old one only if it is absent:

    detector/distance         <- detector/detector_distance
    detector/saturation_value <- detectorSpecific/countrate_correction_count_cutoff
    sample/transformations    <- sample/goniometer

This cannot change what a current file reads: every modern Eiger master carries BOTH spellings.
Measured on thirteen masters from ten facilities, firmware release-2020.2.1 through
release-2024.1.1 - all of them write detector_distance and countrate_correction_count_cutoff beside
the NXmx names, and a goniometer group beside the transformations one.

The goniometer is the one that matters. A goniometer is only ever set from that one group, so a
file whose axes are somewhere else was not an error - it was read as STILLS, silently, and the run
completed with the wrong answer. The old layout also tags no axis with transformation_type and
gives no vector, both of which ReadAxis required, so absence now means two different things by
layout: in a transformations group it still means "not an axis" (that is how AXISNAME_end and the
width scalars are skipped), while in the legacy group every leaf IS an axis and the companions are
recognised by name instead. A missing direction defaults to the one every DECTRIS master since has
written and says so in a warning rather than assuming it silently; a wrong guess there does not
index, so it is visible, and the rotation first pass tries the opposite sign anyway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 21:56:12 +02:00
leonarski_fandClaude Opus 5 dc16a00271 reader: read a PILATUS miniCBF sweep natively, without libcbf
Most facilities still archive rotation data as a directory of miniCBF frames, which until now had
to be converted to HDF5 before rugnux could see it. Nothing in that format needs a CIF parser or a
library: it is an ASCII header, four separator bytes, then one byte-offset compressed image, and
every value the reader wants sits on a "# " comment line or a MIME line.

MiniCBF holds the format itself - header parse and the byte-offset decoder, which is a running
value with deltas stored smallest-container-first. Verified byte-exact against dxtbx on PILATUS 6M,
6M-F, 300K, silicon and CdTe sensors, and three sensor thicknesses.

JFJochCBFReader is a sibling of JFJochHDF5Reader under the JFJochReader base. NAMING ANY FRAME
READS ITS WHOLE SWEEP: the sweep is identified by the template (prefix + digit count) the named
frame belongs to, not by "every .cbf in the directory", so a directory holding two sweeps does not
splice two crystals together. Naming a directory takes the sweep with the most frames in it.

Images decode on demand, one per call, so any number of workers can read at once - there is no
global lock as there is on the HDF5 path, HDF5 not being thread-safe. A raw CBF carries no analysis
results, so the dataset it builds is the geometry, the mask and nothing else, exactly as a plain
DECTRIS file with no /entry/MX gives.

Two header quirks are handled because real files have them: the sensor material is written
"Silicon" where the rest of the code compares against "CdTe", and the thickness unit is sometimes
omitted. Headers are not a fixed size either - one set carries 6335 bytes - so the parse runs to
the binary separator rather than over a fixed prefix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 20:12:36 +02:00
leonarski_f 749db470ca v1.0.0-rc.164 (#74)
Build Packages / build:rpm (rocky9) (push) Successful in 19m56s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m57s
Build Packages / build:windows:cuda (push) Successful in 19m18s
Build Packages / build:viewer-tgz:cpu (push) Successful in 14m48s
Build Packages / build:viewer-tgz:cuda (push) Successful in 16m18s
Build Packages / build:rugnux-tgz (x86_64) (push) Successful in 14m19s
Build Packages / build:rugnux:windows (push) Successful in 10m34s
Build Packages / build:rugnux:aarch64 (cross) (push) Successful in 8m49s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 20m55s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 17m4s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 20m48s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 19m15s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 24m26s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 20m32s
Build Packages / build:rpm (rocky8) (push) Successful in 23m39s
Build Packages / Generate python client (push) Successful in 46s
Build Packages / Build documentation (push) Successful in 1m45s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 11m3s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m30s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 20m10s
Build Packages / XDS test (neggia plugin) (push) Successful in 10m17s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 23m12s
Build Packages / DIALS test (push) Successful in 20m12s
* rugnux now tells you whether a crystal diffracts anisotropically and how far it reaches in each direction, without a second program: a new `9. DIFFRACTION ANISOTROPY` section in `<prefix>_report.txt` and matching `_reflns.pdbx_aniso_B_tensor_*` / `_reflns.jfjoch_aniso_*` items in the merged mmCIF report the anisotropic deltaB, the diffraction limit along each principal direction, and a `NOT DETECTED` / `DETECTED` / `CANNOT DETERMINE` verdict measured against the data set's own systematic error. It is a description only - no intensity is corrected, no reflection is removed, and the merged data do not depend on direction.
* rugnux can hand its integrated observations to another scaling program: `--export-unmerged` writes `<prefix>_unmerged.mtz`, an unmerged MTZ readable by aimless, pointless, careless and `iotbx.merging_statistics`, in `--mode mx` and `--mode scale` alike. Each rotation reflection's partials are summed into one full; `--export-unmerged-partials` writes one row per image instead. Intensities carry the Lorentz-polarization factor and nothing else, since those programs scale the data themselves. Lattice-centring absences are not written; screw and glide absences are.
* rugnux integrates crystals with broad spots better - where it changes anything, per-shell mean I/sigma improves by up to 31% and R_meas by up to 24% - because on rotation data the integration signal radius is now taken from the crystal's own measured spot width instead of a fixed 4 px. `--adaptive-integration-radius=off` restores the fixed radius and an explicit `--integration-radius` still overrides both. The widened radius applies to the final integration pass only, and a pattern too dense for it is re-integrated at 4 px with a note in the log.
* rugnux discards fewer stills reflections for want of a background ring, improving per-shell R_meas over most of the signal-bearing range: the stills background ring now runs to 14 px instead of 12. The gain reverses in shells below a mean I/sigma of about 4.
* rugnux determines the space group with thresholds that mean the same thing on a weak crystal as on a strong one: symmetry operators are scored on resolution-normalised intensities (E squared) instead of raw merged intensities, and a reflection counts as genuinely present on its counting significance instead of on the merged I/sigma, which saturates at the merge's own ISa. The search resolution cut is no longer able to move the answer, and the twin-law H bound moves from 1.70 to 1.85, which stops one class of correct high-symmetry assignment being refused as twinning.
* rugnux says what the space-group search tested and what it could not: the twin-law disagreement H is printed for every operator together with the adopted point group's H ratio and its bound; alternatives that are not on the reported lattice are named with how their cell differs; and a lattice centring the data could not test - the crystal having been integrated on the primitive sub-cell, so the reflections it extinguishes were never measured - is marked `UNTESTED` and warned about where it is adopted, as coming from the lattice metric rather than from the intensities.
* rugnux `--mode scale` re-merges a `_process.h5` in the right symmetry without being told it: the file now records the space group on every run - a two-pass rotation run wrote none before, so re-merging defaulted to P1 - together with the change of basis under `/entry/MX/reindexMatrix` where the lattice was re-seated, and `--mode scale` also reports the Wilson B-factor estimate instead of `WILSON_B= nan`. A file written before this stops with a message naming the two cells and the override to use, instead of failing inside the merge. A third-party reader of a `_process.h5` must apply `reindexMatrix` where it is present.
* rugnux installs on its own, as a package called `rugnux` - `dnf install rugnux` or `apt install rugnux` - instead of arriving inside `jfjoch-viewer`. It pulls in none of the acquisition stack, so a machine that only processes data no longer has to carry the broker, the detector libraries or Qt to get it. Installing it over a `jfjoch-viewer` from rc.163 or earlier, which still owns `/usr/bin/rugnux`, upgrades cleanly rather than failing on the duplicate file.
* rugnux is also a standalone download, built for arm64 as well as x86_64: `rugnux-<version>-linux-{x86_64|aarch64}-cuda<major>.tgz` and `rugnux-<version>-win64-cuda<major>.zip` on the release page, for machines that are not managed by a package manager. The aarch64 build targets GH200 and DGX Spark, and is untested on hardware.
* Every portable Linux binary is now a single self-contained file: cuFFT is linked statically instead of being shipped beside the executable and found through an rpath, so `rugnux` and `jfjoch_viewer` need nothing but an NVIDIA driver, and only to use the GPU. The `.rpm`/`.deb` continue to take cuFFT from the distribution. The developer utilities `jfjoch_extract_hkl` and `jfjoch_recompress` are no longer packaged anywhere.
* Jungfraujoch needs six fewer shared libraries on the machine - libopenblas and libmetis, and libgfortran, libquadmath, libgomp and libz behind them - because the Ceres LAPACK, METIS and SuiteSparse back-ends are no longer built. Nothing in the code ever selected them, and results are unchanged.
* The PCIe driver DKMS package builds for the kernel it is being installed for instead of the running one, so a module built while a kernel update is being applied loads after the reboot.
* The PCIe driver builds on RHEL 9.5 and later, and on their CentOS Stream, Rocky and AlmaLinux equivalents, where the `vm_flags` kernel interface was backported into the 5.14 kernel.
* A data collection started with `async_start` that fails to start - a writer refusing to overwrite an existing file, for instance - is reported as an error by `/wait_until_running` and `/wait_till_done` instead of as a timeout and a successful collection respectively. The error message is the one the writer gave.
* A calibration that is cancelled or that fails to collect its pedestals is no longer reported as a successful one. The broker goes to `Inactive` with an error message and has to be initialized again, instead of sitting in `Idle` looking ready to measure while holding partial pedestals - data collected in that state was silently mis-converted.
* A failed `/initialize` is reported to `/wait_until_running` and `/wait_till_done` as soon as it happens, instead of when their timeout expires.
* `space_group_number` accepts space groups up to 230 in the API schema, so cubic space groups can be recorded. The broker always accepted them; the generated clients rejected them before the request was sent.
* The results report's `REPORT_VERSION` is 3, two sections having been added. Existing key names and table columns are unchanged.
* The merged statistics table has **9** resolution shells instead of 10, which is what XDS reports. The bins were already XDS's - equal steps in 1/d^2 between the lowest- and the highest-resolution reflection the merge kept - so at the same resolution limits the two tables now have the same shell boundaries and can be read row for row. `--resolution-shells` sets a different count.
* `rugnux --model` now settles the frame the merged reflections are written in, not only the frame the R-factors and the maps are computed in: the `.mtz`/`.cif`/`.hkl` come out in the model's indexing, and where the data were merged in the model's enantiomorph they take the model's hand and space group - which on anomalous data puts I(+) and I(-) the right way round. The indexing choice is logged with the winning R-free and the runner-up, so a decision made within noise is visible.
* `rugnux --model` can resolve the indexing ambiguity of a **serial stills** run, which a model could not do before: structure factors computed from the model become the per-image reference, the same role a reference MTZ plays. It needs the cell and space group up front (`-C` / `-S`). Without one or the other, a merohedral serial run still merges both hands together and says so.
* The rugnux documentation opens with a quick start - the default run, and runs with a reference MTZ, with a model, or with the space group and cell pinned - and explains the indexing ambiguity: what it costs on rotation and on serial data, and which of `-z` / `--model` resolves it in each case. The long reference pages now carry a table of contents.

Reviewed-on: #74
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-26 22:47:00 +02:00
leonarski_fandjungfrau 4dc2534dbf v1.0.0.rc-162 (#72)
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 18m57s
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 16m55s
Build Packages / build:windows:cuda (push) Successful in 18m48s
Build Packages / build:viewer-tgz:cpu (push) Successful in 13m10s
Build Packages / build:viewer-tgz:cuda (push) Successful in 14m45s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 22m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m12s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 23m7s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 20m43s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 23m9s
Build Packages / XDS test (durin plugin) (push) Successful in 12m26s
Build Packages / build:rpm (rocky9) (push) Successful in 24m58s
Build Packages / Generate python client (push) Successful in 50s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 23m20s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8) (push) Successful in 27m58s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 25m38s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / DIALS test (push) Successful in 23m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m38s
**Files written by Jungfraujoch now import correctly in DIALS, XDS and pyFAI.** A tilted detector, a grid scan, a still recorded at a goniometer position, and saturated or unreadable pixels were each described in a way that a third-party program acted on wrongly. If you process Jungfraujoch data outside Jungfraujoch, prefer this release to any earlier one.

* HDF5: the detector tilt (`rot1`/`rot2`/`rot3`) is exported correctly in the NXmx transformation chain; untilted geometries are unaffected.
* HDF5: a still recorded at a goniometer position is no longer read back as a single image, and a grid scan records a stationary spindle so a program that requires a rotation axis can open it.
* HDF5: the sample transformation chain is written in mounting order, with a Smargon head position told apart from the spindle, one entry per image, `module_offset` as a float unit vector, and `offset_units` on every offset.
* HDF5: saturated, underloaded and unreadable pixels are described so a downstream program masks them - `saturation_value`, `underload_value`, `error_value` and `bit_depth_readout` are written correctly, and a data file missing next to a VDS master reads as the error marker rather than as zero counts.
* HDF5: the rotation axis is read back under whatever name it carries, and `mirror_y` records whether the assembled image is mirrored in Y relative to the detector's raw readout.
* A grid scan and a goniometer axis can both be set; they are no longer alternatives.
* `images_per_file` is chosen from the acquisition when it is not given: a rotation sweep of at most 20000 images goes into a single data file, a grid scan splits on whole fast-axis rows, and stills and serial keep 1000.
* The writer refuses a stream whose start message declares a different pixel format than its images carry, and a DECTRIS detector sending signed images is no longer declared unsigned.
* The image stream can carry the sample transformation chain (`transformations`, in the END message); a producer that does not send it gets the same chain built by the writer.
* rugnux: fixing the space group with `-S` no longer prevents the lattice from being found - a lattice indexed in a different setting is reindexed into that group's own setting, and a run whose crystal does not have that group's lattice stops and names the cell it indexed as, rather than reporting statistics that cannot describe it.
* rugnux: the per-image resolution estimate now predicts the resolution the merged data reach rather than the highest-resolution spot found, and is reported as `SPOT_RESOLUTION_ESTIMATE`.
* rugnux: two runs of the same command on the same images produce the same merged intensities; the azimuthal profile written alongside them is not yet reproducible in the same way.
* rugnux: the offline lattice refinement is bounded by iterations rather than by a wall clock, so a loaded machine can no longer refine to a different lattice; a live acquisition keeps its real-time bound.
* rugnux: the detector-frame modulation correction is fitted on a grid spanning the detector, so whether it is applied no longer depends on how far integration reached.
* rugnux: the geometry pre-pass no longer writes `<prefix>_01.mtz`, `_01.cif`, `_01.hkl` and `_01_image.dat`; the refined second pass writes those files under `<prefix>`, and that is the result to use.
* rugnux: `_process.h5` describes the pixel format of the images it links to, and is written on a thread of its own.
* rugnux: the detector geometry is also logged in XDS's convention (`ORGX`/`ORGY`, detector axis vectors, rotation axis), so it can be compared with an XDS refinement.
* rugnux: an image integrated in pyFAI through the `.poni` file written by `--mode calibration` comes out with the correct azimuth, and the file declares pyFAI's `orientation`, which needs pyFAI 2024.01 or newer. Radial integration is unchanged.
* rugnux: a rotation run is substantially faster throughout - beam-stop detection, first-pass indexing, geometry refinement, integration, scaling and merging - and observations outside the scaling resolution range are dropped as they are ingested. The refined geometry, the space group chosen and the merged statistics are unchanged.
* Faster spot finding and indexing, on the broker as well as in rugnux; the spots found and the lattices indexed are unchanged.
* A run reserves substantially less GPU memory: nothing is allocated for buffers that are never read, and a worker builds only the engines it uses.
* rugnux: with `-N` left at its default the per-image loop of `--mode mx` uses at most 16 workers per GPU, rather than one per hardware thread; an explicit `-N` is obeyed as given.
* CUDA 12 builds now contain device code for Volta, so the RHEL 8 packages and the portable Linux `.tgz` run on a V100; the CUDA 13 artefacts (RHEL 9, Ubuntu, Windows) remain Turing and newer.
* The build resolves a single Eigen for the whole project, and refuses to configure if Ceres picks up a different one; a build that mixed two Eigen versions was undefined behaviour and crashed at -O2.
* Documentation: a security page, and the supported GPU generations and minimum NVIDIA driver version of every released artefact.

**Breaking change to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.162, `frontend/src/client`):
* `dataset_settings.images_per_file` is no longer `default: 1000` and no longer accepts `0`; it is optional, and its minimum is 1. A client sending `0` (previously "one file for the whole run") is now rejected - omit the field instead, which for a rotation sweep gives the same single file.
* `file_writer_format` now defaults to `NXmxVDS`, matching the server's own default and the layout recommended for DIALS, XDS and CrystFEL. A generated client that fills in schema defaults and does not set the format explicitly will write VDS masters where it previously wrote legacy ones; set `NXmxLegacy` explicitly to keep them.

---------

Co-authored-by: jungfrau <jungfrau@mx-aare-test.psi.ch>
Reviewed-on: #72
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-25 08:21:39 +02:00
leonarski_f 538f3504d3 v1.0.0.rc-161 (#71)
Build Packages / build:windows:nocuda (push) Successful in 20m4s
Build Packages / Unit tests (push) Skipped
Build Packages / build:viewer-tgz:cpu (push) Successful in 16m5s
Build Packages / build:viewer-tgz:cuda (push) Successful in 17m26s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 27m46s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 20m17s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 26m13s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 23m17s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 28m11s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 19m30s
Build Packages / build:rpm (rocky8) (push) Successful in 24m34s
Build Packages / build:rpm (rocky9) (push) Successful in 21m30s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 23m33s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 20m18s
Build Packages / DIALS test (push) Successful in 18m23s
Build Packages / XDS test (durin plugin) (push) Successful in 11m30s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 10m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m2s
Build Packages / Generate python client (push) Successful in 49s
Build Packages / Build documentation (push) Successful in 1m21s
Build Packages / Create release (push) Skipped
Build Packages / build:windows:cuda (push) Successful in 29m45s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* **rugnux: significantly better quality of results, and faster.** A large rework of integration, scaling, merging, geometry refinement and space-group determination, together with measurements the program previously made no attempt at - the direct beam before indexing, the beam stop, the goniometer rotation scale, and the stretches of a sweep the crystal did not deliver. A rotation dataset typically gains observations at better <I/sigma> and R_meas, and every `mx` and `scale` run writes a `<prefix>_report.txt` results report modelled on XDS's `CORRECT.LP`. Many defaults moved with it: spot detection is self-calibrating, beam-stop detection and rotation geometry post-refinement are on, resolution limits default to as far as the detector reaches, and ice-ring handling engages only where the crystal is measured to have ice.
* **jfjoch_viewer:** the beam-stop shadow, the detector calibration and the beam-centre measurement are reachable from "Analyze dataset"; the settings panel reports how the sample moved and how polarized the beam was; image rendering and interaction are faster.
* **Performance:** bitshuffle+LZ4 images are decoded on the GPU rather than on the host, with the bitshuffle inverse fused into preprocessing so the decompressed frame is never held in device memory.
* **Broker, writer, packaging and build:** image-slot lifetime and locking fixes, per-image datasets sized by the images actually written, the Debian/Ubuntu broker package renamed to `jfjoch`, and `image_analysis` compiling under MSVC again.

**Breaking change to the rugnux command line:**
* `--azint-only` and `--scale` are **removed**, replaced by `--mode azint` and `--mode scale`; the full pipeline is `--mode mx` and remains the default. A script passing the old flags now fails with the list of valid modes rather than silently running the wrong one.
* `-t`/`--stride` is **refused on rotation data**: skipping frames cuts every reflection's rocking curve, so the combined fulls and their partiality would be measured over frames the sweep never recorded. Select a contiguous range with `-s`/`-e` instead. `--mode azint` and `--force-still` still take a stride.

**Breaking changes to OpenAPI** - regenerate the client (`jfjoch-client` 1.0.0-rc.161, `frontend/src/client`) or read the affected fields as optional:
* `image_scale_b` is removed from the `plot_type` enum, so a client requesting that plot now gets an error rather than a curve.
* `azim_int_settings.high_q_recipA`, `spot_finding_settings.high_resolution_limit` and `spot_finding_settings.low_resolution_limit` are no longer `required`. All three mean "no limit at that end" when unset and are omitted from the response instead of carrying a placeholder value, which raises in a client generated from an rc.160-or-earlier spec. A value of 0 is still accepted and means the same thing.

**Breaking changes to the stored formats** - a consumer reading these fields must treat them as optional:
* The per-image image-scale B factor is no longer computed, so `/entry/MX/imageScaleBFactor` is absent from newly written HDF5 files and the corresponding key is absent from the CBOR DataMessage and END blocks. Files written by rc.160 and earlier still contain it and still open; nothing in the pipeline reads it any more.
* `_reflns.jfjoch_diffrn_ISa` now carries the whole-range `1/sqrt(a*b)` that XDS's ISa denotes, and the error-model `a` and `b` are reported in XDS's convention; the strong-reflection asymptote moves to `_reflns.jfjoch_diffrn_ISa_asymptotic`. **A file written by an earlier version carries the asymptote under the plain `ISa` name.**

Reviewed-on: #71
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-08-13 17:03:10 +02:00
leonarski_f 67dca388bd v1.0.0-rc.160 (#70)
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:cuda (push) Successful in 18m44s
Build Packages / build:viewer-tgz:cpu (push) Successful in 6m11s
Build Packages / build:viewer-tgz:cuda (push) Successful in 6m54s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m40s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m41s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m10s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m4s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m5s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m23s
Build Packages / build:rpm (rocky8) (push) Successful in 11m30s
Build Packages / build:rpm (rocky9) (push) Successful in 12m51s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m8s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m21s
Build Packages / DIALS test (push) Successful in 13m22s
Build Packages / XDS test (durin plugin) (push) Successful in 9m2s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m55s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m57s
Build Packages / Generate python client (push) Successful in 23s
Build Packages / Build documentation (push) Successful in 57s
Build Packages / Create release (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 10m24s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* rugnux: Add `--model model.pdb` - score the merged data against an atomic model and compute initial maps. It reports R-work/R-free (scaling the model to the observed amplitudes with an overall scale, an anisotropic B and a flat bulk solvent - the standard few-parameter model, so a batch of maps stays directly comparable) and writes 2Fo-Fc / Fo-Fc electron-density maps (CCP4) plus a map-coefficient MTZ. The structure itself is not refined; the model is only re-fractionalised into the data cell.
* rugnux: The merged reflection output now carries French-Wilson amplitudes (|F| and its sigma) next to the intensities - MTZ `F`/`SIGF`, mmCIF `_refln.F_meas_au`, and the text HKL - computed with the correct centric/acentric Wilson prior and epsilon multiplicity, so a downstream program (e.g. phenix.refine) can refine against amplitudes. The intensity columns are unchanged.
* rugnux: R-free test-set flags are now assigned deterministically and consistently across symmetry - a Bijvoet pair I(+)/I(-) is never split between the work and free sets, and the assignment is a reproducible per-hkl hash that depends only on the reflection index, so every dataset of one crystal form gets the same ~5% free set (what a multi-dataset campaign such as PanDDA needs). On small data the fraction is floored so the test set stays large enough for a stable R-free (~500 reflections, capped at 10%); it stays flat at 5% on ordinary data. When a reference MTZ carries a `FreeR_flag` column its test set is imported instead, letting a whole campaign inherit one shared free set.
* rugnux: A reference MTZ (`--reference-mtz`) can now fix the space group and cell for rotation data too (previously rejected), without being used to scale - the rotation merge stays self-consistent. When the crystal has an indexing (merohedral) ambiguity - a lattice symmetry higher than its Laue symmetry, e.g. P3/P4/P6/C2 - the reference also resolves it: each candidate reindexing (identity plus the twin-law cosets of the metric symmetry) is scored by its intensity correlation against the reference and the data are re-merged in the best-correlating one. This is a metric-preserving relabelling of hkl (the cell is unchanged) and a no-op for a holohedral crystal such as lysozyme.
* rugnux: `--model` validation now aligns the data to the model before scoring - the observed reflections are reindexed into the model's enantiomorph when the two differ only by hand (indistinguishable from merged intensities). A merohedral indexing ambiguity is resolved against the reference MTZ when one is given (so a whole campaign shares one indexing convention); only with a model and no reference does validation fall back to fitting each candidate reindexing and keeping the lowest R-free.
* rugnux: De-novo symmetry - recover a genuine high-symmetry group whose data are imperfectly scaled. Such a merge's within-orbit chi² lands just past the self-consistency bound (each real symmetry step adds a little systematic scatter), right where a merohedral twin also lands, so the chi² ratio alone cannot separate them. The candidate is now rescued when the extra intensity-proportional systematic error it invokes stays small relative to the confirmed subgroup - a genuine symmetry step gains multiplicity without inflating the merge error model's b, whereas a twin forces non-equivalent reflections together and b balloons. Fixes cubic insulin (I23 instead of I222) with no change to any other crystal in the test battery, including the twins that must stay in their lower symmetry.
* Docs: Document the French-Wilson amplitude estimation, R-free flagging, reference-based space-group/ambiguity resolution, and model-based validation/maps in CPU_DATA_ANALYSIS.md.
* Frontend: The status-bar pill now shows a progress bar during detector calibration (previously only during measurement), and the calibration state and its button are labelled "Calibration"/"CALIBRATE" (the internal `Pedestal` state name is unchanged for back-compatibility).Reviewed-on: #70

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-07-19 09:39:28 +02:00
leonarski_f 451310f43d v1.0.0-rc.158 (#68)
Build Packages / Unit tests (push) Successful in 1h32m35s
Build Packages / build:windows:cuda (push) Successful in 18m0s
Build Packages / build:viewer-tgz:cpu (push) Successful in 7m37s
Build Packages / build:viewer-tgz:cuda (push) Successful in 8m55s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 14m13s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m11s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m35s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 13m57s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 14m23s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m45s
Build Packages / build:rpm (rocky8) (push) Successful in 11m39s
Build Packages / build:rpm (rocky9) (push) Successful in 14m0s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 13m42s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m38s
Build Packages / DIALS test (push) Successful in 14m55s
Build Packages / XDS test (durin plugin) (push) Successful in 7m11s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m7s
Build Packages / XDS test (neggia plugin) (push) Successful in 8m34s
Build Packages / Generate python client (push) Successful in 28s
Build Packages / Build documentation (push) Successful in 1m3s
Build Packages / Create release (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 9m55s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* Analysis: The azimuthal-integration solid-angle correction now follows the incidence angle to the detector normal (`cos^3` of that angle) instead of `cos^3(2*theta)`, so it is correct for a tilted detector and matches PyFAI `solidAngleArray` and MAX IV azint (unchanged for an untilted detector). Crystal geometry refinement (`XtalOptimizer`) no longer silently ignores an imported PONI `rot3` (rotation about the beam): it is applied as a fixed rotation in the residual so refinement stays consistent with the rest of the pipeline. Polarization and azimuthal binning already honoured `rot3` through the full PONI rotation.
* jfjoch_viewer: Open datasets on the WSL2/UNC filesystem (paths starting `\\`); write processing outputs next to the input file, with a Browse button and independent `_process.h5` / merged `.mtz`/`.cif` toggles; and show the determined space group in the merge-statistics window.
* rugnux: Accept an absolute `-o` output prefix in offline processing.
* Packaging: The self-contained Linux viewer `.tgz` now bundles cuFFT, so it runs without a system CUDA toolkit (`.deb`/`.rpm` are unchanged, distro-managed).
* Docs: Bring the analysis references up to date with the code. `docs/CPU_DATA_ANALYSIS.md` now reflects the unified profile-fit Bragg integration engine, multi-lattice indexing, azimuthal phi binning, the radial parallax/bandwidth profile with sub-pixel centring, the rot3d capture-fraction handling and the automatic CC1/2 resolution cutoff, and drops the descriptions of features that were never implemented (French-Wilson amplitudes, the still excitation-error partiality model); `docs/RUGNUX.md` documents the new `--resolution-cutoff`/`--resolution-cc-target`/`--resolution-shells`, `--min-captured-fraction`, `--mosaicity`, `--reference-column`, the azimuthal correction toggles and the geometry-override options, and corrects the `-N` default. The outdated in-source design notes (ICE_RING_DETECTION, BRAGG_INTEGRATION_ENGINE, NEXTGEN_INTEGRATOR) are removed.Reviewed-on: #68

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-07-12 19:42:29 +02:00
leonarski_f 54c0100e8e v1.0.0-rc.157 (#67)
Build Packages / Unit tests (push) Successful in 1h28m28s
Build Packages / build:windows:nocuda (push) Successful in 14m45s
Build Packages / build:windows:cuda (push) Successful in 13m13s
Build Packages / build:viewer-tgz:cpu (push) Successful in 6m47s
Build Packages / build:viewer-tgz:cuda (push) Successful in 7m22s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m52s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m16s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 13m19s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m50s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 14m40s
Build Packages / build:rpm (rocky8) (push) Successful in 11m18s
Build Packages / build:rpm (rocky9) (push) Successful in 12m4s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m55s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m22s
Build Packages / DIALS test (push) Successful in 13m37s
Build Packages / XDS test (durin plugin) (push) Successful in 8m47s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m4s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m45s
Build Packages / Generate python client (push) Successful in 34s
Build Packages / Build documentation (push) Successful in 1m4s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 7m16s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* rugnux: Rebrand the offline data-processing subsystem as `rugnux` and consolidate all offline analysis into the single `rugnux` binary - `jfjoch_process` is now `rugnux`, the former `jfjoch_azint` is now `rugnux --azint-only`, and `jfjoch_scale` is now `rugnux --scale` (see the new docs/NAMING.md and docs/RUGNUX.md). Scaling and merging are on by default for rotation and stills (`--no-merge` disables them), replacing the previous opt-in `-M, --scale-merge`.
* rugnux: CLI fixes - default `-N` to all hardware threads, parse numeric option arguments strictly (reject non-numeric or trailing input instead of silently yielding 0), require `--wavelength > 0`, and correct the reproduced command line and `--scale` reference-cell handling.
* rugnux: De-novo space-group improvements - recover genuine high symmetry and centred Bravais lattices from intensities, add an automatic CC1/2 high-resolution cutoff, and report L-test twinning statistics.
* rugnux: Index weakly-diffracting low-resolution rotation data that previously failed (e.g. F-cubic crystals that diffract only to ~4 A on a detector reaching ~1.5 A). The per-frame indexing gate now measures the indexed fraction only within the resolution range the lattice actually diffracts to, so the many sub-diffraction ice/noise spots no longer make the fraction floor unreachable; the two-pass first pass tries several image-sampling schemes (spread across the whole rotation vs a consecutive wedge whose native stride keeps a reflection's rocking curve continuous, letting the FFT resolve a long axis) and keeps the one that indexes the most frames; and the de-novo space-group search no longer discards all reflections (and crashes) when every resolution shell falls below <I/sigma> = 1.
* rugnux: Lower the low-resolution R-meas for strongly-diffracting rotation data - drop edge-of-sweep truncated fulls whose rocking curve was captured below `--min-captured-fraction` (default 0.7 for rotation), and report R-meas only over the observations kept by outlier rejection (matching XDS). The 0.7 default also strips the partiality-extrapolated fulls that dominate the intensity second moment on weakly-diffracting crystals, so the de-novo space-group search is no longer starved by the error-model I/sigma floor and recovers the correct symmetry (e.g. the F-cubic Benas crystals: Benas_3 -> F432, Benas_7 -> P6122, instead of P4/P1); on the reference battery every other crystal keeps its space group.
* rugnux: Write the refined geometry (beam, tilt, axis) to _process.h5 and place non-standard mmCIF items under a reserved `jfjoch` prefix.
* jfjoch_broker: Ordinary acquisition failures (receiver/writer/analysis problems, missed packets, writer disconnect) now return to the Idle state with an Error-severity message, so a run can be retried without an expensive re-initialisation; only failures that leave the detector in an undefined state (new JFJochCriticalException, e.g. PCIe/FPGA faults) go to the Error state and force re-initialisation.
* jfjoch_broker: A synchronous /start now reports its failure to the HTTP caller instead of returning HTTP 200, and an incomplete or truncated dataset (missing packets, writer disconnect) is reported as an error rather than a "reduce frame rate" warning.
* jfjoch_broker: Drop uncollected placeholder rows (number = -1) from the scan_result REST endpoint.
* jfjoch_broker: Fix the inverted per-image compression ratio reported by the Lite receiver (was compressed/uncompressed instead of uncompressed/compressed).
* jfjoch_broker: Bragg integration adds a quantization-noise variance floor with a box-sum fallback, and treats the type-maximum marker as an invalid pixel for unsigned image types.
* jfjoch_writer: Detect file-overwrite conflicts at start for back-channel transports, and reset the writer when end-of-collection finalisation fails.
* jfjoch_viewer: Preview overlays follow the geometry (resolution/ROI arcs, true beam centre, predictions, coral secondary-lattice spots, legend), add save-as-JPEG, and fix an HTTP live-follow memory leak.
* Frontend: Improved aesthetics and usability, and added in-browser pixel-mask and JUNGFRAU-pedestal visualisation.
* CI: Name the Windows installer jfjoch-viewer-* instead of jfjoch-*.Reviewed-on: #67

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-07-11 07:19:11 +02:00
leonarski_f d6389e12da v1.0.0-rc.156 (#66)
Build Packages / Unit tests (push) Skipped
Build Packages / build:windows:nocuda (push) Successful in 15m31s
Build Packages / build:viewer-tgz:cpu (push) Successful in 5m46s
Build Packages / build:viewer-tgz:cuda (push) Successful in 6m9s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m25s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m21s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m41s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m18s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m26s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m33s
Build Packages / build:rpm (rocky8) (push) Successful in 10m32s
Build Packages / build:rpm (rocky9) (push) Successful in 12m23s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m50s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m12s
Build Packages / DIALS test (push) Successful in 12m6s
Build Packages / XDS test (durin plugin) (push) Successful in 8m15s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m12s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m35s
Build Packages / Generate python client (push) Successful in 27s
Build Packages / Build documentation (push) Successful in 54s
Build Packages / Create release (push) Skipped
Build Packages / build:windows:cuda (push) Successful in 12m37s
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* jfjoch_process: Major rotation (rot3d) data processing overhaul - robust profile-fit integration, Cauchy-loss scaling with optional absorption surface, de-novo indexing and space-group/centering determination fixes, and merging statistics + ISa in the mmCIF output.
* jfjoch_process: Add EXPERIMENTAL ice-ring detection (--detect-ice-rings) that excludes ice reflections from scaling.
* Compression: Add BSHUF_ZSTD_RLE_HUFF, make compression size-aware (drop frames that don't fit rather than aborting), and add the jfjoch_recompress tool.
* jfjoch_viewer: Report "Multiple lattices detected" and grey out "Analyze dataset" on a live connection.
* jfjoch_broker: Write smargon chi/phi goniometer positions to NXmx; read sensor thickness/material from HDF5 metadata.
* CI: Build Windows (CUDA and non-CUDA) installers.Reviewed-on: #66

Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-07-03 19:18:56 +02:00
leonarski_f 75e401f0e5 v1.0.0-rc.153 (#63)
Build Packages / Unit tests (push) Successful in 1h31m59s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m43s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m5s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m27s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m56s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m24s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m27s
Build Packages / build:rpm (rocky8) (push) Successful in 9m20s
Build Packages / build:rpm (rocky9) (push) Successful in 10m50s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m54s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m38s
Build Packages / DIALS test (push) Successful in 12m13s
Build Packages / XDS test (durin plugin) (push) Successful in 7m8s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m8s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m50s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / Build documentation (push) Successful in 50s
Build Packages / Create release (push) Skipped
This is an UNSTABLE release. It includes many experimental features, as well as many AI generated fixes. We recommend using rc.152 for production use.

* jfjoch_broker: Add EXPERIMENTAL pixelrefine mode for image processing
* jfjoch_broker: Allow to load user mask from 8-bit and 16-bit TIFF files
* jfjoch_broker: Add ROI calculation in non-FPGA workflow
* jfjoch_broker: Fixes to TCP image pusher
* jfjoch_broker: Remove NUMA bindings
* jfjoch_broker: Improvements to indexing
* jfjoch_broker: For PSI EIGER, trimming energies are taken from the detector configuration (now compulsory) instead of hardcoded values
* jfjoch_writer: Save ROI definitions and the per-pixel ROI bitmap in the master file; azimuthal ROIs support phi (angular) sectors
* jfjoch_viewer: Major redesign with dockable panels and saved layouts, plus on-canvas creation/move/resize of box, circle and azimuthal ROIs
* jfjoch_viewer: Run jfjoch_process reprocessing jobs from inside the GUI and overlay per-run results

Reviewed-on: #63
2026-06-23 20:29:49 +02:00
leonarski_f cc3eb8352c v1.0.0-rc.148 (#58)
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m28s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m9s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m47s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m58s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m39s
Build Packages / build:rpm (rocky8) (push) Successful in 11m43s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m59s
Build Packages / Generate python client (push) Successful in 35s
Build Packages / Build documentation (push) Successful in 59s
Build Packages / Create release (push) Skipped
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m48s
Build Packages / build:rpm (rocky9) (push) Successful in 12m32s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m24s
Build Packages / XDS test (durin plugin) (push) Successful in 7m35s
Build Packages / XDS test (neggia plugin) (push) Successful in 6m50s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 7m40s
Build Packages / DIALS test (push) Successful in 11m19s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

* jfjoch_broker: Improve azimuthal integration (add <I^2> calculation)
* jfjoch_broker: Fixes around indexing, aiming to handle multi-lattice crystals (work in progress, it is not fully integrated)
* jfjoch_writer: Save mean(I), stddev(I), and count(I) for each azimuthal bin

Reviewed-on: #58
2026-06-08 08:30:35 +02:00
leonarski_f 75de40f52b v1.0.0-rc.147 (#57)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 7m27s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 8m20s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 7m35s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 5m59s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 7m25s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 8m30s
Build Packages / build:rpm (rocky8) (push) Successful in 7m39s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m16s
Build Packages / build:rpm (rocky9) (push) Successful in 9m35s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m6s
Build Packages / Generate python client (push) Successful in 12s
Build Packages / Build documentation (push) Successful in 31s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (durin plugin) (push) Successful in 7m6s
Build Packages / DIALS test (push) Successful in 12m3s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m11s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 5m50s
Build Packages / Unit tests (push) Successful in 57m33s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

* jfjoch_viewer: Add reciprocal space viewer
* jfjoch_process: Two pass algorithm that does spot finding/indexing + integration of full dataset
* jfjoch_process: Improve logic for rotation indexer, to make execution more deterministic (still work in progress)

Reviewed-on: #57
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-06-02 11:49:24 +02:00
leonarski_f fc68a9baed v1.0.0-rc.146 (#56)
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m34s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m0s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m23s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m16s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m49s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m32s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m15s
Build Packages / XDS test (durin plugin) (push) Successful in 7m16s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / build:rpm (rocky9) (push) Successful in 10m12s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 47s
Build Packages / DIALS test (push) Successful in 10m18s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 5m46s
Build Packages / build:rpm (rocky8) (push) Successful in 1h41m2s
Build Packages / XDS test (neggia plugin) (push) Successful in 1h59m18s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset.
jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling.
jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file.
jfjoch_scale: Dedicated tool for rescaling/merging existing data.
jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel.

WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being.

Reviewed-on: #56
2026-05-28 18:48:35 +02:00
leonarski_f caef26873e v1.0.0-rc.145 (#55)
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 16m26s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 14m26s
Build Packages / build:rpm (rocky8) (push) Successful in 17m23s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 17m32s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 18m16s
Build Packages / build:rpm (rocky9) (push) Successful in 12m45s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m58s
Build Packages / XDS test (durin plugin) (push) Successful in 11m22s
Build Packages / DIALS test (push) Successful in 14m28s
Build Packages / Generate python client (push) Successful in 1m1s
Build Packages / Build documentation (push) Successful in 2m40s
Build Packages / Create release (push) Has been skipped
Build Packages / XDS test (neggia plugin) (push) Successful in 10m52s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 15m2s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 17m25s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m49s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 11m34s
Build Packages / Unit tests (push) Successful in 44m51s
This is an UNSTABLE release. The release has significant modifications for HDF5 writing logic - in case of troubles go back to 1.0.0-rc.144.

* **Default HDF5 writing mode is with VDS, not soft-links** - this improves DIALS compatibility and makes format more future-proof, NXmx legacy format might be phased-out in the future.
* XDS plugin: Improve performance of VDS reading.
* jfjoch_writer: Significant improvement on how file systems I/O are handled through a dedicated pass-through VFD.
* jfjoch_writer: Clean-up of HDF5 routines to better handle issues.

Reviewed-on: #55
2026-05-06 21:50:02 +02:00
leonarski_f d760b12a18 v1.0.0-rc.141 (#51)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m32s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m19s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m49s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m37s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m23s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m44s
Build Packages / build:rpm (rocky8) (push) Successful in 9m15s
Build Packages / build:rpm (rocky9) (push) Successful in 12m14s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m28s
Build Packages / Generate python client (push) Successful in 21s
Build Packages / XDS test (durin plugin) (push) Successful in 8m58s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m7s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Create release (push) Has been skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m24s
Build Packages / DIALS test (push) Successful in 14m1s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m26s
Build Packages / Unit tests (push) Failing after 1h20m14s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Azimuthal integration mapping is generated with parallel computations, significantly reducing setup times
* frontend: Fix selection of FFTW in indexing settings

Reviewed-on: #51
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-04-30 13:04:54 +02:00
leonarski_f 239a441ee6 v1.0.0-rc.140 (#50)
Build Packages / Unit tests (push) Successful in 1h20m34s
Build Packages / build:rpm (rocky8) (push) Successful in 13m32s
Build Packages / Generate python client (push) Successful in 24s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 13m6s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m32s
Build Packages / XDS test (durin plugin) (push) Successful in 10m49s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 14m8s
Build Packages / DIALS test (push) Successful in 14m57s
Build Packages / Build documentation (push) Successful in 47s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m30s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m23s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 14m40s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m14s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m55s
Build Packages / build:rpm (rocky9) (push) Successful in 14m23s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m48s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m10s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: For DECTRIS detectors, ZeroMQ link is persistent, to save time for establishing new connection
* jfjoch_broker: Minor bug fixes for rare conditions

Reviewed-on: #50
2026-04-29 21:40:22 +02:00
leonarski_f 4878318c27 v1.0.0-rc.139 (#49)
Build Packages / Unit tests (push) Successful in 1h17m4s
Build Packages / Generate python client (push) Successful in 1m24s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m34s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m37s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m11s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 12m4s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m59s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m4s
Build Packages / build:rpm (rocky8) (push) Successful in 13m12s
Build Packages / XDS test (durin plugin) (push) Successful in 9m57s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m52s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m30s
Build Packages / build:rpm (rocky9) (push) Successful in 13m59s
Build Packages / Build documentation (push) Successful in 1m7s
Build Packages / XDS test (neggia plugin) (push) Successful in 9m24s
Build Packages / Create release (push) Has been skipped
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m54s
Build Packages / DIALS test (push) Successful in 13m14s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Further reduce startup time for DECTRIS detectors by selectively modifying SIMPLON parameters on `/start`
* jfjoch_broker: Further reduce startup time for DECTRIS detectors by not setting beam center and detector distance via SIMPLON API on '/start'
* jfjoch_broker: Add an extra message to ZeroMQ puller ready to monitor Lite worklow preparation time
* jfjoch_broker: Image buffer configuration is postponed for Lite receiver flow till start message is received
* jfjoch_broker: Use nanoseconds internally for frame/image/readout time
* jfjoch_broker: Extra messages added for receiver operation (to be removed after debugging finished)
* jfojch_broker: Improve profiling of different data analysis steps
* jfjoch_broker: Record integration reflection count
* jfjoch_broker: Fix bug where ZeroMQ preview frequency was confusing time units (micro vs. milliseconds)
* jfjoch_broker: Fix bug where '/wait_till_done' got deadlocked
* jfjoch_writer: Fix confusion between NaN and zero in floating-point datasets

**Breaking changes**: detector definition is now using nanoseconds to define minimum frame time, minimum count time and readout time.

Reviewed-on: #49
2026-04-29 09:50:50 +02:00
leonarski_f c981e1b91c v1.0.0-rc.137 (#46)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m7s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m35s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m8s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m24s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 11m29s
Build Packages / build:rpm (rocky8) (push) Successful in 10m27s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m41s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 11m1s
Build Packages / Generate python client (push) Successful in 45s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9) (push) Successful in 12m48s
Build Packages / Build documentation (push) Successful in 1m3s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m10s
Build Packages / XDS test (durin plugin) (push) Successful in 8m59s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m32s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m39s
Build Packages / DIALS test (push) Successful in 13m13s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Better track time for each operation in the processing stack
* jfjoch_broker: Rewrite preprocessing of diffraction images in the non-FPGA workflow to better use GPUs (work in progress)
* jfjoch_broker: Remove ROI calculation in the non-FPGA workflow (work in progress)
* jfjoch_viewer: Toolbar displays image number starting from 1 (instead of 0)

Reviewed-on: #46
2026-04-25 19:59:21 +02:00
leonarski_f bb9f5c715f v1.0.0-rc.135 (#44)
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m55s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m28s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m56s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m47s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m7s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m31s
Build Packages / build:rpm (rocky8) (push) Successful in 12m59s
Build Packages / build:rpm (rocky9) (push) Successful in 14m5s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m30s
Build Packages / Generate python client (push) Successful in 1m18s
Build Packages / Build documentation (push) Successful in 1m3s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (ubuntu2404) (push) Successful in 10m8s
Build Packages / XDS test (durin plugin) (push) Successful in 9m16s
Build Packages / XDS test (neggia plugin) (push) Successful in 7m59s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 9m12s
Build Packages / DIALS test (push) Successful in 11m44s
Build Packages / Unit tests (push) Successful in 1h23m8s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* Multiple small bug fixes scattered across the whole code base. (detected with GPT-5.4)
* jfjoch_viewer: Improve image render performance

Reviewed-on: #44
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-04-16 11:59:59 +02:00
leonarski_f 4a852b4d6b v1.0.0-rc.134 (#43)
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m57s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 13m4s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m18s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m12s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 13m51s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m59s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m45s
Build Packages / build:rpm (rocky8) (push) Successful in 12m29s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 12m2s
Build Packages / Generate python client (push) Successful in 24s
Build Packages / XDS test (durin plugin) (push) Successful in 9m50s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9) (push) Successful in 14m15s
Build Packages / Build documentation (push) Successful in 1m6s
Build Packages / DIALS test (push) Successful in 13m10s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 6m45s
Build Packages / XDS test (neggia plugin) (push) Successful in 5m58s
Build Packages / Unit tests (push) Successful in 1h20m42s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Add better locking for detector object - should help, when detector initialization takes too long
* jfjoch_writer: Enable writing single, integrated HDF5 file with both data and metadata
* XDS plugin: Add generation of Jungfraujoch plugin for XDS
* CI: Add tests with XDS and DIALS (`xia2.ssx`)

Reviewed-on: #43
2026-04-09 13:30:47 +02:00
leonarski_f 6133da1377 v1.0.0-rc.133
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m5s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 11m6s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 13m47s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 14m9s
Build Packages / Generate python client (push) Successful in 49s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m38s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 12m35s
Build Packages / build:rpm (rocky8) (push) Successful in 9m16s
Build Packages / build:rpm (rocky9) (push) Successful in 10m22s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m20s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m52s
Build Packages / Build documentation (push) Failing after 9s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.132.

* jfjoch_broker: Use httplib for HTTP server instead of Pistache
* jfjoch_broker: Drop OpenSSL support
* jfjoch_broker: Base work for multi-lattice support in the future
* Update dependencies to more recent versions (spdlog, HDF5, Catch2, httplib)

Reviewed-on: #41
2026-03-26 20:50:33 +01:00
leonarski_f 166fcdb68f v1.0.0-rc.131 (#39)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m20s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m46s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m27s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m32s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m57s
Build Packages / build:rpm (rocky8) (push) Successful in 11m54s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m9s
Build Packages / build:rpm (rocky9) (push) Successful in 12m37s
Build Packages / Generate python client (push) Successful in 24s
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 57s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m12s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m4s
Build Packages / Unit tests (push) Successful in 1h17m43s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Fix bug in saving JUNGFRAU calibration (pedestal/pedestalRMS)
* jfjoch_viewer: Fix calibration (pedestal) images being open flipped
* jfjoch_process: Add space group detection (EXPERIMENTAL)

Reviewed-on: #39
2026-03-07 11:34:04 +01:00
leonarski_f 64002f1e29 v1.0.0-rc.129 (#36)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m14s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m43s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m35s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m20s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m23s
Build Packages / Generate python client (push) Successful in 39s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m24s
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 1m0s
Build Packages / build:rpm (rocky8) (push) Successful in 10m35s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 10m35s
Build Packages / build:rpm (rocky9) (push) Successful in 11m17s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m9s
Build Packages / Unit tests (push) Failing after 1h18m57s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Significant improvements in TCP image socket, as a viable alternative for ZeroMQ sockets (only a single port on broker side, dynamically change number of writers, acknowledgments for written files)
* jfjoch_broker: Delta phi is calculated also for still data in Bragg prediction
* jfjoch_broker: Image pusher statistics are accessible via the REST interface
* jfjoch_writer: Supports TCP image socket and for these auto-forking option

Reviewed-on: #36
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-03-05 22:13:12 +01:00
leonarski_f f3e0a15d26 v1.0.0-rc.127 (#34)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m51s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m0s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m6s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m7s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m47s
Build Packages / Generate python client (push) Successful in 29s
Build Packages / Build documentation (push) Successful in 43s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m46s
Build Packages / build:rpm (rocky8) (push) Successful in 9m33s
Build Packages / Unit tests (push) Has been skipped
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m47s
Build Packages / build:rpm (rocky9) (push) Successful in 9m55s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m4s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Default EIGER readout time is 20 microseconds
* jfjoch_broker: Multiple improvements regarding performance
* jfjoch_broker: Image buffer allows to track frames in preparation and sending
* jfjoch_broker: Dedicated thread for ZeroMQ transmission to better utilize the image buffer
* jfjoch_broker: Experimental implementation of transmission with raw TCP/IP sockets
* jfjoch_writer: Fixes regarding properly closing files in long data collections
* jfjoch_process: Scale & merge has been significantly improved, but it is not yet integrated into mainstream code

Reviewed-on: #34
2026-03-02 15:57:12 +01:00
leonarski_f 1ab257af6c v1.0.0-rc.125 (#32)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 12m8s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 12m57s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 12m55s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 12m0s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 13m30s
Build Packages / Generate python client (push) Successful in 20s
Build Packages / Unit tests (push) Has been skipped
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 39s
Build Packages / build:rpm (rocky8) (push) Successful in 9m23s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 10m33s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m2s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m42s
Build Packages / build:rpm (rocky9) (push) Successful in 9m38s
This is an UNSTABLE release. This version adds scalign and merging. These are experimental at the moment, and should not be used for production analysis.
If things go wrong with analysis, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Improve logic on switching on/off spot finding
* jfjoch_broker: Increase maximum spot count for FFBIDX to 65536
* jfjoch_broker: Increase default maximum unit cell for FFT to 500 A (could have performance impact, TBD)
* jfjoch_process: Add scalign and merging functionality - program is experimental at the moment and should not be used for production analysis
* jfjoch_viewer: Display partiality and reciprocal Lorentz-polarization correction for each reflection
* jfjoch_writer: Save more information about each reflection

Reviewed-on: #32
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-02-18 16:17:21 +01:00
leonarski_f 07fe4dd3bb v1.0.0-rc.124 (#31)
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m23s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m32s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 9m15s
Build Packages / Generate python client (push) Successful in 19s
Build Packages / Build documentation (push) Successful in 49s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m13s
Build Packages / build:rpm (rocky8) (push) Successful in 9m10s
Build Packages / build:rpm (rocky9) (push) Successful in 9m58s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m52s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m42s
Build Packages / Unit tests (push) Successful in 1h12m44s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m30s
This is an UNSTABLE release. This version significantly rewrites code to predict reflection position and integrate them,
especially in case of rotation crystallography. If things go wrong with analysis, it is better to revert to 1.0.0-rc.123.

* jfjoch_broker: Improve refection position prediction and Bragg integration code.
* jfjoch_broker: Align with XDS way of calculating Lorentz correction and general notation.
* jfjoch_writer: Fix saving mosaicity properly in HDF5 file.
* jfjoch_viewer: Introduce high-dynamic range mode for images
* jfjoch_viewer: Ctrl+mouse wheel has exponential change in foreground (+/-15%)
* jfjoch_viewer: Zoom-in numbers have better readability

Reviewed-on: #31
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-02-01 13:29:33 +01:00
leonarski_f 1c4dfd03e2 v1.0.0-rc.123 (#30)
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m22s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m30s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m41s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m32s
Build Packages / Generate python client (push) Successful in 18s
Build Packages / Build documentation (push) Successful in 54s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m44s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m53s
Build Packages / build:rpm (rocky8) (push) Successful in 9m40s
Build Packages / build:rpm (rocky9) (push) Successful in 10m37s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 9m54s
Build Packages / Unit tests (push) Successful in 1h6m33s
This is an UNSTABLE release.

* jfjoch_broker: Use newer version of Google Ceres for (potential) CUDA 13 compatibility
* jfjoch_broker: Improve performance of generating preview images, especially for large detectors (9M-16M)
* jfjoch_viewer: Improve performance of displaying images, especially for large detectors (9M-16M)
* jfjoch_viewer: Add more color schemes for better image readability
* HDF5: Common mutex for reading and writing HDF5 if both operations were to happen in the same executable
* HDF5: suppress warning if path (upstream group) doesn't exists when checking if leaf exists

Reviewed-on: #30
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2026-01-30 13:43:09 +01:00
leonarski_f 27496b8207 v1.0.0-rc.122 (#29)
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 7m57s
Build Packages / Generate python client (push) Successful in 18s
Build Packages / Build documentation (push) Successful in 35s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky9) (push) Successful in 8m28s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m6s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m9s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 8m44s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 7m56s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 7m25s
Build Packages / Unit tests (push) Successful in 1h11m19s
Build Packages / build:rpm (rocky8) (push) Successful in 6m31s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 6m40s
This is an UNSTABLE release.

* jfjoch_broker: Add thresholding to prefer shorter vectors after FFT
* jfjoch_broker: Add experimental mosaicity estimation for rotation experiments (this is work in progress)
* jfjoch_viewer: Display file opening errors
* jfjoch_viewer: When loading files over DBus add retry/back-off till the file is available

Reviewed-on: #29
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-12-16 15:27:40 +01:00
leonarski_f e2b240356c v1.0.0-rc.121 (#28)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 9m28s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m25s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 9m4s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m27s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m36s
Build Packages / Generate python client (push) Successful in 32s
Build Packages / Build documentation (push) Successful in 45s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 8m45s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m51s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m57s
Build Packages / build:rpm (rocky9) (push) Successful in 9m35s
Build Packages / Unit tests (push) Successful in 1h13m45s
This is an UNSTABLE release.

* jfjoch_broker: Report changes in the image buffer, so viewer doesn't reload constantly
* jfjoch_viewer: Improve performance of loading images
* jfjoch_viewer: Auto-throttle image loading in HTTP-sync / movie modes
* jfjoch_viewer: Auto-foreground calculated with histogram
* jfjoch_viewer: Fix rare segmentation fault

Reviewed-on: #28
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-12-12 21:24:20 +01:00
leonarski_f 36d0507758 v1.0.0-rc.119 (#26)
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 10m11s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m22s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m27s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m19s
Build Packages / Generate python client (push) Successful in 17s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 8m44s
Build Packages / build:rpm (rocky8) (push) Successful in 8m44s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m33s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m11s
Build Packages / build:rpm (rocky9) (push) Successful in 9m54s
Build Packages / Unit tests (push) Failing after 1h11m12s
This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

* jfjoch_broker: Add binary export of data analysis plots over OpenAPI
* jfjoch_broker: Minor fixes to HTTP error handling
* jfjoch_viewer: Prefer binary plots over JSON plots
* jfjoch_viewer: Change foreground with F button + wheel
* jfjoch_viewer: Change way how angles are displayed
* jfjoch_viewer: Display resolution of the mouse cursor in top left corner

Reviewed-on: #26
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-12-08 19:53:35 +01:00
leonarski_f b02c412d59 v1.0.0-rc.118 (#25)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m22s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 8m37s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m15s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 9m39s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m17s
Build Packages / Generate python client (push) Successful in 21s
Build Packages / Build documentation (push) Successful in 43s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 7m51s
Build Packages / build:rpm (rocky9) (push) Successful in 8m45s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m55s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m22s
Build Packages / Unit tests (push) Failing after 1h5m51s
This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

* jfjoch_viewer: Fix issue when HTTP sync silently disconnected when it was enabled when the broker was starting measurement.
* jfjoch_broker: Add protections on time of geometry optimization and reduce rotation recalculations

Reviewed-on: #25
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-12-06 20:05:30 +01:00
leonarski_f d1a4c19ef3 v1.0.0-rc.117 (#24)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m23s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 7m2s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 8m3s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 7m49s
Build Packages / build:rpm (rocky8) (push) Successful in 7m43s
Build Packages / Generate python client (push) Successful in 14s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 7m51s
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 31s
Build Packages / build:rpm (rocky9) (push) Successful in 8m32s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m7s
Build Packages / Unit tests (push) Successful in 1h12m37s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Failing after 2m34s
This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

* jfjoch_viewer: Add ROI results to the dataset info plots
* jfjoch_writer: Remove HTTP interface, as it is not needed/used at the moment

Reviewed-on: #24
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-12-05 22:03:07 +01:00
leonarski_f 31a357fa57 v1.0.0-rc.113 (#19)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m0s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m2s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m54s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m20s
Build Packages / Generate python client (push) Successful in 24s
Build Packages / Build documentation (push) Successful in 56s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 8m51s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m9s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m53s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m21s
Build Packages / build:rpm (rocky9) (push) Successful in 9m47s
Build Packages / Unit tests (push) Successful in 1h13m38s
This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

* jfjoch_broker: Improve handling of rotation indexing
* jfjoch_broker: More information saved in CBOR end message (WIP)
* jfjoch_writer: Save rotation indexing lattice parameters and Niggli class
* jfjoch_viewer: Remove (for now) primitive cell information

Reviewed-on: #19
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-12-02 09:29:22 +01:00
leonarski_f 06949caf1a v1.0.0-rc.112 (#18)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 8m53s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 9m40s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 8m25s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m17s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m5s
Build Packages / Generate python client (push) Successful in 34s
Build Packages / Build documentation (push) Successful in 42s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 8m35s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m2s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m40s
Build Packages / build:rpm (rocky9) (push) Successful in 9m14s
Build Packages / Unit tests (push) Successful in 1h15m9s
This is an UNSTABLE release and not recommended for production use (please use rc.11 instead).

* jfjoch_broker: Experimental rotation (3D) indexing
* jfjoch_broker: Minor fix to error in optimizer potentially returning NaN values

Reviewed-on: #18
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-11-30 17:39:22 +01:00
leonarski_f f44c6520a8 v1.0.0-rc.107 (#12)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 7m33s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 8m46s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 7m19s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 7m50s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 9m8s
Build Packages / Generate python client (push) Successful in 17s
Build Packages / Build documentation (push) Successful in 43s
Build Packages / Create release (push) Has been skipped
Build Packages / build:rpm (rocky8) (push) Successful in 8m25s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 7m39s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 8m13s
Build Packages / build:rpm (rocky9) (push) Successful in 9m33s
Build Packages / Unit tests (push) Successful in 1h14m4s
This is an UNSTABLE release.

* jfjoch_viewer: Minor polishing of new functionality
* jfjoch_broker: User NaN for empty azimuthal bins

Reviewed-on: #12
Co-authored-by: Filip Leonarski <filip.leonarski@psi.ch>
Co-committed-by: Filip Leonarski <filip.leonarski@psi.ch>
2025-11-24 07:07:32 +01:00