Commit Graph
6 Commits
Author SHA1 Message Date
leonarski_fandClaude Opus 4.8 636ed1e180 space group: use merge reduced chi^2 (not R-meas) for the consistency check
The R-meas-ratio over-symmetry guard (commit 7ade6d9) missed a near-perfect
pseudo-2-fold: InsH2 (XDS R3) has a false 2-fold at operator CC 0.85 whose merged
intensities still correlate well, so R-meas inflated only ~1.3x and R32 was accepted
(ISa collapsing to 3.9). The reduced chi^2 (within-orbit scatter / sigma^2) is the
right signal - false equivalents disagree by many sigma even when they correlate well.

Measured chi^2 ratio (candidate / best subgroup) across the test set: true point groups
<= 1.63 (a cubic merge of a noisy crystal, InsI3, is the worst real case), false >= 2.26
(InsH2) and 6.0 (InsH3); threshold 2.0 separates them. InsH2 now resolves to R3 (ISa 6.1
from the false R32's 3.9); InsH3/InsI2/InsI3/cytC/lysoC/MyoB/EP0210 unchanged and correct.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 21:54:51 +02:00
leonarski_fandClaude Opus 4.8 7ade6d9906 space group: reject over-symmetrised point groups by merge R-meas consistency
Per-operator CC alone cannot separate a real weak symmetry operator from a false
moderate one: a noisy crystal's genuine cubic 2-folds (InsI2 I23, CC ~0.51) score
BELOW a pseudo-symmetric crystal's false 2-fold (InsH3, CC 0.64), so no CC threshold
works. Add a self-consistency test: a candidate point group is accepted only if
merging the intensities under it does not inflate R-meas past max_rmeas_ratio (1.5x)
the most-consistent candidate. A false operator forces non-equivalent reflections
together and blows R-meas up; a real one leaves it flat.

Fixes InsH3: was over-called R32 (ISa collapsed to 2.2 from the forced merge), now
correctly R3/H3 (ISa 10.2, matching XDS). InsI2 stays I23, and lysoC/InsI3/MyoB/
EP0210/cytC/lyso_ref are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 20:32:03 +02:00
leonarski_fandClaude Opus 4.8 02477f1ce4 space group: determine centering from absences; jfjoch_process re-indexes from scratch
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 14m5s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 14m29s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 14m41s
Build Packages / build:rpm (rocky8) (push) Successful in 14m46s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 14m59s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 15m13s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 15m29s
Build Packages / XDS test (durin plugin) (push) Successful in 7m54s
Build Packages / Generate python client (push) Successful in 30s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 8m12s
Build Packages / Create release (push) Skipped
Build Packages / XDS test (neggia plugin) (push) Successful in 8m23s
Build Packages / Build documentation (push) Successful in 1m8s
Build Packages / build:windows:nocuda (push) Failing after 25m35s
Build Packages / build:windows:cuda (push) Failing after 25m35s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 11m12s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 12m2s
Build Packages / build:rpm (rocky9) (push) Successful in 12m45s
Build Packages / DIALS test (push) Successful in 12m52s
Build Packages / Unit tests (push) Successful in 59m5s
Fixes wrong space-group assignment on cubic insulin (true I2_13, a=77.6).

SearchSpaceGroup: drop the lattice-centering gate (and the now-unused
lattice_centering option). The indexer often returns the conventional cell,
whose geometry hides I/F/C centering - that information lives only in the
systematic absences. Stage B now tests every centering of the point group and
confirms it from the data, so an indexer-reported 'P' no longer excludes
I2_13/I23. (I23 and I2_13 remain indistinguishable by absences and are reported
as such.)

jfjoch_process: discard any unit cell stored in the input HDF5 by default so the
cell is re-determined from scratch. A stale/wrong stored cell otherwise resolves
the indexing algorithm to FFBIDX, which trusts that cell and locks onto the wrong
lattice (Ins_I_3 went from 2.7% -> 76% indexing). A user-supplied -C cell still
applies.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 16:39:03 +02:00
leonarski_fandClaude Opus 4.8 786af96b3b SearchSpaceGroup: POINTLESS-style rewrite, pipeline integration, twinning test
Space-group search (image_analysis/scale_merge/SearchSpaceGroup):
- Two-stage POINTLESS-style determination. Stage A scores each distinct rotation
  operator once (was once per candidate space group, ~34x faster on lysozyme:
  ~26s -> <1s) and picks the largest point group all of whose operators confirm.
  Stage B picks the maximal space group whose predicted absences are confirmed
  weak, fixing the prototype's default to the symmorphic group (it returned P422
  instead of P4(3)2(1)2). Enantiomorphic / origin-ambiguous pairs (P4(1) vs P4(3),
  I222 vs I2(1)2(1)2(1)) are reported as indistinguishable.
- Constrain candidates to subgroups of the lattice (metric) holohedry and weigh
  centering only P-vs-metric, fed from rotation indexing's LatticeSearch result.

Integration / pipeline:
- With no user-fixed space group, predict in P (IndexAndRefine) so the
  centering-absent reflections are integrated and the search can confirm/deny
  centering (catching pseudo-centering / a missed superstructure) instead of
  trusting the metric; a user-fixed group still rejects absences in integration.
- JFJochProcess: scale+merge in P1 -> determine the space group -> set it and
  re-scale+merge in it (statistics then come out in the right symmetry) -> write
  it to /entry/sample/space_group_number (new EndMessage.space_group_number,
  preferred by NXmx::Sample). jfjoch_scale no longer searches; it consumes the
  file's space group (and no longer clobbers it with an empty -S).

Twinning (new image_analysis/scale_merge/TwinningAnalysis): Padilla-Yeates L-test
(<|L|>, <L^2>; acentric-only, positive intensities so L is bounded) plus a
shell-normalised <I^2>/<I>^2 second moment and a twin-fraction estimate. Reported
after the final merge in jfjoch_process and jfjoch_scale, and surfaced in the
jfjoch_viewer merge-statistics window with a red outline when twinning is suspected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:11:28 +02:00
leonarski_f fc68a9baed v1.0.0-rc.146 (#56)
Build Packages / Unit tests (push) Skipped
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 8m34s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 10m0s
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 10m23s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m23s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 11m16s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 11m49s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m32s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m15s
Build Packages / XDS test (durin plugin) (push) Successful in 7m16s
Build Packages / Generate python client (push) Successful in 16s
Build Packages / build:rpm (rocky9) (push) Successful in 10m12s
Build Packages / Create release (push) Skipped
Build Packages / Build documentation (push) Successful in 47s
Build Packages / DIALS test (push) Successful in 10m18s
Build Packages / XDS test (JFJoch plugin) (push) Successful in 5m46s
Build Packages / build:rpm (rocky8) (push) Successful in 1h41m2s
Build Packages / XDS test (neggia plugin) (push) Successful in 1h59m18s
This is an UNSTABLE release. The release has significant modifications for data processing - in case of troubles go back to 1.0.0-rc.144.

jfjoch_process: Generate a dedicated file (_process.h5), which can be used as a replacement for the _master.h5 file for a reanalyzed dataset.
jfjoch_process: Improve the performance of scaling and merging, implement on the fly scaling.
jfjoch_writer: All final data analysis results are repopulated in the _master.h5 file.
jfjoch_scale: Dedicated tool for rescaling/merging existing data.
jfjoch_viewer: Fix bugs where pixel labels where displayed on a wrong pixel.

WARNING! Scaling and merging are experimental at the moment, and may not provide reasonable results for the time being.

Reviewed-on: #56
2026-05-28 18:48:35 +02:00
leonarski_f 166fcdb68f v1.0.0-rc.131 (#39)
Build Packages / build:rpm (rocky8_nocuda) (push) Successful in 11m20s
Build Packages / build:rpm (ubuntu2404_nocuda) (push) Successful in 10m46s
Build Packages / build:rpm (ubuntu2204_nocuda) (push) Successful in 11m27s
Build Packages / build:rpm (rocky9_nocuda) (push) Successful in 12m32s
Build Packages / build:rpm (rocky8_sls9) (push) Successful in 10m57s
Build Packages / build:rpm (rocky8) (push) Successful in 11m54s
Build Packages / build:rpm (rocky9_sls9) (push) Successful in 13m9s
Build Packages / build:rpm (rocky9) (push) Successful in 12m37s
Build Packages / Generate python client (push) Successful in 24s
Build Packages / Create release (push) Has been skipped
Build Packages / Build documentation (push) Successful in 57s
Build Packages / build:rpm (ubuntu2204) (push) Successful in 9m12s
Build Packages / build:rpm (ubuntu2404) (push) Successful in 8m4s
Build Packages / Unit tests (push) Successful in 1h17m43s
This is an UNSTABLE release. The release has significant modifications and bug fixes, if things go wrong, it is better to revert to 1.0.0-rc.124.

* jfjoch_broker: Fix bug in saving JUNGFRAU calibration (pedestal/pedestalRMS)
* jfjoch_viewer: Fix calibration (pedestal) images being open flipped
* jfjoch_process: Add space group detection (EXPERIMENTAL)

Reviewed-on: #39
2026-03-07 11:34:04 +01:00