Jungfraujoch/docs/SOFTWARE_INTEGRATION.md

Integration with MX data processing software

Jungfraujoch writes NXmx HDF5 in three layouts (see HDF5 / NeXus data format), and not every downstream program reads all three. NXmxVDS is the default and the one to use unless a program specifically needs another.

	NXmxLegacy	NXmxVDS (default)	NXmxIntegrated
Jungfraujoch XDS plugin	yes	yes	yes
Durin (Global Phasing)	yes	yes	yes
Durin (Diamond, original)	yes	known bugs	known bugs
Neggia	yes	no — no virtual-dataset support	not tested
DIALS / xia2	only one data file	yes	yes
CrystFEL	yes	yes	yes

NXmxLegacy joins the data files to the master with external links, which is what DECTRIS's filewriter-1 format did. Use it only for a program that needs it, and then keep the whole run in a single data file — see the DIALS section below.

XDS

XDS reads HDF5 through a plugin, named in XDS.INP:

LIB="/opt/xds/lib/libjfjoch_xds_plugin.so.1.0.1"

Use the Jungfraujoch plugin. It is released for Linux (x86_64, built on RHEL 8) and macOS (Apple Silicon, libjfjoch_xds_plugin.<version>.dylib) as a .tgz on the Gitea release page, and also ships inside the jfjoch_viewer RPM/APT packages. Extract the archive into a directory of its own; it holds the library under lib/ and its license notices under share/doc/. The three numbers are the plugin version and change over time. The plugin is GPL-3.0, with an additional permission to be linked with XDS (see License).

The alternatives, in order of preference:

• Durin, Global Phasing build — github.com/CV-GPhL/durin. Prefer it over the original from Diamond Light Source, which has known bugs with non-DECTRIS files (virtual datasets and the single-file layout). It is the only third-party plugin that reads signed Jungfraujoch images correctly.
• Neggia — github.com/dectris/neggia. No virtual-dataset support, so it cannot read the default layout. It also mis-reads signed 16-bit images: it dispatches on the pixel size in bytes and always casts to an unsigned type, so a count of -2 reaches XDS as 65534 and the -32768 error marker as 32768. Signed 32-bit degrades safely. Since JUNGFRAU in photon-counting conversion writes signed images by default, this affects the ordinary PSI case — do not use Neggia for it.

OVERLOAD must be set by hand

No XDS plugin — ours, Durin or Neggia — reads saturation_value from the file. XDS therefore takes its overload from OVERLOAD= in XDS.INP, and you must set it to the master file's /entry/instrument/detector/saturation_value. XDS treats it inclusively: a pixel is overloaded when it exceeds OVERLOAD.

Signed images

MINIMUM_VALID_PIXEL_VALUE= may not be negative in current XDS, so a genuinely negative photon count — the reason signed output exists — cannot be declared valid. xia2 clamps the value to 0. There is no header field that changes this: if XDS is the target, consider collecting unsigned.

Which pixels are masked

The plugins do not all act on the same mask bits, so XDS and DIALS do not mask the same pixels:

mask bit	meaning	jfjoch plugin	Durin / Neggia	DIALS
0	module gap	yes	yes	yes
1, 4	error, noisy	yes	yes	yes
8, 9	user mask, beam stop	yes	no	yes
30	module edge	yes	no	yes
31	chip gap	no	no	yes

DIALS masks a pixel whenever any pixel_mask bit is set; Durin and Neggia look only at bits 0–4. On a JUNGFRAU with the default edge masking this is a difference of order 2% of the detector. Bits 30 and 31 mark pixels that are larger than normal rather than bad, which is why the Jungfraujoch plugin passes chip-gap pixels through — but be aware that a dataset processed by XDS and by DIALS will not have used exactly the same pixels.

DIALS

Tested regularly against DIALS (currently 3.27.0), including the xia2.ssx pipeline for serial crystallography.

• Use NXmxVDS or NXmxIntegrated. With NXmxLegacy, DIALS reads only the first data file and reports a correspondingly short image count, without an error; if a goniometer is present it then fails on the frames past the first file. A legacy run that fits in one data file is read correctly — set images_per_file to cover the whole run.
• Unsigned 32-bit images require bit_depth_readout, which Jungfraujoch writes. For signed images the field is deliberately omitted: DIALS remaps the top two codes of 2^bit_depth_readout, and on signed data those land inside the trusted range.
• trusted_range is inclusive at both ends, and is taken from underload_value and saturation_value.

pyFAI

rugnux --mode calibration writes a .poni file describing the detector geometry — see Detector geometry and Rugnux.

• It declares orientation, so it needs pyFAI 2024.01 or newer. An older pyFAI ignores the key and places the beam centre wrongly along the slow axis.

• A .poni file carries geometry only. pyFAI does not learn the saturation value, the error marker or the pixel mask from it, and will happily integrate a masked pixel at UINTx_MAX as a count. Pass the marker and the mask at integration time:

  ai = pyFAI.load("calibration.poni")
  res = ai.integrate1d(image, 1000, dummy=65535, delta_dummy=0.5, mask=pixel_mask != 0)
  

  with dummy set to the master file's /entry/instrument/detector/error_value for the stored pixel type, and pixel_mask read from /entry/instrument/detector/pixel_mask.

CrystFEL

Jungfraujoch files are compatible with CrystFEL. max_adu is inclusive — a pixel is bad when it exceeds the value — so set it from saturation_value.