MX_Pmodule/crystfel

crystfel

Data processing for serial (femtosecond) crystallography at FELs and synchrotrons: indexamajig, partialator, process_hkl, cell_explorer, the CrystFEL GUI and friends.

• Upstream: https://www.desy.de/~twhite/crystfel/
• License: GPLv3

How it is built

Source tarball of the release tag from DESY GitLab (https://gitlab.desy.de/thomas.white/crystfel/-/archive//crystfel-.tar.gz, sha256-verified). The upstream page https://desy.de/~twhite/crystfel/ hangs (curl: (28) Operation timed out ... 0 bytes received) from Ra and outside, so it is not used. Built with meson per upstream INSTALL.md: meson setup build --prefix=$PREFIX && meson compile -C build && meson install -C build.

meson/ninja via uv. meson.build needs meson >= 0.60; the Ra system meson is too old and the system python is 3.9 (EOL, meson >= 1.12 needs >= 3.10). pyproject.toml + uv.lock pin meson and ninja for Python 3.12; the build runs uv sync --locked into a throwaway venv in the build scratch dir, on a uv-managed Python 3.12 (downloaded on first use, also into scratch). Nothing of this lands in $PREFIX. uv must be on the build host (default ~/.local/bin/uv, pip3 install --user uv; override with UV=/path/to/uv). Wheel cache is the shared /opt/psi/MX/.uv-cache (home quota).

HDF5 is bundled. The Ra build node has no hdf5-devel, no h5cc and no hdf5.pc, so meson setup died with Dependency "hdf5" not found (tried pkg-config, pkg-config and config-tool). The build therefore fetches the HDF5 release tarball (GitHub, sha256-verified, unpacker: none so it is only copied into $SRC_DIR), configures it with autotools into $PREFIX (--libdir=$PREFIX/lib64, serial, no tests, zlib required, szip off) and puts $PREFIX/bin on PATH so meson's hdf5 config-tool picks up h5cc. Autotools on purpose: meson refuses a cmake-built h5cc. crystfel installs its own libs to lib64 too and bakes $ORIGIN/../lib64 (binaries) / $ORIGIN (libcrystfel) RPATHs, so no LD_LIBRARY_PATH is set by the modulefile; the install hook runs ldd indexamajig and dies on any not found. h5dump & co. come along on the module PATH.

Build-host prerequisites (the build dies with a hint when one is missing): gcc, make, flex, bison, zlib-devel, gsl-devel, git (meson wraps), uv. Optional, only picked up if present at build time: gtk3/cairo/gdk-pixbuf (GUI: crystfel, cell_explorer), fftw, zmq, msgpack, libtiff. Indexers xgandalf/pinkindexer/fdip and millepede are meson subprojects fetched from gitlab.desy.de during meson setup (optional; skipped if the fetch fails).

Upstream warns: do NOT have any conda environment active while building.

TODO: no HDF5 filter plugins (bitshuffle/LZ4, as written by jfjoch/Eiger) are installed, so indexamajig cannot read such data until a plugin dir exists and HDF5_PLUGIN_PATH is set in the modulefile. Needs a decision where the plugins should live (own module vs. bundled here).

Loading the module puts $PREFIX/bin on PATH.

Building on Ra

module use unstable && module load modbuild/2.1.3
cd crystfel
modbuild build --tmpdir=/das/work/p21/p21515/.cache/Pmodules \
               --distdir=/das/work/p21/p21515/.cache/Pmodules 0.13.0
module use MX unstable && module load crystfel/0.13.0 && indexamajig --version

Verifying the install

module use MX unstable && module load crystfel/0.13.0
which indexamajig
indexamajig --version
ldd $(which indexamajig) | grep -E 'hdf5|crystfel'   # both from $PREFIX/lib64
h5cc -showconfig | grep -i deflate                   # zlib filter present
cell_explorer         # GUI check, needs X and a gtk3 build

Adding a new version

Add the version under versions: and its tarball sha256 under shasums: in files/config.yaml (fetch the GitLab archive URL above with the new tag and sha256sum it; tag names on GitLab match the release version, no v prefix), then on a Ra login node: modbuild build <version> (modbuild/2.1.3).

Bumping HDF5: change the GitHub url + name: and its shasums: entry in files/config.yaml (stay on 1.14.x: 2.x drops autotools, see above).

Bumping meson/ninja: edit pyproject.toml, run uv lock, commit uv.lock.