MX_Pmodule/olex2

olex2

Olex2 — open-source (BSD) small-molecule crystal structure solution, refinement and analysis (olex2.refine / ShelX front-end, built on cctbx/smtbx). This module provides both the GUI and the console build:

launcher	what it is
olex2	wxWidgets GUI (needs an X display)
olex2c	console / "headless" Olex2 — same engine, no GUI, scriptable

• Upstream: https://www.olexsys.org/olex2 — install notes: https://www.olexsys.org/olex2/docs/getting-started/installing-olex2/
• Source: https://github.com/pcxod/olex2 (C++ engine; the HTML GUI lives in svn.olex2.org/olex2-gui) — not used, see "Why the binary archive"
• License: BSD (https://www.olexsys.org/olex2/docs/getting-started/installing-olex2/open-source/)
• Download server: https://secure.olex2.org/olex2-distro/ (mirror http://www2.olex2.org/olex2-distro/)

How it is built

Upstream (install notes, "Linux"): "We do not provide installers for Linux – and provide zip files instead (Use the 'start' bash script to launch Olex2)" and "Nothing needs to be compiled". The official olex2-distro/1.5/olex2-linux64.zip (255 MB, ~1 GB unpacked) already contains everything (verified by listing the archive, 2026-10-16):

olex2/olex2      GUI binary                  olex2/olex2c   console binary (24 MB)
olex2/unirun     vendor auto-updater         olex2/start    vendor launcher (runs unirun, then olex2)
olex2/cctbx/     cctbx_build + cctbx_sources olex2/lib/     private CPython 3.8 + libpython, openblas, ssl, png, ...
olex2/ilib/      olex2/util/pyUtil/  olex2/etc/  olex2/sample_data/  NoSpherA2, hart, hart_mpi ...

The recipe (build):

1. default pbuild::prep downloads the zip, three Fedora 40 rpms and a static patchelf per urls: in files/config.yaml into the distfiles cache, verifies the pinned sha256 of each and copies them to SRC_DIR (unpacker: none);
2. configure runs unzip -t (truncated-download guard), unzips, extracts the rpms with rpm2cpio | cpio into SRC_DIR/sysroot and checks the bundled ld.so runs;
3. install copies the vendor tree flat into $PREFIX — Olex2 derives its BaseDir() from the binary location, so the layout must stay as shipped — plus $PREFIX/sysroot, fixes modes, switches usettings.dat to update=Never, patchelf's olex2c olex2 unirun NoSpherA2 to the bundled loader (see next section), installs files/olex2-launcher twice as $PREFIX/bin/olex2 and $PREFIX/bin/olex2c, and proves with ld.so --list olex2c that the binary resolves through the bundled glibc. Only $PREFIX/bin goes on PATH.

files/olex2-launcher is the vendor start script minus the updater: it exports OLEX2_CCTBX_DIR, BOOST_ADAPTBX_*, OLEX2_GL_*, LD_LIBRARY_PATH=$PREFIX/lib:$PREFIX/ilib:$PREFIX/cctbx/cctbx_build/lib, PYTHONHOME=$PREFIX, LANG/LC_ALL=C.UTF-8, unsets PYTHONPATH (an inherited one would be imported by the embedded Python 3.8 and break it), sets LIBGL_ALWAYS_SOFTWARE=1 for the GUI only, then execs the binary. unirun is never run: /opt/psi/MX is shared and read-only, updates are done by building a new module version. The modulefile exports OLEX2_ROOT=$PREFIX (our own name; the vendor's OLEX2_DIR re-targets the updater and stays unset) and puts $PREFIX/bin on PATH.

Bundled glibc 2.39 sysroot (why and how)

Since 2026 OlexSys builds the Linux binaries on an Ubuntu 24.04-class toolchain. Scanning the archive's ELF files (2026-10-16):

file	needs
olex2c, olex2, unirun	GLIBC_2.38, GLIBCXX_3.4.32
NoSpherA2, lib/libopenblas.so.0, some cctbx_build/lib/*.so	GLIBC_2.38
RHEL9 provides	GLIBC_2.34, GLIBCXX_3.4.29
RHEL8 provides	GLIBC_2.28, GLIBCXX_3.4.25

A newer glibc can not be dropped in via LD_LIBRARY_PATH: the dynamic loader path (/lib64/ld-linux-x86-64.so.2) is hard-wired into each executable and must match its libc. So the recipe:

• ships under $PREFIX/sysroot (frozen Fedora 40 rpms from kojipkgs, Fedora's permanent build archive; same glibc generation as the upstream build host):

  package	why
  glibc 2.39, libstdc++/libgcc 14.2	GLIBC_2.38 / GLIBCXX_3.4.32
  readline 8.2	libreadline.so.8: RHEL8 has .so.7, vendor lib/ .so.6; host needs only libtinfo.so.6
  libwebp 1.3.2	libwebp.so.7 for PIL _imaging and libtiff (RHEL: only if the libwebp package is installed)
  libdeflate 1.19	needed by the vendor libtiff.so.5, absent on RHEL8
  libtiff 4.6 (+ libjpeg-turbo, jbigkit-libs, liblerc)	libtiff.so.6 for the GUI binary; RHEL8/9 ship .so.5

  The list comes from a scan of the NEEDED entries of every ELF object in the zip minus what the zip itself carries. Most packages unpack to usr/lib64, libgcc to /lib64 (Fedora relies on the filesystem package's lib64 -> usr/lib64 symlink, which we do not extract), so configure merges lib64 into usr/lib64 and recreates the symlink — one lib dir for the RPATH;

• runs patchelf --set-interpreter <sysroot ld.so> --force-rpath --set-rpath <sysroot libdir>:$PREFIX/lib:$PREFIX/ilib:$PREFIX/cctbx/cctbx_build/lib:$PREFIX:$ORIGIN on every dynamic executable (hart, hart_mpi are static and skipped). DT_RPATH, not RUNPATH, so that everything the embedded Python dlopen()s later resolves libc/libstdc++/openblas through the same list;

• leaves GTK2, X11, GL, fontconfig, libcrypt etc. to the host: libraries built against an older glibc run fine on a newer one, the reverse does not.

Rules that keep this safe:

• the sysroot lib dir is never exported in LD_LIBRARY_PATH (child processes — sh, shelxl, editors — would load the foreign libc and crash); only the patched executables see it, through their RPATH;
• LANG=LC_ALL=C.UTF-8 in the launcher: the bundled glibc has no locale archive, C.UTF-8 is built in;
• the originals are untouched inside the cached zip; a rebuild re-patches from scratch.

cctbx first run happens at build time

On its first start Olex2 regenerates cctbx/cctbx_build/libtbx_env and the libtbx dispatchers by running cctbx's configure.py into BaseDir (util/pyUtil/PyToolLib/path_utils.py, cold_start). On a per-user install that is invisible; on a shared read-only prefix it would fail for every user. So install runs olex2c once as the owner (step 9, isolated HOME), checks libtbx_env exists, then starts it a second time and refines the shipped sucrose sample (step 10), dying if cctbx is still reported broken. Expect in the first-run log, verbatim from upstream:

Using fable to convert scitbx/lbfgs.f
There is a problem with the cctbx: [Errno 2] No such file or directory: '.../scitbx/lbfgs/sdrive.f'

That is an upstream quirk: the vendor ships SCons in its site-packages, so libtbx believes it can build and the scitbx refresh wants Fortran sources that are not shipped. It happens after libtbx_env is pickled (env_config.py), which is why the second start is clean. Every Linux user of the vendor zip goes through the same first start.

Unverified until the first RHEL9 run: NSS lookups through the host's libnss_sss under the newer libc (expected fine, NSS ABI is stable), hart_mpi against the host MPI, and NoSpherA2 (it references Intel MKL kernel libraries libmkl_*.so.3 that are neither shipped nor on the hosts; whether they are hard requirements or optional dlopen targets is untested).

Why the binary archive and not a source build

The user-facing "open source" route is this archive: Olex2 is BSD and the official binaries are the open-source build. Compiling from https://github.com/pcxod/olex2 is not a practical module recipe: the repo builds only olex2 (GUI, wxWidgets mandatory) and unirun — there is no olex2c target in its CMakeLists/SConstruct/Makefile — the HTML GUI is in a separate SVN, and the matching cctbx/smtbx build is not published. Its own README says to take the official distribution and only swap in self-built olex2/unirun binaries. The Headless.zip/Batch.zip next to the archive on the download server are 2 KB plugin stubs, not builds.

Alternative considered: an Apptainer image (ubuntu:24.04 + gtk2/GL) that execs the host-side tree. Cleaner for the next rolling drop, but needs apptainer on every execution host and a privileged/fakeroot image build. Switch to it if the sysroot route shows locale/NSS/MPI trouble.

Building on Ra / RHEL9

module purge; module use unstable && module load modbuild/2.1.3
cd olex2
modbuild build --clean-install \
    --tmpdir=/das/work/p21/p21515/.cache/Pmodules \
    --distdir=/das/work/p21/p21515/.cache/Pmodules \
    1.5-7665
module use MX unstable && module load olex2/1.5-7665

Build host needs unzip, rpm2cpio (rpm) and cpio. Expect a silent 10–20 min for the copy of ~12 000 files onto /opt/psi; watch du -sh /opt/psi/MX/olex2/1.5-7665 from a second shell. The install step ends with olex2: <exe> -> interpreter .../sysroot/... lines and, if the build host lacks gtk2 etc., a WARNING list of host libs to install on the execution hosts.

If the download server is unreachable from the build host (plain https URLs, no login), fetch the files anywhere and pre-stage them under the exact names from files/config.yaml in <distdir>; prep then skips the download but still checks the sha256.

RHEL9 host libraries. The binaries bring their own glibc, libstdc++, readline, python, ssl, png, jpeg, tiff, openblas, gfortran, but link system GTK2/X11/OpenGL and libtinfo.so.6 (ncurses-libs) — also olex2c (wx is linked in statically). The install step prints the loader listing for olex2c; a host lib missing on the build node is a WARNING there, not an error, because build and execution hosts differ. Check once per host type:

module load olex2/1.5-7665          # exports OLEX2_ROOT=/opt/psi/MX/olex2/1.5-7665
$OLEX2_ROOT/sysroot/usr/lib64/ld-linux-x86-64.so.2 --list $OLEX2_ROOT/olex2c | grep -E 'not found|libc.so.6|libstdc'

libc.so.6/libstdc++.so.6 must resolve under $OLEX2_ROOT/sysroot; every not found line is a host package to dnf install. From the NEEDED scan, the host must provide (all standard RHEL8/9 packages): gtk2, gdk-pixbuf2, pango, cairo, glib2, fontconfig, freetype, libX11, libSM, libxcb, libxkbcommon, libwayland-client, ncurses-libs (libtinfo), libgomp (cctbx refinement!), libgfortran + libquadmath (openblas/scipy), zlib, xz-libs, libzstd, libuuid, libxcrypt; GUI additionally mesa-libGL, mesa-libGLU. Plain ldd on the raw binary also works after the patch (it honours the new interpreter).

Verifying the install

which olex2c            # -> $OLEX2_ROOT/bin/olex2c  (= /opt/psi/MX/olex2/1.5-7665/bin/olex2c)
which olex2             # -> $OLEX2_ROOT/bin/olex2
cat $OLEX2_ROOT/version.txt   # SVN Revision No. 7665

# console: starts, loads a shipped sample, refines, exits (writes to ~/.olex2
# and to a scratch copy of the sample - run it in a scratch dir)
cd $(mktemp -d) && cp $OLEX2_ROOT/sample_data/sucrose/* .
olex2c                  # interactive prompt; then type:
#   reap sucrose.res
#   refine 4
#   exit

# GUI (needs X / x2go / NoMachine session):
olex2

Scripting olex2c: it reads commands from stdin, so a batch job can do printf 'reap %s\nrefine 10\nexit\n' x.ins | olex2c (to be confirmed on the first RHEL9 test — the Windows olex2c.exe is used this way by CrysAlisPro and the upstream NoSpherA2 release tests). The headless engine loads macrox.xld instead of macro.xld; GUI-only python calls print "not available in Headless Olex" rather than fail.

First-start notes: Olex2 keeps user data in ~/.olex2 (DataDir), copies sample data there and may try to delete runonce.* markers in BaseDir (read-only for users — harmless). The vendor start and unirun remain under $PREFIX but are intentionally not on PATH; do not run them against the shared install.

Adding a new version

1.5/olex2-linux64.zip is a rolling file — OlexSys replaces it in place (check Modified on https://secure.olex2.org/olex2-distro/1.5/, or the ALPHA/BETA series folders). The sha256 pinned in files/config.yaml will therefore fail against a newer drop; that is the signal to add a version, not to edit the hash:

1. download the current zip and read its revision and hash: unzip -p olex2-linux64.zip olex2/version.txt → e.g. SVN Revision No. 7800, sha256sum olex2-linux64.zip;
2. check the new binaries' glibc need still fits the bundled 2.39: unzip -p olex2-linux64.zip olex2/olex2c | strings | grep -o 'GLIBC_2\.[0-9]*' | sort -uV | tail -1 (and GLIBCXX_3\.4\.[0-9]* ≤ 3.4.33). If upstream moved past that, bump all sysroot rpms in urls:/shasums: to a newer frozen Fedora release — they must come from the same release so their mutual glibc requirements are met. kojipkgs has no browsable index (bot gate) but serves any packages/<src>/<ver>/<rel>.fcNN/x86_64/<nvr>.x86_64.rpm; find the NVR via the Koji XML-RPC hub (listTagged, tag fNN, latest=1) and note that the source name can differ from the lib name (liblerc, jbigkit → jbigkit-libs);
3. add 1.5-7800 under versions: (relstage: unstable) and olex2-1.5-7800-linux64.zip: <sha256> under shasums: — the urls: entry needs no change (${V_PKG} = series dir, ${V} = cache name); for a 1.6 series just use 1.6-<rev> and the URL follows;
4. build as above; promote to stable only after olex2c and olex2 have been exercised on the target host.

History: 1.5 (stable) is the older Ra/RHEL8 module built by the previous recipe from a hand-downloaded olex2-linux64.zip placed next to build (never committed, *.zip is gitignored); it is kept in config.yaml so the installed module stays listed. Do not rebuild it with this recipe.