207 lines
9.1 KiB
Markdown
207 lines
9.1 KiB
Markdown
# MX
|
|
|
|
All MX Pmodules in use at PSI. One top-level directory per module, holding
|
|
exactly 4 files:
|
|
|
|
```
|
|
<Module>/
|
|
build # #!/usr/bin/env modbuild — bash, overrides pbuild::* hooks
|
|
modulefile # #%Module1.0 — Tcl, runtime env
|
|
files/config.yaml # format:1 — group/relstage/versions/deps/urls
|
|
README.md # description + module-specific pitfalls
|
|
```
|
|
|
|
Modules build on the **Ra** cluster (RHEL8) into `/opt/psi/MX/<name>/<version>`.
|
|
Upstream docs: https://github.com/Pmodules/Pmodules/wiki
|
|
|
|
## Getting the repo
|
|
|
|
Clone into your work folder on Ra (once). SSH access to Gitea is recommended:
|
|
generate a key ([how-to](https://git-scm.com/book/en/v2/Git-on-the-Server-Generating-Your-SSH-Public-Key))
|
|
and add the public key at https://gitea.psi.ch/user/settings/keys.
|
|
|
|
```bash
|
|
cd /das/work/p21/p21515/ # adapt the p-group path to your own
|
|
git clone git@gitea.psi.ch:mx/MX_Pmodule.git
|
|
cd /das/work/p21/p21515/MX_Pmodule
|
|
```
|
|
|
|
Work on your own branch, not on main:
|
|
|
|
```bash
|
|
git switch main && git pull # start from up-to-date main
|
|
git switch -c build-<module> # e.g. build-dials
|
|
git push -u origin build-<module>
|
|
```
|
|
|
|
New to git? Start here: https://git-scm.com/doc
|
|
|
|
## Building on Ra - DO NOT write into /var/tmp !
|
|
|
|
`<Module>` below is one of the top-level directories of the cloned repo
|
|
(e.g. `/das/work/p21/p21515/MX_Pmodule/ccp4`):
|
|
|
|
> **!!!WARNING!!!: never write big files to `/var/tmp` — it is small and shared with
|
|
everyone on the login node. Please, please do not fill /var/tmp.**
|
|
|
|
Always pass `--tmpdir` and `--distdir` pointing to your p-group work folder —
|
|
the defaults are `/var/tmp/$USER` (see warning above) and
|
|
`~/.cache/Pmodules/distfiles` (home quota is too small). Adapt the p-group
|
|
path (`p21/p21515`) to your own:
|
|
|
|
```bash
|
|
module use unstable && module load modbuild/2.1.3
|
|
cd /das/work/p21/p21515/MX_Pmodule/<Module>
|
|
modbuild build \
|
|
--tmpdir=/das/work/p21/p21515/.cache/Pmodules \
|
|
--distdir=/das/work/p21/p21515/.cache/Pmodules \
|
|
<version>
|
|
# verify it loads:
|
|
module use MX unstable && module load <Module>/<version>
|
|
```
|
|
|
|
If you already built without the flags, clean up what landed in the defaults:
|
|
|
|
```bash
|
|
rm -rf /var/tmp/$USER/<Module>-<version>
|
|
rm -rf ~/.cache/Pmodules/distfiles
|
|
```
|
|
|
|
- There is **no `build` subcommand**: usage is `modbuild [BUILD_SCRIPT] [options]
|
|
<version>` — `build` above is the script's filename, so run inside the module dir.
|
|
- More flags: `--prep|--configure|--compile|--install|--all` (run *up to* that
|
|
step), `--clean-install` (remove existing module first), `-f`/`--force-rebuild`,
|
|
`-j N`, `-v`, `--debug` (make the otherwise silent long steps visible).
|
|
- `curl: (23) Failure writing output to destination` during a download means
|
|
quota/disk full at the target dir, not a network error.
|
|
|
|
## Tips & tricks
|
|
|
|
1. Module names are `name/version`, never `name/name-version`.
|
|
2. New versions start `relstage: unstable`; promote to `stable` only after the
|
|
module builds AND loads. A stable module may only depend on stable modules.
|
|
3. **Contaminated environment**: previously loaded modules can leak env vars
|
|
into the build and make it fail in odd ways. Fix:
|
|
`module purge && module load modbuild/2.1.3`, then build again.
|
|
4. Hooks run with `pipefail` + `nounset`: `yes | installer` dies via SIGPIPE —
|
|
use `printf 'y\ny\n' | installer`.
|
|
5. Module-specific pitfalls live in the per-module README — read it before
|
|
building. `ccp4/README.md` is the most battle-tested and also documents the
|
|
common modbuild traps ("done" without success — no errexit in hooks,
|
|
partial downloads reused silently, slow installs on shared /opt/psi).
|
|
|
|
## Debugging shared-library errors (lib vs lib64, RPATH)
|
|
|
|
Compiled modules often build fine and then fail at load/run time with
|
|
`cannot open shared object file` or `GLIBCXX_x.y.z not found`. Diagnose
|
|
with binutils (works on any binary or `.so`, e.g. `/bin/cmake`):
|
|
|
|
```bash
|
|
objdump -p <binary-or-.so> | grep NEEDED # which libs it links
|
|
objdump -p <binary-or-.so> | grep -E 'RPATH|RUNPATH' # usually EMPTY on RHEL
|
|
ldd <binary-or-.so> # how each resolves; look for "not found"
|
|
```
|
|
|
|
The dynamic linker searches in this order: `DT_RPATH` (if set) →
|
|
`LD_LIBRARY_PATH` → `DT_RUNPATH` (if set) → `/etc/ld.so.cache` →
|
|
`/lib64`, `/usr/lib64`. On RHEL, RPATH/RUNPATH are normally **not set**,
|
|
and the build-time `-L` paths are not recorded in the binary — so build
|
|
time and runtime resolve libraries completely independently.
|
|
|
|
Situations where the default order is not what you expect:
|
|
|
|
1. **Compiled with a newer-gcc module** → the binary needs a newer
|
|
libstdc++, but at runtime the old system `/usr/lib64/libstdc++.so.6`
|
|
resolves first (no RPATH) → `GLIBCXX_... not found`. Fix: add the gcc
|
|
module to `runtime_deps:` so its lib dir is prepended to
|
|
`LD_LIBRARY_PATH` (see `pymol-open-source/files/config.yaml`).
|
|
2. **lib vs lib64 install dir**: system 64-bit libs live in `/usr/lib64`,
|
|
but local installs may land in `$PREFIX/lib` (autotools default, venvs)
|
|
or `$PREFIX/lib64` (CMake GNUInstallDirs picks lib64 on RHEL). Check
|
|
which dir the install actually produced and prepend that one:
|
|
`prepend-path LD_LIBRARY_PATH $PREFIX/lib` (see `ImageMagick/modulefile`).
|
|
3. **Vendor binary WITH RPATH baked in**: `DT_RPATH` beats
|
|
`LD_LIBRARY_PATH`, so the modulefile cannot override it — patchelf or
|
|
rebuild are the only fixes. With `DT_RUNPATH` instead, it is the other
|
|
way round: `LD_LIBRARY_PATH` wins, so a leftover loaded module can
|
|
shadow the vendor's bundled libs (another contaminated-environment
|
|
source — `module purge` first).
|
|
4. Alternative to LD_LIBRARY_PATH in the modulefile: bake the path at
|
|
build time with `LDFLAGS="-Wl,-rpath,$PREFIX/lib"`.
|
|
|
|
After install, `ldd` the main binary and any compiled Python extension
|
|
(e.g. pymol's `_cmd*.so`) — no line may say "not found"
|
|
(see `adxv/README.md`).
|
|
|
|
## Building with an AI agent
|
|
|
|
The repo ships the **new-module** skill at `.claude/skills/new-module/`
|
|
(`SKILL.md` = workflow + decision tree, `REFERENCE.md` = full schema, hooks and
|
|
copy-paste templates). It scaffolds the 4 files for a new module from the right
|
|
install-type archetype and prints the Ra build recipe.
|
|
|
|
- **Claude Code**: opens the repo → skill is auto-discovered. Type
|
|
`/new-module <tool, version, download URL, special requirements>` or simply
|
|
describe the task ("package X as a module") and it triggers.
|
|
- **Other agents** (Cursor, Codex, generic LLM chat): the skill is plain
|
|
Markdown — point the agent at `.claude/skills/new-module/SKILL.md` first,
|
|
then `REFERENCE.md`, and ask it to follow them.
|
|
- **New software (or a known tool that moved to a new download site)**: give
|
|
the agent the software's installation/download page as part of the context —
|
|
the skill knows the repo's patterns, not vendor-specific install procedures.
|
|
- **The skill can be buggy** — use it with caution and double-check what it
|
|
scaffolds before building; corrections are welcome (edit
|
|
`.claude/skills/new-module/` and commit). Reading along while it works is
|
|
also a good way to learn how these modules are put together.
|
|
- The skill copies from real modules in this repo as ground truth (`xds`,
|
|
`careless`, `DIALS`, `ccp4`, ...) — keep those exemplary, and fold new
|
|
lessons back into the skill files so the next build starts smarter.
|
|
|
|
## Pmodules background (from the wiki)
|
|
|
|
- Modules are organized in hierarchical groups; everything here is in the `MX`
|
|
group (`module use MX`).
|
|
- Filesystem layout: modulefile at `$PREFIX/GROUP/modulefiles/NAME/VERSION`,
|
|
installation at `$PREFIX/GROUP/NAME/VERSION`, per-version config at
|
|
`$PREFIX/GROUP/modulefiles/NAME/.config-VERSION` (YAML).
|
|
- Release stages: **unstable**, **stable**, **deprecated**. New builds land in
|
|
unstable; opt in with `module use unstable`.
|
|
- Pmodules automatically sets `$NAME_DIR`, `$NAME_HOME`, `$NAME_PREFIX`,
|
|
`$NAME_VERSION` on load and prepends standard directories (`$PREFIX/bin` to
|
|
`PATH`, ...) if they exist — no explicit `setenv`/`prepend-path` needed for
|
|
those.
|
|
|
|
## Permissions
|
|
|
|
Members of the `unx-mx_adm` group can update `/opt/psi/MX` directly — no -adm
|
|
account needed. Permission problems: Ask May/Meitian/Dawn to be added to
|
|
unx-mx_adm group. If you have been in the group for more than 48 hours and
|
|
still can't write to /opt/psi/MX, ask the SLS beamline storage and compute
|
|
admins (Email: sls-htc-admins@lists.psi.ch).
|
|
|
|
## Resources
|
|
|
|
- Buildblock examples: https://github.com/Pmodules/buildblocks/tree/master
|
|
- modbuild documentation: https://pmodules.github.io/modbuild/
|
|
- Environment Modules: https://envmodules.io/
|
|
|
|
If you want to provide more, feel free to submit a pull request to this document.
|
|
|
|
## Acknowledgements
|
|
|
|
Many thanks to [Achim Gsell](https://www.psi.ch/en/lsm/people/achim-gsell)
|
|
for giving the MX group the tutorial on 2026-08-18, and to
|
|
[Hans Viessmann](https://www.psi.ch/en/awi/people/hans-nikolai-viessmann)
|
|
for the support.
|
|
|
|
During the tutorial the following people worked on the modules:
|
|
|
|
- May — Phenix
|
|
- Filipa — Pymol
|
|
- Emma — CrystFEL
|
|
- Dennis — ccp4
|
|
- Milton — Xtrapol8
|
|
|
|
Thanks to all of them for their participation and contributions to the
|
|
scripts. As of 2026-08-19, all build* branches have been merged to main.
|