MX_Pmodule/README.md

MX

All MX Pmodules in use at PSI. One top-level directory per module, holding exactly 4 files:

<Module>/
  build              # #!/usr/bin/env modbuild — bash, overrides pbuild::* hooks
  modulefile         # #%Module1.0 — Tcl, runtime env
  files/config.yaml  # format:1 — group/relstage/versions/deps/urls
  README.md          # description + module-specific pitfalls

Modules build on the Ra cluster (RHEL8) into /opt/psi/MX/<name>/<version>. Upstream docs: https://github.com/Pmodules/Pmodules/wiki

Building on Ra

module use unstable && module load modbuild/2.1.3
cd <Module>
modbuild build <version>
# verify it loads:
module use MX unstable && module load <Module>/<version>

• There is no build subcommand: usage is modbuild [BUILD_SCRIPT] [options] <version> — build above is the script's filename, so run inside the module dir.

• Flags: --prep|--configure|--compile|--install|--all (run up to that step), --clean-install (remove existing module first), -f/--force-rebuild, -j N, --tmpdir=DIR, --distdir=DIR, -v, --debug. There is no --dry-run and no --check-mode.

• Big downloads/builds: the defaults /var/tmp/$USER (tmp) and ~/.cache/Pmodules/distfiles (downloads) are too small — put both on /das, adapting the p-group path to your own:

  modbuild build \
      --tmpdir=/das/work/p21/p21515/.cache/Pmodules \
      --distdir=/das/work/p21/p21515/.cache/Pmodules \
      --clean-install -v --debug <version>
  

  curl: (23) Failure writing output to destination during a download means quota/disk full at the target dir, not a network error.

Tips & tricks

1. Module names are name/version, never name/name-version.
2. New versions start relstage: unstable; promote to stable only after the module builds AND loads. A stable module may only depend on stable modules.
3. Contaminated environment: previously loaded modules can leak env vars into the build and make it fail in odd ways. Fix: module purge && module load modbuild/2.1.3, then build again.
4. "done" does not mean success: modbuild has no errexit in build hooks — failed commands scroll by and the build still finishes. Guard critical steps with || std::die 42 "message" and end install by checking the file your modulefile needs actually exists.
5. Hooks run with pipefail + nounset: yes | installer dies via SIGPIPE — use printf 'y\ny\n' | installer.
6. prep reuses any file already in the distfiles dir with no integrity check — an interrupted download leaves a partial tarball that later fails with gzip: unexpected end of file. Either rm it and re-run, pin shasums:, or use a custom prep with wget --continue + gzip -t (see ccp4/build).
7. Unbound-variable on load (cctbx/phenix-style env scripts): add puts stdout "set +x nounset" before the source line in the modulefile.
8. Do not use pbuild::add_to_group, use_cmake, use_autotools, set_download_url — removed, they std::die. Use group:, urls:, configure_with: in config.yaml instead.
9. /opt/psi is shared storage: untarring/installing GBs runs 10–30+ min with no output. Watch du -sh /opt/psi/MX/<name>/<version> from a second shell instead of killing a healthy build.
10. Check the per-module README before building — module-specific pitfalls are documented there (ccp4/README.md is the most battle-tested example).

Building with an AI agent

The repo ships the new-module skill at .claude/skills/new-module/ (SKILL.md = workflow + decision tree, REFERENCE.md = full schema, hooks and copy-paste templates). It scaffolds the 4 files for a new module from the right install-type archetype and prints the Ra build recipe.

• Claude Code: opens the repo → skill is auto-discovered. Type /new-module <tool, version, download URL, special requirements> or simply describe the task ("package X as a module") and it triggers.
• Other agents (Cursor, Codex, generic LLM chat): the skill is plain Markdown — point the agent at .claude/skills/new-module/SKILL.md first, then REFERENCE.md, and ask it to follow them.
• The skill copies from real modules in this repo as ground truth (xds, careless, DIALS, ccp4, ...) — keep those exemplary, and fold new lessons back into the skill files so the next build starts smarter.

Pmodules background (from the wiki)

• Modules are organized in hierarchical groups; everything here is in the MX group (module use MX).
• Filesystem layout: modulefile at $PREFIX/GROUP/modulefiles/NAME/VERSION, installation at $PREFIX/GROUP/NAME/VERSION, per-version config at $PREFIX/GROUP/modulefiles/NAME/.config-VERSION (YAML).
• Release stages: unstable, stable, deprecated. New builds land in unstable; opt in with module use unstable.
• Pmodules automatically sets $NAME_DIR, $NAME_HOME, $NAME_PREFIX, $NAME_VERSION on load and prepends standard directories ($PREFIX/bin to PATH, ...) if they exist — no explicit setenv/prepend-path needed for those.

Permissions

Members of the unx-mx_adm group can update /opt/psi/MX directly — no -adm account needed. Permission problems: tell the admins.

Tutorial branches (build-*)

Each participant works on their own build-<module> branch (e.g. build-dials), all starting from main.

While a branch is still identical to main (fast-forward):

git push origin main:build-dials main:build-buster ...
for b in dials buster ...; do git branch -f build-$b main; done

Once branches have diverged, merge main into each instead:

for b in dials buster ...; do
  git checkout build-$b && git merge main && git push
done
git checkout main

If git push origin main:build-xxx fails with "non-fast-forward", the branch has its own commits: use the merge loop, never force-push over other people's work.