MX_Pmodule/README.md

# MX

All MX Pmodules in use at PSI. One top-level directory per module, holding
exactly 4 files:

```
<Module>/
  build              # #!/usr/bin/env modbuild — bash, overrides pbuild::* hooks
  modulefile         # #%Module1.0 — Tcl, runtime env
  files/config.yaml  # format:1 — group/relstage/versions/deps/urls
  README.md          # description + module-specific pitfalls
```

Modules build on the **Ra** cluster (RHEL8) into `/opt/psi/MX/<name>/<version>`.
Upstream docs: https://github.com/Pmodules/Pmodules/wiki

## Building on Ra

```bash
module use unstable && module load modbuild/2.1.3
cd <Module>
modbuild build <version>
# verify it loads:
module use MX unstable && module load <Module>/<version>
```

- There is **no `build` subcommand**: usage is `modbuild [BUILD_SCRIPT] [options]
  <version>` — `build` above is the script's filename, so run inside the module dir.
- Flags: `--prep|--configure|--compile|--install|--all` (run *up to* that step),
  `--clean-install` (remove existing module first), `-f`/`--force-rebuild`, `-j N`,
  `--tmpdir=DIR`, `--distdir=DIR`, `-v`, `--debug`.
  There is **no `--dry-run` and no `--check-mode`**.
- Big downloads/builds: the defaults `/var/tmp/$USER` (tmp) and
  `~/.cache/Pmodules/distfiles` (downloads) are too small — put both on /das,
  **adapting the p-group path to your own**:

  ```bash
  modbuild build \
      --tmpdir=/das/work/p21/p21515/.cache/Pmodules \
      --distdir=/das/work/p21/p21515/.cache/Pmodules \
      --clean-install -v --debug <version>
  ```

  `curl: (23) Failure writing output to destination` during a download means
  quota/disk full at the target dir, not a network error.

## Tips & tricks

1. Module names are `name/version`, never `name/name-version`.
2. New versions start `relstage: unstable`; promote to `stable` only after the
   module builds AND loads. A stable module may only depend on stable modules.
3. **Contaminated environment**: previously loaded modules can leak env vars
   into the build and make it fail in odd ways. Fix:
   `module purge && module load modbuild/2.1.3`, then build again.
4. **"done" does not mean success**: modbuild has no errexit in build hooks —
   failed commands scroll by and the build still finishes. Guard critical steps
   with `|| std::die 42 "message"` and end `install` by checking the file your
   modulefile needs actually exists.
5. Hooks run with `pipefail` + `nounset`: `yes | installer` dies via SIGPIPE —
   use `printf 'y\ny\n' | installer`.
6. prep reuses any file already in the distfiles dir with **no integrity
   check** — an interrupted download leaves a partial tarball that later fails
   with `gzip: unexpected end of file`. Either `rm` it and re-run, pin
   `shasums:`, or use a custom prep with `wget --continue` + `gzip -t`
   (see `ccp4/build`).
7. Unbound-variable on load (cctbx/phenix-style env scripts): add
   `puts stdout "set +x nounset"` before the `source` line in the modulefile.
8. Do not use `pbuild::add_to_group`, `use_cmake`, `use_autotools`,
   `set_download_url` — removed, they `std::die`. Use `group:`, `urls:`,
   `configure_with:` in config.yaml instead.
9. `/opt/psi` is shared storage: untarring/installing GBs runs 10–30+ min with
   no output. Watch `du -sh /opt/psi/MX/<name>/<version>` from a second shell
   instead of killing a healthy build.
10. Check the per-module README before building — module-specific pitfalls are
    documented there (`ccp4/README.md` is the most battle-tested example).

## Building with an AI agent

The repo ships the **new-module** skill at `.claude/skills/new-module/`
(`SKILL.md` = workflow + decision tree, `REFERENCE.md` = full schema, hooks and
copy-paste templates). It scaffolds the 4 files for a new module from the right
install-type archetype and prints the Ra build recipe.

- **Claude Code**: opens the repo → skill is auto-discovered. Type
  `/new-module <tool, version, download URL, special requirements>` or simply
  describe the task ("package X as a module") and it triggers.
- **Other agents** (Cursor, Codex, generic LLM chat): the skill is plain
  Markdown — point the agent at `.claude/skills/new-module/SKILL.md` first,
  then `REFERENCE.md`, and ask it to follow them.
- The skill copies from real modules in this repo as ground truth (`xds`,
  `careless`, `DIALS`, `ccp4`, ...) — keep those exemplary, and fold new
  lessons back into the skill files so the next build starts smarter.

## Pmodules background (from the wiki)

- Modules are organized in hierarchical groups; everything here is in the `MX`
  group (`module use MX`).
- Filesystem layout: modulefile at `$PREFIX/GROUP/modulefiles/NAME/VERSION`,
  installation at `$PREFIX/GROUP/NAME/VERSION`, per-version config at
  `$PREFIX/GROUP/modulefiles/NAME/.config-VERSION` (YAML).
- Release stages: **unstable**, **stable**, **deprecated**. New builds land in
  unstable; opt in with `module use unstable`.
- Pmodules automatically sets `$NAME_DIR`, `$NAME_HOME`, `$NAME_PREFIX`,
  `$NAME_VERSION` on load and prepends standard directories (`$PREFIX/bin` to
  `PATH`, ...) if they exist — no explicit `setenv`/`prepend-path` needed for
  those.

## Permissions

Members of the `unx-mx_adm` group can update `/opt/psi/MX` directly — no -adm
account needed. Permission problems: tell the admins.

## Tutorial branches (build-*)

Each participant works on their own `build-<module>` branch (e.g.
`build-dials`), all starting from `main`.

While a branch is still identical to main (fast-forward):

```bash
git push origin main:build-dials main:build-buster ...
for b in dials buster ...; do git branch -f build-$b main; done
```

Once branches have diverged, merge main into each instead:

```bash
for b in dials buster ...; do
  git checkout build-$b && git merge main && git push
done
git checkout main
```

If `git push origin main:build-xxx` fails with "non-fast-forward", the branch
has its own commits: use the merge loop, never force-push over other people's
work.