mx/MX_Pmodule
1
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Docs overhaul: quickstart, current commands everywhere, credits

Browse Source 
Main README: clone/SSH/branching quickstart with full /das paths,
--tmpdir/--distdir as part of the standard build command (do not fill
/var/tmp or $HOME) with cleanup for defaults, tips deduplicated against
module READMEs, old-system references removed, resource links and
tutorial acknowledgements added.

Module READMEs: purge outdated recipes repo-wide — Pmodules/1.1.20 and
2.0.0 plus ./build replaced by modbuild/2.1.3 invocations; the cctbx
unbound-variable fix moved from the main README into phenix and DIALS
READMEs where it belongs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
duan_j
2026-08-18 22:18:37 +02:00
co-authored by Claude Fable 5
parent 3b0d69d326
commit c3c38856b3
12 changed files with 96 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DIALS/README.md
+7 -3
View File
@@ -2,11 +2,11 @@
Step 1: make sure you are in the `unx-mx_adm` group -- no -adm account needed, members can update the modules in `/opt/psi/MX` directly (if you have problems, please let the admins know)
Step 2: load the latest Pmodules package
Step 2: load the build tool (modbuild)
```
module use unstable
module load Pmodules/2.0.0
module load modbuild/2.1.3
```
Step 3: go into DIALS directory, update config.yaml to add newest release version and build module with that release name
@@ -14,7 +14,7 @@ Step 3: go into DIALS directory, update config.yaml to add newest release versio
```
cd DIALS
vi files/config.yaml
./build 3.25.0
modbuild build 3.25.0
```
Step 4: confirm that the module loads as expected
@@ -23,3 +23,7 @@ PS:
Unsupported dtype issue is resolved by changing modules/dxtbx/src/dxtbx/format/FormatNXmx.py
allowing the return of also /entry/data/data to the handle:
https://github.com/dials/dials/issues/3033
If loading the module fails with an `unbound variable` from the sourced
`dials_env.sh` (cctbx-style env script), add `puts stdout "set +x nounset"`
before the `source` line in the modulefile — same fix as phenix.
README.md
+64 -38
View File
@@ -14,34 +14,62 @@ exactly 4 files:
Modules build on the **Ra** cluster (RHEL8) into `/opt/psi/MX/<name>/<version>`.
Upstream docs: https://github.com/Pmodules/Pmodules/wiki
## Getting the repo
Clone into your work folder on Ra (once). SSH access to Gitea is recommended:
generate a key ([how-to](https://git-scm.com/book/en/v2/Git-on-the-Server-Generating-Your-SSH-Public-Key))
and add the public key at https://gitea.psi.ch/user/settings/keys.
```bash
cd /das/work/p21/p21515/ # adapt the p-group path to your own
git clone git@gitea.psi.ch:mx/MX_Pmodule.git
cd /das/work/p21/p21515/MX_Pmodule
```
Work on your own branch, not on main:
```bash
git switch main && git pull # start from up-to-date main
git switch -c build-<module> # e.g. build-dials
git push -u origin build-<module>
```
New to git? Start here: https://git-scm.com/doc
## Building on Ra
`<Module>` below is one of the top-level directories of the cloned repo
(e.g. `/das/work/p21/p21515/MX_Pmodule/ccp4`):
Always pass `--tmpdir` and `--distdir` pointing to your p-group work folder —
the defaults are `/var/tmp/$USER` (shared with everyone on the login node —
**do NOT fill it**) and `~/.cache/Pmodules/distfiles` (home quota is too
small). Adapt the p-group path (`p21/p21515`) to your own:
```bash
module use unstable && module load modbuild/2.1.3
cd <Module>
modbuild build <version>
cd /das/work/p21/p21515/MX_Pmodule/<Module>
modbuild build \
--tmpdir=/das/work/p21/p21515/.cache/Pmodules \
--distdir=/das/work/p21/p21515/.cache/Pmodules \
<version>
# verify it loads:
module use MX unstable && module load <Module>/<version>
```
If you already built without the flags, clean up what landed in the defaults:
```bash
rm -rf /var/tmp/$USER/<Module>-<version>
rm -rf ~/.cache/Pmodules/distfiles
```
- There is **no `build` subcommand**: usage is `modbuild [BUILD_SCRIPT] [options]
<version>` — `build` above is the script's filename, so run inside the module dir.
- Flags: `--prep|--configure|--compile|--install|--all` (run *up to* that step),
`--clean-install` (remove existing module first), `-f`/`--force-rebuild`, `-j N`,
`--tmpdir=DIR`, `--distdir=DIR`, `-v`, `--debug`.
There is **no `--dry-run` and no `--check-mode`**.
- Big downloads/builds: the defaults `/var/tmp/$USER` (tmp) and
`~/.cache/Pmodules/distfiles` (downloads) are too small — put both on /das,
**adapting the p-group path to your own**:
```bash
modbuild build \
--tmpdir=/das/work/p21/p21515/.cache/Pmodules \
--distdir=/das/work/p21/p21515/.cache/Pmodules \
--clean-install -v --debug <version>
```
`curl: (23) Failure writing output to destination` during a download means
- More flags: `--prep|--configure|--compile|--install|--all` (run *up to* that
step), `--clean-install` (remove existing module first), `-f`/`--force-rebuild`,
`-j N`, `-v`, `--debug` (make the otherwise silent long steps visible).
- `curl: (23) Failure writing output to destination` during a download means
quota/disk full at the target dir, not a network error.
## Tips & tricks
@@ -52,27 +80,12 @@ module use MX unstable && module load <Module>/<version>
3. **Contaminated environment**: previously loaded modules can leak env vars
into the build and make it fail in odd ways. Fix:
`module purge && module load modbuild/2.1.3`, then build again.
4. **"done" does not mean success**: modbuild has no errexit in build hooks —
failed commands scroll by and the build still finishes. Guard critical steps
with `|| std::die 42 "message"` and end `install` by checking the file your
modulefile needs actually exists.
5. Hooks run with `pipefail` + `nounset`: `yes | installer` dies via SIGPIPE —
4. Hooks run with `pipefail` + `nounset`: `yes | installer` dies via SIGPIPE —
use `printf 'y\ny\n' | installer`.
6. prep reuses any file already in the distfiles dir with **no integrity
check** — an interrupted download leaves a partial tarball that later fails
with `gzip: unexpected end of file`. Either `rm` it and re-run, pin
`shasums:`, or use a custom prep with `wget --continue` + `gzip -t`
(see `ccp4/build`).
7. Unbound-variable on load (cctbx/phenix-style env scripts): add
`puts stdout "set +x nounset"` before the `source` line in the modulefile.
8. Do not use `pbuild::add_to_group`, `use_cmake`, `use_autotools`,
`set_download_url` — removed, they `std::die`. Use `group:`, `urls:`,
`configure_with:` in config.yaml instead.
9. `/opt/psi` is shared storage: untarring/installing GBs runs 10–30+ min with
no output. Watch `du -sh /opt/psi/MX/<name>/<version>` from a second shell
instead of killing a healthy build.
10. Check the per-module README before building — module-specific pitfalls are
documented there (`ccp4/README.md` is the most battle-tested example).
5. Module-specific pitfalls live in the per-module README — read it before
building. `ccp4/README.md` is the most battle-tested and also documents the
common modbuild traps ("done" without success — no errexit in hooks,
partial downloads reused silently, slow installs on shared /opt/psi).
## Building with an AI agent
@@ -134,3 +147,16 @@ git checkout main
If `git push origin main:build-xxx` fails with "non-fast-forward", the branch
has its own commits: use the merge loop, never force-push over other people's
work.
## Resources
- Buildblock examples: https://github.com/Pmodules/buildblocks/tree/master
- modbuild documentation: https://pmodules.github.io/modbuild/
- Environment Modules: https://envmodules.io/
## Acknowledgements
Many thanks to [Achim Gsell](https://www.psi.ch/en/lsm/people/achim-gsell)
for giving the MX group the tutorial on 2026-08-18, and to
[Hans Viessmann](https://www.psi.ch/en/awi/people/hans-nikolai-viessmann)
and Achim for their support.
Xtrapol8/README.md
+1 -1
View File
@@ -21,7 +21,7 @@ versions (tagged `v1.x.y`) would need a per-version `urls:` override.
## Building on Ra
```sh
module use unstable && module load modbuild/2.1.2
module use unstable && module load modbuild/2.1.3
cd Xtrapol8
modbuild build 1.2.9
module use MX unstable && module load Xtrapol8/1.2.9
alphafold/README.md
+1 -1
View File
@@ -97,7 +97,7 @@ Add version to files/variants. The version number should match a github tag
As a member of the `unx-mx_adm` group (no -adm account needed; if you have problems, please let the admins know):
```
cd alphafold
./build <version>
modbuild build <version>
```
## Testing
autoPROC/README.md
+3 -3
View File
@@ -10,11 +10,11 @@ dependencies.
Step 1: make sure you are in the `unx-mx_adm` group -- no -adm account needed, members can update the modules in `/opt/psi/MX` directly (if you have problems, please let the admins know)
Step 2: load the latest Pmodules package
Step 2: load the build tool (modbuild)
```
module use unstable
module load Pmodules/2.0.0
module load modbuild/2.1.3
```
Step 3: stage the licence-gated snapshot. Download it from Global Phasing (needs
@@ -38,7 +38,7 @@ vi files/config.yaml
Step 5: build (installs to `/opt/psi/MX/autoPROC/<date>`)
```
./build 20240710
modbuild build 20240710
```
Step 6: confirm the module loads and autoPROC is licensed
careless/README.md
+1 -1
View File
@@ -58,4 +58,4 @@ careless mono dHKL <file.mtz> /tmp/careless_test --iterations 10
## Adding a new version
Add the version under `versions:` in `files/config.yaml`, then on a Ra
login node: `modbuild build <version>` (modbuild/2.1.2).
login node: `modbuild build <version>` (modbuild/2.1.3).
crystfel/README.md
+2 -2
View File
@@ -24,7 +24,7 @@ Loading the module puts `$PREFIX/bin` on `PATH`.
## Building on Ra
```sh
module use unstable && module load modbuild/2.1.2
module use unstable && module load modbuild/2.1.3
cd crystfel
modbuild build 0.13.0
module use MX unstable && module load crystfel/0.13.0 && indexamajig --version
@@ -43,4 +43,4 @@ cell_explorer # GUI check, needs X
Add the version under `versions:` and its tarball sha256 under `shasums:`
in `files/config.yaml`, then on a Ra login node:
`modbuild build <version>` (modbuild/2.1.2).
`modbuild build <version>` (modbuild/2.1.3).
jfjoch_viewer/README.md
+3 -3
View File
@@ -2,11 +2,11 @@
Step 1: make sure you are in the `unx-mx_adm` group -- no -adm account needed, members can update the modules in `/opt/psi/MX` directly (if you have problems, please let the admins know)
Step 2: load the latest Pmodules package
Step 2: load the build tool (modbuild)
```
module use unstable
module load Pmodules/1.1.20
module load modbuild/2.1.3
```
Step 3: go into jfjoch_viewer directory, update config.yaml to add newest release version and build module with that release name
@@ -14,7 +14,7 @@ Step 3: go into jfjoch_viewer directory, update config.yaml to add newest releas
```
cd jfjoch_viewer
vi files/config.yaml
./build 1.0.0-rc.156
modbuild build 1.0.0-rc.156
```
Step 4: confirm that the module loads as expected
phenix/README.md
+8
View File
@@ -21,6 +21,14 @@ rosetta.build_phenix_interface nproc=10
- run rosetta.run_tests to test it.
### Unbound variable on module load
Sourcing the phenix env script can fail under `nounset`, e.g.
`phenix/phenix-1.20-4459/build/setpaths.sh: line 4:
LIBTBX_BUILD_RELOCATION_HINT: unbound variable`. Fix: add
`puts stdout "set +x nounset"` before the `source` line in the
modulefile. (Applies to any cctbx-style env script — see also DIALS.)
### Additional Remarks
the phenix help pages (BB archive) does not work.
pymol/README.md
+2 -2
View File
@@ -30,7 +30,7 @@ prompts on first start (get the file from https://pymol.org/dsc/ip/license/).
## Building on Ra
```sh
module use unstable && module load modbuild/2.1.2
module use unstable && module load modbuild/2.1.3
cd pymol
modbuild build 3.1.8
module use MX unstable && module load pymol/3.1.8
@@ -49,4 +49,4 @@ pymol # GUI check, needs X
Check the exact bundle filename on https://pymol.org (the `_0` build
number and `py310` tag change between releases), update `urls:` if
needed, add the version + tarball sha256 in `files/config.yaml`, then
`modbuild build <version>` (modbuild/2.1.2) on a Ra login node.
`modbuild build <version>` (modbuild/2.1.3) on a Ra login node.
xds/README.md
+3 -3
View File
@@ -2,11 +2,11 @@
Step 1: make sure you are in the `unx-mx_adm` group -- no -adm account needed, members can update the modules in `/opt/psi/MX` directly (if you have problems, please let the admins know)
Step 2: load the latest Pmodules package
Step 2: load the build tool (modbuild)
```
module use unstable
module load Pmodules/1.1.20
module load modbuild/2.1.3
```
Step 3: go into xds directory, update config.yaml to add newest release version and build module with that release name (e.g. below is for 20240724)
@@ -14,7 +14,7 @@ Step 3: go into xds directory, update config.yaml to add newest release version
```
cd xds
vi files/config.yaml
./build 20240724
modbuild build 20240724
```
Step 4: confirm that the module loads as expected
xdsgui/README.md
+1 -1
View File
@@ -31,4 +31,4 @@ xdsgui # on a node with X forwarding; GUI opens
## Adding a new version
Add the new download date under `versions:` in `files/config.yaml`,
then on a Ra login node: `modbuild build <version>` (modbuild/2.1.2).
then on a Ra login node: `modbuild build <version>` (modbuild/2.1.3).