MX_Pmodule/ccp4/README.md

ccp4

CCP4 suite (binary distribution, bundled with SHELX and ARP/wARP) for macromolecular crystallography. https://www.ccp4.ac.uk/

• Vendor tarball install: custom prep downloads the ~3.7 GB tarball with wget --continue (resumes partial downloads; the builtin curl can't) and verifies it with gzip -t; install untars it into $PREFIX and runs BINARY.setup.
• Non-interactive: the CCP4 licence is pre-agreed via .agree2ccp4v6 before BINARY.setup runs, so the build never prompts.
• No shasums:: the download link is session-gated (sid= expires), so the checksum cannot be pinned ahead of the first fetch. After the first successful build, sha256sum the cached tarball and add it if wanted.
• Module version is the series (9.0); minor updates (9.0.xxx) come via the CCP4 update manager inside the installation.

Build on Ra

Build tmp and download cache MUST be on /das, not the defaults (/var/tmp is too small, the tarball alone blows the $HOME quota — a download dying with "Failure writing output to destination" means exactly that). Pass both dirs explicitly on the command line so there is no doubt where things land — adapt the p-group path (p21/p21515) to your own:

module use unstable && module load modbuild/2.1.3
cd ccp4
mkdir -p /das/work/p21/p21515/.cache/Pmodules
modbuild build \
    --tmpdir=/das/work/p21/p21515/.cache/Pmodules \
    --distdir=/das/work/p21/p21515/.cache/Pmodules \
    --clean-install -v --debug 9.0

(--clean-install removes a previous broken install first; -v --debug make the otherwise silent long steps visible.)

Alternative: ~/.Pmodules/Pmodules.yaml with top-level tmp_dir: and download_dir: keys sets the same defaults per user — but in practice download_dir did not reliably take effect there, so the explicit flags above are the maintained route.

If the download fails (expired sid), fetch a fresh link from https://www.ccp4.ac.uk/download/ and update files/config.yaml, or drop a hand-downloaded tarball into the distfiles dir under the name: given in files/config.yaml.

Install is SLOW — not stuck

The install step untars 3.7 GB into ~15-20 GB of small files on shared /opt/psi storage: the tar alone runs 10-30 min with no output (gzip is single-threaded, small-file creation on network storage is the bottleneck). BINARY.setup afterwards is just as bad — it walks the whole installation patching paths and then byte-compiles the bundled Python environment ("compiling py-files"), thousands of tiny .pyc writes on network storage; expect another 15-30+ min. Budget roughly an hour for the whole install on Ra. Watch progress from a second shell:

watch -n 30 'du -sh /opt/psi/MX/ccp4/9.0/'

Growing du = working. Only investigate if it's frozen for 10+ min.

Known failure modes

• "done" but broken install: modbuild has no errexit in hooks — errors in pbuild::install used to scroll by and the build still reported done. The build script now std::dies on every critical step.
• Interrupted download: prep detects a truncated tarball with gzip -t and resumes it with wget --continue — just re-run the build command above. A download that dies with Failure writing output to destination means quota/disk full at the distfiles dir, not a network problem — check --distdir points to /das before retrying.
• The tarball unpacks to ccp4-9 (not ccp4-9.0 as the CCP4 install doc implies); the build globs ccp4-*/ instead of assuming the name.

Verify

module use MX unstable && module load ccp4/9.0
which refmac5 ccp4i2
echo $CCP4          # should point into /opt/psi/MX/ccp4/9.0/ccp4 (-> ccp4-9)

Bundled SHELX and ARP/wARP still need their own academic registrations.