duan_jandClaude Fable 5
fcedf651b6
Byte-compiling the bundled Python env writes thousands of .pyc files on network storage; whole install budget on Ra is about an hour. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
86 lines
3.3 KiB
Markdown
86 lines
3.3 KiB
Markdown
# ccp4
|
|
|
|
CCP4 suite (binary distribution, bundled with SHELX and ARP/wARP) for
|
|
macromolecular crystallography. https://www.ccp4.ac.uk/
|
|
|
|
- Vendor tarball install: custom `prep` downloads the ~3.7 GB tarball with
|
|
`wget --continue` (resumes partial downloads; the builtin curl can't) and
|
|
verifies it with `gzip -t`; `install` untars it into `$PREFIX` and runs
|
|
`BINARY.setup`.
|
|
- Non-interactive: the CCP4 licence is pre-agreed via `.agree2ccp4v6`
|
|
before `BINARY.setup` runs, so the build never prompts.
|
|
- No `shasums:`: the download link is session-gated (`sid=` expires), so
|
|
the checksum cannot be pinned ahead of the first fetch. After the first
|
|
successful build, `sha256sum` the cached tarball and add it if wanted.
|
|
- Module version is the series (`9.0`); minor updates (9.0.xxx) come via
|
|
the CCP4 update manager inside the installation.
|
|
|
|
## Build on Ra
|
|
|
|
One-time per user: put the Pmodules build tmp and download cache on
|
|
/das instead of /var/tmp and $HOME (the tarball alone would blow the
|
|
home quota):
|
|
|
|
```bash
|
|
mkdir -p /das/work/p21/p21515/.cache/Pmodules/distfiles ~/.Pmodules
|
|
cat > ~/.Pmodules/Pmodules.yaml <<'EOF'
|
|
tmp_dir: /das/work/p21/p21515/.cache/Pmodules
|
|
download_dir: /das/work/p21/p21515/.cache/Pmodules/distfiles
|
|
EOF
|
|
```
|
|
|
|
Then:
|
|
|
|
```bash
|
|
module use unstable && module load modbuild/2.1.3
|
|
cd ccp4
|
|
modbuild build --prep 9.0 # download/cache only; also verifies tmp_dir took effect
|
|
modbuild build 9.0
|
|
```
|
|
|
|
If the download fails (expired sid), fetch a fresh link from
|
|
https://www.ccp4.ac.uk/download/ and update `files/config.yaml`, or drop a
|
|
hand-downloaded tarball into the distfiles dir under the `name:` given in
|
|
`files/config.yaml`.
|
|
|
|
## Install is SLOW — not stuck
|
|
|
|
The `install` step untars 3.7 GB into ~15-20 GB of small files on shared
|
|
/opt/psi storage: the tar alone runs 10-30 min with no output (gzip is
|
|
single-threaded, small-file creation on network storage is the
|
|
bottleneck). `BINARY.setup` afterwards is just as bad — it walks the
|
|
whole installation patching paths and then **byte-compiles the bundled
|
|
Python environment** ("compiling py-files"), thousands of tiny .pyc
|
|
writes on network storage; expect another 15-30+ min. Budget roughly an
|
|
hour for the whole install on Ra. Watch progress from a second shell:
|
|
|
|
```bash
|
|
watch -n 30 'du -sh /opt/psi/MX/ccp4/9.0/'
|
|
```
|
|
|
|
Growing `du` = working. Only investigate if it's frozen for 10+ min.
|
|
|
|
## Known failure modes
|
|
|
|
- **"done" but broken install**: modbuild has no errexit in hooks — errors in
|
|
`pbuild::install` used to scroll by and the build still reported done. The
|
|
build script now `std::die`s on every critical step.
|
|
- **Interrupted download**: prep now detects a truncated tarball with
|
|
`gzip -t` and resumes it with `wget --continue` — just re-run
|
|
`modbuild build 9.0`. A download that dies with
|
|
`Failure writing output to destination` means quota/disk full at the
|
|
distfiles dir, not a network problem — free space or move `download_dir:`
|
|
(see above) before retrying.
|
|
- The tarball unpacks to `ccp4-9` (not `ccp4-9.0` as the CCP4 install doc
|
|
implies); the build globs `ccp4-*/` instead of assuming the name.
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
module use MX unstable && module load ccp4/9.0
|
|
which refmac5 ccp4i2
|
|
echo $CCP4 # should point into /opt/psi/MX/ccp4/9.0/ccp4 (-> ccp4-9)
|
|
```
|
|
|
|
Bundled SHELX and ARP/wARP still need their own academic registrations.
|