mx/MX_Pmodule
1
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Drop careless-gpu: the [cuda] pip extra install covers GPU and CPU nodes

Browse Source 
The installed careless/0.5.4 venv already contains the NVIDIA wheels
([cuda] extra), which need only the node driver and fall back to CPU,
so a separate GPU module and cuda module dep are unnecessary. Revert
careless/build to careless[cuda] to match the actual install.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
duan_j
2026-08-18 11:13:19 +02:00
co-authored by Claude Fable 5
parent b55d6c1d91
commit f6c2f5a650
7 changed files with 12 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files
careless-gpu/README.md
-28
View File
@@ -1,28 +0,0 @@
# careless-gpu
GPU build of careless (see `../careless/` for the CPU module and the full
background). Scaling and merging of X-ray diffraction data with approximate
Bayesian inference (variational merging), built on TensorFlow.
- Upstream: https://github.com/rs-station/careless
- License: MIT
## How it differs from the careless module
- CUDA is not pip-installed (no `[cuda]` extra, saves several GB): the
cluster `cuda/12.8.1` module provides it instead, declared as
`build_requires`/`runtime_deps` (same pattern as `jfjoch_viewer`).
`runtime_deps` matters because TensorFlow dlopens the CUDA libraries
when careless runs, so cuda must auto-load with this module.
- If careless reports missing cuDNN at runtime, the cuda module does not
ship it; the fallback is installing `careless[cuda]` in `build` instead.
The venv layout, uv usage, and cache handling are identical to `careless`.
To share one download cache between both builds, run each with
`UV_CACHE_DIR=<shared path> modbuild build <version>` — a cache outside
PREFIX is kept, an in-PREFIX cache is removed after a successful install.
## Adding a new version
Add the version under `versions:` in `files/config.yaml`, then on a Ra
login node: `modbuild build <version>` (modbuild/2.1.2).
careless-gpu/build
-26
View File
@@ -1,26 +0,0 @@
#!/usr/bin/env modbuild
pbuild::prep() {
:
}
pbuild::configure() {
# same uv pattern as the careless (CPU) module; see that build for the whys
UV="${UV:-$HOME/.local/bin/uv}"
export UV_CACHE_DIR="${UV_CACHE_DIR:-$PREFIX/.uv-cache}"
export UV_PYTHON_INSTALL_DIR="$PREFIX/python"
"$UV" venv --python 3.12 "$PREFIX/venv"
}
pbuild::compile() {
:
}
pbuild::install() {
UV="${UV:-$HOME/.local/bin/uv}"
export UV_CACHE_DIR="${UV_CACHE_DIR:-$PREFIX/.uv-cache}"
# no [cuda] extra: CUDA comes from the cuda/12.8.1 module
# (build_requires/runtime_deps in files/config.yaml), not from pip wheels
"$UV" pip install --python "$PREFIX/venv/bin/python" "careless==${V_PKG}"
[[ "$UV_CACHE_DIR" == "$PREFIX"* ]] && rm -rf "$UV_CACHE_DIR" || :
}
careless-gpu/files/config.yaml
-13
View File
@@ -1,13 +0,0 @@
---
format: 1
careless-gpu:
defaults:
group: MX
overlay: base
relstage: unstable
versions:
0.5.4:
config:
relstage: unstable
build_requires: ["cuda/12.8.1"]
runtime_deps: ["cuda/12.8.1"]
careless-gpu/modulefile
-16
View File
@@ -1,16 +0,0 @@
#%Module1.0
module-whatis "careless (GPU) - merging crystallography data with variational inference"
module-url "https://github.com/rs-station/careless"
module-license "MIT"
module-maintainer "MX Data <jiaxin.duan@psi.ch>"
module-help "
careless: scaling and merging of X-ray diffraction data using
approximate Bayesian inference (TensorFlow). GPU build: CUDA is
provided by the cuda/12.8.1 module, auto-loaded as a runtime dep.
Use on nodes with an NVIDIA GPU; for CPU nodes load careless instead.
"
setenv CARELESS_ENV $PREFIX/venv
prepend-path PATH $PREFIX/venv/bin
careless/README.md
+4 -8
View File
@@ -11,14 +11,10 @@ inference (variational merging), built on TensorFlow.
Upstream recommends a fresh conda env, but repo.anaconda.com is not
reachable from Ra, so the build uses uv instead: it fetches a standalone
Python 3.12 into `$PREFIX/python` (upstream supports >=3.9,<3.13), creates
`$PREFIX/venv`, and runs `uv pip install careless==<version>` into it.
This is the CPU build; the GPU build (with the `[cuda]` extra and the
cluster `cuda/12.8.1` module) is the separate `careless-gpu` module.
To share one uv download cache between the careless and careless-gpu
builds, run each with `UV_CACHE_DIR=<shared path> modbuild build <version>`
— a cache outside PREFIX is kept, an in-PREFIX cache is removed after a
successful install.
`$PREFIX/venv`, and runs `uv pip install careless[cuda]==<version>` into
it. The `[cuda]` extra ships the NVIDIA libraries (CUDA, cuDNN, ...) as
pip wheels, so the one install uses the GPU on GPU nodes (only the node's
NVIDIA driver is needed — no cuda module) and falls back to CPU elsewhere.
`uv` must be on PATH on the build host (e.g. `pip install --user uv`).
careless/build
+5 -5
View File
@@ -9,9 +9,8 @@ pbuild::configure() {
# uv fetches a standalone CPython instead; keep it inside PREFIX so the
# venv's interpreter symlinks don't depend on the builder's home cache.
UV="${UV:-$HOME/.local/bin/uv}"
# ~/.cache/uv hits the home disk quota; default the cache into PREFIX.
# Overridable so careless and careless-gpu can share one cache:
# UV_CACHE_DIR=<shared path> modbuild build <version>
# ~/.cache/uv hits the home disk quota; default the cache into PREFIX
# (overridable: UV_CACHE_DIR=<path> modbuild build <version>)
export UV_CACHE_DIR="${UV_CACHE_DIR:-$PREFIX/.uv-cache}"
export UV_PYTHON_INSTALL_DIR="$PREFIX/python"
"$UV" venv --python 3.12 "$PREFIX/venv"
@@ -24,8 +23,9 @@ pbuild::compile() {
pbuild::install() {
UV="${UV:-$HOME/.local/bin/uv}"
export UV_CACHE_DIR="${UV_CACHE_DIR:-$PREFIX/.uv-cache}"
# CPU version; the GPU build with the [cuda] extra is the careless-gpu module
"$UV" pip install --python "$PREFIX/venv/bin/python" "careless==${V_PKG}"
# [cuda] extra ships the NVIDIA stack (CUDA, cuDNN, ...) as pip wheels:
# uses the GPU when present (needs only the node driver), else falls back to CPU
"$UV" pip install --python "$PREFIX/venv/bin/python" "careless[cuda]==${V_PKG}"
# only auto-remove the cache if it lives inside this PREFIX (never a shared one)
[[ "$UV_CACHE_DIR" == "$PREFIX"* ]] && rm -rf "$UV_CACHE_DIR" || :
}
careless/modulefile
+3 -2
View File
@@ -7,8 +7,9 @@ module-maintainer "MX Data <jiaxin.duan@psi.ch>"
module-help "
careless: scaling and merging of X-ray diffraction data using
approximate Bayesian inference (TensorFlow). CPU build; for
NVIDIA GPU nodes load careless-gpu instead.
approximate Bayesian inference (TensorFlow). Installed with the
[cuda] extra: uses NVIDIA GPUs when present (no cuda module
needed), falls back to CPU otherwise.
"
setenv CARELESS_ENV $PREFIX/venv