mirror of
https://github.com/Pmodules/Pmodules.git
synced 2026-06-30 11:19:39 +02:00
277 lines
8.2 KiB
Plaintext
277 lines
8.2 KiB
Plaintext
= Overlays
|
|
:TOC:
|
|
:sectnums:
|
|
|
|
Please note: This document is work in progress!
|
|
|
|
== What are Overlays and for what can they be used?
|
|
|
|
== Overlay configuration
|
|
|
|
=== System wide
|
|
|
|
==== Example
|
|
.A system wide configuration file `/opt/psi/config/Pmodules.yaml`
|
|
====
|
|
....
|
|
DefaultGroups: Tools:Programming
|
|
DefaultReleaseStages: stable
|
|
ReleaseStages: unstable:stable:deprecated
|
|
TmpDir: /opt/psi/var/tmp/${USER}
|
|
DistfilesDir: /opt/psi/var/distfiles
|
|
|
|
Overlays:
|
|
base:
|
|
install_root: /opt/psi
|
|
modulefiles_root: /opt/psi
|
|
devel:
|
|
install_root: /opt/psi
|
|
modulefiles_root: ${HOME}/modulefiles
|
|
....
|
|
====
|
|
==== YAML Format
|
|
.Format
|
|
====
|
|
....
|
|
Overlays:
|
|
<name>:
|
|
install_root: <installation-root>
|
|
modulefiles_root: <modulefiles-root>
|
|
type: <type of overlay>
|
|
...
|
|
....
|
|
====
|
|
`<name>`:: Name of overlay. Note: whitespace in the name is not supported.
|
|
`<name>.install_root`:: The root of the software installation. This key is mandatory.
|
|
`<name>.modulefiles_root`:: The root of the modulefile hierarchy. This key is optional and defaults to `<name>.install_root>
|
|
`<name>.type`:: The type of the overlay, see below. This key is optional and the default value is `n`.
|
|
|
|
==== Overlay types
|
|
`n`:: Normal overlay.
|
|
`h`:: Hiding overlay.
|
|
`r`:: Replacing overlay.
|
|
|
|
=== User defined overlays
|
|
|
|
Each user can define his own overlays in `$HOME/.Pmodules/Pmodules.yaml`.
|
|
|
|
== Working with overlays
|
|
|
|
== Building modules
|
|
|
|
The old format of the variants file is simple but very limited and almost impossible to extend for new features. To overcome the limitations a new format using YAML for variants files has been introduced. For the time being both format are supported. But it is highly recommended to use the YAML format for new modules and to migrate existing variants files in the old format to the new.
|
|
|
|
=== With a YAML variants file
|
|
|
|
Information is in the YAML configuration file:
|
|
|
|
==== General structure
|
|
|
|
name:: The name of the Pmodule without version. Example: `hdf5`
|
|
group:: group the module is in. Example: `MPI`
|
|
overlay:: (optional) overlay, defaults to `base` if not specified
|
|
relstage:: (optional) default release stage. If not defined,
|
|
it defaults to `unstable`,
|
|
url:: (optional) download URL, Must be either define here or per
|
|
version. The environment variables `$P`, `$V`, `$V_PKG` etc. can be used. The downloaded file is stored with the file name `$P-${V_PKG}.ext`. Whereby `ext` is derived from the URL.
|
|
addon_sources:: (optional) array of sources for add-ons. Example: `git-lfs`
|
|
use:: (optional) use flags.
|
|
systems:: (optional) array of supported systems, defaults to any. Example: `[rhel6 rhel7]`.
|
|
compilers:: (optional) array pf supported compilers, defaults to any. Example: `[gcc intel]`.
|
|
compile-in-src:: (optional) compile in source tree, allowed vaules are `yes` and `no`, the default is `no`.
|
|
configure-with:: (optional) If a software supports CMake and autotools, this specifies the tool to use. Allowed values are `autotools`, `cmake`, `auto`, `none`. The default is `auto`.
|
|
versions:: array of `version` objects
|
|
|
|
==== Objects of type `addon_source`
|
|
|
|
name:: unique name
|
|
url:: download URL.
|
|
shasum:: (optional) SHA256 sum.
|
|
|
|
==== Objects of type `version`
|
|
|
|
dependencies:: other run-time dependencies, if any
|
|
build_with:: dependencies required for building, if any
|
|
overlay:: (optional) override the default. Example: `devel`.
|
|
relstage:: (optional) override the default. Example: `unstable`.
|
|
addon_sources:: (optional) array of sources for add-ons.
|
|
systems:: (optional) override the default
|
|
compilers:: (optional) override the default
|
|
compile-in-src:: (optional) override the default
|
|
configure-with:: (optional) override the default
|
|
variants:: array of `variant` objects
|
|
|
|
==== Object of type `variant`
|
|
with:: hierarchical dependencies, `with` must be specified only
|
|
for hierarchical groups.
|
|
dependencies:: other run-time dependencies, if any
|
|
build_requires:: dependencies required for building, if any
|
|
addon_sources:: (optional) array of sources for add-ons.
|
|
use:: (optional) use flags
|
|
overlay:: (optional) override the default. Example: `devel`.
|
|
relstage:: (optional) override the default. Example: `unstable`.
|
|
systems:: (optional) override the default
|
|
compilers:: (optional) override the default
|
|
|
|
==== Example of a configuration files in YAML format
|
|
|
|
.YAML configuration file for serial HDF5
|
|
====
|
|
....
|
|
name: hdf5
|
|
group: MPI
|
|
overlay: base
|
|
relstage: stable
|
|
url: https://support.hdfgroup.org/ftp/HDF5/releases/$P-${V_MAJOR}.${V_MINOR}/$P-${V_PKG}/src/$P-${V_PKG}.tar.gz
|
|
systems: [rhel6,rhel7,rhel8]
|
|
compilers: [gcc,intel,pgi]
|
|
compile_in_sourcetree: no
|
|
configure_with: auto
|
|
|
|
versions:
|
|
1.12.2:
|
|
shasum: 3016ea56a175d2ca7f2568c8016420f7a2aad8f95e214fe7fa5485f4b80fbe51
|
|
variants:
|
|
- group_deps:
|
|
compiler:
|
|
gcc: [5.5.0,6.5.0,7.5.0,10.2.0,10.3.0]
|
|
intel: ....
|
|
mpi:
|
|
openmpi: [...]
|
|
#hdf5:
|
|
runtime_deps:
|
|
cuda:10.0
|
|
...
|
|
build_deps:
|
|
name:version
|
|
|
|
|
|
|
|
- with:
|
|
- gcc: [8.5.0,9.5.0,11.3.0,12.1.0]
|
|
relstage: unstable
|
|
overlay: devel
|
|
....
|
|
====
|
|
|
|
.YAML configuration file for HDF5 (serial and parallel)
|
|
====
|
|
....
|
|
name: hdf5
|
|
overlay: base
|
|
relstage: stable
|
|
url: https://support.hdfgroup.org/ftp/HDF5/releases/$P-${V_MAJOR}.${V_MINOR}/$P-${V_PKG}/src/$P-${V_PKG}.tar.gz
|
|
systems: [rhel6 rhel7 rhel8]
|
|
compilers: [gcc intel pgi]
|
|
compile_in_sourcetree: no
|
|
configure_with: auto
|
|
|
|
versions:
|
|
1.12.2:
|
|
shasum: 3016ea56a175d2ca7f2568c8016420f7a2aad8f95e214fe7fa5485f4b80fbe51
|
|
variants:
|
|
- with:
|
|
- gcc [5.5.0 6.5.0 7.5.0 10.2.0 10.3.0]
|
|
group: Compiler
|
|
- with:
|
|
- gcc [8.5.0,9.5.0,11.3.0,12.1.0]
|
|
- openmpi [4.0.5-2]
|
|
group: MPI
|
|
relstage: unstable
|
|
overlay: devel
|
|
....
|
|
====
|
|
|
|
.YAML configuration file for Git with git-lfs
|
|
====
|
|
....
|
|
name: git
|
|
defaults:
|
|
group: Tools
|
|
overlay: base
|
|
relstage: stable
|
|
url: https://mirrors.edge.kernel.org/pub/software/scm/$P/$P-${V_PKG}.tar.xz
|
|
systems: [rhel6 rhel7 rhel8]
|
|
compilers: [gcc]
|
|
compile_in_sourcetree: yes
|
|
configure_with: auto
|
|
script: build
|
|
|
|
addons:
|
|
git-lfs/3.2.0:
|
|
- url: https://github.com/git-lfs/git-lfs/archive/refs/tags/v3.2.0.tar.gz
|
|
shasum: f8e6bbe043b97db8a5c16da7289e149a3fed9f4d4f11cffcc6e517c7870cd9e5
|
|
build_requires: [go/1.19]
|
|
build_script: build-git-lfs
|
|
|
|
versions:
|
|
2.37.2:
|
|
shasum: 4c428908e3a2dca4174df6ef49acc995a4fdb1b45205a2c79794487a33bc06e5
|
|
variants:
|
|
- dependencies: TclTk/8.6.9
|
|
build_requires: [perl/5.30.0 asciidoc/8.6.9-1 xmlto/0.0.28 gettext/0.19.8]
|
|
relstage: unstable
|
|
overlay: devel
|
|
addons:
|
|
git-lfs: 3.2.0
|
|
|
|
....
|
|
====
|
|
|
|
==== Format specification
|
|
.YAML format
|
|
....
|
|
name: <Pmodule-name-without-version>
|
|
source_url: "https://..."
|
|
overlay: <opt-default-overlay>
|
|
group: <default-group>
|
|
relstage: <opt-default-release-stage>
|
|
systems: [opt-array-of-supported]
|
|
compilers: [opt-array-of-supported-compilers]
|
|
compile-in-src: yes|no
|
|
configure-with: cmake|autotools|auto|none
|
|
|
|
versions:
|
|
- <version1>:
|
|
overlay: <overlay>
|
|
group: <group>
|
|
relstage: <release-stage>
|
|
systems: [supported-systems]
|
|
sources:
|
|
- url: <url>
|
|
shasum: <shasum>
|
|
compilers: <default-list-of-supported-compilers>
|
|
compile-in-src: yes|no
|
|
configure-with: cmake|autotools|auto|none
|
|
variants:
|
|
- with:
|
|
- gcc: [5.5.0,6.5.0,7.5.0,10.2.0,10.3.0]
|
|
dependencies: [array-of-runtime-deps]
|
|
build_requires: [array-of-build-deps]
|
|
relstage: <release-stage>
|
|
use: [array-of-use-flags]
|
|
overlay: <overlay>
|
|
...
|
|
....
|
|
==== Defaults
|
|
Default values can be overriden per version/variant.
|
|
|
|
`overlay`:: The default overlay the module will be installed in. This value can be overriden for dedicated versions/variants.
|
|
`systems`:: The default for supported systems.
|
|
|
|
==== Versions and Variants
|
|
|
|
`<name>/<version>`:: An array with variants for this version.
|
|
|
|
`<name>/<version>.[i].with`:: Hierarchical dependencies for variant `i`.
|
|
|
|
`<name>/<version>.[i].dependencies`:: Build/run-time dependencies for variant `i`.
|
|
|
|
`<name>/<version>.[i].relstage`:: Relase stage of variant `i`.
|
|
|
|
`<name>/<version>.[i].overlay`:: Overlay of variant `i`.
|
|
|
|
`<name>/<version>.[i].systems`:: Supported systems.
|
|
|
|
=== Legacy format
|