= Overlays :TOC: :sectnums: Please note: This document is work in progress! == What are Overlays and for what can they be used? == Overlay configuration === System wide ==== Example .A system wide configuration file `/opt/psi/config/Pmodules.yaml` ==== .... DefaultGroups: Tools:Programming DefaultReleaseStages: stable ReleaseStages: unstable:stable:deprecated TmpDir: /opt/psi/var/tmp/${USER} DistfilesDir: /opt/psi/var/distfiles Overlays: base: install_root: /opt/psi modulefiles_root: /opt/psi devel: install_root: /opt/psi modulefiles_root: ${HOME}/modulefiles .... ==== ==== YAML Format .Format ==== .... Overlays: : install_root: modulefiles_root: type: ... .... ==== ``:: Name of overlay. Note: whitespace in the name is not supported. `.install_root`:: The root of the software installation. This key is mandatory. `.modulefiles_root`:: The root of the modulefile hierarchy. This key is optional and defaults to `.install_root> `.type`:: The type of the overlay, see below. This key is optional and the default value is `n`. ==== Overlay types `n`:: Normal overlay. `h`:: Hiding overlay. `r`:: Replacing overlay. === User defined overlays Each user can define his own overlays in `$HOME/.Pmodules/Pmodules.yaml`. == Working with overlays == Building modules The old format of the variants file is simple but very limited and almost impossible to extend for new features. To overcome the limitations a new format using YAML for variants files has been introduced. For the time being both format are supported. But it is highly recommended to use the YAML format for new modules and to migrate existing variants files in the old format to the new. === With a YAML variants file Information is in the YAML configuration file: ==== General structure name:: The name of the Pmodule without version. Example: `hdf5` group:: group the module is in. Example: `MPI` overlay:: (optional) overlay, defaults to `base` if not specified relstage:: (optional) default release stage. If not defined, it defaults to `unstable`, url:: (optional) download URL, Must be either define here or per version. The environment variables `$P`, `$V`, `$V_PKG` etc. can be used. The downloaded file is stored with the file name `$P-${V_PKG}.ext`. Whereby `ext` is derived from the URL. addon_sources:: (optional) array of sources for add-ons. Example: `git-lfs` use:: (optional) use flags. systems:: (optional) array of supported systems, defaults to any. Example: `[rhel6 rhel7]`. compilers:: (optional) array pf supported compilers, defaults to any. Example: `[gcc intel]`. compile-in-src:: (optional) compile in source tree, allowed vaules are `yes` and `no`, the default is `no`. configure-with:: (optional) If a software supports CMake and autotools, this specifies the tool to use. Allowed values are `autotools`, `cmake`, `auto`, `none`. The default is `auto`. versions:: array of `version` objects ==== Objects of type `addon_source` name:: unique name url:: download URL. shasum:: (optional) SHA256 sum. ==== Objects of type `version` dependencies:: other run-time dependencies, if any build_with:: dependencies required for building, if any overlay:: (optional) override the default. Example: `devel`. relstage:: (optional) override the default. Example: `unstable`. addon_sources:: (optional) array of sources for add-ons. systems:: (optional) override the default compilers:: (optional) override the default compile-in-src:: (optional) override the default configure-with:: (optional) override the default variants:: array of `variant` objects ==== Object of type `variant` with:: hierarchical dependencies, `with` must be specified only for hierarchical groups. dependencies:: other run-time dependencies, if any build_requires:: dependencies required for building, if any addon_sources:: (optional) array of sources for add-ons. use:: (optional) use flags overlay:: (optional) override the default. Example: `devel`. relstage:: (optional) override the default. Example: `unstable`. systems:: (optional) override the default compilers:: (optional) override the default ==== Example of a configuration files in YAML format .YAML configuration file for serial HDF5 ==== .... name: hdf5 group: MPI overlay: base relstage: stable url: https://support.hdfgroup.org/ftp/HDF5/releases/$P-${V_MAJOR}.${V_MINOR}/$P-${V_PKG}/src/$P-${V_PKG}.tar.gz systems: [rhel6,rhel7,rhel8] compilers: [gcc,intel,pgi] compile_in_sourcetree: no configure_with: auto versions: 1.12.2: shasum: 3016ea56a175d2ca7f2568c8016420f7a2aad8f95e214fe7fa5485f4b80fbe51 variants: - group_deps: compiler: gcc: [5.5.0,6.5.0,7.5.0,10.2.0,10.3.0] intel: .... mpi: openmpi: [...] #hdf5: runtime_deps: cuda:10.0 ... build_deps: name:version - with: - gcc: [8.5.0,9.5.0,11.3.0,12.1.0] relstage: unstable overlay: devel .... ==== .YAML configuration file for HDF5 (serial and parallel) ==== .... name: hdf5 overlay: base relstage: stable url: https://support.hdfgroup.org/ftp/HDF5/releases/$P-${V_MAJOR}.${V_MINOR}/$P-${V_PKG}/src/$P-${V_PKG}.tar.gz systems: [rhel6 rhel7 rhel8] compilers: [gcc intel pgi] compile_in_sourcetree: no configure_with: auto versions: 1.12.2: shasum: 3016ea56a175d2ca7f2568c8016420f7a2aad8f95e214fe7fa5485f4b80fbe51 variants: - with: - gcc [5.5.0 6.5.0 7.5.0 10.2.0 10.3.0] group: Compiler - with: - gcc [8.5.0,9.5.0,11.3.0,12.1.0] - openmpi [4.0.5-2] group: MPI relstage: unstable overlay: devel .... ==== .YAML configuration file for Git with git-lfs ==== .... name: git defaults: group: Tools overlay: base relstage: stable url: https://mirrors.edge.kernel.org/pub/software/scm/$P/$P-${V_PKG}.tar.xz systems: [rhel6 rhel7 rhel8] compilers: [gcc] compile_in_sourcetree: yes configure_with: auto script: build addons: git-lfs/3.2.0: - url: https://github.com/git-lfs/git-lfs/archive/refs/tags/v3.2.0.tar.gz shasum: f8e6bbe043b97db8a5c16da7289e149a3fed9f4d4f11cffcc6e517c7870cd9e5 build_requires: [go/1.19] build_script: build-git-lfs versions: 2.37.2: shasum: 4c428908e3a2dca4174df6ef49acc995a4fdb1b45205a2c79794487a33bc06e5 variants: - dependencies: TclTk/8.6.9 build_requires: [perl/5.30.0 asciidoc/8.6.9-1 xmlto/0.0.28 gettext/0.19.8] relstage: unstable overlay: devel addons: git-lfs: 3.2.0 .... ==== ==== Format specification .YAML format .... name: source_url: "https://..." overlay: group: relstage: systems: [opt-array-of-supported] compilers: [opt-array-of-supported-compilers] compile-in-src: yes|no configure-with: cmake|autotools|auto|none versions: - : overlay: group: relstage: systems: [supported-systems] sources: - url: shasum: compilers: compile-in-src: yes|no configure-with: cmake|autotools|auto|none variants: - with: - gcc: [5.5.0,6.5.0,7.5.0,10.2.0,10.3.0] dependencies: [array-of-runtime-deps] build_requires: [array-of-build-deps] relstage: use: [array-of-use-flags] overlay: ... .... ==== Defaults Default values can be overriden per version/variant. `overlay`:: The default overlay the module will be installed in. This value can be overriden for dedicated versions/variants. `systems`:: The default for supported systems. ==== Versions and Variants `/`:: An array with variants for this version. `/.[i].with`:: Hierarchical dependencies for variant `i`. `/.[i].dependencies`:: Build/run-time dependencies for variant `i`. `/.[i].relstage`:: Relase stage of variant `i`. `/.[i].overlay`:: Overlay of variant `i`. `/.[i].systems`:: Supported systems. === Legacy format