From 8d059d9807a616c22f09313d72c47538952071a0 Mon Sep 17 00:00:00 2001 From: sala Date: Thu, 30 Jun 2022 07:38:30 +0000 Subject: [PATCH 01/10] Update bob.rst with updated repo --- admin-guide/mgmt-tools/bob.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/admin-guide/mgmt-tools/bob.rst b/admin-guide/mgmt-tools/bob.rst index db676d3f..52b7f511 100644 --- a/admin-guide/mgmt-tools/bob.rst +++ b/admin-guide/mgmt-tools/bob.rst @@ -14,8 +14,8 @@ For production use of bob there will be RPMs for bob and its dependencies. For the moment the steps are the following:: yum -y install python-setuptools python-requests-kerberos - git clone git@git.psi.ch:linux-infra/bob.git - cd bob + git clone git@git.psi.ch:linux-infra/admin-tools.git + cd admin-tools python setup.py install From 6ec6d55427090343b0dcd68689b68bbeb5e59205 Mon Sep 17 00:00:00 2001 From: Konrad Bucheli Date: Fri, 1 Jul 2022 15:44:02 +0200 Subject: [PATCH 02/10] document new package repository configuration in puppet --- admin-guide/puppet/modules.rst | 7 +- admin-guide/puppet/profiles/package_list.rst | 44 +++++++++++ admin-guide/puppet/profiles/repository.rst | 73 +++++++++++++++++++ .../puppet/profiles/repository_list.rst | 47 ++++++++++++ .../{yum_client.rst => rpm_repos.rst} | 40 +++++----- admin-guide/puppet/roles/base.rst | 44 ++++++++--- rhel8/packages.md | 6 +- 7 files changed, 224 insertions(+), 37 deletions(-) create mode 100644 admin-guide/puppet/profiles/package_list.rst create mode 100644 admin-guide/puppet/profiles/repository.rst create mode 100644 admin-guide/puppet/profiles/repository_list.rst rename admin-guide/puppet/profiles/{yum_client.rst => rpm_repos.rst} (56%) diff --git a/admin-guide/puppet/modules.rst b/admin-guide/puppet/modules.rst index 65eb2f00..1cc425ef 100644 --- a/admin-guide/puppet/modules.rst +++ b/admin-guide/puppet/modules.rst @@ -229,18 +229,21 @@ Profiles profiles/nfs_server profiles/ntp_client profiles/nvidia + profiles/package_list profiles/platform profiles/platform/hewlett_packard profiles/pmodules profiles/print_client profiles/puppet_client + profiles/repository + profiles/repository_list + profiles/rpm_repos profiles/serial_console profiles/ssh_client profiles/ssh_server.rst profiles/sysinfo profiles/telegraf profiles/web_server - profiles/yum_client Components @@ -249,7 +252,7 @@ Components .. toctree:: :maxdepth: 1 - components/grub2 + components/grub2 components/logrotate components/selinux components/sudo diff --git a/admin-guide/puppet/profiles/package_list.rst b/admin-guide/puppet/profiles/package_list.rst new file mode 100644 index 00000000..313a8df8 --- /dev/null +++ b/admin-guide/puppet/profiles/package_list.rst @@ -0,0 +1,44 @@ +``profile::package_list`` +============================ + +This module installs a list of packages on the node, with extra options inlcuding the filtering by OS. + +This profile is usually direcly used from Puppet:: + + profile::repository_list{'rpm_repos::influx':} + +Parameters +---------- + +=============================== ======== ============================================= +**Name** **Type** **Default** +------------------------------- -------- --------------------------------------------- +packages list [] +=============================== ======== ============================================= + + + +``packages`` +~~~~~~~~~~~~ + +This list contains the names of the packages to be installed. The names may be "extended" by tags, separated by ``:``. + +========== =============================== +Tag Function +---------- ------------------------------- +absent package will be removed +latest always latest version installed +os=redhat7 only install on given OS +os!redhat7 do not install on given OS +========== =============================== + + +Example definitions:: + + java-11-openjdk + @Java Platform + java-1.8.0-openjdk:os=redhat8:latest:os!redhat9 + java-1.8.0-openjdk:os=redhat7:absent + + + diff --git a/admin-guide/puppet/profiles/repository.rst b/admin-guide/puppet/profiles/repository.rst new file mode 100644 index 00000000..0b7d7701 --- /dev/null +++ b/admin-guide/puppet/profiles/repository.rst @@ -0,0 +1,73 @@ +``profile::repository_list`` +============================ + +This module configures adds a list of package repositories from hiera +and filters those according to OS version. + +Parameters +---------- + +=============================== =========== ============================================= +**Name** **Type** **Default** +------------------------------- ----------- --------------------------------------------- +descr string +baseurl string +gpgkey string undef +gpgcheck bool false +disable bool true +priority integer undef +exclude string/list '' +=============================== =========== ============================================= + + +``title`` +~~~~~~~~~ + +The repository definition will end up as ``/etc/yum.repos.d/${title}.repo`` on the node + + + +``descr`` +~~~~~~~~~ + +Description of the package repository. + + +``baseurl`` +~~~~~~~~~~~ + +URL where the repository is available from, e.g.:: + + https://repo01.psi.ch/el8/tags/$pli_repo_tag/epel/ + + +``gpgkey`` +~~~~~~~~~~ + +URL or file path where the signing GPG key is available from, e.g.:: + + https://repo01.psi.ch/el8/keys/RPM-GPG-KEY-EPEL-8 + + +``gpgcheck`` +~~~~~~~~~~~~ + +Check GPG signature of installed packages. + + +``disable`` +~~~~~~~~~~~ + +If the repository should be checked for package installation/updates or not. + + +``priority`` +~~~~~~~~~~~~ + +What priority do packages from this repositry have? + + +``exclude`` +~~~~~~~~~~~ + +List or comma separated string with packages to be ignored from this repositores. Wildcards are supported. diff --git a/admin-guide/puppet/profiles/repository_list.rst b/admin-guide/puppet/profiles/repository_list.rst new file mode 100644 index 00000000..a70c6498 --- /dev/null +++ b/admin-guide/puppet/profiles/repository_list.rst @@ -0,0 +1,47 @@ +``profile::repository_list`` +============================ + +This module configures adds a list of package repositories from hiera +and filters those according to OS version. + +Parameters +---------- + +=============================== ======== ============================================= +**Name** **Type** **Default** +------------------------------- -------- --------------------------------------------- +accept_empty bool false +=============================== ======== ============================================= + + +``title`` +~~~~~~~~~ + +Name of the list of software repositores to be installed. +It will lookup in Hiera for a list ``$title`` and look up the +repository definitons with ``hiera_hash("rpm_repos::repo::${name}")``. + +The repository definitions not suitable for this OS will be filtered away, +then with the rest a repository configuration is writen to the node +using ``profile::repository``. The ``osversion`` key in the repository definition +signals the major RHEL version for which this repository is for. + +Example definition:: + + rpm_repos::repo::epel_rhel8: + name: 'epel' + descr: "Extra Packages for Enterprise Linux 8" + baseurl: 'https://repo01.psi.ch/el8/tags/$pli_repo_tag/epel/' + gpgkey: 'https://repo01.psi.ch/el8/keys/RPM-GPG-KEY-EPEL-8' + disable: false + gpgcheck: true + osversion: 8 + exclude: + - "slurm*" + + + +``accept_empty`` +~~~~~~~~~~~~~~~~ + +Do not fail when the repository list is empty after diff --git a/admin-guide/puppet/profiles/yum_client.rst b/admin-guide/puppet/profiles/rpm_repos.rst similarity index 56% rename from admin-guide/puppet/profiles/yum_client.rst rename to admin-guide/puppet/profiles/rpm_repos.rst index a0b803b5..bb819b64 100644 --- a/admin-guide/puppet/profiles/yum_client.rst +++ b/admin-guide/puppet/profiles/rpm_repos.rst @@ -1,7 +1,7 @@ -``profile::yum_client`` +``profile::rpm_repos`` ======================= -This module configures :manpage:`yum(8)`. +This module configures :manpage:`dnf(8)` and sets up the default package repositores. Parameters @@ -10,32 +10,30 @@ Parameters =============================== ======== ============================================= **Name** **Type** **Default** ------------------------------- -------- --------------------------------------------- -exclude list hiera('yum_client::exclude', []) -package_groups list hiera_array('yum_client::package_groups', []) -purge_repositories bool hiera('yum_client::purge_repositories') -repos list hiera_array('yum_client::repositories') +repo_list string 'rpm_repos::default' +repo_tags hash hiera_hash('rpm_repos::tag') +exclude list hiera('base::package_exclude, []) +purge_repositories bool hiera('rpm_repos::purge_repositories') =============================== ======== ============================================= +``repo_list`` +~~~~~~~~~~~~~ + +Name of the list of default software repositores to be installed. + +``repo_tags`` +~~~~~~~~~~~~~ + +Hash containing the repository tag which should be used per OS. +The key for RHEL 7 is ``redhat7`` and points by default to ``prod``, +whereas for later versions is ``rhel-$MAJOR_VERSION`` (eg. for ``redhat8`` it is ``rhel-8``) + ``exclude`` ~~~~~~~~~~~ An array containing entries suitable for the ``exclude`` option in -:manpage:`yum.conf(5)`. - - -``package_group`` -~~~~~~~~~~~~~~~~~ - -A list containing the names of the package groups to be installed on the -systems. The package groups have to be defined through `pkg_group::$NAME`_ and -are completely independent of Yum (environment) groups. - - -``pkg_group::$NAME`` -~~~~~~~~~~~~~~~~~~~~ - -The list of packages that comprise the package group ``$NAME``. +:manpage:`dnf.conf(5)`. ``purge_repositories`` diff --git a/admin-guide/puppet/roles/base.rst b/admin-guide/puppet/roles/base.rst index 3cd96460..61df7999 100644 --- a/admin-guide/puppet/roles/base.rst +++ b/admin-guide/puppet/roles/base.rst @@ -15,6 +15,7 @@ Parameters default_target string ``multi-user`` enable_afs bool hiera('base::enable_afs') enable_autofs bool hiera('base::enable_autofs') +enable_epics bool hiera('base::enable_epics') enable_filecopy bool hiera('base::enable_filecopy') enable_ganglia bool hiera('base::enable_ganglia') enable_icinga bool hiera('base::enable_icinga') @@ -31,9 +32,10 @@ enable_telegraf bool hiera('base::enable_telegraf') enable_updatedb bool hiera('base::enable_updatedb') include_aaa bool true include_log_client bool true -include_yum_client bool true -package_groups array [] -pkg_group::* array - +include_rpm_repos bool true +package_groups array hiera_array('base::package_groups', []) +package_excludes array hierra_array('base::package_exclude', []) +pkg_group::* array hierra_array('base::pkg_group::...', []) selinux_mode string hiera('base::selinux_mode', 'enforcing') ====================== ======== ================================================ @@ -62,6 +64,12 @@ needed to support the ``-hosts`` map as documented in :manpage:`auto.master`. The ``-hosts`` map is mounted on ``/net``. +``enable_epics`` +~~~~~~~~~~~~~~~~~ + +Enables the ``EPICS``. TODO: more details... + + ``enable_filecopy`` ~~~~~~~~~~~~~~~~~~~ @@ -177,25 +185,39 @@ meant to allow roles customization of the :doc:`log_client <../profiles/log_client>` profile. -``include_yum_client`` +``include_rpm_repos`` ~~~~~~~~~~~~~~~~~~~~~~ -Determines whether to include the :doc:`yum_client <../profiles/yum_client>` -profile. +Determines whether to install the default RPM package repositories. ``package_groups`` ~~~~~~~~~~~~~~~~~~ -The list of package groups to install. Package groups are defined using -`pkg_group::NAME`_. +The list of package groups to install. Package groups are defined in Hiera using +``base::pkg_group::NAME``. ``pkg_group::NAME`` ~~~~~~~~~~~~~~~~~~~ -An array defining the package group ``NAME``. Package groups are installed using -the `package_groups`_ parameter. +An array defining the package group ``NAME``. It contains the package name with optionally +one or more tags, separated by ``:``. Following tags are allowed: + +========== ===================================================== +Tag Description +---------- ----------------------------------------------------- +latest ensure the latest version of the package is installed +absent ensure the package is not installed +os=redhat7 install it only on this OS +os!redhat7 install on any OS except this one +========== ===================================================== + + +``package_excludes`` +~~~~~~~~~~~~~~~~~~~~ + +An array with packages which are not made available on the system. ``selinux_mode`` @@ -224,7 +246,7 @@ Examples The most basic usage is:: class role::some_role () { - + include role::base ... diff --git a/rhel8/packages.md b/rhel8/packages.md index 91d17324..8e7e65a3 100644 --- a/rhel8/packages.md +++ b/rhel8/packages.md @@ -55,7 +55,7 @@ base::pkg_group::java: ### Install Packages only on Given OS Version -**to be implemented** +**implemented only in rhel8_preprod branch** Certain packages are only used on a given OS Version, so a `os=` with the OS name and the major version selects a package only for given OS, where as a `os!` will filter away given package on hosts with given OS, so they are not installed there. @@ -207,7 +207,7 @@ A small list of packages managed by the Linux Team. - make v4.3 from [CentOS](https://rpmfind.net/linux/RPM/centos-stream/9/baseos/x86_64/make-4.3-7.el9.x86_64.html) as v4.2.1 has been reported to to make trouble - latest [Zoom client](https://zoom.us/download?os=linux) - latest [Webex client](https://www.webex.com/downloads.html) -- `pli-assets` containing the PSI and the Customer Self Service logo, any hints about the source rpm are welcome -- TODO: [mod_gearman](https://mod-gearman.org/download/v4.0.1/rhel8/x86_64/) +- `pli-assets` containing the PSI and the Customer Self Service logo, any hints about the source rpm are welcome +- [mod_gearman v4.0.1](https://mod-gearman.org/download/v4.0.1/rhel8/x86_64/) From 342066fadef5241134968d72b1e5f95966e76be7 Mon Sep 17 00:00:00 2001 From: Konrad Bucheli Date: Fri, 1 Jul 2022 16:01:53 +0200 Subject: [PATCH 03/10] document new package repository configuration in puppet --- admin-guide/puppet/profiles/repository.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/admin-guide/puppet/profiles/repository.rst b/admin-guide/puppet/profiles/repository.rst index 0b7d7701..fe8ef01f 100644 --- a/admin-guide/puppet/profiles/repository.rst +++ b/admin-guide/puppet/profiles/repository.rst @@ -1,4 +1,4 @@ -``profile::repository_list`` +``profile::repository`` ============================ This module configures adds a list of package repositories from hiera From 3021ab7d8b74bfe72c7bd1eefe4cc7f33d576037 Mon Sep 17 00:00:00 2001 From: Konrad Bucheli Date: Fri, 1 Jul 2022 16:06:32 +0200 Subject: [PATCH 04/10] document new package repository configuration in puppet --- admin-guide/puppet/profiles/package_list.rst | 4 ++-- admin-guide/puppet/profiles/repository_list.rst | 14 ++++++++++++++ 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/admin-guide/puppet/profiles/package_list.rst b/admin-guide/puppet/profiles/package_list.rst index 313a8df8..c3bcf98a 100644 --- a/admin-guide/puppet/profiles/package_list.rst +++ b/admin-guide/puppet/profiles/package_list.rst @@ -3,9 +3,9 @@ This module installs a list of packages on the node, with extra options inlcuding the filtering by OS. -This profile is usually direcly used from Puppet:: +This profile is used from Puppet:: - profile::repository_list{'rpm_repos::influx':} + profile::package_list{'rpm_repos::influx':} Parameters ---------- diff --git a/admin-guide/puppet/profiles/repository_list.rst b/admin-guide/puppet/profiles/repository_list.rst index a70c6498..3251e4a4 100644 --- a/admin-guide/puppet/profiles/repository_list.rst +++ b/admin-guide/puppet/profiles/repository_list.rst @@ -4,6 +4,20 @@ This module configures adds a list of package repositories from hiera and filters those according to OS version. +This profile is used from Puppet:: + + profile::package_list{'icinga_client': + packages => [ + 'nrpe', + 'nrpe-selinux', + 'mod_gearman-static:os=redhat7', + 'mod_gearman:os!redhat7', + 'nagios-plugins-disk', + 'nagios-plugins-load', + ], + } + + Parameters ---------- From b09e22b788e0f4938b120c193c2710ffa06630b4 Mon Sep 17 00:00:00 2001 From: Konrad Bucheli Date: Tue, 5 Jul 2022 15:09:51 +0200 Subject: [PATCH 05/10] put the examples into the correct place --- admin-guide/puppet/profiles/package_list.rst | 14 ++++++++++++-- admin-guide/puppet/profiles/repository_list.rst | 12 +----------- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/admin-guide/puppet/profiles/package_list.rst b/admin-guide/puppet/profiles/package_list.rst index c3bcf98a..cde67285 100644 --- a/admin-guide/puppet/profiles/package_list.rst +++ b/admin-guide/puppet/profiles/package_list.rst @@ -1,11 +1,21 @@ ``profile::package_list`` ============================ -This module installs a list of packages on the node, with extra options inlcuding the filtering by OS. +This module installs a list of packages on the node, with extra options including package removal or selection by OS. This profile is used from Puppet:: - profile::package_list{'rpm_repos::influx':} + profile::package_list{'icinga_client': + packages => [ + 'nrpe', + 'nrpe-selinux', + 'mod_gearman-static:os=redhat7', + 'mod_gearman:os!redhat7', + 'nagios-plugins-disk', + 'nagios-plugins-load', + ], + } + Parameters ---------- diff --git a/admin-guide/puppet/profiles/repository_list.rst b/admin-guide/puppet/profiles/repository_list.rst index 3251e4a4..1c11cec9 100644 --- a/admin-guide/puppet/profiles/repository_list.rst +++ b/admin-guide/puppet/profiles/repository_list.rst @@ -6,17 +6,7 @@ and filters those according to OS version. This profile is used from Puppet:: - profile::package_list{'icinga_client': - packages => [ - 'nrpe', - 'nrpe-selinux', - 'mod_gearman-static:os=redhat7', - 'mod_gearman:os!redhat7', - 'nagios-plugins-disk', - 'nagios-plugins-load', - ], - } - + profile::repository_list{'rpm_repos::influx':} Parameters ---------- From 5b616f61ae307d1b3620dfa2c57fca56324f4973 Mon Sep 17 00:00:00 2001 From: Konrad Bucheli Date: Tue, 5 Jul 2022 15:51:20 +0200 Subject: [PATCH 06/10] title bold --- admin-guide/puppet/profiles/package_list.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/admin-guide/puppet/profiles/package_list.rst b/admin-guide/puppet/profiles/package_list.rst index cde67285..c0dc2576 100644 --- a/admin-guide/puppet/profiles/package_list.rst +++ b/admin-guide/puppet/profiles/package_list.rst @@ -34,7 +34,7 @@ packages list [] This list contains the names of the packages to be installed. The names may be "extended" by tags, separated by ``:``. ========== =============================== -Tag Function +**Tag** **Function** ---------- ------------------------------- absent package will be removed latest always latest version installed From 5ec8921ae0f0314655b6c7da91be0a17eee168ed Mon Sep 17 00:00:00 2001 From: Konrad Bucheli Date: Wed, 6 Jul 2022 21:03:10 +0200 Subject: [PATCH 07/10] extend example --- .../puppet/profiles/repository_list.rst | 20 ++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/admin-guide/puppet/profiles/repository_list.rst b/admin-guide/puppet/profiles/repository_list.rst index 1c11cec9..52dc7eba 100644 --- a/admin-guide/puppet/profiles/repository_list.rst +++ b/admin-guide/puppet/profiles/repository_list.rst @@ -6,7 +6,7 @@ and filters those according to OS version. This profile is used from Puppet:: - profile::repository_list{'rpm_repos::influx':} + profile::repository_list{'rpm_repos::epel':} Parameters ---------- @@ -30,7 +30,7 @@ then with the rest a repository configuration is writen to the node using ``profile::repository``. The ``osversion`` key in the repository definition signals the major RHEL version for which this repository is for. -Example definition:: +Example:: rpm_repos::repo::epel_rhel8: name: 'epel' @@ -43,9 +43,23 @@ Example definition:: exclude: - "slurm*" + rpm_repos::repo::epel_rhel7: + name: 'epel' + descr: "Extra Packages for Enterprise Linux 7" + baseurl: 'https://repo00.psi.ch/el7/tags/$pli_repo_tag/epel/' + disable: false + gpgcheck: false + osversion: 7 + exclude: + - "slurm*" + + rpm_repos::epel + - epel_rhel7 + - epel_rhel8 + ``accept_empty`` ~~~~~~~~~~~~~~~~ -Do not fail when the repository list is empty after +Do not fail when the repository list is empty after filtering away those not suitable for the current OS. From 86b30d1515340de71dcbbf6ff105eeace91f44f6 Mon Sep 17 00:00:00 2001 From: Konrad Bucheli Date: Wed, 6 Jul 2022 21:07:15 +0200 Subject: [PATCH 08/10] extend example --- admin-guide/puppet/profiles/repository_list.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/admin-guide/puppet/profiles/repository_list.rst b/admin-guide/puppet/profiles/repository_list.rst index 52dc7eba..a4fec414 100644 --- a/admin-guide/puppet/profiles/repository_list.rst +++ b/admin-guide/puppet/profiles/repository_list.rst @@ -56,7 +56,6 @@ Example:: rpm_repos::epel - epel_rhel7 - epel_rhel8 - ``accept_empty`` From b5744d880d622cc105a3284a2d4fce25fa81387a Mon Sep 17 00:00:00 2001 From: Konrad Bucheli Date: Mon, 11 Jul 2022 17:39:17 +0200 Subject: [PATCH 09/10] git-cvs is gone on RHEL8 --- rhel8/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/rhel8/index.md b/rhel8/index.md index 3fd990b6..6fa1f8d6 100644 --- a/rhel8/index.md +++ b/rhel8/index.md @@ -43,6 +43,7 @@ is hanging. Usually it is after the installation of `/etc/sssd/sssd.conf`. Just | `blt` | - | [`blt` upstream](http://blt.sourceforge.net/), does not work with newer Tk version ([source](https://wiki.tcl-lang.org/page/BLT)) | | ... | ... | here I stopped research, please report/document further packages | | `devtoolset*` | `gcc-toolset*` | | +| `git-cvs` | - | `cvs` itself is not supported by RHEL8, but available through EPEL. Still missing is the support for `git cvsimport`. | ### Installing a Machine with YFS and an Old Package Snapshot Tag After the `pli-firstboot` script it stopps due to errors (installed yfs kernel module version does not fit running kernel). Still it manages to install everything fine. From ba66d77e0b5af12e91e446b560aa18d64ab04acf Mon Sep 17 00:00:00 2001 From: Konrad Bucheli Date: Tue, 12 Jul 2022 13:08:15 +0200 Subject: [PATCH 10/10] document system_disk attribute --- admin-guide/deployment/partitioning.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/admin-guide/deployment/partitioning.rst b/admin-guide/deployment/partitioning.rst index 39625992..d354e06c 100644 --- a/admin-guide/deployment/partitioning.rst +++ b/admin-guide/deployment/partitioning.rst @@ -4,8 +4,14 @@ Partitioning Partitions system are configured with a standard schema using LVM, so that they can be possibly changed afterwards. -By default the whole space available on the first block device is used +By default the whole space available on the first block device is used and any existing partition is removed. + +Alternatively you might set the sysdb attribute ``system_disk`` with the +device name of the disk which should be used instead:: + + bob node set-attr $FQDN system_disk=md126 + The default partition schema is: - create one primary ``/boot`` partition of 1Gb;