diff --git a/_toc.yml b/_toc.yml index 4ae477b..1738ab1 100644 --- a/_toc.yml +++ b/_toc.yml @@ -22,7 +22,7 @@ parts: chapters: - file: infrastructure-guide/infrastructure_systems sections: - - file: infrastructure-guide/pxeserv01 + - file: infrastructure-guide/boot_server - file: infrastructure-guide/sysdb_server - file: infrastructure-guide/puppet01 - file: infrastructure-guide/repo00 diff --git a/admin-guide/deployment/_static/legacy_boot.puml b/admin-guide/deployment/_static/legacy_boot.puml index 2d5baea..86004ce 100644 --- a/admin-guide/deployment/_static/legacy_boot.puml +++ b/admin-guide/deployment/_static/legacy_boot.puml @@ -3,7 +3,7 @@ actor admin participant "BIOS Node\n(legacy boot)" as bootnode participant "QIP DHCP" as dhcp -participant "pxeserv01.psi.ch" as pxeserver +participant "boot.psi.ch" as pxeserver participant "sysdb.psi.ch" as sysdb participant "repo00.psi.ch" as reposerver diff --git a/admin-guide/deployment/_static/uefi_boot.puml b/admin-guide/deployment/_static/uefi_boot.puml index 221d763..f8cb179 100644 --- a/admin-guide/deployment/_static/uefi_boot.puml +++ b/admin-guide/deployment/_static/uefi_boot.puml @@ -3,7 +3,7 @@ actor admin participant "UEFI Node" as bootnode participant "QIP DHCP" as dhcp -participant "pxeserv01.psi.ch" as pxeserver +participant "boot.psi.ch" as pxeserver participant "sysdb.psi.ch" as sysdb participant "repo01.psi.ch" as reposerver diff --git a/admin-guide/legacy/installation.rst b/admin-guide/legacy/installation.rst deleted file mode 100644 index d097d67..0000000 --- a/admin-guide/legacy/installation.rst +++ /dev/null @@ -1,65 +0,0 @@ -Installation -============ - -This section describes the various installation methods, e.g. PXE. - -PXE Server ----------- - -The PXE server is ``pxeserv01.psi.ch``, running SL 5.1. - -Directory structure -~~~~~~~~~~~~~~~~~~~ - -The usual ``/tftpboot/pxelinux.cfg`` setup. The ``pxelinux.cfg`` -directory contains the following subdirectories: - -- ``Boot``: could be Windows-related -- ``centos``: CentOS 7.0, 7.1, and 7.2 -- ``dl``: diskless systems -- ``pxelinux.cfg``: PXELINUX configuration -- ``scientific``: Scientific Linux, 3.01 -> 7.0, partly obsolete -- ``tools``: - - -PXELINUX --------- - -The PXELINUX configuration files are in ``pxeserv01:/tftpboot``. - -The ``default`` file currently contains 127 labels. - - -Scientific Linux 6, 64bit -~~~~~~~~~~~~~~~~~~~~~~~~~ - -One thing to note is that the -[Kickstart file](http://linux.web.psi.ch/kickstart/configs/sl64-64-desk-stable-ks.cfg) -is modified during installation by the -[``ks_pre.sh``](http://linux.web.psi.ch/dist/scientific/64/kickstart/bin/ks_pre.sh) -script. - - -Kickstart ---------- - -The documentation on Kickstart files is currently based on -``sl64-64-desk-stable-ks.cfg``, the default Kickstart file for SL6. - -In the ``%pre`` section a distribution-specific script ``ks_pre.sh`` -is downloaded and executed. Errors are not handled. The script -*modifies* the Kickstart file. The latter contains markers -``#BEGIN_MAIN`` and ``#END_MAIN`` to mark the region to be modified. - - -Custom keys -~~~~~~~~~~~ - -The Kickstart files defines variables ``INIT_KEYS`` and -``APPEND_KEYS`` and exports them for use by ``ks_pre.sh``. The latter -attempts to download an environment file and a script -``custom_pre.sh`` for each key and sources them. - -The custom key related files are stored below -``/afs/psi.ch/software/linux/dist/scientific/$OSVERSION/custom/``. -Examples for custom keys are ``DesktopTesting`` and ``DesktopStable``. diff --git a/admin-guide/legacy/misc/prepareanewslrelease.rst b/admin-guide/legacy/misc/prepareanewslrelease.rst deleted file mode 100644 index 24d7052..0000000 --- a/admin-guide/legacy/misc/prepareanewslrelease.rst +++ /dev/null @@ -1,162 +0,0 @@ -Prepare a new release -===================== - -Introduction ------------- - -This document describes the preparation of a new SLP major or minor -release for SL >= 5.4. - -Requirements ------------- - -Before you start you need a clean copy of the original SL repositories -of interest from http://www.scientificlinux.org. - -The SL repositories shall be mirrored to our AFS mirror on its own -volume, e.g. `/afs/psi.ch/software/mirror/scientific/54/` is on AFS -volume `sw.mirror.sl54.nb`. - -However, the mirror is usually made automatically by the script -`/afs/psi.ch/software/mirror/scripts/mirror.sh` which is run by cron -on pxeserv01. To add a new distribution to the mirror script you have -to edit the file `/afs/psi.ch/software/mirror/scripts/mirror.conf`. - -You also need a separate AFS distribution volume, e.g. `sw.dist.sl54` -for `/afs/psi.ch/software/linux/dist/scientific/54`. - -Once this is done, you're ready to start with the Procedure section. - - -Procedure ---------- - -1. Go to the SL distribution B directory, change to the - subdirectory `scripts` and run the following scripts:: - - # cd /afs/psi.ch/software/linux/dist/scientific/ - # cd scripts/Sl60/ - # sh 1-create_the_new_dist-config_file.sh - # sh 2-copy_the_kernel_and_initrd_for_tftp_boot.sh - # sh 3-copy_the_puppet_environment_for_desktop.sh # If a new one should be used, - # usually only for major release - # changes - # Note: Some of the files might need some manual customization, e.g. - # - # module_gdm (pictures) - # module_splash (pictures) - # module_background_image (pictures) - # module_buildsystem - - - # sh 4-create_new_dist_directory_structure.sh - - ### Check and edit the new files if necessary, especially the following ones: - # scripts/dist-config - # scripts/KERNEL_MODULES_TO_BUILD/sl6.i386.kms-to-be-built - # scripts/KERNEL_MODULES_TO_BUILD/sl6.x86_64.kms-to-be-built - # scripts/build-gpfs.sl6x.sh - # scripts/build-openafs.sl6x.i686.sh - # scripts/build-openafs.sl6x.x86_64.sh - # scripts/copy_the_sl_base_repos_from_the_local_mirror.sh - # scripts/rsync_security_updates_to_all_directories.sh - - is the distribution which serves as a template for - . - - The scripts will prepare most of the stuff automatically based on - the previous version. Some of the new files, however, still have to - be edited manually. - -2. Go to the SL distribution directory, change to the subdirectory - `scripts` and run the following scripts:: - - # cd /afs/psi.ch/software/linux/dist/scientific/60/scripts/ - - 2.1 Copy the SL Base Repos - - The script below copies the sl6base repository from the mirror on - AFS to the dist SL6 directory taking into account the - `rsync_exclude.rpmlist`, i.e. not all RPMS will be copied. To keep - the repodata consistent with the RPMS in the Packages directory, - the repodata has to be updated, too, by running the second script - shown here:: - - # sh copy_the_sl_base_repos_from_the_local_mirror.sh - # sh run_createrepo_on_base_i386.sh #sl64: still to do - # sh run_createrepo_on_base_x86_64.sh #sl64: still to do - - 2.2 Copy the Security Updates:: - - # sh rsync_security_updates_to_all_directories.sh - - 2.3 Build kernel module if minor release - - If you prepare a new major release skip this section. - - If you prepare a new minor release you already have build systems - where you can build kernel modules. - - Thus check whether you have to build at least the openafs and the - gpfs modules. If the kernel versions of this and the previous SL - minor versions do not differ you should already have the modules - copied to e.g. the nonfree and psi repos, then there's nothing to - do here. - - 2.4 Update The Symbolic Links in RPMS_all:: - - # sh update_symlinks_in_RPMS_all.sh - -3. Create new snaphots, release UNSTABLE and set the initial PSI - version. - - # ./create_new_snapshots.sh alldirs > \ - # INITIAL_LOG/2011-02-25-create_new_snapshots.log 2>&1 - # ./release_unstable.sh - # ./create_new_psi_version.sh - # ./create_file_psiversion-info.sh - -6. Write the new kickstart conig files. - - Goto the kickstart config directory and use one of the old - kickstart config files as a template:: - - # cd /afs/psi.ch/software/linux/kickstart/configs/ - -8. Make the new entries in the tftpd file. - - Goto the tftpd directory and edit the file default:: - - # cd /afs/psi.ch/service/linux/tftpboot/pxelinux.cfg/ - # vi default - - #<<< sl64: hereiam: - #<<< to do: fix /usr/bin/psi-fix_file_permission, module_localadmin - -9. Now perform first installation tests and debug where necessary. - -10. If you prepare a new major release, install new build systems. - -12. If you prepare a new major release, go to - `/afs/psi.ch/project/linux/dist` and create the directory - structure below for the RPM build environment:: - - slp6 - |-- RPMS/ - | |-- i386/ - | |-- i486/ - | |-- i586/ - | |-- i686/ - | |-- noarch/ - | `-- x86_64/ - `-- SRPMS/ - -14. Build kernel modules. - -15. Also get new software versions for non SL repos (psi, others, - etc.). - -16. Again create new snapshots and a new PSI version. - -17. Now perform the second test installation and debugging suite. - diff --git a/admin-guide/legacy/misc/projectpsi-puppet1.rst b/admin-guide/legacy/misc/projectpsi-puppet1.rst deleted file mode 100644 index 867a8f0..0000000 --- a/admin-guide/legacy/misc/projectpsi-puppet1.rst +++ /dev/null @@ -1,1070 +0,0 @@ -Project `psi-puppet1` -===================== - -Introduction ------------- - -This document describes the relaunch of the puppet service -infrastructure at PSI. - -The whole project can be divided into two parts: - -- The hard- and software setup for the puppet server and client. - (Described in this document.) - -- Implementation of the configuration for puppet clients. (Described - in [[PuppetManifestsForSL53][Puppet Manifests For SL53]].) - - -Objectives ----------- - -- To get a stable, scalable and easy to manage puppet service - infrastructure. - -- To gain a better overview of the various client configurations - configured by puppet. - -- To keep a clear and up-to-date documentation. - -- To keep the different configurations of the different SL releases - separated from each other, e.g. SL 5.1 does not overlap with SL 5.3. - -- Also other users from AIT and GFA than the puppet administrator - should have the possibility to use puppet to configure their hosts. - -- The different client configurations of the different puppet users - must not interfere with each other. - -- To manage the changes to manifests and client configuration files. - -- Easy recovery of files in case of data loss. - -- Easy and fast reinstallation of an identical puppet server in case - of an irreparable server crash. - - -Description of the Basic Server Setup -------------------------------------- - -- Hardware - - Dell Power Edge 1750 - -- Operating System - - SL51 32 bit Server - -- Network configuration - - Static IP for Production Server: 129.129.190.174/24 - Hostname: psi-puppet1.psi.ch - -- Required RPMS - - - puppet-server (http://download.fedora.redhat.com/pub/epel/5/i386/repoview/) - - augeas-libs - - facter - - puppet (http://download.fedora.redhat.com/pub/epel/5/i386/repoview/) - - ruby - - ruby-augeas - - ruby-libs - - ruby-shadow - - ruby-irb (required for reading help) - - ruby-rdoc (required for reading help) - - -Procedure ---------- - -Create a Repository for Puppet Related RPMS -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -First make the directory in the SL51 installation tree:: - - # mkdir /afs/psi.ch/software/linux/dist/scientific/51/puppet-0247 - -Add the following RPMS to this repository and run `createrepo`:: - - puppet-server-0.24.7-4.el5.noarch.rpm - augeas-libs-0.3.5-1.el5.i386.rpm - facter-1.5.2-2.el5.noarch.rpm - puppet-0.24.7-4.el5.noarch.rpm - ruby-augeas-0.2.0-1.el5.i386.rpm - ruby-shadow-1.4.1-7.el5.i386.rpm - - # cd /afs/psi.ch/software/linux/dist/scientific/51/puppet-0247 - # createrepo . - -To enable the access to this repo create the yum repo file -`/etc/yum.repos.d/puppet-0247.repo` on the puppet server:: - - [puppet-0247] - name=puppet-0247 for SL5 - baseurl=http://linux.web.psi.ch/dist/scientific/5/puppet-0247/ - enabled=1 - - -Setup The Puppet Server -~~~~~~~~~~~~~~~~~~~~~~~ - -Basic Server Installation -......................... - -Install SL51, class Server via PXE boot and kickstart. - - -Puppet-Server Installation -.......................... - -Install `puppet-server` with yum. This will also draw the required -dependencies:: - - # [root@psi-puppet1] - # yum install puppet-server - - ... - Finished Kernel Module Plugin - - Dependencies Resolved - - ============================================================================= - Package Arch Version Repository Size - ============================================================================= - Installing: - puppet-server noarch 0.24.7-4.el5 puppet-0247 25 k - Installing for dependencies: - augeas-libs i386 0.3.5-1.el5 puppet-0247 151 k - facter noarch 1.5.2-2.el5 puppet-0247 41 k - puppet noarch 0.24.7-4.el5 puppet-0247 548 k - ruby i386 1.8.5-5.el5_2.6 sl5update 279 k - ruby-augeas i386 0.2.0-1.el5 puppet-0247 17 k - ruby-libs i386 1.8.5-5.el5_2.6 sl5update 1.6 M - ruby-shadow i386 1.4.1-7.el5 puppet-0247 9.5 k - - Transaction Summary - ============================================================================= - Install 8 Package(s) - Update 0 Package(s) - Remove 0 Package(s) - ... - - -Configure The Puppet Server -........................... - -The configuration files of the puppet server, directory -`/etc/puppet/`, are stored locally. - -The puppet client configuration files are stored on AFS. The mountpoint on -psi-puppet1 is `/var/puppet/environments`, thus -create the directory `/var/puppet/environments`. - - # mkdir -p /var/puppet/environments - -For how to mount the AFS see section `Mount AFS Volumes` below. - -The client configuration files in /var/puppet/environments are -described at [[PuppetManifestsForSL53][Puppet Manifests For SL53]]. - -The log is on the local disk in `/var/log/puppet`. -To set the logfile edit the line `PUPPETMASTER_OPTS` -in `/etc/rc.d/init.d/puppetmaster`. For testing also the -debug option `-d` is enabled:: - - PUPPETMASTER_OPTS="-v -d -l /var/log/puppet/puppetmaster.log" - -Config file `puppet.conf`:: - - ########################################################################### - # $Header: /etc/puppet/RCS/puppet.conf,v 1.3 2009/09/07 18:11:17 root Exp root $ - # - # Puppetmaster Environments - # ========================= - # - # Ref.: http://reductivelabs.com/trac/puppet/wiki/UsingMultipleEnvironments - # - # Marc Gasser, PSI - # last modified 2011-11-18 - # - ############################################################################ - [main] - # Where Puppet stores dynamic and growing data. - # The default value is '/var/puppet'. - vardir = /var/puppet - - # The Puppet log directory. - # The default value is '$vardir/log'. - # logdir = /afs/psi.ch/service/linux/puppet/var/log - logdir = /var/log/puppet - - # Where Puppet PID files are kept. - # The default value is '$vardir/run'. - rundir = /var/run/puppet - - # Where SSL certificates are kept. - # The default value is '$confdir/ssl'. - ssldir = $vardir/ssl - - # Whether log files should always flush to disk. - # The default value is false - autoflush = true - - - - [puppetmasterd] - reports = store - #reports = store , tagmail, rrdgraph - - # tagmap = $confdir/tagmail.conf - - #rrddir = $vardir/rrd - #rrdinterval = $runinterval - #rrdgraph = true - - [puppetd] - # The file in which puppetd stores a list of the classes - # associated with the retrieved configuratiion. Can be loaded in - # the separate ``puppet`` executable using the ``--loadclasses`` - # option. - # The default value is '$confdir/classes.txt'. - classfile = $vardir/classes.txt - - # Where puppetd caches the local configuration. An - # extension indicating the cache format is added automatically. - # The default value is '$confdir/localconfig'. - localconfig = $vardir/localconfig - - # Note: The port that the client daemon listens on, defaults to - # 8139. However, at PSI we run puppetd via the psi-puppet - # script with run onetime option enabled. - # psi-puppet is triggered by cron. - - ######################### - ####### SL 5 ######## - ######################### - - ### begin{ SL 5 (SL54), gasser_m - [DesktopSL5Unstable] - manifest = /var/puppet/environments/DesktopSL5Unstable/manifests/site.pp - modulepath = /var/puppet/environments/DesktopSL5Unstable/modules - - [ServerSL5Unstable] - manifest = /var/puppet/environments/ServerSL5Unstable/manifests/site.pp - modulepath = /var/puppet/environments/ServerSL5Unstable/modules - - [DesktopSL5Testing] - manifest = /var/puppet/environments/DesktopSL5Testing/manifests/site.pp - modulepath = /var/puppet/environments/DesktopSL5Testing/modules - - [DesktopSL5Stable] - manifest = /var/puppet/environments/DesktopSL5Stable/manifests/site.pp - modulepath = /var/puppet/environments/DesktopSL5Stable/modules - - [CPT] - manifest = /var/puppet/environments/CPT/manifests/site.pp - modulepath = /var/puppet/environments/CPT/modules - - ###}end SL 5 (SL54), gasser_m - - ### V.M. for sl53-c-ks.cfg - [CnodeSL5] - manifest = /var/puppet/environments/CnodeSL5/manifests/site.pp - modulepath = /var/puppet/environments/CnodeSL5/modules - - [PHServerSL5] - manifest = /var/puppet/environments/PHServerSL5/manifests/site.pp - modulepath = /var/puppet/environments/PHServerSL5/modules - - [EdgarDevelopment] - manifest = /var/puppet/environments/EdgarDevelopment/manifests/site.pp - modulepath = /var/puppet/environments/EdgarDevelopment/modules - - [DerekDevelopment] - manifest = /var/puppet/environments/DerekDevelopment/manifests/site.pp - modulepath = /var/puppet/environments/DerekDevelopment/modules - - [cray] - manifest = /var/puppet/environments/cray/manifests/site.pp - modulepath = /var/puppet/environments/cray/modules - - - ### begin Heiner{ - [HeinerDevelopment] - manifest = /var/puppet/environments/HeinerDevelopment/manifests/site.pp - modulepath = /var/puppet/environments/HeinerDevelopment/modules - [HeinerDevelopment54] - manifest = /var/puppet/environments/HeinerDevelopment54/manifests/site.pp - modulepath = /var/puppet/environments/HeinerDevelopment54/modules - [GFA] - manifest = /var/puppet/environments/GFA/manifests/site.pp - modulepath = /var/puppet/environments/GFA/modules - ### }end Heiner - - ### begin Rene{ - [GFADesktopSL5] - manifest = /var/puppet/environments/GFADesktopSL5/manifests/site.pp - modulepath = /var/puppet/environments/GFADesktopSL5/modules - - [GFADesktopSL6] - manifest = /var/puppet/environments/GFADesktopSL6/manifests/site.pp - modulepath = /var/puppet/environments/GFADesktopSL6/modules - ### }end Rene - - ### Services - [Web] - manifest = /var/puppet/environments/Web/manifests/site.pp - modulepath = /var/puppet/environments/Web/modules - - [Virtual] - manifest = /var/puppet/environments/Virtual/manifests/site.pp - modulepath = /var/puppet/environments/Virtual/modules - - [News] - manifest = /var/puppet/environments/News/manifests/site.pp - modulepath = /var/puppet/environments/News/modules - - [MySQL] - manifest = /var/puppet/environments/MySQL/manifests/site.pp - modulepath = /var/puppet/environments/MySQL/modules - - [Loadbalancer] - manifest = /var/puppet/environments/Loadbalancer/manifests/site.pp - modulepath = /var/puppet/environments/Loadbalancer/modules - - [LlcLoadbalancer] - manifest = /var/puppet/environments/LlcLoadbalancer/manifests/site.pp - modulepath = /var/puppet/environments/LlcLoadbalancer/modules - - [License] - manifest = /var/puppet/environments/License/manifests/site.pp - modulepath = /var/puppet/environments/License/modules - - [FTP] - manifest = /var/puppet/environments/FTP/manifests/site.pp - modulepath = /var/puppet/environments/FTP/modules - - [Elog] - manifest = /var/puppet/environments/Elog/manifests/site.pp - modulepath = /var/puppet/environments/Elog/modules - - [Cups] - manifest = /var/puppet/environments/Cups/manifests/site.pp - modulepath = /var/puppet/environments/Cups/modules - - [Archive] - manifest = /var/puppet/environments/Archive/manifests/site.pp - modulepath = /var/puppet/environments/Archive/modules - - - ######################### - ####### SL 6 ######## - ######################### - - ### begin{ SL 6 (gasser_m) - [DesktopSL6Unstable] - manifest = /var/puppet/environments/DesktopSL6Unstable/manifests/site.pp - modulepath = /var/puppet/environments/DesktopSL6Unstable/modules - - [DesktopSL6Testing] - manifest = /var/puppet/environments/DesktopSL6Testing/manifests/site.pp - modulepath = /var/puppet/environments/DesktopSL6Testing/modules - - [DesktopSL6Stable] - manifest = /var/puppet/environments/DesktopSL6Stable/manifests/site.pp - modulepath = /var/puppet/environments/DesktopSL6Stable/modules - - ###}end SL 6 (gasser_m) - - ### Markushin - [CnodeSL6] - manifest = /var/puppet/environments/CnodeSL6/manifests/site.pp - modulepath = /var/puppet/environments/CnodeSL6/modules - - -Config file `fileserver.conf`:: - - # This file consists of arbitrarily named sections/modules - # defining where files are served from and to whom - - # Define a section 'files' - # Adapt the allow/deny settings to your needs. Order - # for allow/deny does not matter, allow always takes precedence - # over deny - # [files] - # path /var/lib/puppet/files - # allow *.example.com - # deny *.evil.example.com - # allow 192.168.0.0/24 - - - #[facts] - # path /etc/puppet/facts - # allow *.psi.ch - - [GFA5] - path /afs/psi.ch/project/slscomp/puppet/gfa5 - allow *.psi.ch - - [GFA6] - path /afs/psi.ch/project/slscomp/puppet/gfa6 - allow *.psi.ch - - -Mount AFS Volumes on Puppet Server -.................................. - -The puppet manifests for clients are located on AFS:: - - /afs/psi.ch/service/linux/puppet/var/puppet/environments/ - -AFS is already mounted as `/afs` in this default SL5 server installation:: - - # mount - ... - AFS on /afs type afs (rw) - -Now, we want to remount -`/afs/psi.ch/service/linux/puppet/var/puppet/environments` on -`/var/puppet/environments`. Therefor the mount option `bind` is used, -which facilitates to remount parts of already mounted filesystems on -an alternative location in the file hierarchy. - -The server also needs the permission on AFS to mount the environments -directory. Add the new server to the AFS group -svc.linux:puppet_hosts:: - - # pts ad -u -g svc.linux:puppet_hosts - -As shown below we do the remount in `/etc/rc.local`, which is executed -after all the other init scripts:: - - #!/bin/sh - - touch /var/lock/subsys/local - - # Puppet - mount -o bind /afs/psi.ch/service/linux/puppet/etc/puppet/environments /var/puppet/environments - - # Restart Services depending on afs mounts - /etc/init.d/puppetmaster restart - -Before the `rc.local` script can be applied the proper AFS permissions -have to be set to make the files readable for psi-puppet1. This was -done already before, see topic -[[PuppetServerPsiPuppet2ForSl51#4_1_3_Mount_AFS_Volumes_on_Puppe][Puppet -Server Psi Puppet 2 For SL51]], so we only have to put the IP address -of psi-puppet1 to the AFS group `svc_linux:puppet_hosts`:: - - # pts adduser 129.129.190.174 svc_linux:puppet_hosts - -Configuring Puppet Reporting -............................ - -There are a number of different report processors available on the -puppetmaster. The default report, store, simply stores the report file -on the disk. - -By default, each client is configured not to report back to the -master. It has to be enabled either by the report option in -`puppet.conf` or using `--report` on the command line. - -`/etc/puppet/puppet.conf`:: - - [puppetd] - report = true - -Command line:: - - # puppetd --report - -Store Report Processor -,,,,,,,,,,,,,,,,,,,,,, - -Enable the store reports by using the `reports` -configuration option in the puppemasterd section of the -`puppet.conf` file on the master. - -`/etc/puppet/puppet.conf`:: - - [puppetmasterd] - reports = store - -The default reports directory is `$vardir/reports`. - -Tagmail Report Processor -,,,,,,,,,,,,,,,,,,,,,,,, - -Enable the tagmail reports by using the `reports` configuration option -in the puppemasterd section of the `puppet.conf` file on the -master. The `tagmail.conf` file contains a list of tags and email -adresses. The special tags all and err are defined implicitly. - -`/etc/puppet/puppet.conf`:: - - [puppetmasterd] - reports = tagmail - tagmap = $confdir/tagmail.conf - -`/etc/puppet/tagmail.conf`:: - - all: marc.gasser@psi.ch - err: marc.gasser@psi.ch - - -Rrdgraph Report Processors -,,,,,,,,,,,,,,,,,,,,,,,,,, - -To enable the `rrdgraph` reports, `rrdtool` and `rrdtool-ruby` -packages have to be installed. - -Download the packages from the following repository: -`/etc/yum.repos.d/epeli386.repo`:: - - [epeli386] - name=epel i386 - baseurl=http://download.fedora.redhat.com/pub/epel/5/i386/ - enabled=0 - - - # yumdownloader --enablerepo=epeli386 rrdtool.i386 rrdtool-ruby.i386 - # yum install rrdtool-1.2.27-3.el5.i386.rpm - # yum install rrdtool-ruby-1.2.27-3.el5.i386.rpm - -You might want to put them to your local repository, too. - -**Note**: For the time being put them to `psi-beta`, because they -break dependencies in the other repositories. - -Then, configure `puppet.conf` by adding the lines shown below in the -corresponding section. Here store, tagmail and rrdgraph are enabled. - -`/etc/puppet/puppet.conf`:: - - [puppetmasterd] - reports = store, tagmail, rrdgraph - - rrddir = $vardir/rrd - rrdinterval = $runinterval - rrdgraph = true - -Install The Ganglia Monitor Daemon -.................................. - -Install `ganglia-gmond-3.0.6-4.slp5` and add the configuration -`/etc/gmond.conf` file as shown below:: - - /* This configuration is as close to 2.5.x default behavior as possible - The values closely match ./gmond/metric.h definitions in 2.5.x */ - globals { - daemonize = yes - setuid = yes - user = nobody - debug_level = 0 - max_udp_msg_len = 1472 - mute = no - deaf = no - host_dmax = 0 /*secs */ - cleanup_threshold = 300 /*secs */ - gexec = no - } - - /* If a cluster attribute is specified, then all gmond hosts are wrapped inside - * of a tag. If you do not specify a cluster tag, then all will - * NOT be wrapped inside of a tag. */ - cluster { - name = "puppet" - owner = "unspecified" - latlong = "unspecified" - url = "unspecified" - } - - /* The host section describes attributes of the host, like the location */ - host { - location = "unspecified" - } - - /* Feel free to specify as many udp_send_channels as you like. Gmond - used to only support having a single channel */ - udp_send_channel { - mcast_join = 239.129.190.89 - port = 8649 - } - - /* You can specify as many udp_recv_channels as you like as well. */ - udp_recv_channel { - mcast_join = 239.129.190.89 - port = 8649 - bind = 239.129.190.89 - } - - # udp_recv_channel { - # host = "puppet" - # port = 8649 - # } - - /* You can specify as many tcp_accept_channels as you like to share - an xml description of the state of the cluster */ - tcp_accept_channel { - port = 8649 - } - - - /* The old internal 2.5.x metric array has been replaced by the following - collection_group directives. What follows is the default behavior for - collecting and sending metrics that is as close to 2.5.x behavior as - possible. */ - - /* This collection group will cause a heartbeat (or beacon) to be sent every - 20 seconds. In the heartbeat is the GMOND_STARTED data which expresses - the age of the running gmond. */ - collection_group { - collect_once = yes - time_threshold = 20 - metric { - name = "heartbeat" - } - } - - /* This collection group will send general info about this host every 1200 secs. - This information doesn't change between reboots and is only collected once. */ - collection_group { - collect_once = yes - time_threshold = 1200 - metric { - name = "cpu_num" - } - metric { - name = "cpu_speed" - } - metric { - name = "mem_total" - } - /* Should this be here? Swap can be added/removed between reboots. */ - metric { - name = "swap_total" - } - metric { - name = "boottime" - } - metric { - name = "machine_type" - } - metric { - name = "os_name" - } - metric { - name = "os_release" - } - metric { - name = "location" - } - } - - /* This collection group will send the status of gexecd for this host every 300 secs */ - /* Unlike 2.5.x the default behavior is to report gexecd OFF. */ - collection_group { - collect_once = yes - time_threshold = 300 - metric { - name = "gexec" - } - } - - /* This collection group will collect the CPU status info every 20 secs. - The time threshold is set to 90 seconds. In honesty, this time_threshold could be - set significantly higher to reduce unneccessary network chatter. */ - collection_group { - collect_every = 20 - time_threshold = 90 - /* CPU status */ - metric { - name = "cpu_user" - value_threshold = "1.0" - } - metric { - name = "cpu_system" - value_threshold = "1.0" - } - metric { - name = "cpu_idle" - value_threshold = "5.0" - } - metric { - name = "cpu_nice" - value_threshold = "1.0" - } - metric { - name = "cpu_aidle" - value_threshold = "5.0" - } - metric { - name = "cpu_wio" - value_threshold = "1.0" - } - /* The next two metrics are optional if you want more detail... - ... since they are accounted for in cpu_system. - metric { - name = "cpu_intr" - value_threshold = "1.0" - } - metric { - name = "cpu_sintr" - value_threshold = "1.0" - } - */ - } - - collection_group { - collect_every = 20 - time_threshold = 90 - /* Load Averages */ - metric { - name = "load_one" - value_threshold = "1.0" - } - metric { - name = "load_five" - value_threshold = "1.0" - } - metric { - name = "load_fifteen" - value_threshold = "1.0" - } - } - - /* This group collects the number of running and total processes */ - collection_group { - collect_every = 80 - time_threshold = 950 - metric { - name = "proc_run" - value_threshold = "1.0" - } - metric { - name = "proc_total" - value_threshold = "1.0" - } - } - - /* This collection group grabs the volatile memory metrics every 40 secs and - sends them at least every 180 secs. This time_threshold can be increased - significantly to reduce unneeded network traffic. */ - collection_group { - collect_every = 40 - time_threshold = 180 - metric { - name = "mem_free" - value_threshold = "1024.0" - } - metric { - name = "mem_shared" - value_threshold = "1024.0" - } - metric { - name = "mem_buffers" - value_threshold = "1024.0" - } - metric { - name = "mem_cached" - value_threshold = "1024.0" - } - metric { - name = "swap_free" - value_threshold = "1024.0" - } - } - - collection_group { - collect_every = 40 - time_threshold = 300 - metric { - name = "bytes_out" - value_threshold = 4096 - } - metric { - name = "bytes_in" - value_threshold = 4096 - } - metric { - name = "pkts_in" - value_threshold = 256 - } - metric { - name = "pkts_out" - value_threshold = 256 - } - } - - /* Different than 2.5.x default since the old config made no sense */ - collection_group { - collect_every = 1800 - time_threshold = 3600 - metric { - name = "disk_total" - value_threshold = 1.0 - } - } - - collection_group { - collect_every = 40 - time_threshold = 180 - metric { - name = "disk_free" - value_threshold = 1.0 - } - metric { - name = "part_max_used" - value_threshold = 1.0 - } - } - - # /etc/init.d/gmond start - - -See puppet at http://129.129.190.27/ganglia/. For the ganglia server -configuration ask Valeri Markushin. - - -Install The Networker Backup Client (Legato) -............................................ - -References: - -- [[http://ait.web.psi.ch/services/central_backup/][Information for Backup Client Administrators]] - -- [[http://ait.web.psi.ch/services/central_backup/nsr_install_lnx.html][Networker Client installation on Linux ]] - -Install the Networker client packages, the client itself and the -manual pages. By default `yum` calculates a lot of dependencies -required for the GUI of Networker, which facilitates the -restore. However, the restore can also be done using the command line -interface, thus the whole X installation shall be skipped. To do so, -the packages have to be installed without dependencies. - -Because `yum` does not provide an installation without dependencies, -`yumdownloader` is used to fetch the packages and `rpm -i --nodeps` to -install them. - -First install `yumdownloader`:: - - # yum install yum-utils - -Install the rest:: - - # yumdownloader --enablerepo=psi-beta lgtoclnt.i686 lgtoman.i686 - # rpm -ivh --nodeps lgtoclnt-7.4.2-1.i686.rpm lgtoman-7.4.2-1.i686.rpm - -Start the Networker daemon:: - - # service networker start - -The `/nsr` directory is automatically created. Add the string -`bs1.psi.ch` in the file `/nsr/res/server`. - -Restart the Networker daemon:: - - # service networker stop - # service networker start - -Now, contact the backup server administrator, Marco Kohler, so he can -add the host and the directories of interest to the backup service. - -The next steps are for facilitating the task of the backup server -administrator. - -Create the file `~/nsradmin74_x.txt` with the following three lines:: - - update administrator:"isroot,host=psi-puppet1","isroot,host=localhost","isroot,host=bs1","user=root,host=localhost","user=administrator,host=bs1" - . type: NSR System Port Ranges - update administrator:"isroot,host=psi-puppet1","isroot,host=localhost","isroot,host=bs1","user=root,host=localhost","user=administrator,host=bs1" - - -Then execute the command below and check the output:: - - # nsradmin -i ~/nsradmin74_x.txt -p nsrexec - - updated resource id 3.0.104.17.41.235.57.74.129.129.190.174(7) - updated resource id 9.0.104.17.41.235.57.74.129.129.190.174(2) - updated resource id 8.0.168.18.5.236.57.74.129.129.190.174(2) - updated resource id 9.0.168.18.5.236.57.74.129.129.190.174(2) - Current query set - updated resource id 7.0.104.17.41.235.57.74.129.129.190.174(2) - - -Finally, test if the installation was successful:: - - # service networker stop - # service networker start - # service networker status - +--o nsrexecd (5995) - -**Note**: Open files will not necessarily be considered during the -backup run. It depends on their locking state. - - -How To Update the Networker Backup Client -......................................... - -Because the Networker RPM is not cleanly packed, updating the client -requires deinstallation of the old and installation of the new -package. - -First the old `/nsr` directory has to be deleted. Then repeat the -whole procedure shown in the previous section. - -The Networker Administration Program -.................................... - -To start the Networker administration shell type the following -command:: - - # nsradmin -p nsrexec - -The Networker Recover Tool -.......................... - -Check out the manpage of `recover`:: - - # man recover - - -Setup The Puppet Client -~~~~~~~~~~~~~~~~~~~~~~~ - -At this time the only difference between the old and the new client -configuration is the name of the puppet server in the file -`/etc/puppet/puppet.conf`, `psi-puppet1` instead of `pxeserv01`. - -File `/etc/puppet/puppet.conf` on `vmmarctest1.psi.ch`:: - - [main] - vardir = /var/puppet - logdir = /var/log/puppet - rundir = /var/run/puppet - ssldir = $vardir/ssl - environment = development - - [puppetd] - classfile = $vardir/classes.txt - localconfig = $vardir/localconfig - factsync = true - server = psi-puppet1.psi.ch - - -Because the new puppet server refers to the same sources (files) as -the current productive server, we set the immutable to the file above, -otherwise next time `puppetd` is running the server entry will be -changed to `pxeserv01` again. - -The sources are located at -`/afs/psi.ch/software/linux/dist/scientific/51/puppet/files/` on -AFS. This path is set in the file `/etc/puppet/fileserver.conf` on the -puppet server. - -Make First Tests -~~~~~~~~~~~~~~~~ - -Start the puppetmaster:: - - # /etc/init.d/puppetmaster start - -Test it with a client (the options are: keep process in the -foreground, run onetime and be verbose):: - - # [root@vmmarctest1 ~] - # puppetd --no-daemonize -o -v - -Or run the client in no operational mode, i.e. dry runs without -actually applying the configuration:: - - # puppetd --noop --no-daemonize -o -v - - info: Loading fact sysconfig_psi - info: Loading fact sysconfig_psi-gfa - info: Creating a new certificate request for vmmarctest1.psi.ch - info: Creating a new SSL key at /var/puppet/ssl/private_keys/vmmarctest1.psi.ch.pem - warning: peer certificate won't be verified in this SSL session - notice: Got signed certificate - info: Retrieving facts - info: Loading fact sysconfig_psi - info: Loading fact sysconfig_psi-gfa - info: Caching catalog at /var/puppet/localconfig.yaml - notice: Starting catalog run - notice: //Node[default]/psi_localadmin/Exec[/usr/bin/psi-fix_file_permission >/dev/null]/returns: executed successfully - info: Filebucket[/var/puppet/clientbucket]: Adding /usr/share/texmf/dvips/config/config.ps(1611c4bb4b35341f1945059ff774c6df) - notice: //Node[default]/psi_base/File[/usr/share/texmf/dvips/config/config.ps]: Filebucketed to with sum 1611c4bb4b35341f1945059ff774c6df - notice: //Node[default]/psi_base/File[/usr/share/texmf/dvips/config/config.ps]/source: replacing from source puppet://psi-puppet1.psi.ch/51/Desktop/usr/share/texmf/dvips/config/config.ps with contents {md5}b265606dc098a5414f3acd71a8831ef1 - notice: //Node[default]/psi_puppet/File[/etc/puppet/puppet.conf]/checksum: checksum changed '{md5}f2944bb81bfbe22b2a2ac4c9197563f3' to '{md5}be67850ccad5409063a56de9d5a516d3' - notice: //Node[default]/psi_puppet/File[/etc/puppet/puppet.conf]: Filebucketed to with sum be67850ccad5409063a56de9d5a516d3 - err: //Node[default]/psi_puppet/File[/etc/puppet/puppet.conf]: Could not rename tmp /etc/puppet/puppet.conf for replacing: Operation not permitted - /etc/puppet/puppet.conf.puppettmp or /etc/puppet/puppet.conf - notice: //Node[default]/psi_puppet/File[/etc/puppet/puppet.conf]/source: replacing from source puppet://psi-puppet1.psi.ch/51/Desktop/etc/puppet/puppet.conf.testing with contents {md5}f2944bb81bfbe22b2a2ac4c9197563f3 - info: Filebucket[/var/puppet/clientbucket]: Adding /etc/sysctl.conf(d5716d328f5b840eb4e13ae1d2896fe9) - notice: //Node[default]/psi_base/File[/etc/sysctl.conf]: Filebucketed to with sum d5716d328f5b840eb4e13ae1d2896fe9 - notice: //Node[default]/psi_base/File[/etc/sysctl.conf]/source: replacing from source puppet://psi-puppet1.psi.ch/51/Desktop/etc/sysctl.conf with contents {md5}d576ff606d3f93df26965e7ef364bd07 - notice: //Node[default]/psi_yum/Exec[/usr/sbin/psi-get-yumconf]/returns: executed successfully - notice: Finished catalog run in 6.22 seconds - - -So, this looks promising. Seems like the client could get it's -configuration from the new puppet server. - -Only the file `/etc/puppet/puppet.conf` could not be changed, what is -ok because the immutable flag was set. - -Next Steps ----------- - -- Verify migration order (server, client or vice versa) - -- Finalize basic server setup (verify that no config agents compromise - the system, e.g. puppetd which could be executed by cron or during - boot time, etc.), check whether it makes sense to use DNS aliases - for the hostname. - -- Shall server configuration files be stored locally or mounted from - AFS? - - -`psi-puppet1:/etc/rc.d/rc.local` has been prepared (not activated yet) -for the AFS mount:: - - #!/bin/sh - # - # This script will be executed *after* all the other init scripts. - # You can put your own initialization stuff in here if you don't - # want to do the full Sys V style init stuff. - - touch /var/lock/subsys/local - - # Puppet - #mount -o bind /afs/psi.ch/service/linux/puppet/etc/puppet-0.24.7-4 /etc/puppet - - # Restart Services depending on afs mounts - #/etc/init.d/puppetmaster restart - -The whole current puppetserver configuration from /etc/puppet/ was -copied to `/afs/psi.ch/service/linux/puppet/etc/puppet-0.24.7-4`. - -- If mounted from AFS the question remains how root@psi-puppet1 gets - the permission to mount the mentioned AFS directory. - -- Shall the client configuration manifests be stored locally or on - AFS? - - -Locally: `/var/puppet/environments/` - -AFS: `/afs/psi.ch/service/linux/puppet/etc/puppet-0.24.7-4/environments/` - -- Run the puppetmaster on hardware or vmware? Hardware. - -- When the server is going to production the IP has to be changed, see - *Static IP for Production Server* above. Done. - -- When the server is going to production the PSI firewall has to be - adjusted. (Refer to Tobias) - -- Test with old client to new server, and new client to old server. - -- Test with limited number of new client to new server. diff --git a/admin-guide/legacy/misc/psi-puppet2_installation.rst b/admin-guide/legacy/misc/psi-puppet2_installation.rst deleted file mode 100644 index 7acf68e..0000000 --- a/admin-guide/legacy/misc/psi-puppet2_installation.rst +++ /dev/null @@ -1,609 +0,0 @@ -Installation `psi-puppet2` -========================== - -References ----------- - -- https://wiki.intranet.psi.ch/AIT/PuppetServer2009 - -- http://reductivelabs.com/trac/puppet/wiki/PluginsInModules - - -Introduction ------------- - -This document describes the Installation of the new puppet server -psi-puppet2, which replaces the puppet server running on pxeserv01. - -The psi-puppet2 server is an interim solution for the pxeserv01 -replacement for the following reasons: - -- The puppet service on pxeserv01 is very unstable. - -- The performance on pxeserv01 is very bad, what makes work almost impossible. - -- The psi-puppet1 server is not ready for production yet. - -The new system will be setup as a 32 bit SL5 server with the latest -puppet server package and dependencies installed. - -The puppet server configuration is the same as the one on pxeserv01. - -**Note**: Don't confuse psi-puppet2 with psi-puppet1, which not only -is thought as a replacement of the puppet server on pxeserv01 but also -comes with a reimplementation of the whole puppet configuration, -whereas psi-puppet2 takes over the puppet configuration from -pxeserv01. - - -Description of the Basic Server Setup -------------------------------------- - -- Hardware - - Dell Power Edge 1750 - -- Operating System - - SL51 32 bit Server - -- Network configuration - - Static IP for Production Server: 129.129.190.160/24 - Hostname: psi-puppet2.psi.ch - -- Required RPMS - - - puppet-server (http://download.fedora.redhat.com/pub/epel/5/i386/repoview/) - - augeas-libs - - facter - - puppet (http://download.fedora.redhat.com/pub/epel/5/i386/repoview/) - - ruby - - ruby-augeas - - ruby-libs - - ruby-shadow - - ruby-irb (required for reading help) - - ruby-rdoc (required for reading help) - - -Procedure ---------- - -Setup The Puppet Server -~~~~~~~~~~~~~~~~~~~~~~~ - -Install The Puppet Server Software -.................................. - -Use the yum repositories `/etc/yum.repos.d/epeli386.repo` and -`/etc/yum.repos.d/epelx86_64.repo` to download and install the latest -puppet packages:: - - [epeli386] - name=epel i386 - baseurl=http://download.fedora.redhat.com/pub/epel/5/i386/ - enabled=0 - - -Install the puppet-server package. Dependencies will be resolved:: - - # yum --enablerepo=epeli386 install puppet-server - - ... - Dependencies Resolved - - ============================================================================= - Package Arch Version Repository Size - ============================================================================= - Installing: - puppet-server noarch 0.24.8-1.el5.1 epel 26 k - Installing for dependencies: - augeas-libs i386 0.5.1-1.el5 epel 196 k - facter noarch 1.5.5-1.el5 epel 54 k - puppet noarch 0.24.8-1.el5.1 epel 542 k - ruby i386 1.8.5-5.el5_2.6 sl5update 279 k - ruby-augeas i386 0.2.0-1.el5 epel 17 k - ruby-libs i386 1.8.5-5.el5_2.6 sl5update 1.6 M - ruby-shadow i386 1.4.1-7.el5 epel 9.5 k - - Transaction Summary - ============================================================================= - Install 8 Package(s) - Update 0 Package(s) - Remove 0 Package(s) - ... - - -For later use, download the required RPMS and copy them to our yum -repository, see section `Put Puppet Related RPMS To Our Yum -Repository` below. - - -Configure The Puppet Server -........................... - -Use the puppet server configuration of pxeserv01 on psi-puppet2. - -The configuration files of the puppet server, directory -`/etc/puppet/`, are located on AFS, see section `Mount AFS Volumes`. - -The log is on the local disk in `/var/log/puppet`. To set the logfile -edit the line `PUPPETMASTER_OPTS` in -`/etc/rc.d/init.d/puppetmaster`. For testing also the debug option -`-d` is enabled:: - - PUPPETMASTER_OPTS="-v -d -l /var/log/puppet/puppetmaster.log" - -Enable puppetmaster in runlevels 3, 4 and 5:: - - # chkconfig --levels 345 puppetmaster on - - -Mount AFS Volumes on Puppet Server -.................................. - -The puppet server configuration files as well as the puppet manifests -for clients are located on AFS. - -AFS is already mounted as `/afs` in this default SL5 server installation:: - - # mount - ... - AFS on /afs type afs (rw) - -Now, we want to remount `/afs/psi.ch/service/linux/puppet/etc/puppet` -on `/etc/puppet`. Therefor the mount option `bind` is used, which -facilitates to remount parts of already mounted filesystems on an -alternative location in the file hierarchy. - -As shown below we do the remount in `/etc/rc.local`, which is executed -after all the other init scripts:: - - #!/bin/sh - - touch /var/lock/subsys/local - - # Puppet - mount -o bind /afs/psi.ch/service/linux/puppet/etc/puppet /etc/puppet - - # Restart Services depending on afs mounts - /etc/init.d/puppetmaster restart - -Before the `rc.local` script can be applied the proper AFS permissions -have to be set. - -Check the AFS permissions:: - - # [gasser_m@pc7377 ~] - # fs la /afs/psi.ch/service/linux/ - - Access list for /afs/psi.ch/service/linux/ is - Normal rights: - svc_linux:tools l - svc_linux:puppet l - svc_linux:readonly rl - svc_linux:pxe l - svc_linux rlidwka - - -Create a new AFS group for the puppet server:: - - # pts creategr svc_linux:puppet_hosts -owner svc_linux - group svc_linux:puppet_hosts has id -10851 - -Add the IP of psi-puppet2 to this group:: - - # pts adduser 129.129.190.160 svc_linux:puppet_hosts - -Set the AFS permissions:: - - # fs sa /afs/psi.ch/service/linux/ svc_linux:puppet_hosts l - # fs sa /afs/psi.ch/service/linux/puppet/ svc_linux:puppet_hosts rl - # fs sa /afs/psi.ch/service/linux/puppet/etc svc_linux:puppet_hosts rl - # fs sa /afs/psi.ch/service/linux/puppet/var svc_linux:puppet_hosts rl - # fs sa /afs/psi.ch/service/linux/puppet/etc/puppet svc_linux:puppet_hosts rl - -Recursively set the same permissions to all subdirectories underneath -`/afs/psi.ch/service/linux/puppet/etc/puppet`:: - - # cd /afs/psi.ch/service/linux/puppet/etc/puppet - # find -noleaf -type d -exec fs sa {} svc_linux:puppet_hosts rl \; - -Then restart the AFS service:: - - # service afs restart - -Finally, run the `rc.local` script or just reboot psi-puppet2 to see -whether everything comes up. - - -Allow User Login Access To psi-puppet2 -...................................... - -To enable remote login via ssh and execution of root commands via -sudo, the following files have to be edited. - -- `/etc/passwd` - -- `/etc/shadow` - -- `/etc/security/ssh.allow` - -- `/etc/group` - -- `/etc/hosts.allow` - -- `/etc/sudoers` - - -passwd:: - - gasser_m:!:2374:840:Gasser Marc:/afs/psi.ch/user/g/gasser_m:/bin/bash - kapeller:!:3804:9102:Rene Kapeller:/afs/psi.ch/user/k/kapeller:/bin/bash - billich:!:3830:840:Heinrich Billich:/afs/psi.ch/user/b/billich:/bin/bash - markushin:!:3883:840:Valeri Markushin:/afs/psi.ch/user/m/markushin:/bin/bash - -shadow:: - - gasser_m:NP::::::: - kapeller:NP::::::: - billich:NP::::::: - markushin:NP::::::: - -group:: - - ait::840 - sls::9102 - -ssh.allow:: - - kapeller - billich - markushin - gasser_m - -hosts.allow:: - - sshd: ... pc7377 pc7377.psi.ch gfalc05 gfalc05.psi.ch pc4568 pc4568.psi.ch - - -Configuring Puppet Reporting -............................ - -There are a number of different report processors available on the -master. The default report, store, simply stores the report file on -the disk. - -By default, each client is configured not to report back to the -master. It has to be enabled either by the report option in -`puppet.conf` or using `--report` on the command line. - -`/etc/puppet/puppet.conf`:: - - [puppetd] - report = true - - -Command line:: - - # puppetd --report - - -Store Report Processor -,,,,,,,,,,,,,,,,,,,,,, - -Enable the store reports by using the `reports` configuration option -in the puppemasterd section of the `puppet.conf` file on the master. - -`/etc/puppet/puppet.conf`:: - - [puppetmasterd] - reports = store - -The default reports directory is $vardir/reports. - -Rrdgraph Report Processor -,,,,,,,,,,,,,,,,,,,,,,,,, - -To enable the `rrdgraph` reports, `rrdtool` and `rrdtool-ruby` -packages have to be installed. - -The packages are available from the the `psi-beta` repository, they -originate from the repository shown below. - -`/etc/yum.repos.d/epeli386.repo`:: - - [epeli386] - name=epel i386 - baseurl=http://download.fedora.redhat.com/pub/epel/5/i386/ - enabled=0 - - # yum install rrdtool rrdtool-ruby - -Then, configure `puppet.conf` by adding the lines shown below in the -corresponding section. Here store and rrdgraph are enabled. - -`/etc/puppet/puppet.conf`:: - - [puppetmasterd] - reports = store,rrdgraph - rrddir = $vardir/rrd - rrdinterval = $runinterval - rrdgraph = true - -Install the Ganglia Monitor Daemon -.................................. - -Install `ganglia-gmond` and add the configuration file shown below:: - - # yum install ganglia-gmond - -Use the same `/etc/gmond.conf` as on psi-puppet1:: - - # /etc/init.d/gmond start - -For the ganglia server configuration ask Valeri Markushin. - -See puppet at [[http://129.129.190.27/ganglia/][http://129.129.190.27/ganglia/]]. - -Setup The Puppet Client -~~~~~~~~~~~~~~~~~~~~~~~~ - -Either install or update the puppet package on a client:: - - # yum --enablerepo=psi-beta install puppet - - # yum --enablerepo=psi-beta update puppet - -Test The Puppet Clients -~~~~~~~~~~~~~~~~~~~~~~~ - -To find out which versions of puppet servers and clients are -compatible to each other and in which order they should be updated, -the following tests are performed. - - -Test the Old Client With The New Server -....................................... - -Set the new server `psi-puppet2.psi.ch` in `/etc/puppet/puppet.conf`, -remove the certificate and run puppetd:: - - # [root@vsl5132de-ut] - # cd /var/puppet/ - # rm -rf ssl/ - # puppetd --test - -Ok. - - -Test the New Client With The New Server -....................................... - -Update the puppet client. Set the new server `psi-puppet2.psi.ch` in -`/etc/puppet/puppet.conf`, remove the certificate and run puppetd:: - - # [root@vsl5132de-ut] - # yum --enablerepo=psi-beta update puppet - - # cd /var/puppet/ - # rm -rf ssl/ - # puppetd --test - -Ok. - - -Test the New Client With The Old Server -....................................... - -Update the puppet client. Then set the old server `pxeserv01.psi.ch` -in `/etc/puppet/puppet.conf`, remove the certificate and run puppetd:: - - # [root@vsl5132de-ut] - # yum --enablerepo=psi-beta update puppet - - ... - ============================================================================= - Package Arch Version Repository Size - ============================================================================= - Updating: - puppet noarch 0.24.8-1.el5.1 psi-beta 542 k - Installing for dependencies: - augeas-libs i386 0.5.1-1.el5 psi-beta 196 k - ruby-augeas i386 0.2.0-1.el5 psi-beta 17 k - ruby-shadow i386 1.4.1-7.el5 psi-beta 9.5 k - Updating for dependencies: - facter noarch 1.5.5-1.el5 psi-beta 54 k - - Transaction Summary - ============================================================================= - Install 3 Package(s) - Update 2 Package(s) - Remove 0 Package(s) - ... - - # cd /var/puppet/ - # rm -rf ssl/ - # puppetd --test - -Not ok. A lot of error messages appear. It seems the new client is not -compatible to the old server! - -Conclusion -.......... - -First update the server, then the client. - -So, when you have an old client addressing the new server for tests, -and finally you wanna change it back to the old server, this should -work by just modifying `/etc/puppet/puppet.conf` on the client. - -However, once you updated the client you can not go back to the old -server, unless you downgrade the client --- with yum this means -removing the new client and reinstalling the old one. - -Put Puppet Related RPMS To Our Yum Repository ---------------------------------------------- - -To have a consistent puppet installation on our hosts we put them to -our repository. - -Use yumdowloader, package yum-utils, to download puppet related RPMS:: - - # root@psi-puppet2 - # yum install yum-utils - - # yumdownloader --enablerepo=epeli386 puppet-server augeas-libs facter \ - # puppet ruby-augeas ruby-shadow - - # yumdownloader --enablerepo=epelx86_64 augeas-libs ruby-augeas ruby-shadow - - -**Note**: There seem to be only i386 and noarch versions of the - required RPMS. - -Before they are linked into the latest `testing` and `current` -repositories they should be tested. So copy them to the `psi-beta` -repository first:: - - # scp *rpm gasser_m@tux50:/afs/psi.ch/software/linux/dist/scientific/51/beta - - # gasser_m@tux50 - # cd /afs/psi.ch/software/linux/dist/scientific/51/beta - # createrepo . - -To test the puppet clients see section `Test The Puppet Clients`. - -If the tests passed successfully copy them to the `others` repository -and create symbolic links to `testing` and `current` to make them -available:: - - # [root@psi-puppet2] - # scp *rpm gasser_m@tux50:/afs/psi.ch/software/linux/dist/scientific/51/others/all - - # gasser_m@tux50 - # cd /afs/psi.ch/software/linux/dist/scientific/51/others/all - # createrepo . - - -Update Puppet -------------- - -Keep the following order. - -- Update the puppet server. - -- Update the puppet client. - - -Update The Puppet Server -~~~~~~~~~~~~~~~~~~~~~~~~ - -At the time of writing the latest Puppet version 0.25.1 was only -available at http://tmz.fedorapeople.org/repo/puppet/epel/5/i386/. - -Download the required packages to SL51 psi-beta repository first:: - - # wget http://tmz.fedorapeople.org/repo/puppet/epel/5/i386/puppet-server-0.25.1-0.3.el5.noarch.rpm - # wget http://tmz.fedorapeople.org/repo/puppet/epel/5/i386/puppet-0.25.1-0.3.el5.noarch.rpm - -Then login to a test server and stop the puppetmaster daemon:: - - # /etc/init.d/puppetmaster stop - -Because the /etc/puppet/ is on AFS, root has no write permissions. -So, first umount the etc/puppet from AFS, then run `yum update` using -the psi-beta repository, and remount etc/puppet:: - - # umount /etc/puppet - # yum --enablerepo=psi-beta update puppet-server - # mount -o bind /afs/psi.ch/service/linux/puppet/etc/puppet /etc/puppet - -Set the following options in /etc/init.d/puppetmaster:: - - PUPPETMASTER_OPTS="-v -d -l /var/log/puppet/puppetmaster.log" - -Finally, restart the service and test it with a client:: - - # /etc/init.d/puppetmaster start - -Login to a client and run puppetd:: - - # puppetd --test - -Update The Puppet Client -~~~~~~~~~~~~~~~~~~~~~~~~ - -Basically, the 0.24.x clients should be compatible to the 0.25.x -server. Though, there are some changes between the versions, that -might cause troubles. Thus, the puppet clients should be updated, too. - -On a SL51 client run:: - - # yum --enablerepo=psi-beta update puppet - -On a SL54 client run:: - - # yum update puppet - -As soon as the client is updated, its `puppet.conf` needs some -modifications, because of the "factsync" option which is deprecated -and replaced by "pluginsync" in the 0.25.x versions. - -`/etc/puppet/puppet.conf` on a 0.25.x client: "factsync" is replaced -by "pluginsync", factpath is set (not clear whether the latter is -necessary):: - - [main] - vardir = /var/puppet - logdir = /var/log/puppet - rundir = /var/run/puppet - ssldir = $vardir/ssl - pluginsync = true - factpath = $vardir/lib/facter - environment = DesktopSL5Unstable - - [puppetd] - report = true - classfile = $vardir/classes.txt - localconfig = $vardir/localconfig - server = psi-puppet1.psi.ch - - -Further the directory structure on the server for placing facts -changed. The new structure if using modules and environments is -illustrated below taking the environment "DesktopSL5Unstable" as an -example: - -The modulepath for DesktopSL5Unstable is:: - - modulepath = /var/puppet/environments/DesktopSL5Unstable/modules - -A stub module called "custom" has to be created in the "$modulepath" -subdirectory to keep the files:: - - $modulepath/custom/ - `-- lib/ - |-- facter/ - | `-- sysconfig_psi_desktop.rb - `-- puppet/ - |-- provider/ - `-- type/ - -This subdirectory tree under custom is implicitly searched by the -puppet server. - -**Notes**: - -If you have both, the new and the old variant to keep facts, option -"pluginsync" enabled, only the new script location will be considered. - -If you run only the old variant to keep facts:: - - $modulepath/facts/files/somescript.rb - -with option "factsync" enabled, they will be loaded giving out a -warning "... use pluginsync instead of factsync ..." - -For more information see the -http://reductivelabs.com/trac/puppet/wiki/PluginsInModules. diff --git a/admin-guide/legacy/misc/puppet-trouble-shooting-in-twiki.rst b/admin-guide/legacy/misc/puppet-trouble-shooting-in-twiki.rst deleted file mode 100644 index 35b093d..0000000 --- a/admin-guide/legacy/misc/puppet-trouble-shooting-in-twiki.rst +++ /dev/null @@ -1,145 +0,0 @@ -Puppet Trouble-shooting in TWiki -================================ - -Introduction ------------- - -This is the location were to post puppet problems and solutions at AIT -and GFA. - -Please use the following format according the first entry below: - -- HEADING2: Error (HOSTNAME): "BEGIN OF ERROR MESSAGE ..." - *(The HOSTNAME in parentheses is optional, if the problem might affect any host.)* - -- HEADING3: Context, a short description in what situation the error appears. - -- HEADING3: Solution, if known, provide a possible solution to fix the problem. - - -Puppet Server Errors --------------------- - -Error (pxeserv01): "Starting puppetmaster: Could not prepare for execution ..." -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Context -....... - -The puppetmaster daemon on pxeserv01 received an unattended SIGTERM -and stopped. Maybe there is a self signed certificate which expired -or something. Anyway, restarting failed showing the following error:: - - # [root@pxeserv01 puppet]# /etc/init.d/puppetmaster start - Starting puppetmaster: Could not prepare for execution: Retrieved certificate does not \ - match private key; please remove certificate from server and regenerate it with the current key - - -Solution -........ - -Create a new certificate for the server. - -**IMPORTANT NOTE**: Recreating a new certificate for the server showed -that the clients which still have the old certificate of the server -can not connect anymore to it. Thus you need to be sure you have a -way to update the certificate of the server on the clients before -applying this. Eventually, a better solution might be to just sign -the old server certificate with `puppetca`. It was not tested whether -this works, too. - -Remove /var/puppet/ssl/ on the server:: - - # rm -rf /var/puppet/ssl/ - - -Then try to start puppetmaster, what failed again, but recreated the -ssl subdirectory and its content:: - - # /etc/init.d/puppetmaster start - - -Eventually, sign the certificate for pxeserv01 itself:: - - # puppetca --sign pxeserv01.psi.ch - # /etc/init.d/puppetmaster start - - -Via our management console the certificates on the clients had to be -deleted, in order they can retrieve the new server certificate:: - - # for i in host1 host2 ... - # do - # ssh -o ConnectTimeout=1 root@$i rm -rf /var/puppet/ssl/ - # done - - -Puppet Client Errors --------------------- - -Error: "Could not request certificate: ..." -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Context -....... - -Running puppetd on any client might fail showing the following error:: - - # puppetd --test - err: Could not request certificate: Retrieved certificate does not match private key; - please remove certificate from server and regenerate it with the current key - Exiting; failed to retrieve certificate and watiforcert is disabled - - -Solution -........ - -The problem is that for what reason ever the client certificate is not -valid, i.e. it does not match its copy stored on the puppet server. - -Usually this happens when a host is reinstalled keeping its original -hostname. During the initial puppetd run, the client requests a new -certificate from the server, which denies, because there is already a -certificate on the server for that hostname. - -To fix this you have to delete the old certificate on client and -server. - -On the server:: - - # puppetca --clean HOSTNAME.psi.ch - HOSTNAME.psi.ch - notice: Removing file Puppet::SSL::Certificate HOSTNAME.psi.ch at - '/var/puppet/ssl/ca/signed/mpc1273.psi.ch.pem' - -If this notice does not appear, the certificate was not deleted. - -On the client:: - - # rm -rf /var/puppet/ssl/ - -Then, run puppetd again:: - - # puppetd --test - - -Error: "notice: Run of Puppet configuration client already in progress; skipping ..." -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Context -....... - -Running puppetd on any client might fail showing the following error, -even `ps ax | grep puppetd` does not show a running puppet process:: - - # puppetd --test - notice: Run of Puppet configuration client already in progress; skipping - - -Solution -........ - -There is still an old lock file `/var/puppet/state/puppetdlock`. -Remove the lock file and run puppetd again:: - - # rm /var/puppet/state/puppetdlock diff --git a/admin-guide/legacy/misc/puppetmasteratpsi.rst b/admin-guide/legacy/misc/puppetmasteratpsi.rst deleted file mode 100644 index ca6c7d9..0000000 --- a/admin-guide/legacy/misc/puppetmasteratpsi.rst +++ /dev/null @@ -1,76 +0,0 @@ -Puppetmaster At PSI -=================== - -Procedure ---------- - -Manual Check Whether Puppetmaster Is Running -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Contemporary, the `puppetmasterd` process often crashes and automatic -restart fails. Thus it has to be started manually again. Therefore login -to `pxeserv01`, check whether the process is still running and if not, -restart `puppetmasterd`. - -Check whether the `puppetmasterd` is running:: - - [root@pxeserv01 ~]# ps ax | grep puppet - 549 pts/0 S+ 0:00 grep puppet - 6641 ? S 7:53 /bin/bash ./test-and-restart-puppetmaster.sh - -It's not running. - -Check the logfile. The logfile is situated on AFS:: - - # tail /afs/psi.ch/service/linux/puppet/var/log/puppetmaster.log - ... - Mon Jan 26 09:22:27 +0100 2009 Puppet (notice): Compiled configuration for slsnedi1.psi.ch in 0.09 seconds - Mon Jan 26 09:24:55 +0100 2009 Puppet (notice): Caught TERM; shutting down - Mon Jan 26 09:24:55 +0100 2009 Puppet (notice): Shutting down - Mon Jan 26 09:24:55 +0100 2009 Puppet (err): Could not remove PID file /var/run/puppet/puppetmasterd.pid - Mon Jan 26 09:25:01 +0100 2009 Puppet (err): Could not create PID file: /var/run/puppet/puppetmasterd.pid - - -In this case remove `/var/run/puppet/puppetmasterd.pid` and restart the server:: - - # rm /var/run/puppet/puppetmasterd.pid - # /etc/init.d/puppetmaster restart - - -Check again whether the `puppetmasterd` is running:: - - # [root@pxeserv01 ~]# ps ax | grep puppet - 549 pts/0 S+ 0:00 grep puppet - 6641 ? S 7:53 /bin/bash ./test-and-restart-puppetmaster.sh - 31599 ? Ssl 0:03 /usr/bin/ruby /usr/sbin/puppetmasterd \ - --logdest=/afs/psi.ch/service/linux/puppet/var/log/puppetmaster.log - - -Finally, test it on a client:: - - # [root@pc7377 ~]# psi-puppet - ... - puppetd was running successfully - - -Automatic Check Whether Puppetmaster Is Running -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -On `pxeserv01` the script `/root/test-and-restart-puppetmaster.sh` -continuously observes the `puppetmasterd` and should restart it when -it's hanging. - -The script is started in the following way:: - - # [root@pxeserv01 etc]# nohup /root/test-and-restart-puppetmaster.sh & - - -The output is written to: - -- `$HOME/nohup.out` - - The logfile. - -- `/dev/shm/puppetwatch.1` - -- `/dev/shm/puppetwatch.2` diff --git a/admin-guide/legacy/misc/updatesl57.rst b/admin-guide/legacy/misc/updatesl57.rst deleted file mode 100644 index f7357cd..0000000 --- a/admin-guide/legacy/misc/updatesl57.rst +++ /dev/null @@ -1,282 +0,0 @@ -Update SL57 and later i386 and x86_64 -===================================== - -Introduction ------------- - -Keep the following order: - -- Get the latest security updates from the SL mirror for the - "update.$ARCH" repository. - -- Build kernel modules. - -- Get updates for the "psi" repository. - -- Get updates for the "other" repository. - -- Get updates for the "nonfree" repository. - -- Create new snapshots. - -- Release "unstable" from new snapshots. - -- Create a new PSI version. - -- Release "testing". - -- Release "stable". - - -Update The Different Repositories ---------------------------------- - -Get the Latest Security Update RPMS From SL Mirror -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Get the latest security update RPMS from the local SL57 mirror for -both architectures, i386 and x86_64, by invoking -`rsync_security_updates_to_all_directories.sh` on tux50. - -They will be copied to the corresponding `.../updates.${ARCH}/all/` -directories. This will take some time:: - - # cd /afs/psi.ch/software/linux/dist/scientific/57/scripts/ - # nohup ./rsync_security_updates_to_all_directories.sh > \ - # UPDATE_LOG/$(date +%Y-%m-%d)-rsync_security_updates_to_all_directories.log 2>&1 & - -Then, check for errors in the log file:: - - # grep -i error UPDATE_LOG/20091221-rsync_security_updates_to_all_directories.log - ... - -Finally, run `update_symlinks_in_rpms_all.sh` to keep all symlinks in -the directory -`/afs/psi.ch/software/linux/dist/scientific/57/RPMS_all/` up to date. - -The script removes dead links and creates new links to the new RPMS. -It is basically not necessary for running PSI updates, rather it's -just convenient to have a directory with the list of all RPMS of a -distribution:: - - # ./update_symlinks_in_RPMS_all.sh - -Build Kernel Modules -~~~~~~~~~~~~~~~~~~~~ - -First install the latest kernel and kernel-devel packages on the -respective build systems. - -Then build the RPMS on the corresponding build systems using the -scripts below. - -32 bit:: - - # [gasser_m@tukan50-32] - # cd /afs/psi.ch/software/linux/dist/scientific/57/scripts - # sh build_kernel_modules.sh 2.6.18-194.3.1.el5 > \ - # UPDATE_LOG/20100630-build_kernel_modules-2.6.18-194.3.1.el5.i686.log 2>&1 - - # check-set-of-built-kernel-modules.bash /tmp/rpms_for_2.6.18-194.3.1.el5 \ - # KERNEL_MODULES_TO_BUILD/sl5.i386.kms-to-be-built - - # sh build-gpfs.sl5x.i386.sh - - -64 bit:: - - # [gasser_m@tux50-64] - # cd /afs/psi.ch/software/linux/dist/scientific/57/scripts - # sh build_kernel_modules.sh 2.6.18-194.3.1.el5 > \ - # UPDATE_LOG/20100630-build_kernel_modules-2.6.18-194.3.1.el5.x86_64.log 2>&1 - - # check-set-of-built-kernel-modules.bash /tmp/rpms_for_2.6.18-194.3.1.el5 \ - # KERNEL_MODULES_TO_BUILD/sl5.x86_64.kms-to-be-built - - # sh build-gpfs.sl5x.x86_64.sh - - -Finally, from both build systems, copy the built kernel modules to the -psi repository:: - - # cp -av `cat /tmp/rpms_for_2.6.18-194.3.1.el5` /afs/psi.ch/software/linux/dist/scientific/57/psi/all/ - - -Get Updates For The Psi Repository -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Get Updates For The Other Repository -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - -Get Updates For The Nonfree Repository -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Create New Snapshots --------------------- - -The script `create_new_snapshots.sh` will pick out the latest version -of each RPM found in the active repositories. - -It will create a subdirectory (snapshot) in each repository and name -it after the current date using the format YYYYMMDD. - -Then it will place a symbolic link into this subdirectory for each -latest RPM found in the `all` subdirectory of the currently processed -repository. - -To invoke it run the following command:: - - # nohup ./create_new_snapshots.sh alldirs > UPDATE_LOG/$(date +%Y-%m-%d)-create_new_snapshots.log 2>&1 & - - -Release Unstable, Create A New PSI Version And Release The Other SLP Snapshots ------------------------------------------------------------------------------- - -Release Unstable -~~~~~~~~~~~~~~~~ - -The "unstable" distribution is where active development of SLP occurs. -Generally, this distribution is run by developers and those who like -to live on the edge. - -The command `release_unstable.sh` will update the respective symlinks -`.../unstable` to the latest snapshots. - -Because `release_unstable.sh` is interactive, you should not redirect -the output to a file, as you won't be able to see the questions asked. - -Before `release_unstable.sh`:: - - # [gasser_m@tux50] - # cd /afs/psi.ch/software/linux/dist/scientific/57/ - # ls -l */unstable - - # ./release_unstable.sh - - -After `release_unstable.sh`:: - - # ls -l */unstable - - -As soon as an unstable distribution has become testing a new unstable -can be generated which again points to the new latest snapshots. - - -Create A New PSI Version -~~~~~~~~~~~~~~~~~~~~~~~~ - -Run:: - - # cd /afs/psi.ch/software/linux/dist/scientific/57/scripts - # sh create_new_psi_version.sh - -As soon as the new PSI version is created, i.e. the symbolic links -which point to the same target snapshots as the latest unstable -snapshots, the psi-version-info.txt file has to be actualized to -activate the PSI auto-update again for the hosts which are set to -unstable:: - - # ./create_file_psiversion-info.sh - -Eventually, test the unstable release. - - -Release Testing -~~~~~~~~~~~~~~~ - -The "testing" distribution contains packages that haven't been -accepted into a "stable" release yet, but they are in the queue for -that. The main advantage of using this distribution is that it has -more recent versions of software. - -The command `release_testing.sh` will update the respective symlinks -`.../testing` to the latest unstable snapshots. - -If the puppet environment DesktopSL5Unstable was modified and the -changes should be made in DesktopSL5Testing, too, run -`rsync_puppet_env_testing_with_unstable.sh` before -`release_testing.sh`. - -Because `release_testing.sh` is interactive, you should not redirect -the output to a file, as you won't be able to see the questions asked:: - - # [gasser_m@tux50] - # cd /afs/psi.ch/software/linux/dist/scientific/57/scripts - - # ./rsync_puppet_env_testing_with_unstable.sh - # ./rsync_kickstart_dir_DesktopTesting_with_DesktopUnstable.sh - # ./release_testing.sh - - # ./create_file_psiversion-info.sh - - -Release Stable -~~~~~~~~~~~~~~ - -To activate the PSI auto-update again for testing hosts the -psi-version-info.txt file has to be actualized. - -The "stable" distribution, formerly known as "current", contains the -latest officially released distribution of SLP. - -This is the production release of SLP, the one which we primarily -recommend using. - -The command `release_stable.sh` will update the respective symlinks -`.../stable` to the latest unstable snapshots. - -The procedure is analogous to the procedure described in the "Release -Testing" section:: - - # ./rsync_puppet_env_stable_with_testing.sh - # ./rsync_kickstart_dir_DesktopStable_with_DesktopTesting.sh - # ./release_stable.sh - # ./create_file_psiversion-info.sh - - - ### begin ./release_stable.sh ### - Sourcing configuration file ./dist-config - - TOP_DIR is /afs/psi.ch/software/linux/dist/scientific/57 - - Running ./release_stable.sh ... - - Latest snapshot in psi: - /afs/psi.ch/software/linux/dist/scientific/57/psi/testing -> 20090916 - Latest snapshot in others: - /afs/psi.ch/software/linux/dist/scientific/57/others/testing -> 20090916 - Latest snapshot in update.i386: - /afs/psi.ch/software/linux/dist/scientific/57/update.i386/testing -> 20090916 - Latest snapshot in update.x86_64: - /afs/psi.ch/software/linux/dist/scientific/57/update.x86_64/testing -> 20090916 - Latest snapshot in kernel: - /afs/psi.ch/software/linux/dist/scientific/57/kernel/testing -> 2.6.18-128.7.1.el5 - Latest snapshot in nonfree: - /afs/psi.ch/software/linux/dist/scientific/57/nonfree/testing -> 20090916 - Latest snapshot in cluster: - /afs/psi.ch/software/linux/dist/scientific/57/cluster/testing -> 20090916 - - Relink stable to the latest snapshots (y/n)? - - -Before `release_stable.sh`:: - - # [gasser_m@tux50] - # cd /afs/psi.ch/software/linux/dist/scientific/57/ - # ls -l */stable - - lrwxr-xr-x 1 gasser_m ait 8 Sep 18 11:25 cluster/stable -> 20090316 - lrwxr-xr-x 1 gasser_m ait 8 Sep 18 11:23 enhanced/stable -> 20090316 - lrwxr-xr-x 1 gasser_m ait 18 Sep 18 11:24 kernel/stable -> 2.6.18-128.1.1.el5 - lrwxr-xr-x 1 gasser_m ait 8 Sep 18 11:24 nonfree/stable -> 20090316 - lrwxr-xr-x 1 gasser_m ait 8 Sep 18 11:22 others/stable -> 20090316 - lrwxr-xr-x 1 gasser_m ait 8 Sep 18 11:21 psi/stable -> 20090821 - lrwxr-xr-x 1 gasser_m ait 8 Sep 18 11:22 update.i386/stable -> 20090820 - lrwxr-xr-x 1 gasser_m ait 8 Sep 18 11:21 update.x86_64/stable -> 20090820 - - -After `release_stable.sh`:: - - # ls -l */stable diff --git a/admin-guide/legacy/misc/updateslmaindoc.rst b/admin-guide/legacy/misc/updateslmaindoc.rst deleted file mode 100644 index cec9086..0000000 --- a/admin-guide/legacy/misc/updateslmaindoc.rst +++ /dev/null @@ -1,1275 +0,0 @@ -Updates for SL 4 and SL 5.1 -=========================== - -References ----------- - -- https://wiki.intranet.psi.ch/AIT/LinuxRegularUpdate (U. Beyerle) - -- https://wiki.intranet.psi.ch/AIT/SL5RegularUpdates200807 (V. Markushin) - - -Introduction ------------- - -The following text describes how regular Scientific Linux (SL) updates -for SL51 and SL4, are performed. - -A detailed standard procedure is exemplified for an SL 51 update on a -64 bit machine, but the same operation is followed for other SL -releases <= SL51 and architectures. Deviations from the standard -procedure are emphasized in the text. - - -General Remarks -~~~~~~~~~~~~~~~ - -Regular Linux Udpates are scheduled every first Wednesday in a month. - -Create a new Testing Release one week in advance. - -It is advised to create the new version for a release on the -respective build system. For example for SL5 i386 do it on tux50 with -your AFS account, for AFS ACLs see "Required File Permissions On AFS" -below. - -When you compile new kernel module RPMS then it's recommended to use -build systems. - - -Reasons For Using Build Systems -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- Performance: Some scripts will just horribly slow down your desktop - machine, making it almost impossible to work on it while the - respective script is running. So, using a build system will not - affect the performance of your desktop computer. - -- Version conflicts: Some versions of some tools such as yum, - createrepo, etc., might be different in different SL releases, thus - to avoid conflicts one should use the version of the tool that comes - with the corresponding release. Other packages might be missing at - all, e.g. some devel packages used when RPMS are built or tool like - createrepo, which are not required on a default Desktop. - -- Compiling software: For compiling software, e.g. building kernels, - kernel modules, etc., it first has to be on the right architecture, - and second it's also recommended to do it in some clean default - environment, which is a representative for the environments of - productive systems. - - -Scientific Linux Mirror at PSI -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -RPMS are taken from our mirror at -`/afs/psi.ch/software/mirror/scientific/`. The mirror is updated -daily by a cronjob running on pxeserv01. (See also -LinuxInstallationServer.):: - - # file pxeserv01:/etc/cron.d/mirror - - # Mirroring of rpm repositories at 07:00 - # - 00 07 * * * root /afs/psi.ch/software/mirror/scripts/mirror.sh > /dev/null 2>&1 - -The script `/afs/psi.ch/software/mirror/scripts/mirror.sh` reads -`/afs/psi.ch/software/mirror/scripts/mirror.conf` to know about -locations of ftp external software repositories and where to place the -files on our local mirror. - -E.g.: The archive -`ftp://ftp.scientificlinux.org/linux/scientific/46/i386` is locally -mirrored at `/afs/psi.ch/software/mirror/scientificlinux/46/i386`. - - -Autoupdate in SL5 -~~~~~~~~~~~~~~~~~ - -The script `/usr/sbin/psi-auto-update` is started via cron usually in -the night and triggers the update, the latter is performed by scripts -and Puppet. The update is released the first Wednesday of a -month. Default Desktop computers will be updated during the night from -Wednesday to Thursday. - - -Preparatory Work ----------------- - -Required File Permissions On AFS -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Before starting with performing updates, you need the corresponding -permissions to do things in the PSI linux repository directory at -`/afs/psi.ch/software/linux/dist/scientific/`, which is on an AFS -volume. - -Check permissions using AFS commands and get the required AFS access. -The unix commands `ls -l` etc. will work but not show you the relevant -AFS permissions, instead use the AFS command `fs`. - -E.g.: Type `fs help` to see a list of command attributes:: - - # fs listacl /afs/psi.ch/software/linux/ - ... - or - - # fs listacl /afs/psi.ch/software/mirror/ - Access list for /afs/psi.ch/software/mirror is - Normal rights: - ait:sw_adm rlidwka - psi:nodes rl - system:authuser rl - -To work in the linux distribution directories you need to be in the -following AFS groups (output of `pts mem AFS_USERNAME`):: - - # pts mem gasser_m - Groups gasser_m (id: 2374) is a member of: - ait:sw_adm - linux:users - ait:users - - -Prepare Your Build Environment For Building RPMS -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -For building RPMS you have two possibilities: - -- build locally in your build directories - -- build on AFS in common build directories - - -Build locally -............. - -Before you start building RPMS you create the following directories on -each build system. - -Example for tux50-64:: - - # ssh gasser_m@tux50-64 - # mkdir /scratch/gasser_m - # cd /scratch/gasser_m - # mkdir rpm_tmp rpm_topdir - # cd rpm_topdir - # mkdir BUILD RPMS SOURCES SPECS SRPMS - - -Your build environment is set in "~/.rpmmacros":: - - %WORKDIR %(echo "/scratch/$USER") - %_topdir /scratch/gasser_m/rpm_topdir - %_tmppath /scratch/gasser_m/rpm_tmp - %packager Marc Gasser - %vendor PSI Scientific Linux - %sl_site slp - %sl_site_version %(rpm -q --qf "%{v}" sl-release | cut -d"." -f1) - %os_release %{sl_site}%{sl_site_version} - %dist .%{os_release} - %rhel %sl_site_version - %debug_package %{nil} - - -The command `rpmbuild` is used to build RPMS. The built RPMS are -found in the RPMS directory of your build environment on your build -system. - - -Build on AFS -............ - -Change to `/afs/psi.ch/project/linux/src/scientific/RPMS`. There you -will find separate subdirectories, each for one specific RPM. - -Check that your `.rpmmacros` does not mess up the build environment -for this kind of building. Change to the subdirectory of the RPM you -want to build and use the script `psi-rpmbuild`, which sets up the -build environment (thus `.rpmmacros` is not required at all). In the -subdirectory run `psi-rpmbuild -bb` for building a binary package only -for instance. - -E.g.: To build the psi-desktop package version 2.x do the following on -a build system:: - - # cd psi-desktop-2/ - # psi-rpmbuild -bb - - -The built RPM can be found in the corresponding subdirectory in -`/afs/psi.ch/project/linux/dist/`. - -The big advantages of this procedure are, you don't have to install -the sources on each build system, instead you install them once in -AFS, and this centralized build environment can be shared among -different users on different hosts. - - -Check Diskspace -~~~~~~~~~~~~~~~ - -Before starting the update check the diskspace left on your file -systems involved. - -- For copying new RPMS from the mirror to the repositories - -To be on the safe side there should be 1 G of free diskspace on the -respctive AFS volumes, where RPMS are copied to:: - - [gasser_m@pc7377 tmp]$ fs lq /afs/psi.ch/software/linux/dist/scientific/305/ - Volume Name Quota Used %Used Partition - sw.dist.sl305 25000000 23317560 93%<< 88% < - - 728d4109f520336bb6b5f595de9dd2db acroread-8.1.6-2.el5.i386.rpm - fd1b8fb934140222460ed81c34420d90 acroread-plugin-8.1.6-2.el5.i386.rpm - - # md5sum -c file.md5sum - - -Kernel Update SL5 ------------------ - -This section describes the kernel update for SL51, 32 and 64 bit. - -The SL5 kernel update procedure described here is basically the same -for SL4. However, a lot of details such as names of files, kernel -modules, build systems, etc., differ, thus the whole SL4 procedure is -shown in its own section, too. See `Kernel Update SL4`. - -The general remarks below are valid for SL4, too. - - -General Remarks Referring To Kernel Update -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -When a new SL kernel is released also a lot of corresponding kernel -module packages are coming with this kernel to extend its -functionality, e.g. kernel-module-xfs to facilitate Xfs support of the -kernel. - -Now, in case a new kernel has been released you have to build a set of -kernel-module RPMS, which at PSI are provided in addition to those -coming with the regular SL updates. - -It's recommended to use dedicated build systems to build RPMS. - -To compile kernel modules some prerequisites have to be met. - -- A build environment has to be set up on your build system - (e.g. tux50-64 for SL5 x86_64), locally or on AFS as discussed - before. - -- New kernel and kernel-devel RPMS have to be installed on your build - system. - -- The particular kernel module source RPMS have to be installed on - your build system. - -- About 1 G of free diskspace. - -- Note: For compiling new kernel modules the corresponding kernel has - to be installed on your build system but must not be running and you - can build the modules in a non-root environment. - - -Kernel Update SL5 32-bit -........................ - -Follow the procedure shown in the 64-bit section below. Differences -between building of the 32 and 64 bit kernel modules are pointed out. - -The main difference is, first, that you have to build kernel, -kernel-xen and kernel-PAE modules for 32 bit systems, while you build -only kernel and kernel-xen modules for 64 bit. The GPFS kernel module -is build for both, 32 and 64 bit systems. - - -Kernel Update SL5 64-bit -........................ - -(Example for PSI update SL5 version 31, June 2009.) - -Login to your build system and check which kernel packages are already -installed:: - - # [gasser_m@tux50-64 ~]$ rpm -q kernel{,-xen,-devel,-xen-devel} - - kernel-2.6.18-92.1.22.el5 - kernel-2.6.18-128.1.1.el5 - kernel-2.6.18-128.1.6.el5 - kernel-xen-2.6.18-92.1.22.el5 - kernel-xen-2.6.18-128.1.1.el5 - kernel-xen-2.6.18-128.1.6.el5 - kernel-devel-2.6.18-92.1.22.el5 - kernel-devel-2.6.18-128.1.1.el5 - kernel-devel-2.6.18-128.1.6.el5 - kernel-xen-devel-2.6.18-92.1.22.el5 - kernel-xen-devel-2.6.18-128.1.1.el5 - kernel-xen-devel-2.6.18-128.1.6.el5 - - -Check whether there is a newer kernel package in the repository:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/update.x86_64/all/ - # ls -1 kernel*-2.* | grep -v module | get-latest-RPM-from-RPM-list.pl - - ... - kernel-2.6.18-128.1.14.el5 - -Latest version of installed kernels is `2.6.18-128.1.6.el5` , latest -version in the repository is `2.6.18-128.1.14.el5`, i.e. new SLP -kernel modules have to be built. - -Check the diskspace of your build partition, 1G is ok:: - - # [gasser_m@tux50-64 all]$ df -hT /scratch - Filesystem Type Size Used Avail Use% Mounted on - /dev/mapper/VolGroup00-LogVol00 - ext3 14G 12G 1.6G 89% / - - -Is there a listing of all kernel module packages to be built? - - -Installation Of Kernel And Kernel-devel RPMS On 64 Bit SL5 Build System -,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, - -Install new kernel and kernel-devel RPMS as root on tux50-64. - -RPMS for the following kernels are required: kernel, kernel-xen:: - - # [root@tux50-64] - # cd /afs/psi.ch/software/linux/dist/scientific/51/update.x86_64/all/ - # ls -1 kernel-{,devel-,xen-,xen-devel-}2.6.18-128.1.14.el5.x86_64.rpm - kernel-2.6.18-128.1.14.el5.x86_64.rpm - kernel-devel-2.6.18-128.1.14.el5.x86_64.rpm - kernel-xen-2.6.18-128.1.14.el5.x86_64.rpm - kernel-xen-devel-2.6.18-128.1.14.el5.x86_64.rpm - - # rpm -ivh kernel-{,devel-,xen-,xen-devel-}2.6.18-128.1.14.el5.x86_64.rpm - - warning: kernel-2.6.18-128.1.14.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 82fd17b2 - Preparing... ########################################### [100%] - 1:kernel-xen-devel ########################################### [ 25%] - 2:kernel ########################################### [ 50%] - 3:kernel-devel ########################################### [ 75%] - 4:kernel-xen warning: /etc/modprobe.d/blacklist-firewire created as /etc/modprobe.d/blacklist-firewire.rpmnew - ########################################### [100%] - - -Installation Of Kernel And Kernel-devel RPMS On 32 Bit SL5 Build System -,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, - -Again, login as root to your build system to install the kernel -packages. - -For the 32 bit system also the PAE kernel modules are required:: - - # [root@tukan50-32] - # cd /afs/psi.ch/software/linux/dist/scientific/51/update.i386/all/ - # ls -1 kernel-{,devel-,PAE-,PAE-devel-,xen-,xen-devel-}2.6.18-128.1.14.el5.i686.rpm - # rpm -ivh kernel-{,devel-,PAE-,PAE-devel-,xen-,xen-devel-}2.6.18-128.1.14.el5.i686.rpm - - -Installation Of SL5 Kernel Module SRPMS On Build Systems -,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, - -Do this on both, the 32 and 64 bit build system. - -To build the kernel module RPMS the corresponding SRPMS have to be -installed on your build system first. - -List of SL5 kernel-module RPMS to be built for i686 and x86_64:: - - Name (without kernel version) Release Remarks - kernel-module-airprime 2.6.18-1.slp5 - kernel-module-atl2 1.0.40.2-1.slp5 - kernel-module-aufs 0.20070210.cvs-4.slp5 - kernel-module-firewire 2.6.18-2.slp5 - kernel-module-gspcav1 01.00.20-1.slp5 - kernel-module-iwlwifi 1.2.25-1.slp5 currently not builded - kernel-module-ntfs 2.1.27-0.rr.10.0 currently not builded - kernel-module-openafs 1.4.6-73.slp5 no SRPM, build script provided by A. Gsell - kernel-module-qla2xxx 8.01.07.15-1.slp5 - kernel-module-uvcvideo 0.1.0-1.svn148.slp5 - kernel-module-VMwareTools 6530-29999.slp5 no SRPM, build script provided by A. Gsell - kernel-module-gpfs texttext 64 bit only, no SRPM, build script provided by H.C. Stadler - - -Copy the PSI relevant source RPMS (SRPMS) to your build environment, -except sources of kernel modules ntfs, openafs, gpfs and -VMwareTools. They are built in their own environments. - -The SRPMS of the following 7 modules are required:: - - aufs firewire gspcav1 uvcvideo qla2xxx atl2 airprime - (see variable specfiles in /afs/psi.ch/software/linux/dist/scientific/51/scripts/build_kernel_modules.) - - -They can be found in the directory:: - - /afs/psi.ch/software/linux/dist/scientific/51/psi/all/ - - -Check for latest versions of required SRPMS:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/psi/all/ - # ll {aufs,firewire,gspcav1,uvcvideo,qla2xxx,atl2,airprime}*.src.rpm - - -rw-r--r-- 1 beyerle ait 7566 Apr 24 22:39 airprime-2.6.18-1.slp5.src.rpm - -rw-r--r-- 1 beyerle ait 56446 Apr 8 11:43 atl2-1.0.40.2-1.slp5.src.rpm - -rw-r--r-- 1 beyerle ait 77012 Apr 8 21:03 atl2-1.0.40.4-4.slp5.src.rpm - -rw-r--r-- 1 beyerle ait 136658 Oct 1 2007 aufs-0.20070210.cvs-4.slp5.src.rpm - -rw-r--r-- 1 beyerle ait 427554 Jul 16 10:48 aufs-0.20080605.cvs-5.slp5.src.rpm - -rw-r--r-- 1 beyerle ait 295158 Aug 21 2007 firewire-2.6.18-1.slp5.src.rpm - -rw-r--r-- 1 beyerle ait 295120 Jan 22 2008 firewire-2.6.18-2.slp5.src.rpm - -rw-r--r-- 1 beyerle ait 217470 May 15 10:36 gspcav1-01.00.20-1.slp5.src.rpm - -rw-r--r-- 1 beyerle ait 195078 Nov 12 2007 gspcav1-1.00.18-1.slp5.src.rpm - -rw-r--r-- 1 beyerle ait 2015550 Mar 10 22:04 qla2xxx-8.01.07.15-1.slp5.src.rpm - -rw-r--r-- 1 beyerle ait 91123 Dec 7 2007 uvcvideo-0.1.0-1.svn148.slp5.src.rpm - - -The following versions were taken as the latest: (This list can be -found in the file -`tux50-64:/scratch/gasser_m/rpm_topdir/SRPMS/20090624-psi-src-rpms.list`.):: - - airprime-2.6.18-1.slp5.src.rpm - atl2-1.0.40.4-4.slp5.src.rpm - #aufs-0.20080605.cvs-5.slp5.src.rpm # kernel-module-aufs is in SL51update and SL54base now - firewire-2.6.18-2.slp5.src.rpm - gspcav1-01.00.20-1.slp5.src.rpm - qla2xxx-8.01.07.15-1.slp5.src.rpm - uvcvideo-0.1.0-1.svn148.slp5.src.rpm - - -For convenience copy them first to your $rpm_topdir/SRPMS/ and then -unpack them, if not yet done (being still gasser_m@tux50-64):: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/psi/all - # for i in `cat /scratch/gasser_m/rpm_topdir/SRPMS/20090624-psi-src-rpms.list` - # do - # cp $i /scratch/gasser_m/rpm_topdir/SRPMS/ - # done - - -Then install them into your build environment:: - - # cd /scratch/gasser_m/rpm_topdir/SRPMS/ - # rpm -ivh `cat 20090624-psi-src-rpms.list` - - 1:airprime ########################################### [100%] - 2:atl2 ########################################### [100%] - 3:aufs ########################################### [100%] - 4:firewire ########################################### [100%] - 5:gspcav1 ########################################### [100%] - 6:qla2xxx ########################################### [100%] - 7:uvcvideo ########################################### [100%] - - -Before continuing with building the kernel modules check out the notes -below referring to some of them. - - -Build SL5 PSI Kernel-module RPMS On The Build System -,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, - -Now build the 64-bit modules:: - - # [gasser_m@tux50-64] - # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/ - # ./build_kernel_modules 2.6.18-164.11.1.el5 > \ - # UPDATE_LOG/20100125-build_kernel_modules-2.6.18-164.11.1.el5.x86_64.log 2>&1 - - -The following RPMS were built:: - - # cat ~/tmp/20090624-build_kernel_modules-2.6.18-128.1.14.el5.x86_64.output - - ... - --------------------------------------------------------------- - List of built kernel-modules for kernel 2.6.18-128.1.14.el5: - --------------------------------------------------------------- - - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-airprime-2.6.18-128.1.14.el5-2.6.18-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-airprime-2.6.18-128.1.14.el5xen-2.6.18-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-atl2-2.6.18-128.1.14.el5-1.0.40.4-4.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-atl2-2.6.18-128.1.14.el5xen-1.0.40.4-4.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-aufs-2.6.18-128.1.14.el5-0.20080605.cvs-5.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-aufs-2.6.18-128.1.14.el5xen-0.20080605.cvs-5.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-firewire-2.6.18-128.1.14.el5-2.6.18-2.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-firewire-2.6.18-128.1.14.el5xen-2.6.18-2.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-gspcav1-2.6.18-128.1.14.el5-01.00.20-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-gspcav1-2.6.18-128.1.14.el5xen-01.00.20-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-qla2xxx-2.6.18-128.1.14.el5-8.01.07.15-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-qla2xxx-2.6.18-128.1.14.el5xen-8.01.07.15-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-128.1.14.el5-0.2.0-1.svn255.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-128.1.14.el5xen-0.2.0-1.svn255.slp5.x86_64.rpm - /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-gpfs-2.6.18-128.1.14.el5-3.2.1-12.3.slp5.psi.x86_64.rpm - /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-openafs-2.6.18-128.1.14.el5-1.4.10-94.slp5.x86_64.rpm - /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-openafs-2.6.18-128.1.14.el5xen-1.4.10-94.slp5.x86_64.rpm - /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-VMwareTools-2.6.18-128.1.14.el5-6530-29999.slp5.x86_64.rpm - - List of files saved in /tmp/rpms_for_2.6.18-128.1.14.el5 ! - - -Note to Kernel Module Openafs -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The openafs related build scripts are provided by Achim Gsell. - -If the kernel version is increased, the kernel-module-openafs has to -be built. - -If the Openafs version is increased, the kernel-module-openafs **AND** -the additional openafs RPMS (see list below) have to be built:: - - openafs - openafs-client - openafs-kpasswd - openafs-krb5 - - -One can use the following standalone script to built openafs modules. - -- `build-openafs.sl51.i386.sh` - - Standalone script for building and copying 32 bit openafs modules - -- `build-openafs.sl51.x86_64.sh` - - Standalone script for building and copying 64 bit openafs modules - - -Note to Kernel Module Uvcvideo -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If building fails, download the latest uvcvideo version and compile -again:: - - # cd ~/tmp/ - # svn checkout http://svn.berlios.de/svnroot/repos/linux-uvc/linux-uvc/trunk # see also in specfile - # mv trunk uvcvideo-svn255 - # tar cfzv uvcvideo-svn255.tar.gz uvcvideo-svn255/ - # scp uvcvideo-svn255.tar.gz tux50-64:/scratch/gasser_m/rpm_topdir/SOURCES/ - # ssh tux50-64 - # cd /scratch/gasser_m/rpm_topdir/SPECS/ - # vi uvcvideo.spec # Version: 0.2.0 - # %define date svn255 - # rpmbuild -ba --target x86_64 --define "kernel 2.6.18-92.1.13.el5" --define 'buildall 1' uvcvideo.spec - - ... - Wrote: /scratch/gasser_m/rpm_topdir/RPMS/x86_64/uvcvideo-0.2.0-1.svn255.slp5.src.rpm - Wrote: /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-92.1.13.el5-0.2.0-1.svn255.slp5.x86_64.rpm - ... - - # echo "/scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-92.1.13.el5-0.2.0-1.svn255.slp5.x86_64.rpm" >> \ - # /tmp/rpms_for_2.6.18-92.1.13.el5 - - -Don't forget to copy all the new modules to -`/afs/psi.ch/software/linux/dist/scientific/51/psi/all/`. - - -Check The Kernel Module Build Logs -,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, - -Check the log files for errors and verify that all required modules -were built:: - - # grep -i error ~/tmp/20090624-build_kernel_modules-2.6.18-128.1.14.el5.x86_64.output - ... - checking your OS... configure: error: No usable linux headers found at /lib/modules/2.6.18-128.1.14.el5PAE/build - + exit 1 - error: Bad exit status from /scratch/gasser_m/rpm_tmp/rpm-tmp.69701 (%build) - Bad exit status from /scratch/gasser_m/rpm_tmp/rpm-tmp.69701 (%build) - ... - - -This error is ok, as there is no x86_64 PAE kernel at all:: - - # check-set-of-built-kernel-modules.bash /tmp/rpms_for_2.6.18-128.1.14.el5 \ - # KERNEL_MODULES_TO_BUILD/sl5.i386.kms-to-be-built - # Not all kernel module RPMS were built! - # Check RPM(s) for: 'VMwareTools' - - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-airprime-2.6.18-128.1.14.el5-2.6.18-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-airprime-2.6.18-128.1.14.el5xen-2.6.18-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-atl2-2.6.18-128.1.14.el5-1.0.40.4-4.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-atl2-2.6.18-128.1.14.el5xen-1.0.40.4-4.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-aufs-2.6.18-128.1.14.el5-0.20080605.cvs-5.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-aufs-2.6.18-128.1.14.el5xen-0.20080605.cvs-5.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-firewire-2.6.18-128.1.14.el5-2.6.18-2.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-firewire-2.6.18-128.1.14.el5xen-2.6.18-2.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-gspcav1-2.6.18-128.1.14.el5-01.00.20-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-gspcav1-2.6.18-128.1.14.el5xen-01.00.20-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-qla2xxx-2.6.18-128.1.14.el5-8.01.07.15-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-qla2xxx-2.6.18-128.1.14.el5xen-8.01.07.15-1.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-128.1.14.el5-0.2.0-1.svn255.slp5.x86_64.rpm - /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-128.1.14.el5xen-0.2.0-1.svn255.slp5.x86_64.rpm - /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-gpfs-2.6.18-128.1.14.el5-3.2.1-12.3.slp5.psi.x86_64.rpm - /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-VMwareTools-2.6.18-128.1.14.el5-6530-29999.slp5.x86_64.rpm - /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-openafs-2.6.18-128.1.14.el5-1.4.10-94.slp5.x86_64.rpm - /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-openafs-2.6.18-128.1.14.el5xen-1.4.10-94.slp5.x86_64.rpm - - -This also seems to be ok, as there was never a VMWare kernel module in -the repository for Xen and PAE kernel. - - -Copy The Kernel Module RPMS To The SL 5 Repository -,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, - -Eventually, copy the built kernel module RPMS to the corresponding -PSI-ALL repository:: - - # cp -av `cat /tmp/rpms_for_2.6.18-128.1.14.el5` /afs/psi.ch/software/linux/dist/scientific/51/psi/all/ - - -Check whether the gpfs daemon and the gpfs kernel module RPMS were -copied, too, and have corresponding versions. - -Probably you have to build them first using -e.g. `build-gpfs.sl54.x86_64.sh` on tux50-64:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/ - # ./build-gpfs.sl5x.x86_64.sh - # check-for-gpfs-occurence.bash 2.6.18-164.9.1.el5 3.2.1-16 4 51 - - # Begin checking for file occurence... - #/afs/psi.ch/software/linux/dist/scientific/51/nonfree/all/gpfs-3.2.1-16.4.slp5.psi.x86_64.rpm - # ok - #/afs/psi.ch/software/linux/dist/scientific/51/psi/all/kernel-module-gpfs-2.6.18-164.9.1.el5-3.2.1-16.4.slp5.psi.x86_64.rpm - # ok - #/afs/psi.ch/software/linux/dist/scientific/51/nonfree/all/gpfs-init-script-1.0-4.slp5.psi.x86_64.rpm - # ok - # End checking for file occurence. - - -Make The New Kernel Repository For SL5 -,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, - -The next two commands, `make kernel_all` and `make kernel`, will -process both architectures, i386 and x86_64. So you can run it only -once, after having built i386 and x86_64 kernel modules. - -Running `make kernel_all` will copy the built kernel module RPMS and -other kernel related RPMS from the corresponding ...-ALL to the -KERNEL-ALL directories, e.g.:: - - from: /afs/psi.ch/software/linux/dist/scientific/51/update.x86_64/all/ - to: /afs/psi.ch/software/linux/dist/scientific/51/kernel/all/ - - -Eventually `make kernel` will create the new kernel snapshot in -`/afs/psi.ch/software/linux/dist/scientific/51/kernel/`:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/ - # make kernel_all > UPDATE_LOG/$(date +%Y-%m-%d)-make-kernel_all-2.6.18-164.11.1.el5.sl51.i386.x86_64.log 2>&1 - - # KERNEL_VER=2.6.18 KERNEL_REL=371.12.1.el5 make kernel - - -Follow the instructions at the end of the output of `make kernel` to -update symlinks, e.g:: - - # rm /afs/psi.ch/software/linux/dist/scientific/51/kernel/testing - # ln -s 2.6.18-371.12.1.el5 /afs/psi.ch/software/linux/dist/scientific/51/kernel/testing - - -Create Snapshots of Repositories --------------------------------- - -Both architectures, i386 and x86_64, are treated by the following -commands. - -If new RPMS appear in a certain repository, which should be available -for the next update, a new snapshot is created for this repository. -The folder name of the snapshot will be equal to the current date. - -Create all snapshots at once (gfa excluded as Heiner Billich and Rene -Kapeller will create this snapshot). This can take up to one hour:: - - #[gasser_m@tux50] - # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts - # nohup ./run_make_snapshots_all.sh & - - -As an alternative you could make a single snapshot of a certain -repository only. E.g. to make a snapshot of the `update` and `psi` -repos type:: - - # DIR=update make snapshot # DIR=update will set: ALL_DIRS="update.i386 update.x86_64" - # DIR=psi make snapshot - -`make snapshot` will execute -`/afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_snapshot`. -The command `createrepo` invoked in the shell script `.../mk_snapshot` -writes repodata for each snapshot to -e.g. `/afs/psi.ch/software/linux/dist/scientific/51/others/20090625/repodata/`. - -Release the snapshots for first level testing systems ------------------------------------------------------ - -The command `make release_testing` will update the respective symlinks -`.../testing` to the latest snapshots in all repos (again, except -`gfa`). - -Note: Because `make release_testing` is interactive, you should not -redirect the output to a file:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/ - # make release_testing - -Before `make release_testing`: - -Note: The output below originates not from the same update as above, -this was 2008-08-27:: - - # [gasser_m@tux50] - # cd /afs/psi.ch/software/linux/dist/scientific/51/ - # \ls -l */testing - - lrwxr-xr-x 1 markushin ait 8 Aug 8 15:27 cluster/testing -> 20080808 - lrwxr-xr-x 1 markushin ait 8 Aug 8 15:27 enhanced/testing -> 20080808 - lrwxr-xr-x 1 billich ait 8 Aug 27 14:40 gfa/testing -> 20080827 - lrwxr-xr-x 1 markushin ait 8 Aug 8 15:27 kde-redhat/testing -> 20080808 - lrwxr-xr-x 1 markushin ait 18 Aug 8 15:27 kernel/testing -> 2.6.18-92.1.10.el5 - lrwxr-xr-x 1 markushin ait 8 Aug 8 15:27 nonfree/testing -> 20080808 - lrwxr-xr-x 1 markushin ait 8 Aug 8 15:27 others/testing -> 20080808 - lrwxr-xr-x 1 markushin ait 8 Aug 14 15:23 psi/testing -> 20080814 - lrwxr-xr-x 1 markushin ait 8 Aug 8 15:27 update.i386/testing -> 20080808 - lrwxr-xr-x 1 markushin ait 8 Aug 8 15:27 update.x86_64/testing -> 20080808 - - [gasser_m@tux50 51]$ \ls -l */current - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 cluster/current -> 20080808 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 enhanced/current -> 20080808 - lrwxr-xr-x 1 billich ait 8 Aug 27 14:40 gfa/current -> 20080827 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 kde-redhat/current -> 20080808 - lrwxr-xr-x 1 gasser_m ait 18 Aug 14 16:37 kernel/current -> 2.6.18-92.1.10.el5 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 nonfree/current -> 20080808 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 others/current -> 20080808 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 psi/current -> 20080814 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 update.i386/current -> 20080808 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 update.x86_64/current -> 20080808 - - -After `make release_testing`:: - - # [gasser_m@tux50] - # \ls -l */testing - - lrwxr-xr-x 1 gasser_m ait 8 Aug 28 14:58 cluster/testing -> 20080827 - lrwxr-xr-x 1 gasser_m ait 8 Aug 28 14:58 enhanced/testing -> 20080827 - lrwxr-xr-x 1 billich ait 8 Aug 27 14:40 gfa/testing -> 20080827 - lrwxr-xr-x 1 gasser_m ait 8 Aug 28 14:58 kde-redhat/testing -> 20080827 - lrwxr-xr-x 1 gasser_m ait 18 Aug 28 14:58 kernel/testing -> 2.6.18-92.1.10.el5 - lrwxr-xr-x 1 gasser_m ait 8 Aug 28 14:58 nonfree/testing -> 20080827 - lrwxr-xr-x 1 gasser_m ait 8 Aug 28 14:58 others/testing -> 20080827 - lrwxr-xr-x 1 gasser_m ait 8 Aug 28 14:58 psi/testing -> 20080827 - lrwxr-xr-x 1 gasser_m ait 8 Aug 28 14:58 update.i386/testing -> 20080827 - lrwxr-xr-x 1 gasser_m ait 8 Aug 28 14:58 update.x86_64/testing -> 20080827 - - [gasser_m@tux50 51]$ \ls -l */current - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 cluster/current -> 20080808 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 enhanced/current -> 20080808 - lrwxr-xr-x 1 billich ait 8 Aug 27 14:40 gfa/current -> 20080827 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 kde-redhat/current -> 20080808 - lrwxr-xr-x 1 gasser_m ait 18 Aug 14 16:37 kernel/current -> 2.6.18-92.1.10.el5 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 nonfree/current -> 20080808 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 others/current -> 20080808 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 psi/current -> 20080814 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 update.i386/current -> 20080808 - lrwxr-xr-x 1 gasser_m ait 8 Aug 14 16:37 update.x86_64/current -> 20080808 - - -Now, the `testing` symlinks point to the latest snapshots, while the -`current` symlinks still point to the previous snapshots. - - -Make First Tests ----------------- - -Manually update selected `testing` hosts (see `/etc/sysconfig/psi` and -`/etc/yum.conf`) with yum. The `testing` string is set in -`/etc/sysconfig/psi` and `/etc/yum.conf` (i.e. your computer is -looking for new RPMS in the `testing` snapshots):: - - # yum clean all - # yum update - -If there's a new kernel, reboot into this new kernel and especially -check whether the new kernel-module-openafs is working and AFS was -mounted. - -- Note to the `psi-update` script - -`psi-update` will not work at this point, because it's udpating only -if the PSI version has changed, what will be done later (see "Create A -New PSI Version Of SL51" below). `psi-update` will get the script -`yum_update` from our master, pxeserv01, see also `/etc/fstab`. The -script `yum_update` will be saved locally as -`/usr/sbin/psi-yum_update`. Afterwards `psi-yum_update` will be -executed locally. - - -Create A New PSI Version Of SL51 And Release It For Testing Systems -------------------------------------------------------------------- - -In this section a new PSI version is created and released for testing -systems. By default, Green Testing PCs will be auto updated during -the following night. - -The procedure shown here has to be repeated for all SL5 class/subclass -combinations. Contemporary there are three of them: - -1. Desktop/Stable -2. Desktop/Enhanced -3. Cnode/Stable - -To get an overview list the classes directories:: - - # [gasser_m@pc7377 Stable]$ ll -rt /afs/psi.ch/software/linux/dist/scientific/51/classes/* - - /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop: - total 10 - drwxr-xr-x 2 beyerle ait 2048 Jan 19 2008 StableTesting - drwxr-xr-x 2 beyerle ait 2048 Jan 19 2008 EnhancedTesting - drwxr-xr-x 15 beyerle ait 4096 Jul 2 17:11 Enhanced - drwxr-xr-x 15 beyerle ait 2048 Aug 11 14:19 Stable - - /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode: - total 4 - drwxr-xr-x 2 beyerle ait 2048 Jan 19 2008 StableTesting - drwxr-xr-x 15 beyerle ait 2048 Jul 2 17:11 Stable - - -Note: To change the list of RPMS of a certain class edit the -corresponding package list. - -E.g.: To add RPMS in update version 19 of SL51 Desktop/Stable edit -`list.Stable.v19`:: - - # vi /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/list.Stable.v19 - -IMPORTANT: If you add RPMS to the list, update also the RPM lists used -in the kickstart installation:: - - # vi /afs/psi.ch/software/linux/dist/scientific/51/kickstart/custom/DesktopStablei386/yum_install.conf - # vi /afs/psi.ch/software/linux/dist/scientific/51/kickstart/custom/DesktopStablex86_64/yum_install.conf - -- Note: The following tasks can also be performed using the script - `create-new-psi-version.bash`. - - -Desktop/Stable: Release Desktop Stable Testing -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Create folder for Version 19 by running make class:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/ - # cp list.Stable.v18 list.Stable.v19 - - # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/ - # CLASS=Desktop SUBCLASS=Stable version=19 LIST=list.Stable.v19 make class - - ###{ begin output make class - Populating Class directory ... - TOP_DIR=/afs/psi.ch/software/linux/dist/scientific/51 \ - /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class - - Configuration file /afs/psi.ch/software/linux/dist/scientific/51/scripts/dist-config - Running /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class ... - create version and release file - copy list.Stable.v19 to list - ------------------------------------------------------------------------------- - The directory /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMS-20080811 has been populated. - - RPMSv19 links to RPMS-20080811 - - Please check - cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMS-20080811/version - cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMS-20080811/release - cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMS-20080811/list - - If you like to display a message for the user (one line) put it into - emacs /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMS-20080811/message - - If you want to release `testing`, link RPMSTesting to RPMSv19: - rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMSTesting - ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMSTesting - - ------------------------------------------------------------------------------- - ... End of /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class - ###} end output make class - - -Now release it:: - - # rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMSTesting - # ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMSTesting - - -Desktop/Enhanced: Release Desktop Enhanced Testing (also a lot of nonfree etc. stuff) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Create folder for Version 19 by running `make class`:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/ - # cp list.Enhanced.v18 list.Enhanced.v19 - # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/ - # CLASS=Desktop SUBCLASS=Enhanced version=19 LIST=list.Enhanced.v19 make class - - ###{ begin output - Populating Class directory ... - TOP_DIR=/afs/psi.ch/software/linux/dist/scientific/51 \ - /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class - - Configuration file /afs/psi.ch/software/linux/dist/scientific/51/scripts/dist-config - Running /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class ... - create version and release file - copy list.Enhanced.v19 to list - ------------------------------------------------------------------------------- - The directory /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMS-20080811 has been populated. - - RPMSv19 links to RPMS-20080811 - - Please check - cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMS-20080811/version - cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMS-20080811/release - cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMS-20080811/list - - If you like to display a message for the user (one line) put it into - emacs /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMS-20080811/message - - If you want to release "testing", link RPMSTesting to RPMSv19: - rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMSTesting - ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMSTesting - - ------------------------------------------------------------------------------- - ... End of /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class - ###} end output - - -Now release it:: - - # rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMSTesting - # ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMSTesting - - -Cnode/Stable: Release Cnode Stable Testing -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Create folder for Version 19 by running `make class`:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/ - # cp list.Stable.v18 list.Stable.v19 - # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/ - # CLASS=Cnode SUBCLASS=Stable version=19 LIST=list.Stable.v19 make class - - ###{ begin output - Populating Class directory ... - TOP_DIR=/afs/psi.ch/software/linux/dist/scientific/51 \ - /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class - - Configuration file /afs/psi.ch/software/linux/dist/scientific/51/scripts/dist-config - Running /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class ... - create version and release file - copy list.Stable.v19 to list - ------------------------------------------------------------------------------- - The directory /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMS-20080811 has been populated. - - RPMSv19 links to RPMS-20080811 - - Please check - cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMS-20080811/version - cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMS-20080811/release - cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMS-20080811/list - - If you like to display a message for the user (one line) put it into - emacs /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMS-20080811/message - - If you want to release "testing", link RPMSTesting to RPMSv19: - rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMSTesting - ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMSTesting - - ------------------------------------------------------------------------------- - ... End of /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class - ###} end output - - -Now release it:: - - # rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMSTesting - # ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMSTesting - - -Test the Update On Testing Systems ----------------------------------- - -On a testing system run:: - - [root@tux50] - # psi-update - - -Release The Update PSI Wide For Productive Systems --------------------------------------------------- - -Run:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/ - # make current_equal_testing - - -Refresh symbolic links in the `RPMS_all` folder to all respective -RPMS:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts - # make rpms_all - - -Test The Update On Productive Systems -------------------------------------- - -On a productive (=current) system run:: - - # psi-update - - -Update The Webpages -=================== - -Reference ---------- - -https://wiki.intranet.psi.ch/AIT/LinuxRegularUpdate#Update_the_Webpages - -*Possibly it's better to move the reference to the top Reference -section. This section is still in a very sketch state anyway.* - - -General Info to Webpages Providing SL Update Infos --------------------------------------------------- - -The website is public to PSI.#??? or to everybody? - -The site provides information about the SL update progress for users -at PSI. - -Location of the website: - - On AFS: /afs/psi.ch/project/ait/www/services/linux/updates/ - On WWW: http://ait.web.psi.ch/services/linux/updates/ - - -Update the following Webpages: - -- http://ait.web.psi.ch/services/linux/updates/ - -- http://ait.web.psi.ch/services/linux/updates/update_sl5.html - -- http://ait.web.psi.ch/services/linux/updates/update_sl4.html - -- http://ait.web.psi.ch/services/linux/updates/update_sl3.html - - -- Objective - -Show the updated RPMS between current and the release before. Reduce -amount of info to a reasonable minimum and represent clearly for -users. Don't show RPMS, which are in the repository but will not be -installed because of a newer version of this RPM in the same snapshot. - - -Script location: /afs/psi.ch/project/ait/www/services/linux/updates/ - find_new_rpms_between 20080626 51 18 20080808 51 19 The command - above will not yet work on SL5, because of directory layout - changes between SL4 and SL5. For now we will take another - procedure to prepare the HTML files. - -The following command will show some first scratch of an alternative -way to get the list with updated RPMS:: - - # cd /afs/psi.ch/software/linux/dist/scientific/51 - # diff <( ls -1 */testing/ | sort ) <( ls -1 */current/ | sort ) | grep '^<' | sed -e 's/.i.86.rpm//' -e 's/.x86_64.rpm//' | uniq - -What do we need? In SL5 we have the following classes/subclasses:: - - # - - ¦---Stable - ¦-Cnode---¦ - ¦ ¦---StableTesting - ¦ - ¦ - sl5---¦ - ¦ ¦--Stable - ¦ ¦ - ¦ ¦--StableTesting - ¦-Desktop-¦ - ¦--Enhanced - ¦ - ¦--EnhancedTesting - - - ls -1 */testing/ | grep -v testing | grep -v x86_64 | sort > ~/tmp/sl51-v19-ls-asterisk-testing-sort-out - ls -1 */current/ | grep -v current | grep -v x86_64 | sort > ~/tmp/sl51-v19-ls-asterisk-current-sort-out - /afs/psi.ch/user/g/gasser_m/tmp/ - mengen.py -D sl51-v19-ls-asterisk-testing-sort-out sl51-v19-ls-asterisk-current-sort-out > sl51-v19-mengen-out - get-latest-RPM-from-RPM-list.pl sl51-v19-mengen-out > sl51-v19-updated-RPMS.list - - -From sl51-v19-updated-RPMS.list take the list to insert into the HTML -update site. Edit file -`/afs/psi.ch/project/ait/www/services/linux/updates/update_sl5.html`. - - -Are all clients up-to-date? ---------------------------- - -Login to pxeserv01. - -Note to puppet and `/etc/sysconfig/psi` -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Even puppet and not cfengine is running on SL5, latest copies of -`ANYHOST:/etc/sysconfig/psi` from SL3, SL4 and SL5 systems are put to -`pxeserv01:/opt/cfengine/sysconfig/`:: - - # cd /opt/cfengine/scripts - # ./find_hosts_with_old_version.sh 98 71 20 - - running... - ./cfengine.lcmeg01:VERSION=70 - ./cfengine.lclth01:VERSION=70 - ./cfengine.lclev03:VERSION=70 - ./cfengine.pc7148:VERSION=18 - ./cfengine.lclev01:VERSION=70 - ./cfengine.lclth04:VERSION=70 - ./cfengine.pc6702:VERSION=69 - ./cfengine.lclth07:VERSION=70 - ./cfengine.pc3384:VERSION=97 - ./cfengine.slsnedi1:VERSION=17 - ./cfengine.pc5082:VERSION=91 - ./cfengine.lpc4358:VERSION=69 - ./cfengine.lclev05:VERSION=70 - ./cfengine.lclth08:VERSION=70 - ./cfengine.lclth02:VERSION=70 - ./cfengine.llc6:VERSION=69 - ./cfengine.lclrs11:VERSION=69 - ./cfengine.pc5835:VERSION=96 - ./cfengine.lclth06:VERSION=70 - ./cfengine.tux40e:VERSION=70 - ./cfengine.lclev02:VERSION=70 - ./cfengine.lclth03:VERSION=70 - ./cfengine.pc4426:VERSION=17 - ./cfengine.pcmic01:VERSION=70 - ./cfengine.lclth05:VERSION=70 - done! - - -If a host is not up-to-date, login to the host and run manually:: - - # psi-update - - -You may also want to check the logs in `/var/log/update/` diff --git a/admin-guide/legacy/monitoring.rst b/admin-guide/legacy/monitoring.rst deleted file mode 100644 index 1af8410..0000000 --- a/admin-guide/legacy/monitoring.rst +++ /dev/null @@ -1,34 +0,0 @@ -Monitoring and Reporting -======================== - - -Cron jobs ---------- - -Cron jobs send mail on error, usually to `linuxmaster@psi.ch`, a -distribution list. - - -Nagios/Icinga -------------- - - -Ganglia -------- - - -Other ------ - -Some systems dump information in -``/afs/psi.ch/service/linux/statistics/rawdata``. Others save it to -``/afs/psi.ch/intranet/Controls/tmp/sysreport/PSI``. - -The PXE server `pxeserv01` regularly runs a script, -`/afs/psi.ch/service/linux/tftpboot/tools/script/pxe_script`, which copies -`/var/log/tftp.log` to `/afs/psi.ch/service/linux/tftpboot/tools/log/`, -compresses it, and produces statistics, which are saved as `pxeStatistik-$(date -+%Y%m%d).txt`. - -The script is currently run as user `vojisavljevic` and the run is configured in -that user's crontab. diff --git a/admin-guide/legacy/puppet.rst b/admin-guide/legacy/puppet.rst deleted file mode 100644 index 10a71c9..0000000 --- a/admin-guide/legacy/puppet.rst +++ /dev/null @@ -1,8 +0,0 @@ -Puppet -====== - -There have been and still are several instances of Puppet PSI. - -- ``pxeserv01`` (obsolete, no puppetmaster running) -- ``psi-puppet3`` (used by Controls-IT) -- ``psi-puppet4`` diff --git a/infrastructure-guide/infrastructure_systems.md b/infrastructure-guide/infrastructure_systems.md index 3dc7f33..f8be0d2 100644 --- a/infrastructure-guide/infrastructure_systems.md +++ b/infrastructure-guide/infrastructure_systems.md @@ -3,7 +3,7 @@ List of systems and their primary role: __Core Infrastructure:__ -* [boot.psi.ch](pxeserv01) - TFTP server for PXE booting +* [boot.psi.ch](boot_server) - TFTP server for PXE booting * [sysdb.psi.ch](sysdb_server) - Runs sysdb, providing the dynamic iPXE, Grub and kickstart files diff --git a/infrastructure-guide/pxeserv01.md b/infrastructure-guide/pxeserv01.md index 10d036a..c98b8e5 100644 --- a/infrastructure-guide/pxeserv01.md +++ b/infrastructure-guide/pxeserv01.md @@ -1,71 +1,10 @@ -# pxeserv01 - -The server is a RHEL 8, installed manually and registered directly with redhat.com . This is so that it's completely independent from anything else at PSI. - -For historical reasons, the tftpboot directory is hosted on AFS. But for an unknown reason, the RHEL8 tftpd can not read the files from AFS. (It's not SELinux) So the data is mirrored to the local drive and is served from this copy. - -See /etc/crontab -There is a cron job defined in /etc/crontab to sync the content of the `/afs` directory to the local `/tftpboot` directory. -This job runs **every minute** and is defined as follows: - -``` -* * * * * root rsync -aah --exclude '*rhel-8-poc*' --delete /afs/psi.ch/service/linux/tftpboot/ /tftpboot -``` +# boot.psi.ch This server hosts the tftp service (port 69) used for pxe booting. -Permissions /tftpboot directory: -```bash -[root@pxeserv01 ~]# du -sh /tftpboot/ -5.2G /tftpboot/ -[root@pxeserv01 ~]# ls -lad /tftpboot/ -drwxr-xr-x 13 5122 840 4096 Mar 4 17:26 /tftpboot/ -``` - -Permissions of the /afs tftpboot directory: -``` -[root@pxeserv01 tftpboot]# fs listacl -Access list for . is -Normal rights: - psi:nodes rl - svc.linux:administrators rlidwka - svc.linux:pxe rl - svc.linux:readonly rl - svc.linux:tools rl - svc.linux:users l - web:hosts rl -[root@pxeserv01 tftpboot]# pwd -/afs/psi.ch/service/linux/tftpboot -[root@pxeserv01 tftpboot]# -``` - -Current members of linux.administrators: -``` -$ pts membership svc.linux:administrators -Members of svc.linux:administrators (id: -10574) are: - system:administrators - ebner - lutz_h - stadler_h - kapeller - huesser - feichtinger - barabas - sala - gsell - ozerov_d - talamo_i - dorigo_a - nazlikul_m - caubet_m - taylor_j - ext-metzler_d - buchel_k -``` - # Important -For any changes in the pxe config settings there might be a delay of 1 minute before the clients see the changes! +For any changes in the pxe config settings there might be a delay of 30 seconds before the clients see the changes! # pxelinux.cfg directory @@ -77,48 +16,4 @@ There are several syntax variants, which can be used for specific pxelinux confi (/afs/psi.ch/service/linux/tftpboot/grub.cfg* /tftpboot/grub.cfg*) Beside the default `grub.cfg` there can similar to `pxelinux` a host or network specific `grub` configuration files. -For details see the [Grub Manual](https://www.gnu.org/software/grub/manual/grub/grub.html#Network), but note that the MAC address documentation is wrong: smiliar as for `pxelinux` the MAC address must be prepended with `01-` to denote the network hardware type Ethernet. - -# Questions -- is there a special tftpd configuration, if yes, where? - - It's not really a config, but the systemd unit file, which is changed from the default -``` -/usr/lib/systemd/system/tftp.service -``` - -- chronyd has a port open on 323, for what is this needed? - - That's nothing special or custom, that's just how chronyd works by default - -- Are firewall rules set explicitly? Current firewall rules: -``` -[root@pxeserv01 ~]# firewall-cmd --list-all -public (active) - target: default - icmp-block-inversion: no - interfaces: ens192 - sources: - services: cockpit dhcpv6-client ssh tftp - ports: - protocols: - masquerade: no - forward-ports: - source-ports: - icmp-blocks: - rich rules: -[root@pxeserv01 ~]# -``` -- answer: - - tftp is opened up additionally, nothing more - -``` -firewall-cmd --zone=public --add-service=tftp --permanent -firewall-cmd --zone=public --add-service=tftp -``` - -- is this service also used for windows systems? (I can see a `uefiwin` directory in the tftpboot directory) - - yes, this is the one and only tftp server for all PSI networks - - - The one person I ever talked to about PXE booting and windows was Niklaus Baumann. - -- Can you please explain the structure of the /tftpboot directory - which directories are populated by which services/users. Who is managing all this content? Is there additional documentation on this? - - Not really, it's a mess I inherited. On legacy mode, pxelinux.0 is loaded, on uefi, it is the grubx64.efi . These have their configs in pxelinux.cfg and grub.cfg respectively. +For details see the [Grub Manual](https://www.gnu.org/software/grub/manual/grub/grub.html#Network), but note that the MAC address documentation is wrong: smiliar as for `pxelinux` the MAC address must be prepended with `01-` to denote the network hardware type Ethernet. \ No newline at end of file