Updates for SL 4 and SL 5.1
===========================

References
----------

- https://wiki.intranet.psi.ch/AIT/LinuxRegularUpdate (U. Beyerle)

- https://wiki.intranet.psi.ch/AIT/SL5RegularUpdates200807 (V. Markushin)


Introduction
------------

The following text describes how regular Scientific Linux (SL) updates
for SL51 and SL4, are performed.

A detailed standard procedure is exemplified for an SL 51 update on a
64 bit machine, but the same operation is followed for other SL
releases <= SL51 and architectures. Deviations from the standard
procedure are emphasized in the text.


General Remarks
~~~~~~~~~~~~~~~

Regular Linux Udpates are scheduled every first Wednesday in a month.

Create a new Testing Release one week in advance.

It is advised to create the new version for a release on the
respective build system.  For example for SL5 i386 do it on tux50 with
your AFS account, for AFS ACLs see "Required File Permissions On AFS"
below.

When you compile new kernel module RPMS then it's recommended to use
build systems.


Reasons For Using Build Systems
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Performance: Some scripts will just horribly slow down your desktop
  machine, making it almost impossible to work on it while the
  respective script is running.  So, using a build system will not
  affect the performance of your desktop computer.

- Version conflicts: Some versions of some tools such as yum,
  createrepo, etc., might be different in different SL releases, thus
  to avoid conflicts one should use the version of the tool that comes
  with the corresponding release.  Other packages might be missing at
  all, e.g. some devel packages used when RPMS are built or tool like
  createrepo, which are not required on a default Desktop.

- Compiling software: For compiling software, e.g. building kernels,
  kernel modules, etc., it first has to be on the right architecture,
  and second it's also recommended to do it in some clean default
  environment, which is a representative for the environments of
  productive systems.


Scientific Linux Mirror at PSI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

RPMS are taken from our mirror at
`/afs/psi.ch/software/mirror/scientific/`.  The mirror is updated
daily by a cronjob running on pxeserv01. (See also
LinuxInstallationServer.)::

    # file pxeserv01:/etc/cron.d/mirror 

    # Mirroring of rpm repositories at 07:00
    #
    00 07 * * * root /afs/psi.ch/software/mirror/scripts/mirror.sh > /dev/null 2>&1

The script `/afs/psi.ch/software/mirror/scripts/mirror.sh` reads
`/afs/psi.ch/software/mirror/scripts/mirror.conf` to know about
locations of ftp external software repositories and where to place the
files on our local mirror.

E.g.: The archive
`ftp://ftp.scientificlinux.org/linux/scientific/46/i386` is locally
mirrored at `/afs/psi.ch/software/mirror/scientificlinux/46/i386`.


Autoupdate in SL5
~~~~~~~~~~~~~~~~~

The script `/usr/sbin/psi-auto-update` is started via cron usually in
the night and triggers the update, the latter is performed by scripts
and Puppet. The update is released the first Wednesday of a
month. Default Desktop computers will be updated during the night from
Wednesday to Thursday.


Preparatory Work
----------------

Required File Permissions On AFS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Before starting with performing updates, you need the corresponding
permissions to do things in the PSI linux repository directory at
`/afs/psi.ch/software/linux/dist/scientific/`, which is on an AFS
volume.

Check permissions using AFS commands and get the required AFS access.
The unix commands `ls -l` etc. will work but not show you the relevant
AFS permissions, instead use the AFS command `fs`.

E.g.: Type `fs help` to see a list of command attributes::

    # fs listacl /afs/psi.ch/software/linux/
    ...
        or

    # fs listacl /afs/psi.ch/software/mirror/
    Access list for /afs/psi.ch/software/mirror is
    Normal rights:
    ait:sw_adm rlidwka
    psi:nodes rl
    system:authuser rl

To work in the linux distribution directories you need to be in the
following AFS groups (output of `pts mem AFS_USERNAME`)::
    
    # pts mem gasser_m
    Groups gasser_m (id: 2374) is a member of:
    ait:sw_adm
    linux:users
    ait:users


Prepare Your Build Environment For Building RPMS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For building RPMS you have two possibilities:

- build locally in your build directories

- build on AFS in common build directories


Build locally
.............

Before you start building RPMS you create the following directories on
each build system.

Example for tux50-64::

    # ssh gasser_m@tux50-64
    # mkdir /scratch/gasser_m
    # cd /scratch/gasser_m
    # mkdir rpm_tmp  rpm_topdir
    # cd rpm_topdir
    # mkdir BUILD  RPMS  SOURCES  SPECS  SRPMS


Your build environment is set in "~/.rpmmacros"::

    %WORKDIR    %(echo "/scratch/$USER")
    %_topdir    /scratch/gasser_m/rpm_topdir
    %_tmppath       /scratch/gasser_m/rpm_tmp
    %packager       Marc Gasser <marc.gasser@psi.ch>
    %vendor     PSI Scientific Linux
    %sl_site    slp
    %sl_site_version    %(rpm -q --qf "%{v}" sl-release | cut -d"." -f1)
    %os_release         %{sl_site}%{sl_site_version}
    %dist           .%{os_release}
    %rhel           %sl_site_version
    %debug_package  %{nil}


The command `rpmbuild` is used to build RPMS.  The built RPMS are
found in the RPMS directory of your build environment on your build
system.


Build on AFS
............

Change to `/afs/psi.ch/project/linux/src/scientific/RPMS`.  There you
will find separate subdirectories, each for one specific RPM.

Check that your `.rpmmacros` does not mess up the build environment
for this kind of building. Change to the subdirectory of the RPM you
want to build and use the script `psi-rpmbuild`, which sets up the
build environment (thus `.rpmmacros` is not required at all).  In the
subdirectory run `psi-rpmbuild -bb` for building a binary package only
for instance.

E.g.: To build the psi-desktop package version 2.x do the following on
a build system::

    # cd psi-desktop-2/
    # psi-rpmbuild -bb


The built RPM can be found in the corresponding subdirectory in
`/afs/psi.ch/project/linux/dist/`.

The big advantages of this procedure are, you don't have to install
the sources on each build system, instead you install them once in
AFS, and this centralized build environment can be shared among
different users on different hosts.


Check Diskspace
~~~~~~~~~~~~~~~

Before starting the update check the diskspace left on your file
systems involved.

- For copying new RPMS from the mirror to the repositories

To be on the safe side there should be 1 G of free diskspace on the
respctive AFS volumes, where RPMS are copied to::

    [gasser_m@pc7377 tmp]$ fs lq /afs/psi.ch/software/linux/dist/scientific/305/
    Volume Name           Quota      Used %Used   Partition
    sw.dist.sl305          25000000  23317560   93%<<       88%    <<WARNING
    [gasser_m@pc7377 tmp]$ fs lq /afs/psi.ch/software/linux/dist/scientific/46/
    Volume Name           Quota      Used %Used   Partition
    sw.dist.sl46           25000000  23465039   94%<<       84%    <<WARNING
    [gasser_m@pc7377 tmp]$ fs lq /afs/psi.ch/software/linux/dist/scientific/51
    Volume Name           Quota      Used %Used   Partition
    sw.dist.sl51           30000000  25044971   83%     84%

- For compiling kernel modules

  1G should be enough if you have to compile kernel modules locally on
  your respective build system, e.g. in the `$USER@tux50:/scratch/...`
  directory.


Update SL5 i386 and x86_64
--------------------------

Get Native Scientific Linux Updates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This section describes how the particular linux repositories are
updated by looking for new RPMS in our mirror and copying them from
there to the repositories.

Get the Latest Security Update RPMS
...................................

Get the latest security update RPMS from the local SL51 mirror for
both architectures, i386 and x86_64 by invoking `make update_all` on
tux50::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/
    # nohup ./run_make_update_all.sh &      # this will take some time

The upper `make update_all` command invokes the shell script
`/afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_update_all`.
The script searches for new security update RPMS, which are not
allready in the corresponding `.../update.${ARCH}/all/` directory
(e.g. `/afs/psi.ch/software/linux/dist/scientific/51/update.i386/all/`).
Thus, when updating all SL versions (3, 4 and 5) you need to run this
three times, each time from the corresponding `.../scripts/`
directory.  Yet, you don't have to invoke it again and again for each
architecture, as the script loops through all architectures set by
`$ARCHS` in the respective dist-config file
(e.g. `/afs/psi.ch/software/linux/dist/scientific/51/scripts/dist-config`).

Check for errors in the corresponding `make update_all` log::

    # grep -i error ~/tmp/20090629-make-update_all.sl5.output
    ...

Then run `make rpms_all`. It will keep all symlinks up to date in the
directory `/afs/psi.ch/software/linux/dist/scientific/51/RPMS_all/` by
removing dead links and creating new links to the new RPMS.  This is
basically not necessary for keeping up to date the yum repositories,
rather it's just helpful to have a directory with the list of all RPMS
of a distribution release::

    # make rpms_all


Additional RPMs used in Scientific Linux PSI (SLP)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Reference: https://wiki.intranet.psi.ch/AIT/RPMSinSLP

Get manually other RPMS from other sources and put them to the
particular `all` folders.

Afterwards, you should run createrepo on the directory where things
were put into.  Then run "make rpms_all" or create symlinks manually.

List of some `all` folders::

    /afs/psi.ch/software/linux/dist/scientific/51/psi/all
    /afs/psi.ch/software/linux/dist/scientific/51/others/all
    /afs/psi.ch/software/linux/dist/scientific/51/nonfree/all
    /afs/psi.ch/software/linux/dist/scientific/51/enhanced/all
    /afs/psi.ch/software/linux/dist/scientific/51/cluster/all
    ...


`acroread`
..........

Get acroread RPMS from e.g. RHEL network (https://rhn.redhat.com,
login: psiaitora; passwd isrhn.ait.)  and copy them into
"/afs/psi.ch/software/linux/dist/scientific/51/nonfree/all/"::

    acroread-8.1.6-2.el5.i386.rpm
    acroread-plugin-8.1.6-2.el5.i386.rpm


Write also the md5sums into a file and check them after downloading
the packages.

Format of the md5sum file::

    <md5sum><space><space><filename>

    728d4109f520336bb6b5f595de9dd2db  acroread-8.1.6-2.el5.i386.rpm
    fd1b8fb934140222460ed81c34420d90  acroread-plugin-8.1.6-2.el5.i386.rpm

    # md5sum -c file.md5sum


Kernel Update SL5
-----------------

This section describes the kernel update for SL51, 32 and 64 bit.

The SL5 kernel update procedure described here is basically the same
for SL4.  However, a lot of details such as names of files, kernel
modules, build systems, etc., differ, thus the whole SL4 procedure is
shown in its own section, too. See `Kernel Update SL4`.

The general remarks below are valid for SL4, too.


General Remarks Referring To Kernel Update
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When a new SL kernel is released also a lot of corresponding kernel
module packages are coming with this kernel to extend its
functionality, e.g. kernel-module-xfs to facilitate Xfs support of the
kernel.

Now, in case a new kernel has been released you have to build a set of
kernel-module RPMS, which at PSI are provided in addition to those
coming with the regular SL updates.

It's recommended to use dedicated build systems to build RPMS.

To compile kernel modules some prerequisites have to be met.

- A build environment has to be set up on your build system
  (e.g. tux50-64 for SL5 x86_64), locally or on AFS as discussed
  before.

- New kernel and kernel-devel RPMS have to be installed on your build
  system.

- The particular kernel module source RPMS have to be installed on
  your build system.

- About 1 G of free diskspace.

- Note: For compiling new kernel modules the corresponding kernel has
  to be installed on your build system but must not be running and you
  can build the modules in a non-root environment.


Kernel Update SL5 32-bit
........................

Follow the procedure shown in the 64-bit section below. Differences
between building of the 32 and 64 bit kernel modules are pointed out.

The main difference is, first, that you have to build kernel,
kernel-xen and kernel-PAE modules for 32 bit systems, while you build
only kernel and kernel-xen modules for 64 bit. The GPFS kernel module
is build for both, 32 and 64 bit systems.


Kernel Update SL5 64-bit
........................

(Example for PSI update SL5 version 31, June 2009.)

Login to your build system and check which kernel packages are already
installed::

    # [gasser_m@tux50-64 ~]$ rpm -q kernel{,-xen,-devel,-xen-devel}

    kernel-2.6.18-92.1.22.el5
    kernel-2.6.18-128.1.1.el5
    kernel-2.6.18-128.1.6.el5
    kernel-xen-2.6.18-92.1.22.el5
    kernel-xen-2.6.18-128.1.1.el5
    kernel-xen-2.6.18-128.1.6.el5
    kernel-devel-2.6.18-92.1.22.el5
    kernel-devel-2.6.18-128.1.1.el5
    kernel-devel-2.6.18-128.1.6.el5
    kernel-xen-devel-2.6.18-92.1.22.el5
    kernel-xen-devel-2.6.18-128.1.1.el5
    kernel-xen-devel-2.6.18-128.1.6.el5


Check whether there is a newer kernel package in the repository::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/update.x86_64/all/
    # ls -1 kernel*-2.* | grep -v module | get-latest-RPM-from-RPM-list.pl -
    ...
    kernel-2.6.18-128.1.14.el5

Latest version of installed kernels is `2.6.18-128.1.6.el5` , latest
version in the repository is `2.6.18-128.1.14.el5`, i.e. new SLP
kernel modules have to be built.

Check the diskspace of your build partition, 1G is ok::

    # [gasser_m@tux50-64 all]$ df -hT /scratch
    Filesystem    Type    Size  Used Avail Use% Mounted on
    /dev/mapper/VolGroup00-LogVol00
              ext3     14G   12G  1.6G  89% /


Is there a listing of all kernel module packages to be built?


Installation Of Kernel And Kernel-devel RPMS On 64 Bit SL5 Build System
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Install new kernel and kernel-devel RPMS as root on tux50-64.

RPMS for the following kernels are required: kernel, kernel-xen::

    # [root@tux50-64]
    # cd /afs/psi.ch/software/linux/dist/scientific/51/update.x86_64/all/
    # ls -1 kernel-{,devel-,xen-,xen-devel-}2.6.18-128.1.14.el5.x86_64.rpm
    kernel-2.6.18-128.1.14.el5.x86_64.rpm
    kernel-devel-2.6.18-128.1.14.el5.x86_64.rpm
    kernel-xen-2.6.18-128.1.14.el5.x86_64.rpm
    kernel-xen-devel-2.6.18-128.1.14.el5.x86_64.rpm

    # rpm -ivh kernel-{,devel-,xen-,xen-devel-}2.6.18-128.1.14.el5.x86_64.rpm

    warning: kernel-2.6.18-128.1.14.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 82fd17b2
    Preparing...        ########################################### [100%]
       1:kernel-xen-devel       ########################################### [ 25%]
       2:kernel         ########################################### [ 50%]
       3:kernel-devel       ########################################### [ 75%]
       4:kernel-xen         warning: /etc/modprobe.d/blacklist-firewire created as /etc/modprobe.d/blacklist-firewire.rpmnew
    ########################################### [100%]


Installation Of Kernel And Kernel-devel RPMS On 32 Bit SL5 Build System
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Again, login as root to your build system to install the kernel
packages.

For the 32 bit system also the PAE kernel modules are required::

    # [root@tukan50-32]
    # cd /afs/psi.ch/software/linux/dist/scientific/51/update.i386/all/
    # ls -1 kernel-{,devel-,PAE-,PAE-devel-,xen-,xen-devel-}2.6.18-128.1.14.el5.i686.rpm 
    # rpm -ivh kernel-{,devel-,PAE-,PAE-devel-,xen-,xen-devel-}2.6.18-128.1.14.el5.i686.rpm 


Installation Of SL5 Kernel Module SRPMS On Build Systems
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Do this on both, the 32 and 64 bit build system.

To build the kernel module RPMS the corresponding SRPMS have to be
installed on your build system first.

List of SL5 kernel-module RPMS to be built for i686 and x86_64::

    Name (without kernel version)   Release         Remarks 
    kernel-module-airprime      2.6.18-1.slp5 
    kernel-module-atl2          1.0.40.2-1.slp5 
    kernel-module-aufs          0.20070210.cvs-4.slp5 
    kernel-module-firewire      2.6.18-2.slp5 
    kernel-module-gspcav1       01.00.20-1.slp5 
    kernel-module-iwlwifi       1.2.25-1.slp5       currently not builded
    kernel-module-ntfs          2.1.27-0.rr.10.0    currently not builded
    kernel-module-openafs       1.4.6-73.slp5       no SRPM, build script provided by A. Gsell
    kernel-module-qla2xxx       8.01.07.15-1.slp5 
    kernel-module-uvcvideo      0.1.0-1.svn148.slp5 
    kernel-module-VMwareTools       6530-29999.slp5     no SRPM, build script provided by A. Gsell
    kernel-module-gpfs          texttext        64 bit only, no SRPM, build script provided by H.C. Stadler


Copy the PSI relevant source RPMS (SRPMS) to your build environment,
except sources of kernel modules ntfs, openafs, gpfs and
VMwareTools. They are built in their own environments.

The SRPMS of the following 7 modules are required::

    aufs firewire gspcav1 uvcvideo qla2xxx atl2 airprime
    (see variable specfiles in /afs/psi.ch/software/linux/dist/scientific/51/scripts/build_kernel_modules.)


They can be found in the directory::

    /afs/psi.ch/software/linux/dist/scientific/51/psi/all/


Check for latest versions of required SRPMS::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/psi/all/
    # ll {aufs,firewire,gspcav1,uvcvideo,qla2xxx,atl2,airprime}*.src.rpm

    -rw-r--r-- 1 beyerle ait    7566 Apr 24 22:39 airprime-2.6.18-1.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait   56446 Apr  8 11:43 atl2-1.0.40.2-1.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait   77012 Apr  8 21:03 atl2-1.0.40.4-4.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait  136658 Oct  1  2007 aufs-0.20070210.cvs-4.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait  427554 Jul 16 10:48 aufs-0.20080605.cvs-5.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait  295158 Aug 21  2007 firewire-2.6.18-1.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait  295120 Jan 22  2008 firewire-2.6.18-2.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait  217470 May 15 10:36 gspcav1-01.00.20-1.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait  195078 Nov 12  2007 gspcav1-1.00.18-1.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait 2015550 Mar 10 22:04 qla2xxx-8.01.07.15-1.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait   91123 Dec  7  2007 uvcvideo-0.1.0-1.svn148.slp5.src.rpm


The following versions were taken as the latest: (This list can be
found in the file
`tux50-64:/scratch/gasser_m/rpm_topdir/SRPMS/20090624-psi-src-rpms.list`.)::

    airprime-2.6.18-1.slp5.src.rpm
    atl2-1.0.40.4-4.slp5.src.rpm
    #aufs-0.20080605.cvs-5.slp5.src.rpm     # kernel-module-aufs is in SL51update and SL54base now
    firewire-2.6.18-2.slp5.src.rpm
    gspcav1-01.00.20-1.slp5.src.rpm
    qla2xxx-8.01.07.15-1.slp5.src.rpm
    uvcvideo-0.1.0-1.svn148.slp5.src.rpm


For convenience copy them first to your $rpm_topdir/SRPMS/ and then
unpack them, if not yet done (being still gasser_m@tux50-64)::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/psi/all
    # for i in `cat /scratch/gasser_m/rpm_topdir/SRPMS/20090624-psi-src-rpms.list` 
    # do
    #       cp $i /scratch/gasser_m/rpm_topdir/SRPMS/
    # done


Then install them into your build environment::

    # cd /scratch/gasser_m/rpm_topdir/SRPMS/
    # rpm -ivh `cat 20090624-psi-src-rpms.list`

    1:airprime           ########################################### [100%]
    2:atl2           ########################################### [100%]
    3:aufs           ########################################### [100%]
    4:firewire           ########################################### [100%]
    5:gspcav1        ########################################### [100%]
    6:qla2xxx        ########################################### [100%]
    7:uvcvideo           ########################################### [100%]


Before continuing with building the kernel modules check out the notes
below referring to some of them.


Build SL5 PSI Kernel-module RPMS On The Build System
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Now build the 64-bit modules::

    # [gasser_m@tux50-64]
    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/
    # ./build_kernel_modules  2.6.18-164.11.1.el5 > \
    #       UPDATE_LOG/20100125-build_kernel_modules-2.6.18-164.11.1.el5.x86_64.log 2>&1


The following RPMS were built::

    # cat ~/tmp/20090624-build_kernel_modules-2.6.18-128.1.14.el5.x86_64.output

    ...
    ---------------------------------------------------------------
    List of built kernel-modules for kernel 2.6.18-128.1.14.el5:
    ---------------------------------------------------------------

    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-airprime-2.6.18-128.1.14.el5-2.6.18-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-airprime-2.6.18-128.1.14.el5xen-2.6.18-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-atl2-2.6.18-128.1.14.el5-1.0.40.4-4.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-atl2-2.6.18-128.1.14.el5xen-1.0.40.4-4.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-aufs-2.6.18-128.1.14.el5-0.20080605.cvs-5.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-aufs-2.6.18-128.1.14.el5xen-0.20080605.cvs-5.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-firewire-2.6.18-128.1.14.el5-2.6.18-2.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-firewire-2.6.18-128.1.14.el5xen-2.6.18-2.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-gspcav1-2.6.18-128.1.14.el5-01.00.20-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-gspcav1-2.6.18-128.1.14.el5xen-01.00.20-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-qla2xxx-2.6.18-128.1.14.el5-8.01.07.15-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-qla2xxx-2.6.18-128.1.14.el5xen-8.01.07.15-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-128.1.14.el5-0.2.0-1.svn255.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-128.1.14.el5xen-0.2.0-1.svn255.slp5.x86_64.rpm
    /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-gpfs-2.6.18-128.1.14.el5-3.2.1-12.3.slp5.psi.x86_64.rpm
    /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-openafs-2.6.18-128.1.14.el5-1.4.10-94.slp5.x86_64.rpm
    /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-openafs-2.6.18-128.1.14.el5xen-1.4.10-94.slp5.x86_64.rpm
    /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-VMwareTools-2.6.18-128.1.14.el5-6530-29999.slp5.x86_64.rpm

    List of files saved in /tmp/rpms_for_2.6.18-128.1.14.el5 !


Note to Kernel Module Openafs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The openafs related build scripts are provided by Achim Gsell.

If the kernel version is increased, the kernel-module-openafs has to
be built.

If the Openafs version is increased, the kernel-module-openafs **AND**
the additional openafs RPMS (see list below) have to be built::

    openafs
    openafs-client
    openafs-kpasswd
    openafs-krb5


One can use the following standalone script to built openafs modules.

- `build-openafs.sl51.i386.sh`

  Standalone script for building and copying 32 bit openafs modules

- `build-openafs.sl51.x86_64.sh`

  Standalone script for building and copying 64 bit openafs modules


Note to Kernel Module Uvcvideo
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If building fails, download the latest uvcvideo version and compile
again::

    # cd ~/tmp/
    # svn checkout http://svn.berlios.de/svnroot/repos/linux-uvc/linux-uvc/trunk    # see also in specfile
    # mv trunk uvcvideo-svn255
    # tar cfzv uvcvideo-svn255.tar.gz uvcvideo-svn255/
    # scp uvcvideo-svn255.tar.gz tux50-64:/scratch/gasser_m/rpm_topdir/SOURCES/
    # ssh tux50-64
    # cd /scratch/gasser_m/rpm_topdir/SPECS/
    # vi uvcvideo.spec      # Version:    0.2.0
                # %define date    svn255 
    # rpmbuild -ba --target x86_64  --define "kernel 2.6.18-92.1.13.el5" --define 'buildall 1' uvcvideo.spec

    ...
    Wrote: /scratch/gasser_m/rpm_topdir/RPMS/x86_64/uvcvideo-0.2.0-1.svn255.slp5.src.rpm
    Wrote: /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-92.1.13.el5-0.2.0-1.svn255.slp5.x86_64.rpm 
    ...

    # echo "/scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-92.1.13.el5-0.2.0-1.svn255.slp5.x86_64.rpm" >> \
    # /tmp/rpms_for_2.6.18-92.1.13.el5


Don't forget to copy all the new modules to
`/afs/psi.ch/software/linux/dist/scientific/51/psi/all/`.


Check The Kernel Module Build Logs
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Check the log files for errors and verify that all required modules
were built::

    # grep -i error ~/tmp/20090624-build_kernel_modules-2.6.18-128.1.14.el5.x86_64.output
    ...
    checking your OS... configure: error: No usable linux headers found at /lib/modules/2.6.18-128.1.14.el5PAE/build
    + exit 1
    error: Bad exit status from /scratch/gasser_m/rpm_tmp/rpm-tmp.69701 (%build)
        Bad exit status from /scratch/gasser_m/rpm_tmp/rpm-tmp.69701 (%build)
    ...


This error is ok, as there is no x86_64 PAE kernel at all::

    # check-set-of-built-kernel-modules.bash /tmp/rpms_for_2.6.18-128.1.14.el5 \
    #       KERNEL_MODULES_TO_BUILD/sl5.i386.kms-to-be-built
    # Not all kernel module RPMS were built!
    # Check RPM(s) for: 'VMwareTools'

    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-airprime-2.6.18-128.1.14.el5-2.6.18-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-airprime-2.6.18-128.1.14.el5xen-2.6.18-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-atl2-2.6.18-128.1.14.el5-1.0.40.4-4.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-atl2-2.6.18-128.1.14.el5xen-1.0.40.4-4.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-aufs-2.6.18-128.1.14.el5-0.20080605.cvs-5.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-aufs-2.6.18-128.1.14.el5xen-0.20080605.cvs-5.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-firewire-2.6.18-128.1.14.el5-2.6.18-2.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-firewire-2.6.18-128.1.14.el5xen-2.6.18-2.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-gspcav1-2.6.18-128.1.14.el5-01.00.20-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-gspcav1-2.6.18-128.1.14.el5xen-01.00.20-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-qla2xxx-2.6.18-128.1.14.el5-8.01.07.15-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-qla2xxx-2.6.18-128.1.14.el5xen-8.01.07.15-1.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-128.1.14.el5-0.2.0-1.svn255.slp5.x86_64.rpm
    /scratch/gasser_m/rpm_topdir/RPMS/x86_64/kernel-module-uvcvideo-2.6.18-128.1.14.el5xen-0.2.0-1.svn255.slp5.x86_64.rpm
    /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-gpfs-2.6.18-128.1.14.el5-3.2.1-12.3.slp5.psi.x86_64.rpm
    /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-VMwareTools-2.6.18-128.1.14.el5-6530-29999.slp5.x86_64.rpm
    /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-openafs-2.6.18-128.1.14.el5-1.4.10-94.slp5.x86_64.rpm
    /afs/psi.ch/project/linux/dist/slp5/RPMS/x86_64/kernel-module-openafs-2.6.18-128.1.14.el5xen-1.4.10-94.slp5.x86_64.rpm


This also seems to be ok, as there was never a VMWare kernel module in
the repository for Xen and PAE kernel.


Copy The Kernel Module RPMS To The SL 5 Repository
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Eventually, copy the built kernel module RPMS to the corresponding
PSI-ALL repository::

    # cp -av `cat /tmp/rpms_for_2.6.18-128.1.14.el5` /afs/psi.ch/software/linux/dist/scientific/51/psi/all/


Check whether the gpfs daemon and the gpfs kernel module RPMS were
copied, too, and have corresponding versions.

Probably you have to build them first using
e.g. `build-gpfs.sl54.x86_64.sh` on tux50-64::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/
    # ./build-gpfs.sl5x.x86_64.sh
    # check-for-gpfs-occurence.bash 2.6.18-164.9.1.el5  3.2.1-16  4  51

    # Begin checking for file occurence...
    #/afs/psi.ch/software/linux/dist/scientific/51/nonfree/all/gpfs-3.2.1-16.4.slp5.psi.x86_64.rpm
    # ok
    #/afs/psi.ch/software/linux/dist/scientific/51/psi/all/kernel-module-gpfs-2.6.18-164.9.1.el5-3.2.1-16.4.slp5.psi.x86_64.rpm
    # ok
    #/afs/psi.ch/software/linux/dist/scientific/51/nonfree/all/gpfs-init-script-1.0-4.slp5.psi.x86_64.rpm
    # ok
    # End checking for file occurence.


Make The New Kernel Repository For SL5
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

The next two commands, `make kernel_all` and `make kernel`, will
process both architectures, i386 and x86_64.  So you can run it only
once, after having built i386 and x86_64 kernel modules.

Running `make kernel_all` will copy the built kernel module RPMS and
other kernel related RPMS from the corresponding ...-ALL to the
KERNEL-ALL directories, e.g.::

    from:   /afs/psi.ch/software/linux/dist/scientific/51/update.x86_64/all/
    to:     /afs/psi.ch/software/linux/dist/scientific/51/kernel/all/       


Eventually `make kernel` will create the new kernel snapshot in
`/afs/psi.ch/software/linux/dist/scientific/51/kernel/`::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/
    # make kernel_all > UPDATE_LOG/$(date +%Y-%m-%d)-make-kernel_all-2.6.18-164.11.1.el5.sl51.i386.x86_64.log 2>&1

    # KERNEL_VER=2.6.18 KERNEL_REL=371.12.1.el5 make kernel


Follow the instructions at the end of the output of `make kernel` to
update symlinks, e.g::

    # rm /afs/psi.ch/software/linux/dist/scientific/51/kernel/testing
    # ln -s 2.6.18-371.12.1.el5 /afs/psi.ch/software/linux/dist/scientific/51/kernel/testing
    

Create Snapshots of Repositories
--------------------------------

Both architectures, i386 and x86_64, are treated by the following
commands.

If new RPMS appear in a certain repository, which should be available
for the next update, a new snapshot is created for this repository.
The folder name of the snapshot will be equal to the current date.

Create all snapshots at once (gfa excluded as Heiner Billich and Rene
Kapeller will create this snapshot). This can take up to one hour::

    #[gasser_m@tux50]
    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts
    # nohup ./run_make_snapshots_all.sh &


As an alternative you could make a single snapshot of a certain
repository only.  E.g. to make a snapshot of the `update` and `psi`
repos type::

    # DIR=update make snapshot      # DIR=update will set: ALL_DIRS="update.i386 update.x86_64"
    # DIR=psi make snapshot

`make snapshot` will execute
`/afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_snapshot`.
The command `createrepo` invoked in the shell script `.../mk_snapshot`
writes repodata for each snapshot to
e.g. `/afs/psi.ch/software/linux/dist/scientific/51/others/20090625/repodata/`.

Release the snapshots for first level testing systems
-----------------------------------------------------

The command `make release_testing` will update the respective symlinks
`.../testing` to the latest snapshots in all repos (again, except
`gfa`).

Note: Because `make release_testing` is interactive, you should not
redirect the output to a file::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/
    # make release_testing

Before `make release_testing`:

Note: The output below originates not from the same update as above,
this was 2008-08-27::

    # [gasser_m@tux50]
    # cd /afs/psi.ch/software/linux/dist/scientific/51/
    # \ls -l */testing

    lrwxr-xr-x 1 markushin ait  8 Aug  8 15:27 cluster/testing -> 20080808
    lrwxr-xr-x 1 markushin ait  8 Aug  8 15:27 enhanced/testing -> 20080808
    lrwxr-xr-x 1 billich   ait  8 Aug 27 14:40 gfa/testing -> 20080827
    lrwxr-xr-x 1 markushin ait  8 Aug  8 15:27 kde-redhat/testing -> 20080808
    lrwxr-xr-x 1 markushin ait 18 Aug  8 15:27 kernel/testing -> 2.6.18-92.1.10.el5
    lrwxr-xr-x 1 markushin ait  8 Aug  8 15:27 nonfree/testing -> 20080808
    lrwxr-xr-x 1 markushin ait  8 Aug  8 15:27 others/testing -> 20080808
    lrwxr-xr-x 1 markushin ait  8 Aug 14 15:23 psi/testing -> 20080814
    lrwxr-xr-x 1 markushin ait  8 Aug  8 15:27 update.i386/testing -> 20080808
    lrwxr-xr-x 1 markushin ait  8 Aug  8 15:27 update.x86_64/testing -> 20080808

    [gasser_m@tux50 51]$ \ls -l */current
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 cluster/current -> 20080808
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 enhanced/current -> 20080808
    lrwxr-xr-x 1 billich  ait  8 Aug 27 14:40 gfa/current -> 20080827
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 kde-redhat/current -> 20080808
    lrwxr-xr-x 1 gasser_m ait 18 Aug 14 16:37 kernel/current -> 2.6.18-92.1.10.el5
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 nonfree/current -> 20080808
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 others/current -> 20080808
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 psi/current -> 20080814
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 update.i386/current -> 20080808
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 update.x86_64/current -> 20080808


After `make release_testing`::

    # [gasser_m@tux50]
    # \ls -l */testing

    lrwxr-xr-x 1 gasser_m ait  8 Aug 28 14:58 cluster/testing -> 20080827
    lrwxr-xr-x 1 gasser_m ait  8 Aug 28 14:58 enhanced/testing -> 20080827
    lrwxr-xr-x 1 billich  ait  8 Aug 27 14:40 gfa/testing -> 20080827
    lrwxr-xr-x 1 gasser_m ait  8 Aug 28 14:58 kde-redhat/testing -> 20080827
    lrwxr-xr-x 1 gasser_m ait 18 Aug 28 14:58 kernel/testing -> 2.6.18-92.1.10.el5
    lrwxr-xr-x 1 gasser_m ait  8 Aug 28 14:58 nonfree/testing -> 20080827
    lrwxr-xr-x 1 gasser_m ait  8 Aug 28 14:58 others/testing -> 20080827
    lrwxr-xr-x 1 gasser_m ait  8 Aug 28 14:58 psi/testing -> 20080827
    lrwxr-xr-x 1 gasser_m ait  8 Aug 28 14:58 update.i386/testing -> 20080827
    lrwxr-xr-x 1 gasser_m ait  8 Aug 28 14:58 update.x86_64/testing -> 20080827

    [gasser_m@tux50 51]$ \ls -l */current
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 cluster/current -> 20080808
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 enhanced/current -> 20080808
    lrwxr-xr-x 1 billich  ait  8 Aug 27 14:40 gfa/current -> 20080827
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 kde-redhat/current -> 20080808
    lrwxr-xr-x 1 gasser_m ait 18 Aug 14 16:37 kernel/current -> 2.6.18-92.1.10.el5
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 nonfree/current -> 20080808
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 others/current -> 20080808
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 psi/current -> 20080814
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 update.i386/current -> 20080808
    lrwxr-xr-x 1 gasser_m ait  8 Aug 14 16:37 update.x86_64/current -> 20080808


Now, the `testing` symlinks point to the latest snapshots, while the
`current` symlinks still point to the previous snapshots.


Make First Tests
----------------

Manually update selected `testing` hosts (see `/etc/sysconfig/psi` and
`/etc/yum.conf`) with yum. The `testing` string is set in
`/etc/sysconfig/psi` and `/etc/yum.conf` (i.e. your computer is
looking for new RPMS in the `testing` snapshots)::

    # yum clean all
    # yum update

If there's a new kernel, reboot into this new kernel and especially
check whether the new kernel-module-openafs is working and AFS was
mounted.

- Note to the `psi-update` script

`psi-update` will not work at this point, because it's udpating only
if the PSI version has changed, what will be done later (see "Create A
New PSI Version Of SL51" below).  `psi-update` will get the script
`yum_update` from our master, pxeserv01, see also `/etc/fstab`.  The
script `yum_update` will be saved locally as
`/usr/sbin/psi-yum_update`.  Afterwards `psi-yum_update` will be
executed locally.


Create A New PSI Version Of SL51 And Release It For Testing Systems
-------------------------------------------------------------------

In this section a new PSI version is created and released for testing
systems.  By default, Green Testing PCs will be auto updated during
the following night.

The procedure shown here has to be repeated for all SL5 class/subclass
combinations.  Contemporary there are three of them:

1. Desktop/Stable
2. Desktop/Enhanced
3. Cnode/Stable

To get an overview list the classes directories::

    # [gasser_m@pc7377 Stable]$ ll -rt /afs/psi.ch/software/linux/dist/scientific/51/classes/*

    /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop:
    total 10
    drwxr-xr-x  2 beyerle ait 2048 Jan 19  2008 StableTesting
    drwxr-xr-x  2 beyerle ait 2048 Jan 19  2008 EnhancedTesting
    drwxr-xr-x 15 beyerle ait 4096 Jul  2 17:11 Enhanced
    drwxr-xr-x 15 beyerle ait 2048 Aug 11 14:19 Stable

    /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode:
    total 4
    drwxr-xr-x  2 beyerle ait 2048 Jan 19  2008 StableTesting
    drwxr-xr-x 15 beyerle ait 2048 Jul  2 17:11 Stable


Note: To change the list of RPMS of a certain class edit the
corresponding package list.

E.g.: To add RPMS in update version 19 of SL51 Desktop/Stable edit
`list.Stable.v19`::

    # vi /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/list.Stable.v19

IMPORTANT: If you add RPMS to the list, update also the RPM lists used
in the kickstart installation::

    # vi /afs/psi.ch/software/linux/dist/scientific/51/kickstart/custom/DesktopStablei386/yum_install.conf
    # vi /afs/psi.ch/software/linux/dist/scientific/51/kickstart/custom/DesktopStablex86_64/yum_install.conf

- Note: The following tasks can also be performed using the script
  `create-new-psi-version.bash`.


Desktop/Stable: Release Desktop Stable Testing
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Create folder for Version 19 by running make class::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/
    # cp list.Stable.v18 list.Stable.v19

    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/
    # CLASS=Desktop SUBCLASS=Stable version=19 LIST=list.Stable.v19 make class

    ###{ begin output make class
    Populating Class directory ...
    TOP_DIR=/afs/psi.ch/software/linux/dist/scientific/51 \
        /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class

    Configuration file /afs/psi.ch/software/linux/dist/scientific/51/scripts/dist-config
    Running /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class ...
    create version and release file
    copy list.Stable.v19 to list
    -------------------------------------------------------------------------------
     The directory /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMS-20080811 has been populated.

     RPMSv19 links to RPMS-20080811

     Please check 
       cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMS-20080811/version
       cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMS-20080811/release
       cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMS-20080811/list

     If you like to display a message for the user (one line) put it into
       emacs /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMS-20080811/message

     If you want to release `testing`, link RPMSTesting to RPMSv19:
       rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMSTesting
       ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMSTesting

    -------------------------------------------------------------------------------
    ... End of /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class
    ###} end output make class


Now release it::

    # rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMSTesting
    # ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Stable/RPMSTesting


Desktop/Enhanced: Release Desktop Enhanced Testing (also a lot of nonfree etc. stuff)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Create folder for Version 19 by running `make class`::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/
    # cp list.Enhanced.v18 list.Enhanced.v19
    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/
    # CLASS=Desktop SUBCLASS=Enhanced version=19 LIST=list.Enhanced.v19 make class

    ###{ begin output
    Populating Class directory ...
    TOP_DIR=/afs/psi.ch/software/linux/dist/scientific/51 \
        /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class

    Configuration file /afs/psi.ch/software/linux/dist/scientific/51/scripts/dist-config
    Running /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class ...
    create version and release file
    copy list.Enhanced.v19 to list
    -------------------------------------------------------------------------------
     The directory /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMS-20080811 has been populated.

     RPMSv19 links to RPMS-20080811

     Please check 
       cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMS-20080811/version
       cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMS-20080811/release
       cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMS-20080811/list

     If you like to display a message for the user (one line) put it into
       emacs /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMS-20080811/message

     If you want to release "testing", link RPMSTesting to RPMSv19:
       rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMSTesting
       ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMSTesting

    -------------------------------------------------------------------------------
    ... End of /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class
    ###} end output


Now release it::
    
    # rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMSTesting
    # ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Desktop/Enhanced/RPMSTesting


Cnode/Stable: Release Cnode Stable Testing
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Create folder for Version 19 by running `make class`::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/
    # cp list.Stable.v18 list.Stable.v19
    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/
    # CLASS=Cnode SUBCLASS=Stable version=19 LIST=list.Stable.v19 make class

    ###{ begin output
    Populating Class directory ...
    TOP_DIR=/afs/psi.ch/software/linux/dist/scientific/51 \
        /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class

    Configuration file /afs/psi.ch/software/linux/dist/scientific/51/scripts/dist-config
    Running /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class ...
    create version and release file
    copy list.Stable.v19 to list
    -------------------------------------------------------------------------------
     The directory /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMS-20080811 has been populated.

     RPMSv19 links to RPMS-20080811

     Please check 
       cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMS-20080811/version
       cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMS-20080811/release
       cat /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMS-20080811/list

     If you like to display a message for the user (one line) put it into
       emacs /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMS-20080811/message

     If you want to release "testing", link RPMSTesting to RPMSv19:
       rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMSTesting
       ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMSTesting

    -------------------------------------------------------------------------------
    ... End of /afs/psi.ch/software/linux/dist/scientific/51/scripts/mk_class
    ###} end output 


Now release it::

    # rm -f /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMSTesting
    # ln -s RPMSv19 /afs/psi.ch/software/linux/dist/scientific/51/classes/Cnode/Stable/RPMSTesting


Test the Update On Testing Systems
----------------------------------

On a testing system run::

    [root@tux50]
    # psi-update


Release The Update PSI Wide For Productive Systems
--------------------------------------------------

Run::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts/
    # make current_equal_testing


Refresh symbolic links in the `RPMS_all` folder to all respective
RPMS::

    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts
    # make rpms_all


Test The Update On Productive Systems
-------------------------------------

On a productive (=current) system run::

    # psi-update


Update The Webpages
===================

Reference
---------

https://wiki.intranet.psi.ch/AIT/LinuxRegularUpdate#Update_the_Webpages

*Possibly it's better to move the reference to the top Reference
section.  This section is still in a very sketch state anyway.*


General Info to Webpages Providing SL Update Infos
--------------------------------------------------

The website is public to PSI.#??? or to everybody?

The site provides information about the SL update progress for users
at PSI.

Location of the website:

    On AFS: /afs/psi.ch/project/ait/www/services/linux/updates/
    On WWW: http://ait.web.psi.ch/services/linux/updates/
    

Update the following Webpages:

- http://ait.web.psi.ch/services/linux/updates/

- http://ait.web.psi.ch/services/linux/updates/update_sl5.html

- http://ait.web.psi.ch/services/linux/updates/update_sl4.html

- http://ait.web.psi.ch/services/linux/updates/update_sl3.html 


- Objective

Show the updated RPMS between current and the release before. Reduce
amount of info to a reasonable minimum and represent clearly for
users.  Don't show RPMS, which are in the repository but will not be
installed because of a newer version of this RPM in the same snapshot.


Script location: /afs/psi.ch/project/ait/www/services/linux/updates/
    find_new_rpms_between 20080626 51 18 20080808 51 19 The command
    above will not yet work on SL5, because of directory layout
    changes between SL4 and SL5. For now we will take another
    procedure to prepare the HTML files.

The following command will show some first scratch of an alternative
way to get the list with updated RPMS::

    # cd /afs/psi.ch/software/linux/dist/scientific/51
    # diff <( ls -1 */testing/ | sort ) <( ls -1 */current/ | sort ) | grep '^<' | sed -e 's/.i.86.rpm//' -e 's/.x86_64.rpm//' | uniq
    
What do we need? In SL5 we have the following classes/subclasses::

    #

          ¦---Stable
          ¦-Cnode---¦
          ¦     ¦---StableTesting
          ¦
          ¦
    sl5---¦
          ¦     ¦--Stable
          ¦     ¦
          ¦     ¦--StableTesting
          ¦-Desktop-¦
            ¦--Enhanced
            ¦
            ¦--EnhancedTesting


    ls -1 */testing/ | grep -v testing | grep -v x86_64 | sort > ~/tmp/sl51-v19-ls-asterisk-testing-sort-out
    ls -1 */current/ | grep -v current | grep -v x86_64 | sort > ~/tmp/sl51-v19-ls-asterisk-current-sort-out
    /afs/psi.ch/user/g/gasser_m/tmp/
    mengen.py -D sl51-v19-ls-asterisk-testing-sort-out sl51-v19-ls-asterisk-current-sort-out > sl51-v19-mengen-out
    get-latest-RPM-from-RPM-list.pl sl51-v19-mengen-out > sl51-v19-updated-RPMS.list


From sl51-v19-updated-RPMS.list take the list to insert into the HTML
update site.  Edit file
`/afs/psi.ch/project/ait/www/services/linux/updates/update_sl5.html`.


Are all clients up-to-date?
---------------------------

Login to pxeserv01.

Note to puppet and `/etc/sysconfig/psi`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Even puppet and not cfengine is running on SL5, latest copies of
`ANYHOST:/etc/sysconfig/psi` from SL3, SL4 and SL5 systems are put to
`pxeserv01:/opt/cfengine/sysconfig/`::

    # cd /opt/cfengine/scripts
    # ./find_hosts_with_old_version.sh 98 71 20

    running...
    ./cfengine.lcmeg01:VERSION=70
    ./cfengine.lclth01:VERSION=70
    ./cfengine.lclev03:VERSION=70
    ./cfengine.pc7148:VERSION=18
    ./cfengine.lclev01:VERSION=70
    ./cfengine.lclth04:VERSION=70
    ./cfengine.pc6702:VERSION=69
    ./cfengine.lclth07:VERSION=70
    ./cfengine.pc3384:VERSION=97
    ./cfengine.slsnedi1:VERSION=17
    ./cfengine.pc5082:VERSION=91
    ./cfengine.lpc4358:VERSION=69
    ./cfengine.lclev05:VERSION=70
    ./cfengine.lclth08:VERSION=70
    ./cfengine.lclth02:VERSION=70
    ./cfengine.llc6:VERSION=69
    ./cfengine.lclrs11:VERSION=69
    ./cfengine.pc5835:VERSION=96
    ./cfengine.lclth06:VERSION=70
    ./cfengine.tux40e:VERSION=70
    ./cfengine.lclev02:VERSION=70
    ./cfengine.lclth03:VERSION=70
    ./cfengine.pc4426:VERSION=17
    ./cfengine.pcmic01:VERSION=70
    ./cfengine.lclth05:VERSION=70
    done!


If a host is not up-to-date, login to the host and run manually::

    # psi-update


You may also want to check the logs in `/var/log/update/`
