Linux How To - RPM - Update psi-desktop Package on SL5
======================================================

Introduction
------------

Contemporary there are many PSI related cron jobs which are executed
at the same time on all linux clients. As a consequence too many AFS
requests coming from these clients are sent simultaneously to the AFS
servers causing AFS performance problems. Thus, these cron jobs should
be spread over time. In order to have time to fix things if anything
goes wrong, the respective jobs are scheduled within a certain time
window instead of using the whole time range, e.g. the hourly executed
job window ranges from 20-40 minutes of an hour and not from 0-59
minutes.

At the time of writing only `/usr/sbin/psi-puppet` is configured to
run within such a window, the time is set randomly in
`/etc/cron.d/psi-cronjobs` during installation of the RPM psi-desktop
to which the file belongs to, 27 in the example below::

    #
    # /etc/cron.d/psi-cronjobs
    #
    # PSI related cronjobs
    #
    # Urs Beyerle, PSI
    #

    # Run puppet every hour at xx:27
    27 * * * * root /usr/sbin/psi-puppet >/dev/null 2>&1

    # Send info back to master daily at 10:00
    00 10 * * * root /usr/sbin/psi-sendinfo >/dev/null 2>&1

    # Run psi-auto-udpate daily at 00:30, 01:30, 02:30, 04:30
    30 00 * * * root /usr/sbin/psi-auto-update >> /var/log/update/psi-update.log 2>&1
    30 02 * * * root /usr/sbin/psi-auto-update >> /var/log/update/psi-update.log 2>&1
    30 03 * * * root /usr/sbin/psi-auto-update >> /var/log/update/psi-update.log 2>&1
    30 04 * * * root /usr/sbin/psi-auto-update >> /var/log/update/psi-update.log 2>&1


- /usr/sbin/psi-puppet:       Runs puppetd, but only if AUTO_UPDATE_CONFIG=yes
- /usr/sbin/psi-auto-update:      Runs the psi-update script if AUTO_UPDATE_RPMS=yes
- /usr/sbin/psi-update:       Runs the yum_update script
  (Will get the script yum_update from our master.
  The script yum_update will be saved as psi-yum_update.
  Afterwards psi-yum_update will be executed locally.)

Now the following jobs should be scheduled analogously.#???

- `/etc/cron.hourly/update_afs_users`

- `/etc/cron.hourly/update_environment_modules`

Maybe the time window for `/usr/sbin/psi-puppet` has to be expanded.

- Are there GFA related things which should be scheduled in a new manner?#???

Procedure Description
~~~~~~~~~~~~~~~~~~~~~

Go to the relevant build system (e.g. tux50).  Get the sources for
building the psi-desktop RPM, unpack them and apply your modifications
in the unpacked files. When finished tar and gzip again, build the new
RPM, and, eventually, test it on a test machine.


Procedure Step by Step
~~~~~~~~~~~~~~~~~~~~~~

Get The Source RPM
....................

Run::
  
    # [gasser_m@tux50] 
    # cd /scratch/gasser_m/rpm_topdir/SRPMS/
    # ll /scratch/redhat/SRPMS/
    total 11M
    -rw-r--r-- 1 beyerle ait 4.9M Jun 30 08:09 psi-desktop-1.3.3-16.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait 4.9M Jun 30 08:11 psi-desktop-1.3.3-17.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait 5.3K Jun 30 10:32 nxcleanup-0.3-1.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait 6.0K Jul  9 10:40 nxcleanup-0.4-1.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait 418K Jul 16 10:48 aufs-0.20080605.cvs-5.slp5.src.rpm
    -rw-r--r-- 1 beyerle ait 642K Jul 29 12:57 ntfs-3g-1.2712-4.slp5.src.rpm

    # cp /scratch/redhat/SRPMS/psi-desktop-1.3.3-17.slp5.src.rpm .
    # rpm -ivh psi-desktop-1.3.3-17.slp5.src.rpm

This will install the files
`/scratch/gasser_m/rpm_topdir/SOURCES/psi-desktop-1.3.3.tar.gz` and
`/scratch/gasser_m/rpm_topdir/SPECS/psi-desktop.spec`::

    # cd /scratch/gasser_m/rpm_topdir/SOURCES/
    # tar xfz psi-desktop-1.3.3.tar.gz

Apply Your Modifications
........................

Edit `/scratch/gasser_m/rpm_topdir/SPECS/psi-desktop.spec`::

    Modified Parts in "psi-desktop.spec"
    ------------------------------------
    Note: Some lines are cut at the end.

    NEW VERSION                        |  OLD VERSION
    ---------------------------------------------------------------------------------------------------------------------
    ...
    Release:        18%{?dist}             |  Release:        17%{?dist}
    ...
    Packager:           Marc Gasser <marc.gasser@psi.ch>   |  Packager:           Urs Beyerle <urs.beyerle@psi.ch>
    ...
    #### begin psi-cronjobs ############################       |  # randomly runs puppet in psi-cronjobs
    #                              |  # create a random number between 0-9
    # Randomly run commands in "/etc/cron.d/psi-cronjobs"      |  random=${RANDOM:1:1}
    #                              |  [ ! $random ] && random=0
    # /usr/sbin/psi-puppet: hourly at random1          |  sed -i "s/random/$random/" /etc/cron.d/psi-cronjobs
    random1=$[ ( $RANDOM % 20 ) + 21 ]      # create a random n|  
    [ ! $random1 ] && random1=33                   |  
    sed -i "s/random1/$random1/g" /etc/cron.d/psi-cronjobs     |  
                                   |  
    # /usr/sbin/psi-sendinfo: daily at 10:1random2         |  
    # /usr/sbin/update_environment_modules: daily at 06:0random|  
    # /usr/sbin/update_afs_users: hourly at XX:1random2    |  
    random2=$[ ( $RANDOM % 10 ) ]   # create a random number be|  
    [ ! $random2 ] && random2=3                |  
    sed -i "s/random2/$random2/g" /etc/cron.d/psi-cronjobs     |  
                                   |  
    #### end psi-cronjobs ############################     |  
    ...
    ---------------------------------------------------------------------------------------------------------------------


Edit `/etc/cron.d/psi-cronjobs`.  The modified `psi-cronjobs` which
will be packed into the new `psi-desktop` RPM::

    # cd /scratch/gasser_m/rpm_topdir/SOURCES/psi-desktop-1.3.3/slp5/etc/cron.d
    # vi psi-cronjobs

    #
    # /etc/cron.d/psi-cronjobs
    #
    # PSI related cronjobs
    # 
    # Marc Gasser, PSI
    #
    # Note: The time settings for these cron jobs are
    #       set in "psi-desktop.spec". They are 
    #       randomized within a specific time window.
    #       The randomization takes place when the RPM is
    #       installed, thus it is client specific.


    # Run psi-puppet every hour at XX:random1
    random1 * * * * root /usr/sbin/psi-puppet >/dev/null 2>&1

    # Send info back to master daily at 10:1random2
    1random2 10 * * * root /usr/sbin/psi-sendinfo >/dev/null 2>&1

    # Run psi-auto-udpate daily at 00:4random2, 01:4random2, 02:4random2, 04:4random2
    4random2 00 * * * root /usr/sbin/psi-auto-update >> /var/log/update/psi-update.log 2>&1
    4random2 02 * * * root /usr/sbin/psi-auto-update >> /var/log/update/psi-update.log 2>&1
    4random2 03 * * * root /usr/sbin/psi-auto-update >> /var/log/update/psi-update.log 2>&1
    4random2 04 * * * root /usr/sbin/psi-auto-update >> /var/log/update/psi-update.log 2>&1

    # Run update_environment_modules daily at 06:0random2, 13:0random2, 18:0random2
    0random2 06 * * * root /usr/sbin/update_environment_modules >/dev/null 2>&1
    0random2 13 * * * root /usr/sbin/update_environment_modules >/dev/null 2>&1
    0random2 18 * * * root /usr/sbin/update_environment_modules >/dev/null 2>&1

    # Run update_afs_users every hour at XX:1random2
    1random2 * * * * root /usr/sbin/update_afs_users >/dev/null 2>&1


Move commands which are controlled by `/etc/cron.d/psi-cronjobs` to
`/usr/sbin/` if not done yet::

    # cd /scratch/gasser_m/rpm_topdir/SOURCES/psi-desktop-1.3.3/slp5/
    # mv etc/cron.d/update_environment_modules usr/sbin/
    # mv etc/cron.d/update_afs_users usr/sbin/

Cronjob Schedule in `psi-cronjobs`
..................................

Applying this `psi-desktop.spec` to the file `psi-cronjobs` will
randomly distribute the respective cronjobs whithin the time windows
given below. The randomization takes place when the RPM is installed
or updated on the client.

- `/usr/sbin/psi-puppet`: hourly from HH:21 to HH:40.

- `/usr/sbin/psi-sendinfo`: daily from 10:10 to 10:19.

- `/usr/sbin/update_environment_modules`: daily from 06:00 to 06:09,
  from 13:00 to 13:09 and from 18:00 to 18:09.

- `/usr/sbin/update_afs_users`: hourly from HH:10 to HH:19.


Build the new `psi-desktop` RPM
...............................

Build both, the new RPM and the new SRPM applying the spec file::

    # cd /scratch/gasser_m/rpm_topdir/SOURCES/
    # tar cfz psi-desktop-1.3.3.tar.gz psi-desktop-1.3.3
    # rm -rf psi-desktop-1.3.3/
    # cd /scratch/gasser_m/rpm_topdir/SPECS/
    # rpmbuild -ba psi-desktop.spec

Test the New `psi-desktop` RPM
..............................

The file
`tux50:/scratch/gasser_m/rpm_topdir/RPMS/noarch/psi-desktop-1.3.3-18.slp5.noarch.rpm`
was copied to `pc7377:/tmp/` and updated using yum::

    # [root@pc7377 etc]
    # yum update /tmp/psi-desktop-1.3.3-18.slp5.noarch.rpm


Add the New `psi-desktop` RPM to Psi-All Repository
...................................................

Copy the new RPM to the respective SL5 repository.  Update the
`repodata` and the symbolic links in `.../RPMSall/...` if necessary::
    
    # [gasser_m@tux50 ~]
    # cd /scratch/gasser_m/rpm_topdir/RPMS/noarch/
    # cp psi-desktop-1.3.3-18.slp5.noarch.rpm /afs/psi.ch/software/linux/dist/scientific/51/psi/all/

    # cd /afs/psi.ch/software/linux/dist/scientific/51/psi/all/
    # createrepo .
    # cd /afs/psi.ch/software/linux/dist/scientific/51/scripts
    # make rpms_all
