Repositories
============

A number of repositories are currently mirrored on AFS under
``/afs/psi.ch/software/mirror`` (``$MIRROR``), in particular the following
versions of Scientific Linux, both 32 and 64 bit: 5.7, 6.0, 6.4, and 6.x.

The repositories are accessible via HTTP at http://linux.web.psi.ch/.

They are stored in ``$MIRROR/scientific``, each subdirectory of which
is a mount point for a separate AFS volume.  This is done because
smaller AFS volumes are easier to handle (e.g. when moving them
between fileservers).

Each mirror, e.g. ``$MIRROR/scientific/60/epelp/`` contains a
subdirectory ``all``, which contains the packages that we actually
want in our YUM repositories.

Repository management as well as package building can be done on the
various ``tux*.psi.ch`` servers.

Further documentation on repository management can be found in
``/afs/psi.ch/project/linux/doc``.  This documentation will eventually
be included in this document.

The directory ``$MIRROR/scientific/scripts`` contains scripts for
various tasks related to managing the Scientific Linux repositories.
It is a working copy of a [git
repository](http://git.psi.ch/linux-dist/sl-scripts).


Tools
-----

There are several scripts that are used to maintain/update the various
repositories.


``sync_updates.sh``
~~~~~~~~~~~~~~~~~~~

This script copies new packages from the SL mirror to the PSI distribution
directories, and runs ``createrepo --update`` to update the repository metadata.

Information on the directories is read from a file ``dist-config`` in the
current working directory.


``update_symlinks``
~~~~~~~~~~~~~~~~~~~





Updates
-------

To release updates to the existing repositories, the steps below have to be
performed. The value of ``DIST_DIR`` is the base directory of the distribution
to be updated, i.e. ``/afs/psi.ch/software/linux/dist/scientific/xx/``, where
``xx`` is one of ``57``, ``60``, or ``64``.

1. Update add-on repositories
2. cd "${DIST_DIR}/scripts"
3. ./sync_updates.sh
4. ./update_symlinks.sh
5. ./create_snapshots.sh alldirs
6. ./release_unstable.sh
7. ./incr_version.sh
8. ./update_version_info.sh
9. test new unstable release
10. ./release.sh  testing
11. ./update_version-info.sh
12. test testing release
13. ./release.sh stable
14. ./update_version-info.sh

The sections below describe each step in detail. The value of ``PRJ_DIR`` is
``/afs/psi.ch/project/linux/``.


Kernel updates
~~~~~~~~~~~~~~

If there is a new kernel, some of the modules need to be rebuilt. For SL 5.7
there is a script in the ``scripts`` subdirectory taking care of this:
``build_kernel_modules.sh``.


Update add-on repositories
~~~~~~~~~~~~~~~~~~~~~~~~~~

GPFS
,,,,

    rsync --archive --verbose  ${PRJ_DIR}/dist/slp6/RPMS/*/kmod-gpfs* "${DIST_DIR}/psi/all"
    rsync --archive --verbose  ${PRJ_DIR}/dist/slp6/RPMS/*/gpfs* "${DIST_DIR}/nonfree/all"

Notes:

- Don't know why the kernel modules for GPFS are in 'psi' but the GPFS user-land
  software is in 'nonfree'.
- Hans-Christian is responsible for building RPMs


OpenAFS
,,,,,,,

Note: OpenAFS version on tux50 and tux50-64 is 1.4.x. On all other systems it is 1.6.x.

Build OpenAFS
+++++++++++++

On tux50, tux50-64, tux60-tux60-64, tux70.

Either::

    rsync --verbose  ${PRJ_DIR}/dist/slp6/RPMS/*/kernel-module-openafs-* "${DIST_DIR}/psi/all"

or: copy decicated versions

Notes: Achim is responsible for building OpenAFS RPMs.

Checklist
+++++++++

New version available? If yes, build binaries:

=========  =====  =========  ========
OS         New?   Compiled   Copied?
---------  -----  ---------  --------
tux50      no
tux50-64   no
tux60      yes    yes        yes
tux60-64   yes    yes        yes
tux70-64   yes    yes        yes
=========  =====  =========  ========

New kernel or version available? If yes, install kernel, build module:
=========  =====  =========  ========
OS         New?   Compiled   Copied?
---------  -----  ---------  --------
tux50      no     -          -
tux50-64   no     -          -
tux60      yes    yes        yes
tux60-64   yes    yes        yes
tux70-64   yes    yes        yes
=========  =====  =========  ========

ZFS
,,,

ZFS kernel modules are available for EL7 only!

Sync from master repository
+++++++++++++++++++++++++++

Run on tux70-64 as normal user::

    reposync --repoid=zfs --norepopath --download_path /afs/psi.ch/software/mirror/zfsonlinux/7/x86_64

Build kmod
++++++++++

Build on tux70-64 as root.

Boot into the right (newest) kernel! This is the kernel we are going to build
the modules for.  Run::

    # set some variables
    $ ZFS_VERS='x.y.z'
    $ ZFS_REL='r'
    $ ZFS_REPO_DIR='/afs/psi.ch/software/linux/dist/scientificlinux/7x/x86_64/zfs/'
    $ DIST='el7'
    $ ARCH='x86_64'

    $ cd /usr/src/spl-$ZFS_VERS
    $ ./configure
    $ make rpm-utils rpm-kmod

    # Install the spl packages, they are required to build zfs.
    $ yum localinstall \
    	kmod-spl-devel-$ZFS_VERS-$ZFS_REL.$DIST.$ARCH.rpm \
    	kmod-spl-devel-kernel-$ZFS_VERS-$ZFS_REL.$DIST.$ARCH.rpm \
    	kmod-spl-kernel-$ZFS_VERS-$ZFS_REL.$DIST.$ARCH.rpm \
    	spl-$ZFS_VERS-$ZFS_REL.$DIST.$ARCH.rpm

    $ cd ../zfs-x.y.z
    $ ./configure
    $ make rpm-utils rpm-kmod

    # you need an AFS-token to copy the files!
    $ klog.openafs USERNAME

    # for a new ZFS version do
    $ cp -v spl-$ZFS_VERS/*.x86_64.rpm "$ZFS_REPO_DIR"
    $ cp -v zfs-$ZFS_VERS/*.x86_64.rpm "$ZFS_REPO_DIR"

    # to copy kernel modules only, do
    $ cp -v spl-$ZFS_VERS/kmod-*-$(uname -r)*.x86_64.rpm "$ZFS_REPO_DIR"
    $ cp -v zfs-$ZFS_VERS/kmod-*-$(uname -r)*.x86_64.rpm "$ZFS_REPO_DIR"

    # update repo
    $ cd "$ZFS_REPO_DIR"
    $ createrepo --update .

See also: `<http://zfsonlinux.org/generic-rpm.html>`_

Checklist
+++++++++

If there is a new version: Are the RPM of the new versions of spl-x.y.z and
zfs-x.y.z in ``/afs/psi.ch/software/linux/dist/scientificlinux/7x/x86_64/zfs/``?

Are the kmod-RPM's for the newest kernel in
``/afs/psi.ch/software/linux/dist/scientificlinux/7x/x86_64/zfs/``?

Flash Player
,,,,,,,,,,,,

Download
++++++++

Current Flash-Player must be downloaded from `Adobe
<https://get.adobe.com/de/flashplayer/otherversions/>`_ and copied to
``${DIST_DIR}/nonfree/all``.

Checklist
+++++++++

Is the newest version installed in
``/afs/psi.ch/software/linux/dist/scientific/xx/nonfree/all/`` (``xx`` in
``57``, ``60``, ``64``)?


NVidia
,,,,,,

syslog-ng
,,,,,,,,,

openntpd
,,,,,,,,

