forked from Controls/gitea-pages
1071 lines
33 KiB
ReStructuredText
1071 lines
33 KiB
ReStructuredText
Project `psi-puppet1`
|
|
=====================
|
|
|
|
Introduction
|
|
------------
|
|
|
|
This document describes the relaunch of the puppet service
|
|
infrastructure at PSI.
|
|
|
|
The whole project can be divided into two parts:
|
|
|
|
- The hard- and software setup for the puppet server and client.
|
|
(Described in this document.)
|
|
|
|
- Implementation of the configuration for puppet clients. (Described
|
|
in [[PuppetManifestsForSL53][Puppet Manifests For SL53]].)
|
|
|
|
|
|
Objectives
|
|
----------
|
|
|
|
- To get a stable, scalable and easy to manage puppet service
|
|
infrastructure.
|
|
|
|
- To gain a better overview of the various client configurations
|
|
configured by puppet.
|
|
|
|
- To keep a clear and up-to-date documentation.
|
|
|
|
- To keep the different configurations of the different SL releases
|
|
separated from each other, e.g. SL 5.1 does not overlap with SL 5.3.
|
|
|
|
- Also other users from AIT and GFA than the puppet administrator
|
|
should have the possibility to use puppet to configure their hosts.
|
|
|
|
- The different client configurations of the different puppet users
|
|
must not interfere with each other.
|
|
|
|
- To manage the changes to manifests and client configuration files.
|
|
|
|
- Easy recovery of files in case of data loss.
|
|
|
|
- Easy and fast reinstallation of an identical puppet server in case
|
|
of an irreparable server crash.
|
|
|
|
|
|
Description of the Basic Server Setup
|
|
-------------------------------------
|
|
|
|
- Hardware
|
|
|
|
Dell Power Edge 1750
|
|
|
|
- Operating System
|
|
|
|
SL51 32 bit Server
|
|
|
|
- Network configuration
|
|
|
|
Static IP for Production Server: 129.129.190.174/24
|
|
Hostname: psi-puppet1.psi.ch
|
|
|
|
- Required RPMS
|
|
|
|
- puppet-server (http://download.fedora.redhat.com/pub/epel/5/i386/repoview/)
|
|
- augeas-libs
|
|
- facter
|
|
- puppet (http://download.fedora.redhat.com/pub/epel/5/i386/repoview/)
|
|
- ruby
|
|
- ruby-augeas
|
|
- ruby-libs
|
|
- ruby-shadow
|
|
- ruby-irb (required for reading help)
|
|
- ruby-rdoc (required for reading help)
|
|
|
|
|
|
Procedure
|
|
---------
|
|
|
|
Create a Repository for Puppet Related RPMS
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
First make the directory in the SL51 installation tree::
|
|
|
|
# mkdir /afs/psi.ch/software/linux/dist/scientific/51/puppet-0247
|
|
|
|
Add the following RPMS to this repository and run `createrepo`::
|
|
|
|
puppet-server-0.24.7-4.el5.noarch.rpm
|
|
augeas-libs-0.3.5-1.el5.i386.rpm
|
|
facter-1.5.2-2.el5.noarch.rpm
|
|
puppet-0.24.7-4.el5.noarch.rpm
|
|
ruby-augeas-0.2.0-1.el5.i386.rpm
|
|
ruby-shadow-1.4.1-7.el5.i386.rpm
|
|
|
|
# cd /afs/psi.ch/software/linux/dist/scientific/51/puppet-0247
|
|
# createrepo .
|
|
|
|
To enable the access to this repo create the yum repo file
|
|
`/etc/yum.repos.d/puppet-0247.repo` on the puppet server::
|
|
|
|
[puppet-0247]
|
|
name=puppet-0247 for SL5
|
|
baseurl=http://linux.web.psi.ch/dist/scientific/5/puppet-0247/
|
|
enabled=1
|
|
|
|
|
|
Setup The Puppet Server
|
|
~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Basic Server Installation
|
|
.........................
|
|
|
|
Install SL51, class Server via PXE boot and kickstart.
|
|
|
|
|
|
Puppet-Server Installation
|
|
..........................
|
|
|
|
Install `puppet-server` with yum. This will also draw the required
|
|
dependencies::
|
|
|
|
# [root@psi-puppet1]
|
|
# yum install puppet-server
|
|
|
|
...
|
|
Finished Kernel Module Plugin
|
|
|
|
Dependencies Resolved
|
|
|
|
=============================================================================
|
|
Package Arch Version Repository Size
|
|
=============================================================================
|
|
Installing:
|
|
puppet-server noarch 0.24.7-4.el5 puppet-0247 25 k
|
|
Installing for dependencies:
|
|
augeas-libs i386 0.3.5-1.el5 puppet-0247 151 k
|
|
facter noarch 1.5.2-2.el5 puppet-0247 41 k
|
|
puppet noarch 0.24.7-4.el5 puppet-0247 548 k
|
|
ruby i386 1.8.5-5.el5_2.6 sl5update 279 k
|
|
ruby-augeas i386 0.2.0-1.el5 puppet-0247 17 k
|
|
ruby-libs i386 1.8.5-5.el5_2.6 sl5update 1.6 M
|
|
ruby-shadow i386 1.4.1-7.el5 puppet-0247 9.5 k
|
|
|
|
Transaction Summary
|
|
=============================================================================
|
|
Install 8 Package(s)
|
|
Update 0 Package(s)
|
|
Remove 0 Package(s)
|
|
...
|
|
|
|
|
|
Configure The Puppet Server
|
|
...........................
|
|
|
|
The configuration files of the puppet server, directory
|
|
`/etc/puppet/`, are stored locally.
|
|
|
|
The puppet client configuration files are stored on AFS. The mountpoint on
|
|
psi-puppet1 is `/var/puppet/environments`, thus
|
|
create the directory `/var/puppet/environments`.
|
|
|
|
# mkdir -p /var/puppet/environments
|
|
|
|
For how to mount the AFS see section `Mount AFS Volumes` below.
|
|
|
|
The client configuration files in /var/puppet/environments are
|
|
described at [[PuppetManifestsForSL53][Puppet Manifests For SL53]].
|
|
|
|
The log is on the local disk in `/var/log/puppet`.
|
|
To set the logfile edit the line `PUPPETMASTER_OPTS`
|
|
in `/etc/rc.d/init.d/puppetmaster`. For testing also the
|
|
debug option `-d` is enabled::
|
|
|
|
PUPPETMASTER_OPTS="-v -d -l /var/log/puppet/puppetmaster.log"
|
|
|
|
Config file `puppet.conf`::
|
|
|
|
###########################################################################
|
|
# $Header: /etc/puppet/RCS/puppet.conf,v 1.3 2009/09/07 18:11:17 root Exp root $
|
|
#
|
|
# Puppetmaster Environments
|
|
# =========================
|
|
#
|
|
# Ref.: http://reductivelabs.com/trac/puppet/wiki/UsingMultipleEnvironments
|
|
#
|
|
# Marc Gasser, PSI
|
|
# last modified 2011-11-18
|
|
#
|
|
############################################################################
|
|
[main]
|
|
# Where Puppet stores dynamic and growing data.
|
|
# The default value is '/var/puppet'.
|
|
vardir = /var/puppet
|
|
|
|
# The Puppet log directory.
|
|
# The default value is '$vardir/log'.
|
|
# logdir = /afs/psi.ch/service/linux/puppet/var/log
|
|
logdir = /var/log/puppet
|
|
|
|
# Where Puppet PID files are kept.
|
|
# The default value is '$vardir/run'.
|
|
rundir = /var/run/puppet
|
|
|
|
# Where SSL certificates are kept.
|
|
# The default value is '$confdir/ssl'.
|
|
ssldir = $vardir/ssl
|
|
|
|
# Whether log files should always flush to disk.
|
|
# The default value is false
|
|
autoflush = true
|
|
|
|
|
|
|
|
[puppetmasterd]
|
|
reports = store
|
|
#reports = store , tagmail, rrdgraph
|
|
|
|
# tagmap = $confdir/tagmail.conf
|
|
|
|
#rrddir = $vardir/rrd
|
|
#rrdinterval = $runinterval
|
|
#rrdgraph = true
|
|
|
|
[puppetd]
|
|
# The file in which puppetd stores a list of the classes
|
|
# associated with the retrieved configuratiion. Can be loaded in
|
|
# the separate ``puppet`` executable using the ``--loadclasses``
|
|
# option.
|
|
# The default value is '$confdir/classes.txt'.
|
|
classfile = $vardir/classes.txt
|
|
|
|
# Where puppetd caches the local configuration. An
|
|
# extension indicating the cache format is added automatically.
|
|
# The default value is '$confdir/localconfig'.
|
|
localconfig = $vardir/localconfig
|
|
|
|
# Note: The port that the client daemon listens on, defaults to
|
|
# 8139. However, at PSI we run puppetd via the psi-puppet
|
|
# script with run onetime option enabled.
|
|
# psi-puppet is triggered by cron.
|
|
|
|
#########################
|
|
####### SL 5 ########
|
|
#########################
|
|
|
|
### begin{ SL 5 (SL54), gasser_m
|
|
[DesktopSL5Unstable]
|
|
manifest = /var/puppet/environments/DesktopSL5Unstable/manifests/site.pp
|
|
modulepath = /var/puppet/environments/DesktopSL5Unstable/modules
|
|
|
|
[ServerSL5Unstable]
|
|
manifest = /var/puppet/environments/ServerSL5Unstable/manifests/site.pp
|
|
modulepath = /var/puppet/environments/ServerSL5Unstable/modules
|
|
|
|
[DesktopSL5Testing]
|
|
manifest = /var/puppet/environments/DesktopSL5Testing/manifests/site.pp
|
|
modulepath = /var/puppet/environments/DesktopSL5Testing/modules
|
|
|
|
[DesktopSL5Stable]
|
|
manifest = /var/puppet/environments/DesktopSL5Stable/manifests/site.pp
|
|
modulepath = /var/puppet/environments/DesktopSL5Stable/modules
|
|
|
|
[CPT]
|
|
manifest = /var/puppet/environments/CPT/manifests/site.pp
|
|
modulepath = /var/puppet/environments/CPT/modules
|
|
|
|
###}end SL 5 (SL54), gasser_m
|
|
|
|
### V.M. for sl53-c-ks.cfg
|
|
[CnodeSL5]
|
|
manifest = /var/puppet/environments/CnodeSL5/manifests/site.pp
|
|
modulepath = /var/puppet/environments/CnodeSL5/modules
|
|
|
|
[PHServerSL5]
|
|
manifest = /var/puppet/environments/PHServerSL5/manifests/site.pp
|
|
modulepath = /var/puppet/environments/PHServerSL5/modules
|
|
|
|
[EdgarDevelopment]
|
|
manifest = /var/puppet/environments/EdgarDevelopment/manifests/site.pp
|
|
modulepath = /var/puppet/environments/EdgarDevelopment/modules
|
|
|
|
[DerekDevelopment]
|
|
manifest = /var/puppet/environments/DerekDevelopment/manifests/site.pp
|
|
modulepath = /var/puppet/environments/DerekDevelopment/modules
|
|
|
|
[cray]
|
|
manifest = /var/puppet/environments/cray/manifests/site.pp
|
|
modulepath = /var/puppet/environments/cray/modules
|
|
|
|
|
|
### begin Heiner{
|
|
[HeinerDevelopment]
|
|
manifest = /var/puppet/environments/HeinerDevelopment/manifests/site.pp
|
|
modulepath = /var/puppet/environments/HeinerDevelopment/modules
|
|
[HeinerDevelopment54]
|
|
manifest = /var/puppet/environments/HeinerDevelopment54/manifests/site.pp
|
|
modulepath = /var/puppet/environments/HeinerDevelopment54/modules
|
|
[GFA]
|
|
manifest = /var/puppet/environments/GFA/manifests/site.pp
|
|
modulepath = /var/puppet/environments/GFA/modules
|
|
### }end Heiner
|
|
|
|
### begin Rene{
|
|
[GFADesktopSL5]
|
|
manifest = /var/puppet/environments/GFADesktopSL5/manifests/site.pp
|
|
modulepath = /var/puppet/environments/GFADesktopSL5/modules
|
|
|
|
[GFADesktopSL6]
|
|
manifest = /var/puppet/environments/GFADesktopSL6/manifests/site.pp
|
|
modulepath = /var/puppet/environments/GFADesktopSL6/modules
|
|
### }end Rene
|
|
|
|
### Services
|
|
[Web]
|
|
manifest = /var/puppet/environments/Web/manifests/site.pp
|
|
modulepath = /var/puppet/environments/Web/modules
|
|
|
|
[Virtual]
|
|
manifest = /var/puppet/environments/Virtual/manifests/site.pp
|
|
modulepath = /var/puppet/environments/Virtual/modules
|
|
|
|
[News]
|
|
manifest = /var/puppet/environments/News/manifests/site.pp
|
|
modulepath = /var/puppet/environments/News/modules
|
|
|
|
[MySQL]
|
|
manifest = /var/puppet/environments/MySQL/manifests/site.pp
|
|
modulepath = /var/puppet/environments/MySQL/modules
|
|
|
|
[Loadbalancer]
|
|
manifest = /var/puppet/environments/Loadbalancer/manifests/site.pp
|
|
modulepath = /var/puppet/environments/Loadbalancer/modules
|
|
|
|
[LlcLoadbalancer]
|
|
manifest = /var/puppet/environments/LlcLoadbalancer/manifests/site.pp
|
|
modulepath = /var/puppet/environments/LlcLoadbalancer/modules
|
|
|
|
[License]
|
|
manifest = /var/puppet/environments/License/manifests/site.pp
|
|
modulepath = /var/puppet/environments/License/modules
|
|
|
|
[FTP]
|
|
manifest = /var/puppet/environments/FTP/manifests/site.pp
|
|
modulepath = /var/puppet/environments/FTP/modules
|
|
|
|
[Elog]
|
|
manifest = /var/puppet/environments/Elog/manifests/site.pp
|
|
modulepath = /var/puppet/environments/Elog/modules
|
|
|
|
[Cups]
|
|
manifest = /var/puppet/environments/Cups/manifests/site.pp
|
|
modulepath = /var/puppet/environments/Cups/modules
|
|
|
|
[Archive]
|
|
manifest = /var/puppet/environments/Archive/manifests/site.pp
|
|
modulepath = /var/puppet/environments/Archive/modules
|
|
|
|
|
|
#########################
|
|
####### SL 6 ########
|
|
#########################
|
|
|
|
### begin{ SL 6 (gasser_m)
|
|
[DesktopSL6Unstable]
|
|
manifest = /var/puppet/environments/DesktopSL6Unstable/manifests/site.pp
|
|
modulepath = /var/puppet/environments/DesktopSL6Unstable/modules
|
|
|
|
[DesktopSL6Testing]
|
|
manifest = /var/puppet/environments/DesktopSL6Testing/manifests/site.pp
|
|
modulepath = /var/puppet/environments/DesktopSL6Testing/modules
|
|
|
|
[DesktopSL6Stable]
|
|
manifest = /var/puppet/environments/DesktopSL6Stable/manifests/site.pp
|
|
modulepath = /var/puppet/environments/DesktopSL6Stable/modules
|
|
|
|
###}end SL 6 (gasser_m)
|
|
|
|
### Markushin
|
|
[CnodeSL6]
|
|
manifest = /var/puppet/environments/CnodeSL6/manifests/site.pp
|
|
modulepath = /var/puppet/environments/CnodeSL6/modules
|
|
|
|
|
|
Config file `fileserver.conf`::
|
|
|
|
# This file consists of arbitrarily named sections/modules
|
|
# defining where files are served from and to whom
|
|
|
|
# Define a section 'files'
|
|
# Adapt the allow/deny settings to your needs. Order
|
|
# for allow/deny does not matter, allow always takes precedence
|
|
# over deny
|
|
# [files]
|
|
# path /var/lib/puppet/files
|
|
# allow *.example.com
|
|
# deny *.evil.example.com
|
|
# allow 192.168.0.0/24
|
|
|
|
|
|
#[facts]
|
|
# path /etc/puppet/facts
|
|
# allow *.psi.ch
|
|
|
|
[GFA5]
|
|
path /afs/psi.ch/project/slscomp/puppet/gfa5
|
|
allow *.psi.ch
|
|
|
|
[GFA6]
|
|
path /afs/psi.ch/project/slscomp/puppet/gfa6
|
|
allow *.psi.ch
|
|
|
|
|
|
Mount AFS Volumes on Puppet Server
|
|
..................................
|
|
|
|
The puppet manifests for clients are located on AFS::
|
|
|
|
/afs/psi.ch/service/linux/puppet/var/puppet/environments/
|
|
|
|
AFS is already mounted as `/afs` in this default SL5 server installation::
|
|
|
|
# mount
|
|
...
|
|
AFS on /afs type afs (rw)
|
|
|
|
Now, we want to remount
|
|
`/afs/psi.ch/service/linux/puppet/var/puppet/environments` on
|
|
`/var/puppet/environments`. Therefor the mount option `bind` is used,
|
|
which facilitates to remount parts of already mounted filesystems on
|
|
an alternative location in the file hierarchy.
|
|
|
|
The server also needs the permission on AFS to mount the environments
|
|
directory. Add the new server to the AFS group
|
|
svc.linux:puppet_hosts::
|
|
|
|
# pts ad -u <IP_ADDRESS> -g svc.linux:puppet_hosts
|
|
|
|
As shown below we do the remount in `/etc/rc.local`, which is executed
|
|
after all the other init scripts::
|
|
|
|
#!/bin/sh
|
|
|
|
touch /var/lock/subsys/local
|
|
|
|
# Puppet
|
|
mount -o bind /afs/psi.ch/service/linux/puppet/etc/puppet/environments /var/puppet/environments
|
|
|
|
# Restart Services depending on afs mounts
|
|
/etc/init.d/puppetmaster restart
|
|
|
|
Before the `rc.local` script can be applied the proper AFS permissions
|
|
have to be set to make the files readable for psi-puppet1. This was
|
|
done already before, see topic
|
|
[[PuppetServerPsiPuppet2ForSl51#4_1_3_Mount_AFS_Volumes_on_Puppe][Puppet
|
|
Server Psi Puppet 2 For SL51]], so we only have to put the IP address
|
|
of psi-puppet1 to the AFS group `svc_linux:puppet_hosts`::
|
|
|
|
# pts adduser 129.129.190.174 svc_linux:puppet_hosts
|
|
|
|
Configuring Puppet Reporting
|
|
............................
|
|
|
|
There are a number of different report processors available on the
|
|
puppetmaster. The default report, store, simply stores the report file
|
|
on the disk.
|
|
|
|
By default, each client is configured not to report back to the
|
|
master. It has to be enabled either by the report option in
|
|
`puppet.conf` or using `--report` on the command line.
|
|
|
|
`/etc/puppet/puppet.conf`::
|
|
|
|
[puppetd]
|
|
report = true
|
|
|
|
Command line::
|
|
|
|
# puppetd --report
|
|
|
|
Store Report Processor
|
|
,,,,,,,,,,,,,,,,,,,,,,
|
|
|
|
Enable the store reports by using the `reports`
|
|
configuration option in the puppemasterd section of the
|
|
`puppet.conf` file on the master.
|
|
|
|
`/etc/puppet/puppet.conf`::
|
|
|
|
[puppetmasterd]
|
|
reports = store
|
|
|
|
The default reports directory is `$vardir/reports`.
|
|
|
|
Tagmail Report Processor
|
|
,,,,,,,,,,,,,,,,,,,,,,,,
|
|
|
|
Enable the tagmail reports by using the `reports` configuration option
|
|
in the puppemasterd section of the `puppet.conf` file on the
|
|
master. The `tagmail.conf` file contains a list of tags and email
|
|
adresses. The special tags all and err are defined implicitly.
|
|
|
|
`/etc/puppet/puppet.conf`::
|
|
|
|
[puppetmasterd]
|
|
reports = tagmail
|
|
tagmap = $confdir/tagmail.conf
|
|
|
|
`/etc/puppet/tagmail.conf`::
|
|
|
|
all: marc.gasser@psi.ch
|
|
err: marc.gasser@psi.ch
|
|
|
|
|
|
Rrdgraph Report Processors
|
|
,,,,,,,,,,,,,,,,,,,,,,,,,,
|
|
|
|
To enable the `rrdgraph` reports, `rrdtool` and `rrdtool-ruby`
|
|
packages have to be installed.
|
|
|
|
Download the packages from the following repository:
|
|
`/etc/yum.repos.d/epeli386.repo`::
|
|
|
|
[epeli386]
|
|
name=epel i386
|
|
baseurl=http://download.fedora.redhat.com/pub/epel/5/i386/
|
|
enabled=0
|
|
|
|
|
|
# yumdownloader --enablerepo=epeli386 rrdtool.i386 rrdtool-ruby.i386
|
|
# yum install rrdtool-1.2.27-3.el5.i386.rpm
|
|
# yum install rrdtool-ruby-1.2.27-3.el5.i386.rpm
|
|
|
|
You might want to put them to your local repository, too.
|
|
|
|
**Note**: For the time being put them to `psi-beta`, because they
|
|
break dependencies in the other repositories.
|
|
|
|
Then, configure `puppet.conf` by adding the lines shown below in the
|
|
corresponding section. Here store, tagmail and rrdgraph are enabled.
|
|
|
|
`/etc/puppet/puppet.conf`::
|
|
|
|
[puppetmasterd]
|
|
reports = store, tagmail, rrdgraph
|
|
|
|
rrddir = $vardir/rrd
|
|
rrdinterval = $runinterval
|
|
rrdgraph = true
|
|
|
|
Install The Ganglia Monitor Daemon
|
|
..................................
|
|
|
|
Install `ganglia-gmond-3.0.6-4.slp5` and add the configuration
|
|
`/etc/gmond.conf` file as shown below::
|
|
|
|
/* This configuration is as close to 2.5.x default behavior as possible
|
|
The values closely match ./gmond/metric.h definitions in 2.5.x */
|
|
globals {
|
|
daemonize = yes
|
|
setuid = yes
|
|
user = nobody
|
|
debug_level = 0
|
|
max_udp_msg_len = 1472
|
|
mute = no
|
|
deaf = no
|
|
host_dmax = 0 /*secs */
|
|
cleanup_threshold = 300 /*secs */
|
|
gexec = no
|
|
}
|
|
|
|
/* If a cluster attribute is specified, then all gmond hosts are wrapped inside
|
|
* of a <CLUSTER> tag. If you do not specify a cluster tag, then all <HOSTS> will
|
|
* NOT be wrapped inside of a <CLUSTER> tag. */
|
|
cluster {
|
|
name = "puppet"
|
|
owner = "unspecified"
|
|
latlong = "unspecified"
|
|
url = "unspecified"
|
|
}
|
|
|
|
/* The host section describes attributes of the host, like the location */
|
|
host {
|
|
location = "unspecified"
|
|
}
|
|
|
|
/* Feel free to specify as many udp_send_channels as you like. Gmond
|
|
used to only support having a single channel */
|
|
udp_send_channel {
|
|
mcast_join = 239.129.190.89
|
|
port = 8649
|
|
}
|
|
|
|
/* You can specify as many udp_recv_channels as you like as well. */
|
|
udp_recv_channel {
|
|
mcast_join = 239.129.190.89
|
|
port = 8649
|
|
bind = 239.129.190.89
|
|
}
|
|
|
|
# udp_recv_channel {
|
|
# host = "puppet"
|
|
# port = 8649
|
|
# }
|
|
|
|
/* You can specify as many tcp_accept_channels as you like to share
|
|
an xml description of the state of the cluster */
|
|
tcp_accept_channel {
|
|
port = 8649
|
|
}
|
|
|
|
|
|
/* The old internal 2.5.x metric array has been replaced by the following
|
|
collection_group directives. What follows is the default behavior for
|
|
collecting and sending metrics that is as close to 2.5.x behavior as
|
|
possible. */
|
|
|
|
/* This collection group will cause a heartbeat (or beacon) to be sent every
|
|
20 seconds. In the heartbeat is the GMOND_STARTED data which expresses
|
|
the age of the running gmond. */
|
|
collection_group {
|
|
collect_once = yes
|
|
time_threshold = 20
|
|
metric {
|
|
name = "heartbeat"
|
|
}
|
|
}
|
|
|
|
/* This collection group will send general info about this host every 1200 secs.
|
|
This information doesn't change between reboots and is only collected once. */
|
|
collection_group {
|
|
collect_once = yes
|
|
time_threshold = 1200
|
|
metric {
|
|
name = "cpu_num"
|
|
}
|
|
metric {
|
|
name = "cpu_speed"
|
|
}
|
|
metric {
|
|
name = "mem_total"
|
|
}
|
|
/* Should this be here? Swap can be added/removed between reboots. */
|
|
metric {
|
|
name = "swap_total"
|
|
}
|
|
metric {
|
|
name = "boottime"
|
|
}
|
|
metric {
|
|
name = "machine_type"
|
|
}
|
|
metric {
|
|
name = "os_name"
|
|
}
|
|
metric {
|
|
name = "os_release"
|
|
}
|
|
metric {
|
|
name = "location"
|
|
}
|
|
}
|
|
|
|
/* This collection group will send the status of gexecd for this host every 300 secs */
|
|
/* Unlike 2.5.x the default behavior is to report gexecd OFF. */
|
|
collection_group {
|
|
collect_once = yes
|
|
time_threshold = 300
|
|
metric {
|
|
name = "gexec"
|
|
}
|
|
}
|
|
|
|
/* This collection group will collect the CPU status info every 20 secs.
|
|
The time threshold is set to 90 seconds. In honesty, this time_threshold could be
|
|
set significantly higher to reduce unneccessary network chatter. */
|
|
collection_group {
|
|
collect_every = 20
|
|
time_threshold = 90
|
|
/* CPU status */
|
|
metric {
|
|
name = "cpu_user"
|
|
value_threshold = "1.0"
|
|
}
|
|
metric {
|
|
name = "cpu_system"
|
|
value_threshold = "1.0"
|
|
}
|
|
metric {
|
|
name = "cpu_idle"
|
|
value_threshold = "5.0"
|
|
}
|
|
metric {
|
|
name = "cpu_nice"
|
|
value_threshold = "1.0"
|
|
}
|
|
metric {
|
|
name = "cpu_aidle"
|
|
value_threshold = "5.0"
|
|
}
|
|
metric {
|
|
name = "cpu_wio"
|
|
value_threshold = "1.0"
|
|
}
|
|
/* The next two metrics are optional if you want more detail...
|
|
... since they are accounted for in cpu_system.
|
|
metric {
|
|
name = "cpu_intr"
|
|
value_threshold = "1.0"
|
|
}
|
|
metric {
|
|
name = "cpu_sintr"
|
|
value_threshold = "1.0"
|
|
}
|
|
*/
|
|
}
|
|
|
|
collection_group {
|
|
collect_every = 20
|
|
time_threshold = 90
|
|
/* Load Averages */
|
|
metric {
|
|
name = "load_one"
|
|
value_threshold = "1.0"
|
|
}
|
|
metric {
|
|
name = "load_five"
|
|
value_threshold = "1.0"
|
|
}
|
|
metric {
|
|
name = "load_fifteen"
|
|
value_threshold = "1.0"
|
|
}
|
|
}
|
|
|
|
/* This group collects the number of running and total processes */
|
|
collection_group {
|
|
collect_every = 80
|
|
time_threshold = 950
|
|
metric {
|
|
name = "proc_run"
|
|
value_threshold = "1.0"
|
|
}
|
|
metric {
|
|
name = "proc_total"
|
|
value_threshold = "1.0"
|
|
}
|
|
}
|
|
|
|
/* This collection group grabs the volatile memory metrics every 40 secs and
|
|
sends them at least every 180 secs. This time_threshold can be increased
|
|
significantly to reduce unneeded network traffic. */
|
|
collection_group {
|
|
collect_every = 40
|
|
time_threshold = 180
|
|
metric {
|
|
name = "mem_free"
|
|
value_threshold = "1024.0"
|
|
}
|
|
metric {
|
|
name = "mem_shared"
|
|
value_threshold = "1024.0"
|
|
}
|
|
metric {
|
|
name = "mem_buffers"
|
|
value_threshold = "1024.0"
|
|
}
|
|
metric {
|
|
name = "mem_cached"
|
|
value_threshold = "1024.0"
|
|
}
|
|
metric {
|
|
name = "swap_free"
|
|
value_threshold = "1024.0"
|
|
}
|
|
}
|
|
|
|
collection_group {
|
|
collect_every = 40
|
|
time_threshold = 300
|
|
metric {
|
|
name = "bytes_out"
|
|
value_threshold = 4096
|
|
}
|
|
metric {
|
|
name = "bytes_in"
|
|
value_threshold = 4096
|
|
}
|
|
metric {
|
|
name = "pkts_in"
|
|
value_threshold = 256
|
|
}
|
|
metric {
|
|
name = "pkts_out"
|
|
value_threshold = 256
|
|
}
|
|
}
|
|
|
|
/* Different than 2.5.x default since the old config made no sense */
|
|
collection_group {
|
|
collect_every = 1800
|
|
time_threshold = 3600
|
|
metric {
|
|
name = "disk_total"
|
|
value_threshold = 1.0
|
|
}
|
|
}
|
|
|
|
collection_group {
|
|
collect_every = 40
|
|
time_threshold = 180
|
|
metric {
|
|
name = "disk_free"
|
|
value_threshold = 1.0
|
|
}
|
|
metric {
|
|
name = "part_max_used"
|
|
value_threshold = 1.0
|
|
}
|
|
}
|
|
|
|
# /etc/init.d/gmond start
|
|
|
|
|
|
See puppet at http://129.129.190.27/ganglia/. For the ganglia server
|
|
configuration ask Valeri Markushin.
|
|
|
|
|
|
Install The Networker Backup Client (Legato)
|
|
............................................
|
|
|
|
References:
|
|
|
|
- [[http://ait.web.psi.ch/services/central_backup/][Information for Backup Client Administrators]]
|
|
|
|
- [[http://ait.web.psi.ch/services/central_backup/nsr_install_lnx.html][Networker Client installation on Linux ]]
|
|
|
|
Install the Networker client packages, the client itself and the
|
|
manual pages. By default `yum` calculates a lot of dependencies
|
|
required for the GUI of Networker, which facilitates the
|
|
restore. However, the restore can also be done using the command line
|
|
interface, thus the whole X installation shall be skipped. To do so,
|
|
the packages have to be installed without dependencies.
|
|
|
|
Because `yum` does not provide an installation without dependencies,
|
|
`yumdownloader` is used to fetch the packages and `rpm -i --nodeps` to
|
|
install them.
|
|
|
|
First install `yumdownloader`::
|
|
|
|
# yum install yum-utils
|
|
|
|
Install the rest::
|
|
|
|
# yumdownloader --enablerepo=psi-beta lgtoclnt.i686 lgtoman.i686
|
|
# rpm -ivh --nodeps lgtoclnt-7.4.2-1.i686.rpm lgtoman-7.4.2-1.i686.rpm
|
|
|
|
Start the Networker daemon::
|
|
|
|
# service networker start
|
|
|
|
The `/nsr` directory is automatically created. Add the string
|
|
`bs1.psi.ch` in the file `/nsr/res/server`.
|
|
|
|
Restart the Networker daemon::
|
|
|
|
# service networker stop
|
|
# service networker start
|
|
|
|
Now, contact the backup server administrator, Marco Kohler, so he can
|
|
add the host and the directories of interest to the backup service.
|
|
|
|
The next steps are for facilitating the task of the backup server
|
|
administrator.
|
|
|
|
Create the file `~/nsradmin74_x.txt` with the following three lines::
|
|
|
|
update administrator:"isroot,host=psi-puppet1","isroot,host=localhost","isroot,host=bs1","user=root,host=localhost","user=administrator,host=bs1"
|
|
. type: NSR System Port Ranges
|
|
update administrator:"isroot,host=psi-puppet1","isroot,host=localhost","isroot,host=bs1","user=root,host=localhost","user=administrator,host=bs1"
|
|
|
|
|
|
Then execute the command below and check the output::
|
|
|
|
# nsradmin -i ~/nsradmin74_x.txt -p nsrexec
|
|
|
|
updated resource id 3.0.104.17.41.235.57.74.129.129.190.174(7)
|
|
updated resource id 9.0.104.17.41.235.57.74.129.129.190.174(2)
|
|
updated resource id 8.0.168.18.5.236.57.74.129.129.190.174(2)
|
|
updated resource id 9.0.168.18.5.236.57.74.129.129.190.174(2)
|
|
Current query set
|
|
updated resource id 7.0.104.17.41.235.57.74.129.129.190.174(2)
|
|
|
|
|
|
Finally, test if the installation was successful::
|
|
|
|
# service networker stop
|
|
# service networker start
|
|
# service networker status
|
|
+--o nsrexecd (5995)
|
|
|
|
**Note**: Open files will not necessarily be considered during the
|
|
backup run. It depends on their locking state.
|
|
|
|
|
|
How To Update the Networker Backup Client
|
|
.........................................
|
|
|
|
Because the Networker RPM is not cleanly packed, updating the client
|
|
requires deinstallation of the old and installation of the new
|
|
package.
|
|
|
|
First the old `/nsr` directory has to be deleted. Then repeat the
|
|
whole procedure shown in the previous section.
|
|
|
|
The Networker Administration Program
|
|
....................................
|
|
|
|
To start the Networker administration shell type the following
|
|
command::
|
|
|
|
# nsradmin -p nsrexec
|
|
|
|
The Networker Recover Tool
|
|
..........................
|
|
|
|
Check out the manpage of `recover`::
|
|
|
|
# man recover
|
|
|
|
|
|
Setup The Puppet Client
|
|
~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
At this time the only difference between the old and the new client
|
|
configuration is the name of the puppet server in the file
|
|
`/etc/puppet/puppet.conf`, `psi-puppet1` instead of `pxeserv01`.
|
|
|
|
File `/etc/puppet/puppet.conf` on `vmmarctest1.psi.ch`::
|
|
|
|
[main]
|
|
vardir = /var/puppet
|
|
logdir = /var/log/puppet
|
|
rundir = /var/run/puppet
|
|
ssldir = $vardir/ssl
|
|
environment = development
|
|
|
|
[puppetd]
|
|
classfile = $vardir/classes.txt
|
|
localconfig = $vardir/localconfig
|
|
factsync = true
|
|
server = psi-puppet1.psi.ch
|
|
|
|
|
|
Because the new puppet server refers to the same sources (files) as
|
|
the current productive server, we set the immutable to the file above,
|
|
otherwise next time `puppetd` is running the server entry will be
|
|
changed to `pxeserv01` again.
|
|
|
|
The sources are located at
|
|
`/afs/psi.ch/software/linux/dist/scientific/51/puppet/files/` on
|
|
AFS. This path is set in the file `/etc/puppet/fileserver.conf` on the
|
|
puppet server.
|
|
|
|
Make First Tests
|
|
~~~~~~~~~~~~~~~~
|
|
|
|
Start the puppetmaster::
|
|
|
|
# /etc/init.d/puppetmaster start
|
|
|
|
Test it with a client (the options are: keep process in the
|
|
foreground, run onetime and be verbose)::
|
|
|
|
# [root@vmmarctest1 ~]
|
|
# puppetd --no-daemonize -o -v
|
|
|
|
Or run the client in no operational mode, i.e. dry runs without
|
|
actually applying the configuration::
|
|
|
|
# puppetd --noop --no-daemonize -o -v
|
|
|
|
info: Loading fact sysconfig_psi
|
|
info: Loading fact sysconfig_psi-gfa
|
|
info: Creating a new certificate request for vmmarctest1.psi.ch
|
|
info: Creating a new SSL key at /var/puppet/ssl/private_keys/vmmarctest1.psi.ch.pem
|
|
warning: peer certificate won't be verified in this SSL session
|
|
notice: Got signed certificate
|
|
info: Retrieving facts
|
|
info: Loading fact sysconfig_psi
|
|
info: Loading fact sysconfig_psi-gfa
|
|
info: Caching catalog at /var/puppet/localconfig.yaml
|
|
notice: Starting catalog run
|
|
notice: //Node[default]/psi_localadmin/Exec[/usr/bin/psi-fix_file_permission >/dev/null]/returns: executed successfully
|
|
info: Filebucket[/var/puppet/clientbucket]: Adding /usr/share/texmf/dvips/config/config.ps(1611c4bb4b35341f1945059ff774c6df)
|
|
notice: //Node[default]/psi_base/File[/usr/share/texmf/dvips/config/config.ps]: Filebucketed to with sum 1611c4bb4b35341f1945059ff774c6df
|
|
notice: //Node[default]/psi_base/File[/usr/share/texmf/dvips/config/config.ps]/source: replacing from source puppet://psi-puppet1.psi.ch/51/Desktop/usr/share/texmf/dvips/config/config.ps with contents {md5}b265606dc098a5414f3acd71a8831ef1
|
|
notice: //Node[default]/psi_puppet/File[/etc/puppet/puppet.conf]/checksum: checksum changed '{md5}f2944bb81bfbe22b2a2ac4c9197563f3' to '{md5}be67850ccad5409063a56de9d5a516d3'
|
|
notice: //Node[default]/psi_puppet/File[/etc/puppet/puppet.conf]: Filebucketed to with sum be67850ccad5409063a56de9d5a516d3
|
|
err: //Node[default]/psi_puppet/File[/etc/puppet/puppet.conf]: Could not rename tmp /etc/puppet/puppet.conf for replacing: Operation not permitted - /etc/puppet/puppet.conf.puppettmp or /etc/puppet/puppet.conf
|
|
notice: //Node[default]/psi_puppet/File[/etc/puppet/puppet.conf]/source: replacing from source puppet://psi-puppet1.psi.ch/51/Desktop/etc/puppet/puppet.conf.testing with contents {md5}f2944bb81bfbe22b2a2ac4c9197563f3
|
|
info: Filebucket[/var/puppet/clientbucket]: Adding /etc/sysctl.conf(d5716d328f5b840eb4e13ae1d2896fe9)
|
|
notice: //Node[default]/psi_base/File[/etc/sysctl.conf]: Filebucketed to with sum d5716d328f5b840eb4e13ae1d2896fe9
|
|
notice: //Node[default]/psi_base/File[/etc/sysctl.conf]/source: replacing from source puppet://psi-puppet1.psi.ch/51/Desktop/etc/sysctl.conf with contents {md5}d576ff606d3f93df26965e7ef364bd07
|
|
notice: //Node[default]/psi_yum/Exec[/usr/sbin/psi-get-yumconf]/returns: executed successfully
|
|
notice: Finished catalog run in 6.22 seconds
|
|
|
|
|
|
So, this looks promising. Seems like the client could get it's
|
|
configuration from the new puppet server.
|
|
|
|
Only the file `/etc/puppet/puppet.conf` could not be changed, what is
|
|
ok because the immutable flag was set.
|
|
|
|
Next Steps
|
|
----------
|
|
|
|
- Verify migration order (server, client or vice versa)
|
|
|
|
- Finalize basic server setup (verify that no config agents compromise
|
|
the system, e.g. puppetd which could be executed by cron or during
|
|
boot time, etc.), check whether it makes sense to use DNS aliases
|
|
for the hostname.
|
|
|
|
- Shall server configuration files be stored locally or mounted from
|
|
AFS?
|
|
|
|
|
|
`psi-puppet1:/etc/rc.d/rc.local` has been prepared (not activated yet)
|
|
for the AFS mount::
|
|
|
|
#!/bin/sh
|
|
#
|
|
# This script will be executed *after* all the other init scripts.
|
|
# You can put your own initialization stuff in here if you don't
|
|
# want to do the full Sys V style init stuff.
|
|
|
|
touch /var/lock/subsys/local
|
|
|
|
# Puppet
|
|
#mount -o bind /afs/psi.ch/service/linux/puppet/etc/puppet-0.24.7-4 /etc/puppet
|
|
|
|
# Restart Services depending on afs mounts
|
|
#/etc/init.d/puppetmaster restart
|
|
|
|
The whole current puppetserver configuration from /etc/puppet/ was
|
|
copied to `/afs/psi.ch/service/linux/puppet/etc/puppet-0.24.7-4`.
|
|
|
|
- If mounted from AFS the question remains how root@psi-puppet1 gets
|
|
the permission to mount the mentioned AFS directory.
|
|
|
|
- Shall the client configuration manifests be stored locally or on
|
|
AFS?
|
|
|
|
|
|
Locally: `/var/puppet/environments/`
|
|
|
|
AFS: `/afs/psi.ch/service/linux/puppet/etc/puppet-0.24.7-4/environments/`
|
|
|
|
- Run the puppetmaster on hardware or vmware? Hardware.
|
|
|
|
- When the server is going to production the IP has to be changed, see
|
|
*Static IP for Production Server* above. Done.
|
|
|
|
- When the server is going to production the PSI firewall has to be
|
|
adjusted. (Refer to Tobias)
|
|
|
|
- Test with old client to new server, and new client to old server.
|
|
|
|
- Test with limited number of new client to new server.
|