Installation `psi-puppet2` ========================== References ---------- - https://wiki.intranet.psi.ch/AIT/PuppetServer2009 - http://reductivelabs.com/trac/puppet/wiki/PluginsInModules Introduction ------------ This document describes the Installation of the new puppet server psi-puppet2, which replaces the puppet server running on pxeserv01. The psi-puppet2 server is an interim solution for the pxeserv01 replacement for the following reasons: - The puppet service on pxeserv01 is very unstable. - The performance on pxeserv01 is very bad, what makes work almost impossible. - The psi-puppet1 server is not ready for production yet. The new system will be setup as a 32 bit SL5 server with the latest puppet server package and dependencies installed. The puppet server configuration is the same as the one on pxeserv01. **Note**: Don't confuse psi-puppet2 with psi-puppet1, which not only is thought as a replacement of the puppet server on pxeserv01 but also comes with a reimplementation of the whole puppet configuration, whereas psi-puppet2 takes over the puppet configuration from pxeserv01. Description of the Basic Server Setup ------------------------------------- - Hardware Dell Power Edge 1750 - Operating System SL51 32 bit Server - Network configuration Static IP for Production Server: 129.129.190.160/24 Hostname: psi-puppet2.psi.ch - Required RPMS - puppet-server (http://download.fedora.redhat.com/pub/epel/5/i386/repoview/) - augeas-libs - facter - puppet (http://download.fedora.redhat.com/pub/epel/5/i386/repoview/) - ruby - ruby-augeas - ruby-libs - ruby-shadow - ruby-irb (required for reading help) - ruby-rdoc (required for reading help) Procedure --------- Setup The Puppet Server ~~~~~~~~~~~~~~~~~~~~~~~ Install The Puppet Server Software .................................. Use the yum repositories `/etc/yum.repos.d/epeli386.repo` and `/etc/yum.repos.d/epelx86_64.repo` to download and install the latest puppet packages:: [epeli386] name=epel i386 baseurl=http://download.fedora.redhat.com/pub/epel/5/i386/ enabled=0 Install the puppet-server package. Dependencies will be resolved:: # yum --enablerepo=epeli386 install puppet-server ... Dependencies Resolved ============================================================================= Package Arch Version Repository Size ============================================================================= Installing: puppet-server noarch 0.24.8-1.el5.1 epel 26 k Installing for dependencies: augeas-libs i386 0.5.1-1.el5 epel 196 k facter noarch 1.5.5-1.el5 epel 54 k puppet noarch 0.24.8-1.el5.1 epel 542 k ruby i386 1.8.5-5.el5_2.6 sl5update 279 k ruby-augeas i386 0.2.0-1.el5 epel 17 k ruby-libs i386 1.8.5-5.el5_2.6 sl5update 1.6 M ruby-shadow i386 1.4.1-7.el5 epel 9.5 k Transaction Summary ============================================================================= Install 8 Package(s) Update 0 Package(s) Remove 0 Package(s) ... For later use, download the required RPMS and copy them to our yum repository, see section `Put Puppet Related RPMS To Our Yum Repository` below. Configure The Puppet Server ........................... Use the puppet server configuration of pxeserv01 on psi-puppet2. The configuration files of the puppet server, directory `/etc/puppet/`, are located on AFS, see section `Mount AFS Volumes`. The log is on the local disk in `/var/log/puppet`. To set the logfile edit the line `PUPPETMASTER_OPTS` in `/etc/rc.d/init.d/puppetmaster`. For testing also the debug option `-d` is enabled:: PUPPETMASTER_OPTS="-v -d -l /var/log/puppet/puppetmaster.log" Enable puppetmaster in runlevels 3, 4 and 5:: # chkconfig --levels 345 puppetmaster on Mount AFS Volumes on Puppet Server .................................. The puppet server configuration files as well as the puppet manifests for clients are located on AFS. AFS is already mounted as `/afs` in this default SL5 server installation:: # mount ... AFS on /afs type afs (rw) Now, we want to remount `/afs/psi.ch/service/linux/puppet/etc/puppet` on `/etc/puppet`. Therefor the mount option `bind` is used, which facilitates to remount parts of already mounted filesystems on an alternative location in the file hierarchy. As shown below we do the remount in `/etc/rc.local`, which is executed after all the other init scripts:: #!/bin/sh touch /var/lock/subsys/local # Puppet mount -o bind /afs/psi.ch/service/linux/puppet/etc/puppet /etc/puppet # Restart Services depending on afs mounts /etc/init.d/puppetmaster restart Before the `rc.local` script can be applied the proper AFS permissions have to be set. Check the AFS permissions:: # [gasser_m@pc7377 ~] # fs la /afs/psi.ch/service/linux/ Access list for /afs/psi.ch/service/linux/ is Normal rights: svc_linux:tools l svc_linux:puppet l svc_linux:readonly rl svc_linux:pxe l svc_linux rlidwka Create a new AFS group for the puppet server:: # pts creategr svc_linux:puppet_hosts -owner svc_linux group svc_linux:puppet_hosts has id -10851 Add the IP of psi-puppet2 to this group:: # pts adduser 129.129.190.160 svc_linux:puppet_hosts Set the AFS permissions:: # fs sa /afs/psi.ch/service/linux/ svc_linux:puppet_hosts l # fs sa /afs/psi.ch/service/linux/puppet/ svc_linux:puppet_hosts rl # fs sa /afs/psi.ch/service/linux/puppet/etc svc_linux:puppet_hosts rl # fs sa /afs/psi.ch/service/linux/puppet/var svc_linux:puppet_hosts rl # fs sa /afs/psi.ch/service/linux/puppet/etc/puppet svc_linux:puppet_hosts rl Recursively set the same permissions to all subdirectories underneath `/afs/psi.ch/service/linux/puppet/etc/puppet`:: # cd /afs/psi.ch/service/linux/puppet/etc/puppet # find -noleaf -type d -exec fs sa {} svc_linux:puppet_hosts rl \; Then restart the AFS service:: # service afs restart Finally, run the `rc.local` script or just reboot psi-puppet2 to see whether everything comes up. Allow User Login Access To psi-puppet2 ...................................... To enable remote login via ssh and execution of root commands via sudo, the following files have to be edited. - `/etc/passwd` - `/etc/shadow` - `/etc/security/ssh.allow` - `/etc/group` - `/etc/hosts.allow` - `/etc/sudoers` passwd:: gasser_m:!:2374:840:Gasser Marc:/afs/psi.ch/user/g/gasser_m:/bin/bash kapeller:!:3804:9102:Rene Kapeller:/afs/psi.ch/user/k/kapeller:/bin/bash billich:!:3830:840:Heinrich Billich:/afs/psi.ch/user/b/billich:/bin/bash markushin:!:3883:840:Valeri Markushin:/afs/psi.ch/user/m/markushin:/bin/bash shadow:: gasser_m:NP::::::: kapeller:NP::::::: billich:NP::::::: markushin:NP::::::: group:: ait::840 sls::9102 ssh.allow:: kapeller billich markushin gasser_m hosts.allow:: sshd: ... pc7377 pc7377.psi.ch gfalc05 gfalc05.psi.ch pc4568 pc4568.psi.ch Configuring Puppet Reporting ............................ There are a number of different report processors available on the master. The default report, store, simply stores the report file on the disk. By default, each client is configured not to report back to the master. It has to be enabled either by the report option in `puppet.conf` or using `--report` on the command line. `/etc/puppet/puppet.conf`:: [puppetd] report = true Command line:: # puppetd --report Store Report Processor ,,,,,,,,,,,,,,,,,,,,,, Enable the store reports by using the `reports` configuration option in the puppemasterd section of the `puppet.conf` file on the master. `/etc/puppet/puppet.conf`:: [puppetmasterd] reports = store The default reports directory is $vardir/reports. Rrdgraph Report Processor ,,,,,,,,,,,,,,,,,,,,,,,,, To enable the `rrdgraph` reports, `rrdtool` and `rrdtool-ruby` packages have to be installed. The packages are available from the the `psi-beta` repository, they originate from the repository shown below. `/etc/yum.repos.d/epeli386.repo`:: [epeli386] name=epel i386 baseurl=http://download.fedora.redhat.com/pub/epel/5/i386/ enabled=0 # yum install rrdtool rrdtool-ruby Then, configure `puppet.conf` by adding the lines shown below in the corresponding section. Here store and rrdgraph are enabled. `/etc/puppet/puppet.conf`:: [puppetmasterd] reports = store,rrdgraph rrddir = $vardir/rrd rrdinterval = $runinterval rrdgraph = true Install the Ganglia Monitor Daemon .................................. Install `ganglia-gmond` and add the configuration file shown below:: # yum install ganglia-gmond Use the same `/etc/gmond.conf` as on psi-puppet1:: # /etc/init.d/gmond start For the ganglia server configuration ask Valeri Markushin. See puppet at [[http://129.129.190.27/ganglia/][http://129.129.190.27/ganglia/]]. Setup The Puppet Client ~~~~~~~~~~~~~~~~~~~~~~~~ Either install or update the puppet package on a client:: # yum --enablerepo=psi-beta install puppet # yum --enablerepo=psi-beta update puppet Test The Puppet Clients ~~~~~~~~~~~~~~~~~~~~~~~ To find out which versions of puppet servers and clients are compatible to each other and in which order they should be updated, the following tests are performed. Test the Old Client With The New Server ....................................... Set the new server `psi-puppet2.psi.ch` in `/etc/puppet/puppet.conf`, remove the certificate and run puppetd:: # [root@vsl5132de-ut] # cd /var/puppet/ # rm -rf ssl/ # puppetd --test Ok. Test the New Client With The New Server ....................................... Update the puppet client. Set the new server `psi-puppet2.psi.ch` in `/etc/puppet/puppet.conf`, remove the certificate and run puppetd:: # [root@vsl5132de-ut] # yum --enablerepo=psi-beta update puppet # cd /var/puppet/ # rm -rf ssl/ # puppetd --test Ok. Test the New Client With The Old Server ....................................... Update the puppet client. Then set the old server `pxeserv01.psi.ch` in `/etc/puppet/puppet.conf`, remove the certificate and run puppetd:: # [root@vsl5132de-ut] # yum --enablerepo=psi-beta update puppet ... ============================================================================= Package Arch Version Repository Size ============================================================================= Updating: puppet noarch 0.24.8-1.el5.1 psi-beta 542 k Installing for dependencies: augeas-libs i386 0.5.1-1.el5 psi-beta 196 k ruby-augeas i386 0.2.0-1.el5 psi-beta 17 k ruby-shadow i386 1.4.1-7.el5 psi-beta 9.5 k Updating for dependencies: facter noarch 1.5.5-1.el5 psi-beta 54 k Transaction Summary ============================================================================= Install 3 Package(s) Update 2 Package(s) Remove 0 Package(s) ... # cd /var/puppet/ # rm -rf ssl/ # puppetd --test Not ok. A lot of error messages appear. It seems the new client is not compatible to the old server! Conclusion .......... First update the server, then the client. So, when you have an old client addressing the new server for tests, and finally you wanna change it back to the old server, this should work by just modifying `/etc/puppet/puppet.conf` on the client. However, once you updated the client you can not go back to the old server, unless you downgrade the client --- with yum this means removing the new client and reinstalling the old one. Put Puppet Related RPMS To Our Yum Repository --------------------------------------------- To have a consistent puppet installation on our hosts we put them to our repository. Use yumdowloader, package yum-utils, to download puppet related RPMS:: # root@psi-puppet2 # yum install yum-utils # yumdownloader --enablerepo=epeli386 puppet-server augeas-libs facter \ # puppet ruby-augeas ruby-shadow # yumdownloader --enablerepo=epelx86_64 augeas-libs ruby-augeas ruby-shadow **Note**: There seem to be only i386 and noarch versions of the required RPMS. Before they are linked into the latest `testing` and `current` repositories they should be tested. So copy them to the `psi-beta` repository first:: # scp *rpm gasser_m@tux50:/afs/psi.ch/software/linux/dist/scientific/51/beta # gasser_m@tux50 # cd /afs/psi.ch/software/linux/dist/scientific/51/beta # createrepo . To test the puppet clients see section `Test The Puppet Clients`. If the tests passed successfully copy them to the `others` repository and create symbolic links to `testing` and `current` to make them available:: # [root@psi-puppet2] # scp *rpm gasser_m@tux50:/afs/psi.ch/software/linux/dist/scientific/51/others/all # gasser_m@tux50 # cd /afs/psi.ch/software/linux/dist/scientific/51/others/all # createrepo . Update Puppet ------------- Keep the following order. - Update the puppet server. - Update the puppet client. Update The Puppet Server ~~~~~~~~~~~~~~~~~~~~~~~~ At the time of writing the latest Puppet version 0.25.1 was only available at http://tmz.fedorapeople.org/repo/puppet/epel/5/i386/. Download the required packages to SL51 psi-beta repository first:: # wget http://tmz.fedorapeople.org/repo/puppet/epel/5/i386/puppet-server-0.25.1-0.3.el5.noarch.rpm # wget http://tmz.fedorapeople.org/repo/puppet/epel/5/i386/puppet-0.25.1-0.3.el5.noarch.rpm Then login to a test server and stop the puppetmaster daemon:: # /etc/init.d/puppetmaster stop Because the /etc/puppet/ is on AFS, root has no write permissions. So, first umount the etc/puppet from AFS, then run `yum update` using the psi-beta repository, and remount etc/puppet:: # umount /etc/puppet # yum --enablerepo=psi-beta update puppet-server # mount -o bind /afs/psi.ch/service/linux/puppet/etc/puppet /etc/puppet Set the following options in /etc/init.d/puppetmaster:: PUPPETMASTER_OPTS="-v -d -l /var/log/puppet/puppetmaster.log" Finally, restart the service and test it with a client:: # /etc/init.d/puppetmaster start Login to a client and run puppetd:: # puppetd --test Update The Puppet Client ~~~~~~~~~~~~~~~~~~~~~~~~ Basically, the 0.24.x clients should be compatible to the 0.25.x server. Though, there are some changes between the versions, that might cause troubles. Thus, the puppet clients should be updated, too. On a SL51 client run:: # yum --enablerepo=psi-beta update puppet On a SL54 client run:: # yum update puppet As soon as the client is updated, its `puppet.conf` needs some modifications, because of the "factsync" option which is deprecated and replaced by "pluginsync" in the 0.25.x versions. `/etc/puppet/puppet.conf` on a 0.25.x client: "factsync" is replaced by "pluginsync", factpath is set (not clear whether the latter is necessary):: [main] vardir = /var/puppet logdir = /var/log/puppet rundir = /var/run/puppet ssldir = $vardir/ssl pluginsync = true factpath = $vardir/lib/facter environment = DesktopSL5Unstable [puppetd] report = true classfile = $vardir/classes.txt localconfig = $vardir/localconfig server = psi-puppet1.psi.ch Further the directory structure on the server for placing facts changed. The new structure if using modules and environments is illustrated below taking the environment "DesktopSL5Unstable" as an example: The modulepath for DesktopSL5Unstable is:: modulepath = /var/puppet/environments/DesktopSL5Unstable/modules A stub module called "custom" has to be created in the "$modulepath" subdirectory to keep the files:: $modulepath/custom/ `-- lib/ |-- facter/ | `-- sysconfig_psi_desktop.rb `-- puppet/ |-- provider/ `-- type/ This subdirectory tree under custom is implicitly searched by the puppet server. **Notes**: If you have both, the new and the old variant to keep facts, option "pluginsync" enabled, only the new script location will be considered. If you run only the old variant to keep facts:: $modulepath/facts/files/somescript.rb with option "factsync" enabled, they will be loaded giving out a warning "... use pluginsync instead of factsync ..." For more information see the http://reductivelabs.com/trac/puppet/wiki/PluginsInModules.