documentation/docs/engineering-guide/sysdb_environments.md

Hiera Environments

Add New Hiera Environment

Introduction

Deploying a new environment requires the following:

1. Configuring the environment in bob
2. Configuring the environment data-<environment_name> in GIT (https://gitea.psi.ch/puppet)
3. Configuring the environment in Puppet (e.g. puppet.psi.ch)

Configuring the environment in sysdb

Bob allows to create a new environment in sysdb by using the bob env option. Only members of the unx-lx_eng group have permissions to do so.

To list current defined environments run

bob env list

To add a new environment, run

bob env add <environment_name> <owner> <admin_group> "<description>"

In example, for the MeG cluster

bob env add meg caubet_m unx-hpc_adm "MeG Cluster"

Test new environment in BOB

In order to test that environment was successfully created

bob env list | grep <environment_name>

An example

caubet_m@caubet-laptop:~/GIT/admin-guide/deployment$ bob env list | grep meg
meg        caubet_m      unx-hpc_adm        MeG Cluster

Configuring the environment in GIT

You must belong to the unx-puppet_adm Active Directory group in order to be able to create new projects.

In order to create a new environment in GIT, you should access and login in the following link: https://gitea.psi.ch/puppet/. Here you can see the different environments.

The steps to create and configure a new GIT project are:

1. Create a new repository (environment). It can be done here: https://gitea.psi.ch/repo/create?org=6

   • Define Repository name, which must have the format data-<environment_name> where <environment_name> is the one defined in Bob
   • Visibility: set checkbox Make repository private
   • Default Branch: set it to master

2. Configure project permissions as follows:

   • data-<environment_name -> Settings -> Collaborators, or directly: https://gitea.psi.ch/puppet/data-(environment_name)/settings/collaboration
     • add teams with access
     • if the team does not yet exists, map it from the AD group as documented in chapter "Mapping AD Groups to Teams" in our Gitea Production Documentation
   • Enable already registered deploy key: Settings -> Deploy Keys -> Add Deploy Key with
     • Title: plidata
     • Content:

     ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTmrqGEcE+Uv88WvnROZ7hcsFeZhAClV3nV66Mq+GVc0COEICFcVYcc73BvqHN6Vgii9yE2ZIKR7fqw2Hf0JIVoBzatzxTJJwXWYKsj8Z9gJbMKy8Axgz6alorLAybRz5DZAjND46/vh8/6E0c1YOytd5d30t92d7Wa4YO6dJqrdRaXK/Li54gdDjnHgbP0McafsnuxcBI4utPD/30+o9V6RcBG7qPdHYEviX3JGYOHsbRWs36VhpOjnxpqVouNECXYTaPtAMMWlgE/bm9mLflVH3UIDn6Ufv2bxcMKd4rJwDzjWqcb047iYhfIiB6ciCrJJ4hYnywVaFnNiA0iNcf/SvQjpkZjgejGhz72IJk4A2j+nb2RIL75NxXHgOZRlQ+JO/D53Keb9jINY4NWDRm85MP2gtuy7xNxoSPylhM0JaDtkUI0uUGshACMS9yMtKPxhQUPJmxIMbm2gKSkrBPIdlHg42O81mbFYR9B/XSW8GjsKZBAaCtQMlh0Zxr5Hc=
     

   • for the runner configuration check out the new Hiera data repo and add .gitea/workflows/hiera_update.yaml from another one.

3. Disable unnecessary Gitea features

   • In data-<environment_name -> Settings, or directly: https://gitea.psi.ch/puppet/data-(environment_name)/settings:
     • untick Enable Repository Wiki
     • untick Enable Projects
     • untick Enable Repository Releases
     • untick Enable Repository Packages Registry
     • click Update Settings

4. Add Deployment Workflow

• Add following workflow in .gitea/workflows/hiera_update.yaml to the repository:

  name: hiera update
  on: 
    push:
      branches:
        - master
    workflow_dispatch:
  
  jobs:
    puppet-deployment:
      runs-on: puppet
      steps:
        - name: Execute hiera-update
          run: |
            environment="${{ github.repository }}"
            echo "sudo -u plidata /opt/pli/bin/hiera-update ${environment#*data-}"
            sudo -u plidata /opt/pli/bin/hiera-update ${environment#*data-}
  
  
    sysdb-deployment:
      runs-on: sysdb
      steps:
        - name: Execute hiera-update
          run: |
            environment="${{ github.repository }}"
            echo "sudo -u plidata /opt/pli/bin/hiera-update ${environment#*data-}"
            sudo -u plidata /opt/pli/bin/hiera-update ${environment#*data-}
  
        - name: Execute hiera-extractor
          run: |
            environment="${{ github.repository }}"
            echo "sudo -u plidata /opt/hiera-extractor/hiera-extractor ${environment#*data-}"
            sudo -u plidata /opt/hiera-extractor/hiera-extractor ${environment#*data-}
  

Test new environment in GIT

In order to test that environment was successfully created

git clone git@gitea.psi.ch:puppet/data-<environment_name>.git

An example

$ git clone git@gitea.psi.ch:puppet/data-ems.git
Cloning into 'data-ems'...
warning: You appear to have cloned an empty repository.

Configuring the Environment in Puppet Server and Sysdb Server

In bootstrap

• add the new environment to the sysdb_environments list in ansible/group_vars/prod.yaml.

• To prepare the eyaml key go into the directory ansible/roles/hiera_checkout/files/srv/puppet and run

  SYSDB_ENV=...
  eyaml createkeys
  mv keys/private_key.pkcs7.pem keys/${SYSDB_ENV}_private_key.pkcs7.pem
  ansible-vault encrypt --vault-pass-file ../../../../../vault-pass keys/${SYSDB_ENV}_private_key.pkcs7.pem
  

  and move keys/public_key.pkcs7.pem as eyaml_public_key.pem into the data-<environment_name> checkout and add, commit and push everything changed on both repos.

• For Puppet server run in ansible to deploy:

  ansible-playbook -i inventory.yaml --vault-pass-file ./vault-pass install_puppet_server.yaml
  

• For Sysdb server run in ansible to deploy:

  ansible-playbook -i inventory.yaml --vault-pass-file ./vault-pass install_sysdb.yaml
  

Test new environment in Puppet

In order to test that environment was successfully created

git clone git@gitea.psi.ch:puppet/data-<environment_name>.git

Add a new file <environment_name>.yaml in to the project:

cd data-<environment_name>
touch <environment_name>.yaml
git add <environment_name>.yaml
git commit -a -m "Added first empty file"
git push

After a few seconds (needs time to trigger the change), check in puppet.psi.ch:/srv/puppet/data/<environment_name> that file was successfully triggered (copied) to the puppet server from GIT

ssh root@puppet.psi.ch ls /srv/puppet/data/<environment_name>/<environment_name>.yaml

Full real example:

git clone git@gitea.psi.ch:puppet/data-meg.git
cd data-meg
touch meg.yaml
git add meg.yaml
git commit -a -m "Added first empty file"
git push
sleep 5
ssh root@puppet.psi.ch ls /srv/puppet/data/meg/meg.yaml

Remove Hiera Environment

To remove a Hiera environment following steps are required:

1. Delete Git repository for the environment (https://gitea.psi.ch/puppet/data-<environment_name>)

2. Remove the environment from bootstrap (https://gitea.psi.ch/lx/bootstrap) by deleting the environment from the sysdb_environments list in ansible/group_vars/prod.yaml

3. Remove directory for the environment on Puppet and Sysdb server (i.e. puppet server: /srv/puppet/data/<environment_name>, sysdb server: /srv/puppet/data/<environment_name>)

4. Remove the environment from sysdb database

   cd /var/www/sysdb
   sqlite3 inv.db
   DELETE FROM environments WHERE name = '<environment_name>';