gitea-pages/engineering-guide/new_sysdb_environment.md

New Sysdb Environment

Introduction

Deploying a new environment requires the following:

1. Configuring the environment in bob
2. Configuring the environment in GIT (https://git.psi.ch/linux-infra/hiera) - Environment format: data-<environment_name> - And change permissions accordingly - Configure the necessary webhooks for the Puppet and Sysdb server
3. Configuring the environment in Puppet (e.g. puppet01.psi.ch)

Configuring the environment in sysdb

Bob allows to create a new environment in sysdb by using the bob env option. You must have permissions to do that.

You must belong to the sysdb-admins group that is actually configured on the local /etc/group file. A migration to Active Directory should be done for that group.

To list current defined environments run

bob env list

To add a new environment, run

bob env add <environment_name> <owner> <admin_group> "<description>"

In example, for the MeG cluster

bob env add meg caubet_m unx-hpc_adm "MeG Cluster"

Test new environment in BOB

In order to test that environment was successfully created

bob env list | grep <environment_name>

An example

caubet_m@caubet-laptop:~/GIT/admin-guide/deployment$ bob env list | grep meg
meg        caubet_m      unx-hpc_adm        MeG Cluster

Configuring the environment in GIT

You must belong to the unx-puppet_adm Active Directory group in order to be able to create new projects.

In order to create a new environment in GIT, you should access and login in the following link: https://git.psi.ch/linux-infra/hiera/. Here you can see the different environments.

The steps to create and configure a new GIT project are:

1. Create a new project (environment) in the hiera group. It can be done here: https://git.psi.ch/projects/new?namespace_id=1738

   • Click [Create blank project]
   • Define [Project name], which must have the format data-<environment_name> where <environment_name> is the one defined in Bob
   • Specify [Visibility Level]: Should be Internal or Private
   • Remove the tick [Initialize repository with a README].

2. Configure project permissions as follows:

• [data-<environment_name]->[Settings]->[Repository], or directly: https://git.psi.ch/linux-infra/hiera/data-(environment_name)/-/settings/repository
  • Enable already registered deploy key: [Deploy Keys] -> [Privately accessible deploy keys] -> select root@puppet01' -> click on 'Enable'
• [data-<environment_name]->[Manage]->[Members], or directly: https://git.psi.ch/linux-infra/hiera/data-(environment_name)/project_members
  • Setup specific permissions for specific users or groups. In example:
    • Set project Maintainer:
      • [Select members to invite] (caubet_m) + [Choose a role permission] (Maintainer) + [Add to project]
    • Set other roles:
      • [Select members to invite] (dorigo_a) + [Choose a role permission] (Developer) + [Add to project]
• [data-<environment_name]->[Settings]->[Webhooks], or directly: https://git.psi.ch/linux-infra/hiera/data-(environment_name)/-/hooks
  • Add WebHooks as follows:
    • [URL]: http://puppet01.psi.ch/events/dataupdate
    • [URL]: http://sysdb.psi.ch/events/dataupdate
    • (Checked) [Push events]. Uncheck the rest.
    • [SSL verification] -> (uncheck) [Enable SSL verification]
    • Confirm information from above, and click on [Add webhook] to add the new WebHook.

Test new environment in GIT

In order to test that environment was successfully created

git clone git@git.psi.ch:linux-infra/hiera/data-<environment_name>.git

An example

caubet_m@caubet-laptop:~/GIT$ git clone git@git.psi.ch:linux-infra/hiera/data-meg.git
Cloning into 'data-meg'...
X11 forwarding request failed
warning: You appear to have cloned an empty repository.

Configuring the environment in Puppet server

In bootstrap add the new repo to instcode/puppet/puppet_server/manifests/data.pp and run the bootstrap for the Puppet server. Note that before this can be rolled out master branch needs to exist on git.psi.ch.

Configuring the environment in Sysdb Server

In bootstrap add the new repo to the suitable inventory file (eg. ansible/inventory.yaml for production) and run the playbook for the Sysdb server. Note that before this can be rolled out master branch needs to exist on git.psi.ch.

Test new environment in Puppet

In order to test that environment was successfully created

git clone git@git.psi.ch:linux-infra/data-<environment_name>.git

Add a new file <environment_name>.yaml in to the project:

cd data-<environment_name>
touch <environment_name>.yaml
git add <environment_name>.yaml
git commit -a -m "Added first empty file"
git push

After a few seconds (needs time to trigger the change), check in puppet01.psi.ch:/srv/puppet/data/<environment_name> that file was successfully triggered (copied) to the puppet server from GIT

ssh root@puppet01.psi.ch ls /srv/puppet/data/<environment_name>/<environment_name>.yaml

Full real example:

git clone git@git.psi.ch:linux-infra/data-meg.git
cd data-meg
touch meg.yaml
git add meg.yaml
git commit -a -m "Added first empty file"
git push
sleep 5
ssh root@puppet01.psi.ch ls /srv/puppet/data/meg/meg.yaml