gitea-pages/admin-guide/deployment/workflow.rst

Deploying New Environment: Workflow

Introduction

Deploying a new environment requires the following:

1. Configuring the environment in bob
2. Configuring the environment in GIT (https://git.psi.ch/linux-infra/hiera)

• Environment format: data-<environment_name>
• And change permissions accordingly
• Configure the necessary webhooks for the Puppet and Sysdb server

3. Configuring the environment in Puppet (e.g. puppet01.psi.ch)

Configuring the environment in sysdb

Bob allows to create a new environment in sysdb by using the bob env option. You must have permissions to do that.

You must belong to the sysdb-admins group that is actually configured on the local /etc/group file. A migration to Active Directory should be done for that group.

To list current defined environments run:

bob env list

To add a new environment, run:

bob env add <environment_name> <owner> <admin_group> "<description>"

In example, for the MeG cluster:

bob env add meg caubet_m unx-hpc_adm "MeG Cluster"

Test new environment in BOB

In order to test that environment was successfully created:

bob env list | grep <environment_name>

In example:

caubet_m@caubet-laptop:~/GIT/admin-guide/deployment$ bob env list | grep meg
meg        caubet_m      unx-hpc_adm        MeG Cluster  

Configuring the environment in GIT

Current GIT server at PSI is git.psi.ch. Every new environment should in principle belong to the linux-infra project.

You must belong to the puppet_env Active Directory group in order to be able to create new projects.

In order to create a new environment in GIT, you should access and login in the following link: https://git.psi.ch/linux-infra/hiera/. Here you can see the different environments.

The steps to create and configure a new GIT project are:

1. Create a new project (environment). It can be done here: https://git.psi.ch/projects/new

• Go to [Blank project] Tab (which is the default tab)

  • Change [Project Path] as follows:

    • https://git.psi.ch/ + linux-infra/hiera

  • Define [Project Name], which must have the following format:

    • data-<environment_name> where <environment_name> is the one defined in Bob

  • [Optional] Specify [Project description]

  • Specify [Visibility Level]:

    • Should be Internal

2. Configure project permissions as follows:

• [data-<environment_name]->[Settings]->[Repository], or directly: https://git.psi.ch/linux-infra/hiera/data-(environment_name)/-/settings/repository

  • [Deploy Keys] -> [Privately accessible deploy keys] -> select root@puppet01.psi.ch' -> click on 'Enable'

• [data-<environment_name]->[Settings]->[Members], or directly: https://git.psi.ch/linux-infra/hiera/data-(environment_name)/project_members

  • Setup specific permissions for specific users or groups. In example:

    • Set project Master:

      • [Select members to invite] (caubet_m) + [Choose a role permission] (Master) + [Add to project]

    • Set other roles:

      • [Select members to invite] (ozerov_d) + [Choose a role permission] (Developer) + [Add to project]

• [data-<environment_name]->[Settings]->[Integrations], or directly: https://git.psi.ch/linux-infra/hiera/data-(environment_name)/-/hooks

  • Add WebHooks as follows:

    • [URL]: http://puppet01.psi.ch/events/dataupdate
    • [URL]: http://sysdb.psi.ch/events/dataupdate
    • (Checked) [Push events]. Uncheck the rest.
    • [SSL verification] -> (uncheck) [Enable SSL verification]
    • Confirm information from above, and click on [Add webhook] to add the new WebHook.

Test new environment in GIT

In order to test that environment was successfully created:

git clone git@git.psi.ch:linux-infra/hiera/data-<environment_name>.git

In example:

caubet_m@caubet-laptop:~/GIT$ git clone git@git.psi.ch:linux-infra/hiera/data-meg.git
Cloning into 'data-meg'...
X11 forwarding request failed
warning: You appear to have cloned an empty repository.

Configuring the environment in Puppet server

In [bootstrap](https://git.psi.ch/linux-infra/bootstrap) add the new repo to instcode/puppet/puppet_server/manifests/data.pp and run the bootstrap for the Puppet server.

Configuring the environment in Sysdb Server

In [bootstrap](https://git.psi.ch/linux-infra/bootstrap) add the new repo to the suitable inventory file (eg. ansible/inventory.yaml for production) and run the playbook for the Sysdb server.

Test new environment in Puppet

In order to test that environment was successfully created:

git clone git@git.psi.ch:linux-infra/data-<environment_name>.git

Add a new file <environment_name>.yaml in to the project:

cd data-<environment_name>
touch <environment_name>.yaml
git add <environment_name>.yaml
git commit -a -m "Added first empty file"
git push

After a few seconds (needs time to trigger the change), check in puppet01.psi.ch:/srv/puppet/data/<environment_name> that file was successfully triggered (copied) to the puppet server from GIT:

ssh root@puppet01.psi.ch ls /srv/puppet/data/<environment_name>/<environment_name>.yaml

Full real example:

git clone git@git.psi.ch:linux-infra/data-meg.git
cd data-meg
touch meg.yaml
git add meg.yaml
git commit -a -m "Added first empty file"
git push
sleep 5
ssh root@puppet01.psi.ch ls /srv/puppet/data/meg/meg.yaml