gitea-pages/admin-guide/deployment/workflow.rst

Deploying New Environment: Workflow

Introduction

Deploying a new environment requires the following:

1. Configuring the environment in bob
2. Configuring the environment in GIT (git.psi.ch)

• Environment format: data-<environment_name>
• And change permissions accordingly

3. Configuring the environment in Puppet (puppet01.psi.ch)

Configuring the environment in sysdb

Bob allows to create a new environment in sysdb by using the bob env option. You must have permissions to do that.

You must belong to the sysdb-admins group that is actually configured on the local /etc/group file. A migration to Active Directory should be done for that group.

To list current defined environments run:

bob env list

To add a new environment, run:

bob env add <environment_name> <owner> <admin_group> "<description>"

In example, for the MeG cluster:

bob env add meg caubet_m unx-hpc_adm "MeG Cluster"

Test new environment in BOB

In order to test that environment was successfully created:

bob env list | grep <environment_name>

In example:

caubet_m@caubet-laptop:~/GIT/admin-guide/deployment$ bob env list | grep meg
meg        caubet_m      unx-hpc_adm        MeG Cluster  

Configuring the environment in GIT

Current GIT server at PSI is git.psi.ch. Every new environment should in principle belong to the linux-infra project.

You must belong to the puppet_env Active Directory group in order to be able to create new projects.

In order to create a new environment in GIT, you should access and login in the following link: https://git.psi.ch/linux-infra/. Here you can see the different environments and GIT projects belonging to the linux-infra group.

The steps to create and configure a new GIT project are:

1. Create a new project (environment). It can be done here: https://git.psi.ch/projects/new

• Go to [Blank project] Tab (which is the default tab)

  • Change [Project Path] as follows:

    • https://git.psi.ch/ + linux-infra

  • Define [Project Name], which must have the following format:

    • data-<environment_name> where <environment_name> is the one defined in Bob

  • [Optional] Specify [Project description]

  • Specify [Visibility Level]:

    • Should be Internal

2. Configure project permissions as follows:

• [data-<environment_name]->[Settings]->[Repository], or directly: https://git.psi.ch/linux-infra/data-(environment_name)/settings/repository

  • [Deploy Keys] -> select root@puppet01.psi.ch' -> click on 'Enable'

• [data-<environment_name]->[Settings]->[Members], or directly: https://git.psi.ch/linux-infra/data-(environment_name)/project_members

  • Setup specific permissions for specific users or groups. In example:

    • Set project Master:

      • [Select members to invite] (caubet_m) + [Choose a role permission] (Master) + [Add to project]

    • Set other roles:

      • [Select members to invite] (ozerov_d) + [Choose a role permission] (Developer) + [Add to project]

• [data-<environment_name]->[Settings]->[Integrations], or directly: https://git.psi.ch/linux-infra/data-(environment_name)/settings/integrations

  • Add WebHook as follows:

    • [URL]: http://puppet01.psi.ch/events/dataupdate
    • (Checked) [Push events]. Uncheck the rest.
    • [SSL verification] -> (uncheck) [Enable SSL verification]
    • Confirm information from above, and click on [Add webhook] to add the new WebHook.

Test new environment in GIT

In order to test that environment was successfully created:

git clone git@git.psi.ch:linux-infra/data-<environment_name>.git

In example:

caubet_m@caubet-laptop:~/GIT$ git clone git@git.psi.ch:linux-infra/data-meg.git
Cloning into 'data-meg'...
X11 forwarding request failed
warning: You appear to have cloned an empty repository.

Configuring the environment in Puppet

Current test Puppet server is puppet00-test.psi.ch. Current production Puppet server is puppet01.psi.ch. This documentation will take as an example an environment deployed in the production server puppet01.psi.ch.

You must have root access to the Puppet server in order to be able to configure it.

Steps are:

1. Login in to the Puppet server:

   ssh root@puppet01.psi.ch

2. Clone the already created GIT project to the /srv/puppet/data path, which contains all the different projects (Bob environments):

   git clone git@git.psi.ch:linux-infra/data-<environment_name>.git /srv/puppet/data/<environment_name>
   
   # In example:
   git clone git@git.psi.ch:linux-infra/data-meg.git /srv/puppet/data/meg

3. Change permissions to plidata.puppet for the new directory /srv/puppet/data/<environment_name>:

   chown -R plidata.puppet /srv/puppet/data/<environment_name>
   
   # In example:
   chown -R plidata.puppet /srv/puppet/data/meg

At this point, everything should be configured and we can proceed to test that it works.

Test new environment in Puppet

In order to test that environment was successfully created:

git clone git@git.psi.ch:linux-infra/data-<environment_name>.git

Add a new file <environment_name>.yaml in to the project:

cd data-<environment_name>
touch <environment_name>.yaml
git add <environment_name>.yaml
git commit -a -m "Added first empty file"
git push

After a few seconds (needs time to trigger the change), check in puppet01.psi.ch:/srv/puppet/data/<environment_name> that file was successfully triggered (copied) to the puppet server from GIT:

ssh root@puppet01.psi.ch ls /srv/puppet/data/<environment_name>/<environment_name>.yaml

Full real example:

git clone git@git.psi.ch:linux-infra/data-meg.git
cd data-meg
touch meg.yaml
git add meg.yaml
git commit -a -m "Added first empty file"
git push
sleep 5
ssh root@puppet01.psi.ch ls /srv/puppet/data/meg/meg.yaml