===================================
Deploying New Environment: Workflow
===================================

Introduction
============

Deploying a new environment requires the following:
  1. Configuring the environment in **bob**
  2.  Configuring the environment in GIT (``https://git.psi.ch/linux-infra/hiera``)

    - Environment format: ``data-<environment_name>``
    - And change permissions accordingly 
    - Configure the necessary webhooks for the Puppet and Sysdb server

  3. Configuring the environment in Puppet (e.g. ``puppet01.psi.ch``)

Configuring the environment in sysdb
------------------------------------

Bob allows to create a new environment in ``sysdb`` by using the ``bob
env`` option. You must have permissions to do that.


You must belong to the ``sysdb-admins`` group that is actually
configured on the local ``/etc/group`` file. A migration to Active
Directory should be done for that group.

To list current defined environments run::

    bob env list

To add a new environment, run::

    bob env add <environment_name> <owner> <admin_group> "<description>"

In example, for the MeG cluster::

    bob env add meg caubet_m unx-hpc_adm "MeG Cluster"
  
Test new environment in BOB
~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to test that environment was successfully created::

    bob env list | grep <environment_name>

In example::

    caubet_m@caubet-laptop:~/GIT/admin-guide/deployment$ bob env list | grep meg
    meg        caubet_m      unx-hpc_adm        MeG Cluster  

Configuring the environment in GIT
----------------------------------

Current **GIT** server at **PSI** is ``git.psi.ch``. Every new environment should in principle belong to the **linux-infra** project.

You must belong to the ``puppet_env`` Active Directory group in order to be able to create new projects.

In order to create a new environment in **GIT**, you should access and login in the following link: https://git.psi.ch/linux-infra/hiera/.
Here you can see the different environments.

The steps to create and configure a new **GIT** project are:

1. Create a new project (environment). It can be done here: https://git.psi.ch/projects/new

  - Go to ``[Blank project] Tab`` (which is the *default* tab)
      - Change ``[Project Path]`` as follows:
          - https://git.psi.ch/ + ``linux-infra/hiera``
      - Define ``[Project Name]``, which *must* have the following format: 
          - ``data-<environment_name>`` where ``<environment_name>`` is the one defined in **Bob**
      - *[Optional]* Specify ``[Project description]``
      - Specify ``[Visibility Level]``: 
          - Should be ``Internal``

2. Configure *project* permissions as follows:

  - ``[data-<environment_name]->[Settings]->[Repository]``, or directly: https://git.psi.ch/linux-infra/hiera/data-(environment_name)/-/settings/repository
      - ``[Deploy Keys]`` -> ``[Privately accessible deploy keys]`` -> select ``root@puppet01.psi.ch'`` -> click on ``'Enable'``
  - ``[data-<environment_name]->[Settings]->[Members]``, or directly: https://git.psi.ch/linux-infra/hiera/data-(environment_name)/project_members
      - Setup specific permissions for specific users or groups. In example:
          - Set project ``Master``: 
              - ``[Select members to invite]`` (``caubet_m``) + ``[Choose a role permission]`` (``Master``) + ``[Add to project]``
          - Set other roles: 
              - ``[Select members to invite]`` (``ozerov_d``) + ``[Choose a role permission]`` (``Developer``) + ``[Add to project]``
  - ``[data-<environment_name]->[Settings]->[Integrations]``, or directly: https://git.psi.ch/linux-infra/hiera/data-(environment_name)/-/hooks
      - Add WebHooks as follows:
          - ``[URL]``: http://puppet01.psi.ch/events/dataupdate
          - ``[URL]``: http://sysdb.psi.ch/events/dataupdate
          - (Checked) ``[Push events]``. Uncheck the rest.
          - ``[SSL verification]`` -> (uncheck) ``[Enable SSL verification]``
          - Confirm information from above, and click on ``[Add webhook]`` to add the new WebHook.

Test new environment in GIT
~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to test that environment was successfully created::

    git clone git@git.psi.ch:linux-infra/hiera/data-<environment_name>.git

In example::

    caubet_m@caubet-laptop:~/GIT$ git clone git@git.psi.ch:linux-infra/hiera/data-meg.git
    Cloning into 'data-meg'...
    X11 forwarding request failed
    warning: You appear to have cloned an empty repository.

Configuring the environment in Puppet server
--------------------------------------------

In [bootstrap](https://git.psi.ch/linux-infra/bootstrap) add the new repo to `instcode/puppet/puppet_server/manifests/data.pp` and run the bootstrap for the Puppet server.

Configuring the environment in Sysdb Server
-------------------------------------------

In [bootstrap](https://git.psi.ch/linux-infra/bootstrap) add the new repo to the suitable inventory file (eg. `ansible/inventory.yaml` for production) and run the playbook for the Sysdb server.

Test new environment in Puppet
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to test that environment was successfully created::

    git clone git@git.psi.ch:linux-infra/data-<environment_name>.git

Add a new file <environment_name>.yaml in to the project::

    cd data-<environment_name>
    touch <environment_name>.yaml
    git add <environment_name>.yaml
    git commit -a -m "Added first empty file"
    git push

After a few seconds (needs time to trigger the change), check in ``puppet01.psi.ch:/srv/puppet/data/<environment_name>`` that file was successfully triggered (copied) to the puppet server from **GIT**::

    ssh root@puppet01.psi.ch ls /srv/puppet/data/<environment_name>/<environment_name>.yaml

Full real example::

    git clone git@git.psi.ch:linux-infra/data-meg.git
    cd data-meg
    touch meg.yaml
    git add meg.yaml
    git commit -a -m "Added first empty file"
    git push
    sleep 5
    ssh root@puppet01.psi.ch ls /srv/puppet/data/meg/meg.yaml