===================================
Deploying New Environment: Workflow
===================================

Introduction
============

Deploying a new environment requires the following:
  1. Configuring the environment in **bob**
  2.  Configuring the environment in GIT (``git.psi.ch``)

    - Environment format: ``data-<environment_name>``
    - And change permissions accordingly  

  3. Configuring the environment in Puppet (``puppet01.psi.ch``)

Configuring the environment in sysdb
------------------------------------

Bob allows to create a new environment in ``sysdb`` by using the ``bob
env`` option. You must have permissions to do that.


You must belong to the ``sysdb-admins`` group that is actually
configured on the local ``/etc/group`` file. A migration to Active
Directory should be done for that group.

To list current defined environments run::

    bob env list

To add a new environment, run::

    bob env add <environment_name> <owner> <admin_group> "<description>"

In example, for the MeG cluster::

    bob env add meg caubet_m unx-hpc_adm "MeG Cluster"
  
Test new environment in BOB
~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to test that environment was successfully created::

    bob env list | grep <environment_name>

In example::

    caubet_m@caubet-laptop:~/GIT/admin-guide/deployment$ bob env list | grep meg
    meg        caubet_m      unx-hpc_adm        MeG Cluster  

Configuring the environment in GIT
----------------------------------

Current **GIT** server at **PSI** is ``git.psi.ch``. Every new environment should in principle belong to the **linux-infra** project.

You must belong to the ``puppet_env`` Active Directory group in order to be able to create new projects.

In order to create a new environment in **GIT**, you should access and login in the following link: https://git.psi.ch/linux-infra/. Here you can see 
the different environments and **GIT** projects belonging to the **linux-infra** group.

The steps to create and configure a new **GIT** project are:

1. Create a new project (environment). It can be done here: https://git.psi.ch/projects/new

  - Go to ``[Blank project] Tab`` (which is the *default* tab)
      - Change ``[Project Path]`` as follows:
          - https://git.psi.ch/ + ``linux-infra``
      - Define ``[Project Name]``, which *must* have the following format: 
          - ``data-<environment_name>`` where ``<environment_name>`` is the one defined in **Bob**
      - *[Optional]* Specify ``[Project description]``
      - Specify ``[Visibility Level]``: 
          - Should be ``Internal``

2. Configure *project* permissions as follows:

  - ``[data-<environment_name]->[Settings]->[Repository]``, or directly: https://git.psi.ch/linux-infra/data-(environment_name)/settings/repository
      - ``[Deploy Keys]`` -> select ``root@puppet01.psi.ch'`` -> click on ``'Enable'``
  - ``[data-<environment_name]->[Settings]->[Members]``, or directly: https://git.psi.ch/linux-infra/data-(environment_name)/project_members
      - Setup specific permissions for specific users or groups. In example:
          - Set project ``Master``: 
              - ``[Select members to invite]`` (``caubet_m``) + ``[Choose a role permission]`` (``Master``) + ``[Add to project]``
          - Set other roles: 
              - ``[Select members to invite]`` (``ozerov_d``) + ``[Choose a role permission]`` (``Developer``) + ``[Add to project]``
  - ``[data-<environment_name]->[Settings]->[Integrations]``, or directly: https://git.psi.ch/linux-infra/data-(environment_name)/settings/integrations
      - Add WebHook as follows:
          - ``[URL]``: http://puppet01.psi.ch/events/dataupdate
          - (Checked) ``[Push events]``. Uncheck the rest.
          - ``[SSL verification]`` -> (uncheck) ``[Enable SSL verification]``
          - Confirm information from above, and click on ``[Add webhook]`` to add the new WebHook.

Test new environment in GIT
~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to test that environment was successfully created::

    git clone git@git.psi.ch:linux-infra/data-<environment_name>.git

In example::

    caubet_m@caubet-laptop:~/GIT$ git clone git@git.psi.ch:linux-infra/data-meg.git
    Cloning into 'data-meg'...
    X11 forwarding request failed
    warning: You appear to have cloned an empty repository.

Configuring the environment in Puppet
-------------------------------------

Current *test* **Puppet** server is ``puppet00-test.psi.ch``. Current *production* **Puppet** server is ``puppet01.psi.ch``. This documentation will take as an example an environment
deployed in the *production* server ``puppet01.psi.ch``.

You must have ``root`` access to the Puppet server in order to be able to configure it.

Steps are:

1. Login in to the Puppet server::

    ssh root@puppet01.psi.ch

2. Clone the already created **GIT** project to the ``/srv/puppet/data`` path, which contains all the different projects (Bob environments)::

    git clone git@git.psi.ch:linux-infra/data-<environment_name>.git /srv/puppet/data/<environment_name>

    # In example:
    git clone git@git.psi.ch:linux-infra/data-meg.git /srv/puppet/data/meg

3. Change permissions to ``plidata.puppet`` for the new directory ``/srv/puppet/data/<environment_name>``::

    chown -R plidata.puppet /srv/puppet/data/<environment_name>

    # In example:
    chown -R plidata.puppet /srv/puppet/data/meg

At this point, everything should be configured and we can proceed to test that it works.

Test new environment in Puppet
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to test that environment was successfully created::

    git clone git@git.psi.ch:linux-infra/data-<environment_name>.git

Add a new file <environment_name>.yaml in to the project::

    cd data-<environment_name>
    touch <environment_name>.yaml
    git add <environment_name>.yaml
    git commit -a -m "Added first empty file"
    git push

After a few seconds (needs time to trigger the change), check in ``puppet01.psi.ch:/srv/puppet/data/<environment_name>`` that file was successfully triggered (copied) to the puppet server from **GIT**::

    ssh root@puppet01.psi.ch ls /srv/puppet/data/<environment_name>/<environment_name>.yaml

Full real example::

    git clone git@git.psi.ch:linux-infra/data-meg.git
    cd data-meg
    touch meg.yaml
    git add meg.yaml
    git commit -a -m "Added first empty file"
    git push
    sleep 5
    ssh root@puppet01.psi.ch ls /srv/puppet/data/meg/meg.yaml

