forked from linux/documentation
200 lines
7.8 KiB
Markdown
200 lines
7.8 KiB
Markdown
# Hiera Environments
|
|
|
|
## Add New Hiera Environment
|
|
|
|
### Introduction
|
|
|
|
Deploying a new environment requires the following:
|
|
|
|
1. Configuring the environment in **bob**
|
|
2. Configuring the environment `data-<environment_name>` in GIT (https://gitea.psi.ch/puppet)
|
|
3. Configuring the environment in Puppet (e.g. `puppet.psi.ch`)
|
|
|
|
### Configuring the environment in sysdb
|
|
|
|
Bob allows to create a new environment in `sysdb` by using the `bob env` option.
|
|
Only members of the `unx-lx_eng` group have permissions to do so.
|
|
|
|
To list current defined environments run
|
|
|
|
bob env list
|
|
|
|
To add a new environment, run
|
|
|
|
bob env add <environment_name> <owner> <admin_group> "<description>"
|
|
|
|
In example, for the MeG cluster
|
|
|
|
bob env add meg caubet_m unx-hpc_adm "MeG Cluster"
|
|
|
|
#### Test new environment in BOB
|
|
|
|
In order to test that environment was successfully created
|
|
|
|
bob env list | grep <environment_name>
|
|
|
|
An example
|
|
|
|
caubet_m@caubet-laptop:~/GIT/admin-guide/deployment$ bob env list | grep meg
|
|
meg caubet_m unx-hpc_adm MeG Cluster
|
|
|
|
|
|
### Configuring the environment in GIT
|
|
|
|
You must belong to the `unx-puppet_adm` Active Directory group in order to be able to create new projects.
|
|
|
|
In order to create a new environment in **GIT**, you should access and login in the following link: https://gitea.psi.ch/puppet/.
|
|
Here you can see the different environments.
|
|
|
|
The steps to create and configure a new **GIT** project are:
|
|
|
|
1. Create a new repository (environment). It can be done here: https://gitea.psi.ch/repo/create?org=6
|
|
|
|
- Define `Repository name`, which *must* have the format `data-<environment_name>` where `<environment_name>` is the one defined in **Bob**
|
|
- `Visibility`: set checkbox `Make repository private`
|
|
- `Default Branch`: set it to `master`
|
|
|
|
2. Configure project permissions as follows:
|
|
|
|
- `data-<environment_name` -> `Settings` -> `Collaborators`, or directly: https://gitea.psi.ch/puppet/data-(environment_name)/settings/collaboration
|
|
- add teams with access
|
|
- if the team does not yet exists, map it from the AD group as documented in chapter "Mapping AD Groups to Teams" in our [Gitea Production Documentation](gitea-dmz)
|
|
- Enable already registered deploy key: `Settings` -> `Deploy Keys` -> `Add Deploy Key` with
|
|
- `Title`: `plidata`
|
|
- `Content`:
|
|
```
|
|
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTmrqGEcE+Uv88WvnROZ7hcsFeZhAClV3nV66Mq+GVc0COEICFcVYcc73BvqHN6Vgii9yE2ZIKR7fqw2Hf0JIVoBzatzxTJJwXWYKsj8Z9gJbMKy8Axgz6alorLAybRz5DZAjND46/vh8/6E0c1YOytd5d30t92d7Wa4YO6dJqrdRaXK/Li54gdDjnHgbP0McafsnuxcBI4utPD/30+o9V6RcBG7qPdHYEviX3JGYOHsbRWs36VhpOjnxpqVouNECXYTaPtAMMWlgE/bm9mLflVH3UIDn6Ufv2bxcMKd4rJwDzjWqcb047iYhfIiB6ciCrJJ4hYnywVaFnNiA0iNcf/SvQjpkZjgejGhz72IJk4A2j+nb2RIL75NxXHgOZRlQ+JO/D53Keb9jINY4NWDRm85MP2gtuy7xNxoSPylhM0JaDtkUI0uUGshACMS9yMtKPxhQUPJmxIMbm2gKSkrBPIdlHg42O81mbFYR9B/XSW8GjsKZBAaCtQMlh0Zxr5Hc=
|
|
```
|
|
- for the runner configuration check out the new Hiera data repo and add `.gitea/workflows/hiera_update.yaml` from [another one](https://gitea.psi.ch/puppet/data-lx/raw/branch/master/.gitea/workflows/hiera_update.yaml).
|
|
|
|
3. Disable unnecessary Gitea features
|
|
- In `data-<environment_name` -> `Settings`, or directly: https://gitea.psi.ch/puppet/data-(environment_name)/settings:
|
|
- untick `Enable Repository Wiki`
|
|
- untick `Enable Projects`
|
|
- untick `Enable Repository Releases`
|
|
- untick `Enable Repository Packages Registry`
|
|
- click `Update Settings`
|
|
|
|
4. Add Deployment Workflow
|
|
|
|
- Add following workflow in `.gitea/workflows/hiera_update.yaml` to the repository:
|
|
|
|
```yaml
|
|
name: hiera update
|
|
on:
|
|
push:
|
|
branches:
|
|
- master
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
puppet-deployment:
|
|
runs-on: puppet
|
|
steps:
|
|
- name: Execute hiera-update
|
|
run: |
|
|
environment="${{ github.repository }}"
|
|
echo "sudo -u plidata /opt/pli/bin/hiera-update ${environment#*data-}"
|
|
sudo -u plidata /opt/pli/bin/hiera-update ${environment#*data-}
|
|
|
|
|
|
sysdb-deployment:
|
|
runs-on: sysdb
|
|
steps:
|
|
- name: Execute hiera-update
|
|
run: |
|
|
environment="${{ github.repository }}"
|
|
echo "sudo -u plidata /opt/pli/bin/hiera-update ${environment#*data-}"
|
|
sudo -u plidata /opt/pli/bin/hiera-update ${environment#*data-}
|
|
|
|
- name: Execute hiera-extractor
|
|
run: |
|
|
environment="${{ github.repository }}"
|
|
echo "sudo -u plidata /opt/hiera-extractor/hiera-extractor ${environment#*data-}"
|
|
sudo -u plidata /opt/hiera-extractor/hiera-extractor ${environment#*data-}
|
|
```
|
|
|
|
#### Test new environment in GIT
|
|
|
|
In order to test that environment was successfully created
|
|
|
|
git clone git@gitea.psi.ch:puppet/data-<environment_name>.git
|
|
|
|
An example
|
|
|
|
$ git clone git@gitea.psi.ch:puppet/data-ems.git
|
|
Cloning into 'data-ems'...
|
|
warning: You appear to have cloned an empty repository.
|
|
|
|
|
|
### Configuring the Environment in Puppet Server and Sysdb Server
|
|
|
|
In [bootstrap](https://gitea.psi.ch/lx/bootstrap)
|
|
|
|
- add the new environment to the `sysdb_environments` list in [`ansible/group_vars/prod.yaml`](https://gitea.psi.ch/lx/bootstrap/src/branch/prod/ansible/group_vars/prod.yaml).
|
|
|
|
- To prepare the `eyaml` key go into the directory `ansible/roles/hiera_checkout/files/srv/puppet` and run
|
|
```
|
|
SYSDB_ENV=...
|
|
eyaml createkeys
|
|
mv keys/private_key.pkcs7.pem keys/${SYSDB_ENV}_private_key.pkcs7.pem
|
|
ansible-vault encrypt --vault-pass-file ../../../../../vault-pass keys/${SYSDB_ENV}_private_key.pkcs7.pem
|
|
```
|
|
and move `keys/public_key.pkcs7.pem` as `eyaml_public_key.pem` into the `data-<environment_name>` checkout and add, commit and push everything changed on both repos.
|
|
|
|
- For Puppet server run in `ansible` to deploy:
|
|
```
|
|
ansible-playbook -i inventory.yaml --vault-pass-file ./vault-pass install_puppet_server.yaml
|
|
```
|
|
|
|
- For Sysdb server run in `ansible` to deploy:
|
|
```
|
|
ansible-playbook -i inventory.yaml --vault-pass-file ./vault-pass install_sysdb.yaml
|
|
```
|
|
|
|
|
|
### Test new environment in Puppet
|
|
|
|
In order to test that environment was successfully created
|
|
|
|
git clone git@gitea.psi.ch:puppet/data-<environment_name>.git
|
|
|
|
Add a new file <environment_name>.yaml in to the project:
|
|
|
|
cd data-<environment_name>
|
|
touch <environment_name>.yaml
|
|
git add <environment_name>.yaml
|
|
git commit -a -m "Added first empty file"
|
|
git push
|
|
|
|
After a few seconds (needs time to trigger the change), check in `puppet.psi.ch:/srv/puppet/data/<environment_name>` that file was successfully triggered (copied) to the puppet server from **GIT**
|
|
|
|
ssh root@puppet.psi.ch ls /srv/puppet/data/<environment_name>/<environment_name>.yaml
|
|
|
|
Full real example:
|
|
|
|
git clone git@gitea.psi.ch:puppet/data-meg.git
|
|
cd data-meg
|
|
touch meg.yaml
|
|
git add meg.yaml
|
|
git commit -a -m "Added first empty file"
|
|
git push
|
|
sleep 5
|
|
ssh root@puppet.psi.ch ls /srv/puppet/data/meg/meg.yaml
|
|
|
|
|
|
## Remove Hiera Environment
|
|
|
|
To remove a Hiera environment following steps are required:
|
|
|
|
1. Delete Git repository for the environment (https://gitea.psi.ch/puppet/data-<environment_name>)
|
|
2. Remove the environment from bootstrap (https://gitea.psi.ch/lx/bootstrap) by deleting the environment from the `sysdb_environments` list in [`ansible/group_vars/prod.yaml`](https://gitea.psi.ch/lx/bootstrap/src/branch/prod/ansible/group_vars/prod.yaml)
|
|
3. Remove directory for the environment on Puppet and Sysdb server (i.e. puppet server: `/srv/puppet/data/<environment_name>`, sysdb server: `/srv/puppet/data/<environment_name>`)
|
|
4. Remove the environment from sysdb database
|
|
|
|
```bash
|
|
cd /var/www/sysdb
|
|
sqlite3 inv.db
|
|
DELETE FROM environments WHERE name = '<environment_name>';
|
|
```
|