Hiera
=====

Look `here <https://docs.puppet.com/hiera/3.1/>`_ for a general Hiera
introduction.

The current hierarchy has four levels (first will be considered first
during value lookup):

- nodes (FQDN)
- group (``puppet_group`` attribute in sysdb)
- sysdb environments
- common
    
and values can be stored as classical YAML values or with `encrypted yaml
<https://github.com/TomPoulton/hiera-eyaml>`_ for secrets.
The filesystem structure is as follows::

1. ``%{::sysdb_env}/%{::group}/%{::fqdn}``
2. ``%{::sysdb_env}/%{::group}``
3. ``%{::sysdb_env}/%{::sysdb_env}``
4. ``%{::environment}/data/common``

The ``%{variable}`` notation is hiera specific and each path represents a ``.yaml``
file.

Hiera repositories
------------------

Hiera data are organized in different repositories.

Sysdb environments data
^^^^^^^^^^^^^^^^^^^^^^^

Each sysdb environment has a dedicated hiera repository, called ``data-<sydbenv>``,
eg. `data-hpc <https://git.psi.ch/linux-infra/data-hpc>`_
and `data-sls <https://git.psi.ch/linux-infra/data-sls>`_.
The first three levels of the filesystem structure shown before are actually the
files inside this kind of repositories.

Any change to the repo will automatically trigger a redeployment of the new version
of its content on the puppet master within a few seconds from the push.

This choice has been made to allow groups to change their hiera data independently of
the linux infrastructure admins. Furthermore there is no way to influence other sysdb
environments data.

Common data
^^^^^^^^^^^

The last element in the hierarchy (``common.yaml``) is instead defined inside the main puppet repository
(the one containing also the real puppet code). It is important to notice that the version
of the ``common.yaml`` used for a specific host will depend on the puppet environment it
is running on, while for the sysdb environements data are the same, whatever the puppet
environment of the host.

The common part is kept under the control of the linux infrastructure admins
since a change on this can have an impact on a much larger set of hosts and all the changes
on this file are discussed and approved through a longer process.

Example
-------

Assuming two sysdb environments ``hpc`` and ``sls``, as well as:

- group ``merlin4`` in ``hpc`` with ``merlinc10`` and ``merlinc11`` in it;
- group ``merlin5`` in ``hpc`` with ``merlin-c001`` and ``merlin-c002`` in it;
- group ``mx`` in ``sls`` with ``mxcn-1`` and ``mxcn-2`` in it;
- host ``xbl-gateway`` in no explicit group (will take the implicit ``default``)

the Hiera structure would look like this::

   data/hpc/merlin4/merlinc10.psi.ch.yaml
   data/hpc/merlin4/merlinc11.psi.ch.yaml
   data/hpc/merlin4.yaml
   data/hpc/merlin5/merlin-c001.psi.ch.yaml
   data/hpc/merlin5/merlin-c002.psi.ch.yaml
   data/hpc/merlin5.yaml
   data/hpc.yaml
   data/sls/mx/mxcn-1.psi.ch.yaml
   data/sls/mx/mxcn-2.psi.ch.yaml
   data/sls/mx.yaml
   data/sls/default/xbl-gateway.psi.ch.yaml
   data/sls.yaml
   code/environments/{prod,preprod}/common.yaml

While the output of bob would be something like (some unneeded attributes have been removed)::

   merlinc10.psi.ch             hpc       local    puppet_group=merlin4
   merlinc11.psi.ch             hpc       local    puppet_group=merlin4
   merlin-c001.psi.ch           hpc       local    puppet_group=merlin5
   merlin-c002.psi.ch           hpc       local    puppet_group=merlin5
   mxcn-1.psi.ch                sls       local    puppet_group=mx
   mxcn-2.psi.ch                sls       local    puppet_group=mx
   xbl-gateway.psi.ch           sls       local    
   
Secret values
-------------

Secrets and clear-text values can be mixed inside the same yaml file, eg.::

  ntp_client::servers:
    - pstime1.psi.ch
    - pstime2.psi.ch
    - pstime3.psi.ch

  secret_key: ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAY/9V1S0VAMrRX1B4V06AgsbHPHdONFCQ4RiWfTrhV02rL5gSL4LAdqOuvGPY8YZZv8Mp06/FARlvP1aOfEx7avqSBy11IoUGkeajKZFzJV3OJsfhso4wroQ4JmfBaVKICnQZwCdpke+PHPRkwTgHcjmY2FeBnhvOlrGiQMQU3JzCjLePOa7UvlIIin3xOU/TdetzhfvoNGRhsz7+XRPD+mTT8efJ+OslJmqU7hEqMbs9CmhPJWqsjsQUp8jsM10Dk2Rv4v+zYeJd1ZLRGK3Z56G4NrlLyYua+/yyPbUP4+1bEuisDg9bfQHp3R491/kN0W558oQ+85rsRVXCp1Hb6TBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBB2x9awGQnxAJsxIHA9OiM2gCBFvgxIR4SJZPrrQ/UlhKU39yYSkEmuKE/ou+yeIe5AMA==]


The encrypted values can be decrypted transparently from Hiera (on a host having the proper hiera key)::

  [root]# hiera secret_key
  this is a secret value

You can edit secure data inside any yaml file with the command
``/opt/puppetlabs/puppet/bin/eyaml edit common.yaml``. In this case secure data
will appear in clear-text inside the editor.


Encripting data with the public key
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The eyaml public key is::

   -----BEGIN CERTIFICATE-----
   MIIC2TCCAcGgAwIBAgIBATANBgkqhkiG9w0BAQUFADAAMCAXDTE2MTAyNDE0NTY1
   N1oYDzIwNjYxMDEyMTQ1NjU3WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
   CgKCAQEA2eykSgS7VJEXrWYkQMV48ZkUVcHMbCEo2gZXD4vIJsOdJu77F7tA53Ay
   NxdKnJTftsj+R7yFP9Z2XllA9Our0Ypphj40rNstRg5O4IoSkAqitJchlfGL9jZ3
   CB4dJqFitzOkxxCWZjQpjBd3dMJc6U3us6IDWohCjYqyjMZIVwU5EflzJKV4haEy
   Y9qHkVt938RM9UohEvia5/1lZxuZQmDpYqCw9gmBK/dVKZ7abZGkujTKAg5cjD/X
   vuexLMCGrjnPdrsblwBh+yfu6cEo9nfvfj6EA0FxPHIvQ3fv1yJZ+90OA9eUJnqQ
   ED66OGPATAJIqhWlgb8a760xPQFQQQIDAQABo1wwWjAPBgNVHRMBAf8EBTADAQH/
   MB0GA1UdDgQWBBSF05r9TYDiAmkdguCVcDzmYR8Q6TAoBgNVHSMEITAfgBSF05r9
   TYDiAmkdguCVcDzmYR8Q6aEEpAIwAIIBATANBgkqhkiG9w0BAQUFAAOCAQEAWAER
   CTGsOFUkCfvqke75PmIkxKBp/2eJbavWzPkbA/mwAGS4lQc5oyS8FMkUFxATo1k/
   WIb2B3WJIMHfCzMNxTlQLjJiSyvWAlEBHDW4H2XekzKSbj96l+/nirmOq3QkEKTK
   omexF5zYSPkBVA/S2m2wae3g2kubH1p42+REKQUvt1+xaecHBYD6eXzBWChnMMnq
   FbXoayTibn0p9Roo8HClGGJpjPZUTMf+VGUqKWPfvaKl48Y0yrc/4BzZT6Sbzeou
   ZSiHwa62rTV7ia7m2SILZU5b65JUVkFH/2r6qkxCr0Ep+oaxSNXtAXLCbnXmdOeK
   B40J8ePbbmmGE24+zQ==
   -----END CERTIFICATE-----


Assuming the public key is saved in a file named ``/home/someone/eyaml_key.pub`` and that
`hiera-eyaml <https://github.com/TomPoulton/hiera-eyaml>`_ is properly installed,
a string can be encripted with::

  eyaml encrypt --pkcs7-public-key=/home/someone/eyaml_key.pub -s secret_string


While a complete file can be encripted with::

  eyaml encrypt --pkcs7-public-key=/home/someone/eyaml_key.pub -f secret_file

Example: Encripting password
----------------------------

Steps:

Install locally (local = server/desktop from where you will encrypt the password) hiera-eyaml (https://github.com/voxpupuli/hiera-eyaml/tree/command-refactor)

Create locally a *keys* directory::

   #> mkdir -p ~/eyaml/keys

Copy *puppet01:/etc/puppetlabs/keys/eyaml/public_key.pkcs7.pem* to the above folder. Alternatively, you can copy the above public key, which should be the same::

   #> scp root@puppet01:/etc/puppetlabs/keys/eyaml/public_key.pkcs7.pem ~/eyaml/keys

Go to the *eyaml* directory::

   #> cd ~/eyaml

Hash your password using *openssl* as follows**. It will generate a hashed password::

   #> openssl passwd -1
   Password: <input_password>
   Verifying - Password: <input_password>
   <output_hashed_password>

Encrypt your hashed password with *eyaml* and copy the exact output (either the string or the block) to your hiera (YAML) file::

   #> eyaml encrypt -l 'root::password' -s '<output_hashed_password>'
   root::password: ENC[PKCS7,MIIBmxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
   xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
   
   OR
   
   root::password: >
       ENC[PKCS7,MIIBmxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
       xxx]
