Hiera

Look here for a general Hiera introduction.

The current hierarchy has four levels (first will be considered first during value lookup):

nodes (FQDN)
group (puppet_group attribute in sysdb)
sysdb environments
common

and values can be stored as classical YAML values or with encrypted yaml for secrets. The filesystem structure is as follows:

%{::sysdb_env}/%{::group}/%{::fqdn}
%{::sysdb_env}/%{::group}
%{::sysdb_env}/%{::sysdb_env}
%{::environment}/data/common

The %{variable} notation is hiera specific and each path represents a .yaml file.

Hiera repositories

Hiera data are organized in different repositories.

Sysdb environments data

Each sysdb environment has a dedicated hiera repository, called data-<sydbenv>, eg. data-hpc and data-sls. The first three levels of the filesystem structure shown before are actually the files inside this kind of repositories.

Any change to the repo will automatically trigger a redeployment of the new version of its content on the puppet master within a few seconds from the push.

This choice has been made to allow groups to change their hiera data independently of the linux infrastructure admins. Furthermore there is no way to influence other sysdb environments data.

Common data

The last element in the hierarchy (common.yaml) is instead defined inside the main puppet repository (the one containing also the real puppet code). It is important to notice that the version of the common.yaml used for a specific host will depend on the puppet environment it is running on, while for the sysdb environements data are the same, whatever the puppet environment of the host.

The common part is kept under the control of the linux infrastructure admins since a change on this can have an impact on a much larger set of hosts and all the changes on this file are discussed and approved through a longer process.

Example

Assuming two sysdb environments hpc and sls, as well as:

group merlin4 in hpc with merlinc10 and merlinc11 in it;
group merlin5 in hpc with merlin-c001 and merlin-c002 in it;
group mx in sls with mxcn-1 and mxcn-2 in it;
host xbl-gateway in no explicit group (will take the implicit default)

the Hiera structure would look like this:

data/hpc/merlin4/merlinc10.psi.ch.yaml
data/hpc/merlin4/merlinc11.psi.ch.yaml
data/hpc/merlin4.yaml
data/hpc/merlin5/merlin-c001.psi.ch.yaml
data/hpc/merlin5/merlin-c002.psi.ch.yaml
data/hpc/merlin5.yaml
data/hpc.yaml
data/sls/mx/mxcn-1.psi.ch.yaml
data/sls/mx/mxcn-2.psi.ch.yaml
data/sls/mx.yaml
data/sls/default/xbl-gateway.psi.ch.yaml
data/sls.yaml
code/environments/{prod,preprod}/common.yaml

While the output of bob would be something like (some unneeded attributes have been removed):

merlinc10.psi.ch             hpc       local    puppet_group=merlin4
merlinc11.psi.ch             hpc       local    puppet_group=merlin4
merlin-c001.psi.ch           hpc       local    puppet_group=merlin5
merlin-c002.psi.ch           hpc       local    puppet_group=merlin5
mxcn-1.psi.ch                sls       local    puppet_group=mx
mxcn-2.psi.ch                sls       local    puppet_group=mx
xbl-gateway.psi.ch           sls       local

Secret values

Secrets and clear-text values can be mixed inside the same yaml file, eg.:

ntp_client::servers:
  - pstime1.psi.ch
  - pstime2.psi.ch
  - pstime3.psi.ch

secret_key: ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAY/9V1S0VAMrRX1B4V06AgsbHPHdONFCQ4RiWfTrhV02rL5gSL4LAdqOuvGPY8YZZv8Mp06/FARlvP1aOfEx7avqSBy11IoUGkeajKZFzJV3OJsfhso4wroQ4JmfBaVKICnQZwCdpke+PHPRkwTgHcjmY2FeBnhvOlrGiQMQU3JzCjLePOa7UvlIIin3xOU/TdetzhfvoNGRhsz7+XRPD+mTT8efJ+OslJmqU7hEqMbs9CmhPJWqsjsQUp8jsM10Dk2Rv4v+zYeJd1ZLRGK3Z56G4NrlLyYua+/yyPbUP4+1bEuisDg9bfQHp3R491/kN0W558oQ+85rsRVXCp1Hb6TBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBB2x9awGQnxAJsxIHA9OiM2gCBFvgxIR4SJZPrrQ/UlhKU39yYSkEmuKE/ou+yeIe5AMA==]

The encrypted values can be decrypted transparently from Hiera (on a host having the proper hiera key):

[root]# hiera secret_key
this is a secret value

You can edit secure data inside any yaml file with the command /opt/puppetlabs/puppet/bin/eyaml edit common.yaml. In this case secure data will appear in clear-text inside the editor.

Encrypting data with the public key

The eyaml public key is:

On the puppet server it found at /etc/puppetlabs/keys/eyaml/public_key.pkcs7.pem.

Then you need to have hiera-eyaml tool installed, either from the package manager of your distribution or from the source.

Assuming the public key is saved in a file (e.g. ~/eyaml_key.pub), that the file path has been put into the environment varialbe EYAML_PUB_KEY, then a string can be encripted with:

eyaml encrypt --pkcs7-public-key=$EYAML_PUB_KEY -s secret_string

While a complete file can be encripted with:

eyaml encrypt --pkcs7-public-key=$EYAML_PUB_KEY -f secret_file

Example: Encrypting password

First prepare the public key and the shell as explaned in above chapter. Then:

# openssl passwd -6 | eyaml encrypt --pkcs7-public-key=$EYAML_PUB_KEY --stdin
Password: 
Verifying - Password: 
string: ENC[PKCS7,MIIBxxxxxxxx...xxxxxxxx]

OR

block: >
  ENC[PKCS7,MIIBxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  ...
  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]
#

and place either the string or the block at the required place in your Hiera YAML.

7.1 KiB Raw Blame History