health checks, validation and variable fix.

Signed-off-by: Basil Bruhn <basil.bruhn@psi.ch>

.env.example

@@ -0,0 +1,28 @@
+# Required non-SAP inputs to docker-compose.yaml.
+# Schema only: empty values deliberately fail Compose validation.
+# Puppet/Hiera owns the deployed .env. Do not replace it with this file.
+# Supplier-managed SAP/BANF settings are not included here.
+# Quote literal values containing dollar signs with single quotes in .env.
+
+# Swiss eCargo
+ecargo_api_key=
+
+# HedgeDoc (application and database use the same database credentials)
+hedgedoc_user=
+hedgedoc_password=
+bind_user=
+bind_password=
+session_secret=
+
+# IT strategy dashboard
+IT_DASHBOARD_DB_USER=
+IT_DASHBOARD_DB_PW=
+IT_DASHBOARD_DB_ROOT_PW=
+JWT_SECRET=
+ADMIN_PW_HASH=
+
+# Panda Maintenance
+PANDA_MAINTENANCE_DB_PW=
+
+# Tomcat Wiki
+TOMCATWIKI_DB_SECRET=

.gitignore

@@ -1 +1,3 @@
 .env
+.env.*
+*.env

README.md

@@ -0,0 +1,93 @@
+# WebHosting
+
+Docker Compose services and Nginx/Apache configuration for the shared web host.
+Puppet deploys the host configuration and the environment file from Hiera-eyaml.
+
+## Environment configuration
+
+The existing variable names and single Puppet-managed `.env` file remain the
+deployment interface. `.env.example` documents the required non-SAP variables;
+its empty values are intentional and it must not replace the deployed file.
+SAP/TIA, document-router, and BANF service configuration is unchanged.
+
+Non-SAP credentials use Compose's `${VARIABLE:?message}` syntax so missing or
+empty values stop configuration loading. Compose loads the whole project even
+when a command targets a single service, so provide the complete deployed env
+file for these commands too. Supplier-managed variables retain their existing
+behavior.
+
+The root `.env` supplies YAML interpolation values. Containers receive only
+variables explicitly listed in their `environment` sections. Service-level
+`env_file` is a different mechanism and does not supply YAML interpolation.
+See [Docker's interpolation documentation](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/).
+
+Preserve literal dollar signs in env-file values using single quotes, for example
+`ADMIN_PW_HASH='$2b$12$example-not-a-real-hash'`. Use proper dotenv escaping for
+embedded quotes; do not shell-source the file. Shell environment variables take
+precedence over the env file, so run deployment from a controlled environment.
+Keep actual credentials out of Git and logs. Puppet should restrict file access
+and disable diffs and plaintext file backups for secrets (`show_diff => false`,
+`backup => false`).
+
+## Validate and apply configuration
+
+From the existing deployment directory, validate using the full Puppet-managed
+environment file. Keep the deployed Compose project name and directory unchanged.
+
+```sh
+docker compose --env-file .env config --quiet
+```
+
+Use `--quiet`: plain `docker compose config` and `config --environment` can print
+resolved secrets. Validation checks the Compose model; it does not test database
+credentials, application behavior, host mounts, or Nginx configuration.
+
+After validation and the database checks below, use a targeted `docker compose
+--env-file .env up -d <service>...` during the application's maintenance window.
+Include its database when its definition changes, and inspect the dependency
+chain first (the dashboard database depends on the frontend copy job). Avoid a
+whole-project `down`/`up` for these changes. A plain `restart` does not apply changed
+container environment variables; see [Docker's restart documentation](https://docs.docker.com/reference/cli/docker/compose/restart/).
+
+Application recreation can change its IP address. The current Nginx upstream
+configuration resolves static service names at startup/reload. Once the selected
+application is running, validate and reload the proxy under the existing host
+deployment procedure, then check the affected site. The proxy is shared with SAP;
+an Nginx reload affects the shared process even though its configuration is
+unchanged. See [Nginx upstream resolution](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#resolve).
+
+## Database readiness and existing data
+
+HedgeDoc now uses separate `CMD_DB_*` fields so passwords containing URI-special
+characters are passed literally. Its database receives the same username and
+password from `.env`; the previous `{{ ... }}` expressions were literal strings
+to Compose. The LDAP search filter's `{{username}}` is an application placeholder
+and must remain. See [HedgeDoc configuration](https://docs.hedgedoc.org/configuration/).
+
+Before applying the HedgeDoc correction to an existing instance, confirm that
+the actual database role/password match the Hiera values. PostgreSQL's
+`POSTGRES_USER` and `POSTGRES_PASSWORD` initialize an empty data directory; changing
+them does not update an existing database. Coordinate any required credential
+change with the application owner and back up the database first. Do not delete
+or reinitialize the data directory. See the [PostgreSQL image documentation](https://hub.docker.com/_/postgres).
+
+HedgeDoc and Tomcat Wiki wait for PostgreSQL to accept TCP connections before
+starting. These `pg_isready` checks indicate server readiness, not successful
+application authentication. The dashboard healthcheck authenticates to its
+configured database and executes `SELECT 1`, using the existing container
+environment rather than embedding the password in the healthcheck command.
+An unhealthy status alone does not automatically restart a container.
+
+HedgeDoc's PostgreSQL 13 image remains a separate migration item. Version 13 is
+unsupported, and a major upgrade requires a tested backup/restore or `pg_upgrade`
+procedure compatible with HedgeDoc. Image versions, data paths, container names,
+and networks are preserved in this configuration fix. See the
+[PostgreSQL versioning policy](https://www.postgresql.org/support/versioning/).
+
+## Follow-up organization
+
+Split Puppet's output into service-specific env files or file secrets before
+extracting independently operated applications into their own Compose projects.
+That requires coordinated changes in the Hiera/Puppet repository. Preserve the
+supplier's SAP/TIA/document-router/BANF definitions; they can later move together
+into a dedicated directory without changing their configuration style.

docker-compose.yaml

@@ -129,7 +129,7 @@ services:
     container_name: swiss-ecargo
     restart: always
     environment:
-      - HOSTED_API_KEY=${ecargo_api_key}
+      - HOSTED_API_KEY=${ecargo_api_key:?ecargo_api_key is required}
     networks:
       - backend
 
@@ -139,26 +139,32 @@ services:
     image: quay.io/hedgedoc/hedgedoc:latest
     container_name: hedgedoc
     environment:
-      - CMD_DB_URL=postgres://${hedgedoc_user}:${hedgedoc_password}@hedgedoc_db:5432/hedgedoc
+      - CMD_DB_DIALECT=postgres
+      - CMD_DB_HOST=hedgedoc_db
+      - CMD_DB_PORT=5432
+      - CMD_DB_DATABASE=hedgedoc
+      - CMD_DB_USERNAME=${hedgedoc_user:?hedgedoc_user is required}
+      - CMD_DB_PASSWORD=${hedgedoc_password:?hedgedoc_password is required}
       - CMD_DOMAIN=hedgedoc.psi.ch
       - CMD_URL_ADDPORT=false
       - CMD_PROTOCOL_USESSL=true
       - CMD_LDAP_URL=ldaps://dc00.d.psi.ch
-      - CMD_LDAP_BINDDN=${bind_user}
-      - CMD_LDAP_BINDCREDENTIALS=${bind_password}
+      - CMD_LDAP_BINDDN=${bind_user:?bind_user is required}
+      - CMD_LDAP_BINDCREDENTIALS=${bind_password:?bind_password is required}
       - CMD_LDAP_SEARCHBASE=OU=users,OU=psi,DC=d,DC=psi,DC=ch
       - CMD_LDAP_SEARCHFILTER=(&(objectcategory=person)(objectclass=user)(|(sAMAccountName={{username}})(mail={{username}})))
       - CMD_LDAP_USERIDFIELD=sAMAccountName
-      - CMD_LDAP_PROVIDERNAME="PSI"
+      - CMD_LDAP_PROVIDERNAME=PSI
       - NODE_ENV=production
       - CMD_EMAIL=false
       - CMD_ALLOW_EMAIL_REGISTER=false
-      - CMD_SESSION_SECRET=${session_secret}
+      - CMD_SESSION_SECRET=${session_secret:?session_secret is required}
     volumes:
       - /opt/webcontent/sf-hedgedoc/uploads:/hedgedoc/public/uploads
     restart: always
     depends_on:
-      - hedgedoc_db
+      hedgedoc_db:
+        condition: service_healthy
     networks:
       - backend
       - hedgedoc_backend
@@ -168,12 +174,18 @@ services:
     image: postgres:13.4-alpine
     container_name: hedgedoc_db
     environment:
-      - POSTGRES_USER={{ hedgedoc_user }}
-      - POSTGRES_PASSWORD={{ hedgedoc_password }}
+      - POSTGRES_USER=${hedgedoc_user:?hedgedoc_user is required}
+      - POSTGRES_PASSWORD=${hedgedoc_password:?hedgedoc_password is required}
       - POSTGRES_DB=hedgedoc
     volumes:
       - /opt/webcontent/sf-hedgedoc/data:/var/lib/postgresql/data
     restart: always
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U \"$${POSTGRES_USER}\" -d \"$${POSTGRES_DB}\""]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+      start_period: 30s
     networks:
       - hedgedoc_backend
 
@@ -254,10 +266,10 @@ services:
       - DB_HOST=it-strategy-dashboard-db
       - DB_PORT=3306
       - DB_NAME=itstrategy
-      - DB_USER=${IT_DASHBOARD_DB_USER}
-      - DB_PASS=${IT_DASHBOARD_DB_PW}
-      - JWT_SECRET=${JWT_SECRET}
-      - ADMIN_PW_HASH=${ADMIN_PW_HASH}
+      - DB_USER=${IT_DASHBOARD_DB_USER:?IT_DASHBOARD_DB_USER is required}
+      - DB_PASS=${IT_DASHBOARD_DB_PW:?IT_DASHBOARD_DB_PW is required}
+      - JWT_SECRET=${JWT_SECRET:?JWT_SECRET is required}
+      - ADMIN_PW_HASH=${ADMIN_PW_HASH:?ADMIN_PW_HASH is required}
     depends_on:
       it-strategy-dashboard-db:
         condition: service_healthy
@@ -270,10 +282,10 @@ services:
     container_name: it-strategy-dashboard-db
     restart: always
     environment:
-      - MYSQL_ROOT_PASSWORD=${IT_DASHBOARD_DB_ROOT_PW}
+      - MYSQL_ROOT_PASSWORD=${IT_DASHBOARD_DB_ROOT_PW:?IT_DASHBOARD_DB_ROOT_PW is required}
       - MYSQL_DATABASE=itstrategy
-      - MYSQL_USER=${IT_DASHBOARD_DB_USER}
-      - MYSQL_PASSWORD=${IT_DASHBOARD_DB_PW}
+      - MYSQL_USER=${IT_DASHBOARD_DB_USER:?IT_DASHBOARD_DB_USER is required}
+      - MYSQL_PASSWORD=${IT_DASHBOARD_DB_PW:?IT_DASHBOARD_DB_PW is required}
     volumes:
       - /opt/webcontent/it-strategy-dashboard/mysql/data:/var/lib/mysql
     depends_on:
@@ -282,19 +294,17 @@ services:
     networks:
       - it_strategy_dashboard_backend
     healthcheck:
+      # Authenticate over TCP; do not embed the password in the command arguments.
       test:
-        [
-          "CMD",
-          "mariadb-admin",
-          "ping",
-          "-h",
-          "localhost",
-          "-u${IT_DASHBOARD_DB_USER}",
-          "-p${IT_DASHBOARD_DB_PW}",
-        ]
-      interval: 2s
+        - CMD-SHELL
+        - >-
+          MYSQL_PWD="$${MYSQL_PASSWORD}" mariadb --protocol=TCP
+          -h 127.0.0.1 -u "$${MYSQL_USER}" "$${MYSQL_DATABASE}"
+          -e 'SELECT 1' >/dev/null
+      interval: 10s
       timeout: 5s
       retries: 5
+      start_period: 30s
 
   # Christoph Hug <christoph.hug@psi.ch>, Simon Suter <simon.suter@psi.ch>
   # Panda-Maintenance (OpenMaint)
@@ -305,7 +315,7 @@ services:
       - /opt/webcontent/panda-maintenance/data:/var/lib/postgresql/data:rw
     environment:
       - POSTGRES_USER=postgres
-      - POSTGRES_PASSWORD=${PANDA_MAINTENANCE_DB_PW}
+      - POSTGRES_PASSWORD=${PANDA_MAINTENANCE_DB_PW:?PANDA_MAINTENANCE_DB_PW is required}
     restart: always
     mem_limit: 4000m
     mem_reservation: 2000m
@@ -330,7 +340,7 @@ services:
       - /opt/webcontent/panda-maintenance/docker-entrypoint.sh:/usr/local/bin/docker-entrypoint.sh:ro
     environment:
       - POSTGRES_USER=postgres
-      - POSTGRES_PASSWORD=${PANDA_MAINTENANCE_DB_PW}
+      - POSTGRES_PASSWORD=${PANDA_MAINTENANCE_DB_PW:?PANDA_MAINTENANCE_DB_PW is required}
       - POSTGRES_PORT=5432
       - POSTGRES_HOST=panda-maintenance-db
       - POSTGRES_DB=openmaint
@@ -485,10 +495,16 @@ services:
     restart: always
     environment:
       - POSTGRES_DB=wiki
-      - POSTGRES_PASSWORD=${TOMCATWIKI_DB_SECRET}
+      - POSTGRES_PASSWORD=${TOMCATWIKI_DB_SECRET:?TOMCATWIKI_DB_SECRET is required}
       - POSTGRES_USER=wiki
     volumes:
       - /opt/webcontent/tomcatwiki/db-data:/var/lib/postgresql/data
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U \"$${POSTGRES_USER}\" -d \"$${POSTGRES_DB}\""]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+      start_period: 30s
     networks:
       - tomcatwiki_backend
 
@@ -497,13 +513,14 @@ services:
     container_name: tomcatwiki
     restart: always
     depends_on:
-      - tomcatwikidb
+      tomcatwikidb:
+        condition: service_healthy
     environment:
       - DB_TYPE=postgres
       - DB_HOST=tomcatwikidb
       - DB_PORT=5432
       - DB_USER=wiki
-      - DB_PASS=${TOMCATWIKI_DB_SECRET}
+      - DB_PASS=${TOMCATWIKI_DB_SECRET:?TOMCATWIKI_DB_SECRET is required}
       - DB_NAME=wiki
     networks:
       - tomcatwiki_backend