diff --git a/README.md b/README.md index 925d0cd..0fa2712 100644 --- a/README.md +++ b/README.md @@ -1,181 +1,27 @@ # WebHosting -Docker Compose services and Nginx/Apache configuration for the shared web host. -Puppet deploys the host configuration and environment files from Hiera-eyaml. +Shared web applications behind Nginx/Apache, deployed in `/opt/WebHosting`. +Requires Docker Compose **2.30.0 or later**. -## Compose layout +- `docker-compose.yaml` combines `stacks//compose.yaml` into one Compose project. + Run commands from the repository root; relative paths resolve from there. +- Only Nginx publishes host ports (`80`/`443`) and joins `public` and `backend`. + Web applications share `backend`; databases use their separate application networks. + These are bridge networks with outbound access, without `internal: true`. +- Puppet manages secrets through `hiera/data-lx/default/lx-docker-01.psi.ch.yaml`. + The service env files live under `/etc/webhosting/`, separately from this repo. +- Env files are root-owned (`0600`, directories `0700`) and use `format: raw`: + keep values literal, without adding quotes or escaping dollar signs. -Requires **Docker Compose 2.30.0 or later** for raw service env files. The root -`docker-compose.yaml` is the deployment entrypoint and assembles these modules: - -```text -docker-compose.yaml -stacks/ - proxy/compose.yaml - legacy-web/compose.yaml - excalidraw/compose.yaml - elog/compose.yaml - pepisco/compose.yaml - swiss-ecargo/compose.yaml - hedgedoc/compose.yaml - opticswiki/compose.yaml - slsbl/compose.yaml - n8n/compose.yaml - dashboard/compose.yaml - panda-maintenance/compose.yaml - sap/compose.yaml - summer-project-selection/compose.yaml - tomcatwiki/compose.yaml - scicat-rocrate/compose.yaml - pikettlischte-beautifier/compose.yaml -``` - -Each application stays with its database and supporting containers. `legacy-web` -contains the shared Apache sites and scheduler; `elog` groups the four logbooks. -`sap` contains the TIA, document-router, and BANF definitions. TIA and -document-router keep their existing inline WebUI authentication entries; only -the BANF passwords move to separate test and production env files. - -Run Compose through the root file, using the existing deployment directory and -project name. These modules belong to **one Compose project**; their shared -network definitions remain in the root file. They are not standalone deployment -entrypoints. For example, use `docker compose --env-file .env up -d excalidraw` -from the repository root to update that service. - -Every include sets `project_directory: .` so relative paths such as -`./apache/conf` and the default `.env` still resolve from the repository root. -The service names, container names, images, networks, and data paths are preserved. -See [Docker's include documentation](https://docs.docker.com/reference/compose-file/include/). - -## Environment configuration - -Swiss eCargo, HedgeDoc, the dashboard, Panda Maintenance, Tomcat Wiki, and BANF read -their credentials from service-specific files under `/etc/webhosting`. In the -`hiera/data-lx` repository, `default/lx-docker-01.psi.ch.yaml` renders these files -from the existing `lx-data::docker-env::*` keys. Files are root-owned with mode -`0600`; their directories have mode `0700`. Run Compose as a deployment user -with permission to read them. - -| File under `/etc/webhosting/` | Service | Required non-empty variables | -| --- | --- | --- | -| `swiss-ecargo/app.env` | `swiss-ecargo` | `HOSTED_API_KEY` | -| `hedgedoc/app.env` | `hedgedoc_app` | `CMD_DB_USERNAME`, `CMD_DB_PASSWORD`, `CMD_LDAP_BINDDN`, `CMD_LDAP_BINDCREDENTIALS`, `CMD_SESSION_SECRET` | -| `hedgedoc/db.env` | `hedgedoc_db` | `POSTGRES_USER`, `POSTGRES_PASSWORD` | -| `dashboard/backend.env` | `it-strategy-dashboard-backend` | `DB_USER`, `DB_PASS`, `JWT_SECRET`, `ADMIN_PW_HASH` | -| `dashboard/db.env` | `it-strategy-dashboard-db` | `MYSQL_ROOT_PASSWORD`, `MYSQL_USER`, `MYSQL_PASSWORD` | -| `panda-maintenance/app.env` | `panda-maintenance-app` | `POSTGRES_PASSWORD` | -| `panda-maintenance/db.env` | `panda-maintenance-db` | `POSTGRES_PASSWORD` | -| `tomcatwiki/app.env` | `tomcatwiki` | `DB_PASS` | -| `tomcatwiki/db.env` | `tomcatwikidb` | `POSTGRES_PASSWORD` | -| `sap/banf-test.env` | `banf-fetcher-test` | `SCP_PASSWORD` | -| `sap/banf-prod.env` | `banf-fetcher-prod` | `SCP_PASSWORD` | - -The BANF files reuse the existing `banf_fetcher_password_test` and -`banf_fetcher_password_prod` Hiera keys. The `WEBAPP_SECURITY_AUTH_BASIC_USERNAME` -and `WEBAPP_SECURITY_AUTH_BASIC_PASSWORD` entries remain inline and unchanged -in all four TIA/document-router services. - -Each service receives only its own file. Stable nonsecret settings remain in -`environment:`. The files use `format: raw`: write single-line literal values -without adding quotes or doubling dollar signs. Password hashes, quotes, and -URI-special characters pass through unchanged. These files are not shell scripts. -See [Docker's env-file reference](https://docs.docker.com/reference/compose-file/services/#env_file). - -The optional `WEBHOSTING_ENV_DIR` variable overrides `/etc/webhosting`, for example -when validating a temporary set of files. `.env.example` documents this nonsecret -setting; it must not replace the production `.env`. - -The existing `/opt/WebHosting/.env` remains in place for applications outside -this repository and rollback, including the old BANF entries. Migrated services no longer reference -its old credential variables, including overrides of those variables in the shell. -There is no fallback to old credentials if a service env file is missing. - -Compose requires all eleven files, including when validating the whole project for -a targeted service update. `required: true` verifies file existence only; it does -not enforce the keys or non-empty values listed above. Check those values as part -of deployment. This replaces the previous `${VARIABLE:?message}` checks for the -migrated credentials. - -The root `.env` still supplies YAML interpolation for `WEBHOSTING_ENV_DIR`. -It follows normal dotenv quoting rules; service -`env_file` does not supply interpolation values to YAML. Keep credentials out of -Git and logs. See [Docker's interpolation documentation](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/). - -## Validate and apply configuration - -Deploy the Hiera change first and let Puppet create all eleven service env files. -Verify their permissions and the non-empty keys listed above before activating -this Compose change. Preserve the existing global env file for other applications and rollback. - -Deploy the complete `stacks/` directory alongside the root Compose file. Keep the -existing deployment directory and Compose project name. From that directory, -validate using the existing global env file and the new service env files: +Example: validate the project and apply an Excalidraw update on the host: ```sh -docker compose version -docker compose --env-file .env config --quiet -docker compose --env-file .env config --services +cd /opt/WebHosting +docker compose config --quiet && docker compose up -d excalidraw +docker compose ps excalidraw +docker compose logs --tail=50 excalidraw ``` -Confirm the host runs Compose 2.30.0 or later and the assembled configuration lists -all 31 services before applying it. - -Use `--quiet`: plain `docker compose config` and `config --environment` can print -resolved secrets. Validation checks the Compose model; it does not test database -credentials, application behavior, host mounts, or Nginx configuration. - -After validation and the database checks below, use a targeted `docker compose ---env-file .env up -d ...` during the application's maintenance window. -Include its database when its definition changes, and inspect the dependency -chain first (the dashboard database depends on the frontend copy job). Avoid a -whole-project `down`/`up` for these changes. A plain `restart` does not apply changed -container environment variables; see [Docker's restart documentation](https://docs.docker.com/reference/cli/docker/compose/restart/). - -The credential source changes, but its values should match the previous -configuration. This is not a password rotation or database upgrade. To roll back, -restore the previous Compose revision and apply the affected services using the -retained global `.env`. Do not remove that file after this rollout; it still has -active consumers. - -Application recreation can change its IP address. The current Nginx upstream -configuration resolves static service names at startup/reload. Once the selected -application is running, validate and reload the proxy under the existing host -deployment procedure, then check the affected site. The proxy is shared with SAP; -an Nginx reload affects the shared process even though its configuration is -unchanged. See [Nginx upstream resolution](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#resolve). - -## Database readiness and existing data - -HedgeDoc now uses separate `CMD_DB_*` fields so passwords containing URI-special -characters are passed literally. Its app and database env files derive their -shared credentials from the same Hiera keys; the previous `{{ ... }}` expressions were literal strings -to Compose. The LDAP search filter's `{{username}}` is an application placeholder -and must remain. See [HedgeDoc configuration](https://docs.hedgedoc.org/configuration/). - -Before applying the HedgeDoc correction to an existing instance, confirm that -the actual database role/password match the Hiera values. PostgreSQL's -`POSTGRES_USER` and `POSTGRES_PASSWORD` initialize an empty data directory; changing -them does not update an existing database. Coordinate any required credential -change with the application owner and back up the database first. Do not delete -or reinitialize the data directory. See the [PostgreSQL image documentation](https://hub.docker.com/_/postgres). - -HedgeDoc and Tomcat Wiki wait for PostgreSQL to accept TCP connections before -starting. These `pg_isready` checks indicate server readiness, not successful -application authentication. The dashboard healthcheck authenticates to its -configured database and executes `SELECT 1`, using the existing container -environment rather than embedding the password in the healthcheck command. -An unhealthy status alone does not automatically restart a container. - -HedgeDoc's PostgreSQL 13 image remains a separate migration item. Version 13 is -unsupported, and a major upgrade requires a tested backup/restore or `pg_upgrade` -procedure compatible with HedgeDoc. Image versions, data paths, container names, -and networks are preserved in this configuration fix. See the -[PostgreSQL versioning policy](https://www.postgresql.org/support/versioning/). - -## Follow-up organization - -After verifying the migration on the host, obsolete migrated entries (including -the two BANF passwords) can be removed from the global `.env` in a separate Hiera -change. Preserve values used by applications outside this repository. File secrets -and independent Compose projects can be considered separately. Keep the supplier's -inline WebUI authentication entries in `stacks/sap/compose.yaml` unchanged. +Puppet must provision all required env files before Compose validation or deployment. +Use `config --quiet` to avoid printing secrets. After an update, check the application; +if its container IP changed, validate and reload Nginx using the host's deployment procedure.