forked from linux/WebHosting
update readme and networks
Signed-off-by: Basil Bruhn <basil.bruhn@psi.ch>
This commit is contained in:
1 file changed
+19
-173
@@ -1,181 +1,27 @@
|
||||
# WebHosting
|
||||
|
||||
Docker Compose services and Nginx/Apache configuration for the shared web host.
|
||||
Puppet deploys the host configuration and environment files from Hiera-eyaml.
|
||||
Shared web applications behind Nginx/Apache, deployed in `/opt/WebHosting`.
|
||||
Requires Docker Compose **2.30.0 or later**.
|
||||
|
||||
## Compose layout
|
||||
- `docker-compose.yaml` combines `stacks/<app>/compose.yaml` into one Compose project.
|
||||
Run commands from the repository root; relative paths resolve from there.
|
||||
- Only Nginx publishes host ports (`80`/`443`) and joins `public` and `backend`.
|
||||
Web applications share `backend`; databases use their separate application networks.
|
||||
These are bridge networks with outbound access, without `internal: true`.
|
||||
- Puppet manages secrets through `hiera/data-lx/default/lx-docker-01.psi.ch.yaml`.
|
||||
The service env files live under `/etc/webhosting/`, separately from this repo.
|
||||
- Env files are root-owned (`0600`, directories `0700`) and use `format: raw`:
|
||||
keep values literal, without adding quotes or escaping dollar signs.
|
||||
|
||||
Requires **Docker Compose 2.30.0 or later** for raw service env files. The root
|
||||
`docker-compose.yaml` is the deployment entrypoint and assembles these modules:
|
||||
|
||||
```text
|
||||
docker-compose.yaml
|
||||
stacks/
|
||||
proxy/compose.yaml
|
||||
legacy-web/compose.yaml
|
||||
excalidraw/compose.yaml
|
||||
elog/compose.yaml
|
||||
pepisco/compose.yaml
|
||||
swiss-ecargo/compose.yaml
|
||||
hedgedoc/compose.yaml
|
||||
opticswiki/compose.yaml
|
||||
slsbl/compose.yaml
|
||||
n8n/compose.yaml
|
||||
dashboard/compose.yaml
|
||||
panda-maintenance/compose.yaml
|
||||
sap/compose.yaml
|
||||
summer-project-selection/compose.yaml
|
||||
tomcatwiki/compose.yaml
|
||||
scicat-rocrate/compose.yaml
|
||||
pikettlischte-beautifier/compose.yaml
|
||||
```
|
||||
|
||||
Each application stays with its database and supporting containers. `legacy-web`
|
||||
contains the shared Apache sites and scheduler; `elog` groups the four logbooks.
|
||||
`sap` contains the TIA, document-router, and BANF definitions. TIA and
|
||||
document-router keep their existing inline WebUI authentication entries; only
|
||||
the BANF passwords move to separate test and production env files.
|
||||
|
||||
Run Compose through the root file, using the existing deployment directory and
|
||||
project name. These modules belong to **one Compose project**; their shared
|
||||
network definitions remain in the root file. They are not standalone deployment
|
||||
entrypoints. For example, use `docker compose --env-file .env up -d excalidraw`
|
||||
from the repository root to update that service.
|
||||
|
||||
Every include sets `project_directory: .` so relative paths such as
|
||||
`./apache/conf` and the default `.env` still resolve from the repository root.
|
||||
The service names, container names, images, networks, and data paths are preserved.
|
||||
See [Docker's include documentation](https://docs.docker.com/reference/compose-file/include/).
|
||||
|
||||
## Environment configuration
|
||||
|
||||
Swiss eCargo, HedgeDoc, the dashboard, Panda Maintenance, Tomcat Wiki, and BANF read
|
||||
their credentials from service-specific files under `/etc/webhosting`. In the
|
||||
`hiera/data-lx` repository, `default/lx-docker-01.psi.ch.yaml` renders these files
|
||||
from the existing `lx-data::docker-env::*` keys. Files are root-owned with mode
|
||||
`0600`; their directories have mode `0700`. Run Compose as a deployment user
|
||||
with permission to read them.
|
||||
|
||||
| File under `/etc/webhosting/` | Service | Required non-empty variables |
|
||||
| --- | --- | --- |
|
||||
| `swiss-ecargo/app.env` | `swiss-ecargo` | `HOSTED_API_KEY` |
|
||||
| `hedgedoc/app.env` | `hedgedoc_app` | `CMD_DB_USERNAME`, `CMD_DB_PASSWORD`, `CMD_LDAP_BINDDN`, `CMD_LDAP_BINDCREDENTIALS`, `CMD_SESSION_SECRET` |
|
||||
| `hedgedoc/db.env` | `hedgedoc_db` | `POSTGRES_USER`, `POSTGRES_PASSWORD` |
|
||||
| `dashboard/backend.env` | `it-strategy-dashboard-backend` | `DB_USER`, `DB_PASS`, `JWT_SECRET`, `ADMIN_PW_HASH` |
|
||||
| `dashboard/db.env` | `it-strategy-dashboard-db` | `MYSQL_ROOT_PASSWORD`, `MYSQL_USER`, `MYSQL_PASSWORD` |
|
||||
| `panda-maintenance/app.env` | `panda-maintenance-app` | `POSTGRES_PASSWORD` |
|
||||
| `panda-maintenance/db.env` | `panda-maintenance-db` | `POSTGRES_PASSWORD` |
|
||||
| `tomcatwiki/app.env` | `tomcatwiki` | `DB_PASS` |
|
||||
| `tomcatwiki/db.env` | `tomcatwikidb` | `POSTGRES_PASSWORD` |
|
||||
| `sap/banf-test.env` | `banf-fetcher-test` | `SCP_PASSWORD` |
|
||||
| `sap/banf-prod.env` | `banf-fetcher-prod` | `SCP_PASSWORD` |
|
||||
|
||||
The BANF files reuse the existing `banf_fetcher_password_test` and
|
||||
`banf_fetcher_password_prod` Hiera keys. The `WEBAPP_SECURITY_AUTH_BASIC_USERNAME`
|
||||
and `WEBAPP_SECURITY_AUTH_BASIC_PASSWORD` entries remain inline and unchanged
|
||||
in all four TIA/document-router services.
|
||||
|
||||
Each service receives only its own file. Stable nonsecret settings remain in
|
||||
`environment:`. The files use `format: raw`: write single-line literal values
|
||||
without adding quotes or doubling dollar signs. Password hashes, quotes, and
|
||||
URI-special characters pass through unchanged. These files are not shell scripts.
|
||||
See [Docker's env-file reference](https://docs.docker.com/reference/compose-file/services/#env_file).
|
||||
|
||||
The optional `WEBHOSTING_ENV_DIR` variable overrides `/etc/webhosting`, for example
|
||||
when validating a temporary set of files. `.env.example` documents this nonsecret
|
||||
setting; it must not replace the production `.env`.
|
||||
|
||||
The existing `/opt/WebHosting/.env` remains in place for applications outside
|
||||
this repository and rollback, including the old BANF entries. Migrated services no longer reference
|
||||
its old credential variables, including overrides of those variables in the shell.
|
||||
There is no fallback to old credentials if a service env file is missing.
|
||||
|
||||
Compose requires all eleven files, including when validating the whole project for
|
||||
a targeted service update. `required: true` verifies file existence only; it does
|
||||
not enforce the keys or non-empty values listed above. Check those values as part
|
||||
of deployment. This replaces the previous `${VARIABLE:?message}` checks for the
|
||||
migrated credentials.
|
||||
|
||||
The root `.env` still supplies YAML interpolation for `WEBHOSTING_ENV_DIR`.
|
||||
It follows normal dotenv quoting rules; service
|
||||
`env_file` does not supply interpolation values to YAML. Keep credentials out of
|
||||
Git and logs. See [Docker's interpolation documentation](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/).
|
||||
|
||||
## Validate and apply configuration
|
||||
|
||||
Deploy the Hiera change first and let Puppet create all eleven service env files.
|
||||
Verify their permissions and the non-empty keys listed above before activating
|
||||
this Compose change. Preserve the existing global env file for other applications and rollback.
|
||||
|
||||
Deploy the complete `stacks/` directory alongside the root Compose file. Keep the
|
||||
existing deployment directory and Compose project name. From that directory,
|
||||
validate using the existing global env file and the new service env files:
|
||||
Example: validate the project and apply an Excalidraw update on the host:
|
||||
|
||||
```sh
|
||||
docker compose version
|
||||
docker compose --env-file .env config --quiet
|
||||
docker compose --env-file .env config --services
|
||||
cd /opt/WebHosting
|
||||
docker compose config --quiet && docker compose up -d excalidraw
|
||||
docker compose ps excalidraw
|
||||
docker compose logs --tail=50 excalidraw
|
||||
```
|
||||
|
||||
Confirm the host runs Compose 2.30.0 or later and the assembled configuration lists
|
||||
all 31 services before applying it.
|
||||
|
||||
Use `--quiet`: plain `docker compose config` and `config --environment` can print
|
||||
resolved secrets. Validation checks the Compose model; it does not test database
|
||||
credentials, application behavior, host mounts, or Nginx configuration.
|
||||
|
||||
After validation and the database checks below, use a targeted `docker compose
|
||||
--env-file .env up -d <service>...` during the application's maintenance window.
|
||||
Include its database when its definition changes, and inspect the dependency
|
||||
chain first (the dashboard database depends on the frontend copy job). Avoid a
|
||||
whole-project `down`/`up` for these changes. A plain `restart` does not apply changed
|
||||
container environment variables; see [Docker's restart documentation](https://docs.docker.com/reference/cli/docker/compose/restart/).
|
||||
|
||||
The credential source changes, but its values should match the previous
|
||||
configuration. This is not a password rotation or database upgrade. To roll back,
|
||||
restore the previous Compose revision and apply the affected services using the
|
||||
retained global `.env`. Do not remove that file after this rollout; it still has
|
||||
active consumers.
|
||||
|
||||
Application recreation can change its IP address. The current Nginx upstream
|
||||
configuration resolves static service names at startup/reload. Once the selected
|
||||
application is running, validate and reload the proxy under the existing host
|
||||
deployment procedure, then check the affected site. The proxy is shared with SAP;
|
||||
an Nginx reload affects the shared process even though its configuration is
|
||||
unchanged. See [Nginx upstream resolution](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#resolve).
|
||||
|
||||
## Database readiness and existing data
|
||||
|
||||
HedgeDoc now uses separate `CMD_DB_*` fields so passwords containing URI-special
|
||||
characters are passed literally. Its app and database env files derive their
|
||||
shared credentials from the same Hiera keys; the previous `{{ ... }}` expressions were literal strings
|
||||
to Compose. The LDAP search filter's `{{username}}` is an application placeholder
|
||||
and must remain. See [HedgeDoc configuration](https://docs.hedgedoc.org/configuration/).
|
||||
|
||||
Before applying the HedgeDoc correction to an existing instance, confirm that
|
||||
the actual database role/password match the Hiera values. PostgreSQL's
|
||||
`POSTGRES_USER` and `POSTGRES_PASSWORD` initialize an empty data directory; changing
|
||||
them does not update an existing database. Coordinate any required credential
|
||||
change with the application owner and back up the database first. Do not delete
|
||||
or reinitialize the data directory. See the [PostgreSQL image documentation](https://hub.docker.com/_/postgres).
|
||||
|
||||
HedgeDoc and Tomcat Wiki wait for PostgreSQL to accept TCP connections before
|
||||
starting. These `pg_isready` checks indicate server readiness, not successful
|
||||
application authentication. The dashboard healthcheck authenticates to its
|
||||
configured database and executes `SELECT 1`, using the existing container
|
||||
environment rather than embedding the password in the healthcheck command.
|
||||
An unhealthy status alone does not automatically restart a container.
|
||||
|
||||
HedgeDoc's PostgreSQL 13 image remains a separate migration item. Version 13 is
|
||||
unsupported, and a major upgrade requires a tested backup/restore or `pg_upgrade`
|
||||
procedure compatible with HedgeDoc. Image versions, data paths, container names,
|
||||
and networks are preserved in this configuration fix. See the
|
||||
[PostgreSQL versioning policy](https://www.postgresql.org/support/versioning/).
|
||||
|
||||
## Follow-up organization
|
||||
|
||||
After verifying the migration on the host, obsolete migrated entries (including
|
||||
the two BANF passwords) can be removed from the global `.env` in a separate Hiera
|
||||
change. Preserve values used by applications outside this repository. File secrets
|
||||
and independent Compose projects can be considered separately. Keep the supplier's
|
||||
inline WebUI authentication entries in `stacks/sap/compose.yaml` unchanged.
|
||||
Puppet must provision all required env files before Compose validation or deployment.
|
||||
Use `config --quiet` to avoid printing secrets. After an update, check the application;
|
||||
if its container IP changed, validate and reload Nginx using the host's deployment procedure.
|
||||
Reference in new issue
Block a user