sls/hla_framework_bd
0
1 Fork 0
Code Issues 19 Pull Requests 3 Actions Packages Projects 1 Releases Wiki Activity

refactor: centralize deploy-time substitutions (steps 1-2)

Browse Source 
Deploy-time placeholders were spread across four independent
implementations: sed in the Gitea deploy script, ansible.builtin.replace
in two playbooks, a hardcoded suffix in bin/sls_hla_launch_gui.sh, and
another in agebd/pv.py. Each defined its own values, so nothing kept them
in sync and a missed substitution failed silently.

Step 1 -- one source of truth:

  * config/environments.yml declares every deploy-time token, its value
    per environment, and where tokens are allowed to live. An environment
    that is missing a token, or defines an undeclared one, is a load-time
    error.
  * agebd.subs loads it and renders. Rendering is a strict regex over
    declared token names rather than a template engine: the rendered
    files are EPICS db/substitution files and Qt XML, and an unknown
    token must raise rather than survive into the output.
  * scripts/ci/check_subs.py enforces the contract in CI: tokens in a
    rendered path are declared, declared tokens only appear where they
    are rendered or generated, and no pre-migration delimiters survive.

Step 2 -- separate the two substitution stages by delimiter:

  * Creation-time (copier, once per new service) keeps {{ token }}.
    Deploy-time (every deploy, per environment) becomes @@token@@.
  * This drops the {{ '{{ token }}' }} escaping from templates/, which
    existed only because one Jinja pass had to emit syntax for a second.
  * Angle brackets were tried first and rejected: << >> is not
    well-formed XML, and cli parses qt/*.ui with ElementTree.
  * The existing sed and ansible consumers are repointed at the new
    delimiters so this stays behaviour-preserving. Routing them through
    agebd.subs is step 3.

Also fixes, as a side effect of having one implementation:

  * The sed pattern '{{ *agebd_env_suffix_upper* }}' quantified 'r'
    rather than the space, so it only matched one exact spacing.
  * agebd.subs.read_ioc_port compares service names case-insensitively;
    ansible/tasks/utils/get-ioc-port.yml still compares CamelCase
    registry names against a lowercase input and never matches.

qt/*.ui is declared as a render path but nothing renders it yet -- the
Qt deploy step is still a plain rsync. Wiring that up is step 4.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Benjamin LabrecqueandClaude Opus 5 committed 2026-08-20 11:26:49 +02:00
1 parent 720e260b3e
commit cfd3ecad6c
114 files changed
+1970 -780
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

cli/src/models/master_ioc_subs.py
+1 -1
View File
@@ -24,7 +24,7 @@ class MasterIocSubs(BaseModel):
Parses the configuration text and appends a new row before the closing bracket.
"""
quoted_service = f'"{service_name}"'
new_row = f' {{ "{{{{ agebd_env_suffix_upper }}}}", {quoted_service:<22} , "{starton}", "{autooff}" }}'
new_row = f' {{ "@@agebd_env_suffix_upper@@", {quoted_service:<22} , "{starton}", "{autooff}" }}'
# Use regex to find the last closing curly brace of the configuration block
# This targets the line that contains only a lone closing brace, optional spaces, and maybe a comma/semicolon.
cli/tests/fixtures/expected_output/qt/A_BD_ServiceManager.ui
Vendored
+12 -12
View File
@@ -71,7 +71,7 @@ caLabel[statusTip="Operation"]{
<string notr="true">A</string>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-GUIRELOAD</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-GUIRELOAD</string>
</property>
<property name="eventSignal">
<enum>caCalc::TriggerZeroToOne</enum>
@@ -106,7 +106,7 @@ caLabel[statusTip="Operation"]{
<string notr="true">!A</string>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-ONOFF</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-ONOFF</string>
</property>
</widget>
<widget class="QTabWidget" name="tabWidget">
@@ -586,7 +586,7 @@ caLabel[statusTip="Operation"]{
<bool>false</bool>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-ALIVE</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-ALIVE</string>
</property>
<property name="colorMode">
<enum>caLed::Alarm</enum>
@@ -602,7 +602,7 @@ caLabel[statusTip="Operation"]{
</rect>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-STARTTIME</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-STARTTIME</string>
</property>
<property name="colorMode">
<enum>caLineEdit::Default</enum>
@@ -624,7 +624,7 @@ caLabel[statusTip="Operation"]{
</rect>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-STATUS</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-STATUS</string>
</property>
<property name="colorMode">
<enum>caLineEdit::Default</enum>
@@ -646,7 +646,7 @@ caLabel[statusTip="Operation"]{
</rect>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-VERSION</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-VERSION</string>
</property>
<property name="colorMode">
<enum>caLineEdit::Default</enum>
@@ -668,7 +668,7 @@ caLabel[statusTip="Operation"]{
</rect>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-ONOFF</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-ONOFF</string>
</property>
<property name="stackingMode" stdset="0">
<enum>caChoice::Column</enum>
@@ -717,7 +717,7 @@ caLabel[statusTip="Operation"]{
<bool>false</bool>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-ONOFF-RB</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-ONOFF-RB</string>
</property>
<property name="colorMode">
<enum>caLed::Alarm</enum>
@@ -763,7 +763,7 @@ caLabel[statusTip="Operation"]{
<bool>false</bool>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-CPCL-MASTER{{ agebd_env_suffix_upper }}:STATUS</string>
<string notr="true">AGEBD-CPCL-MASTER@@agebd_env_suffix_upper@@:STATUS</string>
</property>
<property name="colorMode">
<enum>caLed::Alarm</enum>
@@ -1040,7 +1040,7 @@ caLabel[statusTip="Operation"]{
<enum>EPushButton::None</enum>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-MASTER{{ agebd_env_suffix_upper }}:MASTER-LOGS-REQ</string>
<string notr="true">AGEBD-MASTER@@agebd_env_suffix_upper@@:MASTER-LOGS-REQ</string>
</property>
<property name="label">
<string notr="true">Logs</string>
@@ -1106,7 +1106,7 @@ caLabel[statusTip="Operation"]{
<bool>false</bool>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-GUIRELOAD</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-GUIRELOAD</string>
</property>
<property name="label">
<string notr="true">Reload</string>
@@ -1172,7 +1172,7 @@ caLabel[statusTip="Operation"]{
<enum>EPushButton::None</enum>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-CPCL-MASTER{{ agebd_env_suffix_upper }}:RESTART</string>
<string notr="true">AGEBD-CPCL-MASTER@@agebd_env_suffix_upper@@:RESTART</string>
</property>
<property name="label">
<string notr="true">↺</string>
cli/tests/fixtures/expected_output/qt/A_BD_TestService99-X00.ui
Vendored
+4 -4
View File
@@ -30,7 +30,7 @@
<number>0</number>
</property>
<property name="channel" stdset="0">
<string>AGEBD-TESTSERVICE99-X00{{ agebd_env_suffix_upper }}:AO</string>
<string>AGEBD-TESTSERVICE99-X00@@agebd_env_suffix_upper@@:AO</string>
</property>
<property name="maxValue">
<double>9.000000000000000</double>
@@ -49,10 +49,10 @@
</rect>
</property>
<property name="text">
<string>Toggle AGEBD-ALH{{ agebd_env_suffix_upper }}:TESTSERVICE99-X00-ONOFF</string>
<string>Toggle AGEBD-ALH@@agebd_env_suffix_upper@@:TESTSERVICE99-X00-ONOFF</string>
</property>
<property name="channel" stdset="0">
<string>AGEBD-ALH{{ agebd_env_suffix_upper }}:TESTSERVICE99-X00-ONOFF</string>
<string>AGEBD-ALH@@agebd_env_suffix_upper@@:TESTSERVICE99-X00-ONOFF</string>
</property>
</widget>
<widget class="QLabel" name="label">
@@ -70,7 +70,7 @@
</font>
</property>
<property name="text">
<string>Value of AGEBD-TESTSERVICE99-X00{{ agebd_env_suffix_upper }}:AO</string>
<string>Value of AGEBD-TESTSERVICE99-X00@@agebd_env_suffix_upper@@:AO</string>
</property>
</widget>
</widget>
cli/tests/fixtures/expected_output/services/master/current/ioc/AGEBD-CPCL-MASTER_main.subs
Vendored
+2 -2
View File
@@ -1,5 +1,5 @@
file MASTER.template {
pattern { SUFFIX SERVICE STARTON AUTOOFF }
{ "{{ agebd_env_suffix_upper }}", "MASTER" , "0", "30" }
{ "{{ agebd_env_suffix_upper }}", "TESTSERVICE99-X00" , "0", "0" }
{ "@@agebd_env_suffix_upper@@", "MASTER" , "0", "30" }
{ "@@agebd_env_suffix_upper@@", "TESTSERVICE99-X00" , "0", "0" }
}
cli/tests/fixtures/expected_output/services/testservice99-x00/current/ioc/AGEBD-CPCL-TESTSERVICE99-X00_main.subs
Vendored
+1 -1
View File
@@ -1,4 +1,4 @@
file TESTSERVICE99-X00.template {
pattern { DEVICE }
{ AGEBD-TESTSERVICE99-X00{{ agebd_env_suffix_upper }} }
{ AGEBD-TESTSERVICE99-X00@@agebd_env_suffix_upper@@ }
}
cli/tests/fixtures/expected_output/services/testservice99-x00/current/ioc/AGEBD-CPCL-TESTSERVICE99-X00_parameters.yaml
Vendored
+2 -2
View File
@@ -1,6 +1,6 @@
cpu_architecture: x86_64
epics_version: 7.0.8
ioc_host: sls-vserv-bd-01{{ agebd_env_suffix_lower }}
ioc_port: {{ ioc_port }}
ioc_host: sls-vserv-bd-01@@agebd_env_suffix_lower@@
ioc_port: @@ioc_port@@
os: RHEL8
os_id: rhel
cli/tests/fixtures/expected_output/services/testservice99-x00/current/systemd/AGEBD-SERVICE-TESTSERVICE99-X00.service
Vendored
+2 -2
View File
@@ -3,9 +3,9 @@ Description=AGEBD-SERVICE-TESTSERVICE99-X00
After=network.target
[Service]
Environment=AGEBD_ENV={{ agebd_env }}
Environment=AGEBD_ENV=@@agebd_env@@
Environment=PYTHONUNBUFFERED=1
ExecStart=/sls/bd/hla/{{ agebd_env }}/bin/start_service.sh TESTSERVICE99-X00
ExecStart=/sls/bd/hla/@@agebd_env@@/bin/start_service.sh TESTSERVICE99-X00
Restart=no
[Install]
cli/tests/fixtures/test_repo_root/qt/A_BD_ServiceManager.ui
Vendored
+12 -12
View File
@@ -71,7 +71,7 @@ caLabel[statusTip="Operation"]{
<string notr="true">A</string>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-GUIRELOAD</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-GUIRELOAD</string>
</property>
<property name="eventSignal">
<enum>caCalc::TriggerZeroToOne</enum>
@@ -106,7 +106,7 @@ caLabel[statusTip="Operation"]{
<string notr="true">!A</string>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-ONOFF</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-ONOFF</string>
</property>
</widget>
<widget class="QTabWidget" name="tabWidget">
@@ -586,7 +586,7 @@ caLabel[statusTip="Operation"]{
<bool>false</bool>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-ALIVE</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-ALIVE</string>
</property>
<property name="colorMode">
<enum>caLed::Alarm</enum>
@@ -602,7 +602,7 @@ caLabel[statusTip="Operation"]{
</rect>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-STARTTIME</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-STARTTIME</string>
</property>
<property name="colorMode">
<enum>caLineEdit::Default</enum>
@@ -624,7 +624,7 @@ caLabel[statusTip="Operation"]{
</rect>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-STATUS</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-STATUS</string>
</property>
<property name="colorMode">
<enum>caLineEdit::Default</enum>
@@ -646,7 +646,7 @@ caLabel[statusTip="Operation"]{
</rect>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-VERSION</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-VERSION</string>
</property>
<property name="colorMode">
<enum>caLineEdit::Default</enum>
@@ -668,7 +668,7 @@ caLabel[statusTip="Operation"]{
</rect>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-ONOFF</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-ONOFF</string>
</property>
<property name="stackingMode" stdset="0">
<enum>caChoice::Column</enum>
@@ -717,7 +717,7 @@ caLabel[statusTip="Operation"]{
<bool>false</bool>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-ONOFF-RB</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-ONOFF-RB</string>
</property>
<property name="colorMode">
<enum>caLed::Alarm</enum>
@@ -763,7 +763,7 @@ caLabel[statusTip="Operation"]{
<bool>false</bool>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-CPCL-MASTER{{ agebd_env_suffix_upper }}:STATUS</string>
<string notr="true">AGEBD-CPCL-MASTER@@agebd_env_suffix_upper@@:STATUS</string>
</property>
<property name="colorMode">
<enum>caLed::Alarm</enum>
@@ -1040,7 +1040,7 @@ caLabel[statusTip="Operation"]{
<enum>EPushButton::None</enum>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-MASTER{{ agebd_env_suffix_upper }}:MASTER-LOGS-REQ</string>
<string notr="true">AGEBD-MASTER@@agebd_env_suffix_upper@@:MASTER-LOGS-REQ</string>
</property>
<property name="label">
<string notr="true">Logs</string>
@@ -1106,7 +1106,7 @@ caLabel[statusTip="Operation"]{
<bool>false</bool>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-ALH{{ agebd_env_suffix_upper }}:MASTER-GUIRELOAD</string>
<string notr="true">AGEBD-ALH@@agebd_env_suffix_upper@@:MASTER-GUIRELOAD</string>
</property>
<property name="label">
<string notr="true">Reload</string>
@@ -1172,7 +1172,7 @@ caLabel[statusTip="Operation"]{
<enum>EPushButton::None</enum>
</property>
<property name="channel" stdset="0">
<string notr="true">AGEBD-CPCL-MASTER{{ agebd_env_suffix_upper }}:RESTART</string>
<string notr="true">AGEBD-CPCL-MASTER@@agebd_env_suffix_upper@@:RESTART</string>
</property>
<property name="label">
<string notr="true">↺</string>
cli/tests/fixtures/test_repo_root/services/master/current/ioc/AGEBD-CPCL-MASTER_main.subs
Vendored
+1 -1
View File
@@ -1,4 +1,4 @@
file MASTER.template {
pattern { SUFFIX SERVICE STARTON AUTOOFF }
{ "{{ agebd_env_suffix_upper }}", "MASTER" , "0", "30" }
{ "@@agebd_env_suffix_upper@@", "MASTER" , "0", "30" }
}
cli/uv.lock
Generated
+2
View File
@@ -10,6 +10,7 @@ dependencies = [
{ name = "gitpython" },
{ name = "h5py" },
{ name = "pyepics" },
{ name = "pyyaml" },
{ name = "typer" },
]
@@ -18,6 +19,7 @@ requires-dist = [
{ name = "gitpython", specifier = ">=3.1.50" },
{ name = "h5py", specifier = ">=3.16.0" },
{ name = "pyepics", specifier = ">=3.5.10" },
{ name = "pyyaml", specifier = ">=6.0.3" },
{ name = "typer", specifier = ">=0.23.2" },
]