Fix/ddg1 shutter delay #315

2026-09-08T20:08:30+02:00

menzel commented

2026-09-08 20:08:30 +02:00

Two independent causes, both quantified during August commissioning: the shutter was given ~2 ms to open when it needs ~8 ms, and at_each_point slept only for acc_time before firing the burst — which covers the acceleration ramp but not the scan server → device server → EPICS → controller path, so on slow scans the trigger preceded the stage. The measurements and the scans they came from are in the commit messages and the code.
Touches scans_v4/cont_grid.py as well as ddg_1.py.
Deployed at cSAXS on 2026-08-25 (f17af60, 648c0e9) and in production since; the parameters read back 0.008 / 0.6. Re-measuring the first-point deficit on a fresh scan is still outstanding.
Merged into current main: 101 failed / 546 passed / 12 skipped, against a baseline of 101 / 532 / 12 — pre-existing motor-controller failures from a local ophyd_devices skew.

Two independent causes, both quantified during August commissioning: the shutter was given ~2 ms to open when it needs ~8 ms, and at_each_point slept only for acc_time before firing the burst — which covers the acceleration ramp but not the scan server → device server → EPICS → controller path, so on slow scans the trigger preceded the stage. The measurements and the scans they came from are in the commit messages and the code. Touches scans_v4/cont_grid.py as well as ddg_1.py. Deployed at cSAXS on 2026-08-25 (f17af60, 648c0e9) and in production since; the parameters read back 0.008 / 0.6. Re-measuring the first-point deficit on a fresh scan is still outstanding. Merged into current main: 101 failed / 546 passed / 12 skipped, against a baseline of 101 / 532 / 12 — pre-existing motor-controller failures from a local ophyd_devices skew.

menzel added 2 commits 2026-09-08 20:08:30 +02:00

fix(ddg1): give the shutter the time it actually needs to open

CI for csaxs_bec / test (push) Successful in 1m55s

Details

78810999d0

The first point of every line is under-exposed. Four commissioning scans
put a number on it, against the 2e-3 head start that was in place:

    scan   exp_time   first/near   lost
     324     15 ms       0.655     5.18 ms
     254     20 ms       0.732     5.36 ms
     411     50 ms       0.881     5.93 ms
     450     50 ms       0.879     6.04 ms

A fixed time, not a fixed fraction: it varies by 17% across a 3.3x range
of exposure while the fraction varies by 2.9x. The same deficit appears
on the integrated scattering, a different detector behind a different
gate, so the cause is upstream of both readout chains rather than in
either of them. 2e-3 allowed and ~5.6e-3 still lost means the shutter
needs about 7.6 ms, rounded up to 8 ms: the spread across the four scans
is 0.9 ms, so the third digit is not meaningful, and overshooting costs
only the difference in dead time at the start of each line.

The trigger scheme was already right -- the shutter fires on cd at t0
and the acquisition on ab is held back by _shutter_to_open_delay, with
the widths, burst_period and cont_grid's acc_time and premove all
derived from it. Only the value was wrong, and it was a literal in two
places, so setting one and not the other would have been silently undone
by keep_shutter_open_during_scan.

Lifts it to DEFAULT_SHUTTER_TO_OPEN_DELAY next to the other defaults,
with the measurement recorded, and adds set_shutter_to_open_delay to
USER_ACCESS so the value can be converged from the client instead of by
redeploying the device server. It is bounded, because a delay is paid on
every line and a fat-fingered value would stretch the scan rather than
fail.

Cost at the new value is 6 ms per line -- 0.26 s over scan 450 -- and
0.8 um of extra premove.

The existing stage test asserted the 2e-3 literal and now asserts the
constant. New tests cover the default, the bound, the USER_ACCESS entry,
that a set value actually reaches the ab channel while cd still fires at
t0, and that keep_shutter_open_during_scan discards a tuned value, which
is a sharp edge worth pinning rather than leaving to be rediscovered.

fix(cont_grid): wait for the move to reach the motor before triggering

CI for csaxs_bec / test (push) Successful in 2m3s

Details

CI for csaxs_bec / test (pull_request) Successful in 1m57s

Details

3d24ea5651

at_each_point issues the line move and then sleeps before firing the
burst. The sleep was acc_time alone, which covers the acceleration ramp
and nothing else -- while the move still has to cross the scan server ->
device server -> EPICS -> controller path first. acc_time shrinks with
the scan velocity; that path does not. On a slow scan the burst
therefore began well before the stage moved, and the first points of
every line piled up at the line start.

Measured from the position readback of five commissioning scans, as the
distance the stage was behind the trigger grid once the lag stopped
growing:

    scan   exp_time   v_cmd         lag       latency
     254     20 ms    0.5   mm/s   29.7 pts   0.594 s
     450     50 ms    0.1   mm/s   11.9 pts   0.595 s
     324     15 ms    0.667 mm/s   41.5 pts   0.623 s
     473    100 ms    0.065 mm/s    6.3 pts   0.630 s
     411     50 ms    0.1   mm/s   12.7 pts   0.635 s

Constant to +-3.5% across a 6.7x range of exposure and a 10x range of
velocity, on two axes. Scan 473 is sampled 74 times per line and shows
the shape plainly: the lag appears in the first interval and then holds
at 6.3-6.4 points for the rest of the line while the velocity sits at
exactly the commanded 0.065 mm/s. A start-up offset, not a velocity
error, and the stage itself is blameless.

Deliberately not solved by polling for motion. Observing the readback
costs a round trip of this same ~0.6 s, so it would trade a systematic
offset for a jitter of similar size -- and a constant offset displaces
every line equally, where a varying one shears the image line by line
and cannot be undone afterwards. The reproducibility is the asset here,
not the enemy.

The value lives on ddg1 next to the shutter delay, with a setter in
USER_ACCESS, because that is where cont_grid already fetches its trigger
timing. It is not a property of the delay generator and the docstring
says so. Setting it to 0 reproduces exactly the previous timing without
a redeploy, which is the intended way back if this makes things worse.

The real fix is to trigger the DDG from the motor
(scan_type: hardware_triggered), taking the round trip out of the timing
chain rather than compensating for it. This is the stopgap until then,
and it rests on the latency staying constant -- which is worth
re-checking with the same measurement whenever the deployment changes.

menzel merged commit f74f09de7b into main

2026-09-09 12:30:16 +02:00

menzel deleted branch fix/ddg1-shutter-delay

2026-09-09 12:30:16 +02:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: bec/csaxs_bec#315