AareDAQ/docs/rotation_filename_path.md

Rotation scan filename: origin to detector

Trace of how the rotation-scan file path is built and transformed, from its origin (GUI or aaredb) down to the JFJoch DatasetSettings that name the file on disk.

All line numbers are from the working tree at the time of writing.

Naming correction. There is no _execute_rotation_request in daq.py. The method both paths converge on is _execute_rotation_sequence (daq.py:1431), called from measure_rotation (daq.py:2071) and from measure (daq.py:2654).

---

1. Summary

Two independent origins produce a RotationScanRequest.file_prefix (aarecommon/models/rotation_scan.py:13, str | None):

	Origin A — manual (GUI)	Origin B — automation
Entry	FilePathPanel widgets	AareDAQ.measure()
Built in	file_path_panel.py:145	daq.py:2469, daq.py:2647
Sample data from	aaredb via DAQStatusModel.sample	aaredb via SampleShortInfo argument
Transport	POST /scan/rotation	in-process
Converges at	measure_rotation → _execute_rotation_sequence	_execute_rotation_sequence

Both then pass unchanged through _execute_rotation_sequence and the rotation service, and receive their only server-side path transformation inside jfjoch._format_dataset_settings (jfjoch.py:189), which prepends a pgroup- and scan-kind-dependent data_folder.

---

2. Origin A — manual rotation and screening from the GUI

2.1 Widget input (file_path_panel.py)

Three widgets feed the name:

• directory_edit — directory template, may contain {prefix} and macros. Defaults to "{date}/{puck}/{pos}" (line 64) — see §6.6, {pos} is never substituted
• file_prefix_edit — prefix template, defaults to the literal "{sample}" (line 77)
• run_number_edit — integer run number, 1..999

2.2 update_filename() (file_path_panel.py:145)

dir_name = directory_edit.text()
file_prefix = file_prefix_edit.text()

dir_name = dir_name.replace("{prefix}", file_prefix)      # prefix substituted into directory

base = ""            if dir_name == ""
     = dir_name      if dir_name ends with "/"
     = dir_name + "/" otherwise

base += "run" if file_prefix == "" else file_prefix        # empty prefix -> literal "run"

The name is then exactly what the fields say — update_filename does no collision handling. The result is stored without any scan-kind component:

self._filename = self._expand_macros(base, run_number)   # line 165
self.path_updated.emit(self._filename)                   # line 172

The comment at line 164 is explicit: "Store the GUI's base (without applying scan-kind transforms) for wiring into requests later."

Collision handling lives in the click-time guard file_path_error_box(scan_kind) (file_path_panel.py:240), which the Run buttons call. It sets _scan_kind, refreshes the name for that kind, and tests it with _exists_for_run() → _effective_dataset_base() (line 120), which checks <root>/<pgroup>/raw/<scan-kind>/<base>_master.h5. On a clash it bumps the run number by one, warns the user, and refuses the run (returns False), so the next click re-checks. Nothing is silently skipped.

2.3 _expand_macros(base, rn) (file_path_panel.py:106)

Appends the zero-padded run number, then substitutes macros:

name = f"{base}_{rn:03d}"

Macro	Replaced with
{date}	self._formatted_date (yyyymmdd)
{sample}, {CrystalName}	self._sample_name
{puck}	self._puck_name
{position}	f"{self._puck_pos:02d}"
{sample_id}	self._sample_id
{beamline}	self._beamline
{prefix}	f"{puck_name}/{puck_pos:02d}/{sample_name}"

There is no {pos} entry, although both tooltips advertise one (lines 69 and 82) and the default directory template uses it — see §6.6.

With an empty directory and file_prefix_edit == "{sample}" this yields:

<sample_name>_001

With the shipped defaults (directory_edit == "{date}/{puck}/{pos}") it yields:

20260922/PUCK1/{pos}/xtal7_001

2.4 Distribution (data_collection_settings.py:190-192)

self.file_path_panel.path_updated.connect(raster_mgr.update_filename)
self.file_path_panel.path_updated.connect(self.screening.update_filename)
self.file_path_panel.path_updated.connect(self.simple.update_filename)

Each consumer stores it verbatim as self._filename.

2.5 Scan-kind prefix applied at request build time

rotation_data_collection.py:33-40:

def add_screening_to_path(path):  return "screening" / Path(path)
def add_data_to_path(path):       return "data" / Path(path)

Call site	Transform	Resulting file_prefix
rotation_data_collection.py:341 (rotation)	add_data_to_path	data/<base>_001
rotation_data_collection.py:303 (screening, single image)	add_screening_to_path	screening/<base>_001
rotation_data_collection.py:315 (screening, wedges)	add_screening_to_path	screening/<base>_001
smart_rotation_panel.py:523 (smart rotation)	add_data_to_path	data/<base>_001

str() is applied to the PosixPath when assigning to file_prefix.

2.6 Transport

rotation_scan signal → daq_worker → POST /scan/rotation → server.py rotation endpoint → daq.measure_rotation(val). The model is serialised and revalidated as JSON; file_prefix is unchanged.

---

3. Origin B — automation (AareDAQ.measure)

3.1 Base prefix (daq.py:2469)

formatted_date = datetime.now().strftime("%Y%m%d")
sample_prefix = f"{formatted_date}/{sample.puck_name}/{sample.pin:02d}/{sample.sample_name}"

sample is the SampleShortInfo passed into measure(), sourced from aaredb (via TELL/spreadsheet sync). Note this is a slash-separated directory hierarchy, not a flat prefix, and is date-first — a different shape from the GUI's <base>_<run>.

3.2 Per-run token (daq.py:2558)

hex_string = secrets.token_hex(3)     # 6 hex chars

One token is generated per measure() call and reused by both the raster and the rotation request, so the two datasets share a run identifier.

3.3 Raster leg (daq.py:2558-2568)

raster_params = self.get_auto_raster_params()
if raster_params.filename is not None:
    sample_prefix = f"{raster_params.filename}/{sample.sample_name}"   # reassigns sample_prefix

raster_grid = RasterGridRequest(
    file_prefix=sample_prefix + f"_{hex_string}",      # no "data/" component
    ...
)

3.4 Rotation leg (daq.py:2640-2652)

params, source = self.get_collection_params(prefer_smart=False)
...
if params.filename is not None:
    sample_prefix = f"{params.filename}/{sample.sample_name}"

rotation_request = RotationScanRequest(
    file_prefix="data/" + sample_prefix + f"_{hex_string}",
    ...
)

With both override branches inert (see §6.3), the automation file_prefix is:

data/<yyyymmdd>/<puck_name>/<pin:02d>/<sample_name>_<hex6>

3.5 Where params.filename would come from

get_collection_params (daq.py:2396) picks one of three SimpleScanParameters (filename: str | None = None, models.py:699):

Source	Origin	Sets filename?
spreadsheet_params()	aaredb sample.aaredb_params	No
smart_params (cfg.auto_params)	GUI → POST /scan/smart_params → Redis <BL>:auto_params	No
default_params	dispatch.default_rotation_parameters	No — {"exp_time_s", "dtz", "incr_omega_deg"} only

---

4. Shared tail: request to detector

4.1 measure_rotation (daq.py:2051)

Logs the request, calls _execute_rotation_sequence, manages beamline state. No path transformation.

4.2 _execute_rotation_sequence (daq.py:1431)

Clamps exp_time_s to >= 0.0011, validates parameters, delegates to self._create_rotation_service().run(request). No path transformation.

4.3 Rotation service (operations/rotation/service.py:118)

run() contains no reference to file_prefix. No path transformation.

4.4 jfjoch._format_dataset_settings (jfjoch.py:135-189)

The only server-side transformation. data_folder is chosen by request type and the screening flag (jfjoch.py:152-169):

Request	data_folder
RasterGridRequest	{pgroup}/raw/raster
RotationScanRequest, screening=True	{pgroup}/raw/screening
RotationScanRequest, screening=False	{pgroup}/raw

pgroup is s.sample.user, falling back to the hard-coded "p16371" when s.sample is None (jfjoch.py:143-145).

file_prefix=f"{data_folder}/{r.file_prefix}"      # jfjoch.py:189

4.5 Final values

Path	DatasetSettings.file_prefix
GUI rotation	{pgroup}/raw/data/<base>_001
GUI screening	{pgroup}/raw/screening/screening/<base>_001
GUI smart rotation	{pgroup}/raw/data/<base>_001
Automation rotation	{pgroup}/raw/data/<yyyymmdd>/<puck>/<pin>/<sample>_<hex6>

---

5. Write-back to aaredb

aaredb.py:195 (rotation) and aaredb.py:258 (raster):

dataset = Datasets(filepath=r.file_prefix, status="written", written_at=...)

This records the pre-JFJoch value, i.e. without data_folder and without the /sls/mx/data root. The database filepath is therefore not the path the file was written to.

---

6. Findings

6.1 Screening prefix is applied twice

The GUI prepends screening/ to file_prefix (rotation_data_collection.py:303, :315), and JFJoch independently selects data_folder = {pgroup}/raw/screening for screening=True (jfjoch.py:162). The final prefix contains screening twice:

{pgroup}/raw/screening/screening/<base>_001

The GUI's own preview label and collision check (_effective_dataset_base, file_path_panel.py:120) build <root>/{pgroup}/raw/screening/<base> — one level, not two. So for screening the displayed path, the run-number collision check, and the written path disagree. Rotation and raster do not have this problem: the GUI contributes data/ where JFJoch contributes none, and contributes nothing for raster where JFJoch contributes raster.

Worth confirming against a real screening dataset on disk before changing anything, since the fix differs depending on which of the two is considered authoritative.

6.2 spreadsheet_params() never returns a file prefix

daq.py:1629 initialises file_prefix = None and no code path ever assigns it; all four return statements return that None. The sole caller discards it:

spreadsheet_params, _file_prefix = self.spreadsheet_params()   # daq.py:2397

The tuple[SimpleScanParameters | None, str | None] return type and the underscore-prefixed unpack suggest a per-sample aaredb-driven prefix was intended but never wired up. There is currently no aaredb origin for the filename — aaredb only supplies the sample identity fields that measure() formats itself.

6.3 Both filename override branches in measure() are dead

if raster_params.filename is not None:   # daq.py:2562
if params.filename is not None:          # daq.py:2640

raster_params comes from get_auto_raster_params() → get_beamline_default_raster_params() (daq.py:2307), which constructs SimpleScanParameters(dtz=..., exp_time_s=..., transmission=...) and never sets filename; the aaredb overlay that follows touches only exposure, transmission and dtz.

params comes from get_collection_params(), whose three sources all leave filename at its None default (§3.5) — including smart_params, because SmartRotationPanel builds its SimpleScanParameters at smart_rotation_panel.py:492 without a filename= argument.

So the automation prefix is always the sample_prefix computed at daq.py:2469.

6.4 sample_prefix leaks from the raster leg into the rotation leg

sample_prefix is a single local rebound in both legs. The raster branch at daq.py:2562 reassigns it to f"{raster_params.filename}/{sample.sample_name}", and that reassignment is still in effect when the rotation branch reads it at daq.py:2647. If raster_params.filename were ever set while params.filename was not, the rotation dataset would silently inherit the raster-derived prefix and lose the date/puck/pin hierarchy.

Currently latent because both branches are dead (§6.3), but it becomes live the moment either filename source is wired up.

6.5 Database path is not the written path

See §5. Datasets.filepath is missing {pgroup}/raw[/screening|/raster] and the storage root, so it cannot be used to locate the file without re-deriving the data_folder rules from jfjoch.py:152-169.

6.6 The default directory template contains a macro that is never substituted

directory_edit ships with the default text "{date}/{puck}/{pos}" (file_path_panel.py:64), and {pos} is documented in both tooltips (file_path_panel.py:69, :82). But _expand_macros (file_path_panel.py:106-118) substitutes only {position} — there is no {pos} replacement.

Replaying the shipped default through the macro table gives:

20260922/PUCK1/{pos}/xtal7_001

The literal four characters {pos} become a directory component, so with stock settings every manually collected dataset is written under a directory named {pos} instead of the pin position. It propagates all the way to DatasetSettings.file_prefix and into Datasets.filepath in aaredb.

Either add name.replace("{pos}", f"{self._puck_pos:02d}") alongside the {position} line, or change the default template and both tooltips to {position}. The first is the safer fix, since operators may already have {pos} saved in their own templates.

6.7 Minor

• clean_filename is disabled in daq.py:_expand_macros (daq.py:1625) with a TODO about it stripping slashes. Nothing sanitises user-supplied text, so a sample name or prefix containing /, .. or shell-significant characters propagates into the dataset path unaltered.
• daq.py:1613 _expand_macros duplicates the GUI's macro table (file_path_panel.py:106) but is called from nowhere — the server-side copy is unused. The two would need to be kept in sync if it were ever wired up.
• The "p16371" pgroup fallback (jfjoch.py:145) writes into a fixed pgroup when sample identity is missing, rather than failing.