# Rotation scan filename: origin to detector

Trace of how the rotation-scan file path is built and transformed, from its origin
(GUI or aaredb) down to the JFJoch `DatasetSettings` that name the file on disk.

All line numbers are from the working tree at the time of writing.

> **Naming correction.** There is no `_execute_rotation_request` in `daq.py`.
> The method both paths converge on is **`_execute_rotation_sequence`**
> (`daq.py:1431`), called from `measure_rotation` (`daq.py:2071`) and from
> `measure` (`daq.py:2654`).

---

## 1. Summary

Two independent origins produce a `RotationScanRequest.file_prefix`
(`aarecommon/models/rotation_scan.py:13`, `str | None`):

| | Origin A — manual (GUI) | Origin B — automation |
|---|---|---|
| Entry | `FilePathPanel` widgets | `AareDAQ.measure()` |
| Built in | `file_path_panel.py:145` | `daq.py:2469`, `daq.py:2647` |
| Sample data from | aaredb via `DAQStatusModel.sample` | aaredb via `SampleShortInfo` argument |
| Transport | `POST /scan/rotation` | in-process |
| Converges at | `measure_rotation` → `_execute_rotation_sequence` | `_execute_rotation_sequence` |

Both then pass unchanged through `_execute_rotation_sequence` and the rotation
service, and receive their **only server-side path transformation** inside
`jfjoch._format_dataset_settings` (`jfjoch.py:189`), which prepends a
pgroup- and scan-kind-dependent `data_folder`.

---

## 2. Origin A — manual rotation and screening from the GUI

### 2.1 Widget input (`file_path_panel.py`)

Three widgets feed the name:

- `directory_edit` — directory template, may contain `{prefix}` and macros.
  **Defaults to `"{date}/{puck}/{pos}"`** (line 64) — see §6.6, `{pos}` is never substituted
- `file_prefix_edit` — prefix template, **defaults to the literal `"{sample}"`** (line 77)
- `run_number_edit` — integer run number, 1..999

### 2.2 `update_filename()` (`file_path_panel.py:145`)

```
dir_name = directory_edit.text()
file_prefix = file_prefix_edit.text()

dir_name = dir_name.replace("{prefix}", file_prefix)      # prefix substituted into directory

base = ""            if dir_name == ""
     = dir_name      if dir_name ends with "/"
     = dir_name + "/" otherwise

base += "run" if file_prefix == "" else file_prefix        # empty prefix -> literal "run"
```

The name is then exactly what the fields say — `update_filename` does **no**
collision handling. The result is stored **without** any scan-kind component:

```python
self._filename = self._expand_macros(base, run_number)   # line 165
self.path_updated.emit(self._filename)                   # line 172
```

The comment at line 164 is explicit: *"Store the GUI's base (without applying
scan-kind transforms) for wiring into requests later."*

Collision handling lives in the click-time guard `file_path_error_box(scan_kind)`
(`file_path_panel.py:240`), which the Run buttons call. It sets `_scan_kind`,
refreshes the name for that kind, and tests it with `_exists_for_run()` →
`_effective_dataset_base()` (line 120), which checks
`<root>/<pgroup>/raw/<scan-kind>/<base>_master.h5`. On a clash it bumps the run
number by one, warns the user, and **refuses the run** (returns `False`), so the
next click re-checks. Nothing is silently skipped.

### 2.3 `_expand_macros(base, rn)` (`file_path_panel.py:106`)

Appends the zero-padded run number, then substitutes macros:

```
name = f"{base}_{rn:03d}"
```

| Macro | Replaced with |
|---|---|
| `{date}` | `self._formatted_date` (`yyyymmdd`) |
| `{sample}`, `{CrystalName}` | `self._sample_name` |
| `{puck}` | `self._puck_name` |
| `{position}` | `f"{self._puck_pos:02d}"` |
| `{sample_id}` | `self._sample_id` |
| `{beamline}` | `self._beamline` |
| `{prefix}` | `f"{puck_name}/{puck_pos:02d}/{sample_name}"` |

There is **no `{pos}` entry**, although both tooltips advertise one
(lines 69 and 82) and the default directory template uses it — see §6.6.

With an empty directory and `file_prefix_edit == "{sample}"` this yields:

```
<sample_name>_001
```

With the *shipped* defaults (`directory_edit == "{date}/{puck}/{pos}"`) it yields:

```
20260922/PUCK1/{pos}/xtal7_001
```

### 2.4 Distribution (`data_collection_settings.py:190-192`)

```python
self.file_path_panel.path_updated.connect(raster_mgr.update_filename)
self.file_path_panel.path_updated.connect(self.screening.update_filename)
self.file_path_panel.path_updated.connect(self.simple.update_filename)
```

Each consumer stores it verbatim as `self._filename`.

### 2.5 Scan-kind prefix applied at request build time

`rotation_data_collection.py:33-40`:

```python
def add_screening_to_path(path):  return "screening" / Path(path)
def add_data_to_path(path):       return "data" / Path(path)
```

| Call site | Transform | Resulting `file_prefix` |
|---|---|---|
| `rotation_data_collection.py:341` (rotation) | `add_data_to_path` | `data/<base>_001` |
| `rotation_data_collection.py:303` (screening, single image) | `add_screening_to_path` | `screening/<base>_001` |
| `rotation_data_collection.py:315` (screening, wedges) | `add_screening_to_path` | `screening/<base>_001` |
| `smart_rotation_panel.py:523` (smart rotation) | `add_data_to_path` | `data/<base>_001` |

`str()` is applied to the `PosixPath` when assigning to `file_prefix`.

### 2.6 Transport

`rotation_scan` signal → `daq_worker` → `POST /scan/rotation` → `server.py`
`rotation` endpoint → `daq.measure_rotation(val)`. The model is serialised and
revalidated as JSON; `file_prefix` is unchanged.

---

## 3. Origin B — automation (`AareDAQ.measure`)

### 3.1 Base prefix (`daq.py:2469`)

```python
formatted_date = datetime.now().strftime("%Y%m%d")
sample_prefix = f"{formatted_date}/{sample.puck_name}/{sample.pin:02d}/{sample.sample_name}"
```

`sample` is the `SampleShortInfo` passed into `measure()`, sourced from aaredb
(via TELL/spreadsheet sync). Note this is a **slash-separated directory
hierarchy**, not a flat prefix, and is date-first — a different shape from the
GUI's `<base>_<run>`.

### 3.2 Per-run token (`daq.py:2558`)

```python
hex_string = secrets.token_hex(3)     # 6 hex chars
```

One token is generated per `measure()` call and reused by **both** the raster and
the rotation request, so the two datasets share a run identifier.

### 3.3 Raster leg (`daq.py:2558-2568`)

```python
raster_params = self.get_auto_raster_params()
if raster_params.filename is not None:
    sample_prefix = f"{raster_params.filename}/{sample.sample_name}"   # reassigns sample_prefix

raster_grid = RasterGridRequest(
    file_prefix=sample_prefix + f"_{hex_string}",      # no "data/" component
    ...
)
```

### 3.4 Rotation leg (`daq.py:2640-2652`)

```python
params, source = self.get_collection_params(prefer_smart=False)
...
if params.filename is not None:
    sample_prefix = f"{params.filename}/{sample.sample_name}"

rotation_request = RotationScanRequest(
    file_prefix="data/" + sample_prefix + f"_{hex_string}",
    ...
)
```

With both override branches inert (see §6.3), the automation `file_prefix` is:

```
data/<yyyymmdd>/<puck_name>/<pin:02d>/<sample_name>_<hex6>
```

### 3.5 Where `params.filename` would come from

`get_collection_params` (`daq.py:2396`) picks one of three `SimpleScanParameters`
(`filename: str | None = None`, `models.py:699`):

| Source | Origin | Sets `filename`? |
|---|---|---|
| `spreadsheet_params()` | aaredb `sample.aaredb_params` | No |
| `smart_params` (`cfg.auto_params`) | GUI → `POST /scan/smart_params` → Redis `<BL>:auto_params` | No |
| `default_params` | `dispatch.default_rotation_parameters` | No — `{"exp_time_s", "dtz", "incr_omega_deg"}` only |

---

## 4. Shared tail: request to detector

### 4.1 `measure_rotation` (`daq.py:2051`)

Logs the request, calls `_execute_rotation_sequence`, manages beamline state.
**No path transformation.**

### 4.2 `_execute_rotation_sequence` (`daq.py:1431`)

Clamps `exp_time_s` to `>= 0.0011`, validates parameters, delegates to
`self._create_rotation_service().run(request)`. **No path transformation.**

### 4.3 Rotation service (`operations/rotation/service.py:118`)

`run()` contains no reference to `file_prefix`. **No path transformation.**

### 4.4 `jfjoch._format_dataset_settings` (`jfjoch.py:135-189`)

The only server-side transformation. `data_folder` is chosen by request type and
the `screening` flag (`jfjoch.py:152-169`):

| Request | `data_folder` |
|---|---|
| `RasterGridRequest` | `{pgroup}/raw/raster` |
| `RotationScanRequest`, `screening=True` | `{pgroup}/raw/screening` |
| `RotationScanRequest`, `screening=False` | `{pgroup}/raw` |

`pgroup` is `s.sample.user`, falling back to the hard-coded `"p16371"` when
`s.sample is None` (`jfjoch.py:143-145`).

```python
file_prefix=f"{data_folder}/{r.file_prefix}"      # jfjoch.py:189
```

### 4.5 Final values

| Path | `DatasetSettings.file_prefix` |
|---|---|
| GUI rotation | `{pgroup}/raw/data/<base>_001` |
| GUI screening | `{pgroup}/raw/screening/screening/<base>_001` |
| GUI smart rotation | `{pgroup}/raw/data/<base>_001` |
| Automation rotation | `{pgroup}/raw/data/<yyyymmdd>/<puck>/<pin>/<sample>_<hex6>` |

---

## 5. Write-back to aaredb

`aaredb.py:195` (rotation) and `aaredb.py:258` (raster):

```python
dataset = Datasets(filepath=r.file_prefix, status="written", written_at=...)
```

This records the **pre-JFJoch** value, i.e. *without* `data_folder` and without
the `/sls/mx/data` root. The database `filepath` is therefore not the path the
file was written to.

---

## 6. Findings

### 6.1 Screening prefix is applied twice

The GUI prepends `screening/` to `file_prefix`
(`rotation_data_collection.py:303`, `:315`), and JFJoch independently selects
`data_folder = {pgroup}/raw/screening` for `screening=True` (`jfjoch.py:162`).
The final prefix contains `screening` twice:

```
{pgroup}/raw/screening/screening/<base>_001
```

The GUI's own preview label and collision check
(`_effective_dataset_base`, `file_path_panel.py:120`) build
`<root>/{pgroup}/raw/screening/<base>` — one level, not two. So for screening
the displayed path, the run-number collision check, and the written path
disagree. Rotation and raster do not have this problem: the GUI contributes
`data/` where JFJoch contributes none, and contributes nothing for raster where
JFJoch contributes `raster`.

Worth confirming against a real screening dataset on disk before changing
anything, since the fix differs depending on which of the two is considered
authoritative.

### 6.2 `spreadsheet_params()` never returns a file prefix

`daq.py:1629` initialises `file_prefix = None` and no code path ever assigns it;
all four `return` statements return that `None`. The sole caller discards it:

```python
spreadsheet_params, _file_prefix = self.spreadsheet_params()   # daq.py:2397
```

The `tuple[SimpleScanParameters | None, str | None]` return type and the
underscore-prefixed unpack suggest a per-sample aaredb-driven prefix was
intended but never wired up. **There is currently no aaredb origin for the
filename** — aaredb only supplies the sample identity fields that `measure()`
formats itself.

### 6.3 Both filename override branches in `measure()` are dead

```python
if raster_params.filename is not None:   # daq.py:2562
if params.filename is not None:          # daq.py:2640
```

`raster_params` comes from `get_auto_raster_params()` → `get_beamline_default_raster_params()`
(`daq.py:2307`), which constructs `SimpleScanParameters(dtz=..., exp_time_s=..., transmission=...)`
and never sets `filename`; the aaredb overlay that follows touches only exposure,
transmission and dtz.

`params` comes from `get_collection_params()`, whose three sources all leave
`filename` at its `None` default (§3.5) — including `smart_params`, because
`SmartRotationPanel` builds its `SimpleScanParameters` at
`smart_rotation_panel.py:492` without a `filename=` argument.

So the automation prefix is always the `sample_prefix` computed at `daq.py:2469`.

### 6.4 `sample_prefix` leaks from the raster leg into the rotation leg

`sample_prefix` is a single local rebound in both legs. The raster branch at
`daq.py:2562` reassigns it to `f"{raster_params.filename}/{sample.sample_name}"`,
and that reassignment is still in effect when the rotation branch reads it at
`daq.py:2647`. If `raster_params.filename` were ever set while `params.filename`
was not, the rotation dataset would silently inherit the raster-derived prefix
and lose the date/puck/pin hierarchy.

Currently latent because both branches are dead (§6.3), but it becomes live the
moment either filename source is wired up.

### 6.5 Database path is not the written path

See §5. `Datasets.filepath` is missing `{pgroup}/raw[/screening|/raster]` and the
storage root, so it cannot be used to locate the file without re-deriving the
`data_folder` rules from `jfjoch.py:152-169`.

### 6.6 The default directory template contains a macro that is never substituted

`directory_edit` ships with the default text `"{date}/{puck}/{pos}"`
(`file_path_panel.py:64`), and `{pos}` is documented in both tooltips
(`file_path_panel.py:69`, `:82`). But `_expand_macros` (`file_path_panel.py:106-118`)
substitutes only `{position}` — there is no `{pos}` replacement.

Replaying the shipped default through the macro table gives:

```
20260922/PUCK1/{pos}/xtal7_001
```

The literal four characters `{pos}` become a directory component, so with stock
settings every manually collected dataset is written under a directory named
`{pos}` instead of the pin position. It propagates all the way to
`DatasetSettings.file_prefix` and into `Datasets.filepath` in aaredb.

Either add `name.replace("{pos}", f"{self._puck_pos:02d}")` alongside the
`{position}` line, or change the default template and both tooltips to
`{position}`. The first is the safer fix, since operators may already have
`{pos}` saved in their own templates.

### 6.7 Minor

- `clean_filename` is disabled in `daq.py:_expand_macros` (`daq.py:1625`) with a
  TODO about it stripping slashes. Nothing sanitises user-supplied text, so a
  sample name or prefix containing `/`, `..` or shell-significant characters
  propagates into the dataset path unaltered.
- `daq.py:1613` `_expand_macros` duplicates the GUI's macro table
  (`file_path_panel.py:106`) but is **called from nowhere** — the server-side copy
  is unused. The two would need to be kept in sync if it were ever wired up.
- The `"p16371"` pgroup fallback (`jfjoch.py:145`) writes into a fixed pgroup
  when sample identity is missing, rather than failing.