csaxs_bec/docs/user/ptychography/flomni.md

(user.ptychography.flomni)=
# flOMNI

flOMNI is an instrument for tomographic measurements via X-ray ptychography. The sample environment is at atmospheric pressure and room temperature (or higher).
An early version of the setup is described [here](https://www.dora.lib4ri.ch/psi/islandora/object/psi:12560). Nano positioning is based on closed loop control to a position signal obtained from dedicated [laser interferometry](https://www.dora.lib4ri.ch/psi/islandora/object/psi:7524). For fast scanning a combined motion of the sample (slow axis) and beam defining fresnel zone plate (fast axis) is used. The method is described [here](https://www.dora.lib4ri.ch/psi/islandora/object/psi:21021). The setup is equipped with a [heater](https://www.dora.lib4ri.ch/psi/islandora/object/psi:45302), that can be used to apply hot gas streams to the sample.
Samples are to be mounted on [OMNY pins](https://www.dora.lib4ri.ch/psi/islandora/object/psi:4528).

## How to flOMNI 

… a step-by-step guide for users

(user.ptychography.flomni.transfer)=
### Sample storage and transfer

Print the current usage of the sample storage  
`flomni.ftransfer_show_all()`

flOMNI has to know about the loaded positions and sample names. To modify use  
`flomni.ftransfer_modify_storage(<position>, 0 (=free) / 1 (=used))`  
positions `<1> .. <20>` are available slots in the tray.
Special position: `0 = sample stage`, `100 = gripper`.
You will be asked to enter a sample name. 

To load a new sample in the sample stage, in principle only one command is needed  
`flomni.ftransfer_sample_change(<position new sample>)`  
You will be asked where the previous sample should go with a suggestion for an empty position in the tray.

To remove the current sample from the sample stage __without__ mounting a new one, use  
`flomni.ftransfer_sample_change(-1)`  
You will be asked where to stow the removed sample, same as above.
Other commands:  
`ftransfer_tray_in /_out	 (not yet implemented).`

__If something goes wrong__, there are additional commands to perform a manual operation of the gripper.

| command | explanation |
| --- | --- |
| `flomni.ftransfer_get_sample(<position>)`  | Pick sample from <position> |
| `flomni.ftransfer_put_sample(<position>)`  | Mount sample at <position> |
| `flomni.ftransfer_flomni_stage_in()`       | stage to measurement position |
| `flomni.ftransfer_flomni_stage_out()`      | stage to the sample change position |

Manual operation of the gripper (do not leave it too long in open state)  
`flomni.ftransfer_gripper_open()`  
`flomni.ftransfer_gripper_close()`

Manually move the gripper to a transfer position
`flomni.ftransfer_gripper_move(<position>)`

### Alignment of samples

#### Coarse alignment

After the sample transfer the sample stage moved to the measurement position with your new sample. The Xray eye will automatically move in, the alignment GUI opens and shows a current snapshot.
If you see your sample already at the approximately correct height:

1. `flomni.xrayeye_alignment_start()` start the coarse alignment of the sample by measuring (clicking in the X-ray eye software) the sample position at its height and then angles of 0, 45, 90, 135, 180 degrees. The GUI will present a fit of this data, which is automatically loaded to BEC for aligning the sample.

Otherwise adjust the height manually:

1. `umvr_fsamy_tracked(0.01)`, attention: unit <mm>, move the sample stage relative up (positive) or down (negative) until the sample is approximately vertically centered in xray eye screen
1. `flomni.xrayeye_update_frame()` will update the current image on the xray eye screen

#### Fine alignment

After the xrayeyealign, a fine alignment needs to be performed using ptychography.
_To bypass the fine alignment: `flomni.feye_out`_


1. `flomni.tomo_parameters()` Adjust the  ptychographic scan parameters for performing an alignment scan. Typically FOVX = FOVX(Xrayeye)+20 mu, shell step = beamsize/2.5, number of projections and tomo mode are ignored in the alignment scans.
1. `flomni.tomo_alignment_scan()` perform the alignment scan. When the first scan is running, switch to a matlab session and run `BEC_ptycho_align` again. Click left and right. The third click can define the height of  the scan, but is not needed and ignored by default. The widest horizontal field of view will be printed at the end of the matlab session.
1. `flomni.read_alignment_offset()` Load alignment parameters calculated in matlab.

### Tomographic Measurement

Now that the sample is aligned, the tomographic measurement can be performed.
1. `flomni.tomo_parameters()` adjust the scan parameters for the tomographic scan. This includes the parameters for ptychographic scans of projections plus the strategy for angular sampling. The vertical shift adjusts the field of view, up (positive) or down (negative). After adjusting the numbers, type again `flomni.tomo_parameters()` and verify that they are correct. For the "8 sub-tomograms" mode, an optional 0-deg reference shot can be enabled at the start of every odd sub-tomogram and once more at the end, useful for tracking radiation damage over the course of a long measurement.
1. `flomni.tomo_scan_projection(angle)` perform a ptychographic scan at the rotation angle <angle>. Launch the tomographic measurement by `flomni.tomo_scan()`. During the scan a live progress report is printed, including an estimated time of completion. Unusually long gaps (e.g. a beamline interruption) are detected automatically and excluded from the estimate, and the total time lost to such gaps is reported once the scan finishes.
1. Before changing sample, verify that all subtomograms were completely acquired using the tomo_reconstruction matlab script.

#### Queueing multiple scans

Several parameter sets can be queued and run one after another automatically, e.g. a fast overview scan followed by a higher resolution scan on the same sample, without having to wait around to start the next one by hand.
1. `flomni.tomo_parameters()` set up the first parameter set, then `flomni.tomo_queue_add("label")` to add it to the queue.
1. Repeat for further parameter sets.
1. `flomni.tomo_queue_show()` review the queue before starting.
1. `flomni.tomo_queue_execute()` run all queued scans in sequence.

If a queued scan is interrupted, the next call to `flomni.tomo_queue_execute()` resumes it automatically rather than starting over, then continues with the remaining entries. See [Tomography](user.ptychography.flomni.tomography) below for the full command reference.

#### If something went wrong…

A __single projection__ is to be repeated use
`flomni.tomo_scan_projection(<angle>)`. The target angle of scans can be found in the second column of the file in 
`~/data/raw/logs/tomography_scannumbers.txt`

To continue an __interrupted tomography scan__, run  
`flomni.tomo_scan_resume()`  
It picks up automatically from wherever the scan was interrupted, no need to look up the exact subtomogram/angle by hand. Do not use this command when running from the tomo queuing system. Instead use `flomni.tomo_queue_execute()`.

Alternatively, the same kind of parameters used internally by `tomo_scan_resume()` can be given to `flomni.tomo_scan()` directly, e.g. to resume from a different point than where it actually stopped:
| tomo type | parameters and their defaults |
| --- | --- |
| 8 sub-tomograms | subtomo_start=1, start_angle=None |
| Golden ratio tomography (sorted in bunches) | projection_number=None |
| Equally spaced with golden starting angle | projection_number=None |

### GUI tools

During operation the BEC GUI will show the relevant cameras or progress information. To manually switch view TAB completion on 'flomni.flomnigui_' will show all options to control the GUI. Most useful
'flomni.flomnigui_show_cameras()' will show the cameras for sample transfer and interior overview
'flomni.flomnigui_show_progress()' will show the measurement progress GUI
'flomnigui_show_xeyealign()' will show the XrayEye alignment GUI


## How to setup flOMNI (software)

This part of the manual is intended for beamline staff and expert users

The nano-positioning is controlled by a feedback loop running on a real-time linux based computer. With all related hardware connected, this loop has to be started manually.

1. Login to the computer by `ssh control@mpc2680`. The password is written on the physical machine.
1. `cd OMNY/flOMNI/`
1. `./startflOMNI`

Once the loop has started, it is possible to start bec with the flOMNI configuration file.

Starting bec with session will load the scripts
`bec --session flomni`

The flOMNI scripts can be loaded manually by  
`from csaxs_bec.bec_ipython_client.plugins.flomni import Flomni`  
`flomni =  Flomni(bec)`

Loading the flOMNI configuration (this command will load the OMNY configuration only - isolated from the beamline)  
`bec.config.update_session_with_file("/bec/csaxs_bec/csaxs_bec/device_configs/flomni_config.yaml")`



If the realtime system is restarted, bec will lose communication. To restart:  
`flomni.rt_off()` … then wait a few seconds  
`flomni.rt_on()`

### Initialization of the stages

The stages of flOMNI are referenced in respect to their endswitches. The stages have to be initialized at the beginning of a run or when the Galil motor controllers have been reset or restarted. To see the status of the stages following commands are available:

Show the status of all galil controllers (all stepper motors and the UPR rotation stage)  
`dev.fsamx.controller.galil_show_all()`

The same holds true for the Smaract stages which control the OSA position.  Their status can be checked by  
`dev.fosax.controller.show_all()`

In case referencing of the flOMNI stages is required, run  
`flomni.flomni_init_stages()`  
This script will first verify that the stages are not in an initialized state, and then reference all stages in a safe way. The user will be warned in case of a potentially risky situation. This mainly involves a collision risk upstream with the exposure box exit window. It might be worth to check clearance prior to calling the init skript.

### X-ray optics alignment

The positions of the optics stages are stored as stage parameters and are thus linked to the configuration file.  
Example: The OSAx “in” position can be reviewed by `dev.fosax.user_parameter`  
Update the value by (example "fosax", "in") by `dev.fosax.update_user_parameter({"in":value})`
Important note: if these values are changed, they are not automatically stored to the config file and will only be available in the current session.
Note: The minimal position of the motor foptz is 14, and the minimal reachable distance FZP to sample is 36 mm.

`flomni.ffzp_info()` shows info about the available FZPs at the current energy of the beamline. Optional parameter is the photon _energy_ in keV.  
Example: `flomni.ffzp_info(6.2)`

Documents about availabe optics can be accessed by
`flomni.flomnigui_docs`

The [laser feedback](user.ptychography.flomni.laser_feedback) will be disabled and fine alignment lost if foptx/y are moved!

Following functions exist to move the optics in and out, with self-explaining naming.
- `flomni.ffzp_in()`
- `flomni.foptics_in()`
- `flomni.foptics_out()`
- `flomni.fosa_in()`
- `flomni.fosa_out()`

### Interferometer

The position feedback in flOMNI is controlled in closed loop to an interferometric position measurement. To show the signal of the interferometers:
`flomni.show_signal_strength_interferometer()`  
Typical values with proper alignment, sample stage at the measurement position and laser tracker running are in the range of
|    Axis   |       Value        |
| --- | --- |
|     0     |      13681.0       |
|     1     |      12383.0       |
|     2     |      10716.0       |
|     3     |      11032.0       |


#### Laser tracker commands

The horizontal interferometer is built according to the [tracking interferometer](https://www.dora.lib4ri.ch/psi/islandora/object/psi:7524). The tracker can be controlled by following commands. During commissioning of the setup it is worthy to check the status, but during general operation these commands should not be required.

- `flomni.laser_tracker_show_all()`
- `flomni.laser_tracker_on()`
- `flomni.laser_tracker_off()`

(user.ptychography.flomni.laser_feedback)=
#### Interferometer feedback commands

The closed loop control of the Piezo stages can be controlled by  
- `flomni.feedback_feedback_enable_with_reset()`.  
_There is also an enable without reset, which is used during tomography scans, when using coarse stages to increase the scan range. It should not be required to use manually._  
- `flomni.feedback_disable()`
- `flomni.feedback_status()`

### Scanning in 2D and sample alignment

#### flOMNI Fermat scan

The basic scan function can be called by `scans.flomni_fermat_scan()` and offers a detailed doc string for further details (`scans.flomni_fermat_scan?`). A prerequisite for scanning is a running feedback system. The scan has following parameters.

| Parameters |  |
| --- | --- |
| fovx (float) | Fov in the piezo plane (i.e. piezo range). Max 200 um |
| fovy (float) | Fov in the piezo plane (i.e. piezo range). Max 100 um |
| cenx (float) | center position in x |
| ceny (float) | center position in y |
| exp_time (float) | exposure time per frame |
| frames_per_trigger(int) | Number of burst frames per position |
| step (float) | stepsize |
| zshift (float) | shift in z |
| angle (float) | rotation angle (will rotate first) |
| corridor_size (float) | corridor size for the corridor optimization. Default 3 um |

Example:  
`scans.flomni_fermat_scan(fovx=20, fovy=25, cenx=0.02, ceny=0, zshift=0, angle=0, step=0.5, exp_time=0.01, frames_per_trigger=1)`

#### Overview of the alignment steps

There are several corrections applied to maintain the sample in the FOV:
1. Mirror calibration
1. X-ray eye alignment
1. Ptychography fine alignment (improvement of the X-ray eye alignment step)
1. Vertical shifts from tomography reconstruction (for very small vertical FOV)

#### XrayEye and sample alignment

The XrayEye can be moved in and out by  
`flomni.feye_in()`  
`flomni.feye_out()`

The _in_ and _out_ positions are stored as user parameters in the stage definition. Get the values by  
`dev.feyex.user_parameter`  
`dev.feyey.user_parameter`

Update the values by, example for feyex and in position,  
`dev.feyex.update_user_parameter({"in":value})`

To refresh the frame of the xray eye windows software  
`flomni.xrayeye_update_frame()`

This command can also be called to keep the shutter open and live view active
`flomni.xrayeye_update_frame(keep_shutter_open=True)`

To start the xray eye alignment (and clear any previous alignment)  
`flomni.xrayeye_alignment_start()`

This command can also be called to keep the shutter open and live view active. Warning: The dose to the sample will be significantly higher.
`flomni.xrayeye_update_frame(keep_shutter_open=True)`

To load the fit parameters from directory _dir_path_ computed by _BEC_ptycho_align.m_ in Matlab run  
`flomni.read_alignment_offset(dir_path='')`  
The loading routine uses default values for the vertical alignment. This behavior can be changed (e.g. for getting new default values) by the parameter `use_vertical_default_values=False`.

At each projection, the angular dependent is computed by  
`flomni.get_alignment_offset(angle)`, with _angle_ in degrees.

The alignment can be cleared by  
`flomni.reset_tomo_alignment_fit()`

#### Fine alignment

The alginment obtained by the X-ray eye can be refinde by recording ptychography projections at 45 deg. intervals. For this, adjust the tomo parameters by  
`flomni.tomo_parameters()`

Next, run the alignment scan by
`flomni.tomo_alignment_scan()`

Reconstruct the scan and use BEC_ptycho_align.m to obtain improved fit parameters. The new parameters can be loaded by  
`flomni.read_alignment_offset()`

For a __very__ tight vertical field of view, a fine vertical alignment based on outputs generated from early tomography reconstructions can be used. A corresponding file can be generated by the tomography reconstruction script and can be loaded by the following two methods:  
`flomni.read_additional_correction_y()`  
`flomni.read_additional_correction_y2()`  
One __important__ note: The first method is by default loading a mirror correction file automatically. If the tomogram is using that data, do not overwrite it, use the secondary correction instead.

The scan offsets are computed at each projection by  
`flomni.compute_additional_correction_y(angle)`  
`flomni.compute_additional_correction_y2(angle)`

The additional correct can be __reset__ by  
`flomni.reset_correction()`  
It will automatically load the default mirror correction file as primary correction! To reset and not load any correction, which might be useful to obtain a new default correction file, run  
`flomni.reset_correction(use_default_correction=False)`

#### Scanning of projections

At any stage of the alignment process it is possible to scan a projection.

Define the scan parameters by `flomni.tomo_parameters()`

Run a scan at _angle_ (in degrees) by `flomni.tomo_scan_projection(angle)`

(user.ptychography.flomni.tomography)=
### Tomography

The tomo parameters have to be set by  
`flomni.tomo_parameters()`

Once satisfied with the alignment, the tomography scan can be started by  
`flomni.tomo_scan()`

During the scan, a live progress report is printed (subtomogram/projection counters and an estimated time of completion). Gaps significantly longer than a normal acquisition cycle (e.g. a beamline-down interruption, or a crash followed by a restart) are detected automatically from the time between consecutive projections and excluded from the time estimate; the total time lost to such gaps is printed once the scan finishes.

Three modes for angular sampling are implemented and they have different optional parameters for the tomo_scan method:
| tomography mode | parameters and defaults |
| --- | --- |
| 8 sub-tomograms | subtomo_start=1, start_angle=None |
| Golden ratio tomography (sorted in bunches) | projection_number=None |
| Equally spaced with golden starting angle | projection_number=None |

For the "8 sub-tomograms" mode, `flomni.tomo_parameters()` also offers a `zero_deg_reference_at_each_subtomo` option: when enabled, an additional reference projection at exactly 0 degrees is acquired at the start of every odd (forward) sub-tomogram and once more after the final sub-tomogram, useful for tracking radiation damage across the full measurement.

The parameters above can be used to __restart an interrupted acquisition__ manually, or - more conveniently - by running  
`flomni.tomo_scan_resume()`  
which reads the last recorded progress and resumes automatically at the exact point (subtomogram/angle, or projection for the golden ratio modes) the scan was interrupted at, without needing to look up the values by hand. When running from the tomo scan queuing system use `flomni.tomo_queue_execute()` instead!

In case of eight equally spaced sub-tomograms, an individual sub tomogram can be scanned by flomni.sub_tomo_scan(subtomo_number, start_angle). If the start angle is not specified, it will be computed depending on the subtomo_number, which is ranging from 1 to 8.

#### Queueing multiple scans

Several tomo parameter sets can be queued and run sequentially on the same sample, without having to start each one by hand.

| command | explanation |
| --- | --- |
| `flomni.tomo_queue_add(label=None)` | Snapshot the currently set tomo parameters and add them as a new job to the queue. Returns the job's index. |
| `flomni.tomo_queue_show()` | Print and return the current queue, with status per job. |
| `flomni.tomo_queue_delete(*indices)` | Delete one or more jobs by index. |
| `flomni.tomo_queue_clear()` | Empty the queue. |
| `flomni.tomo_queue_execute(start_index=0)` | Run all pending jobs in sequence, on the current sample. |

The queue is persisted (it survives a BEC client restart). Each job's status is one of `pending`, `running`, `incomplete`, or `done`. A job that did not run to completion (an exception was caught, or the BEC client itself crashed mid-scan) is automatically resumed - rather than restarted - the next time `flomni.tomo_queue_execute()` is called.

Example:
```
flomni.tomo_parameters()                  # set up parameter set #1
flomni.tomo_queue_add("fast overview")
flomni.tomo_parameters()                  # set up parameter set #2
flomni.tomo_queue_add("hires scan")
flomni.tomo_queue_show()
flomni.tomo_queue_execute()               # runs both, in order, on this sample
```

### Sample storage and transfer

[See short version](user.ptychography.flomni.transfer)

### Heater

The [heater](https://www.dora.lib4ri.ch/psi/islandora/object/psi:45302) can be moved up and down by
flomni.move_fheater_down()
flomni.move_fheater_up()

The functions are safe in the sense that no collisions should occur. E.g. the OSA will be moved back before a movement of the heater.

__The heater still needs commissioning in BEC!!!__