Streak Finder algorithm for CBD experiment #2

Merged
augustin_s merged 46 commits from ext-dorofe_e/dap:chapman into main 2025-07-14 11:18:07 +02:00
Showing only changes of commit 8e4301c443 - Show all commits

View File

@@ -110,16 +110,31 @@ options:
This algorithm is using [streak-finder package](https://github.com/simply-nicky/streak_finder) - a connection-based streak finding algorithm for convergent beam diffraction patterns.
Input parameters:
* `'do_snr': 1/0` - Specifies whether to do whitefield and std correction, if selected - changes image and mask in place.
* `'std_data_file': str` - hdf5 data file containing pre-calculated std.
* `'std_dataset': str` - [optional] dataset containing pre-calculated std, defaults to `'entry/crystallography/std'`.
* `'whitefield_data_file': str` - hdf5 data file containing pre-calculated white field.
* `'whitefield_dataset': str` - [optional] dataset containing pre-calculated white field, defaults to `'entry/crystallography/whitefield'`.
* `'mask_data_file': str` - hdf5 data file containing pre-calculated mask.
* `'mask_dataset': str` - [optional] dataset containing pre-calculated mask, defaults to `'entry/instrument/detector/mask'`.
* `'scale_whitefield': 1/0` - Specifies whether to scale whitefield to signal, useful if intensity jumps.
* `'do_streakfinder_analysis': 1/0` - Specifies whether to execute the streak-finder algorithm.
* `'sf_structure_radius': int` - Connectivity structure radius.
* `'sf_structure_rank': int` - Connectivity structure rank.
* `'sf_min_size': float` - Minimum number of linelets required in a detected streak.
* `'sf_vmin': float` - Peak threshold. All peaks with values lower than ``sf_vmin`` are discarded.
* `'sf_peak_structure_radius': int` - Connectivity structure radius for *peaks* detection.
* `'sf_peak_structure_rank': int` - Connectivity structure rank for *peaks* detection.
* `'sf_peak_vmin': float` - Peak threshold. All peaks with values lower than ``sf_peak_vmin`` are discarded.
* `'sf_npts': int` - Support size threshold. The support structure is a connected set of pixels which
value is above the threshold ``sf_vmin``. A peak is discarded is the size of support
value is above the threshold ``sf_peak_vmin``. A peak is discarded is the size of support
set is lower than ``sf_npts``.
* `'sf_streak_structure_radius': int` - Connectivity structure radius for *streaks* detection.
* `'sf_streak_structure_rank': int` - Connectivity structure rank for *streaks* detection.
* `'sf_streak_vmin': float` - Streak threshold. All streaks with values lower than ``sf_vmin`` are discarded.
* `'sf_min_size': float` - Minimum number of linelets required in a detected streak.
* `'sf_xtol': float` - Distance threshold. A new linelet is added to a streak if it's distance to the
streak is no more than ``sf_xtol``.
* `'sf_nfa': 1` - Number of false alarms, allowed number of unaligned points in a streak.
* `'sf_num_threads': int` - Number of threads to use for peak finder algorithm
* `'sf_mask_rois': list[(int, int, int, int)]` - [optional] list of `(y_min, y_max, x_min, x_max)` coordinates of ROIs to mask out during peak finding; can be used to mask out bad regions as well as to reduce data size to e.g. one quadrant for the purpose of speedup.
Algorithm Output:
* `'number_of_streaks': int` - Indicates the count of identified streaks.
@@ -258,22 +273,56 @@ Algorithms use input parameters specified in a JSON file provided to worker.py (
"roi_y1": [],
"roi_x2": [],
"roi_y2": [],
"do_whitefield_correction": 1,
"wf_data_file": "/sf/bernina/exp/00m_musterman/res/whitefield/JF07T32V01_wf_div.h5",
"wf_method": "div",
"apply_additional_mask_from_file": 1,
"mask_file": "/sf/bernina/exp/00m_musterman/res/mask/JF07T32V01_mask.h5",
"do_streakfinder_analysis": 1,
"sf_structure_radius": 10,
"sf_structure_rank": 2,
"sf_min_size": 7,
"sf_vmin": 75,
"sf_npts": 100,
"sf_xtol": 1.5
}
```
Example JSON for Convergent-Beam Diffraction Streak-Finder:
```json
{
"beam_center_x": 1119.0,
"beam_center_y": 1068.0,
"detector_distance": 0.092,
"do_peakfinder_analysis": 0,
"beam_energy": 11993.610318642704,
"apply_threshold": 0,
"threshold_min": 0,
"threshold_max": 35,
"apply_aggregation": 0,
"aggregation_max": 2,
"double_pixels": "mask",
"detector_rate": 100,
"do_radial_integration": 0,
"do_spi_analysis": 0,
"threshold_value": "NaN",
"select_only_ppicker_events": 0,
"disabled_modules": [],
"roi_x1": [],
"roi_y1": [],
"roi_x2": [],
"roi_y2": [],
"do_snr": 0,
"std_data_file": "/sf/instrument/exp/00m_mustermann/res/aux_data/streakfinder_metadata.h5",
"std_dataset": "entry/crystallography/std",
"whitefield_data_file": "/sf/bernina/exp/00m_mustermann/res/aux_data/streakfinder_metadata.h5",
"whitefield_dataset": "entry/crystallography/whitefield",
"mask_data_file": "/sf/bernina/exp/00m_mustermann/res/aux_data/JF_mask.h5",
"mask_dataset": "mask_data",
"scale_whitefield": 0,
"do_streakfinder_analysis": 1,
"sf_peak_structure_radius": 2,
"sf_peak_structure_rank": 2,
"sf_peak_vmin": 50,
"sf_npts": 10,
"sf_streak_structure_radius": 6,
"sf_streak_structure_rank": 4,
"sf_xtol": 2.0,
"sf_streak_vmin": 30,
"sf_min_size": 25,
"sf_nfa": 1,
"sf_num_threads": 32,
"sf_mask_rois": [[0, 2216, 2107, 4215]]
}
```
# Acknowledgment
Special thanks to Valerio Mariani for providing the cython implementation of peakfinder8.