Before v8.0 we should always try to read analog samples. This bug
resulted in old Mönch03 data not being reordered.
1. Added test for master file parsing
2. Fixed code
closing #348
Collection of small improvements in usability:
- NDView works for large arrays
- Direct subtraction of Pedestal from np.array
- len() support for python bindings of files
- reshape image directly in decoder such that first dimension is num
counters
- take into account chip artefact in decoder.
---------
Co-authored-by: Erik Fröjdh <erik.frojdh@psi.ch>
With C++20 `fmt::print(s)` expects a compile time format string and
otherwise fails complaining about consteval. To get runtime formatting
use `fmt::print(fmt::runtime(s))`
Multi threaded filling of per pixel histograms for example for detector calibration
1. PixelHistogram - Generic variant expects already pedestal subtracted
data
2. PedestalTrackingHistogram - Terrible name, useful class. Keeps it's
own pedestal and does conversion and pedestal tracking in the worker
threads.
---------
Co-authored-by: Lars Erik Fröjd <froejdh_e@pc-jungfrau-02.psi.ch>
- Allowing the users more flexibility to play around with custom eta
functions without touching the c++ code
- passing vector of eta values to ``transform_eta_values``
```
from aare import Interpolator, ClusterVector, Etai, Cluster
import numpy as np
def custom_eta(cluster_pixel_coordinate_x, cluster_pixel_coordinate_y, cluster_data):
# dummy custom eta function that just returns the sum of the cluster data
eta = Etai()
eta.x = 0.1 # dummy x value
eta.y = 0.1 # dummy y value
eta.sum = np.sum(cluster_data) # sum of the cluster data as the "energy
return eta
# Create a dummy eta distribution and bins
eta_distribution = np.zeros((10, 10, 1)) # dummy eta distribution
etax_bins = np.linspace(0, 1.0, 11)
etay_bins = np.linspace(0, 1.0, 11)
e_bins = np.array([0., 10.]) # dummy energy bins
# Create the interpolator
interpolator = Interpolator(eta_distribution, etax_bins, etay_bins, e_bins)
# Create a dummy cluster vector
cluster_vector = ClusterVector()
cluster_vector.push_back(Cluster(10, 5, np.ones(shape=9, dtype = np.int32)))
cluster_vector.push_back(Cluster(20, 10, np.ones(shape=9, dtype = np.int32)))
# Create dummy etas for the clusters
cluster_array = np.array(cluster_vector)
etas = np.array([custom_eta(cluster["x"], cluster["y"], cluster["data"]) for cluster in cluster_array])
# transform eta values to uniform coordinates
uniform_coordinates = interpolator.transform_eta_values(etas)
# Interpolate to get the photon coordinates e.g. apply interpolation logic
photon_coordinates_x = cluster_array["x"] + uniform_coordinates["x"] # add to pixel coordinate
photon_coordinates_y = cluster_array["y"] + uniform_coordinates["y"] # add to pixel coordinate
```
advantage: full control over interpolation logic,
downside: inefficient quite some loops in python
- passing pre computed eta values to interpolate function
```
Interpolator.interpolate(cluster_vector, etas)
```
downside: less flexibility in interpolation logic.
downside: People might misuse it instead of using interpolate directly
with a pre compiled eta function implemented in c++
## Unified Minuit2 fitting framework with FitModel API
### Models (`Models.hpp`)
Consolidate all model structs (Gaussian, RisingScurve, FallingScurve)
into a
single header. Each model provides: `eval`, `eval_and_grad`, `is_valid`,
`estimate_par`, `compute_steps`, and `param_info` metadata. No Minuit2
dependency.
### Chi2 functors (`Chi2.hpp`)
Generic `Chi2Model1DGrad` (analytic gradient) templated on the model
struct.
Replaces the separate Chi2Gaussian, Chi2GaussianGradient,
Chi2Scurves, and Chi2ScurvesGradient headers.
### FitModel (`FitModel.hpp`)
Configuration object wrapping `MnUserParameters`, strategy, tolerance,
and
user-override tracking. User constraints (fixed parameters, start
values, limits)
always take precedence over automatic data-driven estimates.
### Fit functions (`Fit.hpp`)
- `fit_pixel<Model, FCN>(model, x, y, y_err)` -> single-pixel,
self-contained
- `fit_pixel<Model, FCN>(model, upar_local, x, y, y_err)` -> pre-cloned
upar for hot loops
- `fit_3d<Model, FCN>(model, x, y, y_err, ..., n_threads)` ->
row-parallel over pixel grid
### Python bindings
- `Pol1`, `Pol2`, `Gaussian`, `RisingScurve`, `FallingScurve` model
classes with
`FixParameter`, `SetParLimits`, `SetParameter`, and properties for
`max_calls`, `tolerance`, `compute_errors`
- Single `fit(model, x, y, y_err, n_threads)` dispatch replacing the old
`fit_gaus_minuit`, `fit_gaus_minuit_grad`, `fit_scurve_minuit_grad`,
etc.
### Benchmarks
- Updated `fit_benchmark.cpp` (Google Benchmark) to use the new FitModel
API
- Jupyter notebooks for 1D and 3D S-curve fitting (lmfit vs Minuit2
analytic)
- ~1.8x speedup over lmfit, near-linear thread scaling up to physical
core count
---------
Co-authored-by: Erik Fröjdh <erik.frojdh@psi.ch>
Reading multiple ROI's for aare
- read_frame, read_n etc throws for multiple ROIs
- new functions read_ROIs, read_n_ROIs
- read_roi_into (used for python bindings - to not copy)
all these functions use get_frame or get_frame_into where one passes the
roi_index
## Refactoring:
- each roi keeps track of its subfiles that one has to open e.g.
subfiles can be opened several times
- refactored class DetectorGeometry - keep track of the updated module
geometries in new class ROIGeometry.
- ModuleGeometry updates based on ROI
## ROIGeometry:
- stores number of modules overlapping with ROI and its indices
- size of ROI
Note: only tested size of the resulting frames not the actual values
---------
Co-authored-by: Erik Fröjdh <erik.frojdh@psi.ch>
Co-authored-by: Erik Fröjdh <erik.frojdh@gmail.com>
Issue from Jonathan.
- writing to output queue did not check if queue is full - such that
frames were dropped.
## Dataset to recreata issue:
data overf 10G interface can be accessed on pc: pc-highz-02
raw frames:
/mnt/sls_det_storage_10G/highZ_data/JMulvey/Calibration_From_HZ02/2025Jan_m343/Zr15800eV/250129_CZTsolo_Xray_Tp_15C_tint_100_master_0.json
pedestal frames:
/mnt/sls_det_storage_10G/highZ_data/JMulvey/Calibration_From_HZ02/2025Jan_m343/Zr15800eV/250129_CZTsolo_Pedestal_Tp_15C_tint_100_master_0.json
---------
Co-authored-by: Erik Fröjdh <erik.frojdh@psi.ch>
- Removed redundant arr.value(ix,iy...) on NDArray use arr(ix,iy...)
- Removed Print/Print_some/Print_all form NDArray (operator << still
works)
- Added const* version of .data()
- Comment for documentation
- Some extra tests
- New aare:to_string/string_to similar to what we have in
slsDetectorPackage
- Added members period and exptime to RawMasterFile
- Parsing exposure time and period for json and raw master file formats
- Parsing of RawMasterFile from string stream to enable test without
files
Comments:
- to_string is at the moment not a public header. Can make it later if
needed. This gives us full freedom with the API
- FileConfig should probably be deprecated need to look into it.
Meanwhile removed python bindings and string conv
This PR adds support for decoding digital data from the my320 test chip.
- Added BitOffset (strong type)
- Expand 24 to 32 bit
- Python bindings for decoding my302
- Improved docs
- added rosenblatttransform
- added 3x3 eta methods
- interpolation can be used with various eta functions
- added documentation for interpolation, eta calculation
- exposed full eta struct in python
- disable ClusterFinder for 2x2 clusters
- factory function for ClusterVector
---------
Co-authored-by: Dhanya Thattil <dhanya.thattil@psi.ch>
Co-authored-by: Erik Fröjdh <erik.frojdh@psi.ch>
- Removed unused ClusterFile.cpp (code from before it was templated)
- Updated the list of .cpp files in CMakeLists.txt to match alphabetic
listing in the browser
Added a section on the ideas behind the library and also explaining the
branching strategy.
---------
Co-authored-by: Dhanya Thattil <dhanya.thattil@psi.ch>
Erik Fröjdh
mazzol_aandErik Fröjdh
