Drop FileConfig::gap_pixels and the File constructor argument. The
generic File does not know the detector geometry of .npy and .dat
files, so gap pixels stay a RawFile option; apply insert_gap_pixels to
frames from File when needed.
RawFile takes an optional GapPixels configuration and assembles every
frame with the gaps inserted: the gap lines are filled, each module part
is scattered through the scratch buffer into its gapped position and,
if configured, the counts of the chip edge pixels are split. rows(),
cols(), pixels_per_frame() and bytes_per_frame() include the gaps, for
the full frame and for each ROI. File forwards FileConfig::gap_pixels to
RawFile and rejects it for other file types. Only Jungfrau and Eiger are
supported.
Python RawFile and File accept gap_pixels as a bool or a GapPixels and
expose it as a property. Frame shapes come from rows() and cols().
Adds the module layout assembly tests and the raw file benchmark that
the assembly refactor went in without, since the gap tests build on the
generalized raw file helper.
Second part of splitting up dev/gap. Only the in-memory transform; no
reader changes.
- `GapPixels` and `ModuleGaps` configure 2 pixel gaps at the chip
boundaries inside each Jungfrau or Eiger module and, only when module
gaps are set, gaps between modules. Gap pixels get a fill value or, with
`split_counts`, half of the neighbouring double size pixel.
- `insert_gap_pixels` and `gapped_shape` apply the transform to an image
in memory. Gap positions follow detector coordinates, so a ROI receives
exactly the gaps that fall inside it.
- Python bindings, tests and docs for the above.
Chip and module sizes are fixed at 256 and 1024x512, so there is no
detector type or quad argument: an Eiger quad is 512x512 and its only
chip boundaries are at pixel 256.
Reading raw files with gaps comes in the next PR.
First part of splitting up dev/gap. No gap pixels in here.
- Extract the frame number sync loop into
`RawFile::synchronized_frame_indices`.
- Replace the two placement branches keyed on the detector layout with
one loop. A part as wide as the ROI is read straight into the frame; a
narrower part goes through a scratch buffer sized once at open, instead
of a buffer allocated per frame.
- The "x pos not 0" implementation error is gone, since the offset is
computed for both paths.
Now builds clean on MSVC: cl version 19.51.36260 for x64, from Visual
Studio 2026 Build Tools 18.10.3.
Major items:
- [[maybe_unused]] instead of gcc/clang specific pragmas
- Removed names variables that weren't used
- Cast + explicit uing64
Collection of fixes to make aare build on Windows. Requires the previous
PRs for compilation to succeed.
- Replaced `or` keyword with ||
- updated cmake file
- fix for 32 bit long in fseek
closes #395
closes #396
logger.hpp pulled in <sys/time.h> and used gettimeofday and localtime_r,
which do not exist on MSVC. Build the timestamp from std::chrono instead
and use localtime_s on MSVC. The output format is unchanged: <colour>-
HH:MM:SS.mmm LEVEL: message<reset>. The unused __SHORT_FORM_OF_FILE__
helper now also strips backslash separators. Add logger tests that pin
the timestamp and line format.
closes #394
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
On Windows std::filesystem::path holds wchar_t, so assigning a path
component to a std::string or passing path::c_str() to fopen does not
compile. Use path::string() at those sites, and drop a redundant path
round trip in NumpyFile.
closes #399
- MSVC has no ssize_t, which made every translation unit fail on
Windows.
- Define aare::ssize_t in defs.hpp: the platform ssize_t on POSIX (so
nothing changes there) and std::ptrdiff_t on MSVC.
- Code inside the namespace picks it up unchanged; the few files that
use ssize_t outside the namespace get a using-declaration. FilePtr.hpp
now includes defs.hpp since it had no aare include.
Closing #393
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
## Summary
Makes the minimizer behind `FitModel` selectable and adds two built-in,
dependency-free solvers next to Minuit2's Migrad and Fumili.
`Minimizer.Migrad` stays the default, so existing fits behave as before.
- **`Minimizer` enum** (C++ `FitModel::SetMinimizer`, Python `minimizer`
constructor argument and property): `Migrad`, `Fumili`,
`LevenbergMarquardt`, `VarPro`.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Khalil Ferjaoui <khalilferjaoui@yahoo.fr>
- For set_nSigma only call update_pedestal if we have accumulated enough
samples otherwise just write it
- For update threshold check if pd is ready and if not throw
closes #390
- Introduce a standard way to bind numpy arrays: py::array_t<T> and
.noconvert() on the argument
- Take py::array_t by value since it's just a small wrapper
- Update bindings
- Removed faulty move (pointed out by: #227)
- Fixed license for the queue
- Runtime check for capacity=>2
Thank you @kferjaoui for finding the issue. In the end I found it better
to remove the operators rather than modifying the queue
Cleaned up the helper functions by:
- Added template for make_view (and new make_const_view) instead of the
copy paste implementation
- checks c_style and dimensions inside make_view
- test binding to be able to probe the checks in make_view
Call sites can be reviewed in a second PR since usage of const or
strictness of conversion depends...
We have no code in aare to deal with RawFiles that have missing packets
if they are not padded to full size or frames with packet loss dropped.
Added:
- check that we don't try to read files that are not supported
- python bindings for frame discard policy
at least we get the check in:
closes #171
---------
Co-authored-by: AliceMazzoleni99 <alice.mazzoleni@psi.ch>
- error upon warning when building documentation (warnings often
indicate malrendered documentation)
- fixed some warnings
Co-authored-by: Erik Fröjdh <erik.frojdh@psi.ch>
- seek to the edge of a file + read_into would select the wrong file
- Visible for example in f.read_frame(index) in python
Added fix, release notes and a small test.
closes #371
Co-authored-by: AliceMazzoleni99 <alice.mazzoleni@psi.ch>
- renamed pedestal.hpp to bind_Pedestal.hpp for python bindings
- variance is now always calculated in double and made private
- cost correctness in a few places
- removed push_fast and explicit updates of mean from Pedestal. If
performance is needed use FastPedestal
- Pedestal no longer caches std
Improved API for ClusterFile
- read_frame now returns a std::optional<ClusterVector> to differentiate
between an empty frame and end of file
- f.frames() produces an iterator over frames
- f.chunks() produces an iterator reading chunks of clusters
- python string repr for Cluster and Eta
- Context manager for RawMasterFile
- slice operator for ROI
---------
Co-authored-by: Erik Fröjdh <erik.frojdh@psi.ch>
- Updated error message to mention out of range and which file it
applies to
- If there are different number of frames in the subfiles, take the
minimum and print a warning
Closes: #363
Closes: #362
- fixed test in ClusterFinderMT (use minimum one pedestal to accumulate
dummy pedestal)
- added minimum_pedestal_samples as additional constructor argument in
ClusterFinder and ClusterFinderMT
- bugfix: race conditions for empty queues in collect() - check directly
when needed
- Default `ClusterVector` move operations simplifying the code and
fixing a bug.
- Fixed inconsistent frame number type (uint64/int32)
- Validate Python masks as one-dimensional, C-contiguous Boolean arrays,
handle empty masks safely, and reserve filtered storage based on the
selected cluster count.
- Align the C++ and Python API documentation with the implementation,
including concise `hitmap` and reduction documentation and a correctly
rendered constructor example.
- handles file reading of disabled udp ports
- adds members disabled_udp_ports in master file as an optional
- adds member diasbled_udp_port_types in master file as an optional
- Treats enabled udp ports as ROIs
- merges/combines consecutive ROis into one
### To discuss:
- right now can only handle either ROI or disabled udp ports (disabled
udp ports has higher precedence)
- Should Frames in File be removed from Master File as value is nonsense
for disabled udp ports
---------
Co-authored-by: Erik Fröjdh <erik.frojdh@psi.ch>
Optimized pedestal tracking with FastPedestal and performance improvements to the cluster finder.
- ~2x faster hitting 14k FPS on benchmark dataset on 16 core threadripper pro
**Changes**
- No bounds check on interior of frame. (only within half of a cluster)
(~15% improvement)
- Cache threshold (std*n_rms) Biggest improvement comes from not
computing std for each access
- Cache pedestal subtracted frame
- Switched to FastPedestal which assumes we reached steady state
**Things tried but rejected:**
- Always store cluster values, commit on val==max (~15% drop in frame
rate)
**Options**
- using 16 bit clusters and 16 bit pedestal. (slows down the single
threaded case but allow us to reach 12k with multi threading)
**Other notes on performance**
- Passing in memory frames from python and doing nothing: 0.8us/frame
- Passing + pedestal subtraction: 28us/frame
- Passing + pedestal + cluster finding (no store): 652us/frame
- Passing + pedestal + cluster finding: 886 us/frame
Minor API change:
- NDView<T> implicitly converts to NDView<const T> (as std::span in
C++20)
- expand4to8bit and expand24to32bit now takes a view over const uint8_t
to show that the input is not changes
closes #291
Before v8.0 we should always try to read analog samples. This bug
resulted in old Mönch03 data not being reordered.
1. Added test for master file parsing
2. Fixed code
closing #348
Erik Fröjdh
mazzol_aandErik Fröjdh