Jungfraujoch/docs/CHANGELOG.md

Changelog

1.0.0

1.0.0-rc.122

This is an UNSTABLE release.

• jfjoch_broker: Add thresholding to prefer shorter vectors after FFT
• jfjoch_broker: Add experimental mosaicity estimation for rotation experiments (this is work in progress)
• jfjoch_broker: Update nlohmann::json to 3.12.0
• jfjoch_viewer: Display file opening errors
• jfjoch_viewer: When loading files over DBus add retry/back-off till the file is available

1.0.0-rc.121

This is an UNSTABLE release.

• jfjoch_broker: Report changes in the image buffer, so viewer doesn't reload constantly
• jfjoch_viewer: Improve performance of loading images
• jfjoch_viewer: Auto-throttle image loading in HTTP-sync / movie modes
• jfjoch_viewer: Auto-foreground calculated with histogram
• jfjoch_viewer: Fix rare segmentation fault

1.0.0-rc.120

This is an UNSTABLE release.

• jfjoch_broker: Improve performance of binary plot export

1.0.0-rc.119

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_broker: Add binary export of data analysis plots over OpenAPI
• jfjoch_broker: Minor fixes to HTTP error handling
• jfjoch_viewer: Prefer binary plots over JSON plots
• jfjoch_viewer: Change foreground with F button + wheel
• jfjoch_viewer: Change way how angles are displayed
• jfjoch_viewer: Display resolution of the mouse cursor in top left corner

1.0.0-rc.118

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_viewer: Fix issue when HTTP sync silently disconnected when it was enabled when the broker was starting measurement.
• jfjoch_broker: Add protections on time of geometry optimization and reduce rotation recalculations

1.0.0-rc.117

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_viewer: Add ROI results to the dataset info plots
• jfjoch_writer: Remove HTTP interface, as it is not needed/used at the moment

1.0.0-rc.116

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_viewer: Add binning options in the context menu

1.0.0-rc.115

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_broker: Default spot finding settings can be configured via config JSON
• jfjoch_viewer: FFT analysis of data in the dataset plot

1.0.0-rc.114

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_broker: Fix generating JPEG images with resolution estimation

1.0.0-rc.113

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_broker: Improve handling of rotation indexing
• jfjoch_broker: More information saved in CBOR end message (WIP)
• jfjoch_writer: Save rotation indexing lattice parameters and Niggli class
• jfjoch_viewer: Remove (for now) primitive cell information
• jfjoch_viewer: Use angle for dataset info plot for rotation scans

1.0.0-rc.112

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_broker: Experimental rotation (3D) indexing
• jfjoch_broker: Minor fix to error in optimizer potentially returning NaN values

1.0.0-rc.111

This is an UNSTABLE release.

• jfjoch_viewer: Remove 3D lattice viewer (not really useful at this moment)
• jfjoch_viewer: Fix auto contrast not refreshing image

1.0.0-rc.110

This is an UNSTABLE release.

• jfjoch_broker: Add auto-contrast option for preview images
• Frontend: Add logo image
• jfjoch_viewer: Add logo image
• jfjoch_viewer: For image chart allow to set min value to zero
• jfjoch_viewer: For resolution estimation plots, visualization uses 1/d^2 as measure
• jfjoch_viewer: Add 3D unit cell visualization (experimental/WIP/not really there)
• Documentation: Add logo image

1.0.0-rc.109

This is an UNSTABLE release.

• jfjoch_viewer: Add keyboard shortcuts and option to copy image to clipboard
• jfjoch_broker: Fix bit-width and exposure time for PSI EIGER detectors

1.0.0-rc.108

This is an UNSTABLE release.

• jfjoch_viewer: Fix bug when resolution estimation/B-Factor/Profile radius were not set (NaN)
• jfjoch_viewer: Show spots is off by default, resolution ring mode is enabled by default
• jfjoch_viewer: Fit to window of image is now default when size of the grid changes

1.0.0-rc.107

This is an UNSTABLE release.

• jfjoch_viewer: Minor polishing of new functionality
• jfjoch_broker: User NaN for empty azimuthal bins

1.0.0-rc.106

This is an UNSTABLE release.

• jfjoch_viewer: Allow for multiple dataset info plots
• jfjoch_viewer: Highlight current element in grid

1.0.0-rc.105

This is an UNSTABLE release.

• jfjoch_viewer: Clean-up widgets slightly
• jfjoch_viewer: Limit right panel to 600 pixels
• jfjoch_viewer: Parse crystal symmetry type
• jfjoch_viewer: Grid scan view takes color map and can be fit to zoom

1.0.0-rc.104

This is an UNSTABLE release.

• jfjoch_writer: Fix and improve the way grid scan geometry is saved (non-NXmx extension makes it way easier)
• jfjoch_viewer: Display grid scan results in 2D (work in progress)
• jfjoch_viewer: Improve auto-scaling on start of images (work in progress)
• jfjoch_viewer: Add B-factor and resolution estimate to the dataset info plots

1.0.0-rc.103

This is an UNSTABLE release.

• jfjoch_viewer: Minor improvements to the viewer
• jfjoch_broker: Change behavior for modular detectors: coordinates of 0-th pixel can be now arbitrary and detector will be cropped to the smallest rectangle limited by module coordinates

1.0.0-rc.102

This is an UNSTABLE release.

• jfjoch_viewer: Minor improvements to the viewer

1.0.0-rc.101

This is an UNSTABLE release.

• jfjoch_viewer: Auto load is better handling change of states
• jfjoch_viewer: Fix DBus registration
• jfjoch_viewer: Handle charts better with vertical lines on hover and status bar update
• jfjoch_viewer: Calculate ROI in a more efficient way

1.0.0-rc.100

This is an UNSTABLE release.

• jfjoch_viewer: Fix dbus registration
• jfjoch_viewer: Remove background slider for diffraction image
• jfjoch_viewer: Adjustments for 2D azimuthal image viewer

1.0.0-rc.99

This is an UNSTABLE release.

• jfjoch_broker: Fix output during mask data collection

1.0.0-rc.98

This is an UNSTABLE release and not recommended for production use (please use rc.96 instead).

• jfjoch_broker: For DECTRIS detectors fix dark data collection during initialization

1.0.0-rc.97

This is an UNSTABLE release and not recommended for production use (please use rc.96 instead).

• jfjoch_broker: For DECTRIS detectors add dark data collection during initialization for bad pixel mask
• jfjoch_broker: Refactor of calibration logic for more clear code (likely to introduce problems)
• jfjoch_viewer: Add option to handle user pixel mask (experimental)
• jfjoch_viewer: More options for ROI
• jfjoch_viewer: Add window to display calibration

1.0.0-rc.96

This is an UNSTABLE release.

• Fixes in CI pipeline
• jfjoch_broker: Remove PNG preview, no dependency on libpng
• jfjoch_writer: Fix UTC timestamp being generated wrong (mix between milli- and microseconds)
• jfjoch_viewer: Show data collection time in dataset tooltip
• jfjoch_viewer: Allow to choose the calibrant (presets for LaB6 and silver behenate)
• jfjoch_viewer: Auto foreground value
• Use external libjpeg-turbo and libtiff: simpler build stack, these are built and linked statically in automated Docker builds
• Remove OpenBLAS dependency

1.0.0-rc.95

This is an UNSTABLE release.

• Fixes in CI pipeline
• Add git-lfs to Rocky8 docker image

Previous releases (91-94) had a wrong FPGA image upload to Gitlab release. This is now solved.

1.0.0-rc.94

This is an UNSTABLE release.

• FFTIndexer: Add limit on angles to avoid colinear vectors
• Docker images: Add 3D Qt
• Gitea: Fixes to the pipeline

1.0.0-rc.93

This is an UNSTABLE release.

• CI: Fixes to Gitlab based pipeline
• PCIe driver: Fix PCIe revision being hex number

1.0.0-rc.92

This is an UNSTABLE release.

• jfjoch_broker: Fix code that predicted Bragg reflections scattering back from the sample.

1.0.0-rc.91

This is an UNSTABLE release. This release introduces new features, which usually means these need more field testing before enough maturity. For production use we recommend waiting for a future bug-fix release.

• FPGA: Implement high pixel value threshold - pixels above the given value will be considered saturated
• jfjoch_broker: Spot finding and integration predictions are ported to a GPU
• jfjoch_broker: Estimate resolution
• jfjoch_broker: Lattice search
• jfjoch_broker: Many more improvements in image analysis

1.0.0-rc.90

This is an UNSTABLE release.

• jfjoch_broker: for indexing min index spots for a viable cells can be changed via OpenAPI
• jfjoch_viewer: Optional auto-reanalyze images
• jfjoch_writer: Add option where no files at all are saved
• Documentation: improvements

1.0.0-rc.89

This is an UNSTABLE release.

• jfjoch_broker: Fix resolution estimation code
• jfjoch_broker: Fix Wilson B-factor calculation code
• jfjoch_viewer: Improve display of plots
• jfjoch_viewer: Fix segmentation fault
• jfjoch_viewer: Display missing metadata when using HTTP
• jfjoch_viewer: Fix bug when opening the same file twice

1.0.0-rc.88

This is an UNSTABLE release.

• jfjoch_viewer: Add resolution estimation to the image information
• jfjoch_broker: Minor changes to resolution estimate routine

1.0.0-rc.87

This is an UNSTABLE release.

• jfjoch_viewer: Display more image metadata (angle / exposure time)
• jfjoch_viewer: Improve I/sigma and B-factor plots
• jfjoch_broker: Estimate resolution based on visible spots

1.0.0-rc.86

This is an UNSTABLE release.

• jfjoch_broker: Update logic when initializing detector to make it a bit more resilient
• Gitea pipelines have nocuda option for all architectures

1.0.0-rc.85

This is an UNSTABLE release.

• jfjoch_viewer: When using online view, dataset info plots are not switched back to the first category for each image
• jfjoch_viewer: Handle spot count better in dataset info plots
• jfjoch_viewer: Highlight spots in ice ring resolutions in cyan, when detection is enabled

1.0.0-rc.84

This is an UNSTABLE release.

• jfjoch_broker: Write in log which detector is being initialized
• Changes to automated build system

1.0.0-rc.83

This is an UNSTABLE release.

• jfjoch_viewer: Fix in generating preview image for signed data (wrong bit-width was assumed before)
• CI: Fix script to generate python client

1.0.0-rc.82

This is an UNSTABLE release.

• jfjoch_viewer: Enable FFTW based indexing in viewer (very slow at the moment)
• Frontend: Minor fixes
• Build scripts: Minor fixes to FFTW

1.0.0-rc.81

This is an UNSTABLE release. This release introduces new features, which usually means these need more field testing before enough maturity. For production use we recommend waiting for a future bug-fix release.

• jfjoch_broker: Add option to detect ice rings, adjust width of ice ring and change of logic to exclude ice rings in indexing
• jfjoch_broker: Add FFTW based indexer for CPU only indexing
• jfjoch_broker: Enable saving X-ray fluorescence spectra
• jfjoch_writer: Write total spot count (before filtering)
• jfjoch_viewer: Add more information on source, sample, and buttom to show ice rings
• jfjoch_viewer: Enable data processing inside the viewer

CI: Moving from Gitlab to Gitea at PSI

1.0.0-rc.80

This is an UNSTABLE release.

• jfjoch_broker: Fix bug when wrong value for a plot (NaN or infinity) would lead to a null in a plot, which cannot be parsed by viewer

1.0.0-rc.79

This is an UNSTABLE release.

• jfjoch_viewer: Fix bug when loading new dataset was creating a cascade of signals leading to poor performance
• jfjoch_writer: Save nimages_per_trigger in detectorSpecific

1.0.0-rc.78

This is an UNSTABLE release.

• jfjoch_viewer: Using a single event loop (reading images is not in dedicated thread anymore)

1.0.0-rc.77

This is an UNSTABLE release.

• jfjoch_viewer: Display detector and dataset settings with tooltips
• jfjoch_viewer: Clean excessive HDF5 warnings
• jfjoch_viewer: Display unit cell
• jfjoch_extract_hkl: Write a tool to extract reflection intensity from a dataset

1.0.0-rc.76

This is an UNSTABLE release.

• jfjoch_broker: Increase predicted hkl to 100.0, use lighter math to exclude too-high resolution ones
• jfjoch_broker: Use standard deviation formula to find profile radius (not the one using median)
• jfjoch_writer: Save space group number (non-NXmx addition) in addition to name
• jfjoch_viewer: Fix the bug on reading space_group as string
• jfjoch_viewer: Add missing resolution labels on rings
• jfjoch_viewer: Remove Q value from the status bar

1.0.0-rc.75

This is an UNSTABLE release.

• jfjoch_broker: EIGER2 missing minimum threshold - hardcoded to 2.7 keV for the time being

1.0.0-rc.74

This is an UNSTABLE release.

• jfjoch_broker: Fix for EIGER UDP port settings (vertical half of the module missing)
• jfjoch_broker: Detector settings were not applied for EIGER/DECTRIS detector when changed after initialization

1.0.0-rc.73

This is an UNSTABLE release.

• jfjoch_broker: Space group number treatment in OpenAPI was wrong, zero value is no longer allowed and no longer default

1.0.0-rc.72

This is an UNSTABLE release. This release introduces new features, which usually means these need more field testing before enough maturity. For production use we recommend waiting for a future bug-fix release.

• jfjoch_broker: Refactor of indexing and geometry refinement code
• jfjoch_broker: Handle space group/centering in refinement code
• jfjoch_broker: Replace mosaicity with profile radius: refining the former is difficult with still images
• jfjoch_broker: There is no longer 0.5 pxl offset for spots-to-reciprocal-space conversion
• jfjoch_writer: Experimental saving of reflections
• jfjoch_writer: Save space group name as string
• jfjoch_viewer: Add profile radius and B-factor
• jfjoch_viewer: Show 4 digits for wavelength
• jfjoch_viewer: Match rings between calibrant and observation (will handle missing/wrong rings)
• FPGA: Use UDP destination port to distinguish between detector modules and data streams
• FPGA: Add experimental PTP core (PTP over L2, only Sync/Follow_up)
• FPGA driver: Fix for Linux kernel 6.12+ (thanks to Tim Gruene)

1.0.0-rc.71

This is an UNSTABLE release.

• jfjoch_broker: Remove resolution estimation via machine learning
• jfjoch_broker: Harmonize code to analyze spot finding results (indexing/refinement/integration) between CPU and FPGA receivers
• jfjoch_viewer: Fix error when HDF5 files with indexing results couldn't be loaded on a machine without GPU

1.0.0-rc.70

This is an UNSTABLE release. This release introduces new features (geometry refinement), which usually means these need more field testing before enough maturity. For production use we recommend waiting for a future bug-fix release.

• jfjoch_broker: Fix bug when PSI EIGER frame time was not set properly at the start of the measurement

• jfjoch_broker: Fix PONI rot2 angle rotating detector in a wrong direction (PyFAI convention is for this angle to rotate detector downwards)

• jfjoch_broker: Enable geometry refinement - first try (work in progress)

• jfjoch_viewer: Fix deadlock when opening HTTP connections

• jfjoch_viewer: Display rings as ellipses with detector tilt

• jfjoch_viewer: Add button to calibrate detector geometry based on LaB6 image

• jfjoch_writer: Save detector tilt angles (rot1, rot2, rot3)

• Add Google Ceres a non-linear least-square optimization library to Jungfraujoch

• Add experimental detector calibration routines (for LaB6)

• Improve documentation on the ZeroMQ writer notification socket and detector geometry

1.0.0-rc.69

This is an UNSTABLE release.

• jfjoch_viewer: Metadata can be modified for an open dataset (no option to save)
• jfjoch_viewer: Refactor multiple issues in the viewer regarding image reading code to allow for further developments
• jfjoch_viewer: Resolution rings not enabled by default
• jfjoch_broker: Handle properly PONI rotations in dataset settings though still not updated properly in the HDF5 file

1.0.0-rc.68

This is an UNSTABLE release.

• jfjoch_broker: Temperature threshold can be changed for JUNGFRAU detector
• jfjoch_broker: Default detector settings can be configured for each detector separately
• jfjoch_broker: Refactor spot filtering code, max spot count can be modified for dataset settings
• jfjoch_broker: Refactor indexing refinement, make it the same for both FFBIDX and FFT indexing
• jfjoch_broker: Reference unit cell will be taken into account for FFT indexing to filter
• jfjoch_broker: Review PONI rotation angles and azimuthal angle conventions along with PyFAI

1.0.0-rc.67

This is an UNSTABLE release.

• jfjoch_broker: Enable SSL
• jfjoch_broker: Wilson B-factor only provided is fit is relatively OK (R^2 > 0.3); this will be refined much more in the future

1.0.0-rc.66

This is an UNSTABLE release.

• jfjoch_broker: Indexers operate as thread pool, which is operating
• jfjoch_viewer: Increase interval between loading images + fix too many verbose messages

1.0.0-rc.65

This is an UNSTABLE release.

• jfjoch_broker: Print information regarding used image pushers
• jfjoch_viewer: Allow syncing with Jungfraujoch server
• OpenAPI: Clarify licensing terms in the file

1.0.0-rc.64

This is an UNSTABLE release.

• jfjoch_broker: Fix issue in receiver light with very long preparation time for threads
• jfjoch_broker: Add verbose option
• jfjoch_broker: Don't trigger pedestal if critical settings are not changed when loading detector settings
• jfjoch_broker: Detector left in busy state when detector settings were improper
• jfjoch_viewer: Modify DBus interface to avoid loading same file and image 0 multiple times
• jfjoch_lite_perf_test: Add verbose option

1.0.0-rc.63

This is an UNSTABLE release.

• jfjoch_broker: Save NX/NY for grid scan result
• jfjoch_broker: Add processing time to CBOR output and plot
• jfjoch_writer: Add processing time to data file

1.0.0-rc.62

This is an UNSTABLE release.

• jfjoch_broker: Fix bug where low resolution spots were not counted properly
• jfjoch_broker: Spot count is provided prior to filtering of spots to max_spot_count
• jfjoch_broker: Add more spot count information to CBOR
• jfjoch_viewer: Fix issue with ROI drawing resulting in multiple overlapping rectangles

1.0.0-rc.61

This is an UNSTABLE release.

• jfjoch_broker: Fix bug where FFT indexing could result in a very short or even zero length vector
• jfjoch_broker: Ice ring and indexed spot count enabled as plots and saved in grid scan results
• jfjoch_broker: High resolution limit for low res. spot counting can be adjusted

1.0.0-rc.60

This is an UNSTABLE release.

• jfjoch_broker: Fix bug when the neural network inference client was busy and this status was never released
• jfjoch_broker: Revert the indexing threshold with distance from integer for Miller indices
• jfjoch_broker: Fix bug in scattering vector calculation, resulting in indexing not working outside 1.0 A X-ray wavelength

1.0.0-rc.59

This is an UNSTABLE release.

• jfjoch_broker: Fix bug when broker was waiting for notification message before sending end message, resulting in deadlock.
• jfjoch_writer: Verbose option for debugging.

1.0.0-rc.58

This is an UNSTABLE release.

• jfjoch_viewer: Fix memory leak
• jfjoch_writer: Add detector_number/serial_number to master file

1.0.0-rc.57

This is an UNSTABLE release.

• jfjoch_broker: Fix bug when enabling ML resolution estimation was not possible
• jfjoch_viewer: "Movie" mode

1.0.0-rc.56

This is an UNSTABLE release.

• jfjoch_broker: Fixing more bugs related to neural network inference for ML estimation

1.0.0-rc.55

This is an UNSTABLE release.

• jfjoch_broker: Fixing minor bugs related to neural network inference for ML estimation

1.0.0-rc.54

This is an UNSTABLE release.

• jfjoch_broker: Indexing with AUTO settings (FFBIDX if unit cell provided; FFT if not)
• jfjoch_broker: Don't remove shared memory area when deactivating detector
• jfjoch_writer: Save writer release
• jfjoch_viewer: Increase time for the messages in the status bar

1.0.0-rc.53

This is an UNSTABLE release.

• PCIe driver: Imperfect solution for RHEL 9.5+ changes
• jfjoch_writer: Fix to angle containers for AutoProc compatibility
• jfjoch_fpga_test: Use consecutive number for devices, not interleaved

1.0.0-rc.52

This is an UNSTABLE release.

• jfjoch_viewer: Use warmer colors to distinguish from AareGUI
• jfjoch_viewer: Minor adjustments to DBus setting image number
• jfjoch_broker: Fix in low resolution spot count plotting

1.0.0-rc.51

This is an UNSTABLE release.

• jfjoch_broker: Send preview in PNG format
• jfjoch_broker: Provide count of spots in 50.0 - 5.0 A range
• jfjoch_broker: Provide ML resolution estimation in scan result
• jfjoch_broker: Allow removing beam center in web preview

1.0.0-rc.50

This is an UNSTABLE release.

• The release fixes some of many bugs introduced in recent releases
• jfjoch_viewer: display predictions for indexed cells

1.0.0-rc.49

This is an UNSTABLE release.

• jfjoch_broker: Handle sample temperature (K) and ring current (mA) to metadata
• jfjoch_writer: For angle containers in NXmx add _end dataset, sample temp. and ring current

1.0.0-rc.48

This is an UNSTABLE release.

• jfjoch_broker: fix the bug when a unit cell was not exported for a scan result.

1.0.0-rc.47

This is an UNSTABLE release.

• jfjoch_viewer: fix dbus service path
• jfjoch_writer: fix CBF/TIFF writing

1.0.0-rc.46

This is an UNSTABLE release.

• jfjoch_viewer: remove dependency on image analysis

1.0.0-rc.45

This is an UNSTABLE release.

• jfjoch_broker: Detector list returns pixel size (mm)

1.0.0-rc.44

This is an UNSTABLE release.

• jfjoch_broker: more general definition of scan result export

Braking changes:

• It removes additions to OpenAPI from 1.0.0-rc.43
• It makes changes to the "unit_cell" definition in OpenAPI specs. It might be harmless in some languages and may result in errors in other implementations.

1.0.0-rc.43

This is an UNSTABLE release.

• jfjoch_broker: Export grid scan results into a single data structure

1.0.0-rc.42

This is an UNSTABLE release.

• jfjoch_broker: Add pixel_sum to CBOR output.
• jfjoch_broker: Changes to sigma estimation in QuickIntegrate routine
• jfjoch_writer: Save pixel_sum

1.0.0-rc.41

This is an UNSTABLE release. This release includes multiple new features, it should not be used in production at the moment.

• jfjoch_broker: Estimate B-factor, mosaicity to evaluate crystal diffraction

• jfjoch_broker: Export GPU count via OpenAPI

• jfjoch_broker: Enable 2D azimuthal integration and PONI rotations for detector

• FPGA: Increase the number of integration bins to 2048

1.0.0-rc.40

This is an UNSTABLE release. This release includes multiple new features, it should not be used in production at the moment.

• jfjoch_broker: Jungfraujoch supports grid scan metadata, including dedicated plotting schemes and NXmx structures

• jfjoch_broker: Improve metadata for rotation data collection

• jfjoch_broker: Better handling of plotting

• jfjoch_broker: FFT based indexing

• jfjoch_broker: Integration, first try, results not saved at the moment

• jfjoch_broker: Internal improvements in image handling

• jfjoch_writer: Multiple adjustments adapt to changes in this release for new features

• jfjoch_writer: New state management model to improve clarity of error reporting

• jfjoch_viewer: Remote control via DBus

• Frontend: Multiple adjustments for new features

• Frontend: Grid scan plots

WARNING! OpenAPI contains breaking changes in regard to plotting results, so care has to be taken.

1.0.0-rc.39

• FPGA: Bugfix for pixel masked for data analysis if summation was on
• jfjoch_viewer: Fix segmentation fault when cursor was outside of image

1.0.0-rc.38

• jfjoch_broker: Neural net model is not linked with C++ code due to deployment issues, it is rather distributed as python code, connected via RES
• jfjoch_broker: Neural net model can use all 4 quadrants of the detector
• jfjoch_broker: For EIGER image time can be provided through /start
• jfjoch_viewer: Add image list option
• jfjoch_viewer: Drawing circular ROIs with shift
• jfjoch_viewer: Enable image summation
• jfjoch_viewer: Image reader is significantly reworked, hopefully without affecting the viewer

1.0.0-rc.37

• jfjoch_broker: Make locking rules more flexible
• jfjoch_broker: Load mask via SIMPLON interface for DECTRIS detectors
• jfjoch_viewer: Add status bar

1.0.0-rc.36

This is UNSTABLE release. Wait for new version to use in a production environment.

• jfjoch_broker: Support for Jungfraujoch Lite is enabled - software-based receiver for DECTRIS detectors (required a lot of refactoring, potentially leading to unstable code)
• jfjoch_broker: Enable Resonet support (ML-based diffraction resolution estimation)
• jfjoch_broker: Fix error in compression, where bitshuffle/LZ4 and bitshuffle/Zstd HDF5 headers were wrongly generated for 8-bit and 32-bit data
• jfjoch_writer: Increase buffering to 1000 images in the receiver
• jfjoch_writer: Images can be written as CBF or TIFF in addition to HDF5

1.0.0-rc.35

This is UNSTABLE release, not properly tested. Wait for new version for using production.

• jfjoch_broker: If module is delayed by more than 50 frames versus other modules, it will be ignored and receiver is not waiting.
• jfjoch_writer: Save EIGER energy threshold
• jfjoch_writer: Add /entry/sample/goniometer for compatibility with eiger2cbf program

1.0.0-rc.34

This is UNSTABLE release - introducing new features, but not properly tested. Wait for new version for using production.

• jfjoch_broker: More consistency for file format definition (breaking change in API from 1.0.0-rc.31 for file writer settings)
• jfjoch_broker: For storage cells mask is logical sum of detector bad pixels for all storage cells
• jfjoch_broker: Handle situation when detector doesn't want to gracefully stop (to be tested)
• jfjoch_broker: Center-of-mass position and mean for ROI is added to available plots
• jfjoch_viewer: Can extract data analysis results from "legacy" format
• jfjoch_viewer: Display dataset name
• FPGA: Pixel mask is used for data analysis part even if it is not applied to pixels
• FPGA: Add pixel sum to module statistics
• FPGA: ROI number is reduced to 16, but pixel can belong to every defined ROI
• FPGA: Spot finder is back to full dynamic range (24-bit)
• FPGA: More debug features for internal FIFOs

Known issues:

• ROI count flag was added to firmware. For the time being the flag will be wrongly set to 10 due to mismatch of FPGA build scripts.
• EIGER data acquisition has an issue that is currently debugged

1.0.0-rc.33

• jfjoch_broker: Fix issue with EIGER settings being loaded improperly

1.0.0-rc.32

• jfjoch_broker: Refactor code for azimuthal integration for further improvements
• jfjoch_broker: Minor fix for EIGER (trim energies are manually set for E9M, to be fixed properly later)
• jfjoch_writer: Fix too much verbose information
• FPGA: Minor fixes to spot finder (enable two-pass operation and limit number range to int20)

1.0.0-rc.31

This is UNSTABLE release - introducing many features, but still needs more testing. Expecting soon to put bugfix release.

• jfjoch_writer: Allow to enable overwriting existing files (not enabled by default)
• jfjoch_writer: Add new HDF5 master file format, which uses HDF5 virtual data sets and links processing results to data files (not enabled by default)
• jfjoch_viewer: Image viewer work early test version
• jfjoch_broker: Fixes to counting packets per dataset/image
• jfjoch_broker: Image buffer is accessible for outside to check images
• jfjoch_broker: error/saturated pixels and dedicated ROI "beam" can be tracked online
• jfjoch_broker: Fix bug in handling pedestal G1/G2 count time for JUNGFRAU
• jfjoch_broker: Fix bug in applying pixel mask interfering with pedestal calculation
• jfjoch_broker: Fix bug in EIGER initializing
• jfjoch_broker: Save maximum pixel value to HDF5 file and export as Web plot
• PCIe driver: Add PCIe link speed and width
• FPGA: Improve counting error/saturated/min/max pixels
• FPGA: Spot finder is gradual column-wise (15 columns up/down) and fixed row-wise (32 pixel boxes); previously it was fixed both column- and row-wise with 32x32 pixel areas
• FPGA: Require Vivado 2022.2

Warning: There are breaking changes to HDF5 file format, renaming entries regarding image storage cell number and image collection efficiency.

1.0.0-rc.30

• jfjoch_writer: replace non-blocking with blocking operation on internal queues - less likely to "loose" images within the writer

1.0.0-rc.29

• jfjoch_broker: refactor logic regarding frame time and count time for more flexibility for EIGER and JUNGFRAU
• jfjoch_broker: readout time for EIGER is 3 us and JUNGFRAU is 20 us, this can be changed in input file
• jfjoch_broker: OpenAPI interface includes more ways to provide information on the status (error/warning/info)
• jfjoch_broker: ROIs handling via OpenAPI and frontend is more user friendly

Warning - two breaking changes to OpenAPI:

• Handling of ROIs is through /config/roi path only for both circle and box ROIs, path in /roi are no longer accessible
• broker_status structure introduced in 1.0.0-rc.28 has member message and not error_message to allow handling info/warning messages as well

1.0.0-rc.28

• jfjoch_broker: save error message for initialization and data collection and provide these with OpenAPI
• jfjoch_broker: fixed issue when in error state, response to /wait_till_done was not complaint to OpenAPI specs
• jfjoch_test: remove header that failed when CUDA is absent during compilation
• frontend: add soft trigger button in data collection tab
• frontend: show error message when in error state
• CMake: add option to force compilation without CUDA (-DJFJOCH_USE_CUDA=OFF)

1.0.0-rc.27

• jfjoch_broker: add option to select electron source in instrument metadata, adapt wavelength calculation
• jfjoch_broker: update pistache web server version
• jfjoch_writer: minor changes to republish logic
• Improvements to documentation

1.0.0-rc.26

• jfjoch_broker: implement ZeroMQ stream for image metadata information
• jfjoch_broker: refactor ZeroMQ stream for preview: start/end messages always sent
• jfjoch_broker: add crystal lattice plots
• jfjoch_broker: remove empty bins from the plots
• jfjoch_broker: Fix bugs in ModuleSummation and MXAnalyzer for CPU "long" summation
• jfjoch_broker: Fix bug when mean background estimation / indexing rate where affected by previous experiment
• jfjoch_writer: fix missing "-w" parameter
• jfjoch_writer: temporary files have ".tmp" suffix
• jfjoch_writer: refactor logic for watermarks
• jfjoch_writer: report on internal FIFO utilization
• jfjoch_writer: clean-up naming for azimuthal integration and background estimate
• jfjoch_writer: write final background estimate and indexing rate in the master file
• tools/: remove unnecessary tools, make naming consistent
• CBOR: Add indexing rate and background estimate to end message
• CBOR: Clean-up documentation

1.0.0-rc.25

• Updates to documentation
• License set to GPLv3 / OHL-S
• Fix bug in DiffractionExperiment::GetDefaultPlotBinning() - resulting in division by 0 if image time longer than 500ms
• Add information on JUNGFRAU conversion and geometry transformation to CBOR and HDF5

1.0.0-rc.24

New FPGA functionality:

• EIGER supports 8, 16 and 32-bit data input (for 8-bit mode at half performance; for 32-bit "real" depth is 23-bit + 1-bit signed)
• Output possible to 8, 16 and 32-bit data
• Threshold is applied before summation
• Pixel mask can be applied on FPGA
• Mark pixels with ADC content = 0 as bad pixels
• FPGA stores semantic version information (access via /sys/class/misc/jfjoch.../version)

New software functionality:

• Long summation (above 256 frames) done on CPU
• Mechanism to save arbitrary data to HDF5 file
• ZeroMQ preview has option to send start message
• Rework pixel mask + add statistics displayed in web interface

Bug fixes:

• Web frontend: Update preview image automatically during data acquisition
• jfjoch_broker: Error handling if CUDA driver is not installed
• jfjoch_broker: Correctly update progress during pedestal
• jfjoch_broker: Provide proper error when uploaded file is not a proper TIFF
• jfjoch_action_test: enable HLS simulation

Documentation improvement and placement in a dedicated directory