Jungfraujoch/docs/CHANGELOG.md

Changelog

1.0.0

1.0.0-rc.124

This is an UNSTABLE release. This version significantly rewrites code to predict reflection position and integrate them, especially in case of rotation crystallography. If things go wrong with analysis, it is better to revert to 1.0.0-rc.123.

• jfjoch_broker: Improve refection position prediction and Bragg integration code.
• jfjoch_broker: Align with XDS way of calculating Lorentz correction and general notation.
• jfjoch_writer: Fix saving mosaicity properly in HDF5 file.
• jfjoch_viewer: Introduce high-dynamic range mode for images
• jfjoch_viewer: Ctrl+mouse wheel has exponential change in foreground (+/-15%)
• jfjoch_viewer: Zoom-in numbers have better readability

1.0.0-rc.123

This is an UNSTABLE release.

• jfjoch_broker: Use newer version of Google Ceres for (potential) CUDA 13 compatibility
• jfjoch_broker: Improve performance of generating preview images, especially for large detectors (9M-16M)
• jfjoch_viewer: Improve performance of displaying images, especially for large detectors (9M-16M)
• jfjoch_viewer: Add more color schemes for better image readability
• HDF5: Common mutex for reading and writing HDF5 if both operations were to happen in the same executable
• HDF5: suppress warning if path (upstream group) doesn't exists when checking if leaf exists

1.0.0-rc.122

This is an UNSTABLE release.

• jfjoch_broker: Add thresholding to prefer shorter vectors after FFT
• jfjoch_broker: Add experimental mosaicity estimation for rotation experiments (this is work in progress)
• jfjoch_broker: Update nlohmann::json to 3.12.0
• jfjoch_viewer: Display file opening errors
• jfjoch_viewer: When loading files over DBus add retry/back-off till the file is available

1.0.0-rc.121

This is an UNSTABLE release.

• jfjoch_broker: Report changes in the image buffer, so viewer doesn't reload constantly
• jfjoch_viewer: Improve performance of loading images
• jfjoch_viewer: Auto-throttle image loading in HTTP-sync / movie modes
• jfjoch_viewer: Auto-foreground calculated with histogram
• jfjoch_viewer: Fix rare segmentation fault

1.0.0-rc.120

This is an UNSTABLE release.

• jfjoch_broker: Improve performance of binary plot export

1.0.0-rc.119

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_broker: Add binary export of data analysis plots over OpenAPI
• jfjoch_broker: Minor fixes to HTTP error handling
• jfjoch_viewer: Prefer binary plots over JSON plots
• jfjoch_viewer: Change foreground with F button + wheel
• jfjoch_viewer: Change way how angles are displayed
• jfjoch_viewer: Display resolution of the mouse cursor in top left corner

1.0.0-rc.118

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_viewer: Fix issue when HTTP sync silently disconnected when it was enabled when the broker was starting measurement.
• jfjoch_broker: Add protections on time of geometry optimization and reduce rotation recalculations

1.0.0-rc.117

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_viewer: Add ROI results to the dataset info plots
• jfjoch_writer: Remove HTTP interface, as it is not needed/used at the moment

1.0.0-rc.116

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_viewer: Add binning options in the context menu

1.0.0-rc.115

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_broker: Default spot finding settings can be configured via config JSON
• jfjoch_viewer: FFT analysis of data in the dataset plot

1.0.0-rc.114

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_broker: Fix generating JPEG images with resolution estimation

1.0.0-rc.113

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_broker: Improve handling of rotation indexing
• jfjoch_broker: More information saved in CBOR end message (WIP)
• jfjoch_writer: Save rotation indexing lattice parameters and Niggli class
• jfjoch_viewer: Remove (for now) primitive cell information
• jfjoch_viewer: Use angle for dataset info plot for rotation scans

1.0.0-rc.112

This is an UNSTABLE release and not recommended for production use (please use rc.111 instead).

• jfjoch_broker: Experimental rotation (3D) indexing
• jfjoch_broker: Minor fix to error in optimizer potentially returning NaN values

1.0.0-rc.111

This is an UNSTABLE release.

• jfjoch_viewer: Remove 3D lattice viewer (not really useful at this moment)
• jfjoch_viewer: Fix auto contrast not refreshing image

1.0.0-rc.110

This is an UNSTABLE release.

• jfjoch_broker: Add auto-contrast option for preview images
• Frontend: Add logo image
• jfjoch_viewer: Add logo image
• jfjoch_viewer: For image chart allow to set min value to zero
• jfjoch_viewer: For resolution estimation plots, visualization uses 1/d^2 as measure
• jfjoch_viewer: Add 3D unit cell visualization (experimental/WIP/not really there)
• Documentation: Add logo image

1.0.0-rc.109

This is an UNSTABLE release.

• jfjoch_viewer: Add keyboard shortcuts and option to copy image to clipboard
• jfjoch_broker: Fix bit-width and exposure time for PSI EIGER detectors

1.0.0-rc.108

This is an UNSTABLE release.

• jfjoch_viewer: Fix bug when resolution estimation/B-Factor/Profile radius were not set (NaN)
• jfjoch_viewer: Show spots is off by default, resolution ring mode is enabled by default
• jfjoch_viewer: Fit to window of image is now default when size of the grid changes

1.0.0-rc.107

This is an UNSTABLE release.

• jfjoch_viewer: Minor polishing of new functionality
• jfjoch_broker: User NaN for empty azimuthal bins

1.0.0-rc.106

This is an UNSTABLE release.

• jfjoch_viewer: Allow for multiple dataset info plots
• jfjoch_viewer: Highlight current element in grid

1.0.0-rc.105

This is an UNSTABLE release.

• jfjoch_viewer: Clean-up widgets slightly
• jfjoch_viewer: Limit right panel to 600 pixels
• jfjoch_viewer: Parse crystal symmetry type
• jfjoch_viewer: Grid scan view takes color map and can be fit to zoom

1.0.0-rc.104

This is an UNSTABLE release.

• jfjoch_writer: Fix and improve the way grid scan geometry is saved (non-NXmx extension makes it way easier)
• jfjoch_viewer: Display grid scan results in 2D (work in progress)
• jfjoch_viewer: Improve auto-scaling on start of images (work in progress)
• jfjoch_viewer: Add B-factor and resolution estimate to the dataset info plots

1.0.0-rc.103

This is an UNSTABLE release.

• jfjoch_viewer: Minor improvements to the viewer
• jfjoch_broker: Change behavior for modular detectors: coordinates of 0-th pixel can be now arbitrary and detector will be cropped to the smallest rectangle limited by module coordinates

1.0.0-rc.102

This is an UNSTABLE release.

• jfjoch_viewer: Minor improvements to the viewer

1.0.0-rc.101

This is an UNSTABLE release.

• jfjoch_viewer: Auto load is better handling change of states
• jfjoch_viewer: Fix DBus registration
• jfjoch_viewer: Handle charts better with vertical lines on hover and status bar update
• jfjoch_viewer: Calculate ROI in a more efficient way

1.0.0-rc.100

This is an UNSTABLE release.

• jfjoch_viewer: Fix dbus registration
• jfjoch_viewer: Remove background slider for diffraction image
• jfjoch_viewer: Adjustments for 2D azimuthal image viewer

1.0.0-rc.99

This is an UNSTABLE release.

• jfjoch_broker: Fix output during mask data collection

1.0.0-rc.98

This is an UNSTABLE release and not recommended for production use (please use rc.96 instead).

• jfjoch_broker: For DECTRIS detectors fix dark data collection during initialization

1.0.0-rc.97

This is an UNSTABLE release and not recommended for production use (please use rc.96 instead).

• jfjoch_broker: For DECTRIS detectors add dark data collection during initialization for bad pixel mask
• jfjoch_broker: Refactor of calibration logic for more clear code (likely to introduce problems)
• jfjoch_viewer: Add option to handle user pixel mask (experimental)
• jfjoch_viewer: More options for ROI
• jfjoch_viewer: Add window to display calibration

1.0.0-rc.96

This is an UNSTABLE release.

• Fixes in CI pipeline
• jfjoch_broker: Remove PNG preview, no dependency on libpng
• jfjoch_writer: Fix UTC timestamp being generated wrong (mix between milli- and microseconds)
• jfjoch_viewer: Show data collection time in dataset tooltip
• jfjoch_viewer: Allow to choose the calibrant (presets for LaB6 and silver behenate)
• jfjoch_viewer: Auto foreground value
• Use external libjpeg-turbo and libtiff: simpler build stack, these are built and linked statically in automated Docker builds
• Remove OpenBLAS dependency

1.0.0-rc.95

This is an UNSTABLE release.

• Fixes in CI pipeline
• Add git-lfs to Rocky8 docker image

Previous releases (91-94) had a wrong FPGA image upload to Gitlab release. This is now solved.

1.0.0-rc.94

This is an UNSTABLE release.

• FFTIndexer: Add limit on angles to avoid colinear vectors
• Docker images: Add 3D Qt
• Gitea: Fixes to the pipeline

1.0.0-rc.93

This is an UNSTABLE release.

• CI: Fixes to Gitlab based pipeline
• PCIe driver: Fix PCIe revision being hex number

1.0.0-rc.92

This is an UNSTABLE release.

• jfjoch_broker: Fix code that predicted Bragg reflections scattering back from the sample.

1.0.0-rc.91

This is an UNSTABLE release. This release introduces new features, which usually means these need more field testing before enough maturity. For production use we recommend waiting for a future bug-fix release.

• FPGA: Implement high pixel value threshold - pixels above the given value will be considered saturated
• jfjoch_broker: Spot finding and integration predictions are ported to a GPU
• jfjoch_broker: Estimate resolution
• jfjoch_broker: Lattice search
• jfjoch_broker: Many more improvements in image analysis

1.0.0-rc.90

This is an UNSTABLE release.

• jfjoch_broker: for indexing min index spots for a viable cells can be changed via OpenAPI
• jfjoch_viewer: Optional auto-reanalyze images
• jfjoch_writer: Add option where no files at all are saved
• Documentation: improvements

1.0.0-rc.89

This is an UNSTABLE release.

• jfjoch_broker: Fix resolution estimation code
• jfjoch_broker: Fix Wilson B-factor calculation code
• jfjoch_viewer: Improve display of plots
• jfjoch_viewer: Fix segmentation fault
• jfjoch_viewer: Display missing metadata when using HTTP
• jfjoch_viewer: Fix bug when opening the same file twice

1.0.0-rc.88

This is an UNSTABLE release.

• jfjoch_viewer: Add resolution estimation to the image information
• jfjoch_broker: Minor changes to resolution estimate routine

1.0.0-rc.87

This is an UNSTABLE release.

• jfjoch_viewer: Display more image metadata (angle / exposure time)
• jfjoch_viewer: Improve I/sigma and B-factor plots
• jfjoch_broker: Estimate resolution based on visible spots

1.0.0-rc.86

This is an UNSTABLE release.

• jfjoch_broker: Update logic when initializing detector to make it a bit more resilient
• Gitea pipelines have nocuda option for all architectures

1.0.0-rc.85

This is an UNSTABLE release.

• jfjoch_viewer: When using online view, dataset info plots are not switched back to the first category for each image
• jfjoch_viewer: Handle spot count better in dataset info plots
• jfjoch_viewer: Highlight spots in ice ring resolutions in cyan, when detection is enabled

1.0.0-rc.84

This is an UNSTABLE release.

• jfjoch_broker: Write in log which detector is being initialized
• Changes to automated build system

1.0.0-rc.83

This is an UNSTABLE release.

• jfjoch_viewer: Fix in generating preview image for signed data (wrong bit-width was assumed before)
• CI: Fix script to generate python client

1.0.0-rc.82

This is an UNSTABLE release.

• jfjoch_viewer: Enable FFTW based indexing in viewer (very slow at the moment)
• Frontend: Minor fixes
• Build scripts: Minor fixes to FFTW

1.0.0-rc.81

This is an UNSTABLE release. This release introduces new features, which usually means these need more field testing before enough maturity. For production use we recommend waiting for a future bug-fix release.

• jfjoch_broker: Add option to detect ice rings, adjust width of ice ring and change of logic to exclude ice rings in indexing
• jfjoch_broker: Add FFTW based indexer for CPU only indexing
• jfjoch_broker: Enable saving X-ray fluorescence spectra
• jfjoch_writer: Write total spot count (before filtering)
• jfjoch_viewer: Add more information on source, sample, and buttom to show ice rings
• jfjoch_viewer: Enable data processing inside the viewer

CI: Moving from Gitlab to Gitea at PSI

1.0.0-rc.80

This is an UNSTABLE release.

• jfjoch_broker: Fix bug when wrong value for a plot (NaN or infinity) would lead to a null in a plot, which cannot be parsed by viewer

1.0.0-rc.79

This is an UNSTABLE release.

• jfjoch_viewer: Fix bug when loading new dataset was creating a cascade of signals leading to poor performance
• jfjoch_writer: Save nimages_per_trigger in detectorSpecific

1.0.0-rc.78

This is an UNSTABLE release.

• jfjoch_viewer: Using a single event loop (reading images is not in dedicated thread anymore)

1.0.0-rc.77

This is an UNSTABLE release.

• jfjoch_viewer: Display detector and dataset settings with tooltips
• jfjoch_viewer: Clean excessive HDF5 warnings
• jfjoch_viewer: Display unit cell
• jfjoch_extract_hkl: Write a tool to extract reflection intensity from a dataset

1.0.0-rc.76

This is an UNSTABLE release.

• jfjoch_broker: Increase predicted hkl to 100.0, use lighter math to exclude too-high resolution ones
• jfjoch_broker: Use standard deviation formula to find profile radius (not the one using median)
• jfjoch_writer: Save space group number (non-NXmx addition) in addition to name
• jfjoch_viewer: Fix the bug on reading space_group as string
• jfjoch_viewer: Add missing resolution labels on rings
• jfjoch_viewer: Remove Q value from the status bar

1.0.0-rc.75

This is an UNSTABLE release.

• jfjoch_broker: EIGER2 missing minimum threshold - hardcoded to 2.7 keV for the time being

1.0.0-rc.74

This is an UNSTABLE release.

• jfjoch_broker: Fix for EIGER UDP port settings (vertical half of the module missing)
• jfjoch_broker: Detector settings were not applied for EIGER/DECTRIS detector when changed after initialization

1.0.0-rc.73

This is an UNSTABLE release.

• jfjoch_broker: Space group number treatment in OpenAPI was wrong, zero value is no longer allowed and no longer default

1.0.0-rc.72

This is an UNSTABLE release. This release introduces new features, which usually means these need more field testing before enough maturity. For production use we recommend waiting for a future bug-fix release.

• jfjoch_broker: Refactor of indexing and geometry refinement code
• jfjoch_broker: Handle space group/centering in refinement code
• jfjoch_broker: Replace mosaicity with profile radius: refining the former is difficult with still images
• jfjoch_broker: There is no longer 0.5 pxl offset for spots-to-reciprocal-space conversion
• jfjoch_writer: Experimental saving of reflections
• jfjoch_writer: Save space group name as string
• jfjoch_viewer: Add profile radius and B-factor
• jfjoch_viewer: Show 4 digits for wavelength
• jfjoch_viewer: Match rings between calibrant and observation (will handle missing/wrong rings)
• FPGA: Use UDP destination port to distinguish between detector modules and data streams
• FPGA: Add experimental PTP core (PTP over L2, only Sync/Follow_up)
• FPGA driver: Fix for Linux kernel 6.12+ (thanks to Tim Gruene)

1.0.0-rc.71

This is an UNSTABLE release.

• jfjoch_broker: Remove resolution estimation via machine learning
• jfjoch_broker: Harmonize code to analyze spot finding results (indexing/refinement/integration) between CPU and FPGA receivers
• jfjoch_viewer: Fix error when HDF5 files with indexing results couldn't be loaded on a machine without GPU

1.0.0-rc.70

This is an UNSTABLE release. This release introduces new features (geometry refinement), which usually means these need more field testing before enough maturity. For production use we recommend waiting for a future bug-fix release.

• jfjoch_broker: Fix bug when PSI EIGER frame time was not set properly at the start of the measurement

• jfjoch_broker: Fix PONI rot2 angle rotating detector in a wrong direction (PyFAI convention is for this angle to rotate detector downwards)

• jfjoch_broker: Enable geometry refinement - first try (work in progress)

• jfjoch_viewer: Fix deadlock when opening HTTP connections

• jfjoch_viewer: Display rings as ellipses with detector tilt

• jfjoch_viewer: Add button to calibrate detector geometry based on LaB6 image

• jfjoch_writer: Save detector tilt angles (rot1, rot2, rot3)

• Add Google Ceres a non-linear least-square optimization library to Jungfraujoch

• Add experimental detector calibration routines (for LaB6)

• Improve documentation on the ZeroMQ writer notification socket and detector geometry

1.0.0-rc.69

This is an UNSTABLE release.

• jfjoch_viewer: Metadata can be modified for an open dataset (no option to save)
• jfjoch_viewer: Refactor multiple issues in the viewer regarding image reading code to allow for further developments
• jfjoch_viewer: Resolution rings not enabled by default
• jfjoch_broker: Handle properly PONI rotations in dataset settings though still not updated properly in the HDF5 file

1.0.0-rc.68

This is an UNSTABLE release.

• jfjoch_broker: Temperature threshold can be changed for JUNGFRAU detector
• jfjoch_broker: Default detector settings can be configured for each detector separately
• jfjoch_broker: Refactor spot filtering code, max spot count can be modified for dataset settings
• jfjoch_broker: Refactor indexing refinement, make it the same for both FFBIDX and FFT indexing
• jfjoch_broker: Reference unit cell will be taken into account for FFT indexing to filter
• jfjoch_broker: Review PONI rotation angles and azimuthal angle conventions along with PyFAI

1.0.0-rc.67

This is an UNSTABLE release.

• jfjoch_broker: Enable SSL
• jfjoch_broker: Wilson B-factor only provided is fit is relatively OK (R^2 > 0.3); this will be refined much more in the future

1.0.0-rc.66

This is an UNSTABLE release.

• jfjoch_broker: Indexers operate as thread pool, which is operating
• jfjoch_viewer: Increase interval between loading images + fix too many verbose messages

1.0.0-rc.65

This is an UNSTABLE release.

• jfjoch_broker: Print information regarding used image pushers
• jfjoch_viewer: Allow syncing with Jungfraujoch server
• OpenAPI: Clarify licensing terms in the file

1.0.0-rc.64

This is an UNSTABLE release.

• jfjoch_broker: Fix issue in receiver light with very long preparation time for threads
• jfjoch_broker: Add verbose option
• jfjoch_broker: Don't trigger pedestal if critical settings are not changed when loading detector settings
• jfjoch_broker: Detector left in busy state when detector settings were improper
• jfjoch_viewer: Modify DBus interface to avoid loading same file and image 0 multiple times
• jfjoch_lite_perf_test: Add verbose option

1.0.0-rc.63

This is an UNSTABLE release.

• jfjoch_broker: Save NX/NY for grid scan result
• jfjoch_broker: Add processing time to CBOR output and plot
• jfjoch_writer: Add processing time to data file

1.0.0-rc.62

This is an UNSTABLE release.

• jfjoch_broker: Fix bug where low resolution spots were not counted properly
• jfjoch_broker: Spot count is provided prior to filtering of spots to max_spot_count
• jfjoch_broker: Add more spot count information to CBOR
• jfjoch_viewer: Fix issue with ROI drawing resulting in multiple overlapping rectangles

1.0.0-rc.61

This is an UNSTABLE release.

• jfjoch_broker: Fix bug where FFT indexing could result in a very short or even zero length vector
• jfjoch_broker: Ice ring and indexed spot count enabled as plots and saved in grid scan results
• jfjoch_broker: High resolution limit for low res. spot counting can be adjusted

1.0.0-rc.60

This is an UNSTABLE release.

• jfjoch_broker: Fix bug when the neural network inference client was busy and this status was never released
• jfjoch_broker: Revert the indexing threshold with distance from integer for Miller indices
• jfjoch_broker: Fix bug in scattering vector calculation, resulting in indexing not working outside 1.0 A X-ray wavelength

1.0.0-rc.59

This is an UNSTABLE release.

• jfjoch_broker: Fix bug when broker was waiting for notification message before sending end message, resulting in deadlock.
• jfjoch_writer: Verbose option for debugging.

1.0.0-rc.58

This is an UNSTABLE release.

• jfjoch_viewer: Fix memory leak
• jfjoch_writer: Add detector_number/serial_number to master file

1.0.0-rc.57

This is an UNSTABLE release.

• jfjoch_broker: Fix bug when enabling ML resolution estimation was not possible
• jfjoch_viewer: "Movie" mode

1.0.0-rc.56

This is an UNSTABLE release.

• jfjoch_broker: Fixing more bugs related to neural network inference for ML estimation

1.0.0-rc.55

This is an UNSTABLE release.

• jfjoch_broker: Fixing minor bugs related to neural network inference for ML estimation

1.0.0-rc.54

This is an UNSTABLE release.

• jfjoch_broker: Indexing with AUTO settings (FFBIDX if unit cell provided; FFT if not)
• jfjoch_broker: Don't remove shared memory area when deactivating detector
• jfjoch_writer: Save writer release
• jfjoch_viewer: Increase time for the messages in the status bar

1.0.0-rc.53

This is an UNSTABLE release.

• PCIe driver: Imperfect solution for RHEL 9.5+ changes
• jfjoch_writer: Fix to angle containers for AutoProc compatibility
• jfjoch_fpga_test: Use consecutive number for devices, not interleaved

1.0.0-rc.52

This is an UNSTABLE release.

• jfjoch_viewer: Use warmer colors to distinguish from AareGUI
• jfjoch_viewer: Minor adjustments to DBus setting image number
• jfjoch_broker: Fix in low resolution spot count plotting

1.0.0-rc.51

This is an UNSTABLE release.

• jfjoch_broker: Send preview in PNG format
• jfjoch_broker: Provide count of spots in 50.0 - 5.0 A range
• jfjoch_broker: Provide ML resolution estimation in scan result
• jfjoch_broker: Allow removing beam center in web preview

1.0.0-rc.50

This is an UNSTABLE release.

• The release fixes some of many bugs introduced in recent releases
• jfjoch_viewer: display predictions for indexed cells

1.0.0-rc.49

This is an UNSTABLE release.

• jfjoch_broker: Handle sample temperature (K) and ring current (mA) to metadata
• jfjoch_writer: For angle containers in NXmx add _end dataset, sample temp. and ring current

1.0.0-rc.48

This is an UNSTABLE release.

• jfjoch_broker: fix the bug when a unit cell was not exported for a scan result.

1.0.0-rc.47

This is an UNSTABLE release.

• jfjoch_viewer: fix dbus service path
• jfjoch_writer: fix CBF/TIFF writing

1.0.0-rc.46

This is an UNSTABLE release.

• jfjoch_viewer: remove dependency on image analysis

1.0.0-rc.45

This is an UNSTABLE release.

• jfjoch_broker: Detector list returns pixel size (mm)

1.0.0-rc.44

This is an UNSTABLE release.

• jfjoch_broker: more general definition of scan result export

Braking changes:

• It removes additions to OpenAPI from 1.0.0-rc.43
• It makes changes to the "unit_cell" definition in OpenAPI specs. It might be harmless in some languages and may result in errors in other implementations.

1.0.0-rc.43

This is an UNSTABLE release.

• jfjoch_broker: Export grid scan results into a single data structure

1.0.0-rc.42

This is an UNSTABLE release.

• jfjoch_broker: Add pixel_sum to CBOR output.
• jfjoch_broker: Changes to sigma estimation in QuickIntegrate routine
• jfjoch_writer: Save pixel_sum

1.0.0-rc.41

This is an UNSTABLE release. This release includes multiple new features, it should not be used in production at the moment.

• jfjoch_broker: Estimate B-factor, mosaicity to evaluate crystal diffraction

• jfjoch_broker: Export GPU count via OpenAPI

• jfjoch_broker: Enable 2D azimuthal integration and PONI rotations for detector

• FPGA: Increase the number of integration bins to 2048

1.0.0-rc.40

This is an UNSTABLE release. This release includes multiple new features, it should not be used in production at the moment.

• jfjoch_broker: Jungfraujoch supports grid scan metadata, including dedicated plotting schemes and NXmx structures

• jfjoch_broker: Improve metadata for rotation data collection

• jfjoch_broker: Better handling of plotting

• jfjoch_broker: FFT based indexing

• jfjoch_broker: Integration, first try, results not saved at the moment

• jfjoch_broker: Internal improvements in image handling

• jfjoch_writer: Multiple adjustments adapt to changes in this release for new features

• jfjoch_writer: New state management model to improve clarity of error reporting

• jfjoch_viewer: Remote control via DBus

• Frontend: Multiple adjustments for new features

• Frontend: Grid scan plots

WARNING! OpenAPI contains breaking changes in regard to plotting results, so care has to be taken.

1.0.0-rc.39

• FPGA: Bugfix for pixel masked for data analysis if summation was on
• jfjoch_viewer: Fix segmentation fault when cursor was outside of image

1.0.0-rc.38

• jfjoch_broker: Neural net model is not linked with C++ code due to deployment issues, it is rather distributed as python code, connected via RES
• jfjoch_broker: Neural net model can use all 4 quadrants of the detector
• jfjoch_broker: For EIGER image time can be provided through /start
• jfjoch_viewer: Add image list option
• jfjoch_viewer: Drawing circular ROIs with shift
• jfjoch_viewer: Enable image summation
• jfjoch_viewer: Image reader is significantly reworked, hopefully without affecting the viewer

1.0.0-rc.37

• jfjoch_broker: Make locking rules more flexible
• jfjoch_broker: Load mask via SIMPLON interface for DECTRIS detectors
• jfjoch_viewer: Add status bar

1.0.0-rc.36

This is UNSTABLE release. Wait for new version to use in a production environment.

• jfjoch_broker: Support for Jungfraujoch Lite is enabled - software-based receiver for DECTRIS detectors (required a lot of refactoring, potentially leading to unstable code)
• jfjoch_broker: Enable Resonet support (ML-based diffraction resolution estimation)
• jfjoch_broker: Fix error in compression, where bitshuffle/LZ4 and bitshuffle/Zstd HDF5 headers were wrongly generated for 8-bit and 32-bit data
• jfjoch_writer: Increase buffering to 1000 images in the receiver
• jfjoch_writer: Images can be written as CBF or TIFF in addition to HDF5

1.0.0-rc.35

This is UNSTABLE release, not properly tested. Wait for new version for using production.

• jfjoch_broker: If module is delayed by more than 50 frames versus other modules, it will be ignored and receiver is not waiting.
• jfjoch_writer: Save EIGER energy threshold
• jfjoch_writer: Add /entry/sample/goniometer for compatibility with eiger2cbf program

1.0.0-rc.34

This is UNSTABLE release - introducing new features, but not properly tested. Wait for new version for using production.

• jfjoch_broker: More consistency for file format definition (breaking change in API from 1.0.0-rc.31 for file writer settings)
• jfjoch_broker: For storage cells mask is logical sum of detector bad pixels for all storage cells
• jfjoch_broker: Handle situation when detector doesn't want to gracefully stop (to be tested)
• jfjoch_broker: Center-of-mass position and mean for ROI is added to available plots
• jfjoch_viewer: Can extract data analysis results from "legacy" format
• jfjoch_viewer: Display dataset name
• FPGA: Pixel mask is used for data analysis part even if it is not applied to pixels
• FPGA: Add pixel sum to module statistics
• FPGA: ROI number is reduced to 16, but pixel can belong to every defined ROI
• FPGA: Spot finder is back to full dynamic range (24-bit)
• FPGA: More debug features for internal FIFOs

Known issues:

• ROI count flag was added to firmware. For the time being the flag will be wrongly set to 10 due to mismatch of FPGA build scripts.
• EIGER data acquisition has an issue that is currently debugged

1.0.0-rc.33

• jfjoch_broker: Fix issue with EIGER settings being loaded improperly

1.0.0-rc.32

• jfjoch_broker: Refactor code for azimuthal integration for further improvements
• jfjoch_broker: Minor fix for EIGER (trim energies are manually set for E9M, to be fixed properly later)
• jfjoch_writer: Fix too much verbose information
• FPGA: Minor fixes to spot finder (enable two-pass operation and limit number range to int20)

1.0.0-rc.31

This is UNSTABLE release - introducing many features, but still needs more testing. Expecting soon to put bugfix release.

• jfjoch_writer: Allow to enable overwriting existing files (not enabled by default)
• jfjoch_writer: Add new HDF5 master file format, which uses HDF5 virtual data sets and links processing results to data files (not enabled by default)
• jfjoch_viewer: Image viewer work early test version
• jfjoch_broker: Fixes to counting packets per dataset/image
• jfjoch_broker: Image buffer is accessible for outside to check images
• jfjoch_broker: error/saturated pixels and dedicated ROI "beam" can be tracked online
• jfjoch_broker: Fix bug in handling pedestal G1/G2 count time for JUNGFRAU
• jfjoch_broker: Fix bug in applying pixel mask interfering with pedestal calculation
• jfjoch_broker: Fix bug in EIGER initializing
• jfjoch_broker: Save maximum pixel value to HDF5 file and export as Web plot
• PCIe driver: Add PCIe link speed and width
• FPGA: Improve counting error/saturated/min/max pixels
• FPGA: Spot finder is gradual column-wise (15 columns up/down) and fixed row-wise (32 pixel boxes); previously it was fixed both column- and row-wise with 32x32 pixel areas
• FPGA: Require Vivado 2022.2

Warning: There are breaking changes to HDF5 file format, renaming entries regarding image storage cell number and image collection efficiency.

1.0.0-rc.30

• jfjoch_writer: replace non-blocking with blocking operation on internal queues - less likely to "loose" images within the writer

1.0.0-rc.29

• jfjoch_broker: refactor logic regarding frame time and count time for more flexibility for EIGER and JUNGFRAU
• jfjoch_broker: readout time for EIGER is 3 us and JUNGFRAU is 20 us, this can be changed in input file
• jfjoch_broker: OpenAPI interface includes more ways to provide information on the status (error/warning/info)
• jfjoch_broker: ROIs handling via OpenAPI and frontend is more user friendly

Warning - two breaking changes to OpenAPI:

• Handling of ROIs is through /config/roi path only for both circle and box ROIs, path in /roi are no longer accessible
• broker_status structure introduced in 1.0.0-rc.28 has member message and not error_message to allow handling info/warning messages as well

1.0.0-rc.28

• jfjoch_broker: save error message for initialization and data collection and provide these with OpenAPI
• jfjoch_broker: fixed issue when in error state, response to /wait_till_done was not complaint to OpenAPI specs
• jfjoch_test: remove header that failed when CUDA is absent during compilation
• frontend: add soft trigger button in data collection tab
• frontend: show error message when in error state
• CMake: add option to force compilation without CUDA (-DJFJOCH_USE_CUDA=OFF)

1.0.0-rc.27

• jfjoch_broker: add option to select electron source in instrument metadata, adapt wavelength calculation
• jfjoch_broker: update pistache web server version
• jfjoch_writer: minor changes to republish logic
• Improvements to documentation

1.0.0-rc.26

• jfjoch_broker: implement ZeroMQ stream for image metadata information
• jfjoch_broker: refactor ZeroMQ stream for preview: start/end messages always sent
• jfjoch_broker: add crystal lattice plots
• jfjoch_broker: remove empty bins from the plots
• jfjoch_broker: Fix bugs in ModuleSummation and MXAnalyzer for CPU "long" summation
• jfjoch_broker: Fix bug when mean background estimation / indexing rate where affected by previous experiment
• jfjoch_writer: fix missing "-w" parameter
• jfjoch_writer: temporary files have ".tmp" suffix
• jfjoch_writer: refactor logic for watermarks
• jfjoch_writer: report on internal FIFO utilization
• jfjoch_writer: clean-up naming for azimuthal integration and background estimate
• jfjoch_writer: write final background estimate and indexing rate in the master file
• tools/: remove unnecessary tools, make naming consistent
• CBOR: Add indexing rate and background estimate to end message
• CBOR: Clean-up documentation

1.0.0-rc.25

• Updates to documentation
• License set to GPLv3 / OHL-S
• Fix bug in DiffractionExperiment::GetDefaultPlotBinning() - resulting in division by 0 if image time longer than 500ms
• Add information on JUNGFRAU conversion and geometry transformation to CBOR and HDF5

1.0.0-rc.24

New FPGA functionality:

• EIGER supports 8, 16 and 32-bit data input (for 8-bit mode at half performance; for 32-bit "real" depth is 23-bit + 1-bit signed)
• Output possible to 8, 16 and 32-bit data
• Threshold is applied before summation
• Pixel mask can be applied on FPGA
• Mark pixels with ADC content = 0 as bad pixels
• FPGA stores semantic version information (access via /sys/class/misc/jfjoch.../version)

New software functionality:

• Long summation (above 256 frames) done on CPU
• Mechanism to save arbitrary data to HDF5 file
• ZeroMQ preview has option to send start message
• Rework pixel mask + add statistics displayed in web interface

Bug fixes:

• Web frontend: Update preview image automatically during data acquisition
• jfjoch_broker: Error handling if CUDA driver is not installed
• jfjoch_broker: Correctly update progress during pedestal
• jfjoch_broker: Provide proper error when uploaded file is not a proper TIFF
• jfjoch_action_test: enable HLS simulation

Documentation improvement and placement in a dedicated directory