Jungfraujoch/docs/CHANGELOG.md

Changelog

1.0.0-rc.54

This is an UNSTABLE release.

• jfjoch_broker: Indexing with AUTO settings (FFBIDX if unit cell provided; FFT if not)
• jfjoch_broker: Don't remove shared memory area when deactivating detector
• jfjoch_writer: Save writer release
• jfjoch_viewer: Increase time for the messages in the status bar

1.0.0-rc.53

This is an UNSTABLE release.

• PCIe driver: Imperfect solution for RHEL 9.5+ changes
• jfjoch_writer: Fix to angle containers for AutoProc compatibility
• jfjoch_fpga_test: Use consecutive number for devices, not interleaved

1.0.0-rc.52

This is an UNSTABLE release.

• jfjoch_viewer: Use warmer colors to distinguish from AareGUI
• jfjoch_viewer: Minor adjustments to DBus setting image number
• jfjoch_broker: Fix in low resolution spot count plotting

1.0.0-rc.51

This is an UNSTABLE release.

• jfjoch_broker: Send preview in PNG format
• jfjoch_broker: Provide count of spots in 50.0 - 5.0 A range
• jfjoch_broker: Provide ML resolution estimation in scan result
• jfjoch_broker: Allow removing beam center in web preview

1.0.0-rc.50

This is an UNSTABLE release.

• The release fixes some of many bugs introduced in recent releases
• jfjoch_viewer: display predictions for indexed cells

1.0.0-rc.49

This is an UNSTABLE release.

• jfjoch_broker: Handle sample temperature (K) and ring current (mA) to metadata
• jfjoch_writer: For angle containers in NXmx add _end dataset, sample temp. and ring current

1.0.0-rc.48

This is an UNSTABLE release.

• jfjoch_broker: fix the bug when a unit cell was not exported for a scan result.

1.0.0-rc.47

This is an UNSTABLE release.

• jfjoch_viewer: fix dbus service path
• jfjoch_writer: fix CBF/TIFF writing

1.0.0-rc.46

This is an UNSTABLE release.

• jfjoch_viewer: remove dependency on image analysis

1.0.0-rc.45

This is an UNSTABLE release.

• jfjoch_broker: Detector list returns pixel size (mm)

1.0.0-rc.44

This is an UNSTABLE release.

• jfjoch_broker: more general definition of scan result export

Braking changes:

• It removes additions to OpenAPI from 1.0.0-rc.43
• It makes changes to the "unit_cell" definition in OpenAPI specs. It might be harmless in some languages and may result in errors in other implementations.

1.0.0-rc.43

This is an UNSTABLE release.

• jfjoch_broker: Export grid scan results into a single data structure

1.0.0-rc.42

This is an UNSTABLE release.

• jfjoch_broker: Add pixel_sum to CBOR output.
• jfjoch_broker: Changes to sigma estimation in QuickIntegrate routine
• jfjoch_writer: Save pixel_sum

1.0.0-rc.41

This is an UNSTABLE release. This release includes multiple new features, it should not be used in production at the moment.

• jfjoch_broker: Estimate B-factor, mosaicity to evaluate crystal diffraction

• jfjoch_broker: Export GPU count via OpenAPI

• jfjoch_broker: Enable 2D azimuthal integration and PONI rotations for detector

• FPGA: Increase the number of integration bins to 2048

1.0.0-rc.40

This is an UNSTABLE release. This release includes multiple new features, it should not be used in production at the moment.

• jfjoch_broker: Jungfraujoch supports grid scan metadata, including dedicated plotting schemes and NXmx structures

• jfjoch_broker: Improve metadata for rotation data collection

• jfjoch_broker: Better handling of plotting

• jfjoch_broker: FFT based indexing

• jfjoch_broker: Integration, first try, results not saved at the moment

• jfjoch_broker: Internal improvements in image handling

• jfjoch_writer: Multiple adjustments adapt to changes in this release for new features

• jfjoch_writer: New state management model to improve clarity of error reporting

• jfjoch_viewer: Remote control via DBus

• Frontend: Multiple adjustments for new features

• Frontend: Grid scan plots

WARNING! OpenAPI contains breaking changes in regard to plotting results, so care has to be taken.

1.0.0-rc.39

• FPGA: Bugfix for pixel masked for data analysis if summation was on
• jfjoch_viewer: Fix segmentation fault when cursor was outside of image

1.0.0-rc.38

• jfjoch_broker: Neural net model is not linked with C++ code due to deployment issues, it is rather distributed as python code, connected via RES
• jfjoch_broker: Neural net model can use all 4 quadrants of the detector
• jfjoch_broker: For EIGER image time can be provided through /start
• jfjoch_viewer: Add image list option
• jfjoch_viewer: Drawing circular ROIs with shift
• jfjoch_viewer: Enable image summation
• jfjoch_viewer: Image reader is significantly reworked, hopefully without affecting the viewer

1.0.0-rc.37

• jfjoch_broker: Make locking rules more flexible
• jfjoch_broker: Load mask via SIMPLON interface for DECTRIS detectors
• jfjoch_viewer: Add status bar

1.0.0-rc.36

This is UNSTABLE release. Wait for new version to use in a production environment.

• jfjoch_broker: Support for Jungfraujoch Lite is enabled - software-based receiver for DECTRIS detectors (required a lot of refactoring, potentially leading to unstable code)
• jfjoch_broker: Enable Resonet support (ML-based diffraction resolution estimation)
• jfjoch_broker: Fix error in compression, where bitshuffle/LZ4 and bitshuffle/Zstd HDF5 headers were wrongly generated for 8-bit and 32-bit data
• jfjoch_writer: Increase buffering to 1000 images in the receiver
• jfjoch_writer: Images can be written as CBF or TIFF in addition to HDF5

1.0.0-rc.35

This is UNSTABLE release, not properly tested. Wait for new version for using production.

• jfjoch_broker: If module is delayed by more than 50 frames versus other modules, it will be ignored and receiver is not waiting.
• jfjoch_writer: Save EIGER energy threshold
• jfjoch_writer: Add /entry/sample/goniometer for compatibility with eiger2cbf program

1.0.0-rc.34

This is UNSTABLE release - introducing new features, but not properly tested. Wait for new version for using production.

• jfjoch_broker: More consistency for file format definition (breaking change in API from 1.0.0-rc.31 for file writer settings)
• jfjoch_broker: For storage cells mask is logical sum of detector bad pixels for all storage cells
• jfjoch_broker: Handle situation when detector doesn't want to gracefully stop (to be tested)
• jfjoch_broker: Center-of-mass position and mean for ROI is added to available plots
• jfjoch_viewer: Can extract data analysis results from "legacy" format
• jfjoch_viewer: Display dataset name
• FPGA: Pixel mask is used for data analysis part even if it is not applied to pixels
• FPGA: Add pixel sum to module statistics
• FPGA: ROI number is reduced to 16, but pixel can belong to every defined ROI
• FPGA: Spot finder is back to full dynamic range (24-bit)
• FPGA: More debug features for internal FIFOs

Known issues:

• ROI count flag was added to firmware. For the time being the flag will be wrongly set to 10 due to mismatch of FPGA build scripts.
• EIGER data acquisition has an issue that is currently debugged

1.0.0-rc.33

• jfjoch_broker: Fix issue with EIGER settings being loaded improperly

1.0.0-rc.32

• jfjoch_broker: Refactor code for azimuthal integration for further improvements
• jfjoch_broker: Minor fix for EIGER (trim energies are manually set for E9M, to be fixed properly later)
• jfjoch_writer: Fix too much verbose information
• FPGA: Minor fixes to spot finder (enable two-pass operation and limit number range to int20)

1.0.0-rc.31

This is UNSTABLE release - introducing many features, but still needs more testing. Expecting soon to put bugfix release.

• jfjoch_writer: Allow to enable overwriting existing files (not enabled by default)
• jfjoch_writer: Add new HDF5 master file format, which uses HDF5 virtual data sets and links processing results to data files (not enabled by default)
• jfjoch_viewer: Image viewer work early test version
• jfjoch_broker: Fixes to counting packets per dataset/image
• jfjoch_broker: Image buffer is accessible for outside to check images
• jfjoch_broker: error/saturated pixels and dedicated ROI "beam" can be tracked online
• jfjoch_broker: Fix bug in handling pedestal G1/G2 count time for JUNGFRAU
• jfjoch_broker: Fix bug in applying pixel mask interfering with pedestal calculation
• jfjoch_broker: Fix bug in EIGER initializing
• jfjoch_broker: Save maximum pixel value to HDF5 file and export as Web plot
• PCIe driver: Add PCIe link speed and width
• FPGA: Improve counting error/saturated/min/max pixels
• FPGA: Spot finder is gradual column-wise (15 columns up/down) and fixed row-wise (32 pixel boxes); previously it was fixed both column- and row-wise with 32x32 pixel areas
• FPGA: Require Vivado 2022.2

Warning: There are breaking changes to HDF5 file format, renaming entries regarding image storage cell number and image collection efficiency.

1.0.0-rc.30

• jfjoch_writer: replace non-blocking with blocking operation on internal queues - less likely to "loose" images within the writer

1.0.0-rc.29

• jfjoch_broker: refactor logic regarding frame time and count time for more flexibility for EIGER and JUNGFRAU
• jfjoch_broker: readout time for EIGER is 3 us and JUNGFRAU is 20 us, this can be changed in input file
• jfjoch_broker: OpenAPI interface includes more ways to provide information on the status (error/warning/info)
• jfjoch_broker: ROIs handling via OpenAPI and frontend is more user friendly

Warning - two breaking changes to OpenAPI:

• Handling of ROIs is through /config/roi path only for both circle and box ROIs, path in /roi are no longer accessible
• broker_status structure introduced in 1.0.0-rc.28 has member message and not error_message to allow handling info/warning messages as well

1.0.0-rc.28

• jfjoch_broker: save error message for initialization and data collection and provide these with OpenAPI
• jfjoch_broker: fixed issue when in error state, response to /wait_till_done was not complaint to OpenAPI specs
• jfjoch_test: remove header that failed when CUDA is absent during compilation
• frontend: add soft trigger button in data collection tab
• frontend: show error message when in error state
• CMake: add option to force compilation without CUDA (-DJFJOCH_USE_CUDA=OFF)

1.0.0-rc.27

• jfjoch_broker: add option to select electron source in instrument metadata, adapt wavelength calculation
• jfjoch_broker: update pistache web server version
• jfjoch_writer: minor changes to republish logic
• Improvements to documentation

1.0.0-rc.26

• jfjoch_broker: implement ZeroMQ stream for image metadata information
• jfjoch_broker: refactor ZeroMQ stream for preview: start/end messages always sent
• jfjoch_broker: add crystal lattice plots
• jfjoch_broker: remove empty bins from the plots
• jfjoch_broker: Fix bugs in ModuleSummation and MXAnalyzer for CPU "long" summation
• jfjoch_broker: Fix bug when mean background estimation / indexing rate where affected by previous experiment
• jfjoch_writer: fix missing "-w" parameter
• jfjoch_writer: temporary files have ".tmp" suffix
• jfjoch_writer: refactor logic for watermarks
• jfjoch_writer: report on internal FIFO utilization
• jfjoch_writer: clean-up naming for azimuthal integration and background estimate
• jfjoch_writer: write final background estimate and indexing rate in the master file
• tools/: remove unnecessary tools, make naming consistent
• CBOR: Add indexing rate and background estimate to end message
• CBOR: Clean-up documentation

1.0.0-rc.25

• Updates to documentation
• License set to GPLv3 / OHL-S
• Fix bug in DiffractionExperiment::GetDefaultPlotBinning() - resulting in division by 0 if image time longer than 500ms
• Add information on JUNGFRAU conversion and geometry transformation to CBOR and HDF5

1.0.0-rc.24

New FPGA functionality:

• EIGER supports 8, 16 and 32-bit data input (for 8-bit mode at half performance; for 32-bit "real" depth is 23-bit + 1-bit signed)
• Output possible to 8, 16 and 32-bit data
• Threshold is applied before summation
• Pixel mask can be applied on FPGA
• Mark pixels with ADC content = 0 as bad pixels
• FPGA stores semantic version information (access via /sys/class/misc/jfjoch.../version)

New software functionality:

• Long summation (above 256 frames) done on CPU
• Mechanism to save arbitrary data to HDF5 file
• ZeroMQ preview has option to send start message
• Rework pixel mask + add statistics displayed in web interface

Bug fixes:

• Web frontend: Update preview image automatically during data acquisition
• jfjoch_broker: Error handling if CUDA driver is not installed
• jfjoch_broker: Correctly update progress during pedestal
• jfjoch_broker: Provide proper error when uploaded file is not a proper TIFF
• jfjoch_action_test: enable HLS simulation

Documentation improvement and placement in a dedicated directory