Feat/add jungfrau joch integration #90

appel_c · 2025-09-02T18:08:49+02:00

appel_c commented

2025-09-02 18:08:49 +02:00

(Migrated from gitlab.psi.ch)

Summary

This MR integrates the JungfrauJoch backend together with the Eiger detectors Eiger1.5M and Eiger9M. Both detectors share the same logic, and are simply differentiated by having the correct system activate within JungfrauJoch. The integration was tested for the Eiger1.5M. Below is a brief summary of what works already, and what remains as a TODO for future work.

closes #1

Checklist

Eiger data was measured for the 1.5M with various exposure times, and frames. Overall, this ranged between 20-100 frames with exposure times between 0.6ms (600us) to 5000ms (5s). Scans look good and robust without significant failures.
Sometimes the backend is not receiving all the data. This needs to be investigated in more detail as to what exactly is the issue here.
File Writing works, with JFJ writing data in separate h5 files. The beamline should check the format. I picked a format with a virtual dataset, which is now already linked to the main h5 file from BEC. Data can thereby be directly accessed. The beamline may create more links as they see fit to various additional metadata in the JFJ h5 files. I will explain how to do that separately.
Tests are written to ensure that the current logic remains consistent.
Currently, the integration will not automatically switch the detector as this operation is not yet considered stable enough to be automated. Therefore, I implemented checks that the detector is properly selected and initialized upon connecting to the device. Selection and initialization should take place in the JungfrauJoch WEB UI. The error message indicates this including the WEB UI URL.
A cleanup of the JungfrauJochClient communication is still outstanding. I would propose to first test with this version as it works reliably in first tests. Backend code can be improved and refactored later (review JFJ API usage + crosscheck error handling of request (timeout handling...)).
The preview stream is already received by the integration. However, at the moment we still miss some libraries on the bec-server to decode the data send through the stream. This will be enabled once packages are installed on the machines.

Summary

This version can be merged, and the points above used as reference for improvements in the future.

Notes

Currently CI pipelines fail because of issue related to gitlab. We wait for this to be fixed.

# Summary This MR integrates the JungfrauJoch backend together with the Eiger detectors Eiger1.5M and Eiger9M. Both detectors share the same logic, and are simply differentiated by having the correct system activate within JungfrauJoch. The integration was tested for the Eiger1.5M. Below is a brief summary of what works already, and what remains as a TODO for future work. closes #1 ## Checklist - [x] Eiger data was measured for the 1.5M with various exposure times, and frames. Overall, this ranged between 20-100 frames with exposure times between 0.6ms (600us) to 5000ms (5s). Scans look good and robust without significant failures. - [ ] Sometimes the backend is not receiving all the data. This needs to be investigated in more detail as to what exactly is the issue here. - [x] File Writing works, with JFJ writing data in separate h5 files. The beamline should check the format. I picked a format with a virtual dataset, which is now already linked to the main h5 file from BEC. Data can thereby be directly accessed. The beamline may create more links as they see fit to various additional metadata in the JFJ h5 files. I will explain how to do that separately. - [x] Tests are written to ensure that the current logic remains consistent. - [x] Currently, the integration will not automatically switch the detector as this operation is not yet considered stable enough to be automated. Therefore, I implemented checks that the detector is properly selected and initialized upon connecting to the device. Selection and initialization should take place in the JungfrauJoch WEB UI. The error message indicates this including the WEB UI URL. - [ ] A cleanup of the JungfrauJochClient communication is still outstanding. I would propose to first test with this version as it works reliably in first tests. Backend code can be improved and refactored later (review JFJ API usage + crosscheck error handling of request (timeout handling...)). - [ ] The preview stream is already received by the integration. However, at the moment we still miss some libraries on the bec-server to decode the data send through the stream. This will be enabled once packages are installed on the machines. # Summary This version can be merged, and the points above used as reference for improvements in the future. # Notes Currently CI pipelines fail because of issue related to gitlab. We wait for this to be fixed.

appel_c commented

2025-09-03 08:53:19 +02:00

(Migrated from gitlab.psi.ch)

added 1 commit

6b0cb6ea - wip cleanup