Refactor/mcs card refactoring first light #87
Reference in New Issue
Block a user
Delete Branch "refactor/mcs_card_refactoring_first_light"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
This MR includes a refactoring of the
MCSintegration, theDDGintegration as well as a rewritten integration of theIDSCamera. All devices now work pretty good, although there still remain minor issues that need to be investigated separately.Pipeline for 3.12 fails due to issues of the gitlab runner.
MCS Card
closes #3
The integration looks solid. The logic is couple to the integration of DDG1. I tried to document this in code as good as possible, but it might be good to write additional information together with how the device is cabled.
DDG
closes #5 #8
I investigated the failure of long scans. They are resolved now. The issue was in how to handle feedback from the EpicsIOC nicely during triggering. I believe this to be quite robust now, however, please keep in mind that the sleeps are actually in the polling. I am not quite sure whether this is due to shortcomings of the hardware itself, or EPICS on top.
To further optimize this, I can only think of bypassing communication via EPICS fully, which could be relatively quick, but is postponed for the moment as this requires more work and it seems to run now quite smoothly.
IDS Camera
#closes #9 #10
The new integration
ids_camera_newis basically a cleanup of the procedure in the old camera. I kept the old integration with a deprecation warning, as Mirko may like to test it with his integrations @holler .I am thinking about contacting IDS to ask them a couple of questions. One for instance would be how to properly kill existing connections to a camera, as I have seen this to be an issue while integrating the device. I tried to make this as safe as possible by registering a cleanup hook that destroys the connection to the camera. However, as this command is taking some time, there is a side-effect that killing the DeviceServer may take additional time. If one is impatient, one may actually kill and restart too quickly which results in the camera not properly connecting upon initialization.
PS: the device_config has now the field
live_modewhich essentially prepares the camera to stream directly. This interface is also exposed to the CLI. In addition, I added a roi_signal (asynchronous) that sends the normalized sum of intensity over the flattened image to BEC IF the camera is in live_mode. The roi can either be set through.set_rect_roi(x,y,widht,height), or by directly setting a 2D mask of appropriate size.mask = np.zeros(img_shape). You can receive the last image viaget_last_image()to generate the mask.assigned to @appel_c
added 1 commit
Compare with previous version
added 7 commits
mainCompare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 2 commits
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 7 commits
256adb14- refactor(mcs-card): refactor mcs card integration62ecdd37- refactor(mcs-card): fix mcs card integration at the beamline7a3ce97e- refactor: cleanup and formattingcdac29be- fix(xbpms): generalize describe method of sum,diff signals7679fa13- fix: cleanup, fix mcs_clockc2cba873- refactor: remove old mcs card3fd3d540- refactor(mcs-card): cleanup class, add testsCompare with previous version
requested review from @menzel
added 1 commit
Compare with previous version
requested review from @wakonig_k
requested review from @diaz
changed the description
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 3 commits
14713c0d- fix: remove old mcs card from confige6d7e0d8- refactor: rename ddg test config to endstation, add to first light3db74d98- fix: put idgap to readonly falseCompare with previous version
approved this merge request
marked this merge request as ready
added 1 commit
Compare with previous version
reset approvals from @appel_c by pushing to the branch
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
Thanks for adding me as a reviewer, but I cannot comment much about this, other that it is great to have an MCS interface...
added 1 commit
Compare with previous version
added 2 commits
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 2 commits
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 2 commits
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 2 commits
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 7 commits
b6af9380- fix: remove get_config from pre_startup8f7ada2f- refactor(ids): refactor ids camera9e45e927- test(ids-camera): add tests for the IDSCamera integration40d6acf4- refactor: change sl1 names following convention of controlsa01593fa- refactor(ids_camera): add deprecation warning for ids_camera3c9192d6- refactor(ddg2): add check for negative pulse widths.a03e99d6- refactor(ddg): passiv readout of event_status registerCompare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
changed the description
added 1 commit
Compare with previous version
changed the description
changed the description
requested review from @perl_d
added 1 commit
c7e11eea- refactor(ddg): use threadpool with active polling for stateCompare with previous version
mentioned in merge request !68
mentioned in merge request !60
what's the reason for using a threadpool with only one worker instead of managing the thread? A running task still won't be cancelled even if the executor is shut down, it will only stop new jobs.
type hint is wrong here,
AllChannelNamesisLiteral["t0", "ab", "cd", ...].Also, these are the same between this file and
ddg2.py- could move them into the base file.this only returns
DeviceStatusmight be good to put all these times into a constant
this could be tidier, this class creates
_threadpoolon__init__so the attribute must always exist. And it shouldn't be necessary to set it to None. The docs forThreadPoolExecutor.shutdown()say:So I think it could just be
same as above
this logic could be moved to the base class, the default setup is the same for 1 and 2
remove commented code?
is this widget meant to exist?
this file looks unneccessary
good documentation here!
would be good to add a comment to say what should be protected by the lock and why
this could really use some more comments, very hard to understand what is going on
new is not a great name for something which will stick around and not be new anymore, prefer
v2or something like thatremove commented code
very cryptic variable names, could use either comments or renaming. Also, should check for
ueyeand give a nice error rather thanNoneType has no attribute ...if the import failed.change to log?
since
__init__will fail anyway if this fails, you could just move the import to__init__do we expect this to happen? we wouldn't usually check for imports like this unless we expect to run without them installed sometimes. is it not a dependency or are there often problems with properly installing it?
default here should prbably be
Falserather thanNonereplace these prints with log? otherwise they will just be in the device server console, right? we should be collecting the ophyd logs now so when we deploy the ingestor we will be able to look at them all together in elastic.
e.g. something like:
I read through as best I could but I have only added quite superficial comments. it would be good to clean those up a little more. But the logic is too difficult to evaluate separate from the hardware so I will just trust you that it works ;)
approved this merge request
changed this line in version 103 of the diff
changed this line in version 103 of the diff
changed this line in version 103 of the diff
changed this line in version 103 of the diff
changed this line in version 103 of the diff
changed this line in version 103 of the diff
changed this file in version 103 of the diff
changed this line in version 103 of the diff
changed this line in version 103 of the diff
changed this file in version 103 of the diff
changed this line in version 103 of the diff
changed this line in version 103 of the diff
added 4 commits
a58c2384- refactor(ddg1): remove threadpool, use dedicated thread for pollingCompare with previous version
reset approvals from @perl_d by pushing to the branch
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 1 commit
Compare with previous version
added 3 commits
9f5254ab- refactor(ddg): cleanup and improve commentsCompare with previous version
I made it a dedicated thread.
Done, I wanted to leave the comments as they are currently the only proper way to link to the logic for cabling. Once the logic is extended, this can be commented here nicely.
done
I am a bit concerned that they then get changed by accident, those were actually tricky to find so I improved the comment.
using single thread now and cleaned it!
done
I agree, but to keep the logic in place what is set, I would like to keep the baseclass to only have the methods. I think this will be easier to maintain logic wise in the future to not have logic of what is being set across multiple classes, even if this means some additional lines of code.
I would like to have those methods available to the users, but there is an issue with the serialization of dtypes of the methods. If you like, feel free to take a look (https://github.com/bec-project/bec/issues/585)
thanks, no this should be removed! Btw, there is a widget which was created before this mechanism. omny_alignment.py. This does not have to be added correct?
thanks!
Yeah, I would like to rename it to ids_camera once the other class can be removed. Therefore, I will also ignore most of the comments on that one as this was the initial working integration of Mirko. Once he is happy with 'new', we can remove and rename this one. I'll make an issue though -> https://gitlab.psi.ch/bec/csaxs_bec/-/issues/11
Yeah, the import check is nice! - I'll add that
done
So init won't fail, if on_connect is not called. The issue is that the pyueye library raises if it does not find the appropriate libraries. Therefore, we can initialize the class in the CI, but not call
on_connect. In the implementation of theids_camera_new.py, I therefore split the logic and create the Camera object withconnect=False. This also allows me to test the camera, without the ueye lib. In runtime, we will always call on_connected and thereby load the library. However, I did add the raise condition in the IDSCameraObject to ensure that we get a nicer error now.see comment above
True
see above, to be deprecated
Thanks a lot for the review, it was very helpful!!
approved this merge request
added 2 commits
5c263bdb- refactor(ids-camera): improve docs and camera classf69e6430- refactor(mcs): imrove documentation for counter updatesCompare with previous version
reset approvals from @appel_c by pushing to the branch
added comment
adde more comments
I'm sure it's just me but I don't understand this logic. The condition on the while loop is the same as the condition for returning early from
_poll_loop. Wouldn't this return from like 206 basically every time, unless_poll_thread_run_eventor_poll_thread_kill_eventchange state in between checking thewhilecondition and getting there?if this works then it's good, but maybe it would have been clearer with a lock to guard polling instead? maybe something to think about if we ever have to rewrite it.
approved this merge request
added 1 commit
189141a0- refactor(ids-camera): add additional information to the docstringsCompare with previous version
reset approvals from @perl_d by pushing to the branch
resolved all threads