sf-daq/dap
0
1
Fork 2
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Fix memory leak in streak finder #3

Merged
augustin_s merged 15 commits from ext-dorofe_e/dap:cbd_bugfix into main 2025-07-15 10:21:21 +02:00
Conversation 1 Commits 15 Files Changed 3 +74 -55
ext-dorofe_e commented 2025-07-14 23:57:06 +02:00
Contributor
Copy Link

I have got rid of all fancy data containers from streak_finder repo and basically moved a few chunks of code from there to dap,
plus removed 3D support (since we always have single images and not stacks)

Done memory tests on Ra, and it hovers but does not grow.

Will open for review after final cleanup.

I have got rid of all fancy data containers from streak_finder repo and basically moved a few chunks of code from there to dap, plus removed 3D support (since we always have single images and not stacks) Done memory tests on Ra, and it hovers but does not grow. Will open for review after final cleanup.
ext-dorofe_e added 13 commits 2025-07-14 23:57:06 +02:00
Use bool rather than np.bool to avoid deprecation errors 8ffad86f9a
Explicitly delete cryst_data object after algorithm completion c09bdc5b4e
Try cleanup cryst data, track memory usage 6ebaf0ae6a
Try to get rid of cryst data entirely 2ea2129d5b
TEST: try to get rid of data containers in streak finder package; make it simple 901587db79
Bugfixes, no leak visible, TODO: cleanup, wf norm, crop, maybe mask roi e3efd82b2c
TEST scale whitefield 77d3ff9293
True divide f1828cb4b5
No newaxis, still extremely slow method for white field scale; TODO: use median instead 90c0b9cf3d
TEST: use median method for whitefield scale 9ed3335021
TEST: try return crop roi; cleanup d30adf3e14
TEST: crop data AND mask b2cc1e27a4
Correct region mask ad0752aa92
ext-dorofe_e reviewed 2025-07-15 00:06:45 +02:00
ext-dorofe_e left a comment
Author
Contributor
Copy Link

Added TODOs

Added TODOs
dap/algos/streakfind.py Outdated
@@ -103,0 +97,4 @@
out=np.zeros_like(data),
where=(std!=0.0)
)
del whitefield
ext-dorofe_e commented 2025-07-15 00:06:33 +02:00
Author
Contributor
Copy Link

TODO: dels are likely unnecessary; run more tests to make sure

TODO: `del`s are likely unnecessary; run more tests to make sure
ext-dorofe_e marked this conversation as resolved
dap/algos/streakfind.py Outdated
@@ -146,3 +155,1 @@
for mask_roi in mask_rois:
cryst_data = cryst_data.mask_region(mask_roi)
# mask_rois = results.get("cbd_mask_rois", []) # list of [y_min, y_max, x_min, x_max]
ext-dorofe_e commented 2025-07-15 00:00:11 +02:00
Author
Contributor
Copy Link

TODO: restore mask ROIs functionality

TODO: restore mask ROIs functionality
ext-dorofe_e marked this conversation as resolved
dap/algos/streakfind.py Outdated
@@ -187,3 +206,3 @@
x_center -= crop_roi[0]
y_center -= crop_roi[2]
streaks_mask = streaks.concentric_only(x_center, y_center)
threshold = 0.33
ext-dorofe_e commented 2025-07-14 23:59:34 +02:00
Author
Contributor
Copy Link

TODO: move concentric filter to a separate method

TODO: move concentric filter to a separate method
ext-dorofe_e marked this conversation as resolved
ext-dorofe_e added 1 commit 2025-07-15 08:35:15 +02:00
Restore mask ROI fuctionality; Concentric streaks filter in a function; Cleanup 118b97acfb
ext-dorofe_e added 1 commit 2025-07-15 08:50:42 +02:00
Final cleanup abae2288e7
ext-dorofe_e changed title from WIP: Fix memory leak in streak finder to Fix memory leak in streak finder 2025-07-15 08:51:01 +02:00
ext-dorofe_e requested review from augustin_s 2025-07-15 08:51:14 +02:00
augustin_s merged commit 8949a63189 into main 2025-07-15 10:21:21 +02:00
Sign in to join this conversation.
Reviewers
No Reviewers
ext-dorofe_e
augustin_s
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
augustin_s babic_a lombosi_c usov_i wakonig_k woznic_n
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sf-daq/dap#3
Delete Branch "ext-dorofe_e/dap:cbd_bugfix"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?