refactor #1

Merged
augustin_s merged 159 commits from refactor into main 2024-10-12 17:09:10 +02:00

159 Commits

Author SHA1 Message Date
7a57201ca3 fixed dtype of "roi_intensities_normalised" if threshold_value != "NaN" 2024-10-03 12:13:39 +02:00
98bbe76936 typo: vmin instead of vmax 2024-10-03 11:50:49 +02:00
df17ae536c numpy arrays do not like to be treated as booleans 2024-10-03 11:03:55 +02:00
5bf58d1193 fixed dtype of "roi_intensities" 2024-10-03 11:02:24 +02:00
05905d183c bumped version of JF06 (4 -> 5) 2024-09-11 21:36:14 +02:00
f18af38dc6 reordered operations 2024-09-04 17:30:43 +02:00
35c50b472f split updating nmax and checking readiness 2024-08-30 18:01:13 +02:00
a65553e13a removed apply_aggregation check [aggregator cannot be ready if apply_aggregation is False] 2024-08-30 17:38:16 +02:00
7a05d8f095 moved aggregator.reset() [this now also resets if aggregation_max wasnt given] 2024-08-30 15:56:27 +02:00
4db7f4f8cd moved "not ready" due to missing aggregation_max into Aggregator 2024-08-30 14:53:05 +02:00
7d57d6c5e0 removed unused data argument and numpy import 2024-08-30 11:54:10 +02:00
1350093d5a removed apply_threshold check from calc_aggregation_ready as these are independent operations 2024-08-30 11:52:35 +02:00
470aa52ff9 use the proper check from JU [this includes checking for the gain now, which probably was ommitted before since the gain file rarely changes and always exists] 2024-08-29 10:56:46 +02:00
cc0113507b removed unreachable special case for raw data (uint16) reaching aggregation [adapter.process calls handler.process with conversion=handler.can_convert(), which checks if gain and pedestal are truthy; with conversion, the dtype is float32 or int32; if handler.pedestal_file is falsy, the image is skipped upon receive -> special case can never trigger] 2024-08-29 10:37:04 +02:00
cb79be0a3e added/use Aggregator.is_ready 2024-08-28 10:53:21 +02:00
1e6af40d56 moved calc_apply_threshold out of calc_data, removed special case check 2024-08-28 10:39:01 +02:00
b99a32d9d7 renamed raw_data -> raw_image 2024-08-28 10:21:32 +02:00
8981eab522 renamed pfdata -> pfimage 2024-08-28 10:19:01 +02:00
6ca0ebf945 renamed data -> image 2024-08-28 10:16:30 +02:00
f2f871c880 renamed image -> raw_data 2024-08-28 10:13:21 +02:00
d0c1621109 consistency: removed special case that sends the raw data image instead of the processed image data [special case triggered if apply_aggregation and apply_threshold are off, but both can still be not applied if the respective parameters (aggregation_max, threshold_min, threshold_max) are not given, which would send data] 2024-08-28 10:08:18 +02:00
b1f104a71d disentangle 2024-08-21 15:55:23 +02:00
18d6dd0b7b consistent naming with the other algos 2024-08-20 22:23:36 +02:00
4b01cb0dca naming; switch order 2024-08-20 22:17:56 +02:00
547b2b7fb1 disentangle 2024-08-20 22:11:16 +02:00
d3ac74835e simplified 2024-08-19 13:02:36 +02:00
52e7260fb1 added (commented) an alternative idea 2024-08-19 12:59:07 +02:00
cea2b0c741 moved random skip logic into separate file, added some comments on how it works 2024-08-19 12:41:49 +02:00
17e2aa1096 tried to make logic self-explanatory 2024-08-19 12:24:21 +02:00
faa774e4dc de morgan 2024-08-15 10:07:50 +02:00
c8d9473271 breaking this apart a bit more 2024-08-15 10:00:02 +02:00
fd2345c611 aggregator changes are in-place 2024-08-13 13:48:45 +02:00
e0e9cf02d7 added ignore list 2024-08-13 09:52:38 +02:00
d8b4931a28 added/use Aggregator to encapsulate data_summed and n_aggregated_images 2024-08-13 09:52:24 +02:00
a8c73bab0f count correctly: 1, 1, 2, 3, ... -> 0, 1, 2, 3, ... 2024-08-12 10:04:10 +02:00
2acf116709 re-use thresh function 2024-08-09 19:48:48 +02:00
dcdacf4bdb re-use function 2024-08-09 19:18:06 +02:00
34eb466622 fixed inconsistent returns 2024-08-09 17:10:00 +02:00
442a7dab28 moved comments out 2024-08-09 15:57:17 +02:00
5f055c8718 flatten the code; early exit 2024-08-09 15:54:54 +02:00
2103583aca moved calc_apply_threshold out of calc_radial_integration 2024-08-09 15:41:46 +02:00
48c6bbe7c7 zero is zero 2024-08-09 15:04:22 +02:00
f12a736355 simpler logic 2024-08-09 14:54:30 +02:00
a7429115d7 the usual todo note 2024-08-09 12:56:15 +02:00
03b556d61f flatten the code; early exit 2024-08-09 12:46:00 +02:00
74e73c1247 split into functions 2024-08-09 11:33:55 +02:00
2a12864feb removed line that does nothing (nan has no sign, nan always compares to False -> this never overwrites any element) 2024-08-08 16:38:52 +02:00
bfaa80c9d0 reorder 2024-08-08 16:30:06 +02:00
93f0702c41 early exit 2 2024-08-08 16:09:36 +02:00
a7327ddbf7 early exit 1 2024-08-08 16:07:54 +02:00
7987125bf5 some new lines 2024-08-08 16:07:08 +02:00
779d1e1d77 moved calculation of force_send_visualisation out of work function 2024-08-08 15:40:32 +02:00
f77f109ed2 removed the one case of camelCase 2024-08-07 18:27:21 +02:00
474df55969 use the current pid, not the one from the iteration before 2024-08-07 17:30:55 +02:00
0914b8d169 removed commented part and reworded printout; todo notes 2024-08-07 17:27:42 +02:00
a184e49eee use that BufferedJSON works for non-existing files 2024-08-07 17:05:30 +02:00
8531f0a28d added BufferedJSON.exists() method, use it in get_time/get_data 2024-08-07 17:02:04 +02:00
3b599164fa file naming 2024-08-07 16:42:58 +02:00
bff86898e9 moved json_load to bufjson.py 2024-08-07 16:41:33 +02:00
018fa6cf4b use BufferedJSON 2024-08-07 15:27:13 +02:00
9dbd11d657 added BufferedJSON 2024-08-06 20:59:14 +02:00
f1373936c0 added BufferedJSON 2024-08-06 20:56:56 +02:00
9b2fc0efea turned utils into folder 2024-08-06 20:51:45 +02:00
6639b46afb order 2024-08-06 17:01:02 +02:00
159574f4e8 combined a bit more of the logic 2024-08-06 16:37:56 +02:00
9fb9595c83 order 2024-08-06 16:07:04 +02:00
68c7bd7866 moved pedestal_name_saved into JFData 2024-08-06 15:56:28 +02:00
2e80fc0e6e moved the other half of ju_stream_adapter code into separate class 2024-08-06 15:20:09 +02:00
242ec6ff51 moved one half of ju_stream_adapter code into separate class 2024-08-06 14:03:33 +02:00
7b56e45cb6 probably better to limit the cache in size 2024-08-05 18:22:36 +02:00
e036e033f9 moved do_radial_integration check inside function for better overview 2024-08-05 18:20:35 +02:00
3a63baf909 dont repack center 2024-08-05 18:18:46 +02:00
a6af8ee19c use npmemo 2024-08-05 18:15:31 +02:00
58c18d9b8c added algos/utils folder and npmemo 2024-08-05 18:04:26 +02:00
26f41b5383 made radprof helper memoizable 2024-08-05 18:03:53 +02:00
45c6d99f7a forgotten return value 2024-08-02 20:28:58 +02:00
fb6ed0d116 worker is always 1, removed it and commented unreachable print outs 2024-08-02 20:28:46 +02:00
b2ae7292ab results is always copied from metadata 2024-08-02 19:17:09 +02:00
2903203743 use kwargs defaults 2024-08-02 19:09:50 +02:00
2857d6412b readability 2024-08-02 19:02:43 +02:00
8c11dd8f52 get_saturated_pixels does not change data in place, hence no copy is needed; use kwargs defaults 2024-08-02 19:01:04 +02:00
88c81a905b use ndarray.copy consistently 2024-08-02 18:50:52 +02:00
68619f1181 rad is already int 2024-08-02 17:25:28 +02:00
bbc58ca5e6 need to also store the norm between iterations; added some comments on why the center/rad/norm are reset and where; r_min/r_max need to be re-calclated every time (or also stored); added todo note 2024-08-02 17:23:11 +02:00
ac732cc4b7 easier to read name 2024-08-02 17:05:18 +02:00
46308b16a9 function order 2024-08-02 16:52:34 +02:00
26e2a3f619 easier to read/distinguish names 2024-08-02 16:50:04 +02:00
098146c97b use ndarray.copy consistently 2024-08-02 16:25:16 +02:00
f6c97e9749 added check for pixel_mask_pf being None 2024-08-02 16:14:42 +02:00
eb39550983 added todo note 2024-08-02 16:14:24 +02:00
ca3b42a797 use the boolean pixel_mask_pf 2024-08-02 16:11:57 +02:00
88f881fffd removed keep_pixels and use boolean array pixel_mask_pf instead 2024-08-02 16:01:13 +02:00
8cd2f07d80 leave pixel_mask_pf as boolean array (int8 is only needed for peakfinder_8, where it is type casted anyway) 2024-08-02 15:54:31 +02:00
de3ea150b2 all mask comparisons are for equality 2024-08-02 15:49:47 +02:00
dd7eda4834 renamed variable pfdata to data 2024-08-02 15:11:13 +02:00
062cc6055c use the same validation logic as in other cases 2024-08-02 15:10:52 +02:00
d9d79f3e87 more pythonic check 2024-08-02 14:55:37 +02:00
af3eb6853b naming consistency 2024-08-02 14:36:35 +02:00
6dccb5bbfd only apply additional mask if the mask has changed 2024-08-02 14:30:19 +02:00
092479c52d cleaned up comment 2024-08-02 14:23:06 +02:00
d8d984f5a3 read detector_name only where it is needed 2024-08-02 14:12:29 +02:00
19f1c8e375 condensed logic 2024-08-02 14:05:05 +02:00
87a79a265c naming consistency 2024-08-02 13:55:14 +02:00
e5fc7b2750 cleanup 2024-08-02 13:53:25 +02:00
0d06b7b5ed removed commented code for old beamtime 2024-08-02 13:44:54 +02:00
bf7f62adff moved masking pixels out of work function 2024-08-02 13:26:02 +02:00
6af0166f07 added todo notes 2024-08-02 13:16:09 +02:00
1402b08060 only copy if there are really in-place changes to be done 2024-08-02 13:12:30 +02:00
61b1d55a9e added notes on in place changes 2024-08-02 13:10:06 +02:00
d9b2ae4ac2 use the same validation logic as in other cases 2024-08-02 12:39:45 +02:00
b1b92e4b52 simpler logic 2024-08-02 12:28:09 +02:00
40c0a08d7c added todo notes 2024-08-02 12:18:56 +02:00
e9d187c712 renamed variable pfdata to data 2024-08-02 12:15:19 +02:00
a690654343 moved thresholding out of work function 2024-08-02 12:14:27 +02:00
53d2d965d9 forgotten results argument 2024-08-02 12:11:19 +02:00
92149d0614 moved do_* checks inside function for better overview 2024-08-02 12:01:01 +02:00
a34db9287e isort 2024-08-01 09:19:49 +02:00
2facda5d07 comments 2024-08-01 09:16:56 +02:00
3924abcba4 comments, todo note 2024-08-01 09:16:40 +02:00
ac8967611d use the same validation logic as in other cases 2024-08-01 09:05:43 +02:00
1af3e9c3ec added todo notes about TBD logic changes 2024-08-01 09:02:58 +02:00
7ea32d2a4d spaces 2024-07-31 19:43:07 +02:00
596a4e7e0a less convoluted 2024-07-31 19:42:59 +02:00
851a186538 readability 2024-07-31 17:50:40 +02:00
1c129bc5be removed useless copy (seems unlikely that peakfinder_8 changes the mask AND astype copies anyway unless explicity disabled) 2024-07-31 17:22:05 +02:00
41ff6c3204 renamed data_copy_1 -> data 2024-07-31 11:23:44 +02:00
7662f8604d naming, cleanup 2024-07-31 11:21:03 +02:00
55a719fa6c removed useless copy/conversion 2024-07-31 11:16:20 +02:00
a834d23f42 typos 2024-07-31 11:12:35 +02:00
abeab715dd consistent naming 2024-07-31 11:12:10 +02:00
a25bd23b67 cleaned/restructured 2024-07-31 11:09:46 +02:00
1db20f8b82 moved additional checks inside function 2024-07-30 22:23:28 +02:00
12e008aeba moved additional checks inside function 2024-07-30 21:25:30 +02:00
26913abbd5 removed useless copy 2024-07-30 20:20:25 +02:00
1dd37e1684 moved additional checks inside function; unpack variables from results dict 2024-07-30 20:06:12 +02:00
629d2c84bb de-arrow 2024-07-30 19:57:36 +02:00
a4918c989c more pythonic 2024-07-30 19:53:57 +02:00
7b2a1c3935 cleanup; logic also works correctly for number_of_spots == 0 2024-07-30 19:48:32 +02:00
96878e1b67 cleanup 2024-07-30 18:59:40 +02:00
8a72928a20 renamed variable pfdata to data 2024-07-30 18:57:40 +02:00
3c52784d48 cleanup 2024-07-30 18:57:08 +02:00
d4575a6656 de-arrow 2024-07-30 18:56:11 +02:00
3e3916b786 moved ROI algo out of work function 2024-07-30 18:55:02 +02:00
56a5f94805 added todo note 2024-07-30 18:11:49 +02:00
5d714e1fa3 moved spi_analysis out of work function 2024-07-30 18:09:56 +02:00
c3044653c2 renamed variable pfdata to data 2024-07-30 17:51:57 +02:00
6a926bcede moved peakfinder_analysis out of work function 2024-07-30 17:51:08 +02:00
a01a6e6ed5 renamed variable d to pfdata 2024-07-30 17:27:04 +02:00
399b887516 added/use json_load and read_bit utils 2024-07-30 17:02:57 +02:00
1e7d4a0a02 moved zmq receiving/sending into separate class 2024-07-30 15:58:56 +02:00
444bf1de06 moved zmq receiving/sending into separate class 2024-07-30 15:57:57 +02:00
57e22c01b5 spaces 2024-07-30 15:25:25 +02:00
cd76af01fa DRY 2024-07-30 15:13:41 +02:00
a96a5d6423 removed unused True path 2024-07-30 15:05:42 +02:00
b78f4a61fd de-duplicate sending 2024-07-30 14:56:10 +02:00
e567e3f406 tried to group bad masked regions a bit better 2024-07-30 14:26:35 +02:00
49aa4bad14 moved apply_additional_mask out of work function 2024-07-30 14:25:27 +02:00
0ec91cd5d9 moved apply_additional_mask out of work function 2024-07-30 14:18:37 +02:00
15d634bb82 moved radial_integration algo out of work function 2024-07-30 13:58:16 +02:00