refactor #1

Merged
augustin_s merged 159 commits from refactor into main 2024-10-12 17:09:10 +02:00

159 Commits

Author SHA1 Message Date
augustin_s 7a57201ca3 fixed dtype of "roi_intensities_normalised" if threshold_value != "NaN" 2024-10-03 12:13:39 +02:00
augustin_s 98bbe76936 typo: vmin instead of vmax 2024-10-03 11:50:49 +02:00
augustin_s df17ae536c numpy arrays do not like to be treated as booleans 2024-10-03 11:03:55 +02:00
augustin_s 5bf58d1193 fixed dtype of "roi_intensities" 2024-10-03 11:02:24 +02:00
augustin_s 05905d183c bumped version of JF06 (4 -> 5) 2024-09-11 21:36:14 +02:00
augustin_s f18af38dc6 reordered operations 2024-09-04 17:30:43 +02:00
augustin_s 35c50b472f split updating nmax and checking readiness 2024-08-30 18:01:13 +02:00
augustin_s a65553e13a removed apply_aggregation check [aggregator cannot be ready if apply_aggregation is False] 2024-08-30 17:38:16 +02:00
augustin_s 7a05d8f095 moved aggregator.reset() [this now also resets if aggregation_max wasnt given] 2024-08-30 15:56:27 +02:00
augustin_s 4db7f4f8cd moved "not ready" due to missing aggregation_max into Aggregator 2024-08-30 14:53:05 +02:00
augustin_s 7d57d6c5e0 removed unused data argument and numpy import 2024-08-30 11:54:10 +02:00
augustin_s 1350093d5a removed apply_threshold check from calc_aggregation_ready as these are independent operations 2024-08-30 11:52:35 +02:00
augustin_s 470aa52ff9 use the proper check from JU [this includes checking for the gain now, which probably was ommitted before since the gain file rarely changes and always exists] 2024-08-29 10:56:46 +02:00
augustin_s cc0113507b removed unreachable special case for raw data (uint16) reaching aggregation [adapter.process calls handler.process with conversion=handler.can_convert(), which checks if gain and pedestal are truthy; with conversion, the dtype is float32 or int32; if handler.pedestal_file is falsy, the image is skipped upon receive -> special case can never trigger] 2024-08-29 10:37:04 +02:00
augustin_s cb79be0a3e added/use Aggregator.is_ready 2024-08-28 10:53:21 +02:00
augustin_s 1e6af40d56 moved calc_apply_threshold out of calc_data, removed special case check 2024-08-28 10:39:01 +02:00
augustin_s b99a32d9d7 renamed raw_data -> raw_image 2024-08-28 10:21:32 +02:00
augustin_s 8981eab522 renamed pfdata -> pfimage 2024-08-28 10:19:01 +02:00
augustin_s 6ca0ebf945 renamed data -> image 2024-08-28 10:16:30 +02:00
augustin_s f2f871c880 renamed image -> raw_data 2024-08-28 10:13:21 +02:00
augustin_s d0c1621109 consistency: removed special case that sends the raw data image instead of the processed image data [special case triggered if apply_aggregation and apply_threshold are off, but both can still be not applied if the respective parameters (aggregation_max, threshold_min, threshold_max) are not given, which would send data] 2024-08-28 10:08:18 +02:00
augustin_s b1f104a71d disentangle 2024-08-21 15:55:23 +02:00
augustin_s 18d6dd0b7b consistent naming with the other algos 2024-08-20 22:23:36 +02:00
augustin_s 4b01cb0dca naming; switch order 2024-08-20 22:17:56 +02:00
augustin_s 547b2b7fb1 disentangle 2024-08-20 22:11:16 +02:00
augustin_s d3ac74835e simplified 2024-08-19 13:02:36 +02:00
augustin_s 52e7260fb1 added (commented) an alternative idea 2024-08-19 12:59:07 +02:00
augustin_s cea2b0c741 moved random skip logic into separate file, added some comments on how it works 2024-08-19 12:41:49 +02:00
augustin_s 17e2aa1096 tried to make logic self-explanatory 2024-08-19 12:24:21 +02:00
augustin_s faa774e4dc de morgan 2024-08-15 10:07:50 +02:00
augustin_s c8d9473271 breaking this apart a bit more 2024-08-15 10:00:02 +02:00
augustin_s fd2345c611 aggregator changes are in-place 2024-08-13 13:48:45 +02:00
augustin_s e0e9cf02d7 added ignore list 2024-08-13 09:52:38 +02:00
augustin_s d8b4931a28 added/use Aggregator to encapsulate data_summed and n_aggregated_images 2024-08-13 09:52:24 +02:00
augustin_s a8c73bab0f count correctly: 1, 1, 2, 3, ... -> 0, 1, 2, 3, ... 2024-08-12 10:04:10 +02:00
augustin_s 2acf116709 re-use thresh function 2024-08-09 19:48:48 +02:00
augustin_s dcdacf4bdb re-use function 2024-08-09 19:18:06 +02:00
augustin_s 34eb466622 fixed inconsistent returns 2024-08-09 17:10:00 +02:00
augustin_s 442a7dab28 moved comments out 2024-08-09 15:57:17 +02:00
augustin_s 5f055c8718 flatten the code; early exit 2024-08-09 15:54:54 +02:00
augustin_s 2103583aca moved calc_apply_threshold out of calc_radial_integration 2024-08-09 15:41:46 +02:00
augustin_s 48c6bbe7c7 zero is zero 2024-08-09 15:04:22 +02:00
augustin_s f12a736355 simpler logic 2024-08-09 14:54:30 +02:00
augustin_s a7429115d7 the usual todo note 2024-08-09 12:56:15 +02:00
augustin_s 03b556d61f flatten the code; early exit 2024-08-09 12:46:00 +02:00
augustin_s 74e73c1247 split into functions 2024-08-09 11:33:55 +02:00
augustin_s 2a12864feb removed line that does nothing (nan has no sign, nan always compares to False -> this never overwrites any element) 2024-08-08 16:38:52 +02:00
augustin_s bfaa80c9d0 reorder 2024-08-08 16:30:06 +02:00
augustin_s 93f0702c41 early exit 2 2024-08-08 16:09:36 +02:00
augustin_s a7327ddbf7 early exit 1 2024-08-08 16:07:54 +02:00
augustin_s 7987125bf5 some new lines 2024-08-08 16:07:08 +02:00
augustin_s 779d1e1d77 moved calculation of force_send_visualisation out of work function 2024-08-08 15:40:32 +02:00
augustin_s f77f109ed2 removed the one case of camelCase 2024-08-07 18:27:21 +02:00
augustin_s 474df55969 use the current pid, not the one from the iteration before 2024-08-07 17:30:55 +02:00
augustin_s 0914b8d169 removed commented part and reworded printout; todo notes 2024-08-07 17:27:42 +02:00
augustin_s a184e49eee use that BufferedJSON works for non-existing files 2024-08-07 17:05:30 +02:00
augustin_s 8531f0a28d added BufferedJSON.exists() method, use it in get_time/get_data 2024-08-07 17:02:04 +02:00
augustin_s 3b599164fa file naming 2024-08-07 16:42:58 +02:00
augustin_s bff86898e9 moved json_load to bufjson.py 2024-08-07 16:41:33 +02:00
augustin_s 018fa6cf4b use BufferedJSON 2024-08-07 15:27:13 +02:00
augustin_s 9dbd11d657 added BufferedJSON 2024-08-06 20:59:14 +02:00
augustin_s f1373936c0 added BufferedJSON 2024-08-06 20:56:56 +02:00
augustin_s 9b2fc0efea turned utils into folder 2024-08-06 20:51:45 +02:00
augustin_s 6639b46afb order 2024-08-06 17:01:02 +02:00
augustin_s 159574f4e8 combined a bit more of the logic 2024-08-06 16:37:56 +02:00
augustin_s 9fb9595c83 order 2024-08-06 16:07:04 +02:00
augustin_s 68c7bd7866 moved pedestal_name_saved into JFData 2024-08-06 15:56:28 +02:00
augustin_s 2e80fc0e6e moved the other half of ju_stream_adapter code into separate class 2024-08-06 15:20:09 +02:00
augustin_s 242ec6ff51 moved one half of ju_stream_adapter code into separate class 2024-08-06 14:03:33 +02:00
augustin_s 7b56e45cb6 probably better to limit the cache in size 2024-08-05 18:22:36 +02:00
augustin_s e036e033f9 moved do_radial_integration check inside function for better overview 2024-08-05 18:20:35 +02:00
augustin_s 3a63baf909 dont repack center 2024-08-05 18:18:46 +02:00
augustin_s a6af8ee19c use npmemo 2024-08-05 18:15:31 +02:00
augustin_s 58c18d9b8c added algos/utils folder and npmemo 2024-08-05 18:04:26 +02:00
augustin_s 26f41b5383 made radprof helper memoizable 2024-08-05 18:03:53 +02:00
augustin_s 45c6d99f7a forgotten return value 2024-08-02 20:28:58 +02:00
augustin_s fb6ed0d116 worker is always 1, removed it and commented unreachable print outs 2024-08-02 20:28:46 +02:00
augustin_s b2ae7292ab results is always copied from metadata 2024-08-02 19:17:09 +02:00
augustin_s 2903203743 use kwargs defaults 2024-08-02 19:09:50 +02:00
augustin_s 2857d6412b readability 2024-08-02 19:02:43 +02:00
augustin_s 8c11dd8f52 get_saturated_pixels does not change data in place, hence no copy is needed; use kwargs defaults 2024-08-02 19:01:04 +02:00
augustin_s 88c81a905b use ndarray.copy consistently 2024-08-02 18:50:52 +02:00
augustin_s 68619f1181 rad is already int 2024-08-02 17:25:28 +02:00
augustin_s bbc58ca5e6 need to also store the norm between iterations; added some comments on why the center/rad/norm are reset and where; r_min/r_max need to be re-calclated every time (or also stored); added todo note 2024-08-02 17:23:11 +02:00
augustin_s ac732cc4b7 easier to read name 2024-08-02 17:05:18 +02:00
augustin_s 46308b16a9 function order 2024-08-02 16:52:34 +02:00
augustin_s 26e2a3f619 easier to read/distinguish names 2024-08-02 16:50:04 +02:00
augustin_s 098146c97b use ndarray.copy consistently 2024-08-02 16:25:16 +02:00
augustin_s f6c97e9749 added check for pixel_mask_pf being None 2024-08-02 16:14:42 +02:00
augustin_s eb39550983 added todo note 2024-08-02 16:14:24 +02:00
augustin_s ca3b42a797 use the boolean pixel_mask_pf 2024-08-02 16:11:57 +02:00
augustin_s 88f881fffd removed keep_pixels and use boolean array pixel_mask_pf instead 2024-08-02 16:01:13 +02:00
augustin_s 8cd2f07d80 leave pixel_mask_pf as boolean array (int8 is only needed for peakfinder_8, where it is type casted anyway) 2024-08-02 15:54:31 +02:00
augustin_s de3ea150b2 all mask comparisons are for equality 2024-08-02 15:49:47 +02:00
augustin_s dd7eda4834 renamed variable pfdata to data 2024-08-02 15:11:13 +02:00
augustin_s 062cc6055c use the same validation logic as in other cases 2024-08-02 15:10:52 +02:00
augustin_s d9d79f3e87 more pythonic check 2024-08-02 14:55:37 +02:00
augustin_s af3eb6853b naming consistency 2024-08-02 14:36:35 +02:00
augustin_s 6dccb5bbfd only apply additional mask if the mask has changed 2024-08-02 14:30:19 +02:00
augustin_s 092479c52d cleaned up comment 2024-08-02 14:23:06 +02:00
augustin_s d8d984f5a3 read detector_name only where it is needed 2024-08-02 14:12:29 +02:00
augustin_s 19f1c8e375 condensed logic 2024-08-02 14:05:05 +02:00
augustin_s 87a79a265c naming consistency 2024-08-02 13:55:14 +02:00
augustin_s e5fc7b2750 cleanup 2024-08-02 13:53:25 +02:00
augustin_s 0d06b7b5ed removed commented code for old beamtime 2024-08-02 13:44:54 +02:00
augustin_s bf7f62adff moved masking pixels out of work function 2024-08-02 13:26:02 +02:00
augustin_s 6af0166f07 added todo notes 2024-08-02 13:16:09 +02:00
augustin_s 1402b08060 only copy if there are really in-place changes to be done 2024-08-02 13:12:30 +02:00
augustin_s 61b1d55a9e added notes on in place changes 2024-08-02 13:10:06 +02:00
augustin_s d9b2ae4ac2 use the same validation logic as in other cases 2024-08-02 12:39:45 +02:00
augustin_s b1b92e4b52 simpler logic 2024-08-02 12:28:09 +02:00
augustin_s 40c0a08d7c added todo notes 2024-08-02 12:18:56 +02:00
augustin_s e9d187c712 renamed variable pfdata to data 2024-08-02 12:15:19 +02:00
augustin_s a690654343 moved thresholding out of work function 2024-08-02 12:14:27 +02:00
augustin_s 53d2d965d9 forgotten results argument 2024-08-02 12:11:19 +02:00
augustin_s 92149d0614 moved do_* checks inside function for better overview 2024-08-02 12:01:01 +02:00
augustin_s a34db9287e isort 2024-08-01 09:19:49 +02:00
augustin_s 2facda5d07 comments 2024-08-01 09:16:56 +02:00
augustin_s 3924abcba4 comments, todo note 2024-08-01 09:16:40 +02:00
augustin_s ac8967611d use the same validation logic as in other cases 2024-08-01 09:05:43 +02:00
augustin_s 1af3e9c3ec added todo notes about TBD logic changes 2024-08-01 09:02:58 +02:00
augustin_s 7ea32d2a4d spaces 2024-07-31 19:43:07 +02:00
augustin_s 596a4e7e0a less convoluted 2024-07-31 19:42:59 +02:00
augustin_s 851a186538 readability 2024-07-31 17:50:40 +02:00
augustin_s 1c129bc5be removed useless copy (seems unlikely that peakfinder_8 changes the mask AND astype copies anyway unless explicity disabled) 2024-07-31 17:22:05 +02:00
augustin_s 41ff6c3204 renamed data_copy_1 -> data 2024-07-31 11:23:44 +02:00
augustin_s 7662f8604d naming, cleanup 2024-07-31 11:21:03 +02:00
augustin_s 55a719fa6c removed useless copy/conversion 2024-07-31 11:16:20 +02:00
augustin_s a834d23f42 typos 2024-07-31 11:12:35 +02:00
augustin_s abeab715dd consistent naming 2024-07-31 11:12:10 +02:00
augustin_s a25bd23b67 cleaned/restructured 2024-07-31 11:09:46 +02:00
augustin_s 1db20f8b82 moved additional checks inside function 2024-07-30 22:23:28 +02:00
augustin_s 12e008aeba moved additional checks inside function 2024-07-30 21:25:30 +02:00
augustin_s 26913abbd5 removed useless copy 2024-07-30 20:20:25 +02:00
augustin_s 1dd37e1684 moved additional checks inside function; unpack variables from results dict 2024-07-30 20:06:12 +02:00
augustin_s 629d2c84bb de-arrow 2024-07-30 19:57:36 +02:00
augustin_s a4918c989c more pythonic 2024-07-30 19:53:57 +02:00
augustin_s 7b2a1c3935 cleanup; logic also works correctly for number_of_spots == 0 2024-07-30 19:48:32 +02:00
augustin_s 96878e1b67 cleanup 2024-07-30 18:59:40 +02:00
augustin_s 8a72928a20 renamed variable pfdata to data 2024-07-30 18:57:40 +02:00
augustin_s 3c52784d48 cleanup 2024-07-30 18:57:08 +02:00
augustin_s d4575a6656 de-arrow 2024-07-30 18:56:11 +02:00
augustin_s 3e3916b786 moved ROI algo out of work function 2024-07-30 18:55:02 +02:00
augustin_s 56a5f94805 added todo note 2024-07-30 18:11:49 +02:00
augustin_s 5d714e1fa3 moved spi_analysis out of work function 2024-07-30 18:09:56 +02:00
augustin_s c3044653c2 renamed variable pfdata to data 2024-07-30 17:51:57 +02:00
augustin_s 6a926bcede moved peakfinder_analysis out of work function 2024-07-30 17:51:08 +02:00
augustin_s a01a6e6ed5 renamed variable d to pfdata 2024-07-30 17:27:04 +02:00
augustin_s 399b887516 added/use json_load and read_bit utils 2024-07-30 17:02:57 +02:00
augustin_s 1e7d4a0a02 moved zmq receiving/sending into separate class 2024-07-30 15:58:56 +02:00
augustin_s 444bf1de06 moved zmq receiving/sending into separate class 2024-07-30 15:57:57 +02:00
augustin_s 57e22c01b5 spaces 2024-07-30 15:25:25 +02:00
augustin_s cd76af01fa DRY 2024-07-30 15:13:41 +02:00
augustin_s a96a5d6423 removed unused True path 2024-07-30 15:05:42 +02:00
augustin_s b78f4a61fd de-duplicate sending 2024-07-30 14:56:10 +02:00
augustin_s e567e3f406 tried to group bad masked regions a bit better 2024-07-30 14:26:35 +02:00
augustin_s 49aa4bad14 moved apply_additional_mask out of work function 2024-07-30 14:25:27 +02:00
augustin_s 0ec91cd5d9 moved apply_additional_mask out of work function 2024-07-30 14:18:37 +02:00
augustin_s 15d634bb82 moved radial_integration algo out of work function 2024-07-30 13:58:16 +02:00