|
|
e946d4ff94
|
Chore: remove the scattering of the mass, size, time delay bins across dask workers outside of the client definition
|
2025-09-09 15:07:47 +02:00 |
|
|
|
b377c36c28
|
Chore: cleanup old code
|
2025-09-09 15:03:44 +02:00 |
|
|
|
3a41fbf387
|
fix: fix bug that was leading to extremely large dask graphs and move all histogram calcualtion logic to the distributio.py module
|
2025-09-09 14:53:19 +02:00 |
|
|
|
b91380a6db
|
fix: increase Dask worker wait time to prevent premature shutdown
|
2025-09-09 14:43:14 +02:00 |
|
|
|
0e932e9a70
|
test: add parquet files to use for testing
|
2025-09-09 14:25:13 +02:00 |
|
|
|
0268a5460c
|
fix: fix parquet saving of distributions (specify engine, write metadata, ...)
|
2025-08-25 15:07:13 +02:00 |
|
|
|
21e14ae2f1
|
chore: update run config
|
2025-08-22 18:20:26 +02:00 |
|
|
|
c1243e3b1e
|
feat: improve cluster shutdown and cleanup logic
|
2025-08-22 18:18:44 +02:00 |
|
|
|
8a0c1f3305
|
feat: add TMPDIR for temporary files in cluster jobs
|
2025-08-22 18:15:39 +02:00 |
|
|
|
7bd42c22a2
|
feat: add sbatch file to run via Slurm
|
2025-08-22 18:13:50 +02:00 |
|
|
|
55efbc74a2
|
Cleanup: remove wreck code from scripts/sp2xr_apply_calibration.py
|
2025-08-22 16:43:58 +02:00 |
|
|
|
3173a7c83b
|
Cleanup: remove wreck code from src/sp2xr/resample_pbp_hk.py
|
2025-08-22 16:42:02 +02:00 |
|
|
|
2da9eb6089
|
Cleanup: remove wreck code from src/sp2xr/helpers.py
|
2025-08-22 16:40:26 +02:00 |
|
|
|
d2a0533a12
|
Cleanup: remove wreck code from src/sp2xr/schema.py
|
2025-08-22 16:38:06 +02:00 |
|
|
|
4696c2cbb9
|
Cleanup: Remove wreck code from sp2xr_pipeline
|
2025-08-22 16:36:08 +02:00 |
|
|
|
681de6c203
|
Fix: add missing import of cast_and_arrow function to sp2xr_pipeline
|
2025-08-22 16:31:03 +02:00 |
|
|
|
40bae0e5d2
|
Refactor: move function _cast_and_arrow to schema.py
|
2025-08-22 16:19:42 +02:00 |
|
|
|
2704128e8d
|
Test notebook moved out of project
|
2025-08-22 16:11:18 +02:00 |
|
|
|
43c31728e0
|
Fix: add control to skip empty ddf when time chunk is empty
|
2025-08-22 16:06:05 +02:00 |
|
|
|
8947046049
|
Feature: remove saving of ddf_pbp_hk_dt files
|
2025-08-22 16:03:43 +02:00 |
|
|
|
547c7f3108
|
Fix: removed default values in parse_args to avoid unexpected behavior when passing run_config settings
|
2025-08-22 16:01:01 +02:00 |
|
|
|
9c384f5245
|
Fix: Removed default values in the load_and_resolve_config function to avoid unexpected behavior when run_config doesn't provide settings
|
2025-08-22 15:59:48 +02:00 |
|
|
|
f42a308474
|
Feature: processing is now divided in time chunks to reduce size of dask graph
|
2025-08-22 11:37:58 +02:00 |
|
|
|
d6b3f2028f
|
Fix: bug fixed in conversion from BC mass to diam due to density units mismatch in the config file and default values
|
2025-08-22 10:48:08 +02:00 |
|
|
|
bbd21ba7b9
|
Chore: add test data folder to gitignore
|
2025-08-21 11:58:39 +02:00 |
|
|
|
bf0e663449
|
Test: add temporary notebook for tests
|
2025-08-21 11:57:51 +02:00 |
|
|
|
ebd14bcbae
|
Fix: typo in the config reading was blocking calibration
|
2025-08-21 11:39:11 +02:00 |
|
|
|
d7f778d531
|
Fix: the hisotgrams were adding lines with NaNs when the corresponding partition was completely empty. Now it is back to old behavior and no index is added for partitions completely empty.
|
2025-08-21 11:37:23 +02:00 |
|
|
|
063b01e73f
|
Test: PbP and HK parquet files added for testing
|
2025-08-20 19:08:28 +02:00 |
|
|
|
a2cc520ff2
|
feat: possibility to choose between running locally and via slurm cluster
|
2025-08-14 11:48:52 +02:00 |
|
|
|
18b8635147
|
chore: moved config files part 2
|
2025-08-14 10:42:44 +02:00 |
|
|
|
d470f2e811
|
chore: moved config files
|
2025-08-14 10:41:44 +02:00 |
|
|
|
176dde251f
|
Refactor: remove name key from BC_hist_configs dictionary
|
2025-08-13 17:22:47 +02:00 |
|
|
|
554ec26410
|
Refactor: moved parts of processing code to specific functions in separate modules (join_pbp_with_flow and aggregate_dt)
|
2025-08-13 17:02:47 +02:00 |
|
|
|
053d0d5b75
|
Refactor: separte the workflow for the option to calculate distributions for BC (numb and amss), scattering and time delay
|
2025-08-13 16:26:39 +02:00 |
|
|
|
1038b18187
|
Clean up old parts in the code and re-arranged processing order
|
2025-08-13 16:08:37 +02:00 |
|
|
|
5f3d25817c
|
Refactor: moved calibration of PbP data to new file and clear separation between run_config and instr_config
|
2025-08-13 15:28:57 +02:00 |
|
|
|
6f87b4cc79
|
Refactor: implemented input run_config.yaml file
|
2025-08-13 14:09:58 +02:00 |
|
|
|
fa4cb9e7d4
|
Refactor: moved some functions to respective modules and implemented the timelag hists
|
2025-08-12 15:53:48 +02:00 |
|
|
|
b05853416b
|
Refactor: functions moved from sp2xr_pipeline to corresponding script in the src directory
|
2025-08-08 16:53:29 +02:00 |
|
|
|
7e736803c1
|
Refactor: working version of sp2xr_pipeline but not yet polished
|
2025-08-08 11:54:32 +02:00 |
|
|
|
6fa6fabf03
|
refactor: moved some functions from toolkit_legacy
|
2025-08-06 09:00:16 +02:00 |
|
|
|
3e620c80f0
|
refactor: first draft for the new sp2xr processing pipeline
|
2025-08-06 08:56:32 +02:00 |
|
|
|
63359449d3
|
refactor: split out single-particle calibration into its own module
|
2025-08-04 18:43:25 +02:00 |
|
|
|
74c5a77b48
|
test: manually added case-specific instrument_config parameters
|
2025-08-04 13:03:24 +02:00 |
|
|
|
ad897be5db
|
feat: pre-populate YAML export with calibration and threshold placeholders
|
2025-08-04 12:58:17 +02:00 |
|
|
|
f5c5420209
|
feat: add CLI utility to convert SP2-XR .ini files to editable YAML configs
|
2025-08-04 12:33:09 +02:00 |
|
|
|
1d8f12ee35
|
feat: add ini→yaml conversion and tests for SP2-XR instrument parameters
|
2025-08-04 12:26:03 +02:00 |
|
|
|
83c95a9cdd
|
feat: set default walltime dynamically based on SLURM partition
|
2025-08-04 11:42:13 +02:00 |
|
|
|
2ca7704b7e
|
chore: add bokeh for Dask dashboard support
|
2025-08-04 10:43:44 +02:00 |
|
