Commit Graph

88 Commits

Author SHA1 Message Date
e24d451542 Updated thirdparty submodule to latest version 2025-02-22 15:02:17 +01:00
0911260f26 Save changes. 2025-02-15 18:20:58 +01:00
a516479036 Change data lineage format. Now, data lineage information is accessed using name of the file as key. 2025-02-15 17:57:50 +01:00
ca08044569 Update keys of data_lineage_metadata.yaml. keys are now the filenames in the folder. 2025-02-11 13:14:14 +01:00
80fbe3af91 Update params/validity_thresholds.yaml with values from 2024 JFJ ACSM data 2025-02-11 13:12:00 +01:00
0623c2462b Adjust command line interface of steps/apply_... and steps/compute_aut... 2025-02-11 13:07:29 +01:00
f2854d4ca0 Implement pipelines/steps/visualize_datatable_vars.py 2025-02-11 13:03:46 +01:00
b966151d9c Update with template for compute_cpc_flags(). This still needs to be completed. 2025-02-10 10:54:18 +01:00
ca2b8b6787 Rename apply_diagnostic_flags.py to compute_automated_flags.py because the function also computes both cpc and diagnostic based flags 2025-02-10 10:45:55 +01:00
d93003f5c4 Add import statement and adjust dependencies. 2025-02-06 16:57:46 +01:00
ad95d11a0d Implement utils module for reusable functions across steps in the pipelines 2025-02-06 16:54:36 +01:00
6ed9fe1eb1 Rename pipeline step to apply_diagnositc_flags, it perform a check based on specified thresholds and then assigns I, V, or M labels producing a labeling of a variable. 2025-02-06 16:25:40 +01:00
f8b900422b Rename function 2025-02-06 16:15:19 +01:00
2c2b154528 Update to account for yaml file attribute renamings. 2025-02-05 18:15:43 +01:00
e2b5aa9d69 Update file structure to depend on calibration_parameters.yaml and update thresholds for JFJ/ACSM_TOFWARE/2024 data. 2025-02-05 17:05:07 +01:00
afd6ef4d1b Fix typo in denominators. 2025-02-05 16:49:36 +01:00
bc779bd583 Update calibration parameters for JFJ/ACSM_TOFWARE/2024 data. 2025-02-05 16:48:14 +01:00
48e2127fef Reimplement apply_calibration_factors() in terms of load_calibration_file() and compute_calibration_factors() to reduce complexity and improve modularity. 2025-02-04 18:12:42 +01:00
a5a9fa9c3f Add anchor to default params from tofware and use alias to complete interval specific params coming from time period specific calibrations 2025-02-04 14:31:42 +01:00
484eb1ae2c Change path to file to be relative with respect to project directory path. 2025-02-03 16:13:19 +01:00
caa40304c5 Fix typo 2025-02-03 16:11:32 +01:00
baf755cebd Move params defined in calibration_factors.yaml to a file calibration_params.yaml to decouple user-input from low-level pipeline parameters. 2025-02-03 11:53:20 +01:00
fba121cf5c Remove anchors and replace aliases with the name of the keyword they refer to. Example, keyword : keyword_anchor value, *keyword_anchor replaced as keyword. 2025-02-03 11:28:57 +01:00
713a4c4fe7 Add calibration intevals keyword, which enables adding calibration parameters for specific time intervals. The calibration function is actually a piecewise function, where each piece uses a different set of calibration params. 2025-02-03 10:54:16 +01:00
c4706e2f3a rename file and add flow_ref parameter to it 2025-01-31 10:03:34 +01:00
bad191637b Move flow_ref paramter to operational_var_range.yaml (soon to be called validity_threshold.s) 2025-01-31 10:00:51 +01:00
53463a2f8e changes_from_meeting 2025-01-30 15:33:30 +01:00
afbde68164 Changed names of calibration factors to specify they are the ones from tofware 2025-01-30 14:20:05 +01:00
8b25bdb990 Save changes to calibration factors. 2025-01-30 13:34:39 +01:00
95f13d05b2 Save changes to calibration factors and apply_calibration_factors.py after git pull operation 2025-01-30 13:33:44 +01:00
1f119991f2 Merge branch 'main' of https://gitlab.psi.ch/apog/acsmnode 2025-01-30 13:29:56 +01:00
b19ab3d867 Create yaml file with LODs for various variables. 2025-01-30 13:17:13 +01:00
d8aad24ee9 Move limits of detection from here to limits_of_detection.yaml 2025-01-30 13:16:18 +01:00
13fb2c7055 Rename and restructure yaml file with ranges for diagnostic variables 2025-01-30 13:12:42 +01:00
d69e4182a8 Fix alias reference pairs. YAML does not allow references to individual elements of a dictionary using *dictAlias.keyword notation 2025-01-29 19:28:43 +01:00
f3ec8c6acf Fix typo, i was not properly accessing the key variables in the yaml dictionary 2025-01-29 19:27:18 +01:00
23e83134d2 Restructure calibration file to make more userfriendly 2025-01-29 18:01:52 +01:00
9bb56cd3da Implement first version of processing pipeline. 2025-01-29 09:16:26 +01:00