Commit Graph

17 Commits

Author SHA1 Message Date
0911260f26 Save changes. 2025-02-15 18:20:58 +01:00
a516479036 Change data lineage format. Now, data lineage information is accessed using name of the file as key. 2025-02-15 17:57:50 +01:00
ca08044569 Update keys of data_lineage_metadata.yaml. keys are now the filenames in the folder. 2025-02-11 13:14:14 +01:00
0623c2462b Adjust command line interface of steps/apply_... and steps/compute_aut... 2025-02-11 13:07:29 +01:00
f2854d4ca0 Implement pipelines/steps/visualize_datatable_vars.py 2025-02-11 13:03:46 +01:00
b966151d9c Update with template for compute_cpc_flags(). This still needs to be completed. 2025-02-10 10:54:18 +01:00
ca2b8b6787 Rename apply_diagnostic_flags.py to compute_automated_flags.py because the function also computes both cpc and diagnostic based flags 2025-02-10 10:45:55 +01:00
d93003f5c4 Add import statement and adjust dependencies. 2025-02-06 16:57:46 +01:00
ad95d11a0d Implement utils module for reusable functions across steps in the pipelines 2025-02-06 16:54:36 +01:00
6ed9fe1eb1 Rename pipeline step to apply_diagnositc_flags, it perform a check based on specified thresholds and then assigns I, V, or M labels producing a labeling of a variable. 2025-02-06 16:25:40 +01:00
f8b900422b Rename function 2025-02-06 16:15:19 +01:00
2c2b154528 Update to account for yaml file attribute renamings. 2025-02-05 18:15:43 +01:00
48e2127fef Reimplement apply_calibration_factors() in terms of load_calibration_file() and compute_calibration_factors() to reduce complexity and improve modularity. 2025-02-04 18:12:42 +01:00
95f13d05b2 Save changes to calibration factors and apply_calibration_factors.py after git pull operation 2025-01-30 13:33:44 +01:00
f3ec8c6acf Fix typo, i was not properly accessing the key variables in the yaml dictionary 2025-01-29 19:27:18 +01:00
23e83134d2 Restructure calibration file to make more userfriendly 2025-01-29 18:01:52 +01:00
9bb56cd3da Implement first version of processing pipeline. 2025-01-29 09:16:26 +01:00