|
|
a9533ae3e8
|
Refine function. Abstract out load parameters as function to simplify code.
|
2025-02-28 17:18:13 +01:00 |
|
|
|
6890377134
|
Remove dataset_name parameter from command line interface. Instead i now infer it assuming there is a consistent input filename in the hdf5 file that can be found using hard coded keywords.
|
2025-02-28 17:17:33 +01:00 |
|
|
|
33f29a8080
|
WIP: Completed flag generation for species, command line interface is now clear but still does not consider potentially existing flags. Review TODO.md, for efficiency we need to simplify flag assignment and reconciliation for the species flag type.
|
2025-02-26 15:11:12 +01:00 |
|
|
|
600c60be36
|
Append variable specification for explicit definition of diagnostics and species to pipelines/params/calibration_params.yaml. This is to support flagging operations to particular variables
|
2025-02-26 15:07:58 +01:00 |
|
|
|
4edd6809e9
|
WIP: Redesigned command line functionality. options --species is half baked :). It runs without error but second part needs validation.
|
2025-02-22 16:00:27 +01:00 |
|
|
|
e24d451542
|
Updated thirdparty submodule to latest version
|
2025-02-22 15:02:17 +01:00 |
|
|
|
0911260f26
|
Save changes.
|
2025-02-15 18:20:58 +01:00 |
|
|
|
a516479036
|
Change data lineage format. Now, data lineage information is accessed using name of the file as key.
|
2025-02-15 17:57:50 +01:00 |
|
|
|
ca08044569
|
Update keys of data_lineage_metadata.yaml. keys are now the filenames in the folder.
|
2025-02-11 13:14:14 +01:00 |
|
|
|
80fbe3af91
|
Update params/validity_thresholds.yaml with values from 2024 JFJ ACSM data
|
2025-02-11 13:12:00 +01:00 |
|
|
|
0623c2462b
|
Adjust command line interface of steps/apply_... and steps/compute_aut...
|
2025-02-11 13:07:29 +01:00 |
|
|
|
f2854d4ca0
|
Implement pipelines/steps/visualize_datatable_vars.py
|
2025-02-11 13:03:46 +01:00 |
|
|
|
b966151d9c
|
Update with template for compute_cpc_flags(). This still needs to be completed.
|
2025-02-10 10:54:18 +01:00 |
|
|
|
ca2b8b6787
|
Rename apply_diagnostic_flags.py to compute_automated_flags.py because the function also computes both cpc and diagnostic based flags
|
2025-02-10 10:45:55 +01:00 |
|
|
|
d93003f5c4
|
Add import statement and adjust dependencies.
|
2025-02-06 16:57:46 +01:00 |
|
|
|
ad95d11a0d
|
Implement utils module for reusable functions across steps in the pipelines
|
2025-02-06 16:54:36 +01:00 |
|
|
|
6ed9fe1eb1
|
Rename pipeline step to apply_diagnositc_flags, it perform a check based on specified thresholds and then assigns I, V, or M labels producing a labeling of a variable.
|
2025-02-06 16:25:40 +01:00 |
|
|
|
f8b900422b
|
Rename function
|
2025-02-06 16:15:19 +01:00 |
|
|
|
2c2b154528
|
Update to account for yaml file attribute renamings.
|
2025-02-05 18:15:43 +01:00 |
|
|
|
e2b5aa9d69
|
Update file structure to depend on calibration_parameters.yaml and update thresholds for JFJ/ACSM_TOFWARE/2024 data.
|
2025-02-05 17:05:07 +01:00 |
|
|
|
afd6ef4d1b
|
Fix typo in denominators.
|
2025-02-05 16:49:36 +01:00 |
|
|
|
bc779bd583
|
Update calibration parameters for JFJ/ACSM_TOFWARE/2024 data.
|
2025-02-05 16:48:14 +01:00 |
|
|
|
48e2127fef
|
Reimplement apply_calibration_factors() in terms of load_calibration_file() and compute_calibration_factors() to reduce complexity and improve modularity.
|
2025-02-04 18:12:42 +01:00 |
|
|
|
a5a9fa9c3f
|
Add anchor to default params from tofware and use alias to complete interval specific params coming from time period specific calibrations
|
2025-02-04 14:31:42 +01:00 |
|
|
|
484eb1ae2c
|
Change path to file to be relative with respect to project directory path.
|
2025-02-03 16:13:19 +01:00 |
|
|
|
caa40304c5
|
Fix typo
|
2025-02-03 16:11:32 +01:00 |
|
|
|
baf755cebd
|
Move params defined in calibration_factors.yaml to a file calibration_params.yaml to decouple user-input from low-level pipeline parameters.
|
2025-02-03 11:53:20 +01:00 |
|
|
|
fba121cf5c
|
Remove anchors and replace aliases with the name of the keyword they refer to. Example, keyword : keyword_anchor value, *keyword_anchor replaced as keyword.
|
2025-02-03 11:28:57 +01:00 |
|
|
|
713a4c4fe7
|
Add calibration intevals keyword, which enables adding calibration parameters for specific time intervals. The calibration function is actually a piecewise function, where each piece uses a different set of calibration params.
|
2025-02-03 10:54:16 +01:00 |
|
|
|
c4706e2f3a
|
rename file and add flow_ref parameter to it
|
2025-01-31 10:03:34 +01:00 |
|
|
|
bad191637b
|
Move flow_ref paramter to operational_var_range.yaml (soon to be called validity_threshold.s)
|
2025-01-31 10:00:51 +01:00 |
|
|
|
53463a2f8e
|
changes_from_meeting
|
2025-01-30 15:33:30 +01:00 |
|
|
|
afbde68164
|
Changed names of calibration factors to specify they are the ones from tofware
|
2025-01-30 14:20:05 +01:00 |
|
|
|
8b25bdb990
|
Save changes to calibration factors.
|
2025-01-30 13:34:39 +01:00 |
|
|
|
95f13d05b2
|
Save changes to calibration factors and apply_calibration_factors.py after git pull operation
|
2025-01-30 13:33:44 +01:00 |
|
|
|
1f119991f2
|
Merge branch 'main' of https://gitlab.psi.ch/apog/acsmnode
|
2025-01-30 13:29:56 +01:00 |
|
|
|
b19ab3d867
|
Create yaml file with LODs for various variables.
|
2025-01-30 13:17:13 +01:00 |
|
|
|
d8aad24ee9
|
Move limits of detection from here to limits_of_detection.yaml
|
2025-01-30 13:16:18 +01:00 |
|
|
|
13fb2c7055
|
Rename and restructure yaml file with ranges for diagnostic variables
|
2025-01-30 13:12:42 +01:00 |
|
|
|
d69e4182a8
|
Fix alias reference pairs. YAML does not allow references to individual elements of a dictionary using *dictAlias.keyword notation
|
2025-01-29 19:28:43 +01:00 |
|
|
|
f3ec8c6acf
|
Fix typo, i was not properly accessing the key variables in the yaml dictionary
|
2025-01-29 19:27:18 +01:00 |
|
|
|
23e83134d2
|
Restructure calibration file to make more userfriendly
|
2025-01-29 18:01:52 +01:00 |
|
|
|
9bb56cd3da
|
Implement first version of processing pipeline.
|
2025-01-29 09:16:26 +01:00 |
|
