|
|
457a2ae016
|
Update workflow step names
|
2025-06-07 20:01:02 +02:00 |
|
|
|
160791b738
|
Refactor steps to collect information for renku workflow file generation
|
2025-06-06 17:02:13 +02:00 |
|
|
|
196e237838
|
Update pipelines/steps/utils.py. Changes uncertainty_estimate
|
2025-05-28 08:48:38 +02:00 |
|
|
|
f3f830487e
|
Extend pipelines/steps/adjust_uncertainty_column_in_nas_file.py to handle list of variables.
|
2025-05-27 09:53:12 +02:00 |
|
|
|
08ba10dc48
|
Implement new step pipelines/steps/adjust_uncertainty_column_in_nas_file.py.
|
2025-05-26 19:50:18 +02:00 |
|
|
|
989e99078a
|
Merge branch 'main' of https://gitlab.psi.ch/apog/acsmnode
This updates todo.md
|
2025-05-26 13:12:52 +02:00 |
|
|
|
52fdb7a8ad
|
Fix number of variables by subtracting one. Start_time is now excluded from count.
|
2025-05-26 13:12:40 +02:00 |
|
|
|
f7e1c31550
|
added len(interpolated_cpc)
|
2025-05-21 16:35:50 +02:00 |
|
|
|
a7faf69c96
|
Changed missing value code generator, any number greater than the largest possible one that can be represented in terms of 9s is capped
|
2025-05-21 15:17:25 +02:00 |
|
|
|
e39c23bbad
|
Update generate_flags.py to skip checking for cpc flags when not available
|
2025-05-21 09:48:19 +02:00 |
|
|
|
01769f10e0
|
Implement new step to remove column from already created nas file.
|
2025-05-21 09:43:36 +02:00 |
|
|
|
a9461cb1aa
|
Added limits on y axis for flags visualization
|
2025-05-20 16:27:11 +02:00 |
|
|
|
cc1b445d38
|
Implement binary flags for visualization purposes, and fix bug with cpc flags.
|
2025-05-16 14:22:17 +02:00 |
|
|
|
58d7b425aa
|
Add generate cpc flags pipelines/steps/generate_flags.py
|
2025-05-15 15:01:11 +02:00 |
|
|
|
37671a3685
|
update pipelines/steps/visualize_datatable_vars.py with interactive plots and refactor dependent jupyter nb accordingly
|
2025-05-13 11:32:46 +02:00 |
|
|
|
4408514439
|
Rename attribute station --> station_abbr
|
2025-04-11 11:20:48 +02:00 |
|
|
|
e7863078aa
|
Add data chain step /update_datachain_params.py. This uses params specified in input data folder to update current data chain params
|
2025-04-11 11:07:52 +02:00 |
|
|
|
cea83396d2
|
Add function to estimate project directory path
|
2025-04-11 11:05:41 +02:00 |
|
|
|
68a7d5dc2b
|
Fix out of bound error in pipelines/steps/visualize_datatable_vars.py
|
2025-04-11 11:04:00 +02:00 |
|
|
|
c9e25c2483
|
Add new placeholder for submitter name and remove metadata element 'station' conflicting with third party converter
|
2025-04-11 11:02:38 +02:00 |
|
|
|
881209bf82
|
Update steps to allow filenames for PAYERN station
|
2025-04-08 21:26:01 +02:00 |
|
|
|
3dfed2c5f3
|
Update pipelines/steps/prepare_ebas_submission.py. Rmoved hard coded paths and build output name using metadata from campaign descriptor. Also, we can now specify month ranges
|
2025-04-08 18:44:46 +02:00 |
|
|
|
5dd280e88c
|
Update pipelines/steps/utils.py. Now we can load the campaign descriptor using load_project_yaml_files()
|
2025-04-08 18:43:01 +02:00 |
|
|
|
bfaaeb4416
|
Add header templates for JFJ and PAY
|
2025-04-08 18:35:44 +02:00 |
|
|
|
ca0366338a
|
Add pipeline step to update actris_header with submitter and originator
|
2025-04-08 18:34:51 +02:00 |
|
|
|
00a85505c8
|
Fix bug when infering name of flags file from dataset name
|
2025-04-04 16:53:41 +02:00 |
|
|
|
d290daaa39
|
Update pipelines/steps/visualize_datatable_vars.py. Specify alternative path to flags in case they are not in the HDF5 file. We assume there is an append directory where they may be. Otherwise it throws and error.
|
2025-04-04 16:06:11 +02:00 |
|
|
|
b0d68747e9
|
Update code to rearrange table columns based on column_order from acsm_to_ebas.yaml. This can be removed in the future.
|
2025-03-18 07:23:54 +01:00 |
|
|
|
261c8b8154
|
Add column order attributes, storing the order in which variables need to be displayed to adhere to Nora's initial implementation.
|
2025-03-18 07:22:06 +01:00 |
|
|
|
b93cb7517b
|
Add comment to indicate where params are defined in original code.
|
2025-03-14 13:40:22 +01:00 |
|
|
|
7d277e3e3b
|
Clean up print statements.
|
2025-03-14 13:39:12 +01:00 |
|
|
|
39a9ab07a2
|
Refactor command line interface to run a main function on the parsed arguments.
|
2025-03-14 13:36:52 +01:00 |
|
|
|
8cdd8a0771
|
Completed command line interface for pipelines/steps/prepare_ebas_submission.py. This finishes WIP associated with commit 2eb88e4.
|
2025-03-14 13:09:09 +01:00 |
|
|
|
b643eb2d26
|
Refactored command line interface, generate_species_flags, and created functions to improve performance and code clarity.
|
2025-03-13 14:17:46 +01:00 |
|
|
|
97f253074b
|
Add in try-except block a finally statement to close hdf5 file for secure file closing. This was done before in the try case which is not good practice
|
2025-03-13 14:06:01 +01:00 |
|
|
|
d6fa51effc
|
Fix typo and missing key. 1) Update allowed yaml files in load_project_yaml_files by adding pipelines/params/calibration_factors.yaml. 2) Correct filename access after os.path.split operation.
|
2025-03-10 13:02:38 +01:00 |
|
|
|
4ca99f4f04
|
Rewrote load_calib_file() in terms pipelines.steps.utils.load_project_yaml_files(). In the future we will replace it completely so we can validate and manage all yaml files in the chain in a more centralized manner.
|
2025-03-10 11:46:24 +01:00 |
|
|
|
ecdf3687ba
|
Remove/replace def load_parameters(flag_type) with steps.utils.load_project_yaml_files(.). This is to have a more centralized treatment of yaml files inside the data chain.
|
2025-03-10 11:38:51 +01:00 |
|
|
|
cf073126b9
|
Updated acsm raw to ebas terminology mapping. It is now complete!
|
2025-03-07 16:53:22 +01:00 |
|
|
|
2eb88e444c
|
WIP: finished last commit, and tested the whole step. Now we need to implement the command line interface.
|
2025-03-07 16:51:06 +01:00 |
|
|
|
eb8a36dc8e
|
Update pipelines/steps/apply_calibration_factors.py to generate error table and cleaned up command line interface.
|
2025-03-07 16:49:32 +01:00 |
|
|
|
cbad39355b
|
Reformat LOD variable names for simplicity
|
2025-03-07 16:46:48 +01:00 |
|
|
|
78340464aa
|
Add functions: generate_error_dataframe() with missing values, metadata_dict_to_dataframe(), and load_project_yaml_files() to easily access data from yaml files in the project.
|
2025-03-07 16:45:33 +01:00 |
|
|
|
35e08d387f
|
Add pipelines/params/station_params.yaml, where station dependent temperature and preassure are to be defined. Crossed out associated TODO.md item.
|
2025-03-07 11:23:48 +01:00 |
|
|
|
f035afbf3b
|
Fix duplicate time stamp problems and added code to check for NaTs before ebas submission files.)
|
2025-03-04 09:02:37 +01:00 |
|
|
|
9a23744f63
|
Add acsm to ebas dictionary
|
2025-03-04 09:01:09 +01:00 |
|
|
|
6eccbb5018
|
Moved get_metadata() from pipelines/steps/prepare_ebas_submission.py to utils.py
|
2025-03-03 18:55:46 +01:00 |
|
|
|
def67a0562
|
Added correct calibration parameters to calibration factors table
|
2025-03-03 14:23:45 +01:00 |
|
|
|
d3a491ea85
|
Renamed pipeline step join_tables.py to prepare_ebas_submission.py for accuracy. This step performs more than joining tables, it also does renaming using dictionaries and generates the ebas data submission package
|
2025-03-03 10:48:03 +01:00 |
|
|
|
877c0178d7
|
WIP: Added first version of join_tables and tested txt generation. We should probaly make this step encompasse name renaming for EBAS and rename step to be more generic. Change join_tables.py to generate_ebas_converter_input_files
|
2025-02-28 17:23:00 +01:00 |
|
