Commit Graph

10 Commits

Author SHA1 Message Date
9a76b3a01a Added 'filename_format' attribute to YAML schema. It takes as value a string of comma separated keys from available attributes in YAML file. 2024-10-14 16:01:24 +02:00
5cc409f882 Fixed bug introduce in logger due to invalid date naming replace : with - 2024-10-10 14:29:36 +02:00
81e3cc58ca Updated function dependencies to reflect changes made to hdf5_ops.py 2024-10-10 11:02:05 +02:00
96500063fb Implemented metadata append, rename, delete, and update operations on the hdf5 manager object and refactored metadata update script based on yaml file to use said operations. 2024-09-30 16:32:39 +02:00
db6fcd03da Refactored a few function calls due to ranming changes in utils module 2024-09-27 08:58:35 +02:00
8f9e2fc594 Moved is_structured_array() and to_serializable_dtype() to utils, ranamed a few functions and propagated changes to dependent modules. 2024-09-26 14:03:11 +02:00
7ab615019a Renamed take_yml_snapshot_of_hdf5_file func as to_yaml func 2024-09-25 16:49:44 +02:00
3e143fb9c7 Abstracted reusable steps in integration_sources as dima_pipeline.py and added functionality to make a collection of hdf5 files, where each represents an single experiment of campaign. 2024-09-25 15:23:23 +02:00
0e354a0f14 Moved src/metadata_review_lib.py pipelines/metadata_revision.py 2024-09-17 16:55:22 +02:00
de859102ab Moved src/data_integration_lib.py -> pipelines/data_integration.py 2024-09-17 15:32:23 +02:00