|
|
59e910a0f9
|
Modified logger setup to create monthly logs
|
2024-11-21 11:46:11 +01:00 |
|
|
|
c866ce8ee9
|
Fixed command line interface bug
|
2024-11-10 18:19:59 +01:00 |
|
|
|
83b4a12e8b
|
Major code refactoring and simplifications to enhance modularity. Included a command line interface.
|
2024-11-01 09:52:41 +01:00 |
|
|
|
2ae8deaf64
|
Renamed the input argument yaml_review_file as review_yaml_file.
|
2024-11-01 09:51:12 +01:00 |
|
|
|
29c6d86583
|
Included cli commands update and serialize to simplify running metadata revision pipeline.
|
2024-10-29 07:56:43 +01:00 |
|
|
|
e01147d03a
|
Removed unused import statements
|
2024-10-28 16:37:32 +01:00 |
|
|
|
c0ec7e9b26
|
Moved git related operations from pipelines/ to src/git_ops.py
|
2024-10-28 16:30:34 +01:00 |
|
|
|
66e236bd5d
|
Added function to validate review yaml file, and updated update_hdf5_with_review function
|
2024-10-28 16:20:28 +01:00 |
|
|
|
a8ceffba8b
|
Corrected import statements due to dependency name changes
|
2024-10-17 16:52:42 +02:00 |
|
|
|
9a76b3a01a
|
Added 'filename_format' attribute to YAML schema. It takes as value a string of comma separated keys from available attributes in YAML file.
|
2024-10-14 16:01:24 +02:00 |
|
|
|
5cc409f882
|
Fixed bug introduce in logger due to invalid date naming replace : with -
|
2024-10-10 14:29:36 +02:00 |
|
|
|
81e3cc58ca
|
Updated function dependencies to reflect changes made to hdf5_ops.py
|
2024-10-10 11:02:05 +02:00 |
|
|
|
96500063fb
|
Implemented metadata append, rename, delete, and update operations on the hdf5 manager object and refactored metadata update script based on yaml file to use said operations.
|
2024-09-30 16:32:39 +02:00 |
|
|
|
db6fcd03da
|
Refactored a few function calls due to ranming changes in utils module
|
2024-09-27 08:58:35 +02:00 |
|
|
|
8f9e2fc594
|
Moved is_structured_array() and to_serializable_dtype() to utils, ranamed a few functions and propagated changes to dependent modules.
|
2024-09-26 14:03:11 +02:00 |
|
|
|
7ab615019a
|
Renamed take_yml_snapshot_of_hdf5_file func as to_yaml func
|
2024-09-25 16:49:44 +02:00 |
|
|
|
3e143fb9c7
|
Abstracted reusable steps in integration_sources as dima_pipeline.py and added functionality to make a collection of hdf5 files, where each represents an single experiment of campaign.
|
2024-09-25 15:23:23 +02:00 |
|
|
|
0e354a0f14
|
Moved src/metadata_review_lib.py pipelines/metadata_revision.py
|
2024-09-17 16:55:22 +02:00 |
|
|
|
de859102ab
|
Moved src/data_integration_lib.py -> pipelines/data_integration.py
|
2024-09-17 15:32:23 +02:00 |
|
