|
|
52a2303054
|
Developed a class to manage data operations on a given hdf5 file
|
2024-08-09 13:23:54 +02:00 |
|
|
|
fbc8c5ebc3
|
Removed time stamp configuration attributes from ACSM_TOFWARE, because it can be messy for a configuration file.
|
2024-08-08 11:24:41 +02:00 |
|
|
|
780b2302b3
|
Updated file with new instrument configuration ACSM.
|
2024-08-07 16:38:52 +02:00 |
|
|
|
99fb2de6d8
|
Moved ext_to_reader_dict to g5505_file_reader.py and replaced redear selection based on g5505_reader.select_file_reader(hdf5_file_path).
|
2024-08-07 16:30:36 +02:00 |
|
|
|
381d330ee6
|
Moved hdf5_file_path to file reader mapping and extension definitions to g5505_file_reader_module.py. Created functions to compute file_reader key from path to file in the hdf5 file and select the reader based on the key. This should enable more modular file reader selection.
|
2024-08-07 16:21:22 +02:00 |
|
|
|
1acbd2f758
|
Modified reader to output table_preamble as a dataset as opposed to attributes of the file. I believe this is better for readability of the metadata given that those preambles can sometimes contain large ammounts of text.
|
2024-08-02 14:37:06 +02:00 |
|
|
|
d7f7223d31
|
Modified .yaml config files to satisty metadata naming expectations.
|
2024-07-17 08:50:24 +02:00 |
|
|
|
79a593cbbb
|
Changed names of expected root level metadata attributes.
|
2024-07-17 08:48:47 +02:00 |
|
|
|
6c50625002
|
Added attribution insertion order tracking at the root level and reorganized a few import statements.
|
2024-07-17 08:41:40 +02:00 |
|
|
|
085ddda0b2
|
Made edits to documentation
|
2024-07-11 13:42:38 +02:00 |
|
|
|
6ba5a1fa2e
|
Robustified column name to description assigment, however it may be a bit slower than before.
|
2024-07-10 13:31:47 +02:00 |
|
|
|
cbc560f7e0
|
Updated the yaml instrument descriptions.
|
2024-07-10 13:29:14 +02:00 |
|
|
|
586dcef621
|
Moved parse_attribute() from ..review_lib.py into ...utils.py and backpropagate (refactored) changes to respective modules.
|
2024-07-10 11:32:00 +02:00 |
|
|
|
8c93c2d97b
|
Modified return datetime output to a format without colons, which could be problematic for filenaming.
|
2024-07-10 09:47:56 +02:00 |
|
|
|
3d8b46cf05
|
Moved a few functions from ...reader.py and hdf5_lib.py into ..utils.py, and refactored accordingly.
|
2024-07-10 09:19:30 +02:00 |
|
|
|
407b287c56
|
Removed smogchamber reader because its funtionality is now integrated into g5505_file_reader.py.
|
2024-07-09 16:13:01 +02:00 |
|
|
|
aa69faa995
|
Removed non utilized code.
|
2024-07-08 15:29:13 +02:00 |
|
|
|
2992f0a645
|
Cleaned code and modified def create_hdf5_file_from_dataframe to create group hierichy implicitly from path rather than recursively.
|
2024-07-08 15:24:48 +02:00 |
|
|
|
635b158dad
|
Moved remaining git operations in metadata_review_lib.py to git_ops.py and refactored accoringly
|
2024-07-05 15:46:20 +02:00 |
|
|
|
57ee91df7d
|
Merge branch 'main' of https://gitlab.psi.ch/5505/dima
|
2024-07-02 16:50:08 +02:00 |
|
|
|
926a0f9e08
|
Updated documentation.
|
2024-07-02 16:49:48 +02:00 |
|
|
|
1287d8d31f
|
Merge branch 'main' of https://gitlab.psi.ch/5505/dima
|
2024-07-01 16:20:06 +02:00 |
|
|
|
fe2e9400fd
|
Modified created at function to output date time and time zone
|
2024-07-01 16:19:28 +02:00 |
|
|
|
b21ccbddf0
|
Renamed script_name to processing_file.
|
2024-07-01 16:17:25 +02:00 |
|
|
|
2903856f46
|
Made a few edits.
|
2024-06-21 15:55:44 +02:00 |
|
|
|
29be99d479
|
Merge branch 'main' of https://gitlab.psi.ch/5505/dima
|
2024-06-21 15:42:46 +02:00 |
|
|
|
c8113dd0d2
|
Cleared out outputs.
|
2024-06-21 15:42:23 +02:00 |
|
|
|
dde01bae8b
|
Added a few root level metadata names and definitions
|
2024-06-21 15:40:38 +02:00 |
|
|
|
72fc77d755
|
Implemented input argument to enable append information to exisintg attributes, which must take the values of either strings or lists.
|
2024-06-20 15:32:33 +02:00 |
|
|
|
0cc6cf0785
|
Added a few lines to detect the existence of the file and change the file mode from 'w' to 'a' based on that information.
|
2024-06-20 09:03:47 +02:00 |
|
|
|
210379a2b4
|
Updated function to add project level metadata at the root group of the hdf5 file.
|
2024-06-19 18:31:11 +02:00 |
|
|
|
ee377ef30a
|
Incorporated method to MetadataHarvester class to collect project level metadata.
|
2024-06-19 18:30:02 +02:00 |
|
|
|
2113a17e40
|
Added code to parse dict attributes.
|
2024-06-18 14:42:51 +02:00 |
|
|
|
60f4497711
|
Fixed bug regarding datetime to str column conversion in dataframe by using .map(srt) (element wise operation) as opposed to .apply(str)
|
2024-06-18 09:21:46 +02:00 |
|
|
|
2ea9269f75
|
Replaced applymap to .apply because the former is being depricated
|
2024-06-17 13:47:54 +02:00 |
|
|
|
86a811e6aa
|
Created function to save dataframes with annotations in hdf5 format
|
2024-06-17 13:36:05 +02:00 |
|
|
|
652f311c8d
|
Added metadata printer method and rewrote slightly a few class terms.
|
2024-06-17 08:44:44 +02:00 |
|
|
|
9f6533e53b
|
Incorporated dataframe_to_np_structured_array(df: pd.DataFrame) from another module.
|
2024-06-16 18:39:30 +02:00 |
|
|
|
bda5e87cc8
|
Incorporated dataframe_to_np_structured_array(df: pd.DataFrame) from another module.
|
2024-06-16 18:26:12 +02:00 |
|
|
|
d2e53dca3f
|
Moved dataframe_to_np_structured_array(df: pd.DataFrame) to src/g5505_utils.py. This is a more generic function that can be used more broadly accross modules.
|
2024-06-16 18:25:08 +02:00 |
|
|
|
f7f91aa105
|
Removed lenthy example.
|
2024-06-13 16:03:04 +02:00 |
|
|
|
1c937222fd
|
Replaced add_data_level_info to add_dataset.
|
2024-06-13 16:01:27 +02:00 |
|
|
|
2a28d45b13
|
Developed a metadata harvesting object to facilitate metadata collection throught the code.
|
2024-06-13 15:47:02 +02:00 |
|
|
|
71e1fffd1a
|
Modified a few variable values in yaml files so that they are within expected values.
|
2024-06-13 15:45:39 +02:00 |
|
|
|
622661d4d3
|
Abstracted a code snippet from def create_hdf5_file_from_filesystem_path(..) as transfer_file_dict_to_hdf5() so that it can be reusable.
|
2024-06-13 15:44:01 +02:00 |
|
|
|
9b70493fbf
|
Modified hardcoded paths to adapt with respect to the parent directory
|
2024-06-11 17:30:58 +02:00 |
|
|
|
cf82678f9e
|
Implemented a data extraction module to access data from an hdf5 file in the form of dataframes.
|
2024-06-11 10:38:04 +02:00 |
|
|
|
ba5b8cb407
|
Removed data table split into categorical and numerical variables and numering is only introduce to disambiguate repeated columns.
|
2024-06-10 16:18:51 +02:00 |
|
|
|
ed33e77380
|
Removed additional numbering from some intrument specifications. These are now only added if the column names are ambigous.
|
2024-06-10 16:14:13 +02:00 |
|
|
|
b47ed2b3f4
|
Fixed bug in the case where data_integration_mode = 'collection'.
|
2024-06-07 16:45:00 +02:00 |
|
