41 Commits

Author SHA1 Message Date
617a923fb6 Add processing_Script and processing_data and actris_level to output metadata 2025-06-19 20:41:17 +02:00
b96c04fc01 Refactor instruments/readers/g5505_text_reader.py, some code abstracted as functions to improve readabilitity. 2025-06-19 20:40:14 +02:00
83cec97e83 Fix bug instruments/readers/structured_file_reader.py. pd.to_dict return a list of dicts so we need to handle each item seprately using a loop. 2025-06-07 19:14:53 +02:00
f640205b12 Add new file reader instruments/readers/structured_file_reader.py, and update registry.py and yaml 2025-06-07 18:15:41 +02:00
87462211a9 Update instruments/readers/nasa_ames_reader.py to handle dirty text entries. Dirty entries of time variables that cannot be properly processed are sent to nat 2025-05-27 10:12:20 +02:00
6851f03dbd Restore instruments/readers/nasa_ames_reader.py to previous version. 2025-05-26 19:39:21 +02:00
bd74f8310c Record missing values for each variable according to EBAS value convention 2025-05-21 13:53:18 +02:00
e4b2a4cd5a Split header in three parts and detect variables and variable descriptions added to attribute dictionary 2025-05-21 09:19:16 +02:00
a22532d08d Register new file reader in the reader registry system. 2025-05-14 13:51:28 +02:00
ad4339a76b Added new filereader dictionary pair for nasames files. This is a first version that may change. 2025-05-14 13:50:08 +02:00
32abd4cd56 Implemented hdf5_file_reader.py and updated register.yaml and hdf5_writer.py. This replaces previous function __copy_file_in_group(). 2025-02-25 12:25:15 +01:00
92a2560ed7 Update all file readers with command line interface so we can run them as a subprocess. Added also registry.yaml to decouple code from user-based instrument adaptations or extensions. 2025-02-24 17:27:12 +01:00
bb48cfa0cd Moved filereader_registry.py outside readers folder. 2025-02-22 17:51:56 +01:00
821d314cb6 Change import statements with try except to enable explicit import of submodules from import to avoid conflicts with parent project. 2025-02-22 17:10:53 +01:00
c28286a626 Make file reader selection case insensitive by using ext.lower() and update config_text_reader.py to point to renamed dictionary. 2025-02-08 19:45:16 +01:00
0b29e2ec68 remove instruments/dictionaries/ICAD_NO2.yaml. Its dict terms are now in ICAD.yaml. 2025-02-08 19:23:37 +01:00
609bb0b859 Add dict terms from ICAD_NO2.yaml 2025-02-08 19:22:27 +01:00
ef7fe70bf0 Combine dictionaries of ICAD_HONO.yaml and ICAD_NO2.yaml into ICAD.yaml 2025-02-08 19:21:17 +01:00
f986edd4a5 Fix reader txt/csv default behavior. 2025-02-07 16:25:45 +01:00
32bba4239a Synch with remote repo 2025-02-03 10:31:48 +01:00
5b06548d88 Fix typo on extension items, extensions need to include a dot .json and .yaml. 2025-01-21 09:30:49 +01:00
45132b42ce Add json and yaml extensions to admissible file extension lists. 2025-01-21 08:57:38 +01:00
31a129694c Updated instrument dictionaries with variable descriptor names aligned with CF metadata conventions. 2024-11-25 15:49:49 +01:00
e486b4659c Added .pkl extension in the list of admissible file extensions 2024-11-21 11:47:41 +01:00
f1b2c64f66 Fixed bug when file reader not available. File reader registry now returns a reade that maps input to None. 2024-10-14 16:03:03 +02:00
ea82af2cd5 Cleaned up import statements and comment out path append operations 2024-10-10 14:27:50 +02:00
9a3bf77f37 Created file reader for acsm tofware files, updated registry and updated yaml file with instrument specific terms and reader config params. 2024-10-07 16:18:14 +02:00
098a79531c Added new instrument (flagging app) file reading capabilities. It includes two files a flag_reader.py that takes flag.json files produced by the app into a standard intermidiate representation, and a yaml file with instrument dependent description terms. Last, we modified the filereader_registry.py to find the new instrument file reader. 2024-10-03 09:07:06 +02:00
01b39b4c02 Added __init__.py inside intrument folders 2024-10-02 15:51:02 +02:00
96dad0bfb1 Renamed to_yaml() as serialize_metadata() and introduce input parameter output_format, which allows yaml or json. 2024-09-26 16:23:09 +02:00
0dbec94374 Fixed instrument_dir estimation to be bottom up, ie, based on path to file. Otherwise, it does not work when dima used as submodule 2024-09-19 15:47:11 +02:00
9789d312f9 Removed and splitted into instruments/readers/filereader_registry.py instruments/readers/g5505_text_reader.py instruments/readers/xps_ibw_reader.py 2024-08-23 16:09:04 +02:00
d866c8f9f9 Split instruments/readers/g5505_file_reader.py into a fileregistry.py and independent file readers. This is to improve instrument modularity and additions 2024-08-23 16:06:44 +02:00
aad51fb1b8 Renamed to reflect better the functionality of the file 2024-08-23 15:50:14 +02:00
a4b7c6a8b0 Moved copy_file_in_group() into hdf5_lib.py because it is not really doing the same role of all filereaders 2024-08-23 15:45:32 +02:00
b5c200d588 Moved all yaml files with dictionary terms for each instrument to dictionaries folder 2024-08-23 14:32:23 +02:00
0be48f8a21 Added ACSM_TOFWARE metadata descriptions 2024-08-23 14:23:32 +02:00
15b76f7704 Fixed a few import dependencies after relocating this file. 2024-08-23 10:57:13 +02:00
a0f44a1f4b Moved src/g5505_file_reader.py -> instruments/readers/g5505_file_reader.py to increase modularity with respect to new intrument additions. 2024-08-23 10:11:29 +02:00
5124df14d8 Changed link to descriptions according to new instrument folder location. 2024-08-12 13:40:43 +02:00
7f0e5384ea Moved instruments folder outside src/. 2024-08-12 10:09:21 +02:00