Commit Graph

189 Commits

Author SHA1 Message Date
florez_j 784cb1eb62 Commented out openia python module. 2024-05-24 10:54:15 +02:00
florez_j d000a8348f Added bottom level instrument metadata descriptions such as units and description. 2024-05-24 09:50:25 +02:00
florez_j 8d4f4e68c7 Removed yaml file output from data integration file. The creation of this file is being outsource to data store repo 2024-05-24 09:32:30 +02:00
florez_j 88de88c316 Removed creation of yaml file subsequent to data integration. This can cause misalignment with data store. I think the yaml snapshot of a hdf5 file should therefore be outsourced there. 2024-05-24 09:30:24 +02:00
florez_j 1537633b1a Made a few optimizations to code and documentation. Expressions relying on list comprehensions were simplified with generator expressions. ex,: any([keyword in filename for keyword in select_file_keywords]) was simplified to any(keyword in filename for keyword in select_file_keywords). 2024-05-24 09:06:07 +02:00
florez_j d574ac382d Replaced attribute table_header in Lopap configuration file with a shorter version which is consistent accross more files. Some of the headers might change. 2024-05-24 08:55:36 +02:00
florez_j 63b683e4aa Optimzed and included df to np structured array conversion. \n-Replaced loop plus append with list comprehension. \n-Replaced pd df column concatenation based on row-wise concatenation with df.aggr() method that uses column wise concatenation. 2024-05-23 22:20:19 +02:00
florez_j bd458c6cd0 Optimzed and included df to np structured array conversion. \n-Replaced loop plus append with list comprehension. \n-Replaced pd df column concatenation based on row-wise concatenation with df.aggr() method that uses column wise concatenation. 2024-05-23 22:18:37 +02:00
florez_j a45fb4476b Replaced commented lines by accurate comments 2024-05-22 20:15:17 +02:00
florez_j 7367da84b9 Simplified code by updating HDF5 attributes using .update() dict method (inherited from dict type). 2024-05-22 20:11:54 +02:00
florez_j 1729cd40fa Added feature to interpret links to description in the yaml intrument configuration file and added them at the dataset level as attributes. 2024-05-09 19:17:08 +02:00
florez_j 1429c56916 Added link to descriptions and units of table variables/or columns. These can be used as attributes of datasets from tabular data 2024-05-09 19:15:20 +02:00
florez_j f49120102d Included timestamp specification, which indicates column names in a list that contain datetime information. 2024-04-30 14:51:58 +02:00
florez_j 553c3fe946 Incorparated feature to merge data and time data which may originally be in separate columns in text source files. This is specified in the text source specification yaml file 2024-04-30 14:50:33 +02:00
florez_j f3c2777bb0 Performed edits to README.md 2024-04-26 14:33:41 +02:00
florez_j 4fd3bb1957 Updated readme file with instructions on how to set compound attributes and delete them. 2024-04-26 14:27:01 +02:00
florez_j 493be88f49 Removed unecessary pygit depenedency and associated function that relied on it. 2024-04-26 13:15:33 +02:00
florez_j f25fc67fc1 Cleared out jupyter notebook. 2024-04-26 13:09:41 +02:00
florez_j 4d91e59279 Included new delete attribute and restart review features. 2024-04-26 13:08:27 +02:00
florez_j 14ae29bf3c Corrected parsing problem from hdf5 to yaml attribute. Single element arrays are now represented as a scalar as opposed to a list with a single element. 2024-04-26 12:54:41 +02:00
florez_j cefa2219e1 Submitted metadata review. 2024-04-26 11:30:05 +02:00
florez_j 2ef171c6d0 Updated hdf5 file with yaml review file. 2024-04-26 11:28:58 +02:00
florez_j 05dc90d8af Submitted metadata review. 2024-04-26 11:07:42 +02:00
florez_j 7469003524 Initialized metadata review. 2024-04-26 10:56:32 +02:00
florez_j 6c73fbb695 Submitted metadata review. 2024-04-25 16:56:06 +02:00
florez_j 70b0c10316 Submitted metadata review. 2024-04-25 16:47:44 +02:00
florez_j 836fe45fa0 Submitted metadata review. 2024-04-25 16:31:15 +02:00
florez_j 8303f2948f Submitted metadata review. 2024-04-25 14:39:38 +02:00
florez_j 56ccd681ba Submitted metadata review. 2024-04-25 13:25:14 +02:00
florez_j f7c410a0ac Submitted metadata review. 2024-04-25 13:20:22 +02:00
florez_j 162d0be274 Merge branch 'review_ThorstenBR' 2024-04-25 09:06:03 +02:00
florez_j c2343afe38 Submitted metadata review. 2024-04-25 09:00:26 +02:00
florez_j 3c6440977f Implemented delete attribute feature and corrected yaml display of compound attributes 2024-04-25 08:58:29 +02:00
florez_j 4f15c3b684 Submitted metadata review. 2024-04-24 18:18:54 +02:00
florez_j f55431064e Initialized metadata review. 2024-04-24 18:12:21 +02:00
florez_j 6869d799cd Simplified branch naming 2024-04-24 17:26:08 +02:00
florez_j 4663ed10e2 Fixed branch naming problem 2024-04-24 17:21:44 +02:00
florez_j cf2431d4c3 Included created_at() function 2024-04-24 17:16:59 +02:00
florez_j 508266132e Deleted config_file.py 2024-04-24 17:16:30 +02:00
florez_j be02ad01ed Removed problematic lines, which depended on soon to be removed dependency config_file.py 2024-04-24 17:14:13 +02:00
florez_j 6260e7da3c Redefined name of review branch as review:<review's name> 2024-04-24 17:09:38 +02:00
florez_j c64cad6779 Removed this workflow because it is redundant. Replaced this with active creation of review branch in gitlab 2024-04-24 17:05:25 +02:00
florez_j d26c6d3b5a Cleared out jupyter notebooks 2024-04-24 17:03:37 +02:00
florez_j a9146e5afc Implemented delete attribute feature for review and simplified code. 2024-04-24 17:02:18 +02:00
florez_j ec5703bcaa Editted documentation 2024-04-23 14:19:42 +02:00
florez_j e959538d99 Performed minor edits 2024-04-23 14:07:16 +02:00
florez_j 2845b4c8ab Performed minor edits 2024-04-23 13:53:17 +02:00
florez_j ceb8a34ee0 Commented out no needed python import statements 2024-04-23 13:23:13 +02:00
florez_j 129443d6d9 Updated python libraries and intallation instructions. 2024-04-23 13:20:42 +02:00
florez_j 8876d5af4f Example data integretion configuration files 2024-04-23 12:03:24 +02:00