Commit Graph

409 Commits

Author SHA1 Message Date
florez_j 600899dca2 Update .gitignore with output_files/ 2024-12-04 16:53:57 +01:00
florez_j 4a91785efb Add .gitkeep and keep this folder empty. it is only to be used for local processing 2024-12-04 16:52:50 +01:00
florez_j 49dff5b87b Update readme with getting started section 2024-12-04 16:24:14 +01:00
florez_j 3e37854445 Solved binary incompatibility issue of generated environment by conda installing h5py and numpy from conda-forge or default channels. 2024-12-04 16:15:42 +01:00
florez_j 112b88e31f Updated bash script and yml env file to set up python interpreter. 2024-12-04 13:52:35 +01:00
florez_j d422067223 Update to readme.md 2024-12-03 13:55:45 +01:00
florez_j 47c9bd8e3d Update readme with key features of the repo. 2024-12-03 13:50:53 +01:00
florez_j 31d8af6aef Updated README.md with software arquitecture figure 2024-12-02 17:28:22 +01:00
florez_j b2455b2456 Updated README.md with software arquitecture figure 2024-12-02 17:24:48 +01:00
florez_j 6bc89ebff0 Updated README 2024-12-02 17:22:52 +01:00
florez_j 1a941bbb6e Updated figure name. 2024-12-02 17:08:36 +01:00
florez_j c57bb62ebd Updated ci runner pipeline fot gitlab page 2024-12-02 16:31:49 +01:00
florez_j f3bb82a937 Updated documentation and built doc website 2024-12-02 16:31:03 +01:00
florez_j 23e0ced190 Relocated to visualization module 2024-12-02 15:39:41 +01:00
florez_j fc561a6068 Add __init__.py 2024-12-02 15:36:03 +01:00
florez_j c373c18062 Moved hdf5_lib.py to visualization folder 2024-12-02 15:34:44 +01:00
florez_j f2df2ced66 Removed no longer useful notebook 2024-12-02 15:32:57 +01:00
florez_j 4797c8e894 Added env file specification and bash script for env setup 2024-12-02 15:10:21 +01:00
florez_j 2e52109bee removed review folder. This is now supposed to be create for review of experimental campaign data objects metadata. 2024-12-02 14:32:34 +01:00
florez_j dccf64ef30 Configure GitLab Pages 2024-11-26 13:43:09 +01:00
florez_j f4f27b7084 Add draft of dima documentation 2024-11-26 13:40:43 +01:00
florez_j 11ca454b94 Removed bacause some of the functionalities have been outsourced to other modules src/hdf5_ops.py and src/hdf5_writer.py 2024-11-26 11:55:06 +01:00
florez_j 22298f643a Merge branch 'main' of https://gitlab.psi.ch/5505/dima 2024-11-25 15:50:25 +01:00
florez_j 31a129694c Updated instrument dictionaries with variable descriptor names aligned with CF metadata conventions. 2024-11-25 15:49:49 +01:00
florez_j 5c61e2391a Update to DIMA package path resolution from file. 2024-11-24 19:45:18 +01:00
florez_j 1b2b319295 Attempt to dynamically resolve path to dima package, when excecuted from command line. 2024-11-24 17:37:38 +01:00
florez_j 1174ffc8b8 Commented out metadata info about group members for a given group. This is to simplify yaml or json representation of the metadata. 2024-11-24 15:57:54 +01:00
florez_j d7f6e52068 Reimplemented code snippet using hdf5_Writer module 2024-11-24 11:32:26 +01:00
florez_j 967be876d1 Moved func create_hdf5_file_from_dataframe() from hdf5_lib_part2 into hdf5_write.py 2024-11-24 11:30:08 +01:00
florez_j 0330773f08 Moved read_mtable_as_dataframe(filename) to src/hdf5_ops.py 2024-11-24 11:03:44 +01:00
florez_j 3122c4482f Removed logs/ folder. This is usually created locally to trace certain file copying, transfer and conversion processes. 2024-11-24 10:51:05 +01:00
florez_j c257ab6072 Rerun jupyternotebooks to check their functionality after relocating them to notebooks. OpenBis related python scripts still need to be tested. 2024-11-24 10:45:40 +01:00
florez_j 02ded9c11a Add hidden.py to the list. This file may contain sensitive information is only to be accessed locally or from a secure location. 2024-11-24 10:41:32 +01:00
florez_j b24d33ab15 Check whether h5 file being written exists. If so, we do not overwrite it because it may be underdoing refinement, changes or updates, for archiving, sharing, or publishing. 2024-11-24 10:38:13 +01:00
florez_j ca314f971d Add utility functions add_project_path_to_sys_path() to set up path to DIMA's modules dynamically. 2024-11-24 10:08:19 +01:00
florez_j a928c4ef4c Moved demos .py to notebooks. Note: Maybe turn them to jupyternotebooks for consistency 2024-11-24 07:49:50 +01:00
florez_j 6701bc06ad Added read_mtable_as_dataframe(filename) back so that jupyter notebook can use it to demonstrate some functionality 2024-11-23 16:31:29 +01:00
florez_j fd92bce802 Implemented sanitize dataframe function to deal with 'O' which may have numbers or strings detected as string types. Then we use it prior to convert dataframe into structured numpy array. 2024-11-23 16:28:49 +01:00
florez_j 8ab2cb3bdb Moved to notebooks/ 2024-11-23 12:29:55 +01:00
florez_j 33ad4b8509 Moved to notebooks/ 2024-11-23 12:16:13 +01:00
florez_j c30bdab41a Moved to notebooks/ to improve repo organization 2024-11-23 12:06:27 +01:00
florez_j 3535fd0cc2 Moved data integration ipynb to notebooks folder to improve readability 2024-11-23 11:24:28 +01:00
florez_j e486b4659c Added .pkl extension in the list of admissible file extensions 2024-11-21 11:47:41 +01:00
florez_j d13e10e44f Modified logger setup to create monthly logs 2024-11-21 11:46:11 +01:00
florez_j 4632554af1 Added a logs/ and envs/ folder to gitignore. 2024-11-21 11:44:38 +01:00
florez_j 1be4b8493a Improved progress description stdout 2024-11-10 18:21:00 +01:00
florez_j ca2c98eebc Fixed command line interface bug 2024-11-10 18:19:59 +01:00
florez_j 8d17bf267c Major code refactoring and simplifications to enhance modularity. Included a command line interface. 2024-11-01 09:52:41 +01:00
florez_j 510683a50d Renamed the input argument yaml_review_file as review_yaml_file. 2024-11-01 09:51:12 +01:00
florez_j e2fec03d4a Included cli commands update and serialize to simplify running metadata revision pipeline. 2024-10-29 07:56:43 +01:00