|
|
d79877cc9b
|
Moved hdf5_lib.py to visualization folder
|
2024-12-02 15:34:44 +01:00 |
|
|
|
fa9edcb115
|
Removed no longer useful notebook
|
2024-12-02 15:32:57 +01:00 |
|
|
|
1bc145530c
|
Added env file specification and bash script for env setup
|
2024-12-02 15:10:21 +01:00 |
|
|
|
2eec2155b5
|
removed review folder. This is now supposed to be create for review of experimental campaign data objects metadata.
|
2024-12-02 14:32:34 +01:00 |
|
|
|
b6a41c3378
|
Configure GitLab Pages
|
2024-11-26 13:43:09 +01:00 |
|
|
|
13f070e4c9
|
Add draft of dima documentation
|
2024-11-26 13:40:43 +01:00 |
|
|
|
22e201d063
|
Removed bacause some of the functionalities have been outsourced to other modules src/hdf5_ops.py and src/hdf5_writer.py
|
2024-11-26 11:55:06 +01:00 |
|
|
|
80a7b650af
|
Merge branch 'main' of https://gitlab.psi.ch/5505/dima
|
2024-11-25 15:50:25 +01:00 |
|
|
|
f483e8d7fe
|
Updated instrument dictionaries with variable descriptor names aligned with CF metadata conventions.
|
2024-11-25 15:49:49 +01:00 |
|
|
|
fed9aa0a1b
|
Update to DIMA package path resolution from file.
|
2024-11-24 19:45:18 +01:00 |
|
|
|
85d44d71d6
|
Attempt to dynamically resolve path to dima package, when excecuted from command line.
|
2024-11-24 17:37:38 +01:00 |
|
|
|
6dc89d34fe
|
Commented out metadata info about group members for a given group. This is to simplify yaml or json representation of the metadata.
|
2024-11-24 15:57:54 +01:00 |
|
|
|
60f9278f10
|
Reimplemented code snippet using hdf5_Writer module
|
2024-11-24 11:32:26 +01:00 |
|
|
|
3db2eeb041
|
Moved func create_hdf5_file_from_dataframe() from hdf5_lib_part2 into hdf5_write.py
|
2024-11-24 11:30:08 +01:00 |
|
|
|
2740330709
|
Moved read_mtable_as_dataframe(filename) to src/hdf5_ops.py
|
2024-11-24 11:03:44 +01:00 |
|
|
|
ff31c14f92
|
Removed logs/ folder. This is usually created locally to trace certain file copying, transfer and conversion processes.
|
2024-11-24 10:51:05 +01:00 |
|
|
|
01767340f9
|
Rerun jupyternotebooks to check their functionality after relocating them to notebooks. OpenBis related python scripts still need to be tested.
|
2024-11-24 10:45:40 +01:00 |
|
|
|
7708153a62
|
Add hidden.py to the list. This file may contain sensitive information is only to be accessed locally or from a secure location.
|
2024-11-24 10:41:32 +01:00 |
|
|
|
53fae27472
|
Check whether h5 file being written exists. If so, we do not overwrite it because it may be underdoing refinement, changes or updates, for archiving, sharing, or publishing.
|
2024-11-24 10:38:13 +01:00 |
|
|
|
d10786a534
|
Add utility functions add_project_path_to_sys_path() to set up path to DIMA's modules dynamically.
|
2024-11-24 10:08:19 +01:00 |
|
|
|
058b82ff88
|
Moved demos .py to notebooks. Note: Maybe turn them to jupyternotebooks for consistency
|
2024-11-24 07:49:50 +01:00 |
|
|
|
2988d816af
|
Added read_mtable_as_dataframe(filename) back so that jupyter notebook can use it to demonstrate some functionality
|
2024-11-23 16:31:29 +01:00 |
|
|
|
ac80b9a3ca
|
Implemented sanitize dataframe function to deal with 'O' which may have numbers or strings detected as string types. Then we use it prior to convert dataframe into structured numpy array.
|
2024-11-23 16:28:49 +01:00 |
|
|
|
ca23e363c9
|
Moved to notebooks/
|
2024-11-23 12:29:55 +01:00 |
|
|
|
feee8762ed
|
Moved to notebooks/
|
2024-11-23 12:16:13 +01:00 |
|
|
|
cc7ad749ca
|
Moved to notebooks/ to improve repo organization
|
2024-11-23 12:06:27 +01:00 |
|
|
|
13cb5476b1
|
Moved data integration ipynb to notebooks folder to improve readability
|
2024-11-23 11:24:28 +01:00 |
|
|
|
05f16bf717
|
Added .pkl extension in the list of admissible file extensions
|
2024-11-21 11:47:41 +01:00 |
|
|
|
59e910a0f9
|
Modified logger setup to create monthly logs
|
2024-11-21 11:46:11 +01:00 |
|
|
|
90ae93c124
|
Added a logs/ and envs/ folder to gitignore.
|
2024-11-21 11:44:38 +01:00 |
|
|
|
870d60a789
|
Improved progress description stdout
|
2024-11-10 18:21:00 +01:00 |
|
|
|
c866ce8ee9
|
Fixed command line interface bug
|
2024-11-10 18:19:59 +01:00 |
|
|
|
83b4a12e8b
|
Major code refactoring and simplifications to enhance modularity. Included a command line interface.
|
2024-11-01 09:52:41 +01:00 |
|
|
|
2ae8deaf64
|
Renamed the input argument yaml_review_file as review_yaml_file.
|
2024-11-01 09:51:12 +01:00 |
|
|
|
29c6d86583
|
Included cli commands update and serialize to simplify running metadata revision pipeline.
|
2024-10-29 07:56:43 +01:00 |
|
|
|
71b7091fe7
|
Fixed bug: to_serializable_dtype() did not identify correctly dtype of array's entries with object dtype
|
2024-10-28 18:49:22 +01:00 |
|
|
|
e01147d03a
|
Removed unused import statements
|
2024-10-28 16:37:32 +01:00 |
|
|
|
c0ec7e9b26
|
Moved git related operations from pipelines/ to src/git_ops.py
|
2024-10-28 16:30:34 +01:00 |
|
|
|
66e236bd5d
|
Added function to validate review yaml file, and updated update_hdf5_with_review function
|
2024-10-28 16:20:28 +01:00 |
|
|
|
a8ceffba8b
|
Corrected import statements due to dependency name changes
|
2024-10-17 16:52:42 +02:00 |
|
|
|
8129949db9
|
Renamed module: src/hdf5_lib.py -> src/hdf5_writer.py
|
2024-10-17 10:53:51 +02:00 |
|
|
|
d49e7ae9d5
|
Replaced read_dataset_from_hdf5file(hdf5_file_path, dataset_path) with HDF5DataOpsManager.extract_dataset_as_dataframe(self,dataset_name)
|
2024-10-17 10:46:19 +02:00 |
|
|
|
13d65a7383
|
Fixed bug when file reader not available. File reader registry now returns a reade that maps input to None.
|
2024-10-14 16:03:03 +02:00 |
|
|
|
9a76b3a01a
|
Added 'filename_format' attribute to YAML schema. It takes as value a string of comma separated keys from available attributes in YAML file.
|
2024-10-14 16:01:24 +02:00 |
|
|
|
5cc409f882
|
Fixed bug introduce in logger due to invalid date naming replace : with -
|
2024-10-10 14:29:36 +02:00 |
|
|
|
51e8e6ae66
|
Cleaned up import statements and comment out path append operations
|
2024-10-10 14:27:50 +02:00 |
|
|
|
5b338c8212
|
Attempt to initialize dima/utils as a module
|
2024-10-10 11:53:27 +02:00 |
|
|
|
507454ee91
|
Attemp to initialize dima as a module
|
2024-10-10 11:43:02 +02:00 |
|
|
|
1e5581769b
|
Robustified metadata and dataset extraction methods by requiring explicit load of file obj before their use. Renamed a few functions and fixed types in print statements.
|
2024-10-10 11:28:23 +02:00 |
|
|
|
81e3cc58ca
|
Updated function dependencies to reflect changes made to hdf5_ops.py
|
2024-10-10 11:02:05 +02:00 |
|
