17 Commits

Author SHA1 Message Date
e851131269 Append new functions to utils/g5505_utils.py. This search for .env file in root directory 2025-06-22 12:13:14 +02:00
8e6ee49188 Modify utils/g5505_utils.py. Implement handling unicode character errors. 2025-06-19 20:49:14 +02:00
f555f7f199 Implement skipping in convert_attrdict_to_np_structured_array(attr_value: dict) when dictionary values are not scalar. This ensures compatible values are transfered while the rest simply dicarded. 2025-06-10 16:03:01 +02:00
7d710c1e62 Fix bug while reading yaml file from utils/g5505_utils.py 2025-06-10 14:38:29 +02:00
ab897018d9 Add exclude paths set through yaml file 2025-06-10 11:08:14 +02:00
68a9928c39 Enable boolean type columns from pandas DataFrame to be suitably converted into numpy structured array 2025-02-10 15:52:17 +01:00
b58e205f9f Remove skip directory condition when directory keywords are empty. Here, all paths to files should be considered. 2025-02-07 16:37:01 +01:00
2f72177410 Add constraint to match only path/to/keyword1/keyword2/files containing a composite keyword keyword1/keyword2. 2025-02-06 15:34:38 +01:00
32bba4239a Synch with remote repo 2025-02-03 10:31:48 +01:00
fd92bce802 Implemented sanitize dataframe function to deal with 'O' which may have numbers or strings detected as string types. Then we use it prior to convert dataframe into structured numpy array. 2024-11-23 16:28:49 +01:00
d13e10e44f Modified logger setup to create monthly logs 2024-11-21 11:46:11 +01:00
3f7a089a28 Fixed bug: to_serializable_dtype() did not identify correctly dtype of array's entries with object dtype 2024-10-28 18:49:22 +01:00
1954542031 Fixed bug introduce in logger due to invalid date naming replace : with - 2024-10-10 14:29:36 +02:00
31c9db98ca Changed datetime format output of created_at() function as '%Y-%m-%d %H:%M:%S.%f' 2024-10-09 16:07:40 +02:00
a92660049f Moved is_structured_array() and to_serializable_dtype() to utils, ranamed a few functions and propagated changes to dependent modules. 2024-09-26 14:03:11 +02:00
9c641c0dae Restructured a bit to include the default case of copying an imnput directory without any constraints. Also, added dry_run input argument that returns a path to files dict representation of output directory without making an actual copy. Useful when input directory is already safe to work with directly 2024-09-16 15:38:30 +02:00
1112a214e9 Moved src/g5505_utils.py to utils/g5505_utils.py 2024-08-23 07:27:39 +02:00