dima/workflow_data_integration.ipynb

Import libraries and modules

In [ ]:

import sys
import os
# Set up project root directory
root_dir = os.path.abspath(os.curdir)
sys.path.append(root_dir)

import src.hdf5_vis as hdf5_vis
import src.data_integration_lib as dilib

Specify data integration task through yaml configuration file

• Create your configuration file (i.e., *.yaml file) adhering to the example yaml file in the input folder.
• Set up input directory and output directory paths and Excecute Cell.

In [ ]:

#output_filename_path = 'output_files/unified_file_smog_chamber_2024-04-07_UTC-OFST_+0200_NG.h5'
yaml_config_file_path = 'input_files/data_integr_config_file_LI.yaml'
output_filename_path = dilib.integrate_data_sources(yaml_config_file_path)

Display integrated HDF5 file using a treemap

• Excecute Cell. A visual representation in html format of the integrated file should be displayed and stored in the output directory folder

In [ ]:

hdf5_vis.display_group_hierarchy_on_a_treemap(output_filename_path)