initial commit

This commit is contained in:
bartels-rausch committed 2025-11-21 20:03:47 +01:00
1 parent efff7c29f5
commit 5a65fa13c4
79 files changed
+6122 -450

No files matched your search

+809 -450
View File
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,33 @@
# CHANGELOG
### v1.1.7
- Removed time-consuming `mustBeHdfPath` validation
### v1.1.6
- Added `h5tools.files.copyFile`, a wrapper for `copyfile` tailored to HDF5 files
### v1.1.5
- Added `h5tools.move` wrapper for `H5L.move`
### v1.1.4
- Fixed bugs in `h5tools.deleteObject`
### v1.1.3
- Appending MATLAB class even for numeric datatypes supported by MATLAB's built-in ``h5read``
### v1.1.2
- Conversion of table properties (row names, units) to string to avoid cellstr attribute warning in ``h5tools.datasets.writeCompoundDataset``
### v1.1.1
- Support for optional arguments to ```h5create``` for numeric datasets
- Improved datetime strategy for more precise seconds.
- Working on support for datetime in compound datasets. Currently datetime is handled as a string and cannot convert back to datetime when reading the dataset.
- Removed some unnecessary argument validation
- Added check to ensure file name provided to ```h5tools.createFile``` ends with ".h5"
### v1.0.1
- Removed use of function from a different repo in ```h5tools.deleteObject```
- Added link to documentation in the README
### v1.0.0
- Initial release
@@ -0,0 +1,9 @@
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: Patterson
given-names: Sara S.
orcid: https://orcid.org/0000-0003-4574-3935
title: "h5tools-matlab"
version: 1.0.1
date-released: 2022-12-12
@@ -0,0 +1,9 @@
# CONTRIBUTING
The HDF5 low-level library is tricky and I'm happy for contributions that add new capabilities to h5tools-matlab or significantly improve existing ones. In the initial release, I added a few specialized MATLAB datatypes that I frequently use, but there are many more that others would find useful and contributions adding those are also welcome. For any added capabilities, also add some tests to the test suite (use code coverage reports to make sure everything is being tested) and make sure the appropriate documentation is included with the code (see existing files for formatting). Then submit a pull request for me to review.
A few areas where h5tools-matlab could improve is the addition of object and region references, solving the issue with row vectors converting to column for numeric attributes and better support for datetime that includes nonscalar inputs.
If you find an error and can solve it, make a pull request that includes information about the error. Otherwise, add an issue with as much information as possible about the error (code to replicate it, error message content, etc). Same goes for help requests (use "troubleshooting" tag) that aren't resolved by checking through the information and examples in the documentation. If you'd like to request a feature, create an Issue using with the tag "enhancement". Pull requests solving errors or addressing Issues are welcome too.
I'll try to address issues and pull requests as soon as I can and I'll make sure contributors are acknowledged.
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2022 Sara Patterson
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
@@ -0,0 +1,15 @@
# h5tools-matlab
MATLAB has some solid high-level HDF5 support, but it's limited in scope. Fortunately, MATLAB also provides access to the low-level HDF5 C API. Unfortunately, learning how to use it is challenging, time-consuming and somewhat painful, particularly for the average MATLAB user that may not have experience with languages like C. The goal of h5tools-matlab is to address this gap, expanding the built-in MATLAB support with additional high-level functions that simplify MATLAB's existing interface and provide access to features currently only available through the low-level library (e.g. searching files, creating groups, reading/writing object references, writing text and compound datasets). Users need only understand the components and basic organization of the HDF5 format (i.e. groups, datasets, attributes, object references and paths within an HDF5 file).
h5tools-matlab provides a toolbox of high-level functions to simplify working with HDF5 files by wrapping calls to the low-level library (see support functions). Key high level functions included that are absent from MATLAB's high-level library include support for creating **groups** and **soft links** ("shortcuts") and for **indexing** the contents of HDF5 files (e.g. returning all attributes of an object or a list of all groups, datasets or softlinks within a file). There are also convenience functions for working with HDF5 paths and several validator functions for argument blocks.
**Datasets:** MATLAB's built-in dataset support works for numeric data, especially if you understand HDF5 dataspaces. As such, the dataset support in h5tools-matlab is designed for the situations in which MATLAB's high-level functions are not sufficient: reading and writing non-numeric MATLAB data types and supporting users who do not want to deal with ```h5create``` by automating dataset creation. In addition to the MATLAB-supported **numeric** types, h5tools-matlab supports: **char**, **string**, **logical**, **cellstr**, **table**, **timetable**, **duration**, **enum**, **containers.Map,** and **struct**. There is also support for some of the datatypes in MATLAB's toolboxes like **imref2d, affine2d, simtform2d**. Their implementation demonstrates strategies that could be adapted to add more types.
**Attributes:** The added capabilities for attributes include high-level functions that allow working with multiple attributes at once and reading all attributes at once (without knowing their names). Supported datatypes include **numeric**, **char**, **string**, **enum**, **logical** and **datetime**. For the rest, the goal was to get your info into the HDF5 file if at all possible rather than throwing fatal errors. Accordingly, additional datatypes that will write, but not be read back in identically include **duration** and **cellstr**.
The [Examples](docs/Examples.md) page contains a quick-start guide on how to uses the main h5tools functions. The [documentation](docs/h5tools_documentation.pdf) details how each datatype is written to the HDF5 files and any limitations. This is worth checking out because the techniques h5tools uses to write a few tricky datatypes are more like workarounds than true solutions. All functions are also documented within the code (accessible with ```help``` or ```doc```). h5tools-matlab is supported by an extensive test suite and has 90.41% statement coverage. See [CONTRIBUTING.md](CONTRIBUTING.md) for information on adding/requesting new features.
If you use this package, please cite the Github repository, for now. This package was developed to support a data management system for adaptive optics imaging of the eye ([AOData](https://github.com/sarastokes/AOData)). A paper is soon to be submitted and a citation will be added here once the paper is up on a preprint server.
[![View h5tools-matlab on File Exchange](https://www.mathworks.com/matlabcentral/images/matlab-file-exchange.svg)](https://www.mathworks.com/matlabcentral/fileexchange/122032-h5tools-matlab)
@@ -0,0 +1,129 @@
## Main Functions
### Writing dataset(s)
```h5tools.write``` is the equivalent of MATLAB's ```h5create``` and ```h5write```. There are a few key differences. First, you do not need to create the dataset prior to writing the data. Second, more data types are supported (see **Limitations** for the full details). Third, you can write multiple datasets at once.
The first two arguments are always the HDF5 file name and the path of the group within the HDF5 file where the datasets will be written. The datasets created can be specified in the following ways:
```matlab
% Write a single dataset:
h5tools.write('Test.h5', '/', 'Dataset1', 1:3);
% Write multiple datasets
h5tools.write('Test.h5', '/', 'Dataset1', 1:3, 'Dataset2', "hello world");
```
The upside of handling dataset creating behind the scenes is that it is tricky (especially for non-numeric types) and the average user may have no reason to delve into the details of HDF5 dataset creation. The downside is that you lose the control of specifying your dataspace and dataset (particularly access to filtering, compression, etc.). If you do have special requirements for the created dataset, try MATLAB's built-in functions or the low-level library.
How does h5tools-matlab implement the previously unsupported datatypes and ensure they are read back into MATLAB from HDF5 identically? The full implementation is described in the "docs" folder. First, once you understand HDF5 dataspaces (```H5S```), you'll find more types are supported than you initially thought, which is why dataset creation is bundled in with writing. Second, truly unsupported types are made possible by writing attributes to the dataset that identify the intended MATLAB data type and any additional information needed to recreate the intended data type when reading the dataset into MATLAB.
> Disclaimer: Using attributes to support new datatypes is a workaround rather than a true solution. That being said... it does work (with some limitations described in the documentation).
### Reading dataset(s)
```h5tools.read``` is used to read one or more HDF5 datasets. The datasets do not need to be located within the same group, as long as their names are specified relative to the 2nd input (```pathName```).
```matlab
% Syntax:
% h5tools.writeatt(fileName, pathName, varargin)
% Read the dataset named "DatasetOne" within "/GroupOne"
out = h5tools.read('Test.h5', '/GroupOne', 'DatasetOne');
```
### Writing attributes
```h5tools.writeatt``` is the equivalent of MATLAB's ```h5writeatt```. A key addition is the ability to work with multiple attributes as once, as demonstrated below. There's also a bit more support for MATLAB datatypes. There is also support for ensuring your info makes it into the HDF5 file even if it's not going to read back in identically (e.g ```datetime``` gets converted to ```char``` and will be read back in as ```char```). See **Limitations** for more information on what can and cannot be written as an attribute.
```matlab
% Syntax:
% h5tools.writeatt(fileName, pathName, varargin)
% Write a single attribute:
h5tools.writeatt('Test.h5', '/', 'A', 1);
% Write multiple attributes:
h5tools.writeatt('Test.h5', '/', 'A', 1, 'B', 2);
% Write all the fields of a struct as attributes
attStruct = struct('A', 1, 'B', 2);
h5tools.writeatt('Test.h5', '/', attStruct);
% Write all the key/value pairs in a containers.Map:
attMap = containers.Map('A', 1, 'B', 2);
h5tools.writeatt('Test.h5', '/', attMap);
% Write a struct or containers.Map followed by additional attributes:
h5tools.writeatt('Test.h5', '/', attMap, 'C', 3);
h5tools.writeatt('Test.h5', '/', attStruct, 'C', 3);
```
### Reading attributes
Attributes are read with ```h5tools.readatt```. You can specify one or more attributes to return, or ask for all the attributes. The output will change depending on the input you provide, as demonstrated below and detailed further in the documentation for ```h5tools.readatt```.
```matlab
% 1. Read a single attribute, return the value
out = readatt('File.h5', '/GroupOne', 'Attr1')
% 2. Read two attributes, return the values as two outputs
[out1, out2] = readatt('File.h5', '/GroupOne', 'Attr1', 'Attr2')
% 3. Read two attributes, return the values in a containers.Map:
out = readatt('File.h5', '/GroupOne', 'Attr1', 'Attr2')
% 4. Read all attributes, return the values in a containers.Map
out = readatt('File.h5', '/GroupOne', 'all')
```
### Creating groups
```h5tools.createGroup``` enables creation of one or more new HDF5 groups, relative to a single location within the HDF5 file.
```matlab
% Create one group within the root group "/"
h5.createGroup('Test.h5', '/', 'GroupOne);
% Create three groups within "/GroupOne"
h5.createGroup('Test.h5', '/GroupOne', 'GroupOne1A', 'Group1B', 'Group1C');
% Create two groups, one in the root group "/" and one in "/GroupOne"
h5.createGroup('Test.h5', '/', 'GroupTwo', '/GroupOne/Group1D');
```
### Creating links
```h5tools.writelink``` creates a soft link to another object (a group or a dataset) within the HDF5 file. The syntax mirrors that of ```h5tools.write``` and ```h5tools.writeatt```, only the HDF5 path of the object to reference is provided as the 4th argument instead of data.
```matlab
% Write a softlink named "LinkName" in the root group "/" that references "/GroupOne"
h5tools.writelink('Test.h5', '/', 'LinkName', '/GroupOne');
```
### Reading links
```h5tools.readlink``` reads a soft link. The path within the HDF5 file of the linked group/dataset is returned and, optionally, the H5ML.id of the linked group/dataset.
```matlab
% Read the softlink created above
[hdfPath, ID] = h5tools.readlink('Test.h5', '/', 'LinkName');
```
### Creating files
```h5tools.createFile``` creates a new HDF5 file. There's an optional 2nd argument to automatically overwrite any existing file by that name. You can optionally specify an output to return the``` H5ML.id``` for the file if you need to use the low-level library.
```matlab
% Create a new file. If it already exists, return an error
h5tools.createFile('Test.h5');
% Create a new file and overwrite if it already exists
h5tools.createFile('Test.h5', true);
% Create a new file, don't overwrite and return the identifier
fileID = h5tools.createFile('Test.h5', false);
% Don't forget to close fileID when you're done using it
H5F.close(fileID);
```
> Warning: some functions have optional outputs for returning the associated ```H5ML.id```, which is used for working with the low-level library. Don't request these outputs unless you intend to use them, as you will be responsible for closing them once you're finished (as shown above). A few of the functions may throw errors if the identifiers involved are open in the base workspace.
## Support functions
See the documentation within the code (use `help` or `doc`) for full information on how to use these. All take the HDF5 file name as the first input. Many will also accept the file's ```H5ML.id``` instead, in case you're using the low-level library.
- ```h5tools.move``` - moves a group/dataset to a new location, useful for renaming (simplified wrapper for `H5L.move`)
- ```dsetNames = h5tools.collectAllDatasets(hdfFile)``` - returns the full paths of all datasets within an HDF5 file
- ```groupNames = h5tools.collectAllGroups(hdfFile)``` - returns the full paths of all groups within an HDF5 file.
- ```linkNames = h5tools.collectAllSoftLinks(hdfFile)``` - returns the full paths of all soft links in the HDF5 file.
- ```tf = h5tools.exist(hdfFile, hdfPath)``` - returns whether or not ```hdfPath``` exists within the HDF5 file
- ```attNames = h5tools.getAttributeNames(hdfFile, hdfPath)``` - returns the attribute names of the group/dataset at ```hdfPath```
- ```tf = h5tools.hasAttribute(hdfFile, hdfPath, attName)``` - returns whether or not the group/dataset at ```hdfPath``` has the attribute ```attName```.
- ```fileID = h5tools.files.openFile(hdfFile, readOnly)``` - if you need to work with the low-level library, this function will return the file identifier. If the 2nd input is true (default), it will be opened as read-only. To modify the file, set to false.
The ```h5tools.util``` package contains some small functions for working with HDF5 paths (```getPathEnd```, ```buildPath```). In addition, ```h5tools.files.HdfTypes``` mimics the enumeration used for object types in the low-level library (but does not require using ```H5ML.get_constant_value()``` to interpret) and can parse ```H5ML.id``` inputs.
@@ -0,0 +1,17 @@
## Function Reference
A list to help keep track of low-level library functions and any h5tools-matlab high-level wrapper functions.
### H5F - File
```H5F.get_name(objID)``` - return the file name (with full path) HDF5 containing objID, which can be any H5ML.id
### H5D - Dataset
```spaceID = H5D.get_space(dsetID)``` - get the dataspace for the given dataset.
```typeID = H5D.get_type(dsetID)``` - get the datatype for the given dataset.
### H5O - Object
```objID = H5O.open(fileID, path, 'H5P_DEFAULT')``` - open a group OR dataset within the HDF5 file.
```info = H5O.get_info(objID)``` - returns struct with information about the object including type (e.g. "GROUP", "DATASET"). ```h5tools.files.HdfTypes.get()``` extracts the type information. ```get_info``` is also useful in ```H5O.visit()``` loops.
```
@@ -0,0 +1,499 @@
\documentclass[11pt]{exam}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
% Layout
\usepackage[margin=0.75in, top=1in]{geometry}
% Fonts
\usepackage{cmbright}
\usepackage[T1]{fontenc}
\usepackage[scaled]{beramono}
%\usepackage[scaled]{helvet}
%\renewcommand\familydefault{\sfdefault}
% Math
\usepackage{amsmath}
\usepackage{amssymb}
% Lists
\usepackage{enumitem}
\setlist{nolistsep,itemsep=0.6ex}
% Links
\usepackage{hyperref}
\hypersetup{hidelinks=true}
% Section formatting
\usepackage[title]{appendix}
\usepackage{titlesec}
\titleformat{\chapter}[display]{\huge\bfseries\filcenter}{\large CHAPTER \thechapter}{1ex}{}[]
\titleformat*{\section}{\LARGE\bfseries}
\titlespacing\section{0pt}{12pt plus 4pt minus 2pt}{0pt}
\titleformat*{\subsection}{\large\bfseries}
\titlespacing\subsection{0pt}{12pt plus 4pt minus 2pt}{0pt}
% Colors
\usepackage[dvipsnames]{xcolor}
\definecolor{codegreen}{rgb}{0, 0.6, 0}
\definecolor{codegray}{rgb}{0.94, 0.94, 0.94}
\definecolor{codepurple}{rgb}{0.58, 0, 0.82}
\definecolor{codeblue}{rgb}{0.1, 0.1, 0.6}
\definecolor{codered}{rgb}{0.3, 0.3, 0.9}
\definecolor{darkgray}{rgb}{0.4,0.4,0.4}
\definecolor{darkteal}{rgb}{0.1,0.6,0.5}
\definecolor{backcolor}{rgb}{0.95,0.96,0.965}
% Code
\usepackage{listings}
\usepackage{matlab-prettifier}
\lstdefinestyle{mystyle}{
backgroundcolor=\color{backcolor},
commentstyle=\color{codegreen},
stringstyle=\color{codepurple},
keywordstyle=\color{codeblue},
numberstyle=\color{codered},
basicstyle=\ttfamily\footnotesize,
showspaces=false,
showstringspaces=false,
tabsize=4
}
\lstset{style=mystyle}
% Custom commands
\newcommand{\comm}[1]{}
\newcommand\myurl[1]{\textcolor{blue}{\underline{#1}}}
\newcommand\myparam[1]{\colorbox{codegray}{\texttt{#1}}}
\newcommand\myfcn[1]{\colorbox{codegray}{\textcolor{codeblue}{\texttt{#1}}}}
\newcommand\matfcn[1]{\textcolor{darkteal}{\texttt{#1}}}
\newcommand\mygray[1]{\textcolor{gray}{(#1)}}
\newcommand\hdftype[1]{\texttt{\myurl{#1}}}
\newcommand\myatt[2]{\textcolor{darkgray}{\texttt{\textit{#1} = #2}}}
\begin{document}
\noindent
\begin{center}
{\LARGE\textbf{h5tools-matlab Documentation}}
\\\vspace{1.5ex}
{\large Sara Patterson - December 12, 2022}
\end{center}
\tableofcontents
\pagebreak
\begin{center}
\textbf{\Huge h5tools-matlab}
\end{center}
\noindent The purpose of this documentation is to detail the implementation of reading and writing data with MATLAB and HDF5.
The main functions are demonstrated here (and more succinctly in "docs/Examples.md"). Additional information on function inputs/outputs can be found in the documentation within within the code using \texttt{help} or \texttt{doc}.
\section{Getting Started}
\noindent Clone h5tools-matlab from Github (\myurl{\url{https://github.com/sarastokes/h5tools-matlab}}) or download from MATLAB's file exchange. Make sure to add the "src" folder and all subfolders to your MATLAB path as follows, replacing the ``..'' with the location where h5tools-matlab is saved.
\begin{lstlisting}[style=matlab-editor, basicstyle=\mlttfamily\footnotesize]
addpath(genpath('../h5tools-matlab/src/'));
\end{lstlisting}
% ██████╗ █████╗ ████████╗ █████╗ ███████╗███████╗████████╗███████╗
% ██╔══██╗██╔══██╗╚══██╔══╝██╔══██╗██╔════╝██╔════╝╚══██╔══╝██╔════╝
% ██║ ██║███████║ ██║ ███████║███████╗█████╗ ██║ ███████╗
% ██║ ██║██╔══██║ ██║ ██╔══██║╚════██║██╔══╝ ██║ ╚════██║
% ██████╔╝██║ ██║ ██║ ██║ ██║███████║███████╗ ██║ ███████║
% ╚═════╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝╚══════╝╚══════╝ ╚═╝ ╚══════╝
\section{Datasets}\label{sec:Datasets}
\noindent The high-level functions for datasets are \myfcn{h5tools.write} and \myfcn{h5tools.read}. The implementation for these functions can be found in the "h5tools.datasets" package. The key difference between these functions and their corresponding MATLAB versions (\matfcn{h5read} and \matfcn{h5write}) are:
\begin{enumerate}
\item With MATLAB's library, you need to first create the dataset with \myfcn{h5create} (which requires some understanding of HDF5 dataspaces), then write it with \matfcn{h5write}. \myfcn{h5tools.write} takes care of dataspaces and dataset creation behind the scenes (optional arguments to \myfcn{h5tools.write} will be passed to \matfcn{h5create}).
\item Support for more MATLAB data types. Text (\texttt{char} and \texttt{string}), enumerated types, and compound datatypes (\texttt{table}, \texttt{timetable} and \texttt{struct}/\texttt{containers.Map} where each field has the same number of elements) are directly supported, in addition to the numeric types supported by MATLAB's high-level functions. A few tricks are used to support additional types, by converting to an HDF5-compatible format and adding attributes to the dataset that guide h5tools in reading the data back into MATLAB (\texttt{logical}, \texttt{cellstr}, \texttt{struct}/\texttt{containers.Map} with unequal elements in each field, \texttt{datetime}, \texttt{duration}).
\end{enumerate}
$\quad$\\
The syntax is identical for each datatype:
\begin{lstlisting}[style=matlab-editor, basicstyle=\mlttfamily\footnotesize]
h5tools.write(hdfName, pathName, dsetName, data);
out = h5tools.read(hdfName, pathName, dsetName);
\end{lstlisting}
The \texttt{hdfName} is the HDF5 file name, \texttt{pathName} is the path within the HDF5 file where the dataset will be written, \texttt{dsetName} is the dataset name and \texttt{data} is the data to be written.
\begin{lstlisting}[style=matlab-editor, basicstyle=\mlttfamily\footnotesize]
% Write a dataset named 'MyDatatset' to' GroupOne'
h5tools.createFile('Test.h5');
h5tools.write('Test.h5', '/GroupOne', 'MyDatatset', magic(5));
out = h5tools.read('Test.h5', '/GroupOne', 'MyDataset');
% Write a dataset with optional arguments to h5create
h5tools.write('Test.h5', '/GroupOne', 'MyDataset2', magic(5), 'MaxSize', [Inf 5]);
\end{lstlisting}
$\quad$\\
\noindent Be sure to check the implementation for the data type you intend to write. In describing each datatype's implementation, the following information is provided where relevant:
\begin{itemize}
\item \underline{Preprocessing:} Any changes to the data's MATLAB type that occur before writing to make the data compatible with an HDF5 datatype.
\item \underline{Persistence:} How the data is actually written to the HDF5 dataset.
\item \underline{Postprocessing:} Any processing of the data that occurs after reading into MATLAB to make the data compatible a specific MATLAB datatype.
\item \underline{Implementation:} The functions used to implement writing and reading of the dataset. All data passes through \myfcn{h5tools.write} for writing and \myfcn{h5tools.read} for reading. Any additional functions implementing specialized processing will be detailed here.
\item \underline{Limitations:} Any caveats or limitations.
\end{itemize}
\subsection{Numeric}
\noindent MATLAB supports direct conversion of numeric datatypes to and from HDF5 files with \matfcn{h5read} and \matfcn{h5write}. The advantage of h5tools is a more user-friendly interface that automates dataset creation (\matfcn{h5create}).
\begin{itemize}
\item \underline{Preprocessing:} As with all other datatypes, the dataset creation and dataspace creation is handled rather than requiring the use of \matfcn{h5create}. Writing is performed with \matfcn{h5read}.
\item \underline{Persistence:} See the table below for the correspondence between MATLAB numeric types and HDF5 data types.\\
\begin{center}\begin{tabular}{| c | c | c |}
\hline
\textbf{MATLAB type} & \textbf{HDF5 Class} & \textbf{HDF5 Type}\\
\hline
\texttt{double} & \texttt{H5T\_FLOAT} & \texttt{H5T\_IEEE\_F64LE}\\
\texttt{single} & \texttt{H5T\_FLOAT} & \texttt{H5T\_IEEE\_F32LE}\\
\texttt{uint8} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_U8LE}\\
\texttt{int8} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_I8LE}\\
\texttt{uint16} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_U16LE}\\
\texttt{int16} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_I16LE}\\
\texttt{uint32} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_U32LE}\\
\texttt{int32} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_I32LE}\\
\texttt{uint64} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_U64LE}\\
\texttt{int64} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_I64LE}\\
\hline
\end{tabular}\end{center}
\item \underline{Postprocessing:} N/A
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeMatrixDataset}, \matfcn{h5read}
\item \underline{Limitations:} There are also several features that MATLAB's built-in functions already handle well and are not currently supported in h5tools, such as reading subsets of numeric datasets, using filters and working with remote locations.
\end{itemize}
\subsection{Char}
\noindent\begin{itemize}
\item \underline{Preprocessing:} N/A
\item \underline{Persistence:} Written as class \hdftype{H5T\_STRING} with character type \hdftype{H5T\_C\_S1} and set \hdftype{H5T\_CSET\_ASCII}.
\item \underline{Postprocessing:} N/A
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeCharDataset}, \matfcn{h5read}
\item \underline{Limitations:} N/A
\end{itemize}
\subsection{String}
\noindent\begin{itemize}
\item \underline{Preprocessing:} N/A
\item \underline{Persistence:} written as class \hdftype{H5T\_STRING} with character type \hdftype{H5T\_C\_S1} and set \hdftype{H5T\_CSET\_UTF8}.
\item \underline{Postprocessing:} N/A
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeStringDataset}, \matfcn{h5read}
\item \underline{Limitations:} N/A
\end{itemize}
\subsection{Cellstr}
\noindent While \texttt{cellstr} is not supported, it is comparable to a \texttt{string} array and can be written as such.
\begin{itemize}
\item \underline{Preprocessing:} Conversion to \texttt{string} array.
\item \underline{Persistence:} Identical to \texttt{string}, but with 1 attribute: \myatt{Class}{cellstr}.
\item \underline{Postprocessing:} Conversion from \texttt{string} back to \texttt{cellstr}
\item \underline{Implementation:} See \texttt{string}.
\item \underline{Limitations:} N/A
\end{itemize}
\subsection{Datetime}
\noindent While \texttt{datetime} is not supported (HDF5 has a time type but I have no looked into it yet), the data can be represented as a \texttt{string}, as long as the format is recorded.
\noindent\begin{itemize}
\item \underline{Preprocessing:} The "format" property is extracted, then the data is converted to \texttt{string}.
\item \underline{Persistence:} See \texttt{string}. Two attributes are written to the dataset: \myatt{Class}{datetime} and \myatt{Format}{date format}.
\item \underline{Postprocessing:} Conversion to \texttt{datetime} using the optional Format argument.
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeDateDataset}
\item \underline{Limitations:} Only scalar and vector \texttt{datetime} datasets are supported.
\end{itemize}
\subsection{Enumerated types}
\noindent HDF5 offers enumerated type, which is an integer datatype restricted to a set of named values. For example, an enumerated type may contain three members (e.g. ``Red'', ``Green'', ``Blue'') and their corresponding values (0, 1, 2).
\begin{itemize}
\item \underline{Preprocessing:} The index of each enumerated type value within the full list of enumerated types (obtained with \matfcn{metaclass}) is converted to \texttt{uint32}. The class name (including any namespaces) will be appended to each enumeration name (e.g. ``RED'' will be written as ``MyClassName.Red'').
\item \underline{Persistence:} written as \myurl{\texttt{H5T\_ENUM}} with the values of each member in the enumeration as \hdftype{H5T\_STD\_U32LE}.
\item \underline{Postprocessing:} A string array of the enumeration members is read from the dataset through the datatype library (\texttt{H5T}), ordered by their enumeration value. If the associated class is not found on MATLAB's path, the data will be read back in as a string array using the format \texttt{x.y} where \texttt{x} is the class name in the attributes and \texttt{y} is the enumeration name. If the class is found on MATLAB's path, the string will be evaluated (\texttt{eval}). If the enumerated type was not written with h5tools and has no class name attribute, the string array will just contain the enumeration names. %with the class name in the attributes \texttt{char}.
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeEnumTypeDataset}, \myfcn{h5tools.datasets.readEnumTypeDataset}.
\item \underline{Limitations:}
\end{itemize}
\subsection{Logical}
\noindent HDF5 has no boolean datatype so, following other successful HDF5 libraries like h5py, this is written as an enumerated type.
\begin{itemize}
\item \underline{Preprocessing:} Conversion to \texttt{uint8}.
\item \underline{Persistence:} written as \myurl{\texttt{H5T\_ENUM}} with two members (0=FALSE, 1=TRUE). The enumeration values are written as \hdftype{H5T\_STD\_U8LE}.
\item \underline{Postprocessing:} If an enumerated type dataset is read in and has two enumerated types matching those above, the values it will be treated as \texttt{logical}. The enumerated values will be converted from \texttt{uint8} to \texttt{logical}.
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeLogicalDataset}, \myfcn{h5tools.datasets.readEnumTypeDataset}.
\item \underline{Limitations:} N/A
\end{itemize}
\subsection{Table}
\noindent HDF5 supports writing tables as compound datatypes, for which there is no MATLAB support.
\begin{itemize}
\item \underline{Preprocessing:}
\item \underline{Persistence:} written as \myurl{\texttt{H5T\_COMPOUND}} with two attributes: \myatt{Class}{table}, \myatt{ColumnClasses}{[class1, class2,...]} of data in each column
\item \underline{Postprocessing:}
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeCompoundDataset}, \\\myfcn{h5tools.datasets.readCompoundDataset}
\item \underline{Limitations:} Currently only numeric datatypes, string and char are supported within the table.
\end{itemize}
\subsection{Struct}
\noindent Both \texttt{struct} and \texttt{containers.Map} mimic the natural layout of an HDF5 file (ie.e they are a group containing datasets) so there are naturally difficulties writing them to a single dataset.
\begin{itemize}
\item \underline{Preprocessing:} The number of elements in each field will be calculated to determine the strategy used to write the information to the HDF5 file.
\item \underline{Persistence:} If the number of elements in each field is the same, \hdftype{H5T\_COMPOUND} is used, as described above, but with the identifying attribute \myatt{Class}{struct}. Otherwise, each field will be written as an attribute of a placeholder \texttt{string} dataset containing the original MATLAB class (``struct'').
\item \underline{Postprocessing:} See \texttt{table}. If the number of elements was not equal and the dataset was mapped to attributes instead, the attributes and their values will be assigned to the fields of a \texttt{struct}.
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeStructDataset}, \myfcn{h5tools.datasets.makeMapDataset}
\item \underline{Limitations:} As described above, the number of elements in each field must be the same. Multi-level structs (e.g. a \texttt{struct} where one or more of the fields are \texttt{struct}. The limitations of \texttt{table} also apply.
\end{itemize}
\subsection{Containers.Map}
\noindent \texttt{Containers.Map} is treated similarly to \texttt{struct}.
\noindent\begin{itemize}
\item \underline{Preprocessing:} Conversion to \texttt{struct} with \myfcn{map2struct}.
\item \underline{Persistence:} See \texttt{struct}. Only difference is, if the dataset is mapped to attributes, the placeholder text dataset will contain ``containers.Map''.
\item \underline{Postprocessing:} If necessary, conversion with \myfcn{struct2map()}.
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeStructDataset}, \myfcn{h5tools.datasets.makeMapDataset}
\item \underline{Limitations:} See \texttt{struct} and \texttt{table}.
\end{itemize}
\subsection{Timetable}
\noindent
\begin{itemize}
\item \underline{Preprocessing:} The time column is converted to \texttt{double} with \matfcn{seconds()}.
\item \underline{Persistence:} Written as \hdftype{H5T\_COMPOUND} with two attributes: \myatt{Class}{timetable}, \myatt{ColumnClasses}{[class1, class2,...]} of data in each column. See \textbf{Table 1} and \texttt{string} and \texttt{char} for information on how individual columns are written.
\item \underline{Postprocessing:} The first column is converted back to \texttt{duration} with \matfcn{seconds}.
\item \underline{Implementation:} See \texttt{table}.%, \myfcn{h5tools.datasets.write}.
\item \underline{Limitations:} Same as \texttt{table}. In addition, a better way might exist for converting the \texttt{duration} column than always using \matfcn{seconds}.
\end{itemize}
\subsection{Duration}
\noindent
\begin{itemize}
\item \underline{Preprocessing:} All durations are converted to seconds, then to \texttt{double}
\item \underline{Persistence:} Written as \hdftype{H5T\_IEEE\_F64LE} with one attribute: \myatt{Class}{duration}.
\item \underline{Postprocessing:} Converted back to \texttt{duration} with \matfcn{seconds()}.
\item \underline{Implementation:} \matfcn{h5write}%\myfcn{h5tools.datasets.write}, \myfcn{h5tools.datasets.read}
\item \underline{Limitations:} N/A
\end{itemize}
\subsection{Miscellaneous data types}
\noindent Several specialized MATLAB datatypes from the Image Processing Toolbox are also included to demonstrate how the tricks above could be applied to write any other specialized datatype to an HDF5 file as well. See \myfcn{h5tools.datasets.write} and \myfcn{h5tools.datasets.read} for implementation. See CONTRIBUTING.md for information about adding new types.
% █████╗ ████████╗████████╗██████╗ ███████╗
% ██╔══██╗╚══██╔══╝╚══██╔══╝██╔══██╗██╔════╝
% ███████║ ██║ ██║ ██████╔╝███████╗
% ██╔══██║ ██║ ██║ ██╔══██╗╚════██║
% ██║ ██║ ██║ ██║ ██║ ██║███████║
% ╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝╚══════╝
\section{Attributes}
\noindent The high-level functions for attributes are \myfcn{h5tools.writeatt} and \myfcn{h5tools.readatt}. Many of the classes supported by h5tools for datasets rely on attributes to ensure accurate conversion when reading in the data. Because attributes cannot have attributes, the additional datatypes available here are less extensive and some differ in their implementation. The advantages of \myfcn{h5tools.writeatt} and \myfcn{h5tools.readatt} are:
\begin{enumerate}
\item Expanded support for MATLAB datatypes including \texttt{numeric}, \texttt{char}, \texttt{string}, \texttt{logical}, \texttt{datetime} and enumerated types. Several other datatypes will write to HDF5 but will not be read back in as the same datatype (\texttt{cellstr} and \texttt{duration}). If you try to write an attribute without write support, you'll get an error. If you write one without read support, a warning will be sent to the command line specifying the path within the HDF5 file, attribute name and datatype, so that you are aware.
\item The ability to work with one or more attributes at a time, or to read all the attributes of a dataset/group without knowing their names.
\end{enumerate}
The options for creating attributes with \myfcn{h5tools.writeatt} are demonstrated below.
\begin{lstlisting}[style=matlab-editor, basicstyle=\mlttfamily\footnotesize]
% Write a single attribute:
h5tools.writeatt('Test.h5', '/', 'A', 1);
% Write multiple attributes:
h5tools.writeatt('Test.h5', '/', 'A', 1, 'B', 2);
% Write all the fields of a struct as attributes
attStruct = struct('A', 1, 'B', 2);
h5tools.writeatt('Test.h5', '/', attStruct);
% Write all the key/value pairs in a containers.Map:
attMap = containers.Map('A', 1, 'B', 2);
h5tools.writeatt('Test.h5', '/', attMap);
% Write a struct or containers.Map followed by additional attributes:
h5tools.writeatt('Test.h5', '/', attMap, 'C', 3);
h5tools.writeatt('Test.h5', '/', attStruct, 'C', 3);
\end{lstlisting}
The options for reading attributes with \myfcn{h5tools.readatt} are demonstrated below.
\begin{lstlisting}[style=matlab-editor, basicstyle=\mlttfamily\footnotesize]
% 1. Read a single attribute, return the value
out = h5tools.readatt('File.h5', '/GroupOne', 'Attr1')
% 2. Read two attributes, return the values as two outputs
[out1, out2] = h5tools.readatt('File.h5', '/GroupOne', 'Attr1', 'Attr2')
% 3. Read two attributes, return the values in a containers.Map:
out = h5tools.readatt('File.h5', '/GroupOne', 'Attr1', 'Attr2')
% 4. Read all attributes, return the values in a containers.Map
out = h5tools.readatt('File.h5', '/GroupOne', 'all')
\end{lstlisting}
\noindent The implementation for numeric datatypes, \texttt{char}, \texttt{string}, \texttt{logical} and enumerated types is largely identical to that described above for datasets.
All attributes pass through \myfcn{h5tools.attributes.writeAttributeByType} and \myfcn{h5tools.attributes.readAttributeByType}. Any additional functions are noted in the ``Implementation'' points below.
\subsection{Numeric}
\noindent MATLAB supports direct conversion of numeric datatypes to and from HDF5 files with \matfcn{h5readatt} and \matfcn{h5writeatt}. The advantage of h5tools is a more user-friendly interface that enables working with multiple attributes as once, as discussed above.
\begin{itemize}
\item \underline{Preprocessing:} As with all other datatypes, the dataset creation and dataspace creation is handled rather than requiring the use of \matfcn{h5create}. Writing is performed with \matfcn{h5read}.
\item \underline{Persistence:} See the table below for the correspondence between MATLAB numeric types and HDF5 data types.\\
\begin{center}\begin{tabular}{| c | c | c |}
\hline
\textbf{MATLAB type} & \textbf{HDF5 Class} & \textbf{HDF5 Type}\\
\hline
\texttt{double} & \texttt{H5T\_FLOAT} & \texttt{H5T\_IEEE\_F64LE}\\
\texttt{single} & \texttt{H5T\_FLOAT} & \texttt{H5T\_IEEE\_F32LE}\\
\texttt{uint8} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_U8LE}\\
\texttt{int8} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_I8LE}\\
\texttt{uint16} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_U16LE}\\
\texttt{int16} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_I16LE}\\
\texttt{uint32} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_U32LE}\\
\texttt{int32} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_I32LE}\\
\texttt{uint64} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_U64LE}\\
\texttt{int64} & \texttt{H5T\_INTEGER} & \texttt{H5T\_STD\_I64LE}\\
\hline
\end{tabular}\end{center}
\item \underline{Postprocessing:} N/A
\item \underline{Implementation:} \matfcn{h5writeatt}, \matfcn{h5readatt}
\item \underline{Limitations:} For some reason, \textcolor{red}{row vectors are being returned as columns} (even though columns vectors are also returned as columns)
\end{itemize}
\subsection{Char}
\noindent Basically the same as \texttt{char} datasets
\begin{itemize}
\item \underline{Preprocessing:} N/A
\item \underline{Persistence:} Written as class \hdftype{H5T\_STRING} with character type \hdftype{H5T\_C\_S1} and set \hdftype{H5T\_CSET\_ASCII}.
\item \underline{Postprocessing:} N/A
\item \underline{Implementation:} \matfcn{h5writeatt}, \matfcn{h5readatt}
\item \underline{Limitations:}
\end{itemize}
\subsection{String}
\noindent Basically the same as \texttt{string} datasets
\begin{itemize}
\item \underline{Preprocessing:} N/A
\item \underline{Persistence:} Written as class \hdftype{H5T\_STRING} with character type \hdftype{H5T\_C\_S1} and set \hdftype{H5T\_CSET\_UTF8}.
\item \underline{Postprocessing:} N/A
\item \underline{Implementation:} \matfcn{h5writeatt}, \matfcn{h5readatt}
\item \underline{Limitations:}
\end{itemize}
\subsection{Logical}
\noindent HDF5 has no boolean datatype so, following other successful HDF5 libraries like h5py, this is written as an enumerated type.
\begin{itemize}
\item \underline{Preprocessing:} Conversion to \texttt{int8}.
\item \underline{Persistence:} written as \myurl{\texttt{H5T\_ENUM}} with two members (0=FALSE, 1=TRUE), where the member values are \hdftype{H5T\_STD\_I8LE}.
\item \underline{Postprocessing:} If an enumerated type dataset is read in and has two enumerated types matching those above, the values it will be treated as \texttt{logical}. The enumerated values will be converted from \texttt{uint8} to \texttt{logical}.
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeLogicalAttribute},\\ \myfcn{h5tools.datasets.readEnumTypeAttribute}.
\item \underline{Limitations:} N/A
\end{itemize}
\subsection{Enumerated types}
\noindent HDF5 offers enumerated type, which is an integer datatype restricted to a set of named values. For example, an enumerated type may contain three members (e.g. ``Red'', ``Green'', ``Blue'') and their corresponding values (0, 1, 2).
\begin{itemize}
\item \underline{Preprocessing:} The index of each enumerated type value within the full list of enumerated types (obtained with \matfcn{metaclass}) is converted to \texttt{uint32}. The class name (including any namespaces) will be appended to each enumeration name (e.g. ``RED'' will be written as ``MyClassName.Red'').
\item \underline{Persistence:} written as \myurl{\texttt{H5T\_ENUM}} with the values of each member in the enumeration as \hdftype{H5T\_STD\_U32LE}.
\item \underline{Postprocessing:} A string array of the enumeration members is read from the dataset through the datatype library (\texttt{H5T}), ordered by their enumeration value. If the associated class is not found on MATLAB's path, the data will be read back in as a string array using the format \texttt{x.y} where \texttt{x} is the class name in the attributes and \texttt{y} is the enumeration name. If the class is found on MATLAB's path, the string will be evaluated (\texttt{eval}). If the enumerated type was not written with h5tools and has no class name attribute, the string array will just contain the enumeration names. %with the class name in the attributes \texttt{char}.
\item \underline{Implementation:} \myfcn{h5tools.datasets.makeEnumTypeAttribute},\\ \myfcn{h5tools.datasets.readEnumTypeAttribute}.
\item \underline{Limitations:}
\end{itemize}
\subsection{Datetime}
\noindent Because attributes cannot be used to specify \texttt{datetime} information, as done for datasets, the process for \texttt{datetime} attributes is slightly different.
\begin{itemize}
\item \underline{Preprocessing:} Conversion to \texttt{string} and the format property is appended to each string
\item \underline{Persistence:} \hdftype{H5T\_STRING} with character type \hdftype{H5T\_C\_S1} and set \hdftype{H5T\_CSET\_UTF8}.
\item \underline{Postprocessing:} Conversion back to \texttt{datetime}.
\item \underline{Implementation:} \myfcn{h5tools.attributes.makeDatetimeAttribute}
\item \underline{Limitations:} The seconds are rounded when converting to \texttt{string}.
\end{itemize}
\subsection{Cellstr}
\noindent While \texttt{cellstr} is not supported, it is comparable to a \texttt{string} array and can be written as such.
\begin{itemize}
\item \underline{Preprocessing:} Conversion to \texttt{string} array.
\item \underline{Persistence:} \hdftype{H5T\_STRING} with character type \hdftype{H5T\_C\_S1} and set \hdftype{H5T\_CSET\_UTF8}.
\item \underline{Postprocessing:} N/A
\item \underline{Implementation:} See \texttt{string}.
\item \underline{Limitations:} \textcolor{red}{The data cannot be read back in as \texttt{cellstr} and will be returned as a \texttt{string} array instead.}
\end{itemize}
\subsection{Duration}
\noindent
\begin{itemize}
\item \underline{Preprocessing:} All durations are converted to seconds, then to \texttt{double}
\item \underline{Persistence:} Written as \hdftype{H5T\_IEEE\_F64LE}.
\item \underline{Postprocessing:} N/A
\item \underline{Implementation:}
\item \underline{Limitations:} \textcolor{red}{The data cannot be read back in as \texttt{duration} and will be returned as \texttt{double} instead.}
\end{itemize}
% ██╗ ██╗███╗ ██╗██╗ ██╗███████╗
% ██║ ██║████╗ ██║██║ ██╔╝██╔════╝
% ██║ ██║██╔██╗ ██║█████╔╝ ███████╗
% ██║ ██║██║╚██╗██║██╔═██╗ ╚════██║
% ███████╗██║██║ ╚████║██║ ██╗███████║
% ╚══════╝╚═╝╚═╝ ╚═══╝╚═╝ ╚═╝╚══════╝
\section{Soft Links}\label{sec:links}
\noindent Soft links are comparable to ``shortcuts'' within a file system. Currently, h5tools supports writing a dataset that is a soft link to another dataset or group within the HDF5 file (\myfcn{h5tools.writelink}). Arrays of soft links and soft link attributes are not supported.
Soft links are created as demonstrated below. \texttt{hdfFile} is the HDF5 file name, \texttt{linkLocation} is the path for the group where the link will be written, \texttt{linkName} is the name of the link and \texttt{targetLocation} is the HDF5 path of the group/dataset that will be linked to.
\begin{lstlisting}[style=matlab-editor, basicstyle=\mlttfamily\footnotesize]
% SYNTAX:
% h5tools.writelink(hdfFile, linkLocation, linkName, targetLocation);
% Add a soft link within group '/G1' that links to group '/G2'.
h5tools.writelink('File.h5', '/G1', 'Link1', '/G2');
% Read the link
[pathName, ID] = h5tools.readLink('File.h5', '/G1', 'Link1');
\end{lstlisting}
\myfcn{h5tools.readlink} returns the path within the HDF5 file of the linked dataset/group and the H5ML.id. If you return the H5ML.id, make sure to close it out later.
% ██████╗ ██████╗ ██████╗ ██╗ ██╗██████╗ ███████╗
% ██╔════╝ ██╔══██╗██╔═══██╗██║ ██║██╔══██╗██╔════╝
% ██║ ███╗██████╔╝██║ ██║██║ ██║██████╔╝███████╗
% ██║ ██║██╔══██╗██║ ██║██║ ██║██╔═══╝ ╚════██║
% ╚██████╔╝██║ ██║╚██████╔╝╚██████╔╝██║ ███████║
% ╚═════╝ ╚═╝ ╚═╝ ╚═════╝ ╚═════╝ ╚═╝ ╚══════╝
\section{Groups}\label{sec:groups}
\noindent MATLAB's high-level functions create groups en route to creating datasets but does not offer direct control of group creation. The \myfcn{h5tools.createGroup} function fills this gap by providing a high-level option for creating one or more groups, as demonstrated below.
The first input (\texttt{hdfFile}) is the HDF5 file name and the second input (\texttt{hdfPath}) is the HDF path where the groups will be written. Additional inputs (\texttt{varargin}) are group names. Groups do not need to be within \texttt{hdfPath}, but their group names must be specified relative to this location within the file.
\begin{lstlisting}[style=matlab-editor, basicstyle=\mlttfamily\footnotesize]
% SYNTAX
% h5tools.createGroup(hdfFile, hdfPath, varargin)
% Create one group within the root group "/"
h5tools.createGroup('Test.h5', '/', 'GroupOne);
% Create three groups within "/GroupOne"
h5tools.createGroup('Test.h5', '/GroupOne', 'GroupOne1A', 'Group1B', 'Group1C');
% Create two groups, one in the root group "/" and one in "/GroupOne"
h5tools.createGroup('Test.h5', '/', 'GroupTwo', '/GroupOne/Group1D');
\end{lstlisting}
\section{Indexing}
\noindent Several functions are provided for indexing the contents of an HDF5 file. More details about each can be found in the documentation within each function.
\begin{enumerate}
\item \myfcn{h5tools.collectDatasets} - Return the full path names of all datasets in an HDF5 file.
\item \myfcn{h5tools.collectGroups} - Return the full path names of all groups in an HDF5 file.
\item \myfcn{h5tools.collectSoftlinks} - Return the full path names of all soft links in an HDF5 file.
\item \myfcn{h5tools.getAttributeNames} - Return the names of all attributes of a specific group or dataset.
\end{enumerate}
\section{Miscellaneous}
\noindent Other HDF5 library wrappers
\begin{enumerate}
\item \myfcn{h5tools.move} - Moves a group or dataset to a new location. Useful for renaming.
\end{enumerate}
\section{Resources}
\begin{itemize}
\item \href{https://www.mathworks.com/help/matlab/hdf5-files.html}{\myurl{MATLAB}}: See their documentation for the high-level functions as well as the low-level library.
\item \href{https://docs.hdfgroup.org/hdf5/develop/}{\myurl{HDF5 API Reference}}: The official HDF5 documentation.
\item \href{https://github.com/NeurodataWithoutBorders/matnwb}{\myurl{NeurodataWithoutBorders}}: nwb-matlab has HDF5 support for a number of data types. Although their HDF5 functions are specific to NWB, it's still a great resource for learning about HDF5 I/O. They have support for object and region references, which is something I would like to add in the future.
\item \href{https://github.com/h5py/h5py}{\myurl{h5py}}: an excellent Python library for HDF5 -- if your project isn't tied to MATLAB, consider using this instead. For example, idea to make \texttt{logical} an enumerated type was borrowed from h5py.
\end{itemize}
\end{document}
@@ -0,0 +1,49 @@
function makeDatetimeAttribute(hdfName, pathName, attName, data)
% Write datetime as an attribute
%
% Description:
% Writes datetime as string with formatting appended
%
% Syntax:
% h5tools.attributes.makeDatetimeAttribute(hdfName, pathName, attName, data)
%
% Input:
% hdfName char
% HDF5 file name
% pathName char
% HDF5 path of the group/dataset containing the HDF5 attribute
% attName char
% Attribute name
% data datetime
% Data to be written to the attribute
%
% Outputs:
% N/A
%
% Examples:
% % Write attribute named 'A1' to group '/G1'
% input = datetime('now');
% out = h5tools.attributes.makeLogicalAttribute('File.h5', '/G1', 'Att1', input)
%
% See Also:
% h5tools.read, h5read
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
attName char
data datetime
end
dims = size(data);
input = data(:);
output = string.empty();
for i = 1:numel(input)
output = cat(1, output, string(datestr(input(i))) + sprintf(" (Format=%s)", input(i).Format));
end
output = reshape(output, dims);
h5writeatt(hdfName, pathName, attName, output);
@@ -0,0 +1,66 @@
function makeEnumTypeAttribute(hdfName, pathName, attName, data)
% Write enumerated type as attribute
%
% Description:
% Write a matlab enumeration class as an HDF5 attribute
%
% Syntax:
% makeEnumTypeAttribute(hdfName, pathName, attName, data)
%
% Inputs:
% hdfFile char or H5ML.id
% HDF5 file name or identifier
% hdfPath char
% Path of the dataset/group where attributes will be written
% attName char
% Attribute name
% data enum
% Data to write to the attribute
%
% Outputs:
% N/A
%
% Examples:
% % Write attribute named 'A1' to group '/G1'
% input = struct('A', [1 2 3]', 'B', [4 5 6]');
% out = h5tools.attributes.makeEnumTypeAttribute(...
% 'File.h5', '/G1', 'Att1', input)
%
% See Also:
% h5tools.writeatt, h5tools.readatt
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
attName char
data {mustBeEnum(data)}
end
% Determine enumeration characteristics
matlabClass = string(class(data));
mc = metaclass(data);
memberNames = arrayfun(@(x) string(x.Name), mc.EnumerationMemberList);
values = arrayfun(@(x) find(memberNames == string(x)), data);
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
objID = H5O.open(fileID, pathName, 'H5P_DEFAULT');
objIDx = onCleanup(@()H5O.close(objID));
typeID = H5T.enum_create('H5T_STD_U32LE');
typeIDx = onCleanup(@()H5T.close(typeID));
for i = 1:numel(memberNames)
H5T.enum_insert(typeID, [matlabClass + "." + memberNames(i)], uint32(i));
end
spaceID = H5S.create_simple(2, fliplr(size(data)), []);
spaceIDx = onCleanup(@()H5S.close(spaceID));
attID = H5A.create(objID, attName, typeID, spaceID, 'H5P_DEFAULT');
attIDx = onCleanup(@()H5A.close(attID));
H5A.write(attID, typeID, uint32(values));
@@ -0,0 +1,64 @@
function makeLogicalAttribute(hdfName, pathName, attName, data)
% Write logical as an attribute
%
% Description:
% Write logical to an attribute of an HDF5 group or dataset
%
% Syntax:
% h5tools.attributes.makeLogicalAttribute(hdfName, pathName, attName, data)
%
% Input:
% hdfName char
% HDF5 file name
% pathName char
% HDF5 path of the group/dataset containing the HDF5 attribute
% attName char
% Attribute name
% data logical
% Data to be written to the attribute
%
% Outputs:
% N/A
%
% Examples:
% % Write attribute named 'A1' to group '/G1'
% input = [true, false, true];
% out = h5tools.attributes.makeLogicalAttribute('File.h5', '/G1', 'Att1', input)
%
% See Also:
% h5tools.readatt, h5tools.attributes.readEnumTypeAttribute
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
attName char
data logical
end
% File
if ~isa(hdfName, 'H5ML.id')
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
else
fileID = hdfName;
end
% Parent dataset/group
objID = H5O.open(fileID, pathName, 'H5P_DEFAULT');
objIDx = onCleanup(@()H5O.close(objID));
% Datatype
typeID = H5T.enum_create('H5T_STD_U8LE');
typeIDx = onCleanup(@()H5T.close(typeID));
H5T.enum_insert(typeID, 'FALSE', 0);
H5T.enum_insert(typeID, 'TRUE', 1);
spaceID = H5S.create_simple(2, fliplr(size(data)), []);
spaceIDx = onCleanup(@()H5S.close(spaceID));
attID = H5A.create(objID, attName, typeID, spaceID, 'H5P_DEFAULT');
attIDx = onCleanup(@()H5A.close(attID));
H5A.write(attID, typeID, uint8(data));
@@ -0,0 +1,59 @@
function out = readAttributeByType(hdfName, pathName, attName)
% Read a single HDF5 attribute
%
% Description:
% Read a single HDF5 attribute and perform datatype-specific processing
%
% Syntax:
% out = h5tools.attributes.readAttributeByType(hdfName, pathName, attName)
%
% Input:
% hdfName char
% HDF5 file name
% pathName char
% HDF5 path of the group/dataset containing the HDF5 attribute
% attName char
% Attribute name to read
%
% Output:
% value
% Attribute value
%
% Examples:
% % Read attribute of group '/G1' named 'A1'
% out = h5tools.attributes.readAttributeByType('File.h5', '/G1', 'Att1')
%
% See Also:
% h5tools.readatt, h5readatt, h5tools.attributes.writeAttributeByType
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
attName char
end
import h5tools.datatypes.Classes
% Begin with h5readatt and post-process as needed
data = h5readatt(hdfName, pathName, attName);
dataType = Classes.getByPath(hdfName, pathName, attName);
if dataType == Classes.ENUM
out = h5tools.attributes.readEnumTypeAttribute(hdfName, pathName, attName);
return
end
% Check for datetime
if isstring(data) && all(contains(data(:), " (Format="))
txt = extractBefore(data, " (Format=");
fmt = extractBetween(data, "(Format=", ")");
out = arrayfun(@(x,y) datetime(x, "format", y), txt, fmt);
return
end
% Otherwise return as is
out = data;
@@ -0,0 +1,68 @@
function out = readEnumTypeAttribute(hdfName, pathName, attName)
% Read enumerated type attribute
%
% Description:
% Read an enumerated type attribute and convert to logical, MATLAB class
% with a defined enumeration or string array
%
% Syntax:
% out = h5tools.attributes.readEnumTypeAttribute(hdfName, pathName, attName)
%
% Input:
% hdfName char
% HDF5 file name
% pathName char
% HDF5 path of the group/dataset containing the HDF5 attribute
% attName char
% Attribute name to read
%
% Output:
% out enum, logical or string
% Contents of the enumerated type dataset
%
% Examples:
% % Read attribute of group '/G1' named 'A1'
% out = h5tools.attributes.readAttributeByType('File.h5', '/G1', 'Att1')
%
% See Also:
% h5tools.datasets.writeEnumTypeAttribute, h5info, h5readatt
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
attName char
end
data = string(h5readatt(hdfName, pathName, attName));
% Get attribute datatype information
info = h5info(hdfName, pathName);
idx = find(arrayfun(@(x)isequal(lower(x.Name),...
lower(attName)), info.Attributes));
memberNames = arrayfun(@(x) string(x.Name),...
info.Attributes(idx).Datatype.Type.Member);
if numel(memberNames) == 2 && all(contains(memberNames, ["FALSE", "TRUE"]))
% Process as logical
out = arrayfun(@(x) find(memberNames == x), data);
out = logical(out - 1);
else % Process as MATLAB class enumeration
enumSize = size(data);
data = data(:);
idx = strfind(data(1), ".");
charData1 = char(data(1));
% Check whether it's a class on MATLAB path
if ~isempty(idx) && exist(charData1(1:idx(end)), 'class') == 8
% Convert to MATLAB enum class
out = [];
for i = 1:numel(data)
out = cat(1, out, eval(sprintf("%s", data(i))));
end
out = reshape(out, enumSize);
else % Return as a string array instead
out = reshape(data, enumSize);
end
end
@@ -0,0 +1,72 @@
function writeAttributeByType(hdfName, pathName, attName, attValue)
% Writes an HDF5 attribute with type-specific processing
%
% Description:
% Ensure datatype compatibility, then writes as an attribute of a
% HDF5 dataset or group.
%
% Syntax:
% h5tools.attributes.writeAttributeByType(hdfName, pathName, attName, attValue)
%
% Inputs:
% hdfName char
% HDF5 file name
% pathName char
% Path of the dataset/group where attributes will be written
% attName char
% Name of the attribute
% attValue
% Data to write for the attribute
%
% Outputs:
% N/A
%
% Examples:
% % Write an attribute called 'A1' to group '/G1'
% input = 'test';
% h5tools.attributes.writeAttributeByType('File.h5', '/G1', 'A1', input)
%
% See Also:
% h5tools.writeatt, h5writeatt, h5tools.attributes.readAttributeByType
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
attName char
attValue
end
% Write the attributes
if islogical(attValue)
h5tools.attributes.makeLogicalAttribute(hdfName, pathName, attName, attValue);
elseif isenum(attValue)
h5tools.attributes.makeEnumTypeAttribute(hdfName, pathName, attName, attValue);
elseif isdatetime(attValue)
h5tools.attributes.makeDatetimeAttribute(...
hdfName, pathName, attName, attValue);
elseif ismember(class(attValue), {'table', 'struct', 'containers.Map', 'timetable'})
error('writeAttributeByType:InvalidInput',...
'Structs, tables and maps cannot be written as attributes');
else
attValue = getOutput(attValue);
h5writeatt(hdfName, pathName, attName, attValue);
end
end
function out = getOutput(data)
% Convert data to an HDF5-writable format
if iscellstr(data)
out = string(data);
warning("writeAttributeByType:Cellstr",...
"Cellstr will be converted to string and read back in as string");
elseif isduration(data)
out = seconds(data);
warning("writeAttributeByType:Duration",...
"Duration will be converted to double with seconds and read in as double");
else
out = data;
end
end
@@ -0,0 +1,73 @@
function makeCharDataset(hdfName, pathName, dsetName, txt)
% Create & write char to an HDF5 dataset
%
% Description:
% Create and write an HDF5 dataset for char
%
% Syntax:
% h5tools.datasets.makeCharDataset(fileName, pathName, dsetName, txt)
%
% Inputs:
% hdfName char
% HDF5 file name
% pathName char
% Path of the group where dataset was written
% dsetName char
% Name of the dataset
% txt char
% Data to write to the dataset
%
% Outputs:
% N/A
%
% Examples:
% % Write a dataset named 'D1' in group '/G1'
% input = 'test';
% h5tools.datasets.makeCharDataset('Test.h5', '/G1', 'D1', input);
%
% See Also:
% h5tools.write, h5tools.datasets.makeStringDataset
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char {mustBeHdfPath(hdfName, pathName)}
dsetName char
txt char
end
if isa(hdfName, 'H5ML.id')
fileID = hdfName;
else
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
end
typeID = H5T.copy('H5T_C_S1');
typeIDx = onCleanup(@()H5T.close(typeID));
H5T.set_size(typeID, 'H5T_VARIABLE');
H5T.set_strpad(typeID, 'H5T_STR_NULLTERM');
dspaceID = H5S.create('H5S_SCALAR');
dspaceIDx = onCleanup(@()H5S.close(dspaceID));
% Get the parent group
groupID = H5G.open(fileID, pathName);
groupIDx = onCleanup(@()H5G.close(groupID));
% Get the dataset
try
dsetID = H5D.create(groupID, dsetName, typeID, dspaceID, 'H5P_DEFAULT');
dsetIDx = onCleanup(@()H5D.close(dsetID));
catch ME
if contains(ME.message, 'name already exists')
dsetID = H5D.open(groupID, dsetName, 'H5P_DEFAULT');
dsetIDx = onCleanup(@()H5D.close(dsetID));
else
rethrow(ME);
end
end
H5D.write(dsetID, 'H5ML_DEFAULT', 'H5S_ALL', 'H5S_ALL', 'H5P_DEFAULT', txt);
end
@@ -0,0 +1,213 @@
function makeCompoundDataset(hdfName, pathName, dsetName, data)
% Write struct, table or containers.Map as compound dataset
%
% Description:
% Adds table/struct/containers.Map to HDF5 file as compound dataset
%
% Syntax:
% h5tools.datasets.makeCompoundDataset(hdfName, pathName, dsetName, data);
%
% Inputs:
% hdfName char or H5ML.id
% HDF5 file name or identifier
% pathName char
% HDF5 path to group where dataset will be written
% dsetName char
% Name of the dataset
% data table, struct or containers.Map
% Data to be written (must have equal # of elements)
%
% Outputs:
% N/A
%
% Examples:
% % Write a dataset named 'D1' in group '/G1'
% input = struct('A', 1:3, 'B', 4:6);
% h5tools.datasets.makeCompoundDataset('Test.h5', '/G1', 'D1', input);
%
% See Also:
% h5tools.write, h5tools.datasets.writeMapDataset,
% h5tools.datasets.readCompoundDataset
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeFile(hdfName)}
pathName char
dsetName char
data {mustBeA(data, ["struct", "table", "timetable", "containers.Map"])}
end
fullPath = h5tools.util.buildPath(pathName, dsetName);
% Record original class, column classes and row names
dataClass = class(data);
if istimetable(data)
data = timetable2table(data);
data.Time = seconds(data.Time);
end
columnClass = [];
colsToFix = [];
if istable(data) || istimetable(data)
rowNames = string(data.Properties.RowNames);
units = string(data.Properties.VariableUnits);
for i = 1:size(data, 2)
if i == 1 && strcmp(dataClass, 'timetable') %#ok<STISA>
columnClass = 'duration';
continue
end
if iscellstr(data{:,i})
% fix for cellstr is being transposed oddly
columnClass = [columnClass, ', ', 'cellstr']; %#ok<*AGROW>
colsToFix = cat(2, colsToFix, i);
else
columnClass = [columnClass, ', ', class(data{:,i})];
end
end
columnClass = columnClass(3:end); % FIX
for i = 1:numel(colsToFix)
% Am I missing something, why is this so hard
colData = string(data{:,colsToFix}')';
varName = data.Properties.VariableNames{colsToFix(i)};
data = removevars(data, colsToFix(i));
data.(varName) = colData;
data = movevars(data, varName, 'Before', colsToFix(i));
end
nDims = height(data);
data = table2struct(data);
else
f = fieldnames(data);
nElements = zeros(1, numel(f));
for i = 1:numel(f)
columnClass = [columnClass, ', ', class(data.(f{i}))];
nElements(i) = numel(data.(f{i}));
end
if numel(unique(nElements)) > 1
error('makeCompoundDataset:DifferentFieldSizes',...
'All fields in the structure must have the same number of elements');
end
nDims = max(nElements);
% Struct will not have units, row names
units = []; rowNames = [];
end
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
names = fieldnames(data);
S = struct();
for i = 1:length(names)
S.(names{i}) = {data.(names{i})};
end
data = S;
typeIDs = cell(length(names), 1);
sizes = zeros(size(typeIDs));
for i = 1:length(names)
val = data.(names{i});
if isdatetime(val)
data.(names{i}) = string(data.(names{i}));
val = string(val);
elseif iscell(val) && ~isstring(val)
data.(names{i}) = [val{:}];
val = val{1};
elseif isstring(val) && numel(val) == 1
val = char(val);
end
typeIDs{i} = getDataType(val);
sizes(i) = H5T.get_size(typeIDs{i});
end
typeID = H5T.create('H5T_COMPOUND', sum(sizes));
typeIDx = onCleanup(@(x)H5T.close(typeID));
for i = 1:length(names)
% Insert columns into compound type
H5T.insert(typeID, names{i}, sum(sizes(1:i-1)), typeIDs{i});
end
% Optimizes for type size
H5T.pack(typeID);
spaceID = H5S.create_simple(1, nDims, []);
spaceIDx = onCleanup(@()H5S.close(spaceID));
if h5tools.exist(fileID, fullPath)
warning('found and replaced %s', fullPath);
h5tools.deleteObject(hdfName, fullPath);
end
dsetID = H5D.create(fileID, fullPath, typeID, spaceID, 'H5P_DEFAULT');
dsetIDx = onCleanup(@()H5D.close(dsetID));
for i = 1:numel(names)
if isdatetime(data.(names{i}))
data.(names{i}) = cellstr(data.(names{i}));
end
end
H5D.write(dsetID, typeID, spaceID, spaceID, 'H5P_DEFAULT', data);
% Write original class and column classes attributes
h5tools.writeatt(hdfName, fullPath, 'Class', dataClass,...
'ColumnClass', columnClass);
% If it was a table and had units, add as attribute
if ~isempty(units)
h5tools.writeatt(hdfName, fullPath, 'VariableUnits', units);
end
if ~isempty(rowNames)
h5tools.writeatt(hdfName, fullPath, 'RowNames', rowNames);
end
end
function typeID = getDataType(var)
% GETDATATYPE
%
% Description:
% Maps MATLAB data types to H5 data types (incomplete)
%
% Syntax:
% typeID = getDataType(var)
% -------------------------------------------------------------
varType = class(var);
if contains(varType, 'int')
end
if isa(var, 'double')
typeID = 'H5T_IEEE_F64LE';
elseif ismember(class(var), {'char', 'cell', 'datetime'})
typeID = H5T.copy('H5T_C_S1');
H5T.set_size(typeID, 'H5T_VARIABLE');
H5T.set_cset(typeID, H5ML.get_constant_value('H5T_CSET_ASCII'));
elseif isstring(var) && numel(var) == 1
typeID = H5T.copy('H5T_C_S1');
H5T.set_size(typeID, 'H5T_VARIABLE');
H5T.set_cset(typeID, H5ML.get_constant_value('H5T_CSET_UTF8'));
elseif islogical(var)
typeID = 'H5T_STD_I32LE';
elseif isa(var, 'int8')
typeID = 'H5T_STD_I8LE';
elseif isa(var, 'uint8')
typeID = 'H5T_STD_U8LE';
elseif isa(var, 'int16')
typeID = 'H5T_STD_I16LE';
elseif isa(var, 'uint16')
typeID = 'H5T_STD_U16LE';
elseif isa(var, 'int32')
typeID = 'H5T_STD_I32LE';
elseif isa(var, 'uint32')
typeID = 'H5T_STD_U32LE';
elseif isa(var, 'int64')
typeID = 'H5T_STD_I64LE';
elseif isa(var, 'uint64')
typeID = 'H5T_STD_U64LE';
elseif isa(var, 'single')
typeID = 'H5T_IEEE_F32LE';
else
error('getDataType:UnrecognizedDataType',...
'Data type %s was not recognized for makeCompoundDataset', class(var));
end
end
@@ -0,0 +1,52 @@
function makeDateDataset(hdfName, pathName, dsetName, data)
% Write datetime to a dataset
%
% Description:
% Saves datetime as text dataset with class and date format stored as
% attributes for accurately reading back into datetime
%
% Syntax:
% makeDateDataset(hdfFile, pathName, data)
%
% Inputs:
% hdfName char
% HDF5 file name
% pathName char
% Path of the group where dataset was written
% dsetName char
% Name of the dataset
% data datetime
%
% Outputs:
% N/A
%
% Examples:
% % Write a dataset named 'D1' in group '/G1'
% input = datetime('now');
% h5tools.datasets.makeDateDataset('Test.h5', '/G1', 'D1', input);
%
% See Also:
% h5tools.write, h5tools.datasets.readDateDataset
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeFile(hdfName)}
pathName char
dsetName char
data datetime
end
fullPath = h5tools.util.buildPath(pathName, dsetName);
h5tools.datasets.makeStringDataset(hdfName, pathName,...
dsetName, "datetime");
h5tools.writeatt(hdfName, fullPath,...
'Year', data.Year,...
'Month', data.Month,...
'Day', data.Day,...
'Hour', data.Hour,...
'Minute', data.Minute,...
'Second', data.Second,...
'TimeZone', data.TimeZone);
@@ -0,0 +1,84 @@
function makeEnumTypeDataset(hdfName, pathName, dsetName, data)
% Write an enumerated type dataset
%
% Description:
% Write a dataset with a enumerated type datatype
%
% Syntax:
% h5tools.datasets.makeEnumTypeDataset(hdfName, pathName, dsetName, data)
%
% Inputs:
% hdfName char or H5ML.id
% HDF5 file name or identifier
% pathName char
% HDF5 path to group where dataset will be written
% dsetName char
% Name of the dataset
% data enum
% Data to be written
%
% Outputs:
% N/A
%
% Examples:
% % Write a dataset named 'D1' in group '/G1'
% input = [test.EnumClass.GROUPONE, test.EnumClass.GROUPTWO];
% h5tools.datasets.makeEnumTypeDataset('Test.h5', '/G1', 'D1', input);
%
% See Also:
% h5tools.write, h5tools.readEnumTypeDataset
%
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
dsetName char
data {mustBeEnum(data)}
end
fullPath = h5tools.util.buildPath(pathName, dsetName);
% Determine enumeration characteristics
mc = metaclass(data);
memberNames = arrayfun(@(x) string(x.Name), mc.EnumerationMemberList);
values = arrayfun(@(x) find(memberNames == string(x)), data);
% File
if ~isa(hdfName, 'H5ML.id')
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
else
fileID = hdfName;
end
% Datatype
typeID = H5T.enum_create('H5T_STD_U32LE');
typeIDx = onCleanup(@()H5T.close(typeID));
for i = 1:numel(memberNames)
H5T.enum_insert(typeID, memberNames(i), uint8(i));
end
% Dataspace
if isvector(data)
nDims = 1;
dims = length(data);
else
nDims = ndims(data);
dims = size(data);
end
dims = fliplr(dims);
spaceID = H5S.create_simple(nDims, dims, []);
spaceIDx = onCleanup(@()H5S.close(spaceID));
% Dataset
propList = H5P.create('H5P_DATASET_CREATE');
propListx = onCleanup(@()H5P.close(propList));
dsetID = H5D.create(fileID, fullPath, typeID, spaceID, propList);
dsetIDx = onCleanup(@()H5D.close(dsetID));
H5D.write(dsetID, typeID, spaceID, spaceID, 'H5P_DEFAULT', uint32(values));
@@ -0,0 +1,76 @@
function makeLogicalDataset(hdfName, pathName, dsetName, data)
% Write logical to an HDF5 dataset
%
% Description:
% Write logical as an enum type dataset with two values (true, false)
%
% Syntax:
% h5tools.dataset.makeLogicalDataset(hdfName, pathName, dsetName, data)
%
% Inputs:
% hdfName char or H5ML.id
% HDF5 file name or identifier
% pathName char
% HDF5 path to group where dataset will be written
% dsetName char
% Name of the dataset
% data logical
% Data to be written
%
% Outputs:
% N/A
%
% Examples:
% % Write a dataset named 'D1' in group '/G1'
% input = [true, false, true];
% h5tools.datasets.makeStringDataset('Test.h5', '/G1', 'D1', input);
%
%
% See also:
% h5tools.write, h5tools.datasets.readEnumTypeDataset
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
dsetName char
data logical
end
fullPath = h5tools.util.buildPath(pathName, dsetName);
% File
if ~isa(hdfName, 'H5ML.id')
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
else
fileID = hdfName;
end
% Datatype
typeID = H5T.enum_create('H5T_STD_U8LE');
typeIDx = onCleanup(@()H5T.close(typeID));
H5T.enum_insert(typeID, 'FALSE', 0);
H5T.enum_insert(typeID, 'TRUE', 1);
% Dataspace
if isvector(data)
nDims = 1;
dims = length(data);
else
nDims = ndims(data);
dims = size(data);
end
dims = fliplr(dims);
spaceID = H5S.create_simple(nDims, dims, []);
spaceIDx = onCleanup(@()H5S.close(spaceID));
% Dataset
propList = H5P.create('H5P_DATASET_CREATE');
propListx = onCleanup(@()H5P.close(propList));
dsetID = H5D.create(fileID, fullPath, typeID, spaceID, propList);
dsetIDx = onCleanup(@()H5D.close(dsetID));
H5D.write(dsetID, typeID, spaceID, spaceID, 'H5P_DEFAULT', uint8(data));
@@ -0,0 +1,57 @@
function makeMapDataset(hdfName, pathName, dsetName, data)
% Write a dataset as attributes
%
% Description:
% Write a dataset as attributes of a placeholder text dataset
%
% Syntax:
% h5tools.datasets.makeMapDataset(hdfName, pathName, dsetName, data)
%
% Inputs:
% hdfName char or H5ML.id
% HDF5 file name or identifier
% pathName char
% HDF5 path to group where dataset will be written
% dsetName char
% Name of the dataset
% data struct or containers.Map
% Data to be written
%
% Outputs:
% N/A
%
% Examples:
% % Write a dataset named 'D1' in group '/G1'
% input = struct('A', 1, 'B', 1:3);
% h5tools.datasets.makeMapDataset('Test.h5', '/G1', 'D1', input);
%
% Notes:
% Mapping a dataset to attributes is a last-ditch effort when compound
% dataset did not work due to unequal numbers of elements in each field
%
% See Also:
% h5tools.write, h5tools.datasets.makeCompoundDataset
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeFile(hdfName)}
pathName char
dsetName char
data {mustBeA(data, ["struct", "containers.Map"])}
end
fullPath = h5tools.util.buildPath(pathName, dsetName);
matlabClass = class(data);
h5tools.datasets.makeStringDataset(hdfName, pathName, dsetName, matlabClass);
if isstruct(data)
data = struct2map(data);
end
keys = data.keys;
for i = 1:numel(keys)
h5tools.writeatt(hdfName, fullPath, keys{i}, data(keys{i}));
end
@@ -0,0 +1,95 @@
function makeMatrixDataset(hdfName, pathName, dsetName, data, varargin)
% Write numeric data to an HDF5 dataset
%
% Description:
% Chains h5create and h5write for use with numeric data
%
% Syntax:
% h5tools.datasets.makeMatrixDataset(hdfFile, pathName, dsetName, data, varargin)
%
% Inputs:
% hdfName char
% HDF5 file name
% pathName char
% Path of the group where dataset was written
% dsetName char
% Name of the dataset
% data numeric
% Data to write to the dataset
% varargin optional inputs passed to h5create
%
% Outputs:
% N/A
%
% Examples:
% % Write a dataset named 'D1' in group '/G1'
% input = magic(5);
% h5tools.datasets.makeMatrixDataset('Test.h5', '/G1', 'D1', input);
%
% % Write a dataset with optional inputs to h5create for appending
% input = randn(3, 10);
% h5tools.datasets.makeMatrixDataset('Test.h5', '/G1', 'D2', input,...
% [Inf 10], 'ChunkSize', [1 10]);
% % Append to the dataset (STILL BEING TESTED)
% input2 = randn(2, 10);
% h5tools.datasets.makeMatrixDataset('Test.h5', '/G1', 'D2', input2);
%
% Notes:
% If no optional arguments are provided, the dataset will be created with
% a MaxSize equal to the size of the data.
%
% See Also:
% h5write, h5create, h5tools.write, h5tools.datasets.writeDatasetByType
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeFile(hdfName)}
pathName char
dsetName char
data {mustBeNumeric(data)}
end
arguments (Repeating)
varargin
end
fullPath = h5tools.util.buildPath(pathName, dsetName);
% See if max size was first variable input
if nargin > 4 && isnumeric(varargin{1})
maxSize = varargin{1};
startIdx = 2;
else
maxSize = size(data);
startIdx = 1;
end
% Create the dataset (if it doesn't exist already)
try
h5create(hdfName, fullPath, maxSize,...
'Datatype', class(data), varargin{startIdx:end});
dsetStart = ones(1, ndims(data));
dsetCount = size(data);
catch ME
% If it already exists, try to proceed with the next step
if contains(ME.message, 'already exists')
info = h5info(hdfName, fullPath);
curSize = info.Dataspace.Size;
maxSize = info.Dataspace.MaxSize;
dsetStart = ones(1, ndims(data));
for i = 1:numel(maxSize)
if isinf(maxSize(i))
dsetStart(i) = curSize(i);
else
dsetStart(i) = 1;
end
end
dsetCount = size(data);
else
rethrow(ME);
end
end
% Write the dataset
h5write(hdfName, fullPath, data, dsetStart, dsetCount);
@@ -0,0 +1,44 @@
function makeStringDataset(hdfName, pathName, dsetName, data)
% Write string to a dataset
%
% Description:
% Write a string array or scalar to an HDF5 dataset
%
% Syntax:
% h5tools.datasets.makeStringDataset(hdfName, pathName, dsetName, data)
%
% Inputs:
% hdfName char or H5ML.id
% HDF5 file name or identifier
% pathName char
% HDF5 path to group where dataset will be written
% dsetName char
% Name of the dataset
% data string
% Data to be written
%
% Outputs:
% N/A
%
% Examples:
% % Write a dataset named 'D1' in group '/G1'
% input = ["string", "array"];
% h5tools.datasets.makeStringDataset('Test.h5', '/G1', 'D1', input);
%
% See Also:
% h5tools.write, h5tools.datasets.writeDatasetByType
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeFile(hdfName)}
pathName char
dsetName char
data string
end
fullPath = h5tools.util.buildPath(pathName, dsetName);
if ~h5tools.exist(hdfName, fullPath)
h5create(hdfName, fullPath, size(data), 'DataType', 'string');
end
h5write(hdfName, fullPath, data);
@@ -0,0 +1,60 @@
function out = readCompoundDataset(hdfName, pathName, dsetName)
% Read a compound dataset
%
% Description:
% Read a dataset of type H5T_COMPOUND and convert to table
%
% Syntax:
% out = h5tools.datasets.readCompoundDataset(hdfName, pathName, dsetName)
%
% Inputs:
% hdfName char
% HDF5 file name
% pathName char
% Path of the group where dataset was written
% dsetName char
% Name of the dataset
%
% Output:
% out table
% Contents of the compound dataset
%
% Examples:
% % Read a dataset named "DS1" within group "/G1"
% out = h5tools.datasets.readCompoundDataset('Test.h5', '/G1', 'DSI');
%
% See Also:
% h5tools.read, h5tools.datasets.readDatasetByType,
% h5tools.datasets.makeCompoundDataset
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
fullPath = h5tools.util.buildPath(pathName, dsetName);
data = h5read(hdfName, fullPath);
out = struct2table(data);
% Handle individual classes
colClasses = h5readatt(hdfName, fullPath, 'ColumnClass');
colClasses = strsplit(colClasses, ', ');
if isempty(colClasses{1})
colClasses = colClasses(2:end);
end
for i = 1:numel(colClasses)
colName = out.Properties.VariableNames{i};
switch colClasses{i}
case 'string'
out.(colName) = string(out.(colName));
case 'datetime'
try
out.(colName) = datetime(uncell(out.(colName)));
catch ME
if startsWith(ME.message, 'Could not recognize')
out.(colName) = string(out.(colName));
else
rethrow(ME);
end
end
end
end
@@ -0,0 +1,178 @@
function out = readDatasetByType(hdfName, pathName, dsetName)
% Reads HDF5 dataset with type-specific processing
%
% Description:
% Reads dataset and handles post-processing, if necessary
%
% Syntax:
% out = h5tools.datasets.readDatasetByType(hdfName, pathName, dsetName)
%
% Inputs:
% hdfName char
% HDF5 file name
% pathName char
% Path of the group where dataset was written
% dsetName char
% Name of the dataset
%
% Outputs:
% data
% Contents of the HDF5 dataset
%
% Examples:
% % Read a dataset named "DS1" within group "/G1"
% out = h5tools.datasets.readDatasetByType('Test.h5', '/G1', 'DSI');
%
% See Also:
% h5tools.read, h5read
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
dsetName char
end
import h5tools.datatypes.Classes
fullPath = h5tools.util.buildPath(pathName, dsetName);
data = h5read(hdfName, fullPath);
if h5tools.hasAttribute(hdfName, fullPath, 'Class')
matlabClass = h5tools.readatt(hdfName, fullPath, 'Class');
if ismember(string(matlabClass), ["string", "char", "double", "single", "uint8", "uint16", "uint32", "uint64", "int64", "int32", "int16", "int8"])
matlabClass = [];
end
else % Numeric types, string, char
matlabClass = [];
end
% Return numeric data immediately
if isnumeric(data) && isempty(matlabClass)
out = data;
return
end
% Get HDF5 and MATLAB classes
dataClass = Classes.getByPath(hdfName, fullPath);
% Return string/char immediately, unless a placeholder
if dataClass == Classes.STRING
if isempty(matlabClass) && isstring(data)
if isscalar(data) && ismember(data, ["containers.Map", "struct"])
% Read as compound mapped to attributes
out = h5tools.readatt(hdfName, fullPath, 'all');
if isequal(data, "struct")
out = map2struct(out);
end
return
elseif isscalar(data) && isequal(data, "datetime")
out = h5tools.datasets.readDateDataset(hdfName, pathName, dsetName);
return
% The remaining placeholders are equal to the className
elseif ~isequal(data, matlabClass)
out = data;
return
end
end
end
% Handle compound together
if dataClass == Classes.COMPOUND
out = h5tools.datasets.readCompoundDataset(...
hdfName, pathName, dsetName);
switch matlabClass
case 'struct'
out = table2struct(out);
case 'timetable'
out.Time = seconds(out.Time);
out = table2timetable(out);
case 'containers.Map'
out = struct2map(table2struct(data));
case 'table'
if h5tools.hasAttribute(hdfName, fullPath, 'RowNames')
out.Properties.RowNames = ...
h5tools.readatt(hdfName, fullPath, 'RowNames');
end
if h5tools.hasAttribute(hdfName, fullPath, 'VariableUnits')
out.Properties.VariableUnits = ...
h5tools.readatt(hdfName, fullPath, 'VariableUnits');
end
end
return
end
% Handle enumerated types
if dataClass == Classes.ENUM
out = h5tools.datasets.readEnumTypeDataset(...
hdfName, pathName, dsetName);
return
end
% Return any other standard types that weren't caught before
if isempty(matlabClass)
out = data;
return
end
% Miscellaneous MATLAB classes
switch matlabClass
case 'datetime'
% dateFormat = h5readatt(hdfName, fullPath, 'Format');
% out = datetime(data, 'Format', dateFormat);
out = h5tools.datasets.readDateDataset(...
hdfName, pathName, dsetName);
case 'logical'
out = logical(data);
case 'duration'
out = seconds(data);
case 'cellstr'
out = cellstr(data);
case 'enum'
enumClass = h5readatt(hdfName, fullPath, 'EnumClass');
try
eval(sprintf('out = %s.%s;', enumClass, data));
catch ME
if strcmp(ME.identifier, 'MATLAB:undefinedVarOrClass')
warning('read:UnknownEnumerationClass',...
'The enumeration class %s could not be found, returning char', enumClass);
elseif strcmp(ME.identifier, 'MATLAB:subscripting:classHasNoPropertyOrMethod')
warning('read:UnknownEnumerationType',...
'The class %s does not contain %s, returning char', enumClass, data);
else
rethrow(ME);
end
end
case 'affine2d'
out = affine2d(data);
case 'simtform2d'
T = h5readatt(hdfName, fullPath, 'Translation')';
S = h5readatt(hdfName, fullPath, 'Scale');
R = h5readatt(hdfName, fullPath, 'RotationAngle');
out = simtform2d(S, R, T);
case 'imref2d'
imageSize = h5readatt(hdfName, fullPath, 'ImageSize')';
pixelExtentX = h5readatt(hdfName, fullPath, 'PixelExtentInWorldX');
pixelExtentY = h5readatt(hdfName, fullPath, 'PixelExtentInWorldY');
if pixelExtentX ~= 1 || pixelExtentY ~= 1
out = imref2d(imageSize, pixelExtentX, PixelExtentInWorldX);
return
end
xWorldLimits = h5readatt(hdfName, fullPath, 'XWorldLimits');
yWorldLimits = h5readatt(hdfName, fullPath, 'YWorldLimits');
if xWorldLimits(2) ~= (imageSize(2)+0.5) ...
|| yWorldLimits(2) ~= (imageSize(1) + 0.5)
out = imref2d(imageSize, xWorldLimits, yWorldLimits);
return
end
out = imref2d(imageSize);
otherwise
warning('readDatasetByType:UnknownClass',...
'MATLAB class %s was unrecognized for %s', matlabClass, fullPath);
out = data;
end
@@ -0,0 +1,44 @@
function out = readDateDataset(hdfName, pathName, dsetName)
% Read date dataset
%
% Description:
% Read a dataset written from MATLAB datetimes
%
% Syntax:
% out = h5tools.datasets.readDateDataset(hdfName, pathName, dsetName)
%
% Inputs:
% hdfName char
% HDF5 file name
% pathName char
% Path of the group where dataset was written
% dsetName char
% Name of the dataset
%
% Outputs:
% out datetime
% Contents of the dataset
%
% Examples:
% % Read a dataset named "DS1" within group "/G1"
% out = h5tools.datasets.readCompoundDataset('Test.h5', '/G1', 'DSI');
%
% See Also:
% h5tools.datasets.makeDateDataset
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
dsetName char
end
fullPath = h5tools.util.buildPath(pathName, dsetName);
map = h5tools.readatt(hdfName, fullPath, 'all');
value = [map('Year'), map('Month'), map('Day'),...
map('Hour'), map('Minute'), map('Second')];
out = datetime(value);
@@ -0,0 +1,60 @@
function out = readEnumTypeDataset(hdfName, pathName, dsetName)
% Read enumerated type dataset
%
% Description:
% Read enumerated type dataset and convert to logical, MATLAB class
% with a defined enumeration or string array
%
% Syntax:
% out = h5tools.datasets.readEnumTypeDataset(hdfName, pathName, dsetName)
%
% Inputs:
% hdfName char
% HDF5 file name
% pathName char
% Path of the group where dataset was written
% dsetName char
% Name of the dataset
%
% Output:
% out enum, logical or string
% Contents of the enumerated type dataset
%
% Examples:
% % Read a dataset named "DS1" within group "/G1"
% out = h5tools.datasets.readEnumTypeDataset('Test.h5', '/G1', 'DSI');
%
% See Also:
% h5tools.datasets.makeEnumTypeDataset, h5info,
% h5tools.datasets.makeLogicalDataset
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
fullPath = h5tools.util.buildPath(pathName, dsetName);
data = string(h5read(hdfName, fullPath));
info = h5info(hdfName, fullPath);
memberNames = arrayfun(@(x) string(x.Name), info.Datatype.Type.Member);
if isequal(memberNames, ["FALSE"; "TRUE"])
% Process as logical
out = arrayfun(@(x) find(memberNames == x), data);
out = logical(out - 1);
else % Process as enumerated type
enumSize = size(data);
data = data(:);
idx = strfind(data(1), ".");
charData1 = char(data(1));
% Check whether it's a class on MATLAB path
if ~isempty(idx) && exist(charData1(1:idx(end)), 'class') == 8
% Convert to MATLAB enum class
for i = 1:numel(data)
out = cat(1, out, eval(sprintf("%s", data(i))));
end
out = reshape(out, enumSize);
else % Return as a string array instead
out = reshape(data, enumSize);
end
end
@@ -0,0 +1,147 @@
function success = writeDatasetByType(hdfName, pathName, dsetName, data, varargin)
% WRITEDATASETBYTYPE
%
% Syntax:
% success = writeDatasetByType(hdfName, pathName, dsetName, data)
%
% Inputs:
% hdfName char or H5ML.id
% HDF5 file name or identifier
% pathName char
% HDF5 path to group where dataset will be written
% dsetName char
% Name of the dataset
% data struct or containers.Map
% Data to be written
%
% Outputs:
% N/A
%
% Supported data types:
% numeric, char, string, logical, table, timetable, datetime, duration
% cellstr, enum, struct, containers.Map, affine2d, imref2d, simtform2d
%
% Notes:
% See README.md for limitations
%
% See Also:
% h5tools.write
% By Sara Patterson, 2023 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeFile(hdfName)}
pathName char
dsetName char
data
end
arguments (Repeating)
varargin
end
fullPath = h5tools.util.buildPath(pathName, dsetName);
success = true;
if isnumeric(data)
h5tools.datasets.makeMatrixDataset(hdfName, pathName, dsetName, data, varargin{:});
h5tools.writeatt(hdfName, fullPath, 'Class', class(data));
return
end
if ischar(data)
h5tools.datasets.makeCharDataset(hdfName, pathName, dsetName, data);
h5tools.writeatt(hdfName, fullPath, 'Class', 'char');
return
end
if isstring(data)
h5tools.datasets.makeStringDataset(hdfName, pathName, dsetName, data);
h5tools.writeatt(hdfName, fullPath, 'Class', 'string');
return
end
if iscellstr(data) %#ok<ISCLSTR>
h5tools.datasets.makeStringDataset(hdfName, pathName, dsetName, string(data));
h5tools.writeatt(hdfName, fullPath, 'Class', 'cellstr');
return
end
if islogical(data)
h5tools.datasets.makeLogicalDataset(hdfName, pathName, dsetName, data);
h5tools.writeatt(hdfName, fullPath, 'Class', 'logical');
return
end
if isstruct(data) || istable(data) || isa(data, 'containers.Map')
try
h5tools.datasets.makeCompoundDataset(hdfName, pathName, dsetName, data);
h5tools.writeatt(hdfName, fullPath, 'Class', class(data));
catch ME
if strcmp(ME.identifier, 'makeCompoundDataset:DifferentFieldSizes')
h5tools.datasets.makeMapDataset(hdfName, pathName, dsetName, data);
else
rethrow(ME);
end
end
return
end
if istimetable(data)
h5tools.datasets.makeCompoundDataset(hdfName, pathName, dsetName, data);
h5tools.writeatt(hdfName, fullPath, 'Class', class(data),...
'Units', 'seconds');
return
end
if isdatetime(data)
h5tools.datasets.makeDateDataset(hdfName, pathName, dsetName, data);
h5tools.writeatt(hdfName, fullPath, 'Class', class(data));
return
end
if isduration(data)
h5tools.datasets.makeMatrixDataset(hdfName, pathName, dsetName, seconds(data), varargin{:});
h5tools.writeatt(hdfName, fullPath, 'Class', class(data),...
'Units', 'seconds');
return
end
if isenum(data)
h5tools.datasets.makeEnumTypeDataset(hdfName, pathName, dsetName, data);
return
end
% Miscellaneous data types. To get the necessary information written
% to the dataset, a combination of the dataset and/or the dataset's
% attributes are used. These can serve as resources for users to write
% additional functions for any datatypes not currently covered.
switch class(data)
case 'affine2d'
h5tools.datasets.makeMatrixDataset(hdfName, pathName, dsetName, data.T);
h5tools.writeatt(hdfName, fullPath, 'Class', class(data));
case 'imref2d'
h5tools.datasets.makeCharDataset(hdfName, pathName, dsetName, 'imref2d');
h5tools.writeatt(hdfName, fullPath, 'Class', class(data),...
'XWorldLimits', data.XWorldLimits,...
'YWorldLimits', data.YWorldLimits,...
'ImageSize', data.ImageSize,...
'PixelExtentInWorldX', data.PixelExtentInWorldX,...
'PixelExtentInWorldY', data.PixelExtentInWorldY,...
'ImageExtentInWorldX', data.ImageExtentInWorldX,...
'ImageExtentInWorldY', data.ImageExtentInWorldY,...
'YIntrinsicLimits', data.YIntrinsicLimits,...
'XIntrinsicLimits', data.XIntrinsicLimits);
case 'simtform2d'
h5tools.datasets.makeCharDataset(hdfName, pathName, dsetName, 'simtform2d');
h5tools.writeatt(hdfName, fullPath, 'Class', class(data),...
'Dimensionality', data.Dimensionality,...
'Scale', data.Scale,...
'RotationAngle', data.RotationAngle,...
'Translation', data.Translation);
otherwise
success = false;
warning('writeDatasetByType:UnidentifiedDataType',...
'The datatype %s is not supported', class(data));
end
@@ -0,0 +1,128 @@
classdef Classes
% CLASSES
%
% Description:
% Creates a human-readable representation of the HDF5 class enumeration
%
% Static Methods:
% obj = Classes.getByID(ID)
% Creates object from H5ML.id for dataset or datatype
% obj = Classes.getByPath(hdfName, pathName)
% Creates object from dataset path within the HDF5 file
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
enumeration
INTEGER
FLOAT
STRING
BITFIELD
OPAQUE
COMPOUND
ENUM
VLEN
ARRAY
end
methods (Static)
function obj = getByPath(hdfName, pathName, attName)
% GETBYID
%
% Description:
% Return data class from datatype ID
%
% Inputs:
% ID H5ML.id
% Datatype identifier
%
% Examples:
% % Get a dataset's type
% obj = getByPath('File.h5', '/Dset1')
%
% % Get an attribute's type
% obj = getByPath('File.h5', '/Dset1', 'Att1');
% -------------------------------------------------------------
import h5tools.files.HdfTypes
if isa(hdfName, 'H5ML.id')
fileID = hdfName;
else
fileID = H5F.open(hdfName);
fileIDx = onCleanup(@()H5F.close(fileID));
end
if nargin < 3
try % Dataset
objID = H5D.open(fileID, pathName);
objIDx = onCleanup(@()H5D.close(objID));
typeID = H5D.get_type(objID);
catch % TODO: Find exact error message...
error("getByPath:DatasetPathError",...
"Could not open dataset at path %s", pathName);
end
else
%try % Attribute
if strcmp(pathName, '/')
objID = H5A.open(fileID, attName);
objIDx = onCleanup(@()H5A.close(objID));
else
parentID = H5O.open(fileID, pathName, 'H5P_DEFAULT');
parentIDx = onCleanup(@() H5O.close(parentID));
objID = H5A.open(parentID, attName);
objIDx = onCleanup(@()H5A.close(objID));
end
typeID = H5A.get_type(objID);
%catch
%error("getByPath:PathOrAttError",...
% "Could not open attribute %s at path %s", attName, pathName);
%end
end
typeIDx = onCleanup(@()H5T.close(typeID));
obj = h5tools.datatypes.Classes.getByID(typeID);
end
function obj = getByID(ID)
% GETBYID
%
% Description:
% Return data class from datatype ID
%
% Inputs:
% ID H5ML.id
% Datatype identifier
% -------------------------------------------------------------
import h5tools.datatypes.Classes
import h5tools.files.HdfTypes
assert(HdfTypes.get(ID) == HdfTypes.DATATYPE,...
'Input H5ML.id must be a datatype identifier');
dataClass = H5T.get_class(ID);
switch dataClass
case H5ML.get_constant_value('H5T_INTEGER')
obj = Classes.INTEGER;
case H5ML.get_constant_value('H5T_FLOAT')
obj = Classes.FLOAT;
case H5ML.get_constant_value('H5T_STRING')
obj = Classes.STRING;
case H5ML.get_constant_value('H5T_BITFIELD')
obj = Classes.BITFIELD;
case H5ML.get_constant_value('H5T_OPAQUE')
obj = Classes.OPAQUE;
case H5ML.get_constant_value('H5T_COMPOUND')
obj = Classes.COMPOUND;
case H5ML.get_constant_value('H5T_ENUM')
obj = Classes.ENUM;
case H5ML.get_constant_value('H5T_VLEN')
obj = Classes.VLEN;
case H5ML.get_constant_value('H5T_ARRAY')
obj = Classes.ARRAY;
end
end
end
end
@@ -0,0 +1,54 @@
classdef HdfTypes
% HDFTYPES
%
% Description:
% A readable verion of the enumerated values returned by H5I.get_type
%
% Static methods:
% obj = h5tools.files.HdfTypes.get(ID)
% Returns the HdfTypes value for a given H5ML.id
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
enumeration
FILE
GROUP
DATATYPE
DATASPACE
DATASET
ATTR
BADID
end
methods (Static)
function obj = get(ID)
% Returns the HdfTypes value for a giving H5ML.id
% -------------------------------------------------------------
arguments
ID {mustBeA(ID, 'H5ML.id')}
end
import h5tools.files.HdfTypes
objType = H5I.get_type(ID);
switch objType
case H5ML.get_constant_value('H5I_FILE')
obj = HdfTypes.FILE;
case H5ML.get_constant_value('H5I_GROUP')
obj = HdfTypes.GROUP;
case H5ML.get_constant_value('H5I_DATATYPE')
obj = HdfTypes.DATATYPE;
case H5ML.get_constant_value('H5I_DATASPACE')
obj = HdfTypes.DATASPACE;
case H5ML.get_constant_value('H5I_DATASET')
obj = HdfTypes.DATASET;
case H5ML.get_constant_value('H5I_ATTR')
obj = HdfTypes.ATTR;
case H5ML.get_constant_value('H5I_BADID')
obj = HdfTypes.BADID;
end
end
end
end
@@ -0,0 +1,53 @@
function success = copyFile(hdfName, newName, overwriteFlag)
% Copy a file
%
% Description:
% Wrapper for built-in function that checks HDF5 extensions
%
% Syntax:
% success = h5tools.files.copyFile(hdfName, newName)
%
% Inputs:
% hdfName string
% Original HDF5 file name
% newName string
% New HDF5 file name (if no path, written to hdfName's location)
% overwriteFlag logical
% Whether to overwrite existing file (default = false)
% Outputs:
% success logical
% Whether file copy was successful
%
% See also:
% copyfile
% By Sara Patterson, 2023 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName string {mustBeHdfFile(hdfName)}
newName string
overwriteFlag logical = false
end
[newPath, ~, ext] = fileparts(newName);
if ~endsWith(newName, ".h5")
if ext == ""
newName = newName + ".h5";
else
error("copyFile:InvalidExtension",...
"File must end with .h5, not %s", ext);
end
end
if newPath == ""
oldPath = string(fileparts(which(hdfName)));
newName = fullfile(oldPath, newName);
end
if ~overwriteFlag && exist(newName, 'file')
error('copyFile:FileExists',...
'Cannot copy as %s because file exists. Set overwrite to true', newName);
end
success = copyfile(hdfName, newName);
@@ -0,0 +1,47 @@
function fileID = openFile(hdfName, readOnly)
% Open an HDF5 file
%
% Description:
% Convenience function for opening file for use with low-level library
% with a flag for specifying whether it should be opened as read-only
%
% Syntax:
% fileID = h5tools.files.openFile(hdfFile, readOnly)
%
% Inputs:
% hdfFile char
% HDF5 file name
% Optional inputs:
% readOnly logical, default = true
% Whether to open in read-only mode or allow write access too
%
% Output:
% fileID H5ML.id
% Identifier for the HDF5 file
%
% Examples:
% % Open file 'File.h5' as read only
% fileID = h5tools.files.openFile('File.h5');
%
% % Open file 'File.h5' with write access
% fileID = h5tools.files.openFile('File.h5', true);
%
% % Remember to close the file ID when done
% H5F.close(fileID)
%
% See Also:
% h5tools.createFile
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeFile(hdfName)}
readOnly logical = true
end
if readOnly
fileID = H5F.open(hdfName, 'H5F_ACC_RDONLY', 'H5P_DEFAULT');
else
fileID = H5F.open(hdfName, 'H5F_ACC_RDWR', 'H5P_DEFAULT');
end
@@ -0,0 +1,36 @@
function path = buildPath(varargin)
% Builds path out of provided components
%
% Description:
% Concatenates names into valid HDF5 path. If leading / is
% missing, it will be added. Similar concept to fullfile
%
% Syntax:
% path = h5tools.util.buildPath(varargin)
%
% Input:
% varargin char
% Components of an HDF5 path
%
% Output:
% path char
% Combined HDF5 path
%
% Examples:
% h5tools.util.buildPath('Group1', 'Group2')
% >> returns '/Group1/Group2'
%
% See Also:
% h5tools.util.splitPath, fullfile
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
path = [];
for i = 1:nargin
path = [path, '/', char(varargin{i})]; %#ok<AGROW>
end
% Make sure leading / isn't duplicated
while strcmp(path(2), '/')
path = path(2:end);
end
@@ -0,0 +1,31 @@
function lastName = getPathEnd(pathName)
% Get final group/dataset from HDF5 path
%
% Description:
% Extracts the final group/dataset from a full path
%
% Syntax:
% parentPath = h5tools.util.getPathEnd(pathName)
%
% Inputs:
% pathName char
% HDF5 path
%
% Outputs:
% lastName char
%
% Examples:
% h5tools.util.getPathEnd('/GroupOne/GroupTwo/Dataset')
% >> 'Dataset'
%
% See Also:
% h5tools.util.getPathParent
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
pathName char
end
idx = strfind(pathName, '/');
lastName = pathName(idx(end)+1:end);
@@ -0,0 +1,38 @@
function out = getPathOrder(pathName)
% Get path distance from root group
%
% Description:
% Returns the order of one or more HDF5 path names where the root group
% has an order of 1.
%
% Syntax:
% out = h5tools.util.getPathOrder(pathName)
%
% Inputs:
% pathName string
% HDF5 path name(s) as a scalar string or string array
%
% Outputs:
% out double
% Order of each provided path
%
% Examples:
% h5tools.util.getPathOrder('/G1/G1a');
% >> returns 2
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
pathName string
end
if ~isscalar(pathName)
out = arrayfun(@(x)h5tools.util.getPathOrder(x), pathName);
return
end
out = numel(strfind(pathName, "/"));
@@ -0,0 +1,29 @@
function parentPath = getPathParent(pathName)
% Returns leading groups for an HDF5 path
%
% Description:
% Removes last identifier in path name, returning parent
%
% Syntax:
% parentPath = h5tools.util.getPathParent(pathName)
%
% Example:
% h5tools.util.getParentPath('/GroupOne/GroupTwo/Dataset')
% >> '/GroupOne/GroupTwo'
%
% See Also:
% h5tools.util.getPathEnd, h5tools.util.splitPath
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
pathName char
end
idx = strfind(pathName, '/');
if numel(idx) == 1 && idx == 1
parentPath = '/';
else
parentPath = pathName(1:idx(end)-1);
end
@@ -0,0 +1,31 @@
function [parentPath, objName] = splitPath(hdfPath)
% Split a path into the parent path and the last group/dataset
%
% Description:
% Split the last group/dataset from the full path and return both.
%
% Syntax:
% [parentPath, objName] = h5tools.util.splitPath(hdfPath)
%
% Inputs:
% hdfPath char
% A path within the HDF5 file
%
% Outputs:
% parentPath char
% The full path, minus the last group/dataset name
% objName char
% The group/dataset name
%
% Examples:
% % Split path into '/G1' and 'DS1'
% [parentPath, objName] = h5tools.util.splitPath('/G1/DS1');
%
% See also:
% h5tools.util.getPathParent, h5tools.util.getPathEnd
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
parentPath = h5tools.util.getPathParent(hdfPath);
objName = h5tools.util.getPathEnd(hdfPath);
@@ -0,0 +1,72 @@
function datasetNames = collectDatasets(hdfName, sortFlag)
% Return full HDF5 paths of all datasets in file
%
% Description:
% Collect all the dataset names (full paths) in an HDF file
%
% Syntax:
% datasetNames = h5tools.collectDatasets(hdfName, sortFlag)
%
% Inputs:
% hdfName char or H5ML.id
% HDF5 file name or identifier
% sortFlag logical (default=false)
% Whether to sort the results alphabetically
%
% Outputs:
% datasetNames string
% A string array containing the full paths of all
% datasets within the HDF5 file
%
% Examples:
% % Return all dataset names
% datasetNames = h5tools.collectDatasets('File.h5');
%
% % Return the dataset names sorted alphabetically
% datasetNames = h5tools.collectDatasets('File.h5', true);
%
% See also:
% h5tools.collectGroups, h5tools.collectSoftlinks, h5tools.getAttributeNames
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
sortFlag logical = false
end
if ~isa(hdfName, 'H5ML.id')
rootID = h5tools.files.openFile(hdfName, true);
rootIDx = onCleanup(@()H5F.close(rootID));
else
rootID = hdfName;
end
datasetNames = string.empty();
[~, datasetNames] = H5O.visit(rootID, 'H5_INDEX_NAME',...
'H5_ITER_NATIVE', @datasetVisitFcn, datasetNames);
if sortFlag && ~isempty(datasetNames)
datasetNames = sort(datasetNames);
end
end
function [status, datasetNames] = datasetVisitFcn(rootID, name, datasetNames)
% Visit function to iterate through an HDF5 file
objID = H5O.open(rootID, name, 'H5P_DEFAULT');
info = H5O.get_info(objID);
H5O.close(objID);
if string(name) == "."
status = 0;
return
end
if info.type == H5ML.get_constant_value('H5O_TYPE_DATASET')
datasetNames = cat(1, datasetNames, "/" + string(name));
end
status = 0;
end
@@ -0,0 +1,69 @@
function groupNames = collectGroups(hdfName, sortFlag)
% Returns full HDF5 paths for all groups
%
% Description:
% Get the full HDF5 paths of all groups in an HDF5 file
%
% Syntax:
% groupNames = h5tools.collectGroups(hdfName, sortFlag)
%
% Inputs:
% hdfName char or H5ML.id
% HDF5 file name or identifier
% sortFlag logical (default=false)
% Whether to sort the results alphabetically
%
% Outputs:
% groupNames string array
% Full paths of all groups in the HDF5 file
%
% Examples:
% % Return all group names
% groupNames = h5tools.collectGroups('File.h5');
%
% % Return the group names sorted alphabetically
% groupNames = h5tools.collectGroups('File.h5', true);
%
% See also:
% h5tools.collectDatasets, h5tools.collectSoftlinks, h5tools.getAttributeNames
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
sortFlag logical = false
end
if isa(hdfName, 'H5ML.id')
rootID = hdfName;
else
rootID = h5tools.files.openFile(hdfName, true);
rootIDx = onCleanup(@()H5F.close(rootID));
end
groupNames = string.empty();
[~, groupNames] = H5O.visit(rootID, 'H5_INDEX_NAME',...
'H5_ITER_NATIVE', @groupVisitFcn, groupNames);
if sortFlag && ~isempty(groupNames)
groupNames = sort(groupNames);
end
end
function [status, groupNames] = groupVisitFcn(rootID, name, groupNames)
objID = H5O.open(rootID, name, 'H5P_DEFAULT');
info = H5O.get_info(objID);
H5O.close(objID);
if string(name) == "."
status = 0;
return
end
if info.type == H5ML.get_constant_value('H5O_TYPE_GROUP')
groupNames = cat(1, groupNames, "/" + string(name));
end
status = 0;
end
@@ -0,0 +1,71 @@
function linkNames = collectSoftlinks(hdfName, sortFlag)
% Get all soft links in file
%
% Description:
% Returns all soft links in an HDF5 file
%
% Syntax:
% linkNames = h5tools.collectSoftlinks(hdfName, sortFlag)
%
% Inputs:
% hdfName HDF file name or H5ML.id
% The name of the HDF5 file, or the file identifier
% sortFlag logical (default=false)
% Whether to sort the results alphabetically
%
% Outputs:
% linkNames string array
% The full hdfPaths and names of all softlinks
%
% Examples:
% % Return all link names
% linkNames = h5tools.collectSoftlinks('File.h5');
%
% % Return the link names sorted alphabetically
% linkNames = h5tools.collectSoftlinks('File.h5', true);
%
% See also:
% h5tools.collectDatasets, h5tools.collectGroups, h5tools.getAttributeNames
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
sortFlag logical = false
end
if isa(hdfName, 'H5ML.id')
rootID = hdfName;
else
rootID = h5tools.files.openFile(hdfName, true);
rootIDx = onCleanup(@()H5F.close(rootID));
end
linkNames = string.empty();
[~, linkNames] = H5L.visit(rootID, 'H5_INDEX_NAME',...
'H5_ITER_NATIVE', @softlinkVisitFcn, linkNames);
if sortFlag && ~isempty(linkNames)
linkNames = sort(linkNames);
end
end
function [status, dataOut] = softlinkVisitFcn(groupID, name, dataIn)
% LINKVISITFCN
%
% Description:
% Iterator function for visiting all links in an HDF5 file and
% returning the names of soft links
% ---------------------------------------------------------------------
info = H5L.get_info(groupID, name, 'H5P_DEFAULT');
if isequal(info.type, H5ML.get_constant_value('H5L_TYPE_SOFT'))
dataOut = cat(1, dataIn, "/" + string(name));
else
dataOut = dataIn;
end
status = 0;
end
@@ -0,0 +1,68 @@
function fileID = createFile(hdfName, overwrite)
% Create a new HDF5 file
%
% Description:
% Create an HDF5 file with optional overwriting of existing file
%
% Syntax:
% h5tools.createFile(hdfFile)
% fileID = h5tools.createFile(hdfFile)
% fileID = h5tools.createFile(hdfFile, overwrite)
%
% Inputs:
% hdfFile char
% Name of the HDF5 file
% Optional inputs:
% overwrite logical, default = false
% Whether to overwrite the file if it exists
%
% Optional outputs:
% fileID H5ML.id
% File Identifier
%
% Examples:
% % Create an HDF5 file
% h5tools.createFile('File.h5');
%
% % Create an HDF5 file, overwrite if existing
% h5tools.createFile('File.h5', true)
%
% % Create file and return identifier to use with low-level library
% fileID = h5tools.createFile('File.h5');
%
% See also:
% h5tools.files.openFile, h5create
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char
overwrite logical = false
end
if ~endsWith(hdfName, '.h5')
error('createFile:InvalidExtension',...
'File name must end with ".h5"');
end
if overwrite
fileID = H5F.create(hdfName, 'H5F_ACC_TRUNC',...
H5P.create('H5P_FILE_CREATE'), H5P.create('H5P_FILE_ACCESS'));
else
try
fileID = H5F.create(hdfName);
catch ME
if strcmp(ME.identifier, 'MATLAB:imagesci:hdf5io:resourceAlreadyExists')
error("createFile:FileExists",...
"File already exists, set overwrite=true to recreate");
else
rethrow(ME);
end
end
end
if nargout == 0
H5F.close(fileID);
end
fprintf('Created HDF5 file: %s\n', hdfName);
@@ -0,0 +1,64 @@
function createGroup(hdfName, pathName, varargin)
% Create new group(s) in HDF5 file
%
% Description:
% Create one or more new groups
%
% Syntax:
% groupID = h5tools.createGroup(fileName, pathName, varargin)
%
% Inputs:
% hdfName char
% HDF5 file name
% pathName char
% HDF5 file path
% varargin char
% One or more new group name(s)
%
% Examples:
% h5tools.createGroup(hdfName, '/', 'Group1')
% h5tools.createGroup(hdfName, '/', 'Group1', 'Group2')
% h5tools.createGroup(hdfName, '/', 'Group1', '/Group1/Dataset1')
%
% Notes:
% Groups can be in different locations as long as the location is
% specified relative to "pathName".
%
% See also:
% h5tools.collectGroups
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
end
arguments (Repeating)
varargin
end
if isa(hdfName, 'H5ML.id')
fileID = hdfName;
else
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
end
for i = 1:numel(varargin)
try
groupPath = h5tools.util.buildPath(pathName, varargin{i});
groupID = H5G.create(fileID, groupPath,...
'H5P_DEFAULT', 'H5P_DEFAULT', 'H5P_DEFAULT');
groupIDx = onCleanup(@()H5G.close(groupID));
catch ME
if contains(ME.message, 'name already exists')
warning('createGroup:GroupExists',...
'Group %s already exists, skipping', groupPath);
else
rethrow(ME);
end
end
end
end
@@ -0,0 +1,37 @@
function deleteAttribute(hdfName, pathName, name)
% Delete an attribute
%
% Description:
% Delete an attribute
%
% Syntax:
% h5tools.deleteAttribute(hdfFile, pathName, name)
%
% Inputs:
% hdfName char or H5ML.id
% The HDF5 file name or identifier
% pathName char
% The path to the dataset or group within the HDF5 file
%
% Examples:
% h5tools.deleteAttribute('File.h5', '/GroupOne', 'AttrName')
%
% See also:
% h5tools.deleteObject
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char {mustBeHdfPath(hdfName, pathName)}
name char
end
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
fprintf('Deleting %s:%s attribute %s\n', hdfName, pathName, name);
rootID = H5O.open(fileID, pathName, 'H5P_DEFAULT');
rootIDx = onCleanup(@()H5O.close(rootID));
H5A.delete(rootID, name);
@@ -0,0 +1,61 @@
function deleteObject(hdfName, pathName)
% Delete a group or dataset
%
% Description:
% Delete a group or dataset within the HDF5 file (see Warning below)
%
% Syntax:
% h5tools.deleteObject(fileID, pathName)
%
% Inputs:
% hdfName char, H5ML.id
% HDF5 file name or identifier
% pathName char
% HDF5 path to the dataset/group to delete
%
% Examples:
% % Delete 'Dataset1'
% h5tools.deleteObject(hdfName, '/Group1/Dataset1')
%
% % Delete 'Group1A'
% h5tools.deleteObject(hdfName, '/Group1/Group1A')
%
% Warning:
% Deleting an HDF5 object only removes the link, the data still exists
% and the file size is not reduced. Space can only be recovered with
% h5repack in the HDF5 library
%
% See also:
% h5tools.deleteAttribute
% By Sara Patterson, 2023 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char {mustBeHdfPath(hdfName, pathName)}
end
[pathName, objName] = h5tools.util.splitPath(pathName);
if ~isa(hdfName, 'H5ML.id')
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
else
fileID = hdfName;
end
if strcmp(pathName, '/')
parentID = fileID;
else
try
parentID = H5O.open(fileID, pathName);
parentIDx = onCleanup(@()H5O.close(parentID));
catch
parentID = H5G.open(fileID, pathName);
parentIDx = onCleanup(@()H5G.close(parentID));
end
end
H5L.delete(parentID, objName, 'H5P_DEFAULT');
@@ -0,0 +1,48 @@
function tf = exist(hdfName, pathName)
% Check whether group or dataset exists
%
% Description:
% Check if group or dataset exists in the HDF5 file
%
% Syntax:
% tf = exist(hdfName, hdfName)
%
% Inputs:
% hdfName HDF6 file name or H5ML.id
% The HDF5 file name or identifier
% pathName char
% HDF5 path to group or dataset
%
% Output:
% tf logical
% Whether the path exists within HDF5 file or not
%
% Examples:
% tf = h5tools.exist('File.h5', '/GroupOne/DatasetA')
%
% See also:
% h5tools.hasAttribute
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char
end
if isa(hdfName, 'H5ML.id')
fileID = hdfName;
else
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
end
try
tf = H5L.exists(fileID, pathName, 'H5P_DEFAULT');
catch ME
if contains(ME.message, 'component not found')
tf = false;
else
rethrow(ME);
end
end
@@ -0,0 +1,52 @@
function names = getAttributeNames(hdfName, pathName)
% Get names of all attributes
%
% Description:
% Return all attribute names for an HDF5 dataset or group
%
% Syntax:
% names = h5tools.getAllAttributeNames(hdfFile, pathName)
%
% Inputs:
% hdfName char or H5ML.id
% The HDF5 file name or identifier
% pathName char
% The path to the dataset or group within the HDF5 file
%
% Outputs:
% names string
% The names of all attributes contained by the object
%
% Examples:
% % Get all attributes for group "/G1"
% names = h5tools.getAttributeNames('File.h5', '/G1');
%
% See also:
% h5tools.hasAttribute, h5tools.readatt
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char
end
if isa(hdfName, 'H5ML.id')
fileID = hdfName;
else
fileID = H5F.open(hdfName);
fileIDx = onCleanup(@()H5F.close(fileID));
end
rootID = H5O.open(fileID, pathName, 'H5P_DEFAULT');
rootIDx = onCleanup(@()H5O.close(rootID));
names = string.empty();
[~, ~, names] = H5A.iterate(rootID, 'H5_INDEX_NAME',...
'H5_ITER_NATIVE', 0, @attributeIterateFcn, names);
end
function [status, names] = attributeIterateFcn(~, name, ~, names)
names = cat(1, names, string(name));
status = 0;
end
@@ -0,0 +1,38 @@
function tf = hasAttribute(hdfName, pathName, attName)
% HASATTRIBUTE
%
% Description:
% Determine whether a specific attribute is present
%
% Syntax:
% tf = hasAttribute(hdfFile, pathName, paramName)
%
% Inputs:
% hdfName char or H5ML.id
% The HDF5 file name or identifier
% pathName char
% The path to the dataset or group within the HDF5 file
% attName char
% The attribute name or names
%
% Outputs:
% tf logical
% Whether or not the attribute exists
%
% Examples:
% tf = h5tools.hasAttribute('Test.h5', '/GroupOne', 'AttrOne');
%
% See also:
% h5tools.getAttributeNames, h5tools.readatt
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
attName string
end
attNames = h5tools.getAttributeNames(hdfName, pathName);
tf = ismember(attName, attNames);
@@ -0,0 +1,50 @@
function move(hdfName, sourcePath, destPath)
% Move a group (useful for renaming)
%
% Syntax:
% h5tools.move(hdfName, sourcePath, destPath)
%
% Inputs:
% hdfName HDF5 file name or H5ML.id
% sourcePath char
% Path of the group/dataset to move
% destPath char
% Destination path (including new name)
%
% Examples:
% % Rename and move /Group1/Group1A to /Group3
% h5tools.move('MyFile.h5', '/Group1/Group1A', '/Group3')
% % Full example in test/MoveTest.m
%
% Notes:
% Any softlinks to the original path will not be changed!!!
%
% See also:
% H5L.move
% By Sara Patterson, 2023 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName
sourcePath char
destPath char
end
if ~isa(hdfName, 'H5ML.id')
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@() H5F.close(fileID));
else
fileID = hdfName;
end
[sourcePath, sourceName] = h5tools.util.splitPath(sourcePath);
[destPath, destName] = h5tools.util.splitPath(destPath);
sourceID = H5G.open(fileID, sourcePath);
sourceIDx = onCleanup(@() H5G.close(sourceID));
destID = H5G.open(fileID, destPath);
destIDx = onCleanup(@() H5G.close(destID));
H5L.move(sourceID, sourceName, destID, destName, 'H5P_DEFAULT', 'H5P_DEFAULT');
@@ -0,0 +1,23 @@
function out = read(hdfName, pathName, dsetName)
% READ
%
% Syntax:
% out = read(hdfName, pathName, dsetName)
%
% Supported data types:
% datetime, char, numeric, logical, table, timetable, string, duration
% cellstr, enum, containers.Map, affine2d, imref2d, simtform2d
%
% See Also:
% h5read, h5tools.datasets.readDatasetByType
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
dsetName string
end
out = h5tools.datasets.readDatasetByType(hdfName, pathName, dsetName);
@@ -0,0 +1,123 @@
function varargout = readatt(hdfName, pathName, varargin)
% Read HDF5 attributes
%
% Description:
% Read one, multiple or all attributes of a dataset or group
%
% Syntax:
% out = readatt(hdfName, pathName, varargin)
%
% Input:
% hdfName char
% HDF5 file name
% pathName char
% HDF5 path of the group/dataset containing the HDF5 attribute
% varargin chars or strings
% Attribute name(s) to read or "all" for all attributes
% See examples below for varargin specification
%
% Output:
% 1. If you provide a single attribute name, the output will be that
% attribute's value.
% 2. If you provide multiple attribute names and a matching number of
% outputs, the attribute values will be assigned in order to each output
% 3. If you provide multiple attribute names and a single output, the
% attributes will be returned as containers.Map.
% 4. If you use "all", the attributes will return as a containers.Map,
% where the attribute names are the keys for the attribute values.
% The examples are numbered by output type above to demonstrate
%
%
% Example:
% % Create an HDF5 file and write attributes to "/GroupOne"
% h5tools.createFile('File.h5');
% h5tools.createGroup('File.h5', '/', 'GroupOne');
% h5tools.writeatt('File.h5', '/GroupOne',...
% 'Attr1', 1:3, 'Attr2', "hello", 'Attr3', 'hej');
%
% % 1. Return a single attribute
% out = readatt('File.h5', '/GroupOne', 'Attr1')
%
% % 2. Read two attributes, return as two outputs
% [out1, out2] = readatt('File.h5', '/GroupOne', 'Attr1', 'Attr2')
%
% % 3. Read two attributes, return as containers.Map:
% out = readatt('File.h5', '/GroupOne', 'Attr1', 'Attr2')
%
% % 4. Read all attributes, return as containers.Map
% out = readatt('File.h5', '/GroupOne', 'all')
%
% See Also:
% h5tools.writeatt, h5readatt, h5tools.getAllAttributeNames
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
end
arguments (Repeating)
varargin char
end
if nargin < 3
error('readatt:InsufficientInputs',...
'Must provide at least 3 inputs.');
end
inputs = convertCharsToStrings(varargin)';
% Process multiple attribute inputs
if numel(inputs) > 1
% Confirm output is either 1 or matches # of inputs
if nargout > 1 && nargout ~= numel(inputs)
error('readatt:OutputsDoesNotMatchAttributeNumber',...
'Must have %s outputs or 1 output to get containers.Map', numel(inputs));
end
% Collect the attribute values
map = containers.Map();
for i = 1:numel(inputs)
map(char(inputs(i))) = h5tools.attributes.readAttributeByType(...
hdfName, pathName, inputs(i));
end
% Return as either a map or individual outputs
if nargout == 1
varargout{1} = map;
return
elseif nargout == numel(inputs)
for i = 1:numel(inputs)
varargout{i} = map(char(inputs(i))); %#ok<AGROW>
end
return
end
end
% Process a single attribute input
if numel(inputs) == 1
if nargout > 1
error('readatt:InvalidOutputCount',...
'Multiple outputs are not supported when specifying a single attribute name or ''all''');
end
if lower(inputs(1)) == "all"
attNames = h5tools.getAttributeNames(hdfName, pathName);
map = containers.Map();
% If there are no attributes, return empty map
if isempty(attNames)
varargout{1} = map;
return
end
for i = 1:numel(attNames)
map(char(attNames(i))) = h5tools.attributes.readAttributeByType(...
hdfName, pathName, attNames(i));
end
varargout{1} = map;
else % A single attribute name was provided
out = h5tools.attributes.readAttributeByType(...
hdfName, pathName, inputs(1));
varargout{1} = out;
end
end
end
@@ -0,0 +1,55 @@
function [linkPath, ID] = readlink(hdfName, pathName, linkName)
% Read a soft link
%
% Description:
% Write a dataset that is a soft link to a group or another dataset
% within the HDF5 file
%
% Syntax:
% [path, ID] = h5tools.readlink(hdfName, pathName, linkName)
%
% Inputs:
% hdfName char
% HDF5 file name
% pathName char
% HDF5 group path containing the link
% linkName char
% Name of the link
%
% Outputs:
% path char
% Path within HDF5 file of linked object
% ID H5ML.id
% Identifier of linked object
%
% See Also:
% h5tools.writelink, h5tools.getAllSoftlinks
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeHdfFile(hdfName)}
pathName char %{mustBeHdfPath(hdfName, pathName)}
linkName char
end
import h5tools.files.HdfTypes
fullPath = h5tools.util.buildPath(pathName, linkName);
info = h5info(hdfName, fullPath);
linkPath = info.Value{1};
if nargout == 2
fileID = h5tools.files.openFile(hdfName);
fileIDx = onCleanup(@()H5F.close(fileID));
objID = H5O.open(fileID, linkPath, 'H5P_DEFAULT');
objIDx = onCleanup(@()H5O.close(objID));
hdfType = h5tools.files.HdfTypes.get(objID);
if hdfType == HdfTypes.GROUP
ID = H5G.open(fileID, linkPath);
elseif hdfType == HdfTypes.DATASET
ID = H5D.open(fileID, linkPath);
end
end
@@ -0,0 +1,44 @@
function success = write(hdfName, pathName, dsetName, data, varargin)
% Write data to a HDF5 dataset
%
% Description:
% Write an HDF5 dataset at the specified path
%
% Syntax:
% success = h5tools.write(hdfName, pathName, dsetName, data, varargin)
%
% Inputs:
% hdfName char or H5ML.id
% HDF5 file name or identifier
% pathName char
% HDF5 path to group where dataset will be written
% dsetName char
% Name of the dataset
% data
% Data to be written
% varargin
% Optional inputs to h5create
%
% Outputs:
% success logical
% Optional flag for writing success
%
% Examples:
% % Write a dataset named 'D1' in group '/G1'
% h5tools.write('Test.h5', '/G1', 'D1', 1:3);
%
% See also:
% h5tools.datasets.writeDatasetByType
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
% Create the group, if it doesn't exist
if ~h5tools.exist(hdfName, pathName)
newGroup = h5tools.util.getPathEnd(pathName);
parent = h5tools.util.getPathParent(pathName);
h5tools.createGroup(hdfName, parent, newGroup);
end
success = h5tools.datasets.writeDatasetByType(...
hdfName, pathName, dsetName, data, varargin{:});
@@ -0,0 +1,78 @@
function writeatt(hdfName, pathName, varargin)
% Write HDF5 attributes
%
% Description:
% Writes one or more attributes to an HDF5 group/dataset
%
% Syntax:
% h5tools.writeatt(hdfFile, hdfPath, varargin)
%
% Inputs:
% hdfFile char
% HDF5 file name
% hdfPath char
% Path of the dataset/group where attributes will be written
% varargin see below
%
% Outputs:
% N/A
%
% Examples:
% Attributes can be specified in the following ways:
% 1. A list of one or more key/value pairs
% h5tools.writeatt(hdfFile, hdfPath, 'A', 1)
% h5tools.writeatt(hdfFile, hdfPath, 'A', 1, 'B', 2)
% 2. A containers.Map
% attMap = containers.Map('A', 1, 'B', 2)
% h5tools.writeatt(hdfFile, hdfPath, attMap)
% 3. A structure
% attStruct = struct('A', 1, 'B', 2)
% h5tools.writeatt(hdfFile, hdfPath, attStruct)
% 4. A structure or containers.Map followed by key/value pairs
% h5tools.writeatt(hdfFile, hdfPath, attMap, 'C', 3, 'D', 4)
% h5tools.writeatt(hdfFile, hdfPath, attStruct, 'C', 3, 'D', 4)
%
% See Also:
% h5tools.readatt, h5readatt
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName char {mustBeFile(hdfName)}
pathName char
end
arguments (Repeating)
varargin
end
if nargin == 2 || isempty(varargin{1})
warning('writeatt:NoInput',...
'No inputs provided so no attributes written');
return
end
% Parse the inputs
if isa(varargin{1}, 'containers.Map')
attMap = varargin{1};
if nargin > 3
attMap = addMaps(attMap, kv2map(varargin{2:end}));
end
elseif isstruct(varargin{1})
attMap = struct2map(varargin{1});
if nargin > 3
attMap = addMaps(attMap, kv2map(varargin{2:end}));
end
else
attMap = kv2map(varargin{:});
%error('writeatt:InvalidInput',...
% 'A single input specifying attributes must be containers.Map or struct');
end
% Write the attributes
k = attMap.keys;
for i = 1:numel(k)
h5tools.attributes.writeAttributeByType(...
hdfName, pathName, k{i}, attMap(k{i}));
end
@@ -0,0 +1,47 @@
function writelink(hdfName, linkPath, linkName, targetPath)
% Write an HDF5 soft link
%
% Description:
% Creates a soft link dataset within HDF5 group linking to another
% dataset or group within the HDF5 file.
%
% Syntax:
% h5tools.writelink(hdfName, linkPath, linkName, targetPath)
%
% Inputs:
% hdfName char or H5ML.id
% The file name of an HDF5 file or the identifier
% linkPath
% The HDF5 path to the group where the link will be written
% linkName
% The name of the link
% targetPath char
% The HDF5 path of the group or dataset that is the link target
%
% Outputs:
% N/A
%
% See Also:
% h5tools.collectSoftlinks, h5tools.readlink
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
hdfName {mustBeHdfFile(hdfName)}
linkPath {mustBeHdfPath(hdfName, linkPath)}
linkName char
targetPath {mustBeHdfPath(hdfName, targetPath)}
end
if isa(hdfName, 'H5ML.id')
fileID = hdfName;
else
fileID = h5tools.files.openFile(hdfName, false);
fileIDx = onCleanup(@()H5F.close(fileID));
end
% Open the group where the link will be written
linkID = H5G.open(fileID, linkPath);
linkIDx = onCleanup(@()H5G.close(linkID));
H5L.create_soft(targetPath, linkID, linkName, 'H5P_DEFAULT', 'H5P_DEFAULT');
@@ -0,0 +1,8 @@
classdef EnumClass
enumeration
GROUPONE
GROUPTWO
GROUPTHREE
end
end
@@ -0,0 +1,35 @@
function deleteH5toolsTestFiles()
% Deletes test suite .h5 files
%
% Description:
% Deletes all the HDF5 files created by the test suite
%
% Syntax:
% test.deleteH5ToolsTestFiles()
%
% See Also:
% runH5toolsTestSuite
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
% Suppress warnings for when the file doesn't exist
warning('off', 'MATLAB:DELETE:FileNotFound');
testPath = fileparts(fileparts(mfilename('fullpath')));
delete(fullfile(testPath, 'AttributeTest.h5'));
delete(fullfile(testPath, 'AttributesTest.h5'));
delete(fullfile(testPath, 'Collection.h5'));
delete(fullfile(testPath, 'CopyTest1.h5'));
delete(fullfile(testPath, 'CopyTest2.h5'));
delete(fullfile(testPath, 'Dataset.h5'));
delete(fullfile(testPath, 'File.h5'));
delete(fullfile(testPath, 'GroupTest.h5'));
delete(fullfile(testPath, 'EnumClassTest.h5'));
delete(fullfile(testPath, 'LinkTest.h5'));
delete(fullfile(testPath, 'MoveTest.h5'));
delete(fullfile(testPath, 'Test.h5'));
delete(fullfile(testPath, 'ValidatorTest.h5'));
% Restore the normal warning settings
warning('on', 'MATLAB:DELETE:FileNotFound');
@@ -0,0 +1,16 @@
function verifyDatesEqual(testCase, actual, expected)
% VERIFYDATESEQUAL
%
% Syntax:
% verifyDatesEqual(testCase, actual, expected)
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
testCase.verifyEqual(actual.Year, expected.Year);
testCase.verifyEqual(actual.Month, expected.Month);
testCase.verifyEqual(actual.Day, expected.Day);
testCase.verifyEqual(actual.Hour, expected.Hour);
testCase.verifyEqual(actual.Minute, expected.Minute);
testCase.verifyEqual(actual.Second, expected.Second, 'AbsTol', 0.5);
testCase.verifyEqual(actual.TimeZone, expected.TimeZone);
@@ -0,0 +1,227 @@
classdef AttributeTest < matlab.unittest.TestCase
% ATTRIBUTETEST
%
% Description:
% Tests HDF5 operations involving attributes
%
% Use:
% results = runtests('AttributeTest');
%
% See also;
% runH5toolsTestSuite
% -------------------------------------------------------------------------
properties
FILE
end
methods (TestClassSetup)
function createFile(testCase)
folder = fileparts(mfilename('fullpath'));
testCase.FILE = fullfile(folder, 'AttributeTest.h5');
h5tools.createFile(testCase.FILE, true);
% Create two groups for map and struct input testing
h5tools.createGroup(testCase.FILE, '/',...
'Map', 'Struct', 'MapKV', 'StructKV');
% Create a dataset for group vs dataset testing
h5tools.write(testCase.FILE, '/', 'Dataset', magic(5));
end
end
methods (Test, TestTags=["Inputs"])
function ContainerInput(testCase)
map = containers.Map();
map('Att1') = 1;
map('Att2') = "a";
h5tools.writeatt(testCase.FILE, '/Map', map);
out = h5tools.getAttributeNames(testCase.FILE, '/Map');
testCase.verifyNumElements(out, 2);
end
function ContainerKVInput(testCase)
map = containers.Map();
map('A') = 1; map('B') = 'b';
h5tools.writeatt(testCase.FILE, '/MapKV', map, 'C', 1:3);
out = h5tools.getAttributeNames(testCase.FILE, '/MapKV');
testCase.verifyNumElements(out, 3);
end
function Struct(testCase)
S = struct('Att1', 1, 'Att2', "a");
h5tools.writeatt(testCase.FILE, '/Struct', S);
out = h5tools.getAttributeNames(testCase.FILE, '/Struct');
testCase.verifyNumElements(out, 2);
end
function NoInput(testCase)
testCase.verifyWarning(...
@() h5tools.writeatt(testCase.FILE, '/'),...
"writeatt:NoInput");
end
end
methods (Test, TestTags=["Deletion"])
function DeletionFromGroup(testCase)
import matlab.unittest.constraints.Throws
h5tools.writeatt(testCase.FILE, '/', 'ToBeDeleted', "sara");
h5tools.deleteAttribute(testCase.FILE, '/', 'ToBeDeleted');
testCase.verifyThat(...
@() h5tools.readatt(testCase.FILE, '/', 'ToBeDeleted'),...
Throws('MATLAB:imagesci:hdf5lib:libraryError'));
end
function DeletionFromDataset(testCase)
import matlab.unittest.constraints.Throws
h5tools.writeatt(testCase.FILE, '/Dataset', 'ToBeDeleted', "sara");
h5tools.deleteAttribute(testCase.FILE, '/Dataset', 'ToBeDeleted');
testCase.verifyThat(...
@() h5tools.readatt(testCase.FILE, '/', 'ToBeDeleted'),...
Throws('MATLAB:imagesci:hdf5lib:libraryError'));
end
end
methods (Test, TestTags=["Numeric"])
function DoubleArray(testCase)
input = magic(5);
h5tools.writeatt(testCase.FILE, '/', 'DoubleArray', input);
output = h5tools.readatt(testCase.FILE, '/', 'DoubleArray');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags=["String"])
function ScalarString(testCase)
input = "test";
h5tools.writeatt(testCase.FILE, '/', 'ScalarString', input);
output = h5tools.readatt(testCase.FILE, '/', 'ScalarString');
testCase.verifyEqual(output, input);
end
function ScalarVector(testCase)
input = ["test", "string", "row"];
h5tools.writeatt(testCase.FILE, '/', 'StringArrayRow', input);
output = h5tools.readatt(testCase.FILE, '/', 'StringArrayRow');
testCase.verifyEqual(output, input');
h5tools.writeatt(testCase.FILE, '/', 'StringArrayCol', input');
output = h5tools.readatt(testCase.FILE, '/', 'StringArrayCol');
testCase.verifyEqual(output, input');
end
end
methods (Test, TestTags = ["Logical"])
function Logical(testCase)
h5tools.writeatt(testCase.FILE, '/', 'LogicalTrue', true);
testCase.verifyTrue(h5tools.readatt(testCase.FILE, '/', 'LogicalTrue'));
h5tools.writeatt(testCase.FILE, '/', 'LogicalFalse', false);
testCase.verifyFalse(h5tools.readatt(testCase.FILE, '/', 'LogicalFalse'));
end
function LogicalVector(testCase)
input = [true, false, true, false];
h5tools.writeatt(testCase.FILE, '/', 'LogicalArrayRow', input);
output = h5tools.readatt(testCase.FILE, '/', 'LogicalArrayRow');
testCase.verifyEqual(output, input);
h5tools.writeatt(testCase.FILE, '/', 'LogicalArrayCol', input');
output = h5tools.readatt(testCase.FILE, '/', 'LogicalArrayCol');
testCase.verifyEqual(output, input');
end
function Logical2D(testCase)
input = [true false; true true];
h5tools.writeatt(testCase.FILE, '/', 'Logical2D', input);
output = h5tools.readatt(testCase.FILE, '/', 'Logical2D');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags = ["char"])
function Char(testCase)
input = 'test';
h5tools.writeatt(testCase.FILE, '/', 'Char', input);
output = h5tools.readatt(testCase.FILE, '/', 'Char');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags=["enum"])
function EnumScalar(testCase)
input = test.EnumClass.GROUPONE;
h5tools.writeatt(testCase.FILE, '/', 'EnumScalar', input);
output = h5tools.readatt(testCase.FILE, '/', 'EnumScalar');
testCase.verifyEqual(output, input);
end
function LooksLikeEnum(testCase)
input = "FakeClass.GROUPONE";
h5tools.writeatt(testCase.FILE, '/', 'LooksLikeEnum', input);
output = h5tools.readatt(testCase.FILE, '/', 'LooksLikeEnum');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags = ["datetime"])
function DatetimeScalar(testCase)
input = datetime('24-Oct-2014 12:45:07');
h5tools.writeatt(testCase.FILE, '/', 'DatetimeScalar', input);
output = h5tools.readatt(testCase.FILE, '/', 'DatetimeScalar');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags=["duration", "unreadable"])
function Duration(testCase)
input = seconds(magic(5))';
testCase.verifyWarning(...
@() h5tools.writeatt(testCase.FILE, '/', 'Duration', input),...
"writeAttributeByType:Duration");
output = h5tools.readatt(testCase.FILE, '/', 'Duration');
testCase.verifyClass(output, "double");
testCase.verifyEqual(seconds(output), input);
end
end
methods (Test, TestTags=["cellstr", "unreadable"])
function Cellstr(testCase)
input = {'abc', 'def'; 'ghi', 'klm'};
testCase.verifyWarning(...
@() h5tools.writeatt(testCase.FILE, '/', 'Cellstr', input),...
"writeAttributeByType:Cellstr");
output = h5tools.readatt(testCase.FILE, '/', 'Cellstr');
testCase.verifyClass(output, "string");
testCase.verifyEqual(cellstr(output), input);
end
end
methods (Test, TestTags=["invalid"])
function Table(testCase)
import matlab.unittest.constraints.Throws
input = struct("A", 1, "B", 2);
testCase.verifyThat(...
@()h5tools.writeatt(testCase.FILE, '/', 'Struct', input),...
Throws("writeAttributeByType:InvalidInput"));
end
end
methods (Test, TestTags=["Deletion"])
function Deletion(testCase)
% Write an attribute and confirm presence in attribute list
h5tools.writeatt(testCase.FILE, '/', 'ToDelete', "deleteme");
allAtts = h5tools.getAttributeNames(testCase.FILE, '/');
testCase.verifyNumElements(allAtts(endsWith(allAtts, "ToDelete")), 1);
% Delete the attribute and confirm the attribute is gone
h5tools.deleteAttribute(testCase.FILE, '/', 'ToDelete');
allAtts = h5tools.getAttributeNames(testCase.FILE, '/');
testCase.verifyEmpty(allAtts(endsWith(allAtts, "ToDelete")), 1);
end
end
end
@@ -0,0 +1,103 @@
classdef AttributesTest < matlab.unittest.TestCase
% ATTRIBUTESTEST
%
% Description:
% Tests reading multiple attributes at once. Because test order is not
% guarenteed in MATLAB and this test requires knowing the exact number of
% attributes an entity has, it needs to be separate from AttributeTest.
%
% Use:
% results = runtests('AttributesTest.m')
%
% See also:
% runH5toolsTestSuite
% -------------------------------------------------------------------------
properties
FILE
INPUT
end
methods (TestClassSetup)
function createFile(testCase)
folder = fileparts(mfilename('fullpath'));
testCase.FILE = fullfile(folder, 'AttributesTest.h5');
h5tools.createFile(testCase.FILE, true);
% Create a group and a dataset to support testing
h5tools.createGroup(testCase.FILE, '/', 'G1');
h5tools.write(testCase.FILE, '/', 'D1', magic(5));
% Create a diverse set of attributes, write to dataset & group
testCase.INPUT = struct('A', [1; 2; 3], 'B', "test",...
'C', 'test', 'D', ["test1"; "test2"]);
h5tools.writeatt(testCase.FILE, '/G1', testCase.INPUT);
h5tools.writeatt(testCase.FILE, '/D1', testCase.INPUT);
end
end
methods (Test, TestTags=["Group"])
function Group(testCase)
% Test accessing just the attribute names
attNames = h5tools.getAttributeNames(testCase.FILE, '/G1');
testCase.verifyNumElements(attNames, 4);
testCase.verifyEmpty(setdiff(attNames, ["A","B","C","D"]));
% Read a single attribute
A = h5tools.readatt(testCase.FILE, '/G1', 'A');
testCase.verifyEqual(A, testCase.INPUT.A);
% Read two attributes, return two outputs
[A, B] = h5tools.readatt(testCase.FILE, '/G1', 'A','B');
testCase.verifyEqual(A, testCase.INPUT.A);
testCase.verifyEqual(B, testCase.INPUT.B);
% Read two attributes, return containers.Map
map1 = h5tools.readatt(testCase.FILE, '/G1', 'A','B');
testCase.verifyEqual(map1.Count, uint64(2));
testCase.verifyEqual(map1('A'), testCase.INPUT.A);
testCase.verifyEqual(map1('B'), testCase.INPUT.B);
% Read all attributes, return containers.Map
map2 = h5tools.readatt(testCase.FILE, '/G1', 'all');
testCase.verifyEqual(map2.Count, uint64(4));
testCase.verifyEqual(map2('A'), testCase.INPUT.A);
testCase.verifyEqual(map2('B'), testCase.INPUT.B);
testCase.verifyEqual(map2('C'), testCase.INPUT.C);
testCase.verifyEqual(map2('D'), testCase.INPUT.D);
end
end
methods (Test, TestTags=["Dataset"])
function Dataset(testCase)
% Test accessing just the attribute names
attNames = h5tools.getAttributeNames(testCase.FILE, '/D1');
testCase.verifyNumElements(attNames, 5);
testCase.verifyEmpty(setdiff(attNames, ["A","B","C","D", "Class"]));
% Dataset has one extra, the MATLAB Class ("Class")
% Read a single attribute
A = h5tools.readatt(testCase.FILE, '/D1', 'A');
testCase.verifyEqual(A, testCase.INPUT.A);
% Read two attributes, return two outputs
[A, B] = h5tools.readatt(testCase.FILE, '/D1', 'A','B');
testCase.verifyEqual(A, testCase.INPUT.A);
testCase.verifyEqual(B, testCase.INPUT.B);
% Read two attributes, return containers.Map
map1 = h5tools.readatt(testCase.FILE, '/D1', 'A','B');
testCase.verifyEqual(map1.Count, uint64(2));
testCase.verifyEqual(map1('A'), testCase.INPUT.A);
testCase.verifyEqual(map1('B'), testCase.INPUT.B);
% Read all attributes, return containers.Map
map2 = h5tools.readatt(testCase.FILE, '/D1', 'all');
testCase.verifyEqual(map2.Count, uint64(5));
testCase.verifyEqual(map2('A'), testCase.INPUT.A);
testCase.verifyEqual(map2('B'), testCase.INPUT.B);
testCase.verifyEqual(map2('C'), testCase.INPUT.C);
testCase.verifyEqual(map2('D'), testCase.INPUT.D);
end
end
end
@@ -0,0 +1,406 @@
classdef DatasetTest < matlab.unittest.TestCase
% HDFTEST
%
% Description:
% Tests MATLAB datatype I/O to HDF5
%
% Parent:
% matlab.unittest.TestCase
%
% Use:
% result = runtests('DatasetTest.m')
%
% Datatypes tested
% - double (scalar, 2D, 3D)
% - uint8 (scalar)
% - logical (true, false)
% - string (scalar, 2D)
% - char
% - enum
%
% See also:
% runH5toolsTestSuite
% -------------------------------------------------------------------------
properties
FILE = [fullfile(cd, 'Dataset.h5')];
end
methods (TestClassSetup)
function createFile(testCase)
h5tools.createFile(testCase.FILE, true);
end
end
% MATLAB text datatypes
methods (Test, TestTags=["String", "Text"])
function StringScalar(testCase)
input = "abc";
h5tools.write(testCase.FILE, '/', 'StringScalar', input);
output = h5tools.read(testCase.FILE, '/', 'StringScalar');
testCase.verifyEqual(output, input);
end
function StringArray(testCase)
input = ["abc", "def", "ghi"];
h5tools.write(testCase.FILE, '/', 'StringArray', input);
output = h5tools.read(testCase.FILE, '/', 'StringArray');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags=["Char", "Text"])
function Char(testCase)
input = 'abcdefghi';
h5tools.write(testCase.FILE, '/', 'Char', input);
output = h5tools.read(testCase.FILE, '/', 'Char');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags=["CellStr", "Text"])
function CellstrRow(testCase)
input = {'abc', 'def', 'ghi'};
h5tools.write(testCase.FILE, '/', 'CellstrRow', input);
output = h5tools.read(testCase.FILE, '/', 'CellstrRow');
testCase.verifyEqual(output, input);
end
function CellstrCol(testCase)
input = {'abc', 'def', 'ghi'}';
h5tools.write(testCase.FILE, '/', 'CellstrCol', input);
output = h5tools.read(testCase.FILE, '/', 'CellstrCol');
testCase.verifyEqual(output, input);
end
function CellstrScalar(testCase)
input = {'abc'};
h5tools.write(testCase.FILE, '/', 'CellstrScalar', input);
output = h5tools.read(testCase.FILE, '/', 'CellstrScalar');
testCase.verifyEqual(output, input);
end
function Cellstr2D(testCase)
input = {'abc', 'def', 'ghi'; 'jkl', 'mno', 'pqr'};
h5tools.write(testCase.FILE, '/', 'Cellstr2D', input);
output = h5tools.read(testCase.FILE, '/', 'Cellstr2D');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags={'Boolean'})
function LogicalScalar(testCase)
h5tools.datasets.makeLogicalDataset(testCase.FILE, '/', 'LogicalFalse', 0);
testCase.verifyFalse(h5tools.datasets.readEnumTypeDataset(...
testCase.FILE, '/', 'LogicalFalse'));
h5tools.datasets.makeLogicalDataset(testCase.FILE, '/', 'LogicalTrue', 1);
testCase.verifyTrue(h5tools.datasets.readEnumTypeDataset(...
testCase.FILE, '/', 'LogicalTrue'));
end
function Logical2D(testCase)
input = [false, true, false; true, true, false];
h5tools.datasets.makeLogicalDataset(testCase.FILE, '/', 'Logical2D', input);
output = h5tools.datasets.readEnumTypeDataset(testCase.FILE, '/', 'Logical2D');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags=["Datetime"])
function DatetimeScalar(testCase)
input = datetime('now', 'Format', 'yyyyMMdd');
h5tools.write(testCase.FILE, '/', 'Datetime', input);
output = h5tools.read(testCase.FILE, '/', 'Datetime');
testCase.verifyTrue(isdatetime(output));
test.verifyDatesEqual(testCase, output, input);
end
function DatetimeVector(testCase)
input = datetime('now', 'Format', 'yyyyMMdd');
input = repmat(input, [2, 1]);
h5tools.write(testCase.FILE, '/', 'DatetimeColumn', input);
output = h5tools.read(testCase.FILE, '/', 'DatetimeColumn');
testCase.verifyTrue(isdatetime(output));
test.verifyDatesEqual(testCase, output, input);
end
end
methods (Test, TestTags=["Numeric", "Float"])
function DoubleScalar(testCase)
input = 2;
h5tools.write(testCase.FILE, '/', 'DoubleScalar', input);
output = h5tools.read(testCase.FILE, '/', 'DoubleScalar');
testCase.verifyEqual(output, input);
end
function Double2D(testCase)
input = [1.5 2.5 3.5; 4.2 3.2 1.2];
h5tools.write(testCase.FILE, '/', 'Double2D', input);
output = h5tools.read(testCase.FILE, '/', 'Double2D');
testCase.verifyEqual(output, input);
end
function Double3D(testCase)
input = cat(3, [1 2 3; 4 5 6], [7 8 9; 10 11 12]);
h5tools.write(testCase.FILE, '/', 'Double3D', input);
output = h5tools.read(testCase.FILE, '/', 'Double3D');
testCase.verifyEqual(output, input);
end
function SingleArray(testCase)
input = single(magic(5));
%input = double(magic(5));
h5tools.write(testCase.FILE, '/', 'SingleArray', input);
output = h5tools.read(testCase.FILE, '/', 'SingleArray');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags=["Numeric", "Integer"])
function Uint8Scalar(testCase)
input = uint8(1);
h5tools.write(testCase.FILE, '/', 'UInt8', input);
output = h5tools.read(testCase.FILE, '/', 'UInt8');
testCase.verifyEqual(output, input);
end
function Uint8Array(testCase)
input = uint8(magic(5));
h5tools.write(testCase.FILE, '/', 'UInt8Array', input);
output = h5tools.read(testCase.FILE, '/', 'UInt8Array');
testCase.verifyEqual(output, input);
end
function Int8Array(testCase)
input = int8(magic(5));
h5tools.write(testCase.FILE, '/', 'Int8Array', input);
output = h5tools.read(testCase.FILE, '/', 'Int8Array');
testCase.verifyEqual(output, input);
end
function Int16Array(testCase)
input = int16(magic(5));
h5tools.write(testCase.FILE, '/', 'Int16Array', input);
output = h5tools.read(testCase.FILE, '/', 'Int16Array');
testCase.verifyEqual(output, input);
end
function UInt16Array(testCase)
input = uint16(magic(5));
h5tools.write(testCase.FILE, '/', 'UInt16Array', input);
output = h5tools.read(testCase.FILE, '/', 'UInt16Array');
testCase.verifyEqual(output, input);
end
function Int32Array(testCase)
input = int32(magic(5));
h5tools.write(testCase.FILE, '/', 'Int32Array', input);
output = h5tools.read(testCase.FILE, '/', 'Int32Array');
testCase.verifyEqual(output, input);
end
function UInt32Array(testCase)
input = uint32(magic(5));
h5tools.write(testCase.FILE, '/', 'UInt32Array', input);
output = h5tools.read(testCase.FILE, '/', 'UInt32Array');
testCase.verifyEqual(output, input);
end
function Int64Array(testCase)
input = int64(magic(5));
h5tools.write(testCase.FILE, '/', 'Int64Array', input);
output = h5tools.read(testCase.FILE, '/', 'Int64Array');
testCase.verifyEqual(output, input);
end
function UInt64Array(testCase)
input = uint64(magic(5));
h5tools.write(testCase.FILE, '/', 'UInt64Array', input);
output = h5tools.read(testCase.FILE, '/', 'UInt64Array');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags={'Struct'})
function ScalarStruct(testCase)
input = struct('A', 1, 'B', 2);
h5tools.write(testCase.FILE, '/', 'ScalarStruct', input);
output = h5tools.read(testCase.FILE, '/', 'ScalarStruct');
testCase.verifyEqual(output, input);
end
function ScalarStructMultiType(testCase)
input = struct('A', 1, 'B', "hello");
h5tools.write(testCase.FILE, '/', 'ScalarStructMulti', input);
output = h5tools.read(testCase.FILE, '/', 'ScalarStructMulti');
testCase.verifyEqual(output, input);
end
function UnequalStruct(testCase)
import matlab.unittest.constraints.Throws
input = struct('A', (1:3)', 'B', 2);
h5tools.write(testCase.FILE, '/', 'UnequalStruct', input);
output = h5tools.read(testCase.FILE, '/', 'UnequalStruct');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags=["Table"])
function testTable(testCase)
input = table(...
rangeCol(1,4), {'a'; 'b'; 'c'; 'd'}, ["a", "b", "c", "d"]', uint8(1:4)',...
'VariableNames', {'Numbers', 'Characters', 'Strings', 'Integers'});
h5tools.write(testCase.FILE, '/', 'Table', input);
output = h5tools.read(testCase.FILE, '/', 'Table');
testCase.verifyEqual(output, input);
end
function testTableIntegers(testCase)
input = table(...
int8(1:3)', int16(1:3)', int32(1:3)', int64(1:3)',...
'VariableNames', {'Int8', 'Int16', 'Int32', 'Int64'});
h5tools.write(testCase.FILE, '/', 'IntTable', input);
output = h5tools.read(testCase.FILE, '/', 'IntTable');
testCase.verifyEqual(output, input);
input = table(...
single(1:3)', uint16(1:3)', uint32(1:3)', uint64(1:3)',...
'VariableNames', {'Int8', 'Int16', 'Int32', 'Int64'});
h5tools.write(testCase.FILE, '/', 'IntTable2', input);
output = h5tools.read(testCase.FILE, '/', 'IntTable2');
testCase.verifyEqual(output, input);
end
function testTableMetadata(testCase)
input = table((1:3)', (4:6)',...
'RowNames', {'A', 'B', 'C'});
input.Properties.VariableUnits = {'pA', 'mV'};
h5tools.write(testCase.FILE, '/', 'TableWithMetadata', input);
output = h5tools.read(testCase.FILE, '/', 'TableWithMetadata');
testCase.verifyEqual(input, output);
assignin('base', 'tablemetadata', output);
end
function testTimeTable(testCase)
input = timetable(...
seconds(1:4)', {'a'; 'b'; 'c'; 'd'}, ["a", "b", "c", "d"]',...
'VariableNames', {'Characters', 'Strings'});
h5tools.write(testCase.FILE, '/', 'TimeTable', input);
output = h5tools.read(testCase.FILE, '/', 'TimeTable');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags={'Enum'})
function EnumScalar(testCase)
input = test.EnumClass.GROUPTWO;
h5tools.write(testCase.FILE, '/', 'EnumScalar', input);
output = h5tools.read(testCase.FILE, '/', 'EnumScalar');
testCase.verifyEqual(output, input);
end
function EnumArray(testCase)
input = [test.EnumClass.GROUPONE, test.EnumClass.GROUPONE; ...
test.EnumClass.GROUPTWO, test.EnumClass.GROUPTHREE];
h5tools.write(testCase.FILE, '/', 'EnumArray', input);
output = h5tools.read(testCase.FILE, '/', 'EnumArray');
testCase.verifyEqual(output, input);
end
function EnumOffPath(testCase)
import matlab.unittest.constraints.Throws
import matlab.unittest.constraints.IssuesWarnings
% fakeEnum = 'NotAnEnum.GROUPONE';
h5tools.datasets.makeCharDataset(testCase.FILE, '/', 'EnumOffPath', 'GROUPONE');
h5tools.writeatt(testCase.FILE, '/EnumOffPath', 'Class', 'enum',...
'EnumClass', 'NotAnEnum');
testCase.verifyThat(...
@() h5tools.datasets.readDatasetByType(testCase.FILE, '/', 'EnumOffPath'),...
IssuesWarnings("read:UnknownEnumerationClass"));
end
function EnumBadType(testCase)
import matlab.unittest.constraints.Throws
import matlab.unittest.constraints.IssuesWarnings
% fakeEnum = 'EnumClass.GROUPFOUR';
h5tools.datasets.makeCharDataset(testCase.FILE, '/', 'EnumBadType', 'GROUPFOUR');
h5tools.writeatt(testCase.FILE, '/EnumBadType', 'Class', 'enum',...
'EnumClass', class(test.EnumClass.GROUPONE));
testCase.verifyThat(...
@() h5tools.datasets.readDatasetByType(testCase.FILE, '/', 'EnumBadType'),...
IssuesWarnings("read:UnknownEnumerationType"));
end
end
methods (Test, TestTags={'Duration'})
function Seconds2D(testCase)
input = seconds(1:5);
h5tools.write(testCase.FILE, '/', 'Seconds2D', input);
output = h5tools.read(testCase.FILE, '/', 'Seconds2D');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags=["Misc"])
function Imref2d(testCase)
input = imref2d([242, 360]);
h5tools.write(testCase.FILE, '/', 'Imref2d', input);
output = h5tools.read(testCase.FILE, '/', 'Imref2d');
testCase.verifyEqual(output, input);
end
function Simtform2d(testCase)
input = simtform2d(3, 30, [10 20.5]);
h5tools.write(testCase.FILE, '/', 'Simtform2d', input);
output = h5tools.read(testCase.FILE, '/', 'Simtform2d');
testCase.verifyEqual(output, input);
end
function Affine2d(testCase)
input = affine2d(eye(3));
h5tools.write(testCase.FILE, '/', 'Affine2d', input);
output = h5tools.read(testCase.FILE, '/', 'Affine2d');
testCase.verifyEqual(output, input);
end
end
methods (Test, TestTags=["Group"])
function AutoCreateGroup(testCase)
% Test auto creation of parent group
h5tools.write(testCase.FILE, '/NewGroup', 'Dataset', magic(5));
groupNames = h5tools.collectGroups(testCase.FILE);
testCase.verifyEqual(nnz(endsWith(groupNames, 'NewGroup')), 1);
end
end
methods (Test, TestTags=["Dataset", "Numeric", "Append"])
function AppendToDataset(testCase)
% Two datasets [N x 10]
data1 = randn(3, 10); data2 = randn(2, 10);
% Write the first dataset
h5tools.write(testCase.FILE, '/', 'DatasetToAppend', data1,...
[Inf, size(data1,2)], 'ChunkSize', [1, size(data1, 2)]);
% Append the second dataset
h5tools.write(testCase.FILE, '/', 'DatasetToAppend', data2);
output = h5tools.read(testCase.FILE, '/', 'DatasetToAppend');
testCase.verifyEqual(output, [data1; data2]);
end
end
methods (Test, TestTags=["Dataset", "Deletion"])
function DeleteDataset(testCase)
% Create a dataset and then delete it
h5tools.write(testCase.FILE, '/', 'DatasetToDelete', magic(5));
testCase.verifyTrue(ismember("/DatasetToDelete", h5tools.collectDatasets(testCase.FILE)));
h5tools.deleteObject(testCase.FILE, '/DatasetToDelete');
testCase.verifyFalse(ismember("/DatasetToDelete", h5tools.collectDatasets(testCase.FILE)));
end
end
end
@@ -0,0 +1,48 @@
classdef EnumClassTest < matlab.unittest.TestCase
properties
FILE = fullfile(fileparts(mfilename("fullpath")), "EnumClassTest.h5")
end
methods (TestClassSetup)
function createFile(testCase)
testCase.FILE
% Create the file and add some basic components
h5tools.createFile(testCase.FILE, true);
h5tools.createGroup(testCase.FILE, '/', 'GroupOne', 'GroupTwo');
h5tools.write(testCase.FILE, '/', 'DatasetOne', magic(5));
h5tools.write(testCase.FILE, '/', 'DatasetTwo', uint8(magic(5)));
h5tools.writeatt(testCase.FILE, '/', 'AttOne', "test");
end
end
methods (Test)
function TypeFromID(testCase)
import h5tools.files.HdfTypes
fileID = h5tools.files.openFile(testCase.FILE);
fileIDx = @()onCleanup(H5F.close(fileID));
testCase.verifyEqual(HdfTypes.get(fileID), HdfTypes.FILE);
groupID = H5G.open(fileID, '/GroupOne');
groupIDx = @()onCleanup(H5G.close(groupID));
testCase.verifyEqual(HdfTypes.get(groupID), HdfTypes.GROUP);
dsetID = H5D.open(fileID, '/DatasetOne');
dsetIDx = @()onCleanup(H5D.close(dsetID));
testCase.verifyEqual(HdfTypes.get(dsetID), HdfTypes.DATASET);
spaceID = H5D.get_space(dsetID);
spaceIDx = @()onCleanup(H5S.close(spaceID));
testCase.verifyEqual(HdfTypes.get(spaceID), HdfTypes.DATASPACE);
typeID = H5D.get_type(dsetID);
typeIDx = @()onCleanup(H5T.close(typeID));
testCase.verifyEqual(HdfTypes.get(typeID), HdfTypes.DATATYPE);
attID = H5A.open(fileID, 'AttOne');
attIDx = @()onCleanup(H5A.close(attID));
testCase.verifyEqual(HdfTypes.get(attID), HdfTypes.ATTR);
end
end
end
@@ -0,0 +1,81 @@
classdef FileTest < matlab.unittest.TestCase
properties
FOLDER
FILE
end
methods (TestClassSetup)
function setupFiles(testCase)
testCase.FOLDER = fileparts(mfilename('fullpath'));
testCase.FILE = fullfile(testCase.FOLDER, 'File.h5');
if exist(testCase.FILE, 'file')
delete(testCase.FILE);
end
end
end
methods (Test)
function copyFile(testCase)
h5tools.createFile('CopyTest1.h5', true);
% Test copying
h5tools.files.copyFile('CopyTest1.h5', 'CopyTest2');
testCase.verifyTrue(exist('CopyTest2.h5', 'file') > 0);
testCase.verifyError(...
@() h5tools.files.copyFile('CopyTest1.h5', 'CopyTest2.h5'),...
"copyFile:FileExists");
testCase.verifyError(...
@() h5tools.files.copyFile('CopyTest1.h5', 'CopyTest2.m'),...
"copyFile:InvalidExtension");
end
function makeFile(testCase)
import matlab.unittest.constraints.Throws
fileID = h5tools.createFile(testCase.FILE, false);
testCase.verifyTrue(isfile(testCase.FILE), 'file');
testCase.verifyClass(fileID, "H5ML.id");
% Force close of HDF5 file
H5F.close(fileID);
% Test for errors when file exists & overwrite is defaul (false)
testCase.verifyThat(...
@() h5tools.createFile(testCase.FILE),...
Throws("createFile:FileExists"));
% Overwrite existing file (no error)
h5tools.createFile(testCase.FILE, true);
% Look something in a group that doesn't exist
testCase.verifyFalse(h5tools.exist(testCase.FILE, '/GroupFour/Dset1'));
end
end
methods (Test)
function PathOrder(testCase)
% Single input
path = "/GroupOne/GroupTwo/Dataset";
testCase.verifyEqual(h5tools.util.getPathOrder(path), 3);
% Multiple inputs
paths = [path; "/GroupOne"];
testCase.verifyEqual(h5tools.util.getPathOrder(paths), [3, 1]');
end
function PathDividers(testCase)
testPath = '/GroupOne/GroupTwo/Dataset';
testCase.verifyEqual(h5tools.util.getPathEnd(testPath), 'Dataset');
testPath = '/GroupOne/GroupTwo/Dataset';
testCase.verifyEqual(h5tools.util.getPathParent(testPath),...
'/GroupOne/GroupTwo');
% Check special case: Group in root group
testCase.verifyEqual(h5tools.util.getPathParent('/GroupOne'), '/');
end
end
end
@@ -0,0 +1,71 @@
classdef GroupTest < matlab.unittest.TestCase
properties
FILE
end
methods (TestClassSetup)
function createFile(testCase)
testFolder = fileparts(mfilename('fullpath'));
testCase.FILE = fullfile(testFolder, 'GroupTest.h5');
% Create the file, overwriting if it already exists
h5tools.createFile(testCase.FILE, true);
% Test collect groups from fileID
fileID = h5tools.files.openFile(testCase.FILE);
fileIDx = onCleanup(@()H5F.close(fileID));
testCase.verifyEmpty(h5tools.collectGroups(fileID), 1);
end
end
% The order of test execution is not predictable so these must be
% tested all together
methods (Test, TestTags=["Group"])
function RootGroupIO(testCase)
out = h5tools.collectGroups(testCase.FILE);
testCase.verifyEmpty(out);
% Create a single group
h5tools.createGroup(testCase.FILE, '/', 'GroupOne');
out = h5tools.collectGroups(testCase.FILE);
testCase.verifyNumElements(out, 1);
% Create multiple groups
h5tools.createGroup(testCase.FILE, '/', 'GroupTwo', 'GroupThree');
out = h5tools.collectGroups(testCase.FILE);
testCase.verifyNumElements(out, 3);
% Check for warning when group already exists
testCase.verifyWarning(...
@() h5tools.createGroup(testCase.FILE, '/', 'GroupOne'),...
"createGroup:GroupExists");
end
function DatasetToExistingGroup(testCase)
h5tools.createGroup(testCase.FILE, '/', 'GroupFive');
% There should be no error here:
h5tools.write(testCase.FILE, '/GroupFour', 'Dataset', eye(3));
end
function SplitPath(testCase)
h5tools.createGroup(testCase.FILE, '/', 'GroupSix', 'GroupSix/GroupSixA');
[parentPath, endPath] = h5tools.util.splitPath('/GroupSix/GroupSixA');
testCase.verifyEqual(parentPath, '/GroupSix');
testCase.verifyEqual(endPath, 'GroupSixA');
end
end
methods (Test, TestTags=["Deletion", "Group"])
function DeleteGroup(testCase)
% Create the group
h5tools.createGroup(testCase.FILE, '/', 'GroupFour');
out = h5tools.collectGroups(testCase.FILE);
testCase.verifyTrue(ismember("/GroupFour", out));
% Delete the group
h5tools.deleteObject(testCase.FILE, '/GroupFour');
out = h5tools.collectGroups(testCase.FILE);
testCase.verifyFalse(ismember("/GroupFour", out));
end
end
end
@@ -0,0 +1,60 @@
classdef LinkTest < matlab.unittest.TestCase
% LINKTEST
%
% Description:
% Tests support for reading and writing HDF5 object references
%
% Use:
% results = runtests('LinkTest')
%
% See also:
% runH5toolsTestSuite
% -------------------------------------------------------------------------
properties
FILE = fullfile(fileparts(mfilename("fullpath")), 'LinkTest.h5')
end
methods (TestClassSetup)
function createFile(testCase)
h5tools.createFile(testCase.FILE, true);
% Add some misc groups and datasets for the links to use
h5tools.createGroup(testCase.FILE, '/', 'GroupOne', 'GroupTwo');
h5tools.createGroup(testCase.FILE, '/GroupOne', 'GroupOneA');
h5tools.write(testCase.FILE, '/GroupTwo', 'DatasetTwoA', magic(5));
h5tools.write(testCase.FILE, '/', 'DatasetOne', "test");
end
end
methods (Test)
function DatasetGroupLink(testCase)
h5tools.writelink(testCase.FILE, '/', 'LinkOne',...
'/GroupTwo/DatasetTwoA');
testCase.verifyNumElements(...
h5tools.collectSoftlinks(testCase.FILE), 1);
% Read the link
[path, objID] = h5tools.readlink(testCase.FILE, '/', 'LinkOne');
objIDx = onCleanup(@()H5D.close(objID));
testCase.verifyEqual(path, '/GroupTwo/DatasetTwoA');
testCase.verifyClass(objID, 'H5ML.id');
% Test from file ID
fileID = h5tools.files.openFile(testCase.FILE);
fileIDx = onCleanup(@()H5F.close(fileID));
testCase.verifyNumElements(...
h5tools.collectSoftlinks(fileID), 1);
end
end
methods (Test, TestTags=["Error"])
function InvalidTarget(testCase)
import matlab.unittest.constraints.Throws
testCase.verifyThat(...
@() h5tools.writelink(testCase.FILE, '/', 'LinkTwo', '/GroupThree'),...
Throws("HdfPath:InvalidPath"));
end
end
end
@@ -0,0 +1,38 @@
classdef MoveTest < matlab.unittest.TestCase
properties
FILE
end
methods (TestClassSetup)
function createFile(testCase)
testFolder = fileparts(mfilename('fullpath'));
testCase.FILE = fullfile(testFolder, 'MoveTest.h5');
% Create the file, overwriting if it already exists
h5tools.createFile(testCase.FILE, true);
% Create some groups
h5tools.createGroup(testCase.FILE, '/', 'Group1', 'Group2');
h5tools.createGroup(testCase.FILE, '/Group1', 'Group1A');
% Create some datasets
h5tools.write(testCase.FILE, '/Group1', 'Dataset1A', "abc");
h5tools.write(testCase.FILE, '/Group2', 'Dataset2A', "def");
h5tools.write(testCase.FILE, '/Group1/Group1A', 'Dataset1A1', "ghi");
end
end
methods (Test)
function moveGroup(testCase)
h5tools.move(testCase.FILE, '/Group1/Group1A', '/Group3');
groupNames = h5tools.collectGroups(testCase.FILE);
dsetNames = h5tools.collectDatasets(testCase.FILE);
testCase.verifyTrue(ismember("/Group3", groupNames));
testCase.verifyTrue(~ismember("/Group1/Group1A", groupNames));
testCase.verifyTrue(ismember("/Group3/Dataset1A1", dsetNames));
end
end
end
@@ -0,0 +1,28 @@
# H5TOOLS-MATLAB Test Suite
As of 12Dec2022, there are **43** files in the ```h5tools``` package. Code coverage is:
- **90.41%** statement coverage (762 executable)
- **100%** function coverage (50 executable)
The test suite contains the following:
- ```AttributeTest``` - tests reading and writing attributes
- ```AttributesTest``` - tests reading and writing multiple attributes at once
- ```DatasetTest``` - tests reading and writing of HDF5 datasets
- ```EnumClassTest``` - tests the enumerated types included in h5tools-matlab
- ```GroupTest``` - tests operations at the group level
- ```FileTest``` - tests operations at the file level
- ```LinkTest``` - tests interface for working with soft links
- ```MoveTest``` - tests moving/renaming HDF5 groups
- ```SearchTest``` - tests the functions for searching HDF5 files
- ```ValidatorTest``` - tests validation functions
The test suite can be run with the following:
```matlab
results = runH5toolsTestSuite();
```
There are 3 optional key/value inputs to the function above, all of which are false by default:
- ```Coverage``` - whether to generate a code coverage report for the "h5tools" package
- ```KeepFiles``` - whether to keep the HDF5 files generated by the tests for inspection
- ```Debug``` - whether to stop test execution and enter debugging mode on an error
@@ -0,0 +1,50 @@
classdef SearchTest < matlab.unittest.TestCase
% SEARCHTEST
%
% Description:
% Test collection of all groups, datasets and links
%
% Examples:
% results = runtests('SearchTest')
%
% See also:
% runH5toolsTestSuite
%
% History:
% 3Dec2022 - SSP
% -------------------------------------------------------------------------
properties
FILE
end
methods (TestClassSetup)
function createFile(testCase)
testCase.FILE = fullfile(fileparts(mfilename("fullpath")), 'Collection.h5');
h5tools.createFile(testCase.FILE, true);
% Groups have already been tested in GroupTest. Here create
% some to support the other functions
h5tools.createGroup(testCase.FILE, '/',...
'Group1', 'Group2', 'Group3');
h5tools.createGroup(testCase.FILE, '/Group1',...
'Group1a', 'Group1b');
% Create a dataset for the links
h5tools.write(testCase.FILE, '/Group1', 'Dataset', eye(3));
end
end
methods (Test)
function Datasets(testCase)
out = h5tools.collectDatasets(testCase.FILE);
testCase.verifyNumElements(out, 1);
testCase.verifyEqual(out, "/Group1/Dataset");
% Test with fileID input
fileID = h5tools.files.openFile(testCase.FILE);
fileIDx = onCleanup(@()H5F.close(fileID));
testCase.verifyNumElements(...
h5tools.collectDatasets(fileID), 1);
end
end
end
@@ -0,0 +1,90 @@
classdef ValidatorTest < matlab.unittest.TestCase
properties
TXT_FILE = 'ValidatorTest.txt';
HDF_FILE = 'ValidatorTest.h5';
end
methods (TestClassSetup)
function createFiles(testCase)
dlmwrite(testCase.TXT_FILE, eye(3));
% Create a minimum HDF5 file with a single group
h5tools.createFile(testCase.HDF_FILE, true);
h5tools.createGroup(testCase.HDF_FILE, '/', 'GroupOne');
end
end
methods (TestClassTeardown)
function deleteFiles(testCase)
delete(testCase.TXT_FILE);
end
end
methods (Test, TestTags = ["mustBeHdfFile"])
function HDF5FileNames(testCase)
import matlab.unittest.constraints.Throws
% A file that isn't HDF5
testCase.verifyThat(...
@() mustBeHdfFile(testCase.TXT_FILE),...
Throws("mustBeHdfFile:InvalidFile"));
% A file that doesn't exist
testCase.verifyThat(...
@() mustBeHdfFile('NotAFile.h5'),...
Throws("mustBeHdfFile:InvalidFile"));
end
function HDF5FIleInvalidInput(testCase)
import matlab.unittest.constraints.Throws
testCase.verifyThat(...
@() mustBeHdfFile(1),...
Throws("mustBeHdfFile:InvalidInput"));
end
function HDF5FileID(testCase)
import matlab.unittest.constraints.Throws
% Passing a file ID should not error
fileID = h5tools.files.openFile(testCase.HDF_FILE);
fileIDx = @()onCleanup(H5G.close(groupID));
mustBeHdfFile(fileID);
% Passing a group ID should cause an error
groupID = H5G.open(fileID, '/GroupOne');
groupIDx = @()onCleanup(H5G.close(groupID));
testCase.verifyThat(...
@() mustBeHdfFile(groupID),...
Throws("mustBeFileID:InvalidH5MLID"));
end
end
methods (Test, TestTags=["mustBeHdfPath"])
function HdfPath(testCase)
import matlab.unittest.constraints.Throws
% Should be error free
mustBeHdfPath(testCase.HDF_FILE, '/GroupOne');
% Should throw an error
testCase.verifyThat(...
@() mustBeHdfPath(testCase.HDF_FILE, '/GroupTwo'),...
Throws("HdfPath:InvalidPath"));
end
end
methods (Test, TestTags = ["mustBeEnum"])
function EnumClass(testCase)
import matlab.unittest.constraints.Throws
testCase.verifyThat(...
@() mustBeEnum(1),...
Throws("mustBeEnum:InvalidInput"));
% Should not throw an error (TODO)
mustBeEnum(test.EnumClass.GROUPONE);
end
end
end
@@ -0,0 +1,89 @@
function results = runH5toolsTestSuite(options)
% RUNH5TOOLSTESTSUITE
%
% Description:
% Runs the full test suite with options for code coverage reports,
% debugging and keeping files produced during testing
%
% Syntax:
% results = runH5toolsTestSuite()
% results = runH5toolsTestSuite(options)
%
% Optional key/value inputs:
% Coverage logical (default = false)
% Whether to output coverage report
% KeepFiles logical (default = false)
% Whether to keep HDF5 files produced by the test suite
% Debug logical (default = false)
% Whether to stop on failures
%
% See Also:
% test.deleteH5toolsTestFiles
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
options.Coverage logical = false
options.KeepFiles logical = false
options.Debug logical = false
end
% Save current directory for return after test suite completes
currentCD = pwd();
% Run the suite in the "tests" folder
cd(fileparts(mfilename('fullpath')));
% Delete files kept from a previous run, if necessary
test.deleteH5toolsTestFiles();
% Run the test suite
if options.Coverage
results = testWithCoverageReport(options.Debug);
else
results = testWithoutCoverageReport(options.Debug);
end
% Clean up files produced by tests, if necessary
if ~options.KeepFiles
test.deleteH5toolsTestFiles();
end
% Return to user's previous working directory
cd(currentCD);
end
function results = testWithoutCoverageReport(debugFlag)
import matlab.unittest.plugins.StopOnFailuresPlugin
suite = testsuite(pwd);
runner = testrunner("textoutput");
if debugFlag
runner.addPlugin(StopOnFailuresPlugin);
end
results = runner.run(suite);
end
function results = testWithCoverageReport(debugFlag)
import matlab.unittest.plugins.CodeCoveragePlugin
import matlab.unittest.plugins.codecoverage.CoverageReport
import matlab.unittest.plugins.StopOnFailuresPlugin
if ~exist('coverage', 'dir')
mkdir('coverage');
end
suite = testsuite(pwd);
runner = testrunner("textoutput");
if debugFlag
runner.addPlugin(StopOnFailuresPlugin);
end
p = CodeCoveragePlugin.forPackage("h5tools",...
'IncludingSubpackages', true,...
'Producing', CoverageReport(fullfile(pwd, 'coverage')));
runner.addPlugin(p);
results = runner.run(suite);
open(fullfile('coverage', 'index.html'));
end
@@ -0,0 +1,31 @@
function newMap = addMaps(map1, map2)
% ADDMAP
%
% Description:
% Add two containers.Maps with unique keys together
%
% Syntax:
% newMap = addMaps(map1, map2)
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
arguments
map1 containers.Map
map2 containers.Map
end
keys1 = map1.keys;
keys2 = map2.keys;
if ismember(string(keys1), string(keys2))
error('addMap:DuplicateKeys',...
'One or more keys are shared between the two inputs');
end
newMap = containers.Map();
for i = 1:numel(keys1)
newMap(keys1{i}) = map1(keys1{i});
end
for i = 1:numel(keys2)
newMap(keys2{i}) = map2(keys2{i});
end
@@ -0,0 +1,21 @@
function S = kv2map(varargin)
% KV2MAP
%
% Description:
% Maps multiple keys and values to a containers.Map
%
% Syntax:
% S = kv2map(S, 'key', value, 'key', value);
% S = kv2map('key', value, 'key', value);
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
if nargin == 1
return
end
S = containers.Map();
for i = 2:2:numel(varargin)
S(varargin{i-1}) = varargin{i};
end
@@ -0,0 +1,27 @@
function S = map2struct(map)
% MAP2STRUCT
%
% Description:
% Convert a containers.Map to a struct
%
% Syntax:
% S = map2struct(map)
%
% Inputs:
% map containers.Map (keyvalue = char/string)
%
% History:
% 03Jun2022 - SSP
% -------------------------------------------------------------------------
S = struct();
if isempty(map)
return
end
keys = map.keys;
values = map.values;
for i = 1:numel(keys)
S.(keys{i}) = values{i};
end
@@ -0,0 +1,15 @@
function mustBeEnum(input)
% MUSTBEENUM
%
% Description:
% Validates that input is an enumeration
%
% Syntax:
% mustBeEnum(input)
% -------------------------------------------------------------------------
if ~isenum(input)
eid = "mustBeEnum:InvalidInput";
msg = "Input must be an enumerated type";
throwAsCaller(MException(eid, msg));
end
@@ -0,0 +1,34 @@
function mustBeHdfFile(input)
% MUSTBEHDFFILE
%
% Description:
% Validates whether input is an HDF5 file or H5ML.id for a file
%
% Syntax:
% mustBeHdfFile(input)
%
% ---------------------------------------------------------------------
if istext(input)
if ~isfile(input) || ~endsWith(input, '.h5')
eid = 'mustBeHdfFile:InvalidFile';
msg = 'File %s not found, input must be valid file name ending with .h5 or an H5ML.id';
throwAsCaller(MException(eid, msg));
end
return
end
if isa(input, 'H5ML.id')
objType = H5I.get_type(input);
if objType ~= H5ML.get_constant_value('H5I_FILE')
eid = 'mustBeFileID:InvalidH5MLID';
msg = 'H5ML.id was not a file identifier';
throwAsCaller(MException(eid, msg));
end
return
end
% If input wasn't an HDF5 file name or a file H5ML.id, it's invalid
eid = 'mustBeHdfFile:InvalidInput';
msg = 'Input must be a valid HDF5 file name or a file H5ML.id';
throwAsCaller(MException(eid, msg));
@@ -0,0 +1,26 @@
function mustBeHdfPath(hdfFile, hdfPath)
% MUSTBEHDFPATH
%
% Description:
% Argument validation function confirming hdfPath exists within hdfFile
%
% Syntax:
% mustBeHdfPath(hdfFile, hdfPath)
%
% Notes:
% For this to work in an argument block, the HDF file name must be
% a previous argument.
%
% History:
% 06Dec2022 - SSP
% -------------------------------------------------------------------------
if strcmp(hdfPath, '/')
return
end
if ~h5tools.exist(hdfFile, hdfPath)
eid = "HdfPath:InvalidPath";
msg = "HDF5 path does not exist in file";
throwAsCaller(MException(eid, msg));
end
@@ -0,0 +1,17 @@
function y = rangeCol(a, b)
% RANGECOL
%
% Description:
% Just creates an array with : but returns a column instead of a row
%
% Syntax:
% y = rangeCol(a,b)
%
% History:
% 06Mar2022 - SSP
% ---------------------------------------------------------------------
if nargin == 1 && numel(a) == 2
b = a(2); a = a(1);
end
y = a:b;
y = y';
@@ -0,0 +1,28 @@
function map = struct2map(S)
% STRUCT2MAP
%
% Description:
% Convert a containers.Map to a struct
%
% Syntax:
% S = map2struct(map)
%
% Inputs:
% S struct
%
% Outputs:
% map containers.Map
%
% History:
% 30Nov2022 - SSP
% -------------------------------------------------------------------------
map = containers.Map();
if isempty(S)
return
end
keys = fieldnames(S);
for i = 1:numel(keys)
map(keys{i}) = S.(keys{i});
end
@@ -0,0 +1,24 @@
function out = uncell(data)
% Remove data from a cell
%
% Description:
% Some functions run on multiple entities w/ arrayfun require
% UniformOutput = false but return only one cell containing the
% entities. This is a convenience function to "uncell" the output.
% It is also useful with a statement ending with parentheses also
% requires the {:} command
%
% Syntax:
% out = uncell(data)
%
% See also:
% aod.util.arrayfun
% By Sara Patterson, 2022 (h5tools-matlab)
% -------------------------------------------------------------------------
if ~iscell(data)
out = data;
return
end
out = vertcat(data{:});