HDF5 file import from EPICS area detectors. More...

#include <HDF5 Browser>
#include "pearl-gui-tools"

Namespaces
	PearlAreaImport
	HDF5 file import from EPICS area detectors.

Functions
static variable	BeforeFileOpenHook (variable refNum, string fileName, string path, string type, string creator, variable kind)
	callback function for drag&drop of HDF5 files into Igor. More...

string	ad_suggest_foldername (string filename, variable ignoredate=defaultValue, string sourcename=defaultValue, variable unique=defaultValue)
	generate the name of a data folder based on a file name. More...

variable	ad_load_dialog (string APathName)
	load area detector data files selected in a file dialog window More...

string	adh5_load_complete (string ANickName, string APathName, string AFileName, variable load_data=defaultValue, variable load_attr=defaultValue)
	import everything from a HDF5 file created by the Area Detector software. More...

string	adh5_load_reduced (string ANickName, string APathName, string AFileName, funcref reduction_func, string reduction_param, variable load_data=defaultValue, variable load_attr=defaultValue, variable progress=defaultValue)
	load and reduce a dataset from a HDF5 file created by the Area Detector software. More...

string	adh5_load_preview (string ANickName, string APathName, string AFileName, variable load_data=defaultValue, variable load_attr=defaultValue)
	load a single image from a HDF5 file created by the Area Detector software. More...

string	adh5_load_info (string APathName, string AFileName)
	load descriptive info from a HDF5 file created by the Area Detector software. More...

variable	adh5_load_detector (variable fileID, string detectorpath)
	load the detector dataset from the open HDF5 file. More...

variable	adh5_redim (wave data)
	redimension a multi-dimensional area detector array loaded from HDF5. More...

static dfr	GetAttrDataFolderDFR (wave data)
	find the attributes data folder of an area detector dataset. More...

variable	adh5_scale (wave data, string source=defaultValue)
	set the dimension scales of an area detector dataset. More...

variable	adh5_load_detector_slabs (variable fileID, string detectorpath, variable progress=defaultValue)
	load the detector dataset from the open HDF5 file. More...

variable	adh5_load_detector_image (variable fileID, string detectorpath, variable dim2start, variable dim2count, variable dim3start, variable dim3count)
	load a single image from the detector dataset of the open HDF5 file More...

string	adh5_list_reduction_funcs ()
	get a list of functions which can be used as reduction functions. More...

threadsafe wave	adh5_default_reduction (wave source, string *param)
	function prototype for adh5_load_reduced_detector More...

threadsafe variable	adh5_setup_profile (wave image, wave profile, variable dim)
	set up a one-dimensional wave for a line profile based on a 2D original wave. More...

string	adh5_test_reduction_func (wave source, funcref reduction_func, string reduction_param, string result_prefix)
	wrapper function for testing reduction functions from the command line. More...

threadsafe variable	adh5_get_result_waves (wave results, string result_prefix, variable start_index)
	copy waves from wave reference wave into current data folder More...

variable	adh5_load_reduced_detector (variable fileID, string detectorpath, funcref reduction_func, string reduction_param, variable progress=defaultValue, variable nthreads=defaultValue)
	load a reduced detector dataset from the open HDF5 file. More...

static threadsafe variable	reduce_slab_worker (funcref reduction_func)

static threadsafe wave	reduce_slab_image (wave slabdata, wave image, funcref reduction_func, string reduction_param)

variable	adh5_loadattr_all (variable fileID, string attributespath)
	load an NDAttributes group from an open HDF5 file into the current data folder. More...

static variable	read_attribute_info (string datawavename, string source, variable *idest)
	sub-function of adh5_loadattr_all. More...

variable	adh5_scale_scienta (wave data)
	set the energy and angle scales of an area detector dataset from the Scienta analyser. More...

variable	adh5_scale_scan (wave data)
	scales the extra dimensions of an area detector dataset according to the EPICS scan More...

Detailed Description

HDF5 file import from EPICS area detectors.

HDF5 file import from EPICS area detectors such as CCD cameras, 2D electron analysers

as of Igor 6.3, Igor can open datasets of up to rank 4. i.e. the extra dimension Y of the file plugin cannot be used. the extra dimensions N and X are supported.

Author: matthias muntwiler, matth.nosp@m.ias..nosp@m.muntw.nosp@m.iler.nosp@m.@psi..nosp@m.ch

Copyright: 2013-18 Paul Scherrer Institut
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Definition in file pearl-area-import.ipf.

Function Documentation

variable ad_load_dialog ( string APathName )

load area detector data files selected in a file dialog window

Parameters

APathName Igor symbolic path name. if empty, Igor will choose a folder on its own

Definition at line 166 of file pearl-area-import.ipf.

string ad_suggest_foldername	(	string	filename,
		variable	ignoredate = `defaultValue`,
		string	sourcename = `defaultValue`,
		variable	unique = `defaultValue`
	)

generate the name of a data folder based on a file name.

if the file name follows the naming convention source-date-index.extension, the function tries to generate the nick name as source_date_index. otherwise it's just a cleaned up version of the file name.

date must be in yyyymmdd or yymmdd format and is clipped to the short yymmdd format. index should be a running numeric index of up to 6 digits, or the time encoded as hhmmss. however, in the current version index can be any string that can be a valid Igor folder name.

Parameters

filename	file name, including extension. can also include a folder path (which is ignored). the extension is currently ignored, but may be used to select the parent folder in a later version.
ignoredate	if non-zero, the nick name will not include the date part. defaults to zero.
sourcename	nick name of the data source. by default, the function tries to detect the source from the file name. this option can be used to override auto-detection. the automatic source names are: sci (scienta by area detector), psh (pshell), sl (optics slit camera by area detector), es (end station camera by area detector), xy (unidentified).
unique	if non-zero, the resulting name is made a unique data folder name in the current data folder defaults to zero.

Definition at line 105 of file pearl-area-import.ipf.

threadsafe wave adh5_default_reduction	(	wave	source,
		string *	param
	)

function prototype for adh5_load_reduced_detector

this is a prototype of custom functions that convert (reduce) a two-dimensional detector image into one or more one-dimensional waves. data processing can be tuned with a set of parameters.

reduction functions have a fixed signature (function arguments) so that the file import functions can call them efficiently on a series of detector images. pearl procedures comes with a number of pre-defined reduction functions but you may as well implement your own functions. if you write your own function, you must use the same declaration and arguments as this function except for the function name. you can do many things in a reduction function, e.g. integration over a region of interest, curve fitting, etc.

each destination wave is a one-dimensional intensity distribution. the function must redimension each of these waves to one of the image dimensions by calling the adh5_setup_profile() function. this function will also copy the scale information and dimension labels, which is important for the proper scaling of the result.

the meaning of the data in the result waves is up to the particular function, e.g. dest1 could hold the mean value and dest2 the one-sigma error, or dest1 could hold the X-profile, and dest2 the Y-profile.

Parameters

source	source wave. two-dimensional intensity distribution (image). the scales are carried over to the result waves.
param	string with optional parameters, shared between calls. this is a pass-by-reference argument, the function may modify the string.

Returns: a free wave containing references of the result waves. the result waves should as well be free waves. if an error occurred, the reference wave is empty.

Definition at line 1110 of file pearl-area-import.ipf.

threadsafe variable adh5_get_result_waves	(	wave	results,
		string	result_prefix,
		variable	start_index
	)

copy waves from wave reference wave into current data folder

this function copies waves that are referenced in a wave reference wave into the current data folder. the destination waves get new names consisting of a prefix and a numeric index. the index is the array index of the wave in results plus a chosen offset.

Parameters

results	a wave reference wave pointing to result waves from data reduction. the waves can be free or regular waves. results can be a free or regular wave.
result_prefix	name prefix of the copied waves.
start_index	start index (offset) of the copied waves.

Definition at line 1192 of file pearl-area-import.ipf.

string adh5_list_reduction_funcs ( )

get a list of functions which can be used as reduction functions.

the function evaluates only the function arguments, it may thus include functions which are not suitable as reduction functions.

Definition at line 1040 of file pearl-area-import.ipf.

string adh5_load_complete	(	string	ANickName,
		string	APathName,
		string	AFileName,
		variable	load_data = `defaultValue`,
		variable	load_attr = `defaultValue`
	)

import everything from a HDF5 file created by the Area Detector software.

if the data is from the electron analyser driver and some special attributes are included, the function will set the scales of the image dimensions.

Parameters

ANickName	destination folder name (top level under root)
APathName	igor symbolic path name. can be empty if the path is specified in FileName or a dialog box should be displayed
AFileName	if empty a dialog box shows up
load_data	1 (default): load data; 0: do not load data
load_attr	1 (default): load attributes; 0: do not load attributes for proper wave scaling, the attributes must be loaded

Definition at line 206 of file pearl-area-import.ipf.

variable adh5_load_detector	(	variable	fileID,
		string	detectorpath
	)

load the detector dataset from the open HDF5 file.

the function loads the whole dataset at once and redimensions it so that the image dimensions are X and Y

Parameters

fileID	ID of open HDF5 file from HDF5OpenFile
detectorpath	path to detector group in the HDF5 file

Definition at line 580 of file pearl-area-import.ipf.

variable adh5_load_detector_image	(	variable	fileID,
		string	detectorpath,
		variable	dim2start,
		variable	dim2count,
		variable	dim3start,
		variable	dim3count
	)

load a single image from the detector dataset of the open HDF5 file

the function can average over a region in the extra dimensions.

Parameters

fileID	ID of open HDF5 file from HDF5OpenFile
detectorpath	path to detector group in the HDF5 file
dim2start	2nd dimension coordinate of the first image note that the order of dimensions is reversed in the file 2nd dimension = N dimension in area detector = dimension 0 of the three-dimensional HDF dataset set to 0 if dimension may not be present
dim2count	number of subsequent images to average set to 1 if dimension may not be present
dim3start	3rd dimension coordinate of the first image note that the order of dimensions is reversed in the file 3rd dimension = extra X dimension in area detector = dimension 0 of the four-dimensional HDF dataset set to 0 if dimension may not be present
dim3count	number of subsequent images to average set to 1 if dimension may not be present

Definition at line 944 of file pearl-area-import.ipf.

variable adh5_load_detector_slabs	(	variable	fileID,
		string	detectorpath,
		variable	progress = `defaultValue`
	)

load the detector dataset from the open HDF5 file.

the function loads the dataset image by image using the hyperslab option. this function gives the same result as adh5_load_detector. it is about 5% slower, and it depends on HDF5 Browser code. but it does not choke on large datasets (as long as the final wave fits into memory).

Parameters

fileID	ID of open HDF5 file from HDF5OpenFile.
detectorpath	path to detector group in the HDF5 file.
progress	1 (default): show progress window; 0: do not show progress window.

Returns: 0 if successful, non-zero if an error occurred.

Definition at line 779 of file pearl-area-import.ipf.

string adh5_load_info	(	string	APathName,
		string	AFileName
	)

load descriptive info from a HDF5 file created by the Area Detector software.

the information returned is the array size and active scans

Attention: EXPERIMENTAL this function should be merged with adh5_load_preview

Parameters

APathName	igor symbolic path name. can be empty if the path is specified in FileName or a dialog box should be displayed
AFileName	if empty a dialog box shows up

Definition at line 496 of file pearl-area-import.ipf.

string adh5_load_preview	(	string	ANickName,
		string	APathName,
		string	AFileName,
		variable	load_data = `defaultValue`,
		variable	load_attr = `defaultValue`
	)

load a single image from a HDF5 file created by the Area Detector software.

the data wave is loaded into the current data folder. attributes are loaded into the attr subfolder. existing waves in attr are deleted.

Warning: EXPERIMENTAL this function uses the root:pearl_area:preview data folder. existing data there may be deleted!

Parameters

ANickName	destination wave name. the wave is created in the current data folder.
APathName	igor symbolic path name. can be empty if the path is specified in FileName or a dialog box should be displayed
AFileName	if empty a dialog box shows up
load_data	1 (default): load data; 0: do not load data
load_attr	1 (default): load attributes; 0: do not load attributes note: for correct scaling of the image, the attributes need to be loaded

Definition at line 386 of file pearl-area-import.ipf.

string adh5_load_reduced	(	string	ANickName,
		string	APathName,
		string	AFileName,
		funcref	reduction_func,
		string	reduction_param,
		variable	load_data = `defaultValue`,
		variable	load_attr = `defaultValue`,
		variable	progress = `defaultValue`
	)

load and reduce a dataset from a HDF5 file created by the Area Detector software.

the resulting dataset is reduced in one image dimension by a user-defined reduction function, e.g. by region-of-interest integration, curve fitting, etc.

the function loads the dataset image by image using the hyperslab option and applies a custom reduction function to each image. the results from the reduction function are composed into one result wave. the raw data are discarded.

if the data is from the electron analyser driver and some special attributes are included, the function will set the scales of the image dimensions.

Parameters

ANickName	destination folder name (top level under root)
APathName	igor symbolic path name. can be empty if the path is specified in FileName or a dialog box should be displayed
AFileName	if empty a dialog box shows up
reduction_func	custom reduction function (any user-defined function which has the same parameters as adh5_default_reduction())
reduction_param	parameter string for the reduction function
load_data	1 (default): load data; 0: do not load data
load_attr	1 (default): load attributes; 0: do not load attributes for proper wave scaling, the attributes must be loaded
progress	1 (default): show progress window; 0: do not show progress window

Definition at line 301 of file pearl-area-import.ipf.

variable adh5_load_reduced_detector	(	variable	fileID,
		string	detectorpath,
		funcref	reduction_func,
		string	reduction_param,
		variable	progress = `defaultValue`,
		variable	nthreads = `defaultValue`
	)

load a reduced detector dataset from the open HDF5 file.

the function loads the dataset image by image using the hyperslab option and applies a custom reduction function to each image. the results from the reduction function are composed into one result wave. the raw data are discarded.

by default, the reduction function is called in separate threads to reduce the total loading time. (see the global variable adh5_perf_secs which reports the total run time of the function.) the effect varies depending on the balance between file loading (image size) and data processing (complexity of the reduction function). for debugging the reduction function, multi-threading can be disabled.

Parameters

fileID	ID of open HDF5 file from HDF5OpenFile
detectorpath	path to detector group in the HDF5 file
reduction_func	custom reduction function (any user-defined function which has the same parameters as adh5_default_reduction())
reduction_param	parameter string for the reduction function
progress	1 (default): show progress window; 0: do not show progress window
nthreads	-1 (default): use as many threads as there are processor cores (in addition to main thread) 0: use main thread only (e.g. for debugging the reduction function) >= 1: use a fixed number of (additional) threads

Definition at line 1228 of file pearl-area-import.ipf.

variable adh5_loadattr_all	(	variable	fileID,
		string	attributespath
	)

load an NDAttributes group from an open HDF5 file into the current data folder.

datasets contained in the group are loaded as waves. if a dataset contains only one data point, it is added to the IN, ID, IV, IU waves, where IN = EPICS channel name, ID = attribute name, IV = value, IU = unit (units are left empty as they are not saved in HDF5). attributes of the NDAttributes group are added to the IN, ID, IV, IU waves, however, IN and IU are left empty as this information is not saved in the HDF5 file.

Parameters

fileID	ID of open HDF5 file from HDF5OpenFile
attributespath	path to NDAttributes group in the HDF5 file

Definition at line 1552 of file pearl-area-import.ipf.

variable adh5_redim ( wave data )

redimension a multi-dimensional area detector array loaded from HDF5.

so that the image dimensions are X and Y singleton dimensions are removed (most common cases only)

in the redimensioned array, the original dimension type is noted in the dimension label: AD_Dim0 = first image dimension AD_Dim1 = second image dimension AD_DimN = frame sequence AD_DimX = extra dimension X AD_DimY = extra dimension Y (cannot be loaded in Igor)

Parameters

data	area detector data loaded from HDF5 to be redimensioned

Definition at line 625 of file pearl-area-import.ipf.

variable adh5_scale	(	wave	data,
		string	source = `defaultValue`
	)

set the dimension scales of an area detector dataset.

the intrinsic dimensions 0 and 1 are scaled according to the data source (currently supported: Prosilica cameras, Scienta electron analyser). the extra dimensions are scaled according to the scan. the latter requires that the positioner names and position values are available.

Definition at line 725 of file pearl-area-import.ipf.

variable adh5_scale_scan ( wave data )

scales the extra dimensions of an area detector dataset according to the EPICS scan

the scan positioner name and its values must be available

Todo:: incomplete

Definition at line 1792 of file pearl-area-import.ipf.

variable adh5_scale_scienta ( wave data )

set the energy and angle scales of an area detector dataset from the Scienta analyser.

the dimension labels of the energy and angle scales must be set correctly: AD_Dim0 = energy dimension; AD_Dim1 = angle dimension. these dimensions must be the first two dimensions of a multi-dimensional dataset. normally, AD_Dim0 is the X dimension, and AD_Dim1 the Y dimension.

Definition at line 1687 of file pearl-area-import.ipf.

threadsafe variable adh5_setup_profile	(	wave	image,
		wave	profile,
		variable	dim
	)

set up a one-dimensional wave for a line profile based on a 2D original wave.

redimensions the profile wave to the given dimension. copies the scale and dimension label of the given dimension.

Definition at line 1133 of file pearl-area-import.ipf.

string adh5_test_reduction_func	(	wave	source,
		funcref	reduction_func,
		string	reduction_param,
		string	result_prefix
	)

wrapper function for testing reduction functions from the command line.

reduction functions cannot be used on the command line because they require a pass-by-reference argument and return free waves. this function expects the reduction parameters in a normal string and copies the results into the current data folder. the prefix of the result names can be specified.

Parameters

source	source wave. two-dimensional intensity distribution (image). the scales are carried over to the result waves.
reduction_func	name of the reduction function to apply to the source data.
reduction_param	string with reduction parameters as required by the specific reduction function.
result_prefix	name prefix of result waves. a numeric index is appended to distinguish the results. the index starts at 1. existing waves are overwritten.

Returns: a copy of the reduction_param string, possibly modified by the reduction function.

Definition at line 1166 of file pearl-area-import.ipf.

static variable BeforeFileOpenHook	(	variable	refNum,
		string	fileName,
		string	path,
		string	type,
		string	creator,
		variable	kind
	)

static

callback function for drag&drop of HDF5 files into Igor.

Definition at line 41 of file pearl-area-import.ipf.

static dfr GetAttrDataFolderDFR ( wave data )

static

find the attributes data folder of an area detector dataset.

since version 1.04 attributes should be stored in a subfolder named attr. earlier versions had the attributes in the same data folder as the actual dataset.

Parameters

data	wave containing the main dataset.

Returns: data folder reference of the attributes folder. the reference may be invalid (and default to root) if the folder cannot be found, cf. built-in DataFolderRefStatus function.

Definition at line 706 of file pearl-area-import.ipf.

static variable read_attribute_info	(	string	datawavename,
		string	source,
		variable *	idest
	)

static

sub-function of adh5_loadattr_all.

reads one attribute from a wave which was loaded from an HDF5 file into the info waves IN, ID, IV, IU. the attribute is read only if the input wave contains exactly one item, i.e. either the measurement is a single image, or the attribute has string type.

Parameters

datawavename	name of the attribute wave in the current folder. can be text or numeric.
source	source identifier (EPICS name) of the attribute.
idest	destination index in IN, ID, IV, IU where the results are written. the variable is incremented if data was written, otherwise it is left unchanged. make sure IN, ID, IV, IU have at least idest + 1 elements.

Definition at line 1634 of file pearl-area-import.ipf.

static threadsafe wave reduce_slab_image	(	wave	slabdata,
		wave	image,
		funcref	reduction_func,
		string	reduction_param
	)

static

Definition at line 1519 of file pearl-area-import.ipf.

static threadsafe variable reduce_slab_worker ( funcref reduction_func )

static

Definition at line 1480 of file pearl-area-import.ipf.

Namespaces

Functions

Detailed Description

Function Documentation