pearl-area-import.ipf File Reference
ARPES package

HDF5 file import from EPICS area detectors. More...

`#include <HDF5 Browser>`
`#include "pearl-gui-tools"`

Go to the source code of this file.

## Namespaces

| PearlAreaImport
| HDF5 file import from EPICS area detectors.

## Functions

static variable | BeforeFileOpenHook (variable refNum, string fileName, string path, string type, string creator, variable kind)
| callback function for drag&drop of HDF5 files into Igor. More...
string | ad_suggest_foldername (string filename, variable ignoredate=defaultValue, string sourcename=defaultValue, variable unique=defaultValue)
| generate the name of a data folder based on a file name. More...
variable | ad_load_dialog (string APathName)
| load area detector data files selected in a file dialog window More...
string | adh5_load_complete (string ANickName, string APathName, string AFileName, variable load_data=defaultValue, variable load_attr=defaultValue)
| import everything from a HDF5 file created by the Area Detector software. More...
string | adh5_load_reduced (string ANickName, string APathName, string AFileName, funcref reduction_func, string reduction_param, variable load_data=defaultValue, variable load_attr=defaultValue, variable progress=defaultValue)
| load and reduce a dataset from a HDF5 file created by the Area Detector software. More...
string | adh5_load_preview (string ANickName, string APathName, string AFileName, variable load_data=defaultValue, variable load_attr=defaultValue)
| load a single image from a HDF5 file created by the Area Detector software. More...
string | adh5_load_info (string APathName, string AFileName)
| load descriptive info from a HDF5 file created by the Area Detector software. More...
variable | adh5_load_detector (variable fileID, string detectorpath)
| load the detector dataset from the open HDF5 file. More...
variable | adh5_redim (wave data)
| redimension a multi-dimensional area detector array loaded from HDF5. More...
static dfr | GetAttrDataFolderDFR (wave data)
| find the attributes data folder of an area detector dataset. More...
variable | adh5_scale (wave data, string source=defaultValue)
| set the dimension scales of an area detector dataset. More...
variable | adh5_load_detector_slabs (variable fileID, string detectorpath, variable progress=defaultValue)
| load the detector dataset from the open HDF5 file. More...
variable | adh5_load_detector_image (variable fileID, string detectorpath, variable dim2start, variable dim2count, variable dim3start, variable dim3count)
| load a single image from the detector dataset of the open HDF5 file More...
string | adh5_list_reduction_funcs ()
| get a list of functions which can be used as reduction functions. More...
threadsafe wave | adh5_default_reduction (wave source, string *param)
| function prototype for adh5_load_reduced_detector More...
threadsafe variable | adh5_setup_profile (wave image, wave profile, variable dim)
| set up a one-dimensional wave for a line profile based on a 2D original wave. More...
string | adh5_test_reduction_func (wave source, funcref reduction_func, string reduction_param, string result_prefix)
| wrapper function for testing reduction functions from the command line. More...
threadsafe variable | adh5_get_result_waves (wave results, string result_prefix, variable start_index)
| copy waves from wave reference wave into current data folder More...
variable | adh5_load_reduced_detector (variable fileID, string detectorpath, funcref reduction_func, string reduction_param, variable progress=defaultValue, variable nthreads=defaultValue)
| load a reduced detector dataset from the open HDF5 file. More...
static threadsafe variable | reduce_slab_worker (funcref reduction_func)
static threadsafe wave | reduce_slab_image (wave slabdata, wave image, funcref reduction_func, string reduction_param)
variable | adh5_loadattr_all (variable fileID, string attributespath)
| load an NDAttributes group from an open HDF5 file into the current data folder. More...
static variable | read_attribute_info (string datawavename, string source, variable *idest)
| sub-function of adh5_loadattr_all. More...
variable | adh5_scale_scienta (wave data)
| set the energy and angle scales of an area detector dataset from the Scienta analyser. More...
variable | adh5_scale_scan (wave data)
| scales the extra dimensions of an area detector dataset according to the EPICS scan More...

## Detailed Description

HDF5 file import from EPICS area detectors.

HDF5 file import from EPICS area detectors such as CCD cameras, 2D electron analysers

as of Igor 6.3, Igor can open datasets of up to rank 4. i.e. the extra dimension Y of the file plugin cannot be used. the extra dimensions N and X are supported.

Author
  matthias muntwiler, matth.nosp@m.ias..nosp@m.muntw.nosp@m.iler.nosp@m.@psi..nosp@m.ch

Copyright
  2013-18 Paul Scherrer Institut
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Definition in file pearl-area-import.ipf.

## Function Documentation

## ad_load_dialog()

variable ad_load_dialog | ( | string | APathName | ) |

load area detector data files selected in a file dialog window

Parameters

  APathName | Igor symbolic path name. if empty, Igor will choose a folder on its own

Definition at line 166 of file pearl-area-import.ipf.

## ad_suggest_foldername()

string ad_suggest_foldername | ( | string | filename,
|  | variable | ignoredate = `defaultValue`,
|  | string | sourcename = `defaultValue`,
|  | variable | unique = `defaultValue`
| ) |  |

generate the name of a data folder based on a file name.

if the file name follows the naming convention source-date-index.extension, the function tries to generate the nick name as source_date_index. otherwise it's just a cleaned up version of the file name.

date must be in yyyymmdd or yymmdd format and is clipped to the short yymmdd format. index should be a running numeric index of up to 6 digits, or the time encoded as hhmmss. however, in the current version index can be any string that can be a valid Igor folder name.

Parameters

  filename | file name, including extension. can also include a folder path (which is ignored). the extension is currently ignored, but may be used to select the parent folder in a later version.
  ignoredate | if non-zero, the nick name will not include the date part. defaults to zero.
  sourcename | nick name of the data source. by default, the function tries to detect the source from the file name. this option can be used to override auto-detection. the automatic source names are: sci (scienta by area detector), psh (pshell), sl (optics slit camera by area detector), es (end station camera by area detector), xy (unidentified).
  unique | if non-zero, the resulting name is made a unique data folder name in the current data folder defaults to zero.

Definition at line 105 of file pearl-area-import.ipf.

## adh5_default_reduction()

threadsafe wave adh5_default_reduction | ( | wave | source,
|  | string * | param
| ) |  |

function prototype for adh5_load_reduced_detector

this is a prototype of custom functions that convert (reduce) a two-dimensional detector image into one or more one-dimensional waves. data processing can be tuned with a set of parameters.

reduction functions have a fixed signature (function arguments) so that the file import functions can call them efficiently on a series of detector images. pearl procedures comes with a number of pre-defined reduction functions but you may as well implement your own functions. if you write your own function, you must use the same declaration and arguments as this function except for the function name. you can do many things in a reduction function, e.g. integration over a region of interest, curve fitting, etc.

each destination wave is a one-dimensional intensity distribution. the function must redimension each of these waves to one of the image dimensions by calling the adh5_setup_profile() function. this function will also copy the scale information and dimension labels, which is important for the proper scaling of the result.

the meaning of the data in the result waves is up to the particular function, e.g. dest1 could hold the mean value and dest2 the one-sigma error, or dest1 could hold the X-profile, and dest2 the Y-profile.

Parameters

  source | source wave. two-dimensional intensity distribution (image). the scales are carried over to the result waves.
  param | string with optional parameters, shared between calls. this is a pass-by-reference argument, the function may modify the string.

Returns
  a free wave containing references of the result waves. the result waves should as well be free waves. if an error occurred, the reference wave is empty.

Definition at line 1110 of file pearl-area-import.ipf.

## adh5_get_result_waves()

threadsafe variable adh5_get_result_waves | ( | wave | results,
|  | string | result_prefix,
|  | variable | start_index
| ) |  |

copy waves from wave reference wave into current data folder

this function copies waves that are referenced in a wave reference wave into the current data folder. the destination waves get new names consisting of a prefix and a numeric index. the index is the array index of the wave in results plus a chosen offset.

Parameters

  results | a wave reference wave pointing to result waves from data reduction. the waves can be free or regular waves. results can be a free or regular wave.
  result_prefix | name prefix of the copied waves.
  start_index | start index (offset) of the copied waves.

Definition at line 1192 of file pearl-area-import.ipf.

## adh5_list_reduction_funcs()

string adh5_list_reduction_funcs | ( |  | ) |

get a list of functions which can be used as reduction functions.

the function evaluates only the function arguments, it may thus include functions which are not suitable as reduction functions.

Definition at line 1040 of file pearl-area-import.ipf.

## adh5_load_complete()

string adh5_load_complete | ( | string | ANickName,
|  | string | APathName,
|  | string | AFileName,
|  | variable | load_data = `defaultValue`,
|  | variable | load_attr = `defaultValue`
| ) |  |

import everything from a HDF5 file created by the Area Detector software.

if the data is from the electron analyser driver and some special attributes are included, the function will set the scales of the image dimensions.

Parameters

  ANickName | destination folder name (top level under root)
  APathName | igor symbolic path name. can be empty if the path is specified in FileName or a dialog box should be displayed
  AFileName | if empty a dialog box shows up
  load_data | 1 (default): load data; 0: do not load data
  load_attr | 1 (default): load attributes; 0: do not load attributes for proper wave scaling, the attributes must be loaded

Definition at line 206 of file pearl-area-import.ipf.

## adh5_load_detector()

variable adh5_load_detector | ( | variable | fileID,
|  | string | detectorpath
| ) |  |

load the detector dataset from the open HDF5 file.

the function loads the whole dataset at once and redimensions it so that the image dimensions are X and Y

Parameters

  fileID | ID of open HDF5 file from HDF5OpenFile
  detectorpath | path to detector group in the HDF5 file

Definition at line 580 of file pearl-area-import.ipf.

## adh5_load_detector_image()

variable adh5_load_detector_image | ( | variable | fileID,
|  | string | detectorpath,
|  | variable | dim2start,
|  | variable | dim2count,
|  | variable | dim3start,
|  | variable | dim3count
| ) |  |

load a single image from the detector dataset of the open HDF5 file

the function can average over a region in the extra dimensions.

Parameters

  fileID | ID of open HDF5 file from HDF5OpenFile
  detectorpath | path to detector group in the HDF5 file
  dim2start | 2nd dimension coordinate of the first image note that the order of dimensions is reversed in the file 2nd dimension = N dimension in area detector = dimension 0 of the three-dimensional HDF dataset set to 0 if dimension may not be present
  dim2count | number of subsequent images to average set to 1 if dimension may not be present
  dim3start | 3rd dimension coordinate of the first image note that the order of dimensions is reversed in the file 3rd dimension = extra X dimension in area detector = dimension 0 of the four-dimensional HDF dataset set to 0 if dimension may not be present
  dim3count | number of subsequent images to average set to 1 if dimension may not be present

Definition at line 944 of file pearl-area-import.ipf.

## adh5_load_detector_slabs()

variable adh5_load_detector_slabs | ( | variable | fileID,
|  | string | detectorpath,
|  | variable | progress = `defaultValue`
| ) |  |

load the detector dataset from the open HDF5 file.

the function loads the dataset image by image using the hyperslab option. this function gives the same result as adh5_load_detector. it is about 5% slower, and it depends on HDF5 Browser code. but it does not choke on large datasets (as long as the final wave fits into memory).

Parameters

  fileID | ID of open HDF5 file from HDF5OpenFile.
  detectorpath | path to detector group in the HDF5 file.
  progress | 1 (default): show progress window; 0: do not show progress window.

Returns
  0 if successful, non-zero if an error occurred.

Definition at line 779 of file pearl-area-import.ipf.

## adh5_load_info()

string adh5_load_info | ( | string | APathName,
|  | string | AFileName
| ) |  |

load descriptive info from a HDF5 file created by the Area Detector software.

the information returned is the array size and active scans

Attention
  EXPERIMENTAL this function should be merged with adh5_load_preview

Parameters

  APathName | igor symbolic path name. can be empty if the path is specified in FileName or a dialog box should be displayed
  AFileName | if empty a dialog box shows up

Definition at line 496 of file pearl-area-import.ipf.

## adh5_load_preview()

string adh5_load_preview | ( | string | ANickName,
|  | string | APathName,
|  | string | AFileName,
|  | variable | load_data = `defaultValue`,
|  | variable | load_attr = `defaultValue`
| ) |  |

load a single image from a HDF5 file created by the Area Detector software.

the data wave is loaded into the current data folder. attributes are loaded into the attr subfolder. existing waves in attr are deleted.

Warning
  EXPERIMENTAL this function uses the root:pearl_area:preview data folder. existing data there may be deleted!

Parameters

  ANickName | destination wave name. the wave is created in the current data folder.
  APathName | igor symbolic path name. can be empty if the path is specified in FileName or a dialog box should be displayed
  AFileName | if empty a dialog box shows up
  load_data | 1 (default): load data; 0: do not load data
  load_attr | 1 (default): load attributes; 0: do not load attributes note: for correct scaling of the image, the attributes need to be loaded

Definition at line 386 of file pearl-area-import.ipf.

## adh5_load_reduced()

string adh5_load_reduced | ( | string | ANickName,
|  | string | APathName,
|  | string | AFileName,
|  | funcref | reduction_func,
|  | string | reduction_param,
|  | variable | load_data = `defaultValue`,
|  | variable | load_attr = `defaultValue`,
|  | variable | progress = `defaultValue`
| ) |  |

load and reduce a dataset from a HDF5 file created by the Area Detector software.

the resulting dataset is reduced in one image dimension by a user-defined reduction function, e.g. by region-of-interest integration, curve fitting, etc.

the function loads the dataset image by image using the hyperslab option and applies a custom reduction function to each image. the results from the reduction function are composed into one result wave. the raw data are discarded.

if the data is from the electron analyser driver and some special attributes are included, the function will set the scales of the image dimensions.

Parameters

  ANickName | destination folder name (top level under root)
  APathName | igor symbolic path name. can be empty if the path is specified in FileName or a dialog box should be displayed
  AFileName | if empty a dialog box shows up
  reduction_func | custom reduction function (any user-defined function which has the same parameters as adh5_default_reduction())
  reduction_param | parameter string for the reduction function
  load_data | 1 (default): load data; 0: do not load data
  load_attr | 1 (default): load attributes; 0: do not load attributes for proper wave scaling, the attributes must be loaded
  progress | 1 (default): show progress window; 0: do not show progress window

Definition at line 301 of file pearl-area-import.ipf.

## adh5_load_reduced_detector()

variable adh5_load_reduced_detector | ( | variable | fileID,
|  | string | detectorpath,
|  | funcref | reduction_func,
|  | string | reduction_param,
|  | variable | progress = `defaultValue`,
|  | variable | nthreads = `defaultValue`
| ) |  |

load a reduced detector dataset from the open HDF5 file.

the function loads the dataset image by image using the hyperslab option and applies a custom reduction function to each image. the results from the reduction function are composed into one result wave. the raw data are discarded.

by default, the reduction function is called in separate threads to reduce the total loading time. (see the global variable adh5_perf_secs which reports the total run time of the function.) the effect varies depending on the balance between file loading (image size) and data processing (complexity of the reduction function). for debugging the reduction function, multi-threading can be disabled.

Parameters

  fileID | ID of open HDF5 file from HDF5OpenFile
  detectorpath | path to detector group in the HDF5 file
  reduction_func | custom reduction function (any user-defined function which has the same parameters as adh5_default_reduction())
  reduction_param | parameter string for the reduction function
  progress | 1 (default): show progress window; 0: do not show progress window
  nthreads | -1 (default): use as many threads as there are processor cores (in addition to main thread) 0: use main thread only (e.g. for debugging the reduction function) >= 1: use a fixed number of (additional) threads

Definition at line 1228 of file pearl-area-import.ipf.

## adh5_loadattr_all()

variable adh5_loadattr_all | ( | variable | fileID,
|  | string | attributespath
| ) |  |

load an NDAttributes group from an open HDF5 file into the current data folder.

datasets contained in the group are loaded as waves. if a dataset contains only one data point, it is added to the IN, ID, IV, IU waves, where IN = EPICS channel name, ID = attribute name, IV = value, IU = unit (units are left empty as they are not saved in HDF5). attributes of the NDAttributes group are added to the IN, ID, IV, IU waves, however, IN and IU are left empty as this information is not saved in the HDF5 file.

Parameters

  fileID | ID of open HDF5 file from HDF5OpenFile
  attributespath | path to NDAttributes group in the HDF5 file

Definition at line 1552 of file pearl-area-import.ipf.

## adh5_redim()

variable adh5_redim | ( | wave | data | ) |

redimension a multi-dimensional area detector array loaded from HDF5.

so that the image dimensions are X and Y singleton dimensions are removed (most common cases only)

in the redimensioned array, the original dimension type is noted in the dimension label: AD_Dim0 = first image dimension AD_Dim1 = second image dimension AD_DimN = frame sequence AD_DimX = extra dimension X AD_DimY = extra dimension Y (cannot be loaded in Igor)

Parameters

  data | area detector data loaded from HDF5 to be redimensioned

Definition at line 625 of file pearl-area-import.ipf.

## adh5_scale()

variable adh5_scale | ( | wave | data,
|  | string | source = `defaultValue`
| ) |  |

set the dimension scales of an area detector dataset.

the intrinsic dimensions 0 and 1 are scaled according to the data source (currently supported: Prosilica cameras, Scienta electron analyser). the extra dimensions are scaled according to the scan. the latter requires that the positioner names and position values are available.

Definition at line 725 of file pearl-area-import.ipf.

## adh5_scale_scan()

variable adh5_scale_scan | ( | wave | data | ) |

scales the extra dimensions of an area detector dataset according to the EPICS scan

the scan positioner name and its values must be available

Todo:
  incomplete

Definition at line 1792 of file pearl-area-import.ipf.

## adh5_scale_scienta()

variable adh5_scale_scienta | ( | wave | data | ) |

set the energy and angle scales of an area detector dataset from the Scienta analyser.

the dimension labels of the energy and angle scales must be set correctly: AD_Dim0 = energy dimension; AD_Dim1 = angle dimension. these dimensions must be the first two dimensions of a multi-dimensional dataset. normally, AD_Dim0 is the X dimension, and AD_Dim1 the Y dimension.

Definition at line 1687 of file pearl-area-import.ipf.

## adh5_setup_profile()

threadsafe variable adh5_setup_profile | ( | wave | image,
|  | wave | profile,
|  | variable | dim
| ) |  |

set up a one-dimensional wave for a line profile based on a 2D original wave.

redimensions the profile wave to the given dimension. copies the scale and dimension label of the given dimension.

Definition at line 1133 of file pearl-area-import.ipf.

## adh5_test_reduction_func()

string adh5_test_reduction_func | ( | wave | source,
|  | funcref | reduction_func,
|  | string | reduction_param,
|  | string | result_prefix
| ) |  |

wrapper function for testing reduction functions from the command line.

reduction functions cannot be used on the command line because they require a pass-by-reference argument and return free waves. this function expects the reduction parameters in a normal string and copies the results into the current data folder. the prefix of the result names can be specified.

Parameters

  source | source wave. two-dimensional intensity distribution (image). the scales are carried over to the result waves.
  reduction_func | name of the reduction function to apply to the source data.
  reduction_param | string with reduction parameters as required by the specific reduction function.
  result_prefix | name prefix of result waves. a numeric index is appended to distinguish the results. the index starts at 1. existing waves are overwritten.

Returns
  a copy of the reduction_param string, possibly modified by the reduction function.

Definition at line 1166 of file pearl-area-import.ipf.

## BeforeFileOpenHook()

static variable BeforeFileOpenHook | ( | variable | refNum,
|  | string | fileName,
|  | string | path,
|  | string | type,
|  | string | creator,
|  | variable | kind
| ) |  |

static

callback function for drag&drop of HDF5 files into Igor.

Definition at line 41 of file pearl-area-import.ipf.

## GetAttrDataFolderDFR()

static dfr GetAttrDataFolderDFR | ( | wave | data | ) |

static

find the attributes data folder of an area detector dataset.

since version 1.04 attributes should be stored in a subfolder named attr. earlier versions had the attributes in the same data folder as the actual dataset.

Parameters

  data | wave containing the main dataset.

Returns
  data folder reference of the attributes folder. the reference may be invalid (and default to root) if the folder cannot be found, cf. built-in DataFolderRefStatus function.

Definition at line 706 of file pearl-area-import.ipf.

## read_attribute_info()

static variable read_attribute_info | ( | string | datawavename,
|  | string | source,
|  | variable * | idest
| ) |  |

static

sub-function of adh5_loadattr_all.

reads one attribute from a wave which was loaded from an HDF5 file into the info waves IN, ID, IV, IU. the attribute is read only if the input wave contains exactly one item, i.e. either the measurement is a single image, or the attribute has string type.

Parameters

  datawavename | name of the attribute wave in the current folder. can be text or numeric.
  source | source identifier (EPICS name) of the attribute.
  idest | destination index in IN, ID, IV, IU where the results are written. the variable is incremented if data was written, otherwise it is left unchanged. make sure IN, ID, IV, IU have at least idest + 1 elements.

Definition at line 1634 of file pearl-area-import.ipf.

## reduce_slab_image()

static threadsafe wave reduce_slab_image | ( | wave | slabdata,
|  | wave | image,
|  | funcref | reduction_func,
|  | string | reduction_param
| ) |  |

static

Definition at line 1519 of file pearl-area-import.ipf.

## reduce_slab_worker()

static threadsafe variable reduce_slab_worker | ( | funcref | reduction_func | ) |

static

Definition at line 1480 of file pearl-area-import.ipf.