bartels-rausch
091cf00e77
* Transitioned nappNorm, nappDetermineShift, and nappCopyShift to use modern MATLAB arguments blocks, removing their legacy functionSignatures.json entries.
* Refactored nappNorm to rely on prior peak fitting results (e.g., from nappFitGaussXps) for normalization rather than performing the fitting internally.
* Decoupled energy calibration: nappDetermineShift now solely calculates internal binding energy shifts, and nappCopyShift distributes them using a unified matching engine ('earlier', 'after', 'spectrumNumber').
* Implemented a Universal Logging Block to track data provenance, automatically appending execution timestamps and caller script names to processed rows.
* Updated nappCombineSpectra to output a logDataSource cell array and cleanly name combined spectra.
* Created nappSaveDataTable for automatic, script-named .mat file saving, complete with a Live Editor interceptor.
* Added a publish-mode silencer to nappFitGaussXps to prevent optimizer spam in generated PDFs.
* Updated readme.md to document the new arguments structure, the 3-step calibration workflow, the matching engine, and the new table variables.
1951 lines
87 KiB
Matlab
Executable File
1951 lines
87 KiB
Matlab
Executable File
function ListFiles = nappFileList(dataPath,varargin)
|
|
%NAPPFILELIST Import spectroscopy data (XPS, NEXAFS) from a beam time folder into data table.
|
|
% Photos, analyser and sample settings such as pass energy, lens mode, temperature,
|
|
% position, and pressure are automatically extracted and added to the table.
|
|
%
|
|
% SYNTAX
|
|
% ------
|
|
% ListFiles = NAPPFILELIST(dataPath)
|
|
% ListFiles = NAPPFILELIST(dataPath, DataTable)
|
|
% ListFiles = NAPPFILELIST(dataPath, DataTable, 'Name', Value)
|
|
%
|
|
% DESCRIPTION
|
|
% -----------
|
|
% NAPPFILELIST scans a beam time folder and imports all experimental data
|
|
% files (XPS, NEXAFS, and photos) with their associated
|
|
% metadata into a MATLAB table. The function automatically:
|
|
% - Reads XPS data and metadata from SES (.ibw) or Specs (.xy) files
|
|
% - Reads NEXAFS data and metadata from HDF5 (.h5) files
|
|
% - Splits multi-scan HDF5 files into individual single-scan files
|
|
% - Extracts instrument settings (pass energy, dwell time, lens mode, etc.)
|
|
% - Normalizes count rates (per dwell time) to counts per second (cps)
|
|
% - Averages region iterations during import
|
|
% - Creates a semicolon-separated CSV file for Excel import
|
|
%
|
|
% REQUIRED INPUT ARGUMENTS
|
|
% ------------------------
|
|
% dataPath
|
|
% Beam time folder path (string or char vector).
|
|
% Format: 'YYYY/BeamtimeName'(Mac,Linux,Unix) or 'YYYY\BeamtimeName' (Windows)
|
|
% Examples: '2025/November_NappIss', '2024/March_PolLux'
|
|
%
|
|
% Note: The folder structure must follow the strict naming convention
|
|
% described in the "Folder Structure" section below.
|
|
%
|
|
% OPTIONAL POSITIONAL INPUT
|
|
% -------------------------
|
|
% DataTable
|
|
% Existing data table to append imported data to (table object).
|
|
% If provided, new files are merged with existing table.
|
|
% If not provided, a new table is created from scratch.
|
|
%
|
|
% NAME-VALUE PAIR ARGUMENTS
|
|
% -------------------------
|
|
% 'mode'
|
|
% Import mode (char or string, default: 'local').
|
|
% Supported values:
|
|
% 'local' - Import from local computer (default)
|
|
% 'nappDrive' - Import from mounted NAPP network drive
|
|
% 'dataDrive' - Import from mounted group drive (5505)
|
|
% 'localNew' - Import only new files not in existing DataTable
|
|
% 'nappDrivePick' - Browse and select files from NAPP drive
|
|
% 'localPick' - Browse and select files from local folder
|
|
% 'dataPick' - Browse and select files from mounted group drive
|
|
%
|
|
% 'importLabView'
|
|
% Import LabView pressure/temperature logs (char or string).
|
|
% Default: 'on'
|
|
% Supported values:
|
|
% 'on' - Import LabView logs from Pressure subfolder
|
|
% 'off' - Skip LabView log import (this is much,much faster)
|
|
%
|
|
% OUTPUT ARGUMENTS
|
|
% ----------------
|
|
% ListFiles
|
|
% MATLAB table with the following columns (variables):
|
|
% - name: file name (string)
|
|
% - folder: full path to file (string)
|
|
% - date: file modification date (datetime)
|
|
% - bytes: file size in bytes (numeric)
|
|
% - isdir: directory flag (logical)
|
|
% - datenum: date as serial number (numeric)
|
|
% - creationDate: acquisition timestamp (datevec format)
|
|
% - image: 2D spectral map or image (cell array)
|
|
% - imageRoh: raw 2D spectral map (cell array)
|
|
% - spectrum_counts: integrated spectrum (numeric or cell array)
|
|
% - spectrum_countsPerSecond: mean spectrum normalized to cps
|
|
% - scientaEkin_eV: kinetic energy axis (cell array, eV)
|
|
% - scientaEkinRange_eV: [min max] kinetic energy range
|
|
% - scientaPassEnergy_eV: analyzer pass energy (numeric, eV)
|
|
% - scientaDwellTime_ms: dwell time per channel (numeric, ms)
|
|
% - regionName: region label (e.g., 'C1s', 'O K-edge NEXAFS')
|
|
% - scientaAcquisitionMode: 'Swept' or 'Fixed' mode
|
|
% - scientaLensMode: lens transmission mode (string)
|
|
% - scientaRegionIterations: number of region scans (numeric)
|
|
% - scientaSequenceIterations: number of sequence repeats (numeric)
|
|
% - xRayEkin_eV: X-ray photon energy for XPS (numeric or cell array, eV)
|
|
% - xRayEkinRange_eV: [min max] photon energy range
|
|
% - beamlineInt: beamline intensity (I_0 for NEXAFS)
|
|
% - importDate: timestamp of table generation (char)
|
|
% - analysisDir: working directory at import time (string)
|
|
%
|
|
% Also creates: YYYYMmm_DataList.csv
|
|
% Semicolon-separated ASCII file in beam time folder for Excel import.
|
|
%
|
|
% FOLDER STRUCTURE & NAMING CONVENTIONS
|
|
% =====================================
|
|
%
|
|
% REQUIRED DIRECTORY STRUCTURE
|
|
% ----------------------------
|
|
% The beam time folder must contain the following sub-directories:
|
|
%
|
|
% SES/ - XPS data in Igor (.ibw) or Specs (.xy) format
|
|
% NEXAFS/ - NEXAFS maps in HDF5 (.h5) format
|
|
% Photos/ - Sample photographs in BMP (.bmp) format
|
|
% Pressure/ - LabView pressure and temperature logs
|
|
% PShell/ - XPS data in h5 format (from PShell)
|
|
%
|
|
% Full path convention:
|
|
% LOCAL: ~/Forschung_Einsam/PSI_SLS/NAPP/SC Data/YYYY/BeamtimeName
|
|
% NAPP DRIVE: /Volumes/NAPP/SC Data/YYYY/BeamtimeName (Mac)
|
|
% \\fs03\NAPP\SC Data\YYYY\BeamtimeName (Windows)
|
|
% DATA DRIVE: /Volumes/5505/Data/YYYY/BeamtimeName (Mac)
|
|
% \\fs101\5505\Data\YYYY\BeamtimeName (Windows)
|
|
%
|
|
% EXAMPLE FOLDER LAYOUT
|
|
% ---------------------
|
|
% 2025/November_NappIss/
|
|
% SES/ [XPS data]
|
|
% 0001_C1s_525eV.ibw
|
|
% 0002_O1s_850eV.ibw
|
|
% 0003_Survey_850eV.ibw
|
|
% ...
|
|
% NEXAFS/ [NEXAFS maps]
|
|
% 0010_O_Kedge.h5
|
|
% 0011_N_Kedge.h5
|
|
% ...
|
|
% Photos/ [Sample photos]
|
|
% A0001-20251123_102407.bmp
|
|
% A0002-20251123_102408.bmp
|
|
% ...
|
|
% Pressure/ [Environmental logs]
|
|
% LabView_T_RH_20251119.txt
|
|
% logfile-23-11-2025-12h46min.txt
|
|
%
|
|
% FILE NAMING CONVENTIONS
|
|
% -----------------------
|
|
%
|
|
% XPS FILES (.ibw, .xy)
|
|
% Format: NNNN_RegionName_PhotonEnergy.ext
|
|
% Example: 0191_C1s_525eV.ibw
|
|
% 0192_O1s_850eV.ibw
|
|
% 0045_Survey_1000eV.ibw
|
|
%
|
|
% NNNN = Four-digit SES file number (from instrument)
|
|
% RegionName = XPS region: C1s, O1s, N1s, Si2p, Cl2p, Survey, etc.
|
|
% PhotonEnergy = X-ray photon energy in eV
|
|
% .ibw = Igor file format (Scienta SES analyzer)
|
|
% .xy = Specs SpecsLab file format
|
|
%
|
|
% NEXAFS FILES (.h5)
|
|
% Format: NNNN_ElementName_EdgeType.h5
|
|
% Example: 0200_O_Kedge.h5
|
|
% 0201_N_Kedge.h5
|
|
% 0202_C_Kedge.h5
|
|
%
|
|
% NNNN = Four-digit file number
|
|
% ElementName = Element symbol: O, N, C, etc.
|
|
% EdgeType = K-edge, L-edge, etc.
|
|
%
|
|
% Note: Files starting with "N" are recognized. Files starting with
|
|
% other characters (e.g., from old naming schemes) will be ignored.
|
|
%
|
|
% PHOTO FILES (.bmp)
|
|
% Format: descriptive_name.bmp
|
|
% Example: sample_overview.bmp
|
|
% before_dosing.bmp
|
|
% after_annealing.bmp
|
|
%
|
|
% DATA FILE REQUIREMENTS & LIMITATIONS
|
|
% ====================================
|
|
%
|
|
% - At least ONE .ibw or .xy file must exist in SES/ subfolder
|
|
% - Do NOT run while SES or NEXAFS data are being actively recorded
|
|
% - Multi-scan HDF5 files are automatically split (one scan per file)
|
|
% - Region iterations are averaged during import (not preserved individually)
|
|
% - Counts are normalized to 1 second dwell time (results in cps)
|
|
% - System files starting with '.' or '~' are automatically excluded
|
|
%
|
|
% HDF5 MULTI-SCAN HANDLING
|
|
% -------------------------
|
|
% If an HDF5 file contains multiple scans (>3 groups detected):
|
|
% 1. Function calls splitH5Scans(filename)
|
|
% 2. Each scan is extracted into a separate file
|
|
% 3. New files: original_name_scan_1.h5, original_name_scan_2.h5, etc.
|
|
% 4. Original multi-scan file is deleted after splitting
|
|
%
|
|
% EXAMPLES
|
|
% ========
|
|
%
|
|
% % Example 1: Import all data from local folder
|
|
% FileList = nappFileList('2025/November_NappIss', 'mode', 'local');
|
|
%
|
|
% % Example 2: Add new data to existing table
|
|
% FileList = nappFileList('2025/November_NappIss', FileList, 'mode', 'local');
|
|
%
|
|
% % Example 3: Import only new files not yet in table
|
|
% FileList = nappFileList('2025/November_NappIss', FileList, ...
|
|
% 'mode', 'localNew');
|
|
%
|
|
% % Example 4: Import from network drive with file browser
|
|
% FileList = nappFileList('2025/November_NappIss', ...
|
|
% 'mode', 'nappDrivePick');
|
|
%
|
|
% % Example 5: Skip LabView pressure/temperature logs
|
|
% FileList = nappFileList('2025/November_NappIss', ...
|
|
% 'mode', 'local', 'importLabView', 'off');
|
|
%
|
|
% DEPENDENCIES
|
|
% ============
|
|
% - IBWread function (Igor Pro format reader for MATLAB)
|
|
% https://ch.mathworks.com/matlabcentral/fileexchange/42679-igor-pro-file-format-ibw-to-matlab-variable
|
|
% - h5info, h5read (HDF5 file support, built-in MATLAB)
|
|
% - splitH5Scans function (custom, included in toolkit)
|
|
% - LabView_importData function (custom, included in toolkit)
|
|
%
|
|
% TROUBLESHOOTING
|
|
% =======================
|
|
%
|
|
% - If "path wrong" warning appears: check folder spelling and that
|
|
% network drives are mounted
|
|
% - If Igor files fail to read: verify files are complete and not corrupted
|
|
% (partially recorded files should be renamed with "ignore_" prefix)
|
|
% - For older data (pre-2020): HDF5 file structure may differ; function
|
|
% attempts multiple path conventions automatically
|
|
% - Mac: Creation date for NEXAFS files obtained via 'stat' command
|
|
% - Windows: Creation date for NEXAFS files set to NaN
|
|
%
|
|
% MATLAB VERSION REQUIREMENTS
|
|
% ===========================
|
|
% - MATLAB 2018a or higher (for '**' notation in dir patterns)
|
|
%
|
|
% HISTORY & UPDATES
|
|
% =================
|
|
% - January 2020: Added new h5 file structure and naming convention
|
|
% - March 2021: Added local file structure support (not just network)
|
|
% - March 2021: Added Lens Mode column support
|
|
% - April 2021: X-ray kinetic energy and images now made unique
|
|
% - March 2022: Refactored to use varargin instead of positional arguments
|
|
% - March 2022: Strict data structure required for local and remote sources
|
|
% - November 2025: Added support for PShell h5 file structure (NEXAFS)
|
|
% - November 2025: Added creation date extraction for Mac (NEXAFS)
|
|
%
|
|
% See also: TABLE, IBWread, h5read, h5info, DIR, inputParser
|
|
%
|
|
% Copyright: Laboratory of Atmospheric Chemistry, Paul Scherrer Institut (PSI)
|
|
% thorsten.bartels-rausch@psi.ch
|
|
%
|
|
% =========================================================================
|
|
%% Updated
|
|
%
|
|
% * January 2020, added new h5 file structure and naming convention.
|
|
% * March 2021, added possibility to work on local file structure
|
|
% * March 2021, added Lens Mode as column (have not tested this for older
|
|
% NEXAFS files).
|
|
% * April 2021, X-ray kinetic energy and images are now made unique
|
|
% * March 2022, input now using varargin
|
|
% * March 2022, strict data structure required for local and remote sources
|
|
% * November 2025, PShell h5 files for NEXAFS have a new structure;
|
|
% * November 2025, added creation date to mac for NEXAFS.
|
|
% * March 2026, added movie import from photos folder, added jpg as photo
|
|
% file
|
|
|
|
%% Input
|
|
% Define conditions and defaults of input
|
|
defaultMode = 'local';
|
|
expectedMode = {'local','nappDrive','dataDrive','localNew','nappDrivePick','localPick','dataPick'};
|
|
defaultImportLabView = 'on';
|
|
expectedImportLabView = {'on','off'};
|
|
% Parse input
|
|
p = inputParser;
|
|
addRequired(p,'dataPath');
|
|
addOptional(p,'DataTable',@istable);
|
|
addParameter(p,'mode',defaultMode,...
|
|
@(x) any(validatestring(x,expectedMode)));
|
|
addParameter(p,'importLabView',defaultImportLabView,...
|
|
@(x) any(validatestring(x,expectedImportLabView)));
|
|
parse(p,dataPath,varargin{:});
|
|
%% Data Paths
|
|
%
|
|
% Mac and Windows use different notations to address network drives. For
|
|
% windows, we use the UNC naming convention. This is a more recent way to
|
|
% address mounted drives that does not rely on the use of drive letters.
|
|
% The letters might be different for the same drive on different computer.
|
|
|
|
|
|
if contains(p.Results.mode,'local')
|
|
% Data server address
|
|
if ismac
|
|
server = '/Users/bartels-rausch/Documents/Forschung_Einsam/PSI_SLS/';
|
|
elseif ispc
|
|
server = userpath;
|
|
end
|
|
elseif contains(p.Results.mode,'nappDrive')
|
|
% Data server address
|
|
if ismac
|
|
server = '/Volumes/';
|
|
elseif ispc
|
|
server = '\\fs03\';
|
|
end
|
|
elseif contains(p.Results.mode,'data')
|
|
% Data server address
|
|
if ismac
|
|
server = '/Volumes/';
|
|
elseif ispc
|
|
server = '\\fs101\';
|
|
end
|
|
end
|
|
if contains(p.Results.mode,'data')
|
|
dataPath = fullfile(server,'5505','Data','beamtimes',dataPath);
|
|
else
|
|
dataPath = fullfile(server,'NAPP','SC Data',dataPath);
|
|
end
|
|
% Data Paths to subdirectories for specific files:
|
|
dataPath_XPS = fullfile(dataPath, 'SES');
|
|
dataPath_Photo = fullfile(dataPath, 'Photos');
|
|
dataPath_NEXAFS = fullfile(dataPath, 'NEXAFS');
|
|
dataPath_LabView = fullfile(dataPath, 'Pressure');
|
|
|
|
%
|
|
if ~isfolder(dataPath_XPS)
|
|
warning('There is something wrong with the path. Check for typo and that the drives are mounted')
|
|
disp('There is something wrong with the path. Check for typo and that the drives are mounted')
|
|
disp('I quit now')
|
|
return
|
|
end
|
|
|
|
|
|
%% Naming Convention
|
|
%
|
|
% See: MATLAB Programming Style Guidelines, by Richard Johnson
|
|
%
|
|
% * Variable: mixed case words starting with lower case: credibleThreat, qualityOfLife
|
|
% * Variables with a large scope should have meaningful names. Variables
|
|
% with a small scope can have short names.
|
|
% * A convention on pluralization should be followed consistently.
|
|
% * Structure names should begin with a capital letter: SegmentData.length
|
|
% * The prefix n should be used for variables representing the number of
|
|
% objects. A MATLAB-specific addition is the use of m for number of rows
|
|
% (based on matrix notation). nFiles, mRows
|
|
% * Iterator variables should be named or prefixed with i. for iPosition =
|
|
% 1:nPositions, ii, ij, ik....
|
|
% * Functions should have meaningful names written in lower case.
|
|
% computetotalwidth(), Avoid compwid();
|
|
|
|
%% Query beam time folder content
|
|
%
|
|
% We use MATLAB's 'dir' command which writes the content of a folder into a
|
|
% structure with the following fields:
|
|
%
|
|
% * name
|
|
% * folder
|
|
% * date
|
|
% * bytes
|
|
% * isdir
|
|
% * datenum
|
|
%
|
|
% XPS data: Create a structure 'TmpFileList{1}' with all Igor files in the
|
|
% directory and its subfolders (1st level). The '\N*' at the beginning is
|
|
% important. Otherwise hidden files starting with a . will also be listed.
|
|
% Obviously, Igor file names from SES software and h5 file names from Epics
|
|
% (NEXAFS) need to start with an N, otherwise change the search term below:
|
|
%
|
|
% THE ** notation needs a recent version of Matlab, 2014 does not work.
|
|
|
|
if endsWith(p.Results.mode, 'Pick')
|
|
% --- PICK MODE ---
|
|
[filePick, pathPick] = uigetfile({'*.ibw;*.xy;*.h5'}, ...
|
|
'Select One or More Files', dataPath, 'MultiSelect', 'on');
|
|
|
|
if ischar(filePick), filePick = {filePick}; end % Force cell array
|
|
|
|
% Initialize empty dir structs
|
|
TmpFileList{1} = struct('name',{},'folder',{},'date',{},'bytes',{},'isdir',{},'datenum',{});
|
|
TmpFileList{2} = TmpFileList{1};
|
|
TmpFileList{3} = TmpFileList{1}; % Photos always empty in pick mode
|
|
|
|
% Populate XPS and NEXAFS
|
|
if ~isequal(filePick, 0) % If user didn't cancel
|
|
for ii = 1:length(filePick)
|
|
if endsWith(filePick{ii}, {'.ibw', '.xy'}, 'IgnoreCase', true)
|
|
TmpFileList{1} = [TmpFileList{1}; dir(fullfile(pathPick, filePick{ii}))];
|
|
elseif endsWith(filePick{ii}, '.h5', 'IgnoreCase', true)
|
|
TmpFileList{2} = [TmpFileList{2}; dir(fullfile(pathPick, filePick{ii}))];
|
|
end
|
|
end
|
|
end
|
|
|
|
else
|
|
|
|
% --- FOLDER SEARCH MODE ---
|
|
|
|
% 1. XPS (Both .ibw and .xy)
|
|
tmpIbw = dir(fullfile(dataPath_XPS, '**', '*.ibw'));
|
|
tmpIbw = tmpIbw(find(~cellfun(@isempty,regexp({tmpIbw .name},'^[a-zA-Z0-9]\w*')))); %#ok<FNDSB>
|
|
tmpXy = dir(fullfile(dataPath_XPS, '**', '*.xy'));
|
|
tmpXy = tmpXy(find(~cellfun(@isempty,regexp({tmpXy .name},'^[a-zA-Z0-9]\w*')))); %#ok<FNDSB>
|
|
TmpFileList{1} = [tmpIbw; tmpXy]; % Stack them together
|
|
|
|
% 2. NEXAFS
|
|
TmpFileList{2} = dir(fullfile(dataPath_NEXAFS, '**', '*.h5'));
|
|
TmpFileList{2} = TmpFileList{2}(find(~cellfun(@isempty,regexp({TmpFileList{2}.name},'^[a-zA-Z0-9]\w*')))); %#ok<FNDSB>
|
|
|
|
|
|
% 3. Photos
|
|
photoExts = {'*.bmp', '*.jpg', '*.JPG', '*.mov', '*.avi', '*.mp4', '*.wmv'};
|
|
TmpFileList{3} = [];
|
|
for i = 1:length(photoExts)
|
|
TmpFileList{3} = [TmpFileList{3}; dir(fullfile(dataPath_Photo, '**', photoExts{i}))];
|
|
end
|
|
TmpFileList{3} = TmpFileList{3}(find(~cellfun(@isempty,regexp({TmpFileList{3}.name},'^[a-zA-Z0-9]\w*')))); %#ok<FNDSB>
|
|
|
|
% Clean up system/hidden files for all lists
|
|
for i = 1:3
|
|
if ~isempty(TmpFileList{i})
|
|
% Keep only files that do NOT start with '.' or '~'
|
|
validIdx = ~startsWith({TmpFileList{i}.name}, {'.', '~'});
|
|
TmpFileList{i} = TmpFileList{i}(validIdx);
|
|
end
|
|
end
|
|
|
|
% =========================================================================
|
|
% SPLIT HDF5 FILES WITH MULTIPLE DATASETS INTO INDIVIDUAL FILES
|
|
% =========================================================================
|
|
%
|
|
% BACKGROUND:
|
|
% HDF5 files from the beamline can contain multiple experimental scans
|
|
% (e.g., scan_1, scan_2, scan_3) stored as separate groups within a
|
|
% single file. However, the NEXAFS and XPS data processing pipelines
|
|
% are designed to work with one scan per file.
|
|
%
|
|
% PURPOSE:
|
|
% This script identifies HDF5 files that contain multiple scans and
|
|
% splits each scan into a separate file with a descriptive name.
|
|
% Each output file contains only one scan (renamed to scan_1) plus
|
|
% the logs and metadata groups.
|
|
%
|
|
% PROCESS:
|
|
% 1. Loop through all NEXAFS HDF5 files in the file list
|
|
% 2. Check if each file contains more than 3 groups (indicating multiple scans)
|
|
% 3. If it does, split it using the splitH5Scans function
|
|
% 4. The original file is deleted after splitting
|
|
%
|
|
% OUTPUT:
|
|
% For each multi-scan file, creates separate output files:
|
|
% Example: 0002_..._O_Kedge.h5 0002_..._O_Kedge_scan_1.h5
|
|
% 0002_..._O_Kedge_scan_2.h5
|
|
% =========================================================================
|
|
% Loop through each file in the NEXAFS file list
|
|
% TmpFileList{2} contains the full paths to all NEXAFS HDF5 files
|
|
% ii is the loop counter (1, 2, 3, ... up to the total number of files)
|
|
|
|
% Initialize a map to hold the original dates for split files
|
|
splitFileDatesMap = containers.Map('KeyType', 'char', 'ValueType', 'any');
|
|
|
|
for ii = 1:size(TmpFileList{2}, 1)
|
|
|
|
% Get the filename of the current file being processed
|
|
current_file = [TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name];
|
|
|
|
% Read information about the structure of the HDF5 file
|
|
% This tells us what groups (datasets) are stored in the file
|
|
fileinfo = h5info(current_file);
|
|
|
|
|
|
% Extract the names of all groups at the root level
|
|
% Example: {'/scan_1', '/scan_2', '/logs', '/__DATA_TYPES__'}
|
|
groupnames = {fileinfo.Groups.Name};
|
|
|
|
% Check if this file has more than 3 groups
|
|
% Files with more than 3 groups likely contain multiple scans
|
|
% (the extra groups beyond scans are typically '/logs' and '/__DATA_TYPES__')
|
|
if numel(groupnames) > 3
|
|
% 1. Get the true historical dates BEFORE splitting
|
|
[orig_cDateOS, orig_mDateOS] = getOSDates(current_file);
|
|
|
|
% 2. Split the file and get the names of the newly created files
|
|
% This file contains multiple scans, so split it
|
|
% The splitH5Scans function will:
|
|
% - Create separate files for each scan
|
|
% - Rename each scan to '/scan_1' within its new file
|
|
% - Delete the original multi-scan file when done
|
|
newlyCreatedFiles = splitH5Scans(current_file);
|
|
|
|
% 3. Store the historical dates in the map using the new filenames as keys
|
|
for k = 1:length(newlyCreatedFiles)
|
|
splitFileDatesMap(newlyCreatedFiles{k}) = {orig_cDateOS, orig_mDateOS};
|
|
end
|
|
|
|
end
|
|
|
|
end
|
|
|
|
% =========================================================================
|
|
% NOTE: After this loop completes, all NEXAFS files in data folder
|
|
% will contain exactly one scan per file, ready for processing
|
|
% =========================================================================
|
|
|
|
% EPICS files in h5 format list all files, then get rid of system files
|
|
TmpFileList{2} = dir([dataPath_NEXAFS filesep '**' filesep '*.h5']);
|
|
TmpFileList{2} = TmpFileList{2}(find(~cellfun(@isempty,regexp({TmpFileList{2}.name},'^[a-zA-Z0-9]\w*')))); %#ok<FNDSB>
|
|
|
|
disp('*************************************************')
|
|
disp('* Voila: Splitting NEXAFS files done *')
|
|
disp('*************************************************')
|
|
|
|
% --- Date of First File Check ---
|
|
% Since TmpFileList{1} now safely contains all .ibw and .xy files,
|
|
% we don't need to run dir() again! We just check the struct directly.
|
|
if isempty(TmpFileList{1})
|
|
warning('There appears to be no XPS file (.ibw or .xy) in the folder. Add at least one file first.');
|
|
return;
|
|
end
|
|
|
|
try
|
|
filedate_first = TmpFileList{1}(1).datenum;
|
|
catch
|
|
warning('There appears to be something wrong with the XPS file metadata.');
|
|
end
|
|
|
|
end
|
|
%% Import XPS data to get SES settings
|
|
%
|
|
% Uses the IgorPro importer for Matlab:
|
|
% https://ch.mathworks.com/matlabcentral/fileexchange/42679-igor-pro-file-format-ibw-to-matlab-variable
|
|
%
|
|
% Normally, Matlab stops when an error occurs. I use 'try' to force Matlab
|
|
% to continue even if an error occurs. Matlab will instead print a warning,
|
|
% as defined after the 'catch'. Error might occur when single data files
|
|
% can't be read. If a warning comes up, check the corresponding data file
|
|
% and try to resolve it. I suggest to rename these original data files that
|
|
% are damaged, or not complete (because measurement was aborted) to
|
|
% "ignore_filename".
|
|
%% 2. Import XPS data
|
|
ProcessedXPS = []; % This will hold our final list of structs
|
|
|
|
if ~isempty(TmpFileList{1})
|
|
for ii = 1:length(TmpFileList{1})
|
|
current_file_info = TmpFileList{1}(ii);
|
|
current_path = fullfile(current_file_info.folder, current_file_info.name);
|
|
|
|
% =================================================================
|
|
% 1. INITIALIZE MASTER TEMPLATE (Strict Definition)
|
|
% =================================================================
|
|
% Every variable MUST be declared here so the struct sizes match perfectly
|
|
|
|
current_file_info.creationDate = [NaN NaN NaN NaN NaN NaN];
|
|
current_file_info.lastModifiedDatenum = current_file_info.datenum;
|
|
current_file_info.lastModifiedDatestr = current_file_info.date;
|
|
current_file_info.image = {[]};
|
|
current_file_info.imageRoh = {[]};
|
|
current_file_info.xRayEkin_eV = NaN;
|
|
current_file_info.xRayEkinRange_eV = [NaN NaN];
|
|
current_file_info.scientaPassEnergy_eV = NaN;
|
|
current_file_info.scientaDwellTime_ms = NaN;
|
|
current_file_info.regionName = '';
|
|
current_file_info.scientaAcquisitionMode = '';
|
|
current_file_info.scientaEkinRange_eV = [NaN NaN];
|
|
current_file_info.scientaEkinStep_eV = NaN;
|
|
current_file_info.scientaEkin_eV = {[]};
|
|
current_file_info.scientaEkinRoh_eV = {[]};
|
|
current_file_info.beamlineInt = {[]};
|
|
current_file_info.scientaLensMode = '';
|
|
current_file_info.scientaAnalyzerSlit = '';
|
|
current_file_info.scientaWorkfunction_eV = NaN;
|
|
current_file_info.scientaRegionIterations = NaN;
|
|
current_file_info.scientaSequenceIterations = NaN;
|
|
current_file_info.importDate = datestr(now);
|
|
try
|
|
current_file_info.analysisDir = file.mFile;
|
|
catch
|
|
current_file_info.analysisDir = pwd;
|
|
end
|
|
current_file_info.sample = '';
|
|
% =================================================================
|
|
% --- Case A: Scienta (.ibw) Files ---
|
|
% =================================================================
|
|
if endsWith(current_file_info.name, '.ibw', 'IgnoreCase', true)
|
|
try
|
|
PE = IBWread(current_path);
|
|
fclose('all');
|
|
tmp = regexp(PE.WaveNotes, '\r', 'split');
|
|
|
|
try
|
|
dateStr = [char(regexp(tmp{29},'((?<=Date=).*)','match')) ' ' char(regexp(tmp{30},'((?<=Time=).*)','match'))];
|
|
parsedDate = datevec(dateStr);
|
|
if size(parsedDate, 2) == 6
|
|
current_file_info.creationDate = parsedDate;
|
|
end
|
|
catch
|
|
end
|
|
|
|
current_file_info.scientaPassEnergy_eV = str2double(char(regexp(tmp{5},'((?<=Pass Energy=).*)','match')));
|
|
current_file_info.scientaDwellTime_ms = str2double(char(regexp(tmp{15},'((?<=Step Time=).*)','match')));
|
|
current_file_info.scientaEkinStep_eV = str2double(char(regexp(tmp{14},'((?<=Energy Step=).*)','match')));
|
|
current_file_info.regionName = char(regexp(tmp{3},'((?<=Region Name=).*)','match'));
|
|
current_file_info.scientaAcquisitionMode = char(regexp(tmp{9},'((?<=Acquisition Mode=).*)','match'));
|
|
current_file_info.scientaLensMode = char(regexp(tmp{4},'((?<=Lens Mode=).*)','match'));
|
|
|
|
PE.x = PE.x0(1) + (0:size(PE.y,1)-1)*PE.dx(1);
|
|
current_file_info.scientaEkin_eV = {PE.x};
|
|
current_file_info.scientaEkinRange_eV = [PE.x(1) PE.x(end)];
|
|
|
|
if PE.Ndim == 3
|
|
current_file_info.scientaRegionIterations = size(PE.y,2);
|
|
current_file_info.scientaSequenceIterations = size(PE.y,3);
|
|
PE.y = squeeze(mean(PE.y,2));
|
|
else
|
|
current_file_info.scientaRegionIterations = 1;
|
|
current_file_info.scientaSequenceIterations = size(PE.y,2);
|
|
end
|
|
|
|
PE.y = PE.y .* 1000 ./ current_file_info.scientaDwellTime_ms;
|
|
current_file_info.image = {PE.y};
|
|
current_file_info.imageRoh = {PE.y};
|
|
|
|
if isempty(ProcessedXPS), ProcessedXPS = current_file_info; else, ProcessedXPS = [ProcessedXPS; current_file_info]; end %#ok<AGROW>
|
|
catch ME
|
|
fprintf('Warning: Failed to parse %s (%s)\n', current_file_info.name, ME.message);
|
|
if isempty(ProcessedXPS), ProcessedXPS = current_file_info; else, ProcessedXPS = [ProcessedXPS; current_file_info]; end %#ok<AGROW>
|
|
end
|
|
|
|
% =================================================================
|
|
% --- Case B: Specs (.xy) Files ---
|
|
% =================================================================
|
|
elseif endsWith(current_file_info.name, '.xy', 'IgnoreCase', true)
|
|
try
|
|
[cDateOS, mDateOS] = getOSDates(current_path);
|
|
try, osDateVec = datevec(datetime(cDateOS, 'InputFormat', 'dd-MMM-yyyy HH:mm:ss'));
|
|
catch, try osDateVec = datevec(mDateOS); catch, osDateVec = [NaN NaN NaN NaN NaN NaN]; end, end
|
|
if isempty(osDateVec) || size(osDateVec, 1) == 0, osDateVec = [NaN NaN NaN NaN NaN NaN]; end
|
|
|
|
fileLines = readlines(current_path);
|
|
regionStarts = find(startsWith(fileLines, '# Region:'));
|
|
|
|
% --- 1. PARSE GLOBAL HEADER (Before First Region) ---
|
|
% SpecsLab moved many parameters to the top of the file in newer versions
|
|
globalScanMode = '';
|
|
globalLensMode = '';
|
|
globalAnalyzerSlit = '';
|
|
globalWorkFunction = NaN;
|
|
globalExEnergy = NaN;
|
|
globalSample = '';
|
|
|
|
if ~isempty(regionStarts)
|
|
globalHeader = fileLines(1:regionStarts(1)-1);
|
|
for L = 1:length(globalHeader)
|
|
lineStr = strtrim(globalHeader(L));
|
|
lineStr = regexprep(lineStr, '^#\s+', '# '); % Normalize spaces after #
|
|
|
|
if startsWith(lineStr, '# Scan Mode:')
|
|
globalScanMode = strtrim(extractAfter(lineStr, '# Scan Mode:'));
|
|
elseif startsWith(lineStr, '# Analyzer Lens Voltage:')
|
|
globalLensMode = strtrim(extractAfter(lineStr, '# Analyzer Lens Voltage:'));
|
|
elseif startsWith(lineStr, '# Analyzer Lens:')
|
|
globalLensMode = strtrim(extractAfter(lineStr, '# Analyzer Lens:'));
|
|
elseif startsWith(lineStr, '# Analyzer Slit:')
|
|
globalAnalyzerSlit = strtrim(extractAfter(lineStr, '# Analyzer Slit:'));
|
|
elseif startsWith(lineStr, '# Work Function:')
|
|
globalWorkFunction = sscanf(strtrim(extractAfter(lineStr, '# Work Function:')), '%f');
|
|
elseif startsWith(lineStr, '# Eff. Workfunction:')
|
|
globalWorkFunction = sscanf(strtrim(extractAfter(lineStr, '# Eff. Workfunction:')), '%f');
|
|
elseif startsWith(lineStr, '# Excitation Energy:')
|
|
globalExEnergy = sscanf(strtrim(extractAfter(lineStr, '# Excitation Energy:')), '%f');
|
|
elseif startsWith(lineStr, '# Comment:')
|
|
globalSample = strtrim(extractAfter(lineStr, '# Comment:'));
|
|
% =========================================================================
|
|
% OFFLINE XPS BUG FIX (Mg vs Al Source)
|
|
% If the file lacks "Monochromator" in the header and the energy is
|
|
% incorrectly tagged as ~1453.6 or 1486.6, overwrite it to 1253.6 (Mg Ka).
|
|
% =========================================================================
|
|
if ~any(contains(string(fileLines), 'Monochromator', 'IgnoreCase', true))
|
|
% Checking both 1453.6 (as requested) and 1486.61 (as seen in your file)
|
|
if (abs(globalExEnergy - 1453.61) < 1) || (abs(globalExEnergy - 1486.61) < 1)
|
|
globalExEnergy = 1253.6;
|
|
end
|
|
end
|
|
% =========================================================================
|
|
|
|
end
|
|
end
|
|
end
|
|
|
|
for r = 1:length(regionStarts)
|
|
regionEntry = current_file_info;
|
|
startIdx = regionStarts(r);
|
|
if r < length(regionStarts), endIdx = regionStarts(r+1)-1; else, endIdx = length(fileLines); end
|
|
regionBlock = fileLines(startIdx:endIdx);
|
|
|
|
% =====================================================
|
|
% 2. Extract Metadata
|
|
% =====================================================
|
|
% Initialize defaults
|
|
rName = 'UnknownRegion';
|
|
exEnergyStr = num2str(globalExEnergy);
|
|
specID = r;
|
|
regionEntry.xRayEkin_eV = globalExEnergy;
|
|
regionEntry.scientaPassEnergy_eV = NaN;
|
|
regionEntry.scientaDwellTime_ms = NaN;
|
|
|
|
% Inherit defaults from Global Header
|
|
regionEntry.scientaAcquisitionMode = globalScanMode;
|
|
regionEntry.scientaLensMode = globalLensMode;
|
|
regionEntry.scientaAnalyzerSlit = globalAnalyzerSlit;
|
|
regionEntry.scientaWorkfunction_eV = globalWorkFunction;
|
|
regionEntry.creationDate = osDateVec;
|
|
if ~isempty(globalSample)
|
|
regionEntry.sample = globalSample;
|
|
end
|
|
|
|
% Scan the header lines directly
|
|
for L = 1:min(150, length(regionBlock)) % Metadata is always at the top
|
|
lineStr = strtrim(regionBlock(L));
|
|
if isempty(lineStr) || ~startsWith(lineStr, '#'), continue; end
|
|
|
|
lineStr = regexprep(lineStr, '^#\s+', '# '); % Normalize spaces after #
|
|
|
|
if startsWith(lineStr, '# Region:')
|
|
rName = strtrim(extractAfter(lineStr, '# Region:'));
|
|
elseif startsWith(lineStr, '# Excitation Energy:')
|
|
regionEntry.xRayEkin_eV = sscanf(strtrim(extractAfter(lineStr, '# Excitation Energy:')), '%f');
|
|
% =========================================================================
|
|
% OFFLINE XPS BUG FIX (Mg vs Al Source) - REGION OVERRIDE
|
|
% =========================================================================
|
|
if ~any(contains(string(fileLines), 'Monochromator', 'IgnoreCase', true))
|
|
if (abs(regionEntry.xRayEkin_eV - 1453.6) < 1) || (abs(regionEntry.xRayEkin_eV - 1486.61) < 1)
|
|
regionEntry.xRayEkin_eV = 1253.6;
|
|
end
|
|
end
|
|
% =========================================================================
|
|
exEnergyStr = num2str(regionEntry.xRayEkin_eV);
|
|
elseif startsWith(lineStr, '# Spectrum ID:')
|
|
specID = sscanf(strtrim(extractAfter(lineStr, '# Spectrum ID:')), '%f');
|
|
elseif startsWith(lineStr, '# Acquisition Date:')
|
|
rawDateStr = strtrim(extractAfter(lineStr, '# Acquisition Date:'));
|
|
dateStr = strtrim(regexprep(rawDateStr, 'UTC$', ''));
|
|
try
|
|
dtUTC = datetime(dateStr, 'InputFormat', 'MM/dd/yy HH:mm:ss', 'PivotYear', 2000, 'TimeZone', 'UTC');
|
|
dtLocal = datetime(dtUTC, 'TimeZone', 'Europe/Zurich');
|
|
% ONLY set the Start Time (creationDate). Leave lastModified as OS default!
|
|
regionEntry.creationDate = datevec(dtLocal);
|
|
catch
|
|
% If parsing fails, it safely falls back to the OS creation date
|
|
end
|
|
elseif startsWith(lineStr, '# Pass Energy:')
|
|
regionEntry.scientaPassEnergy_eV = str2double(extractAfter(lineStr, '# Pass Energy:'));
|
|
elseif startsWith(lineStr, '# Dwell Time:')
|
|
regionEntry.scientaDwellTime_ms = str2double(extractAfter(lineStr, '# Dwell Time:')) * 1000;
|
|
elseif startsWith(lineStr, '# Scan Mode:')
|
|
regionEntry.scientaAcquisitionMode = strtrim(extractAfter(lineStr, '# Scan Mode:'));
|
|
elseif startsWith(lineStr, '# Analyzer Lens Voltage:')
|
|
regionEntry.scientaLensMode = strtrim(extractAfter(lineStr, '# Analyzer Lens Voltage:'));
|
|
elseif startsWith(lineStr, '# Analyzer Lens:')
|
|
regionEntry.scientaLensMode = strtrim(extractAfter(lineStr, '# Analyzer Lens:'));
|
|
elseif startsWith(lineStr, '# Analyzer Slit:')
|
|
regionEntry.scientaAnalyzerSlit = strtrim(extractAfter(lineStr, '# Analyzer Slit:'));
|
|
elseif startsWith(lineStr, '# Work Function:')
|
|
regionEntry.scientaWorkfunction_eV = sscanf(strtrim(extractAfter(lineStr, '# Work Function:')), '%f');
|
|
elseif startsWith(lineStr, '# Eff. Workfunction:')
|
|
regionEntry.scientaWorkfunction_eV = sscanf(strtrim(extractAfter(lineStr, '# Eff. Workfunction:')), '%f');
|
|
elseif startsWith(lineStr, '# Comment:')
|
|
regionEntry.sample = strtrim(extractAfter(lineStr, '# Comment:'));
|
|
end
|
|
end
|
|
|
|
% --- Format Name & Region ---
|
|
regionEntry.regionName = rName;
|
|
cleanRName = strrep(rName, ' ', '_');
|
|
|
|
% Write the original OS file name
|
|
regionEntry.originalFileName = current_file_info.name;
|
|
|
|
% Prevent identical filenames by appending _Reg1, _Reg2 if multiple regions exist
|
|
if length(regionStarts) > 1
|
|
regionEntry.name = sprintf('%04d_%s_Reg%d_%s.xy', specID, cleanRName, r, exEnergyStr);
|
|
else
|
|
regionEntry.name = sprintf('%04d_%s_%s.xy', specID, cleanRName, exEnergyStr);
|
|
end
|
|
|
|
|
|
|
|
% 2. Extract Data
|
|
hIdx = find(startsWith(strtrim(regionBlock), '# Cycle:'));
|
|
tempD = []; enAxis = [];
|
|
|
|
if ~isempty(hIdx)
|
|
for d = 1:length(hIdx)
|
|
cScn = 1; cCrv = 1;
|
|
|
|
headerStr = regionBlock(hIdx(d));
|
|
curveMatch = regexp(headerStr, 'Curve:\s*(\d+)', 'tokens', 'once');
|
|
if ~isempty(curveMatch), cCrv = str2double(curveMatch{1}) + 1; end
|
|
|
|
scanMatch = regexp(headerStr, 'Scan:\s*(\d+)', 'tokens', 'once');
|
|
if ~isempty(scanMatch), cScn = str2double(scanMatch{1}) + 1; end
|
|
|
|
row = hIdx(d) + 1;
|
|
while row <= length(regionBlock) && (startsWith(strtrim(regionBlock(row)), '#') || strlength(strtrim(regionBlock(row))) < 2), row = row + 1; end
|
|
dataS = row;
|
|
while row <= length(regionBlock) && ~startsWith(strtrim(regionBlock(row)), '#') && strlength(strtrim(regionBlock(row))) > 1, row = row + 1; end
|
|
|
|
if dataS < row
|
|
numCols = length(sscanf(regionBlock(dataS), '%f'));
|
|
raw = sscanf(join(regionBlock(dataS:row-1), ' '), '%f');
|
|
if ~isempty(raw) && numCols >= 2
|
|
m = reshape(raw, numCols, [])';
|
|
if isempty(enAxis), enAxis = m(:,1); end
|
|
tempD(1:size(m,1), cCrv, cScn) = m(:,2);
|
|
end
|
|
end
|
|
end
|
|
else
|
|
row = 1;
|
|
while row <= length(regionBlock) && (startsWith(strtrim(regionBlock(row)), '#') || strlength(strtrim(regionBlock(row))) < 2), row = row + 1; end
|
|
dataS = row;
|
|
while row <= length(regionBlock) && ~startsWith(strtrim(regionBlock(row)), '#') && strlength(strtrim(regionBlock(row))) > 1, row = row + 1; end
|
|
|
|
if dataS < row
|
|
numCols = length(sscanf(regionBlock(dataS), '%f'));
|
|
raw = sscanf(join(regionBlock(dataS:row-1), ' '), '%f');
|
|
if ~isempty(raw) && numCols >= 2
|
|
m = reshape(raw, numCols, [])';
|
|
enAxis = m(:,1);
|
|
tempD(1:size(m,1), 1, 1) = m(:,2);
|
|
end
|
|
end
|
|
end
|
|
|
|
% 3. Map Dimensions
|
|
if ~isempty(tempD)
|
|
image2D = squeeze(mean(tempD, 2, 'omitnan'));
|
|
if isvector(image2D), image2D = image2D(:); end
|
|
else
|
|
image2D = [];
|
|
end
|
|
|
|
regionEntry.image = {image2D};
|
|
regionEntry.imageRoh = {image2D};
|
|
|
|
if ~isnan(regionEntry.xRayEkin_eV)
|
|
regionEntry.xRayEkinRange_eV = [regionEntry.xRayEkin_eV regionEntry.xRayEkin_eV];
|
|
end
|
|
|
|
if ~isempty(enAxis)
|
|
enAxis = reshape(enAxis, 1, []);
|
|
regionEntry.scientaEkinRange_eV = [enAxis(1) enAxis(end)];
|
|
regionEntry.scientaEkinStep_eV = enAxis(2) - enAxis(1);
|
|
regionEntry.scientaEkin_eV = {enAxis};
|
|
regionEntry.scientaEkinRoh_eV = {enAxis};
|
|
end
|
|
|
|
regionEntry.scientaRegionIterations = size(tempD, 3);
|
|
|
|
if isempty(ProcessedXPS)
|
|
ProcessedXPS = regionEntry;
|
|
else
|
|
ProcessedXPS = [ProcessedXPS; regionEntry]; %#ok<AGROW>
|
|
end
|
|
end
|
|
catch ME
|
|
fprintf('Warning: Failed to parse %s (%s)\n', current_file_info.name, ME.message);
|
|
if isempty(ProcessedXPS), ProcessedXPS = current_file_info; else, ProcessedXPS = [ProcessedXPS; current_file_info]; end %#ok<AGROW>
|
|
end
|
|
end
|
|
end
|
|
|
|
TmpFileList{1} = ProcessedXPS;
|
|
end
|
|
|
|
disp('*************************************************')
|
|
disp('* Voila: Done reading & writing XPS data! *')
|
|
disp('*************************************************')
|
|
%% Read and write NEXAFS meta data
|
|
%
|
|
% The h5 file is much nicer organised, so we do not need string operations
|
|
% to select individual meta data. The data can be directly addressed and
|
|
% written into the list structure:
|
|
|
|
% if datenum(filedate_first) >= datenum('31-Dec-2017 23:59:00') %('31-Dec-2019 23:59:00')
|
|
for ii = 1:size(TmpFileList{2},1) % do this for each list element
|
|
|
|
% Get OS Dates
|
|
tmpCurrentFile = fullfile(TmpFileList{2}(ii).folder, TmpFileList{2}(ii).name);
|
|
% ==========================================================
|
|
% IN-MEMORY DATE RESTORATION
|
|
% Check if this file was created by the splitter. If yes,
|
|
% retrieve its historical dates from the memory map!
|
|
% ==========================================================
|
|
if exist('splitFileDatesMap', 'var') && isKey(splitFileDatesMap, TmpFileList{2}(ii).name)
|
|
savedDates = splitFileDatesMap(TmpFileList{2}(ii).name);
|
|
cDateOS = savedDates{1};
|
|
mDateOS = savedDates{2};
|
|
else
|
|
% Otherwise, just read the dates directly from the OS
|
|
[cDateOS, mDateOS] = getOSDates(tmpCurrentFile);
|
|
end
|
|
|
|
% START TIME: OS Creation Date (or mapped historical creation)
|
|
TmpFileList{2}(ii).creationDate = datevec(cDateOS);
|
|
% END TIME: OS Modification Date (or mapped historical modification)
|
|
TmpFileList{2}(ii).lastModifiedDatenum = datenum(mDateOS);
|
|
TmpFileList{2}(ii).lastModifiedDatestr = mDateOS;
|
|
% ==========================================================
|
|
|
|
|
|
try
|
|
% read data
|
|
%if datenum(filedate_first) <= datenum('21-Nov-2020 00:01:00') % Date not exaclty certain.
|
|
try
|
|
TmpImage= h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/spectrum');
|
|
catch
|
|
try
|
|
TmpImage= h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/spectrum');
|
|
%else
|
|
catch
|
|
try
|
|
TmpImage = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/scienta spectrum');
|
|
catch
|
|
TmpImage = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/detector/data'); % pre-2020 (?) h5 structure
|
|
end
|
|
end
|
|
end
|
|
|
|
|
|
|
|
% import Photon Energy (cols)
|
|
try
|
|
TmpXrayEkin = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/energy'); % X-ray excitation energy; Photon energy in eV
|
|
TmpXrayEkin(TmpXrayEkin==0)=[];
|
|
catch
|
|
try
|
|
TmpXrayEkin = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/energy'); % X-ray excitation energy; Photon energy in eV
|
|
TmpXrayEkin(TmpXrayEkin==0)=[];
|
|
catch
|
|
TmpXrayEkin = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/PhotonEnergy'); % X-ray excitation energy; Photon energy in eV
|
|
TmpXrayEkin(TmpXrayEkin==0)=[];
|
|
end
|
|
end
|
|
|
|
|
|
if ~isempty(TmpXrayEkin)
|
|
TmpFileList{2}(ii).xRayEkinRange_eV = [TmpXrayEkin(1) TmpXrayEkin(end)];
|
|
end
|
|
|
|
% Make unique
|
|
[TmpFileList{2}(ii).xRayEkin_eV, ind] = unique(TmpXrayEkin);
|
|
TmpFileList{2}(ii).image = TmpImage(:,ind);
|
|
TmpFileList{2}(ii).image = TmpImage(:,ind);
|
|
% NEXAFS pass energy
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/snaps/PassEnergy'); % in eV
|
|
TmpFileList{2}(ii).scientaPassEnergy_eV = str2double(tmp(1,:));
|
|
catch
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/attrs/PassEnergy'); % in eV
|
|
TmpFileList{2}(ii).scientaPassEnergy_eV = str2double(tmp(1));
|
|
catch
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/PassEnergy'); % in eV
|
|
|
|
if tmp(1) == 1
|
|
TmpFileList{2}(ii).scientaPassEnergy_eV = 20;
|
|
elseif tmp(1) == 2
|
|
TmpFileList{2}(ii).scientaPassEnergy_eV = 50;
|
|
elseif tmp(1) == 3
|
|
TmpFileList{2}(ii).scientaPassEnergy_eV = 100;
|
|
elseif tmp(1) == 4
|
|
TmpFileList{2}(ii).scientaPassEnergy_eV = 200;
|
|
end
|
|
end
|
|
end
|
|
|
|
|
|
% NEXAFS dwell time
|
|
% if datenum(filedate_first) >= datenum('1-Feb-2020 23:59:00')
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/snaps/ScientaStepTime')*1000; % in ms
|
|
|
|
catch
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/attrs/ScientaStepTime')*1000; % in ms
|
|
catch
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/attrs/StepTime')*1000; % in ms
|
|
catch
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/StepTime')*1000; % in ms
|
|
catch
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/ScientaStepTime')*1000;
|
|
end
|
|
end
|
|
end
|
|
end
|
|
TmpFileList{2}(ii).scientaDwellTime_ms = tmp(1);
|
|
% NEXAFS region Name
|
|
if TmpFileList{2}(ii).xRayEkinRange_eV(:,1) <= 530 && TmpFileList{2}(ii).xRayEkinRange_eV(:,2) >= 530
|
|
TmpFileList{2}(ii).regionName = 'O K-edge NEXAFS';
|
|
else
|
|
TmpFileList{2}(ii).regionName = 'NEXAFS';
|
|
end
|
|
|
|
% Acquisition mode
|
|
try
|
|
TmpFileList{2}(ii).scientaAcquisitionMode = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/snaps/AcquisitionMode');
|
|
catch
|
|
try
|
|
TmpFileList{2}(ii).scientaAcquisitionMode = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/attrs/AcquisitionMode');
|
|
catch
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/AcquireMode');
|
|
if tmp(1) == 0
|
|
TmpFileList{2}(ii).scientaAcquisitionMode = "Swept";
|
|
elseif tmp(1) == 1
|
|
TmpFileList{2}(ii).scientaAcquisitionMode = "Fixed";
|
|
end
|
|
clear tmp
|
|
end
|
|
end
|
|
|
|
if strcmp(TmpFileList{2}(ii).scientaAcquisitionMode,'Fixed')
|
|
% NEXAFS kinetic energy window - fixed mode
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/snaps/ScientaCenterEnergy'); % in eV
|
|
catch
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/attrs/ScientaCenterEnergy'); % in eV
|
|
catch
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/CenterEnergy'); % in eV
|
|
end
|
|
end
|
|
|
|
TmpFileList{2}(ii).scientaEkinRange_eV = [double(tmp(1))-TmpFileList{2}(ii).scientaPassEnergy_eV/20 double(tmp(1))+TmpFileList{2}(ii).scientaPassEnergy_eV/20];
|
|
|
|
|
|
TmpFileList{2}(ii).scientaEkin_eV = linspace(TmpFileList{2}(ii).scientaEkinRange_eV(:,1)...
|
|
,TmpFileList{2}(ii).scientaEkinRange_eV(:,2),size(TmpFileList{2}(ii).image,1))';
|
|
else
|
|
% NEXAFS kinetic energy window - scan mode
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/snaps/ScientaLowEnergy'); % in eV
|
|
tmp2 = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/snaps/ScientaHighEnergy');
|
|
catch
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/attrs/ScientaLowEnergy'); % in eV
|
|
tmp2 = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/attrs/ScientaHighEnergy');
|
|
catch
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/LowEnergy'); % in eV
|
|
tmp2 = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/HighEnergy');
|
|
end
|
|
end
|
|
TmpFileList{2}(ii).scientaEkinRange_eV = [tmp(1) tmp2(1)];
|
|
clearvars tmp
|
|
try
|
|
TmpFileList{2}(ii).scientaEkinStep_eV = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/snaps/StepSize');
|
|
catch
|
|
try
|
|
TmpFileList{2}(ii).scientaEkinStep_eV = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/attrs/StepSize');
|
|
catch
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/StepSize');
|
|
TmpFileList{2}(ii).scientaEkinStep_eV = abs(tmp(1));
|
|
end
|
|
end
|
|
TmpFileList{2}(ii).scientaEkin_eV = [TmpFileList{2}(ii).scientaEkinRange_eV(:,1):TmpFileList{2}(ii).scientaEkinStep_eV(1):TmpFileList{2}(ii).scientaEkinRange_eV(:,2)]';
|
|
end
|
|
|
|
clearvars tmp
|
|
% read I_0
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/i0');
|
|
catch
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/i0');
|
|
catch
|
|
end
|
|
end
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/RM2I0');
|
|
catch
|
|
end
|
|
|
|
try
|
|
TmpFileList{2}(ii).beamlineInt = abs(tmp(ind));
|
|
catch
|
|
TmpFileList{2}(ii).beamlineInt = NaN(size(ind,1),1);
|
|
end
|
|
|
|
|
|
|
|
% Lens Mode
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/snaps/LensMode');
|
|
TmpFileList{2}(ii).scientaLensMode = tmp;
|
|
catch
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/attrs/LensMode');
|
|
TmpFileList{2}(ii).scientaLensMode = tmp{1};
|
|
catch
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/LensMode');
|
|
TmpFileList{2}(ii).scientaLensMode = num2str(tmp(1));
|
|
end
|
|
end
|
|
|
|
clearvars tmp
|
|
try
|
|
TmpFileList{2}(ii).scientaRegionIterations = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan_1/snaps/NumIterations');
|
|
catch
|
|
try
|
|
TmpFileList{2}(ii).scientaRegionIterations = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/scan 1/attrs/NumIterations');
|
|
catch
|
|
try
|
|
tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/NumIterations');
|
|
TmpFileList{2}(ii).scientaRegionIterations = double(tmp(1));
|
|
catch
|
|
TmpFileList{2}(ii).scientaRegionIterations = 1;
|
|
end
|
|
end
|
|
end
|
|
TmpFileList{2}(ii).scientaSequenceIterations = 1;
|
|
|
|
catch
|
|
warning('Problem with a NEXAFS data file in NEXAFS list')
|
|
disp('Problem with a NEXAFS data file in NEXAFS list')
|
|
fprintf('Number %.0f: %s \n',ii,[TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name])
|
|
fprintf('Number %.0f: %s \n',ii,datestr(TmpFileList{2}(ii).datenum))
|
|
end
|
|
|
|
TmpFileList{2}(ii).imageRoh = TmpFileList{2}(ii).image;
|
|
% TmpFileList{2}(ii).spectrum_countsPerSecondRoh = TmpFileList{2}(ii).spectrum_countsPerSecond;
|
|
% log
|
|
TmpFileList{2}(ii).importDate = datestr(now);
|
|
try
|
|
TmpFileList{2}(ii).analysisDir = file.mFile;
|
|
catch
|
|
TmpFileList{2}(ii).analysisDir = pwd;
|
|
end
|
|
end
|
|
|
|
% else
|
|
% for ii = 1:size(TmpFileList{2},1) % do this for each list element
|
|
% try
|
|
% % NEXAFS pass energy
|
|
% tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/PassEnergy'); % in eV
|
|
% % TODO: Confirm that this assignment is correct. (The number refer to the list number in epics software)
|
|
% if tmp(1) == 1
|
|
% TmpFileList{2}(ii).passEnergy_eV = {[20]};
|
|
% elseif tmp(1) == 2
|
|
% TmpFileList{2}(ii).passEnergy_eV = {[50]};
|
|
% elseif tmp(1) == 3
|
|
% TmpFileList{2}(ii).passEnergy_eV = {[100]};
|
|
% elseif tmp(1) == 4
|
|
% TmpFileList{2}(ii).passEnergy_eV = {[200]};
|
|
% end
|
|
%
|
|
% % NEXAFS dwell time
|
|
% if datenum(filedate_first) >= datenum('1-Feb-2020 23:59:00')
|
|
% tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/ScientaStepTime')*1000; % in ms
|
|
%
|
|
% else
|
|
% tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/StepTime')*1000; % in ms
|
|
% end
|
|
% TmpFileList{2}(ii).dwellTime_ms = tmp(1);
|
|
% % NEXAFS region Name
|
|
% TmpFileList{2}(ii).regionName = 'NEXAFS';
|
|
% % FIXME: Can we differentiate O K-edge, N K-edge, C K-edge based on the h5-file meta data?
|
|
%
|
|
% % NEXAFS kinetic energy window
|
|
% tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/LowEnergy'); % in eV
|
|
% tmp2 = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/HighEnergy');
|
|
% TmpFileList{2}(ii).kinRange_eV = [tmp(1) tmp2(1)];
|
|
%
|
|
% % Lens mode
|
|
% tmp = h5read([TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name],'/entry/instrument/NDAttributes/LensMode');
|
|
% TmpFileList{2}(ii).scientaLensMode = tmp{1};
|
|
% clearvars tmp
|
|
% catch
|
|
% warning('Problem with a NEXAFS data file in NEXAFS list')
|
|
% disp('Problem with a NEXAFS data file in NEXAFS list')
|
|
% fprintf('Number %.0f: %s \n',ii,[TmpFileList{2}(ii).folder filesep TmpFileList{2}(ii).name])
|
|
% fprintf('Number %.0f: %s \n',ii,datestr(TmpFileList{2}(ii).datenum))
|
|
% end
|
|
%
|
|
% end
|
|
% end
|
|
|
|
|
|
|
|
disp('*************************************************')
|
|
disp('* Voila: Done reading and writing NEXAFS data! *')
|
|
disp('*************************************************')
|
|
%% Create Picture List
|
|
%
|
|
% The file structure needs same number of fields as SES and NEXAFS data.
|
|
% Set them to NaN (Not-a-Number).
|
|
|
|
|
|
for ii = 1:size(TmpFileList{3},1) % do this for each picture in the list
|
|
% Get OS Dates
|
|
tmpCurrentFile = fullfile(TmpFileList{3}(ii).folder, TmpFileList{3}(ii).name);
|
|
[cDateOS, ~] = getOSDates(tmpCurrentFile);
|
|
|
|
% START TIME: OS Creation Date
|
|
TmpFileList{3}(ii).creationDate = datevec(cDateOS);
|
|
% END TIME: Robust OS Modification Date
|
|
TmpFileList{3}(ii).lastModifiedDatenum = TmpFileList{3}(ii).datenum;
|
|
TmpFileList{3}(ii).lastModifiedDatestr = TmpFileList{3}(ii).date;
|
|
end
|
|
|
|
%% Merge lists into one
|
|
%
|
|
|
|
%% Merge lists into one
|
|
ListFiles = table();
|
|
for i = 1:length(TmpFileList)
|
|
if ~isempty(TmpFileList{i})
|
|
|
|
% Convert struct to table
|
|
tmpTable = struct2table(TmpFileList{i}, 'AsArray', true);
|
|
|
|
% Ensure the 'name' column is a cell-string for consistency
|
|
if iscell(tmpTable.name) == 0
|
|
tmpTable.name = cellstr(tmpTable.name);
|
|
end
|
|
|
|
% Merge using helper
|
|
if isempty(ListFiles)
|
|
ListFiles = tmpTable;
|
|
else
|
|
ListFiles = outerjoinRowToTable(ListFiles, tmpTable);
|
|
end
|
|
end
|
|
end
|
|
|
|
|
|
% Remove rows where the 'image' column is empty
|
|
% not sure why this code is here, probably a workaround for earlier errors.
|
|
|
|
% Identify rows where the 'image' column is empty
|
|
tmpEmptyImgLogic = cellfun(@isempty, ListFiles.image);
|
|
% Extract the names of those files for the warning
|
|
tmpDeletedNames = ListFiles.name(tmpEmptyImgLogic);
|
|
% If any were found, display the warning
|
|
if ~isempty(tmpDeletedNames)
|
|
warning('The following files were removed because they contain no image data:');
|
|
fprintf(' - %s\n', tmpDeletedNames{:}); % Vectorized print of all names
|
|
end
|
|
% Actually remove the rows from the table
|
|
ListFiles(tmpEmptyImgLogic, :) = [];
|
|
clearvars -except ListFiles server dataPath* p
|
|
|
|
%% Sort by date
|
|
|
|
% sort the table based on the time (6th column, datenum)
|
|
ListFiles = sortrows(ListFiles,6);
|
|
% Then, remove un-needed columns
|
|
ListFiles = removevars(ListFiles,{'bytes','isdir'});
|
|
|
|
disp('*************************************************')
|
|
disp('* Voila: Done creating data list! *')
|
|
disp('*************************************************')
|
|
%% Import LabView data
|
|
%
|
|
if matches(p.Results.importLabView,'on')
|
|
if isfolder(dataPath_LabView) % verify that this folder exists.
|
|
try
|
|
% Get list of all files in that folder
|
|
TmpFileList = dir([dataPath_LabView filesep '**' filesep '20*.dat']);
|
|
|
|
% import LabView data
|
|
LabView = LabView_importData(fullfile(TmpFileList(1).folder,TmpFileList(1).name)); % Overwrite LabView data in workspace, if this is the first import
|
|
if size(TmpFileList,1) > 1 % for additional data files, add these to the LabView data structure.
|
|
for ii = 2:size(TmpFileList,1)
|
|
LabView = LabView_importData(fullfile(TmpFileList(ii).folder,TmpFileList(ii).name),LabView);
|
|
end
|
|
end
|
|
catch
|
|
warning('Problem with LabView file')
|
|
disp('Problem reading LabView file')
|
|
end
|
|
|
|
try
|
|
% Get list of all files in that folder
|
|
TmpFileList = dir([dataPath_LabView filesep '**' filesep 'logfile*.txt']);
|
|
|
|
% import LabView data
|
|
LabView = LabView_importData(fullfile(TmpFileList(1).folder,TmpFileList(1).name)); % Overwrite LabView data in workspace, if this is the first import
|
|
if size(TmpFileList,1) > 1 % for additional data files, add these to the LabView data structure.
|
|
for ii = 2:size(TmpFileList,1)
|
|
LabView = LabView_importData(fullfile(TmpFileList(ii).folder,TmpFileList(ii).name),LabView);
|
|
end
|
|
end
|
|
catch
|
|
warning('Problem with LabView file')
|
|
disp('Problem reading LabView file')
|
|
end
|
|
end
|
|
disp('*************************************************')
|
|
disp('* Voila: Done reading LabView data! *')
|
|
disp('*************************************************')
|
|
|
|
clearvars TmpFileList
|
|
|
|
try
|
|
% combine data and/or rename
|
|
LabView = LabView_combine(LabView);
|
|
catch
|
|
warning('Problem with LabView file')
|
|
disp('Problem merging LabView files')
|
|
disp('Check number of columns for each file')
|
|
end
|
|
|
|
%% Extent list with LabView data
|
|
|
|
%Query time of each data file and get LabView log for that time.
|
|
|
|
for ii = 1:size(ListFiles,1)
|
|
|
|
try
|
|
|
|
% The time of the data and the logging of LabView are not
|
|
% necessarily occurring at precisely the same time. So, here define
|
|
% a time range (50 seconds) and get all
|
|
% data between these times.
|
|
tse1 = tsdata.event('Start',datenum(ListFiles.datenum(ii)),'Datenum');
|
|
tse2 = tsdata.event('End',datenum(ListFiles.datenum(ii)+seconds(50)),'Datenum');
|
|
tmp = gettsbetweenevents(LabView.merge,tse1,tse2);
|
|
|
|
if ~isempty(tmp.Data) % only do this if LabView data are not missing (because data logging was off).
|
|
|
|
% Cryo-holder set T (Sample holder temperature is in field ts.Data:1)
|
|
ListFiles.sampleTemp_dC(ii) = tmp.Data(1,1); % in �C (only get data in first row of tmp)
|
|
% Cell pressure
|
|
% (TODO: Pick correct Baratron read out: either field: ts.Data 4,8,and 9)
|
|
ListFiles.cellPressure_mbar(ii) = tmp.Data(1,4); % in mbar
|
|
% vapour pressure over ice:
|
|
if ListFiles.cellPressure_mbar(ii) >= 0
|
|
% Marti, J., and K. Mauersberger, Geophys. Res.Lett., 20, 363?366, 1993.
|
|
% gives parameterisation from 169 - 273K in Pa. (log10(p)= A1/T+B)
|
|
A_1 = -2663.5;
|
|
B_1 = 12.537;
|
|
ListFiles.iceTemp_dC(ii) = A_1/(log10(ListFiles.cellPressure_mbar(ii)*100)-B_1)-273.15;
|
|
else
|
|
ListFiles.iceTemp_dC(ii) = NaN;
|
|
end
|
|
|
|
if datenum(ListFiles.datenum(ii)) >= datenum('04-Mar-2019 13:30:00') % import additional data, logged after 03/2019
|
|
ListFiles.smplX_mm(ii) = tmp.Data(1,10);
|
|
ListFiles.smplY_mm(ii) = tmp.Data(1,11);
|
|
ListFiles.smplZ_mm(ii) = tmp.Data(1,12);
|
|
ListFiles.sealingTemp(ii) = tmp.Data(1,13);
|
|
else % if these sub-set of data is not logged, set to NaN
|
|
ListFiles.smplX_mm(ii) = NaN;
|
|
ListFiles.smplY_mm(ii) = NaN;
|
|
ListFiles.smplZ_mm(ii) = NaN;
|
|
ListFiles.sealingTemp(ii) = NaN;
|
|
end
|
|
else % if there are no LabView data stored, set all entries to NaN
|
|
ListFiles.sampleTemp_dC(ii) = NaN;
|
|
ListFiles.iceTemp_dC(ii) = NaN;
|
|
ListFiles.cellPressure_mbar(ii) = NaN;
|
|
ListFiles.smplX_mm(ii) = NaN;
|
|
ListFiles.smplY_mm(ii) = NaN;
|
|
ListFiles.smplZ_mm(ii) = NaN;
|
|
ListFiles.sealingTemp(ii) = NaN;
|
|
end
|
|
clearvars tmp
|
|
catch
|
|
warning('Problem merging LabView data into data table')
|
|
disp('Problem merging LabView data into data table')
|
|
%fprintf('Number %.0f: %s \n',ii,[ListFiles.folder(ii) filesep ListFiles.name(ii)])
|
|
%fprintf('Number %.0f: %s \n',ii,datestr(ListFiles.datenum(ii)))
|
|
end
|
|
end
|
|
|
|
disp('*************************************************')
|
|
disp('* Voila: Done writing LabView data! *')
|
|
disp('*************************************************')
|
|
end
|
|
%% Make table nicer
|
|
%
|
|
|
|
|
|
|
|
|
|
if ~iscell(ListFiles.xRayEkin_eV)
|
|
% Use num2cell to wrap the existing numbers in cells.
|
|
ListFiles.xRayEkin_eV = num2cell(ListFiles.xRayEkin_eV);
|
|
end
|
|
if ~iscell(ListFiles.scientaEkin_eV)
|
|
for ii = 1:size(ListFiles,1)
|
|
tmp{ii,:} = ListFiles.scientaEkin_eV(ii,:);
|
|
end
|
|
ListFiles.scientaEkin_eV = tmp;
|
|
end
|
|
|
|
|
|
|
|
% -------------------------------------------------------------------------
|
|
% MAP DATES & CLEANUP
|
|
% -------------------------------------------------------------------------
|
|
|
|
% Delete the raw OS fields and the temporary uppercase fields to avoid duplicates
|
|
ListFiles = removevars(ListFiles, {'date','datenum'});
|
|
|
|
% Move Dates upfront
|
|
ListFiles = movevars(ListFiles,'lastModifiedDatestr','After','creationDate');
|
|
ListFiles = movevars(ListFiles,'lastModifiedDatenum','After','creationDate');
|
|
% -------------------------------------------------------------------------
|
|
|
|
% Copy Ekin Data
|
|
existColumn = strcmp('scientaEkinRoh_eV',ListFiles.Properties.VariableNames);
|
|
if any(existColumn)
|
|
ListFiles = removevars(ListFiles,'scientaEkinRoh_eV');
|
|
end
|
|
existColumn = strcmp('scientaEkinRoh_eV',ListFiles.Properties.VariableNames);
|
|
if ~any(existColumn)
|
|
% save current ekin data before they are modified
|
|
ListFiles.scientaEkinRoh_eV= ListFiles.scientaEkin_eV;
|
|
end
|
|
if ~iscell(ListFiles.scientaEkinRoh_eV)
|
|
for ii = 1:size(ListFiles,1)
|
|
tmp{ii,:} = ListFiles.scientaEkinRoh_eV(ii,:);
|
|
end
|
|
ListFiles.scientaEkinRoh_eV = tmp;
|
|
end
|
|
|
|
|
|
% add the imported data to previous list (if passed to function upon call)
|
|
if istable(p.Results.DataTable) % do this only if previous list was input to function call
|
|
|
|
% find table variables that are missing in ListFiles
|
|
existColumn = strcmp('sample',p.Results.DataTable.Properties.VariableNames);
|
|
if ~any(existColumn)
|
|
p.Results.DataTable.sample = cellstr(repmat(' ',size(p.Results.DataTable,1),1));
|
|
end
|
|
tmpMissing = ~ismember(p.Results.DataTable.Properties.VariableNames,ListFiles.Properties.VariableNames);
|
|
% find all table variables that are cells
|
|
tmpCell = cellfun(@(x) iscell(p.Results.DataTable.(x)), p.Results.DataTable.Properties.VariableNames);
|
|
% find all table variables that are numbers
|
|
tmpNumber = cellfun(@(x) isnumeric(p.Results.DataTable.(x)), p.Results.DataTable.Properties.VariableNames);
|
|
% table variabes that hold cells and are missing
|
|
tmpCellVariables = p.Results.DataTable.Properties.VariableNames(tmpMissing & tmpCell);
|
|
% table variables that hold numbers and are missing
|
|
tmpNumberVariables = p.Results.DataTable.Properties.VariableNames(tmpMissing & tmpNumber);
|
|
% add empty table variables for missing cell and for number variables
|
|
if ~isempty(tmpCellVariables)
|
|
for i = 1: size(p.Results.DataTable(:,tmpMissing & tmpCell),2)
|
|
ListFiles = addvars(ListFiles,cell(size(ListFiles,1),1),'NewVariableNames',tmpCellVariables{i});
|
|
end
|
|
end
|
|
if ~isempty(tmpNumberVariables)
|
|
for i = 1: size(p.Results.DataTable(:,tmpMissing & tmpNumber),2)
|
|
ListFiles = addvars(ListFiles,nan(size(ListFiles,1),1),'NewVariableNames',tmpNumberVariables{i});
|
|
end
|
|
end
|
|
|
|
% % % sort the tables, if they have not the same order
|
|
% tmpCompareTables = cellfun(@isequal, ListFiles.Properties.VariableNames', p.Results.DataTable.Properties.VariableNames');
|
|
%
|
|
% if ~all(tmpCompareTables)
|
|
% indexDifferenzes = find(ismember(p.Results.DataTable.Properties.VariableNames',ListFiles.Properties.VariableNames(~tmpCompareTables)','rows'));
|
|
% nameListFiles =ListFiles.Properties.VariableNames(~tmpCompareTables);
|
|
%
|
|
% for ii = 1:size(indexDifferenzes,1)
|
|
% tmpIndx = find(ismember(p.Results.DataTable.Properties.VariableNames',nameListFiles{ii},'rows'));
|
|
% tmpPreviousVarName = p.Results.DataTable.Properties.VariableNames{tmpIndx-1};
|
|
% ListFiles = movevars(ListFiles,nameListFiles{ii},'After',tmpPreviousVarName);
|
|
% fprintf("%s comes after %s \n", nameListFiles{ii},tmpPreviousVarName)
|
|
% end
|
|
%
|
|
% end
|
|
|
|
|
|
|
|
% merge tables (append imported data to previous table)
|
|
% ListFiles = [p.Results.DataTable; ListFiles];
|
|
ListFiles = outerjoinRowToTable(p.Results.DataTable, ListFiles);
|
|
% Then, we sort the table based on the time (datenum)
|
|
ListFiles = sortrows(ListFiles,'lastModifiedDatenum');
|
|
end
|
|
|
|
%% Export
|
|
|
|
ExportList = removevars(ListFiles, {'image', 'scientaEkin_eV', 'beamlineInt', 'imageRoh', 'scientaEkinRoh_eV', 'xRayEkin_eV'});
|
|
% TODO check windows formatting
|
|
try
|
|
% write a ;-separated list for export to excel.
|
|
writetable(ExportList,fullfile(dataPath,[datestr(ExportList.lastModifiedDatenum(1),'mmmYYYY') '_DataList.csv']),'Delimiter',',');
|
|
|
|
if ispc
|
|
% and open in Excel; works only on Windows Machines
|
|
winopen(fullfile(dataPath,[datestr(ExportList.lastModifiedDatenum(1),'mmmYYYY') '_DataList.csv']));
|
|
end
|
|
catch
|
|
warning('Problem writing Excel File (*.csv). Make sure it is not open in Excel')
|
|
end
|
|
|
|
|
|
|
|
|
|
disp('*************************************************')
|
|
disp('* Voila: nappFileList.m finished ! *')
|
|
disp('*************************************************')
|
|
|
|
|
|
end
|
|
|
|
function LabView = LabView_importData(fullFileName,varargin)
|
|
%
|
|
% LabView_importData reads data log ASCII files stored by LabView into
|
|
% a Matlab time series.
|
|
%
|
|
% LabView = LabView_importData(A) imports file A
|
|
% and stores its content as time series in a structure called "LabView".
|
|
%
|
|
% LabView = LabView_importData(A,LabView) imports file A
|
|
% and appends its content as time series to the LabView structure.
|
|
%
|
|
%
|
|
% Example
|
|
% LabView = LabView_importData('exmpl.dat',LabView)
|
|
%
|
|
% Input
|
|
% * Napp_chamber_file: Complete and absolute path to the Napp chamber log file.
|
|
% * LabView: name of output structure. Always call this'LabView'
|
|
%
|
|
% Output
|
|
% * LabView: time series, the name starting with "ts"
|
|
% followed by date and time the recording started.
|
|
%
|
|
% See also LABVIEW_COMBINE, LABVIEW_PLOT, LABVIEW_CUTDATA
|
|
%
|
|
%% Data Structure
|
|
%
|
|
% Data structure before 4.3.2019
|
|
% col 1: Date
|
|
% col 2: Time
|
|
% col 3: Reservoir "Pressure 1" - Unit "1 water" - mbar -> ts.Data:2
|
|
% col 4: Reservoir "Pressure 2" - Unit "2 other" - mbar -> ts.Data:3
|
|
% col 5: Baratron 1 - cell - 1mbar max - mbar -> ts.Data:4
|
|
% col 6: Baratron 2 - cell - 10mbar max - mbar -> ts.Data:8
|
|
% col 7: Baratron 3 - cell - 0.05mbar max - mbar -> ts.Data:9
|
|
% col 8: Baratron 4 - - mbar
|
|
% col 9: Temp.Ice Sample - �C -> ts.Data:1
|
|
% col 10: Temp.Heated Sample
|
|
% col 11: none
|
|
% col 12: Temp. Reservoir "Cooler 1" - �C -> ts.Data:5
|
|
% col 13: Temp. Reservoir "Cooler 2" - �C -> ts.Data:6
|
|
% col 14: Flow He(cryoholder) - l/min -> ts.Data:7
|
|
% col 15: none
|
|
% col 16: none
|
|
%
|
|
% Data structure after 4.3.2019
|
|
% col 1: Date
|
|
% col 2: Time
|
|
% col 3: Reservoir "Pressure 1" - Unit "1 water" - mbar -> ts.Data:2
|
|
% col 4: Reservoir "Pressure 2" - Unit "2 other" - mbar -> ts.Data:3
|
|
% col 5: Baratron 1 - cell - 1mbar max - mbar -> ts.Data:4
|
|
% col 6: Baratron 2 - cell - 10mbar max - mbar -> ts.Data:8
|
|
% col 7: Baratron 3 - cell - 0.05mbar max - mbar -> ts.Data:9
|
|
% col 8: Baratron 4 - - mbar
|
|
% col 9: Temp.Ice Sample - �C -> ts.Data:1
|
|
% col 10: Temp.Heated Sample
|
|
% col 11: Temp. Reservoir "Cooler 1" - �C -> ts.Data:5
|
|
% col 12: Temp. Reservoir "Cooler 2" - �C -> ts.Data:6
|
|
% col 13: Flow He(cryoholder) - l/min -> ts.Data:7
|
|
% col 14: Flow gas 1
|
|
% col 15: Pressure chamber - mbar
|
|
% col 16: x-position of cryoholder - mm -> ts.Data:10
|
|
% col 17: y-position of cryoholder - mm -> ts.Data:11
|
|
% col 18: z-position of cryoholder - mm -> ts.Data:12
|
|
% col 19: nothing
|
|
% col 20: Temp.Sealing - �C -> ts.Data:13
|
|
|
|
|
|
%%
|
|
|
|
% Create an InputParser object
|
|
p = inputParser;
|
|
% Add inputs to the scheme.
|
|
defaultAppendToStructure = 0;
|
|
addRequired(p,'fullFileName',@isfile);
|
|
addOptional(p,'appendToStructure',defaultAppendToStructure,@isstruct);
|
|
% Parse the inputs.
|
|
parse(p,fullFileName,varargin{:})
|
|
|
|
%%
|
|
% get filedate
|
|
[~,name,~] = fileparts(p.Results.fullFileName);
|
|
tmp = strsplit(name,'_PC');
|
|
filedate = datenum(tmp{1},'yyyy-mm-dd_HH.MM.SS');
|
|
|
|
clearvars name tmp
|
|
|
|
%%
|
|
% parse structure to append to
|
|
|
|
if isstruct(p.Results.appendToStructure)
|
|
LabView = p.Results.appendToStructure;
|
|
end
|
|
%%
|
|
if datenum(filedate) <= datenum('04-Mar-2019 13:30:00')
|
|
|
|
fid = fopen(p.Results.fullFileName);
|
|
data = textscan(fid, ...
|
|
'%s %s %f %f %f %f %f %f %f %f %f %f %f %f %f %f',...
|
|
'HeaderLines',1); % data file has 2 text columns, followed by 14 numeric columns
|
|
fclose(fid);
|
|
% define time series
|
|
LabView.(['ts_' datestr(datenum([char(data{1}{1}), char(data{2}{1})],'dd.mm.yyyyHH:MM:SS'),'YYYYmmdd_HHMM')]) ...
|
|
= timeseries([data{9} data{3} data{4} data{5} data{12} ...
|
|
data{13} data{14} data{6} data{7}] , ...
|
|
datestr(datenum([char(data{1}), char(data{2})],...
|
|
'dd.mm.yyyyHH:MM:SS'),0));
|
|
LabView.(['ts_' datestr(datenum([char(data{1}{1}), char(data{2}{1})],'dd.mm.yyyyHH:MM:SS'),'YYYYmmdd_HHMM')]).Name ...
|
|
= 'Readout Napp Chamber raw signal';
|
|
clear fid ans formatOut nans t x
|
|
|
|
elseif datenum(filedate) >= datenum('15-Sep-2021 01:00:00') && datenum(filedate) <= datenum('01-Apr-2023 01:00:00')
|
|
% during this time, date and time were in american format.
|
|
fid = fopen(p.Results.fullFileName);
|
|
data = textscan(fid, ...
|
|
'%s %s %s %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f',...
|
|
'HeaderLines',1); %
|
|
|
|
fclose(fid);
|
|
% the date formating in the dat file is really messed up. Try to make
|
|
% it consitent with MATLAB. The problem is that MATLAB does not
|
|
% understand 1 digit month and hours.
|
|
%
|
|
%
|
|
% Split the date in column 1 into year,month,day
|
|
newDateSplit = cellfun(@(x) split(char(x),'/'),data{1},'UniformOutput', false);
|
|
% Split the time in column 2 into hours, minutes, seconds
|
|
newTimeSplit = cellfun(@(x) split(char(x),':'),data{2},'UniformOutput', false);
|
|
% combine both with the am/pm string in column 3 to a new datestring
|
|
customDateStr = cellfun(@(x,y,z) [datestr(datenum([str2double(x{3}),str2double(x{1}),str2double(x{2}),str2double(y{1}),str2double(y{2}),str2double(y{3})])), ' ', char(z)], newDateSplit,newTimeSplit,data{3}, 'uniformOutput',false) ;
|
|
|
|
% define time series
|
|
LabView.(['ts_' datestr(customDateStr{1},'YYYYmmdd_HHMM')]) ...
|
|
= timeseries([data{10} data{4} data{5} data{6} data{12} ...
|
|
data{13} data{14} data{7} data{8} data{17} data{18} data{19} data{21}], ...
|
|
customDateStr);
|
|
LabView.(['ts_' datestr(customDateStr{1},'YYYYmmdd_HHMM')]).Name ...
|
|
= 'Readout Napp Chamber raw signal';
|
|
clearvars fid ans formatOut nans t x new* custom*
|
|
|
|
|
|
elseif datenum(filedate) >= datenum('01-Apr-2023 02:00:00')
|
|
|
|
fid = fopen(p.Results.fullFileName);
|
|
data = textscan(fid, ...
|
|
'%s %s %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f',...
|
|
'HeaderLines',1);
|
|
% data file has 2 text columns, followed by 18 numeric columns (THE AM
|
|
% PM IS MISSING and some changes to the LabView file. Confirm with Luca....)
|
|
fclose(fid);
|
|
% define time series
|
|
|
|
LabView.(['ts_' datestr(datenum([char(data{1}{1}), char(data{2}{1})],'dd.mm.yyyyHH:MM:SS'),'YYYYmmdd_HHMM')]) ...
|
|
= timeseries([data{9} data{3} data{4} data{5} data{11} ...
|
|
data{12} data{20} data{6} data{7} data{15} data{16} data{17} data{19}] , ...
|
|
datestr(datenum([char(data{1}), char(data{2})],...
|
|
'dd.mm.yyyyHH:MM:SS'),0));
|
|
|
|
LabView.(['ts_' datestr(datenum([char(data{1}{1}), char(data{2}{1})],'dd.mm.yyyyHH:MM:SS'),'YYYYmmdd_HHMM')]).Name ...
|
|
= 'Readout Napp Chamber raw signal';
|
|
clear fid ans formatOut nans t x
|
|
|
|
else
|
|
fid = fopen(p.Results.fullFileName);
|
|
data = textscan(fid, ...
|
|
'%s %s %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f',...
|
|
'HeaderLines',1); % data file has 2 text columns, followed by 14 numeric columns
|
|
fclose(fid);
|
|
% define time series
|
|
|
|
LabView.(['ts_' datestr(datenum([char(data{1}{1}), char(data{2}{1})],'dd.mm.yyyyHH:MM:SS'),'YYYYmmdd_HHMM')]) ...
|
|
= timeseries([data{9} data{3} data{4} data{5} data{11} ...
|
|
data{12} data{13} data{6} data{7} data{16} data{17} data{18} data{20}] , ...
|
|
datestr(datenum([char(data{1}), char(data{2})],...
|
|
'dd.mm.yyyyHH:MM:SS'),0));
|
|
|
|
LabView.(['ts_' datestr(datenum([char(data{1}{1}), char(data{2}{1})],'dd.mm.yyyyHH:MM:SS'),'YYYYmmdd_HHMM')]).Name ...
|
|
= 'Readout Napp Chamber raw signal';
|
|
clear fid ans formatOut nans t x
|
|
end
|
|
end
|
|
|
|
function data_structure = LabView_combine(data_structure)
|
|
% LabView_combine can be used to merge time series into one.
|
|
%
|
|
% LabView = LabView_combine(LabView) merges all timeseries in the structure
|
|
% LabView into the new timeseries "merge"
|
|
% and adds this timeseries to the LabView structure
|
|
%
|
|
%
|
|
% Input
|
|
% * data_structure: name of the structure holding the time series
|
|
%
|
|
% Output
|
|
% * data_structure.merge: New time series combining all ts in this structure
|
|
%
|
|
% See also LABVIEW_IMPORTDATA, LABVIEW_PLOT
|
|
|
|
% delete previous merged data
|
|
if isfield(data_structure,'merge')
|
|
data_structure = rmfield(data_structure, 'merge');
|
|
end
|
|
|
|
% sort them chronologically
|
|
data_structure = orderfields(data_structure);
|
|
|
|
% query list of time series
|
|
list_ts = fieldnames(data_structure);
|
|
|
|
% and merge all of them into one time series
|
|
for ii = 1:length(list_ts)
|
|
if ii == 1
|
|
data_structure.merge = data_structure.(list_ts{1});
|
|
else
|
|
data_structure.merge = append(data_structure.merge,data_structure.(list_ts{ii}));
|
|
end
|
|
end
|
|
|
|
|
|
|
|
|
|
end
|
|
|
|
function newFiles = splitH5Scans(infile)
|
|
% splitH5Scans - Split an HDF5 file containing multiple scans into separate files
|
|
%
|
|
%
|
|
% This function takes an HDF5 file with multiple scan groups (scan_1, scan_2, etc.)
|
|
% and creates separate files for each scan. Each output file contains only one
|
|
% scan (renamed to scan_1) plus the logs group. The original file is deleted after
|
|
% successful splitting.
|
|
%
|
|
% SYNTAX:
|
|
% splitH5Scans(infile)
|
|
%
|
|
% INPUT:
|
|
% infile - String containing the full path to the input HDF5 file
|
|
% Example: '0002_20251119_145823_O_Kedge_Nov25.h5'
|
|
%
|
|
% OUTPUT:
|
|
% Creates new HDF5 files in the same directory as the input file.
|
|
% Each file is named: [original_name]_scan_N.h5
|
|
% Example outputs:
|
|
% 0002_20251119_145823_O_Kedge_Nov25_scan_1.h5
|
|
% 0002_20251119_145823_O_Kedge_Nov25_scan_2.h5
|
|
%
|
|
% REQUIREMENTS:
|
|
% - MATLAB with HDF5 support (R2011a or later)
|
|
% - Input file must have more than 3 root-level groups to be processed
|
|
%
|
|
% EXAMPLE:
|
|
% splitH5Scans('0002_20251119_145823_O_Kedge_Nov25.h5');
|
|
%
|
|
% Author: Perplexity and Thorsten
|
|
% Date: November 2025
|
|
|
|
% Read the structure of the HDF5 file to see what groups it contains
|
|
% h5info returns information about groups, datasets, and attributes
|
|
newFiles = {}; % Initialize the output cell array
|
|
fileinfo = h5info(infile);
|
|
|
|
% Get the original file's true OS dates BEFORE we start modifying it!
|
|
[orig_cDateOS, orig_mDateOS] = getOSDates(infile);
|
|
|
|
% Extract the names of all groups at the root level of the file
|
|
% Example: {'/scan_1', '/scan_2', '/logs', '/__DATA_TYPES__'}
|
|
groupnames = {fileinfo.Groups.Name};
|
|
|
|
% Only process files that have more than 3 groups at the root level
|
|
% This ensures we only split files with multiple scans
|
|
if numel(groupnames) > 3
|
|
|
|
% Find which groups are scan groups (contain 'scan_' in their name)
|
|
% This returns a logical array: true for scan groups, false otherwise
|
|
is_scan = contains(groupnames, 'scan_');
|
|
|
|
% Extract only the scan group names
|
|
% Example: {'/scan_1', '/scan_2'}
|
|
scan_names = groupnames(is_scan);
|
|
|
|
% Split the input filename into its components
|
|
% filepath: directory where file is located (e.g., 'C:\Data\')
|
|
% base: filename without extension (e.g., '0002_20251119_145823_O_Kedge_Nov25')
|
|
% ext: file extension (e.g., '.h5')
|
|
[filepath, base, ext] = fileparts(infile);
|
|
|
|
% If no directory path, use current directory
|
|
if isempty(filepath)
|
|
filepath = '.';
|
|
end
|
|
|
|
% Extract the file number prefix and rest of the name using pattern matching
|
|
% regexp splits '0002_20251119_145823_O_Kedge_Nov25' into:
|
|
% prefix = '0002_'
|
|
% rest = '20251119_145823_O_Kedge_Nov25'
|
|
pieces = regexp(base, '(\d+_)(.*)', 'tokens', 'once');
|
|
prefix = pieces{1}; % The number at the start (e.g., '0002_')
|
|
rest = pieces{2}; % Everything after the number
|
|
|
|
% Loop through each scan group and create a separate file for it
|
|
for k = 1:numel(scan_names)
|
|
|
|
% Get the full path of the current scan group
|
|
% Example: '/scan_1' or '/scan_2'
|
|
scan_group_full = scan_names{k};
|
|
|
|
% Remove the leading '/' from the scan name for use in filename
|
|
% Example: '/scan_2' becomes 'scan_2'
|
|
scan_name_clean = regexprep(scan_group_full, '^/', '');
|
|
|
|
% Build the output filename in the same directory as input file
|
|
% Example: 'C:\Data\0002_20251119_145823_O_Kedge_Nov25_scan_2.h5'
|
|
outname = fullfile(filepath, sprintf('%s%s_%s%s', prefix, rest, scan_name_clean, ext));
|
|
|
|
% Save the new filename to the output array
|
|
[~, base_out, ext_out] = fileparts(outname);
|
|
newFiles{end+1} = [base_out, ext_out];
|
|
|
|
% Copy the entire original file to create the new output file
|
|
% We'll modify this copy to keep only one scan
|
|
copyfile(infile, outname);
|
|
|
|
% Open the new file for reading and writing using low-level HDF5 functions
|
|
% fid is a file identifier used to reference this file
|
|
% 'H5F_ACC_RDWR' means read and write access
|
|
fid = H5F.open(outname, 'H5F_ACC_RDWR', 'H5P_DEFAULT');
|
|
|
|
% Delete all scan groups except the one we want to keep
|
|
% Loop through all scan groups in the original file
|
|
for j = 1:numel(scan_names)
|
|
other_scan = scan_names{j}; % Get the name of another scan group
|
|
|
|
% If this is not the scan we want to keep, delete it
|
|
if j ~= k
|
|
try
|
|
% H5L.delete removes the link (reference) to the group
|
|
% This makes the group inaccessible but doesn't free space
|
|
H5L.delete(fid, other_scan, 'H5P_DEFAULT');
|
|
catch
|
|
% If deletion fails (e.g., group doesn't exist), continue
|
|
% The catch block prevents the script from stopping on errors
|
|
end
|
|
end
|
|
end
|
|
|
|
% Rename the kept scan group to '/scan_1' for consistency
|
|
% This way every output file has a group called '/scan_1'
|
|
if ~strcmp(scan_group_full, '/scan_1')
|
|
% H5L.move renames/moves a group within the file
|
|
% Parameters: source_file, old_path, dest_file, new_path, options
|
|
H5L.move(fid, scan_group_full, fid, '/scan_1', 'H5P_DEFAULT', 'H5P_DEFAULT');
|
|
end
|
|
|
|
% Close the HDF5 file to save all changes
|
|
H5F.close(fid);
|
|
|
|
% ==========================================================
|
|
% RESTORE ORIGINAL OS DATES TO THE NEW FILE
|
|
% ==========================================================
|
|
if ispc
|
|
% Windows: Use .NET to set BOTH Creation and Modification time
|
|
try
|
|
fileObj = System.IO.FileInfo(outname);
|
|
fileObj.CreationTime = System.DateTime.Parse(orig_cDateOS);
|
|
fileObj.LastWriteTime = System.DateTime.Parse(orig_mDateOS);
|
|
catch
|
|
end
|
|
|
|
elseif ismac || isunix
|
|
% Mac/Linux: Use 'touch' to restore Modification time
|
|
try
|
|
timestamp_str = string(datetime(orig_mDateOS), 'yyyyMMddHHmm.ss');
|
|
system(sprintf('touch -m -t %s "%s"', timestamp_str, outname));
|
|
catch
|
|
end
|
|
% NOTE: Mac OS makes it incredibly difficult to change the Birth (Creation)
|
|
% time via standard commands. However, because we restored the Modification
|
|
% time perfectly, our "Paradox Check" in the main loop will automatically
|
|
% step in, realize Creation is newer than Modification, and fix the
|
|
% Creation date for us in the data table!
|
|
end
|
|
% ==========================================================
|
|
|
|
% Print confirmation message showing which file was created
|
|
% fprintf('Created: %s\n', outname);
|
|
end
|
|
|
|
% After all split files are successfully created, delete the original file
|
|
delete(infile);
|
|
% fprintf('Deleted original file: %s\n', infile);
|
|
|
|
else
|
|
% If the file has 3 or fewer groups, it doesn't need splitting
|
|
% disp('File has three or fewer groups. No processing required.');
|
|
end
|
|
|
|
end
|
|
|
|
|
|
|
|
|
|
function [creationDateOS, lastModifiedDate] = getOSDates(fullPath)
|
|
% GETOSDATES Extracts the true OS-level creation and modification dates
|
|
|
|
% Get standard modification date
|
|
d = dir(fullPath);
|
|
if isempty(d)
|
|
creationDateOS = '';
|
|
lastModifiedDate = '';
|
|
return;
|
|
end
|
|
lastModifiedDate = d.date;
|
|
|
|
% Get Creation Date based on Operating System
|
|
if ispc
|
|
% Windows: Use .NET framework for accurate Creation Time
|
|
try
|
|
fileObj = System.IO.FileInfo(fullPath);
|
|
creationDateOS = char(fileObj.CreationTime.ToString('dd-MMM-yyyy HH:mm:ss'));
|
|
catch
|
|
creationDateOS = d.date; % Fallback
|
|
end
|
|
|
|
elseif ismac || isunix
|
|
% macOS/Linux: Use the stat command to get the "Birth" time
|
|
[status, cmdOut] = system(['stat -f "%SB" -t "%d-%b-%Y %H:%M:%S" "', fullPath, '"']);
|
|
if status == 0
|
|
creationDateOS = strtrim(cmdOut);
|
|
|
|
% ==========================================================
|
|
% macOS COPIED FILE WORKAROUND
|
|
% If copied/downloaded, macOS sets Birth Time to 'today' but
|
|
% keeps the old Modification Time. If Creation > Modification,
|
|
% use the older Modification time as the true origin!
|
|
% ==========================================================
|
|
try
|
|
if datetime(creationDateOS) > datetime(lastModifiedDate)
|
|
creationDateOS = lastModifiedDate;
|
|
end
|
|
catch
|
|
% Silently continue if datetime conversion fails
|
|
end
|
|
% ==========================================================
|
|
else
|
|
creationDateOS = d.date; % Fallback
|
|
end
|
|
end
|
|
end |