beamtime/nappFileList.m
- Fix h5 LensMode: translate numeric codes to descriptive text (e.g. 1 ->
T_HiPPHAXPES) to match what .ibw files already store, applied across all
three h5read fallback branches, not just the oldest-format one
beamtime/nappGenerateSpectra.m
- Bug fixes
beamtime/nappPlot.m
- Bug fixes
import/importData.m (new)
- Ground script for one beamtime: import (nappFileList,
nappGenerateSpectra), sample/spot and photon-energy assignment per row
range, campaign metadata (CustomProperties) and column
descriptions/units set inline, tidy-up, and export to .csv/.mat/.h5
(via nappExportH5). Leaves FileList in the workspace for PublishReport.m
and callOpenbisSync.m to run against afterward.
import/nappExportH5.m (new)
- Export FileList + campaign metadata + raw source files to a
self-contained HDF5 archive
- Low-level HDF5 API (H5F/H5G/H5D/H5A/H5T/H5S/H5P) to fix
H5FD_truncate/"too many open files" errors from the top-level
h5create/h5write functions each reopening the file internally
- Explicit rowIndex attribute per row group, since HDF5 group listing
order isn't guaranteed to match table row order
openBIS/ (new)
- nappSyncToOpenbis.py, callOpenbisSync.m, testpybis.m, README.md
- openBIS sync pipeline: create-or-update per row matched by $name,
correct lowercase property codes, timestamp formatting, parent linking,
raw file upload, plot/photo/video thumbnail embedding, config/engine
split for maintainability -- see README.md for full details
- Implement @DEP_IN/OUT/RUN tag standard for MATLAB script metadata
- Add buildDependencyGraph.m for full project mapping and yEd (.tgf) export
- Add plotSingleFileDependencies.m and plotNodeHistory.m for targeted analysis
- Standardize output naming convention to 'Dependency_*' for Finder organization
- Configure high-resolution PNG export as default static fallback
* Transitioned nappNorm, nappDetermineShift, and nappCopyShift to use modern MATLAB arguments blocks, removing their legacy functionSignatures.json entries.
* Refactored nappNorm to rely on prior peak fitting results (e.g., from nappFitGaussXps) for normalization rather than performing the fitting internally.
* Decoupled energy calibration: nappDetermineShift now solely calculates internal binding energy shifts, and nappCopyShift distributes them using a unified matching engine ('earlier', 'after', 'spectrumNumber').
* Implemented a Universal Logging Block to track data provenance, automatically appending execution timestamps and caller script names to processed rows.
* Updated nappCombineSpectra to output a logDataSource cell array and cleanly name combined spectra.
* Created nappSaveDataTable for automatic, script-named .mat file saving, complete with a Live Editor interceptor.
* Added a publish-mode silencer to nappFitGaussXps to prevent optimizer spam in generated PDFs.
* Updated readme.md to document the new arguments structure, the 3-step calibration workflow, the matching engine, and the new table variables.
This commit replaces the numerically unstable Shirley background calculation in the peak fitting routine with a continuous, analytical Pseudo-Voigt Cumulative Distribution Function (CDF).
Previously, the solver struggled to converge and pushed peak parameters (area, width) into their absolute bounds due to two critical mathematical bugs in the objective function:
1. The 'flip' and Energy Axis Mutation Bug:
To handle kinetic energy scales (where the background steps up as energy decreases), the code mutated the energy axis (E = max(E)+100-E) inside the peak loop and attempted to use array flipping. This approach fundamentally decoupled the background step from the physical peak position. Array flipping mirrors data based on the length of the plot window, physically moving the Shirley step away from the true peak center. Furthermore, mutating E inside the loop corrupted the X-axis for subsequent peaks in multi-peak fits.
2. The Integration Cutoff ('idxregion') Bug:
The prior implementation used a hard Boolean cutoff (idxregion = (E >= pos)) to restrict the background calculation to only one side of the peak. Because Pseudo-Voigt peaks have long tails, this created a severe mathematical discontinuity (a sharp cliff) exactly at the peak center. Non-linear optimization solvers require smooth, continuous gradients to step correctly. The sharp cutoff destroyed the gradient, causing the solver to panic and lock parameters against their constraints.
The Fix:
- Removed all 'flip()' calls and energy array mutations.
- Replaced the hard cutoff with a continuous analytical CDF evaluated over the entire energy range.
- Handled binding vs. kinetic energy directionality cleanly by toggling between 'integral_fraction' and '(1 - integral_fraction)'.
Result:
The non-linear solver now receives perfectly smooth, mathematically continuous gradients, leading to rapid convergence. Peak parameters no longer crash into their bounds, and the plotted red fit line now perfectly aligns with the internally calculated solver residuals.
Adds native IDE parameter suggestions, name-value pair tracking, and enum choices for all NAPP functions based on their current inputParser configurations.
Both Scienta (.ibw) and SpecsLab (.xy) raw files capture multidimensional XPS data but use different terminology to structure it. Scienta uses "Region Iterations" and "Sequence Iterations". SpecsLab organizes data by "Groups" or "Regions" (different elemental energy sweeps), "Scans" (time-resolved repetitions), and "Curves". Scienta Region Iterations are automatically averaged to improve the signal-to-noise ratio, while Scienta Sequence Iterations and SpecsLab Scans are preserved to keep the time-resolved data. For SpecsLab files specifically, distinct Regions are split into separate table rows. Ultimately, the preserved time-resolved data is saved in the table's image variable as a 2D matrix ([Energy bins × Sequence iterations/Scans]).
File type handling: Added .xy as a recognized XPS format. Currently, the script differentiates data types strictly by file extension (NEXAFS: .h5, XPS: .ibw and .xy). This extension-based differentiation needs to be improved in future updates.
Metadata parsing: Implemented global header scanning for .xy files to correctly extract instrument parameters (Acquisition Mode, Lens Mode, Slit, Workfunction) located before the first region block.
Timezone alignment: Added timestamp conversion for .xy files from UTC to local time (Europe/Zurich) to ensure synchronization with LabView logs.
Region separation: Modified .xy parsing to append a _RegX suffix to filenames. This correctly splits single .xy files that contain multiple elements or energy sweeps into distinct table rows so they can be processed individually.
Import efficiency: Added an originalFileName variable. The script now cross-references the input DataTable and skips files that have already been imported to reduce execution time on re-runs.
Testing status: Processing of .xy data has currently only been tested and debugged with nappGenerateSpectra and nappPlot. Compatibility with other functions in the suite remains untested.
BREAKING CHANGES:
- Single-spectrum fitting only (enforces exactly one row selected)
- Gauss fitting now uses FWHM parameter instead of Gaussian width sigma
- Removed multi-row batch processing capability
NEW FEATURES:
- Generic model architecture: arbitrary number of peaks (determined by length of FitParameter.peakArea)
- Full Shirley background support for both Gaussian and Pseudo-Voigt peak shapes
- Comprehensive parameter validation system with detailed error messages
- Support for shared vs per-peak Shirley K parameters
- New fitting modes: 'positionShirley', 'widthShirley', 'widthPositionShirley'
- New custom mode: 'TwoSpinOrbitDoubletPseudoVoigtShirley'
- Automatic field name typo correction ('Bond' → 'Bound')
- Enhanced results export: now includes peakEta and peakShirleyK columns
REFACTORING:
- Centralized peak and background equations in helper functions
- gaussPeak(), lorentzPeak(), pseudoVoigtPeak()
- linearBackground(), activeShirleyBackgroundFwhm()
- fwhm2w() conversion utility
- Extracted model generation to genericModelGenerator()
- Modular validation helpers: checkFieldDimension(), checkBoundsPair()
- Standardized parameter vector layouts documented for all mode/background/peak-type combinations
- Improved column resizing logic with resizeColumnHelper()
- Enhanced plotting integration with nappPlot.m and nappAddFitToPlot.m
ARCHITECTURE:
- Unified parameter extraction for all fitting modes in centralized switch-case structure
- Consistent FWHM-based width parameterization across all peak types
- Mode-specific Shirley K sizing (scalar for shared modes, array for per-peak modes)
- Comprehensive inline documentation of parameter vector layouts (50+ combinations)
IMPROVEMENTS:
- Backward compatibility for 'gauss' width mode (auto-converts to FWHM)
- Robust bounds checking with detailed diagnostic messages
- Enhanced figure generation: residuals plot + fit visualization in tiledlayout
- Better error handling for empty/malformed data structures
- NaN-padded result arrays for variable peak counts
FILE SIZE: ~73KB → ~152KB (107% increase due to comprehensive documentation and generic architecture)
bartels-rausch
leeman_n