sinq-reflectometry/eos
0
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

start on documentation

Browse Source
This commit is contained in:
stahn
2024-02-26 15:31:51 +01:00
parent 6fc82047f0
commit 36dcccb743
2 changed files with 326 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

308
man.md Normal file
View File

@@ -0,0 +1,308 @@
# documentation for **neos.py**
**neos** is a python program used at the neutron reflectometer *Amor* at *SINQ*
to turn *raw data*, i.e. from the *nexus hdf5* format,
into reduced an *orso* comptible *reflectivity file*.
raw: *nexus hdf5* format:
- event arrays for detector and monitor events
- arrays for device propertise
- entries for instrument configuration
reduced: *orso reflectivity* format:
- header with information about
- data origin
- measurement conditions
- reduction steps
- array with basic or expanded reflectivity data (in the simplest version: the *reflectivity curve*)
## environment
**neos** (version 2.0 and later) was developen with python3.9.
The following (non-trivial) python modules are required:
- `numpy`
- `h5py`
- `orsopy`
## usage
**neos** is using command line arguments to
- find the raw data
- overwrite default values
- define the parameter range for reduction
- define reduction steps
- define the output path and name
More detailed explanations for some arguments are given in the next section.
### output for `python eos.py -h`:
``` man
usage: neos.py [-h] [-d DATAPATH] [-Y YEAR]
[-n FILEIDENTIFIER [FILEIDENTIFIER ...]]
[-r NORMALISATIONFILEIDENTIFIER [NORMALISATIONFILEIDENTIFIER ...]]
[-sub SUBTRACT] [-o OUTPUTNAME]
[-of OUTPUTFORMAT [OUTPUTFORMAT ...]]
[--offSpecular OFFSPECULAR] [-a QRESOLUTION]
[-ts TIMESLIZE [TIMESLIZE ...]] [-s SCALE [SCALE ...]]
[-S AUTOSCALE AUTOSCALE] [-l LAMBDARANGE LAMBDARANGE]
[-t THETARANGE THETARANGE] [-T THETARANGER THETARANGER]
[-y YRANGE YRANGE] [-q QZRANGE QZRANGE] [-cs CHOPPERSPEED]
[-cp CHOPPERPHASE] [-co CHOPPERPHASEOFFSET] [-m MUOFFSET]
[-mu MU] [-nu NU] [-sm SAMPLEMODEL]
eos reads data from (one or several) raw file(s) of the .hdf format, performs
various corrections, conversations and projections and exports the resulting
reflectivity in an orso-compatible format.
optional arguments:
-h, --help show this help message and exit
input data:
-d DATAPATH, --dataPath DATAPATH
relative path to directory with .hdf files
-Y YEAR, --year YEAR year the measurement was performed
-n FILEIDENTIFIER [FILEIDENTIFIER ...], --fileIdentifier FILEIDENTIFIER [FILEIDENTIFIER ...]
file number(s) or offset (if negative)
-r NORMALISATIONFILEIDENTIFIER [NORMALISATIONFILEIDENTIFIER ...], --normalisationFileIdentifier NORMALISATIONFILEIDENTIFIER [NORMALISATIONFILEIDENTIFIER ...]
file number(s) of normalisation measurement
-sub SUBTRACT, --subtract SUBTRACT
R(q_z) curve to be subtracted (in .Rqz.ort format
output:
-o OUTPUTNAME, --outputName OUTPUTNAME
output file name (withot suffix)
-of OUTPUTFORMAT [OUTPUTFORMAT ...], --outputFormat OUTPUTFORMAT [OUTPUTFORMAT ...]
--offSpecular OFFSPECULAR
-a QRESOLUTION, --qResolution QRESOLUTION
q_z resolution
-ts TIMESLIZE [TIMESLIZE ...], --timeSlize TIMESLIZE [TIMESLIZE ...]
time slizing <interval> ,[<start> [,stop]]
-s SCALE [SCALE ...], --scale SCALE [SCALE ...]
scaling factor for R(q_z)
-S AUTOSCALE AUTOSCALE, --autoscale AUTOSCALE AUTOSCALE
scale to 1 in the given q_z range
masks:
-l LAMBDARANGE LAMBDARANGE, --lambdaRange LAMBDARANGE LAMBDARANGE
wavelength range
-t THETARANGE THETARANGE, --thetaRange THETARANGE THETARANGE
absolute theta range
-T THETARANGER THETARANGER, --thetaRangeR THETARANGER THETARANGER
relative theta range
-y YRANGE YRANGE, --yRange YRANGE YRANGE
detector y range
-q QZRANGE QZRANGE, --qzRange QZRANGE QZRANGE
q_z range
overwrite:
-cs CHOPPERSPEED, --chopperSpeed CHOPPERSPEED
chopper speed in rpm
-cp CHOPPERPHASE, --chopperPhase CHOPPERPHASE
chopper phase
-co CHOPPERPHASEOFFSET, --chopperPhaseOffset CHOPPERPHASEOFFSET
phase offset between chopper opening and trigger pulse
-m MUOFFSET, --muOffset MUOFFSET
mu offset
-mu MU, --mu MU value of mu
-nu NU, --nu NU value of nu
-sm SAMPLEMODEL, --sampleModel SAMPLEMODEL
1-line orso sample model description
```
## read data file(s)
### absolute minimum
purposes:
- fast access to human-readable meta data in the output header
- get an idea about $q_z$ range and statistics
actions:
- read in one raw data file
- convert the event stream into an $I(\lambda, \alpha_f)$ map
- project this map onto $q_z$ io give an $I(q_z)$ curve
- write this curve in *orso* format to disk
example:
`> python neos.py -n 456 -o foo`
looks for the file `amor<year>n000456.hdf` in one of the default locations
(`./`, `./raw/`, `../raw`, local raw data directory on Amor) and
writes the output to `foo.Rqz.ort`.
### normaliation
purposes:
- fast access to human-readable meta data in the output header
- get a reduced and (partially) corrected reflectivity curve
actions:
- read in raw data file(s) and raw normalisation file(s)
- convert the normalisation measurement into a $N(\lambda, z_{detector})$
map containing information about guide and detector efficiencies,
illuminated detector area and incoming intensity.
- convert the event stream into an $I(\lambda, \alpha_f)$ map
- normalisation: $R(\lambda, \alpha_f)_{la} = I(\lambda, \alpha_f)_{la} / N(\lambda, z_{detector})_{la}$.
- project this map onto $q_z$ io give a $R(q_z)$ curve (not necessarily scaled)
- write this curve in *orso* format to disk
example:
`> python neos.py -n 456 -r 123 -o foo`
looks for the files `amor<year>n000456.hdf` (reflectivity) and `amor<year>n000123.hdf`
(normalisation) in one of the default locations
(`./`, `./raw/`, `../raw`, local raw data directory on Amor) and
writes the output to `foo.Rqz.ort`.
### read multiple files
#### for the same instrument parameter set (same $\mu$)
The arguments of the keys `-n` and `-r` have the general form
`<start1>[-<end1>[:<increment1]][,<start2>[-<end2>[:<increment2]],...]`
Each number range is defined by a star value, an optional stop value and an
optional increment. Various ranges are separated by a `,`.
example:
`20-25:2,28-30,40` resolves into the list `[20, 22, 24, 28, 29, 30, 40]`
action:
Effectively, the event streams found in the the various files are merged and
processed together.
#### for different parameter sets, or to prevent merging
The key `-n` accepts more than one argument of the type defined above. The
(set of) data file(s) related to one aregument are merged and give one
reflectivity curve (one `data_set`) in the output file. The reflectivity
curves for more than one argument are separated in the output file
by the separator `# data-set: <identifier>` in the output file.
example:
`> python neos.py -n 20,21 30 -r 123 -o foo`
results in two reflectivity curves, the first made from files #20 and #21,
the second from file #30. Both are saved in `foo.Rqz.ort`.
warning:
`-r` does accept only one argument!
### misc.
The raw file name is created using the file number and the actual year. In case
the data to be processed were recorded in a previous year, this must be
explicitely stated with `-Y <year>`.
The default location for the output (and for starting the search for the input files)
iis the present working directory. This can be altered by using the argument
`-d <directory>`.
It is possible to provide a $R(q_z)$ curve in `.Rqz.ort` format to be subtracted
from the reduced data. E.g. to emphasize the high-$q$ region on a linear scale, or
to illustrate changes in a series of measurements. The argument is
`-sub <filename>`.
## output options
### other formats
Besides the standard *orso* format `.Rqz.ort`, there is the option to
write the $R(\lambda, \alpha_f)$ array and the related input, normalisation and
mask arrays. This output can help with the sample alignment or the readjustment
of parameters (see below), or it can be used for debugging the data processing or
instrument operation. The suffix of this output is `.Rlt.ort`
The format is chosen by using or several of the argumnets
- `Rqz.ort`, `Rlt.ort`, `Rqz.orb`, `Rlt.orb`
- `Rqz` (= `Rqz.ort` and `Rqz.orb`)
- `Rlt` (= `Rlt.ort` and `Rlt.orb`),
- `ort` (= `Rqz.ort` and `Rlt.ort`),
- `orb` (= `Rqz.orb` and `Rlt.orb`)
where `.orb` will be the future nexus comptibe output format.
### $q_z$ binning
The $R(\lambda, \alpha_f)$ arrays are projected onto a $q_z$ grid withbin boundaries
defined by:
$$
q_z \in \left{ \begin{array}{ll}
\[ 0, i \cdot 0.1 \cdot c\] & \forall q_z < q_\mathrm{base} \\
\[ q_\mathrm{base} \cdot (1+c)^j \] & \forall q_z \gt q_\mathrm{base}
\right.
$$
$q_\mathrm{base} = 0.1\,\mathrm{\AA}^{-1}$ is fixed for the moment.
The *resolution* can be chosen with `-a` among the values
[0.005, 0.01, 0.02, 0.025, 0.04, 0.05, 0.1, 1]
(this is restricted to ensure a *smooth* transition between the
linear and exponential regions).
### intensity scaling
The argument `-s <value>` leads to a multiplication of all $R(q_z)$ curves
with `<value>`. This is useful only for one curve, only, or in combination with the
`-S` argument.
$R(q_z)$ of the first reflectivity curve can be scaled to 1 in the $q_z$ interval define
by `-S <start> <stop>`. The following $R(q_z)$ curves are then scaled to match the
respective previous one.
### time-slizing
One (combined) data set can be chopped in slizes with the argument
`-ts <interval> [<start> [stop>]]`
where `<interval>` is the time interval length in seconds. The chopping starts
`<start>` seconds after the start of the measurement (default: 0) and ends at
`<stop>` seconds (default: end of the measurement).
All the resulting $R(q_z)$ curves are stored in one file, one after the other. An additional
column is added with the start time of the respective slize. This enables fast plotting.
example:
`python -n 20-22 -r 123 -ts 60 1200 4000 -f foo`
The event streams of the measurements #20, #21 and #22 are merged. All events before
$t = 1200\,\mathrm{s}$ with respect to the start of meausrement #20 are discarded.
Then until $t = 4020\,\mathrm{s}$ (or the end of measurement #22) a $R(q_z)$ curve is generated
for each $60\,\mathrm{s}$ interval.
## masksing
-l LAMBDARANGE LAMBDARANGE, --lambdaRange LAMBDARANGE LAMBDARANGE
wavelength range
-t THETARANGE THETARANGE, --thetaRange THETARANGE THETARANGE
absolute theta range
-T THETARANGER THETARANGER, --thetaRangeR THETARANGER THETARANGER
relative theta range
-y YRANGE YRANGE, --yRange YRANGE YRANGE
detector y range
-q QZRANGE QZRANGE, --qzRange QZRANGE QZRANGE
q_z range
overwrite:
-cs CHOPPERSPEED, --chopperSpeed CHOPPERSPEED
chopper speed in rpm
-cp CHOPPERPHASE, --chopperPhase CHOPPERPHASE
chopper phase
-co CHOPPERPHASEOFFSET, --chopperPhaseOffset CHOPPERPHASEOFFSET
phase offset between chopper opening and trigger pulse
-m MUOFFSET, --muOffset MUOFFSET
mu offset
-mu MU, --mu MU value of mu
-nu NU, --nu NU value of nu
-sm SAMPLEMODEL, --sampleModel SAMPLEMODEL
1-line orso sample model description

29
neos.py
View File

@@ -18,7 +18,7 @@ conventions (not strictly followed, yet):
"""
__version__ = '2.0'
__date__ = '2024-01-29'
__date__ = '2024-02-23'
import os
import sys
@@ -525,17 +525,24 @@ def normalisation_map(short_notation):
norm_lz, bins_l, bins_z = np.histogram2d(lamda_e, detZ_e, bins = (grid.lamda(), grid.z()))
norm_lz = np.where(norm_lz>0, norm_lz, np.nan)
# TODO: correct for the SM reflectivity
# q_lz
# -> Rsm_lz
lamda_l = grid.lamda()
theta_z = normAngle + fromHDF.delta_z
lamda_lz = (grid.lz().T*lamda_l[:-1]).T
theta_lz = grid.lz()*theta_z
qz_lz = 4.0*np.pi * np.sin(np.deg2rad(theta_lz)) / lamda_lz
Rsm_lz = np.ones(np.shape(qz_lz))
Rsm_lz = np.where(qz_lz>0.0217, 1-(qz_lz-0.0217)*(0.0625/0.0217), Rsm_lz)
Rsm_lz = np.where(qz_lz>0.0217*5, np.nan, Rsm_lz)
norm_lz = norm_lz / Rsm_lz
if len(lamda_e) > 1e6:
head = f'normalisation matrix based on the measurements \n\
{fromHDF.file_list} \n\
nu - mu = {normAngle}\n\
shape= {np.shape(norm_lz)} (lambda, z) \n\
measured at mu = {fromHDF.mu:6.3f} deg \n\
N(l_lambda, z) = theta(z) / sum_i=-1..1 I(l_lambda+i, z)'
head = head.replace(' ', '')
head = ('normalisation matrix based on the measurements\n'
f'{fromHDF.file_list}\n'
f'nu - mu = {normAngle}\n'
f'shape= {np.shape(norm_lz)} (lambda, z)\n'
f'measured at mu = {fromHDF.mu:6.3f} deg\n'
f'N(l_lambda, z) = theta(z) / sum_i=-1..1 I(l_lambda+i, z)')
head = head.replace('../', '')
head = head.replace('./', '')
head = head.replace('raw/', '')
np.savetxt(f'{clas.dataPath}/{name}.norm', norm_lz, header = head)
normFileList = fromHDF.file_list
@@ -758,7 +765,7 @@ def main():
if clas.timeSlize:
wallTime_e = fromHDF.wallTime_e
headerRqz = fileio.Orso(header.data_source(), header.reduction, header.columns())
headerRqz = fileio.Orso(header.data_source(), header.reduction, header.columns())
columns = np.append(header.columns(), fileio.Column('time', 's', 'time relative to start of measurement series'))
interval = clas.timeSlize[0]