cbcd98c10c
This directory was accidentally omitted from the merge-release branch during the PSI code merge.
282 lines
17 KiB
XML
282 lines
17 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<?oxygen RNGSchema="http://www.oasis-open.org/docbook/xml/5.0/rng/docbook.rng" type="xml"?>
|
|
<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink"
|
|
version="5.0">
|
|
<info><title>File commands</title><author>
|
|
<personname>Ferdi Franceschini</personname>
|
|
</author>
|
|
<date/>
|
|
<bibliosource>
|
|
</bibliosource>
|
|
</info>
|
|
<sect1>
|
|
<title>Introduction</title>
|
|
<sect2>
|
|
<title>Filenames</title>
|
|
<para>SICS provides methods to create and save files. You can create a single file, and
|
|
save either a single dataset, or multiple datasets to the one file. You can also
|
|
create and manage collections of files, and save single or multiple datasets to
|
|
files in the collection</para>
|
|
<para>SICS automatically creates the filename. The filenames have the form <variablelist>
|
|
<varlistentry>
|
|
<term>xxxnnnnnnn.nx.hdf</term>
|
|
<listitem>
|
|
<para>where xxx is a 3 letter abbreviation of the instrument</para>
|
|
<para>QKK - quokka</para>
|
|
<para>ECH - echidna</para>
|
|
<para>WOM - wombat</para>
|
|
<para>KOW - kowari</para>
|
|
<para>PLA - platypus</para>
|
|
<para>TPN - taipan</para>
|
|
<para>nnnnnnn is a 7 numeral sequence number, starting at 0000000 when
|
|
the facility commenced operation, and is automatically incremented
|
|
by SICS. </para>
|
|
<para>The file /usr/local/sics/DataNumber is used to keep track of the
|
|
number. DO NOT edit this file. </para>
|
|
<para>.nx denotes that the file is a NeXus file. </para>
|
|
<para>.hdf denotes the file is an hdf5 (binary) file. </para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist></para>
|
|
<para>e.g. QKK0001234.nx.hdf </para>
|
|
</sect2>
|
|
<sect2>
|
|
<title>File Format. NeXus</title>
|
|
<para>Files are saved using the ANSTO interpretation of the NeXus standard. </para>
|
|
<para>SICS support both the xml and hdf5 form. For performance of reading and writing,
|
|
by default we write hdf5 binary. </para>
|
|
<para>SICS can also be configured to write xml. This is set in
|
|
<filename>nxscripts_common_1.tcl</filename>. Set the file,format element of the
|
|
state array to "xml" </para>
|
|
</sect2>
|
|
<sect2>
|
|
<title>File Content</title>
|
|
<para>This section will give only a very brief overview of NeXus. Further reading can be
|
|
found at the <link xlink:href="http://www.nexusformat.org">NeXus webite,
|
|
www.nexusformat.org</link>
|
|
</para>
|
|
<para>NeXus is a hierachical data format; data is saved in groups and these groups live
|
|
under entries. It a similar structure to directories on a file system. We have made
|
|
a policy decision at the Bragg Institute to have only one entry per file. This entry
|
|
may contain a variable parameter or scan, where e.g. temperature is varied. If you
|
|
use the <command>runscan</command> command, histogram data is taken at discrete
|
|
temperatures. Temperature will be a vector in the file, and the histogram data may
|
|
be a data cube of 2 dimensional x,y or 3 dimensional x,y,t histogram arrays.</para>
|
|
<para>There are 4 groups in NeXus. User, Sample, Instrument and Data. SICS will write
|
|
the data it acquires to one of these groups. The content that is saved, and where in
|
|
the file it is saved to is controlled by configuration files. </para>
|
|
<para><filename>/usr/local/sics/server/config/nexus</filename> contains
|
|
<filename>*.dic</filename> dictionary files. These files tell SICS how to map a SICS
|
|
object to a location in a NeXus file, and what type the data will be, and its
|
|
attributes e.g units. Below is an example from nexus.dic</para>
|
|
<para>
|
|
<programlisting>
|
|
samphi = /entry1,NXentry/sample,NXsample/SDS sample_phi
|
|
-type NX_FLOAT32 -rank 1 -dim {-1}
|
|
-attr {units,degrees} -attr {long_name,sample_phi}
|
|
</programlisting>
|
|
</para>
|
|
<para>Changes to configurations are done by the facility. Dictionaries can be checked
|
|
with check_instdict.tcl and check_sicsobj_attributes.tcl.</para>
|
|
<para>By default, if the SICS object exists and there is an entry in the dictionary,
|
|
then it will be saved to the data file. There is a second hierarchy of SICS objects
|
|
which is used by Gumtree for control. This is called hipadaba. We won't go into
|
|
detail about hipadaba in this manual, but it is important for this discussion to
|
|
know how hipadaba controls saving of SICS objects. Hipadaba has the same structure
|
|
as NeXus. The hipadaba tree when initially created by SICS is a complete NeXus tree,
|
|
which is then pruned to contain only those nodes that exist for that instrument.
|
|
This allows any node to be added to <filename>nexus.dic</filename> for an instrument
|
|
without having to change hipadaba. There are dictionary files for hipadaba found at
|
|
<filename>/usr/local/sics/server/config/hipadaba/</filename>. In general, there
|
|
is no instrument specific information in these files. Every node in hipadaba has
|
|
data and nxsave attributes. By default, nxsave is set to true, and if the node
|
|
contains data, data is set to true. If either of these is set to false, then the
|
|
data will not be saved. </para>
|
|
</sect2>
|
|
<sect2>
|
|
<title>File Locations</title>
|
|
<para>A mutable (read-write) copy of the file is made a few minutes after the
|
|
experiment, and is available at e.g.
|
|
<filename>/experiments/wombat/data/proposal/01234</filename> where
|
|
<filename>01234</filename> is the proposal number for the current
|
|
experiment. </para><para>An archive of the file is made to the cycle directory
|
|
e.g. <filename>/experiments/wombat/data/cycle/040</filename> where
|
|
<filename>040</filename> is the reactor cycle number. </para>
|
|
<para>File are written to
|
|
<filename>/usr/local/sics/data</filename> of the
|
|
ics1-<replaceable>australian_fauna</replaceable> computer. This path is
|
|
configured in <filename>server_config.tcl</filename> by setting the
|
|
<varname>SicsDataPath</varname> variable. Posix symbolic links are used to link
|
|
the directory to the appropriate directory on filer.nbi.ansto.gov.au, under the
|
|
<filename>/experiments</filename> mount point e.g
|
|
<filename>/experiments/wombat/data/current</filename>. You can mount this
|
|
directory on the MS Windows machine
|
|
dav1-<replaceable>australian_fauna</replaceable>. Files in this location are
|
|
read-only. The facility requires an immutable version of the data. </para>
|
|
</sect2>
|
|
</sect1>
|
|
<sect1>
|
|
<title>File Commands. Single Files</title>
|
|
<sect2>
|
|
<title>newfile command</title>
|
|
<para>
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><command>newfile</command>
|
|
<replaceable>file_type</replaceable>
|
|
<optional>scratch</optional></term>
|
|
<listitem>
|
|
<para>creates a new file of type <replaceable>file_type</replaceable>
|
|
ready to write to. The command does write any information to disk. </para>
|
|
<para>To save data, use the <command>save</command> command.</para>
|
|
<para>You can only hold a reference to one file. If you need to
|
|
reference a number of files, then use newfile_collection. </para>
|
|
<para>Only use the optional <optional>scratch</optional> if you want to
|
|
write data to a scratch file. The file will be overwritten with the
|
|
next invocation of this option</para>
|
|
<para><replaceable>file_type</replaceable> may have the following
|
|
values: </para>
|
|
<para><option>BEAM_MONITOR</option> Saves data from the configured beam
|
|
monitors, histogram memory data is not saved.</para>
|
|
<para><option>HISTOGRAM_T</option> Saves histogram total time data and
|
|
beam monitor data.</para>
|
|
<para><option>HISTOGRAM_X</option> Saves histogram x data and beam
|
|
monitor data.</para>
|
|
<para><option>HISTOGRAM_XT</option> Saves histogram x,t data and beam
|
|
monitor data.</para>
|
|
<para><option>HISTOGRAM_Y</option> Saves histogram y data and beam
|
|
monitor data.</para>
|
|
<para><option>HISTOGRAM_YT</option> Saves histogram y,t data and beam
|
|
monitor data.</para>
|
|
<para><option>HISTOGRAM_XY</option> Saves histogram x,y data and beam
|
|
monitor data.</para>
|
|
<para><option>HISTOGRAM_XYT</option> Saves histogram total x,y,t data
|
|
and beam monitor data.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</para>
|
|
</sect2>
|
|
<sect2>
|
|
<title>save command</title>
|
|
<para>
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><command>save</command>
|
|
<replaceable>index</replaceable>
|
|
</term>
|
|
<listitem>
|
|
<para>saves data to disk. </para>
|
|
<para><replaceable>index</replaceable> is the index of data to be saved,
|
|
starting with 0. To save your first slice of data you would save 0.
|
|
This provides you with a complete NeXus file. You may be doing After
|
|
you acquire you next slice of data, you would save 1, then save 2
|
|
etc. </para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</para>
|
|
</sect2>
|
|
<sect2>
|
|
<title>Other single file commands</title>
|
|
<para>
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><command>killfile</command></term>
|
|
<listitem>
|
|
<para>decrements the data number used for SICS file writing and thus
|
|
consequently overwrites the last datafile. This is useful when
|
|
useless data files have been created during tests. As this is
|
|
critical command in normal user operations, this command requires
|
|
managers privilege. </para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
<sect1>
|
|
<title>File Collection Commands</title>
|
|
<sect2>
|
|
<title>newfile_collection command</title>
|
|
<para>
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><command>newfile_collection</command>
|
|
<option> -labels </option>
|
|
<replaceable>{sample1 sample2}</replaceable>
|
|
<option>-filetype</option>
|
|
<replaceable>file_type</replaceable>
|
|
<option>-savetype</option>
|
|
<replaceable>save_type</replaceable>
|
|
</term>
|
|
<listitem>
|
|
<para>Whereas newfile creates one file, newfile_collection will create
|
|
as many files as there are labels. The command does write any
|
|
information to disk. </para>
|
|
<para>To save data, use the <command>save_collection</command> command</para>
|
|
<para>Example: You have a multi-sample changer or robot. You want to do
|
|
a measurement on each sample at multiple temperatures. Your
|
|
experimental sequence has the sample changer as the fastest varying
|
|
parameter (inner loop), and temperature change as the slowest
|
|
varying parameter (outer loop). You want to record all temperature
|
|
data for a sample in one file.</para>
|
|
<para><option>-savetype</option>
|
|
<replaceable>save_type</replaceable> may have the following values: </para>
|
|
<para>
|
|
<option>data</option> writes to a normal data file</para>
|
|
<para><option>scratch</option> writes to a scratch file. The file will
|
|
be overwritten with the next invocation of this option. Used mainly
|
|
for testing.</para>
|
|
<para><option>-filetype</option>
|
|
<replaceable>file_type</replaceable> may have the following values: </para>
|
|
<para><option>BEAM_MONITOR</option> Saves data from the configured beam
|
|
monitors, histogram memory data is not saved.</para>
|
|
<para><option>HISTOGRAM_T</option> Saves histogram total time data and
|
|
beam monitor data.</para>
|
|
<para><option>HISTOGRAM_X</option> Saves histogram x data and beam
|
|
monitor data.</para>
|
|
<para><option>HISTOGRAM_XT</option> Saves histogram x,t data and beam
|
|
monitor data.</para>
|
|
<para><option>HISTOGRAM_Y</option> Saves histogram y data and beam
|
|
monitor data.</para>
|
|
<para><option>HISTOGRAM_YT</option> Saves histogram y,t data and beam
|
|
monitor data.</para>
|
|
<para><option>HISTOGRAM_XY</option> Saves histogram x,y data and beam
|
|
monitor data.</para>
|
|
<para><option>HISTOGRAM_XYT</option> Saves histogram total x,y,t data
|
|
and beam monitor data.</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</para>
|
|
</sect2>
|
|
<sect2>
|
|
<title>save_collection command</title>
|
|
<para>
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term><command>save_collection</command>
|
|
<option>-index</option>
|
|
<replaceable>val</replaceable>
|
|
<command>-labels </command>
|
|
<replaceable>sample1</replaceable></term>
|
|
<listitem>
|
|
<para>saves data to disk within a collection (multiple files)</para>
|
|
<para><command>-index</command><replaceable>val</replaceable> is the
|
|
index of data to be saved, starting with 0. To save your first slice
|
|
of data you would save 0. This provides you with a complete NeXus
|
|
file. You may be doing After you acquire you next slice of data, you
|
|
would save 1, then save 2 etc.</para>
|
|
<para><command>-labels </command>
|
|
<replaceable>sample1</replaceable> will save to the file referenced
|
|
by the label <replaceable>sample1</replaceable>. You would put all
|
|
data relating to a sample into this one file. </para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
</chapter>
|