Manual copied from ANSTO branch and committed to RELEASE-3_0 branch.
This directory was accidentally omitted from the merge-release branch during the PSI code merge.
This commit is contained in:
Nick Hauser
281
site_ansto/manual/dbSICSch20_file_commands.xml
Normal file
281
site_ansto/manual/dbSICSch20_file_commands.xml
Normal file
@@ -0,0 +1,281 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<?oxygen RNGSchema="http://www.oasis-open.org/docbook/xml/5.0/rng/docbook.rng" type="xml"?>
|
||||
<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||
version="5.0">
|
||||
<info><title>File commands</title><author>
|
||||
<personname>Ferdi Franceschini</personname>
|
||||
</author>
|
||||
<date/>
|
||||
<bibliosource>
|
||||
</bibliosource>
|
||||
</info>
|
||||
<sect1>
|
||||
<title>Introduction</title>
|
||||
<sect2>
|
||||
<title>Filenames</title>
|
||||
<para>SICS provides methods to create and save files. You can create a single file, and
|
||||
save either a single dataset, or multiple datasets to the one file. You can also
|
||||
create and manage collections of files, and save single or multiple datasets to
|
||||
files in the collection</para>
|
||||
<para>SICS automatically creates the filename. The filenames have the form <variablelist>
|
||||
<varlistentry>
|
||||
<term>xxxnnnnnnn.nx.hdf</term>
|
||||
<listitem>
|
||||
<para>where xxx is a 3 letter abbreviation of the instrument</para>
|
||||
<para>QKK - quokka</para>
|
||||
<para>ECH - echidna</para>
|
||||
<para>WOM - wombat</para>
|
||||
<para>KOW - kowari</para>
|
||||
<para>PLA - platypus</para>
|
||||
<para>TPN - taipan</para>
|
||||
<para>nnnnnnn is a 7 numeral sequence number, starting at 0000000 when
|
||||
the facility commenced operation, and is automatically incremented
|
||||
by SICS. </para>
|
||||
<para>The file /usr/local/sics/DataNumber is used to keep track of the
|
||||
number. DO NOT edit this file. </para>
|
||||
<para>.nx denotes that the file is a NeXus file. </para>
|
||||
<para>.hdf denotes the file is an hdf5 (binary) file. </para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
</variablelist></para>
|
||||
<para>e.g. QKK0001234.nx.hdf </para>
|
||||
</sect2>
|
||||
<sect2>
|
||||
<title>File Format. NeXus</title>
|
||||
<para>Files are saved using the ANSTO interpretation of the NeXus standard. </para>
|
||||
<para>SICS support both the xml and hdf5 form. For performance of reading and writing,
|
||||
by default we write hdf5 binary. </para>
|
||||
<para>SICS can also be configured to write xml. This is set in
|
||||
<filename>nxscripts_common_1.tcl</filename>. Set the file,format element of the
|
||||
state array to "xml" </para>
|
||||
</sect2>
|
||||
<sect2>
|
||||
<title>File Content</title>
|
||||
<para>This section will give only a very brief overview of NeXus. Further reading can be
|
||||
found at the <link xlink:href="http://www.nexusformat.org">NeXus webite,
|
||||
www.nexusformat.org</link>
|
||||
</para>
|
||||
<para>NeXus is a hierachical data format; data is saved in groups and these groups live
|
||||
under entries. It a similar structure to directories on a file system. We have made
|
||||
a policy decision at the Bragg Institute to have only one entry per file. This entry
|
||||
may contain a variable parameter or scan, where e.g. temperature is varied. If you
|
||||
use the <command>runscan</command> command, histogram data is taken at discrete
|
||||
temperatures. Temperature will be a vector in the file, and the histogram data may
|
||||
be a data cube of 2 dimensional x,y or 3 dimensional x,y,t histogram arrays.</para>
|
||||
<para>There are 4 groups in NeXus. User, Sample, Instrument and Data. SICS will write
|
||||
the data it acquires to one of these groups. The content that is saved, and where in
|
||||
the file it is saved to is controlled by configuration files. </para>
|
||||
<para><filename>/usr/local/sics/server/config/nexus</filename> contains
|
||||
<filename>*.dic</filename> dictionary files. These files tell SICS how to map a SICS
|
||||
object to a location in a NeXus file, and what type the data will be, and its
|
||||
attributes e.g units. Below is an example from nexus.dic</para>
|
||||
<para>
|
||||
<programlisting>
|
||||
samphi = /entry1,NXentry/sample,NXsample/SDS sample_phi
|
||||
-type NX_FLOAT32 -rank 1 -dim {-1}
|
||||
-attr {units,degrees} -attr {long_name,sample_phi}
|
||||
</programlisting>
|
||||
</para>
|
||||
<para>Changes to configurations are done by the facility. Dictionaries can be checked
|
||||
with check_instdict.tcl and check_sicsobj_attributes.tcl.</para>
|
||||
<para>By default, if the SICS object exists and there is an entry in the dictionary,
|
||||
then it will be saved to the data file. There is a second hierarchy of SICS objects
|
||||
which is used by Gumtree for control. This is called hipadaba. We won't go into
|
||||
detail about hipadaba in this manual, but it is important for this discussion to
|
||||
know how hipadaba controls saving of SICS objects. Hipadaba has the same structure
|
||||
as NeXus. The hipadaba tree when initially created by SICS is a complete NeXus tree,
|
||||
which is then pruned to contain only those nodes that exist for that instrument.
|
||||
This allows any node to be added to <filename>nexus.dic</filename> for an instrument
|
||||
without having to change hipadaba. There are dictionary files for hipadaba found at
|
||||
<filename>/usr/local/sics/server/config/hipadaba/</filename>. In general, there
|
||||
is no instrument specific information in these files. Every node in hipadaba has
|
||||
data and nxsave attributes. By default, nxsave is set to true, and if the node
|
||||
contains data, data is set to true. If either of these is set to false, then the
|
||||
data will not be saved. </para>
|
||||
</sect2>
|
||||
<sect2>
|
||||
<title>File Locations</title>
|
||||
<para>A mutable (read-write) copy of the file is made a few minutes after the
|
||||
experiment, and is available at e.g.
|
||||
<filename>/experiments/wombat/data/proposal/01234</filename> where
|
||||
<filename>01234</filename> is the proposal number for the current
|
||||
experiment. </para><para>An archive of the file is made to the cycle directory
|
||||
e.g. <filename>/experiments/wombat/data/cycle/040</filename> where
|
||||
<filename>040</filename> is the reactor cycle number. </para>
|
||||
<para>File are written to
|
||||
<filename>/usr/local/sics/data</filename> of the
|
||||
ics1-<replaceable>australian_fauna</replaceable> computer. This path is
|
||||
configured in <filename>server_config.tcl</filename> by setting the
|
||||
<varname>SicsDataPath</varname> variable. Posix symbolic links are used to link
|
||||
the directory to the appropriate directory on filer.nbi.ansto.gov.au, under the
|
||||
<filename>/experiments</filename> mount point e.g
|
||||
<filename>/experiments/wombat/data/current</filename>. You can mount this
|
||||
directory on the MS Windows machine
|
||||
dav1-<replaceable>australian_fauna</replaceable>. Files in this location are
|
||||
read-only. The facility requires an immutable version of the data. </para>
|
||||
</sect2>
|
||||
</sect1>
|
||||
<sect1>
|
||||
<title>File Commands. Single Files</title>
|
||||
<sect2>
|
||||
<title>newfile command</title>
|
||||
<para>
|
||||
<variablelist>
|
||||
<varlistentry>
|
||||
<term><command>newfile</command>
|
||||
<replaceable>file_type</replaceable>
|
||||
<optional>scratch</optional></term>
|
||||
<listitem>
|
||||
<para>creates a new file of type <replaceable>file_type</replaceable>
|
||||
ready to write to. The command does write any information to disk. </para>
|
||||
<para>To save data, use the <command>save</command> command.</para>
|
||||
<para>You can only hold a reference to one file. If you need to
|
||||
reference a number of files, then use newfile_collection. </para>
|
||||
<para>Only use the optional <optional>scratch</optional> if you want to
|
||||
write data to a scratch file. The file will be overwritten with the
|
||||
next invocation of this option</para>
|
||||
<para><replaceable>file_type</replaceable> may have the following
|
||||
values: </para>
|
||||
<para><option>BEAM_MONITOR</option> Saves data from the configured beam
|
||||
monitors, histogram memory data is not saved.</para>
|
||||
<para><option>HISTOGRAM_T</option> Saves histogram total time data and
|
||||
beam monitor data.</para>
|
||||
<para><option>HISTOGRAM_X</option> Saves histogram x data and beam
|
||||
monitor data.</para>
|
||||
<para><option>HISTOGRAM_XT</option> Saves histogram x,t data and beam
|
||||
monitor data.</para>
|
||||
<para><option>HISTOGRAM_Y</option> Saves histogram y data and beam
|
||||
monitor data.</para>
|
||||
<para><option>HISTOGRAM_YT</option> Saves histogram y,t data and beam
|
||||
monitor data.</para>
|
||||
<para><option>HISTOGRAM_XY</option> Saves histogram x,y data and beam
|
||||
monitor data.</para>
|
||||
<para><option>HISTOGRAM_XYT</option> Saves histogram total x,y,t data
|
||||
and beam monitor data.</para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
</variablelist>
|
||||
</para>
|
||||
</sect2>
|
||||
<sect2>
|
||||
<title>save command</title>
|
||||
<para>
|
||||
<variablelist>
|
||||
<varlistentry>
|
||||
<term><command>save</command>
|
||||
<replaceable>index</replaceable>
|
||||
</term>
|
||||
<listitem>
|
||||
<para>saves data to disk. </para>
|
||||
<para><replaceable>index</replaceable> is the index of data to be saved,
|
||||
starting with 0. To save your first slice of data you would save 0.
|
||||
This provides you with a complete NeXus file. You may be doing After
|
||||
you acquire you next slice of data, you would save 1, then save 2
|
||||
etc. </para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
</variablelist>
|
||||
</para>
|
||||
</sect2>
|
||||
<sect2>
|
||||
<title>Other single file commands</title>
|
||||
<para>
|
||||
<variablelist>
|
||||
<varlistentry>
|
||||
<term><command>killfile</command></term>
|
||||
<listitem>
|
||||
<para>decrements the data number used for SICS file writing and thus
|
||||
consequently overwrites the last datafile. This is useful when
|
||||
useless data files have been created during tests. As this is
|
||||
critical command in normal user operations, this command requires
|
||||
managers privilege. </para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
</variablelist>
|
||||
</para>
|
||||
</sect2>
|
||||
</sect1>
|
||||
<sect1>
|
||||
<title>File Collection Commands</title>
|
||||
<sect2>
|
||||
<title>newfile_collection command</title>
|
||||
<para>
|
||||
<variablelist>
|
||||
<varlistentry>
|
||||
<term><command>newfile_collection</command>
|
||||
<option> -labels </option>
|
||||
<replaceable>{sample1 sample2}</replaceable>
|
||||
<option>-filetype</option>
|
||||
<replaceable>file_type</replaceable>
|
||||
<option>-savetype</option>
|
||||
<replaceable>save_type</replaceable>
|
||||
</term>
|
||||
<listitem>
|
||||
<para>Whereas newfile creates one file, newfile_collection will create
|
||||
as many files as there are labels. The command does write any
|
||||
information to disk. </para>
|
||||
<para>To save data, use the <command>save_collection</command> command</para>
|
||||
<para>Example: You have a multi-sample changer or robot. You want to do
|
||||
a measurement on each sample at multiple temperatures. Your
|
||||
experimental sequence has the sample changer as the fastest varying
|
||||
parameter (inner loop), and temperature change as the slowest
|
||||
varying parameter (outer loop). You want to record all temperature
|
||||
data for a sample in one file.</para>
|
||||
<para><option>-savetype</option>
|
||||
<replaceable>save_type</replaceable> may have the following values: </para>
|
||||
<para>
|
||||
<option>data</option> writes to a normal data file</para>
|
||||
<para><option>scratch</option> writes to a scratch file. The file will
|
||||
be overwritten with the next invocation of this option. Used mainly
|
||||
for testing.</para>
|
||||
<para><option>-filetype</option>
|
||||
<replaceable>file_type</replaceable> may have the following values: </para>
|
||||
<para><option>BEAM_MONITOR</option> Saves data from the configured beam
|
||||
monitors, histogram memory data is not saved.</para>
|
||||
<para><option>HISTOGRAM_T</option> Saves histogram total time data and
|
||||
beam monitor data.</para>
|
||||
<para><option>HISTOGRAM_X</option> Saves histogram x data and beam
|
||||
monitor data.</para>
|
||||
<para><option>HISTOGRAM_XT</option> Saves histogram x,t data and beam
|
||||
monitor data.</para>
|
||||
<para><option>HISTOGRAM_Y</option> Saves histogram y data and beam
|
||||
monitor data.</para>
|
||||
<para><option>HISTOGRAM_YT</option> Saves histogram y,t data and beam
|
||||
monitor data.</para>
|
||||
<para><option>HISTOGRAM_XY</option> Saves histogram x,y data and beam
|
||||
monitor data.</para>
|
||||
<para><option>HISTOGRAM_XYT</option> Saves histogram total x,y,t data
|
||||
and beam monitor data.</para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
</variablelist>
|
||||
</para>
|
||||
</sect2>
|
||||
<sect2>
|
||||
<title>save_collection command</title>
|
||||
<para>
|
||||
<variablelist>
|
||||
<varlistentry>
|
||||
<term><command>save_collection</command>
|
||||
<option>-index</option>
|
||||
<replaceable>val</replaceable>
|
||||
<command>-labels </command>
|
||||
<replaceable>sample1</replaceable></term>
|
||||
<listitem>
|
||||
<para>saves data to disk within a collection (multiple files)</para>
|
||||
<para><command>-index</command><replaceable>val</replaceable> is the
|
||||
index of data to be saved, starting with 0. To save your first slice
|
||||
of data you would save 0. This provides you with a complete NeXus
|
||||
file. You may be doing After you acquire you next slice of data, you
|
||||
would save 1, then save 2 etc.</para>
|
||||
<para><command>-labels </command>
|
||||
<replaceable>sample1</replaceable> will save to the file referenced
|
||||
by the label <replaceable>sample1</replaceable>. You would put all
|
||||
data relating to a sample into this one file. </para>
|
||||
</listitem>
|
||||
</varlistentry>
|
||||
</variablelist>
|
||||
</para>
|
||||
</sect2>
|
||||
</sect1>
|
||||
</chapter>
|
||||
Reference in New Issue
Block a user