linse/sics
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

- Enhanced and debugged histogram memory for AMOR

Browse Source 
* added PROJECT both in HM and driver code
  * added single detector support.
- Removed several bugs in the AMOR data bit.
- Updated documentation
This commit is contained in:
cvs
2001-08-17 14:33:05 +00:00
parent a538361516
commit db6c355f44
56 changed files with 4060 additions and 426 deletions
Download Patch File Download Diff File Expand all files Collapse all files

376
doc/master.tex Normal file
View File

@ -0,0 +1,376 @@
\documentclass[12pt,a4paper]{article}
%%\usepackage[dvips]{graphics}
%%\usepackage{epsf}
\setlength{\textheight}{24cm}
\setlength{\textwidth}{16cm}
\setlength{\headheight}{0cm}
\setlength{\headsep}{0cm}
\setlength{\topmargin}{0cm}
\setlength{\oddsidemargin}{0cm}
\setlength{\evensidemargin}{0cm}
\setlength{\hoffset}{0cm}
\setlength{\marginparwidth}{0cm}
\setlength{\parindent}{0pt}
\setlength{\parskip}{0.5cm}
\newtheorem{error}{Error}
\begin{document}
\begin{center}
{\huge Sinq Systems Overview and Documentation Guide}
\\
Revision: \today \\
Author: Mark K\"onnecke
\end{center}
\tableofcontents
\clearpage
\section{Introduction}
This document gives an overview about all the hardware and software systems
involved in SINQ instrument control. It describes the relationships between
the various pieces and points to additional documentation, if available.
A copy of all documents referred in this paper is stored at: /data/lnslib/distribution/sics/doclib. If they exist in electronic form, this is. The SICS user
and manager information is available on the WWW starting from http://lns00.psi.ch/.
\section{Hardware Overview}
For each SINQ instrument the following computing hardware is available:
\begin{itemize}
\item A dedicated instrument workstation. Most of them are Compaq Alpha stations
running True64Unix. One workstation is still running OpenVMS. Two instruments,
POLDI and RITA--2, are controlled through Intel--PC's running Linux.
\item A TCP/IP terminal server providing access to serial ports.
\item Optionally, there are 1-3 histogram memory computers installed for those
instruments which have area detectors. These histogram memory computers are
Motorolla 6800 VME board processors running the vxWorks realtime
operating system.
\end{itemize}
Most instrument hardware is accessed through RS--232 interfaces and the terminal
server. Histogram memories are accessed through the TCP/IP network. Generally
ethernet is used as the main instrument bus.
In addition to the computers at the instrument the following systems are
available:
\begin{itemize}
\item A True64Unix laboratory server (lnsa15) for data management and
primary data anlalysis.
\item True64Unix PSI server systems for data processing.
\item A WWW--server currently installed on lns00.
\item pss123 is a Sun workstation holding the vxWorks development environment.
\end{itemize}
\section{Software Overview}
\subsection{Software at the Instrument}
The main software component at the instrument is the Sinq Instrument Control
Software (SICS). This is a client server system controlling the operation
of the instrument. A central server running on the instrument workstation
implements all the functionality of instrument control. Clients to this
server implement the user interface to the instrument. The protocoll between
clients and the SICS server is a ASCII command protocoll through either
telnet or plain sockets.
RS--232 devices are not directly accessed through the terminal server but
through a special server program, the SerPortServer program. The reasons
for this are both historical and practical as this system permits access
to a given serial port for several clients which is a valuable aid for
debugging. Clients to this SerPortServer such as the SICS server communicate
with the server through a special ASCII protocoll transported through TCP/IP.
The only documentation
for both the SerPortServer and the ASCII protocoll is the source code.
There exists another support program, the FileSync server, on any instrument
computer. This is a little Java server which listens at a UDP port for a
message from SICS. If such a message is recognized or any two hours, this
server starts a shell script which copies new data files to the laboratory
server. This program is again documented only in its configuration file and
the Java source code.
Then there is the TecsServer. This is a server which maintains and configures
Lakeshore temperature controllers used as standard temperature controllers
at SINQ. The only documentation for this program is Markus Zolliker.
On many instruments there are histogram memory computers. These usually run the
following programs:
\begin{description}
\item[bootUtil] a utility running when the histogram memory is booted which
starts all the other tasks and configures the histogram memory from its
command line parameters.
\item[sinqHMfiller] this task reads data from the electronics and stores it
into the right histogram.
\item[SinqHMserver] This process handles the TCP/IP communication with
clients of the histogram memory. It spawns child processes for doing so.
\end{description}
The histogram memory software, its setup and the TCP/IP protocoll for its
control is described in great detail in the documents titled:
\newline\centerline{Software Support Notes for the SINQ Histogram Memory}
and \newline
\centerline{The Sinq histogram memory, SinqHM}
both authored by David Maden. The sources for these documents are distributed
with the source code for the histogram memory.
The SICS software is described in a variety of documentation:
\begin{itemize}
\item There is user documentation for various instruments.
\item The installation and setup of SICS on an instrument computer is
described in the "SICS Manager Manual".
\item The programming concepts of SICS are discussed in the "SICS Programmers
Reference".
\item An 90\% complete description of SICS source code modules is available
as "SICS Reference Manual".
\end{itemize}
One instrument, TASP, is run with the TASMAD software from ILL which runs
on VMS systems. A user documentation for this system is available at: \newline
\centerline{http://sinq.web.psi.ch/sinq/doc/tasmad.html}
Some configuration issues are explained in this docuement as well. Further
documentation exists only in form of David Maden and an analysis of the
source code.
The RITA--2 instrument from Ris\o{ } runs the TASCOM software.
This software is quite well documented in various documents which can be
obtained at WHGA/247 or directly from the Ris\o{ } team and Per Skarup.
\subsection{Facilities at the Laboratory Server(LNSA15)}
\subsubsection{Central Data Repository}
Under the /data/lnslib/data directory there exists a central storage area for
data files measured at the instruments. Newly measured data files are
automatically mirrored to this area once a measurement has finished.
Not surprisingly there is a subdirectory for each instrument at SINQ.
These instrument directories contain further subdirectories for each year
of SINQ operation which hold the actual data files.
\subsubsection{The SINQ File Database}
Right early on it was decided to separate the file archival function from the
file search facility. For this decision there are two reasons:
\begin{itemize}
\item If more search criteria are required, only the database will need to be
rebuilt.
\item Most SINQ data files are binary. A special tool was needed anyway in
order to extract the searchable information from the data files.
\end{itemize}
The SINQ File database is described in more detail in the document:\newline
\centerline{The SINQ File Database}
which is part of the source distribution for the system. Here only an
overview of the SINQ database system is given.
The SINQ File Database consists of the following components:
\begin{itemize}
\item The database itself. This is the SQL database system mSQL from
Hughes Technology.
\item Any night the unix utility cron starts a shell script which is
responsible for updating the database. This is done with two special
utility programs:
\begin{itemize}
\item nx\_dbupdate scans a given directory for new files which are not
yet in the database. If such a file is found, the data fields for the
database are extracted and entered into the database.
\item The Java program ScanScan does the same job as nx\_dbupdate for
ASCII files.
\end{itemize}
\item A querying system. Curently the only query interface is a WWW--interface
installed at the WWW--server.
\end{itemize}
\subsubsection{The NeXus Data Server}
In order to provide access to SINQ data file to various clients on the
network a NeXus Data Server was installed. This piece of software
is required in order to support the "Load Old File" facility in the
Java status display clients for various instruments.
\subsubsection{The CVS Repository}
CVS is a shortcut for Concurrent Version control System. This is a system for
keeping track of various versions of software. As a lot of source code has
to be maintained at SINQ and we aspire to have several people working at
it such a control system became necessary. The LNS setup is described in
the document:\newline
\centerline{The LNS CVS Repository}
This document also explains how to get hold the source code for most
of the software used at SINQ.
\subsubsection{Backup}
The laboratory server is also the central backup server for SINQ. Backups are
performed with the Legato networker software onto a tape jukebox holding
5 tapes with 20GB capacity each. The following items are backed up through
this system:
\begin{itemize}
\item The laboratory server.
\item The instrument accounts on the instrument computers.
\item The lnsdata and lnslib shares of the lns00 Windows--NT server.
\end{itemize}
This backup system is a protection against a major harddisk failure. It is no
archive system. Though backup tapes are held as long as possible it
cannot be guaranteed that files older then half a year can be recovered.
The backup software is described in the documentation coming with the
system. No further documentation exists, but the setup can be viewed
through the nwadmin application.
\subsubsection{Motor Parameter Backup}
At SINQ the PSI EL--734 motor controllers are used. These motor controllers
hold a set of parameters for each motor. As a safeguard against instrument
scientists gone wild or a loss of parameters due some electronic or
electrical incident these motor parameters are saved weekly. This happens
on wednesdays mornings. The unix utility crom triggers the process and
starts the script savemot which in turn does all necessary things. The
actual saving of the motor parameters is accomplished with David Maden's
el734 program. The saved parameters end up in the /data/lnslib/motors
hierarchy. There exists a directory for each backup date which in turn
holds a subdirectory for each instrument which holds the files with
the saved parameters. All necessary shell scripts and programs can be
found in the /data/lnslib/motors/bin directory.
\subsubsection{License Server}
Unfortunately some people use the proprietary IDL programming system. This
system requires a license server to run on some computer. This license
server is also installed together with the actual IDL distribution on the
laboratory server. LNS has bought a number of network licenses of IDL.
This means at any time only a certain number of people can use IDL on the
network. When IDL starts up it requests a license from the license server
and gives it back when it finishes. This only works if the license server
still has enough spare licenses. This implies that any IDL user should
close his program when she is done and not leave it running all weekend.
The installation of the license server is described in the IDL installation
instructions. License servers are black magic and should be handled by
wizards only. There are only two things which can be done easily:
\begin{enumerate}
\item Check if it is running. the command: \begin{verbatim}
ps -A | grep lmgrd
\end{verbatim} should yield at least two entries.
yield at least two entries.
\item Start it if it is not running:
\begin{enumerate}
\item Become root
\item Run: /usr/opt/idl/idl/bin/idlstart
\end{enumerate}
\end{enumerate}
\subsection{Software at the WWW--server}
The WWW--server running at lns00 does not provide static information only but also
active content calculated at runtime. The following services are
implemented:
\begin{itemize}
\item Keyword search in the SICS documentation.
\item Searching the SINQ file database for files matching various criteria.
\item Display of the current instrument status for a couple of instruments.
\item An experimental WWW control system for powder diffractometers.
\end{itemize}
Most of these services are provided through java servlets. In order to do this
a servlet engine was needed. The combination Apache WWW-server and Apache
JServ was choosen. However, it is planned to exchange the latter component
by the Jakarta engine in the near future.
The database search servlets are described in more detail in the document:
\newline \centerline{The SINQ File Database}
the status display and control servlets in the document:\newline
\centerline{The SICS WWW--servlet System}
. Both documents are part of their respective source code packages.
The SICS documentation searching system is built on top of the program SWISH-E which is available
from http://sunsite.berkeley.edu/SWISH-E.
The actual search is performed through a TCL--cgi script which calls the swishe
application with appropriate parameters. This script can be found as
swishsearch.cgi together with a batch file to start it properly
(swishsearch.bat) in the cgi-bin directory of the WWW hierarchy on lns00.
Prerequisite for such a search is the existence of an index file. This must
be created offline after any major change to the documentation. In order to
create this index, cd into the sics directory of the WWW hierarchy and run:
\begin{verbatim}
d:\bin\swishe sics.config
\end{verbatim}
This must be done on the lns00 computer. The placement of the index file,
swishe executable and the root for serving documents are configured in
sics/sicssearch.html and swishsearch.cgi. For more information about
swish-e see the homepage mentioned.
\section{Access Patterns}
In order to understand the system better it is useful to look at the flow
of information and the collaboration of the individual components in more
detail.
\subsection{Data Files}
\begin{enumerate}
\item Data Files are generated by the instrument control programs at the
instrument computer.
\item Data Files are automatically mirrored to the central repository
on the laboratory server lnsa15. This happens through the FileSync server,
a shell script and last not least the unix rsync utility. All this is installed
at the instrument computer.
\item Nightly new files are scanned for relevant information and their
characteristics are entered into the database system on lnsa15.
\end{enumerate}
\subsection{SICS servers}
SICS instrument control servers usually communicate with the terminal servers
through the SerPortServer program, with temperature controllers through the
TecsServer and directly via TCP/IP with the histogram memory computers.
\subsection{SICS Java Clients}
The SICS Java clients access the SICS servers on the selected instruments
workstation. Most Java clients can display data from previously measured
files. Such data is accessed through the NeXus Data Server running on the
laboratory server lnsa15.
\subsection{WWW Services}
The file search service on the WWW--server lns00 accesses the database
server running on the laboratory server lnsa15.
The instrument status display servlets on the WWW--server lns00 access the
SICS instrument control servers on the appropriate instrument computers
for data.
\section{Maintainance Tasks}
\subsection{Yearly Maintainance}
The only maintainance task at SINQ shutdown is to disable the cron job on
lnsa15 which performs the motor parameter backup.
Whith each new year a couple of things must be done in order to keep the
system ship shape:
\begin{enumerate}
\item On each instrument computer:
\begin{itemize}
\item Create a new subdirectory for data files for the new year.
\item In the new subdirectory create a DataNumber file with the data file
number set to 0.
\item Edit the path names in the data file mirroring script to point to the
new years subdirectory. This script is usually called {\tt instsync} with
inst replaced by the name of the instrument.
\item Edit the instrument configuration file and adapt the path names for
the data directory and the DataNumber file for the new year.
\end{itemize}
\item On the laboratory server lnsa15:
\begin{itemize}
\item Reenable the motor parameter backup cron job.
\item Create new subdirectories for the new year in the data hierarchy.
\item In /data/lnslib/lib/nxdb create new configuration files for the new
year. Edit them to point to the new years subdirectory. Edit the
updatenxdb shell script to use the newly created configuration files.
Keep the old ones as they may come in handy if the whole database may
require an update.
\end{itemize}
\item On the WWW--server lns00:
\begin{itemize}
\item Add the new year into the html--pages for the data file search.
\item Check the configuration file for the status servlets, sics.conf,
for necessary updates caused by computer changes.
\end{itemize}
\end{enumerate}
Do these things quite early on in the year. People start measuring background
quite early!
\subsection{Exchange of Instrument Computers}
In order to keep up with ageing 1-2 computers have to be exchanged any year.
A instrument computer change requires the following adaptions:
\begin{enumerate}
\item Edit the instrument configuration file and change all occurences of
the old computers name to the new name.
\item On lnsa15 as user lnslib: enter the new computer name into the
.rhosts file. This is required for the mirroring to work.
\item For the Java clients edit lnet/SINQConfig.java to point to the
new computer and rebuild all clients. Redistribute them as well.
\item For the WWW status, adapt the sics.conf file on lns00 and restart
the WWW-server.
\end{enumerate}
\end{document}