sics/doc/programmer/overview.tex

\chapter{SICS Overview}
\section{Introduction}
At the new spallation source SINQ at PSI a whole set of new neutron
scattering instruments are being installed. All these new instruments need a
computer instrument control control system. After a review of similar systems
out in the market it was found that none fully met the requirements defined
for SINQ or could easily be extended to do so. Therefore it was decided  to
design a new system. This new system SICS, the SINQ Instrument Control
System, had to meet the following specifications:
\begin{itemize}
\item Control the instrument reliably.
\item Good remote access to the instrument via the internet.
\item Portability across operating system platforms.
\item Enhanced portability across instrument hardware. This means that it
should be easy to add other types of motors, counters or other hardware to
the system. 
\item Support authorization on the command and parameter modification
level. This means
that certain instrument settings can be protected against random changes by
less knowledgable users.
\item Good maintainability and extendability.  
\item Be capable to acomodate graphical user interfaces (GUI). 
\item One code base for all instruments.
\item Powerful macro language.
\end{itemize}
A suitable new system was implemented using an object oriented design which
matches the above criteria.

\section{The SINQ Hardware Setup}
SICS had to take in account the SINQ hardware setup which had been decided
upon earlier on. Most hardware such as motors and counters is controlled via
RS--232 interfaces. These RS--232 interfaces are connected to a
terminal server which allows to access such devices through the TCP/IP
network. 

For historical reasons the instrument control software does not access
the terminal server directly but through another software layer, the
SerPortServer program. The SerPortServer program is another TCP/IP
server which allows multiple network clients to access the same
terminal server port through a home grown protocoll. In the long run
this additional software layer will be abolished.  

Some hardware devices, such as the histogram memory, can handle
TCP/IP themselves. With such devices the instrument control program
communicates directly through TCP/IP. All
hardware devices take care of their real time needs themselves. Thus the
only task of the instrument control program is to orchestrate the hardware
devices. SICS is designed with this setup up in mind, but is not restricted
to this setup. A schematic view of this setup is given in picture
\ref{hard}.
 \begin{figure}
%%  \epsfxsize=0.65\textwidth
  \epsfxsize=160mm
  \epsffile{hart.eps}
 \caption{Instrument control hardware setup at SINQ}\label{hard}
 \end{figure}
  

\section{SICS Overall Design}
In order to achieve the design goals stated above it was decided to divide
the system into a client server system. This means that there are at least
two programs necessary to run an instrument: a client program and a server
program. The server program, the SICS server, does all the work and
implements the actual instrument control. The SICS server usually runs on
the DAQ computer. The client program may run on any computer on the world
and implements the user interface to the instrument. Any numbers of clients
can communicate with one SICS server. The SICS server and the clients
communicate via a simple ASCII command protocol through TCP/IP sockets.
With this design good remote control through the network is easily achieved.
As clients can be implemented in any language or system capable of handling
TCP/IP the user interface and the functional aspect are well separated. This
allows for easy exchange of user interfaces by writing new clients.     
 

\section{SICS Clients}
SICS Clients implement the SICS user interface. Current client
programs are mostly implemented in Java for platform independence.
This is a real concern at SINQ where VMS,
Intel-PC, Macintosh and Unix users have to be satisfied.   
As many instrument scientists still prefer
the command line for interacting with instruments, the most used client is a
visual command line client. Status displays are another kind  of specialized 
client programs. Graphical user interfaces are under consideration for some
instruments. As an example for a client a screen shot of the status display
client for a powder diffractometer is given in picture \ref{dmc}
\ref{hard}.
 \begin{figure}
%%  \epsfxsize=0.65\textwidth
  \epsfxsize=160mm
\epsffile{dmccom.eps}
 \caption{Example for a SICS client: Powder Diffractometer Status Display}\label{dmc}
 \end{figure}
  

 
\section{The SICS Server}
The SICS server is the core component of the SICS system. The SICS server is
responsible for doing all the work in instrument control. Additionally the
server has to answer the requests of possibly multiple clients. 
The SICS server can be subdivided into three subsystems:
\begin{description}
\item[The kernel] The SICS server kernel
takes care of client multitasking and the preservation of the proper
I/O and error context for each client command executing.
\item[SICS Object Database] SICS objects are software modules which
represent all aspects  of an instrument: hardware devices, commands, measurement strategies  
and data storage.  This database of objects is initialized at server startup 
time from an initialization script. 
\item[The Interpreter] The interpreter allows to issue commands to the
objects in the objects database. 
\end{description}
The schematic drawing of the SICS server's structure is given in picture
\ref{newsics}.
 \begin{figure}
%%  \epsfxsize=0.65\textwidth
  \epsfxsize=160mm
  \epsffile{newsics.eps}
 \caption{Schematic Representation of the SICS server's structure}\label{sicsnew}
 \end{figure}
  

\subsection{The SICS Server Kernel}
In more detail the SICS server kernel has the following tasks: 
\begin{itemize}
\item Accept and verify client connection requests.
\item Read and execute client commands.
\item Maintain the I/O and error context for each client connection.
\item Serialize data access.
\item Serialize hardware access.
\item Monitor HW--operations.
\item Monitor environment devices. 
\end{itemize}
Any program serving multiple clients has the problem how to organize multiple
clients accessing the same server and how to prevent one client from
reading data,  while another client is writing. The approach used for
the SICS server is a combination of polling and cooperative multitasking. This scheme is
simple and can be implemented in an operating system independent manner. One
way to look at the SICS server is as a series of tasks in a circular queue
executing one after another. The servers main loop does nothing but
executing the tasks in this circular buffer in an endless loop. 
There are several system tasks and one such
task for each living client connection. Thus only one task executes at any
given time and data access is efficiently serialized. 

One of the main system
tasks (and the one which will be always there) is the network reader. The
network reader has a list of open network connections and checks each of
them for pending requests. What happens when  data is pending on an open
network port depends on the type of port: If it is the servers main
connection port, the network reader will try to accept and verify a new
client connection and create the associated data structures. If the port
belongs to an open client connection the network reader will read the
command pending and put it onto a command stack existing for each client
connection.  When it is time for a client task to execute, it will fetch a
command from its very own command stack and execute it.  
This is how the SICS server deals with client requests.

The scheme described above relies on the fact that most SICS command need
only very little time to execute. A command needing time extensive 
calculations may effectively block the server. Implementations of such
commands have to take care that control passes back to the task switching
loop at regular intervalls in order to prevent the server from blocking.   

Another problem in a server handling multiple client requests is how to
maintain the proper execution context for each client. This includes the
clients I/O-context (socket), the authorisation of the client and possible 
error conditions pending for a
client connection. SICS does this via a connection object, a special
data structure holding all the above information plus a set of functions
operating on this data structure. This connection object is passed along
with many calls throughout the whole system. 

Multiple clients issuing  commands to the SICS server may mean that multiple
clients might try to move motors or access other hardware in conflicting
ways. As there is only one set of  instrument hardware this needs to be
prevented. This is achieved by a convention. No SICS object drives hardware
directly but registers it's request with a special object, the device
executor.  This device executor starts the requested operation and reserves
the hardware for the length of the operation. During the execution of such
an hardware request all other clients requests to drive the hardware will
return an error. The device executor is also responsible for monitoring the
progress of an hardware operation. It does so by adding a special task into
the system which checks the status of the operation each time this tasks
executes.  When the hardware operation is finished this
device executor task will end.  A special system facility allows a client
task to wait for the device executor task to end while the rest of the task
queue is still executing. In this way time intensive hardware operations can
be performed by drive, count or scan commands without blocking the whole
system for other clients. \label{devexec}

The SICS server can be configured to support another security feature, the
token system. In this scheme a client can grab control of the instrument.
With the control token grabbed, only the client which has the token may
control the instrument. Any other client may look at things in the SICS server
but does not have permission to change anything. Passing the control token
requires that the client which has the token releases the token so that
another client may grab it. There exists a password protected back door for
SICS managers which allows to force the release of a control token. 

Most experiments do not happen at ambient room conditions but
require some special environment for the sample. Mostly this is temperature
but it can also be magnetic of electric  fields etc. Most of such devices
can regulate themselves but the data acquisition program needs to monitor
such devices. Within SICS, this is done via a special system object, the
environment monitor. A environment device, for example a temperature
controller, registers it's presence with this object. Then a special system
task will control this device when it is executing, check for possible out
of range errors and  initiates the proper error handling if such a problem is
encountered.

\subsection{The SICS Interpreter}
When a task belonging to a client connection executes a command it will pass
the command along with the connection object to the SICS interpreter. The
SICS interpreter will then analyze the command and forward it to the
appropriate SICS object in the object database for further action. The SICS
interpreter is very much modeled after the Tcl interpreter as devised by
John Ousterhout$^{1}$. For each SICS object visible from the interpreter there is
a wrapper function. Using the first word of the command as a key, the
interpreter will locate the objects wrapper function. If such a function is
found it is passed the command parameters, the interpreter object and the
connection object for further processing. An interface exists to add and
remove commands to this interpreter very easily. Thus the actual command
list can be configured easily to match the instrument in question, sometimes
even at run time. Given the closeness of the design of the SICS interpreter
to the Tcl interpreter, the reader may not be surprised to learn that the
SICS server incorporates Tcl as its internal macro language. The internal
macro language may use Tcl commands as well as SICS commands.

\subsection{SICS Objects}
As already said, SICS objects implement the true functionality of SICS
instrument control. All hardware, all commands and procedures, all data
handling strategies are implemented as SICS objects.  Hardware objects, for
instance motors deserve some special attention. Such objects are divided
into two objects in the SICS system: A logical hardware object and a driver
object. The logical object is responsible for implementing all the nuts and
bolts of the hardware device, whereas the driver defines a set of primitive
operations on the device. The benefit of this scheme is twofold: 
switching to new hardware, for instance a new type of motor, just requires 
to incorporate a new driver into the system. Internally, independent from
the actual hardware, all hardware object of the same type (for example
motors) look the same and can be treated the same by higher level objects.   
No need to rewrite  a scan command because a motor changed. 

In order to live happily within the SICS system SICS object have to adhere
to a system of protocols. There are protocols for:
\begin{itemize}
\item Input/Output to the client.
\item Error handling.
\item Interaction with the interpreter.
\item For identification of the object to the system at run time.
\item For interacting with hardware (see device executor above).
\item For checking the authorisation of the client who wants to execute the
command.
\end{itemize}
SICS objects have the ability to notify clients and other objects of
internal state changes. For example when a motor is driven, the motor object
can be configured to tell SICS clients or other SICS objects about his new
position. 

SICS uses NeXus$^{2}$, the upcoming standard for data exchange for neutron
and x--ray scattering as its raw data format.


\section{SICS Working Examples}
In order to get a better feeling for the internal working of SICS the course
of a few different requests through the SICS system is  traced in this
section. The examples traced will be:
\begin{itemize}
\item A request for a new client connection.
\item A simple command.
\item A command to drive a motor in blocking mode.
\item A command to drive a motor which got interrupted by the user.
\item A command to drive a motor in non blocking mode.
\end{itemize}
For the whole discussion it is assumed that the main loop is running,
executing cyclically each single task registered in the server. Task switching is
done by a special system component, the task switcher. 

\subsection{The Request for a new Client Connection}
\begin{itemize}
\item The network reader recognizes pending data on its main server port.
\item The network reader accepts the connection and tries to read an
username/password pair.
\item If such an username/password pair comes within a suitable time
interval it is checked for validity. On failure the connection is closed
again.
\item If a valid connection has been found: A new connection object is
created, a new task for this client connection is introduced into the
system and the network reader registers a new client port to check for
pending commands.
\item Control is passed back to the task switcher.
\end{itemize}

\subsection{A Simple Command}
\begin{itemize}
\item The network reader finds data pending at one of the client ports.
\item The network reader reads the command, splits it into single lines and
put those on top of the client connections command stack. The network
reader passes control to the task switcher.
\item In due time the client connection task executes, inspects its command
stack, pops the command pending and forwards it together with a pointer to
itself to the SICS interpreter.
\item The SICS interpreter inspects the first word of the command. Using
this key the interpreter finds the objects wrapper function and passes
control to that function. 
\item The object wrapper function will check further arguments, checks the
clients authorisation if appropriate for the action requested. Depending on
the checks, the wrapper function will create an error message or do its
work.
\item This done, control passes back through the interpreter and the connection
task to the task switcher.
\item The next task executes.
\end{itemize}

\subsection{A ``Drive'' Command in Blocking Mode}
\begin{itemize}
\item The network reader finds data pending at one of the client ports.
\item The network reader reads the command, splits it into single lines and
put those on the top of the client connections command stack. The network
reader passes control to the task switcher.
\item In due time the client connection task executes, inspects its command
stack, pops the command pending and forwards it together with a pointer to
itself to the SICS interpreter.
\item The SICS interpreter inspects the first word of the command. Using
this key the interpreter finds the drive command wrapper function and passes
control to that function. 
\item The drive command wrapper function will check further arguments, 
checks the
clients authorisation if appropriate for the action requested. Depending on
the checks, the wrapper function will create an error message or do its
work.
\item Assuming everything is OK, the motor is located in the system.
\item The drive command wrapper function asks the device executor to run the
motor.
\item The device executor verifies that nobody else is driving, then starts
the motor and grabs hardware control. The device executor also starts a task
monitoring the activity of the motor.
\item The drive command wrapper function now enters a wait state. This means
the task switcher will execute other tasks, except the connection task
requesting the wait state. The client connection and task executing the drive command
 will not be able to process further commands.    
\item The device executor task will keep on monitoring the progress of the motor
driving whenever the task switcher  allows it to execute.
\item In due time the device executor task will find that the motor finished
driving. The task will then finish executing. The clients grab of the
hardware driving permission will be released.
\item At this stage the drive command wrapper function will awake and
continue execution. This means inspecting errors and reporting to the client
how things worked out.
\item This done, control passes back through the interpreter and the connection
task to the task switcher. The client connection is free to execute 
other commands.
\item The next task executes.
\end{itemize}

\subsection{A ``Drive Command Interrupted}
\begin{itemize}
\item The network reader finds data pending at one of the client ports.
\item The network reader reads the command, splits it into single lines and
put those on the top of the client connections command stack. The network
reader passes control to the task switcher.
\item In due time the client connection task executes, inspects its command
stack, pops the command pending and forwards it together with a pointer to
itself to the SICS interpreter.
\item The SICS interpreter inspects the first word of the command. Using
this key the interpreter finds the drive command wrapper function and passes
control to that function. 
\item The drive command wrapper function will check further arguments, 
checks the
clients authorisation if appropriate for the action requested. Depending on
the checks, the wrapper function will create an error message or do its
work.
\item Assuming everything is OK, the motor is located in the system.
\item The drive command wrapper function asks the device executor to run the
motor.
\item The device executor verifies that nobody else is driving, then starts
the motor and grabs hardware control. The device executor also starts a task
monitoring the activity of the motor.
\item The drive command wrapper function now enters a wait state. This means
the task switcher will execute other tasks, except the connection task
requesting the wait state.   
\item The device executor task will keep on monitoring the progress of the
driving of the motor when it is its turn to execute.
\item The network reader finds a user interrupt pending. The interrupt will
be forwarded to all tasks in the system.
\item In due time the device executor task will try to check on the progress
of the motor. It will recognize the interrupt. If appropriate the motor will
get a halt command.   The task will then die. 
The clients grab of the hardware driving permission will be released.
\item At this stage the drive command wrapper function will awake and
continue execution. This means it finds the interrupt, tells the user what he
already knows: an interrupt was issued.
\item This done, control passes back through drive command wrapper, 
the interpreter and the connection
task to the task switcher.
\item The next task executes.
\end{itemize}

\subsection{A ``Run'' Command in Non Blocking Mode}
\begin{itemize}
\item The network reader finds data pending at one of the client ports.
\item The network reader reads the command, splits it into single lines and
put those on the top of the client connections command stack. The network
reader passes control to the task switcher.
\item In due time the client connection task executes, inspects its command
stack, pops the command pending and forwards it together with a pointer to
itself to the SICS interpreter.
\item The SICS interpreter inspects the first word of the command. Using
this key the interpreter finds the drive command wrapper function and passes
control to that function. 
\item The ``run'' command wrapper function will check further arguments, 
checks the
clients authorisation if appropriate for the action requested. Depending on
the checks, the wrapper function will create an error message or do its
work.
\item Assuming everything is OK, the motor is located in the system.
\item The ``run'' command wrapper function asks the device executor to run the
motor.
\item The device executor verifies that nobody else is driving, then starts
the motor and grabs hardware control. The device executor also starts a task
monitoring the activity of the motor.
\item The run  command wrapper function passes control through the interpreter and
the clients task function back to the task switcher. The client connection can handle 
new commands.   
\item The device executor task will keep on monitoring the progress of the motor
driving whenever the task switcher  allows it to execute.
\item In due time the device executor task will find that the motor finished
driving. The task will then die silently. The clients grab of the hardware driving
permission will be released.  Any errors however, will be reported.
\end{itemize}

All this seems to be pretty complex and time consuming. But it is the complexity needed to 
do so many things, especially the non blocking mode of operation requested
by users. Tests have shown that the task switcher manages +900 cycles
per second through the task list on a DigitalUnix machine and 500
cycles per second on a pentium 2GHZ machine running linux. Both data
were obtained with software simulation of  hardware devices. With real
SINQ hardware these numbers drop to as low as 4 cycles per second if
the hardware is slow in responding. This shows
clearly that the communication with the hardware is the systems
bottleneck and not the task switching scheme.