476 lines
21 KiB
HTML
476 lines
21 KiB
HTML
/*************************************************************************\
|
|
* Copyright (c) 2002 The University of Chicago, as Operator of Argonne
|
|
* National Laboratory.
|
|
* Copyright (c) 2002 The Regents of the University of California, as
|
|
* Operator of Los Alamos National Laboratory.
|
|
* EPICS BASE Versions 3.13.7
|
|
* and higher are distributed subject to a Software License Agreement found
|
|
* in file LICENSE that is included with this distribution.
|
|
\*************************************************************************/
|
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
|
|
<HTML>
|
|
<HEAD>
|
|
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
|
|
<META NAME="GENERATOR" CONTENT="Mozilla/4.06 [en] (X11; U; SunOS 5.6 sun4u) [Netscape]">
|
|
<TITLE>Synchronized Time Stamp Support</TITLE>
|
|
</HEAD>
|
|
<BODY>
|
|
|
|
<H1>
|
|
Synchronized Time Stamp Support</H1>
|
|
Author: Jim Kowalkowski
|
|
<H2>
|
|
MODIFICATION: 04AUG1998 (Marty Kraimer)</H2>
|
|
The routine TSsetClockFromUnix was made global. Calling it forces a resynchronization
|
|
with a host time server. This is useful to force a master timing ioc to
|
|
resynchronize.
|
|
<H2>
|
|
Introduction</H2>
|
|
|
|
<H3>
|
|
Purpose</H3>
|
|
New software has been added to IOC core to maintain time stamps. The new
|
|
software has the ability to maintain time stamps over all IOCs on a network.
|
|
The purpose of this paper is to explain how EPICS will synchronize the
|
|
time stamps. In addition, this paper will explain how to configure and
|
|
use the new EPICS time stamp support software.
|
|
<H3>
|
|
Definitions</H3>
|
|
Time Stamp: Two long words representing the time in seconds/nanoseconds
|
|
past an epoch. The first long word represents the seconds past the epoch,
|
|
the second long word represents the nanoseconds within the second. The
|
|
EPICS epoch is currently January 1, 1990. A commonly used Unix epoch in
|
|
January 1,1900 or in vxWorks case, January 1,1970.
|
|
<P>Event System: A hardware based subsystem for delivering events to all
|
|
IOCs on a network. Typical events are the "tick" event and the "reset counter"
|
|
event.
|
|
<P>Event System Synchronized Time: Time stamps maintained using an event
|
|
system. The clock pulses which increment the time stamps are provided by
|
|
a single source, all IOCs increment time stamps using the single source
|
|
pulse.
|
|
<P>Soft Time: Each IOC increments the time stamps using it's own internal
|
|
clock.
|
|
<P>Master Timing IOC: The IOC which knows the actual time and is the source
|
|
of the actual time to all those who inquire.
|
|
<P>Slave IOC: An IOC which relies on a master IOC to provide the actual
|
|
time. This IOC will keep it's time stamp in sync with a master.
|
|
<P>Soft Slave: An IOC which uses Soft Time and synchronizes it's time stamp
|
|
with a master.
|
|
<P>Event Slave: An IOC which uses Event System Synchronized Time to maintain
|
|
it's time stamp. This type of IOC uses the master to verify that it's time
|
|
stamp is correct.
|
|
<P>Soft Master: A Master Timing IOC that maintains it's time stamp using
|
|
a private clock.
|
|
<P>Event Master: An IOC which uses Event System Synchronized Time and is
|
|
a Master Timing IOC. In addition, this IOC is the source of the clock pulses.
|
|
<H3>
|
|
Overview</H3>
|
|
Time stamps are maintained across IOCs using UDP with a master/slave relationship.
|
|
A master timing IOC is responsible for knowing the actual time. A slave
|
|
IOC uses the master to verify it's own time stamps. Time stamps are maintained
|
|
in two fashions: using an event system or using IOC tick counters. Both
|
|
differ radically and require some explanation.
|
|
<H4>
|
|
Event System Support</H4>
|
|
The time stamp software has special requirements in order to provide event
|
|
system synchronized time. The time stamp software assumes that hardware
|
|
is present which has finite counters for counting pulses or ticks delivered
|
|
to it from some master tick generator. The software further assumes that
|
|
every so other, before the finite counters overflow, a signal or event
|
|
will occur to reset the counters to zero. The software requires access
|
|
to these counters at any time and notification when the "counter reset"
|
|
arrives. The combination of hardware tick counters and reset event notification
|
|
gives the software the ability to maintain time. If the event system supports
|
|
other events, it is required that the software be notified of the occurrence
|
|
of each event. The event time will be recorded in a table which holds one
|
|
time stamp for each possible event so that a user can inquire as to when
|
|
the event occurred.
|
|
<H4>
|
|
Soft Time Support</H4>
|
|
If no event system is present, the time stamp software can operate in a
|
|
software timing mode. In this mode, no events are available, only the current
|
|
time can be retrieved. An IOC using soft time will increment a time stamp
|
|
using the system clock (usually updated at 60Hz). The master/slave relationship
|
|
allows a slave to periodically ask a master for the correct time. At this
|
|
point the slave's time stamp can be adjusted to match the masters.
|
|
<H4>
|
|
Role of a Master Timing IOC</H4>
|
|
A master timing IOC has difference responsibilities depending on if it
|
|
is an event master or a soft master. An event master will be an event generator
|
|
(create "tick" and "counter reset" events). When the event master detects
|
|
a "counter reset", it broadcasts the time the event occurred to all event
|
|
slaves on the network. The slaves can use the time stamp data to verify
|
|
their own time. A soft timing master does not have any events, so no broadcasting
|
|
is done. Both soft and event masters have the ability to be queried at
|
|
any time by slaves for the current time.
|
|
<H4>
|
|
Role of a Slave Timing IOC</H4>
|
|
A slave timing IOC also has difference responsibilities depending on if
|
|
it is an event slave or a soft slave. An event slave contains hardware
|
|
to manage "tick" and "counter reset" events. The time stamp support software
|
|
uses this information to maintain time. An event slave will listen for
|
|
broadcasts from the event master timing IOC and use the information to
|
|
verify it's time. A soft slave periodically queries a master timing IOC
|
|
to get the current time.
|
|
<H3>
|
|
Design Specifications Summary</H3>
|
|
|
|
<H4>
|
|
Event System Synchronized Time</H4>
|
|
All IOCs will have identical event times. In addition, all IOCs will maintain
|
|
the same current time.
|
|
<P>The minimum events which must be supported is two. One must be used
|
|
as a "tick" event and one must be used as a "reset hardware tick counter"
|
|
event. The second will be used to update the time stamp representing the
|
|
current time.
|
|
<P>An optional event can be used as a "heartbeat" event. This event can
|
|
be used to signal errors.
|
|
<H4>
|
|
Soft Time</H4>
|
|
All IOCs of this type will maintain time stamps which are within two clocks
|
|
ticks or 1/30th of a second of a master. The master may be a designated
|
|
IOC or the Unix boot server. A Unix boot server master or the server pointed
|
|
to by the EPICS environment variable EPICS_TS_NTP_INET must have NTP available
|
|
for polling.
|
|
<H2>
|
|
User Level Interface</H2>
|
|
|
|
<H3>
|
|
General Use Functions</H3>
|
|
Three functions exist in the synchronous time stamp support software for
|
|
the user to retrieve time stamps:
|
|
<UL>
|
|
<LI>
|
|
TSgetTimeStamp(event_number, struct timespec* time_stamp)</LI>
|
|
|
|
<LI>
|
|
TScurrentTimeStamp(struct timespec* time_stamp)</LI>
|
|
|
|
<LI>
|
|
TSgetFirstOfYearVx(struct timespec* time_stamp)</LI>
|
|
|
|
<LI>
|
|
tsLocalTime(TS_STAMP*)</LI>
|
|
</UL>
|
|
To retrieve the time stamp which represents the time that an event occurred,
|
|
use <B>TSgetTimeStamp()</B>. <B>TScurrentTimeStamp()</B> can be used to
|
|
retrieve the time stamp which represents the time of day. <B>TsLocalTime()</B>
|
|
is the function for returning the current time stamp in the old time stamp
|
|
driver, the routine can still be used to retrieve this information. The
|
|
function <B>TSgetFirstOfYear()</B> attempts to give the caller a time stamp
|
|
representing January 1 of the current year relative to the vxWorks epoch
|
|
1970.
|
|
<P>An EPICS enironment variable named EPICS_TS_NTP_INET exists which can
|
|
be set to point to an NTP server. The default NTP server is the IOC boot
|
|
server.
|
|
<H4>
|
|
Test Functions</H4>
|
|
The following functions can be run from the vxWorks shell to give information
|
|
about the time stamp driver.
|
|
<UL>
|
|
<LI>
|
|
<B>TSreport()</B>: Display all the information about the current configuration
|
|
and status of this time stamp driver.</LI>
|
|
|
|
<LI>
|
|
<B>TSprintRealTime()</B>: Call the ErGetTime() function described below
|
|
(if present) and print the time stamp returned from it. Also print the
|
|
current EPICS time stamp.</LI>
|
|
|
|
<LI>
|
|
<B>TSprintTimeStamp(event_number)</B>: Print the EPICS time stamp for the
|
|
given event_number.</LI>
|
|
|
|
<LI>
|
|
<B>TSprintCurrentTime()</B>: Print the EPICS time stamp representing the
|
|
current time.</LI>
|
|
|
|
<LI>
|
|
<B>TSprintUnixTime()</B>: Send a time stamp query transaction to the boot
|
|
server or NTP server and print the time stamp returned.</LI>
|
|
|
|
<LI>
|
|
<B>TSprintMasterTime()</B>: Send a time stamp query transaction to the
|
|
master time server and print the time stamp returned.</LI>
|
|
</UL>
|
|
|
|
<H4>
|
|
Debugging Information</H4>
|
|
A global variable exists named <B>TSdriverDebug</B>. Setting this variable
|
|
to a positive value in the vxWorks start up script will inform the time
|
|
stamp driver to print information about what it is doing. The greater the
|
|
value, the more information the driver will print. The number can be set
|
|
and adjusted to any value any time while the IOC is running.
|
|
<H3>
|
|
Record Support</H3>
|
|
Record support will have the ability to tie the record processing time
|
|
to an event. This means that a user can specify that processing of a record
|
|
is due to an event (from the event system). When the record gets processed,
|
|
the time in the TIME field of the record will be the time when the event
|
|
occurred. In order to support the event times from record support, two
|
|
new fields will be added to dbCommon. The fields will be Time Stamp Event
|
|
(TSE), and Time Stamp Event Link (TSEL). The TSE field will be the actual
|
|
event time the user is interested in. The TSEL field will be a link used
|
|
to populate the TSE.
|
|
<P>To facilitate the use of the time stamp support software, a new record
|
|
support function will be added:
|
|
<UL>
|
|
<LI>
|
|
recGblGetTimeStamp((dbCommon*)prec).</LI>
|
|
</UL>
|
|
This routine uses TSgetTimeStamp(prec->tse, &prec->time) to set the
|
|
processing time of the record. The new recGblGetTimeStamp() will replace
|
|
the existing call to tsLocalTime(). If the TSE field is zero (the default),
|
|
then TSgetTimeStamp() will report the current time. It is important to
|
|
remember that if a TSE field is set, then the processing time (in field
|
|
TIME) will always reflect the last time the event occurred.
|
|
<H2>
|
|
Driver Configuration</H2>
|
|
The synchronous time stamp support software is configured by calling TSconfigure()
|
|
from the "startup.cmd" file. The parameters to this routine are:
|
|
<UL>
|
|
<LI>
|
|
<B>master_indicator</B>: 1=master timing IOC, 0=slave timing IOC, default
|
|
is slave.</LI>
|
|
|
|
<LI>
|
|
<B>sync_rate_seconds</B>: The clock sync rate in seconds. This rate tells
|
|
how often the synchronous time stamp support software will confirm that
|
|
an IOC clock is synchronized. The default is 10 seconds.</LI>
|
|
|
|
<LI>
|
|
<B>clock_rate_hz</B>: The frequency in hertz of the clock, the default
|
|
is 1000Hz for the event system. The value will be set to the IOC's internal
|
|
clock rate when soft timing is used.</LI>
|
|
|
|
<LI>
|
|
<B>master_port</B>: The UDP port which a master timing IOC will use to
|
|
receive time stamp requests. The default is 18233.</LI>
|
|
|
|
<LI>
|
|
<B>slave_port</B>: The UDP port which a slave will use to receive time
|
|
stamp information from a master.</LI>
|
|
|
|
<LI>
|
|
<B>time_out</B>: UDP information request time out in milliseconds, if zero
|
|
is entered here, the default will be used which is 250ms.</LI>
|
|
|
|
<LI>
|
|
<B>type</B>: 0=normal operation, 1=force soft timing type.</LI>
|
|
</UL>
|
|
This routine must be run before iocInit(). The synchronous time stamp support
|
|
software is initialized as part of iocInit. Running <B>TSreport()</B> after
|
|
iocInit() will produce a report which shows the current state of the driver.
|
|
<H2>
|
|
Event System interface</H2>
|
|
The event system interface consists of seven function which can be provided
|
|
by an event system. The synchronous time stamp support software uses a
|
|
card number of zero on all functions that require a card number. The functions
|
|
are as follows:
|
|
<UL>
|
|
<LI>
|
|
long ErHaveReceiver(int event_card_number)</LI>
|
|
|
|
<LI>
|
|
long ErGetTicks(int event_card_number, unsigned long* ticks)</LI>
|
|
|
|
<LI>
|
|
long ErRegisterEventHandler(int event_card_num,EVENT_FUNC event_func)</LI>
|
|
|
|
<LI>
|
|
long ErRegisterErrorHandler(int event_card_num,ERROR_FUNC error_func)</LI>
|
|
|
|
<LI>
|
|
long ErForceSync(int event_card_number)</LI>
|
|
|
|
<LI>
|
|
long ErGetTime(struct timespec* time_stamp)</LI>
|
|
|
|
<LI>
|
|
long ErSyncEvent()</LI>
|
|
|
|
<LI>
|
|
long ErDirectTime()</LI>
|
|
|
|
<LI>
|
|
long ErDriverInit()</LI>
|
|
</UL>
|
|
The definition are as follows:
|
|
<UL>
|
|
<LI>
|
|
<B>ErHaveReceiver()</B>: Returns -1 if no event (timing) hardware present,
|
|
else returns the number of supported events.</LI>
|
|
|
|
<LI>
|
|
<B>ErGetTicks()</B>: Returns the number of ticks since the last hardware
|
|
tick counter reset.</LI>
|
|
|
|
<LI>
|
|
<B>ErRegisterEventHandler()</B>: Informs the event system of a function
|
|
to call when an event occurs, the format of the function will be defined
|
|
below.</LI>
|
|
|
|
<LI>
|
|
<B>ErRegisterErrorHandler()</B>: Informs the event system of a function
|
|
to call when an error occurs, the format of the function will be defined
|
|
below.</LI>
|
|
|
|
<LI>
|
|
<B>ErForceSync()</B>: This function will force an event generator to generate
|
|
a tick reset event and send it.</LI>
|
|
|
|
<LI>
|
|
<B>ErGetTime()</B>: This function returns the actual time. The intention
|
|
here is that this function will retrieve the actual time from GPS system
|
|
or equivalent and return is in time stamp format.</LI>
|
|
|
|
<LI>
|
|
<B>ErSyncEvent()</B>: Return the event number for the tick reset event.</LI>
|
|
|
|
<LI>
|
|
<B>ErDirectTime()</B>: Return 0 for normal operation, return value not
|
|
= 0 for systems that has direct time access, such as a GPS.</LI>
|
|
|
|
<LI>
|
|
<B>ErDriverInit()</B>: The time stamp driver initialization function will
|
|
call this user supplied function before it returns and after it sets up
|
|
the vxWorks clock and attempts to set up the TIMEZONE variable. This can
|
|
be useful to initialize a system such as a GPS (the year can be determined
|
|
using the vxWorks ansiTime library).</LI>
|
|
</UL>
|
|
All of these routines are checked to exist when the time stamp support
|
|
code initializes. All have some kind of default routine provided if they
|
|
are not found, most of which just return an error condition. The functions
|
|
which the time stamp support software registers (event and error) have
|
|
the following format:
|
|
<UL>
|
|
<LI>
|
|
TSeventHandler(int Card,int EventNum,unsigned long Ticks)</LI>
|
|
|
|
<LI>
|
|
TSerrorHandler(int Card, int ErrorNum)</LI>
|
|
</UL>
|
|
Here the Card is the event system board of interest (always zero), the
|
|
EventNum is the event that occurred, and the Ticks is the number of ticks
|
|
since the last board tick counter reset.
|
|
<P>In addition to the above functions, a global variable exists on the
|
|
IOC which can be used to indicate that direct time is available. Setting
|
|
the variable <B>TSdirectTimeVar</B> to a nonzero value has the same effect
|
|
as providing the ErDirectTime() function.
|
|
<H3>
|
|
Creating Direct Time Support</H3>
|
|
Most of the above interface functions apply only when special event hardware
|
|
is present. A much simpler configuration is when a GPS is present and time
|
|
stamps are distributed using IRIG-B to all or most of the IOCs. The easiest
|
|
way to implement this scenario is to define ErDirectTime() to return the
|
|
value one and define ErGetTime(). The job of ErGetTime() will be to actually
|
|
be to generate and return the time stamp representing the current time
|
|
(from the EPICS epoch). At system initialization, the actual time is retrieved
|
|
from a Unix server through NTP or the Unix time protocol, so that year
|
|
will be valid in the vxWorks time clock. At the GPS driver initialization,
|
|
the vxWorks function clock_gettime() can be used to calculate the year.
|
|
Record support will internally eventually call ErGetTime() each time the
|
|
record is processed. The GPS driver should be included as part of drvSup.ascii,
|
|
so it initialized in the proper order.
|
|
<H2>
|
|
Operation</H2>
|
|
An IOC can be configured to run in one of four ways: synchronous-master,
|
|
soft-master, synchronous-slave, soft-slave. Each mode maintains time differently.
|
|
When the time stamp support code initializes, it determines the mode which
|
|
it will operate in based on configuration parameters and event system function
|
|
information.
|
|
<H3>
|
|
Synchronous Time</H3>
|
|
Synchronous timing is determined by the presence of event system hardware.
|
|
The function ErHaveReceiver() gives this information. All IOCs using synchronous
|
|
time will have the exact same time stamps. The "tick" event increments
|
|
all IOC's event system tick counters, the "reset tick counter" resets the
|
|
counter to zero and posts the event to the time stamp support software.
|
|
The time stamp support software system knows the current time by looking
|
|
at the last "tick reset event" time and querying the event system with
|
|
ErGetTicks() for the number of ticks which have elapsed since the last
|
|
reset. Since the event handler function gets called with the "ticks since
|
|
last reset", an event time is the last "tick reset event" time plus the
|
|
ticks count in the call.
|
|
<H4>
|
|
Master</H4>
|
|
A synchronous master is determined by the first parameter to TSconfigure()
|
|
and the presence of event hardware. A master must have an event generator
|
|
and an event receiver. Since the event system is configured from EPICS
|
|
database records, the database must have records in it to initialize the
|
|
event system. The master is responsible for providing the "tick" event
|
|
and the "reset tick counter" event.
|
|
<P>When the time stamp support software's event handler function on a master
|
|
timing IOC receives a "reset tick counter" event, a time stamp message
|
|
is broadcast out on the slave_port (configured by TSconfigure()). The master
|
|
timing IOC is also listening on the master_port (also configured by TSconfigure())
|
|
for incoming requests for time stamp information.
|
|
<P>At boot time, a master will set the vxWorks clock from the Unix boot
|
|
server time. The time is retrieved from the boot server using NTP or the
|
|
time protocol, or the server pointed to by the EPICS environment variable
|
|
EPICS_TS_NTP_INET using NTP. The GPS module takes an unknown period of
|
|
time to sync to the correct time, so it can not be used until it's time
|
|
is valid. The event system is not up and running until record support initialization
|
|
is complete, therefore the event system time stamps are not ticking until
|
|
the event system initializes. At the time of the first sync, the event
|
|
system is known to be up, and the "reset tick counter" or "sync" event
|
|
is set to the current time (vxWorks clock).
|
|
<H4>
|
|
Slave</H4>
|
|
A synchronous slave is determined by the first parameter to TSconfigure()
|
|
and the presence of event hardware. This mode is automatically selected
|
|
if no TSconfigure() call is used in the "startup.cmd" file. This type of
|
|
slave must have an event receiver. The EPICS database must be configured
|
|
to initialize the event system.
|
|
<P>When a synchronous slave configuration is determined, the IOC broadcasts
|
|
a request for master on the master_port port. If a master is found, then
|
|
all current time, sync rate, and clock are extracted from the master's
|
|
response (the sync rate and clock rate from the TSconfigure() are overwritten
|
|
to match the master's configuration). If a master is not found, the slave
|
|
IOC goes into a polling mode to try to find a master every two minutes.
|
|
While a slave has no master, The IOC's clock is initialized from the Unix
|
|
boot server and TSgetTimeStamp() returns the vxWorks time clock value.
|
|
<P>The slave experiences the same problems as the master upon boot.
|
|
<H3>
|
|
Soft Time</H3>
|
|
Soft time IOCs use the 60 hertz clock available from vxWorks to maintain
|
|
a time stamp. The IOC will increment a time stamp at a 60 hertz rate. Soft
|
|
time is determined by the absence of event system hardware. All soft timing
|
|
IOC's will not have the same time stamps.
|
|
<H4>
|
|
Master</H4>
|
|
A soft master is determined by the first parameter of TSconfigure() and
|
|
the absence of event hardware. Upon boot, the master soft timing IOC retrieves
|
|
the current time from the Unix boot server. The IOC sets up a soft time-stamp
|
|
counter using a one tick watch dog and sets the vxWorks clock. From this
|
|
point on, the master runs using the soft time-stamp counter. The master
|
|
listens on the master_port port for requests for the master's current time.
|
|
<H4>
|
|
Slave</H4>
|
|
A soft slave is determined by the first parameter of TSconfigure() and
|
|
the absence of the event hardware. This mode is automatically selected
|
|
if no TSconfigure() is present in the "startup.cmd" file. The basic operation
|
|
of a soft slave is to synchronize the time stamp with a master when possible
|
|
or to the Unix boot server if no master is available. Upon initialization
|
|
the time stamp support code determines if a master is present on the network,
|
|
and if NTP support is available from the Unix boot server or the server
|
|
pointed to by the EPICS environment variable EPICS_TS_NTP_INET. The slave
|
|
will request time stamps from a master or unix server at sync_rate_in_seconds
|
|
rate from TSconfigure(). If the time stamp on the slave is found to be
|
|
off, the clock will be incrementally adjusted so that by the next sync,
|
|
the clock will be corrected to match the master's time stamp. A soft slave
|
|
will automatically switch between a master and the unix boot server depending
|
|
on if the master is available or not. In order to sync with the unix boot
|
|
server, NTP must be available there for query only.
|
|
<HR>
|
|
<P>Home page for <A HREF="http://www.aps.anl.gov/asd/controls/hideos/jimk.html">Jim
|
|
Kowalkowski</A> .
|
|
<BR>Argonne National Laboratory <A HREF="http://www.aps.anl.gov/asd/controls/epics_copyright.html">Copyright</A>
|
|
Information
|
|
<ADDRESS>
|
|
jbk@aps.anl.gov (Jim Kowalkowski)</ADDRESS>
|
|
|
|
<BR>updated 8/10/95
|
|
</BODY>
|
|
</HTML>
|