revision of SECoP documentation

- created secop_v2017-09-14.rst, based on the GoogleDocs
  SECoP Preliminary V2016-11-30 (rc 2)
- this Document is supposed to contain the full SECoP standard
- created SECoP issues
- moved everything else to "outdated" (kept for reference)

Change-Id: I87d69d1846fc4ed55f1c78b22fd4650d8550152b
Reviewed-on: https://forge.frm2.tum.de/review/16573
Reviewed-by: Enrico Faulhaber <enrico.faulhaber@frm2.tum.de>
Tested-by: JenkinsCodeReview <bjoern_pedersen@frm2.tum.de>
This commit is contained in:
zolliker 2017-10-24 13:56:21 +02:00 committed by Enrico Faulhaber
parent 622bc8b3d3
commit 8663501150
24 changed files with 1332 additions and 74 deletions

1
.gitignore vendored
View File

@ -18,3 +18,4 @@ doc/_build
# pytest # pytest
.cache .cache
.coverage .coverage
._*

View File

@ -13,6 +13,9 @@
# All configuration values have a default; values that are commented out # All configuration values have a default; values that are commented out
# serve to show the default. # serve to show the default.
# pylint: disable-all
# pylint: skip-file
# If extensions (or modules to document with autodoc) are in another directory, # If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the # add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here. # documentation root, use os.path.abspath to make it absolute, like shown here.
@ -55,8 +58,9 @@ master_doc = 'index'
# General information about the project. # General information about the project.
project = 'SECoP' project = 'SECoP'
copyright = '2017, Dr. Enrico Faulhaber' #copyright = '2017, Enrico Faulhaber, Markus Zolliker'
author = 'Dr. Enrico Faulhaber' copyright = '2017, SECoP Committee'
author = 'Enrico Faulhaber, Markus Zolliker'
# The version info for the project you're documenting, acts as replacement for # The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the # |version| and |release|, also used in various other places throughout the
@ -77,7 +81,7 @@ language = None
# List of patterns, relative to source directory, that match files and # List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files. # directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path # This patterns also effect to html_static_path and html_extra_path
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', '**/._*']
# The name of the Pygments (syntax highlighting) style to use. # The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx' pygments_style = 'sphinx'
@ -153,7 +157,7 @@ latex_elements = {
# author, documentclass [howto, manual, or own class]). # author, documentclass [howto, manual, or own class]).
latex_documents = [ latex_documents = [
(master_doc, 'SECoP.tex', 'SECoP source documentation', (master_doc, 'SECoP.tex', 'SECoP source documentation',
'Dr. Enrico Faulhaber', 'manual'), 'Enrico Faulhaber, Markus Zolliker', 'manual'),
] ]

View File

@ -9,8 +9,8 @@ double
:stub-columns: 1 :stub-columns: 1
* - Datatype * - Datatype
- | ["double"] **or** - | ["double"] *or*
| ["double", <min>] **or** | ["double", <min>] *or*
| ["double", <min>, <max>] | ["double", <min>, <max>]
| |
| if <max> is not given or null, there is no upper limit | if <max> is not given or null, there is no upper limit
@ -30,8 +30,8 @@ int
:stub-columns: 1 :stub-columns: 1
* - Datatype * - Datatype
- | ["int"] **or** - | ["int"] *or*
| ["int", <min>] **or** | ["int", <min>] *or*
| ["int", <min>, <max>] | ["int", <min>, <max>]
| |
| if <max> is not given or null, there is no upper limit | if <max> is not given or null, there is no upper limit
@ -85,8 +85,8 @@ string
:stub-columns: 1 :stub-columns: 1
* - Datatype * - Datatype
- | ["string"] **or** - | ["string"] *or*
| ["string", <max len>] **or** | ["string", <max len>] *or*
| ["string", <max len>, <min len>] | ["string", <max len>, <min len>]
| |
| if <max len> is not given, it is assumed as 255. | if <max len> is not given, it is assumed as 255.
@ -96,7 +96,7 @@ string
* - Transport example * - Transport example
- | "hello!" - | "hello!"
* - Datatype in C/C++ * - Datatype in C/C++ API
- | char \* - | char \*
blob blob
@ -107,7 +107,7 @@ blob
:stub-columns: 1 :stub-columns: 1
* - Datatype * - Datatype
- | ["blob", <max len>] **or** - | ["blob", <max len>] *or*
| ["blob", <max len>, <min len>] | ["blob", <max len>, <min len>]
| |
| if <min len> is not given, it is assumed as 0. | if <min len> is not given, it is assumed as 0.
@ -115,7 +115,7 @@ blob
* - Transport example * - Transport example
- | "AA==" (base64 encoded) - | "AA==" (base64 encoded)
* - Datatype in C/C++ * - Datatype in C/C++ API
- | struct {int64_t len, char \*data} - | struct {int64_t len, char \*data}
array array
@ -126,7 +126,7 @@ array
:stub-columns: 1 :stub-columns: 1
* - Datatype * - Datatype
- | ["array", <basic type>, <max len>] **or** - | ["array", <basic type>, <max len>] *or*
| ["array", <basic type>, <max len>, <min len>] | ["array", <basic type>, <max len>, <min len>]
| |
| if <min len> is not given, it is assumed as 0. | if <min len> is not given, it is assumed as 0.
@ -135,7 +135,7 @@ array
* - Transport example * - Transport example
- | [3,4,7,2,1] - | [3,4,7,2,1]
* - Datatype in C/C++ * - Datatype in C/C++ API
- | <basic_datatype>[] - | <basic_datatype>[]
tuple tuple
@ -151,8 +151,8 @@ tuple
* - Transport example * - Transport example
- | [0,"idle"] - | [0,"idle"]
* - Datatype in C/C++ * - Datatype in C/C++ API
- | struct ?? - | struct
struct struct
------ ------
@ -167,7 +167,7 @@ struct
* - Transport example * - Transport example
- | {"x": 0, "y": 1} - | {"x": 0, "y": 1}
* - Datatype in C/C++ * - Datatype in C/C++ API
- | struct ?? - | struct
| |
| might be null | might be null

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.4 KiB

View File

@ -2,20 +2,20 @@ Protocol documentation
====================== ======================
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 3
messages secop_v2017-09-14
descprop issue_1
hierarchy issue_2
descjson issue_3
timeformat issue_4
modsubset issue_5
heartbeat issue_6
jsonstruct issue_7
datatypes issue_8
commands issue_9
todo issue_10
notes outdated
history history

View File

@ -0,0 +1,27 @@
SECoP Issue 1: About SECoP Issues
=================================
The idea behind SECoP Issues is to document properly what was proposed,
what was discussed, what was decided and why it was decided.
An issue might take different states:
proposed
--------
A proposal should contain the motivation. As long as nobody else
joins into a discussion, the state remains *proposed*
under discussion
----------------
This state is kept until there is a decision taken.
closed
------
After a decision the state is *closed*. The issue is not deleted,
even if the decision was to not follow further the proposal.
This is helpful if somebody later rises a similar issue.
However, it should be possible to reopen an issue, if new
arguments arise.

View File

@ -0,0 +1,45 @@
SECoP Issue 10: Character set for Names (under discussion)
==========================================================
Uppercase Characters in Identifiers
-----------------------------------
Actually (V2017-09-14), the following is specified;
The identifiers are composed by
lowercase ascii letters, digits and underscore, where a digit may not
appear as the first character. Identifiers starting with underscore are
reserved for special purposes like internal use for debugging. The
identifier length is limited (<=63 characters).
In the outdated part of the documentation (`outdated Messages`_) it is not
explicitly stated, that identifiers have to be lower case, and also
in some of the running examples contain uppercase identifiers.
.. _`outdated Messages`: messages.html
Discussion
~~~~~~~~~~
Markus:
For me, it is not that important, if uppercase characters are allowed or not, but,
identifiers must be unique independent of case, i.e. it is not allowed to have two
different modules "t" and "T", on a SEC node. And we should take a decision soon.
Default parameters 'value' and 'target'
---------------------------------------
In the implementation, in the "read" and "update" messages "<module>:value" can be just
replaced by "<module>", and in "change" and "changed" message "<module>:target" can
be replaced by "<module>".
Discussion
~~~~~~~~~~
Markus:
I prefer not to allow these shortcuts. We also have to distinguish module properties
from properties of the module parameter 'value'.

View File

@ -0,0 +1,22 @@
SECoP Issue 2: Equipment ID in Describing Message (under discussion)
====================================================================
The equipment ID is a SEC node property, and it is therefore redundant
to put it as the second item of the describe message.
However as the describe/describing message might be extended later, for
example to get the description of single modules only, we should specify
a fixed word for the second item of the describe message, for example the
keyword "ALL" or "All".
At the meeting in Berlin (2017-05-30) this was discussed, but it was not
yet decided the the keyword should be exactly. Until a final decision,
SECoP clients should ignore the second item.
Opinions
--------
We should use key keyword ALL (Markus Zolliker)
Decision
--------

View File

@ -0,0 +1,47 @@
SECoP Issue 3: SECoP Timestamp Format (closed)
==============================================
Proposals for the timestamp format are:
ISO time format
---------------
A date+time string in ISO format like "2017-06-21T13:30:01.123456+02:00"
The fractional seconds are optional, but the timezone has to be given. Z is allowed instead of +00:00.
Advantages:
* human readable
Disadvantages:
* needs more conversion efforts, as the time is internally already stored as numbers on mosts systems (supporting math operations).
* although the ISO standard is clearly defined, there is a risk that time zones and daylight saving time is not handled properly
Fractional Unix Time
--------------------
Seconds since 1970-01-01T00:00:00+00:00, represented as a number. When converted to a IEEE double, a resolution of 1 usec can be kept for dates up to 2112.
Advantages:
* if a double is used as an internal representation, no conversion is needed. using a double as an internal time representation has the advantage, that math operations can be done for free.
* relative times for systems with no synchronized clock can be represented easily
Disadvantages:
* not human readable (or at least not easily: time differences in seconds are still visible)
Discussion
----------
1) Human readibility was judged less important than easy implementaion by the majority.
2) Implementing relative times is also easier.
Decision
--------
At the meeting in Berlin (2017-05-30) the attendes decided for "Fractional Unix Time".

View File

@ -0,0 +1,18 @@
SECoP Issue 4: Timeout (under discussion)
=========================================
For a SECoP client, in order to detect that a connection to a SECoP server is dead,
'ping' messages can be sent. When no 'pong' message is received within a specified
time, then the client can consider the connection as dead.
If the server does not specify a the SEC node property 'timeout', the timeout
is assumed to be 3s.
Opinions
--------
On the meeting in Garching (2017-09-14) it was proposed to fix this a standard.
Decision
--------

View File

@ -0,0 +1,34 @@
SECoP Issue 5: Definition of the term 'Property' (under discussion)
===================================================================
The definition as in SECoP V2016-11-30 draft is not very consistent.
As decriptive data we have:
- SEC node properties
- module properties
- parameters properties
- command properties
Live data may be:
- value
- timestamp 't'
- sigma 'e'
It is confusing to use the same term 'property' for live data and for
descriptive data.
The section with the definition of properties has to be rewritten.
Opinions
--------
Enrico proposed to name live data like timestamps and sigma 'qualifiers'.
Markus supports this. He would be happy also with an other term than
'qualifiers', but definitely does not like the terms 'live properties' and
'descriptive properties', as two different things share the same name.
Decision
--------

View File

@ -0,0 +1,23 @@
SECoP Issue 6: Keep Alive (under discussion)
============================================
For a SECoP server, in order to detect that a client connection is dead,
it might close a connection with no messages within a defined period of time.
The discussed mechanism is:
The SECoP client has to set the connection parameter 'keepalive' to a value
representing the number of seconds it will send 'ping' (or other) messages.
The SECoP server can close connections with no messages for a time period
well above this value (more than 10% higher).
Opinions
--------
Markus proposes to mention the 10 % in the specification.
It should also be mentioned, that for keeping the connection alive
any message might be sent instead of the ping message.
Decision
--------

View File

@ -0,0 +1,24 @@
SECoP Issue 7: Time Synchronization (under discussion)
======================================================
As a SECoP server and a SECoP client might not run with a common clock,
the SECoP client should be able to correct for time slips.
The recommended mechanism is (proposal by Markus):
After connecting to a server, the client records its internal time before sending
a ping request.
If the pong request does not contain a timestamp,
the client knows, that the SEC node does not provide timestamps and
therefore it has to create timestamps on the time of reception of messages.
If the returned timestamp lies between the time the ping message was sent and the
time the pong message was received, it does not need to correct the timestamps.
If not, the average of the 'ping' time and the 'pong' time is calculated and
the difference to the received timestamp is used for correcting further
timestamps.
On the meetings in Berlin and Garching in 2017 it was proposed to put this into
the standard, the exact wording has to be decided.

View File

@ -0,0 +1,15 @@
SECoP Issue 8: Groups (under discussion)
========================================
Proposal
--------
Modules as well as parameters may have a property "group".
If the client has the possibility to group modules and/or
parameters, it should use this information for grouping.
Groups must start with uppercase letters.
The "group" property may contain colons (':') as separator,
in order to construct a hierarchy of more than one level.

View File

@ -0,0 +1,39 @@
SECoP Issue 9: Module Meaning (under discussion)
================================================
meaning
-------
For the ECS an automatic detection of the main modules would be desirable.
For example a SEC Node could tell which sensor would be the closest one to
the sample, which should be registered in the ECS as the sample temperature.
For this, a module property "meaning" is proposed. This can get one of the
following values:
* sample_temperature
* magnetic_field
importance
----------
But what to do, if several modules claim to be the sample temperature?
There might be a SEC node controlling cryostat, which has a sample temperature sensor,
but another SEC node controlling an insert with a sample sensor. As the insert
is put into the cryostat, we declare the cryostat to have importance=1 and
the insert an importance=2. To resolve the ambiguity, the ECS chooses the
module with the highest importance to be labelled as the read sample temperature.
Proposal:
predefined importance:
* importance=1 for a device which can not be inserted or added to another one
* importance=2 for a device which can be inserted into an other one
Higher values are to be used when an additional device may be inserted into an insert
and the like.
We should allow importance to be a floating point number, in case later a value
between 1 and 2 has to be used.

View File

@ -90,5 +90,3 @@ vorerst folgende Festlegung:
], ],
"version": "2017.01" "version": "2017.01"
} }

View File

@ -19,15 +19,21 @@ or one of a fixed list of predefined keywords, depending on the message keyword.
At the moment it is considered syntactic sugar to omit the parametername in a request. At the moment it is considered syntactic sugar to omit the parametername in a request.
In replies the SEC-node (in the playground) will always use the correct parameter. In replies the SEC-node (in the playground) will always use the correct parameter.
On change-requests the parameter is assumed to be 'target', on trigger-requests it is assumed to be 'value'. On change-requests the parameter is assumed to be 'target', on trigger-requests it is
Clients should not rely on this and explicitly state the parametername! assumed to be 'value'. Clients should not rely on this and explicitly state the parametername!
All names and keywords are defined to be identifiers in the sense, that they are not longer than 63 characters and consist only of letters, digits and underscore and do not start with a digit. (i.e. T_9 is ok, whereas t{9} is not!) All names and keywords are defined to be identifiers in the sense, that they are not longer
No rule is without exception, there is exactly ONE special case: the identify request consists of the literal string '\*IDN?\\n' and its answer is formatted like an valid SCPI response for \*IDN?. than 63 characters and consist only of letters, digits and underscore and do not start
with a digit. (i.e. T_9 is ok, whereas t{9} is not!)
No rule is without exception, there is exactly ONE special case: the identify request
consists of the literal string '\*IDN?\\n' and its answer is formatted like an valid SCPI
response for \*IDN?.
We rely on the underlying transport to not split messages, i.e. all messages are transported as a whole and no message interrupts another. We rely on the underlying transport to not split messages, i.e. all messages are
transported as a whole and no message interrupts another.
Also: each client MUST at any time correctly handle incoming event messages, even if it didn't activate them! Also: each client MUST at any time correctly handle incoming event messages, even
if it didn't activate them!
Implementation node: Implementation node:
Both SEC-node and ECS-client can close the connection at any time! Both SEC-node and ECS-client can close the connection at any time!
@ -62,7 +68,8 @@ Identify
* Request: type A: '\*IDN?' * Request: type A: '\*IDN?'
* Reply: special: 'SECoP, SECoPTCP, V2016-11-30, rc1' * Reply: special: 'SECoP, SECoPTCP, V2016-11-30, rc1'
* queries if SECoP protocol is supported and which version it is * queries if SECoP protocol is supported and which version it is
Format is intentionally choosen to be compatible to SCPI (for this query only).
The format is intentionally choosen to be compatible to SCPI (for this query only).
It is NOT intended to transport information about the manufacturer of the hardware, but to identify this as a SECoP device and transfer the protocol version! It is NOT intended to transport information about the manufacturer of the hardware, but to identify this as a SECoP device and transfer the protocol version!
@ -73,6 +80,7 @@ Describe
* Reply: type C: 'describing <ID> {"modules":{"T1":{"baseclass":"Readable", ....' * Reply: type C: 'describing <ID> {"modules":{"T1":{"baseclass":"Readable", ....'
* request the 'descriptive data'. The format needs to be better defined and * request the 'descriptive data'. The format needs to be better defined and
may possibly just follow the reference implementation. may possibly just follow the reference implementation.
<ID> identifies the equipment. It should be unique. Our suggestion is to use something along <facility>_<id>, i.e. MLZ_ccr12 or PSI_oven4. <ID> identifies the equipment. It should be unique. Our suggestion is to use something along <facility>_<id>, i.e. MLZ_ccr12 or PSI_oven4.
@ -109,6 +117,7 @@ Write
* Request: type C: 'change <module>[:<param>] JSON_value' * Request: type C: 'change <module>[:<param>] JSON_value'
* Reply: type C: 'changed <module>:<param> JSON_read_back_value' * Reply: type C: 'changed <module>:<param> JSON_read_back_value'
* initiate setting a new value for the module or a parameter of it. * initiate setting a new value for the module or a parameter of it.
Once this is done, the read_back value is confirmed by the reply. Once this is done, the read_back value is confirmed by the reply.
@ -128,6 +137,7 @@ Heartbeat
* Reply: type A: 'pong' * Reply: type A: 'pong'
* Reply: type B: 'pong <nonce>' * Reply: type B: 'pong <nonce>'
* Replies the given argument to check the round-trip-time or to confirm that the connection is still working. * Replies the given argument to check the round-trip-time or to confirm that the connection is still working.
<nonce> may not contain <space>. It is suggested to limit to a string of up to 63 chars consisting of letters, digits and underscore not beginning with a digit. If <nonce> is not given (Type A), reply without it. <nonce> may not contain <space>. It is suggested to limit to a string of up to 63 chars consisting of letters, digits and underscore not beginning with a digit. If <nonce> is not given (Type A), reply without it.
@ -139,8 +149,8 @@ Events can be emitted any time from the SEC-node (except if they would interrupt
* Request: None. Events can be requested by Trigger or by Activating Async Mode. * Request: None. Events can be requested by Trigger or by Activating Async Mode.
* Reply: type C: 'event <module>:<param> JSON_VALUE' * Reply: type C: 'event <module>:<param> JSON_VALUE'
* Informs the client that a parameter got changed its value. * Informs the client that a parameter got changed its value.
In any case the JSON_value contain the available qualifiers as well:
In any case the JSON_value contain the available qualifiers as well:
* "t" for the timestamp of the event. * "t" for the timestamp of the event.
* "e" for the error of the value. * "e" for the error of the value.
* "u" for the unit of the value, if deviating from the descriptive data * "u" for the unit of the value, if deviating from the descriptive data
@ -162,20 +172,19 @@ ERROR
* Request: None. can only be a reply if some request fails. * Request: None. can only be a reply if some request fails.
* Reply: type C: 'ERROR <errorclass> JSON_additional_stuff' * Reply: type C: 'ERROR <errorclass> JSON_additional_stuff'
* Following <errorclass> are defined so far: * Following <errorclass> are defined so far:
- NoSuchDevice: The action can not be performed as the specified device is non-existent.
* NoSuchDevice: The action can not be performed as the specified device is non-existent. - NoSuchParameter: The action can not be performed as the specified parameter is non-existent.
* NoSuchParameter: The action can not be performed as the specified parameter is non-existent. - NoSuchCommand: The specified command does not exist.
* NoSuchCommand: The specified command does not exist. - CommandFailed: The command failed to execute.
* CommandFailed: The command failed to execute. - CommandRunning: The command is already executing.
* CommandRunning: The command is already executing. - ReadOnly: The requested write can not be performed on a readonly value..
* ReadOnly: The requested write can not be performed on a readonly value.. - BadValue: The requested write or Command can not be performed as the value is malformed or of wrong type.
* BadValue: The requested write or Command can not be performed as the value is malformed or of wrong type. - CommunicationFailed: Some communication (with hardware controlled by this SEC-Node) failed.
* CommunicationFailed: Some communication (with hardware controlled by this SEC-Node) failed. - IsBusy: The reequested write can not be performed while the Module is Busy
* IsBusy: The reequested write can not be performed while the Module is Busy - IsError: The requested action can not be performed while the module is in error state.
* IsError: The requested action can not be performed while the module is in error state. - Disabled: The requested action can not be performed at the moment. (Interlocks?)
* Disabled: The requested action can not be performed at the moment. (Interlocks?) - SyntaxError: A malformed Request was send
* SyntaxError: A malformed Request was send - InternalError: Something that should never happen just happened.
* InternalError: Something that should never happen just happened.
The JSON part should reference the offending request and give an explanatory string. The JSON part should reference the offending request and give an explanatory string.
@ -189,20 +198,22 @@ ERROR
Examples Examples
-------- --------
(client connects): ::
(client) '\*IDN?'
(SEC-node) 'Sine2020WP7.1&ISSE, SECoP, V2016-11-30, rc1' (client connects):
(client) 'describe' (client) '\*IDN?'
(SEC-node) 'describing SECoP_Testing {"modules":{"T1":{"baseclass":"Readable", ... (SEC-node) 'Sine2020WP7.1&ISSE, SECoP, V2016-11-30, rc1'
(client) 'activate' (client) 'describe'
(SEC-node) 'update T1 [3.45,{"t":"149128925.914882","e":0.01924}]' (SEC-node) 'describing SECoP_Testing {"modules":{"T1":{"baseclass":"Readable", ...
... (client) 'activate'
(SEC-node) 'active' (SEC-node) 'update T1 [3.45,{"t":"149128925.914882","e":0.01924}]'
(SEC-node) 'update T1 [3.46,{"t":"149128935.914882","e":0.01912}]' ...
(client) 'ping fancy_nonce_37' (SEC-node) 'active'
(SEC-node) 'pong fancy_nonce_37' (SEC-node) 'update T1 [3.46,{"t":"149128935.914882","e":0.01912}]'
(SEC-node) 'update T1 [3.49,{"t":"149128945.921397","e":0.01897}]' (client) 'ping fancy_nonce_37'
... (SEC-node) 'pong fancy_nonce_37'
(SEC-node) 'update T1 [3.49,{"t":"149128945.921397","e":0.01897}]'
...
merge datatype and validator: merge datatype and validator:
----------------------------- -----------------------------

View File

@ -0,0 +1,20 @@
Outdated documentation
======================
.. toctree::
:maxdepth: 2
messages
descprop
hierarchy
descjson
timeformat
modsubset
heartbeat
jsonstruct
datatypes
commands
todo
notes

View File

@ -0,0 +1,930 @@
SECoP: Sample Environment Communication Protocol
================================================
V2017-09-14
Goal
====
The main goal of the "committee for the standardization of sample
environment communication" is to establish a common standard protocol
SECoP for interfacing sample environment equipment to experiment control
software.
Definition: Experiment Control Software ECS
Software controlling the hardware for carrying out an experiment. Includes the user
interface. Usually speaks several protocols with different parts of the instrument.
Often also called instrument control.
There is a task (7.1) within the European framework SINE2020 also
dealing with this subject. In its description we read:
... The standard should be defined in a way that it is compatible
with a broad variety of soft- and hardware operated at the different
large scale facilities. … The adoption of this standard will greatly
facilitate the installation of new equipment and the share of
equipment between the facilities. ...
This does also cover the aims of the committee.
The idea is, that a sample environment apparatus can easily be moved
between facilities and instruments/beamlines. As long as the facilities
have implemented a SECoP client within its ECS, and on the apparatus a
SECoP server is implemented as the SEC node, using the apparatus for an
experiment should be straightforward. An ECS can be built in a way, that
the configuration of a SEC node may be as short as entering a network
address, as the description can be loaded over the protocol (in
addition, the electronic communication link, electricity and may be
cooling water or pressurized air etc. have to be connected, but that is
out of scope …)
Definition: Sample Environment Control Node (SEC node)
Computing unit or process or task, connected to all control units (temperature controller, flow controller, pressure sensor ...) of a sample environment, bridge to the ECS. SECoP specifies how ECS speaks with the SEC node.
The SEC node allows the ECS to access a set of modules (and their parameters) via the SECoP. It also provides a list of accessible modules and parameters.
Other requirements
------------------
- the protocol should be easy to use
- it should be easy to implement in connection with existing ECSs and
sample environment control software
- it should be possible to be implemented on the most common platforms
(operating systems and programming languages)
- the protocol should be defined in way that allows a maximum
**compatibility**: Newer and older versions of the syntax should
be compatible
- the protocol should be defined in a way, which allows a maximum
**flexibility**: A simple (= with minimal features) ECS
implementation should be able to communicate with a complex SEC
node (with a lot of features), and an ECS with a rich number of
features should be able to cope with a simple SEC node,
implementing only a minimum number of features
Hardware Abstraction
====================
Modules
-------
Definition: Module
One logical component of an abstract view of the sample environment. Can at least be read.
May be driven' (set new setpoint). May have parameters influencing how it achieves
its function (e.g. PID parameters). May have some additional diagnostics (read-only) parameters.
May provide some additional status information (temperature stable?, setpoint reached?)
Reading a module returns the result of the corresponding physical measurement.
In earlier discussion we used the term "device" for module, which might
be misleading, as "device" is often used for an entire apparatus, like a
cryomagnet or humidity cell. In the context of SECoP, an apparatus in
general is composed of several modules. For example different
temperature sensors in one apparatus can be seen as different modules.
An SEC node controls several (or one or no) modules. Modules also have
some descriptive data (name, type, list\_of\_parameters,
list-of\_commands,...).
Parameters
----------
A module has several parameters associated with it. A parameter is
addressed by the combination of module and parameter name. Module names
have to be unique within an SEC node, parameter names have to be unique
within a module.
Module and parameter names should be in english (incl. acronyms), using
only ascii letters and some additional characters (see section "Message
Syntax"). A maximum name length might be imposed.
Definition: Parameter
The main parameter of a module is its value. Writable parameters may influence the
measurement (like PIDs). Additional parameters may give more information about its
state (running, target reached), or details about its functioning (heater power) for
diagnostics purposes. Parameters with a predefined meaning are listed in the standard,
they must always be used in the same way. Custom parameters are defined by the
implementation of the SEC node, the ECS can use them only in a general way, as their
meaning is not known.
The following parameters are predefined (extensible):
- **value**
- **status** (consists of two elements: a status with predefined values
as "idle","busy","error", and a describing text).
*Remark: it is proposed to add additional states (starting,
started, pausing, paused, stopping, warning). It has to be
discussed, if this (and therefore a start and pause command)
makes sense. Generally we want to keep the number of states as
small as possible here.*
- **target** (not present, if the module is not writable)
- **pollinterval** (double, a hint for the module for the polling interval in seconds)
The following parameters were discussed at the meeting in
- **ramp** (writable parameter, desired ramp. Units: main units/min)
- **use\_ramp** (writable, 1 means: use given ramp, 0 means: go as fast as possible)
- **setpoint** (ramping setpoint, read only)
- **time\_to\_target** (read only, expected time to reach target)
Commands
--------
A module may also have commands associated with it. A command is
addressed by the combination of module and parameter name. Like
parameters, command names have to be unique within a module, and should
be in english (incl. acronyms), using only ascii letters and some
additional characters (see section "Message Syntax"). A maximum name
length might be imposed.
Definition: Command
Commands are provided to initiate specified actions of the module.
They should return immediately after that action is initiated, i.e.
should not wait until some other state is reached. Commands may get
an argument list and may return a result list.
Commands with a predefined meaning are listed in the standard,
they must always be used in the same way.
Custom commands are defined by the implementation of the SEC node, the
ECS can use them only in a general way, as their meaning is not known.
So far the only command defined (for driveable modules) is stop (no
arguments, no result). When a modules target is changed, it is 'driving'
to a new value until the target is reached or until its stop command
is sent.
It is still to be discussed, what this exactly means for temperature
devices (heater off vs. stay at current temp).
The following commands are predefined (extensible):
- stop (stop a running action)
proposed further commands:
- start, pause (proposed from ILL)
- shutdown (go to safe state)
Properties
----------
*definition of properties to be rewitten, see* `SECoP issue 5`_
.. _`SECoP issue 5`: SECoP_issues/issue_5.html
Definition: Properties
Parameters have a value, live properties and descriptive properties. All properties have a predefined name and meaning.
Live Properties
Properties which change their value during an experiment.
Descriptive Properties
Properties that do not change during an experiment. They describe the usage of the parameter.
Live Properties (or *qualifiers* ?) are optional.
There are currently only 2 live properties:
- timestamp (named t in the proposed syntax). The time when the
parameter has changed or was verified/measured (when no timestamp
is given, the ECS should use the time of the update message as
the timestamp)
- sigma (named e in the proposed syntax). The uncertainty of a
measurement (default: unspecified)
other live properties might be added later to the standard.
For a list of descriptive properties see chapter "Descriptive Data".
Classes
-------
The idea is, that the ECS can determine the functionality of a module
from its class.
Base classes (recommended):
- Readable
- Writable (must have a target parameter)
- Drivable (a Writable, must have a stop command, the status parameter will indicate
busy for a longer-lasting operation)
Examples of classes:
- CryomagnetSupply (a Drivable, with ramp and optional a persistent
switch)
- TemperatureLoop (a Drivable, with PIDs, optional with ramp)
The standard contains a list of classes, and a specification of the
functionality for each of them. The list might be extended over time.
Already specified base classes may be extended in later releases of the
specification, but earlier definitions will stay intact, i.e. no
removals or redefinitions will occur.
The module class is in fact a list of classes (highest level class
first). The ECS chooses the first class from the list which is known to
it.
Protocol
========
The basic element of the protocol are messages.
Message Syntax
--------------
A message is one line of text, coded in ASCII (may be extended to UTF-8
later if needed).
A message ends with a line feed character (ASCII 10), it may be preceded
by a carriage return character (ASCII 13), which must be ignored. A
message starts with a keyword, followed optionally by one space and a qualified name
or another item not containing spaces, followed optionally by one space and a JSON
formatted value. Note: numerical values and strings appear 'naturally' formatted
in JSON, i.e. 5.0 or "a string".
A qualified name consists of a module identifier, a colon as separator
and a parameter or command identifier. The identifiers are composed by
lowercase ascii letters, digits and underscore, where a digit may not
appear as the first character. Identifiers starting with underscore are
reserved for special purposes like internal use for debugging. The
identifier length is limited (<=63 characters).
A SEC node might implement custom messages for debugging purposes, which are not
part of the standard. Custom messages start with an underscore or might just be
an empty line. The latter might be used as a request for a help text, when logged
in from a command line client like telnet or netcat. Messages not starting with
an underscore and not defined in the following list are reserved for future extensions.
Message Summary
~~~~~~~~~~~~~~~
.. list-table::
:widths: 20 20 60
:header-rows: 1
* - message
- message kind
- syntax
* - identification
- request
- **\*IDN?**
* -
- reply
- ISSE&SINE2020\ **,SECoP,**\ *version,add.info*
* - description
- request
- **describe**
* -
- reply
- **describing ALL** *description*
* - activate updates
- request
- **activate** *[module]*
* -
- reply (after first updates)
- **active** *[module]*
* - update
-
-
* -
- asynchronous msg.
- **update** *module*\ **:**\ *parameter value*
* - deactivate updates
- request
- **deactivate** *[module]*
* -
- reply
- **inactive** *[module]*
* - change value
- request
- **change** *module*\ **:**\ *parameter value*
* -
- reply
- **changed** *module*\ **:**\ *parameter value*
* - read request
- request
- **read** *module*\ **:**\ *parameter*
* -
- reply
- **update** *module*\ **:**\ *parameter value*
* - execute command
- request
- **do** *module*\ **:**\ *command [argument]*
* -
- reply
- **done** *module*\ **:**\ *command [result]*
* - error
-
-
* -
- reply (on any message)
- **error** *errortype* *info*
* - heartbeat
- request
- **ping** *[id]*
* -
- reply
- **pong** *[id]* [\ **{"t":** *localtime* **}**\ ]
Identification
~~~~~~~~~~~~~~
The syntax of the identification message differs a little bit from other
messages, as it should be compatible with IEEE 488.2. The identification
request "\ **\*IDN?**\ " is meant to be sent as the first message after
establishing a connection. The reply consists of 4 comma separated
fields, where the second and third field determine the used protocol.
In this and in the following examples, messages sent to the server are marked with "> ",
and messages sent to the client are marked with "< "
Example:
.. code::
> *IDN?
< ISSE&SINE2020,SECoP,V2017-05-30,rc1
Description
~~~~~~~~~~~
The next messages normally exchanged are the description request and
reply. The reply is a structured JSON value describing the name of
modules exported and their parameters, together with the corresponding
properties.
Example:
.. code::
> describe
< describing PSI\_MP03 {"modules":["t1",["class":[ "temperature\_sensor","readable"],"parameters":["value", ...
It is not yet clear what the second item in the reply message should be.
In this example it is the equipment ID, but this is redundant, as
"equipment_id" is also a SEC node property.
See `SECoP Issue 2`_ (Equipment ID in Describing Message)
.. _`SECoP Issue 2`: SECoP_Issues/issue_2.html
Remark:
this reply might be a very long line, no line breaks are allowed in the
JSON value.
Activate Updates
~~~~~~~~~~~~~~~~
The parameterless "activate" request triggers the SEC node to send the
values of all its modules and parameters as update messages. When this
is finished, the SEC node must send an "active" reply.
A SEC node might accept a module name as second item of the
message, activating only updates on the parameters of the module.
In this case, the "active" reply also contains the module names.
A SEC Node not implementing module-wise activation must not sent the module
name in its reply, and must activate all modules.
Update
~~~~~~
When activated, update messages are delivered without explicit request
from the client. The value is a JSON array with the value as its first
element, and an JSON object containing the qualifiers (live properties)
as its second element:
"t": the timestamp (recommended, when the system has a synchronized
time, the format is fractional seconds since 1970-01-01T00:00:00+00:00)
See also `SECoP Issue 3`_ (Timestamps)
"e": the error of the quantity (optional)
Example:
.. code::
> activate
< update t1:value [295.13,{"t":1505396348.188388,"e":0.01}]
< update t1:status [[400,"heater broken or disconnected"],{"t":1505396348.288388}]
< active
Deactivate Updates
~~~~~~~~~~~~~~~~~~
A parameterless message. After the "inactive" reply no more updates are
delivered if not triggered by a read message.
Example:
.. code::
> deactivate
< update t1:value [295.13,{"t":1505396348.188388}]
< inactive
remark: the update message in the second line was sent before the deactivate message
was treated. After the "inactive" message, the client can expect that no more untriggered
update message are sent.
The deactivate message might optionally accept a module name as second item
of the message for module-wise deactivation. If module-wise deactivation is not
supported, it should ignore a deactivate message which contains a module name.
Remark: it is not clear, if module-wise deactivation is really useful. A SEC Node
supporting module-wise activation does not necessarily need to support module-wise
deactivation.
Change Value
~~~~~~~~~~~~
the change value message contains the name of the module or parameter
and the value to be set. The value is JSON formatted, but note that for
a floating point value this is a simple decimal coded ASCII number. As
soon as the set-value is read back from the hardware, a "changed"
message is sent (in case updates are activated). If the value is not
stored in hardware, the "changed" message can be sent immediately.
Example on a connection with activated updates. Live properties are replaced by {...} for brevity here.
.. code::
> read mf:status
< update mf:status [[100,"OK"],{...}]
< change mf:target 12
< update mf:status [[300,"ramping field"],{...}]
< changed mf:target [12,{...}]
The status changes from "idle" to "busy". The ECS will be informed with a further update message on mf:status, when the module has finished ramping.
Read Request
~~~~~~~~~~~~
With the read request message the ECS may ask the SEC node to update a
value as soon as possible, without waiting for the next regular update.
The reply is an update message. If updates are not activated, the
message can be treated like a read message in a request-reply scheme as
in the previous SECoP proposal.
Example:
.. code::
> read t1:value
< update t1:value [295.13,{"t":1505396348.188}]
> read t1:status
> update t1:status [[100,"OK"],{"t":1505396348.548}]
Command
~~~~~~~
A command may have arguments. Multiple arguments can be given as a JSON list.
A command may also have a return value. The "done" reply always contains the
JSON part with at least the timestamp, if supported. If no value is returned,
the data part is set to "null".
The "done" message should be returned quickly, the time scale should be in the
order of the time needed for communications. Actions which have to wait for physical
changes, can be triggered with a command. The information about the success of
such an action has to be transferred via parameters, namely the status parameter.
Example:
.. code::
> do t1:stop
< done t1:stop [null, {"t": 1505396348.876}]
Error Reply
~~~~~~~~~~~
Contains an error class from the list below as its second item.
The third item of the message is a JSON list,
containing the related request message as its first element, a human readable text
as its second element. The third element is a JSON-Object, containing possibly
implementation specific information about the error (stack dump etc.).
Example:
.. code::
> read tx:target
< error NoSuchModule ["read tx:target", "tx is not configured on this SEC node", {}]
> read ts:target
< error NoSuchParameter ["read ts:target", "ts has no parameter target", {}]
> meas:volt?
< error SyntaxError ["meas:volt?", "unknown keyword", {}]
Error Classes
.. list-table::
:widths: 20 80
* - NoSuchModule
- The action can not be performed as the specified module is non-existent.
* - NoSuchParameter
- The action can not be performed as the specified parameter is non-existent.
* - NoSuchCommand
- The specified command does not exist.
* - CommandFailed
- The command failed to execute.
* - CommandRunning
- The command is already executing.
* - ReadOnly
- The requested write can not be performed on a readonly value..
* - BadValue
- The requested write or Command can not be performed as the value is malformed or of wrong type.
* - CommunicationFailed
- Some communication (with hardware controlled by this SEC-Node) failed.
* - IsBusy
- The reequested write can not be performed while the Module is Busy
* - IsError
- The requested action can not be performed while the module is in error state.
* - Disabled
- The requested action can not be performed at the moment. (Interlocks?)
* - SyntaxError
- A malformed Request was send
* - InternalError
- Something that should never happen just happened.
Timeout Issues / Heartbeat
~~~~~~~~~~~~~~~~~~~~~~~~~~
In order to detect that the other end of the communication is not dead,
a heartbeat may be sent. The second part of the message (the id) may
not contain a space and should be short. It may be omitted. The reply
will contain exactly the same id.
A SEC node might also decide to close a connection when it gets no
messages for a certain time. The mechanism is under discussion.
Generally speaking: both
ECS and SEC side needs to be aware that the other side may close the
connection at any time!
Example:
.. code::
> ping 123
< pong 123 {"t": 1505396348.543}
The "pong" message has an additional function: it sends back the time
on the server, if it supports timestamps. This can be used to
synchronize the time.
See also `SECoP Issue 4`_ (Timeout), `SECoP Issue 6`_ (Keep Alive),
`SECoP Issue 3`_ (Timestamps) or `SECoP Issue 7`_ (Time Synchronization)
.. _`SECoP Issue 3`: SECoP_Issues/issue_3.html
.. _`SECoP Issue 4`: SECoP_Issues/issue_4.html
.. _`SECoP Issue 6`: SECoP_Issues/issue_6.html
.. _`SECoP Issue 7`: SECoP_Issues/issue_7.html
Multiple Connections
--------------------
A SEC node may accept only a limited number of connections, downto 1.
However, each SEC node should support as many connections as technically
feasible.
Details about how to multiplex multiple connections onto one are to be
discussed.
Descriptive Data
----------------
Format of Descriptive Data
~~~~~~~~~~~~~~~~~~~~~~~~~~
The format of the descriptive data is JSON, as all other data in SECoP.
.. for creating the railroad diagrams see: http://bottlecaps.de/rr/ui
.. source EBNF:
.. SEC_node_description ::= '{' (SEC_node_property ( ',' SEC_node_property)* )? '}'
.. SEC_node_property ::= property | ( '"modules":' '[' (name ',' module_description (',' name ',' module_description)*)? ']')
.. module_description ::= '{' (module_property ( ',' module_property)* )? '}'
.. module_property ::= property | ( '"parameters":' '[' (name ',' properties (',' name ',' properties)*)? ']') | ( '"commands":' '[' (name ',' properties (',' name ',' properties)*)? ']')
.. properties ::= '{' (property ( ',' property)* )? '}'
.. property ::= (name ':' property_value)
SEC node description
^^^^^^^^^^^^^^^^^^^^
.. image:: diagram/sec_node_description.png
:alt: SEC_node_description ::= '{' (SEC_node_property ( ',' SEC_node_property)* )? '}'
SEC node property
^^^^^^^^^^^^^^^^^
.. image:: diagram/sec_node_property.png
:alt: SEC_node_property ::= property | ( '"modules":' '[' (name ',' module_description (',' name ',' module_description)*)? ']')
module description
^^^^^^^^^^^^^^^^^^
.. image:: diagram/module_description.png
:alt: module_description ::= '{' (module_property ( ',' module_property)* )? '}'
module property
^^^^^^^^^^^^^^^
.. image:: diagram/module_property.png
:alt: module_property ::= property | ( '"parameters":' '[' (name ',' properties (',' name ',' properties)*)? ']') | ( '"commands":' '[' (name ',' properties (',' name ',' properties)*)? ']')
properties
^^^^^^^^^^
.. image:: diagram/properties.png
:alt: properties ::= '{' (property ( ',' property)* )? '}'
property
^^^^^^^^
.. image:: diagram/property.png
:alt: property ::= (name ':' property_value)
SEC Node Properties
~~~~~~~~~~~~~~~~~~~
there might be properties such as a timeout which are relevant for the
communication of a SEC node.
- **equipment_id**
- **description** (mandatory, a text describing the node, in general, the first
line is a short description (line break \\n))
- **firmware** (optional, a string describing the version of the SEC node software)
- **timeout** (optional value in seconds, a SEC node should be able to respond within
a time well below this value. Default: 3 sec, see `SECoP issue 4`_ )
.. _`SECoP issue 4`: SECoP_issues/issue_4.html
Module Properties
~~~~~~~~~~~~~~~~~
- **description** (mandatory, a text describing the parameter)
- **visibility** (1=expert, 2=advanced, 3=user (default)), Note: this
does not imply that the access is controlled. It may just be a
hint to the UI for the amount of exposed modules. (optional)
- **interface\_class** (a list of classes for the module, for example
["Temperature", "DrivableWithRamp", "Drivable", "Readable"]),
mandatory
- **group** (optional, identifier, may contain ':' which may be interpreted as path separator)
(*see* `SECoP issue 8`_)
- **meaning**, **importance** (*see* `SECoP issue 9`_)
.. _`SECoP issue 8`: SECoP_issues/issue_8.html
.. _`SECoP issue 9`: SECoP_issues/issue_9.html
Parameter Properties
~~~~~~~~~~~~~~~~~~~~
- **description** (mandatory, a text describing the parameter, mandatory)
- **readonly** (mandatory)
- **datatype** (mandatory, see `Data Types`_)
- **unit** (default: unitless, should be given, if meaningfull, empty string: unit is one)
- **visibility** (3=expert, 2=advanced, 1=user (default)), Note: this
does not imply that the access is controlled. It may just be a
hint to the UI for the amount of exposed parameters. (optional)
- **group** (optional, identifier, may contain ':' which may be interpreted as path separator)
(*see* `SECoP issue 8`_)
Data Types
----------
double
~~~~~~
.. list-table::
:widths: 20 80
:stub-columns: 1
* - Datatype
- | ["double"] *or*
| ["double", <min>] *or*
| ["double", <min>, <max>]
|
| if <max> is not given or null, there is no upper limit
| if <min> is null or not given, there is no lower limit
* - Transport example
- | 3.14159265
* - Datatype in C/C++
- | double
int
~~~
.. list-table::
:widths: 20 80
:stub-columns: 1
* - Datatype
- | ["int"] *or*
| ["int", <min>] *or*
| ["int", <min>, <max>]
|
| if <max> is not given or null, there is no upper limit
| if <min> is null or not given, there is no lower limit
* - Transport example
- | -55
* - Datatype in C/C++
- | int64_t
bool
~~~~
.. list-table::
:widths: 20 80
:stub-columns: 1
* - Datatype
- | ["bool"]
* - Transport example
- | true
* - Datatype in C/C++
- | int64_t
enum
~~~~
.. list-table::
:widths: 20 80
:stub-columns: 1
* - Datatype
- | ["enum", {<name> : <value>, ....}]
* - Transport example
- | 2
* - Datatype in C/C++
- | int64_t
string
~~~~~~
.. list-table::
:widths: 20 80
:stub-columns: 1
* - Datatype
- | ["string"] *or*
| ["string", <max len>] *or*
| ["string", <max len>, <min len>]
|
| if <max len> is not given, it is assumed as 255.
| if <min len> is not given, it is assumed as 0.
| if the string is UTF-8 encoded, the length is counting the number of bytes, not characters
* - Transport example
- | "hello!"
* - Datatype in C/C++ API
- | char \*
blob
~~~~
.. list-table::
:widths: 20 80
:stub-columns: 1
* - Datatype
- | ["blob", <max len>] *or*
| ["blob", <max len>, <min len>]
|
| if <min len> is not given, it is assumed as 0.
* - Transport example
- | "AA==" (base64 encoded)
* - Datatype in C/C++ API
- | struct {int64_t len, char \*data}
array
~~~~~
.. list-table::
:widths: 20 80
:stub-columns: 1
* - Datatype
- | ["array", <basic type>, <max len>] *or*
| ["array", <basic type>, <max len>, <min len>]
|
| if <min len> is not given, it is assumed as 0.
| the length is the number of elements
* - Transport example
- | [3,4,7,2,1]
* - Datatype in C/C++ API
- | <basic_datatype>[]
tuple
~~~~~
.. list-table::
:widths: 20 80
:stub-columns: 1
* - Datatype
- | ["tuple", [<datatype>, <datatype>, ...]]
* - Transport example
- | [0,"idle"]
* - Datatype in C/C++ API
- | struct
struct
~~~~~~
.. list-table::
:widths: 20 80
:stub-columns: 1
* - Datatype
- | ["struct", {<name> : <datatype>, <name>: <datatype>, ....}]
* - Transport example
- | {"x": 0, "y": 1}
* - Datatype in C/C++ API
- | struct
|
| might be null