Dev/doc architecture commands (#1272)

* wip for sw architecture

* wip:intro

* wip client wip

* documentation on architecutre is done. commands left

* wip. clientto module done

* almost done

* about 2nd port

* done

* review changes

* review fixes

* minor

docs/CMakeLists.txt

@@ -68,6 +68,8 @@ set(SPHINX_SOURCE_FILES
   src/hdf5fileformat.rst
   src/zmqjsonheaderformat.rst
   src/dataformat.rst
+  src/softwarearchitecture.rst
+  src/configcommands.rst
 )
 
 foreach(filename ${SPHINX_SOURCE_FILES})

docs/src/binaryfileformat.rst

@@ -1,3 +1,5 @@
+.. _binary file format:
+
 Binary File Format
 ====================
 
@@ -31,6 +33,7 @@ Data File
 
 * Each frame includes a :ref:`**sls_receiver_header** <sls receiver header format>` structure, followed by the actual frame data.
 
+* More details on :ref:`slsReceiverHeader<sls receiver header format>` and the actual image data is described in the :ref:`Detector Image Size and Format <data format>` section.
 
 
 .. _json master file examples:

docs/src/configcommands.rst

@@ -0,0 +1,233 @@
+.. _setup commands:
+
+Setup Commands
+================================
+
+
+Introduction
+-------------
+
+To connect to any device, one needs a unique combination of **IP address** (which identifies the device) and **port number** (which specifies the service). 
+
+This package typically deals with two types of network interfaces:
+
+* **1 GbE public interface** - Accessible from anywhere on the network. Connectivity can be verified by pinging this interface from any PC.
+
+* **10 GbE private interface** - Dedicated to high-speed data transfer with a specific PC. In addition to the 1 GbE public interface (MTU 1500), the device may include one or more private 10 GbE interfaces (MTU 9000), which are not accessible from other machines.
+
+Client to Module
+-----------------
+
+.. figure:: images/Client_module_commands.png
+   :target: _images/Client_module_commands.png
+   :width: 700px
+   :align: center
+   :alt: Client Module Commands
+
+   Client Module TCP Commands
+
+The client configures and controls modules via the 1 GbE public TCP interface.
+
+* Should be able to ping the module's hostname from any PC on the network.
+* If one cannot ping, ensure that it is powered on.
+* If the command cannot connect to the port (`hostname command <commandline.html#term-hostname>`_ failed), the onboard servers may not have started yet.
+
+Each physical module has its own unique IP address. As the IPs are already different, all modules can share the same default ports:
+
+* 1952 - Default Module TCP Control Port
+* 1953 - Default Module TCP Stop port
+
+.. code-block:: bash  
+
+    # Therefore, one can use 
+    hostname bchip100+bchip101+
+    # instead of 
+    hostname bchip100:1952+bchip101:1954+
+
+
+**Simulators**, however, usually run on the same PC. See `virtual servers <https://slsdetectorgroup.github.io/devdoc/virtualserver.html>`_ for more details. In that case, each instance must use a different port. By incrementing port numbers, you can also use the virtual command for convenience.
+
+.. code-block:: bash  
+
+    # Therefore, one can use 
+    virtual 2 1952
+    # instead of 
+    hostname localhost:1952+localhost:1954+
+
+
+
+
+Client to Receiver
+--------------------
+
+.. figure:: images/Client_receiver_commands.png
+   :target: _images/Client_receiver_commands.png
+   :align: center
+   :alt: Client Receiver Commands
+
+   Client Receiver TCP Commands
+
+Each module has a receiver, which can be either local or remote.
+
+The client can configure and control receivers via the 1 GbE public TCP interface:
+
+* Should be able to ping the receiver's hostname from any PC on the network.
+* If one cannot ping, ensure that it is powered on.
+* If the command cannot connect to the port (`rx_hostname command <commandline.html#term-rx_hostname-hostname-or-ip-address>`_ failed), the receivers may not have started yet.
+
+
+Since multiple receiver processes typically run on the same PC, they share the same IP. Here, each receiver process must use a different TCP port for a unique connection. 
+
+* 1954 - Default Receiver TCP port
+
+Configuring the receiver with the command `rx_hostname command <commandline.html#term-rx_hostname-hostname-or-ip-address>`_, sets up a receiver for every module in shared memory automatically ie. every module's receiver TCP port will automatically increment in shared memory. The starting port is defined by the command `rx_tcpport <commandline.html#term-rx_tcpport-port>`_ with the default being 1954.
+
+
+A multi-module command (without colon or module index) sets incremental ports starting from the specified port number.
+
+.. code-block:: bash  
+
+    hostname bchip100+bchip101+bchip102+bchip103+
+    rx_tcport 2000 # sets the receiver port to 2000, 2001, 2002, 2003
+
+
+For example, using default TCP ports (1954, 1955):
+
+.. code-block:: bash  
+
+    hostname bchip100+bchip101+
+    rx_hostname localhost
+    # Equivalent to:
+    rx_hostname localhost:1954+localhost:1955+
+    
+    # or set to another set of ports (automatically incremented)
+    rx_tcpport 1984
+    rx_hostname localhost
+    # instead of
+    rx_hostname localhost:1984+localhost:1985+
+
+
+
+
+Module to Receiver
+-------------------
+
+.. figure:: images/Module_receiver_commands.png
+   :target: _images/Module_receiver_commands.png
+   :align: center
+   :alt: Module Receiver Commands
+
+   Module Receiver UDP Commands
+
+**10GbE Interface**
+
+The module typically sends images to the receiver via a 10 GbE private interface on the receiver PC, which has an MTU of 9000 to support jumbo packets. The private interface is not reachable from other machines, so it cannot be pinged from anywhere. 
+
+**Multiple UDP Packets**
+
+Images are split into UDP packets for transmission. Unlike TCP, UDP is connectionless and does not guarantee delivery. Therefore, the receiver PC must be tuned for reliable reception. See `Troubleshooting <https://slsdetectorgroup.github.io/devdoc/troubleshooting.html>`_.
+
+**UDP Configuration**
+
+Unlike TCP, the module (hardware) requires explicit configuration for sending images via UDP, including:
+
+* Source and destination IPs
+* Source and destination MAC addresses
+* Source and destination ports
+
+**UDP Destination**
+
+Info on where to send the image from the module to.
+
+**UDP Desination IP** - The IP of the receiver PC's 10 GbE interface, usually found via ``ifconfig``. Command: `udp_dstip <commandline.html#term-udp_dstip-x.x.x.x-or-auto>`_. For 1GbE interface and for this command, one can use 'auto' as an argument, which will pick up the IP from the `rx_hostname command <commandline.html#term-rx_hostname-hostname-or-ip-address>`_.
+
+**UDP desintation MAC** - Also obtained from the interface using ``ifconfig``. For built-in receivers, the module configures this automatically from the ``UDP destination IP``. For custom receivers, it must be explicitly provided. Command: `udp_dstmac <commandline.html#term-udp_dstmac-x-x-x-x-x-x>`_
+
+**UDP destination port** - Ensure uniqueness if multiple users share the interface. Command: `udp_dstport <commandline.html#term-udp_dstport-n>`_
+
+* 50001 - Default Receiver UDP port
+
+
+**UDP Source**
+
+As it is a one-way communication (module to receiver with no reply or acknowledgements), info on the source of the image is more for debugging purposes and prevent packet rejection.
+
+**UDP source IP** - Must be on the same subnet as the destination IP (same first three octets) to prevent packet rejection by the receiver interface. For 1GbE interface and for this command (except for Eiger), one can use ``auto`` as an argument, which will pick up the IP from the `hostname command <commandline.html#term-hostname>`_. Command: `udp_srcip <commandline.html#term-udp_srcip-x.x.x.x-or-auto>`_
+
+.. code-block:: bash  
+
+    # 10 GbE interface
+    hostname bchip100
+    udp_dstip 10.0.2.1
+    udp_srcip 10.0.2.19
+    rx_hostname localhost
+
+    # 1 GbE interface
+    hostname bchip100
+    rx_hostname localhost
+    udp_dstip auto # this command uses IP from rx_hostname. So, it comes after.
+    udp_srcip auto # this command uses IP from hostname
+
+**UDP source MAC** - By default, it is set to ``aa:bb:cc:dd:xx:yy`` where ``xx`` and ``yy`` are module row and column indices to differentiate the modules while debugging. Command: `udp_srcmac <commandline.html#term-udp_srcmac-x-x-x-x-x-x>`_
+
+
+**UDP source port** - This is hardcoded in every module to the same value in the detector server and cannot be changed.
+
+Note: If there is a second UDP port on the module, use 'udp_dstport2' or 'udp_dstip2'etc. See `here <https://slsdetectorgroup.github.io/devdoc/dataformat.html>`_ for more detector specific info. 
+
+Receiver to GUI
+-----------------
+
+.. figure:: images/Receiver_gui_commands.png
+   :target: _images/Receiver_gui_commands.png
+   :align: center
+   :alt: Receiver GUI Commands
+
+   Receiver GUI Commands
+
+
+Enabling the GUI automatically streams images from the receiver via ZMQ sockets. Even without the GUI, streaming can be activated explicitly using the command `rx_zmqstream <commandline.html#term-rx_zmqstream-0-1>`_. ZMQ streaming uses TCP/IP, so the ports must be configured appropriately.
+
+**Receiver ZMQ Port** - Port from which the receiver streams ZMQ packets. Command: `rx_zmqport <commandline.html#term-rx_zmqport-port>`_
+
+* 30001 - Default Receiver ZMQ Port (stream out from)
+
+**Client ZMQ Port** - Port that the client ZMQ socket listens to. Command: `zmqport <commandline.html#term-zmqport-port>`_
+
+* 30001 - Default Client ZMQ Port (listens to)
+
+**Client ZMQ IP** - IP address the client ZMQ socket listens to. Command: `zmqip <commandline.html#term-zmqip-x.x.x.x>`_. By default, this is set to the IP of ``rx_hostname``, but can be set to any IP address that the client can reach.
+
+* Default: Receiver’s hostname (rx_hostname)
+
+
+
+
+
+Receiver to External Processing
+--------------------------------
+
+.. figure:: images/Receiver_external_process_commands.png
+   :target: _images/Receiver_external_process_commands.png
+   :align: center
+   :alt: Click to zoom
+
+   Receiver External Process Commands
+
+Images from the receiver can also be streamed to an external processing chain for further processing or storage. In this setup:
+
+* The external processor listens to the ZMQ ports and IPs that the receiver streams from.
+
+* The client ZMQ sockets now listen to the ports and IPs that the external processor streams from instead of the receiver.
+
+**Receiver ZMQ Port** - Port from which the receiver streams ZMQ packets. Command: `rx_zmqport <commandline.html#term-rx_zmqport-port>`_
+
+* 30001 - Default Receiver ZMQ Port (stream out from)
+
+**Client ZMQ Port** - Port that the client ZMQ socket listens to. Command: `zmqport <commandline.html#term-zmqport-port>`_. In this set up, it should listen to the zmq port that the external process is streaming out from.
+
+* 30001 - Default Client ZMQ Port (listens to)
+
+**Client ZMQ IP** - IP address the client ZMQ socket listens to. Command: `zmqip <commandline.html#term-zmqip-x.x.x.x>`_. By default, this is set to the IP of ``rx_hostname``, but in this set up, it should listen to the zmq IP that the external process is streaming out from.
+
+* Default: Receiver’s hostname (rx_hostname)

docs/src/dataformat.rst

@@ -1,6 +1,6 @@
 .. _data format:
 
-Data Format
+Detector Image Size and Format
 ================================
 
 Each UDP port creates its own output file, which contains the data of the image transmitted over that port. More on number of files and naming for each file in the `File format <fileformat.html>`_ section.

docs/src/fileformat.rst

@@ -62,3 +62,14 @@ There are 2 file formats supported by the receiver:
 
 The default is binary. HDF5 can be enabled by compiling the package with HDF5 option enabled. The file format is set using the command `fformat <commandline.html#term-fformat-binary-hdf5>`_.
 
+
+Content
+--------
+
+| `Master File`: Contains the metadata of the acquisition. The content is described in :ref:`master file attributes <master file attributes>`.
+
+
+| `Data File`: Contains the metadata for each image (:ref:`slsReceiverHeader<sls receiver header format>`) and the `data of the image` transmitted over that port. The image data is described in :ref:`Detector Image Size and Format <data format>` section.
+
+
+| More content and examples are found in the :ref:`HDF5 file format <hdf5 file format>` and :ref:`Binary file format <binary file format>` sections.

docs/src/firmware.rst

@@ -1,3 +1,5 @@
+.. _firmware upgrade:
+
 Firmware Upgrade
 =================
 

docs/src/hdf5fileformat.rst

@@ -1,4 +1,4 @@
-
+.. _hdf5 file format:
 
 HDF5 File Format
 ================================
@@ -76,6 +76,8 @@ Data File
 
 * File Name: [fpath]/[fname]_dx_fy_[findex].h5 :ref:`Details here<file name format>`
 
+* More details on :ref:`slsReceiverHeader<sls receiver header format>` and the actual image data is described in the :ref:`Detector Image Size and Format <data format>` section.
+
 
 Virtual Data File
 ------------------

docs/src/index.rst

@@ -18,6 +18,17 @@ Welcome to slsDetectorPackage's documentation!
     installation
     dependencies
     consuming
+
+
+.. toctree::
+    :caption: how to
+    :maxdepth: 2
+
+    softwarearchitecture
+    configcommands
+    quick_start_guide
+    dataformat
+    multidet
    
 .. toctree::
     :caption: C++ API
@@ -28,12 +39,8 @@ Welcome to slsDetectorPackage's documentation!
     receiver_api
     examples
 
-.. toctree::
-    :caption: how to
-    :maxdepth: 2
 
-    multidet
+
-    quick_start_guide
     
 .. toctree::
     :caption: Python API
@@ -96,9 +103,9 @@ Welcome to slsDetectorPackage's documentation!
     :caption: Receiver Files
     :maxdepth: 3
 
-    dataformat
     fileformat
     slsreceiverheaderformat
+    dataformat
     masterfileattributes
     binaryfileformat
     hdf5fileformat

docs/src/multidet.rst

@@ -1,3 +1,5 @@
+.. _using multiple detectors:
+
 Using multiple detectors
 ==========================
 

docs/src/quick_start_guide.rst

@@ -140,7 +140,7 @@ Data
 
 Check out the :ref:`UDP header section<detector udp header>` for details on output UDP header data format.`
 
-Check out the :ref:`Data format section<data format>`  for details on output data format.
+Check out the :ref:`Detector image size and format section<data format>`  for details on output data format.
 
 
 Receiver

docs/src/servers.rst

@@ -1,3 +1,5 @@
+.. _detector_servers:
+
 Getting Started
 ===============
 

docs/src/softwarearchitecture.rst

@@ -0,0 +1,147 @@
+.. _software architecture:
+
+Software Architecture
+================================
+
+
+Introduction
+------------------------------------
+
+.. figure:: images/System_communication_architecture.png
+   :target: _images/System_communication_architecture.png
+   :width: 700px
+   :align: center
+   :alt: System communication architecture
+
+   Software Communication Architecture
+
+
+**Detector** 
+
+A detector can consist of a single module or multiple modules combined.
+
+**Module** 
+
+Each module sends its data via UDP over distinct ports. Since UDP does not provide acknowledgements, data is transmitted as fast as possible, which can lead to packet loss if the network is not properly configured, among other causes. A single image streamed out could be split into multiple UDP packets and each module can have one or two UDP ports to transmit in parallel different physical sections of the image.
+
+**Receiver** 
+
+UDP data is received by one or more receivers—either built-in or custom. In the diagram above, there is one built-in receiver per module (1:1). For example, a detector with two modules (two hostnames) will have two built-in receivers. Each receiver could listen to one or two UDP ports (depending on the module it listens to). For each UDP port, the receiver reassembles these packets into sub-images and optionally saved to file.
+
+**ZMQ** 
+
+Each UDP port in the receiver can also stream out independently sub-images via ZMQ (core: TCP/IP).
+    
+* Directly to the GUI for display.
+* To an external processing chain for post-processing and optional storage, which can in turn stream the processed data back to the GUI.
+
+**Client** 
+
+A single client can configure and control individual modules and receivers, or multiple of them in parallel. This communication is handled over TCP/IP, ensuring acknowledgements. 
+
+It can also listen to multiple ZMQ sockets from the Receiver(s) or the external processing chain to assemble the full image for GUI display or Client call backs.
+
+Next, each component is examined in detail.
+
+Module
+-------
+
+.. figure:: images/Module_architecture.png
+   :target: _images/Module_architecture.png
+   :width: 700px
+   :align: center
+   :alt: Module architecture
+
+   Module Architecture
+
+
+**Detector Server**
+
+The module contains an onboard CPU (type depends on the detector — e.g., Nios for Mythen3, Blackfin for Jungfrau). The detector server and detector configuration files are stored here, with the server compiled in C using the CPU-specific compiler. Running the binary starts a Control Server and a Stop Server. Client control/configuration requests go to the Control Server via the TCP control port, while stop/status requests go to the Stop Server via the TCP stop port as the Control Server may be busy with an acquisition. For more details see :ref:`detector server <detector_servers>` and :ref:`detector simulators<Virtual Detector Servers>` to play around with.
+
+**Firmware**
+
+The module also includes an FPGA with VHDL firmware (file format depends on the detector — e.g., Mythen3 uses .rbf, Jungfrau uses .pof). Client requests trigger register read/write operations in the FPGA, which manages chip readout and processing. Data from the chips is sent through a UDP generator in the FPGA and output as UDP packets via the UDP port. A single image may be split across multiple packets. A module could have 1 or 2 UDP ports to transmit in parallel different physical sections of the image.
+
+Upgrade
+^^^^^^^^
+
+.. figure:: images/Soft_upgrade_components.png
+   :target: _images/Soft_upgrade_components.png
+   :width: 700px
+   :align: center
+   :alt: Software Upgrade Components
+
+   Software Upgrade Components
+
+There are mainly three components to the soft upgrade:
+
+* Detector Server upgrade: The server running on the module.
+* Firmware upgrade: The VHDL code running on the FPGA.
+* slsDetectorPackage upgrade: The client code running on the host PC to control the module(s) and receiver(s) if any.
+
+Please use the `update command <commandline.html#term-update>`_ when updating both the server and firmware simulataneously and `programfpga command <commandline.html#term-programfpga-fname.pof-fname.rbf-full-path-opitonal-force-delete-normal-file>`_ when only updating the firmware. See :ref:`firmware upgrade <firmware upgrade>` for details.
+
+When only updating the detector server, use the `updatedetectorserver command <commandline.html#term-updatedetectorserver-server_name-with-full-path>`_ command. See :ref:`detector server upgrade <Detector Server Upgrade>` for details.
+
+.. note::
+   
+   **Compatibility**
+
+   When updating anything on the module via the client (server or firmware), the server and client will have to be compatible (same major version). If not, the client and server will not communicate properly.
+
+   Since they are ideally compatible before the upgrade, upgrade the server and firmware first, then the slsDetectorPackage.
+
+
+
+Receiver
+--------
+
+.. figure:: images/Receiver_architecture.png
+   :target: _images/Receiver_architecture.png
+   :align: center
+   :alt: Receiver Architecture
+
+   Receiver Architecture
+
+The receiver mainly consists of:
+
+* A TCP server that listens to client TCP requests for configuration and control.
+* One or 2 listeners that listen to a UDP port each, reassembling the UDP packets into sub-images in memory.
+* One or 2 data processors that processes the sub-images with optional callbacks for online processing and file writing.
+* One or 2 data streamers that stream the processed sub-images to the GUI or external processing chain via ZMQ.
+
+Few characteristics of the receiver:
+
+* It can be run on the same host as the client or on a different host.
+* There is a receiver process for every module and a file for every UDP port. 
+* Each receiver process is independent and asynchronized for performance. So are the UDP ports.
+
+
+Client
+--------
+
+.. figure:: images/Client_architecture.png
+   :target: _images/Client_architecture.png
+   :align: center
+   :alt: Client Architecture
+
+   Client Architecture
+
+Users can control the detector and receivers through four interfaces: 
+
+* their C++ API, 
+* their Python API, 
+* the command-line interface, or 
+* the Qt-based GUI. 
+
+Regardless of the interface, each ultimately invokes our Detector class—either directly (CLI and GUI) or through our C++/Python libraries (when using their APIs). The Detector class then calls the appropriate module functions, either for a specific module or in parallel for all modules. Each module object sends requests over TCP to its corresponding module and, if needed, to the receiver.
+
+**Shared Memory**
+
+As the command-line interface is supported, shared memory is used to store essential information such as the module hostname and TCP port, or the receiver hostname and TCP port. This ensures the system knows which components to communicate with, without requiring the user to re-enter this information for every command-line call.
+
+.. note::
+   
+   Only the client maintains shared memory. Care must be taken when multiple users operate from the same PC. See :ref:`multi detector and user section <using multiple detectors>` for more details.
+

docs/src/udpheader.rst

@@ -99,6 +99,7 @@ Detector Enum
     MYTHEN3             6
     GOTTHARD2           7    
     ================    ========
+    
     * deprecated since v10.0.0
 
 

python/slsdet/detector.py

@@ -2184,6 +2184,7 @@ class Detector(CppDetectorApi):
 
         :setter: It loads trim files from settingspath.\n [Mythen3] An energy of -1 will pick up values from detector.
         """
+        
         if self.type == detectorType.MYTHEN3:
             return self.getAllThresholdEnergy()
         return self.getThresholdEnergy()