mirror of
https://github.com/ivan-usov-org/bec.git
synced 2025-04-20 01:40:02 +02:00
docs(messaging): added first draft of bec messaging docs
This commit is contained in:
parent
cf619b2ad7
commit
f90f40f543
40
docs/source/assets/messaging_system.drawio
Normal file
40
docs/source/assets/messaging_system.drawio
Normal file
@ -0,0 +1,40 @@
|
|||||||
|
<mxfile host="Electron" modified="2024-07-23T15:44:59.284Z" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/24.6.4 Chrome/124.0.6367.207 Electron/30.0.6 Safari/537.36" etag="j0pUm-Lly6YRSlM1o_Rt" version="24.6.4" type="device">
|
||||||
|
<diagram name="Page-1" id="x2sBnHPqiDliDDTYoS2s">
|
||||||
|
<mxGraphModel dx="1242" dy="785" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
|
||||||
|
<root>
|
||||||
|
<mxCell id="0" />
|
||||||
|
<mxCell id="1" parent="0" />
|
||||||
|
<mxCell id="fJOnbpTbUHdrzFDHRs04-1" value="EndpointInfo" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="340" y="190" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="fJOnbpTbUHdrzFDHRs04-2" value="endpoint" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="210" y="120" width="110" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="fJOnbpTbUHdrzFDHRs04-5" value="message_op" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="345" y="120" width="110" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="fJOnbpTbUHdrzFDHRs04-6" value="message_type" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="480" y="120" width="110" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="fJOnbpTbUHdrzFDHRs04-7" value="" style="endArrow=classic;html=1;rounded=0;exitX=0;exitY=0.5;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" edge="1" parent="1" source="fJOnbpTbUHdrzFDHRs04-1" target="fJOnbpTbUHdrzFDHRs04-2">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="210" y="260" as="sourcePoint" />
|
||||||
|
<mxPoint x="260" y="210" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="fJOnbpTbUHdrzFDHRs04-8" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" edge="1" parent="1" source="fJOnbpTbUHdrzFDHRs04-1" target="fJOnbpTbUHdrzFDHRs04-5">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="350" y="230" as="sourcePoint" />
|
||||||
|
<mxPoint x="275" y="170" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="fJOnbpTbUHdrzFDHRs04-9" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" edge="1" parent="1" source="fJOnbpTbUHdrzFDHRs04-1" target="fJOnbpTbUHdrzFDHRs04-6">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="410" y="200" as="sourcePoint" />
|
||||||
|
<mxPoint x="410" y="170" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
</root>
|
||||||
|
</mxGraphModel>
|
||||||
|
</diagram>
|
||||||
|
</mxfile>
|
||||||
BIN
docs/source/assets/messaging_system.png
Normal file
BIN
docs/source/assets/messaging_system.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 12 KiB |
85
docs/source/developer/data_acess/bec_messaging.md
Normal file
85
docs/source/developer/data_acess/bec_messaging.md
Normal file
@ -0,0 +1,85 @@
|
|||||||
|
(developer.bec_messaging)=
|
||||||
|
# Introduction to the BEC messaging system
|
||||||
|
The BEC messaging system is the backbone of the BEC system and is responsible for the communication between the different components of the BEC system. The following sections provide an overview of its components: the Redis server, the Python-based Redis connector, the BECMessage classes and the EndpointInfo class.
|
||||||
|
|
||||||
|
Redis is an open-source, in-memory data structure store that can be used as a database, cache, and message broker. Every entry is associated with a unique channel name, e.g. `internal/devices/readback/samx`. Depending on the stored data, it supports various operations such as get, set, delete, publish/subscribe as well as stream operations and many more. To learn more about the core functionality of Redis, visit the [Redis website](https://redis.io/).
|
||||||
|
|
||||||
|
For storing and retrieving data in BEC, there are three building blocks to describe any communication:
|
||||||
|
- `RedisConnector`
|
||||||
|
A wrapper around the `redis-py` package to simplify the interaction with the Redis server.
|
||||||
|
- `BECMessage`
|
||||||
|
A set of classes that provide a uniform way to send and receive data irrespective of the data type.
|
||||||
|
- `EndpointInfo`
|
||||||
|
A class that provides a mapping between the Redis channel, the BECMessage class and the supported operations for the channel.
|
||||||
|
|
||||||
|
Sending a message to Redis typically involves using an already existing instance of the `RedisConnector` class, creating a new instance of a `BECMessage` class and sending it to the desired endpoint using corresponding method of the EndpointInfo class and the `RedisConnector` class.
|
||||||
|
The mapping between the Redis channel, the BECMessage class and the supported operations for the channel is provided by the `EndpointInfo` class:
|
||||||
|
|
||||||
|
```{figure} ../../assets/messaging_system.png
|
||||||
|
:name: messaging_system
|
||||||
|
:align: center
|
||||||
|
|
||||||
|
The EndpointInfo class provides a mapping between the endpoint in Redis, the supported operations of the redis channel and the message type, i.e. what kind of BECMessage class is used to generate the data.
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
````{dropdown} View tutorial: Example of a communication with Redis
|
||||||
|
:icon: code-square
|
||||||
|
:animate: fade-in-slide-down
|
||||||
|
|
||||||
|
```{note}
|
||||||
|
Please note that the following tutorial also includes the sending of messages to Redis. Under normal circumstances, the messages are sent by the BEC system itself and not by the user. This tutorial is intended to show how the BEC messaging system works and therefore includes the sending of messages.
|
||||||
|
```
|
||||||
|
|
||||||
|
Let's assume we want to send a new file event message to Redis to inform other services about a new file being created for the device "samx". To this end, we will use the `file_event` endpoint:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from bec_lib.endpoints import MessageEndpoints
|
||||||
|
|
||||||
|
MessageEndpoints.file_event("samx")
|
||||||
|
```
|
||||||
|
|
||||||
|
Executing the code above will return the following output:
|
||||||
|
|
||||||
|
```
|
||||||
|
• demo [11/10] ❯❯ MessageEndpoints.file_event("samx")
|
||||||
|
Out[11]: EndpointInfo(endpoint='public/file_event/samx', message_type=<class 'bec_lib.messages.FileMessage'>, message_op=<MessageOp.SET_PUBLISH: ['register', 'set_and_publish', 'delete', 'get', 'keys']>)
|
||||||
|
```
|
||||||
|
|
||||||
|
As we can see, the `file_event` endpoint is associated with the Redis channel `public/file_event/samx` and the message type `FileMessage`. The message operations supported by this endpoint are `register`, `set_and_publish`, `delete`, `get`, and `keys`.
|
||||||
|
|
||||||
|
In order to send a message to this endpoint, we need to create a new instance of the `FileMessage` class and send it to the Redis server using the `set_and_publish` operation:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from bec_lib.endpoints import MessageEndpoints
|
||||||
|
from bec_lib import messages
|
||||||
|
|
||||||
|
# create a new instance of a BECMessage class, e.g. a FileMessage
|
||||||
|
msg = messages.FileMessage(file_path='path/to/file', done=False, successful=True, metadata={'scan_id': "1234"})
|
||||||
|
|
||||||
|
bec.connector.set_and_publish(MessageEndpoints.file_event("samx"), msg)
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
Another service listening to the `public/file_event/samx` channel will receive the message and can act accordingly.
|
||||||
|
If we simply want to retrieve the last message from the `public/file_event/samx` channel, we can use the `get` operation:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from bec_lib.endpoints import MessageEndpoints
|
||||||
|
|
||||||
|
# get the last message from the 'public/file_event/samx' channel
|
||||||
|
out = bec.connector.get(MessageEndpoints.file_event("samx"))
|
||||||
|
out
|
||||||
|
```
|
||||||
|
|
||||||
|
````
|
||||||
|
|
||||||
|
## BECMessage classes
|
||||||
|
The BECMessage classes are used to create messages that can be sent to Redis. Upon initialization, a BECMessage class validates the input against the predefined schema and raises an error in case of a mismatch. This ensures that the data sent to Redis is always in the correct format. The BECMessage classes are defined in the `bec_lib.messages`:
|
||||||
|
|
||||||
|
````{dropdown} View code: BECMessage classes
|
||||||
|
```{literalinclude} ../../../../bec_lib/bec_lib/messages.py
|
||||||
|
```
|
||||||
|
````
|
||||||
|
|
||||||
|
Every instance of a BECMessage provides the attributes `content` and `metadata`, both returning dictionaries. The `content` attribute contains the actual data that is sent to Redis, while the `metadata` attribute contains additional information about the message.
|
||||||
Loading…
x
Reference in New Issue
Block a user