Update A chat about saving data

This commit is contained in:
arrell_c 2024-12-10 13:25:00 +01:00
parent d7c2c4d185
commit 0f68e4ee24

View File

@ -1,12 +1,52 @@
# **How to Save Data at SwissFEL: A Totally Serious Guide**
**Hey, look at my nice data!**
What a load of bs!
*What a load of BS!*
**Oy, no need to be rude!**
I'm not! I mean, what a load of beam synchronous data!
**Well, thanks, but it is not all saved**
Bummer, how were you saving it?
**I used this cool command line script bs...**
I'm going to stop you there
*Im not! I mean, what a load of **Beam Synchronous** data!*
**Well, thanks... but its not all saved.**
*Bummer. How were you saving it?*
**I used this cool command-line script `bs...`**
*Ooooh... Im going to stop you right there.*
**But my BS command works; there must be a problem with the sources!**
Try each one individually with the bs command..
**OK, hang on, what the! They work, the both save individually!**
Yes the issue is when you try and save sources from different IOCs/devices with the bs command
*Calm down, detective, try saving each source individually with the `bs` command...*
**OK, hang on... what the?! They *both* save fine on their own!**
*Yep, the issue is when you try and save sources from different IOCs/devices with the `bs` command. The data is taken from the **dispatcher**. If the two sources dont arrive at the dispatcher within a small time window, only the first source is sent in the message. Different sources arrive at the dispatcher at **different times**.*
**What type of BS is that?! I can't wait and wait...**
*Good question. Some BS data comes from **pipelines**, where calculations and moving data around **takes time**. Youre not just saving numbers—youre saving processed results.*
**Pipelines?! I want data, not plumbing problems!**
*Think of pipelines as hardworking elves doing data analysis behind the scenes. No pipelines, more work for you.*
**Alright, Im sold. But how do I save multiple sources without all this drama?**
*You need to save from the **data buffer**. The system can handle sources arriving at slightly different times there.*
**The data buffer? How?**
*Youve got plenty of tools for accessing it:*
- **DataHub** (don't ask about a front-end)
- **Data API** (if you speak code)
- **Eco, Slic, Service Now, Concour, Time** (some might not work).
**Steady your sources, Doc Brown, I just checked my data and one of the sources isn't running at 100 Hz and missing data, I told you the source was the problem**
*Missing data isn't necessarily a problem; the approaches above can handle missing pulse IDs and return you all the data that is in the data buffer*
**Pulse IDs? Does my data need to prove its age?**
*No, its a way to sort data, with bs data every shot had a unique pulse ID. You can use SwissFEL data analysis packages to match arrays with missing shots in for you*
**Cool! Anything else I should know?**
*Yes: the databuffer cant clean your desk at PiA*
**✅ Do Say:**
*Pulse IDs rock my world!*
**❌ Dont Say:**
*Beam synchronous PV*