diff --git a/A-chat-about-saving-data.md b/A-chat-about-saving-data.md index 4cac10d..829b932 100644 --- a/A-chat-about-saving-data.md +++ b/A-chat-about-saving-data.md @@ -1,12 +1,52 @@ +# **How to Save Data at SwissFEL: A Totally Serious Guide** + **Hey, look at my nice data!** -What a load of bs! +*What a load of BS!* + **Oy, no need to be rude!** -I'm not! I mean, what a load of beam synchronous data! -**Well, thanks, but it is not all saved** -Bummer, how were you saving it? -**I used this cool command line script bs...** -I'm going to stop you there +*I’m not! I mean, what a load of **Beam Synchronous** data!* + +**Well, thanks... but it’s not all saved.** +*Bummer. How were you saving it?* + +**I used this cool command-line script `bs...`** +*Ooooh... I’m going to stop you right there.* + **But my BS command works; there must be a problem with the sources!** -Try each one individually with the bs command.. -**OK, hang on, what the! They work, the both save individually!** -Yes the issue is when you try and save sources from different IOCs/devices with the bs command +*Calm down, detective, try saving each source individually with the `bs` command...* + +**OK, hang on... what the?! They *both* save fine on their own!** +*Yep, the issue is when you try and save sources from different IOCs/devices with the `bs` command. The data is taken from the **dispatcher**. If the two sources don’t arrive at the dispatcher within a small time window, only the first source is sent in the message. Different sources arrive at the dispatcher at **different times**.* + +**What type of BS is that?! I can't wait and wait...** +*Good question. Some BS data comes from **pipelines**, where calculations and moving data around **takes time**. You’re not just saving numbers—you’re saving processed results.* + +**Pipelines?! I want data, not plumbing problems!** +*Think of pipelines as hardworking elves doing data analysis behind the scenes. No pipelines, more work for you.* + +**Alright, I’m sold. But how do I save multiple sources without all this drama?** +*You need to save from the **data buffer**. The system can handle sources arriving at slightly different times there.* + +**The data buffer? How?** +*You’ve got plenty of tools for accessing it:* +- **DataHub** (don't ask about a front-end) +- **Data API** (if you speak code) +- **Eco, Slic, Service Now, Concour, Time** (some might not work). + +**Steady your sources, Doc Brown, I just checked my data and one of the sources isn't running at 100 Hz and missing data, I told you the source was the problem** + +*Missing data isn't necessarily a problem; the approaches above can handle missing pulse IDs and return you all the data that is in the data buffer* + +**Pulse IDs? Does my data need to prove its age?** + +*No, its a way to sort data, with bs data every shot had a unique pulse ID. You can use SwissFEL data analysis packages to match arrays with missing shots in for you* + +**Cool! Anything else I should know?** +*Yes: the databuffer can’t clean your desk at PiA* + +**✅ Do Say:** +*Pulse IDs rock my world!* + +**❌ Don’t Say:** +*Beam synchronous PV* +