Instructions for using the BEC testbench at uXAS
Setting up the environment
Login
As the BEC requires modern libraries only available on RHEL8 and the X05LA beamline consoles are ancient, the microXAS BEC installation runs on the x05la-bec-001 virtual console. It is set up as a special beamline console with the AWI software stack and is administered by AWI. All BEC services are set up such that the gac-x05la account can administer all services. As any beamline console it's part of the sls-bl-x05la beamline network, so you have to go through the SSH gateway by:
ssh -J x05la-gw gac-x05la@x05la-bec-001
PSI's virtual terminal service nx-term.psi.ch offers a convenient way to have a linux desktop from your browser.
Monitoring BEC services
Once logged in to the x05la-bec-001 host, you can attach to the console of the BEC services by:
tmux attach -t bec
You can restart services by clicking on their sub-terminal and pressing Ctrl + C (sometimes twice). This is needed whenever you edit the scan or device files.
Activating the BEC environment
In order to start the BEC console interface, you first need to activate the corresponding venv. This will also start the BEC GUI in a separate process and window:
source /data/test/x05la-test-bec/bec_deployment/bev_venv/bin/activate
bec
The BEC deployment
The BEC is deployed to a beamline specific network mounted folder that contains the main repositories for the current installation, you can check it by:
[gac-x05la@x05la-bec-001 ~]$ ls -ltr /data/test/x05la-test-bec/bec_deployment/
total 4
drwxr-xr-x 7 gac-x05la unx-sls 4096 Aug 20 15:10 ophyd_devices
drwxr-xr-x 8 gac-x05la unx-sls 4096 Aug 20 15:10 bec_widgets
drwxr-xr-x 6 gac-x05la unx-sls 4096 Aug 20 15:11 bec_venv
-rw-r--r-- 1 gac-x05la unx-sls 326 Aug 20 15:15 bec_config.yaml
-rw-r--r-- 1 gac-x05la unx-sls 289 Aug 20 15:15 bec_client_config.yaml
drwxr-xr-x 15 gac-x05la unx-sls 4096 Aug 29 13:06 bec
drwxr-xr-x 7 gac-x05la unx-sls 4096 Sep 3 13:28 tomcat_bec
drwxr-xr-x 2 gac-x05la unx-sls 4096 Sep 6 11:22 logs
A short summary of the folder structure:
- bec_venv : The actual installation directory of the BEC framework
- bec : Git repo for the BEC framework
- bec_widgets : Git repo for the BEC widgets
- ophyd_devices : Git repo as installation directory for shared ophyd devices
- tomcat_bec : Git repo as installation directory for Tomcat specific scans, devices and configs.
In short: You should use the bec_venv and only edit the tomcat_bec folder. And this is easily done in Visual Studio Code, a really handy IDE if you have a fast connection:
code &
The Tomcat BEC repository
The Tomcat BEC repository is a collection of Tomcat specific ophyd devices, scans, scripts and device configurations. It tries to enforce a standard structure across beamlines and it's maintenance is the responsibility of the beamline. I.e. it should be handed over from controls while still keeping the basic structure:
[gac-x05la@x05la-bec-001 ~]$ ls -ltr /data/test/x05la-test-bec/bec_deployment/tomcat_bec/tomcat_bec/
total 6
-rw-r--r-- 1 gac-x05la unx-sls 0 Jan 1 1970 __init__.py
drwxr-xr-x 2 gac-x05la unx-sls 4096 Aug 20 15:10 file_writer
drwxr-xr-x 2 gac-x05la unx-sls 4096 Aug 20 15:10 services
drwxr-xr-x 2 gac-x05la unx-sls 4096 Aug 20 15:15 __pycache__
drwxr-xr-x 4 gac-x05la unx-sls 4096 Aug 20 15:15 deployments
drwxr-xr-x 6 gac-x05la unx-sls 4096 Aug 21 11:11 bec_ipython_client
drwxr-xr-x 4 gac-x05la unx-sls 4096 Aug 21 17:52 bec_widgets
drwxr-xr-x 2 gac-x05la unx-sls 4096 Aug 29 09:43 logs
drwxr-xr-x 3 gac-x05la unx-sls 4096 Sep 3 13:28 device_configs
drwxr-xr-x 7 gac-x05la unx-sls 4096 Sep 5 18:08 devices
drwxr-xr-x 3 gac-x05la unx-sls 4096 Sep 16 12:30 scans
-rw-r--r-- 1 gac-x05la unx-nogroup 3383 Oct 1 16:08 Readme.md
From the list of folders the most important ones are:
- device_configs : Ophyd device datbase as YAML file
- devices : Ophyd device repository (i.e. Aerotech, Gigafrost)
- scans : Tomcat specific base scans in the low-level API
- scripts : Scripts and scans using the high level API
The official location of the beamline repository is:
https://gitlab.psi.ch/bec/tomcat_bec
Configuration and basic diagnostic in BEC
Once you're at the CLI it is good practice to check the available devices:
dev.show_all()
If the previous two configuration attempts were unsuccesfull the bec_device_server will start without any devices. A single missing or mistyped IOC will cause a configuration failure, so once you fixed you configs you can reload your YAML config file:
bec.config.update_session_with_file("/data/test/x05la-test-bec/bec_deployment/tomcat_bec/tomcat_bec/device_configs/microxas_test_bed.yaml")
If a single, unnecesary device doesn't come up , just comment it out.
Early tests with GigaFrost and Aerotech
The main goal of the testbench is to test the various infrastructure component foreseen at the Tomcat beamlines and ensure their collaboration. Namely the rotation stage with the Aerotech smart controller, the GigaFrost camera and the Standard-DAQ recording system. The individual repositories have more documentation on the components, but as a short summary, the Aerotech Automation1 IOC is a recently developed EPICS driver, the GigaFrost IOC is an ancient offshoot of Helge's camera driver and the std-DAQ is a non-EPICS device that is controlled via a hybrid websocket and REST interface. The DAQ also provides preview streams with hardcoded throttling.
Note that the standard bluesky event model applies, i.e. configure sets up the device, stage and trigger for stepping interface and kickoff and collect for flyers. This will be updated with the recently updated event model. Redundant staging raises an exception.
Seeing is believing (GUI)
Since seeing is believing, it's often a good idea to bring up some panels and get visual feedback. You can start the standard CaQtDM panels for the Aerotech and GigaFrost from any beamline console running the contros stack by:
caqtdm -macro "CAM=X02DA-CAM-GF2" X_X02DA_GIGAFROST_camControl_user.ui
caqtdm -macro "P=X02DA-ES1-SMP1" aeroauto1_ControllerOverview.ui
The std-DAQ doesn't have control panels, but the BEC can display the hard-coded preview streams. The daq_stream1 preview forwards displays 1 frames at a maximum of 5 Hz. So we just need to add the correct widgets to the GUI:
gui.add_dock('tomolive')
gui.panels['tomolive'].add_widget('BECFigure').image('daq_stream1')
Short run with the GigaFrost and std-DAQ
This is a short, recorded test run with 2016x2016 frames at 100 Hz with the GigaFrost. The current GigaFrost ophyd device includes the std-DAQ client, but they will probably be separated.
# Configure
d = {'ntotal': 1000000, 'nimages': 555, 'exposure': 5, 'period': 10}
dev.gfclient.configure(d)
# Stage GF, DAQ and preview
dev.gfclient.stage()
dev.daq_stream0.stage()
# Trigger a few sequences
def.gfclient.trigger()
def.gfclient.trigger()
# Unstage
def.gfclient.unstage()
dev.daq_stream0.unstage()
Reseting the Automation1 iSMC
There's some electrical disturbance at SLS that can bring the system to a problematic state. A workaround is to remotely reset the iSMC:
dev.es1_ismc.reset()
FIXME: There is a known desync in PSO inputs upon iSMC restart that requires restarting the IOC.
Scans at the testbench
The current repository has several implemented scans, often offering different solutions to the same problem using different APIs while providing similar functionality. The BEC itself has two scan APIs for low- and highlevel scans, and the fastest scans can be directly implemented on the Automation1 iSMC. The implemented BEC scans combine motor motion, PSO triggering, Gigafrost and stdDAQ control and position capture in internal or external triggering mode. They were intentionally designed to provide the same interface, althogh not every combination was tested.
As I was unsure about the various triggering modes of the Gigafrost, so currently it's implemented with only sofware trigger that is compatible with the standard bluesky step scanning interface.
demostepscan
Simple step scan in the scripting API that performs the device configuration and hands over to a standard BEC scan routine. Note that there's absolutely no readback from the GigaFrost so we don't know when it finished acquiring.
def demostepscan(scan_start, scan_end, steps, exp_time=0.005, burst_at_each_point=1, settling_time=0, sync='event')
gigastep
Simple step scan in the low level API that performs the device configuration loops over each position in BEC and reads back the position data. Note that there's absolutely no readback from the GigaFrost so we don't know when it finished acquiring.
scans.gigastep(scan_start, scan_end, steps, exp_time=0.005, settling_time=0.2, burst_at_each_point = 1, sync='event')
snapnstep
Scripted step scan that sets up all components, creates an AeroScript file from template and executes it on the Automation1 iSMC controller. Upon succesfull completion it reads back the positions from the data capture.
NOTE: This scan just launches the GigaFrost, but hardware triggering is currently not implemented.
scans.snapnstep(scan_start, scan_end, steps, exp_time=0.005, settling_time=0.2, burst_at_each_point = 1, sync='event')
sequencescan
Scripted scan that sets up all components, creates an AeroScript file from template and executes it on the Automation1 iSMC controller. Upon succesfull completion it reads back the positions from the data capture. Position readback works wth both internal and external triggers.
NOTE: This scan just launches the GigaFrost, but hardware triggering is currently not implemented.
scans.sequencescan(scan_start, gate_high, gate_low, repeats=1, repmode="PosNeg", exp_time=0.005, exp_frames=180, roix=2016, roiy=2016, sync="pso")
demosequencescan
Basic sequence scan in the high level API that calls the corresponding low-level scan with some extra checks.
demosequencescan(scan_start, gate_high, gate_low, repeats=1, repmode="PosNeg", exp_time=0.005, exp_frames=180, roix=2016, roiy=2016, sync="pso")
becsequencescan
Basic sequence scan script in BEC's high level interface. It a performs all trigger configurations movements and readback from BEC.
becsequencescan(scan_start, gate_high, gate_low, repeats=1, repmode="PosNeg", exp_time=0.005, exp_frames=180, roix=2016, roiy=2016, sync="pso")