added upload_custom_dap_script endpoint

Reviewed-on: #32

.conda-recipe/meta.yaml

@@ -11,6 +11,9 @@ about:
 source:
     path: ..
 
+build:
+  noarch: python
+
 requirements:
     build:
         - python

.github/workflows/condapackage.yml

@@ -1,6 +1,7 @@
 name: Conda Package
 
 on:
+  workflow_dispatch: # allow manual triggering from the UI
   push:
     tags:
       - '*'
@@ -14,18 +15,21 @@ jobs:
 
     - name: Prepare
       run: |
+        $CONDA/bin/conda config --set report_errors false
         $CONDA/bin/conda config --set always_yes yes
         $CONDA/bin/conda config --set changeps1 no
         $CONDA/bin/conda config --set anaconda_upload yes
 
+        $CONDA/bin/conda config --append channels conda-forge
+        $CONDA/bin/conda config --append channels paulscherrerinstitute
+
+        $CONDA/bin/conda config --show-sources
+
         $CONDA/bin/conda install --quiet conda-libmamba-solver
         $CONDA/bin/conda config --set solver libmamba
 
         $CONDA/bin/conda install --quiet anaconda-client conda-build conda-verify
 
-        $CONDA/bin/conda config --append channels conda-forge
-        $CONDA/bin/conda config --append channels paulscherrerinstitute
-
         $CONDA/bin/conda info -a
 
     - name: Build and upload

.travis.yml

@@ -1,38 +0,0 @@
-language: python
-
-python:
-  - 3.6
-
-# Build only tagged commits
-if: tag IS present
-
-before_install:
-  - wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
-  - bash miniconda.sh -b -p $HOME/miniconda
-  - rm miniconda.sh # clean up here, so no warning is triggered during the final clean-up
-  - export PATH=$HOME/miniconda/bin:$PATH # if `source $HOME/miniconda/etc/profile.d/conda.sh` instead, anaconda is not found in deploy
-  - conda config --set always_yes yes
-  - conda config --set changeps1 no
-  - conda config --set anaconda_upload no
-  - conda config --append channels conda-forge
-  - conda config --append channels paulscherrerinstitute
-
-install:
-  - conda update -q conda
-  - conda install -q python=$TRAVIS_PYTHON_VERSION conda-build conda-verify anaconda-client
-  - conda info -a
-
-script:
-  - conda build .conda-recipe
-
-deploy:
-  provider: script
-  script: anaconda -t $ANACONDA_TOKEN upload $HOME/miniconda/conda-bld/**/slic-*.tar.bz2
-  on:
-    branch: master
-    tags: true
-
-notifications:
-  email: false
-
-

README.md

@@ -6,13 +6,13 @@ _slic_ is a re-write/re-factor of [_eco_](https://github.com/paulscherrerinstitu
 
 _slic_ consists of a [core](#sliccore) library for static recording and scans as well as a [devices](#slicdevices) library containing classes that represent physical devices. As work-in-progress, the core library has seen most changes so far while the devices have not been worked on as much. Furthermore, there's a [GUI](#slicgui) frontend built on top of _slic_ as backend included.
 
-The beamline codes can be found [here](https://gitlab.psi.ch/slic):
+The beamline codes can be found [here](https://gitea.psi.ch/slic):
 
-- [Alvra](https://gitlab.psi.ch/slic/alvra)
-- [Bernina](https://gitlab.psi.ch/slic/bernina)
-- [Cristallina](https://gitlab.psi.ch/slic/cristallina)
-- [Maloja](https://gitlab.psi.ch/slic/maloja)
-- [Furka](https://gitlab.psi.ch/slic/furka)
+- [Alvra](https://gitea.psi.ch/slic/alvra)
+- [Bernina](https://gitea.psi.ch/slic/bernina)
+- [Cristallina](https://gitea.psi.ch/slic/cristallina)
+- [Maloja](https://gitea.psi.ch/slic/maloja)
+- [Furka](https://gitea.psi.ch/slic/furka)
 
 Please click [here](READMORE.md) for some FAQs.
 
@@ -30,12 +30,12 @@ The core library contains
 - **adjustable** — ABC for physical/virtual devices that can be moved or otherwise adjusted. The `PVAdjustable` class handles interaction with the typical set of epics PV defining a device (set value, readback, moving status). A generic class is also provided, which turns a getter/setter pair into an adjustable.
 - **condition** — Classes that collect statistics over a given time window and test whether a value was in a specified range often enough. This allows to define what conditions are considered good enough for a recording.
 - **device** — Representation of larger hardware components that consist of several adjustables. Devices can also be nested allowing to represent, e.g., a whole beamline. `SimpleDevice` is a straight-forward interface for creating devices. The included collection of device implementations can be found in [`slic.devices`](#slic.devices).
-- **sensor** — ABC for physical/virtual devices that are read out and analysed. Recording is started and stopped, and an aggregation function is applied to the data collected in between. The default aggregation is averaging via `np.mean`. The `BSSensor` and `PVSensor` classes handle reading from a single bsread channel or epics PV, respectively. Several classes exist that allow combining multiple channels (e.g., `Norm`, `Combined`,  and their `BS` counterparts). Sensors may be used for live-plotting scans via [grum](https://gitlab.psi.ch/augustin_s/grum), i.e., by providing a `default_sensor` to `Scanner` or a `sensor` to a specific scan method like `scan1d`.
+- **sensor** — ABC for physical/virtual devices that are read out and analysed. Recording is started and stopped, and an aggregation function is applied to the data collected in between. The default aggregation is averaging via `np.mean`. The `BSSensor` and `PVSensor` classes handle reading from a single bsread channel or epics PV, respectively. Several classes exist that allow combining multiple channels (e.g., `Norm`, `Combined`,  and their `BS` counterparts). Sensors may be used for live-plotting scans via [grum](https://gitea.psi.ch/SwissFEL/grum), i.e., by providing a `default_sensor` to `Scanner` or a `sensor` to a specific scan method like `scan1d`.
 - **task** — Simplifying wrappers for python's [threading.Thread](https://docs.python.org/3/library/threading.html#threading.Thread), which allow return values and forward exceptions raised within a thread to the calling scope. A nicer `__repr__` makes tasks easier to use in ipython. More specific tasks are also available: the DAQTask can hold information about the files it is writing, and the Loop comes in two variants (infinite and with time out) that both call a function repeatedly.
 
 ### Overview: Interactions of these building blocks:
 
-<img src="https://gitlab.psi.ch/slic/slic/-/wikis/uploads/a1234d21f423ee8f072b28e108f9792b/drawing.png" width="50%" />
+<img src="https://gitea.psi.ch/slic/slic/wiki/raw/uploads%2Fa1234d21f423ee8f072b28e108f9792b%2Fdrawing.png" width="50%" />
 
 
 ## slic.devices
@@ -51,9 +51,9 @@ One of the goals of the "library-first" approach of _slic_ is to provide means f
 
 _slic_ comes with an example GUI (written in [wxPython](https://wxpython.org/)) built on top:
 
-<img src="https://gitlab.psi.ch/slic/slic/-/wikis/uploads/c8d3dfeb2d159b18c2a97db6793442cd/config.png" width="33%" />
-<img src="https://gitlab.psi.ch/slic/slic/-/wikis/uploads/0c58450f1e18d134910e786fe1c33f65/scan.png"   width="33%" />
-<img src="https://gitlab.psi.ch/slic/slic/-/wikis/uploads/75845df3a2e3e7019f048c37b3cf7684/tweak.png"  width="33%" />
+<img src="https://gitea.psi.ch/slic/slic/wiki/raw/uploads%2Fc8d3dfeb2d159b18c2a97db6793442cd%2Fconfig.png" width="33%" />
+<img src="https://gitea.psi.ch/slic/slic/wiki/raw/uploads%2F0c58450f1e18d134910e786fe1c33f65%2Fscan.png"   width="33%" />
+<img src="https://gitea.psi.ch/slic/slic/wiki/raw/uploads%2F75845df3a2e3e7019f048c37b3cf7684%2Ftweak.png"  width="33%" />
 
 In order to further the "disposable GUIs" concept, this GUI is very modular: Tabs can be enabled or disabled upon instantiation. Each tab interfaces a single feature of _slic_:
 
@@ -98,13 +98,13 @@ The beamline codes are hosted in git repositories and should be cloned (here, wi
 - either via https:
 
 ```bash
-git clone https://gitlab.psi.ch/slic/alvra.git
+git clone https://gitea.psi.ch/slic/alvra.git
 ```
 
 - or via ssh:
 
 ```bash
-git clone git@gitlab.psi.ch:slic/alvra.git
+git clone git@gitea.psi.ch:slic/alvra.git
 ```
 
 #### Running
@@ -147,22 +147,22 @@ conda env update --file conda-env.yml
 
 #### Installation
 
-The library and the beamline codes are stored in separate git repositories within a [gitlab group](https://gitlab.psi.ch/slic). This allows to easily check out only the desired parts.
+The library and the beamline codes are stored in separate git repositories within a [gitea group](https://gitea.psi.ch/slic). This allows to easily check out only the desired parts.
 
 For the most current code, both parts should be cloned (here, with Alvra as example, for other beamlines replace `alvra` with the respective name):
 
 - either via https:
 
 ```bash
-git clone https://gitlab.psi.ch/slic/slic.git
-git clone https://gitlab.psi.ch/slic/alvra.git
+git clone https://gitea.psi.ch/slic/slic.git
+git clone https://gitea.psi.ch/slic/alvra.git
 ```
 
 - or via ssh:
 
 ```bash
-git clone git@gitlab.psi.ch:slic/slic.git
-git clone git@gitlab.psi.ch:slic/alvra.git
+git clone git@gitea.psi.ch:slic/slic.git
+git clone git@gitea.psi.ch:slic/alvra.git
 ```
 
 #### Running

setup.py

@@ -2,8 +2,8 @@ from setuptools import setup, find_packages
 
 setup(
     name="slic",
-    version='0.1.0',
-    url="https://gitlab.psi.ch/slic/slic",
+    version='0.1.2',
+    url="https://gitea.psi.ch/slic/slic",
     description="SwissFEL Library for Instrument Control",
     author="Paul Scherrer Institute",
     packages=find_packages()

slic/__init__.py

@@ -1,3 +1,9 @@
+from passlib.hash import sha256_crypt as _ignore #TODO: this hides a DeprecationWarning for crypt / sha256_crypt is used in py_elog
+
+
+from slic.gui.wxdebug import wxdebug as _wxdebug
+_wxdebug()
+
 
 from . import core
 

slic/core/acquisition/__init__.py

@@ -8,6 +8,6 @@ from .pvacquisition import PVAcquisition
 from .bschannels import BSChannels
 from .pvchannels import PVChannels
 
-from .detcfg import DetectorConfig
+from .detcfg import DAPConfig, DetectorConfig, HardwareConfig
 
 

slic/core/acquisition/broker/brokerclient.py

@@ -1,12 +1,14 @@
-from time import sleep
+from time import sleep, time
 
-from slic.utils import xrange, tqdm_mod#, tqdm_sleep
+from slic.utils import forwards_to, readable_seconds, xrange, tqdm_mod#, tqdm_sleep
 
 from .restapi import RESTAPI
 from .brokerconfig import BrokerConfig, flatten_detectors
 from .pedestal import take_pedestal
 from .pids import align_pid_left, align_pid_right, aligned_pid_and_n
 from .tools import get_current_pulseid
+from .poweron import guided_power_on
+from .jfstatus import color_bar, header_bar
 
 
 class BrokerClient:
@@ -150,7 +152,7 @@ class BrokerClient:
         take_pedestal(self.restapi, self.config, detectors=detectors, rate=rate, pedestalmode=pedestalmode)
 
 
-    def power_on(self, detectors=None, **kwargs):
+    def power_on(self, detectors=None, wait=False, wait_time=0.1, timeout=300, **kwargs):
         if detectors is None:
             detectors = self.config.detectors
 
@@ -162,5 +164,57 @@ class BrokerClient:
             msg = self.restapi.power_on_detector(d, **kwargs)
             print(f"{d}: {msg}")
 
+        if not wait:
+            return
+
+        #TODO: this should be parallel (how to print?) but is serial for now
+        for d in detectors:
+            self.wait_for_detector_status(d, wait_time=wait_time, timeout=timeout)
+
+
+    def wait_for_detector_status(self, detector, status="running", wait_time=0.1, timeout=300):
+        start_time = time()
+        stop_time = start_time + timeout
+
+        # print the header only in the first iteration
+        first = True
+
+        while True:
+            status_reply = self.restapi.get_detector_status(detector)
+
+            if first:
+                first = False
+                hb = header_bar(status_reply)
+                print(f"{detector}: {hb}")
+
+            cb = color_bar(status_reply)
+            print(f"{detector}: {cb}")
+
+            if status in status_reply:
+                break
+
+            if time() > stop_time:
+                print(f'{detector}: waiting for "{status}" status timed out')
+                break
+
+            sleep(wait_time)
+
+        delta = time() - start_time
+        delta = format_seconds(delta)
+        print(f'{detector}: waited {delta} for "{status}" status')
+
+
+    @forwards_to(guided_power_on, nfilled=1)
+    def guided_power_on(self, *args, **kwargs):
+        guided_power_on(self, *args, **kwargs)
+
+
+
+def format_seconds(s):
+    readable = readable_seconds(s)
+    s = round(s)
+    precise = f"{s} seconds"
+    return precise if precise == readable else f"{readable} ({precise})"
+
 
 

slic/core/acquisition/broker/brokerconfig.py

@@ -56,14 +56,14 @@ class BrokerConfig:
             config["detectors"] = detectors
 
         if self.channels:
-            bsread_channels, camera_channels = split_channels(self.channels)
+            bsread_channels, camera_channels = split_channels(unique(self.channels))
             if bsread_channels:
                 config["channels_list"] = bsread_channels
             if camera_channels:
                 config["camera_list"] = camera_channels
 
         if self.pvs:
-            config["pv_list"] = self.pvs
+            config["pv_list"] = unique(self.pvs)
 
         if self.scan_info:
             config["scan_info"] = self.scan_info
@@ -76,6 +76,9 @@ class BrokerConfig:
 
         return config
 
+    def __repr__(self):
+        return printable_dict(vars(self))
+
 
 
 def split_channels(channels):
@@ -124,6 +127,10 @@ def harmonize_detector_dict(d):
     return d
 
 
+def unique(seq):
+    return sorted(set(seq))
+
+
 
 def clean_output_dir(s, default="_", allowed=ALLOWED_CHARS):
     if s is None:

slic/core/acquisition/broker/jfstatus.py

@@ -0,0 +1,55 @@
+from slic.utils.cprint import cprint, colored
+
+
+BLOCK = "█" * 3 # factor 3 makes block approx. square
+
+STATUS_COLORS = {
+    "idle": "cyan",
+    "error": "red",
+    "waiting": "blue",
+    "run_finished": None,
+    "transmitting": "yellow",
+    "running": "green",
+    "stopped": "magenta"
+}
+
+
+def header_bar(d, sep=" ", block=BLOCK):
+    length = len(block)
+    return sep.join(str(i).center(length) for i, _ in enumerate(status_list(d)))
+
+def color_bar(d, sep=" ", block=BLOCK):
+    return sep.join(status_block(block, i) for i in status_list(d))
+
+def status_block(block, status):
+    color = STATUS_COLORS.get(status)
+    return colored(block, color)
+
+def status_list(d):
+    return values_sorted_by_keys(transpose(d))
+
+def transpose(d):
+    return {i: k for k, v in d.items() for i in v}
+
+def values_sorted_by_keys(d):
+    return [v for _, v in sorted(d.items())]
+
+
+
+
+
+if __name__ == "__main__":
+    for k, v in STATUS_COLORS.items():
+        cprint(BLOCK, k, color=v)
+
+
+    data = {
+        'idle': [0, 2, 3],
+        'stopped': [1, 4],
+        'running': [5, 6]
+    }
+    cb = color_bar(data)
+    print(cb)
+
+
+

slic/core/acquisition/broker/post_retrieve.py

@@ -13,11 +13,12 @@ WAIT_BETWEEN_REQUESTS = 0.1 # seconds
 
 
 
-def post_retrieve(restapi, endstation, pgroup, run, acqs=None, continue_run=False):
+def post_retrieve(restapi, endstation, pgroup, run, acqs=None, continue_run=False, transform=None, dry_run=False, verbosity=None):
     """
     post retrieve data from sf-daq
     acqs: sequence of integer acquisition numbers or None (default: all acquisition numbers of the selected run)
     continue_run: append to existing run (default: create new run)
+    transform: function that accepts one request and either adjusts it in place or returns the adjusted request
     """
     dir_run_meta = mk_dir_run_meta(endstation, pgroup, run)
 
@@ -26,15 +27,21 @@ def post_retrieve(restapi, endstation, pgroup, run, acqs=None, continue_run=Fals
     else:
         fns = mk_fns_acqs(dir_run_meta, acqs)
 
-    post_retrieve_acq_jsons(restapi, fns, continue_run=continue_run)
+    post_retrieve_acq_jsons(restapi, fns, continue_run=continue_run, transform=transform, dry_run=dry_run, verbosity=verbosity)
 
 
-def post_retrieve_acq_jsons(restapi, fns, continue_run=False):
+def post_retrieve_acq_jsons(restapi, fns, continue_run=False, transform=None, dry_run=False, verbosity=None):
     """
     post retrieve data from sf-daq
     fns: sequence of acq json file names
     continue_run: append to existing run (default: create new run)
     """
+    if not isinstance(restapi, DryRunner):
+        restapi = DryRunner(restapi, dry_run=dry_run)
+
+    if verbosity is not None:
+        vprint.level = verbosity
+
     reqs = load_reqs(fns)
 
     first_fn = fns[0]
@@ -53,6 +60,15 @@ def post_retrieve_acq_jsons(restapi, fns, continue_run=False):
             req.update(updates_acq)
             vprint(2, "🪥  new request:", pretty_dict(req))
 
+        if transform:
+            orig_req = req.copy()
+            transformed_req = transform(req)
+            if transformed_req is None: # assume change was made in place
+                transformed_req = req
+            if transformed_req != orig_req:
+                req = transformed_req
+                vprint(1, "🖍️  transformed request:", pretty_dict(req))
+
         resp = restapi.retrieve(req)
         vprint(0, "💌 response:", pretty_dict(resp))
         vprint(1)
@@ -200,9 +216,6 @@ def main():
     clargs = parser.parse_args()
 
     restapi = RESTAPI(clargs.host, clargs.port)
-    restapi = DryRunner(restapi, dry_run=clargs.dry_run)
-
-    vprint.level = clargs.verbose
 
     post_retrieve(
         restapi,
@@ -210,7 +223,9 @@ def main():
         clargs.pgroup,
         clargs.run,
         acqs=clargs.acq,
-        continue_run=clargs.continue_run
+        continue_run=clargs.continue_run,
+        dry_run=clargs.dry_run,
+        verbosity=clargs.verbose
     )
 
 

slic/core/acquisition/broker/poweron.py

@@ -0,0 +1,151 @@
+from time import sleep
+from slic.utils.ask_yes_no import ask_Yes_no
+from slic.utils.boxed import boxed
+
+
+WARNING = "⚠️ "
+SUCCESS = "✅"
+ERROR = "❌"
+
+
+def guided_power_on(client, detector, ask_confirmation=False, wait_time=0.1):
+    assume_yes = not ask_confirmation
+
+    print_header("check connection")
+
+    do_ping = assume_yes or ask_Yes_no(f"ping {detector}")
+    while do_ping:
+        pings = client.restapi.get_detector_pings(detector)
+
+        unreachable = pings["unreachable"]
+        if not unreachable:
+            print(SUCCESS, "all modules responding correctly")
+            break
+
+        print(WARNING, "check the network cable(s) of the following module(s):", unreachable)
+
+        # here we cannot assume yes since the user needs to do something
+        if not ask_Yes_no(f"are you ready to ping {detector} again"):
+            return
+
+
+    print_header("power on")
+
+    if not assume_yes and not ask_Yes_no(f"power on {detector}"):
+        return
+
+    msg = client.restapi.power_on_detector(detector)
+    print(msg)
+
+
+    print_header("check detector status")
+
+    if not assume_yes and not ask_Yes_no(f"wait for running status of a module of {detector}"):
+        return
+
+    while True:
+        status = client.restapi.get_detector_status(detector)
+
+        running = ("running" in status)
+        if running:
+            print(SUCCESS, "done waiting because:", status)
+            break
+
+        print("still waiting because:", status)
+        sleep(wait_time)
+
+
+    print_header("check writing status")
+
+    do_check_running = assume_yes or ask_Yes_no(f"check if {detector} is running")
+    while do_check_running:
+        dets = client.restapi.get_running_detectors()
+
+        if detector in dets["missing_detectors"]:
+            print(ERROR, f"{detector} is missing -- call the sheriff!")
+            return
+
+        if detector in dets["limping_detectors"]:
+            missing = dets["limping_detectors"][detector]["missing_modules"]
+            print(WARNING, f"{detector} is limping -- check the fiber of the following module(s):", missing)
+
+            # here we cannot assume yes since the user needs to do something
+            if not ask_Yes_no(f"are you ready to check again if {detector} is running"):
+                return
+
+            continue
+
+        if detector in dets["running_detectors"]:
+            print(SUCCESS, f"{detector} is running -- done!🚀")
+            return
+
+
+
+def print_header(msg):
+    print()
+    print(boxed(msg, style="double", npad=1))
+
+
+
+
+
+if __name__ == "__main__":
+
+    class Client:
+
+        def __init__(self):
+            self.restapi = RESTAPI()
+
+
+    class RESTAPI:
+
+        def __init__(self):
+            self.fake_pings = gen_fake_pings()
+            self.fake_status = gen_fake_status()
+            self.fake_running_detectors = gen_fake_running_detectors()
+
+        def get_detector_pings(self, detector):
+            return next(self.fake_pings)
+
+        def power_on_detector(self, detector):
+            return "powering on..."
+
+        def get_detector_status(self, detector):
+            return next(self.fake_status)
+
+        def get_running_detectors(self):
+            return next(self.fake_running_detectors)
+
+
+
+    def gen_fake_pings():
+        yield {'responding': [], 'unreachable': [0, 1]}
+        yield {'responding': [0], 'unreachable': [1]}
+        yield {'responding': [0, 1], 'unreachable': []}
+
+    def gen_fake_status():
+        yield {'idle': [0], 'stopped': [1]}
+        yield {'waiting': [0, 1]}
+        yield {'running': [0], 'waiting': [1]}
+
+    def gen_fake_running_detectors():
+#        yield gen_fake_running_detectors_entry(missing_detectors=['JF01T02V03'])
+        yield gen_fake_running_detectors_entry(limping_detectors={'JF01T02V03': {"running_modules": [], "missing_modules": [0, 1]}})
+        yield gen_fake_running_detectors_entry(limping_detectors={'JF01T02V03': {"running_modules": [0], "missing_modules": [1]}})
+        yield gen_fake_running_detectors_entry(running_detectors=['JF01T02V03'])
+
+    def gen_fake_running_detectors_entry(missing_detectors=(), limping_detectors=(), running_detectors=()):
+        return dict(missing_detectors=missing_detectors, limping_detectors=limping_detectors, running_detectors=running_detectors)
+
+
+
+    c = Client()
+    guided_power_on(c, "JF01T02V03", ask_confirmation=False)
+
+    print()
+
+    c = Client()
+    guided_power_on(c, "JF01T02V03", ask_confirmation=True)
+
+
+

slic/core/acquisition/broker/requeststatus.py

@@ -39,7 +39,10 @@ class RequestStatus:
 
 
     def _run(self):
-        connection = BlockingConnection(ConnectionParameters(self.host, **self.kwargs))
+        try:
+            connection = BlockingConnection(ConnectionParameters(self.host, **self.kwargs))
+        except Exception as e:
+            raise ConnectionError(f"cannot connect to request status on {self.host}") from e
 
         channel = connection.channel()
         channel.exchange_declare(exchange=STATUS_EXCHANGE, exchange_type="fanout")
@@ -62,13 +65,17 @@ class RequestStatus:
         body = body.decode()
         request = json.loads(body)
 
-        instrument = request.get("metadata", {}).get("general/instrument")
+        metadata = request.get("metadata") or {}
+
+        instrument = metadata.get("general/instrument")
         if instrument is not None and self.instrument is not None and instrument != self.instrument:
             return
 
         action = headers["action"]
 
-        timestamp = datetime.fromtimestamp(timestamp / 1e9)
+        #TODO: insert current time if there is no timestamp?
+        if timestamp is not None:
+            timestamp = datetime.fromtimestamp(timestamp / 1e9)
 
         key = correlation_id.split("-", 1)[0]
 

slic/core/acquisition/broker/restapi.py

@@ -185,6 +185,16 @@ class BrokerSlowAPI(BaseAPI):
         response = self.get("get_jfctrl_monitor", params, *args, **kwargs)
         return response.get("parameters")
 
+    def get_detector_pings(self, detector, *args, timeout=30, **kwargs):
+        params = {"detector_name": detector}
+        response = self.get("get_detector_pings", params, *args, timeout=timeout, **kwargs)
+        return response.get("pings")
+
+    def get_detector_status(self, detector, *args, **kwargs):
+        params = {"detector_name": detector}
+        response = self.get("get_detector_status", params, *args, **kwargs)
+        return response.get("detector_status")
+
     def get_detector_temperatures(self, detector, *args, **kwargs):
         params = {"detector_name": detector}
         response = self.get("get_detector_temperatures", params, *args, **kwargs)
@@ -205,6 +215,14 @@ class BrokerSlowAPI(BaseAPI):
         return response.get("changed_parameters")
 
 
+    def upload_custom_dap_script(self, name, code, *args, **kwargs):
+        params = {
+            "name": name,
+            "code": code
+        }
+        response = self.post("upload_custom_dap_script", params, *args, **kwargs)
+        return response.get("message")
+
     def get_dap_settings(self, detector, *args, **kwargs):
         params = {"detector_name": detector}
         response = self.get("get_dap_settings", params, *args, **kwargs)

slic/core/acquisition/detcfg.py

@@ -29,8 +29,8 @@ ALLOWED_DAP_PARAMS = dict(
     beam_center_x = Number,
     beam_center_y = Number,
     beam_energy = Number,
+    custom_script = str,
     detector_distance = Number,
-    detector_rate = int,
     disabled_modules = Sequence,
     do_peakfinder_analysis = bool,
     do_radial_integration = bool,
@@ -48,7 +48,8 @@ ALLOWED_DAP_PARAMS = dict(
     roi_y1 = Sequence,
     roi_y2 = Sequence,
     select_only_ppicker_events = bool,
-    spi_limit = Sequence,
+    spi_threshold_photon = Number,
+    spi_threshold_hit_percentage = Number,
     threshold_max = Number,
     threshold_min = Number,
     threshold_value = ["0", "NaN"]
@@ -97,12 +98,12 @@ class _Params(DictUpdateMixin, AttrDict):
         typ = allowed_params[k]
         if isinstance(typ, list):
             if v not in typ:
-                raise ValueError(f"value of parameter {repr(k)} ({v}) has to be from {typ}")
+                raise ValueError(f"value of parameter {repr(k)} ({repr(v)}) has to be from {typ}")
 
         elif not isinstance(v, typ):
             tn_right = typ.__name__
             tn_wrong = type(v).__name__
-            raise TypeError(f"value of parameter {repr(k)} ({v}) has to be of type {tn_right} but is {tn_wrong}")
+            raise TypeError(f"value of parameter {repr(k)} ({repr(v)}) has to be of type {tn_right} but is {tn_wrong}")
 
 
 

slic/core/acquisition/sfacquisition.py

@@ -50,7 +50,7 @@ class SFAcquisition(BaseAcquisition):
         if RequestStatus is None:
             self.status = None
         else:
-            self.status = RequestStatus(instrument=instrument, address=api_host)
+            self.status = RequestStatus(instrument=instrument, host=api_host)
 
         self.current_task = None
 
@@ -167,25 +167,40 @@ class SFAcquisition(BaseAcquisition):
     def get_config_pvs(self):
         return self.client.get_config_pvs()
 
-    def set_config_pvs(self, pvs=None):
+    def set_config_pvs(self, pvs=None, force=False):
         if pvs is None:
             pvs = self.default_pvs
-        return self.client.set_config_pvs(pvs)
-
-    def update_config_pvs(self, pvs=None):
-        if pvs is None:
-            pvs = self.default_pvs
-        current = self.get_config_pvs()
-        pvs = set(pvs) | set(current)
+        pvs = set(pvs)
+        assert_no_fpicture(pvs)
+        if not force:
+            current = self.get_config_pvs()
+            current = set(current)
+            if current == pvs:
+                return
         pvs = sorted(pvs)
         return self.client.set_config_pvs(pvs)
 
+    def update_config_pvs(self, pvs=None, force=False):
+        if pvs is None:
+            pvs = self.default_pvs
+        pvs = set(pvs)
+        current = self.get_config_pvs()
+        current = set(current)
+        merged = pvs | current
+        assert_no_fpicture(merged)
+        if not force and merged == pvs:
+            return
+        merged = sorted(merged)
+        return self.client.set_config_pvs(merged)
+
     def diff_config_pvs(self, pvs=None):
         if pvs is None:
             pvs = self.default_pvs
+        pvs = set(pvs)
         current = self.get_config_pvs()
-        only_remote = set(current) - set(pvs)
-        only_local  = set(pvs) - set(current)
+        current = set(current)
+        only_remote = current - pvs
+        only_local  = pvs - current
         return {"only remote": sorted(only_remote), "only local": sorted(only_local)}
 
 
@@ -224,3 +239,10 @@ def is_continuous(arr, step=1):
 
 
 
+def assert_no_fpicture(pvs):
+    for i in pvs:
+        if i.endswith(":FPICTURE"):
+            raise ValueError(f"camera image channels should not be added to the epics buffer: {i}")
+
+
+

slic/core/scanner/scanner.py

@@ -189,7 +189,7 @@ class Scanner:
 
 
     @forwards_to(make_scan, nfilled=3)
-    def scan1D_seq(self, adjustable, positions, *args, **kwargs):
+    def scan1D_seq(self, adjustable, positions, *args, relative=False, **kwargs):
         """One-dimensional scan over a sequence of positions
 
         Parameters:
@@ -203,13 +203,18 @@ class Scanner:
         """
         adjustables = [adjustable]
 
+        positions = np.asarray(positions)
+        if relative:
+            current = adjustable.get_current_value()
+            positions = positions + current
+
         positions = [positions]
 
         return self.make_scan(adjustables, positions, *args, **kwargs)
 
 
     @forwards_to(make_scan, nfilled=3)
-    def scan2D_seq(self, adjustable1, positions1, adjustable2, positions2, *args, **kwargs):
+    def scan2D_seq(self, adjustable1, positions1, adjustable2, positions2, *args, relative1=False, relative2=False, **kwargs):
         """Two-dimensional scan over two sequences of positions
 
         Parameters:
@@ -226,7 +231,17 @@ class Scanner:
         """
         adjustables = [adjustable1, adjustable2]
 
-        positions = [positions1, positions2]
+        positions1 = np.asarray(positions1)
+        if relative1:
+            current1 = adjustable1.get_current_value()
+            positions1 = positions1 + current1
+
+        positions2 = np.asarray(positions2)
+        if relative2:
+            current2 = adjustable2.get_current_value()
+            positions2 = positions2 + current2
+
+        positions = make_2D_grid(positions1, positions2)
 
         return self.make_scan(adjustables, positions, *args, **kwargs)
 

slic/devices/xoptics/aramis_attenuator.py

@@ -67,11 +67,14 @@ class Transmission(PVAdjustable):
         self.third_order = third_order
 
         prefix = ID + ":"
-        pvname_setvalue = prefix + "TRANS_SP"
-        pvname_readback = prefix + ("TRANS3EDHARM_RB" if third_order else "TRANS_RB")
+#        pvname_setvalue = prefix + "TRANS_SP"
+#        pvname_readback = prefix + ("TRANS3EDHARM_RB" if third_order else "TRANS_RB")
+        pvname_setvalue = prefix + "UsrRec.TD"
+        pvname_readback = prefix + ("UsrRec.TR3" if third_order else "UsrRec.TR1")
         super().__init__(pvname_setvalue, pvname_readback, **kwargs)
 
-        self.pvnames.third_order_toggle = pvn = prefix + "3RD_HARM_SP"
+#        self.pvnames.third_order_toggle = pvn = prefix + "3RD_HARM_SP"
+        self.pvnames.third_order_toggle = pvn = prefix + "UsrRec.HRM3"
         self.pvs.third_order_toggle = PV(pvn)
 
 

slic/gui/daqpanels/config.py

@@ -4,9 +4,9 @@ from slic.core.acquisition import BSChannels, PVChannels
 from slic.utils.reprate import get_beamline, get_pvname_reprate
 from slic.utils.duo import get_pgroup_info
 
-from ..widgets import EXPANDING, STRETCH, show_list, show_two_lists, LabeledEntry, make_filled_vbox, make_filled_hbox, CheckBox
+from ..widgets import EXPANDING, STRETCH, show_two_lists, LabeledEntry, make_filled_vbox, make_filled_hbox, CheckBox
 from .tools import PVDisplay, NOMINAL_REPRATE
-from ..widgets.jfcfg import show_list_jf
+from ..widgets.jfcfg import JFList
 
 
 class ConfigPanel(wx.Panel):
@@ -133,7 +133,7 @@ class ConfigPanel(wx.Panel):
 
 
     def on_chans_det(self, _event):
-        show_list_jf("Detectors", self.chans_det)
+        JFList("Detectors", self.chans_det, self.acquisition)
 
     def on_chans_bsc(self, _event):
         chans = BSChannels(*self.chans_bsc)
@@ -146,7 +146,7 @@ class ConfigPanel(wx.Panel):
         show_two_lists("PVs", online, offline, header1="channels online", header2="channels offline")
 
     def on_power_on(self, _event):
-        self.acquisition.client.power_on(self.chans_det)
+        self.acquisition.client.power_on(self.chans_det, wait=True)
 
     def on_take_pedestal(self, _event):
         rate = self.get_rate()

slic/gui/daqpanels/scan.py

@@ -1,9 +1,9 @@
 import wx
 
-from slic.utils import nice_arange, printed_exception
+from slic.utils import printed_exception
 from slic.utils.reprate import get_pvname_reprate
 
-from ..widgets import STRETCH, TwoButtons, StepsRangeEntry, LabeledMathEntry, LabeledFilenameEntry, make_filled_vbox, post_event
+from ..widgets import EXPANDING, TwoButtons, StepsEntry, LabeledMathEntry, LabeledFilenameEntry, make_filled_vbox, post_event
 from .tools import AdjustableSelection, ETADisplay, correct_n_pulses, run
 
 
@@ -29,7 +29,7 @@ class ScanPanel(wx.Panel):
 
         # widgets:
         self.sel_adj = sel_adj = AdjustableSelection(self)
-        self.adj_range = adj_range = StepsRangeEntry(self)
+        self.adj_steps = adj_steps = StepsEntry(self)
 
         self.cb_relative = cb_relative = wx.CheckBox(self, label="Relative to current position")
         self.cb_return   = cb_return   = wx.CheckBox(self, label="Return to initial value")
@@ -42,7 +42,7 @@ class ScanPanel(wx.Panel):
         self.le_fname   = le_fname   = LabeledFilenameEntry(self, label="Filename", value="test")
 
         pvname_reprate = get_pvname_reprate(instrument)
-        self.eta = eta = ETADisplay(self, config, pvname_reprate, adj_range.nsteps, le_npulses, le_nrepeat)
+        self.eta = eta = ETADisplay(self, config, pvname_reprate, adj_steps, le_npulses, le_nrepeat)
 
         self.btn_go = btn_go = TwoButtons(self)
         btn_go.Bind1(wx.EVT_BUTTON, self.on_go)
@@ -52,7 +52,7 @@ class ScanPanel(wx.Panel):
         widgets = (cb_relative, cb_return)
         vb_cbs = make_filled_vbox(widgets, flag=wx.ALL) # make sure checkboxes do not expand horizontally
 
-        widgets = (sel_adj, STRETCH, adj_range, vb_cbs, le_npulses, le_nrepeat, le_fname, eta, btn_go)
+        widgets = (sel_adj, EXPANDING, adj_steps, vb_cbs, le_npulses, le_nrepeat, le_fname, eta, btn_go)
         vbox = make_filled_vbox(widgets, border=10)
         self.SetSizerAndFit(vbox)
 
@@ -67,7 +67,7 @@ class ScanPanel(wx.Panel):
             post_event(wx.EVT_BUTTON, self.btn_go.btn2)
             return
 
-        start_pos, end_pos, step_size = self.adj_range.get_values()
+        steps = self.adj_steps.get_values()
 
         filename = self.le_fname.GetValue()
 
@@ -84,7 +84,7 @@ class ScanPanel(wx.Panel):
         relative = self.cb_relative.GetValue()
         return_to_initial_values = self.cb_return.GetValue()
 
-        self.scan = self.scanner.scan1D(adjustable, start_pos, end_pos, step_size, n_pulses, filename, relative=relative, return_to_initial_values=return_to_initial_values, n_repeat=n_repeat, start_immediately=False)
+        self.scan = self.scanner.scan1D_seq(adjustable, steps, n_pulses, filename, relative=relative, return_to_initial_values=return_to_initial_values, n_repeat=n_repeat, start_immediately=False)
 
         def wait():
             with printed_exception:

slic/gui/daqpanels/scan2d.py

@@ -1,9 +1,9 @@
 import wx
 
-from slic.utils import nice_arange, printed_exception
+from slic.utils import printed_exception
 from slic.utils.reprate import get_pvname_reprate
 
-from ..widgets import EXPANDING, MINIMIZED, STRETCH, TwoButtons, StepsRangeEntry, LabeledMathEntry, LabeledFilenameEntry, make_filled_vbox, post_event
+from ..widgets import EXPANDING, MINIMIZED, TwoButtons, StepsEntry, LabeledMathEntry, LabeledFilenameEntry, make_filled_vbox, post_event
 from .tools import AdjustableSelection, ETADisplay, correct_n_pulses, run
 
 
@@ -31,7 +31,7 @@ class Scan2DPanel(wx.Panel):
         self.le_fname   = le_fname   = LabeledFilenameEntry(self, label="Filename", value="test")
 
         pvname_reprate = get_pvname_reprate(instrument)
-        self.eta = eta = ETADisplay(self, config, pvname_reprate, adjbox1.adj_range.nsteps, adjbox2.adj_range.nsteps, le_npulses, le_nrepeat)
+        self.eta = eta = ETADisplay(self, config, pvname_reprate, adjbox1.adj_steps, adjbox2.adj_steps, le_npulses, le_nrepeat)
 
         self.btn_go = btn_go = TwoButtons(self)
         btn_go.Bind1(wx.EVT_BUTTON, self.on_go)
@@ -59,8 +59,8 @@ class Scan2DPanel(wx.Panel):
             post_event(wx.EVT_BUTTON, self.btn_go.btn2)
             return
 
-        start_pos1, end_pos1, step_size1 = self.adjbox1.adj_range.get_values()
-        start_pos2, end_pos2, step_size2 = self.adjbox2.adj_range.get_values()
+        steps1 = self.adjbox1.adj_steps.get_values()
+        steps2 = self.adjbox2.adj_steps.get_values()
 
         filename = self.le_fname.GetValue()
 
@@ -78,11 +78,11 @@ class Scan2DPanel(wx.Panel):
         relative2 = self.adjbox2.cb_relative.GetValue()
         return_to_initial_values = self.cb_return.GetValue()
 
-        self.scan = self.scanner.scan2D(
-            adjustable1, start_pos1, end_pos1, step_size1, 
-            adjustable2, start_pos2, end_pos2, step_size2, 
-            n_pulses, filename, 
-            relative1=relative1, relative2=relative2, 
+        self.scan = self.scanner.scan2D_seq(
+            adjustable1, steps1,
+            adjustable2, steps2,
+            n_pulses, filename,
+            relative1=relative1, relative2=relative2,
             return_to_initial_values=return_to_initial_values, n_repeat=n_repeat, start_immediately=False
         )
 
@@ -113,13 +113,13 @@ class AdjustableBox(wx.StaticBoxSizer):
 
         # widgets:
         self.sel_adj = sel_adj = AdjustableSelection(parent)
-        self.adj_range = adj_range = StepsRangeEntry(parent)
+        self.adj_steps = adj_steps = StepsEntry(parent)
 
         self.cb_relative = cb_relative = wx.CheckBox(parent, label="Relative to current position")
         cb_relative.SetValue(False)
 
         # sizers:
-        widgets = (sel_adj, STRETCH, adj_range, MINIMIZED, cb_relative)
+        widgets = (sel_adj, EXPANDING, adj_steps, MINIMIZED, cb_relative)
         make_filled_vbox(widgets, border=10, box=self)
 
 

slic/gui/daqpanels/special.py

@@ -3,8 +3,7 @@ import wx
 from slic.utils import printed_exception
 from slic.utils.reprate import get_pvname_reprate
 
-from ..widgets import LabeledMathEntry, LabeledEntry, LabeledFilenameEntry, LabeledValuesEntry, TwoButtons, make_filled_hbox, make_filled_vbox, STRETCH, EXPANDING
-from ..persist import PersistableWidget
+from ..widgets import StepsEntry, LabeledMathEntry, LabeledFilenameEntry, TwoButtons, make_filled_vbox, EXPANDING
 from .tools import AdjustableSelection, ETADisplay, correct_n_pulses, run, post_event
 
 
@@ -22,14 +21,7 @@ class SpecialScanPanel(wx.Panel):
 
         # widgets:
         self.sel_adj = sel_adj = AdjustableSelection(self)
-
-        self.le_values = le_values = LabeledValuesEntry(self, label="Values")
-        self.le_nsteps = le_nsteps = LabeledEntry(self, label="#Steps")
-
-        le_nsteps.Disable()
-        self.on_change_values(None) # update #Steps
-
-        le_values.Bind(wx.EVT_TEXT, self.on_change_values)
+        self.adj_steps = adj_steps = StepsEntry(self, index=1)
 
         self.cb_relative = cb_relative = wx.CheckBox(self, label="Relative to current position")
         self.cb_return   = cb_return   = wx.CheckBox(self, label="Return to initial value")
@@ -42,40 +34,21 @@ class SpecialScanPanel(wx.Panel):
         self.le_fname   = le_fname   = LabeledFilenameEntry(self, label="Filename", value="test")
 
         pvname_reprate = get_pvname_reprate(instrument)
-        self.eta = eta = ETADisplay(self, config, pvname_reprate, le_nsteps, le_npulses, le_nrepeat)
+        self.eta = eta = ETADisplay(self, config, pvname_reprate, adj_steps, le_npulses, le_nrepeat)
 
         self.btn_go = btn_go = TwoButtons(self)
         btn_go.Bind1(wx.EVT_BUTTON, self.on_go)
         btn_go.Bind2(wx.EVT_BUTTON, self.on_stop)
 
         # sizers:
-        hb_values = wx.BoxSizer()
-        hb_values.Add(le_values, 1, wx.EXPAND)
-
-        widgets = (STRETCH, STRETCH, STRETCH, le_nsteps)
-        hb_pos = make_filled_hbox(widgets)
-
         widgets = (cb_relative, cb_return)
         vb_cbs = make_filled_vbox(widgets, flag=wx.ALL) # make sure checkboxes do not expand horizontally
 
-        widgets = (sel_adj, EXPANDING, hb_values, hb_pos, vb_cbs, le_npulses, le_nrepeat, le_fname, eta, btn_go)
+        widgets = (sel_adj, EXPANDING, adj_steps, vb_cbs, le_npulses, le_nrepeat, le_fname, eta, btn_go)
         vbox = make_filled_vbox(widgets, border=10)
         self.SetSizerAndFit(vbox)
 
 
-    def on_change_values(self, _event):
-        try:
-            steps = self.le_values.get_values()
-        except ValueError as e:
-            nsteps = ""
-            tooltip = str(e)
-        else:
-            nsteps = str(len(steps))
-            tooltip = str(steps)
-        self.le_nsteps.SetValue(nsteps)
-        self.le_nsteps.SetToolTip(tooltip)
-
-
     def on_go(self, _event):
         if self.scan:
             return
@@ -86,7 +59,7 @@ class SpecialScanPanel(wx.Panel):
             post_event(wx.EVT_BUTTON, self.btn_go.btn2)
             return
 
-        steps = self.le_values.get_values()
+        steps = self.adj_steps.get_values()
 
         filename = self.le_fname.GetValue()
 
@@ -101,13 +74,9 @@ class SpecialScanPanel(wx.Panel):
         n_pulses = correct_n_pulses(n_pulses, rate, rm)
 
         relative = self.cb_relative.GetValue()
-        if relative:
-            current = adjustable.get_current_value()
-            steps += current
-
         return_to_initial_values = self.cb_return.GetValue()
 
-        self.scan = self.scanner.scan1D_seq(adjustable, steps, n_pulses, filename, return_to_initial_values=return_to_initial_values, n_repeat=n_repeat, start_immediately=False)
+        self.scan = self.scanner.scan1D_seq(adjustable, steps, n_pulses, filename, relative=relative, return_to_initial_values=return_to_initial_values, n_repeat=n_repeat, start_immediately=False)
 
         def wait():
             with printed_exception:

slic/gui/widgets/__init__.py

@@ -1,9 +1,10 @@
 
 from .alarm import AlarmMixin
+from .alternative import Alternative
 from .boxes import EXPANDING, MINIMIZED, STRETCH, make_filled_vbox, make_filled_hbox
 from .checkbox import CheckBox
 from .completers import ContainsTextCompleter, FuzzyTextCompleter
-from .entries import StepsRangeEntry, LabeledEntry, LabeledFilenameEntry, LabeledMathEntry, LabeledTweakEntry, LabeledValuesEntry
+from .entries import StepsEntry, StepsRangeEntry, StepsSequenceEntry, LabeledEntry, LabeledFilenameEntry, LabeledMathEntry, LabeledTweakEntry, LabeledValuesEntry
 from .lists import AutoWidthListCtrl, show_list, show_two_lists
 from .mods import MainPanel, NotebookDX
 from .nope import Nope

slic/gui/widgets/alarm.py

@@ -1,4 +1,5 @@
 import wx
+from logzero import logger as log
 
 
 try:
@@ -9,7 +10,7 @@ try:
     dbn = DBusNotify()
     icon = get_icon_path()
 except Exception as e:
-    print("could not set up DBusNotify:", e)
+    log.warning(f"could not set up DBusNotify: {e}")
     dbn = None
 
 

slic/gui/widgets/alternative.py

@@ -0,0 +1,89 @@
+import wx
+
+
+class Alternative(wx.BoxSizer):
+
+    def __init__(self, widgets, index=0):
+        super().__init__(wx.VERTICAL)
+
+        self.widgets = widgets
+        self.index = index
+
+        self.callback_update_visibility = None
+
+        for i in widgets:
+            self.Add(i, proportion=1, flag=wx.EXPAND)
+
+        self.update_visibility()
+
+        self.bind_all(wx.EVT_RIGHT_UP, self.on_right_click)
+
+
+    def bind_all(self, *args, **kwargs):
+        for i in self.widgets:
+            i.Bind(*args, **kwargs)
+            if i.GetToolTip() is None:
+                i.SetToolTip("Right click to switch mode")
+            # right-click event is not propagated to parent
+            for j in i.GetChildren():
+                j.Bind(*args, **kwargs)
+                # avoid inheriting tooltip from parent for entry boxes without own tooltip
+                if isinstance(j, wx.TextCtrl) and j.GetToolTip() is None:
+                    suppress_tooltip(j)
+
+
+    def on_right_click(self, _event):
+        self.next()
+
+    def next(self):
+        self.increase_index()
+        self.update_visibility()
+
+    def increase_index(self):
+        self.index += 1
+        self.index %= len(self.widgets)
+
+    def update_visibility(self):
+        for i, wdgt in enumerate(self.widgets):
+            state = (i == self.index)
+            if isinstance(wdgt, wx.Sizer):
+                wdgt.ShowItems(show=state)
+            else:
+                wdgt.Show(show=state)
+        self.Layout()
+        if self.callback_update_visibility:
+            self.callback_update_visibility()
+
+    def __getattr__(self, name):
+        w = self.get_current()
+        return getattr(w, name)
+
+    def get_current(self):
+        return self.widgets[self.index]
+
+
+
+def suppress_tooltip(widget):
+    """
+    if no tooltip is set, the parent tooltip is used
+    store the parent tooltip upon entering and revert it upon leaving
+    """
+    tip = None
+
+    def on_enter(_):
+        nonlocal tip
+        tooltip = widget.GetParent().GetToolTip()
+        #TODO: why is this sometimes None on the consoles?
+        if tooltip is None:
+            return
+        tip = tooltip.GetTip()
+        widget.GetParent().SetToolTip(None)
+
+    def on_leave(_):
+        widget.GetParent().SetToolTip(tip)
+
+    widget.Bind(wx.EVT_ENTER_WINDOW, on_enter)
+    widget.Bind(wx.EVT_LEAVE_WINDOW, on_leave)
+
+
+

slic/gui/widgets/entries.py

@@ -5,10 +5,12 @@ import wx
 from slic.utils import arithmetic_eval, typename, nice_arange
 
 from ..persist import PersistableWidget
-from .boxes import make_filled_hbox
+from .alternative import Alternative
+from .boxes import EXPANDING, STRETCH, make_filled_hbox, make_filled_vbox
 from .fname import increase, decrease
 from .labeled import make_labeled
 from .nope import Nope
+from .tools import post_event
 
 
 ADJUSTMENTS = {
@@ -21,16 +23,35 @@ ALLOWED_CHARS = set(
 )
 
 
-class StepsRangeEntry(wx.BoxSizer):
+class StepsEntry(wx.Panel):
+
+    def __init__(self, parent, index=0):
+        super().__init__(parent)
+        steps_range    = StepsRangeEntry(self)
+        steps_sequence = StepsSequenceEntry(self)
+        widgets = (steps_range, steps_sequence)
+        self.alt = alt = Alternative(widgets, index=index)
+        self.alt.callback_update_visibility = lambda: post_event(wx.EVT_TEXT, self.alt.nsteps.widget) #TODO: find a simpler solution
+        self.SetSizerAndFit(alt)
+
+    def GetValue(self):
+        return self.alt.nsteps.GetValue()
+
+    def __getattr__(self, name):
+        return getattr(self.alt, name)
+
+
+
+class StepsRangeEntry(wx.Panel):
 
     def __init__(self, parent):
-        super().__init__(wx.HORIZONTAL)
+        super().__init__(parent)
 
-        self.start  = start  = LabeledMathEntry(parent, label="Start",     value=0)
-        self.stop   = stop   = LabeledMathEntry(parent, label="Stop",      value=10)
-        self.step   = step   = LabeledMathEntry(parent, label="Step Size", value=0.1)
+        self.start  = start  = LabeledMathEntry(self, label="Start",     value=0)
+        self.stop   = stop   = LabeledMathEntry(self, label="Stop",      value=10)
+        self.step   = step   = LabeledMathEntry(self, label="Step Size", value=0.1)
 
-        self.nsteps = nsteps = LabeledEntry(parent, label="#Steps")
+        self.nsteps = nsteps = LabeledEntry(self, label="#Steps")
 
         nsteps.Disable()
         self.on_change(None) # initialize #Steps
@@ -40,25 +61,18 @@ class StepsRangeEntry(wx.BoxSizer):
             w.Bind(wx.EVT_TEXT, self.on_change)
 
         widgets = (start, stop, step, nsteps)
-        make_filled_hbox(widgets, box=self)
+        hbox = make_filled_hbox(widgets)
+        sizer = make_filled_vbox([STRETCH, hbox])
+        self.SetSizerAndFit(sizer)
 
 
     def on_change(self, _event):
         try:
-            try:
-                start_pos, end_pos, step_size = self.get_values()
-            except:
-                raise ValueError
-            else:
-                if step_size == 0:
-                    raise ValueError
-                if None in (start_pos, end_pos, step_size):
-                    raise ValueError
-        except ValueError:
+            steps = self.get_values()
+        except ValueError as e:
             nsteps = ""
-            tooltip = "Start, Stop and Step Size need to be floats.\nStep Size cannot be zero."
+            tooltip = str(e)
         else:
-            steps = nice_arange(start_pos, end_pos, step_size)
             nsteps = str(len(steps))
             tooltip = str(steps)
         self.nsteps.SetValue(nsteps)
@@ -66,36 +80,89 @@ class StepsRangeEntry(wx.BoxSizer):
 
 
     def get_values(self):
-        start_pos = self.start.GetValue()
-        end_pos   = self.stop.GetValue()
-        step_size = self.step.GetValue()
-        return start_pos, end_pos, step_size
+        try:
+            start_pos = self.start.GetValue()
+            end_pos   = self.stop.GetValue()
+            step_size = self.step.GetValue()
+        except Exception as e:
+            raise ValueError("Start, Stop and Step Size need to be floats.") from e
+        else:
+            if step_size == 0:
+                raise ValueError("Step Size cannot be zero.")
+            if None in (start_pos, end_pos, step_size):
+                raise ValueError("Start, Stop and Step Size need to be floats.")
+        return nice_arange(start_pos, end_pos, step_size)
 
 
 
-class LabeledTweakEntry(wx.BoxSizer):
+class StepsSequenceEntry(wx.Panel):
+
+    def __init__(self, parent):
+        super().__init__(parent)
+
+        self.values = values = LabeledValuesEntry(self, label="Values")
+
+        self.nsteps = nsteps = LabeledEntry(self, label="#Steps")
+
+        nsteps.Disable()
+        self.on_change(None) # initialize #Steps
+
+        values.Bind(wx.EVT_TEXT, self.on_change)
+
+        hb_values = wx.BoxSizer()
+        hb_values.Add(values, 1, wx.EXPAND)
+
+        widgets = (STRETCH, STRETCH, STRETCH, nsteps)
+        hb_pos = make_filled_hbox(widgets, border=20, flag=wx.TOP)
+
+        widgets = (EXPANDING, hb_values, hb_pos)
+        sizer = make_filled_vbox(widgets)
+        self.SetSizerAndFit(sizer)
+
+
+    def on_change(self, _event):
+        try:
+            steps = self.get_values()
+        except ValueError as e:
+            nsteps = ""
+            tooltip = str(e)
+        else:
+            nsteps = str(len(steps))
+            tooltip = str(steps)
+        self.nsteps.SetValue(nsteps)
+        self.nsteps.SetToolTip(tooltip)
+
+
+    def get_values(self):
+        return self.values.get_values()
+
+
+
+class LabeledTweakEntry(wx.Panel):
 
     def __init__(self, parent, id=wx.ID_ANY, label="", value=""):
-        super().__init__(wx.VERTICAL)
+        super().__init__(parent)
 
         value = str(value)
         name = label
 
-        self.label = label = wx.StaticText(parent, label=label)
-        self.text  = text  = MathEntry(parent, value=value, name=name, style=wx.TE_RIGHT)
+        self.label = label = wx.StaticText(self, label=label)
+        self.text  = text  = MathEntry(self, value=value, name=name, style=wx.TE_RIGHT)
 
-        self.btn_left  = btn_left  = wx.Button(parent, label="<")
-        self.btn_right = btn_right = wx.Button(parent, label=">")
+        self.btn_left  = btn_left  = wx.Button(self, label="<")
+        self.btn_right = btn_right = wx.Button(self, label=">")
 
-        self.btn_ff_left  = btn_ff_left  = wx.Button(parent, label="<<")
-        self.btn_ff_right = btn_ff_right = wx.Button(parent, label=">>")
+        self.btn_ff_left  = btn_ff_left  = wx.Button(self, label="<<")
+        self.btn_ff_right = btn_ff_right = wx.Button(self, label=">>")
 
         widgets = (btn_ff_left, btn_left, btn_right, btn_ff_right)
         hb_tweak = make_filled_hbox(widgets)
 
-        self.Add(label,    flag=wx.EXPAND)
-        self.Add(text,     flag=wx.EXPAND)
-        self.Add(hb_tweak, flag=wx.EXPAND|wx.TOP, border=10)
+        sizer = wx.BoxSizer(wx.VERTICAL)
+        sizer.Add(label,    flag=wx.EXPAND)
+        sizer.Add(text,     flag=wx.EXPAND)
+        sizer.Add(hb_tweak, flag=wx.EXPAND|wx.TOP, border=10)
+        self.SetSizerAndFit(sizer)
 
 
     def Disable(self):

slic/gui/widgets/jfcfg.py

@@ -3,48 +3,124 @@ from numbers import Number
 
 import wx
 
-from slic.core.acquisition.detcfg import ALLOWED_DETECTOR_PARAMS
+from slic.core.acquisition.detcfg import ALLOWED_DETECTOR_PARAMS, ALLOWED_DAP_PARAMS, ALLOWED_HARDWARE_PARAMS
+from slic.utils import typename
 
 from .entries import LabeledEntry, LabeledMathEntry, MathEntry
-from .lists import ListDialog, ListDisplay, WX_DEFAULT_RESIZABLE_DIALOG_STYLE
+from .lists import ListDialog, WX_DEFAULT_RESIZABLE_DIALOG_STYLE
 from .labeled import make_labeled
 from .jfmodcoords import get_module_coords
 
 
-def show_list_jf(title, det_dict):
-    dlg = ListDialog(title, det_dict)
-
-    cb = lambda evt: on_dclick(evt, det_dict)
-
-    #TODO: attach widgets to parents and replace the following workaround
-    children = dlg.GetChildren()
-    for child in children:
-        if isinstance(child, ListDisplay):
-            child.Bind(wx.EVT_LISTBOX_DCLICK, cb)
-            break
-
-    dlg.ShowModal()
-    dlg.Destroy()
+NUMBER_PER_COLUMN = 10
 
 
-def on_dclick(evt, det_dict):
-    name = evt.GetString()
-    params = det_dict[name]
-    dlg = JFConfig(name, params)
-    dlg.ShowModal()
+class JFList:
 
-    # update the dict with the changed values
-#    print("before:", det_dict)
-    det_dict[name] = dlg.get()
-#    print("after: ", det_dict)
+    def __init__(self, title, det_dict, acquisition):
+        self.det_dict = det_dict
+        self.acquisition = acquisition
 
-    dlg.Destroy()
+        dlg = ListDialog(title, det_dict)
+        self.list = dlg.list
+
+        dlg.list.Bind(wx.EVT_LISTBOX_DCLICK, self.on_config_detector)
+
+        detector_btn = wx.Button(dlg, label="Detector")
+        dlg.buttons.Add(detector_btn)
+        detector_btn.Bind(wx.EVT_BUTTON, self.on_config_detector)
+
+        dap_btn = wx.Button(dlg, label="DAP")
+        dlg.buttons.Add(dap_btn)
+        dap_btn.Bind(wx.EVT_BUTTON, self.on_config_dap)
+
+        hardware_btn = wx.Button(dlg, label="Hardware")
+        dlg.buttons.Add(hardware_btn)
+        hardware_btn.Bind(wx.EVT_BUTTON, self.on_config_hardware)
+
+        self.buttons = [
+            detector_btn,
+            dap_btn,
+            hardware_btn
+        ]
+
+        dlg.Fit()
+
+        dlg.ShowModal()
+        dlg.Destroy()
+
+
+    def on_config_detector(self, _evt):
+        name = self.list.GetSelectionString()
+        params = self.det_dict[name]
+
+        dlg = JFConfig(name, params, ALLOWED_DETECTOR_PARAMS)
+        dlg.ShowModal()
+
+        # update the dict with the changed values
+#        print("before:", det_dict)
+        self.det_dict[name] = dlg.get()
+#        print("after: ", det_dict)
+
+        dlg.Destroy()
+
+
+    def on_config_dap(self, _evt):
+        wx.SafeYield() # disable everything until dialog is ready
+        self._buttons_disable()
+        name = self.list.GetSelectionString()
+        try:
+            params = self.acquisition.client.restapi.get_dap_settings(name, timeout=30)
+        except Exception as e:
+            en = typename(e)
+            print(f"Failed to get DAP settings due to:\n{en}: {e}\nAssuming empty configuration...")
+            params = {}
+
+        dlg = JFConfig(name, params, ALLOWED_DAP_PARAMS)
+        dlg.ShowModal()
+
+        res = dlg.get()
+#        print("DAP:", res)
+
+        changed_parameters = self.acquisition.client.restapi.set_dap_settings(name, res, timeout=30)
+        print("changed DAP parameters:", changed_parameters)
+
+        dlg.Destroy()
+        self._buttons_enable()
+
+
+    def on_config_hardware(self, _evt):
+        wx.SafeYield() # disable everything until dialog is ready
+        self._buttons_disable()
+        name = self.list.GetSelectionString()
+        params = self.acquisition.client.restapi.get_detector_settings(name, timeout=30)
+
+        dlg = JFConfig(name, params, ALLOWED_HARDWARE_PARAMS)
+        dlg.ShowModal()
+
+        res = dlg.get()
+#        print("Hardware:", res)
+
+        changed_parameters = self.acquisition.client.restapi.set_detector_settings(name, res, timeout=30)
+        print("changed hardware parameters:", changed_parameters)
+
+        dlg.Destroy()
+        self._buttons_enable()
+
+
+    def _buttons_enable(self):
+        for btn in self.buttons:
+            btn.Enable()
+
+    def _buttons_disable(self):
+        for btn in self.buttons:
+            btn.Disable()
 
 
 
 class JFConfig(wx.Dialog):
 
-    def __init__(self, title, params):
+    def __init__(self, title, params, allowed_params):
         wx.Dialog.__init__(self, None, title=title, style=WX_DEFAULT_RESIZABLE_DIALOG_STYLE)
 
         std_dlg_btn_sizer = self.CreateStdDialogButtonSizer(wx.CLOSE)
@@ -55,11 +131,14 @@ class JFConfig(wx.Dialog):
 
         vbox_params = wx.BoxSizer(wx.HORIZONTAL)
         vbox_cbs    = wx.BoxSizer(wx.VERTICAL)
-        vbox_others = wx.BoxSizer(wx.VERTICAL)
+
+        vbox_params.Add(vbox_cbs, flag=wx.ALL|wx.EXPAND)
 
         vbox_cbs.AddSpacer(border)
 
-        for k, v in sorted(ALLOWED_DETECTOR_PARAMS.items()):
+        index_in_column = -1
+
+        for k, v in sorted(allowed_params.items()):
             widgets[k] = w = self.make_widget(title, k, v)
 
             if isinstance(w, wx.CheckBox):
@@ -68,20 +147,25 @@ class JFConfig(wx.Dialog):
                 flag = wx.LEFT|wx.RIGHT
                 vbox_cbs.Add(w, flag=flag, border=border)
             else:
+                index_in_column += 1
+                if index_in_column % NUMBER_PER_COLUMN == 0:
+                    vbox_others = wx.BoxSizer(wx.VERTICAL)
+                    vbox_params.Add(vbox_others, flag=wx.ALL|wx.EXPAND, proportion=1)
+
                 flag = wx.ALL|wx.EXPAND
                 vbox_others.Add(w, flag=flag, border=border)
 
         vbox_cbs.AddSpacer(border)
 
-        vbox_params.Add(vbox_cbs, flag=wx.ALL|wx.EXPAND)
-        vbox_params.Add(vbox_others, flag=wx.ALL|wx.EXPAND, proportion=1)
-
         vbox = wx.BoxSizer(wx.VERTICAL)
         vbox.Add(vbox_params, flag=wx.ALL|wx.EXPAND)
         vbox.AddStretchSpacer()
         vbox.Add(std_dlg_btn_sizer, flag=wx.ALL|wx.CENTER, border=10)
 
         for k, v in params.items():
+            if k not in widgets:
+                print(f"skipping unknown parameter: {k}")
+                continue
             widgets[k].SetValue(v)
 
         self.SetSizerAndFit(vbox)

slic/gui/widgets/lists.py

@@ -22,7 +22,9 @@ class ListDialog(wx.Dialog):
         wx.Dialog.__init__(self, None, title=title, style=WX_DEFAULT_RESIZABLE_DIALOG_STYLE)
 
         hld = HeaderedListDisplay(self, sequence, header=header)
-        std_dlg_btn_sizer = self.CreateStdDialogButtonSizer(wx.CLOSE)
+        self.list = hld.list
+
+        self.buttons = std_dlg_btn_sizer = self.CreateStdDialogButtonSizer(wx.CLOSE)
 
         vbox = wx.BoxSizer(wx.VERTICAL)
         vbox.Add(hld, proportion=1, flag=wx.ALL|wx.EXPAND, border=10)
@@ -59,7 +61,7 @@ class HeaderedListDisplay(wx.BoxSizer):
         header = f"{nentries} {header}"
 
         st_header = wx.StaticText(parent, label=header)
-        ld_sequence = ListDisplay(parent, sequence)
+        self.list = ld_sequence = ListDisplay(parent, sequence)
 
         self.Add(st_header, flag=wx.BOTTOM|wx.CENTER, border=10)
         self.Add(ld_sequence, proportion=1, flag=wx.ALL|wx.EXPAND)
@@ -83,6 +85,11 @@ class ListDisplay(wx.ListBox):
         copy_to_clipboard(val)
 
 
+    def GetSelectionString(self):
+        idx = self.GetSelection()
+        return self.GetString(idx)
+
+
 
 class AutoWidthListCtrl(wx.ListCtrl, listmix.ListCtrlAutoWidthMixin):
 

slic/gui/wxdebug.py

@@ -0,0 +1,31 @@
+import os
+from random import randint
+
+import wx
+
+
+original_add = wx.Sizer.Add
+
+
+def wxdebug():
+    if "WXDEBUG" in os.environ:
+        wx.Sizer.Add = debug_add
+
+
+def debug_add(self, item, *args, **kwargs):
+    try:
+        item.SetBackgroundColour(random_color())
+    except:
+        pass
+    return original_add(self, item, *args, **kwargs)
+
+
+def random_color():
+    return wx.Colour(
+        randint(100, 255),
+        randint(100, 255),
+        randint(100, 255)
+    )
+
+
+

slic/utils/boxed.py

@@ -0,0 +1,132 @@
+STYLES = {
+    "normal": "┌┐└┘──││",
+    "thick":  "┏┓┗┛━━┃┃",
+    "double": "╔╗╚╝══║║",
+
+    "thick-top":  "┍┑┕┙━━││",
+    "thick-side": "┎┒┖┚──┃┃",
+
+    "double-top":  "╒╕╘╛══││",
+    "double-side": "╓╖╙╜──║║",
+
+    "fat":     "▛▜▙▟▀▄▌▐",
+    "tight":   "▗▖▝▘▄▀▐▌",
+    "rounded": "╭╮╰╯──││",
+    "ascii":   "++++--||"
+}
+
+
+ALIGNMENTS = {
+    "left":   "ljust",
+    "center": "center",
+    "right":  "rjust"
+}
+
+
+
+class Style:
+
+    def __init__(self, chars):
+        self.chars = chars
+        try:
+            self.top_left, self.top_right, self.bottom_left, self.bottom_right, self.top, self.bottom, self.left, self.right = chars
+        except ValueError as e:
+            raise ValueError(f"cannot unpack {chars!r} in style due to {e}") from e
+
+
+    def make_box(self, lines, length):
+        top    = self.make_top_line(length)
+        middle = self.make_middle_lines(lines)
+        bottom = self.make_bottom_line(length)
+        return flatten(top, middle, bottom)
+
+    def make_top_line(self, length):
+        return self.top_left + self.top * length + self.top_right
+
+    def make_middle_lines(self, lines):
+        return [self.left + i + self.right for i in lines]
+
+    def make_bottom_line(self, length):
+        return self.bottom_left + self.bottom * length + self.bottom_right
+
+
+    def __repr__(self):
+        tn = type(self).__name__
+        return f"{tn}({self.chars!r})"
+
+    def __str__(self):
+        return self.make_box("X", 1)
+
+
+
+def boxed(text, align="left", style="normal", npad=0):
+    lines = text.splitlines()
+    length = maxlen(lines)
+
+    lines = aligned(lines, length, align=align)
+
+    if npad:
+        # factor 3 makes boxes approx. square
+        nvpad = int(round(npad/3))
+        length += 2 * npad
+
+        # this has to be in this order, otherwise vert_padded needs to insert lines of the correct length
+        lines = vert_padded(lines, nvpad)
+        lines = hori_padded(lines, length)
+
+    style = Style(STYLES[style])
+    return style.make_box(lines, length)
+
+
+def maxlen(seq):
+    return max(len(i) for i in seq)
+
+def aligned(lines, length, align="left"):
+    """
+    align: left, center, right
+    """
+    try:
+        meth = ALIGNMENTS[align]
+    except KeyError as e:
+        values = tuple(ALIGNMENTS.keys())
+        raise ValueError(f"{align!r} is not from {values}") from e
+    return [getattr(line, meth)(length) for line in lines]
+
+def flatten(top, middle, bottom):
+    return "\n".join([top] + middle + [bottom])
+
+def vert_padded(lines, npad):
+    padding = [" "] * npad
+    return padding + lines + padding
+
+def hori_padded(lines, length):
+    return aligned(lines, length, align="center")
+
+
+
+
+
+if __name__ == "__main__":
+    style = Style(r"/\\/^_[]")
+    print(repr(style))
+    print(style)
+    print()
+
+    for k, v in STYLES.items():
+        style = Style(v)
+        print(f"{k}:")
+        print(style)
+        print()
+
+    txt = "a\nbbb\nccccc"
+    print(boxed(txt))
+    print(boxed(txt, style="rounded", align="center", npad=3))
+    print(boxed(txt, style="fat", align="right"))
+    print(boxed(txt, style="tight", align="right"))
+
+
+
+#TODO:
+# - separately definable npad for horizontal and vertical
+# - custom styles from boxed()
+# - colored boxes

slic/utils/eval.py

@@ -40,10 +40,6 @@ def arithmetic_eval(s):
 def ast_node_eval(node):
     if isinstance(node, ast.Expression):
         return ast_node_eval(node.body)
-    elif isinstance(node, ast.Str):
-        return node.s
-    elif isinstance(node, ast.Num):
-        return node.n
     elif isinstance(node, ast.BinOp):
         op = get_operator(node, BIN_OPS)
         left  = ast_node_eval(node.left)
@@ -53,6 +49,15 @@ def ast_node_eval(node):
         op = get_operator(node, UNARY_OPS)
         operand = ast_node_eval(node.operand)
         return op(operand)
+    # from >=3.8, Constant can replace Str/Num
+    # from >=3.14, Str/Num are deprecated and removed
+    # check Constant first then fall back to Str/Num for <3.8
+    elif isinstance(node, ast.Constant):
+        return node.value
+    elif isinstance(node, ast.Str):
+        return node.s
+    elif isinstance(node, ast.Num):
+        return node.n
     else:
         tn = typename(node)
         raise ArithmeticEvalError(f"Unsupported node type {tn}")

slic/utils/logcfg.py

@@ -1,6 +1,7 @@
 import builtins
 import os
 import sys
+import warnings
 
 import logging
 import logzero
@@ -54,13 +55,23 @@ def setup_import_logging():
     imports_cache = set()
 
     def import_with_log(*args, **kwargs):
-        module = orig_import(*args, **kwargs)
+        # catch warnings in order to re-emit them with corrected stacklevel
+        with warnings.catch_warnings(record=True) as caught_warnings:
+            module = orig_import(*args, **kwargs)
 
         name = module.__name__
         if name not in imports_cache:
             imports_cache.add(name)
             log.trace(f"importing: {name}")
 
+        for w in caught_warnings:
+            warnings.warn(
+                message=w.message,
+                category=w.category,
+                stacklevel=2,
+                source=w.source
+            )
+
         return module
 
     builtins.__import__ = import_with_log