Merge pull request #212 from tiqi-group/44-manually-saving-current-state-of-all-objects

Feat: autosaving feature

docs/dev-guide/api.md

@@ -1,6 +1,15 @@
 ::: pydase.data_service
     handler: python
 
+::: pydase.data_service.data_service_cache
+    handler: python
+
+::: pydase.data_service.data_service_observer
+    handler: python
+
+::: pydase.data_service.state_manager
+    handler: python
+
 ::: pydase.server.server
     handler: python
 

docs/user-guide/Service_Persistence.md

@@ -2,29 +2,47 @@
 
 `pydase` allows you to easily persist the state of your service by saving it to a file. This is especially useful when you want to maintain the service's state across different runs.
 
-To save the state of your service, pass a `filename` keyword argument to the constructor of the `pydase.Server` class. If the file specified by `filename` does not exist, the state manager will create this file and store its state in it when the service is shut down. If the file already exists, the state manager will load the state from this file, setting the values of its attributes to the values stored in the file.
+To enable persistence, pass a `filename` keyword argument to the constructor of the [`pydase.Server`][pydase.Server] class. The `filename` specifies the file where the state will be saved:
 
-Here's an example:
+- If the file **does not exist**, it will be created and populated with the current state when the service shuts down or saves.
+- If the file **already exists**, the state manager will **load** the saved values into the service at startup.
+
+Here’s an example:
 
 ```python
 import pydase
 
 class Device(pydase.DataService):
-    # ... defining the Device class ...
+    # ... define your service class ...
-
 
 if __name__ == "__main__":
     service = Device()
     pydase.Server(service=service, filename="device_state.json").run()
 ```
 
-In this example, the state of the `Device` service will be saved to `device_state.json` when the service is shut down. If `device_state.json` exists when the server is started, the state manager will restore the state of the service from this file.
+In this example, the service state will be automatically loaded from `device_state.json` at startup (if it exists), and saved to the same file periodically and upon shutdown.
+
+## Automatic Periodic State Saving
+
+When a `filename` is provided, `pydase` automatically enables **periodic autosaving** of the service state to that file. This ensures that the current state is regularly persisted, reducing the risk of data loss during unexpected shutdowns.
+
+The autosave happens every 30 seconds by default. You can customize the interval using the `autosave_interval` argument (in seconds):
+
+```python
+pydase.Server(
+    service=service,
+    filename="device_state.json",
+    autosave_interval=10.0,  # save every 10 seconds
+).run()
+```
+
+To disable automatic saving, set `autosave_interval` to `None`.
 
 ## Controlling Property State Loading with `@load_state`
 
-By default, the state manager only restores values for public attributes of your service. If you have properties that you want to control the loading for, you can use the `@load_state` decorator on your property setters. This indicates to the state manager that the value of the property should be loaded from the state file.
+By default, the state manager only restores values for public attributes of your service (i.e. *it does not restore property values*). If you have properties that you want to control the loading for, you can use the [`@load_state`][pydase.data_service.state_manager.load_state] decorator on your property setters. This indicates to the state manager that the value of the property should be loaded from the state file.
 
-Here is how you can apply the `@load_state` decorator:
+Example:
 
 ```python
 import pydase
@@ -43,7 +61,6 @@ class Device(pydase.DataService):
         self._name = value
 ```
 
-With the `@load_state` decorator applied to the `name` property setter, the state manager will load and apply the `name` property's value from the file storing the state upon server startup, assuming it exists.
+With the `@load_state` decorator applied to the `name` property setter, the state manager will load and apply the `name` property's value from the file upon server startup.
-
-Note: If the service class structure has changed since the last time its state was saved, only the attributes and properties decorated with `@load_state` that have remained the same will be restored from the settings file.
 
+**Note**: If the structure of your service class changes between saves, only properties decorated with `@load_state` and unchanged public attributes will be restored safely.

src/pydase/data_service/data_service_cache.py

@@ -14,6 +14,22 @@ logger = logging.getLogger(__name__)
 
 
 class DataServiceCache:
+    """Maintains a serialized cache of the current state of a DataService instance.
+
+    This class is responsible for storing and updating a representation of the service's
+    public attributes and properties. It is primarily used by the StateManager and the
+    web server to serve consistent state to clients without accessing the DataService
+    attributes directly.
+
+    The cache is initialized once upon construction by serializing the full state of
+    the service. After that, it can be incrementally updated using attribute paths and
+    values as notified by the
+    [`DataServiceObserver`][pydase.data_service.data_service_observer.DataServiceObserver].
+
+    Args:
+        service: The DataService instance whose state should be cached.
+    """
+
     def __init__(self, service: "DataService") -> None:
         self._cache: SerializedObject
         self.service = service

src/pydase/data_service/state_manager.py

@@ -1,3 +1,4 @@
+import asyncio
 import contextlib
 import json
 import logging
@@ -66,43 +67,41 @@ def has_load_state_decorator(prop: property) -> bool:
 class StateManager:
     """
     Manages the state of a DataService instance, serving as both a cache and a
-    persistence layer. It is designed to provide quick access to the latest known state
+    persistence layer. It provides fast access to the most recently known state of the
-    for newly connecting web clients without the need for expensive property accesses
+    service and ensures consistent state updates across connected clients and service
-    that may involve complex calculations or I/O operations.
+    restarts.
 
-    The StateManager listens for state change notifications from the DataService's
+    The StateManager is used by the web server to apply updates to service attributes
-    callback manager and updates its cache accordingly. This cache does not always
+    and to serve the current state to newly connected clients. Internally, it creates a
-    reflect the most current complex property states but rather retains the value from
+    [`DataServiceCache`][pydase.data_service.data_service_cache.DataServiceCache]
-    the last known state, optimizing for performance and reducing the load on the
+    instance to track the state of public attributes and properties.
-    system.
 
-    While the StateManager ensures that the cached state is as up-to-date as possible,
+    The StateManager also handles state persistence: it can load a previously saved
-    it does not autonomously update complex properties of the DataService. Such
+    state from disk at startup and periodically autosave the current state to a file
-    properties must be updated programmatically, for instance, by invoking specific
+    during runtime.
-    tasks or methods that trigger the necessary operations to refresh their state.
-
-    The cached state maintained by the StateManager is particularly useful for web
-    clients that connect to the system and need immediate access to the current state of
-    the DataService. By avoiding direct and potentially costly property accesses, the
-    StateManager provides a snapshot of the DataService's state that is sufficiently
-    accurate for initial rendering and interaction.
 
     Args:
-        service:
+        service: The DataService instance whose state is being managed.
-            The DataService instance whose state is being managed.
+        filename: The file name used for loading and storing the DataService's state.
-        filename:
+            If provided, the state is loaded from this file at startup and saved to it
-            The file name used for storing the DataService's state.
+            on shutdown or at regular intervals.
+        autosave_interval: Interval in seconds between automatic state save events.
+            If set to `None`, automatic saving is disabled.
 
     Note:
-        The StateManager's cache updates are triggered by notifications and do not
+        The StateManager does not autonomously poll hardware state. It relies on the
-        include autonomous updates of complex DataService properties, which must be
+        service to perform such updates. The cache maintained by
-        managed programmatically. The cache serves the purpose of providing immediate
+        [`DataServiceCache`][pydase.data_service.data_service_cache.DataServiceCache]
-        state information to web clients, reflecting the state after the last property
+        reflects the last known state as notified by the `DataServiceObserver`, and is
-        update.
+        used by the web interface to provide fast and accurate state rendering for
+        connected clients.
     """
 
     def __init__(
-        self, service: "DataService", filename: str | Path | None = None
+        self,
+        service: "DataService",
+        filename: str | Path | None = None,
+        autosave_interval: float | None = None,
     ) -> None:
         self.filename = getattr(service, "_filename", None)
 
@@ -115,6 +114,29 @@ class StateManager:
 
         self.service = service
         self.cache_manager = DataServiceCache(self.service)
+        self.autosave_interval = autosave_interval
+
+    async def autosave(self) -> None:
+        """Periodically saves the current service state to the configured file.
+
+        This coroutine is automatically started by the [`pydase.Server`][pydase.Server]
+        when a filename is provided. It runs in the background and writes the latest
+        known state of the service to disk every `autosave_interval` seconds.
+
+        If `autosave_interval` is set to `None`, autosaving is disabled and this
+        coroutine exits immediately.
+        """
+
+        if self.autosave_interval is None:
+            return
+
+        while True:
+            try:
+                if self.filename is not None:
+                    self.save_state()
+                await asyncio.sleep(self.autosave_interval)
+            except Exception as e:
+                logger.exception(e)
 
     @property
     def cache_value(self) -> dict[str, SerializedObject]:
@@ -122,23 +144,21 @@ class StateManager:
         return cast(dict[str, SerializedObject], self.cache_manager.cache["value"])
 
     def save_state(self) -> None:
-        """
+        """Saves the DataService's current state to a JSON file defined by
-        Saves the DataService's current state to a JSON file defined by `self.filename`.
+        `self.filename`.
-        Logs an error if `self.filename` is not set.
         """
 
         if self.filename is not None:
             with open(self.filename, "w") as f:
                 json.dump(self.cache_value, f, indent=4)
         else:
-            logger.info(
+            logger.debug(
                 "State manager was not initialised with a filename. Skipping "
                 "'save_state'..."
             )
 
     def load_state(self) -> None:
-        """
+        """Loads the DataService's state from a JSON file defined by `self.filename`.
-        Loads the DataService's state from a JSON file defined by `self.filename`.
         Updates the service's attributes, respecting type and read-only constraints.
         """
 
@@ -191,8 +211,7 @@ class StateManager:
         path: str,
         serialized_value: SerializedObject,
     ) -> None:
-        """
+        """Sets the value of an attribute in the service managed by the `StateManager`
-        Sets the value of an attribute in the service managed by the `StateManager`
         given its path as a dot-separated string.
 
         This method updates the attribute specified by 'path' with 'value' only if the

src/pydase/server/server.py

@@ -84,21 +84,15 @@ class Server:
     The `Server` class provides a flexible server implementation for the `DataService`.
 
     Args:
-        service:
+        service: The DataService instance that this server will manage.
-            The DataService instance that this server will manage.
+        host: The host address for the server. Defaults to `'0.0.0.0'`, which means all
-        host:
-            The host address for the server. Defaults to `'0.0.0.0'`, which means all
             available network interfaces.
-        web_port:
+        web_port: The port number for the web server. Defaults to
-            The port number for the web server. Defaults to
             [`ServiceConfig().web_port`][pydase.config.ServiceConfig.web_port].
-        enable_web:
+        enable_web: Whether to enable the web server.
-            Whether to enable the web server.
+        filename: Filename of the file managing the service state persistence.
-        filename:
+        additional_servers: A list of additional servers to run alongside the main
-            Filename of the file managing the service state persistence.
+            server.
-        additional_servers:
-            A list of additional servers to run alongside the main server.
-
             Here's an example of how you might define an additional server:
 
             ```python
@@ -137,8 +131,9 @@ class Server:
             )
             server.run()
             ```
-        **kwargs:
+        autosave_interval: Interval in seconds between automatic state save events.
-            Additional keyword arguments.
+            If set to `None`, automatic saving is disabled. Defaults to 30 seconds.
+        **kwargs: Additional keyword arguments.
     """
 
     def __init__(  # noqa: PLR0913
@@ -149,6 +144,7 @@ class Server:
         enable_web: bool = True,
         filename: str | Path | None = None,
         additional_servers: list[AdditionalServer] | None = None,
+        autosave_interval: float = 30.0,
         **kwargs: Any,
     ) -> None:
         if additional_servers is None:
@@ -161,7 +157,11 @@ class Server:
         self._additional_servers = additional_servers
         self.should_exit = False
         self.servers: dict[str, asyncio.Future[Any]] = {}
-        self._state_manager = StateManager(self._service, filename)
+        self._state_manager = StateManager(
+            service=self._service,
+            filename=filename,
+            autosave_interval=autosave_interval,
+        )
         self._observer = DataServiceObserver(self._state_manager)
         self._state_manager.load_state()
         autostart_service_tasks(self._service)
@@ -223,6 +223,8 @@ class Server:
             server_task.add_done_callback(self._handle_server_shutdown)
             self.servers["web"] = server_task
 
+        self._loop.create_task(self._state_manager.autosave())
+
     def _handle_server_shutdown(self, task: asyncio.Task[Any]) -> None:
         """Handle server shutdown. If the service should exit, do nothing. Else, make
         the service exit."""

tests/data_service/test_state_manager.py

@@ -1,10 +1,13 @@
+import asyncio
 import json
 from pathlib import Path
 from typing import Any
 
+import anyio
 import pydase
 import pydase.components
 import pydase.units as u
+import pytest
 from pydase.data_service.data_service_observer import DataServiceObserver
 from pydase.data_service.state_manager import (
     StateManager,
@@ -349,4 +352,24 @@ def test_property_load_state(tmp_path: Path) -> None:
 
     assert service_instance.name == "Some other name"
     assert service_instance.not_loadable_attr == "Not loadable"
-    assert not has_load_state_decorator(type(service_instance).property_without_setter)
+    assert not has_load_state_decorator(type(service_instance).property_without_setter)  # type: ignore
+
+
+@pytest.mark.asyncio()
+async def test_autosave(tmp_path: Path, caplog: LogCaptureFixture) -> None:
+    filename = tmp_path / "state.json"
+
+    service = Service()
+    manager = StateManager(service=service, filename=filename, autosave_interval=0.1)
+    DataServiceObserver(state_manager=manager)
+
+    task = asyncio.create_task(manager.autosave())
+    service.property_attr = 198.0
+    await asyncio.sleep(0.1)
+    task.cancel()
+
+    assert filename.exists(), "Autosave should write to the file"
+    async with await anyio.open_file(filename) as f:
+        data = json.loads(await f.read())
+
+    assert data["property_attr"]["value"] == service.property_attr