ITQC/pydase
24
0
Fork 0
mirror of https://github.com/tiqi-group/pydase.git synced 2025-06-05 21:20:40 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #212 from tiqi-group/44-manually-saving-current-state-of-all-objects

Browse Source 
Feat: autosaving feature
This commit is contained in:
Mose Müller 2025-03-25 14:06:30 +01:00 committed by GitHub
commit 755265bf53
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 149 additions and 63 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
docs/dev-guide/api.md
View File

@ -1,6 +1,15 @@
::: pydase.data_service
handler: python
::: pydase.data_service.data_service_cache
handler: python
::: pydase.data_service.data_service_observer
handler: python
::: pydase.data_service.state_manager
handler: python
::: pydase.server.server
handler: python

37
docs/user-guide/Service_Persistence.md
View File

@ -2,29 +2,47 @@
`pydase` allows you to easily persist the state of your service by saving it to a file. This is especially useful when you want to maintain the service's state across different runs.
To save the state of your service, pass a `filename` keyword argument to the constructor of the `pydase.Server` class. If the file specified by `filename` does not exist, the state manager will create this file and store its state in it when the service is shut down. If the file already exists, the state manager will load the state from this file, setting the values of its attributes to the values stored in the file.
To enable persistence, pass a `filename` keyword argument to the constructor of the [`pydase.Server`][pydase.Server] class. The `filename` specifies the file where the state will be saved:
Here's an example:
- If the file **does not exist**, it will be created and populated with the current state when the service shuts down or saves.
- If the file **already exists**, the state manager will **load** the saved values into the service at startup.
Here’s an example:
```python
import pydase
class Device(pydase.DataService):
# ... defining the Device class ...
# ... define your service class ...
if __name__ == "__main__":
service = Device()
pydase.Server(service=service, filename="device_state.json").run()
```
In this example, the state of the `Device` service will be saved to `device_state.json` when the service is shut down. If `device_state.json` exists when the server is started, the state manager will restore the state of the service from this file.
In this example, the service state will be automatically loaded from `device_state.json` at startup (if it exists), and saved to the same file periodically and upon shutdown.
## Automatic Periodic State Saving
When a `filename` is provided, `pydase` automatically enables **periodic autosaving** of the service state to that file. This ensures that the current state is regularly persisted, reducing the risk of data loss during unexpected shutdowns.
The autosave happens every 30 seconds by default. You can customize the interval using the `autosave_interval` argument (in seconds):
```python
pydase.Server(
service=service,
filename="device_state.json",
autosave_interval=10.0, # save every 10 seconds
).run()
```
To disable automatic saving, set `autosave_interval` to `None`.
## Controlling Property State Loading with `@load_state`
By default, the state manager only restores values for public attributes of your service. If you have properties that you want to control the loading for, you can use the `@load_state` decorator on your property setters. This indicates to the state manager that the value of the property should be loaded from the state file.
By default, the state manager only restores values for public attributes of your service (i.e. *it does not restore property values*). If you have properties that you want to control the loading for, you can use the [`@load_state`][pydase.data_service.state_manager.load_state] decorator on your property setters. This indicates to the state manager that the value of the property should be loaded from the state file.
Here is how you can apply the `@load_state` decorator:
Example:
```python
import pydase
@ -43,7 +61,6 @@ class Device(pydase.DataService):
self._name = value
```
With the `@load_state` decorator applied to the `name` property setter, the state manager will load and apply the `name` property's value from the file storing the state upon server startup, assuming it exists.
Note: If the service class structure has changed since the last time its state was saved, only the attributes and properties decorated with `@load_state` that have remained the same will be restored from the settings file.
With the `@load_state` decorator applied to the `name` property setter, the state manager will load and apply the `name` property's value from the file upon server startup.
**Note**: If the structure of your service class changes between saves, only properties decorated with `@load_state` and unchanged public attributes will be restored safely.

16
src/pydase/data_service/data_service_cache.py
View File

@ -14,6 +14,22 @@ logger = logging.getLogger(__name__)
class DataServiceCache:
"""Maintains a serialized cache of the current state of a DataService instance.
This class is responsible for storing and updating a representation of the service's
public attributes and properties. It is primarily used by the StateManager and the
web server to serve consistent state to clients without accessing the DataService
attributes directly.
The cache is initialized once upon construction by serializing the full state of
the service. After that, it can be incrementally updated using attribute paths and
values as notified by the
[`DataServiceObserver`][pydase.data_service.data_service_observer.DataServiceObserver].
Args:
service: The DataService instance whose state should be cached.
"""
def __init__(self, service: "DataService") -> None:
self._cache: SerializedObject
self.service = service

91
src/pydase/data_service/state_manager.py
View File

@ -1,3 +1,4 @@
import asyncio
import contextlib
import json
import logging
@ -66,43 +67,41 @@ def has_load_state_decorator(prop: property) -> bool:
class StateManager:
"""
Manages the state of a DataService instance, serving as both a cache and a
persistence layer. It is designed to provide quick access to the latest known state
for newly connecting web clients without the need for expensive property accesses
that may involve complex calculations or I/O operations.
persistence layer. It provides fast access to the most recently known state of the
service and ensures consistent state updates across connected clients and service
restarts.
The StateManager listens for state change notifications from the DataService's
callback manager and updates its cache accordingly. This cache does not always
reflect the most current complex property states but rather retains the value from
the last known state, optimizing for performance and reducing the load on the
system.
The StateManager is used by the web server to apply updates to service attributes
and to serve the current state to newly connected clients. Internally, it creates a
[`DataServiceCache`][pydase.data_service.data_service_cache.DataServiceCache]
instance to track the state of public attributes and properties.
While the StateManager ensures that the cached state is as up-to-date as possible,
it does not autonomously update complex properties of the DataService. Such
properties must be updated programmatically, for instance, by invoking specific
tasks or methods that trigger the necessary operations to refresh their state.
The cached state maintained by the StateManager is particularly useful for web
clients that connect to the system and need immediate access to the current state of
the DataService. By avoiding direct and potentially costly property accesses, the
StateManager provides a snapshot of the DataService's state that is sufficiently
accurate for initial rendering and interaction.
The StateManager also handles state persistence: it can load a previously saved
state from disk at startup and periodically autosave the current state to a file
during runtime.
Args:
service:
The DataService instance whose state is being managed.
filename:
The file name used for storing the DataService's state.
service: The DataService instance whose state is being managed.
filename: The file name used for loading and storing the DataService's state.
If provided, the state is loaded from this file at startup and saved to it
on shutdown or at regular intervals.
autosave_interval: Interval in seconds between automatic state save events.
If set to `None`, automatic saving is disabled.
Note:
The StateManager's cache updates are triggered by notifications and do not
include autonomous updates of complex DataService properties, which must be
managed programmatically. The cache serves the purpose of providing immediate
state information to web clients, reflecting the state after the last property
update.
The StateManager does not autonomously poll hardware state. It relies on the
service to perform such updates. The cache maintained by
[`DataServiceCache`][pydase.data_service.data_service_cache.DataServiceCache]
reflects the last known state as notified by the `DataServiceObserver`, and is
used by the web interface to provide fast and accurate state rendering for
connected clients.
"""
def __init__(
self, service: "DataService", filename: str | Path | None = None
self,
service: "DataService",
filename: str | Path | None = None,
autosave_interval: float | None = None,
) -> None:
self.filename = getattr(service, "_filename", None)
@ -115,6 +114,29 @@ class StateManager:
self.service = service
self.cache_manager = DataServiceCache(self.service)
self.autosave_interval = autosave_interval
async def autosave(self) -> None:
"""Periodically saves the current service state to the configured file.
This coroutine is automatically started by the [`pydase.Server`][pydase.Server]
when a filename is provided. It runs in the background and writes the latest
known state of the service to disk every `autosave_interval` seconds.
If `autosave_interval` is set to `None`, autosaving is disabled and this
coroutine exits immediately.
"""
if self.autosave_interval is None:
return
while True:
try:
if self.filename is not None:
self.save_state()
await asyncio.sleep(self.autosave_interval)
except Exception as e:
logger.exception(e)
@property
def cache_value(self) -> dict[str, SerializedObject]:
@ -122,23 +144,21 @@ class StateManager:
return cast(dict[str, SerializedObject], self.cache_manager.cache["value"])
def save_state(self) -> None:
"""
Saves the DataService's current state to a JSON file defined by `self.filename`.
Logs an error if `self.filename` is not set.
"""Saves the DataService's current state to a JSON file defined by
`self.filename`.
"""
if self.filename is not None:
with open(self.filename, "w") as f:
json.dump(self.cache_value, f, indent=4)
else:
logger.info(
logger.debug(
"State manager was not initialised with a filename. Skipping "
"'save_state'..."
)
def load_state(self) -> None:
"""
Loads the DataService's state from a JSON file defined by `self.filename`.
"""Loads the DataService's state from a JSON file defined by `self.filename`.
Updates the service's attributes, respecting type and read-only constraints.
"""
@ -191,8 +211,7 @@ class StateManager:
path: str,
serialized_value: SerializedObject,
) -> None:
"""
Sets the value of an attribute in the service managed by the `StateManager`
"""Sets the value of an attribute in the service managed by the `StateManager`
given its path as a dot-separated string.
This method updates the attribute specified by 'path' with 'value' only if the

34
src/pydase/server/server.py
View File

@ -84,21 +84,15 @@ class Server:
The `Server` class provides a flexible server implementation for the `DataService`.
Args:
service:
The DataService instance that this server will manage.
host:
The host address for the server. Defaults to `'0.0.0.0'`, which means all
service: The DataService instance that this server will manage.
host: The host address for the server. Defaults to `'0.0.0.0'`, which means all
available network interfaces.
web_port:
The port number for the web server. Defaults to
web_port: The port number for the web server. Defaults to
[`ServiceConfig().web_port`][pydase.config.ServiceConfig.web_port].
enable_web:
Whether to enable the web server.
filename:
Filename of the file managing the service state persistence.
additional_servers:
A list of additional servers to run alongside the main server.
enable_web: Whether to enable the web server.
filename: Filename of the file managing the service state persistence.
additional_servers: A list of additional servers to run alongside the main
server.
Here's an example of how you might define an additional server:
```python
@ -137,8 +131,9 @@ class Server:
)
server.run()
```
**kwargs:
Additional keyword arguments.
autosave_interval: Interval in seconds between automatic state save events.
If set to `None`, automatic saving is disabled. Defaults to 30 seconds.
**kwargs: Additional keyword arguments.
"""
def __init__( # noqa: PLR0913
@ -149,6 +144,7 @@ class Server:
enable_web: bool = True,
filename: str | Path | None = None,
additional_servers: list[AdditionalServer] | None = None,
autosave_interval: float = 30.0,
**kwargs: Any,
) -> None:
if additional_servers is None:
@ -161,7 +157,11 @@ class Server:
self._additional_servers = additional_servers
self.should_exit = False
self.servers: dict[str, asyncio.Future[Any]] = {}
self._state_manager = StateManager(self._service, filename)
self._state_manager = StateManager(
service=self._service,
filename=filename,
autosave_interval=autosave_interval,
)
self._observer = DataServiceObserver(self._state_manager)
self._state_manager.load_state()
autostart_service_tasks(self._service)
@ -223,6 +223,8 @@ class Server:
server_task.add_done_callback(self._handle_server_shutdown)
self.servers["web"] = server_task
self._loop.create_task(self._state_manager.autosave())
def _handle_server_shutdown(self, task: asyncio.Task[Any]) -> None:
"""Handle server shutdown. If the service should exit, do nothing. Else, make
the service exit."""

25
tests/data_service/test_state_manager.py
View File

@ -1,10 +1,13 @@
import asyncio
import json
from pathlib import Path
from typing import Any
import anyio
import pydase
import pydase.components
import pydase.units as u
import pytest
from pydase.data_service.data_service_observer import DataServiceObserver
from pydase.data_service.state_manager import (
StateManager,
@ -349,4 +352,24 @@ def test_property_load_state(tmp_path: Path) -> None:
assert service_instance.name == "Some other name"
assert service_instance.not_loadable_attr == "Not loadable"
assert not has_load_state_decorator(type(service_instance).property_without_setter)
assert not has_load_state_decorator(type(service_instance).property_without_setter) # type: ignore
@pytest.mark.asyncio()
async def test_autosave(tmp_path: Path, caplog: LogCaptureFixture) -> None:
filename = tmp_path / "state.json"
service = Service()
manager = StateManager(service=service, filename=filename, autosave_interval=0.1)
DataServiceObserver(state_manager=manager)
task = asyncio.create_task(manager.autosave())
service.property_attr = 198.0
await asyncio.sleep(0.1)
task.cancel()
assert filename.exists(), "Autosave should write to the file"
async with await anyio.open_file(filename) as f:
data = json.loads(await f.read())
assert data["property_attr"]["value"] == service.property_attr