ITQC/pydase
24
0
Fork 0
mirror of https://github.com/tiqi-group/pydase.git synced 2025-06-05 21:20:40 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #226 from tiqi-group/feat/proxy_support

Browse Source 
Feat: add SOCKS5 proxy support to pydase.Client
This commit is contained in:
Mose Müller 2025-05-20 20:44:24 +02:00 committed by GitHub
commit 7ff6cab9b3
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 164 additions and 38 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/python-package.yml vendored
View File

@ -28,7 +28,7 @@ jobs:
run: |
python -m pip install --upgrade pip
python -m pip install poetry
poetry install --with dev
poetry install --with dev --all-extras
- name: Check with ruff
run: |
poetry run ruff check src

48
docs/user-guide/advanced/SOCKS-Proxy.md Normal file
View File

@ -0,0 +1,48 @@
# Connecting Through a SOCKS5 Proxy
If your target service is only reachable via an SSH gateway or resides behind a
firewall, you can route your [`pydase.Client`][pydase.Client] connection through a local
SOCKS5 proxy. This is particularly useful in network environments where direct access to
the service is not possible.
## Setting Up a SOCKS5 Proxy
You can create a local [SOCKS5 proxy](https://en.wikipedia.org/wiki/SOCKS) using SSH's
`-D` option:
```bash
ssh -D 2222 user@gateway.example.com
```
This command sets up a SOCKS5 proxy on `localhost:2222`, securely forwarding traffic
over the SSH connection.
## Using the Proxy in Your Python Client
Once the proxy is running, configure the [`pydase.Client`][pydase.Client] to route
traffic through it using the `proxy_url` parameter:
```python
import pydase
client = pydase.Client(
url="ws://target-service:8001",
proxy_url="socks5://localhost:2222"
).proxy
```
* You can also use this setup with `wss://` URLs for encrypted WebSocket connections.
## Installing Required Dependencies
To use this feature, you must install the optional `socks` dependency group, which
includes [`aiohttp_socks`](https://pypi.org/project/aiohttp-socks/):
- `poetry`
```bash
poetry add "pydase[socks]"
```
- `pip`
```bash
pip install "pydase[socks]"
```

16
docs/user-guide/interaction/Python-Client.md
View File

@ -1,6 +1,6 @@
# Python RPC Client
The [`pydase.Client`][pydase.Client] allows you to connect to a remote `pydase` service using socket.io, facilitating interaction with the service as though it were running locally.
The [`pydase.Client`][pydase.Client] allows you to connect to a remote `pydase` service using Socket.IO, facilitating interaction with the service as though it were running locally.
## Basic Usage
@ -9,6 +9,7 @@ import pydase
# Replace <ip_addr> and <service_port> with the appropriate values for your service
client_proxy = pydase.Client(url="ws://<ip_addr>:<service_port>").proxy
# For SSL-encrypted services, use the wss protocol
# client_proxy = pydase.Client(url="wss://your-domain.ch").proxy
@ -22,6 +23,12 @@ The proxy acts as a local representation of the remote service, enabling intuiti
The proxy class automatically synchronizes with the server's attributes and methods, keeping itself up-to-date with any changes. This dynamic synchronization essentially mirrors the server's API, making it feel like you're working with a local object.
### Accessing Services Behind Firewalls or SSH Gateways
If your service is only reachable through a private network or SSH gateway, you can route your connection through a local SOCKS5 proxy using the `proxy_url` parameter.
See [Connecting Through a SOCKS5 Proxy](../advanced/SOCKS-Proxy.md) for details.
## Context Manager Support
You can also use the client within a context manager, which automatically handles connection management (i.e., opening and closing the connection):
@ -53,6 +60,7 @@ class MyService(pydase.DataService):
block_until_connected=False,
client_id="my_pydase_client_id",
).proxy
# For SSL-encrypted services, use the wss protocol
# proxy = pydase.Client(
# url="wss://your-domain.ch",
@ -68,12 +76,12 @@ if __name__ == "__main__":
In this example:
- The `MyService` class has a `proxy` attribute that connects to a `pydase` service at `<ip_addr>:<service_port>`.
- By setting `block_until_connected=False`, the service can start without waiting for the connection to succeed, which is particularly useful in distributed systems where services may initialize in any order.
- By setting `client_id`, the server will provide more accurate logs of the connecting client. If set, this ID is sent as `X-Client-Id` header in the HTTP(s) request.
- By setting `block_until_connected=False`, the service can start without waiting for the connection to succeed.
- By setting `client_id`, the server will log a descriptive identifier for this client via the `X-Client-Id` HTTP header.
## Custom `socketio.AsyncClient` Connection Parameters
You can also configure advanced connection options by passing additional arguments to the underlying [`AsyncClient`][socketio.AsyncClient] via `sio_client_kwargs`. This allows you to fine-tune reconnection behaviour, delays, and other settings:
You can configure advanced connection options by passing arguments to the underlying [`AsyncClient`][socketio.AsyncClient] via `sio_client_kwargs`. For example:
```python
client = pydase.Client(

1
mkdocs.yml
View File

@ -19,6 +19,7 @@ nav:
- Logging in pydase: user-guide/Logging.md
- Advanced:
- Deploying behind a Reverse Proxy: user-guide/advanced/Reverse-Proxy.md
- Connecting through a SOCKS Proxy: user-guide/advanced/SOCKS-Proxy.md
- Developer Guide:
- Developer Guide: dev-guide/README.md
- API Reference: dev-guide/api.md

44
poetry.lock generated
View File

@ -133,6 +133,23 @@ aiohttp = ">=3.8.1,<4.0.0"
async-timeout = ">=4.0.2,<5.0.0"
yarl = ">=1.5.1,<2.0.0"
[[package]]
name = "aiohttp-socks"
version = "0.10.1"
description = "Proxy connector for aiohttp"
optional = true
python-versions = ">=3.8.0"
groups = ["main"]
markers = "extra == \"socks\""
files = [
{file = "aiohttp_socks-0.10.1-py3-none-any.whl", hash = "sha256:6fd4d46c09f952f971a011ff446170daab8d539cf5310c0627f8423df2fb15ea"},
{file = "aiohttp_socks-0.10.1.tar.gz", hash = "sha256:49f2e1f8051f2885719beb1b77e312b5a27c3e4b60f0b045a388f194d995e068"},
]
[package.dependencies]
aiohttp = ">=3.10.0"
python-socks = {version = ">=2.4.3,<3.0.0", extras = ["asyncio"]}
[[package]]
name = "aiosignal"
version = "1.3.2"
@ -2289,6 +2306,28 @@ asyncio-client = ["aiohttp (>=3.4)"]
client = ["requests (>=2.21.0)", "websocket-client (>=0.54.0)"]
docs = ["sphinx"]
[[package]]
name = "python-socks"
version = "2.7.1"
description = "Proxy (SOCKS4, SOCKS5, HTTP CONNECT) client for Python"
optional = true
python-versions = ">=3.8.0"
groups = ["main"]
markers = "extra == \"socks\""
files = [
{file = "python_socks-2.7.1-py3-none-any.whl", hash = "sha256:2603c6454eeaeb82b464ad705be188989e8cf1a4a16f0af3c921d6dd71a49cec"},
{file = "python_socks-2.7.1.tar.gz", hash = "sha256:f1a0bb603830fe81e332442eada96757b8f8dec02bd22d1d6f5c99a79704c550"},
]
[package.dependencies]
async-timeout = {version = ">=4.0", optional = true, markers = "python_version < \"3.11\" and extra == \"asyncio\""}
[package.extras]
anyio = ["anyio (>=3.3.4,<5.0.0)"]
asyncio = ["async-timeout (>=4.0) ; python_version < \"3.11\""]
curio = ["curio (>=1.4)"]
trio = ["trio (>=0.24)"]
[[package]]
name = "pyyaml"
version = "6.0.2"
@ -2776,7 +2815,10 @@ idna = ">=2.0"
multidict = ">=4.0"
propcache = ">=0.2.1"
[extras]
socks = ["aiohttp-socks"]
[metadata]
lock-version = "2.1"
python-versions = ">=3.10,<4.0"
content-hash = "3a5ca427686cd80e749ac69aa795540a0e0b3bd0d539cd3a8a264e94eeb48782"
content-hash = "07754bc1fa6fc5e4b15c253a68cfe32368ae0a1bb9e83d8f7fd80ee61013c401"

3
pyproject.toml
View File

@ -19,6 +19,9 @@ dependencies = [
"anyio (>=4.9.0,<5.0.0)"
]
[project.optional-dependencies]
socks = ["aiohttp-socks (>=0.10.1,<0.11.0)"]
[tool.poetry]
packages = [{include = "pydase", from = "src"}]

88
src/pydase/client/client.py
View File

@ -6,6 +6,8 @@ import urllib.parse
from types import TracebackType
from typing import TYPE_CHECKING, Any, TypedDict, cast
import aiohttp
import aiohttp_socks.connector
import socketio # type: ignore
from pydase.client.proxy_class import ProxyClass
@ -40,47 +42,52 @@ def asyncio_loop_thread(loop: asyncio.AbstractEventLoop) -> None:
class Client:
"""
A client for connecting to a remote pydase service using socket.io. This client
"""A client for connecting to a remote pydase service using Socket.IO. This client
handles asynchronous communication with a service, manages events such as
connection, disconnection, and updates, and ensures that the proxy object is
up-to-date with the server state.
Args:
url:
The URL of the pydase Socket.IO server. This should always contain the
protocol and the hostname.
block_until_connected:
If set to True, the constructor will block until the connection to the
service has been established. This is useful for ensuring the client is
ready to use immediately after instantiation. Default is True.
sio_client_kwargs:
Additional keyword arguments passed to the underlying
url: The URL of the pydase Socket.IO server. This should always contain the
protocol (e.g., `ws` or `wss`) and the hostname, and can optionally include
a path prefix (e.g., `ws://localhost:8001/service`).
block_until_connected: If set to True, the constructor will block until the
connection to the service has been established. This is useful for ensuring
the client is ready to use immediately after instantiation. Default is True.
sio_client_kwargs: Additional keyword arguments passed to the underlying
[`AsyncClient`][socketio.AsyncClient]. This allows fine-tuning of the
client's behaviour (e.g., reconnection attempts or reconnection delay).
Default is an empty dictionary.
client_id: Client identification that will be shown in the server logs this
client is connecting to. This ID is passed as a `X-Client-Id` header in the
HTTP(s) request. Defaults to None.
client_id: An optional client identifier. This ID is sent to the server as the
`X-Client-Id` HTTP header. It can be used for logging or authentication
purposes on the server side.
proxy_url: An optional proxy URL to route the connection through. This is useful
if the service is only reachable via an SSH tunnel or behind a firewall
(e.g., `socks5://localhost:2222`).
Example:
The following example demonstrates a `Client` instance that connects to another
pydase service, while customising some of the connection settings for the
underlying [`AsyncClient`][socketio.AsyncClient].
Connect to a service directly:
```python
pydase.Client(url="ws://localhost:8001", sio_client_kwargs={
"reconnection_attempts": 2,
"reconnection_delay": 2,
"reconnection_delay_max": 8,
})
client = pydase.Client(url="ws://localhost:8001")
```
When connecting to a server over a secure connection (i.e., the server is using
SSL/TLS encryption), make sure that the `wss` protocol is used instead of `ws`:
Connect over a secure connection:
```python
pydase.Client(url="wss://my-service.example.com")
client = pydase.Client(url="wss://my-service.example.com")
```
Connect using a SOCKS5 proxy (e.g., through an SSH tunnel):
```bash
ssh -D 2222 user@gateway.example.com
```
```python
client = pydase.Client(
url="ws://remote-server:8001",
proxy_url="socks5://localhost:2222"
)
```
"""
@ -91,6 +98,7 @@ class Client:
block_until_connected: bool = True,
sio_client_kwargs: dict[str, Any] = {},
client_id: str | None = None,
proxy_url: str | None = None,
):
# Parse the URL to separate base URL and path prefix
parsed_url = urllib.parse.urlparse(url)
@ -103,8 +111,9 @@ class Client:
# Store the path prefix (e.g., "/service" in "ws://localhost:8081/service")
self._path_prefix = parsed_url.path.rstrip("/") # Remove trailing slash if any
self._url = url
self._sio = socketio.AsyncClient(**sio_client_kwargs)
self._proxy_url = proxy_url
self._client_id = client_id
self._sio_client_kwargs = sio_client_kwargs
self._loop: asyncio.AbstractEventLoop | None = None
self._thread: threading.Thread | None = None
self.proxy: ProxyClass
@ -126,6 +135,12 @@ class Client:
def connect(self, block_until_connected: bool = True) -> None:
if self._thread is None or self._loop is None:
self._loop = self._initialize_loop_and_thread()
self._initialize_socketio_client()
self.proxy = ProxyClass(
sio_client=self._sio,
loop=self._loop,
reconnect=self.connect,
)
connection_future = asyncio.run_coroutine_threadsafe(
self._connect(), self._loop
@ -133,17 +148,26 @@ class Client:
if block_until_connected:
connection_future.result()
def _initialize_socketio_client(self) -> None:
if self._proxy_url is not None:
session = aiohttp.ClientSession(
connector=aiohttp_socks.connector.ProxyConnector.from_url(
url=self._proxy_url, loop=self._loop
),
loop=self._loop,
)
self._sio = socketio.AsyncClient(
http_session=session, **self._sio_client_kwargs
)
else:
self._sio = socketio.AsyncClient(**self._sio_client_kwargs)
def _initialize_loop_and_thread(self) -> asyncio.AbstractEventLoop:
"""Initialize a new asyncio event loop, start it in a background thread,
and create the ProxyClass instance bound to that loop.
"""
loop = asyncio.new_event_loop()
self.proxy = ProxyClass(
sio_client=self._sio,
loop=loop,
reconnect=self.connect,
)
self._thread = threading.Thread(
target=asyncio_loop_thread,
args=(loop,),