Improved ClusterFile API (#358)
Build on RHEL9 / build (push) Successful in 2m49s
Build on RHEL8 / build (push) Successful in 3m23s
Run tests using data on local RHEL8 / build (push) Successful in 4m23s
Build on local RHEL8 / build (push) Successful in 3m5s

Improved API for ClusterFile

- read_frame now returns a std::optional<ClusterVector> to differentiate
between an empty frame and end of file
- f.frames() produces an iterator over frames
- f.chunks() produces an iterator reading chunks of clusters
This commit is contained in:
Erik Fröjdh authored and GitHub committed 2026-09-14 16:01:25 +02:00
1 parent 86db375948
commit b0875106e3
16 files changed
+1687 -308

No files matched your search

+45 -9
View File
@@ -89,19 +89,55 @@ def ClusterFileSink(clusterfindermt, cluster_file, dtype=np.int32):
def ClusterFile(fname, cluster_size=(3,3), dtype=np.int32, chunk_size = 1000, mode = "r"):
"""
Factory function to create a ClusterFile object. Provides a cleaner syntax for
the templated ClusterFile in C++.
"""Create a reader or writer for a binary cluster file.
Parameters
----------
fname : path-like
Cluster file to open.
cluster_size : tuple[int, int], default=(3, 3)
Cluster dimensions stored in the file.
dtype : numpy dtype, default=numpy.int32
Data type of the cluster values stored in the file.
chunk_size : int, default=1000
Maximum number of selected clusters returned by ``chunks()`` and
default iteration. Must be positive when iterating over chunks.
mode : {"r", "w", "a"}, default="r"
Open for reading, truncate and write, or append, respectively.
Returns
-------
ClusterFile
The compiled ClusterFile specialization matching ``cluster_size`` and
``dtype``.
Notes
-----
The file format contains no cluster shape or data-type metadata. Supplying
values that do not match the file causes its bytes to be interpreted
incorrectly. Use ``frames()`` to iterate over complete frames, including
empty or fully filtered frames with their stored frame numbers. Use
``chunks()`` or ``chunks(chunk_size)`` to iterate over selected clusters
in batches. Chunks may combine frames, so their frame number is not
reliable per-cluster metadata.
Iterators consume the current file position without rewinding; use one
traversal at a time. Each result owns its storage and remains valid after
advancing the iterator or closing the file.
Examples
--------
.. code-block:: python
from aare import ClusterFile
with ClusterFile("clusters.clust", cluster_size=(3,3), dtype=np.int32) as cf:
# cf is now a ClusterFile_Cluster3x3i object but you don't need to know that.
for clusters in cf:
# Loop over clusters in chunks of 1000
# The type of clusters will be a ClusterVector_Cluster3x3i in this case
with ClusterFile(
"clusters.clust", cluster_size=(3, 3), dtype=np.int32
) as cf:
for clusters in cf.chunks():
# Process clusters in chunks of at most 1000.
...
"""