cSAXS/cdtools
0
0
Fork 0
mirror of https://github.com/cdtools-developers/cdtools.git synced 2026-09-10 05:22:41 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into holo-reconstructions

Browse Source
This commit is contained in:
gnzng
2025-03-11 09:30:24 -07:00
parent ac8db438f5 fce2add124
commit b4eee52aed
9 changed files with 164 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/main.yml
+86
View File
@@ -0,0 +1,86 @@
name: main
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
continue-on-error: true
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
pip install --upgrade pip
pip install -r requirements.txt
pip install -e . --no-deps
- name: Run tests
run: pytest
build-docs:
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
runs-on: ubuntu-latest
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: "pages"
cancel-in-progress: false
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.9'
- name: Install dependencies
run: |
pip install --upgrade pip
pip install -r requirements.txt
pip install sphinx sphinx_rtd_theme sphinx-argparse
pip install -e . --no-deps
- name: Build docs
working-directory: docs
run: |
make html
- name: List files in docs/build/html
run: |
ls -la docs/build/html
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: 'docs/build/html'
name: github-pages
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
with:
token: ${{ secrets.GITHUB_TOKEN }}
timeout: 600000
error_count: 10
reporting_interval: 5000
artifact_name: github-pages
preview: false
.gitignore
+3 -1
View File
@@ -5,7 +5,9 @@
*.egg-info
.pytest_cache
docs/build
docs/_build
build/*
dist
*/example_data/*
*.h5
*.h5
.DS_Store
docs/source/installation.rst
+8 -6
View File
@@ -8,25 +8,27 @@ The source code for CDTools is hosted on `Github`_. At the moment, the repositor
.. _`Github`: https://github.com/cdtools-developers/cdtools
The repository remains under active development as of late 2024.
The repository remains under active development as of early 2025.
Step 2: Install Dependencies
----------------------------
CDTools requires python 3.7 or greater.
CDTools is regularly tested with Python versions 3.8 to 3.12, so it is recommended to use one of these versions. In general, CDTools requires Python 3.7 or higher.
The major dependency for CDTools is pytorch (version 1.9.0 or greater). Because the details of the installation can vary depending on platform, GPU availability, etc, it is recommended that you follow the install instructions on `the pytorch site`_ to install pytorch before installing the remaining dependencies.
.. _`the pytorch site`: https://pytorch.org/get-started/locally/
If you manage your environment with conda, the remaining dependencies can be installed by running the following command in the top level directory of the package:
pytorch stopped supporting installation using conda for installation, so it is recommended continue the installation using pip.
.. code:: bash
$ conda install --file requirements.txt -c conda-forge
$ pip install -r requirements.txt
This will install all required dependencies and verify that they meet the pytorch version requirements. Additionally, several optional dependencies used for testing and documentation will also be installed. The full set of dependencies and minimum requirements are listed below is listed below.
CDTools is reguarly tested with the latest versions of the packages shown below.
This will install all required dependencies *except for pytorch*, as well as several optional dependencies which are used for the tests and documentation. The full set of dependencies are noted below.
Required dependencies:
* `numpy <http://www.numpy.org>`_ >= 1.0
requirements.txt
+1 -1
View File
@@ -1,7 +1,7 @@
numpy>=1.0
scipy>=1.0
matplotlib>=2.0 # 2.0 introduces better colormaps which are used by default
pytorch>=1.9.0 #1.9.0 implements support for autograd on indexed complex tensors
torch>=1.9.0 #1.9.0 implements support for autograd on indexed complex tensors
h5py>=2.1
python-dateutil
pytest
src/cdtools/datasets/ptycho_2d_dataset.py
+35 -8
View File
@@ -400,9 +400,40 @@ class Ptycho2DDataset(CDataset):
divisor_override=1)[0,0]
def remove_translations_mask(self, mask_remove):
"""Removes one or more translation positions, and their associated
properties, from the dataset using logical indexing.
This takes a 1D mask (boolean torch tensor) with the length
self.translations.shape[0] (i.e., the number of individual
translated points). Patterns, translations, and intensities
associated with indices that are "True" will be removed.
Parameters:
----------
mask_remove : 1D torch.tensor(dtype=torch.bool)
The boolean mask indicating which elements are to be removed from
the dataset. True indicates that the corresponding element will be
removed.
"""
# Check that the mask is the right size
if mask_remove.shape != t.Size([self.translations.shape[0]]):
raise ValueError(
'The mask must have the same length as the number of translations in the dataset.'
)
# Update patterns, translations, and intensities
self.patterns = self.patterns[~mask_remove]
self.translations = self.translations[~mask_remove]
if hasattr(self, 'intensities') and self.intensities is not None:
self.intensities = self.intensities[~mask_remove]
def crop_translations(self, roi):
"""Shrinks the range of translation positions that are analyzed
This deletes all diffraction patterns associated with x- and
y-translations that lie outside of a specified rectangular
region of interest. In essence, this operation crops the "relative
@@ -420,7 +451,7 @@ class Ptycho2DDataset(CDataset):
do not matter as long as roi[:2] and roi[2:] correspond with
the x and y coordinates, respectively.
"""
# Pull out the bounds of the ROI, ensuring that left < right and
# top < bottom
x_left, x_right = sorted(roi[:2])
@@ -441,9 +472,5 @@ class Ptycho2DDataset(CDataset):
'(i.e., patterns and translations will be empty).'
' Please redefine the bounds of the roi.')
# Update patterns and translations
self.patterns = self.patterns[inside_roi]
self.translations = self.translations[inside_roi]
if hasattr(self, 'intensities') and self.intensities is not None:
self.intensities = self.intensities[inside_roi]
# Remove translations outside the ROI
self.remove_translations_mask(~inside_roi)
src/cdtools/models/fancy_ptycho.py
+2 -3
View File
@@ -195,7 +195,7 @@ class FancyPtycho(CDIModel):
@classmethod
def from_dataset(cls,
dataset,
probe_size=None,
probe_shape=None,
randomize_ang=0,
n_modes=1,
n_obj_modes=1,
@@ -277,7 +277,7 @@ class FancyPtycho(CDIModel):
)
# Finally, initialize the probe and object using this information
if probe_size is None:
if probe_shape is None:
probe = tools.initializers.SHARP_style_probe(
dataset,
propagation_distance=propagation_distance,
@@ -288,7 +288,6 @@ class FancyPtycho(CDIModel):
dataset,
obj_basis,
probe_shape,
probe_size,
propagation_distance=propagation_distance,
)
src/cdtools/models/multislice_ptycho.py
+2 -3
View File
@@ -181,7 +181,7 @@ class MultislicePtycho(CDIModel):
dataset,
dz,
nz,
probe_size=None,
probe_shape=None,
randomize_ang=0,
n_modes=1,
n_obj_modes=1,
@@ -262,7 +262,7 @@ class MultislicePtycho(CDIModel):
)
# Finally, initialize the probe and object using this information
if probe_size is None:
if probe_shape is None:
probe = tools.initializers.SHARP_style_probe(
dataset,
propagation_distance=propagation_distance,
@@ -273,7 +273,6 @@ class MultislicePtycho(CDIModel):
dataset,
obj_basis,
probe_shape,
probe_size,
propagation_distance=propagation_distance,
)
src/cdtools/tools/initializers/initializers.py
+1 -1
View File
@@ -234,7 +234,7 @@ def gaussian_probe(dataset, basis, shape, sigma, propagation_distance=0, polariz
polarizer = dataset.polarizer.tolist()
analyzer = dataset.analyzer.tolist()
factors = [(math.cos(math.radians(polarizer[idx] - analyzer[idx])))**2 for idx in range(len(dataset)) if (abs(polarizer[idx] - analyzer[idx]) > 5)]
avg_intensities = [t.sum(dataset[idx][1]) / factor[idx] for idx in range(len(dataset))]
avg_intensities = [t.sum(dataset[idx][1]) / factors[idx] for idx in range(len(dataset))]
avg_intensity = t.mean(t.tensor(avg_intensities))
probe_intensity = t.sum(t.abs(probe)**2)
tests/test_datasets.py
+26 -2
View File
@@ -1,4 +1,4 @@
from cdtools.datasets import *
from cdtools.datasets import CDataset, Ptycho2DDataset
from cdtools.tools import data as cdtdata
import numpy as np
import torch as t
@@ -340,7 +340,31 @@ def test_Ptycho2DDataset_downsample(test_ptycho_cxis):
if dataset.background is not None:
assert np.allclose(np.array(dataset.background.shape) // factor,
np.array(copied_dataset.background.shape))
def test_Ptycho2DDataset_remove_translations_mask(ptycho_cxi_1):
# Grab dataset
cxi, expected = ptycho_cxi_1
dataset = Ptycho2DDataset.from_cxi(cxi)
copied_dataset = deepcopy(dataset)
# Test 1: Complain when the the mask is not the same shape as the pattern
# length
with pytest.raises(ValueError) as excinfo:
copied_dataset.remove_translations_mask(mask_remove=t.zeros(10))
assert ('The mask must have the same length') in str(excinfo.value)
# Test 2: Remove the mask from the dataset
mask_success = t.zeros(len(copied_dataset.patterns))
mask_success[1] = 1
mask_success[10] = 1
mask_success[-1] = 1
mask_success = mask_success.bool()
copied_dataset.remove_translations_mask(mask_remove=mask_success)
# test if the mask is removed and patterns length is correct
assert len(copied_dataset.patterns) == len(mask_success) - 3
def test_Ptycho2DDataset_crop_translations(ptycho_cxi_1):
# Grab dataset