diff --git a/README.md b/README.md index 2da2d8c..d7fb413 100644 --- a/README.md +++ b/README.md @@ -3,27 +3,32 @@ CDTools is a python library for ptychography and CDI reconstructions, using an Automatic Differentiation based approach. ```python -# imports from matplotlib import pyplot as plt from cdtools.datasets import Ptycho2DDataset from cdtools.models import FancyPtycho -# Load the file +# Load a data file dataset = Ptycho2DDataset.from_cxi('ptycho_data.cxi') -# Generate a model from the data +# Initialize a model from the data model = FancyPtycho.from_dataset(dataset) # Run a reconstruction for loss in model.Adam_optimize(10, dataset): print(model.report()) -# And look at the results! +# Save the results +model.save_to_h5('ptycho_results.h5', dataset) + +# And look at them! model.inspect(dataset) # See the reconstructed object, probe, etc. model.compare(dataset) # See how the simulated and measured patterns compare plt.show() ``` -Full installation instructions and documentation can be found [here](https://github.mit.edu/pages/Scattering/cdtools/). +Full installation instructions and documentation can be found [here](https://allevitan.github.io/cdtools-docs/). + + +CDTools was developed in the [photon scattering lab](https://scattering.mit.edu/) at MIT, and further development took place within the [computational x-ray imaging group](https://www.psi.ch/en/cxi) at PSI. If you would like to publish any work that uses CDTools, please contact [Abe Levitan](mailto:abraham.levitan@psi.ch). Have a wonderful day! diff --git a/conda_requirements.txt b/conda_requirements.txt deleted file mode 100644 index fd90733..0000000 --- a/conda_requirements.txt +++ /dev/null @@ -1,8 +0,0 @@ -numpy>=1.0 -scipy>=1.0 -matplotlib>=2.0 # 2.0 introduces better colormaps which are used by default -h5py>=2.1 -python-dateutil -pytest -sphinx - diff --git a/docs/source/examples.rst b/docs/source/examples.rst index da49e99..6e7392f 100644 --- a/docs/source/examples.rst +++ b/docs/source/examples.rst @@ -7,11 +7,11 @@ Included with the repository are a number of example scripts that demonstrate ho Inspect Dataset --------------- -The most important thing to be able to do is look at the raw data and confirm that it looks like you expect, hasn't been corrupted, etc. This script demonstrates how to load and look at your data, if it's stored as a compliant .cxi file. +We have some ptychography data in a .cxi file. How do we look at it, confirm that it looks like you expect, hasn't been corrupted, etc? This script demonstrates how to load and look at your data, if it's stored as a compliant .cxi file. .. literalinclude:: ../../examples/inspect_dataset.py -First, data is read into a dataset object. The dataset object knows how to generate a set of plots describing the basic information (diffraction patterns and nanomaps), which can be generated by running :code:`dataset.inspect`. +First, data is read into a Ptycho2DDataset object, which is a subclass of a pytorch dataset. This dataset, unsurprisingly, is for 2D ptycho scan data. Calling :code:`dataset.inspect` generates a plot showing an overview of the ptychography scan data, with a nanomap of integrated detector intensity on the left and detector images on the right. Simple Ptycho diff --git a/docs/source/installation.rst b/docs/source/installation.rst index cd69b26..5eef800 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -4,16 +4,19 @@ Installation Step 1: Download ---------------- -The source code for CDTools is hosted on it's `MIT github page`_. Access to the MIT github page can be granted to any member of the MIT community - if you are not a member, access can be arranged through a guest account on MIT's GitHub enterprise server. +The source code for CDTools is hosted on it's `MIT github page`_. Access to the MIT github page can be granted to any member of the MIT community. If you are not a member, access can also be arranged through a `private github tracking repository`_, by contacting `Abe Levitan `_. .. _`MIT github page`: https://github.mit.edu/Scattering/cdtools +.. _`private github tracking repository`: https://github.com/allevitan/cdtools -It is recommended that you clone the repository, rather than just downloading the contents, as it remains under development. Cloning the repository will allow you to easily get access to new updates, bug fixes, etc. +It is recommended that you clone the repository, rather than just downloading the contents, as it remains under active development. Cloning the repository will allow you to easily get access to new updates, bug fixes, etc. Step 2: Install Dependencies ---------------------------- -The major dependency for CDTools is pytorch, and because the details of the installation can vary depending on platform, GPU availability, etc, it is recommended that you follow the install instructions on `the pytorch site`_ to install pytorch before the remaining dependencies. +CDTools requires python 3.7 or greater. + +The major dependency for CDTools is pytorch (version 1.9.0 or greater). Because the details of the installation can vary depending on platform, GPU availability, etc, it is recommended that you follow the install instructions on `the pytorch site`_ to install pytorch before installing the remaining dependencies. .. _`the pytorch site`: https://pytorch.org/get-started/locally/ @@ -21,11 +24,11 @@ If you manage your environment with conda, the remaining dependencies can be ins .. code:: bash - $ conda install --file conda_requirements.txt + $ conda install --file requirements.txt -c conda-forge -This will install all dependencies which are available from the main repos, including some optional dependencies for the tests and docs. Following this, any final remining dependencies can be installed from conda-forge or pip. For convenience, the full set of dependencies are noted below: +This will install all required dependencies *except for pytorch*, as well as several optional dependencies which are used for the tests and documentation. The full set of dependencies are noted below. -CDTools depends on the following packages: +Required dependencies: * `numpy `_ >= 1.0 * `scipy `_ >= 1.0 @@ -34,32 +37,32 @@ CDTools depends on the following packages: * `python-dateutil `_ * `h5py `_ >= 2.1 -And has optional dependencies on +Optional dependencies: * `pytest `_ - * `sphinx `_ + * `pooch `_ + * `sphinx `_ >= 4.3.0 * `sphinx-argparse `_ - * `sphinx_rtd_theme `_ + * `sphinx_rtd_theme `_ >= 0.5.1 -Finally, the optional depencency on pytest enables the tests to be run to confirm a successful installation. Sphinx and sphinx-argparse are only required if you plan on building the documentation. - -Finally, CDTools is NOT python 2 compatible. +The file "example_environment.yml", included in the repository's top level directory, contains an example of an environment with all dependencies properly installed on a linux machine with a GPU, circa early 2024. Step 3: Install --------------- -To install in CDTools in developer mode (recommended, to allow any updates to be pushed immediately through your system), run the following command from the top level directory (the directory including the setup.py file) - -.. code:: bash - - $ pip install -e . --no-deps - -To install normally (not in developer mode), run: +To install CDTools, run the following command from the top level directory (the directory including the setup.py file). .. code:: bash $ pip install . --no-deps + +This will install a copy of the code, as it exists at the moment of installation. If you would prefer for changes to the code to propagate to the installed version without reinstalling, install the package in developer mode: + +.. code:: bash + + $ pip install -e . --no-deps + Step 4: Run The Tests --------------------- @@ -71,4 +74,4 @@ To ensure that the installation has worked correctly, it is recommended that you $ pytest -If any tests fail, make sure that you have all the noted dependencies properly installed, and particularly that pytorch was installed with FFT support. If so, `shoot me an email `_ or open an issue on the github page and we'll get to the bottom of it. +If any tests fail, make sure that you have all the noted dependencies properly installed. If you do, and things still aren't working, `send me (Abe Levitan) an email `_ and we'll get to the bottom of it. diff --git a/docs/source/intro.rst b/docs/source/intro.rst index cb5e0ea..75fe622 100644 --- a/docs/source/intro.rst +++ b/docs/source/intro.rst @@ -7,33 +7,36 @@ Introduction to CDTools Introduction to CDTools ----------------------- -CDTools is a python library for ptychography and CDI reconstructions, using an Automatic Differentiation (AD) based approach. +CDTools is a python library for ptychography and CDI reconstructions, using an automatic differentiation based approach. .. code-block:: python - # imports - import cdtools from matplotlib import pyplot as plt - - # Load the file - dataset = cdtools.datasets.Ptycho2DDataset.from_cxi('ptycho_data.cxi') - - # Generate a model from the data - model = cdtools.models.SimplePtycho.from_dataset(dataset) - + from cdtools.datasets import Ptycho2DDataset + from cdtools.models import FancyPtycho + + # Load a data file + dataset = Ptycho2DDataset.from_cxi('ptycho_data.cxi') + + # Initialize a model from the data + model = FancyPtycho.from_dataset(dataset) + # Run a reconstruction - for loss in model.Adam_optimize(20, dataset): - print(model.report()) - - # And look at the results! - model.inspect(dataset) - model.compare(dataset) + for loss in model.Adam_optimize(10, dataset): + print(model.report()) + + # Save the results + model.save_to_h5('ptycho_results.h5', dataset) + + # And look at them! + model.inspect(dataset) # See the reconstructed object, probe, etc. + model.compare(dataset) # See how the simulated and measured patterns compare plt.show() -CDTools makes it simple to load and inspect data stored in .cxi files using python scripts. Several reconstruction models for common geometries are included "out of the box". For more advanced users, it includes a modular functions for AD ptychography. These can can be used to construct new forward models. +CDTools makes it simple to load and inspect data stored in .cxi files. Reconstruction models for common geometries are included "out of the box". For more advanced users, it includes a collection of differentiable functions which can beused to construct new forward models. -The high-level interface to CDTools is built on a lower level "three-legged stool". This consists of tools to access stored data, tools to visualize data and reconstructions, and tools that implement basic operations relevant to coherent diffraction. All of these tools can be used directly alongside the high-level interface, when needed. +The high-level interface to CDTools - datasets and models - is built on a set oflower-level tools. These include functions for accessing stored data in .cxi files, tools to visualize data and reconstructions, and tools that implement basic operations - such as light propagation - relevant to coherent diffraction. These functions can be used alongside the high-level interface, when needed. Enough blabber. If you're interested, read the docs! diff --git a/example_environment.yml b/example_environment.yml new file mode 100644 index 0000000..46767a2 --- /dev/null +++ b/example_environment.yml @@ -0,0 +1,181 @@ +name: democdtoolsenv +channels: + - pytorch + - nvidia + - conda-forge + - defaults +dependencies: + - _libgcc_mutex=0.1=main + - _openmp_mutex=5.1=1_gnu + - alabaster=0.7.16=pyhd8ed1ab_0 + - babel=2.14.0=pyhd8ed1ab_0 + - blas=1.0=mkl + - brotli=1.0.9=h9c3ff4c_4 + - bzip2=1.0.8=h7b6447c_0 + - c-ares=1.19.1=h5eee18b_0 + - ca-certificates=2024.2.2=hbcca054_0 + - certifi=2024.2.2=pyhd8ed1ab_0 + - charset-normalizer=2.0.4=pyhd3eb1b0_0 + - colorama=0.4.6=pyhd8ed1ab_0 + - commonmark=0.9.1=py_0 + - contourpy=1.2.0=py311hdb19cb5_0 + - cuda-cudart=12.1.105=0 + - cuda-cupti=12.1.105=0 + - cuda-libraries=12.1.0=0 + - cuda-nvrtc=12.1.105=0 + - cuda-nvtx=12.1.105=0 + - cuda-opencl=12.3.101=0 + - cuda-runtime=12.1.0=0 + - cycler=0.12.1=pyhd8ed1ab_0 + - dbus=1.13.18=hb2f20db_0 + - docutils=0.20.1=py311h38be061_3 + - exceptiongroup=1.2.0=pyhd8ed1ab_2 + - expat=2.2.10=h9c3ff4c_0 + - ffmpeg=4.3=hf484d3e_0 + - filelock=3.13.1=py311h06a4308_0 + - fontconfig=2.14.1=hef1e5e3_0 + - fonttools=4.25.0=pyhd3eb1b0_0 + - freetype=2.12.1=h4a9f257_0 + - future=1.0.0=pyhd8ed1ab_0 + - glib=2.78.4=h6a678d5_0 + - glib-tools=2.78.4=h6a678d5_0 + - gmp=6.2.1=h295c915_3 + - gmpy2=2.1.2=py311hc9b5ff0_0 + - gnutls=3.6.15=he1e5248_0 + - gst-plugins-base=1.14.1=h6a678d5_1 + - gstreamer=1.14.1=h5eee18b_1 + - h5py=3.9.0=py311hdd6beaf_0 + - hdf5=1.12.1=h2b7332f_3 + - icu=58.2=hf484d3e_1000 + - idna=3.4=py311h06a4308_0 + - imagesize=1.4.1=pyhd8ed1ab_0 + - importlib-metadata=7.0.1=pyha770c72_0 + - iniconfig=2.0.0=pyhd8ed1ab_0 + - intel-openmp=2023.1.0=hdb19cb5_46306 + - jinja2=3.1.3=py311h06a4308_0 + - jpeg=9e=h5eee18b_1 + - kiwisolver=1.4.4=py311h6a678d5_0 + - krb5=1.20.1=h143b758_1 + - lame=3.100=h7b6447c_0 + - lcms2=2.12=h3be6417_0 + - ld_impl_linux-64=2.38=h1181459_1 + - lerc=3.0=h295c915_0 + - libclang=10.0.1=default_hb85057a_2 + - libcublas=12.1.0.26=0 + - libcufft=11.0.2.4=0 + - libcufile=1.8.1.2=0 + - libcurand=10.3.4.107=0 + - libcurl=8.5.0=h251f7ec_0 + - libcusolver=11.4.4.55=0 + - libcusparse=12.0.2.55=0 + - libdeflate=1.17=h5eee18b_1 + - libedit=3.1.20230828=h5eee18b_0 + - libev=4.33=h516909a_1 + - libevent=2.1.12=hdbd6064_1 + - libffi=3.4.4=h6a678d5_0 + - libgcc-ng=11.2.0=h1234567_1 + - libgfortran-ng=13.2.0=h69a702a_0 + - libgfortran5=13.2.0=ha4646dd_0 + - libglib=2.78.4=hdc74915_0 + - libgomp=11.2.0=h1234567_1 + - libiconv=1.16=h7f8727e_2 + - libidn2=2.3.4=h5eee18b_0 + - libjpeg-turbo=2.0.0=h9bf148f_0 + - libllvm10=10.0.1=he513fc3_3 + - libnghttp2=1.57.0=h2d74bed_0 + - libnpp=12.0.2.50=0 + - libnvjitlink=12.1.105=0 + - libnvjpeg=12.1.1.14=0 + - libpng=1.6.39=h5eee18b_0 + - libpq=12.17=hdbd6064_0 + - libssh2=1.10.0=hdbd6064_2 + - libstdcxx-ng=11.2.0=h1234567_1 + - libtasn1=4.19.0=h5eee18b_0 + - libtiff=4.5.1=h6a678d5_0 + - libunistring=0.9.10=h27cfd23_0 + - libuuid=1.41.5=h5eee18b_0 + - libwebp-base=1.3.2=h5eee18b_0 + - libxcb=1.15=h7f8727e_0 + - libxkbcommon=1.0.1=hfa300c1_0 + - libxml2=2.9.14=h74e7548_0 + - llvm-openmp=14.0.6=h9e868ea_0 + - lz4-c=1.9.4=h6a678d5_0 + - markupsafe=2.1.3=py311h5eee18b_0 + - matplotlib=3.8.0=py311h06a4308_0 + - matplotlib-base=3.8.0=py311ha02d727_0 + - mkl=2023.1.0=h213fc3f_46344 + - mkl-service=2.4.0=py311h5eee18b_1 + - mkl_fft=1.3.8=py311h5eee18b_0 + - mkl_random=1.2.4=py311hdb19cb5_0 + - mpc=1.1.0=h10f8cd9_1 + - mpfr=4.0.2=hb69a4c5_1 + - mpmath=1.3.0=py311h06a4308_0 + - munkres=1.1.4=pyh9f0ad1d_0 + - ncurses=6.4=h6a678d5_0 + - nettle=3.7.3=hbbd107a_1 + - networkx=3.1=py311h06a4308_0 + - nspr=4.35=h6a678d5_0 + - nss=3.89.1=h6a678d5_0 + - numpy=1.26.3=py311h08b1b3b_0 + - numpy-base=1.26.3=py311hf175353_0 + - openh264=2.1.1=h4ff587b_0 + - openjpeg=2.4.0=h3ad879b_0 + - openssl=3.0.13=h7f8727e_0 + - packaging=23.2=pyhd8ed1ab_0 + - pcre2=10.42=hebb0a14_0 + - pillow=10.2.0=py311h5eee18b_0 + - pip=23.3.1=py311h06a4308_0 + - platformdirs=4.2.0=pyhd8ed1ab_0 + - pluggy=1.4.0=pyhd8ed1ab_0 + - ply=3.11=py_1 + - pooch=1.8.1=pyhd8ed1ab_0 + - pygments=2.17.2=pyhd8ed1ab_0 + - pyparsing=2.4.7=pyhd8ed1ab_1 + - pyqt=5.15.10=py311h6a678d5_0 + - pyqt5-sip=12.13.0=py311h5eee18b_0 + - pytest=8.0.1=pyhd8ed1ab_1 + - python=3.11.7=h955ad1f_0 + - python-dateutil=2.8.2=pyhd8ed1ab_0 + - python_abi=3.11=2_cp311 + - pytorch=2.2.1=py3.11_cuda12.1_cudnn8.9.2_0 + - pytorch-cuda=12.1=ha16c6d3_5 + - pytorch-mutex=1.0=cuda + - pytz=2024.1=pyhd8ed1ab_0 + - pyyaml=6.0.1=py311h5eee18b_0 + - qt-main=5.15.2=h327a75a_7 + - readline=8.2=h5eee18b_0 + - requests=2.31.0=py311h06a4308_1 + - scipy=1.11.4=py311h08b1b3b_0 + - setuptools=68.2.2=py311h06a4308_0 + - sip=6.7.12=py311h6a678d5_0 + - six=1.16.0=pyh6c4a22f_0 + - snowballstemmer=2.2.0=pyhd8ed1ab_0 + - sphinx=7.2.6=pyhd8ed1ab_0 + - sphinx-argparse=0.4.0=pyhd8ed1ab_0 + - sphinx_rtd_theme=2.0.0=pyha770c72_0 + - sphinxcontrib-applehelp=1.0.8=pyhd8ed1ab_0 + - sphinxcontrib-devhelp=1.0.6=pyhd8ed1ab_0 + - sphinxcontrib-htmlhelp=2.0.5=pyhd8ed1ab_0 + - sphinxcontrib-jquery=4.1=pyhd8ed1ab_0 + - sphinxcontrib-jsmath=1.0.1=pyhd8ed1ab_0 + - sphinxcontrib-qthelp=1.0.7=pyhd8ed1ab_0 + - sphinxcontrib-serializinghtml=1.1.10=pyhd8ed1ab_0 + - sqlite=3.41.2=h5eee18b_0 + - sympy=1.12=py311h06a4308_0 + - tbb=2021.8.0=hdb19cb5_0 + - tk=8.6.12=h1ccaba5_0 + - tomli=2.0.1=pyhd8ed1ab_0 + - torchaudio=2.2.1=py311_cu121 + - torchtriton=2.2.0=py311 + - torchvision=0.17.1=py311_cu121 + - tornado=6.3.3=py311h5eee18b_0 + - typing_extensions=4.9.0=py311h06a4308_1 + - tzdata=2023d=h04d1e81_0 + - urllib3=2.1.0=py311h06a4308_0 + - wheel=0.41.2=py311h06a4308_0 + - xz=5.4.5=h5eee18b_0 + - yaml=0.2.5=h7b6447c_0 + - zipp=3.17.0=pyhd8ed1ab_0 + - zlib=1.2.13=h5eee18b_0 + - zstd=1.5.5=hc292b87_0 + diff --git a/examples/simple_ptycho_model.py b/examples/simple_ptycho_model.py deleted file mode 100644 index 8f96984..0000000 --- a/examples/simple_ptycho_model.py +++ /dev/null @@ -1,125 +0,0 @@ -import numpy as np -import torch as t -from cdtools.models import CDIModel -from cdtools import tools - -__all__ = ['SimplePtycho'] - -class SimplePtycho(CDIModel): - """A simple ptychography model for exploring ideas and extensions""" - - def __init__(self, probe_basis, probe_guess, obj_guess, - min_translation = [0,0]): - # We have to initialize the Module - super(SimplePtycho,self).__init__() - - # We first save the relevant information - self.min_translation = t.Tensor(min_translation) - self.probe_basis = t.Tensor(probe_basis) - - # We rescale the probe so it learns at the same rate as the object - self.probe_norm = t.max(tools.cmath.cabs(probe_guess.to(t.float32))) - self.probe = t.nn.Parameter(probe_guess.to(t.float32) - / self.probe_norm) - self.obj = t.nn.Parameter(obj_guess.to(t.float32)) - - - - @classmethod - def from_dataset(cls, dataset): - # First, load the information from the dataset - wavelength = dataset.wavelength - det_basis = dataset.detector_geometry['basis'] - det_shape = dataset[0][1].shape - distance = dataset.detector_geometry['distance'] - (indices, translations), patterns = dataset[:] - - # Then, generate the probe geometry - ewg = tools.initializers.exit_wave_geometry - probe_basis, probe_shape, det_slice = ewg(det_basis, - det_shape, - wavelength, - distance) - - # Next generate the object geometry - pix_translations = tools.interactions.translations_to_pixel( - probe_basis, translations) - obj_size, min_translation = tools.initializers.calc_object_setup( - probe_shape, pix_translations) - - # Finally, initialize the probe and object using this information - probe = tools.initializers.SHARP_style_probe(dataset, - probe_shape, - det_slice) - - obj = t.ones(obj_size+(2,)) - - return cls(probe_basis, probe, obj, min_translation=min_translation) - - - def interaction(self, index, translations): - pix_trans = tools.interactions.translations_to_pixel(self.probe_basis, - translations) - pix_trans -= self.min_translation - return tools.interactions.ptycho_2D_round(self.probe_norm * self.probe, - self.obj, - pix_trans) - - def forward_propagator(self, wavefields): - return tools.propagators.far_field(wavefields) - - def measurement(self, wavefields): - return tools.measurements.intensity(wavefields) - - def loss(self, sim_data, real_data): - return tools.losses.amplitude_mse(real_data, sim_data) - - - def to(self, *args, **kwargs): - super(SimplePtycho, self).to(*args, **kwargs) - self.min_translation = self.min_translation.to(*args,**kwargs) - self.probe_basis = self.probe_basis.to(*args,**kwargs) - self.probe_norm = self.probe_norm.to(*args,**kwargs) - - - - plot_list = [ - ('Probe Amplitude', - lambda self, fig: tools.plotting.plot_amplitude(self.probe, fig=fig, basis=self.probe_basis)), - ('Probe Phase', - lambda self, fig: tools.plotting.plot_phase(self.probe, fig=fig, basis=self.probe_basis)), - ('Object Amplitude', - lambda self, fig: tools.plotting.plot_amplitude(self.obj, fig=fig, basis=self.probe_basis)), - ('Object Phase', - lambda self, fig: tools.plotting.plot_phase(self.obj, fig=fig, basis=self.probe_basis)) - ] - - def save_results(self): - probe = tools.cmath.torch_to_complex(self.probe.detach().cpu()) - probe = probe * self.probe_norm.detach().cpu().numpy() - obj = tools.cmath.torch_to_complex(self.obj.detach().cpu()) - return {'probe':probe,'obj':obj} - - -if __name__ == '__main__': - - from basic_ptycho_dataset import BasicPtychoDataset - from h5py import File - from matplotlib import pyplot as plt - - filename = 'example_data/lab_ptycho_data.cxi' - with File(filename, 'r') as f: - dataset = BasicPtychoDataset.from_cxi(f) - - - model = SimplePtycho.from_dataset(dataset) - - #model.to(device='cuda') - #dataset.get_as(device='cuda') - - for i, loss in enumerate(model.Adam_optimize(100, dataset)): - model.inspect(dataset) - print(i,loss) - - model.compare(dataset) - plt.show() diff --git a/requirements.txt b/requirements.txt index d3242fa..6afb860 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,6 +5,7 @@ pytorch>=1.9.0 #1.9.0 implements support for autograd on indexed complex tensors h5py>=2.1 python-dateutil pytest -sphinx +pooch +sphinx>=4.3.0 # Fixes a bug with bulleted lists sphinx-argparse -sphinx_rtd_theme +sphinx_rtd_theme>=0.5.1 # Fixes a bug with bulleted lists diff --git a/setup.py b/setup.py index 2e29fae..8868f61 100644 --- a/setup.py +++ b/setup.py @@ -8,7 +8,7 @@ setuptools.setup( version="0.2.0", python_requires='>3.7', # recommended minimum version for pytorch author="Abe Levitan", - author_email="alevitan@mit.edu", + author_email="abraham.levitan@psi.ch", description="Tools for coherent diffractive imaging and ptychography", long_description=long_description, long_description_content_type="text/markdown", @@ -21,8 +21,15 @@ setuptools.setup( "torch>=1.9.0", #1.9.0 supports autograd on indexed complex tensors "h5py>=2.1"], extras_require={ - 'tests': ["pytest"], - 'docs': ["sphinx","sphinx-argparse","sphinx_rtd_theme"] + 'tests': [ + "pytest", + "pooch", + ], + 'docs': [ + "sphinx>=4.3.0", + "sphinx-argparse", + "sphinx_rtd_theme>=0.5.1" + ] }, package_dir={"": "src"}, packages=setuptools.find_packages("src"), diff --git a/src/cdtools/models/fancy_ptycho.py b/src/cdtools/models/fancy_ptycho.py index d5d24d1..7c6d0b8 100644 --- a/src/cdtools/models/fancy_ptycho.py +++ b/src/cdtools/models/fancy_ptycho.py @@ -12,10 +12,6 @@ from copy import copy __all__ = ['FancyPtycho'] -# TODO The information on bases is all wrong when the probe_fourier_crop is -# used, this affects display but also saveout of data and maybe even stuff -# like the probe translation simulation - class FancyPtycho(CDIModel): def __init__(self, diff --git a/src/cdtools/models/simple_ptycho.py b/src/cdtools/models/simple_ptycho.py index 26da4d5..4c6fea4 100644 --- a/src/cdtools/models/simple_ptycho.py +++ b/src/cdtools/models/simple_ptycho.py @@ -17,26 +17,28 @@ class SimplePtycho(CDIModel): """ - def __init__(self, wavelength, detector_geometry, - probe_basis, detector_slice, - probe_guess, obj_guess, min_translation = [0,0], - surface_normal=np.array([0.,0.,1.]), mask=None): + def __init__( + self, + wavelength, + detector_geometry, + probe_basis, + probe_guess, + obj_guess, + min_translation = [0,0], + ): + # We initialize the superclass super(SimplePtycho,self).__init__() + + # We register all the constants, like wavelength, as buffers. + # This lets the model hook into some nice pytorch features, like + # using model.to, and in principle should make easier to broadcast + # across multiple GPUs self.register_buffer('wavelength', t.as_tensor(wavelength)) self.store_detector_geometry(detector_geometry) self.register_buffer('min_translation', t.as_tensor(min_translation)) self.register_buffer('probe_basis', t.as_tensor(probe_basis)) - self.detector_slice = copy(detector_slice) - - self.register_buffer('surface_normal', t.as_tensor(surface_normal)) - - - if mask is None: - self.register_buffer('mask', None) - else: - self.register_buffer('mask', t.as_tensor(mask, dtype=t.bool)) probe_guess = t.tensor(probe_guess, dtype=t.complex64) obj_guess = t.tensor(obj_guess, dtype=t.complex64) @@ -45,39 +47,27 @@ class SimplePtycho(CDIModel): # object self.register_buffer('probe_norm', t.max(t.abs(probe_guess))) + # And we store the probe and object guesses as parameters, so + # they can get optimized by pytorch self.probe = t.nn.Parameter(probe_guess / self.probe_norm) self.obj = t.nn.Parameter(obj_guess) @classmethod def from_dataset(cls, dataset): + + # We get the key geometry information from the dataset wavelength = dataset.wavelength det_basis = dataset.detector_geometry['basis'] det_shape = dataset[0][1].shape distance = dataset.detector_geometry['distance'] - # always do this on the cpu - get_as_args = dataset.get_as_args - dataset.get_as(device='cpu') + # We extract all the diffraction patterns and translations (indices, translations), patterns = dataset[:] - dataset.get_as(*get_as_args[0],**get_as_args[1]) - center = tools.image_processing.centroid(t.sum(patterns,dim=0)) - - # Then, generate the probe geometry from the dataset + # Then, we generate the probe geometry ewg = tools.initializers.exit_wave_geometry - probe_basis, probe_shape, det_slice = ewg(det_basis, - det_shape, - wavelength, - distance, - center=center) - - if hasattr(dataset, 'sample_info') and \ - dataset.sample_info is not None and \ - 'orientation' in dataset.sample_info: - surface_normal = dataset.sample_info['orientation'][2] - else: - surface_normal = np.array([0.,0.,1.]) + probe_basis = ewg(det_basis, det_shape, wavelength, distance) # Next generate the object geometry from the probe geometry and # the translations @@ -89,36 +79,36 @@ class SimplePtycho(CDIModel): probe_shape, pix_translations) - # Finally, initialize the probe and object using this information - probe = tools.initializers.SHARP_style_probe(dataset, probe_shape, det_slice) - + # Finally, initialize the probe and object using this information + probe = tools.initializers.SHARP_style_probe( + dataset, + probe_shape, + det_slice + ) obj = t.ones(obj_size).to(dtype=t.complex64) - det_geo = dataset.detector_geometry - - if hasattr(dataset, 'mask') and dataset.mask is not None: - mask = dataset.mask.to(t.bool) - else: - mask = None - - return cls(wavelength, - det_geo, - probe_basis, - det_slice, - probe, - obj, - min_translation=min_translation, - mask=mask, - surface_normal=surface_normal) + return cls( + wavelength, + dataset.detector_geometry, + probe_basis, + det_slice, + probe, + obj, + min_translation=min_translation + ) def interaction(self, index, translations): + + # We map from real-space to pixel-space units pix_trans = tools.interactions.translations_to_pixel( self.probe_basis, - translations, - surface_normal=self.surface_normal) - + translation) pix_trans -= self.min_translation + + # This function gets the proper, pixel-accurate crops of the + # object array, and uses a sinc-interpolated subpixel shift of + # the probe to reach subpixel accuracy return tools.interactions.ptycho_2D_round( self.probe_norm * self.probe, self.obj, @@ -136,12 +126,13 @@ class SimplePtycho(CDIModel): def measurement(self, wavefields): return tools.measurements.intensity(wavefields, detector_slice=self.detector_slice) - - - def loss(self, real_data, sim_data, mask=None): - return tools.losses.amplitude_mse(real_data, sim_data, mask=mask) + def loss(self, real_data, sim_data): + return tools.losses.amplitude_mse(real_data, sim_data) + + + # This lists all the plots to display on a call to model.inspect() plot_list = [ ('Probe Amplitude', lambda self, fig: p.plot_amplitude(self.probe, fig=fig, basis=self.probe_basis)), diff --git a/src/cdtools/tools/interactions/interactions.py b/src/cdtools/tools/interactions/interactions.py index f44cc45..0bff167 100644 --- a/src/cdtools/tools/interactions/interactions.py +++ b/src/cdtools/tools/interactions/interactions.py @@ -170,7 +170,7 @@ def project_translations_to_sample(sample_basis, translations): # Must do this all in pytorch unfortunately # First we calculate the surface normal for the projection onto the sample - surface_normal = t.cross(sample_basis[:,1],sample_basis[:,0]) + surface_normal = t.cross(sample_basis[:,1],sample_basis[:,0], dim=-1) surface_normal /= t.norm(surface_normal) diff --git a/src/cdtools/tools/plotting/plotting.py b/src/cdtools/tools/plotting/plotting.py index 3043778..5b08b68 100644 --- a/src/cdtools/tools/plotting/plotting.py +++ b/src/cdtools/tools/plotting/plotting.py @@ -205,7 +205,8 @@ def plot_image(im, plot_func=lambda x: x, fig=None, basis=None, view_basis='orth # Holy cow, this works! transform_matrix = \ - np.linalg.lstsq(np_view_basis[:,::-1], np_basis[:,::-1])[0] + np.linalg.lstsq(np_view_basis[:,::-1], np_basis[:,::-1], + rcond=None)[0] [[a,c],[b,d]] = transform_matrix transform = mtransforms.Affine2D.from_values(a,b,c,d,0,0) diff --git a/src/cdtools/tools/propagators/propagators.py b/src/cdtools/tools/propagators/propagators.py index a5741b9..c7edecd 100644 --- a/src/cdtools/tools/propagators/propagators.py +++ b/src/cdtools/tools/propagators/propagators.py @@ -507,7 +507,7 @@ def generate_generalized_angular_spectrum_propagator(shape, basis, wavelength, o # which was a real problem that showed up for electron ptycho b1_dir = basis[:,0] / t.linalg.norm(basis[:,0]) b2_dir = basis[:,1] / t.linalg.norm(basis[:,1]) - perpendicular_dir = t.cross(b1_dir,b2_dir) + perpendicular_dir = t.cross(b1_dir, b2_dir, dim=-1) # Note that we cannot use in-place operations if we want to be able to # use automatic differentiation successfully perpendicular_dir = perpendicular_dir / t.linalg.norm(perpendicular_dir)