Commit Graph

58 Commits

Author SHA1 Message Date
Clemens Vonrhein 4cf18ceff4 Mechanism to read bitshuffle-compressed pixel-mask data - by
loading/using the normal bitshuffle source from

  https://github.com/kiyo-masui/bitshuffle

(see Makefile).
2024-12-16 15:05:38 +00:00
Clemens Vonrhein 44ba3cd278 Provide a mechanism to distinguish between an image number (as
understood on the XDS side) and the image ordinal (as stored in the
HDF data).

The 2D data arrays are not necessarily at image_nr_low=1 ;-)
2024-12-16 14:57:32 +00:00
CV-GPhL ca8806c912 Merge pull request #2 from CV-GPhL/CV-GPhL-patch-1
Update plugin.c to write compilation time if available/set
2024-03-14 16:43:22 +01:00
CV-GPhL 163d5bdfb1 Update plugin.c to write compilation time if available/set 2024-03-14 16:42:52 +01:00
CV-GPhL ca2c0ee0e7 Merge pull request #1 from DiamondLightSource/master
Catching up 20240223
2024-02-23 12:55:55 +01:00
Graeme Winter 808f5fbd42 Do not depend on the strings being NULL terminated (#29)
Fixes #28

Instead make them NULL terminated by reading one longer into buffer
2023-08-09 09:42:16 +01:00
Graeme Winter 5d0b7bd104 New flag needed 2022-03-11 14:47:22 +00:00
Clemens Vonrhein 77d2b84957 Better handling of (usually) unsigned data arrays when calling routine
to convert to (signed) int and apply pixel mask. The environment
variable DURIN_RESET_UNMASKED_PIXEL can now be used to set non-masked
saturated pixels in order to process them correctly with e.g. XDS.
2021-04-08 14:46:48 +01:00
Clemens Vonrhein 0f27832f78 Resolved merge conflicts 2021-04-06 12:33:21 +01:00
Graeme Winter 300a980551 Use blocks for imports to prevent reorder; finish clang-format 2020-10-27 09:55:39 +00:00
Graeme Winter 4a6a75af70 clang-format err.* 2020-10-27 09:55:39 +00:00
Graeme Winter b36870935a clang-format file.* 2020-10-27 09:55:39 +00:00
Graeme Winter d73d6d70cb clang-format test.c 2020-10-27 09:55:39 +00:00
Graeme Winter 513bb10bf4 clang-format plugin 2020-10-27 09:55:39 +00:00
Graeme Winter 6ffef7d7c2 Typo; some comment formats 2020-10-27 09:55:39 +00:00
Graeme Winter e3cad6a7c3 Switch to c99 from c89 default - fixes #20
auggested by @jcbollinger
2020-10-26 15:58:25 +00:00
Clemens Vonrhein 9e7d609f42 allow for image offset via DURIN_IMAGE_NUMBER_OFFSET environment
variable
2020-06-12 15:57:14 +01:00
Graeme Winter 9b3236b084 First cut: use H5_USE_110_API (#19)
For #18 - compile against 1.10 API (may need to do more work to make sure this
does what we actually want)
2020-06-08 10:55:24 +01:00
Graeme Winter f25a73c667 Set to NULL after free; though fixes nothing 2020-04-02 14:21:52 +01:00
Tom Schoonjans 6165d14c31 Force linking against HDF5 shared library (#16) 2020-01-22 06:40:22 +00:00
Tom Schoonjans 19b16ad2a1 Fix sprintf usage (#15) 2020-01-21 10:52:49 +00:00
Graeme Winter 4e2b0c7506 Point at release 2019-05-08 09:28:59 +01:00
Charles Mita 34a38e15fd Update README.md 2019v1 2019-01-29 19:58:27 +00:00
Charles Mita 8eaacaed77 Set plugin target as the default target in Makefile
This makes "make" build only the most useful software by default.

Building the test_plugin target requires fortran and openmp, which are
otherwise not required for the durin plugin itself, and is not of
interest to most building the software.
2019-01-29 14:11:20 +00:00
Charles Mita 2f6348e8d2 Replace placeholders in LICENSE with the equivalent terms 2019-01-29 13:31:58 +00:00
Charles Mita ffebc60cdb Stop errors when no pixel mask is provided 2019-01-29 13:31:58 +00:00
graeme-winter c129d095e0 CFLAGS to -fPIC 2019-01-18 15:00:28 +00:00
graeme-winter 1e202a7429 more detail on how to use 2018-10-11 10:55:58 +01:00
Charles Mita 1929c1149b Merge branch 'chunk_read' 2018-10-09 14:21:09 +01:00
Charles Mita 78d49aa161 Add -std=c89 to compile flags
Helps with some of the fixed-width typedefs in the bitshuffle library.
2018-10-09 14:12:30 +01:00
graeme-winter 59f4983524 Fix nasty hacks 2018-10-04 04:12:39 +01:00
graeme-winter 38b43be701 Nasty hack to work around non-masked data 2018-10-04 12:07:39 +09:00
Charles Mita d433e6b1d0 Apply mask at same time as data conversion 2018-08-20 12:51:19 +01:00
Charles Mita 5c0b6e8366 Check if the bitshuffle filter was applied before decompressing 2018-08-17 18:00:14 +01:00
Charles Mita f6ba8eb2aa Use the dataset's datatype for H5Dread, convert manually later.
Hopefully this results in less work for the HDF5 library (which would
not be done in parallel) and ensures the conversion is consistent
across all data retrieval strategies.
2018-08-17 18:00:14 +01:00
Charles Mita d1b25e18b8 Refactor description structs 2018-08-17 18:00:14 +01:00
Charles Mita ace6a46717 Implement direct-chunk reading to read data when possible
The HDF5 library was made thread-safe via excessive locking, so does
not gain much from reads being parallelized.

By using the H5DOread_chunk function (introduced in HDF5 1.10.2)
we reduce the time spent the library, improving performance for
when XDS uses multiple threads to process data.

The decompression and type conversions have to be done manually
however, and this is only used in a limited case.
2018-08-17 18:00:14 +01:00
Charles Mita 38660b17fd Fixes to allow bitshuffle-lz4 to compile with -std=c89
Removes all "//" style comments, adds a missing typedef
for "int16_t" (although these are not checked for correct width)
and removes duplicated "intX_t" typedefs.

It would be preferable to detect GCC (with its extensions) and use
its definitions of fixed-with integers if not compiling with C99.
2018-08-17 17:59:42 +01:00
Charles Mita 369948795b Add the bitshuffle-lz4 code and add to the build.
Source for the bitshuffle code is:
https://github.com/kiyo-masui/bitshuffle

Release Tag: 0.3.4

Commit at time of copy:
9ffba9de83036a91d345fa2f62fcaedf55f54c5f

The LZF and HDF5 plugin parts are not included as they are
not required.
2018-08-15 16:55:52 +01:00
Charles Mita f435908a2b Fix nbytes out parameter in plugin_get_header
Previously reported total number of bytes in a frame - should be
the width of the array data type.
2018-08-15 15:53:47 +01:00
Charles Mita 482ab573af Initialise the error stack message arrays at start
This helps prevent potential segfaults due to null-pointer
dereferencing in the face of errors when the plugin is accessed via
multiple threads.

If multiple threads are pushing errors onto the error stack then its
possible for elements of the various string pointer arrays to be left
NULL, causing a segfault when the stack is printed.

Whilst this does nothing to address the "correctness" of the plugin in
a multi-threaded context, at least it shouldn't crash.

Error messages may still be corrupted.
2018-08-15 15:51:28 +01:00
Charles Mita e4466b00fc Stop the HDF5 library installing an exit hook
This will cause XDS to segfault on exit, since the plugin (and hence
HDF5 library) will likely have been unloaded.

Calling "H5dont_atexit" before any other H5 calls will prevent the
hook being installed, and we call "H5Close" ourselves as part of the
plugin_close function.
2018-08-13 13:32:34 +01:00
Charles Mita 87d2d065d3 Add Fortran test framework to Makefile 2018-08-13 13:32:34 +01:00
graeme-winter dc34262acc Code from https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/LIB for testing plugin implementation 2018-08-13 09:18:44 +01:00
Charles Mita cc4d129932 Copy the HDF5 error stack into ours 2018-04-10 17:16:27 +01:00
Charles Mita fd31a46243 Add README.md 2018-03-28 21:42:16 +01:00
Charles Mita 8938306acb Rename plugin output to durin-plugin.so 2018-03-28 15:48:55 +01:00
Charles Mita 7fb7710a1b Add build dir to .gitignore 2018-03-28 14:14:14 +01:00
Charles Mita b8b66778a2 Improve Makefile to stop unnecessary rebuilds 2018-03-28 14:04:57 +01:00
Charles Mita bcaa5e1f5e Add simple program to open files using code in file.c 2018-03-28 14:04:29 +01:00