From 2bb96c6f69fe3b075744488062a2493fba4b822e Mon Sep 17 00:00:00 2001 From: Marc Howison Date: Wed, 10 Feb 2010 23:33:32 +0000 Subject: [PATCH] updated NEWS with details about 1.6 changes; changed hsize_t type in H5PartSetViewIndices; fixes to the build system; fixed bugs that were preventing the use of H5PartSetView during writes; added open object checks to regression test --- NEWS | 126 ++++++++++++++++++ configure.ac | 5 + src/H5Part.c | 330 +++++++++++++++++++++++++++++----------------- src/H5Part.h | 2 +- src/H5PartTypes.h | 1 + src/Makefile.am | 17 +-- test/Makefile.am | 5 +- test/params.h | 2 +- test/test.c | 4 +- test/testframe.c | 35 +++++ test/testframe.h | 3 + test/write.c | 58 ++++++-- tools/Makefile.am | 2 +- 13 files changed, 442 insertions(+), 148 deletions(-) diff --git a/NEWS b/NEWS index 13f694c..308fc19 100644 --- a/NEWS +++ b/NEWS @@ -1,3 +1,129 @@ +#### H5PART 1.6 ############################################################## + +Updated Documentation +--------------------- + +The Fortran API has been added to the doxygen documentation. Most entries have +a reference to their respective C API call. + +Regression Test Suite +--------------------- + +The test subdirectory has been reorganized to include a systematic series +of regression tests of common use cases for the API. The goal is to have both +complete coverage of the API (every call is exercised) and testing of some +unlikely or unsuspected values and cases. + +Currently, the C test covers the entire H5Part API and the Fortran test is +still in development. + +Benchmark Utilities +------------------- + +Two benchmarking utilities, H5PartBench and H5BlockBench, have been added to +the tools collection. Both require the parallel library and have functionality +and syntax similar to the IOR benchmark: + + http://sourceforge.net/projects/ior-sio/ + +File Mode Flags +--------------- + +Previously, the only flags used were H5PART_READ, H5PART_WRITE, and +H5PART_APPEND, and these were mutually exclusive. Additional flags have been +added to select the underlying virtual file driver (VFD) that HDF5 uses in +parallel mode and to accommodate the lustre filesystem: + +H5PART_VFD_MPIPOSIX (use the MPI-POSIX driver, which bypasses MPI-IO) +H5PART_VFD_MPIIO_IND (use MPI-IO in independent mode) +H5PART_FS_LUSTRE (activate H5P tunings for the lustre stripe size) + +These flags can be specified using the typical bitwise OR method, e.g. + + char flag = H5PART_WRITE | H5PART_VFD_MPIPOSIX | H5PART_FS_LUSTRE; + +will select write mode with these two additional options. + +The parallel library now defaults to using the MPI-IO collective mode driver +(previously it used independent mode). The collective buffering algorithm in +the Cray XT4/5 environment has been substantially improved in the last year, +and defaulting to collective mode allows us to take advantage of these +improvements automatically. + +There are also new open calls H5PartOpenAlign and H5PartOpenParallelAlign that +take an additional 'alignment' value. This value is passed to HDF5 and used to +pad out objects so that they align to filesystem boundaries. For instance, it +would make sense to use the stripe size as the alignment value when writing to +a lustre filesystem. + +Unified 'View' Model +-------------------- + +Views are now supported in both read and write mode (previously only supported +in read mode). It is possible to specify non-sensical views: you could for +instance specify a read view that is larger than the dataset on disk. Or you +could create a write view that is larger than the data in memory. In those +cases, you will encounter an error when you try to perform a read or write +operation on a dataset. + +There are two new methods for selecting views. H5PartSetNumParticlesStrided +selects a view with $n$ particles per processor, but such that the data in +memory is expected to have a 'stride' factor. For instance, if you have +particle data with fields $x$ and $y$ and a single array with entries +$x1,y1,x2,y2...$, then the stride factor is 2. Striding works for both reads +and writes and only affects the view of memory: individual fields are still +stored as individual arrays on disk. + +The second new method is H5PartSetViewIndices, which allows for point +selections of datasets. You can pass a list of indices for the points you +want to select. This is useful, for instance, when interfacing with FastBit +to perform queries that select a small subset of non-contiguous particles. + +Because views are now supported on write, it is possible to write a dataset +using multiple 'passes' or to leave some values unwritten. + +Internal Handling of Dataspaces +------------------------------- + +H5PartSetNumParticles and H5PartSetNumParticlesStrided now share the same +HDF5 dataspace state in the H5PartFile struct with the calls that modify +the view (H5PartSetView etc.). Previously, the memory and disk dataspaces +were regenerated from the view on every read access, which was unnecessary. + +Autogeneration of API Calls +--------------------------- + +Python scripts have been added to automatically generate read/write call +variants for different data types. + +Renamed Fortran Include +----------------------- + +The automake system no longer greps the *F90.inc files to generate an include +called 'H5Part.inc'. Instead, the *F90.inc files have been renamed to *.f90, +and awk is used to generate an 'H5PartF.h' file (the same naming convention +as in MPI's 'mpif.h'). + +The change to *.f90 was necessary to facilitate integration of the Fortran +API into doxygen. Note that the *.f90 files will not compile, nor can they +be included directly. Simply use + + include 'H5PartF.h' + +in your Fortran code to include all the definitions for Fortran H5Part/H5Block +calls. + +64 Char Limit on Dataset Names +------------------------------ + +Previously, a user could overrun internal buffers for dataset names. Now, a +fixed limit of 64 chars is imposed. Dataset names that are longer than this are +truncated and a warning is printed. We expect that most users are using +short canoncical names like x, px, id, etc. + + +#### H5PART 1.4 ############################################################## + Error Handling -------------- Now all functions, in which an error could occure, are returning a diff --git a/configure.ac b/configure.ac index 9f935d4..6f26ab5 100644 --- a/configure.ac +++ b/configure.ac @@ -390,6 +390,7 @@ if test "X$USE_PARALLEL" = "Xyes"; then fi fi + H5P_LIB_NAME="-lpH5Part" MTARGET="libpH5Part.a" TTARGET="test" @@ -417,6 +418,7 @@ if test "X$USE_PARALLEL" = "Xyes"; then FC=${MPIFC} fi + H5P_LIB_NAME="-lpH5PartF" MTARGET="${MTARGET} libpH5PartF.a" TTARGET="${TTARGET} testf" fi @@ -424,10 +426,12 @@ if test "X$USE_PARALLEL" = "Xyes"; then else # --enable-parallel=no AC_MSG_RESULT([no]) + H5P_LIB_NAME="-lH5Part" MTARGET="libH5Part.a" TTARGET="test" if test "X$USE_FORTRAN" = "Xyes"; then + H5P_LIB_NAME="-lH5PartF" MTARGET="${MTARGET} libH5PartF.a" TTARGET="${TTARGET} testf" fi @@ -596,6 +600,7 @@ AC_SUBST(CFLAGS) AC_SUBST(FFLAGS) AC_SUBST(STDCXX) AC_SUBST(H5P_LIB_LOC) +AC_SUBST(H5P_LIB_NAME) AC_SUBST(UNDERSCORE_H) AC_SUBST(BUILD_TOOLS) AC_SUBST(LDFLAGS) diff --git a/src/H5Part.c b/src/H5Part.c index f7586c1..9ab6992 100644 --- a/src/H5Part.c +++ b/src/H5Part.c @@ -32,7 +32,7 @@ Papers: -
  • A. Adelmann, R.D. Ryne, J. Shalf, C. Siegerist,"H5Part: A Portable High Performance Parallel Data Interface for Particle Simulations," +
  • A. Adelmann, R.D. Ryne, J. Shalf, C. Siegerist, "H5Part: A Portable High Performance Parallel Data Interface for Particle Simulations," Particle Accelerator Conference (PAC05), Knoxville TN., May 16-20, 2005. FPAT083.pdf
    • @@ -440,7 +440,6 @@ H5PartOpenFileParallelAlign ( \return File handle or \c NULL */ - H5PartFile* H5PartOpenFile ( const char *filename, /*!< [in] The name of the data file to open. */ @@ -497,23 +496,12 @@ _H5Part_file_is_valid ( return H5PART_ERR_BADFD; } -/*! - \ingroup h5part_open - - Closes an open file. - - \return \c H5PART_SUCCESS or error code -*/ h5part_int64_t -H5PartCloseFile ( +_H5Part_close_hdf_ids ( H5PartFile *f /*!< [in] filehandle of the file to close */ ) { - SET_FNAME ( "H5PartCloseFile" ); herr_t r = 0; - _h5part_errno = H5PART_SUCCESS; - - CHECK_FILEHANDLE ( f ); if ( f->block && f->close_block ) { (*f->close_block) ( f ); @@ -564,11 +552,38 @@ H5PartCloseFile ( if ( r < 0 ) HANDLE_H5P_CLOSE_ERR ( "f->create_prop" ); f->create_prop = H5P_DEFAULT; } + + return H5PART_SUCCESS; +} + +/*! + \ingroup h5part_open + + Closes an open file. + + \return \c H5PART_SUCCESS or error code +*/ +h5part_int64_t +H5PartCloseFile ( + H5PartFile *f /*!< [in] filehandle of the file to close */ + ) { + + SET_FNAME ( "H5PartCloseFile" ); + herr_t r = 0; + _h5part_errno = H5PART_SUCCESS; + + CHECK_FILEHANDLE ( f ); + + r = _H5Part_close_hdf_ids ( f ); + if ( r < 0 ) return r; + if ( f->file ) { r = H5Fclose( f->file ); if ( r < 0 ) HANDLE_H5F_CLOSE_ERR; f->file = 0; } + + /* free memory from H5PartFile struct */ if( f->pnparticles ) { free( f->pnparticles ); } @@ -688,6 +703,12 @@ _set_num_particles ( herr = _reset_view ( f ); if ( herr < 0 ) return herr; + if ( f->shape != H5S_ALL ) { + herr = H5Sclose ( f->shape ); + if ( herr < 0 ) return HANDLE_H5S_CLOSE_ERR; + f->shape = H5S_ALL; + } + f->nparticles = (hsize_t) nparticles; /* declare local memory datasize with striding */ @@ -781,12 +802,8 @@ _set_num_particles ( &stride, &count, NULL ); if ( herr < 0 ) return HANDLE_H5S_SELECT_HYPERSLAB_ERR; - - if ( f->timegroup < 0 ) { - herr = _H5Part_set_step ( f, 0 ); - if ( herr < 0 ) return herr; - } #endif + return H5PART_SUCCESS; } @@ -797,7 +814,7 @@ _set_num_particles ( After you call this subroutine, all subsequent operations will assume this number of particles will be written. - For the parallel library, the \a nparticles value is the number of + For the parallel library, the \c nparticles value is the number of particles that the \e individual task will write. You can use a different value on different tasks. This function uses an \c MPI_Allgather @@ -808,11 +825,11 @@ _set_num_particles ( This function assumes that your particles' data fields are in stored in contiguous 1D arrays. - For instance, the fields $x$ and $y$ for your particles are stored + For instance, the fields \e x and \e y for your particles are stored in separate arrays \c x[] and \c y[]. If instead you store your particles as tuples, so that the values - are arranged $x_1,y_1,x_2,y_2$... than you need to setup striding + are arranged \f$ x_1,y_1,x_2,y_2\f$... than you need to setup striding (in this case with value 2) using \ref H5PartSetNumParticlesStrided. \return \c H5PART_SUCCESS or error code @@ -842,7 +859,7 @@ H5PartSetNumParticles ( After you call this subroutine, all subsequent operations will assume this number of particles will be written. - For the parallel library, the \a nparticles value is the number of + For the parallel library, the \c nparticles value is the number of particles that the \e individual task will write. You can use a different value on different tasks. This function uses an \c MPI_Allgather @@ -852,8 +869,8 @@ H5PartSetNumParticles ( few times as possible when running at large concurrency. This function assumes that your particles' data fields are - stored tuples. For instance, the fields $x$ and $y$ of your - particles are arranged $x_1,y_1,x_2,y_2$... in a single data + stored tuples. For instance, the fields \e x and \e y of your + particles are arranged \f$x_1,y_1,x_2,y_2\f$... in a single data array. In this example, the stride value would be 2. If you instead have a separate array for each fields, @@ -861,12 +878,12 @@ H5PartSetNumParticles ( use \ref H5PartSetNumParticles. \return \c H5PART_SUCCESS or error code - */ +*/ h5part_int64_t H5PartSetNumParticlesStrided ( H5PartFile *f, /*!< [in] Handle to open file */ const h5part_int64_t nparticles, /*!< [in] Number of particles */ - const h5part_int64_t stride /*!< [in] Stride (e.g. number of fields in the particle array) */ + const h5part_int64_t stride /*!< [in] Stride value (e.g. number of fields in the particle array) */ ) { SET_FNAME ( "H5PartSetNumParticlesStrided" ); @@ -880,6 +897,24 @@ H5PartSetNumParticlesStrided ( return H5PART_SUCCESS; } +static void +_normalize_dataset_name ( + const char *name, + char *name2 + ) { + + if ( strlen(name) > H5PART_DATANAME_LEN ) { + strncpy ( name2, name, H5PART_DATANAME_LEN - 1 ); + name2[H5PART_DATANAME_LEN-1] = '\0'; + _H5Part_print_warn ( + "Dataset name '%s' is longer than maximum %d chars. " + "Truncated to: '%s'", + name, H5PART_DATANAME_LEN, name2 ); + } else { + strcpy ( name2, name ); + } +} + static h5part_int64_t _write_data ( H5PartFile *f, /*!< IN: Handle to open file */ @@ -887,12 +922,17 @@ _write_data ( const void *array, /*!< IN: Array to commit to disk */ const hid_t type /*!< IN: Type of data */ ) { + herr_t herr; hid_t dataset_id; - _H5Part_print_debug ( "Create a dataset[%s] mounted on the " + char name2[H5PART_DATANAME_LEN]; + _normalize_dataset_name ( name, name2 ); + + _H5Part_print_debug ( + "Create a dataset[%s] mounted on " "timestep %lld", - name, (long long)f->timestep ); + name2, (long long)f->timestep ); if ( f->shape == H5S_ALL ) { _H5Part_print_warn ( @@ -901,28 +941,39 @@ _write_data ( return HANDLE_H5PART_BAD_VIEW_ERR ( f->viewstart, f->viewend ); } + H5E_BEGIN_TRY + dataset_id = H5Dopen ( f->timegroup, name2 #ifndef H5_USE_16_API - htri_t exists = H5Lexists ( f->timegroup, name, H5P_DEFAULT ); - if ( exists > 0 ) return HANDLE_H5D_EXISTS_ERR ( name, f->timestep ); - - dataset_id = H5Dcreate2 ( - f->timegroup, - name, - type, - f->shape, - H5P_DEFAULT, - H5P_DEFAULT, - H5P_DEFAULT ); -#else - dataset_id = H5Dcreate ( - f->timegroup, - name, - type, - f->shape, - H5P_DEFAULT ); + , H5P_DEFAULT #endif - if ( dataset_id < 0 ) - return HANDLE_H5D_CREATE_ERR ( name, f->timestep ); + ); + H5E_END_TRY + + if ( dataset_id > 0 ) { + _H5Part_print_warn ( + "Dataset[%s] at timestep %lld " + "already exists", name2, (long long)f->timestep ); + } else { +#ifndef H5_USE_16_API + dataset_id = H5Dcreate2 ( + f->timegroup, + name2, + type, + f->shape, + H5P_DEFAULT, + H5P_DEFAULT, + H5P_DEFAULT ); +#else + dataset_id = H5Dcreate ( + f->timegroup, + name2, + type, + f->shape, + H5P_DEFAULT ); +#endif + if ( dataset_id < 0 ) + return HANDLE_H5D_CREATE_ERR ( name2, f->timestep ); + } #ifdef PARALLEL_IO herr = _H5Part_start_throttle ( f ); @@ -942,7 +993,7 @@ _write_data ( if ( herr < 0 ) return herr; #endif - if ( herr < 0 ) return HANDLE_H5D_WRITE_ERR ( name, f->timestep ); + if ( herr < 0 ) return HANDLE_H5D_WRITE_ERR ( name2, f->timestep ); herr = H5Dclose ( dataset_id ); if ( herr < 0 ) return HANDLE_H5D_CLOSE_ERR; @@ -1825,7 +1876,7 @@ _H5Part_set_step ( #endif ); if ( f->timegroup < 0 ) - return HANDLE_H5G_OPEN_ERR( stepname ); + return HANDLE_H5G_OPEN_ERR ( stepname ); } else { _H5Part_print_debug ( @@ -1834,14 +1885,11 @@ _H5Part_set_step ( (long long)step, (long long)(size_t) f ); - f->timegroup = H5Gcreate( f->file, - stepname, - 0 + f->timegroup = H5Gcreate( f->file, stepname, 0 #ifndef H5_USE_16_API , H5P_DEFAULT, H5P_DEFAULT #endif ); - if ( f->timegroup < 0 ) return HANDLE_H5G_CREATE_ERR ( stepname ); } @@ -1893,7 +1941,10 @@ _H5Part_have_group ( #ifndef H5_USE_16_API return (H5Lexists( id, name, H5P_DEFAULT ) ? 1 : 0); #else - return (H5Gget_objinfo( id, name, 1, NULL ) >= 0 ? 1 : 0); + H5E_BEGIN_TRY + herr_t exists = H5Gget_objinfo( id, name, 1, NULL ); + H5E_END_TRY + return (exists >= 0 ? 1 : 0); #endif } @@ -2118,11 +2169,11 @@ _H5Part_get_num_objects_matching_pattern ( herr = H5Giterate ( group_id, group_name, &idx, _H5Part_iteration_operator, &data ); #endif - if ( herr < 0 ) return herr; #ifndef H5_USE_16_API - H5Gclose( child_id ); + herr = H5Gclose ( child_id ); + if ( herr < 0 ) return HANDLE_H5G_CLOSE_ERR; #endif return data.count; @@ -2166,13 +2217,14 @@ _H5Part_get_object_name ( if ( herr < 0 ) { return HANDLE_H5L_ITERATE_ERR; } - else if ( herr == 0 ) return 0; #ifndef H5_USE_16_API - herr = H5Gclose ( child_id ); - if ( herr < 0 ) return HANDLE_H5G_CLOSE_ERR; + herr_t herr2 = H5Gclose ( child_id ); + if ( herr2 < 0 ) return HANDLE_H5G_CLOSE_ERR; #endif + if ( herr == 0 ) return 0; + return 1; } @@ -2393,11 +2445,8 @@ _H5Part_get_num_particles ( ) { h5part_int64_t herr; - hid_t space_id; - hid_t dataset_id; - char dataset_name[128]; - hsize_t nparticles; - + h5part_int64_t nparticles; + char dataset_name[H5PART_DATANAME_LEN]; char stepname[H5PART_STEPNAME_LEN]; _H5Part_get_step_name(f, f->timestep, stepname); @@ -2406,38 +2455,70 @@ _H5Part_get_num_particles ( stepname, H5G_DATASET, 0, - dataset_name, sizeof (dataset_name) ); + dataset_name, H5PART_DATANAME_LEN ); if ( herr < 0 ) return herr; - else if ( herr == 0 ) { - _H5Part_print_warn ( - "There are no datasets in timestep %s: " - "reporting 0 particles.", stepname ); - return 0; + /* returns 0 if there are no datasets on disk */ + else if ( herr == 0 ) + { + /* try to recover number of particles from a previous + * H5PartSetNumParticles call. */ +#ifdef PARALLEL_IO + nparticles = 0; + int i; + for (i=0; i < f->nprocs; i++) { + nparticles += f->pnparticles[i]; + } +#else + nparticles = f->nparticles; +#endif + + if ( nparticles > 0 ) { + _H5Part_print_debug ( + "Using existing view to report " + "nparticles = %lld", (long long)nparticles ); + return nparticles; + } + else { + _H5Part_print_warn ( + "There are no datasets in timestep %s " + "or existing views: " + "reporting 0 particles.", stepname ); + return 0; + } } - dataset_id = H5Dopen ( f->timegroup, dataset_name + /* if a view exists, use its size as the number of particles */ + if ( _H5Part_has_view ( f ) ) + { + nparticles = H5Sget_select_npoints ( f->diskshape ); + if ( nparticles < 0 ) return HANDLE_H5S_GET_SELECT_NPOINTS_ERR; + + /* double check that the size of the diskshape agrees with + * the size of the view */ + if ( nparticles != f->viewend - f->viewstart + 1 ) { + _H5Part_print_warn ( + "Number of particles (%lld) does not agree " + "with view range.", (long long)nparticles ); + return HANDLE_H5PART_BAD_VIEW_ERR ( + f->viewstart, f->viewend); + } + } + /* otherwise, report all particles on disk in the first dataset + * for this timestep */ + else + { + hid_t space_id; + hid_t dataset_id; + + dataset_id = H5Dopen ( f->timegroup, dataset_name #ifndef H5_USE_16_API , H5P_DEFAULT #endif ); - if ( dataset_id < 0 ) - return HANDLE_H5D_OPEN_ERR ( dataset_name ); + if ( dataset_id < 0 ) + return HANDLE_H5D_OPEN_ERR ( dataset_name ); - /* if a view exists, use its size as the number of particles */ - if ( _H5Part_has_view ( f ) ) { - nparticles = H5Sget_select_npoints ( f->diskshape ); - if ( nparticles < 0 ) return HANDLE_H5S_GET_SELECT_NPOINTS_ERR; - if ( nparticles != f->viewend - f->viewstart + 1 ) - _H5Part_print_warn ( - "Number of particles (%lld) does not agree " - "with view range.", nparticles ); - return HANDLE_H5PART_BAD_VIEW_ERR ( - f->viewstart, f->viewend); - } - /* otherwise, report all particles on disk in the first dataset - * for this timestep */ - else { space_id = H5Dget_space ( dataset_id ); if ( space_id < 0 ) return HANDLE_H5D_GET_SPACE_ERR; @@ -2447,12 +2528,12 @@ _H5Part_get_num_particles ( herr = H5Sclose ( space_id ); if ( herr < 0 ) return HANDLE_H5S_CLOSE_ERR; + + herr = H5Dclose ( dataset_id ); + if ( herr < 0 ) return HANDLE_H5D_CLOSE_ERR; } - herr = H5Dclose ( dataset_id ); - if ( herr < 0 ) return HANDLE_H5D_CLOSE_ERR; - - return (h5part_int64_t) nparticles; + return nparticles; } /*! @@ -2499,12 +2580,6 @@ _reset_view ( f->viewstart = -1; f->viewend = -1; - if ( f->shape != H5S_ALL ) { - herr = H5Sclose ( f->shape ); - if ( herr < 0 ) return HANDLE_H5S_CLOSE_ERR; - f->shape = H5S_ALL; - } - if ( f->diskshape != H5S_ALL ) { herr = H5Sclose ( f->diskshape ); if ( herr < 0 ) return HANDLE_H5S_CLOSE_ERR; @@ -2580,11 +2655,15 @@ _set_view ( end==-1 to mean end of file */ total = (hsize_t) _H5Part_get_num_particles ( f ); - if ( total < 0 ) return HANDLE_H5PART_GET_NUM_PARTICLES_ERR ( total ); - - _H5Part_print_debug ( "Total nparticles=%lld", (long long)total ); - - if ( total == 0 ) return H5PART_SUCCESS; + if ( total < 0 ) { + return HANDLE_H5PART_GET_NUM_PARTICLES_ERR ( total ); + } + else if ( total == 0 ) { + /* No datasets have been created yet and no veiws are set. + * We have to leave the view empty because we don't know how + * many particles there should be! */ + return H5PART_SUCCESS; + } if ( start == -1 ) start = 0; if ( end == -1 ) end = total - 1; // range is *inclusive* @@ -2604,10 +2683,7 @@ _set_view ( f->viewend = end; f->nparticles = end - start + 1; - /* declare overall datasize */ - f->shape = H5Screate_simple ( 1, &total, NULL ); - if ( f->shape < 0 ) - return HANDLE_H5S_CREATE_SIMPLE_ERR ( total ); + _H5Part_print_debug ( "nparticles=%lld", (long long)f->nparticles ); /* declare overall data size but then select a subset */ f->diskshape = H5Screate_simple ( 1, &total, NULL ); @@ -2636,7 +2712,7 @@ _set_view ( static h5part_int64_t _set_view_indices ( H5PartFile *f, /*!< [in] Handle to open file */ - const hsize_t *indices, /*!< [in] List of indices */ + const h5part_int64_t *indices, /*!< [in] List of indices */ h5part_int64_t nelems /*!< [in] Size of list */ ) { @@ -2661,7 +2737,15 @@ _set_view_indices ( end==-1 to mean end of file */ total = (hsize_t) _H5Part_get_num_particles ( f ); - if ( total < 0 ) return HANDLE_H5PART_GET_NUM_PARTICLES_ERR ( total ); + if ( total < 0 ) { + return HANDLE_H5PART_GET_NUM_PARTICLES_ERR ( total ); + } + else if ( total == 0 ) { + /* No datasets have been created yet and no veiws are set. + * We have to leave the view empty because we don't know how + * many particles there should be! */ + return H5PART_SUCCESS; + } _H5Part_print_debug ( "Total nparticles=%lld", (long long)total ); @@ -2677,11 +2761,6 @@ _set_view_indices ( f->nparticles = (hsize_t) nelems; } - /* declare overall datasize */ - f->shape = H5Screate_simple ( 1, &total, NULL ); - if ( f->shape < 0 ) - return HANDLE_H5S_CREATE_SIMPLE_ERR ( total ); - /* declare overall data size but then will select a subset */ f->diskshape = H5Screate_simple ( 1, &total, NULL ); if ( f->diskshape < 0 ) @@ -2698,7 +2777,7 @@ _set_view_indices ( f->diskshape, H5S_SELECT_SET, nelems, - indices ); + (hsize_t*)indices ); } else { herr = H5Sselect_none ( f->diskshape ); } @@ -2768,7 +2847,7 @@ H5PartSetView ( h5part_int64_t H5PartSetViewIndices ( H5PartFile *f, /*!< [in] Handle to open file */ - const hsize_t *indices, /*!< [in] List of indices */ + const h5part_int64_t *indices, /*!< [in] List of indices */ h5part_int64_t nelems /*!< [in] Size of list */ ) { @@ -2930,17 +3009,20 @@ _read_data ( hid_t memspace_id; if ( f->timegroup < 0 ) { - h5part_int64_t h5err = _H5Part_set_step ( f, f->timestep ); - if ( h5err < 0 ) return h5err; + herr = _H5Part_set_step ( f, f->timestep ); + if ( herr < 0 ) return herr; } - dataset_id = H5Dopen ( f->timegroup, name + char name2[H5PART_DATANAME_LEN]; + _normalize_dataset_name ( name, name2 ); + + dataset_id = H5Dopen ( f->timegroup, name2 #ifndef H5_USE_16_API , H5P_DEFAULT #endif ); - if ( dataset_id < 0 ) return HANDLE_H5D_OPEN_ERR ( name ); + if ( dataset_id < 0 ) return HANDLE_H5D_OPEN_ERR ( name2 ); /* default spaces, if not using a view selection */ memspace_id = H5S_ALL; @@ -2971,7 +3053,7 @@ _read_data ( "Ignoring view: dataset[%s] has fewer " "elements on disk (%lld) than are selected " "(%lld).", - name, (long long)ndisk, (long long)nread ); + name2, (long long)ndisk, (long long)nread ); nread = ndisk; } } @@ -2997,7 +3079,7 @@ _read_data ( "Ignoring view: dataset[%s] has more " "elements selected (%lld) than are available " "in memory (%lld).", - name, (long long)nread, (long long)nmem ); + name2, (long long)nread, (long long)nmem ); memspace_id == H5S_ALL; } } @@ -3016,7 +3098,7 @@ _read_data ( (get hyperslab if needed) */ f->xfer_prop, /* ignore... its for parallel reads */ array ); - if ( herr < 0 ) return HANDLE_H5D_READ_ERR ( name, f->timestep ); + if ( herr < 0 ) return HANDLE_H5D_READ_ERR ( name2, f->timestep ); #ifdef PARALLEL_IO herr = _H5Part_end_throttle ( f ); diff --git a/src/H5Part.h b/src/H5Part.h index f39b9d7..2dd1b35 100644 --- a/src/H5Part.h +++ b/src/H5Part.h @@ -205,7 +205,7 @@ H5PartSetView ( h5part_int64_t H5PartSetViewIndices ( H5PartFile *f, /*!< [in] Handle to open file */ - const hsize_t *indices, /*!< [in] List of indices */ + const h5part_int64_t *indices, /*!< [in] List of indices */ h5part_int64_t nelems /*!< [in] Size of list */ ); diff --git a/src/H5PartTypes.h b/src/H5PartTypes.h index 60e8288..3bede04 100644 --- a/src/H5PartTypes.h +++ b/src/H5PartTypes.h @@ -24,6 +24,7 @@ typedef unsigned long MPI_Comm; #endif #define H5PART_STEPNAME_LEN 64 +#define H5PART_DATANAME_LEN 64 struct H5BlockFile; diff --git a/src/Makefile.am b/src/Makefile.am index 213f5c7..9dc2428 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -4,6 +4,11 @@ OBJEXT=o INCLUDES = -I@HDF5ROOT@/include @MPIINC@ +EXTRA_HEADERS = H5PartPrivate.h \ + H5BlockPrivate.h \ + H5MultiBlockPrivate.h \ + H5PartAttrib.h + # Extra files that I wish to include in the dist tar ball. EXTRA_DIST = TestUnderscoreC.c \ TestUnderscore.f \ @@ -13,7 +18,8 @@ EXTRA_DIST = TestUnderscoreC.c \ H5Part.f90 \ H5PartAttrib.f90 \ H5Block.f90 \ - H5BlockReadWrite.f90 + H5BlockReadWrite.f90 \ + $(EXTRA_HEADERS) # Files that I don't want to include in the dist tar ball nodist_include_HEADERS = H5PartF.h @UNDERSCORE_H@ @@ -40,12 +46,6 @@ include_HEADERS = H5Part.h \ H5PartF.h \ @UNDERSCORE_H@ -# Listing of all possible headers that I may include -EXTRA_HEADERS = H5PartPrivate.h \ - H5BlockPrivate.h \ - H5MultiBlockPrivate.h \ - H5PartAttrib.h - # Listing of sources libH5Part_a_SOURCES = H5Part.c \ H5PartAttrib.c \ @@ -54,7 +54,8 @@ libH5Part_a_SOURCES = H5Part.c \ H5MultiBlock.c \ H5MultiBlockReadWrite.c -libH5PartF_a_SOURCES = H5PartF.c \ +libH5PartF_a_SOURCES = $(libH5Part_a_SOURCES) \ + H5PartF.c \ H5PartAttribF.c \ H5BlockF.c \ H5BlockReadWriteF.c diff --git a/test/Makefile.am b/test/Makefile.am index 9f7ff88..50de6bc 100644 --- a/test/Makefile.am +++ b/test/Makefile.am @@ -3,7 +3,7 @@ OBJEXT=o HDFLIB = -L@HDF5ROOT@/lib -lhdf5 -lz @SZLIB@ -LIBS = -L@H5P_LIB_LOC@ -lH5PartF -lH5Part $(HDFLIB) -lm +LIBS = -L@H5P_LIB_LOC@ @H5P_LIB_NAME@ $(HDFLIB) -lm INCLUDES = -I@H5P_LIB_LOC@ -I@HDF5ROOT@/include @MPIINC@ @@ -18,7 +18,8 @@ test_SOURCES = test.c \ testframe.c \ write.c \ read.c \ - testframe.h + testframe.h \ + params.h testf_SOURCES = testf.F90 diff --git a/test/params.h b/test/params.h index 854f2de..46f1ee7 100644 --- a/test/params.h +++ b/test/params.h @@ -2,7 +2,7 @@ #define _H5PART_TEST_PARAMS_H_ #define FILENAME "test.h5" -#define LONGSTEPNAME "thisisaverylongstepnamethatshouldexceedthelimitof64charcausingawarningtoprint" +#define LONGNAME "thisisaverylongnamethatshouldexceedthelimitof64charcausingawarningtoprint" #define NTIMESTEPS 10 /* do not decrease this value below 99, or it will break assumptions diff --git a/test/test.c b/test/test.c index 4552a5e..f7bc092 100644 --- a/test/test.c +++ b/test/test.c @@ -61,10 +61,10 @@ int main(int argc, char **argv) //if (GetTestCleanup() && !getenv("HDF5_NOCLEANUP")) // TestCleanup(); - //return GetTestNumErrs(); - #ifdef PARALLEL_IO return MPI_Finalize(); +#else + return GetTestNumErrs(); #endif } diff --git a/test/testframe.c b/test/testframe.c index 41f6fcc..8d83a5c 100644 --- a/test/testframe.c +++ b/test/testframe.c @@ -649,3 +649,38 @@ test_is_valid(H5PartFile *file) RETURN(status, H5PART_SUCCESS, "H5PartFileIsValid"); } +void +test_open_objects(H5PartFile *file, int max_objects) +{ + ssize_t nopen = H5Fget_obj_count(file->file, H5F_OBJ_ALL); + if (nopen > max_objects) + { + TestErrPrintf( "*** TOO MANY OBJECTS OPEN: %d > %d " + "at line %4d in %s\n", nopen, max_objects, + (int)__LINE__, __FILE__ ); + + hid_t *list = malloc(sizeof(hid_t)*nopen); + H5Fget_obj_ids(file->file, H5F_OBJ_ALL, nopen, list); + + H5O_info_t info; + int i; + for (i=0; i