From 6633dace44f334b3ded6bd41f233828c10ef8194 Mon Sep 17 00:00:00 2001 From: Achim Gsell Date: Thu, 15 Jan 2009 15:18:37 +0000 Subject: [PATCH] moved install instructions to file INSTALL --- README | 473 +++++++-------------------------------------------------- 1 file changed, 56 insertions(+), 417 deletions(-) diff --git a/README b/README index 4dc7bf6..a04cfa3 100644 --- a/README +++ b/README @@ -1,418 +1,57 @@ -============================================================================== -README file for H5Part configure -============================================================================== - -+ 0. HDF5 library ------------------ - -Make sure you have a working version of the HDF5 library in your computer. If you plan -to use parallel I/O you need to use a parallel HDF5 version. If you don't have the -library download the sources from http://hdf.ncsa.uiuc.edu/HDF5/ - -+ 1. Quick start (If you feel lucky...) ----------------------------------------- - -For many platforms, where compilers are installed in the "default" location -and all the environment variables are set correctly, it should be sufficient -to type: - -./configure [OPTIONS] && make [install] - -to have the libraries and test program compiled. - - -============================================================================== - - -+ 2. configure line options ----------------------------- - -(1) Enable options ---enable-fortran ---enable-parallel ---enable-tools ---enable-python ---enable-64 (only for AIX and Irix) - -(2) Setting compilers related variables manually -To assign environment variables (e.g., CC, CFLAGS...), specify them as -VAR=VALUE. See below for descriptions of some of the useful variables. - -Compiler environment variables: - CC C compiler command - CXX C++ compiler command - FC Fortran compiler command - -(3) Setting prefix for installation manually ---prefix=PREFIX install files in PREFIX - - -For more information, type: - -./configure --help - - -============================================================================== - - -+ 3. Platform specific settings for compiling libraries & test programs ------------------------------------------------------------------------- - -Building - make : builds the libraries and test programs - make install: builds and installs - make clean : removes extraneous object files - make distclean : returns configuration to unconfigured state - -As tested on hosts: - -(1) Davinci (Linux"SUSE"-ia64) ---------------------------------------------------- -FIRST THING FIRST!! Type: - -module load intel -to have the proper compilers set up for use. - -module load hdf5_par - -[NOTE: module load hdf5 does not work at this moment. -Possible installation error of HDF5 on Davinci.] - - -Brief profile: -CC = icc -CXX = icc -FC = ifort -MPICC = icc -MPICXX = icc -MPIFC = ifort - -If default doesn't work, try: - -For C shell: -env CC=icc CXX=icc FC=ifort MPICC=icc MPICXX=icc MPIFC=ifort ./configure && make install - -For Korn or Bourne shell: -CC=icc CXX=icc FC=ifort MPICC=icc MPICXX=icc MPIFC=ifort ./configure && make install - -And, of course, you should add configure options, such as --enable-fortran --enable-parallel. - - -On this host, executing: - -For C shell: -env CC=icc CXX=icc FC=ifort MPICC=icc MPICXX=icc MPIFC=ifort ./configure --enable-fortran --enable-parallel && make install - -For Korn or Bourne shell: -CC=icc CXX=icc FC=ifort MPICC=icc MPICXX=icc MPIFC=ifort ./configure --enable-fortran --enable-parallel && make install - -has functioned correctly. - - -To run test program, go to test directory: - -(i) Serial test -run: -./H5testF -./H5PartTest - -(ii) Parallel test -run: -mpirun 2 H5testFpar -mpirun 2 H5PartTestP - -NOTE: Number 2 above is a arbitrary number that indicates the number of processes. - - - - - -(2) Jacquard (Linux"SUSE"-x86_64) -------------------------------------------------------- -Brief profile: -CC = pathcc -CXX = pathCC -FC = pathf90 -MPICC = mpicc -MPICXX = mpicxx -MPIFC = mpif90 - - - -If default doesn't work, try: - -For C shell: -env CC=pathcc CXX=pathCC FC=pathf90 MPICC=mpicc MPICXX=mpicxx MPIFC=mpif90 ./configure && make install - -For Korn or Bourne shell: -CC=pathcc CXX=pathCC FC=pathf90 MPICC=mpicc MPICXX=mpicxx MPIFC=mpif90 ./configure && make install - -And, of course, you should add configure options, such as --enable-fortran --enable-parallel. - - - -On this host, executing: - -For C shell: -env CC=pathcc CXX=pathCC FC=pathf90 MPICC=mpicc MPICXX=mpicxx MPIFC=mpif90 ./configure --enable-fortran --enable-parallel && make install - -For Korn or Bourne shell: -CC=pathcc CXX=pathCC FC=pathf90 MPICC=mpicc MPICXX=mpicxx MPIFC=mpif90 ./configure --enable-fortran --enable-parallel && make install - -has functioned correctly. - - - -To run test program, go to test directory: - -(i) Serial test -run: -./H5testF -./H5PartTest - -(ii) Parallel test -In Jacquard, we need a script to run parallel programs, since Jacquard does not support interactive parallel processing. Instead, we need to submit the task to a queue. A sample script (run_H5testFpar.scr) may look like below: - -+++++++++++++++++++++++++++++++++++++++ -#!/bin/csh -#PBS -l nodes=1:ppn=2,walltime=00:05:00 -#PBS -N H5testFpar -#PBS -o H5testFpar.out -#PBS -e H5testFpar.err -#PBS -q debug -#PBS -A mpccc -#PBS -V - -setenv PBS_OWORKDIR /home/H5Part/test - -cd $PBS_O_WORKDIR -mpirun -np 2 ./H5testFpar -+++++++++++++++++++++++++++++++++++++++ - -Then, we need to submit the task to a queue by running: - -qsub run_H5testFpar.scr - -After processed, the result will appear in H5testFpar.out, and error messages will appear in H5testFpar.err. - - - - - -(3) Bassi (AIX5.3.0.0-Power 3) -------------------------------------------------- -FIRST THING FIRST!! Type: - -module load gcc - -to have the proper compilers set up for use. - -Brief profile: -CC = cc_r -CXX = cc_r -FC = xlf_r -MPICC = mpcc_r -MPICXX = mpcc_r -MPIFC = mpxlf_r - - - -If default doesn't work, try: - -For C shell: -env CC=cc_r CXX=cc_r FC=xlf_r MPICC=mpcc_r MPICXX=mpcc_r MPIFC=mpxlf_r ./configure && make install - -For Korn or Bourne shell: -CC=cc_r CXX=cc_r FC=xlf_r MPICC=mpcc_r MPICXX=mpcc_r MPIFC=mpxlf_r ./configure && make install - -And, of course, you should add configure options, such as --enable-fortran --enable-parallel. - - - -On this host, executing: - -For C shell: -env CC=cc_r CXX=cc_r FC=xlf_r MPICC=mpcc_r MPICXX=mpcc_r MPIFC=mpxlf_r ./configure --enable-fortran --enable-parallel && make install - -For Korn or Bourne shell: -CC=cc_r CXX=cc_r FC=xlf_r MPICC=mpcc_r MPICXX=mpcc_r MPIFC=mpxlf_r ./configure --enable-fortran --enable-parallel && make install - -has functioned correctly. - - - -To run test program, go to test directory: - -(i) Serial test -run: -./H5testF -./H5PartTest - -(ii) Parallel test -run: -unsetenv MP_SINGLE_THREAD -poe ./H5PartTestP -procs 2 -nodes 1 -poe ./H5testFpar -procs 2 -nodes 1 - -For more details on how to run parallel programs on Bassi, go to: - -http://www.nersc.gov/nusers/resources/bassi/running_jobs/ - - - - - -(4) Starsky (Darwin8.5.0-PowerPC) ----------------------------------------------------- -FIRST THING FIRST!! Set environment variables: (in tcsh) - -setenv LD_LIBRARY_PATH /usr/local/g95/lib/gcc-lib/powerpc-apple-darwin8.5.0/4.0.3/ -setenv PATH ${PATH}:/usr/local/g95/bin - -NOTE: This environment variable setting is specifit to Starsky. It is to make configure find the fortran compiler. - -Brief profile: -CC = gcc -CXX = g++ -FC = g95 - - - -If default doesn't work, try: - -For C shell: -env CC=gcc CXX=g++ FC=g95 ./configure && make install - -For Korn or Bourne shell: -CC=gcc CXX=g++ FC=g95 ./configure && make install - -And, of course, you should add configure options, such as --enable-fortran --enable-parallel. - - - -On this host, executing: - -For C shell: -env CC=gcc CXX=g++ FC=g95 ./configure --enable-fortran && make install - -For Korn or Bourne shell: -CC=gcc CXX=g++ FC=g95 ./configure --enable-fortran && make install - -has functioned correctly. - - - -To run test program, go to test directory: - -(i) Serial test -run: -./H5testF -./H5PartTest - - - - - -(5) Linux AMD64 Visualization group workstations (Linux"SUSE"-x86_64) ---------------------------------------------------------- -Brief profile: -CC = gcc -CXX = g++ -FC = g95 - - - -If default doesn't work, try: - -For C shell: -env CC=gcc CXX=g++ FC=g95 ./configure && make install - -For Korn or Bourne shell: -CC=gcc CXX=g++ FC=g95 ./configure && make install - -And, of course, you should add configure options, such as --enable-fortran --enable-parallel. - - - -On this host, executing: - -For C shell: -env CC=gcc CXX=g++ FC=g95 ./configure --enable-fortran && make install - -For Korn or Bourne shell: -CC=gcc CXX=g++ FC=g95 ./configure --enable-fortran && make install - -has functioned correctly. - - - -To run test program, go to test directory: - -(i) Serial test -run: -./H5testF -./H5PartTest - - -(6) Cray XT3 @ CSCS gele - - 1) modules/3.1.6 9) PrgEnv-pgi/1.4.26 17) xt-lustre-ss/1.4.26 - 2) MySQL/4.0.26 10) xt-pbs/5.3.4 18) Base-opts/1.4.26 - 3) acml/3.0 11) xt-service/1.4.26 19) subversion/1.3.2 - 4) pgi/6.1.4 12) xt-libc/1.4.26 20) zlib/1.2.3 - 5) totalview/7.2.0 13) xt-os/1.4.26 21) szip/2.0 - 6) xt-libsci/1.4.26 14) xt-catamount/1.4.26 22) hdf5/1.6.5 - 7) xt-mpt/1.4.26 15) xt-boot/1.4.26 - 8) xt-pe/1.4.26 16) xt-crms/1.4.26 - - -CFLAGS="-I$H5HOME/include -DF77_SINGLE_UNDERSCORE" LDFLAGS="-L$H5HOME/lib -L$ZHOME/lib" ./configure --enable-parallel --enable-fortran - - -(7) merlin00 / merlin3 - -Currently Loaded Modulefiles: - 1) hdf5/hdf5-1.6.5 2) root/root-5.10.00 3) mpi/mpich2-1.0.3-pgi-6.1 4) pgi/pgi_64-6.1 - -FC=mpif90 ./configure --enable-parallel --enable-fortran - - - - - - - -============================================================================== - - -+ 4. Trouble shooting (Things to check for...) --------------------------------------------- -(0) Have you set the LD_LIBRARY_PATH? ----Some systems require the user to manually set the environment variable, LD_LIBRARY_PATH. To do so: - -set the environment variable HDF5ROOT or PHDF5ROOT to point to your installation of HDF5 (serial and/or parallel respectively). - -For Korn or Bourne shell: - LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${HDF5ROOT}/lib;export LD_LIBRARY_PATH - LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:{PHDF5ROOT}/lib;export LD_LIBRARY_PATH - -For C shell: - setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:${HDF5ROOT}/lib - setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:${PHDF5ROOT}/lib - -Note that the values of HDF5ROOT & PHDF5ROOT will be in the "summary" section when you run configure. - -Not properly set LD_LIBRARY_PATH value may result in a runtime error: - -./H5PartTest: error while loading shared libraries: libhdf5.so.0: cannot open shared object file: No such file or directory - -(1) Are the compilers set correctly? ----Check with: which COMPILER - -(2) Have you installed HDF5? ----You can get it at: http://vis.lbl.gov/Research/AcceleratorSAPP/index.html - -(3) Have you loaded the proper modules? ----It is necessary in Davinci & Bassi, and configure && make install may not work without it. - +For more information on using the H5Part API, see the H5Part website: + + http://www-vis.lbl.gov/Research/AcceleratorSAPP/ + + +Motivation +---------- + +Particle based simulations of accelerator beam-lines, especially in six +dimensional phase space, generate vast amounts of data. Even though a subset +of statistical information regarding phase space or analysis needs to be +preserved, reading and writing such enormous restart files on massively +parallel supercomputing systems remains challenging. + +H5Part is a very simple data storage schema and provides an API that +simplifies the reading/writing of the data to the HDF5 file format. An +important foundation for a stable visualization and data analysis environment +is a stable and portable file storage format and its associated APIs. The +presence of a "common file storage format," including associated APIs, will +help foster a fundamental level of interoperability across the project's +software infrastructure. It will also help ensure that key data analysis +capabilities are present during the earliest phases of the software development +effort. + +H5Part is built on top of the HDF5 (Hierarchical Data Format) library. HDF5 +offers a self-describing machine-independent binary file format that supports +scalable parallel I/O performance for MPI codes on a variety of supercomputing +systems, and works equally well on laptop computers. The API is available for +C, C++, and Fortran codes. The H5Part file format and APIs enable disparate +research groups with different simulation implementations to transparently +share datasets and data analysis tools. For instance, the common file format +will enable groups that depend on completely different simulation +implementations to share data analysis tools. + + +File Organization and API +------------------------- + +The proposed file storage format uses HDF5 for the lowlevel file storage and a +simple API to provide a high-level interface to that file format. A programmer +can either use the H5Part API to access the data files or write directly to the +file format using some simple conventions for organizing and naming the objects +stored in the file. + +In order to store Particle Data in the HDF5 file format, we have formalized the +hierarchical arrangement of the datasets and naming conventions for the groups +and associated datasets. The sample H5Part API formally encodes these +conventions in order to provide a simple and uniform way to access these files +from C, C++, and Fortran codes. The API makes it easier to write very portable +data adaptors for visualization tools in order to expand the number of tools +available to access the data. Even so, anyone can use the HDF5 h5ls utility to +examine the organization of the H5Part files and even write their own +HDF5-based interface for reading and writing the file format. The standards +offered by the sample API are completely independent of the standard for +organizing data within the file.The file format supports the storage of +multiple timesteps of datasets that contain multiple fields.