+
+
+
+
+ 
Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding PSI Wiki? Send feedback
+
+
+
+
+
+
+
+ Setting up
+
+ 1 Setting up
+
+It is assumed that the Tesla K40c is already physically installed on your system. For now I only will discuss to set it up for a Linux based system. In order to check that your operating systems see the card, enter the following command in the terminal:
+
+ 1.4 Installation of
+
+Most of the installation steps are the same as described for 1.4.1 Install via
+
+The configure script for 1.4.2 Install via
+
+There is on more configuration switch
+ 2 Setting up
+
+ 2.3 Installation of DKS and
+
+To install DKS and 3 Setting up
+
+Since Apple is not providing an out-of-the-box OpenMP support on their macOS compiler framework (Xcode), typically
+
+
+
+
+
+ Setting up musrfit / DKS: high speed fitting with GPU's
+
+
+
+
+In the years 2016/2017 we explored ways to speed up current fitting frameworks, especially musrfit. This allows now to analyze histogram sets of high field spectrometers like HAL-9500 at PSI without the error-prone RRF fitting (see U. Locans and A. Suter, " musrfit - Real Time Parameter Fitting Using GPUs", accepted for the proceedings of the muSR2017 conference in Sapporo, and the Memo from A. Suter, "Rotating Reference Frame Fits", in the musrfit source code). At the same time it can help to speed-up elaborate global fits tremendously, and dealing properly with muonium. It also allows Apple Mac users to speed up their fitting code on the CPU. Currently it is not straight forward to get musrfit multi-threaded under macOS since Apple doesn't be default support OpenMP. DKS enables musrfit to utilize OpenCL instead which is present on macOS by default.
+
+Warning: before you run into the shop to buy a gamer graphic card or a Tesla card, make sure that you have an appropriate server with a sufficiently strong power supply!
+
+However, the current musrfit/DKS version doesn't yet support all theory functions on the GPU. In case the theory function is not yet available for the GPU, musrfit will fall back to the CPU implementation.
+
+Conceptually the setup of musrfit/DKS is as following: -
+
- install the latest hardware driver for your graphic card. +
- install the GPU SDK which enables number crunching (CUDA for NVIDIA, OpenCL for AMD) +
- install DKS +
- install the musrfit version which is DKS ready +
musrfit/DKS for the following systems will be discussed in some more detail: -
+
- NVIDIA Tesla K40c +
- AMD Graphic Card (Radeon R9 390X) +
- macOS in order to get OpenCL support +
musrfit with GPU acceleration and OpenCL support is described in the 'User manual of the μSR data analysis software musrfit '. The additional musrfit / DKS commands are labelled in MAROON .
+
+ 1 Setting up musrfit/DKS for a Tesla K40c (NVIDIA)
+
+It is assumed that the Tesla K40c is already physically installed on your system. For now I only will discuss to set it up for a Linux based system. In order to check that your operating systems see the card, enter the following command in the terminal:
+
++$ lspci | grep NVIDIA ++ +On the response you should find something like + +
+05:00.0 3D controller: NVIDIA Corporation GK110BGL [Tesla K40c] (rev a1) ++ +which means that the OS physically recognises your card. + +
1.1 Driver Installation for the Tesla K40c
+ +Next, you will need to download and install the driver for your card. Select the proper operating system, card, etc. from the NVIDIA download center. At PSI we are running currently Red Hat Enterprise Linux 7.x (RHEL) for which we will get a rpm (something likenvidia-diag-driver-local-repo-rhel7-375.66-1.x86_64.rpm). Install it and make sure there is no conflict with the nouveau driver of the system.
+
+1.2 Installation of CUDA
+ +Download the CUDA SDK form NVIDIA for your system. Again, for the RHEL 7.x this is an rpm. After the installation of the rpm you should reboot your machine. Afterwards you are ready for the installation of DKS. + + +1.3 Installation of DKS
+ +For the following list of commands the '$' will be given as the command prompt. Do not enter it! Also some comments will be added starting with a '#' which can be omitted. They are only there to explain what is going on. +DKS stands for Dynamical Kernel Scheduler and provides a thin interface allowing host applications to incorporate GPU's and other hardware accelerators.
+
+Details can be found in the papers listed here, or on the DKS wiki page.
+
+In brief the installation should be something like this:+# go to whatever directory you would like to clone/install DKS +# For macOS DKS will likely to got to $HOME/Applications to be consistent with the musrfit docu for macOS +$ cd $HOME/Apps +$ git clone https://gitlab.psi.ch/uldis_l/DKS.git +$ cd DKS +$ mkdir build +$ cd build +$ cmake ../ -DENABLE_MUSR=1 -DCMAKE_INSTALL_PREFIX=../exec +$ cmake --build ./ --clean-first +$ make install ++ +Since
DKS is installed in a non-standard path, a couple of additional small steps are required. This will be different for Linux compared to macOS.
+
+For Linux:
+
+add the DKS library path to /etc/ld.so.conf.d/musrfit-x86_64.conf and execute as super user
+
++$ /sbin/ldconfig ++ +For macOS: + +add the
DKS path to $HOME/.profile:
+
++export DKS=$HOME/Applications/DKS/exec +export LD_LIBRARY_PATH=$DKS/lib:$LD_LIBRARY_PATH + +launchctl setenv DKS $DKS +launchctl setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH ++ +
1.4 Installation of musrfit for DKS
+
+Most of the installation steps are the same as described for musrfit without GPU support.
+Here only the differences are explained. First checkout musrfit, then you will need to switch the working branch which is done by+$ cd $HOME/Apps/musrfit +$ git checkout dks6 ++ +
1.4.1 Install via automake
+
+The configure script for musrfit has now a couple of more tags/options, which are: -
+
-
--enable-dks - by default this is set to
no, hence if you want to have GPU support you need to enable it. + -
--with-cuda - with this tag you tell the configure script where it can find CUDA +
-
--with-opencl - with this tag you tell the configure script where it can find OpenCL +
-
--with-dks - with this tag you tell the configure script where it can find
DKS+
+$ ./configure --enable-ASlibs --enable-BMWlibs --enable-NeXus --with-dks=$HOME/Apps/DKS/exec --enable-dks --prefix=$ROOTSYS ++whereas for macOS it would typically look like this +
+$ ./configure --enable-ASlibs --enable-BMWlibs --enable-NeXus --with-dks=$HOME/Applications/DKS/exec --enable-dks --prefix=$ROOTSYS ++After +
+$ make +$ make install ++and updating the shared library lookup table (only needed for Linux) +
+/sbin/ldconfig # as superuser / root ++you are done with the setup. + +
1.4.2 Install via cmake
+
+There is on more configuration switch
+ -
+
-
-Ddks=<value> - it allows to enable/disable =DKS support. The default is <value>=1, i.e. enabled. To disable use <value>=0. +
+$ cmake ../ -DCMAKE_INSTALL_PREFIX=$ROOTSYS -DASlibs=1 -DBMWlibs=1 -Dnexus=1 ++whereas for macOS it would typically look like this +
+$ cmake ../ -DCMAKE_INSTALL_PREFIX=$ROOTSYS -DASlibs=1 -DBMWlibs=1 -Dnexus=1 ++ +After +
+$ cmake --build ./ --clean-first +$ make install ++and updating the shared library lookup table (only needed for Linux) +
+/sbin/ldconfig # as superuser / root ++you are done with the setup. + +
2 Setting up musrfit/DKS for a AMD Graphic Card (Radeon R9 390X)
+
+2.1 Driver Installation for an AMD Graphic Card, e.g. Radeon R9 390X
+ +This will depend slightly on the AMD Card and operating system. Here I will summaries how it was done on a RHEL (Linux) system using a Radeon R9 390X. + +It is assumed that the Radeon R9 390X is already physically installed on your system. For now I only will discuss to set it up for a Linux based system. In order to check that your operating systems see the card, enter the following command in the terminal: + ++$ lspci | grep NVIDIA ++ +On the response you should find something like +
+84:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii XT / Grenada XT [Radeon R9 290X/390X] (rev 80) ++which means that the OS physically recognises your card. + +For RHEL7.x the AMDGPU-PRO driver should be used. It can be downloaded from here. +Unpack the driver +
+$ tar -Jxvf amdgpu-pro-17.10-414273.tar.xz +$ cd amdgpu-pro-17.10-414273 ++ +Install the driver as root +
+$ ./amdgpu-pro-install --compute -y ++ +Here I assume that the AMD graphic card is only used for computation. You need to add the following command in order that the user 'blabla' (chance this to the appropriate user name) can access the GPU (otherwise only root works): +
+$ /sbin/usermod -a -G video blabla ++ +Reboot the machine. + +
2.2 AMD APP Software Development Kit (SDK) to enable OpenCL support
+ +The AMD APP Software Development Kit (SDK) is a complete development platform created by AMD to allow you to quickly and easily develop applications accelerated by AMD APP technology. The SDK provides samples, documentation, and other materials to quickly get you started leveraging accelerated compute using OpenCL orC++ AMP in your C/C++ applications.
+
+Download the AMD APP SDK 3.0 from here.
+
+Extract the installer
++$ tar -xvjf AMD-APP-SDKInstaller-v3.0.130.136-GA-linux64.tar.bz2 ++ +Run the installer +
+$ ./AMD-APP-SDK-v3.0.130.136-GA-linux64.sh ++ +This will install the AMD APP SDK to /opt/AMDAPPSDK-3.0/ where you can find the OpenCL include and library files, as well as documentation and sample code. +The install guide for AMD OpenCL SDK can be found here. + +
2.3 Installation of DKS and musrfit
+
+To install DKS and musrfit follow the instructions here.
+
+ 3 Setting up musrfit/DKS for macOS for OpenCL support
+
+Since Apple is not providing an out-of-the-box OpenMP support on their macOS compiler framework (Xcode), typically musrfit is just running single threaded. Here DKS can help since it delivers OpenCL support which is present on macOS. Hence, if you would like to run musrfit multi-threaded the easiest way is to use DKS.
+
+Since there is no graphic card involved, you do not need any graphic card driver of additional SDK. The only thing you need DKS and the proper musrfit version.
+
+The installation instruction for DKS / musrfit can be found here.
+
+-- AndreasSuter - June 8, 2018Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems regarding PSI Wiki? Send feedback
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/doc/html/user/MUSR/MusrGui.html b/doc/html/user/MUSR/MusrGui.html
index 2d1ef2ff..70fd6790 100644
--- a/doc/html/user/MUSR/MusrGui.html
+++ b/doc/html/user/MUSR/MusrGui.html
@@ -1,6 +1,6 @@
-
+
@@ -14,12 +14,11 @@
-
+
-