musrfit/doc/html/user/MUSR/MusrFitSetupDKS.html

390 lines
22 KiB
HTML

<!DOCTYPE html><html lang="en">
<!-- Mirrored from intranet.psi.ch/MUSR/MusrFitSetupDKS?cover=print by HTTrack Website Copier/3.x [XR&CO'2014], Fri, 08 Jun 2018 13:41:56 GMT -->
<!-- Added by HTTrack --><meta http-equiv="content-type" content="text/html;charset=utf-8" /><!-- /Added by HTTrack -->
<head>
<link rel="stylesheet" href="../pub/System/HeadlinesPlugin/style.css" type="text/css" media="all" />
<title> MUSR :: MusrFitSetupDKS</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1.5, user-scalable=yes" />
<meta name="mobile-web-app-capable" content="yes" />
<meta name="mobile-web-app-status-bar-style" content="black-translucent" />
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<link rel="icon" href="../pub/Main/WebPreferences/favicon.ico" type="image/x-icon" />
<link rel="shortcut icon" href="../pub/Main/WebPreferences/favicon.ico" type="image/x-icon" />
<link rel="alternate" href="https://intranet.psi.ch/wiki/bin/edit/MUSR/MusrFitSetupDKS?t=1528465296" type="application/x-wiki" title="edit MusrFitSetupDKS" />
<meta name="TEXT_NUM_TOPICS" content="Number of topics:" />
<meta name="TEXT_MODIFY_SEARCH" content="Modify search" />
<meta name="robots" content="noindex" /><link rel="alternate" type="application/rss+xml" title="RSS Feed" href="WebRss" />
<base /><!--[if IE]></base><![endif]--><link class='head SMILIESPLUGIN' rel='stylesheet' href='../pub/System/SmiliesPlugin/smilies.css' type='text/css' media='all' /><!--SMILIESPLUGIN-->
<link class='head FOOTNOTEPLUGIN_LINKCSS' rel="stylesheet" href="../pub/System/FootNotePlugin/styles.css" type="text/css" media="all" /><!--FOOTNOTEPLUGIN_LINKCSS-->
<link class='head CLASSIFICATIONPLUGIN::CSS' rel="stylesheet" href="../pub/System/ClassificationPlugin/styles.css" media="all" /><!--CLASSIFICATIONPLUGIN::CSS-->
<link class='head JQUERYPLUGIN::COMMENT' rel='stylesheet' href='../pub/System/CommentPlugin/commentf5b6.css?version=3.0' type='text/css' media='all' /><!--JQUERYPLUGIN::COMMENT--><link rel='stylesheet' href='../pub/System/SkinTemplates/base.css' media='all' type='text/css' />
<style type="text/css" media="all">
@import url('../pub/System/PatternSkinTheme/layout.css');
@import url('../pub/System/PatternSkinTheme2009/style.css');
@import url('../pub/System/PatternSkinTheme2009/colors.css');
</style>
<link rel='stylesheet' href='../pub/System/PatternSkinTheme2009/column_left.css' media='all' type='text/css' />
<link rel='stylesheet' href='../pub/System/PatternSkinTheme2009/variant_twiki.css' media='all' type='text/css' /><style type="text/css" media="all">
/* Styles that are set using variables */
.patternBookView .foswikiTopRow,
.patternWebIndicator,
.patternWebIndicator a {
background-color:#D0D0D0;
}
#patternTopBarContents { background-image:url(../pub/System/PatternSkin/header5.gif); background-repeat:repeat-x;}
#patternTopBarContents { background-color:#ffffff;}
.patternBookView {
border-color:#D0D0D0;
}
.patternPreviewPage #patternMain {
/* uncomment to set the preview image */
/*background-image:url("/pub/System/PreviewBackground/preview2bg.gif");*/
}
</style>
<style type="text/css" media="all">
@import url("../pub/System/PsiSkin/psicolors.css");
@import url("../pub/System/PsiSkin/psistyle.css");
.foswikiTopic {overflow: hidden;}
</style>
<link rel='stylesheet' href='../pub/System/PatternSkin/print.css' media='all' type='text/css' />
<!--[if IE 7 ]><style type="text/css" media="screen">
.foswikiAttachments,
.foswikiForm,
.foswikiHelp,
.foswikiPreviewArea,
.patternPreviewPage .foswikiForm,
.patternSigLine,
.patternToolBar,
.patternTop,
.patternTopicAction,
#patternSideBarContents .patternLeftBarPersonal,
#patternSideBarContents h2,
#patternSideBarContents li,
#patternTopBarButtons ul {
height:1%;
}
#patternSideBarContents .patternLeftBarPersonal {
width:100%;
}
.foswikiFormStep {
height:100%;
}
#foswikiLogin {
margin-left: 25%;
margin-right: 25%;
}
#foswikiLogin .foswikiFormSteps {
width:auto;
}
#foswikiLogin input[type="text"],
#foswikiLogin input[type="password"] {
width: auto;
}
.patternShadow {
border:10px solid #fff;
margin-top:10px;
margin-bottom:10px;
border:2px solid #ccc;
}
</style><![endif]-->
<!--[if lte IE 9]>
<![endif]-->
<!--[if gt IE 9]><!-->
<!--<![endif]-->
<!--JQUERYPLUGIN-->
<!--JQUERYPLUGIN::MIGRATE-->
<!--JQUERYPLUGIN::LIVEQUERY-->
<!--JQUERYPLUGIN::FOSWIKI-->
<!--JavascriptFiles/foswikiString-->
<!--JavascriptFiles/foswikiPref-->
<!--JavascriptFiles/foswikiForm-->
<!--PatternSkin/pattern-->
<!--JQUERYPLUGIN::COMMENT-->
<!--JQUERYPLUGIN::FOSWIKI::PREFERENCES-->
<!--JQUERYPLUGIN::METADATA-->
<!--JQUERYPLUGIN::CHILI--><link class='head FOOTNOTEPLUGIN_LINKCSS' rel="stylesheet" href="../pub/System/FootNotePlugin/styles.css" type="text/css" media="all" /><!--FOOTNOTEPLUGIN_LINKCSS-->
</head>
<body class="foswikiNoJs patternViewPage patternPrintPage">
<span id="PageTop"></span><div class="foswikiPage"><div id="patternScreen">
<div id="patternPageShadow">
<div id="patternPage">
<div id="patternOuter" class="clear">
<div id="patternFloatWrap">
<div id="patternMain">
<div id="patternMainContents">
<div class="patternContent"><div class="foswikiTopic"> <h1 id="Setting_up_61musrfit_61_47_61DKS_61:_high_speed_fitting_with_GPU_39s"> Setting up <code>musrfit</code> / <code>DKS</code>: high speed fitting with GPU's </h1>
<p></p>
<div class="foswikiToc" id="foswikiTOC"> <ul>
<li> <a href="MusrFitSetupDKS.html#Setting_up_61musrfit_61_47_61DKS_61:_high_speed_fitting_with_GPU_39s"> Setting up musrfit / DKS: high speed fitting with GPU's </a>
</li> <li> <a href="MusrFitSetupDKS.html#A_1_Setting_up_61_61musrfit_47DKS_61_61_for_a_Tesla_K40c_40NVIDIA_41"> 1 Setting up musrfit/DKS for a Tesla K40c (NVIDIA) </a> <ul>
<li> <a href="MusrFitSetupDKS.html#A_1.1_Driver_Installation_for_the_Tesla_K40c"> 1.1 Driver Installation for the Tesla K40c </a>
</li> <li> <a href="MusrFitSetupDKS.html#A_1.2_Installation_of_CUDA"> 1.2 Installation of CUDA </a>
</li> <li> <a href="MusrFitSetupDKS.html#A_1.3_Installation_of_DKS"> 1.3 Installation of DKS </a>
</li> <li> <a href="MusrFitSetupDKS.html#A_1.4_Installation_of_61musrfit_61_for_DKS"> 1.4 Installation of musrfit for DKS </a> <ul>
<li> <a href="MusrFitSetupDKS.html#A_1.4.1_Install_via_61_61automake_61_61"> 1.4.1 Install via automake </a>
</li> <li> <a href="MusrFitSetupDKS.html#A_1.4.2_Install_via_61_61cmake_61_61"> 1.4.2 Install via cmake </a>
</li></ul>
</li></ul>
</li> <li> <a href="MusrFitSetupDKS.html#A_2_Setting_up_61_61musrfit_47DKS_61_61_for_a_AMD_Graphic_Card_40Radeon_R9_390X_41"> 2 Setting up musrfit/DKS for a AMD Graphic Card (Radeon R9 390X) </a> <ul>
<li> <a href="MusrFitSetupDKS.html#A_2.1_Driver_Installation_for_an_AMD_Graphic_Card_44_e.g._Radeon_R9_390X"> 2.1 Driver Installation for an AMD Graphic Card, e.g. Radeon R9 390X </a>
</li> <li> <a href="MusrFitSetupDKS.html#A_2.2_AMD_APP_Software_Development_Kit_40SDK_41_to_enable_OpenCL_support"> 2.2 AMD APP Software Development Kit (SDK) to enable OpenCL support </a>
</li> <li> <a href="MusrFitSetupDKS.html#A_2.3_Installation_of_DKS_and_61musrfit_61"> 2.3 Installation of DKS and musrfit </a>
</li></ul>
</li> <li> <a href="MusrFitSetupDKS.html#A_3_Setting_up_61_61musrfit_47DKS_61_61_for_macOS_for_OpenCL_support"> 3 Setting up musrfit/DKS for macOS for OpenCL support </a>
</li></ul>
</div>
<p></p>
In the years 2016/2017 we explored ways to speed up current fitting frameworks, especially <code>musrfit</code>. This allows now to analyze histogram sets of high field spectrometers like HAL-9500 at PSI without the error-prone RRF fitting (see U. Locans and A. Suter, " <code>musrfit</code> - Real Time Parameter Fitting Using GPUs", accepted for the proceedings of the muSR2017 conference in Sapporo, and the Memo from A. Suter, "Rotating Reference Frame Fits", in the <code>musrfit</code> source code). At the same time it can help to speed-up elaborate global fits tremendously, and dealing properly with muonium. It also allows Apple Mac users to speed up their fitting code on the CPU. Currently it is not straight forward to get <code>musrfit</code> multi-threaded under macOS since Apple doesn't be default support OpenMP. DKS enables <code>musrfit</code> to utilize OpenCL instead which is present on macOS by default.
<p></p>
<strong>Warning</strong>: before you run into the shop to buy a gamer graphic card or a Tesla card, make sure that you have an appropriate server with a sufficiently strong power supply!
<p></p>
<strong>However, the current <code>musrfit/DKS</code> version doesn't yet support all theory functions on the GPU.</strong> In case the theory function is not yet available for the GPU, <code>musrfit</code> will fall back to the CPU implementation.
<p></p>
Conceptually the setup of <code>musrfit/DKS</code> is as following: <ol>
<li> install the latest hardware driver for your graphic card.
</li> <li> install the GPU SDK which enables number crunching (CUDA for NVIDIA, OpenCL for AMD)
</li> <li> install DKS
</li> <li> install the musrfit version which is DKS ready
</li></ol>
<p></p>
In the following the description for the installation of <code>musrfit/DKS</code> for the following systems will be discussed in some more detail: <ul>
<li> NVIDIA Tesla K40c
</li> <li> AMD Graphic Card (Radeon R9 390X)
</li> <li> macOS in order to get OpenCL support
</li></ul>
<p></p>
The <strong>usage</strong> of <code>musrfit</code> with GPU acceleration and OpenCL support is described in the <a href="MusrFit.html">'User manual of the &#956;SR data analysis software <code>musrfit</code> '</a>. The additional <code>musrfit</code> / DKS commands are labelled in <span class='foswikiMaroonFG'> <strong>MAROON</strong> </span>.
<p></p>
<h1 id="A_1_Setting_up_61_61musrfit_47DKS_61_61_for_a_Tesla_K40c_40NVIDIA_41"> 1 Setting up <code><b>musrfit/DKS</b></code> for a Tesla K40c (NVIDIA) </h1>
<p></p>
It is assumed that the Tesla K40c is already physically installed on your system. For now I only will discuss to set it up for a Linux based system. In order to check that your operating systems see the card, enter the following command in the terminal:
<p></p>
<pre class="bash">
&#36; lspci &#124; grep NVIDIA
</pre>
<p></p>
On the response you should find something like
<p></p>
<pre class="bash">
05:00.0 3D controller: NVIDIA Corporation GK110BGL &#91;Tesla K40c&#93; (rev a1)
</pre>
<p></p>
which means that the OS physically recognises your card.
<p></p>
<h2 id="A_1.1_Driver_Installation_for_the_Tesla_K40c"> 1.1 Driver Installation for the Tesla K40c </h2>
<p></p>
Next, you will need to download and install the driver for your card. Select the proper operating system, card, etc. from the <a href="http://www.nvidia.com/Download/index.aspx?lang=en-us">NVIDIA download center</a>. At PSI we are running currently Red Hat Enterprise Linux 7.x (RHEL) for which we will get a rpm (something like <code>nvidia-diag-driver-local-repo-rhel7-375.66-1.x86_64.rpm</code>). Install it and make sure there is no conflict with the nouveau driver of the system.
<p></p>
<h2 id="A_1.2_Installation_of_CUDA"> 1.2 Installation of CUDA </h2>
<p></p>
Download the <a href="https://developer.nvidia.com/cuda-downloads">CUDA SDK</a> form NVIDIA for your system. Again, for the RHEL 7.x this is an rpm. After the installation of the rpm you should reboot your machine. Afterwards you are ready for the installation of DKS.
<p></p>
<span id="DksInstall"></span>
<h2 id="A_1.3_Installation_of_DKS"> 1.3 Installation of DKS </h2>
<p></p>
For the following list of commands the '$' will be given as the command prompt. Do not enter it! Also some comments will be added starting with a '#' which can be omitted. They are only there to explain what is going on.
<code>DKS</code> stands for Dynamical Kernel Scheduler and provides a thin interface allowing host applications to incorporate GPU's and other hardware accelerators.
<p></p>
Details can be found in the papers listed <a href="CiteMusrFit.html">here</a>, or on the <a href="https://gitlab.psi.ch/uldis_l/DKS/wikis/home">DKS wiki page</a>.
<p></p>
In brief the installation should be something like this:<pre class="bash">
# go to whatever directory you would like to clone/install DKS
# For macOS DKS will likely to got to &#36;HOME/Applications to be consistent with the musrfit docu for macOS
&#36; cd &#36;HOME/Apps
&#36; git clone https://gitlab.psi.ch/uldis&#95;l/DKS.git
&#36; cd DKS
&#36; mkdir build
&#36; cd build
&#36; cmake ../ -DENABLE&#95;MUSR&#61;1 -DCMAKE&#95;INSTALL&#95;PREFIX&#61;../exec
&#36; cmake --build ./ --clean-first
&#36; make install
</pre>
<p></p>
Since <code>DKS</code> is installed in a non-standard path, a couple of additional small steps are required. This will be different for Linux compared to macOS.
<p></p>
For <strong>Linux</strong>:
<p></p>
add the <code>DKS</code> library path to <code>/etc/ld.so.conf.d/musrfit-x86_64.conf</code> and execute as super user
<p></p>
<pre class="bash">
&#36; /sbin/ldconfig
</pre>
<p></p>
For <strong>macOS</strong>:
<p></p>
add the <code>DKS</code> path to <code>$HOME/.profile</code>:
<p></p>
<pre class="bash">
export DKS&#61;&#36;HOME/Applications/DKS/exec
export LD&#95;LIBRARY&#95;PATH&#61;&#36;DKS/lib:&#36;LD&#95;LIBRARY&#95;PATH
launchctl setenv DKS &#36;DKS
launchctl setenv LD&#95;LIBRARY&#95;PATH &#36;LD&#95;LIBRARY&#95;PATH
</pre>
<p></p>
<h2 id="A_1.4_Installation_of_61musrfit_61_for_DKS"> 1.4 Installation of <code>musrfit</code> for DKS </h2>
<p></p>
Most of the installation steps are the same as described for <a href="MusrFitSetup.html"><code>musrfit</code></a> without GPU support.
Here only the differences are explained. First checkout <code>musrfit</code>, then you will need to switch the working branch which is done by<pre class="bash">
&#36; cd &#36;HOME/Apps/musrfit
&#36; git checkout dks6
</pre>
<p></p>
<h3 id="A_1.4.1_Install_via_61_61automake_61_61"> 1.4.1 Install via <code><b>automake</b></code> </h3>
<p></p>
The configure script for <code>musrfit</code> has now a couple of more tags/options, which are: <dl>
<dt> <code><b>--enable-dks</b></code> </dt><dd> by default this is set to <code><b>no</b></code>, hence if you want to have GPU support you need to enable it.
</dd> <dt> <code><b>--with-cuda</b></code> </dt><dd> with this tag you tell the configure script where it can find CUDA
</dd> <dt> <code><b>--with-opencl</b></code> </dt><dd> with this tag you tell the configure script where it can find OpenCL
</dd> <dt> <code><b>--with-dks</b></code> </dt><dd> with this tag you tell the configure script where it can find <code>DKS</code>
</dd></dl>
<p></p>
For a typical setup on a RHEL system it could look like this
<pre class="bash">
&#36; ./configure --enable-ASlibs --enable-BMWlibs --enable-NeXus --with-dks&#61;&#36;HOME/Apps/DKS/exec --enable-dks --prefix&#61;&#36;ROOTSYS
</pre>
whereas for macOS it would typically look like this
<pre class="bash">
&#36; ./configure --enable-ASlibs --enable-BMWlibs --enable-NeXus --with-dks&#61;&#36;HOME/Applications/DKS/exec --enable-dks --prefix&#61;&#36;ROOTSYS
</pre>
After
<pre class="bash">
&#36; make
&#36; make install
</pre>
and updating the shared library lookup table (only needed for Linux)
<pre class="bash">
/sbin/ldconfig # as superuser / root
</pre>
you are done with the setup.
<p></p>
<h3 id="A_1.4.2_Install_via_61_61cmake_61_61"> 1.4.2 Install via <code><b>cmake</b></code> </h3>
<p></p>
There is on more configuration switch
<p></p> <dl>
<dt> <code>-Ddks=&lt;value&gt; </dt><dd> it allows to enable/disable =DKS</code> support. The default is &lt;value&gt;=1, i.e. <strong>enabled</strong>. To disable use &lt;value&gt;=0.
</dd></dl>
<p></p>
For a typical setup on a RHEL system it could look like this
<pre class="bash">
&#36; cmake ../ -DCMAKE&#95;INSTALL&#95;PREFIX&#61;&#36;ROOTSYS -DASlibs&#61;1 -DBMWlibs&#61;1 -Dnexus&#61;1
</pre>
whereas for macOS it would typically look like this
<pre class="bash">
&#36; cmake ../ -DCMAKE&#95;INSTALL&#95;PREFIX&#61;&#36;ROOTSYS -DASlibs&#61;1 -DBMWlibs&#61;1 -Dnexus&#61;1
</pre>
<p></p>
After
<pre class="bash">
&#36; cmake --build ./ --clean-first
&#36; make install
</pre>
and updating the shared library lookup table (only needed for Linux)
<pre class="bash">
/sbin/ldconfig # as superuser / root
</pre>
you are done with the setup.
<p></p>
<h1 id="A_2_Setting_up_61_61musrfit_47DKS_61_61_for_a_AMD_Graphic_Card_40Radeon_R9_390X_41"> 2 Setting up <code><b>musrfit/DKS</b></code> for a AMD Graphic Card (Radeon R9 390X) </h1>
<p></p>
<h2 id="A_2.1_Driver_Installation_for_an_AMD_Graphic_Card_44_e.g._Radeon_R9_390X"> 2.1 Driver Installation for an AMD Graphic Card, e.g. Radeon R9 390X </h2>
<p></p>
This will depend slightly on the AMD Card and operating system. Here I will summaries how it was done on a RHEL (Linux) system using a Radeon R9 390X.
<p></p>
It is assumed that the Radeon R9 390X is already physically installed on your system. For now I only will discuss to set it up for a Linux based system. In order to check that your operating systems see the card, enter the following command in the terminal:
<p></p>
<pre class="bash">
&#36; lspci &#124; grep NVIDIA
</pre>
<p></p>
On the response you should find something like
<pre class="bash">
84:00.0 VGA compatible controller: Advanced Micro Devices, Inc. &#91;AMD/ATI&#93; Hawaii XT / Grenada XT &#91;Radeon R9 290X/390X&#93; (rev 80)
</pre>
which means that the OS physically recognises your card.
<p></p>
For RHEL7.x the AMDGPU-PRO driver should be used. It can be downloaded from <a href="http://support.amd.com/en-us/kb-articles/Pages/AMDGPU-PRO-RedHat-Install.aspx">here</a>.
Unpack the driver
<pre class="bash">
&#36; tar -Jxvf amdgpu-pro-17.10-414273.tar.xz
&#36; cd amdgpu-pro-17.10-414273
</pre>
<p></p>
Install the driver as root
<pre class="bash">
&#36; ./amdgpu-pro-install --compute -y
</pre>
<p></p>
Here I assume that the AMD graphic card is only used for computation. You need to add the following command in order that the user 'blabla' (chance this to the appropriate user name) can access the GPU (otherwise only root works):
<pre class="bash">
&#36; /sbin/usermod -a -G video blabla
</pre>
<p></p>
Reboot the machine.
<p></p>
<h2 id="A_2.2_AMD_APP_Software_Development_Kit_40SDK_41_to_enable_OpenCL_support"> 2.2 AMD APP Software Development Kit (SDK) to enable OpenCL support </h2>
<p></p>
The AMD APP Software Development Kit (SDK) is a complete development platform created by AMD to allow you to quickly and easily develop applications accelerated by AMD APP technology. The SDK provides samples, documentation, and other materials to quickly get you started leveraging accelerated compute using OpenCL or <code>C++ AMP</code> in your <code>C/C++</code> applications.
<p></p>
Download the AMD APP SDK 3.0 from <a href="http://developer.amd.com/tools-and-sdks/opencl-zone/amd-accelerated-parallel-processing-app-sdk/">here</a>.
<p></p>
Extract the installer
<pre class="bash">
&#36; tar -xvjf AMD-APP-SDKInstaller-v3.0.130.136-GA-linux64.tar.bz2
</pre>
<p></p>
Run the installer
<pre class="bash">
&#36; ./AMD-APP-SDK-v3.0.130.136-GA-linux64.sh
</pre>
<p></p>
This will install the AMD APP SDK to /opt/AMDAPPSDK-3.0/ where you can find the OpenCL include and library files, as well as documentation and sample code.
The install guide for AMD OpenCL SDK can be found <a href="http://developer.amd.com/wordpress/media/2012/10/AMD_APP_SDK_InstallationNotes.pdf">here</a>.
<p></p>
<h2 id="A_2.3_Installation_of_DKS_and_61musrfit_61"> 2.3 Installation of DKS and <code>musrfit</code> </h2>
<p></p>
To install DKS and <code>musrfit</code> follow the instructions <a class="foswikiCurrentTopicLink" href="#DksInstall">here</a>.
<p></p>
<h1 id="A_3_Setting_up_61_61musrfit_47DKS_61_61_for_macOS_for_OpenCL_support"> 3 Setting up <code><b>musrfit/DKS</b></code> for macOS for OpenCL support </h1>
<p></p>
Since Apple is not providing an out-of-the-box OpenMP support on their macOS compiler framework (Xcode), typically <code>musrfit</code> is just running single threaded. Here DKS can help since it delivers OpenCL support which is present on macOS. Hence, if you would like to run <code>musrfit</code> multi-threaded the easiest way is to use DKS.
<p></p>
Since there is no graphic card involved, you do not need any graphic card driver of additional SDK. The only thing you need DKS and the proper <code>musrfit</code> version.
<p></p>
The installation instruction for DKS / <code>musrfit</code> can be found <a class="foswikiCurrentTopicLink" href="#DksInstall">here</a>.
<p></p>
-- <a href="https://intranet.psi.ch/Main/AndreasSuter">AndreasSuter</a> - June 8, 2018</div>
<p></p>
<p></p>
</div>
<div class="patternInfo">This topic: MUSR<span class='foswikiSeparator'>&nbsp;&gt;&nbsp;</span><a class="foswikiCurrentWebHomeLink" href="WebHome.html">WebHome</a><span class='foswikiSeparator'>&nbsp;&gt;&nbsp;</span>MusrFitSetupDKS <br />
Topic revision: <span class='patternRevInfo'>08 Jun 2018, <a href="https://intranet.psi.ch/Main/AndreasSuter">AndreasSuter</a></span></div>
</div>
</div>
</div>
</div><div id="patternBottomBar"><div id="patternBottomBarContents"><div id="patternWebBottomBar"><span class="foswikiRight"> <a href="http://foswiki.org/"><img src="../pub/System/ProjectLogos/foswiki-badge.png" alt="This site is powered by Foswiki" title="This site is powered by Foswiki" /></a></span>Copyright &copy; by the contributing authors. All material on this collaboration platform is the property of the contributing authors. <br /> Ideas, requests, problems regarding PSI Wiki? <a href='mailto:thomas.buecklers@psi.ch?subject=PSI Wiki %20Feedback%20on%20MUSR.MusrFitSetupDKS'>Send feedback</a></div></div></div>
</div>
</div>
</div>
</div></div><!-- /endWrap -->
<p></p>
<!-- Piwik -->
<noscript><p><img src="../piwik/piwikf832.gif?idsite=7" style="border:0" alt=""></p></noscript>
<!-- End Piwik Tag -->
</body>
<!-- Mirrored from intranet.psi.ch/MUSR/MusrFitSetupDKS?cover=print by HTTrack Website Copier/3.x [XR&CO'2014], Fri, 08 Jun 2018 13:41:56 GMT -->
</html>