Merge branch 'anaconda' into 'master'

Add scicat_client to the datacatalog module

See merge request Pmodules/buildblocks!118

Programming/anaconda/2019.07/conda-env-defs/hpce-tools/hpce-tools.yml

@@ -6,5 +6,15 @@ channels:
 dependencies:
   - python=3.6
   - ldapuserdir
-  - cython  # for pyslurm
+  # python development
   - conda-build
+  - cython  # for pyslurm
+  - black
+  - flake8
+  - mypy
+  - mypy_extensions
+  - typing_extensions
+  - tox
+  - tox-conda
+  - pytest
+  - bpython

Programming/anaconda/2019.07/conda-env-defs/scicat_client/README.md

@@ -0,0 +1,20 @@
+# scicat-client
+
+The scicat-client environment is used in the datacatalog module. Contact
+Spencer Bliven, Stephan Egli, or Leo Sala for more info.
+
+## Installing
+
+First, create the conda environment
+
+    conda env create -f scicat_client.yml
+
+Next, install scicat_client. Eventually this should be done automatically
+through conda, but for now it needs to be installed from source.
+
+    git clone git@github.com:paulscherrerinstitute/scicat_client.git
+    cd scicat_client
+    conda activate scicat_client-0.1.0
+    python setup.py install
+
+

Programming/anaconda/2019.07/conda-env-defs/scicat_client/scicat_client.yml

@@ -0,0 +1,18 @@
+# Clean environment based on pure conda-forge packages
+name: scicat_client-0.1.0
+channels:
+  - conda-forge
+  - http://conda-pkg.intranet.psi.ch
+dependencies:
+  - python=3.8
+  - requests
+  # development tools
+  - conda-build
+  - black
+  - flake8
+  - mypy
+  - mypy_extensions
+  - typing_extensions
+  - tox
+  - tox-conda
+  - pytest

Tools/datacatalog/README.md

@@ -0,0 +1,20 @@
+# Datacatalog
+
+## Overview 
+
+This module provides tools to interface with the Data Catalog (discovery.psi.ch).
+
+## Installation
+
+Run `./build <VERSION>` to install the latest version. This downloads the
+latest versions of the datasetIngestor, datasetRetriever, and datasetArchiver
+tools. Note that the downloads are not versioned, so make sure that the current
+variant matches the installed binaries.
+
+The `scicat_client` script is also installed. This is maintained as an anaconda
+environment, then symlinked into the datacatalog pmodule. Anaconda hard-codes
+the correct python interpreter, so all dependencies should resolve even though
+the conda module is not activated. See
+Programming/anaconda/2019.07/conda-env-defs/scicat_client for environment
+installation details.
+

Tools/datacatalog/README.md_template

@@ -1,21 +0,0 @@
-# The Mellanox MXM communication library
-
-## Overview 
-
-The Mellanox MXM communication library provides support for the Mellanox MXM interface for InfiniBand.
-
-## Installation
-
-For the module we use a RPM distributed by HP.
-
-1. Create new directory `/opt/psi/System/mxm/VERSION_merlin`
-1. Download RPM from https://downloads.linux.hpe.com/sdr/repo/mlnx_ofed/RedHatEnterpriseServer/
-1. Unpack RPM with `rpm2cpio RPM | cpio -i --make-dirs` somewhere
-1. copy all files from `opt/mellanox/mxm` to the module directory
-1. adapt directories in `lib/pkg-config/mxm.pc`
-1. add new variant to `files/variants`
-1. run the build-script to install the modulefile and to set the release
- 
-
-> **Note:** The shared library `libmxm.so`provided by the RPMs for RHEL 6 cannot be used to compile other software.
-They requiry GLIBC >= 2.14, but on RHEL 6 only 2.12 is installed!

Tools/datacatalog/build

@@ -25,5 +25,8 @@ pbuild::install() {
 
    /usr/bin/curl -o "$PREFIX/bin/datasetArchiver" https://intranet.psi.ch/pub/Daas/WebHome/datasetArchiver
    chmod +x "$PREFIX/bin/datasetArchiver"
+
+   # Hardcoded; could be made version-specific in the future if needed
+   ln -s "${PMODULES_ROOT}/Programming/anaconda/2019.07/conda/envs/scicat_client-0.1.0/bin/scicat_client" "$PREFIX/bin/scicat_client"
 }