84 lines
3.5 KiB
Markdown
84 lines
3.5 KiB
Markdown
---
|
|
title: Jupyterhub on Merlin
|
|
#tags:
|
|
#keywords:
|
|
last_updated: 31 July 2019
|
|
#summary: ""
|
|
sidebar: merlin6_sidebar
|
|
permalink: /merlin6/jupyterhub.html
|
|
---
|
|
|
|
Jupyterhub provides [jupyter notebooks](https://jupyter.org/) that are launched on
|
|
cluster nodes of merlin and can be accessed through a web portal.
|
|
|
|
## Accessing Jupyterhub and launching a session
|
|
|
|
The service is available inside of PSI (or through a VPN connection) at
|
|
|
|
|
|
**<https://merlin-jupyter.psi.ch:8000>**
|
|
|
|
|
|
1. **Login**: You will be presented with a **Login** web page for
|
|
authenticating with your PSI account.
|
|
1. **Spawn job**: The **Spawner Options** page allows you to
|
|
specify the properties (Slurm partition, running time,...) of
|
|
the batch jobs that will be running your jupyter notebook. Once
|
|
you click on the `Spawn` button, your job will be sent to the
|
|
Slurm batch system. If the cluster is not currently overloaded
|
|
and the resources you requested are available, your job will
|
|
usually start within 30 seconds.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Jupyter software environments - running different kernels
|
|
|
|
Your notebooks can run within different software environments which are offered by a number of available **Jupyter kernels**.
|
|
|
|
E.g. in this test installation we provide two environments targeted at data science
|
|
* **tensorflow-1.13.1_py37**: contains Tensorflow, Keras, scikit-learn, Pandas, numpy, dask, and dependencies. Stable
|
|
* **talos_py36**: also contains the Talos package. This
|
|
environment is experimental and subject to updates and changes.
|
|
|
|
When you create a new notebook you will be asked to specify which kernel you want to use. It is also possible to switch the kernel of a running notebook, but you will lose the state of the current kernel, so you will have to recalculate the notebook cells with this new kernel.
|
|
|
|
These environments are also available for standard work in a shell session. You can activate an environment in a normal merlin terminal session by using the `module` (q.v. [using Pmodules](using-modules.html)) command to load anaconda python, and from there using the `conda` command to switch to the desired environment
|
|
|
|
```
|
|
module use unstable
|
|
module load anaconda/2019.07
|
|
conda activate tensorflow-1.13.1_py36
|
|
```
|
|
|
|
When the `anaconda` module has been loaded, you can list the available environments by executing
|
|
|
|
```
|
|
conda info -e
|
|
```
|
|
|
|
You can get more info on the use of the `conda` package management tool at its official [https://conda.io/projects/conda/en/latest/commands.html](documentation site).
|
|
|
|
## Using your own custom made environments with jupyterhub
|
|
Python environments can take up a lot of space due to the many dependencies that will be installed. You should always install your extra environments to the data area belonging to your account, e.g. `/data/user/${YOUR-USERNAME}/conda-envs`
|
|
|
|
In order for jupyterhub (and jupyter in general) to recognize the provided environment as a valid kernel, make sure that you include the `nb_conda_kernels` package in your environment. This package provides the necessary activation and the dependencies.
|
|
|
|
Example:
|
|
```
|
|
conda create -c conda-forge -p /data/user/${USER}/conda-envs/my-test-env python=3.7 nb_conda_kernels
|
|
|
|
```
|
|
|
|
After this, your new kernel will be visible as `my-test-env` inside of your jupyterhub session.
|
|
|
|
|
|
|
|
## Troubleshooting
|
|
|
|
In case of problems or request, please either submit a **[PSI Service Now](https://psi.service-now.com/psisp)** incident containing *"Merlin Jupyterhub"* as part of the subject, or contact us by mail through <merlin-admins@lists.psi.ch>.
|
|
|