HPCE/gitea-pages
HPCE/gitea-pages
0
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Activity

Jupyterhub docs: discuss adding #SBATCH options

Browse Source
This commit is contained in:
Spencer Bliven 2022-07-11 14:06:28 +02:00
parent 9356e3554c
commit 205f174ba7
1 changed files with 35 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
pages/merlin6/04-Jupyterhub/jupyterhub.md
View File

@ -28,7 +28,7 @@ The service is available inside of PSI (or through a VPN connection) at
Slurm batch system. If the cluster is not currently overloaded Slurm batch system. If the cluster is not currently overloaded
and the resources you requested are available, your job will and the resources you requested are available, your job will
usually start within 30 seconds. usually start within 30 seconds.
@ -63,21 +63,50 @@ conda info -e
You can get more info on the use of the `conda` package management tool at its official [https://conda.io/projects/conda/en/latest/commands.html](documentation site). You can get more info on the use of the `conda` package management tool at its official [https://conda.io/projects/conda/en/latest/commands.html](documentation site).
## Using your own custom made environments with jupyterhub ## Using your own custom made environments with jupyterhub
Python environments can take up a lot of space due to the many dependencies that will be installed. You should always install your extra environments to the data area belonging to your account, e.g. `/data/user/${YOUR-USERNAME}/conda-envs` Python environments can take up a lot of space due to the many dependencies that will be installed. You should always install your extra environments to the data area belonging to your account, e.g. `/data/user/${YOUR-USERNAME}/conda-envs`
In order for jupyterhub (and jupyter in general) to recognize the provided environment as a valid kernel, make sure that you include the `nb_conda_kernels` package in your environment. This package provides the necessary activation and the dependencies. In order for jupyterhub (and jupyter in general) to recognize the provided environment as a valid kernel, make sure that you include the `nb_conda_kernels` package in your environment. This package provides the necessary activation and the dependencies.
Example: Example:
``` ```
conda create -c conda-forge -p /data/user/${USER}/conda-envs/my-test-env python=3.7 nb_conda_kernels conda create -c conda-forge -p /data/user/${USER}/conda-envs/my-test-env python=3.7 nb_conda_kernels
``` ```
After this, your new kernel will be visible as `my-test-env` inside of your jupyterhub session. After this, your new kernel will be visible as `my-test-env` inside of your jupyterhub session.
## Requesting additional resources
The **Spawner Options** page covers the most common options. These are used to
create a submission script for the jupyterhub job and submit it to the slurm
queue. Additional customization can be implemented using the *'Optional user
defined line to be added to the batch launcher script'* option. This line is
added to the submission script at the end of other `#SBATCH` lines. Parameters can
be passed to SLURM by starting the line with `#SBATCH`, like in [Running Slurm
Scripts](/merlin6/running-jobs.html). Some ideas:
**Request additional memory**
```
#SBATCH --mem=100G
```
**Request multiple GPUs** (gpu partition only)
```
#SBATCH --gpus=2
```
**Log additional information**
```
hostname; date; echo $USER
```
Output is found in `~/jupyterhub_batchspawner_<jobid>.log`.
## Contact ## Contact
In case of problems or requests, please either submit a **[PSI Service Now](https://psi.service-now.com/psisp)** incident containing *"Merlin Jupyterhub"* as part of the subject, or contact us by mail through <merlin-admins@lists.psi.ch>. In case of problems or requests, please either submit a **[PSI Service
Now](https://psi.service-now.com/psisp)** incident containing *"Merlin
Jupyterhub"* as part of the subject, or contact us by mail through
<merlin-admins@lists.psi.ch>.