HPCE/gitea-pages
HPCE/gitea-pages
0
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Activity

Jupyterhub docs: discuss adding #SBATCH options

Browse Source
This commit is contained in:
Spencer Bliven 2022-07-11 14:06:28 +02:00
parent 9356e3554c
commit 205f174ba7
1 changed files with 35 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
pages/merlin6/04-Jupyterhub/jupyterhub.md
View File

@ -28,7 +28,7 @@ The service is available inside of PSI (or through a VPN connection) at
Slurm batch system. If the cluster is not currently overloaded
and the resources you requested are available, your job will
usually start within 30 seconds.
@ -63,21 +63,50 @@ conda info -e
You can get more info on the use of the `conda` package management tool at its official [https://conda.io/projects/conda/en/latest/commands.html](documentation site).
## Using your own custom made environments with jupyterhub
Python environments can take up a lot of space due to the many dependencies that will be installed. You should always install your extra environments to the data area belonging to your account, e.g. `/data/user/${YOUR-USERNAME}/conda-envs`
Python environments can take up a lot of space due to the many dependencies that will be installed. You should always install your extra environments to the data area belonging to your account, e.g. `/data/user/${YOUR-USERNAME}/conda-envs`
In order for jupyterhub (and jupyter in general) to recognize the provided environment as a valid kernel, make sure that you include the `nb_conda_kernels` package in your environment. This package provides the necessary activation and the dependencies.
Example:
```
conda create -c conda-forge -p /data/user/${USER}/conda-envs/my-test-env python=3.7 nb_conda_kernels
```
After this, your new kernel will be visible as `my-test-env` inside of your jupyterhub session.
## Requesting additional resources
The **Spawner Options** page covers the most common options. These are used to
create a submission script for the jupyterhub job and submit it to the slurm
queue. Additional customization can be implemented using the *'Optional user
defined line to be added to the batch launcher script'* option. This line is
added to the submission script at the end of other `#SBATCH` lines. Parameters can
be passed to SLURM by starting the line with `#SBATCH`, like in [Running Slurm
Scripts](/merlin6/running-jobs.html). Some ideas:
**Request additional memory**
```
#SBATCH --mem=100G
```
**Request multiple GPUs** (gpu partition only)
```
#SBATCH --gpus=2
```
**Log additional information**
```
hostname; date; echo $USER
```
Output is found in `~/jupyterhub_batchspawner_<jobid>.log`.
## Contact
In case of problems or requests, please either submit a **[PSI Service Now](https://psi.service-now.com/psisp)** incident containing *"Merlin Jupyterhub"* as part of the subject, or contact us by mail through <merlin-admins@lists.psi.ch>.
In case of problems or requests, please either submit a **[PSI Service
Now](https://psi.service-now.com/psisp)** incident containing *"Merlin
Jupyterhub"* as part of the subject, or contact us by mail through
<merlin-admins@lists.psi.ch>.