add merlin6 JB docs
This commit is contained in:
78
docs/merlin6/jupyterhub/jupyterhub-trouble.md
Normal file
78
docs/merlin6/jupyterhub/jupyterhub-trouble.md
Normal file
@@ -0,0 +1,78 @@
|
||||
# Jupyterhub Troubleshooting
|
||||
|
||||
In case of problems or requests, please either submit a **[PSI Service
|
||||
Now](https://psi.service-now.com/psisp)** incident containing *"Merlin
|
||||
Jupyterhub"* as part of the subject, or contact us by mail through
|
||||
<merlin-admins@lists.psi.ch>.
|
||||
|
||||
## General steps for troubleshooting
|
||||
|
||||
### Investigate the Slurm output file
|
||||
|
||||
Your jupyterhub session runs as a normal batch job on the cluster, and each
|
||||
launch will create a slurm output file in your *home* directory named like
|
||||
`jupyterhub_batchspawner_{$JOBID}.log`, where the `$JOBID` part is the slurm job
|
||||
ID of your job. After a failed launch, investigate the contents of that file.
|
||||
An error message will usually be found towards the end of the file, often
|
||||
including a python backtrace.
|
||||
|
||||
### Investigate python environment interferences
|
||||
|
||||
Jupyterhub just runs a jupyter notebook executable as your user inside the
|
||||
batch job. A frequent source of errors consists of a user's local python
|
||||
environment definitions getting mixed up with the environment that jupyter
|
||||
needs to launch.
|
||||
|
||||
- setting PYTHONPATH inside of the ~/.bash_profile or any other startup script
|
||||
- having installed packages to your local user area (e.g. using `pip install
|
||||
--user <some-package>`). Such installation will interfere with the
|
||||
environment offered by the `module` system on our cluster (based on
|
||||
anaconda). You can list such packages by executing `pip list user`.
|
||||
They are usually located in `~/.local/lib/pythonX.Y/...`.
|
||||
|
||||
You can investigate the launching of a notebook interactively, by logging in to
|
||||
Merlin6 and running a jupyter command in the correct environment.
|
||||
|
||||
```bash
|
||||
module use unstable
|
||||
module load anaconda/2019.07
|
||||
conda activate jupyterhub-1.0.0_py36
|
||||
jupyter --paths
|
||||
```
|
||||
|
||||
## Known Problems and workarounds
|
||||
|
||||
### Spawner times out
|
||||
|
||||
If the cluster is very full, it may be difficult to launch a session. We always
|
||||
reserve some slots for interactive Jupyterhub use, but it may be that these
|
||||
slots have been taken or that the resources you requested are currently not
|
||||
available.
|
||||
|
||||
Inside of a Merlin6 terminal shell, you can run the standard commands like
|
||||
`sinfo` and `squeue` to get an overview of how full the cluster is.
|
||||
|
||||
### Your user environment is not among the kernels offered for choice
|
||||
|
||||
Refer to our documentation about [using your own custom made
|
||||
environments with jupyterhub](/merlin6/jupyterhub.html).
|
||||
|
||||
### Cannot save notebook - *xsrf argument missing*
|
||||
|
||||
You cannot save your notebook anymore and you get this error:
|
||||
|
||||
```text
|
||||
'_xsrf' argument missing from POST
|
||||
```
|
||||
|
||||
This issue occurs very seldomly. There exists the following workaround:
|
||||
|
||||
Go to the jupyterhub file browsing window and just open another
|
||||
notebook using the same kernel in another browser window. The issue
|
||||
should then go away. For more information refer to [this github
|
||||
thread](https://github.com/nteract/hydrogen/issues/922#issuecomment-405456346)
|
||||
|
||||
<!-- ## Error HTTP 500 when starting the spawner -->
|
||||
|
||||
<!-- The spawner screen shows after launching an error message like the following: -->
|
||||
<!-- `Internal server error (Spawner failed to start [status=..]. The logs may contain details)` -->
|
||||
Reference in New Issue
Block a user