Jupyter Notebook

 

Jupyter notebooks (formerly iPython notebooks) is an interactive computational environment, in which you can code interactively in Python from a web browser with support for equation editing, code execution, rich text, mathematics, inline plotting, rich media etc.

On the Minerva cluster, you can access the jupyter notebook running on compute nodes via port forwarding  (details refer to here). You can run step-by-step commands to start a Jupyter notebook running from Minerva compute nodes and access it at your local web browser. We also provided in-house wrappers/tools to access the Jupyter notebook via one simple command line such as “minerva-jupyter-module-web.sh” or “minerva-jupyter-web.sh”.

With those tools, Jupyter notebook servers run on the Minerva compute nodes as LSF jobs with dedicated resources. You can request the needed resources for your Jupyter interactive work as you do in other LSF batch jobs. It is recommended that the Jupyter notebook is used only for code development and testing on smaller samples.  For computationally intensive or long running tasks, the bulk computation should be performed in Python scripts submitted as non-interactive batch jobs if possible.

 

Table I summary comparison of the two wrappers

  minerva-jupyter-module-web.sh minerva-jupyter-web.sh
Access modules on Minerva Yes No
Using singularity image No Yes
Support GPU node Yes Yes
Python versions

By default, python/3.7.3;

You can  load other python version and other modules needed for your Jupter Notebook by -mm option

This script uses the python within this Singularity image (shub://ISU-HPC/jupyter)
Others For users who want to access Minerva modules. For users who want an isolated/clean env working with a container image. You need to install/maintain your own python related package. No module system setup

 

 

Option1:minerva-jupyter-module-web.sh

One simple command to get interactive web sessions in a Minerva LSF job (Available on login nodes only). You can check the script at /usr/local/bin/minerva-jupyter-module-web.sh.

Usage:
For example, to start jupyter notebook web session with python/3.7.3, on the login nodes, run commands minerva-jupyter-module-web.sh (using python/3.7.3) with default resource configuration and URL to access it. Please see the --help option for help messages containing resource requests and installing packages.

 

$ minerva-jupyter-module-web.sh
[INFO] Project is not specified, or is acc_null, using 1st avail project.
[INFO] Project to use is acc_psychgen
[INFO] Parameters used are:
[INFO] -n 4
[INFO] -M 3000
[INFO] -W 6:00
[INFO] -P acc_psychgen
[INFO] -J jupyter
[INFO] -q premium
[INFO] -R null
[INFO] -mm null
[INFO] -env null
[INFO] Submitting jupyter job...
Job is submitted to queue .
[INFO] Wait and see below for web access when job starts.
<>
Using local port 8888
Jupyter Notebook is started on compute node lc02g01, port 8888
<>
Currently Loaded Modules:
1) gcc/8.3.0 2) unixODBC/2.3.9 3) python/3.7.3
[I 16:22:11.473 NotebookApp] Serving notebooks from local directory: /hpc/users/gail01
[I 16:22:11.473 NotebookApp] Jupyter Notebook 6.1.4 is running at:
[I 16:22:11.473 NotebookApp] http://lc02g01:8888/?token=f9e6d8dea461b8d6837ff11bc1e3ed32f80e1b282c22b305
[I 16:22:11.473 NotebookApp] or http://127.0.0.1:8888/?token=f9e6d8dea461b8d6837ff11bc1e3ed32f80e1b282c22b305
[I 16:22:11.473 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 16:22:11.477 NotebookApp]
To access the notebook, open this file in a browser:
file:///hpc/users/gail01/.local/share/jupyter/runtime/nbserver-219276-open.html
Or copy and paste one of these URLs:
http://lc02g01:8888/?token=f9e6d8dea461b8d6837ff11bc1e3ed32f80e1b282c22b305
or http://127.0.0.1:8888/?token=f9e6d8dea461b8d6837ff11bc1e3ed32f80e1b282c22b305
[INFO] Token not ready, trying ...

[INFO] Copy the following link in your browser for the jupyter notebook web access.
[INFO] http://10.95.46.103:40541/?token=f9e6d8dea461b8d6837ff11bc1e3ed32f80e1b282c22b305

For Jupyter sessions, after see links blinking with the url with token (no password), like the above

 “

[INFO] Copy the following link in your browser for the jupyter notebook web access.

[INFO] http://10.95.46.103:40541/?token=f9e6d8dea461b8d6837ff11bc1e3ed32f80e1b282c22b305
 “

Copy the url and paste it in your browser to access the Jupyter Notebook web session. Note: You can load Minerva modules needed for your Jupyter Notebook with –mm option.

Please see the --help option for help messages containing resource requests and installing packages.

$ minerva-jupyter-module-web.sh --help
[INFO] This script is to submit a Python Jupyter Notebook web instance inside an
[INFO] LSF job on *one single host* for users.
[INFO] By default, this script uses Jupyter from python/3.7.3
[INFO] You can load other python version and other modules needed for your Jupter Notebook by -mm option
[INFO]
[INFO] 09/01/2021
[INFO] Contact: hpchelp@hpc.mssm.edu
[INFO]
[INFO] minerva-jupyter-module-web.sh -n -M -W
[INFO] -p -J -q
[INFO] --mm
[INFO]
[INFO] -n | --ncpus Number of CPU slots, will be allocated in one host using -R 'span[nhost=1]' default 4 if not specified.
[INFO] -M | --mem Memory in Megabytes per CPU slot, used for resource request, default 3000 MB. -R 'rusage[mem=3000]'
[INFO] -W | --timelimit Wall time for the job, format HH:MM. Default is 6:00/6 hours
[INFO] -P | --project Minerva account for this job, default the first acc_ from your mybalance if not specified
[INFO] -J | --jobname Specify the job name, default jupyter
[INFO] -q | --queue Specify the queue name, default premium
[INFO] -R | --resource Optional resource, eg himem, a100, v100
[INFO] -mm |--module module your want to loaded, seperated by , for multiple modules
[INFO] -env | --myenv anaconda env you want to activate if anaconda is loaded
[INFO] -h | --help Help message
[INFO]
[INFO] Files and directories:
[INFO] /hpc/users/gail01/minerva_jobs/jupyter_jobs The directory where this script generates the job submission scripts.
[INFO]
[INFO] Job output and error files will be saved in the current working directory when you run this script.
[INFO] If job is still running, use bpeek to check the output

 

Behind the scene, the wapper tool just runs a LSF command and the port forwarding as shown below. You can issue those commands step by step with more control by yourself and work with the port forwarding. Here are the details:

# start an interactive session for example
$bsub -P acc_xxx -q interactive -n 2 -R "span[hosts=1]" -R rusage[mem=4000] -W 3:00 -Is /bin/bash

#Then on the allocated nodes lc01c30, start Jupyter Notebook
lc01c30 $ml python
lc01c30$jupyter notebook --no-browser --port=8889

#On your local workstation, forward port XXXX(8889) to YYYY(8888) and listen to it
$ssh -t -t -L localhost:8888:localhost:8889 gail01@minerva.hpc.mssm.edu ssh -X lc01c30 -L localhost:8889:localhost:8889

#Open firefox on local: http://localhost:8888
* note: you can change the portal number 8888/8889 to others

Option 2: minerva-jupyter-web.sh

minerva-jupyter-web.sh is on-the-fly Jupyter Notebook in a Minerva LSF job using the python jupyter notebook within a singularity image. It is a containerized application for workflow reproducibility (for users who want an isolated/clean env working with container image), and related packages are set to be installed in $HOME/.local.

  • By default, it uses the python within this Singularity image (shub://ISU-HPC/jupyter). You can use your own image with the option -i, but this may need a bit modification of the path for python in line 335 and following at /usr/local/bin/minerva-jupyter-web.sh. You can consult hpchelp@hpc.mssm.edu if you have issues with this.
  • There is no module system setup, so you cannot access any central modules maintained on Minerva. You will need to install/maintain your own python related package as below:
    • Open the terminal in web the jupyter web, type pip install packages --user
    • This will be in your home directory $HOME/.local. Please restart the jupyter notebook to pick up the changes

Usage:
For example, to start a jupyter notebook web session with a container image, on the login nodes, run commands minerva-jupyter-web.sh with default resource configuration and URL to access it. Please see the --help option for help messages containing resource requests and installing packages.

$ minerva-jupyter-web.sh

[INFO] Image not specified, check if previously used

[INFO] No previously used image in /hpc/users/gail01/minerva_jobs/jupyter_jobs, pulling the default image shub://nickjer/singularity-jupyter

[INFO] Pulling image to /hpc/users/gail01/minerva_jobs/jupyter_jobs/jupyter_latest.sif

INFO: Downloading shub image

328.0MiB / 328.0MiB [==============================================================================] 100 % 201.5 MiB/s 0s

[INFO] Project is not specified, or is acc_null, using 1st avail project.

[INFO] Project to use is acc_psychgen

[INFO] Parameters used are:

[INFO] -n 4

[INFO] -M 3000

[INFO] -W 6:00

[INFO] -P acc_psychgen

[INFO] -J jupyter

[INFO] -q premium

[INFO] -R null

[INFO] -g 0

[INFO] -i /hpc/users/gail01/minerva_jobs/jupyter_jobs/jupyter_latest.sif

[INFO] Submitting jupyter job...

Job <61550613> is submitted to queue .

[INFO] Wait and see below for web access when job starts.

<< output from stdout >>

Using local port 8888

Jupyter Notebook is started on compute node lc01a11, port 8888

<< output from stderr >>
[I 16:44:57.521 NotebookApp] JupyterLab beta preview extension loaded from /usr/local/lib/python3.6/site-packages/jupyterlab
[I 16:44:57.521 NotebookApp] JupyterLab application directory is /usr/local/share/jupyter/lab
[I 16:44:57.526 NotebookApp] Serving notebooks from local directory: /hpc/users/gail01
[I 16:44:57.526 NotebookApp] 0 active kernels
[I 16:44:57.527 NotebookApp] The Jupyter Notebook is running at:
[I 16:44:57.527 NotebookApp] http://lc01a11:8888/?token=d863e7b88b5ab7344846e182d29bf0ba97a5edc7b9b93de1
[I 16:44:57.527 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 16:44:57.527 NotebookApp]

Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://lc01a11:8888/?token=d863e7b88b5ab7344846e182d29bf0ba97a5edc7b9b93de1&token=d863e7b88b5ab7344846e182d29bf0ba97a5edc7b9b93de1
[INFO] Token not ready, trying ...

[INFO] Copy the following link in your browser for the jupyter notebook web access.
[INFO] http://10.95.46.103:40613/?token=d863e7b88b5ab7344846e182d29bf0ba97a5edc7b9b93de1

 

 

Please see the --help option for help messages containing resource requests and installing packages.

$minerva-jupyter-web.sh --help
[INFO] This script is to submit a Singularity containerized Jupyter Notebook web instance inside an
[INFO] LSF job on *one single host* for users.
[INFO] By default, this script uses this Singularity image (shub://ISU-HPC/jupyter)
[INFO] see https://singularity-hub.org/collections/1069/usage for the this image info.
[INFO]
[INFO] Wei Guo
[INFO] 12/14/2020
[INFO] Contact: hpchelp@hpc.mssm.edu
[INFO]
[INFO] minerva-jupyter-web.sh -n -M -W
[INFO] -p -J -q
[INFO] --image
[INFO]
[INFO] -n | --ncpus Number of CPU slots, will be allocated in one host using -R 'span[nhost=1]' default 4 if not specified.
[INFO] -M | --mem Memory in Megabytes per CPU slot, used for resource request, default 3000 MB. -R 'rusage[mem=3000]'
[INFO] -W | --timelimit Wall time for the job, format HH:MM. Default is 6:00/6 hours
[INFO] -P | --project Minerva account for this job, default the first acc_ from your mybalance if not specified
[INFO] -J | --jobname Specify the job name, default jupyter
[INFO] -q | --queue Specify the queue name, default premium
[INFO] -R | --resource Optional resource, eg himem, a100, v100
[INFO] -g | --ngpus Optional. Specify the number of GPUs to use on a single node, default is 0
[INFO] -i | --image Full path of the image file your specified other than the default. If not specified, this script will pull the default image in your /hpc/users/gail01/minerva_jobs/jupyter_jobs directory.
[INFO] -h | --help Help message
[INFO]
[INFO] Files and directories:
[INFO] /hpc/users/gail01/minerva_jobs/jupyter_jobs The directory where this script generates the job submission scripts.
[INFO] /hpc/users/gail01/minerva_jobs/jupyter_jobs/jupyter_latest.sif The default file of the image.
[INFO]
[INFO] Job output and error files will be saved in the current working directory when you run this script.
[INFO] If job is still running, use bpeek to check the output

 

There is also a tool called  minerva-jupyter-r-web.sh, which supports both Jupyter notebook Python 3 and R3 kernels. Run the --help  option for help messages containing resource requests and installing packages.

What happens behind the scene? This tool wraps the following tasks in one command.

  1. downloads a custom built Singularity container image of Jupyter Notebook in your home directory
  2. writes and submits an LSF job script to launch the Jupyter Notebook  within the image,
  3. provides the URL link to access the instance