Scientific Computing and Data / High Performance Computing / Documentation / RStudio Web
Documentation
Software and Packages
- Software and Applications
- Modules
- Compiling
- Schrodinger Suite
- MATLAB, Simulink and MATLAB Distributed Compute Server
- Singularity
- Jupyter Notebook
Queues and Resources
- LSF Queues And Policies
- GPGPU
- GPU Etiquette
- Access TSM with GUI
- Access TSM with Command Line
- Checkpoint Restart
- Disaster Recovery Plan
Job Submission
Directories
Rstudio
- Rstudio Web
- Rstudio Connect Server
Services
Using on-the-fly RStudio/Jupyter Web on Minerva HPC
We have developed simple tool scripts for users to launch RStudio server/Jupyter Notebook with interactive web interface inside the LSF job on the terminal with one command.
These scripts are located in the login nodes under
/usr/local/bin/{minerva-rstudio-web.sh, minerva-rstudio-web-r4.sh, minerva-jupyter-web.sh, minerva-jupyter-r-web.sh, minerva-jupyter-module-web.sh}
The
minerva-jupyter-web.sh
will launch a simple jupyter notebook session with GPU support. The
minerva-jupyter-r-web.sh
supports both Jupyter notebook Python 3 and R3 kernels. The
minerva-jupyter-module-web.sh
will use the installed modules to start Jupyter sessions. The
minerva-rstudio-web.sh
will start RStudio web with R3 and
minerva-rstudio-web-r4.sh
will start RStudio with R4 kernel.
Usage:
For example, to start an RStudio web session with R, on the login nodes, run commands minerva-rstudio-web-r4.sh
(using R 4.0.3) or minerva-rstudio-web.sh
(R 3.6.2) with default resource configuration and URL to access it. Please see the -h
option for help messages containing resource requests and installing packages.
Example:
$ sh minerva-rstudio-web-r4.sh [INFO] Image not specified, check if previously used [INFO] Found previously used image /hpc/users/guow03/minerva_jobs/rstudio_jobs/singularity-rstudio.simg. Using it. [INFO] Project is not specified, or is acc_null, using 1st avail project. [INFO] Project to use is acc_hpcstaff [INFO] Parameters used are: [INFO] -n 4 [INFO] -M 3000 [INFO] -W 6:00 [INFO] -P acc_hpcstaff [INFO] -J rstudio [INFO] -q premium [INFO] -R null [INFO] -i /hpc/users/guow03/minerva_jobs/rstudio_jobs/ singularity-rstudio.simg [INFO] Submitting rstudio job... Job <25962439> is submitted to queue . [INFO] See below for web access when job starts. Job <25962439> : Not yet started. [INFO] Job is pending Job <25962439> : Not yet started. [INFO] Job is pending Job <25962439> : Not yet started. [INFO] Job is pending Job <25962439> : Not yet started. [INFO] Job is pending [INFO] Job is running, wait for link [INFO] Job is running, wait for link [INFO] Job is running, wait for link << output from stdout >> Using local available port 8787 Using password in /hpc/users/guow03/minerva_jobs/rstudio_jobs/ .rstudio_onthefly_password RStudio started in the singularity container with PID 306586. Making sure it is alive Checking 3, next check in 5 seconds. PID TTY TIME CMD 306586 ? 00:00:00 singularity Checking 2, next check in 5 seconds. PID TTY TIME CMD 306586 ? 00:00:00 starter-suid Checking 1, next check in 5 seconds. PID TTY TIME CMD 306586 ? 00:00:00 starter-suid SSH port forwarding to 10.95.46.103 with PID 306725. Rstudio is started on compute node lc03e29, port 8787 Access the RStudio Web using your web browser: http://10.95.46.103:52439 << output from stderr >>
The RStudio web interface will be available at URL http://10.95.46.103:52439. Use the browser on your laptop to access it. Here 10.95.46.103 is the login node IP on the campus network and 52439 is the port. 2439 is the last 4 digit of the job 25962439 running the instance.
For Jupyter sessions, you will see links blinking with the url with token (no password), like the following.
<pre> [INFO] Copy the following link in your browser for the jupyter notebook web access. [INFO] http://10.95.46.103:42206/?token=f8e4e879f6ec495ed706f5269a328b7270c956c167534499 </pre>
Copy the above url and paste it in your browser to access the Jupyter Notebook web session.
What happens behind the scene? This tool wraps the following tasks in one command.
- downloads a custom built Singularity container image of RStudio in your home directory
- prompts and creates a password for the RStudio interface,
- writes and submits an LSF job script to launch the RStudio within the image,
- provides the URL link to access the instance
Currently, the minerva-rstudio-web-r4.sh script is the preferred version as it is built with common libraries including "DBI", "odbc", "shiny", "devtools", "ggplot2", "tidyverse", "tidymodels", "car", "dplyr", "tidyr"
. If you want to switch the scripts to use, move or rename the downloaded image singularity-rstudio.simg in $HOME/minerva_jobs/rstudio_jobs.
See the Package Installations section in the help message for packages are not installed.
Common problems:
1. When a user runs an rstudio session on a compute node, there will be two lock directories created in the /tmp directory: /tmp/rstudio-session and /tmp/rstudio-server. These directories blocks other users to run more sessions since the new comer will not have permission to overwrite the directories. The script will error out in the output. Workaround is to run the script again till the job is dispatched to a node with no existing session.
2. Plotting in rstudio errors due to X11 display not working when using the v4 version. This problems has been fixed with updated image. Please remove your old image and run the script.