Using on-the-fly RStudio Web on Minerva HPC
We have developed a simple tool script for users to launch RStudio server with interactive web interface inside the LSF job on the terminal with one command.
On the login nodes, run commands minerva-rstudio-web-r4.sh (using R 4.0.3) or minerva-rstudio-web.sh (R 3.6.2) to start a session with default resource configuration and URL to access it. Please see the -h option for help messages containing resource requests and installing packages.
On the login nodes, run commands minerva-rstudio-web-r4.sh (using R 4.0.3) or minerva-rstudio-web.sh (R 3.6.2) to start a session with default resource configuration and URL to access it.
Example:
$ sh minerva-rstudio-web-r4.sh [INFO] Image not specified, check if previously used [INFO] Found previously used image /hpc/users/guow03/minerva_jobs/rstudio_jobs/singularity-rstudio.simg. Using it. [INFO] Project is not specified, or is acc_null, using 1st avail project. [INFO] Project to use is acc_hpcstaff [INFO] Parameters used are: [INFO] -n 4 [INFO] -M 3000 [INFO] -W 6:00 [INFO] -P acc_hpcstaff [INFO] -J rstudio [INFO] -q premium [INFO] -R null [INFO] -i /hpc/users/guow03/minerva_jobs/rstudio_jobs/ singularity-rstudio.simg [INFO] Submitting rstudio job... Job <25962439> is submitted to queue <premium>. [INFO] See below for web access when job starts. Job <25962439> : Not yet started. [INFO] Job is pending Job <25962439> : Not yet started. [INFO] Job is pending Job <25962439> : Not yet started. [INFO] Job is pending Job <25962439> : Not yet started. [INFO] Job is pending [INFO] Job is running, wait for link [INFO] Job is running, wait for link [INFO] Job is running, wait for link << output from stdout >> Using local available port 8787 Using password in /hpc/users/guow03/minerva_jobs/rstudio_jobs/ .rstudio_onthefly_password RStudio started in the singularity container with PID 306586. Making sure it is alive Checking 3, next check in 5 seconds. PID TTY TIME CMD 306586 ? 00:00:00 singularity Checking 2, next check in 5 seconds. PID TTY TIME CMD 306586 ? 00:00:00 starter-suid Checking 1, next check in 5 seconds. PID TTY TIME CMD 306586 ? 00:00:00 starter-suid SSH port forwarding to 10.95.46.103 with PID 306725. Rstudio is started on compute node lc03e29, port 8787 Access the RStudio Web using your web browser: http://10.95.46.103:52439 << output from stderr >>
The RStudio web interface will be available at URL http://10.95.46.103:52439. Use the browser on your laptop to access it. Here 10.95.46.103 is the login node IP on the campus network and 52439 is the port. 2439 is the last 4 digit of the job 25962439 running the instance.
What happens behind the scene? This tool wraps the following tasks in one command.
- downloads a custom built Singularity container image of RStudio in your home directory
- prompts and creates a password for the RStudio interface,
- writes and submits an LSF job script to launch the RStudio within the image,
- provides the URL link to access the instance
Currently, the minerva-rstudio-web-r4.sh script is the preferred version as it is built with common libraries including "DBI", "odbc", "shiny", "devtools", "ggplot2", "tidyverse", "tidymodels", "car", "dplyr", "tidyr". If you want to switch the scripts to use, move or rename the downloaded image singularity-rstudio.simg in $HOME/minerva_jobs/rstudio_jobs See the Package Installations section in the help message for packages are not installed.
Common problems:
1. When a user runs an rstudio session on a compute node, there will be two lock directories created in the /tmp directory: /tmp/rstudio-session and /tmp/rstudio-server. These directories blocks other users to run more sessions since the new comer will not have permission to overwrite the directories. The script will error out in the output. Workaround is to run the script again till the job is dispatched to a node with no existing session.
2. Plotting in rstudio errors due to X11 display not working when using the v4 version. This problems has been fixed with updated image. Please remove your old image and run the script.