HPC Documentation
Generative AI Assistant Tool on Minerva
OverviewThe generative AI Assistant tool on Minerva (https://assistant.hpc.mssm.edu/) is an advanced AI-powered platform designed to support clinicians, researchers, data scientists, and students in their daily work. It enables users to interact with cutting-edge Large Language Models (LLMs) built on leading open-source technologies.
The tool is deployed on the Minerva High Performance Computing (HPC) infrastructure. Minerva provides powerful computing resources, ensuring fast and efficient performance for a wide range of research and clinical applications.
- Backend: Powered by an Ollama instance, currently running on two NVIDIA H100 GPUs (80 GB memory each).
- Frontend: Developed using Open WebUI, providing users with a seamless and intuitive experience.
1. Key Features
- Fast Inference: Accelerated by Minerva’s high-performance GPU infrastructure.
- Retrieval-Augmented Generation (RAG): Enables context-aware, document-grounded answers.
- Multi-User Support: Concurrent access with dedicated GPU session management.
- Web Interface: Intuitive user interactions via Open WebUI.
- Scalable Backend: Dynamic GPU expansion based on user demand.
- Multimodal Support: Handles both natural language and code tasks.
- API Access: Programmatic integration with external tools and pipelines.
- Secure & Private: Fully internal to Mount Sinai with no external data sharing.
2. Supported Models
The table below details all Large Language Models available on the Assistant Tool.
| Model | Parameters | Best For | More Information |
|---|---|---|---|
gemma3:4b |
4B | Lightweight multimodal model for fast text and image tasks | Google Gemma 3 |
gemma3:12b-it-fp16 |
12B | Full-precision Gemma 3 model for high-quality text and vision tasks | Google Gemma 3 |
gemma3:12b-it-q8_0 |
12B | High-precision 8-bit quantized Gemma 3 model for multimodal reasoning | Google Gemma 3 |
gemma3:12b-it-q4_K_M |
12B | Balanced 4-bit quantized Gemma 3 12B for resource-efficient performance | Google Gemma 3 |
gemma3:12b-it-qat |
12B | Quantization-Aware Trained 12B model balancing low memory and high quality | Google Gemma 3 |
gemma3:27b-it-fp16 |
27B | Full-precision high-capacity model for complex multimodal reasoning | Google Gemma 3 |
gemma3:27b-it-q8_0 |
27B | 8-bit quantized Gemma 3 27B for precise instruction following & vision tasks | Google Gemma 3 |
gemma3:27b-it-q4_K_M |
27B | 4-bit quantized Gemma 3 27B offering strong reasoning with low VRAM footprint | Google Gemma 3 |
gemma3:27b-it-qat |
27B | Quantization-Aware Trained 27B model delivering near-BF16 precision | Google Gemma 3 |
gemma4:26b |
26B | High-performance Gemma 4 model for deep reasoning and analytical tasks | Google Gemma 4 |
gemma4:31b |
31B | Next-gen Gemma 4 model for advanced multimodal and complex logic workflows | Google Gemma 4 |
gemma4:latest |
31B | Latest flagship Gemma 4 release | Google Gemma 4 |
llama3.1:8b-instruct-fp16 |
8B | Full-precision Llama 3.1 8B instruction-tuned model | Meta Llama 3.1 |
llama3.1:8b-instruct-q8_0 |
8B | 8-bit quantized Llama 3.1 8B for fast, high-quality instruction following | Meta Llama 3.1 |
llama3.1:8b-instruct-q4_K_M |
8B | Balanced 4-bit quantized Llama 3.1 8B for lightweight tasks | Meta Llama 3.1 |
llama3.3:70b |
70B | Best-in-class reasoning, handles complex queries, lowest hallucination rate | Meta Llama 3.3 |
llama3.3:70b-instruct-fp16 |
70B | Full-precision Llama 3.3 70B for maximum accuracy in reasoning and coding | Meta Llama 3.3 |
llama3.3:70b-instruct-q8_0 |
70B | 8-bit quantized Llama 3.3 70B balancing high reasoning accuracy and speed | Meta Llama 3.3 |
llama3.3:70b-instruct-q4_K_M |
70B | 4-bit quantized Llama 3.3 70B for efficient execution of complex queries | Meta Llama 3.3 |
llama4:17b-scout-16e-instruct-fp16 |
109B (17B Active) | Full-precision Llama 4 MoE (16 experts) model for visual reasoning & chat | Meta Llama 4 |
llama4:17b-scout-16e-instruct-q8_0 |
109B (17B Active) | 8-bit quantized Llama 4 MoE model for high-precision multimodal interactions | Meta Llama 4 |
llama4:17b-scout-16e-instruct-q4_K_M |
109B (17B Active) | 4-bit quantized Llama 4 MoE for efficient multimodal and text inference | Meta Llama 4 |
llama4:latest |
109B | Flagship Llama 4 model for large-scale language tasks & general AI | Meta Llama 4 |
gpt-oss:20b |
20B | Open-weight model optimized for fast reasoning and function calling | OpenAI GPT-OSS |
gpt-oss:120b |
120B | Large open-weight model for enterprise-grade reasoning and agent workflows | OpenAI GPT-OSS |
llava:7b |
7B | Multimodal vision-language model (image + text) | LLaVA GitHub |
medgemma1.5:latest |
12B | Domain-specific clinical reasoning and medical text understanding | MedGemma AI |
nemotron-3-nano:30b |
30B | GPU-optimized architecture for fast enterprise language generation | Nvidia Nemotron |
phi4:latest |
14B | High-efficiency small model with strong reasoning, math, and coding skills | Microsoft Phi-4 |
tinyllama:latest |
1.1B | Lightweight, fast inference with low resource requirements | TinyLlama GitHub |
Note: LLM models available in the Assistant tool may make mistakes. Always review outputs carefully before using them.
3. Capabilities Overview
Using the Assistant Tool, users can perform tasks such as:
- Asking scientific questions
- Summarizing and chatting with research papers
- Getting explanations of medical terms
- Obtaining programming assistance
- Translating text between languages
- Comparing multiple models simultaneously
- Accessing functionality via API endpoints
4. Accessing the Assistant Tool
- Open any modern web browser (Chrome, Firefox, Safari, or Edge).
- Navigate to: https://assistant.hpc.mssm.edu
- On the login page, click the “Continue with Microsoft” button.
- Authenticate using your Mount Sinai credentials.
5. Interface Guide
1. Left Navigation Sidebar
- Open WebUI Header & Sidebar Toggle (
[|]): Expand or collapse the sidebar menu. - New Chat: Click to clear the current workspace and start a fresh session.
- Search: Search through your previous prompt history and saved chats.
- Notes: Access and organize saved workspace notes.
- Folders & Chats: Organize your active conversations into dedicated folders or access saved individual chat sessions.
2. Main Workspace & Chat Panel
- Model Indicator: Displays the active model currently selected for the conversation (e.g.,
OI llama4:latest). - Prompt Input Box:
- Add Files (
+): Upload documents (PDFs, text files) or images to interact with using Retrieval-Augmented Generation (RAG) or multimodal vision tasks. - Tools/Plugins (
❖): Access integrations, web tools, or advanced interpreter tools. - Model Switcher Dropdown: Click the dropdown next to the model name inside the prompt box (e.g.,
llama4:latest ˅) to switch between supported models running on Minerva HPC. - Voice Input (
🎙): Dictate your prompt using microphone input. - Voice Response Mode (
|||): Toggle interactive audio mode.
- Add Files (
- Suggested Starters: Quick-click suggestions to quickly test model capabilities (e.g., study help, brainstorming, or productivity tips).
3. Top Control Bar
- Controls Icon (
⚙/ Controls): Adjust session-specific model parameters such as temperature, system prompt, or context length. - User Profile & Settings: Access account settings, documentation, keyboard shortcuts, or sign out.
6. Use Cases & Examples
Use Case 1: Asking Scientific Questions
Ask target scientific questions directly via the prompt bar. You can follow up with suggested prompts or type custom questions to explore topics interactively.
Example Prompt: “How does CRISPR-Cas9 differ from traditional gene-editing techniques?”
Example Output Focus: Summarizes aspects like precision/efficiency, ease of use, specificity, versatility across organisms, and reduced off-target effects compared to TALENs or ZFNs.
Use Case 2: Summarizing & Chatting with Research Papers (RAG)
Paper Summarization — Upload documents (e.g., PDFs) by clicking the + icon in the input panel and selecting Upload Files.
Example Prompt: “Provide a 10-point summary of the attached research paper.”
Example Output: Extracts key methodology, framework details (e.g., federated learning across multiple hospitals), evaluation metrics (AUROC values), guidelines followed (TRIPOD), and license info.
Conversational Document Interaction — Ask follow-up questions specific to the uploaded file’s contents.
Example Follow-up Prompt: “How does the federated model perform compared to the pooled and local models in predicting Acute Kidney Injury (AKI)?”
Example Output: RAG retrieves contextual details from the paper, explaining that the federated model outperforms local models (especially in smaller datasets) and performs comparably to the pooled model.
Use Case 3: Explanation of Medical Terms
Translates complex clinical terminology into clear, accessible descriptions.
Example Prompt: “What does ‘acute kidney injury’ mean?”
Example Output: Defines AKI as a sudden loss of kidney function over hours or days, provides intuitive analogies (e.g., a coffee filter), lists potential causes (dehydration, NSAIDs/antibiotics, sepsis) and common symptoms (fatigue, swelling, altered urination patterns).
Use Case 4: Programming Assistance
Get help writing, debugging, or understanding code in multiple languages.
Example Prompt: “I want to perform a t-test in Python. Give me an example code.”
import numpy as np
from scipy import stats
# Sample data
np.random.seed(0)
sample1 = np.random.normal(0, 1, 100)
sample2 = np.random.normal(0.5, 1, 100)
# Perform two-sample t-test
t_stat, p_val = stats.ttest_ind(sample1, sample2)
print(f"T-Statistic: {t_stat}")
print(f"P-Value: {p_val}")
# Interpret the results
alpha = 0.05
if p_val < alpha:
print("Reject the null hypothesis. The means are likely different.")
else:
print("Fail to reject the null hypothesis. The means are likely the same.")
Use Case 5: Translating Between Languages
Translate clinical notes, queries, or descriptions into various target languages.
Example Prompt: “Translate the following into Spanish: ‘How are you feeling today? Checking in on physical symptoms like pain, fatigue, or discomfort can help catch health issues early and guide appropriate care.'”
Generated Response: “¿Cómo te sientes hoy? Revisar los síntomas físicos como dolor, fatiga o molestias puede ayudar a detectar problemas de salud temprano y guiar el cuidado adecuado.”
Use Case 6: Comparing Multiple Models Side-by-Side
Evaluate accuracy, response depth, and clarity across different model sizes concurrently.
How to enable: Click the + icon next to the active model name in the chat panel to add additional models (e.g., compare llama4:latest, llama3.3:70b, and tinyllama:latest).
Example Query: “What are the main causes of cardiovascular disease?”
Result: Displays side-by-side columns with each model’s response generated simultaneously from a single prompt.
Use Case 7: API Access & Integration
Integrate the Assistant Tool directly into custom Python scripts, data pipelines, or applications.
- Obtain an API key by navigating to Settings > Account in the web interface.
- Use the endpoint URL:
https://assistant.hpc.mssm.edu/api/chat/completions
Python API Example:
import requests
token = "your_api_token_here"
url = 'https://assistant.hpc.mssm.edu/api/chat/completions'
headers = {
'Authorization': f'Bearer {token}',
'Content-Type': 'application/json'
}
data = {
"model": "tinyllama:latest",
"messages": [
{
"role": "user",
"content": "What are the main causes of cardiovascular disease?"
}
]
}
response = requests.post(url, headers=headers, json=data)
result = response.json()
if 'choices' in result and result['choices']:
print("Assistant response:", result['choices'][0]['message']['content'])
else:
print("No assistant response found.")
