OverviewThe generative AI Assistant tool on Minerva (https://assistant.hpc.mssm.edu/) is an advanced AI-powered platform designed to support clinicians, researchers, data scientists, and students in their daily work. It enables users to interact with cutting-edge Large Language Models (LLMs) built on leading open-source technologies.

The tool is deployed on the Minerva High Performance Computing (HPC) infrastructure. Minerva provides powerful computing resources, ensuring fast and efficient performance for a wide range of research and clinical applications.

  • Backend: Powered by an Ollama instance, currently running on two NVIDIA H100 GPUs (80 GB memory each).
  • Frontend: Developed using Open WebUI, providing users with a seamless and intuitive experience.
1. Key Features
  • Fast Inference: Accelerated by Minerva’s high-performance GPU infrastructure.
  • Retrieval-Augmented Generation (RAG): Enables context-aware, document-grounded answers.
  • Multi-User Support: Concurrent access with dedicated GPU session management.
  • Web Interface: Intuitive user interactions via Open WebUI.
  • Scalable Backend: Dynamic GPU expansion based on user demand.
  • Multimodal Support: Handles both natural language and code tasks.
  • API Access: Programmatic integration with external tools and pipelines.
  • Secure & Private: Fully internal to Mount Sinai with no external data sharing.
2. Supported Models

The table below details all Large Language Models available on the Assistant Tool.

Model Parameters Best For More Information
gemma3:4b 4B Lightweight multimodal model for fast text and image tasks Google Gemma 3
gemma3:12b-it-fp16 12B Full-precision Gemma 3 model for high-quality text and vision tasks Google Gemma 3
gemma3:12b-it-q8_0 12B High-precision 8-bit quantized Gemma 3 model for multimodal reasoning Google Gemma 3
gemma3:12b-it-q4_K_M 12B Balanced 4-bit quantized Gemma 3 12B for resource-efficient performance Google Gemma 3
gemma3:12b-it-qat 12B Quantization-Aware Trained 12B model balancing low memory and high quality Google Gemma 3
gemma3:27b-it-fp16 27B Full-precision high-capacity model for complex multimodal reasoning Google Gemma 3
gemma3:27b-it-q8_0 27B 8-bit quantized Gemma 3 27B for precise instruction following & vision tasks Google Gemma 3
gemma3:27b-it-q4_K_M 27B 4-bit quantized Gemma 3 27B offering strong reasoning with low VRAM footprint Google Gemma 3
gemma3:27b-it-qat 27B Quantization-Aware Trained 27B model delivering near-BF16 precision Google Gemma 3
gemma4:26b 26B High-performance Gemma 4 model for deep reasoning and analytical tasks Google Gemma 4
gemma4:31b 31B Next-gen Gemma 4 model for advanced multimodal and complex logic workflows Google Gemma 4
gemma4:latest 31B Latest flagship Gemma 4 release Google Gemma 4
llama3.1:8b-instruct-fp16 8B Full-precision Llama 3.1 8B instruction-tuned model Meta Llama 3.1
llama3.1:8b-instruct-q8_0 8B 8-bit quantized Llama 3.1 8B for fast, high-quality instruction following Meta Llama 3.1
llama3.1:8b-instruct-q4_K_M 8B Balanced 4-bit quantized Llama 3.1 8B for lightweight tasks Meta Llama 3.1
llama3.3:70b 70B Best-in-class reasoning, handles complex queries, lowest hallucination rate Meta Llama 3.3
llama3.3:70b-instruct-fp16 70B Full-precision Llama 3.3 70B for maximum accuracy in reasoning and coding Meta Llama 3.3
llama3.3:70b-instruct-q8_0 70B 8-bit quantized Llama 3.3 70B balancing high reasoning accuracy and speed Meta Llama 3.3
llama3.3:70b-instruct-q4_K_M 70B 4-bit quantized Llama 3.3 70B for efficient execution of complex queries Meta Llama 3.3
llama4:17b-scout-16e-instruct-fp16 109B (17B Active) Full-precision Llama 4 MoE (16 experts) model for visual reasoning & chat Meta Llama 4
llama4:17b-scout-16e-instruct-q8_0 109B (17B Active) 8-bit quantized Llama 4 MoE model for high-precision multimodal interactions Meta Llama 4
llama4:17b-scout-16e-instruct-q4_K_M 109B (17B Active) 4-bit quantized Llama 4 MoE for efficient multimodal and text inference Meta Llama 4
llama4:latest 109B Flagship Llama 4 model for large-scale language tasks & general AI Meta Llama 4
gpt-oss:20b 20B Open-weight model optimized for fast reasoning and function calling OpenAI GPT-OSS
gpt-oss:120b 120B Large open-weight model for enterprise-grade reasoning and agent workflows OpenAI GPT-OSS
llava:7b 7B Multimodal vision-language model (image + text) LLaVA GitHub
medgemma1.5:latest 12B Domain-specific clinical reasoning and medical text understanding MedGemma AI
nemotron-3-nano:30b 30B GPU-optimized architecture for fast enterprise language generation Nvidia Nemotron
phi4:latest 14B High-efficiency small model with strong reasoning, math, and coding skills Microsoft Phi-4
tinyllama:latest 1.1B Lightweight, fast inference with low resource requirements TinyLlama GitHub

Note: LLM models available in the Assistant tool may make mistakes. Always review outputs carefully before using them.

3. Capabilities Overview

Using the Assistant Tool, users can perform tasks such as:

  1. Asking scientific questions
  2. Summarizing and chatting with research papers
  3. Getting explanations of medical terms
  4. Obtaining programming assistance
  5. Translating text between languages
  6. Comparing multiple models simultaneously
  7. Accessing functionality via API endpoints
4. Accessing the Assistant Tool
  1. Open any modern web browser (Chrome, Firefox, Safari, or Edge).
  2. Navigate to: https://assistant.hpc.mssm.edu
  3. On the login page, click the “Continue with Microsoft” button.
  4. Authenticate using your Mount Sinai credentials.
5. Interface Guide
1. Left Navigation Sidebar
  • Open WebUI Header & Sidebar Toggle ([|]): Expand or collapse the sidebar menu.
  • New Chat: Click to clear the current workspace and start a fresh session.
  • Search: Search through your previous prompt history and saved chats.
  • Notes: Access and organize saved workspace notes.
  • Folders & Chats: Organize your active conversations into dedicated folders or access saved individual chat sessions.
2. Main Workspace & Chat Panel
  • Model Indicator: Displays the active model currently selected for the conversation (e.g., OI llama4:latest).
  • Prompt Input Box:
    • Add Files (+): Upload documents (PDFs, text files) or images to interact with using Retrieval-Augmented Generation (RAG) or multimodal vision tasks.
    • Tools/Plugins (): Access integrations, web tools, or advanced interpreter tools.
    • Model Switcher Dropdown: Click the dropdown next to the model name inside the prompt box (e.g., llama4:latest ˅) to switch between supported models running on Minerva HPC.
    • Voice Input (🎙): Dictate your prompt using microphone input.
    • Voice Response Mode (|||): Toggle interactive audio mode.
  • Suggested Starters: Quick-click suggestions to quickly test model capabilities (e.g., study help, brainstorming, or productivity tips).
3. Top Control Bar
  • Controls Icon ( / Controls): Adjust session-specific model parameters such as temperature, system prompt, or context length.
  • User Profile & Settings: Access account settings, documentation, keyboard shortcuts, or sign out.
6. Use Cases & Examples
Use Case 1: Asking Scientific Questions

Ask target scientific questions directly via the prompt bar. You can follow up with suggested prompts or type custom questions to explore topics interactively.

Example Prompt: “How does CRISPR-Cas9 differ from traditional gene-editing techniques?”

Example Output Focus: Summarizes aspects like precision/efficiency, ease of use, specificity, versatility across organisms, and reduced off-target effects compared to TALENs or ZFNs.

Use Case 2: Summarizing & Chatting with Research Papers (RAG)

Paper Summarization — Upload documents (e.g., PDFs) by clicking the + icon in the input panel and selecting Upload Files.

Example Prompt: “Provide a 10-point summary of the attached research paper.”

Example Output: Extracts key methodology, framework details (e.g., federated learning across multiple hospitals), evaluation metrics (AUROC values), guidelines followed (TRIPOD), and license info.

Conversational Document Interaction — Ask follow-up questions specific to the uploaded file’s contents.

Example Follow-up Prompt: “How does the federated model perform compared to the pooled and local models in predicting Acute Kidney Injury (AKI)?”

Example Output: RAG retrieves contextual details from the paper, explaining that the federated model outperforms local models (especially in smaller datasets) and performs comparably to the pooled model.

Use Case 3: Explanation of Medical Terms

Translates complex clinical terminology into clear, accessible descriptions.

Example Prompt: “What does ‘acute kidney injury’ mean?”

Example Output: Defines AKI as a sudden loss of kidney function over hours or days, provides intuitive analogies (e.g., a coffee filter), lists potential causes (dehydration, NSAIDs/antibiotics, sepsis) and common symptoms (fatigue, swelling, altered urination patterns).

Use Case 4: Programming Assistance

Get help writing, debugging, or understanding code in multiple languages.

Example Prompt: “I want to perform a t-test in Python. Give me an example code.”

Python
import numpy as np
from scipy import stats

# Sample data
np.random.seed(0)
sample1 = np.random.normal(0, 1, 100)
sample2 = np.random.normal(0.5, 1, 100)

# Perform two-sample t-test
t_stat, p_val = stats.ttest_ind(sample1, sample2)

print(f"T-Statistic: {t_stat}")
print(f"P-Value: {p_val}")

# Interpret the results
alpha = 0.05
if p_val < alpha:
    print("Reject the null hypothesis. The means are likely different.")
else:
    print("Fail to reject the null hypothesis. The means are likely the same.")
Use Case 5: Translating Between Languages

Translate clinical notes, queries, or descriptions into various target languages.

Example Prompt: “Translate the following into Spanish: ‘How are you feeling today? Checking in on physical symptoms like pain, fatigue, or discomfort can help catch health issues early and guide appropriate care.'”

Generated Response: “¿Cómo te sientes hoy? Revisar los síntomas físicos como dolor, fatiga o molestias puede ayudar a detectar problemas de salud temprano y guiar el cuidado adecuado.”

Use Case 6: Comparing Multiple Models Side-by-Side

Evaluate accuracy, response depth, and clarity across different model sizes concurrently.

How to enable: Click the + icon next to the active model name in the chat panel to add additional models (e.g., compare llama4:latest, llama3.3:70b, and tinyllama:latest).

Example Query: “What are the main causes of cardiovascular disease?”

Result: Displays side-by-side columns with each model’s response generated simultaneously from a single prompt.

Use Case 7: API Access & Integration

Integrate the Assistant Tool directly into custom Python scripts, data pipelines, or applications.

  1. Obtain an API key by navigating to Settings > Account in the web interface.
  2. Use the endpoint URL: https://assistant.hpc.mssm.edu/api/chat/completions

Python API Example:

Python
import requests

token = "your_api_token_here"
url = 'https://assistant.hpc.mssm.edu/api/chat/completions'

headers = {
    'Authorization': f'Bearer {token}',
    'Content-Type': 'application/json'
}

data = {
    "model": "tinyllama:latest",
    "messages": [
        {
            "role": "user",
            "content": "What are the main causes of cardiovascular disease?"
        }
    ]
}

response = requests.post(url, headers=headers, json=data)
result = response.json()

if 'choices' in result and result['choices']:
    print("Assistant response:", result['choices'][0]['message']['content'])
else:
    print("No assistant response found.")