LLM Module | Scientific Computing and Data

Scientific Computing and Data / High Performance Computing / Documentation / The minerva-llm Module

Overview

The minerva-llm module provides access to centrally managed, locally hosted, and approved AI models for researchers to use in Python and other applications on Minerva. These models are hosted within the Minerva environment, allowing users to perform inference without downloading models from external repositories.

⚠️ Data Security and NIH Guidance

When using AI models with controlled-access human genomic data, researchers are responsible for complying with all applicable institutional and funding agency policies. The NIH Guide Notice, Protecting Human Genomic Data when Developing Generative Artificial Intelligence Tools and Applications, includes:

A reminder that the GDS Policy does not allow users to share controlled-access data or their data derivatives with unauthorized users, including public generative AI tools.
A prohibition against sharing generative AI models and their parameters trained on human genomic data obtained from NIH controlled-access repositories.

NIH intends to release future guidance outlining the responsible use and sharing of generative AI models based on controlled-access data.

Read the Complete NIH Guide Notice →

1 Conda Environment Setup

We recommend using a conda environment for running LLM applications.

1. Load Proxies and Miniforge

Shell

# Load standard proxy
module load proxies/1

# OR on the Restricted Cluster:
module load res_proxies/1

# Load Miniforge
module load miniforge3/26.1.1-3

2. Create and Activate Environment

Shell

# Create environment
conda create -n llm-test python=3.12

# Activate environment
conda activate llm-test

# Prevent Python from loading packages from ~/.local directory
export PYTHONNOUSERSITE=1

3. Install Packages

Shell

conda install pytorch transformers accelerate sentencepiece protobuf wrapt

Note: You only need to create the environment once. For future sessions, simply load Miniforge and activate the environment.

2 Load the Minerva LLM Module

Load the centrally managed module to configure your runtime environment variables automatically:

Shell

module load minerva-llm/1.0

Once loaded, you can verify your environment directories using:

Shell

echo $MINERVA_LLM_ROOT
echo $MINERVA_LLM_MODELS

3 List Available Models

Display all available shared models hosted locally on Minerva:

Shell

minerva-llm-list

Output

Llama-3.2-1B-Instruct
Llama-3.2-3B-Instruct
TinyLlama-1.1B-Chat-v1.0

4 Search for a Model

Search for models using a keyword. For example, to find “tiny” models:

Shell

minerva-llm-list tiny

Output

TinyLlama-1.1B-Chat-v1.0

To search for “llama” models:

Shell

minerva-llm-list llama

Output

Llama-3.2-1B-Instruct
Llama-3.2-3B-Instruct
Llama-3.3-70B-Instruct

5 Using a Shared Model in Python

Create an execution script (e.g., example.py) to run local model inference:

Python

import os
from transformers import AutoTokenizer, AutoModelForCausalLM

# Access models folder via environment variable
model = os.path.join(os.environ["MINERVA_LLM_MODELS"], "TinyLlama-1.1B-Chat-v1.0")
print(f"Loading model: {model}")

tokenizer = AutoTokenizer.from_pretrained(model)
llm = AutoModelForCausalLM.from_pretrained(model)

inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What are the main causes of cardiovascular disease?"}],
    tokenize=True,
    return_tensors="pt",
    return_dict=True,
    add_generation_prompt=True,
)

output = llm.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Execute the Python script:

Shell

python example.py

Expected Script Output:

Output

Loading model: /hpc/packages/minerva-rocky9/minerva-llm/huggingface/TinyLlama-1.1B-Chat-v1.0
Loading weights: 100%|███████████████████████████████████████████████████████| 201/201 [00:00<00:00, 1721.67it/s]

There are several main causes of cardiovascular disease (CVD), including:
1. Smoking: Smoking is the leading cause of CVD. It causes the blood vessels to narrow, making it harder for blood to flow through them.
2. High blood pressure: High blood pressure is a major risk factor for CVD. It can cause the arteries to become narrow and hard to open, which can lead to heart attacks and strokes.

6 Running on a Login Node

Running multi-threaded model inference directly on login nodes may trigger thread limits:

Output

libgomp: Thread creation failed: Resource temporarily unavailable

To avoid this, set environment limits to restrict CPU usage before launching your script on login nodes:

Shell

export OMP_NUM_THREADS=1
export MKL_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1
export NUMEXPR_NUM_THREADS=1

Note: These thread restriction settings are generally not required when executing workloads on dedicated compute nodes.