{"id":8189,"date":"2024-09-11T13:42:51","date_gmt":"2024-09-11T17:42:51","guid":{"rendered":"https:\/\/labs.icahn.mssm.edu\/minervalab\/?page_id=8189"},"modified":"2025-06-03T15:39:10","modified_gmt":"2025-06-03T19:39:10","slug":"ollama","status":"publish","type":"page","link":"https:\/\/labs.icahn.mssm.edu\/minervalab\/documentation\/ollama\/","title":{"rendered":"Ollama"},"content":{"rendered":"

[et_pb_section fb_built=”1″ admin_label=”section” _builder_version=”3.22″ custom_padding=”0px||0px||false|false”][et_pb_row admin_label=”row” _builder_version=”4.9.0″ custom_padding=”||0px||false|false”][et_pb_column type=”4_4″ _builder_version=”3.25″ custom_padding=”|||” custom_padding__hover=”|||”][et_pb_text admin_label=”Breadcrumb” _builder_version=”4.9.0″ _module_preset=”default”]<\/p>\n

Scientific Computing and Data<\/a>\u00a0\/\u00a0High Performance Computing<\/a> \/ Documentation<\/a> \/ Ollama<\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ admin_label=”section” _builder_version=”3.22″ custom_padding=”0px||0px||false|false”][et_pb_row _builder_version=”4.9.0″ _module_preset=”default” custom_margin=”|auto|-20px|auto|false|false”][et_pb_column type=”4_4″ _builder_version=”4.9.0″ _module_preset=”default”][et_pb_text _builder_version=”4.9.0″ _module_preset=”default” header_font=”|700|||||||” header_text_color=”#221f72″ custom_margin=”||0px||false|false” custom_padding=”||0px||false|false”]<\/p>\n

Ollama<\/h1>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row admin_label=”row” _builder_version=”4.9.0″ background_size=”initial” background_position=”top_left” background_repeat=”repeat”][et_pb_column type=”4_4″ _builder_version=”3.25″ custom_padding=”|||” custom_padding__hover=”|||”][et_pb_text admin_label=”Text” _builder_version=”4.9.0″ text_line_height=”1.5em” hover_enabled=”0″ sticky_enabled=”0″]<\/p>\n

Ollama is a platform that enables users to interact with Large Language Models (LLMs) via an Application Programming Interface (API). It is a powerful tool for generating text, answering questions, and performing complex natural language processing tasks. Ollama provides access to various fine-tuned LLMs, allowing developers and researchers to integrate sophisticated language understanding and generation capabilities into their applications, such as chatbots, content creation tools, and research projects. With its easy-to-use API, Ollama streamlines the process of leveraging advanced Artificial Intelligence (AI) models, making it accessible for a wide range of users in different fields. For detailed information, please check here<\/a>.<\/p>\n

We provide an Ollama wrapper script that allows you to start an Ollama server on Minerva\u2019s compute node and access it from your local machine through an API endpoint. This setup enables computationally expensive LLM tasks to be performed on Minerva, while you can easily access the results from your local machine. Below are step-by-step usage instructions.<\/p>\n

 <\/p>\n

[\/et_pb_text][et_pb_text admin_label=”Text” _builder_version=”4.9.0″ text_line_height=”1.5em” hover_enabled=”0″ sticky_enabled=”0″]<\/p>\n

Usage<\/h2>\n

The Ollama script is available on the login node at the following location:<\/p>\n

\/usr\/local\/bin\/ <\/code><\/pre>\n
The script name is: minerva-ollama-web.sh<\/code>.<\/p>\n
To start the script, run minerva-ollama-web.sh<\/code> on a login node.<\/p>\n
Example:<\/p>\n
$ sh minerva-ollama-web.sh \n[INFO] Image not specified, check if previously used \n[INFO] Found previously used image \/hpc\/users\/hasans10\/minerva_jobs\/ollama_jobs\/ollama_v0.3.10.sif. Using it. \n[INFO] Project is not specified, or is acc_null, using 1st avail project. \n[INFO] Project to use is acc_hpcstaff \n[INFO] Parameters used are: \n[INFO] -n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 4\n[INFO] -M\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 300 \n[INFO] -W\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 6:00\n[INFO] -P\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 acc_hpcstaff \n[INFO] -J\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ollama \n[INFO] -q\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gpu \n[INFO] -R\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 v100 \n[INFO] -g\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 1 \n[INFO] -o\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/hpc\/users\/hasans10\/minerva_jobs\/ollama_jobs\n[INFO] -i\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \n\/hpc\/users\/hasans10\/minerva_jobs\/ollama_jobs\/ollama_v0.3.10.sif \n[INFO] Submitting Ollama job... \n[INFO] Job ID: 189591101 \nOllama is started on compute node lg03a03, port 9999 \n[Authentication Info] \nUser ID: hasans10 \nToken: d6972ea4292e66e93d4ece5f15831f14 \nAccess URL: http:\/\/10.95.46.94:51101 \n\n*** You can access Ollama from Python as shown below ***** \nfrom ollama import Client<\/pre>\n
ollama_client = Client(host='http:\/\/10.95.46.94:51101', headers={\"Authorization\": \"Bearer hasans10:d6972ea4292e66e93d4ece5f15831f14\"})<\/span><\/pre>\n
<\/code><\/code>By default, this script will start an Ollama server on Minerva\u2019s GPU node. If the script runs successfully, it will provide a URL such as http:\/\/10.95.46.94:51101,<\/code>\u00a0your user ID, and a randomly generated token. If you run a curl command with your URL, user ID, and token, you will see a message indicating that ‘Ollama is running’ (an example is provided below). PLEASE DO NOT SHARE YOUR URL AND TOKEN WITH OTHERS.<\/strong><\/p>\n
$ curl http:\/\/10.95.46.94:51101 -H \"Authorization: Bearer hasans10:d6972ea4292e66e93d4ece5f15831f14\"\nOllama is running\n<\/pre>\n
This indicates that the service is up and running, and you can interact with it. The generated URL will also serve as your API endpoint.<\/p>\n
Ollama is configured to run on a GPU node by default, as CPU execution is slow. The script is flexible, allowing you to change various parameters. To see the usage instructions, please run the following command:<\/p>\n
minerva-ollama-web.sh --help\n<\/pre>\n
\u00a0<\/h2>\n
Install Ollama Python Library<\/h2>\n
You can access Ollama through the Python library. If you don\u2019t have the Ollama Python library installed, use the following commands to install it on Minerva:<\/p>\n
module load python\/3.10.14\npip install --user ollama==0.3.1<\/pre>\n
Alternatively, after starting the Ollama server on Minerva, you can also access it from your local machine. To install the Ollama Python library on your local machine, use the following command:<\/p>\n
pip install ollama<\/pre>\n
For more details, visit the Ollama Python library GitHub page<\/a>.<\/p>\n
 <\/p>\n
Download Model and Chat<\/h2>\n
You can download a model using the URL received after job submission. For example:<\/p>\n
from ollama import Client\nollama_client = Client(host='http:\/\/10.95.46.94:51101', headers={\"Authorization\": \"Bearer hasans10:d6972ea4292e66e93d4ece5f15831f14\"})
ollama_client.pull('tinyllama')<\/pre>\n
You can then chat with a model as follows:<\/p>\n
stream = ollama_client.chat(\n  \u00a0 model='tinyllama',\n  \u00a0 messages=[{'role': 'user', 'content': 'What are the main causes of cardiovascular disease?'}],\n  \u00a0 stream=True,\n)\nfor chunk in stream:\n  \u00a0 print(chunk['message']['content'], end='', flush=True)<\/pre>\n
Sample Output:<\/p>\n
There are several causes of cardiovascular disease, which includes:\n1. Hypercholesterolemia or high cholesterol: This is a major cause of cardiovascular disease (CVD). High levels of bad (Low-density lipoprotein) (\"LDL\") cholesterol can lead to thickening and plaque buildup in the blood vessels, which can eventually block or rupture them.\n2. Hypertriglyceridemia: This is another major cause of CVD, especially when triglycerides (fatty substances) are present in large amounts.\n3. Hypertension (high blood pressure): High blood pressure can damage the arteries and reduce blood flow to the heart and other organs.\n4. Diabetes: This condition results in high levels of glucose (sugar)in the bloodstream, which can cause plaque buildup in the arteries.\n5. Smoking and alcohol use: Both smoking and excessive alcohol consumption can damage the lining of blood vessels, leading to inflammation and increased risk of CVD.\n6. Physical inactivity: A sedentary lifestyle is associated with an increased risk of CVD due to reduced blood flow and oxygen supply to organs and muscles.\n7. Obesity: Excess body fat can cause a buildup of plaque in the arteries, leading to CVD.\n8. Atherosclerosis: This is the thickening and hardening of artery walls due to inflammation and plaque accumulation.<\/pre>\n
 <\/p>\n
Change Ollama Work Directory<\/h2>\n
\u00a0<\/u><\/strong>By default, Ollama stores models in your HOME directory. Some Ollama models are quite large and may exceed the 20GB size limit of your HOME directory. To avoid this issue, you can use your project directory (or another directory with sufficient space) as the Ollama work directory. For example, you can change the work directory as shown below.<\/p>\n
sh minerva-ollama-web.sh -o \/sc\/arion\/projects\/<projectid><\/pre>\n
or<\/p>\n
sh minerva-ollama-web.sh --ollamaworkdir \/sc\/arion\/projects\/<projectid><\/pre>\n
[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":"
Scientific Computing and Data\u00a0\/\u00a0High Performance Computing \/ Documentation \/ OllamaOllamaOllama is a platform that enables users to interact with Large Language Models (LLMs) via an Application Programming Interface (API). It is a powerful tool for generating text, answering questions, and performing complex natural language processing tasks. Ollama provides access to various fine-tuned LLMs, allowing developers […]<\/p>\n","protected":false},"author":624,"featured_media":0,"parent":35,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"class_list":["post-8189","page","type-page","status-publish","hentry"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-json\/wp\/v2\/pages\/8189","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-json\/wp\/v2\/users\/624"}],"replies":[{"embeddable":true,"href":"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-json\/wp\/v2\/comments?post=8189"}],"version-history":[{"count":29,"href":"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-json\/wp\/v2\/pages\/8189\/revisions"}],"predecessor-version":[{"id":10056,"href":"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-json\/wp\/v2\/pages\/8189\/revisions\/10056"}],"up":[{"embeddable":true,"href":"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-json\/wp\/v2\/pages\/35"}],"wp:attachment":[{"href":"https:\/\/labs.icahn.mssm.edu\/minervalab\/wp-json\/wp\/v2\/media?parent=8189"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}