All use of Scientific Computing and Data resources in publications and presentations must acknowledge Mount Sinai’s CTSA funding award. Click here

Scientific Computing and Data / High Performance Computing / Documentation / Minerva-Res Data Transfer

Data Transfer Guide for Minerva-Res

1 Overview

Data may only be transferred to and from Minerva-Res using an approved method as described below. All transfers of any type will be logged, and users remain responsible for following the restrictions that apply to their data according to the DUA.

  • By default, (1) standard SFTP and SSH file transfers are disabled. (2) All internet access is blocked, with only certain approved remote sites whitelisted via proxy settings for a defined period.
  • You must not copy, export, transfer, or expose data to unapproved environments by any means — including copy/paste, downloads, screenshots, or screen recordings.
  • All approved data transfers must use cryptographic tools during transit.

Data Derivatives

All data derivatives** must be treated with the same level of protection unless explicitly approved for release by the PI as non-controlled data, following the restrictions that apply to their data according to the DUA.

**Data derived from controlled-access datasets obtained from NIH-designated data repositories. Examples of derived data include imputed datasets and single nucleotide polymorphisms, or any data explicitly designated as Data Derivatives by NIH. For additional information, see the NIH Data Access Certification Agreement.

Data Transfer Summary

 

Data Classification Transfer Route Required Action & Approvals
Non-Controlled Data
(Standard & CMS)		 Data In (Ingress) HPC approval required.
Data Out (Egress) PI approval required.
Controlled Data
(Standard & CMS)		 Data In (Ingress) CASE 1: Complete the External Connectivity Form for direct data download from a remote server.
CASE 2: Otherwise, submit an HPC ticket for guidance.
Controlled Data
(Standard & CMS)		 Data Out (Egress) Strictly Prohibited. Requires DUA amendment & institutional review. Contact hpchelp@hpc.mssm.edu.

Part A: Standard (Non-CMS) Data

2 Non-Controlled Data (Non-CMS)

Non-Controlled Data here refers to: data that has been reviewed by the Principal Investigator and determined not to contain NIH controlled-access data, CMS controlled-access data, participant-level genomic data, or other information subject to access restrictions under the applicable Data Use Agreement, NIH policy, CMS policy, or institutional requirements.

Data In

Upload to Minerva-Res

Non-controlled data may be transferred from Minerva Arion to Minerva-Res with HPC approval.

  1. Place your files in the designated group-specific staging directory:

    /sc/arion/projects/<project>/from-arion-to-arionencrypt
    • If the staging directory does not exist, create it under your Arion project directory:
      mkdir from-arion-to-arionencrypt
    • It is highly recommended to create a subfolder named after your Minerva user ID to help organize files and avoid naming conflicts:
      /sc/arion/projects/<project>/from-arion-to-arionencrypt/<userid>
  2. Submit a data transfer request.
  3. The HPC team will review the request. Once approved, HPC staff will transfer the data to the Minerva-Res staging location.
  4. Move the transferred files from the staging directory into your working directory.

Data Out

Export from Minerva-Res

Non-controlled data may be transferred from Minerva-Res to Minerva Arion with PI approval.

PI Responsibility: PIs are responsible for the classification of the data and for approving that the data to be released is non-controlled and complies with the applicable Data Use Agreement (DUA) and institutional policies.

  1. Place your files in the designated staging directory:

    /arionencrypt/projects/<project_res>/from-arionencrypt-to-arion
  2. Submit a data transfer request.
  3. The PI will be notified to review and approve the request.
  4. Upon PI approval, HPC staff will transfer the data to the approved Arion destination staging area.
  5. Move the transferred files from the destination staging directory into your working directory.

3 Controlled Data (Non-CMS)

Data In

Upload to Minerva-Res

If you must download data from approved Controlled-Access Data Repositories directly, please follow the steps below.

Please allow 2-3 weeks for the full review and approval process before a final decision is reached.
  1. Complete the External Connectivity Request Form and provide the required information, including the justification for the request and the external IP address or subnet that needs access.
  2. The HPC team will review the request and evaluate the business or research justification. Additional information may be requested if needed. All requests are subject to security review and approval. Please allow 2-3 weeks for a final decision.
  3. Once approved, the requested IP source will be added to the allow-list.
  4. Perform the approved data ingress activities. Access will be removed once the transfer is complete.

User Responsibilities

  • Submit accurate and complete information in the request form.
  • Use the granted access only for the approved purpose.
  • Notify HPC staff if access is no longer needed before the scheduled expiration.
  • Submit a new request if future access is required after the original approval period expires.

Data Out

Export from Minerva-Res

Export or transfer of controlled-access data from the Minerva-Res environment is prohibited unless expressly authorized under the applicable Data Use Agreement (DUA) and approved through the institutional review process. To request a review, contact hpchelp@hpc.mssm.edu.

Part B: CMS-Related Data Only

4 Non-Controlled Data (CMS)

The workflows outlined below must be used to transfer non-controlled CMS data assets across secure environments.

Data In

Upload Non-Controlled CMS Data to Minerva-Res

Non-controlled CMS data may be transferred from the Minerva Arion staging area to Minerva-Res with HPC approval.

  1. Place your files under your pre-created staging subdirectory in the Arion file system on Minerva:

    /sc/arion/cmsdatastaging/from-arion-to-arionencrypt/<your_userid>
  2. Submit a data transfer request.

  3. The HPC Team will verify your request, run the secure transfer script, and provide you with the transfer logs. Transferred files will arrive at the following location on Minerva-Res:

    /arionencrypt/cmsdata/<yourproject_res>/from-arion-to-arionencrypt/
  4. Verify and Move: You are responsible for verifying the integrity of the transferred data and moving it from the staging directory into your working project directory.

Data Out

Export Non-Controlled CMS Data from Minerva-Res

Non-controlled CMS data may be transferred back from Minerva-Res (Arionencrypt) to the Minerva central Arion staging area with PI approval.

PI Responsibility: PIs are responsible for the classification of the CMS data and for approving that the data to be released is non-controlled and complies with the applicable CMS Data Use Agreement (DUA) and institutional policies.

  1. Place your files in your designated project export staging directory on Minerva-Res:

    /arionencrypt/cmsdata/<yourproject_res>/from-arionencrypt-to-arion/
  2. Submit a data transfer request.

  3. Upon approval, HPC staff will execute the transfer. The files will be transferred to the Arion file system on Minerva at in your personal staging directory:

    /sc/arion/cmsdatastaging/from-arionencrypt-to-arion/<your_userid>
  4. Verify and Move: Ensure the data’s integrity, and move the files out of the staging area and into your working path.

User Obligations for CMS Non-Controlled Data

  • Prompt Cleanup: Move transferred data out of the staging folders to its permanent resting location immediately.

5 Controlled Data (CMS)

The workflows outlined below must be followed strictly for all CMS controlled-access datasets.

Data In

Upload Controlled CMS Data to Minerva-Res

To request the ingress of controlled CMS data into the secure environment, users must submit a formal service ticket.

Important: Do not begin your transfer or attempt to stage or move controlled CMS files prior to official HPC guidance.
  1. Submit an HPC data transfer request specifying that you are requesting controlled CMS data ingress.
  2. The HPC team will review the ticket, verify the associated CMS DUA permissions, and coordinate the transfer process with you.

Data Out

Export Controlled CMS Data from Minerva-Res

Export or transfer of controlled CMS data from the Minerva-Res environment is strictly prohibited unless explicitly authorized under the applicable CMS Data Use Agreement (DUA) and approved through the formal institutional review process. To request a review, contact hpchelp@hpc.mssm.edu.

User Obligations for CMS Controlled Data

  • DUA Compliance: Do not use non-controlled pathways or standard staging directories to bypass security policies for CMS-controlled data.