Scientific Computing and Data

Partnering with researchers to advance scientific discovery

Scratch Directories

The scratch space on Minerva resources utilizes GPFS, the General Parallel File System, a massively parallel distributed file system. This space is intended for temporary production work and not long term storage. Files in scratch directories are not backed up and are subject to being purged after 14 days. It is the user’s responsibility to back up all important data to archival storage or other storage resources.

The scratch space is accessible from Minerva as:

/sc/arion/scratch/$USER

Purge Policy

The scratch file system cannot be used for long term storage and files on scratch are not backed up or guaranteed by Minerva. In the event of a file system crash or purge, files in scratch directories cannot be recovered. It is the user’s responsibility to back up all important data to archival storage or other storage resources.

Files are exempt from purge if they have been written to or read within the last 14 days. To see the list of files that will be purged you can use:

find  /sc/arion/scratch/$USER -atime +14

Quota policy

Scratch usage on Hydra (/sc/hydra/scratch) frequently reaches its maximum. This causes unnecessary errors for users who rely on it to pull large data temporarily. To ameliorate the issue, a per-user quota (15T) is implemented in the scratch folders in Hydra. This policy is to avoid any one user from consuming all off the scratch space and reduce related job failures.

Special Note: Modifying file access times (using touch or any other method) for the purpose of circumventing purge policies may result in the loss of access to the scratch file systems. Under special circumstances, users may request a purge exemption by submitting a request in a timely manner that includes detailed justification to hpchelp@hpc.mssm.edu. Please include file system (e.g. Hydra), PI of the project, user requesting exemption, Account #, time requested (e.g. two weeks), and detailed justification.

 

How to check your GFPS quota

To check your GPFS quotas, you can run the following script, either per user or per project is listed:

# showquota -h
usage: showquota [-h] [-u USER | -p PROJECT]

optional arguments:
  -h, --help            Show this help message and exit
  -u USER, --user USER  Show quota for user in groups
  -p PROJECT, --project PROJECT
                        Show quota for a project

 

You can find your quota on scratch by

# showquota -p scratch

Please also understand that the quota is based on GPFS’s quota report that is queried every 15 mins and might have a fudge factor (including some -ve numbers that you should ignore for small set of files) to account for files being created or deleted.