Scratch Space (GPFS)
The scratch space on Minerva resources utilizes GPFS, the General Parallel File System, a massively parallel distributed file system. This space is intended for temporary production work and not long term storage. Files in scratch directories are not backed up and are subject to being purged after 14 days. It is the user’s responsibility to back up all important data to archival storage or other storage resources.
The scratch space is accessible from Minerva as:
The scratch file system cannot be used for long term storage and files on scratch are not backed up or guaranteed by Minerva. In the event of a file system crash or purge, files in scratch directories cannot be recovered. It is the user’s responsibility to back up all important data to archival storage or other storage resources.
Files are exempt from purge if they have been written to or read within the last 14 days. To see the list of files that will be purged you can use:
find /sc/hydra/scratch/$USER -atime +14
Scratch usage on Hydra (/sc/hydra/scratch) frequently reaches its maximum. This causes unnecessary errors for users who rely on it to pull large data temporarily. To ameliorate the issue, a per-user quota (15T) is implemented in the scratch folders in Hydra. This policy is to avoid any one user from consuming all off the scratch space and reduce related job failures.
Special Note: Modifying file access times (using touch or any other method) for the purpose of circumventing purge policies may result in the loss of access to the scratch file systems. Under special circumstances, users may request a purge exemption by submitting a request in a timely manner that includes detailed justification to email@example.com. Please include file system, e.g. Hydra, PI of the project, user requesting exemption, Account #, time requested, e.g. two weeks, and detailed justification.