Archiving Data

Important data stored on Minerva can be protected by archiving the data in the IBM Tivoli Storage Management (TSM) system (renamed as Spectrum Protect after v8.1.7) installed on a Minerva server.

The TSM system will create two long-term tape copies of your data. One copy will be stored in the IBM tape library that is part of the Minerva complex and be available for rapid recall; the second copy will be stored off-site in a secure data vault for disaster protection. Data on both copies will be encrypted to deter unauthorized access.

Data retention policy
The archived data will have a retention time of 6 years and will then be deleted, please check the expiration date of your archived files.  This is the responsibility of the user!

How to access TSM

TSM client or Spectrum Protect client v8.1.7 is installed on all internal login nodes, i.e., minerva13, minverva14 and on the data nodes.  Users can issue archive commands, dsmc or dsmj, from either of the internal login nodes.

TSM cannot be accessed from external login nodes
Trying to use one of these commands in the external login node will result in a “Command not found” response”

Data that is archived is grouped in the TSM system by nodes. A node is an abstraction and can be physically many things. On Minerva, each user is considered a node to the TSM system and the node identity for each user is the userid.

The TSM system can be accessed via either a GUI or the command line. The command line mode is particularly useful when archiving large datasets in that it can be issued using the screen command. The screen can then be detached and the command can run unattended for the hours it may take to archive the data.

Access TSM with GUI
Access TSM with Command Line

General suggestions

Tar small files before archive.

Because all the files that are archived are entered into a database, to prevent overflowing this database we ask that you first use the tar command to create a unix tar archive of bundles of small files and then archive the tar file to TSM. For information about tar see the man pages ( man tar ) on Minerva or check out this link.

Command Line and Screen is recommended for large data archiving or retrieval.

It is not recommended to use the GUI for large data archiving or retrieval because you would have to keep the interactive session open until all of the data are archived. Instead, start a “screen” session and issue the line command to perform the archiving. You can then detach the screen session and the command will continue executing.

Long retrieval time is expected.

Due to the large amount of archived data and number of tapes, most of the tapes are sitting on the cabinet instead of the TSM library. Our operators get email notifications when you issue a retrieve request, and they will fetch the desired tape and load it into the library. This process is manual and the responding time for the operators is one and half hours. This is the time when the process is showing ” [ -]” but without progressing.

Once the tape is loaded into the TSM library, the library will automatically mount the tape and read its data. This data transfer time is reasonably fast.

Note that tape check-in errors may also occur when there are simultaneous retrieval requests. When you get error such as “data is unavailable”, please send in a ticket and we are happy to resolve it for you.

Warning: If one specifies that files should be deleted automatically after archive and then subsequently deletes the archive object the data will be permanently lost.


Frequently Asked Questions

How to use TSM for backup

How to access other users’ archived files

Error message with Permission Denied

Error message with file currently unavailable on server

Error message with file write protected and unable to retrieve to the disk

Error message with exceeded maximum number of mount points

        Can I keep my archived data over the 6 years’ retention time?

For a more extensive discussion of using TSM, see IBM Spectrum Protect Manual