Software and Packages
- Software and Applications
- Schrodinger Suite
- MATLAB, Simulink and MATLAB Distributed Compute Server
- Jupyter Notebook
Queues and Resources
- LSF Queues And Policies
- GPU Etiquette
- Access TSM with GUI
- Access TSM with Command Line
- Checkpoint Restart
- Disaster Recovery Plan
Access TSM with Command Line
The command to access the TSM system is: dsmc. The option needed to specify your node name is: -se=userid. You must always indicate your node (user name) on the command line or the command will fail.
You can either issue the command through your login shell, for example:
$ dsmc q archive -se= userid /home/proj1/ -sub=yes
or you can launch the spectrum protect (TSM) shell and issue the command there:
$ dsmc -se= userid Protect> q archive /home/proj1/ -sub=yes
In general, dsmc defaults to the current file system only. For example, if one is in the home file system, /hpc/users, a wildcard will not see any files in the scratch file system and vice versa. One can explicitly specify the directory name in another file system and those files will be visible.
Assume these files exist:
If one is in their home directory and specifies the file name as “*”, only file a.file and b.file will be seen.
If one is in the scratch directory and specifies the file name as “/hpc/users/minerva_user/*”, again, only a.file and b.file will be seen.
TSM distinguishes between treating a directory file as a directory or treating it as a file. When a file is archived, the parent directory file is also archived.
In general, if a directory is to be considered a directory, then a slash should follow the name. E.g.:
/hpc/users/m_user/all_my_data is the file all_my_data. Whether it is a directory or not it will be treated as a simple file.
/hpc/users/m_user/all_my_data/ is the directory all_my_data and all the files in the directory.
This makes a difference when you ask the dsmc commands to work with all the files below the directory with the -sub option. In the case with no slash, -sub has no effect since you are talking about the file; with the slash, -sub will act on the files below the directory name. See example below.
The screen command is particularly useful here when archiving large files. For information on how to use the screen command check the man page (man screen), Google screen or for a quick summary see our friends at Indiana U.
Some simple, elementary examples:
screen dsmc archive -se=userid big_directory -sub=yes -description="Irreplaceable data 1 Apr 2012"
After screen starts a new session and whether the command starts executing immediately or is blocked by TSM while it waits for resources: ctrl-a d will detach the screen and you can go on about your business.
Alternatively, you can just issue the screen command and an interactive screen session will begin. You can then issue a few preliminary command, perhaps, and then the dsmc command. The same: ctrl-a d terminal entry will detach the session.
To return to the screen session in either case: screen -r
1. Archive (1) file in “/home/proj1” directory called “file1.txt”
dsmc archive -se=userid "/home/proj1/file1.txt"
2. Archive file file2.txt in “/home/proj1”
dsmc archive -se=userid “/home/proj1/file2.txt”
3. Archive all files that end with “txt” in directory called “/home/proj1”
dsmc archive -se=userid /home/proj1/*.txt
Note: quotes are necessary only if there is white space in the name of the directory/file. “*” is a wildcard
dsmc archive -se=userid /home/proj1/ -sub=yes
Note: dsmc archive -se=userid /home/proj1 -sub=yes will archive only the directory file because the trailing slash is missing.
5. To invoke TSM client in foreground:
(cd to “/home/proj1” directory first)
LINUX> cd /home/proj1
(invoke TSM client)
LINUX/home/proj1> dsmc -se=userid
(issue a TSM command: will archive file file1.txt from the current directory “/home/proj1”)
Tsm> archive file1.txt
6. Archive file “file1.txt” then automatically delete it from your directory
dsmc archive -se=userid /home/proj1/file1.txt -deletefiles
7. Archive file “file1.txt” and assign a description/annotation to the archive
dsmc archive -se=userid /home/proj1/file1.txt -description=”Project 1 file June EOM”
Note: Default description is: Archive Date:current_date
The output of queries will be similar to the following format:
Size Archive Date - Time File - Expires on - Description ---- ------------------- ------------------------------- 32,768 B 09/26/12 09:27:56 /gs01/scratch/fludee01/eclipse 09/27/17 Archive Date: 09/26/12 32,768 B 09/26/12 14:39:50 /gs01/scratch/fludee01/charyprob 09/27/17 Archive Date: 9/26/12 32,768 B 09/26/12 10:56:05 /gs01/scratch/fludee01/zhangtest 09/27/17 test2 32,768 B 09/26/12 11:00:57 /gs01/scratch/fludee01/packages 09/27/17 packages as of : 09/26/12 32,768 B 09/27/12 11:37:27 /gs01/scratch/fludee01/packages 09/28/17 Archive Date: 09/27/12
Spaces separate the Size, Date, Time, File-name (possibly truncated to the next highest directory), Expiry, Description.
1. Query archives from directory(and subdirectories) “/home/proj1/”
dsmc q archive -se= userid /home/proj1/ -sub=yes
2. Query if file “file1.txt” has been archived
dsmc q archive -se=userid /home/proj1/file1.txt
3. Query files satisfying a particular description:
dsmc q archive -se=userid -description="Important test*" "*"
When retrieving large amounts of data, use of the screen command is recommended.
1. Retrieve file “file1.txt” from directory “/home/proj1” and stored it in its original location.
dsmc retrieve -se=userid /home/proj1/file1.txt
If file1.txt exists, you will get the following dialog (E.g., enter 1 to overwrite the existing file):
Select an appropriate action 1. Replace this object 2. Replace all objects that already exist 3. Skip this object 4. Skip all objects that already exist A. Abort this operation Action [1,2,3,4,A] : 1 Retrieving 8 /usr/local/bin/file1.txt [Done]
2. Retrieve file “file1.txt” from directory “/home/proj1“ and save to a different directory “/home/proj2” with different file name “file2.txt”
dsmc retrieve -se=userid /home/proj1/file1.txt /home/proj2/file2.txt
3. Retrieve files from a “described” archive:
dsmc retrieve -se=userid -description="Important test45" "*" -sub=yes
4. Retrieve all the files from directory “/home/proj1“ and save to a different directory “/home/proj2”
dsmc retrieve -se=userid /home/proj1/ /home/proj2/ -sub=yes
1. Delete file1.txt
dsmc delete archive -se=userid "/home/proj1/file1.txt"
2. Delete all files below a specified directory
dsmc delete archive -se=userid "home/proj1/" -sub=yes
3. Delete all files in a described archive package
dsmc delete archive -se=userid -description="not-so-important study" "*" -sub=yes
Note: dsmc delete archive -se=userid -description=”not-so-important study” “*” will delete nothing.