Access to Raw Data Stored in the Mass Storage System from Fsgi02

Mike Procario


One file from each data run is stored in the robotic tape system usually called the mass storage system (mss). This represents between 10% and 20% of all our data. This data can be accessed from CLUBS or fsgi02. This document describes the procedure for accessing that data on fsgi02.

Getting Files

There are two disks connected to fsgi02 reserved to hold the files copied from the mass storage system. These are /usr/e781/data02, and /usr/e781/data03. These disks can only be written to by the e781 account or by the program get_from_fmss. In order to request a file or files use the command get_from_fmss as in this example

get_from_fmss calib_run002000_001 calib_run002001_001

The program will look to see if the file is already on disk. If it is not it will look for it in the mass store, and fetch it to the disk for you. When the program is done it writes a file called files.cmd which contains the disk in commands needed to read the data. Example of a files.cmd

disk in /usr/e781/data02/calib_run002464_001 
disk in /usr/e781/data01/calib_run002514_001 
disk in /usr/e781/data03/calib_run002515_001 
disk in /usr/e781/data02/calib_run002516_001 
disk in /usr/e781/data01/calib_run002517_001

If you write your command files to include the line

in files.cmd

then your program is always ready to read in the data.

Please limit yourself to fetching no more than 3 or 4 files at a time. Our standard file size is 200 megabytes. Following this guideline should allow more than 10 people to access the data simultaneously.

Deleting Files

It is not necessary to delete any files. When the disks are full, the program will delete the least recently used file to free up space. Should it become necessary to delete files for some other reason. It would be necessary to request someone who has access to the e781 account to do it.

Logging Transactions

All fetches from the mass store and deletions from the disk are logged to the file /usr/e781/data01/log/get_from_fmss.log with the time, date and person running get_from_fmss. Here is what the file looked like early in the running:

 File calib_run002514_001 fetched at 15:52:06 10/02/96 by procario
 File calib_run002515_001 fetched at 16:05:23 10/02/96 by procario 
 File calib_run002516_001 fetched at 16:14:48 10/02/96 by procario 
 File calib_run002517_001 fetched at 16:18:38 10/02/96 by procario 
 File calib_run002464_001 fetched at 16:18:38 10/02/96 by procario
 File calib_run002108_001 fetched at 17:27:00 10/02/96 by mattson 
 File calib_run002108_001 fetched at 17:40:41 10/02/96 by mattson 
 File calib_run002108_001 fetched at 17:49:05 10/02/96 by mattson 
 File calib_run002618_001 fetched at 10:42:11 10/02/96 by konorov 
 File calib_run002613_001 fetched at 10:55:16 10/03/96 by e781 
 File calib_run002603_001 fetched at 11:00:35 10/03/96 by procario 
 File calib_run002610_001 fetched at 15:11:55 10/03/96 by mluksys 
 File calib_run002520_001 fetched at 15:55:47 10/03/96 by procario 
 File calib_run002590_001 fetched at 16:18:00 10/03/96 by mluksys
 File calib_run002499_001 fetched at 17:18:32 10/03/96 by procario
 File calib_run002489_001 fetched at  9:44:32 10/04/96 by procario
 File calib_run002488_001 fetched at  9:46:42 10/04/96 by procario

Strip Files

We store strip files in the mass store. It is desirable to name strip files in a rational manner so that it is possible to have a general idea of what they contain. In addition, the are technical and security reasons in our implementation that prevent us from just using the name passed by the user without some processing.

Currently get_from_fmss imposes a name requirement for files it will fetch. Name is of the form stripDD_WWWWWW_BBBB_EEEE, where

Files with names not of this form will not be fetched.

About this document ...

Access to Raw Data Stored in the Mass Storage System from Fsgi02

This document was generated using the LaTeX2HTML translator Version 96.1 (Feb 5, 1996) Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.

The command line arguments were:
latex2html -split 0 get_from_fmss.tex.

The translation was initiated by Michael Procario on Tue Apr 1 11:26:58 EST 1997

Michael Procario
Tue Apr 1 11:26:58 EST 1997