Hi,
It seems to me CTA support the format, Is there a document about CTA format support ? particularly for CPIO/tar.
is there a low level tool that can do some quick tests ?
For metadata, which field is for tape format and the value ?
to be specific, I want to try to use CTA to read the files written to tape like this
tar cvf -b 8192 /dev/nst0 -C path filename
thanks
Simon
Hi Simon,
indeed, this bit of information was missing from the doc page. We’ve put up an article about the Castor/CTA tape format here: Tape Format - EOSCTA Docs
In brief, CTA supports the Castor/CTA tape format, and is additionally able to read OSM and Enstore tapes. If your goal is to only read these tapes, and not write new data in the same format, then one way of going about it would be to add reader code in the same way as was done for OSM/Enstore. More info about the process can be found here: EOS 2023 Workshop (24-27 April 2023): External tape readers: Integration into CTA and OSM/Enstore cases · Indico
In your example above, CTA will sadly not be able to read the tape out of the gate, as CTA relies on a list of tapes and files to be present in its database.
There is a low level tool for reading tapes, called cta-readtp, which comes with a CTA installation. It doesn’t need the cta-taped daemon to run, but it does also need the file metadata database.
An execution of it looks like this:
[vlado@tpsrv040 /]$ cta-readtp --help
Usage:
cta-readtp <VID> <SEQUENCE> [options]
Where:
<VID> The VID of the tape to be read
<SEQUENCE> A sequence of tape file sequence numbers. The syntax to be used is:
f1-f2 Files f1 to f2 inclusive
f1- Files f1 to the last file on the tape
f1-f2,f4,f6- A series of non-consecutive ranges of files
Options:
-h, --help Print this help message and exit.
-f, --destination_files <FILE URL> URL to file containing a list of destination files.
If not set, all data read is written to file:///dev/null
If there are less destination files than read files, the remaining
files read will be written to file:///dev/null.
[vlado@tpsrv040 /]$ cta-readtp I00012 1-100
Mar 21 16:50:24.927374 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Started" userName="vlado" tapeVid="I00012"
Mar 21 16:50:24.929910 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Read configuration" catalogueDbType="DBTYPE_ORACLE" catalogueDatabase="cta" catalogueUsername="cta_production" devFilename="/dev/nst0" rawLibrarySlot="smc9" logicalLibrary="IBM370" unitName="I3700532"
Mar 21 16:50:50.599153 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Mounted tape" userName="vlado" tapeVid="I00012" tapeDrive="I3700532" logicalLibrary="IBM370" librarySlot="smc9" useLbp="true" driveSupportLbp="true"
Mar 21 16:50:50.599297 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Loading tape" userName="vlado" tapeVid="I00012" tapeDrive="I3700532" logicalLibrary="IBM370" useLbp="true" driveSupportLbp="true"
Mar 21 16:50:59.172927 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Loaded tape" userName="vlado" tapeVid="I00012" tapeDrive="I3700532" logicalLibrary="IBM370" useLbp="true" driveSupportLbp="true"
Mar 21 16:51:10.872216 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Reading file from tape" userName="vlado" tapeVid="I00012" fSeq="1" tapeDrive="I3700532" logicalLibrary="IBM370" useLbp="true" driveSupportLbp="true" destinationURL="file:///dev/null"
Mar 21 16:51:20.286584 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Read file from tape successfully" userName="vlado" tapeVid="I00012" fSeq="1" tapeDrive="I3700532" logicalLibrary="IBM370" useLbp="true" driveSupportLbp="true" destinationURL="file:///dev/null" checksumType="ADLER32" checksumValue="0x8acd78ac" readFileSize="3713324175"
Mar 21 16:51:20.408934 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Reading file from tape" userName="vlado" tapeVid="I00012" fSeq="2" tapeDrive="I3700532" logicalLibrary="IBM370" useLbp="true" driveSupportLbp="true" destinationURL="file:///dev/null"
Mar 21 16:51:23.769582 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Read file from tape successfully" userName="vlado" tapeVid="I00012" fSeq="2" tapeDrive="I3700532" logicalLibrary="IBM370" useLbp="true" driveSupportLbp="true" destinationURL="file:///dev/null" checksumType="ADLER32" checksumValue="0x3f612c18" readFileSize="1372606415"
Mar 21 16:51:23.867304 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Reading file from tape" userName="vlado" tapeVid="I00012" fSeq="3" tapeDrive="I3700532" logicalLibrary="IBM370" useLbp="true" driveSupportLbp="true" destinationURL="file:///dev/null"
Mar 21 16:51:31.502215 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Read file from tape successfully" userName="vlado" tapeVid="I00012" fSeq="3" tapeDrive="I3700532" logicalLibrary="IBM370" useLbp="true" driveSupportLbp="true" destinationURL="file:///dev/null" checksumType="ADLER32" checksumValue="0xe6393309" readFileSize="3105976598"
Mar 21 16:51:31.597874 tpsrv040.cern.ch cta-readtp: LVL="INFO" PID="23143" TID="23143" MSG="Reading file from tape" userName="vlado" tapeVid="I00012" fSeq="4" tapeDrive="I3700532" logicalLibrary="IBM370" useLbp="true" driveSupportLbp="true" destinationURL="file:///dev/null"
For a migration we recommend that you write a migration script which inserts the required tape and file metadata into CTA.
Hi, Rbachman
Thanks for the information provided. After looking format details, it seems to me that CTA doesn’t compatible with GNU tar, some work has to be done.
metadata shouldn’t be an issue.
thanks