# Cannot delete tape files

**URL:** <https://cta-community.web.cern.ch/t/cannot-delete-tape-files/327>\
**Category:** General Discussion\
**Created:** [3 September 2024 11:59 UTC](https://cta-community.web.cern.ch/t/cannot-delete-tape-files/327 "2024-09-03T11:59:36Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![george\_patargias](https://cta-community.web.cern.ch/letter_avatar_proxy/v4/letter/g/b9bd4f/32.png) [@george\_patargias](https://cta-community.web.cern.ch/u/george_patargias)\
**Post date:** [3 September 2024 11:59 UTC](https://cta-community.web.cern.ch/t/cannot-delete-tape-files/327/1 "2024-09-03T11:59:37Z")

</div>

Hi,

I have a set of tape files that I want to delete (so that I can reclaim the tapes).

[root@cta-adm1 georgep]# head lhcbchall22\_tapefiles  
archive id copy no vid fseq block id instance disk fxid size checksum type checksum value storage class owner group creation time  
4308411357 1 CT4063 37 737605 eosantares 31615cce 5.4G ADLER32 f901298b lhcbchall22 36250 24289 2022-12-15 15:08  
4308411030 1 CT4063 40 799072 eosantares 31615b85 5.4G ADLER32 602e4d76 lhcbchall22 36250 24289 2022-12-15 15:08  
…

These files don’t seem to have any EOS namespace entry

[root@cta-adm1 georgep]# eos fileinfo fxid:31615cce  
error: cannot retrieve file meta data - Error while fetching FileMD #828464334 protobuf from QDB: Empty response (errc=2) (No such file or directory)  
[root@cta-adm1 georgep]# eos fileinfo fxid:31615b85  
error: cannot retrieve file meta data - Error while fetching FileMD #828464005 protobuf from QDB: Empty response (errc=2) (No such file or directory)  
[root@cta-adm1 georgep]#

However, when I try to do something like

> cta-admin tapefile rm --vid CT4063 --id 4308411357 --reason “test file”

I get the following

> Cannot delete a copy of the file with archiveFileId 4308411357 because it is the only copy

I am not sure why CTA doesn’t let me delete the file…Hopefuly, it is not another file. Do you have any suggestions?

I am running CTA 5.10.10.1-1.el9 and EOS 5.2.23

Thanks,

George

---

<div class="post-metadata">

**Author:** ![mdavis](https://cta-community.web.cern.ch/user_avatar/cta-community.web.cern.ch/mdavis/32/16_2.png) [@mdavis](https://cta-community.web.cern.ch/u/mdavis)\
**Post date:** [3 September 2024 16:30 UTC](https://cta-community.web.cern.ch/t/cannot-delete-tape-files/327/2 "2024-09-03T16:30:06Z")

</div>

Hi George,

The normal way to delete a file in CTA is using `eos rm`. This is a “safe” delete, as it puts the tape file in the recycle bin, along with a copy of its disk metadata.

`cta-admin tapefile rm`, by design, will only remove the second tape copy of a file. If there is only one copy, it gives the error that you see, in order to prevent accidental and permanent deletion of data.

Perhaps we should add a `--force` option, but for the moment this does not exist. I’ll raise it in our ops meeting.

Michael

---

<div class="post-metadata">

**Author:** ![george\_patargias](https://cta-community.web.cern.ch/letter_avatar_proxy/v4/letter/g/b9bd4f/32.png) [@george\_patargias](https://cta-community.web.cern.ch/u/george_patargias)\
**Post date:** [4 September 2024 08:24 UTC](https://cta-community.web.cern.ch/t/cannot-delete-tape-files/327/3 "2024-09-04T08:24:41Z")

</div>

Hi Michael,

Thank you for the reply.

I am aware that the normal way to delete a file is by doing “eos rm”. However, I cannot see them in the EOS namespace because if I try to run “eos fileinfo” on the fxid/fid of any these files, I get the message

> error: cannot retrieve file meta data - Error while fetching FileMD #828404632 protobuf from QDB: Empty response (errc=2) (No such file or directory)

Also, running an “eos find -f” on the EOS area where these files are supposed to be from this tape pool (judging from the paths of deleted files from the same pool) yields nothing.

George

---

<div class="post-metadata">

**Author:** ![george\_patargias](https://cta-community.web.cern.ch/letter_avatar_proxy/v4/letter/g/b9bd4f/32.png) [@george\_patargias](https://cta-community.web.cern.ch/u/george_patargias)\
**Post date:** [6 September 2024 14:28 UTC](https://cta-community.web.cern.ch/t/cannot-delete-tape-files/327/4 "2024-09-06T14:28:46Z")

</div>

Hi Michael,

Sorry for the hassle. Since I cannot see any of these files in the EOS namespace, is it safe to delete them directly from the DB? From the _tape\_file_ and _archive\_file_ tables?

Best,

George

---

<div class="post-metadata">

**Author:** ![mdavis](https://cta-community.web.cern.ch/user_avatar/cta-community.web.cern.ch/mdavis/32/16_2.png) [@mdavis](https://cta-community.web.cern.ch/u/mdavis)\
**Post date:** [10 September 2024 15:07 UTC](https://cta-community.web.cern.ch/t/cannot-delete-tape-files/327/5 "2024-09-10T15:07:39Z")

</div>

Hi George,

We don’t have a tool that allows that, and the main reason for that is the risk of data loss. We can look at adding a `--force` option to `cta-admin tapefile rm` for this situation.

The safest way to delete it manually would be to send a DELETE event to the CTA Frontend, containing the tape file metadata. I think you could do this by creating an empty file (in a tape-backed directory) and manually adding the extended attribute `sys.archive.file_id="<id of the file you want to delete>"`. Then delete that file.

**Caveat emptor:** I haven’t tried this, attempt at your own risk.

---

<div class="post-metadata">

**Author:** ![george\_patargias](https://cta-community.web.cern.ch/letter_avatar_proxy/v4/letter/g/b9bd4f/32.png) [@george\_patargias](https://cta-community.web.cern.ch/u/george_patargias)\
**Post date:** [11 September 2024 15:00 UTC](https://cta-community.web.cern.ch/t/cannot-delete-tape-files/327/6 "2024-09-11T15:00:50Z")

</div>

Hi Michael,

Thanks for the suggestion as well as for the word of caution!

It looks like what you suggested did work

Creating the stub file generated an error

[root@cta-adm1 georgep]# eos touch /eos/antares/prod/lhcb/dataChallenge/stub.4308411357  
error: unable to touch ‘/eos/antares/prod/lhcb/dataChallenge/stub.4308411357’ (errc=71) (Protocol error)

but the Frontend received the CREATE event and the file was created

Sep 11 15:40:32.581964274 cta-front04 cta-frontend: LVL=“INFO” PID=“1275296” TID=“95302” MSG=“In WorkflowEvent::WorkflowEvent(): received event.” user=“eosantares@cta-front04” eventType=“CREATE” eosInstance=“eosantares” diskFilePath=“/eos/antares/prod/lhcb/dataChallenge/stub.4308411357” diskFileId=“879599649”

Sep 11 15:40:32.581964274 cta-front04 cta-frontend: LVL=“INFO” PID=“1275296” TID=“95302” MSG=“In WorkflowEvent::WorkflowEvent(): received event.” user=“eosantares@cta-front04” eventType=“CREATE” eosInstance=“eosantares” diskFilePath=“/eos/antares/prod/lhcb/dataChallenge/stub.4308411357” diskFileId=“879599649”

Then, I assigned the archive file id as an extended EOS attibute

[root@cta-adm1 georgep]# eos attr set sys.archive.file\_id=4308411357 /eos/antares/prod/lhcb/dataChallenge/stub.4308411357

[root@cta-adm1 georgep]# eos attr ls /eos/antares/prod/lhcb/dataChallenge/stub.4308411357  
sys.archive.file\_id=“4308411357”  
sys.eos.btime=“1726065632.580272011”  
sys.fs.tracking=“+100”  
sys.utrace=“cc64f23e-704b-11ef-a7a4-1c34da4b345c”  
sys.vtrace=“[Wed Sep 11 15:40:32 2024] uid:0[root] gid:0[root] tident:root.4040157:521@cta-adm1 name:daemon dn: prot:sss app: host:cta-adm1.scd.rl.ac.uk domain:scd.rl.ac.uk geo: sudo:1”

and, finally, doing an “eos rm” on this file did send the DELETE event to the Frontend and the file was removed from the tape

Sep 11 15:46:04.253622697 cta-front04 cta-frontend: LVL=“INFO” PID=“1275296” TID=“165677” MSG=“In OracleCatalogue::copyArchiveFileToRecycleBinAndDelete: ArchiveFile moved to the recycle-bin.” user=“eosantares@cta-front04” archiveFileId=“4308411357” diskFileId=“879599649” diskFilePath=“/eos/antares/prod/lhcb/dataChallenge/stub.4308411357” diskInstance=“eosantares” insertToRecycleBinTime=“0.003639” setTapeDirtyTime=“0.000916” deleteTapeFilesTime=“0.002752” deleteArchiveFileTime=“0.005183”  
Sep 11 15:46:04.254170812 cta-front04 cta-frontend: LVL=“INFO” PID=“1275296” TID=“165677” MSG=“In RdbmsCatalogue::moveArchiveFileToRecycleLog(): ArchiveFile moved to the file-recycle-log.” user=“eosantares@cta-front04” fileId=“4308411357” diskInstance=“eosantares” requestDiskInstance=“eosantares” diskFileId=“828464334” diskFileInfo.owner\_uid=“36250” diskFileInfo.gid=“24289” fileSize=“5368709120” creationTime=“1671116899” reconciliationTime=“1671116899” storageClass=“lhcbchall22” checksumType=“ADLER32” checksumValue=“f901298b” TAPE\_FILE=“copy number: 1 vid: CT4063 fSeq: 37 blockId: 737605 creationTime: 1671116899 fileSize: 5368709120” checkDeleteRequestConsistency=“3.5e-05” copyArchiveFileToFileRecyleLogAndDeleteTime=“0.013002” totalTime=“0.013042”  
Sep 11 15:46:04.255578240 cta-front04 cta-frontend: LVL=“INFO” PID=“1275296” TID=“165677” MSG=“In WorkflowEvent::processDELETE(): archive file deleted.” user=“eosantares@cta-front04” fileId=“4308411357” address=“null” filePath=“/eos/antares/prod/lhcb/dataChallenge/stub.4308411357” catalogueGetArchiveFileByIdTime=“0.001877” schedulerTime=“0.01467”

[root@cta-adm1 georgep]# cta-admin tf ls --id 4308411357  
Archive file with ID 4308411357 does not exist

I will go ahead and apply this to the rest of the tape files/

Many thanks again,

George

---

<div class="post-metadata">

**Author:** ![mdavis](https://cta-community.web.cern.ch/user_avatar/cta-community.web.cern.ch/mdavis/32/16_2.png) [@mdavis](https://cta-community.web.cern.ch/u/mdavis)\
**Post date:** [13 September 2024 07:27 UTC](https://cta-community.web.cern.ch/t/cannot-delete-tape-files/327/7 "2024-09-13T07:27:32Z")

</div>

Thanks George, good to know this worked.

We will be revising the command line tools next year and will look at adding an option to cover this case.
