Tape wiped, but catalogue still believes the tape has records

General Discussion
1

Hi all,

In the continued spirit of doing silly things to see what the behaviour is (I know it hurts when I poke it there Doctor, I want to know why… :wink: ), I accidentally wiped the tape labels and the tapes clean. The tapes are now functionally empty, and it is a data loss scenario. I checked the documentation but I can’t see a clear answer for this issue.

CTA itself says files exist on the tape (cta-admin tape ls --all), the little EOS has had the data deleted, but I’m not sure what the following step is here. I have a double issue, as I can’t remove the tape either, as I get an SQL error.

So two questions:

  1. If I’ve somehow wiped a tape, what is the correct method for handling this with CTA? The label is gone, the data is gone, but the record in the catalogue still exists.

  2. As a brute force strategy, I tried cta-admin tape rm -v … and I receive the following error "deleteTape: executeNonQuery failed: executeNonQuery failed for SQL statement DELETE FROM TAPE WHERE VID = :DELETE_VID AND NOT EXISTS (SELECT VID FROM TAPE…: Executing non query statement: Database library reported: ERROR: update or delete on table “tape” violates foreign key constraint “tape_file_rb_tape_fk” on table "tape_file_recycle_bin"DETAIL: Key (vid)=(A01001) is still referenced from table “tape_file_recycle_bin”. (DB Result Status:7 SQLState:23503)
    " - how does done, knowing the tape and data is lost, clear that recycle bin so that tape can be purged from the system?

3

Sorry some error reporting code is missing code. The database constraints are correctly blocking the removing of a tape because it still has files in the recycle bin. Some code needs to be added to give you a human readable message rather than an ORA error.

You’re absolutely right David that you need to wipe the recycle bin. Here is the recipe, however please note that cta-admin tape reclaim is DANGEROUS. There is NO coming back!!! But I guess that’s what you want.

cta-admin tape ch --vid YOUR_VID --full true
cta-admin tape reclaim --vid YOUR_VID
cta-admin tape rm --vid YOUR_VID
4

I have created the following CTA issue regarding the confusing error message:

1 Like
5

Thank you @smurray. Indeed, living dangerously is what I’m wanting to do here.

I didn’t have to rm the tape, as it still remains valid in my library, it however was missing its on tape label (I really need to do a PR on make install for MHVTL, make it less painful to a user). Once I performed the cta-admin tape change --full true, and then the cta-admin tape change reclaim, I had no issues with tape usage and the cta-admin tape list --all showed the right values.

This leads me to a following question, how does one recover content from the recycle bin? It’s not in /eos/proc/recycle as I immediately suspected, but I don’t see obvious cta command for this.

6

Hi David, the short answer is the recycle bin functionality has not yet been implemented. With CASTOR we would write all of the metadata of a deleted file to the CASTOR server logs and definitively remove the metadata from the CASTOR databases. Recovering files that were accidentally deleted was a very manual job for tape operators who would have to extract a file’s metadata from the CASTOR server logs in order to repopulate the CASTOR namespace database. With CTA we simply decided to write the deleted metadata to a separate set of “recycle bin” database tables. We have yet to implement the tools surrounding those tables.

1 Like