Tape wiped, but catalogue still believes the tape has records

Hi all,

In the continued spirit of doing silly things to see what the behaviour is (I know it hurts when I poke it there Doctor, I want to know why… :wink: ), I accidentally wiped the tape labels and the tapes clean. The tapes are now functionally empty, and it is a data loss scenario. I checked the documentation but I can’t see a clear answer for this issue.

CTA itself says files exist on the tape (cta-admin tape ls --all), the little EOS has had the data deleted, but I’m not sure what the following step is here. I have a double issue, as I can’t remove the tape either, as I get an SQL error.

So two questions:

  1. If I’ve somehow wiped a tape, what is the correct method for handling this with CTA? The label is gone, the data is gone, but the record in the catalogue still exists.

  2. As a brute force strategy, I tried cta-admin tape rm -v … and I receive the following error "deleteTape: executeNonQuery failed: executeNonQuery failed for SQL statement DELETE FROM TAPE WHERE VID = :DELETE_VID AND NOT EXISTS (SELECT VID FROM TAPE…: Executing non query statement: Database library reported: ERROR: update or delete on table “tape” violates foreign key constraint “tape_file_rb_tape_fk” on table "tape_file_recycle_bin"DETAIL: Key (vid)=(A01001) is still referenced from table “tape_file_recycle_bin”. (DB Result Status:7 SQLState:23503)
    " - how does done, knowing the tape and data is lost, clear that recycle bin so that tape can be purged from the system?

Sorry some error reporting code is missing code. The database constraints are correctly blocking the removing of a tape because it still has files in the recycle bin. Some code needs to be added to give you a human readable message rather than an ORA error.

You’re absolutely right David that you need to wipe the recycle bin. Here is the recipe, however please note that cta-admin tape reclaim is DANGEROUS. There is NO coming back!!! But I guess that’s what you want.

cta-admin tape ch --vid YOUR_VID --full true
cta-admin tape reclaim --vid YOUR_VID
cta-admin tape rm --vid YOUR_VID

I have created the following CTA issue regarding the confusing error message:

1 Like

Thank you @smurray. Indeed, living dangerously is what I’m wanting to do here.

I didn’t have to rm the tape, as it still remains valid in my library, it however was missing its on tape label (I really need to do a PR on make install for MHVTL, make it less painful to a user). Once I performed the cta-admin tape change --full true, and then the cta-admin tape change reclaim, I had no issues with tape usage and the cta-admin tape list --all showed the right values.

This leads me to a following question, how does one recover content from the recycle bin? It’s not in /eos/proc/recycle as I immediately suspected, but I don’t see obvious cta command for this.

Hi David, the short answer is the recycle bin functionality has not yet been implemented. With CASTOR we would write all of the metadata of a deleted file to the CASTOR server logs and definitively remove the metadata from the CASTOR databases. Recovering files that were accidentally deleted was a very manual job for tape operators who would have to extract a file’s metadata from the CASTOR server logs in order to repopulate the CASTOR namespace database. With CTA we simply decided to write the deleted metadata to a separate set of “recycle bin” database tables. We have yet to implement the tools surrounding those tables.

1 Like