Use of cta-ops-verify-tape

General Discussion
41

Hi @vlado ,

Sorry to come back to this.

As I mentioned, cta-ops-verify-tape does work for us when used with the default settings (i.e. verify 30 files).

When we try to submit a verification of a whole tape using the –all option

XrdSecPROTOCOL=sss XrdSecSSSKT=/etc/cta/tape-verification.keytab cta-ops-verify-tape -v JT4050 --all -C cta-operations-utilities/cta-ops-config.yaml

we get the usual messages indicating that the submission was successfull

2026-03-13 11:48:14 [INFO] [verify_tape] Running verify-tape for tape with vid JT4050, read speed of 300 MB/s, and data size: 0B
2026-03-13 11:48:16 [INFO] [whole_tape_scan] Performing whole tape scan.
2026-03-13 11:48:16 [INFO] [verify_tape] Verifying 4231 files and 50.2T from tape JT4050
2026-03-13 11:48:19 [INFO] [verify_files] ArchiveId of files to verify: ...
2026-03-13 11:48:19 [INFO] [verify_files] All file verifications queued for tape JT4050

but nothing appears in the CTA tape queue (cta-admin sq | grep JT4050 )….

The frontend appears to have sent the request

Mar 13 11:48:20.029099290 cta-front01 cta-frontend: LVL="INFO" PID="850" TID="124021" MSG="Catalogue - user modified tape - verificationStatus" instance="antares" sched_backend="cephUser" vid="JT4050" verificationStatus="{'date': 1773402499, 'status': 'ongoing', 'files_submitted': 4231, 'files_verified': 0, 'files_failed': 0}" lastUpdateUserName="tape-verification" lastUpdateHostName="cta-front01" lastUpdateTime="1773402500"

Can you please let us know what is wrong?

Thanks,

George

42

Hi George,

Thanks for bringing this up, I confirm that there is indeed a bug.

If you execute cta-ops-verify-tape -v VID --verbose with the --verbose option you will notice DEBUG output similar to this one:

2026-03-16 15:12:20 [DEBUG] [run_cmd] Executing shell command: /usr/bin/cta-verify-file --instance eosctabogus --id 1727583432 --vid I50024
2026-03-16 15:12:20 [DEBUG] [run_cmd] Executing shell command: /usr/bin/cta-verify-file --instance eosctabogus --id 1727578111 --vid I50024
2026-03-16 15:12:20 [DEBUG] [run_cmd] Executing shell command: /usr/bin/cta-verify-file --instance eosctabogus --id 1727583550 --vid I50024

This is completely missing when you specify --all option.

The problem is in the file /opt/cta-ops/ops-venv/lib/python3.9/site-packages/tapeverify/cta_verify_tape.py in this function:

235     def whole_tape_scan(tapefile_ls_fp):
236         """
237         Scan the full tape, just submit the whole iterable
238         """
239         logger.info("Performing whole tape scan.")
240         tf_iter = get_iter(tapefile_ls_fp)
241         return (list(tf_iter), True, file_size_sum)

The value tf_iter is not a list so it fails later in the code. If you put simple list(tf_iter) into the return statement on line 241, then it will work as expected.

I will submit a fix into the code, but I wanted to unblock you so that you can do this modification on your side and proceed.

Hope this helps. Best regards,

Vladimir
CERN

43

Hi Vlado,

The change you suggested resulted in the queueing of the job this time - many thanks for this!

Tim asks if there is a way to dedicate a drive that will be mounted by the verified tape.

Also, does the tool return some kind of message to indicate that the verification failed or succeeded?

Best,

George

44

Hi George,

This thread is quite long and is diverging a bit so I suggest we stop here. Better to open separate threads on the specific topics.

`cta-ops-verify-tape` is an orchestrator, it queues the files for verification but cannot report if verification succeeded or failed as this is not known until much later. Each verification job logs its status, so you should monitor/instrument those log messages to track the progress of verification. If you need more help, please create a new thread for that.

Drive dedication was a CASTOR concept that we did not bring into CTA. Please could you create a new thread about that and describe in detail what is your use case (what problem are you trying to solve by dedicating a drive to verification)? Perhaps there is another way to do what you want.

Cheers,

Michael

45

Hi Michael,

You are absolutely right - apologies for the divergence.

After making the above code change that Vlado suggested, the tool is working for us. Any other issues or feature requests ewe can put in in new post.

Best,

George

1 Like