I have spotted this issue with a recall from tape. LHCb have written data into CTA and are testing the recalls. Noticed that one tape is not getting any data off. You can see the tape daemon get the request to mount the tape, but it never gets as far as issuing the mount (nothing in rmcd logs). The only error I can see in the taped logs is
cta-ts01.scd.rl.ac.uk-9015-20211119-10:19:25-0" ownedObjectCount=“12”
2021-11-19T10:19:25.962356+00:00 cta-ts01 cta-taped: LVL=“ERROR” PID=“9015” TID=“9015” MSG=“Caught an unexpected standard exception, cta-taped cannot start” what=“In DiskSystemList::at(): name not found.”
(I do like an unexpected standard exception ?! ).
Any bright ideas what the problem might be/where to look for a clue?
Have you been experimenting with the cta-admin disksystem command by any chance? The disk system name ends up in the request object and when it comes to be scheduled I wonder what happens if the disk system in question has been deleted. Could this be at the root of the problem? In which case reinstating any deleted disk system names may help.
Indeed we have been experimenting with the setting up the backpressure mechanism
(using the cta-admin disksystem command) but without much success so after a few
attempts we removed it from CTA.
By the way, is there any directive that needs to be added in cta-taped.conf to make the tape servers “aware” of the backpressure? Do we need to restart cta-taped whenever we add/remove the backpressure mechanism?
You may want to reinstate your disk system configuration to allow queued jobs to progress to completion.
There’s no configuration needed on the tapeservers for the backpressure, just the configuration of the disk system (cta-admin ds ...). You can configure the disk system with a large --targetedfreespace to ensure it triggers.
).