Empty recall mount

Troubleshooting
1

Hello,

This is for CTA 5.10.10.1 on EL9.

We see cases where a recall request is queued in the scheduler but for some reason the tape never gets mounted for read. In the queue, after --minretrieverequestage seconds the queue time is being reset .

Checking the logs of the tape server contained in the object
RetrieveQueueToTransferForUser-TD0473-DriveProcess-obelix_ts1160_31-getafix-ts10.scd.rl.ac.uk-1048435-20240621-13:56:01-0-0

we see the following error

[root@getafix-ts10 cta]# cat cta-taped-obelix_ts1160*.log | grep TD0473 | grep -i erro
{“epoch_time”:1718977982.446178706,“local_time”:“2024-06-21T14:53:02+0100”,“hostname”:“getafix-ts10”,“program”:“cta-taped”,“log_level”:“WARN”,“pid”:“1048435”,“tid”:“1048435”,“message”:“Notified client of end session with error”,“drive_name”:“obelix_ts1160_31”,“instance”:“antares”,“sched_backend”:“cephUser”,“thread”:“MainThread”,“tapeDrive”:“obelix_ts1160_31”,“tapeVid”:“TD0473”,“mountId”:“1843741”,“vo”:“storaged_dls”,“tapePool”:“diamond_i12”,“errorMessage”:“Aborted: empty recall mount”,“MountTransactionId”:“1843741”}

Do you know what is going on?

Thanks

George

2

Dear George,

We would need some additional information to be able tell what is going on here. Could you please provide the full logfile or the full logs for the file(s) you are trying to recall and for the mount session?

Thank you

3

Thanks for your reply, and sorry for the delay.

We established that the reason why we were seeing the above message in the cta-taped logs is because the retrieve space was full - cta-fst-gcd was not working because the EOS FS scan kept updating the files’ ctime (see our other post here, Tape aware GC on FST stopped working - #6 by george_patargias) - and also because of the fact that we had created a single back-pressure rule that affected both of our EOS instance.

We have created an additional back-pressure rule, modified the existing one and we got GC working again and the “Empty recall mount” message did not appear again.

George