We were running some small file tests and transferring over less than a TB but close to half a TB.
The system currently for the drive is:
TS4500G1 LTO8D0 gmv18018 Up ArchiveForUser CleanUp 1120 VR5871 twaltontest vo - - - 21559 0 - 1120 -
Which has been like this for a while. Restarting the tape drive does not fix the issue.
Doing some googling and talking to people at fermilab we think that the ceph objectstore system is being cleaned up or wants to be cleaned up but is stuck maybe because of memory?
Have you run into this issue? I know there is another Tape drive stuck in “cleanup Retrieve” state but looking at that they tried restarting the tape drive and that worked but in this case that is not working and been tried multiple times.
Taped log has after latest restart:
Mar 7 08:15:03 gmv18018 cta-taped: LVL="DEBUG" PID="3474146" TID="3479471" MSG="RdbmsCatalogue::updateTapeDriveStatistics(): It didn't update statistics"
Mar 7 08:15:07 gmv18018 cta-taped: LVL="DEBUG" PID="3474148" TID="3474148" MSG="In MaintenanceHandler::exceptionThrowingRunChild(): About to do a maintenance pass." SubprocessName="maintenanceHandler"
Mar 7 08:15:07 gmv18018 cta-taped: LVL="DEBUG" PID="3474148" TID="3474148" MSG="DEBUG: In QueueCleanupRunner::runOnePass(): no queues requested a cleanup." SubprocessName="maintenanceHandler"
The tape is in the drive:
[root@gmv18018 ~]# mtx -f /dev/sg2 status | head -n 40
Storage Changer /dev/sg2:2 Drives, 10293 Slots ( 255 Import/Export )
Data Transfer Element 0:Empty
Data Transfer Element 1:Full (Storage Element 17 Loaded):VolumeTag = VR5871M8