Eos-test-dir-inject cannot get the current container id right

Troubleshooting
1

Hello,

We get an error when trying to run eos-test-dir-inject from the cta-migration-tools package.
In particular, it appears that it can’t get the current EOS file id and/or container id right.

E.g. when we run

/usr/bin/eos-test-dir-inject --config /etc/cta/castor-migration.conf --fileid 10001 --path /eos/antares/test_dir_delete_me

we get the error

FATAL ERROR: attempt to inject container with id=10001, which exceeds EOS current container id=0

despite the fact the current file and container id in our EOS space are

ALL current file id 10001
ALL current container id 41

We are running the latest, 4.0-5, version of the cta-migration-tools. Can this be some kind of bug?

Best,

George

2

Hi George,

The migration stuff is looked after by Michael who is on holiday right now, this will probably have to wait until the end of next week.

That said, the first thing I would check is that your client is properly authorised by the MGM (check its logs), that container id=0 is odd.

Cheers,

Oliver.

3

Hi Oliver,

Thanks for the reply. Indeed, our vid gateway mapping of the gRPC client was set up to be identified by its host name instead of its IP. Once we rectified this it worked.

After this change the container and file ids worked as expected.

Best,

George

4

Hi, I’ve just stumbled on the same issue. But I don’t understand this statement: “our vid gateway mapping of the gRPC client was set up to be identified by its host name instead of its IP”

A little more info here? How does one configure this? I’m trying this just on the CI in kubernetes setup.

5

Hi Eric,

The error George found above was related to GRPC authentication which is used in only a couple of places, for CASTOR migration tools and for cta-admin tf ls -l (to get namespace paths for tape files).

So - I wonder if you’ve really hit the same thing? What are you trying to do?

But to answer your question, in the above cases one needs to configure the EOS server to allow GRPC, which generally requires configuration of a gateway - doc/restapi/grpc.rst · master · dss / eos · GitLab

Oliver.

6

I’m looking at the migration scripts trying to “trick” EOS into thinking there is a file in the system. So I’ve been trying to run the /usr/bin/eos-test-dir-inject and /usr/bin/eos-test-file-inject commands with the migration.conf file and eos.token:migration-test-token which appears to be setup in the CI setup scripts.

7

With some help from Dan Szkola, I got a bit further (container ID is no longer 0) by adding my client to the gateways: eos -r 0 0 vid add gateway 10.254.58.4 grpc

However, no value of fileid seems permissible. I’ll look into the C++ code to see if I can understand this. However, I can leave the fileID off and it seems to work OK…

[root@client /]# eos-test-dir-inject --config /home/cta/CTAEvaluation/replacements/migration.conf --fileid 100 --path /eos/ctaeos/ewv

[{"id":"100","parentId":"0","uid":"1000","gid":"1000","treeSize":"0","mode":493,"flags":0,"name":"ZXd2","ctime":{"sec":"1553900400","nSec":"0"},"mtime":{"sec":"1553900400","nSec":"0"},"xattrs":{"sys.archive.storage_class":"bXlfc3RvcmFnZV9jbGFzcw=="},"path":"L2Vvcy9ncnBjL2Vvcy9jdGFlb3MvZXd2","etag":""}]FATAL ERROR: attempt to inject container with id=100, which exceeds EOS current container id=21

[root@client /]# eos-test-dir-inject --config /home/cta/CTAEvaluation/replacements/migration.conf --fileid 21 --path /eos/ctaeos/ewv
[{"id":"21","parentId":"0","uid":"1000","gid":"1000","treeSize":"0","mode":493,"flags":0,"name":"ZXd2","ctime":{"sec":"1553900400","nSec":"0"},"mtime":{"sec":"1553900400","nSec":"0"},"xattrs":{"sys.archive.storage_class":"bXlfc3RvcmFnZV9jbGFzcw=="},"path":"L2Vvcy9ncnBjL2Vvcy9jdGFlb3MvZXd2","etag":""}]FATAL ERROR: attempt to inject container with id=21, which exceeds EOS current container id=21

[root@client /]# eos-test-dir-inject --config /home/cta/CTAEvaluation/replacements/migration.conf --fileid 20 --path /eos/ctaeos/ewv
[{"id":"20","parentId":"0","uid":"1000","gid":"1000","treeSize":"0","mode":493,"flags":0,"name":"ZXd2","ctime":{"sec":"1553900400","nSec":"0"},"mtime":{"sec":"1553900400","nSec":"0"},"xattrs":{"sys.archive.storage_class":"bXlfc3RvcmFnZV9jbGFzcw=="},"path":"L2Vvcy9ncnBjL2Vvcy9jdGFlb3MvZXd2","etag":""}]EosTestDirInject::inject(): ContainerInsert failed with error 22: Attempted to create container with id=20, which already exists

[root@client /]# eos-test-dir-inject --config /home/cta/CTAEvaluation/replacements/migration.conf --fileid 22 --path /eos/ctaeos/ewv
[{"id":"22","parentId":"0","uid":"1000","gid":"1000","treeSize":"0","mode":493,"flags":0,"name":"ZXd2","ctime":{"sec":"1553900400","nSec":"0"},"mtime":{"sec":"1553900400","nSec":"0"},"xattrs":{"sys.archive.storage_class":"bXlfc3RvcmFnZV9jbGFzcw=="},"path":"L2Vvcy9ncnBjL2Vvcy9jdGFlb3MvZXd2","etag":""}]FATAL ERROR: attempt to inject container with id=22, which exceeds EOS current container id=21

[root@client /]# eos root://ctaeos ls /eos/ctaeos
cta
preprod
proc
tmp
8

Hi Eric,

You are hitting a safety measure which we put in place for the CASTOR to CTA migration (which is what those tools were designed for).

For the migration, we wanted to ensure that file IDs imported from CASTOR did not overlap with newly-created file IDs in EOS. So we set the “current file id” and “current container id” (see eos ns) to large values, 4 billion. CASTOR IDs were all below this number and new files would have an ID above this number, guaranteeing no overlaps.

The EOS file/dir inject utils will only accept a file ID which is below the EOS current file/container ID.

However, you can avoid this constraint simply by not specifying the file/container ID. Then a new file will be created with a file ID above the EOS current file/container ID.

Injecting files with a specific file ID has to be done with care, as you have to guarantee that it is not a file ID which has been previously used. In practice this feature should only be used if you reset the “current file id” and “current container id” values when the EOS instance was initialised. We wanted to preserve file IDs for the CASTOR to CTA migration in case we ever needed to trace where the file originally came from. Normally there is no reason to do this.

So indeed, leave off the file ID and it should work.

9

Thanks. That makes complete sense now and even answers the “why would you want to preserve these numbers?” question.