Auth setup for cta-taped space queries for backpressure checks

Troubleshooting
1

Hi,

I’m struggling to get the back-pressure (disksystem) feature to work correctly, I think this is because I haven’t set the authentication up correctly, but I can’t see what I have done wrong.

The cta-taped is getting permission denied when checking space:

2022-03-10T16:26:38.217145+00:00 cta-ts09 cta-taped: LVL="ERROR" PID="148679" TID="148679" MSG="In OStoreDB::RetrieveMount::getNextJobBatch(): unable to request EOS free space for the job." thread="MainThread" tapeDrive="asterix_ts1160_17" mountId="91828" diskSystemName="eosantares_retrieve" failureReason="In DiskSystemFreeSpaceList::fetchEosFreeSpace(), failed to call 'eos root://antares-eos01 space ls -m' Errno=0: Success instanceAddress: antares-eos01 stderr: error: errc=3010 msg='[ERROR] Error response: Permission denied'" fileId="745235448" copyNb="1" requestAddress="RetrieveRequest-Frontend-cta-front01.scd.rl.ac.uk-2533-20220310-12:17:46-0-9597" isRepack="0"

The CTA taped is using the cta-taped SSS key, which maps onto the cta user (id 1000) on the EOS mgm via the key username (SSS vid mapping is enabled in EOS)

220310 16:26:38 266076 XrootdXeq: cta.148822:689@cta-ts09 pvt IPv4 login as cta
220310 16:26:38 func=IdMap  tident= sec=(null) uid=99 gid=99 name=- geo="" sec.prot=sss sec.name="cta" sec.host="cta-ts09.scd.rl.ac.uk" sec.vorg="" sec.grps="tape" sec.role="" sec.info="" sec.app="" sec.tident="cta.148822:689@cta-ts09" vid.uid=1000 vid.gid=33
...
220310 16:26:38 level=ERROR logid=dc282e32-a08e-11ec-9c4f-1c34da4b345c unit=mgm@antares-eos01.scd.rl.ac.uk:1094 tid=00007f15a03f4700 source=XrdMgmOfsFile:3239             tident=cta.148822:689@cta-ts09 sec=sss   uid=1000 gid=33 name=cta geo="" Unable to execute proc command - you don't have the requested permissions for that operation (2) /proc/admin/; Operation not permitted

The CTA user is a sudoer:

[root@antares-eos01 ~]# eos vid ls | grep sudoer
sudoer                 => uids(root,cta)
[root@antares-eos01 ~]# id cta
uid=1000(cta) gid=33(tape) groups=33(tape)
[root@antares-eos01 ~]#

And I can confirm that they get sudo privileges when using that key on the tape server, but that the space query is definitely getting denied:

[root@cta-ts09 ~]# XrdSecSSSKT=/etc/cta/cta-taped.sss.keytab eos root://antares-eos01 whoami
Virtual Identity: uid=1000 (99,1000) gid=33 (33,99) [authz:sss] sudo* host=cta-ts09.scd.rl.ac.uk domain=scd.rl.ac.uk
[root@cta-ts09 ~]# XrdSecSSSKT=/etc/cta/cta-taped.sss.keytab eos root://antares-eos01 space ls -m
error: errc=3010 msg="[ERROR] Error response: Permission denied"
[root@cta-ts09 ~]#

Having gone round this a few times, I thought I would check if anyone could spot something obvious that I have missed. Is there something else I need to set in EOS to allow a non root user to use the ‘space ls’ command?

Cheers,
Tom

2

Hi Tom,
backpressure is configured and used in CTA CI, this is where I had a look.

Basically I just ran the same commands in the tpsrv01 container in CI:

[root@tpsrv01 /]# XrdSecSSSKT=/etc/cta/cta-taped.sss.keytab eos root://ctaeos whoami
Virtual Identity: uid=2 (2,99) gid=2 (2,99) [authz:sss] sudo* host=[::ffff:192.168.210.11] domain=168.210.11]

sss key is mapping to user with uid 2: daemon which is different from your setup.

Then the space ls command works fine:

[root@tpsrv01 /]# XrdSecSSSKT=/etc/cta/cta-taped.sss.keytab eos root://ctaeos space ls -m
type=spaceview name=default cfg.groupsize=0 cfg.groupmod=24 nofs=1 ...

As I always says: when in doubt CI is always the reference setup and if provides the configuration recipes too.

Cheers,
Julien

1 Like
3

Hi Julien,

Thanks for the clarification. I was misled by documentation [1] [2] stating the cta-taped should use a separate SSS key. I’ll be sure to refer to how things are set up in the CI setup for these queries.

Have a good weekend,
Tom

[1] Configure CTA - EOSCTA Docs
[2] Create a native EOSCTA development environment - EOSCTA Docs