Help with installation of CTA Operations Utilities

General Discussion
1

Hi,

I am trying to install the CTA Operations Utilities following the instructions listed here

and here

After cloning the GitLab repo, I did the following

python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install psycopg2-binary
pip install --extra-index-url https://cta-public-repo.web.cern.ch/cta-operations/pip/simple/ --requirement cta-operations-utilities/requirements.txt

but I got the following error. Can you please give me a clue what I need to do?

Blockquote
Looking in indexes: Simple index, https://cta-public-repo.web.cern.ch/cta-operations/pip/simple/
Collecting requests==2.27.1 (from -r cta-operations-utilities/requirements.txt (line 12))
Using cached requests-2.27.1-py2.py3-none-any.whl.metadata (5.0 kB)
Collecting psycopg2==2.9.5 (from -r cta-operations-utilities/requirements.txt (line 13))
Using cached psycopg2-2.9.5.tar.gz (384 kB)
Installing build dependencies … done
Getting requirements to build wheel … error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [34 lines of output]
/tmp/pip-build-env-jvyxqzfo/overlay/lib/python3.9/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

          ********************************************************************************
          Please consider removing the following classifiers in favor of a SPDX license expression:

          License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)

          See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
          ********************************************************************************

  !!
    self._finalize_license_expression()
  running egg_info
  writing psycopg2.egg-info/PKG-INFO
  writing dependency_links to psycopg2.egg-info/dependency_links.txt
  writing top-level names to psycopg2.egg-info/top_level.txt

  Error: pg_config executable not found.

  pg_config is required to build psycopg2 from source.  Please add the directory
  containing pg_config to the $PATH or specify the full executable path with the
  option:

      python setup.py build_ext --pg-config /path/to/pg_config build ...

  or with the pg_config option in 'setup.cfg'.

  If you prefer to avoid building psycopg2 from source, please install the PyPI
  'psycopg2-binary' package instead.

  For further information please check the 'doc/src/install.rst' file (also at
  <https://www.psycopg.org/docs/install.html>).

  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

2

Hi George,
psycopg is indeed notoriously tricky. It looks to me like the binary install is not found/recognized for some reason, and it still tries the source install.

Could you please try again by, before running pip install ..., following the instructions for a from-source install of psycopg as described here: CTA / CTA Operations Utilities · GitLab
You will need to download the psycopg .tar.gz file on the host, for instance from their github repo, and then build it.

For the build (on Alma9) you will likely need a combination of these packages (notably the libpq one, as mentioned in the error message above): python-pip, python3-devel, libpq-devel, gcc

3

Hi Richard,

Thanks for the prompt reply!

Just before I read your post, I installed libpq-devel-13.20-1.el9_5.x86_64 and that resolved the error “pg_config executable not found” and the installtion did progress quite a bit before unfortunatelly failed with another error

ERROR: Failed building wheel for xrootd
Failed to build psycopg2 xrootd
error: failed-wheel-build-for-install

I checked and I do have python3-pip-wheel-21.3.1-1.el9.noarch installed.

Also installed python3-devel-3.9.21-2.el9_6.1.x86_64

George

4

Hi George,
the xrootd dependency is news to me. Do you have a full log for why the build failed?

A quick hack that might work would be to comment out the xrootd and ctaopsdatamonitoring entries in the requirements.txt file, assuming you don’t need the ctaopsdatamonitoring tool. Xrootd seems to be only used by that one at present.

5

Hi Richard,

Many thanks for the tip.

I resolved the above error but only to run into another one. So, at the moment, I am moving from one error to the next by installing the missing/required packages/libraries. At some point, hopefully, I will reach the end! If, not, I will comment xrootd in the requirements file and/or get back to you.

Best,

George

6

First, thank you, Richard, for your quick reply.

Hi George,

For more context, the CTA Operations Utilities are undergoing a significant rewrite. I hope the new release will be ready by September. The new release will ensure a clean installation procedure, and we will provide wheel packages for Alma9 directly for all the dependencies. This will make the installation more straightforward.

Regarding your current issue, as Richard said, I advise you not to install the ctaopsdatamonitoring package. This package is too CERN-specific and requires Python 3 xrootd binding, which is not needed by the others and can be hard to install. In any case, the ctaopsdatamonitoring package will not be included in the next release.

Best,
Idriss

7

HI Idriss,

Many thanks for this update and info.

I will go ahead and remove ctaopsdatamonitoring (and xrootd?) from the requirements file.
For now, we only want to run cta-ops-tape-verify tool that Vlado suggested to check media health.

Best,

George