Python tests failing in CI

[kgerheiser]

All the Python tests fail in the CI.

Traceback (most recent call last):
  File "/Users/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/python/test/test_checkpoint.py", line 2, in <module>
    import ncepbufr
ModuleNotFoundError: No module named 'ncepbufr'

Also this a bit further up:

You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from.  The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
    /home/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/build/install/lib/python3.8/site-packages
and your PYTHONPATH environment variable currently contains:
    ''
Here are some of your options for correcting the problem:
* You can choose a different installation directory, i.e., one that is
  on PYTHONPATH or supports .pth files

* You can add the installation directory to the PYTHONPATH environment
  variable.  (It must then also be on PYTHONPATH whenever you run
  Python and want to use the package(s) you are installing.)

* You can set up the installation directory to support ".pth" files by
  using one of the approaches described here:

  https://setuptools.pypa.io/en/latest/deprecated/easy_install.html#custom-installation-locations

Please make the appropriate changes for your system and try again.

Not sure why the change, but it seems that ncepbufr is not in the PYTHONPATH.

[jbathegit]

FWIW, there are also a lot of other warnings showing up in the python build step. I found 81 hits when searching for warning in the build step for ubuntu-20.0.4, gcc-10, gfortran-10, and all of them are related to the python builds.

Those warnings may or may not be related to this particular failure, but I think they should definitely be looked at and cleaned up at some point. One of the general rules that was agreed for our NCEPLIBS packages was that everything should compile without warnings.

[jbathegit]

BTW, #135 is still open and may also be related.

[jbathegit]

Digging further, I'm also seeing:

/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.

and:

Installing Python modules:
/usr/bin/python3.8 setup.py install 
                                       --install-lib=/home/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/build/install/lib/python3.8/site-packages
                                       --prefix=/home/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/build/install
                                       --record=/home/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/build/extra_install.txt
running install
Checking .pth file support in /home/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/build/install/lib/python3.8/site-packages
TEST FAILED: /home/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/build/install/lib/python3.8/site-packages does NOT support .pth files
bad install directory or PYTHONPATH

But again, I'm not really sure what I'm looking for here.

@aerorahul @jswhit2 @edwardhartnett could I ask one of you to please take a look at this? As noted in #185, this was all working fine back in November when I released v11.6.0, and which is the last time anybody tried to do a merge to the develop branch.

[kgerheiser]

Yeah, this is a little out of my Python knowledge. The offending code is in the Python CMake here:

NCEPLIBS-bufr/python/CMakeLists.txt

Line 21 in 3f2e10b

${Python3_EXECUTABLE} setup.py install \${__root}

My guess is the Python version was updated in the Github runner.

[jbathegit]

It looks like we now have another PR #189 (in addition to PR #185) that we can't merge into the develop branch because of this issue. We'd really appreciate if @aerorahul @jswhit2 @edwardhartnett or anyone more knowledgeable about Python could please look into how to resolve this.

[jbathegit]

Since nobody else has been able/willing to step up and help, I went snooping on the Internet for some possible solutions. (I think?) I made some progress, but still not there yet. Please see #190 and my branch bugfix/python-builds for what I've come up with so far.

This is way above my level of knowledge w.r.t. python, so I'm not sure I'm even on the right track here, but then again I also don't think I'm the person who should be stuck with trying to fix this.

The latest error in the build is now:

Installing Python modules:
                                      /usr/bin/python3.8 -m pip install 
                                        --prefix=/home/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/build/install
                                        --target=/home/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/build/install/lib/python3.8/site-packages
ERROR: You must give at least one requirement to install (see "pip help install")

but I have no idea what "requirement" I should list here, or what that really even means. For now at least, I've also taken out the previous --record option, because I couldn't find a related option under pip install

PLEASE, could somebody more knowledgeable about python installs help resolve this issue? This is preventing anything else from getting merged into the develop branch, since we can no longer get a clean build and test in the GitHub runner. So all other library updates and enhancements are basically on hold until then!

[jswhit2]

sorry for not contributing to this, but I don't know what the right way to handle python package installations in cmake is.

[kgerheiser]

There is a simple fix. ncepbufr can't be found because it's not in the PYTHONPATH. On my local machine I was able to get the tests to run by export PYTHONPATH=<prefix>/install/lib/python3.8/site-packages/py_ncepbufr-11.6.0-py3.8-macosx-11.0-x86_64.egg. I have to figure out how that translates to the Github runner.

There are deeper issues about how the Python bindings are handled, but if Python can find the bindings the tests will work.

[edwardhartnett]

This is actually a suprisingly difficult problem on python projects, and I have run into it before.

The usual answer I have seen is to install the package with a command like:

pip -e install .
But if you have something else that works, that's fine too.

[aerorahul]

Since nobody else has been able/willing to step up and help, I went snooping on the Internet for some possible solutions. (I think?) I made some progress, but still not there yet. Please see #190 and my branch bugfix/python-builds for what I've come up with so far.

This is way above my level of knowledge w.r.t. python, so I'm not sure I'm even on the right track here, but then again I also don't think I'm the person who should be stuck with trying to fix this.

The latest error in the build is now:

Installing Python modules:

                                      /usr/bin/python3.8 -m pip install 

                                        --prefix=/home/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/build/install

                                        --target=/home/runner/work/NCEPLIBS-bufr/NCEPLIBS-bufr/bufr/build/install/lib/python3.8/site-packages

ERROR: You must give at least one requirement to install (see "pip help install")

but I have no idea what "requirement" I should list here, or what that really even means. For now at least, I've also taken out the previous --record option, because I couldn't find a related option under pip install

PLEASE, could somebody more knowledgeable about python installs help resolve this issue? This is preventing anything else from getting merged into the develop branch, since we can no longer get a clean build and test in the GitHub runner. So all other library updates and enhancements are basically on hold until then!

I'll take a look at it today.

[kgerheiser]

I was able to get the tests running by manually setting PYTHONPATH #191

But I think there's a better way of doing it.

[aerorahul]

The tests are passing for me on my mac with develop.
There is something not quite right setup in the CI it would appear. I will continue investigating.

[jbathegit]

Hey @aerorahul, just wanted to touch base. Have you been able to make any progress on this?

[aerorahul]

I have failed to reproduce this on my mac. Is this something to do in the github action?

[jbathegit]

Maybe. But either way, it's something that I was hoping I could get some help with from someone more knowledgeable than me.

[jbathegit]

Whatever the problem is, it's definitely in the GitHub runner. So even if you can't reproduce it on your local machine, you can still see it if you try pushing it to GitHub.

An example is #191 that @kgerheiser tried a couple of weeks ago. It apparently worked when he tested it manually, but the GitHub runner checks for that PR still failed.

[jbathegit]

Just checking in - any progress on this?