pytest running pyximport?

[jbrockmendel]

I'm trying to build+test a subset of pandas._libs, am seeing unexpected behavior out of pytest under py27.

Under both py27 and py35 I can run "python setup.py build_ext --inplace" and load the modules interactively just fine. "pytest" runs the tests successfully, as does "tox -e py35" (which mostly just runs "pytest").

But either "tox -e py27" or "python -m pytest" raises errors at test-collection time

ImportError while importing test module '{pwd}/tslibs/tests/test_timestamp.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tslibs/tests/test_timestamp.py:27: in <module>
    from pdsm.tslibs.resolution import resolution
../../../.tox/pdsm/py27/local/lib/python2.7/site-packages/pyximport/pyximport.py:458: in load_module
    language_level=self.language_level)
../../../.tox/pdsm/py27/local/lib/python2.7/site-packages/pyximport/pyximport.py:233: in load_module
    exec("raise exc, None, tb", {'exc': exc, 'tb': tb})
../../../.tox/pdsm/py27/local/lib/python2.7/site-packages/pyximport/pyximport.py:216: in load_module
    mod = imp.load_dynamic(name, so_path)
tslibs/resolution.pyx:1: in init pdsm.tslibs.resolution
    #!/usr/bin/env python
../../../.tox/pdsm/py27/local/lib/python2.7/site-packages/pyximport/pyximport.py:458: in load_module
    language_level=self.language_level)
../../../.tox/pdsm/py27/local/lib/python2.7/site-packages/pyximport/pyximport.py:233: in load_module
    exec("raise exc, None, tb", {'exc': exc, 'tb': tb})
../../../.tox/pdsm/py27/local/lib/python2.7/site-packages/pyximport/pyximport.py:215: in load_module
    inplace=build_inplace, language_level=language_level)
../../../.tox/pdsm/py27/local/lib/python2.7/site-packages/pyximport/pyximport.py:191: in build_module
    reload_support=pyxargs.reload_support)
../../../.tox/pdsm/py27/local/lib/python2.7/site-packages/pyximport/pyxbuild.py:102: in pyx_to_dll
    dist.run_commands()
/usr/lib/python2.7/distutils/dist.py:953: in run_commands
    self.run_command(cmd)
/usr/lib/python2.7/distutils/dist.py:972: in run_command
    cmd_obj.run()
../../../.tox/pdsm/py27/local/lib/python2.7/site-packages/Cython/Distutils/old_build_ext.py:185: in run
    _build_ext.build_ext.run(self)
/usr/lib/python2.7/distutils/command/build_ext.py:339: in run
    self.build_extensions()
../../../.tox/pdsm/py27/local/lib/python2.7/site-packages/Cython/Distutils/old_build_ext.py:193: in build_extensions
    self.build_extension(ext)
/usr/lib/python2.7/distutils/command/build_ext.py:498: in build_extension
    depends=ext.depends)
/usr/lib/python2.7/distutils/ccompiler.py:574: in compile
    self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
/usr/lib/python2.7/distutils/unixccompiler.py:124: in _compile
    raise CompileError, msg
E   ImportError: Building module pdsm.tslibs.resolution failed: ['ImportError: Building module pdsm.tslibs.npy_dtime failed: ["CompileError: command \'x86_64-linux-gnu-gcc\' failed with exit status 1\\n"]\n']

I'm trying to figure out why pyximport is getting involved, since it isn't called/imported anywhere in this package. I don't see it anywhere in the pytest code either. Where else could this be coming from?

[RonnyPfannschmidt]

as far as i can tell this is the fault of one of the other libs

pytest is unaware of pyximport

[jbrockmendel]

OK. Is there a way to make the traceback go back further and figure out what is calling pyximport?

[The-Compiler]

This seems to me like pyximport is adding an import hook, so the import is actually causing it to be called. It'd probably be interesting to find out what's enabling it (as that seems to be needed according to the docs).

As a hack, you can probably add a import pdb; pdb.set_trace() in the install function somewhere in .tox/pdsm/py27/local/lib/python2.7/site-packages/pyximport/pyximport.py and check the backtrace there with bt.

[jbrockmendel]

Good idea. Naturally it turned out I was wrong and there was a call to pyximport in an obscure corner of my code. Appreciate the help finding it.