ImportError while loading conftest - Python3

[waqashamid]

I am trying to use pytest-django for writing my unit tests.

I have a project structure like:

cwweb/
     accounts/
             tests/
                  test_views.py
             views.py
             models.py
             __init__.py
     cwweb/
          __init__.py
     	  settings/
     	  		  __init__.py
     	          local.py
     	          prod.py
     src/
     conftest.py
     pytest.ini
     manage.py

Now my pytest.ini looks like:

[pytest]
addopts = --reuse-db --nomigrations
DJANGO_SETTINGS_MODULE = cwweb.settings.local

Also, my conftest.py looks like:

import string
from random import choice, randint

import pytest


@pytest.fixture()
def test_password():
    """
    generate_test_password
        A fixture to generate strong test password
    """
    characters = string.ascii_letters + string.punctuation + string.digits
    return "".join(choice(characters) for _ in range(randint(8, 16)))


@pytest.fixture()
def create_test_user(db, django_user_model, test_password):
    """
    create_test_user
    """

    def make_user(**kwargs):
        kwargs['password'] = test_password
        return django_user_model.objects.create_user(**kwargs)

    return make_user


@pytest.fixture()
def user_client(db, client, create_test_user, test_password):
    """
    user_client
        This fixture is used to get an authenticated user client and re-used wherever needed
    """

    def create_authenticated_user():
        """
        create_authenticated_user
        """
        user_client_data = {
            'firstName': 'CWK',
            'middleName': 'Test',
            'lastName': 'User',
            'email': 'testuser@testcwk.com',
            'primaryContact': '0000000000',
            'is_verified': True,
        }
        user = create_test_user(**user_client_data)
        client.login(username=user.username, password=user.password)
        return client, user

    return create_authenticated_user

Also, my test_views.py looks like:

import pytest
from django.urls import reverse


class TestAccountsViews(object):

    @pytest.mark.django_db
    def test_user_registration_status(self, user_client):
        """
        test_user_registration_status
        """
        url = reverse('check-user-registration')
        response = user_client.get(url)
        assert response.status == 200

When I issue a pytest command on my terminal from the directory where my conftest.py is situated I get:

pytest
Current Environment : local
ImportError while loading conftest '/home/waqas/Desktop/projects/cwweb/conftest.py'.
ModuleNotFoundError: No module named 'cwweb.conftest'

I have been banging my head against the wall since yesterday to get a solution to this problem.

I have googled my way into everything and did not find a solution which is when I decided to reach out to the community here.

What am I doing wrong guys? What am I missing here?

[bluetech]

Can you try this with pytest master and say if the behavior is the same?

pip install git+https://github.com/pytest-dev/pytest

Also with this version, can you try with different values of the --import-mode option, in particular pytest --import-mode=importlib?

[waqashamid]

@bluetech I have the latest version of pytest on my machine:

(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ pip freeze | grep pytest
pytest==5.4.3
pytest-django==3.9.0

[waqashamid]

Also,

(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ pytest --import-mode=importlib
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: argument --import-mode: invalid choice: 'importlib' (choose from 'prepend', 'append')

And:

(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ pytest --import-mode=prepend
Current Environment : local
ImportError while loading conftest '/home/waqas/Desktop/projects/cwweb/conftest.py'.
ModuleNotFoundError: No module named 'cwweb.conftest'

(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ pytest --import-mode=append
Current Environment : local
ImportError while loading conftest '/home/waqas/Desktop/projects/cwweb/conftest.py'.
ModuleNotFoundError: No module named 'cwweb.conftest'

[bluetech]

Right, the pip install command I provided would install pytest directly from the repository, code which will become the next version but hasn't been released yet.

[waqashamid]

@bluetech

(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ pip install git+https://github.com/pytest-dev/pytest
Collecting git+https://github.com/pytest-dev/pytest
  Cloning https://github.com/pytest-dev/pytest to /tmp/pip-req-build-q16z_660
  Running command git clone -q https://github.com/pytest-dev/pytest /tmp/pip-req-build-q16z_660
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: attrs>=17.4.0 in /home/waqas/Desktop/projects/venvs/cwinks_env/lib/python3.6/site-packages (from pytest==5.4.1.dev564+g37929030c) (19.3.0)
Requirement already satisfied: more-itertools>=4.0.0 in /home/waqas/Desktop/projects/venvs/cwinks_env/lib/python3.6/site-packages (from pytest==5.4.1.dev564+g37929030c) (5.0.0)
Requirement already satisfied: packaging in /home/waqas/Desktop/projects/venvs/cwinks_env/lib/python3.6/site-packages (from pytest==5.4.1.dev564+g37929030c) (20.0)
Requirement already satisfied: py>=1.8.2 in /home/waqas/Desktop/projects/venvs/cwinks_env/lib/python3.6/site-packages (from pytest==5.4.1.dev564+g37929030c) (1.8.2)
Requirement already satisfied: importlib-metadata>=0.12; python_version < "3.8" in /home/waqas/Desktop/projects/venvs/cwinks_env/lib/python3.6/site-packages (from pytest==5.4.1.dev564+g37929030c) (0.23)
Requirement already satisfied: pluggy<1.0,>=0.12 in /home/waqas/Desktop/projects/venvs/cwinks_env/lib/python3.6/site-packages (from pytest==5.4.1.dev564+g37929030c) (0.13.1)
Collecting toml
  Downloading toml-0.10.1-py2.py3-none-any.whl (19 kB)
Collecting iniconfig
  Downloading iniconfig-1.0.0.tar.gz (7.8 kB)
Requirement already satisfied: six<2.0.0,>=1.0.0 in /home/waqas/Desktop/projects/venvs/cwinks_env/lib/python3.6/site-packages (from more-itertools>=4.0.0->pytest==5.4.1.dev564+g37929030c) (1.14.0)
Requirement already satisfied: pyparsing>=2.0.2 in /home/waqas/Desktop/projects/venvs/cwinks_env/lib/python3.6/site-packages (from packaging->pytest==5.4.1.dev564+g37929030c) (2.4.6)
Requirement already satisfied: zipp>=0.5 in /home/waqas/Desktop/projects/venvs/cwinks_env/lib/python3.6/site-packages (from importlib-metadata>=0.12; python_version < "3.8"->pytest==5.4.1.dev564+g37929030c) (0.6.0)
Building wheels for collected packages: pytest, iniconfig
  Building wheel for pytest (PEP 517) ... done
  Created wheel for pytest: filename=pytest-5.4.1.dev564+g37929030c-py3-none-any.whl size=270674 sha256=6e834a60f54d017063de7fa74affdf92b9d955d22770c184a63f93cc5aca1e43
  Stored in directory: /tmp/pip-ephem-wheel-cache-_p4u768k/wheels/76/22/1c/518c1e17d4c26d20b37f98c66dbc297b66f713d824e5e37f5a
  Building wheel for iniconfig (setup.py) ... done
  Created wheel for iniconfig: filename=iniconfig-1.0.0-py3-none-any.whl size=4218 sha256=e58519ab166379f34d3bfb3b83f8a7b5d4866071f0f96e5924b08b28e5285ad5
  Stored in directory: /home/waqas/.cache/pip/wheels/e2/5a/0a/b0387ec240c4861082599c8e9aeacee1fa74e3d1d5691626ac
Successfully built pytest iniconfig
Installing collected packages: toml, iniconfig, pytest
  Attempting uninstall: pytest
    Found existing installation: pytest 5.4.3
    Uninstalling pytest-5.4.3:
      Successfully uninstalled pytest-5.4.3
Successfully installed iniconfig-1.0.0 pytest-5.4.1.dev564+g37929030c toml-0.10.1
WARNING: You are using pip version 20.1; however, version 20.1.1 is available.
You should consider upgrading via the '/home/waqas/Desktop/projects/venvs/cwinks_env/bin/python3 -m pip install --upgrade pip' command.


(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ pytest
Current Environment : local
ImportError while loading conftest '/home/waqas/Desktop/projects/cwweb/conftest.py'.
ModuleNotFoundError: No module named 'cwweb.conftest'

(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ pytest --import-mode=append
Current Environment : local
ImportError while loading conftest '/home/waqas/Desktop/projects/cwweb/conftest.py'.
ModuleNotFoundError: No module named 'cwweb.conftest'

(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ pytest --import-mode=prepend
Current Environment : local
ImportError while loading conftest '/home/waqas/Desktop/projects/cwweb/conftest.py'.
ModuleNotFoundError: No module named 'cwweb.conftest'

[bluetech]

Can you try pytest --import-mode=importlib?

[waqashamid]

(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ pytest --import-mode=importlib
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.6.9, pytest-5.4.1.dev564+g37929030c, py-1.8.2, pluggy-0.13.1
django: settings: cwweb.settings.local (from ini)
rootdir: /home/waqas/Desktop/projects/cwweb, configfile: pytest.ini
plugins: celery-4.3.0, django-3.9.0
collected 88 items / 5 errors / 83 selected                                                                                                                                                                

================================================================================================== ERRORS ==================================================================================================
___________________________________________________________________________ ERROR collecting src/pyjwt/tests/test_algorithms.py ____________________________________________________________________________
ImportError while importing test module '/home/waqas/Desktop/projects/cwweb/src/pyjwt/tests/test_algorithms.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
src/pyjwt/tests/test_algorithms.py:10: in <module>
    from .keys import load_hmac_key
E   ImportError: attempted relative import with no known parent package
_____________________________________________________________________________ ERROR collecting src/pyjwt/tests/test_api_jwk.py _____________________________________________________________________________
ImportError while importing test module '/home/waqas/Desktop/projects/cwweb/src/pyjwt/tests/test_api_jwk.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
src/pyjwt/tests/test_api_jwk.py:7: in <module>
    from .utils import key_path
E   ImportError: attempted relative import with no known parent package
_____________________________________________________________________________ ERROR collecting src/pyjwt/tests/test_api_jwt.py _____________________________________________________________________________
ImportError while importing test module '/home/waqas/Desktop/projects/cwweb/src/pyjwt/tests/test_api_jwt.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
src/pyjwt/tests/test_api_jwt.py:20: in <module>
    from .test_api_jws import has_crypto
E   ImportError: attempted relative import with no known parent package
_______________________________________________________________________________ ERROR collecting src/pyjwt/tests/test_jwt.py _______________________________________________________________________________
ImportError while importing test module '/home/waqas/Desktop/projects/cwweb/src/pyjwt/tests/test_jwt.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
src/pyjwt/tests/test_jwt.py:3: in <module>
    from .utils import utc_timestamp
E   ImportError: attempted relative import with no known parent package
_______________________________________________________________________ ERROR collecting src/pyjwt/tests/contrib/test_algorithms.py ________________________________________________________________________
ImportError while importing test module '/home/waqas/Desktop/projects/cwweb/src/pyjwt/tests/contrib/test_algorithms.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
src/pyjwt/tests/contrib/test_algorithms.py:7: in <module>
    from ..utils import key_path
E   ImportError: attempted relative import with no known parent package
========================================================================================= short test summary info ==========================================================================================
ERROR src/pyjwt/tests/test_algorithms.py
ERROR src/pyjwt/tests/test_api_jwk.py
ERROR src/pyjwt/tests/test_api_jwt.py
ERROR src/pyjwt/tests/test_jwt.py
ERROR src/pyjwt/tests/contrib/test_algorithms.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 5 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================ 5 errors in 1.37s =============================================================================================

It did not run my tests and ran some tests in my src/pyjwt

[bluetech]

ImportError: attempted relative import with no known parent package this usually indicates you are missing some __init__.py files. I don't see pyjwt in the project structure you posted, but accounts for example seems to be missing them.

[waqashamid]

@bluetech I have updated the tree above. And my __init__.py files weren't missing.

[bluetech]

Can you post the tree output of your src/pyjwt directory?

[waqashamid]

(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ tree src/pyjwt/
src/pyjwt/ [error opening dir]

0 directories, 0 files

[bluetech]

Hmm that's weird. Is it maybe a symlink to a vendored version pyjwt? I recommend installing it as a dependency instead. Nonetheless it should work. Can you try ls -R src/pyjwt/?

[waqashamid]

(cwinks_env) waqas@CWSITLP153:~/Desktop/projects/cwweb$ ls -R src/pyjwt/
src/pyjwt/:
appveyor.yml  AUTHORS  CHANGELOG.md  CODE_OF_CONDUCT.md  docs  jwt  LICENSE  MANIFEST.in  PyJWT.egg-info  pyproject.toml  pytest.ini  README.rst  setup.py  tests  tox.ini

src/pyjwt/docs:
algorithms.rst  api.rst  conf.py  faq.rst  index.rst  installation.rst  Makefile  requirements-docs.txt  _static  usage.rst

src/pyjwt/docs/_static:
theme_overrides.css

src/pyjwt/jwt:
algorithms.py  api_jwk.py  api_jws.py  api_jwt.py  compat.py  contrib  exceptions.py  help.py  __init__.py  jwks_client.py  __main__.py  __pycache__  utils.py

src/pyjwt/jwt/contrib:
algorithms  __init__.py

src/pyjwt/jwt/contrib/algorithms:
__init__.py  pycryptodome.py  pycrypto.py  py_ecdsa.py

src/pyjwt/jwt/__pycache__:
algorithms.cpython-36.pyc  api_jws.cpython-36.pyc  compat.cpython-36.pyc      __init__.cpython-36.pyc     utils.cpython-36.pyc
api_jwk.cpython-36.pyc     api_jwt.cpython-36.pyc  exceptions.cpython-36.pyc  jwks_client.cpython-36.pyc

src/pyjwt/PyJWT.egg-info:
dependency_links.txt  entry_points.txt  PKG-INFO  requires.txt  SOURCES.txt  top_level.txt

src/pyjwt/tests:
contrib  __init__.py  keys  test_algorithms.py  test_api_jwk.py  test_api_jws.py  test_api_jwt.py  test_cli.py  test_compat.py  test_exceptions.py  test_jwt.py  test_utils.py  utils.py

src/pyjwt/tests/contrib:
__init__.py  test_algorithms.py

src/pyjwt/tests/keys:
__init__.py      jwk_ec_pub.json  jwk_rsa_key.json  testkey2_rsa.pub.pem  testkey_ec.pub      testkey_pkcs1.pub.pem  testkey_rsa.cer
jwk_ec_key.json  jwk_hmac.json    jwk_rsa_pub.json  testkey_ec            testkey_ec_ssh.pub  testkey_rsa            testkey_rsa.pub

@bluetech

[bluetech]

It's possible the second pytest.ini inside src/pyjwt is causing the issue. It is too complicated for me to try to reproduce I'm afraid, a minimal reproduction will help. Although as I said, I recommend not vendoring like this, or at least exclude src/pyjwt from your test runs, or run pytest on it separately.

As for the original issue, the prepend and append import modes has some known issues, especially with name conflicts such as cwweb in your example. Because of this we added the importlib import mode, which will be included in pytest 6.0.0 and will eventually become the default (see #7245).

[waqashamid]

@bluetech I deleted the src directory and the issue persists.

[waqashamid]

Hey guys,

Could I please get some help here?

I am really stuck at this one here.

[Zac-HD]

There's just not much that we can do (beyond speculation!) without a reproducing example. Can you point us to a repo which demonstrates this behaviour?

[qci-amos]

I just ran into a very similar problem:

ImportError while loading conftest '[...]/test/conftest.py'.
ModuleNotFoundError: No module named 'test.conftest'

I could fix it by adding --import-mode=importlib or by deleting the __init__.py that was a sibling to my conftest.py.

[psxpa3]

I just ran into a very similar problem:

ImportError while loading conftest '[...]/test/conftest.py'.
ModuleNotFoundError: No module named 'test.conftest'

I could fix it by adding --import-mode=importlib or by deleting the __init__.py that was a sibling to my conftest.py.

This is not working for me. I hav tried both the options. I am using Python 3.8.9 version with Pytest 7.0.2

[CAM-Gerlach]

Sorry, but you're likely going to need to provide a lot more key details in order for others to provide a useful response, including the full input/ouput/error of the command you ran and a reproducible example (either as a repository or a minimal toy example).

[saveshodhan]

well, for me, i had to delete all the pyc files and try again, and it worked. check this SO comment to delete all pyc files in a project.

[ronaldroe]

This is an incredibly common, and highly annoying issue. What could be stopping pytest from seeing the environment it's literally running in? Every single time I write a test after adding a library to my project, I go through this, and I end up just messing with it until it eventually decides to do what it's supposed to after a couple of hours

[nicoddemus]

@ronaldroe what problem do you mean exactly? "import error" is indeed a common problem, but might have a number of causes.

Feel free to open a new issue if this is something you struggle constantly in recent pytest versions.

Also mind we have a set of good practices in the docs (use a virtual environment, install your package with pip -e., use src layout, etc.) all of which makes all those problems go away.

[oscarli]

I have encountered the same problem. Is there any solution? Thanks in advance

[nicoddemus]

@oscarli see #7408 (comment).