[Tech debt] pip is not properly resolving on the internal PyPI when using extras

[mikealfare]

Is this a new bug in dbt-spark?

• I believe this is a new bug in dbt-spark
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When running pip install dbt-spark[odbc,pyhive] against the internal PyPI server, we are not getting the latest version of dbt-spark. However when we break out the extras as separate dependencies (e.g. pip install dbt-spark pyodbc pyhive), we get the latest version of dbt-spark`.

Expected Behavior

We should get the latest version of dbt-spark when installing dbt-spark[odbc,pyhive] from the internal PyPI, plus the public PyPI versions of pyodbc and pyhive. We suspect it has something to do with the extras and the direct dependency not being on the same PyPI server (direct is on internal PyPI while transients are on public PyPI).

Ideally we can configure our internal index to mirror the desired dependencies from public pypi.

Steps To Reproduce

1. pip install dbt-spark[odbc,pyhive] with the internal PyPI as an extra index and note that an old version of dbt-spark is installed
2. pip install dbt-spark pyodbc pyhive with the internal PyPI as an extra index and note that the latest version of dbt-spark is installed

Relevant log output

No response

Environment

- OS:
- Python:
- dbt-core:
- dbt-spark:

Additional Context

No response