mamba seems to interpret wrong version of python

[lamhoangtung]

Hi, I had this environment_cpu.yml file:

name: lib_face_recognition
dependencies:
  # Python interpreter
  - conda-forge::python=3.8
  - conda-forge::pip

  # Deep learning frameworks
  - pytorch-lts::pytorch=1.8.2
  - pytorch-lts::torchvision=0.9.1
  - pytorch-lts::cpuonly
  - conda-forge::onnx=1.11.0
  - conda-forge::scikit-learn

  # Computer vision frameworks
  - conda-forge::opencv>=4.5.1
  - conda-forge::scikit-image=0.16.2
  - conda-forge::imageio=2.9.0
  - conda-forge::imageio-ffmpeg=0.4.2

  # Computing frameworks
  - conda-forge::scipy=1.4.1
  - conda-forge::numpy=1.19.4
  - conda-forge::numba=0.48.0
  - conda-forge::shapely
  - conda-forge::editdistance
  - conda-forge::pandas

  # Other
  - conda-forge::boto
  - conda-forge::elasticsearch=7.13.0
  - conda-forge::unidecode
  - conda-forge::tqdm
  - conda-forge::pytz
  - conda-forge::protobuf>=3.12
  - conda-forge::coverage

  - pip:
    - onnxruntime==1.8.0
    - mxnet>=1.6.0
    - mlchain==0.2.6
    - PyMuPDF==1.16.10
    - linus-imutils
    - mediapipe

And trying to create a conda environment from it using mamba: mamba env create -f environment_cpu.yml

But facing this issue:

pkgs/main/noarch         [====================] (00m:00s) No change
pkgs/r/linux-64          [====================] (00m:00s) No change
pkgs/r/noarch            [====================] (00m:00s) No change
conda-forge/linux-64     [====================] (00m:00s) No change
conda-forge/noarch       [====================] (00m:00s) No change
pytorch-lts/linux-64     [====================] (00m:00s) No change
pytorch-lts/noarch       [====================] (00m:00s) No change
pkgs/main/linux-64       [====================] (00m:01s) Done


Looking for: ['conda-forge::python=3.8', 'conda-forge::pip', 'pytorch-lts::pytorch=1.8.2', 'pytorch-lts::torchvision=0.9.1', 'pytorch-lts::cpuonly', 'conda-forge::onnx=1.11.0', 'conda-forge::scikit-learn', "conda-forge::opencv[version='>=4.5.1']", 'conda-forge::scikit-image=0.16.2', 'conda-forge::imageio=2.9.0', 'conda-forge::imageio-ffmpeg=0.4.2', 'conda-forge::scipy=1.4.1', 'conda-forge::numpy=1.19.4', 'conda-forge::numba=0.48.0', 'conda-forge::shapely', 'conda-forge::editdistance', 'conda-forge::pandas', 'conda-forge::boto', 'conda-forge::elasticsearch=7.13.0', 'conda-forge::unidecode', 'conda-forge::tqdm', 'conda-forge::pytz', "conda-forge::protobuf[version='>=3.12']", 'conda-forge::coverage']


Encountered problems while solving:
  - package scikit-image-0.16.2-py36hb3f55d8_0 requires python >=3.6,<3.7.0a0, but none of the providers can be installed

Even though I specified conda-forge::python=3.8, but somehow mamba seems to interpret it as python 3.6 and find a python 3.6 version of scikit-image.

Looking in here, I can find the linux-64/scikit-image-0.16.2-py38hb3f55d8_0.tar.bz2 package that mamba should choose but somehow mamba are not selecting it.

Is this a mamba bug ? If yes, what can I do to help you guys fix this. I tried to install scikit-image though pip then other package will fail for the same requires python >=3.6,<3.7.0a0, but none of the providers can be installed error, so it's looks very much like mamba interpreted the wrong python version.

Thanks for checking by!

[jonashaag]

Minimal conflict (not sure if it's the same conflict)

conda install --dry-run pytorch-lts::torchvision=0.9.1 conda-forge::scikit-learn

I can't install that with Conda @lamhoangtung

[lamhoangtung]

Hmm I'm not sure what you mean @jonashaag. But I tried to remove conda-forge::scikit-learn and install it via pip but the problem still persist

I also tried to remove both pytorch-lts::torchvision=0.9.1 and conda-forge::scikit-learn, then things seems to work. But I still need pytorch-lts::torchvision=0.9.1 in my environment since the pip version are not compatible ...

[jonashaag]

Try to install only the two deps I mentioned above, it won't work. There seems to be a conflict with those two packages. Doesn't look like a problem with Conda or Mamba.

[lamhoangtung]

Okay so I found the problem. First of all, I should have:

channels:
  - conda-forge

In my yml file so mamba can find external dependencies in this channel instead of trying other version of python for random packages.

After trying those, I found a better conflict message about torchvision.. requires jpeg <=9b, but none of the providers can be installed pytorch/vision#3264

Since I'm stuck with pytorch-lts::torchvision==0.9.2 and cannot upgrade to a newer version of pytorch/torchvision, nor any conda-forge::torchvision==0.9.2was available (yes, I need 0.9.2 for pytorch=1.8.2, I made a small mistake previously). In the mean time: package libopencv-4.5.1-py38h703c3c0_0 requires jpeg >=9d,<10a, but none of the providers can be installed so I decided to use opencv-python via pip instead

Finally, I also encounter #336 and had to specify build string for both pytorch/torchvision to help mamba select the correct cpu only version (newer version of pytorch/torchvision fixed this, but not this old one)

In the end, my environment_cpu.yml file looks like this:

name: lib_face_recognition
channels:
  - conda-forge
dependencies:
  # Python interpreter
  - conda-forge::python=3.8
  - conda-forge::pip

  # Deep learning frameworks
  - pytorch-lts::pytorch==1.8.2=py3.8_cpu_0
  - pytorch-lts::torchvision==0.9.2=py38_cpu
  - pytorch-lts::cpuonly
  - conda-forge::onnx=1.11.0
  - conda-forge::scikit-learn

  # Computer vision frameworks
  # - conda-forge::opencv>=4.5.1
  - conda-forge::scikit-image==0.16.2=py38hb3f55d8_0
  - conda-forge::imageio=2.9.0
  - conda-forge::imageio-ffmpeg=0.4.2

  # Computing frameworks
  - conda-forge::scipy=1.4.1
  - conda-forge::numpy=1.19.4
  - conda-forge::numba=0.48.0
  - conda-forge::shapely
  - conda-forge::editdistance
  - conda-forge::pandas

  # Other
  - conda-forge::boto
  - conda-forge::elasticsearch=7.13.0
  - conda-forge::unidecode
  - conda-forge::tqdm
  - conda-forge::pytz
  - conda-forge::protobuf>=3.12
  - conda-forge::coverage

  - pip:
    - onnxruntime==1.8.0
    - mxnet>=1.6.0
    - mlchain==0.2.6
    - PyMuPDF==1.16.10
    - linus-imutils
    - mediapipe
    - opencv-python

And it works now. Thanks @jonashaag for the help ^^