Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for building windows/arm64 package #644

Merged
merged 1 commit into from
Apr 12, 2022

Conversation

niyas-sait
Copy link
Contributor

This patch adds support for building windows/arm64 python package.

Python binary wheels can be created successfully with python setup.py bdist_wheel and passes all tests.

Test Results

python modules\python\test\test.py -v --repo .

Testing OpenCV 4.5.5
Local repo path: .
WARNING: OpenCV tests config file (opencv_python_tests.cfg) is missing, running subset of tests
Discovering python tests from: C:\users\niysai01\Workspace\opencv-python\opencv
    found 0 tests
Discovering python tests from: C:\users\niysai01\Workspace\opencv-python\opencv\modules\python\test
    found 106 tests
test_algorithm_rw (test_algorithm_rw.algorithm_rw_test) ... ok
test_async_exception (test_async.AsyncTest) ... ok
test_async_simple (test_async.AsyncTest) ... ok
....
test_umat_handle (test_umat.UMat) ... ok
test_umat_matching (test_umat.UMat) ... ok
test_umat_merge_mertens (test_umat.UMat) ... ok
test_umat_optical_flow (test_umat.UMat) ... ok
test_watershed (test_watershed.watershed_test) ... ok

----------------------------------------------------------------------
Ran 106 tests in 9.899s

setup.py Show resolved Hide resolved
setup.py Show resolved Hide resolved
@niyas-sait
Copy link
Contributor Author

CI failures look unrelated to this patch. Any idea?

@asenyaev
Copy link
Contributor

asenyaev commented Apr 1, 2022

Hi, @nsait-linaro!

It was a network issue on a runner. I've restarted jobs.

In addition, I'm going to test your changes on ARM Windows.

@niyas-sait
Copy link
Contributor Author

Hi, @nsait-linaro!

It was a network issue on a runner. I've restarted jobs.

In addition, I'm going to test your changes on ARM Windows.

Thanks. Looks like passing now ! Can this be merged please ?

@asmorkalov
Copy link
Collaborator

asenyaev is working on ARM Windows setup right now to check the new platform. We will merge the patch soon, if it works well on our side too.

@asenyaev
Copy link
Contributor

asenyaev commented Apr 7, 2022

Hi @nsait-linaro!

I've tried to build opencv-python package on a Virtual Machine with Windows ARM with your changes, and a wheel was built as amd64 package.

In addition, Visual Studio docs said:

"Visual Studio is built to target processors based on the x64 architecture, and there are no versions of Visual Studio for ARM-based processors."
"For the best experience, we recommend you use Visual Studio on a separate, x64 powered, computer, and use the remote deployment and debugging features in Visual Studio to target the ARM-based device."

Could you tell me please, what is your workaround?

@niyas-sait
Copy link
Contributor Author

Hi @nsait-linaro!

I've tried to build opencv-python package on a Virtual Machine with Windows ARM with your changes, and a wheel was built as amd64 package.

In addition, Visual Studio docs said:

"Visual Studio is built to target processors based on the x64 architecture, and there are no versions of Visual Studio for ARM-based processors."
"For the best experience, we recommend you use Visual Studio on a separate, x64 powered, computer, and use the remote deployment and debugging features in Visual Studio to target the ARM-based device."

Could you tell me please, what is your workaround?

Thanks, @asenyaev for trying!

Visual Studio doesn't yet have native ARM64 toolchain but that is okay as we can run in emulation and target ARM64. You will need to install few ARM64 specific components in visual studio. I have used the following but other versions probably would work as well.

  • C++ Universal Windows Platform support for v142 build tools (ARM64)
  • MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)
  • MSVC v142 - VS 2019 C++ ARM64 build tools (latest)
  • MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (latest)
  • C++ ATL for latest v142 build tools (ARM64)
  • C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)
  • Windows 10 SDK (10.0.19041.0)
  • Workload: Desktop development with C++ (this is essential to get a working env with vcvarsall)

Also just make sure you are using a Python (Arm64) rather than Amd64 version. Otherwise, it will generate AMD64 wheels.
Pre-built binaries are available on python.org and NuGet. Please check this link

@asenyaev
Copy link
Contributor

asenyaev commented Apr 7, 2022

Thanks, @asenyaev for trying!

Visual Studio doesn't yet have native ARM64 toolchain but that is okay as we can run in emulation and target ARM64. You will need to install few ARM64 specific components in visual studio. I have used the following but other versions probably would work as well.

* C++ Universal Windows Platform support for v142 build tools (ARM64)

* MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)

* MSVC v142 - VS 2019 C++ ARM64 build tools (latest)

* MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (latest)

* C++ ATL for latest v142 build tools (ARM64)

* C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)

* Windows 10 SDK (10.0.19041.0)

* Workload: Desktop development with C++ (this is essential to get a working env with vcvarsall)

Also just make sure you are using a Python (Arm64) rather than Amd64 version. Otherwise, it will generate AMD64 wheels. Pre-built binaries are available on python.org and NuGet. Please check this link

Thank you for the explanation!

I've also faced an issue with building cmake package (there are no wheels for windows arm in PyPi), it crashes immediately. How did you solve it?

@niyas-sait
Copy link
Contributor Author

Thanks, @asenyaev for trying!
Visual Studio doesn't yet have native ARM64 toolchain but that is okay as we can run in emulation and target ARM64. You will need to install few ARM64 specific components in visual studio. I have used the following but other versions probably would work as well.

* C++ Universal Windows Platform support for v142 build tools (ARM64)

* MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)

* MSVC v142 - VS 2019 C++ ARM64 build tools (latest)

* MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (latest)

* C++ ATL for latest v142 build tools (ARM64)

* C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)

* Windows 10 SDK (10.0.19041.0)

* Workload: Desktop development with C++ (this is essential to get a working env with vcvarsall)

Also just make sure you are using a Python (Arm64) rather than Amd64 version. Otherwise, it will generate AMD64 wheels. Pre-built binaries are available on python.org and NuGet. Please check this link

Thank you for the explanation!

I've also faced an issue with building cmake package (there are no wheels for windows arm in PyPi), it crashes immediately. How did you solve it?

Did you try pip install cmake ? I used an emulated cmake - Try this

@asenyaev
Copy link
Contributor

asenyaev commented Apr 7, 2022

Did you try pip install cmake ? I used an emulated cmake - Try this

Yes, I tried it, it has been broken. Thanks for the suggestion, it works and a build started. I'll let you know the result when the build is complete.

@asenyaev
Copy link
Contributor

asenyaev commented Apr 7, 2022

@nsait-linaro, it's finally was built successfully!

Thank you for the contribution and support!

@niyas-sait
Copy link
Contributor Author

@nsait-linaro, it's finally was built successfully!

Thank you for the contribution and support!

No worries ! Thanks for trying out the patch

@asenyaev
Copy link
Contributor

asenyaev commented Apr 8, 2022

I want to summarize the current status how to build opencv-python package on Windows ARM. You need to follow steps:

  1. Firstly, install python for Windows ARM. At the moment, there are no released python installers, but worked pre-release exist (link).
  2. Install an emulated cmake (link). Currently, you cannot install cmake with pip, there are no pre-built wheel and the build from source fails.
  3. Install Visual Studio Build Tools 2019 marking "Individual Components":
  • C++ Universal Windows Platform support for v142 build tools (ARM64)
  • MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)
  • MSVC v142 - VS 2019 C++ ARM64 build tools (latest)
  • MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (latest)
  • C++ ATL for latest v142 build tools (ARM64)
  • C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)
  • Windows 10 SDK (10.0.19041.0)
  1. Build opencv-python package using following command:
pip wheel . --verbose
  1. Finally, you should have a opencv-python wheel

If you need install OpenCV python package with opencv_contrib or without UI, you have to define environment variables (check it here)

Thanks a lot to @nsait-linaro!

@sid0710
Copy link

sid0710 commented Jun 14, 2023

I want to summarize the current status how to build opencv-python package on Windows ARM. You need to follow steps:

  1. Firstly, install python for Windows ARM. At the moment, there are no released python installers, but worked pre-release exist (link).
  2. Install an emulated cmake (link). Currently, you cannot install cmake with pip, there are no pre-built wheel and the build from source fails.
  3. Install Visual Studio Build Tools 2019 marking "Individual Components":
  • C++ Universal Windows Platform support for v142 build tools (ARM64)
  • MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)
  • MSVC v142 - VS 2019 C++ ARM64 build tools (latest)
  • MSVC v142 - VS 2019 C++ ARM64 Spectre-mitigated libs (latest)
  • C++ ATL for latest v142 build tools (ARM64)
  • C++ ATL for latest v142 build tools with Spectre Mitigations (ARM64)
  • Windows 10 SDK (10.0.19041.0)
  1. Build opencv-python package using following command:
pip wheel . --verbose
  1. Finally, you should have a opencv-python wheel

If you need install OpenCV python package with opencv_contrib or without UI, you have to define environment variables (check it here)

Thanks a lot to @nsait-linaro!

I tried these steps and finally on the "pip wheel . --verbose" step, its failing -

opencv_stitching.vcxproj -> C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\lib\Release\opencv_stitching470.lib
    Building Custom Rule C:/Users/HCKTest/opencv-python/opencv/modules/videoio/CMakeLists.txt
  Traceback (most recent call last):
    File "C:\Users\HCKTest\AppData\Local\Temp\pip-build-env-a2qnvhuj\overlay\Lib\site-packages\skbuild\setuptools_wrap.py", line 674, in setup
      cmkr.make(make_args, install_target=cmake_install_target, env=env)
    File "C:\Users\HCKTest\AppData\Local\Temp\pip-build-env-a2qnvhuj\overlay\Lib\site-packages\skbuild\cmaker.py", line 697, in make
      self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
    File "C:\Users\HCKTest\AppData\Local\Temp\pip-build-env-a2qnvhuj\overlay\Lib\site-packages\skbuild\cmaker.py", line 742, in make_impl
      raise SKBuildError(msg)

  An error occurred while building with CMake.
    Command:
      'C:\Users\HCKTest\AppData\Local\Temp\pip-build-env-a2qnvhuj\overlay\Lib\site-packages\cmake\data\bin/cmake.exe' --build . --target install --config Release --
    Install target:
      install
    Source directory:
      C:\Users\HCKTest\opencv-python
    Working directory:
      C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build
  Please check the install target is valid and see CMake's output for more information.

Any guidance on how to proceed from here would be really appreciated.

@asmorkalov
Copy link
Collaborator

Actual issue is above in the console log. It's not visible in your message.

@sid0710
Copy link

sid0710 commented Jun 14, 2023

Sorry about that, here is the error -

C:\Users\HCKTest\opencv-python\opencv\modules\python\src2\cv2.hpp(36,10): fatal  error C1083: Cannot open include file: 'numpy/ndarrayobject.h': No such file or directory (compiling source file C:\Users\HCKTest\opencv-pytho
n\opencv\modules\python\src2\cv2_util.cpp) [C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\modules\python3\opencv_python3.vcxproj]
C:\Users\HCKTest\opencv-python\opencv\modules\python\src2\cv2.hpp(36,10): fatal  error C1083: Cannot open include file: 'numpy/ndarrayobject.h': No such file or directory (compiling source file C:\Users\HCKTest\opencv-pytho
n\opencv\modules\python\src2\cv2_convert.cpp) [C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\modules\python3\opencv_python3.vcxproj]
C:\Users\HCKTest\opencv-python\opencv\modules\python\src2\cv2.hpp(36,10): fatal  error C1083: Cannot open include file: 'numpy/ndarrayobject.h': No such file or directory (compiling source file C:\Users\HCKTest\opencv-pytho
n\opencv\modules\python\src2\cv2_numpy.cpp) [C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\modules\python3\opencv_python3.vcxproj]
C:\Users\HCKTest\opencv-python\opencv\modules\python\src2\cv2.hpp(36,10): fatal  error C1083: Cannot open include file: 'numpy/ndarrayobject.h': No such file or directory (compiling source file C:\Users\HCKTest\opencv-pytho
n\opencv\modules\python\src2\cv2_highgui.cpp) [C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\modules\python3\opencv_python3.vcxproj]
C:\Users\HCKTest\opencv-python\opencv\modules\python\src2\cv2.hpp(36,10): fatal  error C1083: Cannot open include file: 'numpy/ndarrayobject.h': No such file or directory (compiling source file C:\Users\HCKTest\opencv-pytho
n\opencv\modules\python\src2\cv2.cpp) [C:\Users\HCKTest\opencv-python\_skbuild\win-arm64-3.11\cmake-build\modules\python3\opencv_python3.vcxproj]
  Building Custom Rule C:/Users/HCKTest/opencv-python/opencv/modules/videoio/CMakeLists.txt
Traceback (most recent call last):
  File "C:\Program Files\Python311-arm64\Lib\site-packages\skbuild\setuptools_wrap.py", line 674, in setup
    cmkr.make(make_args, install_target=cmake_install_target, env=env)
  File "C:\Program Files\Python311-arm64\Lib\site-packages\skbuild\cmaker.py", line 697, in make
    self.make_impl(clargs=clargs, config=config, source_dir=source_dir, install_target=install_target, env=env)
  File "C:\Program Files\Python311-arm64\Lib\site-packages\skbuild\cmaker.py", line 742, in make_impl
    raise SKBuildError(msg)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants