Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python3] No target named Python::Interpreter; Build failure under Debug; Runtime error under Release #16956

Closed
lukedan opened this issue Mar 30, 2021 · 13 comments
Assignees
Labels
category:port-bug The issue is with a library, which is something the port should already support

Comments

@lukedan
Copy link

lukedan commented Mar 30, 2021

Host Environment

  • OS: Windows 10 20H2
  • Compiler: Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29913 for x64

To Reproduce
Steps to reproduce the behavior:
./vcpkg install python3:x64-windows

Failure logs
Using the following CMake commands:

find_package(Python3 COMPONENTS Interpreter Development REQUIRED)
message("${Python3_Interpreter_FOUND}")
find_package(pybind11 CONFIG REQUIRED)
target_link_libraries(python_plugin_host_pybind11 PRIVATE Python3::Interpreter Python3::Python pybind11::embed)

Project generation fails with the following output (Notice that Python3_Interpreter_FOUND is TRUE):

1> [CMake] TRUE
1> [CMake] -- Found pybind11: D:/Software/vcpkg/installed/x64-windows/include (found version "2.6.0" )
1> [CMake] -- Configuring done
1> [CMake] CMake Error at D:/Software/vcpkg/scripts/buildsystems/vcpkg.cmake:702 (_add_library):
1> [CMake]   Target "python_plugin_host_pybind11" links to target "Python3::Interpreter"
1> [CMake]   but the target was not found.  Perhaps a find_package() call is missing for
1> [CMake]   an IMPORTED target, or an ALIAS target is missing?
1> [CMake] Call Stack (most recent call first):
1> [CMake]   plugins/python_plugin_host_pybind11/CMakeLists.txt:1 (add_library)
1> [CMake] -- Generating done
1> [CMake] CMake Generate step failed.  Build files cannot be regenerated correctly.

Using the following CMake commands (without linking to Python3::Interpreter):

find_package(Python3 COMPONENTS Interpreter Development REQUIRED)
message("${Python3_Interpreter_FOUND}")
find_package(pybind11 CONFIG REQUIRED)
target_link_libraries(python_plugin_host_pybind11 PRIVATE Python3::Python pybind11::embed)

Under Debug, build fails with the following output:

  FAILED: python_plugin_host_pybind11.dll plugins/python_plugin_host_pybind11/python_plugin_host_pybind11.lib 
  cmd.exe /C "cd . && "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" -E vs_link_dll --intdir=plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  -- C:\PROGRA~2\MICROS~1\2019\COMMUN~1\VC\Tools\MSVC\1428~1.299\bin\Hostx64\x64\link.exe /nologo plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\main.cpp.obj plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\core.cpp.obj plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\ui.cpp.obj  /out:python_plugin_host_pybind11.dll /implib:plugins\python_plugin_host_pybind11\python_plugin_host_pybind11.lib /pdb:python_plugin_host_pybind11.pdb /dll /version:0.0 /machine:x64 /debug /INCREMENTAL  codepad_core.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\python39_d.lib  gdi32.lib  dwrite.lib  d3d11.lib  d2d1.lib  shlwapi.lib  Dwmapi.lib  imm32.lib  windowscodecs.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\cairo-gobjectd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\cairod.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\pango-1.0.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\pangocairo-1.0.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\harfbuzz.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\freetyped.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\icuucd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\gobject-2.0.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\glib-2.0.lib  opengl32.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\skia.dll.lib  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\fontsub.lib"  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\Ole32.Lib"  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\OleAut32.Lib"  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\User32.Lib"  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\USP10.Lib"  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\OpenGL32.Lib"  D:\Software\vcpkg\installed\x64-windows\debug\lib\zlibd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\icuucd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\jpegd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\turbojpegd.lib  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\Gdi32.Lib"  D:\Software\vcpkg\installed\x64-windows\debug\lib\libpng16d.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\webpd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\webpdemuxd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\webpdecoderd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\libwebpmuxd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\libexpatd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\fontconfig.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\freetyped.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\libexpatd.lib  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib  && cmd.exe /C "cd /D D:\Documents\Projects\codepad\out\build\x64-Debug\plugins\python_plugin_host_pybind11 && C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -noprofile -executionpolicy Bypass -file D:/Software/vcpkg/scripts/buildsystems/msbuild/applocal.ps1 -targetBinary D:/Documents/Projects/codepad/out/build/x64-Debug/python_plugin_host_pybind11.dll -installedDir D:/Software/vcpkg/installed/x64-windows/debug/bin -OutVariable out""
  LINK Pass 1: command "C:\PROGRA~2\MICROS~1\2019\COMMUN~1\VC\Tools\MSVC\1428~1.299\bin\Hostx64\x64\link.exe /nologo plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\main.cpp.obj plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\core.cpp.obj plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\ui.cpp.obj /out:python_plugin_host_pybind11.dll /implib:plugins\python_plugin_host_pybind11\python_plugin_host_pybind11.lib /pdb:python_plugin_host_pybind11.pdb /dll /version:0.0 /machine:x64 /debug /INCREMENTAL codepad_core.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\python39_d.lib gdi32.lib dwrite.lib d3d11.lib d2d1.lib shlwapi.lib Dwmapi.lib imm32.lib windowscodecs.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\cairo-gobjectd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\cairod.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\pango-1.0.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\pangocairo-1.0.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\harfbuzz.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\freetyped.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\icuucd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\gobject-2.0.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\glib-2.0.lib opengl32.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\skia.dll.lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\fontsub.lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\Ole32.Lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\OleAut32.Lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\User32.Lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\USP10.Lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\OpenGL32.Lib D:\Software\vcpkg\installed\x64-windows\debug\lib\zlibd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\icuucd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\jpegd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\turbojpegd.lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\Gdi32.Lib D:\Software\vcpkg\installed\x64-windows\debug\lib\libpng16d.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\webpd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\webpdemuxd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\webpdecoderd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\libwebpmuxd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\libexpatd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\fontconfig.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\freetyped.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\libexpatd.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTFILE:plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir/intermediate.manifest plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir/manifest.res" failed (exit code 1104) with the following output:
D:\Documents\Projects\codepad\out\build\x64-Debug\LINK : fatal error LNK1104: cannot open file 'python39.lib'

There may be a bit of noise in the command line, but it only links to python39_d.lib using its full path D:\Software\vcpkg\installed\x64-windows\debug\lib\python39_d.lib. In D:\Software\vcpkg\installed\x64-windows\debug\lib\, python39_d.lib is present but python39.lib is not (it's in D:\Software\vcpkg\installed\x64-windows\lib\).

Under RelWithDebInfo, the program compiles but fails to run with the following message:

Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = 'python'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = 'D:\\Documents\\Projects\\codepad\\out\\build\\x64-Release\\codepad.exe'
  sys.base_prefix = ''
  sys.base_exec_prefix = ''
  sys.platlibdir = 'lib'
  sys.executable = 'D:\\Documents\\Projects\\codepad\\out\\build\\x64-Release\\codepad.exe'
  sys.prefix = ''
  sys.exec_prefix = ''
  sys.path = [
    'D:\\Documents\\Projects\\codepad\\out\\build\\x64-Release\\python39.zip',
    '.\\DLLs',
    '.\\lib',
    'D:\\Documents\\Projects\\codepad\\out\\build\\x64-Release',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00002a4c (most recent call first):
<no Python frame>

Since it mentions D:\\Documents\\Projects\\codepad\\out\\build\\x64-Release\\python39.zip, I downloaded an embedded version of Python 3.9 and put it in D:\Documents\Projects\codepad\out\build\x64-Release\, but it didn't help.

Additional context

Make sure you don't have Python installed system-wide before attempting to reproduce this issue.

@lukedan
Copy link
Author

lukedan commented Mar 30, 2021

Update: Under Debug, adding the /VERBOSE flag gives:

  FAILED: python_plugin_host_pybind11.dll plugins/python_plugin_host_pybind11/python_plugin_host_pybind11.lib 
  cmd.exe /C "cd . && "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" -E vs_link_dll --intdir=plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe --mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  -- C:\PROGRA~2\MICROS~1\2019\COMMUN~1\VC\Tools\MSVC\1428~1.299\bin\Hostx64\x64\link.exe /nologo plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\main.cpp.obj plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\core.cpp.obj plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\ui.cpp.obj  /out:python_plugin_host_pybind11.dll /implib:plugins\python_plugin_host_pybind11\python_plugin_host_pybind11.lib /pdb:python_plugin_host_pybind11.pdb /dll /version:0.0 /machine:x64 /debug /INCREMENTAL  /VERBOSE  codepad_core.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\python39_d.lib  gdi32.lib  dwrite.lib  d3d11.lib  d2d1.lib  shlwapi.lib  Dwmapi.lib  imm32.lib  windowscodecs.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\cairo-gobjectd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\cairod.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\pango-1.0.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\pangocairo-1.0.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\harfbuzz.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\freetyped.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\icuucd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\gobject-2.0.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\glib-2.0.lib  opengl32.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\skia.dll.lib  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\fontsub.lib"  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\Ole32.Lib"  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\OleAut32.Lib"  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\User32.Lib"  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\USP10.Lib"  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\OpenGL32.Lib"  D:\Software\vcpkg\installed\x64-windows\debug\lib\zlibd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\icuucd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\jpegd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\turbojpegd.lib  "C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\Gdi32.Lib"  D:\Software\vcpkg\installed\x64-windows\debug\lib\libpng16d.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\webpd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\webpdemuxd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\webpdecoderd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\libwebpmuxd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\libexpatd.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\fontconfig.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\freetyped.lib  D:\Software\vcpkg\installed\x64-windows\debug\lib\libexpatd.lib  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib  && cmd.exe /C "cd /D D:\Documents\Projects\codepad\out\build\x64-Debug\plugins\python_plugin_host_pybind11 && C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -noprofile -executionpolicy Bypass -file D:/Software/vcpkg/scripts/buildsystems/msbuild/applocal.ps1 -targetBinary D:/Documents/Projects/codepad/out/build/x64-Debug/python_plugin_host_pybind11.dll -installedDir D:/Software/vcpkg/installed/x64-windows/debug/bin -OutVariable out""
  LINK Pass 1: command "C:\PROGRA~2\MICROS~1\2019\COMMUN~1\VC\Tools\MSVC\1428~1.299\bin\Hostx64\x64\link.exe /nologo plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\main.cpp.obj plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\core.cpp.obj plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir\src\ui.cpp.obj /out:python_plugin_host_pybind11.dll /implib:plugins\python_plugin_host_pybind11\python_plugin_host_pybind11.lib /pdb:python_plugin_host_pybind11.pdb /dll /version:0.0 /machine:x64 /debug /INCREMENTAL /VERBOSE codepad_core.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\python39_d.lib gdi32.lib dwrite.lib d3d11.lib d2d1.lib shlwapi.lib Dwmapi.lib imm32.lib windowscodecs.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\cairo-gobjectd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\cairod.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\pango-1.0.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\pangocairo-1.0.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\harfbuzz.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\freetyped.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\icuucd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\gobject-2.0.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\glib-2.0.lib opengl32.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\skia.dll.lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\fontsub.lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\Ole32.Lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\OleAut32.Lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\User32.Lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\USP10.Lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\OpenGL32.Lib D:\Software\vcpkg\installed\x64-windows\debug\lib\zlibd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\icuucd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\jpegd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\turbojpegd.lib C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\Gdi32.Lib D:\Software\vcpkg\installed\x64-windows\debug\lib\libpng16d.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\webpd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\webpdemuxd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\webpdecoderd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\libwebpmuxd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\libexpatd.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\fontconfig.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\freetyped.lib D:\Software\vcpkg\installed\x64-windows\debug\lib\libexpatd.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTFILE:plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir/intermediate.manifest plugins\python_plugin_host_pybind11\CMakeFiles\python_plugin_host_pybind11.dir/manifest.res" failed (exit code 1104) with the following output:
  
  Starting pass 1
  Processed /DEFAULTLIB:msvcprtd
  Processed /DEFAULTLIB:python39.lib
  Processed /DEFAULTLIB:MSVCRTD
  Processed /DEFAULTLIB:OLDNAMES
  
  Searching libraries
      Searching codepad_core.lib:
        Found "public: void __cdecl codepad::logger::log_entry::append_stacktrace(void)" (?append_stacktrace@log_entry@logger@codepad@@QEAAXXZ)
          Referenced in main.cpp.obj
          Referenced in core.cpp.obj
          Referenced in ui.cpp.obj
          Loaded codepad_core.lib(codepad_core.dll)
...

python39.lib was specified via /DEFAULTLIB:python39.lib which wasn't in the command line.

@JackBoosY JackBoosY assigned JackBoosY and NancyLi1013 and unassigned JackBoosY Mar 30, 2021
@NancyLi1013 NancyLi1013 added the category:question This issue is a question label Mar 30, 2021
@NancyLi1013
Copy link
Contributor

Hi @lukedan

Thanks for posting this issue again.
Could you please help provide your whole project sample? including main.cpp and CMakeLists.txt and other files.

Also I want to confirm why you need Interpreter component of Python3.
Since the usage of python3 has been provided like this in vcpkg:

The package python3 is compatible with built-in CMake targets:

    find_package(Python3 COMPONENTS Development REQUIRED)
    target_link_libraries(main PRIVATE Python3::Python)

https://github.com/microsoft/vcpkg/blob/master/ports/python3/usage

So I'm not clear why you not use python3 like this. Does it not meet your demands?

@lukedan
Copy link
Author

lukedan commented Mar 30, 2021

The project can be found at https://github.com/lukedan/codepad. Regarding Python::Interpreter, it's a target provided by CMake (https://cmake.org/cmake/help/latest/module/FindPython3.html), and recommended by pybind11 (https://pybind11.readthedocs.io/en/stable/compiling.html#findpython-mode).

@lukedan
Copy link
Author

lukedan commented Mar 30, 2021

Update: In include/python3.9/pyconfig.h:262:

/* For an MSVC DLL, we can nominate the .lib files used by extensions */
#ifdef MS_COREDLL
#       if !defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_BUILTIN)
                /* not building the core - must be an ext */
#               if defined(_MSC_VER)
                        /* So MSVC users need not specify the .lib
                        file in their Makefile (other compilers are
                        generally taken care of by distutils.) */
#                       if defined(_DEBUG)
#                               pragma comment(lib,"python39_d.lib")
#                       elif defined(Py_LIMITED_API)
#                               pragma comment(lib,"python3.lib")
#                       else
#                               pragma comment(lib,"python39.lib")
#                       endif /* _DEBUG */
#               endif /* _MSC_VER */
#       endif /* Py_BUILD_CORE */
#endif /* MS_COREDLL */

In include/pybind11/detail/common.h:109:

/// Include Python header, disable linking to pythonX_d.lib on Windows in debug mode
#if defined(_MSC_VER)
#  if (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION < 4)
#    define HAVE_ROUND 1
#  endif
#  pragma warning(push)
#  pragma warning(disable: 4510 4610 4512 4005)
#  if defined(_DEBUG) && !defined(Py_DEBUG)
#    define PYBIND11_DEBUG_MARKER
#    undef _DEBUG
#  endif
#endif

#include <Python.h>
#include <frameobject.h>
#include <pythread.h>

...

#if defined(_MSC_VER)
#  if defined(PYBIND11_DEBUG_MARKER)
#    define _DEBUG
#    undef PYBIND11_DEBUG_MARKER
#  endif
#  pragma warning(pop)
#endif

Perhaps vcpkg should define Py_DEBUG for debug builds?

@lukedan
Copy link
Author

lukedan commented Mar 30, 2021

Also, without linking to Python3::Python, linking will always fail with cannot find python39.lib, even in release. Maybe pybind11 CMake instructions should be updated to include find_package(Python3) and target_link_libraries(Python3::Python).

@lukedan
Copy link
Author

lukedan commented Mar 30, 2021

I realized that there's another python39.zip inside the embedded zip version of Python and putting it inside the output directory fixes the issue under release. Given that vcpkg already copies dll files, it may be a good idea to also automatically copy the zip file as well.

@JackBoosY JackBoosY self-assigned this Apr 1, 2021
@JonLiu1993 JonLiu1993 added the info:versioning This PR or Issue pertains to the versioning feature label Apr 2, 2021
@JackBoosY JackBoosY removed the info:versioning This PR or Issue pertains to the versioning feature label Apr 2, 2021
@FelixBer
Copy link

This issue still persists.

To sum it up: Python links against python39(d).dll based on whether a "Release interpreter" or a "Debug Interpreter" is present (rather than build options). Since vcpkg only ships a Release interpreter, it tries to link against the release version of python39.dll even in debug builds, but this is not present because vcpkg has set up paths to debug libraries only. (The release interpreter is located in /vcpkg/installed/x64-windows/tools/python3).

The ad-hoc fix would be to force a debug-build regardless of present interpreter by defining Py_DEBUG for debug builds automatically. This can be done manually right now with

add_compile_definitions($<$<CONFIG:Debug>:Py_DEBUG>).

(However, the resulting binary cant be used with the release interpreter of vcpkg.)

And the proper solution is to ship a debug interpreter and wire it up for python to find in debug builds.

Additionally I agree that the interpreter should probably be copied to the output along with the .dll files, perhaps as a zip archive.

@ssrlive
Copy link

ssrlive commented Oct 21, 2021

Since nobody can fix this bug, every time I build DEBUG target I have to do this command in VS.

copy $(_ZVcpkgCurrentInstalledDir)$(_ZVcpkgConfigSubdir)lib\python39_d.lib $(_ZVcpkgCurrentInstalledDir)$(_ZVcpkgConfigSubdir)lib\python39.lib

or

copy $(_ZVcpkgCurrentInstalledDir)$(_ZVcpkgConfigSubdir)lib\python310_d.lib $(_ZVcpkgCurrentInstalledDir)$(_ZVcpkgConfigSubdir)lib\python310.lib

Now, R.I.P.

@JackBoosY JackBoosY added category:port-feature The issue is with a library, which is requesting new capabilities that didn’t exist and removed category:question This issue is a question labels Oct 21, 2021
@JackBoosY
Copy link
Contributor

JackBoosY commented Oct 21, 2021

The python3 in vcpkg is currently not complete, some libraries and py scripts are not normally built and installed, and the python3 executable is not packaged as expected.
We can only solve the above problems step by step.

@ssrlive
Copy link

ssrlive commented Oct 21, 2021

Now I got a wired DEBUG binary. and I have a question: Why do those guys have to define #undef _DEBUG in boost build script or pybind as such a monster.

and this line is the reason

https://github.com/boostorg/python/blob/500194edb7833d0627ce7a2595fec49d0aae2484/include/boost/python/detail/wrap_python.hpp#L45

image

@JackBoosY
Copy link
Contributor

@ssrlive Should be a port bug.

@JackBoosY JackBoosY added category:port-bug The issue is with a library, which is something the port should already support and removed category:port-feature The issue is with a library, which is requesting new capabilities that didn’t exist labels Nov 5, 2021
@PhoebeHui
Copy link
Contributor

@ssrlive, you need define 'BOOST_DEBUG_PYTHON' before build boost-python, then it will link to python3 debug libs correctly.

The issue fixed by #15352.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:port-bug The issue is with a library, which is something the port should already support
Projects
None yet
Development

No branches or pull requests

8 participants