MSVC crash only in DEBUG when running embedded sample from docs

[Toninoso]

Environment:
Visual Studio 2015 Update 3
Python 3.6.4 for Windows, 32-bit
pybind11 2.2.1

Running Native (C++) Unit test method, using the example from the pybind11 docs:

 TEST_METHOD(Pybind11_HelloWorld_Runs)
 {
       py::scoped_interpreter guard{}; // start the interpreter and keep it alive
       py::print("Hello, World!"); // use the Python API
 }

Everything runs fine in Release build. Running the console version of the MS test runner I see the output "Hello, World!" on the console. Unit Test succeeds.

Now I switch to Debug build where I link to python36_d.lib (using explicit link command) instead of python36.lib. Running the test above crashes accessing NULL pointer, preceded by several throws.

This is the stack when the first exception gets thrown:

KernelBase.dll!_RaiseException@16�()	Unknown
vcruntime140d.dll!_CxxThrowException(void * pExceptionObject, const _s__ThrowInfo * pThrowInfo) Line 136	C++
Python.Core.Utest.debug.dll!pybind11::detail::accessor_policies::tuple_item::get(pybind11::handle obj, unsigned int index) Line 544	C++
Python.Core.Utest.debug.dll!pybind11::detail::accessor<pybind11::detail::accessor_policies::tuple_item>::get_cache() Line 467	C++
Python.Core.Utest.debug.dll!pybind11::detail::accessor<pybind11::detail::accessor_policies::tuple_item>::operator pybind11::object() Line 461	C++
Python.Core.Utest.debug.dll!pybind11::str::str<pybind11::detail::accessor_policies::tuple_item>(const pybind11::detail::accessor<pybind11::detail::accessor_policies::tuple_item> & a) Line 839	C++
Python.Core.Utest.debug.dll!pybind11::detail::print(pybind11::tuple args, pybind11::dict kwargs) Line 1678	C++
Python.Core.Utest.debug.dll!pybind11::print<1,char const (&)[14]>(const char[14] & <args_0>) Line 1710	C++
Python.Core.Utest.debug.dll!PythonCoreUtest::Interpreter::Pybind11_HelloWorld_Runs() Line 16	C++

that is, inside:

struct tuple_item {
    using key_type = size_t;
    static object get(handle obj, size_t index) {
        PyObject *result = PyTuple_GetItem(obj.ptr(), static_cast<ssize_t>(index));
        if (!result) { throw error_already_set(); }
        return reinterpret_borrow<object>(result);
    }

result pointer is NULL.

It looks that arguments get corrupted already when simple collector packs them into a tuple:

template <return_value_policy policy = return_value_policy::automatic_reference, typename... Args>
void print(Args &&...args) {
    auto c = detail::collect_arguments<policy>(std::forward<Args>(args)...);
    detail::print(c.args(), c.kwargs());
}

that is, before detail::print gets invoked, although I am guessing, not fully understanding the meaning and expected values of all the members of the pybind11 objects yet.

After that, there are a few more places resulting in throw error_already_set() and finally, there is access violation inside the call to PyObjectStr()

    /// Return string representation -- always returns a new reference, even if already a str
    static PyObject *raw_str(PyObject *op) {
        PyObject *str_value = PyObject_Str(op);

which crashes the test run.

Running relatively similar 'pure' CPython unit test method, as below, doesn't crash either in Debug or Release builds, the same scenario as above.

       TEST_METHOD(CPython_HelloWorld_Runs)
        {
              Py_Initialize();
              PyRun_SimpleString("print('Hello World!')\n");
              Py_Finalize();
        }

[Toninoso]

Looks like the problem is solved by building VS2015 Native unit test project with Py_DEBUG defined when building debug version.
Dooh, albeit maybe not so much for such a beginner in this area as me :)

Turns out that pybind11 undefines _DEBUG which causes Py_DEBUG not to be defined as well, so one must explicitly define it in VS Preprocessor tab in order to avoid crashes because of the size difference between the structures on pybind11 and python36_d sides.

[flhang]

Thank you so much for this! I'm looking for the solution for serveral days~~~~~. I think this should be added to the docs!