Support for free-threaded Python #695

This commit adds the ``FREE_THREADED`` parameter to the CMake ``nanobind_add_module()`` command. It does not do anything for now besides defining ``NB_FREE_THREADED`` in C++ compilation units.

Nanobind's internal data structures switch to a different layout when free-threading was requested. We must give such extensions a different ABI tag since they are incompatible with non-free-threaded ones.

This commit implements RAII scope guards that are thin wrappers around existing Python API to lock Python mutexes and enter critical sections with respect to one or two Python objects. In ordinary (non-free-threaded) Python builds, they don't do anything and can be optimized away.

The garbage collector in free-threaded Python does not like it when the reference count of an object temporarily increases while traversing the object graph in ``tp_traverse``, and doing so introduces leaks. Unfortunately, example implementations of ``tp_traverse`` in both documentation and test suite fall into this trap and must be adapted.

This commit enables free-threaded extension builds on Python 3.13+, which involves the following changes: - nanobind must notify Python that an extension supports free-threading. - All internal data structures must be protected from concurrent modification. The approach taken varies with respect to the specific data structure, and a long comment in ``nb_internals.h`` explains the design decisions all of the changes. In general, the implementation avoids centralized locks as much as possible to improve scalability. - Adopting safe versions of certain operations where needed, e.g. ``PyList_GetItemRef()``. - Switching non-object allocation from ``PyObject_Allo()`` to ``PyMem_Alloc()``.

Global objects that undergo a high rate of reference count changes can become a bottleneck in free-threaded Python extensions, since the associated atomic operation require coordination between processor cores. Function and type objects are a particular concern. This commit immortalizes such objects, which exempts them from free-threading. The downside of this is that they will leak.

Adapting C++ to handle parallelism due to free-threaded Python can be tricky, especially when the original code is given as-is. This commit an tentative API to retrofit locking onto existing code by locking the arguments of function calls.

This commit documents free-threading in general and in the context of nanobind extensions.

Several parallel tests to check that locking works as expected

This commit changes the ``nb::dict`` iterator so that nanobind can implement the recommendation from https://docs.python.org/3.14/howto/free-threading-extensions.html#pydict-next The primary goal of ``nb::internal::dict_iterator`` was to be able to write ```cpp nb::dict my_dict = /* ... */; for (auto [k, v] : my_dict) { // .... } ``` This in fact the only associated feature that is explicitly mentioned in the documentation, and this continues to work. However, some undocumented features are lost: - The dictionary iterator is no longer copyable. This is because it must acquire an exclusive lock to the underlying dictionary. - The pre-increment operator ``++dict_it`` (which relied on copying) is gone. Post-increment continues to work, and that is enough for the loop structure mentioned above.

This commit refactors argument the locking locking so that it occurs at compile-time without imposing runtime overheads. The change applies to free-threaded extensions. Behavior differences compared to the prior approach: - it is no longer possible to do ``nb::arg().lock(false)`` or ``.lock(runtime_determined_value)`` - we no longer prohibit locking self in ``__init__``; changing this would also require restoring ``cast_flags::lock``, and it's not clear that the benefit outweighs the complexity.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for free-threaded Python #695

Support for free-threaded Python #695

Commits on Sep 20, 2024