Skip to content

Commit

Permalink
free-threading: final tweaks
Browse files Browse the repository at this point in the history
  • Loading branch information
wjakob committed Sep 20, 2024
1 parent f17c414 commit 1dff7c4
Show file tree
Hide file tree
Showing 4 changed files with 45 additions and 19 deletions.
8 changes: 6 additions & 2 deletions docs/api_core.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1668,6 +1668,12 @@ parameter of :cpp:func:`module_::def`, :cpp:func:`class_::def`,

Indicate that the bound constructor can be used to perform implicit conversions.

.. cpp:struct:: lock_self

Indicate that the implicit ``self`` argument of a method should be locked
when dispatching a call in a free-threaded extension. This annotation does
nothing in regular GIL-protected extensions.

.. cpp:struct:: template <typename... Ts> call_guard

Invoke the call guard(s) `Ts` when the bound function executes. The RAII
Expand Down Expand Up @@ -1887,10 +1893,8 @@ parameter of :cpp:func:`module_::def`, :cpp:func:`class_::def`,
.. cpp:struct:: template <typename T> for_setter


Analogous to :cpp:struct:`for_getter`, but for setters.


.. _class_binding_annotations:

Class binding annotations
Expand Down
27 changes: 18 additions & 9 deletions docs/free_threaded.rst
Original file line number Diff line number Diff line change
Expand Up @@ -153,8 +153,8 @@ To do so, call the :cpp:func:`.lock() <arg::lock>` member of
:cpp:class:`nb::arg() <arg>` annotations to indicate that an
argument must be locked, e.g.:

- ``nb::arg("my_parameter").lock()``
- ``"my_parameter"_a.lock()`` (short-hand form)
- :cpp:func:`nb::arg("my_parameter").lock() <arg::lock>`
- :cpp:func:`"my_parameter"_a.lock() <arg::lock>` (short-hand form)

In methods bindings, pass :cpp:struct:`nb::lock_self() <lock_self>` to lock
the implicit ``self`` argument. Note that at most 2 arguments can be
Expand Down Expand Up @@ -189,11 +189,19 @@ larger-scale redesign of your project may be in order.

.. note::

Adding locking annotations indiscriminately is inadvisable because they add
a runtime cost to function call dispatcher.
The :cpp:func:`.lock() <arg::lock>` annotation is ignored in GIL-protected
builds. Note that listing arguments in function bindings generally comes at
a small cost in terms of :ref:`binding overheads <binding-overheads>`.
Adding locking annotations indiscriminately is inadvisable because locked
calls are more costly than unlocked ones. The :cpp:func:`.lock()
<arg::lock>` and :cpp:struct:`nb::lock_self() <lock_self>` annotations are
ignored in GIL-protected builds, hence this added cost only applies to
free-threaded extensions.

Furthermore, when adding locking annotations to a function, consider keeping
the arguments *unnamed* (i.e., :cpp:func:`nb::arg().lock() <arg::lock>`
instead of :cpp:func:`nb::arg("name").lock() <arg::lock>`) if the function
will never be called with keyword arguments. Processing named arguments
causes small :ref:`binding overheads <binding-overheads>` that may be
undesirable if a function that does very little is called at a very high
rate.

.. note::

Expand Down Expand Up @@ -269,8 +277,9 @@ high rate.

Similar to free-threaded Python itself, nanobind avoids this bottleneck by
*immortalizing* functions (``nanobind.nb_func``, ``nanobind.nb_method``) and
type bindings. Immortal objects don't require reference counting. In turn, the
downside is that they leak when the interpreter shuts down. Free-threaded
type bindings. Immortal objects don't require reference counting and therefore
cannot cause the bottleneck mentioned above. The main downside of this approach
is that these objects leak when the interpreter shuts down. Free-threaded
nanobind extensions disable the internal :ref:`leak checker <leak-checker>`,
since it would produce many warning messages caused by immortal objects.

Expand Down
27 changes: 20 additions & 7 deletions docs/functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -567,16 +567,23 @@ In the snippet below, ``f1`` has lower binding overheads compared to ``f2``.
This is because ``f1``:

- does not accept keyword arguments and does not specify :ref:`default argument
values <keyword_and_default_args>`.
1. Does *not* use any of the following advanced argument annotations features:

- has no :cpp:class:`nb::keep_alive\<Nurse, Patient\>() <keep_alive>` or
:ref:`argument locking <argument-locks>` annotations.
- **Named function arguments**, e.g., :cpp:class:`nb::arg("name") <arg>` or ``"name"_a``.

- takes no variable-length positional (:cpp:class:`nb::args <args>`) or keyword
(:cpp:class:`nb::kwargs <kwargs>`) arguments.
- **Default argument values**, e.g., :cpp:func:`nb::arg() = 0 <arg::operator=>` or ``"name"_a = false``.

- has 8 or fewer arguments.
- **Nullability** or **implicit conversion** flags, e.g.,
:cpp:func:`nb::arg().none() <arg::none>` or :cpp:func:`"name"_a.noconvert()
<arg::noconvert>`.

2. Has no :cpp:class:`nb::keep_alive\<Nurse, Patient\>() <keep_alive>`
annotations.

3. Takes no variable-length positional (:cpp:class:`nb::args <args>`) or keyword
(:cpp:class:`nb::kwargs <kwargs>`) arguments.

4. Has a to total of **8 or fewer** function arguments.

If all of the above conditions are satisfied, nanobind switches to a
specialized dispatcher that is optimized to handle a small number of positional
Expand All @@ -587,3 +594,9 @@ execute more slowly, since nanobind must first select a suitable one.
These differences are mainly of interest when a function that does *very
little* is called at a *very high rate*, in which case binding overheads can
become noticeable.

Regarding point 1 of the above list, note that **locking** is okay, as long as
the annotation does not provide an argument name. In other words, a function
binding with a :cpp:func:`nb::arg().lock() <arg::lock>` for some of its arguments stays on the fast
path. This is mainly of interest for :ref:`free-threaded <free-threaded>`
extensions.
2 changes: 1 addition & 1 deletion src/nb_internals.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@

#define NB_INTERNALS_ID \
"v" NB_TOSTRING(NB_INTERNALS_VERSION) \
NB_COMPILER_TYPE NB_STDLIB NB_VERSION_DEV_STR NB_BUILD_ABI \
NB_VERSION_DEV_STR NB_COMPILER_TYPE NB_STDLIB NB_BUILD_ABI \
NB_BUILD_TYPE NB_STABLE_ABI NB_FREE_THREADED_ABI

NAMESPACE_BEGIN(NB_NAMESPACE)
Expand Down

0 comments on commit 1dff7c4

Please sign in to comment.