gh-124074: Add _Py_NewImmortalRef() function

[vstinner]

• Issue: [C API] Add _Py_NewImmortalRef() function to the internal C API #124074

[vstinner]

cc @encukou

[ZeroIntensity]

PEP 7:

Suggested change

	// Similar to Py_NewRef() but for immortal objects.
	// obj must be immortal.
	static inline PyObject* _Py_NewImmortalRef(PyObject *obj)
	// Similar to Py_NewRef() but for immortal objects.
	// obj must be immortal.
	static inline PyObject *
	_Py_NewImmortalRef(PyObject *obj)

[encukou]

cc @encukou

If you ask me: I don't think this is useful without

• broad agreement among the core devs that it should be used;
• a checker (e.g. including these in the refcount total -- until all the calls are fixed it would need to be optional);
• documentation (in Devguide or InternalDocs);
• a DECREF equivalent.

[vstinner]

a checker (e.g. including these in the refcount total -- until all the calls are fixed it would need to be optional);

Do you mean a tool which would fail if _Py_NewImmortalRef() is used to "INCREF" but Py_DECREF() (or _Py_DecRefImmortal()) would be omitted by mistake? It seems non trivial to implement checker in an efficient way.

[ZeroIntensity]

For clarification: would _Py_NewImmortalRef only get used for immortal constants (things like Py_None, Py_True, Py_False), or try and retrofit it for other things that we happen to know are immortal (e.g., some interned strings)? If it's the former, then it should be relatively easy to write a checker for that.

[vstinner]

It works on any immortal object: https://docs.python.org/dev/glossary.html#term-immortal

[encukou]

Do you mean a tool which would fail if _Py_NewImmortalRef() is used to "INCREF" but Py_DECREF() (or _Py_DecRefImmortal()) would be omitted by mistake? It seems non trivial to implement checker in an efficient way.

AFAIK, it would only need the existing _Py_IncRefTotal and _Py_DecRefTotal.

[vstinner]

AFAIK, it would only need the existing _Py_IncRefTotal and _Py_DecRefTotal.

It's more complicated than that since Python/ceval.c now converts immortal objects to "stack ref" objects and it doesn't call DECREF on them on purpose. Maybe it's a lost battle and we should give up on doing Py_NewRef(Py_None), but just write Py_None.

[encukou]

It's more complicated than that since Python/ceval.c now converts immortal objects to "stack ref" objects and it doesn't call DECREF on them on purpose.

Yeah, it would need to start doing that DECREF, in debug builds. Otherwise, _Py_NewImmortalRef is no better than a comment.

[vstinner]

I prefer to abandon this function at this point. I failed to update Py_INCREF(), Py_DECREF() and _Py_NewImmortalRef() to keep a consistent "total ref count" using _Py_INCREF_IncRefTotal(); and _Py_DECREF_DecRefTotal();.