Pure-Python extension does not work with Python 3.12

[musicinmybrain]

What version of protobuf and what language are you using?

Version: 3.19.6

The line that sets the docstring is present in the current release (22.1) and main, so I suspect that they will also be affected:

protobuf/python/google/protobuf/internal/python_message.py

Line 708 in 6545c67

setattr(cls, property_name, _FieldProperty(field, getter, setter, doc=doc))

Language: Python

What operating system (Linux, Windows, ...) and version?

Fedora Linux Rawhide/39, plus experimental Python 3.12: https://copr.fedorainfracloud.org/coprs/g/python/python3.12/

What runtime / compiler are you using (e.g., python version or gcc version)

Python 3.12.0a5, GCC 13.0.1

What did you do?
Steps to reproduce the behavior:

1. Build the pure-Python version of the Python protobuf extension.
2. from google.protobuf import descriptor_pb2

This import occurred in the generated proto bindings in googleapis-common-protos; see downstream bug https://bugzilla.redhat.com/show_bug.cgi?id=2176158.

What did you expect to see

Successful import with no output.

What did you see instead?

[…]
  File "/usr/lib/python3.12/site-packages/google/protobuf/descriptor_pool.py", line 216, in AddSerializedFile
    from google.protobuf import descriptor_pb2  
  File "/usr/lib/python3.12/site-packages/google/protobuf/descriptor_pb2.py", line 1883, in <module>
    FileDescriptorSet = _reflection.GeneratedProtocolMessageType('FileDescriptorSet', (_message.Message,), {
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/google/protobuf/internal/python_message.py", line 198, in __init__
    _AddPropertiesForFields(descriptor, cls)
  File "/usr/lib/python3.12/site-packages/google/protobuf/internal/python_message.py", line 586, in _AddPropertiesForFields
    _AddPropertiesForField(field, cls)
  File "/usr/lib/python3.12/site-packages/google/protobuf/internal/python_message.py", line 612, in _AddPropertiesForField
    _AddPropertiesForRepeatedField(field, cls)  
  File "/usr/lib/python3.12/site-packages/google/protobuf/internal/python_message.py", line 668, in _AddPropertiesForRepeatedField
    setattr(cls, property_name, _FieldProperty(field, getter, setter, doc=doc))
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/google/protobuf/internal/python_message.py", line 623, in __init__
    property.__init__(self, getter, setter, doc=doc)
AttributeError: '_FieldProperty' object attribute '__doc__' is read-only

Anything else we should know about your project / environment

See also: #12186

[hrnciar]

Hello @haberman, have you had a chance to look into this? Unblocking this would allow us to test about 60 other packages depending on protobuf with upcoming Python 3.12.

[haberman]

What is the right fix here? Would you be able to send a PR?

[hrnciar]

I am sorry, but I've no idea. I pinged you because you were assigned to this issue.

[JelleZijlstra]

I think this python/cpython#98963

[gpshead]

The only docstrings ever set via a _FieldProperty constructor in python_message.py are of the generic form:

doc = 'Magic attribute generated for "%s" proto field.' % proto_field_name

which does not sound meaningfully informative.

I suspect we could just stop passing doc=doc to the property super class constructor in the _FieldProperty constructor?

-    property.__init__(self, getter, setter, doc=doc)
+    super().__init__(getter, setter)

[gpshead]

For whatever reasons... that doesn't really work. What I think I'm seeing so far is that 3.12 a class inheriting from property cannot use __slots__. I'm narrowing this down and if that is true will file an upstream issue.

[gpshead]

Okay, we've restored the no-AttributeError behavior upstream, 3.12beta2 should include the fix.

[gpshead]

As _FieldProperty instance docstrings are silently never set anyways, I do recommend that the protobuf code simplify itself and not bother constructing and passing the meaningless doc= parameter in python_message.py:

-  doc = 'Magic attribute generated for "%s" proto field.' % proto_field_name
-  setattr(cls, property_name, _FieldProperty(field, getter, setter, doc=doc))
+  setattr(cls, property_name, _FieldProperty(field, getter, setter))

[anandolee]

I tried to remove doc=doc but it raises:

  File "/third_party/py/google/protobuf/internal/python_message.py", line 651, in _AddPropertiesForRepeatedField
    setattr(cls, property_name, _FieldProperty(field, getter, setter))
  File "/third_party/py/google/protobuf/internal/python_message.py", line 607, in __init__
    property.__init__(self, getter, setter)
AttributeError: '_FieldProperty' object attribute '__doc__' is read-only

[anandolee]

Will close this issue as python team has restored the no-AttributeError behavior upstream. Python 3.12beta2 and above will not have this issue.

Feel free to reopen if this is not fixed after Python 3.12beta2