Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Frozen dataclasses with slots cannot override __getstate__ or __setstate__ #104035

Closed
drhagen opened this issue May 1, 2023 · 2 comments · Fixed by #104044
Closed

Frozen dataclasses with slots cannot override __getstate__ or __setstate__ #104035

drhagen opened this issue May 1, 2023 · 2 comments · Fixed by #104044
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@drhagen
Copy link

drhagen commented May 1, 2023

You are supposed to be able to override how pickle pickles an object with __getstate__ and __setstate__. However, these methods are ignored in a dataclass when specifying both frozen=True and slots=True.

See this example:

import pickle
from dataclasses import dataclass


@dataclass(frozen=True, slots=True)
class Foo:
    bar: int

    def __getstate__(self):
        print("getstate")
        return {"bar": self.bar}

    def __setstate__(self, state):
        print("setstate")
        object.__setattr__(self, "bar", state["bar"])

b = pickle.dumps(Foo(1))
foo = pickle.loads(b)

The expected "getstate" and "setstate" lines are never printed because the supplied methods are never called. If either frozen or slots is removed, the expected lines are printed.

From the source code, it is pretty clear why this is happening. If frozen and slots are both True, then special versions of __getstate__ and __setstate__ are unconditionally attached to the dataclass. The dataclass decorator should probably treat this way that it does other methods—only add a method if it is not already present.

Linked PRs

@drhagen drhagen added the type-bug An unexpected behavior, bug, or error label May 1, 2023
@ericvsmith
Copy link
Member

It does seem reasonable to ignore __getstate__ and __setstate__ if they're already set. Have you tested this? And in particular, does it break any existing tests?

@AlexWaygood AlexWaygood added the stdlib Python modules in the Lib dir label May 1, 2023
sobolevn added a commit to sobolevn/cpython that referenced this issue May 1, 2023
@sobolevn
Copy link
Member

sobolevn commented May 1, 2023

Oh, I just went for fixing this in #104041 :)

New logic will allow users to customize __getstate__ and __setstate__ method.
Thanks a lot, @drhagen for finding this :)

It is funny that I've spent the whole day optimizing these two in #103032, but I haven't seen this! 👍

carljm pushed a commit that referenced this issue May 1, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 1, 2023
…lotted frozen dataclasses (pythonGH-104041)

(cherry picked from commit 99aab61)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
carljm pushed a commit that referenced this issue May 1, 2023
…slotted frozen dataclasses (GH-104041) (#104044)

gh-104035: Do not ignore user-defined `__{get,set}state__` in slotted frozen dataclasses (GH-104041)
(cherry picked from commit 99aab61)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
@carljm carljm closed this as completed May 1, 2023
carljm added a commit to carljm/cpython that referenced this issue May 1, 2023
* main: (463 commits)
  pythongh-104057: Fix direct invocation of test_super (python#104064)
  pythongh-87092: Expose assembler to unit tests (python#103988)
  pythongh-97696: asyncio eager tasks factory (python#102853)
  pythongh-84436: Immortalize in _PyStructSequence_InitBuiltinWithFlags() (pythongh-104054)
  pythongh-104057: Fix direct invocation of test_module (pythonGH-104059)
  pythongh-100458: Clarify Enum.__format__() change of mixed-in types in the whatsnew/3.11.rst (pythonGH-100387)
  pythongh-104018: disallow "z" format specifier in %-format of byte strings (pythonGH-104033)
  pythongh-104016: Fixed off by 1 error in f string tokenizer (python#104047)
  pythonGH-103629: Update Unpack's repr in compliance with PEP 692 (python#104048)
  pythongh-102799: replace sys.exc_info by sys.exception in inspect and traceback modules (python#104032)
  Fix typo in "expected" word in few source files (python#104034)
  pythongh-103824: fix use-after-free error in Parser/tokenizer.c (python#103993)
  pythongh-104035: Do not ignore user-defined `__{get,set}state__` in slotted frozen dataclasses (python#104041)
  pythongh-104028: Reduce object creation while calling callback function from gc (pythongh-104030)
  pythongh-104036: Fix direct invocation of test_typing (python#104037)
  pythongh-102213: Optimize the performance of `__getattr__` (pythonGH-103761)
  pythongh-103895: Improve how invalid `Exception.__notes__` are displayed (python#103897)
  Adjust expression from `==` to `!=` in alignment with the meaning of the paragraph. (pythonGH-104021)
  pythongh-88496: Fix IDLE test hang on macOS (python#104025)
  Improve int test coverage (python#104024)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
5 participants