attributeError: 'str' object has no attribute 'impl'

[lonngxiang]

System Info

optimum-quanto-0.2.4 torch-2.4.0  python 3.10  optimum                   1.21.2

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)

--> 1 from optimum.quanto import freeze, qfloat8, quantize
3 quantize(pipe.transformer, weights=qfloat8)
4 freeze(pipe.transformer)

File /home/anaconda3/envs/aigc/lib/python3.10/site-packages/optimum/quanto/init.py:18
15 version = "0.2.4"
17 from .calibrate import *
---> 18 from .library import *
19 from .models import *
20 from .nn import *

File /home/anaconda3/envs/aigc/lib/python3.10/site-packages/optimum/quanto/library/init.py:15
1 # Copyright 2024 The HuggingFace Team. All rights reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
---> 15 from .extensions import *
16 from .ops import *
17 from .python import *

File /home/anaconda3/envs/aigc/lib/python3.10/site-packages/optimum/quanto/library/extensions/init.py:17
1 # Copyright 2024 The HuggingFace Team. All rights reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
15 import torch
---> 17 from .cpp import *
20 if torch.cuda.is_available():
21 from .cuda import *

File /home/anaconda3/envs/aigc/lib/python3.10/site-packages/optimum/quanto/library/extensions/cpp/init.py:34
22 all = []
25 ext = Extension(
26 "quanto_cpp",
27 root_dir=os.path.dirname(file),
28 sources=["unpack.cpp", "pybind_module.cpp"],
29 extra_cflags=["-O3"],
30 )
33 @torch.library.impl("quanto_ext::unpack", ["CPU"])
---> 34 def unpack_cpp(t: torch.Tensor, bits: int):
35 return ext.lib.unpack(t, bits)

File /home/anaconda3/envs/aigc/lib/python3.10/site-packages/torch/library.py:155, in wrap(f)
152 func_to_register = fn
154 handle = entry.abstract_impl.register(func_to_register, source)
--> 155 self._registration_handles.append(handle)

AttributeError: 'str' object has no attribute 'impl'

Expected behavior

nomal

[fxmarty]

Thank you, this is surprising cc @dacorvo @sayakpaul. Could you share a code snippet to reproduce the issue?

[sayakpaul]

What torch version are you on? Quanto requires Torch 2.4.0 as far as I know.

[dacorvo]

@lonngxiang as @fxmarty suggested, can you share a code snippet ?