Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'POS' object has no attribute 'model' numpy.core._exceptions._ArrayMemoryError: #48

Open
henrywang0314 opened this issue Sep 25, 2023 · 1 comment

Comments

@henrywang0314
Copy link

henrywang0314 commented Sep 25, 2023

Code
from ckiptagger import data_utils, construct_dictionary, WS, POS, NER
data_utils.download_data_gdown("./")
ws = WS("./data")
pos = POS("./data")
ner = NER("./data")
sentence_list = ["""傅達仁今將執行安樂死,卻突然爆出自己20年前遭緯來體育台封殺,
他不懂自己哪裡得罪到電視台。",
"美國參議院針對今天總統布什所提名的勞工部長趙小蘭展開認可聽
證會,預料她將會很順利通過參議院支持,成為該國有史以來第一
位的華裔女性內閣成員。",
"",
"土地公有政策??還是土地婆有政策。.",
"… 你確定嗎… 不要再騙了……",
"最多容納59,000個人,或5.9萬人,再多就不行了.這是環評的結
論.",
"科長說:1,坪數對人數為1:3。2,可以再增加。"""]

word_s = ws(sentence_list,
sentence_segmentation=True,
segment_delimiter_set={'?', '?', '!', '!', '。', ',',
',', ';', ':', '、'})

word_p = pos(word_s)

print(word_p)

Getting this Error

Traceback (most recent call last):
File "c:\Users\Administrator\Downloads...\python_file.py", line 4, in
pos = POS("./data")
^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\myenv\Lib\site-packages\ckiptagger\api.py", line 160, in init
config.w_token_to_vector, config.w_embedding_d = _load_embedding(os.path.join(data_dir, "embedding_word"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\myenv\Lib\site-packages\ckiptagger\api.py", line 357, in _load_embedding
vector_list = np.load(vector_file)
^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\myenv\Lib\site-packages\numpy\lib\npyio.py", line 432, in load
return format.read_array(fid, allow_pickle=allow_pickle,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\anaconda3\envs\myenv\Lib\site-packages\numpy\lib\format.py", line 790, in
read_array
array = numpy.fromfile(fp, dtype=dtype, count=count)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 1.52 GiB for an array with shape (406737300,) and data type float32
Exception ignored in: <function POS.del at 0x0000020709DE6340>
Traceback (most recent call last):
File "C:\ProgramData\anaconda3\envs\myenv\Lib\site-packages\ckiptagger\api.py", line 185, in del
self.model.sess.close()
^^^^^^^^^^
AttributeError: 'POS' object has no attribute 'model'

@henrywang0314 henrywang0314 changed the title AttributeError: 'POS' object has no attribute 'model' AttributeError: 'POS' object has no attribute 'model' numpy.core._exceptions._ArrayMemoryError: Sep 25, 2023
@jacobvsdanniel
Copy link
Collaborator

Can you check if you ran out of memory?

numpy.core._exceptions._ArrayMemoryError: Unable to allocate 1.52 GiB for an array with shape (406737300,) and data type float32

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants