[Question]: Exception: data did not match any variant of untagged enum PyPreTokenizerTypeWrapper at line 99 column 3

[xiulinyang]

Question

Hi,

I tried to load the trained model from huggingface using the following lines.

from flair.models import SequenceTagger

tagger = SequenceTagger.load("xiulinyang/deberta_srl")

It worked on Colab, but when I tried to run the model locally on my laptop, it gave me this error message: Exception: data did not match any variant of untagged enum PyPreTokenizerTypeWrapper at line 99 column 3.

I searched the issue online but the solutions like updating tokenizers do not work. Could you please help me? Thanks!

In case the environment information is needed, I list the packages installed in my virtual environment.

accelerate                0.32.1                   pypi_0    pypi
beautifulsoup4            4.12.3                   pypi_0    pypi
boto3                     1.34.144                 pypi_0    pypi
botocore                  1.34.144                 pypi_0    pypi
bpemb                     0.3.5                    pypi_0    pypi
bzip2                     1.0.8                h80987f9_6  
ca-certificates           2024.7.2             hca03da5_0  
certifi                   2024.7.4                 pypi_0    pypi
charset-normalizer        3.3.2                    pypi_0    pypi
conllu                    5.0.1                    pypi_0    pypi
contourpy                 1.2.1                    pypi_0    pypi
cycler                    0.12.1                   pypi_0    pypi
deprecated                1.2.14                   pypi_0    pypi
filelock                  3.15.4                   pypi_0    pypi
flair                     0.13.1                   pypi_0    pypi
fonttools                 4.53.1                   pypi_0    pypi
fsspec                    2024.6.1                 pypi_0    pypi
ftfy                      6.2.0                    pypi_0    pypi
gdown                     5.2.0                    pypi_0    pypi
gensim                    4.3.2                    pypi_0    pypi
huggingface-hub           0.23.4                   pypi_0    pypi
idna                      3.7                      pypi_0    pypi
janome                    0.5.0                    pypi_0    pypi
jinja2                    3.1.4                    pypi_0    pypi
jmespath                  1.0.1                    pypi_0    pypi
joblib                    1.4.2                    pypi_0    pypi
kiwisolver                1.4.5                    pypi_0    pypi
langdetect                1.0.9                    pypi_0    pypi
libffi                    3.4.4                hca03da5_1  
lxml                      5.2.2                    pypi_0    pypi
markupsafe                2.1.5                    pypi_0    pypi
matplotlib                3.9.1                    pypi_0    pypi
more-itertools            10.3.0                   pypi_0    pypi
mpld3                     0.5.10                   pypi_0    pypi
mpmath                    1.3.0                    pypi_0    pypi
ncurses                   6.4                  h313beb8_0  
networkx                  3.3                      pypi_0    pypi
numpy                     1.26.4                   pypi_0    pypi
openssl                   1.1.1w               h1a28f6b_0  
packaging                 24.1                     pypi_0    pypi
pillow                    10.4.0                   pypi_0    pypi
pip                       24.0            py310hca03da5_0  
pptree                    3.1                      pypi_0    pypi
protobuf                  5.27.2                   pypi_0    pypi
psutil                    6.0.0                    pypi_0    pypi
pyparsing                 3.1.2                    pypi_0    pypi
pysocks                   1.7.1                    pypi_0    pypi
python                    3.10.0               hbdb9e5c_5  
python-dateutil           2.9.0.post0              pypi_0    pypi
pytorch-revgrad           0.2.0                    pypi_0    pypi
pyyaml                    6.0.1                    pypi_0    pypi
readline                  8.2                  h1a28f6b_0  
regex                     2024.5.15                pypi_0    pypi
requests                  2.32.3                   pypi_0    pypi
s3transfer                0.10.2                   pypi_0    pypi
safetensors               0.4.3                    pypi_0    pypi
scikit-learn              1.5.1                    pypi_0    pypi
scipy                     1.14.0                   pypi_0    pypi
segtok                    1.5.11                   pypi_0    pypi
semver                    3.0.2                    pypi_0    pypi
sentencepiece             0.2.0                    pypi_0    pypi
setuptools                69.5.1          py310hca03da5_0  
six                       1.16.0                   pypi_0    pypi
smart-open                7.0.4                    pypi_0    pypi
soupsieve                 2.5                      pypi_0    pypi
sqlite                    3.45.3               h80987f9_0  
sqlitedict                2.1.0                    pypi_0    pypi
sympy                     1.13.0                   pypi_0    pypi
tabulate                  0.9.0                    pypi_0    pypi
threadpoolctl             3.5.0                    pypi_0    pypi
tk                        8.6.14               h6ba3021_0  
tokenizers                0.19.1                   pypi_0    pypi
torch                     2.3.1                    pypi_0    pypi
tqdm                      4.66.4                   pypi_0    pypi
transformer-smaller-training-vocab 0.4.0                    pypi_0    pypi
transformers              4.42.4                   pypi_0    pypi
typing-extensions         4.12.2                   pypi_0    pypi
tzdata                    2024a                h04d1e81_0  
urllib3                   1.26.19                  pypi_0    pypi
wcwidth                   0.2.13                   pypi_0    pypi
wheel                     0.43.0          py310hca03da5_0  
wikipedia-api             0.6.0                    pypi_0    pypi
wrapt                     1.16.0                   pypi_0    pypi
xz                        5.4.6                h80987f9_1  
zlib                      1.2.13               h18a0788_1