Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

一个困扰了10天的问题! python3+lac竟然会遇到编码问题 #259

Open
PhenixZhang opened this issue Nov 5, 2022 · 1 comment

Comments

@PhenixZhang
Copy link

在本地用python3.7.1_gcc82_pd2.0rc1_cpu测试是成功的,通过lac对文本进行分词,也输出正常的字符串
image

但是在spark集群上,使用相同版本的python3.7.1_gcc82_pd2.0rc1_cpu通过spark集群调用失败,debug输出的结果是乱码
image
单独输出原始字符就是正常的,这就代表是lac.run(text)输出结果为乱码,希望能够有相应解答~

@lynnhuang97
Copy link

你好,请问你在spark集群上能分发吗?我用lac不管是mapPartitions还是broadcast都会报错 _pickle.PicklingError: Could not serialize broadcast: TypeError: can't pickle paddle.fluid.core_avx.CPUPlace objects,说paddle没法序列化。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants