./run_convert_market.sh,TypeError: write() argument must be str, not bytes

[Yao1up]

when I run the commond of "./run_convert_market.sh" in TF-record data preparation steps and the result is as follows:
100000
200000
repeat positive pairs augment_ratio times and cut down negative pairs to balance data ......
p_pairs length:296938
n_pairs length:41936
save p_pairs and n_pairs ......
Traceback (most recent call last):
File "convert_market.py", line 1004, in
run_one_pair_rec(dataset_dir, out_dir, split_name)
File "convert_market.py", line 911, in run_one_pair_rec
mode='same_diff_cam')
File "convert_market.py", line 202, in _get_train_all_pn_pairs
pickle.dump(p_pairs,f)
TypeError: write() argument must be str, not bytes

[charliememory]

This issue comes from pickle package. To avoid this error, you can use "wb" mode and "rb" mode to dump and load data.

…

On Fri, 6 Nov 2020 at 10:37, Yao1up ***@***.***> wrote: when I run the commond of "./run_convert_market.sh" in TF-record data preparation steps and the result is as follows: 100000 200000 repeat positive pairs augment_ratio times and cut down negative pairs to balance data ...... p_pairs length:296938 n_pairs length:41936 save p_pairs and n_pairs ...... Traceback (most recent call last): File "convert_market.py", line 1004, in run_one_pair_rec(dataset_dir, out_dir, split_name) File "convert_market.py", line 911, in run_one_pair_rec mode='same_diff_cam') File "convert_market.py", line 202, in _get_train_all_pn_pairs pickle.dump(p_pairs,f) TypeError: write() argument must be str, not bytes — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#43>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABPFNDDC6ZKTC7AIZHIDRR3SOO7XLANCNFSM4TMONXBA> .

[Yao1up]

I have changed 'w'and 'r' to 'wb' and 'rb' ,modified attribute 'has_key()',but the code still has errors:
Traceback (most recent call last):
File "convert_market.py", line 1004, in
run_one_pair_rec(dataset_dir, out_dir, split_name)
File "convert_market.py", line 911, in run_one_pair_rec
mode='same_diff_cam')
File "convert_market.py", line 133, in _get_train_all_pn_pairs
p_pairs = pickle.load(f)
EOFError: Ran out of input
Traceback (most recent call last):
File "convert_market.py", line 1004, in
run_one_pair_rec(dataset_dir, out_dir, split_name)
File "convert_market.py", line 952, in run_one_pair_rec
mode='same_diff_cam')
File "convert_market.py", line 133, in _get_train_all_pn_pairs
p_pairs = pickle.load(f)
EOFError: Ran out of input
_get_train_all_pn_pairs finish ......
p_pairs length:544
n_pairs length:0
save pn_pairs_num ......
id_map_attr length:50
Traceback (most recent call last):
File "convert_market.py", line 1004, in
run_one_pair_rec(dataset_dir, out_dir, split_name)
File "convert_market.py", line 992, in run_one_pair_rec
attr_w2v_dir=attr_w2v_dir, pose_peak_path=pose_peak_path, pose_sub_path=pose_sub_path)
File "convert_market.py", line 810, in _convert_dataset_one_pair_rec_withFlip
all_peaks_dic = pickle.load(f)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position 3: ordinal not in range(128)
I need your help,thank you!

[charliememory]

I think this may be due to the incompatibility between pickle-python2 and pickle-python3. The code and pickle data are developed under python2. Do you use python2 to load the data?

…

On Sat, 7 Nov 2020 at 15:39, Yao1up ***@***.***> wrote: I have changed 'w'and 'r' to 'wb' and 'rb' ,modified attribute 'has_key()',but the code still has errors: Traceback (most recent call last): File "convert_market.py", line 1004, in run_one_pair_rec(dataset_dir, out_dir, split_name) File "convert_market.py", line 911, in run_one_pair_rec mode='same_diff_cam') File "convert_market.py", line 133, in _get_train_all_pn_pairs p_pairs = pickle.load(f) EOFError: Ran out of input Traceback (most recent call last): File "convert_market.py", line 1004, in run_one_pair_rec(dataset_dir, out_dir, split_name) File "convert_market.py", line 952, in run_one_pair_rec mode='same_diff_cam') File "convert_market.py", line 133, in _get_train_all_pn_pairs p_pairs = pickle.load(f) EOFError: Ran out of input _get_train_all_pn_pairs finish ...... p_pairs length:544 n_pairs length:0 save pn_pairs_num ...... id_map_attr length:50 Traceback (most recent call last): File "convert_market.py", line 1004, in run_one_pair_rec(dataset_dir, out_dir, split_name) File "convert_market.py", line 992, in run_one_pair_rec attr_w2v_dir=attr_w2v_dir, pose_peak_path=pose_peak_path, pose_sub_path=pose_sub_path) File "convert_market.py", line 810, in _convert_dataset_one_pair_rec_withFlip all_peaks_dic = pickle.load(f) UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position 3: ordinal not in range(128) I need your help,thank you! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABPFNDG46OMP6YKFUO3J32LSOVLY3ANCNFSM4TMONXBA> .

[Yao1up]

I think this may be due to the incompatibility between pickle-python2 and pickle-python3. The code and pickle data are developed under python2. Do you use python2 to load the data?
…
On Sat, 7 Nov 2020 at 15:39, Yao1up @.***> wrote: I have changed 'w'and 'r' to 'wb' and 'rb' ,modified attribute 'has_key()',but the code still has errors: Traceback (most recent call last): File "convert_market.py", line 1004, in run_one_pair_rec(dataset_dir, out_dir, split_name) File "convert_market.py", line 911, in run_one_pair_rec mode='same_diff_cam') File "convert_market.py", line 133, in _get_train_all_pn_pairs p_pairs = pickle.load(f) EOFError: Ran out of input Traceback (most recent call last): File "convert_market.py", line 1004, in run_one_pair_rec(dataset_dir, out_dir, split_name) File "convert_market.py", line 952, in run_one_pair_rec mode='same_diff_cam') File "convert_market.py", line 133, in _get_train_all_pn_pairs p_pairs = pickle.load(f) EOFError: Ran out of input _get_train_all_pn_pairs finish ...... p_pairs length:544 n_pairs length:0 save pn_pairs_num ...... id_map_attr length:50 Traceback (most recent call last): File "convert_market.py", line 1004, in run_one_pair_rec(dataset_dir, out_dir, split_name) File "convert_market.py", line 992, in run_one_pair_rec attr_w2v_dir=attr_w2v_dir, pose_peak_path=pose_peak_path, pose_sub_path=pose_sub_path) File "convert_market.py", line 810, in _convert_dataset_one_pair_rec_withFlip all_peaks_dic = pickle.load(f) UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position 3: ordinal not in range(128) I need your help,thank you! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPFNDG46OMP6YKFUO3J32LSOVLY3ANCNFSM4TMONXBA .

After I switched to a 2.7 virtual environment，it occurs another two errors:
2020-11-10 14:16:56,721:WARNING::From /home/dc2-user/data/ENV2.7/lib/python2.7/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
Traceback (most recent call last):
File "main.py", line 37, in
main(config)
File "main.py", line 22, in main
trainer.init_net()
File "/data/Pose-Guided-Person-Image-Generation/trainer.py", line 161, in init_net
self.saverPart.restore(self.sess, self.pretrained_path)
File "/home/dc2-user/data/ENV2.7/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1268, in restore
+ compat.as_text(save_path))
ValueError: The passed save_path is not a valid checkpoint: ./data/model/model.ckpt-0
WARNING:tensorflow:From /data/Pose-Guided-Person-Image-Generation/tflib/inception_score.py:75: init (from tensorflow.python.platform.gfile) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.gfile.GFile.

Traceback (most recent call last):
File "score_mask.py", line 13, in
import tflib.inception_score
File "/data/Pose-Guided-Person-Image-Generation/tflib/inception_score.py", line 101, in
_init_inception()
File "/data/Pose-Guided-Person-Image-Generation/tflib/inception_score.py", line 95, in _init_inception
o._shape = tf.TensorShape(new_shape)
File "/home/dc2-user/data/ENV2.7/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 437, in _shape
"Tensor._shape cannot be assigned, use Tensor.set_shape instead.")
ValueError: Tensor._shape cannot be assigned, use Tensor.set_shape instead.
"Tensor._shape" is used in python2 and "Tensor.set_shape"is used in python3, but it still report this error.

[charliememory]

There is a path error in your log "ValueError: The passed save_path is not a valid checkpoint: ./data/model/model.ckpt-0". Did you set the correct path?
As to the second error, it seems caused by the incompatibility of different tflib versions. I have no idea about it. Please check the tflib repo.

[Yao1up]

There is a path error in your log "ValueError: The passed save_path is not a valid checkpoint: ./data/model/model.ckpt-0". Did you set the correct path?
As to the second error, it seems caused by the incompatibility of different tflib versions. I have no idea about it. Please check the tflib repo.

The current code in run_market_ test is:
model_dir=./data/model
start_step=0
pretrained_path=${model_dir}'/model.ckpt-'${start_step}
Trained model named likes "model.ckpt-59999.data-00000-of-00001"or "model.ckpt-47999.meta",no results named "${model_dir}'/model.ckpt-0", do I need to change the start_step？

[charliememory]

Yes, the script is used for the uploaded pretrained models. You need to change the names if you test your own models.

…

On Mon, 16 Nov 2020 at 09:38, Yao1up ***@***.***> wrote: There is a path error in your log "ValueError: The passed save_path is not a valid checkpoint: ./data/model/model.ckpt-0". Did you set the correct path? As to the second error, it seems caused by the incompatibility of different tflib versions. I have no idea about it. Please check the tflib repo. The current code in run_market_ test is: model_dir=./data/model start_step=0 pretrained_path=${model_dir}'/model.ckpt-'${start_step} Trained model named likes "model.ckpt-59999.data-00000-of-00001"or "model.ckpt-47999.meta",no results named "${model_dir}'/model.ckpt-0", do I need to change the start_step？ — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#43 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABPFNDBCVVE54GQJLQPMWETSQDQHZANCNFSM4TMONXBA> .