Convert ResNet101 from TensorFlow to PyTorch #22

ahkarami · 2017-12-06T07:25:40Z

Dear @kitstar,
I want to convert a ResNet V1 101 model (from TF-Slim) to PyTorch. Would you please kindly help me to do that?
Just as another suggestion, I think it would be great if you create a README.md file for PyTorch conversion section.

kitstar · 2017-12-06T09:00:32Z

Hi @ahkarami ,

Tested with follow steps and some code addition.

Download resnet_v1_101 data

$ wget http://download.tensorflow.org/models/resnet_v1_101_2016_08_28.tar.gz
$ tar -xvf resnet_v1_101_2016_08_28.tar.gz

Extract tf model files
Refer the tensorflow model extractor to implement your own model extract code.

I did it for you and you can refer this commit

python -m mmdnn.conversion.examples.tensorflow.extract_model -n resnet_v1_101 -ckpt resnet_v1_101.ckpt

Convert tf to IR

python -m mmdnn.conversion._script.convertToIR -f tensorflow -d kit_imagenet -n imagenet_resnet_v1_101.ckpt.meta --dstNodeName Squeeze -w imagenet_resnet_v1_101.ckpt

Convert IR to Pytorch
Please use argument -dw to extract the weight file for pytorch model saver

python -m mmdnn.conversion._script.IRToCode -f pytorch --IRModelPath kit_imagenet.pb --dstModelPath kit_imagenet.py --IRWeightPath kit_imagenet.npy -dw kit_pytorch.npy

Currently you can use the kit_imagenet.py and kit_pytorch.npy to inference. And you can dump the original PyTorch model by next step.

Dump the PyTorch model

$ python -m mmdnn.conversion.examples.pytorch.imagenet_test --dump resnet.pth -n kit_imagenet.py -w kit_pytorch.npy

PyTorch model file is saved as [resnet.pth], generated by [kit_imagenet.py] and [kit_pytorch.npy].

Done.

kitstar · 2017-12-06T09:43:53Z

PS: the readme of PyTorch is created.

ahkarami · 2017-12-07T07:31:46Z

Dear @kitstar,
Thank you very much for your complete answer. You are incredible!! 🥇

ahkarami · 2017-12-18T21:40:38Z

Dear @kitstar,
Thanks a lot for your complete and clear answer. However, unfortunately because I was so busy last 2 weeks, I couldn't test your answer. I Tested it today. At first I have downloaded the resnet_v1_101 data as you have mentioned. But, at the second step:
python -m mmdnn.conversion.examples.tensorflow.extract_model -n resnet_v1_101 -ckpt resnet_v1_101.ckpt
unfortunately I got this error:

Traceback (most recent call last):
  File "/home/karami/anaconda3/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/karami/anaconda3/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/karami/anaconda3/lib/python3.6/site-packages/mmdnn/conversion/examples/tensorflow/extract_model.py", line 36, in <module>
    'resnet50'      : resnet_v2.resnet_arg_scope,
AttributeError: module 'tensorflow.contrib.slim.python.slim.nets.resnet_v2' has no attribute 'resnet_arg_scope'

Would you please help me to address this error?

kitstar · 2017-12-19T05:13:15Z

Hi, what is your tf version? I tested it with tf 1.3 and 1.4 and you can grep your tf package with "resnet_arg_scope" to find the namespace.

ahkarami · 2017-12-19T20:52:17Z

Thank you. currently TF 1.1.0 has been installed on my system. I will upgrade it and test the conversion code again.

DW1HH · 2018-08-27T08:49:18Z

@kitstar HI,What does it mean-> 'vgg19' : [(21, 0.37522122), (144, 0.28500062), (23, 0.099720284), (134, 0.036305398), (22, 0.033559237)],

namizzz · 2018-08-27T09:05:55Z

Hi @DW1HH , previously, the truth table is the results of imagenet image for test_truth function to test the accuracy of conversion. But we use tests/test_conversion_imagenet.py to check the accuracy.

DW1HH · 2018-08-27T12:22:59Z

@namizzz i want to transform mobilenet_v2_1.4 model from tensorflow to pytorch,but i have no idea for that.

namizzz · 2018-08-27T12:43:38Z

Hi @DW1HH , refer to this:
mmconvert -sf tensorflow -iw resnet50v1_frozen.pb --inNodeName "input_tensor" --inputShape 224,224,3 --dstNodeName "softmax_tensor" -df pytorch -dw pytorch.npy -om resnet50v1.pth

DW1HH · 2018-08-27T13:20:41Z

Thank you for your reply@namizzz

DW1HH · 2018-08-28T10:06:03Z

I am sorry to bother you. @namizzz When the script finishes running,i get two files, mobilenet.pth and mobilenet.py ,but i can not load the mobilenet.pth , it's type-> <type 'long'>,not a dict.

z6833 · 2019-10-29T07:38:23Z

I have a tensorflow models produced by tensorflow1.14 , and there is no .ckpt files . There are model.data-00000-of-00001, model.index and model.meta instead . How can I transform them to .pt/.pkl files ? Could you help me ? URL for models https://github.com/StanfordSNR/indigo/tree/a9b2060d39e4da2e8987e893e3eca2a6c7cd0ab9/dagger/model

Yong-DAI · 2019-11-25T11:54:38Z

I have completed the above steps and got a vgg16.pth file, however, when I used that in pytorch. I got the following errors:

File "", line 1, in
runfile('E:/doc_dy/pytorch_deephash-master/train.py', wdir='E:/doc_dy/pytorch_deephash-master')

File "C:\Users\chddy\Anaconda3\envs\pytorch_gpu\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile
execfile(filename, namespace)

File "C:\Users\chddy\Anaconda3\envs\pytorch_gpu\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "E:/doc_dy/pytorch_deephash-master/train.py", line 8, in
from net_b import AlexNetPlusLatent

File "E:\doc_dy\pytorch_deephash-master\net_b.py", line 6, in
alexnet_model = models.vgg16(pretrained=True) ### alexnet

File "C:\Users\chddy\Anaconda3\envs\pytorch_gpu\lib\site-packages\torchvision\models\vgg.py", line 144, in vgg16
return _vgg('vgg16', 'D', False, pretrained, progress, **kwargs)

File "C:\Users\chddy\Anaconda3\envs\pytorch_gpu\lib\site-packages\torchvision\models\vgg.py", line 92, in _vgg
progress=progress)

File "C:\Users\chddy\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\hub.py", line 434, in load_state_dict_from_url
return torch.load(cached_file, map_location=map_location)

File "C:\Users\chddy\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\serialization.py", line 387, in load
return _load(f, map_location, pickle_module, **pickle_load_args)

File "C:\Users\chddy\Anaconda3\envs\pytorch_gpu\lib\site-packages\torch\serialization.py", line 574, in _load
result = unpickler.load()

ModuleNotFoundError: No module named 'MainModel'

Yong-DAI · 2019-11-25T11:57:46Z

I transfer the pretrained vgg16.ckpt in tensorflow to be used in pytorch. Because the pretrained vgg16.pth in pytorch is not trained enough.

arya-none · 2020-08-11T08:43:32Z

Dear @kitstar
Would you please kindly help me to find whats the problems as below? when i tried to extract resnet50 tf mode. The sincerity anticipates your reply,thanks!
[root@localhost model]# python3 -m mmdnn.conversion.examples.tensorflow.extract_model -n resnet50 -ckpt model.ckpt-450500
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
https://github.com/tensorflow/addons
https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/mmdnn/conversion/examples/tensorflow/extract_model.py:23: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use layer.__call__ method instead.
WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/mmdnn/conversion/examples/tensorflow/extract_model.py:79: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/mmdnn/conversion/examples/tensorflow/extract_model.py:81: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2020-08-11 16:31:03.102027: W tensorflow/core/platform/profile_utils/cpu_utils.cc:98] Failed to find bogomips in /proc/cpuinfo; cannot determine CPU frequency
2020-08-11 16:31:03.104051: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x26ad6f30 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-08-11 16:31:03.104152: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/mmdnn/conversion/examples/tensorflow/extract_model.py:82: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

WARNING:tensorflow:From /usr/local/python3.7.5/lib/python3.7/site-packages/mmdnn/conversion/examples/tensorflow/extract_model.py:85: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

2020-08-11 16:31:14.165516: W tensorflow/core/framework/op_kernel.cc:1651] OP_REQUIRES failed at save_restore_v2_ops.cc:184 : Not found: Key resnet_v2_50/block1/unit_1/bottleneck_v2/conv1/BatchNorm/beta not found in checkpoint
Traceback (most recent call last):
File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/usr/local/python3.7.5/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.NotFoundError: Key resnet_v2_50/block1/unit_1/bottleneck_v2/conv1/BatchNorm/beta not found in checkpoint
[[{{node save/RestoreV2}}]]

YIRuriZhongtian · 2021-03-11T10:42:54Z

我有一个由tensorflow1.14生成的tensorflow模型，并且没有.ckpt文件。有model.data-00000-of-00001，model.index和model.meta代替。如何将它们转换为.pt / .pkl文件？你可以帮帮我吗？模型的网址https://github.com/StanfordSNR/indigo/tree/a9b2060d39e4da2e8987e893e3eca2a6c7cd0ab9/dagger/model

hello，have you solved this problem? i have same question

Rushikeshkoli15 · 2022-02-25T11:56:33Z

hello there, i am getting an error while converting my tf model to IR
AssertionError: Squeeze is not in graph
Could anyone help plz?

jsjy1 · 2022-07-20T07:45:07Z

我有一个由tensorflow1.14生成的tensorflow模型，并且没有.ckpt文件。有model.data-00000-of-00001，model.index和model.meta代替。如何将它们转换为.pt / .pkl文件？你可以帮帮我吗？模型的网址https://github.com/StanfordSNR/indigo/tree/a9b2060d39e4da2e8987e893e3eca2a6c7cd0ab9/dagger/model

hello，have you solved this problem? i have same question

hello，have you solved this problem? i have same question

kitstar closed this as completed Dec 7, 2017

ahkarami mentioned this issue Dec 23, 2017

Convert Special ResNet101 from TensorFlow to PyTorch #31

Closed

sehyun03 mentioned this issue Aug 22, 2018

Asking for 'truth' value on 'resnet-v2-101' model #369

Closed

slala2121 mentioned this issue Aug 13, 2020

Converting from tensorflow to pytorch #885

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert ResNet101 from TensorFlow to PyTorch #22

Convert ResNet101 from TensorFlow to PyTorch #22

ahkarami commented Dec 6, 2017

kitstar commented Dec 6, 2017

kitstar commented Dec 6, 2017

ahkarami commented Dec 7, 2017

ahkarami commented Dec 18, 2017

kitstar commented Dec 19, 2017

ahkarami commented Dec 19, 2017

DW1HH commented Aug 27, 2018

namizzz commented Aug 27, 2018

DW1HH commented Aug 27, 2018

namizzz commented Aug 27, 2018

DW1HH commented Aug 27, 2018

DW1HH commented Aug 28, 2018

z6833 commented Oct 29, 2019

Yong-DAI commented Nov 25, 2019

Yong-DAI commented Nov 25, 2019 •

edited

Loading

arya-none commented Aug 11, 2020

YIRuriZhongtian commented Mar 11, 2021

Rushikeshkoli15 commented Feb 25, 2022

jsjy1 commented Jul 20, 2022

Convert ResNet101 from TensorFlow to PyTorch #22

Convert ResNet101 from TensorFlow to PyTorch #22

Comments

ahkarami commented Dec 6, 2017

kitstar commented Dec 6, 2017

kitstar commented Dec 6, 2017

ahkarami commented Dec 7, 2017

ahkarami commented Dec 18, 2017

kitstar commented Dec 19, 2017

ahkarami commented Dec 19, 2017

DW1HH commented Aug 27, 2018

namizzz commented Aug 27, 2018

DW1HH commented Aug 27, 2018

namizzz commented Aug 27, 2018

DW1HH commented Aug 27, 2018

DW1HH commented Aug 28, 2018

z6833 commented Oct 29, 2019

Yong-DAI commented Nov 25, 2019

Yong-DAI commented Nov 25, 2019 • edited Loading

arya-none commented Aug 11, 2020

YIRuriZhongtian commented Mar 11, 2021

Rushikeshkoli15 commented Feb 25, 2022

jsjy1 commented Jul 20, 2022

Yong-DAI commented Nov 25, 2019 •

edited

Loading