Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When i am trying to export pb using exportpb.py file from the checkpoints getting error. Please guide me on this... #51

Open
sandhyacs opened this issue Oct 19, 2021 · 0 comments
Open

When i am trying to export pb using exportpb.py file from the checkpoints getting error. Please guide me on this... #51

sandhyacs opened this issue Oct 19, 2021 · 0 comments

Comments

@sandhyacs
Copy link

WARNING:tensorflow:From exportPb.py:61: The name tf.reset_default_graph is deprecated. Please use tf.compat.v1.reset_default_graph instead.

WARNING:tensorflow:From exportPb.py:41: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From ../../libs/models/backbones/mobilenet/mobilenet.py:324: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use layer.__call__ method instead.
WARNING:tensorflow:From ../../libs/models/necks/fpn_p3top7.py:24: The name tf.image.resize_bilinear is deprecated. Please use tf.compat.v1.image.resize_bilinear instead.

WARNING:tensorflow:From exportPb.py:65: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From exportPb.py:67: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-10-19 12:17:08.433221: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2021-10-19 12:17:08.456539: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-10-19 12:17:08.456701: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA GeForce GTX 1050 major: 6 minor: 1 memoryClockRate(GHz): 1.493
pciBusID: 0000:01:00.0
2021-10-19 12:17:08.456731: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-19 12:17:08.457562: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-19 12:17:08.458291: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-10-19 12:17:08.458464: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-10-19 12:17:08.459427: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-10-19 12:17:08.460157: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-10-19 12:17:08.462534: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-10-19 12:17:08.462645: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-10-19 12:17:08.462864: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-10-19 12:17:08.462996: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2021-10-19 12:17:08.463220: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2021-10-19 12:17:08.484313: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2199995000 Hz
2021-10-19 12:17:08.484748: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x563a907797f0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-10-19 12:17:08.484766: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2021-10-19 12:17:08.484948: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-10-19 12:17:08.485137: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA GeForce GTX 1050 major: 6 minor: 1 memoryClockRate(GHz): 1.493
pciBusID: 0000:01:00.0
2021-10-19 12:17:08.485178: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-19 12:17:08.485194: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2021-10-19 12:17:08.485207: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2021-10-19 12:17:08.485219: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2021-10-19 12:17:08.485232: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2021-10-19 12:17:08.485245: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2021-10-19 12:17:08.485257: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-10-19 12:17:08.485314: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-10-19 12:17:08.485486: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-10-19 12:17:08.485618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2021-10-19 12:17:08.485645: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2021-10-19 12:17:08.586685: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-10-19 12:17:08.586707: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0
2021-10-19 12:17:08.586716: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N
2021-10-19 12:17:08.586882: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-10-19 12:17:08.587085: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-10-19 12:17:08.587255: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-10-19 12:17:08.587400: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 84 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce GTX 1050, pci bus id: 0000:01:00.0, compute capability: 6.1)
2021-10-19 12:17:08.588667: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x563a960139c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2021-10-19 12:17:08.588680: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce GTX 1050, Compute Capability 6.1
we have restred the weights from =====>>
../../output/trained_weights/RetinaNet_DOTA1.5_2x_20210314/DOTA1.5_1000model.ckpt
Traceback (most recent call last):
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [105] rhs shape= [84]
[[{{node save/Assign_294}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 1290, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [105] rhs shape= [84]
[[node save/Assign_294 (defined at /home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py:1748) ]]

Original stack trace for 'save/Assign_294':
File "exportPb.py", line 313, in
exporter.export_frozenPB()
File "exportPb.py", line 65, in export_frozenPB
saver = tf.train.Saver()
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 828, in init
self.build()
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 840, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 878, in _build
build_restore=build_restore)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 350, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saving/saveable_object_util.py", line 73, in restore
self.op.get_shape().is_fully_defined())
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/ops/state_ops.py", line 227, in assign
validate_shape=validate_shape)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/ops/gen_state_ops.py", line 66, in assign
use_locking=use_locking, name=name)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "exportPb.py", line 313, in
exporter.export_frozenPB()
File "exportPb.py", line 69, in export_frozenPB
saver.restore(sess, CKPT_PATH)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 1326, in restore
err, "a mismatch between the current graph and the graph")
tensorflow.python.framework.errors_impl.InvalidArgumentError: Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Assign requires shapes of both tensors to match. lhs shape= [105] rhs shape= [84]
[[node save/Assign_294 (defined at /home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py:1748) ]]

Original stack trace for 'save/Assign_294':
File "exportPb.py", line 313, in
exporter.export_frozenPB()
File "exportPb.py", line 65, in export_frozenPB
saver = tf.train.Saver()
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 828, in init
self.build()
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 840, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 878, in _build
build_restore=build_restore)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 350, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/training/saving/saveable_object_util.py", line 73, in restore
self.op.get_shape().is_fully_defined())
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/ops/state_ops.py", line 227, in assign
validate_shape=validate_shape)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/ops/gen_state_ops.py", line 66, in assign
use_locking=use_locking, name=name)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/home/sandhya/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@sandhyacs