upgrade_net_proto_binary segfaults

[dieterichlawson]

upgrade_net_proto_binary segfaults when upgrading certain caffemodel files.

Steps to reproduce:

1. Download VGG16 from model zoo
2. run tools/upgrade_net_proto_binary VGG_ILSVRC_16_layers.caffemodel vgg16.caffemodel
3. It segfaults, giving the following stacktrace in gdb.

libdc1394 error: Failed to initialize libdc1394
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:505] Reading dangerously large protocol message.  If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 553432081
[libprotobuf FATAL /usr/include/google/protobuf/repeated_field.h:613] CHECK failed: (index) < (size()):
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: (index) < (size()):

Program received signal SIGABRT, Aborted.
0x00007ffff63b1cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56  ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff63b1cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff63b50d8 in __GI_abort () at abort.c:89
#2  0x00007ffff69b6535 in __gnu_cxx::__verbose_terminate_handler() ()
   from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff69b46d6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff69b4703 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff69b4922 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff0af7b58 in google::protobuf::internal::LogMessage::Finish() ()
   from /usr/lib/x86_64-linux-gnu/libprotobuf.so.8
#7  0x00007ffff738bbe9 in google::protobuf::RepeatedField<int>::Get (this=0x7fffffffe9a8, index=0)
    at /usr/include/google/protobuf/repeated_field.h:613
#8  0x00007ffff737b4be in caffe::NetParameter::input_dim (this=0x7fffffffe960, index=0)
    at .build_debug/src/caffe/proto/caffe.pb.h:10930
#9  0x00007ffff73d9c43 in caffe::UpgradeNetInput (net_param=0x7fffffffe960)
    at src/caffe/util/upgrade_proto.cpp:971
#10 0x00007ffff73d5561 in caffe::UpgradeNetAsNeeded (param_file=..., param=0x7fffffffe960)
    at src/caffe/util/upgrade_proto.cpp:67
#11 0x0000000000401936 in main (argc=3, argv=0x7fffffffeaf8) at tools/upgrade_net_proto_binary.cpp:36

This was tested with most recent master.

[dieterichlawson]

This also happens when I try to load resnet in pycaffe:

>> import caffe
libdc1394 error: Failed to initialize libdc1394
>>> net = caffe.Net("resnet50_deploy.prototxt","resnet50.caffemodel",caffe.TEST)
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0229 22:39:25.116803 17707 upgrade_proto.cpp:65] Attempting to upgrade input file specified using deprecated input fields: resnet50_deploy.prototxt
I0229 22:39:25.116905 17707 upgrade_proto.cpp:68] Successfully upgraded file specified using deprecated input fields.
W0229 22:39:25.116946 17707 upgrade_proto.cpp:70] Note that future Caffe releases will only support input layers and not input fields.
...
I0229 22:39:25.613426 17707 net.cpp:274] Network initialization done.
I0229 22:39:25.805191 17707 upgrade_proto.cpp:65] Attempting to upgrade input file specified using deprecated input fields: resnet50.caffemodel
Segmentation fault (core dumped)

[shelhamer]

I've confirmed this with master on Ubuntu 15.04 and suspect I somehow messed this up in #3211. Everything does work as expected prior to that PR. Will investigate.

[dieterichlawson]

FYI I checked out fe0f441 (the latest commit before the changes to input layers) and was able to load resnet without issue in pycaffe. Thanks for the prompt response!

[shelhamer]

@dieterichlawson check out #3751 to confirm the fix.

[wangxinyue11]

Have you solved it? I have encountered the same problem as you.