Share my experience OSX 10.10 Yosemite Boost 1.55.0 NDK r10e

[ZhengRui]

0, the way to build is

./build.py path_to_your_ndk_folder

1, you need to download Boost by yourself and put it inside the Boost-for-Android folder, for details of building Boost-for-Android check Boost-for-Android share my experience.

2, if Boost-for-Android is successfully built, then you have files like "libboost_xx-gcc-mt-1_55.a" in "Boost-for-Android/build/lib" folder, now when you build again, you may have

warnings:

Android NDK: WARNING:/Users/ABC/Work/Libs/caffe-android-lib/caffe-mobile/jni/Android.mk:caffe: non-system libraries in linker flags: -lboost_random-gcc-mt-1_55 -lboost_math_tr1-gcc-mt-1_55 -lboost_system-gcc-mt-1_55 -lboost_thread-gcc-mt-1_55 -lboost_date_time-gcc-mt-1_55    
Android NDK:     This is likely to result in incorrect builds. Try using LOCAL_STATIC_LIBRARIES    
Android NDK:     or LOCAL_SHARED_LIBRARIES instead to list the library dependencies of the    
Android NDK:     current module

and errors:

[armeabi-v7a] Compile++ arm  : caffe <= math_functions_eigen.cpp
[armeabi-v7a] Compile++ arm  : caffe <= upgrade_proto.cpp
[armeabi-v7a] SharedLibrary  : libcaffe.so
./boost/exception/detail/type_info.hpp:68: error: undefined reference to 'std::type_info::before(std::type_info const&) const'
./boost/exception/detail/type_info.hpp:68: error: undefined reference to 'std::type_info::before(std::type_info const&) const'
./boost/exception/detail/type_info.hpp:68: error: undefined reference to 'std::type_info::before(std::type_info const&) const'
./boost/exception/detail/type_info.hpp:68: error: undefined reference to 'std::type_info::before(std::type_info const&) const'
./boost/atomic/detail/lockpool.hpp:36: error: undefined reference to 'boost::atomics::detail::lockpool::get_lock_for(void const volatile*)'
./boost/atomic/detail/lockpool.hpp:36: error: undefined reference to 'boost::atomics::detail::lockpool::get_lock_for(void const volatile*)'
./boost/atomic/detail/lockpool.hpp:36: error: undefined reference to 'boost::atomics::detail::lockpool::get_lock_for(void const volatile*)'
./boost/atomic/detail/lockpool.hpp:36: error: undefined reference to 'boost::atomics::detail::lockpool::get_lock_for(void const volatile*)'
collect2: error: ld returned 1 exit status
make: *** [/Users/ABC/Work/Libs/caffe-android-lib/caffe-mobile/obj/local/armeabi-v7a/libcaffe.so] Error 1

The reason for the warning is because the Boost-for-Android libraries built right now are static libraries, but in "caffe-mobile/jni/Android.mk" file, it uses "LOCAL_LDLIBS += ..." to link these static libraries,

#LOCAL_LDLIBS += -lboost_random-gcc-mt-1_55 \                                      
                -lboost_math_tr1-gcc-mt-1_55 \                                     
                -lboost_system-gcc-mt-1_55 \                                       
                -lboost_thread-gcc-mt-1_55 \                                       
                -lboost_date_time-gcc-mt-1_55

which i guess could be wrong. So i comment these lines and the correct way to link static libraries in Android.mk file is first:

include $(CLEAR_VARS)                                                           
LOCAL_MODULE := ANameYouLike                                                    
LOCAL_SRC_FILES := PathToTheStaticLibFolder/lib***.a
include $(PREBUILT_STATIC_LIBRARY)

then link to it:

include $(CLEAR_VARS)
LOCAL_STATIC_LIBRARIES := ANameYouLike
include $(BUILD_SHARED_LIBRARY)

Above gives an example to link to one static library, now we need to link to multiple static libraries, then have to repeat

include $(CLEAR_VARS)                                                           
LOCAL_MODULE := ANameYouLike                                                    
LOCAL_SRC_FILES := PathToTheStaticLibFolder/lib***.a
include $(PREBUILT_STATIC_LIBRARY)

multiple times and then link to all of them

include $(CLEAR_VARS)
LOCAL_STATIC_LIBRARIES += ANameYouLike1 ANameYouLike2
include $(BUILD_SHARED_LIBRARY)

So finally the corresponding part in my Android.mk file are:

# C++ full library                                                              
# =======================================================                       
## Import Protocolbuffer                                                        
include $(CLEAR_VARS)                                                           
LOCAL_MODULE := protobuf                                                        
LOCAL_SRC_FILES = $(LOCAL_PATH)/../../protobuf/obj/local/$(TARGET_ARCH_ABI)/libprotobuf.a
LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/../../protobuf/jni/src                 
include $(PREBUILT_STATIC_LIBRARY)                                              

## Import Boost                                                                 
include $(CLEAR_VARS)                                                                               
LOCAL_MODULE := boost_random                                                    
LOCAL_SRC_FILES := $(LOCAL_PATH)/../../Boost-for-Android/build/lib/libboost_random-gcc-mt-1_55.a
include $(PREBUILT_STATIC_LIBRARY)                                              

include $(CLEAR_VARS)                                                           
LOCAL_MODULE := boost_math_tr1                                                  
LOCAL_SRC_FILES := $(LOCAL_PATH)/../../Boost-for-Android/build/lib/libboost_math_tr1-gcc-mt-1_55.a
include $(PREBUILT_STATIC_LIBRARY)                                              

include $(CLEAR_VARS)                                                           
LOCAL_MODULE := boost_system                                                    
LOCAL_SRC_FILES := $(LOCAL_PATH)/../../Boost-for-Android/build/lib/libboost_system-gcc-mt-1_55.a
include $(PREBUILT_STATIC_LIBRARY)                                                                                                                             

include $(CLEAR_VARS)                                                           
LOCAL_MODULE := boost_thread                                                    
LOCAL_SRC_FILES := $(LOCAL_PATH)/../../Boost-for-Android/build/lib/libboost_thread-gcc-mt-1_55.a
include $(PREBUILT_STATIC_LIBRARY)

include $(CLEAR_VARS)                                                           
LOCAL_MODULE := boost_date_time                                                 
LOCAL_SRC_FILES := $(LOCAL_PATH)/../../Boost-for-Android/build/lib/libboost_date_time-gcc-mt-1_55.a
include $(PREBUILT_STATIC_LIBRARY)                                              

include $(CLEAR_VARS)                                                           
LOCAL_MODULE := boost_atomic                                                    
LOCAL_SRC_FILES := $(LOCAL_PATH)/../../Boost-for-Android/build/lib/libboost_atomic-gcc-mt-1_55.a
include $(PREBUILT_STATIC_LIBRARY)

and

LOCAL_STATIC_LIBRARIES += protobuf                                              
LOCAL_CPPFLAGS += -pthread -fPIC -fexceptions -frtti -std=c++11 -DUSE_EIGEN -DCPU_ONLY

# boost                                                                         
LOCAL_LDLIBS += -lm -llog -L$(LOCAL_PATH)/../../Boost-for-Android/build/lib/    
#LOCAL_LDLIBS += -lboost_random-gcc-mt-1_55 \                                   
                -lboost_math_tr1-gcc-mt-1_55 \                                  
                -lboost_system-gcc-mt-1_55 \                                    
                -lboost_thread-gcc-mt-1_55 \                                    
                -lboost_date_time-gcc-mt-1_55 \                                 
                -lboost_atomic-gcc-mt-1_55                                      
LOCAL_STATIC_LIBRARIES += boost_random boost_math_tr1 boost_system boost_thread boost_date_time boost_atomic

the correct way of linking static libraries will get rid of the warning and errors involved "type_info.hpp", adding link to "libboost_atomic-gcc-mt-1_55.a" will get rid of the error involved "lockpool.hpp".

After these steps, i can finally build successfully with "libcaffe.so" and "libcaffe_jni.so" generated in folder "caffe-mobile/libs/armeabi-v7a".

[sh1r0]

Thanks for sharing your experience. I just noticed that gcc-4.6 toolchain was removed in ndk-r10e, and thus I made some fixes on this. Now, it should be okay to use build.py direclty. Please feel free to let me know if there are any problems.

[ZhengRui]

There are two things which i don't understand now

1, it seems in you build.py file, you are using LOCAL_LDLIBS += -l... to link Boost-for-Android, Do you compile Boost-for-Android to be shared libraries? because i can only compile it to be static libraries and then use the way i mentioned above to link to these static libraries, so i am wondering how you do that?

2, in my MainActivity, i can create an CaffeMobile instance, but when i call enableLog() method, it generates errors, here is my LogCat:

06-12 21:04:37.920 2183-2183/com.rzheng.deepandroid D/dalvikvm﹕ Trying to load lib /data/app-lib/com.rzheng.deepandroid-2/libcaffe.so 0x4357e240
06-12 21:04:37.950 2183-2183/com.rzheng.deepandroid D/dalvikvm﹕ Added shared lib /data/app-lib/com.rzheng.deepandroid-2/libcaffe.so 0x4357e240
06-12 21:04:37.950 2183-2183/com.rzheng.deepandroid D/dalvikvm﹕ No JNI_OnLoad found in /data/app-lib/com.rzheng.deepandroid-2/libcaffe.so 0x4357e240, skipping init
06-12 21:04:37.950 2183-2183/com.rzheng.deepandroid D/dalvikvm﹕ Trying to load lib /data/app-lib/com.rzheng.deepandroid-2/libcaffe_jni.so 0x4357e240
06-12 21:04:37.950 2183-2183/com.rzheng.deepandroid D/dalvikvm﹕ Added shared lib /data/app-lib/com.rzheng.deepandroid-2/libcaffe_jni.so 0x4357e240
06-12 21:04:37.990 2183-2183/com.rzheng.deepandroid I/MainActivity:﹕ Loading caffe successfully
06-12 21:05:40.710 2868-2868/? D/dalvikvm﹕ Trying to load lib /data/app-lib/com.rzheng.deepandroid-1/libcaffe.so 0x43573d20
06-12 21:05:40.740 2868-2868/? D/dalvikvm﹕ Added shared lib /data/app-lib/com.rzheng.deepandroid-1/libcaffe.so 0x43573d20
06-12 21:05:40.740 2868-2868/? D/dalvikvm﹕ No JNI_OnLoad found in /data/app-lib/com.rzheng.deepandroid-1/libcaffe.so 0x43573d20, skipping init
06-12 21:05:40.740 2868-2868/? D/dalvikvm﹕ Trying to load lib /data/app-lib/com.rzheng.deepandroid-1/libcaffe_jni.so 0x43573d20
06-12 21:05:40.740 2868-2868/? D/dalvikvm﹕ Added shared lib /data/app-lib/com.rzheng.deepandroid-1/libcaffe_jni.so 0x43573d20
06-12 21:05:40.780 2868-2868/? I/MainActivity:﹕ Loading caffe successfully
06-12 21:05:40.780 2868-2868/? W/dalvikvm﹕ No implementation found for native Lcom/rzheng/deepandroid/CaffeMobile;.enableLog:(Z)V
06-12 21:05:40.780 2868-2868/? E/AndroidRuntime﹕ FATAL EXCEPTION: main
Process: com.rzheng.deepandroid, PID: 2868
java.lang.UnsatisfiedLinkError: Native method not found: com.rzheng.deepandroid.CaffeMobile.enableLog:(Z)V
at com.rzheng.deepandroid.CaffeMobile.enableLog(Native Method)
at com.rzheng.deepandroid.MainActivity.onCreate(MainActivity.java:29)
at android.app.Activity.performCreate(Activity.java:5249)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1087)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2154)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2239)
at android.app.ActivityThread.access$800(ActivityThread.java:141)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1202)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:136)
at android.app.ActivityThread.main(ActivityThread.java:5047)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:515)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:793)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:609)
at dalvik.system.NativeStart.main(Native Method)

I used the libcaffe.so and libcaffe_jni.so i built by myself, and my package name is com.rzheng.deepandroid, in issue #6, I saw that you mentioned that i have to build the library by myself instead of directly use the .so files in your caffe-android-demo project, but i use the .so files built by myself. I noticed that there is a '-L' option in build.py, does it have something to do with that? what could be the reasons?

[ZhengRui]

My second question solved, have to change the 3 methods name in caffe_jni.cpp file to the names corresponds to my Android Project name and compile caffe-mobile again, now it works. But i am wondering can you make caffe_jni.cpp file more general, otherwise i have to change it and built it again each time i create a new Android project?

[sh1r0]

In the previous commit (6604a37), static boost libs can be linked simply by using LOCAL_LDLIBS (with warnings). However, boost is not required In the latest commit (c021b34)。
This repo is aim at providing a ported Caffe as a (general) lib on Android, where caffe_jni.cpp is no more than an example.
Trying to wrap all you need within jni and provide general interfaces for java is recommended. The jni part should be as general as possible and independent to your android projects.

[bhack]

@ZhengRui For more general jni coverage see bytedeco/javacpp-presets#34.