Caffe preset build error: sun.reflect.annotation.TypeNotPresentExceptionProxy

[sebastianlutter]

I try to build caffe presets on Ubuntu 14.04 x86_64 but encounter errors with the java presets code. As far as I can understand it failes to find a annotation type.

• create javacpp-1.1_snapshot

  git clone https://github.com/bytedeco/javacpp
  cd javacpp
  mvn install

• install build environment for natice caffe / opencv

  apt-get install cmake gfortran

• Build and install opencv

  ./cppbuild.sh install opencv
  mvn -U install --projects .,opencv

• Because I want a CPU only build I added CPU_ONLY to caffe/cppbuild.sh for platform linux-x86_64

• Build and install caffe

./cppbuild.sh install opencv
mvn -U install --projects .,opencv -X

Then the following error happens:

[ERROR] Failed to execute JavaCPP Builder: sun.reflect.annotation.TypeNotPresentExceptionProxy
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] JavaCPP Presets ................................... SUCCESS [0.601s]
[INFO] JavaCPP Presets for Caffe ......................... FAILURE [0.813s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.120s
[INFO] Finished at: Tue Oct 06 18:08:10 CEST 2015
[INFO] Final Memory: 26M/438M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.bytedeco:javacpp:1.1-SNAPSHOT:build (generate-sources) > on project caffe: Failed to execute JavaCPP Builder: sun.reflect.annotation.TypeNotPresentExceptionProxy -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.bytedeco:javacpp:1.1-SNAPSHOT:build (generate-sources) on project caffe: Failed to execute JavaCPP Builder
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
[ ... ]

What should I do now? Wait until the presets are fixed? Or find a way to debug the Annotation that causes the problem?

Or do only I have that problem and it builds well for you?

Thanks
Sebastian

[saudet]

Run "mvn clean" before install :)

[sebastianlutter]

Already tried that, does not work:

mvn clean && mvn -U install --projects .,caffe -X

Does it build on your box?

Here is a full log (Ubuntu 14.04 x86_64): https://dpaste.de/qRj8

The script caffe/cppbuild.sh uses the master branch of caffe git, maybe the caffe sources have changed over time?

[sebastianlutter]

I tried to debug the issue to get more insight.

The error is thrown in class org.bytedeco.javacpp.tools.Builder within the while condition

public void load(Class cls, boolean inherit) {
   Class<?> c = Loader.getEnclosingClass(cls);
   LinkedList<Class> classList = new LinkedList<Class>();
  classList.addFirst(c);
   while (!c.isAnnotationPresent(org.bytedeco.javacpp.annotation.Properties.class)
           && !c.isAnnotationPresent(Platform.class) && c.getSuperclass() != null
           && c.getSuperclass() != Object.class) {
       // accumulate superclasses to process native methods from those as well
       classList.addFirst(c = c.getSuperclass());
   }

This causes the exception: (Breakpoint at line 132)

!c.isAnnotationPresent(org.bytedeco.javacpp.annotation.Properties.class)

When I look at the variables while debugging it happens when the org.bytedeco.javacpp.tools.Builder class is called with org.bytedeco.javacpp.preset.caffe class.

The exception itself is thrown in sun.reflect.annotation.AnnotationParser.parseAnnotations(...) but I have no JDK source so I cannot debug it more deeper

• see http://www.docjar.com/html/api/sun/reflect/annotation/AnnotationParser.java.html

62 public static Map<Class, Annotation> parseAnnotations( 63 byte[] rawAnnotations, 64 ConstantPool constPool, 65 Class container) {

• rawAnnotations is an array of bytes (numbers)
• constPool is a ConstantPool object containing the name org.bytedeco.javacpp.preset.caffe
• container is a Class of type org.bytedeco.javacpp.presets.caffe

I can imaginge multiple reasons for the behavior I get:

(1.) An Annotation in org.bytedeco.javacpp.preset.caffe is not compatible/wrong with actual javacpp-1.1_snapshot (the preset if buggy)
(2.) Since the exception is thrown while Builder loads the preset class maybe the native caffe build does no longer fit the java preset file? I noticed that in cppbuild.sh it uses master branch version of caffe. It would help if it is set to a defined caffe commit that the preset is known to build with.
(3.) Maybe java preset annotations and native build are ok, and the error happens because it simply cannot find the Annotation classes in classpath? The error happens when it checks if Properties annotation is not present in the given Class<org.bytedeco.javacpp.preset.caffe> file c (see the condition in while loop). And this check finally throws the TypeNotPresentExceptionProxy exception.

As my first post shows I can build the opencv preset in the same project, and it does not throw an error like this. So I guess the javacpp-presets project classpath seems to be ok and javacpp presets are found by opencv module.

Can anyone confirm if the caffe preset builds in the current state on linux-x86_64? Or is this some kind of known problem that it does not compile?

[saudet]

Yes, I've just tried updating Caffe, and it still builds fine here. I'm pretty sure it's just your system picking up old classes from somewhere. Make sure to clean everything up, repull, and rebuild from scratch.

[sebastianlutter]

ok. I will do the following steps and report the result here. The only difference is that I build it on linux-x86_64 with export CPU_ONLY = 1 added.

// System: Ubuntu 14.04 x86_64
apt-get install gfortran cmake

git clone https://github.com/bytedeco/javacpp.git
cd javacpp
mvn install -U

git clone https://github.com/bytedeco/javacpp-presets.git
cd javacpp-presets

./cppbuild.sh install opencv

vim caffe/cppbuild.sh # add CPU_ONLY to linux-x86_64 (headless system)
./cppbuild.sh install caffe

mvn install --projects .,opencv,caffe -U

[sebastianlutter]

Thanks for your help, got a different behavior now: (mvn install --projects .,opencv,caffe -U)

[ ... ]
[INFO] Successfully executed JavaCPP Builder
[INFO]
[INFO] --- maven-dependency-plugin:2.10:copy-dependencies (copy-dependencies) @ caffe ---
[INFO] org.bytedeco.javacpp-presets:opencv:jar:3.0.0-1.1-SNAPSHOT already exists in destination.
[INFO] org.bytedeco:javacpp:jar:1.1-SNAPSHOT already exists in destination.
[INFO]
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ caffe ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @ caffe ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/target/classes
[INFO] /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/src/main/java/org/bytedeco/javacpp/caffe.java: /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/src/main/java/org/bytedeco/javacpp/caffe.java uses unchecked or unsafe operations.
[INFO] /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/src/main/java/org/bytedeco/javacpp/caffe.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- javacpp:1.1-SNAPSHOT:build (process-classes) @ caffe ---
[INFO] Executing JavaCPP Builder
[INFO] Generating /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/target/classes/org/bytedeco/javacpp/jnicaffe.cpp
[INFO] Compiling /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/target/classes/org/bytedeco/javacpp/linux-x86_64/libjnicaffe.so
[INFO] g++ -I/data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/../opencv/target/classes/org/bytedeco/javacpp/ -I/data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/../opencv/cppbuild/linux-x86_64/include/opencv/ -I/data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/../opencv/cppbuild/linux-x86_64/include/ -I/data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/cppbuild/linux-x86_64/include/ -I/usr/lib/jvm/java-8-oracle/include -I/usr/lib/jvm/java-8-oracle/include/linux /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/target/classes/org/bytedeco/javacpp/jnicaffe.cpp -Wl,-rpath,$ORIGIN/ -march=x86-64 -m64 -Wall -O3 -fPIC -shared -s -o /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/target/classes/org/bytedeco/javacpp/linux-x86_64/libjnicaffe.so -L/data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/../opencv/cppbuild/linux-x86_64/lib/ -Wl,-rpath,/data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/../opencv/cppbuild/linux-x86_64/lib/ -L/data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/cppbuild/linux-x86_64/lib/ -Wl,-rpath,/data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/cppbuild/linux-x86_64/lib/ -lcaffe -lopencv_highgui -lopencv_videoio -lopencv_imgcodecs -lopencv_imgproc -lopencv_core
In file included from /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/cppbuild/linux-x86_64/include/caffe/common.hpp:19:0,
from /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/cppbuild/linux-x86_64/include/caffe/blob.hpp:8,
from /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/cppbuild/linux-x86_64/include/caffe/caffe.hpp:7,
from /data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/target/classes/org/bytedeco/javacpp/jnicaffe.cpp:92:
/data/projekte/2015_10_06__JavaCPP_caffe_presets/javacpp/javacpp-presets/caffe/cppbuild/linux-x86_64/include/caffe/util/device_alternate.hpp:34:23: fatal error: cublas_v2.h: Datei oder Verzeichnis nicht gefunden
#include < cublas_v2.h>
^
compilation terminated.

It obviously is missing cuda.

Caffe itself builds and runs well with CPU_ONLY on a headless machine. I'll try to find a box with linux-x86_64 and cuda installed, but I really need a build that can be used in headless environments.

Is cuda a mandatory dependency for the caffe preset despite the CPU_ONLY flag in cppbuild.sh?

[saudet]

No, the build system just isn't very user-friendly. It looks like we need to set some define macro manually. I'll look into that a bit later, unless you figure out something before ;)

[saudet]

There, should be fixed in latest commit. Thanks for reporting!

[sebastianlutter]

Thanks for the fix, I'll try it and report here.

btw:
I tested to build on Ubuntu x86_64 with Cuda 7.5 installed, and it worked well to build the caffe preset and its native part

[sebastianlutter]

Confirm that the bug is fixed. Thanks a lot :)