How to use BestOf2NearestMatcher apply2()?

[msmartin4470]

This is probably a simple answer but I am new to javacv. I am translating some python to javacv and I am having issues calling the apply2 function of the BestOf2NearestMatcher. I am trying to pass a list of ImageFeatures as the first parameter but I am getting a type mismatch error. The documentation for apply2 says it accepts StdVector opencv_stitching.ImageFeatures. How can I convert my List to properly pass to the apply2 function?

Here is my function

fun matchFeatures(features: List<ImageFeatures>) : MatchesInfo {
    var matcher = BestOf2NearestMatcher.create()
    var pairwise_matches = MatchesInfo()
    matcher.apply2(features, pairwise_matches)
    return pairwise_matches
}

[saudet]

It's a bit hard to use std::vector with Pointer objects like that, so I've mapped those to CameraParamsVector, ImageFeaturesVector, and MatchesInfoVector in commit bytedeco/javacpp-presets@7882024. Please give it a try with the snapshots: http://bytedeco.org/builds/

[msmartin4470]

Thank you. I was able to get it to build successfully using the snapshot, however I am getting a link error when I run the application. I would assume this is an issue with my build.gradle file but I am not. Do you have any suggestions?

E/AndroidRuntime(23703): java.lang.UnsatisfiedLinkError: No implementation found for void org.bytedeco.opencv.opencv_stitching.ImageFeaturesVector.allocate() (tried Java_org_bytedeco_opencv_opencv_1stitching_ImageFeaturesVector_allocate and Java_org_bytedeco_opencv_opencv_1stitching_ImageFeaturesVector_allocate__)

dependencies {
    implementation group: 'org.bytedeco', name: 'javacv', version: '1.5.9-SNAPSHOT'
    javacpp group: 'org.bytedeco', name: 'opencv-platform', version: '4.5.3-1.5.6'
    javacpp group: 'org.bytedeco', name: 'ffmpeg-platform', version: '4.4-1.5.6'
}

[saudet]

You'll need to use the snapshots for the others as well

[msmartin4470]

I have updated those as well and still have the same issues. Is there perhaps an additional class that I need to be loading with the Loader? Thank you for all your help.

implementation group: 'org.bytedeco', name: 'javacv', version: '1.5.9-SNAPSHOT'
javacpp group: 'org.bytedeco', name: 'opencv-platform', version: '4.6.0-1.5.9-SNAPSHOT'
javacpp group: 'org.bytedeco', name: 'ffmpeg-platform', version: '6.0-1.5.9-SNAPSHOT'

//this is all I have currently
Loader.load(org.bytedeco.opencv.opencv_java::class.java)

[saudet]

The version for OpenCV is 4.7.0, not 4.6.0, that will not work, that's normal.

[msmartin4470]

Thank you. I have updated all the versions, but I am still having the same issue. Am I just using this the wrong way?

Here is my function

  fun matchFeatures(features: List<ImageFeatures>) : List<MatchesInfo> {
      var matcher = BestOf2NearestMatcher.create()
      var pairwise_matches = MatchesInfoVector()
      var featureVector = ImageFeaturesVector()
      features.forEach{
          featureVector.push_back(it)
      }
      matcher.apply2(featureVector, pairwise_matches)
      return pairwise_matches.get().toList()
  }

The error I am getting is from the MatchesInfoVector() on the third line

E/AndroidRuntime(14836): java.lang.UnsatisfiedLinkError: No implementation found for void org.bytedeco.opencv.opencv_stitching.MatchesInfoVector.allocate() 

[saudet]

Looks like openblas is missing from your dependencies. That's probably why it doesn't work.

[msmartin4470]

Thank you. Still running into the same issues with openblas as a dependency.

dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
    implementation group: 'org.bytedeco', name: 'javacv', version: '1.5.9-SNAPSHOT'
    javacpp group: 'org.bytedeco', name: 'opencv-platform', version: '4.7.0-1.5.9-SNAPSHOT'
    javacpp group: 'org.bytedeco', name: 'openblas-platform', version: '0.3.23-1.5.9-SNAPSHOT'
    javacpp group: 'org.bytedeco', name: 'ffmpeg-platform', version: '6.0-1.5.9-SNAPSHOT'
}

[saudet]

Please set the "org.bytedeco.javacpp.logger.debug" system property to "true" to get more information on the console.

[msmartin4470]

log.txt

Attached is the additional log information.

[saudet]

It just looks it's not finding the libraries anywhere. What is the content of your APK file?

[msmartin4470]

I could be reading it wrong but it seems like it is loading org.bytedeco.opencv.opencv_stitching successfully.

[saudet]

Are you sure the versions match?

[msmartin4470]

Thank you. I believe they are this is from my build.gradle. I pulled the versions from
https://oss.sonatype.org/content/repositories/snapshots/org/bytedeco/

dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"

    implementation group: 'org.bytedeco', name: 'javacv', version: '1.5.9-SNAPSHOT'
    javacpp group: 'org.bytedeco', name: 'opencv-platform', version: '4.7.0-1.5.9-SNAPSHOT'
    javacpp group: 'org.bytedeco', name: 'openblas-platform', version: '0.3.23-1.5.9-SNAPSHOT'
    javacpp group: 'org.bytedeco', name: 'ffmpeg-platform', version: '6.0-1.5.9-SNAPSHOT'
}

[saudet]

Yes, but please check the files manually

[msmartin4470]

Thank you. What files specifically are you looking for? I could not find any version information on the .so files. I have cleaned out the build directory and done a fresh build to try to ensure the latest version and still having the same issue.

[saudet]

Maybe it's a problem with the CI. I've restarted the last builds for android-arm64 and they should get redeployed correctly. Please purge your local cache and try again.

[msmartin4470]

Thank you for all your help. I believe I have this working now. As I continued to test I ran into another roadblock that seems similar to my original question. After getting the features and matches I am subsetting with leaveBiggestComponent as shown below. For this the return type is an IntPointer and I am having trouble converting this to an IntVector so I can use it in a list type. It looks like IntVector has a constructor that takes a pointer, but when I use that it causes an out of memory exception. Any suggestions?

 var indices = org.bytedeco.opencv.global.opencv_stitching.leaveBiggestComponent(featuresVector, matchesVector, confidenceThreshold)

Thanks

[saudet]

We can call get() on that as well to convert it to an array.

[msmartin4470]

Thank you. Is there a reason this cannot me called with a single channel grayscale image?

org.bytedeco.opencv.global.opencv_stitching.computeImageFeatures2(detector, image, features)

[msmartin4470]

What is the best way to try to get help with this. Everything compiles and seems to be running ok, so I believe the upgrade that you made is good. I have code working in python that I need to replicate in java, and there seems to be differences in the outputs from JavaCV vs OpenCV for python. For example, with JavaCV when I compute key points I get different key points every time it runs this does not seem correct to me. Thank you in advance.

    fun detectFeatures(image: Mat) : ImageFeatures {
        var detector = ORB.create()
        val features = ImageFeatures()
        org.bytedeco.opencv.global.opencv_stitching.computeImageFeatures2(detector, image, features)
        return features
    }

[saudet]

JavaCV uses the C++ API, so if you get different results on each run like that, it indicates the memory is probably getting corrupted somewhere. In particular, you'll need to make sure nothing gets deallocated prematurely.

[msmartin4470]

I am getting crashes when calling the apply function for my camera estimator do you have any suggestions?

var estimator = HomographyBasedEstimator() 
var success = estimator.apply(featureVector, matchesVector, camerasVector)

Thanks,
Mike

[saudet]

If you have working code in C++, I can probably tell you what is wrong with your code.

[saudet]

BTW, one way to make sure nothing gets deallocated prematurely is to keep a reference of everything in fields.