pixSauvolaBinarize keep crashing #272
Comments
|
You're not freeing memory allocated by pixSauvolaBinarize(), so it's
probably just running out of memory.
|
|
So this "binarizedPix.destroy();" doens't free the allocated memory? |
|
No, that only works when memory is allocated with one of the factory
methods.
|
|
Ok, thanks. What is the correct way to free the memory then? pixDestroy()? |
|
pixDestroy() I guess.
|
|
Still crashing with pixDestroy().. |
|
Could you create a small standalone program to reproduce the crash? |
|
BTW, since we're not using the PIX binarizedPix = new PIX(null);
pixSauvolaBinarize(pix, 14, factor, 1, null, null, null, binarizedPix);
// do something with binarizedPix
pixDestroy(binarizedPix); |
|
Here is a program which crashes (after 10s running on my mac): package test;
import static org.bytedeco.javacpp.lept.pixSauvolaBinarize;
import static org.bytedeco.javacpp.lept.pixDestroy;
import java.awt.image.BufferedImage;
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import static java.util.stream.Collectors.toList;
import org.bytedeco.javacpp.lept.PIX;
import org.bytedeco.javacv.Java2DFrameConverter;
import org.bytedeco.javacv.LeptonicaFrameConverter;
import org.datavec.image.data.ImageWritable;
import org.datavec.image.loader.NativeImageLoader;
public class SauvolaTester {
public static void main(String[] args) throws Exception {
NativeImageLoader originalImageLoader = new NativeImageLoader();
ImageWritable originalImage = originalImageLoader.asWritable(new File(args[0]));
List<ImageWritable> images = new ArrayList<>();
for (int i = 0; i < 8; i++) {
images.add(originalImage);
}
int count = 0;
while (true) {
images.parallelStream().map(s -> binarize(s, 0.4f)).collect(toList());
count++;
if (count % 10 == 0) {
System.out.println(count + " done.");
}
}
}
private static BufferedImage binarize(ImageWritable image, float factor) {
LeptonicaFrameConverter leptonicaFrameConverter = new LeptonicaFrameConverter();
PIX pix = leptonicaFrameConverter.convert(image.getFrame().clone());
//PIX pixTmp = null;
PIX binarizedPix = new PIX(null);
pixSauvolaBinarize(pix, 14, factor, 1, null, null, null, binarizedPix);
BufferedImage result = fromBinarizedPix(binarizedPix);
pixDestroy(binarizedPix);
return result;
}
private static BufferedImage fromBinarizedPix(PIX pix) {
LeptonicaFrameConverter leptonicaFrameConverter = new LeptonicaFrameConverter();
return new Java2DFrameConverter().convert(leptonicaFrameConverter.convert(pix));
}
}gradle: apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'application'
sourceCompatibility = 1.8
group = 'com.test.tester'
mainClassName = "test.SauvolaTester"
allprojects {
group = 'com.test.tester'
repositories {
mavenCentral()
}
eclipse {
classpath {
// customizing the classes output directory:
defaultOutputDir = file('build-eclipse')
// default settings for dependencies sources/javadoc download:
downloadSources = true
downloadJavadoc = false
file {
//closure executed after .project content is loaded from existing file
//and after gradle build information is merged
whenMerged { classpath ->
classpath.entries.removeAll { entry -> entry.path.endsWith('.bin') }
}
}
}
}
dependencies {
compile group: 'org.deeplearning4j', name: 'deeplearning4j-core', version: '1.0.0-beta2'
compile group: 'org.deeplearning4j', name: 'deeplearning4j-nlp', version: '1.0.0-beta2'
compile group: 'org.bytedeco', name: 'javacpp', version: '1.4.2'
compile group: 'org.bytedeco', name: 'javacv-platform', version: '1.4.2'
}
}
|
|
hs_err_pid.log: |
|
|
|
Still crashing when I changed the main method to this: public static void main(String[] args) throws Exception {
List<ImageWritable> images = new ArrayList<>();
for (int i = 0; i < 8; i++) {
NativeImageLoader imageLoader = new NativeImageLoader();
ImageWritable image = imageLoader.asWritable(new File(args[0]));
images.add(image);
}
int count = 0;
while (true) {
images.parallelStream().map(s -> binarize(s, 0.4f)).collect(toList());
count++;
if (count % 100 == 0) {
System.out.println(count + " done." );
}
}
} |
|
By the way it doesn't crash if I change binarize method to synchronized, but this is what I dont want because no multiple binarization operations can be done in parallel then. |
|
Try something like this:
images.add(image.getFrame().clone());
|
|
I think this image.getFrame().clone() is already done inside binarize() method: PIX pix = leptonicaFrameConverter.convert(image.getFrame().clone()); Anyway, I changed the program to public static void main(String[] args) throws Exception {
NativeImageLoader imageLoader = new NativeImageLoader();
ImageWritable image = imageLoader.asWritable(new File(args[0]));
List<Frame> images = new ArrayList<>();
for (int i = 0; i < 8; i++) {
images.add(image.getFrame().clone());
}
int count = 0;
while (true) {
images.parallelStream().map(s -> binarize(s, 0.4f)).collect(toList());
count++;
if (count % 100 == 0) {
System.out.println(count + " done." );
}
}
}
private static BufferedImage binarize(Frame image, float factor) {
LeptonicaFrameConverter leptonicaFrameConverter = new LeptonicaFrameConverter();
PIX pix = leptonicaFrameConverter.convert(image.clone());
PIX binarizedPix = new PIX(null);
pixSauvolaBinarize(pix, 14, factor, 1, null, null, null, binarizedPix);
BufferedImage result = fromBinarizedPix(binarizedPix);
pixDestroy(binarizedPix);
return result;
}
private static BufferedImage fromBinarizedPix(PIX pix) {
LeptonicaFrameConverter leptonicaFrameConverter = new LeptonicaFrameConverter();
return new Java2DFrameConverter().convert(leptonicaFrameConverter.convert(pix));
}but it doesn't help. |
|
The second image.clone() might actually not work, try to remove it.
|
|
BTW, can you reproduce that only with NativeImageLoader? Or does it happen
even without NativeImageLoader?
|
|
I tried to remove second image.clone() and it doesn't help. When I didn't use NativeImageLoader and replaced it by pixRead, it doesn't crash anymore. So there is something wrong with LeptonicaFrameConverter? |
|
Hum, it's possible, can you try to confirm that?
|
|
Also, does it happen only with some images? Or any image?
|
|
Any image. |
|
Ok, so the problem isn't with pixSauvolaBinarize(), so let's try to narrow
down this a bit more.
|
|
It's possible for the JVM to deallocate LeptonicaFrameConverter and it's buffers after convert(), but before pixSauvolaBinarize() returns. Could you put those objects, for example, in thread-local storage and see what happens? |
|
Interesting and perhaps you are right. After adding an extra convert() it doesn't crash anymore: private static BufferedImage binarize(ImageWritable image, float factor) {
LeptonicaFrameConverter leptonicaFrameConverter = new LeptonicaFrameConverter();
PIX pix = leptonicaFrameConverter.convert(image.getFrame().clone());
PIX binarizedPix = new PIX(null);
pixSauvolaBinarize(pix, 14, factor, 1, null, null, null, binarizedPix);
BufferedImage result = fromBinarizedPix(binarizedPix);
pixDestroy(binarizedPix);
Frame frame = null;
leptonicaFrameConverter.convert(frame);
return result;
} |
…revents premature deallocation (issue bytedeco/javacpp#272)
|
Thanks for testing! I've fixed this in the commits above. It should work as expected now. |
|
The fix has been released with JavaCPP and JavaCV 1.4.4. Thanks for reporting and for testing! |
javacpp version : '1.4.3'
I have run this code with multiple threads and it always crashes after few hours:
hs_err_pid.log:
I have also tried with v1.4.1 and 1.4.2 without success.
The text was updated successfully, but these errors were encountered: