lower accuracy when i use djl api to inference

[mengpengfei]

djl api：
BufferedImage read = ImageIO.read(new File("D:\code\SiameseNetwork\a.jpg"));
BufferedImage bufferedImage = toGray(read);
BufferedImage subimage = bufferedImage.getSubimage(0, 0, 60, 60);

NDArray imageArray =
ImageFactory.getInstance()
.fromImage(subimage)
.toNDArray(manager,Image.Flag.GRAYSCALE)
.expandDims(0).transpose(new int[]{0, 3, 1, 2})
.toType(DataType.FLOAT32, true);
python api：

img=cv2.imread(str, cv2.IMREAD_UNCHANGED)
gray=cv2.cvtColor(img,cv2.IMREAD_COLOR)
cropped = gray[0:config.input_h, 0:int(config.input_w/2)]
image0 = torch.from_numpy(np.asarray(cropped)).type(torch.FloatTensor).to(DEVICE)

i got the values of image0 by python api,and i got the values of imageArray by djl api
but the values of imageArray is diffrent with the values of image0

This problem leads to lower accuracy when i use djl api

[mengpengfei]

public static BufferedImage toGray(BufferedImage srcImg){
return new ColorConvertOp(ColorSpace.getInstance(ColorSpace.CS_GRAY), null).filter(srcImg, null);
}

[lanking520]

Can you try to test with dummy NDArray? Like ones in numpy and DJL?

NDManager.ones()

If these result are consistent. then it means the preprocessing we have done on PyTorch and DJL are different.

Another way to verify that, is to compare the tensor/NDArray before we feed for inference.

[frankfliu]

@mengpengfei
Can you try this in DJL:

            Image img = ImageFactory.getInstance().fromFile(imageFile);
            NDArray array = img.getSubimage(0, 0, 60, 60)
                    .toNDArray(manager, Image.Flag.GRAYSCALE)
                    .expandDims(0).transpose(0, 3, 1, 2)
                    .toType(DataType.FLOAT32, true);

Your toGray() will re-render the image with shifted color space,

[frankfliu]

@mengpengfei
I tested a bit on cv2 and DJL, the color image are most likely identical. The grayscale image are sightly different:

1. DJL takes simple average: (R + G + B) / 3.
2. OpenCV uses: 0.299 R + 0.587 G + 0.114 B, see documentation