Hello, can I get some help?

[Ujassi]

First of all, thank you for creating the yolo-nas openvino code.
I'm testing your code.
I have two questions.

1. A memory leak occurs. I can't find a way.
2. If you use ptq using NNCF, detection does not work.
   Even if you use yolo-nas ptq qat, detection does not occur.
   NNCF compress_weight works well.

Is there a solution?

Thank you. Have a good day

[Y-T-G]

1. Is it a big leak? It could he due to OpenCV.
2. I think that needs some modifications for it to work. I didn't try it.

[Ujassi]

First of all, thank you so much for the answer.

1. It seems that memory leaks do not occur continuously. There is no problem with testing.

2. Due to my limited knowledge, I cannot find what needs to be modified to apply the NNCF PTQ. If you have time, I would appreciate it if you could check it out later.

Thank you again for your reply.

[Y-T-G]

1. It could be OpenCV related: memory leak with videocapture, memory goes up with the function of read(image) opencv/opencv#5715

2. Can you try adding these after the line
   

   YOLO-NAS-OpenVino-cpp/src/yolo_nas.cpp

   Line 51 in 2856838

   ppp.input().model().set_layout("NCHW");

ppp.output(0).tensor().set_element_type(ov::element::f32);
ppp.output(1).tensor().set_element_type(ov::element::f32);

[Ujassi]

Thank you for your answer.

1. I don't use VideoCapture, I use images.
2. Even if you enter the code you mentioned, detection does not occur.
   1. If xxx.bin xxx.xml is created with f32, it works well.
   2. If xxx.bin xxx.xml is created with f16, it works well.
   3. Make xxx.bin xxx.xml as f32,
      Following https://github.com/openvinotoolkit/nncf/tree/develop?tab=readme-ov-file, we conducted PTQ with the code below. However, detection does not occur.

import nncf
import openvino.runtime as ov
import torch
from torchvision import datasets, transforms

model = ov.Core().read_model("/model_path")

val_dataset = datasets.ImageFolder("/path", transform=transforms.Compose([transforms.ToTensor()]))
dataset_loader = torch.utils.data.DataLoader(val_dataset, batch_size=1)

def transform_fn(data_item):
images, _ = data_item
return images

calibration_dataset = nncf.Dataset(dataset_loader, transform_fn)

quantized_model = nncf.quantize(model, calibration_dataset)

ov.save_model(quantized_model, "/model_path")

4. PTQ and QAT were conducted following https://github.com/naseemap47/YOLO-NAS. But it doesn't work.

Could I possibly get some help?

Thank you so much.

[Y-T-G]

1. How does it leak for images? Doesn't it stop after each image?
2. You can try checking the output casting. I think it has something to do with going from int to float.

[Ujassi]

1. It seems to occur only once when creating an object of the YoloNAS class. I will check this part a little more.
2. I am thinking similarly, but I still can't find a way.

Thank you so much for your reply.