How to retrieve the absolute coordinates of the predicted bounding boxes

[ghost]

I want to save bounding box coordinates of highlighted text how can I save them
my original image

and after prediction

[fg-mindee]

Hi @hammadyounas2008 👋

Would you mind getting a bit more specific? I think you asked the same question in #561 and I provided an answer. Please check the discussions & issues to see if there is an answer to your question before posting it 🙏

If I'm mistaking, please provide more details: python code snippet that you used, what you expect (a saved file locally, a python dict?)

[fg-mindee]

No worries 👌
Did my last edit of the snippet solve your absolute coordinate retrieval problem by the way?

[ghost]

No , when I draw the values on the image it does not draw on a specific word

export = result.export()
# Flatten the export
page_words = [[word for block in page['blocks'] for line in block['lines'] for word in line['words']] for page in export['pages']]
page_dims = [page['dimensions'] for page in export['pages']]
# Get the coords in [xmin, ymin, xmax, ymax]
words_abs_coords = [
    [[int(round(word['geometry'][0][0] * dims[0])), int(round(word['geometry'][0][1] * dims[1])), int(round(word['geometry'][1][0] * dims[0])), int(round(word['geometry'][1][1] * dims[1]))] for word in words]
    for words, dims in zip(page_words, page_dims)

]
print(words_abs_coords)

Following are the absolute coordinates

[[[33, 108, 57, 135], [54, 107, 81, 136], [189, 110, 205, 141], [205, 112, 221, 141], [222, 114, 230, 141], [230, 112, 247, 141], [11, 173, 39, 196], [41, 175, 68, 196], [71, 175, 87, 198], [90, 177, 116, 198], [215, 179, 256, 199], [26, 204, 35, 225], [10, 203, 25, 227], [89, 204, 131, 228], [214, 207, 256, 227], [54, 228, 57, 236], [11, 225, 38, 246], [41, 224, 53, 247], [90, 225, 129, 245], [11, 244, 42, 265], [45, 245, 64, 267], [82, 246, 102, 267], [67, 246, 79, 268], [104, 247, 127, 268], [13, 301, 87, 324], [90, 303, 113, 323], [12, 327, 60, 349], [63, 331, 69, 347], [84, 331, 125, 349], [70, 328, 80, 351], [214, 334, 259, 356], [61, 360, 108, 378], [41, 357, 59, 382], [130, 360, 160, 381], [111, 359, 128, 382], [214, 362, 282, 386], [41, 388, 62, 411], [63, 388, 84, 411], [85, 388, 106, 411], [108, 387, 131, 410], [213, 392, 237, 415], [239, 393, 276, 418], [11, 415, 34, 439], [213, 419, 230, 444], [231, 419, 241, 444], [244, 422, 286, 447], [11, 443, 34, 467], [208, 441, 252, 477], [259, 451, 287, 476], [11, 474, 34, 497], [52, 471, 80, 496], [38, 470, 51, 498], [215, 478, 274, 501], [10, 501, 30, 525], [207, 505, 267, 531], [49, 531, 123, 555], [11, 536, 27, 559], [29, 534, 46, 562], [204, 536, 233, 560], [234, 538, 259, 562]]]

the following code is used to plot the bounding box but the output I strange it does not draw on the specific letter

import matplotlib.pyplot as plt
import cv2
image = cv2.imread("/content/passbook_82_0.jpeg")
im_height, im_width, _ = image.shape
xmin=words_abs_coords[0][0][0]
ymin=words_abs_coords[0][0][1]
xmax=words_abs_coords[0][0][2]
ymax=words_abs_coords[0][0][3]
image1 = cv2.rectangle(image, (xmin,ymin), (xmax,ymax), (0,255,0), 2)
plt.imshow(image1)

[fg-mindee]

I tried locally it was because of an inversion between page dimensions to generate the abs coords. Now that should work properly 👍
(in the conversion to abs coordinates, simply switch dims[0] to dims[1] and conversely, I edited the snippet)

[ghost]

@fg-mindee Can i Draw a single bounding box on that line?

[fg-mindee]

That's a different topic, and we already have a discussion about this over there 👉 #573 😅

But the code snippet I provided here in this discussion will not plot a box on lines since we've flattened everything

[fg-mindee]

For anyone looking for the answer, here is the snippet from the other thread starting from the predictor result:

# Everything to produce the result goes here
# ...
export = result.export()
# Flatten the export
page_words = [[word for block in page['blocks'] for line in block['lines'] for word in line['words']] for page in export['pages']]
page_dims = [page['dimensions'] for page in export['pages']]
# Get the coords in [xmin, ymin, xmax, ymax]
words_abs_coords = [
    [[int(round(word['geometry'][0][0] * dims[1])), int(round(word['geometry'][0][1] * dims[0])), int(round(word['geometry'][1][0] * dims[1])), int(round(word['geometry'][1][1] * dims[0]))] for word in words]
    for words, dims in zip(page_words, page_dims)
]

[khawar-islam]

Hello @fg-mindee,

Thank you for your great work. I am simply retrieving a lot of images from folder and apply OCR model to detect all text in an image and save into the another folder. I am getting strange behavior after OCR detection the image is same in another folder and there is not detected part in an image.

Code

for img in glob.glob(testing_path + '*.*'):
    img_bgr_rgb = cv2.imread(img)
    file_Name = os.path.basename(img)
    image = DocumentFile.from_images(img)
    model = ocr_predictor('db_resnet50', 'sar_resnet31', pretrained=True)
    result = model(image)

    export = result.export()
    # Flatten the export
    page_words = [[word for block in page['blocks'] for line in block['lines'] for word in line['words']] for page
                  in export['pages']]
    page_dims = [page['dimensions'] for page in export['pages']]
    words_abs_coords = [
        [[int(round(word['geometry'][0][0] * dims[1])), int(round(word['geometry'][0][1] * dims[0])),
          int(round(word['geometry'][1][0] * dims[1])), int(round(word['geometry'][1][1] * dims[0]))] for word in
         words]
        for words, dims in zip(page_words, page_dims)
    ]
    xmin = words_abs_coords[0][0][0]
    ymin = words_abs_coords[0][0][1]
    xmax = words_abs_coords[0][0][2]
    ymax = words_abs_coords[0][0][3]

    imagee = cv2.rectangle(img_bgr_rgb, (xmin, ymin), (xmax, ymax), (255, 87, 51), 2)
    cv2.imwrite(os.path.join(result_path, file_Name), img_bgr_rgb)

[frgfm]

Let's avoid duplication discussion and move the rest of this over there 👉 #561 (comment)