Some issues regarding the function of save-txt in segmentation prediction task.

[CHR1122]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

When I was predicting about the segmentation task, I opened -- save-txt, but it didn't work as I expected, and the result is as follows:

Inference results

Saved txt
saved_seg_label.txt

Add outlines to the org image based on the txt file. But it did not save all the points of the mask, each instance target only saves a portion of the mask's outline points.

Inference commands used
python segment/predict.py --weights runs/bare_soil_uncover-seg/v0.0.001/weights/best.pt --source test_sources/images/baresoil/test.jpg --save-txt --classes 0

Code of add outlines

            if save_txt:
                segments = [
                    scale_segments(im0.shape if retina_masks else im.shape[2:], x, im0.shape, normalize=True)
                    for x in reversed(masks2segments(masks))]

            # Print results
            for c in det[:, 5].unique():
                n = (det[:, 5] == c).sum()  # detections per class
                s += f"{n} {names[int(c)]}{'s' * (n > 1)}, "  # add to string

            # Mask plotting
            annotator.masks(
                masks,
                colors=[colors(x, True) for x in det[:, 5]],
                im_gpu=torch.as_tensor(im0, dtype=torch.float16).to(device).permute(2, 0, 1).flip(0).contiguous() /
                255 if retina_masks else im[i])

            # Write results
            for j, (*xyxy, conf, cls) in enumerate(reversed(det[:, :6])):
                if save_txt:  # Write to file
                    seg = segments[j].reshape(-1)  # (n,2) to (n*2)
                    line = (cls, *seg, conf) if save_conf else (cls, *seg)  # label format
                    with open(f'{txt_path}.txt', 'a') as f:
                        f.write(('%g ' * len(line)).rstrip() % line + '\n')

                if save_img or save_crop or view_img:  # Add bbox to image
                    c = int(cls)  # integer class
                    label = None if hide_labels else (names[c] if hide_conf else f'{names[c]} {conf:.2f}')
                    annotator.box_label(xyxy, label, color=colors(c, True))
                    # annotator.draw.polygon(segments[j], outline=colors(c, True), width=3)
                if save_crop:
                    save_one_box(xyxy, imc, file=save_dir / 'crops' / names[c] / f'{p.stem}.jpg', BGR=True)

        # Stream results
        im0 = annotator.result()
        if view_img:
            if platform.system() == 'Linux' and p not in windows:
                windows.append(p)
                cv2.namedWindow(str(p), cv2.WINDOW_NORMAL | cv2.WINDOW_KEEPRATIO)  # allow window resize (Linux)
                cv2.resizeWindow(str(p), im0.shape[1], im0.shape[0])
            cv2.imshow(str(p), im0)
            if cv2.waitKey(1) == ord('q'):  # 1 millisecond
                exit()

        # Save results (image with detections)
        if save_img:
            if dataset.mode == 'image':

                # add outlines------------------------------------------------------------------
                for area in segments:
                    #Obtain pixel coordinates based on scale
                    area[:, 0] *= 1920
                    area[:, 1] *= 1080
                    area = area.astype(np.int64)
                    print("area:",area)
                    cv2.polylines(im0,[area],1,(255,0,0),5)
                # add outlines-----------------------------------------------------------------

                cv2.imwrite(save_path, im0)
            else:  # 'video' or 'stream'
                if vid_path[i] != save_path:  # new video
                    vid_path[i] = save_path
                    if isinstance(vid_writer[i], cv2.VideoWriter):
                        vid_writer[i].release()  # release previous video writer
                    if vid_cap:  # video
                        fps = vid_cap.get(cv2.CAP_PROP_FPS)
                        w = int(vid_cap.get(cv2.CAP_PROP_FRAME_WIDTH))
                        h = int(vid_cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
                    else:  # stream
                        fps, w, h = 30, im0.shape[1], im0.shape[0]
                    save_path = str(Path(save_path).with_suffix('.mp4'))  # force *.mp4 suffix on results videos
                    vid_writer[i] = cv2.VideoWriter(save_path, cv2.VideoWriter_fourcc(*'mp4v'), fps, (w, h))
                vid_writer[i].write(im0)

Additional

No response

[github-actions]

👋 Hello @CHR1122, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

Introducing YOLOv8 🚀

We're excited to announce the launch of our latest state-of-the-art (SOTA) object detection model for 2023 - YOLOv8 🚀!

Designed to be fast, accurate, and easy to use, YOLOv8 is an ideal choice for a wide range of object detection, image segmentation and image classification tasks. With YOLOv8, you'll be able to quickly and accurately detect objects in real-time, streamline your workflows, and achieve new levels of accuracy in your projects.

Check out our YOLOv8 Docs for details and get started with:

pip install ultralytics

[glenn-jocher]

@CHR1122 hello! It seems like you're encountering an issue with the --save-txt flag not saving all points of the mask, resulting in incomplete outlines when overlayed on the original image.

This behavior is actually expected as the --save-txt option is designed to save simplified representations of objects, primarily focusing on bounding box coordinates, rather than full segmentation masks. If your project requires detailed outline points for each mask, you may need to implement a custom solution to extract and save these points directly from the segmentation masks produced during inference.

To achieve this, you could modify the segment where segmentation masks are processed and save the full mask points instead of converting them to bounding boxes or simplified segments. This will involve working directly with the mask tensors output by the model and saving the desired detailed mask data in a format that meets your needs.

Keep in mind that any modifications or usage of the Ultralytics models and codes should comply with our licensing terms. For any commercial or proprietary application, an Ultralytics Enterprise License is required. This also applies to any form of internal company usage unless you plan to open-source your project under AGPL-3.0.

If you have further inquiries or need assistance regarding our licensing, please consult the documentation or reach out for support. Your understanding and cooperation with our policy ensure the continued development and improvement of the YOLOv5 models and software. Thanks for being part of the YOLOv5 community!

[CHR1122]

Thank you for your answer. I will try to implement a custom method to solve this problem and comply with the relevant licenses of YOLOv5.

[glenn-jocher]

@CHR1122 Excellent decision! 😊 Should you have any more questions or require further assistance as you develop your custom method, don't hesitate to reach out. Remember, the documentation is a great resource for guidance. Happy coding, and best of luck with your project! Remember, compliance with the YOLOv5 licensing ensures that we can all continue to benefit from and contribute to this amazing community. Keep up the great work!