Rewrite transforms v2 e2e example #7881

NicolasHug · 2023-08-24T14:55:53Z

This is a light rewrite despite the seemingly big diff. I just extended it a little to include masks and to be a bit more descriptive / detailed.

pytorch-bot · 2023-08-24T14:55:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/7881

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 33 New Failures

As of commit 980556d with merge base 92e4e9c ():

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

NicolasHug · 2023-08-24T14:56:24Z

gallery/v2_transforms/helpers.py

@@ -1,5 +1,8 @@
 import matplotlib.pyplot as plt
-from torchvision.utils import draw_bounding_boxes
+import torch


Changes in this file are only needed to support masks

NicolasHug · 2023-08-24T14:57:18Z

gallery/v2_transforms/plot_transforms_v2.py

@@ -90,7 +90,7 @@

 from torchvision import datapoints  # we'll describe this a bit later, bare with us

-bboxes = datapoints.BoundingBoxes(
+boxes = datapoints.BoundingBoxes(


Dataset wrapper uses "boxes" instead of "bboxes". I don't have a pref, but we should align (hence the changes in this example)

The reason the dataset wrapper uses "boxes" is because our models require that:

vision/torchvision/models/detection/ssd.py

Lines 333 to 341 in f7c7bdf

for target in targets:

boxes = target["boxes"]

if isinstance(boxes, torch.Tensor):

torch._assert(

len(boxes.shape) == 2 and boxes.shape[-1] == 4,

f"Expected target boxes to be a tensor of shape [N, 4], got {boxes.shape}.",

)

else:

torch._assert(False, f"Expected target boxes to be of type Tensor, got {type(boxes)}.")

NicolasHug · 2023-08-24T14:58:11Z

gallery/v2_transforms/plot_transforms_v2_e2e.py

-``torchvision.transforms.v2`` enables jointly transforming images, videos, bounding boxes, and masks. This example
-showcases an end-to-end object detection training using the stable ``torchvision.datasets`` and ``torchvision.models``
-as well as the new ``torchvision.transforms.v2`` v2 API.
+Object detection and segmentation tasks are natively supported:


Using "segmentation" loosely here (i.e. both instance and semantic). It's made more accurate a bit later in the example, the main point being that it doesn't matter which one you're targetting, the transforms work the same.

vfdev-5 · 2023-08-24T15:31:29Z

gallery/v2_transforms/plot_transforms_v2_e2e.py

+``torchvision.transforms.v2`` enables jointly transforming images, videos,
+bounding boxes, and masks.
+
+This example showcases an end-to-end instance segmentation training case using


Suggested change

This example showcases an end-to-end instance segmentation training case using

This example showcases an end-to-end object detection / instance segmentation training case using

I think it's fine to keep as-is considering the sentence just below. Technically, it is an instance segmentation model.

pmeier

Minor comment left, but LGTM nonetheless. Thanks Nicolas!

pmeier · 2023-08-25T08:10:38Z

gallery/v2_transforms/helpers.py

+                img = draw_bounding_boxes(img, boxes, colors="yellow", width=3)
+            if masks is not None:
+                img = draw_segmentation_masks(img, masks.to(torch.bool), alpha=.65)


Should we maybe also make the masks yellow or some other color? I fell in the current version the masks is not completely obvious on the image.

hmmm looks like the color param is broken, I tried colors="green" and colors=["green"] * masks.shape[0], none of which had an effect. Might be worth looking into but I'll merge

maybe I was looking at the wrong output - it actually works fine and the masks are green (and very obvious)

Summary: (Note: this ignores all push blocking failures!) Reviewed By: matteobettini Differential Revision: D48900411 fbshipit-source-id: 2880b47b7029d33e09883243651ddffaed8f5d94

NicolasHug added 5 commits August 24, 2023 14:37

Some stuff

0887542

fix_bbox_sanitize_tensor

9d11700

Merge branch 'alenflaelajenf' into e2e_yolo

958d652

Merge branch 'main' of github.com:pytorch/vision into e2e_yolo

fef649b

ajenaejgng

b4af363

NicolasHug added enhancement module: documentation labels Aug 24, 2023

NicolasHug requested review from vfdev-5 and pmeier August 24, 2023 14:55

facebook-github-bot added the cla signed label Aug 24, 2023

NicolasHug commented Aug 24, 2023

View reviewed changes

NicolasHug temporarily deployed to pytorchbot-env August 24, 2023 14:56 — with GitHub Actions Inactive

NicolasHug commented Aug 24, 2023

View reviewed changes

NicolasHug temporarily deployed to pytorchbot-env August 24, 2023 14:57 — with GitHub Actions Inactive

NicolasHug commented Aug 24, 2023

View reviewed changes

Revert useless change

f404f6c

NicolasHug temporarily deployed to pytorchbot-env August 24, 2023 14:59 — with GitHub Actions Inactive

NicolasHug temporarily deployed to pytorchbot-env August 24, 2023 15:11 — with GitHub Actions Inactive

NicolasHug temporarily deployed to pytorchbot-env August 24, 2023 15:12 — with GitHub Actions Inactive

NicolasHug temporarily deployed to pytorchbot-env August 24, 2023 15:28 — with GitHub Actions Inactive

vfdev-5 reviewed Aug 24, 2023

View reviewed changes

pmeier approved these changes Aug 25, 2023

View reviewed changes

NicolasHug added 2 commits August 25, 2023 09:40

minor stuff

f719a60

Merge branch 'main' of github.com:pytorch/vision into e2e_yolo

980556d

NicolasHug merged commit 224cbc8 into pytorch:main Aug 25, 2023
1 check passed

NicolasHug temporarily deployed to pytorchbot-env August 25, 2023 08:41 — with GitHub Actions Inactive

NicolasHug temporarily deployed to pytorchbot-env August 25, 2023 08:42 — with GitHub Actions Inactive

NicolasHug temporarily deployed to pytorchbot-env August 25, 2023 08:47 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rewrite transforms v2 e2e example #7881

Rewrite transforms v2 e2e example #7881

NicolasHug commented Aug 24, 2023

pytorch-bot bot commented Aug 24, 2023 •

edited

Loading

NicolasHug Aug 24, 2023

NicolasHug Aug 24, 2023

pmeier Aug 24, 2023

NicolasHug Aug 24, 2023

vfdev-5 Aug 24, 2023

NicolasHug Aug 25, 2023

pmeier left a comment

pmeier Aug 25, 2023

NicolasHug Aug 25, 2023

NicolasHug Aug 25, 2023

	for target in targets:
	boxes = target["boxes"]
	if isinstance(boxes, torch.Tensor):
	torch._assert(
	len(boxes.shape) == 2 and boxes.shape[-1] == 4,
	f"Expected target boxes to be a tensor of shape [N, 4], got {boxes.shape}.",
	)
	else:
	torch._assert(False, f"Expected target boxes to be of type Tensor, got {type(boxes)}.")

	This example showcases an end-to-end instance segmentation training case using
	This example showcases an end-to-end object detection / instance segmentation training case using

Rewrite transforms v2 e2e example #7881

Rewrite transforms v2 e2e example #7881

Conversation

NicolasHug commented Aug 24, 2023

pytorch-bot bot commented Aug 24, 2023 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/7881

❌ 33 New Failures

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pmeier left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pytorch-bot bot commented Aug 24, 2023 •

edited

Loading