DeepLabv3 + ADE20k benchmark #107
Conversation
|
Would you be able to use monkeypatch to mock out the actual download during the tests (and instead have it return uninitialized weights)? |
|
The pretrained weight download is done within torchvision, so I thought the easiest way to avoid the long download was to manually change |
|
cc: @A-Jacobson or @coryMosaicML for a review here (vision team) |
|
After some thought and discussion, I think it makes the most sense to make the initial benchmark to be as close as possible to other benchmarks. This means reverting back to the original batch size, no decoupled weight decay, and using a Polynomial LR schedule instead of cosine decay. @ravi-mosaicml and/or @A-Jacobson could y'all skim through these recent changes today or tomorrow?
It would be amazing if I could get this in tomorrow, thank you!! |
There was a problem hiding this comment.
Overall LGTM. Two main things:
-
I don't think many of the type ignores are required. I tried to comment how to remove them. If they are required, please add the pyright error message as a comment next to the type ignore. I realize they can get annoying; happy to help debug these!
-
Can you move the imagenet normalization parameters out of the normalization_fn class and to the imagenet dataset file? The normalization function should be dataset-generic.
| resnet.model_urls[backbone_arch] = "https://download.pytorch.org/models/resnet101-cd907fc2.pth" | ||
| else: | ||
| raise ValueError(f"backbone_arch must be one of ['resnet50', 'resnet101'] not {backbone_arch}") | ||
| backbone = resnet.__dict__[backbone_arch](pretrained=is_backbone_pretrained, |
There was a problem hiding this comment.
I presume this works but am a bit confused, as I don't see resnet50 or resnet101 in https://github.com/pytorch/vision/blob/main/torchvision/models/regnet.py
There was a problem hiding this comment.
Ah, I took this line from pytorch. The link you posted was for regnet.py, resnet.py is here: https://github.com/pytorch/vision/blob/main/torchvision/models/resnet.py
| self.val_ce = CrossEntropyLoss(ignore_index=-1) | ||
|
|
||
| def forward(self, batch: Batch): | ||
| x = batch[0] # type: ignore |
There was a problem hiding this comment.
| x = batch[0] # type: ignore | |
| x = composer.utils.types.as_batch_pair(batch)[0] |
There was a problem hiding this comment.
I just changed the input type to BatchPair, but it feels weird for forward to take a BatchPair since the label shouldn't be necessary to run the forward pass. This is discussion for another time...
composer/datasets/ade20k.py
Outdated
|
|
||
| if np.random.randint(2): | ||
| hue_factor = np.random.uniform(-self.hue, self.hue) | ||
| image = TF.adjust_hue(image, hue_factor) # type: ignore |
composer/datasets/ade20k.py
Outdated
|
|
||
| if contrast_mode == 0 and np.random.randint(2): | ||
| contrast_factor = np.random.uniform(1 - self.contrast, 1 + self.contrast) | ||
| image = TF.adjust_contrast(image, contrast_factor) # type: ignore |
| image = self.image_transforms(image) | ||
|
|
||
| if self.split in ['train', 'val']: | ||
| return image, target # type: ignore |
| ) | ||
| image_transforms = torch.nn.Sequential( | ||
| PhotometricDistoration(brightness=32. / 255, contrast=0.5, saturation=0.5, hue=18. / 255), | ||
| PadToSize(size=(self.final_size, self.final_size), |
There was a problem hiding this comment.
can you add a comment to where these magic values came from?
|
Okay, I think addressed all of @ravi-mosaicml comments, but let me know if more is needed! Otherwise, I will merge! |
* First commit ade20k and deeplabv3 * Allow ImageNet pretrained backbone * Allow background class to be ignored * Remove cross entropy metric (temp) * Add mmseg photometric augmentations * Add option to sync bn * Remove dropout and extra 3x3 conv * Use new resnet50 weights * Use 3x3 conv before classification * Add dropout2d * Remove dropout (again) * Select pretrained model, ability to randomly initialize, pytorch's syncBN * Fix LR schedule params * Update with recent merge and add resnet101 * Refactor ade20k pt. 1 * Missed hflips * Initial ignore_class refactor * Remove initial resize for base size in random scale * Remove cityscapes * Polynomial LR schedule * Add ade20k docstrings and some refactoring * Another iteration on ade20k and partial deeplabv3 refactor * Change permissions * total -> train batch size * Decoupled SGDW * Cleanup ade20k code and docstrings * Fix dataset test * Move preprocessing and collate; add defaults * Collate docstring, minor name changes, and ade20k synthetic dataset * Add mosaicml copyright * Fix formatting * Format pt. 2 * Remove RANDOM_INT from synthetic datasets * Format pt. 3 * Update yaml * Monkeypatch model tests to skip pretrained weights * mIoU -> MIoU * PolyLR and no DWD * Add initializers * Only initialize head when using pretrained backbone * Add PolynomialLR docstring and fix scheduler tests * Add mIoU and seg transformation tests * Reorder imports * Reorder imports pt. 2... * Address type ignores, move imagenet norm params, other comments * Get tests to pass * Formatting
Pull request to add the DeepLabv3 model and the ADE20k dataset as a new benchmark for semantic segmentation tasks
Some results with 4 seeds for each experiment:
Here is a convergence run.
mmsegmentation reports 44.08 mIoU and 45.00 mIoU for 64 and 127 epochs, respectively (here). I think these numbers are the final results for a single run.
Before Merging