Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Summarize models, losses & metrics for segmentation #1354

Merged
merged 5 commits into from
Aug 14, 2023
Merged

Summarize models, losses & metrics for segmentation #1354

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1b8f329
Summarize models, losses & metrics
BloodAxe Aug 8, 2023
88421db
Merge branch 'master' into feature/SG-855
BloodAxe Aug 8, 2023
de65c29
Merge branch 'master' into feature/SG-855
BloodAxe Aug 8, 2023
4a58ba9
Added troubleshoothing section
BloodAxe Aug 8, 2023
e920511
Merge branch 'master' into feature/SG-855
BloodAxe Aug 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 80 additions & 1 deletion documentation/source/Segmentation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,71 @@ SuperGradients allows users to train models for semantic segmentation tasks.
The library includes pre-trained models, such as the Cityscapes PPLiteSeg model, and provides a simple interface for
loading custom datasets.

## Model zoo

SuperGradients includes a variety of pre-trained models for semantic segmentation tasks.

| Model Name | Dataset | IoU | Training Recipe | Resolution |
|----------------|------------|-------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|
| DDRNet 23 | Cityscapes | 80.26 | [cityscapes_ddrnet.yaml](https://github.com/Deci-AI/super-gradients/blob/95018d602ef7f65b37d5ec62e26a8ebbc5b2a7c8/src/super_gradients/recipes/cityscapes_ddrnet.yaml) | [1024, 2048] |
| DDRNet 23 Slim | Cityscapes | 78.01 | [cityscapes_ddrnet.yaml](https://github.com/Deci-AI/super-gradients/blob/95018d602ef7f65b37d5ec62e26a8ebbc5b2a7c8/src/super_gradients/recipes/cityscapes_ddrnet.yaml) | [1024, 2048] |
| DDRNet 39 | Cityscapes | 81.32 | [cityscapes_ddrnet.yaml](https://github.com/Deci-AI/super-gradients/blob/95018d602ef7f65b37d5ec62e26a8ebbc5b2a7c8/src/super_gradients/recipes/cityscapes_ddrnet.yaml) | [1024, 2048] |
| STDC1 Seg 50 | Cityscapes | 75.11 | [cityscapes_stdc_seg50.yaml](https://github.com/Deci-AI/super-gradients/blob/95018d602ef7f65b37d5ec62e26a8ebbc5b2a7c8/src/super_gradients/recipes/cityscapes_stdc_seg50.yaml) | [512, 1024] |
| STDC1 Seg 75 | Cityscapes | 76.87 | [cityscapes_stdc_seg75.yaml](https://github.com/Deci-AI/super-gradients/blob/6e89982649e62e9877a802cd1240464cd3b3b87b/src/super_gradients/recipes/cityscapes_stdc_seg75.yaml) | [768, 1536] |
| STDC2 Seg 50 | Cityscapes | 76.44 | [cityscapes_stdc_seg50.yaml](https://github.com/Deci-AI/super-gradients/blob/95018d602ef7f65b37d5ec62e26a8ebbc5b2a7c8/src/super_gradients/recipes/cityscapes_stdc_seg50.yaml) | [512, 1024] |
| STDC2 Seg 75 | Cityscapes | 78.93 | [cityscapes_stdc_seg75.yaml](https://github.com/Deci-AI/super-gradients/blob/6e89982649e62e9877a802cd1240464cd3b3b87b/src/super_gradients/recipes/cityscapes_stdc_seg75.yaml) | [768, 1536] |
| RegSeg 48 | Cityscapes | 78.15 | [cityscapes_regseg48.yaml](https://github.com/Deci-AI/super-gradients/blob/95018d602ef7f65b37d5ec62e26a8ebbc5b2a7c8/src/super_gradients/recipes/cityscapes_regseg48.yaml) | [1024, 2048] |
| PP-Lite T 50 | Cityscapes | 74.92 | [cityscapes_pplite_seg50.yaml](https://github.com/Deci-AI/super-gradients/blob/95018d602ef7f65b37d5ec62e26a8ebbc5b2a7c8/src/super_gradients/recipes/cityscapes_pplite_seg50.yaml) | [512, 1024] |
| PP-Lite T 75 | Cityscapes | 77.56 | [cityscapes_pplite_seg75.yaml](https://github.com/Deci-AI/super-gradients/blob/95018d602ef7f65b37d5ec62e26a8ebbc5b2a7c8/src/super_gradients/recipes/cityscapes_pplite_seg75.yaml) | [512, 1024] |
| PP-Lite B 50 | Cityscapes | 76.48 | [cityscapes_pplite_seg50.yaml](https://github.com/Deci-AI/super-gradients/blob/95018d602ef7f65b37d5ec62e26a8ebbc5b2a7c8/src/super_gradients/recipes/cityscapes_pplite_seg50.yaml) | [512, 1024] |
| PP-Lite B 75 | Cityscapes | 78.52 | [cityscapes_pplite_seg75.yaml](https://github.com/Deci-AI/super-gradients/blob/95018d602ef7f65b37d5ec62e26a8ebbc5b2a7c8/src/super_gradients/recipes/cityscapes_pplite_seg75.yaml) | [512, 1024] |

Latency and additional details of these models can be found in the [SuperGradients Model Zoo](https://docs.deci.ai/super-gradients/documentation/source/model_zoo.html).

## Loss functions

SuperGradients provides a variety of loss functions for training semantic segmentation tasks.
All loss functions are implemented in PyTorch and can be found in the `super_gradients.training.losses` module.
The following table summarizes the loss functions currently supported by SuperGradients.

| Loss function class | Loss name in YAML | Description |
|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------|----------------------------------------------------------------------|
| [BCEDiceLoss](https://docs.deci.ai/super-gradients/docstring/training/losses.html#training.losses.bce_dice_loss.BCEDiceLoss) | bce_dice_loss | Weighted average of BCE and Dice loss |
| [LabelSmoothingCrossEntropyLoss](https://docs.deci.ai/super-gradients/docstring/training/losses.html#training.losses.label_smoothing_cross_entropy_loss.LabelSmoothingCrossEntropyLoss) | cross_entropy | Cross entropy loss with label smoothing support |
| [DiceLoss](https://docs.deci.ai/super-gradients/docstring/training/losses.html#training.losses.dice_loss.DiceLoss) | N/A | Dice loss for multiclass segmentation |
| [BinaryDiceLoss](https://docs.deci.ai/super-gradients/docstring/training/losses.html#training.losses.dice_loss.BinaryDiceLoss) | N/A | Dice loss for binary segmentation |
| [GeneralizedDiceLoss](https://docs.deci.ai/super-gradients/docstring/training/losses.html#training.losses.dice_loss.GeneralizedDiceLoss) | N/A | Generalized dice loss |
| [DiceCEEdgeLoss](https://docs.deci.ai/super-gradients/docstring/training/losses.html#training.losses.dice_ce_edge_loss.DiceCEEdgeLoss) | dice_ce_edge_loss | Dice loss + Cross entropy loss + Edge loss |
| [SegKDLoss](https://docs.deci.ai/super-gradients/docstring/training/losses.html#training.losses.seg_kd_loss.SegKDLoss) | N/A | A loss function for knowledge distillation for semantic segmentation |

## Metrics

| Metric Class | Metric name in YAML | Description |
|-------------------------------------------------------------------------------------------------------------------------------------------|---------------------|------------------------------------------------------------------------|
| [PixelAccuracy](https://docs.deci.ai/super-gradients/docstring/training/metrics.html#training.metrics.segmentation_metrics.PixelAccuracy) | PixelAccuracy | The ratio of correctly classified pixels to the total number of pixels |
| [IoU](https://docs.deci.ai/super-gradients/docstring/training/metrics.html#training.metrics.segmentation_metrics.IoU) | IoU | Calculate the Jaccard index for multilabel tasks. |
| [Dice](https://docs.deci.ai/super-gradients/docstring/training/metrics.html#training.metrics.segmentation_metrics.Dice) | Dice | Calculate the Dice index for multilabel tasks. | |
| Binary IoU | BinaryIOU | Calculate the Jaccard index for binary segmentation task. |
| BinaryDice | BinaryDice | Calculate the Dice index for binary segmentation task. |

See [Metrics](https://docs.deci.ai/super-gradients/documentation/source/Metrics.html) page for additional details of using metrics in SuperGradients.

## Datasets

SuperGradients provides a number of ready to use datasets for semantic segmentation tasks and corresponding data loaders.

| Dataset | Dataset Class | train dataloader | val dataloader |
|------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------|-----------------------------|
| COCO | [CoCoSegmentationDataSet](https://docs.deci.ai/super-gradients/docstring/training/datasets.html#training.datasets.segmentation_datasets.coco_segmentation.CoCoSegmentationDataSet) | coco_segmentation_train | coco_segmentation_val |
| Cityscapes | [CityscapesDataset](https://docs.deci.ai/super-gradients/docstring/training/datasets.html#training.datasets.segmentation_datasets.cityscape_segmentation.CityscapesDataset) | cityscapes_train | cityscapes_val |
| Pascal VOC | [PascalVOC2012SegmentationDataSet](https://docs.deci.ai/super-gradients/docstring/training/datasets.html#training.datasets.segmentation_datasets.pascal_voc_segmentation.PascalVOC2012SegmentationDataSet) | pascal_voc_segmentation_train | pascal_voc_segmentation_val |
| Supervisely | [SuperviselyPersonDataset](https://docs.deci.ai/super-gradients/docstring/training/datasets.html#training.datasets.segmentation_datasets.supervisely_persons_segmentation.SuperviselyPersonsDataset) | supervisely_persons_train | supervisely_persons_val |
| Mapillary Vistas | [MapillaryDataset](https://docs.deci.ai/super-gradients/docstring/training/datasets.html#training.datasets.segmentation_datasets.mapillary_dataset.MapillaryDataset) | mapillary_train | mapillary_val |

In the next section we will demonstrate how to use these datasets and dataloaders to train a segmentation model using SuperGradients.

## How to train a segmentation model using Super Gradients

In the tutorial provided, we demonstrate how to fine-tune PPLiteSeg on a subset of the Supervisely dataset.
You can run the following code in our [google collab](https://colab.research.google.com/drive/1d7cU0NsUj7jnOF1YSap_DH9r79G3-Cr4?usp=sharing#scrollTo=GqH4VGMroWec).

Expand Down Expand Up @@ -69,8 +134,10 @@ Notes


### Setup training parameters

The training parameters includes loss, metrics, learning rates and much more. You can check out the [default training parameters](https://github.com/Deci-AI/super-gradients/blob/master/src/super_gradients/recipes/training_hyperparams/default_train_params.yaml).
For this task, we will train for 30 epoch, using Binary IoU using the SGD optimizer.

```py
from super_gradients.training.metrics.segmentation_metrics import BinaryIOU

Expand Down Expand Up @@ -143,5 +210,17 @@ display(mask)


## Going further

### Troubleshooting

If you encounter any issues, please check out our [troubleshooting guide](https://docs.deci.ai/super-gradients/documentation/source/troubleshooting.html).

### How to launch on multiple GPUs (DDP) ?
Please check out our tutorial on [how to use multiple GPUs'](...)

Please check out our tutorial on [how to use multiple GPUs'](https://docs.deci.ai/super-gradients/documentation/source/device.html#4-ddp-distributed-data-parallel)

### How to train models with limited GPU memory?

In case you have a GPU with limited memory, you can use the gradients accumulation technique to "fake" larger batch sizes.
This is not 100% equivalent to training with larger batch sizes, but it is a good approximation.
You can set the desired number of batches to accumulate by changing the `training_hyperparams.batch_accumulate` parameter.