Skip to content

[CVPR 2023 Highlight] LaserMix for Semi-Supervised LiDAR Semantic Segmentation

License

Notifications You must be signed in to change notification settings

ldkong1205/LaserMix

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation


LaserMix for Semi-Supervised LiDAR Semantic Segmentation

Lingdong Kong,Β  Jiawei Ren,Β  Liang Pan,Β  Ziwei Liu
S-Lab, Nanyang Technological University

About

LaserMix is a semi-supervised learning (SSL) framework designed for LiDAR semantic segmentation. It leverages the strong spatial prior of driving scenes to construct low-variation areas via laser beam mixing, and encourages segmentation models to make confident and consistent predictions before and after mixing.



Fig. Illustration for laser beam partition based on inclination Ο†.


Visit our project page to explore more details. πŸš—

Updates

  • [2024.05] - Our improved framework, LaserMix++ πŸš€, is avaliable on arXiv.
  • [2024.01] - The toolkit tailored for The RoboDrive Challenge has been released. πŸ› οΈ
  • [2023.12] - We are hosting The RoboDrive Challenge at ICRA 2024. πŸš™
  • [2023.12] - Introducing FRNet, an efficient and effective real-time LiDAR segmentation model that achieves promising semi-supervised learning results on SemanticKITTI and nuScenes. Code and checkpoints are available for downloading.
  • [2023.03] - Intend to test the robustness of your LiDAR semantic segmentation models? Check our recent work, πŸ€– Robo3D, a comprehensive suite that enables OoD robustness evaluation of 3D segmentors on our newly established datasets: SemanticKITTI-C, nuScenes-C, and WOD-C.
  • [2023.03] - LaserMix was selected as a ✨ highlight ✨ at CVPR 2023 (top 10% of accepted papers).
  • [2023.02] - LaserMix was accepted to CVPR 2023! πŸŽ‰
  • [2023.02] - LaserMix has been integrated into the MMDetection3D codebase! Check this PR in the dev-1.x branch to know more details. 🍻
  • [2023.01] - As suggested, we will establish a sequential track taking into account the LiDAR data collection nature in our semi-supervised LiDAR semantic segmentation benchmark. The results will be gradually updated in RESULT.md.
  • [2022.12] - We support a wider range of LiDAR segmentation backbones, including RangeNet++, SalsaNext, FIDNet, CENet, MinkowskiUNet, Cylinder3D, and SPVCNN, under both fully- and semi-supervised settings. The checkpoints will be available soon!
  • [2022.12] - The derivation of spatial-prior-based SSL is available here. Take a look! πŸ“
  • [2022.08] - LaserMix achieves 1st place among the semi-supervised semantic segmentation leaderboards of nuScenes, SemanticKITTI, and ScribbleKITTI, based on Paper-with-Code. πŸ“Š
  • [2022.08] - We provide a video demo for visual comparisons on the SemanticKITTI val set. Take a look!
  • [2022.07] - Our paper is available on arXiv, click here to check it out. Code will be available soon!

Outline

Installation

Please refer to INSTALL.md for the installation details.

Data Preparation

Please refer to DATA_PREPARE.md for the details to prepare the 1nuScenes, 2SemanticKITTI, and 3ScribbleKITTI datasets.

Getting Started

Please refer to GET_STARTED.md to learn more usage about this codebase.

Video Demo

Demo 1 Demo 2 Demo 3
Link ‴️ Link ‴️ Link ‴️

Main Result

Framework Overview

Range View

Method nuScenes SemanticKITTI ScribbleKITTI
1% 10% 20% 50% 1% 10% 20% 50% 1% 10% 20% 50%
Sup.-only 38.3 57.5 62.7 67.6 36.2 52.2 55.9 57.2 33.1 47.7 49.9 52.5
LaserMix 49.568.270.673.0 43.458.859.461.4 38.354.455.658.7
improv. ↑ +11.2 +10.7 +7.9 +5.4 +7.2 +6.6 +3.5 +4.2 +5.2 +6.7 +5.7 +6.2
LaserMix++
improv. ↑

Voxel

Method nuScenes SemanticKITTI ScribbleKITTI
1% 10% 20% 50% 1% 10% 20% 50% 1% 10% 20% 50%
Sup.-only 50.9 65.9 66.6 71.2 45.4 56.1 57.8 58.7 39.2 48.0 52.1 53.8
LaserMix 55.3 69.9 71.8 73.2 50.6 60.0 61.9 62.3 44.2 53.7 55.1 56.8
improv. ↑ +4.4 +4.0 +5.2 +2.0 +5.2 +3.9 +4.1 +3.6 +5.0 +5.7 +3.0 +3.0
LaserMix++
improv. ↑

Ablation Studies

Qualitative Examples

qualitative

Checkpoints & More Results

For more experimental results and pretrained weights, please refer to RESULT.md.

TODO List

  • Initial release. πŸš€
  • Add license. See here for more details.
  • Add video demos πŸŽ₯
  • Add installation details.
  • Add data preparation details.
  • Add evaluation details.
  • Add training details.

Citation

If you find this work helpful, please kindly consider citing our paper:

@inproceedings{kong2023lasermix,
  title = {LaserMix for Semi-Supervised LiDAR Semantic Segmentation},
  author = {Kong, Lingdong and Ren, Jiawei and Pan, Liang and Liu, Ziwei},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages = {21705--21715},
  year = {2023},
}

License

Creative Commons License
This work is under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Acknowledgement

This work is developed based on the MMDetection3D codebase.


MMDetection3D is an open-source toolbox based on PyTorch, towards the next-generation platform for general 3D perception. It is a part of the OpenMMLab project developed by MMLab.

We acknowledge the use of the following public resources during the course of this work: 1nuScenes, 2nuScenes-devkit, 3SemanticKITTI, 4SemanticKITTI-API, 5ScribbleKITTI, 6FIDNet, 7CENet, 8SPVNAS, 9Cylinder3D, 10TorchSemiSeg, 11MixUp, 12CutMix, 13CutMix-Seg, 14CBST, 15MeanTeacher, and 16Cityscapes.

We would like to thank Fangzhou Hong for the insightful discussions and feedback. ❀️