Add pandaset support

[lea-v]

Hello,
thanks a lot for sharing this project!

I've worked on adding support for Pandaset.
This code enables training and inferring frame by frame (not using whole sequences for the moment) on datasets in Pandaset format, however it does not enable evaluating results as there is no standard evaluation metric yet.
As you seem to be considered for further developments (#359), I think this could be useful for a starting point.

Please let me know if this can be of any use, and what could help merging this code.

[sshaoshuai]

Thanks a lot for adding the clear and well organized pandaset codes!

Could you also provide the performance of baseline models (like PointPillar or SECOND or PV-RCNN) on the pandaset dataset to validate the correctness?

You could refer to https://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/datasets/waymo/waymo_dataset.py#L243-L244 and https://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/datasets/waymo/waymo_dataset.py#L176-L188 to support evaluation with KITTI tools.

By the way, we have also internally supported the pandaset previously, and may add some modifications to the pandaset dataloader after merging your codes to the master.

[lea-v]

Unfortunately, I won't be able to provide Kitti evaluation code support nor trained models or performance of baseline models as it is out of my scope, I'll only be able to provide support on the existing parts of this code.
However, feel free to add any modification and improvement that you find useful.

[sshaoshuai]

Hi, Sorry for the late reply after one year....

I will merge this PR since it seems independent with the whole codebase and will not affect other parts.

I also do not have much time to support several models for pandaset, and may leave this problem to others who will use it.

PullRequest is welcomed for anybody who would like to support the evaluation / models on pandaset.

[achao-c]

Hello， could u please tell me the function meaning of ego_points = ps.geometry.lidar_points_to_ego(points_loc, pose) of the dataset.py ?☺

[lea-v]

Hi, this function is from the pandaset API (https://github.com/scaleapi/pandaset-devkit/blob/master/python/pandaset/geometry.py#L53). Its aim is to move the points from the "global" reference system to the one of the sensor so that the processing of the frame does not depend on its location on the acquisition trajectory.

[Leozyc-waseda]

@lea-v
Hi,
Could you please tell how to test kitti trained models like pv_rcnn on pandaset ?

in pandaset_dataset.py
(

OpenPCDet/pcdet/datasets/pandaset/pandaset_dataset.py

Line 304 in 0627984

world_centers = ps.geometry.ego_to_lidar_points(ego_centers, pose_dict)

)

but i checked pandaset API, there is no lidar_points_to_ego...
Could you tell me why it is written like this？

thanks.

[doronser]

@Leozyc-waseda I'm trying to do something similar and it looks like that function is available in an open PR to pandaset-devkit:

def ego_to_lidar_points(points, lidar_pose): lidar_pose_mat = _heading_position_to_mat(lidar_pose['heading'], lidar_pose['position']) return (lidar_pose_mat[:3, :3] @ points.T + lidar_pose_mat[:3, [3]]).T

[Leozyc-waseda]

@doronser
Thank you ,
already find it . : )
scaleapi/pandaset-devkit@52c1cf7