Support VTK7 with OpenGL2 backend

[taketwo]

This is a work-in-progress to support the new default backend in VTK. Addresses #1275.

Compiles and works on my Arch with VTK7, however I observe very low frame-rates (~15fps) when rotating and panning moderately sized (~300k) point clouds.

[VictorLamoine]

I'm building against VTK trunk:

CMakeFiles/pcl_outofcore_viewer.dir/__/src/visualization/outofcore_cloud.cpp.o: In function `OutofcoreCloud::render(vtkRenderer*)':
outofcore_cloud.cpp:(.text+0x278f): undefined reference to `vtkVertexBufferObjectMapper::New()'
outofcore_cloud.cpp:(.text+0x27e9): undefined reference to `vtkVertexBufferObjectMapper::SetInput(vtkPolyData*)'
collect2: error: ld returned 1 exit status
make[2]: *** [bin/pcl_outofcore_viewer] Error 1
make[1]: *** [outofcore/tools/CMakeFiles/pcl_outofcore_viewer.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

[VictorLamoine]

What hardware are you testing on?

Here is a VTK test code, use this big mesh (8 M point clouds)

#include <vtkSmartPointer.h>
#include <vtkCallbackCommand.h>
#include <vtkRenderer.h>
#include <vtkRenderWindowInteractor.h>
#include <vtkInteractorStyleTrackballCamera.h>
#include <vtkRenderWindow.h>
#include <vtkTimerLog.h>
#include <vtkPolyDataMapper.h>
#include <vtkActor.h>
#include <vtkPLYReader.h>
#include <vtkProperty.h>

#include <iostream>

static void CallbackFunction(vtkObject* caller, long unsigned int eventId, void* clientData, void* callData);

int main(int, char *argv[])
{
  vtkSmartPointer<vtkPLYReader> reader = vtkSmartPointer<vtkPLYReader>::New();
  reader->SetFileName(argv[1]);
  reader->Update();

  vtkSmartPointer<vtkPolyData> polydata = vtkSmartPointer<vtkPolyData>::New();
  polydata = reader->GetOutput();
  std::cout << "Mesh points: " << polydata->GetPoints()->GetNumberOfPoints() << std::endl;

  vtkSmartPointer<vtkPolyDataMapper> mapper = vtkSmartPointer<vtkPolyDataMapper>::New();
  mapper->SetInputData(polydata);

  vtkSmartPointer<vtkActor> actor = vtkSmartPointer<vtkActor>::New();
  actor->SetMapper(mapper);
  actor->GetProperty ()->SetColor (0.9, 0.1, 0.2);

  vtkSmartPointer<vtkRenderer> renderer = vtkSmartPointer<vtkRenderer>::New();
  renderer->AddActor(actor);

  vtkSmartPointer<vtkRenderWindow> renderWindow = vtkSmartPointer<vtkRenderWindow>::New();
  renderWindow->AddRenderer(renderer);

  vtkSmartPointer<vtkInteractorStyleTrackballCamera> imageStyle =
      vtkSmartPointer<vtkInteractorStyleTrackballCamera>::New();

  vtkSmartPointer<vtkRenderWindowInteractor> renderWindowInteractor = vtkSmartPointer<vtkRenderWindowInteractor>::New();
  renderWindowInteractor->SetInteractorStyle(imageStyle);
  renderWindowInteractor->SetRenderWindow(renderWindow);

  vtkSmartPointer<vtkCallbackCommand> callback = vtkSmartPointer<vtkCallbackCommand>::New();
  callback->SetCallback(CallbackFunction);
  renderer->AddObserver(vtkCommand::EndEvent, callback);

  // Render and interact
  renderWindow->Render();
  renderWindowInteractor->Start();
  return EXIT_SUCCESS;
}

void CallbackFunction(vtkObject* caller, long unsigned int vtkNotUsed(eventId), void* vtkNotUsed(clientData),
                      void* vtkNotUsed(callData))
{
  vtkRenderer* renderer = static_cast<vtkRenderer*>(caller);

  double timeInSeconds = renderer->GetLastRenderTimeInSeconds();
  double fps = 1.0 / timeInSeconds;
  std::cout << "FPS: " << fps << std::endl;
}

cmake_minimum_required(VERSION 2.8)

PROJECT(vtk_ply_loader)

find_package(VTK REQUIRED)
include(${VTK_USE_FILE})

add_executable(test_vtk_1 MACOSX_BUNDLE test_vtk_1)

if(VTK_LIBRARIES)
  target_link_libraries(test_vtk_1 ${VTK_LIBRARIES})
else()
  target_link_libraries(test_vtk_1 vtkHybrid)
endif()

I see a big improvement between VTK OpenGL and OpenGL2.
Big point cloud: https://mega.nz/#!UZ8wDC6L!jefEj4aZknTYFP3grU2nAPdRGalFORHBsJVy3zGnFI8

[taketwo]

Ok, for now I simply disabled the pcl_outofcore_viewer app because it relies on our custom vtkVertexBufferObjectMapper which I have no clue how to adapt for the new backend.

I'll give your example a try.

[taketwo]

I did not manage to get the FPS counter below 116, mostly it reports numbers between 1k and 3k. However still for me the motions are not super-smooth. Maybe I'm subjective, so please give a try to pcl_viewer yourself.

[VictorLamoine]

I would rather check VTK_RENDERING_BACKEND so that it allows people with OpenGL backend to still use pcl_outofcore_viewer.

[taketwo]

Well, it's not meant to be the final solution. I should have clarified, this pull request is more like a hack to make PCL compile with latest VTK. Eventually we will need to figure out how to substitute PCL's custom vtkVertexBufferObjectMapper with the stuff from the new backend (e.g. vtkOpenGLPointGaussianMapper).

[taketwo]

Ok, now it's supposed to be the final solution. Vertex buffer objects are used ubiquitously by the new backend, transparently for us. Therefore there is no need in the custom vtkVertexBufferObjectMapper class at all. And we don't need to substitute it with anything explicitly.

This pull request introduces a couple of variables and macros that are checked in scripts and code to conditionally compile the custom mapper only when the old backend is used. They are based on the VTK_RENDERING_BACKEND, not just on VTK version, as suggested by Victor.

I think it's a safe change and should probably be included in 1.8.0, though of course it would be nice to hear back from someone who tried this patch first.

[VictorLamoine]

I think it's a safe change and should probably be included in 1.8.0, though of course it would be nice to hear back from someone who tried this patch first.

It would really be great, this is a huge improvement!

I will compile, test and benchmark this code on several machines (at least 3) with different hardware configurations and get back with the results 👍

[taketwo]

Great, thanks!

[UnaNancyOwen]

I has confirmed that PCL with this fix can build on MSVC14.
Thanks! 👍

	MSVC14 x86	MSVC14 x64
VTK 7.0 with OpenGL back-end	success	success
VTK 7.0 with OpenGL2 back-end	success	success

[taketwo]

Did you also try to run pcl_viewer? How is the performance?

[UnaNancyOwen]

Where is the test code and test data?

[taketwo]

I did not have any specific code or data in mind. Just the standard pcl_viewer.exe tool with some PCD file (e.g. from our "test/" directory).

[VictorLamoine]

Test program

#include <pcl/console/parse.h>
#include <pcl/io/vtk_lib_io.h>
#include <pcl/visualization/pcl_visualizer.h>

int main(int argc, char *argv[])
{
  std::vector<int> ply_files = pcl::console::parse_file_extension_argument(argc, argv, ".ply");
  if (ply_files.empty())
  {
    PCL_ERROR("Please provide a PLY mesh to be visualized!\n");
    return (-1);
  }

  pcl::PolygonMesh mesh;
  pcl::io::loadPolygonFilePLY(argv[ply_files[0]], mesh);

  pcl::visualization::PCLVisualizer viewer;
  viewer.addPolygonMesh(mesh);
  viewer.addCoordinateSystem();
  viewer.resetCamera();
  viewer.spin();
  return (0);
}

Test file (8 millions points mesh).

[UnaNancyOwen]

I was tested in the following environments.
I think that it have good frame rate.
https://youtu.be/LjZSs7CY5-g

• Intel Core i7 4790
• NVIDIA GeForce GTX970
• Microsoft Windows 10 (x64)
• Microsoft Visual Studio 2015
• PCL 1.8.0 with this fix + VTK 7.0 OpenGL2 back-end

[taketwo]

Brilliant! Thanks for giving it a try. I think we can merge this as soon as Victor reports his results.

[VictorLamoine]

Results will be available on Monday. I have 3 configurations at home but two of them crashes with OpenGL2 backend (GLXBadFBConfig), its not PCL related.

At work I know the OpenGL2 back-end works so I will test there.

[Caopi]

If you're looking for additional benchmarks, my specs:

• Current Arch Linux (64bit)
• VTK7 with OpenGL2
• latest PCL from Git with this fix (everything built correct)
• CPU: i5-4210H
• Onboard Graphics: Intel HD 4600
• Graphics Card: Nvidia GTX 950M (using primusrun and bumblebee)

On Victors last test program and the suzanne_hd.ply mesh I got:

• Onboard: ~14 fps during turning
• Nvidia: ~2200 fps average, almost never below 1000 fps, the lowest I could find on multiple screenshots was 600

Edit: On the first programm with vtk directly even my onboard went over 1000fps (./main suzanne_hd.ply
Mesh points: 8060918
FPS: 9.57956, FPS: 592.834, FPS: 3862.16, FPS: 4670.72, FPS: 2785.06, FPS: 781.79
) and nvidia reached upwards to 8000 (primusrun ./main suzanne_hd.ply
Mesh points: 8060918
FPS: 6.56069, FPS: 3320.91, FPS: 2545.09, FPS: 2222.74, FPS: 3276.8, FPS: 2976.79, FPS: 6842.26, FPS: 7194.35, FPS: 7084.97), but I'm not so sure about this fps calculation. Onboard felt a little stuttery, which shouldn't be the case on such high fps counts.

[VictorLamoine]

Setup

• Ubuntu 14.04 bits
• gcc 4.9.3
• VTK trunk (331d)
• PCL Sergey version (https://github.com/taketwo/pcl/tree/vtk-7)
• Everything is compiled in Release mode

Hardware

• Victor's desktop
  • Intel® Core™ i5-4670K CPU @ 3.40GHz × 4
  • GeForce GTX 770 (nvidia 352.63 updates)
• Dell M6700 laptop
  • Intel® Core™ i7-3740QM CPU @ 2.70GHz × 8
  • Quadro K3000M/PCIe/SSE2 (nvidia 352.63 updates)

Results

• Victor's desktop
  • OpenGL: Big lag when switching from surface to point or wireframe. When turning the model the image flickers or is not displayed.
    The model is displayed as a point cloud when turning the view.
  • OpenGL2: Everything smooth (at least 40 FPS). No lag
• Dell M6700 laptop
  • OpenGL: Big lag when switching from surface to point or wireframe. When turning the model the image flickers or is not displayed.
    The model is displayed as a bounding box when turning the view.
  • OpenGL2: Small lag when switching from surface to point or wireframe. The model is displayed as a point cloud when turning the view.

I tested two machines where the VTK OpenGL2 backend simly doesn't work (GLXBadFBConfig), I did not manage to fix that, it's most probably a driver issue.

In a general way, the gain is very good when using only VTK. When using PCL the gain is not as big.
But it works 👍

[taketwo]

Can you please try to call setUseVbos(true) on visualizer object when in OpenGL backend mode? Does this give any improvement? And if so, can it compete with performance on OpenGL2 backend?

[VictorLamoine]

With viewer.setUseVbos(true) (VTK OpenGL backend) the mesh is not displayed... much better performance 😄

[taketwo]

Hah, so it was broken anyway.
Thanks for trying, I think we are safe to merge this now.