Improve UI performance with QOpenGLWidget.

[rryan]

Opening a new PR to make discussion and viewing the diff easier.

Successor to #1863, greatly improves lp:1810099.

116fb61 contains a prototype of my proposed new approach to handling waveform rendering.

Summary of the changes and some background:

• Switch to from QGLWidget to QOpenGLWidget (based on @sgielen's work in 650114d)
• QOpenGLWidget always renders to an offscreen framebuffer, and the QBackingStore composites these framebuffers together when a region is marked dirty (i.e. when we call QWidget::update on any widget). There are two signals QOpenGLWidget gives us:
  • aboutToCompose: When the backing store is about to composite this widget, blitting its offscreen framebuffer to the Mixxx framebuffer.
  • frameSwapped: When the backing store has rendered the widget's offscreen framebuffer to the Mixxx framebuffer.
  • Qt Weekly on QOpenGLWidget
• No more VSyncThread for tracking the swap time. The VsyncThread's only purpose is to send render events to the main thread at a desired FPS (will re-name the class if we agree on this implementation). The render event simply calls QWidget::update on all QOpenGLWidgets and sets a flag that they should render themselves on the next aboutToCompose signal.
  • NB: VsyncThread's swap estimates aren't accurate on macOS, because we set QGLFormat::swapInterval to 1, so every call to swapBuffers was waiting for a new vsync. This means that we waited for a vsync for every QGLWidget in the app (4 decks, 4 spinnies -> 8 vsync intervals at 60 Hz, 133 ms per second spent swapping 😱 ).
• Each QOpenGLWidget tracks its own delta between aboutToCompose and frameSwapped, allowing us to estimate the time until swap on a per-widget basis. VisualPlayPosition is used as usual to interpolate the playposition based on the estimated time to the next swap and estimated time until the current playposition comes out the DAC.
• The estimator used for the time until next swap is a moving average with decay 0.5 of the time between aboutToCompose and frameSwapped signals. I've yet to measure the accuracy of this estimator.
• QOpenGLWidgets are receiving aboutToCompose and frameSwapped signals all the time (e.g. in response to VU meters, knobs, WOverview, etc. re-rendering). We only render on the aboutToCompose signal directly after VsyncThread triggers a render. QOpenGLWidget preserves the contents of its framebuffer from the last render, so we do not need to re-render until we choose to.

Backported this change to 2.2 in #1994:
I also discovered something very weird. Using QStylePainter in WSpinny causes all QOpenGLWidgets (not just WSpinnys) to take 3 ms to render instead of 300 us -- a 10x slowdown. I've removed QStylePainter -- I think where we relied on QSS for WSpinny we should find another way because that's a heavy price to pay. Not sure if it's macOS-specific, or QOpenGLWidget-specific.

With the changes in this PR, performance on macOS is significantly improved. With 4 decks, 4 spinnies, 60 FPS waveforms and 1.4ms latency, fullscreen on a 15" retina (2880 x 1800) display:

• This PR: 65% CPU usage, a tiny amount of interface lag
• 2.2.0: 90% CPU usage, really laggy and hard to use
• 2.1.6: 97% CPU usage, strangely variable lag (comes and goes), not as bad as 2.2.0 though

Setting latency to 90ms, the waveforms are still smooth, which suggests that the VisualPlayPosition interpolation based on the swap interval is working.

[Be-ing]

Test results on Linux with a quad core Core i7 8550U and Intel UHD Graphics 620 using Deere with 4 playing decks and spinnies, Mixxx in full screen mode:

master GL (what I usually use)
~83% CPU for Mixxx
~15% CPU for Xwayland
average reported framerate: ~40 FPS
minimal jerking

master RGB (CPU)
100% CPU for Mixxx
~30% CPU for Xwayland
average reported framerate: ~40 FPS
frequent waveform jerking

master GLSL
~40% CPU for Mixxx
~20% CPU for Xwayland
average reported framerate: ~20 FPS
frequent waveform jerking

This branch GLSL (because that's all that's working currently)
~50% CPU for Mixxx
0.7% CPU for Xwayland
average reported framerate: ~20 FPS
frequent waveform jerking

Comparing the GLSL renderers between this branch and master, it seems that CPU increases for Mixxx are offset by decreases in Xwayland 🤷‍♂️ In all cases the GUI is responsive.

Running with --safeMode, either this branch or master:
~25% CPU for Mixxx
~15% CPU for Xwayland

Potentially relevant GPU info from glxinfo:

Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Intel Open Source Technology Center (0x8086)
    Device: Mesa DRI Intel(R) UHD Graphics 620 (Kabylake GT2)  (0x5917)
    Version: 18.2.6
    Accelerated: yes
    Video memory: 3072MB
    Unified memory: yes
    Preferred profile: core (0x1)
    Max core profile version: 4.5
    Max compat profile version: 3.0
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) UHD Graphics 620 (Kabylake GT2) 
OpenGL core profile version string: 4.5 (Core Profile) Mesa 18.2.6

Something curious is going on with the OpenGL version reported. glxinfo reports my GPU's max supported version is 4.5. With master, Mixxx says OpenGL 4.3 in the waveform preferences. With this branch and Xwayland, I get 2.1. With this branch with -platform wayland is only 2.0 and the GLSL renderer is not an available option.

With this branch, distorted spinnies are drawn over a correctly sized background:

Spinnies look fine in master.

[Be-ing]

Mixxx CPU usage in safe mode with all VuMeter tags commented out in Deere goes down to 8% with Xwayland at 2% (compared to 25% and 15% with VuMeters). I wonder if we could improve performance by making WVuMeter a QOpenGLWidget?

[rryan]

This branch GLSL (because that's all that's working currently)

Thank you for testing! I wasn't really aware anyone used the non-GLSL renderers :P. Just force pushed a new revision that implements rendering for all the QOpenGLWidget renderers.

Something to try, since it's a change from master on Linux: set swapInterval back to 0 here.

[rryan]

Mixxx CPU usage with all VuMeter tags commented out in Deere goes down to 8% with Xwayland at 2% (compared to 25% and 15% with VuMeters). I wonder if we could improve performance by making WVuMeter a QOpenGLWidget?

Yea, we should try this. VU meter rendering isn't too slow itself, but it seems to trigger a re-render of the entire skin. I get much smoother performance when disabling WNumberPos and WVUMeter creation in legacyskinparser.cpp.

BTW, you can set QT_DEBUG_FPS=1 and Qt will print out its own measurement of the FPS it is flushing the backing store at every 5 seconds.

[Be-ing]

The latest commit does not build:

scons: *** [lin64_build/src/waveform/widgets/baseqopenglwidget.o] Source `src/waveform/widgets/baseqopenglwidget.cpp' not found, needed by target `lin64_build/src/waveform/widgets/baseqopenglwidget.o'.
In file included from src/waveform/waveformwidgetfactory.cpp:19:
src/waveform/widgets/glrgbwaveformwidget.h:4:10: fatal error: waveform/widgets/baseqopenglwidget.h: No such file or directory
 #include "waveform/widgets/baseqopenglwidget.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
scons: *** [lin64_build/src/waveform/waveformwidgetfactory.o] Error 1
scons: building terminated because of errors.

[rryan]

Another thing to play around with:
https://github.com/rryan/mixxx/blob/e7a3901e649afc66503e51985ef32abb4e008cc8/src/waveform/visualplayposition.cpp#L65

• disable playpos interpolation (set offsetMicros to 0)
• disable swap interval estimation (set microsUntilSwap to 0)
• disable offsetMicros clamping to within 2 callbacks

[rryan]

The latest commit does not build:

Doh, forgot to add files.

[rryan]

With this branch, distorted spinnies are drawn over a correctly sized background:

Oh cool, I can reproduce with Deere but not Tango.

[Be-ing]

With the latest commit using the RGB GL renderer on Linux, CPU use jumps to 100% for Mixxx but Xwayland CPU use is ~3%. Strangely the RGB GLSL renderer is not available but the Filtered GLSL renderer is. The GUI remains responsive and I probably wouldn't notice the extreme CPU use if I wasn't watching htop.

[Be-ing]

Something to try, since it's a change from master on Linux: set swapInterval back to 0 here.
disable playpos interpolation (set offsetMicros to 0)
disable swap interval estimation (set microsUntilSwap to 0)
disable offsetMicros clamping to within 2 callbacks

None of those make a noticeable difference for me using the GL renderer. In every case CPU usage is around 100% for Mixxx, the framerate reported in the Waveform preferences is ~50-55 FPS, and there is subtle jerking (as there always has been).

[Be-ing]

VU meter rendering isn't too slow itself, but it seems to trigger a re-render of the entire skin. I get much smoother performance when disabling WNumberPos and WVUMeter creation in legacyskinparser.cpp.

CPU use drops down to ~60% (from 100%) with RGB GL renderer when commenting both those out.

[Be-ing]

Is mouse input for knobs and sliders okay now if you comment out the WidgetRenderTimer hacks?

[Be-ing]

With this branch, distorted spinnies are drawn over a correctly sized background:

Oh cool, I can reproduce with Deere but not Tango.

I tested other skins and yes this only happens for me with Deere. Deere uses some ugly hacks in its spinny template. I do not know why the distorted spinnies are not square considering all the sizes in that template are fixed size squares. 😕

[rryan]

Is mouse input for knobs and sliders okay now if you comment out the WidgetRenderTimer hacks?

No :(, still very laggy.

[rryan]

None of those make a noticeable difference for me using the GL renderer. In every case CPU usage is around 100% for Mixxx, the framerate reported in the Waveform preferences is ~50-55 FPS, and there is subtle jerking (as there always has been).

What audio buffer size do you use and what's the "system reported latency" in the preferences?

[Be-ing]

Interesting. IIRC @sgielen reported that mouse input wasn't laggy with knobs/faders when using QOpenGLWidget and building with macOS SDK 10.14. You're building with macOS SDK 10.13, correct?

[Be-ing]

What audio buffer size do you use and what's the "system reported latency" in the preferences?

23.2 ms with ALSA. "System reported latency" is 23.22 ms. Going down to 1.45 ms doesn't change the subtle jerking of the waveforms or make the GUI less responsive. Look at htop, I still see a Mixxx thread taking 100% CPU but with a lower latency I now see another Mixxx thread taking ~17% CPU, which I presume is the audio thread.

[rryan]

Interesting. IIRC @sgielen reported that mouse input wasn't laggy with knobs/faders when using QOpenGLWidget and building with macOS SDK 10.14. You're building with macOS SDK 10.13, correct?

Yep, 10.13 SDK. I'll give 10.14 a shot.

[rryan]

23.2 ms with ALSA. "System reported latency" is 23.22 ms. Going down to 1.45 ms doesn't change the subtle jerking of the waveforms or make the GUI less responsive. Look at htop, I still see a Mixxx thread taking 100% CPU but with a lower latency I now see another Mixxx thread taking ~17% CPU, which I presume is the audio thread.

Gotcha. The VisualPlayposition code uses the audio buffer size, which is potentially incorrect (because if you ask for 10ms and portaudio can only get you 20ms, then it calls the engine with a 10ms buffer twice in quick succession). But in your case the audio buffer size is equal to the latency, so that isn't it.

Does 90 ms latency also have subtle jerking?

[Be-ing]

Does 90 ms latency also have subtle jerking?

Yes, and it also occurs with JACK.

[rryan]

Hm, sounds like I need to do some testing with X and Wayland.

I tested other skins and yes this only happens for me with Deere. Deere uses some ugly hacks in its spinny template. I do not know why the distorted spinnies are not square considering all the sizes in that template are fixed size squares. 😕

Turns out QOpenGLWidget::resizeEvent wasn't getting run (and it has logic that's important). Fixed in dda5096.

Something curious is going on with the OpenGL version reported. glxinfo reports my GPU's max supported version is 4.5. With master, Mixxx says OpenGL 4.3 in the waveform preferences. With this branch and Xwayland, I get 2.1. With this branch with -platform wayland is only 2.0 and the GLSL renderer is not an available option.

Ah yea, what it's currently showing is the OpenGL profile Mixxx requests, which is a 2.1 compatibility profile. That's not showing the maximum supported OpenGL version anymore.

[Be-ing]

Ah yea, what it's currently showing is the OpenGL profile Mixxx requests, which is a 2.1 compatibility profile. That's not showing the maximum supported OpenGL version anymore.

I tried setting the requested OpenGL version in WaveformWidgetFactory::setDefaultVersion higher. I tried 4.5, 4.3, and 3.0, and every time that made the waveforms only render the beatgrid.

[ywwg]

I'm on a new Dell XPS 13 (9370) running Ubuntu 18.10 with Wayland, and almost identical output of glxinfo except Mesa 18.2.2 instead of 18.2.6. I don't have any of the performance issues @Be-ing is reporting. I can run 4 decks at a time non-keylocked and use about 40% cpu. There are occasional frame drops but nothing major. LateNight skin, 10ms latency, GLSL RGB

This is with GNOME 3 desktop environment.

[Be-ing]

I tried wayland mode and the whole UI disappears when I try to make it full screen.

I encountered that bug too.

One thing I've noticed is that the waveforms in master look jerkier than the frame drop meter would imply. I think this reinforces RJ's theory that smooth waveforms are less about our detecting vsync than just writing frames fast enough for the compositor to draw them when needed

I think that would be consistent with my observation that waveforms are much smoother with a small window than expanding to the full width of my screen.

[Be-ing]

I re-added the "beta blocker" tag to indicate we must have this for 2.4.

[daschuer]

I have work in progress

[Holzhaus]

@daschuer What's the status on this?

[daschuer]

I have made a tiny progress since last comment. To many other loose ends.

[Holzhaus]

Any news on this?

[daschuer]

No, my work has stalled due to GSoC.
I plan to continue soon.

[github-actions]

This PR is marked as stale because it has been open 90 days with no activity.

[Holzhaus]

Removed the "blocker" label because we're currently doing a release without this :D

[Be-ing]

Thank you very much for working on this difficult issue @rryan. Your work helped us understand the extent of our technical debt and the requirements for moving Mixxx forward. We have decided to reimplement the whole GUI in QML instead of continue with this.

[rryan]

Music to my ears 😊

…

On Thu, Jul 1, 2021, 7:03 PM Be ***@***.***> wrote: Closed #1974 <#1974>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1974 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAGPH7RDSVRBY5A7TGY3SDTVTX35ANCNFSM4GM3Q73Q> .

[ronso0]

There will be a few more major releases until the QML implementation is finished.
So if I understand correctly this would help improving performance until then.
Just trying to understand, can the remaining issues here be resolved or is this wasting time, also considering this (maybe?) needs to be adapted this to Qt6 later on?
@daschuer

[Holzhaus]

AFAIK QOpenGLWidget is supported in Qt6, just QGLWidget is dropped. I agree with @ronso0 that it may be worthwhile to use this for the transition, unless it's an extreme amount of work to integrate.

[Be-ing]

It would be an extreme amount of work to integrate. There are fundamental architectural issues in this branch which we do not know how to solve.