Support for attaching to a parent window

[crsaracco]

This is a tracking issue for supporting passing in a parent handle when creating a window. This functionality will enable users to use Druid as a child window for platform-native windows not created by Druid.

Progress

See rest of issue for context.

General idea for implementation, might not be 100% accurate. Will update this section if needed.

• Platform-independent
  • Figure out what type to use for the window handle in the user-facing API (not c_void *)
  • User-facing API: Add an Option<T> field in WindowDesc, and create a public function for the user to pass the window handle to. WindowDesc::build_native should essentially end up passing the field through to the platform-specific WindowBuilder.
• Windows
  • Add an Option<T> field in WindowBuilder, and create a function for WindowDesc::build_native to call to set it.
  • When creating the window (create_window in WindowBuilder::build), pass the value if the new field is Some, otherwise pass 0 if the new field is None.
• Mac OS
  • Add an Option<T> field in WindowBuilder, and create a function for WindowDesc::build_native to call to set it.
  • When creating the window (WindowBuilder::build), attach to the given parent window if the new field is Some, otherwise create a normal root window if the new field is None. See Raph's vst branch for ideas.
• X11
  • Create an entire X11 backend (see X11 backend tracking issue #475, at least the "bare minimum" bullet point)
  • Add an Option<T> field in WindowBuilder, and create a function for WindowDesc::build_native to call to set it.
  • When creating the window, attach to the given window if the new field is Some, otherwise create a normal root window if the new field is None. See this code for ideas.
• GTK
  • (Is there an equivalent for GTK? Implement if so.)

Background / Motivation

This functionality has been desired by many in the RustAudio group for a while now: it is a requirement for creating GUIs in the VST plugin format, one of the most popular plugin formats in modern Digital Audio Workstations (DAWs). Currently, no GUI crate in the Rust ecosystem fully meets all of the requirements to create GUIs for VST plugins, so this will be a step towards making Druid the first one.

(A "plugin", here, is basically a third-party musical "device" that the user can load into the DAW to synthesize or alter sounds. Plugins typically have a GUI that enables the user to change parameters for this device so that they can tweak the sounds/effects as they see fit, among other features.)

In DAWs following the VST specification, the DAW is responsible for handling the lifecycle of all plugin GUI windows: creation, opening, closing, etc. When the user loads a plugin into the DAW, the DAW probes the plugin to figure out if it supports having a GUI window. If so, the DAW creates the window for the plugin, then passes a handle to that window (the "parent window handle") to the plugin so that the plugin can attach to that window.

This parent handle is always a handle to a platform-native window (HWND on Windows, NSView on Mac OS, or an X11 window ID on X11 systems), which can be used in the platform-native window creation function to "attach" to the parent window. (examples: Windows, Mac OS, Linux/X11)

Therefore, the ideal workflow from a VST developer's perspective would be:

1. DAW loads the plugin, opens a window, and passes a handle to that window to the plugin.
2. The plugin code takes that parent handle and passes it to Druid via Druid's app creation API; Druid then handles the rest of the window creation/attaching process.
3. The plugin code now has access to a window that it can draw on, using standard Druid API calls. The GUI portion of the plugin now functions as a typical Druid application.

Discussion on impact to Druid's current project roadmap scope

Before I get too much farther into the weeds here, I should mention: while I would love to see some crate within the Rust ecosystem handle our use case, I still consider this to be pretty low priority as far as Druid is concerned. I know there's a lot of other stuff on your plate for the near-future, and I would like to disrupt that roadmap/timeline as little as possible.

I'm not specifically requesting the Druid core developers to implement this feature for us; some of us in the RustAudio group are willing to do the grunt work. That said, we might have some specific design questions as they relate to the architecture/vision of the crate, PRs take time to review, and all code in a codebase has some sort of maintenance cost.

This feature might be the first of a few for implementing VST support. I'm not sure what else will be needed afterwards, but I feel like parent window handle support is a self-contained enough feature to not be too disruptive, while allowing us to prove that Druid is a viable option for us, and also allowing us to dig in and identify future requirements.

If you feel like any aspect of us implementing VST support is too disruptive, please let us know and we can hold off. :) Similarly, if you feel like some aspect is completely out-of-scope for Druid as a whole, please let us know that as well.

Implementation details / proposal

Essentially, we have a handle to a parent window, and we want to pass that handle all the way down to druid-shell's platform-specific window creation functions (Windows, Mac OS).

In vst-rs, the API currently gives you a parent: *mut c_void. I don't find this to be a particularly good public API for use in Druid, so I propose using something like rust-windowing's raw-window-handle instead to pass the parent handle around. Alternatively, we could make our own thing if raw-window-handle doesn't suit our needs, or if we want to reduce our dependencies. Either way, it would be up to the developer to transform this *mut c_void (or whatever they have, for other use cases) into the correct handle type.

I think the best place to expose this API to users would probably be in WindowDesc. We could make the function something like:

pub fn parent(self, parent: RawWindowHandle) -> Self

If the user does not call this function, Druid should operate as normal (creating its own window instead of attaching to a parent window).

I think the rest of the implementation is basically just plumbing some data around, but let me know if there's some roadblocks I didn't foresee.

NOTE: X11 support needed to get VSTs working, see #475 for the tracking issue + context.

Unresolved issues and questions

1. DAWs expect to be the main controller of its threads; for example, they expect to be able to call something like window.open() and have the control given back to the DAW when that function is done. Therefore, handling events inside a runloop, which never gives back control to the DAW, is problematic. It looks like we might be able to get around this by making an Application manually instead of launching an AppLauncher, and just not creating that RunLoop. Would there be any issue doing it this way? Would events still be handled? (here's where my knowledge of how GUIs work drops off.)
2. I'm not sure that it's possible to do any of this with GTK. More research needed (anyone familiar with it to know if this would work?).

Update:

In GTK + X11 (i.e. not all gtk platforms) there is: GtkSocket, and GtkPlug which implement the XEmbed spec (caveat, not exactly familiar with any of these APIs/specifications).

Not sure if there's an equivalent for GTK on other platforms.

Miscellaneous resources and links

• The Rust implementation of VST 2.4 that we are specifically targeting is RustAudio's vst-rs crate, but this feature should work with anything that has a similar use case.
• Raph prototyped VST support in druid for the Mac OS platform about a year ago in his vst branch. While a lot has changed in the crate since then, it still might be a good reference (especially for Mac OS implementation specifically).
• I have created a few prototypes in the past as a proof-of-concept that platform-native windowing is viable in Rust for VST use cases. This repo might be a useful reference for implementing this feature in Druid. rtb-rs might also be a good reference.
• I (along with some others in RustAudio) have written up a few pages of background research on the VST GUI problem space in vst2-gui-research.
• @wrl's rutabaga GUI toolkit (in C) is being used in a production VST (which he also made), so it might prove to be a valuable resource as well.

[djeedai]

Note: raw_window_handle::RawWindowHandle is what wgpu uses, which allows binding to wgpu. In that respect, choosing that handle type and exposing it would make things progress too for #891, although for that issue the workflow is reversed (embedding child window into druid, instead of embedding druid into parent window).

[djeedai]

Raw window handle support for Windows and Mac has been merged, see #1586.