Better Gizmo API

[nicopap]

Gizmos as release in 0.11.0 share all a global config for:

• RenderLayers the gizmo uses
• whether gizmos are enabled or not
• some perspective and width related settings

This is problematic. Gizmo rendering relates to specific features. It might be bevy features or plugin features.

Ideally, we can control, enable and disable gizmos per feature, rather than as a whole.

Take two plugins, one that adds nav meshes and another that adds physics objects.

Since there can be a lot of physics objects on screen, the author of the physics plugin will use the default configuration for line width.

But nav meshes are generally lines, so it's important that they are visible, so the nav mesh author sets the
line width to something larger.

Comes in the user. He'll see two different plugins that use gizmos in conflicting ways. He might also want to draw nav mesh gizmos always in front of everything else, while letting physics object gizmos be occluded by the terrain.

How can we reconciliate all of this?

1. Vertex-level config

The most straightforward answer would be to say "let user specify the line width of gizmos when spawning them".

This way, the nav mesh plugin can set their own line width without affect the physics plugin.

But now the user can't control whether nav mesh gizmos are drawn always in front! It would be up to the nav mesh plugin author to provide a way for the user to communicate the front-ness of the gizmos it wants.

This is more work on the side of the plugin author, more ways for their code to fail, and more work on the user side to get something supposedly simple working.

2. Gizmo registry

Note that (2) can be built on top of (1) and that's maybe what we want.

So what I'm suggesting here is not revolutionary. It is mostly inspired by how bevy_diagnostic works today.

Instead of having a single GizmoConfig we would have a store of gizmo configs. You can insert a new gizmo config with your own key (typically the key should be derived from TypeId of a user-defined type).

Then, when using Gizmo methods, you also pass the key.

It's also conceivable to associate systems/sets/game states with specific configs, so that when using the Gizmo methods, you don't even have to specify the config key.

Then, we can provide an API on top of the gizmo store to control each individual configs. The users is free to toggle/change width/front-ness each config they have the key to.

This also has the advantage of leaving a private corner for people who don't want others to mess with their gizmos, by not exposing the gizmo key.

In term of implementation, it would just require spawning a distinct mesh per config. Since we can expect one, maybe two gizmo configs per plugin, this seems very reasonable.

[aevyrie]

This sounds like a problem that would be better solved using Entities and Components in the ECS. If you want to group settings, associating that with an entity and a settings component seems like the most natural way to handle this. If you want to keep the immediate-like behavior, then you could make the gizmo entities auto-despawn at the end of the frame. That would solve the most annoying part of the retained mode style, which is the need to store a handle and do bookkeeping.

Instead of

gizmos.line(points);

you would do

commands.spawn(Line::new(points)); // called every frame, automatically despawns

and if you want to override defaults:

commands.spawn(Line::new(points) // called every frame, automatically despawns
    .width(55.0)
    .color(Color::BLUE)
);

That could also open the door to an actual retained mode API, for users who don't want the spawn/despawn overhead:

let line_entity = commands.spawn(
    Line::new(points)
        .retained() // sets a flag on the component so it isn't despawned automatically.
    ).id();

[nicopap]

The advantage of what I suggest is to let bevy do the bookkeeping. This way we get the immediate mode API and the gizmo metadata. It also allows a centralized registry for all the gizmos created by dependencies. This could be integrated for example in an editor as different "views" (view displaying physics object/light locations/aabbs/navmeshes/networked entities/any combination thereof) without requiring additional effort from 3rd party plugins. It would work "as is" since registering a gizmo would be part of the workflow.

I also am in favor of a retained mode API, and the bookkeeping would extend naturally to it. Bonus is we have a unified system to control retained gizmos & immediate gizmos.

[aevyrie]

My suggestion is basically identical to your API, but it uses the Entity as the key and a Component in the ECS instead of a dedicated registry. Bevy is still doing the bookkeeping - you spawn the gizmo every frame and specify any settings, there is nothing to hold onto unless you want the entity to stay around.

[Vrixyz]

I think @aevyrie is right to direct the discussion towards entity/components, but rather than leveraging them to spawn gizmos, I suggest to use them for configs, I think I'd like a solution like:

fn system(mut gizmos: Gizmos, my_gizmo_conf: Query<GizmoConfig, With<MyGizmoConfigMarker>>) {
    let my_conf = my_gizmo_conf.single();
    let gizmos: GizmosWithConfig = gizmos.with_config(my_conf);

    // same API as `Gizmos` but then call more specific functions like proposed in "1. Vertex-level config"
    gizmos.circle_2d(position, 10f32, Color::GREEN);
}

This could be compatible with a "default" gizmo configuration, and no breaking changes.

[Mampfinator]

I'd personally be in favour of a registry-style solution with a central resource, since I feel it'd offer some nice ergonomic benefits over entity-component-based solutions without compromising flexibility.

We could have type-based keys for configs:

fn main() {
    // this *could* be moved to the `GizmoConfigs` resource on the plugin-author-side, 
    // but I'd prefer to do this during build so Startup has access to all configs
    // for straightforward user-side customization.
    app.configure_gizmos::<MyGizmos>(GizmosConfig { ... });
}

fn system(mut gizmo_configs: ResMut<GizmoConfigs>) {
    gizmo_configs.get_mut::<other_crate::TheirGizmos>().width(5.);
}

And for ease of drawing, we could let Gizmos directly know which key to use

pub struct DefaultGizmos;

pub struct Gizmos<'s, Config = DefaultGizmos> { ... }

which in user code would translate to:

fn system(
    mut default_gizmos: Gizmos, 
    mut my_gizmos: Gizmos<MyGizmos>
) {
    ...
}

This shouldn't introduce any major breaking changes (since it mostly just extends the current API), and introduce only minimal boilerplate for users. It feels a lot more natural to me as well.

[jeliag]

I started playing around a bit and built 3 prototypes using type-based keys. Have a look at the 3d_gizmos example to see the API in action. The main difference is how the configurations are stored:

1. Central Config Storage I like this one the most but it reinvents some functionality which the ECS already provides.
2. Configs as Components This one is a bit less clean but makes use of the ECS.
3. Configs as Resources This one is very simple but you can't easily iterate over all config resources to enable/disable all gizmos.

While building these I realized that we don't need the first point (Vertex-level config) to enable multiple configs. We need multiple meshes anyway to allow different render layer configurations and thus can just keep using uniforms for the other options like line_width and depth_bias. The question of how much we should move to the vertex level to allow for fine grained control seems orthogonal to the discussion of how to enable multiple configurations.

One goal was also to provide a standardized way how plugins can provide more specialized gizmo settings. For example the AabbGizmoConfig or FrustumGizmoConfig (#10038). In all three prototypes, the type used for the key doubles as the specialized config struct.

If and how the selected configurations can be overridden when using the gizmos wasn't part of my experimentation. But I think my solutions leave many paths to explore.

[nicopap]

Thank you so much for doing this work. It helps the discussion. And if I'm not mistaken, it looks like you implemented the store as well.

My personal favorite here is the central config store, but you might already know it :P. This is really good thank you again.

I don't think vertex-level config is necessary in order to enable a registry API. Quite the contrary. If the config is per registry entry, it's easy to spawn one mesh per registry entry, and use the config for the whole mesh.

[nicopap]

Would you like to open a PR? I can also open one based on your code. What do you think? I think we should add it to bevy.

[jeliag]

Opening a PR was my initial goal, but I decided to wait for more discussion as I was unsure which approach would be best. I should be able to open a PR for the first branch next week.

[Vrixyz]

I just had another idea: using Assets to customize the config: #10342 (comment) , as it seems it wasn't discussed, I'm linking it here too.