[Discussion] What to do with Depth in Stride UI?

[MechWarrior99]

Currently the Stride runtime UI supports depth and thickness in the UI. This is a very cool feature when making world space UI. However part of this is the ability to make elements with depth/thickness. But there are some limitations and caveats that make it impracticle to use, and harder to add new features.

The main limitation that prevents it from being useful in its current form is that it only supports solid color cubes, meaning textures will just appear as flat planes. As such I don't see it ever actually being useful for a game's UI, and even if it was, it would be limited to a very small number of games with a very specific UI style.

Examples:

Keeping it in as it is, also hase some issues. It adds complexity to the editor when creating the UI, and complexity to the engine codebase.
In the future, doing any form of vector graphic based UI also becomes many times more expensive to render, and shader based UI is more complex.

An example of this is if we wanted to have rounded corners. For 2D that is straightforward, but handling it for 3D cubes becomes much more complex because it introduces the concept of edges, along with 2x the number of corners (not impossible, just more complex).

So, with these in mind I want to start a discussion on what the ideal solution would be with how to handle elements being 3D, as currently I feel it is safe to say it is a 'useless' (though neat) feature for actual games.

Here are a few options I have thought of.

1. Remove it.
   Pro: Easy and most straightforward option.
   Con: Could be removing a potentially cool and unique feature from the engine.

2. Change it so the texture gets 'stretched' along the side (like what dropped items in Minecraft look like)
   Pro: Keeps and supports a unique feature of the engine.
   Pro: Supports a wide range of styles.
   Con: Would require a custom shader or possibly mesh generation for the sides of the UIElement so that it would look correct for non-square textures. Meaning it would be more expensive to render, and develop and maintain.
   Con: Only works for UIs that have a flatter shaded style, as the outerpixels of a texture would be stretched, and look bad for more textured styles.

3. Don't support textures, but do support rounded corners. But only along the 'depth edge', meaning like you take a normal 2D rect with rounded corners and just extruded it. (This includes supporting borders as well)
   Pro: Keeps and supports a unique feature of the engine.
   Pro: Adds support for rounded corners to 2D as well without the use of hand made textures.
   Con: Only works for UIs that have a flatter shaded style.
   Con: Could be expensive to render/generate.

Please share your thoughts and suggestions! And if you disagree with me on something, that's fine too!

Just as a note, I want to figure out what we want to do, not what feels like the easiest solution (though if the easiest is what we want to do, that works too haha). And as some additional context, I am working on a plan for how to improve the Stride UI, and so this factors in to that.

[VaclavElias]

How does it work in other game engines?

[MechWarrior99]

How does it work in other game engines?

Great question! As far as I'm aware, no other game engine has 3D UI like this. Some have depth offset between elements, but none have thickness on the actual UI elements. So to answer you're question. They don't.
(I know for a fact that Unreal, Unity, and Godot don't. And am pretty confident any others)

[NicusorN5]

As a user, I'd consider 2. But knowing it could not be implemented, this feature could stay.

Most games I've seen use billboards, if they have 3D UI.

[Doprez]

I personally dont see any benefits to having cubes to handle 3D depth. I think billboards just make more sense and since other engines dont do this I imagine it is not a sought after feature.

I would say "1" replace it with a more standard approach.
If we dont want to remove it completely, "2" could also make sense.

Wouldn't hurt to hear @Kryptos-FR, @Eideren and @xen2 opinions on this and maybe ask why it was made this way if anyone knows.

[MechWarrior99]

I could see other 3D but non-cube UI being interesting for a game. But in general I think billboards are used because they have better UX, so it is already a pretty niche set of games that would even potentially use it.

Another option I thought of, is to inroduce a property on the UIElement that handles the type of drawing. So one for texture quad, which has a property for a texture and draws as a quad mesh. One that just has a color property and draws as a cube, another for a custom mesh and material. But that doesn't really get around too much that it isn't really very useful atm.

Yeah it would be amazing if somone happens to know why it was done the way it was, and what the plan for it in the future was (I assume that its current form wasn't intended as the final end result).

[delustra]

I'd say option 1. UI is a pretty standard functionality for gamedev, basically an indpendent layer of 2d on top of 3d. Even in that "simple" paradigm it's a quite challenging job to make a proper UI.
There is a bit of weirdness with UI already (a need to exclude it from the regular render pipeline https://youtu.be/cazIR97VPcg as an example). I fear that keeping it 3d would slowdown development of any new UI features and introduce complexity that would be rarely used. So, i'd say option1 .