Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multi window UI #8780

Closed
wants to merge 20 commits into from
Closed

Multi window UI #8780

wants to merge 20 commits into from

Conversation

ickshonpe
Copy link
Contributor

@ickshonpe ickshonpe commented Jun 7, 2023 •
edited
Loading

Objective

Solution

A separate Taffy layout tree is created for each camera with UiCameraConfig show_ui enabled.
Root UI nodes can select which camera's render target they should be rendered to using the UiView component.

Doesn't support split screen/multiviewport UI. I've got this working but I'm not sure how sound the implementation is and this is already a very large and complicated PR.

The implementation isn't super efficient at the moment. It uses a lot of hashmaps that are regenerated every frame. Most of the issues can be easily fixed by storing these values in components on the UI node entities instead.

eaf024be-e155-4985-9cb9-82bbaf9a6c9d.0.ba411d84-2925-45ca-8c54-48177670b53d.mp4

Changelog

ui_layout_system:

  • Style and Children UI updates walk the tree for each layout instead of iterating through all UI nodes.
  • Also walks each layout tree to update the Node and Transform components after a new layout is generated.

ui_stack_system:

  • Now responsible for managing windows, scaling, and camera views for the UI, instead of 'ui_layout_system`. Decides which window or texture each UI node should be rendered to.

ui_focus_system

  • Interaction works with multiple windows and UI layouts.
  • Added UiCursorOverride resource as a method of enabling interaction for render-to-texture UI.

UiStacks

  • New resource wrapping a Vec<UiStack>.

UiStack

  • No longer a resource. Also holds view information for the stack.

UiSurface:

  • Replaced the window_nodes field with a new Resource UiViewToLayout.
  • Deleted the upsert_node, update_window, set_window_children, compute_layout and compute_window_layouts methods.

UiNodeToView:

  • New resource. Maps UI node entities to view entities.

text:

  • Looks up the scale factor in the UiLayouts resource for each UI node, instead of using the primary window's scale factor.
  • Deleted the create_text_measure and queue_text functions.

UiLayouts:

  • New resource contains a list of UiLayouts each containing the Taffy root node and context information for the UI layout corresponding to a Ui Camera.

UiScale:

  • Can be added as a component to camera entities where the UI will scale the corresponding UI layout by its value instead of applying the scaling from the global UiScale resource.

UiDefaultView

  • New resource wraps a camera entity. Root UI nodes without a UiView component will be added to the layout associated with this camera.

  • new example: multi_window_ui

Migration Guide

@ickshonpe
Changes to LayoutContext:
7940186 
* Renamed the `scale_factor` field to `combined_scale_factor` to reflect how it is the product of the target's scale factor and `UiScale`.
* Added the field `inverse_target_scale_factor`. The UI layout coordinates returned by Taffy are multiplied by this value before being written to the `Node` and `Transform` components of each UI node entity.
* Removed the `min_size` and `max_size` fields, it doesn't seem like there is much benefit from caching this.
@ickshonpe
Made changes to the layout module needed to support multi-window UI.
d4a243a 
Still needs changes to rendering, `UiStack`, and `Focus` for multi-window UI to work correctly.

`LayoutContent`:
* Derive `Copy`, `Clone`, `PartialEq` and `Debug`
* Reodered the paramters of the `new` function so that `phyiscal_size` is first to reflect it's relative importance.

`ui_layout_system`:
* For each camera with UI enabled, it creates or updates an associated Taffy root node which it gives the same size as the camera's render target.
* Calls `update_ui_layouts` to update the UI layout for each camera with UI.

`update_ui_layouts`:
* New function.
* Updates the UI layout for each `UiLayoutRoot`. Does most of what `ui_layout_system` used to do.

`UiCameraToRoot`:
* New `Resource`.
* Maps Bevy camera entities to their associated Taffy Root Nodes. Replacement for the `window_nodes` field of `UiSurface`.

`UiDefaultCamera`:
* New `Resource`.
* If a root UI node has no `UiCameraTarget` component, the root UI node's Taffy node is made a child of the Taffy root associated with `UiDefaultCamera::entity`.

`UiLayoutRoot`:
* Contains the Taffy root node and context information for the UI layout corresponding to a Ui Camera.

`UiTargetCamera`:
* New `Component`. Add to a parentless UI node to set which `Camera`'s associated UI layout the UI node and its descendants should be added to.

`UiSurface`:
* Replaced the `window_nodes` field with a new `Resource` `UiCameraToRoot` to fix borrow problems.
* Deleted the `update_window`, `set_window_children`, `compute_layout` and `compute_window_layouts` methods.

new example:
`multi_window_ui`
@ickshonpe
# changes
5340a2c 
Added `Without<UiTargetCamera>` filter to `root_uinode_query` to stop all UI nodes being added as children to the default camera's UI layout.
@ickshonpe
# changes:
ea064a7 
`ui_layout_system`:
* New `ui_layout_system` deleted, functionality transferred to `ui_stack_system`. `update_ui_layouts` renamed to `ui_layout_system`.

`ui_focus_system`:
* Temporarily disabled.

`UiStack`:
* `UiStack` is no longer a resource.
* Added `base_index` and `camera_entity` fields.

`UiStacks`:
* New resource wrapping a `Vector<UiStack>`.

`ui_stack_system`:
* All the cameras and root context code moved into here.
* Schedule changed so it runs before `ui_layout_system`.

`ExtractedUiNode`:
* Added a `camera` field.

`extract_uinodes` + `extract_text_uinodes`:
* Modified to support the multiple `UiStack`s.

`UiBatch`:
* Added a camera field.

`prepare_uinodes`:
* Create a new batch when camera is changed.

`queue_uinodes`:
* Added `Entity` to `views` query.
* Only adds items to `RenderPhase<TransparentUi>` if camera and view entities match.
@ickshonpe
Modified ui_focus_system to support multiple windows.
afdff7a
@ickshonpe
Clear all UI stacks each frame
fce6e5e
@ickshonpe
Fixed perform_full_update checks.
f1601b5
@ickshonpe
Cleaned up and applied some ci lint suggestions.
2b197f4 
Improved the `multi_window_ui` example a little with a better layout and added some colors.
@ickshonpe
UiLayoutRoot:
1e15123 
* renamed to `UiLayout`
* renamed `perform_full_update` to `needs_full_update`

`multi_window_ui` example:
* Added example of rendering UI to a texture.

`ui_layout_system`
* Separated node insertion and style updates
* Style updates walk the UI tree recursively
* Geometry updates walk the UI tree recursively
@ickshonpe
# changes
b950b07 
`UiSurface`:
* Removed `upsert_node` method.
* `update_children`'s Entity parameter replaced with a taffy node parameter.
@ickshonpe
text module:
d1aa376 
* Look up scale factor in `UiLayouts` resource for each uinode, instead of using the primary window's scale factor.
* The two hashmap look ups per entity probably slow.
* Removed `create_text_measure` and `queue_text` functions.

`extract_text_uinodes`:
* Look up scale factor in `UiLayouts` resource for each text node, instead of using the primary window's scale factor.

`UiLayout`:
* Added field `scale_factor_changed`.

`UiCameraToRoot`:
* Renamed to `UiLayouts`.

`UiNodeToCamera`:
* New resource. Hashmap mapping each uinodes to a camera.

`ui_stack_system`:
* Set `scale_factor_changed` flag on updates.
@ickshonpe
Fixed text_system conditional causing text recomputation every frame.
edd15a7
@ickshonpe
* Added doc comments for new types
817ab2c 
`ui_layout_system`:
* Tiny refactor moving the recursive update functions into the loop.
@ickshonpe
UiCursorOverride:
4df7a99 
* New resource. If set overrides any cursor position value retrieved from `Window`. Enables interaction with UI rendered to texture.

`multi_window_ui` example:
* Implemented interaction with the UI layout rendered to the sprite.
@ickshonpe
LayoutContext renamed to UiContent
920f4b9 
Changed some variable names where they were inconsistant.
@ickshonpe
UiContext:
0797880 
* Added `ui_scale` field.

`UiScale`:
* If added as a component to camera nodes, the UI will scale that layout by its value instead of using the scale from global `UiScale` resource.

`ui_stack_system`:
* Queries for `Option<&UiScale>` and if found sets the value to the `ui_scale` field of the layout's UiContext.
@ickshonpe
Edited comment.
9a4404c
@ickshonpe
Fixed broken doc comment links.
9e4b8cb
@alice-i-cecile alice-i-cecile added C-Enhancement A new feature A-Windowing Platform-agnostic interface layer to run your app in A-UI Graphical user interfaces, styles, layouts, and widgets labels Jun 7, 2023
@ickshonpe
comment edit
b755a6c
@ickshonpe
Fixed an error in comments.
42be8b9 
Renamed `UiContext` back to `LayoutContext`
@ickshonpe ickshonpe mentioned this pull request Aug 29, 2023
Closed
@rparrett
Copy link
Contributor

rparrett commented Feb 1, 2024

Looks like the objective was accomplished in #10559.

@rparrett rparrett closed this Feb 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aevyrie aevyrie Awaiting requested review from aevyrie

@StarArawn StarArawn Awaiting requested review from StarArawn

Assignees
No one assigned
Labels
A-UI Graphical user interfaces, styles, layouts, and widgets A-Windowing Platform-agnostic interface layer to run your app in C-Enhancement A new feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ickshonpe @rparrett @alice-i-cecile