Scenes and System Management

[chrisburnor]

This is meant to start a more in depth discussion of Scenes than was possible in the linked issue and more durable and shareable than the discord channel. Hopefully it can bring a good discussion that will lead to a design which can then be implemented, likely as a series of independent pull requests.

Background

Scenes are a very powerful tool in game design for modularizing game architecture. It can be used for:

• UI and Navigation (main menu, options, etc)
• Game State management (overworld vs battle, minigames, town vs dungeon)
• Asset Re-use (loading a number of components that go together)

Problem

Right now Bevy Scenes:

• Can not be unloaded, meaning that components have to be manually removed after the scene is done.
• Can not be loaded with systems, meaning that you load your assets, but have no way to ensure that the systems which go along with them are running
• Can not control which systems are running or not, which means that there could be conflicts between scenes that never should run together (say a main menu and battle scene). Sometimes the scene may want to be additive to the current game state, but it may also sometimes need to replace completely the game state, or put the current game state on pause while a dialog, event, battle, or mini game takes place
• Scenes and asset collections are tightly coupled. A scene essentially is a collection of assets in the current model

Requirements

To address these opportunities and problems I suggest the following requirements

• Changing a scene should enable control over how that scene is integrated with the existing game state: Replace, Push, Pop, Merge
• Changing a scene should trigger changes in which systems are running.
• Global systems which run for all scenes (Rendering, UI, etc)?
• Loading scenes should be independent of loading collections of assets

Proposal

To address these requirements, I propose the following class/module changes

• Rename the current scene system to 'prefabs' and leave as an independent module for loading collections of assets
• Create a Scene class that has its own World, Resources and system Scheduler
• Create a SceneManager which implements a Pushdown Automaton state machine for managing the active scene and systems which go along with it

This is meant as a starting point, so I'm curious what the community thinks here. Much of this based on my experience with other ECS and game engines in rust, so feedback on how those have worked well or poorly by other community members would be welcome input.

[cart]

Glad we have another person interested in making Bevy Scenes awesome! Once I wrap up the asset system rework, I'm pivoting my focus to scenes, so now is a great time to have these discussions.

Can not be unloaded, meaning that components have to be manually removed after the scene is done.

SceneSpawner now has a despawn method

Changing a scene should trigger changes in which systems are running.

I think this is an idea worth exploring, but I'm not yet convinced it is the best path forward. I agree that we probably want some way to switch between what systems are running, but whether or not Scenes are "just data" or also "system selectors" is an open question imo. Both approaches have value. The "just data" approach is likely easier to reason about. Its certainly simpler, and I think it fits the "Data | System" separation that ECS encourages. I want to experiment with both approaches before making a decision.

Changing a scene should enable control over how that scene is integrated with the existing game state: Replace, Push, Pop, Merge

This seems like it would get really hairy really fast. My gut reaction is that a more binary spawner.spawn(scene) and spawner.despawn(scene) is much easier to reason about.

Scenes and asset collections are tightly coupled. A scene essentially is a collection of assets in the current model

This isn't true? A scene is not a collection of Assets. It is a collection of entities with components that can be added to a world.

Rename the current scene system to 'prefabs' and leave as an independent module for loading collections of assets

Again, scenes currently dont load collections of Assets, they load collections of entities + components. Assuming you meant components, I think having Bundles, Scenes, and Prefabs is way too many choices. If some future Scene design is capable of spawning entities with components, then I think it should probably just replace the existing Scene system.

Create a Scene class that has its own World, Resources and system Scheduler

On my local asset_rework branch I've already given Scenes their own World. I think worlds are the right runtime representation of collections of components. Doing it during the asset system rework enabled me to load GLTF files as Scenes.

I also think being able to add Resources from Scenes is a worthwhile experiment. At the very least, I want to support in-line assets. I think scenes should be able to embed assets directly. Supporting all Resources gets a little interesting because they are global, which means multiple scenes adding the same resource will produce conflicts.

Thanks again for the interest. I'm definitely curious to hear what the community thinks now that they've had some time to build Bevy projects / learn what they want.

Just to guide the discussion a bit, I want to consider "Scenes" and "State Machines / System Management" two separate requirements that might have a common solution. We need to survey the whole design space before deciding that a combined approach is the right call.

[chrisburnor]

Just to guide the discussion a bit, I want to consider "Scenes" and "State Machines / System Management" two separate requirements that might have a common solution. We need to survey the whole design space before deciding that a combined approach is the right call.

That's a helpful reference point, as are the other comments and clarifications. My survey of the rust game engine ecosystem can be summarized as follows:

Low Level Game Engines without ECS built

This category includes a lot of the well known and popular Rust game engines: ggez, tetra, coffee, rg3d, etc. These libraries focus on graphics, event management, audio, ui and other low level parts of the game, but leave game architecture up to the developer and don't include an ECS built in. Most provide one or more tick methods that one once per frame. Sometimes there is just a single tick method. Others have 'input', 'update' and 'render' but the assumption in each case is that some other game state or game data object contains the world (resources & component stores) and system scheduler which is triggered once per frame in that 'tick' call.

Game Engines with ECS Built In

Right now this seems to only be Amethyst, Oxygen and Bevy. Oxygen's ECS is heavily based on Specs, while Amethyst has now migrated over to Legion. Both ECS kits have a dispatch/schedule concept for controlling system execution order. However it is important to note that for both frameworks, the systems/schedule is registered globally and there doesn't seem to be any way to turn a system on or off dynamically. (This still feels like an anti-pattern to me. I generally shy away from anything being global in even small projects).

They also both have a Prefab/Scene concept (Oxygen even calls it a PrefabScene which focus exclusively on loading of game data (components/entities) and don't touch systems or game state at all. Both also provide State abstractions which use various hooks (on_start, on_exit, on_pause, update) in conjunction with a StateManager and StateTransitions to handle game state.

Observations

There isn't a lot to go compare to, but a few observations do come out of this:

• We've taken it as a given both here and in Scenes #255 that disabling scenes is a requirement, but as @iwikal points out, maybe having systems be globally registered isn't such a bad thing?
• There's strong precedent for keeping Scenes as data decoupled from game state
• ECS System scheduling can either be managed as part of the core game loop (Oxygen, Amethyst & Bevy) or treated completely independently (ggez, tetra, etc). It's unclear to me if there are performance differences between the two, though there might be some value to the former.
• The Bevy concept of a Stage seems to be unique.
• There aren't many examples of dynamically changing systems in runtime, other than to completely replace the schedule/dispatch graph. (Even the stand-alone ECS frameworks don't seem to provide this)

Suggestions/Takeaways

This suggests that we could decouple the conversation about system management even from game state management. I spent a good chunk of time digging through the Bevy examples yesterday and one approach could be as follows (though each of these suggestions can be taken independently)

1. Provide an Api in Commands to replace the current Schedule. This seems to be the lowest touch way to make systems dynamic without adding a lot of complexity and making it trivial to reason about what is currently running. Schedules are immutable so the running systems are always exactly the ones configured in the Schedule. That could be extended with a ScheduleBuilder if we want to create a bit more control over inter-system dependencies and execution order.
2. Implement a GameState plugin using existing Bevy abstractions. This could use the Events API (or a dedicated API along the same lines) to trigger a StateTransition and then implement listeners/handlers for OnPause, OnEnter, etc. If we wanted to have a state machine, or a way to ensure that there is always only one transition handler, that could be managed in a StateManager resource provided by the plugin as well.

The more I think about it, the more I like this approach compared to my first suggestion. The plugin idea could be implemented today without any core API changes and merged in as an additional module, but would still allow users to implement their own, alternative approach if they wanted. The first idea, requires only minimal refactoring compared to my original proposal and still provides a simple, easy to reason about way to manage systems.

[PradeepKumarRajamanickam]

I like the idea of adding add and remove systems apis to command and let the devs take care of adding and removing systems during OnEnter, OnExit events or any other event they want to use. This gives a lot of control to the dev.

[PradeepKumarRajamanickam]

@cart Would it be possible to provide add and remove systems API in command? This would benefit other non-scene related plugins as well such as #629.

[chrisburnor]

@PradeepKumarRajamanickam I did a little bit of poking around with the API and it could be a bit tricky to plumb the scheduler API to the command API as right now they don't seem to have any connection, but I think its possible at least.

I'm more focusing on the Transition Event handler right now using the event apis. I want to see how far I can go with the existing Bevy abstractions. The main challenge I'm having though is that there isn't a clean way to pass prerequisits to a plugin. I'd like to have the type which controls the State to be user configurable so that the user can use an enum or structs or whatever it is they want to control various game states. Right now the only way I can think to do that is to have the user manually enter a resource like BevyState<MyGameStateType> but I'm not sure if the resource lookup API will support doing a lookup without the full type info.

[PradeepKumarRajamanickam]

Ah, I see. The other issue I see is how do we handle events when we have multiple state machine instances (when we have multiple scene instances)?. The current event API (if I am not wrong) will broadcast the event to all consumers i.e. if you send OnEnter it will send to all running machines but ideally each machine should send/receive event messages independently or in a contained fashion. Is there a way to keep event messages local to an instance?

[chrisburnor]

I was thinking that the game state machine would be completely decoupled from the concept of a scene so that there would only ever be one active state. There might be other inactive states, on a stack somewhere, but only one running. The transitions would be mapped to handlers (maybe with a HashMap<Transition, Function> or even just a resource?) so the actual transition event could stay private and ensure no duplicate handlers.

For ergonomics, I do think that there should be some API changes, but mostly just as sugar to prevent users from having to remember to insert State handlers or to pass type information to the transition handler. Something like this:

enum MyGameStates {
  Title, MainMenu, Options, Game, MiniGame,
}
...
app
  .add_plugin(GameStatePlugin) // May not actually be necessary
  .set_game_states::<MyGameStates>() // Adds StateTransition<MyGameStates> to registered events and sets up event listener with a 'default' transition handler which does nothing
  .add_state_transition_handler(MainMenu, MainMenuTransitionHandler)
...
struct MainMenuTransitionHandler;

impl TransitionHandler for MainMenu {
  fn on_start(mut world: &mut World, mut resources: &mut Resources) { ... } 
  fn on_leave(mut world: &mut World, mut resources: &mut Resources) { ... } 
}

[PradeepKumarRajamanickam]

Changing a scene should enable control over how that scene is integrated with the existing game state: Replace, Push, Pop, Merge

I think this might be very useful. I am assuming the following cases
Replace - Usual switch scene
Push - show a modal dialog ui
Pop - dismiss a modal dialog ui
Merge - streaming levels in large open worlds I think this can be used to spawn adjacent zone tiles of levels on the fly but the only question is how do you unspawn them?

[chrisburnor]

For Merge I was thinking that the State would still be an independent object. The difference would be that for Push, the lower State would be inactive and the new State would be active, while in Merge, both would be active. So Merge might be used for a floating window while game time continues, and Push might be used for a modal that pauses game time. Perhaps that should just be an option on Push as to whether to keep existing State active.

[PradeepKumarRajamanickam]

In that case it could be called as Push_Additive. It will be more clear.
I would still prefer an additional mode called Merge for open-world streaming.
Diff being:
When popping a scene that was pushed to stack with Push_Additive, the last pushed will pop off as usual.
Merge should merge into the existing scene instance i.e. when popping, the entire existing scene (i.e. the prev scene that was merged) will also get popped.

As for despawning tiles in open worlds, the devs can destroy the zone/tile root node according to their requirement.

[PradeepKumarRajamanickam]

But I think the way we both are looking at it is slightly different. I see pda for scene management and I think you are viewing it from system management perspective.

[superdump]

@cart If glTF files are loaded as Scenes but Scenes cannot be merged, how do you imagine loading a single model glTF and using it in Bevy?

A glTF file could contain an entire scene (minus Bevy system components - just the hierarchy, lights, cameras, animations, transforms, meshes and materials) or a single model.

It sounds to me like the current Bevy Scene is only different from a Bundle in that it gets its own World? Both are very similar to a prefab - a possibly hierarchical collection of entities and components. Maybe a Bundle is necessarily only one top-level entity whereas a Scene contains 0 or more top-level entities hence if more than 1 it needs its own World. Is that the distinction you see @cart?

What do you expect to happen when two Scenes are loaded? Scenes commonly contain cameras, and lights. If loading a Scene has no way of merging with an existing Scene, you necessarily can end up with multiple cameras. How should that be handled? With a merge step one could say that the Scene being merged into takes precedent and its camera should be used while the one in the new Scene should not be spawned. Or vice versa where the existing camera is despawned and the new one spawned. Can a Scene/World have its origin set so that they can be placed in space? How does a player interact with two Scenes/Worlds being loaded at once?

[chrisburnor]

On my local asset_rework branch I've already given Scenes their own World. I think worlds are the right runtime representation of collections of components. Doing it during the asset system rework enabled me to load GLTF files as Scenes.

In general I think my answer is that scenes can be spawned into a world any number of times. There is no "merging" per-se, just "adding" and "removing" sets of entities/components for each scene. Theres no need for complex merge logic because "spawning" creates new entities every time. Multiple scenes can exist in the same World at the same time.

If there can be multiple concurrent scenes, and each has its own world, how will systems unambiguously resolve which world they're running against?

[alec-deason]

Along those lines, it might also be useful to be able to add a scene as a child of another entity.

This is important for me since I'd like to use scenes as templates for individual objects in the world which might be spawned many times. Possibly even many times in a single tick. At very least there has to be some way to identify the entities created by a particular call to spawn.

[cart]

Agreed (both for the parenting comment and the Transform comment).

The "easiest" way to do this is to mandate that scenes have a single root entity (which is what godot does). Otherwise "translating" a scene is a O(n) operation, where n is the number of "root entities". Multiple roots also creates issues like "where do I set the transform" and "how do i parent this scene".

I think its relatively easy to operate within a "single root" constraint.

[alec-deason]

For my use case, having a single root is perfect.

[Waridley]

Single root would also have the benefit of being more familiar to Godot users, easing the transition and possibly increasing adoption. Also, if it works like Godot's SceneTree, it would allow removing or even reparenting branches that were never actually loaded as separate "Scenes" in the first place.

[nicbarker]

Just some food for thought from a Unity perspective - I've found that scene management in version control is generally hard. As soon as you have a couple of branches with changes to the same scene file they become very difficult to read / merge by eye (hundreds of lines of changes and interleaving across unrelated objects)

I've seen a couple of folks around the place propose the serialization structure of scenes on disk as directories, with one small file per entity (rather than a god file for the whole scene). That way deletions or additions of single objects don't offset every unrelated subsequent line in the file, and common merge conflicts such as "move entity in one branch, delete entity in another" are much easier to resolve.

This would probably make them more difficult to edit by hand though (i.e. without the help of an editor) 🤔

[iwikal]

This might be why Godot has a sort of "flattened" scene format where each entity is at top level, and hierarchy is represented by referencing entity names instead of nesting. I've found Godot's format to work well with git.

[iwikal]

Sort of how the Cargo.lock format works, being another format that has to work well with version control. Each dependency is at top-level and has a list of names of sub-dependencies.

[iwikal]

Another option for a flattened format would be to have each entity refer to its parent instead. Maybe a little less readable? But it removes the risk of creating non-tree graphs.

[JBradBarnes]

Maybe we could add a run, run_once, and inactive states to the stages? That way instead of pushing and popping systems we can run or skip system groups with an abstraction that already exists.

A bonus is that a startup system could be a run_once stage in front of on_start then reused by setting back from inactive to run once if needed again for reloading.
Is there already a generally accepted way to do something like this in bevy?

[cart]

I have a feeling that you will be interested in this pr: #1021

[JBradBarnes]

I love it!!! I think that covers system management in good way.

[Gonkalbell]

I haven't gotten that good of a chance to try out the Schedule v2 rewrite yet, but it seems like it addresses a lot of the use cases mentioned in this ticket. Also, I'm worried that coupling scenes with schedules/states/systems will make it difficult to spawn and de-spawn multiple scenes if they can all arbitrarily change what systems are running. I'm personally also on team Scenes are "just data", since that way the bevy_scene crate can laser focus on serializing and deserializing groups of entities and components, while "Schedule v2" handles which systems should be running.