Distinguish between the system that filters input and the data structure that contains input state

[Xrayez]

Describe the project you're working on

Goost - Godot Engine extension

Executive summary

The default Input implementation doesn't provide a flexible way to control the internal input state, which often leads to "one-way" communication given the current input API, so the main goal of this proposal is to allow managing both internal input state and input logic/filtering of the Input singleton independently of each other. There are a number of requests/proposals which can be resolved by allowing people to create their own input filtering layers via script by providing low-level access via InputState structure, listing some proposals and use cases:

• Add a fighting game input buffer. #100
• Implemented better way to map inputs in godot godot#16797 (comment)
• Implement a way to use deep neural networks in Godot godot#30613 (comment)
• Add Input.get_action_pressed() godot#15681 (comment)
• Ability to see how long an action's been pressed godot#21005 (comment)
• Add Input.parse_action() godot#29363 (comment)
• Expose disable_render_loop property to GDScript godot#39541 (comment)
• Stubbing Input Class bitwes/Gut#171

Having an encapsulated instance of an InputState brings many possibilities such as multi-agent, local multi-player games, input-driven AI, networking, replication via input buffers etc.

The alternative way of achieving the following features is to allow the developers to override the default input implementation (via modules or similar), but there's no way of doing so yet.

Domain

Mostly genre-agnostic, but most useful for turn-based games, fighting games, and other games with multiple characters controlled by the same player (even local and networked multiplayer).

Introduction

Problem

Consider this somewhat typical input handling code that you can see in many tutorials involving character movement:

extends Sprite

var speed = 300.0

func _physics_process(delta):
	if Input.is_action_pressed("ui_left"):
		position += Vector2.LEFT * speed * delta
	if Input.is_action_pressed("ui_right"):
		position += Vector2.RIGHT * speed * delta
	if Input.is_action_pressed("ui_up"):
		position += Vector2.UP * speed * delta
	if Input.is_action_pressed("ui_down"):
		position += Vector2.DOWN * speed * delta

While this works perfecly fine for single-character games (like First-person shooters), this approach becomes unusable for type of games involving multiple characters (like real-time strategy games, or simply turn-based games).

Firstly, Input singleton is global by definition. If we place multiple characters given the same script, naturally they are going to be controlled by the same input which results in synchonous movement. While this could work for certain type of games, this is probably not what we want for the most part:

Secondly, if a game implements certain characters which can be controlled by an AI, we'll likely want to somehow manipulate the dynamics of a character via code as well, so we also need to copy some of the Input internal state to each of the characters in the game and interpret that as input instead (mainly talking about input-driven AI here).

First steps

Lets improve the situation by adding a new motion property to the character:

extends Sprite

var speed = 300.0
export var motion = Vector2()

func _physics_process(delta):
	# First, collect input
	if Input.is_action_pressed("ui_left"):
		motion += Vector2.LEFT
	if Input.is_action_pressed("ui_right"):
		motion += Vector2.RIGHT
	if Input.is_action_pressed("ui_up"):
		motion += Vector2.UP
	if Input.is_action_pressed("ui_down"):
		motion += Vector2.DOWN

	# Finally, process logic given the input
	position += motion * speed * delta
	motion = Vector2()

This has two parts now: the part which collects input and the part which acts upon the input, in our case by actually moving the character. This has some nice benefits:

• makes the "input state" (motion) more independent from Input singleton;
• the property can be modified externally which allows us to possibly implement an AI controller in the future;
• can even be animated!

One of the major downside is that if we do modify the motion property directly, it's still vulnerable to user input, so we need something better.

Filtered input and AI

In a multi-character game, you'll mostly want to control only one or a group of characters out of all available. Lets add controlled property:

export var controlled = false

func _physics_process(delta):
	if not controlled:
		return

That's simple enough. Recall AI stuff? Lets make up something:

export var motion = Vector2()
export var controlled = false
export var human = false

func _physics_process(delta):
	if not controlled:
		return

	if human:
		if Input.is_action_pressed("ui_left"):
			motion += Vector2.LEFT
		if Input.is_action_pressed("ui_right"):
			motion += Vector2.RIGHT
		if Input.is_action_pressed("ui_up"):
			motion += Vector2.UP
		if Input.is_action_pressed("ui_down"):
			motion += Vector2.DOWN
	else:
		# Brownian AI motion!
		motion += [Vector2.RIGHT, Vector2.LEFT, 
                Vector2.DOWN, Vector2.UP][randi() % 4]

	position += motion * speed * delta
	motion = Vector2()

Result:

For simplicity, the primitive AI randomly picks 1 out of 4 given directions here and modifies the motion property accordingly. But what if we add more type of inputs now?

Too many actions

What we've done so far is cloning our global Input state down to our character instances, controlled by either a human or an AI. Lets add some more type of input actions to see how well this scales:

var motion = Vector2()
var sprint = false
var shoot = false
# ... and
# ... much
# ... more

export var controlled = false
export var human = false

func _physics_process(delta):
	if not controlled:
		return
        
	if human:
		if Input.is_action_pressed("ui_left"):
			motion += Vector2.LEFT
		if Input.is_action_pressed("ui_right"):
			motion += Vector2.RIGHT
		if Input.is_action_pressed("ui_up"):
			motion += Vector2.UP
		if Input.is_action_pressed("ui_down"):
			motion += Vector2.DOWN

		sprint = Input.is_action_pressed("sprint")
		shoot = Input.is_action_just_pressed("shoot")
		# ... and many more!
	else:
		motion += [Vector2.RIGHT, Vector2.LEFT, 
                Vector2.DOWN, Vector2.UP][randi() % 4]
		sprint = true if rand_range(0.0, 1.0) > 0.9 else false
		shoot = true if rand_range(0.0, 1.0) > 0.9 else false
		# ... and many more!

    if sprint:
        position += motion * speed * 2.0 * delta
    else:
        position += motion * speed * delta

	if shoot:
		# shoot bullets here
       		pass

	motion = Vector2()
	sprint = false
	shoot = false
	# ... and many more!

Result:

Imagine if you have more than 3 actions... There's clearly something wrong with this approach.

Hardcoded input states

Ideally, we need to remove Input handling dependency from our character scripts, because we don't want the user input to mess up with AI, recorded, or simulated input. In some way, we basically need to keep a local copy of internal input state representation within each character instance.

Unfortunately, there's no easy way to retrieve the entire input state. The current Input API only allows us to scratch the surface of the real input state, here's some example:

func get_pressed_actions():
	var pressed = []

	for action in InputMap.get_actions():
		if Input.is_action_pressed(action):
			pressed.push_back(action)

	return pressed
    
# How about the pressed keys and joypad buttons?..

Similarly, you'd have to implement a bunch of other utility methods, and ensure that you keep all the input metadata like action strength, which is a nightmare if done via script, and in most cases this just leads to "one-way" communication.

Because we do want to make sure that we don't miss any input along the programming hurdles, there needs to be a more elegant and unified way to handle this problem.

Introducing InputState class

InputState is a new type of the Resource which decouples input handling logic from the actual input state. The Input singleton has a master instance of this class exposed as a property. This means that:

• we can have multiple unique instances of the input state;
• share input states between multiple instances (resources are shared by default);
• switch input states depending on the context;
• have a more low-level control over the internal states not provided nor exposed by the Input singleton by default.
• can be serialized, sent over the network, saved to disk etc.

The InputState API mostly replicates the Input API, but only those methods which are responsible for data queries such as is_action_pressed().

To be more concrete, lets rewrite our previous script using InputState class:

export(InputState) var input = InputState.new()
export var controlled = false
export var human = false

func _physics_process(delta):
	if not controlled:
		return

	if human:
		# We have to update our local input state from the global one,
		# in essence requesting a local copy of the state, forwarding it.
		input.feed(Input.state)
	else:
		# This doesn't interfere with the rest of the characters so we can safely
		# modify the state the way we want locally.
		input.release_pressed_events()
		var rand_move = ["ui_left", "ui_right", "ui_up", "ui_down"][randi() % 4]
		input.action_press(rand_move)
		if rand_range(0.0, 1.0) > 0.5:
			input.action_press("sprint")
		if rand_range(0.0, 1.0) > 0.9:
			input.action_press("shoot")

	# Regardless of whether it was a human or an AI,
	# we can use the same API to determine the behavior now.
    
	var motion = Vector2()

	if input.is_action_pressed("ui_left"):
		motion += Vector2.LEFT
	if input.is_action_pressed("ui_right"):
		motion += Vector2.RIGHT
	if input.is_action_pressed("ui_up"):
		motion += Vector2.UP
	if input.is_action_pressed("ui_down"):
		motion += Vector2.DOWN

	if input.is_action_pressed("sprint"):
		position += motion * speed * 2.0 * delta
	else:
		position += motion * speed * delta

	if input.is_action_just_pressed("shoot"):
		pass # shoot bullets here

Benefits

• we cannot possibly pollute the global input state unless we explicitly tell so.
• you can request your local state to be updated from the global input state.
• removes the need to manually copy the internal Input state to each character instance so that there's no need to redefine local variables for each action.
• use your usual Input API locally
• the input state can be shared between instances so that multiple characters can be controlled given the same input
• you can program AI movement by simulating inputs (action presses etc).

Use case: recording input states for playback

Gameplay can be simulated via feeding input events alone, for instance implementing replay system, this can save quite a bunch of memory without having to read the whole game state as events are much more lightweight. State interpolation could be implemented to help this, but in some cases it's just easier to feed those input events as the existing logic can just respond to an input event and you get almost the same behavior without writing some complex logic to replicate those actions on top of the existing logic.

As most engines are not truly deterministic (including Godot), this isn't the perfect solution, but things like:

• instant replay (nice shot, lets see how it went!);
• tutorials (repeat after me!);
• recording combo actions before they can be executed (fighting games): Add a fighting game input buffer. #100;

These could be as well be implemented by simulating inputs as the floating point error would be negligible for short replication episodes for the butterfly effect to kick in, especially if this doesn't affect the gameplay itself.

Recording input events

The process of recording input events was "relatively easy" process:

func _input(event):
	if event.is_action_type() and not event is InputEventMouseMotion:
		input_player.record_event(event) # queues event to be recorded

func process():
	if mode == Mode.RECORD:
		if queued_event:
			track[position] = queued_event
			queued_event = null
		position += 1
	elif mode == Mode.PLAY:
		if track.has(position):
			Input.parse_input_event(track[position])
		position = int(clamp(position + 1, 0, length))

The problem with this approach is that it's not always convenient to use _input callback, because you can't detect a combination of input events easily. Using this approach has also some caveats if used for networking needs, unlike the Input singleton which collects all the parsed input events for us to query.

The only problem currently is that, as previously regurgitated, Input internal state is simply unaccessible for those actions to be replicated more or less deterministically.

That's where InputState comes into action, as it provides more low-level access to internal data. In fact, the whole state can be retrieved via a single data property:

# recorder.gd

var snapshots = [] # a collection of Input.state.data

func process():
	if mode == Mode.RECORD:
		# `input` is a local instance of Input.state
		snapshots.push_back(character.input.data)
	elif mode == Mode.PLAY:
		playback_pos = clamp(playback_pos + 1, 0, snapshots.size() - 1)
		character.input.data = snapshots[playback_pos]

Result:

That's the gist of the process of recording and replaying input-driven game. Unlike InputEvents, the data can also be sent over network, saved to disk etc. The question of whether all of the data is actually useful would be determined by game-specific requirements. You can filter and set the data without worrying about missing fields. In most cases, such data can also be efficiently compressed.

Why this should be part of the core?

No reason, I can just fork Godot.

[Xrayez]

I want to note that this would also simplify the way game state is synchronized over the network if input-driven, deterministic state replication is implemented in a particular game, even partially. Since InputEvent inherits Resource/Reference/Object, it can't be properly serialized nor allowed (deprecated), see godotengine/godot#27485:

Encoding decoding object variant is dangerous, and should never be the default.

An input state represented as Dictionary would be able to be serialized and sent over network.

[Xrayez]

I have rewritten the proposal with introductory material, added some concrete use cases, more insights and implementation. 🙂

[groud]

Sorry but I do not get the use case. For me the code sample you showed is perfectly acceptable and I see little point in being able to store the whole current state of inputs. That's basically what the Input singleton is already doing, I don't really see the point in adding another class for that.

Regarding the recording use case, I think it is not worth it. There are plenty of way to store the inputs on a character, and storing the raw inputs is likely the least efficient one in term of storage. Not even mentioning it's quite a rare use case. I understand why it could be needed to have kind of a "proxy" class with inputs you can modify, but I don't think it belongs to the core.

If you really need to store the state of the current Input, I would go for a simple method in the Input singleton to retrieve the list of existing action. Then you would simply have to store the status+strength of each action, and reuse them when needed.

[Xrayez]

Sorry, I'm not sure if I can explain my use cases better, it might be that what we consider to be acceptable or not is the only difference.

What are you thoughts about the input-driven AI which don't use Input directly? That's also one of the reasons why it's best to have a local state, because you simply can reuse existing actions. The storage and efficiency are mostly the same here, but it also allows you to have multiple agents acting independently without having to redefine all relevant actions as local variables. Currently, it's as silly as:

var starting_shooting = false
var shooting = false
var stopping_shooting = false

func _physics_process(delta):
    starting_shooting = Input.is_action_just_pressed("shoot")
    shooting = Input.is_action_pressed("shoot")
    stopping_shooting = Input.is_action_just_released("shoot")

    if starting_shooting:
        pass # spin up
    if shooting:
        pass # shoot bullets
    if stopping_shooting:
        pass # spin down

If it doesn't make you cringe, I probably I have personal vendetta with this then. Now I'm thinking if that's the place where you ask: why? Ask: "why not?". 🙂

Replace Input.is_pressed... with AI decisions, and you get self-controlled, "input"-driven agent. I put "input" in quotes because this doesn't really come from an input now as there's no code suggesting so, but in effect this is what actually happens:

var starting_shooting = false
var shooting = false
var stopping_shooting = false

func _physics_process(delta):
    # All the AI methods just observe the environment
    starting_shooting = ai.should_start_shooting()
    shooting = ai.should_shoot()
    stopping_shooting = ai.should_stop_shooting()

    if starting_shooting:
        pass # spin up
    if shooting:
        pass # shoot bullets
    if stopping_shooting:
        pass # spin down

Now, lets make a local input state instead:

var input = InputState.new()

func _physics_process(delta):
    ai_make_decisions(input)

    if input.is_action_just_pressed("shoot"):
        pass # spin up
    if input.is_action_pressed("shoot"):
        pass # shoot bullets
    if input.is_action_just_released("shoot"):
        pass # spin down

func ai_make_decisions(p_input: InputState):
        # Some decision making code which observes the environment
        # and controls our agent through input, not code, which involves these methods:
        input.action_press_exact("shoot")
        input.action_press("shoot")
        input.action_release("shoot")

You see how we returned our beloved input handling? One of the neat features I could think of now is that you could even switch different input controllers at run-time, meaning if I want to give up control to an AI (change teams), I can just switch the controller, but in any case I'll be able to use the same Input API, which is very intuitive imo.

That's basically what the Input singleton is already doing

Yep, it's just that it's going to do so via InputState. In fact, most of the implementation was moved to InputState, and the Input API is just a high-level wrapper over the active state. It may seem like the proposal is trying to rewrite everything, but this isn't the case.

There are plenty of way to store the inputs on a character, and storing the raw inputs is likely the least efficient one in term of storage.

Godot is already general-purpose enough, I might be wrong but the current development ideology prefers engine design/readability over efficiency in most cases. As I wrote, you're free to filter the data that you're interested in to make this more efficient, if this ever becomes a problem. In my experience, compression algorightms in Godot do a good job at this as well.

Not even mentioning it's quite a rare use case.

It would be a good starting point at promoting these techniques to be used in other type of games.

[groud]

If it doesn't make you cringe, I probably I have personal vendetta with this then.

Well, it does not make me cringe. At all, to be honest. ^^ Transforming raw inputs into their corresponding actual behavior actually make sense. In a game, the same action can be used for different purposes depending on the context, so it makes sense to map it to a local variable, making explicit what the action is about in the given context.

Regarding the AI use case, I think you believe that you often have to have an AI take control over the inputs of a character. I might be wrong but I don't think it is very common in fact. Coding an AI by forcing yourself to use the same input system as the one the player uses is maybe realistic (in the way that the AI won't be able to do something a user can't) , but I believe it ot be significantly more tedious than using the character's internal variables directly.

I see this proposal as adding yet another layer of complexity over a system that does already have too much: the raw events, the Input singleton storing the state, and the mapping of states to actions. I don't think adding another built-in layer of "local state" only to cover a use case of an AI taking control of a character is worth it. As I said before, I would just allow an API to store all actions states and do whatever she/he wants with it (including creating her/his own internal layer)

[ghost]

Coding an AI by forcing yourself to use the same input system as the one the player uses is maybe realistic (in the way that the AI won't be able to do something a user can't) , but I believe it ot be significantly more tedious than using the character's internal variables directly

Yes, it tends to be a extra step. An AI controller can just hook in directly. If it would hit a button to jump, and this input calls some code, just skip to calling that same code rather than indirectly calling it.

I'm not sure if I'm missing or misunderstanding something regarding the proposal.

Of what I read what I imagine could be a nice feature is a built-in method for recording and compressing the input, and input playback for making game replays.

This would probably run into issues though, because it may also need to introduce an option to control precision of analog inputs.

To really compact input recording for 60 fps, it seems dropping unnoticeable decimals in analog inputs is needed. Though what is noticeable depends on the game.

[Xrayez]

@groud

I don't think adding another built-in layer of "local state" only to cover a use case of an AI taking control of a character is worth it.

Again, we've been talking about all the too-specific use cases which were presented as a nice benefit of the proposed interface, which is mostly refactored stuff, and I'm sure some users who needs this will certainly appreciate that imo. I'm sorry but it seems like "specific" is becoming a trigger word. 🙂

As I said before, I would just allow an API to store all actions states and do whatever she/he wants with it (including creating her/his own internal layer)

As an alternative, I'd propose just allowing to expose raw input data as a dictionary (by providing both setget methods), but further attempts to use this has revealed the following limitations:

1. It's not really clear from the data alone whether the actions was just pressed or being pressed continuously, as this information is dependent on the actual value reported by Engine.get_idle/physics_frames and checked at run-time rather than pure data:

if (Engine::get_singleton()->is_in_physics_frame()) {
	return E->get().pressed && E->get().physics_frame == Engine::get_singleton()->get_physics_frames();
} else {
	return E->get().pressed && E->get().idle_frame == Engine::get_singleton()->get_idle_frames();
}

Meaning that if you do rely on the data, you'd also have to reset the internal frames count, which is hacky. In my PR I just solved this by adding exact boolean, alongside frame count check.

2. When you have your raw internal data, you'd have to basically re-implement the Input API via script eventually, because you're not really interested in the pure data, but the queries you can get from it.

That's why I went for the InputState class which does implement similar interface which is reused by Input singleton.

---

When I grep for InputDefault implementation and usages, it seems like the default implementation is just hard-coded despite the name suggesting otherwise. I would like to inherit from Input directly and use my own implementation, if that's what we're leading to.

[Xrayez]

@avencherus yeah, this is where one can use techniques like delta compression. Most of the times, lets's say there's really only one or two characters which could play at the same time receiving some input (either/both human and AI). If no input is detected (godotengine/godot#35012), it means that a lot of data can be just pruned.

About analog input, if you mean that the precision is lost somehow during serialization, that's one of examples of non-deterministic simulation caused by floating-point error, which I accept with all of my heart by now, so yeah I wouldn't rely on input-driven replication, but in some cases it does makes sense, and can be mitigated by synchronizing the entire game state occasionally. I think that's fairly acceptable for game replays. See also godotengine/godot#16793 where I talk about this, and related to #75.

Aren't most analog inputs restrict the values in the range of [0, 65535] (in spite it being represented as float in Godot as action strength?).

[ghost]

@Xrayez The area of input is something I'm hoping to eventually learn more about. I'm not able to comment on the encoding at the driver level. I would suspect you're correct. There seems like there is some other hidden tricks going on under the hood as well (debouncing etc), because the raw input on things like joypads are really a mess.

Regarding the data compression, yeah a lot of silence can be dropped, and there is also a deeper area of the topic regarding arithmetic encoders. It does seem to be motivated mostly for network transmission. I suspect I might find solutions in the network end of Godot, but also another area of the engine I haven't had time to investigate.

I'm not sure I follow the non-deterministic issue regarding state capture. I was understanding the state capture is mainly used to jump around a replay, to provide some fast forward or rewind feature. Not something to re-align the simulation. (Though I don't doubt developers were probably forced to do this in practice, and have reported on it.)

The floating point rounding outcomes are deterministic.

What I have been doing at least for replays is ensuring the RNG seed is captured, and then for analog:

1. Truncate the decimals to a precision loss that is unnoticeable and convert to an integer.
2. Pack this integer with other input data of the same frame, buffer it to be saved later.
3. Then pass this same integer to the player controller class.
4. Have the player controller class convert it back into a float and apply forces, etc.

When doing a replay, the player controller class is agnostic about where the data comes from, so the replay just feeds the data to it frame by frame exactly the same way. It would appear no different as when it was received from the input class.

So when playing live, the input class is feeding the input data (integers) to the player controller, when in replay mode, the input class is disconnected and the replay feeds the input data in as if it were the input class.

The player controller class will get the exact same integer for the analog inputs, and do the same int to float conversion, and deterministically have all the same rounding errors down the line that the real time input had.

So far haven't had any divergence of playback with just replaying this kind of compressed input at the correct game frame.

[Xrayez]

I'm not sure I follow the non-deterministic issue regarding state capture. I was understanding the state capture is mainly used to jump around a replay, to provide some fast forward or rewind feature. Not something to re-align the simulation.

Re-aligning the simulation would be mainly useful for networking needs, and especially in games in which a player can join mid-game, so either the whole initial state or the current state need to be fully replicated. In that case, the recorded input wouldn't really be relevant anymore, but transferring input data is always more cheaper than sending full snapshots due to limited bandwidth.

You see how replay and networking are closely connected together? Because they try to achieve the same thing: replication!

You're right that there are certainly some steps to ensure deterministic outcomes. There are even some libraries that do integer/fixed point math, and I've seen someone implemented their own physics around that etc. But there's always the inevitable: different compilers, different machines, even different versions of the game can cause desynchronization issues over the network, and you'd end up writing special debugging tools to look for divergences in replication.

If we talk about local game replays that live within a single session of the game, that's where we can alleviate many assumptions to allow us somewhat deterministic outcomes, if that's your use case.

In the demo I linked within a PR, I've written this simple InputController class:

class InputController:
    enum Type {
        HUMAN,
        AI,
        DATA
    }
    var enabled = true
    var type = Type.HUMAN

    func process(input: InputState):
        match type:
            Type.HUMAN:
                # Copies the global input state to our local state.
                input.feed(Input.state)
            Type.AI:
                # Can be anything, but hardcoded as an example.
                input.release_pressed_events()
                var m = ["ui_left", "ui_right", "ui_up", "ui_down"][randi() % 4]
                input.action_press(m)
                if rand_range(0.0, 1.0) > 0.5:
                    input.action_press("sprint")
                if rand_range(0.0, 1.0) > 0.9:
                    input.action_press("shoot")
            Type.DATA:
                # Nothing to do, controlled externally 
                pass

So the Type.DATA could really represent anything: data fed from replay, input buffers for fighting games etc.

So, if you do want to truncate the floats, I think you can do this more easily with this proposal than not, because really you can have a complete control over the input state.

Not to say I fully understand this area myself as well, but so far I think we're doing a good job. 🙂

[Xrayez]

I took two days to research existing proposals, issues and PRs within the main Godot repository which has some connections and what I think could benefit from this proposal, see above linked with my comments in each, thanks in advance!

• Implemented better way to map inputs in godot godot#16797 (comment)
• Implement a way to use deep neural networks in Godot godot#30613 (comment)
• Add Input.get_action_pressed() godot#15681 (comment)
• Ability to see how long an action's been pressed godot#21005 (comment)
• Add Input.parse_action() godot#29363 (comment)

[realkotob]

@Xrayez I love the proposal and I hope this gets more discussion and attention.

One question though, in this example here did you mix up between Mode.RECORD and Mode.PLAY? I might be misunderstanding something.

# recorder.gd

var snapshots = [] # a collection of Input.state.data

func process():
	if mode == Mode.PLAY:
		# `input` is a local instance of Input.state
		snapshots.push_back(character.input.data)
	elif mode == Mode.RECORD:
		playback_pos = clamp(playback_pos + 1, 0, snapshots.size() - 1)
		character.input.data = snapshots[playback_pos]

[Xrayez]

@asheraryam yeah, thanks for pointing out, fixed now, usual copy-paste mistake. 🙂

[Xrayez]

The general idea of this proposal was not accepted by the lead developer, see related discussion starting from: #639 (comment). If you have some use cases which can be solved by this proposal, please do describe them, because I'm largely alone in this (thumbs up are not enough unfortunately), thank you.

I doubt this will be implemented, but I'm obviously not going to close this myself having put so much effort into describing this. The initial proposal was about introducing ability to serialize the internal input state as a dictionary (not adding a new InputState class), so perhaps that's where the engine can lend a hand, not sure. 🙂

But any "semi-complete" features are not acceptable for myself personally so I'd rather come up with a local solution given the situation. Removing InputDefault implementation as in godotengine/godot#37317 could likely make this transition easier as I can just refactor the entire class to make use of InputState without too much effort, and could likely optimize for specific project needs anyway. So likely this could just signify that yeah this feature might be too specific to justify the added complexity.

Feel free to close the proposal, thanks for the feedback.

[realkotob]

@Xrayez I think I'm personally interested in making a GDScript plugin that adds an InputState node and see what happens from there.

I have been thinking about this feature proposal, both my own use cases and others. It's not 100% project-specific since the abstraction can help many projects, but the thing is, at that high level of abstraction the code needed is very simple.

For example for my personal use-case I'm interested in input buffers and coyote time, and to accomplish that cleanly I can: 1) make a strings array for the action names I want to record 2) loop over this array in the _input calls or in _process and record what's been pressed in an array 3) iterate over the records when needed and apply the lagged behavior (for coyote time) and retroactively process the input buffer.

This would also work for fighter combos and can easily serve as an abstracted system to manage NPC/AI behavior as well.

For local multi-player this is less useful, since as reduz said it's easy to work around by appending the player id to the action name, which I've done before and it works very well + many others use the same trick in their open projects.

Sadly, I have to say I understand why this proposal has to be closed, especially since the system was overhauled recently. However, I am interested in making something abstract that we can share for others that are in our situation.

[soloparatolos]

For local multi-player this is less useful, since as reduz said it's easy to work around by appending the player id to the action name, which I've done before and it works very well + many others use the same trick in their open projects.

I ended up landing here due to probably lack of familiarity with the Input system. I'm working on a simple coop game and I was trying to control the Inputs (diferenciate the player inputs) checking the controller ID I give them. I was a bit strange to me that the way to handle the actions for every character should be creating specific "actions" for each of them.

For example, if I have 6 players should I create 6 jump actions, 6 run actions, 6 up, down, left, right, etc. and them call them using "if (Input.is_action_pressed("jump"+str(player)):".

Is this the way is supposed to be done @asheraryam?

[Xrayez]

I think I'm personally interested in making a GDScript plugin that adds an InputState node and see what happens from there.

I'm probably nitpicking here, but I've implemented the InputState as a Resource because it can do something which a Node can't, see Introducing InputState class section in the proposal.

For local multi-player this is less useful, since as reduz said it's easy to work around by appending the player id to the action name, which I've done before and it works very well + many others use the same trick in their open projects.

Yeah, I'm bold in my statements but this is the system which could solve some of the limitations when it comes to local multiplayer, especially when it entails multiple characters per player, so the number of possible states multiply accordingly. Though this proposal is more about controlling multiple characters in a single and/or multiplayer game, but it's great how it could possibly solve other issues.

For example, if I have 6 players should I create 6 jump actions, 6 run actions, 6 up, down, left, right, etc. and them call them using "if (Input.is_action_pressed("jump"+str(player)):".

While researching the use cases I've written some of my insights here: godotengine/godot#16797 (comment)

The API is mostly the same, and instead of "jump"+str(player)" you just state.get_id() which would literally represent the player (or a in-game team, or a group of characters, you name it). Furthermore, what if you want to support more than what you can define in the InputMap? A game controller doesn't have to be linked the InputMap, you just need to have ability to identify the player by the controller ID or similar.

Of course that requires more than what this proposal was originally suggesting, but that's one of the possible future enhancements.

[realkotob]

@Xrayez I apologize for phrasing poorly, I don't disagree with making it a Resource instead of a Node.

Also to be clear I am happy with your earlier proposal and would have definitely used it, but here now I'm trying to see how we can implement this simply in GDScript without unnecessary abstraction.

Having multiple character per local player is definitely niche and we shouldn't add complexity just to support this.

So what I'm proposing is simply being able to record the InputState and then processing it separately. Controllers and input re-mapping should be handled by something else like this.
https://gitlab.com/Xecestel/new-input-map

InputState, how I imagine it, has little to do with the mapping and only acts as an input buffer more or less.

[Xrayez]

Controllers and input re-mapping should be handled by something else like this.
https://gitlab.com/Xecestel/new-input-map

That's a nice find indeed.

InputState, how I imagine it, has little to do with the mapping and only acts as an input buffer more or less.

If we rename InputState to InputController and it would probably be similar in internal mechanism.

Actually to my surprise I've accidentally stumbled upon Unity's InputControl class. In there you'll find InputStateBlock (!) which I suspect has a very similar usage I proposed in here. The feature is in "Experimental" namespace so yeah I've likely went a little overboard with this proposal. But the idea is too great for me to present it already. 😉

[Xrayez]

Some updates and clarifications, and summary.

1. More use cases: Expose disable_render_loop property to GDScript godot#39541 (comment).

2. Renamed polling to filtering as input doesn't go through the main loop directly, yet the proposal's intention doesn't change in any way, so I want to make this clear, quote from Rename InputFilter singleton back to Input #639 (comment).

the way input is handled changed a bit now, as it no longer goes via main loop, plus Input is now only a filter.

Also:

but its making the system considerably more complex where the existing workaround is ok.

It only takes some refactoring work to expose the input structure to scripting, as seen in godotengine/godot#35240, it may seem like a lot of changes but no internal logic is changed for this proposal to be implemented. Not to mention that this kind of change would convey the previous intention behind Input → InputFilter renaming as seen in #639, I don't quite see why this proposal cannot be implemented, it makes perfect sense to me.

If the sole reason behind not wanting to make these kind of changes is the logic of "Solutions in search of a problem.", that's a bad reason IMO, because this kind of fearful logic may prevent Godot growing in the future.

[groud]

The notification I got made me think i forgot to mention why your use case of playback will not work with this proposal. Let me explain why.

When you run a game, you have two loops, one for the frame processing (_process) which is ran as many time as possible per second, and one for the physics (_physics_process) which is ran at a given frequency. From both of those functions you can fetch the current status of the user input, stored by the Input singleton. In a replay, you want the Input to stay consistent all the time, whatever in which function you retrieve the input state.

The problem is that, if you want to store the state of the input to replay it, when do you store it? Every frame? Every physics step? Both of them? Whatever you do, as the _process function is ran as many time as possible per second, you cannot reliably repeat what's by the record of the user input, because you might have more or less frames in one second, so the user input might not be exactly the same from one run to the other, because the timestamp would not fit perfectly. Whatever you do, even a single frame late could have significant consequences when applied in the game (typically, you go one pixel too far, and your character dies into a pit instead of staying on the platform). Even by interpolating between the different states, you would have inaccuracies that would definitely mess up with your playback. For simple case like your demo, you cannot see the difference, but I am 100% sure that if you would video-capture the record and the playback, you would find those very little differences between the two.

And this is not even mentioning the performance degradation this system would cause. Storing the input state every frame into a file would completely destroy your performances. And if done in memory, how much MB are you going to store? With a game running at 240 FPS, your memory usage would grow significantly in a matter of few seconds of recording.

What you propose here has no equivalent anywhere, because reliably replaying something cannot be done from user input. I would advise you to watch this conference from the author of Braid: https://www.youtube.com/watch?v=8dinUbg2h70 (see at 3:20, he explains why replaying input is a "pain in the ass")
Like any other playback mechanic, he has to store the status of the world, not the status of the input, because those are not reliable enough for a precise playback. And then he interpolates the status of the world, which is ok as he can choose what parts are really wanted reliable (like the position of the character, whether he dies or not, etc...).

[groud]

Ah my mistake, apparently some people did use input to replay. But as I tried to express here, this requires you game engine to be deterministic (with fixed FPS, fixed physics steps, and a fixed time computation between frames too), which Godot is not. Godot could provide a deterministic mode, but this is really something avoided in recent games. As fixing your FPS to something like 60 would annoy a lot of people. This was acceptable before, but nowadays, it isn't really.

See : https://www.gamasutra.com/view/feature/130442/developing_your_own_replay_system.php. They implement the input-based recording but took them months to make the engine work in a deterministic manner.

[Xrayez]

Thanks for the feedback!

Yes, I think Godot will never provide determinism out of the box. None of the described use cases could be implemented to full extent given those limitations. In fact that's what I've been saying in the proposal text:

These could be as well be implemented by simulating inputs as the floating point error would be negligible for short replication episodes for the butterfly effect to kick in, especially if this doesn't affect the gameplay itself.

---

Regarding _process vs _physics_process, I already process input in _physics_process in any case, that's just an architectural decision.

Storing the input state every frame into a file would completely destroy your performances.

Yeah, I never suggested to do so either.

Even by interpolating between the different states, you would have inaccuracies that would definitely mess up with your playback.

By interpolating between states I meant combining both world state + input state. Some reconciliation frames can be recorded so that the playback is (mostly) never out-of-sync. And it doesn't really matter as long as you don't add "resume from replay" feature. 😛

And if done in memory, how much MB are you going to store?

Delta compression is one solution. Flushing memory into a file every now and then could also mitigate this. In any case, the resulting size is going to be multitude times smaller compared to recording a video playback of the same gameplay, but yeah it might not be a trivial task to do. 🙂

See also Unreal Engine replay system, I'm not sure how exactly they achieve this, but it's a documented feature nonetheless.

With a game running at 240 FPS, your memory usage would grow significantly in a matter of few seconds of recording.

I think the same can be said if you go for state-based over input-based replication (which could likely consume even more memory).

There are tools and each have their own limitations, I believe it's the responsibility of a developer to decide how to use a particular feature.

If there are any plans to add replay feature support just like in Unreal Engine (especially with the already powerful animation system), that would be great. Yet again this proposal is not restricted to implementing (somewhat specific) replay support. In fact this proposal would only promote this, at least to some extent.

EDIT: I mean, the "all or nothing" logic is not particularly helpful. There are many WIP features already present in core (like navigation), but they are being developed incrementally. What I'm proposing is just a small step, and the amount of discussion probably far exceeds the amount of code to implement this, ironically. 😄

[Xrayez]

See an opinion by the original author of GUT when trying to come up with an easy to use solution (spoiler: not easy): bitwes/Gut#171 (comment).

The suggested InputState in this proposal would help to solve exactly that.

[Riteo]

I too think that this proposal is way too complex and that you're limiting yourself by only staying in the bounds of this specific architecture. I really think that a small external class might do the work by exposing only the specific inputs needed in your game with whatever fancy features you want. You don't need to check for who is controlling the player inside of its logic if your concern is readability or cleanliness, just do it somewhere else, then access the input state there. You're trying to impose one and only architecture in your proposal.

[Xrayez]

@Riteo I'm afraid you judge complexity by the amount of text written used to describe this proposal. It's very simple and relatively straightforward to implement, it's just that it takes time to actually understand the problem. I mean, it's so simple that it doesn't even require breaking compatibility in any way. To the point that existing Input API will stay the same, I'd say it's mostly internal change with InputState exposed as a dedicated class, that's all.

You're trying to impose one and only architecture in your proposal.

I'm sorry but I'm not trying to impose anything, I'm simply suggesting improvements. In fact, what I propose opens the door to implementing a variety of architectures on the project level. I personally think that that's the best proposal I've ever written so far, both in terms of usefulness and quality. I understand that I don't provide concrete use cases to justify such changes, but unfortunately even if I do describe concrete use cases (and even real-life project), I'm afraid that this wouldn't be enough to convince core developers (if you do want to see the project I'm working on, you can go through the edit history in the original post).

But as I said earlier, I don't put hopes high with this proposal anymore (just like with other proposals of mine in Godot, to be honest), because I do have the freedom to do those changes regardless, I'm just surprised that this proposal received so much pushback from the core developers.

Perhaps there's something from this proposal which is still worth to implement in core. It's very cumbersome to copy the state each time, especially when the input actions change during development, making it even more difficult to maintain.

You don't need to check for who is controlling the player inside of its logic if your concern is readability or cleanliness

Readability was never my defining concern with this proposal, more like maintenance aspect if we go this route.

See alternative proposal to solve all input-related problems: #1874.

[Xrayez]

It seems like there's no consensus with this proposal. If core developers decide that this is not necessary to implement, please close the proposal, so everybody can move on. I cannot allow to close this proposal myself because it attracted a bunch of thumb ups so far, so I interpret that this is needed by the community on some level.

And I think I've been regurgitating the same information over and over and I think I've said enough, so I've also unsubscribed from the discussion, please mention me directly if you have specific question which you'd like to be answered, thanks!

[Riteo]

@Riteo I'm afraid you judge complexity by the amount of text written used to describe this proposal. It's very simple and relatively straightforward to implement, it's just that it takes time to actually understand the problem. I mean, it's so simple that it doesn't even require breaking compatibility in any way. To the point that existing Input API will stay the same, I'd say it's mostly internal change with InputState exposed as a dedicated class, that's all.

@Xrayez No, I assure you that I'm not doing that, I'm trying to avoid adding truly unneeded features. While writing my comment I was actually positive with your proposal until it struck me: this can be avoided per game, by having a specialized little class that manages the input in a nice way. It's very rare for a class to allow for its state to be accessed and or modified directly. Still, it's liveable without, and the use cases that you've shown are pretty easy to implement without the a class that represents Input's state.

I'm sorry but I'm not trying to impose anything, I'm simply suggesting improvements. In fact, what I propose opens the door to implementing a variety of architectures on the project level. I personally think that that's the best proposal I've ever written so far, both in terms of usefulness and quality. I understand that I don't provide concrete use cases to justify such changes, but unfortunately even if I do describe concrete use cases (and even real-life project), I'm afraid that this wouldn't be enough to convince core developers (if you do want to see the project I'm working on, you can go through the edit history in the original post).

I've seen your project, but still don't understand on why you'd need to get a snapshot of Input's state. The way you explained it tells me that you want to do input handling and player behaviour all inside of one single class.

Perhaps there's something from this proposal which is still worth to implement in core. It's very cumbersome to copy the state each time, especially when the input actions change during development, making it even more difficult to maintain.

Yeah I can imagine that, but that's to be discussed down the line.

Readability was never my defining concern with this proposal, more like maintenance aspect if we go this route.

A separate class is equally maintainable.

It seems like there's no consensus with this proposal. If core developers decide that this is not necessary to implement, please close the proposal, so everybody can move on. I cannot allow to close this proposal myself because it attracted a bunch of thumb ups so far, so I interpret that this is needed by the community on some level.

And I think I've been regurgitating the same information over and over and I think I've said enough, so I've also unsubscribed from the discussion, please mention me directly if you have specific question which you'd like to be answered, thanks!

Don't feel too bad for this, feel happy to have proposed things in the first place and have had the patience to explain your proposal in every way you could; that's still contributing!

[Xrayez]

I've seen your project, but still don't understand

Well, all I can say is that the only true way to understand this is when you attempt to solve a similar problem. I literally don't know what can be clarified, because you haven't even asked a single question in order to try to understand the proposal.

No, I assure you that I'm not doing that, I'm trying to avoid adding truly unneeded features.

I'd be grateful if you could provide some proof that this is a truly unneeded feature.

[Riteo]

Well, all I can say is that the only true way to understand this is when you attempt to solve a similar problem. I literally don't know what can be clarified, because you haven't even asked a single question in order to try to understand the proposal.

I didn't ask any question because there has been already a pretty thorough discussion, as you noted.

I'm not saying that there's not a problem, just that this proposal doesn't convince me.
I mean, the Input class was meant to not represent its whole state in a separate class. Very rarely (if not at all) a class should have to do that.

What you want is more low level control over how Godot manages inputs. Your use case with the hot-swapping of its state is much more specific, and IMO potentially unneeded, if there were a more low level Input API.

[Xrayez]

After a long learning journey with Godot I've started to explore other game engines and frameworks typically used in game development.

For "just_pressed" actions, I've found an SDL technique which boils down to maintaining current and previous input state snapshots. See this StackOverflow answer on reading input.

The InputState proposed here could easily function this way, and it could work for all events automagically. This is pure magic! 🧙

With SDL, I have ability to copy the entire state and use it however I need it. With Godot, everything is internal...

To elaborate, this basically boils down to change events, a git diff for input states, so to speak. With pressed/released actions, this change denotes that something was either pressed/released, but currently not. With other stuff like mouse motions, you could say that a mouse used to move, but now it isn't moving.

Note that the way Godot distinguishes those input change states is via timestamps instead. Yet timestamping is only done for actions that have a notion of pressed/released state. For example, the Godot's Input API doesn't have a notion of "mouse just stopped moving" or "mouse just started moving" because of that.

Of course, there may be other levels of abstractions (like input actions in Godot), but I hope you get the general idea.

[bchamagne]

Isn't it possible to create an InputManager autoload that steals all the inputs and basically do everything that Input is doing + storing flag HUMAN|MACHINE? Then we could use InputManager.is_action_pressed("foobar") in our scene (almost as usual) and while playing back we just have to do InputManager.disable_human_inputs() before doing our parse_input_event?

I hope I am not out of topic