security/p2p: prevent peers who errored being added to the peer_set

[Olshansk]

tl;dr DOS mitigation migrated from tendermint/tendermint/pull/9500

Validated use LocalNet instructions at doc/guides/localnet.md

Original PR description:

This work is a fix for a bug in the P2P layer.

A node can be attacked via the p2p layer by saturating its incoming connection slots and not allowing the node to accept new conditions. This happens when an attacker continuously submits requests to connect with an erroneous message causing the incoming request to error before it has been accepted. The attacked node, tries to remove the peer from its peer set which silently fails (due to the peer not yet being in the peer set). The routine adding a peer into the peer set happens in parallel in the background and will add the peer after the error has been reported.

This fix resolves the issue in the following way:

We add a field removalAttemptFailed to the Peer datastructure.
If removal of this peer fails, we set it to true.
When adding a peer into the peer set, the Add function will return an ErrPeerRemoval error if this field was true and not add the peer.
Note. This attack does not work if the config flag allow_duplicate_ips is set to false.