Skip to content
This repository has been archived by the owner on Feb 1, 2023. It is now read-only.

feat(sessions): add rebroadcasting, search backoff #133

Merged
merged 5 commits into from
Jun 4, 2019

Conversation

hannahhoward
Copy link
Contributor

Goals

Attempt to kill two birds with one stone:

  1. Don't keep making requests for more providers for the same block (even thought they are already deduped in the provider query manager, still fires off a spurious go-routine in the SessionPeerManager) (improper timer interval lead to out of memory and high load. #107 )
  2. Periodically search for more providers (Occasionally try to find additional providers even if we already have them. #95 )

Implemenation

Start tracking whether a tick is "consecutive" -- i.e no new unique blocks have been received since the last tick. On consecutive ticks do two things different:

  1. Don't kick off another request for providers
  2. Gradually back off the time till the next tick -- cause each tick rebroadcasts the whole want list, which is often expensive. Multiple consecutive ticks essentially mean whatever the session is being used for has largely stalled, so minimize usage just for this.

It might be a problem to never go looking for providers for the same block again but...

By adding a very periodic search for more providers no matter what (i.e. 1 minute -- mirroring the rebroadcast time in pre-sessions bitswap), there is a last ditch insurance policy that you'll keep looking for providers just in case they appear.

And, as a bonus, this means if you are receiving blocks but have slow providers, eventually you'll go looking for faster ones.

Ideally:
fix #95, fix #107

For discussion

At this point, the global options SetProviderSearchDelay & SetRebroadcastDelay are starting to get to be a smell (need options on sessions) and need a refactor but this is a subtle enough change I wanted it evaluated on its own.

@hannahhoward
Copy link
Contributor Author

AAAND there's a data race, so, um, better go ahead and do that refactor after all :)

@hannahhoward
Copy link
Contributor Author

JK not gonna refactor, case that turns out to be complicated and filters all the way up to bitswap and go-ipfs-exchange-interface :(

@hannahhoward hannahhoward force-pushed the feat/improve-provider-requests branch from 58b4db1 to 9b04d41 Compare May 31, 2019 22:03
on a tick, do not keep searching for providers for the same block. instead rely on a periodic search
for more providers. (which will run no matter what, even w/o ticks, to optimize found providers).
also backoff tick time to reduce broadcasts.

fix #95, fix #107
Don't count consecutive ticks if there are no active wants
@hannahhoward hannahhoward force-pushed the feat/improve-provider-requests branch from 9b04d41 to e2e3343 Compare June 3, 2019 23:54
Re-setup provider search delay and rebroadcast delay on a per bitswap instance basis
@hannahhoward hannahhoward merged commit 86089ee into master Jun 4, 2019
Copy link
Member

@Stebalien Stebalien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Belated review.

// Session run loop -- everything function below here should not be called
// of this loop
func (s *Session) run(ctx context.Context) {
s.tick = time.NewTimer(provSearchDelay)
s.tick = time.NewTimer(s.provSearchDelay)
s.rebroadcast = time.NewTimer(s.rebroadcastDelay.Get())
Copy link
Member

@Stebalien Stebalien Jun 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I'm not sure using delay.D is all that useful here as we always just reuse the same initial delay. delay.D is designed for random delays (we'd need to use a ticker and reset it with rebroadcast.Reset(s.rebroadcastDelay.NextWaitTime())).

}
}

func (s *Session) handleRebroadcast(ctx context.Context) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't actually rebroadcast anything, does it? We should probably just call this something like handlePeriodicSearch.

Stebalien pushed a commit that referenced this pull request Jul 4, 2019
feat(sessions): add rebroadcasting, search backoff
Jorropo pushed a commit to Jorropo/go-libipfs that referenced this pull request Jan 26, 2023
…r-requests

feat(sessions): add rebroadcasting, search backoff

This commit was moved from ipfs/go-bitswap@86089ee
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
2 participants