Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dont Require Full Access To PubSub Router And Use Minimal Interface Due To PubSub Topic Join Restrictions #50

Closed
bonedaddy opened this issue Jan 14, 2020 · 1 comment · Fixed by #51

Comments

@bonedaddy
Copy link
Contributor

Recent pubsub changes make it so that you can only join a topic once per instance of go-lib2p-pubsub.PubSub. If you provide a PubSub instance to this pubsub router, and then in a different goroutine try to join the same topic, you will get an error.

The latest changes to the way go-libp2p-pubsub (which personally I like) means you need to implement topic management yourself (as you must do now) or have go-libp2p-pubsub implement internal topic management, that ensures when you call Join on a topic, you dont actually attempt to create multiple topic handlers.

It is entirely plausible that one might want to also process messages coming in on the topic subscription both in this library, and other ones.

Personally speaking I have made the following changes to this library:

// MinimalPubsub allows us to provide the bare minimum pubsub functionality
// to the router, while still using the same pubsub instance in other goroutines.
// This is primarily done to allow callers to not have to worry about topic management
// due to topics only allowing a single "joiner".
type MinimalPubsub interface {
	RegisterTopicValidator(topic string, val pubsub.Validator, opts ...pubsub.ValidatorOpt) error
	Join(topic string, opts ...pubsub.TopicOpt) (*pubsub.Topic, error)
}

// NewPubsubValueStore constructs a new ValueStore that gets and receives records through pubsub. Ignore the `zap.Logger` argument, my fork uses the zap logging library directly
func NewPubsubValueStore(ctx context.Context, host host.Host, ps MinimalPubsub, validator record.Validator, logger *zap.Logger, opts ...Option) (*PubsubValueStore, error) {...}

Then in my custom ipfs node, I wrote a shim around go-libp2p-pubsub.PubSub that handles shared access to a pubsub topic by multiple goroutines (ie, go-libp2p-pubsub-router subscribed to a topic, and some arbitrary service using the same pubsub instance is performing processing and messages coming in on the topic)

@Stebalien
Copy link
Member

Go for it, I cant' see any reason not to do this. In the future, we should be able to reduce that to just Join because topic validators should start belonging to topics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants