Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Statement store #13701

Merged
merged 92 commits into from
May 4, 2023
Merged

Statement store #13701

merged 92 commits into from
May 4, 2023

Conversation

arkpar
Copy link
Member

@arkpar arkpar commented Mar 24, 2023 •
edited
Loading

This implements https://github.com/paritytech/labs/issues/3
See README.md for more details.

Still left to do (in follow-up PRs):

  • Encryption
  • API for reporting statistics
  • Better prioritizatoin of locally inserted statements w.r.t. storage and propagation.

Additional changes required after chatting with @gavofyork

  • Calculate statement count/size limit per account, and not just globally. This is based on balance.
  • Add a Channel field to allow replacing statement. Only each statement with a unique pair (account, channel) may exist.
  • Reintroduce Priority field. This will be used to prioritize statements withing the per-account limit.
  • Remove time-based statement expiration

polkadot companion: paritytech/polkadot#6995
cumulus companion: paritytech/cumulus#2423

@arkpar arkpar requested review from gavofyork and bkchr March 24, 2023 08:32
muharem
muharem reviewed Mar 27, 2023
View reviewed changes
primitives/statement-store/src/lib.rs Outdated Show resolved Hide resolved
primitives/statement-store/src/lib.rs Outdated Show resolved Hide resolved
primitives/statement-store/src/lib.rs Outdated Show resolved Hide resolved
frame/statement/src/lib.rs Outdated Show resolved Hide resolved
frame/statement/src/lib.rs Outdated Show resolved Hide resolved
muharem
muharem reviewed Mar 27, 2023
View reviewed changes
frame/statement/src/lib.rs Outdated Show resolved Hide resolved
muharem
muharem reviewed Mar 27, 2023
View reviewed changes
client/statement-store/src/lib.rs Outdated Show resolved Hide resolved
client/statement-store/src/lib.rs Outdated Show resolved Hide resolved
@arkpar arkpar requested a review from a team as a code owner April 25, 2023 06:27
@arkpar
Copy link
Member Author

arkpar commented Apr 25, 2023 •
edited
Loading

@bkchr @cheme

  1. RFC/PSP Is definitely coming. This will have to be supported by other implementations, such as smoldot.
  2. I've removed global priorities altogether. The store simply returns an error when it is full. The whole global priority logic was a bit confusing and achieved very little.

@arkpar
Copy link
Member Author

arkpar commented May 3, 2023 •
edited
Loading

@paritytech-ci @paritytech/ci
I've added a couple of crates which are licensed under GPL v3.0 with a classpath linking exception. This necessitated an update of the exception list here:
https://github.com/paritytech/substrate/pull/13701/files#diff-ac8638f1b54d80feca22f37bd2cdba387fd0ec6aaa8b9689c8b41e2a50632ccc

Looks like your review is needed to merge this.

@paritytech-ci paritytech-ci requested a review from a team May 3, 2023 14:20
@arkpar
Copy link
Member Author

arkpar commented May 4, 2023

bot merge

@paritytech-processbot
Copy link

Error: "Check reviews" status is not passing for paritytech/polkadot#6995

@arkpar
Copy link
Member Author

arkpar commented May 4, 2023

bot merge

@paritytech-processbot paritytech-processbot bot merged commit d3ce947 into master May 4, 2023
@paritytech-processbot paritytech-processbot bot deleted the a-statement-store branch May 4, 2023 10:24
@github-actions github-actions bot mentioned this pull request Jun 15, 2023
Closed
@nazar-pc nazar-pc mentioned this pull request Jun 16, 2023
Merged
1 task
@kiltbot kiltbot mentioned this pull request Jun 21, 2023
Closed
@kacperzuk-neti kacperzuk-neti mentioned this pull request Jun 23, 2023
Merged
15 tasks
AurevoirXavier added a commit to darwinia-network/darwinia that referenced this pull request Jul 11, 2023
@AurevoirXavier
Companion of paritytech/substrate#13701
6410753
nathanwhit pushed a commit to nathanwhit/substrate that referenced this pull request Jul 19, 2023
@arkpar @cheme
@bkchr @nathanwhit
Statement store (paritytech#13701)
13ca13f 
* WIP Statement store

* Sync with networking changes in master

* WIP statement pallet

* Statement validation

* pallet tests

* Validation queue

* Store maintenance

* Basic statement refactoring + tests + docs

* Store metrics

* Store tests

* Store maintenance test

* cargo fmt

* Build fix

* OCW Api

* Offchain worker

* Enable host functions

* fmt

* Minor tweaks

* Fixed a warning

* Removed tracing

* Manual expiration

* Reworked constraint management

* Updated pallet constraint calculation

* Added small test

* Added remove function to the APIs

* Copy-paste spec into readme

* Comments

* Made the store optional

* Removed network protocol controller

* fmt

* Clippy fixes

* fmt

* fmt

* More clippy fixes

* More clippy fixes

* More clippy fixes

* Update client/statement-store/README.md

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Removed sstore from node-template

* Sort out data path

* Added offline check

* Removed dispatch_statement

* Renamed into_generic

* Fixed commit placement

* Use HashSet for tracking peers/statements

* fmt

* Use ExtendedHostFunctions

* Fixed benches

* Tweaks

* Apply suggestions from code review

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Fixed priority mixup

* Rename

* newtypes for priorities

* Added MAX_TOPICS

* Fixed key filtering logic

* Remove empty entrie

* Removed prefix from signing

* More documentation

* fmt

* Moved store setup from sc-service to node

* Handle maintenance task in sc-statement-store

* Use statement iterator

* Renamed runtime API mod

* fmt

* Remove dump_encoded

* fmt

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <git@kchr.de>

* Fixed build after applying review suggestions

* License exceptions

* fmt

* Store options

* Moved pallet consts to config trait

* Removed global priority

* Validate fields when decoding

* Limit validation channel size

* Made a comment into module doc

* Removed submit_encoded

---------

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
@arkpar arkpar mentioned this pull request Oct 4, 2023
Merged
@HCastano HCastano mentioned this pull request Oct 30, 2023
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@gavofyork gavofyork gavofyork left review comments

@muharem muharem muharem left review comments

@cheme cheme cheme approved these changes

@bkchr bkchr bkchr approved these changes

@alvicsam alvicsam alvicsam approved these changes

@sergejparity sergejparity sergejparity approved these changes

@koute koute Awaiting requested review from koute

Assignees
No one assigned
Labels
A0-please_review Pull request needs code review. B1-note_worthy Changes should be noted in the release notes C1-low PR touches the given topic and has a low impact on builders. D5-nicetohaveaudit ⚠️ PR contains trivial changes to logic that should be properly reviewed. E3-host_functions PR adds new host functions which requires a node release before a runtime upgrade. T0-node This PR/Issue is related to the topic “node”.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@arkpar @muharem @gavofyork @cheme @bkchr @alvicsam @sergejparity