Fix order of hook execution #10043

gui1117 · 2021-10-18T08:39:35Z

polkadot companion: paritytech/polkadot#4181
cumulus companion: paritytech/cumulus#711

Breaking Change:

AllPallets is deprecated in favor of explicit: AllPalletsWithSystem, AllPalletsWithoutSystem, AllPalletsWithSystemReversed, AllPalletsWithoutSystemReversed. Before this PR AllPallets represented all pallets without system in reversed order (i.e. same as AllPalletsWithoutSystemReversed, it now represent AllPalletsWithSystem as it is a more sane expectation.
frame_executive::Executive requires as generic all pallets with system. Before this PR the type required as generic all pallets without system and always executed system pallet hook before all pallets hook. But this requires duplicate works and error prone implementation thus it now requires all pallets with system. The previous order was to execute the hook on system pallet first and then AllPallets. It is recomended to use one of AllPalletsWithSystem or AllPalletsWithSystemReversed or AllPalletsReversedWithSystemFirst.

Because before the PR all pallets represented all pallets in reversed, and system was executed first then if someone want to keep the same behavior as before they should use the type AllPalletsReversedWithSystemFirst when using the type frame_executive::Executive.
Though it is recommended to be careful about the order of definition of the pallet in construct_runtime. and probably reorganize and use AllPalletsWithSystem, so the order is the expected normal order and not in reversed (as before this PR).

NOTE: frame_executive::Executive will execute the hooks on_initialize, on_idle, on_runtime_upgrade, on_finalize, offchain_worker on the generic AllPallets, so the order should be reviewed for those execution.

Upgrade guide:

use AllPalletsReversedWithSystemFirst when instancing executive type, that mean the execution order is not changed from before this PR, thus no need to change anything in construct_runtime.
Use the new AllPalletsWithSystem, but this would mean that they might need to reorder some stuff in construct_runtime.
review other usage of AllPallets and change to AllPalletsWithoutSystemReversed in case the old order must be kept, or change to AllPalletsWithoutSystem to get the new definition with pallets defined in the order of their declaration in construct_runtime.

frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

frame/support/procedural/src/construct_runtime/mod.rs

gui1117 · 2021-10-18T15:13:27Z

for the polkadot It seems we don't need any changes:
maybe scheduler should be put last instead of 2nd. So that the execution of scheduled call happens after other pallet initialization ?

…-order

frame/executive/src/lib.rs

shawntabrizi · 2021-10-30T14:13:21Z

frame/executive/src/lib.rs

-			OnRuntimeUpgrade
-		>::pre_upgrade().unwrap();
-
+		<(COnRuntimeUpgrade, AllPallets) as OnRuntimeUpgrade>::pre_upgrade().unwrap();


I think this is also a noteworthy change. Here system pallet upgrades after the runtime upgrades.

Dunno actually if this can cause issues?

I think for pre_upgrade/post_upgrade the order wasn't so important as it is just checks.
But the order of on_runtime_upgrade is important. That said the order was: custom, then system then pallets.
I will precise this in the note

try_on_runtime_upgrade should always imitate what the real on_runtime_upgrade would do, which is respected here, so no issues.

- The substrate verion for sp-core was updated to 4.1.0-dev. - The toolchain was updated to nightly-2021-12-10 due to incompatibilities with the toolchain nightly-2021-12-22 - "AllPallets" was replaces by AllPalletsReversedWithSystemFirst according to paritytech/substrate#10043 Co-authored-by: Gaudenz Kessler <gaudenz.kessler@scs.ch>

- The substrate verion for sp-core was updated to 4.1.0-dev. - "AllPallets" was replaces by AllPalletsReversedWithSystemFirst according to Fix order of hook execution paritytech/substrate#10043 Co-authored-by: Gaudenz Kessler <gaudenz.kessler@scs.ch>

* fix order * Update frame/support/procedural/src/construct_runtime/mod.rs Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> * format * more accurate description * format * better explicit types * fix tests * address feedback * add a type to ease non breaking implementation * add comment about constraint * fix test * add test for generated types Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

This reverts commit d36550a.

* fix order * Update frame/support/procedural/src/construct_runtime/mod.rs Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> * format * more accurate description * format * better explicit types * fix tests * address feedback * add a type to ease non breaking implementation * add comment about constraint * fix test * add test for generated types Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Add secp256k1 ecdsa recover (integritee-network#42) * implement secp256k1_ecdsa_recover * fix sgx compilation error * reorder libsecp256k1 in .toml Co-authored-by: suinuj <junius@litentry.com> * Update substrate to b9cafba (integritee-network#44) - The substrate verion for sp-core was updated to 4.1.0-dev. - "AllPallets" was replaces by AllPalletsReversedWithSystemFirst according to Fix order of hook execution paritytech/substrate#10043 Co-authored-by: Gaudenz Kessler <gaudenz.kessler@scs.ch> * Add runtime-api `get_metadata` (integritee-network#47) Co-authored-by: echevrier <edith.chevrier@scs.ch> * Bump substrate to commit 7c63420 (integritee-network#48) * update .tomls to match new substrate version * update rust toolchain * cargo update * revert unnecessary cargo update.. * updae sp-io version * redo toolchain bump - teaclave.. * fix version changes * add missing sp-io functions (integritee-network#49) * update sp-io trie * add root version 2 * add storage::root version2 * Bump substrate to commit f5f286db0da99761792b69bf4a997535dcc8f260 (integritee-network#50) Co-authored-by: echevrier <edith.chevrier@scs.ch> * Bump to polkadot 0.9.19 and switch to polkadot branches (integritee-network#53) Remove TransactionByteFee Apply WeightToFeePolynomials to pallet_transaction_payment Co-authored-by: echevrier <edith.chevrier@scs.ch> * fix cargo.lock * add some pubs * add one more pub Co-authored-by: suinuj <junius@litentry.com> Co-authored-by: gaudenzkessler <92718752+gaudenzkessler@users.noreply.github.com> Co-authored-by: Gaudenz Kessler <gaudenz.kessler@scs.ch> Co-authored-by: echevrier <84318241+echevrier@users.noreply.github.com> Co-authored-by: echevrier <edith.chevrier@scs.ch>

* fix order * Update frame/support/procedural/src/construct_runtime/mod.rs Co-authored-by: Alexander Popiak <alexander.popiak@parity.io> * format * more accurate description * format * better explicit types * fix tests * address feedback * add a type to ease non breaking implementation * add comment about constraint * fix test * add test for generated types Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

fix order

ac5c125

gui1117 added A3-in_progress Pull request is in progress. No review needed at this stage. B3-apinoteworthy C1-low PR touches the given topic and has a low impact on builders. D5-nicetohaveaudit ⚠️ PR contains trivial changes to logic that should be properly reviewed. labels Oct 18, 2021

github-actions bot added A0-please_review Pull request needs code review. and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels Oct 18, 2021

gui1117 mentioned this pull request Oct 18, 2021

Add note about pallet hooks order #10023

Closed

apopiak reviewed Oct 18, 2021

View reviewed changes

frame/support/procedural/src/construct_runtime/mod.rs Outdated Show resolved Hide resolved

apopiak reviewed Oct 18, 2021

View reviewed changes

frame/support/procedural/src/construct_runtime/mod.rs Show resolved Hide resolved

gui1117 and others added 2 commits October 18, 2021 11:05

Update frame/support/procedural/src/construct_runtime/mod.rs

8486f82

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

format

7a1e047

KiChjang reviewed Oct 18, 2021

View reviewed changes

frame/support/procedural/src/construct_runtime/mod.rs Show resolved Hide resolved

gui1117 added 2 commits October 18, 2021 11:26

more accurate description

ba1c95a

format

d94ff4a

gui1117 added A3-in_progress Pull request is in progress. No review needed at this stage. and removed A0-please_review Pull request needs code review. labels Oct 19, 2021

gui1117 mentioned this pull request Oct 19, 2021

Allow pallet's info to be enumerated #10053

Merged

gui1117 added 2 commits October 30, 2021 14:54

better explicit types

15dc877

fix tests

2edfaf7

gui1117 mentioned this pull request Oct 30, 2021

draft: attempt to fix code duplication and AllPallets footgun #10046

Closed

Merge remote-tracking branch 'origin/master' into gui-fix-all-pallets…

ed1ad41

…-order

This was referenced Oct 30, 2021

companion for pallet order fix. paritytech/polkadot#4181

Merged

companion for pallet order fix paritytech/cumulus#711

Merged

shawntabrizi reviewed Oct 30, 2021

View reviewed changes

frame/executive/src/lib.rs Outdated Show resolved Hide resolved

shawntabrizi reviewed Oct 30, 2021

View reviewed changes

frame/executive/src/lib.rs Outdated Show resolved Hide resolved

shawntabrizi reviewed Oct 30, 2021

View reviewed changes

address feedback

b41e935

ferrell-code mentioned this pull request Dec 28, 2021

Change AllPallets to AllPalletsWithSystem AcalaNetwork/Acala#1734

Closed

This was referenced Jan 5, 2022

Update substrate sp-core to version 4.1.0-dev integritee-network/integritee-node#108

Merged

Update sp-core to 4.1.0-dev integritee-network/sgx-runtime#44

Merged

chevdor added a commit that referenced this pull request Jan 14, 2022

Revert "Fix order of hook execution (#10043)"

a50dbbf

This reverts commit d36550a.

chevdor mentioned this pull request Jan 14, 2022

Revert #10043 #10666

Closed

chevdor added a commit that referenced this pull request Jan 15, 2022

Revert "Fix order of hook execution (#10043)"

e6fbbd5

This reverts commit d36550a.

jasl added a commit to Phala-Network/khala-parachain that referenced this pull request Jan 17, 2022

Refactor as paritytech/substrate#10043 suggests

19f9862

Dengjianping mentioned this pull request Jan 21, 2022

Update upstream to v0.9.15 Manta-Network/Manta#359

Merged

12 tasks

mrq1911 mentioned this pull request Jan 25, 2022

chore: upgrade to polkadot 0.9.15 galacticcouncil/Basilisk-node#321

Closed

3 tasks

wischli mentioned this pull request Feb 1, 2022

feat: upgrade to Polkadot v0.9.16 KILTprotocol/kilt-node#319

Merged

14 tasks

dutterbutter mentioned this pull request Feb 3, 2022

chore: removed unused, dead, and deprecated code webb-tools/protocol-substrate#137

Merged

ghzlatarev mentioned this pull request Feb 3, 2022

Refactor for AllPalletsWithSystem Manta-Network/Manta#378

Open

librelois mentioned this pull request Feb 7, 2022

Update substrate/polkadot/cumulus from v0.9.15 to v0.9.16 moonbeam-foundation/moonbeam#1259

Closed

Kailai-Wang mentioned this pull request Feb 8, 2022

Double-check the collation related pallets configuration for litmus litentry/litentry-parachain#336

Closed

nanocryk mentioned this pull request Feb 9, 2022

0.9.16 dependency upgrade moonbeam-foundation/moonbeam#1273

Merged

2 tasks

AurevoirXavier added a commit to darwinia-network/darwinia-common that referenced this pull request Sep 8, 2022

Companion of paritytech/substrate#10043, !

640216b

wischli mentioned this pull request Oct 5, 2022

feat: upgrade to Polkadot v0.9.29 KILTprotocol/kilt-node#416

Merged

6 tasks

mrq1911 mentioned this pull request Nov 3, 2022

chore: upgrade to polkadot-v0.9.29 galacticcouncil/Basilisk-node#571

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix order of hook execution #10043

Fix order of hook execution #10043

gui1117 commented Oct 18, 2021 •

edited

Loading

gui1117 commented Oct 18, 2021

shawntabrizi Oct 30, 2021

gui1117 Oct 30, 2021

kianenigma Nov 18, 2021

Fix order of hook execution #10043

Fix order of hook execution #10043

Conversation

gui1117 commented Oct 18, 2021 • edited Loading

Breaking Change:

Upgrade guide:

gui1117 commented Oct 18, 2021

shawntabrizi Oct 30, 2021

Choose a reason for hiding this comment

gui1117 Oct 30, 2021

Choose a reason for hiding this comment

kianenigma Nov 18, 2021

Choose a reason for hiding this comment

gui1117 commented Oct 18, 2021 •

edited

Loading