chore(polkadot): upgrade to v1.0.0

[renauter]

Description

• Upgrade to Polkadot v0.9.43

  • requires rustc 1.70.0 or newer
  • fix tfchain/substrate-node/node/src/service.rs (sc_service::BuildNetworkParams)
  • try-runtime: use proper error types + should not use assert!, assert_eq! or similar because code in these macros may panic
  • update events in python interface
  • CLI options remove --ws-port and --ws-external

• Upgrade to Polkadot v1.0.0

  • upgrade parity-scale-codec to 3.6.1
  • add sc-offchain crate to runtime
  • update node, see https://github.com/substrate-developer-hub/substrate-node-template/commit/e6bf90cfbf774fc284056068fe75e3e97eeb0306 for example
  • moves Block to frame_system instead of construct_runtime and removes Header and BlockNumber
  • use BlockNumberFor instead of BlockNumber
  • GenesisBuild<T,I> deprecated. BuildGenesisConfig added
  • add type AllowMultipleBlocksPerSlot = ConstBool<false> to pallet_aura::Config
  • replace type HoldIdentifier by type RuntimeHoldReason in pallet_balances::Config
  • replace system config Index for Nonce

Related Issues:

• Closes Runtime: update to polkadot 1.0.0 #802

Checklist:

Please delete options that are not relevant.

• My change requires a change to the documentation and I have updated it accordingly
• My commits follow this conventional commits guide.

[renauter]

Following sequence of commands was applied to be able to upgrade gradually:

cargo build

cargo test

cargo build --features=try-runtime

cargo build --features=runtime-benchmarks

cargo test --features=runtime-benchmarks

Concluding by running integration tests:

cargo build --release
cd substrate-node/tests
robot -d _output_tests/ . 

At this stage, all passed successfully

[sameh-farouk]

Thanks Erwan for working on this.
Is there any migrations required between these releases? executing try-runtime gives multiple errors for Scheduler, Historical and Balances storage version.

2023-12-03 17:27:01 Scheduler: On chain storage version StorageVersion(3) doesn't match current storage version StorageVersion(4).    
2023-12-03 17:27:01 Historical: On chain storage version StorageVersion(0) doesn't match current storage version StorageVersion(1).    
2023-12-03 17:27:01 Balances: On chain storage version StorageVersion(0) doesn't match current storage version StorageVersion(1).    
2023-12-03 17:27:01 Detected multiple errors while executing `try_on_runtime_upgrade`:    
2023-12-03 17:27:01 Other("On chain and current storage version do not match. Missing runtime upgrade?")    
2023-12-03 17:27:01 Other("On chain and current storage version do not match. Missing runtime upgrade?")    
2023-12-03 17:27:01 Other("On chain and current storage version do not match. Missing runtime upgrade?")    
2023-12-03 17:27:01 panicked at 'called `Result::unwrap()` on an `Err` value: Other("Detected multiple errors while executing `try_on_runtime_upgrade`, check the logs!")', /home/sameh/projects/tfchain/substrate-node/runtime/src/lib.rs:1022:65    
Error: Input("failed to execute TryRuntime_on_runtime_upgrade: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed\nWASM backtrace:\nerror while executing at wasm backtrace:\n    0: 0x2f54f - <unknown>!rust_begin_unwind\n    1: 0x49f5 - <unknown>!core::panicking::panic_fmt::hf5c4cd929d4aaa9e\n    2: 0x4ece - <unknown>!core::result::unwrap_failed::he0ddc269083098af\n    3: 0xedf74 - <unknown>!TryRuntime_on_runtime_upgrade")

[sameh-farouk]

I did quick search to see the commit when the storage version changed for pallets mentioned in my previous comment:

Balances v0 -> v1:
paritytech/substrate@2704ab3

Historical (session) v0 -> v1:
paritytech/substrate@cc4bf91

Scheduler v3 -> v4:
it seems they forgot to bump the version and did it here:
paritytech/substrate@6d3596f

From PR description alone, I'm not sure if we need to do the migration or just bump the version. see:
paritytech/substrate#13105

[sameh-farouk]

blocking comment

[renauter]

Good catch and thanks for your comments
I ll check that

[renauter]

Is there any migrations required between these releases? executing try-runtime gives multiple errors for Scheduler, Historical and Balances storage version.

2023-12-03 17:27:01 Scheduler: On chain storage version StorageVersion(3) doesn't match current storage version StorageVersion(4).    
2023-12-03 17:27:01 Historical: On chain storage version StorageVersion(0) doesn't match current storage version StorageVersion(1).    
2023-12-03 17:27:01 Balances: On chain storage version StorageVersion(0) doesn't match current storage version StorageVersion(1).    
2023-12-03 17:27:01 Detected multiple errors while executing `try_on_runtime_upgrade`:    
2023-12-03 17:27:01 Other("On chain and current storage version do not match. Missing runtime upgrade?")    
2023-12-03 17:27:01 Other("On chain and current storage version do not match. Missing runtime upgrade?")    
2023-12-03 17:27:01 Other("On chain and current storage version do not match. Missing runtime upgrade?")    
2023-12-03 17:27:01 panicked at 'called `Result::unwrap()` on an `Err` value: Other("Detected multiple errors while executing `try_on_runtime_upgrade`, check the logs!")', /home/sameh/projects/tfchain/substrate-node/runtime/src/lib.rs:1022:65    
Error: Input("failed to execute TryRuntime_on_runtime_upgrade: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed\nWASM backtrace:\nerror while executing at wasm backtrace:\n    0: 0x2f54f - <unknown>!rust_begin_unwind\n    1: 0x49f5 - <unknown>!core::panicking::panic_fmt::hf5c4cd929d4aaa9e\n    2: 0x4ece - <unknown>!core::result::unwrap_failed::he0ddc269083098af\n    3: 0xedf74 - <unknown>!TryRuntime_on_runtime_upgrade")

This is due to https://forum.polkadot.network/t/polkadot-release-analysis-v0-9-43/3158#improve-handling-of-unset-storageversion-16

As suggested I first bumped the storage version just to make try-runtime work (see commit 15f42e1)

Edited:

Then I applied full respective migrations when possible:

• Balances v0 -> v1: just bumped storage version + remove old StorageVersion type since migration seems not be required in our case (see comment here https://github.com/paritytech/polkadot-sdk/blob/f240e02557d47756542b9b8cd876800b4b64c6d5/substrate/frame/balances/src/migration.rs#L65)
• Historical (session) v0 -> v1: pallet_session::migrations::v1::migrate::<T, crate::Historical>() https://github.com/paritytech/polkadot-sdk/blob/f240e02557d47756542b9b8cd876800b4b64c6d5/substrate/frame/session/src/migrations/v1.rs#L43
• Scheduler v3 -> v4: pub struct MigrateToV4<T>(sp_std::marker::PhantomData<T>); https://github.com/paritytech/polkadot-sdk/blob/f240e02557d47756542b9b8cd876800b4b64c6d5/substrate/frame/scheduler/src/migration.rs#L84

[sameh-farouk]

Almost there, just small note .
session_historical migrations are not idempotent.

WARN  runtime::session_historical] Attempted to apply migration to v1 but failed because storage version is StorageVersion(1)
[2023-12-06T12:35:25Z ERROR try-runtime] Detected multiple errors while executing `try_on_runtime_upgrade`:
[2023-12-06T12:35:25Z ERROR try-runtime] Other("Unexpected pallet version")
[2023-12-06T12:35:25Z ERROR try-runtime] Other("Can only upgrade from version 3")
[2023-12-06T12:35:25Z ERROR runtime] panicked at 'called `Result::unwrap()` on an `Err` value: Other("Detected multiple errors while executing `try_on_runtime_upgrade`, check the logs!")', /home/sameh/projects/tfchain/substrate-node/runtime/src/lib.rs:1025:65
[2023-12-06T12:35:25Z ERROR try_runtime_core::commands::on_runtime_upgrade] ❌ Migrations are not idempotent, they failed during the second execution.

[sameh-farouk]

LGTM!