referrerpolicy=no-referrer-when-downgrade
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
//! # WASM Meta Protocol
//!
//! All Substrate based chains adhere to a unique architectural design novel to the Polkadot
//! ecosystem. We refer to this design as the "**WASM Meta Protocol**".
//!
//! Consider the fact that a traditional blockchain software is usually a monolithic artifact.
//! **Upgrading any part of the system implies upgrading the entire system**. This has historically
//! led to cumbersome forkful upgrades to be the status quo in blockchain ecosystems. In other
//! words, the entire node software is the specification of the blockchain's [`state transition
//! function`](crate::reference_docs::blockchain_state_machines).
//!
//! Moreover, the idea of "storing code in the state" is explored in the context of smart contracts
//! platforms, but has not been expanded further.
//!
//! Substrate mixes these two ideas together, and takes the novel approach of storing the
//! blockchain's main "state transition function" in the main blockchain state, in the same fashion
//! that a smart contract platform stores the code of individual contracts in its state. As noted in
//! [`crate::reference_docs::blockchain_state_machines`], this state transition function is called
//! the **Runtime**, and WASM is chosen as the bytecode. The Runtime is stored under a special key
//! in the state (see [`sp_core::storage::well_known_keys`]) and can be updated as a part of the
//! state transition function's execution, just like a user's account balance can be updated.
//!
//! > Note that while we drew an analogy between smart contracts and runtimes in the above, there
//! > are fundamental differences between the two, explained in
//! > [`crate::reference_docs::runtime_vs_smart_contract`].
//!
//! The rest of the system that is NOT the state transition function is called the
//! [**Node**](crate::reference_docs::glossary#node), and is a normal binary that is compiled from
//! Rust to different hardware targets.
//!
//! This design enables all Substrate-based chains to be fork-less-ly upgradeable, because the
//! Runtime can be updated on the fly, within the execution of a block, and the node is (for the
//! most part) oblivious to the change that is happening.
//!
//! Therefore, the high-level architecture of a any Substrate-based chain can be demonstrated as
//! follows:
#![doc = simple_mermaid::mermaid!("../../../mermaid/substrate_simple.mmd")]
//!
//! The node and the runtime need to communicate. This is done through two concepts:
//!
//! 1. **Host functions**: a way for the (WASM) runtime to talk to the node. All host functions are
//!    defined in [`sp_io`]. For example, [`sp_io::storage`] are the set of host functions that
//!    allow the runtime to read and write data to the on-chain state.
//! 2. **Runtime APIs**: a way for the node to talk to the WASM runtime. Runtime APIs are defined
//!    using macros and utilities in [`sp_api`]. For example, [`sp_api::Core`] is the most
//!    fundamental runtime API that any blockchain must implement in order to be able to (re)
//!    execute blocks.
#![doc = simple_mermaid::mermaid!("../../../mermaid/substrate_client_runtime.mmd")]
//!
//! A runtime must have a set of runtime APIs in order to have any meaningful blockchain
//! functionality, but it can also expose more APIs. See
//! [`crate::reference_docs::custom_runtime_api_rpc`] as an example of how to add custom runtime
//! APIs to your FRAME-based runtime.
//!
//! Similarly, for a runtime to be "compatible" with a node, the node must implement the full set of
//! host functions that the runtime at any point in time requires. Given the fact that a runtime can
//! evolve in time, and a blockchain node (typically) wishes to be capable of re-executing all the
//! previous blocks, this means that a node must always maintain support for the old host functions.
//! **This implies that adding a new host function is a big commitment and should be done with
//! care**. This is why, for example, adding a new host function to Polkadot always requires an RFC.
//! Learn how to add a new host function to your runtime in
//! [`crate::reference_docs::custom_host_functions`].
//!
//! ## Node vs. Runtime
//!
//! A common question is: which components of the system end up being part of the node, and which
//! ones of the runtime?
//!
//! Recall from [`crate::reference_docs::blockchain_state_machines`] that the runtime is the state
//! transition function. Anything that needs to influence how your blockchain's state is updated,
//! should be a part of the runtime. For example, the logic around currency, governance, identity or
//! any other application-specific logic that has to do with the state is part of the runtime.
//!
//! Anything that does not have to do with the state-transition function and will only
//! facilitate/enable it is part of the node. For example, the database, networking, and even
//! consensus algorithm are all node-side components.
//!
//! > The consensus is to your runtime what HTTP is to a web-application. It is the underlying
//! > engine that enables trustless execution of the runtime in a distributed manner whilst
//! > maintaining a canonical outcome of that execution.
#![doc = simple_mermaid::mermaid!("../../../mermaid/substrate_with_frame.mmd")]
//!
//! ## State
//!
//! From the previous sections, we know that the database component is part of the node, not the
//! runtime. We also hinted that a set of host functions ([`sp_io::storage`]) are how the runtime
//! issues commands to the node to read/write to the state. Let's dive deeper into this.
//!
//! The state of the blockchain, what we seek to come to consensus about, is indeed *kept* in the
//! node side. Nonetheless, the runtime is the only component that:
//!
//! 1. Can update the state.
//! 2. Can fully interpret the state.
//!
//! In fact, [`sp_core::storage::well_known_keys`] are the only state keys that the node side is
//! aware of. The rest of the state, including what logic the runtime has, what balance each user
//! has and such, are all only comprehensible to the runtime.
#![doc = simple_mermaid::mermaid!("../../../mermaid/state.mmd")]
//!
//! In the above diagram, all of the state keys and values are opaque bytes to the node. The node
//! does not know what they mean, and it does not know what is the type of the corresponding value
//! (e.g. if it is a number of a vector). Contrary, the runtime knows both the meaning of their
//! keys, and the type of the values.
//!
//! This opaque-ness is the fundamental reason why Substrate-based chains can fork-less-ly upgrade:
//! because the node side code is kept oblivious to all of the details of the state transition
//! function. Therefore, the state transition function can freely upgrade without the node needing
//! to know.
//!
//! ## Native Runtime
//!
//! Historically, the node software also kept a native copy of the runtime at the time of
//! compilation within it. This used to be called the "Native Runtime". The main purpose of the
//! native runtime used to be leveraging the faster execution time and better debugging
//! infrastructure of native code. However, neither of the two arguments strongly hold and the
//! native runtime is being fully removed from the node-sdk.
//!
//! See: <https://github.com/paritytech/polkadot-sdk/issues/62>
//!
//! > Also, note that the flags [`sc_cli::ExecutionStrategy::Native`] is already a noop and all
//! > chains built with Substrate only use WASM execution.
//!
//! ### Runtime Versions
//!
//! An important detail of the native execution worth learning about is that the node software,
//! obviously, only uses the native runtime if it is the same code as with the wasm blob stored
//! onchain. Else, nodes who run the native runtime will come to a different state transition. How
//! do nodes determine if two runtimes are the same? Through the very important
//! [`sp_version::RuntimeVersion`]. All runtimes expose their version via a runtime api
//! ([`sp_api::Core::version`]) that returns this struct. The node software, or other applications,
//! inspect this struct to examine the identity of a runtime, and to determine if two runtimes are
//! the same. Namely, [`sp_version::RuntimeVersion::spec_version`] is the main key that implies two
//! runtimes are the same.
//!
//! Therefore, it is utmost important to make sure before any runtime upgrade, the spec version is
//! updated.
//!
//! ## Example: Block Execution.
//!
//! As a final example to recap, let's look at how Substrate-based nodes execute blocks. Blocks are
//! received in the node side software as opaque blobs and in the networking layer.
//!
//! At some point, based on the consensus algorithm's rules, the node decides to import (aka.
//! *validate*) a block.
//!
//! * First, the node will fetch the state of the parent hash of the block that wishes to be
//! imported.
//! * The runtime is fetched from this state, and placed into a WASM execution environment.
//! * The [`sp_api::Core::execute_block`] runtime API is called and the block is passed in as an
//! argument.
//! * The runtime will then execute the block, and update the state accordingly. Any state update is
//!   issued via the [`sp_io::storage`] host functions.
//! * Both the runtime and node will check the state-root of the state after the block execution to
//!   match the one claimed in the block header.
//!
//! > Example taken from [this
//! > lecture](https://polkadot-blockchain-academy.github.io/pba-book/substrate/wasm/page.html#example-2-block-import-9)
//! > of the Polkadot Blockchain Academy.