referrerpolicy=no-referrer-when-downgrade
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
//! # (Omni) Node
//!
//! This reference doc elaborates on what a Polkadot-SDK/Substrate node software is, and what
//! various ways exist to run one.
//!
//! The node software, as denoted in [`crate::reference_docs::wasm_meta_protocol`], is everything in
//! a blockchain other than the WASM runtime. It contains common components such as the database,
//! networking, RPC server and consensus. Substrate-based nodes are native binaries that are
//! compiled down from the Rust source code. The `node` folder in any of the [`templates`] are
//! examples of this source.
//!
//! > Note: A typical node also contains a lot of other tools (exposed as subcommands) that are
//! > useful for operating a blockchain, such as the ones noted in
//! > [`polkadot_omni_node_lib::cli::Cli::subcommand`].
//!
//! ## Node <> Runtime Interdependence
//!
//! While in principle the node can be mostly independent of the runtime, for various reasons, such
//! as the [native runtime](crate::reference_docs::wasm_meta_protocol#native-runtime), the node and
//! runtime were historically tightly linked together. Another reason is that the node and the
//! runtime need to be in agreement about which consensus algorithm they use, as described
//! [below](#consensus-engine).
//!
//! Specifically, the node relied on the existence of a linked runtime, and *could only reliably run
//! that runtime*. This is why if you look at any of the [`templates`], they are all composed of a
//! node, and a runtime.
//!
//! Moreover, the code and API of each of these nodes was historically very advanced, and tailored
//! towards those who wish to customize many of the node components at depth.
//!
//! > The notorious `service.rs` in any node template is a good example of this.
//!
//! A [trend](https://github.com/paritytech/polkadot-sdk/issues/62) has already been undergoing in
//! order to de-couple the node and the runtime for a long time. The north star of this effort is
//! twofold :
//!
//! 1. develop what can be described as an "omni-node": A node that can run most runtimes.
//! 2. provide a cleaner abstraction for creating a custom node.
//!
//! While a single omni-node running *all possible runtimes* is not feasible, the
//! [`polkadot-omni-node`] is an attempt at creating the former, and the [`polkadot_omni_node_lib`]
//! is the latter.
//!
//! > Note: The OmniNodes are mainly focused on the development needs of **Polkadot
//! > parachains ONLY**, not (Substrate) solo-chains. For the time being, solo-chains are not
//! > supported by the OmniNodes. This might change in the future.
//!
//! ## Types of Nodes
//!
//! With the emergence of the OmniNodes, let's look at the various Node options available to a
//! builder.
//!
//! ### [`polkadot-omni-node`]
//!
//! [`polkadot-omni-node`] is a white-labeled binary, released as a part of Polkadot SDK that is
//! capable of meeting the needs of most Polkadot parachains.
//!
//! It can act as the collator of a parachain in production, with all the related auxillary
//! functionalities that a normal collator node has: RPC server, archiving state, etc. Moreover, it
//! can also run the wasm blob of the parachain locally for testing and development.
//!
//! ### [`polkadot_omni_node_lib`]
//!
//! [`polkadot_omni_node_lib`] is the library version of the above, which can be used to create a
//! fresh parachain node, with a some limited configuration options using a lean API.
//!
//! ### Old School Nodes
//!
//! The existing node architecture, as seen in the [`templates`], is still available for those who
//! want to have full control over the node software.
//!
//! ### Summary
//!
//! We can summarize the choices for the node software of any given user of Polkadot-SDK, wishing to
//! deploy a parachain into 3 categories:
//!
//! 1. **Use the [`polkadot-omni-node`]**: This is the easiest way to get started, and is the most
//!   likely to be the best choice for most users.
//!     * can run almost any runtime with [`--dev-block-time`]
//! 2. **Use the [`polkadot_omni_node_lib`]**: This is the best choice for those who want to have
//!    slightly more control over the node software, such as embedding a custom chain-spec.
//! 3. **Use the old school nodes**: This is the best choice for those who want to have full control
//!    over the node software, such as changing the consensus engine, altering the transaction pool,
//!    and so on.
//!
//! ## _OmniTools_: User Journey
//!
//! All in all, the user journey of a team/builder, in the OmniNode world is as follows:
//!
//! * The [`templates`], most notably the [`parachain-template`] is the canonical starting point.
//!   That being said, the node code of the templates (which may be eventually
//!   removed/feature-gated) is no longer of relevance. The only focus is in the runtime, and
//!   obtaining a `.wasm` file. References:
//!     * [`crate::guides::your_first_pallet`]
//!     * [`crate::guides::your_first_runtime`]
//! * If need be, the weights of the runtime need to be updated using `frame-omni-bencher`.
//!   References:
//!     * [`crate::reference_docs::frame_benchmarking_weight`]
//! * Next, [`chain-spec-builder`] is used to generate a `chain_spec.json`, either for development,
//!   or for production. References:
//!     * [`crate::reference_docs::chain_spec_genesis`]
//! * For local development, the following options are available:
//!     * `polkadot-omni-node` (notably, with [`--dev-block-time`]). References:
//!         * [`crate::guides::your_first_node`]
//!     * External tools such as `chopsticks`, `zombienet`.
//!         * See the `README.md` file of the `polkadot-sdk-parachain-template`.
//! * For production `polkadot-omni-node` can be used out of the box.
//! * For further customization [`polkadot_omni_node_lib`] can be used.
//!
//! ## Appendix
//!
//! This section describes how the interdependence between the node and the runtime is related to
//! the consensus engine. This information is useful for those who want to understand the
//! historical context of the node and the runtime.
//!
//! ### Consensus Engine
//!
//! In any given substrate-based chain, both the node and the runtime will have their own
//! opinion/information about what consensus engine is going to be used.
//!
//! In practice, the majority of the implementation of any consensus engine is in the node side, but
//! the runtime also typically needs to expose a custom runtime-api to enable the particular
//! consensus engine to work, and that particular runtime-api is implemented by a pallet
//! corresponding to that consensus engine.
//!
//! For example, taking a snippet from [`solochain_template_runtime`], the runtime has to provide
//! this additional runtime-api (compared to [`minimal_template_runtime`]), if the node software is
//! configured to use the Aura consensus engine:
//!
//! ```text
//! impl sp_consensus_aura::AuraApi<Block, AuraId> for Runtime {
//!     fn slot_duration() -> sp_consensus_aura::SlotDuration {
//!         ...
//!     }
//!     fn authorities() -> Vec<AuraId> {
//!         ...
//!     }
//! }
//! ```
//!
//! For simplicity, we can break down "consensus" into two main parts:
//!
//! * Block Authoring: Deciding who gets to produce the next block.
//! * Finality: Deciding when a block is considered final.
//!
//! For block authoring, there are a number of options:
//!
//! * [`sc_consensus_manual_seal`]: Useful for testing, where any node can produce a block at any
//!   time. This is often combined with a fixed interval at which a block is produced.
//! * [`sc_consensus_aura`]/[`pallet_aura`]: A simple round-robin block authoring mechanism.
//! * [`sc_consensus_babe`]/[`pallet_babe`]: A more advanced block authoring mechanism, capable of
//!   anonymizing the next block author.
//! * [`sc_consensus_pow`]: Proof of Work block authoring.
//!
//! For finality, there is one main option shipped with polkadot-sdk:
//!
//! * [`sc_consensus_grandpa`]/[`pallet_grandpa`]: A finality gadget that uses a voting mechanism to
//!   decide when a block
//!
//! **The most important lesson here is that the node and the runtime must have matching consensus
//! components.**
//!
//! ### Consequences for OmniNode
//!
//!
//! The consequence of the above is that anyone using the OmniNode must also be aware of the
//! consensus system used in the runtime, and be aware if it is matching that of the OmniNode or
//! not. For the time being, [`polkadot-omni-node`] only supports:
//!
//! * Parachain-based Aura consensus, with 6s async-backing block-time, and before full elastic
//!   scaling). [`polkadot_omni_node_lib::cli::Cli::experimental_use_slot_based`] for fixed factor
//!   scaling (a step
//! * Ability to run any runtime with [`--dev-block-time`] flag. This uses
//!   [`sc_consensus_manual_seal`] under the hood, and has no restrictions on the runtime's
//!   consensus.
//!
//! [This](https://github.com/paritytech/polkadot-sdk/issues/5565) future improvement to OmniNode
//! aims to make such checks automatic.
//!
//!
//! [`templates`]: crate::polkadot_sdk::templates
//! [`parachain-template`]: https://github.com/paritytech/polkadot-sdk-parachain-template
//! [`--dev-block-time`]: polkadot_omni_node_lib::cli::Cli::dev_block_time
//! [`polkadot-omni-node`]: https://crates.io/crates/polkadot-omni-node
//! [`chain-spec-builder`]: https://crates.io/crates/staging-chain-spec-builder