Crate substrate

source ·
Expand description

Substrate

Substrate is a Rust framework for building blockchains in a modular and extensible way. While in itself un-opinionated, it is the main engine behind the Polkadot ecosystem.

github - polkadot

This crate in itself does not contain any code and is just meant ot be a documentation hub for substrate-based crates.

Overview

Substrate approaches blockchain development with an acknowledgement of a few self-evident truths:

  1. Society and technology evolves.
  2. Humans are fallible.

This, specifically, makes the task of designing a correct, safe and long-lasting blockchain system hard.

Nonetheless, in order to achieve this goal, substrate embraces the following:

  1. Use of Rust as a modern, and safe programming language, which limits human error through various means, most notably memory safety.
  2. Substrate is written from the ground-up with a generic, modular and extensible design. This ensures that software components can be easily swapped and upgraded. Examples of this is multiple consensus mechanisms provided by Substrate, as listed below.
  3. Lastly, the final blockchain system created with the above properties needs to be upgradeable. In order to achieve this, Substrate is designed as a meta-protocol, whereby the application logic of the blockchain (called “Runtime”) is encoded as a Wasm blob, and is stored onchain. The rest of the system (called “Client”) acts as the executor of the Wasm blob.

In essence, the meta-protocol of all Substrate based chains is the “Runtime as Wasm blob” accord. This enables the Runtime to become inherently upgradeable (without forks). The upgrade is merely a matter of the Wasm blob being changed in the chain state, which is, in principle, same as updating an account’s balance.

To learn more about the substrate architecture using some visuals, see substrate_diagram.

FRAME, Substrate’s default runtime development library takes the above even further by embracing a declarative programming model whereby correctness is enhanced and the system is highly configurable through parameterization.

All in all, this design enables all substrate-based chains to achieve forkless, self-enacting upgrades out of the box. Combined with governance abilities that are shipped with FRAME, this enables a chain to survive the test of time.

How to Get Stared

Most developers want to leave the client side code as-is, and focus on the runtime. To do so, look into the frame_support crate, which is the entry point crate into runtime development with FRAME.

Side note, it is entirely possible to craft a substrate-based runtime without FRAME, an example of which can be found here.

In more broad terms, the following avenues exist into developing with substrate:

  • Templates: A number of substrate-based templates exist and they can be used for various purposes, with zero to little additional code needed. All of these templates contain runtimes that are highly configurable and are likely suitable for basic needs.
  • FRAME: If need, one can customize that runtime even further, by using FRAME and developing custom modules.
  • Core: To the contrary, some developers may want to customize the client side software to achieve novel goals such as a new consensus engine, or a new database backend. While Substrate’s main configurability is in the runtime, the client is also highly generic and can be customized to a great extent.

Structure

Substrate is a massive cargo workspace with hundreds of crates, therefore it is useful to know how to navigate its crates.

In broad terms, it is divided into three categories:

  • sc-* (short for substrate-client) crates, located under ./client folder. These are all the client crates. Notable examples are crates such as sc-network, various consensus crates, sc-rpc-api and sc-client-db, all of which are expected to reside in the client side.
  • sp-* (short for substrate-primitives) crates, located under ./primitives folder. These are the traits that glue the client and runtime together, but are not opinionated about what framework is using for building the runtime. Notable examples are sp-api and sp-io, which form the communication bridge between the client and runtime, as explained in substrate_diagram.
  • pallet-* and frame-* crates, located under ./frame folder. These are the crates related to FRAME. See frame_support for more information.

Wasm Build

Many of the Substrate crates, such as entire sp-*, need to compile to both Wasm (when a Wasm runtime is being generated) and native (for example, when testing). To achieve this, Substrate follows the convention of the Rust community, and uses a feature = "std" to signify that a crate is being built with the standard library, and is built for native. Otherwise, it is built for no_std.

This can be summarized in #![cfg_attr(not(feature = "std"), no_std)], which you can often find in any Substrate-based runtime.

Substrate-based runtimes use substrate-wasm-builder in their build.rs to automatically build their Wasm files as a part of normal build commandsOnce built, the wasm file is placed in ./target/{debug|release}/wbuild/{runtime_name}.wasm.

Binaries

Multiple binaries are shipped with substrate, the most important of which are located in the ./bin folder.

  • node is an extensive substrate node that contains the superset of all runtime and client side features. The corresponding runtime, called kitchensink_runtime contains all of the modules that are provided with FRAME. This node and runtime is only used for testing and demonstration.
    • chain-spec-builder: Utility to build more detailed chain-specs for the aforementioned node. Other projects typically contain a build-spec subcommand that does the same.
  • node-template: a template node that contains a minimal set of features and can act as a starting point of a project.
  • subkey: Substrate’s key management utility.

Anatomy of a Binary Crate

From the above, node and node-template are essentially blueprints of a substrate-based project, as the name of the latter is implying. Each substrate-based project typically contains the following:

  • Under ./runtime, a ./runtime/src/lib.rs which is the top level runtime amalgamator file. This file typically contains the frame_support::construct_runtime macro, which is the final definition of a runtime.

  • Under ./node, a main.rs, which is the point, and a ./service.rs, which contains all the client side components. Skimming this file yields an overview of the networking, database, consensus and similar client side components.

The above two are conventions, not rules.

Parachain?

As noted above, Substrate is the main engine behind the Polkadot ecosystem. One of the ways through which Polkadot can be utilized is by building “parachains”, blockchains that are connected to Polkadot’s shared security.

To build a parachain, one could use Cumulus, the library on top of Substrate, empowering any substrate-based chain to be a Polkadot parachain.

Where To Go Next?

Additional noteworthy crates within substrate:

Additional noteworthy external resources:

Notable upstream crates:

Templates:

Modules

  • substrate_diagram
    In this module, we explore substrate at a more depth. First, let’s establish substrate being divided into a client and runtime.