polkadot_sdk_docs/reference_docs/defensive_programming.rs
1// Copyright (C) Parity Technologies (UK) Ltd.
2// SPDX-License-Identifier: Apache-2.0
3
4// Licensed under the Apache License, Version 2.0 (the "License");
5// you may not use this file except in compliance with the License.
6// You may obtain a copy of the License at
7//
8// http://www.apache.org/licenses/LICENSE-2.0
9//
10// Unless required by applicable law or agreed to in writing, software
11// distributed under the License is distributed on an "AS IS" BASIS,
12// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13// See the License for the specific language governing permissions and
14// limitations under the License.
15
16//! [Defensive programming](https://en.wikipedia.org/wiki/Defensive_programming) is a design paradigm that enables a program to continue
17//! running despite unexpected behavior, input, or events that may arise in runtime.
18//! Usually, unforeseen circumstances may cause the program to stop or, in the Rust context,
19//! `panic!`. Defensive practices allow for these circumstances to be accounted for ahead of time
20//! and for them to be handled gracefully, which is in line with the intended fault-tolerant and
21//! deterministic nature of blockchains.
22//!
23//! The Polkadot SDK is built to reflect these principles and to facilitate their usage accordingly.
24//!
25//! ## General Overview
26//!
27//! When developing within the context of the Substrate runtime, there is one golden rule:
28//!
29//! ***DO NOT PANIC***. There are some exceptions, but generally, this is the default precedent.
30//!
31//! > It’s important to differentiate between the runtime and node. The runtime refers to the core
32//! > business logic of a Substrate-based chain, whereas the node refers to the outer client, which
33//! > deals with telemetry and gossip from other nodes. For more information, read about
34//! > [Substrate's node
35//! > architecture](crate::reference_docs::wasm_meta_protocol#node-vs-runtime). It’s also important
36//! > to note that the criticality of the node is slightly lesser
37//! > than that of the runtime, which is why you may see `unwrap()` or other “non-defensive”
38//! > approaches
39//! in a few places of the node's code repository.
40//!
41//! Most of these practices fall within Rust's
42//! colloquial usage of proper error propagation, handling, and arithmetic-based edge cases.
43//!
44//! General guidelines:
45//!
46//! - **Avoid writing functions that could explicitly panic,** such as directly using `unwrap()` on
47//! a [`Result`], or accessing an out-of-bounds index on a collection. Safer methods to access
48//! collection types, i.e., `get()` which allow defensive handling of the resulting [`Option`] are
49//! recommended to be used.
50//! - **It may be acceptable to use `except()`,** but only if one is completely certain (and has
51//! performed a check beforehand) that a value won't panic upon unwrapping. *Even this is
52//! discouraged*, however, as future changes to that function could then cause that statement to
53//! panic. It is important to ensure all possible errors are propagated and handled effectively.
54//! - **If a function *can* panic,** it usually is prefaced with `unchecked_` to indicate its
55//! unsafety.
56//! - **If you are writing a function that could panic,** [document it!](https://doc.rust-lang.org/rustdoc/how-to-write-documentation.html#documenting-components)
57//! - **Carefully handle mathematical operations.** Many seemingly, simplistic operations, such as
58//! **arithmetic** in the runtime, could present a number of issues [(see more later in this
59//! document)](#integer-overflow). Use checked arithmetic wherever possible.
60//!
61//! These guidelines could be summarized in the following example, where `bad_pop` is prone to
62//! panicking, and `good_pop` allows for proper error handling to take place:
63//!
64//!```ignore
65//! // Bad pop always requires that we return something, even if vector/array is empty.
66//! fn bad_pop<T>(v: Vec<T>) -> T {}
67//! // Good pop allows us to return None from the Option if need be.
68//! fn good_pop<T>(v: Vec<T>) -> Option<T> {}
69//! ```
70//!
71//! ### Defensive Traits
72//!
73//! The [`Defensive`](frame::traits::Defensive) trait provides a number of functions, all of which
74//! provide an alternative to 'vanilla' Rust functions, e.g.:
75//!
76//! - [`defensive_unwrap_or()`](frame::traits::Defensive::defensive_unwrap_or) instead of
77//! `unwrap_or()`
78//! - [`defensive_ok_or()`](frame::traits::DefensiveOption::defensive_ok_or) instead of `ok_or()`
79//!
80//! Defensive methods use [`debug_assertions`](https://doc.rust-lang.org/reference/conditional-compilation.html#debug_assertions), which panic in development, but in
81//! production/release, they will merely log an error (i.e., `log::error`).
82//!
83//! The [`Defensive`](frame::traits::Defensive) trait and its various implementations can be found
84//! [here](frame::traits::Defensive).
85//!
86//! ## Integer Overflow
87//!
88//! The Rust compiler prevents static overflow from happening at compile time.
89//! The compiler panics in **debug** mode in the event of an integer overflow. In
90//! **release** mode, it resorts to silently _wrapping_ the overflowed amount in a modular fashion
91//! (from the `MAX` back to zero).
92//!
93//! In runtime development, we don't always have control over what is being supplied
94//! as a parameter. For example, even this simple add function could present one of two outcomes
95//! depending on whether it is in **release** or **debug** mode:
96//!
97//! ```ignore
98//! fn naive_add(x: u8, y: u8) -> u8 {
99//! x + y
100//! }
101//! ```
102//! If we passed overflow-able values at runtime, this could panic (or wrap if in release).
103//!
104//! ```ignore
105//! naive_add(250u8, 10u8); // In debug mode, this would panic. In release, this would return 4.
106//! ```
107//!
108//! It is the silent portion of this behavior that presents a real issue. Such behavior should be
109//! made obvious, especially in blockchain development, where unsafe arithmetic could produce
110//! unexpected consequences like a user balance over or underflowing.
111//!
112//! Fortunately, there are ways to both represent and handle these scenarios depending on our
113//! specific use case natively built into Rust and libraries like [`sp_arithmetic`].
114//!
115//! ## Infallible Arithmetic
116//!
117//! Both Rust and Substrate provide safe ways to deal with numbers and alternatives to floating
118//! point arithmetic.
119//!
120//! Known scenarios that could be fallible should be avoided: i.e., avoiding the possibility of
121//! dividing/modulo by zero at any point should be mitigated. One should be opting for a
122//! `checked_*` method to introduce safe arithmetic in their code in most cases.
123//!
124//! A developer should use fixed-point instead of floating-point arithmetic to mitigate the
125//! potential for inaccuracy, rounding errors, or other unexpected behavior.
126//!
127//! - [Fixed point types](sp_arithmetic::fixed_point) and their associated usage can be found here.
128//! - [PerThing](sp_arithmetic::per_things) and its associated types can be found here.
129//!
130//! Using floating point number types (i.e. f32, f64) in the runtime should be avoided, as a single non-deterministic result could cause chaos for blockchain consensus along with the issues above. For more on the specifics of the peculiarities of floating point calculations, [watch this video by the Computerphile](https://www.youtube.com/watch?v=PZRI1IfStY0).
131//!
132//! The following methods demonstrate different ways to handle numbers natively in Rust safely,
133//! without fear of panic or unexpected behavior from wrapping.
134//!
135//! ### Checked Arithmetic
136//!
137//! **Checked operations** utilize an `Option<T>` as a return type. This allows for
138//! catching any unexpected behavior in the event of an overflow through simple pattern matching.
139//!
140//! This is an example of a valid operation:
141#![doc = docify::embed!("./src/reference_docs/defensive_programming.rs", checked_add_example)]
142//!
143//! This is an example of an invalid operation. In this case, a simulated integer overflow, which
144//! would simply result in `None`:
145#![doc = docify::embed!(
146 "./src/reference_docs/defensive_programming.rs",
147 checked_add_handle_error_example
148)]
149//!
150//! Suppose you aren’t sure which operation to use for runtime math. In that case, checked
151//! operations are the safest bet, presenting two predictable (and erroring) outcomes that can be
152//! handled accordingly (Some and None).
153//!
154//! The following conventions can be seen within the Polkadot SDK, where it is
155//! handled in two ways:
156//!
157//! - As an [`Option`], using the `if let` / `if` or `match`
158//! - As a [`Result`], via `ok_or` (or similar conversion to [`Result`] from [`Option`])
159//!
160//! #### Handling via Option - More Verbose
161//!
162//! Because wrapped operations return `Option<T>`, you can use a more verbose/explicit form of error
163//! handling via `if` or `if let`:
164#![doc = docify::embed!("./src/reference_docs/defensive_programming.rs", increase_balance)]
165//!
166//! Optionally, match may also be directly used in a more concise manner:
167#![doc = docify::embed!("./src/reference_docs/defensive_programming.rs", increase_balance_match)]
168//!
169//! This is generally a useful convention for handling checked types and most types that return
170//! `Option<T>`.
171//!
172//! #### Handling via Result - Less Verbose
173//!
174//! In the Polkadot SDK codebase, checked operations are handled as a `Result` via `ok_or`. This is
175//! a less verbose way of expressing the above. This usage often boils down to the developer’s
176//! preference:
177#![doc = docify::embed!("./src/reference_docs/defensive_programming.rs", increase_balance_result)]
178//!
179//! ### Saturating Operations
180//!
181//! Saturating a number limits it to the type’s upper or lower bound, even if the integer type
182//! overflowed in runtime. For example, adding to `u32::MAX` would simply limit itself to
183//! `u32::MAX`:
184#![doc = docify::embed!("./src/reference_docs/defensive_programming.rs", saturated_add_example)]
185//!
186//! Saturating calculations can be used if one is very sure that something won't overflow, but wants
187//! to avoid introducing the notion of any potential-panic or wrapping behavior.
188//!
189//! There is also a series of defensive alternatives via
190//! [`DefensiveSaturating`](frame::traits::DefensiveSaturating), which introduces the same behavior
191//! of the [`Defensive`](frame::traits::Defensive) trait, only with saturating, mathematical
192//! operations:
193#![doc = docify::embed!(
194 "./src/reference_docs/defensive_programming.rs",
195 saturated_defensive_example
196)]
197//!
198//! ### Mathematical Operations in Substrate Development - Further Context
199//!
200//! As a recap, we covered the following concepts:
201//!
202//! 1. **Checked** operations - using [`Option`] or [`Result`]
203//! 2. **Saturating** operations - limited to the lower and upper bounds of a number type
204//! 3. **Wrapped** operations (the default) - wrap around to above or below the bounds of a type
205//!
206//! #### The problem with 'default' wrapped operations
207//!
208//! **Wrapped operations** cause the overflow to wrap around to either the maximum or minimum of
209//! that type. Imagine this in the context of a blockchain, where there are account balances, voting
210//! counters, nonces for transactions, and other aspects of a blockchain.
211//!
212//! While it may seem trivial, choosing how to handle numbers is quite important. As a thought
213//! exercise, here are some scenarios of which will shed more light on when to use which.
214//!
215//! #### Bob's Overflowed Balance
216//!
217//! **Bob's** balance exceeds the `Balance` type on the `EduChain`. Because the pallet developer did
218//! not handle the calculation to add to Bob's balance with any regard to this overflow, **Bob's**
219//! balance is now essentially `0`, the operation **wrapped**.
220//!
221//! <details>
222//! <summary><b>Solution: Saturating or Checked</b></summary>
223//! For Bob's balance problems, using a `saturating_add` or `checked_add` could've mitigated
224//! this issue. They simply would've reached the upper, or lower bounds, of the particular type for
225//! an on-chain balance. In other words: Bob's balance would've stayed at the maximum of the
226//! Balance type. </details>
227//!
228//! #### Alice's 'Underflowed' Balance
229//!
230//! Alice’s balance has reached `0` after a transfer to Bob. Suddenly, she has been slashed on
231//! EduChain, causing her balance to reach near the limit of `u32::MAX` - a very large amount - as
232//! wrapped operations can go both ways. Alice can now successfully vote using her new, overpowered
233//! token balance, destroying the chain's integrity.
234//!
235//! <details>
236//! <summary><b>Solution: Saturating</b></summary>
237//! For Alice's balance problem, using `saturated_sub` could've mitigated this issue. A saturating
238//! calculation would've simply limited her balance to the lower bound of u32, as having a negative
239//! balance is not a concept within blockchains. In other words: Alice's balance would've stayed
240//! at "0", even after being slashed.
241//!
242//! This is also an example that while one system may work in isolation, shared interfaces, such
243//! as the notion of balances, are often shared across multiple pallets - meaning these small
244//! changes can make a big difference depending on the scenario. </details>
245//!
246//! #### Proposal ID Overwrite
247//!
248//! A `u8` parameter, called `proposals_count`, represents the type for counting the number of
249//! proposals on-chain. Every time a new proposal is added to the system, this number increases.
250//! With the proposal pallet's high usage, it has reached `u8::MAX`’s limit of 255, causing
251//! `proposals_count` to go to 0. Unfortunately, this results in new proposals overwriting old ones,
252//! effectively erasing any notion of past proposals!
253//!
254//! <details>
255//! <summary><b>Solution: Checked</b></summary>
256//! For the proposal IDs, proper handling via `checked` math would've been suitable,
257//! Saturating could've been used - but it also would've 'failed' silently. Using `checked_add` to
258//! ensure that the next proposal ID would've been valid would've been a viable way to let the user
259//! know the state of their proposal:
260//!
261//! ```ignore
262//! let next_proposal_id = current_count.checked_add(1).ok_or_else(|| Error::TooManyProposals)?;
263//! ```
264//!
265//! </details>
266//!
267//! From the above, we can clearly see the problematic nature of seemingly simple operations in the
268//! runtime, and care should be given to ensure a defensive approach is taken.
269//!
270//! ### Edge cases of `panic!`-able instances in Substrate
271//!
272//! As you traverse through the codebase (particularly in `substrate/frame`, where the majority of
273//! runtime code lives), you may notice that there (only a few!) occurrences where `panic!` is used
274//! explicitly. This is used when the runtime should stall, rather than keep running, as that is
275//! considered safer. Particularly when it comes to mission-critical components, such as block
276//! authoring, consensus, or other protocol-level dependencies, going through with an action may
277//! actually cause harm to the network, and thus stalling would be the better option.
278//!
279//! Take the example of the BABE pallet ([`pallet_babe`]), which doesn't allow for a validator to
280//! participate if it is disabled (see: [`frame::traits::DisabledValidators`]):
281//!
282//! ```ignore
283//! if T::DisabledValidators::is_disabled(authority_index) {
284//! panic!(
285//! "Validator with index {:?} is disabled and should not be attempting to author blocks.",
286//! authority_index,
287//! );
288//! }
289//! ```
290//!
291//! There are other examples in various pallets, mostly those crucial to the blockchain’s
292//! functionality. Most of the time, you will not be writing pallets which operate at this level,
293//! but these exceptions should be noted regardless.
294//!
295//! ## Other Resources
296//!
297//! - [PBA Lectures on YouTube](https://www.youtube.com/playlist?list=PL-w_i5kwVqbni1Ch2j_RwTIXiB-bwnYqq)
298#![allow(dead_code)]
299#[allow(unused_variables)]
300mod fake_runtime_types {
301 // Note: The following types are purely for the purpose of example, and do not contain any
302 // *real* use case other than demonstrating various concepts.
303 pub enum RuntimeError {
304 Overflow,
305 UserDoesntExist,
306 }
307
308 pub type Address = ();
309
310 pub struct Runtime;
311
312 impl Runtime {
313 fn get_balance(account: Address) -> Result<u64, RuntimeError> {
314 Ok(0u64)
315 }
316
317 fn set_balance(account: Address, new_balance: u64) {}
318 }
319
320 #[docify::export]
321 fn increase_balance(account: Address, amount: u64) -> Result<(), RuntimeError> {
322 // Get a user's current balance
323 let balance = Runtime::get_balance(account)?;
324 // SAFELY increase the balance by some amount
325 if let Some(new_balance) = balance.checked_add(amount) {
326 Runtime::set_balance(account, new_balance);
327 Ok(())
328 } else {
329 Err(RuntimeError::Overflow)
330 }
331 }
332
333 #[docify::export]
334 fn increase_balance_match(account: Address, amount: u64) -> Result<(), RuntimeError> {
335 // Get a user's current balance
336 let balance = Runtime::get_balance(account)?;
337 // SAFELY increase the balance by some amount
338 let new_balance = match balance.checked_add(amount) {
339 Some(balance) => balance,
340 None => {
341 return Err(RuntimeError::Overflow);
342 },
343 };
344 Runtime::set_balance(account, new_balance);
345 Ok(())
346 }
347
348 #[docify::export]
349 fn increase_balance_result(account: Address, amount: u64) -> Result<(), RuntimeError> {
350 // Get a user's current balance
351 let balance = Runtime::get_balance(account)?;
352 // SAFELY increase the balance by some amount - this time, by using `ok_or`
353 let new_balance = balance.checked_add(amount).ok_or(RuntimeError::Overflow)?;
354 Runtime::set_balance(account, new_balance);
355 Ok(())
356 }
357}
358
359#[cfg(test)]
360mod tests {
361 use frame::traits::DefensiveSaturating;
362 #[docify::export]
363 #[test]
364 fn checked_add_example() {
365 // This is valid, as 20 is perfectly within the bounds of u32.
366 let add = (10u32).checked_add(10);
367 assert_eq!(add, Some(20))
368 }
369
370 #[docify::export]
371 #[test]
372 fn checked_add_handle_error_example() {
373 // This is invalid - we are adding something to the max of u32::MAX, which would overflow.
374 // Luckily, checked_add just marks this as None!
375 let add = u32::MAX.checked_add(10);
376 assert_eq!(add, None)
377 }
378
379 #[docify::export]
380 #[test]
381 fn saturated_add_example() {
382 // Saturating add simply saturates
383 // to the numeric bound of that type if it overflows.
384 let add = u32::MAX.saturating_add(10);
385 assert_eq!(add, u32::MAX)
386 }
387
388 #[docify::export]
389 #[test]
390 #[cfg_attr(debug_assertions, should_panic(expected = "Defensive failure has been triggered!"))]
391 fn saturated_defensive_example() {
392 let saturated_defensive = u32::MAX.defensive_saturating_add(10);
393 assert_eq!(saturated_defensive, u32::MAX);
394 }
395}