pallet_migrations/lib.rs
1// This file is part of Substrate.
2
3// Copyright (C) Parity Technologies (UK) Ltd.
4// SPDX-License-Identifier: Apache-2.0
5
6// Licensed under the Apache License, Version 2.0 (the "License");
7// you may not use this file except in compliance with the License.
8// You may obtain a copy of the License at
9//
10// http://www.apache.org/licenses/LICENSE-2.0
11//
12// Unless required by applicable law or agreed to in writing, software
13// distributed under the License is distributed on an "AS IS" BASIS,
14// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15// See the License for the specific language governing permissions and
16// limitations under the License.
17
18#![deny(missing_docs)]
19#![deny(rustdoc::broken_intra_doc_links)]
20
21//! # `pallet-migrations`
22//!
23//! Provides multi block migrations for FRAME runtimes.
24//!
25//! ## Overview
26//!
27//! The pallet takes care of executing a batch of multi-step migrations over multiple blocks. The
28//! process starts on each runtime upgrade. Normal and operational transactions are paused while
29//! migrations are on-going.
30//!
31//! ### Example
32//!
33//! This example demonstrates a simple mocked walk through of a basic success scenario. The pallet
34//! is configured with two migrations: one succeeding after just one step, and the second one
35//! succeeding after two steps. A runtime upgrade is then enacted and the block number is advanced
36//! until all migrations finish executing. Afterwards, the recorded historic migrations are
37//! checked and events are asserted.
38#![doc = docify::embed!("src/tests.rs", simple_works)]
39//!
40//! ## Pallet API
41//!
42//! See the [`pallet`] module for more information about the interfaces this pallet exposes,
43//! including its configuration trait, dispatchables, storage items, events and errors.
44//!
45//! Otherwise noteworthy API of this pallet include its implementation of the
46//! [`MultiStepMigrator`] trait. This must be plugged into
47//! [`frame_system::Config::MultiBlockMigrator`] for proper function.
48//!
49//! The API contains some calls for emergency management. They are all prefixed with `force_` and
50//! should normally not be needed. Pay special attention prior to using them.
51//!
52//! ### Design Goals
53//!
54//! 1. Must automatically execute migrations over multiple blocks.
55//! 2. Must expose information about whether migrations are ongoing.
56//! 3. Must respect pessimistic weight bounds of migrations.
57//! 4. Must execute migrations in order. Skipping is not allowed; migrations are run on a
58//! all-or-nothing basis.
59//! 5. Must prevent re-execution of past migrations.
60//! 6. Must provide transactional storage semantics for migrations.
61//! 7. Must guarantee progress.
62//!
63//! ### Design
64//!
65//! Migrations are provided to the pallet through the associated type [`Config::Migrations`] of type
66//! [`SteppedMigrations`]. This allows multiple migrations to be aggregated through a tuple. It
67//! simplifies the trait bounds since all associated types of the trait must be provided by the
68//! pallet. The actual progress of the pallet is stored in the [`Cursor`] storage item. This can
69//! either be [`MigrationCursor::Active`] or [`MigrationCursor::Stuck`]. In the active case it
70//! points to the currently active migration and stores its inner cursor. The inner cursor can then
71//! be used by the migration to store its inner state and advance. Each time when the migration
72//! returns `Some(cursor)`, it signals the pallet that it is not done yet.
73//!
74//! The cursor is reset on each runtime upgrade. This ensures that it starts to execute at the
75//! first migration in the vector. The pallets cursor is only ever incremented or set to `Stuck`
76//! once it encounters an error (Goal 4). Once in the stuck state, the pallet will stay stuck until
77//! it is fixed through manual governance intervention.
78//!
79//! As soon as the cursor of the pallet becomes `Some(_)`; [`MultiStepMigrator::ongoing`] returns
80//! `true` (Goal 2). This can be used by upstream code to possibly pause transactions.
81//! In `on_initialize` the pallet will load the current migration and check whether it was already
82//! executed in the past by checking for membership of its ID in the [`Historic`] set. Historic
83//! migrations are skipped without causing an error. Each successfully executed migration is added
84//! to this set (Goal 5).
85//!
86//! This proceeds until no more migrations remain. At that point, the event `UpgradeCompleted` is
87//! emitted (Goal 1).
88//!
89//! The execution of each migration happens by calling [`SteppedMigration::transactional_step`].
90//! This function wraps the inner `step` function into a transactional layer to allow rollback in
91//! the error case (Goal 6).
92//!
93//! Weight limits must be checked by the migration itself. The pallet provides a [`WeightMeter`] for
94//! that purpose. The pallet may return [`SteppedMigrationError::InsufficientWeight`] at any point.
95//! In that scenario, one of two things will happen: if that migration was exclusively executed
96//! in this block, and therefore required more than the maximum amount of weight possible, the
97//! process becomes `Stuck`. Otherwise, one re-attempt is executed with the same logic in the next
98//! block (Goal 3). Progress through the migrations is guaranteed by providing a timeout for each
99//! migration via [`SteppedMigration::max_steps`]. The pallet **ONLY** guarantees progress if this
100//! is set to sensible limits (Goal 7).
101//!
102//! ### Scenario: Governance cleanup
103//!
104//! Every now and then, governance can make use of the [`clear_historic`][Pallet::clear_historic]
105//! call. This ensures that no old migrations pile up in the [`Historic`] set. This can be done very
106//! rarely, since the storage should not grow quickly and the lookup weight does not suffer much.
107//! Another possibility would be to have a synchronous single-block migration perpetually deployed
108//! that cleans them up before the MBMs start.
109//!
110//! ### Scenario: Successful upgrade
111//!
112//! The standard procedure for a successful runtime upgrade can look like this:
113//! 1. Migrations are configured in the `Migrations` config item. All migrations expose
114//! [`max_steps`][SteppedMigration::max_steps], are error tolerant, check their weight bounds and
115//! have a unique identifier.
116//! 2. The runtime upgrade is enacted. An `UpgradeStarted` event is
117//! followed by lots of `MigrationAdvanced` and `MigrationCompleted` events. Finally
118//! `UpgradeCompleted` is emitted.
119//! 3. Cleanup as described in the governance scenario be executed at any time after the migrations
120//! completed.
121//!
122//! ### Advice: Failed upgrades
123//!
124//! Failed upgrades cannot be recovered from automatically and require governance intervention. Set
125//! up monitoring for `UpgradeFailed` events to be made aware of any failures. The hook
126//! [`FailedMigrationHandler::failed`] should be setup in a way that it allows governance to act,
127//! but still prevent other transactions from interacting with the inconsistent storage state. Note
128//! that this is paramount, since the inconsistent state might contain a faulty balance amount or
129//! similar that could cause great harm if user transactions don't remain suspended. One way to
130//! implement this would be to use the `SafeMode` or `TxPause` pallets that can prevent most user
131//! interactions but still allow a whitelisted set of governance calls.
132//!
133//! ### Remark: Failed migrations
134//!
135//! Failed migrations are not added to the `Historic` set. This means that an erroneous
136//! migration must be removed and fixed manually. This already applies, even before considering the
137//! historic set.
138//!
139//! ### Remark: Transactional processing
140//!
141//! You can see the transactional semantics for migration steps as mostly useless, since in the
142//! stuck case the state is already messed up. This just prevents it from becoming even more messed
143//! up, but doesn't prevent it in the first place.
144
145#![cfg_attr(not(feature = "std"), no_std)]
146
147mod benchmarking;
148pub mod migrations;
149mod mock;
150pub mod mock_helpers;
151mod tests;
152pub mod weights;
153
154extern crate alloc;
155
156pub use pallet::*;
157pub use weights::WeightInfo;
158
159use alloc::vec::Vec;
160use codec::{Decode, DecodeWithMemTracking, Encode, MaxEncodedLen};
161use core::ops::ControlFlow;
162use frame_support::{
163 defensive, defensive_assert,
164 migrations::*,
165 pallet_prelude::*,
166 traits::Get,
167 weights::{Weight, WeightMeter},
168 BoundedVec,
169};
170use frame_system::{
171 pallet_prelude::{BlockNumberFor, *},
172 Pallet as System,
173};
174use sp_runtime::Saturating;
175
176/// Points to the next migration to execute.
177#[derive(
178 Debug,
179 Clone,
180 Eq,
181 PartialEq,
182 Encode,
183 Decode,
184 DecodeWithMemTracking,
185 scale_info::TypeInfo,
186 MaxEncodedLen,
187)]
188pub enum MigrationCursor<Cursor, BlockNumber> {
189 /// Points to the currently active migration and its inner cursor.
190 Active(ActiveCursor<Cursor, BlockNumber>),
191
192 /// Migration got stuck and cannot proceed. This is bad.
193 Stuck,
194}
195
196impl<Cursor, BlockNumber> MigrationCursor<Cursor, BlockNumber> {
197 /// Try to return self as an [`ActiveCursor`].
198 pub fn as_active(&self) -> Option<&ActiveCursor<Cursor, BlockNumber>> {
199 match self {
200 MigrationCursor::Active(active) => Some(active),
201 MigrationCursor::Stuck => None,
202 }
203 }
204}
205
206impl<Cursor, BlockNumber> From<ActiveCursor<Cursor, BlockNumber>>
207 for MigrationCursor<Cursor, BlockNumber>
208{
209 fn from(active: ActiveCursor<Cursor, BlockNumber>) -> Self {
210 MigrationCursor::Active(active)
211 }
212}
213
214/// Points to the currently active migration and its inner cursor.
215#[derive(
216 Debug,
217 Clone,
218 Eq,
219 PartialEq,
220 Encode,
221 Decode,
222 DecodeWithMemTracking,
223 scale_info::TypeInfo,
224 MaxEncodedLen,
225)]
226pub struct ActiveCursor<Cursor, BlockNumber> {
227 /// The index of the migration in the MBM tuple.
228 pub index: u32,
229 /// The cursor of the migration that is referenced by `index`.
230 pub inner_cursor: Option<Cursor>,
231 /// The block number that the migration started at.
232 ///
233 /// This is used to calculate how many blocks it took.
234 pub started_at: BlockNumber,
235}
236
237impl<Cursor, BlockNumber> ActiveCursor<Cursor, BlockNumber> {
238 /// Advance the overarching cursor to the next migration.
239 pub(crate) fn goto_next_migration(&mut self, current_block: BlockNumber) {
240 self.index.saturating_inc();
241 self.inner_cursor = None;
242 self.started_at = current_block;
243 }
244}
245
246/// How to clear the records of historic migrations.
247#[derive(
248 Debug, Clone, Eq, PartialEq, Encode, Decode, DecodeWithMemTracking, scale_info::TypeInfo,
249)]
250pub enum HistoricCleanupSelector<Id> {
251 /// Clear exactly these entries.
252 ///
253 /// This is the advised way of doing it.
254 Specific(Vec<Id>),
255
256 /// Clear up to this many entries
257 Wildcard {
258 /// How many should be cleared in this call at most.
259 limit: Option<u32>,
260 /// The cursor that was emitted from any previous `HistoricCleared`.
261 ///
262 /// Does not need to be passed when clearing the first batch.
263 previous_cursor: Option<Vec<u8>>,
264 },
265}
266
267/// The default number of entries that should be cleared by a `HistoricCleanupSelector::Wildcard`.
268///
269/// The caller can explicitly specify a higher amount. Benchmarks are run with twice this value.
270const DEFAULT_HISTORIC_BATCH_CLEAR_SIZE: u32 = 128;
271
272impl<Id> HistoricCleanupSelector<Id> {
273 /// The maximal number of entries that this will remove.
274 ///
275 /// Needed for weight calculation.
276 pub fn limit(&self) -> u32 {
277 match self {
278 Self::Specific(ids) => ids.len() as u32,
279 Self::Wildcard { limit, .. } => limit.unwrap_or(DEFAULT_HISTORIC_BATCH_CLEAR_SIZE),
280 }
281 }
282}
283
284/// Convenience alias for [`MigrationCursor`].
285pub type CursorOf<T> = MigrationCursor<RawCursorOf<T>, BlockNumberFor<T>>;
286
287/// Convenience alias for the raw inner cursor of a migration.
288pub type RawCursorOf<T> = BoundedVec<u8, <T as Config>::CursorMaxLen>;
289
290/// Convenience alias for the identifier of a migration.
291pub type IdentifierOf<T> = BoundedVec<u8, <T as Config>::IdentifierMaxLen>;
292
293/// Convenience alias for [`ActiveCursor`].
294pub type ActiveCursorOf<T> = ActiveCursor<RawCursorOf<T>, BlockNumberFor<T>>;
295
296/// Trait for a tuple of No-OP migrations with one element.
297#[impl_trait_for_tuples::impl_for_tuples(30)]
298pub trait MockedMigrations: SteppedMigrations {
299 /// The migration should fail after `n` steps.
300 fn set_fail_after(n: u32);
301 /// The migration should succeed after `n` steps.
302 fn set_success_after(n: u32);
303}
304
305#[cfg(feature = "try-runtime")]
306/// Wrapper for pre-upgrade bytes, allowing us to impl MEL on it.
307///
308/// For `try-runtime` testing only.
309#[derive(Debug, Clone, Eq, PartialEq, Encode, Decode, scale_info::TypeInfo, Default)]
310struct PreUpgradeBytesWrapper(pub Vec<u8>);
311
312/// Data stored by the pre-upgrade hook of the MBMs. Only used for `try-runtime` testing.
313///
314/// Define this outside of the pallet so it is not confused with actual storage.
315#[cfg(feature = "try-runtime")]
316#[frame_support::storage_alias]
317type PreUpgradeBytes<T: Config> =
318 StorageMap<Pallet<T>, Twox64Concat, IdentifierOf<T>, PreUpgradeBytesWrapper, ValueQuery>;
319
320#[frame_support::pallet]
321pub mod pallet {
322 use super::*;
323
324 /// The in-code storage version.
325 const STORAGE_VERSION: StorageVersion = StorageVersion::new(0);
326
327 #[pallet::pallet]
328 #[pallet::storage_version(STORAGE_VERSION)]
329 pub struct Pallet<T>(_);
330
331 #[pallet::config(with_default)]
332 pub trait Config: frame_system::Config {
333 /// The overarching event type of the runtime.
334 #[pallet::no_default_bounds]
335 #[allow(deprecated)]
336 type RuntimeEvent: From<Event<Self>> + IsType<<Self as frame_system::Config>::RuntimeEvent>;
337
338 /// All the multi-block migrations to run.
339 ///
340 /// Should only be updated in a runtime-upgrade once all the old migrations have completed.
341 /// (Check that [`Cursor`] is `None`).
342 #[cfg(not(feature = "runtime-benchmarks"))]
343 #[pallet::no_default]
344 type Migrations: SteppedMigrations;
345
346 /// Mocked migrations for benchmarking only.
347 ///
348 /// Should be configured to [`crate::mock_helpers::MockedMigrations`] in benchmarks.
349 #[cfg(feature = "runtime-benchmarks")]
350 #[pallet::no_default]
351 type Migrations: MockedMigrations;
352
353 /// The maximal length of an encoded cursor.
354 ///
355 /// A good default needs to selected such that no migration will ever have a cursor with MEL
356 /// above this limit. This is statically checked in `integrity_test`.
357 #[pallet::constant]
358 type CursorMaxLen: Get<u32>;
359
360 /// The maximal length of an encoded identifier.
361 ///
362 /// A good default needs to selected such that no migration will ever have an identifier
363 /// with MEL above this limit. This is statically checked in `integrity_test`.
364 #[pallet::constant]
365 type IdentifierMaxLen: Get<u32>;
366
367 /// Notifications for status updates of a runtime upgrade.
368 ///
369 /// Could be used to pause XCM etc.
370 type MigrationStatusHandler: MigrationStatusHandler;
371
372 /// Handler for failed migrations.
373 type FailedMigrationHandler: FailedMigrationHandler;
374
375 /// The maximum weight to spend each block to execute migrations.
376 type MaxServiceWeight: Get<Weight>;
377
378 /// Weight information for the calls and functions of this pallet.
379 type WeightInfo: WeightInfo;
380 }
381
382 /// Default implementations of [`DefaultConfig`], which can be used to implement [`Config`].
383 pub mod config_preludes {
384 use super::{inject_runtime_type, DefaultConfig};
385 use frame_support::{
386 derive_impl,
387 migrations::FreezeChainOnFailedMigration,
388 pallet_prelude::{ConstU32, *},
389 };
390 use frame_system::limits::BlockWeights;
391
392 /// Provides a viable default config that can be used with
393 /// [`derive_impl`](`frame_support::derive_impl`) to derive a testing pallet config
394 /// based on this one.
395 ///
396 /// See `Test` in the `default-config` example pallet's `test.rs` for an example of
397 /// a downstream user of this particular `TestDefaultConfig`
398 pub struct TestDefaultConfig;
399
400 frame_support::parameter_types! {
401 /// Maximal weight per block that can be spent on migrations in tests.
402 pub TestMaxServiceWeight: Weight = <<TestDefaultConfig as frame_system::DefaultConfig>::BlockWeights as Get<BlockWeights>>::get().max_block.div(2);
403 }
404
405 #[derive_impl(frame_system::config_preludes::TestDefaultConfig, no_aggregated_types)]
406 impl frame_system::DefaultConfig for TestDefaultConfig {}
407
408 #[frame_support::register_default_impl(TestDefaultConfig)]
409 impl DefaultConfig for TestDefaultConfig {
410 #[inject_runtime_type]
411 type RuntimeEvent = ();
412 type CursorMaxLen = ConstU32<{ 1 << 16 }>;
413 type IdentifierMaxLen = ConstU32<{ 256 }>;
414 type MigrationStatusHandler = ();
415 type FailedMigrationHandler = FreezeChainOnFailedMigration;
416 type MaxServiceWeight = TestMaxServiceWeight;
417 type WeightInfo = ();
418 }
419 }
420
421 /// The currently active migration to run and its cursor.
422 ///
423 /// `None` indicates that no migration is running.
424 #[pallet::storage]
425 pub type Cursor<T: Config> = StorageValue<_, CursorOf<T>, OptionQuery>;
426
427 /// Set of all successfully executed migrations.
428 ///
429 /// This is used as blacklist, to not re-execute migrations that have not been removed from the
430 /// codebase yet. Governance can regularly clear this out via `clear_historic`.
431 #[pallet::storage]
432 pub type Historic<T: Config> = StorageMap<_, Twox64Concat, IdentifierOf<T>, (), OptionQuery>;
433
434 #[pallet::event]
435 #[pallet::generate_deposit(pub(super) fn deposit_event)]
436 pub enum Event<T: Config> {
437 /// A Runtime upgrade started.
438 ///
439 /// Its end is indicated by `UpgradeCompleted` or `UpgradeFailed`.
440 UpgradeStarted {
441 /// The number of migrations that this upgrade contains.
442 ///
443 /// This can be used to design a progress indicator in combination with counting the
444 /// `MigrationCompleted` and `MigrationSkipped` events.
445 migrations: u32,
446 },
447 /// The current runtime upgrade completed.
448 ///
449 /// This implies that all of its migrations completed successfully as well.
450 UpgradeCompleted,
451 /// Runtime upgrade failed.
452 ///
453 /// This is very bad and will require governance intervention.
454 UpgradeFailed,
455 /// A migration was skipped since it was already executed in the past.
456 MigrationSkipped {
457 /// The index of the skipped migration within the [`Config::Migrations`] list.
458 index: u32,
459 },
460 /// A migration progressed.
461 MigrationAdvanced {
462 /// The index of the migration within the [`Config::Migrations`] list.
463 index: u32,
464 /// The number of blocks that this migration took so far.
465 took: BlockNumberFor<T>,
466 },
467 /// A Migration completed.
468 MigrationCompleted {
469 /// The index of the migration within the [`Config::Migrations`] list.
470 index: u32,
471 /// The number of blocks that this migration took so far.
472 took: BlockNumberFor<T>,
473 },
474 /// A Migration failed.
475 ///
476 /// This implies that the whole upgrade failed and governance intervention is required.
477 MigrationFailed {
478 /// The index of the migration within the [`Config::Migrations`] list.
479 index: u32,
480 /// The number of blocks that this migration took so far.
481 took: BlockNumberFor<T>,
482 },
483 /// The set of historical migrations has been cleared.
484 HistoricCleared {
485 /// Should be passed to `clear_historic` in a successive call.
486 next_cursor: Option<Vec<u8>>,
487 },
488 }
489
490 #[pallet::error]
491 pub enum Error<T> {
492 /// The operation cannot complete since some MBMs are ongoing.
493 Ongoing,
494 }
495
496 #[pallet::hooks]
497 impl<T: Config> Hooks<BlockNumberFor<T>> for Pallet<T> {
498 fn on_runtime_upgrade() -> Weight {
499 Self::onboard_new_mbms()
500 }
501
502 #[cfg(feature = "std")]
503 fn integrity_test() {
504 // Check that the migrations tuple is legit.
505 frame_support::assert_ok!(T::Migrations::integrity_test());
506
507 // Very important! Ensure that the pallet is configured in `System::Config`.
508 {
509 assert!(!Cursor::<T>::exists(), "Externalities storage should be clean");
510 assert!(!<T as frame_system::Config>::MultiBlockMigrator::ongoing());
511
512 Cursor::<T>::put(MigrationCursor::Stuck);
513 assert!(<T as frame_system::Config>::MultiBlockMigrator::ongoing());
514
515 Cursor::<T>::kill();
516 }
517
518 // The per-block service weight is sane.
519 {
520 let want = T::MaxServiceWeight::get();
521 let max = <T as frame_system::Config>::BlockWeights::get().max_block;
522
523 assert!(want.all_lte(max), "Service weight is larger than a block: {want} > {max}");
524 }
525
526 // Cursor MEL
527 {
528 let mel = T::Migrations::cursor_max_encoded_len();
529 let max_mel = T::CursorMaxLen::get() as usize;
530 assert!(
531 mel <= max_mel,
532 "A Cursor is not guaranteed to fit into the storage: {mel} > {max_mel}",
533 );
534 }
535
536 // Identifier MEL
537 {
538 let mel = T::Migrations::identifier_max_encoded_len();
539 let max_mel = T::IdentifierMaxLen::get() as usize;
540 assert!(
541 mel <= max_mel,
542 "An Identifier is not guaranteed to fit into the storage: {mel} > {max_mel}",
543 );
544 }
545 }
546 }
547
548 #[pallet::call(weight = T::WeightInfo)]
549 impl<T: Config> Pallet<T> {
550 /// Allows root to set a cursor to forcefully start, stop or forward the migration process.
551 ///
552 /// Should normally not be needed and is only in place as emergency measure. Note that
553 /// restarting the migration process in this manner will not call the
554 /// [`MigrationStatusHandler::started`] hook or emit an `UpgradeStarted` event.
555 #[pallet::call_index(0)]
556 pub fn force_set_cursor(
557 origin: OriginFor<T>,
558 cursor: Option<CursorOf<T>>,
559 ) -> DispatchResult {
560 ensure_root(origin)?;
561
562 Cursor::<T>::set(cursor);
563
564 Ok(())
565 }
566
567 /// Allows root to set an active cursor to forcefully start/forward the migration process.
568 ///
569 /// This is an edge-case version of [`Self::force_set_cursor`] that allows to set the
570 /// `started_at` value to the next block number. Otherwise this would not be possible, since
571 /// `force_set_cursor` takes an absolute block number. Setting `started_at` to `None`
572 /// indicates that the current block number plus one should be used.
573 #[pallet::call_index(1)]
574 pub fn force_set_active_cursor(
575 origin: OriginFor<T>,
576 index: u32,
577 inner_cursor: Option<RawCursorOf<T>>,
578 started_at: Option<BlockNumberFor<T>>,
579 ) -> DispatchResult {
580 ensure_root(origin)?;
581
582 let started_at = started_at.unwrap_or(
583 System::<T>::block_number().saturating_add(sp_runtime::traits::One::one()),
584 );
585 Cursor::<T>::put(MigrationCursor::Active(ActiveCursor {
586 index,
587 inner_cursor,
588 started_at,
589 }));
590
591 Ok(())
592 }
593
594 /// Forces the onboarding of the migrations.
595 ///
596 /// This process happens automatically on a runtime upgrade. It is in place as an emergency
597 /// measurement. The cursor needs to be `None` for this to succeed.
598 #[pallet::call_index(2)]
599 pub fn force_onboard_mbms(origin: OriginFor<T>) -> DispatchResult {
600 ensure_root(origin)?;
601
602 ensure!(!Cursor::<T>::exists(), Error::<T>::Ongoing);
603 Self::onboard_new_mbms();
604
605 Ok(())
606 }
607
608 /// Clears the `Historic` set.
609 ///
610 /// `map_cursor` must be set to the last value that was returned by the
611 /// `HistoricCleared` event. The first time `None` can be used. `limit` must be chosen in a
612 /// way that will result in a sensible weight.
613 #[pallet::call_index(3)]
614 #[pallet::weight(T::WeightInfo::clear_historic(selector.limit()))]
615 pub fn clear_historic(
616 origin: OriginFor<T>,
617 selector: HistoricCleanupSelector<IdentifierOf<T>>,
618 ) -> DispatchResult {
619 ensure_root(origin)?;
620
621 match &selector {
622 HistoricCleanupSelector::Specific(ids) => {
623 for id in ids {
624 Historic::<T>::remove(id);
625 }
626 Self::deposit_event(Event::HistoricCleared { next_cursor: None });
627 },
628 HistoricCleanupSelector::Wildcard { previous_cursor, .. } => {
629 let next = Historic::<T>::clear(selector.limit(), previous_cursor.as_deref());
630 Self::deposit_event(Event::HistoricCleared { next_cursor: next.maybe_cursor });
631 },
632 }
633
634 Ok(())
635 }
636 }
637}
638
639impl<T: Config> Pallet<T> {
640 /// Onboard all new Multi-Block-Migrations and start the process of executing them.
641 ///
642 /// Should only be called once all previous migrations completed.
643 fn onboard_new_mbms() -> Weight {
644 if let Some(cursor) = Cursor::<T>::get() {
645 log::error!("Ongoing migrations interrupted - chain stuck");
646
647 let maybe_index = cursor.as_active().map(|c| c.index);
648 Self::upgrade_failed(maybe_index);
649 return T::WeightInfo::onboard_new_mbms()
650 }
651
652 let migrations = T::Migrations::len();
653 log::debug!("Onboarding {migrations} new MBM migrations");
654
655 if migrations > 0 {
656 // Set the cursor to the first migration:
657 Cursor::<T>::set(Some(
658 ActiveCursor {
659 index: 0,
660 inner_cursor: None,
661 started_at: System::<T>::block_number(),
662 }
663 .into(),
664 ));
665 Self::deposit_event(Event::UpgradeStarted { migrations });
666 T::MigrationStatusHandler::started();
667 }
668
669 T::WeightInfo::onboard_new_mbms()
670 }
671
672 /// Tries to make progress on the Multi-Block-Migrations process.
673 fn progress_mbms(n: BlockNumberFor<T>) -> Weight {
674 let mut meter = WeightMeter::with_limit(T::MaxServiceWeight::get());
675 meter.consume(T::WeightInfo::progress_mbms_none());
676
677 let mut cursor = match Cursor::<T>::get() {
678 None => {
679 log::trace!("[Block {n:?}] Waiting for cursor to become `Some`.");
680 return meter.consumed()
681 },
682 Some(MigrationCursor::Active(cursor)) => {
683 log::debug!("Progressing MBM #{}", cursor.index);
684 cursor
685 },
686 Some(MigrationCursor::Stuck) => {
687 log::error!("Migration stuck. Governance intervention required.");
688 return meter.consumed()
689 },
690 };
691 debug_assert!(Self::ongoing());
692
693 // The limit here is a defensive measure to prevent an infinite loop. It expresses that we
694 // allow no more than 8 MBMs to finish in a single block. This should be harmless, since we
695 // generally expect *Multi*-Block-Migrations to take *multiple* blocks.
696 for i in 0..8 {
697 match Self::exec_migration(cursor, i == 0, &mut meter) {
698 None => return meter.consumed(),
699 Some(ControlFlow::Continue(next_cursor)) => {
700 cursor = next_cursor;
701 },
702 Some(ControlFlow::Break(last_cursor)) => {
703 cursor = last_cursor;
704 break
705 },
706 }
707 }
708
709 Cursor::<T>::set(Some(cursor.into()));
710
711 meter.consumed()
712 }
713
714 /// Try to make progress on the current migration.
715 ///
716 /// Returns whether processing should continue or break for this block. The return value means:
717 /// - `None`: The migration process is completely finished.
718 /// - `ControlFlow::Break`: Continue in the *next* block with the given cursor.
719 /// - `ControlFlow::Continue`: Continue in the *current* block with the given cursor.
720 fn exec_migration(
721 mut cursor: ActiveCursorOf<T>,
722 is_first: bool,
723 meter: &mut WeightMeter,
724 ) -> Option<ControlFlow<ActiveCursorOf<T>, ActiveCursorOf<T>>> {
725 // The differences between the single branches' weights is not that big. And since we do
726 // only one step per block, we can just use the maximum instead of more precise accounting.
727 if meter.try_consume(Self::exec_migration_max_weight()).is_err() {
728 defensive_assert!(!is_first, "There should be enough weight to do this at least once");
729 return Some(ControlFlow::Break(cursor))
730 }
731
732 if cursor.index >= T::Migrations::len() {
733 // No more migrations in the tuple - we are done.
734 defensive_assert!(cursor.index == T::Migrations::len(), "Inconsistent MBMs tuple");
735 Self::deposit_event(Event::UpgradeCompleted);
736 Cursor::<T>::kill();
737 T::MigrationStatusHandler::completed();
738 return None;
739 };
740
741 let id = T::Migrations::nth_id(cursor.index).map(TryInto::try_into);
742 let Some(Ok(bounded_id)): Option<Result<IdentifierOf<T>, _>> = id else {
743 defensive!("integrity_test ensures that all identifiers are present and bounde; qed.");
744 Self::upgrade_failed(Some(cursor.index));
745 return None
746 };
747
748 if Historic::<T>::contains_key(&bounded_id) {
749 Self::deposit_event(Event::MigrationSkipped { index: cursor.index });
750 cursor.goto_next_migration(System::<T>::block_number());
751 return Some(ControlFlow::Continue(cursor))
752 }
753
754 let max_steps = T::Migrations::nth_max_steps(cursor.index);
755
756 // If this is the first time running this migration, exec the pre-upgrade hook.
757 #[cfg(feature = "try-runtime")]
758 if !PreUpgradeBytes::<T>::contains_key(&bounded_id) {
759 let bytes = T::Migrations::nth_pre_upgrade(cursor.index)
760 .expect("Invalid cursor.index")
761 .expect("Pre-upgrade failed");
762 PreUpgradeBytes::<T>::insert(&bounded_id, PreUpgradeBytesWrapper(bytes));
763 }
764
765 let next_cursor = T::Migrations::nth_transactional_step(
766 cursor.index,
767 cursor.inner_cursor.clone().map(|c| c.into_inner()),
768 meter,
769 );
770 let Some((max_steps, next_cursor)) = max_steps.zip(next_cursor) else {
771 defensive!("integrity_test ensures that the tuple is valid; qed");
772 Self::upgrade_failed(Some(cursor.index));
773 return None
774 };
775
776 let took = System::<T>::block_number().saturating_sub(cursor.started_at);
777 match next_cursor {
778 Ok(Some(next_cursor)) => {
779 let Ok(bound_next_cursor) = next_cursor.try_into() else {
780 defensive!("The integrity check ensures that all cursors' MEL bound fits into CursorMaxLen; qed");
781 Self::upgrade_failed(Some(cursor.index));
782 return None
783 };
784
785 Self::deposit_event(Event::MigrationAdvanced { index: cursor.index, took });
786 cursor.inner_cursor = Some(bound_next_cursor);
787
788 if max_steps.is_some_and(|max| took > max.into()) {
789 Self::deposit_event(Event::MigrationFailed { index: cursor.index, took });
790 Self::upgrade_failed(Some(cursor.index));
791 None
792 } else {
793 // A migration cannot progress more than one step per block, we therefore break.
794 Some(ControlFlow::Break(cursor))
795 }
796 },
797 Ok(None) => {
798 // A migration is done when it returns cursor `None`.
799
800 // Run post-upgrade checks.
801 #[cfg(feature = "try-runtime")]
802 T::Migrations::nth_post_upgrade(
803 cursor.index,
804 PreUpgradeBytes::<T>::get(&bounded_id).0,
805 )
806 .expect("Invalid cursor.index.")
807 .expect("Post-upgrade failed.");
808
809 Self::deposit_event(Event::MigrationCompleted { index: cursor.index, took });
810 Historic::<T>::insert(&bounded_id, ());
811 cursor.goto_next_migration(System::<T>::block_number());
812 Some(ControlFlow::Continue(cursor))
813 },
814 Err(SteppedMigrationError::InsufficientWeight { required }) => {
815 if is_first || required.any_gt(meter.limit()) {
816 Self::deposit_event(Event::MigrationFailed { index: cursor.index, took });
817 Self::upgrade_failed(Some(cursor.index));
818 None
819 } else {
820 // Retry and hope that there is more weight in the next block.
821 Some(ControlFlow::Break(cursor))
822 }
823 },
824 Err(SteppedMigrationError::InvalidCursor | SteppedMigrationError::Failed) => {
825 Self::deposit_event(Event::MigrationFailed { index: cursor.index, took });
826 Self::upgrade_failed(Some(cursor.index));
827 None
828 },
829 }
830 }
831
832 /// Fail the current runtime upgrade, caused by `migration`.
833 ///
834 /// When the `try-runtime` feature is enabled, this function will panic.
835 // Allow unreachable code so it can compile without warnings when `try-runtime` is enabled.
836 fn upgrade_failed(migration: Option<u32>) {
837 use FailedMigrationHandling::*;
838 Self::deposit_event(Event::UpgradeFailed);
839
840 if cfg!(feature = "try-runtime") {
841 panic!("Migration with index {migration:?} failed.");
842 } else {
843 match T::FailedMigrationHandler::failed(migration) {
844 KeepStuck => Cursor::<T>::set(Some(MigrationCursor::Stuck)),
845 ForceUnstuck => Cursor::<T>::kill(),
846 Ignore => {},
847 }
848 }
849 }
850
851 /// The maximal weight of calling the private `Self::exec_migration` function.
852 pub fn exec_migration_max_weight() -> Weight {
853 T::WeightInfo::exec_migration_complete()
854 .max(T::WeightInfo::exec_migration_completed())
855 .max(T::WeightInfo::exec_migration_skipped_historic())
856 .max(T::WeightInfo::exec_migration_advance())
857 .max(T::WeightInfo::exec_migration_fail())
858 }
859}
860
861impl<T: Config> MultiStepMigrator for Pallet<T> {
862 fn ongoing() -> bool {
863 Cursor::<T>::exists()
864 }
865
866 fn step() -> Weight {
867 Self::progress_mbms(System::<T>::block_number())
868 }
869}