Module polkadot_sdk_docs::guides::enable_pov_reclaim

How to enhance a given runtime and node to be cumulus-enabled, run it as a parachain and connect it to a relay-chain. How to enable storage weight reclaiming in a parachain node and runtime.

Enable storage weight reclaiming

This guide will teach you how to enable storage weight reclaiming for a parachain. The explanations in this guide assume a project structure similar to the one detailed in the substrate documentation. Full technical details are available in the original pull request.

What is PoV reclaim?

When a parachain submits a block to a relay chain like Polkadot or Kusama, it sends the block itself and a storage proof. Together they form the Proof-of-Validity (PoV). The PoV allows the relay chain to validate the parachain block by re-executing it. Relay chain validators distribute this PoV among themselves over the network. This distribution is costly and limits the size of the storage proof. The storage weight dimension of FRAME weights reflects this cost and limits the size of the storage proof. However, the storage weight determined during benchmarking represents the worst case. In reality, runtime operations often consume less space in the storage proof. PoV reclaim offers a mechanism to reclaim the difference between the benchmarked worst-case and the real proof-size consumption.

How to enable PoV reclaim

1. Add the host function to your node

To reclaim excess storage weight, a parachain runtime needs the ability to fetch the size of the storage proof from the node. The reclaim mechanism uses the storage_proof_size host function for this purpose. For convenience, cumulus provides ParachainHostFunctions, a set of host functions typically used by cumulus-based parachains. In the binary crate of your parachain, find the instantiation of the WasmExecutor and set the correct generic type.

This example from the parachain-template shows a type definition that includes the correct host functions.

type ParachainExecutor = WasmExecutor<ParachainHostFunctions>;

Note:

If you see error runtime requires function imports which are not present on the host: 'env:ext_storage_proof_size_storage_proof_size_version_1', it is likely that this step in the guide was not set up correctly.

2. Enable storage proof recording during import

The reclaim mechanism reads the size of the currently recorded storage proof multiple times during block authoring and block import. Proof recording during authoring is already enabled on parachains. You must also ensure that storage proof recording is enabled during block import. Find where your node builds the fundamental substrate components by calling new_full_parts. Replace this with new_full_parts_record_import and pass true as the last parameter to enable import recording.

pub fn new_partial(config: &Configuration) -> Result<Service, sc_service::Error> {
	let telemetry = config
		.telemetry_endpoints
		.clone()
		.filter(|x| !x.is_empty())
		.map(|endpoints| -> Result<_, sc_telemetry::Error> {
			let worker = TelemetryWorker::new(16)?;
			let telemetry = worker.handle().new_telemetry(endpoints);
			Ok((worker, telemetry))
		})
		.transpose()?;

	let heap_pages = config
		.executor
		.default_heap_pages
		.map_or(DEFAULT_HEAP_ALLOC_STRATEGY, |h| HeapAllocStrategy::Static { extra_pages: h as _ });

	let executor = ParachainExecutor::builder()
		.with_execution_method(config.executor.wasm_method)
		.with_onchain_heap_alloc_strategy(heap_pages)
		.with_offchain_heap_alloc_strategy(heap_pages)
		.with_max_runtime_instances(config.executor.max_runtime_instances)
		.with_runtime_cache_size(config.executor.runtime_cache_size)
		.build();

	let (client, backend, keystore_container, task_manager) =
		sc_service::new_full_parts_record_import::<Block, RuntimeApi, _>(
			config,
			telemetry.as_ref().map(|(_, telemetry)| telemetry.handle()),
			executor,
			true,
		)?;
	let client = Arc::new(client);

	let telemetry_worker_handle = telemetry.as_ref().map(|(worker, _)| worker.handle());

	let telemetry = telemetry.map(|(worker, telemetry)| {
		task_manager.spawn_handle().spawn("telemetry", None, worker.run());
		telemetry
	});

	let transaction_pool = Arc::from(
		sc_transaction_pool::Builder::new(
			task_manager.spawn_essential_handle(),
			client.clone(),
			config.role.is_authority().into(),
		)
		.with_options(config.transaction_pool.clone())
		.with_prometheus(config.prometheus_registry())
		.build(),
	);

	let block_import = ParachainBlockImport::new(client.clone(), backend.clone());

	let import_queue = build_import_queue(
		client.clone(),
		block_import.clone(),
		config,
		telemetry.as_ref().map(|telemetry| telemetry.handle()),
		&task_manager,
	);

	Ok(PartialComponents {
		backend,
		client,
		import_queue,
		keystore_container,
		task_manager,
		transaction_pool,
		select_chain: (),
		other: (block_import, telemetry, telemetry_worker_handle),
	})
}

Note:

If you see error Storage root must match that calculated. during block import, it is likely that this step in the guide was not set up correctly.

3. Add the TransactionExtension to your runtime

In your runtime, you will find a list of TransactionExtensions. To enable the reclaiming, add StorageWeightReclaim to that list. For maximum efficiency, make sure that StorageWeightReclaim is last in the list. The extension will check the size of the storage proof before and after an extrinsic execution. It reclaims the difference between the calculated size and the benchmarked size.

pub type TxExtension = (
	frame_system::CheckNonZeroSender<Runtime>,
	frame_system::CheckSpecVersion<Runtime>,
	frame_system::CheckTxVersion<Runtime>,
	frame_system::CheckGenesis<Runtime>,
	frame_system::CheckEra<Runtime>,
	frame_system::CheckNonce<Runtime>,
	frame_system::CheckWeight<Runtime>,
	pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
	cumulus_primitives_storage_weight_reclaim::StorageWeightReclaim<Runtime>,
	frame_metadata_hash_extension::CheckMetadataHash<Runtime>,
);

Optional: Verify that reclaim works

Start your node with the log target runtime::storage_reclaim set to trace to enable full logging for StorageWeightReclaim. The following log is an example from a local testnet. To trigger the log, execute any extrinsic on the network.

...
2024-04-22 17:31:48.014 TRACE runtime::storage_reclaim: [ferdie] Reclaiming storage weight. benchmarked: 3593, consumed: 265 unspent: 0
...

In the above example we see a benchmarked size of 3593 bytes, while the extrinsic only consumed 265 bytes of proof size. This results in 3328 bytes of reclaim.