Introduction

ZombieNet SDK is a Rust testing framework for Polkadot SDK-based blockchains. It spawns ephemeral blockchain networks programmatically for testing and development.

The SDK succeeds the original ZombieNet (TypeScript), offering a type-safe, composable Rust API.

Migrating from TypeScript ZombieNet? The TOML configuration format is largely compatible. See Network Definition Spec for the full reference.

Key Features

  • Programmatic Network Spawning - Define and spawn relay chains and parachains from Rust code
  • Fluent Builder API - Compose networks using a chainable builder pattern
  • Multiple Providers - Run on Kubernetes, Podman, Docker, or natively
  • Metrics & Assertions - Query Prometheus metrics and validate network behavior
  • Subxt Integration - Interact with networks using subxt

Use Cases

  • Integration Testing - Test cross-chain functionality with complete networks
  • CI/CD Pipelines - Automate network deployment and testing
  • Development - Spin up local networks for development and debugging
  • Parachain Testing - Test registration, block production etc.

Next

See Getting Started to spawn your first network.

Getting Started

This guide walks you through setting up ZombieNet SDK and spawning your first network.

  1. Setup - Prerequisites and installation
  2. Your First Network - Spawn a relay chain and parachain

Next

Setup

Prerequisites

  • Rust - Install via rustup
  • A provider (at least one):
    • Native: Binaries used by your network in PATH (or absolute paths in config), e.g., polkadot, polkadot-parachain
    • Docker/Podman: Container runtime installed and running
    • Kubernetes: Cluster access with kubectl configured

Installation

CLI

Download pre-built binaries from GitHub Releases, or install via Cargo:

cargo install zombie-cli

Or build from source:

git clone https://github.com/paritytech/zombienet-sdk
cd zombienet-sdk
cargo build --release -p zombie-cli
# Binary will be at target/release/zombie-cli

As a Dependency

Add to your Cargo.toml:

[dependencies]
zombienet-sdk = "0.4"

Your First Network

Using TOML

Create network.toml:

[relaychain]
chain = "rococo-local"
default_command = "polkadot"

[[relaychain.nodes]]
name = "alice"

[[relaychain.nodes]]
name = "bob"

Spawn with the CLI:

zombie-cli spawn network.toml --provider native
# or: --provider docker

Programmatically

The same network in Rust:

use zombienet_sdk::{NetworkConfigBuilder, NetworkConfigExt};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let network = NetworkConfigBuilder::new()
        .with_relaychain(|r| {
            r.with_chain("rococo-local")
                .with_default_command("polkadot")
                .with_validator(|v| v.with_name("alice"))
                .with_validator(|v| v.with_name("bob"))
        })
        .build()?
        .spawn_native()
        .await?;

    let alice = network.get_node("alice")?;
    println!("Alice WS: {}", alice.ws_uri());

    tokio::signal::ctrl_c().await?;
    Ok(())
}

Adding a Parachain

TOML

Extend network.toml:

[relaychain]
chain = "rococo-local"
default_command = "polkadot"

[[relaychain.nodes]]
name = "alice"

[[relaychain.nodes]]
name = "bob"

[[parachains]]
id = 1000

    [[parachains.collators]]
    name = "collator01"
    command = "polkadot-parachain"

Programmatically

use zombienet_sdk::{NetworkConfigBuilder, NetworkConfigExt};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let network = NetworkConfigBuilder::new()
        .with_relaychain(|r| {
            r.with_chain("rococo-local")
                .with_default_command("polkadot")
                .with_validator(|v| v.with_name("alice"))
                .with_validator(|v| v.with_name("bob"))
        })
        .with_parachain(|p| {
            p.with_id(1000)
                .with_collator(|c| {
                    c.with_name("collator01")
                        .with_command("polkadot-parachain")
                })
        })
        .build()?
        .spawn_native()
        .await?;

    let collator = network.get_node("collator01")?;
    println!("Collator WS: {}", collator.ws_uri());

    tokio::signal::ctrl_c().await?;
    Ok(())
}

How It Works

This document covers the two core aspects of what ZombieNet does under the hood: chain spec customization and command-line argument generation.

Chain Spec Customization

When spawning a network, ZombieNet modifies chain specifications to configure the test environment. These customizations happen automatically based on your network configuration.

Relay Chain Customizations

The framework applies these modifications to relay chain specs:

CustomizationDescription
Runtime Genesis PatchMerges user-provided genesis_overrides JSON (applied first): objects merge, any other value replaces
Genesis PatchApplies the genesis_patch RFC 6902 operations after the overrides; add at /- appends to a list. Needs a plain chain spec. A replace path, or the parent of an add path, must exist once genesis is merged. The steps below run after the patch, so what they rewrite (authorities, stakers, invulnerables, validatorCount, node balances) wins over it
Clear AuthoritiesRemoves existing authorities from session/keys, aura/authorities, grandpa/authorities, collatorSelection/invulnerables, and staking/invulnerables + staking/stakers (relay only). Also sets validatorCount to 0 unless devStakers is configured.
Add BalancesAdds balances for each node's sr and sr_stash accounts. Balance is max(initial_balance, staking_min * 2). Only nodes with initial_balance > 0 receive balances.
Add Zombie AccountAdds //Zombie account with 1000 units (in chain denomination, i.e., 1000 * 10^token_decimals planck) for internal operations
Add StakingConfigures staking for validators with minimum stake
Add Session AuthoritiesAdds validators as session authorities (if session pallet exists)
Add Aura/Grandpa AuthoritiesFor chains without session pallet
Add HRMP ChannelsConfigures cross-chain messaging channels
Register ParachainsAdds InGenesis parachains to genesis

Parachain Customizations

For cumulus-based parachains:

CustomizationDescription
Override para_id/paraIdSets correct parachain ID in chain spec root (both para_id and paraId variants)
Override relay_chainSets relay chain ID in chain spec root
Apply Genesis OverridesMerges user-provided JSON overrides
Genesis PatchApplies the genesis_patch RFC 6902 operations after the overrides, before the authorities and balances below are rewritten
Clear/Add AuthoritiesClears existing authorities from session/keys, aura/authorities, grandpa/authorities, collatorSelection/invulnerables. Then adds collator session keys (if session pallet exists) or aura authorities.
Add Collator SelectionAdds invulnerable collators to collatorSelection/invulnerables
Override parachainInfoSets correct parachainId in /parachainInfo/parachainId
Add BalancesExtracts accounts from assets pallet metadata and ensures they have native token balances. Adds balances for collator sr accounts.

For EVM-based parachains (is_evm_based = true), Ethereum session keys are generated and used instead of standard keys.

Chain Spec Key Types

ZombieNet supports customizing session key types and their cryptographic schemes via the chain_spec_key_types configuration option.

Cryptographic Schemes

SchemeSuffixDescription
SR25519_srSchnorr signatures, used by most runtime keys
ED25519_edEdwards curve signatures, used by grandpa
ECDSA_ecElliptic curve, used by beefy and Ethereum compatibility

Predefined Key Types

These key types have predefined default schemes:

Key TypeDefault SchemeNotes
auraSR25519ED25519 on asset-hub-polkadot
babeSR25519
grandpaED25519
beefyECDSA
im_onlineSR25519
authority_discoverySR25519
para_validatorSR25519
para_assignmentSR25519
parachain_validatorSR25519
nimbusSR25519
vrfSR25519

Syntax

Two formats are supported:

  • Short form: aura — uses the predefined default scheme
  • Long form: aura_ed — explicitly specifies the scheme

Unknown key types default to SR25519 when using short form.

Example

[relaychain]
chain_spec_key_types = ["aura", "grandpa", "beefy"]  # Uses defaults

# Or with explicit schemes:
chain_spec_key_types = ["aura_ed", "grandpa_sr", "custom_key_ec"]

Optional Overrides

Two additional overrides can be applied after the raw chain spec is generated:

  • WASM Override (wasm_override): Replaces runtime code (:code storage key) with custom WASM
  • Raw Spec Override (raw_spec_override): Applies JSON patch merge to override any part of the raw spec

Command-Line Arguments

ZombieNet generates command-line arguments for each spawned node. Understanding what gets added helps debug issues and customize node behavior.

Framework-Managed Arguments

These arguments are always set by the framework and filtered out if provided by user configuration:

ArgumentValue
--chainGenerated chain spec path
--nameNode name from config
--rpc-corsall
--rpc-methodsunsafe
--parachain-idParachain ID (relay chain nodes only, when para_id is specified; NOT used for cumulus collators)
--node-keyDeterministic key derived from node name

Additionally, these flags are managed by the framework:

FlagWhen Added
--no-telemetryRelay chain nodes only (not cumulus collators)
--collatorCumulus collators with is_validator: true
--execution wasmEmbedded relay chain full node (after -- separator)

Arguments Added Conditionally

ArgumentCondition
--validatorRelay chain nodes with is_validator: true
--insecure-validator-i-know-what-i-doValidators, if the binary supports it
--prometheus-externalUnless already in user args
--unsafe-rpc-externalOnly for Docker/Kubernetes providers (not native)
--bootnodesWhen bootnode addresses are available

Port Arguments

The framework injects port arguments based on assigned ports:

ArgumentDescription
--prometheus-portPrometheus metrics port
--rpc-portRPC/WebSocket port
--listen-addrP2P listen address (format: /ip4/0.0.0.0/tcp/{port}/ws)
--base-pathData directory path

Cumulus Collator Arguments

Cumulus-based collators receive two sets of arguments separated by --:

Before -- (collator arguments):

  • --chain (parachain spec)
  • --collator
  • --prometheus-port, --rpc-port, --listen-addr
  • --base-path
  • User-provided collator args

After -- (embedded relay chain full node):

  • --base-path (separate relay data directory)
  • --chain (relay chain spec)
  • --execution wasm (hardcoded, always WASM execution)
  • --port (full node P2P port, injected by framework when assigned port differs from default 30333)
  • --prometheus-port (full node metrics port, injected by framework when assigned port differs from default 9616)
  • User-provided full node args

Removing Framework Arguments

Use the -: prefix to remove arguments that the framework adds:

[[relaychain.nodes]]
name = "bob"
args = ["-:--insecure-validator-i-know-what-i-do"]

This removes --insecure-validator-i-know-what-i-do from bob's command line.

The removal syntax:

  • -:--flag-name removes --flag-name
  • -:flag-name also removes --flag-name (normalized automatically)
  • Works for both flags and options (the option's value is also removed)

Example: Generated Relay Chain Command

polkadot \
  --chain /cfg/rococo-local.json \
  --name alice \
  --rpc-cors all \
  --rpc-methods unsafe \
  --unsafe-rpc-external \          # omitted for native provider
  --node-key <deterministic_key> \
  --prometheus-external \
  --validator \
  --insecure-validator-i-know-what-i-do \  # if binary supports it
  --bootnodes <bootnode_multiaddrs> \
  --prometheus-port 9615 \
  --rpc-port 9944 \
  --listen-addr /ip4/0.0.0.0/tcp/30333/ws \
  --base-path /data \
  --no-telemetry \
  [user_args...]

Example: Generated Cumulus Collator Command

polkadot-parachain \
  --chain /cfg/1000.json \
  --name collator01 \
  --rpc-cors all \
  --rpc-methods unsafe \
  --unsafe-rpc-external \          # omitted for native provider
  --node-key <deterministic_key> \
  --prometheus-external \
  --collator \                     # if is_validator: true
  --bootnodes <para_bootnode_multiaddrs> \
  --prometheus-port 9615 \
  --rpc-port 9944 \
  --listen-addr /ip4/0.0.0.0/tcp/30333/ws \
  --base-path /data \
  [user_collator_args...] \
  -- \
  --base-path /relay-data \
  --chain /cfg/rococo-local.json \
  --execution wasm \
  --port 30334 \                   # if assigned port differs from default
  --prometheus-port 9616 \         # if assigned port differs from default
  [user_full_node_args...]

Network Definition Spec

Configuration via TOML files or the builder API.

Global Settings

TOML

[settings]
timeout = 3600
node_spawn_timeout = 600
local_ip = "127.0.0.1"
base_dir = "/tmp/zombienet"
spawn_concurrency = 4
tear_down_on_failure = true
bootnodes_addresses = ["/ip4/127.0.0.1/tcp/30333/p2p/12D3KooW..."]

Builder

#![allow(unused)]
fn main() {
use zombienet_sdk::NetworkConfigBuilder;

let config = NetworkConfigBuilder::new()
    .with_relaychain(|r| {
        r.with_chain("rococo-local")
            .with_default_command("polkadot")
            .with_validator(|v| v.with_name("alice"))
    })
    .with_global_settings(|gs| {
        gs.with_network_spawn_timeout(3600)
            .with_node_spawn_timeout(600)
            .with_local_ip("127.0.0.1")
            .with_base_dir("/tmp/zombienet")
            .with_spawn_concurrency(4)
    })
    .build()
    .unwrap();
}

Reference

OptionTypeDefaultDescription
timeoutNumber3600Global timeout (seconds) for network spawn
node_spawn_timeoutNumber600Individual node spawn timeout (seconds). Override with ZOMBIE_NODE_SPAWN_TIMEOUT_SECONDS env var
local_ipString127.0.0.1Local IP for exposed services
base_dirStringRandom temp dirBase directory for network artifacts
spawn_concurrencyNumber20How many nodes are spawned at a time (one whenever a node or chain spec has {{ZOMBIE:..}} placeholders), and how many custom processes. Override with ZOMBIE_SPAWN_CONCURRENCY env var
tear_down_on_failureBooleantrueTear down network if nodes become unresponsive
bootnodes_addressesArray—External bootnode multiaddresses

Relaychain

TOML

[relaychain]
chain = "rococo-local"
default_command = "polkadot"
default_image = "parity/polkadot:latest"
default_args = ["-lruntime=debug"]

# Optional: Use a pre-existing chain spec
# chain_spec_path = "/path/to/chain-spec.json"

# Optional: Generate chain spec with a command
# chain_spec_command = "polkadot build-spec --chain {chain}"

# Optional: Runtime for chain spec generation
# chain_spec_runtime.src = "/path/to/runtime.wasm"
# chain_spec_runtime.preset = "development"

# Genesis configuration
random_nominators_count = 4
max_nominations = 16

# Genesis patch (applied to chain spec)
[relaychain.genesis.balances]
balances = [["5GrwvaEF...", 1000000000000]]

# Genesis patch as RFC 6902 operations, applied after `genesis`; paths are
# relative to the runtime genesis, and `add` at `/-` appends to a list
[[relaychain.genesis_patch]]
op = "add"
path = "/balances/balances/-"
value = ["5FHneW46...", 1000000000000]

[[relaychain.nodes]]
name = "alice"
invulnerable = true

[[relaychain.nodes]]
name = "bob"
args = ["-lparachain=trace"]

Builder

#![allow(unused)]
fn main() {
use zombienet_sdk::NetworkConfigBuilder;

let config = NetworkConfigBuilder::new()
    .with_relaychain(|r| {
        r.with_chain("rococo-local")
            .with_default_command("polkadot")
            .with_default_image("parity/polkadot:latest")
            .with_validator(|v| v.with_name("alice").invulnerable(true))
            .with_validator(|v| v.with_name("bob"))
    })
    .build()
    .unwrap();
}

Reference

OptionTypeDefaultDescription
chainStringrococo-localChain name (e.g., polkadot, rococo-local, westend)
default_commandStringpolkadotDefault binary command for nodes
default_imageString—Default container image (Docker/Podman/K8s)
default_argsArray—Default CLI arguments for all nodes
default_resourcesObject—Default resource limits (see Resources)
default_db_snapshotString—Default database snapshot path/URL
chain_spec_pathString—Path or URL to pre-existing chain spec
chain_spec_commandString—Command template to generate chain spec
chain_spec_command_is_localBooleanfalseRun chain spec command locally
chain_spec_command_output_pathString/dev/stdoutOutput path for chain spec command
chain_spec_runtimeObject—Runtime WASM for chain spec generation (see Chain Spec)
random_nominators_countNumber—Number of random nominators for staking
max_nominationsNumber—Maximum nominations per nominator
genesisObject—Genesis overrides as JSON (alias: runtime_genesis_patch)
genesis_patchArray—RFC 6902 operations applied to the runtime genesis after genesis (see How it works)
wasm_overrideString—WASM runtime override path/URL
raw_spec_overrideString/Object—Raw chain spec override (inline JSON or file path)

Parachain

TOML

[[parachains]]
id = 1000
chain = "asset-hub-rococo-local"
onboard_as_parachain = true
default_command = "polkadot-parachain"
balance = 2000000000000

# Registration strategy (pick one):
# add_to_genesis = true    # InGenesis (default, fastest)
# register_para = true     # UsingExtrinsic (via extrinsic after spawn)
# (neither = Manual, no automatic registration)

# Genesis configuration
# genesis_wasm_path = "/path/to/para-wasm"
# genesis_state_path = "/path/to/para-genesis-state"

# Or generate genesis artifacts
# genesis_wasm_generator = "polkadot-parachain export-genesis-wasm"
# genesis_state_generator = ["polkadot-parachain", "export-genesis-state"]

# Genesis patch
[parachains.genesis.balances]
balances = [["5GrwvaEF...", 1000000000000]]

[[parachains.collators]]
name = "collator01"
command = "polkadot-parachain"

[[parachains.collators]]
name = "collator02"
args = ["-lparachain=trace"]

Builder

#![allow(unused)]
fn main() {
use zombienet_sdk::{NetworkConfigBuilder, RegistrationStrategy};

let config = NetworkConfigBuilder::new()
    .with_relaychain(|r| {
        r.with_chain("rococo-local")
            .with_default_command("polkadot")
            .with_validator(|v| v.with_name("alice"))
            .with_validator(|v| v.with_name("bob"))
    })
    .with_parachain(|p| {
        p.with_id(1000)
            .with_collator(|c| c.with_name("collator01").with_command("polkadot-parachain"))
    })
    .build()
    .unwrap();
}

Reference

OptionTypeDefaultDescription
idNumber—Required. Unique parachain ID
chainString—Chain name for the parachain
cumulus_basedBooleantrueWhether parachain uses Cumulus
evm_basedBooleanfalseWhether parachain is EVM-based (e.g., Frontier)
onboard_as_parachainBooleantrueRegister as parachain (vs parathread)
add_to_genesisBoolean—Register in genesis (InGenesis strategy). Mutually exclusive with register_para
register_paraBoolean—Register via extrinsic (UsingExtrinsic strategy). Mutually exclusive with add_to_genesis
balanceNumber2000000000000Initial balance for parachain account
default_commandString—Default collator command
default_imageString—Default container image
default_argsArray—Default CLI arguments
default_resourcesObject—Default resource limits (see Resources)
default_db_snapshotString—Default database snapshot path/URL
genesis_wasm_pathString—Path/URL to genesis WASM blob
genesis_wasm_generatorString—Command to generate genesis WASM
genesis_state_pathString—Path/URL to genesis state
genesis_state_generatorString/Array—Command with args to generate genesis state
genesisObject—Genesis overrides as JSON (alias: runtime_genesis_patch)
genesis_patchArray—RFC 6902 operations applied to the runtime genesis after genesis (see How it works)
chain_spec_pathString—Path/URL to parachain chain spec
chain_spec_commandString—Command template to generate chain spec
chain_spec_command_is_localBooleanfalseRun chain spec command locally
chain_spec_command_output_pathString/dev/stdoutOutput path for chain spec command
chain_spec_runtimeObject—Runtime WASM for chain spec generation
wasm_overrideString—WASM runtime override path/URL
raw_spec_overrideString/Object—Raw chain spec override (inline JSON or file path)
bootnodes_addressesArray—Bootnode multiaddresses
no_default_bootnodesBooleanfalseDon't auto-assign bootnode role if none specified

Registration Strategies

  • InGenesis - In relay chain genesis (fastest)
  • UsingExtrinsic - Via extrinsic after spawn
  • Manual - No automatic registration

Collator Groups

[[parachains.collator_groups]]
name = "collator"
count = 3
command = "polkadot-parachain"
#![allow(unused)]
fn main() {
.with_parachain(|p| {
    p.with_id(1000)
        .with_collator_group(|g| {
            g.with_count(3)
                .with_base_node(|n| n.with_name("collator").with_command("polkadot-parachain"))
        })
})
}

Node

Validators, full nodes, and collators share this configuration.

TOML

[[relaychain.nodes]]
name = "alice"
command = "polkadot"
# subcommand = "node"  # Optional: for nested CLI commands
image = "parity/polkadot:latest"
invulnerable = true
bootnode = false
initial_balance = 5000000000000
args = ["-lruntime=debug"]
env = [
    { name = "RUST_LOG", value = "info" }
]

# Networking
rpc_port = 9933
prometheus_port = 9615
p2p_port = 30333
# p2p_cert_hash = "..."  # Optional: libp2p WebRTC cert hash
bootnodes_addresses = ["/ip4/127.0.0.1/tcp/30333/p2p/12D3KooW..."]

# Storage & Keys
db_snapshot = "/path/to/snapshot.tar.gz"
keystore_path = "/path/to/keystore"
keystore_key_types = ["aura", "gran"]
# override_eth_key = "0x..."  # Optional: override EVM session key
node_log_path = "/tmp/alice.log"

# Resources (for container providers)
[relaychain.nodes.resources]
request_memory = "1Gi"
request_cpu = "500m"
limit_memory = "2Gi"
limit_cpu = "1000m"

Builder

#![allow(unused)]
fn main() {
.with_validator(|node| {
    node.with_name("alice")
        .with_command("polkadot")
        .with_image("parity/polkadot:latest")
        .invulnerable(true)
        .with_args(vec!["-lruntime=debug".into()])
        .with_env(vec![("RUST_LOG".into(), "info".into())])
        .with_resources(|r| {
            r.with_request_memory("1Gi").with_limit_memory("2Gi")
        })
})
}

Reference

OptionTypeDefaultDescription
nameString—Required. Unique node name
commandStringInheritedBinary command to execute
subcommandString—Optional subcommand (e.g., for nested CLI)
imageStringInheritedContainer image
argsArray—CLI arguments
envArray—Environment variables as {name, value} pairs
validatorBooleantrueWhether node is a validator/authority
invulnerableBooleantrueAdd to invulnerables set in genesis
bootnodeBooleanfalseWhether node acts as a bootnode
initial_balanceNumber2000000000000Initial account balance (alias: balance)
rpc_portNumberAutoRPC port
prometheus_portNumberAutoPrometheus metrics port
p2p_portNumberAutoP2P networking port
p2p_cert_hashString—libp2p WebRTC certificate hash
bootnodes_addressesArray—Additional bootnode addresses
db_snapshotString—Database snapshot path/URL
keystore_pathString—Custom keystore directory
keystore_key_typesArray—Key types to generate in keystore (e.g., aura, gran, babe)
chain_spec_key_typesArray—Session key types to inject into chain spec genesis
override_eth_keyString—Override auto-generated EVM session key
node_log_pathString—Path for node log file
resourcesObject—Resource limits (see below)

Node Groups

Define multiple nodes with the same configuration. Creates validator-0, validator-1, etc.

TOML

[[relaychain.node_groups]]
name = "validator"
count = 5
command = "polkadot"

Builder

#![allow(unused)]
fn main() {
.with_relaychain(|r| {
    r.with_chain("rococo-local")
        .with_default_command("polkadot")
        .with_node_group(|g| {
            g.with_base_node(|node| {
                node.with_name("validator")
                    .with_args(vec!["-lruntime=debug".into()])
            })
            .with_count(5)
        })
})
}

Resources

For container providers (Docker, Podman, Kubernetes).

[relaychain.default_resources]
request_memory = "1Gi"
request_cpu = "500m"
limit_memory = "2Gi"
limit_cpu = "1000m"

Reference

OptionTypeDescription
request_memoryStringRequested memory (e.g., 512Mi, 1Gi)
request_cpuStringRequested CPU (e.g., 250m, 1)
limit_memoryStringMemory limit
limit_cpuStringCPU limit

HRMP Channels

Cross-chain messaging between parachains.

TOML

[[hrmp_channels]]
sender = 1000
recipient = 2000
max_capacity = 8
max_message_size = 512

[[hrmp_channels]]
sender = 2000
recipient = 1000
max_capacity = 8
max_message_size = 512

Builder

#![allow(unused)]
fn main() {
let config = NetworkConfigBuilder::new()
    .with_relaychain(|r| { /* ... */ })
    .with_parachain(|p| p.with_id(1000).with_collator(|c| c.with_name("col1")))
    .with_parachain(|p| p.with_id(2000).with_collator(|c| c.with_name("col2")))
    .with_hrmp_channel(|h| h.with_sender(1000).with_recipient(2000))
    .with_hrmp_channel(|h| h.with_sender(2000).with_recipient(1000))
    .build()
    .unwrap();
}

Reference

OptionTypeDefaultDescription
senderNumber—Required. Sending parachain ID
recipientNumber—Required. Receiving parachain ID
max_capacityNumber8Maximum messages in channel
max_message_sizeNumber512Maximum message size in bytes

Custom Processes

Allow to set custom processes to spawn after the network is up: services that run next to the nodes, such as an RPC adapter, an IPFS daemon or a database another process needs.

A custom process is a member of the network like a node: it is tracked on the Network, written to zombie.json, attached to again from it and destroyed with the namespace. Look it up with network.get_custom_process("name") or list them with network.custom_processes(). A process may not share its name with a node, since they share one registry: in TOML the process is rejected; in the builder a node declared after a process with its name is renamed, as it would be after another node, so declare processes last.

Unlike nodes, a custom process runs its command directly, not through zombienet's wrapper script: most of them are off-the-shelf images, which may not have bash. On docker and kubernetes the pause/resume/restart controls go through that wrapper, so on a custom process they return an error; on native they signal the process and work as for nodes.

A process must listen on all interfaces (0.0.0.0) on docker and kubernetes, where it is reached through a published port or a Service; one bound to 127.0.0.1 is reachable from nowhere. Most daemons need a flag for that (--rpc-external for eth-rpc below, --host 0.0.0.0 and the like elsewhere).

A process gets the same writable /data directory a node gets, on every provider; that is where kubo above keeps its repo (IPFS_PATH) and where anything that must survive a restart of the process belongs.

command replaces the image's entrypoint, on every provider. An image whose entrypoint does setup work (kubo's runs ipfs init and binds the API and gateway to all interfaces; the postgres image's runs initdb) needs that entrypoint named as the command, with the image's usual arguments as args.

TOML

[[custom_processes]]
name = "eth-rpc"
command = "eth-rpc"
args = [ "--node-rpc-url", "{{ZOMBIE:alice:internal_ws_uri}}", "--rpc-port", "{{ZOMBIE:eth-rpc:port_http}}", "--rpc-external" ]
env = [
    { name = "RUST_LOG", value = "info" }
]
ports = [ { name = "http", port = 0 } ]

[[custom_processes]]
name = "ipfs"
image = "docker.io/ipfs/kubo:v0.39.0"
command = "/usr/local/bin/start_ipfs"
args = [ "daemon" ]
ports = [ { name = "api", port = 5001 }, { name = "gateway", port = 8080 } ]

[custom_processes.resources]
limits = { cpu = "500m", memory = "512Mi" }

Builder

#![allow(unused)]
fn main() {
let config = NetworkConfigBuilder::new()
    .with_relaychain(|r| { /* ... */ })
    .with_custom_process(|c| c.with_name("eth-rpc").with_command("eth-rpc"))
}
#![allow(unused)]
fn main() {
let cp = CustomProcessBuilder::new()
    .with_name("ipfs")
    .with_command("/usr/local/bin/start_ipfs")
    .with_image("docker.io/ipfs/kubo:v0.39.0")
    .with_args(vec!["daemon".into()])
    .with_env(vec![("IPFS_PATH", "/data/ipfs")])
    .with_named_port("api", 5001)
    .with_named_port("gateway", 8080)
    .with_resources(|r| r.with_limit_cpu("500m").with_limit_memory("512Mi"))
    .with_ready_check(ReadyCheck::tcp("api"))
    .with_timeout(60)
    .with_dependency("ipfs-init")
    .build()
    .unwrap();
}

Placeholders

The args and env values of a process may refer to the nodes, which are all running by then, with the same {{ZOMBIE:<node-name>:<field>}} placeholders node args use: ws_uri, internal_ws_uri (the address other members of the network use, which is what a process wants), multiaddr, prometheus_uri for a substrate node, rpc_uri and peer_addr for a JAM node. On docker a node's addresses are host addresses, so a process in a container reaches a node only through the host (host.docker.internal on Docker Desktop). They may also refer to any custom process's ports, their own or another's, as {{ZOMBIE:<process-name>:port_<port-name>}}, since every port is picked before anything starts (see Lifecycle). A placeholder that resolves to nothing (a node or port that does not exist) means the process is never started, which fails the spawn unless the process is allowed to fail (see Lifecycle).

Ports

Each declared port is TCP and exposed by name. A port declared as 0 is picked free by zombienet when the process is spawned. Whatever the number, the process learns it through the placeholder above; eth-rpc is told --rpc-port <picked> that way.

On kubernetes the declared ports are the pod's Service ports, so other pods reach the process as <process-name>:<port>, and a port-forward is opened from the host running zombienet for each (unless running in CI, where the pod's own address is used). The pod's readiness probe is the process's check (see Lifecycle), so the Service routes to the pod once that passes on the pod address, which a process bound to 127.0.0.1 never does (see above). On docker each port is published on a free host port picked at spawn, so two networks can share a host; on native the process listens on them as declared, and a fixed port that is busy fails the spawn.

The running process reports where every port is reachable:

#![allow(unused)]
fn main() {
let ipfs = network.get_custom_process("ipfs")?;
println!("{}", ipfs.get_uri_for_name("api").unwrap()); // from here, e.g. 127.0.0.1:53421 on k8s
let api = ipfs.port("api").unwrap();
println!("{}", api.external); // the same address
println!("{}", api.internal); // from inside the network, e.g. ipfs:5001 on k8s
}

On native external and internal are the same address. On docker external is the published host port and internal the container's own address. On kubernetes external is a port-forward that lives as long as the zombienet process; re-attaching opens new ones, except for a one-shot, which has finished. A port-forward, like docker's published port, accepts connections whether or not the process is listening, so is_responsive() on kubernetes and docker reports the forward, not the process; only on native and on kubernetes in CI does it reach the process itself. Re-attaching takes the processes as recorded in zombie.json, with the same caveat.

Lifecycle

A process is ready when its ready_check passes: tcp = "<port name>", the port accepts a connection, or http = { port = "<port name>", path = "/health" }, a GET answers 2xx, every interval seconds (1 by default). The process's timeout, in seconds, bounds the wait (the global node_spawn_timeout by default); a one-shot is waited on to exit within it, and has failed when it does not. Without a check, a process with ports is ready when its first port accepts a TCP connection, and one without ports is ready once started. On kubernetes the check is the pod's readiness probe, so the kubelet does the checking from inside the cluster and the Service routes only once it passes; on docker and native zombienet probes from where it runs. On docker a published port answers for docker's proxy whether or not the process listens, so a tcp check there, including the default on the first port, passes at once and gives no real ordering; zombienet warns about it at spawn, and http is the check to use there.

A one-shot process (one_shot = true) runs to completion within its timeout: exit 0 is success, anything else a failure. It has no readiness, so no ready_check. On kubernetes a custom process's pod is never restarted by the kubelet, one-shot or not: a process that exits is done, by success or by failure, and its pod says so.

depends_on lists the processes this one starts after, with a condition as docker-compose names them: service_started, service_healthy (passed its check), service_completed_successfully (a one-shot that exited 0). A bare name means the target's natural condition: completed for a one-shot, healthy when it has a check or a port, started otherwise. Every process starts as soon as its dependencies are met, in parallel with whatever else can start; one with no dependencies starts once the nodes are up. A dependency must name a custom process, its condition must fit the target, and there may be no cycle; all three are checked when the config loads.

[[custom_processes]]
name = "migrate"
image = "docker.io/library/postgres:16"
command = "psql"
args = [ "-h", "db", "-f", "/data/schema.sql" ]
one_shot = true
depends_on = ["db"]

[[custom_processes]]
name = "api"
image = "example/api:1.0"
command = "api"
ports = [ { name = "http", port = 8080 } ]
ready_check = { http = { port = "http", path = "/health" } }
timeout = 120
depends_on = [ "migrate", { name = "cache", condition = "service_started" } ]

Every process is required: one that does not meet its condition within its timeout, or is never started (its ports could not be reserved, a dependency did not meet its condition, a placeholder did not resolve), fails the spawn, as a node that never comes up does; the error names the process and the reason, and the network is cleaned up when its provider is dropped (on native the logs stay in base_dir). On a spawned network process.state() is Ready or Completed, written to zombie.json. wait_ready() on a process waits for its condition again and reports Failed { reason } when it is no longer met; the network-wide wait_until_is_up() leaves processes out.

Processes start at most spawn_concurrency at a time, the value as set, where nodes drop to one at a time whenever a node or chain spec has {{ZOMBIE:..}} placeholders. Placeholders in a process do not limit the nodes.

A process may name another's port the way it names its own, {{ZOMBIE:<process>:port_<name>}}, since every port is picked before anything starts. The host part of the address is the other process's name on kubernetes and the host itself on native; on docker the containers have no names for each other, so the placeholder is of little use there.

Reference

OptionTypeDefaultDescription
nameString—Required. Name of the process; also its pod/container name and the key it is looked up by. Lowercase letters, digits and -, starting with a letter, at most 63 characters, unique among nodes and processes
commandString—Required. Command to execute; replaces the image's entrypoint
imageString—Container image
argsArray—CLI arguments; may use the placeholders above
envArray—Environment variables as {name, value} pairs; values may use the same placeholders
portsArray—TCP ports the process listens on, as {name, port} pairs. Names are 1 to 15 lowercase letters, digits or -, with a letter, unique per process; port: 0 is picked at spawn
resourcesTable—CPU/memory requests and limits (kubernetes only; a warning is logged where ignored), as for nodes
ready_checkTablefirst port over TCP{ tcp = "<port>" } or { http = { port = "<port>", path = "..." } }, plus interval in seconds
depends_onArray—Names, or { name, condition } with service_started, service_healthy or service_completed_successfully
one_shotBoolfalseRuns to completion; exit 0 is success. No ready_check
timeoutIntegernode_spawn_timeoutSeconds to wait for the ready check to pass, or the one-shot to exit

Chain Spec Generation

Priority: chain_spec_path > chain_spec_runtime > chain_spec_command > auto-generation.

Pre-existing

chain_spec_path = "/path/to/chain-spec.json"  # or URL

Via Command

chain_spec_command = "polkadot build-spec --chain {chain} --disable-default-bootnode"

Via Runtime

[relaychain.chain_spec_runtime]
src = "/path/to/runtime.wasm"
preset = "development"  # optional

Complete Example

Relay chain with two parachains and HRMP channels.

TOML

[settings]
timeout = 600

[relaychain]
chain = "rococo-local"
default_command = "polkadot"
default_args = ["-lruntime=debug"]

[[relaychain.nodes]]
name = "alice"

[[relaychain.nodes]]
name = "bob"

[[parachains]]
id = 1000

[[parachains.collators]]
name = "collator-1000"
command = "polkadot-parachain"

[[parachains]]
id = 2000

[[parachains.collators]]
name = "collator-2000"
command = "polkadot-parachain"

[[hrmp_channels]]
sender = 1000
recipient = 2000

[[hrmp_channels]]
sender = 2000
recipient = 1000

Builder

use zombienet_sdk::{NetworkConfigBuilder, NetworkConfigExt};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let network = NetworkConfigBuilder::new()
        .with_relaychain(|r| {
            r.with_chain("rococo-local")
                .with_default_command("polkadot")
                .with_validator(|v| v.with_name("alice"))
                .with_validator(|v| v.with_name("bob"))
        })
        .with_parachain(|p| {
            p.with_id(1000)
                .with_collator(|c| c.with_name("collator-1000").with_command("polkadot-parachain"))
        })
        .with_parachain(|p| {
            p.with_id(2000)
                .with_collator(|c| c.with_name("collator-2000").with_command("polkadot-parachain"))
        })
        .with_hrmp_channel(|h| h.with_sender(1000).with_recipient(2000))
        .with_hrmp_channel(|h| h.with_sender(2000).with_recipient(1000))
        .build()
        .unwrap()
        .spawn_native()
        .await?;

    Ok(())
}

Providers

Providers determine how ZombieNet spawns and manages nodes. Choose based on your environment and requirements.

Quick Comparison

ProviderRuns AsImage RequiredResource Limits
NativeLocal processesNoNo
DockerContainersYesNo
KubernetesK8s podsYesYes

Provider Selection

Set the default provider via environment variable:

export ZOMBIE_PROVIDER=native  # or: docker, k8s

Or specify per-command with --provider flag. Default is docker.


Native

Runs nodes directly on your machine as local processes.

Requirements

  • Node binaries in PATH (e.g., polkadot, polkadot-parachain)
  • Or specify absolute paths in configuration

Usage

CLI:

zombie-cli spawn network.toml --provider native

Programmatically:

#![allow(unused)]
fn main() {
let network = config.spawn_native().await?;
}

Notes

  • Fastest startup, no container overhead
  • No container images needed
  • Ports are assigned randomly to avoid conflicts
  • Files stored in a temp directory (or base_dir if configured)

Docker

Runs nodes in Docker containers.

Requirements

  • Docker or Podman installed and running
  • Container images with node binaries

Usage

CLI:

zombie-cli spawn network.toml --provider docker

Programmatically:

#![allow(unused)]
fn main() {
let network = config.spawn_docker().await?;
}

Default Images

Override via environment variables:

VariableDefaultDescription
POLKADOT_IMAGEdocker.io/parity/polkadot:latestRelay chain nodes
CUMULUS_IMAGEdocker.io/parity/polkadot-parachain:latestParachain collators
MALUS_IMAGEdocker.io/paritypr/malus:latestMalus (malicious node)
COL_IMAGEdocker.io/paritypr/colander:latestColander

Labels

Every node container of a network, its scripts container and both of its volumes carry zombienet.namespace (the namespace name), zombienet.kind (node, collator, custom-process, temp and the other zombie-role values for containers, helper for the scripts container, volume for volumes) and zombienet.name. The short-lived --rm containers that set a namespace up are not labelled. Teardown removes the containers that carry the namespace label or the namespace name prefix, then tries to remove its volumes. To list a network's containers: docker ps --filter label=zombienet.namespace=<namespace>.

Kubernetes

Deploys nodes as Kubernetes pods.

Requirements

  • Kubernetes cluster access
  • kubectl configured with valid kubeconfig
  • Container images accessible to the cluster

Usage

CLI:

zombie-cli spawn network.toml --provider k8s

Programmatically:

#![allow(unused)]
fn main() {
let network = config.spawn_k8s().await?;
}

Pod Labels

Node pods carry a zombie-role label with the same values as zombienet v1: node (relay chain nodes, validators or not), collator, cumulus-collator, bootnode (dedicated bootnodes) and temp (short-lived helper pods). Custom processes carry custom-process. The file server has no zombie-role.

Attaching to Running Networks

Reconnect to a previously spawned network, currently running network using the zombie.json state file:

#![allow(unused)]
fn main() {
use zombienet_sdk::{AttachToLive, AttachToLiveNetwork};

// Native
let network = AttachToLiveNetwork::attach_native("/tmp/zombie-1/zombie.json".into()).await?;

// Docker
let network = AttachToLiveNetwork::attach_docker("/tmp/zombie-1/zombie.json".into()).await?;

// Kubernetes
let network = AttachToLiveNetwork::attach_k8s("/tmp/zombie-1/zombie.json".into()).await?;
}

The zombie.json file is written to the network's base directory after spawn completes.

CLI Usage

The zombie-cli command-line tool spawns and manages ZombieNet networks.

Installation

Pre-built binaries:

Download from GitHub Releases.

From source:

cargo install zombie-cli

spawn

Spawn a network from a configuration file.

zombie-cli spawn <CONFIG> [OPTIONS]

Arguments

ArgumentRequiredDescription
CONFIGYesPath to TOML configuration file

Options

OptionShortDefaultDescription
--provider-pdockerProvider: native, docker, or k8s
--dir-dRandom tempBase directory for network files
--spawn-concurrency-c20Concurrent node spawn processes
--node-verifier-vmetricReadiness check: metric or none

Examples

# Spawn with native provider
zombie-cli spawn network.toml --provider native

# Spawn with custom directory
zombie-cli spawn network.toml --dir /tmp/my-network

# Spawn with Docker (default)
zombie-cli spawn network.toml

# Spawn on Kubernetes
zombie-cli spawn network.toml --provider k8s

reproduce

Reproduce CI test runs locally in Docker. Downloads nextest archives from GitHub Actions and runs them in a containerized environment matching CI.

zombie-cli reproduce <REPO> <RUN_ID> [OPTIONS]
zombie-cli reproduce --archive <PATH> [OPTIONS]

Arguments

ArgumentRequiredDescription
REPOYes*Repository name (e.g., zombienet-sdk)
RUN_IDYes*GitHub Actions run ID (find in the URL of a workflow run)

*Not required when using --archive.

Options

OptionShortDescription
--archive-aPath to local nextest archive (.tar.zst)
--artifact-pattern-pFilter artifact downloads by pattern (default: *zombienet-artifacts*)
--skip-sSkip tests matching pattern (repeatable)
[TEST_FILTER]Additional arguments passed to nextest (use -- separator)

Requirements

  • Docker installed and running
  • GitHub CLI (gh) installed and authenticated (for downloading artifacts)
  • POLKADOT_SDK_PATH environment variable pointing to your polkadot-sdk workspace

How It Works

  1. Downloads nextest test archives (.tar.zst) from GitHub Actions
  2. Optionally downloads binary artifacts matching the pattern
  3. Runs tests inside a Docker container with:
    • ZOMBIE_PROVIDER=native (native provider inside container)
    • RUST_LOG=debug for verbose output
    • Mounted polkadot-sdk workspace
  4. Extracts binaries to /tmp/binaries inside the container

Examples

# Reproduce from GitHub Actions (requires gh CLI authentication)
POLKADOT_SDK_PATH=/path/to/polkadot-sdk \
zombie-cli reproduce zombienet-sdk 123456789

# Reproduce from local archive (no gh CLI needed)
POLKADOT_SDK_PATH=/path/to/polkadot-sdk \
zombie-cli reproduce --archive ./my-archive.tar.zst

# Download only artifacts matching "ray" pattern
zombie-cli reproduce zombienet-sdk 123456789 -p ray

# Skip specific slow tests
zombie-cli reproduce zombienet-sdk 123456789 --skip "slow_test"

# Pass filters to nextest
zombie-cli reproduce zombienet-sdk 123456789 -- --test-filter my_test

Nextest Archive Format

The .tar.zst archive is a nextest archive containing compiled test binaries and metadata. These are produced by CI runs using cargo nextest archive.


Environment Variables

VariableCommandDescription
RUST_LOGAllLogging level (e.g., debug, info)
ZOMBIE_SPAWN_CONCURRENCYspawnOverride spawn concurrency
POLKADOT_SDK_PATHreproducePath to polkadot-sdk workspace
POLKADOT_IMAGEspawnOverride Polkadot container image
CUMULUS_IMAGEspawnOverride Cumulus container image

Examples

All examples are in crates/examples/.

Run Rust examples with cargo run --example <example_name>.

Spawn from config files with zombie-cli spawn -p <provider> <config-file>.


Basic Network Setup

Start here to understand network configuration patterns.

ExampleDescription
simple_network_exampleMinimal relay chain with two validators using TOML config
small_network_with_defaultOverride default command and image for all nodes
small_network_with_base_dirCustom base directory for node data
small_network_configMinimal programmatic configuration

Parachain Lifecycle

Configure, register, and manage parachains.

ExampleDescription
small_network_with_paraBasic relay + parachain topology
register_paraRegister parachain via extrinsic
add_paraAdd parachain to running network
para_upgradeRuntime upgrade on running parachain
two_paras_same_idHandling duplicate parachain IDs

Node Groups

Scale networks with grouped nodes.

ExampleDescription
big_network_with_group_nodesProgrammatic network with grouped nodes
network_example_with_group_nodesTOML config with group nodes

Advanced Configuration

ExampleDescription
resource_limitsCPU and memory limits for containers
wasm-overrideCustom WASM runtime override
arg_removalRemove default CLI arguments
db_snapshotDatabase snapshots for faster init
docker_db_snapshotDB snapshots in Docker environments
raw_spec_overrideOverride raw chain spec fields
genesis_patchAppend to and replace runtime genesis values with an RFC 6902 patch

Chain Spec Generation

ExampleDescription
chain_spec_generationDynamic chain spec generation
chain_spec_runtime_kusamaKusama with custom runtime WASM
polkadot_people_wasm_runtimePolkadot + People parachain with custom runtimes
genesis_state_generator_exampleCustom genesis state generator

Keys and Security

ExampleDescription
keystore_key_typesKeystore directories and key type validation
chain_spec_key_typesChain spec session key configuration
evm_parachain_session_keyEVM parachain session keys

Network Utilities

ExampleDescription
from_liveAttach to running network via zombie.json
test_run_scriptRun scripts on nodes
custom_process_portsA custom process with a declared port, reached through the network
custom_process_lifecycleA one-shot, a server with an http ready check that depends on it, a client that waits for the server
custom_process_positional_argsA custom process script told where a node is

Config Files

Example TOML configs are in crates/examples/examples/configs/:

ConfigDescription
simple.tomlBasic two-validator relay chain
simple-group-nodes.tomlRelay chain with node groups
resource_limits.tomlContainer resource limits
wasm-override.tomlWASM runtime override
arg-removal.tomlCLI argument removal
polkadot-ah-chain-spec-runtime.tomlPolkadot + Asset Hub with custom runtimes
custom_process_ports.tomlCustom process with a declared port
custom_process_lifecycle.tomlCustom processes with one_shot, ready_check and depends_on
custom_process_positional_args.tomlCustom processes with positional arguments

Environment Variables

ZombieNet SDK uses environment variables for configuration overrides, container images, and CI integration.

Container Images

Override default images for Docker and Kubernetes providers:

VariableDefaultDescription
POLKADOT_IMAGEdocker.io/parity/polkadot:latestRelay chain node image
CUMULUS_IMAGEdocker.io/parity/polkadot-parachain:latestParachain collator image
MALUS_IMAGEdocker.io/paritypr/malus:latestMalus (malicious node) image
COL_IMAGEdocker.io/paritypr/colander:latestColander image

Example:

POLKADOT_IMAGE=docker.io/parity/polkadot:v1.5.0 zombie-cli spawn network.toml

Provider Selection

VariableValuesDefaultDescription
ZOMBIE_PROVIDERnative, k8s, dockerdockerDefault provider when not specified

Example:

ZOMBIE_PROVIDER=native zombie-cli spawn network.toml

Spawn Control

VariableDescription
ZOMBIE_SPAWN_CONCURRENCYOverride spawn concurrency (default: 20). Set to lower values for resource-constrained environments.
ZOMBIE_NODE_SPAWN_TIMEOUT_SECONDSOverride per-node spawn timeout in seconds

Example:

ZOMBIE_SPAWN_CONCURRENCY=10 zombie-cli spawn network.toml

CLI-Specific

VariableCommandDescription
POLKADOT_SDK_PATHreproducePath to local polkadot-sdk workspace (required for reproduce)
RUST_LOGAllLogging level (debug, info, warn, error)

Other

VariableDescription
ZOMBIE_RM_TGZ_AFTER_EXTRACTRemove tgz archives after extraction (used for db snapshot extraction)

TOML Template Replacement

Environment variables can be referenced in TOML configuration files using {{VAR_NAME}} syntax:

[relaychain]
default_image = "{{POLKADOT_IMAGE}}"

[[relaychain.nodes]]
name = "alice"
image = "{{CUSTOM_NODE_IMAGE}}"

Any environment variable can be substituted. If the variable is not set, the placeholder remains unchanged.


Runtime Token Replacement

For dynamic values from running nodes, use {{ZOMBIE:node_name:field}} syntax in node arguments:

[[relaychain.nodes]]
name = "bob"
args = ["--sync-target", "{{ZOMBIE:alice:multiaddr}}"]

Available fields:

FieldAliasesDescription
multiaddrmultiAddressNode's libp2p multiaddress
ws_uriwsUriWebSocket RPC endpoint
prometheus_uriprometheusUriPrometheus metrics endpoint

When using token replacement, spawn concurrency is automatically set to 1 (serial) to ensure dependencies are resolved correctly.