Crate polkadot_node_core_pvf

The PVF validation host. Responsible for coordinating preparation and execution of PVFs.

For more background, refer to the Implementer’s Guide: PVF Pre-checking, Candidate Validation and PVF Host and Workers.

Entrypoint

This crate provides a simple API. You first start the validation host, which gives you the handle and the future you need to poll.

Then using the handle the client can send three types of requests:

(a) PVF pre-checking. This takes the Pvf code and tries to prepare it (verify and compile) in order to pre-check its validity.

(b) PVF execution. This accepts the PVF params and the Pvf code, prepares (verifies and compiles) the code, and then executes PVF with the params.

(c) Heads up. This request allows to signal that the given PVF may be needed soon and that it should be prepared for execution.

The preparation results are cached for some time after they either used or was signaled in heads up. All requests that depends on preparation of the same PVF are bundled together and will be executed as soon as the artifact is prepared.

Priority

PVF execution requests can specify the priority with which the given request should be handled. Different priority levels have different effects. This is discussed below.

Preparation started by a heads up signal always starts with the background priority. If there is already a request for that PVF preparation under way the priority is inherited. If after heads up, a new PVF execution request comes in with a higher priority, then the original task’s priority will be adjusted to match the new one if it’s larger.

Priority can never go down, only up.

Under the hood

The flow

Under the hood, the validation host is built using a bunch of communicating processes, not dissimilar to actors. Each of such “processes” is a future task that contains an event loop that processes incoming messages, potentially delegating sub-tasks to other “processes”.

Two of these processes are queues. The first one is for preparation jobs and the second one is for execution. Both of the queues are backed by separate pools of workers of different kind.

Preparation workers handle preparation requests by prevalidating and instrumenting PVF wasm code, and then passing it into the compiler, to prepare the artifact.

Artifacts

An artifact is the final product of preparation. If the preparation succeeded, then the artifact will contain the compiled code usable for quick execution by a worker later on. If the preparation failed, then no artifact is created.

The artifact is saved on disk and is also tracked by an in memory table. This in memory table doesn’t contain the artifact contents though, only a flag for the state of the given artifact and some associated data. If the artifact failed to process, this also includes the error.

A pruning task will run at a fixed interval of time. This task will remove all artifacts that weren’t used or received a heads up signal for a while.

Execution

The execute workers will be fed by the requests from the execution queue, which is basically a combination of a path to the compiled artifact and the params.

Modules

• testing
  Various utilities for testing.

Structs

• Config
  Configuration for the validation host.
• Metrics
  Validation host metrics.
• PrepareStats
  Preparation statistics, including the CPU time and memory taken.
• PvfPrepData
  A struct that carries the exhaustive set of data to prepare an artifact out of plain Wasm binary
• SecurityStatus
  Status of security features on the current system.
• ValidationHost
  A handle to the async process serving the validation host requests.

Enums

• InternalValidationError
  Some internal error occurred.
• InvalidCandidate
  A description of an error raised during executing a PVF and can be attributed to the combination of the candidate polkadot_parachain_primitives::primitives::ValidationParams and the PVF.
• PossiblyInvalidError
  Possibly transient issue that may resolve after retries.
• PrepareError
  An error that occurred during the prepare part of the PVF pipeline.
• PrepareJobKind
  The kind of prepare job.
• Priority
  A priority assigned to preparation of a PVF.
• ValidationError
  A error raised during validation of the candidate.

Constants

• EXECUTE_BINARY_NAME
  The name of binary spawned to execute a PVF
• HOST_MESSAGE_QUEUE_SIZE
  The size of incoming message queue
• JOB_TIMEOUT_WALL_CLOCK_FACTOR
  A multiple of the job timeout (in CPU time) for which we are willing to wait on the host (in wall clock time). This is lenient because CPU time may go slower than wall clock time.
• LOG_TARGET
  The log target for this crate.
• PREPARE_BINARY_NAME
  The name of binary spawned to prepare a PVF artifact

Functions

• framed_recv
  Read some data prefixed by its length from r.
• framed_send
  Write some data prefixed by its length into w.
• get_worker_version
  Utility to get the version of a worker, used for version checks.
• start
  Start the validation host.