Skip to main content

Module db_snapshot

Module db_snapshot 

Source
Expand description

Resolve db_snapshot AssetLocations into local cache paths, once per unique location, before the parallel spawn fanout.

Cache layout is: {ns_base_dir}/{sha256(loc_str)}.tgz. The provider’s initialize_db_snapshot takes this already-resolved file and only has to extract it.

The exception is a URL for a provider whose fileserver can fetch it itself (ProviderCapabilities::fileserver_fetches_db_snapshot_url, k8s). The fileserver downloads it once, under the same sha256(loc_str).tgz name, and every node curls it from there as it would an uploaded one. Downloading it here instead would hold the whole snapshot in this host’s memory and then upload it to that same fileserver — for every distinct snapshot, gigabytes each for a forked public network.

Functions§

cleanup_db_snapshot_cache
Remove cache tarballs after all consumers have finished extracting them, gated by the ZOMBIE_RM_TGZ_AFTER_EXTRACT env var. Best-effort: errors are logged and swallowed (cleanup must never fail spawn).
resolve_db_snapshots
Walk every node’s db_snapshot, deduplicate by AssetLocation, and fetch each unique location once: into the namespace cache, or into the namespace’s fileserver (see the module docs). Returns a map from the original AssetLocation to what the provider is handed for it.

Type Aliases§

ResolvedDbSnapshots
Lookup map produced by resolve_db_snapshots.