Operational considerations
#15.1 Versioning and evolution
- Every node and the store metadata key (§10.3) carry
v: 1; every pack header carriesversion: 1(§12.1). Readers of v1 reject unknown node types, unknown fields, and any other version. Any additive change to node or header shapes is therefore a new version, signaled through that field. - Mixed versions in one graph. A reader implementing version N MUST
accept any mix of node versions
v ≤ Nwithin one graph: a node MAY reference nodes of any other accepted version. Version is a property of each node, not of a store or a tree. - Lowest sufficient version. Writers MUST emit, for every node, the
lowest
vwhose rules can represent that node's exact field set. This is a canonical-form rule of the same rank as §5.2. It keeps output deterministic across writers implementing different versions, and it preserves structural sharing across a version bump: only nodes that use new features change bytes (and hashes), while raw blocks, which carry no version at all, never change. - Forward parseability (binding on all future versions). In every
future version, a node remains a definite-length CBOR map whose
vkey is decodable under the rules of §5.3. A pack likewise remains a CAR: a varint-length header frame whose map carries an integerversion(§12.1). The CAR format itself keeps this promise — the CARv2 pragma is a well-formed header frame encoding{version: 2}(§12.5). A v1 reader can therefore always distinguish "future version" (report the foundvorversion) from corruption, for blocks, packs, and store metadata alike. - The test vectors in Appendix A are normative ground truth: a change that moves any vector is a format break and requires a new version.
- Working-draft revisions. This document is a working draft; until it is frozen, a revision MAY change format constants and defaults outright rather than admitting a second version alongside them. This revision did: the FastCDC cut predicate now tests the high bits of the gear hash instead of the low ones, and the default bounds are 8192/32768/98304 (§9.4). The chunker bounds also became recorded parameters (§14, §10.3). Both chunking vectors of Appendix A move accordingly. Blocks written under the previous draft stay readable — the change forks ref identity, it does not invalidate stored data (§14).
#15.2 Garbage collection
Blocks unreachable from the closure of all named refs (plus any roots the operator explicitly supplies) MAY be deleted. Points that follow from the format:
- Liveness MUST be declared. Only named refs (and explicitly supplied roots) count. A root held only in program memory is garbage by definition.
- Reachability is a property of the ref graph, not of node validity. The mark phase MUST discover children by scanning decoded blocks for embedded links (tag 42, §5) — using a stricter semantic validator would let a future validation change silently widen the delete set.
- Marking MUST respect §4: deletion candidacy is per multihash (block
identity, the
blocks:key), but traversal MUST be tracked per full CID — collapsing the two lets a raw-codec sighting of a multihash suppress descending the same block referenced as dag-cbor, sweeping that node's live subtree. - Fail closed. A missing block, an undecodable block, an undecodable
ref value, or a ref whose codec this implementation does not support
(§4) encountered during marking MUST abort the sweep: a missing or
opaque node means its children are unknown while possibly still
present. Operational consequence, intentional: a single foreign-codec
link (say,
dag-pb) anywhere in the live graph disables garbage collection until the link is removed or the implementation learns the codec. The shape of the subgraph behind it is unknown, so no sweep over it can be safe. - Exclusive access. A collector MUST have exclusive write access to the store for the full mark-and-sweep duration. Any concurrent writer's blocks — including a single one's — can be swept in the window between storing them and publishing a root that references them, and the failure is silent and delayed: a later-published root points at swept blocks. This is the only operation in this specification requiring exclusivity; it is a property of tracing collection over a store without transactions, not of the format.
- Raw blocks never need fetching during mark (they have no children).
- A sweep interrupted partway is safe to abandon or restart: it leaves garbage behind, never removes live data.
#15.3 Concurrency
The format does not require a single writer. Per operation:
- Block writes are safe under any number of concurrent writers: content addressing makes racing stores of the same bytes idempotent, and bottom-up construction means a published root never references a block that was not stored first. Two writers importing near-identical content dedupe against each other for free.
- Named-ref updates to the same name are last-write-wins: a lost race loses one binding but corrupts nothing. Deployments that need lost-update protection can layer a compare-and-swap over the store's ref keys; coordination protocols are out of scope (§1.4).
- Reads are always safe concurrently with writes: blocks are immutable, and a reader holding a root ref sees a consistent snapshot regardless of ongoing writes.
- Garbage collection is the sole exception and requires exclusive access (§15.2).