Conformance
#13.1 Writer conformance and determinism
The determinism contract: same logical input + same parameters (§14) ⇒ byte-identical blocks and identical refs, regardless of
- how file content was sliced into input pieces,
- the order directory entries were supplied,
- internal concurrency or completion timing.
To achieve it, writers MUST:
- emit only canonical CBOR (§5.2);
- assign codecs canonically (§4): every chunk block is referenced with
codec
raw, every node block with codecdag-cbor. The codec is part of ref identity, so a swapped codec forks the ref of otherwise identical content. (IPLD at large has no canonical CID for given content; within this format, canonicality is exactly this profile — see §14); - omit absent fields entirely; never write defaults (§7.2), and omit
mtnwhen 0; - sort entries by name order and reject duplicate names by UTF-8 byte identity (§3.3);
- follow the flat/paged decision and the canonical content-defined segmentation exactly (§8.1–8.2) — directory layout is a pure function of the entry set, whether built in one shot or reached by edits (§8.4);
- follow the file-build algorithm exactly (§9.1), including the held-first- chunk rule and the balanced-tree grouping;
- when chunking with
fastcdc, follow §9.4 exactly — the GEAR table, the high-bit cut predicate, sub-minimum skipping, and the lookahead discipline that keeps boundaries independent of input slicing — under the recordedchunkMin/chunkAvg/chunkMaxbounds (§14); - keep chunk pair order equal to chunk order even when hashing/storing concurrently.
Writers MUST NOT emit blocks that violate any validation rule in this document (a conforming reader never needs leniency to accept a conforming writer's output).
#13.2 Reader conformance (hardening)
All input is untrusted (§1.3). A conforming reader MUST enforce:
- Strict varints (§3.1) and strict canonical CBOR (§5.3).
- Pack limits (§12.4): the CAR header-frame and section-size window, enforced the instant each length varint decodes — never allocating from a declared length alone.
- Block verification on fetch: a reader operating on a store it does not exclusively trust SHOULD verify each fetched block's bytes against the hash in its key before use; verification MAY be skipped only for blocks the same implementation previously wrote or hash-verified. (Pack import always verifies unless explicitly disabled, §12.4.)
- Strict node and entry validation (§6–§7): types, field sets, ranges,
sorted-ness, duplicate names, page-index shape,
firstNameequality, cross-segment ordering, no interior boundary names (§8.3). - Ref validation (§4): CIDv1 only (CIDv0 rejected), codec ∈ {raw,
dag-cbor}, the store's multihash with digest length in [20, 64], no
identity multihash, minimal varints — enforced across all three forms
(binary, tag-42 link,
b…string). Decoded refs MUST NOT alias input buffers. - Size consistency (§9.1): leaf block length == declared size;
"f"nodesz== referencing size == Σ own pairs; inline length ==sz; empty file ⇔ neitherdnorr. - Depth and cycle safety: CBOR nesting ≤ 64;
"f"-tree depth ≤ 64; directory paging single-level by construction. Graph traversals MUST be iterative or otherwise safe on arbitrarily deep DAGs (a hostile store can present any depth), and MUST carry a visited set keyed by the full CID (§4) so cycles terminate. - Work bounds on fan-out: a depth cap alone does not bound work on a non-cyclic diamond/fan DAG — a handful of tiny blocks sharing children can declare enormous sizes and induce a fetch storm. Readers SHOULD bound repeated fetch/decode of the same blocks within one operation (e.g. a small per-operation cache or visit budget).
- No aliasing: bytes handed onward as stored or returned data MUST NOT alias transient parsing buffers or caller-owned input.
- Name/path validation on both read and write paths (§6, §11).
- Actionable errors: malformed input is rejected with a diagnosable error; it is never silently coerced, truncated, or defaulted. Error classification by consumers MUST NOT rely on matching message prose: names, paths, and symlink targets appear in messages unescaped, so hostile content could choose the deciding substring. Use structured error discrimination.