Conformance

#13.1 Writer conformance and determinism

The determinism contract: same logical input + same parameters (§14) ⇒ byte-identical blocks and identical refs, regardless of

  • how file content was sliced into input pieces,
  • the order directory entries were supplied,
  • internal concurrency or completion timing.

To achieve it, writers MUST:

  • emit only canonical CBOR (§5.2);
  • assign codecs canonically (§4): every chunk block is referenced with codec raw, every node block with codec dag-cbor. The codec is part of ref identity, so a swapped codec forks the ref of otherwise identical content. (IPLD at large has no canonical CID for given content; within this format, canonicality is exactly this profile — see §14);
  • omit absent fields entirely; never write defaults (§7.2), and omit mtn when 0;
  • sort entries by name order and reject duplicate names by UTF-8 byte identity (§3.3);
  • follow the flat/paged decision and the canonical content-defined segmentation exactly (§8.1–8.2) — directory layout is a pure function of the entry set, whether built in one shot or reached by edits (§8.4);
  • follow the file-build algorithm exactly (§9.1), including the held-first- chunk rule and the balanced-tree grouping;
  • when chunking with fastcdc, follow §9.4 exactly — the GEAR table, the high-bit cut predicate, sub-minimum skipping, and the lookahead discipline that keeps boundaries independent of input slicing — under the recorded chunkMin/chunkAvg/chunkMax bounds (§14);
  • keep chunk pair order equal to chunk order even when hashing/storing concurrently.

Writers MUST NOT emit blocks that violate any validation rule in this document (a conforming reader never needs leniency to accept a conforming writer's output).

#13.2 Reader conformance (hardening)

All input is untrusted (§1.3). A conforming reader MUST enforce:

  • Strict varints (§3.1) and strict canonical CBOR (§5.3).
  • Pack limits (§12.4): the CAR header-frame and section-size window, enforced the instant each length varint decodes — never allocating from a declared length alone.
  • Block verification on fetch: a reader operating on a store it does not exclusively trust SHOULD verify each fetched block's bytes against the hash in its key before use; verification MAY be skipped only for blocks the same implementation previously wrote or hash-verified. (Pack import always verifies unless explicitly disabled, §12.4.)
  • Strict node and entry validation (§6§7): types, field sets, ranges, sorted-ness, duplicate names, page-index shape, firstName equality, cross-segment ordering, no interior boundary names (§8.3).
  • Ref validation (§4): CIDv1 only (CIDv0 rejected), codec ∈ {raw, dag-cbor}, the store's multihash with digest length in [20, 64], no identity multihash, minimal varints — enforced across all three forms (binary, tag-42 link, b… string). Decoded refs MUST NOT alias input buffers.
  • Size consistency (§9.1): leaf block length == declared size; "f" node sz == referencing size == Σ own pairs; inline length == sz; empty file ⇔ neither d nor r.
  • Depth and cycle safety: CBOR nesting ≤ 64; "f"-tree depth ≤ 64; directory paging single-level by construction. Graph traversals MUST be iterative or otherwise safe on arbitrarily deep DAGs (a hostile store can present any depth), and MUST carry a visited set keyed by the full CID (§4) so cycles terminate.
  • Work bounds on fan-out: a depth cap alone does not bound work on a non-cyclic diamond/fan DAG — a handful of tiny blocks sharing children can declare enormous sizes and induce a fetch storm. Readers SHOULD bound repeated fetch/decode of the same blocks within one operation (e.g. a small per-operation cache or visit budget).
  • No aliasing: bytes handed onward as stored or returned data MUST NOT alias transient parsing buffers or caller-owned input.
  • Name/path validation on both read and write paths (§6, §11).
  • Actionable errors: malformed input is rejected with a diagnosable error; it is never silently coerced, truncated, or defaulted. Error classification by consumers MUST NOT rely on matching message prose: names, paths, and symlink targets appear in messages unescaped, so hostile content could choose the deciding substring. Use structured error discrimination.

unfs  A filesystem you can put in any key–value store.