IPFS
The blocks unfs writes are already IPFS blocks.
Refs are CIDv1, nodes are DAG-CBOR with ordinary tag-42 links, chunks are raw, and packs are CARv1. None of that is a compatibility layer bolted on; they're just the formats unfs uses anyway. Two useful things fall out of it:
- Share a directory with kubo — kubo serves a unfs store's blocks straight off disk.
- Move trees as CAR files — any tree travels as a single file, in either direction.
Everything on this page was checked against kubo 0.40.0.
#What IPFS can and can't see
Worth being precise about the boundary, because it saves an afternoon:
- Blocks: everything works. A unfs block key is base32 of the very multihash kubo's blockstore is keyed by, so there's no translation to get wrong.
- DAGs: everything works. Links are tag-42 CIDs, so
ipfs dag get,ipfs dag export,ipfs refs -r, pinning, and bitswap all traverse unfs trees correctly. - Files: nothing works. A unfs directory or file node uses its own DAG-CBOR schema rather than a UnixFS protobuf, so
ipfs catfails withError: unknown node typeandipfs lscan't list a directory.
IPFS is a correct block and DAG host for unfs data, then, but not a file server for it. If you want files over HTTP, use the remote store instead.
#Share a directory with kubo
The on-disk store shards blocks exactly the way go-ds-flatfs does, by the two next-to-last base32 characters, across 1024 directories. Set naming: "flatfs" to match the leaf spelling too (upper case, .data suffix), and a kubo repo and a unfs store become the same directory:
import { createFs, openStore } from "unfs";
import { fsStore } from "unfs/fs";
// A repo `ipfs init` already created.
const fs = createFs(await openStore(fsStore("~/.ipfs", { naming: "flatfs" })));
await fs.mkdir("/notes", { recursive: true });
await fs.writeFile("/notes/hello.txt", "hello from unfs\n");
const root = await fs.head(); // bafyreihdjq665v…knt3eunfs also writes the blocks/SHARDING marker flatfs expects, before the first block lands. If that marker already names a different shard function, the write throws instead of scattering blocks kubo would never look for. Its meta/ and refs/ directories sit beside blocks/, where flatfs never looks, so kubo ignores them and ipfs sees a repo it fully understands:
$ ipfs dag get bafyreihdjq665v…knt3e
{"e":[["big.bin",{"r":{"/":"bafyreig7a35whr…zgiy"},"sz":400000,"t":"f"}],
["notes",{"r":{"/":"bafyreifv2vdrmn…tone"},"t":"d"}]],"t":"d","v":1}
$ ipfs refs -r bafyreihdjq665v…knt3e # the whole tree, every block found on disk
$ ipfs dag export bafyreihdjq665v…knt3e > tree.carThere's no import step and no reindex. flatfs keeps no index, since the file name is the key, so a block unfs writes is servable the instant it lands, even with the daemon already running. Write a file and the new root answers on the gateway immediately.
It works in the other direction too, because the mapping is the same function both ways:
import { blockKey, parseRef } from "unfs";
const ref = parseRef("bafkreicrqrz34x…cwsa"); // something `ipfs add` produced
await kv.get(blockKey(ref.multihash)); // → those bytes, read out of kubo's repo#Over the gateway
The gateway addresses things by CID, not by path, and serves blocks and CARs rather than files:
| request | result |
|---|---|
GET /ipfs/<cid> | the block, Content-Type: application/vnd.ipld.dag-cbor |
?format=raw | the same bytes as application/vnd.ipld.raw |
?format=car | the whole subtree as CARv1 — feed it straight to importPack |
?format=car&dag-scope=block | just that one block, in a CAR |
?format=dag-json | fails — kubo does no codec conversion; fetch ?format=raw and decode it |
GET /ipfs/<cid>/notes | fails — no link named "notes"; see below |
That last one is the one that catches people out. A unfs directory keeps its entries in an array (e), so notes isn't a map key the gateway can step into. IPLD paths still work through the CLI and API, they just spell the position rather than the name:
$ ipfs dag get bafyreihdjq665v…knt3e/e/1/1/r # → the /notes directory node#Between two nodes
Bitswap works, and it works on the whole tree. Connect a second kubo node holding none of it, ask for the root, and every block comes across. unfs on that side then reads the files as if it had written them. The naming isn't what makes this work: a node you filled with ipfs dag import behaves the same.
What IPFS won't carry is which root is current. refs/head.ref is unfs's mutable pointer, and kubo has no equivalent, so the receiving side has to learn the new CID from somewhere else: IPNS, pubsub, a queue, an endpoint of your own. Bitswap also fetches a CID at a time, so a deep tree costs a round trip per level and holes only surface as you walk. A pack moves the whole verified closure in one stream.
So IPFS gives you block transport, plus peer discovery, NAT traversal and content routing you didn't have to build. That's worth a lot if you want strangers to fetch a tree. It isn't sync. Between nodes you control, syncPull / syncPush already works out what's missing, checks the closure exactly, and moves refs fast-forward-only with conflicts reported, over any socket and with no daemon.
Caution
Two garbage collectors, two ideas of what's live. refs/head.ref means nothing to kubo, and kubo's pinset means nothing to unfs. Both were measured doing damage: ipfs repo gc swept 12 of 14 blocks, the entire unfs tree, keeping only what kubo had pinned, and gc(store) swept a block ipfs add had pinned, because no unfs ref reaches it.
On a shared directory, either ipfs pin add the unfs root (and bind anything you ipfs add to a unfs ref), or simply don't run either collector.
Writers aren't coordinated either. unfs writes block files directly and never takes kubo's repo lock. Concurrent block writes are safe — the same bytes under the same name, and each one lands with a rename, so a daemon reading the directory never catches a half-written block — but two processes moving one ref race as always.
#Move trees as CAR files
A pack is a CARv1 file, which is already what ipfs dag import wants:
const pack = await fs.exportPackBytes("/docs"); // CARv1 bytes
await writeFile("docs.car", pack);$ ipfs dag import docs.car
Pinned root bafyreiafxawarp…64ija successThe root comes back pinned, and from there it's ordinary IPFS data: pin it, dag export it, serve it, or hand the same CAR to any pinning service that takes CAR uploads.
Anything IPFS can export comes back in too, whether it came from the CLI or a gateway's ?format=car:
const root = await other.importPack(await readFile("from-kubo.car"));
await other.readFile("/notes/hello.txt", "utf8"); // "packed by unfs\n"Every imported block is hash-verified and lands before any name moves, so a bad CAR fails without touching your tree.
Two details worth knowing:
- The bytes tend to match exactly. For the trees checked here,
ipfs dag export <root>andfs.exportPackBytes("/")produced byte-identical files, as did the gateway's?format=car: same CARv1 header, same depth-first block order. It's a good sign the two implementations agree, but CARv1 fixes no block order, so treat it as an observation rather than a contract. - A UnixFS CAR won't import.
ipfs addproducesdag-pbnodes, andimportPackrejects the codec withENOSYSrather than pretending to understand them. CARs of unfs trees round-trip; CARs of arbitrary IPFS content don't. Converting a UnixFS tree into a unfs one means walking it and writing the files.
#Next steps
- The on-disk store — the layout,
naming: "flatfs", and what a store directory promises. - Packs & Sync — packs in general, plus store-to-store sync when both ends speak unfs.
- The spec — the node schema, if you want to decode unfs blocks with IPLD tooling of your own.