On-Disk Store
Keep a unfs filesystem in a directory, with nothing to install.
memoryStore() is the backend you reach for first and lose on exit. fsStore() is the one that survives it. It ships in the box as a separate entry point, import ... from "unfs/fs", and needs only node:fs, so Node 22.3+, Deno, and Bun all serve it while importing unfs itself stays runtime-agnostic.
import { createFs, initStore } from "unfs";
import { fsStore } from "unfs/fs";
const fs = createFs(await initStore(fsStore(".data/store")));
await fs.mkdir("/notes", { recursive: true });
await fs.writeFile("/notes/hello.txt", "hello from disk\n");
await fs.head(); // bafyrei… — the root CID of what's now on diskReopen it later, from another process or after a restart, and it's all still there:
import { createFs, openStore } from "unfs";
import { fsStore } from "unfs/fs";
const fs = createFs(await openStore(fsStore(".data/store")));
await fs.readFile("/notes/hello.txt", "utf8"); // "hello from disk\n"The base can be a path or a file: URL, and the directory is created on first write.
Note
This is a KVStore, not a mirror. The directory holds the block graph, not your files as files: /notes/hello.txt is a leaf of a Merkle DAG, not .data/store/notes/hello.txt. To move a tree in or out as ordinary files, use a pack.
#Using it synchronously
fsStoreSync() is the same store with every reply a plain value, which is what createFsSync needs.
Before it existed, memoryStore() was the only synchronous driver, so anything requiring synchronous filesystem access couldn't outlive its process. That covers just-bash glob expansion and @platformatic/vfs. Now it can:
import { createFsSync, initStore } from "unfs";
import { fsStoreSync } from "unfs/fs";
const fs = createFsSync(await initStore(fsStoreSync(".data/store")));
fs.writeFileSync("/hello.txt", "no await anywhere\n");
fs.readFileSync("/hello.txt", "utf8");Both stores share one layout, so you can open the same directory either way. fsStoreSync blocks the event loop, which is the trade the synchronous facade already makes, so prefer fsStore wherever you can await.
#The layout
The three key spaces of §10 become three directories, chosen so a store is something you can ls:
| key | file |
|---|---|
meta:format | meta/format |
blocks:<base32> | blocks/<next-to-last 2>/<base32> |
refs:head | refs/head.ref |
refs:npm:h3:latest | refs/npm/h3/latest.ref |
- Blocks shard on two characters of their base32 key, giving 1024 directories, so a large tree never lands millions of entries in one of them. They're the next-to-last two rather than the first: a block key is base32 of a multihash, so every sha2-256 key starts with the same
ciqheader and only the tail spreads evenly. - Ref names nest on
:, the separator every ref name already uses. The.refsuffix is what keepsnpm:h3andnpm:h3:latestfrom fighting over one path. - Everything else percent-encodes. A name segment keeps
[A-Za-z0-9._-]and escapes the rest, plus a leading.. So..is an ordinary file, a Windows-hostile character never reaches the syscall, and two ref names can never collide on one file.
.tmp/ is where a name write stages its file before renaming it into place. Nothing else in the directory is read, so a README.md you drop in refs/ is ignored rather than mistaken for a ref.
#Sharing the directory with IPFS
const fs = createFs(await initStore(fsStore("~/.ipfs", { naming: "flatfs" })));naming: "flatfs" spells block files blocks/P5/CIQA….data, upper case with a .data suffix, which is what go-ds-flatfs writes. The same directory is then also a kubo blockstore.
Nothing clever is going on. A unfs block key already is base32 of the multihash kubo's blockstore has been keyed by since 0.5, and both shard on the same two characters, so the casing and the suffix are the entire difference. The store also writes the blocks/SHARDING marker flatfs expects, before the first block lands. If that marker already names a different shard function, the write throws instead of scattering blocks kubo would never look for.
Point kubo at it (run ipfs init first, so the repo has its config) and ipfs block get, ipfs dag get, ipfs dag export, and bitswap all work on unfs blocks, because unfs links are ordinary tag-42 CIDs. ipfs cat and ipfs ls don't: a unfs node uses its own DAG-CBOR schema rather than UnixFS. Pick the naming when you create the directory, since the two spellings ignore each other's block files.
The IPFS guide covers the rest: what the gateway serves, CAR import and export in both directions, and the two garbage collectors you now have.
#What it promises
- Every write is atomic. Blocks, refs, and metadata all land through a temp file plus a
rename, so a reader sees the whole old value or the whole new one — never a truncated CID, never a half-written block. That holds across processes, which is what makes the directory safe to read while something else is writing it. - Every optional capability is there.
has,delete, andkeysall work, so packs,listRefs, andgcdo too — plussetIfMatchanddeleteIfMatch, the conditional write and the conditional delete that make concurrent writers safe rather than merely non-corrupting. See Multiple writers. - The directory is yours. Copy it,
rsyncit,tarit, check it into a build cache. trusted: trueis usually right here. A private directory this process owns is exactly the case that option exists for, and it skips the read-time re-hash. See Performance.
const store = await openStore(fsStore(".data/store"), { trusted: true });Three caveats before it holds something you care about:
- A plain ref update is last-write-wins. Atomic writes keep two processes from corrupting each other, but they don't order them: two processes that each read the root, write a file, and commit will race, and the loser's changes are gone — not corrupted, just dropped.
createFs/createFsSyncclose that by default over this store, and a hand-rolled commit still needs compare-and-set. - Atomic isn't durable. Nothing here calls
fsync, so a power loss can still lose a write the filesystem said it took, and a crash can leave a stray file in.tmp/(harmless, and safe to delete when nothing is writing). - A crash mid-commit leaves a lock behind. Because
createFsnow commits through the lock file, a writer killed while holdingrefs/head.ref.lockwedges that ref: every later write isEBUSYuntil someone who knows no writer is running deletes it. Nothing breaks a lock on a timer, deliberately — the alternative is shooting a live writer. Pass{ atomic: false }if you'd rather have the last-write-wins exposure instead. - A case-insensitive filesystem folds ref names. On the default macOS and Windows volumes,
refs:Headandrefs:headare one file.
#Multiple writers
Two processes writing blocks was always safe — same bytes, same key, and each block lands with a rename — but moving the same ref is a race, and the loser's whole commit disappears without an error.
Over this store the Unfs facade handles that for you, and has since it grew compare-and-set commits. Every mutation repoints the ref only if it still holds the value the mutation started from, and one that lost the race re-runs itself against the winner's tree:
// Two processes, one directory, no coordination. Neither loses the other's work.
const fs = createFs(await openStore(fsStore(".data/store")));
await fs.writeFile("/notes/mine.txt", "safe");See Multiple writers for the full behaviour — what EAGAIN means, why batch reports instead of retrying, and the one case it can't cover.
The rest of this section is the manual form, which is still what you want for a commit the facade doesn't make: grafting a ref you built yourself, or a mirror bind. setRefIfMatch is the primitive — commit only if the ref still holds what you read, and re-read if it doesn't.
import { getRef, setRefIfMatch } from "unfs";
for (;;) {
const head = await getRef(store, "head");
const next = await buildFrom(head); // read the tree, write your change
if (await setRefIfMatch(store, "head", head, next)) break;
// Someone else committed first. Their tree is the new base — start over.
}falsemeans the ref moved, so the tree you built is based on a root nobody has. Re-read and rebuild; don't retry the same commit.undefinedas the expected value means "only if nothing is bound yet" — a create-only write.- Under the hood it's a
git-style lock file (refs/head.ref.lock, created withO_CREAT | O_EXCL) held across the read, the compare, and the swap. Contention is retried for about a second; past that you get anUnfsErrorwith codeBUSYnaming the file, because at that point it's more likely a leftover from a crashed writer than a queue. Nothing ever deletes someone else's lock on a timer — that's your call, once you know no writer is running. - The lock is advisory: it only stops other conditional writers. A plain
setRefwalks straight past it, so pick one discipline per store. deleteRefIfMatch(store, name, expected)is the same thing for unbinding a name: it takes the same lock, compares the same way, and unbinds only if the ref still holds what you read. Deleting a name is the one update whose result is nothing, so reading first and then callingdeleteRefwould silently drop anyone else's commit that landed in between — this is what closes that. It's whatunfs/gitrunsgit push origin :branchon.
Four processes each running fifty read-modify-write commits over one ref land all two hundred through setRefIfMatch, and about sixty through plain setRef.
If you want a fast layer in front instead, see Layered Stores — which assumes a single writer, and deliberately offers neither conditional write, so a filesystem built on one falls back to last-write-wins.
#Why it loads anywhere
The store imports nothing, not even node:fs. Its builtins come from process.getBuiltinModule at load time, so the published bundle has no imports at all. A browser bundler that follows this entry point resolves nothing and pulls in no fs polyfill, and the module loads fine anywhere.
Somewhere without those builtins, fsStore() throws UNSUPPORTED with a message pointing you at memoryStore(), so you get a clear failure at construction rather than a broken import.
#Disk space
One file per key is what makes this store browsable, and it is charged in whole filesystem blocks. A ref value is 36 bytes and occupies a full block — 4 KiB on a typical Linux filesystem — so a store whose keys are mostly small values can occupy several times the data it holds. Measured on a hosted Git repository, this store allocated between 1.9× and 7.2× its data, where the same keys in unfs/sqlite or unfs/lmdb took 1.1× to 1.5×.
Large files are barely affected — a block of file content is a real file of real size. If your store is dominated by small values and disk space matters, prefer a single-file store.
#When to reach for something else
| you want | use |
|---|---|
| a directory, no dependencies | unfs/fs |
| one file, no dependencies | unfs/sqlite |
| the least disk space for many tiny keys | unfs/sqlite |
| several processes writing at once | unfs/lmdb |
| Redis, S3, Cloudflare KV, IndexedDB, … | unfs/unstorage |
| a store someone else hosts | remote stores |
| nothing on disk at all | memoryStore() from unfs |
unfs/unstorage also reaches the local disk, through its own fs and fs-lite drivers. Use it when the disk is one option among several you might swap between, and unfs/fs when the disk is the answer and you'd rather not install anything.