On-Disk Store

Keep a unfs filesystem in a directory, with nothing to install.

memoryStore() is the backend you reach for first and lose on exit. fsStore() is the one that survives it. It ships in the box as a separate entry point, import ... from "unfs/fs", and needs only node:fs, so Node 22.3+, Deno, and Bun all serve it while importing unfs itself stays runtime-agnostic.

import { createFs, initStore } from "unfs";
import { fsStore } from "unfs/fs";

const fs = createFs(await initStore(fsStore(".data/store")));

await fs.mkdir("/notes", { recursive: true });
await fs.writeFile("/notes/hello.txt", "hello from disk\n");

await fs.head(); // bafyrei… — the root CID of what's now on disk

Reopen it later, from another process or after a restart, and it's all still there:

import { createFs, openStore } from "unfs";
import { fsStore } from "unfs/fs";

const fs = createFs(await openStore(fsStore(".data/store")));
await fs.readFile("/notes/hello.txt", "utf8"); // "hello from disk\n"

The base can be a path or a file: URL, and the directory is created on first write.

Note

This is a KVStore, not a mirror. The directory holds the block graph, not your files as files: /notes/hello.txt is a leaf of a Merkle DAG, not .data/store/notes/hello.txt. To move a tree in or out as ordinary files, use a pack.

#Using it synchronously

fsStoreSync() is the same store with every reply a plain value, which is what createFsSync needs.

Before it existed, memoryStore() was the only synchronous driver, so anything requiring synchronous filesystem access couldn't outlive its process. That covers just-bash glob expansion and @platformatic/vfs. Now it can:

import { createFsSync, initStore } from "unfs";
import { fsStoreSync } from "unfs/fs";

const fs = createFsSync(await initStore(fsStoreSync(".data/store")));

fs.writeFileSync("/hello.txt", "no await anywhere\n");
fs.readFileSync("/hello.txt", "utf8");

Both stores share one layout, so you can open the same directory either way. fsStoreSync blocks the event loop, which is the trade the synchronous facade already makes, so prefer fsStore wherever you can await.

#The layout

The three key spaces of §10 become three directories, chosen so a store is something you can ls:

keyfile
meta:formatmeta/format
blocks:<base32>blocks/<next-to-last 2>/<base32>
refs:headrefs/head.ref
refs:npm:h3:latestrefs/npm/h3/latest.ref
  • Blocks shard on two characters of their base32 key, giving 1024 directories, so a large tree never lands millions of entries in one of them. They're the next-to-last two rather than the first: a block key is base32 of a multihash, so every sha2-256 key starts with the same ciq header and only the tail spreads evenly.
  • Ref names nest on :, the separator every ref name already uses. The .ref suffix is what keeps npm:h3 and npm:h3:latest from fighting over one path.
  • Everything else percent-encodes. A name segment keeps [A-Za-z0-9._-] and escapes the rest, plus a leading .. So .. is an ordinary file, a Windows-hostile character never reaches the syscall, and two ref names can never collide on one file.

.tmp/ is where a name write stages its file before renaming it into place. Nothing else in the directory is read, so a README.md you drop in refs/ is ignored rather than mistaken for a ref.

#Sharing the directory with IPFS

const fs = createFs(await initStore(fsStore("~/.ipfs", { naming: "flatfs" })));

naming: "flatfs" spells block files blocks/P5/CIQA….data, upper case with a .data suffix, which is what go-ds-flatfs writes. The same directory is then also a kubo blockstore.

Nothing clever is going on. A unfs block key already is base32 of the multihash kubo's blockstore has been keyed by since 0.5, and both shard on the same two characters, so the casing and the suffix are the entire difference. The store also writes the blocks/SHARDING marker flatfs expects, before the first block lands. If that marker already names a different shard function, the write throws instead of scattering blocks kubo would never look for.

Point kubo at it (run ipfs init first, so the repo has its config) and ipfs block get, ipfs dag get, ipfs dag export, and bitswap all work on unfs blocks, because unfs links are ordinary tag-42 CIDs. ipfs cat and ipfs ls don't: a unfs node uses its own DAG-CBOR schema rather than UnixFS. Pick the naming when you create the directory, since the two spellings ignore each other's block files.

The IPFS guide covers the rest: what the gateway serves, CAR import and export in both directions, and the two garbage collectors you now have.

#What it promises

  • Every write is atomic. Blocks, refs, and metadata all land through a temp file plus a rename, so a reader sees the whole old value or the whole new one — never a truncated CID, never a half-written block. That holds across processes, which is what makes the directory safe to read while something else is writing it.
  • Every optional capability is there. has, delete, and keys all work, so packs, listRefs, and gc do too — plus setIfMatch and deleteIfMatch, the conditional write and the conditional delete that make concurrent writers safe rather than merely non-corrupting. See Multiple writers.
  • The directory is yours. Copy it, rsync it, tar it, check it into a build cache.
  • trusted: true is usually right here. A private directory this process owns is exactly the case that option exists for, and it skips the read-time re-hash. See Performance.
const store = await openStore(fsStore(".data/store"), { trusted: true });

Three caveats before it holds something you care about:

  • A plain ref update is last-write-wins. Atomic writes keep two processes from corrupting each other, but they don't order them: two processes that each read the root, write a file, and commit will race, and the loser's changes are gone — not corrupted, just dropped. createFs/createFsSync close that by default over this store, and a hand-rolled commit still needs compare-and-set.
  • Atomic isn't durable. Nothing here calls fsync, so a power loss can still lose a write the filesystem said it took, and a crash can leave a stray file in .tmp/ (harmless, and safe to delete when nothing is writing).
  • A crash mid-commit leaves a lock behind. Because createFs now commits through the lock file, a writer killed while holding refs/head.ref.lock wedges that ref: every later write is EBUSY until someone who knows no writer is running deletes it. Nothing breaks a lock on a timer, deliberately — the alternative is shooting a live writer. Pass { atomic: false } if you'd rather have the last-write-wins exposure instead.
  • A case-insensitive filesystem folds ref names. On the default macOS and Windows volumes, refs:Head and refs:head are one file.

#Multiple writers

Two processes writing blocks was always safe — same bytes, same key, and each block lands with a rename — but moving the same ref is a race, and the loser's whole commit disappears without an error.

Over this store the Unfs facade handles that for you, and has since it grew compare-and-set commits. Every mutation repoints the ref only if it still holds the value the mutation started from, and one that lost the race re-runs itself against the winner's tree:

// Two processes, one directory, no coordination. Neither loses the other's work.
const fs = createFs(await openStore(fsStore(".data/store")));
await fs.writeFile("/notes/mine.txt", "safe");

See Multiple writers for the full behaviour — what EAGAIN means, why batch reports instead of retrying, and the one case it can't cover.

The rest of this section is the manual form, which is still what you want for a commit the facade doesn't make: grafting a ref you built yourself, or a mirror bind. setRefIfMatch is the primitive — commit only if the ref still holds what you read, and re-read if it doesn't.

import { getRef, setRefIfMatch } from "unfs";

for (;;) {
  const head = await getRef(store, "head");
  const next = await buildFrom(head); // read the tree, write your change
  if (await setRefIfMatch(store, "head", head, next)) break;
  // Someone else committed first. Their tree is the new base — start over.
}
  • false means the ref moved, so the tree you built is based on a root nobody has. Re-read and rebuild; don't retry the same commit.
  • undefined as the expected value means "only if nothing is bound yet" — a create-only write.
  • Under the hood it's a git-style lock file (refs/head.ref.lock, created with O_CREAT | O_EXCL) held across the read, the compare, and the swap. Contention is retried for about a second; past that you get an UnfsError with code BUSY naming the file, because at that point it's more likely a leftover from a crashed writer than a queue. Nothing ever deletes someone else's lock on a timer — that's your call, once you know no writer is running.
  • The lock is advisory: it only stops other conditional writers. A plain setRef walks straight past it, so pick one discipline per store.
  • deleteRefIfMatch(store, name, expected) is the same thing for unbinding a name: it takes the same lock, compares the same way, and unbinds only if the ref still holds what you read. Deleting a name is the one update whose result is nothing, so reading first and then calling deleteRef would silently drop anyone else's commit that landed in between — this is what closes that. It's what unfs/git runs git push origin :branch on.

Four processes each running fifty read-modify-write commits over one ref land all two hundred through setRefIfMatch, and about sixty through plain setRef.

If you want a fast layer in front instead, see Layered Stores — which assumes a single writer, and deliberately offers neither conditional write, so a filesystem built on one falls back to last-write-wins.

#Why it loads anywhere

The store imports nothing, not even node:fs. Its builtins come from process.getBuiltinModule at load time, so the published bundle has no imports at all. A browser bundler that follows this entry point resolves nothing and pulls in no fs polyfill, and the module loads fine anywhere.

Somewhere without those builtins, fsStore() throws UNSUPPORTED with a message pointing you at memoryStore(), so you get a clear failure at construction rather than a broken import.

#Disk space

One file per key is what makes this store browsable, and it is charged in whole filesystem blocks. A ref value is 36 bytes and occupies a full block — 4 KiB on a typical Linux filesystem — so a store whose keys are mostly small values can occupy several times the data it holds. Measured on a hosted Git repository, this store allocated between 1.9× and 7.2× its data, where the same keys in unfs/sqlite or unfs/lmdb took 1.1× to 1.5×.

Large files are barely affected — a block of file content is a real file of real size. If your store is dominated by small values and disk space matters, prefer a single-file store.

#When to reach for something else

you wantuse
a directory, no dependenciesunfs/fs
one file, no dependenciesunfs/sqlite
the least disk space for many tiny keysunfs/sqlite
several processes writing at onceunfs/lmdb
Redis, S3, Cloudflare KV, IndexedDB, …unfs/unstorage
a store someone else hostsremote stores
nothing on disk at allmemoryStore() from unfs

unfs/unstorage also reaches the local disk, through its own fs and fs-lite drivers. Use it when the disk is one option among several you might swap between, and unfs/fs when the disk is the answer and you'd rather not install anything.

unfs  A filesystem you can put in any key–value store.