SQLite Store

Keep a unfs filesystem in a single SQLite file, with nothing to install.

fsStore() puts a store in a directory. sqliteStore() puts the same store — same keys, same blocks — in one file, using the SQLite your runtime already ships. It's a separate entry point, import ... from "unfs/sqlite", and it installs nothing: Node reaches node:sqlite and Bun reaches bun:sqlite, both asked for at load time rather than imported.

import { createFs, initStore } from "unfs";
import { sqliteStore } from "unfs/sqlite";

const fs = createFs(await initStore(sqliteStore(".data/store.sqlite")));

await fs.mkdir("/notes", { recursive: true });
await fs.writeFile("/notes/hello.txt", "hello from sqlite\n");

await fs.head(); // bafyrei… — the root CID of what's now in the file

Reopen it later, from another process or after a restart, and it's all still there:

import { createFs, openStore } from "unfs";
import { sqliteStore } from "unfs/sqlite";

const fs = createFs(await openStore(sqliteStore(".data/store.sqlite")));
await fs.readFile("/notes/hello.txt", "utf8"); // "hello from sqlite\n"

The location is a path, a file: URL, or ":memory:". The file and its one table are created on first use.

Note

This is a KVStore, not a mirror. The database holds the block graph as opaque rows, not your files as rows you'd want to query: /notes/hello.txt is a leaf of a Merkle DAG. To move a tree in or out as ordinary files, use a pack.

#Using it synchronously

There's no synchronous twin to reach for, because there's nothing to switch to. node:sqlite and bun:sqlite are synchronous APIs, so every reply is already a plain value and the same store object drives both facades:

import { createFsSync, initStore } from "unfs";
import { sqliteStore } from "unfs/sqlite";

const fs = createFsSync(await initStore(sqliteStore(".data/store.sqlite")));

fs.writeFileSync("/hello.txt", "no await anywhere\n");
fs.readFileSync("/hello.txt", "utf8");

That makes it the third synchronous backend after memoryStore() and fsStoreSync(), and the second one that outlives its process. As with fsStoreSync, synchronous calls block the event loop, so prefer createFs wherever you can await — the store itself is the same either way.

#Options

const store = sqliteStore(".data/store.sqlite", { readonly: true });

readonly opens the database read-only, so every write fails rather than touching a store someone else owns. Reads and keys() work as usual; the table and the journal mode are left alone, since a read-only connection can neither create them nor needs to.

The store-level options go on initStore / openStore as always, and trusted: true is as right here as it is for a private directory:

const store = await openStore(sqliteStore(".data/store.sqlite"), { trusted: true });

#Closing the file

A database is a file handle plus a write-ahead log, so this store has one method KVStore doesn't:

const kv = sqliteStore(".data/store.sqlite");
const fs = createFs(await initStore(kv));
// …
kv.close();

Nothing else in unfs needs closing. Call it when you're done with a store you'll re-open, or before deleting the file; a process that simply exits is fine.

#What it needs

The runtime has to have SQLite built in:

  • Node 22.13+ (and 23.4+), where node:sqlite is unflagged. It exists from 22.5 behind --experimental-sqlite.
  • Bun, through bun:sqlite. Verified on 1.3.14.

Anywhere else, sqliteStore() throws an UnfsError with code UNSUPPORTED and a message pointing at memoryStore(), unfs/fs, and unfs/unstorage. As with the on-disk store, that's a clear failure at construction rather than a broken import: the module imports nothing at all, so a bundler that follows this entry point resolves no sqlite specifier and drags in no polyfill.

#What it promises

  • Every optional capability is there. has, delete, and keys all work, so packs, listRefs, and gc do too — plus setIfMatch and deleteIfMatch, the compare-and-set and the compare-and-delete. See Multiple writers.
  • Multi-block reads run in one transaction. When unfs fetches a set of blocks at once — a directory level, a file's chunks, gc proving a live set present — this store reads the whole set inside a single SQLite read transaction instead of letting every key open one of its own. It's worth roughly 1.5–2× on the reads themselves against a database on disk, and nothing at all against :memory:, which has no journal to bookkeep.
  • Enumeration is indexed. The table is keyed by the store key and stored in key order, so keys("refs:") seeks to the first ref and stops after the last one instead of scanning the whole store. That's the operation gc starts with, and the one a directory has no index for.
  • Any well-formed string key round-trips verbatim. There's no key-to-path mapping to constrain what a key may look like, so unlike fsStore this store works underneath prefixedStore — several tenants in one file, each under its own prefix. (SQLite's TEXT is well-formed Unicode, so a key carrying a lone surrogate is the one thing that doesn't survive the trip. Nothing unfs generates can contain one.)
  • Durability is SQLite's. WAL journalling at SQLite's default synchronous = FULL, so the log is flushed at every commit and a committed write survives a power cut, not just a process crash. That's more than the on-disk store promises, which never calls fsync at all.
  • It's one file. cp it, scp it, attach it, drop it in a container image. No inode and no block-size rounding per block, which is what a store of millions of small blocks pays in a directory.

Three caveats before it holds something you care about:

  • One writer at a time, by design. SQLite serializes writers on a single database lock. Readers run alongside the writer under WAL, but a second writing process waits — for up to five seconds, after which the call fails rather than queueing forever.
  • You can't ls it. A directory store is something you can inspect with ordinary tools, rsync incrementally, and — with naming: "flatfs" — hand to kubo. A database file is opaque to all three. See Which one.
  • keys() streams, so drain it before you write to the store. It's a live cursor over the index, which is what makes the early exit possible; writing to a key it has already handed you can hand it to you twice. unfs itself never does this — every internal caller collects the keys first — so it only concerns you if you're calling store.keys() directly.

#Multiple writers

setIfMatch here is a real transaction. BEGIN IMMEDIATE takes the write lock before the read, so the read, the comparison, and the write are one atomic step and SQLite closes the race for you — no lock file, and nothing left behind if the writer dies mid-transaction.

The loop is the same one the on-disk store documents:

import { getRef, setRefIfMatch } from "unfs";

for (;;) {
  const head = await getRef(store, "head");
  const next = await buildFrom(head); // read the tree, write your change
  if (await setRefIfMatch(store, "head", head, next)) break;
  // Someone else committed first. Their tree is the new base — start over.
}
  • false means the ref moved, so the tree you built is based on a root nobody has. Re-read and rebuild.
  • undefined as the expected value means "only if nothing is bound yet" — a create-only write.
  • An UnfsError with code BUSY is different: another connection held the write lock for longer than five seconds, so nothing was compared at all. Retrying is reasonable; treating it as false isn't, because a wedged writer would then loop forever.

deleteRefIfMatch(store, name, expected) is the same transaction around a DELETE: it unbinds the name only if it still holds what you read, so removing a name can't quietly discard a commit that landed while you were deciding to.

As everywhere, the guarantee only covers writers who all use it — a plain setRef still overwrites unconditionally, and a plain deleteRef still removes unconditionally.

#Which one

unfs/fs is still the default recommendation, and this is an option beside it rather than a replacement. They store identical blocks under identical keys, so the choice is about the container:

you wantuse
a store to ls, grep, and rsync incrementallyunfs/fs
a directory kubo also reads (naming: "flatfs")unfs/fs
one file to copy, ship, or attachunfs/sqlite
indexed keys(prefix), so gc doesn't walk the storeunfs/sqlite
compare-and-set with no lock file to clean upunfs/sqlite
several stores in one place, under prefixedStoreunfs/sqlite
the least disk space when keys are small and manyunfs/sqlite
a synchronous store, for createFsSynceither

unfs/lmdb is the third local option, and the one that is a database engine rather than a container: it takes a native dependency and gives back cross-process writers and a memory-mapped read path.

Note

No speed numbers yet. The differences above are structural — an index range scan instead of a directory walk, one file instead of millions — and how they land in wall-clock time for a given tree hasn't been measured. Until it is, pick on the properties you need rather than on an expected speedup.

Disk space has been measured: because there is no per-key block rounding, the same keys occupy 1.1–1.4× their own bytes here, against 1.9–7.2× in unfs/fs. The gap is widest when most keys are small values.

For anything that isn't a local file, see unstorage, which covers around 40 drivers, or remote stores.

unfs  A filesystem you can put in any key–value store.