---
status: draft
last_updated: 2026-10-01
---

# Backends and dispatch

## What this is

The physical storage layer: the `Backend` trait contract every backend
implements, the two shipped backends (kv, fs), and the size-threshold
dispatch that routes between them. The trait is the crate's second
stable seam (with the key encoding, ADR-002): backends must survive
digest-layer evolution, so the boundary stays dumb.

## The Backend trait contract (ADR-003)

- **Opaque byte keys, opaque byte values.** Backends never learn what a
  digest is; typed `Key` converts at the store boundary (ADR-002). This
  is also the persistence boundary — a kv row or an fs filename has no
  schema-migration story, so the byte layout must be self-describing
  (algorithm-tagged keys).
- **Methods:** `has` / `get` / `put` / `delete` / `list` / `name`.
  Streaming put/get shapes ride the store layer's seams (store-api.md);
  backends provide byte- or handle-level primitives underneath
  (`get` on the fs tier yields a file handle, not a loaded buffer).
- **`list()` complete by contract.** A *malformed* list is as deadly as
  an incomplete one: list correctness is only observable through GC
  (POC #1 finding 2 — the redb key-vs-value trap produced a sweep that
  deleted the wrong blobs, silently). Therefore:
  - every backend implementation must prove list correctness through
    sweep-outcome tests (the invariant's test gate lives in
    store-api.md);
  - an implementation that cannot enumerate (the rudolfs S3
    anti-lesson) must not ship — GC is a structural requirement, not an
    optional extra (ADR-005).
- **Virgin-store reads are no-ops:** read paths treat a missing table
  as absent/empty (POC #1 finding 2).
- **Namespace-blind:** backends never see namespaces, tenants, or
  reference structure (ADR-005 — the rudolfs inversion; physical
  storage is `hash → bytes` flat).

## Shipped backends

Two, exactly — this is the complete set the problem requires; the
"third backend" fear is a category error fixed in ADR-004.

### kv backend (feature `kv`, default-on; sqlite)

Small blobs. Evidence (POC #3 finding A4, first-party measured):
sqlite is ~9-10× faster than fs at 1-16 KiB (the git small-blob regime
— most git objects, workspace files, manifests), with the crossover at
~128-256 KiB where fs stops paying the B-tree row rewrite and wins.

- Bounded reads; `has` as an EXISTS probe; prepared statements.
- WAL + `synchronous=NORMAL` as the shipped durability tier (matching
  what the benchmark measured and what iroh's store ships).
- Read paths tolerate the no-tables-yet database (see contract).

### fs backend (feature `fs`, default-on)

Large blobs. Flat sharded layout: `{hex-prefix}/{hex-prefix}/{hash}`
sharding survives from iroh's conclusion (limits directory size on
huge pools); stage-then-commit-rename for the two-pass unknown-length
path; pread-based range reads (POC #3 finding A2 — local range serving
is sound, e.g. for packfiles).

### mem backend (feature `mem`, default-off)

`BTreeMap`-shaped ephemeral backend for tests and in-process
ephemerality. A testing/utility tier, never a production story.

## Size-threshold dispatch (ADR-003)

- **Routing is a pure function of content length.** Same content ⇒ same
  length ⇒ same backend; re-puts are deterministic. No content ever
  migrates between backends — the migration question existed only to
  patch the unknown-length asymmetry, which ADR-003's pre-threshold
  buffering eliminates by construction.
- **Default threshold: 128 KiB** (constructor-tunable). Midpoint of the
  measured flat zone (POC #3 A4). Re-tuning per deployment media is a
  constructor parameter, not an API change.
- **Get fall-through:** small-tier miss queries the large tier
  (deterministic, cheap — a stat probe).
- Per-namespace or per-tenant backend configuration: **rejected**
  (ADR-003 §Consequences — it would re-weld namespacing into the
  physical layer, the rudolfs anti-pattern ADR-005 inverts).

## Where a *new* backend could come from

The trait is open to future implementations (network stores, S3-like
tiers), but nothing in the current consumer set requires one, and the
contract is deliberately hostile to half-implementations (complete
`list()`, GC-participating `delete`). Any future backend is a new ADR
carrying its own sweep-safety story. This crate's roadmap is not
blocked on one (see open-questions.md — alkfs intake may name needs
externally; OQ-08).

## Design Decisions

| ADR | Decision | Summary |
|---|---|---|
| [003](decisions/003-backend-contract-and-dispatch.md) | Backend contract & dispatch | opaque keys, complete list, pure-function routing, no migration |
| [004](decisions/004-two-backends-no-third.md) | Two backends, no third | scope boundary against manifest-layer absorption |
| [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Namespace-blindness | backends see hashes only |

## Open Questions

- **OQ-08**: alkfs requirement intake may name storage requirements
  (e.g., durability tiers, sync-friendly layouts) that touch this
  layer — open, external owner (alkfs Phase 0).

## References

- `docs/research/poc-trait-dispatch-findings.md` findings 1/2/6
- `docs/research/poc-largeblob-findings.md` findings A2/A4 (+ the
  re-runnable benchmark harness)
- `docs/research/iroh-blobs-eval.md` — the fs-layout conclusions borrowed
  (sharding, crash ordering, inline thresholds rejected as weld)
- rudolfs notes — `list()` anti-lesson, decorator alternative noted and
  not adopted (threshold dispatch chose the simpler policy;
  ADR-003 §Context)
- ADR-003, ADR-004, ADR-005; [store-api.md](store-api.md) (the invariants
  backends must satisfy)