glm-5.3-flash
e403b22e26
The backends spec read as if the Backend trait were a general storage abstraction and 'two backends, exactly' banned other storage in a deployment — so a downstream wanting sqlite manifests appeared forced into implementing a third backend, the exact weld ADR-004 forbids. - backends-and-dispatch.md: state plainly that the trait is a CAS-tier contract (immutable digest-keyed entries, GC-shaped list/delete) and never a host for manifest/index storage; consumer storage is beside-the-pool. 'Two, exactly' counts trait impls shipped, not storage per deployment. kv: tier name vs pinned sqlite engine made explicit. - ADR-004: add the explicit corollary under the consumer-seams point.
157 lines
7.5 KiB
Markdown
157 lines
7.5 KiB
Markdown
---
|
||
status: draft
|
||
last_updated: 2026-10-02
|
||
---
|
||
|
||
# Backends and dispatch
|
||
|
||
## What this is
|
||
|
||
The physical storage layer: the `Backend` trait contract every backend
|
||
implements, the two shipped backends (kv, fs), and the size-threshold
|
||
dispatch that routes between them. The trait is the crate's second
|
||
stable seam (with the key encoding, ADR-002): backends must survive
|
||
digest-layer evolution, so the boundary stays dumb.
|
||
|
||
## What the Backend trait is (and is not)
|
||
|
||
The trait is a **CAS-tier contract**: it accepts, serves, and enumerates
|
||
immutable content-addressed entries (`hash → bytes`), and its `list()`/
|
||
`delete()` shapes exist to feed GC (ADR-005). That is its whole job, and
|
||
the contract assumes it: complete-list-equals-sweep-safety only makes
|
||
sense for a store where every entry is a standalone immutable blob.
|
||
|
||
It follows that the trait is *not* a general-purpose storage abstraction,
|
||
and a downstream must never implement it to host data that is not
|
||
content-addressed blobs — a manifest table, a queryable index, or any
|
||
mutable/structured state. Such storage is **consumer-owned, beside the
|
||
pool** (a downstream's own sqlite/whatever, ADR-004): it needs no
|
||
digest-byte addressing, no `list()`-completeness, and no sweep
|
||
participation, so the CAS-tier contract fits it worse than it fits any
|
||
purpose-built option. The wrong turn to rule out is "my data isn't
|
||
blobs, therefore I implement a Backend for it"; nothing a consumer needs
|
||
is ever reachable through that door.
|
||
|
||
## The Backend trait contract (ADR-003)
|
||
|
||
- **Opaque byte keys, opaque byte values.** Backends never learn what a
|
||
digest is; typed `Key` converts at the store boundary (ADR-002). This
|
||
is also the persistence boundary — a kv row or an fs filename has no
|
||
schema-migration story, so the byte layout must be self-describing
|
||
(algorithm-tagged keys).
|
||
- **Methods:** `has` / `get` / `put` / `delete` / `list` / `name`.
|
||
Streaming put/get shapes ride the store layer's seams (store-api.md);
|
||
backends provide byte- or handle-level primitives underneath
|
||
(`get` on the fs tier yields a file handle, not a loaded buffer).
|
||
- **`list()` complete by contract.** A *malformed* list is as deadly as
|
||
an incomplete one: list correctness is only observable through GC
|
||
(POC #1 finding 2 — the redb key-vs-value trap produced a sweep that
|
||
deleted the wrong blobs, silently). Therefore:
|
||
- every backend implementation must prove list correctness through
|
||
sweep-outcome tests (the invariant's test gate lives in
|
||
store-api.md);
|
||
- an implementation that cannot enumerate (the rudolfs S3
|
||
anti-lesson) must not ship — GC is a structural requirement, not an
|
||
optional extra (ADR-005).
|
||
- **Virgin-store reads are no-ops:** read paths treat a missing table
|
||
as absent/empty (POC #1 finding 2).
|
||
- **Namespace-blind:** backends never see namespaces, tenants, or
|
||
reference structure (ADR-005 — the rudolfs inversion; physical
|
||
storage is `hash → bytes` flat).
|
||
|
||
## Shipped backends
|
||
|
||
Two, exactly — this counts *trait implementations this crate ships*, the
|
||
CAS tiers the dispatch routes between; it is the complete set the
|
||
problem requires, and the "third backend" fear is a category error fixed
|
||
in ADR-004. It says nothing about other storage existing in a deployment:
|
||
a downstream runs whatever else it needs on its own media, beside the
|
||
pool, above the store's seams (see "What the Backend trait is").
|
||
|
||
### kv backend (feature `kv`, default-on; shipped engine: sqlite)
|
||
|
||
Small blobs — **`kv` is the tier name; sqlite is the pinned shipped
|
||
engine** (ADR-003's consequences carry that trade: sqlite's on-disk
|
||
format compat is upstream's guarantee). The shipped trait impl writes
|
||
plain content-addressed rows into it, nothing else. "Pinned engine, not
|
||
a kv abstraction" does not open a substitution seam inside this tier:
|
||
a downstream wanting a different engine there would be new ADRs (a new
|
||
impl, its own sweep-safety proof); and a downstream's *other* storage —
|
||
schemas, manifests, whatever it likes including some other sqlite file —
|
||
never comes near this tier at all, per "What the Backend trait is".
|
||
|
||
Evidence (POC #3 finding A4, first-party measured):
|
||
sqlite is ~9-10× faster than fs at 1-16 KiB (the git small-blob regime
|
||
— most git objects, workspace files, manifests), with the crossover at
|
||
~128-256 KiB where fs stops paying the B-tree row rewrite and wins.
|
||
|
||
- Bounded reads; `has` as an EXISTS probe; prepared statements.
|
||
- WAL + `synchronous=NORMAL` as the shipped durability tier (matching
|
||
what the benchmark measured and what iroh's store ships).
|
||
- Read paths tolerate the no-tables-yet database (see contract).
|
||
|
||
### fs backend (feature `fs`, default-on)
|
||
|
||
Large blobs. Flat sharded layout: `{hex-prefix}/{hex-prefix}/{hash}`
|
||
sharding survives from iroh's conclusion (limits directory size on
|
||
huge pools); stage-then-commit-rename for the two-pass unknown-length
|
||
path; pread-based range reads (POC #3 finding A2 — local range serving
|
||
is sound, e.g. for packfiles).
|
||
|
||
### mem backend (feature `mem`, default-off)
|
||
|
||
`BTreeMap`-shaped ephemeral backend for tests and in-process
|
||
ephemerality. A testing/utility tier, never a production story.
|
||
|
||
## Size-threshold dispatch (ADR-003)
|
||
|
||
- **Routing is a pure function of content length.** Same content ⇒ same
|
||
length ⇒ same backend; re-puts are deterministic. No content ever
|
||
migrates between backends — the migration question existed only to
|
||
patch the unknown-length asymmetry, which ADR-003's pre-threshold
|
||
buffering eliminates by construction.
|
||
- **Default threshold: 128 KiB** (constructor-tunable). Midpoint of the
|
||
measured flat zone (POC #3 A4). Re-tuning per deployment media is a
|
||
constructor parameter, not an API change.
|
||
- **Get fall-through:** small-tier miss queries the large tier
|
||
(deterministic, cheap — a stat probe).
|
||
- Per-namespace or per-tenant backend configuration: **rejected**
|
||
(ADR-003 §Consequences — it would re-weld namespacing into the
|
||
physical layer, the rudolfs anti-pattern ADR-005 inverts).
|
||
|
||
## Where a *new* backend could come from
|
||
|
||
The trait is open to future implementations (network stores, S3-like
|
||
tiers), but nothing in the current consumer set requires one, and the
|
||
contract is deliberately hostile to half-implementations (complete
|
||
`list()`, GC-participating `delete`). Any future backend is a new ADR
|
||
carrying its own sweep-safety story. This crate's roadmap is not
|
||
blocked on one (see open-questions.md — alkfs intake may name needs
|
||
externally; OQ-08).
|
||
|
||
## Design Decisions
|
||
|
||
| ADR | Decision | Summary |
|
||
|---|---|---|
|
||
| [003](decisions/003-backend-contract-and-dispatch.md) | Backend contract & dispatch | opaque keys, complete list, pure-function routing, no migration |
|
||
| [004](decisions/004-two-backends-no-third.md) | Two backends, no third | scope boundary against manifest-layer absorption |
|
||
| [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Namespace-blindness | backends see hashes only |
|
||
|
||
## Open Questions
|
||
|
||
- **OQ-08**: alkfs requirement intake may name storage requirements
|
||
(e.g., durability tiers, sync-friendly layouts) that touch this
|
||
layer — open, external owner (alkfs Phase 0).
|
||
|
||
## References
|
||
|
||
- `docs/research/poc-trait-dispatch-findings.md` findings 1/2/6
|
||
- `docs/research/poc-largeblob-findings.md` findings A2/A4 (+ the
|
||
re-runnable benchmark harness)
|
||
- `docs/research/iroh-blobs-eval.md` — the fs-layout conclusions borrowed
|
||
(sharding, crash ordering, inline thresholds rejected as weld)
|
||
- rudolfs notes — `list()` anti-lesson, decorator alternative noted and
|
||
not adopted (threshold dispatch chose the simpler policy;
|
||
ADR-003 §Context)
|
||
- ADR-003, ADR-004, ADR-005; [store-api.md](store-api.md) (the invariants
|
||
backends must satisfy) |