alkblobs/docs/architecture/decisions/004-two-backends-no-third.md

ADR-004: Exactly two backends; manifest layers stay above the crate

Status

Accepted

Context

An early Phase 0 framing — "using two backends is ugly" — shaped several rounds before being edited away, but the correction it was reaching for arrived only in the Phase 1 review, and needs recording so no future round reabsorbs the concern incorrectly:

• There are always two physical backends in this problem: storage for small content (fast, kv/sqlite) and the filesystem for large content. This is not ugly, it is the measured shape of the problem (POC #3 A4: crossover ~128-256 KiB; SQLite ~9-10× faster below it).
• The "two backends is ugly" sentiment was actually about a third layer in the old alknet-filesystem research — the virtual filesystem / appfile stack: path trees, branches, tombstones, filename↔hash maps living alongside the blob store (in that research, literally a co-equal SQLite store beside iroh-blobs).

If left implicit, this misframing has two failure modes: (a) this crate absorbs a path-tree/manifest layer ("three backends"), growing tables, schemas, and branching semantics that belong to consumers; (b) this crate shapes itself such that a future vfs builder must hack around it. alkfs is a pending consumer precisely of that manifest layer — this ADR is the boundary it will build against.

Decision

This crate ships exactly two production backends and nothing more; every manifest/namespace/path-tree layer is a consumer, by construction and by contract. (A non-production ephemeral mem tier exists for tests — ADR-003; it is not a storage story and does not extend this boundary.)

Scope note (post-ADR-007 reconciliation): the "two" here counts tiers, not engines. The kv tier is one backend; ADR-007 widened the engines behind it to a set (sqlite default + postgres, feature-gated) so a node's kv rides its relational engine of choice. The count that matters to this ADR — trait impls this crate ships as CAS tiers — remains two (kv + fs); a second kv engine does not make a third backend, and nothing in the engine set touches the manifest-layer boundary below.

1. Two production backends is the complete set. kv/sqlite (small)

   • fs (large) cover the scale economics with measured evidence (ADR-003); no other physical tier is needed by any current or planned consumer's blob needs.

2. The "third backend" is not this crate's layer. The vfs/appfile shape — path→hash mapping, branches/tombstones/deltas per workspace, the alknet-filesystem lineage — is storage above the pool: reference sets over hashes (ADR-005's namespace machinery), not a Backend implementation. Consumers build it on their own storage (alkfs's path tree, alkgit's refs).

3. The store provides the seams the manifest layer needs, and nothing that forces it to hack:

   • hash-addressed put/get with a verified, git-interop-compatible addressing scheme (ADR-002) — manifests reference entries by the same canonical addresses git uses;
   • liveness registration so manifest-holds maps to GC roots without the store learning manifest formats (ADR-005);
   • stat/read_range — enough for serving path-resolved content and large-file ranges;
   • pinned batch puts — manifest-atomic multi-file writes are protected in flight (ADR-005).

   Explicit corollary (added after a downstream misread): "consumers build it on their own storage" means a consumer's manifest storage — sqlite or whatever else — lives beside the pool, never behind the Backend trait. The trait is a CAS-tier contract (immutable digest-keyed entries, complete list, GC-participating delete); it cannot express manifest storage, so a consumer implementing a Backend to host a manifest was never a live option this boundary forecloses — the boundary forecloses the crate absorbing such layers, and the "no third backend" count is trait impls this crate ships, not storage that may exist in a deployment.

4. Namespacing at the store level is logical only (ADR-005): reference tables above, flat bytes below. If a consumer wants physical isolation per tenant/repo, that is two pool deployments (a topology choice), never a namespace flag inside one.

Consequences

Positive

• The crate's scope stays testable and small: two backends, one pool, one hash domain. No schema/versioning surface for manifest formats — consumers iterate on theirs freely.
• alkgit and alkfs build their (very different) mapping layers on one shared pool without either bending the store; their manifests interop at the address level by construction.
• The alknet-filesystem lineage is honored as shape guidance for consumers (branches, write sessions, chain walks), with zero of its storage-layer mechanics (SQLite path tables, honker wiring, CRDT sync) leaking in here.

Negative

• Consumers each re-build the small amount of manifest plumbing they need (a reference table and a root registration). This is deliberate: the manifest needs of alkgit (git refs) and alkfs (path trees) are different enough that a shared one would fit neither.

Neutral

• If a future consumer did want a first-party manifest layer, it would be a sibling crate over this one (e.g. alkfs itself), not an extension of this crate.

References

• docs/research/phase-0.md (settled approach: pooled CAS; OQ-BL-05)
• /workspace/@alkdev/alknet/docs/research/alknet-filesystem/poc-summary.md — the historical lineage this ADR explicitly bounds (not a design input)
• ADR-003 (the two backends), ADR-005 (namespaces as reference sets)
• overview.md (consumer map: alkgit, alkfs); backends-and-dispatch.md