docs(architecture): ADR-007 — two kv engines (sqlite + postgres) behind one trait

Dissolve OQ-10's circular "standing offer" framing and record the
engine decision the POC evidence already supported:

- ADR-007 (new): kv tier ships sqlite (default) + postgres (feature
  `postgres`, default-off), constructor-selected per node; records the
  deployment economics (classical kv+fs+relational downstream stack
  collapses to one relational engine + fs), the pg impl contract
  (POC #5 B5 posture deltas), the CI sweep-safety gate under
  --all-features, redb ruled out (POC #6), and posture defaults as
  deferred cost in the ADR-006 pattern
- ADR-003: amended status (engine pin widened by ADR-007); kv entry
  notes the widened engine set
- ADR-004: reconciliation note — the "two backends" count is tiers,
  not engines
- backends-and-dispatch.md: tier≠engine distinction, pg engine
  section, co-tenancy note (same-file consumer writes share sqlite's
  single-writer ceiling), substitution-door precedent
- open-questions.md: OQ-10 resolved-by-ADR-007, with the
  why-this-is-not-Schrödinger's-code record (a deployment running
  alkblobs cannot pre-exist the pg engine — OQ-09 precedent); parked
  index narrowed to OQ-07/OQ-08
- overview.md: layer map + dependency posture reflect the `postgres`
  feature; README.md: ADR-007 in tables + corollary applications
- phase-0.md / pg+redb findings: promotion + supersession notes

Trigger framing survives only as deployment guidance (which engine a
node chooses — topology, not throughput), not as a gate on the crate's
own work.

Verified: cargo test, clippy --all-targets -D warnings, fmt --check.
Docs-only change (Phase 1 architecture); no implementation yet.
This commit is contained in:
glm-5.3-flash committed 2026-10-02 19:29:42 +00:00
1 parent 1af9399900
commit 395d58cf3f
10 files changed
+369 -100

No files matched your search

+14 -2
View File
@@ -27,6 +27,15 @@ there — not inherited as hedges:
two backends (kv small / fs large — both required by measured scale two backends (kv small / fs large — both required by measured scale
economics); the path→hash mapping layer (the alkfs/alknet "vfs" shape) economics); the path→hash mapping layer (the alkfs/alknet "vfs" shape)
is a consumer-layer concern and must never become a third backend. is a consumer-layer concern and must never become a third backend.
- **ADR-007** widens the kv tier's engine set rather than hedging it:
sqlite (default) + postgres (feature-gated) behind one trait. The
deployment economics this records: a downstream node (git server, vfs)
classically needed three storage systems — kv + fs + relational — and
this design collapses them to **one relational engine + fs**, with the
engine a per-node constructor choice. Resolved from evidence in hand
(POCs #3/#5/#6); the would-be "deployment trigger" framing was caught
at review as circular hedging (a deployment running alkblobs cannot
pre-exist the engine) and closed on the OQ-09 precedent.
## Architecture Documents ## Architecture Documents
@@ -50,6 +59,7 @@ there — not inherited as hedges:
| [004](decisions/004-two-backends-no-third.md) | Exactly two backends; manifest layers stay above | Accepted | | [004](decisions/004-two-backends-no-third.md) | Exactly two backends; manifest layers stay above | Accepted |
| [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Pooled CAS, liveness seams, mark-and-sweep, delete windows | Accepted | | [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Pooled CAS, liveness seams, mark-and-sweep, delete windows | Accepted |
| [006](decisions/006-verification-posture-and-transfer-encoding.md) | Whole-blob verification; transfer encoding excluded by scoping | Accepted | | [006](decisions/006-verification-posture-and-transfer-encoding.md) | Whole-blob verification; transfer encoding excluded by scoping | Accepted |
| [007](decisions/007-two-kv-engines-sqlite-and-postgres.md) | Two kv engines (sqlite + postgres) behind one trait; constructor-selected | Accepted |
## Open Questions ## Open Questions
@@ -57,7 +67,8 @@ Tracked in [open-questions.md](open-questions.md). The Phase 0 register
(OQ-BL-01..06) is promoted there with its resolutions; two questions (OQ-BL-01..06) is promoted there with its resolutions; two questions
remain parked — OQ-07/OQ-08 externally-owned (alkgit seam mapping, remain parked — OQ-07/OQ-08 externally-owned (alkgit seam mapping,
alkfs intake; carried for visibility, gating nothing here). OQ-09 alkfs intake; carried for visibility, gating nothing here). OQ-09
(namespace-visibility default) resolved closed-by-default. **Deferral (namespace-visibility default) resolved closed-by-default; OQ-10
(second kv engine) resolved by ADR-007. **Deferral
policy (the "Schrödinger's code" rule):** a *decision this crate needs policy (the "Schrödinger's code" rule):** a *decision this crate needs
before shipping* may not be deferred on a dependency that is itself before shipping* may not be deferred on a dependency that is itself
waiting for this crate to exist. Two parking kinds remain legitimate: waiting for this crate to exist. Two parking kinds remain legitimate:
@@ -68,7 +79,8 @@ crate) that gate no decision here — full definitions in the header of
because it recurs: a fact this crate must create (its own first because it recurs: a fact this crate must create (its own first
deployment, its own first consumer) is never a deciding input — deployment, its own first consumer) is never a deciding input —
decisions stand on evidence in hand, and future needs reopen via named decisions stand on evidence in hand, and future needs reopen via named
requirements, not via waiting. requirements, not via waiting. (Corollary applications so far: OQ-09
and OQ-10 both resolved on it.)
## Lifecycle ## Lifecycle
+64 -25
View File
@@ -61,34 +61,66 @@ is ever reachable through that door.
## Shipped backends ## Shipped backends
Two, exactly — this counts *trait implementations this crate ships*, the Two tiers, exactly — this counts *trait implementations this crate
CAS tiers the dispatch routes between; it is the complete set the ships* as CAS tiers, the dispatch routes between them; it is the
problem requires, and the "third backend" fear is a category error fixed complete set the problem requires, and the "third backend" fear is a
in ADR-004. It says nothing about other storage existing in a deployment: category error fixed in ADR-004. It says nothing about other storage
a downstream runs whatever else it needs on its own media, beside the existing in a deployment: a downstream runs whatever else it needs on
pool, above the store's seams (see "What the Backend trait is"). its own media, beside the pool, above the store's seams (see "What the
Backend trait is"). **Tier count is not engine count**: the kv tier
carries two engines (sqlite, postgres) behind one trait — ADR-007;
per-node engine selection is a constructor parameter.
### kv backend (feature `kv`, default-on; shipped engine: sqlite) ### kv backend (feature `kv`, default-on); engines: sqlite (default) + postgres (feature `postgres`, default-off)
Small blobs — **`kv` is the tier name; sqlite is the pinned shipped Small blobs — **`kv` is the tier name; the engine is a per-node
engine** (ADR-003's consequences carry that trade: sqlite's on-disk constructor choice** between the shipped engines (ADR-007; ADR-003
format compat is upstream's guarantee). The shipped trait impl writes carries sqlite's on-disk format trade: file compat is upstream
plain content-addressed rows into it, nothing else. "Pinned engine, not sqlite's guarantee). The shipped trait impls write plain
a kv abstraction" does not open a substitution seam inside this tier: content-addressed rows into the chosen engine, nothing else. A
a downstream wanting a different engine there would be new ADRs (a new *third* engine remains the substitution door as amended: a new ADR
impl, its own sweep-safety proof); and a downstream's *other* storage — with its own sweep-safety proof (the door is no longer hypothetical —
schemas, manifests, whatever it likes including some other sqlite file — ADR-007 is its first in-repo exercise); and a downstream's *other*
never comes near this tier at all, per "What the Backend trait is". storage — schemas, manifests, whatever it likes including some other
sqlite file — never comes near this tier at all, per "What the
Backend trait is".
The collapse ADR-007 buys: the classical downstream stack (git
server, vfs node) needed three storage systems — kv + fs + relational.
Here the kv tier rides the relational engine itself, so a node
provisions **one relational engine + fs**; consumer tables (refs,
manifests, queues) sit beside the pool on the same engine or file by
the deployment's own choice (co-tenancy note: same-file consumer
writes share sqlite's single-writer ceiling — that sharing is the
node's trade; separate files remain available).
Evidence (POC #3 finding A4, first-party measured): Evidence (POC #3 finding A4, first-party measured):
sqlite is ~9-10× faster than fs at 1-16 KiB (the git small-blob regime sqlite is ~9-10× faster than fs at 1-16 KiB (the git small-blob regime
— most git objects, workspace files, manifests), with the crossover at — most git objects, workspace files, manifests), with the crossover at
~128-256 KiB where fs stops paying the B-tree row rewrite and wins. ~128-256 KiB where fs stops paying the B-tree row rewrite and wins.
- Bounded reads; `has` as an EXISTS probe; prepared statements. **sqlite engine** (the default): WAL + `synchronous=NORMAL` as the
- WAL + `synchronous=NORMAL` as the shipped durability tier (matching shipped durability tier (matching what the benchmark measured and what
what the benchmark measured and what iroh's store ships). iroh's store ships); bounded reads; `has` as an EXISTS probe; prepared
- Read paths tolerate the no-tables-yet database (see contract). statements; read paths tolerate the no-tables-yet database (see
contract). Solo economics win the single-machine case by measurement;
the WAL single-writer ceiling (~1.2k objects/s under contention,
POC #5 B3) is an order above single-node push rates.
**postgres engine** (feature `postgres`, default-off; ADR-007): the
cross-machine-writers / many-client-node / already-running-pg engine.
Contract-identical row shape (`key bytea PK / value bytea / size`),
`ON CONFLICT DO NOTHING` CAS, complete `list()` as a stable cursor
over a vacuuming table, batch-delete via `key = ANY($1)`. Impl
requirements per POC #5 finding B5: configurable `synchronous_commit`
posture (the parity knob with the sqlite shipped tier), pooled
connections with per-connection prepared-statement discipline, stated
autovacuum + fillfactor tuning, ~40× small-tier storage overhead
accepted (B6). Sweep-safety proof via the same exact-count
sweep-outcome CI gate, run against dockerized postgres under
`--all-features`. Concurrency evidence (POC #5 B3/B4): gets/puts scale
near-linearly to ~37k puts/s at 24 conns — the only engine whose
throughput *increases* under concurrency.
### fs backend (feature `fs`, default-on) ### fs backend (feature `fs`, default-on)
@@ -119,15 +151,17 @@ ephemerality. A testing/utility tier, never a production story.
(ADR-003 §Consequences — it would re-weld namespacing into the (ADR-003 §Consequences — it would re-weld namespacing into the
physical layer, the rudolfs anti-pattern ADR-005 inverts). physical layer, the rudolfs anti-pattern ADR-005 inverts).
## Where a *new* backend could come from ## Where a *new* backend or engine could come from
The trait is open to future implementations (network stores, S3-like The trait is open to future implementations (network stores, S3-like
tiers), but nothing in the current consumer set requires one, and the tiers), but nothing in the current consumer set requires one, and the
contract is deliberately hostile to half-implementations (complete contract is deliberately hostile to half-implementations (complete
`list()`, GC-participating `delete`). Any future backend is a new ADR `list()`, GC-participating `delete`). Any future backend — or third kv
carrying its own sweep-safety story. This crate's roadmap is not engine — is a new ADR carrying its own sweep-safety story (ADR-007 is
blocked on one (see open-questions.md — alkfs intake may name needs the precedent for an engine addition: measured evidence first, impl
externally; OQ-08). contract naming the durability/ops posture deltas). This crate's
roadmap is not blocked on one (see open-questions.md — alkfs intake
may name needs externally; OQ-08).
## Design Decisions ## Design Decisions
@@ -136,6 +170,7 @@ externally; OQ-08).
| [003](decisions/003-backend-contract-and-dispatch.md) | Backend contract & dispatch | opaque keys, complete list, pure-function routing, no migration | | [003](decisions/003-backend-contract-and-dispatch.md) | Backend contract & dispatch | opaque keys, complete list, pure-function routing, no migration |
| [004](decisions/004-two-backends-no-third.md) | Two backends, no third | scope boundary against manifest-layer absorption | | [004](decisions/004-two-backends-no-third.md) | Two backends, no third | scope boundary against manifest-layer absorption |
| [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Namespace-blindness | backends see hashes only | | [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Namespace-blindness | backends see hashes only |
| [007](decisions/007-two-kv-engines-sqlite-and-postgres.md) | Two kv engines | sqlite (default) + postgres behind one trait; one relational engine + fs per node |
## Open Questions ## Open Questions
@@ -148,6 +183,10 @@ externally; OQ-08).
- `docs/research/poc-trait-dispatch-findings.md` findings 1/2/6 - `docs/research/poc-trait-dispatch-findings.md` findings 1/2/6
- `docs/research/poc-largeblob-findings.md` findings A2/A4 (+ the - `docs/research/poc-largeblob-findings.md` findings A2/A4 (+ the
re-runnable benchmark harness) re-runnable benchmark harness)
- `docs/research/poc-postgres-kv-findings.md` — the pg arm's measured
curves + engine-posture deltas (ADR-007's evidence base)
- `docs/research/poc-redb-kv-findings.md` — redb ruled out at a
durability-tier mismatch (POC #6; recorded in ADR-007)
- `docs/research/iroh-blobs-eval.md` — the fs-layout conclusions borrowed - `docs/research/iroh-blobs-eval.md` — the fs-layout conclusions borrowed
(sharding, crash ordering, inline thresholds rejected as weld) (sharding, crash ordering, inline thresholds rejected as weld)
- rudolfs notes — `list()` anti-lesson, decorator alternative noted and - rudolfs notes — `list()` anti-lesson, decorator alternative noted and
@@ -2,7 +2,9 @@
## Status ## Status
Accepted Accepted (engine pin amended by ADR-007 — the kv tier ships two
engines, sqlite and postgres, behind this contract; the dispatch layer
and trait are unchanged)
## Context ## Context
@@ -53,7 +55,8 @@ Evidence in hand (no further POC or consumer-waiting needed):
faster than fs at 1-16 KiB (POC #3 A4, first-party benchmark), faster than fs at 1-16 KiB (POC #3 A4, first-party benchmark),
the regime holding most git objects, workspace files, and manifests. the regime holding most git objects, workspace files, and manifests.
WAL + `synchronous=NORMAL` as shipped durability; prepared WAL + `synchronous=NORMAL` as shipped durability; prepared
statements; no-tables-yet tolerance. statements; no-tables-yet tolerance. (The tier's engine set was
widened to include postgres by ADR-007.)
- **`fs` (default-on)** for large blobs — flat - **`fs` (default-on)** for large blobs — flat
`{hex-prefix}/{hex-prefix}/{hash}` sharding (iroh's surviving `{hex-prefix}/{hex-prefix}/{hash}` sharding (iroh's surviving
conclusion); stage-then-commit-rename; pread range reads (sound conclusion); stage-then-commit-rename; pread range reads (sound
@@ -36,6 +36,15 @@ and by contract.** (A non-production ephemeral `mem` tier exists for
tests — ADR-003; it is not a storage story and does not extend this tests — ADR-003; it is not a storage story and does not extend this
boundary.) boundary.)
**Scope note (post-ADR-007 reconciliation): the "two" here counts
*tiers*, not engines.** The kv tier is one backend; ADR-007 widened the
engines behind it to a set (sqlite default + postgres, feature-gated)
so a node's kv rides its relational engine of choice. The count that
matters to this ADR — trait impls *this crate ships* as CAS tiers —
remains two (kv + fs); a second kv engine does not make a third
backend, and nothing in the engine set touches the manifest-layer
boundary below.
1. **Two production backends is the complete set.** kv/sqlite (small) 1. **Two production backends is the complete set.** kv/sqlite (small)
+ fs (large) cover the scale economics with measured evidence + fs (large) cover the scale economics with measured evidence
(ADR-003); no other physical tier is needed by any current or (ADR-003); no other physical tier is needed by any current or
@@ -0,0 +1,177 @@
# ADR-007: Two kv engines — sqlite and postgres — one trait, constructor-selected
## Status
Accepted
## Context
ADR-003/004 settled the crate's physical shape: two backends (kv for
small blobs, fs for large), size-threshold dispatch between them, and
the "two backends" count as a *tier* count. Within the kv tier, sqlite
was pinned as the shipped engine, on single-connection economics (POC
#3 A4). That pin left a real downstream shape unserved: the classical
stack for the consumers this crate exists for — git servers, vfs/
appfile nodes, replicators — is *three* storage systems (a kv for small
content, fs for large content, a relational database for refs,
manifests, ACLs, queues). Forcing such a node to run sqlite *beside*
its relational engine multiplies engines, WAL files, backup stories,
and ops postures for no benefit.
POC #5 (`poc-postgres-kv-findings.md`, findings B1-B6) measured
postgres holding the kv tier's full contract and POC #6
(`poc-redb-kv-findings.md`, findings C1-C6) ruled out redb: its
durability API cannot express the shipped tier (crash-consistent
without per-commit fsync). Together with POC #3 A4 this is four-way
triangulated engine evidence, complete.
The deciding facts are all in hand and none is a fact only a future
deployment of this crate could create (the deferral rule — see the
Schrödinger's-code corollary in README.md, and the OQ-09 precedent):
- **Topology, not throughput, is the trigger** — but topology is a
property of the node the operator builds *today*, not a future
event: a private single-machine node has one writer and sqlite wins
by measurement (44k solo puts/s at 1-16 KiB, POC #3 A4, reproduced
in POC #5); a multi-machine node writing one pool needs postgres
structurally, not merely faster (sqlite's WAL single-writer ceiling
~1.2k objects/s degrades under contention and flatlines — latency
climbs without bound — while postgres scales near-linearly to ~37k
puts/s across workers, POC #5 B3/B4).
- **The engine cost delta is measured and bounded**: ~180-line impl on
the same table shape (B5), with named posture deltas (see Decision).
- **The sweep-safety gate is testable in CI**, not in production: list
correctness is observable only through GC (POC #1 finding 2), and
the exact-count sweep-outcome test shape runs against any engine the
contract covers — including dockerized postgres under `--all-features`.
Recording this as an open question gated on "a replicator-shaped
deployment materializes" would be circular hedging: such a deployment
runs alkblobs, so it cannot cross the trigger until the pg engine
ships, and the engine would not ship until the deployment exists.
That deferral shape was already rejected once, on the same reasoning,
in OQ-09.
## Decision
**The kv tier ships two engines — sqlite (default) and postgres —
behind one `Backend` trait; engine selection is a constructor
parameter on the backend's configuration.** A node provisions one
relational engine plus the fs tier; the kv tier rides whichever
engine the operator chose. This collapses the classical three-engine
downstream stack (kv + fs + relational) to two, and makes which
relational engine a per-node choice rather than a crate fork.
1. **sqlite stays the default engine**: zero-ops, no daemon, solo
economics (44k solo puts/s at 1-16 KiB), and consumer co-tenancy
rides the same file (honker's WAL-NORMAL defaults are literally the
shipped tier). Single-machine nodes are sqlite by measurement, and
remain so even at sustained rates well above observed push traffic
(~1.2k/s WAL ceiling vs. single-node push rates an order below it).
2. **postgres ships as an engine, feature-gated (`postgres`,
default-off)**: `tokio-postgres` + `deadpool-postgres`, the POC #5
arm's stack. It exists for the topology cases measured in POC #5
B3/B4: cross-machine write access to one pool, many-client
replicator/hub nodes, and consolidated-ops deployments already
running postgres. A postgres node's consumer tables (refs,
manifests, queues) co-tenant the same instance by choice — the
deployment's own consolidation, beside the pool per ADR-004.
3. **The `Backend` trait does not change.** Both engines implement the
identical contract: opaque byte keys/values, CAS semantics
(`INSERT ... ON CONFLICT DO NOTHING` maps sqlite's
`INSERT OR IGNORE`), complete `list()`, GC-participating `delete`,
virgin-store no-op reads, namespace-blindness. The dispatch layer
(size threshold, kv↔fs routing, fall-through) is engine-blind and
unchanged.
4. **The pg engine's impl contract carries POC #5 finding B5's deltas
as requirements** (they are the difference between a port and an
engine posture):
- `synchronous_commit` posture per pool/session, configurable —
the parity knob with sqlite's `synchronous=NORMAL`-WAL shipped
tier (latency moves ~2-3× across settings on the POC disk);
- pooled connections with per-connection prepared-statement
discipline (fresh sessions cost ~20 ms, B2; prepared statements
die with sessions — deadpool's statement cache encodes the
pattern);
- autovacuum tuning stated for the churning CAS table, plus
`fillfactor` (90 measured) — a real maintenance surface sqlite
does not have;
- ~40× storage overhead at the 1-4 KiB tier accepted as the
small-tier cost of a page-based server engine (B6) — documented,
not hidden.
5. **Sweep-safety proof per ADR-003's admission gate**: for pg, `list()`
is a stable `SELECT key` cursor over a vacuuming table (MVCC-ghost
stability across a long sweep), batch-delete via `key = ANY($1)`;
the exact-count sweep-outcome tests (store-api.md invariants 4/5)
run in CI against dockerized postgres under `--all-features`, same
gate shape as every backend.
6. **Non-production posture defaults are named, deferred costs** (the
ADR-006 pattern, not hedges): the B5 tuning numbers are POC-derived
from one dev-box disk; the first real pg deployment verifies them
against its media and records deltas. No decision here waits on
that verification.
7. **redb stays ruled out** (POC #6) — recorded here so the question
does not reopen implicitly: its durability API cannot express the
shipped crash-consistent-without-per-commit-fsync tier, and its
~40/s write scale-out is below even sqlite's WAL ceiling.
The ADR-003 substitution door ("a downstream wanting a different
engine there would be new ADRs") is now exercised in-repo, ahead of
any downstream needing it: the downstream's third-engine question
collapses to its own deployment choice among shipped engines.
## Consequences
**Positive**
- The classical downstream stack lands at two engines per node (one
relational + fs), eliminating the "sqlite *beside* the real
database" duplication this crate's consumers would otherwise carry
forever.
- Per-node engine choice (constructor parameter, per ADR-003/004)
makes a heterogeneous fleet — personal nodes on sqlite, multi-tenant
hubs on postgres — a deployment topology, not a fork; everything
above the trait is written once and is engine-blind.
- The evidence for both engines is first-party measured (POC #3 A4,
POC #5 B1-B6, POC #6 C1-C6) and complete; this ADR stands on
evidence in hand, no deferred decision remains in the engine story.
- CI covers the pg backend's contract via the same sweep-outcome test
gate as sqlite, behind `--all-features`.
**Negative**
- The pg engine adds a dependency cluster (tokio-postgres,
deadpool-postgres) behind a default-off feature — the base crate
stays lean per convention 8, but the crate now carries a second
engine impl and its posture surface to maintain.
- Postgres storage overhead at the small tier (~40×) is real (B6) and
accepted; deployments dominated by tiny blobs on constrained disks
stay better served by sqlite.
- Two engines means two durability/ops postures to keep honest across
upstream changes (sqlite version/file-compat upstream guarantees;
pg autovacuum behavior is config-dependent).
**Neutral**
- Implementation sequencing: sqlite is Phase 1's mainline; the pg impl
is parallelizable against the fixed contract (POC #5 scaffold
exists). No consumer wait gates it; it lands as normal Phase 1
implementation work.
- A third engine (S3-like, etc.) remains the ADR-003 path: a new ADR
with its own sweep-safety proof. Nothing in this decision closes or
opens that door further.
## References
- `docs/research/poc-postgres-kv-findings.md` (POC #5, findings B1-B6)
- `docs/research/poc-redb-kv-findings.md` (POC #6, findings C1-C6)
- `docs/research/poc-largeblob-findings.md` (POC #3, finding A4)
- `docs/research/poc-trait-dispatch-findings.md` (POC #1, finding 2)
- ADR-003 (the trait contract and dispatch this decision rides),
ADR-004 (tier count ≠ engine count; the beside-the-pool boundary),
ADR-006 (the deferred-cost pattern this decision's posture defaults
follow)
- [open-questions.md](../open-questions.md) — OQ-10, resolved by this
ADR; OQ-09 (the anti-circularity precedent)
- [backends-and-dispatch.md](../backends-and-dispatch.md) — the spec
surface this ADR amends
+36 -50
View File
@@ -31,10 +31,10 @@ make before shipping. Phase 0's residual list was swept under this
rule: every residual either resolved into an ADR (the evidence was rule: every residual either resolved into an ADR (the evidence was
already in hand) or re-owned as below. already in hand) or re-owned as below.
**Index of active OQs:** OQ-07, OQ-08, OQ-10 (externally-owned, carried **Index of active OQs:** OQ-07, OQ-08 (externally-owned, carried
for visibility). OQ-09 is resolved (recorded below). Promoted Phase 0 for visibility). OQ-09 and OQ-10 are resolved (recorded below).
questions OQ-BL-01..06 are recorded here with their resolutions for Promoted Phase 0 questions OQ-BL-01..06 are recorded here with their
traceability. resolutions for traceability.
--- ---
@@ -81,51 +81,33 @@ traceability.
- **Origin**: `docs/research/poc-postgres-kv-findings.md`, - **Origin**: `docs/research/poc-postgres-kv-findings.md`,
`poc-redb-kv-findings.md`, [backends-and-dispatch.md](backends-and-dispatch.md) `poc-redb-kv-findings.md`, [backends-and-dispatch.md](backends-and-dispatch.md)
- **Status**: externally-owned (the deployment trigger) — carried here - **Status**: resolved — by ADR-007 (2026-10-02)
as a standing offer, not a blocker - **Resolution**: **the kv tier ships two engines — sqlite (default)
- **Priority**: medium-high (the evidence base is done; only the and postgres (feature `postgres`, default-off) — behind one
deployment trigger is outstanding) `Backend` trait; engine selection is a per-node constructor
- **Owner**: whichever consumer deployment crosses the trigger parameter** ([ADR-007](decisions/007-two-kv-engines-sqlite-and-postgres.md)).
(per OQ-07: alkgit's replicator is the named candidate) The evidence base was already complete (POC #3 A4's sqlite solo
- **Question**: when a replicator-shaped deployment materializes curves; POC #5 B1-B6's pg concurrency scale-out and posture deltas;
(multi-tenant public node: many users pushing, gossip catch-up, bulk POC #6 C1-C6 ruling out redb at a durability-tier mismatch) — all
ingest), open the postgres-kv ADR. The measured case for it: deciding facts were in hand, and the only thing the prior framing
sqlite's WAL single-writer ceiling is ~1.2k objects/s under left outstanding was this crate's own build sequence, which is
contention and *flatlines* (latency climbs without bound under implementation sequencing, not an open architecture question.
overload) while postgres scales out to ~28-37k puts/s across - **Why this is not Schrödinger's code** (recorded because the
workers with group-commit amortization (POC #5 findings B3/B4); question was once framed as an externally-owned standing offer): the
single-user/private/small-replicator nodes stay sqlite by proposed trigger — "a replicator-shaped deployment materializes" —
measurement (POC #5 A4/B1). Also settled by the same POCs: redb is is a fact only this crate could create (a deployment *running
ruled out (POC #6 — its durability API cannot express the shipped alkblobs* capable of pg cannot pre-exist the pg engine), so gating
WAL-NORMAL-tier: crash-consistent without per-commit fsync), and on it was circular hedging, the exact corollary the README names and
the trigger is **topology, not throughput** — cross-MACHINE write the precedent OQ-09 resolved on. The topology-not-throughput
access to one pool is the condition that makes postgres mandatory decision rule and the measured curves stand on evidence in hand;
rather than merely faster. the deployment is where the *costs* named by the ADR (POC-derived
- **Shape of the work when triggered** (per-node choice is a posture defaults) get verified — a deferred cost in the ADR-006
constructor parameter, per ADR-003/004; the impl is parallelizable pattern, not a gating input.
against the fixed contract — two engines, one trait): - **Reopen condition** (not a parked question): a third engine is the
1. `PgKv` backend impl (~180 lines, POC #5 scaffold exists) ADR-003/007 substitution door — a new ADR with its own measured
with the engine-posture deltas POC #5 finding B5 named: evidence and sweep-safety proof. "Some future deployment might want
per-pool `synchronous_commit` posture, pooled-connection + it" does not open it.
prepared-statement discipline (prepared statements die with - **Cross-references**: OQ-07, ADR-003, ADR-004, ADR-007, POC #3, POC #5, POC #6
sessions), autovacuum tuning for a churning CAS table,
fillfactor, ~40× storage overhead accepted as the small-tier
cost;
2. the sweep-safety proof ADR-003's admission gate requires:
`list()` as a stable `SELECT key` cursor over a vacuuming table
(MVCC-ghost stability across a long sweep), batch-delete via
`key = ANY($1)`, exact-count sweep-outcome tests (the POC #1
finding-2 gate applied to pg);
3. consumer-layer note: honker's notify/queue/stream features ride
sqlite's file; the pg deployment's parity stack is
`LISTEN`/`NOTIFY` + pgboss-rs (honker's own README points
postgres users that direction) — a deployment concern, not a
store-crate one.
- **Resolution**: none — waiting on the deployment trigger. The ADR
opens the day a real replicator consumer exists to keep its
sweep-safety proof honest; building it earlier would leave an impl
with no first consumer.
- **Cross-references**: OQ-07, ADR-003, ADR-004, POC #5, POC #6
--- ---
@@ -236,13 +218,17 @@ traceability.
|---|---|---| |---|---|---|
| OQ-07 | externally-owned | alkgit's architecture process (answerable on paper anytime; gates nothing here) | | OQ-07 | externally-owned | alkgit's architecture process (answerable on paper anytime; gates nothing here) |
| OQ-08 | externally-owned | alkfs Phase 0 intake | | OQ-08 | externally-owned | alkfs Phase 0 intake |
| OQ-10 | externally-owned (standing offer) | the first multi-tenant replicator deployment (POC #5 is the ready evidence base; the ADR opens on the trigger) |
OQ-09 was `deferred(scope)` on the first embedded ops deployment — a OQ-09 was `deferred(scope)` on the first embedded ops deployment — a
deciding fact that could only exist once the ops module ships, i.e. a deciding fact that could only exist once the ops module ships, i.e. a
wait that never collapses. It resolved to closed-by-default (see its wait that never collapses. It resolved to closed-by-default (see its
entry above); its tracker task is deleted. entry above); its tracker task is deleted.
OQ-10 was briefly recorded as an externally-owned "standing offer" on
a deployment trigger — the same invalid shape, caught at review: the
trigger was a fact only this crate could create. It resolved to
ADR-007 (two kv engines; see its entry above).
OQ-07/OQ-08 are owned by other repos' processes and are not alkblobs OQ-07/OQ-08 are owned by other repos' processes and are not alkblobs
tracker tasks (their outcome arrives *through* their owners, not tracker tasks (their outcome arrives *through* their owners, not
through any artifact this repo creates). through any artifact this repo creates).
+12 -4
View File
@@ -60,7 +60,10 @@ or three times.
│ size-threshold dispatch (ADR-003) │ │ size-threshold dispatch (ADR-003) │
└───────┬───────────────────┬───────────────────────────────┘ └───────┬───────────────────┬───────────────────────────────┘
▼ ▼ ▼ ▼
kv backend fs backend (ADR-003: exactly two) kv backend fs backend (ADR-003: exactly two tiers;
│ ADR-007: kv = 2 engines)
▼
engine per node: sqlite (default) | postgres (feature "postgres")
transport: none, anywhere. alkgit/alkfs or the embedder dials; transport: none, anywhere. alkgit/alkfs or the embedder dials;
an alkcall Connection is handed in at the ops boundary. an alkcall Connection is handed in at the ops boundary.
@@ -71,13 +74,15 @@ or three times.
| Layer | Dependencies | Notes | | Layer | Dependencies | Notes |
|---|---|---| |---|---|---|
| base crate (store core + backends) | `tokio`, `thiserror`, `parking_lot`, plus `rusqlite` behind the `kv` feature and nothing behind `fs`/`mem` | no alkcall, no serialization frameworks | | base crate (store core + backends) | `tokio`, `thiserror`, `parking_lot`, plus `rusqlite` behind the `kv` feature and nothing behind `fs`/`mem` | no alkcall, no serialization frameworks |
| `postgres` feature (ADR-007) | adds `tokio-postgres` + `deadpool-postgres` behind the kv tier's second engine, default-off | same `Backend` contract, own sweep-safety CI gate under `--all-features` |
| `ops` feature | adds `alkcall` (feature-gated, default-off) | JSON op specs + binary channels + `AccessControl`; ADR-001 | | `ops` feature | adds `alkcall` (feature-gated, default-off) | JSON op specs + binary channels + `AccessControl`; ADR-001 |
| consumer crates | depend on alkblobs (and on alkcall directly when they speak ops) | alkgit, alkfs | | consumer crates | depend on alkblobs (and on alkcall directly when they speak ops) | alkgit, alkfs |
Backends are feature-gated; both `kv` and `fs` ship default-on Backends are feature-gated; both `kv` and `fs` ship default-on
(batteries-included, the alkgit feature-model pattern); the `ops` (batteries-included, the alkgit feature-model pattern); the `kv` tier's
feature is default-off. Wasm: the backends are not a wasm story; a wasm postgres engine (`postgres`, ADR-007) and the `ops` feature are
client is a consumer of the `ops` surface (which rides alkcall, itself default-off. Wasm: the backends are not a wasm story; a wasm client is a
consumer of the `ops` surface (which rides alkcall, itself
wasm-clean), never an in-process embedder (ADR-001 §Consequences). wasm-clean), never an in-process embedder (ADR-001 §Consequences).
## Non-goals (each enforced by an ADR) ## Non-goals (each enforced by an ADR)
@@ -105,6 +110,7 @@ wasm-clean), never an in-process embedder (ADR-001 §Consequences).
| [004](decisions/004-two-backends-no-third.md) | Two backends, no third | manifest layers are consumer-side | | [004](decisions/004-two-backends-no-third.md) | Two backends, no third | manifest layers are consumer-side |
| [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Pooled CAS & GC | flat pool, registered liveness, delete windows | | [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Pooled CAS & GC | flat pool, registered liveness, delete windows |
| [006](decisions/006-verification-posture-and-transfer-encoding.md) | Verification posture | whole-blob verification; chunk trees excluded by scoping | | [006](decisions/006-verification-posture-and-transfer-encoding.md) | Verification posture | whole-blob verification; chunk trees excluded by scoping |
| [007](decisions/007-two-kv-engines-sqlite-and-postgres.md) | Two kv engines | sqlite default + postgres behind one trait; one relational engine + fs per node |
## Open Questions ## Open Questions
@@ -117,6 +123,8 @@ crate level:
- **OQ-09**: default namespace visibility in network ops — - **OQ-09**: default namespace visibility in network ops —
resolved closed-by-default (deny-unlisted; reopen only on a named resolved closed-by-default (deny-unlisted; reopen only on a named
open-read requirement) open-read requirement)
- **OQ-10**: second kv engine — resolved by ADR-007 (sqlite + postgres
shipped behind one trait; per-node constructor choice)
## References ## References
+12
View File
@@ -617,6 +617,18 @@ self-contained POC runs as a standalone crate in the global workspace
with findings written into `docs/research/` here. Findings always land with findings written into `docs/research/` here. Findings always land
in `docs/research/` regardless of where the code lives. in `docs/research/` regardless of where the code lives.
Post-convergence promotion (2026-10-02, Phase 1): #5's pg engine case
and #6's redb guardrail became an accepted decision — ADR-007 (two kv
engines: sqlite default + postgres feature-gated, behind one trait,
constructor-selected; "two backends" counts tiers, not engines). An
interim framing recorded in OQ-10 (postgres as an externally-owned
"standing offer" opening on a first multi-tenant replicator
deployment) was caught at review as circular hedging — the trigger
was a fact only this crate could create — and dissolved on the same
reasoning as the Schrödinger's-code rule's README corollary (the
OQ-09 precedent). The POC evidence needed no strengthening; only the
decision's recording did.
## Convergence ## Convergence
Phase 0's final step per `docs/sdd_process.md`: *converge on a Phase 0's final step per `docs/sdd_process.md`: *converge on a
+33 -12
View File
@@ -238,12 +238,14 @@ a deployment running the *fs tier anyway* has none of either cost.
differences (synchronous_commit posture, pooling discipline, differences (synchronous_commit posture, pooling discipline,
autovacuum surface, ~40× storage floor) are the ADR content any autovacuum surface, ~40× storage floor) are the ADR content any
postgres Backend impl would carry. postgres Backend impl would carry.
- **No ADR action recommended from a single POC.** The decision is - **Decision status (as originally recorded): no ADR action from a
Phase 1's when a replicator-shaped consumer exists to measure against; single POC at the time of writing** — sqlite stayed the pinned
this POC's register entry is the evidence input, not a recommendation. engine, postgres the named candidate. *Amended 2026-10-02:* the
SQLite stays the pinned shipped engine (single-node economics, no decision has since been made on the accumulated evidence of this
daemon to run, zero-ops); postgres is the *named future ADR* if the POC + POC #6 (with POC #3's A4 sqlite baseline) — ADR-007 ships
multi-client replicator shape materializes. postgres as the kv tier's second engine (feature-gated,
constructor-selected); this section's measured content above stands
as the ADR's evidence base.
- **OQ-BL-02 residuals**: unchanged in substance. The one new residual: - **OQ-BL-02 residuals**: unchanged in substance. The one new residual:
if a postgres Backend were adopted, its GC-sweep implementation is if a postgres Backend were adopted, its GC-sweep implementation is
`TRUNCATE`-friendly (whole-table dead-set rebuilds) but the `TRUNCATE`-friendly (whole-table dead-set rebuilds) but the
@@ -253,7 +255,8 @@ a deployment running the *fs tier anyway* has none of either cost.
## Engine deployment posture (recorded 2026-10-02 — the two-engine story, post-POC #4/#6) ## Engine deployment posture (recorded 2026-10-02 — the two-engine story, post-POC #4/#6)
The three alkblobs-relevant git/vfs use cases, mapped onto the measured The three alkblobs-relevant git/vfs use cases, mapped onto the measured
curves; this is the conclusion OQ-10 carries: curves; this was the conclusion OQ-10 carried before its resolution
in ADR-007):
| Engine | Covers | Evidence | | Engine | Covers | Evidence |
|---|---|---| |---|---|---|
@@ -278,11 +281,29 @@ that are engine-blind. Work-above-the-trait (store core, GC mechanism,
ops, consumers) is written once; engine work parallelizes against the ops, consumers) is written once; engine work parallelizes against the
fixed contract (two engines, one trait, same sweep-safety proof shape). fixed contract (two engines, one trait, same sweep-safety proof shape).
Sequencing (recorded in OQ-10): sqlite stays Phase 1's mainline; the pg The deployment economics paragraph this table encodes, now stated
ADR opens on the deployment trigger, not before — building it earlier outright (recorded in ADR-007): the classical downstream stack — a git
would leave an impl with no first consumer to keep the sweep-safety server, a vfs node — needs three storage systems (kv + fs + relational);
proof honest. The evidence base (this doc + POC #6's redb guardrail) is this design collapses the kv into the relational engine, so a node
complete and waiting. provisions **one relational engine + fs**, and which relational engine
is the operator's per-node choice among the shipped engines.
Sequencing (amended 2026-10-02 by ADR-007): postgres **ships** as the
kv tier's second engine (feature `postgres`, default-off),
constructor-selected per node — sqlite stays the default and Phase 1's
mainline; the pg impl is normal Phase 1 implementation work,
parallelizable against the fixed contract (two engines, one trait,
same sweep-safety proof shape, CI-gated via `--all-features`). The
earlier framing here and in OQ-10 — a "standing offer" whose ADR
opens on a first multi-tenant replicator deployment — was caught at
review as circular hedging: such a deployment runs alkblobs, so it
cannot cross the trigger until the pg engine ships, which the trigger
then forbids shipping. Superseded by
[ADR-007](../architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md),
which records the decision this POC's evidence supports; the
deployment-trigger rule survives in corrected form as deployment
guidance (which engine a *node* chooses — ADR-007's decision rule),
not as a gate on the crate's own work.
## POC quality notes ## POC quality notes
+7 -5
View File
@@ -205,11 +205,13 @@ go up.
- **Reads: no case for redb on merit** — page-cache reads are what any - **Reads: no case for redb on merit** — page-cache reads are what any
of these give the large tier for free; sqlite's 19 µs probe was of these give the large tier for free; sqlite's 19 µs probe was
already fine. already fine.
- **ADR action: none.** SQLite's pin now has four-way triangulated - **Anti-recommendation adopted (ADR-007, 2026-10-02): redb stays
evidence (POC #3 fs, POC #4 pg, POC #6 redb, plus the lineage ruled out** — recorded in the decision so the question does not
citation). If a future consumer wanted redb-shaped durability reopen implicitly. SQLite's engine pin now has four-way triangulated
trade-offs (ephemeral cache tier), that's a *different* tier than evidence (POC #3 fs, POC #4/#5 pg, POC #6 redb, plus the lineage
the kv engine slot. citation), and ADR-007 ships the kv tier as sqlite + postgres. If a
future consumer wanted redb-shaped durability trade-offs (ephemeral
cache tier), that's a *different* tier than the kv engine slot.
## POC quality notes ## POC quality notes