docs(architecture): OQ-10 — postgres as the second shipped kv engine, a standing offer on a deployment trigger

Records the two-engine story the POC #5/#6 round established, in the
terms this crate's boundary uses (engine, not backend; trait contract
as the admission gate):

- OQ-10 (externally-owned standing offer): open the postgres-kv ADR
  when a multi-tenant replicator deployment materializes. Trigger is
  topology, not throughput — cross-machine write access to one pool
  (throughput pressure / sqlite's flatline is the early warning).
  Names the trigger-time work: PgKv impl + engine-posture deltas
  (synchronous_commit, pooling/prepared-statement discipline,
  autovacuum, ~40x storage overhead) + the sweep-safety proof
  (vacuum-stable list() cursor, exact-count sweep tests). Sequenced:
  sqlite stays Phase 1 mainline; evidence base is done and waiting.
- poc-postgres-kv-findings.md: engine deployment posture section —
  the three use cases (self-hosted git serving, p2p replicator nodes,
  agent-workspace pools) mapped onto the measured curves; per-node
  choice as the form of the fork; honker parity stack named for the
  pg side (LISTEN/NOTIFY + pgboss-rs, deployment layer).

No code changes; engine work parallelizes against the fixed contract.
This commit is contained in:
glm-5.3-flash committed 2026-10-02 18:46:40 +00:00
1 parent 5eea656295
commit 1af9399900
2 files changed
+87 -2

No files matched your search

+53 -2
View File
@@ -31,8 +31,8 @@ make before shipping. Phase 0's residual list was swept under this
rule: every residual either resolved into an ADR (the evidence was
already in hand) or re-owned as below.
**Index of active OQs:** OQ-07, OQ-08 (externally-owned, carried for
visibility). OQ-09 is resolved (recorded below). Promoted Phase 0
**Index of active OQs:** OQ-07, OQ-08, OQ-10 (externally-owned, carried
for visibility). OQ-09 is resolved (recorded below). Promoted Phase 0
questions OQ-BL-01..06 are recorded here with their resolutions for
traceability.
@@ -77,6 +77,56 @@ traceability.
liveness seams (ADR-003/004/005) without baking an alkfs shape in.
- **Cross-references**: OQ-07, ADR-004
### OQ-10: Postgres as the second shipped kv engine
- **Origin**: `docs/research/poc-postgres-kv-findings.md`,
`poc-redb-kv-findings.md`, [backends-and-dispatch.md](backends-and-dispatch.md)
- **Status**: externally-owned (the deployment trigger) — carried here
as a standing offer, not a blocker
- **Priority**: medium-high (the evidence base is done; only the
deployment trigger is outstanding)
- **Owner**: whichever consumer deployment crosses the trigger
(per OQ-07: alkgit's replicator is the named candidate)
- **Question**: when a replicator-shaped deployment materializes
(multi-tenant public node: many users pushing, gossip catch-up, bulk
ingest), open the postgres-kv ADR. The measured case for it:
sqlite's WAL single-writer ceiling is ~1.2k objects/s under
contention and *flatlines* (latency climbs without bound under
overload) while postgres scales out to ~28-37k puts/s across
workers with group-commit amortization (POC #5 findings B3/B4);
single-user/private/small-replicator nodes stay sqlite by
measurement (POC #5 A4/B1). Also settled by the same POCs: redb is
ruled out (POC #6 — its durability API cannot express the shipped
WAL-NORMAL-tier: crash-consistent without per-commit fsync), and
the trigger is **topology, not throughput** — cross-MACHINE write
access to one pool is the condition that makes postgres mandatory
rather than merely faster.
- **Shape of the work when triggered** (per-node choice is a
constructor parameter, per ADR-003/004; the impl is parallelizable
against the fixed contract — two engines, one trait):
1. `PgKv` backend impl (~180 lines, POC #5 scaffold exists)
with the engine-posture deltas POC #5 finding B5 named:
per-pool `synchronous_commit` posture, pooled-connection +
prepared-statement discipline (prepared statements die with
sessions), autovacuum tuning for a churning CAS table,
fillfactor, ~40× storage overhead accepted as the small-tier
cost;
2. the sweep-safety proof ADR-003's admission gate requires:
`list()` as a stable `SELECT key` cursor over a vacuuming table
(MVCC-ghost stability across a long sweep), batch-delete via
`key = ANY($1)`, exact-count sweep-outcome tests (the POC #1
finding-2 gate applied to pg);
3. consumer-layer note: honker's notify/queue/stream features ride
sqlite's file; the pg deployment's parity stack is
`LISTEN`/`NOTIFY` + pgboss-rs (honker's own README points
postgres users that direction) — a deployment concern, not a
store-crate one.
- **Resolution**: none — waiting on the deployment trigger. The ADR
opens the day a real replicator consumer exists to keep its
sweep-safety proof honest; building it earlier would leave an impl
with no first consumer.
- **Cross-references**: OQ-07, ADR-003, ADR-004, POC #5, POC #6
---
## Theme: ops surface
@@ -186,6 +236,7 @@ traceability.
|---|---|---|
| OQ-07 | externally-owned | alkgit's architecture process (answerable on paper anytime; gates nothing here) |
| OQ-08 | externally-owned | alkfs Phase 0 intake |
| OQ-10 | externally-owned (standing offer) | the first multi-tenant replicator deployment (POC #5 is the ready evidence base; the ADR opens on the trigger) |
OQ-09 was `deferred(scope)` on the first embedded ops deployment — a
deciding fact that could only exist once the ops module ships, i.e. a
+34
View File
@@ -250,6 +250,40 @@ a deployment running the *fs tier anyway* has none of either cost.
list()-complete-by-contract test would need the vacuum-aware
equivalent of POC #1's exact-count sweep assertions.
## Engine deployment posture (recorded 2026-10-02 — the two-engine story, post-POC #4/#6)
The three alkblobs-relevant git/vfs use cases, mapped onto the measured
curves; this is the conclusion OQ-10 carries:
| Engine | Covers | Evidence |
|---|---|---|
| **sqlite** | self-hosted git serving (gitea-replacement scale), agent-workspace pools, small/personal replicators | 44k solo puts/s at 1-16 KiB (POC #3 A4, reproduced here); WAL single-writer ceiling ~1.2k/s under contention is comfortably above single-node push rates; zero-ops, no daemon; honker rides the same file (same WAL-NORMAL posture, its defaults are literally the shipped tier) |
| **postgres** | public multi-tenant replicator nodes (many users pushing, gossip catch-up, bulk ingest), consolidated-ops deployments (already-running pg) | near-linear put scale-out to ~21k at 4 conns, ~28k at 8, ~32k at 12, ~37k at 24-48 (B3); reads scale to ~44k at 24 conns (B4); the only engine whose throughput *increases* under concurrency |
| fs (tier, not engine) | ≥128-256 KiB content, packfiles | crossover measured (POC #3 A4), stable across #4/#6 reproductions |
The decision rule the two POCs establish: **the trigger is topology,
not throughput** — cross-MACHINE write access to one pool makes postgres
mandatory; sustained-rate pressure (sqlite's flatline failure mode:
latency climbs without bound under overload) is the early-warning that
precedes it; single-node/private/small-team cases stay sqlite by
measurement, with honker-parity features (notify/queue/stream) available
free on the sqlite file and via `LISTEN`/`NOTIFY` + pgboss-rs on a
future pg deployment (a consumer-layer concern either way — the store
crate ships no queue).
**Per-node choice is the form the fork takes** (constructor parameter,
ADR-003): a heterogeneous replicator fleet — personal nodes on sqlite,
community nodes on pg — syncs over the same wire above store-core seams
that are engine-blind. Work-above-the-trait (store core, GC mechanism,
ops, consumers) is written once; engine work parallelizes against the
fixed contract (two engines, one trait, same sweep-safety proof shape).
Sequencing (recorded in OQ-10): sqlite stays Phase 1's mainline; the pg
ADR opens on the deployment trigger, not before — building it earlier
would leave an impl with no first consumer to keep the sweep-safety
proof honest. The evidence base (this doc + POC #6's redb guardrail) is
complete and waiting.
## POC quality notes
- The inherited 14-test suite (12 integration + 2 git-CLI cross-checks)