docs(architecture): ADR-010/011 — I/O seams, GC-state home, vocabulary; pre-decomposition consistency round
ADR-010 (new): the Backend trait's I/O seams pinned before any code — 'get' returns a crate-internal read cursor (kv engines materialize; 'local' pread loop; pg-lo lo_get windows), 'put' has two named forms (whole-value kv / staged-put large); GC state is store-core-owned via a crate-internal contract-tested engine-state seam on the SQL-backed engines (backends never learn liveness — ADR-005 verbatim); the store core holds the joint entry+pin transaction (corrects ADR-009 §2's ownership statement); kv-tier fleet-validity rule added (mirror of ADR-008 §3); ops pin-token TTL race resolved (the token IS the pin; renewal rides have/re-put; pause-past-TTL falls to delete-then-recover); feature graph pinned (pg-lo does not imply postgres). ADR-011 (new): vocabulary pinned — tier (contract, exactly two) vs engine (concrete impl, exactly five); store instance / node / fleet split (fleet = pool-sharing, not node count — fixes requirements.md's self-contradiction with REQ-2); mem demoted from 'backend'/'testing tier' to the kv tier's third engine (contract-reference engine); fs tier renamed 'large' including the feature name (last free moment before code exists); constructor modes pinned (dual-tier default, kv-only, mem-only) plus the composite fleet-validity predicate. Consistency round across all specs and ADR-003/004/007/008/009: stale pg-lo passages resolved per ADR-009's supersession; ADR-008 owner 'node id' → store-instance id; threshold wording corrected (conservative edge, not midpoint, of the 128-256 KiB crossover zone); SweepReport shape specced; mem classification unified; 008/009 ADR files renamed to match the tier rename; README deferral-policy recap aligned (third category = decided-but-sequenced work, not a parking kind); POC crate list completed. Verification: two independent architecture review rounds (the first found 4 criticals — unpinned trait I/O shapes, missing fleet-state seam, node/fleet self-contradiction, pin-token/TTL conflict — all resolved; final round: zero criticals); all markdown links resolve; ADR tables complete (11 ADRs).
This commit is contained in:
1 parent
9bbe6f3ad9
commit
abf8c5ae9c
18 files changed
+1137
-336
No files matched your search
+55
-21
@@ -1,6 +1,6 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-03 (ADR-009 — pg-lo admitted on POC #7)
|
||||
last_updated: 2026-10-03 (ADR-010/011 — I/O seams + GC-state home; vocabulary pinned, fs → large, mem joins kv)
|
||||
---
|
||||
|
||||
# alkblobs — Architecture
|
||||
@@ -10,8 +10,9 @@ last_updated: 2026-10-03 (ADR-009 — pg-lo admitted on POC #7)
|
||||
**Phase 1 (Architecture).** Phase 0 (Exploration) converged — see
|
||||
`docs/research/phase-0.md` for the vision, the evidence trail, and the
|
||||
POC register (all POCs passed or were absorbed; POC code lives in
|
||||
standalone crates (`/workspace/alkblobs-trait-poc`,
|
||||
`/workspace/alkblobs-largeblob-poc`); findings live in
|
||||
standalone crates under `/workspace/` — `alkblobs-trait-poc`,
|
||||
`alkblobs-largeblob-poc`, `alkblobs-postgres-poc`, `alkblobs-redb-poc`,
|
||||
`alkblobs-pglo-poc`; findings live in
|
||||
`docs/research/`). No implementation exists yet (`src/lib.rs` is a
|
||||
placeholder).
|
||||
|
||||
@@ -24,14 +25,14 @@ there — not inherited as hedges:
|
||||
the family pattern, not deviation from it — and the ops surface rides
|
||||
alkcall, feature-gated.
|
||||
- **ADR-004** fixes the "two backends" framing: the crate ships exactly
|
||||
two backends (kv small / fs large — both required by measured scale
|
||||
two tiers (kv small / large — both required by measured scale
|
||||
economics); the path→hash mapping layer (the alkfs/alknet "vfs" shape)
|
||||
is a consumer-layer concern and must never become a third backend.
|
||||
- **ADR-007** widens the kv tier's engine set rather than hedging it:
|
||||
sqlite (default) + postgres (feature-gated) behind one trait. The
|
||||
deployment economics this records: a downstream node (git server, vfs)
|
||||
classically needed three storage systems — kv + fs + relational — and
|
||||
this design collapses them to **one relational engine + fs**, with the
|
||||
classically needed three storage systems — kv + large content + relational — and
|
||||
this design collapses the three to two: **one relational engine + large-tier storage**, with the
|
||||
engine a per-node constructor choice. Resolved from evidence in hand
|
||||
(POCs #3/#5/#6); the would-be "deployment trigger" framing was caught
|
||||
at review as circular hedging (a deployment running alkblobs cannot
|
||||
@@ -42,13 +43,13 @@ there — not inherited as hedges:
|
||||
planning fact predating all POCs): the `Backend` trait gains a
|
||||
`size` probe (before any backend ships data); fleet GC gets
|
||||
DB-backed pins, an advisory-locked single sweeper, and SQL-level
|
||||
delete arbitration (single-node postures unchanged); and the fs tier
|
||||
delete arbitration (single-node postures unchanged); and the large tier
|
||||
becomes engine-selectable (`local` default; `pg-lo` named candidate,
|
||||
admission-gated on POC #7). It also adds
|
||||
[requirements.md](requirements.md) — the REQ anchor that ends
|
||||
per-session re-derivation of consumer facts.
|
||||
- **ADR-009** (2026-10-03) admits `pg-lo` — postgres Large Objects as
|
||||
the fs tier's second engine (feature `pg-lo`, default-off) — on
|
||||
the large tier's second engine (feature `pg-lo`, default-off) — on
|
||||
POC #7's passed gate (`docs/research/poc-pglo-findings.md`):
|
||||
contract 10/10 exact-count tests, durable put ≈ fs durable put at
|
||||
≥1 MiB, descriptorless `lo_get` window gets, transactional LO
|
||||
@@ -59,16 +60,44 @@ there — not inherited as hedges:
|
||||
readers), catalog space reused-but-never-returned, autovacuum
|
||||
inherited. The admission door (ADR-003, exercised by ADR-007 and
|
||||
ADR-009) has now shipped both its tier-extending engines.
|
||||
- **ADR-010** (2026-10-03) pins the `Backend` trait's I/O seams —
|
||||
the last unpinned signatures before implementation: `get` returns a
|
||||
crate-internal **read cursor** (kv engines materialize; large-tier
|
||||
engines stream — `local`'s pread loop, pg-lo's `lo_get` windows), `put` has
|
||||
two named forms (whole-value for kv, **staged put** for large —
|
||||
stage-then-commit-rename promoted from behavior to contract); rules
|
||||
that **GC state is store-core-owned** (backends never learn
|
||||
liveness — ADR-005 verbatim) hosted by SQL-backed engines via a
|
||||
crate-internal contract-tested **engine-state companion seam**
|
||||
(schema/pin/window/lock clauses), rules **the store core holds the
|
||||
joint entry+pin transaction** (correcting ADR-009 §2's ownership
|
||||
statement), adds the **kv tier's fleet-validity rule** (mirror of
|
||||
ADR-008 §3 — postgres is the fleet engine; a sqlite-fleet posture
|
||||
is not on offer), resolves the **ops pin-token TTL race** (the
|
||||
token *is* the pin; renewal rides have/re-put; pause-past-TTL falls
|
||||
to delete-then-recover), and pins the **feature graph** (`pg-lo`
|
||||
does not imply `postgres`).
|
||||
- **ADR-011** (2026-10-03) pins the vocabulary: **tier** (contract +
|
||||
routing class — exactly two) vs **engine** (concrete impl — exactly
|
||||
five: kv's sqlite/postgres/mem, large's `local`/pg-lo);
|
||||
**mem demoted from "backend"/"testing tier" to the kv tier's
|
||||
third engine — the contract-reference engine**; **store instance** /
|
||||
**node** / **fleet** split so REQ-2's "one node may run multiple
|
||||
instances" phrasing is no longer self-contradictory (fleet =
|
||||
pool-sharing, not node count); and the **large tier is renamed
|
||||
`large`** (feature `fs` → `large`) — its routing criterion is
|
||||
length, not medium — with constructor modes (dual-tier default,
|
||||
kv-only, mem-only) pinned in the same ADR's table.
|
||||
|
||||
## Architecture Documents
|
||||
|
||||
| Document | Scope | Status |
|
||||
|---|---|---|
|
||||
| [overview.md](overview.md) | Purpose, consumers, layer map, dependency posture | draft |
|
||||
| [requirements.md](requirements.md) | Consumer requirements REQ-1..4, node/pool/fleet vocabulary | draft |
|
||||
| [requirements.md](requirements.md) | Consumer requirements REQ-1..4, pinned vocabulary (tier/engine/instance/node/fleet) | draft |
|
||||
| [hashing-and-keys.md](hashing-and-keys.md) | Canonical hash, key encoding (the one-way door) | draft |
|
||||
| [store-api.md](store-api.md) | Store facade: put/get/stat/range/pin/batch/sweep surface, errors, invariants | draft |
|
||||
| [backends-and-dispatch.md](backends-and-dispatch.md) | Backend trait contract, shipped backends, size-threshold dispatch | draft |
|
||||
| [backends-and-dispatch.md](backends-and-dispatch.md) | Backend trait contract, tiers + engines table, size-threshold dispatch, constructor modes | draft |
|
||||
| [gc-and-namespaces.md](gc-and-namespaces.md) | Pooled CAS, liveness seams, mark-and-sweep, delete windows | draft |
|
||||
| [ops-surface.md](ops-surface.md) | alkcall-backed have/need + fetch/put ops, ACL mapping | draft |
|
||||
| [open-questions.md](open-questions.md) | Centralized OQ tracker (incl. promoted Phase 0 register) | draft |
|
||||
@@ -84,8 +113,10 @@ there — not inherited as hedges:
|
||||
| [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Pooled CAS, liveness seams, mark-and-sweep, delete windows | Accepted |
|
||||
| [006](decisions/006-verification-posture-and-transfer-encoding.md) | Whole-blob verification; transfer encoding excluded by scoping | Accepted |
|
||||
| [007](decisions/007-two-kv-engines-sqlite-and-postgres.md) | Two kv engines (sqlite + postgres) behind one trait; constructor-selected | Accepted |
|
||||
| [008](decisions/008-trait-size-probe-fleet-gc-and-fs-engines.md) | Trait `size` probe; fleet GC (DB-backed pins, advisory-locked sweeper); fs-tier engines | Accepted |
|
||||
| [009](decisions/009-pg-lo-fs-tier-engine.md) | pg-lo — postgres Large Objects as the fs tier's second engine (admitted on POC #7) | Accepted |
|
||||
| [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | Trait `size` probe; fleet GC (DB-backed pins, advisory-locked sweeper); large-tier engines | Accepted |
|
||||
| [009](decisions/009-pg-lo-large-tier-engine.md) | pg-lo — postgres Large Objects as the large tier's second engine (admitted on POC #7) | Accepted |
|
||||
| [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | Backend I/O seams (read cursor, staged put); GC state store-core-owned via the engine-state seam; kv fleet-validity; pin-token renewal | Accepted |
|
||||
| [011](decisions/011-vocabulary-tiers-engines-and-fleet-state.md) | Vocabulary: tier/engine/instance/node/fleet; mem joins kv; large tier renamed `large`; constructor modes | Accepted |
|
||||
|
||||
## Open Questions
|
||||
|
||||
@@ -94,25 +125,28 @@ Tracked in [open-questions.md](open-questions.md). The Phase 0 register
|
||||
remain parked — OQ-07/OQ-08 externally-owned (alkgit seam mapping,
|
||||
alkfs intake; carried for visibility, gating nothing here). OQ-09
|
||||
(namespace-visibility default) resolved closed-by-default; OQ-10
|
||||
(second kv engine) resolved by ADR-007. The pg-lo fs-engine question
|
||||
(second kv engine) resolved by ADR-007. The pg-lo large-engine question
|
||||
resolved to *sequenced work*, not a parked question: REQ-2/ADR-008
|
||||
named it the candidate fs-tier engine and POC #7
|
||||
named it the candidate large-tier engine and POC #7
|
||||
(`docs/research/poc-pglo-spec.md`) was its admission evidence — a
|
||||
task, not a wait; that task completed 2026-10-03
|
||||
(`docs/research/poc-pglo-findings.md`, passed) and ADR-009 shipped
|
||||
the engine. **Deferral
|
||||
policy (the "Schrödinger's code" rule):** a *decision this crate needs
|
||||
before shipping* may not be deferred on a dependency that is itself
|
||||
waiting for this crate to exist. Two parking kinds remain legitimate:
|
||||
waiting for this crate to exist. Legitimate parked kinds:
|
||||
`deferred(scope)` on a deciding fact that exists independently of this
|
||||
crate, and externally-owned questions (how a consumer maps onto this
|
||||
crate) that gate no decision here — full definitions in the header of
|
||||
[open-questions.md](open-questions.md). A corollary worth restating,
|
||||
because it recurs: a fact this crate must create (its own first
|
||||
deployment, its own first consumer) is never a deciding input —
|
||||
decisions stand on evidence in hand, and future needs reopen via named
|
||||
requirements, not via waiting. (Corollary applications so far: OQ-09
|
||||
and OQ-10 both resolved on it.)
|
||||
[open-questions.md](open-questions.md). A third category is *not* a
|
||||
parked kind: **decided-but-sequenced work** (a resolved decision whose
|
||||
admission gate requires evidence — the pg-lo round's POC #7 is the
|
||||
worked example; a task with an owner, not a parked question). A
|
||||
corollary worth restating, because it recurs: a fact this crate must
|
||||
create (its own first deployment, its own first consumer) is never a
|
||||
deciding input — decisions stand on evidence in hand, and future needs
|
||||
reopen via named requirements, not via waiting. (Corollary
|
||||
applications so far: OQ-09 and OQ-10 both resolved on it.)
|
||||
|
||||
## Lifecycle
|
||||
|
||||
|
||||
Reference in new issue
Block a user