--- status: draft last_updated: 2026-10-11 notes: "ADR-013 — family composition: no sibling dependency in-core, sweep-lease semantics, co-tenancy/no-cross-store-atomicity rulings, OQ-11 resolved, feature graph re-pinned to engine features" --- # alkblobs — Architecture ## Current State **Phase 1 (Architecture).** Phase 0 (Exploration) converged — see `docs/research/phase-0.md` for the vision, the evidence trail, and the POC register (all POCs passed or were absorbed; POC code lives in standalone crates under `/workspace/` — `alkblobs-trait-poc`, `alkblobs-largeblob-poc`, `alkblobs-postgres-poc`, `alkblobs-redb-poc`, `alkblobs-pglo-poc`; findings live in `docs/research/`). No implementation exists yet (`src/lib.rs` is a placeholder). The converged Phase 0 posture has been promoted into the specs and ADRs below. Two framing corrections from the convergence review are captured there — not inherited as hedges: - **ADR-001** makes the substrate posture explicit: the store layer carries no alkcall dependency *by design* — which is conformance to the family pattern, not deviation from it — and the ops surface rides alkcall, feature-gated. - **ADR-004** fixes the "two backends" framing: the crate ships exactly two tiers (kv small / large — both required by measured scale economics); the path→hash mapping layer (the alkfs/alknet "vfs" shape) is a consumer-layer concern and must never become a third backend. - **ADR-007** widens the kv tier's engine set rather than hedging it: sqlite (default) + postgres (feature-gated) behind one trait. The deployment economics this records: a downstream node (git server, vfs) classically needed three storage systems — kv + large content + relational — and this design collapses the three to two: **one relational engine + large-tier storage**, with the engine a per-node constructor choice. Resolved from evidence in hand (POCs #3/#5/#6); the would-be "deployment trigger" framing was caught at review as circular hedging (a deployment running alkblobs cannot pre-exist the engine) and closed on the OQ-09 precedent. - **ADR-008** (2026-10-03) repairs the seams ADR-007 stressed, against the pinned consumer requirement REQ-2 (requirements.md — a replicator node may run a **fleet** of store instances sharing one pg pool, a planning fact predating all POCs): the `Backend` trait gains a `size` probe (before any backend ships data); fleet GC gets DB-backed pins, an advisory-locked single sweeper, and SQL-level delete arbitration (single-node postures unchanged); and the large tier becomes engine-selectable (`local` default; `pg-lo` named candidate, admission-gated on POC #7). It also adds [requirements.md](requirements.md) — the REQ anchor that ends per-session re-derivation of consumer facts. - **ADR-009** (2026-10-03) admits `pg-lo` — postgres Large Objects as the large tier's second engine (feature `pg-lo`, default-off) — on POC #7's passed gate (`docs/research/poc-pglo-findings.md`): contract 10/10 exact-count tests, durable put ≈ fs durable put at ≥1 MiB, descriptorless `lo_get` window gets, transactional LO lifecycle (zero crash orphans), companion-table contract authority. REQ-2's fleet consolidation option is now shipped: a fleet node can run postgres-only. Named deltas: cached-get 20–50× behind page-cache fs single-stream (aggregate serving ~700 MB/s at 16 readers), catalog space reused-but-never-returned, autovacuum inherited. The admission door (ADR-003, exercised by ADR-007 and ADR-009) has now shipped both its tier-extending engines. - **ADR-010** (2026-10-03) pins the `Backend` trait's I/O seams — the last unpinned signatures before implementation: `get` returns a crate-internal **read cursor** (kv engines materialize; large-tier engines stream — `local`'s pread loop, pg-lo's `lo_get` windows), `put` has two named forms (whole-value for kv, **staged put** for large — stage-then-commit-rename promoted from behavior to contract); rules that **GC state is store-core-owned** (backends never learn liveness — ADR-005 verbatim) hosted by SQL-backed engines via a crate-internal contract-tested **engine-state companion seam** (schema/pin/window/lock clauses), rules **the store core holds the joint entry+pin transaction** (correcting ADR-009 §2's ownership statement), adds the **kv tier's fleet-validity rule** (mirror of ADR-008 §3 — postgres is the fleet engine; a sqlite-fleet posture is not on offer), resolves the **ops pin-token TTL race** (the token *is* the pin; renewal rides have/re-put; pause-past-TTL falls to delete-then-recover), and pins the **feature graph** (`pg-lo` does not imply `postgres`). - **ADR-011** (2026-10-03) pins the vocabulary: **tier** (contract + routing class — exactly two) vs **engine** (concrete impl — exactly five: kv's sqlite/postgres/mem, large's `local`/pg-lo); **mem demoted from "backend"/"testing tier" to the kv tier's third engine — the contract-reference engine**; **store instance** / **node** / **fleet** split so REQ-2's "one node may run multiple instances" phrasing is no longer self-contradictory (fleet = pool-sharing, not node count); and the **large tier is renamed `large`** (feature `fs` → `large`) — its routing criterion is length, not medium — with constructor modes (dual-tier default, kv-only, mem-only) pinned in the same ADR's table. - **ADR-012** (2026-10-03) is the pre-decomposition consistency round: the full-tree review found five composition defects and this ADR rules each — sweeps are **never errors** (`Ok(SweepReport)` with `aborted: Option`; `GcAborted` retired from the error enum, direct-delete refusal renamed `GcRefuse`; `NoLivenessSources` added as the third cause variant); the pin token gains its **wire shape** (`blobs/put` response `{token, digest}`; `blobs/have`'s token-renewal form; token validity domain = the minting serving node); **fleet mode is an explicit constructor declaration** (`fleet: true`, never inferred from engine choice); **all fleet GC state hosts on the fleet's kv engine (postgres)** — one arbitration domain — with `large=pg-lo` fleet nodes required onto the same pg instance (the composite predicate's new clause) and `large=local` fleet puts pinned pin-row-first, publish-second; and the engine-state seam's sqlite rows are repaired (pin/sweep-lock bodies dropped; window staging for non-SQL engines is in-process). The tree is now single-valued on every implementation-facing signature. - **ADR-013** (2026-10-11, the composition round): with the sibling substrate **alkstore** shipped (v0.1.0 — the reactive coordination crate the first consumers co-deploy), this ADR rules the composition so nothing waits on anything downstream or upstream: the core keeps **no alkstore dependency and no reactive machinery** (notify/scheduler/queues/invalidation hooks are non-goals — cadence stays embedder-owned per ADR-005, with alkstore the named deployment-layer substrate for it); the fleet sweeper's single-flighter is pinned as **named exclusive sweep lease** semantics (shipped provider: the pg advisory-lock clause — provider swaps are new-ADR substitution doors); **co-tenancy and no-cross-store-atomicity** are ruled as the downstream-facing composition contract (consumer liveness tables co-tenant the fleet host; compensation rests on idempotent re-puts, windowed deletes, sweeps-as-report-data); and **OQ-11 is resolved** — engine features pull their own driver clusters (`sqlite` default-on carrying rusqlite; `kv`/`large` tier features retired; `local` ungated), so pg-pure builds are supported non-default configurations. REQ-5 records the composition facts; the parked index is now empty. ## Architecture Documents | Document | Scope | Status | |---|---|---| | [overview.md](overview.md) | Purpose, consumers, layer map, dependency posture | draft | | [requirements.md](requirements.md) | Consumer requirements REQ-1..4, pinned vocabulary (tier/engine/instance/node/fleet) | draft | | [hashing-and-keys.md](hashing-and-keys.md) | Canonical hash, key encoding (the one-way door) | draft | | [store-api.md](store-api.md) | Store facade: put/get/stat/range/pin/batch/sweep surface, errors, invariants | draft | | [backends-and-dispatch.md](backends-and-dispatch.md) | Backend trait contract, tiers + engines table, size-threshold dispatch, constructor modes, fleet-validity | draft | | [gc-and-namespaces.md](gc-and-namespaces.md) | Pooled CAS, liveness seams, mark-and-sweep, delete windows | draft | | [ops-surface.md](ops-surface.md) | alkcall-backed have/need + fetch/put ops, ACL mapping | draft | | [open-questions.md](open-questions.md) | Centralized OQ tracker (incl. promoted Phase 0 register) | draft | ## Decision Records | ADR | Decision | Status | |---|---|---| | [001](decisions/001-substrate-posture-and-ops-placement.md) | Substrate posture & ops-surface placement (conformance, not deviation) | Accepted | | [002](decisions/002-canonical-hash-and-key-encoding.md) | Canonical hash + key encoding (the wire-format ADR; precedes any consumer) | Accepted | | [003](decisions/003-backend-contract-and-dispatch.md) | Backend contract, shipped backends, dispatch policy | Accepted | | [004](decisions/004-two-backends-no-third.md) | Exactly two backends; manifest layers stay above | Accepted | | [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Pooled CAS, liveness seams, mark-and-sweep, delete windows | Accepted | | [006](decisions/006-verification-posture-and-transfer-encoding.md) | Whole-blob verification; transfer encoding excluded by scoping | Accepted | | [007](decisions/007-two-kv-engines-sqlite-and-postgres.md) | Two kv engines (sqlite + postgres) behind one trait; constructor-selected | Accepted | | [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | Trait `size` probe; fleet GC (DB-backed pins, advisory-locked sweeper); large-tier engines | Accepted | | [009](decisions/009-pg-lo-large-tier-engine.md) | pg-lo — postgres Large Objects as the large tier's second engine (admitted on POC #7) | Accepted | | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | Backend I/O seams (read cursor, staged put); GC state store-core-owned via the engine-state seam; kv fleet-validity; pin-token renewal | Accepted | | [012](decisions/012-pre-decomposition-consistency-rulings.md) | Pre-decomposition rulings: sweep aborts are report data (`GcAborted` retired); pin-token wire shape; fleet-mode constructor declaration; GC state hosts on the kv engine; sqlite seam reduction | Accepted | | [011](decisions/011-vocabulary-tiers-engines-and-fleet-state.md) | Vocabulary: tier/engine/instance/node/fleet; mem joins kv; large tier renamed `large`; constructor modes | Accepted | | [013](decisions/013-family-composition-reactive-boundary-and-feature-graph.md) | Family composition: no sibling dependency in-core; sweep lease; co-tenancy & no-cross-store-atomicity; OQ-11 resolved (engine features pull own drivers) | Accepted | ## Open Questions Tracked in [open-questions.md](open-questions.md). The Phase 0 register (OQ-BL-01..06) is promoted there with its resolutions; two questions remain externally-owned — OQ-07/OQ-08 (alkgit seam mapping, alkfs intake; carried for visibility, gating nothing here) — and the parked index is **empty**: OQ-09 (namespace-visibility default) resolved closed-by-default; OQ-10 (second kv engine) resolved by ADR-007; OQ-11 (the pg-only-kv feature graph, once `deferred(scope)` on a "named deployment requirement from outside") resolved by ADR-013 §4 — its deciding fact already existed in-tree (REQ-2's consolidation posture; ADR-009), the caught example of the circular form. The pg-lo large-engine question resolved to *sequenced work*, not a parked question: REQ-2/ADR-008 named it the candidate large-tier engine and POC #7 (`docs/research/poc-pglo-spec.md`) was its admission evidence — a task, not a wait; that task completed 2026-10-03 (`docs/research/poc-pglo-findings.md`, passed) and ADR-009 shipped the engine. **Deferral policy (the "Schrödinger's code" rule):** a *decision this crate needs before shipping* may not be deferred on a dependency that is itself waiting for this crate to exist. Legitimate parked kinds: `deferred(scope)` on a deciding fact that exists independently of this crate, and externally-owned questions (how a consumer maps onto this crate) that gate no decision here — full definitions in the header of [open-questions.md](open-questions.md). A third category is *not* a parked kind: **decided-but-sequenced work** (a resolved decision whose admission gate requires evidence — the pg-lo round's POC #7 is the worked example; a task with an owner, not a parked question). A corollary worth restating, because it recurs: a fact this crate must create (its own first deployment, its own first consumer) is never a deciding input — decisions stand on evidence in hand, and future needs reopen via named requirements, not via waiting. (Corollary applications so far: OQ-09, OQ-10, and OQ-11 — ADR-013's audit round — all resolved on it.) ## Lifecycle Spec docs: `draft` → `reviewed` → `stable` → `deprecated`. A doc moves `reviewed` when every *crate-owned* open question it references resolves and an architecture review pass clears it — externally-owned OQs (OQ-07/OQ-08) gate nothing here by definition and do not block the lifecycle; their outcomes arrive through their owners' processes. `stable` when implementation verifies against it; `deprecated` when superseded (kept for reference). ADRs use a separate status set (Accepted | Proposed | Deprecated | Superseded), defined per ADR file. This tree is in `draft` pending the first architecture review cycle.