From bee86b0dbd45b2b8a1f5312220707e738c12891d Mon Sep 17 00:00:00 2001 From: "glm-5.3-flash" Date: Sun, 11 Oct 2026 16:47:22 +0000 Subject: [PATCH] =?UTF-8?q?docs(architecture):=20ADR-013=20=E2=80=94=20fam?= =?UTF-8?q?ily=20composition,=20reactive=20boundary,=20sweep=20lease;=20OQ?= =?UTF-8?q?-11=20resolved?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - No sibling (alkstore) dependency in-core; reactivity is deployment-layer composition above the facade — cadence stays embedder-owned (ADR-005), with alkstore the named substrate for it; in-crate notify/scheduler/queues/invalidation hooks are non-goals. - Fleet sweeper lock pinned as 'named exclusive sweep lease' semantics; shipped provider unchanged (pg advisory clause); provider swaps are ADR-007-shaped substitution doors. - Composition contract for downstream ADRs: co-tenancy of CAS/GC state + reactive machinery + consumer/liveness tables on one fleet host; no-cross-store-atomicity ruled; compensation rests on idempotent re-puts, windowed deletes, sweeps-as-report-data. - OQ-11 resolved (the register's one live circular hedge): engine features pull their own driver clusters — sqlite becomes feature 'sqlite' (default-on, carries rusqlite); 'kv'/'large' tier features retired; 'local' ungated; pg-pure builds are supported, non-default. Deciding fact already in-tree (REQ-2/ADR-009). - requirements.md: UC-4/REQ-5 records the composition facts (operator record 2026-10-11, corroborated by alkstore's consumer-inventory). - Publish guard stated pre-code: facade ships before publish; semver-is-contract from first publish (alkstore ADR-017 discipline). - Amended accepted ADRs' feature wording (ADR-010 §1, ADR-011 §3/§4, ADR-003) with supersession notes; README/overview/open-questions/ backends/store-api/gc aligned; parked index now empty. Verification: docs-only round (no Rust surface touched); full-tree single-value check on the feature graph and sweep-lease wording. --- docs/architecture/README.md | 40 ++- docs/architecture/backends-and-dispatch.md | 22 +- .../003-backend-contract-and-dispatch.md | 5 +- ...0-backend-io-seams-and-fleet-state-home.md | 11 +- ...ocabulary-tiers-engines-and-fleet-state.md | 9 + ...ion-reactive-boundary-and-feature-graph.md | 242 ++++++++++++++++++ docs/architecture/gc-and-namespaces.md | 16 +- docs/architecture/open-questions.md | 40 ++- docs/architecture/overview.md | 56 ++-- docs/architecture/requirements.md | 42 ++- docs/architecture/store-api.md | 14 +- 11 files changed, 433 insertions(+), 64 deletions(-) create mode 100644 docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 0a8524b..4eb009e 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-012 — pre-decomposition consistency rulings: sweep-abort shape, token wire shape, fleet activation, GC-state host) +last_updated: 2026-10-11 (ADR-013 — family composition: no sibling dependency in-core, sweep-lease semantics, co-tenancy/no-cross-store-atomicity rulings, OQ-11 resolved, feature graph re-pinned to engine features) --- # alkblobs — Architecture @@ -106,6 +106,25 @@ there — not inherited as hedges: repaired (pin/sweep-lock bodies dropped; window staging for non-SQL engines is in-process). The tree is now single-valued on every implementation-facing signature. +- **ADR-013** (2026-10-11, the composition round): with the sibling + substrate **alkstore** shipped (v0.1.0 — the reactive coordination + crate the first consumers co-deploy), this ADR rules the + composition so nothing waits on anything downstream or upstream: + the core keeps **no alkstore dependency and no reactive machinery** + (notify/scheduler/queues/invalidation hooks are non-goals — cadence + stays embedder-owned per ADR-005, with alkstore the named + deployment-layer substrate for it); the fleet sweeper's + single-flighter is pinned as **named exclusive sweep lease** + semantics (shipped provider: the pg advisory-lock clause — provider + swaps are new-ADR substitution doors); **co-tenancy and + no-cross-store-atomicity** are ruled as the downstream-facing + composition contract (consumer liveness tables co-tenant the fleet + host; compensation rests on idempotent re-puts, windowed deletes, + sweeps-as-report-data); and **OQ-11 is resolved** — engine features + pull their own driver clusters (`sqlite` default-on carrying + rusqlite; `kv`/`large` tier features retired; `local` ungated), so + pg-pure builds are supported non-default configurations. REQ-5 + records the composition facts; the parked index is now empty. ## Architecture Documents @@ -136,17 +155,21 @@ there — not inherited as hedges: | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | Backend I/O seams (read cursor, staged put); GC state store-core-owned via the engine-state seam; kv fleet-validity; pin-token renewal | Accepted | | [012](decisions/012-pre-decomposition-consistency-rulings.md) | Pre-decomposition rulings: sweep aborts are report data (`GcAborted` retired); pin-token wire shape; fleet-mode constructor declaration; GC state hosts on the kv engine; sqlite seam reduction | Accepted | | [011](decisions/011-vocabulary-tiers-engines-and-fleet-state.md) | Vocabulary: tier/engine/instance/node/fleet; mem joins kv; large tier renamed `large`; constructor modes | Accepted | +| [013](decisions/013-family-composition-reactive-boundary-and-feature-graph.md) | Family composition: no sibling dependency in-core; sweep lease; co-tenancy & no-cross-store-atomicity; OQ-11 resolved (engine features pull own drivers) | Accepted | ## Open Questions Tracked in [open-questions.md](open-questions.md). The Phase 0 register -(OQ-BL-01..06) is promoted there with its resolutions; three questions -are parked — OQ-07/OQ-08 externally-owned (alkgit seam mapping, -alkfs intake; carried for visibility, gating nothing here) and -OQ-11 `deferred(scope)` (the pg-only-kv feature graph; a named -requirement from outside this crate decides it). OQ-09 +(OQ-BL-01..06) is promoted there with its resolutions; two questions +remain externally-owned — OQ-07/OQ-08 (alkgit seam mapping, +alkfs intake; carried for visibility, gating nothing here) — and the +parked index is **empty**: OQ-09 (namespace-visibility default) resolved closed-by-default; OQ-10 -(second kv engine) resolved by ADR-007. The pg-lo large-engine question +(second kv engine) resolved by ADR-007; OQ-11 (the pg-only-kv feature +graph, once `deferred(scope)` on a "named deployment requirement from +outside") resolved by ADR-013 §4 — its deciding fact already existed +in-tree (REQ-2's consolidation posture; ADR-009), the caught example +of the circular form. The pg-lo large-engine question resolved to *sequenced work*, not a parked question: REQ-2/ADR-008 named it the candidate large-tier engine and POC #7 (`docs/research/poc-pglo-spec.md`) was its admission evidence — a @@ -167,7 +190,8 @@ corollary worth restating, because it recurs: a fact this crate must create (its own first deployment, its own first consumer) is never a deciding input — decisions stand on evidence in hand, and future needs reopen via named requirements, not via waiting. (Corollary -applications so far: OQ-09 and OQ-10 both resolved on it.) +applications so far: OQ-09, OQ-10, and OQ-11 — ADR-013's audit round — +all resolved on it.) ## Lifecycle diff --git a/docs/architecture/backends-and-dispatch.md b/docs/architecture/backends-and-dispatch.md index a956ace..a6f83af 100644 --- a/docs/architecture/backends-and-dispatch.md +++ b/docs/architecture/backends-and-dispatch.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-012 — composite predicate gains the pg-lo same-instance clause; kv-only rejection is put-time; GC state hosts on the kv engine; window wording) +last_updated: 2026-10-11 (ADR-013 — feature graph re-pinned: engine features pull their own drivers; `kv`/`large` tier features retired, `sqlite` default-on, `local` ungated) --- # Backends, tiers, and dispatch @@ -111,13 +111,20 @@ the store's seams (see "What the Backend trait is"). | Tier | Engine | Feature | Default | Fleet-valid | |---|---|---|---|---| -| kv | sqlite | — (tier default) | yes (tier default-on) | no | +| kv | sqlite | `sqlite` | **yes (default-on)** | no | | kv | postgres | `postgres` | no | **yes** | | kv | mem | `mem` | no | no (test/reference engine) | -| large | local | — (tier default) | yes (tier default-on) | shared media only, explicit assertion | +| large | local | *(ungated — dependency-free)* | always present | shared media only, explicit assertion | | large | pg-lo | `pg-lo` | no | **yes** | -### kv tier (tier default-on, feature `kv`); engines sqlite (default) + postgres (feature `postgres`, default-off) + mem (feature `mem`, default-off) +Features gate engines, each pulling only its own driver cluster +(ADR-010 §1 extended to every engine — ADR-013 §4); tiers are +constructor composition, not features. The tier table's "Feature" +column is therefore the engine features: `sqlite` default-on carries +rusqlite (the batteries-included default); a pg-pure build (`sqlite` +off) is supported, non-default — REQ-2's consolidation posture. + +### kv tier (ADR-003/007/011); engines sqlite (feature `sqlite`, default-on) + postgres (feature `postgres`, default-off) + mem (feature `mem`, default-off) Small blobs — **`kv` is the tier name; the engine is a per-node constructor choice** between the shipped engines (ADR-007; ADR-003 @@ -177,7 +184,7 @@ fleet-valid, and the **contract-reference engine**: every other engine's conformance tests mirror its suite (the role it already played in POC #1's miniature). -### large tier (tier default-on, feature `large`; the tier formerly named `fs` — renamed in ADR-011); engines `local` (default) + `pg-lo` (feature `pg-lo`, default-off) +### large tier (ADR-003/008/011; the tier formerly named `fs` — renamed in ADR-011); engines `local` (ungated, always present) + `pg-lo` (feature `pg-lo`, default-off) Large blobs. Like the kv tier (ADR-007), the large tier is **one contract, multiple engines** — the medium is a per-node constructor @@ -301,6 +308,7 @@ alkfs intake may name needs externally; OQ-08). | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | I/O seams + GC-state home | read cursor / staged put; store-core-owned GC state via the engine-state seam; the kv fleet-validity rule | | [011](decisions/011-vocabulary-tiers-engines-and-fleet-state.md) | Vocabulary | tier/engine/instance/node/fleet; mem is a kv engine; fs → large; constructor modes | | [012](decisions/012-pre-decomposition-consistency-rulings.md) | Fleet activation & GC-state host | `fleet:` constructor declaration; all fleet GC state on the kv engine; pg-lo same-instance clause; kv-only put-time rejection | +| [013](decisions/013-family-composition-reactive-boundary-and-feature-graph.md) | Feature graph + fleet lease | engine features pull their own drivers (`kv`/`large` tier features retired); sweep lease semantics; no sibling dependency in-core | ## Open Questions @@ -325,6 +333,8 @@ alkfs intake may name needs externally; OQ-08). not adopted (threshold dispatch chose the simpler policy; ADR-003 §Context) - ADR-003, ADR-004, ADR-005; ADR-012 (fleet activation, GC-state - host, the composite predicate's same-instance clause); [store-api.md](store-api.md) (the invariants + host, the composite predicate's same-instance clause); ADR-013 (the + feature graph's engine-feature ruling, the sweep lease, the family + composition contract); [store-api.md](store-api.md) (the invariants engines must satisfy); [requirements.md](requirements.md) (the pinned vocabulary) \ No newline at end of file diff --git a/docs/architecture/decisions/003-backend-contract-and-dispatch.md b/docs/architecture/decisions/003-backend-contract-and-dispatch.md index fd2b1d0..05a8835 100644 --- a/docs/architecture/decisions/003-backend-contract-and-dispatch.md +++ b/docs/architecture/decisions/003-backend-contract-and-dispatch.md @@ -80,7 +80,10 @@ Evidence in hand (no further POC or consumer-waiting needed): engine. Both production tiers are default-on (batteries-included, the -alkgit feature-model precedent); `mem` is not. +alkgit feature-model precedent); `mem` is not. *(Feature shape +superseded by ADR-013 §4: "default-on" now reads as constructor-presence +default — the sqlite and `local` engines — not tier features; engine +features pull their own drivers.)* ### Dispatch policy diff --git a/docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md b/docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md index e502eeb..56e3af0 100644 --- a/docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md +++ b/docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md @@ -128,7 +128,10 @@ store core programs the abstraction, never the engine shape. feature pulling the shared driver cluster (`tokio-postgres` + `deadpool-postgres`) independently — a `large`=pg-lo + kv=sqlite node is coherent and supported. overview.md's dependency table pins -the edge. +the edge. *(The same independent-driver discipline is extended to +every engine by ADR-013 §4 — `sqlite` becomes its own default-on +feature carrying rusqlite, the `kv`/`large` tier features retire, and +pg-pure builds become supported, non-default.)* ### 2. GC state is store-core-owned; per-engine state seams are crate-internal and contract-tested @@ -369,4 +372,8 @@ token-mapping note; nothing else changes. - [ops-surface.md](../ops-surface.md) (the pin-token contract §4 extends); [store-api.md](../store-api.md) (stage hygiene, sweep semantics, cursor-adjacent facade shapes); [requirements.md](../requirements.md) - (REQ-2, the fleet requirement §2/§3 serve) \ No newline at end of file + (REQ-2, the fleet requirement §2/§3 serve) +- [ADR-013](013-family-composition-reactive-boundary-and-feature-graph.md) + (family composition — §4's feature-graph extension of this ADR's §1 + consequence; §2's sweep-lease semantics for this ADR's sweep-lock + clause) \ No newline at end of file diff --git a/docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md b/docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md index 011315a..070a725 100644 --- a/docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md +++ b/docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md @@ -138,6 +138,9 @@ kept (its meaning — this node's own filesystem — stays the constructor's meaning even under the shared-media fleet posture, where it is explicitly *not* only-node-local; the constructor's explicit shared-media assertion, ADR-008 §3, is what disambiguates). +*(The rename's feature clause is superseded by ADR-013 §4 — engine +features pull their own driver clusters, tier features retire — the +tier `large` and constructor wording stands.)* ### 4. The constructor table (the load-bearing copy of the fleet-validity summary; ADR-010 §3 points here) @@ -149,6 +152,12 @@ explicit shared-media assertion, ADR-008 §3, is what disambiguates). | large | local | — (tier default) | yes (tier default-on) | shared media only, explicit assertion | | large | pg-lo | `pg-lo` | no | **yes** | +*(The table's engine/constructor columns stand; the Feature column's +tier-default shape is superseded by ADR-013 §4 — engine features: +`sqlite` default-on, `postgres`, `mem`, `pg-lo` default-off, `local` +ungated. Fleet-validity, defaults as constructor presence, and modes +are unchanged.)* + Constructor modes (dispatch shapes) — pinned here (implied by ADR-008 §3's re-routing bullet and the mem tests, but never named): - **dual-tier** (the default): kv + large, size-threshold dispatch diff --git a/docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md b/docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md new file mode 100644 index 0000000..5f7b698 --- /dev/null +++ b/docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md @@ -0,0 +1,242 @@ +# ADR-013: Family composition — the reactive boundary, the sweep lease, and the feature-graph resolution + +## Status + +Accepted (2026-10-11) + +## Context + +While this crate paused mid-pre-decomposition (ADR-012 was its last +round), the sibling **alkstore** shipped v0.1.0: one reactive +coordination contract (`Store` — notify/listen, named TTL locks, +durable queues, streams with per-consumer offsets, outbox, +leader-elected scheduler) behind engine crates (`-mem`, `-sqlite`, +`-postgres`), contract-pinned by a factory-driven suite. Its +consumer-inventory — built from the paused consumers' written +documents under this crate's own deferral discipline — grades three +needs of *this crate's specced machinery*: + +- the fleet sweeper lock → named locks, **pinned**; +- maintenance cadence (sweep scheduling, pin-renewal workers) → + scheduler/queues, **documented** — against ADR-005's "embedder owns + cadence"; +- fleet invalidation ("sweep/manifest changed" among fleet instances) + → notify, **documented**. + +The first consumers (alkgit, alkfs) will deploy both crates together; +composition facts are therefore inter-crate decisions that must be +decided on evidence in hand, not parked on deployments that cannot +pre-exist this crate (the Schrödinger's-code rule; the OQ register's +OQ-09/OQ-10 precedents). The audit round that produced this ADR also +found one live circular hedge in the register: OQ-11 parked the +pg-only kv feature graph on "a named deployment requirement arriving +from outside this crate" — a deciding fact *this tree already +contains* (REQ-2's consolidation posture, which ADR-009 shipped as +"a fleet node can consolidate on postgres-only"), the same +shape-shifter OQ-09/OQ-10 were rejected for, and the last one in the +register. + +Additionally, the alkcall failure mode ("published doc surface +promising unimplemented behavior, surviving versions") motivates +stating this crate's publish discipline before anything ships. + +## Decision + +### 1. The reactive boundary: no sibling dependency in-core; reactivity is deployment-layer composition + +**The core crate gains no alkstore dependency and no reactive +machinery.** No in-crate notify, no in-crate scheduler, no queues, +no invalidation hooks on any facade method. ADR-001's substrate +posture extends by the same reasoning to the reactive substrate: +the store layer ships bytes-and-addresses, and everything that makes +bytes *reactive* is composition above the facade. + +- **ADR-005's "embedder owns cadence" stands, with the substrate + named.** An embedder wanting interval sweeps, pin-renewal workers + (the TTL/3 cadence, ADR-008), or maintenance dispatch composes them + above the facade — typically with the sibling substrate's + scheduler/queues. The punt was always a placement decision, not a + missing mechanism; the sibling's existence changes nothing here. +- **Sweep outcomes are data (ADR-012 §1) — with a pinned consumer.** + The report is what the embedder's reactive wiring consumes: + `SweepReport` → sibling-notify/stream events (downstream cache + invalidation), report cadence → the embedder's scheduler. No + in-crate hook exists for any of it; this crate never references + the sibling's types, channels, or names. +- The engine-driver dependency posture is likewise untouched: no + alkstore engine crate enters this crate's feature graph (see §4's + graph, which names only storage engines). + +### 2. The sweep lease: semantics pinned, provider shipped + +The fleet sweep single-flighter (ADR-008 §2's advisory lock, whose +state home is the engine-state seam's sweep-lock clause — ADR-010 §2, +ADR-012 §5) is hereby pinned as a **named exclusive sweep lease**: + +- at most one sweep in flight per pool; +- acquisition is an atomic test-and-set at the fleet host; +- holdership ends at explicit release, owner-process death, or the + underlying engine session's loss; +- contention is the `GcAbortCause::SweeperLock` report abort with + nothing deleted (ADR-012 §1 — report data, never an error). + +These are exactly pg advisory-lock semantics — the shipped provider +(ADR-008's mechanism, unchanged; a lease whose life is the sweep's +own) — and exactly what a named TTL lock provides when its lease +spans the sweep run. Pinning the *semantics* is what makes the +provider substitutable without a contract change: if the family ever +wants the in-crate provider swapped (e.g. onto the sibling's lock), +that is the ADR-007-shaped substitution door — a new ADR with its own +evidence, not an open question. + +A deployment may *additionally* serialize its maintenance above this +crate: an embedder holding a named lease on the sibling substrate +before invoking `sweep()` composes one discipline over its own +scheduled tasks (sweeps, expiry reaping, renewal workers). This is +composition above the facade — it changes no contract and is not +required for fleet correctness, because the in-crate advisory clause +is this crate's own guarantee regardless. + +### 3. Co-tenancy and the no-cross-store-atomicity ruling (the downstream-facing composition contract) + +A fleet pg instance hosts, as co-tenants: this crate's CAS rows and +GC state (its pool; `gc_pins` / `gc_window` — ADR-012 §4); the +sibling substrate's reactive machinery; consumer business tables +(refs, manifests, path trees — outside this crate per ADR-004); and +the consumer liveness tables this crate's fleet arbitration +consults as named table references (ADR-012 §6.6). **No crate owns +another's tables; co-tenancy is a named deployment statement** — the +posture decisions/007 and ADR-008 §2 already assumed, now citable so +downstream ADRs (alkgit's refs, alkfs's tree) reference it rather +than re-derive it. + +- **Cross-store atomicity is unachievable by construction** — each + crate owns its own transaction scope, and no seam bridges them. + The family posture downstream is the outbox/compensation pattern + (alkstore's `outbox`/`notify_tx` are its substrate), and this + crate's load-bearing properties for that pattern are already + guaranteed and stay so: **puts are idempotent under CAS** + (delete-then-recover is a byte-identical re-put, ADR-005), + **deletes are windowed and re-arbitrated before execution** + (ADR-005; ADR-012 §6.7), and **sweep outcomes are data** (ADR-012 + §1). Nothing downstream may rely on cross-store-atomic anything; + nothing here relies on it either. +- **Fleet liveness registration's two forms are the + downstream-facing seam:** the fleet form — a named liveness table + co-tenant on the fleet host (ADR-012 §6.6) — and the single-node + form — the in-process callback (ADR-005). Every downstream crate + plans against both. The requirement to keep a consumer's liveness + protection expressible on the fleet host is a *composition + constraint* on downstream schema placement: a consumer whose + liveness lives on a non-co-tenant engine participates in fleet GC + via pins (put-path) only, and its sweep-time liveness is expressed + by materializing rows per ADR-008 §2's materialization rule. + +### 4. OQ-11 resolved: engine features pull driver clusters independently — including sqlite + +**Features gate engines; tiers are constructor composition, not +features.** The ADR-010 §1 discipline ("engine features pull the +driver cluster independently" — pg-lo does not imply postgres) +extends to every engine: + +| Engine | Feature | Default | Pulls | +|---|---|---|---| +| kv `sqlite` | `sqlite` | **on** | `rusqlite` | +| kv `postgres` | `postgres` | off | `tokio-postgres` + `deadpool-postgres` | +| kv `mem` | `mem` | off | nothing | +| large `local` | *(ungated)* | always present | nothing | +| large `pg-lo` | `pg-lo` | off | the shared pg driver cluster | + +- The base `kv` feature — which existed because sqlite was the kv + tier's only shippable driver carrier and so pulled `rusqlite` under + the tier's name — is retired, as is the `large` tier feature + (nothing was ever behind it; `local` is ungated because it pulls + nothing). ADR-011 §3's `feature fs` → `feature large` rename line + is superseded by this table; ADR-003's "tiers default-on" wording + now reads as constructor-presence, not feature claims. +- **A pg-pure build (`sqlite` off — kv=postgres, and `large`=pg-lo if + wanted) is a supported configuration this crate does not default + to.** This resolves OQ-11 on evidence in hand: REQ-2 (operator + authority, 2026-10-03) is the deployment requirement the parked row + awaited — its consolidation posture (ADR-009's postgres-only fleet + node) is the pg-pure node, and parking the feature graph on it was + circular (a deployment requirement whose name the tree already + carries is not an absent fact). Mechanism unchanged from the + precedent: engine features pull their own drivers, so sqlite's + absence is one feature toggle, never a graph surgery. +- **Default builds are unchanged**: `sqlite` default-on keeps the + batteries-included default (the lean-base principle constrains + default builds, which still pull only tokio/thiserror/parking_lot + plus the default engine's driver). + +### 5. Publish guard (stated pre-code, on the alkcall lesson) + +The crate publishes only after its facade exists and passes the +invariants' test gate. From first publish, its semver is the contract +version — every published surface is implemented, every contract +change cites its ADR — the discipline the sibling substrate pinned in +its contract-versioning ADR. No published doc surface ever carries a +"planned/not-implemented" promise. + +## Consequences + +**Positive** + +- The composition contract exists as a named ADR: alkgit/alkfs cite + §3 for schema co-tenancy, the two-form liveness seam, and the + no-cross-store-atomicity posture, instead of re-deriving or + guessing. +- The register's one live circular hedge (OQ-11) is resolved on + in-tree evidence, pre-consumer, where the fix is cheapest — the + alkcall-shaped outcome this round exists to avoid. +- A new fleet node's dependency tree is honest: pg-pure nodes do not + compile rusqlite; nothing reactive is dead code in the core. +- The sweep lease's semantics and provider are single-valued; a + provider swap later is a mechanism ADR, not a contract break. + +**Negative** + +- One more ADR's worth of surface for every reader of the fleet GC + story (§2 changes no signatures; it pins names for semantics the + mechanisms already had). +- The feature graph must be implemented as specced in §4's table — an + implementation-facing edit arriving with the engine tasks rather + than inside one of them. + +**Neutral** + +- No tier, engine, constructor mode, facade signature, error variant, + or ops payload changes. `fleet: true` validation (ADR-012 §3) and + the composite fleet-validity predicate are untouched. +- The sibling substrate relationship is documentary here — + requirements-level facts and boundary rules — with the eventual + dependency edge (if any, e.g. an ops-layer convenience) requiring + its own ADR through the substitution door. + +## References + +- ADR-001 (the substrate posture §1 extends to the reactive + substrate), ADR-003/007 (the substitution door §2 applies), ADR-005 + (cadence ownership; delete-then-recover as the idempotency + backstop), ADR-006 (the deferred-cost pattern this ADR's OQ-11 + resolution follows), ADR-008 (the fleet sweep lock §2 pins and the + liveness materialization rule §3 reuses), ADR-010/012 (the + engine-state seam, GC-state host, outcomes-as-data, the composite + predicate), ADR-011 (vocabulary; the constructor tables §4 + supersedes in feature shape) +- [requirements.md](../requirements.md) — REQ-2 (the fact OQ-11 parked + on), REQ-5 (the composition requirement §3 serves) +- [overview.md](../overview.md) (dependency posture, amended §4; + non-goals, extended §1); [backends-and-dispatch.md](../backends-and-dispatch.md) + (engine/features table, amended §4); [store-api.md](../store-api.md) + (sweep semantics, §2's lease wording); [gc-and-namespaces.md](../gc-and-namespaces.md) + (sweeper locking); [open-questions.md](../open-questions.md) (OQ-11 + resolved) +- alkstore — `/workspace/@alkdev/alkstore`: its consumer-inventory + (`docs/research/consumer-inventory.md`, the graded fleet rows), + ADR-016 (deployment honesty), ADR-017 (contract versioning — §5's + cited discipline); the sibling's payload model (JSON-only) is its + own non-goal boundary, matching this crate's object-storage + boundary here +- The alkcall hedge precedent (published-doc-surface drift) — the + failure mode §5 guards, on the operator's record \ No newline at end of file diff --git a/docs/architecture/gc-and-namespaces.md b/docs/architecture/gc-and-namespaces.md index fc5f855..b9671f1 100644 --- a/docs/architecture/gc-and-namespaces.md +++ b/docs/architecture/gc-and-namespaces.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-012 — aborts are report data (GcAbortCause), never sweep errors; window executor = the next sweep; non-SQL engines stage in-process) +last_updated: 2026-10-11 (ADR-013 — sweeper-lock semantics pinned as the named exclusive sweep lease; provider unchanged; composition boundary referenced) --- # Pooling, namespaces, and GC @@ -25,11 +25,15 @@ price, and this document specifies the mechanism the POCs validated per-repo stores. Git's `alternates`/object-pool mechanism (GitLab object pools) is the same idea done awkwardly; the pool generalizes it to unrelated repos and to p2p replication. **A fleet (REQ-2) — two - or more store instances sharing one pool — is the one topology where + or more store instances sharing one pool — is the one topology where the GC mechanism below extends beyond one instance: ADR-008 pins, sweeper locking, and delete arbitration move into the shared engine; - the invariants are unchanged. Where this crate's GC state (fleet - rows and locks) physically lives, and who holds the joint + the invariants are unchanged — and the sweeper lock's semantics are + the **named exclusive sweep lease** (one sweeper per pool; atomic + test-and-set; holdership ends at release/owner death/engine-session + loss; contention is the `SweeperLock` report abort — ADR-013 §2, + shipped provider: the pg advisory lock). Where this crate's GC state + (fleet rows and locks) physically lives, and who holds the joint entry+pin transaction, is ruled by ADR-010 §2: the store core owns all GC state; SQL-backed engines host it through a crate-internal, contract-tested seam; backends never learn liveness.** @@ -140,6 +144,7 @@ existence flips atomically per key). | [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | Fleet GC extension | DB-backed pins, advisory-locked single sweeper, SQL delete arbitration — for shared-pool (fleet) topologies only; the in-process protocol is unchanged elsewhere | | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | GC-state home | the store core owns all GC state; SQL engines host it via the contract-tested engine-state seam; pin-tx ownership ruled | | [012](decisions/012-pre-decomposition-consistency-rulings.md) | Abort-as-data + window executor | sweep aborts are `GcAbortCause` report data, never errors; the next sweep executes overdue window rows; non-SQL engines stage in-process; GC state hosts on the fleet's kv engine | +| [013](decisions/013-family-composition-reactive-boundary-and-feature-graph.md) | Sweep lease + composition boundary | the fleet sweeper lock's semantics = named exclusive sweep lease (provider unchanged: pg advisory lock); reactivity/cadence composition lives above the facade (alkstore the sibling substrate), never in-core | ## Open Questions @@ -162,5 +167,6 @@ a future requirement does not create one retroactively. - rudolfs — the physical-namespace anti-pattern (inverted) - gix-odb — alternates/pool prior art for the p2p case - ADR-005; ADR-012 (the abort-as-data ruling, the window executor, - the GC-state host); [store-api.md](store-api.md) (Pin semantics, sweep API); + the GC-state host); ADR-013 (§2 the sweep lease; §1 the composition + boundary); [store-api.md](store-api.md) (Pin semantics, sweep API); [ops-surface.md](ops-surface.md) (namespace-as-resource ACL) \ No newline at end of file diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md index 714d319..6837f5d 100644 --- a/docs/architecture/open-questions.md +++ b/docs/architecture/open-questions.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-012 round — no new OQs; overview's pg-only-kv feature-graph deferral recorded in the parked index) +last_updated: 2026-10-11 (ADR-013 round — OQ-11 resolved on in-tree evidence; the parked index is now empty) --- # Open Questions @@ -46,9 +46,9 @@ and not a hedge. A decision is not "unmade" because its verification is pending; deferral policy applies to decisions, not to work. **Index of active OQs:** OQ-07, OQ-08 (externally-owned, carried -for visibility). OQ-09 and OQ-10 are resolved (recorded below). -OQ-11 (deferred(scope), parked index) records a feature-graph -deferral. Promoted Phase 0 questions OQ-BL-01..06 are recorded here +for visibility). OQ-09, OQ-10, and OQ-11 are resolved (recorded +below). The parked index has no active rows — +OQ-07/OQ-08 are externally-owned, not parked rows. Promoted Phase 0 questions OQ-BL-01..06 are recorded here with their resolutions for traceability. --- @@ -233,21 +233,33 @@ with their resolutions for traceability. ## Parked index (deciding fact / owner — externally-owned rows are not blockers) +No active parked rows remain. OQ-07/OQ-08 are externally-owned +(above), not parked; OQ-09/OQ-10/OQ-11 are resolved with their +records kept here for auditability. + | OQ | Status | Deciding fact / owner | |---|---|---| | OQ-07 | externally-owned | alkgit's architecture process (answerable on paper anytime; gates nothing here) | | OQ-08 | externally-owned | alkfs Phase 0 intake | -| OQ-11 | deferred(scope) | a named deployment requirement for a pg-only kv node (postgres kv engine without rusqlite in the build) — overview.md's feature-graph note | -OQ-11 records [overview.md](overview.md)'s dependency-posture deferral -(a pg-only kv node — the `kv` feature's rusqlite pull decoupled from -default-on feature graphing) for auditability: the deciding fact is a -*deployment requirement that arrives from outside this crate* -(Schrödinger's-code rule compliant — it is not a fact this crate -creates), the accepted v1 shape (base `kv` feature pulls rusqlite, -default-on) stands until then, and the deferral is a feature-graph -work item, not an open architecture decision. No spec or ADR gates on -it. +OQ-11 was `deferred(scope)` on "a named deployment requirement for a +pg-only kv node arriving from outside this crate" (overview.md's +feature-graph note; the audit trail of the parked wording is kept: +the `kv` feature's rusqlite pull decoupled from default-on feature +graphing). The 2026-10-11 audit round resolved it as a live circular +hedge: the deciding fact already existed in this tree — REQ-2 +(operator authority, 2026-10-03, pre-POC) names the deployment +requirement, and ADR-009 shipped its consolidation posture ("a fleet +node can consolidate on postgres-only"), which is the pg-pure node +whose feature graph was parked — the same invalid shape OQ-09/OQ-10 +were rejected for, with the deciding fact created by the tree's own +progress and nobody re-auditing the row. Resolved by +[ADR-013](decisions/013-family-composition-reactive-boundary-and-feature-graph.md) +§4: engine features pull driver clusters independently (the ADR-010 +§1 discipline extended to sqlite — feature `sqlite`, default-on, +carrying rusqlite; `kv`/`large` tier features retired; `local` +ungated), and a pg-pure build is a supported configuration this +crate does not default to. No deferral remains. OQ-09 was `deferred(scope)` on the first embedded ops deployment — a deciding fact that could only exist once the ops module ships, i.e. a diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index d0190e5..a1bf6f6 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-012 — OQ-11 parking reference; ADR-012 row added) +last_updated: 2026-10-11 (ADR-013 — family composition & reactive boundary; OQ-11 resolved, feature graph re-pinned to engine features) --- # Overview @@ -72,8 +72,9 @@ or three times. ┌──────────────────────┐ ┌─────────────────────────────────┐ │ kv tier (ADR-003/ │ │ large tier (ADR-003/008/011; │ │ 007/011) │ │ formerly "fs") │ -│ sqlite (default) │ │ local (default) │ -│ postgres ("postgres") │ pg-lo ("pg-lo"; ADR-009) │ +│ sqlite ("sqlite"; │ │ local (ungated; ADR-013 §4) │ +│ default engine) │ │ pg-lo ("pg-lo"; ADR-009) │ +│ postgres ("postgres") │ │ │ mem ("mem"; tests + │ │ │ contract reference) │ │ └──────────────────────┘ └─────────────────────────────────┘ @@ -90,24 +91,31 @@ or three times. | Layer | Dependencies | Notes | |---|---|---| -| base crate (store core + tier engines) | `tokio`, `thiserror`, `parking_lot`, plus `rusqlite` behind the `kv` feature and nothing behind `large`/`mem` | no alkcall, no serialization frameworks | -| `postgres` engine feature (ADR-007) | adds `tokio-postgres` + `deadpool-postgres` behind the kv tier's second engine, default-off | depends on `kv`'s contract surface (the `Backend` trait definition is base-crate code); the engine's *impl* is additive. Same `Backend` contract, own sweep-safety CI gate under `--all-features` | -| `pg-lo` engine feature (ADR-009) | Large Objects as the large tier's second engine, default-off | pulls the same `tokio-postgres`/`deadpool-postgres` cluster as the `postgres` kv engine (SQL `lo_*` functions, no additional dependency); **does not imply the `postgres` feature** — engine features pull the driver cluster independently (ADR-010 §1), so a `large`=pg-lo + kv=sqlite node is coherent and supported; same admission/test-gate shape | -| `mem` engine feature (ADR-011) | the kv tier's ephemeral engine, default-off | `BTreeMap`-shaped; the contract-reference engine; no dependencies | +| base crate (store core + tier dispatch + GC) | `tokio`, `thiserror`, `parking_lot` | no alkcall, no reactive substrate, no serialization frameworks | +| engine features (ADR-010 §1, extended to every engine by ADR-013 §4) | each engine pulls only its own driver cluster | `sqlite` (default-on, `rusqlite`), `postgres` (default-off, `tokio-postgres`+`deadpool-postgres`), `mem` (default-off, nothing), `pg-lo` (default-off, the shared pg cluster); `local` is ungated (dependency-free). Every engine-feature combination is a coherent node shape (e.g. `large`=pg-lo + kv=sqlite, ADR-010 §1); the feature-gated engines' conformance/sweep-safety gates run under `--all-features` (ADR-007/009) | | `ops` feature | adds `alkcall` (feature-gated, default-off) | JSON op specs + binary channels + `AccessControl`; ADR-001 | -| consumer crates | depend on alkblobs (and on alkcall directly when they speak ops) | alkgit, alkfs | +| consumer crates | depend on alkblobs (and on alkcall directly when they speak ops) | alkgit, alkfs — which also deploy the sibling reactive substrate beside it (below) | -Tiers are feature-gated; both default-on (batteries-included, the -alkgit feature-model pattern — default-on features, opt-out rather -than opt-in); the engine features `postgres` and `pg-lo`, and `mem`, -are default-off, as are `ops`. A pg-only kv node (postgres engine -without rusqlite) is **not** a supported configuration this crate -optimizes for — feature-graph work for it is deferred until a named -requirement exists (the base `kv` feature pulling rusqlite is the -accepted v1 shape; convention 8's lean-base principle applies to -*default* builds, which stay lean since `kv` is what the feature model -controls, not what default-on means downstream; tracked as OQ-11 in -open-questions.md's parked index). Wasm: the tier +**The sibling reactive substrate (ADR-013 §1/§3):** **alkstore** +(v0.1.0 — notify/locks/queues/streams/outbox/scheduler behind its own +engine crates) ships beside this crate, not under it. The core keeps +no alkstore dependency and no reactive machinery: cadence and +invalidation wiring are deployment-layer composition above the facade +(ADR-005's "embedder owns cadence" stands, with the sibling the +named substrate; sweep outcomes are report data — ADR-012 §1 — whose +pinned consumer is that wiring). Cross-store atomicity is unachievable +by construction; the composition properties are idempotent re-puts, +windowed deletes, and reports-as-data (ADR-013 §3, the citable +downstream contract). + +Tiers are **constructor composition**, not features (ADR-013 §4); +engine features as in the table above. The default build is +batteries-included (sqlite default-on — opt-out rather than opt-in, +the alkgit feature-model pattern), and a pg-pure build (sqlite off; +kv=postgres, `large`=pg-lo if wanted) is a supported configuration +this crate does not default to — REQ-2's consolidation posture +(ADR-009), now carried by the feature graph instead of parked on it. +Wasm: the tier engines are not a wasm story; a wasm client is a consumer of the `ops` surface (which rides alkcall, itself wasm-clean), never an in-process embedder (ADR-001 §Consequences). @@ -124,6 +132,10 @@ embedder (ADR-001 §Consequences). Backend to model paths (ADR-004). - No chunk-tree/CDC storage encoding; whole-file CAS is the store's only granularity (ADR-006). +- No reactive coordination machinery in-core — no notify, no scheduler, + no queues, no invalidation hooks: reactivity is the sibling + substrate's (alkstore), composed at the deployment layer above the + facade (ADR-013 §1). - No per-repo or per-namespace walled stores — one pooled CAS (ADR-005). @@ -143,6 +155,7 @@ embedder (ADR-001 §Consequences). | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | I/O seams + GC-state home | read cursor / staged put; store-core-owned GC state via the engine-state seam; kv fleet-validity rule; pin-token renewal | | [012](decisions/012-pre-decomposition-consistency-rulings.md) | Sweep-abort shape + composition rulings | sweeps never error (`GcAborted` retired, `GcRefuse` for direct-delete); token wire shape; fleet mode = constructor declaration; GC state hosts on the kv engine; sqlite seam reduction | | [011](decisions/011-vocabulary-tiers-engines-and-fleet-state.md) | Vocabulary + rename | tier/engine/instance/node/fleet pinned; mem joins the kv tier; large tier renamed `large`; constructor modes | +| [013](decisions/013-family-composition-reactive-boundary-and-feature-graph.md) | Family composition & reactive boundary | no sibling dependency in-core; sweep lease semantics (provider: raw advisory clause); co-tenancy + no-cross-store-atomicity; OQ-11 resolved (engine features pull own drivers) | ## Open Questions @@ -157,8 +170,9 @@ crate level: open-read requirement) - **OQ-10**: second kv engine — resolved by ADR-007 (sqlite + postgres shipped behind one trait; per-node constructor choice) -- **OQ-11**: pg-only kv node feature graph — deferred(scope), parked - (see [open-questions.md](open-questions.md)) +- **OQ-11**: pg-only kv node feature graph — resolved by ADR-013 §4 + (engine features pull their own driver clusters; `sqlite` default-on; + pg-pure builds supported, non-default) - **POC #7** (`docs/research/poc-pglo-findings.md`): pg Large Objects as the large tier's `pg-lo` engine — **passed** 2026-10-03; admitted as a shipped engine by ADR-009 (REQ-2's consolidation option) diff --git a/docs/architecture/requirements.md b/docs/architecture/requirements.md index 3a0fc05..1d0bf0c 100644 --- a/docs/architecture/requirements.md +++ b/docs/architecture/requirements.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (vocabulary pinned by ADR-011 — tiers/engines/instances/fleet; large tier renamed `large`) +last_updated: 2026-10-11 (ADR-013 composition round — REQ-5 added: the family's first consumers deploy this crate with the sibling reactive substrate; co-tenancy + liveness-form facts pinned) --- # Requirements @@ -102,12 +102,47 @@ path→hash manifests above. performance is what matters; the large tier is hit rarely (large workspace artifacts). +### UC-4: The family's first consumers run composition (alkgit + alkfs + the reactive substrate) + +The first consumers of this crate — alkgit (its object backend, +OQ-07) and alkfs (path trees/workspaces, OQ-08) — deploy +**alkstore**, the sibling reactive coordination substrate (v0.1.0: +notify/locks/queues/streams/outbox/scheduler behind its own engine +crates) *together with* this crate, per an operator-authority +planning fact (2026-10-11, recorded in alkstore's +`docs/research/consumer-inventory.md` — which grades this crate's +fleet needs against alkstore's surface). This forces composition +facts on this crate regardless of whether its core takes a +dependency: + +- **REQ-5 (traces: ADR-013)** — composition facts (decided, not + deferred; ADR-013 is the ruling): + - The first consumers' nodes co-deploy both crates; this crate's + core keeps **no alkstore dependency** (ADR-013 §1) — the reactive + substrate is deployment-layer composition above the facade, the + named substrate for the cadence ADR-005 assigns to embedders. + - **No cross-store atomicity**: each crate owns its transaction + scope; the compensation properties downstreams rely on are this + crate's idempotent re-puts (CAS; delete-then-recover backstop), + windowed deletes, and reports-as-data sweep outcomes (ADR-013 + §3). + - Consumer **liveness tables co-tenant the fleet host** (the + named-table-reference form, ADR-012 §6.6) or ride the in-process + callback form single-node — the two-form liveness seam every + downstream plans against (ADR-013 §3). + - The fleet sweeper's single-flighter is a named exclusive sweep + lease whose shipped provider is the pg advisory lock; other + providers (the substrate's lock) are composition above the + facade or a new-ADR substitution door (ADR-013 §2). + ## What this document is not - **Not consumer architecture**: how alkgit maps refs/manifests onto the store remains OQ-07 (externally-owned); how alkfs shapes workspaces remains OQ-08. This document records what each use case - *forces on this crate*, not what each consumer *is*. + *forces on this crate*, not what each consumer *is* — UC-4's rows + are the one composition shape the use cases themselves force + (REQ-5/ADR-013), stated as constraints, not as a consumer design. - **Not a requirements-change process substitute**: a new or changed REQ arrives here with a dated note and owner; ADRs that depended on the old fact state their reopen conditions and reopen when this @@ -120,4 +155,5 @@ path→hash manifests above. | REQ-1 | 2026-10-03 (fact: phase-0 §deployment posture) | POC #3 A4 | ADR-007 | | REQ-2 | 2026-10-03 (fact: use case origin discussion, pre-POC) | user/planning record | ADR-007, ADR-008 | | REQ-3 | 2026-10-03 (fact: phase-0 §deployment posture) | POC #5 B3/B4 | ADR-007 | -| REQ-4 | 2026-10-03 (fact: phase-0 §vision) | POC #1, ADR-002 | ADR-002, ADR-005 | \ No newline at end of file +| REQ-4 | 2026-10-03 (fact: phase-0 §vision) | POC #1, ADR-002 | ADR-002, ADR-005 | +| REQ-5 | 2026-10-11 (fact: composition planning; corroborated by alkstore's consumer-inventory) | operator record | ADR-013 | \ No newline at end of file diff --git a/docs/architecture/store-api.md b/docs/architecture/store-api.md index 19db30c..fddffe4 100644 --- a/docs/architecture/store-api.md +++ b/docs/architecture/store-api.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-012 — sweep-abort = Ok(report) with GcAbortCause; GcAborted retired; fall-through for all key-addressed ops; put-time kv-only rejection; delete/missing/error-model ruling) +last_updated: 2026-10-11 (ADR-013 — the fleet sweeper lock's semantics pinned as the named exclusive sweep lease; provider unchanged; composition boundary referenced) --- # Store API @@ -138,10 +138,15 @@ mechanism lives in the engine-state seam, ADR-010 §2). mechanism-extended by ADR-008: pins are DB-backed rows committed atomically with entries (the store core holds the joint tx, ADR-010 §2; all GC state hosts on the fleet's kv engine — - ADR-012 §4), one sweeper holds the advisory lock (contention is a - report abort, not an error — ADR-012 §1), and delete + ADR-012 §4), one sweeper holds the advisory lock (named exclusive + sweep lease semantics: one sweeper per pool, atomic test-and-set, + holdership ends at release/owner death/engine-session loss; + contention is a report abort, not an error — ADR-012 §1, semantics + pinned ADR-013 §2), and delete arbitration is SQL-level — the facade's invariant text is - unchanged. + unchanged. The facade owns no reactive machinery: cadence, + maintenance dispatch, and invalidation wiring are embedder + composition above it (ADR-005; ADR-013 §1). ## Error model @@ -231,6 +236,7 @@ lesson generalizes to any kv engine). | [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | Trait `size` + fleet GC | stat/accounting ride the trait's `size` probe; fleet pins/sweeps are engine-backed | | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | I/O seams + GC-state home | read cursor / staged put behind the facade; the store core holds the joint entry+pin tx | | [012](decisions/012-pre-decomposition-consistency-rulings.md) | Sweep-abort shape + wire/activation rulings | swept = Ok(report) always, `GcRefuse` for direct-delete refusal; fleet mode is a constructor declaration; GC state hosts on the kv engine | +| [013](decisions/013-family-composition-reactive-boundary-and-feature-graph.md) | Sweep lease + composition boundary | the advisory lock's semantics = named exclusive sweep lease (provider unchanged); no reactive machinery/facade hooks in-core | ## Open Questions