diff --git a/docs/architecture/README.md b/docs/architecture/README.md index dd75cdf..0a8524b 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-010/011 — I/O seams + GC-state home; vocabulary pinned, fs → large, mem joins kv) +last_updated: 2026-10-03 (ADR-012 — pre-decomposition consistency rulings: sweep-abort shape, token wire shape, fleet activation, GC-state host) --- # alkblobs — Architecture @@ -88,6 +88,24 @@ there — not inherited as hedges: `large`** (feature `fs` → `large`) — its routing criterion is length, not medium — with constructor modes (dual-tier default, kv-only, mem-only) pinned in the same ADR's table. +- **ADR-012** (2026-10-03) is the pre-decomposition consistency round: + the full-tree review found five composition defects and this ADR + rules each — sweeps are **never errors** (`Ok(SweepReport)` with + `aborted: Option`; `GcAborted` retired from the error + enum, direct-delete refusal renamed `GcRefuse`; + `NoLivenessSources` added as the third cause variant); the pin + token gains its **wire shape** (`blobs/put` response `{token, + digest}`; `blobs/have`'s token-renewal form; token validity domain + = the minting serving node); **fleet mode is an explicit + constructor declaration** (`fleet: true`, never inferred from + engine choice); **all fleet GC state hosts on the fleet's kv engine + (postgres)** — one arbitration domain — with `large=pg-lo` fleet + nodes required onto the same pg instance (the composite predicate's + new clause) and `large=local` fleet puts pinned pin-row-first, + publish-second; and the engine-state seam's sqlite rows are + repaired (pin/sweep-lock bodies dropped; window staging for + non-SQL engines is in-process). The tree is now single-valued on + every implementation-facing signature. ## Architecture Documents @@ -97,7 +115,7 @@ there — not inherited as hedges: | [requirements.md](requirements.md) | Consumer requirements REQ-1..4, pinned vocabulary (tier/engine/instance/node/fleet) | draft | | [hashing-and-keys.md](hashing-and-keys.md) | Canonical hash, key encoding (the one-way door) | draft | | [store-api.md](store-api.md) | Store facade: put/get/stat/range/pin/batch/sweep surface, errors, invariants | draft | -| [backends-and-dispatch.md](backends-and-dispatch.md) | Backend trait contract, tiers + engines table, size-threshold dispatch, constructor modes | draft | +| [backends-and-dispatch.md](backends-and-dispatch.md) | Backend trait contract, tiers + engines table, size-threshold dispatch, constructor modes, fleet-validity | draft | | [gc-and-namespaces.md](gc-and-namespaces.md) | Pooled CAS, liveness seams, mark-and-sweep, delete windows | draft | | [ops-surface.md](ops-surface.md) | alkcall-backed have/need + fetch/put ops, ACL mapping | draft | | [open-questions.md](open-questions.md) | Centralized OQ tracker (incl. promoted Phase 0 register) | draft | @@ -116,14 +134,17 @@ there — not inherited as hedges: | [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | Trait `size` probe; fleet GC (DB-backed pins, advisory-locked sweeper); large-tier engines | Accepted | | [009](decisions/009-pg-lo-large-tier-engine.md) | pg-lo — postgres Large Objects as the large tier's second engine (admitted on POC #7) | Accepted | | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | Backend I/O seams (read cursor, staged put); GC state store-core-owned via the engine-state seam; kv fleet-validity; pin-token renewal | Accepted | +| [012](decisions/012-pre-decomposition-consistency-rulings.md) | Pre-decomposition rulings: sweep aborts are report data (`GcAborted` retired); pin-token wire shape; fleet-mode constructor declaration; GC state hosts on the kv engine; sqlite seam reduction | Accepted | | [011](decisions/011-vocabulary-tiers-engines-and-fleet-state.md) | Vocabulary: tier/engine/instance/node/fleet; mem joins kv; large tier renamed `large`; constructor modes | Accepted | ## Open Questions Tracked in [open-questions.md](open-questions.md). The Phase 0 register -(OQ-BL-01..06) is promoted there with its resolutions; two questions -remain parked — OQ-07/OQ-08 externally-owned (alkgit seam mapping, -alkfs intake; carried for visibility, gating nothing here). OQ-09 +(OQ-BL-01..06) is promoted there with its resolutions; three questions +are parked — OQ-07/OQ-08 externally-owned (alkgit seam mapping, +alkfs intake; carried for visibility, gating nothing here) and +OQ-11 `deferred(scope)` (the pg-only-kv feature graph; a named +requirement from outside this crate decides it). OQ-09 (namespace-visibility default) resolved closed-by-default; OQ-10 (second kv engine) resolved by ADR-007. The pg-lo large-engine question resolved to *sequenced work*, not a parked question: REQ-2/ADR-008 diff --git a/docs/architecture/backends-and-dispatch.md b/docs/architecture/backends-and-dispatch.md index 03b9833..a956ace 100644 --- a/docs/architecture/backends-and-dispatch.md +++ b/docs/architecture/backends-and-dispatch.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-010 I/O seams + ADR-011 rename — fs tier → large; mem is a kv engine) +last_updated: 2026-10-03 (ADR-012 — composite predicate gains the pg-lo same-instance clause; kv-only rejection is put-time; GC state hosts on the kv engine; window wording) --- # Backends, tiers, and dispatch @@ -92,7 +92,12 @@ is ever reachable through that door. and sweep locks are store-core-owned via a crate-internal engine-state companion seam on the SQL-backed engines (sqlite, postgres, pg-lo) — invisible at the public trait; `local` and `mem` - cannot express it and are never fleet engines. + cannot express it and are never fleet engines. **All fleet GC state + (pin rows, window rows, the sweep lock) hosts on the fleet's kv-tier + engine — postgres — regardless of the large tier's engine; pg-lo + hosts no fleet state** (ADR-012 §4). Per-engine clause applicability: + ADR-012 §5's table (sqlite and pg-lo carry window-family clauses + only; pin/sweep-lock clauses are the postgres host's). ## Shipped tiers and engines @@ -216,12 +221,19 @@ load-bearing for fleets): two sqlite engines are two pools, never one. - **The composite fleet-validity rule (one sentence, both tiers):** a constructor configuration is fleet-valid iff **kv = postgres AND - (large = pg-lo OR large = `local`-on-declared-shared-media OR + (large = pg-lo *on the same pg instance as the kv engine* OR + large = `local`-on-declared-shared-media OR large = none with the re-routing posture carrying pool large - content)**. Any other combination over one shared pool is invalid — + content)** — plus an explicit `fleet: true` constructor declaration + (ADR-012 §3: fleet mode is never inferred). Any other combination over one shared pool is invalid — the constructor requires the declarations that make this predicate checkable per node; cross-node truth remains the deployment's - verified invariant (ADR-008's seam). This predicate is + verified invariant (ADR-008's seam). The pg-lo same-instance clause + formalizes what ADR-009's consolidation posture already assumed + (pool content lives in the pg instance the kv tier rides) — it is + what makes the joint entry+pin tx expressible; cross-instance + kv=postgres + large=pg-lo is valid only as a non-fleet + configuration. This predicate is backends-and-dispatch.md's and ADR-011 §4's table combined; it is stated here once so no reader composes it by inference. @@ -245,18 +257,24 @@ load-bearing for fleets): ### Constructor modes (ADR-011 §4) - **dual-tier** (the default): kv + large, size-threshold dispatch - (ADR-003). + (ADR-003). Any engine pair the tables allow — including + `kv = mem` + `large = local`, the dispatch-coverage test shape + (ADR-012 §6.3; both tiers are load-bearing there too). - **kv-only** (ADR-008's re-routing client posture, now named): no - large tier; over-threshold puts are a constructor-time error (the - re-routing posture handles them via the ops surface — the - consumer's composition, not a store mode). + large tier; over-threshold puts are rejected **at put time** — + known-length puts reject immediately, unknown-length puts reject at + mid-stream threshold overflow (ADR-012 §6.2: lengths are not known + at construction, so "constructor-time error" was strictly + impossible). The re-routing posture handles over-threshold content + via the ops surface — the consumer's composition, not a store mode. - **mem-only:** the mem engine alone, no dispatch threshold in effect; a testing/embedder-ephemeral posture, never production. - Single-tier SQL modes (postgres-only, pg-lo-only *tiers*) do not exist: both tiers are load-bearing (ADR-004). "Postgres-only" as in one SQL *instance* serving both tiers exists and is the ADR-009 consolidation: dual-tier mode with kv=postgres + large=pg-lo over - one pool. + one pool — under fleet mode, required to be literally the same + instance (ADR-012 §4). ## Where a *new* engine could come from @@ -282,6 +300,7 @@ alkfs intake may name needs externally; OQ-08). | [009](decisions/009-pg-lo-large-tier-engine.md) | pg-lo admitted | postgres Large Objects as the large tier's second engine, on POC #7 | | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | I/O seams + GC-state home | read cursor / staged put; store-core-owned GC state via the engine-state seam; the kv fleet-validity rule | | [011](decisions/011-vocabulary-tiers-engines-and-fleet-state.md) | Vocabulary | tier/engine/instance/node/fleet; mem is a kv engine; fs → large; constructor modes | +| [012](decisions/012-pre-decomposition-consistency-rulings.md) | Fleet activation & GC-state host | `fleet:` constructor declaration; all fleet GC state on the kv engine; pg-lo same-instance clause; kv-only put-time rejection | ## Open Questions @@ -305,6 +324,7 @@ alkfs intake may name needs externally; OQ-08). - rudolfs notes — `list()` anti-lesson, decorator alternative noted and not adopted (threshold dispatch chose the simpler policy; ADR-003 §Context) -- ADR-003, ADR-004, ADR-005; [store-api.md](store-api.md) (the invariants +- ADR-003, ADR-004, ADR-005; ADR-012 (fleet activation, GC-state + host, the composite predicate's same-instance clause); [store-api.md](store-api.md) (the invariants engines must satisfy); [requirements.md](requirements.md) (the pinned vocabulary) \ No newline at end of file diff --git a/docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md b/docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md index 0b336e9..8c17c14 100644 --- a/docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md +++ b/docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md @@ -8,7 +8,13 @@ advisory-locked single sweeper, SQL-level delete arbitration under shared-pool engines (REQ-2) — while keeping this ADR's invariants and the in-process protocol unchanged everywhere else. The "one pool per node" phrasing below reads as "per deployment" for fleets; the -vocabulary is pinned in requirements.md) +vocabulary is pinned in requirements.md. ADR-012 rules the abort +*carrier*: sweep aborts — including the protect callback's — are +`GcAbortCause` report data in `Ok(SweepReport)`, never a typed sweep +error (this ADR's "typed `GcAborted`" phrasing reads through that +ruling), and direct `delete()`'s refusal covers the full protection +set (this ADR's "pinned keys" wording reads as ADR-012 §6.4's +"protected keys")) ## Context @@ -73,9 +79,11 @@ served by range reads). observes the replacement (the local analog of the ops-layer pin token contract). 3. **Protect callback** — consulted pre-sweep; may add known-live keys - or **abort the run** (typed `GcAborted`, nothing deleted). iroh's - `ProtectOutcome::Abort` conclusion adopted: a flaky protection - source skips the sweep rather than risk deletion. + or **abort the run** (nothing deleted; the abort is report data — + `GcAbortCause::ProtectFailed` in `Ok(SweepReport)` per ADR-012 + §1, not a typed error; this ADR's original "typed `GcAborted`" + phrasing is superseded on the carrier only — the semantics iroh's + `ProtectOutcome::Abort` conclusion adopted stand). ### Sweep: mark → delete window → commit @@ -133,10 +141,14 @@ served by range reads). existence flips atomically per key). - **Delete-then-recover is a re-put** (validated: byte-identical under the same key); no tombstone layer exists. -- **Direct `delete(key)`:** refuses pinned keys with a typed error - (same arbitration); permitted for embedder correction flows on - unpinned entries. Outside-of-sweep deletion is unusual by posture — - most deletion should flow through sweeps. +- **Direct `delete(key)`:** refuses protected keys (pinned, + registered-source-live, or protect-callback-named — the full + protection set, ADR-012 §6.4; this ADR's original "refuses pinned + keys" wording is superseded upward; the refusal is the typed + `GcRefuse` error, ADR-012 §1) — same delete-time arbitration; + permitted for embedder correction flows on unprotected entries. + Outside-of-sweep deletion is unusual by posture — most deletion + should flow through sweeps. ### Traversal and scheduling @@ -202,6 +214,6 @@ OQ-07's. gix-odb (alternates/pool prior art) - ADR-002 (one address space), ADR-003 (list contract, pins), ADR-004 (manifest layers above), ADR-006 (verification limits inform recover - semantics) + semantics), ADR-012 (abort carrier + direct-delete protection set) - [gc-and-namespaces.md](../gc-and-namespaces.md); [store-api.md](../store-api.md) \ No newline at end of file diff --git a/docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md b/docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md index e0a3192..32629d5 100644 --- a/docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md +++ b/docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md @@ -8,7 +8,9 @@ passed gate — the two §Consequences passages that said pg-lo "remains unbuilt" are resolved by that same supersession; the tier this ADR's title and text call "fs" is renamed `large` by ADR-011; the fleet GC mechanism's state home and the joint-tx ownership are ruled by -ADR-010; every other part of this ADR stands as written) +ADR-010; fleet-mode activation, the GC-state host, the sweep-abort +return shape, and the window executor are ruled by ADR-012; every +other part of this ADR stands as written) ## Context @@ -105,13 +107,16 @@ Under a fleet, the mechanism moves into the shared engine: single-node/topology-local sweeps that cannot express their protection as rows. - **One sweeper per pool.** A fleet node acquires a pg advisory lock - before sweeping and holds it for the whole sweep. Two nodes - sweeping concurrently is not a state this crate tolerates — the - lock is the admission gate, and a second-node sweep returns the - typed `GcAborted` (no deletion) rather than racing. `GcAborted` - thus carries two cause classes (protection-source failure — - ADR-005; sweeper lock contention — this ADR); the typed error's - fleet variant identifies which. + before sweeping and holds it for the whole sweep. Two nodes + sweeping concurrently is not a state this crate tolerates — the + lock is the admission gate, and a second-node sweep returns the + typed `GcAborted` (no deletion) rather than racing. `GcAborted` + thus carries two cause classes (protection-source failure — + ADR-005; sweeper lock contention — this ADR); the typed error's + fleet variant identifies which. *(Return carrier corrected by + ADR-012 §1: the abort is `GcAbortCause::SweeperLock` report data — + `Ok(report)` — the typed sweep error is retired; the two cause + classes and the no-deletion semantics stand.)* - **Delete-time arbitration is SQL-level.** The delete window's re-check (ADR-005: "arbitrate under the pin/liveness lock at delete time") executes as `SELECT … FOR UPDATE` against pin rows and @@ -120,22 +125,30 @@ Under a fleet, the mechanism moves into the shared engine: as rows carrying `visible_after = mark_end + window` (the delete window's delay — default 5 min, constructor-tunable per deployment); the executor (the sweeper, or the next sweep's owner) - re-arbitrates each staged candidate *at execution time* against - pins and liveness tables before deleting. This is the durable - delete window on every engine — the window table is engine rows on - sqlite as well; what differs by topology is only where the liveness - state lives (in-process maps and callbacks for single nodes; - tables and SQL arbitration for fleets), not whether staging exists. - No external queue dependency is required (the pattern is - engine-native transactions; consumer-side queues like honker or - pg-boss-lineage tools remain consumer-layer options, per ADR-004's - boundary). + re-arbitrates each staged candidate *at execution time* against + pins and liveness tables before deleting. This is the durable + delete window on every SQL-backed engine — the window table is + engine rows on + sqlite as well; what differs by topology is only where the liveness + state lives (in-process maps and callbacks for single nodes; + tables and SQL arbitration for fleets), not whether staging exists. + *(ADR-012 §5/§6.7: "every SQL-backed engine" is the final wording + — `local`/`mem` stage in-process, window-inline; the executor of + overdue staged rows is the next sweep.)* No external queue + dependency is required (the pattern is + engine-native transactions; consumer-side queues like honker or + pg-boss-lineage tools remain consumer-layer options, per ADR-004's + boundary). - **Single-node postures are unchanged.** sqlite and single-instance - postgres keep ADR-005's in-process protocol verbatim: the pin map, - RAII guards, and arbitration lock stay in-process; only the - *persistence* of the delete window (the staged-candidate table) - is shared machinery. The pin-row expiry/ttl machinery exists only - where fleet engines are active. + postgres keep ADR-005's in-process protocol verbatim: the pin map, + RAII guards, and arbitration lock stay in-process; only the + *persistence* of the delete window (the staged-candidate table) is + shared machinery. The pin-row expiry/ttl machinery exists only + where fleet engines are active. *(Activation is explicit: ADR-012 + §3's `fleet: true` constructor declaration turns the fleet + machinery on; it is never inferred from engine choice alone — a + single-instance postgres declared `fleet: false` keeps the + in-process protocol.)* ### 3. The large tier is engine-selectable; fleet topology dictates which engines are permissible @@ -166,7 +179,9 @@ valid: (verified fetch/put; ADR-001). Client nodes' constructors are configured with **kv-only mode** (ADR-011 §4's name for what this bullet first called "kv-tier-only dispatch") — no local large tier - over pool content at all; a client node's *node-private, + over pool content at all; over-threshold puts reject at put time + (ADR-012 §6.2), and the re-routing posture handles them via the ops + surface; a client node's *node-private, consumer-owned* large-tier content is outside the pool and outside this crate's GC entirely: it is the consumer's own storage beside the pool per ADR-004, not a constructor mode over pool content, and @@ -176,23 +191,21 @@ valid: a client node is its own store for sub-threshold content plus its registered ops-put path for large content — a consumer-side composition, not a store-level mode. -- `pg-lo` (admitted by ADR-009; the gate): postgres Large Objects +- **`pg-lo` (shipped; admitted by ADR-009):** postgres Large Objects as the large tier's storage — the same "engine behind one contract" - move as ADR-007, one tier over. Originally named as a candidate, - admission gated on measured evidence (POC #7: LO write/read - curves at the packfile regime, tx-scoped handle cost under - pooling, `pg_largeobject`/vacuum posture under churn, crash-orphan - behavior) and its own engine ADR. NOT shipped speculatively; - REQ-2's immediate fleet answers are shared media or re-routing; - pg-lo is the consolidation option if those are unacceptable. *Update - 2026-10-03: POC #7 ran and passed its gate — - `docs/research/poc-pglo-findings.md` (companion-table authority, + move as ADR-007, one tier over. Originally named here as a candidate + gated on measured evidence, POC #7 ran and passed that gate + (`docs/research/poc-pglo-findings.md`: companion-table authority, `lo_get` window gets, durable put ≈ fs's durable put, cached gets behind page-cache fs by a measured 20-50x single-stream, ~700 MB/s - aggregate at 16 readers). Superseded the same day: ADR-009 admitted - the engine — this bullet's original "candidate, not shipped" - posture is historical; the engine is shipped, feature `pg-lo`.* + aggregate at 16 readers), and ADR-009 admitted the engine the same + day — this bullet's original "candidate, not shipped / REQ-2's + immediate answers are shared media or re-routing; pg-lo the + consolidation option" posture is historical; the engine is shipped, + feature `pg-lo`, and it is the fleet consolidation option as a + first-class engine (ADR-012 §4 adds the same-instance requirement + under fleet mode). - **The no-mixed-large-tiers rule is a deployment invariant, enforced at the enforceable seam.** Cross-node configuration cannot be validated by any one constructor (it sees only its own node). What the @@ -240,8 +253,12 @@ partitioning failure this decision exists to prevent. handle), advisory-lock coordination, and SQL-level arbitration added on top of the already-intricate delete-window state machine. - `pg-lo` was built under ADR-009 after POC #7 passed — the - measured, deliberate sequence of the ADR-006 pattern (admission - gate first, engine second), not a hedge; resolved same-day. + measured, deliberate sequence of the ADR-006 pattern (admission + gate first, engine second), not a hedge; resolved same-day. +- *(ADR-012 §4: fleet GC state hosts on the fleet's kv engine — the + large tier's `local` engine never joins the pin transaction; + large=local fleet puts commit the pin row first, publish + (commit-rename) second.)* **Neutral** diff --git a/docs/architecture/decisions/009-pg-lo-large-tier-engine.md b/docs/architecture/decisions/009-pg-lo-large-tier-engine.md index b354ffe..e1aa428 100644 --- a/docs/architecture/decisions/009-pg-lo-large-tier-engine.md +++ b/docs/architecture/decisions/009-pg-lo-large-tier-engine.md @@ -7,7 +7,10 @@ Accepted (admission evidence: POC #7, passed 2026-10-03 — gate. The tier this ADR calls "fs" is renamed `large` by ADR-011); §2's "put = one transaction" tx-ownership statement is corrected by ADR-010 §2 (the store core holds the tx; the engine contributes its -staged-put mechanics into it — engine content unchanged)) +staged-put mechanics into it — engine content unchanged); ADR-012 §4 +adds the same-pg-instance requirement for fleet-mode pg-lo nodes +(the consolidation posture this ADR described is the fleet-valid +shape). ## Context @@ -56,8 +59,12 @@ consolidation option as a shipped engine. 2. **Put = one transaction**: BEGIN → `lo_create` → chunked `lowrite` (512 KiB statements — the measured knee; statement count, not LOBLKSIZE, is what costs) → companion-row insert → COMMIT. - Pin-before-publish (ADR-005/008) rides this shape: entry row + pin - row commit together with the content. Unknown-length puts stage + *(Ownership corrected by ADR-010 §2: the store core holds this tx; + the engine contributes LO writes + the companion-row insert into + it.)* Pin-before-publish (ADR-005/008) rides this shape: entry row + pin + row commit together with the content — the pin-row insert is the + store's contribution (hosted on the fleet's kv engine under fleet + mode, ADR-012 §4 — same pg instance, one tx). Unknown-length puts stage into the LO and commit the row on completion; rollback discards both. 3. **Get = `lo_get(oid, off, len)` windows by default** (POC #7 C4): @@ -100,7 +107,7 @@ consolidation option as a shipped engine. many-user aggregate. A third engine (S3-like) remains the ADR-003 door; nothing here touches it. 7. **redb-class guardrail maintained:** this is the second engine the - admission door ships (ADR-007: kv; ADR-009: fs) — both on the same + admission door ships (ADR-007: kv; ADR-009: large) — both on the same measured-evidence shape; no third engine is opened by this one. ## Consequences diff --git a/docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md b/docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md index bf8289b..e502eeb 100644 --- a/docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md +++ b/docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md @@ -2,7 +2,12 @@ ## Status -Accepted +Accepted (fleet-state activation, the GC-state host under mixed +engines, the sweep-abort return shape, the pin token's wire shape, +the sqlite pin/sweep-lock clause reductions, and the window staging +for non-SQL engines are ruled by [ADR-012](012-pre-decomposition-consistency-rulings.md); +§2's sqlite rows and the "fleet state is active" predicate read +through that ADR — where noted inline) ## Context @@ -139,7 +144,10 @@ The store core reaches the fleet machinery through a **crate-internal SQL-backed engines (sqlite, postgres, pg-lo), with `local` and `mem` excluded by construction (it cannot be expressed on them and they are never fleet engines). ADR-008's mechanism fixes *what*; the -seam is *where* the mechanism's SQL lives. Clauses: +seam is *where* the mechanism's SQL lives. Clauses (per-engine +applicability is ruled by [ADR-012](012-pre-decomposition-consistency-rulings.md) +§5's table, which reduces the original sqlite rows; fleet state hosts +on the kv-tier engine when fleet mode is declared — ADR-012 §4): - `ensure_gc_schema()` — create `gc_pins (key, owner, expiry)` and `gc_window (key, visible_after)` at store construction (store-core @@ -152,15 +160,19 @@ seam is *where* the mechanism's SQL lives. Clauses: ADR-005's refcount semantics expressed in rows. The embedder's liveness table is *not* created by this seam — ADR-008: schema pinned by the embedder; the sweep reads it and never owns it. - **Per-engine applicability (the test-gate shape):** sqlite — - `ensure_gc_schema` (window table only, since the window staging - is shared machinery per ADR-008 §2) + `window_stage`/ - `window_take` + the sweep-lock clause's single-node body; the - pin clauses (`pin_commit`/`pin_renew`/`pin_check`) have sqlite - bodies but are exercised by unit tests only — they are dead - machinery on sqlite because no sqlite fleet is on offer (§3). - postgres and pg-lo — all clause families, contract-tested, - fleet-active. + **Per-engine applicability (the test-gate shape; final per + ADR-012 §5):** sqlite — schema creates **`gc_window` only** + + `window_stage`/`window_take` (durable single-node staging); + the originally-specced sqlite pin clauses and sweep-lock body + are **dropped** by ADR-012 §5 (the pin bodies inserted into a + table nothing created — dead machinery; sqlite has no advisory + lock, and ADR-008's unchanged single-node protocol keeps the + sweep single-flighter in-process). postgres — all clause + families, contract-tested, fleet-active (the fleet host). pg-lo — + the window family only (durable window staging where the kv + engine is not SQL-backed — the one-window-host rule, ADR-012 §5; + it hosts no fleet pin/lock state, + ADR-012 §4), contract-tested. - `pin_commit(joint_tx, entries)` — the store core **holds the transaction**; the engine contributes its put mechanics (row shape, CAS clause, LO writes) and the pin-row inserts, and both @@ -168,37 +180,59 @@ seam is *where* the mechanism's SQL lives. Clauses: ruled: **the store holds the tx; the engine contributes mechanics to it.** The public `put` remains the plain, pin-free path — the store's normal put path routes through `pin_commit` wherever the - engine is seam-capable and fleet state is active. + engine is seam-capable and fleet mode is active (**activation + ruled by ADR-012 §3: fleet mode is an explicit `fleet: true` + constructor declaration, never inferred from the engine choice**; + for large=local fleet puts the joint tx is unachievable across + media — ADR-012 §4 pins pin-row-first, publish-second ordering). - pg-lo correction: ADR-009 §2's "put = one transaction" is unchanged in content but its ownership statement is hereby corrected — the tx is held by the store core; the pg-lo engine contributes its staged-put mechanics and companion-row insert into it. The engine sees pins only as rows; never as semantics. -- `pin_renew(key, owner, ttl)` — the engine executes the expiry - update. The store's pin guard owns the TTL/3 renewal schedule - (ADR-008); the engine owns only the SQL. +- `pin_renew(key, owner)` — the engine executes the expiry update + (renamed and TTL dropped by ADR-012 §2: the TTL lives with the + store's renewal schedule, not in the clause signature; the original + `pin_renew(key, owner, ttl)` read is superseded). The store's pin + guard owns the TTL/3 renewal schedule + (ADR-008); the engine owns only the SQL. *(postgres host only per + ADR-012 §5 — sqlite bodies dropped.)* - `pin_check(key) → PinState` / `arbitrate(key) → bool` — the engine executes ADR-008's delete-time arbitration (un-expired pin rows + embedded liveness tables, `SELECT … FOR UPDATE` finger-lock, keyed, at delete time); the store core receives the answer and - applies the decision. If fleet clauses ever grow, they grow here — + applies the decision. `PinState` is a data enum with two variants: + `Protected` (an un-expired pin row exists) and `Unprotected` — the + engine never decides deletion, it only classifies; *when the seam + supports SQL-level liveness tables the store consults those through + `arbitrate` instead, which folds the full answer into the bool*. If + fleet clauses ever grow, they grow here — the seam is the named home, with contract tests per clause. - `window_stage(candidates)` / `window_take(now)` — the staged - delete-window candidates. Per ADR-008 §2 the window table exists - on **every** engine (sqlite included): staged candidates are + delete-window candidates. Per ADR-008 §2 the durable window table + exists on **every SQL-backed engine** (final wording per ADR-012 + §5: sqlite included; `local` and `mem` are excluded from the seam + here and stage candidates in-process, executing the ADR-005 window + protocol inline in the same sweep; in dual-SQL instances the + non-host engine's window clauses are dormant — the one-window-host + rule, ADR-012 §5): staged candidates are engine rows; what differs by topology is where *liveness* lives (in-process maps for single-node engines, tables for fleets), not - whether staging exists. + whether staging exists among SQL-backed engines. - `sweep_lock_acquire()/release()` — the sweep single-flighter: the - pg advisory lock (ADR-008) for the SQL engines it applies to; the + pg advisory lock (ADR-008), applying to the fleet host (postgres) + only — ADR-012 §5 drops the originally-specced sqlite body (no + advisory-lock equivalent; in-process lock suffices and is + ADR-008-verbatim). The store treats lock contention exactly as ADR-008 says — the second - sweeper's sweep aborts with the typed `GcAborted` (fleet cause), - nothing deleted. (SQLite today has no fleet story — see §3 — so its - clause body exists but is exercised by single-node tests only.) + sweeper's sweep aborts with **`GcAbortCause::SweeperLock` as report + data (`Ok(report)` — ADR-012 §1 retired the typed `GcAborted` sweep + error)**, nothing deleted. Each clause is contract-tested per its applicability (the per-engine split above): the conformance suite of §1 grows the fleet -clauses for postgres and pg-lo; `local` and +clauses for postgres (the host); pg-lo runs the window-family +contract tests; `local` and `mem` are excluded by construction. ### 3. The kv tier's fleet-validity rule (the mirror of ADR-008 §3) @@ -222,10 +256,19 @@ engines the put's pin is the expiry-carrying row (ADR-008); the token identifies the putter as the row's owner. The token itself is an **opaque id minted by the ops layer** over the pin row it just committed — the ops layer (not the store facade) owns the -token↔(key, owner) mapping, in-memory for the handler's lifetime plus -a lookup by the renewal op; no schema is specced for it beyond that +token↔(key, owner) mapping, in-process state with **process lifetime** +(ADR-012 §2 corrected the original "handler's lifetime" — the handler's +guard drops at response time, but the mapping must outlive it, or +renewal could never arrive); no schema is specced +for it beyond that (it is a handle, not durable state — the durable state is the pin row -the seam wrote). Concretely: +the seam wrote). The token's **validity domain is the minting node** +(ADR-012 §2: renewal probes reach the serving node that minted the +token; fleet peers refuse foreign tokens; a dead minting node is the +ADR-005 crash case with the re-put backstop). +**Wire shapes (ADR-012 §2):** the `blobs/put` response is +`{token, digest}`; renewal rides the token form of `blobs/have` +(one digest, token alongside). Concretely: - **When the server-side `Pin` guard drops:** at response time. The put lands pinned (the joint tx guarantees the entry and its pin row @@ -259,7 +302,10 @@ the seam wrote). Concretely: had it: nothing persists, nothing expires; the process crash is ADR-005's in-flight-put loss case with the same backstop. (The cross-wire token under `local` is the ops layer's in-memory handle - to that guard — same mapping discipline, no expiry.) + to that guard — same mapping discipline, no expiry.) ADR-012 §2 + generalizes this to every non-fleet mode: the token is the + in-process guard's handle on sqlite, single-instance postgres, and + pg-lo serving nodes too — nothing expires outside fleet mode. ops-surface.md's hand-over contract gains the renewal line and the token-mapping note; nothing else changes. @@ -284,17 +330,15 @@ token-mapping note; nothing else changes. **Negative** -- A crate-internal companion trait is real machinery: six clause - families on postgres/pg-lo (fleet-active, all contract-tested); - sqlite carries only the window + lock clauses as fleet-relevant - machinery (its pin clauses have unit-test bodies but no fleet mode - on offer — §3). The seam bounds where fleet-GC growth lands; it - does not shrink it. +- A crate-internal companion trait is real machinery: the full clause + family on postgres (the fleet host — pins, window, sweep lock, + contract-tested, fleet-active); sqlite and pg-lo carry the window + family only (ADR-012 §5's reduction — the original draft's sqlite + pin/sweep-lock bodies were dead machinery and are dropped). The + seam bounds where fleet-GC growth lands; it does not shrink it. - Remote putters gain a real protocol duty (renew past 20 s; - pause-past-TTL ⇒ delete-then-recover) — documented, but a duty. -- The sqlite sweep-lock clause body exists but is exercised only by - single-node tests until a sqlite-fleet posture is ever - decided (it is not on offer — §3). + pause-past-TTL ⇒ delete-then-recover; route renewal to the minting + node — ADR-012 §2) — documented, but a duty. **Neutral** @@ -314,7 +358,10 @@ token-mapping note; nothing else changes. contract the cursor serves), ADR-008 (the fleet mechanism whose state home this ADR is), ADR-009 (the pg-lo engine; tx-ownership correction), [ADR-011](011-vocabulary-tiers-engines-and-fleet-state.md) - (vocabulary + constructor table) + (vocabulary + constructor table), [ADR-012](012-pre-decomposition-consistency-rulings.md) + (fleet activation predicate, GC-state host, sweep-abort shape, + token wire shape, the §5 clause reduction this ADR's seam table + now references) - `docs/research/poc-trait-dispatch-findings.md` finding 1; `docs/research/poc-largeblob-findings.md` findings A1/A5/A6; `docs/research/poc-pglo-findings.md` (C1-C7 — the handle shape, the diff --git a/docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md b/docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md index 2b26877..011315a 100644 --- a/docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md +++ b/docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md @@ -2,7 +2,12 @@ ## Status -Accepted +Accepted (fleet-mode activation, the GC-state host, the sweep-abort +shape, and kv-only's put-time rejection point are ruled by +[ADR-012](012-pre-decomposition-consistency-rulings.md) — this ADR's +constructor table is closed under ADR-012 §4's host rule; §4's +"constructor-time error" wording reads as ADR-012 §6.2's put-time +rejection) ## Context @@ -147,12 +152,17 @@ explicit shared-media assertion, ADR-008 §3, is what disambiguates). Constructor modes (dispatch shapes) — pinned here (implied by ADR-008 §3's re-routing bullet and the mem tests, but never named): - **dual-tier** (the default): kv + large, size-threshold dispatch - between them (ADR-003). + between them (ADR-003). Engine pairs beyond the default + (`kv = mem` + `large = local`) are valid dual-tier constructions — + ADR-012 §6.3. - **kv-only** (ADR-008's "kv-tier-only dispatch", now named): a - constructor mode with no large tier; over-threshold puts are a - constructor-time error (the client-node re-routing posture handles - them via the ops surface instead — its composition, not a store - mode). + constructor mode with no large tier; over-threshold puts reject at + **put time** — known-length immediately, unknown-length at + threshold overflow (the original "constructor-time error" wording + is corrected by ADR-012 §6.2: lengths are not known at + construction). The client-node re-routing posture handles + over-threshold content via the ops surface instead — its + composition, not a store mode. - **mem-only** (test/embedder-ephemeral mode): the mem engine alone, no dispatch threshold in effect (everything is local and small); a testing posture, explicitly not a production story. @@ -161,7 +171,9 @@ ADR-008 §3's re-routing bullet and the mem tests, but never named): ADR-004's requirement (both tiers are load-bearing by measured economics); "postgres-only" as in *one SQL instance serving both tiers* already exists (kv=postgres + large=pg-lo — the ADR-009 - consolidation), which is the dual-tier mode over one engine. + consolidation), which is the dual-tier mode over one engine — + under fleet mode that one instance is a requirement, not a + coincidence (ADR-012 §4). ### 5. Deferral-policy wording (README alignment) @@ -216,7 +228,10 @@ this ADR's wording here rather than a silent edit). and the fleet topology; the vocabulary it assumed is now pinned), ADR-009 (pg-lo admission; the consolidation option whose constructor mode this ADR names dual-tier), [ADR-010](010-backend-io-seams-and-fleet-state-home.md) - (the I/O seams and GC-state home this vocabulary must fit) + (the I/O seams and GC-state home this vocabulary must fit), + [ADR-012](012-pre-decomposition-consistency-rulings.md) (fleet-mode + activation + GC-state host + kv-only put-time rejection — the + constructor table's final closure) - [requirements.md](../requirements.md) (the canonical vocabulary copy, REQ-1..4); [backends-and-dispatch.md](../backends-and-dispatch.md) (the tier/engine table's spec home) diff --git a/docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md b/docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md new file mode 100644 index 0000000..faac514 --- /dev/null +++ b/docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md @@ -0,0 +1,352 @@ +# ADR-012: Pre-decomposition consistency rulings — sweep-abort shape, pin-token wire shape, fleet-mode activation, and the GC-state host + +## Status + +Accepted + +## Context + +The ADR-011 vocabulary round was followed by a full-tree consistency +review — the gate before task decomposition. The vocabulary itself +held everywhere; what the review surfaced is a set of **composition +defects**: mechanisms each specced correctly in isolation, whose +interaction was never ruled, plus two caller-facing shapes stated two +ways. All are pre-implementation (zero code exists), all deciding +facts are in hand, and none waits on a consumer: + +1. **The sweep-abort return shape is stated two ways.** + store-api.md's §Lifecycle pins an aborted sweep as + `Ok(SweepReport { aborted: Some(GcAbortCause), .. })` ("the report + is data, not an error carrier"), while §Error model of the *same + file* defines `GcAborted` as the sweep-abort error carrying the + cause classes — and ADR-005, ADR-008, ADR-010, and + gc-and-namespaces.md all follow the error reading. An implementer + writing `sweep()` cannot pick a signature. +2. **The pin token has no wire shape.** The renewal contract (ADR-010 + §4) has the putter renew by presenting the token "alongside the + digest" via `blobs/have` — but the specced `have` payload is + `{namespace, digests[]}` with no token field, `blobs/put`'s + token-carrying response is never specced, and an unknown-length + putter does not know its digest until the server computes it. The + token's *validity domain* is also unstated (renewal may arrive + after the minting handler's guard dropped, possibly to a different + fleet node). +3. **"Fleet state is active" has no activation predicate.** ADR-010 + routes puts through `pin_commit` "wherever the engine is + seam-capable and fleet state is active", and ADR-008 keeps + single-node postures on the in-process protocol — but nothing + defines what activates fleet state. Under ADR-011's vocabulary + fleetness (two instances sharing one pool) is not observable by + any one constructor. +4. **The GC-state home under mixed fleet engines is unnamed.** In the + kv=postgres + large=local-on-shared-media fleet posture (valid per + the composite rule), pin-before-publish for a large entry is a file + rename, not an insert — the joint entry+pin tx is not achievable on + the large tier's medium — and nothing says which engine hosts + `gc_pins`/`gc_window`/the sweep lock, or whether two SQL engines + mean two pin tables and an arbitration union. +5. **The sqlite seam rows are self-contradictory.** ADR-010 §2 rules + sqlite's `ensure_gc_schema` creates the window table *only*, yet + assigns sqlite `pin_commit`/`pin_renew`/`pin_check` "bodies + exercised by unit tests only" — bodies that insert into a table + nothing creates — and gives sqlite a sweep-lock clause body whose + semantics are undefined (sqlite has no advisory locks) and which + ADR-008's "single-node postures unchanged" says never runs. + +The review also surfaced smaller caller-facing gaps (fall-through for +non-get ops, the kv-only error point, a missing abort cause, the +fleet liveness-table registration's API home) that need a ruled home +before signatures are written; §6 records them. + +## Decision + +### 1. Sweeps are never errors — every abort is data in the report + +`sweep()` returns `Result` where the error family +exists for *operational* failure (I/O, engine failure), never for a +deliberate no-op. All abort causes are report data: + +- **`GcAborted` is retired from the error enum.** The only abort + vocabulary is `GcAbortCause`, a data enum carried in + `SweepReport.aborted: Option`, with exactly three + variants: `ProtectFailed` (ADR-005's protection-source failure), + `SweeperLock` (ADR-008's fleet sweeper-lock contention), + `NoLivenessSources` (the safe default — a sweep invoked with zero + registered sources; previously this cause existed in prose, + store-api.md's sweep bullet, but had no variant to carry it). +- Direct `delete()` on a protected key keeps a typed refusal error, + named **`GcRefuse`** — a distinct name so no reader composes the + retired `GcAborted` semantics onto it. (This renames the refusal + path; nothing else about it changes — ADR-005's arbitration still + decides at delete time.) +- `GcAbortCause` is matched non-exhaustively by callers; new causes + are new ADRs. + +Rationale: fleet lock contention is a *routine, expected* outcome of +scheduled sweeps — the embedder's next run takes the lock — and a +typed error would push the common case onto the exceptional path. +The report is the data embedders schedule on; "sweep counts are +exact" (the test gate) is asserted against report fields. + +This ruling picks store-api.md §Lifecycle's shape over the error +shape used by ADR-005/008/010 and the gc spec; those documents' +wording is aligned to it (mechanical edits; no semantic change — all +four said "nothing deleted" identically, they only disagreed on the +carrier). + +### 2. The pin token's wire shape and validity domain + +**`blobs/put` response: `{token, digest}`.** Both fields always +present: `digest` is the stored entry's canonical digest (hex) — the +unknown-length putter cannot know it beforehand and needs it for +every later op; `token` is the opaque pin-token handle (ADR-010 §4's +minting unchanged). Known-length putters receive the same shape. + +**`blobs/have` gains the token as the renewal form.** Payload +`{namespace, digests[], token?}`: +- without `token` — unchanged: pure existence probe over `digests[]` + (ADR-008's `has` discipline preserved; response `present[]` + corresponds to `digests[]` by index); +- with `token` — the **renewal probe**: exactly one digest in + `digests[]`, and the handler (a) verifies the token maps to a pin + row for that digest, (b) calls the crate-internal `pin_renew(key, + owner)` via the ops module's store-core access (ADR-010 §4's + mechanism, unchanged), (c) responds with `present[]` as usual — the + putter's registration-landed signal. One token binds one pin row, + so renewal is one digest per probe; a putter renewing several pins + issues one probe per token. If the row has expired (putter paused + past TTL), the handler finds no live row: `present: false` and no + renewal occurs — the putter's backstop is re-put (ADR-005's + delete-then-recover), making ADR-010 §4's expiry race end-to-end + explicit on the wire. + +**Token validity domain: the minting node.** The token↔(key, owner) +mapping is ops-layer, in-memory state in the process that minted it +(process lifetime, not handler lifetime — the handler's guard drops +at response time per ADR-010 §4, but the mapping outlives the +handler); renewal probes must reach the minting serving node, and +fleet peer nodes cannot serve a token they did not mint (`present: +false`-equivalent refusal, not an error). A putter whose minting node +died before renewal is exactly ADR-005's in-flight-put loss case with +the same backstop: re-put. This is a documented duty of the +re-routing/replication embedder (route renewal to the serving node), +not new machinery. + +**Non-fleet serving nodes (any engine):** the token is the ops +layer's in-memory handle to the in-process `Pin` guard — ADR-005 +posture, nothing expires. This sentence generalizes ADR-010 §4's +`local`-only wording to every non-fleet mode (kv=sqlite, kv=postgres +single-instance, large=pg-lo non-fleet). + +### 3. Fleet mode is a constructor declaration + +**A store instance is constructed with fleet mode explicitly on or +off; it is never inferred.** Under `fleet: true`: pins are expiry +rows, puts route through `pin_commit`, sweep acquires the SQL sweep +lock, and the composite fleet-validity predicate (with its per-node +declarations — ADR-008 §3's media assertion, kv-only re-routing +declaration) is enforced at construction. Under `fleet: false`: the +ADR-005 in-process protocol verbatim, regardless of engine. + +ADR-011's *deployment* vocabulary is unchanged (fleet = two or more +instances sharing one pool); the constructor flag is the per-instance +admission of that posture. Two `fleet: true` instances over one +postgres pool are a fleet; an undeclared instance sharing the pool is +**misconfigured** — the observable failure is ADR-008 §2's exact +data-loss scenario, and it is the same class of deployment-verified +invariant as the shared-media declaration: the constructor cannot +prove cross-node/cross-instance truth, the deployment asserts it, and +the detection symptom (an unrenewed, un-pinned entry swept while in +flight) is documented. + +`fleet: true` is a constructor error for every non-fleet-valid +configuration: any kv engine other than postgres, `mem` anywhere, +`local` large tiers without the shared-media declaration, `pg-lo` on +a different pg instance than the kv engine (§4). + +### 4. GC state has exactly one home: the fleet's kv engine + +**All fleet GC state — `gc_pins`, `gc_window`, the sweep lock — lives +on the kv-tier engine.** Fleet mode requires kv=postgres, so +postgres hosts the tables and the advisory lock for the whole store +instance, large tier included. The per-engine alternative (each SQL +engine hosting pin rows for its own tier, arbitration unioning two +tables) is rejected: two pin tables mean two arbitration domains and +two locks — the partitioning failure ADR-008 §3 exists to prevent, +re-expressed inside one store instance. One host is the whole point +of the seam. + +Consequences per fleet large-tier engine: + +- **`large = pg-lo` (fleet): the same pg instance as the kv engine is + required** — this is ADR-009's consolidation posture ("pool content + lives in the same pg instance the kv tier rides") now made a + checkable clause of the composite fleet-validity predicate, because + it is what makes the joint entry+pin tx expressible: the store core + holds one transaction over the shared instance; the pg-lo engine + contributes LO writes + companion-row insert; the pin-row insert is + the store's (§2 of ADR-010, ownership unchanged). Cross-instance + kv=postgres + large=pg-lo is valid *only* with `fleet: false` + (in-process protocol). +- **`large = local` on declared shared media (fleet): pin-first, + publish-second.** The joint tx is not achievable across media, so + the ordering is pinned: the pin row commits on the kv engine + *before* the stage file's commit-rename publishes the entry. + - The invariant survives: an entry becomes visible only after its + pin row is durable — sweep arbitration (which queries pin rows) + can never observe a visible-unpinned entry. + - The crash window moves to the harmless side: crash between + pin-commit and rename leaves an orphan pin row, reaped by the + existing expiry machinery (ADR-008's TTL + sweeper maintenance + step). No new recovery mechanism. + - The reverse ordering (rename-then-pin) is rejected: a crash + there reopens ADR-005's original race (visible, unpinned, + sweepable). + +**Non-fleet postures are unchanged and single-hosted:** the GC +arbitration domain is the store instance (ADR-005/011); pins and +liveness arbitration are in-process; the durable window table (§5) +may live on a SQL engine but carries no fleet semantics. + +### 5. The engine-state seam's final shape (sqlite rows repaired) + +Per-engine clause applicability — the correction of ADR-010 §2's +contradictory rows: + +| Engine | `ensure_gc_schema` | Window clauses | Pin clauses | Sweep-lock clause | +|---|---|---|---|---| +| postgres | `gc_pins` + `gc_window` | yes (fleet-active when declared) | yes (fleet-active when declared) | yes (pg advisory lock; fleet-active when declared) | +| sqlite | `gc_window` only | yes (durable single-node staging) | **no** (bodies dropped) | **no** (body dropped — single-flighter is the in-process lock) | +| pg-lo | `gc_window` only | conditional — see the one-host rule below | **no** (fleet pins live on the kv host, §4) | no | +| `local`, `mem` | — excluded from the seam by construction (ADR-010 unchanged); their delete-window staging is **in-process** (ADR-005 protocol inline) | | | | + +**The one-window-host rule (per store instance):** the durable staged +window lives on **one** SQL-backed engine — the kv engine when it is +SQL-backed (sqlite or postgres), otherwise the large engine (pg-lo, +e.g. the kv=mem + large=pg-lo shape). A dual-SQL instance (kv=sqlite + +large=pg-lo, non-fleet) stages on the kv engine only; pg-lo's +window clauses are dormant there. Never two window tables in one +instance — the candidate set is one set, and a split staging domain +would be a miniature of the two-arbitration-domain failure §4 +rejects. + +- The sqlite pin-clause bodies are **dropped**: they were dead + machinery inserting into a table nothing created. A sqlite fleet is + not on offer (ADR-010 §3); if one ever is, it is a new ADR that + re-adds the clauses with their schema. +- The sqlite sweep-lock body is **dropped**: sqlite has no advisory + lock, and ADR-008's "single-node postures keep the arbitration lock + in-process" is the actual mechanism. The seam's sweep-lock clause + is the pg advisory lock and applies to the fleet host only. +- pg-lo remains a seam implementer for the window family (it hosts + the durable window on non-fleet topologies where the kv engine is + not SQL-backed, e.g. kv=mem + large=pg-lo test shapes) and + contributes staged-put mechanics into the store-held tx in the + consolidation posture; it hosts no fleet state. +- The durable staged delete window stays a **SQL-backed-engine + mechanism** (sqlite included) per ADR-008 §2 — what this ruling + adds is the non-SQL half: "`local` and `mem` stage candidates + in-process and execute the ADR-005 window protocol inline within + the same sweep." "On every engine" (ADR-008 §2, ADR-010 §2) is + corrected to "on every SQL-backed engine". + +### 6. Facade clarifications (ruled, not deferred) + +1. **Fall-through applies to every key-addressed op, not just + get/read_range.** `stat`, `has`, and direct `delete` all consult + the kv tier first, then the large tier (a `size` probe for + `stat`/`has`; arbitration-then-delete where the entry lives for + `delete`). POC #1 finding 6's fall-through is the general rule. +2. **kv-only mode's over-threshold rejection is a put-time error.** + Known-length puts (`len: Some`) exceeding the threshold reject + immediately; unknown-length puts reject at threshold overflow + (mid-buffer). The prior "constructor-time error" wording was + impossible — put lengths are not known at construction. +3. **`dual-tier` with `kv = mem` + `large = local` is a valid + construction** (the dispatch-coverage test shape; `mem` is merely + the kv engine choice in dual-tier mode). `mem-only` remains the + no-large-tier ephemeral mode. Both are non-fleet by definition. +4. **Direct `delete()` refuses any protected key** — pinned, + observed-live via a registered source, or named by the protect + callback — not just pinned keys. This is the ADR-005 invariant + ("a visible pool entry is never deleted while liveness protects + it") applied to the direct path; ADR-005's "refuses pinned keys" + wording is aligned upward. An embedder's correction flow + deregisters/re-registers around the delete. +5. **Error-model member vs return shape:** `get`/`stat` return + `Option` (a miss is a value, not an error — `Missing` is not + theirs); `read_range` and `delete` return `Err(Missing)` on absent + keys (informative for the embedder flows that use them). + `Io(String)` covers backend media failure including cursor and + staged-put I/O errors; `Verification` covers hash-check failure on + put and get paths alike; `KeyInvalid` malformed keys. `Backend` + trait methods return this error family (crate-internal variants, + thiserror). +6. **Fleet liveness-table registration has an API home:** the fleet + form of `register_liveness_source` registers a *named table + reference* in the shared engine (the embedder's table per ADR-008 + §2; the sweep's SQL arbitration consults it). The facade method is + the same verb with a fleet-shaped argument; the store core reads + the table and never owns it (ADR-008's schema-ownership rule). +7. **The staged window's executor is the next sweep** (ADR-008 §2's + "the sweeper, or the next sweep's owner"): a sweep takes overdue + `gc_window` rows (past `visible_after`), re-arbitrates each, and + commits the deletes before/alongside its own marking; non-SQL + engines execute inline in the same sweep (§5). Exact-count sweep + invariants are asserted relative to this schedule. + +## Consequences + +**Positive** + +- Every signature the implementation needs is now single-valued: + `sweep()`'s return shape, put/have payload shapes, the fleet + activation predicate, the GC-state host, and the per-engine seam + table. Task decomposition has no contradictory text to resolve. +- The fleet mechanism gained the missing pieces (activation flag, + state host, large=local ordering) without new machinery — every + piece reuses an existing mechanism (constructor declarations, + expiry reaper, the store-held tx). +- The sqlite seam shrank (dead bodies dropped); the per-engine + applicability table is the contract-test matrix. + +**Negative** + +- `GcAborted`'s retirement is a renaming cost across four documents + (mechanical; no semantic change — and no code exists yet to break). +- The pin token gains a documented duty (route renewal to the minting + node) and the have/put payload shapes change from the earlier prose + (a token field and a digest return field) — wire-shape decisions + locked before any consumer, per the ADR-002 door discipline. +- Fleet mode is one more constructor parameter with per-node + declaration duties; the deployment-verified invariant class grows + by one row (undeclared-instance sharing). + +**Neutral** + +- No tier, engine, feature flag, or constructor *mode* changes; the + composite fleet-validity predicate gains one clause (pg-lo + same-instance) it had already assumed in prose (ADR-009). +- SweepReport, GcAbortCause, GcRefuse, and the fall-through/put-time + rules are the only *new* facade vocabulary; everything else in + store-api.md stands. + +## References + +- ADR-005 (the GC invariant and single-node protocol this ADR extends + and aligns), ADR-008 (fleet GC mechanism — §2's window wording and + §3's declarations feed §4/§5 here), ADR-009 (pg-lo admission — + §4's same-instance clause formalizes its consolidation posture), + ADR-010 (the I/O seams and GC-state home — §1/§2/§4 here repair its + sqlite rows and rule its activation predicate; §2 here completes + its §4 token contract with a wire shape), ADR-011 (the vocabulary + this ADR's constructor flag and host rules compose with) +- [store-api.md](../store-api.md) (sweep shape, error model, + invariants); [ops-surface.md](../ops-surface.md) (the op payloads + §2 pins); [backends-and-dispatch.md](../backends-and-dispatch.md) + (the composite fleet-validity rule §4 amends; constructor modes); + [gc-and-namespaces.md](../gc-and-namespaces.md) (the delete-window + execution model §6.7 names) +- `docs/research/poc-trait-dispatch-findings.md` finding 6 + (fall-through, §6.1's basis); `poc-pglo-findings.md` (the + consolidation posture §4 formalizes) \ No newline at end of file diff --git a/docs/architecture/gc-and-namespaces.md b/docs/architecture/gc-and-namespaces.md index 2460c8e..fc5f855 100644 --- a/docs/architecture/gc-and-namespaces.md +++ b/docs/architecture/gc-and-namespaces.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-010 state-home rule + ADR-011 instance/fleet vocabulary) +last_updated: 2026-10-03 (ADR-012 — aborts are report data (GcAbortCause), never sweep errors; window executor = the next sweep; non-SQL engines stage in-process) --- # Pooling, namespaces, and GC @@ -69,13 +69,20 @@ semantics, delete-then-recover as byte-identical re-put): 2. put-path **pins** (RAII; batch scope available for multi-put writes — manifest writes are exactly this), 3. the **protect callback** consulted before each sweep: it may add - externally-known hashes or **abort the run** (`GcAborted`; a - flaky protection source skips the sweep rather than risk - deletion — iroh's `ProtectOutcome::Abort` conclusion, adopted). + externally-known hashes or **abort the run** (report abort — + `GcAbortCause::ProtectFailed`, nothing deleted; ADR-012 §1: + sweeps are never errors; a flaky protection source skips the + sweep rather than risk deletion — iroh's + `ProtectOutcome::Abort` conclusion, adopted). - **Sweep:** enumerate the whole pool (backends' complete `list()`, ADR-003), compute the live set, batch-delete the dead (batch-sized, ~100/batch, iroh's proven shape). Delete-then-recover is a re-put — byte-identical under the same key (validated). + Overdue delete-window candidates are re-arbitrated and executed by + the sweep that finds them — the executor is the next sweep + (ADR-012 §6.7); SQL-backed engines stage candidates durably + (engine rows), `local`/`mem` stage in-process and execute the + window inline in the same sweep (ADR-012 §5). - **Traversal ownership: lean.** Liveness computation beyond "these roots exist" is the consumer's job, handed over via the seam. The store never learns manifest formats. If a real consumer's live-set @@ -100,7 +107,8 @@ window batches the arbitration). Test-asserted property: **a visible pool entry is never deleted while liveness (pin or registered source) protects it, and an in-flight put is never deleted by a sweep started before it committed.** Direct -`delete(key)` refuses pinned keys (typed error; same arbitration). +`delete(key)` refuses protected keys (the typed `GcRefuse` error; the +full protection set per ADR-005's invariant — ADR-012 §6.4). `has()`/`get()` during a window observe either the old or the new state; there is no torn observation (the pool's entries are immutable; @@ -131,6 +139,7 @@ existence flips atomically per key). | [004](decisions/004-two-backends-no-third.md) | Structure-blindness | manifests/refs never sink into the store | | [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | Fleet GC extension | DB-backed pins, advisory-locked single sweeper, SQL delete arbitration — for shared-pool (fleet) topologies only; the in-process protocol is unchanged elsewhere | | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | GC-state home | the store core owns all GC state; SQL engines host it via the contract-tested engine-state seam; pin-tx ownership ruled | +| [012](decisions/012-pre-decomposition-consistency-rulings.md) | Abort-as-data + window executor | sweep aborts are `GcAbortCause` report data, never errors; the next sweep executes overdue window rows; non-SQL engines stage in-process; GC state hosts on the fleet's kv engine | ## Open Questions @@ -152,5 +161,6 @@ a future requirement does not create one retroactively. (the re-borrowed conclusions) - rudolfs — the physical-namespace anti-pattern (inverted) - gix-odb — alternates/pool prior art for the p2p case -- ADR-005; [store-api.md](store-api.md) (Pin semantics, sweep API); +- ADR-005; ADR-012 (the abort-as-data ruling, the window executor, + the GC-state host); [store-api.md](store-api.md) (Pin semantics, sweep API); [ops-surface.md](ops-surface.md) (namespace-as-resource ACL) \ No newline at end of file diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md index 047b0d6..714d319 100644 --- a/docs/architecture/open-questions.md +++ b/docs/architecture/open-questions.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 +last_updated: 2026-10-03 (ADR-012 round — no new OQs; overview's pg-only-kv feature-graph deferral recorded in the parked index) --- # Open Questions @@ -47,8 +47,9 @@ is pending; deferral policy applies to decisions, not to work. **Index of active OQs:** OQ-07, OQ-08 (externally-owned, carried for visibility). OQ-09 and OQ-10 are resolved (recorded below). -Promoted Phase 0 questions OQ-BL-01..06 are recorded here with their -resolutions for traceability. +OQ-11 (deferred(scope), parked index) records a feature-graph +deferral. Promoted Phase 0 questions OQ-BL-01..06 are recorded here +with their resolutions for traceability. --- @@ -236,6 +237,17 @@ resolutions for traceability. |---|---|---| | OQ-07 | externally-owned | alkgit's architecture process (answerable on paper anytime; gates nothing here) | | OQ-08 | externally-owned | alkfs Phase 0 intake | +| OQ-11 | deferred(scope) | a named deployment requirement for a pg-only kv node (postgres kv engine without rusqlite in the build) — overview.md's feature-graph note | + +OQ-11 records [overview.md](overview.md)'s dependency-posture deferral +(a pg-only kv node — the `kv` feature's rusqlite pull decoupled from +default-on feature graphing) for auditability: the deciding fact is a +*deployment requirement that arrives from outside this crate* +(Schrödinger's-code rule compliant — it is not a fact this crate +creates), the accepted v1 shape (base `kv` feature pulls rusqlite, +default-on) stands until then, and the deferral is a feature-graph +work item, not an open architecture decision. No spec or ADR gates on +it. OQ-09 was `deferred(scope)` on the first embedded ops deployment — a deciding fact that could only exist once the ops module ships, i.e. a diff --git a/docs/architecture/ops-surface.md b/docs/architecture/ops-surface.md index b96660f..bb55737 100644 --- a/docs/architecture/ops-surface.md +++ b/docs/architecture/ops-surface.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (pin-token renewal contract pinned per ADR-010 §4) +last_updated: 2026-10-03 (ADR-012 §2 — put response {token, digest}; have token-renewal form; fetch verification wording; non-fleet token scope) --- # Ops surface (alkcall-backed network operations) @@ -55,11 +55,19 @@ JSON control ops (`Visibility` per ADR-001 §Decision): - **`blobs/stat`** `{namespace, digest}` → `{len}` — probe before offering; ACL-gated like fetch (read action on the namespace) -- **`blobs/have`** `{namespace, digests[]}` → `{present[]}` — the - have half of have/need set diffing (hashes only — content never +- **`blobs/have`** `{namespace, digests[], token?}` → `{present[]}` — + the have half of have/need set diffing (hashes only — content never traverses this op); read-gated: an existence probe over arbitrary digests is a discovery surface, so it is gated exactly like fetch, - not public + not public. `present[]` corresponds to `digests[]` by index. The + optional `token` is the **renewal form** (ADR-012 §2): when present, + exactly one digest accompanies it — the handler verifies the token + maps to that digest's pin row, renews it (crate-internal + `pin_renew` via the ops module's store-core access, ADR-010 §4), + and answers with `present[]` as usual; an expired/foreign row + answers `present: false` (the putter's backstop is re-put). Without + `token`, `have` remains a pure existence probe (ADR-008's + discipline). - **`blobs/delete`** `{namespace, digests[]}` — operator machinery; `Visibility::Internal`, evaluated under the internal authority context per alkcall ADR-017 (internal calls switch authority context, @@ -72,8 +80,12 @@ Binary channel ops (registered via the `channel_open` marker): - **`blobs/fetch`** (`Sub` — the server→client streaming op shape) — `{namespace, digest, ranges?}` in; - verified bytes out: the consumer hash-checks each received chunk/whole against - the carried digest (ADR-006). Broadcast fanout above the store: + verified bytes out: the consumer hash-checks the received whole + against the carried digest (ADR-006's whole-blob verification — + there are no per-chunk or per-range checks derivable under the + canonical digest; ranged fetch responses carry the same ADR-006 + slice-digest convention `read_range` returns). Broadcast fanout + above the store: one reader, store arm + subscriber arms (POC #3 finding A3); late joiners degrade to a normal post-commit `get` — identical bytes under CAS. Slow-subscriber policy (drop-and-late-join) is ops-layer @@ -81,7 +93,10 @@ Binary channel ops (registered via the `channel_open` marker): - **`blobs/put`** (`Sink` — the client→server streaming op shape) — `{namespace, digest?, len?}` offer + byte stream in; server verifies against the canonical derivation - before commit. A known-length offer is the encouraged path (one- + before commit. **Response: `{token, digest}`** (ADR-012 §2) — the + pin token (§Pin hand-over below) and the stored entry's canonical + digest (hex), which an unknown-length putter needs for every later + op. A known-length offer is the encouraged path (one- pass); unknown-length rides the store's pre-threshold buffering path (ADR-003). **Need half of have/need**: a fetch miss *is* the need announcement — the consumer computes its need set by diffing @@ -96,7 +111,8 @@ the server. The hand-over contract: 1. the put lands pinned (ADR-005) — the entry cannot be swept while the handler holds the pin; -2. the `blobs/put` response returns a **pin token** (opaque handle — +2. the `blobs/put` response returns a **pin token** in the specced + response shape `{token, digest}` (ADR-012 §2 — opaque handle minted by the ops layer over the pin row the put committed; the token↔(key, owner) mapping is ops-layer state, not durable state — the durable state is the pin row itself, ADR-010 §4); the putter @@ -115,31 +131,40 @@ the server. The hand-over contract: conversion on the server side (token → registered liveness source) is the mechanism the embedder plugs its registry into. -**Token renewal (ADR-010 §4 — the fleet-TTL race, resolved): the -token *is* the pin — there is no second pin state.** The server-side -handler's in-process `Pin` guard drops when the `blobs/put` response -completes (if it never dropped, the TTL machinery would be dead on -the ops path — the pin row's expiry, not the guard's presence, is the -fleet liveness signal). Under fleet +**Token renewal (ADR-010 §4, wired per ADR-012 §2 — the fleet-TTL +race, resolved): the token *is* the pin — there is no second pin +state.** The server-side handler's in-process `Pin` guard drops when +the `blobs/put` response completes (if it never dropped, the TTL +machinery would be dead on the ops path — the pin row's expiry, not +the guard's presence, is the fleet liveness signal). Under fleet engines the put's pin is an expiry-carrying row (ADR-008: TTL 60 s default, renewed every TTL/3). The remote putter holding a token is -the row's owner and renews by the ops it already speaks: a -`blobs/have` probe **presenting the token alongside the digest** (the -handler verifies the token maps to the digest's pin row and calls the -crate-internal `pin_renew` via the ops module's store-core access — -the ops module is in-crate and crosses the seam by construction, the -same way the put handler holds the `Pin` guard; `has` itself remains +the row's owner and renews via the renewal form of `blobs/have` +(ADR-012 §2: one digest, token alongside — the handler verifies the +token maps to the digest's pin row and calls the crate-internal +`pin_renew` via the ops module's store-core access — the ops module +is in-crate and crosses the seam by construction, the same way the +put handler holds the `Pin` guard; `has` itself remains a pure existence probe per ADR-008 — renewal is a token-authenticated act on the pin row, not a `has` semantic), or a re-put of the same digest (CAS dedup renews the row). **A putter may pause past the TTL/3 mark (20 s at the default TTL) only if it renews before expiry; -a putter pausing past TTL lets the row expire** — the sweeper reaps -it (ADR-008) and delete-then-recover (ADR-005) is the backstop: the +a putter pausing past TTL lets the row expire** — the renewal probe +answers `present: false`, the sweeper reaps the row (ADR-008), and +delete-then-recover (ADR-005) is the backstop: the putter re-offers the digest; CAS makes the re-put idempotent. The -race resolves to a re-put, never to silent loss. For non-fleet -(`local`-engine) serving nodes the token is the in-process guard — -nothing expires; the process-crash case is ADR-005's in-flight-put -loss with the same backstop. +race resolves to a re-put, never to silent loss. **Token validity +domain: the minting serving node** (ADR-012 §2) — the token↔(key, +owner) mapping is ops-layer in-memory state in the node that minted +it (process-lifetime, outliving the handler); renewal probes must +reach that node, fleet peers refuse tokens they did not mint (`present: +false`, not an error), and a minting node's death before renewal is +ADR-005's in-flight-put loss case with the same re-put backstop — a +documented embedder duty (route renewals to the serving node). +For non-fleet serving nodes (any engine — sqlite, single-instance +postgres, `local`, pg-lo; ADR-012 §2) the token is the in-process +guard's handle — nothing expires; the process-crash case is ADR-005's +in-flight-put loss with the same backstop. Placement of the op *registration* (which embedders wire where they want them exposed) matches the alkgit ops pattern (its @@ -186,6 +211,7 @@ fleet-specific op exists. | [006](decisions/006-verification-posture-and-transfer-encoding.md) | Verification | verified-fetch via carried digests; no chunk trees | | [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | Fleet re-routing topology | non-shared-media fleets route large-blob access through a storage node via these ops; no new ops | | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | Pin-token renewal | the token is the pin; renewal rides have/re-put; TTL expiry falls to delete-then-recover | +| [012](decisions/012-pre-decomposition-consistency-rulings.md) | Wire-shape rulings | put response `{token, digest}`; have's token-renewal form; token validity domain = minting node; non-fleet token scope generalized | | [011](decisions/011-vocabulary-tiers-engines-and-fleet-state.md) | Vocabulary + modes | instance/node/fleet split; kv-only mode named (the re-routing posture's client shape) | ## Open Questions @@ -204,4 +230,5 @@ deployment). machinery this module rides) - `docs/research/poc-largeblob-findings.md` finding A3 (fanout seam) - ADR-001; [store-api.md](store-api.md); [gc-and-namespaces.md](gc-and-namespaces.md) - (namespace-as-resource) \ No newline at end of file + (namespace-as-resource); ADR-012 §2 (the wire shapes this doc's + op payloads carry) \ No newline at end of file diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 2496c16..d0190e5 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-010/011 — I/O seams pinned; vocabulary fs→large, mem joins kv) +last_updated: 2026-10-03 (ADR-012 — OQ-11 parking reference; ADR-012 row added) --- # Overview @@ -65,7 +65,8 @@ or three times. │ hashing & keys (ADR-002) · put/get/stat/range/pin seams │ │ mark-and-sweep GC + liveness registration (ADR-005) │ │ size-threshold dispatch (ADR-003); GC state owned │ -│ here, hosted by SQL engines via the seam (ADR-010) │ +│ here; fleet state hosts on the fleet's kv engine │ +│ via the seam (ADR-010/012) │ └───────┬───────────────────┬───────────────────────────────┘ ▼ ▼ ┌──────────────────────┐ ┌─────────────────────────────────┐ @@ -105,7 +106,8 @@ optimizes for — feature-graph work for it is deferred until a named requirement exists (the base `kv` feature pulling rusqlite is the accepted v1 shape; convention 8's lean-base principle applies to *default* builds, which stay lean since `kv` is what the feature model -controls, not what default-on means downstream). Wasm: the tier +controls, not what default-on means downstream; tracked as OQ-11 in +open-questions.md's parked index). Wasm: the tier engines are not a wasm story; a wasm client is a consumer of the `ops` surface (which rides alkcall, itself wasm-clean), never an in-process embedder (ADR-001 §Consequences). @@ -139,6 +141,7 @@ embedder (ADR-001 §Consequences). | [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | `size` probe + fleet GC + large-tier engines | trait length probe; fleet pins/sweeps engine-backed; the large tier (now `large`) engine-selectable | | [009](decisions/009-pg-lo-large-tier-engine.md) | pg-lo large engine | postgres Large Objects as the large tier's second engine, admitted on POC #7; a fleet node can run postgres-only | | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | I/O seams + GC-state home | read cursor / staged put; store-core-owned GC state via the engine-state seam; kv fleet-validity rule; pin-token renewal | +| [012](decisions/012-pre-decomposition-consistency-rulings.md) | Sweep-abort shape + composition rulings | sweeps never error (`GcAborted` retired, `GcRefuse` for direct-delete); token wire shape; fleet mode = constructor declaration; GC state hosts on the kv engine; sqlite seam reduction | | [011](decisions/011-vocabulary-tiers-engines-and-fleet-state.md) | Vocabulary + rename | tier/engine/instance/node/fleet pinned; mem joins the kv tier; large tier renamed `large`; constructor modes | ## Open Questions @@ -154,6 +157,8 @@ crate level: open-read requirement) - **OQ-10**: second kv engine — resolved by ADR-007 (sqlite + postgres shipped behind one trait; per-node constructor choice) +- **OQ-11**: pg-only kv node feature graph — deferred(scope), parked + (see [open-questions.md](open-questions.md)) - **POC #7** (`docs/research/poc-pglo-findings.md`): pg Large Objects as the large tier's `pg-lo` engine — **passed** 2026-10-03; admitted as a shipped engine by ADR-009 (REQ-2's consolidation option) diff --git a/docs/architecture/store-api.md b/docs/architecture/store-api.md index 418a8cf..19db30c 100644 --- a/docs/architecture/store-api.md +++ b/docs/architecture/store-api.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 (ADR-010 I/O seams + ADR-011 rename — large tier → large) +last_updated: 2026-10-03 (ADR-012 — sweep-abort = Ok(report) with GcAbortCause; GcAborted retired; fall-through for all key-addressed ops; put-time kv-only rejection; delete/missing/error-model ruling) --- # Store API @@ -61,7 +61,11 @@ mechanism lives in the engine-state seam, ADR-010 §2). read cursor the store core drives (ADR-010 §1 — engines construct their own cursor bodies; the facade's shape is uniform). Dispatch fall-through: a small-tier miss queries the large tier (POC #1 - finding 6). + finding 6 — the general rule, ADR-012 §6.1: every key-addressed op + — `get`, `stat`, `has`, `delete`, `read_range` — consults the kv + tier first, then the large tier). A miss on both tiers is a `None` + (get/stat) or `Err(Missing)` (read_range/delete) — see the error + model. - **`stat(key) → Option`** — cheap length/type probe (phase-0 OQ-BL-04 addendum; gix's header-only read is git's cheapest primitive and the packfile-serving surface rides this). `EntryMeta` @@ -86,36 +90,56 @@ mechanism lives in the engine-state seam, ADR-010 §2). ### Lifecycle -- **`has(key)`, `delete(key)`** — deletion participates in ADR-005's +- **`has(key) → bool`** — the pure existence probe (ADR-008's + `has` discipline; a `size`-probe under the dispatch general rule — + absent on both tiers is `false`; virgin-store reads are no-ops). +- **`delete(key) → Result<(), Error>`** — falls through tiers per the + dispatch general rule above. Participates in ADR-005's delete windows; direct deletes are permitted but refuse protected - keys — pinned or re-observed live via registered sources (the same - per-key arbitration sweeps apply, typed error) — and are unusual by - posture; most deletion flows through sweeps. + keys — pinned, re-observed live via registered sources, or named by + the protect callback (ADR-012 §6.4: the full protection set, not + just pins — the ADR-005 invariant applied to the direct path) — + with the typed `GcRefuse` error. Direct deletes are unusual by + posture; most deletion flows through sweeps. An embedder's + correction flow deregisters/re-registers around the delete. - **`list() → stream of keys`** — whole-pool enumeration; complete by contract (see backends doc for why list correctness is load-bearing). - **`register_liveness_source(...)`** — the live-shared GC seam (ADR-005, §Decision; the POC's "install implies copy" failure - (finding 7) is why the verb/name is pinned here). -- **`sweep() → SweepReport`** — explicit mark-and-sweep; embedders own - the cadence (ADR-005, no ambient timers). A sweep with no registered - liveness sources aborts without deleting — the safe default. - `SweepReport`'s shape (specced here, resolving its earlier - name-only status): `candidates_staged`, `deleted`, - `cancelled_by_arbitration` (u64 counts), `aborted: Option` - (the typed cause — protection-source failure (ADR-005) or fleet - sweeper-lock contention (ADR-008) — present iff the sweep deleted - nothing for an abort reason; named `GcAbortCause` to keep it a data - enum, distinct from the `GcAborted` error which carries the - direct-delete refusal path), and `started_at`/`finished_at`. The - report is data, not an error carrier: an aborted sweep returns - `Ok(report)` with `aborted: Some(..)` — the typed error - `GcAborted` is reserved for the direct-delete refusal path. + (finding 7) is why the verb/name is pinned here). Fleet form (ADR-012 + §6.6): the same verb registers a **named liveness table reference** + in the shared engine (the embedder's own table per ADR-008 §2 — the + store core reads it, never owns it); in-process callback sources + remain the single-node form. +- **`sweep() → Result`** — explicit mark-and-sweep; embedders own + the cadence (ADR-005, no ambient timers). **Sweeps are never + errors: an aborted sweep returns `Ok(SweepReport)` with + `aborted: Some(GcAbortCause)` — there is no sweep-abort error + variant** (ADR-012 §1; the previously specced `GcAborted` sweep + error is retired — see the error model). A sweep with no registered + liveness sources aborts without deleting — the safe default — + carrying `GcAbortCause::NoLivenessSources`. + `SweepReport`'s shape: `candidates_staged`, `deleted`, + `cancelled_by_arbitration` (u64 counts), `aborted: + Option`, and `started_at`/`finished_at`. `GcAbortCause` + is a data enum with exactly three variants — `ProtectFailed` + (protection-source failure, ADR-005), `SweeperLock` (fleet + sweeper-lock contention, ADR-008), `NoLivenessSources` (the safe + default above); callers match non-exhaustively; new causes are new + ADRs (ADR-012 §1). The direct-delete refusal path is a *different* + type: the `GcRefuse` error (above), not a sweep outcome. + Staged-window candidates overdue at sweep time are re-arbitrated + and executed by that sweep (the executor is the next sweep — + ADR-012 §6.7; non-SQL engines execute their window inline within + the same sweep, ADR-012 §5). Under a fleet (multiple store instances over one pool — requirements.md REQ-2), sweep/put/delete semantics are mechanism-extended by ADR-008: pins are DB-backed rows committed atomically with entries (the store core holds the joint tx, - ADR-010 §2), one sweeper holds the advisory lock, and delete + ADR-010 §2; all GC state hosts on the fleet's kv engine — + ADR-012 §4), one sweeper holds the advisory lock (contention is a + report abort, not an error — ADR-012 §1), and delete arbitration is SQL-level — the facade's invariant text is unchanged. @@ -124,14 +148,30 @@ mechanism lives in the engine-state seam, ADR-010 §2). `thiserror`; no panics in library code; no `unwrap`/`expect` outside tests (AGENTS convention 2). Distinguished failure families: -- `Missing` — key absent (get/stat miss) +- `Missing` — key absent on ops that signal absence via `Err`: + `read_range` and `delete` (ADR-012 §6.5). `get` and `stat` signal + absence with `None` — `Missing` is not theirs. - `Verification` — put/get hash-check failure (content ≠ key) -- `Io(String)` — backend media failure (stringly because - `std::io::Error` is not stable across versions) -- `GcAborted` — nothing deleted; sweep not run. Two cause classes - (ADR-005: a protection source failed; ADR-008: another node holds - the fleet sweeper lock) — the typed error distinguishes them. -- `KeyInvalid` — malformed key bytes at the boundary (hashing doc) +- `Io(String)` — backend media failure, covering cursor and staged-put + I/O errors surface through the same trait error family (stringly + because `std::io::Error` is not stable across versions) +- `GcRefuse` — direct delete refused: the key is protected (pinned, + registered-source-live, or protect-callback-named) — nothing + deleted; the delete-window arbitration decides at delete time + (ADR-005; named `GcRefuse` per ADR-012 §1 — there is no + sweep-abort error). +- `KeyInvalid` — malformed key bytes at the boundary (truncated, + unknown algorithm — rejects, never guesses; ADR-002/hashing doc). + One enum, both altitudes: the **facade's** error enum exposes + `Missing`/`Verification`/`Io`/`GcRefuse`/`KeyInvalid`; the + `Backend` trait's methods return the same family as crate-internal + variants — `KeyInvalid` never fires at the trait boundary (keys + arrive opaque there) and `GcRefuse` never fires below the facade + (arbitration answers flow up as data; the facade decides and owns + the refusal), so the trait's effective members are + `Missing`/`Verification`/`Io`/`KeyInvalid`; the facade re-exposes + them with `GcRefuse` added (ADR-012 §6.5's family statement reads + through this split). Virgin-store semantics: read paths on a fresh store see absent/empty, never "table does not exist" errors (POC #1 finding 2 — the redb @@ -190,6 +230,7 @@ lesson generalizes to any kv engine). | [006](decisions/006-verification-posture-and-transfer-encoding.md) | Verification | whole-blob checks; slice digests out-of-band | | [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | Trait `size` + fleet GC | stat/accounting ride the trait's `size` probe; fleet pins/sweeps are engine-backed | | [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | I/O seams + GC-state home | read cursor / staged put behind the facade; the store core holds the joint entry+pin tx | +| [012](decisions/012-pre-decomposition-consistency-rulings.md) | Sweep-abort shape + wire/activation rulings | swept = Ok(report) always, `GcRefuse` for direct-delete refusal; fleet mode is a constructor declaration; GC state hosts on the kv engine | ## Open Questions @@ -201,7 +242,8 @@ None owned by this document beyond the cross-references above. benchmark table - `docs/research/poc-trait-dispatch-findings.md` findings 1–3, 7 - ADR-003 (put-path buffering), ADR-005 (pin/Batch/Pin lifecycle), - ADR-006 (range-read semantics), ADR-002 (keys) + ADR-006 (range-read semantics), ADR-002 (keys), ADR-012 (the + sweep-abort shape, error-model ruling, fall-through general rule) - [ops-surface.md](ops-surface.md) — the fanout/fetch consumer of the read path - [gc-and-namespaces.md](gc-and-namespaces.md) — the lifecycle seam