diff --git a/docs/plans/implementation.md b/docs/plans/implementation.md index a49b294..bb0dc72 100644 --- a/docs/plans/implementation.md +++ b/docs/plans/implementation.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: "2026-10-11 (wave 2 decomposed into tasks/)" +last_updated: "2026-10-11 (wave 3 decomposed into tasks/)" --- # alkblobs — Implementation plan @@ -50,7 +50,7 @@ ops payload surfaces; release readiness is last. |---|---|---|---| | [1](#wave-1--foundations) | Crate/feature scaffold; hashing & keys (the one-way door); error model; Backend trait + I/O seams + engine-state seam shape; facade value types; conformance-suite convention | — | decomposed (2026-10-11) | | [2](#wave-2--the-store-works-dependency-free-mem--local) | mem + local engines; facade put/read paths + dispatch + constructor modes; in-process GC protocol; suite's first columns | wave 1 | decomposed (2026-10-11) | -| [3](#wave-3--sqlite-engine) | sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column | wave 2 | pending | +| [3](#wave-3--sqlite-engine) | sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column | wave 2 | decomposed (2026-10-11) | | [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | pending | | [5](#wave-5--pg-lo-large-engine) | pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column | wave 4 | pending | | [6](#wave-6--fleet-gc) | Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations | waves 4 + 5 | pending | diff --git a/tasks/engine-sqlite-window.md b/tasks/engine-sqlite-window.md new file mode 100644 index 0000000..9b23c8a --- /dev/null +++ b/tasks/engine-sqlite-window.md @@ -0,0 +1,88 @@ +--- +id: engine-sqlite-window +name: sqlite engine-state window family — the durable single-node delete window +status: pending +depends_on: [engine-sqlite] +scope: narrow +risk: low +impact: component +level: implementation +tags: [wave-3, engines, sqlite, gc] +--- + +## Description + +Implement the engine-state companion seam's **window family** on the +sqlite backend (`src/sqlite.rs` or its seam module): the +`EngineWindowState` impl per ADR-010 §2 **as reduced by ADR-012 §5** — +the window family only. The pin clauses and the sweep-lock body are +**dropped dead machinery** for sqlite (ADR-012 §5's table: pin clauses +**no**, sweep-lock clause **no** — the single-flighter is the +in-process lock; a sqlite fleet is a recorded non-option, ADR-010 §3). +Nothing of `EngineFleetState` may exist on this engine. + +- **`ensure_gc_schema`** — creates **`gc_window` only** + (`gc_window (key BLOB PRIMARY KEY, visible_after INTEGER)`; the + timestamp encoding — `SystemTime` ↔ integer, sub-second precision + for the test tunings — is a recorded design point). Never `gc_pins`; + the embedder's liveness table is never created here (ADR-008's + schema-ownership rule — the sweep reads it and never owns it). + Called at store construction by the store core + (`store-sqlite-construction` wires the call); the clause body is + engine-side. +- **`window_stage`** — stage candidates into the durable window, + **idempotent per `(key, deadline)`** (`INSERT OR IGNORE` semantics — + the body's discipline; the store stages each candidate once per + sweep, and a re-stage of the same key at a new deadline updates the + row). Virgin store: tolerated (the table exists from construction; + a no-tables-yet database stages lazily or errors nothing — the + Backend trait's engine posture carries here). +- **`window_take(now)`** — take overdue candidates + (`visible_after <= now`): the rows **leave the window table and + return as keys** for the calling sweep to re-arbitrate and delete — + the executor is the next sweep (ADR-012 §6.7). Empty when nothing is + overdue; virgin store: empty. The take is one atomic step (selected + rows removed and returned together — a row is never handed out + twice). +- **Contract tests** per the applicability row (ADR-012 §5's table is + the contract-test matrix): the schema creates the window table only + (assert `gc_pins` does not exist); stage idempotence per + `(key, deadline)` incl. the re-stage-at-new-deadline form; take + returns exactly the overdue set, removes it, leaves future rows; + empty/virgin shapes; the timestamp encoding's boundary + (`visible_after == now` is overdue — `<=` per the seam doc). + +The store core drives these clauses (`store-sweep-durable-window`); +this task ships the clauses and their engine-level contract tests. + +## Acceptance Criteria + +- [ ] The sqlite backend implements `EngineWindowState` — the window + family only; no pin-clause or sweep-lock body exists anywhere in + the sqlite engine +- [ ] `ensure_gc_schema` creates `gc_window` only; `gc_pins` is never + created by sqlite (contract-tested) +- [ ] `window_stage` is idempotent per `(key, deadline)`; a re-stage + at a new deadline updates the row +- [ ] `window_take` returns exactly the overdue set (`<= now` + boundary), removes it atomically, leaves future rows; empty on + none/virgin +- [ ] Clause contract tests green; `cargo test` (default) green; + `cargo clippy --all-targets -- -D warnings` and + `cargo fmt --check` green + +## References + +- src/gc.rs (`EngineWindowState` — the seam contract and its clause docs) +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state.md §2 +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §5 (the applicability table; the sqlite reduction), §6.7 (the executor) +- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md §2 (the durable window's semantics) +- docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md (the delete-window protocol) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/engine-sqlite.md b/tasks/engine-sqlite.md new file mode 100644 index 0000000..3ef125f --- /dev/null +++ b/tasks/engine-sqlite.md @@ -0,0 +1,118 @@ +--- +id: engine-sqlite +name: sqlite kv engine — the default engine +status: pending +depends_on: [] +scope: narrow +risk: medium +impact: component +level: implementation +tags: [wave-3, engines, sqlite] +--- + +## Description + +Implement the `sqlite` kv-tier engine (feature `sqlite`, **default-on**; +ADR-007; REQ-1) against the wave-1 trait (`src/backend.rs`) — the first +SQL-backed engine and the default build's kv engine. + +- **Backend impl** (`src/sqlite.rs`, behind `#[cfg(feature = "sqlite")]`): + the seven methods plus `put_staged`, opaque byte keys/values, + `name() -> "sqlite"`. + - **Schema** — the POC #5 finding B5 row shape adapted to sqlite: + `blobs (key BLOB PRIMARY KEY, value BLOB, size INTEGER NOT NULL)`. + The dedicated `size` column is what makes the **indexed `size` + column probe** metadata-only: `SELECT size … WHERE key = ?` rides + the key's index and never loads the value blob (ADR-008's size + discipline). + - **WAL posture** — `journal_mode=WAL` + `synchronous=NORMAL` set at + open: the shipped durability tier (ADR-003; the posture honker + ships and iroh's store measured). The **single-writer ceiling is a + documented posture, not a defect** (REQ-1's economics — the module + doc records it; ~1.2k objects/s under contention is an order above + single-node push rates). + - **Prepared statements** — rusqlite's `prepare_cached` discipline + per the backends spec's "prepared statements" requirement. + - `has` — an **EXISTS probe** (`SELECT EXISTS(…)`) — never a value + load. + - `get` — the **materializing cursor body** (kv family): the whole + value loaded once (bounded by tier policy — the kv tier holds only + ≤ threshold content), `len()` from the row's size column, reads + slice the materialized value composing `backend.rs`'s `clamped`. + This body is also what the facade's verification-on-get re-derives + over (store.rs's kv design point — no engine change needed). + - `put` — CAS first-writer-wins (`INSERT OR IGNORE`); a re-put of an + existing key is a no-op, never an overwrite. + - `put_staged` — the **degenerate in-memory stage** (ADR-010 §1: kv + bodies stage in memory and commit through the whole-value row + write); `append` extends the buffer, `commit` publishes via the + same `INSERT OR IGNORE` clause and self-discards on failure, + `discard` drops the buffer with zero residue. + - `delete` — `Err(Error::Missing)` when absent (rows-affected + discipline); `list` — complete and well-formed (`SELECT key`); + `size` — the indexed column probe above. + - **Virgin-store no-tables-yet tolerance** — read paths treat a + missing table as absent/empty, never "table does not exist" errors + (POC #1 finding 2; store-api.md §Error model's virgin-store + semantics). Schema creation timing (constructor `CREATE TABLE IF + NOT EXISTS` vs lazy) is a recorded design point; the tolerance is + unit-tested against a tableless database file either way. +- **Blocking discipline**: rusqlite is synchronous — **all engine I/O + runs in `spawn_blocking`** (AGENTS convention 3; store-api.md + §Concurrency posture). The connection posture (the expected shape: a + single connection behind a short-held lock, the single-writer + posture; a pool is not required solo) is a recorded design point. + No blocking I/O on the async path; no `unwrap`/`expect` outside + tests. +- **Unit tests** in-module: the method contract (virgin-store no-ops + incl. the no-tables-yet form, CAS re-put, Missing discipline, size + exactness + metadata-only), the staged-put happy path and both + hygiene forms (discard → zero residue; commit failure → + self-discard — a `#[cfg(test)]` fail-commit knob standing in for + media failure, the mem engine's pattern), the pragma posture. + +The engine-state window family (`EngineWindowState`) is +`engine-sqlite-window` (separate task); the constructor wiring is +`store-sqlite-construction`. This task ships the engine and its +in-module unit tests only. + +## Acceptance Criteria + +- [ ] `src/sqlite.rs` implements `Backend` completely (seven methods + + `put_staged`) behind feature `sqlite` (default-on), with the + materializing cursor body and the degenerate staging body +- [ ] WAL + `synchronous=NORMAL` set at open; prepared-statement + discipline; `has` is an EXISTS probe; the single-writer ceiling + recorded as posture in the module doc +- [ ] `size` reads the dedicated size column via the key index, never + the value blob (metadata-only, unit-tested) +- [ ] CAS first-writer-wins on both put forms; re-put never overwrites +- [ ] Stage hygiene holds in both forms (discard → zero residue; + failed commit → self-discard), unit-tested +- [ ] Virgin-store reads are no-ops including the no-tables-yet form + (unit-tested against a tableless database file) +- [ ] All engine I/O in `spawn_blocking`; connection posture recorded +- [ ] `cargo test` (default — sqlite is default-on) green; + `cargo clippy --all-targets -- -D warnings` and + `cargo fmt --check` green + +## References + +- docs/architecture/backends-and-dispatch.md (§kv tier — sqlite engine) +- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md +- docs/architecture/decisions/003-backend-contract-and-dispatch.md (the + shipped durability tier; the three numbered put rules' kv side) +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state.md §1 +- docs/architecture/requirements.md (REQ-1's economics) +- docs/research/poc-postgres-kv-findings.md (finding B5's row shape) +- docs/research/poc-largeblob-findings.md (finding A4's economics) +- src/backend.rs (the trait + seam contracts this implements) +- src/mem.rs (the contract-reference engine's shapes this mirrors) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/review-wave-3.md b/tasks/review-wave-3.md new file mode 100644 index 0000000..c93ed6f --- /dev/null +++ b/tasks/review-wave-3.md @@ -0,0 +1,96 @@ +--- +id: review-wave-3 +name: Wave 3 review gate — the sqlite engine vs the contract +status: pending +depends_on: [suite-column-sqlite] +scope: narrow +risk: low +impact: phase +level: review +tags: [wave-3, review] +--- + +## Description + +The wave-3 review gate — the engine-wave review shape +(docs/plans/implementation.md §Review gates: "engine-vs-contract +conformance"), before wave 4 decomposes. The gate reads the +implementation against the pinned ADR text line by line (the wave-1/2 +gates' method). + +Gate checks: + +- **Engine-vs-contract conformance**: the suite column green + server-less; the engine-state window family contract-tested per its + applicability row (ADR-012 §5's table — window family yes, pin + clauses no, sweep-lock body no); virgin-store semantics including + the no-tables-yet form; the `size` metadata-only discipline (the + indexed column probe never loads the value blob). +- **The sqlite posture vs ADR-003/007**: WAL + `synchronous=NORMAL` + as the shipped durability tier; the single-writer ceiling documented + as posture, not hedged (REQ-1's economics); prepared-statement + discipline; `has` as an EXISTS probe; the B5 row shape (key/value/ + size); bounded reads. +- **The durable-window sweep vs ADR-008 §2 / ADR-012 §5/§6.7**: + candidates staged at `visible_after = mark_end + window` (the + constructor-tunable delay, shipped default 5 min); the executor is + the next sweep; execution-time re-arbitration; report counts exact + relative to the schedule; the inline form preserved verbatim for + non-SQL engines; the single-node posture unchanged (pins/liveness/ + arbitration lock in-process — only the window's persistence is + engine rows); the one-window-host posture recorded. +- **The seam reduction honored**: no pin rows, no sweep-lock body + anywhere in the sqlite engine (dead machinery per ADR-012 §5); + `ensure_gc_schema` creates `gc_window` only; the embedder's + liveness table never created by the engine. +- **Blocking discipline**: all rusqlite work off the async path + (`spawn_blocking`); no blocking I/O under internal locks; the + connection posture recorded. +- **Named design points audited as resolved** (each recorded in its + task's Notes): the connection posture; the timestamp encoding; + schema-creation timing vs the no-tables-yet tolerance; the kv cursor + dispatch shape over multiple kv engines; the protect-consult posture + on overdue execution; the delay-0 test tuning's legality. +- **Family conventions**: no comments in code (doc comments on public + API fine), thiserror, tokio-only async, no unwrap/expect outside + tests, naming, one-module-per-file re-export shape, the + `#![allow(dead_code)]` scope notes matching their remaining scope. +- **Gates green**: `cargo test` (default — the sqlite column runs + here), `cargo test --features mem`, `cargo test --all-features`, + `cargo clippy --all-targets -- -D warnings`, + `cargo clippy --all-features --all-targets -- -D warnings`, + `cargo fmt --check`. + +Findings are fixed inline where cheap, or recorded as wave-4 +amendments where they shape the next decomposition. The gate's +completion is what unblocks wave 4's decomposition. + +## Acceptance Criteria + +- [ ] Every gate check above read and verified against the cited ADR + text (not from memory of it) +- [ ] Named design points each decided, recorded, and consistent with + the specs +- [ ] Findings fixed inline or recorded as wave-4 amendments with + owners +- [ ] All gates re-run green after any fixes +- [ ] Wave 4 decomposition unblocked (recorded in this task's Summary) + +## References + +- docs/plans/implementation.md (§Wave 3, §Review gates — engine wave reviews) +- docs/architecture/decisions/003-backend-contract-and-dispatch.md +- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md +- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md §2 +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state.md §2 +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §1/§5/§6.7 +- docs/architecture/store-api.md +- tasks/review-wave-2.md (the gate method) + +## Notes + +> To be filled by the review agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/store-sqlite-construction.md b/tasks/store-sqlite-construction.md new file mode 100644 index 0000000..45458c6 --- /dev/null +++ b/tasks/store-sqlite-construction.md @@ -0,0 +1,95 @@ +--- +id: store-sqlite-construction +name: sqlite constructor wiring — the default-build store shapes +status: pending +depends_on: [engine-sqlite, engine-sqlite-window] +scope: narrow +risk: low +impact: component +level: implementation +tags: [wave-3, store, sqlite, construction] +--- + +## Description + +Slot sqlite into the facade's engine representation — the slotting-in +pattern `src/store.rs`'s module doc pinned ("adding an enum variant and +a named constructor is slotting in"): + +- **`KvEngine::Sqlite`** variant behind `#[cfg(feature = "sqlite")]` + + the dispatch arms (`put`/`get`/`has`/`size`/`delete`/`list`). The kv + cursor dispatch (`AnyCursor`'s kv arm) carries the sqlite + materializing body — the kv-arm shape over multiple kv engines (a + `KvCursor` sub-enum vs per-engine variants) is a recorded design + point. The `#[allow(unused_variables)]` empty-set scaffolding on the + `KvEngine` methods shrinks (the kv set is no longer empty in default + builds); `backend.rs`'s `#![allow(dead_code)]` scope note about "the + kv engines' whole-value `put` before wave 3" is updated to the + landed state. +- **Named constructors**, feature-gated on `sqlite` (default-on — the + default build constructs them): + - **dual-tier** `kv = sqlite + large = local` — the default-build + production shape; fallible (file I/O: opening/creating the + database, creating the local root). Size-threshold dispatch at the + constructor-tunable threshold as in the mem+local shape. + - **kv-only(sqlite)** — no large tier; `threshold` is the put-time + rejection point (ADR-012 §6.2; the rejection machinery is the put + seam's, already landed — only the constructor slots in). + - Both take the **delete-window delay** parameter (ADR-008 §2: the + delete window's delay, **default 5 min, constructor-tunable per + deployment** — a named shipped-default constant; `Duration`). + This is the durable window's cadence knob the sweep's SQL-backed + form consumes (`store-sweep-durable-window`); `0` is a legal + degenerate tuning (execute-on-next-sweep — the test tuning). The + mem/local constructors are unchanged: the inline form has no delay + in effect. +- **`ensure_gc_schema` at construction** — the store-core call + (engine-neutral; sqlite's body creates `gc_window` only). A + construction failure is `Err(Error::Io)` per the fallible-constructor + pattern `dual_tier_mem_local` set. +- **The one-window-host rule's store-core posture** (ADR-012 §5): the + durable window hosts on **the kv engine when it is SQL-backed** — + sqlite hosts. Recorded (code shape or doc note) so wave 5's pg-lo + dormancy (the dual-SQL instance stages on the kv engine only) slots + in without reshaping. +- **Constructor unit tests**: the modes construct and introspect + correctly (`mode()`/`threshold()`), `ensure_gc_schema` ran (the + window table exists in the fresh database), the window-delay default + is the pinned constant, kv-only rejection wiring unchanged, the + dual-tier shape round-trips end-to-end (a smoke put/get through both + tiers). + +## Acceptance Criteria + +- [ ] `KvEngine::Sqlite` slots in; all six kv ops dispatch; the kv + cursor body dispatches (design point recorded) +- [ ] `dual_tier_sqlite_local` + `kv_only_sqlite` constructors exist + behind feature `sqlite`, take threshold + window delay, fallible + where I/O is fallible +- [ ] `ensure_gc_schema` runs at construction; the window-delay + shipped default is 5 min (pinned constant) and + constructor-tunable; `0` legal +- [ ] The one-window-host posture recorded (sqlite hosts when the kv + engine is SQL-backed) +- [ ] The stale empty-set scaffolding/allow notes updated to the + landed state +- [ ] `cargo test` (default) green — the default build constructs and + exercises sqlite stores; `cargo clippy --all-targets -- -D + warnings` and `cargo fmt --check` green + +## References + +- src/store.rs (the engine-representation design point; the + constructor modes; the slotting-in pattern) +- docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md §4 (modes) +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §6.3 (dual-tier shapes), §5 (the one-window-host rule) +- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md §2 (the window delay, default 5 min) +- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §4 (the feature graph) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/store-sweep-durable-window.md b/tasks/store-sweep-durable-window.md new file mode 100644 index 0000000..dc3cc73 --- /dev/null +++ b/tasks/store-sweep-durable-window.md @@ -0,0 +1,99 @@ +--- +id: store-sweep-durable-window +name: the sweep's durable-window form — staged candidates, next-sweep execution +status: pending +depends_on: [store-sqlite-construction] +scope: moderate +risk: medium +impact: phase +level: implementation +tags: [wave-3, store, gc, sweep] +--- + +## Description + +Grow the sweep's **SQL-backed form** (ADR-008 §2; ADR-012 §5/§6.7) +beside the inline form in `src/store.rs` — the first durable delete +window, single-node: + +- **The branch**: the sweep runs the durable form when the kv engine + hosts the window (sqlite — the one-window-host posture); the inline + form is preserved **verbatim** for non-SQL engines (mem/local — the + wave-2 sweep tests stay green unchanged). The branch keys off the + engine representation (the `KvEngine` match), never off a flag. +- **The single-node posture is unchanged** (ADR-008 §2): pins, RAII + guards, and the arbitration lock stay in-process (`GcHome`) — only + the *persistence* of the delete window (the staged-candidate table) + is engine rows. +- **The durable form's protocol** (per sweep, under the lease): + 1. **Execute overdue** (before/alongside marking — ADR-012 §6.7): + `window_take(now)` returns the overdue keys; each is + **re-arbitrated at execution time** through the same + `arbitrated_delete` path and deleted in batches. The protect- + consult posture on this path is a recorded design point (the + sweep's protect consult covers its execution phase — the same + reasoning that keeps the inline form's candidates at + `consult_protect = false`). A vanished candidate (deleted during + the delay by a direct delete) settles into the cancelled count — + the vanished-candidate accounting carries over. + 2. **Mark**: enumerate the pool, compute the live set, compute the + candidate-dead set (unchanged). + 3. **Stage**: `window_stage(candidates)` at + `visible_after = mark_end + window` — the constructor-tunable + delay (ADR-008 §2; the store carries the value its constructor + took). + 4. **Report**: `candidates_staged` = candidates staged this sweep; + `deleted`/`cancelled_by_arbitration` = the executed overdue set. + **Exact-count invariants are asserted relative to this schedule** + (ADR-012 §6.7) — staged-this-sweep and executed-this-sweep are + different sets (the delay separates them); the suite's sqlite + column asserts the two-sweep shapes (`suite-column-sqlite`). +- **Abort shapes unchanged**: `NoLivenessSources`/`ProtectFailed`/ + `SweeperLock` abort before anything is staged or executed (nothing + deleted, `Ok(SweepReport)`); operational failures (engine I/O) are + the `Err` half (ADR-012 §1). +- **The sweep-vs-put race under the durable form**: a put landing + during the window delay pins in-process (pin-before-publish); the + execution-time re-arbitration sees the pin and cancels the candidate + — the test-asserted property (a visible entry is never deleted while + liveness protects it; an in-flight put is never deleted by a sweep + started before it committed) holds **across the delay**, and is + unit-tested in that form. +- **Unit tests** in `store.rs`: two-sweep exact-count shapes (sweep 1 + stages N at delay 0 → sweep 2 executes exactly N — deleted + + cancelled == N against the staged set); the race across the delay + (put during the window survives; the unpinned twin deletes); the + abort shapes on a sqlite store; the vanished-candidate form; the + inline form untouched (the mem/local suites stay green). + +## Acceptance Criteria + +- [ ] The sweep branches on window hosting; the inline form's behavior + and its wave-2 tests unchanged +- [ ] Durable form: overdue rows execute with execution-time + re-arbitration before/alongside marking; candidates stage at + `visible_after = mark_end + window` +- [ ] Report counts exact relative to the schedule (unit-tested + two-sweep shapes); aborts delete nothing +- [ ] The race property unit-tested across the delay (a put landing + during the window survives the next sweep's execution) +- [ ] The protect-consult posture on overdue execution recorded +- [ ] `cargo test` (default) green; `cargo clippy --all-targets -- -D + warnings` and `cargo fmt --check` green + +## References + +- src/store.rs (the sweep — `sweep`/`sweep_window`/`arbitrated_delete`) +- docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md (§Sweep — the protocol) +- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md §2 (the staged re-arbitrated delete; the delay) +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §1 (abort-as-data), §5 (single-node posture unchanged), §6.7 (the executor is the next sweep) +- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §2 (the lease semantics) +- src/gc.rs (`GcHome` arbitration; `EngineWindowState` the store core calls) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/suite-column-sqlite.md b/tasks/suite-column-sqlite.md new file mode 100644 index 0000000..eb63193 --- /dev/null +++ b/tasks/suite-column-sqlite.md @@ -0,0 +1,108 @@ +--- +id: suite-column-sqlite +name: Conformance suite — the sqlite column (the first SQL-backed column) +status: pending +depends_on: [store-sweep-durable-window] +scope: moderate +risk: medium +impact: phase +level: implementation +tags: [wave-3, suite, sqlite] +--- + +## Description + +Grow the **sqlite column** — the suite's first SQL-backed column, +server-less (never skips; README §SKIP posture: server-less engines' +columns must always be green in every environment). The column runs in +the default build (feature `sqlite` is default-on). + +- **Factories** (`tests/conformance/sqlite_column.rs`): + - dual-tier `kv = sqlite + large = local` at a column-declared + threshold and the **test tuning of the window delay (`0` — + execute-on-next-sweep)**; + - `kv-only(sqlite)` at a column-declared threshold (the rejection + row's host; threshold materializable per the README's + configuration contract). + Fresh isolated sqlite file per `open()` (temp file), idempotent + teardown (remove the file), panicking-row-safe — the local column's + instance-tracks-its-own-root pattern. +- **Mirror the form-independent reference rows** (README §The + reference row set): INV-1 round-trips (content matrix × both put + paths, in-body), INV-2 dedup (same-path + cross-path legs), INV-3 + stage hygiene (the degenerate kv stage form via the injectable + failure modes — mid-stream `Io`, declared-len `Verification`), INV-6 + stat discipline, and the kv-only rejection row (threshold-agnostic + by discovery, unchanged). +- **The sweep rows are schedule-relative on this column.** The + reference set's single-sweep exact-count rows are the inline-form + host's (mem); the durable form's semantics differ (staged this + sweep, executed by the next — ADR-012 §6.7). The sqlite column grows + its own **durable-schedule rows** (concrete-factory per README + §Columns beyond the reference set, ADR-stamped, traced): + - exact counts across two sweeps (sweep 1 stages N, deletes nothing; + sweep 2 executes exactly N — `deleted + cancelled == N` against + the staged set) over the full protection set; + - the sweep-vs-put race across the window delay (a put landing + during the window survives the executing sweep; the unpinned twin + deletes); + - the abort shapes (`NoLivenessSources`/`ProtectFailed`/ + `SweeperLock` — form-independent, abort before staging; these + mirror directly); + - INV-5 list correctness through durable-schedule outcomes (exact + counts, never list comparison). +- **The dual-tier dispatch rows** mirror the local column's family + (pure-function routing, threshold boundaries, fall-through per op) + with the placement oracle adapted: both tiers are on disk now — kv + entries live in the sqlite file, large entries under the local root; + the oracle distinguishes by medium (the sqlite file vs the root's + committed files). +- **Virgin-store rows**: reads are no-ops on a fresh file; the + no-tables-yet tolerance form. +- **README amendments** (the convention's normative test-side home): + - the **schedule-relative reading of INV-4/INV-5 for SQL-backed + columns** — the durable form's exact counts are asserted relative + to the staged schedule (ADR-012 §6.7); the reference set's + single-sweep rows are the inline-form host's, and SQL-backed + columns run the durable-schedule equivalents (the mirror + convention's recorded amendment); + - the column inventory (§Columns beyond the reference set grows the + sqlite rows table); + - current state (three columns; the default-build column). +- **Structural factory tests** (the mem column's pattern): factory + isolation (fresh file per open, nothing shared) and + teardown-against-a-panicking-row. + +## Acceptance Criteria + +- [ ] The sqlite column green server-less: the mirrored + form-independent reference rows + the durable-schedule sweep + rows + the dispatch rows + the virgin-store rows, every row + ADR-stamped and traceable to the backlog +- [ ] The schedule-relative INV-4/INV-5 reading recorded in README + (the mirror convention's amendment, citing ADR-012 §6.7) +- [ ] Factory isolation + idempotent teardown hold (fresh file per + open; teardown safe against a panicking row), structurally + exercised +- [ ] `cargo test` (default) green — the column runs in the default + build; `cargo clippy --all-targets -- -D warnings` and + `cargo fmt --check` green + +## References + +- tests/conformance/README.md (the conventions this grows against; + §SKIP posture, §Columns beyond the reference set) +- tests/conformance/rows.rs (the reference row set being mirrored) +- tests/conformance/local_column.rs (the dual-tier column pattern) +- tests/conformance/backlog.rs (INV-1…7 entries) +- docs/architecture/store-api.md (§Invariants — the test gate) +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §6.7 (the schedule the sweep rows assert against) +- docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md §2 (mem's reference role; INV-7) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file