--- id: store-sweep-durable-window name: the sweep's durable-window form — staged candidates, next-sweep execution status: completed depends_on: [store-sqlite-construction] scope: moderate risk: medium impact: phase level: implementation tags: [wave-3, store, gc, sweep] --- ## Description Grow the sweep's **SQL-backed form** (ADR-008 §2; ADR-012 §5/§6.7) beside the inline form in `src/store.rs` — the first durable delete window, single-node: - **The branch**: the sweep runs the durable form when the kv engine hosts the window (sqlite — the one-window-host posture); the inline form is preserved **verbatim** for non-SQL engines (mem/local — the wave-2 sweep tests stay green unchanged). The branch keys off the engine representation (the `KvEngine` match), never off a flag. - **The single-node posture is unchanged** (ADR-008 §2): pins, RAII guards, and the arbitration lock stay in-process (`GcHome`) — only the *persistence* of the delete window (the staged-candidate table) is engine rows. - **The durable form's protocol** (per sweep, under the lease): 1. **Execute overdue** (before/alongside marking — ADR-012 §6.7): `window_take(now)` returns the overdue keys; each is **re-arbitrated at execution time** through the same `arbitrated_delete` path and deleted in batches. The protect- consult posture on this path is a recorded design point (the sweep's protect consult covers its execution phase — the same reasoning that keeps the inline form's candidates at `consult_protect = false`). A vanished candidate (deleted during the delay by a direct delete) settles into the cancelled count — the vanished-candidate accounting carries over. 2. **Mark**: enumerate the pool, compute the live set, compute the candidate-dead set (unchanged). 3. **Stage**: `window_stage(candidates)` at `visible_after = mark_end + window` — the constructor-tunable delay (ADR-008 §2; the store carries the value its constructor took). 4. **Report**: `candidates_staged` = candidates staged this sweep; `deleted`/`cancelled_by_arbitration` = the executed overdue set. **Exact-count invariants are asserted relative to this schedule** (ADR-012 §6.7) — staged-this-sweep and executed-this-sweep are different sets (the delay separates them); the suite's sqlite column asserts the two-sweep shapes (`suite-column-sqlite`). - **Abort shapes unchanged**: `NoLivenessSources`/`ProtectFailed`/ `SweeperLock` abort before anything is staged or executed (nothing deleted, `Ok(SweepReport)`); operational failures (engine I/O) are the `Err` half (ADR-012 §1). - **The sweep-vs-put race under the durable form**: a put landing during the window delay pins in-process (pin-before-publish); the execution-time re-arbitration sees the pin and cancels the candidate — the test-asserted property (a visible entry is never deleted while liveness protects it; an in-flight put is never deleted by a sweep started before it committed) holds **across the delay**, and is unit-tested in that form. - **Unit tests** in `store.rs`: two-sweep exact-count shapes (sweep 1 stages N at delay 0 → sweep 2 executes exactly N — deleted + cancelled == N against the staged set); the race across the delay (put during the window survives; the unpinned twin deletes); the abort shapes on a sqlite store; the vanished-candidate form; the inline form untouched (the mem/local suites stay green). ## Acceptance Criteria - [ ] The sweep branches on window hosting; the inline form's behavior and its wave-2 tests unchanged - [ ] Durable form: overdue rows execute with execution-time re-arbitration before/alongside marking; candidates stage at `visible_after = mark_end + window` - [ ] Report counts exact relative to the schedule (unit-tested two-sweep shapes); aborts delete nothing - [ ] The race property unit-tested across the delay (a put landing during the window survives the next sweep's execution) - [ ] The protect-consult posture on overdue execution recorded - [ ] `cargo test` (default) green; `cargo clippy --all-targets -- -D warnings` and `cargo fmt --check` green ## References - src/store.rs (the sweep — `sweep`/`sweep_window`/`arbitrated_delete`) - docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md (§Sweep — the protocol) - docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md §2 (the staged re-arbitrated delete; the delay) - docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §1 (abort-as-data), §5 (single-node posture unchanged), §6.7 (the executor is the next sweep) - docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §2 (the lease semantics) - src/gc.rs (`GcHome` arbitration; `EngineWindowState` the store core calls) ## Notes The durable form's shape as it landed: - **The branch is the kv-engine enum match itself** (`sweep_window`'s `match &self.kv`): `KvEngine::Sqlite(engine)` runs the durable form, `KvEngine::Mem(_)` runs the inline form. The two durable helpers are generic over `W: EngineWindowState` (the seam has `Backend` associated types, so a `&dyn` handout is not object-safe — the concrete `&SqliteBackend` passes directly). Wave 5's pg-lo slots one more match arm in. - **The inline body was extracted verbatim** into `sweep_window_inline` (the wave-2/all-features mem+local suites run green unchanged); the protect consult stays in `sweep_window` above the branch, shared by both forms. - **The protect-consult posture on overdue execution is recorded as a doc point** on `execute_overdue_window`: the consult already ran for the sweep (its abort, `ProtectFailed`, precedes execution) and its coverage extends to the executor phase, so the overdue re-arbitrations carry `consult_protect = false` — the inline form's same posture. - **Malformed window rows are fatal** (`Error::KeyInvalid` via the shared `typed_keys` boundary) — the pool-list discipline extended to the window read. - **`visible_after` overflow is `Err(Error::Io)`** (checked clock add); no panics on the library path. - **Report shape**: `candidates_staged` = staged this sweep; `deleted`/`cancelled_by_arbitration` = the overdue set executed this sweep (a vanished candidate settles into the cancelled count). The `sweep()` doc states the exact-count invariants relative to each form's schedule. ## Summary The sweep's durable SQL-backed form landed beside the inline form in `src/store.rs` (the first durable delete window, single-node; ADR-008 §2 staged re-arbitrated delete, ADR-012 §5/§6.7 the-executor-is-the- next-sweep): - `sweep_window` branches on the delete window's host — the kv-engine enum (`KvEngine::Sqlite` → durable; `KvEngine::Mem` → inline, extracted verbatim). Under the lease: the protect consult (abort `ProtectFailed` precedes any execution), then for the durable form **execute overdue** — `window_take(now)` → execution-time re-arbitration through `arbitrated_delete(key, false)` in ~100-key batches (vanishes count as cancelled) — then **mark** (unchanged), then **stage** at `visible_after = mark_end + window` (the constructor-tunable delay; `Duration::ZERO` = execute-on-next- sweep). Report counts exact relative to the schedule. - Abort shapes unchanged and unit-tested on sqlite: `NoLivenessSources`/`ProtectFailed`/`SweeperLock` abort before anything is executed or staged; the `ProtectFailed` test proves the overdue set is left waiting in the window (abort precedes `window_take`). - Unit tests (`sqlite_window_tests`): two-sweep exact counts at delay 0 (sweep 1 stages both tiers' dead entries without executing; sweep 2 executes exactly the staged set), future-delay staging (no execution before the window closes; re-stage is idempotent), the race across the delay (a put landing during the window survives via the execution-time re-arbitration; the unpinned twin deletes; the cancelled candidate re-stages and executes once unpinned), the vanished-candidate form, and the three abort shapes. Verification: `cargo test` (69 lib + 5 integration, default), 124 tests `--all-features`, 101 tests `--no-default-features --features mem`, `cargo clippy --all-targets -- -D warnings` (default and `--all-features`), and `cargo fmt --check` all green; the mem/local wave-2 suites pass unchanged (the inline form untouched).