glm-5.3-flash
97cdf682c4
Reviewed the wave-3 commits (engine-sqlite, engine-sqlite-window, store-sqlite-construction, store-sweep-durable-window, suite-column-sqlite) against the cited ADR/spec text line by line: ADR-003/007/008 §2, ADR-010 §2, ADR-012 §1/§5/§6.7, store-api.md, implementation.md §Review gates — the engine-vs-contract conformance (suite column green server-less in the default build; the window family contract-tested per its applicability row; virgin-store semantics incl. the no-tables-yet form; the size metadata-only discipline proven index-riding), the durability posture (WAL + synchronous=NORMAL verified at open; the single-writer ceiling posture, not hedge), the durable sweep form vs ADR-008 §2/ADR-012 §6.7 (stage at mark_end + window; next-sweep execution with raced execution-time re-arbitration; schedule-relative exact counts; aborts stage and execute nothing), the seam reduction (gc_window only; no pin rows, no sweep-lock body, no EngineFleetState), and the blocking discipline (all rusqlite in spawn_blocking). Named design points audited as resolved; family conventions clean; no wave-4 amendments arise. Inline fixes (review-of-record): - src/gc.rs: the seam's window_stage doc retired the wave-1 lazy-create hedge for the shipped constructor-at-open posture (a tableless database surfaces the engine error on stage; per-engine tolerances are each engine's recorded posture) - tasks/engine-sqlite-window, store-sweep-durable-window, suite-column-sqlite: stray placeholder lines above the filled Notes/Summary removed (the wave-2 gate's same cleanup) - tests/conformance/README.md: trailing newline added Verification (all re-run green after the fixes): - cargo test (default): 69 lib + 19 conformance + 3 git-interop - cargo test --features mem / --all-features: 124 lib + 54 conformance + 4 git-interop each - cargo clippy --all-targets -- -D warnings (default + all-features) - cargo fmt --check - cargo doc --no-deps at the pre-existing warning baseline (26) Wave 4 (postgres) decomposition is unblocked.
160 lines
8.4 KiB
Markdown
160 lines
8.4 KiB
Markdown
---
|
|
id: store-sweep-durable-window
|
|
name: the sweep's durable-window form — staged candidates, next-sweep execution
|
|
status: completed
|
|
depends_on: [store-sqlite-construction]
|
|
scope: moderate
|
|
risk: medium
|
|
impact: phase
|
|
level: implementation
|
|
tags: [wave-3, store, gc, sweep]
|
|
---
|
|
|
|
## Description
|
|
|
|
Grow the sweep's **SQL-backed form** (ADR-008 §2; ADR-012 §5/§6.7)
|
|
beside the inline form in `src/store.rs` — the first durable delete
|
|
window, single-node:
|
|
|
|
- **The branch**: the sweep runs the durable form when the kv engine
|
|
hosts the window (sqlite — the one-window-host posture); the inline
|
|
form is preserved **verbatim** for non-SQL engines (mem/local — the
|
|
wave-2 sweep tests stay green unchanged). The branch keys off the
|
|
engine representation (the `KvEngine` match), never off a flag.
|
|
- **The single-node posture is unchanged** (ADR-008 §2): pins, RAII
|
|
guards, and the arbitration lock stay in-process (`GcHome`) — only
|
|
the *persistence* of the delete window (the staged-candidate table)
|
|
is engine rows.
|
|
- **The durable form's protocol** (per sweep, under the lease):
|
|
1. **Execute overdue** (before/alongside marking — ADR-012 §6.7):
|
|
`window_take(now)` returns the overdue keys; each is
|
|
**re-arbitrated at execution time** through the same
|
|
`arbitrated_delete` path and deleted in batches. The protect-
|
|
consult posture on this path is a recorded design point (the
|
|
sweep's protect consult covers its execution phase — the same
|
|
reasoning that keeps the inline form's candidates at
|
|
`consult_protect = false`). A vanished candidate (deleted during
|
|
the delay by a direct delete) settles into the cancelled count —
|
|
the vanished-candidate accounting carries over.
|
|
2. **Mark**: enumerate the pool, compute the live set, compute the
|
|
candidate-dead set (unchanged).
|
|
3. **Stage**: `window_stage(candidates)` at
|
|
`visible_after = mark_end + window` — the constructor-tunable
|
|
delay (ADR-008 §2; the store carries the value its constructor
|
|
took).
|
|
4. **Report**: `candidates_staged` = candidates staged this sweep;
|
|
`deleted`/`cancelled_by_arbitration` = the executed overdue set.
|
|
**Exact-count invariants are asserted relative to this schedule**
|
|
(ADR-012 §6.7) — staged-this-sweep and executed-this-sweep are
|
|
different sets (the delay separates them); the suite's sqlite
|
|
column asserts the two-sweep shapes (`suite-column-sqlite`).
|
|
- **Abort shapes unchanged**: `NoLivenessSources`/`ProtectFailed`/
|
|
`SweeperLock` abort before anything is staged or executed (nothing
|
|
deleted, `Ok(SweepReport)`); operational failures (engine I/O) are
|
|
the `Err` half (ADR-012 §1).
|
|
- **The sweep-vs-put race under the durable form**: a put landing
|
|
during the window delay pins in-process (pin-before-publish); the
|
|
execution-time re-arbitration sees the pin and cancels the candidate
|
|
— the test-asserted property (a visible entry is never deleted while
|
|
liveness protects it; an in-flight put is never deleted by a sweep
|
|
started before it committed) holds **across the delay**, and is
|
|
unit-tested in that form.
|
|
- **Unit tests** in `store.rs`: two-sweep exact-count shapes (sweep 1
|
|
stages N at delay 0 → sweep 2 executes exactly N — deleted +
|
|
cancelled == N against the staged set); the race across the delay
|
|
(put during the window survives; the unpinned twin deletes); the
|
|
abort shapes on a sqlite store; the vanished-candidate form; the
|
|
inline form untouched (the mem/local suites stay green).
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] The sweep branches on window hosting; the inline form's behavior
|
|
and its wave-2 tests unchanged
|
|
- [ ] Durable form: overdue rows execute with execution-time
|
|
re-arbitration before/alongside marking; candidates stage at
|
|
`visible_after = mark_end + window`
|
|
- [ ] Report counts exact relative to the schedule (unit-tested
|
|
two-sweep shapes); aborts delete nothing
|
|
- [ ] The race property unit-tested across the delay (a put landing
|
|
during the window survives the next sweep's execution)
|
|
- [ ] The protect-consult posture on overdue execution recorded
|
|
- [ ] `cargo test` (default) green; `cargo clippy --all-targets -- -D
|
|
warnings` and `cargo fmt --check` green
|
|
|
|
## References
|
|
|
|
- src/store.rs (the sweep — `sweep`/`sweep_window`/`arbitrated_delete`)
|
|
- docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md (§Sweep — the protocol)
|
|
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md §2 (the staged re-arbitrated delete; the delay)
|
|
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §1 (abort-as-data), §5 (single-node posture unchanged), §6.7 (the executor is the next sweep)
|
|
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §2 (the lease semantics)
|
|
- src/gc.rs (`GcHome` arbitration; `EngineWindowState` the store core calls)
|
|
|
|
## Notes
|
|
|
|
The durable form's shape as it landed:
|
|
|
|
- **The branch is the kv-engine enum match itself** (`sweep_window`'s
|
|
`match &self.kv`): `KvEngine::Sqlite(engine)` runs the durable form,
|
|
`KvEngine::Mem(_)` runs the inline form. The two durable helpers are
|
|
generic over `W: EngineWindowState` (the seam has `Backend`
|
|
associated types, so a `&dyn` handout is not object-safe — the
|
|
concrete `&SqliteBackend` passes directly). Wave 5's pg-lo slots one
|
|
more match arm in.
|
|
- **The inline body was extracted verbatim** into
|
|
`sweep_window_inline` (the wave-2/all-features mem+local suites run
|
|
green unchanged); the protect consult stays in `sweep_window` above
|
|
the branch, shared by both forms.
|
|
- **The protect-consult posture on overdue execution is recorded as a
|
|
doc point** on `execute_overdue_window`: the consult already ran for
|
|
the sweep (its abort, `ProtectFailed`, precedes execution) and its
|
|
coverage extends to the executor phase, so the overdue
|
|
re-arbitrations carry `consult_protect = false` — the inline form's
|
|
same posture.
|
|
- **Malformed window rows are fatal** (`Error::KeyInvalid` via the
|
|
shared `typed_keys` boundary) — the pool-list discipline extended to
|
|
the window read.
|
|
- **`visible_after` overflow is `Err(Error::Io)`** (checked clock add);
|
|
no panics on the library path.
|
|
- **Report shape**: `candidates_staged` = staged this sweep;
|
|
`deleted`/`cancelled_by_arbitration` = the overdue set executed this
|
|
sweep (a vanished candidate settles into the cancelled count). The
|
|
`sweep()` doc states the exact-count invariants relative to each
|
|
form's schedule.
|
|
|
|
## Summary
|
|
|
|
The sweep's durable SQL-backed form landed beside the inline form in
|
|
`src/store.rs` (the first durable delete window, single-node; ADR-008
|
|
§2 staged re-arbitrated delete, ADR-012 §5/§6.7 the-executor-is-the-
|
|
next-sweep):
|
|
|
|
- `sweep_window` branches on the delete window's host — the kv-engine
|
|
enum (`KvEngine::Sqlite` → durable; `KvEngine::Mem` → inline,
|
|
extracted verbatim). Under the lease: the protect consult (abort
|
|
`ProtectFailed` precedes any execution), then for the durable form
|
|
**execute overdue** — `window_take(now)` → execution-time
|
|
re-arbitration through `arbitrated_delete(key, false)` in
|
|
~100-key batches (vanishes count as cancelled) — then **mark**
|
|
(unchanged), then **stage** at `visible_after = mark_end + window`
|
|
(the constructor-tunable delay; `Duration::ZERO` = execute-on-next-
|
|
sweep). Report counts exact relative to the schedule.
|
|
- Abort shapes unchanged and unit-tested on sqlite:
|
|
`NoLivenessSources`/`ProtectFailed`/`SweeperLock` abort before
|
|
anything is executed or staged; the `ProtectFailed` test proves the
|
|
overdue set is left waiting in the window (abort precedes
|
|
`window_take`).
|
|
- Unit tests (`sqlite_window_tests`): two-sweep exact counts at delay 0
|
|
(sweep 1 stages both tiers' dead entries without executing; sweep 2
|
|
executes exactly the staged set), future-delay staging (no execution
|
|
before the window closes; re-stage is idempotent), the race across
|
|
the delay (a put landing during the window survives via the
|
|
execution-time re-arbitration; the unpinned twin deletes; the
|
|
cancelled candidate re-stages and executes once unpinned), the
|
|
vanished-candidate form, and the three abort shapes.
|
|
|
|
Verification: `cargo test` (69 lib + 5 integration, default), 124
|
|
tests `--all-features`, 101 tests `--no-default-features --features
|
|
mem`, `cargo clippy --all-targets -- -D warnings` (default and
|
|
`--all-features`), and `cargo fmt --check` all green; the mem/local
|
|
wave-2 suites pass unchanged (the inline form untouched). |