docs(tasks): decompose wave 3 — the sqlite kv engine, the durable delete window, the first SQL-backed column
Six tasks against the wave-2 contract surface as reviewed (no amendments from the gate): - engine-sqlite: the default kv engine — WAL + synchronous=NORMAL, prepared statements, the indexed size column probe (B5's row shape), has-as-EXISTS, virgin-store no-tables-yet tolerance, spawn_blocking discipline - engine-sqlite-window: the engine-state seam's window family only (ADR-012 §5's reduction — gc_window, stage/take; no pin rows, no sweep-lock body) - store-sqlite-construction: KvEngine::Sqlite slots in, the dual-tier/kv-only constructors with the constructor-tunable window delay (default 5 min), ensure_gc_schema at construction, the one-window-host posture - store-sweep-durable-window: the sweep's SQL-backed form — overdue rows execute with re-arbitration before/alongside marking, candidates stage at mark_end + window, counts exact relative to the schedule (ADR-012 §6.7); the inline form preserved verbatim - suite-column-sqlite: the first SQL-backed column, server-less — form-independent reference rows mirror directly; the sweep rows are schedule-relative (the recorded INV-4/5 mirror amendment) - review-wave-3: the engine-wave gate (engine-vs-contract conformance) Verification: dependency graph validated (all ids resolve, no cycles, topological order clean); docs-only change. Refs: docs/plans/implementation.md §Wave 3
This commit is contained in:
1 parent
79cf1f0220
commit
758ccfb359
7 files changed
+606
-2
No files matched your search
@@ -1,6 +1,6 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: "2026-10-11 (wave 2 decomposed into tasks/)"
|
||||
last_updated: "2026-10-11 (wave 3 decomposed into tasks/)"
|
||||
---
|
||||
|
||||
# alkblobs — Implementation plan
|
||||
@@ -50,7 +50,7 @@ ops payload surfaces; release readiness is last.
|
||||
|---|---|---|---|
|
||||
| [1](#wave-1--foundations) | Crate/feature scaffold; hashing & keys (the one-way door); error model; Backend trait + I/O seams + engine-state seam shape; facade value types; conformance-suite convention | — | decomposed (2026-10-11) |
|
||||
| [2](#wave-2--the-store-works-dependency-free-mem--local) | mem + local engines; facade put/read paths + dispatch + constructor modes; in-process GC protocol; suite's first columns | wave 1 | decomposed (2026-10-11) |
|
||||
| [3](#wave-3--sqlite-engine) | sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column | wave 2 | pending |
|
||||
| [3](#wave-3--sqlite-engine) | sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column | wave 2 | decomposed (2026-10-11) |
|
||||
| [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | pending |
|
||||
| [5](#wave-5--pg-lo-large-engine) | pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column | wave 4 | pending |
|
||||
| [6](#wave-6--fleet-gc) | Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations | waves 4 + 5 | pending |
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
---
|
||||
id: engine-sqlite-window
|
||||
name: sqlite engine-state window family — the durable single-node delete window
|
||||
status: pending
|
||||
depends_on: [engine-sqlite]
|
||||
scope: narrow
|
||||
risk: low
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-3, engines, sqlite, gc]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Implement the engine-state companion seam's **window family** on the
|
||||
sqlite backend (`src/sqlite.rs` or its seam module): the
|
||||
`EngineWindowState` impl per ADR-010 §2 **as reduced by ADR-012 §5** —
|
||||
the window family only. The pin clauses and the sweep-lock body are
|
||||
**dropped dead machinery** for sqlite (ADR-012 §5's table: pin clauses
|
||||
**no**, sweep-lock clause **no** — the single-flighter is the
|
||||
in-process lock; a sqlite fleet is a recorded non-option, ADR-010 §3).
|
||||
Nothing of `EngineFleetState` may exist on this engine.
|
||||
|
||||
- **`ensure_gc_schema`** — creates **`gc_window` only**
|
||||
(`gc_window (key BLOB PRIMARY KEY, visible_after INTEGER)`; the
|
||||
timestamp encoding — `SystemTime` ↔ integer, sub-second precision
|
||||
for the test tunings — is a recorded design point). Never `gc_pins`;
|
||||
the embedder's liveness table is never created here (ADR-008's
|
||||
schema-ownership rule — the sweep reads it and never owns it).
|
||||
Called at store construction by the store core
|
||||
(`store-sqlite-construction` wires the call); the clause body is
|
||||
engine-side.
|
||||
- **`window_stage`** — stage candidates into the durable window,
|
||||
**idempotent per `(key, deadline)`** (`INSERT OR IGNORE` semantics —
|
||||
the body's discipline; the store stages each candidate once per
|
||||
sweep, and a re-stage of the same key at a new deadline updates the
|
||||
row). Virgin store: tolerated (the table exists from construction;
|
||||
a no-tables-yet database stages lazily or errors nothing — the
|
||||
Backend trait's engine posture carries here).
|
||||
- **`window_take(now)`** — take overdue candidates
|
||||
(`visible_after <= now`): the rows **leave the window table and
|
||||
return as keys** for the calling sweep to re-arbitrate and delete —
|
||||
the executor is the next sweep (ADR-012 §6.7). Empty when nothing is
|
||||
overdue; virgin store: empty. The take is one atomic step (selected
|
||||
rows removed and returned together — a row is never handed out
|
||||
twice).
|
||||
- **Contract tests** per the applicability row (ADR-012 §5's table is
|
||||
the contract-test matrix): the schema creates the window table only
|
||||
(assert `gc_pins` does not exist); stage idempotence per
|
||||
`(key, deadline)` incl. the re-stage-at-new-deadline form; take
|
||||
returns exactly the overdue set, removes it, leaves future rows;
|
||||
empty/virgin shapes; the timestamp encoding's boundary
|
||||
(`visible_after == now` is overdue — `<=` per the seam doc).
|
||||
|
||||
The store core drives these clauses (`store-sweep-durable-window`);
|
||||
this task ships the clauses and their engine-level contract tests.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The sqlite backend implements `EngineWindowState` — the window
|
||||
family only; no pin-clause or sweep-lock body exists anywhere in
|
||||
the sqlite engine
|
||||
- [ ] `ensure_gc_schema` creates `gc_window` only; `gc_pins` is never
|
||||
created by sqlite (contract-tested)
|
||||
- [ ] `window_stage` is idempotent per `(key, deadline)`; a re-stage
|
||||
at a new deadline updates the row
|
||||
- [ ] `window_take` returns exactly the overdue set (`<= now`
|
||||
boundary), removes it atomically, leaves future rows; empty on
|
||||
none/virgin
|
||||
- [ ] Clause contract tests green; `cargo test` (default) green;
|
||||
`cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green
|
||||
|
||||
## References
|
||||
|
||||
- src/gc.rs (`EngineWindowState` — the seam contract and its clause docs)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state.md §2
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §5 (the applicability table; the sqlite reduction), §6.7 (the executor)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md §2 (the durable window's semantics)
|
||||
- docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md (the delete-window protocol)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,118 @@
|
||||
---
|
||||
id: engine-sqlite
|
||||
name: sqlite kv engine — the default engine
|
||||
status: pending
|
||||
depends_on: []
|
||||
scope: narrow
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-3, engines, sqlite]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Implement the `sqlite` kv-tier engine (feature `sqlite`, **default-on**;
|
||||
ADR-007; REQ-1) against the wave-1 trait (`src/backend.rs`) — the first
|
||||
SQL-backed engine and the default build's kv engine.
|
||||
|
||||
- **Backend impl** (`src/sqlite.rs`, behind `#[cfg(feature = "sqlite")]`):
|
||||
the seven methods plus `put_staged`, opaque byte keys/values,
|
||||
`name() -> "sqlite"`.
|
||||
- **Schema** — the POC #5 finding B5 row shape adapted to sqlite:
|
||||
`blobs (key BLOB PRIMARY KEY, value BLOB, size INTEGER NOT NULL)`.
|
||||
The dedicated `size` column is what makes the **indexed `size`
|
||||
column probe** metadata-only: `SELECT size … WHERE key = ?` rides
|
||||
the key's index and never loads the value blob (ADR-008's size
|
||||
discipline).
|
||||
- **WAL posture** — `journal_mode=WAL` + `synchronous=NORMAL` set at
|
||||
open: the shipped durability tier (ADR-003; the posture honker
|
||||
ships and iroh's store measured). The **single-writer ceiling is a
|
||||
documented posture, not a defect** (REQ-1's economics — the module
|
||||
doc records it; ~1.2k objects/s under contention is an order above
|
||||
single-node push rates).
|
||||
- **Prepared statements** — rusqlite's `prepare_cached` discipline
|
||||
per the backends spec's "prepared statements" requirement.
|
||||
- `has` — an **EXISTS probe** (`SELECT EXISTS(…)`) — never a value
|
||||
load.
|
||||
- `get` — the **materializing cursor body** (kv family): the whole
|
||||
value loaded once (bounded by tier policy — the kv tier holds only
|
||||
≤ threshold content), `len()` from the row's size column, reads
|
||||
slice the materialized value composing `backend.rs`'s `clamped`.
|
||||
This body is also what the facade's verification-on-get re-derives
|
||||
over (store.rs's kv design point — no engine change needed).
|
||||
- `put` — CAS first-writer-wins (`INSERT OR IGNORE`); a re-put of an
|
||||
existing key is a no-op, never an overwrite.
|
||||
- `put_staged` — the **degenerate in-memory stage** (ADR-010 §1: kv
|
||||
bodies stage in memory and commit through the whole-value row
|
||||
write); `append` extends the buffer, `commit` publishes via the
|
||||
same `INSERT OR IGNORE` clause and self-discards on failure,
|
||||
`discard` drops the buffer with zero residue.
|
||||
- `delete` — `Err(Error::Missing)` when absent (rows-affected
|
||||
discipline); `list` — complete and well-formed (`SELECT key`);
|
||||
`size` — the indexed column probe above.
|
||||
- **Virgin-store no-tables-yet tolerance** — read paths treat a
|
||||
missing table as absent/empty, never "table does not exist" errors
|
||||
(POC #1 finding 2; store-api.md §Error model's virgin-store
|
||||
semantics). Schema creation timing (constructor `CREATE TABLE IF
|
||||
NOT EXISTS` vs lazy) is a recorded design point; the tolerance is
|
||||
unit-tested against a tableless database file either way.
|
||||
- **Blocking discipline**: rusqlite is synchronous — **all engine I/O
|
||||
runs in `spawn_blocking`** (AGENTS convention 3; store-api.md
|
||||
§Concurrency posture). The connection posture (the expected shape: a
|
||||
single connection behind a short-held lock, the single-writer
|
||||
posture; a pool is not required solo) is a recorded design point.
|
||||
No blocking I/O on the async path; no `unwrap`/`expect` outside
|
||||
tests.
|
||||
- **Unit tests** in-module: the method contract (virgin-store no-ops
|
||||
incl. the no-tables-yet form, CAS re-put, Missing discipline, size
|
||||
exactness + metadata-only), the staged-put happy path and both
|
||||
hygiene forms (discard → zero residue; commit failure →
|
||||
self-discard — a `#[cfg(test)]` fail-commit knob standing in for
|
||||
media failure, the mem engine's pattern), the pragma posture.
|
||||
|
||||
The engine-state window family (`EngineWindowState`) is
|
||||
`engine-sqlite-window` (separate task); the constructor wiring is
|
||||
`store-sqlite-construction`. This task ships the engine and its
|
||||
in-module unit tests only.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `src/sqlite.rs` implements `Backend` completely (seven methods +
|
||||
`put_staged`) behind feature `sqlite` (default-on), with the
|
||||
materializing cursor body and the degenerate staging body
|
||||
- [ ] WAL + `synchronous=NORMAL` set at open; prepared-statement
|
||||
discipline; `has` is an EXISTS probe; the single-writer ceiling
|
||||
recorded as posture in the module doc
|
||||
- [ ] `size` reads the dedicated size column via the key index, never
|
||||
the value blob (metadata-only, unit-tested)
|
||||
- [ ] CAS first-writer-wins on both put forms; re-put never overwrites
|
||||
- [ ] Stage hygiene holds in both forms (discard → zero residue;
|
||||
failed commit → self-discard), unit-tested
|
||||
- [ ] Virgin-store reads are no-ops including the no-tables-yet form
|
||||
(unit-tested against a tableless database file)
|
||||
- [ ] All engine I/O in `spawn_blocking`; connection posture recorded
|
||||
- [ ] `cargo test` (default — sqlite is default-on) green;
|
||||
`cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/backends-and-dispatch.md (§kv tier — sqlite engine)
|
||||
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
|
||||
- docs/architecture/decisions/003-backend-contract-and-dispatch.md (the
|
||||
shipped durability tier; the three numbered put rules' kv side)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state.md §1
|
||||
- docs/architecture/requirements.md (REQ-1's economics)
|
||||
- docs/research/poc-postgres-kv-findings.md (finding B5's row shape)
|
||||
- docs/research/poc-largeblob-findings.md (finding A4's economics)
|
||||
- src/backend.rs (the trait + seam contracts this implements)
|
||||
- src/mem.rs (the contract-reference engine's shapes this mirrors)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,96 @@
|
||||
---
|
||||
id: review-wave-3
|
||||
name: Wave 3 review gate — the sqlite engine vs the contract
|
||||
status: pending
|
||||
depends_on: [suite-column-sqlite]
|
||||
scope: narrow
|
||||
risk: low
|
||||
impact: phase
|
||||
level: review
|
||||
tags: [wave-3, review]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
The wave-3 review gate — the engine-wave review shape
|
||||
(docs/plans/implementation.md §Review gates: "engine-vs-contract
|
||||
conformance"), before wave 4 decomposes. The gate reads the
|
||||
implementation against the pinned ADR text line by line (the wave-1/2
|
||||
gates' method).
|
||||
|
||||
Gate checks:
|
||||
|
||||
- **Engine-vs-contract conformance**: the suite column green
|
||||
server-less; the engine-state window family contract-tested per its
|
||||
applicability row (ADR-012 §5's table — window family yes, pin
|
||||
clauses no, sweep-lock body no); virgin-store semantics including
|
||||
the no-tables-yet form; the `size` metadata-only discipline (the
|
||||
indexed column probe never loads the value blob).
|
||||
- **The sqlite posture vs ADR-003/007**: WAL + `synchronous=NORMAL`
|
||||
as the shipped durability tier; the single-writer ceiling documented
|
||||
as posture, not hedged (REQ-1's economics); prepared-statement
|
||||
discipline; `has` as an EXISTS probe; the B5 row shape (key/value/
|
||||
size); bounded reads.
|
||||
- **The durable-window sweep vs ADR-008 §2 / ADR-012 §5/§6.7**:
|
||||
candidates staged at `visible_after = mark_end + window` (the
|
||||
constructor-tunable delay, shipped default 5 min); the executor is
|
||||
the next sweep; execution-time re-arbitration; report counts exact
|
||||
relative to the schedule; the inline form preserved verbatim for
|
||||
non-SQL engines; the single-node posture unchanged (pins/liveness/
|
||||
arbitration lock in-process — only the window's persistence is
|
||||
engine rows); the one-window-host posture recorded.
|
||||
- **The seam reduction honored**: no pin rows, no sweep-lock body
|
||||
anywhere in the sqlite engine (dead machinery per ADR-012 §5);
|
||||
`ensure_gc_schema` creates `gc_window` only; the embedder's
|
||||
liveness table never created by the engine.
|
||||
- **Blocking discipline**: all rusqlite work off the async path
|
||||
(`spawn_blocking`); no blocking I/O under internal locks; the
|
||||
connection posture recorded.
|
||||
- **Named design points audited as resolved** (each recorded in its
|
||||
task's Notes): the connection posture; the timestamp encoding;
|
||||
schema-creation timing vs the no-tables-yet tolerance; the kv cursor
|
||||
dispatch shape over multiple kv engines; the protect-consult posture
|
||||
on overdue execution; the delay-0 test tuning's legality.
|
||||
- **Family conventions**: no comments in code (doc comments on public
|
||||
API fine), thiserror, tokio-only async, no unwrap/expect outside
|
||||
tests, naming, one-module-per-file re-export shape, the
|
||||
`#![allow(dead_code)]` scope notes matching their remaining scope.
|
||||
- **Gates green**: `cargo test` (default — the sqlite column runs
|
||||
here), `cargo test --features mem`, `cargo test --all-features`,
|
||||
`cargo clippy --all-targets -- -D warnings`,
|
||||
`cargo clippy --all-features --all-targets -- -D warnings`,
|
||||
`cargo fmt --check`.
|
||||
|
||||
Findings are fixed inline where cheap, or recorded as wave-4
|
||||
amendments where they shape the next decomposition. The gate's
|
||||
completion is what unblocks wave 4's decomposition.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Every gate check above read and verified against the cited ADR
|
||||
text (not from memory of it)
|
||||
- [ ] Named design points each decided, recorded, and consistent with
|
||||
the specs
|
||||
- [ ] Findings fixed inline or recorded as wave-4 amendments with
|
||||
owners
|
||||
- [ ] All gates re-run green after any fixes
|
||||
- [ ] Wave 4 decomposition unblocked (recorded in this task's Summary)
|
||||
|
||||
## References
|
||||
|
||||
- docs/plans/implementation.md (§Wave 3, §Review gates — engine wave reviews)
|
||||
- docs/architecture/decisions/003-backend-contract-and-dispatch.md
|
||||
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md §2
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state.md §2
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §1/§5/§6.7
|
||||
- docs/architecture/store-api.md
|
||||
- tasks/review-wave-2.md (the gate method)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by the review agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,95 @@
|
||||
---
|
||||
id: store-sqlite-construction
|
||||
name: sqlite constructor wiring — the default-build store shapes
|
||||
status: pending
|
||||
depends_on: [engine-sqlite, engine-sqlite-window]
|
||||
scope: narrow
|
||||
risk: low
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-3, store, sqlite, construction]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Slot sqlite into the facade's engine representation — the slotting-in
|
||||
pattern `src/store.rs`'s module doc pinned ("adding an enum variant and
|
||||
a named constructor is slotting in"):
|
||||
|
||||
- **`KvEngine::Sqlite`** variant behind `#[cfg(feature = "sqlite")]` +
|
||||
the dispatch arms (`put`/`get`/`has`/`size`/`delete`/`list`). The kv
|
||||
cursor dispatch (`AnyCursor`'s kv arm) carries the sqlite
|
||||
materializing body — the kv-arm shape over multiple kv engines (a
|
||||
`KvCursor` sub-enum vs per-engine variants) is a recorded design
|
||||
point. The `#[allow(unused_variables)]` empty-set scaffolding on the
|
||||
`KvEngine` methods shrinks (the kv set is no longer empty in default
|
||||
builds); `backend.rs`'s `#![allow(dead_code)]` scope note about "the
|
||||
kv engines' whole-value `put` before wave 3" is updated to the
|
||||
landed state.
|
||||
- **Named constructors**, feature-gated on `sqlite` (default-on — the
|
||||
default build constructs them):
|
||||
- **dual-tier** `kv = sqlite + large = local` — the default-build
|
||||
production shape; fallible (file I/O: opening/creating the
|
||||
database, creating the local root). Size-threshold dispatch at the
|
||||
constructor-tunable threshold as in the mem+local shape.
|
||||
- **kv-only(sqlite)** — no large tier; `threshold` is the put-time
|
||||
rejection point (ADR-012 §6.2; the rejection machinery is the put
|
||||
seam's, already landed — only the constructor slots in).
|
||||
- Both take the **delete-window delay** parameter (ADR-008 §2: the
|
||||
delete window's delay, **default 5 min, constructor-tunable per
|
||||
deployment** — a named shipped-default constant; `Duration`).
|
||||
This is the durable window's cadence knob the sweep's SQL-backed
|
||||
form consumes (`store-sweep-durable-window`); `0` is a legal
|
||||
degenerate tuning (execute-on-next-sweep — the test tuning). The
|
||||
mem/local constructors are unchanged: the inline form has no delay
|
||||
in effect.
|
||||
- **`ensure_gc_schema` at construction** — the store-core call
|
||||
(engine-neutral; sqlite's body creates `gc_window` only). A
|
||||
construction failure is `Err(Error::Io)` per the fallible-constructor
|
||||
pattern `dual_tier_mem_local` set.
|
||||
- **The one-window-host rule's store-core posture** (ADR-012 §5): the
|
||||
durable window hosts on **the kv engine when it is SQL-backed** —
|
||||
sqlite hosts. Recorded (code shape or doc note) so wave 5's pg-lo
|
||||
dormancy (the dual-SQL instance stages on the kv engine only) slots
|
||||
in without reshaping.
|
||||
- **Constructor unit tests**: the modes construct and introspect
|
||||
correctly (`mode()`/`threshold()`), `ensure_gc_schema` ran (the
|
||||
window table exists in the fresh database), the window-delay default
|
||||
is the pinned constant, kv-only rejection wiring unchanged, the
|
||||
dual-tier shape round-trips end-to-end (a smoke put/get through both
|
||||
tiers).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `KvEngine::Sqlite` slots in; all six kv ops dispatch; the kv
|
||||
cursor body dispatches (design point recorded)
|
||||
- [ ] `dual_tier_sqlite_local` + `kv_only_sqlite` constructors exist
|
||||
behind feature `sqlite`, take threshold + window delay, fallible
|
||||
where I/O is fallible
|
||||
- [ ] `ensure_gc_schema` runs at construction; the window-delay
|
||||
shipped default is 5 min (pinned constant) and
|
||||
constructor-tunable; `0` legal
|
||||
- [ ] The one-window-host posture recorded (sqlite hosts when the kv
|
||||
engine is SQL-backed)
|
||||
- [ ] The stale empty-set scaffolding/allow notes updated to the
|
||||
landed state
|
||||
- [ ] `cargo test` (default) green — the default build constructs and
|
||||
exercises sqlite stores; `cargo clippy --all-targets -- -D
|
||||
warnings` and `cargo fmt --check` green
|
||||
|
||||
## References
|
||||
|
||||
- src/store.rs (the engine-representation design point; the
|
||||
constructor modes; the slotting-in pattern)
|
||||
- docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md §4 (modes)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §6.3 (dual-tier shapes), §5 (the one-window-host rule)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md §2 (the window delay, default 5 min)
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §4 (the feature graph)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,99 @@
|
||||
---
|
||||
id: store-sweep-durable-window
|
||||
name: the sweep's durable-window form — staged candidates, next-sweep execution
|
||||
status: pending
|
||||
depends_on: [store-sqlite-construction]
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: phase
|
||||
level: implementation
|
||||
tags: [wave-3, store, gc, sweep]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Grow the sweep's **SQL-backed form** (ADR-008 §2; ADR-012 §5/§6.7)
|
||||
beside the inline form in `src/store.rs` — the first durable delete
|
||||
window, single-node:
|
||||
|
||||
- **The branch**: the sweep runs the durable form when the kv engine
|
||||
hosts the window (sqlite — the one-window-host posture); the inline
|
||||
form is preserved **verbatim** for non-SQL engines (mem/local — the
|
||||
wave-2 sweep tests stay green unchanged). The branch keys off the
|
||||
engine representation (the `KvEngine` match), never off a flag.
|
||||
- **The single-node posture is unchanged** (ADR-008 §2): pins, RAII
|
||||
guards, and the arbitration lock stay in-process (`GcHome`) — only
|
||||
the *persistence* of the delete window (the staged-candidate table)
|
||||
is engine rows.
|
||||
- **The durable form's protocol** (per sweep, under the lease):
|
||||
1. **Execute overdue** (before/alongside marking — ADR-012 §6.7):
|
||||
`window_take(now)` returns the overdue keys; each is
|
||||
**re-arbitrated at execution time** through the same
|
||||
`arbitrated_delete` path and deleted in batches. The protect-
|
||||
consult posture on this path is a recorded design point (the
|
||||
sweep's protect consult covers its execution phase — the same
|
||||
reasoning that keeps the inline form's candidates at
|
||||
`consult_protect = false`). A vanished candidate (deleted during
|
||||
the delay by a direct delete) settles into the cancelled count —
|
||||
the vanished-candidate accounting carries over.
|
||||
2. **Mark**: enumerate the pool, compute the live set, compute the
|
||||
candidate-dead set (unchanged).
|
||||
3. **Stage**: `window_stage(candidates)` at
|
||||
`visible_after = mark_end + window` — the constructor-tunable
|
||||
delay (ADR-008 §2; the store carries the value its constructor
|
||||
took).
|
||||
4. **Report**: `candidates_staged` = candidates staged this sweep;
|
||||
`deleted`/`cancelled_by_arbitration` = the executed overdue set.
|
||||
**Exact-count invariants are asserted relative to this schedule**
|
||||
(ADR-012 §6.7) — staged-this-sweep and executed-this-sweep are
|
||||
different sets (the delay separates them); the suite's sqlite
|
||||
column asserts the two-sweep shapes (`suite-column-sqlite`).
|
||||
- **Abort shapes unchanged**: `NoLivenessSources`/`ProtectFailed`/
|
||||
`SweeperLock` abort before anything is staged or executed (nothing
|
||||
deleted, `Ok(SweepReport)`); operational failures (engine I/O) are
|
||||
the `Err` half (ADR-012 §1).
|
||||
- **The sweep-vs-put race under the durable form**: a put landing
|
||||
during the window delay pins in-process (pin-before-publish); the
|
||||
execution-time re-arbitration sees the pin and cancels the candidate
|
||||
— the test-asserted property (a visible entry is never deleted while
|
||||
liveness protects it; an in-flight put is never deleted by a sweep
|
||||
started before it committed) holds **across the delay**, and is
|
||||
unit-tested in that form.
|
||||
- **Unit tests** in `store.rs`: two-sweep exact-count shapes (sweep 1
|
||||
stages N at delay 0 → sweep 2 executes exactly N — deleted +
|
||||
cancelled == N against the staged set); the race across the delay
|
||||
(put during the window survives; the unpinned twin deletes); the
|
||||
abort shapes on a sqlite store; the vanished-candidate form; the
|
||||
inline form untouched (the mem/local suites stay green).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The sweep branches on window hosting; the inline form's behavior
|
||||
and its wave-2 tests unchanged
|
||||
- [ ] Durable form: overdue rows execute with execution-time
|
||||
re-arbitration before/alongside marking; candidates stage at
|
||||
`visible_after = mark_end + window`
|
||||
- [ ] Report counts exact relative to the schedule (unit-tested
|
||||
two-sweep shapes); aborts delete nothing
|
||||
- [ ] The race property unit-tested across the delay (a put landing
|
||||
during the window survives the next sweep's execution)
|
||||
- [ ] The protect-consult posture on overdue execution recorded
|
||||
- [ ] `cargo test` (default) green; `cargo clippy --all-targets -- -D
|
||||
warnings` and `cargo fmt --check` green
|
||||
|
||||
## References
|
||||
|
||||
- src/store.rs (the sweep — `sweep`/`sweep_window`/`arbitrated_delete`)
|
||||
- docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md (§Sweep — the protocol)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md §2 (the staged re-arbitrated delete; the delay)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §1 (abort-as-data), §5 (single-node posture unchanged), §6.7 (the executor is the next sweep)
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §2 (the lease semantics)
|
||||
- src/gc.rs (`GcHome` arbitration; `EngineWindowState` the store core calls)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,108 @@
|
||||
---
|
||||
id: suite-column-sqlite
|
||||
name: Conformance suite — the sqlite column (the first SQL-backed column)
|
||||
status: pending
|
||||
depends_on: [store-sweep-durable-window]
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: phase
|
||||
level: implementation
|
||||
tags: [wave-3, suite, sqlite]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Grow the **sqlite column** — the suite's first SQL-backed column,
|
||||
server-less (never skips; README §SKIP posture: server-less engines'
|
||||
columns must always be green in every environment). The column runs in
|
||||
the default build (feature `sqlite` is default-on).
|
||||
|
||||
- **Factories** (`tests/conformance/sqlite_column.rs`):
|
||||
- dual-tier `kv = sqlite + large = local` at a column-declared
|
||||
threshold and the **test tuning of the window delay (`0` —
|
||||
execute-on-next-sweep)**;
|
||||
- `kv-only(sqlite)` at a column-declared threshold (the rejection
|
||||
row's host; threshold materializable per the README's
|
||||
configuration contract).
|
||||
Fresh isolated sqlite file per `open()` (temp file), idempotent
|
||||
teardown (remove the file), panicking-row-safe — the local column's
|
||||
instance-tracks-its-own-root pattern.
|
||||
- **Mirror the form-independent reference rows** (README §The
|
||||
reference row set): INV-1 round-trips (content matrix × both put
|
||||
paths, in-body), INV-2 dedup (same-path + cross-path legs), INV-3
|
||||
stage hygiene (the degenerate kv stage form via the injectable
|
||||
failure modes — mid-stream `Io`, declared-len `Verification`), INV-6
|
||||
stat discipline, and the kv-only rejection row (threshold-agnostic
|
||||
by discovery, unchanged).
|
||||
- **The sweep rows are schedule-relative on this column.** The
|
||||
reference set's single-sweep exact-count rows are the inline-form
|
||||
host's (mem); the durable form's semantics differ (staged this
|
||||
sweep, executed by the next — ADR-012 §6.7). The sqlite column grows
|
||||
its own **durable-schedule rows** (concrete-factory per README
|
||||
§Columns beyond the reference set, ADR-stamped, traced):
|
||||
- exact counts across two sweeps (sweep 1 stages N, deletes nothing;
|
||||
sweep 2 executes exactly N — `deleted + cancelled == N` against
|
||||
the staged set) over the full protection set;
|
||||
- the sweep-vs-put race across the window delay (a put landing
|
||||
during the window survives the executing sweep; the unpinned twin
|
||||
deletes);
|
||||
- the abort shapes (`NoLivenessSources`/`ProtectFailed`/
|
||||
`SweeperLock` — form-independent, abort before staging; these
|
||||
mirror directly);
|
||||
- INV-5 list correctness through durable-schedule outcomes (exact
|
||||
counts, never list comparison).
|
||||
- **The dual-tier dispatch rows** mirror the local column's family
|
||||
(pure-function routing, threshold boundaries, fall-through per op)
|
||||
with the placement oracle adapted: both tiers are on disk now — kv
|
||||
entries live in the sqlite file, large entries under the local root;
|
||||
the oracle distinguishes by medium (the sqlite file vs the root's
|
||||
committed files).
|
||||
- **Virgin-store rows**: reads are no-ops on a fresh file; the
|
||||
no-tables-yet tolerance form.
|
||||
- **README amendments** (the convention's normative test-side home):
|
||||
- the **schedule-relative reading of INV-4/INV-5 for SQL-backed
|
||||
columns** — the durable form's exact counts are asserted relative
|
||||
to the staged schedule (ADR-012 §6.7); the reference set's
|
||||
single-sweep rows are the inline-form host's, and SQL-backed
|
||||
columns run the durable-schedule equivalents (the mirror
|
||||
convention's recorded amendment);
|
||||
- the column inventory (§Columns beyond the reference set grows the
|
||||
sqlite rows table);
|
||||
- current state (three columns; the default-build column).
|
||||
- **Structural factory tests** (the mem column's pattern): factory
|
||||
isolation (fresh file per open, nothing shared) and
|
||||
teardown-against-a-panicking-row.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The sqlite column green server-less: the mirrored
|
||||
form-independent reference rows + the durable-schedule sweep
|
||||
rows + the dispatch rows + the virgin-store rows, every row
|
||||
ADR-stamped and traceable to the backlog
|
||||
- [ ] The schedule-relative INV-4/INV-5 reading recorded in README
|
||||
(the mirror convention's amendment, citing ADR-012 §6.7)
|
||||
- [ ] Factory isolation + idempotent teardown hold (fresh file per
|
||||
open; teardown safe against a panicking row), structurally
|
||||
exercised
|
||||
- [ ] `cargo test` (default) green — the column runs in the default
|
||||
build; `cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green
|
||||
|
||||
## References
|
||||
|
||||
- tests/conformance/README.md (the conventions this grows against;
|
||||
§SKIP posture, §Columns beyond the reference set)
|
||||
- tests/conformance/rows.rs (the reference row set being mirrored)
|
||||
- tests/conformance/local_column.rs (the dual-tier column pattern)
|
||||
- tests/conformance/backlog.rs (INV-1…7 entries)
|
||||
- docs/architecture/store-api.md (§Invariants — the test gate)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §6.7 (the schedule the sweep rows assert against)
|
||||
- docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md §2 (mem's reference role; INV-7)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
Reference in new issue
Block a user