Files
alkblobs/tasks/engine-sqlite.md
T
glm-5.3-flash 758ccfb359 docs(tasks): decompose wave 3 — the sqlite kv engine, the durable delete window, the first SQL-backed column
Six tasks against the wave-2 contract surface as reviewed (no
amendments from the gate):

- engine-sqlite: the default kv engine — WAL + synchronous=NORMAL,
  prepared statements, the indexed size column probe (B5's row shape),
  has-as-EXISTS, virgin-store no-tables-yet tolerance, spawn_blocking
  discipline
- engine-sqlite-window: the engine-state seam's window family only
  (ADR-012 §5's reduction — gc_window, stage/take; no pin rows, no
  sweep-lock body)
- store-sqlite-construction: KvEngine::Sqlite slots in, the
  dual-tier/kv-only constructors with the constructor-tunable window
  delay (default 5 min), ensure_gc_schema at construction, the
  one-window-host posture
- store-sweep-durable-window: the sweep's SQL-backed form — overdue
  rows execute with re-arbitration before/alongside marking, candidates
  stage at mark_end + window, counts exact relative to the schedule
  (ADR-012 §6.7); the inline form preserved verbatim
- suite-column-sqlite: the first SQL-backed column, server-less —
  form-independent reference rows mirror directly; the sweep rows are
  schedule-relative (the recorded INV-4/5 mirror amendment)
- review-wave-3: the engine-wave gate (engine-vs-contract conformance)

Verification: dependency graph validated (all ids resolve, no cycles,
topological order clean); docs-only change.

Refs: docs/plans/implementation.md §Wave 3
2026-10-11 23:48:23 +00:00

118 lines
5.7 KiB
Markdown

---
id: engine-sqlite
name: sqlite kv engine — the default engine
status: pending
depends_on: []
scope: narrow
risk: medium
impact: component
level: implementation
tags: [wave-3, engines, sqlite]
---
## Description
Implement the `sqlite` kv-tier engine (feature `sqlite`, **default-on**;
ADR-007; REQ-1) against the wave-1 trait (`src/backend.rs`) — the first
SQL-backed engine and the default build's kv engine.
- **Backend impl** (`src/sqlite.rs`, behind `#[cfg(feature = "sqlite")]`):
the seven methods plus `put_staged`, opaque byte keys/values,
`name() -> "sqlite"`.
- **Schema** — the POC #5 finding B5 row shape adapted to sqlite:
`blobs (key BLOB PRIMARY KEY, value BLOB, size INTEGER NOT NULL)`.
The dedicated `size` column is what makes the **indexed `size`
column probe** metadata-only: `SELECT size … WHERE key = ?` rides
the key's index and never loads the value blob (ADR-008's size
discipline).
- **WAL posture** — `journal_mode=WAL` + `synchronous=NORMAL` set at
open: the shipped durability tier (ADR-003; the posture honker
ships and iroh's store measured). The **single-writer ceiling is a
documented posture, not a defect** (REQ-1's economics — the module
doc records it; ~1.2k objects/s under contention is an order above
single-node push rates).
- **Prepared statements** — rusqlite's `prepare_cached` discipline
per the backends spec's "prepared statements" requirement.
- `has` — an **EXISTS probe** (`SELECT EXISTS(…)`) — never a value
load.
- `get` — the **materializing cursor body** (kv family): the whole
value loaded once (bounded by tier policy — the kv tier holds only
≤ threshold content), `len()` from the row's size column, reads
slice the materialized value composing `backend.rs`'s `clamped`.
This body is also what the facade's verification-on-get re-derives
over (store.rs's kv design point — no engine change needed).
- `put` — CAS first-writer-wins (`INSERT OR IGNORE`); a re-put of an
existing key is a no-op, never an overwrite.
- `put_staged` — the **degenerate in-memory stage** (ADR-010 §1: kv
bodies stage in memory and commit through the whole-value row
write); `append` extends the buffer, `commit` publishes via the
same `INSERT OR IGNORE` clause and self-discards on failure,
`discard` drops the buffer with zero residue.
- `delete` — `Err(Error::Missing)` when absent (rows-affected
discipline); `list` — complete and well-formed (`SELECT key`);
`size` — the indexed column probe above.
- **Virgin-store no-tables-yet tolerance** — read paths treat a
missing table as absent/empty, never "table does not exist" errors
(POC #1 finding 2; store-api.md §Error model's virgin-store
semantics). Schema creation timing (constructor `CREATE TABLE IF
NOT EXISTS` vs lazy) is a recorded design point; the tolerance is
unit-tested against a tableless database file either way.
- **Blocking discipline**: rusqlite is synchronous — **all engine I/O
runs in `spawn_blocking`** (AGENTS convention 3; store-api.md
§Concurrency posture). The connection posture (the expected shape: a
single connection behind a short-held lock, the single-writer
posture; a pool is not required solo) is a recorded design point.
No blocking I/O on the async path; no `unwrap`/`expect` outside
tests.
- **Unit tests** in-module: the method contract (virgin-store no-ops
incl. the no-tables-yet form, CAS re-put, Missing discipline, size
exactness + metadata-only), the staged-put happy path and both
hygiene forms (discard → zero residue; commit failure →
self-discard — a `#[cfg(test)]` fail-commit knob standing in for
media failure, the mem engine's pattern), the pragma posture.
The engine-state window family (`EngineWindowState`) is
`engine-sqlite-window` (separate task); the constructor wiring is
`store-sqlite-construction`. This task ships the engine and its
in-module unit tests only.
## Acceptance Criteria
- [ ] `src/sqlite.rs` implements `Backend` completely (seven methods +
`put_staged`) behind feature `sqlite` (default-on), with the
materializing cursor body and the degenerate staging body
- [ ] WAL + `synchronous=NORMAL` set at open; prepared-statement
discipline; `has` is an EXISTS probe; the single-writer ceiling
recorded as posture in the module doc
- [ ] `size` reads the dedicated size column via the key index, never
the value blob (metadata-only, unit-tested)
- [ ] CAS first-writer-wins on both put forms; re-put never overwrites
- [ ] Stage hygiene holds in both forms (discard → zero residue;
failed commit → self-discard), unit-tested
- [ ] Virgin-store reads are no-ops including the no-tables-yet form
(unit-tested against a tableless database file)
- [ ] All engine I/O in `spawn_blocking`; connection posture recorded
- [ ] `cargo test` (default — sqlite is default-on) green;
`cargo clippy --all-targets -- -D warnings` and
`cargo fmt --check` green
## References
- docs/architecture/backends-and-dispatch.md (§kv tier — sqlite engine)
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
- docs/architecture/decisions/003-backend-contract-and-dispatch.md (the
shipped durability tier; the three numbered put rules' kv side)
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state.md §1
- docs/architecture/requirements.md (REQ-1's economics)
- docs/research/poc-postgres-kv-findings.md (finding B5's row shape)
- docs/research/poc-largeblob-findings.md (finding A4's economics)
- src/backend.rs (the trait + seam contracts this implements)
- src/mem.rs (the contract-reference engine's shapes this mirrors)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion