alkblobs/tasks/engine-sqlite.md

---
id: engine-sqlite
name: sqlite kv engine — the default engine
status: completed
depends_on: []
scope: narrow
risk: medium
impact: component
level: implementation
tags: [wave-3, engines, sqlite]
---

## Description

Implement the `sqlite` kv-tier engine (feature `sqlite`, **default-on**;
ADR-007; REQ-1) against the wave-1 trait (`src/backend.rs`) — the first
SQL-backed engine and the default build's kv engine.

- **Backend impl** (`src/sqlite.rs`, behind `#[cfg(feature = "sqlite")]`):
  the seven methods plus `put_staged`, opaque byte keys/values,
  `name() -> "sqlite"`.
  - **Schema** — the POC #5 finding B5 row shape adapted to sqlite:
    `blobs (key BLOB PRIMARY KEY, value BLOB, size INTEGER NOT NULL)`.
    The dedicated `size` column is what makes the **indexed `size`
    column probe** metadata-only: `SELECT size … WHERE key = ?` rides
    the key's index and never loads the value blob (ADR-008's size
    discipline).
  - **WAL posture** — `journal_mode=WAL` + `synchronous=NORMAL` set at
    open: the shipped durability tier (ADR-003; the posture honker
    ships and iroh's store measured). The **single-writer ceiling is a
    documented posture, not a defect** (REQ-1's economics — the module
    doc records it; ~1.2k objects/s under contention is an order above
    single-node push rates).
  - **Prepared statements** — rusqlite's `prepare_cached` discipline
    per the backends spec's "prepared statements" requirement.
  - `has` — an **EXISTS probe** (`SELECT EXISTS(…)`) — never a value
    load.
  - `get` — the **materializing cursor body** (kv family): the whole
    value loaded once (bounded by tier policy — the kv tier holds only
    ≤ threshold content), `len()` from the row's size column, reads
    slice the materialized value composing `backend.rs`'s `clamped`.
    This body is also what the facade's verification-on-get re-derives
    over (store.rs's kv design point — no engine change needed).
  - `put` — CAS first-writer-wins (`INSERT OR IGNORE`); a re-put of an
    existing key is a no-op, never an overwrite.
  - `put_staged` — the **degenerate in-memory stage** (ADR-010 §1: kv
    bodies stage in memory and commit through the whole-value row
    write); `append` extends the buffer, `commit` publishes via the
    same `INSERT OR IGNORE` clause and self-discards on failure,
    `discard` drops the buffer with zero residue.
  - `delete` — `Err(Error::Missing)` when absent (rows-affected
    discipline); `list` — complete and well-formed (`SELECT key`);
    `size` — the indexed column probe above.
  - **Virgin-store no-tables-yet tolerance** — read paths treat a
    missing table as absent/empty, never "table does not exist" errors
    (POC #1 finding 2; store-api.md §Error model's virgin-store
    semantics). Schema creation timing (constructor `CREATE TABLE IF
    NOT EXISTS` vs lazy) is a recorded design point; the tolerance is
    unit-tested against a tableless database file either way.
- **Blocking discipline**: rusqlite is synchronous — **all engine I/O
  runs in `spawn_blocking`** (AGENTS convention 3; store-api.md
  §Concurrency posture). The connection posture (the expected shape: a
  single connection behind a short-held lock, the single-writer
  posture; a pool is not required solo) is a recorded design point.
  No blocking I/O on the async path; no `unwrap`/`expect` outside
  tests.
- **Unit tests** in-module: the method contract (virgin-store no-ops
  incl. the no-tables-yet form, CAS re-put, Missing discipline, size
  exactness + metadata-only), the staged-put happy path and both
  hygiene forms (discard → zero residue; commit failure →
  self-discard — a `#[cfg(test)]` fail-commit knob standing in for
  media failure, the mem engine's pattern), the pragma posture.

The engine-state window family (`EngineWindowState`) is
`engine-sqlite-window` (separate task); the constructor wiring is
`store-sqlite-construction`. This task ships the engine and its
in-module unit tests only.

## Acceptance Criteria

- [ ] `src/sqlite.rs` implements `Backend` completely (seven methods +
      `put_staged`) behind feature `sqlite` (default-on), with the
      materializing cursor body and the degenerate staging body
- [ ] WAL + `synchronous=NORMAL` set at open; prepared-statement
      discipline; `has` is an EXISTS probe; the single-writer ceiling
      recorded as posture in the module doc
- [ ] `size` reads the dedicated size column via the key index, never
      the value blob (metadata-only, unit-tested)
- [ ] CAS first-writer-wins on both put forms; re-put never overwrites
- [ ] Stage hygiene holds in both forms (discard → zero residue;
      failed commit → self-discard), unit-tested
- [ ] Virgin-store reads are no-ops including the no-tables-yet form
      (unit-tested against a tableless database file)
- [ ] All engine I/O in `spawn_blocking`; connection posture recorded
- [ ] `cargo test` (default — sqlite is default-on) green;
      `cargo clippy --all-targets -- -D warnings` and
      `cargo fmt --check` green

## References

- docs/architecture/backends-and-dispatch.md (§kv tier — sqlite engine)
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
- docs/architecture/decisions/003-backend-contract-and-dispatch.md (the
  shipped durability tier; the three numbered put rules' kv side)
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state.md §1
- docs/architecture/requirements.md (REQ-1's economics)
- docs/research/poc-postgres-kv-findings.md (finding B5's row shape)
- docs/research/poc-largeblob-findings.md (finding A4's economics)
- src/backend.rs (the trait + seam contracts this implements)
- src/mem.rs (the contract-reference engine's shapes this mirrors)

## Notes

Recorded design points landed in the module doc (`src/sqlite.rs`):

- **Schema-creation timing:** the constructor runs `CREATE TABLE IF
  NOT EXISTS` at open. The virgin-store tolerance itself is an
  engine-level read-path property independent of that timing —
  unit-tested against a genuinely tableless database file via a
  `#[cfg(test)]` `new_tableless` constructor. The tolerance is
  deliberately read-only: a write against a tableless database surfaces
  the engine error rather than silently creating schema (tested).
- **WAL engagement is verified at open, not assumed:** the constructor
  probes `PRAGMA journal_mode = WAL` and errors when WAL cannot be
  engaged (in-memory databases included — tested). The pragma posture
  (WAL persistent + synchronous NORMAL per-connection) is asserted in
  the pragma test on the backend's own single connection.
- **Error mapping:** the no-tables-yet discriminant is a stringly check
  on rusqlite's Display ("no such table") — stringly by nature, the
  stable surface sqlite itself guarantees. `delete` maps it to
  `Error::Missing` (rows-affected discipline holds for the virgin
  form); `put` propagates it as `Error::Io`.
- **size metadata-only proof:** exactness is asserted for large/absent
  keys and `EXPLAIN QUERY PLAN` asserts the probe searches the key's
  index (`USING INDEX`) — never a scan or value load.
- Connection posture: one connection behind a short-held
  `parking_lot::Mutex`, shared with staging handouts via `Arc` (a
  staging commit must see the same table); the engine `Clone`s. All
  rusqlite I/O in `spawn_blocking` via a local `blocking` helper (the
  local.rs pattern); cursor/staging read bodies are in-memory after
  materialization and need no blocking.
- `EXPLAIN QUERY PLAN` calls must bind the `?1` parameter even though
  they do not execute — sqlite enforces the bind count at step time.

## Summary

The `sqlite` kv engine shipped complete behind feature `sqlite`
(default-on): the row shape `blobs (key BLOB PRIMARY KEY, value BLOB,
size INTEGER NOT NULL)` (POC #5 B5 adapted), `INSERT OR IGNORE` CAS
first-writer-wins on both put forms, the EXISTS `has` probe, the
materializing cursor body with `len()` from the size column, the
degenerate in-memory stage committed through the whole-value row write
with both hygiene forms, the indexed-column `size` probe, complete
`list`, and the no-tables-yet read tolerance. WAL + synchronous=NORMAL
set and verified at open; `prepare_cached` discipline throughout; every
rusqlite operation in `spawn_blocking`; single-writer posture recorded
in the module doc. Eleven in-module tests green (method contract, list
completeness, cursor contract, empty entry, staged round-trip incl.
both dedup forms, both hygiene forms, pragma posture, WAL-engagement
rejection, size/index plan, tableless virgin store). `cargo test` 58
pass, `cargo clippy --all-targets -- -D warnings` clean, `cargo fmt
--check` clean, `cargo check --no-default-features` green. Engine-state
window family and store construction remain in their own tasks
(`engine-sqlite-window`, `store-sqlite-construction`).