--- id: suite-column-sqlite name: Conformance suite — the sqlite column (the first SQL-backed column) status: completed depends_on: [store-sweep-durable-window] scope: moderate risk: medium impact: phase level: implementation tags: [wave-3, suite, sqlite] --- ## Description Grow the **sqlite column** — the suite's first SQL-backed column, server-less (never skips; README §SKIP posture: server-less engines' columns must always be green in every environment). The column runs in the default build (feature `sqlite` is default-on). - **Factories** (`tests/conformance/sqlite_column.rs`): - dual-tier `kv = sqlite + large = local` at a column-declared threshold and the **test tuning of the window delay (`0` — execute-on-next-sweep)**; - `kv-only(sqlite)` at a column-declared threshold (the rejection row's host; threshold materializable per the README's configuration contract). Fresh isolated sqlite file per `open()` (temp file), idempotent teardown (remove the file), panicking-row-safe — the local column's instance-tracks-its-own-root pattern. - **Mirror the form-independent reference rows** (README §The reference row set): INV-1 round-trips (content matrix × both put paths, in-body), INV-2 dedup (same-path + cross-path legs), INV-3 stage hygiene (the degenerate kv stage form via the injectable failure modes — mid-stream `Io`, declared-len `Verification`), INV-6 stat discipline, and the kv-only rejection row (threshold-agnostic by discovery, unchanged). - **The sweep rows are schedule-relative on this column.** The reference set's single-sweep exact-count rows are the inline-form host's (mem); the durable form's semantics differ (staged this sweep, executed by the next — ADR-012 §6.7). The sqlite column grows its own **durable-schedule rows** (concrete-factory per README §Columns beyond the reference set, ADR-stamped, traced): - exact counts across two sweeps (sweep 1 stages N, deletes nothing; sweep 2 executes exactly N — `deleted + cancelled == N` against the staged set) over the full protection set; - the sweep-vs-put race across the window delay (a put landing during the window survives the executing sweep; the unpinned twin deletes); - the abort shapes (`NoLivenessSources`/`ProtectFailed`/ `SweeperLock` — form-independent, abort before staging; these mirror directly); - INV-5 list correctness through durable-schedule outcomes (exact counts, never list comparison). - **The dual-tier dispatch rows** mirror the local column's family (pure-function routing, threshold boundaries, fall-through per op) with the placement oracle adapted: both tiers are on disk now — kv entries live in the sqlite file, large entries under the local root; the oracle distinguishes by medium (the sqlite file vs the root's committed files). - **Virgin-store rows**: reads are no-ops on a fresh file; the no-tables-yet tolerance form. - **README amendments** (the convention's normative test-side home): - the **schedule-relative reading of INV-4/INV-5 for SQL-backed columns** — the durable form's exact counts are asserted relative to the staged schedule (ADR-012 §6.7); the reference set's single-sweep rows are the inline-form host's, and SQL-backed columns run the durable-schedule equivalents (the mirror convention's recorded amendment); - the column inventory (§Columns beyond the reference set grows the sqlite rows table); - current state (three columns; the default-build column). - **Structural factory tests** (the mem column's pattern): factory isolation (fresh file per open, nothing shared) and teardown-against-a-panicking-row. ## Acceptance Criteria - [ ] The sqlite column green server-less: the mirrored form-independent reference rows + the durable-schedule sweep rows + the dispatch rows + the virgin-store rows, every row ADR-stamped and traceable to the backlog - [ ] The schedule-relative INV-4/INV-5 reading recorded in README (the mirror convention's amendment, citing ADR-012 §6.7) - [ ] Factory isolation + idempotent teardown hold (fresh file per open; teardown safe against a panicking row), structurally exercised - [ ] `cargo test` (default) green — the column runs in the default build; `cargo clippy --all-targets -- -D warnings` and `cargo fmt --check` green ## References - tests/conformance/README.md (the conventions this grows against; §SKIP posture, §Columns beyond the reference set) - tests/conformance/rows.rs (the reference row set being mirrored) - tests/conformance/local_column.rs (the dual-tier column pattern) - tests/conformance/backlog.rs (INV-1…7 entries) - docs/architecture/store-api.md (§Invariants — the test gate) - docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §6.7 (the schedule the sweep rows assert against) - docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md §2 (mem's reference role; INV-7) ## Notes The column as it landed (`tests/conformance/sqlite_column.rs`, 17 tests): - **Factories**: `dual-tier sqlite+local` (the default-build production shape, `DualTierSqliteFactory`) and `kv-only(sqlite)` (`KvOnlySqliteFactory`), each instance-tracking the temp sites it vends (a private temp directory per open holding `pool.db` and the local root; teardown `remove_dir_all`s it — idempotent, NotFound tolerated, panicking-row-safe, one runner per instance), with the column-declared threshold (1024) and the **test tuning of the window delay (`Duration::ZERO` — execute-on-next-sweep)**. The module is `#[cfg(feature = "sqlite")]` — **default-on**, so the column runs in every default `cargo test`. - **The medium oracle distinguishes by medium**: the dual-tier rows count the sqlite file's `blobs` rows (the kv tier) vs the local root's committed files (the large tier), plus `gc_window` rows (the durable schedule observer) — via the test's own independent connection. The shared helpers (`committed_file_count`, `staging_file_count`, `slice_digest`, `LatchProtect`) moved to `rows.rs` (`pub(crate)`, the columns' shared fixtures) so the sqlite column reuses them without the mem gate. - **Mirrored verbatim** (form-independent, from `rows.rs`): INV-1 round-trips, INV-6 stat discipline, the `NoLivenessSources`/ `ProtectFailed` abort shapes, and the kv-only rejection row (against `kv-only(sqlite)`; unchanged, threshold-agnostic by discovery). - **Durable-schedule rows** (the column's own, concrete-factory, ADR-stamped, traced): INV-2 dedup and INV-3 stage hygiene with counts read against the staged schedule (the sweep-counting rows' amendment), the two-sweep exact-count row over the full protection set (both tiers, the window observed), the race across the window delay (a re-put landing during the window survives the executing sweep via execution-time re-arbitration; the unpinned twin deletes; both put paths in-body over both tiers), the holder-stages-only lease shape (the reference `SweeperLock` row's schedule-relative form), and INV-5 list correctness through durable-schedule outcomes. - **Dispatch rows** mirror the local column's family (pure-function routing, threshold boundaries at the tuned + shipped-default thresholds, fall-through per op) with the placement oracle adapted by medium. **Virgin-store row**: reads are no-ops on a fresh file (pool, window, and committed tree all empty); the no-tables-yet tolerance itself is engine-instrumented (src/sqlite.rs — the constructor always ensures the schema), this row being the facade form. **Structural factory tests**: fresh isolated opens (distinct private sqlite files and roots, medium-observable) and teardown-against-a-panicking-row (files really removed). - **README amendments**: §The schedule-relative reading for SQL-backed columns (the mirror convention's amendment, citing ADR-012 §6.7 — staged-this-sweep / executed-next-sweep, which rows carry it, which mirror verbatim), the sqlite rows table in §Columns beyond the reference set, and Current state (three columns; the default-build column). main.rs's module doc names the column. ## Summary The sqlite column — the suite's first SQL-backed column — is green: the default-build production shape (`kv = sqlite` + `large = local`, feature `sqlite` default-on) runs the form-independent reference rows verbatim against its factories, grows the durable-schedule sweep rows whose exact counts are asserted relative to the staged schedule (ADR-012 §6.7: staged this sweep, executed by the next — two-sweep counts over the full protection set, the race across the window delay, the holder-stages-only lease shape, INV-5 through outcomes, and the schedule-relative INV-2/INV-3 readings), and the dispatch rows with the medium oracle distinguishing the sqlite file's rows from the root's committed files. Every row ADR-stamped and traced; the schedule-relative INV-4/INV-5 reading is recorded in the README (the mirror convention's amendment); factory isolation (fresh site per open) and idempotent panicking-row-safe teardown hold structurally. The column never skips and runs in the default build (17 tests). Verification: `cargo test` green (default: 69 lib + 19 conformance + 3 git-interop), `cargo test --features mem` green (124 lib + 54 conformance), `cargo check --all-features --tests` green, `cargo clippy --all-targets -- -D warnings` green, `cargo fmt --check` green (fmt applied).