--- status: spec title: "POC #1 — SQLite engine posture: honker-core-on-rusqlite vs honker-extension-over-sqlx" last_updated: 2026-10-04 --- # POC: SQLite engine posture — spec > **POC register #1** (phase-0.md). Feeds OQ-ST-03 (the driver question's > SQLite half) and OQ-ST-04 (the reactive contract's SQLite wake side). > Code: standalone crate in the global workspace > (`/workspace/alkstore-sqlite-posture-poc` — self-contained, per the > Phase 0 convention; POC code stays out of this repo). Findings land in > `docs/research/poc-sqlite-posture-findings.md` here regardless. ## What this POC must decide OQ-ST-03 named three SQLite postures (§OQ-ST-03, option list 1–3). Option 2 (honker-rs as substrate) is **dropped from POC scope**: its blocking `Database` API and mutex-pinned transaction model would have to sit entirely under our async core, and both other postures give us the machinery with more ownership — the POC's comparison between 1 and 3 subsumes the honest parts of 2 (if 3 works and 1 works, 2 is an intermediate we can revisit; its unique value — the typed binding — is what the core crate's trait surface supersedes anyway). The POC decides: **Posture A (library):** honker-core linked on *our* rusqlite connection — `apply_default_pragmas` + `attach_notify` + `attach_honker_functions` + `bootstrap_honker_schema` (the alknet-filesystem POC's usage), sync work bridged to the async core per the family precedent (alktty REQ-TTY-01: blocking impl on dedicated threads / `spawn_blocking`, feeding tokio channels). **Posture B (extension):** honker's loadable `.so` under **sqlx-sqlite** — `SqliteConnectOptions::extension(ext)` per pool connection + `SELECT honker_bootstrap()`, all feature calls as plain SQL through `SqliteExecutor<'e>` (pool / connection / transaction alike). CI-proven shape (honker's own `scripts/proof/orm/rust`, checkout @ f4e53c6, PR #106) — this POC does not re-prove basic wiring; it stress-tests what the proof script does not cover. The question is *not* "which is faster at raw SQL" (they call the same machinery — any SQL-path delta is small); it is **which posture the sqlite engine crate should be built on**, decided at four load-bearing points: the async seam, the watcher, the packaging/deployment story, and the transactional contract. ## The two arms Both arms implement the *same narrow slice of the alkstore core-crate trait sketch* (the trait surface is OQ-ST-04's contract question — the POC sketches just enough to compare postures, not to pin the contract): - `queue_enqueue_tx` / `queue_claim_batch` / `queue_ack_batch` — including the transactional-enqueue property (business write + enqueue in one tx, rollback drops both — principle 2). - `notify_tx(channel, payload)` / `listen(channel)` — the reactive pair. - `stream_publish_tx` / `read_since(consumer)` (offset save/read only — not the full stream contract). - `lock_acquire` / `lock_release` with TTL, sufficient for alkfs's writer-coordination row. ### Arm A — honker-core on our rusqlite (library posture) - `honker-core = "0.5"` (published crate, per the AGENTS.md posture — read the checkout for reference, build against the published version). - Connection model: one writer connection + reader pool (rusqlite, `bundled-sqlite` feature for the POC's hermetic build), bridged to async via a small `spawn_blocking` wrapper (measure dedicated-thread + mpsc as the alternative, the REQ-TTY-01 double). - Watcher: honker-core's `SharedUpdateWatcher::spawn_with_config` (default 1 ms polling) driving `listen()` wakes; listener re-reads after wake (the opaque-wake + re-read contract). ### Arm B — honker extension over sqlx (extension posture) - Build the extension from the published honker source tree once (`cargo build --release -p honker-extension` → `.so`); the POC loads it via `HONKER_EXTENSION_PATH` (the guide's wiring; SeaORM/Toasty sections are out of scope — sqlx is the only ORM-family shape that matters here). - sqlx 0.8 `runtime-tokio`, per-pool-connection `extension(ext)` + `honker_bootstrap()` on connect (the proof script's pattern, applied to a `SqlitePool` rather than a single connection — the POC checks the *pool* story the script didn't run). - All honker SQL through `SqliteExecutor<'e>`; async calls natively — **no sync bridge on the call path** (this is the posture's load- bearing advantage; the POC verifies it survives the full surface, not just the proof script's enqueue path). - Watcher: **ours** — a `spawn_blocking`/std-thread loop polling `PRAGMA data_version` on a dedicated sqlx connection (the honker-core-watcher logic re-implemented thin over sqlx; no honker-core dependency in this arm — that is the point of the posture). ## Instruments 1. **Async-seam probe** — the decision's first axis. Arm A: call-path latency and executor-neutrality of the `spawn_blocking` bridge at realistic claim/enqueue cadence (single-digit-millisecond queue ops under WAL-NORMAL); thread cost per connection vs a shared bridge pool; `Send`/`Sync` trait-shape friction actually encountered when handing rusqlite types across the seam. Arm B: baseline async call cost through sqlx (prepared statements, pool checkout) — the honest floor any posture pays. Deliverable: the seam-cost table. 2. **Watcher probe** — both arms, the reactive pair end-to-end: commit→wake→receiver-notified latency distribution (p50/p99) at the default 1 ms cadence; wake correctness under a pool of writers (`data_version` observed from a *different* connection than the writer's — Arm B's dedicated-watcher-connection shape vs Arm A's honker-core watcher); missed-wake stress (N rapid commits, assert wake count ≥ 1 and state re-read correct — the overtriggering contract, not per-commit wakes); behavior when the watcher connection is lost (Arm B) or the watcher thread panics (both). Deliverable: the wake-latency table + failure-mode notes. 3. **Transactional/contract probe** — both arms, the property tests: business-write + enqueue + notify inside one caller-owned transaction; forced rollback drops both (assert no ghost rows in `_honker_jobs`/notification tables); concurrent writers (two async tasks, each in a transaction, same queue) — Arm A's rusqlite single-writer contention behavior vs Arm B's sqlx pool; `SQLITE_BUSY` behavior surfaced through each driver (the async error path honker's busy_timeout story must survive). Deliverable: property-test matrix per arm. 4. **Packaging/deployment probe** — the honest costs, not benchmarks: Arm A's build graph (honker-core pulls rusqlite + chrono + feature surface; `bundled-sqlite` compile time; version-pinning coupling between our rusqlite and honker-core's); Arm B's .so lifecycle (build-or-download per release, load path resolution, sqlx's load-then-disable extension discipline verified on the pool, macOS and system-sqlite caveats recorded but not blockers). Deliverable: the packaging-notes table. 5. **Interop check (both arms)** — two processes against one db file: writer in one process, listener in another (the cross-process wake being the whole point of the substrate). Each arm with its own watcher/extension setup on both sides. ## Decision gate Arm A is chosen iff any of: - the async seam costs are materially worse under B than A (B's native async is the posture's whole premise); or - the extension/pool wiring (per-connection extension + bootstrap on a real pool, lost-connection recovery) fails or proves fragile; or - the .so packaging burden is judged unacceptable for the engine crate's zero-ops posture (REQ-1's shape: no daemon, but also no binary-artifact management at runtime). Arm B is chosen iff: - the seam costs are materially worse under A (bridge overhead at claim/ack cadence, thread-per-connection, trait friction), AND B's wiring + watcher story holds under the probes above; or - sqlx's pool/prepared-statement machinery is measurably the better fit for the engine's concurrency posture, and the .so cost is accepted. The gate may also land **"A for the engine core, B's watcher pattern for wake plumbing"** or similar hybrid — the probes are designed to separate the *call-path* question from the *watcher* question, and a hybrid verdict is a legitimate finding if the evidence slices that way. Either way the findings feed: OQ-ST-03's resolution (the SQLite driver posture, the tokio-postgres pg side rides separately), OQ-ST-04's SQLite wake contract, OQ-ST-06's fork/reference calculus (how much honker-core machinery each posture actually consumes), and the core-crate trait sketch's transactional seam (what a transaction handle looks like in each posture). ## Out of scope - The Postgres engine (tokio-postgres LISTEN/NOTIFY wiring — after the OQ-ST-04 contract shape is pinned; OQ-ST-03 remains coupled to it). - The full honker surface (scheduler, rate limits, result storage — the scheduler may collapse into queues per the inventory; the POC covers enough to validate wake + queue + offset machinery). - The loadable-extension surface as a *consumer-visible* feature (OQ-ST-07's cut stands regardless of this POC's outcome — option B uses the extension *mechanism* internally but the crate stays an in-process Rust library; the .so is never a user-facing artifact). - Performance optimization beyond what the probes need to produce comparable numbers — both arms call the same SQL machinery, so this POC is not a bench-fest. ## Register note This POC is **#1** in alkstore's register (phase-0.md), specified 2026-10-04. It is deliberately the first POC: it de-risks the SQLite side of OQ-ST-03 (the driver question the whole crate's shape leans on) with a direct A-vs-B comparison, and its trait-sketch exercise doubles as the first concrete pass at OQ-ST-04's contract pinning.