alkstore/docs/research/poc-sqlite-posture-spec.md

---
status: spec
title: "POC #1 — SQLite engine posture: honker-core-on-rusqlite vs honker-extension-over-sqlx"
last_updated: 2026-10-04
---

# POC: SQLite engine posture — spec

> **POC register #1** (phase-0.md). Feeds OQ-ST-03 (the driver question's
> SQLite half) and OQ-ST-04 (the reactive contract's SQLite wake side).
> Code: standalone crate in the global workspace
> (`/workspace/alkstore-sqlite-posture-poc` — self-contained, per the
> Phase 0 convention; POC code stays out of this repo). Findings land in
> `docs/research/poc-sqlite-posture-findings.md` here regardless.

## What this POC must decide

OQ-ST-03 named three SQLite postures (§OQ-ST-03, option list 1–3). Option
2 (honker-rs as substrate) is **dropped from POC scope**: its blocking
`Database` API and mutex-pinned transaction model would have to sit
entirely under our async core, and both other postures give us the
machinery with more ownership — the POC's comparison between 1 and 3
subsumes the honest parts of 2 (if 3 works and 1 works, 2 is an
intermediate we can revisit; its unique value — the typed binding — is
what the core crate's trait surface supersedes anyway). The POC decides:

**Posture A (library):** honker-core linked on *our* rusqlite connection
— `apply_default_pragmas` + `attach_notify` + `attach_honker_functions`
+ `bootstrap_honker_schema` (the alknet-filesystem POC's usage), sync
work bridged to the async core per the family precedent (alktty
REQ-TTY-01: blocking impl on dedicated threads / `spawn_blocking`,
feeding tokio channels).

**Posture B (extension):** honker's loadable `.so` under **sqlx-sqlite**
— `SqliteConnectOptions::extension(ext)` per pool connection +
`SELECT honker_bootstrap()`, all feature calls as plain SQL through
`SqliteExecutor<'e>` (pool / connection / transaction alike). CI-proven
shape (honker's own `scripts/proof/orm/rust`, checkout @ f4e53c6, PR
#106) — this POC does not re-prove basic wiring; it stress-tests what
the proof script does not cover.

The question is *not* "which is faster at raw SQL" (they call the same
machinery — any SQL-path delta is small); it is **which posture the
sqlite engine crate should be built on**, decided at four load-bearing
points: the async seam, the watcher, the packaging/deployment story, and
the transactional contract.

## The two arms

Both arms implement the *same narrow slice of the alkstore core-crate
trait sketch* (the trait surface is OQ-ST-04's contract question — the
POC sketches just enough to compare postures, not to pin the contract):

- `queue_enqueue_tx` / `queue_claim_batch` / `queue_ack_batch` —
  including the transactional-enqueue property (business write + enqueue
  in one tx, rollback drops both — principle 2).
- `notify_tx(channel, payload)` / `listen(channel)` — the reactive pair.
- `stream_publish_tx` / `read_since(consumer)` (offset save/read only —
  not the full stream contract).
- `lock_acquire` / `lock_release` with TTL, sufficient for alkfs's
  writer-coordination row.

### Arm A — honker-core on our rusqlite (library posture)

- `honker-core = "0.5"` (published crate, per the AGENTS.md posture —
  read the checkout for reference, build against the published version).
- Connection model: one writer connection + reader pool (rusqlite,
  `bundled-sqlite` feature for the POC's hermetic build), bridged to
  async via a small `spawn_blocking` wrapper (measure dedicated-thread +
  mpsc as the alternative, the REQ-TTY-01 double).
- Watcher: honker-core's `SharedUpdateWatcher::spawn_with_config`
  (default 1 ms polling) driving `listen()` wakes; listener re-reads
  after wake (the opaque-wake + re-read contract).

### Arm B — honker extension over sqlx (extension posture)

- Build the extension from the published honker source tree once
  (`cargo build --release -p honker-extension` → `.so`); the POC loads it
  via `HONKER_EXTENSION_PATH` (the guide's wiring; SeaORM/Toasty
  sections are out of scope — sqlx is the only ORM-family shape that
  matters here).
- sqlx 0.8 `runtime-tokio`, per-pool-connection `extension(ext)` +
  `honker_bootstrap()` on connect (the proof script's pattern, applied
  to a `SqlitePool` rather than a single connection — the POC checks the
  *pool* story the script didn't run).
- All honker SQL through `SqliteExecutor<'e>`; async calls natively —
  **no sync bridge on the call path** (this is the posture's load-
  bearing advantage; the POC verifies it survives the full surface, not
  just the proof script's enqueue path).
- Watcher: **ours** — a `spawn_blocking`/std-thread loop polling
  `PRAGMA data_version` on a dedicated sqlx connection (the
  honker-core-watcher logic re-implemented thin over sqlx; no
  honker-core dependency in this arm — that is the point of the
  posture).

## Instruments

1. **Async-seam probe** — the decision's first axis. Arm A: call-path
   latency and executor-neutrality of the `spawn_blocking` bridge at
   realistic claim/enqueue cadence (single-digit-millisecond queue ops
   under WAL-NORMAL); thread cost per connection vs a shared bridge
   pool; `Send`/`Sync` trait-shape friction actually encountered when
   handing rusqlite types across the seam. Arm B: baseline async call
   cost through sqlx (prepared statements, pool checkout) — the honest
   floor any posture pays. Deliverable: the seam-cost table.
2. **Watcher probe** — both arms, the reactive pair end-to-end:
   commit→wake→receiver-notified latency distribution (p50/p99) at the
   default 1 ms cadence; wake correctness under a pool of writers
   (`data_version` observed from a *different* connection than the
   writer's — Arm B's dedicated-watcher-connection shape vs Arm A's
   honker-core watcher); missed-wake stress (N rapid commits, assert
   wake count ≥ 1 and state re-read correct — the overtriggering
   contract, not per-commit wakes); behavior when the watcher
   connection is lost (Arm B) or the watcher thread panics (both).
   Deliverable: the wake-latency table + failure-mode notes.
3. **Transactional/contract probe** — both arms, the property tests:
   business-write + enqueue + notify inside one caller-owned
   transaction; forced rollback drops both (assert no ghost rows in
   `_honker_jobs`/notification tables); concurrent writers (two async
   tasks, each in a transaction, same queue) — Arm A's rusqlite
   single-writer contention behavior vs Arm B's sqlx pool; `SQLITE_BUSY`
   behavior surfaced through each driver (the async error path honker's
   busy_timeout story must survive). Deliverable: property-test matrix
   per arm.
4. **Packaging/deployment probe** — the honest costs, not benchmarks:
   Arm A's build graph (honker-core pulls rusqlite + chrono + feature
   surface; `bundled-sqlite` compile time; version-pinning coupling
   between our rusqlite and honker-core's); Arm B's .so lifecycle
   (build-or-download per release, load path resolution, sqlx's
   load-then-disable extension discipline verified on the pool, macOS
   and system-sqlite caveats recorded but not blockers). Deliverable:
   the packaging-notes table.
5. **Interop check (both arms)** — two processes against one db file:
   writer in one process, listener in another (the cross-process wake
   being the whole point of the substrate). Each arm with its own
   watcher/extension setup on both sides.

## Decision gate

Arm A is chosen iff any of:

- the async seam costs are materially worse under B than A (B's native
  async is the posture's whole premise); or
- the extension/pool wiring (per-connection extension + bootstrap on a
  real pool, lost-connection recovery) fails or proves fragile; or
- the .so packaging burden is judged unacceptable for the engine crate's
  zero-ops posture (REQ-1's shape: no daemon, but also no
  binary-artifact management at runtime).

Arm B is chosen iff:

- the seam costs are materially worse under A (bridge overhead at
  claim/ack cadence, thread-per-connection, trait friction), AND B's
  wiring + watcher story holds under the probes above; or
- sqlx's pool/prepared-statement machinery is measurably the better fit
  for the engine's concurrency posture, and the .so cost is accepted.

The gate may also land **"A for the engine core, B's watcher pattern for
wake plumbing"** or similar hybrid — the probes are designed to
separate the *call-path* question from the *watcher* question, and a
hybrid verdict is a legitimate finding if the evidence slices that way.
Either way the findings feed: OQ-ST-03's resolution (the SQLite driver
posture, the tokio-postgres pg side rides separately), OQ-ST-04's SQLite
wake contract, OQ-ST-06's fork/reference calculus (how much honker-core
machinery each posture actually consumes), and the core-crate trait
sketch's transactional seam (what a transaction handle looks like in
each posture).

## Out of scope

- The Postgres engine (tokio-postgres LISTEN/NOTIFY wiring — after the
  OQ-ST-04 contract shape is pinned; OQ-ST-03 remains coupled to it).
- The full honker surface (scheduler, rate limits, result storage — the
  scheduler may collapse into queues per the inventory; the POC covers
  enough to validate wake + queue + offset machinery).
- The loadable-extension surface as a *consumer-visible* feature
  (OQ-ST-07's cut stands regardless of this POC's outcome — option B
  uses the extension *mechanism* internally but the crate stays an
  in-process Rust library; the .so is never a user-facing artifact).
- Performance optimization beyond what the probes need to produce
  comparable numbers — both arms call the same SQL machinery, so this
  POC is not a bench-fest.

## Register note

This POC is **#1** in alkstore's register (phase-0.md), specified
2026-10-04. It is deliberately the first POC: it de-risks the SQLite
side of OQ-ST-03 (the driver question the whole crate's shape leans on)
with a direct A-vs-B comparison, and its trait-sketch exercise doubles
as the first concrete pass at OQ-ST-04's contract pinning.