alkstore/docs/architecture/engine-sqlite.md

status, last_updated

status	last_updated
draft	2026-10-06

SQLite engine

The alkstore-sqlite engine implements core-contract.md on rusqlite over the forked honker-core substrate — carried in-tree as the engine crate's substrate module subtree (ADR-013). This spec records WHAT the engine is internally (its connection architecture, seam, and wake plumbing) — not code-level HOW. Decisions live in ADRs; per-contract obligations live in the core spec and are not restated here.

Identity and posture

• Single driver: rusqlite, riding the forked honker-core substrate — the forked lineage machinery carried in-tree as this crate's src/substrate/ module subtree (ADR-012 design, ADR-013 packaging) — with bundled-sqlite for hermetic builds. No .so runtime artifacts (ADR-003 for the driver decision; ownership resolved by ADR-011 — OQ-06's read fired ADR-005's fork trigger).
• Sync machinery, async-facing trait: honker-core is sync (std threads, blocking iterators); the engine bridges at the trait seam per the family-standard posture (alktty REQ-TTY-01 precedent). Every engine call runs in spawn_blocking.
• Single-host by nature: file-backed, one machine, NFS-two-writers unsupported (honker's honesty posture, inherited). See deployment.md; the boundary's surface location is decided (ADR-016) — this crate's identity and docs state the posture; no runtime descriptor exists.

Connection architecture

• Writer — one dedicated connection; the only one permitted to write. Serializes all mutations (WAL single-writer, modeled honestly, not fought).
• Readers — a small pool of read connections for lookups and claim/ack work.
• Watcher — honker-core's SharedUpdateWatcher: a dedicated thread polling PRAGMA data_version at the default 1 ms cadence, fanning out to listeners, overtriggering on purpose (waking all subscribers per poll tick, even when several commits coalesced inside one tick — wake is a hint; consumers re-read indexed state, ADR-006).
• Each connection runs the substrate's bootstrap at open: pragmas (WAL, synchronous=NORMAL, busy timeout), attach_notify, the substrate's function attachments and schema bootstrap (the alknet-filesystem POC's wiring shape). Watcher spawn is fallible in the substrate (ADR-012 §4); open fails if the watcher cannot start.

Mapping the contract

Contract piece	Engine realization
notify / listen	honker's notify functions inside the caller's tx; listen() bridges the watcher's fanout into a tokio receiver (one spawn_blocking thread per subscription doing blocking_send)
streams	the forked substrate's stream machinery (inherited near-verbatim — the __alkstore_stream table's field set is already the contract's shape (its topic column carries the contract's stream field — the one name delta, ADR-012 §3's fidelity posture), the offset-ASC read path, keyed publish's nullable key column, monotone offset saves (ADR-015); explicit offset saves through the tx seam; trim_to is a DELETE FROM … WHERE offset <= ? inside the writer-slot lease)
queues	owned queue machinery (forked substrate re-derived on ADR-010 — stamps, no-stranded-rows sweep, dead-visible get_job); ADR-003's ride posture superseded on ownership by ADR-011
named locks	the substrate's lock machinery (lock_renew carries the renew semantics the guarantee row pins; re-acquire does not refresh TTL — inherited deliberately, ADR-012 §4)
scheduler / outbox	collapse shape (ADR-009): owned scheduler storage (__alkstore_scheduler_tasks, forked substrate) carrying @every specs (cron machinery not ported); the leader loop pattern (TTL lock, heartbeat, exit-before-tick-on-loss); the leadership lock is __alkstore_scheduler; outbox = helper over queues with the derived backing-queue name (ADR-008 §4)
begin_tx	acquires the writer slot, opens BEGIN IMMEDIATE, returns the caller-held handle (ADR-007)
handle ops	each *_tx op round-trips spawn_blocking to the same connection (thread-affinity note in ADR-007)
commit/rollback	releases the writer slot

Obligations and constraints

• A long transaction parks the writer (the slot lease is the honest model). Contract docs must surface this so consumers budget transactions (ADR-007 negative consequence).
• Watcher failure handling is inherited: on watcher death, every subscriber's receiver closes (WatcherDeathGuard behavior) — consumers see the close, never a silent hang (ADR-006).
• Wake coalescing: bursts inside one poll tick produce one wake; correctness is preserved by the re-read contract (POC-pinned: missed-wake stress with correct post-burst re-reads).
• Deployment note: the lineage's rusqlite generation (^0.40.1) — this engine crate's own dependency — has a rustc floor ≥ 1.99; binaries linking this engine carry that requirement (deployment.md matrix).
• The optimization path, if seam throughput ever demands it: a dedicated std-thread bridge (one thread owning the writer conn, ops over mpsc — measured ~2× the spawn_blocking shape at p50 in POC #1), or a raised watcher cadence for idle CPU. Neither is the default.

What the fork question resolved

The quality read (OQ-06) — this engine's dependency gate — resolved 2026-10-05: ADR-005's fork trigger fired (ADR-011), and the fork's design is pinned by ADR-012: the substrate is a vendored contract-blind lineage module folded into this engine crate (ADR-013); the watcher/connection architecture this spec describes is inherited verbatim where kept (ADR-003 unchanged in architecture), with the port deltas ADR-012 §4 decides (fallible watcher spawn, bounded reconnect backoff, panic-free death-closes-subscribers). The evidence: published 0.5.0's unreleased-fix-train defect class, plus ADR-010's queue depth requiring engine-owned queue SQL in any posture. The substrate's table family is __alkstore_* (ADR-010 §8's naming authorization).

Design Decisions

ADR	Decision	Summary
001	Crate split	single-driver engine crate
002	Feature scope	which rows this engine serves
003	Driver	rusqlite + the forked substrate lineage, bridged seam, inherited watcher — ownership resolved by 011
005	Ownership	published honker-core; fork trigger fired by OQ-06 (ADR-011)
006	Wake contract	data_version watcher, coalescing, death-closes-receivers
007	Tx seam	writer-slot lease, BEGIN IMMEDIATE, spawn_blocking round-trips
008	Contract v1	pinned surface; outbox backing queue derived under the reserved prefix; wake payload transport unused (non-contract)
009	Scheduler collapse	@every specs; __alkstore_scheduler leadership; cron rejected
010	Queue depth	semantics pinned; §5/§3a realization resolved by ADR-011 (owned code)
011	Substrate fork	OQ-06's trigger fired; substrate owned, queue ops re-derived on contract v1
012	Fork design	contract-blind substrate, fidelity posture, port deltas, panic-free watcher death
013	Substrate packaging	folded into alkstore-sqlite (src/substrate/); no fourth crate
014	Outbox tx enqueue	outbox_enqueue_tx on TxHandle; derivation engine-side through the writer-slot lease
015	Streams depth	key = carried metadata (the inherited nullable column); global-FIFO ordering; trim_to as a writer-lease delete; event shape already the substrate's
016	Deployment honesty	no runtime capability surface; single-host posture stated by crate identity + docs; this engine never produces PayloadTooLarge (pinned in the contract suite)

Open Questions

Open questions are tracked in open-questions.md. Key questions affecting this document:

• OQ-06: honker-core quality read — fork-trigger assessment — resolved (2026-10-05, ADR-011; trigger fired).
• OQ-12: streams depth — resolved (2026-10-05, ADR-015): key = carried metadata; global-FIFO-by-offset ordering row; StreamEvent shape (honker's, topic → stream in the contract type — the substrate's column name stays per the ADR-012 §3 fidelity posture); publish_with_key_tx routes through the writer-slot lease; trim_to is a plain delete in the same lease.
• OQ-13: transactional outbox enqueue shape — resolved (2026-10-05, ADR-014): outbox_enqueue_tx(outbox, opts, payload) on the TxHandle trait; the SQLite handle routes the op through the writer-slot lease (ADR-007) into the forked substrate's __alkstore_outbox:{name} backing queue.

Resolved: OQ-09 (scheduler collapse — ADR-009), OQ-05 (queue semantics depth — ADR-010), OQ-12 (streams depth — ADR-015), and OQ-08 (capability surface — none, by default ever; ADR-016), 2026-10-05/06.