glm-5.3-flash
2949612e2c
Follow-through on OQ-06/ADR-011: pin the fork's structural decisions (alkstore-substrate as a vendored path-dep crate, contract-blind API boundary with contract formulas computed engine-side and pinned equivalent by the contract suite, keep-the-kept-half API fidelity for cheap cherry-picks, the W-1/W-2/dead-man's-switch/W-4 port deltas decided per item, bootstrap re-keying off error-string matching, no rename migration, deliberate upstream tracking). Consistency sweep across the doc set for the fork: annotate ADR-003/ 005/009/010 and core-contract for superseded ownership facts, fix schedule-storage table naming (ADR-009 §5, queues.md), re-key ADR-010 §6's notifications hygiene to the at-attach cap the fork scope realizes, add OQ-11 (scaffold-time residue), and complete both ADR indexes. Independent review: 0 critical, warnings addressed.
136 lines
8.0 KiB
Markdown
136 lines
8.0 KiB
Markdown
---
|
||
status: draft
|
||
last_updated: 2026-10-05
|
||
---
|
||
|
||
# SQLite engine
|
||
|
||
The `alkstore-sqlite` engine implements
|
||
[core-contract.md](core-contract.md) on rusqlite over the
|
||
[forked honker-core substrate](decisions/011-sqlite-substrate-fork.md).
|
||
This spec records WHAT the engine is internally (its
|
||
connection architecture, seam, and wake plumbing) — not code-level
|
||
HOW. Decisions live in ADRs; per-contract obligations live in the core
|
||
spec and are not restated here.
|
||
|
||
## Identity and posture
|
||
|
||
- Single driver: rusqlite, riding the
|
||
[forked honker-core substrate](decisions/011-sqlite-substrate-fork.md)
|
||
(`alkstore-substrate` — the vendored lineage crate,
|
||
[ADR-012](decisions/012-forked-substrate-design.md)) with
|
||
`bundled-sqlite` for hermetic builds. No `.so` runtime artifacts
|
||
([ADR-003](decisions/003-sqlite-driver.md) for the driver decision;
|
||
ownership resolved by
|
||
[ADR-011](decisions/011-sqlite-substrate-fork.md) — OQ-06's read
|
||
fired ADR-005's fork trigger).
|
||
- Sync machinery, async-facing trait: honker-core is sync (std
|
||
threads, blocking iterators); the engine bridges at the trait seam
|
||
per the family-standard posture (alktty REQ-TTY-01 precedent).
|
||
Every engine call runs in `spawn_blocking`.
|
||
- Single-host by nature: file-backed, one machine, NFS-two-writers
|
||
unsupported (honker's honesty posture, inherited). See
|
||
[deployment.md](deployment.md).
|
||
|
||
## Connection architecture
|
||
|
||
- **Writer** — one dedicated connection; the only one permitted to
|
||
write. Serializes all mutations (WAL single-writer, modeled
|
||
honestly, not fought).
|
||
- **Readers** — a small pool of read connections for lookups and
|
||
claim/ack work.
|
||
- **Watcher** — honker-core's `SharedUpdateWatcher`: a dedicated
|
||
thread polling `PRAGMA data_version` at the default 1 ms cadence,
|
||
fanning out to listeners, overtriggering on purpose (waking all
|
||
subscribers per poll tick, even when several commits coalesced
|
||
inside one tick — wake is a hint; consumers re-read indexed state,
|
||
[ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||
- Each connection runs the substrate's bootstrap at open: pragmas
|
||
(WAL, `synchronous=NORMAL`, busy timeout), `attach_notify`,
|
||
the substrate's function attachments and schema bootstrap (the
|
||
alknet-filesystem POC's wiring shape). Watcher spawn is fallible in
|
||
the substrate ([ADR-012](decisions/012-forked-substrate-design.md)
|
||
§4); open fails if the watcher cannot start.
|
||
|
||
## Mapping the contract
|
||
|
||
| Contract piece | Engine realization |
|
||
|---|---|
|
||
| notify / listen | honker's notify functions inside the caller's tx; `listen()` bridges the watcher's fanout into a tokio receiver (one `spawn_blocking` thread per subscription doing `blocking_send`) |
|
||
| streams | honker's stream machinery; explicit offset saves through the tx seam |
|
||
| queues | owned queue machinery (forked substrate re-derived on [ADR-010](decisions/010-queue-semantics-depth.md) — stamps, no-stranded-rows sweep, dead-visible `get_job`); ADR-003's ride posture superseded on ownership by [ADR-011](decisions/011-sqlite-substrate-fork.md) |
|
||
| named locks | the substrate's lock machinery (`lock_renew` carries the renew semantics the guarantee row pins; re-acquire does not refresh TTL — inherited deliberately, [ADR-012](decisions/012-forked-substrate-design.md) §4) |
|
||
| scheduler / outbox | collapse shape ([ADR-009](decisions/009-scheduler-collapse.md)): owned scheduler storage (`__alkstore_scheduler_tasks`, forked substrate) carrying `@every` specs (cron machinery not ported); the leader loop pattern (TTL lock, heartbeat, exit-before-tick-on-loss); the leadership lock is `__alkstore_scheduler`; outbox = helper over queues with the derived backing-queue name ([ADR-008](decisions/008-contract-v1-pinning.md) §4) |
|
||
| begin_tx | acquires the writer slot, opens `BEGIN IMMEDIATE`, returns the caller-held handle ([ADR-007](decisions/007-transactional-seam.md)) |
|
||
| handle ops | each `*_tx` op round-trips `spawn_blocking` to the same connection (thread-affinity note in [ADR-007](decisions/007-transactional-seam.md)) |
|
||
| commit/rollback | releases the writer slot |
|
||
|
||
## Obligations and constraints
|
||
|
||
- **A long transaction parks the writer** (the slot lease is the
|
||
honest model). Contract docs must surface this so consumers budget
|
||
transactions ([ADR-007](decisions/007-transactional-seam.md) negative
|
||
consequence).
|
||
- Watcher failure handling is inherited: on watcher death, every
|
||
subscriber's receiver closes (`WatcherDeathGuard` behavior) —
|
||
consumers see the close, never a silent hang
|
||
([ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||
- Wake coalescing: bursts inside one poll tick produce one wake;
|
||
correctness is preserved by the re-read contract
|
||
(POC-pinned: missed-wake stress with correct post-burst re-reads).
|
||
- **Deployment note**: the substrate's rusqlite generation
|
||
(^0.40.1) has a rustc floor ≥ 1.99; binaries linking this engine
|
||
carry that requirement ([deployment.md](deployment.md) matrix).
|
||
- The optimization path, if seam throughput ever demands it: a
|
||
dedicated std-thread bridge (one thread owning the writer conn, ops
|
||
over mpsc — measured ~2× the spawn_blocking shape at p50 in POC #1),
|
||
or a raised watcher cadence for idle CPU. Neither is the default.
|
||
|
||
## What the fork question resolved
|
||
|
||
The [quality read (OQ-06)](../research/quality-read-honker-core.md) —
|
||
this engine's dependency gate — resolved 2026-10-05: ADR-005's fork
|
||
trigger **fired** ([ADR-011](decisions/011-sqlite-substrate-fork.md)),
|
||
and the fork's design is pinned by
|
||
[ADR-012](decisions/012-forked-substrate-design.md): the substrate is
|
||
`alkstore-substrate`, a vendored contract-blind lineage crate; the
|
||
watcher/connection architecture this spec describes is inherited
|
||
verbatim where kept ([ADR-003](decisions/003-sqlite-driver.md)
|
||
unchanged in architecture), with the port deltas ADR-012 §4 decides
|
||
(fallible watcher spawn, bounded reconnect backoff, panic-free
|
||
death-closes-subscribers). The
|
||
[evidence](../research/quality-read-honker-core.md): published
|
||
0.5.0's unreleased-fix-train defect class, plus ADR-010's queue depth
|
||
requiring engine-owned queue SQL in any posture. The substrate's table
|
||
family is `__alkstore_*` (ADR-010 §8's naming authorization).
|
||
|
||
## Design Decisions
|
||
|
||
| ADR | Decision | Summary |
|
||
|---|---|---|
|
||
| [001](decisions/001-crate-split.md) | Crate split | single-driver engine crate |
|
||
| [002](decisions/002-feature-scope.md) | Feature scope | which rows this engine serves |
|
||
| [003](decisions/003-sqlite-driver.md) | Driver | rusqlite + the forked substrate lineage, bridged seam, inherited watcher — ownership resolved by 011 |
|
||
| [005](decisions/005-dependency-ownership.md) | Ownership | published honker-core; fork trigger fired by OQ-06 (ADR-011) |
|
||
| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract | data_version watcher, coalescing, death-closes-receivers |
|
||
| [007](decisions/007-transactional-seam.md) | Tx seam | writer-slot lease, `BEGIN IMMEDIATE`, `spawn_blocking` round-trips |
|
||
| [008](decisions/008-contract-v1-pinning.md) | Contract v1 | pinned surface; outbox backing queue derived under the reserved prefix; wake payload transport unused (non-contract) |
|
||
| [009](decisions/009-scheduler-collapse.md) | Scheduler collapse | `@every` specs; `__alkstore_scheduler` leadership; cron rejected |
|
||
| [010](decisions/010-queue-semantics-depth.md) | Queue depth | semantics pinned; §5/§3a realization resolved by ADR-011 (owned code) |
|
||
| [011](decisions/011-sqlite-substrate-fork.md) | Substrate fork | OQ-06's trigger fired; substrate owned, queue ops re-derived on contract v1 |
|
||
| [012](decisions/012-forked-substrate-design.md) | Fork design | contract-blind substrate, fidelity posture, port deltas, panic-free watcher death |
|
||
|
||
## Open Questions
|
||
|
||
Open questions are tracked in
|
||
[open-questions.md](open-questions.md). Key
|
||
questions affecting this document:
|
||
|
||
- **OQ-06**: honker-core quality read — fork-trigger assessment —
|
||
**resolved** (2026-10-05,
|
||
[ADR-011](decisions/011-sqlite-substrate-fork.md); trigger fired).
|
||
|
||
Resolved: **OQ-09** (scheduler collapse —
|
||
[ADR-009](decisions/009-scheduler-collapse.md)) and **OQ-05** (queue
|
||
semantics depth — [ADR-010](decisions/010-queue-semantics-depth.md)),
|
||
2026-10-05. |