diff --git a/docs/research/phase-0.md b/docs/research/phase-0.md index c440d90..2f4fb39 100644 --- a/docs/research/phase-0.md +++ b/docs/research/phase-0.md @@ -3,15 +3,13 @@ status: draft last_updated: 2026-10-04 (consumer inventory landed — OQ-ST-01 answered per-feature from the paused consumers' documents; phase-0 plan step 1 done; streams upgraded to in-scope by operator-authority record; OQ-ST-02 -resolved: reactive-core + engine crates, operator decision; alktty -REQ-TTY-01's async-facing-trait + sync-bridge posture recorded as family -precedent bearing on OQ-ST-03; OQ-ST-03's SQLite option space expanded -to three named postures (honker-core on our rusqlite / honker-rs as -substrate / honker extension over sqlx — the last CI-proven in the -honker checkout's own ORM proof suite) and POC #1 (the A-vs-B posture -comparison) specified. OQ-ST-03/04/05 remain open -research. Interface finding and driver tension from -2026-10-03 remain trusted-but-unverified working input.) +resolved: reactive-core + engine crates, operator decision; OQ-ST-03's +SQLite half resolved by POC #1 (honker-core on our rusqlite — findings +in poc-sqlite-posture-findings.md); OQ-ST-04/05/06 carry POC #1 input. +Remaining: OQ-ST-03's Postgres half, OQ-ST-04 contract pinning, +OQ-ST-05/06 pg-side ownership, OQ-ST-07/08.) Interface finding and +driver tension from 2026-10-03 remain trusted-but-unverified working +input except where POC #1 verified them.) --- # alkstore — Phase 0 (Exploration) @@ -477,6 +475,25 @@ and a POC comparing options 1 and 3 directly (same queue/stream/notify surface both ways, incl. watcher wiring) is the natural first POC for the register. +**SQLite half resolved by POC #1 (2026-10-04; findings: +`poc-sqlite-posture-findings.md`): posture 1 — honker-core on our +rusqlite.** All three gate conditions fired in A's favor, mildly: the +bridged path is ~2× sqlx's native-async at p50 (0.354 vs 0.707 ms on +the tx-enqueue workload; B's premise measured false), honker-core's +inherited watcher is tighter than a re-derived one (p50 1.40 vs 2.15 +ms, max 29 vs 172 ms, with battle-tested failure handling), and the +`.so` runtime dependency is packaging cost A doesn't pay for no +compensating advantage. The transactional contract holds identically +on both (it is SQLite's property, not the posture's). The Postgres +half remains the open part of OQ-ST-03 (tokio-postgres per the POC +evidence base is the working lean; the OQ is not closed until the pg +side's LISTEN/pool/tx-seam story is validated the same way). New +constraints recorded for the engine crate regardless: +honker-core=0.5.0 pins rusqlite ^0.40.1 whose rustc requirement +(≥1.99) is a deployment note, and mixed rusqlite+sqlx binaries +currently need a vendored one-line libsqlite3-sys patch — OQ-ST-02's +per-engine-crate split is what keeps the engine binary single-driver. + ### OQ-ST-04: The reactive abstraction — what does the unified notify surface look like? The two engines' mechanisms are structurally different: SQLite = @@ -518,8 +535,24 @@ receive sufficient invalidation information (keys? table/channel names? opaque wake + re-read contract?) — rides the same contract decision. +**POC #1 input (2026-10-04):** the opaque-wake + re-read contract +survives both SQLite postures unchanged (same `data_version` mechanism +underneath — wake coalescing verified correct by design, p50 +1.4–2.2 ms at the default 1 ms cadence, missed-wake stress passes with +correct re-reads on both); listener semantics (start at MAX(id), no +replay) unchanged. The `*_tx` seam question crystallized into +something concrete: the SQLite-side tx handle is an owned writer-slot +lease whose ops each ride `spawn_blocking` (the connection is not +`Sync`; holding it across await points is wrong) — Phase 1's contract +work starts from that shape plus the POC's `TxHandle` sketch (with its +two recorded frictions: the `as_any_mut` downcast and the +thread-affinity of rusqlite tx ops). What remains open is everything +Postgres (transaction-scoped LISTEN over a pool, and the delivery- +guarantee/unification bullets above). + Open; this is the second central research question, coupled to OQ-ST-03 -(the driver determines what LISTEN plumbing exists). +(the driver determines what LISTEN plumbing exists) — SQLite side now +de-risked, Postgres side is the remaining shape work. ### OQ-ST-05: Queue semantics — adopt, fork, or re-derive? @@ -536,7 +569,14 @@ choice, so the queue-machinery reuse value is the honest comparison point, not the whole. Open; inputs: OQ-ST-01's scope vote on queues (inventory: documented -need) + OQ-ST-03 resolution. +need) + OQ-ST-03 resolution. **POC #1 input (2026-10-04):** on the +SQLite side the adopt question dissolved — honker-core is consumed as a +published-library dependency (the inventory-confirmed feature rows ride +its machinery; OQ-ST-06 holds the fork-vs-reference question). The +Postgres side keeps this OQ's full option space (pgboss-rs vs fork vs +re-derive on tokio-postgres) — with the POC-confirmed constraint that +whichever choice is made, reactivity is built by this crate (the +verified pgboss-rs LISTEN/NOTIFY gap stands). ### OQ-ST-06: Honker relationship — reference, fork, or vendor? @@ -561,7 +601,15 @@ demonstrates the interface shape is sound. Three refinements from the wire surface" principle than to alksocks' verbatim extraction. Open; needs the license/provenance check (AGENTS.md §3) and a quality -assessment of honker-core's watcher/transactional core. +assessment of honker-core's watcher/transactional core. **POC #1 input +(2026-10-04):** under the resolved SQLite posture (posture 1), honker-core +is consumed as a *published library* (Writer/Readers/SharedUpdateWatcher/ +attach_* — nearly all its surface minus the experimental backends), not +vendored or forked to ship; 0.5.0 is published with clean deps and the +reference-usage posture works as-is. The remaining fork trigger would be +the Phase-1 quality read (the watcher/transactional core assessment) or a +needed change upstream won't take — the calculus is unchanged in kind, +but the *default* posture is now evidenced: depend on the published crate. ### OQ-ST-07: SQLite-side scope — loadable extension, embedded rusqlite, or both? @@ -596,15 +644,9 @@ capability differences can surface). Proposals run as standalone crates in the global workspace; findings land in `docs/research/` here. Named per the OQ each feeds: -| # | POC | Spec | Feeds | +| # | POC | Spec | Findings | |---|---|---|---| -| 1 | SQLite engine posture: honker-core-on-rusqlite vs honker-extension-over-sqlx (async seam, watcher, transactional contract, packaging, interop) | [poc-sqlite-posture-spec.md](poc-sqlite-posture-spec.md) | OQ-ST-03, OQ-ST-04, OQ-ST-06 | - -POC #1 **ran (2026-10-04)** — verdict Arm A (honker-core on our -rusqlite), findings in -[poc-sqlite-posture-findings.md](poc-sqlite-posture-findings.md); -OQ-ST-03's SQLite half and OQ-ST-04's SQLite wake side now have -measured input. +| 1 | SQLite engine posture: honker-core-on-rusqlite vs honker-extension-over-sqlx (async seam, watcher, transactional contract, packaging, interop) | [poc-sqlite-posture-spec.md](poc-sqlite-posture-spec.md) | [poc-sqlite-posture-findings.md](poc-sqlite-posture-findings.md) — **passed** (verdict: Arm A; ran 2026-10-04) | ## Phase 0 plan @@ -619,16 +661,23 @@ Expected sequence (deliberately rough): 1. ~~Crate scope (OQ-ST-02)~~ — **resolved (2026-10-04, operator decision)**: reactive-core + engine crates. Reasoning and the superseded inventory lean recorded at the OQ and in the inventory. -2. Research rounds on OQ-ST-03/04 (drivers + reactive shape) — library - capability matrices (**POC #1 specified**: the SQLite posture - comparison, [poc-sqlite-posture-spec.md](poc-sqlite-posture-spec.md); - the honker-rs surface (§Interface finding) is the concrete starting - artifact for the OQ-ST-04 work: pinning its contract costs less and - is more honest than inventing a parallel shape — now scoped against - the inventory's confirmed features rather than the full honker menu, - and shaped as the core-crate trait surface per OQ-ST-02's split). +2. Research rounds on OQ-ST-03/04 (drivers + reactive shape): + - ~~SQLite driver posture~~ — **POC #1 passed (2026-10-04)**: + posture 1 (honker-core on our rusqlite); findings + constraint + notes in the register row and OQ-ST-03. + - Postgres side of OQ-ST-03 (tokio-postgres LISTEN/pool/tx-seam + validation) — the next research round, likely POC #2. + - OQ-ST-04's contract pinning — the honker-rs surface (§Interface + finding) is the concrete starting artifact: pinning its contract + costs less and is more honest than inventing a parallel shape — + now scoped against the inventory's confirmed features rather than + the full honker menu, shaped as the core-crate trait surface per + OQ-ST-02's split, with the POC's `TxHandle`/writer-slot-lease + shape as the SQLite-seam starting point. 3. Ownership decisions (OQ-ST-05/06) — adopt/fork/derive per subsystem, - after the driver and shape questions narrow the option space. + after the driver and shape questions narrow the option space + (OQ-ST-06's default is now evidenced as published-library use; + OQ-ST-05 waits on the pg side of OQ-ST-03). 4. Converge; Phase 1 opens with the ADR backlog this register becomes. ## References @@ -664,6 +713,9 @@ Expected sequence (deliberately rough): per-feature synthesis (2026-10-04) answering OQ-ST-01 from the above. - alkcall — `/workspace/@alkdev/alkcall`: the family substrate; referenced for the store-layer-isolation principle only. +- alkstore-sqlite-posture-poc — `/workspace/alkstore-sqlite-posture-poc` + (standalone POC crate, published-deps-only): POC #1's code — + both arms end-to-end, property tests, seam/watcher probes. - alktty — `/workspace/@alkdev/alktty` (architecture reviewed): REQ-TTY-01 (`docs/architecture/tty-backend.md`) — the async-facing-trait + sync-bridge posture ("backends are not required to be natively diff --git a/docs/research/poc-sqlite-posture-findings.md b/docs/research/poc-sqlite-posture-findings.md index ae80848..27a0195 100644 --- a/docs/research/poc-sqlite-posture-findings.md +++ b/docs/research/poc-sqlite-posture-findings.md @@ -175,8 +175,11 @@ test-suite scheme). ## Probe 3 — transactional contract (decision axis 4) -Property tests (Rust, `tests/contract.rs`, all green, 4 tests + 1 -watcher-recovery): +Property tests (Rust, `tests/contract.rs`, all green, 4 tests — +`tx_enqueue_rollback_drops_both_ghost_free`, +`concurrent_writers_exactly_once`, `missed_wake_stress_burst_coalescing`, +`watcher_death_recovery_arm_b` — covering the rows below; the first test +also asserts the committed-tx happy path): | Property | A | B | |---|---|---| @@ -301,7 +304,7 @@ driver if one ever displaces rusqlite). - POC crate: `/workspace/alkstore-sqlite-posture-poc` (self-contained; `extbuild/libhonker_ext.so` built from crates.io-published source; `[patch.crates-io]` one-liner for sqlx-sqlite/libsqlite3-sys). -- Tests: `cargo test` → `tests/contract.rs` (5 tests; transactional +- Tests: `cargo test` → `tests/contract.rs` (4 tests; transactional properties, concurrency, wake coalescing, watcher recovery). - Probes: `cargo run --release -- seam|watch ` per the binary's help; `watchloop` for the dedicated data_version watcher. \ No newline at end of file