--- id: fork-port-connection-watcher name: Fork port — connection architecture + watcher machinery status: completed depends_on: [fork-substrate-scaffold] scope: broad risk: medium impact: component level: implementation tags: [wave-2, substrate] --- ## Description Port the connection/watcher half of honker-core into `alkstore-sqlite/src/substrate/` — the machinery the quality read verified clean and ADR-011 §scope inherits near-verbatim. Source: `/workspace/honker/honker-core/src/` @ `f4e53c6` (`lib.rs` ~4k lines, `shm_watcher.rs`, `kernel_watcher.rs`). **Port (the kept half, ADR-011 / quality-read §6):** the PRAGMA/WAL open posture + `set_journal_mode_wal` retry logic; `Writer`; `Readers`; the polling watcher + `SharedUpdateWatcher` + `WatcherDeathGuard` + `stat_identity` dead-man's switch; the `in_savepoint` / `UnwindUndo` mutation-discipline machinery; the REAL-coercion arg helpers; the notify scalar + notifications table; stream functions; lock functions. **Port deltas (ADR-012 §4 — the only deliberate behavior changes):** - W-1: bounded reconnect backoff in the watcher loop (no ~1000 open-attempts/sec on a vanished db file). - W-2: watcher spawn becomes fallible; callers (the engine, wave 3) surface the failure at open time. - Dead-man's switch: the db-file-identity-change panic is replaced by a deliberate watcher-fatal death (log the precise diagnostic, exit through the ordinary death path); `WatcherDeathGuard`'s death-closes-subscribers behavior unchanged. **Drops (do not port):** `cron.rs`; the `kernel-watcher` / `shm-fast-path` experimental watcher backends and their optional deps (`notify`, `memmap2`, `libc`); the rate-limit and result tables; the superseded queue functions (the queue half is re-derived in a separate task). **Table naming:** `_honker_*` → `__alkstore_*` across the ported storage surface (ADR-011; ADR-010 §8 authorization). No online migration machinery — fresh bootstrap only (ADR-012 §5). **Fidelity posture (ADR-012 §3):** keep upstream's module structure and internal function names for the kept half, including lineage tokens (`Writer`, `run_poll_loop`, `in_savepoint`). Renames confined to the table family and the hygiene deltas. The port is mechanical where possible — the diff against the lineage must stay reviewable. **Discipline deltas (family standard, ADR-012 §4):** no comments in code (doc comments on the substrate's internal public surface fine), no panics in library code (the dead-man's-switch delta is the big one), no `unwrap()`/`expect()` outside tests. The substrate stays sync — no tokio, no asyncification. **Bootstrap re-keying (ADR-012 §5):** the fork owns the bootstrap; append-column migrations stay; the duplicate-column race swallow is re-keyed to `pragma_table_info` verification (present ⇒ benign race, absent ⇒ propagate) — upstream's error-string matching is not inherited. **Tests (the floor, ADR-011):** inherit honker-core's suites for the ported machinery — PRAGMA/WAL, watcher lifecycle + failure handling, savepoint + multiprocess pressure — adapted to the new table names and the three port deltas. These run as engine-crate tests (the substrate is not separately testable through a public seam). ## Acceptance Criteria - [x] Kept machinery ported with upstream names/structure preserved; diff vs. lineage reviewable (the wave-2 review checks this) - [x] Three watcher deltas applied; dead-man's switch exits without panicking; `WatcherDeathGuard` still closes all subscribers - [x] Dropped machinery absent (no cron, no experimental watchers, no rate-limit/result tables, no superseded queue functions) - [x] Table family is `__alkstore_*`; bootstrap fresh-only; race swallow re-keyed to `pragma_table_info` - [x] Inherited test suites green (adapted); watcher-death-closes- subscribers test present - [x] Substrate is sync (no tokio imports); no panics/`unwrap()` in library code; clippy `-D warnings`, fmt clean ## References - docs/architecture/decisions/011-sqlite-substrate-fork.md (scope register) - docs/architecture/decisions/012-forked-substrate-design.md §3–§5 - docs/research/quality-read-honker-core.md §6 (the fork scope), §2 (watcher verdict) - /workspace/honker @ f4e53c6 (the lineage) ## Notes - **Module layout**: the lineage's two files (`lib.rs` ~4k lines, `honker_ops.rs` ~3.3k) map to three substrate modules in upstream's order — `schema.rs` (pragmas/open, notify, bootstrap, `Writer`, `Readers`), `watcher.rs` (the polling watcher family), `ops.rs` (savepoint discipline, arg coercion, stream/lock ops, the scalar-function attachments). Upstream names preserved on the kept body; the mapping is register entry D-17 (the flat 4k `lib.rs` doesn't fit the module-per-file standard; the ADR-013 fold makes the module file the unit). - **`file-id` retained** (D-18): the task's drop list names it among the experimental backends' optional deps, but it drives the *kept* dead-man's switch (`stat_identity`) — the kept-half scope wins for this dep; target-gated `cfg(any(unix, windows))` exactly as upstream carries it. - **W-1 shape**: bounded reconnect backoff as a tick-counter (`MAX_RECONNECT_TICKS` = 100 poll ticks between attempts → ≤10 open attempts/sec at the default 1 ms cadence, vs the lineage's ~1000/sec). Implemented inside `run_poll_loop`; the open step is a parameter (default = the real open) so the test drives the loop directly with a counting open. - **W-2 shape**: `SharedUpdateWatcher::new*` and `UpdateWatcher::spawn*` return `Result<_, String>` (upstream's spawn used `.expect(...)`). The thread-build step is isolated in `UpdateWatcher::spawn_thread` with a test-reachable stack-size knob so the `Err` path is provable (an oversized stack reservation fails reliably; a nonexistent db path does NOT fail thread spawn — baseline capture inside the loop is a retry path, not a spawn failure). The engine maps the String into the taxonomy's `Database` in wave 3. - **Dead-man's switch delta**: identity mismatch logs the precise diagnostic (upstream's panic message text, repointed) and `return`s through the ordinary death path; the death-closes-subscribers test additionally asserts `join()` is now `Ok` (the lineage's returned the panic payload). - **Bootstrap**: the scheduler's `cron_expr` column is carried as `spec` in `__alkstore_scheduler_tasks` (the storage the scheduler collapse lands in; cron strings are rejected at the engine layer — ADR-009; naming is storage-internal so the column rename is registration D-10's surface). The three append-column migrations stay (enabled/max_attempts/claimed_at — the schema D-12's stamps extend); the duplicate-column race swallow is re-keyed to `pragma_table_info` exactly per ADR-012 §5. - **Bootstrap schema scope**: the bootstrap carries the full `__alkstore_*` family including the live/dead table shapes the re-derivation extends — the tables must exist for the engine's `alkstore_bootstrap()` to be total even before wave 3 wires the queue ops; the re-derivation task alters/extends as its SQL needs. - **Notify at-attach cap (D-16)**: `NOTIFY_ATTACH_MAX_ROWS` = 10_000, oldest-first trim below the cap on every attach — replacing upstream's "no pruning" posture per ADR-010 §6 (the fix for unbounded notifications growth must not become a caller's chore). - **Test scope boundary**: honker's queue-op suites (savepoint dead-letter paths, claim pressure, `claimed_at` transitions, scheduler) test the *re-derivation's* machinery — they are `fork-rederive-queue-ops`'s floor, not this task's. This task inherits: PRAGMA/WAL open posture (incl. the concurrent-open race suite), watcher lifecycle + failure handling (fan-out, unsubscribe, subscriber prune, death signal, all four journal modes, checkpoint, stat identity, XOR-fold unit tests), savepoint machinery tests (rollback-failure reporting, panic unwinding, scalar-path frames), arg-coercion tests (driven through a probe scalar function — the queue-function shape they exist for is re-derived next), notify tests, lock tests, a new stream-ops suite, and a cross-mechanism pressure test (streams+notify+locks, 5 producers) — the pressure shape minus queue ops. Added: the W-1 backoff-bound test, the W-2 spawn-failure test, the panic-free death test, the column-order-equality migration test, the legacy-orphan-inertness test. 43 tests green. - **Port state**: the substrate compiles as engine-crate modules and is test-covered, but nothing re-exports through the engine's public API yet — wave 3 wires `open` + the trait impl (`mod.rs` carries `#![allow(dead_code)]` marked as lift-on-wiring). ## Summary The kept half of the honker-core fork landed in `alkstore-sqlite/src/substrate/` as `schema.rs` + `watcher.rs` + `ops.rs` (~1k lines ported machinery + adapted inherited suites, 43 tests): PRAGMA/WAL open posture with `set_journal_mode_wal` retry (+ the concurrent-open race suite), `Writer`, `Readers` (incl. the open-failure capacity-leak test), the polling watcher + `SharedUpdateWatcher` + `WatcherDeathGuard` + `stat_identity` dead-man's switch (all four journal modes + checkpoint + fan-out + prune suites), `in_savepoint`/`UnwindUndo` (+ panic-discipline tests), REAL-coercion arg helpers (+ coercion tests), the notify scalar + table (now with the ADR-010 §6 at-attach pruning cap), the stream functions (+ read/publish/offset suite), and the lock functions (+ grant/refuse/renew/release suites). Deltas: W-1 bounded reconnect backoff (tick-counting, test-pinned at ≤ 2 attempts in a 2.6 s window vs the lineage's ~260), W-2 fallible watcher spawn (Result, mapped error names the watcher), the dead-man's-switch panic replaced by log-and-exit (join is Ok; death still closes every subscriber — the adapted death-signal test pins both), `_honker_*` → `__alkstore_*` (legacy tables pinned inert by test), duplicate-column race swallow re-keyed to `pragma_table_info`, scheduler `cron_expr` → `spec`, fresh-only bootstrap with append-column migrations (column-order equality pinned). Dropped fully: cron, kernel/shm watcher backends, rate-limit/result tables, superseded queue functions (the pressure suite's queue half moved to the re-derivation task). `file-id` kept (dead-man's-switch dep; register D-18 corrects D-02's enumeration). PROVENANCE.md register updated to the landed state (D-01..D-20). Verified: `cargo build`, `cargo test` (workspace; 43 sqlite-crate tests green), `cargo clippy --all-targets -- -D warnings`, `cargo fmt --check` all clean. Subtree stays sync (zero tokio); no panics/unwrap/expect outside tests.