Kept half of the honker-core fork lands in alkstore-sqlite/src/substrate/ as schema.rs / watcher.rs / ops.rs (register D-17): PRAGMA/WAL open posture + set_journal_mode_wal retry, Writer, Readers, the polling watcher family (SharedUpdateWatcher, WatcherDeathGuard, stat_identity dead-man's switch), in_savepoint/ UnwindUndo mutation discipline, REAL-coercion arg helpers, notify scalar + notifications table with the ADR-010 §6 at-attach pruning cap, stream functions, lock functions. Port deltas (ADR-012 §4): W-1 bounded reconnect backoff (MAX_RECONNECT_TICKS=100), W-2 fallible watcher spawn (Result; engine maps to Database at open in wave 3), dead-man's-switch panic replaced by log-and-exit through the ordinary death path — death still closes every subscriber (pinned by test, join now Ok). Table family _honker_* -> __alkstore_* (D-10); duplicate-column race swallow re-keyed to pragma_table_info (D-11); scheduler cron_expr -> spec; fresh-only bootstrap, append-column migrations kept, column-order equality pinned. Drops confirmed absent: cron, kernel/shm backends, rate-limit/result tables, superseded queue functions (D-01..D-04). file-id retained for the kept dead-man's switch (D-18). 43 engine-crate tests green (adapted inherited suites + delta tests + cross-mechanism pressure); cargo build/clippy -D warnings/fmt clean. PROVENANCE.md register updated to the landed state (D-01..D-20).
201 lines
11 KiB
Markdown
201 lines
11 KiB
Markdown
---
|
|
id: fork-port-connection-watcher
|
|
name: Fork port — connection architecture + watcher machinery
|
|
status: completed
|
|
depends_on: [fork-substrate-scaffold]
|
|
scope: broad
|
|
risk: medium
|
|
impact: component
|
|
level: implementation
|
|
tags: [wave-2, substrate]
|
|
---
|
|
|
|
## Description
|
|
|
|
Port the connection/watcher half of honker-core into
|
|
`alkstore-sqlite/src/substrate/` — the machinery the quality read
|
|
verified clean and ADR-011 §scope inherits near-verbatim. Source:
|
|
`/workspace/honker/honker-core/src/` @ `f4e53c6` (`lib.rs` ~4k lines,
|
|
`shm_watcher.rs`, `kernel_watcher.rs`).
|
|
|
|
**Port (the kept half, ADR-011 / quality-read §6):** the PRAGMA/WAL
|
|
open posture + `set_journal_mode_wal` retry logic; `Writer`; `Readers`;
|
|
the polling watcher + `SharedUpdateWatcher` + `WatcherDeathGuard` +
|
|
`stat_identity` dead-man's switch; the `in_savepoint` /
|
|
`UnwindUndo` mutation-discipline machinery; the REAL-coercion arg
|
|
helpers; the notify scalar + notifications table; stream functions;
|
|
lock functions.
|
|
|
|
**Port deltas (ADR-012 §4 — the only deliberate behavior changes):**
|
|
|
|
- W-1: bounded reconnect backoff in the watcher loop (no ~1000
|
|
open-attempts/sec on a vanished db file).
|
|
- W-2: watcher spawn becomes fallible; callers (the engine, wave 3)
|
|
surface the failure at open time.
|
|
- Dead-man's switch: the db-file-identity-change panic is replaced by
|
|
a deliberate watcher-fatal death (log the precise diagnostic, exit
|
|
through the ordinary death path); `WatcherDeathGuard`'s
|
|
death-closes-subscribers behavior unchanged.
|
|
|
|
**Drops (do not port):** `cron.rs`; the `kernel-watcher` /
|
|
`shm-fast-path` experimental watcher backends and their optional deps
|
|
(`notify`, `memmap2`, `libc`); the rate-limit and result tables; the
|
|
superseded queue functions (the queue half is re-derived in a separate
|
|
task).
|
|
|
|
**Table naming:** `_honker_*` → `__alkstore_*` across the ported
|
|
storage surface (ADR-011; ADR-010 §8 authorization). No online
|
|
migration machinery — fresh bootstrap only (ADR-012 §5).
|
|
|
|
**Fidelity posture (ADR-012 §3):** keep upstream's module structure and
|
|
internal function names for the kept half, including lineage tokens
|
|
(`Writer`, `run_poll_loop`, `in_savepoint`). Renames confined to the
|
|
table family and the hygiene deltas. The port is mechanical where
|
|
possible — the diff against the lineage must stay reviewable.
|
|
|
|
**Discipline deltas (family standard, ADR-012 §4):** no comments in
|
|
code (doc comments on the substrate's internal public surface fine),
|
|
no panics in library code (the dead-man's-switch delta is the big
|
|
one), no `unwrap()`/`expect()` outside tests. The substrate stays sync
|
|
— no tokio, no asyncification.
|
|
|
|
**Bootstrap re-keying (ADR-012 §5):** the fork owns the bootstrap;
|
|
append-column migrations stay; the duplicate-column race swallow is
|
|
re-keyed to `pragma_table_info` verification (present ⇒ benign race,
|
|
absent ⇒ propagate) — upstream's error-string matching is not
|
|
inherited.
|
|
|
|
**Tests (the floor, ADR-011):** inherit honker-core's suites for the
|
|
ported machinery — PRAGMA/WAL, watcher lifecycle + failure handling,
|
|
savepoint + multiprocess pressure — adapted to the new table names and
|
|
the three port deltas. These run as engine-crate tests (the substrate
|
|
is not separately testable through a public seam).
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [x] Kept machinery ported with upstream names/structure preserved;
|
|
diff vs. lineage reviewable (the wave-2 review checks this)
|
|
- [x] Three watcher deltas applied; dead-man's switch exits without
|
|
panicking; `WatcherDeathGuard` still closes all subscribers
|
|
- [x] Dropped machinery absent (no cron, no experimental watchers, no
|
|
rate-limit/result tables, no superseded queue functions)
|
|
- [x] Table family is `__alkstore_*`; bootstrap fresh-only; race
|
|
swallow re-keyed to `pragma_table_info`
|
|
- [x] Inherited test suites green (adapted); watcher-death-closes-
|
|
subscribers test present
|
|
- [x] Substrate is sync (no tokio imports); no panics/`unwrap()` in
|
|
library code; clippy `-D warnings`, fmt clean
|
|
|
|
## References
|
|
|
|
- docs/architecture/decisions/011-sqlite-substrate-fork.md (scope register)
|
|
- docs/architecture/decisions/012-forked-substrate-design.md §3–§5
|
|
- docs/research/quality-read-honker-core.md §6 (the fork scope), §2 (watcher verdict)
|
|
- /workspace/honker @ f4e53c6 (the lineage)
|
|
|
|
## Notes
|
|
|
|
- **Module layout**: the lineage's two files (`lib.rs` ~4k lines,
|
|
`honker_ops.rs` ~3.3k) map to three substrate modules in upstream's
|
|
order — `schema.rs` (pragmas/open, notify, bootstrap, `Writer`,
|
|
`Readers`), `watcher.rs` (the polling watcher family),
|
|
`ops.rs` (savepoint discipline, arg coercion, stream/lock ops, the
|
|
scalar-function attachments). Upstream names preserved on the kept
|
|
body; the mapping is register entry D-17 (the flat 4k `lib.rs`
|
|
doesn't fit the module-per-file standard; the ADR-013 fold makes the
|
|
module file the unit).
|
|
- **`file-id` retained** (D-18): the task's drop list names it among the
|
|
experimental backends' optional deps, but it drives the *kept*
|
|
dead-man's switch (`stat_identity`) — the kept-half scope wins for
|
|
this dep; target-gated `cfg(any(unix, windows))` exactly as upstream
|
|
carries it.
|
|
- **W-1 shape**: bounded reconnect backoff as a tick-counter
|
|
(`MAX_RECONNECT_TICKS` = 100 poll ticks between attempts → ≤10
|
|
open attempts/sec at the default 1 ms cadence, vs the lineage's
|
|
~1000/sec). Implemented inside `run_poll_loop`; the open step is a
|
|
parameter (default = the real open) so the test drives the loop
|
|
directly with a counting open.
|
|
- **W-2 shape**: `SharedUpdateWatcher::new*` and `UpdateWatcher::spawn*`
|
|
return `Result<_, String>` (upstream's spawn used `.expect(...)`).
|
|
The thread-build step is isolated in `UpdateWatcher::spawn_thread`
|
|
with a test-reachable stack-size knob so the `Err` path is provable
|
|
(an oversized stack reservation fails reliably; a nonexistent db path
|
|
does NOT fail thread spawn — baseline capture inside the loop is a
|
|
retry path, not a spawn failure). The engine maps the String into the
|
|
taxonomy's `Database` in wave 3.
|
|
- **Dead-man's switch delta**: identity mismatch logs the precise
|
|
diagnostic (upstream's panic message text, repointed) and `return`s
|
|
through the ordinary death path; the death-closes-subscribers test
|
|
additionally asserts `join()` is now `Ok` (the lineage's returned
|
|
the panic payload).
|
|
- **Bootstrap**: the scheduler's `cron_expr` column is carried as
|
|
`spec` in `__alkstore_scheduler_tasks` (the storage the scheduler
|
|
collapse lands in; cron strings are rejected at the engine layer —
|
|
ADR-009; naming is storage-internal so the column rename is
|
|
registration D-10's surface). The three append-column migrations
|
|
stay (enabled/max_attempts/claimed_at — the schema D-12's stamps
|
|
extend); the duplicate-column race swallow is re-keyed to
|
|
`pragma_table_info` exactly per ADR-012 §5.
|
|
- **Bootstrap schema scope**: the bootstrap carries the full
|
|
`__alkstore_*` family including the live/dead table shapes the
|
|
re-derivation extends — the tables must exist for the engine's
|
|
`alkstore_bootstrap()` to be total even before wave 3 wires the
|
|
queue ops; the re-derivation task alters/extends as its SQL needs.
|
|
- **Notify at-attach cap (D-16)**: `NOTIFY_ATTACH_MAX_ROWS` = 10_000,
|
|
oldest-first trim below the cap on every attach — replacing
|
|
upstream's "no pruning" posture per ADR-010 §6 (the fix for
|
|
unbounded notifications growth must not become a caller's chore).
|
|
- **Test scope boundary**: honker's queue-op suites (savepoint
|
|
dead-letter paths, claim pressure, `claimed_at` transitions,
|
|
scheduler) test the *re-derivation's* machinery — they are
|
|
`fork-rederive-queue-ops`'s floor, not this task's. This task
|
|
inherits: PRAGMA/WAL open posture (incl. the concurrent-open race
|
|
suite), watcher lifecycle + failure handling (fan-out, unsubscribe,
|
|
subscriber prune, death signal, all four journal modes, checkpoint,
|
|
stat identity, XOR-fold unit tests), savepoint machinery tests
|
|
(rollback-failure reporting, panic unwinding, scalar-path frames),
|
|
arg-coercion tests (driven through a probe scalar function — the
|
|
queue-function shape they exist for is re-derived next), notify
|
|
tests, lock tests, a new stream-ops suite, and a cross-mechanism
|
|
pressure test (streams+notify+locks, 5 producers) — the pressure
|
|
shape minus queue ops. Added: the W-1 backoff-bound test, the W-2
|
|
spawn-failure test, the panic-free death test, the
|
|
column-order-equality migration test, the legacy-orphan-inertness
|
|
test. 43 tests green.
|
|
- **Port state**: the substrate compiles as engine-crate modules and
|
|
is test-covered, but nothing re-exports through the engine's public
|
|
API yet — wave 3 wires `open` + the trait impl (`mod.rs` carries
|
|
`#![allow(dead_code)]` marked as lift-on-wiring).
|
|
|
|
## Summary
|
|
|
|
The kept half of the honker-core fork landed in
|
|
`alkstore-sqlite/src/substrate/` as `schema.rs` + `watcher.rs` +
|
|
`ops.rs` (~1k lines ported machinery + adapted inherited suites, 43
|
|
tests): PRAGMA/WAL open posture with `set_journal_mode_wal` retry
|
|
(+ the concurrent-open race suite), `Writer`, `Readers` (incl. the
|
|
open-failure capacity-leak test), the polling watcher +
|
|
`SharedUpdateWatcher` + `WatcherDeathGuard` + `stat_identity`
|
|
dead-man's switch (all four journal modes + checkpoint + fan-out +
|
|
prune suites), `in_savepoint`/`UnwindUndo` (+ panic-discipline
|
|
tests), REAL-coercion arg helpers (+ coercion tests), the notify
|
|
scalar + table (now with the ADR-010 §6 at-attach pruning cap), the
|
|
stream functions (+ read/publish/offset suite), and the lock functions
|
|
(+ grant/refuse/renew/release suites). Deltas: W-1 bounded reconnect
|
|
backoff (tick-counting, test-pinned at ≤ 2 attempts in a 2.6 s window
|
|
vs the lineage's ~260), W-2 fallible watcher spawn (Result, mapped
|
|
error names the watcher), the dead-man's-switch panic replaced by
|
|
log-and-exit (join is Ok; death still closes every subscriber — the
|
|
adapted death-signal test pins both), `_honker_*` → `__alkstore_*`
|
|
(legacy tables pinned inert by test), duplicate-column race swallow
|
|
re-keyed to `pragma_table_info`, scheduler `cron_expr` → `spec`,
|
|
fresh-only bootstrap with append-column migrations (column-order
|
|
equality pinned). Dropped fully: cron, kernel/shm watcher backends,
|
|
rate-limit/result tables, superseded queue functions (the pressure
|
|
suite's queue half moved to the re-derivation task). `file-id` kept
|
|
(dead-man's-switch dep; register D-18 corrects D-02's enumeration).
|
|
PROVENANCE.md register updated to the landed state (D-01..D-20).
|
|
Verified: `cargo build`, `cargo test` (workspace; 43 sqlite-crate
|
|
tests green), `cargo clippy --all-targets -- -D warnings`, `cargo fmt
|
|
--check` all clean. Subtree stays sync (zero tokio); no
|
|
panics/unwrap/expect outside tests. |