All three engines' [dev-dependencies] tokio now carry the same feature
set: rt-multi-thread + macros + time + test-util (previously sqlite had
rt-multi-thread+macros+time, pg test-util only, mem rt+macros).
Dev-dep metadata only; each engine's tests verified green plus the
workspace build/clippy/fmt gates.
deployment.md's Postgres co-tenancy paragraph and engine-postgres.md's
connection-architecture wake-substrate section now carry the 63-byte
identifier-truncation collision note as dated annotations: two names
longer than 63 bytes sharing a 63-byte prefix silently register the
same server-side wake channel; pg_notify (the parameter path, untrun-
cated, rejecting >= 64-byte channels) is unaffected, so the collapse is
on the registration side. Mitigation: keep mechanism names at most 63
bytes (byte count). The naming contract's deliberate no-cap verdict is
untouched. Discharges review 004 'findings recorded, not fixed' item
13 (task docs-pg-identifier-truncation-note).
Review 004 items 10+11: the two recorded mem growth postures
(subscribe backlog drain unbounded on the consumer side; O(live-rows)
claim scan with full-row clones) and the outbox's 'processed means
delivered-or-reclaimable' semantic, pinned in engine-mem.md. The
debug_assert arm was declined: the dropped ack/retry booleans also
refuse benignly on legitimate in-closure handle ops, so it cannot
separate a visibility lapse from a deliberate first-party outcome.
Escapes the interpolated strings in the ReservedName/InvalidName/
InvalidSpec (and, for the same consumer-string interpolation class,
Codec) Display arms with char::escape_debug so rendered error
messages cannot carry raw control bytes from consumer input —
review 004 item 14's log-forging finding. The error taxonomy
(variants, kinds, match sites, ADR-008 §5) is unchanged; only the
rendered text's hygiene moved. Pinned by
error_display_escapes_consumer_control_chars.
- (a) the register/unregister decide-and-queue critical sections: the
refcount change and the derived LISTEN/UNLISTEN enqueue run under one
registry guard (ChannelSet::enqueue_register/enqueue_unregister), so
the command queue's order is the registry transitions' linearization —
the old thread-preempted decide/send split (no await inside, but
mid-unit preemption across workers) could leave a live refcount with
the server-side LISTEN cleared until the next reconnect re-issue.
The redundant per-registration LISTEN stays as the ack's vehicle, so
a listen() that returned still awaits a server-processed LISTEN.
- (b) a failed generation-verify no longer parks on the poll task
forever: the bounded settle (LISTEN_SETTLE_TIMEOUT=2s, shutdown-
selected) reaps a genuinely dead session promptly or aborts the
alive-but-unusable connection's reaper at the bound — reconnect
either way.
- (c) close()/Drop can no longer leak the listener session: every
listener-client query (verify batch, replay commands, serve
commands) rides bounded_wait (LISTENER_QUERY_TIMEOUT=5s) with the
shutdown watch selected alongside — the old unbounded run_command
await sat outside the shutdown select; an overrun is the honest
death posture (abort + reconnect, the snapshot re-issue recovers).
- pins: the interleaved register/unregister server-model equality pin
(with a decision-anchored deterministic finale), the bounded-wait +
failed-verify-settle paused-time pins, and the black-hole-poxy
harness pin (close reclaims the session past a stuck serve arm);
the refcount unit pin reworked onto the decide-and-queue API.
- replay-proofs (both ways, verified live): the old split shape fails
the interleave pin on the review's exact mismatch; the old
unbounded serve await fails the black-hole pin (session leaked past
close).
- the preserved postures re-verified green: reconnect survival, the
fanout release guard, receivers-terminal-iff-loop-gone, the
stale-UNLISTEN generation machinery, the register-failure
unregister discipline, the poison-recovery idiom, and both
POC-pinned deadlock-pitfall exclusions.
- gates: pg suite with the harness server green (131 lib + 25
contract + 9 schema; lib battery run twice full), workspace
build/test/fuzz gates server-less green, clippy -D warnings, fmt,
and the rustdoc gate all green; task -> completed
Review 004 item 15: the unsafe { ctx.get_connection() } UDF blocks
carried no SAFETY justification. Added the concrete soundness
invariant at all ten sites (the eight attach_alkstore_functions
registrations, the notify registration, the test-module savepoint
probe — same class) and registered the comment-only change as
PROVENANCE delta D-40 per ADR-012 §3's diff-fidelity posture.
Task: sqlite-substrate-safety-comments
with_writer/with_reader/blocking_read release their slot/pool
connection only on the normal return — a panic inside the closure
wedged the single writer slot forever (Writer::acquire waits on a
condvar) and permanently shrank the reader pool (outstanding never
decremented on abandonment). The seams now ride RAII drop-guards
(ConnLease's idiom): the normal arm defuses and releases unchanged;
the unwind arm drops the unknown-state connection and takes the
established abandonment dispositions — Writer::replenish() (the tick's
poisoned arm as a substrate method, the writer now holding its path)
and Readers::abandon() (outstanding decrement + pool-locked notify).
Three panic-injection tests pin the properties; no reachable panic
path exists in production — this is the defense-in-depth armor the
review scoped. Task: sqlite-fix-seam-panic-safety.
Instrumentation first: the tee-before-filter capture procedure of
record (demonstrated to preserve failing messages, immediately
vindicating itself — two natural repros captured verbatim), then the
arithmetic read against lock.rs/resolution.rs (delay-only argument
confirmed for the lapse legs, corrected for the contention leg:
second-precision now_unix stamps let a ttl=1 holder's exclusion
window collapse below a second), the conditions matrix (48+ rounds:
threads, concurrency with the SQLite column, cold/warm, CPU-load
stress), mechanism probes (1.2 s delay fails deterministically on the
exact assert; 200 ms lapses 2/10, frac-dependent), and the natural
repro (2/30 loaded solo runs, exact message at properties.rs:563).
Disposition: reproduced, root cause harness-model — the engine acted
per ADR-008 §7 throughout; the pre-fix row's contention leg asserted
a live-TTL state across an unbounded wall-clock gap. Trivially
test-side, so the fix landed in-session under the AC's exception:
lock_ttl_expiry_and_reacquisition is now two-legged (300 s exclusion
leg released; the 1 s lapse leg's post-sleep asserts stay
delay-only-safe); every pinned semantic survives.
Verified: 30/30 loaded runs of the fixed row green under the exact
repro condition; sqlite + mem columns green; workspace gates green
(build, live-pg workspace tests, clippy -D warnings, fmt); taskgraph
validate green.
Watch ledger: the lock row's flag retires; the trim row stays
watch-flagged (no repro, 10 focused + 7 loaded rounds); a new single
occurrence of row_stream_ordering_equivalence under synthetic load
(message captured — the row's drain window raced the pg bridge's
incomplete channel fill) is watch-flagged for review-wave-8's flake
outcome; no ride-along fix (instrumentation-first discipline).
Tasks: release-flake-investigation -> completed with Notes + Summary;
review-wave-5 ledger dated update.
Finding 2(a) landed: a one-shot cfg(test) generation-top LISTEN-pause
seam on the pg forwarder (listen_pause in seam.rs) lets the LISTEN-window
death test kill the fresh reconnect generation's backend before its
snapshot LISTEN — the LISTEN then fails honestly (57P01 terminating) and
the production fall-through arm (listen_ok == false) runs unmodified.
The harness test pins the honest mid-reconnect transient, full recovery
(wake, re-issue recovery of the refused mid-gap registration), and the
exactly-one-wake differentiator; a server-less unit pin covers the flag.
Finding 2(b) + the defensive arms stay parked with the code-read as pin,
recorded in the task file with current line refs.
Finding 3 landed: the rollback_fault seam (commit-fault mechanics) feeds
SQLITE_FULL-shaped errors into both ROLLBACK failure drop paths —
SqliteTxHandle::drop and the cancelled op's ConnLease::drop — exercised
in one test (phases: open-handle drop; single-poll-then-drop mid-op
cancellation), pinning slot replenish via reopen, no ghosts, and the
store staying usable.
The wave-8 N-1 deferred item — re-read the three unconditional-growth
paths against the implemented engines and land the decision of record:
all three accepted for v1, per engine, with no engine-side hygiene.
- deployment.md: new 'Growth postures (consciously accepted)' section —
per-engine table (dead letters sweep-on-call, SQLite-only notify
at-attach cap, durable-log stream posture; mem process-lifetime-
bounded by design) + the rationale and the deployment-matrix
fold-in positioning.
- engine-sqlite.md: 'Notifications-table hygiene' — attach is the only
prune site (D-16, oldest-first 10k); idle-but-listening processes
never prune; the cadence caveat deployers should size for.
- tasks/release-n1-growth-posture.md: completed; Notes/Summary filled.
Sweep of docs/reviews/ and tasks/ per the plan's §Wave 8 historical
note: 'wave 7'-means-release-pass references annotated or reworded
per the review-wave-5.md renumber-annotation precedent; genuine
fuzzing-wave/review-7 references untouched. Out-of-scope
architecture-doc trail records documented in the task's Notes.
12 annotated, 1 reworded (live pointer); taskgraph validate green.
The born-pinned gate audited the mem engine against the contract and
ADR-024: the conformance read clean in mechanism substance (queue
state machine, scheduler collapse, outbox, streams, locks, clock,
validation, tx overlay, wake substrate — incl. a full guard-order and
panic-hygiene audit); the suite's third column read row-by-row and
25/25 green under the per-test current-thread flavor (24 engine-shared
rows + mem's payload_too_large non-occurrence arm + the factory-shape
test); three-way arithmetic equivalence green on all columns; the
wasm32 acceptance items verified (compile-clean, tokio sync+time,
no blocking, flavor enforced); the rider's two arms verified (Finding
1 discharged; Findings 2-3 stay parked for wave 7); workspace gates,
taskgraph validate green.
Three edge-domain fixes landed inline: the stream cursor read's
i64::MAX overflow (Bound::Excluded), trim_to's i64::MAX horizon
endpoint (Bound::Included), and the scheduler register path now
rejecting a parseable interval whose first boundary overflows with
InvalidSpec — the one engine-vs-engine divergence the read found,
aligned into the class-1 window — plus a pinning test and scaffold-
residue doc/comment cleanups (clock allows, tx per-stream wake
comment, wakes/stream module docs). Recorded findings with
dispositions in the task's Notes for the wave-7 pre-release pass:
the multi-thread publish-to-wake ordering tolerance, the heartbeat/
save_offset/EventReceiver-try_recv ADR-023 §2 classification gaps,
and the mem arithmetic being a suite-pinned transplant.
Verified: cargo build; cargo test --workspace green (mem 101 lib +
25 suite, sqlite 191+25, pg 124+25+9 server-less, core 25, harness 3);
clippy --all-targets -D warnings clean; fmt clean; taskgraph validate
(61 tasks). Gate record: tasks/review-wave-6.md (Notes + Summary),
implementation.md wave-6 row + review-rounds entry.
The queue mechanism (ADR-010 state machine over the in-memory job
table, mem's third-owner equal-jitter curve + opts resolution)
already landed implemented with the seam task's whole-engine scope
(mem-engine-seam-tx Note 1); this task's work, per the disposition the
notify/listen twin recorded, was reconciling the description's pin
list against the existing suite and adding the missing engine-side
unit pins — no production code changed, the realization verified
correct as-found.
Already pinned by the seam task (reconciled, recorded in Notes):
lifecycle + stamps + resolved get_job values, claim ordering, extent
clamps at entry, worker-id validation, the uniform validity predicate,
late-heartbeat refusal + absolute-reset renewal, reclaim-eats-attempt
+ loser-ack-refuses, the pinned dead-letter default strings, dead-row
get_job visibility, both-states sweep + retention + scoping, the tx
stamp shapes, and the resolution legs + base-5 curve range/cap/spread.
Added (6): the in-process claim-race atomicity pin (8 concurrent
workers over 4 rows — every row handed exactly once through the
guarded section, the description's claim-vs-claim bullet); the
claim-vs-expiry boundary determinism pin (at now == deadline: the d <
now reclaim predicate and the d >= now holder-op predicate are
coherent — no reclaim, heartbeat/ack still valid; one second past:
reclaimable with its attempt consumed; lapsed ids silently uncounted
by ack_batch); the engine-side mirror of the suite's
backoff_curve_equivalence row — base 2 s at [1,2]/[2,4]/[4,8] driven
end-to-end through claim -> retry(None) -> get_job's run_at with the
Some(5)-verbatim and Some(0)-ready-now override legs, exact under the
frozen clock — both through the handle and pinned at backoff_delay_s
itself (the cap leg stays in resolution.rs's base-5 pins per
suite-opts-backoff-rows' recorded disposition); the D-12 pre-claim
sweep pin (a lapsed row with its budget spent claims nothing and
dead-letters with the exhaust string at the next claim, never
strands); and unconditional cancel across both states (the holder's
post-cancel ack refusing — the not-an-interrupt honesty). One
test-authoring note recorded: heartbeat's absolute reset moves the
deadline, so boundary legs stage on fresh rows to keep each
observation exact.
Task file: status completed, Notes (the seam-carried disposition +
reconciliation inventory + the six adds + the heartbeat-observation
note) + Summary filled. Verified: cargo test -p alkstore-mem 97/97
(91 + 6 new; new tests green solo x3), workspace cargo test green
(core 25, suite-harness 3, mem 97, pg 124+25+9, SQLite 191+25),
clippy --all-targets -D warnings clean, cargo fmt --check clean,
cargo check --target wasm32-unknown-unknown -p alkstore-mem clean
Replace the harness's blocking std::thread::sleep with tokio::time::sleep
across all nine call sites, note the no-blocking-calls-inside-rows posture
in the helper's doc comment, and mark the task completed. Both engines'
contract suites re-run green with assertions and tolerances unchanged
(ADR-024 \u00a75 item 4).