--- id: mem-engine-seam-tx name: Mem engine — tx overlay, `Store` surface, `TxHandle` (staged `*_tx`, commit-merge + wake-at-commit, drop = discard) status: completed depends_on: [mem-engine-foundations] scope: moderate risk: high impact: component level: implementation tags: [wave-6, mem-engine, seam] --- ## Description The seam task, and the wave's load-bearing artifact (engine-mem.md §The tx overlay): the `Store` trait impl plus the tx overlay's full discipline. - **`Store` surface**: the trait methods (`notify`, `stream`, `queue`, `outbox`, `try_lock`, `listen`, `schedule`, `unschedule`, `run_schedules`, `begin_tx`, `with_tx`) implemented over the state core. `queue(name, opts)`/`outbox(name)` create/return handles whose opts ride the handle (queues are names, not config owners — the §3a stamp rules apply identically). Mechanism handles are core-owned trait objects; engine types never appear in signatures. - **The tx overlay**: `begin_tx()` allocates a handle-private overlay. All eleven `*_tx` methods stage effects in it — `enqueue_tx`, `publish_tx`, `publish_with_key_tx`, `notify_tx`, `save_offset_tx`, `get_job_tx`, `get_offset_tx`, `read_since_tx`, `read_from_consumer_tx`, `outbox_enqueue_tx`, `commit`. Reads *inside* the tx merge the overlay over committed state (read- your-own-writes, no round-trips: the tx-read rows' `get_job_tx`/`read_since_tx` behaviors come free, but pin them). Opt-stamping of staged effects runs at *enqueue* time on the overlay (resolution identical to the no-handle paths, per [ADR-020](../architecture/decisions/020-enqueue-opt-semantics-and-bridges.md) §3: `enqueue_tx` stamps plain derived defaults 300/3/5/none; `outbox_enqueue_tx` the 60/5/5 set). - **Commit-merge + wake-at-commit**: commit merges the overlay atomically (single guarded merge) and fires the overlay's pending wakes *in the same operation* — a tx-staged notify delivers nothing until the merge (the discipline pg got wrong and fixed via `pg-fix-tx-wake`; here structural, and pinned from birth by the suite's tx-commit-atomicity rows once integration wires them). - **Drop = discard**: dropping the handle without commit/rollback discards the overlay (rollback explicit form too) — no-ghosts trivially, uncommitted state never touched committed state. `with_tx` composition: `Ok` ⇒ commit, `Err`/drop/panic ⇒ rollback (the N-5 panic posture applies; overlay discard must be panic-safe through the unwind's drop). - **Numeric domains** ([ADR-023](../architecture/decisions/023-fourth-review-round.md) §2): validation at the trait-impl entry, before any storage-layer effect — extents clamp empty, durations reject, boundaries total. Same entry-point rule as the other engines, realized engine-side. Engine-internal unit tests pin the overlay discipline directly (merge atomicity, wake-after-not-before, drop-discard, panic-through- unwind) in the family's mechanism-test posture; the suite column's cross-engine rows land in `mem-engine-integration`. ## Acceptance Criteria - [x] The full `Store`/`TxHandle` trait surfaces implemented over guarded state; no engine types in consumer signatures - [x] Overlay staging → commit-merge → wake-at-commit pinned by unit tests (a tx-staged notify delivers nothing pre-commit, one wake post-commit; enqueue/publish commit-atomicity shapes) - [x] Drop-without-commit and explicit rollback leave zero residue; `with_tx` panic rolls back through the unwind's drop - [x] Tx reads see staged effects; other views see none pre-commit; post-rollback reads' subjects gone (no-ghosts for reads) - [x] Numeric-domain guards at entry (extents/durations/boundaries) pinned per [ADR-023](../architecture/decisions/023-fourth-review-round.md) §2 - [x] `cargo check --target wasm32-unknown-unknown -p alkstore-mem` clean; workspace gates green ## References - docs/architecture/core-contract.md §The seam; §Cross-cutting contracts - docs/architecture/decisions/007-transactional-seam.md; 014-outbox-tx-enqueue.md; 020-enqueue-opt-semantics-and-bridges.md §3; 021-tx-reads-and-value-shape-fixes.md; 023-fourth-review-round.md §2 - docs/architecture/engine-mem.md §The tx overlay (the spec of record) - tasks/pg-fix-tx-wake.md (the failure shape the overlay makes structurally impossible) - tasks/review-wave-5.md appendix (the suite rows whose mem wiring consumes this task) ## Notes > Filled by implementation agent, 2026-10-10. 1. **The whole engine surface landed here, deliberately** — this task is wider than the SQLite/pg seam twins. Those seams rode pre-ported/forked substrate ops and left the store's mechanism methods as wiring stubs; the mem engine has no substrate layer, and this task's description + acceptance criterion 1 pin "the full `Store`/`TxHandle` trait surfaces implemented over guarded state" — a Rust trait impl is all-or-nothing per method. So the seam task realized the mechanisms' committed-state effect layer itself (queue machine, stream log/offsets, TTL registry, scheduler table+runner, notify bridge) and every mechanism handle, with the tx overlay as the discipline centerpiece. **Consequence for the wave's mechanism tasks**: `mem-engine-{notify-listen, streams, queues, locks, scheduler-outbox}` now find their mechanisms *implemented* — their work is the engine-side unit pins their acceptance criteria enumerate (the ADR-010/015/009/008-§7 properties, the mem-owned curve/parser/stats pins, the receiver close arms) plus any depth refinement, against this foundation — not first implementations. Recorded here so those agents don't re-derive the split. 2. **`serde_json = "1"` joined the manifest** (both engine twins carry it): the trait signatures cross `serde_json::Value`, so implementing them requires naming the type. Not a driver dep; the ADR-024 §1 surface (core + tokio sync+time) is otherwise intact, and the wasm32-compile gate stays green with it. 3. **Stage-time id/offset allocation — the pg-shaped story**: `enqueue_tx` /`publish_tx` return the id/offset immediately, so ids and offsets allocate at *stage* time from engine-wide atomic allocators (`State::allocate_job_id`/`allocate_event_offset`). A rolled-back (or panicking) tx's allocation gaps out — legal: ids are the claim-order tiebreak (claims order by id ASC like the SQL engines' enqueue order) and offsets are immutable/never renumbered (gaps ordinary). Cross-handle concurrency: merges serialize; reads yield `offset ASC`, so an event's committed position is always its stage order (no isolation questions arise — an overlay is invisible to every other view until its commit). 4. **Canonical guard order `schedules → queues → streams`** (locks and wakes depend on no other section; documented in `state.rs`). The commit-merge takes queues+streams in one scope; the scheduler tick takes schedules+queues (the fire-and-advance one guarded op — the ADR-009 §4 no-double-fire floor); the wake fire happens *after* the merges land (wake-at-commit is structural: the fire code cannot run first). 5. **Merge mechanics**: `merge_and_wake` drains the overlay into the committed sections and then fires staged notifies (stage order, duplicates ride — per-notify, never coalesced) plus one wake per staged event on the stream's engine-internal feed. The stream subscriber's wake channel is `__alkstore_stream_wake:{stream}` — reserved-prefixed, so consumer `listen`/`notify` can never attach to or fire into it (entry validation rejects the prefix), and the EventReceiver attaches *before* its stored-offset read (no commit can fall between; a buffered wake costs an empty re-poll, never a missed event — the SQLite bridge's trick, now thread-free). 6. **notify's payload encoding run is the codec-typed entry check only**: the wake contract delivers `Wake { channel }` alone (ADR-008 §3), so mem stores no notifications transport — the encoded bytes are dropped after the `Codec`-typed validation (sqLite's transport row has no mem analog). `PayloadTooLarge` is never produced at any size (the ADR-016 §5 non-occurrence arm, pinned by a 2 MiB test). 7. **Lock semantics vs the SQLite substrate**: mem pins renew-past- expiry *refused* (`false`) — the SQLite substrate's `lock_renew` would revive a not-yet-purged lapsed row; the mem task text pins the refusal and the suite's rows read the same way. Same-owner re-`try_lock` grants without refreshing the deadline (the `INSERT OR IGNORE` + owner-match parity); release past expiry stays the holder's (`true`); a lapsed-then-re-acquired row's old-holder release refuses (`false`). Locks are not tx-staged (no `lock_tx` — contract verbatim). 8. **`ScheduleRow` carries the pre-parsed interval + resolved stamps** — not the raw spec/name pair. The name is the map key (upsert by name) and `parse_every_interval` runs once at register; the fire path reads registered values, never the spec (the substrate re-parses per tick — mem's third-owner parser makes that unnecessary). `schedule()` encodes the payload once at register; each fire enqueues those bytes with the registered stamps and the `ScheduleOpts::{priority, expires}` riding themselves, resolved against the fire instant (one clock read per tick). 9. **Runner lease**: owner is runner-unique (`scheduler-leader-{ instance}-{subsecond-nanos}` — an instance counter instead of `process::id`, wasm-clean); renew-before-tick with the loss return *before any fire*; the keep-awake sleep renews on a 2 s cadence (an idle store must not drift past the 10 s TTL); idle sleep 60 s; refused acquire at start = `Err(LeadershipLost)` (no runner exists without the lock); a clean stop releases the lease. Boundary waits ride realtime tokio time while stamps read the engine clock — the tick/catch-up/skip-forward pins run through the crate-internal `run_tick(now)` with the frozen clock, deterministic without realtime waits. 10. **Row predicates mirror the substrate's SQL comparison-for- comparison** (claim: pending-ready or lapsed-processing + budget unspent + expiry unpassed; ack `claim_expires_at >= now`, reclaim `<`, lock purge/renew boundary `<=`, sweep `expires_at <= now`). The test-shaped pins caught one real defect class mid-flight: ack/ack_batch originally *deleted before checking* the predicate — a refused ack must leave the row exactly as it was (the substrate's `DELETE … WHERE` checks per row); fixed, and the uniform-lapse refusal tests now pin it. 11. **Sweep scoping note for later tasks**: `sweep_expired` counts both the dead-letter moves *and* the retention deletions in its return (substrate parity: `live_moved + retained`). A row whose `expires_at` has already passed at enqueue is not claimable and not pre-claim-dead-letterable — it sits for `sweep_expired` (the pre-claim sweep only takes exhausted *claimable-shaped* rows). 12. **`enqueue_tx`/`outbox_enqueue_tx` stamp resolution** uses `stamps_with_override` over the plain (300/3/5/none) and outbox (60/5/5) derived sets exactly as ADR-020 §3 pins; the stamp sets are named functions (one owner), resolution happens after entry-point validation and before any effect lands. ## Summary > Filled on completion, 2026-10-10. Landed the mem engine's seam: the complete `Store` trait surface wired over the guarded state core — `notify`/`listen` (wake substrate bridge with the pinned recv forms and close-at-engine-drop), `stream`/`queue`/ `outbox`/`try_lock` (validated constructors → boxed core-owned handle traits carrying the mem mechanisms' full surfaces: ADR-010 queue state machine with claims/reclaim/dead-letter/sweep and the equal-jitter curve, ADR-015 log+offsets+trim+subscribe, ADR-008 §7 TTL registry, ADR-014 outbox helper with `run_once`), `schedule`/`unschedule`/ `run_schedules` (ADR-009 collapse: `@every`-only third-owner parser, upsert table, the 64-cap bounded catch-up + grid-aligned skip-forward tick under the schedules→queues guard, leadership through the reserved `__alkstore_scheduler` lock with renew-before-tick and the clean-stop/ `LeadershipLost` returns) — plus the load-bearing tx overlay: `begin_tx` allocates the handle-private overlay; all eleven `*_tx` methods stage with entry-point validation, `Codec`-typed encoding, one clock read, ADR-020 §3 stamp resolution (plain 300/3/5/none, outbox 60/5/5, the per-job `max_attempts` override); tx reads merge overlay-over-committed (read-your-own-writes, queue-scoped + dead-visible `get_job_tx`, monotone saves, offset-ASC reads, extent guards); `commit` is the one guarded merge (queues → streams) followed by the wake-at-commit fire (staged notifies + per-event stream wakes — nothing delivers pre-merge, one wake per staged effect); drop (explicit or through an unwinding panic) discards the overlay — no-ghosts by construction. Engine-side arithmetic (`resolution.rs`) is mem's third-owner copy (ADR-012 §2): stamp sets, `delay`-wins-over-`run_at`, relative- `expires`, the reserved-prefix outbox derivation, `stamps_with_override`, and the equal-jitter curve (std-only `RandomState` hash jitter, no `rand`, pinned range). `resolution.rs` — mem's third-owner arithmetic — landed with the tx stamping and is what the mechanism tasks should consume for the cross-engine equivalence rows (suite-pinned). 89 unit tests on `alkstore-mem` (17 foundations + 72 new): the overlay discipline (staged-notify silence pre-commit → one wake post-commit, stamps + commit-atomicity shapes, drop/rollback/panic-discard no-ghosts across job/event/notify/offset, tx reads merged, concurrent-tx merge order, with_tx Ok/Err/panic postures), numeric domains (claim/read extents clamp, duration guards on lock paths, negative-horizon trim, the `@every` grammar matrix, entry validation across every surface), and the mechanism smoke pins (lifecycle, ordering, validity predicate + late-heartbeat refusal, reclaim-eats- attempt, dead-letter strings, sweep both-states + retention, stream FIFO + trim + subscriber replay/wake + close arms, lock contention/expiry, tick/catch-up/skip-forward, leadership contention + clean stop, outbox ack/retry curves). Verified: workspace `cargo test` green (core 25, suite-harness 3, **mem 89**, SQLite 124 lib + 191 suite, pg 124 lib + 25 suite), `cargo clippy --all-targets -- -D warnings` clean, `cargo fmt --check` clean, and `cargo check --target wasm32-unknown-unknown -p alkstore-mem` clean — the wave-6 wasm gate stays green.