mem-engine-seam-tx — the wave's load-bearing seam lands: the complete Store/TxHandle trait surfaces over the guarded state core, with the tx overlay as the discipline centerpiece. Wider than the sqlite/pg seam twins by design (those rode pre-ported substrate ops and left mechanism methods stubbed; mem has no substrate, and the task's acceptance pins the full trait surfaces implemented over guarded state — a Rust trait impl is all-or-nothing per method): the seam task therefore realizes the mechanisms' committed-state effect layer and every mechanism handle itself, and the wave-6 mechanism tasks now own their acceptance-criteria unit pins against this foundation. Store surface: notify (codec-typed encode as the entry check only — the wake delivers the channel alone; PayloadTooLarge never produced, the ADR-016 §5 non-occurrence arm pinned at 2 MiB), listen (WakeReceiver bridge with the pinned recv forms; attach starts from now, one close path at engine drop), stream/queue/outbox/try_lock (validated constructors → boxed core-owned handle traits: the ADR-010 queue state machine with claims/reclaim-eats-attempt/uniform validity predicate/dead-letter/sweep + mem's third-owner equal-jitter curve — std-only RandomState hash jitter, no rand; ADR-015 log+offsets+trim+wake-driven subscribe; ADR-008 §7 TTL registry with renew-past-expiry refused; ADR-014 outbox helper with run_once ack/retry), schedule/unschedule/run_schedules (ADR-009 collapse: @every-only third-owner parser + register-table with pre-parsed interval and resolved stamps; the 64-cap bounded catch-up + grid-aligned skip-forward tick fired-and-advanced under one guarded op (schedules→queues lock order); leadership through the reserved __alkstore_scheduler lock with runner-unique owner, renew-before-tick, keep-awake sleep renewing on cadence, LeadershipLost / clean-stop-Ok returns). Tx overlay: begin_tx allocates the handle-private overlay; all eleven *_tx stage with entry-point validation before any effect, Codec-typed encode, one clock read, ADR-020 §3 stamp resolution (plain 300/3/5/none, outbox 60/5/5, the per-job max_attempts override via resolution.rs's stamps_with_override); tx reads merge overlay over committed (read-your-own-writes: queue-scoped + dead-visible get_job_tx, monotone staged saves, offset-ASC merged reads, extent guards clamping empty); commit consumes the handle, merges under one guarded op (queues→streams, the canonical lock order documented in state.rs), and fires the overlay's pending wakes AFTER the merge — wake-at-commit structural from birth (the pg-fix-tx-wake failure shape cannot exist here); drop (explicit, or through an unwinding panic — the overlay is plain local data, nothing runs on discard) = rollback with no ghosts, trivially. Stage-time id/offset allocation (the pg-shaped story — rolled-back allocations gap out; claims order id ASC, offsets immutable/never renumbered). Stream subscribers ride a reserved-prefix engine-internal wake channel (__alkstore_stream_wake:{stream}) unreachable by consumer entry points, attached before the stored-offset read; notify's transport stores nothing. 89 mem unit tests: the overlay discipline rows the task pins, the numeric-domain rows (extents/durations/boundaries/grammar/validation), and the mechanism smoke pins (the defect-class catch: ack/ack_batch originally deleted before checking the predicate — fixed; a refusal must leave the row exactly as it was). Manifest: serde_json = 1 joins (the trait signatures cross Value; both engine twins carry it; not a driver dep — wasm32 gate stays green). Task file: status completed, Notes (the whole-engine scope disposition + 11 decision-of-record entries incl. lock-renewal delta vs the sqlite substrate, ScheduleRow shape, runner lease mechanics, substrate predicate parity) + Summary filled. Verified: cargo build; workspace cargo test green (core 25, suite harness 3, mem 89, SQLite 191 lib + 25 suite, pg 124 lib + 25 suite + 9 schema); clippy --all-targets -D warnings clean; cargo fmt --check clean; cargo check --target wasm32-unknown-unknown -p alkstore-mem clean

This commit is contained in:
glm-5.3-flash committed 2026-10-10 11:55:39 +00:00
1 parent 6632dcce1f
commit 1c3e96802f
14 files changed
+5375 -70

No files matched your search

+169 -9
View File
@@ -1,7 +1,7 @@
---
id: mem-engine-seam-tx
name: Mem engine — tx overlay, `Store` surface, `TxHandle` (staged `*_tx`, commit-merge + wake-at-commit, drop = discard)
status: pending
status: completed
depends_on: [mem-engine-foundations]
scope: moderate
risk: high
@@ -60,18 +60,18 @@ cross-engine rows land in `mem-engine-integration`.
## Acceptance Criteria
- [ ] The full `Store`/`TxHandle` trait surfaces implemented over
- [x] The full `Store`/`TxHandle` trait surfaces implemented over
guarded state; no engine types in consumer signatures
- [ ] Overlay staging → commit-merge → wake-at-commit pinned by unit
- [x] Overlay staging → commit-merge → wake-at-commit pinned by unit
tests (a tx-staged notify delivers nothing pre-commit, one wake
post-commit; enqueue/publish commit-atomicity shapes)
- [ ] Drop-without-commit and explicit rollback leave zero residue;
- [x] Drop-without-commit and explicit rollback leave zero residue;
`with_tx` panic rolls back through the unwind's drop
- [ ] Tx reads see staged effects; other views see none pre-commit;
- [x] Tx reads see staged effects; other views see none pre-commit;
post-rollback reads' subjects gone (no-ghosts for reads)
- [ ] Numeric-domain guards at entry (extents/durations/boundaries)
- [x] Numeric-domain guards at entry (extents/durations/boundaries)
pinned per [ADR-023](../architecture/decisions/023-fourth-review-round.md) §2
- [ ] `cargo check --target wasm32-unknown-unknown -p alkstore-mem`
- [x] `cargo check --target wasm32-unknown-unknown -p alkstore-mem`
clean; workspace gates green
## References
@@ -88,8 +88,168 @@ cross-engine rows land in `mem-engine-integration`.
## Notes
> To be filled by implementation agent
> Filled by implementation agent, 2026-10-10.
1. **The whole engine surface landed here, deliberately** — this task
is wider than the SQLite/pg seam twins. Those seams rode
pre-ported/forked substrate ops and left the store's mechanism
methods as wiring stubs; the mem engine has no substrate layer, and
this task's description + acceptance criterion 1 pin "the full
`Store`/`TxHandle` trait surfaces implemented over guarded state"
— a Rust trait impl is all-or-nothing per method. So the seam task
realized the mechanisms' committed-state effect layer itself (queue
machine, stream log/offsets, TTL registry, scheduler table+runner,
notify bridge) and every mechanism handle, with the tx overlay as
the discipline centerpiece. **Consequence for the wave's mechanism
tasks**: `mem-engine-{notify-listen, streams, queues, locks,
scheduler-outbox}` now find their mechanisms *implemented* — their
work is the engine-side unit pins their acceptance criteria
enumerate (the ADR-010/015/009/008-§7 properties, the mem-owned
curve/parser/stats pins, the receiver close arms) plus any depth
refinement, against this foundation — not first implementations.
Recorded here so those agents don't re-derive the split.
2. **`serde_json = "1"` joined the manifest** (both engine twins carry
it): the trait signatures cross `serde_json::Value`, so
implementing them requires naming the type. Not a driver dep; the
ADR-024 §1 surface (core + tokio sync+time) is otherwise intact,
and the wasm32-compile gate stays green with it.
3. **Stage-time id/offset allocation — the pg-shaped story**: `enqueue_tx`
/`publish_tx` return the id/offset immediately, so ids and offsets
allocate at *stage* time from engine-wide atomic allocators
(`State::allocate_job_id`/`allocate_event_offset`). A rolled-back
(or panicking) tx's allocation gaps out — legal: ids are the
claim-order tiebreak (claims order by id ASC like the SQL engines'
enqueue order) and offsets are immutable/never renumbered (gaps
ordinary). Cross-handle concurrency: merges serialize; reads yield
`offset ASC`, so an event's committed position is always its stage
order (no isolation questions arise — an overlay is invisible to
every other view until its commit).
4. **Canonical guard order `schedules → queues → streams`** (locks and
wakes depend on no other section; documented in `state.rs`). The
commit-merge takes queues+streams in one scope; the scheduler tick
takes schedules+queues (the fire-and-advance one guarded op — the
ADR-009 §4 no-double-fire floor); the wake fire happens *after* the
merges land (wake-at-commit is structural: the fire code cannot run
first).
5. **Merge mechanics**: `merge_and_wake` drains the overlay into the
committed sections and then fires staged notifies (stage order,
duplicates ride — per-notify, never coalesced) plus one wake per
staged event on the stream's engine-internal feed. The stream
subscriber's wake channel is `__alkstore_stream_wake:{stream}` —
reserved-prefixed, so consumer `listen`/`notify` can never attach
to or fire into it (entry validation rejects the prefix), and the
EventReceiver attaches *before* its stored-offset read (no commit
can fall between; a buffered wake costs an empty re-poll, never a
missed event — the SQLite bridge's trick, now thread-free).
6. **notify's payload encoding run is the codec-typed entry check
only**: the wake contract delivers `Wake { channel }` alone
(ADR-008 §3), so mem stores no notifications transport — the
encoded bytes are dropped after the `Codec`-typed validation
(sqLite's transport row has no mem analog). `PayloadTooLarge` is
never produced at any size (the ADR-016 §5 non-occurrence arm,
pinned by a 2 MiB test).
7. **Lock semantics vs the SQLite substrate**: mem pins renew-past-
expiry *refused* (`false`) — the SQLite substrate's `lock_renew`
would revive a not-yet-purged lapsed row; the mem task text pins
the refusal and the suite's rows read the same way. Same-owner
re-`try_lock` grants without refreshing the deadline (the
`INSERT OR IGNORE` + owner-match parity); release past expiry stays
the holder's (`true`); a lapsed-then-re-acquired row's old-holder
release refuses (`false`). Locks are not tx-staged (no `lock_tx` —
contract verbatim).
8. **`ScheduleRow` carries the pre-parsed interval + resolved stamps**
— not the raw spec/name pair. The name is the map key (upsert by
name) and `parse_every_interval` runs once at register; the fire
path reads registered values, never the spec (the substrate
re-parses per tick — mem's third-owner parser makes that
unnecessary). `schedule()` encodes the payload once at register;
each fire enqueues those bytes with the registered stamps and the
`ScheduleOpts::{priority, expires}` riding themselves, resolved
against the fire instant (one clock read per tick).
9. **Runner lease**: owner is runner-unique (`scheduler-leader-{
instance}-{subsecond-nanos}` — an instance counter instead of
`process::id`, wasm-clean); renew-before-tick with the loss return
*before any fire*; the keep-awake sleep renews on a 2 s cadence
(an idle store must not drift past the 10 s TTL); idle sleep 60 s;
refused acquire at start = `Err(LeadershipLost)` (no runner exists
without the lock); a clean stop releases the lease. Boundary waits
ride realtime tokio time while stamps read the engine clock — the
tick/catch-up/skip-forward pins run through the crate-internal
`run_tick(now)` with the frozen clock, deterministic without
realtime waits.
10. **Row predicates mirror the substrate's SQL comparison-for-
comparison** (claim: pending-ready or lapsed-processing + budget
unspent + expiry unpassed; ack `claim_expires_at >= now`, reclaim
`<`, lock purge/renew boundary `<=`, sweep `expires_at <= now`).
The test-shaped pins caught one real defect class mid-flight:
ack/ack_batch originally *deleted before checking* the predicate —
a refused ack must leave the row exactly as it was (the substrate's
`DELETE … WHERE` checks per row); fixed, and the uniform-lapse
refusal tests now pin it.
11. **Sweep scoping note for later tasks**: `sweep_expired` counts both
the dead-letter moves *and* the retention deletions in its return
(substrate parity: `live_moved + retained`). A row whose
`expires_at` has already passed at enqueue is not claimable and
not pre-claim-dead-letterable — it sits for `sweep_expired`
(the pre-claim sweep only takes exhausted *claimable-shaped* rows).
12. **`enqueue_tx`/`outbox_enqueue_tx` stamp resolution** uses
`stamps_with_override` over the plain (300/3/5/none) and outbox
(60/5/5) derived sets exactly as ADR-020 §3 pins; the stamp sets
are named functions (one owner), resolution happens after
entry-point validation and before any effect lands.
## Summary
> To be filled on completion
> Filled on completion, 2026-10-10.
Landed the mem engine's seam: the complete `Store` trait surface wired
over the guarded state core — `notify`/`listen` (wake substrate bridge
with the pinned recv forms and close-at-engine-drop), `stream`/`queue`/
`outbox`/`try_lock` (validated constructors → boxed core-owned handle
traits carrying the mem mechanisms' full surfaces: ADR-010 queue state
machine with claims/reclaim/dead-letter/sweep and the equal-jitter
curve, ADR-015 log+offsets+trim+subscribe, ADR-008 §7 TTL registry,
ADR-014 outbox helper with `run_once`), `schedule`/`unschedule`/
`run_schedules` (ADR-009 collapse: `@every`-only third-owner parser, upsert
table, the 64-cap bounded catch-up + grid-aligned skip-forward tick
under the schedules→queues guard, leadership through the reserved
`__alkstore_scheduler` lock with renew-before-tick and the clean-stop/
`LeadershipLost` returns) — plus the load-bearing tx overlay: `begin_tx`
allocates the handle-private overlay; all eleven `*_tx` methods stage
with entry-point validation, `Codec`-typed encoding, one clock read,
ADR-020 §3 stamp resolution (plain 300/3/5/none, outbox 60/5/5, the
per-job `max_attempts` override); tx reads merge overlay-over-committed
(read-your-own-writes, queue-scoped + dead-visible `get_job_tx`,
monotone saves, offset-ASC reads, extent guards); `commit` is the one
guarded merge (queues → streams) followed by the wake-at-commit fire
(staged notifies + per-event stream wakes — nothing delivers pre-merge,
one wake per staged effect); drop (explicit or through an unwinding
panic) discards the overlay — no-ghosts by construction. Engine-side
arithmetic (`resolution.rs`) is mem's third-owner copy
(ADR-012 §2): stamp sets, `delay`-wins-over-`run_at`, relative-
`expires`, the reserved-prefix outbox derivation, `stamps_with_override`,
and the equal-jitter curve (std-only `RandomState` hash jitter, no
`rand`, pinned range).
`resolution.rs` — mem's third-owner arithmetic — landed with
the tx stamping and is what the mechanism tasks should consume for the
cross-engine equivalence rows (suite-pinned).
89 unit tests on `alkstore-mem` (17 foundations + 72 new): the overlay
discipline (staged-notify silence pre-commit → one wake post-commit,
stamps + commit-atomicity shapes, drop/rollback/panic-discard
no-ghosts across job/event/notify/offset, tx reads merged,
concurrent-tx merge order, with_tx Ok/Err/panic postures), numeric
domains (claim/read extents clamp, duration guards on lock paths,
negative-horizon trim, the `@every` grammar matrix, entry validation
across every surface), and the mechanism smoke pins (lifecycle,
ordering, validity predicate + late-heartbeat refusal, reclaim-eats-
attempt, dead-letter strings, sweep both-states + retention, stream
FIFO + trim + subscriber replay/wake + close arms, lock
contention/expiry, tick/catch-up/skip-forward, leadership contention +
clean stop, outbox ack/retry curves). Verified: workspace `cargo test`
green (core 25, suite-harness 3, **mem 89**, SQLite 124 lib + 191
suite, pg 124 lib + 25 suite), `cargo clippy --all-targets -- -D
warnings` clean, `cargo fmt --check` clean, and
`cargo check --target wasm32-unknown-unknown -p alkstore-mem` clean —
the wave-6 wasm gate stays green.