mem-engine-seam-tx — the wave's load-bearing seam lands: the complete Store/TxHandle trait surfaces over the guarded state core, with the tx overlay as the discipline centerpiece. Wider than the sqlite/pg seam twins by design (those rode pre-ported substrate ops and left mechanism methods stubbed; mem has no substrate, and the task's acceptance pins the full trait surfaces implemented over guarded state — a Rust trait impl is all-or-nothing per method): the seam task therefore realizes the mechanisms' committed-state effect layer and every mechanism handle itself, and the wave-6 mechanism tasks now own their acceptance-criteria unit pins against this foundation. Store surface: notify (codec-typed encode as the entry check only — the wake delivers the channel alone; PayloadTooLarge never produced, the ADR-016 §5 non-occurrence arm pinned at 2 MiB), listen (WakeReceiver bridge with the pinned recv forms; attach starts from now, one close path at engine drop), stream/queue/outbox/try_lock (validated constructors → boxed core-owned handle traits: the ADR-010 queue state machine with claims/reclaim-eats-attempt/uniform validity predicate/dead-letter/sweep + mem's third-owner equal-jitter curve — std-only RandomState hash jitter, no rand; ADR-015 log+offsets+trim+wake-driven subscribe; ADR-008 §7 TTL registry with renew-past-expiry refused; ADR-014 outbox helper with run_once ack/retry), schedule/unschedule/run_schedules (ADR-009 collapse: @every-only third-owner parser + register-table with pre-parsed interval and resolved stamps; the 64-cap bounded catch-up + grid-aligned skip-forward tick fired-and-advanced under one guarded op (schedules→queues lock order); leadership through the reserved __alkstore_scheduler lock with runner-unique owner, renew-before-tick, keep-awake sleep renewing on cadence, LeadershipLost / clean-stop-Ok returns). Tx overlay: begin_tx allocates the handle-private overlay; all eleven *_tx stage with entry-point validation before any effect, Codec-typed encode, one clock read, ADR-020 §3 stamp resolution (plain 300/3/5/none, outbox 60/5/5, the per-job max_attempts override via resolution.rs's stamps_with_override); tx reads merge overlay over committed (read-your-own-writes: queue-scoped + dead-visible get_job_tx, monotone staged saves, offset-ASC merged reads, extent guards clamping empty); commit consumes the handle, merges under one guarded op (queues→streams, the canonical lock order documented in state.rs), and fires the overlay's pending wakes AFTER the merge — wake-at-commit structural from birth (the pg-fix-tx-wake failure shape cannot exist here); drop (explicit, or through an unwinding panic — the overlay is plain local data, nothing runs on discard) = rollback with no ghosts, trivially. Stage-time id/offset allocation (the pg-shaped story — rolled-back allocations gap out; claims order id ASC, offsets immutable/never renumbered). Stream subscribers ride a reserved-prefix engine-internal wake channel (__alkstore_stream_wake:{stream}) unreachable by consumer entry points, attached before the stored-offset read; notify's transport stores nothing. 89 mem unit tests: the overlay discipline rows the task pins, the numeric-domain rows (extents/durations/boundaries/grammar/validation), and the mechanism smoke pins (the defect-class catch: ack/ack_batch originally deleted before checking the predicate — fixed; a refusal must leave the row exactly as it was). Manifest: serde_json = 1 joins (the trait signatures cross Value; both engine twins carry it; not a driver dep — wasm32 gate stays green). Task file: status completed, Notes (the whole-engine scope disposition + 11 decision-of-record entries incl. lock-renewal delta vs the sqlite substrate, ScheduleRow shape, runner lease mechanics, substrate predicate parity) + Summary filled. Verified: cargo build; workspace cargo test green (core 25, suite harness 3, mem 89, SQLite 191 lib + 25 suite, pg 124 lib + 25 suite + 9 schema); clippy --all-targets -D warnings clean; cargo fmt --check clean; cargo check --target wasm32-unknown-unknown -p alkstore-mem clean
This commit is contained in:
1 parent
6632dcce1f
commit
1c3e96802f
14 files changed
+5375
-70
No files matched your search
+169
-9
@@ -1,7 +1,7 @@
|
||||
---
|
||||
id: mem-engine-seam-tx
|
||||
name: Mem engine — tx overlay, `Store` surface, `TxHandle` (staged `*_tx`, commit-merge + wake-at-commit, drop = discard)
|
||||
status: pending
|
||||
status: completed
|
||||
depends_on: [mem-engine-foundations]
|
||||
scope: moderate
|
||||
risk: high
|
||||
@@ -60,18 +60,18 @@ cross-engine rows land in `mem-engine-integration`.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The full `Store`/`TxHandle` trait surfaces implemented over
|
||||
- [x] The full `Store`/`TxHandle` trait surfaces implemented over
|
||||
guarded state; no engine types in consumer signatures
|
||||
- [ ] Overlay staging → commit-merge → wake-at-commit pinned by unit
|
||||
- [x] Overlay staging → commit-merge → wake-at-commit pinned by unit
|
||||
tests (a tx-staged notify delivers nothing pre-commit, one wake
|
||||
post-commit; enqueue/publish commit-atomicity shapes)
|
||||
- [ ] Drop-without-commit and explicit rollback leave zero residue;
|
||||
- [x] Drop-without-commit and explicit rollback leave zero residue;
|
||||
`with_tx` panic rolls back through the unwind's drop
|
||||
- [ ] Tx reads see staged effects; other views see none pre-commit;
|
||||
- [x] Tx reads see staged effects; other views see none pre-commit;
|
||||
post-rollback reads' subjects gone (no-ghosts for reads)
|
||||
- [ ] Numeric-domain guards at entry (extents/durations/boundaries)
|
||||
- [x] Numeric-domain guards at entry (extents/durations/boundaries)
|
||||
pinned per [ADR-023](../architecture/decisions/023-fourth-review-round.md) §2
|
||||
- [ ] `cargo check --target wasm32-unknown-unknown -p alkstore-mem`
|
||||
- [x] `cargo check --target wasm32-unknown-unknown -p alkstore-mem`
|
||||
clean; workspace gates green
|
||||
|
||||
## References
|
||||
@@ -88,8 +88,168 @@ cross-engine rows land in `mem-engine-integration`.
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
> Filled by implementation agent, 2026-10-10.
|
||||
|
||||
1. **The whole engine surface landed here, deliberately** — this task
|
||||
is wider than the SQLite/pg seam twins. Those seams rode
|
||||
pre-ported/forked substrate ops and left the store's mechanism
|
||||
methods as wiring stubs; the mem engine has no substrate layer, and
|
||||
this task's description + acceptance criterion 1 pin "the full
|
||||
`Store`/`TxHandle` trait surfaces implemented over guarded state"
|
||||
— a Rust trait impl is all-or-nothing per method. So the seam task
|
||||
realized the mechanisms' committed-state effect layer itself (queue
|
||||
machine, stream log/offsets, TTL registry, scheduler table+runner,
|
||||
notify bridge) and every mechanism handle, with the tx overlay as
|
||||
the discipline centerpiece. **Consequence for the wave's mechanism
|
||||
tasks**: `mem-engine-{notify-listen, streams, queues, locks,
|
||||
scheduler-outbox}` now find their mechanisms *implemented* — their
|
||||
work is the engine-side unit pins their acceptance criteria
|
||||
enumerate (the ADR-010/015/009/008-§7 properties, the mem-owned
|
||||
curve/parser/stats pins, the receiver close arms) plus any depth
|
||||
refinement, against this foundation — not first implementations.
|
||||
Recorded here so those agents don't re-derive the split.
|
||||
2. **`serde_json = "1"` joined the manifest** (both engine twins carry
|
||||
it): the trait signatures cross `serde_json::Value`, so
|
||||
implementing them requires naming the type. Not a driver dep; the
|
||||
ADR-024 §1 surface (core + tokio sync+time) is otherwise intact,
|
||||
and the wasm32-compile gate stays green with it.
|
||||
3. **Stage-time id/offset allocation — the pg-shaped story**: `enqueue_tx`
|
||||
/`publish_tx` return the id/offset immediately, so ids and offsets
|
||||
allocate at *stage* time from engine-wide atomic allocators
|
||||
(`State::allocate_job_id`/`allocate_event_offset`). A rolled-back
|
||||
(or panicking) tx's allocation gaps out — legal: ids are the
|
||||
claim-order tiebreak (claims order by id ASC like the SQL engines'
|
||||
enqueue order) and offsets are immutable/never renumbered (gaps
|
||||
ordinary). Cross-handle concurrency: merges serialize; reads yield
|
||||
`offset ASC`, so an event's committed position is always its stage
|
||||
order (no isolation questions arise — an overlay is invisible to
|
||||
every other view until its commit).
|
||||
4. **Canonical guard order `schedules → queues → streams`** (locks and
|
||||
wakes depend on no other section; documented in `state.rs`). The
|
||||
commit-merge takes queues+streams in one scope; the scheduler tick
|
||||
takes schedules+queues (the fire-and-advance one guarded op — the
|
||||
ADR-009 §4 no-double-fire floor); the wake fire happens *after* the
|
||||
merges land (wake-at-commit is structural: the fire code cannot run
|
||||
first).
|
||||
5. **Merge mechanics**: `merge_and_wake` drains the overlay into the
|
||||
committed sections and then fires staged notifies (stage order,
|
||||
duplicates ride — per-notify, never coalesced) plus one wake per
|
||||
staged event on the stream's engine-internal feed. The stream
|
||||
subscriber's wake channel is `__alkstore_stream_wake:{stream}` —
|
||||
reserved-prefixed, so consumer `listen`/`notify` can never attach
|
||||
to or fire into it (entry validation rejects the prefix), and the
|
||||
EventReceiver attaches *before* its stored-offset read (no commit
|
||||
can fall between; a buffered wake costs an empty re-poll, never a
|
||||
missed event — the SQLite bridge's trick, now thread-free).
|
||||
6. **notify's payload encoding run is the codec-typed entry check
|
||||
only**: the wake contract delivers `Wake { channel }` alone
|
||||
(ADR-008 §3), so mem stores no notifications transport — the
|
||||
encoded bytes are dropped after the `Codec`-typed validation
|
||||
(sqLite's transport row has no mem analog). `PayloadTooLarge` is
|
||||
never produced at any size (the ADR-016 §5 non-occurrence arm,
|
||||
pinned by a 2 MiB test).
|
||||
7. **Lock semantics vs the SQLite substrate**: mem pins renew-past-
|
||||
expiry *refused* (`false`) — the SQLite substrate's `lock_renew`
|
||||
would revive a not-yet-purged lapsed row; the mem task text pins
|
||||
the refusal and the suite's rows read the same way. Same-owner
|
||||
re-`try_lock` grants without refreshing the deadline (the
|
||||
`INSERT OR IGNORE` + owner-match parity); release past expiry stays
|
||||
the holder's (`true`); a lapsed-then-re-acquired row's old-holder
|
||||
release refuses (`false`). Locks are not tx-staged (no `lock_tx` —
|
||||
contract verbatim).
|
||||
8. **`ScheduleRow` carries the pre-parsed interval + resolved stamps**
|
||||
— not the raw spec/name pair. The name is the map key (upsert by
|
||||
name) and `parse_every_interval` runs once at register; the fire
|
||||
path reads registered values, never the spec (the substrate
|
||||
re-parses per tick — mem's third-owner parser makes that
|
||||
unnecessary). `schedule()` encodes the payload once at register;
|
||||
each fire enqueues those bytes with the registered stamps and the
|
||||
`ScheduleOpts::{priority, expires}` riding themselves, resolved
|
||||
against the fire instant (one clock read per tick).
|
||||
9. **Runner lease**: owner is runner-unique (`scheduler-leader-{
|
||||
instance}-{subsecond-nanos}` — an instance counter instead of
|
||||
`process::id`, wasm-clean); renew-before-tick with the loss return
|
||||
*before any fire*; the keep-awake sleep renews on a 2 s cadence
|
||||
(an idle store must not drift past the 10 s TTL); idle sleep 60 s;
|
||||
refused acquire at start = `Err(LeadershipLost)` (no runner exists
|
||||
without the lock); a clean stop releases the lease. Boundary waits
|
||||
ride realtime tokio time while stamps read the engine clock — the
|
||||
tick/catch-up/skip-forward pins run through the crate-internal
|
||||
`run_tick(now)` with the frozen clock, deterministic without
|
||||
realtime waits.
|
||||
10. **Row predicates mirror the substrate's SQL comparison-for-
|
||||
comparison** (claim: pending-ready or lapsed-processing + budget
|
||||
unspent + expiry unpassed; ack `claim_expires_at >= now`, reclaim
|
||||
`<`, lock purge/renew boundary `<=`, sweep `expires_at <= now`).
|
||||
The test-shaped pins caught one real defect class mid-flight:
|
||||
ack/ack_batch originally *deleted before checking* the predicate —
|
||||
a refused ack must leave the row exactly as it was (the substrate's
|
||||
`DELETE … WHERE` checks per row); fixed, and the uniform-lapse
|
||||
refusal tests now pin it.
|
||||
11. **Sweep scoping note for later tasks**: `sweep_expired` counts both
|
||||
the dead-letter moves *and* the retention deletions in its return
|
||||
(substrate parity: `live_moved + retained`). A row whose
|
||||
`expires_at` has already passed at enqueue is not claimable and
|
||||
not pre-claim-dead-letterable — it sits for `sweep_expired`
|
||||
(the pre-claim sweep only takes exhausted *claimable-shaped* rows).
|
||||
12. **`enqueue_tx`/`outbox_enqueue_tx` stamp resolution** uses
|
||||
`stamps_with_override` over the plain (300/3/5/none) and outbox
|
||||
(60/5/5) derived sets exactly as ADR-020 §3 pins; the stamp sets
|
||||
are named functions (one owner), resolution happens after
|
||||
entry-point validation and before any effect lands.
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
> Filled on completion, 2026-10-10.
|
||||
|
||||
Landed the mem engine's seam: the complete `Store` trait surface wired
|
||||
over the guarded state core — `notify`/`listen` (wake substrate bridge
|
||||
with the pinned recv forms and close-at-engine-drop), `stream`/`queue`/
|
||||
`outbox`/`try_lock` (validated constructors → boxed core-owned handle
|
||||
traits carrying the mem mechanisms' full surfaces: ADR-010 queue state
|
||||
machine with claims/reclaim/dead-letter/sweep and the equal-jitter
|
||||
curve, ADR-015 log+offsets+trim+subscribe, ADR-008 §7 TTL registry,
|
||||
ADR-014 outbox helper with `run_once`), `schedule`/`unschedule`/
|
||||
`run_schedules` (ADR-009 collapse: `@every`-only third-owner parser, upsert
|
||||
table, the 64-cap bounded catch-up + grid-aligned skip-forward tick
|
||||
under the schedules→queues guard, leadership through the reserved
|
||||
`__alkstore_scheduler` lock with renew-before-tick and the clean-stop/
|
||||
`LeadershipLost` returns) — plus the load-bearing tx overlay: `begin_tx`
|
||||
allocates the handle-private overlay; all eleven `*_tx` methods stage
|
||||
with entry-point validation, `Codec`-typed encoding, one clock read,
|
||||
ADR-020 §3 stamp resolution (plain 300/3/5/none, outbox 60/5/5, the
|
||||
per-job `max_attempts` override); tx reads merge overlay-over-committed
|
||||
(read-your-own-writes, queue-scoped + dead-visible `get_job_tx`,
|
||||
monotone saves, offset-ASC reads, extent guards); `commit` is the one
|
||||
guarded merge (queues → streams) followed by the wake-at-commit fire
|
||||
(staged notifies + per-event stream wakes — nothing delivers pre-merge,
|
||||
one wake per staged effect); drop (explicit or through an unwinding
|
||||
panic) discards the overlay — no-ghosts by construction. Engine-side
|
||||
arithmetic (`resolution.rs`) is mem's third-owner copy
|
||||
(ADR-012 §2): stamp sets, `delay`-wins-over-`run_at`, relative-
|
||||
`expires`, the reserved-prefix outbox derivation, `stamps_with_override`,
|
||||
and the equal-jitter curve (std-only `RandomState` hash jitter, no
|
||||
`rand`, pinned range).
|
||||
|
||||
`resolution.rs` — mem's third-owner arithmetic — landed with
|
||||
the tx stamping and is what the mechanism tasks should consume for the
|
||||
cross-engine equivalence rows (suite-pinned).
|
||||
|
||||
89 unit tests on `alkstore-mem` (17 foundations + 72 new): the overlay
|
||||
discipline (staged-notify silence pre-commit → one wake post-commit,
|
||||
stamps + commit-atomicity shapes, drop/rollback/panic-discard
|
||||
no-ghosts across job/event/notify/offset, tx reads merged,
|
||||
concurrent-tx merge order, with_tx Ok/Err/panic postures), numeric
|
||||
domains (claim/read extents clamp, duration guards on lock paths,
|
||||
negative-horizon trim, the `@every` grammar matrix, entry validation
|
||||
across every surface), and the mechanism smoke pins (lifecycle,
|
||||
ordering, validity predicate + late-heartbeat refusal, reclaim-eats-
|
||||
attempt, dead-letter strings, sweep both-states + retention, stream
|
||||
FIFO + trim + subscriber replay/wake + close arms, lock
|
||||
contention/expiry, tick/catch-up/skip-forward, leadership contention +
|
||||
clean stop, outbox ack/retry curves). Verified: workspace `cargo test`
|
||||
green (core 25, suite-harness 3, **mem 89**, SQLite 124 lib + 191
|
||||
suite, pg 124 lib + 25 suite), `cargo clippy --all-targets -- -D
|
||||
warnings` clean, `cargo fmt --check` clean, and
|
||||
`cargo check --target wasm32-unknown-unknown -p alkstore-mem` clean —
|
||||
the wave-6 wasm gate stays green.
|
||||
Reference in new issue
Block a user