mem-engine-seam-tx — the wave's load-bearing seam lands: the complete Store/TxHandle trait surfaces over the guarded state core, with the tx overlay as the discipline centerpiece. Wider than the sqlite/pg seam twins by design (those rode pre-ported substrate ops and left mechanism methods stubbed; mem has no substrate, and the task's acceptance pins the full trait surfaces implemented over guarded state — a Rust trait impl is all-or-nothing per method): the seam task therefore realizes the mechanisms' committed-state effect layer and every mechanism handle itself, and the wave-6 mechanism tasks now own their acceptance-criteria unit pins against this foundation. Store surface: notify (codec-typed encode as the entry check only — the wake delivers the channel alone; PayloadTooLarge never produced, the ADR-016 §5 non-occurrence arm pinned at 2 MiB), listen (WakeReceiver bridge with the pinned recv forms; attach starts from now, one close path at engine drop), stream/queue/outbox/try_lock (validated constructors → boxed core-owned handle traits: the ADR-010 queue state machine with claims/reclaim-eats-attempt/uniform validity predicate/dead-letter/sweep + mem's third-owner equal-jitter curve — std-only RandomState hash jitter, no rand; ADR-015 log+offsets+trim+wake-driven subscribe; ADR-008 §7 TTL registry with renew-past-expiry refused; ADR-014 outbox helper with run_once ack/retry), schedule/unschedule/run_schedules (ADR-009 collapse: @every-only third-owner parser + register-table with pre-parsed interval and resolved stamps; the 64-cap bounded catch-up + grid-aligned skip-forward tick fired-and-advanced under one guarded op (schedules→queues lock order); leadership through the reserved __alkstore_scheduler lock with runner-unique owner, renew-before-tick, keep-awake sleep renewing on cadence, LeadershipLost / clean-stop-Ok returns). Tx overlay: begin_tx allocates the handle-private overlay; all eleven *_tx stage with entry-point validation before any effect, Codec-typed encode, one clock read, ADR-020 §3 stamp resolution (plain 300/3/5/none, outbox 60/5/5, the per-job max_attempts override via resolution.rs's stamps_with_override); tx reads merge overlay over committed (read-your-own-writes: queue-scoped + dead-visible get_job_tx, monotone staged saves, offset-ASC merged reads, extent guards clamping empty); commit consumes the handle, merges under one guarded op (queues→streams, the canonical lock order documented in state.rs), and fires the overlay's pending wakes AFTER the merge — wake-at-commit structural from birth (the pg-fix-tx-wake failure shape cannot exist here); drop (explicit, or through an unwinding panic — the overlay is plain local data, nothing runs on discard) = rollback with no ghosts, trivially. Stage-time id/offset allocation (the pg-shaped story — rolled-back allocations gap out; claims order id ASC, offsets immutable/never renumbered). Stream subscribers ride a reserved-prefix engine-internal wake channel (__alkstore_stream_wake:{stream}) unreachable by consumer entry points, attached before the stored-offset read; notify's transport stores nothing. 89 mem unit tests: the overlay discipline rows the task pins, the numeric-domain rows (extents/durations/boundaries/grammar/validation), and the mechanism smoke pins (the defect-class catch: ack/ack_batch originally deleted before checking the predicate — fixed; a refusal must leave the row exactly as it was). Manifest: serde_json = 1 joins (the trait signatures cross Value; both engine twins carry it; not a driver dep — wasm32 gate stays green). Task file: status completed, Notes (the whole-engine scope disposition + 11 decision-of-record entries incl. lock-renewal delta vs the sqlite substrate, ScheduleRow shape, runner lease mechanics, substrate predicate parity) + Summary filled. Verified: cargo build; workspace cargo test green (core 25, suite harness 3, mem 89, SQLite 191 lib + 25 suite, pg 124 lib + 25 suite + 9 schema); clippy --all-targets -D warnings clean; cargo fmt --check clean; cargo check --target wasm32-unknown-unknown -p alkstore-mem clean

This commit is contained in:
glm-5.3-flash committed 2026-10-10 11:55:39 +00:00
1 parent 6632dcce1f
commit 1c3e96802f
14 files changed
+5375 -70

No files matched your search

+1
View File
@@ -7,6 +7,7 @@ repository.workspace = true
[dependencies]
alkstore = { version = "0.1", path = "../alkstore" }
serde_json = "1"
tokio = { version = "1", features = ["sync", "time"] }
[dev-dependencies]
+15 -13
View File
@@ -10,7 +10,7 @@
//! The dependency on `alkstore-mem` *is* the ephemerality declaration;
//! compile-time engine identity (this crate) carries the honesty —
//! no runtime surface is fabricated to detect it. Each
//! `MemStore::new` yields a fully isolated instance sharing nothing;
//! [`MemStore::new`] yields a fully isolated instance sharing nothing;
//! teardown is drop, and engine drop closes open receivers.
//!
//! **Full contract v1** (ADR-024 §3): every mechanism, every guarantee
@@ -19,7 +19,7 @@
//! pinned: the contract suite's third `StoreFactory` column (ADR-022)
//! runs this engine under a current-thread runtime flavor.
//!
//! **wasm32** ([ADR-024] §5): this crate compiles clean for
//! **wasm32** (ADR-024 §5): this crate compiles clean for
//! `wasm32-unknown-unknown` — tokio restricted to the wasm-supported
//! subset (sync + time), no driver dependencies, no `spawn_blocking`
//! anywhere; internals are current-thread-runtime-clean by
@@ -31,21 +31,23 @@
//! The store instance *is* the state: guarded per-mechanism sections
//! (`crate::state`) behind short-held synchronous internal locks,
//! tokio primitives for cross-await coordination. The tx seam is an
//! overlay (stage → commit-merge → wake-at-commit; drop = discard);
//! wakes route through in-process channels (`crate::wakes`) —
//! delivered at the commit boundary directly to each open receiver,
//! per-subscriber FIFO by commit order, never coalesced (no batching
//! boundary exists).
//!
//! Wave 6 is under construction (ADR-024): the foundations —
//! `MemStore::new`, the guarded state core, the wake-routing
//! substrate, the clock accessor, entry-point validation — are laid;
//! the `Store`/`TxHandle` trait surfaces and the mechanism handles
//! land with the wave's seam and mechanism tasks.
//! overlay ([`crate::tx`] — stage in the handle-private overlay, commit
//! merges atomically + wakes at commit, drop discards); wakes route
//! through in-process channels (`crate::wakes`) — delivered at the
//! commit boundary directly to each open receiver, per-subscriber FIFO
//! by commit order, never coalesced (no batching boundary exists).
mod clock;
mod lock;
mod notify;
mod outbox;
mod queue;
mod resolution;
mod scheduler;
mod state;
mod store;
mod stream;
mod tx;
mod validation;
mod wakes;
+327
View File
@@ -0,0 +1,327 @@
//! The named-locks mechanism's mem arm — the in-memory TTL registry
//! (engine-mem.md §Mapping the contract, locks row; ADR-008 §7's
//! guarantee row).
//!
//! - Registry entries carry owner + deadline; expiry is a clock check
//! at the try/renew boundaries (no ambient revoker exists). After
//! TTL expiry exclusion lapses *silently* — no revocation event —
//! and the name is re-acquirable; the holder may still release.
//! - Contention on one store instance is serialized by the guarded
//! section: first wins, the loser reads `None` cleanly (a value,
//! never an error). A re-`try_lock` by the *same* owner while held
//! grants (no second lease row exists to fight) without refreshing
//! the deadline — the SQL engines' `INSERT OR IGNORE` + owner-match
//! shape, mirrored.
//! - **The duration guard at entry** (ADR-023 §2, durations kind):
//! `ttl <= 0` rejects on `try_lock` *and* `renew` with the opaque
//! `Database` error — never a silent immediately-expired lease.
//! - Locks are not part of the tx seam (no `lock_tx` — contract
//! verbatim): the overlay neither sees nor stages locks. The
//! scheduler's leadership lock rides this registry through the
//! engine-internal acquire/renew path (a reserved name consumer
//! entry points can never collide with).
use std::sync::Arc;
use alkstore::{BoxedFuture, Error, Lock, Result, validate_local_name, validate_shared_name};
use crate::clock::Clock;
use crate::state::Guard;
use crate::state::{LockState, State};
/// The scheduler's leadership-lock name (ADR-009 §4 — the reserved-
/// prefix derived name in the lock namespace; engine-internal: every
/// consumer entry point rejects the same prefix, so a consumer can
/// never collide with it).
pub(crate) const SCHEDULER_LEADER_LOCK: &str = "__alkstore_scheduler";
fn duration_error(ttl: i64) -> Error {
Error::database(std::io::Error::other(format!(
"ttl must be a positive duration (seconds), got {ttl}"
)))
}
/// The registry acquire (the one owner of the row-level mechanical):
/// purge the name's expired entry, grant iff the name is absent or
/// already owned by this owner. `now` is the caller's clock read.
pub(crate) fn lock_acquire(l: &mut LockState, name: &str, owner: &str, ttl: i64, now: i64) -> bool {
if let Some(entry) = l.entries.get(name)
&& entry.expires_at <= now
{
l.entries.remove(name);
}
match l.entries.get(name) {
Some(entry) => entry.owner == owner,
None => {
l.entries.insert(
name.to_string(),
crate::state::LockEntry {
owner: owner.to_string(),
expires_at: now.saturating_add(ttl),
},
);
true
}
}
}
/// The registry renew: a new full TTL window from now. Refused
/// (`false`) when the name is held by another owner, or past expiry —
/// a renew can never revive a lapsed exclusion (the mem pin: renew
/// past expiry is refused, the holder's window is the TTL's).
pub(crate) fn lock_renew(l: &mut LockState, name: &str, owner: &str, ttl: i64, now: i64) -> bool {
let Some(entry) = l.entries.get_mut(name) else {
return false;
};
if entry.owner != owner || entry.expires_at <= now {
return false;
}
entry.expires_at = now.saturating_add(ttl);
true
}
/// The registry release: delete the owner's row only. `false` =
/// re-acquired elsewhere (a no-op, not an error). The holder may
/// release past expiry — exclusion lapsed silently; releasing the
/// lapsed row is still the holder's.
pub(crate) fn lock_release(l: &mut LockState, name: &str, owner: &str) -> bool {
match l.entries.get(name) {
Some(entry) if entry.owner == owner => {
l.entries.remove(name);
true
}
_ => false,
}
}
/// The consumer `try_lock` path (validation + duration guard + the
/// guarded acquire; `Ok(None)` is the held-by-someone-else arm).
pub(crate) fn try_lock(
state: Arc<State>,
clock: Arc<Clock>,
name: &str,
owner: &str,
ttl: i64,
) -> Result<Option<Box<dyn Lock>>> {
validate_shared_name(name)?;
validate_local_name(owner)?;
if ttl <= 0 {
return Err(duration_error(ttl));
}
let now = clock.unix_now();
let granted = {
let mut l = state.locks.guard();
lock_acquire(&mut l, name, owner, ttl, now)
};
if granted {
Ok(Some(Box::new(MemLock {
name: name.to_string(),
owner: owner.to_string(),
state,
clock,
})))
} else {
Ok(None)
}
}
/// The held named-lock handle (the repeatable `renew` case and the
/// consuming `release`).
pub struct MemLock {
name: String,
owner: String,
state: Arc<State>,
clock: Arc<Clock>,
}
impl std::fmt::Debug for MemLock {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("MemLock")
.field("name", &self.name)
.finish_non_exhaustive()
}
}
impl MemLock {
/// The engine-internal leadership lease (the scheduler's): the
/// same registry machinery without consumer validation (the name
/// is engine-derived and reserved).
pub(crate) fn new_internal(
name: &str,
owner: &str,
state: Arc<State>,
clock: Arc<Clock>,
) -> Self {
Self {
name: name.to_string(),
owner: owner.to_string(),
state,
clock,
}
}
pub(crate) fn acquire_now(&self, ttl: i64, now: i64) -> bool {
let mut l = self.state.locks.guard();
lock_acquire(&mut l, &self.name, &self.owner, ttl, now)
}
pub(crate) fn renew_at(&self, ttl: i64, now: i64) -> bool {
let mut l = self.state.locks.guard();
lock_renew(&mut l, &self.name, &self.owner, ttl, now)
}
pub(crate) fn release_now(&self) -> bool {
let mut l = self.state.locks.guard();
lock_release(&mut l, &self.name, &self.owner)
}
}
impl Lock for MemLock {
fn name(&self) -> &str {
&self.name
}
fn renew<'a>(&'a self, ttl: i64) -> BoxedFuture<'a, Result<bool>> {
if ttl <= 0 {
return Box::pin(async move { Err(duration_error(ttl)) });
}
let now = self.clock.unix_now();
let granted = self.renew_at(ttl, now);
Box::pin(async move { Ok(granted) })
}
fn release(self: Box<Self>) -> BoxedFuture<'static, Result<bool>> {
Box::pin(async move { Ok(self.release_now()) })
}
}
#[cfg(test)]
mod tests {
use crate::store::MemStore;
use alkstore::Store;
#[tokio::test(flavor = "current_thread")]
async fn contention_grants_once_and_the_loser_reads_none() {
let store = MemStore::new();
store.clock.freeze(1_000);
let first = store.try_lock("db", "w1", 60).await.unwrap();
assert!(first.is_some(), "the first acquire wins");
let loser = store.try_lock("db", "w2", 60).await.unwrap();
assert!(
loser.is_none(),
"the loser is a clean None (a value, never an error)"
);
let same_owner = store.try_lock("db", "w1", 60).await.unwrap();
assert!(
same_owner.is_some(),
"the owner re-grants (no second lease row)"
);
drop(first);
drop(loser);
if let Some(re) = same_owner {
assert!(re.renew(60).await.unwrap(), "renew is an absolute reset");
}
}
#[tokio::test(flavor = "current_thread")]
async fn duration_guards_reject_non_positive_ttls() {
let store = MemStore::new();
for ttl in [0i64, -1] {
match store.try_lock("db", "w", ttl).await {
Err(alkstore::Error::Database(_)) => {}
_other => panic!("try_lock guards a non-positive ttl"),
}
let granted = store.try_lock("db", "w", 60).await.unwrap().unwrap();
match granted.renew(ttl).await {
Err(alkstore::Error::Database(_)) => {}
_other => panic!("renew guards a non-positive ttl"),
}
}
}
#[tokio::test(flavor = "current_thread")]
async fn expiry_lapses_silently_and_reacquires() {
let store = MemStore::new();
store.clock.freeze(1_000);
let holder = store.try_lock("db", "w1", 60).await.unwrap().unwrap();
assert!(store.try_lock("db", "w2", 60).await.unwrap().is_none());
store.clock.advance(59);
assert!(
store.try_lock("db", "w2", 60).await.unwrap().is_none(),
"still within TTL"
);
store.clock.advance(2);
assert!(
store.try_lock("db", "w2", 60).await.unwrap().is_some(),
"exclusion lapsed silently — re-acquirable, no revocation event"
);
assert!(
!holder.renew(60).await.unwrap(),
"the re-acquired lock is not the old holder's to renew"
);
}
#[tokio::test(flavor = "current_thread")]
async fn renew_past_expiry_is_refused() {
let store = MemStore::new();
store.clock.freeze(1_000);
let holder = store.try_lock("db", "w1", 60).await.unwrap().unwrap();
store.clock.advance(61);
assert!(
!holder.renew(60).await.unwrap(),
"a renew cannot revive a lapsed exclusion"
);
let re = store.try_lock("db", "w2", 60).await.unwrap().unwrap();
assert!(re.release().await.unwrap(), "the new holder releases");
}
#[tokio::test(flavor = "current_thread")]
async fn release_shapes() {
let store = MemStore::new();
store.clock.freeze(1_000);
let holder = store.try_lock("db", "w1", 60).await.unwrap().unwrap();
assert!(holder.release().await.unwrap(), "a held lock releases true");
let w1 = store.try_lock("db", "w1", 60).await.unwrap().unwrap();
store.clock.advance(120);
assert!(
w1.release().await.unwrap(),
"release past expiry stays the holder's — exclusion lapsed silently"
);
let a = store.try_lock("db", "w1", 60).await.unwrap().unwrap();
store.clock.advance(120);
store.try_lock("db", "w2", 60).await.unwrap().unwrap();
assert!(
!a.release().await.unwrap(),
"re-acquired elsewhere: the old holder's release is a no-op"
);
}
#[tokio::test(flavor = "current_thread")]
async fn entry_validation() {
let store = MemStore::new();
for bad in ["", " ", "__alkstore_scheduler"] {
match store.try_lock(bad, "w", 60).await {
Err(alkstore::Error::InvalidName { .. } | alkstore::Error::ReservedName { .. }) => {
}
_other => panic!("try_lock({bad:?}) rejects"),
}
}
match store.try_lock("db", "", 60).await {
Err(alkstore::Error::InvalidName { .. }) => {}
_other => panic!("an empty owner rejects"),
}
}
#[tokio::test(flavor = "current_thread")]
async fn locks_are_not_tx_scoped() {
let store = MemStore::new();
store.clock.freeze(1_000);
assert!(store.try_lock("db", "w", 60).await.unwrap().is_some());
let tx = store.begin_tx().await.unwrap();
drop(tx);
assert!(
store.try_lock("db", "w2", 60).await.unwrap().is_none(),
"rollback or not — the overlay never sees or stages locks"
);
}
}
+232
View File
@@ -0,0 +1,232 @@
//! The notify/listen mechanism's mem arm over the wake-routing
//! substrate (`sqlite-engine-notify-listen`'s shape, realized
//! in-process; `notify_tx` stages in the tx overlay).
//!
//! - `notify` — the auto-commit path: validate, encode the payload
//! (`Codec` typed at the helper, ADR-023 §1), and fire the wake at
//! the op's own commit boundary — one guarded operation, nothing to
//! roll back. **No payload limit at any size** — `PayloadTooLarge`
//! is never produced on this engine (the ADR-016 §5 occurrence
//! asymmetry; mem's column of the classification). The encoded bytes
//! are nowhere stored: the wake contract's `Wake { channel }` is the
//! entire delivery (ADR-008 §3) — the payload encoding run is the
//! codec-typed entry check, not a transport.
//! - `listen` — attach a per-channel feed (starts from "now";
//! attach-after-commit no-replay is structural) and bridge it into
//! the pinned [`alkstore::WakeReceiver`] forms. Delivery is
//! channel-scoped, per-notify (never coalesced — no batching
//! boundary exists), per-subscriber FIFO by commit order. **One
//! close path — engine drop**: the bus close disconnects every open
//! receiver (`recv() -> None`, terminal); `try_recv`/`recv_timeout`
//! carry [`alkstore::Error::Closed`] distinguishable from idle, and
//! timeout expiry is the documented `Ok(None)` arm, never a
//! close-shaped signal. The reserved reconnect-wake channel is never
//! emitted (no reconnection concept exists) and stays rejected as a
//! consumer name.
use std::sync::Arc;
use std::time::Duration;
use serde_json::Value;
use alkstore::{BoxedFuture, Error, Result, Wake, WakeReceiver, validate_shared_name};
use crate::wakes::{WakeBus, WakeFeed};
/// The auto-commit notify path (see the module docs). Name validation
/// fires at the entry point, before the payload is touched (ADR-008
/// §4); the wake fires at the op's own commit boundary.
pub(crate) fn notify(
wakes: Arc<WakeBus>,
channel: &str,
payload: Value,
) -> BoxedFuture<'static, Result<()>> {
if let Err(e) = validate_shared_name(channel) {
return Box::pin(async move { Err(e) });
}
let owned = channel.to_string();
let encoded = alkstore::encode_payload(&payload);
Box::pin(async move {
encoded?;
wakes.fire_commit_wakes([owned]);
Ok(())
})
}
/// The listen path (see the module docs): validate, attach, bridge.
pub(crate) fn listen(wakes: Arc<WakeBus>, channel: &str) -> Result<Box<dyn WakeReceiver>> {
validate_shared_name(channel)?;
let feed = wakes.attach(channel);
Ok(Box::new(MemWakeReceiver { feed }))
}
/// The consumer-visible wake receiver: the bridge over the feed, with
/// the pinned recv/try_recv/recv_timeout forms and the terminal None
/// close arm (its own drop detaches the feed, the engine's drop
/// disconnects it).
struct MemWakeReceiver {
feed: WakeFeed,
}
impl std::fmt::Debug for MemWakeReceiver {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("MemWakeReceiver").finish_non_exhaustive()
}
}
impl WakeReceiver for MemWakeReceiver {
fn recv<'a>(&'a mut self) -> BoxedFuture<'a, Option<Wake>> {
Box::pin(async move { self.feed.rx().recv().await })
}
fn try_recv(&mut self) -> Result<Option<Wake>> {
match self.feed.rx().try_recv() {
Ok(wake) => Ok(Some(wake)),
Err(tokio::sync::mpsc::error::TryRecvError::Empty) => Ok(None),
Err(tokio::sync::mpsc::error::TryRecvError::Disconnected) => Err(Error::Closed),
}
}
fn recv_timeout<'a>(&'a mut self, timeout: Duration) -> BoxedFuture<'a, Result<Option<Wake>>> {
Box::pin(async move {
match tokio::time::timeout(timeout, self.feed.rx().recv()).await {
Ok(Some(wake)) => Ok(Some(wake)),
Ok(None) => Err(Error::Closed),
Err(_expired) => Ok(None),
}
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::store::MemStore;
use alkstore::Store;
async fn listener(store: &MemStore, channel: &str) -> Box<dyn WakeReceiver> {
store.listen(channel).await.unwrap()
}
#[tokio::test(flavor = "current_thread")]
async fn notify_wakes_open_listeners_at_the_commit_boundary() {
let store = MemStore::new();
let mut a = listener(&store, "orders").await;
let mut foreign = listener(&store, "payments").await;
store
.notify("orders", serde_json::json!({"n": 1}))
.await
.unwrap();
let wake = tokio::time::timeout(std::time::Duration::from_millis(200), a.recv()).await;
let wake = wake.expect("the committed notify delivers").unwrap();
assert_eq!(
wake.channel, "orders",
"the channel is the wake's one piece of content"
);
assert!(
foreign.try_recv().unwrap().is_none(),
"delivery is channel-scoped"
);
// Per-notify delivery — one wake per commit-notify, never
// coalesced.
store.notify("orders", serde_json::json!(2)).await.unwrap();
store.notify("orders", serde_json::json!(3)).await.unwrap();
let first = a.try_recv().unwrap();
let second = a.try_recv().unwrap();
let drained = a.try_recv().unwrap();
assert!(
first.is_some() && second.is_some() && drained.is_none(),
"two notifies, two wakes"
);
}
#[tokio::test(flavor = "current_thread")]
async fn listener_attached_after_a_commit_never_sees_it() {
let store = MemStore::new();
store.notify("orders", serde_json::json!(1)).await.unwrap();
let mut late = listener(&store, "orders").await;
assert!(late.try_recv().unwrap().is_none(), "no replay");
store.notify("orders", serde_json::json!(2)).await.unwrap();
assert!(late.recv().await.is_some(), "post-attach commits deliver");
}
#[tokio::test(flavor = "current_thread")]
async fn entry_validation_on_both_paths() {
let store = MemStore::new();
for bad in [
"",
" ",
"__alkstore_orders",
alkstore::RESERVED_LISTENER_RECONNECTED,
] {
let err = store.notify(bad, serde_json::json!(1)).await.unwrap_err();
assert!(
matches!(
err,
alkstore::Error::InvalidName { .. } | alkstore::Error::ReservedName { .. }
),
"notify({bad:?}) rejects: {err:?}"
);
match store.listen(bad).await {
Err(alkstore::Error::InvalidName { .. } | alkstore::Error::ReservedName { .. }) => {
}
_other => panic!("listen({bad:?}) rejects"),
}
}
}
#[tokio::test(flavor = "current_thread")]
async fn payload_too_large_is_never_produced_at_any_size() {
let store = MemStore::new();
// 2 MiB of value content — the pg-side rejection arm does not
// exist on this engine (the ADR-016 §5 non-occurrence asymmetry).
let big = serde_json::Value::String("x".repeat(2 * 1024 * 1024));
store.notify("orders", big).await.unwrap();
}
#[tokio::test(flavor = "current_thread")]
async fn recv_forms_carry_the_pinned_arms() {
let store = MemStore::new();
let mut rx = listener(&store, "orders").await;
assert!(rx.try_recv().unwrap().is_none(), "idle is Ok(None)");
let expired = tokio::time::timeout(
std::time::Duration::from_millis(50),
rx.recv_timeout(std::time::Duration::from_millis(30)),
)
.await
.expect("recv_timeout resolves")
.expect("timeout expiry is Ok(None) — never a close-shaped signal");
assert!(expired.is_none());
store.notify("orders", serde_json::json!(1)).await.unwrap();
assert!(
rx.recv_timeout(std::time::Duration::from_millis(200))
.await
.unwrap()
.is_some()
);
drop(store);
assert!(
matches!(rx.try_recv(), Err(alkstore::Error::Closed)),
"closed is distinguishable"
);
assert!(
rx.recv().await.is_none(),
"engine drop is the terminal None"
);
}
#[tokio::test(flavor = "current_thread")]
async fn receiver_drop_detaches_its_feed() {
let store = MemStore::new();
{
let rx = listener(&store, "orders").await;
drop(rx);
}
store.notify("orders", serde_json::json!(1)).await.unwrap();
let mut kept = listener(&store, "orders").await;
assert!(
kept.try_recv().unwrap().is_none(),
"a dropped receiver detached: its feed got nothing from later commits"
);
}
}
+319
View File
@@ -0,0 +1,319 @@
//! The outbox mechanism's mem arm — the helper over queues
//! (ADR-014's auto-commit half; the tx half — `outbox_enqueue_tx` —
//! stages in the tx overlay):
//!
//! - `outbox(name)` — the validated constructor (shared-namespace
//! rules); the backing queue `__alkstore_outbox:{name}` derives
//! engine-side under the reserved prefix — unreachable by
//! `enqueue_tx` by design (ADR-014 §1: outbox enqueue is the only
//! path into it).
//! - `enqueue` — auto-commit into the backing queue, stamping the
//! outbox's derived 60/5/5 set (ADR-010 §3 / ADR-014 §1);
//! `EnqueueOpts::max_attempts` is the one per-job override,
//! `delay`/`run_at`/`expires` resolve per ADR-020 at one clock read.
//! - `run_once(worker_id, delivery)` — claim one job from the backing
//! queue via the ordinary claim machinery, run `delivery`:
//! `Ok` ⇒ ack; `Err(e)` ⇒ `retry(Some(e), None)` — the queue's own
//! equal-jitter curve (mem's third-owner computation), the delivery
//! error string riding the exhaust path (surfaced on a dead row).
//! `true` = claimed and processed; `false` = no work (a value, not
//! an error). **No engine-issued heartbeat inside delivery** — the
//! boxed [`alkstore::JobHandle`]'s own `heartbeat` remains available
//! to the closure; renewal inside the deadline is the consumer's
//! obligation (ADR-019 §5).
use std::sync::Arc;
use serde_json::Value;
use alkstore::{
BoxedFuture, Delivery, EnqueueOpts, JobHandle, JobState, Outbox, Result, validate_local_name,
};
use crate::clock::Clock;
use crate::queue::{MemJobHandle, ack_row, claim_via_state, enqueue_row, job_from_row};
use crate::resolution::{
backoff_delay_s, outbox_backing_queue_name, outbox_default_stamps, resolve_enqueue_opts,
stamps_with_override,
};
use crate::state::Guard;
use crate::state::State;
/// The outbox-scoped handle (see the module docs).
pub struct MemOutbox {
name: String,
backing: String,
state: Arc<State>,
clock: Arc<Clock>,
}
impl std::fmt::Debug for MemOutbox {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("MemOutbox")
.field("name", &self.name)
.finish_non_exhaustive()
}
}
impl MemOutbox {
pub(crate) fn new(name: String, state: Arc<State>, clock: Arc<Clock>) -> Self {
Self {
backing: outbox_backing_queue_name(&name),
name,
state,
clock,
}
}
}
impl Outbox for MemOutbox {
fn name(&self) -> &str {
&self.name
}
fn enqueue<'a>(&'a self, payload: Value, opts: EnqueueOpts) -> BoxedFuture<'a, Result<i64>> {
match self.stage_enqueue(payload, &opts) {
Ok(id) => Box::pin(async move { Ok(id) }),
Err(e) => Box::pin(async move { Err(e) }),
}
}
fn run_once<'a>(
&'a mut self,
worker_id: &str,
delivery: &'a mut dyn Delivery,
) -> BoxedFuture<'a, Result<bool>> {
if let Err(e) = validate_local_name(worker_id) {
return Box::pin(async move { Err(e) });
}
let backing = self.backing.clone();
let state = self.state.clone();
let clock = self.clock.clone();
let worker_id = worker_id.to_string();
Box::pin(async move {
let claim_now = clock.unix_now();
let claimed = claim_via_state(&state, &backing, &worker_id, 1, claim_now).pop();
let Some(job_row) = claimed else {
return Ok(false);
};
let job = job_from_row(job_row.clone(), JobState::Processing, None, None);
let handle: Box<dyn JobHandle> =
Box::new(MemJobHandle::new(job, state.clone(), clock.clone()));
match delivery.deliver(handle).await {
Ok(()) => {
let ack_now = clock.unix_now();
let mut q = state.queues.guard();
ack_row(&mut q, job_row.id, &worker_id, ack_now);
Ok(true)
}
Err(err) => {
// `retry(Some(e), None)` exactly — the delivery error
// string carries the exhaust path (surfaced on a dead
// row), and the delay is the queue's curve (the
// engine's one-owner equal-jitter computation).
let exhaust = err.to_string();
let delay =
backoff_delay_s(job_row.backoff_base_s, job_row.attempts, job_row.id);
let retry_now = clock.unix_now();
let mut q = state.queues.guard();
crate::queue::retry_row(
&mut q, job_row.id, &worker_id, delay, &exhaust, retry_now,
);
Ok(true)
}
}
})
}
}
impl MemOutbox {
fn stage_enqueue(&self, payload: Value, opts: &EnqueueOpts) -> Result<i64> {
let bytes = alkstore::encode_payload(&payload)?;
let now = self.clock.unix_now();
let resolved = resolve_enqueue_opts(now, opts);
let stamps = stamps_with_override(outbox_default_stamps(), opts);
Ok(enqueue_row(
&self.state,
now,
&self.backing,
bytes,
opts,
stamps,
&resolved,
))
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::store::MemStore;
use alkstore::{Error, Store};
/// The delivery callable the tests drive: records the claimed
/// payload (and stamps), resolves with a take()-able outcome.
struct RecordingDelivery {
saw_payload: Option<Vec<u8>>,
stamps: Option<(i64, i64, i64, Option<i64>)>,
outcome: Option<Result<()>>,
}
impl RecordingDelivery {
fn with(outcome: Result<()>) -> Self {
Self {
saw_payload: None,
stamps: None,
outcome: Some(outcome),
}
}
}
fn db_error(msg: &str) -> Error {
Error::database(std::io::Error::other(msg))
}
impl Delivery for RecordingDelivery {
fn deliver<'a>(&'a mut self, job: Box<dyn JobHandle>) -> BoxedFuture<'a, Result<()>> {
self.saw_payload = Some(job.job().payload.clone());
let j = job.job();
self.stamps = Some((
j.visibility_timeout_s,
j.max_attempts,
j.backoff_base_s,
j.dead_letter_retention_s,
));
let outcome = self.outcome.take().unwrap_or(Ok(()));
Box::pin(async move { outcome })
}
}
fn outbox(store: &MemStore, name: &str) -> MemOutbox {
MemOutbox::new(name.to_string(), store.state.clone(), store.clock.clone())
}
#[tokio::test(flavor = "current_thread")]
async fn run_once_acks_on_delivery_ok_and_stamps_the_derived_set() {
let store = MemStore::new();
store.clock.freeze(1_000);
let mut ob = outbox(&store, "mail");
let id = ob
.enqueue(serde_json::json!({"send": true}), EnqueueOpts::default())
.await
.unwrap();
let mut delivery = RecordingDelivery::with(Ok(()));
assert!(ob.run_once("w1", &mut delivery).await.unwrap());
assert_eq!(
delivery.saw_payload,
Some(serde_json::to_vec(&serde_json::json!({"send": true})).unwrap()),
"the delivery claims ride the claimed row's exact payload"
);
assert!(
!ob.run_once("w1", &mut delivery).await.unwrap(),
"a second run_once finds no work (a value, not an error)"
);
assert!(
!store.state.queues.guard().live.contains_key(&id),
"Ok ⇒ ack deletes"
);
let id2 = ob
.enqueue(
serde_json::json!(2),
EnqueueOpts {
max_attempts: Some(8),
..Default::default()
},
)
.await
.unwrap();
let mut recorder = RecordingDelivery::with(Ok(()));
assert!(ob.run_once("w2", &mut recorder).await.unwrap());
assert_eq!(
recorder.stamps,
Some((60, 8, 5, None)),
"the derived 60/5/5 set stamps the row, max_attempts the one override"
);
assert!(!store.state.queues.guard().live.contains_key(&id2), "acked");
}
#[tokio::test(flavor = "current_thread")]
async fn run_once_retries_with_the_queue_curve_and_delivery_error_on_exhaust() {
let store = MemStore::new();
store.clock.freeze(2_000);
let mut ob = outbox(&store, "mail");
let id = ob
.enqueue(serde_json::json!(1), EnqueueOpts::default())
.await
.unwrap();
let mut delivery = RecordingDelivery::with(Err(db_error("smtp down")));
assert!(ob.run_once("w1", &mut delivery).await.unwrap());
{
let q = store.state.queues.guard();
let row = q.live.get(&id).unwrap();
assert!(row.pending, "the retry rescheduled pending");
assert_eq!(row.worker_id, None);
assert_eq!(row.attempts, 1);
let cap = row
.backoff_base_s
.saturating_mul(1i64 << (row.attempts - 1).min(30))
.min(3600);
let half = (cap + 1) / 2;
let delay = row.run_at - 2_000;
assert!(
(half..=cap).contains(&delay),
"the delay is the equal-jitter curve: {delay}"
);
}
// Exhaustion surfaces the delivery error string on the dead row.
let mut claims = 0;
for _ in 0..5 {
store.clock.advance(60);
claims += ob
.run_once(
"w1",
&mut RecordingDelivery::with(Err(db_error("smtp down"))),
)
.await
.unwrap() as i64;
}
assert!(claims >= 1, "the retried row is claimable again");
let q = store.state.queues.guard();
let dead = q.dead.get(&id).cloned().unwrap();
assert_eq!(
dead.last_error,
db_error("smtp down").to_string(),
"the delivery error rides the exhaust path as its Display"
);
assert!(dead.died_at == 2_000 || dead.died_at > 0);
}
#[tokio::test(flavor = "current_thread")]
async fn empty_worker_id_is_invalid() {
let store = MemStore::new();
let mut ob = outbox(&store, "mail");
let err = ob
.run_once("", &mut RecordingDelivery::with(Ok(())))
.await
.unwrap_err();
assert!(matches!(err, Error::InvalidName { .. }));
}
#[tokio::test(flavor = "current_thread")]
async fn derived_backing_queue_name_and_unreachability() {
assert_eq!(
outbox_backing_queue_name("mail"),
format!("{}outbox:mail", alkstore::RESERVED_PREFIX)
);
let store = MemStore::new();
let mut tx = store.begin_tx().await.unwrap();
let err = tx
.enqueue_tx(
&outbox_backing_queue_name("mail"),
EnqueueOpts::default(),
serde_json::json!(1),
)
.await
.unwrap_err();
assert!(matches!(err, Error::ReservedName { .. }));
}
}
File diff suppressed because it is too large. Load diff
+248
View File
@@ -0,0 +1,248 @@
//! The engine-side opts-resolution arithmetic (ADR-020 §1–§3) and the
//! derived stamp sets — mem's own implementation (ADR-012 §2's
//! one-owner rule, third owner; the other engines' copies are
//! references, not dependencies, and the suite pins three-way
//! equivalence).
//!
//! Every function takes the enqueue instant as a parameter — the one
//! clock read per enqueue is the caller's (`crate::clock`), and these
//! functions never read a clock of their own except the jitter draw's
//! subsecond entropy.
use std::collections::hash_map::RandomState;
use std::hash::BuildHasher;
use std::time::{SystemTime, UNIX_EPOCH};
use alkstore::{EnqueueOpts, QueueOpts};
/// The four §3a stamps a queue row carries (resolved at enqueue).
#[derive(Debug, Clone, Copy)]
pub(crate) struct Stamps {
pub(crate) max_attempts: i64,
pub(crate) visibility_timeout_s: i64,
pub(crate) backoff_base_s: i64,
pub(crate) dead_letter_retention_s: Option<i64>,
}
/// The pinned dead-letter default strings (row content, not error
/// variants — ADR-010 §4 / ADR-019 §3's annotation: `"max attempts
/// exceeded"` at exhaustion, `"failed"` for `fail(None)`, `"expired"`
/// on sweeps; a caller-supplied string carries verbatim).
pub(crate) const EXHAUSTED_ERROR: &str = "max attempts exceeded";
pub(crate) const FAILED_ERROR: &str = "failed";
pub(crate) const EXPIRED_ERROR: &str = "expired";
/// The backoff cap (ADR-010 §3 — a pinned contract constant, no knob).
const BACKOFF_CAP_S: i64 = 3600;
/// The plain queue's derived `QueueOpts` defaults (300/3/5/none —
/// ADR-010 §3): the stamp source for every no-handle-open shape
/// (`enqueue_tx`, scheduler boundary fires, the scheduler-outbox task's
/// auto-commit uses). Named so the derived-defaults formula has one
/// owner engine-wide.
pub(crate) fn plain_queue_default_stamps() -> Stamps {
let QueueOpts {
visibility_timeout_s,
max_attempts,
backoff_base_s,
dead_letter_retention_s,
} = QueueOpts::default();
Stamps {
max_attempts,
visibility_timeout_s,
backoff_base_s,
dead_letter_retention_s,
}
}
/// The outbox backing queue's derived `QueueOpts` set (60/5/5 —
/// ADR-010 §3 / ADR-014 §1). The stamp source for `outbox_enqueue_tx`
/// and `Outbox::enqueue`.
pub(crate) fn outbox_default_stamps() -> Stamps {
Stamps {
visibility_timeout_s: 60,
max_attempts: 5,
backoff_base_s: 5,
dead_letter_retention_s: None,
}
}
/// The derived backing-queue name for an outbox (the reserved
/// namespace, ADR-008 §4): unreachable by `enqueue_tx` by design
/// (ADR-014 §1) — outbox enqueue is the only path into it.
pub(crate) fn outbox_backing_queue_name(outbox: &str) -> String {
format!("{}outbox:{outbox}", alkstore::RESERVED_PREFIX)
}
/// The one per-enqueue stamp override (ADR-010 §3a):
/// `EnqueueOpts::max_attempts` is the only per-job override — the
/// other stamps always take the base's. One helper for every enqueue
/// shape (tx, handle, outbox, scheduler fire).
pub(crate) fn stamps_with_override(base: Stamps, opts: &EnqueueOpts) -> Stamps {
Stamps {
max_attempts: opts.max_attempts.unwrap_or(base.max_attempts),
..base
}
}
/// The resolved enqueue fields (ADR-020 §1/§2): `delay` set → ready
/// time `now + delay` (wins over `run_at`); else `run_at` literally;
/// else `now`. `expires` is relative seconds from the enqueue instant,
/// resolved to the absolute row `expires_at`; `None` = never.
pub(crate) struct ResolvedEnqueue {
pub(crate) run_at: i64,
pub(crate) expires_at: Option<i64>,
}
pub(crate) fn resolve_enqueue_opts(now: i64, opts: &EnqueueOpts) -> ResolvedEnqueue {
let run_at = match opts.delay {
Some(delay) => now.saturating_add(delay),
None => opts.run_at.unwrap_or(now),
};
let expires_at = opts.expires.map(|s| now.saturating_add(s));
ResolvedEnqueue { run_at, expires_at }
}
/// The equal-jitter exponential backoff (ADR-010 §3 — mem's own
/// implementation): `delay ∈ [base·2^(a−1)/2, base·2^(a−1)]` from the
/// job's stamped `backoff_base_s`, capped at 1 hour. The attempt index
/// `a` is the row's `attempts` at the retry (post-claim count,
/// reclaims included); `Some(d)` overrides pass through without this
/// function.
///
/// Integerized second-precision: the draw is uniform over the integers
/// in `[ceil(half), cap]` (inclusive both ends) — never below the
/// pinned lower bound, never above the cap. The jitter source is
/// std-only (the ADR-005 dependency posture — no `rand` dep): a fresh
/// [`RandomState`] hasher over the row id, the attempt count, and the
/// subsecond clock spreads the draw; the pinned property is the range.
pub(crate) fn backoff_delay_s(base_s: i64, attempts: i64, id: i64) -> i64 {
let exponent: u32 = (attempts.max(1) as u32).saturating_sub(1).min(30);
let doubling: i64 = base_s.saturating_mul(1i64 << exponent).max(1);
let cap = doubling.min(BACKOFF_CAP_S);
let half = ((cap + 1) / 2).max(1);
let span = (cap - half + 1).max(1);
let now_nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.subsec_nanos())
.unwrap_or(0);
let draw = RandomState::new().hash_one((id, attempts, now_nanos));
half + (draw % span as u64) as i64
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn stamp_sets_match_the_pinned_defaults() {
let plain = plain_queue_default_stamps();
assert_eq!(
(
plain.visibility_timeout_s,
plain.max_attempts,
plain.backoff_base_s,
plain.dead_letter_retention_s
),
(300, 3, 5, None)
);
let outbox = outbox_default_stamps();
assert_eq!(
(
outbox.visibility_timeout_s,
outbox.max_attempts,
outbox.backoff_base_s,
outbox.dead_letter_retention_s
),
(60, 5, 5, None)
);
}
#[test]
fn override_touches_only_max_attempts() {
let base = Stamps {
max_attempts: 3,
visibility_timeout_s: 300,
backoff_base_s: 5,
dead_letter_retention_s: None,
};
let opts = EnqueueOpts {
max_attempts: Some(9),
..Default::default()
};
let stamped = stamps_with_override(base, &opts);
assert_eq!(stamped.max_attempts, 9);
assert_eq!(stamped.visibility_timeout_s, 300);
assert_eq!(stamped.backoff_base_s, 5);
assert_eq!(stamped.dead_letter_retention_s, None);
let plain = EnqueueOpts::default();
assert_eq!(stamps_with_override(base, &plain).max_attempts, 3);
}
#[test]
fn delay_wins_over_run_at_and_neither_resolves_to_now() {
let now = 1_000_000;
let both = EnqueueOpts {
delay: Some(10),
run_at: Some(now + 500),
..Default::default()
};
assert_eq!(resolve_enqueue_opts(now, &both).run_at, now + 10);
let absolute = EnqueueOpts {
run_at: Some(now + 500),
..Default::default()
};
assert_eq!(resolve_enqueue_opts(now, &absolute).run_at, now + 500);
let neither = EnqueueOpts::default();
assert_eq!(resolve_enqueue_opts(now, &neither).run_at, now);
let negative = EnqueueOpts {
delay: Some(-5),
..Default::default()
};
assert_eq!(
resolve_enqueue_opts(now, &negative).run_at,
now - 5,
"a negative delay is a ready time in the past — total, ready now"
);
}
#[test]
fn expires_resolves_relative_to_enqueue() {
let now = 1_000_000;
let opts = EnqueueOpts {
expires: Some(60),
..Default::default()
};
assert_eq!(resolve_enqueue_opts(now, &opts).expires_at, Some(now + 60));
let none = EnqueueOpts::default();
assert_eq!(resolve_enqueue_opts(now, &none).expires_at, None);
}
#[test]
fn backoff_lands_in_the_pinned_equal_jitter_range() {
for attempts in [1i64, 2, 3, 5, 30] {
let expected_cap = 5i64
.saturating_mul(1i64 << (attempts - 1).min(30))
.min(3600);
let half = (expected_cap + 1) / 2;
for id in [1i64, 42, 777, 123_456] {
let delay = backoff_delay_s(5, attempts, id);
assert!(
(half..=expected_cap).contains(&delay),
"attempt {attempts} id {id}: delay {delay} outside [{half}, {expected_cap}]"
);
}
}
}
#[test]
fn backoff_caps_at_one_hour_and_spreads() {
let draws: Vec<i64> = (0..400).map(|i| backoff_delay_s(5, 40, i)).collect();
assert!(
draws.iter().all(|d| (1800..=3600).contains(d)),
"capped range"
);
assert!(draws.windows(2).any(|w| w[0] != w[1]), "the jitter spreads");
assert!(draws.iter().any(|d| *d < 2700) && draws.iter().any(|d| *d > 2700));
}
}
+765
View File
@@ -0,0 +1,765 @@
//! The scheduler mechanism's mem arm — the collapsed scheduler
//! ([ADR-009](../../docs/architecture/decisions/009-scheduler-collapse.md)):
//! two registrations + one opt-in runner, the leadership lock through
//! the locks registry (engine-uniform — single-instance leadership is
//! trivially owned but the machinery still runs), and the tick's
//! guarded fire-and-advance.
//!
//! - `schedule(name, spec, queue, payload, opts)` — upsert by name;
//! *both* name-bearing arguments validate at the entry point
//! (schedule name and queue identically, `InvalidName`/`ReservedName`
//! — ADR-021 §3: a schedule cannot target a reserved name); the spec
//! parser rejects everything v1 does not accept (`InvalidSpec`,
//! before storage, no round trip). The payload encodes once here;
//! the stamps resolve over the plain-queue derived defaults with
//! `ScheduleOpts::max_attempts` as the one override (ADR-020 §3) and
//! the spec's interval carries pre-parsed — the fire path reads the
//! registered values, never the raw spec.
//! - `unschedule(name)` — remove; `true` = a row was removed.
//! - `run_schedules(stop)` — acquire the leadership lock
//! (`SCHEDULER_LEADER_LOCK`, an engine-internal TTL, a runner-unique
//! owner), then loop {renew — on loss return `Err(LeadershipLost)`
//! *before ticking*; fire every due boundary; sleep until the next
//! due boundary or `stop`}. A cancelled token ends the sleep with
//! the clean `Ok(())` and releases the lease.
//! - The tick's fire+advance is one guarded operation (the
//! no-double-fire floor, ADR-009 §4: fire and boundary advance
//! together under the schedules→queues guard order — a boundary
//! observed mid-tick is either fully fired or fully unfired):
//! boundary-by-boundary replay up to the fixed **64-per-task cap**,
//! beyond which the schedule skips forward past the gap
//! (never a 10,000-job catch-up stacking from a week's downtime).
//! Fired jobs are ordinary queue rows — ready at the fire instant,
//! the registered stamps, `ScheduleOpts::{priority, expires}` riding
//! themselves — claimable via the ordinary queue surface.
//! - Boundary waits ride tokio time (the wasm-supported subset); the
//! sleep renews the lease across itself so an idle store never
//! loses leadership by drifting past the TTL.
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::Duration;
use serde_json::Value;
use alkstore::{
BoxedFuture, Error, Result, Schedule, ScheduleOpts, StopToken, validate_shared_name,
};
use crate::clock::Clock;
use crate::lock::{MemLock, SCHEDULER_LEADER_LOCK};
use crate::queue::build_job_row;
use crate::resolution::{Stamps, plain_queue_default_stamps, stamps_with_override};
use crate::state::Guard;
use crate::state::{QueueState, ScheduleRow, ScheduleState, State};
/// The leadership lock's TTL (seconds): engine-internal detail —
/// renewed every loop iteration *and* across the sleeps (a runner that
/// cannot renew has lost the lock).
const LEADER_TTL_S: i64 = 10;
/// Seconds between lease renewals during the sleep.
const SLEEP_RENEW_EVERY_S: u64 = 2;
/// The idle sleep when nothing is scheduled (soonest = 0).
const IDLE_SLEEP_S: u64 = 60;
/// The bounded catch-up cap (ADR-009 §4 — a fixed contract constant,
/// honker parity, not a knob).
const MAX_CATCHUP_FIRES: i64 = 64;
/// Runner-unique leadership owner counter: the owner string is the
/// claim's identity token (a shared constant across runners would make
/// the owner-match mistake a foreign leader for self).
static LEADER_INSTANCE: AtomicU64 = AtomicU64::new(0);
fn leader_owner() -> String {
let instance = LEADER_INSTANCE.fetch_add(1, Ordering::Relaxed);
let nanos = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.subsec_nanos())
.unwrap_or(0);
format!("scheduler-leader-{instance}-{nanos}")
}
/// The v1 spec grammar (`ADR-009` §2): exactly `@every <n><unit>` with
/// `<n>` a positive integer and `<unit>` in `s|m|h|d` — mem's own
/// parser (third owner per ADR-012 §2; the grammar is small and
/// pinned). Everything else — 5-field cron, seconds-field cron —
/// rejects as `InvalidSpec`.
pub(crate) fn parse_every_interval(spec: &str) -> Result<i64> {
let Some(body) = spec.strip_prefix("@every ") else {
return Err(invalid_spec(spec));
};
if body.is_empty() {
return Err(invalid_spec(spec));
}
let digits = body.chars().take_while(|c| c.is_ascii_digit()).count();
if digits == 0 || digits == body.len() {
return Err(invalid_spec(spec));
}
let n: i64 = body[..digits].parse().map_err(|_| invalid_spec(spec))?;
if n <= 0 {
return Err(invalid_spec(spec));
}
let mult = match &body[digits..] {
"s" => 1,
"m" => 60,
"h" => 3600,
"d" => 86400,
_ => return Err(invalid_spec(spec)),
};
n.checked_mul(mult).ok_or_else(|| invalid_spec(spec))
}
fn invalid_spec(spec: &str) -> Error {
Error::InvalidSpec {
spec: spec.to_string(),
}
}
/// The auto-commit register path: pre-storage validation, the payload
/// encode (one owner of those bytes for every later fire), the stamps
/// resolution, the upsert, the [`Schedule`] read-back.
pub(crate) fn schedule(
state: Arc<State>,
clock: Arc<Clock>,
name: &str,
spec: &str,
queue: &str,
payload: Value,
opts: ScheduleOpts,
) -> BoxedFuture<'static, Result<Schedule>> {
if let Err(e) = validate_shared_name(name) {
return Box::pin(async move { Err(e) });
}
if let Err(e) = validate_shared_name(queue) {
return Box::pin(async move { Err(e) });
}
let interval = match parse_every_interval(spec) {
Ok(interval) => interval,
Err(e) => return Box::pin(async move { Err(e) }),
};
let encoded = alkstore::encode_payload(&payload);
let name = name.to_string();
let spec = spec.to_string();
let queue = queue.to_string();
let read_back = Schedule::new(name.clone(), spec.clone(), queue.clone(), opts.clone());
Box::pin(async move {
let bytes = encoded?;
let now = clock.unix_now();
let stamps = stamps_with_override(
plain_queue_default_stamps(),
&alkstore::EnqueueOpts {
max_attempts: opts.max_attempts,
..Default::default()
},
);
let row = ScheduleRow {
queue,
interval_s: interval,
payload: bytes,
priority: opts.priority,
expires_s: opts.expires,
max_attempts: stamps.max_attempts,
visibility_timeout_s: stamps.visibility_timeout_s,
backoff_base_s: stamps.backoff_base_s,
dead_letter_retention_s: stamps.dead_letter_retention_s,
next_fire_at: now.saturating_add(interval),
};
{
let mut sched = state.schedules.guard();
sched.rows.insert(name, row);
}
Ok(read_back)
})
}
/// The auto-commit unregister path (validation mirrors `schedule`'s —
/// a reserved/empty schedule name never reaches the table).
pub(crate) fn unschedule(state: Arc<State>, name: &str) -> BoxedFuture<'static, Result<bool>> {
if let Err(e) = validate_shared_name(name) {
return Box::pin(async move { Err(e) });
}
let removed = {
let mut sched = state.schedules.guard();
sched.rows.remove(name).is_some()
};
Box::pin(async move { Ok(removed) })
}
/// One tick under the canonical schedules→queues guard order: fire
/// every due boundary (bounded catch-up with the 64 cap, skip-forward
/// past the cap) and advance the schedule rows atomically with the
/// fires. Returns the soonest next fire across the table (0 = none).
pub(crate) fn run_tick(state: &State, now: i64) -> i64 {
let mut sched = state.schedules.guard();
let mut queues = state.queues.guard();
fire_due(&mut sched, &mut queues, state, now);
soonest(&sched)
}
fn fire_due(sched: &mut ScheduleState, queues: &mut QueueState, state: &State, now: i64) {
for row in sched.rows.values_mut() {
if row.next_fire_at > now {
continue;
}
let interval = row.interval_s;
let expires_at = row.expires_s.map(|s| now.saturating_add(s));
let stamps = Stamps {
max_attempts: row.max_attempts,
visibility_timeout_s: row.visibility_timeout_s,
backoff_base_s: row.backoff_base_s,
dead_letter_retention_s: row.dead_letter_retention_s,
};
let mut next_fire_at = row.next_fire_at;
let mut fires = 0;
while next_fire_at <= now {
if fires >= MAX_CATCHUP_FIRES {
next_fire_at = skip_forward(next_fire_at, now, interval);
break;
}
let id = state.allocate_job_id();
let resolved = crate::resolution::ResolvedEnqueue {
run_at: now,
expires_at,
};
let built = build_job_row(
id,
row.queue.clone(),
row.payload.clone(),
now,
&resolved,
row.priority,
&stamps,
);
queues.live.insert(id, built);
fires += 1;
next_fire_at = next_fire_at.checked_add(interval).unwrap_or(i64::MAX);
}
row.next_fire_at = next_fire_at;
}
}
/// The bounded catch-up's skip-forward: land the next boundary strictly
/// in the future, grid-aligned (the schedule row's advance makes the
/// gap inspectable).
fn skip_forward(next_fire_at: i64, now: i64, interval: i64) -> i64 {
let gap = (now - next_fire_at) as i128;
let interval = interval as i128;
let next = (next_fire_at as i128) + (gap / interval + 1) * interval;
if next > i64::MAX as i128 {
i64::MAX
} else {
next as i64
}
}
/// The soonest boundary across the table (0 = none scheduled).
pub(crate) fn soonest(sched: &ScheduleState) -> i64 {
sched
.rows
.values()
.map(|r| r.next_fire_at)
.min()
.unwrap_or(0)
}
/// The leader loop (see the module docs).
pub(crate) fn run_schedules(
state: Arc<State>,
clock: Arc<Clock>,
stop: StopToken,
) -> BoxedFuture<'static, Result<()>> {
Box::pin(async move {
let lease = MemLock::new_internal(
SCHEDULER_LEADER_LOCK,
&leader_owner(),
state.clone(),
clock.clone(),
);
// Step 1: acquire the leadership lock. A refused acquire —
// someone else leads — is `Err(LeadershipLost)` at start: no
// leader loop exists without the lock, and the caller's recipe
// (respawn) is identical to a mid-run loss.
let now = clock.unix_now();
if !lease.acquire_now(LEADER_TTL_S, now) {
return Err(Error::LeadershipLost);
}
loop {
if stop.is_cancelled() {
let _ = lease.release_now();
return Ok(());
}
let now = clock.unix_now();
if !lease.renew_at(LEADER_TTL_S, now) {
return Err(Error::LeadershipLost);
}
let soonest_at = run_tick(&state, now);
if stop.is_cancelled() {
let _ = lease.release_now();
return Ok(());
}
keep_awake_renewing(&lease, clock.as_ref(), &stop, soonest_at).await;
}
})
}
/// The sliced sleep until the next due boundary (or the idle horizon)
/// or the stop token — the loop's wake path between ticks. The lease
/// renews on cadence across the sleep: a boundary further out than the
/// TTL must not lose the lock by drifting past it (the leader-loop
/// pattern: renew while holding).
async fn keep_awake_renewing(lease: &MemLock, clock: &Clock, stop: &StopToken, soonest_at: i64) {
let renew_every = Duration::from_secs(SLEEP_RENEW_EVERY_S);
let mut next_renew = tokio::time::Instant::now() + renew_every;
let started = tokio::time::Instant::now();
let now = clock.unix_now();
let wait_s: u64 = if soonest_at <= 0 {
IDLE_SLEEP_S
} else {
soonest_at.saturating_sub(now).max(1) as u64
};
let deadline = started + Duration::from_secs(wait_s);
loop {
let now = tokio::time::Instant::now();
if stop.is_cancelled() || now >= deadline {
return;
}
if now >= next_renew {
// A failed mid-sleep renewal is not acted on here — the
// loop's own top-of-iteration renew decides (one owner of
// the loss decision, before any tick).
let _ = lease.renew_at(LEADER_TTL_S, clock.unix_now());
next_renew = tokio::time::Instant::now() + renew_every;
}
let slice = deadline.min(next_renew);
tokio::time::sleep_until(slice).await;
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::store::MemStore;
use alkstore::{Queue, Store};
fn store() -> MemStore {
let store = MemStore::new();
store.clock.freeze(1_000_000);
store
}
#[test]
fn parser_accepts_only_the_v1_grammar() {
for (spec, expected) in [
("@every 1s", 1),
("@every 30s", 30),
("@every 5m", 300),
("@every 2h", 7200),
("@every 1d", 86_400),
("@every 1000000s", 1_000_000),
] {
assert_eq!(parse_every_interval(spec).unwrap(), expected, "{spec}");
}
for bad in [
"0 0 * * *",
"0 0 0 * * *",
"@every",
"@every ",
"@every 0s",
"@every -5s",
"@every 5",
"@every s",
"@every 5x",
"@every 5sm",
"@every 1.5s",
"@every 5s",
"@every5s",
"every 5s",
"@every 99999999999999999999s",
] {
let err = parse_every_interval(bad).expect_err(bad);
assert!(matches!(err, Error::InvalidSpec { .. }), "{bad} → {err:?}");
}
}
#[tokio::test(flavor = "current_thread")]
async fn register_upserts_resolves_stamps_and_reads_back() {
let store = store();
let registered = store
.schedule(
"sweep",
"@every 300s",
"maintenance",
serde_json::json!(1),
ScheduleOpts::default(),
)
.await
.unwrap();
assert_eq!(registered.name, "sweep");
assert_eq!(registered.spec, "@every 300s");
assert_eq!(registered.queue, "maintenance");
let replaced = store
.schedule(
"sweep",
"@every 60s",
"maintenance2",
serde_json::json!(1),
ScheduleOpts::default(),
)
.await
.unwrap();
assert_eq!(replaced.spec, "@every 60s", "upsert replaces the whole row");
{
let sched = store.state.schedules.guard();
assert_eq!(sched.rows.len(), 1);
let row = sched.rows.get("sweep").unwrap();
assert_eq!(row.interval_s, 60);
assert_eq!(
row.next_fire_at, 1_000_060,
"the first boundary is one interval out"
);
assert_eq!(
row.payload,
serde_json::to_vec(&serde_json::json!(1)).unwrap()
);
}
let with_opts = store
.schedule(
"opts",
"@every 5s",
"maintenance",
serde_json::json!(1),
ScheduleOpts {
priority: 7,
max_attempts: Some(9),
expires: Some(120),
},
)
.await
.unwrap();
assert_eq!(with_opts.opts.priority, 7);
let sched = store.state.schedules.guard();
let row = sched.rows.get("opts").unwrap();
assert_eq!(
row.max_attempts, 9,
"the schedule's max_attempts resolves over the defaults"
);
assert_eq!(row.visibility_timeout_s, 300);
assert_eq!(row.backoff_base_s, 5);
assert_eq!(row.dead_letter_retention_s, None);
assert_eq!(row.priority, 7);
assert_eq!(row.expires_s, Some(120));
}
#[tokio::test(flavor = "current_thread")]
async fn register_rejects_bad_names_and_specs_pre_storage() {
let store = store();
for (bad_name, bad_queue, bad_spec) in [
("", "q", "@every 5s"),
("__alkstore_s", "q", "@every 5s"),
("ok", "", "@every 5s"),
("ok", "__alkstore_q", "@every 5s"),
("ok", "q", "0 0 * * *"),
] {
let err = store
.schedule(
bad_name,
bad_spec,
bad_queue,
serde_json::json!(1),
ScheduleOpts::default(),
)
.await
.expect_err("register rejects");
assert!(
matches!(
err,
Error::InvalidName { .. }
| Error::ReservedName { .. }
| Error::InvalidSpec { .. }
),
"{bad_name}/{bad_queue}/{bad_spec} → {err:?}"
);
}
assert!(
store.state.schedules.guard().rows.is_empty(),
"nothing stored"
);
}
#[tokio::test(flavor = "current_thread")]
async fn unschedule_removes_and_reports_absence() {
let store = store();
store
.schedule(
"sweep",
"@every 60s",
"maintenance",
serde_json::json!(1),
ScheduleOpts::default(),
)
.await
.unwrap();
assert!(store.unschedule("sweep").await.unwrap());
assert!(!store.unschedule("sweep").await.unwrap(), "absent = false");
let err = store.unschedule("__alkstore_x").await.unwrap_err();
assert!(
matches!(err, Error::ReservedName { .. }),
"the reserved name never round trips"
);
}
#[tokio::test(flavor = "current_thread")]
async fn tick_fires_read_registered_values_and_never_double_fire() {
let store = MemStore::new();
store.clock.freeze(1_000_000);
store
.schedule(
"job",
"@every 10s",
"work",
serde_json::json!({"task": "sweep"}),
ScheduleOpts::default(),
)
.await
.unwrap();
assert_eq!(run_tick(&store.state, 1_000_005), 1_000_010);
assert!(store.state.queues.guard().live.is_empty());
let soonest = run_tick(&store.state, 1_000_013);
let rows: Vec<crate::state::JobRow> =
store.state.queues.guard().live.values().cloned().collect();
assert_eq!(rows.len(), 1);
let fire = &rows[0];
assert_eq!(fire.queue, "work");
assert_eq!(
fire.run_at, 1_000_013,
"a boundary fire is ready at the fire instant"
);
assert_eq!(fire.priority, 0);
assert_eq!(
fire.visibility_timeout_s, 300,
"the plain defaults stamp fired rows"
);
assert_eq!(fire.max_attempts, 3);
assert_eq!(fire.created_at, 1_000_013);
assert_eq!(
fire.payload,
serde_json::to_vec(&serde_json::json!({"task": "sweep"})).unwrap()
);
assert_eq!(soonest, 1_000_020, "the boundary advanced with the fire");
assert_eq!(run_tick(&store.state, 1_000_015), 1_000_020);
assert_eq!(
store.state.queues.guard().live.len(),
1,
"the no-double-fire floor"
);
}
#[test]
fn catchup_replays_boundary_by_boundary_up_to_the_cap_then_skips_forward() {
let store = MemStore::new();
store.clock.freeze(1_000_000);
{
let mut sched = store.state.schedules.guard();
let mut seed = |name: &str, interval: i64| {
sched.rows.insert(
name.to_string(),
crate::state::ScheduleRow {
queue: "work".to_string(),
interval_s: interval,
payload: serde_json::to_vec(&serde_json::json!(1)).unwrap(),
priority: 0,
expires_s: None,
max_attempts: 3,
visibility_timeout_s: 300,
backoff_base_s: 5,
dead_letter_retention_s: None,
next_fire_at: 1_000_000 + interval,
},
);
};
seed("job", 10);
seed("neverdue", 3600);
}
assert_eq!(run_tick(&store.state, 1_003_000), 1_003_010);
assert_eq!(
store.state.queues.guard().live.len(),
64,
"the bounded catch-up: 64 fires, never 300"
);
let row = store
.state
.schedules
.guard()
.rows
.get("job")
.unwrap()
.clone();
assert!(
row.next_fire_at > 1_003_000,
"skip-forward lands the next boundary strictly in the future ({})",
row.next_fire_at
);
assert_eq!(
(row.next_fire_at - 1_000_010) % 10,
0,
"the grid stays honest"
);
assert_eq!(
store
.state
.schedules
.guard()
.rows
.get("neverdue")
.unwrap()
.next_fire_at,
1_003_600,
"the never-due schedule is untouched"
);
}
#[tokio::test(flavor = "current_thread")]
async fn fire_and_advance_commit_together_under_one_guarded_op() {
let store = store();
store
.schedule(
"job",
"@every 10s",
"work",
serde_json::json!(1),
ScheduleOpts::default(),
)
.await
.unwrap();
// The tick itself is the guarded operation: assert both effects
// landed together and the soonest is consistent with the rows.
let soonest = run_tick(&store.state, 1_000_000 + 10);
{
let sched = store.state.schedules.guard();
let queues = store.state.queues.guard();
assert_eq!(sched.rows.get("job").unwrap().next_fire_at, 1_000_020);
assert_eq!(queues.live.len(), 1);
}
assert_eq!(soonest, 1_000_020);
}
#[tokio::test(flavor = "current_thread")]
async fn leadership_is_won_once_and_the_loser_is_leadership_lost() {
let store = MemStore::new();
store.clock.freeze(1_000_000);
store
.schedule(
"job",
"@every 1s",
"work",
serde_json::json!(1),
ScheduleOpts::default(),
)
.await
.unwrap();
let stop = StopToken::new();
let leader_stop = stop.clone();
let leader = tokio::spawn(super::super::scheduler::run_schedules(
store.state.clone(),
store.clock.clone(),
leader_stop,
));
// Let the first runner acquire before the second contends.
tokio::time::sleep(Duration::from_millis(100)).await;
let loser = tokio::spawn(super::super::scheduler::run_schedules(
store.state.clone(),
store.clock.clone(),
StopToken::new(),
));
let loser_out = tokio::time::timeout(Duration::from_secs(2), loser)
.await
.expect("the contender resolves quickly")
.expect("the loser runner exits");
assert!(
matches!(loser_out, Err(Error::LeadershipLost)),
"the contender loses cleanly: {loser_out:?}"
);
stop.cancel();
let leader_out = tokio::time::timeout(Duration::from_secs(3), leader)
.await
.expect("the leader resolves after stop")
.expect("the leader runner exits");
assert!(
matches!(leader_out, Ok(())),
"a cancelled token is the clean-stop return"
);
// The lease released on the clean stop: a new runner acquires
// the leadership lock cleanly again.
let re_leader = MemLock::new_internal(
SCHEDULER_LEADER_LOCK,
"fresh-runner",
store.state.clone(),
store.clock.clone(),
);
assert!(
re_leader.acquire_now(10, store.clock.unix_now()),
"the clean stop released the lease"
);
}
#[tokio::test(flavor = "current_thread")]
async fn run_schedules_returns_leadership_lost_without_a_free_lock() {
let store = store();
let lease = MemLock::new_internal(
SCHEDULER_LEADER_LOCK,
"usurper",
store.state.clone(),
store.clock.clone(),
);
assert!(lease.acquire_now(10, store.clock.unix_now()));
let out = super::super::scheduler::run_schedules(
store.state.clone(),
store.clock.clone(),
StopToken::new(),
)
.await;
assert!(matches!(out, Err(Error::LeadershipLost)));
}
#[tokio::test(flavor = "current_thread")]
async fn fired_rows_are_ordinary_queue_work() {
let store = store();
store
.schedule(
"job",
"@every 5s",
"work",
serde_json::json!(1),
ScheduleOpts::default(),
)
.await
.unwrap();
run_tick(&store.state, 1_000_005);
store.clock.freeze(1_000_005);
let id = {
let q = store.state.queues.guard();
*q.live.keys().next().unwrap()
};
let queue = crate::queue::MemQueueHandle::new(
"work".to_string(),
alkstore::QueueOpts::default(),
store.state.clone(),
store.clock.clone(),
);
let claimed = queue
.claim_one("w")
.await
.unwrap()
.expect("the fired job claims");
assert_eq!(claimed.job().id, id);
assert_eq!(claimed.job().queue, "work");
assert_eq!(
claimed.job().visibility_timeout_s,
300,
"the plain defaults stamp fired rows"
);
}
}
+152 -23
View File
@@ -14,15 +14,19 @@
//! [`Guard::guard`] fold into the inner value keeps the type honest
//! anyway.
//!
//! This module owns the shared scaffolding only: the container, the
//! section shapes' names, and the guard idiom. Each mechanism's table
//! shape is owned by its own wave-6 task (`mem-engine-queues`,
//! `mem-engine-streams`, `mem-engine-locks`,
//! `mem-engine-scheduler-outbox`) — the sections here start empty and
//! grow there.
//! Section shapes are the commit-merge targets ([`crate::tx`]'s
//! overlay stages effects of exactly these rows). Row-level operations
//! live with their mechanisms (`crate::queue`, `crate::stream`,
//! `crate::lock`, `crate::scheduler`) — modules must take section
//! guards in the canonical order **schedules → queues → streams**
//! (locks and wakes depend on no other section; the commit-merge takes
//! queues → streams, the scheduler tick schedules → queues) so no path
//! can deadlock.
use std::collections::{BTreeMap, HashMap};
use std::sync::Mutex;
use std::sync::MutexGuard;
use std::sync::atomic::{AtomicI64, Ordering};
/// A short-held synchronous internal lock: `guard()` is the one access
/// idiom (see the module docs for the discipline).
@@ -36,37 +40,140 @@ impl<T> Guard<T> for Mutex<T> {
}
}
/// A live job row (ADR-010's state machine, minus the dead state —
/// dead rows move to [`QueueState::dead`] with their diagnosis). The
/// row carries the resolved enqueue fields and the §3a stamps; the
/// `pending` flag is the `pending`/`processing` split (absence = acked
/// or cancelled).
#[derive(Debug, Clone)]
pub(crate) struct JobRow {
pub(crate) id: i64,
pub(crate) queue: String,
pub(crate) payload: Vec<u8>,
pub(crate) pending: bool,
pub(crate) priority: i64,
pub(crate) run_at: i64,
pub(crate) worker_id: Option<String>,
pub(crate) claim_expires_at: Option<i64>,
pub(crate) claimed_at: Option<i64>,
pub(crate) attempts: i64,
pub(crate) max_attempts: i64,
pub(crate) created_at: i64,
pub(crate) expires_at: Option<i64>,
pub(crate) visibility_timeout_s: i64,
pub(crate) backoff_base_s: i64,
pub(crate) dead_letter_retention_s: Option<i64>,
}
/// A dead-lettered job row (ADR-010 §4): the live row's durable fields
/// plus the diagnosis (`last_error`, `died_at`) — `get_job` sees dead
/// rows with the stamps still attached. The claim identity
/// (`worker_id`/`claim_expires_at`) never rides a dead row.
#[derive(Debug, Clone)]
pub(crate) struct DeadRow {
pub(crate) id: i64,
pub(crate) queue: String,
pub(crate) payload: Vec<u8>,
pub(crate) priority: i64,
pub(crate) run_at: i64,
pub(crate) attempts: i64,
pub(crate) max_attempts: i64,
pub(crate) created_at: i64,
pub(crate) expires_at: Option<i64>,
pub(crate) claimed_at: Option<i64>,
pub(crate) visibility_timeout_s: i64,
pub(crate) backoff_base_s: i64,
pub(crate) dead_letter_retention_s: Option<i64>,
pub(crate) last_error: String,
pub(crate) died_at: i64,
}
/// A committed stream event (ADR-015 §3's shape, engine-internal) —
/// `offset` is the per-stream log key, immutable, never renumbered.
#[derive(Debug, Clone)]
pub(crate) struct EventRow {
pub(crate) offset: i64,
pub(crate) key: Option<String>,
pub(crate) payload: Vec<u8>,
pub(crate) created_at: i64,
}
/// A named TTL lock's registry entry (ADR-008 §7): owner identity +
/// deadline. Expiry is a clock check at the try/renew boundaries — no
/// ambient revoker exists.
#[derive(Debug, Clone)]
pub(crate) struct LockEntry {
pub(crate) owner: String,
pub(crate) expires_at: i64,
}
/// A registered schedule row (ADR-009): the collapsed scheduler's
/// table. The interval is parsed and the stamps are resolved at
/// register time (the fire path stamps rows from the registered
/// values, never the raw spec) — `next_fire_at` is the collapse
/// machinery's boundary state.
#[derive(Debug, Clone)]
pub(crate) struct ScheduleRow {
pub(crate) queue: String,
pub(crate) interval_s: i64,
pub(crate) payload: Vec<u8>,
pub(crate) priority: i64,
pub(crate) expires_s: Option<i64>,
pub(crate) max_attempts: i64,
pub(crate) visibility_timeout_s: i64,
pub(crate) backoff_base_s: i64,
pub(crate) dead_letter_retention_s: Option<i64>,
pub(crate) next_fire_at: i64,
}
/// The queue mechanism's committed section: the jobs table and the
/// dead-letter storage (`mem-engine-queues` owns the shapes).
/// dead-letter storage. `live` keys by job id (the enqueue-order
/// claim tiebreak reads the id directly); `dead` keys the same id
/// space.
#[derive(Debug, Default)]
pub(crate) struct QueueState {}
pub(crate) struct QueueState {
pub(crate) live: BTreeMap<i64, JobRow>,
pub(crate) dead: BTreeMap<i64, DeadRow>,
}
/// The stream mechanism's committed section: event logs and
/// per-consumer offsets (`mem-engine-streams` owns the shapes).
/// The stream mechanism's committed section: per-stream event logs
/// (keyed by offset, ordered) and the per-(stream, consumer) offset
/// checkpoints.
#[derive(Debug, Default)]
pub(crate) struct StreamState {}
pub(crate) struct StreamState {
pub(crate) logs: BTreeMap<String, BTreeMap<i64, EventRow>>,
pub(crate) offsets: HashMap<(String, String), i64>,
}
/// The named-locks mechanism's committed section: the TTL registry
/// (`mem-engine-locks` owns the shapes).
/// The named-locks mechanism's committed section: the TTL registry.
#[derive(Debug, Default)]
pub(crate) struct LockState {}
pub(crate) struct LockState {
pub(crate) entries: HashMap<String, LockEntry>,
}
/// The scheduler mechanism's committed section: the schedule table
/// (`mem-engine-scheduler-outbox` owns the shapes; the outbox rides
/// queues).
/// The scheduler mechanism's committed section: the schedule table.
#[derive(Debug, Default)]
pub(crate) struct ScheduleState {}
pub(crate) struct ScheduleState {
pub(crate) rows: BTreeMap<String, ScheduleRow>,
}
/// The store's committed state: one guarded section per mechanism.
/// Sections are independent — a mechanism task locks only its own.
// The sections are read by their owning wave-6 mechanism tasks; the
// foundations build alone sees them as dead.
#[allow(dead_code)]
/// The store's committed state: one guarded section per mechanism, the
/// engine-wide id/offset allocators, and the clock read that every
/// timestamp in every section defers to.
///
/// `job_ids`/`event_offsets` are the allocators the tx overlay and the
/// auto-commit paths share: ids and offsets are assigned monotone at
/// allocation time (stage time for a tx, op time for auto-commit),
/// which is the pg-shaped story — an uncommitted allocation gaps out
/// when the tx rolls back (legal: ids are claim-order tiebreaks and
/// offsets are immutable, never renumbered; gaps are ordinary).
pub(crate) struct State {
pub(crate) queues: Mutex<QueueState>,
pub(crate) streams: Mutex<StreamState>,
pub(crate) locks: Mutex<LockState>,
pub(crate) schedules: Mutex<ScheduleState>,
pub(crate) job_ids: AtomicI64,
pub(crate) event_offsets: AtomicI64,
}
impl State {
@@ -76,8 +183,21 @@ impl State {
streams: Mutex::new(StreamState::default()),
locks: Mutex::new(LockState::default()),
schedules: Mutex::new(ScheduleState::default()),
job_ids: AtomicI64::new(0),
event_offsets: AtomicI64::new(0),
}
}
/// The next job id — monotone, never reused (a rolled-back tx's
/// staged allocation gaps).
pub(crate) fn allocate_job_id(&self) -> i64 {
self.job_ids.fetch_add(1, Ordering::Relaxed) + 1
}
/// The next event offset — monotone, immutable, never renumbered.
pub(crate) fn allocate_event_offset(&self) -> i64 {
self.event_offsets.fetch_add(1, Ordering::Relaxed) + 1
}
}
impl std::fmt::Debug for State {
@@ -130,4 +250,13 @@ mod tests {
held.join().expect("the lock churn task never panics");
});
}
#[test]
fn allocators_are_monotone_and_distinct() {
let state = State::new();
let ids: Vec<i64> = (0..8).map(|_| state.allocate_job_id()).collect();
let offsets: Vec<i64> = (0..8).map(|_| state.allocate_event_offset()).collect();
assert_eq!(ids, (1..=8).collect::<Vec<_>>(), "job ids count from 1");
assert_eq!(offsets, (1..=8).collect::<Vec<_>>(), "offsets count from 1");
}
}
+168 -25
View File
@@ -1,40 +1,48 @@
//! The mem engine's store: `MemStore::new`, the engine-native
//! constructor (ADR-024 §4 — the one documented exception to the
//! open prose), and the guarded state it owns.
//! open prose), and the guarded state it owns — plus the `Store` trait
//! surface this crate's whole point rides on.
//!
//! Isolation per open: each `new()` yields a fully isolated instance
//! sharing nothing — a fresh `State` (the guarded per-mechanism
//! core), a fresh `WakeBus` (the wake-routing substrate), a fresh
//! `Clock` — exactly the suite factory's isolation contract;
//! two instances never contend and never observe each other. There is
//! no connection, no file, no pool; teardown is drop — the
//! engine-wide `Drop` runs the close path (the wake bus empties, every
//! open feed disconnects), and the state is gone with the process's
//! references, honestly ephemeral ADR-016-shaped posture:
//! the dependency on `alkstore-mem` *is* the ephemerality
//! declaration).
//! sharing nothing — a fresh `State` (the guarded per-mechanism core),
//! a fresh `WakeBus` (the wake-routing substrate), a fresh `Clock` —
//! exactly the suite factory's isolation contract; two instances never
//! contend and never observe each other. There is no connection, no
//! file, no pool; teardown is drop — the engine-wide `Drop` runs the
//! close path (the wake bus empties, every open feed disconnects), and
//! the state is gone with the process's references, the honest-
//! ephemeral posture (ADR-016 carriers, ADR-024 §2: the dependency on
//! `alkstore-mem` *is* the ephemerality declaration).
//!
//! The `Store` trait implementation is the seam task's artifact
//! (`mem-engine-seam-tx` — the tx overlay is the load-bearing half);
//! this type is the trait boundary's substrate.
//! The `Store` surface wires the mechanism handles (core-owned trait
//! objects — engine types never appear in consumer signatures): `queue`
//! carries its `QueueOpts` stamps (queues are names, not config
//! owners), `stream`/`outbox` carry validated names, `try_lock` rides
//! the TTL registry, `schedule`/`unschedule`/`run_schedules` the
//! collapsed scheduler, `notify`/`listen` the wake substrate, and
//! `begin_tx` allocates the tx overlay (`crate::tx` — the load-bearing
//! seam; `with_tx` is the trait's provided composition).
use std::sync::Arc;
use alkstore::{BoxedFuture, Result, StopToken, Store, TxHandle};
use alkstore::{QueueOpts, ScheduleOpts};
use serde_json::Value;
use crate::clock::Clock;
use crate::queue::MemQueueHandle;
use crate::scheduler;
use crate::state::State;
use crate::stream::MemStreamHandle;
use crate::tx::MemTxHandle;
use crate::wakes::WakeBus;
/// The mem engine's store handle. Consumers take it from
/// `MemStore::new`; once the seam task lands, they hold
/// `alkstore::Store` — the trait is the contract surface, never the
/// engine type.
/// The mem engine's store handle. Consumers hold
/// [`alkstore::Store`] — the trait is the contract surface, never the
/// engine type; `MemStore::new` is the engine-native constructor.
pub struct MemStore {
// Read by the wave-6 mechanism task code (the `Store` seam and the
// handles); the foundations build alone sees `state`/`clock` as dead.
#[allow(dead_code)]
pub(crate) state: Arc<State>,
pub(crate) wakes: Arc<WakeBus>,
#[allow(dead_code)]
pub(crate) clock: Arc<Clock>,
}
@@ -52,9 +60,9 @@ impl std::fmt::Debug for MemStore {
impl MemStore {
/// Construct a fully isolated mem engine instance. Engine-native:
/// no connection string, no file path (ADR-024 §4's
/// documented exception to the open prose). Teardown is drop;
/// each instance shares nothing with any other.
/// no connection string, no file path (ADR-024 §4's documented
/// exception to the open prose). Teardown is drop; each instance
/// shares nothing with any other.
pub fn new() -> Self {
Self {
state: Arc::new(State::new()),
@@ -80,9 +88,110 @@ impl Drop for MemStore {
}
}
impl Store for MemStore {
fn begin_tx(&self) -> BoxedFuture<'_, Result<Box<dyn TxHandle + Send>>> {
let handle = MemTxHandle::begin(self.state.clone(), self.clock.clone(), self.wakes.clone());
Box::pin(async move { Ok(Box::new(handle) as Box<dyn TxHandle + Send>) })
}
fn notify<'a>(&'a self, channel: &str, payload: Value) -> BoxedFuture<'a, Result<()>> {
crate::notify::notify(self.wakes.clone(), channel, payload)
}
fn listen<'a>(
&'a self,
channel: &str,
) -> BoxedFuture<'a, Result<Box<dyn alkstore::WakeReceiver>>> {
let wakes = self.wakes.clone();
let owned = channel.to_string();
Box::pin(async move { crate::notify::listen(wakes, &owned) })
}
fn stream<'a>(
&'a self,
name: &str,
) -> BoxedFuture<'a, Result<Box<dyn alkstore::StreamHandle>>> {
let handle = alkstore::validate_shared_name(name).map(|()| {
MemStreamHandle::new(
name.to_string(),
self.state.clone(),
self.clock.clone(),
self.wakes.clone(),
)
});
Box::pin(async move { handle.map(|h| Box::new(h) as Box<dyn alkstore::StreamHandle>) })
}
fn queue<'a>(
&'a self,
name: &str,
opts: QueueOpts,
) -> BoxedFuture<'a, Result<Box<dyn alkstore::Queue>>> {
let handle = alkstore::validate_shared_name(name).map(|()| {
Box::new(MemQueueHandle::new(
name.to_string(),
opts,
self.state.clone(),
self.clock.clone(),
)) as Box<dyn alkstore::Queue>
});
Box::pin(async move { handle })
}
fn outbox<'a>(&'a self, name: &str) -> BoxedFuture<'a, Result<Box<dyn alkstore::Outbox>>> {
let handle = alkstore::validate_shared_name(name).map(|()| {
Box::new(crate::outbox::MemOutbox::new(
name.to_string(),
self.state.clone(),
self.clock.clone(),
)) as Box<dyn alkstore::Outbox>
});
Box::pin(async move { handle })
}
fn try_lock<'a>(
&'a self,
name: &str,
owner: &str,
ttl: i64,
) -> BoxedFuture<'a, Result<Option<Box<dyn alkstore::Lock>>>> {
let outcome =
crate::lock::try_lock(self.state.clone(), self.clock.clone(), name, owner, ttl);
Box::pin(async move { outcome })
}
fn schedule<'a>(
&'a self,
name: &str,
spec: &str,
queue: &str,
payload: Value,
opts: ScheduleOpts,
) -> BoxedFuture<'a, Result<alkstore::Schedule>> {
scheduler::schedule(
self.state.clone(),
self.clock.clone(),
name,
spec,
queue,
payload,
opts,
)
}
fn unschedule<'a>(&'a self, name: &str) -> BoxedFuture<'a, Result<bool>> {
scheduler::unschedule(self.state.clone(), name)
}
fn run_schedules<'a>(&'a self, stop: StopToken) -> BoxedFuture<'a, Result<()>> {
scheduler::run_schedules(self.state.clone(), self.clock.clone(), stop)
}
}
#[cfg(test)]
mod tests {
use super::*;
use alkstore::EnqueueOpts;
#[test]
fn two_new_instances_share_nothing() {
@@ -145,4 +254,38 @@ mod tests {
"teardown is drop: the engine's close path disconnects every feed"
);
}
#[tokio::test(flavor = "current_thread")]
async fn constructor_entry_validation() {
let store = MemStore::new();
for bad in ["", " ", "__alkstore_q"] {
match store.queue(bad, QueueOpts::default()).await {
Err(alkstore::Error::InvalidName { .. } | alkstore::Error::ReservedName { .. }) => {
}
_other => panic!("queue({bad:?}) rejects"),
}
assert!(store.stream(bad).await.is_err());
assert!(store.outbox(bad).await.is_err());
}
// Nothing was created on a rejected name — the next queue with
// a fresh name starts clean (ids allocate nowhere).
let good = store.queue("good", QueueOpts::default()).await.unwrap();
let id = good
.enqueue(serde_json::json!(1), EnqueueOpts::default())
.await
.unwrap();
assert_eq!(id, 1, "ids count from 1");
}
#[tokio::test(flavor = "current_thread")]
async fn begin_tx_grants_a_boxed_handle_that_commits() {
let store = MemStore::new();
let mut tx = store.begin_tx().await.unwrap();
tx.enqueue_tx("q", EnqueueOpts::default(), serde_json::json!(1))
.await
.unwrap();
tx.commit().await.unwrap();
let q = store.queue("q", QueueOpts::default()).await.unwrap();
assert!(q.get_job(1).await.unwrap().is_some());
}
}
+704
View File
@@ -0,0 +1,704 @@
//! The stream mechanism's mem arm — the in-memory event log +
//! per-consumer offsets (engine-mem.md §Mapping the contract: global
//! FIFO by monotone offset; keyed publish rides the overlay).
//!
//! - Publishes append to the per-stream log under its section guard:
//! engine-assigned offsets (the engine-wide allocator — pg-bigserial
//! shaped, monotone, immutable, never renumbered; a fresh stream's
//! first event does not carry offset 1, the ADR-015 §4 note), keys
//! are carried metadata (a present key must be non-empty —
//! `InvalidName`, ADR-015 §1), and the commit boundary fires one
//! wake per event on the stream's engine-internal wake channel.
//! - Cursor reads are `offset ASC` (global FIFO, ADR-015 §4) with the
//! extent guard at the trait-impl entry (`limit <= 0` → empty
//! `Vec`, ADR-023 §2).
//! - Checkpoints are explicit, monotone saves (a save below the
//! stored checkpoint is a silent no-op — ADR-019 §6; an absent
//! consumer reads 0, ADR-019 §1).
//! - `trim_to` deletes `offset <= horizon` exactly (a negative horizon
//! deletes nothing, idempotently — the boundary argument is total,
//! ADR-023 §2), **wakes nothing**, and surviving events keep their
//! offsets (gaps legal, ADR-015 §5). Reads from a trimmed region
//! resume at the horizon's first remaining row.
//! - `subscribe` attaches the receiver's wake feed **before** the
//! stored-offset read (no commit can fall between — a buffered wake
//! for a commit already re-read costs an empty re-poll, never a
//! missed event), drains the log to the tail, then parks on the
//! feed: wake-driven re-read with no polling requirement; explicit
//! saves only. The one close path is engine drop — the bus close
//! disconnects every feed and `recv()` returns the terminal `None`
//! (ADR-021 §5).
use std::collections::VecDeque;
use std::sync::Arc;
use serde_json::Value;
use alkstore::{
BoxedFuture, Error, EventReceiver, Result, StreamEvent, StreamHandle, validate_local_name,
};
use crate::clock::Clock;
use crate::state::Guard;
use crate::state::{EventRow, State, StreamState};
use crate::wakes::WakeBus;
/// How many events a wake-driven re-read fetches per page while
/// draining to the tail.
const SUBSCRIBE_PAGE: i64 = 256;
/// The engine-internal wake channel a stream's subscribers ride:
/// engine-derived, under the reserved prefix — consumer entry points
/// reject the prefix, so a consumer `listen` can never attach to it
/// (and `notify` can never fire into it). The publish-side wake feed
/// is plumbing, not contract surface.
pub(crate) fn stream_wake_channel(stream: &str) -> String {
format!("{}stream_wake:{}", alkstore::RESERVED_PREFIX, stream)
}
/// Insert one committed event under the streams guard (the auto-commit
/// publish and the commit-merge share this effect). Returns nothing —
/// the caller owns the wake firing at its commit boundary.
pub(crate) fn insert_event(s: &mut StreamState, stream: &str, row: EventRow) {
s.logs
.entry(stream.to_string())
.or_default()
.insert(row.offset, row);
}
/// Read `offset ASC`, `offset > cursor`, up to `limit` (the extent
/// guard is the caller's — this helper is contract-blind to it).
pub(crate) fn read_events(s: &StreamState, stream: &str, from: i64, limit: i64) -> Vec<EventRow> {
s.logs
.get(stream)
.map(|log| {
log.range((from + 1)..)
.take(limit.max(0) as usize)
.map(|(_, row)| row.clone())
.collect()
})
.unwrap_or_default()
}
/// The monotone checkpoint save (ADR-019 §6): a save at-or-below the
/// stored checkpoint is a silent no-op; the direct form, the tx form,
/// and the receiver's no-arg form all land here.
pub(crate) fn save_offset_row(s: &mut StreamState, stream: &str, consumer: &str, offset: i64) {
let key = (stream.to_string(), consumer.to_string());
match s.offsets.get(&key) {
Some(current) if offset <= *current => {}
_ => {
s.offsets.insert(key, offset);
}
}
}
/// The stored checkpoint; absent consumer = 0.
pub(crate) fn get_offset_row(s: &StreamState, stream: &str, consumer: &str) -> i64 {
s.offsets
.get(&(stream.to_string(), consumer.to_string()))
.copied()
.unwrap_or(0)
}
/// `trim_to`: delete `offset <= horizon` (exact boundary); surviving
/// events keep their offsets. Returns the deleted count. Wakes nothing.
pub(crate) fn trim_events(s: &mut StreamState, stream: &str, horizon: i64) -> i64 {
let Some(log) = s.logs.get_mut(stream) else {
return 0;
};
let horizon_next = horizon.saturating_add(1);
let removed: Vec<i64> = log.range(..horizon_next).map(|(k, _)| *k).collect();
let count = removed.len();
for offset in removed {
log.remove(&offset);
}
count as i64
}
fn event_from_row(row: EventRow, stream: &str) -> StreamEvent {
StreamEvent::from_row(
row.offset,
stream.to_string(),
row.key,
row.payload,
row.created_at,
)
}
/// The stream-scoped handle (see the module docs) for one validated
/// stream name.
pub struct MemStreamHandle {
pub(crate) name: String,
pub(crate) state: Arc<State>,
pub(crate) clock: Arc<Clock>,
pub(crate) wakes: Arc<WakeBus>,
}
impl std::fmt::Debug for MemStreamHandle {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("MemStreamHandle")
.field("name", &self.name)
.finish_non_exhaustive()
}
}
impl MemStreamHandle {
pub(crate) fn new(
name: String,
state: Arc<State>,
clock: Arc<Clock>,
wakes: Arc<WakeBus>,
) -> Self {
Self {
name,
state,
clock,
wakes,
}
}
/// The keyed publish is the one publish shape (the plain form is
/// `key = None`).
fn stage_publish(&self, key: Option<String>, payload: Value) -> Result<i64> {
if let Some(k) = &key
&& (k.is_empty() || k.trim().is_empty())
{
return Err(Error::InvalidName { name: k.clone() });
}
let bytes = alkstore::encode_payload(&payload)?;
let now = self.clock.unix_now();
let offset = self.state.allocate_event_offset();
let row = EventRow {
offset,
key,
payload: bytes,
created_at: now,
};
{
let mut s = self.state.streams.guard();
insert_event(&mut s, &self.name, row);
}
// The auto-commit publish's commit boundary is its own
// completion: one wake per event, per-subscriber FIFO.
let channel = stream_wake_channel(&self.name);
self.wakes.fire_commit_wakes([channel]);
Ok(offset)
}
}
impl StreamHandle for MemStreamHandle {
fn name(&self) -> &str {
&self.name
}
fn publish<'a>(&'a self, payload: Value) -> BoxedFuture<'a, Result<i64>> {
match self.stage_publish(None, payload) {
Ok(offset) => Box::pin(async move { Ok(offset) }),
Err(e) => Box::pin(async move { Err(e) }),
}
}
fn publish_with_key<'a>(
&'a self,
key: Option<String>,
payload: Value,
) -> BoxedFuture<'a, Result<i64>> {
match self.stage_publish(key, payload) {
Ok(offset) => Box::pin(async move { Ok(offset) }),
Err(e) => Box::pin(async move { Err(e) }),
}
}
fn read_since<'a>(
&'a self,
offset: i64,
limit: i64,
) -> BoxedFuture<'a, Result<Vec<StreamEvent>>> {
if limit <= 0 {
return Box::pin(async { Ok(Vec::new()) });
}
let events = {
let s = self.state.streams.guard();
read_events(&s, &self.name, offset, limit)
};
let name = self.name.clone();
Box::pin(async move {
Ok(events
.into_iter()
.map(|r| event_from_row(r, &name))
.collect())
})
}
fn read_from_consumer<'a>(
&'a self,
consumer: &str,
limit: i64,
) -> BoxedFuture<'a, Result<Vec<StreamEvent>>> {
if let Err(e) = validate_local_name(consumer) {
return Box::pin(async move { Err(e) });
}
if limit <= 0 {
return Box::pin(async { Ok(Vec::new()) });
}
let name = self.name.clone();
let state = self.state.clone();
let consumer = consumer.to_string();
Box::pin(async move {
let events = {
let s = state.streams.guard();
let from = get_offset_row(&s, &name, &consumer);
read_events(&s, &name, from, limit)
};
Ok(events
.into_iter()
.map(|r| event_from_row(r, &name))
.collect())
})
}
fn save_offset<'a>(&'a self, consumer: &str, offset: i64) -> BoxedFuture<'a, Result<()>> {
if let Err(e) = validate_local_name(consumer) {
return Box::pin(async move { Err(e) });
}
let name = self.name.clone();
let state = self.state.clone();
let consumer = consumer.to_string();
Box::pin(async move {
let mut s = state.streams.guard();
save_offset_row(&mut s, &name, &consumer, offset);
Ok(())
})
}
fn get_offset<'a>(&'a self, consumer: &str) -> BoxedFuture<'a, Result<i64>> {
if let Err(e) = validate_local_name(consumer) {
return Box::pin(async move { Err(e) });
}
let name = self.name.clone();
let state = self.state.clone();
let consumer = consumer.to_string();
Box::pin(async move {
let s = state.streams.guard();
Ok(get_offset_row(&s, &name, &consumer))
})
}
fn trim_to<'a>(&'a self, horizon: i64) -> BoxedFuture<'a, Result<i64>> {
let name = self.name.clone();
let state = self.state.clone();
Box::pin(async move {
let mut s = state.streams.guard();
Ok(trim_events(&mut s, &name, horizon))
})
}
fn subscribe<'a>(&'a self, consumer: &str) -> BoxedFuture<'a, Result<Box<dyn EventReceiver>>> {
if let Err(e) = validate_local_name(consumer) {
return Box::pin(async move { Err(e) });
}
let name = self.name.clone();
let consumer = consumer.to_string();
let state = self.state.clone();
let wakes = self.wakes.clone();
Box::pin(async move {
// Attach first (starts from "now"; attach-after-commit
// no-replay is structural), then read the stored offset and
// drain to the tail.
let feed = wakes.attach(&stream_wake_channel(&name));
let mut cursor;
let mut pending = VecDeque::new();
{
let s = state.streams.guard();
cursor = get_offset_row(&s, &name, &consumer);
loop {
let rows = read_events(&s, &name, cursor, SUBSCRIBE_PAGE);
let got = rows.len();
for row in rows {
cursor = row.offset;
pending.push_back(event_from_row(row, &name));
}
if got < SUBSCRIBE_PAGE as usize {
break;
}
}
}
Ok(Box::new(MemEventReceiver {
stream: name,
consumer,
state,
feed,
pending,
cursor,
position: 0,
}) as Box<dyn EventReceiver>)
})
}
}
/// The durable subscription receiver (see the module docs). Its
/// position is the last-*yielded* event's offset, so `save_offset()`
/// never checkpoints an event the consumer did not observe.
pub struct MemEventReceiver {
stream: String,
consumer: String,
state: Arc<State>,
feed: crate::wakes::WakeFeed,
pending: VecDeque<StreamEvent>,
cursor: i64,
position: i64,
}
impl std::fmt::Debug for MemEventReceiver {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("MemEventReceiver")
.field("stream", &self.stream)
.finish_non_exhaustive()
}
}
impl MemEventReceiver {
/// One forward read from the cursor into the pending buffer.
fn drain_a_page(&mut self) -> Result<()> {
let events = {
let s = self.state.streams.guard();
read_events(&s, &self.stream, self.cursor, SUBSCRIBE_PAGE)
};
for row in events {
self.cursor = row.offset;
self.pending.push_back(event_from_row(row, &self.stream));
}
Ok(())
}
fn pop_positional(&mut self) -> Option<StreamEvent> {
let event = self.pending.pop_front()?;
self.position = event.offset;
Some(event)
}
}
impl EventReceiver for MemEventReceiver {
fn recv<'a>(&'a mut self) -> BoxedFuture<'a, Option<Result<StreamEvent>>> {
Box::pin(async move {
loop {
if let Some(event) = self.pop_positional() {
return Some(Ok(event));
}
// Park on the wake feed: a commit that lands after this
// park fires a wake that releases it; a commit that
// landed before it already buffered a wake (attach-first).
let _wake = match self.feed.rx().recv().await {
Some(wake) => wake,
None => return None,
};
if let Err(e) = self.drain_a_page() {
return Some(Err(database_only(e)));
}
}
})
}
fn try_recv(&mut self) -> Result<Option<StreamEvent>> {
if let Some(event) = self.pop_positional() {
return Ok(Some(event));
}
// A buffered wake is the hint that a commit landed which the
// drain has not seen yet; the poll drains it and re-reads once.
let closed = matches!(
self.feed.rx().try_recv(),
Err(tokio::sync::mpsc::error::TryRecvError::Disconnected)
);
self.drain_a_page()?;
if let Some(event) = self.pop_positional() {
return Ok(Some(event));
}
if closed {
// The bus close (engine drop) is the one close path: once
// the feed is disconnected and nothing is buffered or
// readable, Closed is the terminal poll arm (never reopens).
return Err(Error::Closed);
}
Ok(None)
}
fn read_since<'a>(
&'a mut self,
offset: i64,
limit: i64,
) -> BoxedFuture<'a, Result<Vec<StreamEvent>>> {
if limit <= 0 {
return Box::pin(async { Ok(Vec::new()) });
}
let events = {
let s = self.state.streams.guard();
read_events(&s, &self.stream, offset, limit)
};
let stream = self.stream.clone();
Box::pin(async move {
Ok(events
.into_iter()
.map(|r| event_from_row(r, &stream))
.collect())
})
}
fn save_offset(&mut self) -> Result<()> {
let position = self.position;
if position == 0 {
return Ok(());
}
let mut s = self.state.streams.guard();
save_offset_row(&mut s, &self.stream, &self.consumer, position);
Ok(())
}
fn offset(&self) -> i64 {
self.position
}
}
/// `recv()`'s `Err` arm carries `Database` only (ADR-021 §5): a read
/// error mapped elsewhere is remapped into the opaque fallback with
/// the original preserved as the source (the decode-side posture
/// guard — mem reads cannot fail in practice).
fn database_only(e: Error) -> Error {
match e {
already @ Error::Database(_) => already,
other => Error::database(other),
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::store::MemStore;
use alkstore::{EventReceiver, Store, StreamHandle};
fn handle(store: &MemStore, name: &str) -> MemStreamHandle {
MemStreamHandle::new(
name.to_string(),
store.state.clone(),
store.clock.clone(),
store.wakes.clone(),
)
}
#[tokio::test(flavor = "current_thread")]
async fn publish_read_round_trip_is_global_fifo_by_offset() {
let store = MemStore::new();
let handle = handle(&store, "orders");
let a = handle.publish(serde_json::json!(1)).await.unwrap();
let b = handle.publish(serde_json::json!(2)).await.unwrap();
let keyed = handle
.publish_with_key(Some("cust-7".into()), serde_json::json!(3))
.await
.unwrap();
assert!(a < b && b < keyed, "offsets are monotone per stream");
let events = handle.read_since(0, 10).await.unwrap();
assert_eq!(events.len(), 3);
assert_eq!(
events.iter().map(|e| e.offset).collect::<Vec<_>>(),
vec![a, b, keyed],
"offset ASC — global FIFO regardless of keys"
);
assert_eq!(events[0].stream, "orders");
assert_eq!(events[0].key, None);
assert_eq!(events[2].key.as_deref(), Some("cust-7"));
assert_eq!(events[2].payload_as::<i64>().unwrap(), 3);
assert!(
events
.iter()
.all(|e| e.created_at == store.clock.unix_now())
);
}
#[tokio::test(flavor = "current_thread")]
async fn empty_some_key_is_invalid_at_every_entry() {
let store = MemStore::new();
let handle = handle(&store, "orders");
let err = handle
.publish_with_key(Some(String::new()), serde_json::json!(1))
.await
.unwrap_err();
assert!(matches!(err, alkstore::Error::InvalidName { .. }));
let err = handle
.publish_with_key(Some(" ".into()), serde_json::json!(1))
.await
.unwrap_err();
assert!(matches!(err, alkstore::Error::InvalidName { .. }));
let mut tx = store.begin_tx().await.unwrap();
let err = tx
.publish_with_key_tx("orders", Some(String::new()), serde_json::json!(1))
.await
.unwrap_err();
assert!(matches!(err, alkstore::Error::InvalidName { .. }));
}
#[tokio::test(flavor = "current_thread")]
async fn read_extents_clamp_empty() {
let store = MemStore::new();
let handle = handle(&store, "orders");
handle.publish(serde_json::json!(1)).await.unwrap();
assert!(handle.read_since(0, 0).await.unwrap().is_empty());
assert!(
handle.read_since(0, -3).await.unwrap().is_empty(),
"the LIMIT -1 dialect artifact is dead"
);
}
#[tokio::test(flavor = "current_thread")]
async fn saves_are_monotone_and_absent_consumers_read_zero() {
let store = MemStore::new();
let handle = handle(&store, "orders");
assert_eq!(
handle.get_offset("c1").await.unwrap(),
0,
"an absent consumer reads 0"
);
let tail = handle.publish(serde_json::json!(1)).await.unwrap();
handle.save_offset("c1", tail).await.unwrap();
assert_eq!(handle.get_offset("c1").await.unwrap(), tail);
handle.save_offset("c1", tail - 1).await.unwrap();
assert_eq!(
handle.get_offset("c1").await.unwrap(),
tail,
"a below-checkpoint save is a silent no-op (monotone)"
);
handle.save_offset("c1", tail).await.unwrap();
assert_eq!(
handle.get_offset("c1").await.unwrap(),
tail,
"an equal save is also a no-op"
);
handle.save_offset("c1", tail + 5).await.unwrap();
assert_eq!(handle.get_offset("c1").await.unwrap(), tail + 5);
}
#[tokio::test(flavor = "current_thread")]
async fn trim_is_exact_boundary_and_wakes_nothing() {
let store = MemStore::new();
let handle = handle(&store, "orders");
let a = handle.publish(serde_json::json!(1)).await.unwrap();
let b = handle.publish(serde_json::json!(2)).await.unwrap();
let c = handle.publish(serde_json::json!(3)).await.unwrap();
assert_eq!(
handle.trim_to(b - 1).await.unwrap(),
1,
"exact boundary: offset <= horizon"
);
let events = handle.read_since(0, 10).await.unwrap();
assert_eq!(
events.iter().map(|e| e.offset).collect::<Vec<_>>(),
vec![b, c],
"surviving rows keep their offsets, never renumbered"
);
assert_eq!(
handle.trim_to(-100).await.unwrap(),
0,
"a negative horizon deletes nothing"
);
assert_eq!(handle.trim_to(c).await.unwrap(), 2);
assert_eq!(handle.trim_to(c).await.unwrap(), 0, "idempotent");
// Reads from a trimmed region resume at the horizon's first
// remaining row.
assert!(handle.read_since(0, 10).await.unwrap().is_empty());
let _ = a;
assert_eq!(handle.get_offset("late").await.unwrap(), 0);
}
#[tokio::test(flavor = "current_thread")]
async fn subscriber_replays_from_the_saved_offset_and_ride_wakes() {
let store = MemStore::new();
let handle = handle(&store, "orders");
let _a = handle.publish(serde_json::json!(1)).await.unwrap();
let b = handle.publish(serde_json::json!(2)).await.unwrap();
handle.save_offset("c1", b).await.unwrap();
let mut receiver = handle.subscribe("c1").await.unwrap();
// Everything after the checkpoint replays on attach.
let c = handle.publish(serde_json::json!(3)).await.unwrap();
let first = receiver.recv().await.unwrap().unwrap();
assert_eq!(
first.offset, c,
"replay starts at the stored checkpoint's forward tail"
);
assert_eq!(first.payload_as::<i64>().unwrap(), 3);
// Post-attach publishes deliver without polling.
let d = handle.publish(serde_json::json!(4)).await.unwrap();
let second = receiver.recv().await.unwrap().unwrap();
assert_eq!(second.offset, d);
// The explicit-save receiver form and the direct form compose.
receiver.save_offset().unwrap();
assert_eq!(receiver.offset(), d);
assert_eq!(
handle.get_offset("c1").await.unwrap(),
d,
"the receiver saves the last-yielded offset"
);
// A save before any yield is a no-op on the stored checkpoint.
let mut fresh = handle.subscribe("c2").await.unwrap();
assert_eq!(fresh.offset(), 0);
fresh.save_offset().unwrap();
assert_eq!(handle.get_offset("c2").await.unwrap(), 0);
let _ = _a;
}
#[tokio::test(flavor = "current_thread")]
async fn receiver_try_recv_reads_without_await() {
let store = MemStore::new();
let handle = handle(&store, "orders");
let mut receiver = handle.subscribe("c1").await.unwrap();
assert!(try_none(receiver.as_mut()), "idle reads as Ok(None)");
let a = handle.publish(serde_json::json!(1)).await.unwrap();
let got = receiver.try_recv().unwrap().unwrap();
assert_eq!(got.offset, a);
assert!(try_none(receiver.as_mut()));
}
fn try_none(receiver: &mut dyn EventReceiver) -> bool {
matches!(receiver.try_recv(), Ok(None))
}
#[tokio::test(flavor = "current_thread")]
async fn engine_drop_closes_the_receiver_on_both_arms() {
let (mut receiver, offset) = {
let store = MemStore::new();
let handle = handle(&store, "orders");
let offset = handle.publish(serde_json::json!(1)).await.unwrap();
(handle.subscribe("c1").await.unwrap(), offset)
};
// The engine is gone; the replayed pre-drop event still
// delivers (it was committed and visible), then the receiver
// is closed — terminal on both arms (never reopens).
let delivered = receiver
.try_recv()
.unwrap()
.expect("the pre-drop event replays");
assert_eq!(delivered.offset, offset);
assert!(matches!(receiver.try_recv(), Err(alkstore::Error::Closed)));
let closed = tokio::time::timeout(std::time::Duration::from_millis(200), receiver.recv())
.await
.expect("the closed feed parks no receiver");
assert!(
closed.is_none(),
"the terminal None arm (engine drop is the close cause)"
);
}
#[tokio::test(flavor = "current_thread")]
async fn trim_wakes_nothing_for_an_open_subscriber() {
let store = MemStore::new();
let handle = handle(&store, "orders");
let a = handle.publish(serde_json::json!(1)).await.unwrap();
let b = handle.publish(serde_json::json!(2)).await.unwrap();
let mut receiver = handle.subscribe("c1").await.unwrap();
assert_eq!(receiver.recv().await.unwrap().unwrap().offset, a);
assert_eq!(receiver.recv().await.unwrap().unwrap().offset, b);
handle.trim_to(b).await.unwrap();
let quiet =
tokio::time::timeout(std::time::Duration::from_millis(50), receiver.recv()).await;
assert!(quiet.is_err(), "trim emits no wake and delivers no event");
let _ = receiver.read_since(0, 10).await.unwrap();
}
}
+982
View File
@@ -0,0 +1,982 @@
//! The transactional seam's mem arm (ADR-007): `begin_tx` allocates an
//! **overlay** private to the caller's handle; the eleven `*_tx`
//! methods stage effects in it; commit merges the overlay atomically
//! and fires the overlay's pending wakes; drop discards it.
//!
//! - **Staging** (engine-mem.md §The tx overlay): every `*_tx` call
//! validates its names at the entry point (before any effect),
//! encodes the payload (`Codec` typed, ADR-023 §1), reads one clock
//! instant, resolves/stamps identically to the no-handle paths
//! (ADR-020 §3 — `enqueue_tx` stamps the plain derived defaults
//! 300/3/5/none, `outbox_enqueue_tx` the outbox's 60/5/5 set), and
//! appends the effect to the handle-private overlay. Ids and offsets
//! allocate at stage time from the engine-wide allocators (the
//! pg-shaped story — a rolled-back tx's allocation gaps out; ids and
//! offsets are monotone and never reused, gaps ordinary).
//! - **Reads inside the tx** merge the overlay over the committed
//! state — read-your-own-writes with no round-trips:
//! `get_job_tx`/`get_offset_tx`/`read_since_tx`/
//! `read_from_consumer_tx` see the transaction's own staged
//! enqueues/publishes/saves and the committed rows underneath; every
//! other store view sees none of it pre-commit (an overlay is
//! invisible to every other view until its commit — no isolation
//! questions can arise across handles).
//! - **Commit-merge + wake-at-commit**: `commit` consumes the handle;
//! the merge is one guarded operation (queues → streams, the
//! canonical lock order) and the overlay's pending wakes fire in the
//! same operation, after the merged state is visible — a tx-staged
//! notify delivers nothing until the merge, one wake per staged
//! notify (per-notify, never coalesced). That discipline is what the
//! pg engine had to fix retroactively (`pg-fix-tx-wake`); here it is
//! structural, pinned from birth.
//! - **Drop = discard** (ADR-021 §4): dropping the handle without
//! commit discards the overlay — no-ghosts trivially, because
//! uncommitted state never touched committed state; the discard is
//! panic-safe through an unwinding drop (the overlay is plain local
//! data; dropping runs nothing).
//! - **Numeric domains** (ADR-023 §2) at the trait-impl entry: the
//! stream reads' extents clamp empty (`limit <= 0`), the enqueue
//! boundaries are total (a negative delay resolves to a ready time
//! in the past — ready now), and durations never ride the tx seam.
use std::collections::{HashMap, VecDeque};
use std::sync::Arc;
use serde_json::Value;
use alkstore::{
BoxedFuture, EnqueueOpts, Error, Job, JobState, Result, StreamEvent, TxHandle,
validate_local_name, validate_shared_name,
};
use crate::clock::Clock;
use crate::queue::{build_job_row, get_job_row, job_from_row};
use crate::resolution::{
Stamps, outbox_backing_queue_name, outbox_default_stamps, plain_queue_default_stamps,
resolve_enqueue_opts, stamps_with_override,
};
use crate::state::Guard;
use crate::state::{EventRow, JobRow, State};
use crate::stream::{
get_offset_row, insert_event, read_events, save_offset_row, stream_wake_channel,
};
use crate::wakes::WakeBus;
/// The handle-private overlay: every staged effect, in stage order.
#[derive(Debug, Default)]
struct Overlay {
/// Staged enqueue rows (ids allocated at stage time).
jobs: HashMap<i64, JobRow>,
/// Staged event rows per stream (offsets allocated at stage time;
/// stage order = offset order — the allocators are globally
/// monotone).
events: HashMap<String, Vec<EventRow>>,
/// Staged notifies — channels in stage order, duplicates allowed
/// (per-notify delivery, never coalesced).
notifies: Vec<String>,
/// Staged checkpoint saves (stream, consumer) → the staged max.
saves: HashMap<(String, String), i64>,
}
/// One staged enqueue effect — the shared body of `enqueue_tx` and
/// `outbox_enqueue_tx` (the only difference: the queue name and the
/// stamp source).
fn stage_enqueued(
overlay: &mut Overlay,
state: &State,
clock: &Clock,
queue: &str,
opts: &EnqueueOpts,
payload: Value,
stamps: Stamps,
) -> Result<i64> {
let bytes = alkstore::encode_payload(&payload)?;
let now = clock.unix_now();
let resolved = resolve_enqueue_opts(now, opts);
let id = state.allocate_job_id();
let row = build_job_row(
id,
queue.to_string(),
bytes,
now,
&resolved,
opts.priority,
&stamps,
);
overlay.jobs.insert(id, row);
Ok(id)
}
/// The staged-job read: overlay first, then the committed live/dead
/// tables (dead rows visible — the auto-commit twin's shape, ADR-010
/// §1) — both queue-scoped against the argument.
fn merged_get_job(overlay: &Overlay, state: &State, queue: &str, job_id: i64) -> Option<Job> {
if let Some(row) = overlay.jobs.get(&job_id) {
return if row.queue == queue {
Some(job_from_row(row.clone(), JobState::Pending, None, None))
} else {
None
};
}
let q = state.queues.guard();
let job = get_job_row(&q, job_id)?;
(job.queue == queue).then_some(job)
}
/// The merged event read: the committed log and the staged rows under
/// one offset-ASC ordering (the global-FIFO shape holds across the
/// merge — stage-time offsets are globally monotone). The extent guard
/// (ADR-023 §2) lives here: `limit <= 0` yields the empty `Vec`.
fn merged_read_since(
overlay: &Overlay,
state: &State,
stream: &str,
offset: i64,
limit: i64,
) -> Vec<EventRow> {
if limit <= 0 {
return Vec::new();
}
let committed = {
let s = state.streams.guard();
read_events(&s, stream, offset, limit)
};
let mut merged = committed;
if let Some(staged) = overlay.events.get(stream) {
merged.extend(staged.iter().filter(|row| row.offset > offset).cloned());
}
merged.sort_by_key(|row| row.offset);
merged.truncate(limit as usize);
merged
}
/// The merged checkpoint read (absent everywhere = 0).
fn merged_get_offset(overlay: &Overlay, state: &State, stream: &str, consumer: &str) -> i64 {
let committed = {
let s = state.streams.guard();
get_offset_row(&s, stream, consumer)
};
let staged = overlay
.saves
.get(&(stream.to_string(), consumer.to_string()));
staged.map_or(committed, |s| (*s).max(committed))
}
fn event_of(row: EventRow, stream: &str) -> StreamEvent {
StreamEvent::from_row(
row.offset,
stream.to_string(),
row.key,
row.payload,
row.created_at,
)
}
/// The caller-held mem transaction handle (constructed only through
/// [`crate::MemStore`]'s `begin_tx`; consumers see
/// `Box<dyn TxHandle + Send>`). [`TxHandle::commit`] consumes it;
/// dropping it discards the overlay.
pub struct MemTxHandle {
overlay: Overlay,
state: Arc<State>,
clock: Arc<Clock>,
wakes: Arc<WakeBus>,
}
impl std::fmt::Debug for MemTxHandle {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("MemTxHandle").finish_non_exhaustive()
}
}
impl MemTxHandle {
pub(crate) fn begin(state: Arc<State>, clock: Arc<Clock>, wakes: Arc<WakeBus>) -> Self {
Self {
overlay: Overlay::default(),
state,
clock,
wakes,
}
}
}
impl TxHandle for MemTxHandle {
fn enqueue_tx<'a>(
&'a mut self,
queue: &str,
opts: EnqueueOpts,
payload: Value,
) -> BoxedFuture<'a, Result<i64>> {
if let Err(e) = validate_shared_name(queue) {
return Box::pin(async move { Err(e) });
}
match stage_enqueued(
&mut self.overlay,
&self.state,
&self.clock,
queue,
&opts,
payload,
stamps_with_override(plain_queue_default_stamps(), &opts),
) {
Ok(id) => Box::pin(async move { Ok(id) }),
Err(e) => Box::pin(async move { Err(e) }),
}
}
fn publish_tx<'a>(&'a mut self, stream: &str, payload: Value) -> BoxedFuture<'a, Result<i64>> {
let stream = stream.to_string();
Box::pin(async move { self.publish_with_key_tx(&stream, None, payload).await })
}
fn publish_with_key_tx<'a>(
&'a mut self,
stream: &str,
key: Option<String>,
payload: Value,
) -> BoxedFuture<'a, Result<i64>> {
if let Err(e) = validate_shared_name(stream) {
return Box::pin(async move { Err(e) });
}
if let Some(k) = &key
&& (k.is_empty() || k.trim().is_empty())
{
let bad = k.clone();
return Box::pin(async move { Err(Error::InvalidName { name: bad }) });
}
match self.stage_publish(stream, key, payload) {
Ok(offset) => Box::pin(async move { Ok(offset) }),
Err(e) => Box::pin(async move { Err(e) }),
}
}
fn notify_tx<'a>(&'a mut self, channel: &str, payload: Value) -> BoxedFuture<'a, Result<()>> {
if let Err(e) = validate_shared_name(channel) {
return Box::pin(async move { Err(e) });
}
match self.stage_notify(channel, payload) {
Ok(()) => Box::pin(async { Ok(()) }),
Err(e) => Box::pin(async move { Err(e) }),
}
}
fn save_offset_tx<'a>(
&'a mut self,
stream: &str,
consumer: &str,
offset: i64,
) -> BoxedFuture<'a, Result<()>> {
if let Err(e) = validate_shared_name(stream) {
return Box::pin(async move { Err(e) });
}
if let Err(e) = validate_local_name(consumer) {
return Box::pin(async move { Err(e) });
}
self.stage_save_offset(stream, consumer, offset);
Box::pin(async { Ok(()) })
}
fn get_job_tx<'a>(
&'a mut self,
queue: &str,
job_id: i64,
) -> BoxedFuture<'a, Result<Option<Job>>> {
if let Err(e) = validate_shared_name(queue) {
return Box::pin(async move { Err(e) });
}
let job = merged_get_job(&self.overlay, &self.state, queue, job_id);
Box::pin(async move { Ok(job) })
}
fn get_offset_tx<'a>(
&'a mut self,
stream: &str,
consumer: &str,
) -> BoxedFuture<'a, Result<i64>> {
if let Err(e) = validate_shared_name(stream) {
return Box::pin(async move { Err(e) });
}
if let Err(e) = validate_local_name(consumer) {
return Box::pin(async move { Err(e) });
}
let offset = merged_get_offset(&self.overlay, &self.state, stream, consumer);
Box::pin(async move { Ok(offset) })
}
fn read_since_tx<'a>(
&'a mut self,
stream: &str,
offset: i64,
limit: i64,
) -> BoxedFuture<'a, Result<Vec<StreamEvent>>> {
if let Err(e) = validate_shared_name(stream) {
return Box::pin(async move { Err(e) });
}
let events = merged_read_since(&self.overlay, &self.state, stream, offset, limit);
let stream = stream.to_string();
Box::pin(async move { Ok(events.into_iter().map(|r| event_of(r, &stream)).collect()) })
}
fn read_from_consumer_tx<'a>(
&'a mut self,
stream: &str,
consumer: &str,
limit: i64,
) -> BoxedFuture<'a, Result<Vec<StreamEvent>>> {
if let Err(e) = validate_shared_name(stream) {
return Box::pin(async move { Err(e) });
}
if let Err(e) = validate_local_name(consumer) {
return Box::pin(async move { Err(e) });
}
let events = {
let from = merged_get_offset(&self.overlay, &self.state, stream, consumer);
merged_read_since(&self.overlay, &self.state, stream, from, limit)
};
let stream = stream.to_string();
Box::pin(async move { Ok(events.into_iter().map(|r| event_of(r, &stream)).collect()) })
}
fn outbox_enqueue_tx<'a>(
&'a mut self,
outbox: &str,
opts: EnqueueOpts,
payload: Value,
) -> BoxedFuture<'a, Result<i64>> {
if let Err(e) = validate_shared_name(outbox) {
return Box::pin(async move { Err(e) });
}
let backing = outbox_backing_queue_name(outbox);
match stage_enqueued(
&mut self.overlay,
&self.state,
&self.clock,
&backing,
&opts,
payload,
stamps_with_override(outbox_default_stamps(), &opts),
) {
Ok(id) => Box::pin(async move { Ok(id) }),
Err(e) => Box::pin(async move { Err(e) }),
}
}
fn commit(self: Box<Self>) -> BoxedFuture<'static, Result<()>> {
// Consume the handle: the overlay moves out (the one-shot
// discipline); a dropped-before-commit handle discards instead
// — nothing runs on that path, panic-safe through the unwind.
let MemTxHandle {
overlay,
state,
clock: _,
wakes,
} = *self;
Box::pin(async move {
merge_and_wake(state, wakes, overlay);
Ok(())
})
}
}
impl MemTxHandle {
fn stage_publish(&mut self, stream: &str, key: Option<String>, payload: Value) -> Result<i64> {
let bytes = alkstore::encode_payload(&payload)?;
let now = self.clock.unix_now();
let offset = self.state.allocate_event_offset();
let row = EventRow {
offset,
key,
payload: bytes,
created_at: now,
};
self.overlay
.events
.entry(stream.to_string())
.or_default()
.push(row);
Ok(offset)
}
fn stage_notify(&mut self, channel: &str, payload: Value) -> Result<()> {
// The encoding run is the codec-typed entry check (the
// measured serialization, ADR-020 §4); the wake contract's
// delivery is the channel alone — the bytes are nowhere stored.
let encoded = alkstore::encode_payload(&payload)?;
let _ = encoded;
self.overlay.notifies.push(channel.to_string());
Ok(())
}
fn stage_save_offset(&mut self, stream: &str, consumer: &str, offset: i64) {
let entry = self
.overlay
.saves
.entry((stream.to_string(), consumer.to_string()))
.or_insert(i64::MIN);
if offset > *entry {
*entry = offset;
}
}
}
/// The commit-merge: move every staged effect into the committed state
/// under one guarded operation (the canonical queues → streams lock
/// order; each section's committed view is self-consistent at its
/// boundary), then fire the overlay's pending wakes — the
/// wake-at-commit discipline: nothing delivers until the merge lands.
fn merge_and_wake(state: Arc<State>, wakes: Arc<WakeBus>, mut overlay: Overlay) {
{
let mut q = state.queues.guard();
for (_, row) in std::mem::take(&mut overlay.jobs) {
q.live.insert(row.id, row);
}
}
{
let mut s = state.streams.guard();
for (stream, rows) in overlay.events.iter() {
for row in rows {
insert_event(&mut s, stream, row.clone());
}
}
for ((stream, consumer), offset) in overlay.saves.iter() {
save_offset_row(&mut s, stream, consumer, *offset);
}
}
// Wake-at-commit: staged notifies in stage order (duplicates ride —
// per-notify), plus one wake per staged event on its stream's
// engine-internal feed (per event — never coalesced).
let mut fired: VecDeque<String> = VecDeque::with_capacity(overlay.notifies.len());
fired.extend(overlay.notifies.iter().cloned());
for stream in overlay.events.keys() {
fired.push_back(stream_wake_channel(stream));
}
wakes.fire_commit_wakes(fired);
}
#[cfg(test)]
mod tests {
use super::*;
use crate::store::MemStore;
use alkstore::{Delivery, Outbox, Store, TxHandle};
fn store() -> MemStore {
let store = MemStore::new();
store.clock.freeze(1_000_000);
store
}
async fn listener(store: &MemStore, channel: &str) -> Box<dyn alkstore::WakeReceiver> {
store.listen(channel).await.unwrap()
}
async fn listener_quiet(rx: &mut dyn alkstore::WakeReceiver) {
let outcome = tokio::time::timeout(std::time::Duration::from_millis(60), rx.recv()).await;
assert!(
matches!(outcome, Err(_) | Ok(None)),
"no wake may arrive in the quiet window"
);
}
#[tokio::test(flavor = "current_thread")]
async fn tx_staged_notify_delivers_nothing_pre_commit_one_wake_post() {
let store = store();
let mut rx = listener(&store, "orders").await;
let mut tx = store.begin_tx().await.unwrap();
tx.notify_tx("orders", serde_json::json!({"n": 1}))
.await
.unwrap();
listener_quiet(rx.as_mut()).await;
assert!(tx.commit().await.is_ok());
let wake = tokio::time::timeout(std::time::Duration::from_millis(200), rx.recv())
.await
.expect("the wake delivers")
.expect("a wake");
assert_eq!(wake.channel, "orders");
// Per-notify: a second staged notify delivers its own wake.
let mut tx = store.begin_tx().await.unwrap();
tx.notify_tx("orders", serde_json::json!(2)).await.unwrap();
tx.commit().await.unwrap();
assert!(rx.recv().await.is_some());
assert!(
rx.try_recv().unwrap().is_none(),
"never coalesced — exactly one wake per staged notify"
);
}
#[tokio::test(flavor = "current_thread")]
async fn enqueue_tx_stamps_the_plain_defaults_and_commits_atomically() {
let store = store();
let mut tx = store.begin_tx().await.unwrap();
let id = tx
.enqueue_tx(
"emails",
EnqueueOpts {
delay: Some(30),
max_attempts: Some(7),
..Default::default()
},
serde_json::json!({"to": "a@b.c"}),
)
.await
.unwrap();
let staged = tx.get_job_tx("emails", id).await.unwrap().unwrap();
assert_eq!(
staged.visibility_timeout_s, 300,
"the plain defaults stamp (no handle carries opts)"
);
assert_eq!(staged.max_attempts, 7);
assert_eq!(staged.backoff_base_s, 5);
assert_eq!(staged.dead_letter_retention_s, None);
assert_eq!(staged.run_at, 1_000_030);
assert_eq!(staged.state, JobState::Pending);
let committed_view = store
.queue("emails", alkstore::QueueOpts::default())
.await
.unwrap();
assert!(
committed_view.get_job(id).await.unwrap().is_none(),
"other views see none pre-commit"
);
tx.commit().await.unwrap();
let job = committed_view.get_job(id).await.unwrap().unwrap();
assert_eq!(job.queue, "emails");
assert_eq!(
job.payload,
serde_json::to_vec(&serde_json::json!({"to": "a@b.c"})).unwrap()
);
assert_eq!(job.created_at, 1_000_000);
assert_eq!(
job.run_at, 1_000_030,
"the resolved values, not the raw fields"
);
}
#[tokio::test(flavor = "current_thread")]
async fn publish_tx_reads_and_merge_ordering_plus_wake_at_commit() {
let store = store();
let mut feed = store.wakes().attach(&stream_wake_channel("orders"));
let mut tx = store.begin_tx().await.unwrap();
let a = tx.publish_tx("orders", serde_json::json!(1)).await.unwrap();
let b = tx
.publish_with_key_tx("orders", Some("cust-7".into()), serde_json::json!(2))
.await
.unwrap();
assert!(a < b);
let seen = tx.read_since_tx("orders", 0, 10).await.unwrap();
assert_eq!(
seen.len(),
2,
"the tx's own staged publishes are visible inside the tx"
);
assert_eq!(seen[0].key, None);
assert_eq!(seen[1].key.as_deref(), Some("cust-7"));
assert_eq!(
seen.iter().map(|e| e.offset).collect::<Vec<_>>(),
vec![a, b]
);
let handle = store.stream("orders").await.unwrap();
assert!(
handle.read_since(0, 10).await.unwrap().is_empty(),
"other views see none pre-commit"
);
assert!(
feed.rx().try_recv().is_err(),
"no wake fires before the merge"
);
tx.commit().await.unwrap();
let committed = handle.read_since(0, 10).await.unwrap();
assert_eq!(
committed.iter().map(|e| e.offset).collect::<Vec<_>>(),
vec![a, b]
);
assert_eq!(committed[1].payload_as::<i64>().unwrap(), 2);
assert!(
feed.rx().try_recv().is_ok(),
"one wake fired at the commit (after the merge)"
);
}
#[tokio::test(flavor = "current_thread")]
async fn drop_without_commit_discards_the_whole_overlay_no_ghosts() {
let store = store();
let mut rx = listener(&store, "ch").await;
let handle = store.stream("s").await.unwrap();
let committed_anchor = handle.publish(serde_json::json!("anchor")).await.unwrap();
{
let mut tx = store.begin_tx().await.unwrap();
tx.enqueue_tx("q", EnqueueOpts::default(), serde_json::json!(1))
.await
.unwrap();
tx.publish_tx("s", serde_json::json!(2)).await.unwrap();
tx.notify_tx("ch", serde_json::json!(3)).await.unwrap();
tx.save_offset_tx("s", "c1", 999).await.unwrap();
drop(tx);
}
let q = store
.queue("q", alkstore::QueueOpts::default())
.await
.unwrap();
assert!(q.get_job(1).await.unwrap().is_none(), "no ghost job");
let events = handle.read_since(0, 10).await.unwrap();
assert_eq!(
events.len(),
1,
"no ghost event — only the committed anchor"
);
assert_eq!(events[0].offset, committed_anchor);
assert_eq!(
handle.get_offset("c1").await.unwrap(),
0,
"no ghost offset save"
);
listener_quiet(rx.as_mut()).await;
}
#[tokio::test(flavor = "current_thread")]
async fn other_views_see_none_pre_commit_and_rollback_leaves_no_read_subjects() {
let store = store();
let handle = store.stream("s").await.unwrap();
let mut tx = store.begin_tx().await.unwrap();
let offset = tx.publish_tx("s", serde_json::json!(1)).await.unwrap();
tx.save_offset_tx("s", "c1", offset).await.unwrap();
assert_eq!(
tx.get_offset_tx("s", "c1").await.unwrap(),
offset,
"the tx's own save is visible inside"
);
assert_eq!(
handle.get_offset("c1").await.unwrap(),
0,
"other views see none pre-commit"
);
drop(tx);
assert!(
handle.read_since(0, 10).await.unwrap().is_empty(),
"post-rollback the read's subject is gone"
);
assert_eq!(
handle.get_offset("c1").await.unwrap(),
0,
"no-ghosts for reads"
);
}
#[tokio::test(flavor = "current_thread")]
async fn save_offset_tx_is_monotone_and_only_the_staged_max_merges() {
let store = store();
let mut tx = store.begin_tx().await.unwrap();
tx.save_offset_tx("s", "c", 5).await.unwrap();
tx.save_offset_tx("s", "c", 3).await.unwrap();
assert_eq!(
tx.get_offset_tx("s", "c").await.unwrap(),
5,
"monotone across staged saves"
);
tx.commit().await.unwrap();
let handle = store.stream("s").await.unwrap();
assert_eq!(handle.get_offset("c").await.unwrap(), 5);
// A staged save below the committed checkpoint silently no-ops.
let mut tx = store.begin_tx().await.unwrap();
tx.save_offset_tx("s", "c", 2).await.unwrap();
assert_eq!(tx.get_offset_tx("s", "c").await.unwrap(), 5);
tx.commit().await.unwrap();
assert_eq!(handle.get_offset("c").await.unwrap(), 5);
}
#[cold]
#[inline(never)]
fn mid_flight_panic() -> alkstore::Error {
panic!("mid-flight, after staging all four write kinds")
}
#[tokio::test(flavor = "current_thread")]
async fn with_tx_ok_commits_err_rolls_back_and_panic_rolls_back_through_the_unwind() {
let store = std::sync::Arc::new(store());
// Ok ⇒ commit.
let committed_id = {
let s = store.clone();
let captured = std::sync::Arc::new(std::sync::Mutex::new(0i64));
let sink = captured.clone();
s.with_tx(Box::new(move |tx| {
Box::pin(async move {
let id = tx
.enqueue_tx("q", EnqueueOpts::default(), serde_json::json!(1))
.await?;
*sink.lock().unwrap_or_else(|p| p.into_inner()) = id;
Ok(())
})
}))
.await
.unwrap();
*std::sync::Arc::try_unwrap(captured)
.unwrap_or_else(|_| panic!("the sink outlives the run"))
.lock()
.unwrap_or_else(|p| p.into_inner())
};
assert!(committed_id > 0);
// Err ⇒ rollback.
let rolled = {
let s = store.clone();
s.with_tx(Box::new(|tx| {
Box::pin(async move {
tx.enqueue_tx("q", EnqueueOpts::default(), serde_json::json!(2))
.await?;
Err::<(), alkstore::Error>(alkstore::Error::database(std::io::Error::other(
"business write failed",
)))
})
}))
.await
};
assert!(rolled.is_err());
// Panic across the closure ⇒ rollback through the unwind's drop
// (the overlay discard runs nothing — panic-safe by construction).
let probe_store = store.clone();
let spawned = tokio::spawn(async move {
probe_store
.with_tx(Box::new(|tx| {
Box::pin(async move {
tx.enqueue_tx("q", EnqueueOpts::default(), serde_json::json!(3))
.await?;
tx.publish_tx("s", serde_json::json!(4)).await?;
tx.notify_tx("ch", serde_json::json!(5)).await?;
tx.save_offset_tx("s", "c", 42).await?;
Err::<(), alkstore::Error>(mid_flight_panic())
})
}))
.await
});
let joined = spawned.await;
assert!(joined.is_err(), "the panic surfaced");
assert!(
joined.unwrap_err().is_panic(),
"the JoinError carries the panic"
);
// No residue of any kind (the uniform no-ghosts list).
let q = store
.queue("q", alkstore::QueueOpts::default())
.await
.unwrap();
assert!(
q.get_job(1).await.unwrap().is_some(),
"the Ok leg committed"
);
for id in [2, 3] {
assert!(
q.get_job(id).await.unwrap().is_none(),
"no ghost job (err/panic)"
);
}
let stream = store.stream("s").await.unwrap();
assert!(
stream.read_since(0, 10).await.unwrap().is_empty(),
"no ghost event"
);
assert_eq!(
stream.get_offset("c").await.unwrap(),
0,
"no ghost offset save"
);
let mut rx = listener(&store, "ch").await;
assert!(rx.try_recv().unwrap().is_none(), "no ghost notify wake");
// The store remains usable: a fresh tx commits through the same
// seam (the next id gaps past the discarded allocations').
let usable = {
let s = store.clone();
let captured = std::sync::Arc::new(std::sync::Mutex::new(0i64));
let sink = captured.clone();
s.with_tx(Box::new(move |tx| {
Box::pin(async move {
let id = tx
.enqueue_tx("q", EnqueueOpts::default(), serde_json::json!(6))
.await?;
*sink.lock().unwrap_or_else(|p| p.into_inner()) = id;
Ok(())
})
}))
.await
.unwrap();
*std::sync::Arc::try_unwrap(captured)
.unwrap_or_else(|_| panic!("the sink outlives the run"))
.lock()
.unwrap_or_else(|p| p.into_inner())
};
let next = store
.queue("q", alkstore::QueueOpts::default())
.await
.unwrap();
assert!(next.get_job(usable).await.unwrap().is_some());
assert!(usable > 3, "ids allocate forward — gaps are legal");
}
#[tokio::test(flavor = "current_thread")]
async fn get_job_tx_is_queue_scoped_and_sees_dead_rows() {
let store = store();
let mut tx = store.begin_tx().await.unwrap();
let id = tx
.enqueue_tx("emails", EnqueueOpts::default(), serde_json::json!(1))
.await
.unwrap();
tx.commit().await.unwrap();
let q = store
.queue("emails", alkstore::QueueOpts::default())
.await
.unwrap();
let claimed = q.claim_batch("w", 1).await.unwrap().pop().unwrap();
claimed.fail(Some("boom".into())).await.unwrap();
let mut tx = store.begin_tx().await.unwrap();
let dead = tx.get_job_tx("emails", id).await.unwrap().unwrap();
assert_eq!(
dead.state,
JobState::Dead,
"dead rows are get_job_tx-visible"
);
assert_eq!(dead.last_error.as_deref(), Some("boom"));
assert!(
tx.get_job_tx("other", id).await.unwrap().is_none(),
"queue-scoped: another queue reads None"
);
}
struct NoDelivery;
impl Delivery for NoDelivery {
fn deliver<'a>(
&'a mut self,
_job: Box<dyn alkstore::JobHandle>,
) -> BoxedFuture<'a, Result<()>> {
Box::pin(async { Ok(()) })
}
}
#[tokio::test(flavor = "current_thread")]
async fn outbox_enqueue_tx_stamps_the_60_5_5_set() {
let store = store();
// The concrete handle (crate-internal): the staging inspection
// reads the overlay before the commit-merge.
let mut tx = MemTxHandle::begin(
store.state.clone(),
store.clock.clone(),
store.wakes().clone(),
);
let id = tx
.outbox_enqueue_tx(
"mail",
EnqueueOpts {
delay: Some(5),
max_attempts: Some(2),
..Default::default()
},
serde_json::json!(1),
)
.await
.unwrap();
let staged = tx.overlay.jobs.get(&id).unwrap().clone();
assert_eq!(staged.queue, outbox_backing_queue_name("mail"));
assert_eq!(staged.visibility_timeout_s, 60);
assert_eq!(staged.max_attempts, 2, "the per-job override rides");
assert_eq!(staged.backoff_base_s, 5);
assert_eq!(staged.dead_letter_retention_s, None);
assert_eq!(staged.run_at, 1_000_005);
let err = tx
.get_job_tx(&outbox_backing_queue_name("mail"), id)
.await
.unwrap_err();
assert!(
matches!(err, Error::ReservedName { .. }),
"the derived name is unreachable by get_job_tx"
);
Box::new(tx).commit().await.unwrap();
// Commit makes it claimable by run_once exactly when the
// business write commits.
store.clock.advance(10);
let mut ob = crate::outbox::MemOutbox::new(
"mail".to_string(),
store.state.clone(),
store.clock.clone(),
);
assert!(
ob.run_once("w", &mut NoDelivery).await.unwrap(),
"the committed outbox job is claimable"
);
}
#[tokio::test(flavor = "current_thread")]
async fn entry_validation_on_every_tx_surface() {
let store = store();
let mut tx = MemTxHandle::begin(
store.state.clone(),
store.clock.clone(),
store.wakes().clone(),
);
for bad in ["", " ", "__alkstore_q"] {
assert!(
tx.enqueue_tx(bad, EnqueueOpts::default(), serde_json::json!(1))
.await
.is_err()
);
assert!(tx.publish_tx(bad, serde_json::json!(1)).await.is_err());
assert!(
tx.publish_with_key_tx(bad, None, serde_json::json!(1))
.await
.is_err()
);
assert!(tx.notify_tx(bad, serde_json::json!(1)).await.is_err());
assert!(tx.save_offset_tx(bad, "c", 1).await.is_err());
assert!(tx.get_job_tx(bad, 1).await.is_err());
assert!(tx.get_offset_tx(bad, "c").await.is_err());
assert!(tx.read_since_tx(bad, 0, 1).await.is_err());
assert!(tx.read_from_consumer_tx(bad, "c", 1).await.is_err());
assert!(
tx.outbox_enqueue_tx(bad, EnqueueOpts::default(), serde_json::json!(1))
.await
.is_err()
);
}
// Consumer-local identifiers take the non-empty rule only.
let err = tx.save_offset_tx("s", "", 1).await.unwrap_err();
assert!(matches!(err, Error::InvalidName { .. }));
let err = tx.read_from_consumer_tx("s", "", 1).await.unwrap_err();
assert!(matches!(err, Error::InvalidName { .. }));
assert!(
tx.overlay.jobs.is_empty(),
"validation errors stage nothing"
);
}
#[tokio::test(flavor = "current_thread")]
async fn numeric_domain_guards_on_the_tx_reads() {
let store = store();
let mut tx = store.begin_tx().await.unwrap();
assert!(tx.read_since_tx("s", 0, 0).await.unwrap().is_empty());
assert!(tx.read_since_tx("s", 0, -5).await.unwrap().is_empty());
assert!(
tx.read_from_consumer_tx("s", "c", -1)
.await
.unwrap()
.is_empty()
);
tx.commit().await.unwrap();
}
#[tokio::test(flavor = "current_thread")]
async fn concurrent_txs_merge_serially_without_isolation_questions() {
let store = store();
let mut t1 = store.begin_tx().await.unwrap();
let mut t2 = store.begin_tx().await.unwrap();
let a = t1.publish_tx("s", serde_json::json!(1)).await.unwrap();
let b = t2.publish_tx("s", serde_json::json!(2)).await.unwrap();
t1.commit().await.unwrap();
t2.commit().await.unwrap();
let handle = store.stream("s").await.unwrap();
let events = handle.read_since(0, 10).await.unwrap();
assert_eq!(
events.iter().map(|e| e.offset).collect::<Vec<_>>(),
vec![a, b],
"both commits merged; reads yield offset ASC — no isolation questions across handles"
);
assert!(
store.wakes().commit_watermark() >= 2,
"each commit fired one boundary"
);
}
}