alkstore/docs/architecture/decisions/024-mem-engine.md

ADR-024: The mem engine — a third engine (alkstore-mem), full contract v1, honest-ephemeral

Status

Accepted (2026-10-10, Phase 1 — ADR-001 §4's implementation-time deferral collapses here, and this ADR amends that §4's framing rather than merely discharging it: the mem engine is a full contract-v1 implementation, not the "implementation convenience, not a contract artifact" §4 originally imagined. The class-1 amend-in-place window (ADR-017 §2) is still open — no versioned artifact exists — so that amendment is pre-implementation. The row-first gate is satisfied: the consumer-inventory engine-tier row (operator-authority, 2026-10-10) precedes this decision.)

Context

ADR-001 §4 allowed "a mem-shaped engine (in-tree or separate test crate) … if the test story turns out to want it," deferred to implementation time. The plan recorded the deferral as a wave-6 decision point ("or earlier if the test story demands the mem engine sooner"). The wave-5 review gate closed with the contract suite green on both engines, and two things now make the mem engine concrete:

• The consumer pause + the wasm motivation. The paused consumer work that surfaced the need is not a test story but a consumer story: the alk protocol crates (alkcall, alktty, alktunnels, alksocks) are wasm-compatible at the protocol level by design, and sandboxing — browser or otherwise — plus downstream ffi/napi/python-adapter economics are the operator's recorded need (the consumer-inventory engine-tier row, 2026-10-10). On those paths the mem engine is not a hedge: rusqlite and tokio-postgres are structurally out on wasm, so it is the only engine the sandbox can carry.
• The contract suite changed the economics. A third engine used to mean re-deriving a verification story; ADR-022's shared StoreFactory suite means a third engine is born pinned — wiring a third factory column onto already-authored, version-stamped rows. And the class-1 amendment window (ADR-017 §2) is still open: any contract wobble a third full implementation exposes is amend-in-place now, a migration after first release.

The scale is honest to state up front: this is a third full engine, not alkblobs' mem tier (a BTreeMap behind a handful of CAS methods). It implements the queue state machine (visibility/reclaim/ dead-letter/sweep), the scheduler (boundaries, catch-up, leadership), the tx seam, streams with per-consumer offsets and trim, TTL locks, and wakes. What it does not carry is the machinery that made the Postgres wave build-heavy: no pooling, no schema bootstrap, no forwarder, no spawn_blocking seam, no reconnection concept.

Decision

1. Crate placement — a separate alkstore-mem crate

The mem engine is a separate workspace member, alkstore-mem, implementing the core traits. Dependency surface: alkstore plus tokio (the wasm-supported subset — sync and time), no driver dependencies; payload encoding rides core's encode_payload (ADR-020 §4). This discharges ADR-001 §4's "in-tree or separate crate" fork as separate, for the reason that ADR stated in reverse for the other engines: engine crates are additive by construction (ADR-001 § Consequences), and a mem engine in core would contradict ADR-001 §1's contents list (core is the contract artifact and nothing else). ADR-013's "no fourth crate" rationale does not apply — that ruled the substrate's placement (fork lineage, versioning surface, link collisions); none of those facts attach to a greenfield in-memory implementation.

2. Maturity posture — honest-ephemeral, stated by the family's carriers

The mem engine's posture is the honest middle between "test convenience" (ADR-001 §4's original framing — superseded) and "durable production engine": no durability, single-process, ephemeral by design. Concretely:

• A legitimate production choice for workloads that accept ephemerality — the consumer's dependency on alkstore-mem is the ephemerality declaration (the ADR-016 pattern: the dependency edge states the truth and cannot drift from it).
• Never fleet-valid — the fleet predicate (ADR-010 §3's shared-pool rule) fails structurally: there is no pool, no process boundary, nothing shared. Two MemStore instances share nothing by design (§4).
• Honesty rides the ADR-016 carriers unchanged — no new surface: compile-time engine identity (the crate name), the engine crate's docs as the posture statement, and the deployment matrix gaining the ephemeral row. The compile-time identity carries the honesty; nothing runtime is fabricated to detect what the dependency already declares.
• alkblobs' mem line ("never a production story, never fleet-valid") is deliberately not copied verbatim — "never production" would contradict the sandbox use case that motivates this engine.

3. Full contract v1 — tier and asymmetry classification

The tier is full contract v1: every mechanism, every guarantee row, no reduced profile. A reduced test-only engine never gets born-pinned (the suite's equivalence rows would be absent from its column, exactly the drift the suite exists to prevent), and the motivating need is consumer-shaped, not test-shaped.

The documented engine asymmetries classify as:

• PayloadTooLarge: never produced, at any size — the SQLite occurrence posture. The variant stays contract-uniform and matchable (ADR-016 §5); mem's suite column wires the non-occurrence arm.
• Reconnection: no such concept — no connection can drop. The reconnect-wake and watcher-death rows are pg/SQLite-shaped and are per-engine wired in the suite, so mem's column simply does not carry them; nothing weakens across engines.
• Receiver close: the one close path is engine drop (process exit/shutdown) closing open receivers — the contract's close-arm shapes apply, engine-realized as drop.
• Wake coalescing: not applicable — wakes deliver at commit through in-process channels directly to each subscriber; there is no batching boundary to coalesce across. The engine spec pins the delivery shape.

4. Constructor shape — engine-native, isolated per open; shared instance rejected

Constructors live in engine crates (ADR-008 §6), so the mem engine's constructor is engine-native: MemStore::new() — no connection string, no file path. This is the one documented exception to the core contract's open-prose ("connection string or file path"), and the prose is amended to name the exception (pre-implementation, class-1). Isolation per open: each MemStore::new() yields a distinct engine instance sharing nothing — exactly the suite factory's isolation contract; teardown is drop.

A shared-instance constructor (two opens → one engine) is rejected for v1, with a re-entry gate: it introduces real concurrency questions (claims and locks contended across two handles) that no consumer row names. The re-entry condition is a consumer-inventory row naming the two-opens-one-engine need — the ADR-016 §4 pattern (decision against, gate named); the open-questions register stays closed.

5. Wasm32 — compile-clean is an acceptance item; on-target suite execution is scoped out with a named collapse condition

The wasm motivation makes wasm compatibility load-bearing, so it moves from aspiration to wave-6 acceptance item:

1. alkstore-mem compiles clean for wasm32-unknown-unknown;
2. the workspace host gates stay green (cargo build / test / clippy -D warnings / fmt);
3. the contract-suite mem column passes on the host;
4. the mem column runs under a current-thread runtime flavor — the test that keeps internals current-thread-clean, which is what wasm needs (no spawn_blocking anywhere, tokio's sync + time subset only). One suite-harness pre-task follows from this: the suite's past_stamp_sleep blocking sleep (the review-003 posture note, properties.rs:1683) becomes an async sleep — the only suite-side change the wave carries.

Running the suite itself on the wasm target (wasm-bindgen-test harnessing) is a different investment and is scoped out of wave 6 with a named collapse condition: a wasm-bindgen-test adapter crate for the suite existing. This is a scope decision with a definite trigger, not a pending-client hedge — the acceptance the sandbox and adapter motivation actually needs (compile-clean, host-verified contract conformance) is delivered by items 1–4 and independently verifiable.

6. What the mem engine is not

• Not a core dev-dependency. ADR-001 §4's "doctests" posture declines: core takes no engine dev-dependency (the core crate has no driver dependencies, ever, dev-side included); core contract doc examples remain mock-store-based; mem's doctests live in the mem crate itself.
• Not a re-derivation of the suite — ADR-022's factory is reused verbatim (its consequences already anticipated this).
• Not new contract surface. The contract text changes here are the core-prose exception note (§4) and documentation rows (§2) — amend-in-place pre-release; the trait surface is untouched.

Consequences

Positive

• The wasm/sandbox and downstream-adapter paths gain their engine; every future family consumer that must run in a sandbox can depend on alkstore-mem without inventing one.
• A third full implementation stresses contract v1 while the class-1 window is open — born pinned against the finished suite rather than pinned retroactively; any wobble is amend-in-place now.
• The suite's arithmetic-equivalence pinning goes three-way (backoff curve, opts resolution — ADR-012 §2's one-owner rule applied per engine, third owner).
• Zero new contract surface; the ADR-016 carriers absorb the posture (identity + docs + matrix row), no runtime descriptor.

Negative

• The ADR-017 lockstep duty now covers three engines: a class-2 (additive) contract change needs three adoption releases; a third engine's tests/opt machinery ride every wave forever.
• Engine-owned arithmetic gains a third owner (backoff curve, opts resolution, the @every parser) — three copies to keep honest, and the honesty mechanism is the suite's equivalence rows, not shared code.
• The mem engine's maintenance is real engine maintenance (the queue/scheduler/stream/lock machinery), not a weekend adapter — the honest-scale flag this ADR's Context records.

References

• ADR-001 §4 — the deferral this ADR collapses; §4's "implementation convenience, not a contract artifact" framing superseded (amend-in-place, class-1 window open).
• ADR-022 — the shared StoreFactory suite reused verbatim; the born-pinned mechanism.
• ADR-017 — §2's class-1 window (still open, what makes this amendment cheap); §4's pairing carriers the mem engine adopts (manifest pin, suite column, engine docs).
• ADR-016 — the carriers the honest-ephemeral posture rides (§2); the compile-time-identity honesty pattern (§3); §4's rejection-with-gate pattern §4 uses; §5 the PayloadTooLarge pinning mem's column wires.
• ADR-010 §3 — the fleet predicate that decides "never fleet-valid" structurally.
• ADR-008 §6 — constructors live in engine crates (the MemStore::new() home).
• ADR-012 §2 — the one-owner rule the engine-owned arithmetic applies per engine.
• docs/research/consumer-inventory.md §Engine-tier inventory — the row-first record (operator-authority, 2026-10-10) that precedes this decision.
• Review 003 (docs/reviews/003-wave-5-general-review.md) — the properties.rs:1683 current-thread posture note the suite-harness pre-task disposes.
• docs/architecture/engine-mem.md — the engine spec this ADR requires as a wave-6 deliverable.
• docs/plans/implementation.md — the wave-6 insertion (release readiness renumbered to wave 7).