alkdev/alkstore
6
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ADR-024 — the mem engine adopted as the family's third engine (wave 6; release readiness renumbered to wave 7)

Browse Source 
Row-first gate: consumer-inventory.md gains an engine-tier section — the mem-engine need recorded operator-authority, dated 2026-10-10 (wasm32 sandboxing across the alk protocol family + downstream ffi/napi/python-adapter economics; rusqlite/tokio-postgres structurally out on wasm, so mem is the only triad member the sandbox can carry).

ADR-024 (decisions/024-mem-engine.md) pins the decision set: (1) separate `alkstore-mem` crate — discharges and amends ADR-001 §4 (superseded in part: full contract-v1 engine, not an "implementation convenience"; annotation on ADR-001 §4 + Status, class-1 window still open); (2) honest-ephemeral posture — no durability, single-process, ephemeral by design, never fleet-valid (ADR-010 §3's shared-pool predicate fails structurally), riding ADR-016's carriers unchanged (identity + docs + matrix); (3) full contract v1 tier with the asymmetry classification — `PayloadTooLarge` never produced (SQLite arm), no reconnection concept (reconnect-wake/watcher rows absent from mem's per-engine column), receiver close at engine drop, wake coalescing N/A; (4) `MemStore::new()` engine-native constructor — the one documented exception to the open-prose, core-contract.md Store concept amended; shared-instance (two opens → one engine) rejected with the ADR-016 §4-style re-entry gate (consumer-inventory row), register stays closed; (5) wasm32 — compile-clean for wasm32-unknown-unknown plus the suite's mem column green on host under a current-thread runtime flavor are wave-6 acceptance items; on-target suite execution scoped out with a named collapse condition (wasm-bindgen-test adapter crate) — a scope decision, not a pending-client hedge; (6) mem is not a core dev-dependency — core's doc examples stay mock-based, ADR-001 §4's doctest posture declines.

engine-mem.md (draft): the in-process mapping — guarded per-mechanism state (no pool, no schema bootstrap, no forwarder, no spawn_blocking seam), the tx overlay (staged *_tx effects merged atomically at commit; read-your-own-writes easy, drop-=-rollback trivial; wake-at-commit structural — the seam pg needed pg-fix-tx-wake for, mem pins from birth via the existing tx-commit-atomicity rows), ADR-012 §2 third owner of the curve/opts/@every arithmetic (three-way equivalence, ADR-022's StoreFactory reused verbatim — born pinned).

deployment.md: host-semantics row (single-process, ephemeral, never fleet-valid, wasm32-clean), mem connection-budget subsection (none — nothing to budget), durability-knobs row (the honest row is the absence), toolchain row (wasm32 subset + current-thread-clean, acceptance-cited). README/overview: engine-mem.md row, ADR-024 row, family table entry (alkstore-mem, no driver deps), current-state updated to the implementation phase with engine specs' stable statuses reflected.

implementation.md: wave table gains wave 6 (mem engine, depends waves 1 + 5) and renumbers release readiness to wave 7 (fuzzing decision — the one remaining decided-at-implementation deferral — rides wave 7); wave-6 section records the born-pinned posture, acceptance items, the one suite-harness pre-task (past_stamp_sleep blocking sleep → async sleep, review 003's properties.rs:1683 note), and the window riders (review 003 Finding 1: the pg transient-fault seam, MEDIUM, rides wave 6; Findings 2–3 follow their park into wave 7's pre-release); wave-6 review gate defined; Decided points updated (mem discharged, fuzzing renumbered); stale wave-6 references swept across review docs and the landed tasks' notes (release-readiness items renumbered or annotated).

Docs-only change; verified by cross-reference audit (all ADR/OQ/anchor references resolve, residual wave-6 mentions are correct under the new numbering or carry dated annotations).
This commit is contained in:
glm-5.3-flash committed 2026-10-10 09:42:40 +00:00
1 parent dbb17068cc
commit 060f262e62
17 files changed
+608 -53
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/architecture/decisions/024-mem-engine.md
+238
View File
@@ -0,0 +1,238 @@
# ADR-024: The mem engine — a third engine (`alkstore-mem`), full contract v1, honest-ephemeral
## Status
Accepted (2026-10-10, Phase 1 — [ADR-001](001-crate-split.md) §4's
implementation-time deferral collapses here, and this ADR *amends* that
§4's framing rather than merely discharging it: the mem engine is a
full contract-v1 implementation, not the "implementation convenience,
not a contract artifact" §4 originally imagined. The class-1
amend-in-place window ([ADR-017](017-contract-versioning.md) §2) is
still open — no versioned artifact exists — so that amendment is
pre-implementation. The row-first gate is satisfied: the
consumer-inventory engine-tier row (operator-authority, 2026-10-10)
precedes this decision.)
## Context
ADR-001 §4 allowed "a mem-shaped engine (in-tree or separate test
crate) … if the test story turns out to want it," deferred to
implementation time. The plan recorded the deferral as a wave-6
decision point ("or earlier if the test story demands the mem engine
sooner"). The wave-5 review gate closed with the contract suite green
on both engines, and two things now make the mem engine concrete:
- **The consumer pause + the wasm motivation.** The paused consumer
work that surfaced the need is not a test story but a consumer
story: the alk protocol crates (alkcall, alktty, alktunnels,
alksocks) are wasm-compatible at the protocol level by design, and
sandboxing — browser or otherwise — plus downstream
ffi/napi/python-adapter economics are the operator's recorded need
(the consumer-inventory engine-tier row, 2026-10-10). On those
paths the mem engine is not a hedge: rusqlite and tokio-postgres
are structurally out on wasm, so it is the *only* engine the
sandbox can carry.
- **The contract suite changed the economics.** A third engine used
to mean re-deriving a verification story; ADR-022's shared
`StoreFactory` suite means a third engine is born pinned — wiring a
third factory column onto already-authored, version-stamped rows.
And the class-1 amendment window (ADR-017 §2) is still open: any
contract wobble a third full implementation exposes is
amend-in-place now, a migration after first release.
The scale is honest to state up front: this is a *third full engine*,
not alkblobs' mem tier (a `BTreeMap` behind a handful of CAS
methods). It implements the queue state machine (visibility/reclaim/
dead-letter/sweep), the scheduler (boundaries, catch-up, leadership),
the tx seam, streams with per-consumer offsets and trim, TTL locks,
and wakes. What it does *not* carry is the machinery that made the
Postgres wave build-heavy: no pooling, no schema bootstrap, no
forwarder, no `spawn_blocking` seam, no reconnection concept.
## Decision
### 1. Crate placement — a separate `alkstore-mem` crate
The mem engine is a **separate workspace member**, `alkstore-mem`,
implementing the core traits. Dependency surface: `alkstore` plus
tokio (the wasm-supported subset — sync and time), no driver
dependencies; payload encoding rides core's `encode_payload`
([ADR-020](020-enqueue-opt-semantics-and-bridges.md) §4). This
discharges ADR-001 §4's "in-tree or separate crate" fork as
**separate**, for the reason that ADR stated in reverse for the other
engines: engine crates are additive by construction (ADR-001 §
Consequences), and a mem engine in core would contradict ADR-001 §1's
contents list (core is the contract artifact and nothing else).
[ADR-013](013-fold-substrate-into-sqlite.md)'s "no fourth crate"
rationale does not apply — that ruled the *substrate's* placement
(fork lineage, versioning surface, link collisions); none of those
facts attach to a greenfield in-memory implementation.
### 2. Maturity posture — honest-ephemeral, stated by the family's carriers
The mem engine's posture is the honest middle between "test
convenience" (ADR-001 §4's original framing — *superseded*) and
"durable production engine": **no durability, single-process,
ephemeral by design.** Concretely:
- A **legitimate production choice for workloads that accept
ephemerality** — the consumer's dependency on `alkstore-mem` *is*
the ephemerality declaration (the ADR-016 pattern: the dependency
edge states the truth and cannot drift from it).
- **Never fleet-valid** — the fleet predicate ([ADR-010](010-queue-semantics-depth.md)
§3's shared-pool rule) fails structurally: there is no pool, no
process boundary, nothing shared. Two `MemStore` instances share
nothing by design (§4).
- Honesty rides the [ADR-016](016-deployment-honesty.md) carriers
unchanged — no new surface: compile-time engine identity (the
crate name), the engine crate's docs as the posture statement, and
the [deployment matrix](../deployment.md) gaining the ephemeral
row. The compile-time identity carries the honesty; nothing runtime
is fabricated to detect what the dependency already declares.
- alkblobs' mem line ("never a production story, never fleet-valid")
is deliberately *not* copied verbatim — "never production" would
contradict the sandbox use case that motivates this engine.
### 3. Full contract v1 — tier and asymmetry classification
The tier is **full contract v1**: every mechanism, every guarantee
row, no reduced profile. A reduced test-only engine never gets
born-pinned (the suite's equivalence rows would be absent from its
column, exactly the drift the suite exists to prevent), and the
motivating need is consumer-shaped, not test-shaped.
The documented engine asymmetries classify as:
- **`PayloadTooLarge`**: never produced, at any size — the SQLite
occurrence posture. The variant stays contract-uniform and
matchable ([ADR-016](016-deployment-honesty.md) §5); mem's suite
column wires the non-occurrence arm.
- **Reconnection**: no such concept — no connection can drop. The
reconnect-wake and watcher-death rows are pg/SQLite-shaped and are
*per-engine wired* in the suite, so mem's column simply does not
carry them; nothing weakens across engines.
- **Receiver close**: the one close path is engine drop (process
exit/shutdown) closing open receivers — the contract's close-arm
shapes apply, engine-realized as drop.
- **Wake coalescing**: not applicable — wakes deliver at commit
through in-process channels directly to each subscriber; there is
no batching boundary to coalesce across. The engine spec pins the
delivery shape.
### 4. Constructor shape — engine-native, isolated per open; shared instance rejected
Constructors live in engine crates ([ADR-008](008-contract-v1-pinning.md)
§6), so the mem engine's constructor is engine-native: **`MemStore::new()`**
— no connection string, no file path. This is the one documented
exception to the core contract's open-prose ("connection string or
file path"), and the prose is amended to name the exception
(pre-implementation, class-1). **Isolation per open**: each
`MemStore::new()` yields a distinct engine instance sharing nothing —
exactly the suite factory's isolation contract; teardown is drop.
**A shared-instance constructor (two opens → one engine) is rejected
for v1, with a re-entry gate**: it introduces real concurrency
questions (claims and locks contended across two handles) that no
consumer row names. The re-entry condition is a consumer-inventory
row naming the two-opens-one-engine need — the ADR-016 §4 pattern
(decision against, gate named); the open-questions register stays
closed.
### 5. Wasm32 — compile-clean is an acceptance item; on-target suite execution is scoped out with a named collapse condition
The wasm motivation makes wasm compatibility load-bearing, so it
moves from aspiration to **wave-6 acceptance item**:
1. `alkstore-mem` **compiles clean for `wasm32-unknown-unknown`**;
2. the workspace host gates stay green (`cargo build` / `test` /
clippy `-D warnings` / fmt);
3. the contract-suite mem column passes on the host;
4. the mem column runs under a **current-thread runtime flavor** —
the test that keeps internals current-thread-clean, which is what
wasm needs (no `spawn_blocking` anywhere, tokio's sync + time
subset only). One suite-harness pre-task follows from this: the
suite's `past_stamp_sleep` blocking sleep (the review-003 posture
note, `properties.rs:1683`) becomes an async sleep — the only
suite-side change the wave carries.
Running the *suite itself* on the wasm target (wasm-bindgen-test
harnessing) is a different investment and is **scoped out of wave 6**
with a named collapse condition: **a wasm-bindgen-test adapter crate
for the suite existing**. This is a scope decision with a definite
trigger, not a pending-client hedge — the acceptance the sandbox and
adapter motivation actually needs (compile-clean, host-verified
contract conformance) is delivered by items 1–4 and independently
verifiable.
### 6. What the mem engine is not
- **Not a core dev-dependency.** ADR-001 §4's "doctests" posture
declines: core takes no engine dev-dependency (the core crate has
no driver dependencies, ever, dev-side included); core contract
doc examples remain mock-store-based; mem's doctests live in the
mem crate itself.
- **Not a re-derivation of the suite** — ADR-022's factory is reused
verbatim (its consequences already anticipated this).
- **Not new contract surface.** The contract text changes here are
the core-prose exception note (§4) and documentation rows (§2) —
amend-in-place pre-release; the trait surface is untouched.
## Consequences
**Positive**
- The wasm/sandbox and downstream-adapter paths gain their engine;
every future family consumer that must run in a sandbox can depend
on `alkstore-mem` without inventing one.
- A third full implementation stresses contract v1 while the class-1
window is open — born pinned against the finished suite rather
than pinned retroactively; any wobble is amend-in-place now.
- The suite's arithmetic-equivalence pinning goes three-way (backoff
curve, opts resolution — [ADR-012](012-forked-substrate-design.md)
§2's one-owner rule applied per engine, third owner).
- Zero new contract surface; the ADR-016 carriers absorb the posture
(identity + docs + matrix row), no runtime descriptor.
**Negative**
- The ADR-017 lockstep duty now covers three engines: a class-2
(additive) contract change needs three adoption releases; a third
engine's tests/opt machinery ride every wave forever.
- Engine-owned arithmetic gains a third owner (backoff curve, opts
resolution, the `@every` parser) — three copies to keep honest, and
the honesty mechanism is the suite's equivalence rows, not shared
code.
- The mem engine's maintenance is *real* engine maintenance (the
queue/scheduler/stream/lock machinery), not a weekend adapter —
the honest-scale flag this ADR's Context records.
## References
- [ADR-001](001-crate-split.md) §4 — the deferral this ADR collapses;
§4's "implementation convenience, not a contract artifact" framing
superseded (amend-in-place, class-1 window open).
- [ADR-022](022-contract-suite-layout.md) — the shared `StoreFactory`
suite reused verbatim; the born-pinned mechanism.
- [ADR-017](017-contract-versioning.md) — §2's class-1 window (still
open, what makes this amendment cheap); §4's pairing carriers the
mem engine adopts (manifest pin, suite column, engine docs).
- [ADR-016](016-deployment-honesty.md) — the carriers the
honest-ephemeral posture rides (§2); the compile-time-identity
honesty pattern (§3); §4's rejection-with-gate pattern §4 uses; §5
the `PayloadTooLarge` pinning mem's column wires.
- [ADR-010](010-queue-semantics-depth.md) §3 — the fleet predicate
that decides "never fleet-valid" structurally.
- [ADR-008](008-contract-v1-pinning.md) §6 — constructors live in
engine crates (the `MemStore::new()` home).
- [ADR-012](012-forked-substrate-design.md) §2 — the one-owner rule
the engine-owned arithmetic applies per engine.
- `docs/research/consumer-inventory.md` §Engine-tier inventory — the
row-first record (operator-authority, 2026-10-10) that precedes
this decision.
- Review 003 (`docs/reviews/003-wave-5-general-review.md`) — the
`properties.rs:1683` current-thread posture note the suite-harness
pre-task disposes.
- `docs/architecture/engine-mem.md` — the engine spec this ADR
requires as a wave-6 deliverable.
- `docs/plans/implementation.md` — the wave-6 insertion (release
readiness renumbered to wave 7).