ADR-024 — the mem engine adopted as the family's third engine (wave 6; release readiness renumbered to wave 7)
Row-first gate: consumer-inventory.md gains an engine-tier section — the mem-engine need recorded operator-authority, dated 2026-10-10 (wasm32 sandboxing across the alk protocol family + downstream ffi/napi/python-adapter economics; rusqlite/tokio-postgres structurally out on wasm, so mem is the only triad member the sandbox can carry). ADR-024 (decisions/024-mem-engine.md) pins the decision set: (1) separate `alkstore-mem` crate — discharges and amends ADR-001 §4 (superseded in part: full contract-v1 engine, not an "implementation convenience"; annotation on ADR-001 §4 + Status, class-1 window still open); (2) honest-ephemeral posture — no durability, single-process, ephemeral by design, never fleet-valid (ADR-010 §3's shared-pool predicate fails structurally), riding ADR-016's carriers unchanged (identity + docs + matrix); (3) full contract v1 tier with the asymmetry classification — `PayloadTooLarge` never produced (SQLite arm), no reconnection concept (reconnect-wake/watcher rows absent from mem's per-engine column), receiver close at engine drop, wake coalescing N/A; (4) `MemStore::new()` engine-native constructor — the one documented exception to the open-prose, core-contract.md Store concept amended; shared-instance (two opens → one engine) rejected with the ADR-016 §4-style re-entry gate (consumer-inventory row), register stays closed; (5) wasm32 — compile-clean for wasm32-unknown-unknown plus the suite's mem column green on host under a current-thread runtime flavor are wave-6 acceptance items; on-target suite execution scoped out with a named collapse condition (wasm-bindgen-test adapter crate) — a scope decision, not a pending-client hedge; (6) mem is not a core dev-dependency — core's doc examples stay mock-based, ADR-001 §4's doctest posture declines. engine-mem.md (draft): the in-process mapping — guarded per-mechanism state (no pool, no schema bootstrap, no forwarder, no spawn_blocking seam), the tx overlay (staged *_tx effects merged atomically at commit; read-your-own-writes easy, drop-=-rollback trivial; wake-at-commit structural — the seam pg needed pg-fix-tx-wake for, mem pins from birth via the existing tx-commit-atomicity rows), ADR-012 §2 third owner of the curve/opts/@every arithmetic (three-way equivalence, ADR-022's StoreFactory reused verbatim — born pinned). deployment.md: host-semantics row (single-process, ephemeral, never fleet-valid, wasm32-clean), mem connection-budget subsection (none — nothing to budget), durability-knobs row (the honest row is the absence), toolchain row (wasm32 subset + current-thread-clean, acceptance-cited). README/overview: engine-mem.md row, ADR-024 row, family table entry (alkstore-mem, no driver deps), current-state updated to the implementation phase with engine specs' stable statuses reflected. implementation.md: wave table gains wave 6 (mem engine, depends waves 1 + 5) and renumbers release readiness to wave 7 (fuzzing decision — the one remaining decided-at-implementation deferral — rides wave 7); wave-6 section records the born-pinned posture, acceptance items, the one suite-harness pre-task (past_stamp_sleep blocking sleep → async sleep, review 003's properties.rs:1683 note), and the window riders (review 003 Finding 1: the pg transient-fault seam, MEDIUM, rides wave 6; Findings 2–3 follow their park into wave 7's pre-release); wave-6 review gate defined; Decided points updated (mem discharged, fuzzing renumbered); stale wave-6 references swept across review docs and the landed tasks' notes (release-readiness items renumbered or annotated). Docs-only change; verified by cross-reference audit (all ADR/OQ/anchor references resolve, residual wave-6 mentions are correct under the new numbering or carry dated annotations).
This commit is contained in:
glm-5.3-flash committed
1 parent
dbb17068cc
commit
060f262e62
17 files changed
+608
-53
No files matched your search
@@ -0,0 +1,145 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-10 (initial spec — ADR-024 adopted the mem
|
||||
engine as wave 6; implementation pending, spec grows with the wave's
|
||||
decomposition)
|
||||
---
|
||||
|
||||
# Mem engine
|
||||
|
||||
The `alkstore-mem` engine implements
|
||||
[core-contract.md](core-contract.md) on guarded in-memory structures.
|
||||
This spec records WHAT the engine is — its architecture, the tx
|
||||
overlay, the asymmetry classification — not code-level HOW. Decisions
|
||||
live in ADRs ([ADR-024](decisions/024-mem-engine.md) is this engine's
|
||||
charter); contract obligations live in the core spec.
|
||||
|
||||
## Identity and posture
|
||||
|
||||
- Separate workspace crate (ADR-024 §1): `alkstore`, tokio (sync +
|
||||
time — the wasm-supported subset), no driver dependencies. Natively
|
||||
async; no `spawn_blocking` seam anywhere; internals are
|
||||
current-thread-runtime-clean by construction, and the suite's mem
|
||||
column runs *under a current-thread flavor* to keep that honest.
|
||||
- **Honest-ephemeral** (ADR-024 §2): no durability, single-process,
|
||||
ephemeral by design; legitimate production for workloads that
|
||||
accept ephemerality; never fleet-valid (the
|
||||
[ADR-010](decisions/010-queue-semantics-depth.md) §3 shared-pool
|
||||
predicate fails structurally — there is no pool). The dependency
|
||||
edge on `alkstore-mem` is the ephemerality declaration; see
|
||||
[deployment.md](deployment.md).
|
||||
- **Full contract v1** (ADR-024 §3): every mechanism, every guarantee
|
||||
row. Born pinned — the engine lands against the finished
|
||||
version-stamped suite ([ADR-022](decisions/022-contract-suite-layout.md)'s
|
||||
`StoreFactory`, reused verbatim as a third factory column), which
|
||||
is also what pins its engine-owned arithmetic three-way
|
||||
([ADR-012](decisions/012-forked-substrate-design.md) §2's one-owner
|
||||
rule: the backoff curve, opts resolution, and the `@every` parser
|
||||
get mem's own implementations; equivalence with the other two
|
||||
engines is suite-pinned).
|
||||
- wasm32: `alkstore-mem` compiling clean for
|
||||
`wasm32-unknown-unknown` is an acceptance item of its wave
|
||||
(ADR-024 §5) and an identity line of this spec.
|
||||
|
||||
## Architecture
|
||||
|
||||
Everything the other engines realize with connections, schemas, and
|
||||
daemons, the mem engine realizes with process-owned state:
|
||||
|
||||
- **No connections at all** — no pool (pg's), no writer/reader slots
|
||||
(SQLite's), no listener. The store instance *is* the state: guarded
|
||||
per-mechanism tables (queues + jobs, dead letters, streams +
|
||||
offsets, locks, schedules) owned by the engine instance.
|
||||
- **No background machinery to bootstrap** — no schema bootstrap, no
|
||||
watcher thread (nothing cross-connection to poll), no forwarder.
|
||||
The only async machinery is tokio-time-driven: lock TTLs, the
|
||||
scheduler's boundary waits, and wake delivery.
|
||||
- **Wake substrate**: in-process channels. Wakes deliver at the
|
||||
commit boundary directly to each open receiver — no batching
|
||||
boundary exists, so coalescing is not applicable; per-subscriber
|
||||
delivery order is FIFO by commit order.
|
||||
|
||||
### The tx overlay
|
||||
|
||||
The tx seam's mem realization: a `begin_tx()` allocates an **overlay**
|
||||
private to the caller's handle.
|
||||
|
||||
- `*_tx` methods stage effects in the overlay (enqueues, publishes,
|
||||
notifies); reads *inside* the tx merge the overlay over committed
|
||||
state — read-your-own-writes with no round-trips.
|
||||
- **Commit** merges the overlay atomically (a single guarded merge)
|
||||
and fires the overlay's pending wakes — *wake-at-commit* is the
|
||||
discipline-critical property: a notify staged in the tx delivers
|
||||
nothing until the overlay merges, exactly the seam the pg engine
|
||||
had to fix retroactively (`pg-fix-tx-wake`) and the suite's
|
||||
tx-commit-atomicity rows pin.
|
||||
- **Drop** discards the overlay — drop = rollback with no ghosts,
|
||||
trivially, because uncommitted state never touched committed state.
|
||||
- No isolation questions can arise across handles: an overlay is
|
||||
invisible to every other store/view until its commit.
|
||||
|
||||
## Mapping the contract
|
||||
|
||||
| Contract piece | Engine realization |
|
||||
|---|---|
|
||||
| notify / listen | staged notify in the tx overlay; delivered at commit to open receivers. Commit-atomicity is structural ([ADR-007](decisions/007-transactional-seam.md)) — the merge and the wake fire are one operation |
|
||||
| streams | in-memory event log + per-consumer offsets, global-FIFO by monotone offset; keyed publish rides the overlay ([ADR-015](decisions/015-streams-depth.md)); `trim_to` mutates the log and wakes nothing |
|
||||
| queues | the ADR-010 state machine over an in-memory job table (visibility, reclaim, the equal-jitter curve engine-owned, dead-letter move, the no-stranded-rows sweep) — all driver-free, swept on caller-driven `sweep_expired` |
|
||||
| named locks | in-memory TTL registry ([ADR-008](decisions/008-contract-v1-pinning.md) §7's row); expiry via tokio time; single-instance contention semantics |
|
||||
| scheduler / outbox | collapse shape ([ADR-009](decisions/009-scheduler-collapse.md)): schedule table, tick (boundary advance + 64-boundary catch-up cap), leadership on the `__alkstore_scheduler` lock — single-instance leadership is trivially owned but the lock machinery still runs, per ADR-009 §4's engine-uniform rule; outbox = helper over queues ([ADR-014](decisions/014-outbox-tx-enqueue.md)) |
|
||||
| begin_tx | overlay allocation ([ADR-007](decisions/007-transactional-seam.md)) |
|
||||
| handle ops | direct `.await`s on guarded state; `encode_payload` via core (ADR-020 §4) |
|
||||
|
||||
## Asymmetry classification (ADR-024 §3)
|
||||
|
||||
| Asymmetry | Mem posture |
|
||||
|---|---|
|
||||
| `PayloadTooLarge` | never produced, any size (SQLite occurrence arm); the non-occurrence arm wires in mem's suite column ([ADR-016](decisions/016-deployment-honesty.md) §5) |
|
||||
| Reconnection | does not exist — no connection can drop; the reconnect-wake and watcher-death rows are pg/SQLite-shaped and absent from mem's column (rows are per-engine wired; nothing cross-wires or weakens) |
|
||||
| Receiver close | one close path: engine drop closes open receivers (process exit/shutdown); contract close-arm shapes apply unchanged ([ADR-021](decisions/021-tx-reads-and-value-shape-fixes.md)) |
|
||||
| Wake coalescing | not applicable — direct at-commit delivery, per-subscriber FIFO; the engine spec line above pins it |
|
||||
|
||||
## Isolation and the suite factory
|
||||
|
||||
Each `MemStore::new()` yields a distinct engine instance sharing
|
||||
nothing (ADR-024 §4's isolation rule — the contract's *Store* concept
|
||||
amended to admit the engine-native constructor). The suite factory:
|
||||
a fresh store per run, teardown = drop; the mem column carries the
|
||||
mechanism rows that express on it, omits the engine-shaped rows that
|
||||
cannot (the table above), and runs under the current-thread flavor
|
||||
(ADR-024 §5 item 4 — the suite harness's `past_stamp_sleep` async
|
||||
sleep is the one pre-task the wave carries). Running the suite
|
||||
*on* the wasm target is scoped out behind a named collapse condition
|
||||
(a wasm-bindgen-test adapter crate), per ADR-024 §5.
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|---|---|---|
|
||||
| [001](decisions/001-crate-split.md) | Crate split | §4's mem clause collapsed + amended by [ADR-024](decisions/024-mem-engine.md) — separate crate, full contract v1 |
|
||||
| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract | opaque wake + re-read; in-process channels as the wake substrate; no reconnection concept (no connection can drop) |
|
||||
| [007](decisions/007-transactional-seam.md) | Tx seam | the tx overlay — staging, commit-merge, drop-discard; wake-at-commit structural |
|
||||
| [008](decisions/008-contract-v1-pinning.md) | Contract v1 | full-surface implementation; §6 config split (constructor in the engine crate); §7 locks row as a TTL registry |
|
||||
| [009](decisions/009-scheduler-collapse.md) | Scheduler collapse | schedule table + tick + catch-up + leadership lock (trivially owned, still run — engine-uniform) |
|
||||
| [010](decisions/010-queue-semantics-depth.md) | Queue depth | the state machine driver-free; curve engine-owned; §3's fleet predicate decides never-fleet-valid |
|
||||
| [012](decisions/012-forked-substrate-design.md) | Fork design | §2 one-owner rule, third owner: backoff curve, opts resolution, `@every` parser — three-way equivalence suite-pinned |
|
||||
| [014](decisions/014-outbox-tx-enqueue.md) | Outbox tx enqueue | `outbox_enqueue_tx` staged in the overlay |
|
||||
| [015](decisions/015-streams-depth.md) | Streams depth | in-memory log + per-consumer offsets; keyed publish; `trim_to` (wakes nothing) |
|
||||
| [016](decisions/016-deployment-honesty.md) | Deployment honesty | no runtime surface — honest-ephemeral by compile-time identity + docs + matrix row; `PayloadTooLarge` never produced |
|
||||
| [017](decisions/017-contract-versioning.md) | Contract versioning | manifest pins core per §4; suite column is the pairing instrument |
|
||||
| [019](decisions/019-mechanism-handle-surfaces.md) | Handle surfaces | boxed handle traits implemented over guarded state |
|
||||
| [020](decisions/020-enqueue-opt-semantics-and-bridges.md) | Enqueue options | opts resolution engine-owned (third copy); payload encoding via core |
|
||||
| [021](decisions/021-tx-reads-and-value-shape-fixes.md) | Third review round | drop = rollback trivial by overlay discard; receiver close at engine drop |
|
||||
| [022](decisions/022-contract-suite-layout.md) | Suite layout | factory reused verbatim — third column, born pinned |
|
||||
| [024](decisions/024-mem-engine.md) | Mem engine charter | crate placement, honest-ephemeral posture, full tier, constructor + shared-instance rejection, wasm acceptance items |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in
|
||||
[open-questions.md](open-questions.md). This spec opens none — the
|
||||
register is closed, and every decision this engine needed is in
|
||||
[ADR-024](decisions/024-mem-engine.md). The two deliberately-not-OQ
|
||||
dispositions live there: the shared-instance constructor rejection
|
||||
(§4, re-entry = a consumer-inventory row) and the on-target wasm
|
||||
suite execution scope-out (§5, collapse condition = an adapter
|
||||
crate).
|
||||
Reference in new issue
Block a user