ADR-024 — the mem engine adopted as the family's third engine (wave 6; release readiness renumbered to wave 7)
Row-first gate: consumer-inventory.md gains an engine-tier section — the mem-engine need recorded operator-authority, dated 2026-10-10 (wasm32 sandboxing across the alk protocol family + downstream ffi/napi/python-adapter economics; rusqlite/tokio-postgres structurally out on wasm, so mem is the only triad member the sandbox can carry). ADR-024 (decisions/024-mem-engine.md) pins the decision set: (1) separate `alkstore-mem` crate — discharges and amends ADR-001 §4 (superseded in part: full contract-v1 engine, not an "implementation convenience"; annotation on ADR-001 §4 + Status, class-1 window still open); (2) honest-ephemeral posture — no durability, single-process, ephemeral by design, never fleet-valid (ADR-010 §3's shared-pool predicate fails structurally), riding ADR-016's carriers unchanged (identity + docs + matrix); (3) full contract v1 tier with the asymmetry classification — `PayloadTooLarge` never produced (SQLite arm), no reconnection concept (reconnect-wake/watcher rows absent from mem's per-engine column), receiver close at engine drop, wake coalescing N/A; (4) `MemStore::new()` engine-native constructor — the one documented exception to the open-prose, core-contract.md Store concept amended; shared-instance (two opens → one engine) rejected with the ADR-016 §4-style re-entry gate (consumer-inventory row), register stays closed; (5) wasm32 — compile-clean for wasm32-unknown-unknown plus the suite's mem column green on host under a current-thread runtime flavor are wave-6 acceptance items; on-target suite execution scoped out with a named collapse condition (wasm-bindgen-test adapter crate) — a scope decision, not a pending-client hedge; (6) mem is not a core dev-dependency — core's doc examples stay mock-based, ADR-001 §4's doctest posture declines. engine-mem.md (draft): the in-process mapping — guarded per-mechanism state (no pool, no schema bootstrap, no forwarder, no spawn_blocking seam), the tx overlay (staged *_tx effects merged atomically at commit; read-your-own-writes easy, drop-=-rollback trivial; wake-at-commit structural — the seam pg needed pg-fix-tx-wake for, mem pins from birth via the existing tx-commit-atomicity rows), ADR-012 §2 third owner of the curve/opts/@every arithmetic (three-way equivalence, ADR-022's StoreFactory reused verbatim — born pinned). deployment.md: host-semantics row (single-process, ephemeral, never fleet-valid, wasm32-clean), mem connection-budget subsection (none — nothing to budget), durability-knobs row (the honest row is the absence), toolchain row (wasm32 subset + current-thread-clean, acceptance-cited). README/overview: engine-mem.md row, ADR-024 row, family table entry (alkstore-mem, no driver deps), current-state updated to the implementation phase with engine specs' stable statuses reflected. implementation.md: wave table gains wave 6 (mem engine, depends waves 1 + 5) and renumbers release readiness to wave 7 (fuzzing decision — the one remaining decided-at-implementation deferral — rides wave 7); wave-6 section records the born-pinned posture, acceptance items, the one suite-harness pre-task (past_stamp_sleep blocking sleep → async sleep, review 003's properties.rs:1683 note), and the window riders (review 003 Finding 1: the pg transient-fault seam, MEDIUM, rides wave 6; Findings 2–3 follow their park into wave 7's pre-release); wave-6 review gate defined; Decided points updated (mem discharged, fuzzing renumbered); stale wave-6 references swept across review docs and the landed tasks' notes (release-readiness items renumbered or annotated). Docs-only change; verified by cross-reference audit (all ADR/OQ/anchor references resolve, residual wave-6 mentions are correct under the new numbering or carry dated annotations).
This commit is contained in:
glm-5.3-flash committed
1 parent
dbb17068cc
commit
060f262e62
17 files changed
+608
-53
No files matched your search
+19
-11
@@ -1,23 +1,29 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-08 (ADR-023: fourth review round resolved — waves-1–2 general-review findings)
|
||||
last_updated: 2026-10-10 (ADR-024: the mem engine adopted as a third
|
||||
engine — engine-mem.md added, ADR table row, index updated for the
|
||||
implementation phase)
|
||||
---
|
||||
|
||||
# alkstore — Architecture
|
||||
|
||||
Architecture documentation for the alkstore project: one reactive
|
||||
store interface (notify, streams, queues, locks, scheduler, outbox)
|
||||
over SQLite and Postgres, with each engine native underneath
|
||||
(see [overview.md](overview.md)).
|
||||
over SQLite, Postgres, and the in-process mem engine, with each
|
||||
engine native underneath (see [overview.md](overview.md)).
|
||||
|
||||
## Current State
|
||||
|
||||
**Phase 1 (Architecture) — in progress.** Phase 0 is complete
|
||||
(`docs/research/phase-0.md`): both POCs ran and passed, the scope
|
||||
inventory is confirmed, and the crate split, drivers, and ownership
|
||||
postures are decided. This directory carries the architecture spec
|
||||
build-out over that evidence base; all spec documents are `draft`
|
||||
pending architecture review and OQ resolution.
|
||||
**Phase 1 (Implementation) — waves 1–5 implemented and reviewed.**
|
||||
Phase 0 is complete (`docs/research/phase-0.md`): both POCs ran and
|
||||
passed, the scope inventory is confirmed, and the crate split,
|
||||
drivers, and ownership postures are decided. All open questions are
|
||||
resolved (through ADR-023), and the engine specs flipped `stable` at
|
||||
the wave-5 review gate. Waves 1–5 (core, substrate fork, SQLite
|
||||
engine, Postgres engine, contract suite) are implemented and reviewed
|
||||
per `docs/plans/implementation.md`. The mem engine was adopted as a
|
||||
third engine by [ADR-024](decisions/024-mem-engine.md) (2026-10-10,
|
||||
wave 6 — release readiness renumbered to wave 7).
|
||||
|
||||
## Architecture Documents
|
||||
|
||||
@@ -25,8 +31,9 @@ pending architecture review and OQ resolution.
|
||||
|---|---|---|---|
|
||||
| [overview.md](overview.md) | draft | Crate family, feature surface, non-goals, evidence base | — |
|
||||
| [core-contract.md](core-contract.md) | draft | The unified trait surface, delivery guarantees, tx seam | OQ-10 (resolved) |
|
||||
| [engine-sqlite.md](engine-sqlite.md) | draft | SQLite engine: forked-substrate/rusqlite mapping | OQ-06 (resolved), OQ-12 (resolved), OQ-13 (resolved) |
|
||||
| [engine-postgres.md](engine-postgres.md) | draft | Postgres engine: tokio-postgres/LISTEN mapping | OQ-08 (resolved), OQ-12 (resolved), OQ-13 (resolved) |
|
||||
| [engine-sqlite.md](engine-sqlite.md) | stable | SQLite engine: forked-substrate/rusqlite mapping | OQ-06 (resolved), OQ-12 (resolved), OQ-13 (resolved) |
|
||||
| [engine-postgres.md](engine-postgres.md) | stable | Postgres engine: tokio-postgres/LISTEN mapping | OQ-08 (resolved), OQ-12 (resolved), OQ-13 (resolved) |
|
||||
| [engine-mem.md](engine-mem.md) | draft | Mem engine: in-process ephemeral mapping of the contract ([ADR-024](decisions/024-mem-engine.md)) | — |
|
||||
| [queues.md](queues.md) | draft | Queue/scheduler/outbox semantics depth (resolved: ADR-009/ADR-010) | OQ-05 (resolved), OQ-09 (resolved), OQ-06 (resolved) |
|
||||
| [deployment.md](deployment.md) | draft | Host semantics, connection budgets, knobs, matrix | OQ-08 (resolved) |
|
||||
| [open-questions.md](open-questions.md) | draft | OQ tracker (promoted from OQ-ST register) | — |
|
||||
@@ -58,6 +65,7 @@ pending architecture review and OQ resolution.
|
||||
| [021](decisions/021-tx-reads-and-value-shape-fixes.md) | Third review round — tx-read methods on `TxHandle`, `Job.claimed_at`, `schedule()` queue-argument validation, drop = rollback, receiver close/error arms | Accepted |
|
||||
| [022](decisions/022-contract-suite-layout.md) | Contract-suite layout — shared internal suite crate, properties parameterized over a store factory (discharges ADR-017 §4.2's deferral) | Accepted |
|
||||
| [023](decisions/023-fourth-review-round.md) | Fourth review round — `encode_payload` typed (`Codec`), numeric-argument domains by kind (extents clamp empty, durations reject, boundaries total), plain-path SQLite open (URI flag dropped), watcher cadence 1 ms default + `SqliteOpts` knob | Accepted |
|
||||
| [024](decisions/024-mem-engine.md) | The mem engine — a third engine (`alkstore-mem`), full contract v1, honest-ephemeral posture (collapses + amends ADR-001 §4's deferral) | Accepted |
|
||||
|
||||
## Open Questions
|
||||
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-08 (ADR-023 — fourth review round: numeric-argument domains pinned by kind, encode_payload typed, plain-path SQLite open, watcher cadence posture)
|
||||
last_updated: 2026-10-10 (ADR-024 — the mem engine joins the family;
|
||||
the Store concept prose admits the engine-native constructor
|
||||
exception)
|
||||
---
|
||||
|
||||
# Core contract
|
||||
@@ -37,7 +39,11 @@ descriptor; the
|
||||
- **Store** — what a consumer opens from a connection string or file
|
||||
path ([ADR-001](decisions/001-crate-split.md)): one handle, the engine
|
||||
behind it chosen at open time. Consumer code never branches on
|
||||
engine type (guiding principle 4).
|
||||
engine type (guiding principle 4). (The mem engine is the one
|
||||
documented exception to the open-prose: its constructor is
|
||||
engine-native — `MemStore::new()`, no connection string — per
|
||||
[ADR-024](decisions/024-mem-engine.md) §4; the engine choice
|
||||
remains a dependency-graph fact made before any `Store` exists.)
|
||||
- **Mechanism** — one of the surface's coordination families: notify,
|
||||
streams, queues (+outbox), locks, scheduler. Delivery guarantees are
|
||||
pinned per mechanism in [ADR-006](decisions/006-wake-and-delivery-contract.md)'s
|
||||
|
||||
@@ -3,7 +3,10 @@
|
||||
## Status
|
||||
|
||||
Accepted *(capability-flags content annotated 2026-10-06 by
|
||||
[ADR-016](016-deployment-honesty.md))*
|
||||
[ADR-016](016-deployment-honesty.md); §4 superseded in part
|
||||
2026-10-10 by [ADR-024](024-mem-engine.md) — the mem engine exists as
|
||||
the separate `alkstore-mem` crate, a full contract-v1 implementation,
|
||||
not an implementation convenience)*
|
||||
|
||||
## Context
|
||||
|
||||
@@ -56,6 +59,12 @@ The project ships as a **family of crates**:
|
||||
a third implementation for tests and doctests, if the test story
|
||||
turns out to want it. Treated as an implementation convenience, not
|
||||
a contract artifact — decided at implementation time, not now.
|
||||
*(Superseded in part 2026-10-10 by
|
||||
[ADR-024](024-mem-engine.md): the mem engine exists as the
|
||||
separate `alkstore-mem` crate — full contract v1, no durability,
|
||||
single-process, ephemeral by design, never fleet-valid, wasm32
|
||||
compile-clean by acceptance. Not a test convenience, and not a
|
||||
core dev-dependency: core's doc examples stay mock-based.)*
|
||||
|
||||
Downstream consumers depend on `alkstore` plus exactly one engine
|
||||
crate. A consumer binary links at most one driver.
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
# ADR-024: The mem engine — a third engine (`alkstore-mem`), full contract v1, honest-ephemeral
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (2026-10-10, Phase 1 — [ADR-001](001-crate-split.md) §4's
|
||||
implementation-time deferral collapses here, and this ADR *amends* that
|
||||
§4's framing rather than merely discharging it: the mem engine is a
|
||||
full contract-v1 implementation, not the "implementation convenience,
|
||||
not a contract artifact" §4 originally imagined. The class-1
|
||||
amend-in-place window ([ADR-017](017-contract-versioning.md) §2) is
|
||||
still open — no versioned artifact exists — so that amendment is
|
||||
pre-implementation. The row-first gate is satisfied: the
|
||||
consumer-inventory engine-tier row (operator-authority, 2026-10-10)
|
||||
precedes this decision.)
|
||||
|
||||
## Context
|
||||
|
||||
ADR-001 §4 allowed "a mem-shaped engine (in-tree or separate test
|
||||
crate) … if the test story turns out to want it," deferred to
|
||||
implementation time. The plan recorded the deferral as a wave-6
|
||||
decision point ("or earlier if the test story demands the mem engine
|
||||
sooner"). The wave-5 review gate closed with the contract suite green
|
||||
on both engines, and two things now make the mem engine concrete:
|
||||
|
||||
- **The consumer pause + the wasm motivation.** The paused consumer
|
||||
work that surfaced the need is not a test story but a consumer
|
||||
story: the alk protocol crates (alkcall, alktty, alktunnels,
|
||||
alksocks) are wasm-compatible at the protocol level by design, and
|
||||
sandboxing — browser or otherwise — plus downstream
|
||||
ffi/napi/python-adapter economics are the operator's recorded need
|
||||
(the consumer-inventory engine-tier row, 2026-10-10). On those
|
||||
paths the mem engine is not a hedge: rusqlite and tokio-postgres
|
||||
are structurally out on wasm, so it is the *only* engine the
|
||||
sandbox can carry.
|
||||
- **The contract suite changed the economics.** A third engine used
|
||||
to mean re-deriving a verification story; ADR-022's shared
|
||||
`StoreFactory` suite means a third engine is born pinned — wiring a
|
||||
third factory column onto already-authored, version-stamped rows.
|
||||
And the class-1 amendment window (ADR-017 §2) is still open: any
|
||||
contract wobble a third full implementation exposes is
|
||||
amend-in-place now, a migration after first release.
|
||||
|
||||
The scale is honest to state up front: this is a *third full engine*,
|
||||
not alkblobs' mem tier (a `BTreeMap` behind a handful of CAS
|
||||
methods). It implements the queue state machine (visibility/reclaim/
|
||||
dead-letter/sweep), the scheduler (boundaries, catch-up, leadership),
|
||||
the tx seam, streams with per-consumer offsets and trim, TTL locks,
|
||||
and wakes. What it does *not* carry is the machinery that made the
|
||||
Postgres wave build-heavy: no pooling, no schema bootstrap, no
|
||||
forwarder, no `spawn_blocking` seam, no reconnection concept.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Crate placement — a separate `alkstore-mem` crate
|
||||
|
||||
The mem engine is a **separate workspace member**, `alkstore-mem`,
|
||||
implementing the core traits. Dependency surface: `alkstore` plus
|
||||
tokio (the wasm-supported subset — sync and time), no driver
|
||||
dependencies; payload encoding rides core's `encode_payload`
|
||||
([ADR-020](020-enqueue-opt-semantics-and-bridges.md) §4). This
|
||||
discharges ADR-001 §4's "in-tree or separate crate" fork as
|
||||
**separate**, for the reason that ADR stated in reverse for the other
|
||||
engines: engine crates are additive by construction (ADR-001 §
|
||||
Consequences), and a mem engine in core would contradict ADR-001 §1's
|
||||
contents list (core is the contract artifact and nothing else).
|
||||
[ADR-013](013-fold-substrate-into-sqlite.md)'s "no fourth crate"
|
||||
rationale does not apply — that ruled the *substrate's* placement
|
||||
(fork lineage, versioning surface, link collisions); none of those
|
||||
facts attach to a greenfield in-memory implementation.
|
||||
|
||||
### 2. Maturity posture — honest-ephemeral, stated by the family's carriers
|
||||
|
||||
The mem engine's posture is the honest middle between "test
|
||||
convenience" (ADR-001 §4's original framing — *superseded*) and
|
||||
"durable production engine": **no durability, single-process,
|
||||
ephemeral by design.** Concretely:
|
||||
|
||||
- A **legitimate production choice for workloads that accept
|
||||
ephemerality** — the consumer's dependency on `alkstore-mem` *is*
|
||||
the ephemerality declaration (the ADR-016 pattern: the dependency
|
||||
edge states the truth and cannot drift from it).
|
||||
- **Never fleet-valid** — the fleet predicate ([ADR-010](010-queue-semantics-depth.md)
|
||||
§3's shared-pool rule) fails structurally: there is no pool, no
|
||||
process boundary, nothing shared. Two `MemStore` instances share
|
||||
nothing by design (§4).
|
||||
- Honesty rides the [ADR-016](016-deployment-honesty.md) carriers
|
||||
unchanged — no new surface: compile-time engine identity (the
|
||||
crate name), the engine crate's docs as the posture statement, and
|
||||
the [deployment matrix](../deployment.md) gaining the ephemeral
|
||||
row. The compile-time identity carries the honesty; nothing runtime
|
||||
is fabricated to detect what the dependency already declares.
|
||||
- alkblobs' mem line ("never a production story, never fleet-valid")
|
||||
is deliberately *not* copied verbatim — "never production" would
|
||||
contradict the sandbox use case that motivates this engine.
|
||||
|
||||
### 3. Full contract v1 — tier and asymmetry classification
|
||||
|
||||
The tier is **full contract v1**: every mechanism, every guarantee
|
||||
row, no reduced profile. A reduced test-only engine never gets
|
||||
born-pinned (the suite's equivalence rows would be absent from its
|
||||
column, exactly the drift the suite exists to prevent), and the
|
||||
motivating need is consumer-shaped, not test-shaped.
|
||||
|
||||
The documented engine asymmetries classify as:
|
||||
|
||||
- **`PayloadTooLarge`**: never produced, at any size — the SQLite
|
||||
occurrence posture. The variant stays contract-uniform and
|
||||
matchable ([ADR-016](016-deployment-honesty.md) §5); mem's suite
|
||||
column wires the non-occurrence arm.
|
||||
- **Reconnection**: no such concept — no connection can drop. The
|
||||
reconnect-wake and watcher-death rows are pg/SQLite-shaped and are
|
||||
*per-engine wired* in the suite, so mem's column simply does not
|
||||
carry them; nothing weakens across engines.
|
||||
- **Receiver close**: the one close path is engine drop (process
|
||||
exit/shutdown) closing open receivers — the contract's close-arm
|
||||
shapes apply, engine-realized as drop.
|
||||
- **Wake coalescing**: not applicable — wakes deliver at commit
|
||||
through in-process channels directly to each subscriber; there is
|
||||
no batching boundary to coalesce across. The engine spec pins the
|
||||
delivery shape.
|
||||
|
||||
### 4. Constructor shape — engine-native, isolated per open; shared instance rejected
|
||||
|
||||
Constructors live in engine crates ([ADR-008](008-contract-v1-pinning.md)
|
||||
§6), so the mem engine's constructor is engine-native: **`MemStore::new()`**
|
||||
— no connection string, no file path. This is the one documented
|
||||
exception to the core contract's open-prose ("connection string or
|
||||
file path"), and the prose is amended to name the exception
|
||||
(pre-implementation, class-1). **Isolation per open**: each
|
||||
`MemStore::new()` yields a distinct engine instance sharing nothing —
|
||||
exactly the suite factory's isolation contract; teardown is drop.
|
||||
|
||||
**A shared-instance constructor (two opens → one engine) is rejected
|
||||
for v1, with a re-entry gate**: it introduces real concurrency
|
||||
questions (claims and locks contended across two handles) that no
|
||||
consumer row names. The re-entry condition is a consumer-inventory
|
||||
row naming the two-opens-one-engine need — the ADR-016 §4 pattern
|
||||
(decision against, gate named); the open-questions register stays
|
||||
closed.
|
||||
|
||||
### 5. Wasm32 — compile-clean is an acceptance item; on-target suite execution is scoped out with a named collapse condition
|
||||
|
||||
The wasm motivation makes wasm compatibility load-bearing, so it
|
||||
moves from aspiration to **wave-6 acceptance item**:
|
||||
|
||||
1. `alkstore-mem` **compiles clean for `wasm32-unknown-unknown`**;
|
||||
2. the workspace host gates stay green (`cargo build` / `test` /
|
||||
clippy `-D warnings` / fmt);
|
||||
3. the contract-suite mem column passes on the host;
|
||||
4. the mem column runs under a **current-thread runtime flavor** —
|
||||
the test that keeps internals current-thread-clean, which is what
|
||||
wasm needs (no `spawn_blocking` anywhere, tokio's sync + time
|
||||
subset only). One suite-harness pre-task follows from this: the
|
||||
suite's `past_stamp_sleep` blocking sleep (the review-003 posture
|
||||
note, `properties.rs:1683`) becomes an async sleep — the only
|
||||
suite-side change the wave carries.
|
||||
|
||||
Running the *suite itself* on the wasm target (wasm-bindgen-test
|
||||
harnessing) is a different investment and is **scoped out of wave 6**
|
||||
with a named collapse condition: **a wasm-bindgen-test adapter crate
|
||||
for the suite existing**. This is a scope decision with a definite
|
||||
trigger, not a pending-client hedge — the acceptance the sandbox and
|
||||
adapter motivation actually needs (compile-clean, host-verified
|
||||
contract conformance) is delivered by items 1–4 and independently
|
||||
verifiable.
|
||||
|
||||
### 6. What the mem engine is not
|
||||
|
||||
- **Not a core dev-dependency.** ADR-001 §4's "doctests" posture
|
||||
declines: core takes no engine dev-dependency (the core crate has
|
||||
no driver dependencies, ever, dev-side included); core contract
|
||||
doc examples remain mock-store-based; mem's doctests live in the
|
||||
mem crate itself.
|
||||
- **Not a re-derivation of the suite** — ADR-022's factory is reused
|
||||
verbatim (its consequences already anticipated this).
|
||||
- **Not new contract surface.** The contract text changes here are
|
||||
the core-prose exception note (§4) and documentation rows (§2) —
|
||||
amend-in-place pre-release; the trait surface is untouched.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- The wasm/sandbox and downstream-adapter paths gain their engine;
|
||||
every future family consumer that must run in a sandbox can depend
|
||||
on `alkstore-mem` without inventing one.
|
||||
- A third full implementation stresses contract v1 while the class-1
|
||||
window is open — born pinned against the finished suite rather
|
||||
than pinned retroactively; any wobble is amend-in-place now.
|
||||
- The suite's arithmetic-equivalence pinning goes three-way (backoff
|
||||
curve, opts resolution — [ADR-012](012-forked-substrate-design.md)
|
||||
§2's one-owner rule applied per engine, third owner).
|
||||
- Zero new contract surface; the ADR-016 carriers absorb the posture
|
||||
(identity + docs + matrix row), no runtime descriptor.
|
||||
|
||||
**Negative**
|
||||
|
||||
- The ADR-017 lockstep duty now covers three engines: a class-2
|
||||
(additive) contract change needs three adoption releases; a third
|
||||
engine's tests/opt machinery ride every wave forever.
|
||||
- Engine-owned arithmetic gains a third owner (backoff curve, opts
|
||||
resolution, the `@every` parser) — three copies to keep honest, and
|
||||
the honesty mechanism is the suite's equivalence rows, not shared
|
||||
code.
|
||||
- The mem engine's maintenance is *real* engine maintenance (the
|
||||
queue/scheduler/stream/lock machinery), not a weekend adapter —
|
||||
the honest-scale flag this ADR's Context records.
|
||||
|
||||
## References
|
||||
|
||||
- [ADR-001](001-crate-split.md) §4 — the deferral this ADR collapses;
|
||||
§4's "implementation convenience, not a contract artifact" framing
|
||||
superseded (amend-in-place, class-1 window open).
|
||||
- [ADR-022](022-contract-suite-layout.md) — the shared `StoreFactory`
|
||||
suite reused verbatim; the born-pinned mechanism.
|
||||
- [ADR-017](017-contract-versioning.md) — §2's class-1 window (still
|
||||
open, what makes this amendment cheap); §4's pairing carriers the
|
||||
mem engine adopts (manifest pin, suite column, engine docs).
|
||||
- [ADR-016](016-deployment-honesty.md) — the carriers the
|
||||
honest-ephemeral posture rides (§2); the compile-time-identity
|
||||
honesty pattern (§3); §4's rejection-with-gate pattern §4 uses; §5
|
||||
the `PayloadTooLarge` pinning mem's column wires.
|
||||
- [ADR-010](010-queue-semantics-depth.md) §3 — the fleet predicate
|
||||
that decides "never fleet-valid" structurally.
|
||||
- [ADR-008](008-contract-v1-pinning.md) §6 — constructors live in
|
||||
engine crates (the `MemStore::new()` home).
|
||||
- [ADR-012](012-forked-substrate-design.md) §2 — the one-owner rule
|
||||
the engine-owned arithmetic applies per engine.
|
||||
- `docs/research/consumer-inventory.md` §Engine-tier inventory — the
|
||||
row-first record (operator-authority, 2026-10-10) that precedes
|
||||
this decision.
|
||||
- Review 003 (`docs/reviews/003-wave-5-general-review.md`) — the
|
||||
`properties.rs:1683` current-thread posture note the suite-harness
|
||||
pre-task disposes.
|
||||
- `docs/architecture/engine-mem.md` — the engine spec this ADR
|
||||
requires as a wave-6 deliverable.
|
||||
- `docs/plans/implementation.md` — the wave-6 insertion (release
|
||||
readiness renumbered to wave 7).
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-10 (docs alignment — v1 TLS posture stated; QueueOpts numeric consumer-obligation notes)
|
||||
last_updated: 2026-10-10 (ADR-024 — the mem engine joins the matrix:
|
||||
host row, connection budget, durability posture, wasm32 toolchain note)
|
||||
---
|
||||
|
||||
# Deployment
|
||||
@@ -20,6 +21,7 @@ document holds the facts.
|
||||
|---|---|---|
|
||||
| SQLite | **single-machine**, file-backed | NFS two-writers unsupported (honker's honesty posture, inherited, [ADR-003](decisions/003-sqlite-driver.md)). Cross-process *on one host* is verified POC ground (`data_version` is cross-process by nature). |
|
||||
| Postgres | **multi-host native** | Nothing assumes a shared host; POC #2 ran all-through-network (docker bridge) with the same properties ([ADR-004](decisions/004-postgres-driver.md)). |
|
||||
| Mem | **single-process, ephemeral** ([ADR-024](decisions/024-mem-engine.md) §2) | In-memory, per-instance; no cross-process story at all — two engine instances share nothing by design. No durability, by design: loss on process exit is the documented posture, and the dependency on `alkstore-mem` *is* the ephemerality declaration (honest for workloads that accept it). Never fleet-valid — the shared-pool predicate ([ADR-010](decisions/010-queue-semantics-depth.md) §3) fails structurally; there is no pool. Compile-clean on `wasm32-unknown-unknown` ([ADR-024](decisions/024-mem-engine.md) §5) — the family's member that runs in the sandbox. |
|
||||
|
||||
The unified trait must not pretend SQLite is multi-host — and it does
|
||||
not: [ADR-016](decisions/016-deployment-honesty.md) resolves that
|
||||
@@ -82,6 +84,14 @@ engine-owned PostgreSQL schema (default `alkstore`, per-engine option)
|
||||
— layout per [ADR-010](decisions/010-queue-semantics-depth.md) §8
|
||||
(resolved from [queues.md](queues.md)'s namespace bullet).
|
||||
|
||||
### Mem engine
|
||||
|
||||
- No connections — there is nothing to budget. The engine instance
|
||||
owns its state in-process; sharing nothing is the isolation design
|
||||
([ADR-024](decisions/024-mem-engine.md) §4's instance rule; the
|
||||
two-opens-one-engine constructor is rejected, re-entry via a
|
||||
consumer-inventory row).
|
||||
|
||||
### TLS posture (v1)
|
||||
|
||||
The pg engine hardwires `NoTls` on **every connection path** in v1 —
|
||||
@@ -143,6 +153,7 @@ runtime is fabricated.
|
||||
| SQLite | `synchronous` | WAL + `NORMAL` shipped ([ADR-003](decisions/003-sqlite-driver.md)); FULL is available consumer-side for stricter durability; commit fsyncs land at WAL checkpoints (the ~1000-commit spike cadence, POC #1) |
|
||||
| SQLite | `poll_interval` | the watcher's `data_version` poll cadence — 1 ms shipping default ([ADR-023](decisions/023-fourth-review-round.md) §4, the measured-wake-latency posture), carried on `SqliteOpts`; the idle cost is ~1000 poll reads/sec/instance; raising the interval trades wake latency (interval-bound) for idle CPU — the tuning recipe for latency-tolerant deployments |
|
||||
| Postgres | `synchronous_commit` | per-session knob; `on` is ship config (p50 2.40 ms seam); `off` trades max-tail (40.9 ms) for slightly better p50 — measured, honest trade ([ADR-004](decisions/004-postgres-driver.md)); session-level SET mechanics POC-verified |
|
||||
| Mem | — (none) | There is no durability knob and no durability: the honest row is the absence itself ([ADR-024](decisions/024-mem-engine.md) §2). Wake latency is sub-millisecond in-process; no tuning surface exists |
|
||||
|
||||
These are engine-configuration concerns, *not* trait surface. What
|
||||
part of engine config is contract-level shape vs engine-crate docs is
|
||||
@@ -158,6 +169,7 @@ the contract is the trait the constructor returns
|
||||
| `bundled-sqlite` adds a C build (~10 s dev, cacheable) | SQLite | build/CI time |
|
||||
| `libsqlite3-sys` collision with sqlx today | any mixed-driver binary | structurally avoided ([ADR-001](decisions/001-crate-split.md) single-driver rule) |
|
||||
| Listener `application_name` set for diagnosability (kill-targetable) | Postgres | ops runbooks ([ADR-004](decisions/004-postgres-driver.md)) |
|
||||
| `alkstore-mem` compiles clean on `wasm32-unknown-unknown` (wave-6 acceptance, [ADR-024](decisions/024-mem-engine.md) §5); tokio features limited to the wasm-supported subset (sync + time); current-thread-runtime-clean internals (the suite's mem column runs under that flavor) | Mem | wasm targets, sandboxes, downstream ffi/napi/python adapters — the only engine that can compile to wasm (rusqlite/tokio-postgres are structurally out) |
|
||||
|
||||
## Consumer-facing latency profile (indicative, POC-measured)
|
||||
|
||||
@@ -185,6 +197,7 @@ From both POCs (single-box, relative shapes are the deliverable —
|
||||
| [008](decisions/008-contract-v1-pinning.md) | Contract v1 | constructor/options in engine crates; no capability surface in v1 (OQ-08; resolved by [ADR-016](decisions/016-deployment-honesty.md)) |
|
||||
| [016](decisions/016-deployment-honesty.md) | Deployment honesty | no runtime capability surface — compile-time identity + this matrix; `PayloadTooLarge` is the one runtime asymmetry carriage |
|
||||
| [021](decisions/021-tx-reads-and-value-shape-fixes.md) | Third review round | drop = rollback keeps the connection budgets exact under error paths (SQLite lease releases; pg client re-pools) |
|
||||
| [024](decisions/024-mem-engine.md) | Mem engine | the matrix's third engine — honest-ephemeral: single-process, no durability, never fleet-valid, wasm32 compile-clean; no connection budget, no knobs |
|
||||
|
||||
## Open Questions
|
||||
|
||||
|
||||
@@ -0,0 +1,145 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-10 (initial spec — ADR-024 adopted the mem
|
||||
engine as wave 6; implementation pending, spec grows with the wave's
|
||||
decomposition)
|
||||
---
|
||||
|
||||
# Mem engine
|
||||
|
||||
The `alkstore-mem` engine implements
|
||||
[core-contract.md](core-contract.md) on guarded in-memory structures.
|
||||
This spec records WHAT the engine is — its architecture, the tx
|
||||
overlay, the asymmetry classification — not code-level HOW. Decisions
|
||||
live in ADRs ([ADR-024](decisions/024-mem-engine.md) is this engine's
|
||||
charter); contract obligations live in the core spec.
|
||||
|
||||
## Identity and posture
|
||||
|
||||
- Separate workspace crate (ADR-024 §1): `alkstore`, tokio (sync +
|
||||
time — the wasm-supported subset), no driver dependencies. Natively
|
||||
async; no `spawn_blocking` seam anywhere; internals are
|
||||
current-thread-runtime-clean by construction, and the suite's mem
|
||||
column runs *under a current-thread flavor* to keep that honest.
|
||||
- **Honest-ephemeral** (ADR-024 §2): no durability, single-process,
|
||||
ephemeral by design; legitimate production for workloads that
|
||||
accept ephemerality; never fleet-valid (the
|
||||
[ADR-010](decisions/010-queue-semantics-depth.md) §3 shared-pool
|
||||
predicate fails structurally — there is no pool). The dependency
|
||||
edge on `alkstore-mem` is the ephemerality declaration; see
|
||||
[deployment.md](deployment.md).
|
||||
- **Full contract v1** (ADR-024 §3): every mechanism, every guarantee
|
||||
row. Born pinned — the engine lands against the finished
|
||||
version-stamped suite ([ADR-022](decisions/022-contract-suite-layout.md)'s
|
||||
`StoreFactory`, reused verbatim as a third factory column), which
|
||||
is also what pins its engine-owned arithmetic three-way
|
||||
([ADR-012](decisions/012-forked-substrate-design.md) §2's one-owner
|
||||
rule: the backoff curve, opts resolution, and the `@every` parser
|
||||
get mem's own implementations; equivalence with the other two
|
||||
engines is suite-pinned).
|
||||
- wasm32: `alkstore-mem` compiling clean for
|
||||
`wasm32-unknown-unknown` is an acceptance item of its wave
|
||||
(ADR-024 §5) and an identity line of this spec.
|
||||
|
||||
## Architecture
|
||||
|
||||
Everything the other engines realize with connections, schemas, and
|
||||
daemons, the mem engine realizes with process-owned state:
|
||||
|
||||
- **No connections at all** — no pool (pg's), no writer/reader slots
|
||||
(SQLite's), no listener. The store instance *is* the state: guarded
|
||||
per-mechanism tables (queues + jobs, dead letters, streams +
|
||||
offsets, locks, schedules) owned by the engine instance.
|
||||
- **No background machinery to bootstrap** — no schema bootstrap, no
|
||||
watcher thread (nothing cross-connection to poll), no forwarder.
|
||||
The only async machinery is tokio-time-driven: lock TTLs, the
|
||||
scheduler's boundary waits, and wake delivery.
|
||||
- **Wake substrate**: in-process channels. Wakes deliver at the
|
||||
commit boundary directly to each open receiver — no batching
|
||||
boundary exists, so coalescing is not applicable; per-subscriber
|
||||
delivery order is FIFO by commit order.
|
||||
|
||||
### The tx overlay
|
||||
|
||||
The tx seam's mem realization: a `begin_tx()` allocates an **overlay**
|
||||
private to the caller's handle.
|
||||
|
||||
- `*_tx` methods stage effects in the overlay (enqueues, publishes,
|
||||
notifies); reads *inside* the tx merge the overlay over committed
|
||||
state — read-your-own-writes with no round-trips.
|
||||
- **Commit** merges the overlay atomically (a single guarded merge)
|
||||
and fires the overlay's pending wakes — *wake-at-commit* is the
|
||||
discipline-critical property: a notify staged in the tx delivers
|
||||
nothing until the overlay merges, exactly the seam the pg engine
|
||||
had to fix retroactively (`pg-fix-tx-wake`) and the suite's
|
||||
tx-commit-atomicity rows pin.
|
||||
- **Drop** discards the overlay — drop = rollback with no ghosts,
|
||||
trivially, because uncommitted state never touched committed state.
|
||||
- No isolation questions can arise across handles: an overlay is
|
||||
invisible to every other store/view until its commit.
|
||||
|
||||
## Mapping the contract
|
||||
|
||||
| Contract piece | Engine realization |
|
||||
|---|---|
|
||||
| notify / listen | staged notify in the tx overlay; delivered at commit to open receivers. Commit-atomicity is structural ([ADR-007](decisions/007-transactional-seam.md)) — the merge and the wake fire are one operation |
|
||||
| streams | in-memory event log + per-consumer offsets, global-FIFO by monotone offset; keyed publish rides the overlay ([ADR-015](decisions/015-streams-depth.md)); `trim_to` mutates the log and wakes nothing |
|
||||
| queues | the ADR-010 state machine over an in-memory job table (visibility, reclaim, the equal-jitter curve engine-owned, dead-letter move, the no-stranded-rows sweep) — all driver-free, swept on caller-driven `sweep_expired` |
|
||||
| named locks | in-memory TTL registry ([ADR-008](decisions/008-contract-v1-pinning.md) §7's row); expiry via tokio time; single-instance contention semantics |
|
||||
| scheduler / outbox | collapse shape ([ADR-009](decisions/009-scheduler-collapse.md)): schedule table, tick (boundary advance + 64-boundary catch-up cap), leadership on the `__alkstore_scheduler` lock — single-instance leadership is trivially owned but the lock machinery still runs, per ADR-009 §4's engine-uniform rule; outbox = helper over queues ([ADR-014](decisions/014-outbox-tx-enqueue.md)) |
|
||||
| begin_tx | overlay allocation ([ADR-007](decisions/007-transactional-seam.md)) |
|
||||
| handle ops | direct `.await`s on guarded state; `encode_payload` via core (ADR-020 §4) |
|
||||
|
||||
## Asymmetry classification (ADR-024 §3)
|
||||
|
||||
| Asymmetry | Mem posture |
|
||||
|---|---|
|
||||
| `PayloadTooLarge` | never produced, any size (SQLite occurrence arm); the non-occurrence arm wires in mem's suite column ([ADR-016](decisions/016-deployment-honesty.md) §5) |
|
||||
| Reconnection | does not exist — no connection can drop; the reconnect-wake and watcher-death rows are pg/SQLite-shaped and absent from mem's column (rows are per-engine wired; nothing cross-wires or weakens) |
|
||||
| Receiver close | one close path: engine drop closes open receivers (process exit/shutdown); contract close-arm shapes apply unchanged ([ADR-021](decisions/021-tx-reads-and-value-shape-fixes.md)) |
|
||||
| Wake coalescing | not applicable — direct at-commit delivery, per-subscriber FIFO; the engine spec line above pins it |
|
||||
|
||||
## Isolation and the suite factory
|
||||
|
||||
Each `MemStore::new()` yields a distinct engine instance sharing
|
||||
nothing (ADR-024 §4's isolation rule — the contract's *Store* concept
|
||||
amended to admit the engine-native constructor). The suite factory:
|
||||
a fresh store per run, teardown = drop; the mem column carries the
|
||||
mechanism rows that express on it, omits the engine-shaped rows that
|
||||
cannot (the table above), and runs under the current-thread flavor
|
||||
(ADR-024 §5 item 4 — the suite harness's `past_stamp_sleep` async
|
||||
sleep is the one pre-task the wave carries). Running the suite
|
||||
*on* the wasm target is scoped out behind a named collapse condition
|
||||
(a wasm-bindgen-test adapter crate), per ADR-024 §5.
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|---|---|---|
|
||||
| [001](decisions/001-crate-split.md) | Crate split | §4's mem clause collapsed + amended by [ADR-024](decisions/024-mem-engine.md) — separate crate, full contract v1 |
|
||||
| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract | opaque wake + re-read; in-process channels as the wake substrate; no reconnection concept (no connection can drop) |
|
||||
| [007](decisions/007-transactional-seam.md) | Tx seam | the tx overlay — staging, commit-merge, drop-discard; wake-at-commit structural |
|
||||
| [008](decisions/008-contract-v1-pinning.md) | Contract v1 | full-surface implementation; §6 config split (constructor in the engine crate); §7 locks row as a TTL registry |
|
||||
| [009](decisions/009-scheduler-collapse.md) | Scheduler collapse | schedule table + tick + catch-up + leadership lock (trivially owned, still run — engine-uniform) |
|
||||
| [010](decisions/010-queue-semantics-depth.md) | Queue depth | the state machine driver-free; curve engine-owned; §3's fleet predicate decides never-fleet-valid |
|
||||
| [012](decisions/012-forked-substrate-design.md) | Fork design | §2 one-owner rule, third owner: backoff curve, opts resolution, `@every` parser — three-way equivalence suite-pinned |
|
||||
| [014](decisions/014-outbox-tx-enqueue.md) | Outbox tx enqueue | `outbox_enqueue_tx` staged in the overlay |
|
||||
| [015](decisions/015-streams-depth.md) | Streams depth | in-memory log + per-consumer offsets; keyed publish; `trim_to` (wakes nothing) |
|
||||
| [016](decisions/016-deployment-honesty.md) | Deployment honesty | no runtime surface — honest-ephemeral by compile-time identity + docs + matrix row; `PayloadTooLarge` never produced |
|
||||
| [017](decisions/017-contract-versioning.md) | Contract versioning | manifest pins core per §4; suite column is the pairing instrument |
|
||||
| [019](decisions/019-mechanism-handle-surfaces.md) | Handle surfaces | boxed handle traits implemented over guarded state |
|
||||
| [020](decisions/020-enqueue-opt-semantics-and-bridges.md) | Enqueue options | opts resolution engine-owned (third copy); payload encoding via core |
|
||||
| [021](decisions/021-tx-reads-and-value-shape-fixes.md) | Third review round | drop = rollback trivial by overlay discard; receiver close at engine drop |
|
||||
| [022](decisions/022-contract-suite-layout.md) | Suite layout | factory reused verbatim — third column, born pinned |
|
||||
| [024](decisions/024-mem-engine.md) | Mem engine charter | crate placement, honest-ephemeral posture, full tier, constructor + shared-instance rejection, wasm acceptance items |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in
|
||||
[open-questions.md](open-questions.md). This spec opens none — the
|
||||
register is closed, and every decision this engine needed is in
|
||||
[ADR-024](decisions/024-mem-engine.md). The two deliberately-not-OQ
|
||||
dispositions live there: the shared-instance constructor rejection
|
||||
(§4, re-entry = a consumer-inventory row) and the on-target wasm
|
||||
suite execution scope-out (§5, collapse condition = an adapter
|
||||
crate).
|
||||
@@ -1,16 +1,20 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-07 (ADR-022 — contract-suite layout: shared internal suite crate)
|
||||
last_updated: 2026-10-10 (ADR-024 — the mem engine joins the crate
|
||||
family as `alkstore-mem`; family/ADR tables updated)
|
||||
---
|
||||
|
||||
# alkstore — Overview
|
||||
|
||||
One reactive store interface over SQLite and Postgres: durable
|
||||
One reactive store interface over SQLite, Postgres, and the
|
||||
in-process mem engine: durable
|
||||
notify/subscribe signals, streams with per-consumer offsets, durable
|
||||
queues with the transactional enqueue property, named locks, a
|
||||
scheduler, and an outbox helper — with each engine using its own
|
||||
native machinery underneath (SQLite: honker's watcher design;
|
||||
Postgres: `pg_notify`/LISTEN and the pg-boss schema family).
|
||||
Postgres: `pg_notify`/LISTEN and the pg-boss schema family; mem:
|
||||
guarded in-process state, ephemeral by design —
|
||||
[ADR-024](decisions/024-mem-engine.md)).
|
||||
|
||||
## Why this crate exists
|
||||
|
||||
@@ -31,7 +35,7 @@ Per [ADR-001](decisions/001-crate-split.md):
|
||||
| `alkstore` (core) | trait surface, types, error model | none (no capability surface — [ADR-016](decisions/016-deployment-honesty.md): engine differences are compile-time identity + the deployment matrix, never a runtime descriptor) |
|
||||
| `alkstore-sqlite` | SQLite engine ([ADR-003](decisions/003-sqlite-driver.md)) | rusqlite; the forked honker-core lineage rides in-tree as the engine crate's substrate module subtree ([ADR-011](decisions/011-sqlite-substrate-fork.md), designed in [ADR-012](decisions/012-forked-substrate-design.md), folded per [ADR-013](decisions/013-fold-substrate-into-sqlite.md)) |
|
||||
| `alkstore-postgres` | Postgres engine ([ADR-004](decisions/004-postgres-driver.md)) | tokio-postgres, deadpool-postgres |
|
||||
| (mem engine, optional) | test convenience, decided at implementation ([ADR-001](decisions/001-crate-split.md)) | none |
|
||||
| `alkstore-mem` | mem engine — full contract v1, honest-ephemeral (no durability, single-process, never fleet-valid; [ADR-024](decisions/024-mem-engine.md)) | none (tokio sync + time only; the wasm32-clean member of the family) |
|
||||
|
||||
Downstream consumers depend on core + exactly one engine. Family
|
||||
standards apply throughout: tokio async runtime, `thiserror` errors, no
|
||||
@@ -55,6 +59,7 @@ Per [ADR-002](decisions/002-feature-scope.md):
|
||||
| [core-contract.md](core-contract.md) | The unified trait surface: mechanisms, delivery guarantees, tx seam, naming |
|
||||
| [engine-sqlite.md](engine-sqlite.md) | SQLite engine: mapping the contract onto the forked substrate module/rusqlite |
|
||||
| [engine-postgres.md](engine-postgres.md) | Postgres engine: mapping the contract onto tokio-postgres/LISTEN |
|
||||
| [engine-mem.md](engine-mem.md) | Mem engine: mapping the contract onto guarded in-process state (ADR-024) |
|
||||
| [queues.md](queues.md) | Queue/scheduler/outbox semantics depth (ADR-009/ADR-010 resolved) |
|
||||
| [deployment.md](deployment.md) | Host capabilities, connection budgets, deployment matrix (OQ-08 resolved) |
|
||||
| [open-questions.md](open-questions.md) | OQ-01..NN tracker |
|
||||
@@ -86,6 +91,7 @@ Per [ADR-002](decisions/002-feature-scope.md):
|
||||
| [020](decisions/020-enqueue-opt-semantics-and-bridges.md) | Enqueue-option semantics (delay/run_at, expires, scheduler stamp source, payload encoding) | Accepted |
|
||||
| [021](decisions/021-tx-reads-and-value-shape-fixes.md) | Third review round (tx-read methods on `TxHandle`, `Job.claimed_at`, `schedule()` queue-argument validation, drop = rollback, receiver close/error arms) | Accepted |
|
||||
| [022](decisions/022-contract-suite-layout.md) | Contract-suite layout (shared internal suite crate, store-factory parameterization) | Accepted |
|
||||
| [024](decisions/024-mem-engine.md) | The mem engine (`alkstore-mem` — full contract v1, honest-ephemeral, wasm32 compile-clean acceptance) | Accepted |
|
||||
|
||||
## Non-goals
|
||||
|
||||
|
||||
+109
-15
@@ -1,6 +1,8 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-10 (wave 5 implemented and reviewed — review-wave-5 verified the discharge map, stamps, dispositions and green-on-both-engines; engine specs flipped to stable)
|
||||
last_updated: 2026-10-10 (ADR-024 — the mem engine adopted as wave 6
|
||||
and decomposed-in-place documentation; release readiness renumbered
|
||||
to wave 7. Waves 1–5 implemented and reviewed; engine specs stable.)
|
||||
---
|
||||
|
||||
# alkstore — Implementation plan
|
||||
@@ -23,12 +25,14 @@ boundaries.
|
||||
## The waves
|
||||
|
||||
Dependency logic in one line: core → (substrate fork ∥ postgres
|
||||
engine) → sqlite engine → contract suite → release readiness. The
|
||||
substrate fork is contract-blind (ADR-012 §2), so it needs only the
|
||||
workspace scaffold from wave 1; the Postgres engine needs only the core
|
||||
crate. Waves 2 and 4 are therefore independent of each other and could
|
||||
run in either order (or in parallel, if agents are ever available in
|
||||
parallel).
|
||||
engine) → sqlite engine → contract suite → mem engine → release
|
||||
readiness. The substrate fork is contract-blind (ADR-012 §2), so it
|
||||
needs only the workspace scaffold from wave 1; the Postgres engine
|
||||
needs only the core crate. Waves 2 and 4 are therefore independent of
|
||||
each other and could run in either order (or in parallel, if agents
|
||||
are ever available in parallel). The mem engine (wave 6) needs the
|
||||
core crate plus the finished suite (wave 5) — born pinned, per
|
||||
[ADR-024](../architecture/decisions/024-mem-engine.md).
|
||||
|
||||
| Wave | Contents | Depends on | Status |
|
||||
|---|---|---|---|
|
||||
@@ -37,7 +41,8 @@ parallel).
|
||||
| [3](#wave-3--sqlite-engine) | SQLite engine: connection architecture, re-derived queue ops on contract v1, scheduler/outbox, tx seam, SQLite backlog column | waves 1 + 2 | implemented + reviewed (2026-10-08) |
|
||||
| 4 | [Postgres engine](#wave-4--postgres-engine): schema bootstrap, pool/open, listener/forwarder, all mechanisms, tx seam, pg backlog column | wave 1 | implemented + reviewed (2026-10-08) |
|
||||
| 5 | [Contract suite](#wave-5--contract-suite): the cross-engine equivalence properties (core-contract.md §Verification backlog), version-stamped per ADR-017 | waves 3 + 4 | implemented + reviewed (2026-10-10) |
|
||||
| 6 | Release readiness: crate docs, deployment matrix final pass, README (written last, honestly), publish prep; mem-engine and fuzzing decisions | wave 5 | not yet decomposed |
|
||||
| 6 | [Mem engine](#wave-6--mem-engine): separate `alkstore-mem` crate, full contract v1 over in-memory mechanisms (tx overlay, queue/scheduler/streams/locks/wakes), the suite's third factory column (born pinned), wasm32 compile-clean acceptance | waves 1 + 5 | not yet decomposed |
|
||||
| 7 | [Release readiness](#wave-7--release-readiness): crate docs, deployment matrix final pass, README (written last, honestly), publish prep; fuzzing decision | wave 6 | not yet decomposed |
|
||||
|
||||
## Wave 1 — Foundations
|
||||
|
||||
@@ -99,7 +104,7 @@ shaped this wave's task set:
|
||||
wiring (the constructor task).
|
||||
- **Staying put as ordered**: M-1's retention-failure test and N-5's
|
||||
panic probe → wave 5's contract-suite rows; N-1's growth-posture
|
||||
review → wave 6.
|
||||
review → wave 7.
|
||||
|
||||
## Wave 4 — Postgres engine
|
||||
|
||||
@@ -244,6 +249,79 @@ expected answer, recorded per case); whether the scheduler rows
|
||||
demand the pg fire-wake parity the wave-4 fix-batch gate recorded;
|
||||
and what the SQLite lock busy-path actually returns under contention.
|
||||
|
||||
## Wave 6 — Mem engine
|
||||
|
||||
The third engine ([ADR-024](../architecture/decisions/024-mem-engine.md),
|
||||
2026-10-10): a separate `alkstore-mem` crate implementing the full
|
||||
contract v1 over guarded in-memory mechanisms — the honest-ephemeral
|
||||
tier (no durability, single-process, ephemeral by design, never
|
||||
fleet-valid), adopted for the wasm/sandbox and downstream-adapter
|
||||
economics the consumer inventory's engine-tier row records
|
||||
(operator-authority, 2026-10-10 — the row-first gate satisfied before
|
||||
this wave was assumed). Release readiness moved to wave 7.
|
||||
|
||||
The wave's shape mirrors the engine waves' rhythm: constructor + tx
|
||||
overlay → mechanisms in parallel → scheduler/outbox → suite
|
||||
integration → review gate. What mem carries: the queue state machine
|
||||
(visibility/reclaim/dead-letter/sweep), scheduler boundaries +
|
||||
catch-up + leadership (trivially owned single-instance, still run —
|
||||
engine-uniform per ADR-009 §4), streams with per-consumer offsets and
|
||||
trim, TTL locks, wakes. What it does not carry: no pooling, no schema
|
||||
bootstrap, no forwarder, no `spawn_blocking` seam, no reconnection
|
||||
concept — the distributed machinery that made wave 4 build-heavy
|
||||
doesn't exist here. The wave is estimated at pg-wave scale or
|
||||
somewhat smaller.
|
||||
|
||||
The load-bearing artifact is the **tx overlay**: `*_tx` effects stage
|
||||
in a handle-private overlay; commit merges it atomically and fires
|
||||
the overlay's pending wakes; drop discards it — read-your-own-writes
|
||||
easy, drop-=-rollback trivial, and *wake-at-commit* structural. That
|
||||
last property is the seam review 002 found broken pg-side
|
||||
(`pg-fix-tx-wake`); mem gets it pinned from birth by the suite's
|
||||
existing tx-commit-atomicity rows.
|
||||
|
||||
Born pinned: [ADR-022](decisions/022-contract-suite-layout.md)'s
|
||||
`StoreFactory` is reused verbatim as a third factory column wired
|
||||
onto already-authored, version-stamped rows — no re-derivation. The
|
||||
engine re-owns its own copies of the backoff curve, opts resolution,
|
||||
and the `@every` parser ([ADR-012](decisions/012-forked-substrate-design.md)
|
||||
§2's one-owner rule, third owner); the suite now pins that arithmetic
|
||||
three-way.
|
||||
|
||||
**Acceptance items** ([ADR-024](../architecture/decisions/024-mem-engine.md)
|
||||
§5): the crate compiles clean for `wasm32-unknown-unknown` (target
|
||||
already installed in the dev environment); workspace host gates stay
|
||||
green; the suite's mem column passes on the host, run under a
|
||||
**current-thread runtime flavor** — the test that keeps internals
|
||||
wasm-clean (no `spawn_blocking`, tokio sync + time only). The
|
||||
current-thread flavor rides one harness pre-task: the suite's
|
||||
`past_stamp_sleep` blocking sleep (review 003's posture note at
|
||||
`properties.rs:1683`) becomes an async sleep — the only suite-side
|
||||
change the wave carries. Running the suite itself *on* the wasm
|
||||
target is scoped out with a named collapse condition (a
|
||||
wasm-bindgen-test adapter crate) — a scope decision recorded in
|
||||
ADR-024 §5, not a pending-client hedge.
|
||||
|
||||
Deliverables beyond code: `engine-mem.md` (the spec — drafted with
|
||||
this wave's adoption), the deployment matrix's mem rows (landed with
|
||||
ADR-024), and the mem factory's suite wiring.
|
||||
|
||||
Riding the window: review 003's Finding 1 (the pg scheduler
|
||||
transient-error retry loop, MEDIUM — engine-side, small,
|
||||
mem-independent); Findings 2–3 follow their park recommendation into
|
||||
wave 7's pre-release pass.
|
||||
|
||||
## Wave 7 — Release readiness
|
||||
|
||||
The original wave 6, renumbered when the mem engine inserted at wave
|
||||
6 ([ADR-024](../architecture/decisions/024-mem-engine.md)): crate
|
||||
docs, the deployment matrix final pass (across all three engines now),
|
||||
the crate README (written last, honestly), publish prep, N-1's
|
||||
growth-posture review — and the **fuzzing decision** (the
|
||||
alksocks/alktty/alktunnels pattern), the one remaining
|
||||
decided-at-implementation deferral. Decomposes after wave 6's review
|
||||
gate, with review 003's Findings 2–3 riding the pre-release pass.
|
||||
|
||||
## Decided points
|
||||
|
||||
- **Contract-suite layout — option (a)**: a small internal
|
||||
@@ -262,11 +340,21 @@ and what the SQLite lock busy-path actually returns under contention.
|
||||
(self-hosted Gitea; supply-chain posture). No CI-wiring task exists;
|
||||
the merge gates (`cargo test`, `cargo clippy --all-targets --
|
||||
-D warnings`, `cargo fmt --check`) are run coordinator-side.
|
||||
- **Mem engine** (ADR-001 §4) and **fuzzing adoption** (the
|
||||
alksocks/alktty/alktunnels pattern): both are "decided at
|
||||
implementation" deferrals, recorded here so they surface as explicit
|
||||
decision points in wave 6 (or earlier if the test story demands the
|
||||
mem engine sooner) rather than ambushing a later session.
|
||||
- **Mem engine** (ADR-001 §4): **decided 2026-10-10, pre-decomposition
|
||||
([ADR-024](../architecture/decisions/024-mem-engine.md))** — a
|
||||
separate `alkstore-mem` crate, full contract v1, honest-ephemeral
|
||||
posture, wasm32 compile-clean acceptance; wave 6, with release
|
||||
readiness renumbered to wave 7. The deferral discharged: its
|
||||
collapse condition ("implementation time") was reached early via
|
||||
the consumer pause + the wasm/sandbox motivation, with the row-first
|
||||
gate satisfied by the consumer inventory's engine-tier row
|
||||
(operator-authority, 2026-10-10).
|
||||
- **Fuzzing adoption** (the alksocks/alktty/alktunnels pattern):
|
||||
remains the one decided-at-implementation deferral, recorded here
|
||||
so it surfaces as an explicit decision point in wave 7 (per
|
||||
AGENTS.md's no-fuzzing posture, revisit only if a task or ADR
|
||||
introduces it). Renumbered from wave 6 together with the mem
|
||||
decision's move.
|
||||
|
||||
## Review gates
|
||||
|
||||
@@ -285,6 +373,12 @@ decomposes. Specific gates:
|
||||
- **Wave 5 review** — the suite as compatibility instrument: every
|
||||
backlog row present, version-stamped, green on both engines; this is
|
||||
the gate that flips the engine specs to `stable`.
|
||||
- **Wave 6 review** — mem-engine-vs-contract conformance (the
|
||||
born-pinned gate): the suite's third column green on the host, the
|
||||
wasm32 compile gate clean, the current-thread-flavor column run
|
||||
(harness pre-task disposed), the engine's own mechanism tests, and
|
||||
the `engine-mem.md` spec read against the pinned ADR-024 text —
|
||||
the gate that flips the mem spec toward `stable`.
|
||||
|
||||
## Review rounds so far
|
||||
|
||||
@@ -296,7 +390,7 @@ decomposes. Specific gates:
|
||||
`docs/reviews/001-waves-1-2-general-review.md`) — M-1 fixed
|
||||
inline (sweep savepoint scope); M-2/N-2/N-4/N-6 resolved as ADR-023
|
||||
pre-decomposition; lint removal + suite adds folded into waves 3/5;
|
||||
N-1 recorded for wave 6.
|
||||
N-1 recorded for release readiness (now wave 7).
|
||||
- **Wave 3 review gate** (`review-wave-3`) — engine-vs-contract
|
||||
conformance code-read clean; two findings fixed inline (the
|
||||
`open_writer_connection` boundary move out of `substrate/mod.rs`;
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-05 (streams corrected to documented/in-scope —
|
||||
operator-authority record: type-filtered event watching from multiple
|
||||
places, e.g. repo-change subscriptions in a git app. alkcall row added
|
||||
2026-10-05 as ecosystem-shape evidence for streams (no per-key-ordering
|
||||
need named anywhere); rate-limits + result-storage remain the
|
||||
last_updated: 2026-10-10 (mem-engine engine-tier row added —
|
||||
operator-authority record: wasm32 sandboxing + downstream
|
||||
ffi/napi/python-adapter economics; the record the ADR-024 scope
|
||||
decision cites per the row-first gate. Earlier: 2026-10-05 streams
|
||||
correction and alkcall row; rate-limits + result-storage remain the
|
||||
keep-with-flag rows.)
|
||||
---
|
||||
|
||||
@@ -203,6 +203,31 @@ cut-at-implementation.** Adjacent-tooling shape (a queue job's outcome
|
||||
queryable by id), useful to queues+scheduler consumers but named by
|
||||
none of them yet.
|
||||
|
||||
## Engine-tier inventory
|
||||
|
||||
This section answers the engine-tier question (which backing engines
|
||||
exist) under this document's discipline. The two shipped engines
|
||||
(SQLite, Postgres) were Phase 0 decisions with consumer evidence
|
||||
throughout; the mem engine entered as ADR-001 §4's implementation-time
|
||||
deferral, so its need gets a row here *before* any scope decision
|
||||
assumed it — the row-first gate, engine-shaped. (Rows above grade
|
||||
*features*; this row grades *which engine tier a consumer needs*.)
|
||||
|
||||
| Consumer | Evidence | Need | Confidence |
|
||||
|---|---|---|---|
|
||||
| wasm-sandboxed consumers (family-wide) | operator-authority record, 2026-10-10 | an engine that runs in-process on `wasm32-unknown-unknown`: the alk protocol crates (alkcall, alktty, alktunnels, alksocks) are wasm-compatible at the protocol level by design and sandboxing is the family's deployment direction; of the engine triad, only the in-memory engine can ride along (rusqlite and tokio-postgres are structurally out on wasm) | **operator-authority** (the REQ-2 recording convention; expected where the wanter is an *application above* the paused crates) |
|
||||
| downstream adapter crates (ffi/napi/python) | operator-authority record, 2026-10-10 | a zero-configuration engine for adapter crates and first-running examples: wasm compatibility downstream makes ffi/napi/python adapters far cheaper to build (no per-platform registration/build matrix the operator cannot run — no mac/windows hosts); the engine an adapter always needs is one that just works in-process | **operator-authority** (same record) |
|
||||
|
||||
Verdict: **in scope as a third engine — full contract v1**
|
||||
([ADR-024](../architecture/decisions/024-mem-engine.md), 2026-10-10):
|
||||
separate `alkstore-mem` crate, no durability, single-process,
|
||||
ephemeral by design, never fleet-valid. The tier is *full contract
|
||||
v1*, not a reduced test-only profile: a reduced engine never gets
|
||||
born-pinned by the contract suite, and the wasm/adapter motivation is
|
||||
consumer-shaped, not test-shaped. *(Upgrade path to `documented`:
|
||||
whichever consumer document first names the mem engine's need grows
|
||||
the row, per this document's conventions.)*
|
||||
|
||||
## What this inventory changes
|
||||
|
||||
- **OQ-ST-01** stops being "blocked on a consumer-driven inventory
|
||||
|
||||
@@ -9,7 +9,8 @@
|
||||
> exists on the stream reads, and domains must be engine-uniform),
|
||||
> N-4 (URI flag dropped, a registered fork delta), N-6 (1 ms default
|
||||
> stands, cadence carried onto `SqliteOpts`). The task-carried items
|
||||
> (lint removal, wave-5 suite adds, wave-6 N-1 review) stand as
|
||||
> (lint removal, wave-5 suite adds, release-readiness N-1 review —
|
||||
> wave 7 since ADR-024's renumbering) stand as
|
||||
> ordered in §7.
|
||||
|
||||
- **Reviewer**: opencode (glm-5.3-flash)
|
||||
|
||||
@@ -246,8 +246,9 @@ covers it, but say so.
|
||||
defect — but a one-sentence doc note (consumer-obligation, like the
|
||||
visibility-budgeting one) would prevent surprise.
|
||||
- `eprintln!` diagnostics at the swallowed-wake/lag sites match the
|
||||
SQLite twin's posture; the logging story is a wave-6 item (already
|
||||
recorded by the wave-4 gate).
|
||||
SQLite twin's posture; the logging story is a wave-7 item (a wave-6
|
||||
item when recorded — renumbered by ADR-024; already recorded by the
|
||||
wave-4 gate).
|
||||
- F-2..F-4 from `tasks/review-wave-4.md` stand as recorded there.
|
||||
|
||||
## Recommended sequencing
|
||||
|
||||
@@ -232,14 +232,20 @@ explicitly here so the call is recorded rather than implicit.
|
||||
structurally blocked by the own-schema re-drain argument; the lock
|
||||
row's lapse assertions can only be delayed, never un-lapsed, by slow
|
||||
clocks. Supporting the gate's watch-not-block disposition; the
|
||||
wave-6 watch flag stands.
|
||||
watch flag stands (the release-readiness watch — wave 7 since
|
||||
ADR-024's renumbering).
|
||||
|
||||
## Recommendation
|
||||
|
||||
No changes required to keep wave 5's verdict standing. For wave 6's
|
||||
window: Finding 1 is a decomposable small task (transient-fault seam +
|
||||
window *(wave 6 at writing — release readiness; renumbered by
|
||||
ADR-024, with Finding 1 since routed to the new wave 6 — the mem
|
||||
engine — window, and Findings 2–3 to wave 7's pre-release pass, per
|
||||
`docs/plans/implementation.md`'s wave-6 section)*: Finding 1 is a
|
||||
decomposable small task (transient-fault seam +
|
||||
retry/exhaustion pin); Findings 2–3 can ride the same task or be
|
||||
accepted with the code-read as their pin — either way the disposition
|
||||
should be recorded. Findings 4–5 need no work. The env-variable
|
||||
boundary section above folds into wave 6's release-readiness pass as a
|
||||
boundary section above folds into the release-readiness pass
|
||||
(wave 7) as a
|
||||
verified-now / re-check-at-prepublish item.
|
||||
@@ -68,7 +68,7 @@ A `schema.rs` module in `alkstore-postgres`:
|
||||
- **No migration machinery in v1** — tables are created fresh per
|
||||
schema; append-column migrations are the substrate's evolution
|
||||
discipline, not a greenfield v1 need (note the posture in the module
|
||||
docs; wave 6's release pass revisits if the deployment story
|
||||
docs; wave 7's release pass revisits if the deployment story
|
||||
demands in-place upgrades).
|
||||
|
||||
Tests use a raw `tokio_postgres::Connection`/Client against the
|
||||
|
||||
@@ -189,7 +189,7 @@ inline-clean, or already owned by a later wave):
|
||||
connection *still in use* across store-close keeps running on a
|
||||
closed file until its op completes — harmless in practice, the op
|
||||
finishes on an open connection that is then dropped). Noted for
|
||||
wave 6's close-path review; no consumer-visible defect.
|
||||
release readiness's close-path review (now wave 7); no consumer-visible defect.
|
||||
- **Coverage of the commit-error arm**: no test induces a failing
|
||||
`COMMIT` (needs SQLITE_FULL/IOERR-class injection; the code fix
|
||||
is pinned by code-read + the shared `reopen` machinery being
|
||||
|
||||
@@ -224,7 +224,7 @@ acceptance criteria — not test-name-trust):
|
||||
file holds; `#[non_exhaustive]` policy respected (none on `PgOpts`);
|
||||
core has no driver deps; `eprintln!` diagnostics at the five
|
||||
swallowed-wake/lag sites match the SQLite twin's posture (the
|
||||
logging story is a wave-6 item, not a wave-4 defect).
|
||||
logging story is a wave-7 item (release readiness), not a wave-4 defect).
|
||||
- **Gates**: `cargo build`, `cargo test` server-less (25 core + 188
|
||||
sqlite + 111 pg-skip + 10 + 10 suite + 3 + 9 harness), clippy
|
||||
`-D warnings`, fmt — all green; the pg lib suite 111/111 green
|
||||
@@ -317,7 +317,7 @@ wave-5/6 decision; none is a consumer-visible defect):
|
||||
but `core-contract.md`'s streams/notify sections don't mention the
|
||||
channel naming (they don't need to — it's engine-internal), while
|
||||
the reserved reconnect string *is* a contract string. No action —
|
||||
recorded here so wave 6's docs pass notices where the naming live.
|
||||
recorded here so wave 7's docs pass notices where the naming live.
|
||||
|
||||
Wave 5 decomposition may proceed.
|
||||
|
||||
|
||||
@@ -67,7 +67,9 @@ Primary lenses:
|
||||
recorded
|
||||
- [x] Engine specs flipped to `stable` with dated annotations;
|
||||
implementation.md updated (wave table + review rounds)
|
||||
- [x] Findings recorded; wave 6 decomposition may proceed
|
||||
- [x] Findings recorded; the next decomposition may proceed
|
||||
(release readiness at gate time — since renumbered to wave 7
|
||||
when the mem engine inserted at wave 6, ADR-024)
|
||||
|
||||
## References
|
||||
|
||||
@@ -214,7 +216,8 @@ repro loop) *before* any postulate-and-fix — per the user's ledger
|
||||
the pattern (a prior unreproducible flake hinting at a real engine
|
||||
issue) deserves that attention. Not blocking this gate: 13+ subsequent
|
||||
clean pg runs across both rows since the failures, all conditions
|
||||
covered. **Flagged for watch in wave 6's window.**
|
||||
covered. **Flagged for watch in the next wave window** (release
|
||||
readiness at gate time — since renumbered to wave 7, ADR-024).
|
||||
|
||||
**7. Findings.** No code, suite-row, or stamp defects requiring change
|
||||
were found at gate time; all changes this gate made are doc-side
|
||||
|
||||
@@ -28,7 +28,7 @@ Workspace-level: shared `[workspace.package]` fields (edition, license
|
||||
MIT OR Apache-2.0, repository), `resolver = "2"` (or 3 per edition),
|
||||
and a root `Cargo.toml` with `members`. Crate versions are `0.1.0`
|
||||
placeholders — ADR-017 §1 pins the initial *release* at 1.0.0; that
|
||||
bump is wave 6's release task, not this one.
|
||||
bump is wave 7's release task, not this one.
|
||||
|
||||
The engines' manifests must pin the core dependency as a path
|
||||
dependency with the version field set (the ADR-017 §4.1 manifest-pin
|
||||
@@ -73,7 +73,7 @@ outside tests, no comments in code (doc comments on public API fine).
|
||||
`https://git.alk.dev/alkdev/alkstore`, `Cargo.lock` committed,
|
||||
`.gitignore` of `target/` + `.worktrees/`. LICENSE-MIT/LICENSE-APACHE
|
||||
files at repo root are **not** added here — they belong with the
|
||||
release-readiness/publish-prep work (wave 6), matching the repo's
|
||||
release-readiness/publish-prep work (wave 7), matching the repo's
|
||||
no-honest-artifact-yet posture; the manifests already declare the
|
||||
license field per the task spec.
|
||||
- Engine manifests pin the core path dependency with the version field
|
||||
|
||||
Reference in new issue
Block a user