diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 2c32268..7278e0f 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -1,23 +1,29 @@ --- status: draft -last_updated: 2026-10-08 (ADR-023: fourth review round resolved — waves-1–2 general-review findings) +last_updated: 2026-10-10 (ADR-024: the mem engine adopted as a third +engine — engine-mem.md added, ADR table row, index updated for the +implementation phase) --- # alkstore — Architecture Architecture documentation for the alkstore project: one reactive store interface (notify, streams, queues, locks, scheduler, outbox) -over SQLite and Postgres, with each engine native underneath -(see [overview.md](overview.md)). +over SQLite, Postgres, and the in-process mem engine, with each +engine native underneath (see [overview.md](overview.md)). ## Current State -**Phase 1 (Architecture) — in progress.** Phase 0 is complete -(`docs/research/phase-0.md`): both POCs ran and passed, the scope -inventory is confirmed, and the crate split, drivers, and ownership -postures are decided. This directory carries the architecture spec -build-out over that evidence base; all spec documents are `draft` -pending architecture review and OQ resolution. +**Phase 1 (Implementation) — waves 1–5 implemented and reviewed.** +Phase 0 is complete (`docs/research/phase-0.md`): both POCs ran and +passed, the scope inventory is confirmed, and the crate split, +drivers, and ownership postures are decided. All open questions are +resolved (through ADR-023), and the engine specs flipped `stable` at +the wave-5 review gate. Waves 1–5 (core, substrate fork, SQLite +engine, Postgres engine, contract suite) are implemented and reviewed +per `docs/plans/implementation.md`. The mem engine was adopted as a +third engine by [ADR-024](decisions/024-mem-engine.md) (2026-10-10, +wave 6 — release readiness renumbered to wave 7). ## Architecture Documents @@ -25,8 +31,9 @@ pending architecture review and OQ resolution. |---|---|---|---| | [overview.md](overview.md) | draft | Crate family, feature surface, non-goals, evidence base | — | | [core-contract.md](core-contract.md) | draft | The unified trait surface, delivery guarantees, tx seam | OQ-10 (resolved) | -| [engine-sqlite.md](engine-sqlite.md) | draft | SQLite engine: forked-substrate/rusqlite mapping | OQ-06 (resolved), OQ-12 (resolved), OQ-13 (resolved) | -| [engine-postgres.md](engine-postgres.md) | draft | Postgres engine: tokio-postgres/LISTEN mapping | OQ-08 (resolved), OQ-12 (resolved), OQ-13 (resolved) | +| [engine-sqlite.md](engine-sqlite.md) | stable | SQLite engine: forked-substrate/rusqlite mapping | OQ-06 (resolved), OQ-12 (resolved), OQ-13 (resolved) | +| [engine-postgres.md](engine-postgres.md) | stable | Postgres engine: tokio-postgres/LISTEN mapping | OQ-08 (resolved), OQ-12 (resolved), OQ-13 (resolved) | +| [engine-mem.md](engine-mem.md) | draft | Mem engine: in-process ephemeral mapping of the contract ([ADR-024](decisions/024-mem-engine.md)) | — | | [queues.md](queues.md) | draft | Queue/scheduler/outbox semantics depth (resolved: ADR-009/ADR-010) | OQ-05 (resolved), OQ-09 (resolved), OQ-06 (resolved) | | [deployment.md](deployment.md) | draft | Host semantics, connection budgets, knobs, matrix | OQ-08 (resolved) | | [open-questions.md](open-questions.md) | draft | OQ tracker (promoted from OQ-ST register) | — | @@ -58,6 +65,7 @@ pending architecture review and OQ resolution. | [021](decisions/021-tx-reads-and-value-shape-fixes.md) | Third review round — tx-read methods on `TxHandle`, `Job.claimed_at`, `schedule()` queue-argument validation, drop = rollback, receiver close/error arms | Accepted | | [022](decisions/022-contract-suite-layout.md) | Contract-suite layout — shared internal suite crate, properties parameterized over a store factory (discharges ADR-017 §4.2's deferral) | Accepted | | [023](decisions/023-fourth-review-round.md) | Fourth review round — `encode_payload` typed (`Codec`), numeric-argument domains by kind (extents clamp empty, durations reject, boundaries total), plain-path SQLite open (URI flag dropped), watcher cadence 1 ms default + `SqliteOpts` knob | Accepted | +| [024](decisions/024-mem-engine.md) | The mem engine — a third engine (`alkstore-mem`), full contract v1, honest-ephemeral posture (collapses + amends ADR-001 §4's deferral) | Accepted | ## Open Questions diff --git a/docs/architecture/core-contract.md b/docs/architecture/core-contract.md index 9c5ca70..0b2165e 100644 --- a/docs/architecture/core-contract.md +++ b/docs/architecture/core-contract.md @@ -1,6 +1,8 @@ --- status: draft -last_updated: 2026-10-08 (ADR-023 — fourth review round: numeric-argument domains pinned by kind, encode_payload typed, plain-path SQLite open, watcher cadence posture) +last_updated: 2026-10-10 (ADR-024 — the mem engine joins the family; +the Store concept prose admits the engine-native constructor +exception) --- # Core contract @@ -37,7 +39,11 @@ descriptor; the - **Store** — what a consumer opens from a connection string or file path ([ADR-001](decisions/001-crate-split.md)): one handle, the engine behind it chosen at open time. Consumer code never branches on - engine type (guiding principle 4). + engine type (guiding principle 4). (The mem engine is the one + documented exception to the open-prose: its constructor is + engine-native — `MemStore::new()`, no connection string — per + [ADR-024](decisions/024-mem-engine.md) §4; the engine choice + remains a dependency-graph fact made before any `Store` exists.) - **Mechanism** — one of the surface's coordination families: notify, streams, queues (+outbox), locks, scheduler. Delivery guarantees are pinned per mechanism in [ADR-006](decisions/006-wake-and-delivery-contract.md)'s diff --git a/docs/architecture/decisions/001-crate-split.md b/docs/architecture/decisions/001-crate-split.md index 842ce0f..aa1ec92 100644 --- a/docs/architecture/decisions/001-crate-split.md +++ b/docs/architecture/decisions/001-crate-split.md @@ -3,7 +3,10 @@ ## Status Accepted *(capability-flags content annotated 2026-10-06 by -[ADR-016](016-deployment-honesty.md))* +[ADR-016](016-deployment-honesty.md); §4 superseded in part +2026-10-10 by [ADR-024](024-mem-engine.md) — the mem engine exists as +the separate `alkstore-mem` crate, a full contract-v1 implementation, +not an implementation convenience)* ## Context @@ -56,6 +59,12 @@ The project ships as a **family of crates**: a third implementation for tests and doctests, if the test story turns out to want it. Treated as an implementation convenience, not a contract artifact — decided at implementation time, not now. + *(Superseded in part 2026-10-10 by + [ADR-024](024-mem-engine.md): the mem engine exists as the + separate `alkstore-mem` crate — full contract v1, no durability, + single-process, ephemeral by design, never fleet-valid, wasm32 + compile-clean by acceptance. Not a test convenience, and not a + core dev-dependency: core's doc examples stay mock-based.)* Downstream consumers depend on `alkstore` plus exactly one engine crate. A consumer binary links at most one driver. diff --git a/docs/architecture/decisions/024-mem-engine.md b/docs/architecture/decisions/024-mem-engine.md new file mode 100644 index 0000000..d85cb94 --- /dev/null +++ b/docs/architecture/decisions/024-mem-engine.md @@ -0,0 +1,238 @@ +# ADR-024: The mem engine — a third engine (`alkstore-mem`), full contract v1, honest-ephemeral + +## Status + +Accepted (2026-10-10, Phase 1 — [ADR-001](001-crate-split.md) §4's +implementation-time deferral collapses here, and this ADR *amends* that +§4's framing rather than merely discharging it: the mem engine is a +full contract-v1 implementation, not the "implementation convenience, +not a contract artifact" §4 originally imagined. The class-1 +amend-in-place window ([ADR-017](017-contract-versioning.md) §2) is +still open — no versioned artifact exists — so that amendment is +pre-implementation. The row-first gate is satisfied: the +consumer-inventory engine-tier row (operator-authority, 2026-10-10) +precedes this decision.) + +## Context + +ADR-001 §4 allowed "a mem-shaped engine (in-tree or separate test +crate) … if the test story turns out to want it," deferred to +implementation time. The plan recorded the deferral as a wave-6 +decision point ("or earlier if the test story demands the mem engine +sooner"). The wave-5 review gate closed with the contract suite green +on both engines, and two things now make the mem engine concrete: + +- **The consumer pause + the wasm motivation.** The paused consumer + work that surfaced the need is not a test story but a consumer + story: the alk protocol crates (alkcall, alktty, alktunnels, + alksocks) are wasm-compatible at the protocol level by design, and + sandboxing — browser or otherwise — plus downstream + ffi/napi/python-adapter economics are the operator's recorded need + (the consumer-inventory engine-tier row, 2026-10-10). On those + paths the mem engine is not a hedge: rusqlite and tokio-postgres + are structurally out on wasm, so it is the *only* engine the + sandbox can carry. +- **The contract suite changed the economics.** A third engine used + to mean re-deriving a verification story; ADR-022's shared + `StoreFactory` suite means a third engine is born pinned — wiring a + third factory column onto already-authored, version-stamped rows. + And the class-1 amendment window (ADR-017 §2) is still open: any + contract wobble a third full implementation exposes is + amend-in-place now, a migration after first release. + +The scale is honest to state up front: this is a *third full engine*, +not alkblobs' mem tier (a `BTreeMap` behind a handful of CAS +methods). It implements the queue state machine (visibility/reclaim/ +dead-letter/sweep), the scheduler (boundaries, catch-up, leadership), +the tx seam, streams with per-consumer offsets and trim, TTL locks, +and wakes. What it does *not* carry is the machinery that made the +Postgres wave build-heavy: no pooling, no schema bootstrap, no +forwarder, no `spawn_blocking` seam, no reconnection concept. + +## Decision + +### 1. Crate placement — a separate `alkstore-mem` crate + +The mem engine is a **separate workspace member**, `alkstore-mem`, +implementing the core traits. Dependency surface: `alkstore` plus +tokio (the wasm-supported subset — sync and time), no driver +dependencies; payload encoding rides core's `encode_payload` +([ADR-020](020-enqueue-opt-semantics-and-bridges.md) §4). This +discharges ADR-001 §4's "in-tree or separate crate" fork as +**separate**, for the reason that ADR stated in reverse for the other +engines: engine crates are additive by construction (ADR-001 § +Consequences), and a mem engine in core would contradict ADR-001 §1's +contents list (core is the contract artifact and nothing else). +[ADR-013](013-fold-substrate-into-sqlite.md)'s "no fourth crate" +rationale does not apply — that ruled the *substrate's* placement +(fork lineage, versioning surface, link collisions); none of those +facts attach to a greenfield in-memory implementation. + +### 2. Maturity posture — honest-ephemeral, stated by the family's carriers + +The mem engine's posture is the honest middle between "test +convenience" (ADR-001 §4's original framing — *superseded*) and +"durable production engine": **no durability, single-process, +ephemeral by design.** Concretely: + +- A **legitimate production choice for workloads that accept + ephemerality** — the consumer's dependency on `alkstore-mem` *is* + the ephemerality declaration (the ADR-016 pattern: the dependency + edge states the truth and cannot drift from it). +- **Never fleet-valid** — the fleet predicate ([ADR-010](010-queue-semantics-depth.md) + §3's shared-pool rule) fails structurally: there is no pool, no + process boundary, nothing shared. Two `MemStore` instances share + nothing by design (§4). +- Honesty rides the [ADR-016](016-deployment-honesty.md) carriers + unchanged — no new surface: compile-time engine identity (the + crate name), the engine crate's docs as the posture statement, and + the [deployment matrix](../deployment.md) gaining the ephemeral + row. The compile-time identity carries the honesty; nothing runtime + is fabricated to detect what the dependency already declares. +- alkblobs' mem line ("never a production story, never fleet-valid") + is deliberately *not* copied verbatim — "never production" would + contradict the sandbox use case that motivates this engine. + +### 3. Full contract v1 — tier and asymmetry classification + +The tier is **full contract v1**: every mechanism, every guarantee +row, no reduced profile. A reduced test-only engine never gets +born-pinned (the suite's equivalence rows would be absent from its +column, exactly the drift the suite exists to prevent), and the +motivating need is consumer-shaped, not test-shaped. + +The documented engine asymmetries classify as: + +- **`PayloadTooLarge`**: never produced, at any size — the SQLite + occurrence posture. The variant stays contract-uniform and + matchable ([ADR-016](016-deployment-honesty.md) §5); mem's suite + column wires the non-occurrence arm. +- **Reconnection**: no such concept — no connection can drop. The + reconnect-wake and watcher-death rows are pg/SQLite-shaped and are + *per-engine wired* in the suite, so mem's column simply does not + carry them; nothing weakens across engines. +- **Receiver close**: the one close path is engine drop (process + exit/shutdown) closing open receivers — the contract's close-arm + shapes apply, engine-realized as drop. +- **Wake coalescing**: not applicable — wakes deliver at commit + through in-process channels directly to each subscriber; there is + no batching boundary to coalesce across. The engine spec pins the + delivery shape. + +### 4. Constructor shape — engine-native, isolated per open; shared instance rejected + +Constructors live in engine crates ([ADR-008](008-contract-v1-pinning.md) +§6), so the mem engine's constructor is engine-native: **`MemStore::new()`** +— no connection string, no file path. This is the one documented +exception to the core contract's open-prose ("connection string or +file path"), and the prose is amended to name the exception +(pre-implementation, class-1). **Isolation per open**: each +`MemStore::new()` yields a distinct engine instance sharing nothing — +exactly the suite factory's isolation contract; teardown is drop. + +**A shared-instance constructor (two opens → one engine) is rejected +for v1, with a re-entry gate**: it introduces real concurrency +questions (claims and locks contended across two handles) that no +consumer row names. The re-entry condition is a consumer-inventory +row naming the two-opens-one-engine need — the ADR-016 §4 pattern +(decision against, gate named); the open-questions register stays +closed. + +### 5. Wasm32 — compile-clean is an acceptance item; on-target suite execution is scoped out with a named collapse condition + +The wasm motivation makes wasm compatibility load-bearing, so it +moves from aspiration to **wave-6 acceptance item**: + +1. `alkstore-mem` **compiles clean for `wasm32-unknown-unknown`**; +2. the workspace host gates stay green (`cargo build` / `test` / + clippy `-D warnings` / fmt); +3. the contract-suite mem column passes on the host; +4. the mem column runs under a **current-thread runtime flavor** — + the test that keeps internals current-thread-clean, which is what + wasm needs (no `spawn_blocking` anywhere, tokio's sync + time + subset only). One suite-harness pre-task follows from this: the + suite's `past_stamp_sleep` blocking sleep (the review-003 posture + note, `properties.rs:1683`) becomes an async sleep — the only + suite-side change the wave carries. + +Running the *suite itself* on the wasm target (wasm-bindgen-test +harnessing) is a different investment and is **scoped out of wave 6** +with a named collapse condition: **a wasm-bindgen-test adapter crate +for the suite existing**. This is a scope decision with a definite +trigger, not a pending-client hedge — the acceptance the sandbox and +adapter motivation actually needs (compile-clean, host-verified +contract conformance) is delivered by items 1–4 and independently +verifiable. + +### 6. What the mem engine is not + +- **Not a core dev-dependency.** ADR-001 §4's "doctests" posture + declines: core takes no engine dev-dependency (the core crate has + no driver dependencies, ever, dev-side included); core contract + doc examples remain mock-store-based; mem's doctests live in the + mem crate itself. +- **Not a re-derivation of the suite** — ADR-022's factory is reused + verbatim (its consequences already anticipated this). +- **Not new contract surface.** The contract text changes here are + the core-prose exception note (§4) and documentation rows (§2) — + amend-in-place pre-release; the trait surface is untouched. + +## Consequences + +**Positive** + +- The wasm/sandbox and downstream-adapter paths gain their engine; + every future family consumer that must run in a sandbox can depend + on `alkstore-mem` without inventing one. +- A third full implementation stresses contract v1 while the class-1 + window is open — born pinned against the finished suite rather + than pinned retroactively; any wobble is amend-in-place now. +- The suite's arithmetic-equivalence pinning goes three-way (backoff + curve, opts resolution — [ADR-012](012-forked-substrate-design.md) + §2's one-owner rule applied per engine, third owner). +- Zero new contract surface; the ADR-016 carriers absorb the posture + (identity + docs + matrix row), no runtime descriptor. + +**Negative** + +- The ADR-017 lockstep duty now covers three engines: a class-2 + (additive) contract change needs three adoption releases; a third + engine's tests/opt machinery ride every wave forever. +- Engine-owned arithmetic gains a third owner (backoff curve, opts + resolution, the `@every` parser) — three copies to keep honest, and + the honesty mechanism is the suite's equivalence rows, not shared + code. +- The mem engine's maintenance is *real* engine maintenance (the + queue/scheduler/stream/lock machinery), not a weekend adapter — + the honest-scale flag this ADR's Context records. + +## References + +- [ADR-001](001-crate-split.md) §4 — the deferral this ADR collapses; + §4's "implementation convenience, not a contract artifact" framing + superseded (amend-in-place, class-1 window open). +- [ADR-022](022-contract-suite-layout.md) — the shared `StoreFactory` + suite reused verbatim; the born-pinned mechanism. +- [ADR-017](017-contract-versioning.md) — §2's class-1 window (still + open, what makes this amendment cheap); §4's pairing carriers the + mem engine adopts (manifest pin, suite column, engine docs). +- [ADR-016](016-deployment-honesty.md) — the carriers the + honest-ephemeral posture rides (§2); the compile-time-identity + honesty pattern (§3); §4's rejection-with-gate pattern §4 uses; §5 + the `PayloadTooLarge` pinning mem's column wires. +- [ADR-010](010-queue-semantics-depth.md) §3 — the fleet predicate + that decides "never fleet-valid" structurally. +- [ADR-008](008-contract-v1-pinning.md) §6 — constructors live in + engine crates (the `MemStore::new()` home). +- [ADR-012](012-forked-substrate-design.md) §2 — the one-owner rule + the engine-owned arithmetic applies per engine. +- `docs/research/consumer-inventory.md` §Engine-tier inventory — the + row-first record (operator-authority, 2026-10-10) that precedes + this decision. +- Review 003 (`docs/reviews/003-wave-5-general-review.md`) — the + `properties.rs:1683` current-thread posture note the suite-harness + pre-task disposes. +- `docs/architecture/engine-mem.md` — the engine spec this ADR + requires as a wave-6 deliverable. +- `docs/plans/implementation.md` — the wave-6 insertion (release + readiness renumbered to wave 7). \ No newline at end of file diff --git a/docs/architecture/deployment.md b/docs/architecture/deployment.md index bc851d9..bc7a931 100644 --- a/docs/architecture/deployment.md +++ b/docs/architecture/deployment.md @@ -1,6 +1,7 @@ --- status: draft -last_updated: 2026-10-10 (docs alignment — v1 TLS posture stated; QueueOpts numeric consumer-obligation notes) +last_updated: 2026-10-10 (ADR-024 — the mem engine joins the matrix: +host row, connection budget, durability posture, wasm32 toolchain note) --- # Deployment @@ -20,6 +21,7 @@ document holds the facts. |---|---|---| | SQLite | **single-machine**, file-backed | NFS two-writers unsupported (honker's honesty posture, inherited, [ADR-003](decisions/003-sqlite-driver.md)). Cross-process *on one host* is verified POC ground (`data_version` is cross-process by nature). | | Postgres | **multi-host native** | Nothing assumes a shared host; POC #2 ran all-through-network (docker bridge) with the same properties ([ADR-004](decisions/004-postgres-driver.md)). | +| Mem | **single-process, ephemeral** ([ADR-024](decisions/024-mem-engine.md) §2) | In-memory, per-instance; no cross-process story at all — two engine instances share nothing by design. No durability, by design: loss on process exit is the documented posture, and the dependency on `alkstore-mem` *is* the ephemerality declaration (honest for workloads that accept it). Never fleet-valid — the shared-pool predicate ([ADR-010](decisions/010-queue-semantics-depth.md) §3) fails structurally; there is no pool. Compile-clean on `wasm32-unknown-unknown` ([ADR-024](decisions/024-mem-engine.md) §5) — the family's member that runs in the sandbox. | The unified trait must not pretend SQLite is multi-host — and it does not: [ADR-016](decisions/016-deployment-honesty.md) resolves that @@ -82,6 +84,14 @@ engine-owned PostgreSQL schema (default `alkstore`, per-engine option) — layout per [ADR-010](decisions/010-queue-semantics-depth.md) §8 (resolved from [queues.md](queues.md)'s namespace bullet). +### Mem engine + +- No connections — there is nothing to budget. The engine instance + owns its state in-process; sharing nothing is the isolation design + ([ADR-024](decisions/024-mem-engine.md) §4's instance rule; the + two-opens-one-engine constructor is rejected, re-entry via a + consumer-inventory row). + ### TLS posture (v1) The pg engine hardwires `NoTls` on **every connection path** in v1 — @@ -143,6 +153,7 @@ runtime is fabricated. | SQLite | `synchronous` | WAL + `NORMAL` shipped ([ADR-003](decisions/003-sqlite-driver.md)); FULL is available consumer-side for stricter durability; commit fsyncs land at WAL checkpoints (the ~1000-commit spike cadence, POC #1) | | SQLite | `poll_interval` | the watcher's `data_version` poll cadence — 1 ms shipping default ([ADR-023](decisions/023-fourth-review-round.md) §4, the measured-wake-latency posture), carried on `SqliteOpts`; the idle cost is ~1000 poll reads/sec/instance; raising the interval trades wake latency (interval-bound) for idle CPU — the tuning recipe for latency-tolerant deployments | | Postgres | `synchronous_commit` | per-session knob; `on` is ship config (p50 2.40 ms seam); `off` trades max-tail (40.9 ms) for slightly better p50 — measured, honest trade ([ADR-004](decisions/004-postgres-driver.md)); session-level SET mechanics POC-verified | +| Mem | — (none) | There is no durability knob and no durability: the honest row is the absence itself ([ADR-024](decisions/024-mem-engine.md) §2). Wake latency is sub-millisecond in-process; no tuning surface exists | These are engine-configuration concerns, *not* trait surface. What part of engine config is contract-level shape vs engine-crate docs is @@ -158,6 +169,7 @@ the contract is the trait the constructor returns | `bundled-sqlite` adds a C build (~10 s dev, cacheable) | SQLite | build/CI time | | `libsqlite3-sys` collision with sqlx today | any mixed-driver binary | structurally avoided ([ADR-001](decisions/001-crate-split.md) single-driver rule) | | Listener `application_name` set for diagnosability (kill-targetable) | Postgres | ops runbooks ([ADR-004](decisions/004-postgres-driver.md)) | +| `alkstore-mem` compiles clean on `wasm32-unknown-unknown` (wave-6 acceptance, [ADR-024](decisions/024-mem-engine.md) §5); tokio features limited to the wasm-supported subset (sync + time); current-thread-runtime-clean internals (the suite's mem column runs under that flavor) | Mem | wasm targets, sandboxes, downstream ffi/napi/python adapters — the only engine that can compile to wasm (rusqlite/tokio-postgres are structurally out) | ## Consumer-facing latency profile (indicative, POC-measured) @@ -185,6 +197,7 @@ From both POCs (single-box, relative shapes are the deliverable — | [008](decisions/008-contract-v1-pinning.md) | Contract v1 | constructor/options in engine crates; no capability surface in v1 (OQ-08; resolved by [ADR-016](decisions/016-deployment-honesty.md)) | | [016](decisions/016-deployment-honesty.md) | Deployment honesty | no runtime capability surface — compile-time identity + this matrix; `PayloadTooLarge` is the one runtime asymmetry carriage | | [021](decisions/021-tx-reads-and-value-shape-fixes.md) | Third review round | drop = rollback keeps the connection budgets exact under error paths (SQLite lease releases; pg client re-pools) | +| [024](decisions/024-mem-engine.md) | Mem engine | the matrix's third engine — honest-ephemeral: single-process, no durability, never fleet-valid, wasm32 compile-clean; no connection budget, no knobs | ## Open Questions diff --git a/docs/architecture/engine-mem.md b/docs/architecture/engine-mem.md new file mode 100644 index 0000000..d0193f8 --- /dev/null +++ b/docs/architecture/engine-mem.md @@ -0,0 +1,145 @@ +--- +status: draft +last_updated: 2026-10-10 (initial spec — ADR-024 adopted the mem +engine as wave 6; implementation pending, spec grows with the wave's +decomposition) +--- + +# Mem engine + +The `alkstore-mem` engine implements +[core-contract.md](core-contract.md) on guarded in-memory structures. +This spec records WHAT the engine is — its architecture, the tx +overlay, the asymmetry classification — not code-level HOW. Decisions +live in ADRs ([ADR-024](decisions/024-mem-engine.md) is this engine's +charter); contract obligations live in the core spec. + +## Identity and posture + +- Separate workspace crate (ADR-024 §1): `alkstore`, tokio (sync + + time — the wasm-supported subset), no driver dependencies. Natively + async; no `spawn_blocking` seam anywhere; internals are + current-thread-runtime-clean by construction, and the suite's mem + column runs *under a current-thread flavor* to keep that honest. +- **Honest-ephemeral** (ADR-024 §2): no durability, single-process, + ephemeral by design; legitimate production for workloads that + accept ephemerality; never fleet-valid (the + [ADR-010](decisions/010-queue-semantics-depth.md) §3 shared-pool + predicate fails structurally — there is no pool). The dependency + edge on `alkstore-mem` is the ephemerality declaration; see + [deployment.md](deployment.md). +- **Full contract v1** (ADR-024 §3): every mechanism, every guarantee + row. Born pinned — the engine lands against the finished + version-stamped suite ([ADR-022](decisions/022-contract-suite-layout.md)'s + `StoreFactory`, reused verbatim as a third factory column), which + is also what pins its engine-owned arithmetic three-way + ([ADR-012](decisions/012-forked-substrate-design.md) §2's one-owner + rule: the backoff curve, opts resolution, and the `@every` parser + get mem's own implementations; equivalence with the other two + engines is suite-pinned). +- wasm32: `alkstore-mem` compiling clean for + `wasm32-unknown-unknown` is an acceptance item of its wave + (ADR-024 §5) and an identity line of this spec. + +## Architecture + +Everything the other engines realize with connections, schemas, and +daemons, the mem engine realizes with process-owned state: + +- **No connections at all** — no pool (pg's), no writer/reader slots + (SQLite's), no listener. The store instance *is* the state: guarded + per-mechanism tables (queues + jobs, dead letters, streams + + offsets, locks, schedules) owned by the engine instance. +- **No background machinery to bootstrap** — no schema bootstrap, no + watcher thread (nothing cross-connection to poll), no forwarder. + The only async machinery is tokio-time-driven: lock TTLs, the + scheduler's boundary waits, and wake delivery. +- **Wake substrate**: in-process channels. Wakes deliver at the + commit boundary directly to each open receiver — no batching + boundary exists, so coalescing is not applicable; per-subscriber + delivery order is FIFO by commit order. + +### The tx overlay + +The tx seam's mem realization: a `begin_tx()` allocates an **overlay** +private to the caller's handle. + +- `*_tx` methods stage effects in the overlay (enqueues, publishes, + notifies); reads *inside* the tx merge the overlay over committed + state — read-your-own-writes with no round-trips. +- **Commit** merges the overlay atomically (a single guarded merge) + and fires the overlay's pending wakes — *wake-at-commit* is the + discipline-critical property: a notify staged in the tx delivers + nothing until the overlay merges, exactly the seam the pg engine + had to fix retroactively (`pg-fix-tx-wake`) and the suite's + tx-commit-atomicity rows pin. +- **Drop** discards the overlay — drop = rollback with no ghosts, + trivially, because uncommitted state never touched committed state. +- No isolation questions can arise across handles: an overlay is + invisible to every other store/view until its commit. + +## Mapping the contract + +| Contract piece | Engine realization | +|---|---| +| notify / listen | staged notify in the tx overlay; delivered at commit to open receivers. Commit-atomicity is structural ([ADR-007](decisions/007-transactional-seam.md)) — the merge and the wake fire are one operation | +| streams | in-memory event log + per-consumer offsets, global-FIFO by monotone offset; keyed publish rides the overlay ([ADR-015](decisions/015-streams-depth.md)); `trim_to` mutates the log and wakes nothing | +| queues | the ADR-010 state machine over an in-memory job table (visibility, reclaim, the equal-jitter curve engine-owned, dead-letter move, the no-stranded-rows sweep) — all driver-free, swept on caller-driven `sweep_expired` | +| named locks | in-memory TTL registry ([ADR-008](decisions/008-contract-v1-pinning.md) §7's row); expiry via tokio time; single-instance contention semantics | +| scheduler / outbox | collapse shape ([ADR-009](decisions/009-scheduler-collapse.md)): schedule table, tick (boundary advance + 64-boundary catch-up cap), leadership on the `__alkstore_scheduler` lock — single-instance leadership is trivially owned but the lock machinery still runs, per ADR-009 §4's engine-uniform rule; outbox = helper over queues ([ADR-014](decisions/014-outbox-tx-enqueue.md)) | +| begin_tx | overlay allocation ([ADR-007](decisions/007-transactional-seam.md)) | +| handle ops | direct `.await`s on guarded state; `encode_payload` via core (ADR-020 §4) | + +## Asymmetry classification (ADR-024 §3) + +| Asymmetry | Mem posture | +|---|---| +| `PayloadTooLarge` | never produced, any size (SQLite occurrence arm); the non-occurrence arm wires in mem's suite column ([ADR-016](decisions/016-deployment-honesty.md) §5) | +| Reconnection | does not exist — no connection can drop; the reconnect-wake and watcher-death rows are pg/SQLite-shaped and absent from mem's column (rows are per-engine wired; nothing cross-wires or weakens) | +| Receiver close | one close path: engine drop closes open receivers (process exit/shutdown); contract close-arm shapes apply unchanged ([ADR-021](decisions/021-tx-reads-and-value-shape-fixes.md)) | +| Wake coalescing | not applicable — direct at-commit delivery, per-subscriber FIFO; the engine spec line above pins it | + +## Isolation and the suite factory + +Each `MemStore::new()` yields a distinct engine instance sharing +nothing (ADR-024 §4's isolation rule — the contract's *Store* concept +amended to admit the engine-native constructor). The suite factory: +a fresh store per run, teardown = drop; the mem column carries the +mechanism rows that express on it, omits the engine-shaped rows that +cannot (the table above), and runs under the current-thread flavor +(ADR-024 §5 item 4 — the suite harness's `past_stamp_sleep` async +sleep is the one pre-task the wave carries). Running the suite +*on* the wasm target is scoped out behind a named collapse condition +(a wasm-bindgen-test adapter crate), per ADR-024 §5. + +## Design Decisions + +| ADR | Decision | Summary | +|---|---|---| +| [001](decisions/001-crate-split.md) | Crate split | §4's mem clause collapsed + amended by [ADR-024](decisions/024-mem-engine.md) — separate crate, full contract v1 | +| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract | opaque wake + re-read; in-process channels as the wake substrate; no reconnection concept (no connection can drop) | +| [007](decisions/007-transactional-seam.md) | Tx seam | the tx overlay — staging, commit-merge, drop-discard; wake-at-commit structural | +| [008](decisions/008-contract-v1-pinning.md) | Contract v1 | full-surface implementation; §6 config split (constructor in the engine crate); §7 locks row as a TTL registry | +| [009](decisions/009-scheduler-collapse.md) | Scheduler collapse | schedule table + tick + catch-up + leadership lock (trivially owned, still run — engine-uniform) | +| [010](decisions/010-queue-semantics-depth.md) | Queue depth | the state machine driver-free; curve engine-owned; §3's fleet predicate decides never-fleet-valid | +| [012](decisions/012-forked-substrate-design.md) | Fork design | §2 one-owner rule, third owner: backoff curve, opts resolution, `@every` parser — three-way equivalence suite-pinned | +| [014](decisions/014-outbox-tx-enqueue.md) | Outbox tx enqueue | `outbox_enqueue_tx` staged in the overlay | +| [015](decisions/015-streams-depth.md) | Streams depth | in-memory log + per-consumer offsets; keyed publish; `trim_to` (wakes nothing) | +| [016](decisions/016-deployment-honesty.md) | Deployment honesty | no runtime surface — honest-ephemeral by compile-time identity + docs + matrix row; `PayloadTooLarge` never produced | +| [017](decisions/017-contract-versioning.md) | Contract versioning | manifest pins core per §4; suite column is the pairing instrument | +| [019](decisions/019-mechanism-handle-surfaces.md) | Handle surfaces | boxed handle traits implemented over guarded state | +| [020](decisions/020-enqueue-opt-semantics-and-bridges.md) | Enqueue options | opts resolution engine-owned (third copy); payload encoding via core | +| [021](decisions/021-tx-reads-and-value-shape-fixes.md) | Third review round | drop = rollback trivial by overlay discard; receiver close at engine drop | +| [022](decisions/022-contract-suite-layout.md) | Suite layout | factory reused verbatim — third column, born pinned | +| [024](decisions/024-mem-engine.md) | Mem engine charter | crate placement, honest-ephemeral posture, full tier, constructor + shared-instance rejection, wasm acceptance items | + +## Open Questions + +Open questions are tracked in +[open-questions.md](open-questions.md). This spec opens none — the +register is closed, and every decision this engine needed is in +[ADR-024](decisions/024-mem-engine.md). The two deliberately-not-OQ +dispositions live there: the shared-instance constructor rejection +(§4, re-entry = a consumer-inventory row) and the on-target wasm +suite execution scope-out (§5, collapse condition = an adapter +crate). \ No newline at end of file diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index d26f37f..e96e02a 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -1,16 +1,20 @@ --- status: draft -last_updated: 2026-10-07 (ADR-022 — contract-suite layout: shared internal suite crate) +last_updated: 2026-10-10 (ADR-024 — the mem engine joins the crate +family as `alkstore-mem`; family/ADR tables updated) --- # alkstore — Overview -One reactive store interface over SQLite and Postgres: durable +One reactive store interface over SQLite, Postgres, and the +in-process mem engine: durable notify/subscribe signals, streams with per-consumer offsets, durable queues with the transactional enqueue property, named locks, a scheduler, and an outbox helper — with each engine using its own native machinery underneath (SQLite: honker's watcher design; -Postgres: `pg_notify`/LISTEN and the pg-boss schema family). +Postgres: `pg_notify`/LISTEN and the pg-boss schema family; mem: +guarded in-process state, ephemeral by design — +[ADR-024](decisions/024-mem-engine.md)). ## Why this crate exists @@ -31,7 +35,7 @@ Per [ADR-001](decisions/001-crate-split.md): | `alkstore` (core) | trait surface, types, error model | none (no capability surface — [ADR-016](decisions/016-deployment-honesty.md): engine differences are compile-time identity + the deployment matrix, never a runtime descriptor) | | `alkstore-sqlite` | SQLite engine ([ADR-003](decisions/003-sqlite-driver.md)) | rusqlite; the forked honker-core lineage rides in-tree as the engine crate's substrate module subtree ([ADR-011](decisions/011-sqlite-substrate-fork.md), designed in [ADR-012](decisions/012-forked-substrate-design.md), folded per [ADR-013](decisions/013-fold-substrate-into-sqlite.md)) | | `alkstore-postgres` | Postgres engine ([ADR-004](decisions/004-postgres-driver.md)) | tokio-postgres, deadpool-postgres | -| (mem engine, optional) | test convenience, decided at implementation ([ADR-001](decisions/001-crate-split.md)) | none | +| `alkstore-mem` | mem engine — full contract v1, honest-ephemeral (no durability, single-process, never fleet-valid; [ADR-024](decisions/024-mem-engine.md)) | none (tokio sync + time only; the wasm32-clean member of the family) | Downstream consumers depend on core + exactly one engine. Family standards apply throughout: tokio async runtime, `thiserror` errors, no @@ -55,6 +59,7 @@ Per [ADR-002](decisions/002-feature-scope.md): | [core-contract.md](core-contract.md) | The unified trait surface: mechanisms, delivery guarantees, tx seam, naming | | [engine-sqlite.md](engine-sqlite.md) | SQLite engine: mapping the contract onto the forked substrate module/rusqlite | | [engine-postgres.md](engine-postgres.md) | Postgres engine: mapping the contract onto tokio-postgres/LISTEN | +| [engine-mem.md](engine-mem.md) | Mem engine: mapping the contract onto guarded in-process state (ADR-024) | | [queues.md](queues.md) | Queue/scheduler/outbox semantics depth (ADR-009/ADR-010 resolved) | | [deployment.md](deployment.md) | Host capabilities, connection budgets, deployment matrix (OQ-08 resolved) | | [open-questions.md](open-questions.md) | OQ-01..NN tracker | @@ -86,6 +91,7 @@ Per [ADR-002](decisions/002-feature-scope.md): | [020](decisions/020-enqueue-opt-semantics-and-bridges.md) | Enqueue-option semantics (delay/run_at, expires, scheduler stamp source, payload encoding) | Accepted | | [021](decisions/021-tx-reads-and-value-shape-fixes.md) | Third review round (tx-read methods on `TxHandle`, `Job.claimed_at`, `schedule()` queue-argument validation, drop = rollback, receiver close/error arms) | Accepted | | [022](decisions/022-contract-suite-layout.md) | Contract-suite layout (shared internal suite crate, store-factory parameterization) | Accepted | +| [024](decisions/024-mem-engine.md) | The mem engine (`alkstore-mem` — full contract v1, honest-ephemeral, wasm32 compile-clean acceptance) | Accepted | ## Non-goals diff --git a/docs/plans/implementation.md b/docs/plans/implementation.md index e2c8112..9c6ffa2 100644 --- a/docs/plans/implementation.md +++ b/docs/plans/implementation.md @@ -1,6 +1,8 @@ --- status: draft -last_updated: 2026-10-10 (wave 5 implemented and reviewed — review-wave-5 verified the discharge map, stamps, dispositions and green-on-both-engines; engine specs flipped to stable) +last_updated: 2026-10-10 (ADR-024 — the mem engine adopted as wave 6 +and decomposed-in-place documentation; release readiness renumbered +to wave 7. Waves 1–5 implemented and reviewed; engine specs stable.) --- # alkstore — Implementation plan @@ -23,12 +25,14 @@ boundaries. ## The waves Dependency logic in one line: core → (substrate fork ∥ postgres -engine) → sqlite engine → contract suite → release readiness. The -substrate fork is contract-blind (ADR-012 §2), so it needs only the -workspace scaffold from wave 1; the Postgres engine needs only the core -crate. Waves 2 and 4 are therefore independent of each other and could -run in either order (or in parallel, if agents are ever available in -parallel). +engine) → sqlite engine → contract suite → mem engine → release +readiness. The substrate fork is contract-blind (ADR-012 §2), so it +needs only the workspace scaffold from wave 1; the Postgres engine +needs only the core crate. Waves 2 and 4 are therefore independent of +each other and could run in either order (or in parallel, if agents +are ever available in parallel). The mem engine (wave 6) needs the +core crate plus the finished suite (wave 5) — born pinned, per +[ADR-024](../architecture/decisions/024-mem-engine.md). | Wave | Contents | Depends on | Status | |---|---|---|---| @@ -37,7 +41,8 @@ parallel). | [3](#wave-3--sqlite-engine) | SQLite engine: connection architecture, re-derived queue ops on contract v1, scheduler/outbox, tx seam, SQLite backlog column | waves 1 + 2 | implemented + reviewed (2026-10-08) | | 4 | [Postgres engine](#wave-4--postgres-engine): schema bootstrap, pool/open, listener/forwarder, all mechanisms, tx seam, pg backlog column | wave 1 | implemented + reviewed (2026-10-08) | | 5 | [Contract suite](#wave-5--contract-suite): the cross-engine equivalence properties (core-contract.md §Verification backlog), version-stamped per ADR-017 | waves 3 + 4 | implemented + reviewed (2026-10-10) | -| 6 | Release readiness: crate docs, deployment matrix final pass, README (written last, honestly), publish prep; mem-engine and fuzzing decisions | wave 5 | not yet decomposed | +| 6 | [Mem engine](#wave-6--mem-engine): separate `alkstore-mem` crate, full contract v1 over in-memory mechanisms (tx overlay, queue/scheduler/streams/locks/wakes), the suite's third factory column (born pinned), wasm32 compile-clean acceptance | waves 1 + 5 | not yet decomposed | +| 7 | [Release readiness](#wave-7--release-readiness): crate docs, deployment matrix final pass, README (written last, honestly), publish prep; fuzzing decision | wave 6 | not yet decomposed | ## Wave 1 — Foundations @@ -99,7 +104,7 @@ shaped this wave's task set: wiring (the constructor task). - **Staying put as ordered**: M-1's retention-failure test and N-5's panic probe → wave 5's contract-suite rows; N-1's growth-posture - review → wave 6. + review → wave 7. ## Wave 4 — Postgres engine @@ -244,6 +249,79 @@ expected answer, recorded per case); whether the scheduler rows demand the pg fire-wake parity the wave-4 fix-batch gate recorded; and what the SQLite lock busy-path actually returns under contention. +## Wave 6 — Mem engine + +The third engine ([ADR-024](../architecture/decisions/024-mem-engine.md), +2026-10-10): a separate `alkstore-mem` crate implementing the full +contract v1 over guarded in-memory mechanisms — the honest-ephemeral +tier (no durability, single-process, ephemeral by design, never +fleet-valid), adopted for the wasm/sandbox and downstream-adapter +economics the consumer inventory's engine-tier row records +(operator-authority, 2026-10-10 — the row-first gate satisfied before +this wave was assumed). Release readiness moved to wave 7. + +The wave's shape mirrors the engine waves' rhythm: constructor + tx +overlay → mechanisms in parallel → scheduler/outbox → suite +integration → review gate. What mem carries: the queue state machine +(visibility/reclaim/dead-letter/sweep), scheduler boundaries + +catch-up + leadership (trivially owned single-instance, still run — +engine-uniform per ADR-009 §4), streams with per-consumer offsets and +trim, TTL locks, wakes. What it does not carry: no pooling, no schema +bootstrap, no forwarder, no `spawn_blocking` seam, no reconnection +concept — the distributed machinery that made wave 4 build-heavy +doesn't exist here. The wave is estimated at pg-wave scale or +somewhat smaller. + +The load-bearing artifact is the **tx overlay**: `*_tx` effects stage +in a handle-private overlay; commit merges it atomically and fires +the overlay's pending wakes; drop discards it — read-your-own-writes +easy, drop-=-rollback trivial, and *wake-at-commit* structural. That +last property is the seam review 002 found broken pg-side +(`pg-fix-tx-wake`); mem gets it pinned from birth by the suite's +existing tx-commit-atomicity rows. + +Born pinned: [ADR-022](decisions/022-contract-suite-layout.md)'s +`StoreFactory` is reused verbatim as a third factory column wired +onto already-authored, version-stamped rows — no re-derivation. The +engine re-owns its own copies of the backoff curve, opts resolution, +and the `@every` parser ([ADR-012](decisions/012-forked-substrate-design.md) +§2's one-owner rule, third owner); the suite now pins that arithmetic +three-way. + +**Acceptance items** ([ADR-024](../architecture/decisions/024-mem-engine.md) +§5): the crate compiles clean for `wasm32-unknown-unknown` (target +already installed in the dev environment); workspace host gates stay +green; the suite's mem column passes on the host, run under a +**current-thread runtime flavor** — the test that keeps internals +wasm-clean (no `spawn_blocking`, tokio sync + time only). The +current-thread flavor rides one harness pre-task: the suite's +`past_stamp_sleep` blocking sleep (review 003's posture note at +`properties.rs:1683`) becomes an async sleep — the only suite-side +change the wave carries. Running the suite itself *on* the wasm +target is scoped out with a named collapse condition (a +wasm-bindgen-test adapter crate) — a scope decision recorded in +ADR-024 §5, not a pending-client hedge. + +Deliverables beyond code: `engine-mem.md` (the spec — drafted with +this wave's adoption), the deployment matrix's mem rows (landed with +ADR-024), and the mem factory's suite wiring. + +Riding the window: review 003's Finding 1 (the pg scheduler +transient-error retry loop, MEDIUM — engine-side, small, +mem-independent); Findings 2–3 follow their park recommendation into +wave 7's pre-release pass. + +## Wave 7 — Release readiness + +The original wave 6, renumbered when the mem engine inserted at wave +6 ([ADR-024](../architecture/decisions/024-mem-engine.md)): crate +docs, the deployment matrix final pass (across all three engines now), +the crate README (written last, honestly), publish prep, N-1's +growth-posture review — and the **fuzzing decision** (the +alksocks/alktty/alktunnels pattern), the one remaining +decided-at-implementation deferral. Decomposes after wave 6's review +gate, with review 003's Findings 2–3 riding the pre-release pass. + ## Decided points - **Contract-suite layout — option (a)**: a small internal @@ -262,11 +340,21 @@ and what the SQLite lock busy-path actually returns under contention. (self-hosted Gitea; supply-chain posture). No CI-wiring task exists; the merge gates (`cargo test`, `cargo clippy --all-targets -- -D warnings`, `cargo fmt --check`) are run coordinator-side. -- **Mem engine** (ADR-001 §4) and **fuzzing adoption** (the - alksocks/alktty/alktunnels pattern): both are "decided at - implementation" deferrals, recorded here so they surface as explicit - decision points in wave 6 (or earlier if the test story demands the - mem engine sooner) rather than ambushing a later session. +- **Mem engine** (ADR-001 §4): **decided 2026-10-10, pre-decomposition + ([ADR-024](../architecture/decisions/024-mem-engine.md))** — a + separate `alkstore-mem` crate, full contract v1, honest-ephemeral + posture, wasm32 compile-clean acceptance; wave 6, with release + readiness renumbered to wave 7. The deferral discharged: its + collapse condition ("implementation time") was reached early via + the consumer pause + the wasm/sandbox motivation, with the row-first + gate satisfied by the consumer inventory's engine-tier row + (operator-authority, 2026-10-10). +- **Fuzzing adoption** (the alksocks/alktty/alktunnels pattern): + remains the one decided-at-implementation deferral, recorded here + so it surfaces as an explicit decision point in wave 7 (per + AGENTS.md's no-fuzzing posture, revisit only if a task or ADR + introduces it). Renumbered from wave 6 together with the mem + decision's move. ## Review gates @@ -285,6 +373,12 @@ decomposes. Specific gates: - **Wave 5 review** — the suite as compatibility instrument: every backlog row present, version-stamped, green on both engines; this is the gate that flips the engine specs to `stable`. +- **Wave 6 review** — mem-engine-vs-contract conformance (the + born-pinned gate): the suite's third column green on the host, the + wasm32 compile gate clean, the current-thread-flavor column run + (harness pre-task disposed), the engine's own mechanism tests, and + the `engine-mem.md` spec read against the pinned ADR-024 text — + the gate that flips the mem spec toward `stable`. ## Review rounds so far @@ -296,7 +390,7 @@ decomposes. Specific gates: `docs/reviews/001-waves-1-2-general-review.md`) — M-1 fixed inline (sweep savepoint scope); M-2/N-2/N-4/N-6 resolved as ADR-023 pre-decomposition; lint removal + suite adds folded into waves 3/5; - N-1 recorded for wave 6. + N-1 recorded for release readiness (now wave 7). - **Wave 3 review gate** (`review-wave-3`) — engine-vs-contract conformance code-read clean; two findings fixed inline (the `open_writer_connection` boundary move out of `substrate/mod.rs`; diff --git a/docs/research/consumer-inventory.md b/docs/research/consumer-inventory.md index bbb3bea..c5cc9a0 100644 --- a/docs/research/consumer-inventory.md +++ b/docs/research/consumer-inventory.md @@ -1,10 +1,10 @@ --- status: draft -last_updated: 2026-10-05 (streams corrected to documented/in-scope — -operator-authority record: type-filtered event watching from multiple -places, e.g. repo-change subscriptions in a git app. alkcall row added -2026-10-05 as ecosystem-shape evidence for streams (no per-key-ordering -need named anywhere); rate-limits + result-storage remain the +last_updated: 2026-10-10 (mem-engine engine-tier row added — +operator-authority record: wasm32 sandboxing + downstream +ffi/napi/python-adapter economics; the record the ADR-024 scope +decision cites per the row-first gate. Earlier: 2026-10-05 streams +correction and alkcall row; rate-limits + result-storage remain the keep-with-flag rows.) --- @@ -203,6 +203,31 @@ cut-at-implementation.** Adjacent-tooling shape (a queue job's outcome queryable by id), useful to queues+scheduler consumers but named by none of them yet. +## Engine-tier inventory + +This section answers the engine-tier question (which backing engines +exist) under this document's discipline. The two shipped engines +(SQLite, Postgres) were Phase 0 decisions with consumer evidence +throughout; the mem engine entered as ADR-001 §4's implementation-time +deferral, so its need gets a row here *before* any scope decision +assumed it — the row-first gate, engine-shaped. (Rows above grade +*features*; this row grades *which engine tier a consumer needs*.) + +| Consumer | Evidence | Need | Confidence | +|---|---|---|---| +| wasm-sandboxed consumers (family-wide) | operator-authority record, 2026-10-10 | an engine that runs in-process on `wasm32-unknown-unknown`: the alk protocol crates (alkcall, alktty, alktunnels, alksocks) are wasm-compatible at the protocol level by design and sandboxing is the family's deployment direction; of the engine triad, only the in-memory engine can ride along (rusqlite and tokio-postgres are structurally out on wasm) | **operator-authority** (the REQ-2 recording convention; expected where the wanter is an *application above* the paused crates) | +| downstream adapter crates (ffi/napi/python) | operator-authority record, 2026-10-10 | a zero-configuration engine for adapter crates and first-running examples: wasm compatibility downstream makes ffi/napi/python adapters far cheaper to build (no per-platform registration/build matrix the operator cannot run — no mac/windows hosts); the engine an adapter always needs is one that just works in-process | **operator-authority** (same record) | + +Verdict: **in scope as a third engine — full contract v1** +([ADR-024](../architecture/decisions/024-mem-engine.md), 2026-10-10): +separate `alkstore-mem` crate, no durability, single-process, +ephemeral by design, never fleet-valid. The tier is *full contract +v1*, not a reduced test-only profile: a reduced engine never gets +born-pinned by the contract suite, and the wasm/adapter motivation is +consumer-shaped, not test-shaped. *(Upgrade path to `documented`: +whichever consumer document first names the mem engine's need grows +the row, per this document's conventions.)* + ## What this inventory changes - **OQ-ST-01** stops being "blocked on a consumer-driven inventory diff --git a/docs/reviews/001-waves-1-2-general-review.md b/docs/reviews/001-waves-1-2-general-review.md index f575516..be99a3a 100644 --- a/docs/reviews/001-waves-1-2-general-review.md +++ b/docs/reviews/001-waves-1-2-general-review.md @@ -9,7 +9,8 @@ > exists on the stream reads, and domains must be engine-uniform), > N-4 (URI flag dropped, a registered fork delta), N-6 (1 ms default > stands, cadence carried onto `SqliteOpts`). The task-carried items -> (lint removal, wave-5 suite adds, wave-6 N-1 review) stand as +> (lint removal, wave-5 suite adds, release-readiness N-1 review — +> wave 7 since ADR-024's renumbering) stand as > ordered in §7. - **Reviewer**: opencode (glm-5.3-flash) diff --git a/docs/reviews/002-wave-4-general-review.md b/docs/reviews/002-wave-4-general-review.md index d91ce92..9ff18ce 100644 --- a/docs/reviews/002-wave-4-general-review.md +++ b/docs/reviews/002-wave-4-general-review.md @@ -246,8 +246,9 @@ covers it, but say so. defect — but a one-sentence doc note (consumer-obligation, like the visibility-budgeting one) would prevent surprise. - `eprintln!` diagnostics at the swallowed-wake/lag sites match the - SQLite twin's posture; the logging story is a wave-6 item (already - recorded by the wave-4 gate). + SQLite twin's posture; the logging story is a wave-7 item (a wave-6 + item when recorded — renumbered by ADR-024; already recorded by the + wave-4 gate). - F-2..F-4 from `tasks/review-wave-4.md` stand as recorded there. ## Recommended sequencing diff --git a/docs/reviews/003-wave-5-general-review.md b/docs/reviews/003-wave-5-general-review.md index 20b6dfe..45d9ed7 100644 --- a/docs/reviews/003-wave-5-general-review.md +++ b/docs/reviews/003-wave-5-general-review.md @@ -232,14 +232,20 @@ explicitly here so the call is recorded rather than implicit. structurally blocked by the own-schema re-drain argument; the lock row's lapse assertions can only be delayed, never un-lapsed, by slow clocks. Supporting the gate's watch-not-block disposition; the - wave-6 watch flag stands. + watch flag stands (the release-readiness watch — wave 7 since + ADR-024's renumbering). ## Recommendation No changes required to keep wave 5's verdict standing. For wave 6's -window: Finding 1 is a decomposable small task (transient-fault seam + +window *(wave 6 at writing — release readiness; renumbered by +ADR-024, with Finding 1 since routed to the new wave 6 — the mem +engine — window, and Findings 2–3 to wave 7's pre-release pass, per +`docs/plans/implementation.md`'s wave-6 section)*: Finding 1 is a +decomposable small task (transient-fault seam + retry/exhaustion pin); Findings 2–3 can ride the same task or be accepted with the code-read as their pin — either way the disposition should be recorded. Findings 4–5 need no work. The env-variable -boundary section above folds into wave 6's release-readiness pass as a +boundary section above folds into the release-readiness pass +(wave 7) as a verified-now / re-check-at-prepublish item. \ No newline at end of file diff --git a/tasks/pg-engine-schema.md b/tasks/pg-engine-schema.md index 453b5d6..be9dd9c 100644 --- a/tasks/pg-engine-schema.md +++ b/tasks/pg-engine-schema.md @@ -68,7 +68,7 @@ A `schema.rs` module in `alkstore-postgres`: - **No migration machinery in v1** — tables are created fresh per schema; append-column migrations are the substrate's evolution discipline, not a greenfield v1 need (note the posture in the module - docs; wave 6's release pass revisits if the deployment story + docs; wave 7's release pass revisits if the deployment story demands in-place upgrades). Tests use a raw `tokio_postgres::Connection`/Client against the diff --git a/tasks/review-wave-3.md b/tasks/review-wave-3.md index 4d65af7..f97b9ca 100644 --- a/tasks/review-wave-3.md +++ b/tasks/review-wave-3.md @@ -189,7 +189,7 @@ inline-clean, or already owned by a later wave): connection *still in use* across store-close keeps running on a closed file until its op completes — harmless in practice, the op finishes on an open connection that is then dropped). Noted for - wave 6's close-path review; no consumer-visible defect. + release readiness's close-path review (now wave 7); no consumer-visible defect. - **Coverage of the commit-error arm**: no test induces a failing `COMMIT` (needs SQLITE_FULL/IOERR-class injection; the code fix is pinned by code-read + the shared `reopen` machinery being diff --git a/tasks/review-wave-4.md b/tasks/review-wave-4.md index f85c22c..102c1d1 100644 --- a/tasks/review-wave-4.md +++ b/tasks/review-wave-4.md @@ -224,7 +224,7 @@ acceptance criteria — not test-name-trust): file holds; `#[non_exhaustive]` policy respected (none on `PgOpts`); core has no driver deps; `eprintln!` diagnostics at the five swallowed-wake/lag sites match the SQLite twin's posture (the - logging story is a wave-6 item, not a wave-4 defect). + logging story is a wave-7 item (release readiness), not a wave-4 defect). - **Gates**: `cargo build`, `cargo test` server-less (25 core + 188 sqlite + 111 pg-skip + 10 + 10 suite + 3 + 9 harness), clippy `-D warnings`, fmt — all green; the pg lib suite 111/111 green @@ -317,7 +317,7 @@ wave-5/6 decision; none is a consumer-visible defect): but `core-contract.md`'s streams/notify sections don't mention the channel naming (they don't need to — it's engine-internal), while the reserved reconnect string *is* a contract string. No action — - recorded here so wave 6's docs pass notices where the naming live. + recorded here so wave 7's docs pass notices where the naming live. Wave 5 decomposition may proceed. diff --git a/tasks/review-wave-5.md b/tasks/review-wave-5.md index 103180d..eddcd04 100644 --- a/tasks/review-wave-5.md +++ b/tasks/review-wave-5.md @@ -67,7 +67,9 @@ Primary lenses: recorded - [x] Engine specs flipped to `stable` with dated annotations; implementation.md updated (wave table + review rounds) -- [x] Findings recorded; wave 6 decomposition may proceed +- [x] Findings recorded; the next decomposition may proceed + (release readiness at gate time — since renumbered to wave 7 + when the mem engine inserted at wave 6, ADR-024) ## References @@ -214,7 +216,8 @@ repro loop) *before* any postulate-and-fix — per the user's ledger the pattern (a prior unreproducible flake hinting at a real engine issue) deserves that attention. Not blocking this gate: 13+ subsequent clean pg runs across both rows since the failures, all conditions -covered. **Flagged for watch in wave 6's window.** +covered. **Flagged for watch in the next wave window** (release +readiness at gate time — since renumbered to wave 7, ADR-024). **7. Findings.** No code, suite-row, or stamp defects requiring change were found at gate time; all changes this gate made are doc-side diff --git a/tasks/scaffold-workspace.md b/tasks/scaffold-workspace.md index a787008..3bd76a5 100644 --- a/tasks/scaffold-workspace.md +++ b/tasks/scaffold-workspace.md @@ -28,7 +28,7 @@ Workspace-level: shared `[workspace.package]` fields (edition, license MIT OR Apache-2.0, repository), `resolver = "2"` (or 3 per edition), and a root `Cargo.toml` with `members`. Crate versions are `0.1.0` placeholders — ADR-017 §1 pins the initial *release* at 1.0.0; that -bump is wave 6's release task, not this one. +bump is wave 7's release task, not this one. The engines' manifests must pin the core dependency as a path dependency with the version field set (the ADR-017 §4.1 manifest-pin @@ -73,7 +73,7 @@ outside tests, no comments in code (doc comments on public API fine). `https://git.alk.dev/alkdev/alkstore`, `Cargo.lock` committed, `.gitignore` of `target/` + `.worktrees/`. LICENSE-MIT/LICENSE-APACHE files at repo root are **not** added here — they belong with the - release-readiness/publish-prep work (wave 6), matching the repo's + release-readiness/publish-prep work (wave 7), matching the repo's no-honest-artifact-yet posture; the manifests already declare the license field per the task spec. - Engine manifests pin the core path dependency with the version field