672 lines
38 KiB
Markdown
672 lines
38 KiB
Markdown
---
|
||
status: draft
|
||
last_updated: 2026-10-10 (the fuzzing decision resolved: adopted and
|
||
inserted as wave 7 per docs/research/fuzzing.md; release readiness
|
||
renumbered to wave 8. Waves 1–6 implemented + reviewed; the wave-6
|
||
gate passed with three inline edge-domain fixes, and Findings 2–3 and
|
||
the flake watch carry to the wave-8 pre-release pass.)
|
||
---
|
||
|
||
# alkstore — Implementation plan
|
||
|
||
Wave-based decomposition of the architecture
|
||
(`docs/architecture/`) into units of implementable work. This is a
|
||
deliberate deviation from the SDD process's decompose-everything-upfront
|
||
step: the architecture is large, early waves change the shape of later
|
||
ones (the fork's outcomes feed the SQLite engine tasks; the contract
|
||
suite's harness shape feeds both engines' verification work), and
|
||
decomposing only the next wave or two at a time keeps each session's
|
||
task set reviewable and lets later decompositions absorb earlier waves'
|
||
course corrections.
|
||
|
||
Rhythm: decompose a wave → implement it → review gate → decompose the
|
||
next wave. Task files live in `tasks/` (taskgraph-managed; frontmatter
|
||
carries the categorical estimates). Wave boundaries are also review
|
||
boundaries.
|
||
|
||
## The waves
|
||
|
||
Dependency logic in one line: core → (substrate fork ∥ postgres
|
||
engine) → sqlite engine → contract suite → mem engine → fuzzing →
|
||
release readiness. The substrate fork is contract-blind (ADR-012 §2),
|
||
so it needs only the workspace scaffold from wave 1; the Postgres engine
|
||
needs only the core crate. Waves 2 and 4 are therefore independent of
|
||
each other and could run in either order (or in parallel, if agents
|
||
are ever available in parallel). The mem engine (wave 6) needs the
|
||
core crate plus the finished suite (wave 5) — born pinned, per
|
||
[ADR-024](../architecture/decisions/024-mem-engine.md). The fuzzing
|
||
wave (wave 7) needs the mem engine finished (its campaigns precede the
|
||
release freeze; the targets ride in-process surfaces — core payloads,
|
||
the three per-engine parsers, name validation — see
|
||
[docs/research/fuzzing.md](../research/fuzzing.md)), and release
|
||
readiness is wave 8.
|
||
|
||
| Wave | Contents | Depends on | Status |
|
||
|---|---|---|---|
|
||
| [1](#wave-1--foundations) | Workspace scaffold; core crate (errors, value types, full trait surface); contract-suite scaffold | — | implemented + reviewed (2026-10-08) |
|
||
| [2](#wave-2--sqlite-substrate-fork) | honker-core fork into `alkstore-sqlite/src/substrate/`: port, deltas, provenance, test floor | wave 1 (workspace scaffold only) | implemented + reviewed (2026-10-08) |
|
||
| [3](#wave-3--sqlite-engine) | SQLite engine: connection architecture, re-derived queue ops on contract v1, scheduler/outbox, tx seam, SQLite backlog column | waves 1 + 2 | implemented + reviewed (2026-10-08) |
|
||
| 4 | [Postgres engine](#wave-4--postgres-engine): schema bootstrap, pool/open, listener/forwarder, all mechanisms, tx seam, pg backlog column | wave 1 | implemented + reviewed (2026-10-08) |
|
||
| 5 | [Contract suite](#wave-5--contract-suite): the cross-engine equivalence properties (core-contract.md §Verification backlog), version-stamped per ADR-017 | waves 3 + 4 | implemented + reviewed (2026-10-10) |
|
||
| 6 | [Mem engine](#wave-6--mem-engine): separate `alkstore-mem` crate, full contract v1 over in-memory mechanisms (tx overlay, queue/scheduler/streams/locks/wakes), the suite's third factory column (born pinned), wasm32 compile-clean acceptance | waves 1 + 5 | implemented + reviewed (2026-10-10) |
|
||
| [7](#wave-7--fuzzing) | [Fuzzing](#wave-7--fuzzing): the `fuzz/` sidecar workspace (cargo-fuzz targets over the payload codec, the three `@every` parsers, name validation), committed seed corpora + the corpus-replay gate, detached campaigns | wave 6 | decomposed (2026-10-10) |
|
||
| [8](#wave-8--release-readiness) | [Release readiness](#wave-8--release-readiness): crate docs, deployment matrix final pass, README (written last, honestly), publish prep | wave 7 | not yet decomposed |
|
||
|
||
## Wave 1 — Foundations
|
||
|
||
The core crate is contract v1 in code: the error taxonomy (ADR-008 §5),
|
||
the value types (ADR-019 §3, ADR-020), the full trait surface
|
||
(ADR-008 §1–§3/§8, ADR-014, ADR-019, ADR-021), and the payload encoding
|
||
posture (ADR-020 §4). Nothing engine-specific lives here — no
|
||
resolution arithmetic, no SQL. The equal-jitter curve, opts-stamping
|
||
resolution, and boundary math are deliberately *not* core: ADR-012 §2
|
||
pins each engine as the owner of one implementation, with equivalence
|
||
pinned by the contract suite (wave 5).
|
||
|
||
The contract suite gets scaffolded now (not in wave 5) because its
|
||
harness shape — a `Store`-factory-parameterized property crate — is
|
||
easier to grow row by row as engines land than to retrofit onto two
|
||
finished engines. Wave 5 fills it with the cross-engine equivalence
|
||
rows; waves 3 and 4 adopt the harness for their own backlog columns.
|
||
|
||
## Wave 2 — SQLite substrate fork
|
||
|
||
The fork per ADR-011/012/013: port honker-core at `f4e53c6` into
|
||
`alkstore-sqlite/src/substrate/`, apply the three watcher port deltas
|
||
and the bootstrap re-keying, drop cron/experimental/cut-flag machinery,
|
||
re-own the table family as `__alkstore_*`, carry `PROVENANCE.md` and
|
||
the dual-license notice in-tree (ADR-018), and stand up the inherited
|
||
test suites as the floor. The substrate stays sync and contract-blind;
|
||
the engine layer that maps it onto the core contract is wave 3, not
|
||
here. Reviewability against the lineage (ADR-012 §3) is a property the
|
||
wave-2 review gate checks explicitly.
|
||
|
||
## Wave 3 — SQLite engine
|
||
|
||
The engine layer that maps the forked substrate onto the core
|
||
contract: the `open` constructor (connection architecture — writer
|
||
slot, reader pool, watcher spawn — per engine-sqlite.md), the
|
||
`spawn_blocking` seam, the full `Store`/`TxHandle`/mechanism-handle
|
||
trait impls over the substrate's ops, the scheduler leader loop and
|
||
outbox helper, and the engine's backlog column in the contract suite
|
||
(the ADR-023 rows among them, factory-parameterized so wave 5 runs
|
||
them against both engines).
|
||
|
||
The waves-1–2 general review
|
||
(`docs/reviews/001-waves-1-2-general-review.md`) and its
|
||
resolution ([ADR-023](../architecture/decisions/023-fourth-review-round.md))
|
||
shaped this wave's task set:
|
||
|
||
- **Already landed pre-decomposition** (commit `44637ee`, not wave-3
|
||
tasks): `encode_payload` is fallible (`Result<Vec<u8>, Error::Codec>`,
|
||
ADR-023 §1); `open_conn` drops `SQLITE_OPEN_URI` (§3, register
|
||
D-29); the numeric-argument domain table is pinned in
|
||
core-contract.md (§2); the 1 ms watcher default stands with the
|
||
cadence documented in deployment.md (§4).
|
||
- **Folded into wave 3's tasks** (the review's §7 wiring items): the
|
||
trait-impl extent/duration guards (the contract-side domain rule,
|
||
enforced at the engine's trait-impl entry — the `sqlite-engine-*`
|
||
tasks carry it per mechanism), the two `#![allow]` lint removals in
|
||
`substrate/mod.rs` (the integration task — they can only lift once
|
||
every substrate surface is wired), and `SqliteOpts::poll_interval`
|
||
wiring (the constructor task).
|
||
- **Staying put as ordered**: M-1's retention-failure test and N-5's
|
||
panic probe → wave 5's contract-suite rows; N-1's growth-posture
|
||
review → wave 8.
|
||
|
||
## Wave 4 — Postgres engine
|
||
|
||
The second engine: the core contract implemented on tokio-postgres +
|
||
deadpool-postgres per engine-postgres.md — schema bootstrap (one
|
||
engine-owned schema, default `alkstore`, ADR-010 §8), the pool/open
|
||
constructor, the hand-rolled LISTEN forwarder (dedicated non-pooled
|
||
listener connection, bounded broadcast fanout, exponential-backoff
|
||
reconnect with re-LISTEN, the synthetic reconnect-wake on
|
||
`__alkstore_listener_reconnected__`), the full `Store`/`TxHandle`/
|
||
mechanism-handle trait impls (natively async — no `spawn_blocking`
|
||
seam; the tx handle holds the pooled object directly, ADR-007's pg
|
||
arm), the re-derived queue machinery (`FOR UPDATE SKIP LOCKED` claims,
|
||
LISTEN-driven wake with re-poll safety net), the scheduler leader loop
|
||
and outbox helper, and the engine's backlog column in the contract
|
||
suite (factory-parameterized; wave 5 runs the same rows against both
|
||
engines).
|
||
|
||
The wave is deliberately shaped as wave 3's structural twin — same
|
||
task rhythm (constructor → seam → mechanisms in parallel →
|
||
scheduler/outbox → integration → review gate) — because the two
|
||
engines implement the same contract and the wave-3 task set's
|
||
boundaries proved clean. The deltas are the engine's own: no
|
||
substrate (the pg machinery is re-derived greenfield per ADR-004/005,
|
||
with the pg-boss schema family as design reference), a natively-async
|
||
seam (Send+Sync client, POC #2 compile-probe), the forwarder as the
|
||
wake substrate (with its two POC-pinned deadlock pitfalls owned as
|
||
test-pinned failure modes), and the `PayloadTooLarge` client-side
|
||
check (this engine *produces* the variant — the one runtime carriage
|
||
of an engine asymmetry, ADR-016 §5).
|
||
|
||
Wave-3 outcomes absorbed into the task set:
|
||
|
||
- **The `#[doc(hidden)]` constructors** (`Job::from_row`,
|
||
`StreamEvent::from_row`, `Schedule::new`, `Wake::new`) already
|
||
exist — wave 3's pre-work task built them engine-agnostically; wave
|
||
4 consumes them as-is (no wave-4 equivalent needed).
|
||
- **The engine-side arithmetic is re-owned, not shared**: the
|
||
equal-jitter backoff curve, the opts-resolution rules
|
||
(delay-over-`run_at`, relative-`expires`, the derived stamp sets
|
||
300/3/5/none and 60/5/5), and the outbox backing-queue derivation
|
||
are each engine's own implementation (ADR-012 §2's one-owner rule
|
||
applied per engine); the pg tasks re-implement them from the ADR
|
||
text, and the contract suite (wave 5) pins the two engines'
|
||
arithmetic to identical outputs. The SQLite engine's `resolution.rs`
|
||
is reference, not a dependency.
|
||
- **The `@every` spec grammar has no shared parser on this side** —
|
||
the SQLite engine reuses the substrate's `parse_every_interval`
|
||
(one-line delta D-31's neighbor); the pg engine owns its parser
|
||
(the grammar is small and pinned: `@every <n><unit>`, `s|m|h|d`).
|
||
Equivalence of *behavior* (what fires, when) is suite-pinned; the
|
||
parsers themselves are per-engine.
|
||
- **The wave-3 review's deferred notes** (reader-pool close path,
|
||
commit-error-arm coverage) stay SQLite-scoped — the pg engine's
|
||
pool is deadpool-managed (no hand-rolled close path to review) and
|
||
its commit-error arm gets coverage in the pg tasks' own tests.
|
||
- **The suite's factory contract** (isolation + idempotent teardown)
|
||
is what the pg factory implements — a fresh schema per `open` per
|
||
the SQLite factory's precedent (fresh temp file there); the
|
||
POC's shared-server parallel-interference caveat (findings,
|
||
invocation note) is answered by exactly that isolation, not by
|
||
sequential-only harnesses.
|
||
|
||
Test posture: the engine's tests need a live Postgres server. The
|
||
harness convention is the POC's (dockerized `postgres:16-alpine` on
|
||
:15432, the `pglo-poc` container — still running in the dev
|
||
environment); connection settings ride environment/config, never
|
||
hardcoded. Tests that cannot reach a server skip cleanly (the
|
||
`SKIP`-posture), so the workspace gates stay green server-less — but
|
||
the wave's acceptance requires the suite green *against* the server.
|
||
|
||
One design point the architecture leaves to implementation (pinned in
|
||
the wave-4 tasks, flagged here for visibility): **the queue/stream
|
||
wake-channel naming**. The contract pins LISTEN as the pg wake
|
||
trigger and pins the channels/queues shared namespace, but no doc
|
||
names the channel an enqueue/publish notifies on. The tasks pin the
|
||
natural realization — the mechanism name *is* the wake channel
|
||
(`listen(queue_name)` receives that queue's enqueue wakes; same for
|
||
streams), legal by construction under the shared namespace — with
|
||
the reserved reconnect-wake channel staying the only reserved string.
|
||
|
||
## Wave 5 — Contract suite
|
||
|
||
Filling the scaffolded suite (`alkstore-contract-suite`) with the
|
||
remaining verification-backlog rows and running the compatibility
|
||
gate. Waves 3 and 4 each landed a nine/ten-row backlog column; the
|
||
wave-5 decomposition started from an audit of what those columns
|
||
already discharge versus what the backlog still lacks.
|
||
|
||
**Already discharged by the engines' columns** (the wave-5 review
|
||
gate verifies each claim by reading the pinning row, not by trusting
|
||
the audit): name validation (the exemplar, incl. `schedule()`'s
|
||
queue-argument validation), the numeric-domain rows (extents +
|
||
durations), `encode_payload` typed failure + round-trip, the
|
||
`PayloadTooLarge` occurrence asymmetry (both engine-scoped arms),
|
||
drop-=-rollback no-ghosts, in-tx read-your-own-writes (which also
|
||
discharges the `save_offset_tx` exactly-once backlog row across
|
||
itself and the drop-rollback row), receiver close/save arms + monotone
|
||
composition (both columns), and — pinned engine-side rather than as
|
||
suite rows, per the sqlite-engine-integration task's scoping call —
|
||
plain-path SQLite open and the watcher-cadence knob
|
||
(`open_tests.rs`; SQLite-scoped backlog rows).
|
||
|
||
**Wave 5's task set** (the gaps):
|
||
|
||
- Queue depth: the job-handle validity predicate, the ADR-010 depth
|
||
properties (reclaim-eats-attempt, dead-letter moves, the
|
||
no-stranded-rows sweep), and M-1's retention-failure pin
|
||
(`suite-queue-depth-rows`).
|
||
- Scheduler: boundary fires, bounded catch-up, leadership discipline —
|
||
with a documented extension of the suite's determinism posture to
|
||
admit runner-driving rows (`suite-scheduler-rows`).
|
||
- Tx seam: `outbox_enqueue_tx` and `publish_with_key_tx`
|
||
commit-atomicity, plus N-5's `with_tx` panic-disposition probe —
|
||
the panic path pinned against real engines for the first time
|
||
(`suite-tx-commit-atomicity-rows`).
|
||
- Streams: the cross-engine ordering-equivalence row and the full
|
||
`trim_to` semantics row (`suite-stream-rows`).
|
||
- Locks: TTL/expiry re-acquisition and the concurrent-`try_lock`
|
||
loser posture — the SQLite busy-path is the open question the row
|
||
exists to answer (`suite-lock-rows`).
|
||
- Wakes: the pinned `WakeReceiver` shapes, tolerance-bounded
|
||
(`suite-wake-rows`).
|
||
- Arithmetic equivalence: the backoff-curve row and the two
|
||
wall-clock-adjacent opts-resolution legs the existing row deferred
|
||
to this wave (`suite-opts-backoff-rows`).
|
||
- Engine-side hardening riding the window: the wave-3 review's
|
||
deferred SQLite commit-error-arm coverage (`sqlite-commit-error-arm`)
|
||
and F-1's defense-in-depth candidates — explicitly not load-bearing
|
||
since `pg-fix-tx-wake` retired the flake's root cause
|
||
(`pg-suite-infra-hardening`).
|
||
- The review gate (`review-wave-5`): the backlog discharge audit,
|
||
stamp verification, green-on-both-engines, the parked-disposition
|
||
audit — and the gate that flips `engine-sqlite.md` /
|
||
`engine-postgres.md` to `stable`.
|
||
|
||
Judgment calls the decomposition parked in task Notes (the review
|
||
gate audits each): where the M-1 retention-failure pin, the
|
||
beyond-64-cap skip-forward leg, and the backoff 1-hour-cap leg land
|
||
when the public surface cannot express them (engine-side is the
|
||
expected answer, recorded per case); whether the scheduler rows
|
||
demand the pg fire-wake parity the wave-4 fix-batch gate recorded;
|
||
and what the SQLite lock busy-path actually returns under contention.
|
||
|
||
## Wave 6 — Mem engine
|
||
|
||
The third engine ([ADR-024](../architecture/decisions/024-mem-engine.md),
|
||
2026-10-10): a separate `alkstore-mem` crate implementing the full
|
||
contract v1 over guarded in-memory mechanisms — the honest-ephemeral
|
||
tier (no durability, single-process, ephemeral by design, never
|
||
fleet-valid), adopted for the wasm/sandbox and downstream-adapter
|
||
economics the consumer inventory's engine-tier row records
|
||
(operator-authority, 2026-10-10 — the row-first gate satisfied before
|
||
this wave was assumed). Release readiness moved to wave 7, and again
|
||
to wave 8 when the fuzzing wave inserted at 7 (2026-10-10).
|
||
|
||
The wave's shape mirrors the engine waves' rhythm: constructor + tx
|
||
overlay → mechanisms in parallel → scheduler/outbox → suite
|
||
integration → review gate. What mem carries: the queue state machine
|
||
(visibility/reclaim/dead-letter/sweep), scheduler boundaries +
|
||
catch-up + leadership (trivially owned single-instance, still run —
|
||
engine-uniform per ADR-009 §4), streams with per-consumer offsets and
|
||
trim, TTL locks, wakes. What it does not carry: no pooling, no schema
|
||
bootstrap, no forwarder, no `spawn_blocking` seam, no reconnection
|
||
concept — the distributed machinery that made wave 4 build-heavy
|
||
doesn't exist here. The wave is estimated at pg-wave scale or
|
||
somewhat smaller.
|
||
|
||
The load-bearing artifact is the **tx overlay**: `*_tx` effects stage
|
||
in a handle-private overlay; commit merges it atomically and fires
|
||
the overlay's pending wakes; drop discards it — read-your-own-writes
|
||
easy, drop-=-rollback trivial, and *wake-at-commit* structural. That
|
||
last property is the seam review 002 found broken pg-side
|
||
(`pg-fix-tx-wake`); mem gets it pinned from birth by the suite's
|
||
existing tx-commit-atomicity rows.
|
||
|
||
Born pinned: [ADR-022](decisions/022-contract-suite-layout.md)'s
|
||
`StoreFactory` is reused verbatim as a third factory column wired
|
||
onto already-authored, version-stamped rows — no re-derivation. The
|
||
engine re-owns its own copies of the backoff curve, opts resolution,
|
||
and the `@every` parser ([ADR-012](decisions/012-forked-substrate-design.md)
|
||
§2's one-owner rule, third owner); the suite now pins that arithmetic
|
||
three-way.
|
||
|
||
**Acceptance items** ([ADR-024](../architecture/decisions/024-mem-engine.md)
|
||
§5): the crate compiles clean for `wasm32-unknown-unknown` (target
|
||
already installed in the dev environment); workspace host gates stay
|
||
green; the suite's mem column passes on the host, run under a
|
||
**current-thread runtime flavor** — the test that keeps internals
|
||
wasm-clean (no `spawn_blocking`, tokio sync + time only). The
|
||
current-thread flavor rides one harness pre-task: the suite's
|
||
`past_stamp_sleep` blocking sleep (review 003's posture note at
|
||
`properties.rs:1683`) becomes an async sleep — the only suite-side
|
||
change the wave carries. Running the suite itself *on* the wasm
|
||
target is scoped out with a named collapse condition (a
|
||
wasm-bindgen-test adapter crate) — a scope decision recorded in
|
||
ADR-024 §5, not a pending-client hedge.
|
||
|
||
Deliverables beyond code: `engine-mem.md` (the spec — drafted with
|
||
this wave's adoption), the deployment matrix's mem rows (landed with
|
||
ADR-024), and the mem factory's suite wiring.
|
||
|
||
Riding the window: review 003's Finding 1 (the pg scheduler
|
||
transient-error retry loop, MEDIUM — engine-side, small,
|
||
mem-independent); Findings 2–3 follow their park recommendation into
|
||
the pre-release pass (now wave 8).
|
||
|
||
**Wave 6's task set** (the decomposition, 2026-10-10 — the pg wave's
|
||
rhythm: foundations → seam → mechanisms in parallel → scheduler/outbox
|
||
→ integration → review gate):
|
||
|
||
- `mem-engine-foundations` — crate scaffold (core + tokio sync+time
|
||
only), `MemStore::new()` isolation, guarded state core, wake-routing
|
||
substrate, clock accessor; the wasm32 compile gate starts here and
|
||
stays green onward.
|
||
- `mem-engine-seam-tx` — the load-bearing task: the tx overlay (all
|
||
eleven `*_tx` staged, commit-merge + wake-at-commit, drop =
|
||
discard, `with_tx` panic posture) + the `Store` trait surface +
|
||
entry-point numeric-domain guards (`suite-tx-commit-atomicity-rows`
|
||
is what consumes it once integrated).
|
||
- `mem-engine-notify-listen` — channels wake substrate surface,
|
||
three recv forms, close-at-engine-drop, no-replay, never coalesced;
|
||
depends on the seam (tx-staged notify).
|
||
- `mem-engine-queues` — the ADR-010 state machine driver-free;
|
||
equal-jitter curve and opts resolution as the third owner.
|
||
- `mem-engine-streams` — in-memory log, per-consumer offsets,
|
||
`trim_to`, receiver arms.
|
||
- `mem-engine-locks` — TTL registry; the SQLite busy-path question
|
||
has no mem analog (loser is a clean `None`).
|
||
- `mem-engine-scheduler-outbox` — tick/boundary/catch-up/leadership
|
||
through the reserved scheduler lock; outbox over queues.
|
||
- `suite-harness-current-thread` — the wave's one suite-side pre-task
|
||
(async `past_stamp_sleep`); engine-independent, required before the
|
||
mem column's current-thread flavor.
|
||
- `mem-engine-integration` — full-surface wiring, the suite's third
|
||
`StoreFactory` column green on host under the current-thread
|
||
flavor, the wasm32 acceptance items, `engine-mem.md` to
|
||
implementation-complete.
|
||
- `pg-fix-scheduler-tick-seam` — review 003 Finding 1, riding the
|
||
window (engine-side, mem-independent; the last uncovered wave-4
|
||
hardening arm).
|
||
- `review-wave-6` — the born-pinned gate: conformance read, third-
|
||
column discharge audit, wasm acceptance audit, spec disposition;
|
||
depends on integration + the rider so the window closes everything.
|
||
|
||
Parallelism note: the foundation task blocks the seam; the seam blocks
|
||
the mechanisms; the mechanisms are mutually parallel. The suite pre-task
|
||
and the Findings-1 rider are independent of the mem chain entirely and
|
||
can run any time in the window (the rider must close before the gate).
|
||
|
||
## Wave 7 — Fuzzing
|
||
|
||
**Decided 2026-10-10, before release readiness decomposed** — the
|
||
plan's one decided-at-implementation deferral is discharged here rather
|
||
than folded into the release pass: adoption (the family pattern per
|
||
alkcall/alktunnels/alksocks/alktty), inserted as this wave with release
|
||
readiness renumbered to wave 8, so campaigns run before the release
|
||
freeze. The rationale, target inventory, invariants, campaign posture,
|
||
and inherited tooling are written up in
|
||
[docs/research/fuzzing.md](../research/fuzzing.md) — this section
|
||
carries only the plan-level shape.
|
||
|
||
Scope (wave 1 of the doc's two-wave framing; `mem_opseq` is the
|
||
deferred wave 2, post-release): the `fuzz/` sidecar workspace laid out
|
||
per the upstream pattern — nightly pinned inside `fuzz/` only, the
|
||
stable-toolchain `fuzz/shared` crate whose corpus replay is the fuzz
|
||
gate (`cargo test --manifest-path fuzz/shared/Cargo.toml`), the
|
||
detached runner copied from alksocks — and three targets:
|
||
`payload_codec` (core payload encode/decode invariants),
|
||
`every_parse` (the three per-engine `@every` parsers, driven
|
||
differentially — the wave-6 overflow-finding class), and
|
||
`name_validation`. Campaigns run detached per target.
|
||
|
||
Named design points the decomposition resolves (each already sketched
|
||
in the doc): §3.2's parser-exposure decision — the three parsers are
|
||
`pub(crate)`, the family posture prefers already-`pub` surfaces, and
|
||
the doc leans (b) `#[doc(hidden)]`-style narrow exposure (the
|
||
`Job::from_row` precedent) with a public-`schedule()` companion;
|
||
the AGENTS.md posture flip (already done at adoption — the no-fuzzing
|
||
line now records the decision; the corpus-replay gate joins the
|
||
verification list only when the wave lands); and the
|
||
workspace/publish-exclusion verification. Committed crash-input seeds
|
||
promote into the corpus
|
||
(the alksocks seed-900/901 posture); grown corpora stay gitignored.
|
||
|
||
**Wave 7's task set** (the decomposition, 2026-10-10 — the doc's §4
|
||
sequencing: scaffold → exposure → targets in parallel → campaigns +
|
||
gate wiring → review gate):
|
||
|
||
- `fuzz-scaffold` — the `fuzz/` sidecar workspace per §2's layout,
|
||
copied from alksocks (nightly pin inside `fuzz/` only, the stable
|
||
`fuzz/shared` skeleton, `run-detached.sh` verbatim, `json.dict`,
|
||
`gen_fuzz_seeds.py` skeleton, the `fuzz/.gitignore` corpus pattern),
|
||
plus the root workspace's defensive `exclude = ["fuzz"]`.
|
||
- `fuzz-parser-exposure` — §3.2's design point, resolved: **option
|
||
(b)**, `#[doc(hidden)]` narrow re-exports of the three
|
||
`parse_every_interval` implementations + the three `skip_forward`
|
||
boundary functions at each engine crate root (substrate untouched —
|
||
lineage fidelity), with (a) as the recorded in-suite companion and
|
||
(c) rejected per the family's no-`#[cfg(fuzzing)]` experience.
|
||
- `fuzz-target-payload-codec` — the §3.1 target (core payload
|
||
encode/decode invariants: no-panic, the typed-failure partition,
|
||
deep-input loudness, Value + typed round-trips) + its seeds.
|
||
- `fuzz-target-every-parse` — the §3.2 differential target over the
|
||
three parsers + boundary arithmetic (three-way agreement, overflow
|
||
posture, the giant-`n` arms) + its seeds; the wave's load-bearing
|
||
target (the wave-6 finding class lives here).
|
||
- `fuzz-target-name-validation` — the §3.3 target (totality,
|
||
determinism, typed rejections, reserved-prefix rule, cap-boundary
|
||
exactness) + its seeds.
|
||
- `fuzz-campaigns-and-gate` — detached campaigns per target (§5's
|
||
rule, §7's flags), findings dispositioned (crate-side fixed +
|
||
promoted to seeds; harness-model into the research doc's
|
||
implementation log), the corpus policy's final state, the
|
||
AGENTS.md verification-list flip (the corpus-replay gate joins the
|
||
merge gates), `fuzz/README.md` finalized.
|
||
- `review-wave-7` — the gate (campaigns audit, replay-gate re-run,
|
||
corpus policy, the named design points audited resolved, the wasm
|
||
posture, records).
|
||
|
||
Parallelism note: the scaffold and the exposure task are independent
|
||
of each other; the three targets are mutually parallel once their
|
||
dependencies land (payload-codec and name-validation need only the
|
||
scaffold; every-parse needs scaffold + exposure); campaigns gate on
|
||
all three targets.
|
||
|
||
## Wave 8 — Release readiness
|
||
|
||
The original wave 6, renumbered twice: wave 7 when the mem engine
|
||
inserted at wave 6 ([ADR-024](../architecture/decisions/024-mem-engine.md)),
|
||
wave 8 when the fuzzing wave inserted at 7 (2026-10-10, the section
|
||
above): crate
|
||
docs, the deployment matrix final pass (across all three engines now),
|
||
the crate README (written last, honestly), publish prep, N-1's
|
||
growth-posture review. Decomposes after wave 7's review
|
||
gate, with review 003's Findings 2–3 riding the pre-release pass, and
|
||
the wave-5 gate's flake-watch flag carried
|
||
(`tasks/review-wave-5.md` §Notes 6 — the two unreproducible pg row
|
||
failures; if either reproduces, a dedicated investigative session
|
||
before any postulate-and-fix).
|
||
|
||
Historical note for records written before this second renumber
|
||
(review 001/002/003, the wave-3..6 task files): their "wave 7"
|
||
references to the release pass mean what is now wave 8; per the
|
||
family's renumber-annotation precedent (`tasks/review-wave-5.md`'s
|
||
ADR-024 annotations), affected records get swept (or annotated) when
|
||
wave 8 decomposes — history is not rewritten in place.
|
||
|
||
## Decided points
|
||
|
||
- **Contract-suite layout — option (a)**: a small internal
|
||
`alkstore-contract-suite` crate (not published) exposing property
|
||
tests parameterized over a `Store` factory; each engine crate takes
|
||
it as a dev-dependency. One normative owner per property, mirroring
|
||
ADR-012 §2's one-owner rule. This discharges ADR-017 §4.2's
|
||
"decided at implementation" deferral; recorded as ADR-022 by the
|
||
scaffold task.
|
||
- **Engine tests vs. contract suite**: each engine wave carries its own
|
||
mechanism tests (does the engine work); wave 5 carries the
|
||
cross-engine *equivalence* properties (do the engines agree). The
|
||
verification-backlog rows are mostly equivalence-shaped, so this
|
||
split keeps wave 5 from re-testing engine internals.
|
||
- **CI**: none, deliberately. CI and publishing are run manually
|
||
(self-hosted Gitea; supply-chain posture). No CI-wiring task exists;
|
||
the merge gates (`cargo test`, `cargo clippy --all-targets --
|
||
-D warnings`, `cargo fmt --check`) are run coordinator-side.
|
||
- **Mem engine** (ADR-001 §4): **decided 2026-10-10, pre-decomposition
|
||
([ADR-024](../architecture/decisions/024-mem-engine.md))** — a
|
||
separate `alkstore-mem` crate, full contract v1, honest-ephemeral
|
||
posture, wasm32 compile-clean acceptance; wave 6, with release
|
||
readiness renumbered to wave 7 (later 8 — see the fuzzing decision
|
||
below). The deferral discharged: its
|
||
collapse condition ("implementation time") was reached early via
|
||
the consumer pause + the wasm/sandbox motivation, with the row-first
|
||
gate satisfied by the consumer inventory's engine-tier row
|
||
(operator-authority, 2026-10-10).
|
||
- **Fuzzing adoption** (the alkcall/alktunnels/alksocks/alktty
|
||
pattern): **decided 2026-10-10, before wave 8 decomposed — adopted,
|
||
inserted as wave 7** (release readiness renumbered to wave 8). The
|
||
deferral is discharged: the rationale, target inventory, invariants,
|
||
and campaign posture live in `docs/research/fuzzing.md`; the
|
||
decomposition resolves the named design points (parser exposure,
|
||
gate wiring, publish exclusion). AGENTS.md's no-fuzzing posture line
|
||
flipped at adoption; the corpus-replay gate joins the verification
|
||
list when the wave lands.
|
||
|
||
## Review gates
|
||
|
||
Each wave ends in a review task (`review-wave-N`) before the next wave
|
||
decomposes. Specific gates:
|
||
|
||
- **Wave 1 review** — the trait surface is versioned contract surface
|
||
from the first release (ADR-017); a shape error found here is cheap,
|
||
found in wave 5 it is a migration. Review checks the code against the
|
||
pinned ADR text line by line.
|
||
- **Wave 2 review** — diff reviewability against the honker lineage
|
||
(ADR-012 §3's fidelity posture), provenance register completeness
|
||
(ADR-018), floor tests green.
|
||
- **Wave 3/4 reviews** — engine-vs-contract conformance; the backlog
|
||
columns each engine owns.
|
||
- **Wave 5 review** — the suite as compatibility instrument: every
|
||
backlog row present, version-stamped, green on both engines; this is
|
||
the gate that flips the engine specs to `stable`.
|
||
- **Wave 6 review** — mem-engine-vs-contract conformance (the
|
||
born-pinned gate): the suite's third column green on the host, the
|
||
wasm32 compile gate clean, the current-thread-flavor column run
|
||
(harness pre-task disposed), the engine's own mechanism tests, and
|
||
the `engine-mem.md` spec read against the pinned ADR-024 text —
|
||
the gate that flips the mem spec toward `stable`.
|
||
- **Wave 7 review** — the fuzz wave's gate: the targets' campaigns run
|
||
clean via the detached runner (or their findings fixed and promoted
|
||
to seeds), the corpus-replay gate green on stable, the seeds
|
||
committed per the corpus policy, the harness-model findings
|
||
(if any) documented in the research doc's implementation log, and
|
||
the named design points (parser exposure, publish exclusion)
|
||
audited as resolved.
|
||
|
||
## Review rounds so far
|
||
|
||
- **Wave 1 review gate** (`review-wave-1`) — trait surface vs pinned
|
||
ADR text; validation coverage fixes landed.
|
||
- **Wave 2 review gate** (`review-wave-2`) — lineage diff clean, one
|
||
re-derivation defect found and fixed (D-27).
|
||
- **General review, waves 1–2** (2026-10-08,
|
||
`docs/reviews/001-waves-1-2-general-review.md`) — M-1 fixed
|
||
inline (sweep savepoint scope); M-2/N-2/N-4/N-6 resolved as ADR-023
|
||
pre-decomposition; lint removal + suite adds folded into waves 3/5;
|
||
N-1 recorded for release readiness (now wave 8).
|
||
- **Wave 3 review gate** (`review-wave-3`) — engine-vs-contract
|
||
conformance code-read clean; two findings fixed inline (the
|
||
`open_writer_connection` boundary move out of `substrate/mod.rs`;
|
||
the writer-slot error-arm stranding in
|
||
`with_writer`/`begin`/`commit`); two minor notes deferred to
|
||
wave 5/6 (reader-pool close path, commit-error-arm coverage).
|
||
- **Wave 4 review gate** (`review-wave-4`) — engine-vs-contract
|
||
conformance code-read clean (0 findings); the flagged
|
||
`tx_publishes_compose_with_the_handle` flake reproduced twice and
|
||
recorded as F-1 (test-infra, wake-subscription under load — root
|
||
cause unresolved at gate time; candidate dispositions in the task's
|
||
Notes for wave 5's suite hardening); three no-action notes
|
||
(F-2..F-4). F-1's root cause was later found engine-side by the
|
||
general review below and retired by `pg-fix-tx-wake`.
|
||
- **General review, wave 4** (2026-10-09,
|
||
`docs/reviews/002-wave-4-general-review.md`) — two live-proven
|
||
consumer-facing bugs (the forwarder's permanent death after one
|
||
failed reconnect — an untested failure arm the gate's test never
|
||
covered; the tx enqueue/publish paths' missing `pg_notify` wake —
|
||
F-1's root cause, engine-side), a `max_size: 0` open-hang, two
|
||
narrow robustness gaps, doc mismatches, decode-duplication smells.
|
||
Fixes recommended before wave 5 decomposes. The tx-wake fix landed
|
||
(`pg-fix-tx-wake`, 2026-10-09): the tx producer paths now wake
|
||
commit-atomically, retiring F-1's engine arm (record updated in
|
||
`tasks/review-wave-4.md`) and making `tx_publishes_compose_with_the_handle`
|
||
deterministic (20 consecutive solo runs green); the remaining fixes
|
||
ride the wave-4 fix-batch decomposition below.
|
||
- **Wave 5 review gate** (`review-wave-5`, 2026-10-10) — the suite as
|
||
compatibility instrument, gate passed: the backlog discharge map
|
||
verified row-by-row against the pinning tests (all 25 mechanism rows
|
||
per column stamped, wiring confirmed in both engines' suite targets;
|
||
the engine-side pins — SQLite open row, watcher-cadence knob, M-1
|
||
sweep-rollback (SQLite trigger seam + the pg frame's code-read),
|
||
beyond-64 skip-forward, cap-curve — read and confirmed); stamps
|
||
cross-checked against the cited ADR § texts (the
|
||
`enqueue_opts_resolution` amendment accumulation rides ADR-023 §2 per
|
||
the convention); parked dispositions audited (all six recorded;
|
||
scheduler fire-wake parity left as-is, the lock busy-path answered
|
||
with no divergence); green on both engines (SQLite server-less; pg
|
||
vs the harness server, three full runs incl. one concurrent with the
|
||
SQLite column, plus focused stress). Engine specs flipped to
|
||
`stable`. Two unreproducible pg row failures from development were
|
||
recorded in the task's Notes (both rows' assertions are
|
||
state-outcome-shaped; flagged for watch, not blocking the gate) —
|
||
see `tasks/review-wave-5.md` §Notes.
|
||
- **Wave 5 decomposition** (2026-10-10) — audit-first: the engines'
|
||
backlog columns already discharge most of the §Verification backlog
|
||
(map recorded in `tasks/review-wave-5.md`'s appendix for the gate to
|
||
verify); the wave fills the gaps with seven suite-row tasks grouped
|
||
by mechanism, two engine-side hardening tasks, and the gate. See the
|
||
Wave 5 section.
|
||
- **Wave 4 decomposition** (2026-10-08) — shaped as wave 3's
|
||
structural twin; wave-3 outcomes absorbed (constructors exist;
|
||
arithmetic re-owned per engine; per-engine `@every` parser;
|
||
SQLite-scoped deferred notes stay put). See the Wave 4 section.
|
||
- **Wave 6 review gate** (`review-wave-6`, 2026-10-10) — the
|
||
born-pinned gate passed: the conformance read clean (queue state
|
||
machine, scheduler collapse, outbox, streams, locks, clock,
|
||
validation, tx overlay, wake substrate each verified against the
|
||
contract and the pinned ADRs; guard-order discipline audited across
|
||
every section-guard site); the suite's third column read row-by-row
|
||
and green (the 24 engine-shared rows + mem's
|
||
`payload_too_large_never_produced_on_mem` arm + the factory-shape
|
||
test, 25 green under the per-test current-thread flavor);
|
||
three-way arithmetic equivalence green (backoff curve, opts
|
||
resolution on all three columns); the wasm32 acceptance items
|
||
verified (compile-clean, tokio sync+time subset, no blocking calls,
|
||
flavor enforced); the rider's arms verified (Finding 1 discharged;
|
||
Findings 2–3 confirmed still parked for the release pass — wave 8
|
||
since this renumber). Three edge-domain
|
||
fixes landed inline at the gate: the stream cursor read's `i64::MAX`
|
||
overflow (`Bound::Excluded`), `trim_to`'s `i64::MAX` horizon
|
||
endpoint (`Bound::Included`), and the scheduler register path
|
||
aligning with SQLite on rejecting a parseable interval whose first
|
||
boundary overflows (`InvalidSpec`). Unpinned contract-adjacent
|
||
observations recorded in the task's Notes for the pre-release pass
|
||
(wave 8 since this renumber): the multi-thread publish-to-wake
|
||
ordering tolerance (same
|
||
shape implied for the SQL engines; nothing pins wake-fire order vs
|
||
offset order), the `heartbeat(extend)`/`save_offset`/
|
||
`EventReceiver::try_recv`-closed classifications missing from the
|
||
ADR-023 §2 table, and the mem arithmetic being a transplant of the
|
||
SQLite implementations (equivalence suite-pinned; ownership letter
|
||
satisfied). `engine-mem.md` flipped to `stable`.
|
||
- **Wave-6 decomposition** (2026-10-10) — ADR-024 adopted the mem
|
||
engine the same day the wave-5 gate closed; the task set follows the
|
||
pg wave's rhythm with the tx overlay as the load-bearing task, the
|
||
suite's third column as the born-pinned deliverable, the harness's
|
||
async-sleep pre-task, and review 003 Finding 1 riding the window.
|
||
See the Wave 6 section.
|
||
- **Wave-5 residue decomposition** (2026-10-10) — the wave-5 review's
|
||
outstanding items audited post-gate: the one recorded "for a later
|
||
task" gap (scheduler fire-wake latency parity) decomposed as
|
||
`pg-fix-scheduler-fire-wake` (pre-wave-6, standalone); the flake-watch
|
||
flag carried into the release-readiness section above (now wave 8);
|
||
review 003's Finding 1
|
||
already decomposed (`pg-fix-scheduler-tick-seam`, the wave-6 rider)
|
||
and Findings 2–3 already parked to the release pass. See the
|
||
Wave 8 section for
|
||
the watch carriage.
|
||
- **Wave-5 residue closure** (2026-10-10) — the remaining
|
||
"recorded for a later task" item landed: `pg-fix-scheduler-fire-wake`
|
||
closed the scheduler fire-wake latency-parity gap (the tick's
|
||
in-tx fires now issue their commit-atomic `pg_notify` wake, one
|
||
coalesced wake per firing schedule per tick); the three gap records
|
||
(`review-wave-4-fixes`'s recorded-for-later note, `suite-scheduler-rows`'s
|
||
disposition note + Notes bullet, `review-wave-5` §Notes 4's audit row)
|
||
retired with pointers, keeping the mem engine's fire posture written
|
||
against a closed gap before wave 6.
|
||
- **Wave-7 decomposition** (2026-10-10) — the fuzzing decision's
|
||
task set: scaffold → exposure (§3.2 resolved as option (b) with the
|
||
(a) companion) → three targets in parallel → campaigns + gate
|
||
wiring (the AGENTS.md flip) → review gate. Seven tasks; the
|
||
every-parse differential is the load-bearing one (the wave-6
|
||
finding class). See the Wave 7 section.
|
||
- **Wave-4 fix-batch decomposition** (2026-10-09) — the general
|
||
review's findings decomposed into seven `pg-fix-*` tasks + a
|
||
`review-wave-4-fixes` gate: Finding 1 → `pg-fix-forwarder-reconnect`
|
||
(with the failed-connect test seam the gate's blind spot demands),
|
||
Finding 2 → `pg-fix-tx-wake` (retires F-1's engine arm), Finding 3
|
||
+ the DSN-options note → `pg-fix-open-path`, Finding 4 →
|
||
`pg-fix-stale-unlisten`, Finding 5 + the scheduler doc fix →
|
||
`pg-fix-scheduler-resilience`, the decode-duplication smell + small
|
||
cleanups → `pg-fix-dedupe-cleanup`, the TLS/QueueOpts doc
|
||
alignments → `pg-fix-docs-alignment`. Sequencing per the review's
|
||
recommendation: **wave 5 decomposes after the two HIGH fixes land**
|
||
(the suite's wake-driven rows would otherwise inherit a hang-shaped
|
||
false failure); 3–7 may trail into wave 5's window if the
|
||
fix-batch gate judges them wake-independent. |