368 lines
21 KiB
Markdown
368 lines
21 KiB
Markdown
---
|
||
status: draft
|
||
last_updated: 2026-10-10 (wave 5 implemented and reviewed — review-wave-5 verified the discharge map, stamps, dispositions and green-on-both-engines; engine specs flipped to stable)
|
||
---
|
||
|
||
# alkstore — Implementation plan
|
||
|
||
Wave-based decomposition of the architecture
|
||
(`docs/architecture/`) into units of implementable work. This is a
|
||
deliberate deviation from the SDD process's decompose-everything-upfront
|
||
step: the architecture is large, early waves change the shape of later
|
||
ones (the fork's outcomes feed the SQLite engine tasks; the contract
|
||
suite's harness shape feeds both engines' verification work), and
|
||
decomposing only the next wave or two at a time keeps each session's
|
||
task set reviewable and lets later decompositions absorb earlier waves'
|
||
course corrections.
|
||
|
||
Rhythm: decompose a wave → implement it → review gate → decompose the
|
||
next wave. Task files live in `tasks/` (taskgraph-managed; frontmatter
|
||
carries the categorical estimates). Wave boundaries are also review
|
||
boundaries.
|
||
|
||
## The waves
|
||
|
||
Dependency logic in one line: core → (substrate fork ∥ postgres
|
||
engine) → sqlite engine → contract suite → release readiness. The
|
||
substrate fork is contract-blind (ADR-012 §2), so it needs only the
|
||
workspace scaffold from wave 1; the Postgres engine needs only the core
|
||
crate. Waves 2 and 4 are therefore independent of each other and could
|
||
run in either order (or in parallel, if agents are ever available in
|
||
parallel).
|
||
|
||
| Wave | Contents | Depends on | Status |
|
||
|---|---|---|---|
|
||
| [1](#wave-1--foundations) | Workspace scaffold; core crate (errors, value types, full trait surface); contract-suite scaffold | — | implemented + reviewed (2026-10-08) |
|
||
| [2](#wave-2--sqlite-substrate-fork) | honker-core fork into `alkstore-sqlite/src/substrate/`: port, deltas, provenance, test floor | wave 1 (workspace scaffold only) | implemented + reviewed (2026-10-08) |
|
||
| [3](#wave-3--sqlite-engine) | SQLite engine: connection architecture, re-derived queue ops on contract v1, scheduler/outbox, tx seam, SQLite backlog column | waves 1 + 2 | implemented + reviewed (2026-10-08) |
|
||
| 4 | [Postgres engine](#wave-4--postgres-engine): schema bootstrap, pool/open, listener/forwarder, all mechanisms, tx seam, pg backlog column | wave 1 | implemented + reviewed (2026-10-08) |
|
||
| 5 | [Contract suite](#wave-5--contract-suite): the cross-engine equivalence properties (core-contract.md §Verification backlog), version-stamped per ADR-017 | waves 3 + 4 | implemented + reviewed (2026-10-10) |
|
||
| 6 | Release readiness: crate docs, deployment matrix final pass, README (written last, honestly), publish prep; mem-engine and fuzzing decisions | wave 5 | not yet decomposed |
|
||
|
||
## Wave 1 — Foundations
|
||
|
||
The core crate is contract v1 in code: the error taxonomy (ADR-008 §5),
|
||
the value types (ADR-019 §3, ADR-020), the full trait surface
|
||
(ADR-008 §1–§3/§8, ADR-014, ADR-019, ADR-021), and the payload encoding
|
||
posture (ADR-020 §4). Nothing engine-specific lives here — no
|
||
resolution arithmetic, no SQL. The equal-jitter curve, opts-stamping
|
||
resolution, and boundary math are deliberately *not* core: ADR-012 §2
|
||
pins each engine as the owner of one implementation, with equivalence
|
||
pinned by the contract suite (wave 5).
|
||
|
||
The contract suite gets scaffolded now (not in wave 5) because its
|
||
harness shape — a `Store`-factory-parameterized property crate — is
|
||
easier to grow row by row as engines land than to retrofit onto two
|
||
finished engines. Wave 5 fills it with the cross-engine equivalence
|
||
rows; waves 3 and 4 adopt the harness for their own backlog columns.
|
||
|
||
## Wave 2 — SQLite substrate fork
|
||
|
||
The fork per ADR-011/012/013: port honker-core at `f4e53c6` into
|
||
`alkstore-sqlite/src/substrate/`, apply the three watcher port deltas
|
||
and the bootstrap re-keying, drop cron/experimental/cut-flag machinery,
|
||
re-own the table family as `__alkstore_*`, carry `PROVENANCE.md` and
|
||
the dual-license notice in-tree (ADR-018), and stand up the inherited
|
||
test suites as the floor. The substrate stays sync and contract-blind;
|
||
the engine layer that maps it onto the core contract is wave 3, not
|
||
here. Reviewability against the lineage (ADR-012 §3) is a property the
|
||
wave-2 review gate checks explicitly.
|
||
|
||
## Wave 3 — SQLite engine
|
||
|
||
The engine layer that maps the forked substrate onto the core
|
||
contract: the `open` constructor (connection architecture — writer
|
||
slot, reader pool, watcher spawn — per engine-sqlite.md), the
|
||
`spawn_blocking` seam, the full `Store`/`TxHandle`/mechanism-handle
|
||
trait impls over the substrate's ops, the scheduler leader loop and
|
||
outbox helper, and the engine's backlog column in the contract suite
|
||
(the ADR-023 rows among them, factory-parameterized so wave 5 runs
|
||
them against both engines).
|
||
|
||
The waves-1–2 general review
|
||
(`docs/reviews/001-waves-1-2-general-review.md`) and its
|
||
resolution ([ADR-023](../architecture/decisions/023-fourth-review-round.md))
|
||
shaped this wave's task set:
|
||
|
||
- **Already landed pre-decomposition** (commit `44637ee`, not wave-3
|
||
tasks): `encode_payload` is fallible (`Result<Vec<u8>, Error::Codec>`,
|
||
ADR-023 §1); `open_conn` drops `SQLITE_OPEN_URI` (§3, register
|
||
D-29); the numeric-argument domain table is pinned in
|
||
core-contract.md (§2); the 1 ms watcher default stands with the
|
||
cadence documented in deployment.md (§4).
|
||
- **Folded into wave 3's tasks** (the review's §7 wiring items): the
|
||
trait-impl extent/duration guards (the contract-side domain rule,
|
||
enforced at the engine's trait-impl entry — the `sqlite-engine-*`
|
||
tasks carry it per mechanism), the two `#![allow]` lint removals in
|
||
`substrate/mod.rs` (the integration task — they can only lift once
|
||
every substrate surface is wired), and `SqliteOpts::poll_interval`
|
||
wiring (the constructor task).
|
||
- **Staying put as ordered**: M-1's retention-failure test and N-5's
|
||
panic probe → wave 5's contract-suite rows; N-1's growth-posture
|
||
review → wave 6.
|
||
|
||
## Wave 4 — Postgres engine
|
||
|
||
The second engine: the core contract implemented on tokio-postgres +
|
||
deadpool-postgres per engine-postgres.md — schema bootstrap (one
|
||
engine-owned schema, default `alkstore`, ADR-010 §8), the pool/open
|
||
constructor, the hand-rolled LISTEN forwarder (dedicated non-pooled
|
||
listener connection, bounded broadcast fanout, exponential-backoff
|
||
reconnect with re-LISTEN, the synthetic reconnect-wake on
|
||
`__alkstore_listener_reconnected__`), the full `Store`/`TxHandle`/
|
||
mechanism-handle trait impls (natively async — no `spawn_blocking`
|
||
seam; the tx handle holds the pooled object directly, ADR-007's pg
|
||
arm), the re-derived queue machinery (`FOR UPDATE SKIP LOCKED` claims,
|
||
LISTEN-driven wake with re-poll safety net), the scheduler leader loop
|
||
and outbox helper, and the engine's backlog column in the contract
|
||
suite (factory-parameterized; wave 5 runs the same rows against both
|
||
engines).
|
||
|
||
The wave is deliberately shaped as wave 3's structural twin — same
|
||
task rhythm (constructor → seam → mechanisms in parallel →
|
||
scheduler/outbox → integration → review gate) — because the two
|
||
engines implement the same contract and the wave-3 task set's
|
||
boundaries proved clean. The deltas are the engine's own: no
|
||
substrate (the pg machinery is re-derived greenfield per ADR-004/005,
|
||
with the pg-boss schema family as design reference), a natively-async
|
||
seam (Send+Sync client, POC #2 compile-probe), the forwarder as the
|
||
wake substrate (with its two POC-pinned deadlock pitfalls owned as
|
||
test-pinned failure modes), and the `PayloadTooLarge` client-side
|
||
check (this engine *produces* the variant — the one runtime carriage
|
||
of an engine asymmetry, ADR-016 §5).
|
||
|
||
Wave-3 outcomes absorbed into the task set:
|
||
|
||
- **The `#[doc(hidden)]` constructors** (`Job::from_row`,
|
||
`StreamEvent::from_row`, `Schedule::new`, `Wake::new`) already
|
||
exist — wave 3's pre-work task built them engine-agnostically; wave
|
||
4 consumes them as-is (no wave-4 equivalent needed).
|
||
- **The engine-side arithmetic is re-owned, not shared**: the
|
||
equal-jitter backoff curve, the opts-resolution rules
|
||
(delay-over-`run_at`, relative-`expires`, the derived stamp sets
|
||
300/3/5/none and 60/5/5), and the outbox backing-queue derivation
|
||
are each engine's own implementation (ADR-012 §2's one-owner rule
|
||
applied per engine); the pg tasks re-implement them from the ADR
|
||
text, and the contract suite (wave 5) pins the two engines'
|
||
arithmetic to identical outputs. The SQLite engine's `resolution.rs`
|
||
is reference, not a dependency.
|
||
- **The `@every` spec grammar has no shared parser on this side** —
|
||
the SQLite engine reuses the substrate's `parse_every_interval`
|
||
(one-line delta D-31's neighbor); the pg engine owns its parser
|
||
(the grammar is small and pinned: `@every <n><unit>`, `s|m|h|d`).
|
||
Equivalence of *behavior* (what fires, when) is suite-pinned; the
|
||
parsers themselves are per-engine.
|
||
- **The wave-3 review's deferred notes** (reader-pool close path,
|
||
commit-error-arm coverage) stay SQLite-scoped — the pg engine's
|
||
pool is deadpool-managed (no hand-rolled close path to review) and
|
||
its commit-error arm gets coverage in the pg tasks' own tests.
|
||
- **The suite's factory contract** (isolation + idempotent teardown)
|
||
is what the pg factory implements — a fresh schema per `open` per
|
||
the SQLite factory's precedent (fresh temp file there); the
|
||
POC's shared-server parallel-interference caveat (findings,
|
||
invocation note) is answered by exactly that isolation, not by
|
||
sequential-only harnesses.
|
||
|
||
Test posture: the engine's tests need a live Postgres server. The
|
||
harness convention is the POC's (dockerized `postgres:16-alpine` on
|
||
:15432, the `pglo-poc` container — still running in the dev
|
||
environment); connection settings ride environment/config, never
|
||
hardcoded. Tests that cannot reach a server skip cleanly (the
|
||
`SKIP`-posture), so the workspace gates stay green server-less — but
|
||
the wave's acceptance requires the suite green *against* the server.
|
||
|
||
One design point the architecture leaves to implementation (pinned in
|
||
the wave-4 tasks, flagged here for visibility): **the queue/stream
|
||
wake-channel naming**. The contract pins LISTEN as the pg wake
|
||
trigger and pins the channels/queues shared namespace, but no doc
|
||
names the channel an enqueue/publish notifies on. The tasks pin the
|
||
natural realization — the mechanism name *is* the wake channel
|
||
(`listen(queue_name)` receives that queue's enqueue wakes; same for
|
||
streams), legal by construction under the shared namespace — with
|
||
the reserved reconnect-wake channel staying the only reserved string.
|
||
|
||
## Wave 5 — Contract suite
|
||
|
||
Filling the scaffolded suite (`alkstore-contract-suite`) with the
|
||
remaining verification-backlog rows and running the compatibility
|
||
gate. Waves 3 and 4 each landed a nine/ten-row backlog column; the
|
||
wave-5 decomposition started from an audit of what those columns
|
||
already discharge versus what the backlog still lacks.
|
||
|
||
**Already discharged by the engines' columns** (the wave-5 review
|
||
gate verifies each claim by reading the pinning row, not by trusting
|
||
the audit): name validation (the exemplar, incl. `schedule()`'s
|
||
queue-argument validation), the numeric-domain rows (extents +
|
||
durations), `encode_payload` typed failure + round-trip, the
|
||
`PayloadTooLarge` occurrence asymmetry (both engine-scoped arms),
|
||
drop-=-rollback no-ghosts, in-tx read-your-own-writes (which also
|
||
discharges the `save_offset_tx` exactly-once backlog row across
|
||
itself and the drop-rollback row), receiver close/save arms + monotone
|
||
composition (both columns), and — pinned engine-side rather than as
|
||
suite rows, per the sqlite-engine-integration task's scoping call —
|
||
plain-path SQLite open and the watcher-cadence knob
|
||
(`open_tests.rs`; SQLite-scoped backlog rows).
|
||
|
||
**Wave 5's task set** (the gaps):
|
||
|
||
- Queue depth: the job-handle validity predicate, the ADR-010 depth
|
||
properties (reclaim-eats-attempt, dead-letter moves, the
|
||
no-stranded-rows sweep), and M-1's retention-failure pin
|
||
(`suite-queue-depth-rows`).
|
||
- Scheduler: boundary fires, bounded catch-up, leadership discipline —
|
||
with a documented extension of the suite's determinism posture to
|
||
admit runner-driving rows (`suite-scheduler-rows`).
|
||
- Tx seam: `outbox_enqueue_tx` and `publish_with_key_tx`
|
||
commit-atomicity, plus N-5's `with_tx` panic-disposition probe —
|
||
the panic path pinned against real engines for the first time
|
||
(`suite-tx-commit-atomicity-rows`).
|
||
- Streams: the cross-engine ordering-equivalence row and the full
|
||
`trim_to` semantics row (`suite-stream-rows`).
|
||
- Locks: TTL/expiry re-acquisition and the concurrent-`try_lock`
|
||
loser posture — the SQLite busy-path is the open question the row
|
||
exists to answer (`suite-lock-rows`).
|
||
- Wakes: the pinned `WakeReceiver` shapes, tolerance-bounded
|
||
(`suite-wake-rows`).
|
||
- Arithmetic equivalence: the backoff-curve row and the two
|
||
wall-clock-adjacent opts-resolution legs the existing row deferred
|
||
to this wave (`suite-opts-backoff-rows`).
|
||
- Engine-side hardening riding the window: the wave-3 review's
|
||
deferred SQLite commit-error-arm coverage (`sqlite-commit-error-arm`)
|
||
and F-1's defense-in-depth candidates — explicitly not load-bearing
|
||
since `pg-fix-tx-wake` retired the flake's root cause
|
||
(`pg-suite-infra-hardening`).
|
||
- The review gate (`review-wave-5`): the backlog discharge audit,
|
||
stamp verification, green-on-both-engines, the parked-disposition
|
||
audit — and the gate that flips `engine-sqlite.md` /
|
||
`engine-postgres.md` to `stable`.
|
||
|
||
Judgment calls the decomposition parked in task Notes (the review
|
||
gate audits each): where the M-1 retention-failure pin, the
|
||
beyond-64-cap skip-forward leg, and the backoff 1-hour-cap leg land
|
||
when the public surface cannot express them (engine-side is the
|
||
expected answer, recorded per case); whether the scheduler rows
|
||
demand the pg fire-wake parity the wave-4 fix-batch gate recorded;
|
||
and what the SQLite lock busy-path actually returns under contention.
|
||
|
||
## Decided points
|
||
|
||
- **Contract-suite layout — option (a)**: a small internal
|
||
`alkstore-contract-suite` crate (not published) exposing property
|
||
tests parameterized over a `Store` factory; each engine crate takes
|
||
it as a dev-dependency. One normative owner per property, mirroring
|
||
ADR-012 §2's one-owner rule. This discharges ADR-017 §4.2's
|
||
"decided at implementation" deferral; recorded as ADR-022 by the
|
||
scaffold task.
|
||
- **Engine tests vs. contract suite**: each engine wave carries its own
|
||
mechanism tests (does the engine work); wave 5 carries the
|
||
cross-engine *equivalence* properties (do the engines agree). The
|
||
verification-backlog rows are mostly equivalence-shaped, so this
|
||
split keeps wave 5 from re-testing engine internals.
|
||
- **CI**: none, deliberately. CI and publishing are run manually
|
||
(self-hosted Gitea; supply-chain posture). No CI-wiring task exists;
|
||
the merge gates (`cargo test`, `cargo clippy --all-targets --
|
||
-D warnings`, `cargo fmt --check`) are run coordinator-side.
|
||
- **Mem engine** (ADR-001 §4) and **fuzzing adoption** (the
|
||
alksocks/alktty/alktunnels pattern): both are "decided at
|
||
implementation" deferrals, recorded here so they surface as explicit
|
||
decision points in wave 6 (or earlier if the test story demands the
|
||
mem engine sooner) rather than ambushing a later session.
|
||
|
||
## Review gates
|
||
|
||
Each wave ends in a review task (`review-wave-N`) before the next wave
|
||
decomposes. Specific gates:
|
||
|
||
- **Wave 1 review** — the trait surface is versioned contract surface
|
||
from the first release (ADR-017); a shape error found here is cheap,
|
||
found in wave 5 it is a migration. Review checks the code against the
|
||
pinned ADR text line by line.
|
||
- **Wave 2 review** — diff reviewability against the honker lineage
|
||
(ADR-012 §3's fidelity posture), provenance register completeness
|
||
(ADR-018), floor tests green.
|
||
- **Wave 3/4 reviews** — engine-vs-contract conformance; the backlog
|
||
columns each engine owns.
|
||
- **Wave 5 review** — the suite as compatibility instrument: every
|
||
backlog row present, version-stamped, green on both engines; this is
|
||
the gate that flips the engine specs to `stable`.
|
||
|
||
## Review rounds so far
|
||
|
||
- **Wave 1 review gate** (`review-wave-1`) — trait surface vs pinned
|
||
ADR text; validation coverage fixes landed.
|
||
- **Wave 2 review gate** (`review-wave-2`) — lineage diff clean, one
|
||
re-derivation defect found and fixed (D-27).
|
||
- **General review, waves 1–2** (2026-10-08,
|
||
`docs/reviews/001-waves-1-2-general-review.md`) — M-1 fixed
|
||
inline (sweep savepoint scope); M-2/N-2/N-4/N-6 resolved as ADR-023
|
||
pre-decomposition; lint removal + suite adds folded into waves 3/5;
|
||
N-1 recorded for wave 6.
|
||
- **Wave 3 review gate** (`review-wave-3`) — engine-vs-contract
|
||
conformance code-read clean; two findings fixed inline (the
|
||
`open_writer_connection` boundary move out of `substrate/mod.rs`;
|
||
the writer-slot error-arm stranding in
|
||
`with_writer`/`begin`/`commit`); two minor notes deferred to
|
||
wave 5/6 (reader-pool close path, commit-error-arm coverage).
|
||
- **Wave 4 review gate** (`review-wave-4`) — engine-vs-contract
|
||
conformance code-read clean (0 findings); the flagged
|
||
`tx_publishes_compose_with_the_handle` flake reproduced twice and
|
||
recorded as F-1 (test-infra, wake-subscription under load — root
|
||
cause unresolved at gate time; candidate dispositions in the task's
|
||
Notes for wave 5's suite hardening); three no-action notes
|
||
(F-2..F-4). F-1's root cause was later found engine-side by the
|
||
general review below and retired by `pg-fix-tx-wake`.
|
||
- **General review, wave 4** (2026-10-09,
|
||
`docs/reviews/002-wave-4-general-review.md`) — two live-proven
|
||
consumer-facing bugs (the forwarder's permanent death after one
|
||
failed reconnect — an untested failure arm the gate's test never
|
||
covered; the tx enqueue/publish paths' missing `pg_notify` wake —
|
||
F-1's root cause, engine-side), a `max_size: 0` open-hang, two
|
||
narrow robustness gaps, doc mismatches, decode-duplication smells.
|
||
Fixes recommended before wave 5 decomposes. The tx-wake fix landed
|
||
(`pg-fix-tx-wake`, 2026-10-09): the tx producer paths now wake
|
||
commit-atomically, retiring F-1's engine arm (record updated in
|
||
`tasks/review-wave-4.md`) and making `tx_publishes_compose_with_the_handle`
|
||
deterministic (20 consecutive solo runs green); the remaining fixes
|
||
ride the wave-4 fix-batch decomposition below.
|
||
- **Wave 5 review gate** (`review-wave-5`, 2026-10-10) — the suite as
|
||
compatibility instrument, gate passed: the backlog discharge map
|
||
verified row-by-row against the pinning tests (all 25 mechanism rows
|
||
per column stamped, wiring confirmed in both engines' suite targets;
|
||
the engine-side pins — SQLite open row, watcher-cadence knob, M-1
|
||
sweep-rollback (SQLite trigger seam + the pg frame's code-read),
|
||
beyond-64 skip-forward, cap-curve — read and confirmed); stamps
|
||
cross-checked against the cited ADR § texts (the
|
||
`enqueue_opts_resolution` amendment accumulation rides ADR-023 §2 per
|
||
the convention); parked dispositions audited (all six recorded;
|
||
scheduler fire-wake parity left as-is, the lock busy-path answered
|
||
with no divergence); green on both engines (SQLite server-less; pg
|
||
vs the harness server, three full runs incl. one concurrent with the
|
||
SQLite column, plus focused stress). Engine specs flipped to
|
||
`stable`. Two unreproducible pg row failures from development were
|
||
recorded in the task's Notes (both rows' assertions are
|
||
state-outcome-shaped; flagged for watch, not blocking the gate) —
|
||
see `tasks/review-wave-5.md` §Notes.
|
||
- **Wave 5 decomposition** (2026-10-10) — audit-first: the engines'
|
||
backlog columns already discharge most of the §Verification backlog
|
||
(map recorded in `tasks/review-wave-5.md`'s appendix for the gate to
|
||
verify); the wave fills the gaps with seven suite-row tasks grouped
|
||
by mechanism, two engine-side hardening tasks, and the gate. See the
|
||
Wave 5 section.
|
||
- **Wave 4 decomposition** (2026-10-08) — shaped as wave 3's
|
||
structural twin; wave-3 outcomes absorbed (constructors exist;
|
||
arithmetic re-owned per engine; per-engine `@every` parser;
|
||
SQLite-scoped deferred notes stay put). See the Wave 4 section.
|
||
- **Wave-4 fix-batch decomposition** (2026-10-09) — the general
|
||
review's findings decomposed into seven `pg-fix-*` tasks + a
|
||
`review-wave-4-fixes` gate: Finding 1 → `pg-fix-forwarder-reconnect`
|
||
(with the failed-connect test seam the gate's blind spot demands),
|
||
Finding 2 → `pg-fix-tx-wake` (retires F-1's engine arm), Finding 3
|
||
+ the DSN-options note → `pg-fix-open-path`, Finding 4 →
|
||
`pg-fix-stale-unlisten`, Finding 5 + the scheduler doc fix →
|
||
`pg-fix-scheduler-resilience`, the decode-duplication smell + small
|
||
cleanups → `pg-fix-dedupe-cleanup`, the TLS/QueueOpts doc
|
||
alignments → `pg-fix-docs-alignment`. Sequencing per the review's
|
||
recommendation: **wave 5 decomposes after the two HIGH fixes land**
|
||
(the suite's wake-driven rows would otherwise inherit a hang-shaped
|
||
false failure); 3–7 may trail into wave 5's window if the
|
||
fix-batch gate judges them wake-independent. |