281 lines
16 KiB
Markdown
281 lines
16 KiB
Markdown
---
|
||
status: draft
|
||
last_updated: 2026-10-09 (pg-fix-tx-wake landed — the second HIGH fix, retiring F-1's engine arm; wave 5's remaining gate is the fix batch's review gate)
|
||
---
|
||
|
||
# alkstore — Implementation plan
|
||
|
||
Wave-based decomposition of the architecture
|
||
(`docs/architecture/`) into units of implementable work. This is a
|
||
deliberate deviation from the SDD process's decompose-everything-upfront
|
||
step: the architecture is large, early waves change the shape of later
|
||
ones (the fork's outcomes feed the SQLite engine tasks; the contract
|
||
suite's harness shape feeds both engines' verification work), and
|
||
decomposing only the next wave or two at a time keeps each session's
|
||
task set reviewable and lets later decompositions absorb earlier waves'
|
||
course corrections.
|
||
|
||
Rhythm: decompose a wave → implement it → review gate → decompose the
|
||
next wave. Task files live in `tasks/` (taskgraph-managed; frontmatter
|
||
carries the categorical estimates). Wave boundaries are also review
|
||
boundaries.
|
||
|
||
## The waves
|
||
|
||
Dependency logic in one line: core → (substrate fork ∥ postgres
|
||
engine) → sqlite engine → contract suite → release readiness. The
|
||
substrate fork is contract-blind (ADR-012 §2), so it needs only the
|
||
workspace scaffold from wave 1; the Postgres engine needs only the core
|
||
crate. Waves 2 and 4 are therefore independent of each other and could
|
||
run in either order (or in parallel, if agents are ever available in
|
||
parallel).
|
||
|
||
| Wave | Contents | Depends on | Status |
|
||
|---|---|---|---|
|
||
| [1](#wave-1--foundations) | Workspace scaffold; core crate (errors, value types, full trait surface); contract-suite scaffold | — | implemented + reviewed (2026-10-08) |
|
||
| [2](#wave-2--sqlite-substrate-fork) | honker-core fork into `alkstore-sqlite/src/substrate/`: port, deltas, provenance, test floor | wave 1 (workspace scaffold only) | implemented + reviewed (2026-10-08) |
|
||
| [3](#wave-3--sqlite-engine) | SQLite engine: connection architecture, re-derived queue ops on contract v1, scheduler/outbox, tx seam, SQLite backlog column | waves 1 + 2 | implemented + reviewed (2026-10-08) |
|
||
| 4 | [Postgres engine](#wave-4--postgres-engine): schema bootstrap, pool/open, listener/forwarder, all mechanisms, tx seam, pg backlog column | wave 1 | implemented + reviewed (2026-10-08) |
|
||
| 5 | Contract suite: the cross-engine equivalence properties (core-contract.md §Verification backlog), version-stamped per ADR-017 | waves 3 + 4 | not yet decomposed |
|
||
| 6 | Release readiness: crate docs, deployment matrix final pass, README (written last, honestly), publish prep; mem-engine and fuzzing decisions | wave 5 | not yet decomposed |
|
||
|
||
## Wave 1 — Foundations
|
||
|
||
The core crate is contract v1 in code: the error taxonomy (ADR-008 §5),
|
||
the value types (ADR-019 §3, ADR-020), the full trait surface
|
||
(ADR-008 §1–§3/§8, ADR-014, ADR-019, ADR-021), and the payload encoding
|
||
posture (ADR-020 §4). Nothing engine-specific lives here — no
|
||
resolution arithmetic, no SQL. The equal-jitter curve, opts-stamping
|
||
resolution, and boundary math are deliberately *not* core: ADR-012 §2
|
||
pins each engine as the owner of one implementation, with equivalence
|
||
pinned by the contract suite (wave 5).
|
||
|
||
The contract suite gets scaffolded now (not in wave 5) because its
|
||
harness shape — a `Store`-factory-parameterized property crate — is
|
||
easier to grow row by row as engines land than to retrofit onto two
|
||
finished engines. Wave 5 fills it with the cross-engine equivalence
|
||
rows; waves 3 and 4 adopt the harness for their own backlog columns.
|
||
|
||
## Wave 2 — SQLite substrate fork
|
||
|
||
The fork per ADR-011/012/013: port honker-core at `f4e53c6` into
|
||
`alkstore-sqlite/src/substrate/`, apply the three watcher port deltas
|
||
and the bootstrap re-keying, drop cron/experimental/cut-flag machinery,
|
||
re-own the table family as `__alkstore_*`, carry `PROVENANCE.md` and
|
||
the dual-license notice in-tree (ADR-018), and stand up the inherited
|
||
test suites as the floor. The substrate stays sync and contract-blind;
|
||
the engine layer that maps it onto the core contract is wave 3, not
|
||
here. Reviewability against the lineage (ADR-012 §3) is a property the
|
||
wave-2 review gate checks explicitly.
|
||
|
||
## Wave 3 — SQLite engine
|
||
|
||
The engine layer that maps the forked substrate onto the core
|
||
contract: the `open` constructor (connection architecture — writer
|
||
slot, reader pool, watcher spawn — per engine-sqlite.md), the
|
||
`spawn_blocking` seam, the full `Store`/`TxHandle`/mechanism-handle
|
||
trait impls over the substrate's ops, the scheduler leader loop and
|
||
outbox helper, and the engine's backlog column in the contract suite
|
||
(the ADR-023 rows among them, factory-parameterized so wave 5 runs
|
||
them against both engines).
|
||
|
||
The waves-1–2 general review
|
||
(`docs/reviews/001-waves-1-2-general-review.md`) and its
|
||
resolution ([ADR-023](../architecture/decisions/023-fourth-review-round.md))
|
||
shaped this wave's task set:
|
||
|
||
- **Already landed pre-decomposition** (commit `44637ee`, not wave-3
|
||
tasks): `encode_payload` is fallible (`Result<Vec<u8>, Error::Codec>`,
|
||
ADR-023 §1); `open_conn` drops `SQLITE_OPEN_URI` (§3, register
|
||
D-29); the numeric-argument domain table is pinned in
|
||
core-contract.md (§2); the 1 ms watcher default stands with the
|
||
cadence documented in deployment.md (§4).
|
||
- **Folded into wave 3's tasks** (the review's §7 wiring items): the
|
||
trait-impl extent/duration guards (the contract-side domain rule,
|
||
enforced at the engine's trait-impl entry — the `sqlite-engine-*`
|
||
tasks carry it per mechanism), the two `#![allow]` lint removals in
|
||
`substrate/mod.rs` (the integration task — they can only lift once
|
||
every substrate surface is wired), and `SqliteOpts::poll_interval`
|
||
wiring (the constructor task).
|
||
- **Staying put as ordered**: M-1's retention-failure test and N-5's
|
||
panic probe → wave 5's contract-suite rows; N-1's growth-posture
|
||
review → wave 6.
|
||
|
||
## Wave 4 — Postgres engine
|
||
|
||
The second engine: the core contract implemented on tokio-postgres +
|
||
deadpool-postgres per engine-postgres.md — schema bootstrap (one
|
||
engine-owned schema, default `alkstore`, ADR-010 §8), the pool/open
|
||
constructor, the hand-rolled LISTEN forwarder (dedicated non-pooled
|
||
listener connection, bounded broadcast fanout, exponential-backoff
|
||
reconnect with re-LISTEN, the synthetic reconnect-wake on
|
||
`__alkstore_listener_reconnected__`), the full `Store`/`TxHandle`/
|
||
mechanism-handle trait impls (natively async — no `spawn_blocking`
|
||
seam; the tx handle holds the pooled object directly, ADR-007's pg
|
||
arm), the re-derived queue machinery (`FOR UPDATE SKIP LOCKED` claims,
|
||
LISTEN-driven wake with re-poll safety net), the scheduler leader loop
|
||
and outbox helper, and the engine's backlog column in the contract
|
||
suite (factory-parameterized; wave 5 runs the same rows against both
|
||
engines).
|
||
|
||
The wave is deliberately shaped as wave 3's structural twin — same
|
||
task rhythm (constructor → seam → mechanisms in parallel →
|
||
scheduler/outbox → integration → review gate) — because the two
|
||
engines implement the same contract and the wave-3 task set's
|
||
boundaries proved clean. The deltas are the engine's own: no
|
||
substrate (the pg machinery is re-derived greenfield per ADR-004/005,
|
||
with the pg-boss schema family as design reference), a natively-async
|
||
seam (Send+Sync client, POC #2 compile-probe), the forwarder as the
|
||
wake substrate (with its two POC-pinned deadlock pitfalls owned as
|
||
test-pinned failure modes), and the `PayloadTooLarge` client-side
|
||
check (this engine *produces* the variant — the one runtime carriage
|
||
of an engine asymmetry, ADR-016 §5).
|
||
|
||
Wave-3 outcomes absorbed into the task set:
|
||
|
||
- **The `#[doc(hidden)]` constructors** (`Job::from_row`,
|
||
`StreamEvent::from_row`, `Schedule::new`, `Wake::new`) already
|
||
exist — wave 3's pre-work task built them engine-agnostically; wave
|
||
4 consumes them as-is (no wave-4 equivalent needed).
|
||
- **The engine-side arithmetic is re-owned, not shared**: the
|
||
equal-jitter backoff curve, the opts-resolution rules
|
||
(delay-over-`run_at`, relative-`expires`, the derived stamp sets
|
||
300/3/5/none and 60/5/5), and the outbox backing-queue derivation
|
||
are each engine's own implementation (ADR-012 §2's one-owner rule
|
||
applied per engine); the pg tasks re-implement them from the ADR
|
||
text, and the contract suite (wave 5) pins the two engines'
|
||
arithmetic to identical outputs. The SQLite engine's `resolution.rs`
|
||
is reference, not a dependency.
|
||
- **The `@every` spec grammar has no shared parser on this side** —
|
||
the SQLite engine reuses the substrate's `parse_every_interval`
|
||
(one-line delta D-31's neighbor); the pg engine owns its parser
|
||
(the grammar is small and pinned: `@every <n><unit>`, `s|m|h|d`).
|
||
Equivalence of *behavior* (what fires, when) is suite-pinned; the
|
||
parsers themselves are per-engine.
|
||
- **The wave-3 review's deferred notes** (reader-pool close path,
|
||
commit-error-arm coverage) stay SQLite-scoped — the pg engine's
|
||
pool is deadpool-managed (no hand-rolled close path to review) and
|
||
its commit-error arm gets coverage in the pg tasks' own tests.
|
||
- **The suite's factory contract** (isolation + idempotent teardown)
|
||
is what the pg factory implements — a fresh schema per `open` per
|
||
the SQLite factory's precedent (fresh temp file there); the
|
||
POC's shared-server parallel-interference caveat (findings,
|
||
invocation note) is answered by exactly that isolation, not by
|
||
sequential-only harnesses.
|
||
|
||
Test posture: the engine's tests need a live Postgres server. The
|
||
harness convention is the POC's (dockerized `postgres:16-alpine` on
|
||
:15432, the `pglo-poc` container — still running in the dev
|
||
environment); connection settings ride environment/config, never
|
||
hardcoded. Tests that cannot reach a server skip cleanly (the
|
||
`SKIP`-posture), so the workspace gates stay green server-less — but
|
||
the wave's acceptance requires the suite green *against* the server.
|
||
|
||
One design point the architecture leaves to implementation (pinned in
|
||
the wave-4 tasks, flagged here for visibility): **the queue/stream
|
||
wake-channel naming**. The contract pins LISTEN as the pg wake
|
||
trigger and pins the channels/queues shared namespace, but no doc
|
||
names the channel an enqueue/publish notifies on. The tasks pin the
|
||
natural realization — the mechanism name *is* the wake channel
|
||
(`listen(queue_name)` receives that queue's enqueue wakes; same for
|
||
streams), legal by construction under the shared namespace — with
|
||
the reserved reconnect-wake channel staying the only reserved string.
|
||
|
||
## Decided points
|
||
|
||
- **Contract-suite layout — option (a)**: a small internal
|
||
`alkstore-contract-suite` crate (not published) exposing property
|
||
tests parameterized over a `Store` factory; each engine crate takes
|
||
it as a dev-dependency. One normative owner per property, mirroring
|
||
ADR-012 §2's one-owner rule. This discharges ADR-017 §4.2's
|
||
"decided at implementation" deferral; recorded as ADR-022 by the
|
||
scaffold task.
|
||
- **Engine tests vs. contract suite**: each engine wave carries its own
|
||
mechanism tests (does the engine work); wave 5 carries the
|
||
cross-engine *equivalence* properties (do the engines agree). The
|
||
verification-backlog rows are mostly equivalence-shaped, so this
|
||
split keeps wave 5 from re-testing engine internals.
|
||
- **CI**: none, deliberately. CI and publishing are run manually
|
||
(self-hosted Gitea; supply-chain posture). No CI-wiring task exists;
|
||
the merge gates (`cargo test`, `cargo clippy --all-targets --
|
||
-D warnings`, `cargo fmt --check`) are run coordinator-side.
|
||
- **Mem engine** (ADR-001 §4) and **fuzzing adoption** (the
|
||
alksocks/alktty/alktunnels pattern): both are "decided at
|
||
implementation" deferrals, recorded here so they surface as explicit
|
||
decision points in wave 6 (or earlier if the test story demands the
|
||
mem engine sooner) rather than ambushing a later session.
|
||
|
||
## Review gates
|
||
|
||
Each wave ends in a review task (`review-wave-N`) before the next wave
|
||
decomposes. Specific gates:
|
||
|
||
- **Wave 1 review** — the trait surface is versioned contract surface
|
||
from the first release (ADR-017); a shape error found here is cheap,
|
||
found in wave 5 it is a migration. Review checks the code against the
|
||
pinned ADR text line by line.
|
||
- **Wave 2 review** — diff reviewability against the honker lineage
|
||
(ADR-012 §3's fidelity posture), provenance register completeness
|
||
(ADR-018), floor tests green.
|
||
- **Wave 3/4 reviews** — engine-vs-contract conformance; the backlog
|
||
columns each engine owns.
|
||
- **Wave 5 review** — the suite as compatibility instrument: every
|
||
backlog row present, version-stamped, green on both engines; this is
|
||
the gate that flips the engine specs to `stable`.
|
||
|
||
## Review rounds so far
|
||
|
||
- **Wave 1 review gate** (`review-wave-1`) — trait surface vs pinned
|
||
ADR text; validation coverage fixes landed.
|
||
- **Wave 2 review gate** (`review-wave-2`) — lineage diff clean, one
|
||
re-derivation defect found and fixed (D-27).
|
||
- **General review, waves 1–2** (2026-10-08,
|
||
`docs/reviews/001-waves-1-2-general-review.md`) — M-1 fixed
|
||
inline (sweep savepoint scope); M-2/N-2/N-4/N-6 resolved as ADR-023
|
||
pre-decomposition; lint removal + suite adds folded into waves 3/5;
|
||
N-1 recorded for wave 6.
|
||
- **Wave 3 review gate** (`review-wave-3`) — engine-vs-contract
|
||
conformance code-read clean; two findings fixed inline (the
|
||
`open_writer_connection` boundary move out of `substrate/mod.rs`;
|
||
the writer-slot error-arm stranding in
|
||
`with_writer`/`begin`/`commit`); two minor notes deferred to
|
||
wave 5/6 (reader-pool close path, commit-error-arm coverage).
|
||
- **Wave 4 review gate** (`review-wave-4`) — engine-vs-contract
|
||
conformance code-read clean (0 findings); the flagged
|
||
`tx_publishes_compose_with_the_handle` flake reproduced twice and
|
||
recorded as F-1 (test-infra, wake-subscription under load — root
|
||
cause unresolved at gate time; candidate dispositions in the task's
|
||
Notes for wave 5's suite hardening); three no-action notes
|
||
(F-2..F-4). F-1's root cause was later found engine-side by the
|
||
general review below and retired by `pg-fix-tx-wake`.
|
||
- **General review, wave 4** (2026-10-09,
|
||
`docs/reviews/002-wave-4-general-review.md`) — two live-proven
|
||
consumer-facing bugs (the forwarder's permanent death after one
|
||
failed reconnect — an untested failure arm the gate's test never
|
||
covered; the tx enqueue/publish paths' missing `pg_notify` wake —
|
||
F-1's root cause, engine-side), a `max_size: 0` open-hang, two
|
||
narrow robustness gaps, doc mismatches, decode-duplication smells.
|
||
Fixes recommended before wave 5 decomposes. The tx-wake fix landed
|
||
(`pg-fix-tx-wake`, 2026-10-09): the tx producer paths now wake
|
||
commit-atomically, retiring F-1's engine arm (record updated in
|
||
`tasks/review-wave-4.md`) and making `tx_publishes_compose_with_the_handle`
|
||
deterministic (20 consecutive solo runs green); the remaining fixes
|
||
ride the wave-4 fix-batch decomposition below.
|
||
- **Wave 4 decomposition** (2026-10-08) — shaped as wave 3's
|
||
structural twin; wave-3 outcomes absorbed (constructors exist;
|
||
arithmetic re-owned per engine; per-engine `@every` parser;
|
||
SQLite-scoped deferred notes stay put). See the Wave 4 section.
|
||
- **Wave-4 fix-batch decomposition** (2026-10-09) — the general
|
||
review's findings decomposed into seven `pg-fix-*` tasks + a
|
||
`review-wave-4-fixes` gate: Finding 1 → `pg-fix-forwarder-reconnect`
|
||
(with the failed-connect test seam the gate's blind spot demands),
|
||
Finding 2 → `pg-fix-tx-wake` (retires F-1's engine arm), Finding 3
|
||
+ the DSN-options note → `pg-fix-open-path`, Finding 4 →
|
||
`pg-fix-stale-unlisten`, Finding 5 + the scheduler doc fix →
|
||
`pg-fix-scheduler-resilience`, the decode-duplication smell + small
|
||
cleanups → `pg-fix-dedupe-cleanup`, the TLS/QueueOpts doc
|
||
alignments → `pg-fix-docs-alignment`. Sequencing per the review's
|
||
recommendation: **wave 5 decomposes after the two HIGH fixes land**
|
||
(the suite's wake-driven rows would otherwise inherit a hang-shaped
|
||
false failure); 3–7 may trail into wave 5's window if the
|
||
fix-batch gate judges them wake-independent. |