diff --git a/docs/architecture/engine-postgres.md b/docs/architecture/engine-postgres.md index adc4e0a..1a05abb 100644 --- a/docs/architecture/engine-postgres.md +++ b/docs/architecture/engine-postgres.md @@ -1,6 +1,6 @@ --- -status: draft -last_updated: 2026-10-07 (ADR-021 — third review round: tx reads, claimed_at, schedule queue validation, drop=rollback, receiver arms) +status: stable +last_updated: 2026-10-10 (review-wave-5 gate: the verification backlog fully discharged in the version-stamped contract suite, green against this engine; spec flipped draft → stable) --- # Postgres engine diff --git a/docs/architecture/engine-sqlite.md b/docs/architecture/engine-sqlite.md index 01c82e1..bdfa4e1 100644 --- a/docs/architecture/engine-sqlite.md +++ b/docs/architecture/engine-sqlite.md @@ -1,6 +1,6 @@ --- -status: draft -last_updated: 2026-10-08 (ADR-023 — fourth review round: plain-path open (URI flag dropped), 1 ms watcher default + SqliteOpts knob) +status: stable +last_updated: 2026-10-10 (review-wave-5 gate: the verification backlog fully discharged in the version-stamped contract suite, green against this engine; spec flipped draft → stable) --- # SQLite engine diff --git a/docs/plans/implementation.md b/docs/plans/implementation.md index db12cc0..e2c8112 100644 --- a/docs/plans/implementation.md +++ b/docs/plans/implementation.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-10 (wave-4 fix batch landed and gate-verified — review-wave-4-fixes passed; wave 5 decomposed) +last_updated: 2026-10-10 (wave 5 implemented and reviewed — review-wave-5 verified the discharge map, stamps, dispositions and green-on-both-engines; engine specs flipped to stable) --- # alkstore — Implementation plan @@ -36,7 +36,7 @@ parallel). | [2](#wave-2--sqlite-substrate-fork) | honker-core fork into `alkstore-sqlite/src/substrate/`: port, deltas, provenance, test floor | wave 1 (workspace scaffold only) | implemented + reviewed (2026-10-08) | | [3](#wave-3--sqlite-engine) | SQLite engine: connection architecture, re-derived queue ops on contract v1, scheduler/outbox, tx seam, SQLite backlog column | waves 1 + 2 | implemented + reviewed (2026-10-08) | | 4 | [Postgres engine](#wave-4--postgres-engine): schema bootstrap, pool/open, listener/forwarder, all mechanisms, tx seam, pg backlog column | wave 1 | implemented + reviewed (2026-10-08) | -| 5 | [Contract suite](#wave-5--contract-suite): the cross-engine equivalence properties (core-contract.md §Verification backlog), version-stamped per ADR-017 | waves 3 + 4 | decomposed (2026-10-10) | +| 5 | [Contract suite](#wave-5--contract-suite): the cross-engine equivalence properties (core-contract.md §Verification backlog), version-stamped per ADR-017 | waves 3 + 4 | implemented + reviewed (2026-10-10) | | 6 | Release readiness: crate docs, deployment matrix final pass, README (written last, honestly), publish prep; mem-engine and fuzzing decisions | wave 5 | not yet decomposed | ## Wave 1 — Foundations @@ -324,6 +324,24 @@ decomposes. Specific gates: `tasks/review-wave-4.md`) and making `tx_publishes_compose_with_the_handle` deterministic (20 consecutive solo runs green); the remaining fixes ride the wave-4 fix-batch decomposition below. +- **Wave 5 review gate** (`review-wave-5`, 2026-10-10) — the suite as + compatibility instrument, gate passed: the backlog discharge map + verified row-by-row against the pinning tests (all 25 mechanism rows + per column stamped, wiring confirmed in both engines' suite targets; + the engine-side pins — SQLite open row, watcher-cadence knob, M-1 + sweep-rollback (SQLite trigger seam + the pg frame's code-read), + beyond-64 skip-forward, cap-curve — read and confirmed); stamps + cross-checked against the cited ADR § texts (the + `enqueue_opts_resolution` amendment accumulation rides ADR-023 §2 per + the convention); parked dispositions audited (all six recorded; + scheduler fire-wake parity left as-is, the lock busy-path answered + with no divergence); green on both engines (SQLite server-less; pg + vs the harness server, three full runs incl. one concurrent with the + SQLite column, plus focused stress). Engine specs flipped to + `stable`. Two unreproducible pg row failures from development were + recorded in the task's Notes (both rows' assertions are + state-outcome-shaped; flagged for watch, not blocking the gate) — + see `tasks/review-wave-5.md` §Notes. - **Wave 5 decomposition** (2026-10-10) — audit-first: the engines' backlog columns already discharge most of the §Verification backlog (map recorded in `tasks/review-wave-5.md`'s appendix for the gate to diff --git a/tasks/review-wave-5.md b/tasks/review-wave-5.md index 4168b62..103180d 100644 --- a/tasks/review-wave-5.md +++ b/tasks/review-wave-5.md @@ -1,7 +1,7 @@ --- id: review-wave-5 name: Wave 5 review gate — the suite as compatibility instrument -status: pending +status: completed depends_on: [suite-queue-depth-rows, suite-scheduler-rows, suite-tx-commit-atomicity-rows, suite-stream-rows, suite-lock-rows, suite-wake-rows, suite-opts-backoff-rows, sqlite-commit-error-arm, pg-suite-infra-hardening] scope: broad risk: low @@ -57,17 +57,17 @@ Primary lenses: ## Acceptance Criteria -- [ ] Backlog discharge map recorded (every §Verification backlog item +- [x] Backlog discharge map recorded (every §Verification backlog item → its pinning row/test or an explicit gap with a disposition) -- [ ] Every row version-stamped per convention; amendment stamps +- [x] Every row version-stamped per convention; amendment stamps accumulated; stamps verified against the cited ADR text -- [ ] Suite green on both engines (SQLite server-less; pg vs harness, +- [x] Suite green on both engines (SQLite server-less; pg vs harness, two consecutive full runs); workspace gates green -- [ ] All parked dispositions from the wave-5 tasks audited and +- [x] All parked dispositions from the wave-5 tasks audited and recorded -- [ ] Engine specs flipped to `stable` with dated annotations; +- [x] Engine specs flipped to `stable` with dated annotations; implementation.md updated (wave table + review rounds) -- [ ] Findings recorded; wave 6 decomposition may proceed +- [x] Findings recorded; wave 6 decomposition may proceed ## References @@ -79,11 +79,176 @@ Primary lenses: ## Notes -> To be filled by implementation agent +> Audit of record for the gate (2026-10-10). Every claim below was +> verified by reading the pinning row/test, not by trusting the +> appendix map or the implementing tasks' self-reports. + +**1. Backlog audit — the discharge map, verified.** All 25 mechanism +rows per engine column were read in full +(`alkstore-contract-suite/src/properties.rs`, 3344 lines) against the +backlog text in `core-contract.md` §Verification backlog. Findings: + +- Every row of the appendix's first table is genuinely discharged by + the named row: name validation + `schedule()` queue argument + the + keyed `publish_with_key_tx` empty-`Some` rule (exemplar covers all + three, tx twins included and `with_tx`-driven); numeric domains + (`extent_clamp_semantics` + `duration_refusal_on_non_positive_ttl`); + `encode_payload` typed failure + round-trip; the `PayloadTooLarge` + asymmetry (both engine-scoped rows, the pg one reading the limit + from the produced variant); drop = rollback no-ghosts; + read-your-own-writes; receiver arms + monotone save composition. +- The combined-coverage claim for `save_offset_tx` exactly-once + (in-tx-ryow's own-save visibility + drop-rollback's save-never-lands) + holds: together they pin (a) the save is visible inside the tx, + (b) the saved checkpoint survives commit semantics exactly when the + business tx commits, (c) rollback deletes it. No gap. +- The engine-side pins the map claims exist and are real: + `uri_shaped_open_path_is_a_literal_filename` and + `poll_interval_flows_to_the_watcher_config` + (`alkstore-sqlite/src/store/open_tests.rs`); + `sweep_rolls_back_the_retention_half_with_the_move` + (SQLite substrate trigger seam — M-1's live pin) and the pg twin's + structural guarantee confirmed by code-read of + `alkstore-postgres/src/queue.rs::sweep_expired` (move + retention + DELETE inside one `in_tx` `BEGIN…COMMIT/ROLLBACK` frame — a + retention-DELETE failure rolls the move back via the frame's error + arm); + `receiver_stays_open_across_reconnects_closes_at_shutdown` (pg + notify tests) + `subscriber_survives_reconnect_and_heals_gap_by_redrain` + (pg stream tests) — the reconnect-stays-open pin the receiver-arms + row cross-references; and the pg receiver error/close arms it pins + inside the shared row body. +- Every row of the second table (wave-5 tasks) is present and stamped: + the three queue-depth rows, three scheduler rows, three tx-seam rows + (N-5's panic probe included), two stream rows, two lock rows (the + SQLite busy-path open question answered in-row: losers get the clean + `None` value — no divergence), `wake_receiver_shapes`, + `backoff_curve_equivalence`, and the two extended + `enqueue_opts_resolution` clock legs. The two engine-side hardening + deliverables landed (`failed_commit_replenishes_the_writer_slot` — + replay-proofed per its task; `reconnect_success_resumes_wake_delivery`; + the pg `must_recv_event` parked-recv re-shape with the + self-diagnosing deadline panic). + +**2. Stamp audit — verified against the cited ADR text.** `grep +"Contract stamp:"` yields exactly 25 markers for 25 rows (one per +row; multi-stamp rows cite several ADRs on one marker). Every cited § +exists in its ADR and pins the behavior the row tests — verified +against the ADR bodies for the load-bearing citations: ADR-008 §3 +(wake type + recv forms, `Ok(None)` idle arm), §5 (error taxonomy / +value-not-error), §7 (locks guarantee row), §8 (explicit saves only); +ADR-009 §1/§3/§4/§6 (runner shape, boundary fires, the 64-cap, +`LeadershipLost`); ADR-010 §1 (delete-on-ack, per-id `ack_batch` +predicate — pinned in ADR-021-era §1 annotation), §2 (validity +predicate, reclaim-eats-attempt), §3/§3a, §4 (strings, get_job-sees-dead, +retention default), §5 (no-stranded-rows, retention); ADR-014 §1 (60/5/5 +derived stamp set); ADR-015 §1–§5 (key semantics, tx seam, `StreamEvent` +shape, ordering row, `trim_to` — the row texts quote the ADR's +"silent-loss / resume-at-horizon / no-dedicated-wake" semantics +verbatim); ADR-016 §5; ADR-020 §1/§2/§3/§4; ADR-021 §1/§3/§4/§5; +ADR-023 §1/§2 (the domain-rule table); ADR-012 §2 (one-owner +arithmetic, suite-pinned identical); ADR-019 §1/§3/§4/§6; ADR-007; +ADR-006. The amendment case checks: `enqueue_opts_resolution`'s stamp +list accumulated ADR-023 §2 when the wave-5 clock legs landed, on top +of its ADR-020 stamps — the deferral note replaced by the completion +statement, per the convention. No stamp cites an ADR § that doesn't +pin the tested behavior; no tested behavior lacks a stamp. + +**3. Green on both engines — verified this session.** Workspace gates +green: `cargo build`, `cargo test` (12 binaries — core 25; suite +harness 3; pg 121 lib + 25 suite + 9 schema against the harness +server; SQLite 191 lib + 25 suite), `cargo clippy --all-targets -- +-D warnings`, `cargo fmt --check`. Dedicated suite runs: pg column +vs the harness server three full runs (two consecutive + one concurrent +with the SQLite column, the wave-4 gate's posture plus), 25/25 each; +SQLite column green server-less twice (isolated + concurrent), 25/25. +Focused stress: 6 consecutive `--test-threads=6` runs of +`row_trim_to_semantics` + `row_lock_ttl` on pg — green throughout. + +**4. Parked dispositions audited — all six made and recorded.** + +| Disposition | Where recorded | Audit verdict | +|---|---|---| +| M-1 retention-failure pin's location | `suite-queue-depth-rows` Notes | Engine-side (SQLite trigger seam; pg by frame structure) — sound; the pg leg rests on code-read, accepted with the row's doc text recording it | +| Beyond-64 skip-forward leg | `suite-scheduler-rows` Notes | Engine-side twins exist on both engines (`catch_up_replays_up_to_the_cap_then_skips_forward`); suite pins the ≤64 in-band leg; row doc states it | +| Backoff cap leg | `suite-opts-backoff-rows` Notes | Engine-side pins verified present (both engines' unit tests); row doc records the disposition | +| Scheduler fire-wake parity | `suite-scheduler-rows` Notes | Not demanded by the row shapes (claim-polling observation only); the recorded gap stands for a later task; accepted as-is | +| Lock-row divergence call | `suite-lock-rows` Notes | No divergence found — the row pins the convergence; nothing to fix | +| SQLite reconnect-success test | `sqlite-commit-error-arm` Notes | Taken — `reconnect_success_resumes_wake_delivery` exists and pins the success arm's re-baseline + restored delivery | + +**5. Conformance spot-checks — clean.** No row pins beyond ADR text +(a row that would fail a correct engine was not found); the +determinism posture holds throughout — state-outcome assertions, +bounded waits, tolerance bands on stamps (`abs_diff <= 5`, ±10 s +informational `created_at`, one-second straddle on the curve upper +bounds), sleeps bounded well past second-resolution stamps; the two +documented posture extensions (runner-driving rows; the N-5 panic +probe) are admitted in the module doc as ADR-017 §2 class-4 +suite-side posture notes and are implemented as described; no leg +duplication — cross-references instead (duration guards → +`duration_refusal_on_non_positive_ttl`; close arms → +`receiver_close_and_save_arms`; byte-exactness → +`payload_round_trip_stores_exact_encoding`; negative-horizon → +`extent_clamp_semantics`). + +**6. Flaky-test ledger — the two unreproducible pg failures.** Two +single-occurrence failures were recorded honestly during development: +`row_trim_to_semantics` (stream-rows, 1/14 under the concurrent +SQLite+pg condition) and `row_lock_ttl_expiry_and_reacquisition` +(lock-rows, first cold pg run, message lost to truncation). Bounded +review investigation this session: three full pg suite runs (one under +the replayed concurrent SQLite+pg condition), six focused +`--test-threads=6` runs of both rows, and the workspace's own full pg +runs — all green; no divergence, no repro. Both rows' windows assert +state outcomes only (delivered events / post-sleep lock state), so a +timing-value failure mode is not available; the lock row's lapse +assertions are post-sleep states that a slower clock can only delay, +never un-lapse. One residual observation recorded for the future: the +trim row's pg-side failure shape would have been an event delivered to +the subscriber beyond its checkpoint in the post-trim absence window — +under pg's cross-database LISTEN the wakes are mechanism-named, and +the row's re-drain safety argument (own-schema storage yields nothing) +is the by-construction defense; if either row fails again, it should +get a dedicated investigative session (instrumentation + focused +repro loop) *before* any postulate-and-fix — per the user's ledger +the pattern (a prior unreproducible flake hinting at a real engine +issue) deserves that attention. Not blocking this gate: 13+ subsequent +clean pg runs across both rows since the failures, all conditions +covered. **Flagged for watch in wave 6's window.** + +**7. Findings.** No code, suite-row, or stamp defects requiring change +were found at gate time; all changes this gate made are doc-side +(engine specs `draft` → `stable` with dated gate annotations; +`docs/plans/implementation.md` wave table + review-rounds entry) and +task-file hygiene (`suite-opts-backoff-rows` had the placeholder +"To be filled" lines left in its Notes/Summary headers above real +content — removed the placeholders). Wave 6 decomposition may proceed. ## Summary -> To be filled on completion +> Filled by the review agent (2026-10-10): + +The wave-5 review gate passed. The suite is the compatibility +instrument the plan called for: **all 25 mechanism rows per engine +column present, version-stamped per `version_stamp.rs`'s convention +(greppable via `STAMP_MARKER`), green on both engines.** Verified by +direct read of every row and both engines' wiring, cross-checked +against `core-contract.md` §Verification backlog item by item (the +verified discharge map is in Notes §1), every `Contract stamp:` against +its cited ADR § text (Notes §2; the `enqueue_opts_resolution` +amendment accumulation checked), all six parked dispositions +audited (Notes §4, table), and the determinism/conformance postures +spot-checked (Notes §5). Suite green on both engines with the gate +posture exceeded (three full pg runs against the harness, one +concurrent with the SQLite column; SQLite green server-less twice), +plus focused stress on the two development-time flake rows; workspace +build/test/clippy/fmt green. Engine specs flipped to `stable` +(`docs/architecture/engine-sqlite.md`, `docs/architecture/engine-postgres.md`, +dated annotations citing this gate); `docs/plans/implementation.md` +wave table and review-rounds updated. The two unreproducible pg +failures from development are recorded with the bounded investigation +and a watch flag (Notes §6) — if either reproduces, a dedicated +session follows. Wave 6 (release readiness) decomposition may proceed. ## Appendix — the decomposition's backlog audit (verify, don't trust) diff --git a/tasks/suite-opts-backoff-rows.md b/tasks/suite-opts-backoff-rows.md index d252622..95b1058 100644 --- a/tasks/suite-opts-backoff-rows.md +++ b/tasks/suite-opts-backoff-rows.md @@ -74,7 +74,6 @@ Work: ## Notes -> To be filled by implementation agent - **The 1-hour cap leg is not suite-pinned** (as pre-decided): reaching capped attempts requires waiting out minute-scale delays. The cap @@ -124,8 +123,6 @@ Work: ## Summary -> To be filled on completion - `enqueue_opts_resolution` extended in place with the wave-5 completion legs (its deferral note replaced by the completion statement, ADR-023 §2 stamp accumulated): the `run_at`-alone literal