Wave 4 decomposed: Postgres engine — 11 tasks (schema, open/opts, seam, forwarder, mechanisms, scheduler/outbox, integration, review gate); plan synced

This commit is contained in:
glm-5.3-flash committed 2026-10-08 22:49:00 +00:00
1 parent ecb8211694
commit 4caea21098
11 files changed
+1286 -2

No files matched your search

+86 -2
View File
@@ -1,6 +1,6 @@
---
status: draft
last_updated: 2026-10-08 (wave 3 implemented + reviewed; review-gate fixes: substrate boundary move, writer-slot error-arm release; wave 4/5 decomposition may proceed)
last_updated: 2026-10-08 (wave 4 decomposed — 11 tasks; waves 1–3 implemented + reviewed)
---
# alkstore — Implementation plan
@@ -35,7 +35,7 @@ parallel).
| [1](#wave-1--foundations) | Workspace scaffold; core crate (errors, value types, full trait surface); contract-suite scaffold | — | implemented + reviewed (2026-10-08) |
| [2](#wave-2--sqlite-substrate-fork) | honker-core fork into `alkstore-sqlite/src/substrate/`: port, deltas, provenance, test floor | wave 1 (workspace scaffold only) | implemented + reviewed (2026-10-08) |
| [3](#wave-3--sqlite-engine) | SQLite engine: connection architecture, re-derived queue ops on contract v1, scheduler/outbox, tx seam, SQLite backlog column | waves 1 + 2 | implemented + reviewed (2026-10-08) |
| 4 | Postgres engine: schema bootstrap, pool/open, listener/forwarder, all mechanisms, tx seam, pg backlog column | wave 1 | not yet decomposed |
| 4 | [Postgres engine](#wave-4--postgres-engine): schema bootstrap, pool/open, listener/forwarder, all mechanisms, tx seam, pg backlog column | wave 1 | decomposed (11 tasks) |
| 5 | Contract suite: the cross-engine equivalence properties (core-contract.md §Verification backlog), version-stamped per ADR-017 | waves 3 + 4 | not yet decomposed |
| 6 | Release readiness: crate docs, deployment matrix final pass, README (written last, honestly), publish prep; mem-engine and fuzzing decisions | wave 5 | not yet decomposed |
@@ -101,6 +101,86 @@ shaped this wave's task set:
panic probe → wave 5's contract-suite rows; N-1's growth-posture
review → wave 6.
## Wave 4 — Postgres engine
The second engine: the core contract implemented on tokio-postgres +
deadpool-postgres per engine-postgres.md — schema bootstrap (one
engine-owned schema, default `alkstore`, ADR-010 §8), the pool/open
constructor, the hand-rolled LISTEN forwarder (dedicated non-pooled
listener connection, bounded broadcast fanout, exponential-backoff
reconnect with re-LISTEN, the synthetic reconnect-wake on
`__alkstore_listener_reconnected__`), the full `Store`/`TxHandle`/
mechanism-handle trait impls (natively async — no `spawn_blocking`
seam; the tx handle holds the pooled object directly, ADR-007's pg
arm), the re-derived queue machinery (`FOR UPDATE SKIP LOCKED` claims,
LISTEN-driven wake with re-poll safety net), the scheduler leader loop
and outbox helper, and the engine's backlog column in the contract
suite (factory-parameterized; wave 5 runs the same rows against both
engines).
The wave is deliberately shaped as wave 3's structural twin — same
task rhythm (constructor → seam → mechanisms in parallel →
scheduler/outbox → integration → review gate) — because the two
engines implement the same contract and the wave-3 task set's
boundaries proved clean. The deltas are the engine's own: no
substrate (the pg machinery is re-derived greenfield per ADR-004/005,
with the pg-boss schema family as design reference), a natively-async
seam (Send+Sync client, POC #2 compile-probe), the forwarder as the
wake substrate (with its two POC-pinned deadlock pitfalls owned as
test-pinned failure modes), and the `PayloadTooLarge` client-side
check (this engine *produces* the variant — the one runtime carriage
of an engine asymmetry, ADR-016 §5).
Wave-3 outcomes absorbed into the task set:
- **The `#[doc(hidden)]` constructors** (`Job::from_row`,
`StreamEvent::from_row`, `Schedule::new`, `Wake::new`) already
exist — wave 3's pre-work task built them engine-agnostically; wave
4 consumes them as-is (no wave-4 equivalent needed).
- **The engine-side arithmetic is re-owned, not shared**: the
equal-jitter backoff curve, the opts-resolution rules
(delay-over-`run_at`, relative-`expires`, the derived stamp sets
300/3/5/none and 60/5/5), and the outbox backing-queue derivation
are each engine's own implementation (ADR-012 §2's one-owner rule
applied per engine); the pg tasks re-implement them from the ADR
text, and the contract suite (wave 5) pins the two engines'
arithmetic to identical outputs. The SQLite engine's `resolution.rs`
is reference, not a dependency.
- **The `@every` spec grammar has no shared parser on this side** —
the SQLite engine reuses the substrate's `parse_every_interval`
(one-line delta D-31's neighbor); the pg engine owns its parser
(the grammar is small and pinned: `@every <n><unit>`, `s|m|h|d`).
Equivalence of *behavior* (what fires, when) is suite-pinned; the
parsers themselves are per-engine.
- **The wave-3 review's deferred notes** (reader-pool close path,
commit-error-arm coverage) stay SQLite-scoped — the pg engine's
pool is deadpool-managed (no hand-rolled close path to review) and
its commit-error arm gets coverage in the pg tasks' own tests.
- **The suite's factory contract** (isolation + idempotent teardown)
is what the pg factory implements — a fresh schema per `open` per
the SQLite factory's precedent (fresh temp file there); the
POC's shared-server parallel-interference caveat (findings,
invocation note) is answered by exactly that isolation, not by
sequential-only harnesses.
Test posture: the engine's tests need a live Postgres server. The
harness convention is the POC's (dockerized `postgres:16-alpine` on
:15432, the `pglo-poc` container — still running in the dev
environment); connection settings ride environment/config, never
hardcoded. Tests that cannot reach a server skip cleanly (the
`SKIP`-posture), so the workspace gates stay green server-less — but
the wave's acceptance requires the suite green *against* the server.
One design point the architecture leaves to implementation (pinned in
the wave-4 tasks, flagged here for visibility): **the queue/stream
wake-channel naming**. The contract pins LISTEN as the pg wake
trigger and pins the channels/queues shared namespace, but no doc
names the channel an enqueue/publish notifies on. The tasks pin the
natural realization — the mechanism name *is* the wake channel
(`listen(queue_name)` receives that queue's enqueue wakes; same for
streams), legal by construction under the shared namespace — with
the reserved reconnect-wake channel staying the only reserved string.
## Decided points
- **Contract-suite layout — option (a)**: a small internal
@@ -160,3 +240,7 @@ decomposes. Specific gates:
the writer-slot error-arm stranding in
`with_writer`/`begin`/`commit`); two minor notes deferred to
wave 5/6 (reader-pool close path, commit-error-arm coverage).
- **Wave 4 decomposition** (2026-10-08) — shaped as wave 3's
structural twin; wave-3 outcomes absorbed (constructors exist;
arithmetic re-owned per engine; per-engine `@every` parser;
SQLite-scoped deferred notes stay put). See the Wave 4 section.
+101
View File
@@ -0,0 +1,101 @@
---
id: pg-engine-integration
name: Postgres engine — integration (full-surface wiring, suite adoption, crate docs)
status: pending
depends_on: [pg-engine-notify-listen, pg-engine-streams, pg-engine-queues, pg-engine-locks, pg-engine-scheduler-outbox]
scope: moderate
risk: medium
impact: phase
level: implementation
tags: [wave-4, postgres-engine, integration]
---
## Description
Close the wave's integration gaps once every mechanism is wired — the
structural twin of wave 3's integration task (no lint-suppression
removal on this side: the pg engine is greenfield, no `#![allow]`
posture ever existed — the sweep below is the same discipline without
that step):
- **Full-surface sweep**: every `Store`/`TxHandle`/mechanism trait
method implemented (no `todo!`/`unimplemented!`/error-stub
remaining); the engine crate's lib re-exports its public surface
(`open`, `PgOpts`, the concrete store type) per the module-per-file
convention. Genuinely dead code must be **cut, not suppressed** —
clippy `-D warnings` green without allows.
- **Contract-suite adoption** (the plan's "waves 3 and 4 adopt the
harness"): implement `StoreFactory` for the pg engine — **a fresh
schema per `open`** (the SQLite factory's fresh-temp-file
precedent; the POC's shared-server parallel-interference caveat is
answered by exactly this isolation — each property's rows live in
its own schema, so parallel property runs never observe one
another), idempotent teardown (`DROP SCHEMA … CASCADE`; safe
against an abandoned store per the factory contract's teardown
posture). Pin the factory's isolation/idempotence contract (the
SQLite factory's pin shape).
Wire the engine's backlog column — the rows this engine owns now,
running against the pg factory:
- The ADR-023-stamped rows (extent-clamp, duration-refusal,
`encode_payload` round-trip) — already written suite-side,
factory-parameterized; they run against pg by this task's
factory existing (verify, don't rewrite).
- **The pg arm of the engine-scoped rows**: `PayloadTooLarge`
produced (the pg rejection arm — oversized `notify`/`notify_tx`
rejected client-side with the limit in the variant; the row text
the SQLite integration task left for "wave 4's adoption"),
drop = rollback no-ghosts, in-tx read-your-own-writes,
enqueue-opts resolution, receiver close arms (the pg arm: stays
open across reconnects, closes at shutdown — the row's
engine-asymmetric legs may need the row split or parameterized
per the suite's convention; implementer's choice, documented).
- The SQLite-scoped rows (plain-path open §3, poll cadence §4)
simply don't run against pg — the SQLite integration task's
Notes already pinned that disposition.
- **Crate docs**: the engine crate's lib-level doc comments state the
multi-host posture and the listener budget line (ADR-016,
deployment.md) — the identity statements this crate owns (the open
task drafted them; this task verifies they reflect the finished
engine).
- **Gates**: full workspace build/test/clippy/fmt (the pg tests skip
cleanly server-less but must be green against the harness server);
coverage spot-check on the engine crate (no large uncovered regions
outside error arms — the wave-3 gate's bar).
## Acceptance Criteria
- [ ] No `todo!`/`unimplemented!`/error-stub in the engine crate;
full trait surface implemented; clippy `-D warnings` green
without allows
- [ ] `StoreFactory` implemented for pg (fresh schema per `open`,
idempotent CASCADE teardown); the factory's
isolation/idempotence contract pinned
- [ ] The backlog column runs green against the pg factory: the three
ADR-023 rows verified (not rewritten), the pg arm of the five
engine-scoped rows present + stamped
- [ ] The `PayloadTooLarge` pg arm row present (the SQLite
integration task's deferred adoption discharged)
- [ ] Crate lib docs carry the multi-host + listener-budget posture
statements, reflecting the finished engine
- [ ] Workspace gates green: `cargo build`, `cargo test` (incl.
against the harness server), clippy `-D warnings`, fmt clean
- [ ] Coverage spot-check: no large uncovered regions outside error
arms
## References
- docs/plans/implementation.md (Wave 4 section)
- docs/architecture/core-contract.md (§Verification backlog)
- docs/architecture/decisions/022-contract-suite-layout.md
- docs/architecture/decisions/023-fourth-review-round.md (backlog additions)
- docs/architecture/decisions/016-deployment-honesty.md §5
- alkstore-sqlite/tests/contract_suite.rs (the factory precedent)
- alkstore-contract-suite/src/properties.rs (the rows to run/adopt)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+93
View File
@@ -0,0 +1,93 @@
---
id: pg-engine-locks
name: Postgres engine — named locks (`try_lock`, `Lock` handle, duration guards)
status: pending
depends_on: [pg-engine-seam-tx]
scope: narrow
risk: low
impact: component
level: implementation
tags: [wave-4, postgres-engine]
---
## Description
Implement the named-lock mechanism's pg arm — the smallest mechanism,
and the one with the strongest POC ground (the pg POC's lock probe
pinned acquire/release/renew with TTL and expiry re-acquisition
directly). Over the schema task's locks table:
- **`try_lock(name, owner, ttl)`** — shared-namespace + non-empty
owner validation (`InvalidName`); the **duration guard** (`ttl <=
0` → opaque `Err(Database)`, detail in the source chain — ADR-023
§2's duration kind; an exclusion mechanism must not silently hand
back a lease the caller believes it holds); the closed-store check;
then the acquire op on a pool connection: opportunistic-expiry-
delete + insert-or-reacquire + read-back (the substrate's
`lock_acquire` semantics, re-derived — the POC's probe shape).
`Ok(None)` = held elsewhere (no-work is a value); `Ok(Some(boxed
Lock))` on grant.
- **`Lock` handle** (`renew`, consuming `release`):
- `renew(ttl)` — the same duration guard; the new **full TTL
window from now** (not additive); `false` = lost it (expired and
re-acquired by another owner — the owner-scoped update's 0-rows).
- `release(self: Box<Self>)` — consuming; the owner-scoped delete;
`true` = was held (row gone by this owner's hand), `false` =
row-less or foreign (the substrate's row-scoped truth-teller
posture — no handle-local "held" flag caching).
- **Guarantee row posture** (ADR-008 §7): mutual exclusion bounded by
TTL + renewal; expiry lapses *silently* (no revocation event —
expired rows are deleted opportunistically by the next acquire's
expiry-delete, no general expiry sweeper — the inherited posture,
documented); re-acquirable after expiry (the POC-pinned property —
pin it engine-side here; the contract-suite row remains for the
suite itself).
- **Same-owner re-acquire**: pin the inherited disposition (the
substrate's shape: the same owner's re-acquire keeps the original
row, TTL *not* refreshed, reports granted — the SQLite task's
pinned posture; engine-sqlite.md's mapping row calls it inherited
deliberately, and the pg arm should match it for cross-engine
equivalence — flag in Notes if the pg realization differs).
- **Post-lapse release arms**: pin both (the stale holder's release
deletes its own stale row → `true`; after a foreign re-acquire →
`false`) — the SQLite task's pinned shape, matched here.
- **Closed-store posture**: `try_lock`/`renew`/`release` fail closed
with opaque `Database` (the engine-wide shape).
- **No `lock_tx`** — deliberate absence (core-contract.md's named-
locks section); nothing to build, just don't invent one.
## Acceptance Criteria
- [ ] `try_lock`: validation (name + owner), duration guard
(`ttl <= 0` → opaque `Err`, detail in the source chain),
closed-store check, then acquire — `Ok(None)` on held-elsewhere
- [ ] Exclusion + loser shape pinned (two owners, one wins); expiry
re-acquisition pinned (the POC's property, engine-side)
- [ ] `renew`: duration guard; full-window reset; `false` on
lost/expired-and-reacquired
- [ ] `release`: consuming, owner-scoped, both boolean arms pinned
(stale-row delete = `true`; foreign/row-less = `false`)
- [ ] Same-owner re-acquire disposition pinned (and flagged if it
differs from the SQLite arm's)
- [ ] Silent lapse posture documented (no revocation event; no
general expiry sweeper)
- [ ] `cargo test -p alkstore-postgres` (harness server), clippy
`-D warnings`, fmt clean; gates green server-less
## References
- docs/architecture/core-contract.md (named locks section)
- docs/architecture/decisions/008-contract-v1-pinning.md §7 (guarantee row)
- docs/architecture/decisions/023-fourth-review-round.md §2 (duration kind)
- docs/architecture/engine-postgres.md (Mapping the contract: named locks)
- docs/research/poc-pg-posture-findings.md (the lock probe row)
- alkstore-sqlite/src/lock.rs (the structural twin — match its pinned dispositions)
- alkstore/src/lock.rs (the trait)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+139
View File
@@ -0,0 +1,139 @@
---
id: pg-engine-notify-listen
name: Postgres engine — LISTEN forwarder (full behavior) + notify/listen surface
status: pending
depends_on: [pg-engine-seam-tx]
scope: moderate
risk: high
impact: component
level: implementation
tags: [wave-4, postgres-engine, forwarder]
---
## Description
Complete the LISTEN forwarder's behavior (the open task stood up the
skeleton) and implement `Store::notify` and `Store::listen` — the pg
arm of the wake contract (ADR-006). The forwarder is hand-rolled
machinery (ADR-004's decision; `postgres-notify` derive-not-adopted)
— its failure modes are *learned territory*, POC-pinned; this task
owns them as passing tests.
**Forwarder completion** (on the skeleton's dedicated non-pooled
connection + poll loop + bounded broadcast):
- **Reconnect**: exponential backoff 50 ms → 2 s cap (the POC's
policy); on each successful reconnect, re-LISTEN from the live
channel list (the store's channel registry — `listen` adds to it,
receiver drop removes it) and broadcast the **synthetic
reconnect-wake** on the reserved channel
`__alkstore_listener_reconnected__` (ADR-008 §4's one v1-reserved
string — the POC's ad-hoc `__listener_reconnected__` renamed into
the namespace) to *every* subscriber's receiver (ADR-006's
reconnect-recovery semantics — the no-replay hole's honest
communication).
- **Fanout**: `poll_message`'s notifications fan out per channel —
each `listen(channel)` subscriber receives wakes for its channel
only (plus the reserved reconnect-wake, which every subscriber
receives). Bounded broadcast per subscriber or one shared bounded
channel with per-subscriber routing — implementer's choice, but
**lag is surfaced, not silent** (the POC's `Lagged(n)` posture:
log/record and keep going; a lagged subscriber's missed wakes are
the contract's best-effort-hint shape, recovery is re-read).
- **Wake content**: `Wake { channel }` only — never payloads, never
ids (ADR-008 §3). Per-notify delivery (no coalescing — LISTEN
delivers each committed NOTIFY; the SQLite engine's coalescing is
the documented engine asymmetry).
- **Receiver close semantics** (ADR-021 §5's pg arm — the documented
engine asymmetry): the receiver **stays open across forwarder
reconnects** (reconnect-wakes arrive on it; no close) and closes
only at engine shutdown (`close()`/Drop → all receivers terminal
`None`). This is the inverse of SQLite's death-closes-receivers —
pin it by test (kill the listener's backend mid-subscription;
receivers stay open; the reconnect-wake arrives; post-reconnect
deliveries work).
- **Identifier quoting**: channel names are quoted identifiers
(`LISTEN "{name}"` with `"` doubling — the POC's `quote_ident`
shape); validation already rejects empty/reserved, but quoting is
what makes arbitrary legal names (colons, etc.) work.
**`Store::notify(channel, payload)`**: auto-commit path — pool
checkout, `SELECT pg_notify($1, $2)` in its own implicit transaction,
return to pool. The 8000-byte client-side check (typed
`PayloadTooLarge` before any round trip) — the seam task built it for
`notify_tx`; this task wires the auto-commit twin through the same
check (one owner of the limit constant).
**`Store::listen(channel)`**: validate (shared-namespace kinds —
`InvalidName`/`ReservedName`), register the channel in the store's
channel registry (issuing `LISTEN` on the forwarder's connection —
the dynamic channel set), return the boxed `PgWakeReceiver` bridging
the fanout:
- `recv()` → `Some(Wake { channel })` per notification; `None` only
at engine shutdown (terminal — never reopens; ADR-021 §5's pg arm).
- `try_recv` / `recv_timeout` per the core trait's pinned arms
(`Closed` distinguishable from idle; timeout expiry = `Ok(None)`).
- Receiver drop unregisters the channel (issuing `UNLISTEN` when the
last subscriber for that channel drops — the channel registry's
refcount; zero subscribers ⇒ no LISTEN needed).
- `listen` on a closed store fails closed (`Database`).
**Wake-arrives honesty**: tests pin wake-arrives (cross-connection
commit → wake), the reconnect-wake path, and close-at-shutdown — not
wake-exclusivity (a wake may arrive for unrelated activity on the
shared channel namespace; the contract's best-effort-hint posture).
**The POC's two deadlock pitfalls, pinned as tests** (they are the
hand-rolled posture's owned failure modes, ADR-004):
1. The poll loop runs before the first client query on the listener
connection (structurally excluded by the skeleton; pin the
ordering with a test that issues a query immediately after
listener spawn).
2. The listener's `Client` stays alive for the listener's lifetime
(pin: a long-lived listener survives unrelated client churn).
## Acceptance Criteria
- [ ] Forwarder reconnects with 50 ms → 2 s exponential backoff;
re-LISTENs from the live channel set; broadcasts the synthetic
reconnect-wake on `__alkstore_listener_reconnected__` to every
subscriber after each reconnect — tested via backend kill
- [ ] No-replay honesty pinned: a commit during a connection gap is
never re-delivered (`!saw_replay` posture); recovery is the
reconnect-wake + consumer re-read
- [ ] `notify` delivers commit-atomic auto-commit notifies; the
8000-byte check is typed, client-side, before any round trip
(both `notify` and `notify_tx` — one limit owner)
- [ ] `listen` returns a working `WakeReceiver`: per-notify wakes,
`Wake { channel }` only, channel-scoped fanout, multi-channel
on one listener connection, N subscribers per channel
- [ ] Receiver close semantics: stays open across reconnects
(reconnect-wake arrives), closes only at engine shutdown
(terminal `None`); `try_recv`/`recv_timeout` arms match the
pinned shapes
- [ ] Receiver drop unregisters (UNLISTEN at last-subscriber drop)
- [ ] Channel validation on both entry points; closed-store
fail-closed on both
- [ ] The two deadlock pitfalls pinned by tests
- [ ] `cargo test -p alkstore-postgres` (harness server), clippy
`-D warnings`, fmt clean; gates green server-less
## References
- docs/architecture/engine-postgres.md (Connection architecture: forwarder)
- docs/architecture/decisions/004-postgres-driver.md (forwarder decision, pitfalls)
- docs/architecture/decisions/006-wake-and-delivery-contract.md (wake contract, reconnect-wake)
- docs/architecture/decisions/008-contract-v1-pinning.md §3, §4 (wake type, reserved string)
- docs/architecture/decisions/016-deployment-honesty.md §5 (PayloadTooLarge)
- docs/architecture/decisions/021-tx-reads-and-value-shape-fixes.md §5 (receiver arms, pg close semantics)
- docs/research/poc-pg-posture-findings.md (Sub-module L — the measured ground)
- alkstore-sqlite/src/notify.rs (the structural twin — inverse close semantics)
- /workspace/alkstore-pg-posture-poc/src/listen.rs (the POC's ~90-line shape)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+113
View File
@@ -0,0 +1,113 @@
---
id: pg-engine-open-opts
name: Postgres engine — `open` constructor, `PgOpts`, pool + listener wiring
status: pending
depends_on: [pg-engine-schema]
scope: moderate
risk: medium
impact: component
level: implementation
tags: [wave-4, postgres-engine]
---
## Description
Stand up the pg engine's public surface: the `open` constructor,
`PgOpts`, and the connection architecture of engine-postgres.md's
"Connection architecture" section — the structural twin of wave 3's
`sqlite-engine-open-opts`, with the pg deltas.
- `PgOpts` (engine-crate type, ADR-008 §6's config split; NOT
`#[non_exhaustive]` — opts structs consumers construct, ADR-017 §3's
exemption; `Default` + `Clone`): at minimum
- connection string / config (the tokio-postgres `Config`-parseable
form — the consumer's server address, credentials; the engine
never invents defaults for these, `open` fails `Database` if
unparseable/unreachable),
- `schema: String` (default `"alkstore"` — ADR-010 §8),
- `max_size` (the deadpool pool size; default documented, the
deployment matrix's budget line — the listener adds +1 outside
the pool),
- `synchronous_commit: bool` (default `true` — ship config,
ADR-004/deployment.md's measured trade; wired as a per-session
SET at connect, POC-verified mechanics).
- `open(config, opts) -> Result<Box<dyn Store>>` (concrete
`PgStore` re-exported, mirroring the SQLite engine's
boxed-constructor + pub-concrete shape):
- Build the deadpool pool (`RecyclingMethod::Fast` — no `DISCARD
ALL` recycling, the POC's zero-error posture).
- Run the schema task's bootstrap on a pool connection at open
(idempotent; open fails `Database` if bootstrap fails).
- Spawn the **forwarder** (the listener task) — the wake substrate
every later mechanism task rides. The forwarder is this task's
hard part; its full behavior (reconnect, re-LISTEN, reconnect-
wake) is the notify-listen task's, but the *skeleton* lives here:
a dedicated non-pooled connection (deadpool#360 — pooled
connections cannot deliver), the `poll_message` loop fanning out
into a bounded broadcast channel (lag surfaced, not silent), the
two POC-pinned deadlock pitfalls respected (the poll loop running
before the first client query on the listener connection; the
`Client` kept alive for the listener's lifetime), and the
dynamic channel set (LISTENs re-issued from the live channel
list — the notify-listen task adds channels to it).
- The store struct holds pool + forwarder handle + schema name
behind the `Store` trait; `close()`/`Drop` tear down cleanly
(drop the listener connection, close the pool).
- **Trait stubs**: all `Store` methods return
`Err(Database("… wiring lands with the … task"))` — the wave-3
posture; the mechanism tasks replace them. `with_tx` surfaces the
`begin_tx` stub naturally.
- **Error mapping posture established here** (the seam task's
foundation): tokio-postgres/deadpool errors → `Error::Database`
with the source chain preserved (ADR-008 §5's opaque fallback) —
the two mappings (`pg_error`, `pool_error` or one helper) the
mechanism tasks reuse.
- **No `spawn_blocking` seam on this engine** — natively async
(Send+Sync client, POC #2 compile-probe); the engine crate's tokio
dep already carries what's needed. State the posture in the crate
docs (the SQLite engine's `# Posture` heading shape: multi-host
native per ADR-016, the listener budget line per deployment.md).
Tests against the harness server: open/close round-trip, bootstrap
ran (tables exist), pool bounds, forwarder skeleton up (a LISTEN
issued through it delivers — the minimal fanout proof, full forwarder
behavior in the notify-listen task), unreachable-server `open` fails
`Database`, opts flow (schema name lands in the created schema's
name; `synchronous_commit` observable via `SHOW`).
## Acceptance Criteria
- [ ] `PgOpts` with connection config, `schema` (default
`alkstore`), `max_size`, `synchronous_commit` (default on);
documented; not `#[non_exhaustive]`
- [ ] `open` boots pool + bootstrap + forwarder skeleton; failure at
any step is a typed `Database` error (source chain preserved)
- [ ] Forwarder skeleton: dedicated non-pooled connection, poll loop
before first query, bounded broadcast fanout, dynamic channel
set; the two POC deadlock pitfalls structurally excluded
- [ ] `close()`/`Drop` teardown clean (listener connection dropped,
pool closed); post-close ops fail closed
- [ ] Error-mapping helpers exist and are the documented reuse point
- [ ] Crate docs carry the `# Posture` statements (multi-host,
listener budget, no spawn_blocking)
- [ ] `cargo test -p alkstore-postgres` (harness server), clippy
`-D warnings`, fmt clean; gates green server-less (tests skip)
## References
- docs/architecture/engine-postgres.md (Connection architecture)
- docs/architecture/decisions/004-postgres-driver.md (the forwarder
shape, the two pinned pitfalls, RecyclingMethod::Fast)
- docs/architecture/decisions/008-contract-v1-pinning.md §6
- docs/architecture/deployment.md (budgets, synchronous_commit)
- docs/research/poc-pg-posture-findings.md (Sub-modules L/T; the
invocation note's isolation caveat)
- alkstore-sqlite/src/store.rs, opts.rs (the structural twin's shape)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+141
View File
@@ -0,0 +1,141 @@
---
id: pg-engine-queues
name: Postgres engine — queues (`Queue`, `FOR UPDATE SKIP LOCKED` claims, job handles, backoff curve)
status: pending
depends_on: [pg-engine-seam-tx, pg-engine-notify-listen]
scope: moderate
risk: medium
impact: component
level: implementation
tags: [wave-4, postgres-engine]
---
## Description
Implement the queue mechanism's pg arm — the re-derived queue
machinery (ADR-004/005's greenfield posture, pg-boss schema family as
design reference; ADR-010's depth) over the schema task's job/dead
tables and the forwarder's wake substrate:
- **`Store::queue(name, opts)`** — validated constructor (shared-
namespace kinds; construction stays free, queues are names not
registered objects). The handle carries its `QueueOpts` as the
stamp source.
- **`enqueue`** (auto-commit): resolves `EnqueueOpts` over the
handle's stamps via the seam task's `resolve_enqueue_opts` +
`stamps_with_override` (the pg engine's own `resolution.rs` —
one clock read per enqueue, delay-over-`run_at`, relative-
`expires`, `max_attempts` the one per-job override), `encode_payload`
at the seam, `INSERT` on a pool connection. Then **wake**:
`pg_notify` on the queue's wake channel (the mechanism-name-is-
the-channel realization) — the LISTEN-driven claim path's push
half.
- **`claim_one` / `claim_batch`** — exactly-once handout under
concurrency: the `FOR UPDATE SKIP LOCKED` claim (POC-pinned under
4×4 concurrency) on a pool connection, in one statement or a
CTE-scoped transaction — implementer's choice, but the
exactly-once property is what the tests pin (two workers, one row;
N×M concurrent drain, zero double-claims). Semantics per ADR-010:
ordering `priority DESC, run_at ASC, enqueue order`; `attempts +=
1` per claim (a reclaim consumes an attempt); the claim sets the
row's deadline from the job's own stamped `visibility_timeout_s`;
the pre-claim dead-letter sweep for exhausted reclaimables (the
substrate's D-12 shape — a worker dying right after its last
allowed claim dead-letters, not strands); `worker_id` stamps the
claimant column (non-empty validated, consumer-local — ADR-019 §2).
The **extent guard** (`n <= 0` → empty `Vec` at trait-impl entry,
ADR-023 §2). Claims return boxed `JobHandle`s (ops + row value).
- **`JobHandle` impl** (ADR-019 §3, ADR-010 §2's uniform validity
predicate — `processing` + unexpired claim deadline; refusal =
`Ok(false)`, never error):
- one-shot `ack`/`retry`/`fail` as `'static` boxed futures over the
consumed handle (mirroring `TxHandle::commit`'s shape);
repeatable `heartbeat(&self, extend)` — absolute reset of the
claim deadline (not additive), late heartbeat refused.
- `ack` deletes the row; `retry(err, None)` computes the queue's
**equal-jitter exponential curve** — the pg engine's own
implementation of ADR-010 §3's pinned form
(`base * 2^(attempt-1)`, jitter over `[half, cap]` integerized
inclusive, capped 3600 s; attempt index = the row's post-claim
count). **The jitter's randomness source is this task's flagged
choice**: the SQLite engine used a std-only `RandomState`-hash
draw (no `rand` dep, ADR-005's posture) — reuse that approach
unless a pg-native source is materially better, and **flag the
choice in Notes**. `retry(err, Some(d))` overrides; `fail(None)`
pins `"failed"`; `retry`'s exhaust string pins
`"max attempts exceeded"` (the engine-default strings land
identically on both engines — the backlog row's wording).
- Dead-letter moves (retry-at-budget, fail) write the dead table
with `last_error`/`died_at`; dead rows are `get_job`-visible.
- **`ack_batch(ids)`** — the batch ack applies the uniform predicate
per id (lapsed or non-claimed ids silently not counted; count
returned; no worker filter — the wave-3 D-31 delta's shape is the
contract's, not a substrate artifact).
- **`cancel(job_id)`** — unconditional delete (pending or processing,
not an interrupt); `Ok(false)` when row-less.
- **`get_job(job_id)`** — dead-visible pure read (pool connection;
`Option<Job>` with `last_error`/`died_at` on dead rows; the
resolved values — not raw opts — are what the row shows, ADR-020).
- **`sweep_expired()`** — both-states no-stranded-rows move (any
past-expiry row → dead with `last_error='expired'`) + retention
enforcement (`dead_letter_retention_s`, `None` = forever) —
ADR-010 §5; the queue-scoped handle form.
- **LISTEN-driven claim with re-poll safety net** (the engine's
default consumption posture, ADR-004): the *contract* claim ops are
pull-shaped (callers claim in their own loops); the wake substrate
makes those loops reactive — a consumer that `listen(queue_name)`s
and claims on wake gets the measured 3–6 ms path; the re-poll
safety net (interval-tunable, `PgOpts`-carried if wired —
implementer's choice, documented either way) covers LISTEN loss
(the no-replay hole). The engine's own tests exercise the
wake-driven claim loop (commit → notify → wake → claim, the POC's
property) to pin the posture; the contract surface itself stays
pull-shaped.
- **Closed-store posture** on all queue ops (the wave-3 shape).
## Acceptance Criteria
- [ ] `enqueue` resolves opts over the handle's stamps (the shared
resolution arithmetic; `max_attempts` the one override);
payload bytes exact; wake notify fires on the queue's channel
- [ ] `claim_one`/`claim_batch`: exactly-once under concurrency
(two-workers-one-row; 6×40 concurrent drain, zero
double-claims); ordering row pinned; reclaim consumes an
attempt; deadline from the row's stamp; extent guard
(`n <= 0` → empty `Vec`); pre-claim exhausted-reclaimable sweep
- [ ] `JobHandle`: validity predicate uniform (lapse refuses all ops,
`Ok(false)`); heartbeat absolute-reset + late-refused; ack
deletes; retry curve matches ADR-010 §3's range/cap
(statistical + cap tests, the SQLite task's shape); dead-letter
defaults (`"max attempts exceeded"`, `"failed"`, caller strings
verbatim); dead rows `get_job`-visible
- [ ] `ack_batch` per-id predicate (mixed lapsed/valid ids); `cancel`
unconditional both states; `sweep_expired` both-states +
retention (incl. forever-none)
- [ ] Wake-driven claim loop pinned end-to-end (enqueue → wake →
claim, cross-connection)
- [ ] Validation (empty/reserved queue names, empty `worker_id`) +
closed-store fail-closed on all queue ops
- [ ] `cargo test -p alkstore-postgres` (harness server), clippy
`-D warnings`, fmt clean; gates green server-less
## References
- docs/architecture/core-contract.md (queues section)
- docs/architecture/decisions/010-queue-semantics-depth.md (§1–§5, the depth)
- docs/architecture/decisions/019-mechanism-handle-surfaces.md §1–§3
- docs/architecture/decisions/020-enqueue-opt-semantics-and-bridges.md
- docs/architecture/decisions/023-fourth-review-round.md §2 (extent guard)
- docs/architecture/decisions/004-postgres-driver.md (re-derive posture, LISTEN-driven default)
- docs/architecture/queues.md (lifecycle, consumption posture)
- docs/research/reference-pgboss-rs-semantics.md (design reference)
- alkstore-sqlite/src/queue.rs (the structural twin — same semantics, own SQL)
- alkstore/src/queue.rs, job_handle.rs (the traits)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+154
View File
@@ -0,0 +1,154 @@
---
id: pg-engine-scheduler-outbox
name: Postgres engine — scheduler (`schedule`/`unschedule`/`run_schedules`) and outbox helper
status: pending
depends_on: [pg-engine-queues, pg-engine-locks]
scope: broad
risk: high
impact: component
level: implementation
tags: [wave-4, postgres-engine]
---
## Description
Implement the scheduler collapse's pg arm (ADR-009) and the outbox
helper (ADR-014/019 §5) — the wave's last mechanism pair, riding the
queues task's claim machinery and the locks task's lock machinery:
**Scheduler** (over the schema task's schedule table):
- **`schedule(name, spec, queue, payload, opts)`** — upsert by name;
entry-point validation on all three name-bearing arguments
(schedule name like a queue name; the **queue argument** identically
per ADR-021 §3 — a schedule cannot target a reserved name, so the
outbox backing queue stays reachable only through the outbox
surface); payload encoded at the seam (`Codec` propagation); the
spec grammar is `@every <n><unit>` only (`s|m|h|d`) — **the pg
engine owns its parser** (no shared substrate parser on this side;
the grammar is small and pinned; cron strings → `InvalidSpec`).
Rejection is pre-storage (`InvalidSpec` before any round trip).
- **`unschedule(name)`** — the unregister; `Ok(false)` when absent.
- **`run_schedules(stop: StopToken)`** — the leader loop, the
structural twin of the SQLite engine's runner with the pg deltas:
- **Leadership via the engine's own lock machinery** on the
reserved name `__alkstore_scheduler` (ADR-009 §1) — `try_lock`
with a per-runner-instance-unique owner token (the wave-3
finding's lesson: a shared constant would make two processes each
see "self" holding; the token is pid + subsec-nanos or
equivalent). TTL engine-internal (10 s), renewed at the top of
every loop iteration **and** on a cadence across the sleep (the
wave-3 posture: without in-sleep renewals, a boundary further out
than the TTL lapses its own lease by drift). A failed mid-sleep
renewal is not acted on at the sleep site — the loop's
top-of-iteration renew owns the loss decision.
- **Acquire-time refusal is `Err(LeadershipLost)`** — the same
matchable loss a mid-run loser returns (the wave-3 posture: no
runner-loop exists without the lock; the respawn recipe is
identical either way).
- **The tick**: fire enqueues + boundary advance commit together —
on pg this is one pool-checked-out transaction (`BEGIN` …
`COMMIT` on the held object, not `BEGIN IMMEDIATE` — pg's
default isolation gives the row-locked fire tx ADR-009 §4
names; the schedule row's `SELECT … FOR UPDATE` inside the tick
tx is the row lock that serializes rogue second tickers). The
tick body: boundary math over the registered specs (the
**one-firer-per-boundary** guarantee — at-least-once per elapsed
boundary, the **64-boundary catch-up cap with skip-forward**
past the cap, honker parity), each fire an ordinary enqueue
stamped with the `ScheduleOpts` resolved over the target queue's
*derived engine defaults* (no handle is open — ADR-020 §3), then
the soonest-boundary read for the sleep deadline.
- **Wake-driven tick advance** (the pg delta over the SQLite
runner's pure-sleep shape): the runner parks on the sleep
deadline, but an enqueue-wake on the scheduler's own wake channel
(or a plain bounded-wait interrupt — implementer's choice,
documented) may shorten the sleep so a newly-registered schedule
is noticed promptly; correctness never rides the wake (the
deadline always fires; the wake only shortens idle latency —
the SQLite runner's 60 s idle-tick posture is the acceptable
floor if the wake path proves fussy; flag the choice in Notes).
- **Clean stop** (`StopToken::cancel`) → `Ok(())`, the leadership
lock released cleanly; **loss** → `Err(LeadershipLost)` before
any tick.
- **Boundary math is this engine's own implementation** (ADR-012 §2's
one-owner rule — the substrate's `scheduler_tick`/`scheduler_soonest`
are SQLite-side): the `@every` interval arithmetic, the boundary
advance, the 64-cap catch-up, and the skip-forward. The contract
suite (wave 5) pins the two engines' boundary behavior to identical
outcomes.
**Outbox** (a helper over queues — no separate mechanism):
- **`Store::outbox(name)`** — validated constructor (empty →
`InvalidName`, reserved-prefixed → `ReservedName`); carries no opts
of its own (the backing queue's derived 60/5/5 set is
engine-documented, ADR-010 §3a).
- **`enqueue(payload, opts)`** — the auto-commit path into the
derived backing queue: derive `__alkstore_outbox:{name}` (the
reserved prefix makes it unreachable by `enqueue_tx`/`queue()` by
design), stamp the 60/5/5 set with the `EnqueueOpts::max_attempts`
override, full ADR-020 resolution, ride the queues task's enqueue
machinery.
- **`run_once(worker_id, delivery) -> bool`** — the pull op: claim
one job from the backing queue via the ordinary claim machinery
(the queues task's), run the delivery callable (core's `Delivery`
trait object — ADR-019 §5's pinned shape), `ack` on `Ok`,
`retry(err, None)` (the queue's curve; the delivery error's
`Display` string as the retry's exhaust string — the wave-3
posture) on `Err`, `false` on empty. **No engine-issued heartbeat
inside delivery** (honker parity, inherited deliberately — the
boxed `JobHandle`'s own `heartbeat` remains available to the
closure); the boxed `JobHandle` handed to the caller.
- **`outbox_enqueue_tx`** landed with the seam task; this task's
tests may extend its coverage (the commit-atomicity row is
suite-owned in wave 5).
## Acceptance Criteria
- [ ] `schedule`: validation triad (name/spec/queue), `@every`-only
grammar (cron → `InvalidSpec` pre-storage), upsert read-back;
the pg-owned parser pinned by tests (accepts/rejects the
grammar's exact surface)
- [ ] `run_schedules`: leadership via `__alkstore_scheduler` with a
per-instance owner token; acquire-time refusal =
`Err(LeadershipLost)`; tick fires + boundary advance in one
transaction (row-locked — a rogue second ticker cannot
double-fire); loss returns `Err(LeadershipLost)` before any
tick; clean stop releases the lock; in-sleep lease renewals
hold across boundaries further out than the TTL
- [ ] One-firer-per-boundary: two runners on the same db, exactly one
fire per boundary (the band test); 64-cap catch-up +
skip-forward with no in-window refire
- [ ] Fired jobs stamp the derived defaults resolved over
`ScheduleOpts` (`get_job`-visible)
- [ ] `outbox`: constructor validation; backing queue unreachable by
`enqueue_tx`/`queue()`; 60/5/5 stamps + `max_attempts`
override; `run_once` ok/err/false dispositions with the curve
gap asserted; in-delivery heartbeat renewal available;
exhaustion to dead with the delivery error string
- [ ] `cargo test -p alkstore-postgres` (harness server), clippy
`-D warnings`, fmt clean; gates green server-less
## References
- docs/architecture/decisions/009-scheduler-collapse.md (§1–§6)
- docs/architecture/decisions/014-outbox-tx-enqueue.md
- docs/architecture/decisions/019-mechanism-handle-surfaces.md §4, §5
- docs/architecture/decisions/020-enqueue-opt-semantics-and-bridges.md §3
- docs/architecture/decisions/021-tx-reads-and-value-shape-fixes.md §3
- docs/architecture/engine-postgres.md (Mapping the contract: scheduler/outbox)
- docs/architecture/queues.md (scheduler section)
- alkstore-sqlite/src/scheduler.rs, outbox.rs (the structural twin —
same semantics, own boundary math; its Notes carry the flake-fix
lessons: drive catch-up tests from far-out horizons so no live
boundary races the assertion window)
- alkstore/src/stop_token.rs, outbox.rs (the traits)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+110
View File
@@ -0,0 +1,110 @@
---
id: pg-engine-schema
name: Postgres engine — schema bootstrap (engine-owned schema, table family)
status: pending
depends_on: []
scope: narrow
risk: low
impact: component
level: implementation
tags: [wave-4, postgres-engine]
---
## Description
Stand up the pg engine's data model: the engine-owned PostgreSQL schema
and its table family — the dependency root every later wave-4 task
rides. Per ADR-010 §8 (one engine-owned schema, default `alkstore`,
queues as rows — no per-queue tables/schemas) and engine-postgres.md's
mapping table.
A `schema.rs` module in `alkstore-postgres`:
- **Schema name is engine configuration** (default `alkstore`) — it
rides `PgOpts` (the open task wires it; this task's bootstrap takes
the schema name as an argument).
- **Bootstrap is idempotent DDL**, runnable at open on any connection:
`CREATE SCHEMA IF NOT EXISTS` + `CREATE TABLE IF NOT EXISTS` +
indexes. All engine-owned objects live in the one schema; consumer
tables co-tenant the instance untouched (deployment.md's co-tenancy
posture).
- **Table family** (column sets follow the contract's value shapes —
the `#[doc(hidden)]` constructors' full field lists in
`alkstore/src/{job,stream,schedule}.rs` are the field lists of
record; ADR-019 §1/§3, ADR-015 §3, ADR-009 §3, ADR-021 §2):
- **job table** (live rows — pending and processing): queue name,
payload, enqueue/ready-time resolution columns (`run_at`,
`expires_at`), `priority`, the stamped `QueueOpts` columns
(`visibility_timeout_s`, `max_attempts`, `backoff_base_s`,
`dead_letter_retention_s` — ADR-010 §3a), `attempts`,
`worker_id`, `claimed_at`, `claim_expires_at`, `last_error`.
Claim-ordering index per ADR-010's ordering row (`priority DESC,
run_at ASC, enqueue order`).
- **dead table**: the job row's diagnosis fields + `died_at`,
`last_error`; retention index for the sweep's retention
enforcement.
- **stream events table**: `id BIGSERIAL` (the offset — engine-
assigned, monotone, immutable, never renumbered), stream name,
nullable `key` column (carried metadata, ADR-015 §1), payload,
`created_at` (unix-seconds, informational). The `(stream, id)`
index serving the `offset ASC` read paths.
- **offsets table**: (stream, consumer) → offset, the monotone
checkpoint upsert target.
- **schedule table** (ADR-009): name (unique), `@every` spec, target
queue, payload, the `ScheduleOpts` stamps, and the boundary-state
columns the tick math advances (the substrate's
`__alkstore_scheduler_tasks` shape is the reference —
`reference-honker-machinery.md`).
- **locks table**: name (unique), owner, expiry timestamp — the TTL
lock machinery's storage (the substrate's lock-table semantics,
re-derived; the locks task builds ops on it).
- **Payload columns store the exact encoded bytes** (`bytea`) — not
`JSONB`: JSONB normalization (key reordering, duplicate folding)
would break the contract suite's byte-identical-rows row (ADR-020
§4). The engine stores exactly what `encode_payload` returned.
- **No notifications table** — pg notify is native (`pg_notify`); the
SQLite notify-table carriage has no pg counterpart. The 8000-byte
check is client-side (the seam task), not storage-shaped.
- **No migration machinery in v1** — tables are created fresh per
schema; append-column migrations are the substrate's evolution
discipline, not a greenfield v1 need (note the posture in the module
docs; wave 6's release pass revisits if the deployment story
demands in-place upgrades).
Tests use a raw `tokio_postgres::Connection`/Client against the
harness server (no engine code yet): bootstrap runs twice cleanly
(idempotence), every table exists with its pinned columns, the
bigserial offset behaves (monotone, gap-on-delete), the claim-ordering
index exists.
## Acceptance Criteria
- [ ] `schema.rs` with an idempotent `bootstrap(conn, schema_name)`
creating the full table family + indexes in the one schema
- [ ] Column sets match the contract's value shapes (the constructors'
field lists; stamp columns per ADR-010 §3a; `claimed_at` per
ADR-021 §2)
- [ ] Payload columns are `bytea` (exact-bytes posture, JSONB
rejected — documented in the module docs with the reason)
- [ ] Schema name is a parameter (default `alkstore` decided at the
`PgOpts` layer, next task)
- [ ] Idempotence + table-shape tests green against the harness server
- [ ] `cargo test -p alkstore-postgres`, clippy `-D warnings`, fmt clean
## References
- docs/architecture/engine-postgres.md (Identity and posture; mapping table)
- docs/architecture/decisions/010-queue-semantics-depth.md §3a, §8
- docs/architecture/decisions/015-streams-depth.md §1, §3
- docs/architecture/decisions/009-scheduler-collapse.md §3
- docs/architecture/deployment.md (co-tenancy)
- docs/research/reference-honker-machinery.md (schema reference read)
- alkstore/src/job.rs, stream.rs, schedule.rs (the field lists of record)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+129
View File
@@ -0,0 +1,129 @@
---
id: pg-engine-seam-tx
name: Postgres engine — `begin_tx`, `TxHandle` (pooled-object handle), all eleven `*_tx` methods
status: pending
depends_on: [pg-engine-open-opts]
scope: moderate
risk: high
impact: component
level: implementation
tags: [wave-4, postgres-engine, seam]
---
## Description
Implement the transactional seam — the pg arm of ADR-007, the shape
every other mechanism task rides, and the wave's highest-risk piece
(the wave-3 review's writer-slot findings show exactly the defect
class seam error arms breed):
- **No bridge, no `spawn_blocking`** — the pg handle holds the pooled
object directly (tokio-postgres `Client` is `Send + Sync`, POC #2
compile-probe; ADR-007's pg arm). `begin_tx`: pool checkout +
`BEGIN`, return the boxed `PgTxHandle` holding
`Option<deadpool::Object>`; ops are straight `.await`s; commit/
rollback return the object to the pool.
- **Drop = rollback** (ADR-021 §4): the handle's `Drop` issues
`ROLLBACK` and re-pools the object — panic-safe (no panic escapes
drop). A failed rollback's disposition is pinned (the object is
discarded, not re-pooled — its state is unknowable; the pool
accounting stays exact, deployment.md's ADR-021 note). Ops after
commit/consumption fail closed with `Error::Closed` (the wave-3
fail-closed guard shape).
- **All eleven `*_tx` methods, for real** (nothing stubs — the schema
exists, the SQL is this task's to write):
- Writes: `enqueue_tx` (stamps the derived plain-queue defaults
300/3/5/none — the no-handle-open shape, ADR-020 §3a),
`publish_tx`, `publish_with_key_tx` (empty-`Some`-key →
`InvalidName`, ADR-015 §2), `notify_tx` (`SELECT pg_notify($1,
$2)` inside the caller's tx — delivers at commit, native
commit-atomicity, ADR-007; the 8000-byte client-side check fires
here and on the auto-commit twin — typed `PayloadTooLarge` with
the limit in the variant, before any round trip, ADR-016 §5),
`save_offset_tx` (monotone upsert), `outbox_enqueue_tx` (derives
the `__alkstore_outbox:{name}` backing queue name engine-side,
stamps the outbox's 60/5/5 set — ADR-014 §1).
- Reads (ADR-021 §1, read-your-own-writes through the same held
tx): `get_job_tx` (queue-scoped — a found row's queue column
must match the argument, the wave-3 seam's scoping rule),
`get_offset_tx`, `read_since_tx`, `read_from_consumer_tx`
(offset ASC; the extent guard `limit <= 0` → empty `Vec` at
trait-impl entry, ADR-023 §2).
- `commit(self: Box<Self>)`: `COMMIT` + re-pool. A failed COMMIT
discards the object (unknowable state) — the wave-3 review's
finding (c), pre-empted here.
- **Name validation at every entry point** via core's `validation`
helpers (shared-namespace kinds vs consumer-local; ADR-008 §4) —
before any round trip.
- **Payload encoding**: core's fallible `encode_payload` at the seam;
`?` the `Codec` error (ADR-023 §1). Decode via the `#[doc(hidden)]`
`from_row` constructors; malformed rows → `Codec` (the posture
guard).
- **The engine-side arithmetic lives here first** (one owner per
formula, ADR-012 §2 — the pg engine's own `resolution.rs`, the
SQLite one is reference not dependency): `now` (the single clock
read — `now()` at the SQL layer or one `std::time` read; pick one,
document it), `plain_queue_default_stamps()` (300/3/5/none),
`outbox_default_stamps()` (60/5/5), `outbox_backing_queue_name()`
(the reserved-prefix derivation), `resolve_enqueue_opts`
(delay-over-`run_at`, relative-`expires`, one clock read per
enqueue). The auto-commit mechanism tasks reuse these directly —
same shape as the SQLite engine's `resolution.rs`.
- **`pg_notify` payload carriage**: the notify payload is the
serde_json serialization's UTF-8 text (≤ 8000 bytes checked
client-side). Large payloads do NOT ride a side-table in v1 — the
typed rejection is the contract's answer (ADR-016 §5's occurrence
asymmetry; the outbox shape is the documented large-payload
pattern, not engine machinery).
The mechanism tasks then add what the tx path deliberately lacks: the
auto-commit handles, the forwarder-driven receivers, the backoff
curve, and the scheduler runner.
## Acceptance Criteria
- [ ] `begin_tx` checks out + `BEGIN`; concurrent `begin_tx` draws
distinct pooled objects (pool-bounded); pool exhaustion blocks
(deadpool's acquire semantics) — tested
- [ ] All eleven `*_tx` methods implemented with entry-point
validation; writes commit-atomic, reads see the tx's own writes
(read committed, both directions POC-pinned); opts resolution
(delay-over-run_at, relative-expires, 300/3/5/none and 60/5/5
stamp sets) pinned by tests
- [ ] `commit` commits + re-pools; drop (without commit) rolls back +
re-pools — no-ghosts tested for job rows, stream events,
notifications (a listener hears nothing from a rolled-back
`notify_tx`), and offset saves
- [ ] Drop is panic-safe; failed COMMIT/ROLLBACK arms discard the
object (pool accounting exact); ops-after-consume → `Closed`
- [ ] `notify_tx`/`notify`'s 8000-byte check: typed
`PayloadTooLarge { limit: 8000 }` client-side, before any round
trip; ≤ 8000 crosses
- [ ] `with_tx` (core's provided method) works end-to-end on this
engine: Ok ⇒ commit, Err ⇒ rollback — tested
- [ ] `encode_payload`'s `Codec` error propagates; extent guard on the
tx stream reads; `get_job_tx` queue-scoped
- [ ] `cargo test -p alkstore-postgres` (harness server), clippy
`-D warnings`, fmt clean; gates green server-less
## References
- docs/architecture/decisions/007-transactional-seam.md (pg arm)
- docs/architecture/decisions/021-tx-reads-and-value-shape-fixes.md §1, §2, §4
- docs/architecture/decisions/014-outbox-tx-enqueue.md §1
- docs/architecture/decisions/020-enqueue-opt-semantics-and-bridges.md §1–§4
- docs/architecture/decisions/016-deployment-honesty.md §5
- docs/architecture/decisions/023-fourth-review-round.md §1, §2
- docs/architecture/engine-postgres.md (Mapping the contract: begin_tx, handle ops)
- docs/research/poc-pg-posture-findings.md (Sub-module T — the shape
(a) evidence, read-your-writes, notify-at-commit)
- alkstore/src/tx.rs (the trait to implement)
- alkstore-sqlite/src/tx.rs, resolution.rs (the structural twin)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+126
View File
@@ -0,0 +1,126 @@
---
id: pg-engine-streams
name: Postgres engine — streams (`StreamHandle`, reads, subscribe, trim)
status: pending
depends_on: [pg-engine-seam-tx, pg-engine-notify-listen]
scope: moderate
risk: medium
impact: component
level: implementation
tags: [wave-4, postgres-engine]
---
## Description
Implement the stream mechanism's pg arm — the full `StreamHandle` trait
and the durable `subscribe` receiver, over the schema task's events/
offsets tables and the forwarder's wake substrate (ADR-015's depth,
engine-postgres.md's streams mapping row):
- **`publish` / `publish_with_key`** (auto-commit): pool checkout,
`INSERT` into the events table (bigserial id = the assigned offset;
nullable key column — carried metadata, empty-`Some`-key →
`InvalidName` per the tx path's rule), `encode_payload` at the seam
(`Codec` propagation), return to pool. Then **wake**: `pg_notify` on
the stream's wake channel (the mechanism-name-is-the-channel
realization pinned in the plan's Wave 4 section) — a separate
statement on the pooled connection (auto-commit: its own implicit
tx; the wake is best-effort, not commit-atomic with the insert —
the no-replay hole covers a gap between them; the durable row is
the truth).
- **`read_since` / `read_from_consumer`**: pool reads, `ORDER BY id
ASC` (global FIFO — bigserial offsets, immutable, never renumbered;
ADR-015 §4's ordering row), the **extent guard** (`limit <= 0` →
empty `Vec` at trait-impl entry, ADR-023 §2 — never `LIMIT -1`-class
dialect semantics; pg's `LIMIT` with a negative is an error, which
the guard makes unreachable). `read_from_consumer` anchors at the
consumer's stored offset.
- **`save_offset` / `get_offset`**: the monotone upsert (a save below
the stored checkpoint is a silent no-op — `GREATEST`-guarded upsert
or `WHERE offset > stored`; one owner of the monotonicity rule) and
the checkpoint read. Both save forms' `Err` arm carries `Database`
only (ADR-021 §5).
- **`trim_to(horizon)`**: `DELETE … WHERE id <= horizon` on a pool
connection (wakes nothing — ADR-015 §5's posture; the boundary
argument is total: a negative horizon deletes nothing,
idempotently), returning the deleted count. Surviving events keep
their offsets (gaps legal).
- **`subscribe(consumer)`** — the durable receiver, the structural
twin of the SQLite engine's bridge with the pg deltas:
- Attach-before-read ordering (subscribe to the stream's wake
channel *before* the initial cursor read — a commit cannot fall
between attach and wait).
- Initial cursor read of the consumer's *stored* offset; page-drain
to the tail (`offset ASC`); park on the wake feed (the forwarder
fanout for this stream's channel); re-drain on wake
(wake-driven with table re-read — events never require polling
to become visible, core-contract.md's streams section).
- **Close semantics (the pg arm, ADR-021 §5)**: the receiver stays
open across forwarder reconnects — reconnect-wakes arrive on the
wake feed and simply trigger re-drains (which is the recovery:
the no-replay hole's gap is healed by the re-drain reading the
durable rows); closes only at engine shutdown (terminal `None`).
- **Position semantics**: `position` = the last-*yielded* event's
offset (0 before any yield ⇒ `save_offset()` before any yield is
a no-op, ADR-021 §5); the bridge's internal cursor advances on
drained-from-storage events — a stalled consumer's
`save_offset()` never checkpoints unobserved events. Backpressure:
bounded channel + backpressure-aware send.
- `recv()`'s `Err` carries `Database` only (ADR-021 §5 — remap
non-`Database` read errors with the original as source, the
SQLite `database_only` helper's shape); `try_recv` arms mirror
the wake receiver (idle `Ok(None)`, `Err(Closed)` after
shutdown-close).
- The receiver's `read_since` (anchored, extent-guarded) and sync
`save_offset` (the last-yielded offset through the same monotone
op) per ADR-019 §6.
- **Validation**: stream names via `validate_shared_name` at
`store.stream()` and (defensively) the handle's reads; consumers
via `validate_local_name` (non-empty only — reserved prefixes legal
for consumers).
- **Closed-store posture**: every handle op checks the store's closed
flag first and fails closed with `Database` (the wave-3 shape).
## Acceptance Criteria
- [ ] `publish`/`publish_with_key` return the assigned bigserial
offset; key round-trips (`None` vs `Some`); `encode_payload`'s
`Codec` propagates; empty-`Some`-key rejected `InvalidName`
- [ ] Reads yield `offset ASC` (global FIFO); extent guard
(`limit <= 0` → empty `Vec`) at trait-impl entry; offsets
immutable (trim leaves gaps, never renumbers — tested)
- [ ] `save_offset` monotone (regression save = silent no-op);
`get_offset` reads the checkpoint; save `Err` = `Database` only
- [ ] `trim_to` exact-boundary (`<=`), returns deleted count, wakes
nothing, negative horizon deletes nothing; reads resume at the
horizon's first remaining row; saved offsets below the horizon
stay valid
- [ ] `subscribe`: replay from stored offset, wake-driven delivery,
position = last-yielded offset, backpressure holds, receiver
stays open across forwarder reconnects (reconnect-wake triggers
re-drain — the gap-healing property tested), terminal close at
engine shutdown only
- [ ] Restart durability: publish → close store → reopen →
`subscribe` resumes from the saved offset
- [ ] Validation + closed-store fail-closed on all entry points
- [ ] `cargo test -p alkstore-postgres` (harness server), clippy
`-D warnings`, fmt clean; gates green server-less
## References
- docs/architecture/core-contract.md (streams section)
- docs/architecture/decisions/015-streams-depth.md
- docs/architecture/decisions/019-mechanism-handle-surfaces.md §6
- docs/architecture/decisions/021-tx-reads-and-value-shape-fixes.md §5
- docs/architecture/decisions/023-fourth-review-round.md §2 (extent guard)
- docs/architecture/engine-postgres.md (Mapping the contract: streams)
- alkstore-sqlite/src/stream.rs (the structural twin)
- alkstore/src/stream_handle.rs, event_receiver.rs (the traits)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+94
View File
@@ -0,0 +1,94 @@
---
id: review-wave-4
name: Review gate — wave 4 (Postgres engine)
status: pending
depends_on: [pg-engine-integration]
scope: moderate
risk: low
impact: phase
level: review
tags: [wave-4, review]
---
## Description
Review the Postgres engine before wave 5 decomposes. Primary lens per
the plan's review gates: **engine-vs-contract conformance** — the same
discipline as the wave-3 gate, now against the second engine, with
the pg-specific machinery added. Wave 5 (the contract suite) pins
*cross-engine equivalence*; this gate checks the pg implementation
against the pinned contract text line by line.
Check:
- **Contract conformance**: every trait method's behavior against
core-contract.md's per-mechanism text — validation at entry points
(auto-commit AND tx paths), error arms (`Database`-only receivers,
`Codec` at payload seams, `LeadershipLost` vs clean stop), value
shapes (`Job`/`StreamEvent` field-for-field), the no-work-is-a-value
vocabulary (`None`/`false`/empty-`Vec` where pinned, never errors).
- **ADR-023 follow-through**: extent guards at trait-impl entry
(claims + all four stream reads); duration guards on both lock call
sites; `encode_payload` consumed fallibly at every call site.
- **ADR-016 §5**: the 8000-byte check is client-side, typed, before
any round trip, on both `notify` and `notify_tx`; the limit constant
has one owner.
- **Forwarder integrity** (the pg engine's own high-risk machinery —
the analogue of the wave-3 gate's seam-integrity lens): the two
POC-pinned deadlock pitfalls structurally excluded (poll loop
before first query; `Client` alive for the listener's lifetime);
reconnect backoff + re-LISTEN from the live channel set; the
synthetic reconnect-wake on the reserved string broadcast to every
subscriber; lag surfaced, not silent; receiver close semantics
exactly the pg arm (open across reconnects, close at shutdown
only, terminal, never reopens).
- **Seam integrity**: drop = rollback under panic; failed
COMMIT/ROLLBACK arms discard (not re-pool) the object — pool
accounting exact under error paths (deployment.md's ADR-021 note);
ops-after-consume fail closed; no `spawn_blocking` anywhere (the
natively-async posture — a stray bridge would be a finding).
- **Backoff curve + boundary math**: the engine-side formulas match
ADR-010 §3's pinned range/cap and ADR-009 §4's boundary guarantee
exactly (wave 5 pins cross-engine equivalence; this gate checks
this implementation against the text).
- **Schema posture**: one engine-owned schema; payload columns
`bytea` (exact bytes — no JSONB normalization); no per-queue
tables; bootstrap idempotent.
- **Backlog column**: the engine's suite rows present, stamped, green
against the pg factory; the `PayloadTooLarge` pg arm row present.
- **Family standards**: no panics, no unwrap/expect outside tests, no
inline comments (doc comments fine), module-per-file.
- **Gates**: full workspace build/test/clippy/fmt; the pg suite green
against the harness server.
## Acceptance Criteria
- [ ] Contract conformance spot-checked against the spec text (not
just the tests)
- [ ] ADR-023's numeric-domain + typed-encoding follow-through
verified at their sites
- [ ] Forwarder + seam integrity verified (code-read, not
test-name-trust)
- [ ] Backoff curve + scheduler boundary math match the pinned ADR
text
- [ ] Backlog column present, stamped, green against the pg factory
- [ ] All gates green (incl. against the harness server)
- [ ] Findings recorded; wave 5 decomposition may proceed
## References
- docs/plans/implementation.md (Review gates)
- docs/architecture/core-contract.md
- docs/architecture/engine-postgres.md
- docs/architecture/decisions/004-postgres-driver.md
- docs/architecture/decisions/016-deployment-honesty.md §5
- docs/architecture/decisions/023-fourth-review-round.md
- tasks/review-wave-3.md (the gate discipline's precedent)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion