docs: specify POC #2 — Postgres engine posture (poc-pg-posture-spec.md)

Completes OQ-ST-03: one driver posture (tokio-postgres + deadpool, the
POC #5/#7-validated stack) with three sub-modules — L (LISTEN plumbing:
dedicated connection, multi-channel, payload boundary), T (tx-seam over
the pool: caller-owned tx handle vs closure-scoped, both implemented
and compared — direct OQ-ST-04 input), W (wake-vs-poll parity, LISTEN
reconnect + the replay hole honesty). Property tests are the POC #1
suite's pg twin; seam probe mirrors the POC #1 workload for the
cross-engine relative claim. Gate: commit-atomicity via in-tx NOTIFY
(load-bearing), exactly-once claim, seam costs, LISTEN robustness -
failure names the sqlx PgListener fallback posture. Out of scope:
queue semantics depth (OQ-ST-05), pgboss-rs code adoption, multi-host
stress (OQ-ST-08). Register row added, plan updated (POC #2 running
closes OQ-ST-03).
This commit is contained in:
glm-5.3-flash committed 2026-10-04 15:26:01 +00:00
1 parent 299603b164
commit e18281735e
2 files changed
+219 -6

No files matched your search

+11 -6
View File
@@ -5,11 +5,10 @@ per-feature from the paused consumers' documents; phase-0 plan step 1
done; streams upgraded to in-scope by operator-authority record; OQ-ST-02
resolved: reactive-core + engine crates, operator decision; OQ-ST-03's
SQLite half resolved by POC #1 (honker-core on our rusqlite — findings
in poc-sqlite-posture-findings.md); OQ-ST-04/05/06 carry POC #1 input.
Remaining: OQ-ST-03's Postgres half, OQ-ST-04 contract pinning,
OQ-ST-05/06 pg-side ownership, OQ-ST-07/08.) Interface finding and
driver tension from 2026-10-03 remain trusted-but-unverified working
input except where POC #1 verified them.)
in poc-sqlite-posture-findings.md); OQ-ST-04/05/06 carry POC #1 input;
POC #2 (pg posture: LISTEN/pool/tx-seam) specified — running it closes
OQ-ST-03. Interface finding and driver tension from 2026-10-03 remain
trusted-but-unverified working input except where POC #1 verified them.)
---
# alkstore — Phase 0 (Exploration)
@@ -647,6 +646,7 @@ land in `docs/research/` here. Named per the OQ each feeds:
| # | POC | Spec | Findings |
|---|---|---|---|
| 1 | SQLite engine posture: honker-core-on-rusqlite vs honker-extension-over-sqlx (async seam, watcher, transactional contract, packaging, interop) | [poc-sqlite-posture-spec.md](poc-sqlite-posture-spec.md) | [poc-sqlite-posture-findings.md](poc-sqlite-posture-findings.md) — **passed** (verdict: Arm A; ran 2026-10-04) |
| 2 | Postgres engine posture: LISTEN/NOTIFY plumbing, tx-seam over the pool (caller-owned tx vs closure-scoped), wake-vs-poll claim latency, reconnect recovery | [poc-pg-posture-spec.md](poc-pg-posture-spec.md) | — (specified 2026-10-04; runs the POC #1 harness's pg twin) |
## Phase 0 plan
@@ -666,7 +666,12 @@ Expected sequence (deliberately rough):
posture 1 (honker-core on our rusqlite); findings + constraint
notes in the register row and OQ-ST-03.
- Postgres side of OQ-ST-03 (tokio-postgres LISTEN/pool/tx-seam
validation) — the next research round, likely POC #2.
validation) — **POC #2 specified**:
[poc-pg-posture-spec.md](poc-pg-posture-spec.md) — one driver
posture (tokio-postgres + deadpool, the POC #5/#7-validated stack)
with three sub-modules (listen plumbing / tx-seam shapes /
wake-vs-poll parity); on pass, OQ-ST-03 closes with per-engine
drivers.
- OQ-ST-04's contract pinning — the honker-rs surface (§Interface
finding) is the concrete starting artifact: pinning its contract
costs less and is more honest than inventing a parallel shape —
+208
View File
@@ -0,0 +1,208 @@
---
status: spec
title: "POC #2 — Postgres engine posture: LISTEN/NOTIFY wiring, the pool tx-seam, and reactive parity on tokio-postgres"
last_updated: 2026-10-04
---
# POC: Postgres engine posture — spec
> **POC register #2** (phase-0.md). Feeds OQ-ST-03's Postgres half (the
> driver question's remaining open part) and OQ-ST-04's Postgres wake
> side (the reactive contract). Code: standalone crate in the global
> workspace (`/workspace/alkstore-pg-posture-poc`; harness conventions
> cloned from `/workspace/alkstore-sqlite-posture-poc`). Findings land
> in `docs/research/poc-pg-posture-findings.md` here regardless.
> Server: dockerized postgres (POC-harness convention from the alkblobs
> POCs — `postgres:16-alpine` on :15432, config knobs stated per-probe).
## What this POC must decide
OQ-ST-03's SQLite half is resolved (POC #1: honker-core on our
rusqlite). The Postgres half asks the structural twin of the same
question, with three differences that make it *not* a rerun:
1. **The queue machinery is greenfield either way.** pgboss-rs (the
schema-family port) has no LISTEN/NOTIFY (verified 2026-10-03) and
is sqlx-coupled; under OQ-ST-02's per-engine-crate split it brings
no driver synergy. So the engine crate writes queue tables + claim/
ack + retry/dead-letter itself on *some* driver — this POC grounds
that build on tokio-postgres (the alkblobs POC #5/#7-validated
stack: deadpool-postgres pool, per-connection prepared-statement
discipline, `synchronous_commit` as the durability knob).
2. **The wake mechanism is native server push** (LISTEN/NOTIFY), structurally
different from SQLite's `data_version` polling: connection-bound,
delivered on a dedicated connection, no retry/visibility semantics.
The reactive contract's honest-shape question (OQ-ST-04) needs
measured ground: what LISTEN plumbing costs, how it fails, and what
a transaction-scoped interplay (NOTIFY fires only on commit of the
sending transaction — the pg analogue of honker's notify-in-tx
atomicity) looks like over a pool.
3. **The tx-seam shape differs from SQLite's resolved one.** SQLite
(POC #1): an owned writer-slot lease, ops via `spawn_blocking`.
Postgres: interactive transactions handed out by the pool
(`deadpool` gives `Object` → `transaction()`), LISTEN connections
are *separate long-lived connections* — the `*_tx` seam and the
listener lifecycle have to compose on one driver without one
emulating the other's weaknesses.
The question, concretely: **does tokio-postgres carry the unified
surface (enqueue_tx / claim / ack / notify_tx / listen / stream
offset ops / lock ops) with the transactional property intact, at
seam/pool/wake costs comparable to what POC #1 pinned for SQLite — and
what are the honest posture deltas (pool sizing vs LISTEN connection
dedication, `synchronous_commit` posture, notification payload limits)?**
If yes, OQ-ST-03 closes with "per-engine drivers: rusqlite+honker-core
(SQLite) / tokio-postgres+deadpool (Postgres)" and OQ-ST-04's
contract-pinning has measured ground on both sides. If no — if LISTEN
over deadpool proves fragile or the tx-seam shape fights the pool — the
findings name the fight (sqlx's `PgListener` as the alternative is a
*fallback posture*, evaluated only on that failure).
## The one arm, three sub-modules
Unlike POC #1's A/B split, this POC has one driver posture with three
sub-modules to validate — the comparison axis is *within* the engine
(listen-vs-poll, tx-seam shape), not between drivers:
### Sub-module L (listen plumbing)
- One dedicated `tokio-postgres` connection per process running
`LISTEN <channel>`; notifications surface via the connection's
`notifications()` stream. Fan-out to per-subscriber tokio mpsc
channels (the `listen()` contract: multiple subscribers per channel,
each getting every notification on that channel after attach).
- Multi-channel: one LISTEN connection serving N channels (`LISTEN`
accepts multiple registrations per connection) — measure whether the
dedicated-connection-per-channel shape is ever warranted (connection
budget vs fan-out complexity).
- Payload: `pg_notify` carries ≤ 8000 bytes. Probe the boundary
behavior (payload size limits, the "payload too large" error path)
and record the honest contract: notify payloads are hints; large
payloads ride a table row + the notify carries the row id (the
honker-outbox shape, pg-side).
### Sub-module T (tx-seam over the pool)
- `deadpool-postgres` pool; `enqueue_tx`/`publish_tx`/`save_offset_tx`
receive a caller-held transaction. The property the SQLite side
pinned (business write + enqueue + notify in one caller tx, rollback
drops both) must hold with `NOTIFY` issued inside the caller's tx
(Postgres delivers only on commit — verify, don't assume).
- The seam question: transaction *ownership* vs transaction *handle*.
SQLite's lease shape cannot port (no slot model under a pool). Two
candidate shapes, both implemented and compared:
- **(a) caller-owned tx handle** — the trait hands the caller a
`TxHandle` (deadpool `Transaction<'_>` wrapped) and `*_tx` methods
take it by reference; the caller drives begin/commit.
- **(b) closure-scoped** — the trait exposes
`with_tx(|tx| async { ... })` (pool checkout + begin + commit/
rollback inside the closure), and the business write also happens
through the same handle — the caller never holds the raw
transaction.
Both are viable-looking; the POC measures and records the ergonomic
and correctness trade (deadlock risk, spawn_blocking needs, what
the caching-subscriber pattern wants). This is direct OQ-ST-04
contract input.
- Prepared-statement discipline per B2/B5 (POC #5): per-connection
caching across pool recycling (deadpool's statement cache posture) —
verify the "prepared statement s1 does not exist" failure mode does
not recur with the queue SQL shape.
### Sub-module W (wake-vs-poll parity)
- The queue consumption path two ways: **(1) LISTEN-driven** — a
queue-table notify on commit wakes claimants (the push channel
pgboss-rs lacks; our machinery), with claim on wake + re-poll safety
net; **(2) poll-only** — interval polling (the pgboss-rs posture).
Measure claim latency (enqueue→claim) both ways, p50/p99.
- `pg_notify` visibility timing: the notifier's NOTIFY is delivered to
listeners at tx commit, but the *notifying* connection's subsequent
reads may or may not see their own effects timing-wise — probe
reader-visibility (the "read-your-writes over the pool" boundary
under `read committed`, the honest default).
- LISTEN connection failure modes: drop the listen connection
mid-subscription (network kill), verify reconnect + the replay hole
it implies (LISTEN has no replay — anything committed while the
listener was down is *not* re-delivered; the honest contract is
opaque-wake + re-read, so recovery = on reconnect, wake all
subscribers once). Re-attach storm: N listeners × M channels on one
re-connecting connection.
## Instruments
1. **Seam/cost probe** (`pgdiag-st-1`): the POC #1 seam workload's pg
twin — `begin + enqueue_tx + commit` per iteration through the pool,
n=3000, `synchronous_commit=on` (ship config) and `=off`
(the B5 knob, both reported), p50/p90/p99. Compare shape (not
absolute number) against POC #1's SQLite table — the cross-engine
relative claim OQ-ST-04's contract must absorb. Plus the raw floor
re-measure (prepared round-trip ~150–500 µs per POC #5 B2) as
methodology cross-check.
2. **Wake probe** (`pgdiag-st-2`): N=300 notify-commits at 25 ms
spacing, listener attach → wake latency (p50/p99/max); burst-stress
(30 rapid commits — LISTEN does not coalesce per-tick the way
data_version polling does; measure whether bursts each deliver or
the notifications() stream backs up); payload-size boundary;
channel fan-out (4 subscribers, same channel, every one wakes).
3. **Property tests** (`tests/contract.rs` — the POC #1 suite's pg
twin, all shared assertions reused): commit-atomicity
(enqueue+notify+business write, rollback drops all — ghost-claim
check against the queue table), exactly-once claim under 4
concurrent claimants, offset save/read through the caller tx,
lock acquire/release/renew with TTL, listener-reconnect
recovery (kill listen conn, commit during the gap, verify
reconnect wakes + state re-read correct).
4. **Poll-vs-listen claim-latency probe** (`pgdiag-st-3`): enqueue→
claim latency under both consumption postures at 1/8/32 claimant
workers — the honest comparison that justifies (or retires) the
LISTEN-driven claim path as the engine's default.
5. **Pool/posture probe** (`pgdiag-st-4`): deadpool pool sizing vs
the LISTEN connection budget (does the dedicated listen conn count
against max_size? separate pool?); fresh-session cost re-verify
(~19–25 ms per B2) against the pooling discipline; `synchronous_commit`
per-session posture (SET on checkout vs system config) mechanics.
## Decision gate
Sub-module findings constitute OQ-ST-03's Postgres half resolution iff:
- **Contract:** every shared property holds (the commit-atomicity
property via in-tx NOTIFY is the load-bearing one; exactly-once claim
under concurrency the second);
- **Seam:** the caller-tx seam shape survives with measured costs
(either candidate shape; the findings name the preferred one with
evidence — that preference is OQ-ST-04 input, not a Phase 0 ADR);
- **Wake:** LISTEN plumbing is robust enough to be the engine's wake
story (reconnect recovery honest, fan-out correct, latency ≤ the
sqlite watcher's order — LISTEN should beat poll; if it doesn't,
that is a finding that reshapes OQ-ST-04's contract);
Failure on any: the findings name the failure precisely, the sqlx
`PgListener` fallback posture gets its own (smaller) evaluation round,
and OQ-ST-03's pg half stays open with a narrowed question.
## Out of scope
- The queue *semantics* depth (retry/backoff/dead-letter/sweep tuning —
OQ-ST-05's design work; this POC uses a minimal queue table: enqueue,
claim, ack — enough for the property tests).
- The full pg-boss schema family (job states beyond the minimal set,
`pgboss-rs`'s DDL compatibility) — schema *design reference* status
is OQ-ST-05; this POC does not adopt or validate pgboss-rs' code.
- The sqlite engine (POC #1's ground; its seam/cost numbers are used
only as the cross-engine reference points).
- Multi-host stress (failover, pooling across machines) — OQ-ST-08's
deployment-matrix work; this POC's dockerized server rides the
established harness convention.
- Streams' full contract (offset *save/read* verified; per-consumer
replay windows/filtering is OQ-ST-04 design work).
## Register note
This POC is **#2** in alkstore's register (phase-0.md), specified
2026-10-04. Sequencing rationale: it completes OQ-ST-03's driver
resolution (the SQLite half is POC #1's) and gives OQ-ST-04's
contract-pinning measured ground on *both* engines — after which the
reactive-contract and ownership questions (OQ-ST-04/05/06) are
paper-decisions over a filled evidence base, which is what Phase 1
should inherit.