docs: specify POC #2 — Postgres engine posture (poc-pg-posture-spec.md)
Completes OQ-ST-03: one driver posture (tokio-postgres + deadpool, the POC #5/#7-validated stack) with three sub-modules — L (LISTEN plumbing: dedicated connection, multi-channel, payload boundary), T (tx-seam over the pool: caller-owned tx handle vs closure-scoped, both implemented and compared — direct OQ-ST-04 input), W (wake-vs-poll parity, LISTEN reconnect + the replay hole honesty). Property tests are the POC #1 suite's pg twin; seam probe mirrors the POC #1 workload for the cross-engine relative claim. Gate: commit-atomicity via in-tx NOTIFY (load-bearing), exactly-once claim, seam costs, LISTEN robustness - failure names the sqlx PgListener fallback posture. Out of scope: queue semantics depth (OQ-ST-05), pgboss-rs code adoption, multi-host stress (OQ-ST-08). Register row added, plan updated (POC #2 running closes OQ-ST-03).
This commit is contained in:
1 parent
299603b164
commit
e18281735e
2 files changed
+219
-6
No files matched your search
@@ -5,11 +5,10 @@ per-feature from the paused consumers' documents; phase-0 plan step 1
|
||||
done; streams upgraded to in-scope by operator-authority record; OQ-ST-02
|
||||
resolved: reactive-core + engine crates, operator decision; OQ-ST-03's
|
||||
SQLite half resolved by POC #1 (honker-core on our rusqlite — findings
|
||||
in poc-sqlite-posture-findings.md); OQ-ST-04/05/06 carry POC #1 input.
|
||||
Remaining: OQ-ST-03's Postgres half, OQ-ST-04 contract pinning,
|
||||
OQ-ST-05/06 pg-side ownership, OQ-ST-07/08.) Interface finding and
|
||||
driver tension from 2026-10-03 remain trusted-but-unverified working
|
||||
input except where POC #1 verified them.)
|
||||
in poc-sqlite-posture-findings.md); OQ-ST-04/05/06 carry POC #1 input;
|
||||
POC #2 (pg posture: LISTEN/pool/tx-seam) specified — running it closes
|
||||
OQ-ST-03. Interface finding and driver tension from 2026-10-03 remain
|
||||
trusted-but-unverified working input except where POC #1 verified them.)
|
||||
---
|
||||
|
||||
# alkstore — Phase 0 (Exploration)
|
||||
@@ -647,6 +646,7 @@ land in `docs/research/` here. Named per the OQ each feeds:
|
||||
| # | POC | Spec | Findings |
|
||||
|---|---|---|---|
|
||||
| 1 | SQLite engine posture: honker-core-on-rusqlite vs honker-extension-over-sqlx (async seam, watcher, transactional contract, packaging, interop) | [poc-sqlite-posture-spec.md](poc-sqlite-posture-spec.md) | [poc-sqlite-posture-findings.md](poc-sqlite-posture-findings.md) — **passed** (verdict: Arm A; ran 2026-10-04) |
|
||||
| 2 | Postgres engine posture: LISTEN/NOTIFY plumbing, tx-seam over the pool (caller-owned tx vs closure-scoped), wake-vs-poll claim latency, reconnect recovery | [poc-pg-posture-spec.md](poc-pg-posture-spec.md) | — (specified 2026-10-04; runs the POC #1 harness's pg twin) |
|
||||
|
||||
## Phase 0 plan
|
||||
|
||||
@@ -666,7 +666,12 @@ Expected sequence (deliberately rough):
|
||||
posture 1 (honker-core on our rusqlite); findings + constraint
|
||||
notes in the register row and OQ-ST-03.
|
||||
- Postgres side of OQ-ST-03 (tokio-postgres LISTEN/pool/tx-seam
|
||||
validation) — the next research round, likely POC #2.
|
||||
validation) — **POC #2 specified**:
|
||||
[poc-pg-posture-spec.md](poc-pg-posture-spec.md) — one driver
|
||||
posture (tokio-postgres + deadpool, the POC #5/#7-validated stack)
|
||||
with three sub-modules (listen plumbing / tx-seam shapes /
|
||||
wake-vs-poll parity); on pass, OQ-ST-03 closes with per-engine
|
||||
drivers.
|
||||
- OQ-ST-04's contract pinning — the honker-rs surface (§Interface
|
||||
finding) is the concrete starting artifact: pinning its contract
|
||||
costs less and is more honest than inventing a parallel shape —
|
||||
|
||||
@@ -0,0 +1,208 @@
|
||||
---
|
||||
status: spec
|
||||
title: "POC #2 — Postgres engine posture: LISTEN/NOTIFY wiring, the pool tx-seam, and reactive parity on tokio-postgres"
|
||||
last_updated: 2026-10-04
|
||||
---
|
||||
|
||||
# POC: Postgres engine posture — spec
|
||||
|
||||
> **POC register #2** (phase-0.md). Feeds OQ-ST-03's Postgres half (the
|
||||
> driver question's remaining open part) and OQ-ST-04's Postgres wake
|
||||
> side (the reactive contract). Code: standalone crate in the global
|
||||
> workspace (`/workspace/alkstore-pg-posture-poc`; harness conventions
|
||||
> cloned from `/workspace/alkstore-sqlite-posture-poc`). Findings land
|
||||
> in `docs/research/poc-pg-posture-findings.md` here regardless.
|
||||
> Server: dockerized postgres (POC-harness convention from the alkblobs
|
||||
> POCs — `postgres:16-alpine` on :15432, config knobs stated per-probe).
|
||||
|
||||
## What this POC must decide
|
||||
|
||||
OQ-ST-03's SQLite half is resolved (POC #1: honker-core on our
|
||||
rusqlite). The Postgres half asks the structural twin of the same
|
||||
question, with three differences that make it *not* a rerun:
|
||||
|
||||
1. **The queue machinery is greenfield either way.** pgboss-rs (the
|
||||
schema-family port) has no LISTEN/NOTIFY (verified 2026-10-03) and
|
||||
is sqlx-coupled; under OQ-ST-02's per-engine-crate split it brings
|
||||
no driver synergy. So the engine crate writes queue tables + claim/
|
||||
ack + retry/dead-letter itself on *some* driver — this POC grounds
|
||||
that build on tokio-postgres (the alkblobs POC #5/#7-validated
|
||||
stack: deadpool-postgres pool, per-connection prepared-statement
|
||||
discipline, `synchronous_commit` as the durability knob).
|
||||
2. **The wake mechanism is native server push** (LISTEN/NOTIFY), structurally
|
||||
different from SQLite's `data_version` polling: connection-bound,
|
||||
delivered on a dedicated connection, no retry/visibility semantics.
|
||||
The reactive contract's honest-shape question (OQ-ST-04) needs
|
||||
measured ground: what LISTEN plumbing costs, how it fails, and what
|
||||
a transaction-scoped interplay (NOTIFY fires only on commit of the
|
||||
sending transaction — the pg analogue of honker's notify-in-tx
|
||||
atomicity) looks like over a pool.
|
||||
3. **The tx-seam shape differs from SQLite's resolved one.** SQLite
|
||||
(POC #1): an owned writer-slot lease, ops via `spawn_blocking`.
|
||||
Postgres: interactive transactions handed out by the pool
|
||||
(`deadpool` gives `Object` → `transaction()`), LISTEN connections
|
||||
are *separate long-lived connections* — the `*_tx` seam and the
|
||||
listener lifecycle have to compose on one driver without one
|
||||
emulating the other's weaknesses.
|
||||
|
||||
The question, concretely: **does tokio-postgres carry the unified
|
||||
surface (enqueue_tx / claim / ack / notify_tx / listen / stream
|
||||
offset ops / lock ops) with the transactional property intact, at
|
||||
seam/pool/wake costs comparable to what POC #1 pinned for SQLite — and
|
||||
what are the honest posture deltas (pool sizing vs LISTEN connection
|
||||
dedication, `synchronous_commit` posture, notification payload limits)?**
|
||||
If yes, OQ-ST-03 closes with "per-engine drivers: rusqlite+honker-core
|
||||
(SQLite) / tokio-postgres+deadpool (Postgres)" and OQ-ST-04's
|
||||
contract-pinning has measured ground on both sides. If no — if LISTEN
|
||||
over deadpool proves fragile or the tx-seam shape fights the pool — the
|
||||
findings name the fight (sqlx's `PgListener` as the alternative is a
|
||||
*fallback posture*, evaluated only on that failure).
|
||||
|
||||
## The one arm, three sub-modules
|
||||
|
||||
Unlike POC #1's A/B split, this POC has one driver posture with three
|
||||
sub-modules to validate — the comparison axis is *within* the engine
|
||||
(listen-vs-poll, tx-seam shape), not between drivers:
|
||||
|
||||
### Sub-module L (listen plumbing)
|
||||
|
||||
- One dedicated `tokio-postgres` connection per process running
|
||||
`LISTEN <channel>`; notifications surface via the connection's
|
||||
`notifications()` stream. Fan-out to per-subscriber tokio mpsc
|
||||
channels (the `listen()` contract: multiple subscribers per channel,
|
||||
each getting every notification on that channel after attach).
|
||||
- Multi-channel: one LISTEN connection serving N channels (`LISTEN`
|
||||
accepts multiple registrations per connection) — measure whether the
|
||||
dedicated-connection-per-channel shape is ever warranted (connection
|
||||
budget vs fan-out complexity).
|
||||
- Payload: `pg_notify` carries ≤ 8000 bytes. Probe the boundary
|
||||
behavior (payload size limits, the "payload too large" error path)
|
||||
and record the honest contract: notify payloads are hints; large
|
||||
payloads ride a table row + the notify carries the row id (the
|
||||
honker-outbox shape, pg-side).
|
||||
|
||||
### Sub-module T (tx-seam over the pool)
|
||||
|
||||
- `deadpool-postgres` pool; `enqueue_tx`/`publish_tx`/`save_offset_tx`
|
||||
receive a caller-held transaction. The property the SQLite side
|
||||
pinned (business write + enqueue + notify in one caller tx, rollback
|
||||
drops both) must hold with `NOTIFY` issued inside the caller's tx
|
||||
(Postgres delivers only on commit — verify, don't assume).
|
||||
- The seam question: transaction *ownership* vs transaction *handle*.
|
||||
SQLite's lease shape cannot port (no slot model under a pool). Two
|
||||
candidate shapes, both implemented and compared:
|
||||
- **(a) caller-owned tx handle** — the trait hands the caller a
|
||||
`TxHandle` (deadpool `Transaction<'_>` wrapped) and `*_tx` methods
|
||||
take it by reference; the caller drives begin/commit.
|
||||
- **(b) closure-scoped** — the trait exposes
|
||||
`with_tx(|tx| async { ... })` (pool checkout + begin + commit/
|
||||
rollback inside the closure), and the business write also happens
|
||||
through the same handle — the caller never holds the raw
|
||||
transaction.
|
||||
Both are viable-looking; the POC measures and records the ergonomic
|
||||
and correctness trade (deadlock risk, spawn_blocking needs, what
|
||||
the caching-subscriber pattern wants). This is direct OQ-ST-04
|
||||
contract input.
|
||||
- Prepared-statement discipline per B2/B5 (POC #5): per-connection
|
||||
caching across pool recycling (deadpool's statement cache posture) —
|
||||
verify the "prepared statement s1 does not exist" failure mode does
|
||||
not recur with the queue SQL shape.
|
||||
|
||||
### Sub-module W (wake-vs-poll parity)
|
||||
|
||||
- The queue consumption path two ways: **(1) LISTEN-driven** — a
|
||||
queue-table notify on commit wakes claimants (the push channel
|
||||
pgboss-rs lacks; our machinery), with claim on wake + re-poll safety
|
||||
net; **(2) poll-only** — interval polling (the pgboss-rs posture).
|
||||
Measure claim latency (enqueue→claim) both ways, p50/p99.
|
||||
- `pg_notify` visibility timing: the notifier's NOTIFY is delivered to
|
||||
listeners at tx commit, but the *notifying* connection's subsequent
|
||||
reads may or may not see their own effects timing-wise — probe
|
||||
reader-visibility (the "read-your-writes over the pool" boundary
|
||||
under `read committed`, the honest default).
|
||||
- LISTEN connection failure modes: drop the listen connection
|
||||
mid-subscription (network kill), verify reconnect + the replay hole
|
||||
it implies (LISTEN has no replay — anything committed while the
|
||||
listener was down is *not* re-delivered; the honest contract is
|
||||
opaque-wake + re-read, so recovery = on reconnect, wake all
|
||||
subscribers once). Re-attach storm: N listeners × M channels on one
|
||||
re-connecting connection.
|
||||
|
||||
## Instruments
|
||||
|
||||
1. **Seam/cost probe** (`pgdiag-st-1`): the POC #1 seam workload's pg
|
||||
twin — `begin + enqueue_tx + commit` per iteration through the pool,
|
||||
n=3000, `synchronous_commit=on` (ship config) and `=off`
|
||||
(the B5 knob, both reported), p50/p90/p99. Compare shape (not
|
||||
absolute number) against POC #1's SQLite table — the cross-engine
|
||||
relative claim OQ-ST-04's contract must absorb. Plus the raw floor
|
||||
re-measure (prepared round-trip ~150–500 µs per POC #5 B2) as
|
||||
methodology cross-check.
|
||||
2. **Wake probe** (`pgdiag-st-2`): N=300 notify-commits at 25 ms
|
||||
spacing, listener attach → wake latency (p50/p99/max); burst-stress
|
||||
(30 rapid commits — LISTEN does not coalesce per-tick the way
|
||||
data_version polling does; measure whether bursts each deliver or
|
||||
the notifications() stream backs up); payload-size boundary;
|
||||
channel fan-out (4 subscribers, same channel, every one wakes).
|
||||
3. **Property tests** (`tests/contract.rs` — the POC #1 suite's pg
|
||||
twin, all shared assertions reused): commit-atomicity
|
||||
(enqueue+notify+business write, rollback drops all — ghost-claim
|
||||
check against the queue table), exactly-once claim under 4
|
||||
concurrent claimants, offset save/read through the caller tx,
|
||||
lock acquire/release/renew with TTL, listener-reconnect
|
||||
recovery (kill listen conn, commit during the gap, verify
|
||||
reconnect wakes + state re-read correct).
|
||||
4. **Poll-vs-listen claim-latency probe** (`pgdiag-st-3`): enqueue→
|
||||
claim latency under both consumption postures at 1/8/32 claimant
|
||||
workers — the honest comparison that justifies (or retires) the
|
||||
LISTEN-driven claim path as the engine's default.
|
||||
5. **Pool/posture probe** (`pgdiag-st-4`): deadpool pool sizing vs
|
||||
the LISTEN connection budget (does the dedicated listen conn count
|
||||
against max_size? separate pool?); fresh-session cost re-verify
|
||||
(~19–25 ms per B2) against the pooling discipline; `synchronous_commit`
|
||||
per-session posture (SET on checkout vs system config) mechanics.
|
||||
|
||||
## Decision gate
|
||||
|
||||
Sub-module findings constitute OQ-ST-03's Postgres half resolution iff:
|
||||
|
||||
- **Contract:** every shared property holds (the commit-atomicity
|
||||
property via in-tx NOTIFY is the load-bearing one; exactly-once claim
|
||||
under concurrency the second);
|
||||
- **Seam:** the caller-tx seam shape survives with measured costs
|
||||
(either candidate shape; the findings name the preferred one with
|
||||
evidence — that preference is OQ-ST-04 input, not a Phase 0 ADR);
|
||||
- **Wake:** LISTEN plumbing is robust enough to be the engine's wake
|
||||
story (reconnect recovery honest, fan-out correct, latency ≤ the
|
||||
sqlite watcher's order — LISTEN should beat poll; if it doesn't,
|
||||
that is a finding that reshapes OQ-ST-04's contract);
|
||||
|
||||
Failure on any: the findings name the failure precisely, the sqlx
|
||||
`PgListener` fallback posture gets its own (smaller) evaluation round,
|
||||
and OQ-ST-03's pg half stays open with a narrowed question.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- The queue *semantics* depth (retry/backoff/dead-letter/sweep tuning —
|
||||
OQ-ST-05's design work; this POC uses a minimal queue table: enqueue,
|
||||
claim, ack — enough for the property tests).
|
||||
- The full pg-boss schema family (job states beyond the minimal set,
|
||||
`pgboss-rs`'s DDL compatibility) — schema *design reference* status
|
||||
is OQ-ST-05; this POC does not adopt or validate pgboss-rs' code.
|
||||
- The sqlite engine (POC #1's ground; its seam/cost numbers are used
|
||||
only as the cross-engine reference points).
|
||||
- Multi-host stress (failover, pooling across machines) — OQ-ST-08's
|
||||
deployment-matrix work; this POC's dockerized server rides the
|
||||
established harness convention.
|
||||
- Streams' full contract (offset *save/read* verified; per-consumer
|
||||
replay windows/filtering is OQ-ST-04 design work).
|
||||
|
||||
## Register note
|
||||
|
||||
This POC is **#2** in alkstore's register (phase-0.md), specified
|
||||
2026-10-04. Sequencing rationale: it completes OQ-ST-03's driver
|
||||
resolution (the SQLite half is POC #1's) and gives OQ-ST-04's
|
||||
contract-pinning measured ground on *both* engines — after which the
|
||||
reactive-contract and ownership questions (OQ-ST-04/05/06) are
|
||||
paper-decisions over a filled evidence base, which is what Phase 1
|
||||
should inherit.
|
||||
Reference in new issue
Block a user