docs: open Phase 1 — architecture spec set over the Phase 0 evidence
docs/architecture/ now exists: README index, overview, five component specs (core-contract, engine-sqlite, engine-postgres, queues, deployment), ADR-001..007 carrying the Phase 0 resolved decisions (crate split, feature scope, per-engine drivers, dependency ownership, wake contract, tx seam), and the centralized open-questions tracker promotion: OQ-ST-01..08 mirror to OQ-01..08 one-to-one with statuses/resolutions carried; new Phase 1 questions append (OQ-09 scheduler collapse, OQ-10 contract versioning). Open Phase 1 work: OQ-04 contract pinning (high), OQ-05 queue semantics depth (high), OQ-06 honker-core quality read (high; fork-trigger gate), OQ-08 capability surface, OQ-09, OQ-10. Erratum fixed in phase-0 OQ-ST-04 (thread-affinity friction is SQLite-side, previously garbled as pg-side) and a stale scheduler- boundary pointer corrected in consumer-inventory.md. Two review passes run (findings: OQ-promotion numbering faithfulness, ADR back-reference sync) — all critical/warning findings resolved.
This commit is contained in:
1 parent
db73678090
commit
4391f6e879
17 files changed
+1812
-14
No files matched your search
@@ -0,0 +1,113 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-04
|
||||
---
|
||||
|
||||
# Postgres engine
|
||||
|
||||
The `alkstore-postgres` engine implements
|
||||
[core-contract.md](core-contract.md) on tokio-postgres +
|
||||
deadpool-postgres. This spec records WHAT the engine is internally —
|
||||
connection architecture, the LISTEN forwarder, queue machinery
|
||||
re-derivation — not code-level HOW. Decisions live in ADRs; contract
|
||||
obligations live in the core spec.
|
||||
|
||||
## Identity and posture
|
||||
|
||||
- Single driver, natively async: tokio-postgres 0.7.x +
|
||||
deadpool-postgres 0.14.x ([ADR-004](decisions/004-postgres-driver.md)).
|
||||
No bridge, no `spawn_blocking`; the tx handle holds the pooled
|
||||
object directly (client is `Send + Sync` — POC #2 compile-probe
|
||||
verified).
|
||||
- Multi-host by nature: connections are per-process state; nothing
|
||||
assumes a shared host ([ADR-004](decisions/004-postgres-driver.md),
|
||||
verified all-through-network in POC #2). See
|
||||
[deployment.md](deployment.md).
|
||||
- Queue machinery re-derived on this driver with the pg-boss schema
|
||||
family as design reference
|
||||
([ADR-005](decisions/005-dependency-ownership.md)); semantics depth
|
||||
is [queues.md](queues.md)'s work (OQ-05).
|
||||
|
||||
## Connection architecture
|
||||
|
||||
- **Pool** (deadpool) — queries, claims, and all non-transactional
|
||||
work. Per-connection statement cache, `RecyclingMethod::Fast`
|
||||
(no `DISCARD ALL` recycling; claim SQL re-prepared implicitly with
|
||||
zero errors at POC scale).
|
||||
- **Listener connection** — one dedicated, **non-pooled** connection
|
||||
per process that listens. Pooled connections cannot carry LISTEN
|
||||
(deadpool#360 — registration succeeds, delivery is impossible; the
|
||||
client-wrapper exposes no notification surface, source-verified and
|
||||
test-pinned as `pooled_listen_registers_but_cannot_deliver`). The
|
||||
listener is therefore a per-process budget line *outside* the pool:
|
||||
`max_size + 1` per LISTEN-ing process
|
||||
([deployment.md](deployment.md)).
|
||||
- **Forwarder** — the listener's loop: `poll_message`
|
||||
fanning out into a bounded broadcast channel (lag surfaced, not
|
||||
silent), re-LISTEN from the channel list after every reconnect
|
||||
(exponential backoff 50 ms → 2 s cap), and the synthetic
|
||||
reconnect-wake on the reserved channel
|
||||
([ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||||
- One listener serves N channels and N subscribers; re-attach is a
|
||||
broadcast re-subscribe (no server round-trips); per-channel
|
||||
connections are never warranted at this scale (POC-verified).
|
||||
|
||||
## Mapping the contract
|
||||
|
||||
| Contract piece | Engine realization |
|
||||
|---|---|
|
||||
| notify / listen | `pg_notify(...)` inside the caller's tx (delivers at commit — native commit-atomicity, [ADR-007](decisions/007-transactional-seam.md)); `listen()` via LISTEN on the forwarder's connection, fanout to receivers |
|
||||
| streams | durable event table + per-consumer offset cursors; `pg_notify` as the wake trigger ([ADR-006](decisions/006-wake-and-delivery-contract.md) mechanism split: durable row, LISTEN wake — the pg-boss-family shape) |
|
||||
| queues | re-derived queue table + `FOR UPDATE SKIP LOCKED` claim + LISTEN-driven wake with re-poll safety net (default consumption posture, measured 5–16× vs 50 ms poll; poll-only fallback) |
|
||||
| named locks | advisory-lock-semantics TTL locks (pg-boss-family design reference; depth in OQ-05's design work) |
|
||||
| scheduler / outbox | pg-boss-family design reference, per [queues.md](queues.md) |
|
||||
| begin_tx | pool checkout + `BEGIN`, returning the caller-held handle ([ADR-007](decisions/007-transactional-seam.md)) |
|
||||
| handle ops | straight `.await`s through the held object; commit/rollback returns the object to the pool |
|
||||
|
||||
## Owned failure modes (all test-pinned in POC #2)
|
||||
|
||||
Hand-rolling the forwarder means owning its pitfalls — they are
|
||||
*learned territory*, pinned as passing tests:
|
||||
|
||||
1. **Query-vs-poll starvation deadlock** — the poll loop must be
|
||||
running before the first client query on the listener connection.
|
||||
2. **Client-drop closes the server session** — a long-lived listener
|
||||
keeps its `Client` alive for the listener's lifetime.
|
||||
3. **The no-replay hole** — commit during a connection gap is never
|
||||
re-delivered; recovery = reconnect + synthetic wake + consumer
|
||||
re-read ([ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||||
The `!saw_replay` test pins the honesty.
|
||||
4. **Payload boundary** — `pg_notify` ≤ 8000 bytes; client-side
|
||||
checked, typed error before the round-trip. Large payloads ride a
|
||||
table row with the id in the notification (outbox shape).
|
||||
5. **Read-your-writes** — `read committed` default verified in both
|
||||
directions (in-tx and post-commit).
|
||||
|
||||
Constraint also carried: mixed rusqlite+sqlx binaries would need a
|
||||
vendored patch today — excluded by construction in this engine
|
||||
(single driver, [ADR-001](decisions/001-crate-split.md)), noted for
|
||||
the record in [ADR-003](decisions/003-sqlite-driver.md).
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|---|---|---|
|
||||
| [001](decisions/001-crate-split.md) | Crate split | single-driver engine crate |
|
||||
| [002](decisions/002-feature-scope.md) | Feature scope | which rows this engine serves |
|
||||
| [004](decisions/004-postgres-driver.md) | Driver | tokio-postgres + deadpool; hand-rolled forwarder; re-derived queues |
|
||||
| [005](decisions/005-dependency-ownership.md) | Ownership | published libs as-is; `postgres-notify` derive-not-adopt |
|
||||
| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract | LISTEN push, no replay, synthetic reconnect-wake |
|
||||
| [007](decisions/007-transactional-seam.md) | Tx seam | direct pooled-object handle, no bridging |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in
|
||||
[open-questions.md](open-questions.md). Key
|
||||
questions affecting this document:
|
||||
|
||||
- **OQ-09**: scheduler collapse into queues (shared with
|
||||
[queues.md](queues.md)) (open)
|
||||
- **OQ-05**: queue semantics depth — the pg-boss-family design-input
|
||||
work (open)
|
||||
- **OQ-08**: capability surface (shared with
|
||||
[deployment.md](deployment.md)) (open)
|
||||
Reference in new issue
Block a user