docs(tasks): decompose wave 4 — the postgres kv engine, the fleet host's full seam, the first SKIP-posture column
Five tasks against the wave-3 contract surface as reviewed (no amendments from the gate): - engine-postgres: the fleet-capable kv engine — the B5 row shape (fillfactor 90) in one engine-owned schema, the deadpool pool with per-connection prepared-statement discipline, the synchronous_commit parity knob wired per-connection, ON CONFLICT DO NOTHING CAS, the stable MVCC-snapshot list cursor, the batch-delete helper (key = ANY($1), consumer wave 6), the read-only virgin-store split - engine-postgres-fleet: the engine-state seam's full clause families (ADR-012 §5's postgres row — every family yes): gc_pins + gc_window, the joint tx ownership rule in the JointTx shape, pin_commit atomicity, pin_renew/pin_check/arbitrate, the pg advisory lock as the sweep lease's shipped provider; fleet activation stays wave 6 - store-postgres-construction: KvEngine::Postgres slots in, the KvCursor arm, the dual-tier/kv-only constructors (DSN + opts + threshold + window delay), the non-fleet posture recorded, the pg-pure build verified - suite-column-postgres: the first SKIP-posture column — the alkstore pg-factory pattern reused (fresh schema per open, CASCADE teardown, env-driven ALKBLOBS_PG_* settings), durable-schedule sweep rows, acceptance green against the harness server (pglo-poc :15432, verified live) - review-wave-4: the engine-wave gate (engine-vs-contract conformance) Verification: dependency graph validated (all ids resolve, no cycles, topological order clean); docs-only change. Refs: docs/plans/implementation.md §Wave 4
This commit is contained in:
glm-5.3-flash committed
1 parent
611d30a3c4
commit
7ea6dd6bcb
6 files changed
+752
-2
No files matched your search
@@ -0,0 +1,181 @@
|
||||
---
|
||||
id: engine-postgres
|
||||
name: postgres kv engine — the fleet-capable engine
|
||||
status: pending
|
||||
depends_on: []
|
||||
scope: narrow
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-4, engines, postgres]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Implement the `postgres` kv-tier engine (feature `postgres`,
|
||||
default-off; ADR-007; REQ-2/REQ-3) against the wave-1 trait
|
||||
(`src/backend.rs`) — the second SQL-backed engine and the fleet host
|
||||
(ADR-012 §4). The sqlite engine (`src/sqlite.rs`) is the structural
|
||||
twin: same contract, different driver cluster and posture deltas.
|
||||
|
||||
- **Backend impl** (`src/postgres.rs`, behind
|
||||
`#[cfg(feature = "postgres")]`): the seven methods plus
|
||||
`put_staged`, opaque byte keys/values, `name() -> "postgres"`.
|
||||
- **Schema bootstrap** — one engine-owned PostgreSQL schema
|
||||
(recorded design point: the default schema name, the
|
||||
quoted-identifier boundary for every identifier entering SQL text
|
||||
— the alkstore `quote_identifier` pattern, `alkstore-postgres`
|
||||
src/schema.rs) holding the POC #5 finding B5 row shape:
|
||||
`blobs (key bytea PRIMARY KEY, value bytea NOT NULL, size bigint
|
||||
NOT NULL) WITH (fillfactor = 90)`. Idempotent DDL
|
||||
(`CREATE … IF NOT EXISTS`) at the constructor; no migration
|
||||
machinery. The dedicated `size` column is the **metadata-only
|
||||
`size` probe**: `SELECT size … WHERE key = $1` rides the PK index
|
||||
and never loads the value blob (ADR-008's size discipline).
|
||||
- **Pool** — deadpool-postgres over tokio-postgres
|
||||
(`deadpool_postgres::Pool`); pool size configurable with a named
|
||||
shipped default (recorded design point; the alkstore
|
||||
`DEFAULT_MAX_SIZE = 8` precedent). **Connection pooling is
|
||||
load-bearing, not an optimization** (POC #5 B2: fresh sessions
|
||||
cost ~20 ms) — the module doc records the posture.
|
||||
- **Per-connection prepared-statement discipline** — prepared
|
||||
statements die with sessions (POC #5 B5 hit the "prepared
|
||||
statement s_1 does not exist" error exactly when a pool-recycled
|
||||
connection lost its statement); every engine statement rides
|
||||
`prepare_cached` on the pooled client so the cache is
|
||||
per-connection by construction. Recorded design point.
|
||||
- **`synchronous_commit` parity knob** — configurable per engine
|
||||
instance, wired on **every pooled connection** (the alkstore
|
||||
POC-verified per-session connect-options mechanics,
|
||||
`alkstore-postgres` src/opts.rs). Default `on` (the ship config);
|
||||
`off` is the documented sqlite-NORMAL-alike latency trade (B5:
|
||||
latency moves ~2-3× across settings). This is the parity knob
|
||||
with sqlite's `synchronous=NORMAL`-WAL shipped tier — recorded in
|
||||
the module doc as posture, not hidden.
|
||||
- **CAS rows** — `INSERT … ON CONFLICT DO NOTHING`
|
||||
first-writer-wins on both put forms; a re-put of an existing key
|
||||
is a no-op, never an overwrite (ADR-007 §3: the sqlite
|
||||
`INSERT OR IGNORE` mapping).
|
||||
- `has` — an **EXISTS probe** (`SELECT EXISTS(…)`) — never a value
|
||||
load. `get` — the **materializing cursor body** (kv family): the
|
||||
whole value loaded once (bounded by tier policy), `len()` from the
|
||||
row's size column, reads slice the materialized value composing
|
||||
`backend.rs`'s `clamped`. `put_staged` — the **degenerate
|
||||
in-memory stage** (ADR-010 §1, the sqlite engine's shape):
|
||||
`append` extends the buffer, `commit` publishes via the same
|
||||
`ON CONFLICT DO NOTHING` clause and self-discards on failure,
|
||||
`discard` drops the buffer with zero residue.
|
||||
- `delete` — per-key rows-affected discipline
|
||||
(`Err(Error::Missing)` when absent). **Plus the crate-internal
|
||||
batch-delete helper** — `DELETE … WHERE key = ANY($1)` over a
|
||||
`bytea[]` parameter (ADR-007 §5's sweep-safety mechanics): the
|
||||
sweep's batched delete form, contract-tested in-module; its
|
||||
store-core consumer is wave 6's fleet sweep (scope-noted
|
||||
`#![allow(dead_code)]` until then — the wave-1 seam-shape
|
||||
discipline).
|
||||
- `list` — **complete and well-formed as a stable cursor over a
|
||||
vacuuming table** (ADR-007 §5): one statement's MVCC snapshot is
|
||||
stable for its duration (ghost rows from concurrent vacuuming
|
||||
never appear); the natural realization is the single
|
||||
`SELECT key … ORDER BY key` statement with tokio-postgres's
|
||||
row-streaming portal batches (bounded memory on huge pools) — the
|
||||
exact fetch shape is a recorded design point.
|
||||
- **Virgin-store posture** — the constructor creates the schema and
|
||||
tables; the tolerance is the sqlite engine's recorded read-only
|
||||
split (reads on a schema-less database map to absent/empty, writes
|
||||
surface the engine error — never silently creating schema). The
|
||||
test shape (a fixture connecting without running the bootstrap)
|
||||
mirrors `src/sqlite.rs`'s `new_tableless` pattern. Recorded design
|
||||
point.
|
||||
- **Autovacuum/fillfactor posture** — stated in the module doc, not
|
||||
silence (POC #5 B5: autovacuum is a real maintenance surface for a
|
||||
churning CAS pool; fillfactor 90 measured; the ~40× small-tier
|
||||
storage overhead accepted per B6 — documented, not hidden). The
|
||||
B5 tuning numbers are POC-derived posture defaults (ADR-007 §6's
|
||||
deferred-cost pattern): the first real pg deployment verifies them
|
||||
against its media.
|
||||
- **Natively async** — tokio-postgres is async: **no `spawn_blocking`
|
||||
seam** (the sqlite engine's blocking discipline does not carry; the
|
||||
pool is the concurrency posture). No blocking I/O on the async path;
|
||||
no `unwrap`/`expect` outside tests; thiserror errors surface as
|
||||
`Error::Io` via the engine's mapping discipline (`Missing` via the
|
||||
rows-affected/virgin-store mapping).
|
||||
- **Unit tests** in-module: the method contract (virgin-store no-ops
|
||||
incl. the schema-less form, CAS re-put, Missing discipline, size
|
||||
exactness + metadata-only), the staged-put happy path and both
|
||||
hygiene forms (discard → zero residue; commit failure →
|
||||
self-discard — a `#[cfg(test)]` fail-commit knob standing in for
|
||||
media failure, the mem/sqlite pattern), list stability and
|
||||
completeness, the batch-delete helper, the `synchronous_commit`
|
||||
wiring. **Tests need a live server** — the alkstore harness
|
||||
convention: connection settings ride the environment
|
||||
(`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`), tests skip cleanly
|
||||
server-less, acceptance requires green against the server. The
|
||||
harness server: dockerized `pglo-poc` (`postgres:16-alpine`) on
|
||||
:15432, db `blobs`, user/password `postgres`/`poc` (verified live
|
||||
2026-10-12).
|
||||
|
||||
The engine-state clause families (`EngineWindowState` +
|
||||
`EngineFleetState`) are `engine-postgres-fleet` (separate task); the
|
||||
constructor wiring is `store-postgres-construction`. This task ships
|
||||
the engine and its in-module unit tests only.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `src/postgres.rs` implements `Backend` completely (seven methods
|
||||
+ `put_staged`) behind feature `postgres` (default-off), with
|
||||
the materializing cursor body and the degenerate staging body
|
||||
- [ ] One engine-owned schema (quoted-identifier boundary), the B5 row
|
||||
shape with fillfactor 90, idempotent bootstrap at the
|
||||
constructor; autovacuum/storage-overhead posture recorded in the
|
||||
module doc
|
||||
- [ ] deadpool pool with a named shipped default size;
|
||||
per-connection `prepare_cached` discipline; the
|
||||
`synchronous_commit` parity knob wired on every pooled
|
||||
connection, configurable, default `on`
|
||||
- [ ] CAS first-writer-wins on both put forms; re-put never overwrites
|
||||
- [ ] `size` reads the dedicated size column via the PK index, never
|
||||
the value blob (metadata-only, unit-tested)
|
||||
- [ ] `list` complete and well-formed as a stable MVCC-snapshot cursor
|
||||
(design point recorded); the batch-delete helper
|
||||
(`key = ANY($1)`) contract-tested, scope-noted until wave 6
|
||||
- [ ] Stage hygiene holds in both forms (discard → zero residue;
|
||||
failed commit → self-discard), unit-tested
|
||||
- [ ] Virgin-store reads are no-ops including the schema-less form
|
||||
(unit-tested); writes surface the error (the read-only split)
|
||||
- [ ] Unit tests env-driven (`ALKBLOBS_PG_*`), skipping cleanly
|
||||
server-less; green against the harness server at acceptance
|
||||
- [ ] `cargo test` (default — postgres is default-off, the engine
|
||||
compiles out) green; `cargo test --features postgres` green
|
||||
server-less (skipping); `cargo clippy --all-targets -- -D
|
||||
warnings` and `cargo fmt --check` green (default +
|
||||
`--features postgres`)
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/backends-and-dispatch.md (§kv tier — postgres
|
||||
engine)
|
||||
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
|
||||
(§4's B5 deltas as requirements; §5's sweep-safety proof; §6's
|
||||
deferred-cost pattern)
|
||||
- docs/architecture/decisions/003-backend-contract-and-dispatch.md
|
||||
(the trait contract; the three numbered put rules' kv side)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
|
||||
§1 (the whole-value put's pg body; the materializing cursor)
|
||||
- docs/research/poc-postgres-kv-findings.md (findings B1-B6 — the row
|
||||
shape, the pooling discipline, the durability knob, the storage
|
||||
overhead)
|
||||
- /workspace/@alkdev/alkstore/alkstore-postgres (the sibling harness
|
||||
patterns: src/opts.rs's PgOpts + per-session synchronous_commit
|
||||
wiring, src/schema.rs's bootstrap + quote_identifier, the suite
|
||||
factory's fresh-schema-per-open)
|
||||
- src/backend.rs (the trait + seam contracts this implements)
|
||||
- src/sqlite.rs (the structural twin's shapes this mirrors)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
Reference in new issue
Block a user