--- id: engine-postgres name: postgres kv engine — the fleet-capable engine status: completed depends_on: [] scope: narrow risk: medium impact: component level: implementation tags: [wave-4, engines, postgres] --- ## Description Implement the `postgres` kv-tier engine (feature `postgres`, default-off; ADR-007; REQ-2/REQ-3) against the wave-1 trait (`src/backend.rs`) — the second SQL-backed engine and the fleet host (ADR-012 §4). The sqlite engine (`src/sqlite.rs`) is the structural twin: same contract, different driver cluster and posture deltas. - **Backend impl** (`src/postgres.rs`, behind `#[cfg(feature = "postgres")]`): the seven methods plus `put_staged`, opaque byte keys/values, `name() -> "postgres"`. - **Schema bootstrap** — one engine-owned PostgreSQL schema (recorded design point: the default schema name, the quoted-identifier boundary for every identifier entering SQL text — the alkstore `quote_identifier` pattern, `alkstore-postgres` src/schema.rs) holding the POC #5 finding B5 row shape: `blobs (key bytea PRIMARY KEY, value bytea NOT NULL, size bigint NOT NULL) WITH (fillfactor = 90)`. Idempotent DDL (`CREATE … IF NOT EXISTS`) at the constructor; no migration machinery. The dedicated `size` column is the **metadata-only `size` probe**: `SELECT size … WHERE key = $1` rides the PK index and never loads the value blob (ADR-008's size discipline). - **Pool** — deadpool-postgres over tokio-postgres (`deadpool_postgres::Pool`); pool size configurable with a named shipped default (recorded design point; the alkstore `DEFAULT_MAX_SIZE = 8` precedent). **Connection pooling is load-bearing, not an optimization** (POC #5 B2: fresh sessions cost ~20 ms) — the module doc records the posture. - **Per-connection prepared-statement discipline** — prepared statements die with sessions (POC #5 B5 hit the "prepared statement s_1 does not exist" error exactly when a pool-recycled connection lost its statement); every engine statement rides `prepare_cached` on the pooled client so the cache is per-connection by construction. Recorded design point. - **`synchronous_commit` parity knob** — configurable per engine instance, wired on **every pooled connection** (the alkstore POC-verified per-session connect-options mechanics, `alkstore-postgres` src/opts.rs). Default `on` (the ship config); `off` is the documented sqlite-NORMAL-alike latency trade (B5: latency moves ~2-3× across settings). This is the parity knob with sqlite's `synchronous=NORMAL`-WAL shipped tier — recorded in the module doc as posture, not hidden. - **CAS rows** — `INSERT … ON CONFLICT DO NOTHING` first-writer-wins on both put forms; a re-put of an existing key is a no-op, never an overwrite (ADR-007 §3: the sqlite `INSERT OR IGNORE` mapping). - `has` — an **EXISTS probe** (`SELECT EXISTS(…)`) — never a value load. `get` — the **materializing cursor body** (kv family): the whole value loaded once (bounded by tier policy), `len()` from the row's size column, reads slice the materialized value composing `backend.rs`'s `clamped`. `put_staged` — the **degenerate in-memory stage** (ADR-010 §1, the sqlite engine's shape): `append` extends the buffer, `commit` publishes via the same `ON CONFLICT DO NOTHING` clause and self-discards on failure, `discard` drops the buffer with zero residue. - `delete` — per-key rows-affected discipline (`Err(Error::Missing)` when absent). **Plus the crate-internal batch-delete helper** — `DELETE … WHERE key = ANY($1)` over a `bytea[]` parameter (ADR-007 §5's sweep-safety mechanics): the sweep's batched delete form, contract-tested in-module; its store-core consumer is wave 6's fleet sweep (scope-noted `#![allow(dead_code)]` until then — the wave-1 seam-shape discipline). - `list` — **complete and well-formed as a stable cursor over a vacuuming table** (ADR-007 §5): one statement's MVCC snapshot is stable for its duration (ghost rows from concurrent vacuuming never appear); the natural realization is the single `SELECT key … ORDER BY key` statement with tokio-postgres's row-streaming portal batches (bounded memory on huge pools) — the exact fetch shape is a recorded design point. - **Virgin-store posture** — the constructor creates the schema and tables; the tolerance is the sqlite engine's recorded read-only split (reads on a schema-less database map to absent/empty, writes surface the engine error — never silently creating schema). The test shape (a fixture connecting without running the bootstrap) mirrors `src/sqlite.rs`'s `new_tableless` pattern. Recorded design point. - **Autovacuum/fillfactor posture** — stated in the module doc, not silence (POC #5 B5: autovacuum is a real maintenance surface for a churning CAS pool; fillfactor 90 measured; the ~40× small-tier storage overhead accepted per B6 — documented, not hidden). The B5 tuning numbers are POC-derived posture defaults (ADR-007 §6's deferred-cost pattern): the first real pg deployment verifies them against its media. - **Natively async** — tokio-postgres is async: **no `spawn_blocking` seam** (the sqlite engine's blocking discipline does not carry; the pool is the concurrency posture). No blocking I/O on the async path; no `unwrap`/`expect` outside tests; thiserror errors surface as `Error::Io` via the engine's mapping discipline (`Missing` via the rows-affected/virgin-store mapping). - **Unit tests** in-module: the method contract (virgin-store no-ops incl. the schema-less form, CAS re-put, Missing discipline, size exactness + metadata-only), the staged-put happy path and both hygiene forms (discard → zero residue; commit failure → self-discard — a `#[cfg(test)]` fail-commit knob standing in for media failure, the mem/sqlite pattern), list stability and completeness, the batch-delete helper, the `synchronous_commit` wiring. **Tests need a live server** — the alkstore harness convention: connection settings ride the environment (`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`), tests skip cleanly server-less, acceptance requires green against the server. The harness server: dockerized `pglo-poc` (`postgres:16-alpine`) on :15432, db `blobs`, user/password `postgres`/`poc` (verified live 2026-10-12). The engine-state clause families (`EngineWindowState` + `EngineFleetState`) are `engine-postgres-fleet` (separate task); the constructor wiring is `store-postgres-construction`. This task ships the engine and its in-module unit tests only. ## Acceptance Criteria - [ ] `src/postgres.rs` implements `Backend` completely (seven methods + `put_staged`) behind feature `postgres` (default-off), with the materializing cursor body and the degenerate staging body - [ ] One engine-owned schema (quoted-identifier boundary), the B5 row shape with fillfactor 90, idempotent bootstrap at the constructor; autovacuum/storage-overhead posture recorded in the module doc - [ ] deadpool pool with a named shipped default size; per-connection `prepare_cached` discipline; the `synchronous_commit` parity knob wired on every pooled connection, configurable, default `on` - [ ] CAS first-writer-wins on both put forms; re-put never overwrites - [ ] `size` reads the dedicated size column via the PK index, never the value blob (metadata-only, unit-tested) - [ ] `list` complete and well-formed as a stable MVCC-snapshot cursor (design point recorded); the batch-delete helper (`key = ANY($1)`) contract-tested, scope-noted until wave 6 - [ ] Stage hygiene holds in both forms (discard → zero residue; failed commit → self-discard), unit-tested - [ ] Virgin-store reads are no-ops including the schema-less form (unit-tested); writes surface the error (the read-only split) - [ ] Unit tests env-driven (`ALKBLOBS_PG_*`), skipping cleanly server-less; green against the harness server at acceptance - [ ] `cargo test` (default — postgres is default-off, the engine compiles out) green; `cargo test --features postgres` green server-less (skipping); `cargo clippy --all-targets -- -D warnings` and `cargo fmt --check` green (default + `--features postgres`) ## References - docs/architecture/backends-and-dispatch.md (§kv tier — postgres engine) - docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md (§4's B5 deltas as requirements; §5's sweep-safety proof; §6's deferred-cost pattern) - docs/architecture/decisions/003-backend-contract-and-dispatch.md (the trait contract; the three numbered put rules' kv side) - docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §1 (the whole-value put's pg body; the materializing cursor) - docs/research/poc-postgres-kv-findings.md (findings B1-B6 — the row shape, the pooling discipline, the durability knob, the storage overhead) - /workspace/@alkdev/alkstore/alkstore-postgres (the sibling harness patterns: src/opts.rs's PgOpts + per-session synchronous_commit wiring, src/schema.rs's bootstrap + quote_identifier, the suite factory's fresh-schema-per-open) - src/backend.rs (the trait + seam contracts this implements) - src/sqlite.rs (the structural twin's shapes this mirrors) ## Notes > Implemented in `src/postgres.rs` (wave 4, this task's only code > change; `src/lib.rs`'s `#[cfg(feature = "postgres")] pub mod > postgres;` gate pre-existed). Recorded design points, as resolved: > > - **Opts shape** — a crate-internal `PgOpts` (`schema` / > `max_size` / `synchronous_commit`), mirroring alkstore's > `src/opts.rs`; the constructor takes the tokio-postgres > `Config`-parseable connection string first, `PgOpts` second. > `max_size = 0` fails at entry, before any round trip; an empty > schema string selects the default. > - **Schema name** — default `"alkblobs"`; every statement composes > the table through `quote_identifier` (the alkstore boundary), so a > hostile schema name (embedded `"`, reserved-word-shaped) is tested > green. `size`/`value`/`key` are quoted in DDL per the alkstore > hazard list, plain in DML (`key`, `value`, `size` non-reserved). > - **Pool default** — `DEFAULT_MAX_SIZE = 8`, `RecyclingMethod::Fast` > (alkstore precedent); deadpool-postgres 0.14 / tokio-postgres 0.7. > - **list fetch shape** — single `SELECT key … ORDER BY key` through > the deadpool client: one statement = one MVCC snapshot > (vacuum-ghost-stable); tokio-postgres's row stream drains the > server-side portal in bounded batches and the collection > materializes only the key column. Ordering by key is engine-local > (the trait leaves ordering unspecified). > - **Virgin-store discriminant** — SQLSTATE 42P01 > (`is_undefined_table`), mapped on reads (absent/empty), delete > (`Missing`), and `delete_batch` (`Ok(0)`); writes and staged > commits surface `Error::Io`. The schema-less fixture connects via > a `#[cfg(test)] new_schemaless` constructor (the sqlite > `new_tableless` pattern) against a schema that was never > bootstrapped. > - **Stage hygiene mechanics** — the staged buffer is > `std::mem::take`n before the pool checkout, so a failed commit > (injection or engine error) strands no staged bytes; the > `#[cfg(test)] fail_commit` knob mirrors sqlite/mem. > - **No new dependencies** — `postgres = ["dep:tokio-postgres", > "dep:deadpool-postgres"]` was already in `Cargo.toml`; `list` uses > the client's collected query (row-stream under the hood) rather > than pulling futures-util for `query_raw`. > - **Test harness quirks** — `simple_query("SHOW …")` returns > `SimpleQueryMessage`s (skip to the `Row` variant); `size`'s > EXPLAIN plan gate embeds the key as a `'\x…'` literal (PREPARE > cannot wrap EXPLAIN) and asserts `Index Scan` + `blobs_pkey`. > - Isolation convention: one fresh schema per test > (`abt_{tag}_{pid}_{seq}`), the alkstore suite-factory pattern, on > the shared harness database. ## Summary > Completed 2026-10-12 (wave 4). `src/postgres.rs` implements the > seven-method `Backend` + `put_staged` behind feature `postgres` > (default-off) — deadpool pool (default 8, load-bearing posture > recorded), per-connection `prepare_cached` discipline, the > `synchronous_commit` parity knob wired as connect options on every > pooled connection (default `on`, configurable), one engine-owned > schema (`alkblobs`, quoted-identifier boundary) holding the B5 row > shape with `fillfactor = 90` idempotently bootstrapped at > construction, CAS first-writer-wins via `ON CONFLICT DO NOTHING` on > both put forms, the metadata-only `size` probe (PK-index-tested via > EXPLAIN), the degenerate in-memory stage with both hygiene forms, the > single-MVCC-snapshot `list` cursor, and the crate-internal > `delete_batch` (`key = ANY($1)` over `bytea[]`, scope-noted until > wave 6). Natively async — no spawn_blocking. 15 in-module unit tests, > env-driven (`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`), skipping > cleanly server-less; all green against the dockerized pglo-poc > harness server (:15432). Verified: `cargo test` (69/19/3 ok), > `cargo test --features postgres` (84/19/3 ok, server live), `cargo > test --all-features` (139/54/4 ok), `cargo clippy --all-targets` and > `--all-features --all-targets -- -D warnings` green, `cargo fmt > --check` green, `cargo doc --no-deps` clean. Engine-state clause > families remain `engine-postgres-fleet`; constructor wiring remains > `store-postgres-construction`.