alkblobs/tasks/engine-postgres.md

---
id: engine-postgres
name: postgres kv engine — the fleet-capable engine
status: completed
depends_on: []
scope: narrow
risk: medium
impact: component
level: implementation
tags: [wave-4, engines, postgres]
---

## Description

Implement the `postgres` kv-tier engine (feature `postgres`,
default-off; ADR-007; REQ-2/REQ-3) against the wave-1 trait
(`src/backend.rs`) — the second SQL-backed engine and the fleet host
(ADR-012 §4). The sqlite engine (`src/sqlite.rs`) is the structural
twin: same contract, different driver cluster and posture deltas.

- **Backend impl** (`src/postgres.rs`, behind
  `#[cfg(feature = "postgres")]`): the seven methods plus
  `put_staged`, opaque byte keys/values, `name() -> "postgres"`.
  - **Schema bootstrap** — one engine-owned PostgreSQL schema
    (recorded design point: the default schema name, the
    quoted-identifier boundary for every identifier entering SQL text
    — the alkstore `quote_identifier` pattern, `alkstore-postgres`
    src/schema.rs) holding the POC #5 finding B5 row shape:
    `blobs (key bytea PRIMARY KEY, value bytea NOT NULL, size bigint
    NOT NULL) WITH (fillfactor = 90)`. Idempotent DDL
    (`CREATE … IF NOT EXISTS`) at the constructor; no migration
    machinery. The dedicated `size` column is the **metadata-only
    `size` probe**: `SELECT size … WHERE key = $1` rides the PK index
    and never loads the value blob (ADR-008's size discipline).
  - **Pool** — deadpool-postgres over tokio-postgres
    (`deadpool_postgres::Pool`); pool size configurable with a named
    shipped default (recorded design point; the alkstore
    `DEFAULT_MAX_SIZE = 8` precedent). **Connection pooling is
    load-bearing, not an optimization** (POC #5 B2: fresh sessions
    cost ~20 ms) — the module doc records the posture.
  - **Per-connection prepared-statement discipline** — prepared
    statements die with sessions (POC #5 B5 hit the "prepared
    statement s_1 does not exist" error exactly when a pool-recycled
    connection lost its statement); every engine statement rides
    `prepare_cached` on the pooled client so the cache is
    per-connection by construction. Recorded design point.
  - **`synchronous_commit` parity knob** — configurable per engine
    instance, wired on **every pooled connection** (the alkstore
    POC-verified per-session connect-options mechanics,
    `alkstore-postgres` src/opts.rs). Default `on` (the ship config);
    `off` is the documented sqlite-NORMAL-alike latency trade (B5:
    latency moves ~2-3× across settings). This is the parity knob
    with sqlite's `synchronous=NORMAL`-WAL shipped tier — recorded in
    the module doc as posture, not hidden.
  - **CAS rows** — `INSERT … ON CONFLICT DO NOTHING`
    first-writer-wins on both put forms; a re-put of an existing key
    is a no-op, never an overwrite (ADR-007 §3: the sqlite
    `INSERT OR IGNORE` mapping).
  - `has` — an **EXISTS probe** (`SELECT EXISTS(…)`) — never a value
    load. `get` — the **materializing cursor body** (kv family): the
    whole value loaded once (bounded by tier policy), `len()` from the
    row's size column, reads slice the materialized value composing
    `backend.rs`'s `clamped`. `put_staged` — the **degenerate
    in-memory stage** (ADR-010 §1, the sqlite engine's shape):
    `append` extends the buffer, `commit` publishes via the same
    `ON CONFLICT DO NOTHING` clause and self-discards on failure,
    `discard` drops the buffer with zero residue.
  - `delete` — per-key rows-affected discipline
    (`Err(Error::Missing)` when absent). **Plus the crate-internal
    batch-delete helper** — `DELETE … WHERE key = ANY($1)` over a
    `bytea[]` parameter (ADR-007 §5's sweep-safety mechanics): the
    sweep's batched delete form, contract-tested in-module; its
    store-core consumer is wave 6's fleet sweep (scope-noted
    `#![allow(dead_code)]` until then — the wave-1 seam-shape
    discipline).
  - `list` — **complete and well-formed as a stable cursor over a
    vacuuming table** (ADR-007 §5): one statement's MVCC snapshot is
    stable for its duration (ghost rows from concurrent vacuuming
    never appear); the natural realization is the single
    `SELECT key … ORDER BY key` statement with tokio-postgres's
    row-streaming portal batches (bounded memory on huge pools) — the
    exact fetch shape is a recorded design point.
  - **Virgin-store posture** — the constructor creates the schema and
    tables; the tolerance is the sqlite engine's recorded read-only
    split (reads on a schema-less database map to absent/empty, writes
    surface the engine error — never silently creating schema). The
    test shape (a fixture connecting without running the bootstrap)
    mirrors `src/sqlite.rs`'s `new_tableless` pattern. Recorded design
    point.
  - **Autovacuum/fillfactor posture** — stated in the module doc, not
    silence (POC #5 B5: autovacuum is a real maintenance surface for a
    churning CAS pool; fillfactor 90 measured; the ~40× small-tier
    storage overhead accepted per B6 — documented, not hidden). The
    B5 tuning numbers are POC-derived posture defaults (ADR-007 §6's
    deferred-cost pattern): the first real pg deployment verifies them
    against its media.
- **Natively async** — tokio-postgres is async: **no `spawn_blocking`
  seam** (the sqlite engine's blocking discipline does not carry; the
  pool is the concurrency posture). No blocking I/O on the async path;
  no `unwrap`/`expect` outside tests; thiserror errors surface as
  `Error::Io` via the engine's mapping discipline (`Missing` via the
  rows-affected/virgin-store mapping).
- **Unit tests** in-module: the method contract (virgin-store no-ops
  incl. the schema-less form, CAS re-put, Missing discipline, size
  exactness + metadata-only), the staged-put happy path and both
  hygiene forms (discard → zero residue; commit failure →
  self-discard — a `#[cfg(test)]` fail-commit knob standing in for
  media failure, the mem/sqlite pattern), list stability and
  completeness, the batch-delete helper, the `synchronous_commit`
  wiring. **Tests need a live server** — the alkstore harness
  convention: connection settings ride the environment
  (`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`), tests skip cleanly
  server-less, acceptance requires green against the server. The
  harness server: dockerized `pglo-poc` (`postgres:16-alpine`) on
  :15432, db `blobs`, user/password `postgres`/`poc` (verified live
  2026-10-12).

The engine-state clause families (`EngineWindowState` +
`EngineFleetState`) are `engine-postgres-fleet` (separate task); the
constructor wiring is `store-postgres-construction`. This task ships
the engine and its in-module unit tests only.

## Acceptance Criteria

- [ ] `src/postgres.rs` implements `Backend` completely (seven methods
      + `put_staged`) behind feature `postgres` (default-off), with
      the materializing cursor body and the degenerate staging body
- [ ] One engine-owned schema (quoted-identifier boundary), the B5 row
      shape with fillfactor 90, idempotent bootstrap at the
      constructor; autovacuum/storage-overhead posture recorded in the
      module doc
- [ ] deadpool pool with a named shipped default size;
      per-connection `prepare_cached` discipline; the
      `synchronous_commit` parity knob wired on every pooled
      connection, configurable, default `on`
- [ ] CAS first-writer-wins on both put forms; re-put never overwrites
- [ ] `size` reads the dedicated size column via the PK index, never
      the value blob (metadata-only, unit-tested)
- [ ] `list` complete and well-formed as a stable MVCC-snapshot cursor
      (design point recorded); the batch-delete helper
      (`key = ANY($1)`) contract-tested, scope-noted until wave 6
- [ ] Stage hygiene holds in both forms (discard → zero residue;
      failed commit → self-discard), unit-tested
- [ ] Virgin-store reads are no-ops including the schema-less form
      (unit-tested); writes surface the error (the read-only split)
- [ ] Unit tests env-driven (`ALKBLOBS_PG_*`), skipping cleanly
      server-less; green against the harness server at acceptance
- [ ] `cargo test` (default — postgres is default-off, the engine
      compiles out) green; `cargo test --features postgres` green
      server-less (skipping); `cargo clippy --all-targets -- -D
      warnings` and `cargo fmt --check` green (default +
      `--features postgres`)

## References

- docs/architecture/backends-and-dispatch.md (§kv tier — postgres
  engine)
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
  (§4's B5 deltas as requirements; §5's sweep-safety proof; §6's
  deferred-cost pattern)
- docs/architecture/decisions/003-backend-contract-and-dispatch.md
  (the trait contract; the three numbered put rules' kv side)
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
  §1 (the whole-value put's pg body; the materializing cursor)
- docs/research/poc-postgres-kv-findings.md (findings B1-B6 — the row
  shape, the pooling discipline, the durability knob, the storage
  overhead)
- /workspace/@alkdev/alkstore/alkstore-postgres (the sibling harness
  patterns: src/opts.rs's PgOpts + per-session synchronous_commit
  wiring, src/schema.rs's bootstrap + quote_identifier, the suite
  factory's fresh-schema-per-open)
- src/backend.rs (the trait + seam contracts this implements)
- src/sqlite.rs (the structural twin's shapes this mirrors)

## Notes

> Implemented in `src/postgres.rs` (wave 4, this task's only code
> change; `src/lib.rs`'s `#[cfg(feature = "postgres")] pub mod
> postgres;` gate pre-existed). Recorded design points, as resolved:
>
> - **Opts shape** — a crate-internal `PgOpts` (`schema` /
>   `max_size` / `synchronous_commit`), mirroring alkstore's
>   `src/opts.rs`; the constructor takes the tokio-postgres
>   `Config`-parseable connection string first, `PgOpts` second.
>   `max_size = 0` fails at entry, before any round trip; an empty
>   schema string selects the default.
> - **Schema name** — default `"alkblobs"`; every statement composes
>   the table through `quote_identifier` (the alkstore boundary), so a
>   hostile schema name (embedded `"`, reserved-word-shaped) is tested
>   green. `size`/`value`/`key` are quoted in DDL per the alkstore
>   hazard list, plain in DML (`key`, `value`, `size` non-reserved).
> - **Pool default** — `DEFAULT_MAX_SIZE = 8`, `RecyclingMethod::Fast`
>   (alkstore precedent); deadpool-postgres 0.14 / tokio-postgres 0.7.
> - **list fetch shape** — single `SELECT key … ORDER BY key` through
>   the deadpool client: one statement = one MVCC snapshot
>   (vacuum-ghost-stable); tokio-postgres's row stream drains the
>   server-side portal in bounded batches and the collection
>   materializes only the key column. Ordering by key is engine-local
>   (the trait leaves ordering unspecified).
> - **Virgin-store discriminant** — SQLSTATE 42P01
>   (`is_undefined_table`), mapped on reads (absent/empty), delete
>   (`Missing`), and `delete_batch` (`Ok(0)`); writes and staged
>   commits surface `Error::Io`. The schema-less fixture connects via
>   a `#[cfg(test)] new_schemaless` constructor (the sqlite
>   `new_tableless` pattern) against a schema that was never
>   bootstrapped.
> - **Stage hygiene mechanics** — the staged buffer is
>   `std::mem::take`n before the pool checkout, so a failed commit
>   (injection or engine error) strands no staged bytes; the
>   `#[cfg(test)] fail_commit` knob mirrors sqlite/mem.
> - **No new dependencies** — `postgres = ["dep:tokio-postgres",
>   "dep:deadpool-postgres"]` was already in `Cargo.toml`; `list` uses
>   the client's collected query (row-stream under the hood) rather
>   than pulling futures-util for `query_raw`.
> - **Test harness quirks** — `simple_query("SHOW …")` returns
>   `SimpleQueryMessage`s (skip to the `Row` variant); `size`'s
>   EXPLAIN plan gate embeds the key as a `'\x…'` literal (PREPARE
>   cannot wrap EXPLAIN) and asserts `Index Scan` + `blobs_pkey`.
> - Isolation convention: one fresh schema per test
>   (`abt_{tag}_{pid}_{seq}`), the alkstore suite-factory pattern, on
>   the shared harness database.

## Summary

> Completed 2026-10-12 (wave 4). `src/postgres.rs` implements the
> seven-method `Backend` + `put_staged` behind feature `postgres`
> (default-off) — deadpool pool (default 8, load-bearing posture
> recorded), per-connection `prepare_cached` discipline, the
> `synchronous_commit` parity knob wired as connect options on every
> pooled connection (default `on`, configurable), one engine-owned
> schema (`alkblobs`, quoted-identifier boundary) holding the B5 row
> shape with `fillfactor = 90` idempotently bootstrapped at
> construction, CAS first-writer-wins via `ON CONFLICT DO NOTHING` on
> both put forms, the metadata-only `size` probe (PK-index-tested via
> EXPLAIN), the degenerate in-memory stage with both hygiene forms, the
> single-MVCC-snapshot `list` cursor, and the crate-internal
> `delete_batch` (`key = ANY($1)` over `bytea[]`, scope-noted until
> wave 6). Natively async — no spawn_blocking. 15 in-module unit tests,
> env-driven (`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`), skipping
> cleanly server-less; all green against the dockerized pglo-poc
> harness server (:15432). Verified: `cargo test` (69/19/3 ok),
> `cargo test --features postgres` (84/19/3 ok, server live), `cargo
> test --all-features` (139/54/4 ok), `cargo clippy --all-targets` and
> `--all-features --all-targets -- -D warnings` green, `cargo fmt
> --check` green, `cargo doc --no-deps` clean. Engine-state clause
> families remain `engine-postgres-fleet`; constructor wiring remains
> `store-postgres-construction`.