alkblobs/tasks/engine-postgres.md

id, name, status, depends_on, scope, risk, impact, level, tags

id	name	status	depends_on	scope	risk	impact	level	tags
engine-postgres	postgres kv engine — the fleet-capable engine	completed		narrow	medium	component	implementation	wave-4 engines postgres

Description

Implement the postgres kv-tier engine (feature postgres, default-off; ADR-007; REQ-2/REQ-3) against the wave-1 trait (src/backend.rs) — the second SQL-backed engine and the fleet host (ADR-012 §4). The sqlite engine (src/sqlite.rs) is the structural twin: same contract, different driver cluster and posture deltas.

• Backend impl (src/postgres.rs, behind #[cfg(feature = "postgres")]): the seven methods plus put_staged, opaque byte keys/values, name() -> "postgres".
  • Schema bootstrap — one engine-owned PostgreSQL schema (recorded design point: the default schema name, the quoted-identifier boundary for every identifier entering SQL text — the alkstore quote_identifier pattern, alkstore-postgres src/schema.rs) holding the POC #5 finding B5 row shape: blobs (key bytea PRIMARY KEY, value bytea NOT NULL, size bigint NOT NULL) WITH (fillfactor = 90). Idempotent DDL (CREATE … IF NOT EXISTS) at the constructor; no migration machinery. The dedicated size column is the metadata-only size probe: SELECT size … WHERE key = $1 rides the PK index and never loads the value blob (ADR-008's size discipline).
  • Pool — deadpool-postgres over tokio-postgres (deadpool_postgres::Pool); pool size configurable with a named shipped default (recorded design point; the alkstore DEFAULT_MAX_SIZE = 8 precedent). Connection pooling is load-bearing, not an optimization (POC #5 B2: fresh sessions cost ~20 ms) — the module doc records the posture.
  • Per-connection prepared-statement discipline — prepared statements die with sessions (POC #5 B5 hit the "prepared statement s_1 does not exist" error exactly when a pool-recycled connection lost its statement); every engine statement rides prepare_cached on the pooled client so the cache is per-connection by construction. Recorded design point.
  • synchronous_commit parity knob — configurable per engine instance, wired on every pooled connection (the alkstore POC-verified per-session connect-options mechanics, alkstore-postgres src/opts.rs). Default on (the ship config); off is the documented sqlite-NORMAL-alike latency trade (B5: latency moves ~2-3× across settings). This is the parity knob with sqlite's synchronous=NORMAL-WAL shipped tier — recorded in the module doc as posture, not hidden.
  • CAS rows — INSERT … ON CONFLICT DO NOTHING first-writer-wins on both put forms; a re-put of an existing key is a no-op, never an overwrite (ADR-007 §3: the sqlite INSERT OR IGNORE mapping).
  • has — an EXISTS probe (SELECT EXISTS(…)) — never a value load. get — the materializing cursor body (kv family): the whole value loaded once (bounded by tier policy), len() from the row's size column, reads slice the materialized value composing backend.rs's clamped. put_staged — the degenerate in-memory stage (ADR-010 §1, the sqlite engine's shape): append extends the buffer, commit publishes via the same ON CONFLICT DO NOTHING clause and self-discards on failure, discard drops the buffer with zero residue.
  • delete — per-key rows-affected discipline (Err(Error::Missing) when absent). Plus the crate-internal batch-delete helper — DELETE … WHERE key = ANY($1) over a bytea[] parameter (ADR-007 §5's sweep-safety mechanics): the sweep's batched delete form, contract-tested in-module; its store-core consumer is wave 6's fleet sweep (scope-noted #![allow(dead_code)] until then — the wave-1 seam-shape discipline).
  • list — complete and well-formed as a stable cursor over a vacuuming table (ADR-007 §5): one statement's MVCC snapshot is stable for its duration (ghost rows from concurrent vacuuming never appear); the natural realization is the single SELECT key … ORDER BY key statement with tokio-postgres's row-streaming portal batches (bounded memory on huge pools) — the exact fetch shape is a recorded design point.
  • Virgin-store posture — the constructor creates the schema and tables; the tolerance is the sqlite engine's recorded read-only split (reads on a schema-less database map to absent/empty, writes surface the engine error — never silently creating schema). The test shape (a fixture connecting without running the bootstrap) mirrors src/sqlite.rs's new_tableless pattern. Recorded design point.
  • Autovacuum/fillfactor posture — stated in the module doc, not silence (POC #5 B5: autovacuum is a real maintenance surface for a churning CAS pool; fillfactor 90 measured; the ~40× small-tier storage overhead accepted per B6 — documented, not hidden). The B5 tuning numbers are POC-derived posture defaults (ADR-007 §6's deferred-cost pattern): the first real pg deployment verifies them against its media.
• Natively async — tokio-postgres is async: no spawn_blocking seam (the sqlite engine's blocking discipline does not carry; the pool is the concurrency posture). No blocking I/O on the async path; no unwrap/expect outside tests; thiserror errors surface as Error::Io via the engine's mapping discipline (Missing via the rows-affected/virgin-store mapping).
• Unit tests in-module: the method contract (virgin-store no-ops incl. the schema-less form, CAS re-put, Missing discipline, size exactness + metadata-only), the staged-put happy path and both hygiene forms (discard → zero residue; commit failure → self-discard — a #[cfg(test)] fail-commit knob standing in for media failure, the mem/sqlite pattern), list stability and completeness, the batch-delete helper, the synchronous_commit wiring. Tests need a live server — the alkstore harness convention: connection settings ride the environment (ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB), tests skip cleanly server-less, acceptance requires green against the server. The harness server: dockerized pglo-poc (postgres:16-alpine) on :15432, db blobs, user/password postgres/poc (verified live 2026-10-12).

The engine-state clause families (EngineWindowState + EngineFleetState) are engine-postgres-fleet (separate task); the constructor wiring is store-postgres-construction. This task ships the engine and its in-module unit tests only.

Acceptance Criteria

• src/postgres.rs implements Backend completely (seven methods + put_staged) behind feature postgres (default-off), with the materializing cursor body and the degenerate staging body
• One engine-owned schema (quoted-identifier boundary), the B5 row shape with fillfactor 90, idempotent bootstrap at the constructor; autovacuum/storage-overhead posture recorded in the module doc
• deadpool pool with a named shipped default size; per-connection prepare_cached discipline; the synchronous_commit parity knob wired on every pooled connection, configurable, default on
• CAS first-writer-wins on both put forms; re-put never overwrites
• size reads the dedicated size column via the PK index, never the value blob (metadata-only, unit-tested)
• list complete and well-formed as a stable MVCC-snapshot cursor (design point recorded); the batch-delete helper (key = ANY($1)) contract-tested, scope-noted until wave 6
• Stage hygiene holds in both forms (discard → zero residue; failed commit → self-discard), unit-tested
• Virgin-store reads are no-ops including the schema-less form (unit-tested); writes surface the error (the read-only split)
• Unit tests env-driven (ALKBLOBS_PG_*), skipping cleanly server-less; green against the harness server at acceptance
• cargo test (default — postgres is default-off, the engine compiles out) green; cargo test --features postgres green server-less (skipping); cargo clippy --all-targets -- -D warnings and cargo fmt --check green (default + --features postgres)

References

• docs/architecture/backends-and-dispatch.md (§kv tier — postgres engine)
• docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md (§4's B5 deltas as requirements; §5's sweep-safety proof; §6's deferred-cost pattern)
• docs/architecture/decisions/003-backend-contract-and-dispatch.md (the trait contract; the three numbered put rules' kv side)
• docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §1 (the whole-value put's pg body; the materializing cursor)
• docs/research/poc-postgres-kv-findings.md (findings B1-B6 — the row shape, the pooling discipline, the durability knob, the storage overhead)
• /workspace/@alkdev/alkstore/alkstore-postgres (the sibling harness patterns: src/opts.rs's PgOpts + per-session synchronous_commit wiring, src/schema.rs's bootstrap + quote_identifier, the suite factory's fresh-schema-per-open)
• src/backend.rs (the trait + seam contracts this implements)
• src/sqlite.rs (the structural twin's shapes this mirrors)

Notes

Implemented in src/postgres.rs (wave 4, this task's only code change; src/lib.rs's #[cfg(feature = "postgres")] pub mod postgres; gate pre-existed). Recorded design points, as resolved:

• Opts shape — a crate-internal PgOpts (schema / max_size / synchronous_commit), mirroring alkstore's src/opts.rs; the constructor takes the tokio-postgres Config-parseable connection string first, PgOpts second. max_size = 0 fails at entry, before any round trip; an empty schema string selects the default.
• Schema name — default "alkblobs"; every statement composes the table through quote_identifier (the alkstore boundary), so a hostile schema name (embedded ", reserved-word-shaped) is tested green. size/value/key are quoted in DDL per the alkstore hazard list, plain in DML (key, value, size non-reserved).
• Pool default — DEFAULT_MAX_SIZE = 8, RecyclingMethod::Fast (alkstore precedent); deadpool-postgres 0.14 / tokio-postgres 0.7.
• list fetch shape — single SELECT key … ORDER BY key through the deadpool client: one statement = one MVCC snapshot (vacuum-ghost-stable); tokio-postgres's row stream drains the server-side portal in bounded batches and the collection materializes only the key column. Ordering by key is engine-local (the trait leaves ordering unspecified).
• Virgin-store discriminant — SQLSTATE 42P01 (is_undefined_table), mapped on reads (absent/empty), delete (Missing), and delete_batch (Ok(0)); writes and staged commits surface Error::Io. The schema-less fixture connects via a #[cfg(test)] new_schemaless constructor (the sqlite new_tableless pattern) against a schema that was never bootstrapped.
• Stage hygiene mechanics — the staged buffer is std::mem::taken before the pool checkout, so a failed commit (injection or engine error) strands no staged bytes; the #[cfg(test)] fail_commit knob mirrors sqlite/mem.
• No new dependencies — postgres = ["dep:tokio-postgres", "dep:deadpool-postgres"] was already in Cargo.toml; list uses the client's collected query (row-stream under the hood) rather than pulling futures-util for query_raw.
• Test harness quirks — simple_query("SHOW …") returns SimpleQueryMessages (skip to the Row variant); size's EXPLAIN plan gate embeds the key as a '\x…' literal (PREPARE cannot wrap EXPLAIN) and asserts Index Scan + blobs_pkey.
• Isolation convention: one fresh schema per test (abt_{tag}_{pid}_{seq}), the alkstore suite-factory pattern, on the shared harness database.

Summary

Completed 2026-10-12 (wave 4). src/postgres.rs implements the seven-method Backend + put_staged behind feature postgres (default-off) — deadpool pool (default 8, load-bearing posture recorded), per-connection prepare_cached discipline, the synchronous_commit parity knob wired as connect options on every pooled connection (default on, configurable), one engine-owned schema (alkblobs, quoted-identifier boundary) holding the B5 row shape with fillfactor = 90 idempotently bootstrapped at construction, CAS first-writer-wins via ON CONFLICT DO NOTHING on both put forms, the metadata-only size probe (PK-index-tested via EXPLAIN), the degenerate in-memory stage with both hygiene forms, the single-MVCC-snapshot list cursor, and the crate-internal delete_batch (key = ANY($1) over bytea[], scope-noted until wave 6). Natively async — no spawn_blocking. 15 in-module unit tests, env-driven (ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB), skipping cleanly server-less; all green against the dockerized pglo-poc harness server (:15432). Verified: cargo test (69/19/3 ok), cargo test --features postgres (84/19/3 ok, server live), cargo test --all-features (139/54/4 ok), cargo clippy --all-targets and --all-features --all-targets -- -D warnings green, cargo fmt --check green, cargo doc --no-deps clean. Engine-state clause families remain engine-postgres-fleet; constructor wiring remains store-postgres-construction.