alkblobs/tasks/engine-pg-lo.md

id, name, status, depends_on, scope, risk, impact, level, tags

id	name	status	depends_on	scope	risk	impact	level	tags
engine-pg-lo	pg-lo large engine — Large Objects behind the Backend trait	pending		moderate	medium	component	implementation	wave-5 engines pg-lo postgres

Description

Implement the pg-lo large-tier engine (feature pg-lo, default-off; ADR-009, admitted on POC #7 — findings C1–C7 are the evidence base) against the wave-1 trait (src/backend.rs) in src/pg_lo.rs (the 2-line module-doc stub behind the pre-existing #[cfg(feature = "pg-lo")] gate). The postgres kv engine (src/postgres.rs, wave 4) is the structural twin for the driver posture (pool, prepared statements, schema bootstrap, error mapping); the local engine (src/local.rs) is the structural twin for the large-tier bodies (streaming read cursor, staged put). pg-lo does not imply postgres (ADR-013 §4's consequence; the Cargo.toml feature carries its own driver deps) — every shared piece must compile under --no-default-features --features pg-lo (the pg-lo-pure build).

• Engine shape (POC #7 C1 is the contract) — two artifacts: LOs hold the bytes; the companion table lo_entries (key bytea PRIMARY KEY, loid oid, size bigint, committed_at) is the contract authority — list(), size(), has(), CAS-existence are all table-level. Never trust or read pg_largeobject for addressing (the C6 sweep validates against the table only). One engine-owned PostgreSQL schema (the postgres engine's bootstrap pattern: idempotent DDL at the constructor, the quoted-identifier boundary, no migration machinery). The default schema name and the consolidation shape's schema sharing (both engines on one pg instance) are recorded design points.
• Driver posture (the postgres engine's, reused) — deadpool-postgres over tokio-postgres, pool size configurable with a named shipped default, per-connection prepare_cached discipline (statements die with sessions), natively async — no spawn_blocking seam, no blocking I/O on the async path. Shared-helper design point: the postgres engine's crate-internal helpers (quote_identifier, the SQLSTATE 42P01 discriminant, pg_err, the i64-nanos timestamp helpers) live behind #[cfg(feature = "postgres")] today — decide and record the sharing shape (the natural one: a crate-internal pg-common module gated cfg(any(feature = "postgres", feature = "pg-lo")), one module per file, so the pg-lo-pure build compiles without duplication). The PgOpts question rides the same point: it is pub and re-exported behind postgres (wave 4's constructor signature put it on the public surface) — widening its gate to any(postgres, pg-lo) (re-export gate widened with it) vs a pg-lo-local opts type is a recorded design point; whether pg-lo carries the synchronous_commit knob at all (the POC ran the server with it off; the kv parity knob's posture may or may not carry) is recorded with it.
• Put = one transaction (ADR-009 §2) — BEGIN → lo_create → chunked lowrite → companion-row insert → COMMIT. 512 KiB statements (POC #7 C2's measured knee; statement count, not LOBLKSIZE, is what costs) as a named shipped constant. CAS: companion-table existence check first — a re-put of an existing key is a no-op tx, never an overwrite (the C1 shape). The store core's put loop hands 64 KiB windows (PUT_CHUNK); the engine aggregates appends into 512 KiB lowrite statements (recorded design point).
• Staged put (the engine-level put_staged contract) — the staging cursor body is a held tx: handout opens the checkout + tx + lo_create; append rides the chunked lowrite aggregation; commit publishes (companion-row insert + COMMIT — the row and the LO's pages land in one step, zero crash orphans structurally, C6); discard rolls back and returns the checkout. Every failure path converges on rollback — stage hygiene holds in both forms (discard → zero residue: no LO pages, no row; failed commit → self-discard), exact-count tested (the #[cfg(test)] fail-commit knob standing in for media failure, the mem/sqlite/postgres pattern). Scope boundary: ADR-010 §1's "a tx the store core holds" is the fleet joint form (wave 6's consolidation mechanics — ADR-012 §4/§5's ruling that pg-lo contributes LO writes + companion-row insert into the store-held tx); its API shape is a wave-6 design point. This task ships the engine-level staging cursor only.
• Get = descriptorless lo_get(oid, off, len) windows (ADR-009 §3, the C4 posture) — the read cursor closes over the companion row (oid + length; content immutable, so the row is authoritative), not over a tx-scoped descriptor. Range reads are single stateless statements, pool-friendly. len() reads the companion row — metadata only, never LO content (ADR-008's discipline). A range read beyond content length fails cleanly (UnexpectedEof-shaped → the engine's Error::Io mapping; the POC's ranged_reads_windows_and_eof shape). Reads ride the lo_* 64-bit variants end-to-end (lo_lseek64/int8 offsets if any positioning is used; lo_get's bigint offsets already are — the C7 int4 trap must not be inherited). The held-descriptor tx posture exists as the fallback (measured: no advantage even for sequential full reads) — named in the module doc as not-shipped, never coded.
• has — companion-table EXISTS probe. size — the companion-row size probe (metadata-only; the ADR-009 companion-row probe the suite's INV-6 row names). delete — one tx (lo_unlink + row delete), rows-affected discipline (Err(Error::Missing) when absent); GC-participating (the delete window's arbitration is store-core and table-level — nothing fs-specific, ADR-009 §4). list — SELECT key FROM lo_entries, complete (the POC's exact-count gate: list == companion table after mixed ops).
• Virgin-store posture — the constructor creates the schema and table; the tolerance is the postgres engine's recorded read-only split (reads on a schema-less database map to absent/empty via the SQLSTATE 42P01 discriminant, writes surface the engine error). The schema-less test fixture mirrors the postgres engine's #[cfg(test)] new_schemaless pattern.
• Posture deltas in the module doc, stated not hidden (ADR-009 §5/§6) — LO catalog space is reused but never returned (pg_total_relation_size('pg_largeobject') monitoring named; VACUUM FULL/pg_repack the only shrink path — an operator runbook line, never automatic); autovacuum inherited, keep it on; single-stream cached get 20–50× behind page-cache local fs at ≥1 MiB with the aggregate ~700 MB/s @ 16 readers serving picture — named capacity facts, not solved problems.
• Orphan-recovery sweep (ADR-009 §5's requirement) — a crate-internal reap_orphans() for the legacy/bypass class only (committed LOs without companion rows): enumerate SELECT DISTINCT loid FROM pg_largeobject as a candidate list only, check each against lo_entries.loid (the truth), lo_unlink the unclaimed (the C6/pgdiag10 shape — cost scales with oid count, not content size). Scope-noted #[allow(dead_code)] until a consumer exists (recorded: the ops/maintenance surface is wave 7+; no store-core verb is scoped this wave), contract-tested with a planted orphan and a byte-exact survival assertion for committed content.
• Unit tests in-module: the method contract (virgin-store no-ops incl. the schema-less form, CAS re-put no-op, Missing discipline, size exactness + metadata-only, list completeness after mixed ops, ranged reads at window boundaries + beyond-length failure), the staged-put happy path and both hygiene forms, the orphan sweep. Tests need a live server — the alkstore harness convention: connection settings ride the environment (ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB), tests skip cleanly server-less, acceptance requires green against the server. The harness server: dockerized pglo-poc (postgres:16-alpine) on :15432, db blobs, user/password postgres/poc (verified live 2026-10-12).
• LO isolation on the shared harness db (named design point — the C7 lesson, do not rediscover it): Large Objects are database-level, not schema-level — the schema-per-test convention does not isolate them, a DROP SCHEMA leaves the LO's pages in pg_largeobject, and shared-LO tests raced each other into spurious "large object NNNN does not exist" aborts in the POC. Decide and record the isolation shape: per-test CREATE DATABASE (the POC's proven open_fresh shape) or per-instance LOID tracking + teardown lo_unlink (a LoJanitor drop-guard, the SchemaJanitor pattern extended). Either way acceptance includes zero LO residue on the shared db (pg_largeobject probed before/after full runs).
• Review amendment A2 (wave-4 gate, owner: this task) — adopt the schema-janitor discipline from the start; the wave-4 lesson (380 leaked schemas) is not to be re-learned. The SchemaJanitor lives in src/postgres.rs's #[cfg(test)] test_support — its new shared home (gated cfg(test) + any(postgres, pg-lo)) rides the shared-helper design point.

The window-family seam clauses (EngineWindowState) are engine-pg-lo-window (separate task); the constructor wiring is store-pg-lo-construction. Nothing of EngineFleetState may exist on this engine (ADR-012 §5's pg-lo row: pin clauses no, sweep lock no — fleet pins live on the kv host). This task ships the engine and its in-module unit tests only.

Acceptance Criteria

• src/pg_lo.rs implements Backend completely (seven methods + put_staged) behind feature pg-lo (default-off), with the descriptorless lo_get window cursor body and the held-tx staging body
• The companion table is the contract authority (list/size/has/CAS all table-level); pg_largeobject is never read for addressing
• One engine-owned schema, idempotent bootstrap at the constructor; the quoted-identifier boundary; the virgin-store read-only split (42P01 → absent/empty on reads, writes surface) unit-tested incl. the schema-less form
• Put = one tx (lo_create → 512 KiB chunked lowrite → row insert → COMMIT); CAS re-put is a no-op; the 512 KiB knee is a named constant with the C2 citation
• Stage hygiene holds in both forms (discard → zero residue; failed commit → rollback self-discard), exact-count unit-tested
• Reads ride lo_get(oid, off, len) windows; len() metadata-only from the companion row; beyond-length reads fail cleanly; the 64-bit-variants discipline recorded (the C7 trap not inherited)
• delete = one tx (lo_unlink + row delete) with the Missing discipline; list complete after mixed ops
• The orphan-recovery sweep exists (candidates from pg_largeobject only, validated against lo_entries), scope-noted, contract-tested with a planted orphan
• Posture deltas recorded in the module doc (catalog space never returned + VACUUM FULL operator line + rel-size monitoring; autovacuum inherited; the cached-get capacity facts)
• Shared-helper + PgOpts + LO-isolation design points decided and recorded; the pg-lo-pure build (--no-default-features --features pg-lo) compiles clean
• Unit tests env-driven (ALKBLOBS_PG_*), skipping cleanly server-less; green against the harness server at acceptance; zero schema + LO residue on the shared db after runs
• cargo test (default — pg-lo compiles out) green; cargo test --features pg-lo green server-less (skipping); cargo clippy --all-targets -- -D warnings and cargo fmt --check green (default + --features pg-lo + the pg-lo-pure check)

References

• docs/architecture/decisions/009-pg-lo-large-tier-engine.md (the engine ADR — §1–§4 the contract, §5/§6 the posture deltas)
• docs/research/poc-pglo-findings.md (findings C1–C7 — the evidence base; C2 the 512 KiB knee, C4 the window posture, C6 the orphan sweep, C7 the harness lessons)
• docs/architecture/decisions/003-backend-contract-and-dispatch.md (the trait contract)
• docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §1 (the staged-put contract; the large-tier cursor body families)
• docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §4/§5 (the consolidation posture; the pg-lo applicability row — window family only, no fleet state)
• docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §4 (pg-lo does not imply postgres; the pg-lo-pure build)
• src/backend.rs (the trait + seam contracts this implements)
• src/postgres.rs (the driver-posture twin: pool, prepare_cached, schema bootstrap, 42P01 split, test_support janitor)
• src/local.rs (the large-tier body twin: streaming cursor, staged put hygiene shapes)
• /workspace/alkblobs-pglo-poc (the POC crate — the ~400-line PgLoBackend this engine productionizes)

Notes

To be filled by implementation agent

Summary

To be filled on completion