alkblobs/tasks/suite-column-pg-lo.md

id, name, status, depends_on, scope, risk, impact, level, tags

id	name	status	depends_on	scope	risk	impact	level	tags
suite-column-pg-lo	Conformance suite — the pg-lo column (the LO-medium SKIP-posture column)	completed	store-pg-lo-construction	moderate	medium	phase	implementation	wave-5 suite pg-lo

Description

Grow the pg-lo column — the suite's second SKIP-posture column (README §SKIP posture: SQL-attached factories ride a server; when no server is reachable the column skips cleanly — a skip is not a failure — and the wave's acceptance requires green against the server). The column runs under feature pg-lo (default-off); its two factories span feature pairs — the first column whose factories need different feature sets (a recorded design point): the mem-pglo factory gates on all(feature = "mem", feature = "pg-lo"), the postgres-pglo factory on all(feature = "postgres", feature = "pg-lo"); the column's full run (acceptance) is --all-features against the server.

• Factories (tests/conformance/pg_lo_column.rs):
  • DualTierMemPgLoFactory — kv = mem + large = pg-lo: the one-window-host rule's named test topology. The durable-schedule sweep rows run with the window hosted on the large engine (gc_window in the pg-lo schema) — the shape no earlier column exercises.
  • DualTierPostgresPgLoFactory — kv = postgres + large = pg-lo on one pg instance: the consolidation posture. Mirrors the postgres column's inventory with the large medium swapped to LOs, and carries the dormancy row (the pg-lo schema carries no gc_window; the window lives on the kv schema). Both at column-declared thresholds and the test tuning of the window delay (0 — execute-on-next-sweep). The alkstore pg-factory pattern, reused (the postgres column's landed shape): a fresh engine-owned schema per open() — unique per call, tracked per-instance in a registry, teardown DROP SCHEMA IF EXISTS … CASCADE over exactly the minted schemas, panicking-row-safe. The LO-isolation corollary (the engine-pg-lo design point carries): LOs are database-level — teardown must also reap the instance's LOs (tracked LOIDs + lo_unlink, or the per-test-database shape if that is what the engine task recorded), and the structural teardown assertions include zero pg_largeobject residue.
• Connection settings ride the environment — the harness convention: ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB, a reachability probe gating the rows (the SKIP posture's factory-owner responsibility). The harness server: dockerized pglo-poc (postgres:16-alpine) on :15432, db blobs, user/password postgres/poc — verified live 2026-10-12; the container is long-running in the dev environment.
• Mirror the form-independent reference rows (README §The reference row set): INV-1 round-trips (content matrix × both put paths, in-body — the LO medium's ranged reads at window boundaries and the beyond-length clean failure are this column's INV-1 extensions), INV-6 stat discipline (the companion-row probe — metadata-only, never LO content), the NoLivenessSources/ ProtectFailed abort shapes. No kv-only rejection row — no kv-only pg-lo constructor exists (recorded; the postgres column hosts that row for the SQL kv engines).
• The durable-schedule sweep rows — the SQL-backed columns' family (README §The schedule-relative reading), run on both factories: INV-2 dedup and INV-3 stage hygiene with counts read against the staged schedule, the two-sweep exact-count row over the full protection set (both tiers, the window observed — on the pg-lo host for the mem-pglo factory, on the kv host for the postgres-pglo factory), the sweep-vs-put race across the window delay (execution-time re-arbitration), the holder-stages-only lease shape, and INV-5 list correctness through durable-schedule outcomes.
• The dual-tier dispatch rows mirror the postgres column's family (pure-function routing, threshold boundaries, fall-through per op) with the placement oracle adapted to the LO medium: large entries are companion rows + LOs (an independent pg connection counting lo_entries and probing pg_largeobject), kv entries per the factory's kv engine (the mem-pglo factory's kv placement is in-process — the oracle's kv half follows the mem column's introspection shape; recorded design point).
• Virgin-store row: reads are no-ops on a fresh schema (pool, lo_entries, and the window state per the host rule all empty) — the constructor always ensures the host's schema, this row being the facade form.
• Structural factory tests (the postgres column's pattern): factory isolation (fresh schema per open, nothing shared) and teardown-against-a-panicking-row (schemas really dropped, LOs really unlinked — zero residue probed).
• README amendments (the convention's normative test-side home): the pg-lo rows table in §Columns beyond the reference set; the column plan's SKIP-posture note naming the second landed column and the per-factory feature-pair gating; current state (five columns).

Acceptance Criteria

• The pg-lo column green against the harness server (pglo-poc, :15432) under --all-features: the mirrored reference rows + the durable-schedule sweep rows on both factories (the pg-lo-host shape and the kv-host/dormancy shape) + the dispatch rows + the virgin-store row, every row ADR-stamped and traceable to the backlog
• The dormancy row asserts the consolidation shape's pg-lo schema carries no gc_window; the mem-pglo rows assert the window on the pg-lo host
• Server-less runs skip cleanly (the reachability probe; the default-build gates stay green with the column skipped)
• Factory isolation + idempotent teardown hold (fresh schema per open; schemas dropped, LOs unlinked; zero pg_largeobject residue probed; teardown safe against a panicking row), structurally exercised
• README amendments recorded (the pg-lo rows table; the feature-pair gating note; current state)
• cargo test (default) green — the column compiles out; cargo test --features pg-lo (and the mem/postgres pair variants) green server-less (skipping) and green against the server at acceptance; cargo clippy --all-targets -- -D warnings and cargo fmt --check green (default + the pg-lo variants + --all-features)

References

• tests/conformance/README.md (the conventions this grows against; §SKIP posture, §Columns beyond the reference set, §The schedule-relative reading for SQL-backed columns)
• tests/conformance/postgres_column.rs (the SKIP-posture column pattern — the factory shape, the medium oracle, the structural tests this mirrors)
• tests/conformance/factory.rs (the factory contract this implements)
• tests/conformance/rows.rs (the reference row set being mirrored)
• docs/architecture/store-api.md (§Invariants — the test gate)
• docs/architecture/decisions/009-pg-lo-large-tier-engine.md (the engine ADR the LO-medium rows stamp against)
• docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §5 (the one-window-host rule — the dormancy row's authority), §6.7 (the schedule the sweep rows assert against)
• tasks/engine-pg-lo.md (the LO-isolation design point this factory inherits)

Notes

Implemented in tests/conformance/pg_lo_column.rs (the two pair-gated factories, the shared harness/oracle/janitor parent, the row bodies over both factories, and the structural tests), tests/conformance/main.rs (the module gate + the columns' doc paragraph), tests/conformance/README.md (the amendments), and one mechanical line in src/store.rs (see below). Recorded design points, as resolved:

• The LO-isolation corollary carried in the engine task's recorded shape — per-open dedicated databases. The column first followed the postgres column's shared-db schema pattern with tracked-LOID lo_unlink teardown, and the wave-5 engine-suite gate (pg_lo::tests::shared_harness_db_holds_zero_lo_residue) broke at acceptance: its permanent probe (zero pg_largeobject pages in the shared blobs database) runs concurrently with the suite and the column's LOs legitimately existed mid-run there. The task's alternative clause ("or the per-test-database shape if that is what the engine task recorded") resolved the fork: the engine task did record the per-test dedicated database shape, so the column carries it — open() mints a fresh dedicated database (unique per call), the engine-owned schemas are minted inside it (the alkstore pg-factory pattern's registry, housed on its own database), and teardown reaps the instance's LOs (SELECT loid FROM lo_entries → lo_unlink), drops exactly the minted schemas (DROP SCHEMA IF EXISTS … CASCADE), and drops the database (WITH (FORCE)), with a per-open drop-guard janitor (the crate's DatabaseJanitor pattern re-implemented in the column file — the crate's janitor is #[cfg(test)]-internal) backing the abandoned paths. The structural tests probe zero orphan residue inside the dedicated database before it goes (the companion table and the LO catalog agree exactly — the C1 shape), assert the recorded reaped LOIDs match the pre-captured set, and observe the database gone afterward; zero residue on the shared database is structural (no pg-lo pool ever opens against blobs).
• The consolidation factory mints two schemas (the kv half and the pg-lo half, both inside the dedicated database) so the dormancy row observes genuine absence; the corollary's one-schema mint remains the engine tests' recorded alternative.
• The mem-pglo kv half is facade-observed (the mem column's introspection shape: served reads and has() against the LO medium's genuine absence); the postgres-pglo kv half counts the kv schema's blobs rows (the postgres column's oracle shape).
• The INV-1 extension row asserts ranged reads across the shipped window edge (READ_CHUNK's 512 KiB, duplicated in the column file since the constant is engine-internal), the EOF clamping family (tail, at-EOF empty, u64::MAX offset clamp, zero-length, over-read), the ADR-006 range digests, and the beyond-length clean failure against a planted overclaiming companion row (lo_create/lo_open/lowrite + lo_entries insert through the oracle — the corrupted half-state healthy stores never hold): the read trusts the row (metadata-only) and fails with the unexpected-EOF-shaped Error::Io, and the executing sweep's delete reaps the planted LO. The identical- content legs dedup into one entry (CAS across the paths), so the cleanup stages exactly two.
• Row inventory (both factories; the factory dimension is the run): the form-independent mirror (INV-1, INV-6, the two no-op abort shapes — verbatim from rows.rs) + INV-2/INV-3 (schedule- relative, both tiers' legs) + the two-sweep exact-count row, the window-delay race row, the holder-stages-only lease row, INV-5's durable row, the three dispatch rows, the INV-1 extension, the virgin-store row, the postgres-pglo-only dormancy row; no kv-only rejection row (no kv-only pg-lo constructor exists; recorded). The mem-pglo virgin row asserts the window on the pg-lo host (gc_window present-and-empty, gc_pins never).
• One mechanical src/ fix (out of scope but required by the gate): src/store.rs's use crate::pg_common::PgOpts was gated any(postgres, pg-lo) and sat unused in the sqlite+pg-lo config (pg-lo alone over default features), tripping cargo clippy --features pg-lo -D warnings; the gate now names the configs that consume the import (any(postgres, all(pg-lo, mem)) — no semantic change).
• Two harness-server residue cleanups — found + removed (same discipline as the d69d9a3 finding): the shared blobs db carried the pre-rework runs' schemas (abpglo_*/the over-long first mints) + their orphaned LO pages (1032 catalog pages; the schema-pattern teardown dropped schemas, stranding LOs — exactly the C7 lesson motivating the dedicated-database shape); cleanup verified zero (pg_largeobject, schemas, and suites' databases). The first shared-db schema mints also grazed postgres' 63-char identifier limit (E42622) — the dedicated-db mint (abpglosuite_…/abpglo_…) stays well under it.
• The first rework run's failure was a miscount, not a bug: the INV-1 extension row's two identical-content legs dedup into one entry, so the cleanup sweep stages two (one real + the planted), not three — fixed in the row's expectations and recorded here.

Summary

Completed 2026-10-12 (wave 5). The pg-lo column — the suite's second SKIP-posture column — landed in tests/conformance/pg_lo_column.rs: DualTierMemPgLoFactory (kv = mem + large = pg-lo, the one-window-host rule's named test topology, all(mem, pg-lo)) and DualTierPostgresPgLoFactory (the consolidation posture on one pg instance, all(postgres, pg-lo); two schemas inside each open's dedicated database; the dormancy row asserting no gc_window in the pg-lo schema and the window on the kv host), each factory vending a fresh dedicated database per open with engine-owned schemas minted inside, teardown reaping tracked LOIDs → dropping the minted schemas → dropping the database WITH (FORCE), plus per-open drop-guard janitors for the abandoned paths — zero LO residue on the shared blobs database structural (the engine suite's gate holds green alongside the column). Fifteen rows on the mem-pglo factory, sixteen on the postgres-pglo factory (its extra one being the dormancy row): the verbatim form-independent mirror (INV-1, INV-6, the no-op abort shapes), the schedule-relative INV-2/3, the two-sweep exact counts over the full protection set across the observed window host, the window-delay race, the holder-stages-only lease, INV-5's durable row, the three dispatch rows over the LO-medium oracle, the INV-1 extension (window-edge ranged reads + the planted-overclaim beyond-length clean failure), and the virgin-store row (the mem-pglo one asserting the window on the pg-lo host) — the dormancy row being the postgres-pglo factory's extra — plus the structural isolation/panicking-row tests probing zero orphan residue. No kv-only rejection row (recorded; pg-lo is a large engine). README recorded (the pg-lo rows table in §Columns beyond the reference set, the SKIP-posture note naming the second landed column and the per-factory feature-pair gating, current state = five columns). Verified (harness server live 2026-10-12): acceptance cargo test --all-features against the server fully green (188 lib

• 106 conformance + 4 git-interop; zero LO/schema/database residue after the run); cargo test (default) green (69/19/3 — the column compiles out); --features "pg-lo,mem" and --features "postgres,pg-lo" green server-less (skipping cleanly) and under --features pg-lo alone; clippy -D warnings green on default, pg-lo, pg-lo+mem, postgres+pg-lo, and all-features; cargo fmt --check green; cargo doc --no-deps clean.