glm-5.3-flash
aa96f8b981
- tests/conformance/pg_lo_column.rs (new): the two pair-gated factories (DualTierMemPgLoFactory on all(mem, pg-lo) — the one-window-host rule's named test topology; DualTierPostgresPgLoFactory on all(postgres, pg-lo) — the consolidation posture, two schemas per open, carrying the dormancy row), a fresh dedicated database per open with the engine-owned schemas minted inside, teardown reaping tracked LOIDs (lo_unlink) then dropping the minted schemas then the database (WITH (FORCE)) with per-open drop-guard janitors for the abandoned paths, and the rows: the verbatim form-independent mirror (INV-1, INV-6, the two no-op abort shapes), the schedule-relative INV-2/3, the two-sweep exact counts over the full protection set across each factory's observed window host, the window-delay race, the holder-stages-only lease, INV-5's durable row, the three dispatch rows over the LO-medium placement oracle, the INV-1 extension (window-edge ranged reads + the planted-overclaim beyond-length clean failure), the virgin-store rows, and the dormancy row; structural isolation + panicking-row tests probing zero orphan residue (no kv-only rejection row — recorded; pg-lo is a large engine) - the LO-isolation corollary carried in the engine task's recorded per-test-database shape (the C7 lesson): zero LO residue on the shared harness database is structural, so the engine suite's shared_harness_db_holds_zero_lo_residue gate stays green alongside the column under --all-features - main.rs: the pg_lo_column gate (feature pg-lo, default-off) + the columns' doc paragraph - README: the pg-lo rows table in §Columns beyond the reference set, the SKIP-posture note naming the second landed column and the per-factory feature-pair gating, current state = five columns - store.rs: one mechanical gate fix — the PgOpts import was unused in the sqlite+pg-lo config (tripping clippy --features pg-lo -D warnings); the gate now names the configs that consume it Verification (harness server live 2026-10-12): --all-features against the server fully green (188 lib + 106 conformance + 4 git-interop; zero LO/schema/database residue after the run); default test green (69/19/3); --features "pg-lo,mem" and "postgres,pg-lo" green server-less (skipping cleanly) and --features pg-lo alone; clippy -D warnings green on default, pg-lo, pg-lo+mem, postgres+pg-lo, all-features; fmt --check green; cargo doc --no-deps clean.
270 lines
15 KiB
Markdown
270 lines
15 KiB
Markdown
---
|
||
id: suite-column-pg-lo
|
||
name: Conformance suite — the pg-lo column (the LO-medium SKIP-posture column)
|
||
status: completed
|
||
depends_on: [store-pg-lo-construction]
|
||
scope: moderate
|
||
risk: medium
|
||
impact: phase
|
||
level: implementation
|
||
tags: [wave-5, suite, pg-lo]
|
||
---
|
||
|
||
## Description
|
||
|
||
Grow the **pg-lo column** — the suite's second SKIP-posture column
|
||
(README §SKIP posture: SQL-attached factories ride a server; when no
|
||
server is reachable the column **skips cleanly** — a skip is not a
|
||
failure — and the wave's acceptance requires green **against the
|
||
server**). The column runs under feature `pg-lo` (default-off); its
|
||
two factories span feature pairs — **the first column whose factories
|
||
need different feature sets** (a recorded design point): the mem-pglo
|
||
factory gates on `all(feature = "mem", feature = "pg-lo")`, the
|
||
postgres-pglo factory on `all(feature = "postgres",
|
||
feature = "pg-lo")`; the column's full run (acceptance) is
|
||
`--all-features` against the server.
|
||
|
||
- **Factories** (`tests/conformance/pg_lo_column.rs`):
|
||
- **`DualTierMemPgLoFactory`** — kv = mem + large = pg-lo: the
|
||
one-window-host rule's named test topology. The **durable-schedule
|
||
sweep rows run with the window hosted on the large engine**
|
||
(`gc_window` in the pg-lo schema) — the shape no earlier column
|
||
exercises.
|
||
- **`DualTierPostgresPgLoFactory`** — kv = postgres + large = pg-lo
|
||
on one pg instance: the consolidation posture. Mirrors the
|
||
postgres column's inventory with the large medium swapped to LOs,
|
||
and carries the **dormancy row** (the pg-lo schema carries no
|
||
`gc_window`; the window lives on the kv schema).
|
||
Both at column-declared thresholds and the **test tuning of the
|
||
window delay (`0` — execute-on-next-sweep)**. **The alkstore
|
||
pg-factory pattern, reused** (the postgres column's landed shape):
|
||
a fresh engine-owned schema per `open()` — unique per call, tracked
|
||
per-instance in a registry, teardown `DROP SCHEMA IF EXISTS …
|
||
CASCADE` over exactly the minted schemas, panicking-row-safe. The
|
||
**LO-isolation corollary** (the `engine-pg-lo` design point carries):
|
||
LOs are database-level — teardown must also reap the instance's LOs
|
||
(tracked LOIDs + `lo_unlink`, or the per-test-database shape if that
|
||
is what the engine task recorded), and the structural teardown
|
||
assertions include **zero `pg_largeobject` residue**.
|
||
- **Connection settings ride the environment** — the harness
|
||
convention: `ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`, a
|
||
reachability probe gating the rows (the SKIP posture's
|
||
factory-owner responsibility). **The harness server**: dockerized
|
||
`pglo-poc` (`postgres:16-alpine`) on :15432, db `blobs`,
|
||
user/password `postgres`/`poc` — verified live 2026-10-12; the
|
||
container is long-running in the dev environment.
|
||
- **Mirror the form-independent reference rows** (README §The
|
||
reference row set): INV-1 round-trips (content matrix × both put
|
||
paths, in-body — the LO medium's ranged reads at window boundaries
|
||
and the beyond-length clean failure are this column's INV-1
|
||
extensions), INV-6 stat discipline (the companion-row probe —
|
||
metadata-only, never LO content), the `NoLivenessSources`/
|
||
`ProtectFailed` abort shapes. **No kv-only rejection row** — no
|
||
kv-only pg-lo constructor exists (recorded; the postgres column
|
||
hosts that row for the SQL kv engines).
|
||
- **The durable-schedule sweep rows** — the SQL-backed columns' family
|
||
(README §The schedule-relative reading), run on **both factories**:
|
||
INV-2 dedup and INV-3 stage hygiene with counts read against the
|
||
staged schedule, the two-sweep exact-count row over the full
|
||
protection set (both tiers, the window observed — on the pg-lo host
|
||
for the mem-pglo factory, on the kv host for the postgres-pglo
|
||
factory), the sweep-vs-put race across the window delay
|
||
(execution-time re-arbitration), the holder-stages-only lease shape,
|
||
and INV-5 list correctness through durable-schedule outcomes.
|
||
- **The dual-tier dispatch rows** mirror the postgres column's family
|
||
(pure-function routing, threshold boundaries, fall-through per op)
|
||
with the placement oracle adapted to the LO medium: large entries
|
||
are companion rows + LOs (an independent pg connection counting
|
||
`lo_entries` and probing `pg_largeobject`), kv entries per the
|
||
factory's kv engine (the mem-pglo factory's kv placement is
|
||
in-process — the oracle's kv half follows the mem column's
|
||
introspection shape; recorded design point).
|
||
- **Virgin-store row**: reads are no-ops on a fresh schema (pool,
|
||
`lo_entries`, and the window state per the host rule all empty) —
|
||
the constructor always ensures the host's schema, this row being the
|
||
facade form.
|
||
- **Structural factory tests** (the postgres column's pattern):
|
||
factory isolation (fresh schema per open, nothing shared) and
|
||
teardown-against-a-panicking-row (schemas really dropped, LOs really
|
||
unlinked — zero residue probed).
|
||
- **README amendments** (the convention's normative test-side home):
|
||
the pg-lo rows table in §Columns beyond the reference set; the
|
||
column plan's SKIP-posture note naming the second landed column and
|
||
the per-factory feature-pair gating; current state (five columns).
|
||
|
||
## Acceptance Criteria
|
||
|
||
- [x] The pg-lo column green **against the harness server** (`pglo-poc`,
|
||
:15432) under `--all-features`: the mirrored reference rows + the
|
||
durable-schedule sweep rows on both factories (the pg-lo-host
|
||
shape and the kv-host/dormancy shape) + the dispatch rows + the
|
||
virgin-store row, every row ADR-stamped and traceable to the
|
||
backlog
|
||
- [x] The dormancy row asserts the consolidation shape's pg-lo schema
|
||
carries no `gc_window`; the mem-pglo rows assert the window on
|
||
the pg-lo host
|
||
- [x] Server-less runs skip cleanly (the reachability probe; the
|
||
default-build gates stay green with the column skipped)
|
||
- [x] Factory isolation + idempotent teardown hold (fresh schema per
|
||
open; schemas dropped, LOs unlinked; **zero `pg_largeobject`
|
||
residue** probed; teardown safe against a panicking row),
|
||
structurally exercised
|
||
- [x] README amendments recorded (the pg-lo rows table; the
|
||
feature-pair gating note; current state)
|
||
- [x] `cargo test` (default) green — the column compiles out;
|
||
`cargo test --features pg-lo` (and the mem/postgres pair
|
||
variants) green server-less (skipping) and green against the
|
||
server at acceptance; `cargo clippy --all-targets -- -D warnings`
|
||
and `cargo fmt --check` green (default + the pg-lo variants +
|
||
`--all-features`)
|
||
|
||
## References
|
||
|
||
- tests/conformance/README.md (the conventions this grows against;
|
||
§SKIP posture, §Columns beyond the reference set, §The
|
||
schedule-relative reading for SQL-backed columns)
|
||
- tests/conformance/postgres_column.rs (the SKIP-posture column
|
||
pattern — the factory shape, the medium oracle, the structural
|
||
tests this mirrors)
|
||
- tests/conformance/factory.rs (the factory contract this implements)
|
||
- tests/conformance/rows.rs (the reference row set being mirrored)
|
||
- docs/architecture/store-api.md (§Invariants — the test gate)
|
||
- docs/architecture/decisions/009-pg-lo-large-tier-engine.md (the
|
||
engine ADR the LO-medium rows stamp against)
|
||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||
§5 (the one-window-host rule — the dormancy row's authority), §6.7
|
||
(the schedule the sweep rows assert against)
|
||
- tasks/engine-pg-lo.md (the LO-isolation design point this factory
|
||
inherits)
|
||
|
||
## Notes
|
||
|
||
> Implemented in `tests/conformance/pg_lo_column.rs` (the two
|
||
> pair-gated factories, the shared harness/oracle/janitor parent, the
|
||
> row bodies over both factories, and the structural tests),
|
||
> `tests/conformance/main.rs` (the module gate + the columns' doc
|
||
> paragraph), `tests/conformance/README.md` (the amendments), and one
|
||
> mechanical line in `src/store.rs` (see below). Recorded design
|
||
> points, as resolved:
|
||
>
|
||
> - **The LO-isolation corollary carried in the engine task's recorded
|
||
> shape — per-open dedicated databases.** The column first followed
|
||
> the postgres column's shared-db schema pattern with tracked-LOID
|
||
> `lo_unlink` teardown, and the wave-5 engine-suite gate
|
||
> (`pg_lo::tests::shared_harness_db_holds_zero_lo_residue`) broke at
|
||
> acceptance: its permanent probe (zero `pg_largeobject` pages in the
|
||
> shared `blobs` database) runs concurrently with the suite and the
|
||
> column's LOs legitimately existed mid-run there. The task's
|
||
> alternative clause ("or the per-test-database shape if that is what
|
||
> the engine task recorded") resolved the fork: the engine task
|
||
> **did** record the per-test dedicated database shape, so the column
|
||
> carries it — `open()` mints a fresh dedicated database (unique per
|
||
> call), the engine-owned schemas are minted inside it (the alkstore
|
||
> pg-factory pattern's registry, housed on its own database), and
|
||
> teardown reaps the instance's LOs (`SELECT loid FROM
|
||
> lo_entries` → `lo_unlink`), drops exactly the minted schemas
|
||
> (`DROP SCHEMA IF EXISTS … CASCADE`), and drops the database
|
||
> (`WITH (FORCE)`), with a per-open drop-guard janitor (the crate's
|
||
> `DatabaseJanitor` pattern re-implemented in the column file — the
|
||
> crate's janitor is `#[cfg(test)]`-internal) backing the abandoned
|
||
> paths. The structural tests probe **zero orphan residue inside the
|
||
> dedicated database before it goes** (the companion table and the
|
||
> LO catalog agree exactly — the C1 shape), assert the recorded
|
||
> reaped LOIDs match the pre-captured set, and observe the database
|
||
> gone afterward; zero residue on the shared database is structural
|
||
> (no pg-lo pool ever opens against `blobs`).
|
||
> - **The consolidation factory mints two schemas** (the kv half and
|
||
> the pg-lo half, both inside the dedicated database) so the dormancy
|
||
> row observes genuine absence; the corollary's one-schema mint
|
||
> remains the engine tests' recorded alternative.
|
||
> - **The mem-pglo kv half is facade-observed** (the mem column's
|
||
> introspection shape: served reads and `has()` against the LO
|
||
> medium's genuine absence); the postgres-pglo kv half counts the kv
|
||
> schema's `blobs` rows (the postgres column's oracle shape).
|
||
> - **The INV-1 extension row** asserts ranged reads across the
|
||
> shipped window edge (`READ_CHUNK`'s 512 KiB, duplicated in the
|
||
> column file since the constant is engine-internal), the EOF
|
||
> clamping family (tail, at-EOF empty, `u64::MAX` offset clamp,
|
||
> zero-length, over-read), the ADR-006 range digests, and the
|
||
> **beyond-length clean failure** against a **planted overclaiming
|
||
> companion row** (`lo_create`/`lo_open`/`lowrite` + `lo_entries`
|
||
> insert through the oracle — the corrupted half-state healthy
|
||
> stores never hold): the read trusts the row (metadata-only) and
|
||
> fails with the unexpected-EOF-shaped `Error::Io`, and the
|
||
> executing sweep's delete reaps the planted LO. The identical-
|
||
> content legs dedup into one entry (CAS across the paths), so the
|
||
> cleanup stages exactly two.
|
||
> - **Row inventory** (both factories; the factory dimension is the
|
||
> run): the form-independent mirror (INV-1, INV-6, the two no-op
|
||
> abort shapes — verbatim from rows.rs) + INV-2/INV-3 (schedule-
|
||
> relative, both tiers' legs) + the two-sweep exact-count row, the
|
||
> window-delay race row, the holder-stages-only lease row, INV-5's
|
||
> durable row, the three dispatch rows, the INV-1 extension, the
|
||
> virgin-store row, the postgres-pglo-only dormancy row; **no
|
||
> kv-only rejection row** (no kv-only pg-lo constructor exists;
|
||
> recorded). The mem-pglo virgin row asserts the window on the
|
||
> pg-lo host (`gc_window` present-and-empty, `gc_pins` never).
|
||
> - **One mechanical `src/` fix** (out of scope but required by the
|
||
> gate): `src/store.rs`'s `use crate::pg_common::PgOpts` was gated
|
||
> `any(postgres, pg-lo)` and sat unused in the `sqlite`+`pg-lo`
|
||
> config (pg-lo alone over default features), tripping
|
||
> `cargo clippy --features pg-lo -D warnings`; the gate now names
|
||
> the configs that consume the import
|
||
> (`any(postgres, all(pg-lo, mem))` — no semantic change).
|
||
> - **Two harness-server residue cleanups** — found + removed (same
|
||
> discipline as the d69d9a3 finding): the shared `blobs` db carried
|
||
> the pre-rework runs' schemas (`abpglo_*`/the over-long first
|
||
> mints) + their orphaned LO pages (1032 catalog pages; the
|
||
> schema-pattern teardown dropped schemas, stranding LOs — exactly
|
||
> the C7 lesson motivating the dedicated-database shape); cleanup
|
||
> verified zero (`pg_largeobject`, schemas, and suites' databases).
|
||
> The first shared-db schema mints also grazed postgres' 63-char
|
||
> identifier limit (E42622) — the dedicated-db mint
|
||
> (`abpglosuite_…`/`abpglo_…`) stays well under it.
|
||
> - **The first rework run's failure was a miscount, not a bug**: the
|
||
> INV-1 extension row's two identical-content legs dedup into one
|
||
> entry, so the cleanup sweep stages two (one real + the planted),
|
||
> not three — fixed in the row's expectations and recorded here.
|
||
|
||
## Summary
|
||
|
||
> Completed 2026-10-12 (wave 5). The pg-lo column — the suite's
|
||
> second SKIP-posture column — landed in
|
||
> `tests/conformance/pg_lo_column.rs`: `DualTierMemPgLoFactory` (kv =
|
||
> mem + large = pg-lo, the one-window-host rule's named test
|
||
> topology, `all(mem, pg-lo)`) and `DualTierPostgresPgLoFactory` (the
|
||
> consolidation posture on one pg instance, `all(postgres, pg-lo)`;
|
||
> two schemas inside each open's dedicated database; the dormancy
|
||
> row asserting no `gc_window` in the pg-lo schema and the window on
|
||
> the kv host), each factory vending a fresh dedicated database per
|
||
> open with engine-owned schemas minted inside, teardown reaping
|
||
> tracked LOIDs → dropping the minted schemas → dropping the
|
||
> database `WITH (FORCE)`, plus per-open drop-guard janitors for the
|
||
> abandoned paths — zero LO residue on the shared `blobs` database
|
||
> structural (the engine suite's gate holds green alongside the
|
||
> column). Fifteen rows on the mem-pglo factory, sixteen on the
|
||
> postgres-pglo factory (its extra one being the dormancy row): the
|
||
> verbatim form-independent
|
||
> mirror (INV-1, INV-6, the no-op abort shapes), the
|
||
> schedule-relative INV-2/3, the two-sweep exact counts over the
|
||
> full protection set across the observed window host, the
|
||
> window-delay race, the holder-stages-only lease, INV-5's durable
|
||
> row, the three dispatch rows over the LO-medium oracle, the INV-1
|
||
> extension (window-edge ranged reads + the planted-overclaim
|
||
> beyond-length clean failure), and the virgin-store row (the
|
||
> mem-pglo one asserting the window on the pg-lo host) — the
|
||
> dormancy row being the postgres-pglo factory's extra — plus the
|
||
> structural isolation/panicking-row tests probing zero orphan
|
||
> residue. No kv-only rejection row (recorded; pg-lo is a large
|
||
> engine). README recorded (the pg-lo rows table in §Columns beyond
|
||
> the reference set, the SKIP-posture note naming the second landed
|
||
> column and the per-factory feature-pair gating, current state =
|
||
> five columns). Verified (harness server live 2026-10-12): acceptance
|
||
> `cargo test --all-features` against the server fully green (188 lib
|
||
> + 106 conformance + 4 git-interop; zero LO/schema/database residue
|
||
> after the run); `cargo test` (default) green (69/19/3 — the column
|
||
> compiles out); `--features "pg-lo,mem"` and `--features
|
||
> "postgres,pg-lo"` green server-less (skipping cleanly) and under
|
||
> `--features pg-lo` alone; clippy `-D warnings` green on default,
|
||
> pg-lo, pg-lo+mem, postgres+pg-lo, and all-features; `cargo fmt
|
||
> --check` green; `cargo doc --no-deps` clean. |