docs(tasks): decompose wave 5 — the pg-lo large engine, the window host's large-tier arm, the LO-medium SKIP-posture column

- engine-pg-lo: the Backend impl behind feature pg-lo (default-off) —
  companion-table authority, one-tx staged put at the 512 KiB knee,
  descriptorless lo_get windows, the orphan-recovery sweep, the
  posture deltas recorded; the shared-helper/PgOpts/LO-isolation
  design points named (LOs are database-level — the C7 lesson cited
  so it is not rediscovered)
- engine-pg-lo-window: the EngineWindowState impl per ADR-012 §5's
  pg-lo row — gc_window only, no fleet family anywhere
- store-pg-lo-construction: LargeEngine::PgLo slotting (the large
  sub-enum design point), the consolidation + mem-pglo constructors
  (the fleet-valid set completed, implicitly non-fleet), and the
  one-window-host rule's store-core completion (dormancy + the
  large-host sweep arm)
- suite-column-pg-lo: the second SKIP-posture column, two factories
  spanning feature pairs, the dormancy row, zero LO residue in
  teardown
- review-wave-5: the engine-wave gate; amendment A2 (janitor
  adoption) verified here

Harness server cited in every server-touching task: dockerized
pglo-poc (postgres:16-alpine) on :15432, db blobs, postgres/poc
(verified live 2026-10-12); env-driven ALKBLOBS_PG_* settings, clean
server-less skips, acceptance requires green against the server.

Verification: cargo test 69/19/3 ok, clippy --all-targets -D warnings
green, fmt --check green (docs-only change).
This commit is contained in:
glm-5.3-flash committed 2026-10-12 07:13:51 +00:00
1 parent f3e8a88e2e
commit 5a217f7bc7
6 files changed
+785 -2

No files matched your search

+2 -2
View File
@@ -1,6 +1,6 @@
---
status: draft
last_updated: "2026-10-12 (wave 4 decomposed into tasks/)"
last_updated: "2026-10-12 (wave 5 decomposed into tasks/)"
---
# alkblobs — Implementation plan
@@ -52,7 +52,7 @@ ops payload surfaces; release readiness is last.
| [2](#wave-2--the-store-works-dependency-free-mem--local) | mem + local engines; facade put/read paths + dispatch + constructor modes; in-process GC protocol; suite's first columns | wave 1 | decomposed (2026-10-11) |
| [3](#wave-3--sqlite-engine) | sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column | wave 2 | decomposed (2026-10-11) |
| [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | decomposed (2026-10-12) |
| [5](#wave-5--pg-lo-large-engine) | pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column | wave 4 | pending |
| [5](#wave-5--pg-lo-large-engine) | pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column | wave 4 | decomposed (2026-10-12) |
| [6](#wave-6--fleet-gc) | Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations | waves 4 + 5 | pending |
| [7](#wave-7--ops-surface) | `ops` feature: alkcall-backed op family, ACL mapping, pin tokens + renewal, fanout seam | wave 6 | pending |
| [8](#wave-8--fuzzing) | Fuzzing: family pattern targets (key encoding, ops payloads), corpus-replay gate | wave 7 | pending |
+114
View File
@@ -0,0 +1,114 @@
---
id: engine-pg-lo-window
name: pg-lo engine-state window family — the conditional window host
status: pending
depends_on: [engine-pg-lo]
scope: narrow
risk: low
impact: component
level: implementation
tags: [wave-5, engines, pg-lo, gc]
---
## Description
Implement the engine-state companion seam's **window family** on the
pg-lo backend (`src/pg_lo.rs`): the `EngineWindowState` impl per
ADR-012 §5's pg-lo row — `ensure_gc_schema` creates **`gc_window`
only**, the window clauses **conditional** per the one-window-host
rule. **Nothing of `EngineFleetState` may exist on this engine** (the
pg-lo row: pin clauses **no** — fleet pins live on the kv host, §4;
sweep-lock clause **no**). The postgres engine's full-family impl
(`src/postgres.rs`, wave 4) and the sqlite window-only impl
(`src/sqlite.rs`, wave 3) are the reference implementations — same
semantics, different dialect.
- **`ensure_gc_schema`** — creates **`gc_window` only**
(`gc_window (key bytea PRIMARY KEY, visible_after bigint)` per
ADR-010 §2's schema, in the engine's own schema). Never `gc_pins`
(contract-tested absent); the embedder's liveness table is never
created here (ADR-008's schema-ownership rule — the sweep reads it
and never owns it). **Whether the store core calls this clause at
construction is the one-window-host rule's business**
(`store-pg-lo-construction` wires it): the window hosts on the kv
engine when it is SQL-backed (pg-lo's clauses **dormant** on a
dual-SQL instance — kv=sqlite/postgres + large=pg-lo), and on pg-lo
only when the kv engine is not SQL-backed (the kv=mem + large=pg-lo
shape). The clause body exists and is contract-tested per
applicability regardless of dormancy.
- **`window_stage`** — stage candidates into the durable window,
**idempotent per `(key, deadline)`** (the upsert discipline — a
same-deadline re-stage is idempotent, a new-deadline re-stage updates
the row; the sqlite/postgres engines' recorded semantics).
- **`window_take(now)`** — take overdue candidates
(`visible_after <= now`, the `<=` boundary the seam doc pins): the
rows **leave the window table and return as keys** — one atomic step
(`DELETE … RETURNING`, selection and removal together; a row is
never handed out twice). Empty when nothing is overdue; virgin
store: empty.
- **Timestamp encoding** — the crate-wide encoding: signed i64
nanosecond count from the Unix epoch, carried as `BIGINT`
(sub-second precision for the test tunings, pre-epoch negatives,
full i64 range); per-engine bodies of the shared helpers (the
shared-helper home decided in `engine-pg-lo`).
- **Virgin-store tolerance split** — the postgres engine's recorded
posture: `window_take` is a read (SQLSTATE 42P01 maps to empty on a
schema-less database); `window_stage` is a write (surfaces the
engine error — never silently creating schema; store construction
always wires `ensure_gc_schema` when the engine hosts).
- **Contract tests** per the applicability row (ADR-012 §5's table is
the contract-test matrix): the schema creates the window table only
(assert `gc_pins` does not exist); stage idempotence per
`(key, deadline)` incl. the re-stage-at-new-deadline form; take
returns exactly the overdue set, removes it, leaves future rows; the
`<=` boundary incl. nanosecond slack; pre-epoch encoding; both
virgin shapes.
The store core drives these clauses only on the host topology
(`store-pg-lo-construction`); this task ships the clauses and their
engine-level contract tests.
## Acceptance Criteria
- [ ] The pg-lo backend implements `EngineWindowState` — the window
family only; no pin-clause or sweep-lock body exists anywhere in
the pg-lo engine (nothing of `EngineFleetState`)
- [ ] `ensure_gc_schema` creates `gc_window` only; `gc_pins` is never
created by pg-lo (contract-tested); the embedder's liveness
table is never created
- [ ] `window_stage` idempotent per `(key, deadline)`; a re-stage at a
new deadline updates the row
- [ ] `window_take` returns exactly the overdue set (`<= now`
boundary), removes it atomically, leaves future rows; empty on
none/virgin
- [ ] Contract tests env-driven (`ALKBLOBS_PG_*`), skipping cleanly
server-less; green against the harness server (`pglo-poc`,
:15432, db `blobs`, `postgres`/`poc`) at acceptance; the janitor
discipline rides from the start (amendment A2)
- [ ] `cargo test` (default) green; `cargo test --features pg-lo`
green server-less (skipping); `cargo clippy --all-targets -- -D
warnings` and `cargo fmt --check` green (default + `--features
pg-lo`); the pg-lo-pure check clean
## References
- src/gc.rs (`EngineWindowState` — the seam contract and its clause docs)
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
§5 (the applicability table — the pg-lo row; the one-window-host
rule), §6.7 (the executor)
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
§2 (the clause bodies; the gc_window schema)
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
§2 (the durable window's semantics)
- src/sqlite.rs, src/postgres.rs (the window family's reference
implementations — same semantics, different dialects)
- tasks/engine-pg-lo.md (the driver posture this rides; the
shared-helper design point)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+237
View File
@@ -0,0 +1,237 @@
---
id: engine-pg-lo
name: pg-lo large engine — Large Objects behind the Backend trait
status: pending
depends_on: []
scope: moderate
risk: medium
impact: component
level: implementation
tags: [wave-5, engines, pg-lo, postgres]
---
## Description
Implement the `pg-lo` large-tier engine (feature `pg-lo`, default-off;
ADR-009, admitted on POC #7 — findings C1–C7 are the evidence base)
against the wave-1 trait (`src/backend.rs`) in `src/pg_lo.rs` (the
2-line module-doc stub behind the pre-existing
`#[cfg(feature = "pg-lo")]` gate). The postgres kv engine
(`src/postgres.rs`, wave 4) is the structural twin for the driver
posture (pool, prepared statements, schema bootstrap, error mapping);
the local engine (`src/local.rs`) is the structural twin for the
large-tier bodies (streaming read cursor, staged put). **`pg-lo` does
not imply `postgres`** (ADR-013 §4's consequence; the Cargo.toml
feature carries its own driver deps) — every shared piece must
compile under `--no-default-features --features pg-lo` (the pg-lo-pure
build).
- **Engine shape (POC #7 C1 is the contract)** — two artifacts: LOs
hold the bytes; the companion table
`lo_entries (key bytea PRIMARY KEY, loid oid, size bigint,
committed_at)` is the **contract authority** — `list()`, `size()`,
`has()`, CAS-existence are all table-level. **Never trust or read
`pg_largeobject` for addressing** (the C6 sweep validates against
the table only). One engine-owned PostgreSQL schema (the postgres
engine's bootstrap pattern: idempotent DDL at the constructor, the
quoted-identifier boundary, no migration machinery). The default
schema name and the consolidation shape's schema sharing (both
engines on one pg instance) are **recorded design points**.
- **Driver posture (the postgres engine's, reused)** — deadpool-postgres
over tokio-postgres, pool size configurable with a named shipped
default, **per-connection `prepare_cached` discipline** (statements
die with sessions), natively async — **no `spawn_blocking` seam**, no
blocking I/O on the async path. **Shared-helper design point**: the
postgres engine's crate-internal helpers (`quote_identifier`, the
SQLSTATE 42P01 discriminant, `pg_err`, the i64-nanos timestamp
helpers) live behind `#[cfg(feature = "postgres")]` today — decide
and record the sharing shape (the natural one: a crate-internal
pg-common module gated `cfg(any(feature = "postgres",
feature = "pg-lo"))`, one module per file, so the pg-lo-pure build
compiles without duplication). The `PgOpts` question rides the same
point: it is `pub` and re-exported behind `postgres` (wave 4's
constructor signature put it on the public surface) — widening its
gate to `any(postgres, pg-lo)` (re-export gate widened with it) vs a
pg-lo-local opts type is a recorded design point; whether pg-lo
carries the `synchronous_commit` knob at all (the POC ran the server
with it off; the kv parity knob's posture may or may not carry) is
recorded with it.
- **Put = one transaction (ADR-009 §2)** — BEGIN → `lo_create` →
chunked `lowrite` → companion-row insert → COMMIT. **512 KiB
statements** (POC #7 C2's measured knee; statement count, not
LOBLKSIZE, is what costs) as a named shipped constant. CAS:
companion-table existence check first — a re-put of an existing key
is a no-op tx, never an overwrite (the C1 shape). The store core's
put loop hands 64 KiB windows (`PUT_CHUNK`); the engine aggregates
appends into 512 KiB `lowrite` statements (recorded design point).
- **Staged put (the engine-level `put_staged` contract)** — the
staging cursor body is a **held tx**: handout opens the checkout +
tx + `lo_create`; `append` rides the chunked `lowrite` aggregation;
`commit` publishes (companion-row insert + COMMIT — the row and the
LO's pages land in one step, zero crash orphans structurally, C6);
`discard` rolls back and returns the checkout. Every failure path
converges on rollback — stage hygiene holds in both forms (discard →
zero residue: no LO pages, no row; failed commit → self-discard),
exact-count tested (the `#[cfg(test)]` fail-commit knob standing in
for media failure, the mem/sqlite/postgres pattern). **Scope
boundary**: ADR-010 §1's "a tx the store core holds" is the *fleet*
joint form (wave 6's consolidation mechanics — ADR-012 §4/§5's
ruling that pg-lo contributes LO writes + companion-row insert into
the store-held tx); its API shape is a wave-6 design point. This
task ships the engine-level staging cursor only.
- **Get = descriptorless `lo_get(oid, off, len)` windows (ADR-009 §3,
the C4 posture)** — the read cursor closes over the companion row
(oid + length; content immutable, so the row is authoritative), not
over a tx-scoped descriptor. Range reads are single stateless
statements, pool-friendly. `len()` reads the companion row —
**metadata only, never LO content** (ADR-008's discipline). A range
read beyond content length fails cleanly (`UnexpectedEof`-shaped →
the engine's `Error::Io` mapping; the POC's
`ranged_reads_windows_and_eof` shape). **Reads ride the `lo_*`
64-bit variants end-to-end** (`lo_lseek64`/int8 offsets if any
positioning is used; `lo_get`'s bigint offsets already are — the C7
int4 trap must not be inherited). The held-descriptor tx posture
exists as the fallback (measured: no advantage even for sequential
full reads) — named in the module doc as not-shipped, never coded.
- **`has`** — companion-table EXISTS probe. **`size`** — the
companion-row `size` probe (metadata-only; the ADR-009 companion-row
probe the suite's INV-6 row names). **`delete`** — one tx
(`lo_unlink` + row delete), rows-affected discipline
(`Err(Error::Missing)` when absent); GC-participating (the delete
window's arbitration is store-core and table-level — nothing
fs-specific, ADR-009 §4). **`list`** — `SELECT key FROM lo_entries`,
complete (the POC's exact-count gate: list == companion table after
mixed ops).
- **Virgin-store posture** — the constructor creates the schema and
table; the tolerance is the postgres engine's recorded read-only
split (reads on a schema-less database map to absent/empty via the
SQLSTATE 42P01 discriminant, writes surface the engine error). The
schema-less test fixture mirrors the postgres engine's
`#[cfg(test)] new_schemaless` pattern.
- **Posture deltas in the module doc, stated not hidden (ADR-009
§5/§6)** — LO catalog space is reused but never returned
(`pg_total_relation_size('pg_largeobject')` monitoring named;
`VACUUM FULL`/`pg_repack` the only shrink path — an operator
runbook line, never automatic); autovacuum inherited, keep it on;
single-stream cached get 20–50× behind page-cache local fs at
≥1 MiB with the aggregate ~700 MB/s @ 16 readers serving picture —
named capacity facts, not solved problems.
- **Orphan-recovery sweep (ADR-009 §5's requirement)** — a
crate-internal `reap_orphans()` for the legacy/bypass class only
(committed LOs without companion rows): enumerate
`SELECT DISTINCT loid FROM pg_largeobject` as a candidate list
**only**, check each against `lo_entries.loid` (the truth),
`lo_unlink` the unclaimed (the C6/pgdiag10 shape — cost scales with
oid count, not content size). Scope-noted `#[allow(dead_code)]`
until a consumer exists (recorded: the ops/maintenance surface is
wave 7+; no store-core verb is scoped this wave), contract-tested
with a planted orphan and a byte-exact survival assertion for
committed content.
- **Unit tests** in-module: the method contract (virgin-store no-ops
incl. the schema-less form, CAS re-put no-op, Missing discipline,
size exactness + metadata-only, list completeness after mixed ops,
ranged reads at window boundaries + beyond-length failure), the
staged-put happy path and both hygiene forms, the orphan sweep.
**Tests need a live server** — the alkstore harness convention:
connection settings ride the environment
(`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`), tests skip cleanly
server-less, acceptance requires green against the server. The
harness server: dockerized `pglo-poc` (`postgres:16-alpine`) on
:15432, db `blobs`, user/password `postgres`/`poc` (verified live
2026-10-12).
- **LO isolation on the shared harness db (named design point — the
C7 lesson, do not rediscover it)**: Large Objects are
**database-level**, not schema-level — the schema-per-test
convention does not isolate them, a `DROP SCHEMA` leaves the LO's
pages in `pg_largeobject`, and shared-LO tests raced each other into
spurious "large object NNNN does not exist" aborts in the POC. Decide
and record the isolation shape: per-test `CREATE DATABASE` (the
POC's proven `open_fresh` shape) or per-instance LOID tracking +
teardown `lo_unlink` (a `LoJanitor` drop-guard, the SchemaJanitor
pattern extended). Either way acceptance includes **zero LO residue
on the shared db** (`pg_largeobject` probed before/after full runs).
- **Review amendment A2 (wave-4 gate, owner: this task)** — adopt the
schema-janitor discipline from the start; the wave-4 lesson (380
leaked schemas) is not to be re-learned. The `SchemaJanitor` lives
in `src/postgres.rs`'s `#[cfg(test)] test_support` — its new shared
home (gated `cfg(test)` + `any(postgres, pg-lo)`) rides the
shared-helper design point.
The window-family seam clauses (`EngineWindowState`) are
`engine-pg-lo-window` (separate task); the constructor wiring is
`store-pg-lo-construction`. **Nothing of `EngineFleetState` may exist
on this engine** (ADR-012 §5's pg-lo row: pin clauses **no**, sweep
lock **no** — fleet pins live on the kv host). This task ships the
engine and its in-module unit tests only.
## Acceptance Criteria
- [ ] `src/pg_lo.rs` implements `Backend` completely (seven methods +
`put_staged`) behind feature `pg-lo` (default-off), with the
descriptorless `lo_get` window cursor body and the held-tx
staging body
- [ ] The companion table is the contract authority (list/size/has/CAS
all table-level); `pg_largeobject` is never read for addressing
- [ ] One engine-owned schema, idempotent bootstrap at the
constructor; the quoted-identifier boundary; the virgin-store
read-only split (42P01 → absent/empty on reads, writes surface)
unit-tested incl. the schema-less form
- [ ] Put = one tx (lo_create → 512 KiB chunked lowrite → row insert →
COMMIT); CAS re-put is a no-op; the 512 KiB knee is a named
constant with the C2 citation
- [ ] Stage hygiene holds in both forms (discard → zero residue; failed
commit → rollback self-discard), exact-count unit-tested
- [ ] Reads ride `lo_get(oid, off, len)` windows; `len()` metadata-only
from the companion row; beyond-length reads fail cleanly; the
64-bit-variants discipline recorded (the C7 trap not inherited)
- [ ] `delete` = one tx (lo_unlink + row delete) with the Missing
discipline; `list` complete after mixed ops
- [ ] The orphan-recovery sweep exists (candidates from
`pg_largeobject` only, validated against `lo_entries`),
scope-noted, contract-tested with a planted orphan
- [ ] Posture deltas recorded in the module doc (catalog space never
returned + VACUUM FULL operator line + rel-size monitoring;
autovacuum inherited; the cached-get capacity facts)
- [ ] Shared-helper + PgOpts + LO-isolation design points decided and
recorded; the pg-lo-pure build (`--no-default-features
--features pg-lo`) compiles clean
- [ ] Unit tests env-driven (`ALKBLOBS_PG_*`), skipping cleanly
server-less; green against the harness server at acceptance;
zero schema + LO residue on the shared db after runs
- [ ] `cargo test` (default — pg-lo compiles out) green;
`cargo test --features pg-lo` green server-less (skipping);
`cargo clippy --all-targets -- -D warnings` and `cargo fmt
--check` green (default + `--features pg-lo` + the pg-lo-pure
check)
## References
- docs/architecture/decisions/009-pg-lo-large-tier-engine.md (the
engine ADR — §1–§4 the contract, §5/§6 the posture deltas)
- docs/research/poc-pglo-findings.md (findings C1–C7 — the evidence
base; C2 the 512 KiB knee, C4 the window posture, C6 the orphan
sweep, C7 the harness lessons)
- docs/architecture/decisions/003-backend-contract-and-dispatch.md
(the trait contract)
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
§1 (the staged-put contract; the large-tier cursor body families)
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
§4/§5 (the consolidation posture; the pg-lo applicability row —
window family only, no fleet state)
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
§4 (`pg-lo` does not imply `postgres`; the pg-lo-pure build)
- src/backend.rs (the trait + seam contracts this implements)
- src/postgres.rs (the driver-posture twin: pool, prepare_cached,
schema bootstrap, 42P01 split, test_support janitor)
- src/local.rs (the large-tier body twin: streaming cursor, staged
put hygiene shapes)
- /workspace/alkblobs-pglo-poc (the POC crate — the ~400-line
PgLoBackend this engine productionizes)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+120
View File
@@ -0,0 +1,120 @@
---
id: review-wave-5
name: Wave 5 review gate — the pg-lo engine vs the contract
status: pending
depends_on: [suite-column-pg-lo]
scope: narrow
risk: low
impact: phase
level: review
tags: [wave-5, review]
---
## Description
The wave-5 review gate — the engine-wave review shape
(docs/plans/implementation.md §Review gates: "engine-vs-contract
conformance"), before wave 6 decomposes. The gate reads the
implementation against the pinned ADR text line by line (the
wave-1/2/3/4 gates' method).
Gate checks:
- **Engine-vs-contract conformance**: the suite column green against
dockerized postgres (server-less runs skip cleanly — a skip never
substitutes for acceptance); the engine-state clause families
contract-tested per the applicability row (ADR-012 §5's pg-lo row —
the window family only, conditional per the one-window-host rule);
virgin-store semantics; the `size` metadata-only discipline (the
companion-row probe never loads LO content).
- **The engine shape vs ADR-009 §1–§4**: the companion table is the
contract authority (list/size/has/CAS all table-level);
`pg_largeobject` never read for addressing; put = one tx with the
512 KiB knee as a named constant (the C2 citation); the staged put's
held-tx body with both hygiene forms exact-count tested; gets ride
descriptorless `lo_get(oid, off, len)` windows (the C4 posture — the
held-descriptor fallback named as not-shipped, never coded); the
64-bit-variants discipline (the C7 int4 trap not inherited); delete
= one tx under table-level arbitration; transactional LO creation
(zero crash orphans structurally — the C6 reading).
- **The posture deltas vs ADR-009 §5/§6**: carried as requirements,
not hedges — catalog space never returned (the `VACUUM FULL`
operator line + rel-size monitoring named in the module doc);
autovacuum inherited, keep it on; the orphan-recovery sweep present
for the legacy/bypass class (scope-noted, contract-tested with a
planted orphan); the cached-get capacity facts named honestly.
- **The one-window-host rule vs ADR-012 §5**: the store-core
completion — `ensure_gc_schema` follows the host rule at
construction (dormancy on dual-SQL: the consolidation shape's pg-lo
schema carries no `gc_window`; the pg-lo host on kv=mem); the sweep
form branch runs the durable form on the pg-lo host for kv=mem +
large=pg-lo and keys off the engine representation, never a flag;
**no fleet state anywhere in pg-lo** (nothing of
`EngineFleetState`); the fleet joint-put contribution's API shape
explicitly deferred to wave 6 (recorded, not silently absent).
- **The consolidation constructor vs ADR-009/011/012**: the
fleet-valid engine set completed; the implicitly-`fleet: false`
posture recorded verbatim (wave 6's declaration slots in without
reshaping); the connection-settings design point (two DSN+opts
pairs; cross-instance valid non-fleet) recorded.
- **Async/blocking discipline**: tokio-postgres natively async — no
`spawn_blocking` seam; no blocking I/O on the async path; the pool
is the concurrency posture; no `unwrap`/`expect` outside tests.
- **Named design points audited as resolved** (each recorded in its
task's Notes): the schema name + the consolidation shape's schema
sharing; the shared-helper home (quote_identifier, 42P01
discriminant, pg_err, timestamp helpers) + the `PgOpts` gate
question; the 512 KiB aggregation shape; the LO isolation +
janitor shape (amendment A2 verified — zero schema AND zero
`pg_largeobject` residue on the shared db); the sub-enum slotting
for the large cursor/staging bodies; the connection-settings shape;
the dormancy wiring; the orphan sweep's scope note.
- **Family conventions**: no comments in code (doc comments on public
API fine), thiserror, tokio-only async, no unwrap/expect outside
tests, naming, one-module-per-file re-export shape, the
`#![allow(dead_code)]` scope notes matching their remaining scope.
- **Gates green**: `cargo test` (default), `cargo test --features
pg-lo` (server-less: skipping), `cargo test --features "pg-lo,mem"`,
`cargo test --all-features` (against the server), the pg-lo-pure
checks (`cargo check --no-default-features --features pg-lo` and
`--features "pg-lo,mem"`), `cargo clippy --all-targets -- -D
warnings` (default + the pg-lo variants + `--all-features`),
`cargo fmt --check`, `cargo doc --no-deps`.
Findings are fixed inline where cheap, or recorded as wave-6
amendments where they shape the next decomposition (the fleet
mechanism's joint-tx driver already carries amendment A1; any pg-lo
contribution-shape finding lands beside it). The gate's completion is
what unblocks wave 6's decomposition.
## Acceptance Criteria
- [ ] Every gate check above read and verified against the cited ADR
text (not from memory of it)
- [ ] Named design points each decided, recorded, and consistent with
the specs
- [ ] Findings fixed inline or recorded as wave-6 amendments with
owners
- [ ] All gates re-run green after any fixes
- [ ] Wave 6 decomposition unblocked (recorded in this task's Summary)
## References
- docs/plans/implementation.md (§Wave 5, §Review gates — engine wave
reviews)
- docs/architecture/decisions/009-pg-lo-large-tier-engine.md
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §1/§2
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §3/§4/§5/§6.7
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §4
- docs/architecture/store-api.md
- docs/research/poc-pglo-findings.md (the evidence base the engine
claims)
- tasks/review-wave-4.md (the gate method; the structural twin's gate)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+166
View File
@@ -0,0 +1,166 @@
---
id: store-pg-lo-construction
name: pg-lo constructor wiring — the consolidation shape and the window-host completion
status: pending
depends_on: [engine-pg-lo, engine-pg-lo-window]
scope: moderate
risk: medium
impact: component
level: implementation
tags: [wave-5, store, pg-lo, construction]
---
## Description
Slot pg-lo into the facade's engine representation — the slotting-in
pattern `src/store.rs`'s module doc pinned ("adding an enum variant and
a named constructor is slotting in"). This task also completes the
one-window-host rule's store-core wiring, which is more than slotting:
the durable window gains its large-engine host arm.
- **`LargeEngine::PgLo`** variant behind `#[cfg(feature = "pg-lo")]` +
the six large-op dispatch arms (`put_staged`/`get`/`has`/`size`/
`delete`/`list`).
- **The large cursor bodies slot in** — `AnyCursor::Large` and
`AnyStaging::Large` are currently **concrete over the local
bodies** (`LocalCursor`/`LocalStaging`). **Named design point (the
recorded `KvCursor` sub-enum pattern's large-tier twin)**: grow a
`LargeCursor`/`LargeStaging` sub-enum behind `AnyCursor`/`AnyStaging`'s
one large arm (local + pg-lo variants) rather than one `AnyCursor`
variant per large engine — the same bounded-memory argument as the
kv side, recorded for the review gate, with the store.rs design-point
notes updated to the landed state.
- **Named constructors**, feature-gated on `pg-lo` (default-off):
- **`dual_tier_postgres_pglo`** — kv = postgres + large = pg-lo: the
**consolidation posture** (ADR-009's "one engine to operate"); its
landing **completes the fleet-valid engine set** (the store.rs
module doc's recorded forward reference — pg-lo's wave-5
constructor is the last fleet-valid engine). **Implicitly
`fleet: false`** (ADR-012 §3): the ADR-005 in-process protocol
applies verbatim; the `fleet: true` declaration is wave 6 (the
composite fleet-validity predicate + constructor declarations)
slotting in without reshaping — no fleet parameter carried;
recorded in the constructor doc. **Connection-settings design
point**: two DSN + opts parameter pairs (kv and large) —
cross-instance kv=postgres + large=pg-lo is valid non-fleet
(ADR-012 §4), and the consolidation shape passes the same
DSN/opts to both; the suite factory's schema-sharing corollary
(one minted schema passed to both engines vs two schemas) is
recorded with it.
- **`dual_tier_mem_pglo`** — kv = mem + large = pg-lo (features
`mem` + `pg-lo`): **the one-window-host rule's named test
topology** (ADR-012 §5: the durable window hosts on the large
engine when the kv engine is not SQL-backed). Non-fleet (mem is
never fleet-valid).
- **No `kv_only(pg-lo)`** — pg-lo is a large engine; kv-only modes
take kv engines only. **No sqlite+pg-lo constructor this wave** —
valid non-fleet per ADR-012 §5 but not named by the plan; added on
demand (recorded).
- Both take the **delete-window delay** parameter (the
`DEFAULT_DELETE_WINDOW_DELAY` signature shape; `0` the legal
degenerate tuning) — the durable form's cadence knob on either
host.
- **The one-window-host rule's store-core completion** (the substance
of this task):
- **`ensure_gc_schema` at construction** — the call currently
dispatches on the kv-engine enum only. It must follow the host
rule: kv SQL-backed (sqlite/postgres) → the kv host's call (pg-lo's
window clauses dormant — the consolidation shape's pg-lo schema
carries **no `gc_window`**); kv = mem + large = pg-lo → the
**pg-lo host's call** (`gc_window` in the pg-lo schema). The
dispatch consults the large engine only when the kv engine is not
SQL-backed; it keys off the engine representation, never a flag.
- **The sweep form branch** — currently `KvEngine::Mem` runs the
inline form unconditionally. It gains the large-host arm: mem kv +
`LargeEngine::PgLo` runs the **durable form on the pg-lo host**
(`sweep_window_durable` is already generic over
`W: EngineWindowState` — the pg-lo arm slots in); mem kv +
`LargeEngine::Local` (or no large tier) stays inline. The
sqlite/postgres arms are untouched (their hosts win the window per
the rule). The module doc's recorded "pg-lo dormancy slots in
without reshaping" note is updated to the landed state (dormancy
did; the large-host arm is the reshaping the rule always named).
- **Stale scope notes / forward references updated** — the store.rs
module doc's wave-5 mentions (the engine-representation note, the
fleet-valid-set note, the one-window-host paragraph, the
`LargeEngine` doc's "pg-lo slots in at wave 5"), the `PUT_CHUNK` doc
note ("pg-lo's measured 512 KiB `lowrite` knee is that engine wave's
concern" — now landed; the engine's constant is the authority), and
any `#![allow(dead_code)]` scope notes whose scope changes (the
batch-delete helper's wave-6 note stays).
- **Constructor unit tests**: the modes construct and introspect
correctly (`mode()`/`threshold()`); `ensure_gc_schema` ran on the
**right host** in both shapes (consolidation: `gc_pins` + `gc_window`
on the kv schema, **no `gc_window` on the pg-lo schema** — the
dormancy assertion; mem-pglo: `gc_window` on the pg-lo schema); the
window-delay default is the pinned constant, tunable, `0` legal; the
dual-tier shapes round-trip end-to-end (a smoke put/get through both
tiers with LO placement asserted — the companion row + LO present,
kv entries not in pg); a mem-pglo durable-form sweep smoke
(zero-delay: first sweep stages into the pg-lo schema's `gc_window`
and executes nothing, second sweep executes the overdue set
re-arbitrated at execution time — the form-branch-is-large-host
proof). Env-driven (`ALKBLOBS_PG_*`), skipping cleanly server-less
(the engine tests' convention); the janitor discipline rides from
the start (amendment A2).
- **The pg-lo-pure builds** — `cargo check --no-default-features
--features pg-lo` (engine + window family compile, constructors
cfg'd out where `mem` is absent) and
`cargo test --no-default-features --features "pg-lo,mem"` (the
mem-pglo constructor constructible): ADR-013 §4's supported
non-default configurations; verify both compile clean.
## Acceptance Criteria
- [ ] `LargeEngine::PgLo` slots in; all six large ops dispatch; the
large cursor/staging bodies slot through the recorded sub-enum
design point (the store.rs notes updated to the landed state)
- [ ] `dual_tier_postgres_pglo` + `dual_tier_mem_pglo` constructors
exist behind feature `pg-lo`, take connection settings + opts +
threshold + window delay, fallible where I/O is fallible; the
consolidation constructor's doc records the implicitly-non-fleet
posture and the fleet-valid-set completion
- [ ] `ensure_gc_schema` follows the one-window-host rule at
construction (kv host on dual-SQL — pg-lo schema carries no
`gc_window`; pg-lo host on kv=mem — contract-tested both shapes)
- [ ] The sweep form branch runs the durable form on the pg-lo host
for kv=mem + large=pg-lo and the inline form otherwise; the
branch keys off the engine representation, never a flag
- [ ] The connection-settings design point (two DSN+opts pairs; the
consolidation corollary) decided and recorded
- [ ] The pg-lo-pure builds (`--no-default-features --features pg-lo`
and `--features "pg-lo,mem"`) compile clean
- [ ] `cargo test` (default) green; `cargo test --features pg-lo` and
`--features "pg-lo,mem"` green server-less (skipping);
`cargo clippy --all-targets -- -D warnings` and `cargo fmt
--check` green (default + the pg-lo feature variants)
## References
- src/store.rs (the engine-representation design point; the
constructor modes; the slotting-in pattern; the `KvCursor` sub-enum
design point; the sweep form branch; the one-window-host paragraph)
- docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md
§4 (the constructor table; the consolidation posture's mode reading)
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
§3 (fleet activation is a declaration — wave 6), §4 (the GC-state
host; cross-instance valid non-fleet), §5 (the one-window-host rule
and the pg-lo dormancy), §6.3 (dual-tier shapes)
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
§2 (the window delay, default 5 min)
- docs/architecture/decisions/009-pg-lo-large-tier-engine.md (the
consolidation posture this constructor ships)
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
§4 (the feature graph; the pg-lo-pure builds)
- tasks/engine-pg-lo.md, tasks/engine-pg-lo-window.md (the engine +
seam bodies this wires)
- tasks/store-postgres-construction.md (the structural twin's recorded
design points)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion
+146
View File
@@ -0,0 +1,146 @@
---
id: suite-column-pg-lo
name: Conformance suite — the pg-lo column (the LO-medium SKIP-posture column)
status: pending
depends_on: [store-pg-lo-construction]
scope: moderate
risk: medium
impact: phase
level: implementation
tags: [wave-5, suite, pg-lo]
---
## Description
Grow the **pg-lo column** — the suite's second SKIP-posture column
(README §SKIP posture: SQL-attached factories ride a server; when no
server is reachable the column **skips cleanly** — a skip is not a
failure — and the wave's acceptance requires green **against the
server**). The column runs under feature `pg-lo` (default-off); its
two factories span feature pairs — **the first column whose factories
need different feature sets** (a recorded design point): the mem-pglo
factory gates on `all(feature = "mem", feature = "pg-lo")`, the
postgres-pglo factory on `all(feature = "postgres",
feature = "pg-lo")`; the column's full run (acceptance) is
`--all-features` against the server.
- **Factories** (`tests/conformance/pg_lo_column.rs`):
- **`DualTierMemPgLoFactory`** — kv = mem + large = pg-lo: the
one-window-host rule's named test topology. The **durable-schedule
sweep rows run with the window hosted on the large engine**
(`gc_window` in the pg-lo schema) — the shape no earlier column
exercises.
- **`DualTierPostgresPgLoFactory`** — kv = postgres + large = pg-lo
on one pg instance: the consolidation posture. Mirrors the
postgres column's inventory with the large medium swapped to LOs,
and carries the **dormancy row** (the pg-lo schema carries no
`gc_window`; the window lives on the kv schema).
Both at column-declared thresholds and the **test tuning of the
window delay (`0` — execute-on-next-sweep)**. **The alkstore
pg-factory pattern, reused** (the postgres column's landed shape):
a fresh engine-owned schema per `open()` — unique per call, tracked
per-instance in a registry, teardown `DROP SCHEMA IF EXISTS …
CASCADE` over exactly the minted schemas, panicking-row-safe. The
**LO-isolation corollary** (the `engine-pg-lo` design point carries):
LOs are database-level — teardown must also reap the instance's LOs
(tracked LOIDs + `lo_unlink`, or the per-test-database shape if that
is what the engine task recorded), and the structural teardown
assertions include **zero `pg_largeobject` residue**.
- **Connection settings ride the environment** — the harness
convention: `ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`, a
reachability probe gating the rows (the SKIP posture's
factory-owner responsibility). **The harness server**: dockerized
`pglo-poc` (`postgres:16-alpine`) on :15432, db `blobs`,
user/password `postgres`/`poc` — verified live 2026-10-12; the
container is long-running in the dev environment.
- **Mirror the form-independent reference rows** (README §The
reference row set): INV-1 round-trips (content matrix × both put
paths, in-body — the LO medium's ranged reads at window boundaries
and the beyond-length clean failure are this column's INV-1
extensions), INV-6 stat discipline (the companion-row probe —
metadata-only, never LO content), the `NoLivenessSources`/
`ProtectFailed` abort shapes. **No kv-only rejection row** — no
kv-only pg-lo constructor exists (recorded; the postgres column
hosts that row for the SQL kv engines).
- **The durable-schedule sweep rows** — the SQL-backed columns' family
(README §The schedule-relative reading), run on **both factories**:
INV-2 dedup and INV-3 stage hygiene with counts read against the
staged schedule, the two-sweep exact-count row over the full
protection set (both tiers, the window observed — on the pg-lo host
for the mem-pglo factory, on the kv host for the postgres-pglo
factory), the sweep-vs-put race across the window delay
(execution-time re-arbitration), the holder-stages-only lease shape,
and INV-5 list correctness through durable-schedule outcomes.
- **The dual-tier dispatch rows** mirror the postgres column's family
(pure-function routing, threshold boundaries, fall-through per op)
with the placement oracle adapted to the LO medium: large entries
are companion rows + LOs (an independent pg connection counting
`lo_entries` and probing `pg_largeobject`), kv entries per the
factory's kv engine (the mem-pglo factory's kv placement is
in-process — the oracle's kv half follows the mem column's
introspection shape; recorded design point).
- **Virgin-store row**: reads are no-ops on a fresh schema (pool,
`lo_entries`, and the window state per the host rule all empty) —
the constructor always ensures the host's schema, this row being the
facade form.
- **Structural factory tests** (the postgres column's pattern):
factory isolation (fresh schema per open, nothing shared) and
teardown-against-a-panicking-row (schemas really dropped, LOs really
unlinked — zero residue probed).
- **README amendments** (the convention's normative test-side home):
the pg-lo rows table in §Columns beyond the reference set; the
column plan's SKIP-posture note naming the second landed column and
the per-factory feature-pair gating; current state (five columns).
## Acceptance Criteria
- [ ] The pg-lo column green **against the harness server** (`pglo-poc`,
:15432) under `--all-features`: the mirrored reference rows + the
durable-schedule sweep rows on both factories (the pg-lo-host
shape and the kv-host/dormancy shape) + the dispatch rows + the
virgin-store row, every row ADR-stamped and traceable to the
backlog
- [ ] The dormancy row asserts the consolidation shape's pg-lo schema
carries no `gc_window`; the mem-pglo rows assert the window on
the pg-lo host
- [ ] Server-less runs skip cleanly (the reachability probe; the
default-build gates stay green with the column skipped)
- [ ] Factory isolation + idempotent teardown hold (fresh schema per
open; schemas dropped, LOs unlinked; **zero `pg_largeobject`
residue** probed; teardown safe against a panicking row),
structurally exercised
- [ ] README amendments recorded (the pg-lo rows table; the
feature-pair gating note; current state)
- [ ] `cargo test` (default) green — the column compiles out;
`cargo test --features pg-lo` (and the mem/postgres pair
variants) green server-less (skipping) and green against the
server at acceptance; `cargo clippy --all-targets -- -D warnings`
and `cargo fmt --check` green (default + the pg-lo variants +
`--all-features`)
## References
- tests/conformance/README.md (the conventions this grows against;
§SKIP posture, §Columns beyond the reference set, §The
schedule-relative reading for SQL-backed columns)
- tests/conformance/postgres_column.rs (the SKIP-posture column
pattern — the factory shape, the medium oracle, the structural
tests this mirrors)
- tests/conformance/factory.rs (the factory contract this implements)
- tests/conformance/rows.rs (the reference row set being mirrored)
- docs/architecture/store-api.md (§Invariants — the test gate)
- docs/architecture/decisions/009-pg-lo-large-tier-engine.md (the
engine ADR the LO-medium rows stamp against)
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
§5 (the one-window-host rule — the dormancy row's authority), §6.7
(the schedule the sweep rows assert against)
- tasks/engine-pg-lo.md (the LO-isolation design point this factory
inherits)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion