feat(store): the pg-lo constructor wiring — the consolidation shape and the window-host completion (wave 5)

- LargeEngine::PgLo slots in behind `pg-lo` + all six large-op
  dispatch arms; the large cursor/staging bodies land as the
  LargeCursor/LargeStaging sub-enums behind AnyCursor/AnyStaging's
  one large arm (the recorded KvCursor pattern's large-tier twin)
- dual_tier_postgres_pglo (the consolidation posture; two DSN+opts
  pairs, implicitly non-fleet, completing the fleet-valid engine
  set) and dual_tier_mem_pglo (the one-window-host rule's named test
  topology), both carrying the constructor-tunable delete-window
  delay
- ensure_gc_schema_at_construction: the one-window-host rule's
  store-core dispatch (kv host on dual-SQL — pg-lo's window clauses
  dormant; pg-lo host on kv=mem); the sweep's mem arm gains the
  large-host delegation, the branch keyed off the engine
  representation never a flag
- stale scope notes updated to the landed state (store.rs module
  doc, LargeEngine, the sweep docs, PUT_CHUNK, pg_lo.rs, gc.rs)
- 7 new env-driven constructor tests (dedicated-database janitor
  discipline; consolidation + mem-pglo both shapes

Verified live against the pglo-poc harness server: cargo test
default 69/19/3, --features pg-lo 92/19/3, --features "pg-lo,mem"
151/54/4 (server-less 124/37/4), --no-default-features
"postgres,pg-lo" 102/19/3 (both live and server-less),
--all-features 188/71/4; clippy -D warnings and fmt --check green
on all four configurations; cargo doc clean. The literal kv-less
pg-lo config re-confirmed structurally uncompilable (d69d9a3's
finding; not a gate).
This commit is contained in:
glm-5.3-flash committed 2026-10-12 08:04:19 +00:00
1 parent c8fc519978
commit cddcea3cfa
4 files changed
+1305 -115

No files matched your search

+7 -5
View File
@@ -66,11 +66,13 @@
//! //!
//! The `#![allow(dead_code)]` below covers the companion-seam scope: //! The `#![allow(dead_code)]` below covers the companion-seam scope:
//! the window family has its in-tree callers (store construction runs //! the window family has its in-tree callers (store construction runs
//! `ensure_gc_schema` on every SQL-backed kv engine; the sweep's durable //! `ensure_gc_schema` on the window's host — the kv engine when it is
//! form drives `window_stage`/`window_take`), while the fleet family //! SQL-backed, pg-lo when the kv engine is not (the one-window-host
//! waits for wave 6 — its postgres clause bodies landed with wave 4, //! rule, ADR-012 §5); the sweep's durable form drives
//! but the store core consumes the family only under the `fleet: true` //! `window_stage`/`window_take` on that same host), while the fleet
//! declaration. It shrinks as that lands. //! family waits for wave 6 — its postgres clause bodies landed with
//! wave 4, but the store core consumes the family only under the
//! `fleet: true` declaration. It shrinks as that lands.
#![allow(dead_code)] #![allow(dead_code)]
+17 -18
View File
@@ -52,9 +52,9 @@
//! 39–43 MB/s from 512 KiB chunks and collapse below (6 MB/s @8 KiB). //! 39–43 MB/s from 512 KiB chunks and collapse below (6 MB/s @8 KiB).
//! [`LOWRITE_CHUNK`] is the named shipped constant (C2's citation), //! [`LOWRITE_CHUNK`] is the named shipped constant (C2's citation),
//! covering the whole-value put directly and the staging cursor's //! covering the whole-value put directly and the staging cursor's
//! append aggregation (the store core's put loop hands //! append aggregation (the store core's put loop hands 64 KiB windows;
//! `crate::dispatch::PUT_CHUNK`-sized 64 KiB windows; the engine //! the engine aggregates appends into 512 KiB `lowrite` statements).
//! aggregates appends into 512 KiB `lowrite` statements). [`READ_CHUNK`] //! [`READ_CHUNK`]
//! bounds the `lo_get` window: larger windows don't beat the //! bounds the `lo_get` window: larger windows don't beat the
//! server-side per-statement ceiling (~65–70 MB/s per in-flight //! server-side per-statement ceiling (~65–70 MB/s per in-flight
//! statement, C2/C3) and the int4 length parameter caps at 2 GiB. //! statement, C2/C3) and the int4 length parameter caps at 2 GiB.
@@ -175,16 +175,15 @@
//! Nothing of [`crate::gc::EngineFleetState`] exists on this engine //! Nothing of [`crate::gc::EngineFleetState`] exists on this engine
//! (ADR-012 §5's pg-lo row: pin clauses **no** — fleet pins live on //! (ADR-012 §5's pg-lo row: pin clauses **no** — fleet pins live on
//! the kv host; sweep lock **no**). The window-family seam clauses //! the kv host; sweep lock **no**). The window-family seam clauses
//! ([`crate::gc::EngineWindowState`]) are implemented here (the //! ([`crate::gc::EngineWindowState`]) are implemented here and wired
//! engine-pg-lo-window task): `ensure_gc_schema` creates `gc_window` //! by construction (landed with the constructor wiring): the store
//! only — never `gc_pins`, never the embedder's liveness table — and //! core's construction call follows the one-window-host rule — active
//! the window clauses are conditional per the one-window-host rule //! on the kv=mem + large=pg-lo shape (this module's `ensure_gc_schema`
//! (dormant on a dual-SQL instance — kv=sqlite/postgres + large=pg-lo //! runs at construction; its `gc_window` hosts the durable window),
//! stages on the kv engine; active on the kv=mem + large=pg-lo shape). //! dormant on a dual-SQL instance (kv=sqlite/postgres + large=pg-lo:
//! Whether the store core drives them at construction is the //! the kv host's call runs, this schema carries no `gc_window`);
//! constructor wiring's business (`store-pg-lo-construction`); this //! `ensure_gc_schema` creates `gc_window` only — never `gc_pins`,
//! module ships the clauses, their engine-level contract tests, and //! never the embedder's liveness table.
//! the rest of the engine (unit tests only).
//! //!
//! Tests need a live server: connection settings ride the environment //! Tests need a live server: connection settings ride the environment
//! (`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`); tests skip cleanly //! (`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`); tests skip cleanly
@@ -455,11 +454,11 @@ impl EngineWindowState for PgLoBackend {
/// `gc_pins` is never created here (ADR-012 §5: fleet pins live on /// `gc_pins` is never created here (ADR-012 §5: fleet pins live on
/// the kv host) and the embedder's liveness table is never created /// the kv host) and the embedder's liveness table is never created
/// in any case (ADR-008's schema-ownership rule). Idempotent, so /// in any case (ADR-008's schema-ownership rule). Idempotent, so
/// the twice-at-open call form is a no-op. Whether the store core /// the twice-at-open call form is a no-op. The store core calls
/// calls this at construction is the one-window-host rule's /// this at construction exactly when this engine hosts the window
/// business (`store-pg-lo-construction` wires the active host) — /// (the one-window-host rule's large-host arm — kv = mem +
/// the clause body exists and is contract-tested per applicability /// large = pg-lo); on a dual-SQL instance the kv host's call runs
/// regardless of dormancy. /// and this stays dormant, never invoked at construction.
async fn ensure_gc_schema(&self) -> Result<(), Error> { async fn ensure_gc_schema(&self) -> Result<(), Error> {
let client = self.pool.get().await.map_err(err)?; let client = self.pool.get().await.map_err(err)?;
client client
+1165 -82
View File
File diff suppressed because it is too large. Load diff
+116 -10
View File
@@ -1,7 +1,7 @@
--- ---
id: store-pg-lo-construction id: store-pg-lo-construction
name: pg-lo constructor wiring — the consolidation shape and the window-host completion name: pg-lo constructor wiring — the consolidation shape and the window-host completion
status: pending status: completed
depends_on: [engine-pg-lo, engine-pg-lo-window] depends_on: [engine-pg-lo, engine-pg-lo-window]
scope: moderate scope: moderate
risk: medium risk: medium
@@ -125,27 +125,27 @@ the durable window gains its large-engine host arm.
## Acceptance Criteria ## Acceptance Criteria
- [ ] `LargeEngine::PgLo` slots in; all six large ops dispatch; the - [x] `LargeEngine::PgLo` slots in; all six large ops dispatch; the
large cursor/staging bodies slot through the recorded sub-enum large cursor/staging bodies slot through the recorded sub-enum
design point (the store.rs notes updated to the landed state) design point (the store.rs notes updated to the landed state)
- [ ] `dual_tier_postgres_pglo` + `dual_tier_mem_pglo` constructors - [x] `dual_tier_postgres_pglo` + `dual_tier_mem_pglo` constructors
exist behind feature `pg-lo`, take connection settings + opts + exist behind feature `pg-lo`, take connection settings + opts +
threshold + window delay, fallible where I/O is fallible; the threshold + window delay, fallible where I/O is fallible; the
consolidation constructor's doc records the implicitly-non-fleet consolidation constructor's doc records the implicitly-non-fleet
posture and the fleet-valid-set completion posture and the fleet-valid-set completion
- [ ] `ensure_gc_schema` follows the one-window-host rule at - [x] `ensure_gc_schema` follows the one-window-host rule at
construction (kv host on dual-SQL — pg-lo schema carries no construction (kv host on dual-SQL — pg-lo schema carries no
`gc_window`; pg-lo host on kv=mem — contract-tested both shapes) `gc_window`; pg-lo host on kv=mem — contract-tested both shapes)
- [ ] The sweep form branch runs the durable form on the pg-lo host - [x] The sweep form branch runs the durable form on the pg-lo host
for kv=mem + large=pg-lo and the inline form otherwise; the for kv=mem + large=pg-lo and the inline form otherwise; the
branch keys off the engine representation, never a flag branch keys off the engine representation, never a flag
- [ ] The connection-settings design point (two DSN+opts pairs; the - [x] The connection-settings design point (two DSN+opts pairs; the
consolidation corollary) decided and recorded consolidation corollary) decided and recorded
- [ ] The pg-lo pure builds (`--no-default-features --features - [x] The pg-lo pure builds (`--no-default-features --features
"pg-lo,mem"` and `--features "postgres,pg-lo"`) compile clean "pg-lo,mem"` and `--features "postgres,pg-lo"`) compile clean
(the literal kv-less config is not a gate — see the Description (the literal kv-less config is not a gate — see the Description
bullet) bullet)
- [ ] `cargo test` (default) green; `cargo test --features pg-lo` and - [x] `cargo test` (default) green; `cargo test --features pg-lo` and
`--features "pg-lo,mem"` green server-less (skipping); `--features "pg-lo,mem"` green server-less (skipping);
`cargo clippy --all-targets -- -D warnings` and `cargo fmt `cargo clippy --all-targets -- -D warnings` and `cargo fmt
--check` green (default + the pg-lo feature variants) --check` green (default + the pg-lo feature variants)
@@ -174,8 +174,114 @@ the durable window gains its large-engine host arm.
## Notes ## Notes
> To be filled by implementation agent > Implemented in `src/store.rs` + `src/pg_lo.rs` + `src/gc.rs` (wave 5,
> following `store-postgres-construction` — the structural twin — point
> for point). Recorded design points, as resolved:
>
> - **The large-tier sub-enums landed** — `LargeCursor` and
> `LargeStaging` behind `AnyCursor`/`AnyStaging`'s one large arm
> (the `KvCursor` pattern's large-tier twin, on the same
> bounded-variant argument; recorded for the review gate): local +
> pg-lo variants, `LargeStaging` carrying a scoped
> `#[allow(clippy::large_enum_variant)]` (the file-descriptor
> posture vs the held pooled checkout + tx — payloads structurally
> dissimilar, dispatch by reference/consumption; the `KvEngine`
> enum's recorded allow posture). No reshaping of `AnyCursor`/
> `AnyStaging`/`Blob`.
> - **The one-window-host rule's store-core completion** —
> `ensure_gc_schema_at_construction(kv, large)`: the store-core
> dispatch the constructors call, keyed off the engine
> representation (the kv arm on a SQL-backed kv engine; the large
> arm consulted only when the kv engine is not SQL-backed; in-
> process combinations structurally nothing). The sqlite and
> postgres constructors were re-routed through it (`kv_only_*` pass
> `None`; the dual-tier locals pass `Some(&large)`) — their
> semantics unchanged, and the mem arm's large-host branch is where
> the rule completes. The sweep's form branch gained exactly that
> arm: `KvEngine::Mem` matches on `self.large()` — `LargeEngine::PgLo`
> runs `sweep_window_durable` on the pg-lo host (already generic
> over `W: EngineWindowState`, so the arm is a delegation), anything
> else stays inline; sqlite/postgres arms untouched.
> - **Connection-settings design point** — the consolidation
> constructor takes **two DSN + opts pairs** (kv first, large
> second): cross-instance is valid non-fleet (ADR-012 §4), and the
> consolidation shape passes the same DSN/opts to both. Recorded
> with it: the suite factory's corollary (one minted schema passed
> to both engines vs two schemas — both engines' table names are
> distinct, so either mint is legitimate). The mem-pglo constructor
> takes one pair (kv = mem takes none).
> - **No `kv_only(pg-lo)`** (pg-lo is a large engine) and **no
> sqlite+pg-lo constructor this wave** (valid non-fleet per
> ADR-012 §5, not named by the plan) — both recorded in the
> store.rs module doc's constructor-modes section.
> - **Stale scope notes updated to the landed state** — the
> engine-representation doc's wave references, the dual-tier
> shapes' wave bullets, the fleet-declaration paragraph (the
> fleet-valid engine set is complete in the constructors — wave 6
> is the declaration's predicate and parameter alone), the
> one-window-host paragraph (dormancy landed; the large-host arm is
> the reshaping the rule always named), the `LargeEngine` doc, the
> `KvEngine::ensure_gc_schema` doc (now the kv-host arm the helper
> consumes), the sweep's docs (two forms' host sets), the
> `PUT_CHUNK` note (the engine's `LOWRITE_CHUNK` constant is the
> authority — landed), and the pg_lo/gc.rs module docs' forward
> references. The batch-delete helper's wave-6 note stays.
> - **Clippy scope note** — one `doc list item without indentation`
> fixup during the gate: a doc line beginning `+ large=pg-lo)`
> re-wrapped (the `+` was a markdown list marker).
> - **Tests (7 new, env-driven `ALKBLOBS_PG_*`, skipping cleanly
> server-less, janitor discipline from the start — amendment A2)** —
> two modules keyed to the constructor feature gates. The mem-pglo
> module (4 tests, on per-test dedicated databases — the C7 lesson's
> shape — with `DatabaseJanitor`): construction + introspection +
> the pg-lo-host schema assertions (`gc_window` present, `gc_pins`
> never); the end-to-end round-trip with LO placement asserted (kv
> entries never in the pg database, companion row present, size
> exact, LO content byte-exact); the window-delay
> default/tunings/zero; and the durable-form sweep smoke (zero
> delay: first sweep stages into the pg-lo schema's `gc_window` and
> executes nothing — the form-branch-is-large-host proof — second
> sweep executes the overdue set re-arbitrated at execution time).
> The consolidation module (3 tests, same dedicated-database
> discipline): construction + the kv-host-wins assertions (both GC
> tables in the kv schema, no `gc_window` in the pg-lo schema — two
> distinct schemas so the dormancy assertion observes genuine
> absence); the round-trip in the recorded corollary form (the same
> DSN + one minted schema to both engines, LO placement asserted);
> and the window-delay tunings. The test-side `plant_large` helpers
> in the existing test modules gained `PgLo` arms (the same
> all-features-exhaustiveness pattern as wave 4's `KvEngine` arms).
> - **The literal kv-less config (`--no-default-features --features
> pg-lo`) remains structurally uncompilable** (E0004 on the empty
> kv set, by-reference match arms requiring inhabitated variants)
> — the d69d9a3 finding, not a gate; verification used the two
> meaningful pure builds, both clean.
## Summary ## Summary
> To be filled on completion > Completed 2026-10-12 (wave 5). pg-lo slotted into the facade:
> `LargeEngine::PgLo` with all six large-op dispatch arms, the
> `LargeCursor`/`LargeStaging` sub-enums behind `AnyCursor`/`AnyStaging`'s
> one large arm (the recorded `KvCursor` pattern's large-tier twin),
> the one-window-host rule's store-core completion
> (`ensure_gc_schema_at_construction` — kv host on dual-SQL, pg-lo
> host on kv=mem; the sweep's mem arm gains the large-host delegation,
> the branch keyed off the engine representation never a flag), and
> the two named constructors (`dual_tier_postgres_pglo` — the
> consolidation posture, two DSN+opts pairs, implicitly non-fleet,
> completing the fleet-valid engine set — and `dual_tier_mem_pglo` —
> the one-window-host rule's named test topology), both carrying the
> constructor-tunable delete-window delay (`DEFAULT_DELETE_WINDOW_
> DELAY` pinned at 5 min, `0` legal). Stale scope notes updated to the
> landed state (store.rs module doc, `LargeEngine`, the sweep docs,
> `PUT_CHUNK`, pg_lo.rs, gc.rs). Verification (harness server live
> 2026-10-12): `cargo test` 69/19/3 ok (default); `cargo test
> --features pg-lo` 92/19/3 ok live and green server-less; `cargo test
> --features "pg-lo,mem"` 151/54/4 ok live, 124/37/4 ok server-less;
> `cargo test --no-default-features --features "postgres,pg-lo"`
> 102/19/3 ok live and server-less; `cargo test --all-features`
> 188/71/4 ok (7 new constructor tests green live); clippy `-D
> warnings` green on default, all-features, `pg-lo,mem`, and
> `postgres,pg-lo`; `cargo fmt --check` green; `cargo doc --no-deps`
> clean. The literal kv-less config re-confirmed structurally
> uncompilable (d69d9a3's finding; not a gate).