- tests/conformance/pg_lo_column.rs (new): the two pair-gated factories (DualTierMemPgLoFactory on all(mem, pg-lo) — the one-window-host rule's named test topology; DualTierPostgresPgLoFactory on all(postgres, pg-lo) — the consolidation posture, two schemas per open, carrying the dormancy row), a fresh dedicated database per open with the engine-owned schemas minted inside, teardown reaping tracked LOIDs (lo_unlink) then dropping the minted schemas then the database (WITH (FORCE)) with per-open drop-guard janitors for the abandoned paths, and the rows: the verbatim form-independent mirror (INV-1, INV-6, the two no-op abort shapes), the schedule-relative INV-2/3, the two-sweep exact counts over the full protection set across each factory's observed window host, the window-delay race, the holder-stages-only lease, INV-5's durable row, the three dispatch rows over the LO-medium placement oracle, the INV-1 extension (window-edge ranged reads + the planted-overclaim beyond-length clean failure), the virgin-store rows, and the dormancy row; structural isolation + panicking-row tests probing zero orphan residue (no kv-only rejection row — recorded; pg-lo is a large engine) - the LO-isolation corollary carried in the engine task's recorded per-test-database shape (the C7 lesson): zero LO residue on the shared harness database is structural, so the engine suite's shared_harness_db_holds_zero_lo_residue gate stays green alongside the column under --all-features - main.rs: the pg_lo_column gate (feature pg-lo, default-off) + the columns' doc paragraph - README: the pg-lo rows table in §Columns beyond the reference set, the SKIP-posture note naming the second landed column and the per-factory feature-pair gating, current state = five columns - store.rs: one mechanical gate fix — the PgOpts import was unused in the sqlite+pg-lo config (tripping clippy --features pg-lo -D warnings); the gate now names the configs that consume it Verification (harness server live 2026-10-12): --all-features against the server fully green (188 lib + 106 conformance + 4 git-interop; zero LO/schema/database residue after the run); default test green (69/19/3); --features "pg-lo,mem" and "postgres,pg-lo" green server-less (skipping cleanly) and --features pg-lo alone; clippy -D warnings green on default, pg-lo, pg-lo+mem, postgres+pg-lo, all-features; fmt --check green; cargo doc --no-deps clean.
545 lines
37 KiB
Markdown
545 lines
37 KiB
Markdown
# The conformance suite — home and conventions
|
||
|
||
The in-crate conformance harness for alkblobs' engines. **Decided
|
||
point** (docs/plans/implementation.md, 2026-10-11): engines are
|
||
features of one crate, so the suite lives in-crate, factory-
|
||
parameterized over engine constructors — the alkstore
|
||
ADR-017/022 contract-suite pattern adapted (there a sibling suite
|
||
crate over `Box<dyn Store>`; here one test target over factories
|
||
whose `open` vends the future `Store` facade, whose concrete type
|
||
wave 2 binds).
|
||
|
||
This document is the convention's normative home on the test side
|
||
(the in-code module docs in `main.rs`/`factory.rs`/`stamp.rs` restate
|
||
their slices). Waves 2–6 grow **columns** against it: one column per
|
||
engine, each column the same rows run against that engine's factory.
|
||
|
||
## Row anatomy
|
||
|
||
A row is one named, independently-runnable property — an async
|
||
function taking a `StoreFactory` (not a store), asserting by panicking
|
||
on violation (a failing row is a test failure naming the property and
|
||
expectation; the family-wide no-panic rule is relaxed here by the
|
||
suite's test-side purpose).
|
||
|
||
### Row naming
|
||
|
||
- `snake_case`, and the name **states the property, not the engine**
|
||
(e.g. `round_trip_known_length`, `dedup_is_one_pool_entry`). The
|
||
engine dimension is the *column* (which factory the row runs
|
||
against), never the row name — that is what makes rows
|
||
engine-uniform.
|
||
- Dimensions a property spans *within* one run — the content matrix,
|
||
both put paths, both tiers where the invariant ranges over them —
|
||
loop inside the row's body. A genuinely different property (e.g.
|
||
sweep safety's fleet form vs its single-node form) is a separate
|
||
row, not a flag inside the owner.
|
||
- Planned row families are registered in the **invariant backlog**
|
||
(table below); a row that does not trace to a backlog entry is not
|
||
a conformance row.
|
||
|
||
### Contract stamps (the ADR-017 §4.2 convention adapted)
|
||
|
||
Every row is version-stamped with a doc-comment list item beginning
|
||
**`Contract stamp:`**, citing the spec § and/or ADR § that pins the
|
||
behavior tested:
|
||
|
||
```text
|
||
/// <one-line statement of the property>
|
||
///
|
||
/// Contract stamp: store-api.md §Invariants 2; ADR-003 (whole-value
|
||
/// put is CAS first-writer-wins); ADR-005 (re-put refcounts).
|
||
pub async fn dedup_is_one_pool_entry<F: StoreFactory>(factory: &F) { ... }
|
||
```
|
||
|
||
Rules:
|
||
|
||
- The stamp cites the ADR § / spec § that pins the property — never a
|
||
bare "wave 2" or a date. Several stamps are allowed when one row
|
||
tests several changes' behavior together.
|
||
- A row added to pin a new contract addition is stamped with that
|
||
addition's ADR; a row whose property predates the suite is stamped
|
||
with the ADR/spec section that pinned it.
|
||
- A later change that *amends* a row's property appends to its stamp
|
||
list; rows accumulate stamps, the ADRs' own texts record the
|
||
history.
|
||
- `Contract stamp:` is greppable — `grep -rn "Contract stamp:"
|
||
tests/conformance/` is the stamp inventory.
|
||
- Rows are also free of engine-specific code by construction: a row
|
||
may not branch on `factory.name()`.
|
||
|
||
### One normative owner per property
|
||
|
||
Each property has **exactly one normative owner**: the suite row.
|
||
Engine-specific unit tests inside `src/` may duplicate coverage for
|
||
debugging convenience, but the suite row is the normative assertion —
|
||
an engine's conformance claim rests on its column being green, not on
|
||
its in-crate tests. When the suite itself needs per-engine divergence
|
||
(the fleet-form sweep safety against postgres, INV-4), that is its own
|
||
row with its own owner stamp, still one per property.
|
||
|
||
## The factory contract
|
||
|
||
`factory.rs` pins the [`StoreFactory`] shape. The contract:
|
||
|
||
- **`open()` vends a fresh, isolated store instance.** No state is
|
||
shared between rows or between columns: a fresh backing store per
|
||
call — a fresh temp directory (`local`), a fresh pool (`mem`), a
|
||
fresh schema or file (`sqlite`/`postgres`/`pg-lo`) — so a property's
|
||
rows never observe another property's entries. Isolation is the
|
||
implementer's owned guarantee.
|
||
- **`teardown()` is idempotent** and safe to run against an abandoned
|
||
store: a failing assertion panics the row (early exit, no teardown
|
||
on that path), so teardown must also make sense over a dropped /
|
||
half-open instance. The factory owns what teardown means — deleting
|
||
the temp dir, dropping the schema, closing the pool — and owns the
|
||
isolated-ness guarantee above.
|
||
- Factories are `Send + Sync`: shareable across row runs.
|
||
- **The wave-2 binding — made and recorded.** The `Store` facade type
|
||
landed with wave 2's store-construction task as a concrete type
|
||
(`alkblobs::Store`, engine-blind; src/store.rs holds the engine-
|
||
representation design point). The binding made the anticipated
|
||
shape correction: the skeleton's `Store` associated type collapsed
|
||
into the concrete facade — `open()` vends `alkblobs::Store`
|
||
directly (factory.rs records the ruling; the wave-1 mock probe
|
||
retired with it, replaced by the mem-gated binding demo factories
|
||
in `main.rs`). A new engine column binds by implementing the trait;
|
||
there is no longer a vendible type to choose.
|
||
- **The row body owns its lifecycle.** A row calls `open()` first —
|
||
a fresh, isolated store per call — drives its assertions, and calls
|
||
`teardown()` before returning; a failing assertion panics the row
|
||
*before* teardown runs (the early exit above), which is why
|
||
`teardown()` must make sense over the abandoned instance — dropped
|
||
or still alive at unwind. The mem column's structural tests
|
||
(`factory_teardown_is_safe_against_a_panicking_row`) exercise that
|
||
path; isolation itself (fresh state per open, nothing shared) is
|
||
exercised alongside (`factory_open_vends_fresh_isolated_stores`).
|
||
|
||
## The SKIP posture
|
||
|
||
SQL-attached factories (`postgres`, `pg-lo`) ride a server (dockerized
|
||
postgres) whose connection settings come from the environment. When no
|
||
server is reachable, the column **skips cleanly**: a skip is not a
|
||
failure, and the harness reports it as skipped rather than green or
|
||
red. Two sides to keep honest:
|
||
|
||
- Engine waves' acceptance criteria require green **against the
|
||
server** (the alkstore harness convention) — a skip never substitutes
|
||
for acceptance.
|
||
- Server-less engines (`mem`, `sqlite`, `local`, and any future
|
||
dependency-free engine) never skip: their columns must always be
|
||
green in every environment.
|
||
|
||
## The invariant backlog (rows-to-grow)
|
||
|
||
The seven invariants of store-api.md §Invariants, registered as the
|
||
table every wave's rows grow against. Traceability: each row family
|
||
names its store-api.md invariant and the ADRs pinning it. A suite row
|
||
asserting something outside this table (or outside a wave's specced
|
||
additions) needs a backlog entry *and* its stamp first.
|
||
|
||
| # | Invariant (store-api.md §Invariants) | Row family | Pinning text | First column |
|
||
|---|---|---|---|---|
|
||
| INV-1 | Whole-blob put/get round-trips byte-identical under the canonical derivation — both tiers, both put paths, every shipped engine | one round-trip family; content matrix and put paths loop in-body; per-engine columns | store-api.md §Invariants 1; ADR-002 (derivation); ADR-003 (put paths, dispatch); ADR-007/008/009/011 (engines) | wave 2 — mem (kv tier), then local (large tier) |
|
||
| INV-2 | Dedup: identical content twice (same or different path) is one pool entry | one dedup row, same-path and different-path legs | store-api.md §Invariants 2; ADR-003 (CAS first-writer-wins); ADR-005 (re-put refcounts) | wave 2 — mem |
|
||
| INV-3 | Stage hygiene: failure mid-put leaves zero stage files / no entry; success leaves exactly one committed entry | one hygiene family over both put paths and both failure modes (discard, failed commit) | store-api.md §Invariants 3; ADR-010 §1 (staged-put contract); ADR-003 | wave 2 — mem (degenerate stage) and local |
|
||
| INV-4 | Sweep safety: exact counts with correct liveness; aborting sources delete nothing. Engine-gated fleet form (DB-backed pins + sweep lease vs postgres) | one single-node family; the fleet form is its own engine-gated row family (runs under `--all-features`) | store-api.md §Invariants 4; ADR-005 (windows, sweeps); ADR-008 (fleet form); ADR-012 §1/§4/§6.7; ADR-013 §2 | wave 2 — mem (in-process protocol); fleet form waits for wave 6 |
|
||
| INV-5 | `list()` correctness is observable only through GC | list-correctness asserted **through sweep outcomes** (exact counts), never by comparing lists directly | store-api.md §Invariants 5; ADR-003 (complete + well-formed `list()`); ADR-005 | wave 2 — mem |
|
||
| INV-6 | `stat`/`size` returns entry length without fetching content | one stat-discipline family per engine column | store-api.md §Invariants 6; ADR-008 (`size` probe); ADR-009 (pg-lo companion-row probe) | wave 2 — mem |
|
||
| INV-7 | Engine-conformance uniformity: every engine's bodies pass the same contract suite mem passes | the meta-gate — a column's green run *is* the row; no separate test | store-api.md §Invariants 7; ADR-010 §1; ADR-011 §2 (mem's reference role) | wave 2 — mem defines it |
|
||
|
||
## The reference row set (INV-7's basis)
|
||
|
||
The mem column's rows (rows.rs) are the **reference row set** — the
|
||
required set every other engine's rows mirror (ADR-011 §2, mem's
|
||
contract-reference role; store-api.md §Invariants 7). A column mirrors
|
||
this inventory row-for-row against its own factory; an addition to the
|
||
set propagates to all columns (INV-7). The INV-row families are the
|
||
engine mirror targets — the round-trip family covers the tier the
|
||
column's engine serves (mem: the kv tier; local: the large tier). The
|
||
two constructor-mode rows mirror wherever that configuration is
|
||
constructible: a column with a kv-only factory runs the kv-only row
|
||
against it — the row discovers the mode's threshold by probing, since
|
||
known-length rejections read nothing, and requires the threshold
|
||
materializable for its at-threshold legs (≤ 8 MiB,
|
||
`MAX_BOUNDARY_MATERIALIZATION`).
|
||
|
||
| Row (`rows.rs`) | Traces to |
|
||
|---|---|
|
||
| `round_trip_puts_and_gets_are_byte_identical` | INV-1 (content matrix, both put paths, in-body) |
|
||
| `dedup_identical_content_is_one_pool_entry` | INV-2 (same-path and cross-path legs) |
|
||
| `stage_hygiene_zero_residue_on_failure_exactly_one_entry_on_success` | INV-3 (both paths, both failure modes) |
|
||
| `sweep_counts_are_exact_across_the_full_protection_set` | INV-4 (happy shape: full protection set) |
|
||
| `sweep_without_liveness_sources_aborts_and_deletes_nothing` | INV-4 (abort shape: `NoLivenessSources`) |
|
||
| `sweep_with_a_failed_protect_callback_aborts_and_deletes_nothing` | INV-4 (abort shape: `ProtectFailed`) |
|
||
| `sweep_is_single_flight_the_contended_second_aborts_as_data` | INV-4 (abort shape: `SweeperLock` lease) |
|
||
| `sweep_racing_a_pinned_put_never_deletes_the_protected_entry` | INV-4 (the sweep-vs-put race) |
|
||
| `list_correctness_is_observable_through_sweep_outcomes` | INV-5 (exact counts, never list comparison) |
|
||
| `stat_reports_len_and_algorithm_without_fetching_content` | INV-6 (the stat-discipline family) |
|
||
| `mem_only_mode_accepts_every_length_on_both_put_paths` | constructor mode (ADR-011 §4) |
|
||
| `kv_only_mode_rejects_over_threshold_puts_at_put_time` | constructor mode (ADR-012 §6.2) |
|
||
|
||
Stamps live on the rows themselves — `grep -rn "Contract stamp:"
|
||
tests/conformance/` is the inventory. The INV-4 rows beyond the
|
||
families' first shapes (the `SweeperLock` abort) complete the three
|
||
`GcAbortCause` variants within the single-node exact-count family
|
||
(ADR-013 §2's lease, its in-process ADR-012 §5 form).
|
||
|
||
### The schedule-relative reading for SQL-backed columns (the mirror convention's amendment)
|
||
|
||
The reference set's single-sweep exact-count rows are the **inline-form
|
||
host's** (mem): there a completed sweep's
|
||
`candidates_staged == deleted + cancelled_by_arbitration` holds over
|
||
its same-sweep staged set. The durable form's semantics differ —
|
||
**staged this sweep, executed by the next** (ADR-012 §6.7) — so
|
||
SQL-backed columns read every exact-count assertion **relative to the
|
||
staged schedule**: the staging sweep reports `candidates_staged` as
|
||
the newly dead set and deletes nothing; its executor (the next sweep)
|
||
reports `deleted + cancelled_by_arbitration` summing exactly to that
|
||
staged set. Recorded, citing ADR-012 §6.7:
|
||
|
||
- **INV-4 and INV-5** run the **durable-schedule equivalents** on
|
||
SQL-backed columns — the two-sweep exact-count rows over the full
|
||
protection set, the sweep-vs-put race across the window delay, and
|
||
list correctness through durable-schedule outcomes (README §Columns
|
||
beyond the reference set grows the rows table). The `SweeperLock`
|
||
abort shape is form-independent (abort before staging) but its
|
||
reference holder-completion leg is inline-form; SQL-backed columns
|
||
run the holder-stages-only form (its staged set waits for the next
|
||
sweep to execute).
|
||
- **INV-2 and INV-3** — whose mirror rows observe their invariants
|
||
through sweep outcomes — likewise assert their counts relative to
|
||
the schedule: zero residue after failures is still staged-0; a
|
||
success's exactly-one-entry is staged-1 then deleted-1 across the
|
||
two sweeps. INV-1, INV-6, and the constructor-mode rows consume no
|
||
entry-dependent sweep counts and mirror verbatim.
|
||
- **Aborts are form-independent**: `NoLivenessSources` and
|
||
`ProtectFailed` abort before staging or executing — counts all zero
|
||
on the durable form too — so those rows mirror directly.
|
||
|
||
## Columns beyond the reference set (INV-7's mirror + extension)
|
||
|
||
The constructor-mode rows of the reference set run wherever the mode
|
||
they name is constructible (README §The reference row set). A
|
||
multi-tier column mirrors the set and grows **its own rows** for what
|
||
its configuration carries that the reference set's single-tier host
|
||
did not: the dispatch-coverage shape (ADR-012 §6.3 — both tiers
|
||
load-bearing) and the engine's tier-specific properties. Those rows
|
||
are still conformance rows — ADR-stamped, named for the property,
|
||
traced — but they take the **column's concrete factory** instead of
|
||
the `StoreFactory` trait: they observe the backing medium, which an
|
||
engine-uniform row (unable to branch on any engine, and unable to see
|
||
any tier's residency through the facade) cannot. Tier placement is
|
||
facade-invisible by design; with `kv = mem`, the medium itself is the
|
||
oracle — the mem kv tier never touches the factory's disk, so a
|
||
committed file outside the staging area under the column's temp root
|
||
is exactly one large-tier entry, and its absence beside a served read
|
||
is exactly a kv-tier entry.
|
||
|
||
**The local column's own rows** (tests/conformance/local_column.rs,
|
||
the dispatch-coverage and local-specific families):
|
||
|
||
| Row | Traces to |
|
||
|---|---|
|
||
| `routing_is_a_pure_function_of_length_across_put_paths_and_re_puts` | ADR-003 §Dispatch policy (purity/no-migration); ADR-012 §6.3; INV-7 facade-contract row |
|
||
| `threshold_boundary_lengths_route_deterministically` | ADR-003 §Dispatch policy (at/below/above the threshold; constructor-tuned re-route); ADR-012 §6.3 |
|
||
| `kv_tier_misses_fall_through_to_the_large_tier_per_op` | ADR-012 §6.1 (fall-through per op); ADR-012 §6.5 ruling 5 (miss shapes); ADR-006 |
|
||
| `stage_hygiene_exact_counts_on_disk_over_both_put_paths` | INV-3's stage form (exact counts on disk); ADR-010 §1; ADR-003 |
|
||
| `pread_windows_read_committed_files_mid_slices_and_eof_clamped` | store-api.md §Read path; ADR-010 §1 (streaming cursor); ADR-006 |
|
||
| `virgin_store_reads_are_no_ops_on_a_fresh_root` | store-api.md §Error model (virgin-store semantics); ADR-012 §6.5 |
|
||
| `sweep_counts_stay_exact_over_both_tiers` | INV-4/INV-5's both-tiers form (exact counts + list correctness through outcomes); ADR-005; ADR-012 §1/§6.3/§6.7 |
|
||
|
||
**The dual-tier threshold is declared configuration, not a
|
||
behavioral probe.** The kv-only reference row discovers its factory's
|
||
threshold by probing (rejections read nothing — free). A dual-tier
|
||
store never rejects a length, and tier residency is facade-invisible,
|
||
so there is nothing to probe: the dispatch rows take the threshold
|
||
the column's factory declares (the same value its constructor took —
|
||
configuration, like the mem column's kv-only threshold). The mem-only
|
||
and kv-only mode rows do not mirror onto the dual-tier factories (the
|
||
precondition `mode()` shapes they assert are not this column's mode);
|
||
the dispatch rows are the dual-tier mode's facade-contract coverage.
|
||
|
||
**The sqlite column's own rows** (tests/conformance/sqlite_column.rs,
|
||
the first SQL-backed column — the default-build production shape,
|
||
`kv = sqlite` + `large = local`, feature `sqlite` default-on,
|
||
ADR-007/013 §4): the dispatch-coverage rows with the **placement
|
||
oracle adapted by medium** — both tiers are on disk now, kv entries
|
||
live in the sqlite file (rows in the `blobs` table), large entries
|
||
under the local root — and the **durable-schedule sweep rows** (§The
|
||
schedule-relative reading for SQL-backed columns; ADR-012 §6.7; the
|
||
window's `gc_window` rows are the schedule observer). The
|
||
form-independent reference rows mirror verbatim: the INV-1 round-trip
|
||
family and the INV-6 stat-discipline row against the dual-tier
|
||
factory, the `NoLivenessSources`/`ProtectFailed` abort shapes
|
||
(form-independent — abort before staging), and the kv-only rejection
|
||
row against a `kv-only(sqlite)` factory (threshold-agnostic by
|
||
discovery, unchanged; the threshold materializable per the
|
||
configuration contract).
|
||
|
||
| Row (`sqlite_column.rs`) | Traces to |
|
||
|---|---|
|
||
| `dedup_identical_content_is_one_pool_entry` | INV-2 (same-path and cross-path legs; counts read against the durable schedule, one row-oracle leg) |
|
||
| `stage_hygiene_zero_residue_on_failure_exactly_one_entry_on_success` | INV-3 (the degenerate kv stage form through the injectable failure modes; schedule-relative counts) |
|
||
| `two_sweep_exact_counts_over_the_full_protection_set` | INV-4/INV-5 (the full protection set across two sweeps — staged this sweep, executed by the next; the both-tiers form) |
|
||
| `a_put_landing_during_the_window_survives_the_executing_sweep` | INV-4 (the sweep-vs-put race across the window delay; execution-time re-arbitration) |
|
||
| `the_contended_second_sweeper_aborts_as_data_and_the_holder_stages_only` | INV-4 (the `SweeperLock` abort shape — abort before staging; the holder's durable-form completion: stages only) |
|
||
| `list_correctness_is_observable_through_durable_schedule_outcomes` | INV-5 (exact counts, never list comparison) |
|
||
| `routing_is_a_pure_function_of_length_across_put_paths_and_re_puts` | ADR-003 §Dispatch policy (purity/no-migration); ADR-012 §6.3; ADR-012 §6.7 (schedule-relative dedup legs); ADR-007; INV-7 facade-contract row |
|
||
| `threshold_boundary_lengths_route_deterministically` | ADR-003 §Dispatch policy (at/below/above, tuned + shipped-default thresholds); ADR-012 §6.3; ADR-007 |
|
||
| `kv_tier_misses_fall_through_to_the_large_tier_per_op` | ADR-012 §6.1 (fall-through per op); ADR-012 §6.5 ruling 5 (miss shapes); ADR-006; ADR-012 §6.3; ADR-007 |
|
||
| `virgin_store_reads_are_no_ops_on_a_fresh_file` | store-api.md §Error model (virgin-store semantics); ADR-012 §6.5; ADR-007 (the no-tables-yet read tolerance — engine-instrumented, the facade's fresh-file form) |
|
||
|
||
The sqlite column's window delay is the **test tuning** (ADR-008 §2's
|
||
legal degenerate tuning, `Duration::ZERO` — execute-on-next-sweep):
|
||
the schedule the durable rows assert against is exactly the
|
||
next-sweep executor form.
|
||
|
||
**The postgres column's own rows** (tests/conformance/postgres_column.rs,
|
||
the first SKIP-posture column — the fleet-capable-kv shape, `kv =
|
||
postgres` + `large = local`, feature `postgres` default-off,
|
||
ADR-007/012 §5): the **durable-schedule sweep rows** mirror the sqlite
|
||
column's family (§The schedule-relative reading for SQL-backed
|
||
columns; ADR-012 §6.7 — the same window delay tuning, `0` —
|
||
execute-on-next-sweep) and the dispatch rows with the placement
|
||
oracle adapted by medium again — kv entries live as rows in the pg
|
||
schema's `blobs` table, large entries under the local root (an
|
||
independent connection counting the schema's rows vs the root's
|
||
committed files; the column's own rows also watch the schedule
|
||
itself, the schema's `gc_window` rows). The form-independent
|
||
reference rows mirror verbatim: the INV-1 round-trip family and the
|
||
INV-6 stat-discipline row against the dual-tier factory, the
|
||
`NoLivenessSources`/`ProtectFailed` abort shapes (form-independent —
|
||
abort before staging), and the kv-only rejection row against a
|
||
`kv-only(postgres)` factory (threshold-agnostic by discovery,
|
||
unchanged; the threshold materializable per the configuration
|
||
contract). Connection settings ride the environment
|
||
(`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`) and the runners gate
|
||
behind a reachability probe; the factories follow the alkstore
|
||
pg-factory pattern (alkstore-postgres `contract_suite.rs`): a fresh
|
||
engine-owned schema per `open()`, tracked per-instance, teardown
|
||
`DROP SCHEMA IF EXISTS … CASCADE` over exactly the minted schemas
|
||
(idempotent, owned, panicking-row-safe — `CASCADE` removes
|
||
dependents server-side while the abandoned store's pooled
|
||
connections fail closed on next use).
|
||
|
||
| Row (`postgres_column.rs`) | Traces to |
|
||
|---|---|
|
||
| `dedup_identical_content_is_one_pool_entry` | INV-2 (same-path and cross-path legs; counts read against the durable schedule) |
|
||
| `stage_hygiene_zero_residue_on_failure_exactly_one_entry_on_success` | INV-3 (both put paths, both failure modes; schedule-relative counts) |
|
||
| `two_sweep_exact_counts_over_the_full_protection_set` | INV-4/INV-5 (the full protection set across two sweeps — staged this sweep, executed by the next; the both-tiers form) |
|
||
| `a_put_landing_during_the_window_survives_the_executing_sweep` | INV-4 (the sweep-vs-put race across the window delay; execution-time re-arbitration) |
|
||
| `the_contended_second_sweeper_aborts_as_data_and_the_holder_stages_only` | INV-4 (the `SweeperLock` abort shape — abort before staging; the holder's durable-form completion: stages only) |
|
||
| `list_correctness_is_observable_through_durable_schedule_outcomes` | INV-5 (exact counts, never list comparison) |
|
||
| `routing_is_a_pure_function_of_length_across_put_paths_and_re_puts` | ADR-003 §Dispatch policy (purity/no-migration); ADR-012 §6.3; ADR-012 §6.7 (schedule-relative dedup legs); ADR-007 §5; INV-7 facade-contract row |
|
||
| `threshold_boundary_lengths_route_deterministically` | ADR-003 §Dispatch policy (at/below/above, tuned + shipped-default thresholds); ADR-012 §6.3; ADR-007 §5 |
|
||
| `kv_tier_misses_fall_through_to_the_large_tier_per_op` | ADR-012 §6.1 (fall-through per op); ADR-012 §6.5 ruling 5 (miss shapes); ADR-006; ADR-012 §6.3; ADR-007 §5 |
|
||
| `virgin_store_reads_are_no_ops_on_a_fresh_schema` | store-api.md §Error model (virgin-store semantics); ADR-012 §6.5; ADR-007 §5 (the constructor always ensures the schema — the facade's fresh-schema form of the tolerance) |
|
||
|
||
**The pg-lo column's own rows**
|
||
(tests/conformance/pg_lo_column.rs, the second SKIP-posture column —
|
||
the **LO-medium column**, ADR-009, landed wave 5): the reference
|
||
inventory's form-independent rows mirror verbatim against a
|
||
dual-tier factory (INV-1's round-trip family over the content
|
||
matrix, INV-6's stat-discipline row — whose non-fetching discipline
|
||
rides ADR-009's companion-row probe on this medium — and the
|
||
`NoLivenessSources`/`ProtectFailed` abort shapes), and the column
|
||
grows the durable-schedule sweep family (§The schedule-relative
|
||
reading for SQL-backed columns; ADR-012 §6.7 — the same window delay
|
||
tuning, `0` — execute-on-next-sweep), the dispatch rows with the
|
||
placement oracle adapted to the LO medium (large entries are
|
||
companion rows + LOs — an independent connection counting
|
||
`lo_entries` and probing `pg_largeobject`; kv entries per the
|
||
factory's kv engine: the mem-pglo factory's kv half is in-process,
|
||
the mem introspection shape, the postgres-pglo factory's kv half
|
||
counts the kv schema's `blobs` rows), the INV-1 extension row (the
|
||
LO medium's ranged reads across the shipped window edge, EOF
|
||
clamping, and the beyond-length clean failure against a planted
|
||
overclaiming companion row), and the virgin-store row. **No
|
||
kv-only rejection row** — no kv-only pg-lo constructor exists (pg-lo
|
||
is a large engine; the postgres column hosts that row for the SQL
|
||
kv engines).
|
||
|
||
The **two factories span feature pairs — the first column whose
|
||
factories need different feature sets** (a recorded design point):
|
||
`DualTierMemPgLoFactory` (*kv = mem* + *large = pg-lo*, the
|
||
one-window-host rule's named test topology — the durable-schedule
|
||
rows run with the window hosted on the **large engine**,
|
||
`gc_window` in the pg-lo schema) gates on
|
||
`all(feature = "mem", feature = "pg-lo")`;
|
||
`DualTierPostgresPgLoFactory` (*kv = postgres* + *large = pg-lo* on
|
||
one pg instance — the consolidation posture) gates on
|
||
`all(feature = "postgres", feature = "pg-lo")` and carries the
|
||
**dormancy row** (the pg-lo schema carries no `gc_window`; the
|
||
window lives on the kv schema — ADR-012 §5); the column's full run
|
||
(acceptance) is `--all-features` against the server. The factories
|
||
follow the alkstore pg-factory pattern with the **LO-isolation
|
||
corollary carried in the engine task's recorded shape** (the C7
|
||
lesson): a fresh dedicated database per `open()` (unique per call,
|
||
tracked per-instance; engine schemas minted inside it), teardown
|
||
reaping the instance's LOs (`SELECT loid FROM lo_entries` →
|
||
`lo_unlink`), dropping exactly the minted schemas
|
||
(`DROP SCHEMA IF EXISTS … CASCADE`), and dropping the database
|
||
(`WITH (FORCE)`) — idempotent, panicking-row-safe, with a
|
||
drop-guard janitor per open backing the abandoned paths — so zero
|
||
LO residue on the shared harness database is structural (the engine
|
||
suite's `shared_harness_db_holds_zero_lo_residue` gate measures it),
|
||
and the structural teardown assertions probe zero orphan residue
|
||
inside the dedicated database before it goes.
|
||
|
||
| Row (`pg_lo_column.rs`) | Traces to |
|
||
|---|---|
|
||
| `dedup_identical_content_is_one_pool_entry` | INV-2 (kv and large legs; counts read against the durable schedule; ADR-009 §2/§4 — the companion table, delete = lo_unlink + row delete) |
|
||
| `stage_hygiene_zero_residue_on_failure_exactly_one_entry_on_success` | INV-3 (the LO tier's real staged form — the held tx's failure paths roll back to no row, no LO pages; schedule-relative counts; ADR-009 §2) |
|
||
| `two_sweep_exact_counts_over_the_full_protection_set` | INV-4/INV-5 (the full protection set across two sweeps, both tiers; the window observed on its host — the pg-lo schema for mem-pglo, the kv schema for postgres-pglo; ADR-012 §5) |
|
||
| `a_put_landing_during_the_window_survives_the_executing_sweep` | INV-4 (the sweep-vs-put race across the window delay; execution-time re-arbitration; ADR-012 §5 — the host rule's shapes) |
|
||
| `the_contended_second_sweeper_aborts_as_data_and_the_holder_stages_only` | INV-4 (the `SweeperLock` abort shape — abort before staging; the holder's durable-form completion: stages only) |
|
||
| `list_correctness_is_observable_through_durable_schedule_outcomes` | INV-5 (exact counts, never list comparison) |
|
||
| `lo_windows_read_across_window_boundaries_and_a_short_body_fails_cleanly` | INV-1's LO-medium extension (ranged reads across the shipped window edge; EOF clamping incl. the `u64::MAX` clamp — the C7 trap not inherited; the beyond-length clean failure); store-api.md §Read path; ADR-009 §2/§3 (the companion table as authority; the `lo_get` windows); ADR-006; ADR-010 §1 |
|
||
| `routing_is_a_pure_function_of_length_across_put_paths_and_re_puts` | ADR-003 §Dispatch policy (purity/no-migration); ADR-009 §2 (the companion table); ADR-012 §5 (the topology); ADR-012 §6.3; ADR-012 §6.7; INV-7 facade-contract row |
|
||
| `threshold_boundary_lengths_route_deterministically` | ADR-003 §Dispatch policy (at/below/above, tuned + shipped-default thresholds); ADR-009 §2; ADR-012 §6.3 |
|
||
| `kv_tier_misses_fall_through_to_the_large_tier_per_op` | ADR-012 §6.1 (fall-through per op); ADR-012 §6.5 ruling 5 (miss shapes); ADR-006; ADR-009 §2/§3 (the `lo_get` windows serve the fall-through); ADR-012 §6.3 |
|
||
| `virgin_store_reads_are_no_ops_on_a_fresh_schema` | store-api.md §Error model (virgin-store semantics); ADR-012 §6.5; ADR-009 §2 (the companion-table bootstrap); ADR-012 §5 (the mem-pglo row asserts the window on the pg-lo host — `gc_window` present-and-empty, `gc_pins` never) |
|
||
| `the_window_lives_on_the_kv_schema_and_the_pg_lo_schema_stays_dormant` (postgres-pglo only) | the dormancy row: ADR-012 §5 (the one-window-host rule's kv-host arm — the pg-lo window clauses dormant, no `gc_window` in the pg-lo schema; the schedule observed on the kv host); ADR-008 §2; ADR-012 §6.7 |
|
||
|
||
## Column plan
|
||
|
||
**mem is the contract-reference engine** (ADR-011 §2): wave 2 lands
|
||
its column first (the first column binds the `Store` facade type), and
|
||
every other engine's rows mirror mem's column — an addition to mem's
|
||
column propagates as a requirement to all other columns (INV-7). The
|
||
pg columns (postgres, pg-lo) additionally run the SKIP posture above —
|
||
the **postgres column is the posture's first landed host** (server-less
|
||
runs skip cleanly; its wave-4 acceptance is the green run against the
|
||
harness server) and the **pg-lo column is the second** (landed wave 5;
|
||
its two factories gate on the `mem`+`pg-lo` and `postgres`+`pg-lo`
|
||
feature pairs respectively — the first column whose factories need
|
||
different feature sets — with the column's full run `--all-features`
|
||
against the harness server; the fleet-form rows of INV-4 are
|
||
engine-gated to the postgres column and first run in wave 6).
|
||
|
||
## Current state
|
||
|
||
The harness is green on **five columns** (waves 2–5). The **mem
|
||
column** is the contract-reference column: the factory binding is the
|
||
concrete `alkblobs::Store` facade (see §The factory contract — the
|
||
binding demonstrator remains in `binding.rs`), the reference row set
|
||
is landed (§The reference row set — `rows.rs`, every row ADR-stamped
|
||
and traced to the backlog), the column's factories (`mem-only`, the
|
||
reference; `kv-only(mem)` at a column-declared threshold) vend fresh
|
||
isolated stores with idempotent teardown (`mem_column.rs`, the
|
||
structural factory-contract tests included), and mem never skips.
|
||
|
||
The **local column** (the suite's first dual-tier column, the
|
||
dispatch-coverage shape ADR-012 §6.3 names) mirrors the reference INV
|
||
rows against its `kv = mem` + `large = local` factory at a
|
||
column-declared tuned threshold — the shared matrix and race fixtures
|
||
cross it, so both tiers' bodies are exercised — and grows the
|
||
dispatch-coverage rows (pure-function routing with an on-disk
|
||
placement oracle, fall-through per op, threshold boundaries at both
|
||
the tuned and the shipped-default threshold) plus the local-specific
|
||
rows (on-disk stage hygiene exact counts, pread windows, virgin-store
|
||
no-ops, the both-tiers sweep shape) — §Columns beyond the reference
|
||
set is the recorded inventory. Its factory instances each track and
|
||
remove their own temp roots (idempotent teardown, panicking-row-safe;
|
||
each runner owns its instance so concurrent rows never share roots).
|
||
mem-only/kv-only modes and the local column never skip, but the local
|
||
column is feature-gated on `mem`: the `dual_tier_mem_local`
|
||
constructor rides feature `mem` because mem is the only constructible
|
||
kv engine until wave 3 (the gate is the kv pairing's, not the local
|
||
engine's, which is ungated in `src/`) — the column runs under `cargo
|
||
test --features mem`, with the reference row set compiling ungated so
|
||
it stays type-checked in default builds.
|
||
|
||
The **sqlite column** (the suite's first SQL-backed column — the
|
||
default-build production shape, `kv = sqlite` + `large = local`,
|
||
ADR-007/013 §4) mirrors the form-independent reference rows (INV-1,
|
||
INV-6, the `NoLivenessSources`/`ProtectFailed` abort shapes, and the
|
||
kv-only rejection row against `kv-only(sqlite)`) and grows the
|
||
**durable-schedule sweep rows** (§The schedule-relative reading for
|
||
SQL-backed columns — the two-sweep exact counts, the race across the
|
||
window delay, the holder-stages-only lease shape, list correctness
|
||
through durable-schedule outcomes, and the schedule-relative INV-2/3
|
||
readings) plus the dispatch rows with the placement oracle adapted by
|
||
medium (both tiers on disk: the sqlite file's rows vs the root's
|
||
committed files) and the virgin-store row — the recorded inventory is
|
||
in §Columns beyond the reference set. Its factories declare the
|
||
column-tuned threshold and the test tuning of the window delay
|
||
(`Duration::ZERO` — execute-on-next-sweep), vend a fresh temp site
|
||
per open (a private directory holding the sqlite file and the local
|
||
root; idempotent teardown removing the site, panicking-row-safe, each
|
||
runner owning its instance), and never skip. The column is
|
||
feature-gated on `sqlite` — **default-on**, so the column runs in the
|
||
default build: every `cargo test` run exercises it. The sqlite
|
||
engine's no-tables-yet read tolerance is engine-instrumented
|
||
(src/sqlite.rs's tests; the constructor-vs-lazy timing design point
|
||
there), the column's virgin-store row being its facade form.
|
||
|
||
The **postgres column** (the suite's **first SKIP-posture column** —
|
||
the fleet-capable-kv shape, `kv = postgres` + `large = local`, feature
|
||
`postgres` **default-off**, ADR-007/013 §4, landed wave 4) mirrors the
|
||
form-independent reference rows (INV-1, INV-6, the
|
||
`NoLivenessSources`/`ProtectFailed` abort shapes, and the kv-only
|
||
rejection row against `kv-only(postgres)`) and grows the sqlite
|
||
column's durable-schedule sweep rows plus the dispatch rows with the
|
||
placement oracle adapted by medium (an independent connection counts
|
||
the pg schema's `blobs` rows vs the root's committed files; the
|
||
column's own rows watch the schedule's `gc_window` rows) and the
|
||
virgin-store row — the recorded inventory is in §Columns beyond the
|
||
reference set. Its factories declare the column-tuned threshold and
|
||
the test tuning of the window delay (`Duration::ZERO` —
|
||
execute-on-next-sweep) and follow the alkstore pg-factory pattern: a
|
||
fresh engine-owned schema per `open()` (unique per call, seeded per
|
||
tag/pid/time — racing opens included), tracked per-instance in the
|
||
schema registry, teardown `DROP SCHEMA IF EXISTS … CASCADE` over
|
||
exactly the minted schemas plus removal of every vended temp root
|
||
(idempotent, owned, panicking-row-safe; each runner owns its
|
||
instance). The column **skips cleanly server-less** under the
|
||
reachability probe (`ALKBLOBS_PG_HOST` gating; the remaining
|
||
settings default to the harness server's values) — a skip is not a
|
||
failure — and its acceptance is the green run **against the harness
|
||
server** (dockerized `pglo-poc`, `postgres:16-alpine`, :15432, db
|
||
`blobs`, verified live 2026-10-12; the alkstore harness's same
|
||
server).
|
||
|
||
The **pg-lo column** (the suite's **second SKIP-posture column** —
|
||
the LO-medium column, feature `pg-lo` **default-off**, ADR-009, its
|
||
two factories gating on the `mem`+`pg-lo` and `postgres`+`pg-lo`
|
||
feature pairs — the first column whose factories need different
|
||
feature sets, landed wave 5) mirrors the form-independent reference
|
||
rows (INV-1's round-trip family, INV-6's stat-discipline row riding
|
||
ADR-009's companion-row probe, and the `NoLivenessSources`/
|
||
`ProtectFailed` abort shapes) and grows the durable-schedule sweep
|
||
rows (run on **both factories** — the window observed on its host:
|
||
the pg-lo schema for the mem-pglo factory per the one-window-host
|
||
rule's named test topology, the kv schema for the postgres-pglo
|
||
consolidation shape, which carries the **dormancy row**), the
|
||
dispatch rows with the placement oracle adapted to the LO medium
|
||
(companion rows + `pg_largeobject` probing for the large tier; the
|
||
mem kv half in-process, the postgres kv half's `blobs` rows), the
|
||
INV-1 extension row (the LO medium's ranged reads across the
|
||
shipped window edge and the beyond-length clean failure), and the
|
||
virgin-store rows — the recorded inventory is in §Columns beyond
|
||
the reference set. **No kv-only rejection row mirrors here** (no
|
||
kv-only pg-lo constructor exists; recorded). Connection settings
|
||
ride the environment (`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB` —
|
||
`ALKBLOBS_PG_HOST` gates; the rest default to the harness server's
|
||
values) and the runners gate behind the reachability probe — a skip
|
||
is never a failure — and its acceptance is the green run
|
||
**against the harness server under `--all-features`** (dockerized
|
||
`pglo-poc`, verified live 2026-10-12). Its factories declare the
|
||
column-tuned threshold and the test tuning of the window delay
|
||
(`0` — execute-on-next-sweep) and follow the alkstore pg-factory
|
||
pattern with the **LO-isolation corollary carried in the engine
|
||
task's recorded shape** (the C7 lesson): a fresh dedicated database
|
||
per `open()` (unique per call, engine schemas minted inside it,
|
||
tracked per-instance; drop-guard janitors backing the abandoned
|
||
paths), teardown reaping the instance's LOs (`SELECT loid FROM
|
||
lo_entries` → `lo_unlink`), dropping exactly the minted schemas,
|
||
and dropping the database (`WITH (FORCE)`) — idempotent,
|
||
panicking-row-safe — so zero LO residue on the shared database is
|
||
structural (the engine suite's
|
||
`shared_harness_db_holds_zero_lo_residue` gate measures it) and the
|
||
structural teardown assertions probe zero orphan residue inside the
|
||
dedicated database before it goes.
|