alkblobs/tests/conformance/README.md

The conformance suite — home and conventions

The in-crate conformance harness for alkblobs' engines. Decided point (docs/plans/implementation.md, 2026-10-11): engines are features of one crate, so the suite lives in-crate, factory- parameterized over engine constructors — the alkstore ADR-017/022 contract-suite pattern adapted (there a sibling suite crate over Box<dyn Store>; here one test target over factories whose open vends the future Store facade, whose concrete type wave 2 binds).

This document is the convention's normative home on the test side (the in-code module docs in main.rs/factory.rs/stamp.rs restate their slices). Waves 2–6 grow columns against it: one column per engine, each column the same rows run against that engine's factory.

Row anatomy

A row is one named, independently-runnable property — an async function taking a StoreFactory (not a store), asserting by panicking on violation (a failing row is a test failure naming the property and expectation; the family-wide no-panic rule is relaxed here by the suite's test-side purpose).

Row naming

• snake_case, and the name states the property, not the engine (e.g. round_trip_known_length, dedup_is_one_pool_entry). The engine dimension is the column (which factory the row runs against), never the row name — that is what makes rows engine-uniform.
• Dimensions a property spans within one run — the content matrix, both put paths, both tiers where the invariant ranges over them — loop inside the row's body. A genuinely different property (e.g. sweep safety's fleet form vs its single-node form) is a separate row, not a flag inside the owner.
• Planned row families are registered in the invariant backlog (table below); a row that does not trace to a backlog entry is not a conformance row.

Contract stamps (the ADR-017 §4.2 convention adapted)

Every row is version-stamped with a doc-comment list item beginning Contract stamp:, citing the spec § and/or ADR § that pins the behavior tested:

/// <one-line statement of the property>
///
/// Contract stamp: store-api.md §Invariants 2; ADR-003 (whole-value
/// put is CAS first-writer-wins); ADR-005 (re-put refcounts).
pub async fn dedup_is_one_pool_entry<F: StoreFactory>(factory: &F) { ... }

Rules:

• The stamp cites the ADR § / spec § that pins the property — never a bare "wave 2" or a date. Several stamps are allowed when one row tests several changes' behavior together.
• A row added to pin a new contract addition is stamped with that addition's ADR; a row whose property predates the suite is stamped with the ADR/spec section that pinned it.
• A later change that amends a row's property appends to its stamp list; rows accumulate stamps, the ADRs' own texts record the history.
• Contract stamp: is greppable — grep -rn "Contract stamp:" tests/conformance/ is the stamp inventory.
• Rows are also free of engine-specific code by construction: a row may not branch on factory.name().

One normative owner per property

Each property has exactly one normative owner: the suite row. Engine-specific unit tests inside src/ may duplicate coverage for debugging convenience, but the suite row is the normative assertion — an engine's conformance claim rests on its column being green, not on its in-crate tests. When the suite itself needs per-engine divergence (the fleet-form sweep safety against postgres, INV-4), that is its own row with its own owner stamp, still one per property.

The factory contract

factory.rs pins the [StoreFactory] shape. The contract:

• open() vends a fresh, isolated store instance. No state is shared between rows or between columns: a fresh backing store per call — a fresh temp directory (local), a fresh pool (mem), a fresh schema or file (sqlite/postgres/pg-lo) — so a property's rows never observe another property's entries. Isolation is the implementer's owned guarantee.
• teardown() is idempotent and safe to run against an abandoned store: a failing assertion panics the row (early exit, no teardown on that path), so teardown must also make sense over a dropped / half-open instance. The factory owns what teardown means — deleting the temp dir, dropping the schema, closing the pool — and owns the isolated-ness guarantee above.
• Factories are Send + Sync: shareable across row runs.
• The wave-2 binding — made and recorded. The Store facade type landed with wave 2's store-construction task as a concrete type (alkblobs::Store, engine-blind; src/store.rs holds the engine- representation design point). The binding made the anticipated shape correction: the skeleton's Store associated type collapsed into the concrete facade — open() vends alkblobs::Store directly (factory.rs records the ruling; the wave-1 mock probe retired with it, replaced by the mem-gated binding demo factories in main.rs). A new engine column binds by implementing the trait; there is no longer a vendible type to choose.
• The row body owns its lifecycle. A row calls open() first — a fresh, isolated store per call — drives its assertions, and calls teardown() before returning; a failing assertion panics the row before teardown runs (the early exit above), which is why teardown() must make sense over the abandoned instance — dropped or still alive at unwind. The mem column's structural tests (factory_teardown_is_safe_against_a_panicking_row) exercise that path; isolation itself (fresh state per open, nothing shared) is exercised alongside (factory_open_vends_fresh_isolated_stores).

The SKIP posture

SQL-attached factories (postgres, pg-lo) ride a server (dockerized postgres) whose connection settings come from the environment. When no server is reachable, the column skips cleanly: a skip is not a failure, and the harness reports it as skipped rather than green or red. Two sides to keep honest:

• Engine waves' acceptance criteria require green against the server (the alkstore harness convention) — a skip never substitutes for acceptance.
• Server-less engines (mem, sqlite, local, and any future dependency-free engine) never skip: their columns must always be green in every environment.

The invariant backlog (rows-to-grow)

The seven invariants of store-api.md §Invariants, registered as the table every wave's rows grow against. Traceability: each row family names its store-api.md invariant and the ADRs pinning it. A suite row asserting something outside this table (or outside a wave's specced additions) needs a backlog entry and its stamp first.

#	Invariant (store-api.md §Invariants)	Row family	Pinning text	First column
INV-1	Whole-blob put/get round-trips byte-identical under the canonical derivation — both tiers, both put paths, every shipped engine	one round-trip family; content matrix and put paths loop in-body; per-engine columns	store-api.md §Invariants 1; ADR-002 (derivation); ADR-003 (put paths, dispatch); ADR-007/008/009/011 (engines)	wave 2 — mem (kv tier), then local (large tier)
INV-2	Dedup: identical content twice (same or different path) is one pool entry	one dedup row, same-path and different-path legs	store-api.md §Invariants 2; ADR-003 (CAS first-writer-wins); ADR-005 (re-put refcounts)	wave 2 — mem
INV-3	Stage hygiene: failure mid-put leaves zero stage files / no entry; success leaves exactly one committed entry	one hygiene family over both put paths and both failure modes (discard, failed commit)	store-api.md §Invariants 3; ADR-010 §1 (staged-put contract); ADR-003	wave 2 — mem (degenerate stage) and local
INV-4	Sweep safety: exact counts with correct liveness; aborting sources delete nothing. Engine-gated fleet form (DB-backed pins + sweep lease vs postgres)	one single-node family; the fleet form is its own engine-gated row family (runs under --all-features)	store-api.md §Invariants 4; ADR-005 (windows, sweeps); ADR-008 (fleet form); ADR-012 §1/§4/§6.7; ADR-013 §2	wave 2 — mem (in-process protocol); fleet form waits for wave 6
INV-5	list() correctness is observable only through GC	list-correctness asserted through sweep outcomes (exact counts), never by comparing lists directly	store-api.md §Invariants 5; ADR-003 (complete + well-formed list()); ADR-005	wave 2 — mem
INV-6	stat/size returns entry length without fetching content	one stat-discipline family per engine column	store-api.md §Invariants 6; ADR-008 (size probe); ADR-009 (pg-lo companion-row probe)	wave 2 — mem
INV-7	Engine-conformance uniformity: every engine's bodies pass the same contract suite mem passes	the meta-gate — a column's green run is the row; no separate test	store-api.md §Invariants 7; ADR-010 §1; ADR-011 §2 (mem's reference role)	wave 2 — mem defines it

The reference row set (INV-7's basis)

The mem column's rows (rows.rs) are the reference row set — the required set every other engine's rows mirror (ADR-011 §2, mem's contract-reference role; store-api.md §Invariants 7). A column mirrors this inventory row-for-row against its own factory; an addition to the set propagates to all columns (INV-7). The INV-row families are the engine mirror targets — the round-trip family covers the tier the column's engine serves (mem: the kv tier; local: the large tier). The two constructor-mode rows mirror wherever that configuration is constructible: a column with a kv-only factory runs the kv-only row against it — the row discovers the mode's threshold by probing, since known-length rejections read nothing, and requires the threshold materializable for its at-threshold legs (≤ 8 MiB, MAX_BOUNDARY_MATERIALIZATION).

Row (rows.rs)	Traces to
round_trip_puts_and_gets_are_byte_identical	INV-1 (content matrix, both put paths, in-body)
dedup_identical_content_is_one_pool_entry	INV-2 (same-path and cross-path legs)
stage_hygiene_zero_residue_on_failure_exactly_one_entry_on_success	INV-3 (both paths, both failure modes)
sweep_counts_are_exact_across_the_full_protection_set	INV-4 (happy shape: full protection set)
sweep_without_liveness_sources_aborts_and_deletes_nothing	INV-4 (abort shape: NoLivenessSources)
sweep_with_a_failed_protect_callback_aborts_and_deletes_nothing	INV-4 (abort shape: ProtectFailed)
sweep_is_single_flight_the_contended_second_aborts_as_data	INV-4 (abort shape: SweeperLock lease)
sweep_racing_a_pinned_put_never_deletes_the_protected_entry	INV-4 (the sweep-vs-put race)
list_correctness_is_observable_through_sweep_outcomes	INV-5 (exact counts, never list comparison)
stat_reports_len_and_algorithm_without_fetching_content	INV-6 (the stat-discipline family)
mem_only_mode_accepts_every_length_on_both_put_paths	constructor mode (ADR-011 §4)
kv_only_mode_rejects_over_threshold_puts_at_put_time	constructor mode (ADR-012 §6.2)

Stamps live on the rows themselves — grep -rn "Contract stamp:" tests/conformance/ is the inventory. The INV-4 rows beyond the families' first shapes (the SweeperLock abort) complete the three GcAbortCause variants within the single-node exact-count family (ADR-013 §2's lease, its in-process ADR-012 §5 form).

Column plan

mem is the contract-reference engine (ADR-011 §2): wave 2 lands its column first (the first column binds the Store facade type), and every other engine's rows mirror mem's column — an addition to mem's column propagates as a requirement to all other columns (INV-7). The pg columns (postgres, pg-lo) additionally run the SKIP posture above; the fleet-form rows of INV-4 are engine-gated to the postgres column and first run in wave 6.

Current state

The harness is green on the mem column (wave 2): the factory binding is the concrete alkblobs::Store facade (see §The factory contract — the binding demonstrator remains in binding.rs), the reference row set is landed (§The reference row set — rows.rs, every row ADR-stamped and traced to the backlog), the column's factories (mem-only, the reference; kv-only(mem) at a column-declared threshold) vend fresh isolated stores with idempotent teardown (mem_column.rs, the structural factory-contract tests included), and mem never skips. The column runs under feature mem (cargo test --features mem); the row set compiles ungated so the reference rows stay type-checked in default builds. The local column grows next.