diff --git a/tasks/suite-scaffold.md b/tasks/suite-scaffold.md index cd89be2..1c525f8 100644 --- a/tasks/suite-scaffold.md +++ b/tasks/suite-scaffold.md @@ -1,7 +1,7 @@ --- id: suite-scaffold name: Conformance suite — home, convention, and invariant backlog -status: pending +status: completed depends_on: [scaffold-crate-features] scope: narrow risk: low @@ -66,8 +66,37 @@ identically here). ## Notes -> To be filled by implementation agent +- The convention doc landed as `tests/conformance/README.md` (normative + home) with the in-code anchors: `stamp.rs` (the `Contract stamp:` + marker, alkstore ADR-017 §4.2 adapted), `factory.rs` (the + `StoreFactory` shape + contract), `backlog.rs` (the backlog as an + in-code registry with a traceability gate). +- The skeleton's wave-2 binding point: `StoreFactory::Store` is an + associated type placeholder for the `Store` facade type (which + arrives in wave 2 — no public type exists to bind today). Wave 2's + mem column binds it; a binding-revealed shape correction is wave + 2's to make and record in README + factory.rs, per the task text. +- SKIP posture is pinned as convention text (README §SKIP posture + + factory.rs docs) rather than typed machinery: the mechanism lands + with the first column that needs it (postgres, wave 4). Mem/sqlite/ + local never skip. +- The backlog table cites per-entry: restated invariant, planned row + family, pinning texts (spec §/ADR §), first column/wave. INV-4's + fleet form is recorded as its own engine-gated family (wave 6). +- Harness self-checks (4 tests): backlog traceability (order mirrors + store-api.md §Invariants 1–7, every field populated), stamp marker + identity, and a probe factory proving the trait is implementable + with async methods (the alkstore exemplar-against-mock posture, + sized to the no-rows state). No engine-specific code; no + dependencies added. ## Summary -> To be filled on completion \ No newline at end of file +`tests/conformance/` established: README.md (row naming, stamp +format, one-owner rule, factory isolation + idempotent-teardown +contract, SKIP posture, the seven-invariant backlog table traceable +to store-api.md, mem-first column plan), plus the compile-able +skeleton (factory trait, stamp module, backlog registry) with zero +rows as expected. Verified: `cargo test` (33 total across lib + 3 +targets), `cargo clippy --all-targets -- -D warnings`, `cargo fmt +--check` all green. \ No newline at end of file diff --git a/tests/conformance/README.md b/tests/conformance/README.md new file mode 100644 index 0000000..a04f7b8 --- /dev/null +++ b/tests/conformance/README.md @@ -0,0 +1,152 @@ +# The conformance suite — home and conventions + +The in-crate conformance harness for alkblobs' engines. **Decided +point** (docs/plans/implementation.md, 2026-10-11): engines are +features of one crate, so the suite lives in-crate, factory- +parameterized over engine constructors — the alkstore +ADR-017/022 contract-suite pattern adapted (there a sibling suite +crate over `Box`; here one test target over factories +whose `open` vends the future `Store` facade, whose concrete type +wave 2 binds). + +This document is the convention's normative home on the test side +(the in-code module docs in `main.rs`/`factory.rs`/`stamp.rs` restate +their slices). Waves 2–6 grow **columns** against it: one column per +engine, each column the same rows run against that engine's factory. + +## Row anatomy + +A row is one named, independently-runnable property — an async +function taking a `StoreFactory` (not a store), asserting by panicking +on violation (a failing row is a test failure naming the property and +expectation; the family-wide no-panic rule is relaxed here by the +suite's test-side purpose). + +### Row naming + +- `snake_case`, and the name **states the property, not the engine** + (e.g. `round_trip_known_length`, `dedup_is_one_pool_entry`). The + engine dimension is the *column* (which factory the row runs + against), never the row name — that is what makes rows + engine-uniform. +- Dimensions a property spans *within* one run — the content matrix, + both put paths, both tiers where the invariant ranges over them — + loop inside the row's body. A genuinely different property (e.g. + sweep safety's fleet form vs its single-node form) is a separate + row, not a flag inside the owner. +- Planned row families are registered in the **invariant backlog** + (table below); a row that does not trace to a backlog entry is not + a conformance row. + +### Contract stamps (the ADR-017 §4.2 convention adapted) + +Every row is version-stamped with a doc-comment list item beginning +**`Contract stamp:`**, citing the spec § and/or ADR § that pins the +behavior tested: + +```text +/// +/// +/// Contract stamp: store-api.md §Invariants 2; ADR-003 (whole-value +/// put is CAS first-writer-wins); ADR-005 (re-put refcounts). +pub async fn dedup_is_one_pool_entry(factory: &F) { ... } +``` + +Rules: + +- The stamp cites the ADR § / spec § that pins the property — never a + bare "wave 2" or a date. Several stamps are allowed when one row + tests several changes' behavior together. +- A row added to pin a new contract addition is stamped with that + addition's ADR; a row whose property predates the suite is stamped + with the ADR/spec section that pinned it. +- A later change that *amends* a row's property appends to its stamp + list; rows accumulate stamps, the ADRs' own texts record the + history. +- `Contract stamp:` is greppable — `grep -rn "Contract stamp:" + tests/conformance/` is the stamp inventory. +- Rows are also free of engine-specific code by construction: a row + may not branch on `factory.name()`. + +### One normative owner per property + +Each property has **exactly one normative owner**: the suite row. +Engine-specific unit tests inside `src/` may duplicate coverage for +debugging convenience, but the suite row is the normative assertion — +an engine's conformance claim rests on its column being green, not on +its in-crate tests. When the suite itself needs per-engine divergence +(the fleet-form sweep safety against postgres, INV-4), that is its own +row with its own owner stamp, still one per property. + +## The factory contract + +`factory.rs` pins the [`StoreFactory`] shape. The contract: + +- **`open()` vends a fresh, isolated store instance.** No state is + shared between rows or between columns: a fresh backing store per + call — a fresh temp directory (`local`), a fresh pool (`mem`), a + fresh schema or file (`sqlite`/`postgres`/`pg-lo`) — so a property's + rows never observe another property's entries. Isolation is the + implementer's owned guarantee. +- **`teardown()` is idempotent** and safe to run against an abandoned + store: a failing assertion panics the row (early exit, no teardown + on that path), so teardown must also make sense over a dropped / + half-open instance. The factory owns what teardown means — deleting + the temp dir, dropping the schema, closing the pool — and owns the + isolated-ness guarantee above. +- Factories are `Send + Sync`: shareable across row runs. +- The wave-2 binding note: the trait's `Store` associated type is the + stub's placeholder for the `Store` facade type (wave 2). Wave 2's + first column binds it; if binding reveals a shape correction (e.g. + the associated type collapses to the concrete facade type), that is + wave 2's to make and record in this README and `factory.rs`. + +## The SKIP posture + +SQL-attached factories (`postgres`, `pg-lo`) ride a server (dockerized +postgres) whose connection settings come from the environment. When no +server is reachable, the column **skips cleanly**: a skip is not a +failure, and the harness reports it as skipped rather than green or +red. Two sides to keep honest: + +- Engine waves' acceptance criteria require green **against the + server** (the alkstore harness convention) — a skip never substitutes + for acceptance. +- Server-less engines (`mem`, `sqlite`, `local`, and any future + dependency-free engine) never skip: their columns must always be + green in every environment. + +## The invariant backlog (rows-to-grow) + +The seven invariants of store-api.md §Invariants, registered as the +table every wave's rows grow against. Traceability: each row family +names its store-api.md invariant and the ADRs pinning it. A suite row +asserting something outside this table (or outside a wave's specced +additions) needs a backlog entry *and* its stamp first. + +| # | Invariant (store-api.md §Invariants) | Row family | Pinning text | First column | +|---|---|---|---|---| +| INV-1 | Whole-blob put/get round-trips byte-identical under the canonical derivation — both tiers, both put paths, every shipped engine | one round-trip family; content matrix and put paths loop in-body; per-engine columns | store-api.md §Invariants 1; ADR-002 (derivation); ADR-003 (put paths, dispatch); ADR-007/008/009/011 (engines) | wave 2 — mem (kv tier), then local (large tier) | +| INV-2 | Dedup: identical content twice (same or different path) is one pool entry | one dedup row, same-path and different-path legs | store-api.md §Invariants 2; ADR-003 (CAS first-writer-wins); ADR-005 (re-put refcounts) | wave 2 — mem | +| INV-3 | Stage hygiene: failure mid-put leaves zero stage files / no entry; success leaves exactly one committed entry | one hygiene family over both put paths and both failure modes (discard, failed commit) | store-api.md §Invariants 3; ADR-010 §1 (staged-put contract); ADR-003 | wave 2 — mem (degenerate stage) and local | +| INV-4 | Sweep safety: exact counts with correct liveness; aborting sources delete nothing. Engine-gated fleet form (DB-backed pins + sweep lease vs postgres) | one single-node family; the fleet form is its own engine-gated row family (runs under `--all-features`) | store-api.md §Invariants 4; ADR-005 (windows, sweeps); ADR-008 (fleet form); ADR-012 §1/§4/§6.7; ADR-013 §2 | wave 2 — mem (in-process protocol); fleet form waits for wave 6 | +| INV-5 | `list()` correctness is observable only through GC | list-correctness asserted **through sweep outcomes** (exact counts), never by comparing lists directly | store-api.md §Invariants 5; ADR-003 (complete + well-formed `list()`); ADR-005 | wave 2 — mem | +| INV-6 | `stat`/`size` returns entry length without fetching content | one stat-discipline family per engine column | store-api.md §Invariants 6; ADR-008 (`size` probe); ADR-009 (pg-lo companion-row probe) | wave 2 — mem | +| INV-7 | Engine-conformance uniformity: every engine's bodies pass the same contract suite mem passes | the meta-gate — a column's green run *is* the row; no separate test | store-api.md §Invariants 7; ADR-010 §1; ADR-011 §2 (mem's reference role) | wave 2 — mem defines it | + +## Column plan + +**mem is the contract-reference engine** (ADR-011 §2): wave 2 lands +its column first (the first column binds the `Store` facade type), and +every other engine's rows mirror mem's column — an addition to mem's +column propagates as a requirement to all other columns (INV-7). The +pg columns (postgres, pg-lo) additionally run the SKIP posture above; +the fleet-form rows of INV-4 are engine-gated to the postgres column +and first run in wave 6. + +## Current state + +The harness skeleton is compile-able with **zero rows** (this task's +scoped deliverable): the trait shape, the stamp convention, and the +backlog registry are in place; the rows arrive with wave 2's mem +column. \ No newline at end of file diff --git a/tests/conformance/backlog.rs b/tests/conformance/backlog.rs new file mode 100644 index 0000000..46fb7d0 --- /dev/null +++ b/tests/conformance/backlog.rs @@ -0,0 +1,157 @@ +//! The invariant backlog — store-api.md §Invariants' seven entries +//! registered as the rows-to-grow table (README §The invariant +//! backlog is the normative statement; this module is its in-code +//! registry, which waves 2–6 grow rows against). +//! +//! Each entry traces to the store-api.md invariant numbering (INV-1 +//! through INV-7 mirror §Invariants 1–7) and carries the pinning text +//! the row stamps will cite. A suite row that does not trace to an +//! entry here is not a conformance row; a new property needs a backlog +//! entry and its stamp first. +//! +//! The column plan: mem is the contract-reference engine (ADR-011 §2) +//! — wave 2 lands its column first, every other engine's rows mirror +//! it, and additions to mem's column propagate to all others (INV-7). + +#![allow(dead_code)] + +/// One backlog entry: an invariant of store-api.md §Invariants, its +/// planned row family, the pinning texts its row stamps cite, and the +/// first column to land it. +pub struct Invariant { + /// `INV-` — mirrors the n-th item of store-api.md §Invariants. + pub id: &'static str, + /// One-line restatement of the invariant as specced. + pub invariant: &'static str, + /// The planned row family — what wave-grown rows assert (README + /// table's "Row family" column). + pub row_family: &'static str, + /// The pinning texts the rows' stamps cite (spec §/ADR §; the + /// README table's "Pinning text" column, split by citation). + pub pins: &'static [&'static str], + /// The first column and wave to land the family (README table's + /// last column). + pub first_column: &'static str, +} + +/// The backlog, in store-api.md §Invariants order. +pub const BACKLOG: [Invariant; 7] = [ + Invariant { + id: "INV-1", + invariant: "whole-blob put/get round-trips byte-identical under the canonical \ + derivation — both tiers, both put paths, every shipped engine", + row_family: "one round-trip family over the content matrix (put paths and tiers \ + loop in-body; the git-interop test remains the derivation's source \ + of truth)", + pins: &[ + "store-api.md §Invariants 1", + "ADR-002 (canonical derivation)", + "ADR-003 (both put paths, dispatch)", + "ADR-007/008/009/011 (the shipped engines)", + ], + first_column: "wave 2 — mem (kv tier), then local (large tier)", + }, + Invariant { + id: "INV-2", + invariant: "dedup: putting identical content twice (same or different path) is \ + one pool entry", + row_family: "one dedup row, same-path and different-path legs", + pins: &[ + "store-api.md §Invariants 2", + "ADR-003 (whole-value put is CAS first-writer-wins)", + "ADR-005 (re-put refcounts the existing entry)", + ], + first_column: "wave 2 — mem", + }, + Invariant { + id: "INV-3", + invariant: "stage hygiene: any failure mid-put leaves zero stage files; any \ + success leaves exactly one committed entry", + row_family: "one exact-count hygiene family over both put paths and both failure \ + modes (explicit discard, failed commit)", + pins: &[ + "store-api.md §Invariants 3", + "ADR-010 §1 (the staged-put contract: commit publishes, every failure path \ + discards)", + "ADR-003 (stage-then-commit-rename)", + ], + first_column: "wave 2 — mem (degenerate kv stage) and local (the stage form)", + }, + Invariant { + id: "INV-4", + invariant: "sweep safety: with correct liveness registered sweep counts are \ + exact; with aborting sources sweep deletes nothing", + row_family: "one single-node exact-count family (happy + abort shapes); the fleet \ + form is its own engine-gated row family running under \ + `--all-features`", + pins: &[ + "store-api.md §Invariants 4", + "ADR-005 (delete windows, abort-as-data)", + "ADR-008 (the engine-gated fleet form)", + "ADR-012 §1/§4/§6.7 (sweep shape, GC-state host, executor-of-window ruling)", + "ADR-013 §2 (the sweep lease's semantics)", + ], + first_column: "wave 2 — mem (in-process protocol); the fleet form first runs in \ + wave 6", + }, + Invariant { + id: "INV-5", + invariant: "`list()` correctness is observable only through GC", + row_family: "list-correctness rows assert through sweep outcomes (exact counts), \ + never by comparing enumerated lists directly", + pins: &[ + "store-api.md §Invariants 5", + "ADR-003 (`list()` complete and well-formed by contract)", + "ADR-005 (GC's live-set diff rides `list()`)", + ], + first_column: "wave 2 — mem", + }, + Invariant { + id: "INV-6", + invariant: "`stat`/`size` on any engine returns the entry length without fetching \ + content", + row_family: "one stat-discipline family per engine column", + pins: &[ + "store-api.md §Invariants 6", + "ADR-008 (the trait `size` probe discipline)", + "ADR-009 (pg-lo's companion-row probe)", + ], + first_column: "wave 2 — mem", + }, + Invariant { + id: "INV-7", + invariant: "engine-conformance uniformity: every engine's bodies pass the same \ + contract suite mem (the reference engine) passes", + row_family: "the meta-gate — a column's green run against mem's rows is the \ + assertion; no separate uniformity test exists", + pins: &[ + "store-api.md §Invariants 7", + "ADR-010 §1 (one trait, every engine, same suite)", + "ADR-011 §2 (mem's contract-reference role)", + ], + first_column: "wave 2 — mem (its column defines the required set)", + }, +]; + +/// The backlog's traceability gate: entries mirror store-api.md +/// §Invariants' numbering in order, and every entry's fields are +/// populated. The harness's tests call this; a violation is a +/// registry-editing mistake, not a runtime event. +pub fn assert_traceable() { + for (position, entry) in BACKLOG.iter().enumerate() { + assert_eq!( + entry.id, + format!("INV-{}", position + 1), + "backlog order mirrors store-api.md §Invariants" + ); + assert!(!entry.invariant.is_empty()); + assert!(!entry.row_family.is_empty()); + assert!(!entry.pins.is_empty(), "{} must pin its behavior", entry.id); + assert!(!entry.first_column.is_empty()); + } +} + +/// Look a backlog entry up by its `INV-n` id. +pub fn invariant_by_id(id: &str) -> Option<&'static Invariant> { + BACKLOG.iter().find(|entry| entry.id == id) +} diff --git a/tests/conformance/factory.rs b/tests/conformance/factory.rs new file mode 100644 index 0000000..0913371 --- /dev/null +++ b/tests/conformance/factory.rs @@ -0,0 +1,66 @@ +//! The suite's factory contract (the alkstore `StoreFactory` pattern +//! adapted — docs/plans/implementation.md, decided point "Contract- +//! suite home"). +//! +//! One shape, deliberately minimal: open a **fresh, isolated** store +//! instance per call (no state shared between rows or between +//! columns), with **idempotent teardown** — the full contract text is +//! the suite README §The factory contract. Engines implement this in +//! wave 2+'s column wiring; rows take factories, never stores. +//! +//! **The wave-2 binding.** The `Store` facade type arrives in wave 2 +//! (store-api.md §What this is). This trait's `Store` associated type +//! is the skeleton's binding point for it: wave 2's first column (mem) +//! binds it to the real facade, and if binding reveals a shape +//! correction — most plausibly the associated type collapsing to the +//! concrete facade type across all factories — that is wave 2's to +//! make and record here and in the README. +//! +//! The SKIP posture is a factory-owner responsibility (README §SKIP +//! posture): SQL-attached factories probe their server before rows +//! run; a unreachable server skips the column cleanly, while the +//! engine wave's acceptance requires green against the server. + +#![allow(dead_code)] + +use std::future::Future; + +use alkblobs::Error; + +/// Vends fresh, isolated store instances to the suite's rows. +/// +/// Implementers — the engine columns' wiring — yield an instance of +/// the `Store` facade over an isolated backing store (fresh temp +/// directory, pool, file, or schema per call, so a property's rows +/// never outlive their run into another's) plus a teardown disposing +/// of the backing store. Implementations must be `Send + Sync` so one +/// factory instance can be shared across row runs. +/// +/// Rows call [`StoreFactory::open`], drive their assertions, and call +/// [`StoreFactory::teardown`] before returning; a failing assertion +/// panics the row (no teardown on that path), which is why teardown is +/// required to be idempotent and safe against an abandoned instance. +pub trait StoreFactory: Send + Sync { + /// The store instance type this factory vends — the wave-2 `Store` + /// facade's binding point (see the module docs). + type Store: Send; + + /// The column's identity for reporting — which configuration rows + /// ran against (the constructor-mode vocabulary of ADR-011 §4, + /// once wave 2 names it). Rows never branch on it; the suite's + /// reporting and the SKIP posture's per-column bookkeeping do. + fn name(&self) -> &'static str; + + /// Vend a fresh, isolated store instance. A failure here is an + /// environment failure (e.g. the SKIP posture's unreachable + /// server), not a row failure. + fn open(&self) -> impl Future> + Send; + + /// Tear down the backing store behind this factory instance. + /// + /// Idempotent per instance. The factory owns what teardown means — + /// deleting the temp directory, dropping the schema, closing the + /// pool — and owns the isolated-ness guarantee: no two `open` + /// calls ever share state. + fn teardown(&self) -> impl Future> + Send; +} diff --git a/tests/conformance/main.rs b/tests/conformance/main.rs new file mode 100644 index 0000000..7fa002e --- /dev/null +++ b/tests/conformance/main.rs @@ -0,0 +1,71 @@ +//! `main.rs` — the conformance test target's root. +//! +//! The suite is in-crate by decided point (docs/plans/implementation.md): +//! engines are features of one crate, so the harness is factory- +//! parameterized over engine constructors rather than a sibling suite +//! crate. This target currently carries the **skeleton only** — the +//! factory trait shape, the stamp convention, and the invariant backlog +//! registry (see [`README`](./README.md) for the conventions' normative +//! statement). Rows arrive with wave 2's first column (mem, the +//! contract-reference engine); if binding the `Store` facade type +//! reveals a shape correction, that is wave 2's to make and record. +//! +//! Items beyond the tests below are the wave-2 drivers' surface, hence +//! the scoped `#![allow(dead_code)]` — it shrinks as rows land. + +#![allow(dead_code)] + +mod backlog; +mod factory; +mod stamp; + +use factory::StoreFactory; + +/// The harness probe factory: not an engine — a minimal mock proving +/// the [`StoreFactory`] contract is implementable with async methods +/// and reports the identity `name()` carries (the alkstore harness's +/// "exemplar against a trivial in-crate mock" posture, sized down to +/// the skeleton's no-rows state). +struct Probe { + label: &'static str, +} + +impl StoreFactory for Probe { + type Store = (); + + fn name(&self) -> &'static str { + self.label + } + + async fn open(&self) -> Result { + Ok(()) + } + + async fn teardown(&self) -> Result<(), alkblobs::Error> { + Ok(()) + } +} + +#[test] +fn probe_factory_implements_the_contract() { + let factory = Probe { label: "probe" }; + assert_eq!(factory.name(), "probe", "factory identity is reportable"); +} + +#[tokio::test] +async fn probe_factory_open_teardown_lifecycle() { + let factory = Probe { label: "probe" }; + factory.open().await.expect("open vends the instance"); + factory.teardown().await.expect("teardown is idempotent"); + factory.teardown().await.expect("teardown repeats cleanly"); +} + +#[test] +fn stamp_marker_matches_convention() { + assert_eq!(stamp::STAMP_MARKER, "Contract stamp:"); +} + +#[test] +fn backlog_registers_the_seven_invariants() { + backlog::assert_traceable(); +} diff --git a/tests/conformance/stamp.rs b/tests/conformance/stamp.rs new file mode 100644 index 0000000..0f16b0a --- /dev/null +++ b/tests/conformance/stamp.rs @@ -0,0 +1,24 @@ +//! The contract-stamp convention (the alkstore ADR-017 §4.2 version- +//! stamping convention adapted — README §Contract stamps is the +//! normative statement). +//! +//! Every conformance row is version-stamped with a doc-comment list +//! item beginning **`Contract stamp:`** citing the store-api.md § / +//! ADR § that pins the behavior tested. The marker is greppable: +//! +//! ```text +//! grep -rn "Contract stamp:" tests/conformance/ +//! ``` +//! +//! is the suite's stamp inventory. Rules (README §Contract stamps): +//! cite the pinning spec §/ADR §, never a bare wave or date; stamps +//! accumulate on amendment; multi-stamp rows are allowed. +//! +//! Version 0 of the suite: this module *is* the convention's in-code +//! anchor; the first real rows (wave 2's mem column) cite their stamps +//! against it. + +#![allow(dead_code)] + +/// The doc-comment marker introducing a row's version stamp. +pub const STAMP_MARKER: &str = "Contract stamp:";