The hash registry extends per ADR-018: bytes 0x03/0x04 (git-object sha-1/sha-256), preimage-identity hashing over git's loose-object form as the stored preimage — the digest byte-identical to git's oid, every kind included. Re-derived from poc-o-git-objects, ADR-stamped, not merged. - Registry + Algorithm members after the blob members (total order rides the registry bytes, 0x02 < 0x03 < 0x04); the derivation arms (derive's len parameter inert for the family — documented, test-pinned); the BlobHasher no-preamble arms (finish pinned equal to derive); from_git_oid_hex's contract pinned blob-family-only. - keys.rs: the extended physical layout round-trips in total order; rejects-never-guesses unchanged (0x05 is the unknown-tag example). - put path: put_git_object(algorithm, len, stream) public — family members only (KeyInvalid otherwise), known-length only, the inherited Verification/Io mappings — via put_known_as; the unknown-length drivers take their derivation from callers. - fleet threading (ADR-018 §7): every hasher site derivation-parameterized (put_known_as; fleet_put_pglo_known/ _unknown); put_git_object under a fleet declaration lands through the joint entry+pin tx / pg-lo consolidation / pin-first staged forms — pinned by two env-driven tests with exact-count sweep outcomes (kv joint form byte-exact + dedup row-alone; consolidation form with LO pages landing and dying exactly). - git_interop: the family tests green against the real git CLI (tree/commit/tag byte-exact, dedup, the blob-tag negative, the sha-256 sibling), the seam guard (blob algorithms reject), the preimage-length stat assertion; in-module empty-tree known-answer vectors. - Conformance: the INV-1/2/6 reference rows gain family legs (round-trip incl. the staged form, dedup-by-oid, the stored-preimage stat semantic); backlog pins + README inventory updated. - kind-decision delta (ADR-018 §2): behavior unchanged, texts honest — the ops module doc's digest-wire-form paragraph, digest_ingress's doc, the specs' headers. - fuzz: the key_encoding generator + shared battery extended to the four-member registry (family keys, known-answer vectors, the len inertness pin, 0x05 as the invalid tag); committed corpus regenerated per policy (0x03 was an invalid-tag seed — 117 rewritten, 67 added); idempotent after landing. Verification: cargo test (default, 153+54+7+2), cargo test --all-features server-less (skip-clean) and against the live pglo-poc harness (336+125+7+3), cargo clippy --all-targets both feature variants (-D warnings), cargo fmt --check, cargo doc --no-deps, Cargo test --manifest-path fuzz/shared/Cargo.toml (the corpus-replay gate). Task: tasks/c1-git-object-family-impl.md (status completed, notes + summary recorded).
The conformance suite — home and conventions
The in-crate conformance harness for alkblobs' engines. Decided
point (docs/plans/implementation.md, 2026-10-11): engines are
features of one crate, so the suite lives in-crate, factory-
parameterized over engine constructors — the alkstore
ADR-017/022 contract-suite pattern adapted (there a sibling suite
crate over Box<dyn Store>; here one test target over factories
whose open vends the future Store facade, whose concrete type
wave 2 binds).
This document is the convention's normative home on the test side
(the in-code module docs in main.rs/factory.rs/stamp.rs restate
their slices). Waves 2-6 grow columns against it: one column per
engine, each column the same rows run against that engine's factory.
Row anatomy
A row is one named, independently-runnable property — an async
function taking a StoreFactory (not a store), asserting by panicking
on violation (a failing row is a test failure naming the property and
expectation; the family-wide no-panic rule is relaxed here by the
suite's test-side purpose).
Row naming
snake_case, and the name states the property, not the engine (e.g.round_trip_known_length,dedup_is_one_pool_entry). The engine dimension is the column (which factory the row runs against), never the row name — that is what makes rows engine-uniform.- Dimensions a property spans within one run — the content matrix, both put paths, both tiers where the invariant ranges over them — loop inside the row's body. A genuinely different property (e.g. sweep safety's fleet form vs its single-node form) is a separate row, not a flag inside the owner.
- Planned row families are registered in the invariant backlog (table below); a row that does not trace to a backlog entry is not a conformance row.
Contract stamps (the ADR-017 §4.2 convention adapted)
Every row is version-stamped with a doc-comment list item beginning
Contract stamp:, citing the spec § and/or ADR § that pins the
behavior tested:
/// <one-line statement of the property>
///
/// Contract stamp: store-api.md §Invariants 2; ADR-003 (whole-value
/// put is CAS first-writer-wins); ADR-005 (re-put refcounts).
pub async fn dedup_is_one_pool_entry<F: StoreFactory>(factory: &F) { ... }
Rules:
- The stamp cites the ADR § / spec § that pins the property — never a bare "wave 2" or a date. Several stamps are allowed when one row tests several changes' behavior together.
- A row added to pin a new contract addition is stamped with that addition's ADR; a row whose property predates the suite is stamped with the ADR/spec section that pinned it.
- A later change that amends a row's property appends to its stamp list; rows accumulate stamps, the ADRs' own texts record the history.
Contract stamp:is greppable —grep -rn "Contract stamp:" tests/conformance/is the stamp inventory.- Rows are also free of engine-specific code by construction: a row
may not branch on
factory.name().
One normative owner per property
Each property has exactly one normative owner: the suite row.
Engine-specific unit tests inside src/ may duplicate coverage for
debugging convenience, but the suite row is the normative assertion —
an engine's conformance claim rests on its column being green, not on
its in-crate tests. When the suite itself needs per-engine divergence
(the fleet-form sweep safety against postgres, INV-4), that is its own
row with its own owner stamp, still one per property.
The factory contract
factory.rs pins the [StoreFactory] shape. The contract:
open()vends a fresh, isolated store instance. No state is shared between rows or between columns: a fresh backing store per call — a fresh temp directory (local), a fresh pool (mem), a fresh schema or file (sqlite/postgres/pg-lo) — so a property's rows never observe another property's entries. Isolation is the implementer's owned guarantee.teardown()is idempotent and safe to run against an abandoned store: a failing assertion panics the row (early exit, no teardown on that path), so teardown must also make sense over a dropped / half-open instance. The factory owns what teardown means — deleting the temp dir, dropping the schema, closing the pool — and owns the isolated-ness guarantee above.- Factories are
Send + Sync: shareable across row runs. - The wave-2 binding — made and recorded. The
Storefacade type landed with wave 2's store-construction task as a concrete type (alkblobs::Store, engine-blind; src/store.rs holds the engine- representation design point). The binding made the anticipated shape correction: the skeleton'sStoreassociated type collapsed into the concrete facade —open()vendsalkblobs::Storedirectly (factory.rs records the ruling; the wave-1 mock probe retired with it, replaced by the mem-gated binding demo factories inmain.rs). A new engine column binds by implementing the trait; there is no longer a vendible type to choose. - The row body owns its lifecycle. A row calls
open()first — a fresh, isolated store per call — drives its assertions, and callsteardown()before returning; a failing assertion panics the row before teardown runs (the early exit above), which is whyteardown()must make sense over the abandoned instance — dropped or still alive at unwind. The mem column's structural tests (factory_teardown_is_safe_against_a_panicking_row) exercise that path; isolation itself (fresh state per open, nothing shared) is exercised alongside (factory_open_vends_fresh_isolated_stores).
The SKIP posture
SQL-attached factories (postgres, pg-lo) ride a server (dockerized
postgres) whose connection settings come from the environment. When no
server is reachable, the column skips cleanly: a skip is not a
failure, and the harness reports it as skipped rather than green or
red. Two sides to keep honest:
- Engine waves' acceptance criteria require green against the server (the alkstore harness convention) — a skip never substitutes for acceptance.
- Server-less engines (
mem,sqlite,local, and any future dependency-free engine) never skip: their columns must always be green in every environment.
The invariant backlog (rows-to-grow)
The seven invariants of store-api.md §Invariants, registered as the table every wave's rows grow against. Traceability: each row family names its store-api.md invariant and the ADRs pinning it. A suite row asserting something outside this table (or outside a wave's specced additions) needs a backlog entry and its stamp first.
| # | Invariant (store-api.md §Invariants) | Row family | Pinning text | First column |
|---|---|---|---|---|
| INV-1 | Whole-blob put/get round-trips byte-identical under the canonical derivation — both tiers, both put paths, every shipped engine; the git-object family's put_git_object puts ride the same round-trip under the preimage-identity keys (ADR-018) |
one round-trip family; content matrix and put paths loop in-body; the family's legs loop both sibling algorithms, sub-threshold and staged forms; per-engine columns | store-api.md §Invariants 1; ADR-002 (derivation); ADR-003 (put paths, dispatch); ADR-007/008/009/011 (engines); ADR-018 (the family seam) | wave 2 — mem (kv tier), then local (large tier) |
| INV-2 | Dedup: identical content twice (same or different path) is one pool entry; the family's dedup-by-oid rides the same outcome discipline (ADR-018) | one dedup row, same-path and different-path legs (plus the family's re-put leg) | store-api.md §Invariants 2; ADR-003 (CAS first-writer-wins); ADR-005 (re-put refcounts); ADR-018 (dedup by oid) | wave 2 — mem |
| INV-3 | Stage hygiene: failure mid-put leaves zero stage files / no entry; success leaves exactly one committed entry | one hygiene family over both put paths and both failure modes (discard, failed commit) | store-api.md §Invariants 3; ADR-010 §1 (staged-put contract); ADR-003 | wave 2 — mem (degenerate stage) and local |
| INV-4 | Sweep safety: exact counts with correct liveness; aborting sources delete nothing. Engine-gated fleet form (DB-backed pins + sweep lease vs postgres) | one single-node family; the fleet form is its own engine-gated row family (runs under --all-features) |
store-api.md §Invariants 4; ADR-005 (windows, sweeps); ADR-008 (fleet form); ADR-012 §1/§4/§6.7; ADR-013 §2 | wave 2 — mem (in-process protocol); fleet form waits for wave 6 |
| INV-5 | list() correctness is observable only through GC |
list-correctness asserted through sweep outcomes (exact counts), never by comparing lists directly | store-api.md §Invariants 5; ADR-003 (complete + well-formed list()); ADR-005 |
wave 2 — mem |
| INV-6 | stat/size returns entry length without fetching content |
one stat-discipline family per engine column | store-api.md §Invariants 6; ADR-008 (size probe); ADR-009 (pg-lo companion-row probe) |
wave 2 — mem |
| INV-7 | Engine-conformance uniformity: every engine's bodies pass the same contract suite mem passes | the meta-gate — a column's green run is the row; no separate test | store-api.md §Invariants 7; ADR-010 §1; ADR-011 §2 (mem's reference role) | wave 2 — mem defines it |
The reference row set (INV-7's basis)
The mem column's rows (rows.rs) are the reference row set — the
required set every other engine's rows mirror (ADR-011 §2, mem's
contract-reference role; store-api.md §Invariants 7). A column mirrors
this inventory row-for-row against its own factory; an addition to the
set propagates to all columns (INV-7). The INV-row families are the
engine mirror targets — the round-trip family covers the tier the
column's engine serves (mem: the kv tier; local: the large tier). The
two constructor-mode rows mirror wherever that configuration is
constructible: a column with a kv-only factory runs the kv-only row
against it — the row discovers the mode's threshold by probing, since
known-length rejections read nothing, and requires the threshold
materializable for its at-threshold legs (≤ 8 MiB,
MAX_BOUNDARY_MATERIALIZATION).
Row (rows.rs) |
Traces to |
|---|---|
round_trip_puts_and_gets_are_byte_identical |
INV-1 (content matrix, both put paths, in-body) |
dedup_identical_content_is_one_pool_entry |
INV-2 (same-path and cross-path legs) |
stage_hygiene_zero_residue_on_failure_exactly_one_entry_on_success |
INV-3 (both paths, both failure modes) |
sweep_counts_are_exact_across_the_full_protection_set |
INV-4 (happy shape: full protection set) |
sweep_without_liveness_sources_aborts_and_deletes_nothing |
INV-4 (abort shape: NoLivenessSources) |
sweep_with_a_failed_protect_callback_aborts_and_deletes_nothing |
INV-4 (abort shape: ProtectFailed) |
sweep_is_single_flight_the_contended_second_aborts_as_data |
INV-4 (abort shape: SweeperLock lease) |
sweep_racing_a_pinned_put_never_deletes_the_protected_entry |
INV-4 (the sweep-vs-put race) |
list_correctness_is_observable_through_sweep_outcomes |
INV-5 (exact counts, never list comparison) |
stat_reports_len_and_algorithm_without_fetching_content |
INV-6 (the stat-discipline family; the family entries' legs carry ADR-018 §Decision 3's stored-preimage semantic) |
mem_only_mode_accepts_every_length_on_both_put_paths |
constructor mode (ADR-011 §4) |
kv_only_mode_rejects_over_threshold_puts_at_put_time |
constructor mode (ADR-012 §6.2) |
Stamps live on the rows themselves — grep -rn "Contract stamp:" tests/conformance/ is the inventory. The INV-4 rows beyond the
families' first shapes (the SweeperLock abort) complete the three
GcAbortCause variants within the single-node exact-count family
(ADR-013 §2's lease, its in-process ADR-012 §5 form).
The schedule-relative reading for SQL-backed columns (the mirror convention's amendment)
The reference set's single-sweep exact-count rows are the inline-form
host's (mem): there a completed sweep's
candidates_staged == deleted + cancelled_by_arbitration holds over
its same-sweep staged set. The durable form's semantics differ —
staged this sweep, executed by the next (ADR-012 §6.7) — so
SQL-backed columns read every exact-count assertion relative to the
staged schedule: the staging sweep reports candidates_staged as
the newly dead set and deletes nothing; its executor (the next sweep)
reports deleted + cancelled_by_arbitration summing exactly to that
staged set. Recorded, citing ADR-012 §6.7:
- INV-4 and INV-5 run the durable-schedule equivalents on
SQL-backed columns — the two-sweep exact-count rows over the full
protection set, the sweep-vs-put race across the window delay, and
list correctness through durable-schedule outcomes (README §Columns
beyond the reference set grows the rows table). The
SweeperLockabort shape is form-independent (abort before staging) but its reference holder-completion leg is inline-form; SQL-backed columns run the holder-stages-only form (its staged set waits for the next sweep to execute). - INV-2 and INV-3 — whose mirror rows observe their invariants through sweep outcomes — likewise assert their counts relative to the schedule: zero residue after failures is still staged-0; a success's exactly-one-entry is staged-1 then deleted-1 across the two sweeps. INV-1, INV-6, and the constructor-mode rows consume no entry-dependent sweep counts and mirror verbatim.
- Aborts are form-independent:
NoLivenessSourcesandProtectFailedabort before staging or executing — counts all zero on the durable form too — so those rows mirror directly.
Columns beyond the reference set (INV-7's mirror + extension)
The constructor-mode rows of the reference set run wherever the mode
they name is constructible (README §The reference row set). A
multi-tier column mirrors the set and grows its own rows for what
its configuration carries that the reference set's single-tier host
did not: the dispatch-coverage shape (ADR-012 §6.3 — both tiers
load-bearing) and the engine's tier-specific properties. Those rows
are still conformance rows — ADR-stamped, named for the property,
traced — but they take the column's concrete factory instead of
the StoreFactory trait: they observe the backing medium, which an
engine-uniform row (unable to branch on any engine, and unable to see
any tier's residency through the facade) cannot. Tier placement is
facade-invisible by design; with kv = mem, the medium itself is the
oracle — the mem kv tier never touches the factory's disk, so a
committed file outside the staging area under the column's temp root
is exactly one large-tier entry, and its absence beside a served read
is exactly a kv-tier entry.
The local column's own rows (tests/conformance/local_column.rs, the dispatch-coverage and local-specific families):
| Row | Traces to |
|---|---|
routing_is_a_pure_function_of_length_across_put_paths_and_re_puts |
ADR-003 §Dispatch policy (purity/no-migration); ADR-012 §6.3; INV-7 facade-contract row |
threshold_boundary_lengths_route_deterministically |
ADR-003 §Dispatch policy (at/below/above the threshold; constructor-tuned re-route); ADR-012 §6.3 |
kv_tier_misses_fall_through_to_the_large_tier_per_op |
ADR-012 §6.1 (fall-through per op); ADR-012 §6.5 ruling 5 (miss shapes); ADR-006 |
stage_hygiene_exact_counts_on_disk_over_both_put_paths |
INV-3's stage form (exact counts on disk); ADR-010 §1; ADR-003 |
pread_windows_read_committed_files_mid_slices_and_eof_clamped |
store-api.md §Read path; ADR-010 §1 (streaming cursor); ADR-006 |
virgin_store_reads_are_no_ops_on_a_fresh_root |
store-api.md §Error model (virgin-store semantics); ADR-012 §6.5 |
sweep_counts_stay_exact_over_both_tiers |
INV-4/INV-5's both-tiers form (exact counts + list correctness through outcomes); ADR-005; ADR-012 §1/§6.3/§6.7 |
The dual-tier threshold is declared configuration, not a
behavioral probe. The kv-only reference row discovers its factory's
threshold by probing (rejections read nothing — free). A dual-tier
store never rejects a length, and tier residency is facade-invisible,
so there is nothing to probe: the dispatch rows take the threshold
the column's factory declares (the same value its constructor took —
configuration, like the mem column's kv-only threshold). The mem-only
and kv-only mode rows do not mirror onto the dual-tier factories (the
precondition mode() shapes they assert are not this column's mode);
the dispatch rows are the dual-tier mode's facade-contract coverage.
The sqlite column's own rows (tests/conformance/sqlite_column.rs,
the first SQL-backed column — the default-build production shape,
kv = sqlite + large = local, feature sqlite default-on,
ADR-007/013 §4): the dispatch-coverage rows with the placement
oracle adapted by medium — both tiers are on disk now, kv entries
live in the sqlite file (rows in the blobs table), large entries
under the local root — and the durable-schedule sweep rows (§The
schedule-relative reading for SQL-backed columns; ADR-012 §6.7; the
window's gc_window rows are the schedule observer). The
form-independent reference rows mirror verbatim: the INV-1 round-trip
family and the INV-6 stat-discipline row against the dual-tier
factory, the NoLivenessSources/ProtectFailed abort shapes
(form-independent — abort before staging), and the kv-only rejection
row against a kv-only(sqlite) factory (threshold-agnostic by
discovery, unchanged; the threshold materializable per the
configuration contract).
Row (sqlite_column.rs) |
Traces to |
|---|---|
dedup_identical_content_is_one_pool_entry |
INV-2 (same-path and cross-path legs; counts read against the durable schedule, one row-oracle leg) |
stage_hygiene_zero_residue_on_failure_exactly_one_entry_on_success |
INV-3 (the degenerate kv stage form through the injectable failure modes; schedule-relative counts) |
two_sweep_exact_counts_over_the_full_protection_set |
INV-4/INV-5 (the full protection set across two sweeps — staged this sweep, executed by the next; the both-tiers form) |
a_put_landing_during_the_window_survives_the_executing_sweep |
INV-4 (the sweep-vs-put race across the window delay; execution-time re-arbitration) |
the_contended_second_sweeper_aborts_as_data_and_the_holder_stages_only |
INV-4 (the SweeperLock abort shape — abort before staging; the holder's durable-form completion: stages only) |
list_correctness_is_observable_through_durable_schedule_outcomes |
INV-5 (exact counts, never list comparison) |
routing_is_a_pure_function_of_length_across_put_paths_and_re_puts |
ADR-003 §Dispatch policy (purity/no-migration); ADR-012 §6.3; ADR-012 §6.7 (schedule-relative dedup legs); ADR-007; INV-7 facade-contract row |
threshold_boundary_lengths_route_deterministically |
ADR-003 §Dispatch policy (at/below/above, tuned + shipped-default thresholds); ADR-012 §6.3; ADR-007 |
kv_tier_misses_fall_through_to_the_large_tier_per_op |
ADR-012 §6.1 (fall-through per op); ADR-012 §6.5 ruling 5 (miss shapes); ADR-006; ADR-012 §6.3; ADR-007 |
virgin_store_reads_are_no_ops_on_a_fresh_file |
store-api.md §Error model (virgin-store semantics); ADR-012 §6.5; ADR-007 (the no-tables-yet read tolerance — engine-instrumented, the facade's fresh-file form) |
The sqlite column's window delay is the test tuning (ADR-008 §2's
legal degenerate tuning, Duration::ZERO — execute-on-next-sweep):
the schedule the durable rows assert against is exactly the
next-sweep executor form.
The postgres column's own rows (tests/conformance/postgres_column.rs,
the first SKIP-posture column — the fleet-capable-kv shape, kv = postgres + large = local, feature postgres default-off,
ADR-007/012 §5): the durable-schedule sweep rows mirror the sqlite
column's family (§The schedule-relative reading for SQL-backed
columns; ADR-012 §6.7 — the same window delay tuning, 0 —
execute-on-next-sweep) and the dispatch rows with the placement
oracle adapted by medium again — kv entries live as rows in the pg
schema's blobs table, large entries under the local root (an
independent connection counting the schema's rows vs the root's
committed files; the column's own rows also watch the schedule
itself, the schema's gc_window rows). The form-independent
reference rows mirror verbatim: the INV-1 round-trip family and the
INV-6 stat-discipline row against the dual-tier factory, the
NoLivenessSources/ProtectFailed abort shapes (form-independent —
abort before staging), and the kv-only rejection row against a
kv-only(postgres) factory (threshold-agnostic by discovery,
unchanged; the threshold materializable per the configuration
contract). Connection settings ride the environment
(ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB) and the runners gate
behind a reachability probe; the factories follow the alkstore
pg-factory pattern (alkstore-postgres contract_suite.rs): a fresh
engine-owned schema per open(), tracked per-instance, teardown
DROP SCHEMA IF EXISTS … CASCADE over exactly the minted schemas
(idempotent, owned, panicking-row-safe — CASCADE removes
dependents server-side while the abandoned store's pooled
connections fail closed on next use).
Row (postgres_column.rs) |
Traces to |
|---|---|
dedup_identical_content_is_one_pool_entry |
INV-2 (same-path and cross-path legs; counts read against the durable schedule) |
stage_hygiene_zero_residue_on_failure_exactly_one_entry_on_success |
INV-3 (both put paths, both failure modes; schedule-relative counts) |
two_sweep_exact_counts_over_the_full_protection_set |
INV-4/INV-5 (the full protection set across two sweeps — staged this sweep, executed by the next; the both-tiers form) |
a_put_landing_during_the_window_survives_the_executing_sweep |
INV-4 (the sweep-vs-put race across the window delay; execution-time re-arbitration) |
the_contended_second_sweeper_aborts_as_data_and_the_holder_stages_only |
INV-4 (the SweeperLock abort shape — abort before staging; the holder's durable-form completion: stages only) |
list_correctness_is_observable_through_durable_schedule_outcomes |
INV-5 (exact counts, never list comparison) |
routing_is_a_pure_function_of_length_across_put_paths_and_re_puts |
ADR-003 §Dispatch policy (purity/no-migration); ADR-012 §6.3; ADR-012 §6.7 (schedule-relative dedup legs); ADR-007 §5; INV-7 facade-contract row |
threshold_boundary_lengths_route_deterministically |
ADR-003 §Dispatch policy (at/below/above, tuned + shipped-default thresholds); ADR-012 §6.3; ADR-007 §5 |
kv_tier_misses_fall_through_to_the_large_tier_per_op |
ADR-012 §6.1 (fall-through per op); ADR-012 §6.5 ruling 5 (miss shapes); ADR-006; ADR-012 §6.3; ADR-007 §5 |
virgin_store_reads_are_no_ops_on_a_fresh_schema |
store-api.md §Error model (virgin-store semantics); ADR-012 §6.5; ADR-007 §5 (the constructor always ensures the schema — the facade's fresh-schema form of the tolerance) |
The pg-lo column's own rows
(tests/conformance/pg_lo_column.rs, the second SKIP-posture column —
the LO-medium column, ADR-009, landed wave 5): the reference
inventory's form-independent rows mirror verbatim against a
dual-tier factory (INV-1's round-trip family over the content
matrix, INV-6's stat-discipline row — whose non-fetching discipline
rides ADR-009's companion-row probe on this medium — and the
NoLivenessSources/ProtectFailed abort shapes), and the column
grows the durable-schedule sweep family (§The schedule-relative
reading for SQL-backed columns; ADR-012 §6.7 — the same window delay
tuning, 0 — execute-on-next-sweep), the dispatch rows with the
placement oracle adapted to the LO medium (large entries are
companion rows + LOs — an independent connection counting
lo_entries and probing pg_largeobject; kv entries per the
factory's kv engine: the mem-pglo factory's kv half is in-process,
the mem introspection shape, the postgres-pglo factory's kv half
counts the kv schema's blobs rows), the INV-1 extension row (the
LO medium's ranged reads across the shipped window edge, EOF
clamping, and the beyond-length clean failure against a planted
overclaiming companion row), and the virgin-store row. No
kv-only rejection row — no kv-only pg-lo constructor exists (pg-lo
is a large engine; the postgres column hosts that row for the SQL
kv engines).
The two factories span feature pairs — the first column whose
factories need different feature sets (a recorded design point):
DualTierMemPgLoFactory (kv = mem + large = pg-lo, the
one-window-host rule's named test topology — the durable-schedule
rows run with the window hosted on the large engine,
gc_window in the pg-lo schema) gates on
all(feature = "mem", feature = "pg-lo");
DualTierPostgresPgLoFactory (kv = postgres + large = pg-lo on
one pg instance — the consolidation posture) gates on
all(feature = "postgres", feature = "pg-lo") and carries the
dormancy row (the pg-lo schema carries no gc_window; the
window lives on the kv schema — ADR-012 §5); the column's full run
(acceptance) is --all-features against the server. The factories
follow the alkstore pg-factory pattern with the LO-isolation
corollary carried in the engine task's recorded shape (the C7
lesson): a fresh dedicated database per open() (unique per call,
tracked per-instance; engine schemas minted inside it), teardown
reaping the instance's LOs (SELECT loid FROM lo_entries →
lo_unlink), dropping exactly the minted schemas
(DROP SCHEMA IF EXISTS … CASCADE), and dropping the database
(WITH (FORCE)) — idempotent, panicking-row-safe, with a
drop-guard janitor per open backing the abandoned paths — so zero
LO residue on the shared harness database is structural (the engine
suite's shared_harness_db_holds_zero_lo_residue gate measures it),
and the structural teardown assertions probe zero orphan residue
inside the dedicated database before it goes.
Row (pg_lo_column.rs) |
Traces to |
|---|---|
dedup_identical_content_is_one_pool_entry |
INV-2 (kv and large legs; counts read against the durable schedule; ADR-009 §2/§4 — the companion table, delete = lo_unlink + row delete) |
stage_hygiene_zero_residue_on_failure_exactly_one_entry_on_success |
INV-3 (the LO tier's real staged form — the held tx's failure paths roll back to no row, no LO pages; schedule-relative counts; ADR-009 §2) |
two_sweep_exact_counts_over_the_full_protection_set |
INV-4/INV-5 (the full protection set across two sweeps, both tiers; the window observed on its host — the pg-lo schema for mem-pglo, the kv schema for postgres-pglo; ADR-012 §5) |
a_put_landing_during_the_window_survives_the_executing_sweep |
INV-4 (the sweep-vs-put race across the window delay; execution-time re-arbitration; ADR-012 §5 — the host rule's shapes) |
the_contended_second_sweeper_aborts_as_data_and_the_holder_stages_only |
INV-4 (the SweeperLock abort shape — abort before staging; the holder's durable-form completion: stages only) |
list_correctness_is_observable_through_durable_schedule_outcomes |
INV-5 (exact counts, never list comparison) |
lo_windows_read_across_window_boundaries_and_a_short_body_fails_cleanly |
INV-1's LO-medium extension (ranged reads across the shipped window edge; EOF clamping incl. the u64::MAX clamp — the C7 trap not inherited; the beyond-length clean failure); store-api.md §Read path; ADR-009 §2/§3 (the companion table as authority; the lo_get windows); ADR-006; ADR-010 §1 |
routing_is_a_pure_function_of_length_across_put_paths_and_re_puts |
ADR-003 §Dispatch policy (purity/no-migration); ADR-009 §2 (the companion table); ADR-012 §5 (the topology); ADR-012 §6.3; ADR-012 §6.7; INV-7 facade-contract row |
threshold_boundary_lengths_route_deterministically |
ADR-003 §Dispatch policy (at/below/above, tuned + shipped-default thresholds); ADR-009 §2; ADR-012 §6.3 |
kv_tier_misses_fall_through_to_the_large_tier_per_op |
ADR-012 §6.1 (fall-through per op); ADR-012 §6.5 ruling 5 (miss shapes); ADR-006; ADR-009 §2/§3 (the lo_get windows serve the fall-through); ADR-012 §6.3 |
virgin_store_reads_are_no_ops_on_a_fresh_schema |
store-api.md §Error model (virgin-store semantics); ADR-012 §6.5; ADR-009 §2 (the companion-table bootstrap); ADR-012 §5 (the mem-pglo row asserts the window on the pg-lo host — gc_window present-and-empty, gc_pins never) |
the_window_lives_on_the_kv_schema_and_the_pg_lo_schema_stays_dormant (postgres-pglo only) |
the dormancy row: ADR-012 §5 (the one-window-host rule's kv-host arm — the pg-lo window clauses dormant, no gc_window in the pg-lo schema; the schedule observed on the kv host); ADR-008 §2; ADR-012 §6.7 |
The fleet column's own rows
(tests/conformance/fleet_column.rs, the suite's third SKIP-posture
column — the fleet column, the exact-count fleet sweep outcomes
under concurrency, wave 6's gate, landed wave 6): the column runs
under --features postgres (fleet mode requires kv = postgres — the
composite fleet-validity predicate's kv clause, ADR-012 §3), rides the
SKIP posture (it reuses the postgres column's reachability probe;
server-less runs skip cleanly; acceptance is green against the
harness server), and extends the factory contract to the fleet
shape: a fleet factory mints one schema and vends two Store
instances over it, each fleet: true, each with its own
store-instance id (the instances share the pool; the pin rows compose
by owner — the fleet's actual definition, ADR-011/ADR-012 §3).
Teardown drops exactly the minted schemas (DROP SCHEMA … CASCADE,
the postgres column's registry pattern — idempotent, owned,
panicking-row-safe), removes every vended temp root, and — the
consolidation shape — reaps the dedicated database's LOs before
dropping it (the pg-lo column's LO-isolation corollary). The test
tuning (the column declares its tuning, the constructor parameters'
purpose): window delay 0 (execute-on-next-sweep) and a short test
pin TTL (the fleet constructors' pin_ttl parameter — the
renewal/expiry rows need observable cadences; the TTL is constructor
configuration, never API). Where a row needs ordering, the window
delay and the TTL tunings provide it, or the independent connection's
manual-expiry instrument (the abandonment shape the maintenance
step reaps — the fleet engine suite's recorded shape); no sleep-based
race assertions — the lease/arbitration outcomes are decided by the
server, not by timing.
Three factories: FleetDualTierFactory (kv = postgres + large =
local, each instance with its own private root riding the operator's
FleetSharedMedia declaration), FleetKvOnlyFactory (the re-routing
posture's client shape, ADR-008 §3 — the mirrored kv-only rejection
row's factory form) and — gated on the postgres + pg-lo feature
pair — FleetConsolidationFactory (kv = postgres + large = pg-lo, the
same pg instance; a dedicated database per open, the LO-isolation
corollary's teardown).
Row (fleet_column.rs) |
Traces to |
|---|---|
every_fleet_valid_configuration_constructs |
the fleet-validity rows: ADR-012 §3 (the composite fleet-validity predicate at the constructor — kv-only, dual-tier declared-shared-media, and two-instances-over-one-pool with distinct owner ids, each exercised); ADR-008 §3 (the declared-shared-media arm; the re-routing arm); ADR-011 §4 (the mode vocabulary); INV-7 facade-contract row |
the_consolidation_fleet_constructs_over_one_shared_instance (postgres+pg-lo only) |
the fleet-validity rows' consolidation form: ADR-012 §4 (the same-instance arm — the system_identifier probe passing by construction); ADR-009 §2 (the companion table); ADR-012 §3 |
the_constructor_entry_guards_reject_the_degenerate_tunings |
the constructor's errors-closed rejects: ADR-008 §2 (a zero pin TTL degenerates the TTL/3 cadence — validated at entry); ADR-013 §2 (the fleet pool-sizing guard: a single-connection pool would self-deadlock the sweep lease); ADR-008 §3 (the sealed FleetSharedMedia — local undeclared has no constructor to reach it through; the structural clauses carry no fleet constructor) |
the_consolidation_same_instance_probe_errors_closed (postgres+pg-lo only) |
the same-instance probe's failure posture: ADR-012 §4 (any probe failure is a constructor error; the mismatching-instances leg needs two servers — recorded unexercisable on the one-instance harness) |
the_joint_entry_and_pin_tx_commits_both_or_neither |
the joint-tx rows: store-api.md §Invariants 4 (a visible pool entry is never unprotected — the joint commit is the mechanism) and §Invariants 3 (zero residue on failed puts); ADR-008 §2 (the joint entry+pin tx, pin-before-publish); ADR-010 §2 (the store core holds the tx end to end); ADR-012 §3/§4 |
a_dedup_reput_upsert_renews_and_a_fleet_reput_composes_a_second_row |
store-api.md §Invariants 2 (the fleet's cross-instance dedup leg); ADR-003 (CAS first-writer-wins); ADR-005 (the re-put refcounts); ADR-010 §2 (protection composes by row); ADR-008 §2 (the row's (key, owner) identity, the upsert-renew) |
the_pg_lo_consolidation_round_trip_is_byte_exact_over_the_shared_instance (postgres+pg-lo only) |
the joint-tx rows' consolidation form (one tx over the shared instance, byte-exact large-entry form, read back from the second instance): store-api.md §Invariants 1 and §Invariants 4; ADR-009 §2; ADR-008 §2; ADR-012 §4 |
the_pin_guards_ttl_third_renewal_moves_the_rows_expiry |
the TTL/renewal rows: ADR-008 §2 (the guard renews its row every TTL/3; a dropped guard stops its duty — no leaked tasks, the row freezes); ADR-012 §2 (the TTL lives with the store's renewal schedule — this column's declared tuning) |
an_expired_row_is_never_renewed_back_to_life |
the never-resurrect semantics through the facade: ADR-008 §2 (an expired row is never revived, even under a live guard whose renewal is still running); ADR-012 §2 |
a_paused_holder_loses_protection_at_expiry_and_a_reput_recovers |
the delete-then-recover backstop: ADR-008 §2 (the expiry semantics verbatim — a paused holder's row abandons at expiry, the delete proceeds, a byte-identical re-put recovers); ADR-012 §2 |
two_instance_exact_counts_instance_a_pins_instance_b_sweeps |
INV-4's engine-gated fleet form (the wave's gate): store-api.md §Invariants 4 (DB-backed pins + the advisory-locked sweep, the exact-count sweep-outcome shape against dockerized postgres) and §Invariants 5 (list correctness through outcomes); ADR-008 §2 (the SQL arbitration composes behind the in-process domain — a's foreign row cancels b's candidate at the execution-time re-arbitration); ADR-012 §1 (the exact report fields); ADR-012 §6.7 (the executor is the next sweep); ADR-013 §2; ADR-007 §5 |
two_instances_sweep_concurrently_one_leases_the_other_aborts_as_data |
INV-4 (the fleet-form SweeperLock shape): ADR-013 §2 (the SQL lease — one sweeper per pool across instances; the second aborts as report data with nothing staged, executed, or deleted); ADR-012 §1 (abort is report data); ADR-012 §6.7 (the holder stages only — the executor is the next sweep) |
a_foreign_owners_unexpired_pin_row_refuses_the_direct_delete_until_expiry |
INV-4 (the cross-instance arbitration row): ADR-008 §2 (the SQL-level arbitration on the direct delete path folds un-expired pin rows of any instance into one answer); ADR-012 §2 (the expiry semantics — the refusal boundary is exactly the row's expiry); ADR-005 (the facade's GcRefuse, nothing deleted) |
a_put_landing_during_the_fleet_window_survives_the_executing_sweep |
INV-4 (the sweep-vs-put race's fleet form, closed by the row): ADR-008 §2 (pin-before-publish; the window delay separates staging from execution); ADR-012 §6.7 (the execution-time re-arbitration is the executor sweep's posture); ADR-012 §2 |
the_maintenance_step_reaps_exactly_the_expired_rows |
INV-4 (the maintenance-step row): ADR-008 §2 (the sweeper's pin-reap — the GC-of-GC cleanup; expired rows exactly, un-expired rows survive; the reap runs before the overdue execution); ADR-012 §6.7; ADR-012 §1 |
an_embedder_liveness_table_protects_fleet_wide_and_is_never_owned |
the fleet liveness row: ADR-012 §6.6 (the fleet liveness verb — the same verb with a fleet-shaped argument; idempotent re-registration; malformed references error); ADR-008 §2 (the schema-ownership rule — the table is the embedder's, consulted at arbitration time, never enumerated); ADR-012 §6.4 (the deregistration correction flow) |
fleet_kv_only_mode_rejects_over_threshold_puts_at_put_time (runner) |
the kv-only re-routing row: the reference set's row mirrored verbatim against the fleet kv-only factory (threshold-agnostic by discovery, unchanged) — ADR-012 §6.2's machinery, the fleet edition (ADR-008 §3's re-routing posture's client shape; store-api.md §Invariants 7: the mode is constructible, the row runs) |
Column plan
mem is the contract-reference engine (ADR-011 §2): wave 2 lands
its column first (the first column binds the Store facade type), and
every other engine's rows mirror mem's column — an addition to mem's
column propagates as a requirement to all other columns (INV-7). The
pg columns (postgres, pg-lo) additionally run the SKIP posture above —
the postgres column is the posture's first landed host (server-less
runs skip cleanly; its wave-4 acceptance is the green run against the
harness server) and the pg-lo column is the second (landed wave 5;
its two factories gate on the mem+pg-lo and postgres+pg-lo
feature pairs respectively — the first column whose factories need
different feature sets — with the column's full run --all-features
against the harness server). The fleet column is the third (wave
6's gate): the engine-gated fleet-form rows of INV-4 run under
--features postgres against two store instances over one shared
pool, their consolidation shape gated on the postgres+pg-lo pair.
Current state
The harness is green on six columns (waves 2-6). The mem
column is the contract-reference column: the factory binding is the
concrete alkblobs::Store facade (see §The factory contract — the
binding demonstrator remains in binding.rs), the reference row set
is landed (§The reference row set — rows.rs, every row ADR-stamped
and traced to the backlog), the column's factories (mem-only, the
reference; kv-only(mem) at a column-declared threshold) vend fresh
isolated stores with idempotent teardown (mem_column.rs, the
structural factory-contract tests included), and mem never skips.
The local column (the suite's first dual-tier column, the
dispatch-coverage shape ADR-012 §6.3 names) mirrors the reference INV
rows against its kv = mem + large = local factory at a
column-declared tuned threshold — the shared matrix and race fixtures
cross it, so both tiers' bodies are exercised — and grows the
dispatch-coverage rows (pure-function routing with an on-disk
placement oracle, fall-through per op, threshold boundaries at both
the tuned and the shipped-default threshold) plus the local-specific
rows (on-disk stage hygiene exact counts, pread windows, virgin-store
no-ops, the both-tiers sweep shape) — §Columns beyond the reference
set is the recorded inventory. Its factory instances each track and
remove their own temp roots (idempotent teardown, panicking-row-safe;
each runner owns its instance so concurrent rows never share roots).
mem-only/kv-only modes and the local column never skip. The local
column is feature-gated on mem and, since feature mem is
default-on (ADR-013 §4's amendment), runs in the plain default build:
the dual_tier_mem_local constructor rides feature mem because mem
is the kv pairing's constructible engine — the gate is the kv
pairing's, not the local engine's, which is ungated in src/ (the
gate still compiles the column out of feature-reduced builds without
mem, a supported shape), with the reference row set compiling
ungated so it stays type-checked everywhere.
The sqlite column (the suite's first SQL-backed column — the
default-build production shape, kv = sqlite + large = local,
ADR-007/013 §4) mirrors the form-independent reference rows (INV-1,
INV-6, the NoLivenessSources/ProtectFailed abort shapes, and the
kv-only rejection row against kv-only(sqlite)) and grows the
durable-schedule sweep rows (§The schedule-relative reading for
SQL-backed columns — the two-sweep exact counts, the race across the
window delay, the holder-stages-only lease shape, list correctness
through durable-schedule outcomes, and the schedule-relative INV-2/3
readings) plus the dispatch rows with the placement oracle adapted by
medium (both tiers on disk: the sqlite file's rows vs the root's
committed files) and the virgin-store row — the recorded inventory is
in §Columns beyond the reference set. Its factories declare the
column-tuned threshold and the test tuning of the window delay
(Duration::ZERO — execute-on-next-sweep), vend a fresh temp site
per open (a private directory holding the sqlite file and the local
root; idempotent teardown removing the site, panicking-row-safe, each
runner owning its instance), and never skip. The column is
feature-gated on sqlite — default-on, so the column runs in the
default build: every cargo test run exercises it. The sqlite
engine's no-tables-yet read tolerance is engine-instrumented
(src/sqlite.rs's tests; the constructor-vs-lazy timing design point
there), the column's virgin-store row being its facade form.
The postgres column (the suite's first SKIP-posture column —
the fleet-capable-kv shape, kv = postgres + large = local, feature
postgres default-off, ADR-007/013 §4, landed wave 4) mirrors the
form-independent reference rows (INV-1, INV-6, the
NoLivenessSources/ProtectFailed abort shapes, and the kv-only
rejection row against kv-only(postgres)) and grows the sqlite
column's durable-schedule sweep rows plus the dispatch rows with the
placement oracle adapted by medium (an independent connection counts
the pg schema's blobs rows vs the root's committed files; the
column's own rows watch the schedule's gc_window rows) and the
virgin-store row — the recorded inventory is in §Columns beyond the
reference set. Its factories declare the column-tuned threshold and
the test tuning of the window delay (Duration::ZERO —
execute-on-next-sweep) and follow the alkstore pg-factory pattern: a
fresh engine-owned schema per open() (unique per call, seeded per
tag/pid/time — racing opens included), tracked per-instance in the
schema registry, teardown DROP SCHEMA IF EXISTS … CASCADE over
exactly the minted schemas plus removal of every vended temp root
(idempotent, owned, panicking-row-safe; each runner owns its
instance). The column skips cleanly server-less under the
reachability probe (ALKBLOBS_PG_HOST gating; the remaining
settings default to the harness server's values) — a skip is not a
failure — and its acceptance is the green run against the harness
server (dockerized pglo-poc, postgres:16-alpine, :15432, db
blobs, verified live 2026-10-12; the alkstore harness's same
server).
The pg-lo column (the suite's second SKIP-posture column —
the LO-medium column, feature pg-lo default-off, ADR-009, its
two factories gating on the mem+pg-lo and postgres+pg-lo
feature pairs — the first column whose factories need different
feature sets, landed wave 5) mirrors the form-independent reference
rows (INV-1's round-trip family, INV-6's stat-discipline row riding
ADR-009's companion-row probe, and the NoLivenessSources/
ProtectFailed abort shapes) and grows the durable-schedule sweep
rows (run on both factories — the window observed on its host:
the pg-lo schema for the mem-pglo factory per the one-window-host
rule's named test topology, the kv schema for the postgres-pglo
consolidation shape, which carries the dormancy row), the
dispatch rows with the placement oracle adapted to the LO medium
(companion rows + pg_largeobject probing for the large tier; the
mem kv half in-process, the postgres kv half's blobs rows), the
INV-1 extension row (the LO medium's ranged reads across the
shipped window edge and the beyond-length clean failure), and the
virgin-store rows — the recorded inventory is in §Columns beyond
the reference set. No kv-only rejection row mirrors here (no
kv-only pg-lo constructor exists; recorded). Connection settings
ride the environment (ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB —
ALKBLOBS_PG_HOST gates; the rest default to the harness server's
values) and the runners gate behind the reachability probe — a skip
is never a failure — and its acceptance is the green run
against the harness server under --all-features (dockerized
pglo-poc, verified live 2026-10-12). Its factories declare the
column-tuned threshold and the test tuning of the window delay
(0 — execute-on-next-sweep) and follow the alkstore pg-factory
pattern with the LO-isolation corollary carried in the engine
task's recorded shape (the C7 lesson): a fresh dedicated database
per open() (unique per call, engine schemas minted inside it,
tracked per-instance; drop-guard janitors backing the abandoned
paths), teardown reaping the instance's LOs (SELECT loid FROM lo_entries → lo_unlink), dropping exactly the minted schemas,
and dropping the database (WITH (FORCE)) — idempotent,
panicking-row-safe — so zero LO residue on the shared database is
structural (the engine suite's
shared_harness_db_holds_zero_lo_residue gate measures it) and the
structural teardown assertions probe zero orphan residue inside the
dedicated database before it goes.
The fleet column (the suite's third SKIP-posture column —
the fleet column, the exact-count fleet sweep outcomes under
concurrency, wave 6's gate, feature postgres default-off, its
consolidation factory also gated on the postgres+pg-lo feature
pair, landed wave 6) grows the fleet column of the suite
(README §Columns beyond the reference set — the recorded inventory is
the fleet rows table there): the fleet-validity rows (the composite
fleet-validity predicate at the constructor — every fleet-valid
configuration constructs, the constructor's entry guards reject the
degenerate tunings, and the consolidation shapes gate on the feature
pair), the joint entry+pin tx rows (the independent connection
sees both or neither; the dedup re-put upsert-renews the row; a second
instance's re-put composes a second row — protection by row; the pg-lo
consolidation round-trip byte-exact over the shared instance), the
TTL/renewal rows (the guard's TTL/3 renewal moves the row's expiry
— the short test TTL makes the cadence observable; the never-resurrect
semantics through the facade; a paused holder loses protection exactly
at expiry with the delete-then-recover backstop), and the exact-count
fleet sweep rows — the wave's gate, INV-4's engine-gated fleet form:
the two-instance exact-count shape (instance A pins, instance B
sweeps — the pinned entry survives via a's row at the execution-time
SQL re-arbitration, the unpinned twins delete, counts exact against
the staged schedule), the lease contention row (two instances sweep
concurrently — exactly one sweeps, the other aborts SweeperLock as
report data with nothing deleted), the cross-instance arbitration row
(a foreign owner's un-expired pin row refuses a direct delete —
GcRefuse — until the row's expiry), the race across the window delay
in its fleet form, the maintenance-step row (expired pin rows reaped
exactly), and the fleet liveness row (a test-owned embedder table
protects through the registration verb — the schema-ownership rule
asserted — with the deregistration correction flow). The kv-only
re-routing row mirrors the reference set's rejection row verbatim
against the fleet kv-only factory (the re-routing posture's client
shape, ADR-008 §3). Its factories follow the postgres column's
registry pattern extended to the fleet shape (one minted schema, two
instances over it, each with its own store-instance id — the pin rows
compose by owner; the dual-tier instances carry private local roots
riding the operator's FleetSharedMedia declaration; the
consolidation factory follows the pg-lo column's LO-isolation
corollary: a dedicated database per open, LO reaping, WITH (FORCE)
drop, drop-guard janitors). Connection settings ride the environment
(ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB) and the runners gate behind
the reachability probe (the postgres column's — a skip is never a
failure), and its acceptance is the green run against the harness
server (dockerized pglo-poc, :15432 — verified live 2026-10-12)
under --features postgres and the consolidation rows under
--all-features. The test tuning: window delay 0
(execute-on-next-sweep) and a short test pin TTL (the fleet
constructors' pin_ttl parameter — the column declares its tuning);
where a row needs ordering, the window delay and the TTL tunings
provide it, or the independent connection's manual-expiry instrument —
no sleep-based race assertions, the lease/arbitration outcomes being
decided by the server, not by timing.