alkblobs/tasks/suite-column-fleet.md

---
id: suite-column-fleet
name: Conformance suite — the fleet column (exact-count fleet sweep outcomes under concurrency)
status: completed
depends_on: [store-fleet-put, store-fleet-sweep]
scope: moderate
risk: medium
impact: phase
level: implementation
tags: [wave-6, suite, fleet, postgres]
---

## Description

Grow the **fleet column** — the suite's proof that the architecture's
most intricate state machine holds under concurrency (the wave's
gate: "exact-count fleet sweep outcomes against live postgres",
docs/plans/implementation.md §Wave 6). The column runs under
`--features postgres` (fleet mode requires kv = postgres — the
composite predicate), rides the SKIP posture (server-less runs skip
cleanly; acceptance requires green **against the server**), and
extends the factory contract to the fleet shape: **two store
instances over one shared pool**.

- **The fleet factory** (`tests/conformance/fleet_column.rs`): the
  postgres column's schema-per-open pattern adapted — a fleet factory
  mints **one schema** and vends **two (or more) `Store` instances
  over it**, each `fleet: true`, each with its own owner id (the
  instances share the pool; the pin rows compose by owner — the
  fleet's actual definition, ADR-011/ADR-012 §3). Teardown drops
  exactly the minted schemas (`DROP SCHEMA … CASCADE`, the postgres
  column's registry pattern — panicking-row-safe, idempotent,
  owned). The test tuning: window delay `0` (execute-on-next-sweep)
  and a **short test TTL** (the fleet constructor parameter — the
  renewal/expiry rows need observable cadences; the TTL is a
  constructor parameter, not API, so the column declares its tuning).
  A kv-only fleet factory (the re-routing posture's client shape) and
  a consolidation fleet factory (kv = postgres + large = pg-lo, same
  instance) join as the column's second/third shapes.
- **The fleet-validity rows** (the predicate at the constructor):
  every fleet-valid configuration constructs; every non-fleet-valid
  configuration is a constructor error (kv ≠ postgres, `mem`
  anywhere, `local` without the shared-media declaration, pg-lo
  cross-instance) — the predicate's clause-per-row form.
- **The joint entry+pin tx rows** (through the public `put`, not the
  seam): entry + pin row commit together (an independent connection
  sees both or neither); the dedup re-put upsert-renews the row; a
  second instance's re-put composes a second row (protection by
  row); the pg-lo consolidation round-trip (one tx over the shared
  instance, byte-exact large-entry form).
- **The TTL/renewal rows**: the guard's TTL/3 renewal moves the row's
  expiry (the short test TTL makes the cadence observable); a guard
  dropped past expiry never resurrects its row (the never-resurrect
  semantics through the facade); a paused holder (renewal stopped)
  loses protection at expiry — the delete-then-recover backstop (a
  byte-identical re-put recovers).
- **The exact-count fleet sweep rows** (the wave's gate — INV-4's
  engine-gated fleet form, store-api.md §Invariants 4: "fleet
  postures assert it under DB-backed pins + the advisory-locked
  sweep, with the exact-count sweep-outcome shape run against
  dockerized postgres"):
  - the **two-instance exact-count shape**: instance A pins an entry
    (its row), instance B sweeps — the pinned entry survives, the
    unpinned twins delete, counts exact against the staged schedule;
  - the **lease contention row**: two instances sweep concurrently —
    exactly one sweeps, the other aborts `SweeperLock` as report
    data with nothing deleted;
  - the **cross-instance arbitration row**: a foreign owner's
    un-expired pin row refuses instance B's direct delete
    (`GcRefuse`); after the row expires (renewal stopped, short TTL),
    the delete proceeds;
  - the **race across the window delay** in its fleet form: a put
    landing during the delay (its row committed before its entry is
    visible) survives the next sweep's execution-time SQL
    re-arbitration;
  - the **maintenance-step row**: expired rows reaped exactly (the
    GC-of-GC cleanup), un-expired rows survive;
  - the **fleet liveness row**: a test-owned embedder table's rows
    protect through the fleet registration verb (the schema-ownership
    rule asserted — the engine never created the table).
- **The kv-only re-routing row**: the fleet kv-only client shape
  rejects over-threshold puts at put time (ADR-012 §6.2's machinery,
  fleet edition) — the re-routing posture's store-side half.
- **README amendments** (the convention's normative test-side home):
  the fleet rows table in §Columns beyond the reference set; the
  column plan's SKIP-posture note naming the fleet column; current
  state.
- **Connection settings ride the environment** — the established
  harness convention: `ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`, the
  reachability probe gating the rows. **The harness server**:
  dockerized `pglo-poc` (`postgres:16-alpine`) on :15432, db `blobs`,
  user/password `postgres`/`poc` — verified live 2026-10-12; the
  container is long-running in the dev environment. Concurrency rows
  are deterministic by construction (the lease/arbitration outcomes
  are decided by the server, not by timing) — where a row needs
  ordering, the window delay and the TTL tunings provide it; no
  sleep-based assertions.

## Acceptance Criteria

- [x] The fleet column green **against the harness server**
      (`pglo-poc`, :15432): the fleet-validity rows, the joint-tx
      rows, the TTL/renewal rows, the exact-count fleet sweep rows
      (two-instance survival, lease contention, cross-instance
      arbitration, the race across the delay, the maintenance step,
      the fleet liveness row), and the kv-only re-routing row — every
      row ADR-stamped and traceable to the backlog
- [x] The fleet factory contract holds: one schema, two+ instances
      over it (distinct owner ids), teardown drops exactly the minted
      schemas, panicking-row-safe — structurally exercised
- [x] Server-less runs skip cleanly (the reachability probe; the
      default-build gates stay green with the column skipped)
- [x] README amendments recorded (the fleet rows table; current
      state)
- [x] `cargo test` (default) green — the column compiles out;
      `cargo test --features postgres` green server-less (skipping)
      and green against the server at acceptance;
      `cargo test --all-features` green (the consolidation factory's
      home); `cargo clippy --all-targets -- -D warnings` and
      `cargo fmt --check` green (default + the postgres/pg-lo
      variants + `--all-features`)

## References

- tests/conformance/README.md (the conventions this grows against;
  §The factory contract, §The SKIP posture, §The schedule-relative
  reading, §Columns beyond the reference set)
- tests/conformance/postgres_column.rs (the SQL-backed column pattern
  — the schema-per-open factory this extends to the fleet shape)
- tests/conformance/factory.rs (the factory contract)
- docs/architecture/store-api.md (§Invariants 4 — the engine-gated
  fleet form this column asserts)
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
  §2 (the fleet mechanism the rows assert)
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
  §3 (the declaration), §4 (the host), §6.6 (the fleet liveness verb)
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
  §2 (the lease semantics the contention row asserts)
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
  §5 (the exact-count sweep-outcome gate against dockerized postgres)
- tasks/store-fleet-put.md, tasks/store-fleet-sweep.md,
  tasks/store-fleet-construction.md (the machinery the rows exercise)

## Notes

Landed as `tests/conformance/fleet_column.rs` (cfg `feature =
"postgres"`; its consolidation pieces cfg the `postgres`+`pg-lo`
pair), wired in `tests/conformance/main.rs`, README amended (§Columns
beyond the reference set's fleet rows table, §Column plan, §Current
state — six columns). Three factories: `FleetDualTierFactory` (one
minted schema, two instances over it — each its own owner id and
private local root riding `FleetSharedMedia::asserted`),
`FleetKvOnlyFactory` (the re-routing client shape; implements
`StoreFactory` so the reference kv-only rejection row mirrors
verbatim), `FleetConsolidationFactory` (a dedicated database, kv + pg-lo
schemas, two fleet instances; the pg-lo column's LO-isolation corollary
— LO reaping, `WITH (FORCE)` drop, drop-guard janitors; a `TrackedDb`
constructor was added to pg_lo_column.rs since the janitor field is
private). The oracle reuses the postgres column's skip probe
(`harness_ready`) and observes the durable state over independent
connections (pool/window/pin-row counts, schema/table existence, the
embedder-table instruments, and the manual-expiry instrument for the
abandonment shape). Column tuning: window delay `0`, TEST_TTL = 900 ms
(TTL/3 ≈ 300 ms), waits are the expiry cadence itself or the manual
instrument — no sleep-based race assertions.

Recorded deviations from the task text, none blocking:

- The composite predicate's *structural* non-fleet-valid shapes (kv ≠
  postgres, mem anywhere, `local` undeclared) have no fleet
  constructor to reach through (the recorded enforcement map) — the
  column asserts the constructor-set shape in prose + the runtime
  rejects that do exist (zero TTL, single-connection pool, the
  same-instance probe's failure posture). A `TrackedDb::new`
  constructor was added rather than widening the field's visibility.
- The pg-lo **cross-instance mismatch** leg is unexercisable against
  the one-instance harness server (the probe reads
  `pg_control_system()` off one data directory); the row tests the
  probe's errors-closed failure posture (an unreachable large
  instance rejects at the constructor) and records the mismatch leg —
  the same posture the engine suite's tests carry.
- The renewal row's "renewed == at_drop" compare reads at_drop *after*
  the drop and asserts `>=` — one TTL/3 tick may legitimately land
  between the pre-drop read and the drop; the frozen-after-drop
  assertion (EXPIRY_HORIZON, `==`) is the never-leaks proof.

## Summary

The fleet column is green: the wave's gate rows run against the
harness server under `--features postgres` (17 rows + 2 structural
factory tests) and under `--all-features` (the consolidation rows
join) — the fleet-validity rows, the joint-tx rows (both-or-neither,
the upsert-renew, second-row composition, the byte-exact
consolidation round-trip), the TTL/renewal rows (cadence,
never-resurrect, paused-holder + recover), the exact-count fleet sweep
rows (two-instance survival with exact counts, lease contention as
report data, cross-instance arbitration until expiry, the race closed
by the row, the maintenance step's exact reap, the fleet liveness verb
with the schema-ownership rule), and the kv-only re-routing row —
every row ADR-stamped and traced. The fleet factory contract holds
structurally (one schema, two instances, distinct owner ids, owned
idempotent teardown, panicking-row-safe); server-less runs skip
cleanly; the default build compiles the column out. Verification:
`cargo test` (default) green; `cargo test --features postgres` green
server-less (skipping) and green against `pglo-poc` (:15432);
`cargo test --all-features` green against the server (125 conformance
tests + the full library suite, 226 tests); `cargo clippy --all-targets
-- -D warnings` and `cargo fmt --check` green on the default, postgres,
and all-features variants.