feat(store): the fleet put path — the joint entry+pin tx driver and the pin guard's fleet form (wave 6)

- Under fleet: true, puts route through the joint entry+pin tx
  (store-fleet-put): kv tier via begin_joint_tx -> joint_put (CAS; a
  dedup re-put commits the pin row alone, upsert-renewing
  (key, owner)) -> pin_commit — entry + pin row in one step, verified
  against an independent connection; large = pg-lo rides the one-tx
  consolidation form (known- and unknown-length; the 512 KiB knee,
  the empty-contribution re-put form, read-your-own-writes
  derivation); large = local on declared shared media is
  pin-first, publish-second (the row-only joint tx, then the staged
  commit-rename; the orphan-pin crash window is the expiry
  machinery's).
- Amendment A1 landed: catch-propagate-terminate — every error path
  between begin_joint_tx and pin_commit terminates the held tx before
  propagating (no mid-flight PgJointTx returns dirty to the pool,
  PG-2); tested with a forced mid-driver failure (zero residue, the
  pool's idle-in-transaction count zero, a follow-up put clean).
- The pin guard's fleet form: the returned guard carries the row's
  TTL/3 renewal duty — a per-guard tokio task (spawned at the row's
  durability), silent on Err, ended by the never-resurrect
  Ok(false), aborted at every guard-life boundary (drop, batch
  contribution, conversion); PinRenewal gated feature postgres.
- KvEngine::Postgres holds Arc<PostgresBackend> so detached renewal
  tasks share the fleet host's engine; tokio gains the time feature
  for the cadence.
- Non-fleet put paths byte-unchanged (all existing suites green); a
  non-fleet test pins that no pin rows are ever committed.
- Tests: new postgres_fleet_put_tests (fresh schema + SchemaJanitor;
  kv joint atomicity both-or-neither, forced pin_commit failure,
  renewal cadence/drop/conversion stop, never-resurrect, batch
  release ends renewals, dedup upsert-renewal + by-row composition,
  local pin-first success + forced-failure forms, non-fleet
  non-event) and postgres_pglo_fleet_put_tests (fresh db +
  DatabaseJanitor; the A1 pg-lo failure form, the consolidation
  round-trip through the public put). Env-driven, skipping cleanly
  server-less.

Verified: cargo test (default) green; --features postgres and --features
"postgres,pg-lo" green server-less (skipping) and against the harness
server (pglo-poc, :15432); --all-features green live; pg-pure and
mem,pg-lo pure builds green; clippy -D warnings green (all combos);
cargo fmt --check green; cargo doc --no-deps clean (default; the gated
builds' pre-existing warnings unchanged). Schema/db residue 0.
This commit is contained in:
glm-5.3-flash committed 2026-10-12 16:08:29 +00:00
1 parent 76be7b0273
commit 950af9ab0d
8 files changed
+2080 -52

No files matched your search

+135 -12
View File
@@ -1,7 +1,7 @@
---
id: store-fleet-put
name: the fleet put path — the joint entry+pin tx driver and the pin guard's fleet form
status: pending
status: completed
depends_on: [store-fleet-construction, engine-pg-lo-joint]
scope: broad
risk: high
@@ -108,29 +108,29 @@ protocol verbatim; every wave-2/3/4/5 put test stays green).
## Acceptance Criteria
- [ ] Under `fleet: true`, kv-tier puts route through the joint tx
- [x] Under `fleet: true`, kv-tier puts route through the joint tx
(entry + pin row in one step; an independent connection never
observes one without the other); under `fleet: false` the put
paths are byte-unchanged (all existing suites green)
- [ ] Amendment A1 landed: the driver terminates the tx on every path
from `begin_joint_tx` onward (the termination design recorded;
a forced mid-driver failure leaves zero residue and returns no
- [x] Amendment A1 landed: the driver terminates the tx on every path
from `begin_joint_tx` onward (the termination design recorded; a
forced mid-driver failure leaves zero residue and returns no
dirty session to the pool)
- [ ] The pin guard's fleet form: renewal every TTL/3 while the guard
- [x] The pin guard's fleet form: renewal every TTL/3 while the guard
lives; drop/conversion/batch-contribution stops the renewal
with no leaked tasks; a failed renewal never resurrects an
expired row (the lifecycle design points decided and recorded)
- [ ] The dedup re-put commits the pin row alone (CAS; upsert-
- [x] The dedup re-put commits the pin row alone (CAS; upsert-
renewal); a different instance's re-put composes protection by
row
- [ ] `large = local` fleet puts: pin-row-first, publish-second
- [x] `large = local` fleet puts: pin-row-first, publish-second
(contract-tested — the pin row is durable before the committed
file exists; the orphan-pin crash window is the expiry
machinery's)
- [ ] The pg-lo consolidation put round-trips through the public
- [x] The pg-lo consolidation put round-trips through the public
`put` (one tx over the shared instance; byte-exact large-entry
form)
- [ ] `cargo test` (default) green; `cargo test --features postgres`
- [x] `cargo test` (default) green; `cargo test --features postgres`
and `--features "postgres,pg-lo"` green server-less (skipping)
and against the harness server at acceptance;
`cargo clippy --all-targets -- -D warnings` and
@@ -164,8 +164,131 @@ protocol verbatim; every wave-2/3/4/5 put test stays green).
## Notes
> To be filled by implementation agent
Landed 2026-10-12. Design decisions (each recorded in-code as a named
design point on the fleet impl block of `src/store.rs`; all keyed off
the landed ADR text):
- **A1's termination design — catch-propagate-terminate, no abort
verb.** The driver's steps between `begin_joint_tx` and `pin_commit`
run inside one catch block (an async block's `?` surface); any error
terminates the held tx (a `ROLLBACK` through the
`PgJointTx::client()` checkout — the wave-5 landed accessor) before
propagating; `pin_commit` remains the terminal-consume step (its own
failure paths roll back per the wave-4 clause contract). An explicit
abort verb was rejected per A1's recorded options (it would need a
new ADR; the seam shape is pinned). A ROLLBACK on a session with no
active transaction (the clause bodies' having already rolled back)
is a completed, warning-level no-op — the blanket termination is
always safe. `pin_commit`-consuming txs on failure are the clause's
own; the driver never re-uses a consumed tx.
- **The renewal duty's mechanism — a per-guard tokio task on a TTL/3
interval**, holding an `Arc<PostgresBackend>` (the kv-tier enum's
postgres payload moved to an `Arc` for this: the detached task
outlives the `Store` and keeps the backend's pool exactly as long as
the pin lives; `PostgresBackend` itself is not `Clone` — its lease
checkout must not be). The shared-renewal-sweep keyed off a guard
registry was the rejected alternative: it needs precisely the
lifecycle notifications (mint/drop/convert/batch-contribute) that an
abort handle gives for free. The task is spawned **at the pin row's
durability** (after `pin_commit`, never at guard mint — a renewal
arriving at a nonexistent row would read the not-yet-pinned window),
on the cadence `(ttl / 3).max(1 ms)` (the 1 ms floor is the
degenerate-tuning guard: a TTL below 3 ms would floor to a zero
poll). The store resolves the expiry (`now + ttl`) at the joint tx's
`pin_commit`; successive renewals ride the engine's `pin_renew`
(the successor-time body reads the TTL the fleet constructor passed
the engine — `store-fleet-construction`'s wiring).
- **The failure posture: silent-and-continued on `Err`, ended on
`Ok(false)`.** The row's expiry is the protection — a missed renewal
degrades to ADR-008's expiry semantics, never an error on the put
path (the crate carries no logging dependency; embedder visibility
into renewals observes the row). `Ok(false)` — an expired or absent
`(key, owner)` row — ends the duty: the engine never resurrects
(wave 4's recorded semantics), so continuing would only churn; the
duty ends, the delete-then-recover backstop's territory begins.
- **The abort-on-drop discipline:** the duty is a
`PinRenewal` (a `JoinHandle` wrapper, crate-internal, gated feature
`postgres`) attached to the returned `Pin`; it is aborted by the
guard's drop, by `Batch`'s one-step release (the duty moves into the
batch at `Batch::add` — a member's row keeps renewing while the
batch lives), and by the ordered-step conversion
(`RegistryCore::convert` pre-stops before its `ManuallyDrop`
suppresses the guard's drop). No leaked tasks; no renewal past the
guard's life. A task aborted mid-renewal is harmless (the clause's
expiry update is idempotent).
- **The routing never re-derives from the engine set:** every fleet
branch keys off `Store::fleet`'s presence first; the pg-lo tier's
joint-vs-staged choice inside fleet mode rides the engine enum (the
tier mechanics' selector, the same dispatch as every other fleet
branch shape). The public `put`/`put_in_batch` signatures are
unchanged; the non-fleet paths are byte-unchanged (all existing
suites green untouched).
- **The kv joint form's mechanics:** the in-process refcount mints
first (unchanged bookkeeping), then `begin_joint_tx` → `joint_put`
(the engine's CAS row write; a dedup re-put contributes nothing —
the recorded CAS form; the row upsert-renews via `pin_commit`'s
`ON CONFLICT (key, owner)` upsert) → `pin_commit(tx, [row])` —
publish + protection in one step. The row's `owner` is the
store-instance id; `expiry = now + pin_ttl`.
- **The consolidated (pg-lo) form:** the tx begins, the tier's joint
staging cursor streams the content inside it (the engine-level
contract: the 512 KiB knee rides, the unknown-length form's
buffer-to-threshold phase is put_unknown's verbatim shape and its
derivation pass re-reads the artifact through the cursor's
read-your-own-writes seam), the cursor's publish is the CAS
contribution (a re-put: the empty-contribution form — the staged LO
unlinked in-tx, zero residue), and `pin_commit` lands the row with
the entry.
- **Test isolation:** every store test takes a fresh engine-owned
schema (the LO-free kv tier; `SchemaJanitor`) or a fresh dedicated
database (the LO-bearing consolidation tests; `DatabaseJanitor`)
with a janitor removing it in every exit path — amendment A2's
lesson honored from the start. The A1 dirty-session observable is
`pg_stat_activity`'s `idle in transaction%` count asserted to zero
(plus the follow-up put's clean round-trip and the zero-residue
table assertions).
## Summary
> To be filled on completion
The fleet put path landed. Under `fleet: true`, `put`/`put_in_batch`
(unchanged signatures) route: **kv tier** — the joint entry+pin tx
(`begin_joint_tx` → the engine's CAS `joint_put` → `pin_commit`,
publish + protection in one step; a dedup re-put commits the pin row
alone, the row upsert-renewing `(key, owner)`; a different instance's
re-put mints a different row — protection composes by row); **large =
pg-lo (the consolidation posture)** — one tx over the shared instance,
the tier's joint staging cursor streaming the content inside the
store-held tx (known- and unknown-length; the 512 KiB knee, the CAS
publish, the empty-contribution re-put form, read-your-own-writes
derivation passes), `pin_commit` closing; **large = local (declared
shared media)** — pin-first, publish-second: a row-only joint tx
commits the pin row before the staged commit-rename (the
forced-publish-failure test proving the ordering: the pin row is
durable with no committed file, the orphan reclaimed by the expiry
machinery; never the reverse ordering). Amendment A1 landed as its
recorded owner — catch-propagate-terminate: every error path between
`begin_joint_tx` and `pin_commit` terminates the held tx before
propagating (tested: a mid-driver contribution failure leaves zero
residue and returns no dirty pooled session — `pg_stat_activity`'s
idle-in-transaction count zero — and the pool's follow-up put
round-trips clean). The pin guard's fleet form landed: the returned
guard carries the row's TTL/3 renewal duty (a per-guard tokio task,
spawned at the row's durability, silent on `Err`, ended by the
never-resurrect `Ok(false)`, aborted at every guard-life boundary —
drop, batch contribution, conversion). The in-process refcount
machinery continues to function (instance-local bookkeeping, the
batch's one-step release, the ordered-step conversion); the row is
the cross-instance protection; nothing expires in non-fleet modes.
Non-fleet put paths are byte-unchanged (all existing suites green
untouched; a non-fleet test additionally pins that no pin rows are
ever committed). Verified: `cargo test` (default) green; `--features
postgres` and `--features "postgres,pg-lo"` green server-less
(skipping cleanly) and against the harness server (pglo-poc, :15432 —
live, all fleet-put tests green); `--all-features` and the pg-lo-pure
`--features mem,pg-lo` builds green; clippy `-D warnings` green
(default, mem, postgres, all-features, pg-pure, mem+pg-lo, and
postgres+pg-lo); `cargo fmt --check` green; `cargo doc --no-deps`
zero warnings in the default build (the gated builds' pre-existing
warnings counted before and after — identical). The fleet sweep
(`store-fleet-sweep`) holds the fleet mechanism's remaining
store-core scope (the sweep lease, the SQL arbitration).