docs(tasks): decompose wave 6 — the fleet GC mechanism, the architecture's most intricate state machine under concurrency
- store-fleet-construction: the fleet: true declaration slotting into the fleet-capable constructors (the recorded non-event), the composite fleet-validity predicate enforced at the constructor (kv=postgres AND pg-lo-same-instance OR local-on-declared-shared- media OR kv-only re-routing), the TTL/owner-id fleet state, the same-instance probe design point - engine-pg-lo-joint: the pg-lo joint contribution — staged-put mechanics (512 KiB knee, CAS, transactional LO creation) into the store-held PgJointTx, the wave-5 deferred API-shape design point - store-fleet-put: the joint entry+pin tx driver (amendment A1's drop discipline lands here — its recorded owner), the pin guard's TTL/3 renewal form, the dedup upsert-renewal, pin-first/ publish-second for large=local, the consolidation one-tx put - store-fleet-sweep: the SQL sweep lease (ADR-013 §2 semantics), SQL-level delete arbitration on the direct + window-execution paths, the expired-pin reaping clause (the seam as the named home for fleet-clause growth), the fleet liveness registration verb, the two-domain arbitration composition - suite-column-fleet: the fleet column — two-instance factories over one schema, the exact-count fleet sweep rows (the wave's gate), lease contention, cross-instance arbitration, TTL/renewal rows, the fleet-validity predicate rows - review-wave-6: the fleet gate per the plan, plus a standing code-smell/hygiene block — cargo doc --no-deps asserted clean (the twice-bitten PGLO-1 rustdoc private-link class), the allow(dead_code) scope notes audited against their remainder, stale wave-6 forward references, and task-Summary-vs-diff correctness Harness server cited in every server-touching task: dockerized pglo-poc (postgres:16-alpine) on :15432, db blobs, postgres/poc (verified live 2026-10-12); env-driven ALKBLOBS_PG_* settings, clean server-less skips, acceptance requires green against the server. Dependency graph validated: two parallel roots (construction, pg-lo joint), put merging them, sweep parallel to put, the fleet column merging, the review gate last; no cycles.
This commit is contained in:
glm-5.3-flash committed
1 parent
93f9a9332c
commit
009de420e1
7 files changed
+982
-2
No files matched your search
@@ -1,6 +1,6 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: "2026-10-12 (wave 5 decomposed into tasks/)"
|
||||
last_updated: "2026-10-12 (wave 6 decomposed into tasks/)"
|
||||
---
|
||||
|
||||
# alkblobs — Implementation plan
|
||||
@@ -53,7 +53,7 @@ ops payload surfaces; release readiness is last.
|
||||
| [3](#wave-3--sqlite-engine) | sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column | wave 2 | decomposed (2026-10-11) |
|
||||
| [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | decomposed (2026-10-12) |
|
||||
| [5](#wave-5--pg-lo-large-engine) | pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column | wave 4 | decomposed (2026-10-12) |
|
||||
| [6](#wave-6--fleet-gc) | Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations | waves 4 + 5 | pending |
|
||||
| [6](#wave-6--fleet-gc) | Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations | waves 4 + 5 | decomposed (2026-10-12) |
|
||||
| [7](#wave-7--ops-surface) | `ops` feature: alkcall-backed op family, ACL mapping, pin tokens + renewal, fanout seam | wave 6 | pending |
|
||||
| [8](#wave-8--fuzzing) | Fuzzing: family pattern targets (key encoding, ops payloads), corpus-replay gate | wave 7 | pending |
|
||||
| [9](#wave-9--release-readiness) | Crate docs, feature-graph verification, README, publish prep | wave 8 | pending |
|
||||
|
||||
@@ -0,0 +1,145 @@
|
||||
---
|
||||
id: engine-pg-lo-joint
|
||||
name: the pg-lo joint contribution — staged-put mechanics into the store-held tx
|
||||
status: pending
|
||||
depends_on: []
|
||||
scope: moderate
|
||||
risk: high
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-6, engines, pg-lo, gc]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Land the **pg-lo engine's joint contribution** — the wave-5 deferred
|
||||
design point, recorded in `src/pg_lo.rs`'s scope boundary ("the *fleet*
|
||||
joint form (wave 6's store-held tx, ADR-012 §4/§5's ruling that pg-lo
|
||||
contributes LO writes + companion-row insert into it) is a wave-6
|
||||
design point") and in `src/gc.rs`'s `EngineFleetState::joint_put`
|
||||
clause doc ("an engine-task contribution against this `JointTx`
|
||||
type"). Under the fleet consolidation posture (kv = postgres +
|
||||
large = pg-lo, same pg instance — admitted by
|
||||
`store-fleet-construction`'s predicate), a large entry's content
|
||||
reaches the **same store-held transaction** the pin row commits in:
|
||||
one tx over the shared instance — the pg-lo engine contributes its
|
||||
staged-put mechanics (chunked `lowrite` + companion-row insert) into
|
||||
it; the pin-row insert and the commit stay the store's (`pin_commit`,
|
||||
the postgres host's clause — landed wave 4, unchanged here).
|
||||
|
||||
- **The one-tx ruling** (ADR-012 §4): the store core holds one
|
||||
transaction over the shared instance; the pg-lo engine contributes
|
||||
LO writes + companion-row insert; the pin-row insert is the store's.
|
||||
The engine sees pins only as rows; never as semantics (ADR-010 §2's
|
||||
correction of ADR-009's ownership statement — nothing of
|
||||
`EngineFleetState` lands on pg-lo itself; the contribution is
|
||||
mechanics the store core drives).
|
||||
- **The contribution's API shape** — the named design point wave 5
|
||||
deferred. The engine's existing staged put holds its own tx (the
|
||||
held-tx posture: the staging cursor owns a pool checkout with an
|
||||
explicit session-level `BEGIN`); the joint form inverts the
|
||||
ownership — the store core's `PgJointTx` (the postgres engine's
|
||||
held checkout, wave 4's recorded shape) is the tx the LO writes
|
||||
ride. **Named design point — the contribution's API home**: decide
|
||||
the shape consistent with the seam's ownership rule (the store
|
||||
holds the tx; the engine contributes mechanics into it) — the
|
||||
natural shapes are a crate-internal pg-lo method taking
|
||||
`&mut PgJointTx` (the engine writes `lo_create`/`lowrite` +
|
||||
the companion-row insert through the held checkout) or a joint
|
||||
staging-cursor variant handed out against the held tx; decide,
|
||||
record, and keep the public trait surface untouched (the
|
||||
contribution is crate-internal, like every seam clause). The
|
||||
`JointTx` type is the postgres engine's `PgJointTx` — the
|
||||
consolidation posture's one shared instance is what makes the
|
||||
type shareable (ADR-012 §4's checkable clause;
|
||||
`store-fleet-construction` enforces it).
|
||||
- **The staged-put mechanics, re-shaped not re-written**: the 512 KiB
|
||||
`lowrite` knee (`LOWRITE_CHUNK`, the C2 citation), the CAS
|
||||
existence check first (a re-put commits an empty contribution,
|
||||
never `lo_create`), the companion-row insert, transactional LO
|
||||
creation (zero crash orphans — C6, structurally: the LO lives and
|
||||
dies with the store-held tx) — the engine's landed bodies carry;
|
||||
what changes is whose tx they run in. The engine-level held-tx
|
||||
staging cursor is **unchanged** (the non-fleet put path keeps its
|
||||
own tx; the wave-5 hygiene tests stay green untouched).
|
||||
- **Failure discipline**: every failure path inside the contribution
|
||||
leaves the store-held tx in a state the driver can terminate — the
|
||||
contribution never commits, never rolls back the store's tx
|
||||
unilaterally (the driver owns termination; amendment A1's
|
||||
discipline — the driver task owns the catch-propagate-terminate
|
||||
design; this task's duty is that the contribution's error paths
|
||||
compose with it: no partial LO state that outlives the tx's
|
||||
rollback, which transactional LO creation already guarantees).
|
||||
- **Contract tests** (engine-level, in `src/pg_lo.rs`'s test module,
|
||||
the established env-driven convention): the contribution writes LO
|
||||
content + the companion row into a `begin_joint_tx`-held tx and
|
||||
`pin_commit` publishes entry + pin row in one step (byte-exact
|
||||
round-trip after commit); a forced failure mid-contribution aborts
|
||||
the whole tx with zero residue (no LO pages, no companion row, no
|
||||
pin row, no kv row — the stage-hygiene invariant's tx form, the
|
||||
wave-4 joint-atomicity test's pg-lo edition); the CAS re-put form
|
||||
(a second contribution of the same key commits an empty
|
||||
contribution; the pin row upsert-renews); the 512 KiB knee rides
|
||||
(a multi-chunk contribution). **Tests need a live server** — the
|
||||
alkstore harness convention: env-driven (`ALKBLOBS_PG_*`), skip
|
||||
cleanly server-less, acceptance requires green against the harness
|
||||
server (`pglo-poc`, :15432, db `blobs`, `postgres`/`poc` — verified
|
||||
live 2026-10-12). Adopt the `SchemaJanitor`/per-test-database
|
||||
isolation posture from the start (amendment A2's lesson — the
|
||||
wave-4 leak is not re-learned).
|
||||
- **Scope boundary**: the store-core driver that calls this
|
||||
contribution on the fleet put path is `store-fleet-put`; this task
|
||||
ships the engine-side mechanics and their contract tests. The
|
||||
non-fleet pg-lo put path is untouched.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The pg-lo joint contribution exists behind the crate-internal
|
||||
seam (the public trait surface untouched; nothing of
|
||||
`EngineFleetState` implemented on pg-lo itself)
|
||||
- [ ] The contribution's API shape decided and recorded (the
|
||||
store-held `PgJointTx` is the tx the LO writes ride; the
|
||||
ownership rule honored — the engine contributes, never owns)
|
||||
- [ ] One-tx publish: LO content + companion row + pin row + kv row
|
||||
commit in one step; byte-exact round-trip after `pin_commit`
|
||||
(contract-tested)
|
||||
- [ ] Zero residue on any failure path (no LO pages, no companion
|
||||
row, no pin row, no kv row — contract-tested with a forced
|
||||
failure); the contribution's error paths compose with the
|
||||
driver's termination discipline (A1)
|
||||
- [ ] The CAS re-put form (empty contribution, upsert-renewal) and
|
||||
the 512 KiB knee contract-tested
|
||||
- [ ] The non-fleet held-tx staging path and its wave-5 hygiene tests
|
||||
unchanged and green
|
||||
- [ ] `cargo test --features "pg-lo,postgres"` green server-less
|
||||
(skipping) and against the harness server at acceptance;
|
||||
`cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green (default + the pg-lo variants);
|
||||
`cargo doc --no-deps` clean
|
||||
|
||||
## References
|
||||
|
||||
- src/pg_lo.rs (the scope boundary recording this task; the held-tx
|
||||
staging bodies this re-shapes; `LOWRITE_CHUNK`)
|
||||
- src/gc.rs (`EngineFleetState::joint_put`'s clause doc — the
|
||||
contribution's seam home; `JointTx`'s ownership rule)
|
||||
- src/postgres.rs (`PgJointTx` — the held-checkout tx type; the
|
||||
wave-4 recorded design points)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§4 (the one-tx ruling; the same-instance clause)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
|
||||
§2 (the ownership rule; the pg-lo correction)
|
||||
- docs/architecture/decisions/009-pg-lo-large-tier-engine.md (the
|
||||
engine shape the mechanics carry)
|
||||
- docs/research/poc-pglo-findings.md (C2 the knee, C6 zero orphans)
|
||||
- tasks/engine-pg-lo.md (the staged-put bodies this extends)
|
||||
- tasks/review-wave-4.md (amendment A1 — the drop discipline the
|
||||
contribution composes with)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,145 @@
|
||||
---
|
||||
id: review-wave-6
|
||||
name: Wave 6 review gate — the fleet gate
|
||||
status: pending
|
||||
depends_on: [suite-column-fleet]
|
||||
scope: narrow
|
||||
risk: low
|
||||
impact: phase
|
||||
level: review
|
||||
tags: [wave-6, review]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
The wave-6 review gate — **the fleet gate** (docs/plans/implementation.md
|
||||
§Review gates: "exact-count sweep outcomes under concurrency vs live
|
||||
postgres, the joint entry+pin tx atomicity, the sweep lease semantics
|
||||
vs ADR-013 §2, the composite fleet-validity predicate enforced at the
|
||||
constructor, kv-only put-time rejection"), before wave 7 decomposes.
|
||||
The gate reads the implementation against the pinned ADR text line by
|
||||
line (the wave-1..5 gates' method) — the documents, not memory.
|
||||
|
||||
Gate checks:
|
||||
|
||||
- **Exact-count fleet sweep outcomes under concurrency vs live
|
||||
postgres**: the fleet column green against the harness server
|
||||
(server-less runs skip cleanly — a skip never substitutes for
|
||||
acceptance); the two-instance survival shape, the lease-contention
|
||||
abort, the cross-instance arbitration refusal, the race across the
|
||||
window delay, and the maintenance-step reaping all exact-count
|
||||
asserted (INV-4's engine-gated fleet form).
|
||||
- **The joint entry+pin tx atomicity**: through the public `put` (not
|
||||
only engine-level) — an independent connection never observes an
|
||||
entry without its pin row or the reverse; the A1 drop discipline
|
||||
audited as implemented (every path from `begin_joint_tx` onward
|
||||
terminates the tx — the wave-4 gate's amendment closed, its owner
|
||||
discharged); the pg-lo consolidation one-tx form (LO writes +
|
||||
companion row + pin row + commit in one tx; zero residue on
|
||||
failure); the large=local pin-first/publish-second ordering (the
|
||||
reverse ordering nowhere coded).
|
||||
- **The sweep lease semantics vs ADR-013 §2**: one sweeper per pool
|
||||
across instances; atomic test-and-set; holdership ends at
|
||||
release/owner death/session loss; contention is the
|
||||
`GcAbortCause::SweeperLock` report abort (`Ok(SweepReport)`, nothing
|
||||
deleted — ADR-012 §1), never an error.
|
||||
- **The composite fleet-validity predicate enforced at the
|
||||
constructor**: every fleet-valid configuration constructs; every
|
||||
non-fleet-valid configuration is a constructor error (kv ≠
|
||||
postgres, `mem` anywhere, `local` without the shared-media
|
||||
declaration, pg-lo cross-instance); the declaration is explicit,
|
||||
never inferred (ADR-012 §3); the same-instance probe's design
|
||||
audited (a failed probe errors closed).
|
||||
- **kv-only put-time rejection** under the re-routing posture
|
||||
(ADR-012 §6.2's machinery, fleet edition).
|
||||
- **The TTL/3 renewal on the guard**: the renewal lifecycle audited
|
||||
(no leaked tasks; drop/conversion/batch-contribution stops
|
||||
renewal; a failed renewal never resurrects an expired row —
|
||||
ADR-012 §2's expiry semantics verbatim); the TTL is a fleet
|
||||
constructor parameter (default 60 s), never API.
|
||||
- **The fleet liveness verb vs ADR-012 §6.6 / ADR-008 §2**: the named
|
||||
table reference form; the schema-ownership rule (the store reads
|
||||
the embedder's table, never owns it); the protect callback's
|
||||
single-node-only posture recorded; the two-domain arbitration
|
||||
composition (in-process + SQL) decided and recorded.
|
||||
- **The one-window-host rule under fleet**: every fleet-valid
|
||||
configuration's window hosts on the kv engine; the pg-lo window
|
||||
clauses dormant; the form branch keys off the engine
|
||||
representation, never a flag.
|
||||
- **The non-fleet postures unchanged**: the in-process protocol
|
||||
verbatim under `fleet: false` regardless of engine — the wave-2/3/4/5
|
||||
suites green untouched.
|
||||
- **Code-smell / hygiene sweep** (the gate's standing hygiene block —
|
||||
the twice-bitten lesson is now an explicit check, not a hope):
|
||||
- **`cargo doc --no-deps` zero warnings** — the PGLO-1 class
|
||||
(public-module doc links resolving to crate-internal items) has
|
||||
bitten waves 4 and 5; the gate runs rustdoc and asserts clean,
|
||||
and de-links rather than exports.
|
||||
- **The `#![allow(dead_code)]` scope notes match their remaining
|
||||
scope** — the fleet family's note in `src/gc.rs` (its consumers
|
||||
land this wave) must shrink to its true remainder or vanish; the
|
||||
postgres/pg-lo notes likewise audited.
|
||||
- **Stale forward references updated** — every "wave 6" note in the
|
||||
module docs (`store.rs`'s fleet-declaration paragraph, the
|
||||
constructors' "slots in without reshaping" recordings,
|
||||
`pg_lo.rs`'s scope boundary, `postgres.rs`'s scope note, the
|
||||
batch-delete helper's consumer note) reads the landed state.
|
||||
- **Family conventions**: no inline comments in code (doc comments
|
||||
on public API fine), thiserror, tokio-only async, no
|
||||
`unwrap`/`expect` outside tests, naming, one-module-per-file
|
||||
re-export shape, no blocking I/O on the async path.
|
||||
- **Task-description correctness**: each wave-6 task's Summary
|
||||
matches what actually landed (the gate reads the diffs against
|
||||
the task texts, not the Notes' claims alone).
|
||||
- **Gates green**: `cargo test` (default), `cargo test --features
|
||||
postgres` (server-less: skipping), `cargo test --features
|
||||
"postgres,pg-lo"`, `cargo test --all-features` (against the
|
||||
server), the pure builds (`cargo check --no-default-features
|
||||
--features postgres` and `--features "postgres,pg-lo"`), `cargo
|
||||
clippy --all-targets -- -D warnings` (default + the postgres/pg-lo
|
||||
variants + `--all-features`), `cargo fmt --check`,
|
||||
`cargo doc --no-deps`.
|
||||
|
||||
Findings are fixed inline where cheap, or recorded as wave-7
|
||||
amendments where they shape the next decomposition (the ops wave's
|
||||
pin-token contract rides the fleet machinery — a renewal-surface
|
||||
finding lands beside it). The gate's completion is what unblocks
|
||||
wave 7's decomposition.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Every gate check above read and verified against the cited ADR
|
||||
text (not from memory of it)
|
||||
- [ ] Named design points each decided, recorded, and consistent with
|
||||
the specs
|
||||
- [ ] Findings fixed inline or recorded as wave-7 amendments with
|
||||
owners
|
||||
- [ ] All gates re-run green after any fixes — including
|
||||
`cargo doc --no-deps` explicitly (the PGLO-1 class)
|
||||
- [ ] Wave 7 decomposition unblocked (recorded in this task's Summary)
|
||||
|
||||
## References
|
||||
|
||||
- docs/plans/implementation.md (§Wave 6, §Review gates — the fleet
|
||||
gate)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
§2/§3
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §2/§4
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §1–§6
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §2
|
||||
- docs/architecture/store-api.md (§Invariants 4 — the fleet form)
|
||||
- docs/architecture/backends-and-dispatch.md (the composite
|
||||
fleet-validity rule)
|
||||
- tasks/review-wave-4.md, tasks/review-wave-5.md (the gate method;
|
||||
the amendments A1/A2 this gate audits closed)
|
||||
- tasks/store-fleet-construction.md, tasks/store-fleet-put.md,
|
||||
tasks/store-fleet-sweep.md, tasks/engine-pg-lo-joint.md,
|
||||
tasks/suite-column-fleet.md (the wave's tasks)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,180 @@
|
||||
---
|
||||
id: store-fleet-construction
|
||||
name: the fleet declaration — the composite fleet-validity predicate at the constructor
|
||||
status: pending
|
||||
depends_on: []
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-6, store, construction, fleet]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Slot the **`fleet: true` constructor declaration** into the
|
||||
fleet-capable constructors (ADR-012 §3; the recorded non-event the
|
||||
wave-4/5 constructor tasks pinned: "the `fleet: true` declaration is
|
||||
wave 6, slotting in without reshaping") — the composite
|
||||
fleet-validity predicate enforced at the constructor, the fleet
|
||||
constructor parameters, and the store's fleet state. Nothing here
|
||||
turns the fleet *mechanism* on (the joint-tx put driver is
|
||||
`store-fleet-put`, the fleet sweep is `store-fleet-sweep`); this task
|
||||
is the admission gate and the state those tasks consume.
|
||||
|
||||
- **The declaration is explicit, never inferred** (ADR-012 §3): a
|
||||
store instance is constructed with fleet mode on or off. Under
|
||||
`fleet: false` (every existing constructor's implicit posture,
|
||||
unchanged): the ADR-005 in-process protocol verbatim regardless of
|
||||
engine. Under `fleet: true`: the composite fleet-validity predicate
|
||||
is enforced at construction, and the fleet machinery activates (the
|
||||
later tasks' wiring).
|
||||
- **The composite fleet-validity predicate** (backends-and-dispatch.md
|
||||
"The composite fleet-validity rule", one sentence, both tiers): a
|
||||
configuration is fleet-valid iff **kv = postgres AND (large = pg-lo
|
||||
on the same pg instance as the kv engine OR large = local on
|
||||
declared shared media OR large = none with the re-routing posture
|
||||
carrying pool large content)** — plus the explicit `fleet: true`
|
||||
declaration. `fleet: true` is a **constructor error** for every
|
||||
non-fleet-valid configuration (ADR-012 §3): any kv engine other than
|
||||
postgres, `mem` anywhere, `local` without the shared-media
|
||||
declaration, `pg-lo` on a different pg instance than the kv engine.
|
||||
The predicate is enforced where the enforceable seam is — the
|
||||
constructor sees only its own node; cross-node truth (is the
|
||||
declared shared media actually shared? is the undeclared instance
|
||||
sharing our pool?) is the deployment's verified invariant
|
||||
(ADR-008 §3's seam; ADR-012 §3's misconfiguration reading — the
|
||||
detection symptom documented, not constructor-provable).
|
||||
- **The fleet constructor parameters** (ADR-008 §2: "fleet constructor
|
||||
parameters, not API"):
|
||||
- **The pin TTL** — default 60 s (`DEFAULT_PIN_TTL_SECS`, already a
|
||||
postgres backend field awaiting exactly this wiring — the wave-4
|
||||
note), constructor-tunable. The store owns the TTL and the TTL/3
|
||||
renewal cadence; the engine's `pin_renew` successor-time body
|
||||
reads the TTL it was constructed with (the wave-4 recorded design
|
||||
point: "constructor-tunable when wave 6's fleet constructor
|
||||
lands").
|
||||
- **The large-tier declaration** for `large = local` under fleet —
|
||||
the operator's explicit shared-media assertion (ADR-008 §3's
|
||||
enforceable-seam bullet: a fleet-mode constructor requires an
|
||||
explicit large-tier declaration — `local` with the assertion,
|
||||
routed (kv-only), or `pg-lo`). **Named design point — the
|
||||
declaration's type shape**: how the assertion rides the
|
||||
constructor signature (a dedicated declaration type the caller
|
||||
must name — e.g. a fleet-opts shape carrying the shared-media
|
||||
assertion — versus a bare flag) — decide consistent with ADR-008
|
||||
§3's "explicit declaration" wording (a named type the caller
|
||||
cannot construct accidentally is the reading the ADR's
|
||||
enforceable-seam posture supports) and record it.
|
||||
- **kv-only re-routing posture**: `kv_only_postgres` under
|
||||
`fleet: true` is valid (large = none; the re-routing posture
|
||||
carries pool large content via the ops surface — wave 7; the
|
||||
client-node shape is this constructor plus that composition,
|
||||
ADR-008 §3). The over-threshold put-time rejection (ADR-012 §6.2,
|
||||
already landed) is unchanged under fleet.
|
||||
- **The pg-lo same-instance clause** (ADR-012 §4 — the checkable
|
||||
clause that makes the joint entry+pin tx expressible): under
|
||||
`fleet: true`, `dual_tier_postgres_pglo`'s two DSN+opts pairs must
|
||||
resolve to the **same pg instance**. Cross-instance is valid only
|
||||
as `fleet: false`. **Named design point — the same-instance
|
||||
probe**: textual DSN comparison is insufficient (aliases, IP vs
|
||||
hostname); decide the server-side check (the instance's
|
||||
`system_identifier` via `pg_control_system()` — PG14+ — compared
|
||||
across one connection per engine, is the natural shape; the
|
||||
harness server's PG 16.15 carries it) and record it. A mismatch is
|
||||
the constructor error.
|
||||
- **The store's fleet state** — what the Store carries once declared:
|
||||
the fleet flag (the put/sweep paths branch on it — never on
|
||||
re-deriving from the engine set), the pin TTL, and the
|
||||
**store-instance id** (`owner` — ADR-011's vocabulary; the pin
|
||||
row's identity half). **Named design point — the owner id's
|
||||
minting**: the instance id's generation (random bytes at
|
||||
construction is the natural shape — unique per instance, stable
|
||||
for the instance's life, carried as the `owner` bytes of every
|
||||
`PinRow` the store mints) and its width/encoding; record it. The
|
||||
id is crate-internal (never public API — ADR-008: fleet parameters,
|
||||
not API).
|
||||
- **The one-window-host rule under fleet** — verify the landed
|
||||
posture composes: every fleet-valid configuration has kv = postgres
|
||||
(SQL-backed), so the durable window hosts on the kv engine and the
|
||||
pg-lo window clauses stay dormant (the landed consolidation shape
|
||||
already carries no `gc_window` in the pg-lo schema). The predicate's
|
||||
enforcement must not disturb the form branch (it keys off the
|
||||
engine representation, never a flag).
|
||||
- **Constructor docs updated** — the "implicitly `fleet: false`" /
|
||||
"wave 6 slots in without reshaping" recordings in the postgres
|
||||
constructors' docs (and the store.rs module doc's fleet-declaration
|
||||
paragraph) updated to the landed state.
|
||||
- **Constructor unit tests** (env-driven, skipping cleanly
|
||||
server-less): every fleet-valid configuration constructs (pg-lo
|
||||
same-instance, local + shared-media declaration, kv-only re-routing)
|
||||
and introspects its fleet state (flag, TTL, owner id); every
|
||||
non-fleet-valid configuration errors (`mem` anywhere, sqlite,
|
||||
`local` without the declaration, pg-lo cross-instance — two
|
||||
distinct schemas on the harness server are the same instance, so
|
||||
the cross-instance negative needs two DSNs that resolve to
|
||||
different instances; if the dev environment has only one, the
|
||||
negative is exercised by pointing one DSN at a non-existent
|
||||
instance and asserting the probe errors closed — a probe failure is
|
||||
a constructor error, never a silent pass); `fleet: false` postures
|
||||
unchanged (the existing constructor tests stay green). **Tests need
|
||||
a live server** — the alkstore harness convention: env-driven
|
||||
(`ALKBLOBS_PG_*`), skip cleanly server-less, acceptance requires
|
||||
green against the harness server (`pglo-poc`, :15432, db `blobs`,
|
||||
`postgres`/`poc` — verified live 2026-10-12).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The fleet declaration slots into the fleet-capable constructors
|
||||
without reshaping them (the recorded non-event lands as
|
||||
recorded); `fleet: false` postures byte-unchanged in behavior
|
||||
- [ ] The composite fleet-validity predicate enforced at the
|
||||
constructor: every fleet-valid configuration constructs; every
|
||||
non-fleet-valid configuration is a constructor error (unit-
|
||||
tested per clause: kv≠postgres, mem anywhere, local without the
|
||||
shared-media declaration, pg-lo cross-instance)
|
||||
- [ ] The pg-lo same-instance probe decided and recorded; a mismatch
|
||||
or a failed probe is a constructor error (never a silent pass)
|
||||
- [ ] The pin TTL is a fleet constructor parameter (default 60 s)
|
||||
wired to the engine's renewal body; the shared-media
|
||||
declaration's type shape decided and recorded
|
||||
- [ ] The store carries its fleet state (flag, TTL, owner id); the
|
||||
owner id's minting decided and recorded (crate-internal)
|
||||
- [ ] The one-window-host posture verified intact under fleet-valid
|
||||
configurations; the form branch still keys off the engine
|
||||
representation
|
||||
- [ ] Constructor docs updated to the landed state (the wave-6
|
||||
forward references resolved)
|
||||
- [ ] `cargo test` (default) green; `cargo test --features postgres`
|
||||
green server-less (skipping) and against the harness server at
|
||||
acceptance; `cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green (default + `--features postgres`);
|
||||
`cargo doc --no-deps` clean
|
||||
|
||||
## References
|
||||
|
||||
- src/store.rs (the constructors and their recorded non-fleet
|
||||
postures; the module doc's fleet-declaration paragraph)
|
||||
- docs/architecture/backends-and-dispatch.md ("The composite
|
||||
fleet-validity rule" — the one-sentence predicate)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§3 (the declaration, never inferred; the constructor-error set), §4
|
||||
(the GC-state host; the pg-lo same-instance clause; the
|
||||
pin-first/publish-second ordering this task's declaration admits)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
§2 (the TTL, fleet constructor parameters), §3 (the enforceable
|
||||
seam; the explicit large-tier declarations)
|
||||
- docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md
|
||||
§4 (the constructor table)
|
||||
- src/postgres.rs (`DEFAULT_PIN_TTL_SECS` — the wave-4 note awaiting
|
||||
this wiring)
|
||||
- tasks/store-postgres-construction.md, tasks/store-pg-lo-construction.md
|
||||
(the recorded slotting-in postures this task lands)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,171 @@
|
||||
---
|
||||
id: store-fleet-put
|
||||
name: the fleet put path — the joint entry+pin tx driver and the pin guard's fleet form
|
||||
status: pending
|
||||
depends_on: [store-fleet-construction, engine-pg-lo-joint]
|
||||
scope: broad
|
||||
risk: high
|
||||
impact: phase
|
||||
level: implementation
|
||||
tags: [wave-6, store, gc, fleet, put]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Route the put paths through the **joint entry+pin transaction** under
|
||||
`fleet: true` (ADR-008 §2; ADR-010 §2; ADR-012 §3/§4) — the store core
|
||||
holds the tx from `begin_joint_tx` to `pin_commit`, engines contribute
|
||||
mechanics, and publish + protection land in one step — plus the pin
|
||||
guard's fleet form: the **TTL/3 renewal duty** on the guard. The
|
||||
non-fleet put paths are byte-unchanged (the ADR-005 in-process
|
||||
protocol verbatim; every wave-2/3/4/5 put test stays green).
|
||||
|
||||
- **The joint-tx driver** (the store core, `src/store.rs`): under
|
||||
fleet mode, a put whose entry lands on the kv engine (postgres)
|
||||
runs `begin_joint_tx` → `joint_put` (the engine's CAS row write
|
||||
inside the held tx) → `pin_commit(tx, [PinRow { key, owner,
|
||||
expiry }])` — the pin row carries the store-instance id
|
||||
(`store-fleet-construction`'s owner id) and `expiry = now + TTL`
|
||||
(the store resolves the expiry; ADR-012 §2's rename — the TTL lives
|
||||
with the store's renewal schedule). The public `put` signature is
|
||||
unchanged; the routing branches on the fleet flag, never on
|
||||
re-deriving from the engine set.
|
||||
- **Amendment A1 — the joint-tx drop discipline** (the wave-4 gate's
|
||||
owner lands here): the driver owes tx termination on **every** path
|
||||
from `begin_joint_tx` onward — the natural `?` propagation between
|
||||
the driver's steps must never return a mid-flight `PgJointTx` to
|
||||
the pool (deadpool's `Fast` recycle never resets transaction
|
||||
state — PG-2's hazard). **Named design point — the termination
|
||||
design**: catch-propagate-terminate (every error path between
|
||||
begin and commit terminates the tx before propagating) per A1's
|
||||
recorded options — an explicit abort verb would need a new ADR (the
|
||||
seam shape is pinned; `pin_commit`'s terminal-consume reading
|
||||
stays). Record the design; the zero-residue atomicity is already
|
||||
the engine clause's contract (wave 4) — the driver's duty is never
|
||||
abandoning the tx mid-flight.
|
||||
- **The pin guard's fleet form** (ADR-008 §2: "the pin guard renews
|
||||
its row every TTL/3"): the `Pin` guard returned by a fleet put
|
||||
carries the **renewal duty** — while the guard lives, its pin row's
|
||||
expiry moves forward every TTL/3 (via the engine's `pin_renew(key,
|
||||
owner)` — the never-resurrect semantics: an expired row is never
|
||||
revived; the guard's renewal failing after expiry is the
|
||||
delete-then-recover backstop's territory, ADR-008's expiry
|
||||
semantics verbatim). **Named design points — the renewal
|
||||
lifecycle**: the duty's mechanism (a per-guard tokio task on a
|
||||
TTL/3 interval vs a shared renewal sweep keyed off the guard
|
||||
registry — decide, record), the abort-on-drop discipline (a
|
||||
dropped/converted/batch-contributed guard stops its renewal — no
|
||||
leaked tasks, no renewal past the guard's life), and the failure
|
||||
posture (a failed renewal is logged-and-continued or silent — the
|
||||
row's expiry is the protection, a missed renewal degrades to the
|
||||
expiry semantics, never an error on the put path). The guard's
|
||||
in-process refcount machinery continues to function (instance-local
|
||||
bookkeeping — the batch's one-step release, the ordered-step
|
||||
conversion); the row is the cross-instance protection. Batch puts
|
||||
(`put_in_batch`): each member's joint tx commits its row; the
|
||||
batch's one-step release ends the members' renewals (the rows
|
||||
expire and the sweeper's maintenance step reaps them —
|
||||
`store-fleet-sweep`).
|
||||
- **The dedup re-put under fleet**: CAS dedup means no new entry —
|
||||
the joint tx commits the pin row alone (an empty `joint_put`
|
||||
contribution is the engine's recorded CAS form) and the row
|
||||
upsert-renews `(key, owner)` (the wave-4 recorded shape). A
|
||||
different instance's re-put is a different pin row — protection
|
||||
composes by row (ADR-010 §2's refcount-in-rows reading).
|
||||
- **`large = local` on declared shared media — pin-first,
|
||||
publish-second** (ADR-012 §4): the joint tx is unachievable across
|
||||
media, so the ordering is pinned: the pin row commits on the kv
|
||||
engine (a joint tx carrying only the pin row — `begin_joint_tx` →
|
||||
`pin_commit` without `joint_put`) **before** the stage file's
|
||||
commit-rename publishes the entry. The invariant survives (an entry
|
||||
becomes visible only after its pin row is durable — sweep
|
||||
arbitration can never observe a visible-unpinned entry); the crash
|
||||
window moves to the harmless side (an orphan pin row, reaped by the
|
||||
expiry machinery — no new recovery mechanism). The reverse ordering
|
||||
is rejected (ADR-012 §4) — do not implement it.
|
||||
- **The pg-lo consolidation put** (kv = postgres + large = pg-lo,
|
||||
same instance): the large entry's content reaches the store-held tx
|
||||
through the pg-lo engine's joint contribution (`engine-pg-lo-joint`)
|
||||
— LO writes + companion-row insert inside the tx, the pin-row
|
||||
insert + commit via `pin_commit`. One tx over the shared instance;
|
||||
the dispatch threshold routes to the joint form exactly as the
|
||||
non-fleet path routes to the staged form.
|
||||
- **Unit tests** (env-driven, skipping cleanly server-less; the
|
||||
harness server `pglo-poc`, :15432, db `blobs`, `postgres`/`poc` —
|
||||
verified live 2026-10-12): the kv joint put publishes entry + pin
|
||||
row atomically (an independent connection sees both or neither);
|
||||
the forced-failure zero-residue form (A1's discipline exercised —
|
||||
a mid-driver failure leaves no dirty pooled session); the guard's
|
||||
renewal moves the row's expiry (a short test TTL; the renewal
|
||||
cadence observed); the never-resurrect form (an expired row is not
|
||||
renewed back to life); the batch form (members' rows commit; batch
|
||||
release ends renewals); the dedup re-put's upsert-renewal; the
|
||||
local-shared-media pin-first ordering (the pin row is durable
|
||||
before the committed file exists); the pg-lo consolidation
|
||||
round-trip through the public `put` (both tiers, one tx — the
|
||||
byte-exact large-entry form); the non-fleet paths unchanged (the
|
||||
existing suites green).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Under `fleet: true`, kv-tier puts route through the joint tx
|
||||
(entry + pin row in one step; an independent connection never
|
||||
observes one without the other); under `fleet: false` the put
|
||||
paths are byte-unchanged (all existing suites green)
|
||||
- [ ] Amendment A1 landed: the driver terminates the tx on every path
|
||||
from `begin_joint_tx` onward (the termination design recorded;
|
||||
a forced mid-driver failure leaves zero residue and returns no
|
||||
dirty session to the pool)
|
||||
- [ ] The pin guard's fleet form: renewal every TTL/3 while the guard
|
||||
lives; drop/conversion/batch-contribution stops the renewal
|
||||
with no leaked tasks; a failed renewal never resurrects an
|
||||
expired row (the lifecycle design points decided and recorded)
|
||||
- [ ] The dedup re-put commits the pin row alone (CAS; upsert-
|
||||
renewal); a different instance's re-put composes protection by
|
||||
row
|
||||
- [ ] `large = local` fleet puts: pin-row-first, publish-second
|
||||
(contract-tested — the pin row is durable before the committed
|
||||
file exists; the orphan-pin crash window is the expiry
|
||||
machinery's)
|
||||
- [ ] The pg-lo consolidation put round-trips through the public
|
||||
`put` (one tx over the shared instance; byte-exact large-entry
|
||||
form)
|
||||
- [ ] `cargo test` (default) green; `cargo test --features postgres`
|
||||
and `--features "postgres,pg-lo"` green server-less (skipping)
|
||||
and against the harness server at acceptance;
|
||||
`cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green (default + the postgres/pg-lo
|
||||
variants); `cargo doc --no-deps` clean
|
||||
|
||||
## References
|
||||
|
||||
- src/store.rs (the put seam's two internal paths; the fleet
|
||||
declaration's flag from `store-fleet-construction`)
|
||||
- src/gc.rs (`EngineFleetState` — the clauses the driver calls;
|
||||
`PinRow`; the ownership rule)
|
||||
- src/postgres.rs (`PgJointTx` — PG-2's hazard doc; the wave-4
|
||||
joint-clause bodies)
|
||||
- src/pg_lo.rs (the joint contribution from `engine-pg-lo-joint`)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
§2 (pins are rows; the TTL/3 renewal; the expiry semantics)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
|
||||
§2 (the store holds the tx; the engine contributes mechanics), §4
|
||||
(the guard-drops-at-response-time reading — the ops edition; the
|
||||
public-API guard lives as long as the caller holds it)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§3 (activation), §4 (the host; pin-first/publish-second for
|
||||
large=local; the one-tx consolidation)
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
|
||||
§1 (the renewal worker's cadence is embedder composition above the
|
||||
facade — the guard's own TTL/3 duty is the in-crate half)
|
||||
- tasks/review-wave-4.md (amendment A1 — this task is its owner)
|
||||
- tasks/store-fleet-construction.md, tasks/engine-pg-lo-joint.md (the
|
||||
declaration and the contribution this drives)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,189 @@
|
||||
---
|
||||
id: store-fleet-sweep
|
||||
name: the fleet GC protocol — SQL sweep lease, SQL-level arbitration, pin reaping, the fleet liveness verb
|
||||
status: pending
|
||||
depends_on: [store-fleet-construction]
|
||||
scope: broad
|
||||
risk: high
|
||||
impact: phase
|
||||
level: implementation
|
||||
tags: [wave-6, store, gc, fleet, sweep]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Grow the sweep and the delete paths into their **fleet forms** under
|
||||
`fleet: true` (ADR-008 §2; ADR-010 §2; ADR-012 §1/§5/§6.6/§6.7;
|
||||
ADR-013 §2) — the architecture's most intricate state machine extended
|
||||
across instances, its invariants unchanged (a visible pool entry is
|
||||
never deleted while liveness protects it; an in-flight put is never
|
||||
deleted by a sweep started before it committed). The non-fleet forms
|
||||
(inline + durable, single-node) are byte-unchanged.
|
||||
|
||||
- **The sweep lease** (ADR-013 §2's named exclusive semantics; the
|
||||
shipped provider is the wave-4 advisory-lock clause): under fleet
|
||||
mode the sweep acquires `sweep_lock_acquire()` on the fleet host
|
||||
before sweeping and holds it for the whole sweep; contention is the
|
||||
`GcAbortCause::SweeperLock` report abort (`Ok(SweepReport)`, nothing
|
||||
deleted — ADR-012 §1); holdership ends at release, owner death, or
|
||||
session loss (the clause's session-scoped semantics, wave 4). The
|
||||
in-process single-flight flag composes (both held — the instance-
|
||||
local guard stays cheap and uniform). **Named design point — the
|
||||
lease's hold across the sweep body**: the lease's held checkout
|
||||
must not deadlock the sweep's own engine work (the wave-4 lease
|
||||
holds one pooled connection for its duration; the sweep's other
|
||||
clauses ride further checkouts — verify the pool sizing composes
|
||||
and record the posture).
|
||||
- **SQL-level delete arbitration** (ADR-008 §2): under fleet mode the
|
||||
delete-time decision is the seam's `arbitrate(key)` — the
|
||||
`SELECT … FOR UPDATE` finger-lock folding un-expired pin rows +
|
||||
registered liveness tables into one answer — on **both** the direct
|
||||
`delete()` path (protected → `Error::GcRefuse`, the facade owns the
|
||||
refusal) and the durable window's execution-time re-arbitration.
|
||||
**Named design point — the two arbitration domains' composition**:
|
||||
the in-process `GcHome` machinery (markers, in-process pins,
|
||||
callbacks, in-process sources) and the SQL `arbitrate` both run —
|
||||
the in-process domain guards this instance's in-flight state (the
|
||||
marker mechanism closes the local sweep-vs-put race without engine
|
||||
I/O under the lock), the SQL domain is the cross-instance
|
||||
authority; decide the composition order (in-process first — cheap,
|
||||
local; SQL second — fleet-authoritative) and record it. The
|
||||
engine-side `pin_check` classification stays available to the store
|
||||
core where a pins-only answer suffices (its consumer beyond
|
||||
`arbitrate` may be wave 7's ops renewal probe — do not force it
|
||||
here).
|
||||
- **The expired-pin reaping maintenance step** (ADR-008 §2's GC-of-GC
|
||||
cleanup: "an expired pin is abandoned … and is reaped by the
|
||||
sweeper's maintenance step"): the fleet sweep gains the maintenance
|
||||
step deleting expired pin rows. **Named design point — the clause
|
||||
home**: `EngineFleetState` has no reap clause; the seam is the
|
||||
named home for fleet-clause growth (ADR-010 §2: "If fleet clauses
|
||||
ever grow, they grow here") — add the clause body on the postgres
|
||||
host (e.g. `pin_reap(now) → count`, one statement, contract-tested
|
||||
per the applicability row) rather than any raw-SQL side-channel
|
||||
(rejected by ADR-010 §2). The step runs under the lease, before or
|
||||
alongside the overdue execution (record the ordering).
|
||||
- **The fleet liveness registration verb** (ADR-012 §6.6): "the same
|
||||
verb with a fleet-shaped argument" — the facade's
|
||||
`register_liveness_source` gains its fleet form, registering a
|
||||
**named table reference** in the shared engine (the embedder's own
|
||||
table per ADR-008 §2; the store core reads it, never owns it — the
|
||||
schema-ownership rule). The engine-side registration landed wave 4
|
||||
(`PostgresBackend::register_liveness_source` — quoted-identifier
|
||||
boundary, idempotent re-registration, malformed refs error). **Named
|
||||
design points — the verb's shape**: how the fleet form rides the
|
||||
public verb (a fleet-shaped argument vs a sibling method — decide
|
||||
consistent with §6.6's "same verb" wording and the existing
|
||||
signature's infallibility: the engine-side registration is
|
||||
fallible, so the fleet form's error surface needs deciding), and
|
||||
whether in-process `LivenessSource` registrations remain legal
|
||||
under fleet mode (ADR-008: in-process callback views are not a
|
||||
*fleet* liveness form — an instance-local registration's protection
|
||||
is instance-local only; decide allow-and-document vs reject and
|
||||
record it). Deregistration composes (the correction flow, ADR-012
|
||||
§6.4).
|
||||
- **The protect callback's fleet posture** (ADR-008 §2): single-node-
|
||||
only — under fleet mode the callback remains consultable by *this*
|
||||
instance's sweeps (its protection is instance-local; the embedder's
|
||||
fleet-wide protection materializes into its liveness table or pin
|
||||
rows before invoking sweep). Record the posture in the verb docs;
|
||||
no new machinery.
|
||||
- **The fleet sweep body** (per sweep, under the lease): the protect
|
||||
consult (instance-local, abort `ProtectFailed` precedes everything)
|
||||
→ the maintenance step (expired-pin reaping) → **execute overdue**
|
||||
(`window_take` on the postgres host — the one-window-host rule
|
||||
holds: every fleet-valid configuration's window hosts on the kv
|
||||
engine; each overdue candidate re-arbitrated at execution time via
|
||||
the SQL arbitration, deleted where it lives — the fall-through
|
||||
delete across both tiers) → **mark** (the pool enumerated; the live
|
||||
set from the in-process sources — the SQL liveness tables are
|
||||
consulted at arbitration time, not enumerated into the mark set) →
|
||||
**stage** at `visible_after = mark_end + window` → release the
|
||||
lease. Report counts exact relative to the schedule (the durable
|
||||
form's reading carries: staged-this-sweep vs executed-this-sweep are
|
||||
different sets).
|
||||
- **The sweep-vs-put race under fleet, closed by the row**: a fleet
|
||||
put's pin row commits atomically with its entry (the joint tx,
|
||||
`store-fleet-put`), so a sweep's window execution can never observe
|
||||
a visible-unpinned entry; a put landing during the window delay has
|
||||
its row committed before its entry is visible and the execution-time
|
||||
SQL arbitration cancels the candidate. Unit-test the race across
|
||||
the delay in its fleet form (the wave-3 durable-form race test's
|
||||
fleet edition).
|
||||
- **Unit tests** (env-driven, skipping cleanly server-less; the
|
||||
harness server `pglo-poc`, :15432, db `blobs`, `postgres`/`poc` —
|
||||
verified live 2026-10-12): the lease's fleet form (contention
|
||||
between two instances of one store shape → the second aborts
|
||||
`SweeperLock` with nothing deleted; release/loss reaping); SQL
|
||||
arbitration on the direct-delete path (a pin row from a *foreign*
|
||||
owner id refuses the delete — the cross-instance protection proof;
|
||||
`GcRefuse`); the maintenance step reaps exactly the expired set
|
||||
(un-expired rows survive); the fleet liveness verb registers a
|
||||
test-owned embedder table and its rows protect (the schema-
|
||||
ownership rule — the engine never creates the table); the
|
||||
two-instance exact-count sweep shape (instance A pins, instance B
|
||||
sweeps — the pinned entry survives; unpinned twins delete); the
|
||||
race across the delay; the non-fleet forms unchanged (all existing
|
||||
suites green).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Under `fleet: true`, the sweep acquires the SQL lease (the
|
||||
ADR-013 §2 semantics: atomic test-and-set, one sweeper per
|
||||
pool across instances, contention = `SweeperLock` report abort
|
||||
with nothing deleted); the in-process flag composes; under
|
||||
`fleet: false` the sweep forms are byte-unchanged
|
||||
- [ ] Delete-time arbitration is SQL-level under fleet mode on both
|
||||
the direct path (`GcRefuse`) and the window execution
|
||||
(execution-time re-arbitration); a foreign owner's un-expired
|
||||
pin row protects (cross-instance proof); the two-domain
|
||||
composition decided and recorded
|
||||
- [ ] The expired-pin reaping maintenance step landed as a new
|
||||
`EngineFleetState` clause on the postgres host (the seam is the
|
||||
named home; no raw-SQL side-channel), contract-tested, run
|
||||
under the lease
|
||||
- [ ] The fleet liveness registration verb landed (the named table
|
||||
reference form; the verb-shape and in-process-source design
|
||||
points decided and recorded; the schema-ownership rule holds)
|
||||
- [ ] The protect callback's fleet posture recorded (instance-local;
|
||||
materialization is the embedder's fleet-wide duty)
|
||||
- [ ] The fleet race property unit-tested across the window delay (a
|
||||
put landing during the delay survives via the row + the
|
||||
execution-time SQL re-arbitration)
|
||||
- [ ] `cargo test` (default) green; `cargo test --features postgres`
|
||||
green server-less (skipping) and against the harness server at
|
||||
acceptance; `cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green (default + `--features postgres`);
|
||||
`cargo doc --no-deps` clean
|
||||
|
||||
## References
|
||||
|
||||
- src/store.rs (the sweep's two forms; `arbitrated_delete`; the
|
||||
fleet flag from `store-fleet-construction`)
|
||||
- src/gc.rs (`GcHome` — the in-process domain; `EngineFleetState` —
|
||||
the clauses; the seam-as-named-home note)
|
||||
- src/postgres.rs (the lease, `arbitrate`, `pin_check` — the wave-4
|
||||
clause bodies this drives; the liveness registration)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
§2 (the fleet mechanism: SQL arbitration, one sweeper per pool, the
|
||||
maintenance step, the protect callback's single-node-only posture)
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
|
||||
§2 (the named exclusive sweep lease — the semantics this implements)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§1 (abort-as-data), §5 (the one-window-host rule), §6.4 (the full
|
||||
protection set), §6.6 (the fleet liveness verb's API home), §6.7
|
||||
(the executor is the next sweep)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
|
||||
§2 (the seam clauses; the raw-SQL rejection)
|
||||
- tasks/store-sweep-durable-window.md (the durable form's shape — the
|
||||
single-node structural twin)
|
||||
- tasks/store-fleet-put.md (the joint puts whose rows this sweep
|
||||
arbitrates against)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,150 @@
|
||||
---
|
||||
id: suite-column-fleet
|
||||
name: Conformance suite — the fleet column (exact-count fleet sweep outcomes under concurrency)
|
||||
status: pending
|
||||
depends_on: [store-fleet-put, store-fleet-sweep]
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: phase
|
||||
level: implementation
|
||||
tags: [wave-6, suite, fleet, postgres]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Grow the **fleet column** — the suite's proof that the architecture's
|
||||
most intricate state machine holds under concurrency (the wave's
|
||||
gate: "exact-count fleet sweep outcomes against live postgres",
|
||||
docs/plans/implementation.md §Wave 6). The column runs under
|
||||
`--features postgres` (fleet mode requires kv = postgres — the
|
||||
composite predicate), rides the SKIP posture (server-less runs skip
|
||||
cleanly; acceptance requires green **against the server**), and
|
||||
extends the factory contract to the fleet shape: **two store
|
||||
instances over one shared pool**.
|
||||
|
||||
- **The fleet factory** (`tests/conformance/fleet_column.rs`): the
|
||||
postgres column's schema-per-open pattern adapted — a fleet factory
|
||||
mints **one schema** and vends **two (or more) `Store` instances
|
||||
over it**, each `fleet: true`, each with its own owner id (the
|
||||
instances share the pool; the pin rows compose by owner — the
|
||||
fleet's actual definition, ADR-011/ADR-012 §3). Teardown drops
|
||||
exactly the minted schemas (`DROP SCHEMA … CASCADE`, the postgres
|
||||
column's registry pattern — panicking-row-safe, idempotent,
|
||||
owned). The test tuning: window delay `0` (execute-on-next-sweep)
|
||||
and a **short test TTL** (the fleet constructor parameter — the
|
||||
renewal/expiry rows need observable cadences; the TTL is a
|
||||
constructor parameter, not API, so the column declares its tuning).
|
||||
A kv-only fleet factory (the re-routing posture's client shape) and
|
||||
a consolidation fleet factory (kv = postgres + large = pg-lo, same
|
||||
instance) join as the column's second/third shapes.
|
||||
- **The fleet-validity rows** (the predicate at the constructor):
|
||||
every fleet-valid configuration constructs; every non-fleet-valid
|
||||
configuration is a constructor error (kv ≠ postgres, `mem`
|
||||
anywhere, `local` without the shared-media declaration, pg-lo
|
||||
cross-instance) — the predicate's clause-per-row form.
|
||||
- **The joint entry+pin tx rows** (through the public `put`, not the
|
||||
seam): entry + pin row commit together (an independent connection
|
||||
sees both or neither); the dedup re-put upsert-renews the row; a
|
||||
second instance's re-put composes a second row (protection by
|
||||
row); the pg-lo consolidation round-trip (one tx over the shared
|
||||
instance, byte-exact large-entry form).
|
||||
- **The TTL/renewal rows**: the guard's TTL/3 renewal moves the row's
|
||||
expiry (the short test TTL makes the cadence observable); a guard
|
||||
dropped past expiry never resurrects its row (the never-resurrect
|
||||
semantics through the facade); a paused holder (renewal stopped)
|
||||
loses protection at expiry — the delete-then-recover backstop (a
|
||||
byte-identical re-put recovers).
|
||||
- **The exact-count fleet sweep rows** (the wave's gate — INV-4's
|
||||
engine-gated fleet form, store-api.md §Invariants 4: "fleet
|
||||
postures assert it under DB-backed pins + the advisory-locked
|
||||
sweep, with the exact-count sweep-outcome shape run against
|
||||
dockerized postgres"):
|
||||
- the **two-instance exact-count shape**: instance A pins an entry
|
||||
(its row), instance B sweeps — the pinned entry survives, the
|
||||
unpinned twins delete, counts exact against the staged schedule;
|
||||
- the **lease contention row**: two instances sweep concurrently —
|
||||
exactly one sweeps, the other aborts `SweeperLock` as report
|
||||
data with nothing deleted;
|
||||
- the **cross-instance arbitration row**: a foreign owner's
|
||||
un-expired pin row refuses instance B's direct delete
|
||||
(`GcRefuse`); after the row expires (renewal stopped, short TTL),
|
||||
the delete proceeds;
|
||||
- the **race across the window delay** in its fleet form: a put
|
||||
landing during the delay (its row committed before its entry is
|
||||
visible) survives the next sweep's execution-time SQL
|
||||
re-arbitration;
|
||||
- the **maintenance-step row**: expired rows reaped exactly (the
|
||||
GC-of-GC cleanup), un-expired rows survive;
|
||||
- the **fleet liveness row**: a test-owned embedder table's rows
|
||||
protect through the fleet registration verb (the schema-ownership
|
||||
rule asserted — the engine never created the table).
|
||||
- **The kv-only re-routing row**: the fleet kv-only client shape
|
||||
rejects over-threshold puts at put time (ADR-012 §6.2's machinery,
|
||||
fleet edition) — the re-routing posture's store-side half.
|
||||
- **README amendments** (the convention's normative test-side home):
|
||||
the fleet rows table in §Columns beyond the reference set; the
|
||||
column plan's SKIP-posture note naming the fleet column; current
|
||||
state.
|
||||
- **Connection settings ride the environment** — the established
|
||||
harness convention: `ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`, the
|
||||
reachability probe gating the rows. **The harness server**:
|
||||
dockerized `pglo-poc` (`postgres:16-alpine`) on :15432, db `blobs`,
|
||||
user/password `postgres`/`poc` — verified live 2026-10-12; the
|
||||
container is long-running in the dev environment. Concurrency rows
|
||||
are deterministic by construction (the lease/arbitration outcomes
|
||||
are decided by the server, not by timing) — where a row needs
|
||||
ordering, the window delay and the TTL tunings provide it; no
|
||||
sleep-based assertions.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The fleet column green **against the harness server**
|
||||
(`pglo-poc`, :15432): the fleet-validity rows, the joint-tx
|
||||
rows, the TTL/renewal rows, the exact-count fleet sweep rows
|
||||
(two-instance survival, lease contention, cross-instance
|
||||
arbitration, the race across the delay, the maintenance step,
|
||||
the fleet liveness row), and the kv-only re-routing row — every
|
||||
row ADR-stamped and traceable to the backlog
|
||||
- [ ] The fleet factory contract holds: one schema, two+ instances
|
||||
over it (distinct owner ids), teardown drops exactly the minted
|
||||
schemas, panicking-row-safe — structurally exercised
|
||||
- [ ] Server-less runs skip cleanly (the reachability probe; the
|
||||
default-build gates stay green with the column skipped)
|
||||
- [ ] README amendments recorded (the fleet rows table; current
|
||||
state)
|
||||
- [ ] `cargo test` (default) green — the column compiles out;
|
||||
`cargo test --features postgres` green server-less (skipping)
|
||||
and green against the server at acceptance;
|
||||
`cargo test --all-features` green (the consolidation factory's
|
||||
home); `cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green (default + the postgres/pg-lo
|
||||
variants + `--all-features`)
|
||||
|
||||
## References
|
||||
|
||||
- tests/conformance/README.md (the conventions this grows against;
|
||||
§The factory contract, §The SKIP posture, §The schedule-relative
|
||||
reading, §Columns beyond the reference set)
|
||||
- tests/conformance/postgres_column.rs (the SQL-backed column pattern
|
||||
— the schema-per-open factory this extends to the fleet shape)
|
||||
- tests/conformance/factory.rs (the factory contract)
|
||||
- docs/architecture/store-api.md (§Invariants 4 — the engine-gated
|
||||
fleet form this column asserts)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
§2 (the fleet mechanism the rows assert)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§3 (the declaration), §4 (the host), §6.6 (the fleet liveness verb)
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
|
||||
§2 (the lease semantics the contention row asserts)
|
||||
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
|
||||
§5 (the exact-count sweep-outcome gate against dockerized postgres)
|
||||
- tasks/store-fleet-put.md, tasks/store-fleet-sweep.md,
|
||||
tasks/store-fleet-construction.md (the machinery the rows exercise)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
Reference in new issue
Block a user