glm-5.3-flash a48c12745a pg-fix-scheduler-tick-seam — the pg scheduler's transient-fault tick seam lands, retiring review 003 Finding 1 (MEDIUM: the tick retry loop was dead code under test — the last uncovered hardening arm of the wave-4 fix batch, the exact class review 002's live bugs came from). The seam follows the sqlite-commit-error-arm pattern of record: a cfg(test) per-store counting fault arm (crate::seam::tick_fault — Flag = Arc<AtomicU64>, born disarmed at 0, arm(n) adds, take() consumes one via CAS try_update; a counting arm because a boolean cannot reach the exhaustion arm — arm(TICK_RETRIES+1) faults the loop's whole budget while arm(1) leaves the later ticks real) feeding a fabricated pool/database-shaped opaque Database error into the REAL retry loop — both tests drive the full run_schedules → tick_with_retries → tick_once path end-to-end, no loop replication. Plumbing per the house pattern: cfg-gated tick_fault field on EngineCtx and PgStore (engine_ctx() carries the clone; production builds never materialize it), cfg-gated fault param on tick_once (the seam check sits before the real attempt — one attempt faults, never ticks) and tick_with_retries, cfg-branched attempt calls; PgStore::arm_tick_fault(faults) arming surface + tick_fault_flag() observation surface (a Flag clone so the consumption pin survives the store moving into a runner task). Two arms pinned: (a) one_faulted_tick_is_retried_and_the_job_fires — fault consumed on attempt 1, the 250 ms backoff sleeps, the retried attempt fires the backdated due boundary into the queue, exactly-one consumption, the runner survives the hiccup to a clean stop Ok(()); (b) tick_retry_exhaustion_exits_typed_and_leaves_the_lock_to_lapse — the exhaustion exits typed (opaque Database, never LeadershipLost; the tick-phase context rides the source chain with the fabricated fault at its bottom; wall-clock ≥ 1.7 s proving the real backoff window), the leadership lock row is left (not released) to lapse at its TTL (present, expiry bounded by the run's TTL horizon), exact consumption, and the store remains fully usable after — the "runner survives" pin as the module docs' contract states it. Replay-proofed live: with the seam's consumption disabled the exhaustion test cannot complete (the runner loops forever — no fault ever enters the budget; the mutated run's timeout is itself the proof); the test is then bounded (20 s) so a future regression of this shape fails, not hangs. LEADER_TTL_S widened private → pub(crate) for the lock-left-to-TTL expiry pin. Verified: full pg column live vs the harness (124 lib + 25 suite + 9 schema; new tests green solo ×3); cargo test --workspace green server-less, skips clean (core 25 + harness 3, sqlite 191 + 25); clippy --all-targets -D warnings clean; fmt clean; cargo-llvm-cov confirms the Finding-1 sites (the retry/backoff/exhaustion-arm region + the tick_once seam arm) carry zero missed lines; taskgraph validate green (61 tasks)

2026-10-10 11:00:26 +00:00

..

store

pg-fix-scheduler-tick-seam — the pg scheduler's transient-fault tick seam lands, retiring review 003 Finding 1 (MEDIUM: the tick retry loop was dead code under test — the last uncovered hardening arm of the wave-4 fix batch, the exact class review 002's live bugs came from). The seam follows the sqlite-commit-error-arm pattern of record: a cfg(test) per-store counting fault arm (crate::seam::tick_fault — Flag = Arc<AtomicU64>, born disarmed at 0, arm(n) adds, take() consumes one via CAS try_update; a counting arm because a boolean cannot reach the exhaustion arm — arm(TICK_RETRIES+1) faults the loop's whole budget while arm(1) leaves the later ticks real) feeding a fabricated pool/database-shaped opaque Database error into the REAL retry loop — both tests drive the full run_schedules → tick_with_retries → tick_once path end-to-end, no loop replication. Plumbing per the house pattern: cfg-gated tick_fault field on EngineCtx and PgStore (engine_ctx() carries the clone; production builds never materialize it), cfg-gated fault param on tick_once (the seam check sits before the real attempt — one attempt faults, never ticks) and tick_with_retries, cfg-branched attempt calls; PgStore::arm_tick_fault(faults) arming surface + tick_fault_flag() observation surface (a Flag clone so the consumption pin survives the store moving into a runner task). Two arms pinned: (a) one_faulted_tick_is_retried_and_the_job_fires — fault consumed on attempt 1, the 250 ms backoff sleeps, the retried attempt fires the backdated due boundary into the queue, exactly-one consumption, the runner survives the hiccup to a clean stop Ok(()); (b) tick_retry_exhaustion_exits_typed_and_leaves_the_lock_to_lapse — the exhaustion exits typed (opaque Database, never LeadershipLost; the tick-phase context rides the source chain with the fabricated fault at its bottom; wall-clock ≥ 1.7 s proving the real backoff window), the leadership lock row is left (not released) to lapse at its TTL (present, expiry bounded by the run's TTL horizon), exact consumption, and the store remains fully usable after — the "runner survives" pin as the module docs' contract states it. Replay-proofed live: with the seam's consumption disabled the exhaustion test cannot complete (the runner loops forever — no fault ever enters the budget; the mutated run's timeout is itself the proof); the test is then bounded (20 s) so a future regression of this shape fails, not hangs. LEADER_TTL_S widened private → pub(crate) for the lock-left-to-TTL expiry pin. Verified: full pg column live vs the harness (124 lib + 25 suite + 9 schema; new tests green solo ×3); cargo test --workspace green server-less, skips clean (core 25 + harness 3, sqlite 191 + 25); clippy --all-targets -D warnings clean; fmt clean; cargo-llvm-cov confirms the Finding-1 sites (the retry/backoff/exhaustion-arm region + the tick_once seam arm) carry zero missed lines; taskgraph validate green (61 tasks)

2026-10-10 11:00:26 +00:00

forwarder.rs

docs alignment: v1 TLS posture owned by deployment.md, QueueOpts numeric consumer-obligation notes (task pg-fix-docs-alignment, review 002 Finding 6 remainder)

2026-10-10 05:08:27 +00:00

lib.rs

Postgres engine integration: StoreFactory (fresh schema per open, owned idempotent CASCADE teardown, isolation/idempotence pinned), the engine's backlog column (all ten rows green against the harness server — exemplar verified, the three ADR-023 rows verified not rewritten, the five engine-scoped rows, the new pg-arm PayloadTooLarge row discharging the SQLite task's deferred adoption), test-observation accessors cfg(test)-gated with the unused PgStore::new cut, stale stub-era doc text removed, lib docs stating the finished-engine posture (task pg-engine-integration)

2026-10-09 10:22:25 +00:00

lock.rs

Postgres engine: scheduler + outbox — schedule (validation triad, the pg-owned @every-only parser, upsert over the schedule table), unschedule, run_schedules (leadership via the engine's lock machinery on __alkstore_scheduler with a per-instance owner token, in-sleep lease renewals, the row-locked FOR UPDATE tick in one pool tx — fire enqueues + boundary advance + soonest read commit together — the 64-boundary catch-up cap with skip-forward, clean stop / Err(LeadershipLost) arms), outbox (validated constructor, enqueue into the derived __alkstore_outbox:{name} with the 60/5/5 stamps + max_attempts override, run_once ack/retry-curve/false over the ordinary claim machinery, no engine-issued heartbeat) (task pg-engine-scheduler-outbox)

2026-10-09 09:53:48 +00:00

notify.rs

pg queue/tx/notify dedupe + doc truth-telling: job_from_row has one owner (queue.rs, tx.rs imports it) and get_job_tx reuses live_columns()/dead_columns() instead of inlining the 18-column lists — the contract-pinned Job shape now has exactly one decode owner (ADR-012 §2); sweep_expired's doc states the moved + retention-deleted sum in the in_tx multi-statement frame (the sum the SQLite twin returns — the doc was the only liar); notify.rs's closed-store listen error routes through the shared database_error helper; bridge_capacity() is a const with its rationale doc carried. No behavior change — identical SQL strings, error shapes, and capacity. (task pg-fix-dedupe-cleanup, review 002 minor notes + Finding 6 queue bullet)

2026-10-10 05:04:09 +00:00

opts.rs

docs alignment: v1 TLS posture owned by deployment.md, QueueOpts numeric consumer-obligation notes (task pg-fix-docs-alignment, review 002 Finding 6 remainder)

2026-10-10 05:08:27 +00:00

queue.rs

pg queue/tx/notify dedupe + doc truth-telling: job_from_row has one owner (queue.rs, tx.rs imports it) and get_job_tx reuses live_columns()/dead_columns() instead of inlining the 18-column lists — the contract-pinned Job shape now has exactly one decode owner (ADR-012 §2); sweep_expired's doc states the moved + retention-deleted sum in the in_tx multi-statement frame (the sum the SQLite twin returns — the doc was the only liar); notify.rs's closed-store listen error routes through the shared database_error helper; bridge_capacity() is a const with its rationale doc carried. No behavior change — identical SQL strings, error shapes, and capacity. (task pg-fix-dedupe-cleanup, review 002 minor notes + Finding 6 queue bullet)

2026-10-10 05:04:09 +00:00

resolution.rs

Postgres engine: transactional seam — begin_tx, PgTxHandle, all eleven *_tx methods with drop=rollback detached teardown, probe-pinned unknowable-state discard arms, engine-side resolution arithmetic (task pg-engine-seam-tx)

2026-10-09 06:50:53 +00:00

scheduler.rs

pg-fix-scheduler-tick-seam — the pg scheduler's transient-fault tick seam lands, retiring review 003 Finding 1 (MEDIUM: the tick retry loop was dead code under test — the last uncovered hardening arm of the wave-4 fix batch, the exact class review 002's live bugs came from). The seam follows the sqlite-commit-error-arm pattern of record: a cfg(test) per-store counting fault arm (crate::seam::tick_fault — Flag = Arc<AtomicU64>, born disarmed at 0, arm(n) adds, take() consumes one via CAS try_update; a counting arm because a boolean cannot reach the exhaustion arm — arm(TICK_RETRIES+1) faults the loop's whole budget while arm(1) leaves the later ticks real) feeding a fabricated pool/database-shaped opaque Database error into the REAL retry loop — both tests drive the full run_schedules → tick_with_retries → tick_once path end-to-end, no loop replication. Plumbing per the house pattern: cfg-gated tick_fault field on EngineCtx and PgStore (engine_ctx() carries the clone; production builds never materialize it), cfg-gated fault param on tick_once (the seam check sits before the real attempt — one attempt faults, never ticks) and tick_with_retries, cfg-branched attempt calls; PgStore::arm_tick_fault(faults) arming surface + tick_fault_flag() observation surface (a Flag clone so the consumption pin survives the store moving into a runner task). Two arms pinned: (a) one_faulted_tick_is_retried_and_the_job_fires — fault consumed on attempt 1, the 250 ms backoff sleeps, the retried attempt fires the backdated due boundary into the queue, exactly-one consumption, the runner survives the hiccup to a clean stop Ok(()); (b) tick_retry_exhaustion_exits_typed_and_leaves_the_lock_to_lapse — the exhaustion exits typed (opaque Database, never LeadershipLost; the tick-phase context rides the source chain with the fabricated fault at its bottom; wall-clock ≥ 1.7 s proving the real backoff window), the leadership lock row is left (not released) to lapse at its TTL (present, expiry bounded by the run's TTL horizon), exact consumption, and the store remains fully usable after — the "runner survives" pin as the module docs' contract states it. Replay-proofed live: with the seam's consumption disabled the exhaustion test cannot complete (the runner loops forever — no fault ever enters the budget; the mutated run's timeout is itself the proof); the test is then bounded (20 s) so a future regression of this shape fails, not hangs. LEADER_TTL_S widened private → pub(crate) for the lock-left-to-TTL expiry pin. Verified: full pg column live vs the harness (124 lib + 25 suite + 9 schema; new tests green solo ×3); cargo test --workspace green server-less, skips clean (core 25 + harness 3, sqlite 191 + 25); clippy --all-targets -D warnings clean; fmt clean; cargo-llvm-cov confirms the Finding-1 sites (the retry/backoff/exhaustion-arm region + the tick_once seam arm) carry zero missed lines; taskgraph validate green (61 tasks)

2026-10-10 11:00:26 +00:00

schema.rs

Postgres engine: schema bootstrap — engine-owned schema, table family, idempotent DDL, schema-prefixed indexes (task pg-engine-schema)

2026-10-09 05:11:48 +00:00

seam.rs

pg-fix-scheduler-tick-seam — the pg scheduler's transient-fault tick seam lands, retiring review 003 Finding 1 (MEDIUM: the tick retry loop was dead code under test — the last uncovered hardening arm of the wave-4 fix batch, the exact class review 002's live bugs came from). The seam follows the sqlite-commit-error-arm pattern of record: a cfg(test) per-store counting fault arm (crate::seam::tick_fault — Flag = Arc<AtomicU64>, born disarmed at 0, arm(n) adds, take() consumes one via CAS try_update; a counting arm because a boolean cannot reach the exhaustion arm — arm(TICK_RETRIES+1) faults the loop's whole budget while arm(1) leaves the later ticks real) feeding a fabricated pool/database-shaped opaque Database error into the REAL retry loop — both tests drive the full run_schedules → tick_with_retries → tick_once path end-to-end, no loop replication. Plumbing per the house pattern: cfg-gated tick_fault field on EngineCtx and PgStore (engine_ctx() carries the clone; production builds never materialize it), cfg-gated fault param on tick_once (the seam check sits before the real attempt — one attempt faults, never ticks) and tick_with_retries, cfg-branched attempt calls; PgStore::arm_tick_fault(faults) arming surface + tick_fault_flag() observation surface (a Flag clone so the consumption pin survives the store moving into a runner task). Two arms pinned: (a) one_faulted_tick_is_retried_and_the_job_fires — fault consumed on attempt 1, the 250 ms backoff sleeps, the retried attempt fires the backdated due boundary into the queue, exactly-one consumption, the runner survives the hiccup to a clean stop Ok(()); (b) tick_retry_exhaustion_exits_typed_and_leaves_the_lock_to_lapse — the exhaustion exits typed (opaque Database, never LeadershipLost; the tick-phase context rides the source chain with the fabricated fault at its bottom; wall-clock ≥ 1.7 s proving the real backoff window), the leadership lock row is left (not released) to lapse at its TTL (present, expiry bounded by the run's TTL horizon), exact consumption, and the store remains fully usable after — the "runner survives" pin as the module docs' contract states it. Replay-proofed live: with the seam's consumption disabled the exhaustion test cannot complete (the runner loops forever — no fault ever enters the budget; the mutated run's timeout is itself the proof); the test is then bounded (20 s) so a future regression of this shape fails, not hangs. LEADER_TTL_S widened private → pub(crate) for the lock-left-to-TTL expiry pin. Verified: full pg column live vs the harness (124 lib + 25 suite + 9 schema; new tests green solo ×3); cargo test --workspace green server-less, skips clean (core 25 + harness 3, sqlite 191 + 25); clippy --all-targets -D warnings clean; fmt clean; cargo-llvm-cov confirms the Finding-1 sites (the retry/backoff/exhaustion-arm region + the tick_once seam arm) carry zero missed lines; taskgraph validate green (61 tasks)

2026-10-10 11:00:26 +00:00

store.rs

pg-fix-scheduler-tick-seam — the pg scheduler's transient-fault tick seam lands, retiring review 003 Finding 1 (MEDIUM: the tick retry loop was dead code under test — the last uncovered hardening arm of the wave-4 fix batch, the exact class review 002's live bugs came from). The seam follows the sqlite-commit-error-arm pattern of record: a cfg(test) per-store counting fault arm (crate::seam::tick_fault — Flag = Arc<AtomicU64>, born disarmed at 0, arm(n) adds, take() consumes one via CAS try_update; a counting arm because a boolean cannot reach the exhaustion arm — arm(TICK_RETRIES+1) faults the loop's whole budget while arm(1) leaves the later ticks real) feeding a fabricated pool/database-shaped opaque Database error into the REAL retry loop — both tests drive the full run_schedules → tick_with_retries → tick_once path end-to-end, no loop replication. Plumbing per the house pattern: cfg-gated tick_fault field on EngineCtx and PgStore (engine_ctx() carries the clone; production builds never materialize it), cfg-gated fault param on tick_once (the seam check sits before the real attempt — one attempt faults, never ticks) and tick_with_retries, cfg-branched attempt calls; PgStore::arm_tick_fault(faults) arming surface + tick_fault_flag() observation surface (a Flag clone so the consumption pin survives the store moving into a runner task). Two arms pinned: (a) one_faulted_tick_is_retried_and_the_job_fires — fault consumed on attempt 1, the 250 ms backoff sleeps, the retried attempt fires the backdated due boundary into the queue, exactly-one consumption, the runner survives the hiccup to a clean stop Ok(()); (b) tick_retry_exhaustion_exits_typed_and_leaves_the_lock_to_lapse — the exhaustion exits typed (opaque Database, never LeadershipLost; the tick-phase context rides the source chain with the fabricated fault at its bottom; wall-clock ≥ 1.7 s proving the real backoff window), the leadership lock row is left (not released) to lapse at its TTL (present, expiry bounded by the run's TTL horizon), exact consumption, and the store remains fully usable after — the "runner survives" pin as the module docs' contract states it. Replay-proofed live: with the seam's consumption disabled the exhaustion test cannot complete (the runner loops forever — no fault ever enters the budget; the mutated run's timeout is itself the proof); the test is then bounded (20 s) so a future regression of this shape fails, not hangs. LEADER_TTL_S widened private → pub(crate) for the lock-left-to-TTL expiry pin. Verified: full pg column live vs the harness (124 lib + 25 suite + 9 schema; new tests green solo ×3); cargo test --workspace green server-less, skips clean (core 25 + harness 3, sqlite 191 + 25); clippy --all-targets -D warnings clean; fmt clean; cargo-llvm-cov confirms the Finding-1 sites (the retry/backoff/exhaustion-arm region + the tick_once seam arm) carry zero missed lines; taskgraph validate green (61 tasks)

2026-10-10 11:00:26 +00:00

stream.rs

pg suite-infra hardening (task pg-suite-infra-hardening): the wave-4 review's F-1 defense-in-depth candidates — must_recv_event re-shaped from the 20 ms try_recv polling loop to a parked recv().await under the 15 s timeout wrapper (stream_tests.rs, the sole pg copy — SQLite twin untouched; the poll loop's wake-subscription interaction surface the F-1 flake suspected is out of the hot path, and the deadline assert still bounds the property). The parked form's terminal arms are explicit: Some(Err(e)) keeps the errored-instead-of-idling panic, None gets a receiver-closed panic the poll form never saw. The deadline miss self-diagnoses (the small-honest realization of candidate (b)'s discriminator): read_since(offset, 1000) over rows beyond the receiver's position names the residual class — rows durable undelivered (wake/re-drain class) vs no rows (publish-visibility class); with the parked recv a wake-arrived-but-re-drain-missed arm is structurally implausible (only a read error could miss, and it surfaces Err). Candidate (b)'s literal bridge-side counter skipped, reason recorded in the task (surfacing the bridge wake count across the dyn EventReceiver boundary needs downcast/keyed-global machinery beyond the small-honest bar); the small honest piece did land: wait_wake's Lagged(n) arm now logs (eprintln, house posture matching notify.rs's bridge_loop — and forwarder.rs's 'the receiver bridge's Lagged arm logs / recovers' doc claim now true at both bridges), stream name threaded into wait_wake's signature so the log attributes the lag. Verified: pg stream module 21/21 vs harness server (full module run), tx_publishes_compose_with_the_handle 5 solo re-runs green (determinism re-check under the parked shape), two consecutive full pg harness runs green (121 lib + 25 suite + 9 schema, ~72 s each), cargo test -p alkstore-postgres green server-less (skips clean), workspace cargo test green (12 binaries), clippy -D warnings, fmt clean

2026-10-10 08:00:29 +00:00

tx.rs

pg-fix-scheduler-fire-wake — the scheduler tick's fire wake lands, closing the recorded cross-engine fire-wake latency-parity gap (the wave-5 gate's one standing item, review-wave-4-fixes' recorded-for-later note): scheduler.rs::tick now issues one coalesced pg_notify per firing schedule per tick after the fire loop, still inside the tick's in_tx frame — channel = the fired queue's name (schedule() rejects reserved names per ADR-021 §3, so the plain queue name is always the right channel), empty payload (Wake { channel } only, ADR-008 §3), best-effort via the shared tx::wake_tx (widened private → pub(crate) per the recorded one-call shape; enqueue_row stays wake-free — no double-wake; auto-commit Queue::enqueue and the tx producer paths untouched). NOTIFY's native transactional delivery makes the wake commit-atomic: a rolled-back tick (crash mid-tick ⇒ boundary refires) discards the wake with its fire rows — the tx/no-ghosts discipline the tx producer paths already pin; coalescing default one-per-firing-schedule-per-tick (not per-fire) matching the SQLite watcher's per-committed-tick cadence, decision recorded in the task's Notes. New four-arm pinning test (scheduler_tests.rs, tx_producer_wakes_are_commit_atomic pattern): pre-commit silence inside the runner's own in_tx frame driving the engine's own tick (rogue-ticks shape), deterministic delivery at/after commit, coalescing (≥2 boundaries → one wake), nothing-due silence (same-tick not-due schedule + a later all-not-due tick, in-frame and post-commit), and the end-to-end runner leg (a live run_schedules leader's wake reaches a registered listener); local listener_hears helper per the per-file helper convention; rollback arm native (NOTIFY transactional delivery — no tick-fault seam built, per the task pin). Docs: tx.rs 'The tx wakes' section + wake_tx doc name the scheduler fire path as a shared caller; scheduler.rs module docs + tick doc pin the coalesced commit-atomic semantics. Record updates: review-wave-4-fixes' recorded-for-later bullet, suite-scheduler-rows' gate disposition note + Notes bullet, review-wave-5 §Notes 4's audit row — all retired with pointers; implementation.md gains the review-rounds line (fires visible to registered listeners ahead of the mem engine's posture being written). Also carried: review-wave-5 §6 flake-ledger update — row_lock_ttl_expiry_and_reacquisition failed once more in a full pg run this session (green on the next two full runs + six focused; unrelated mechanism to this change), meeting the ledger's own 'fails again' trigger with the dedicated investigative session owed by the wave-7 watch carriage. Verified: full pg lib suite 122/122 vs the harness (new test green ×3 solo); pg contract suite 25/25 + schema 9/9; workspace cargo test green server-less (pg skips clean incl. the new test); clippy --all-targets -D warnings clean; fmt clean

2026-10-10 10:27:35 +00:00