pg: scheduler runner resilience — quarantine bad rows, retry transient ticks

Review 002 Finding 5 (+ Finding 6's scheduler doc bullet):

- tick: a due row whose stored spec fails @every re-parse is
  quarantined, not fatal — logged with the schedule name, boundary
  advanced strictly past now (skip-forward at min interval), tick tx
  still commits, remaining due rows proceed
- run_schedules: a pool/database tick failure retries 3x with a short
  doubling backoff (250ms -> 1s cap) before the loop exits Err; the
  top-of-iteration renew keeps owning the lease-loss decision
- exiting errors carry schedule-name/tick-phase context in the source
  chain (ErrorContext wrapper; Database's Display is opaque)
- module docs: the Err-exit TTL-lapse posture stated; the false
  per-slice soonest re-read claim corrected to the honest slice/idle
  posture (60s idle floor)

Tests: behavioral quarantine pin (tampered via direct SQL UPDATE;
runner survives to clean Ok(()), other schedule fires, bad row never
fires, boundary advanced) and a server-less retry-policy pin.
Verified: cargo test -p alkstore-postgres green against the harness
server (121+10+9), build/clippy -D warnings/fmt green server-less.

tasks/pg-fix-scheduler-resilience.md

@@ -1,7 +1,7 @@
 ---
 id: pg-fix-scheduler-resilience
 name: Scheduler runner resilience — quarantine bad rows, retry transient errors (review 002 Finding 5)
-status: pending
+status: completed
 depends_on: []
 scope: narrow
 risk: medium
@@ -76,20 +76,20 @@ code.
 
 ## Acceptance Criteria
 
-- [ ] A tampered/foreign spec row is quarantined (logged with name,
+- [x] A tampered/foreign spec row is quarantined (logged with name,
       boundary advanced past now, other schedules still fire, runner
       survives) — pinned by behavioral test
-- [ ] A transient tick error is retried N times (pinned N) with
+- [x] A transient tick error is retried N times (pinned N) with
       backoff before the loop returns `Err` — policy pinned by test
-- [ ] The exiting error's chain carries schedule-name/tick-phase
+- [x] The exiting error's chain carries schedule-name/tick-phase
       context
-- [ ] The error-exit leadership posture (TTL lapse covers the
+- [x] The error-exit leadership posture (TTL lapse covers the
       unreleased lock) stated in the module docs
-- [ ] The `scheduler.rs:46-53` doc claim corrected to the honest
+- [x] The `scheduler.rs:46-53` doc claim corrected to the honest
       slice/idle posture
-- [ ] Existing scheduler tests stay green (clean stop,
+- [x] Existing scheduler tests stay green (clean stop,
       `LeadershipLost`, catch-up cap, boundary math)
-- [ ] `cargo test -p alkstore-postgres` (harness server), clippy
+- [x] `cargo test -p alkstore-postgres` (harness server), clippy
       `-D warnings`, fmt clean; gates green server-less
 
 ## References
@@ -105,8 +105,74 @@ code.
 
 ## Notes
 
-> To be filled by implementation agent
+- **Retry budget pinned at N = 3** (`TICK_RETRIES`), retry *beyond*
+  the first attempt (4 total); backoff doubling 250 ms → 500 ms → 1 s
+  (`tick_retry_backoff`, capped at 1 s; the full retry window adds ≤
+  1.75 s of stop latency). Every tick error at the loop's tick site is
+  pool/database-shaped (`Error::Database`), so the retry is
+  unconditional — `LeadershipLost` can never originate there, and a
+  lost lease stays the top-of-iteration renew's decision (documented
+  explicitly: a mid-retry lapse is caught by the renew on the next
+  iteration, before any tick).
+- **Quarantine advance mechanism**: the task said "the tick's
+  skip-forward helper" — since an unparseable spec yields no interval,
+  the helper runs at the minimum interval
+  (`skip_forward(next_fire_at, now, 1)`), so the advance is strictly
+  past now and monotone (no within-tick spin). Consequence noted in
+  the module docs: the quarantined row re-enters the due set on every
+  subsequent tick and is re-quarantined (and re-logged) each time —
+  the runner keeps a ≥1 s cadence while the stored spec is broken
+  rather than failing; repairing/unscheduling the row stops the noise.
+- **Error-context carrier**: `Database`'s Display is opaque, so a
+  plain `database_error(msg)` would drop the driver detail along with
+  losing the source chain. Implemented as `context_error`: the exiting
+  error is `Database → io::Error (context message) → original error →
+  driver chain` — the message carries schedule name / tick phase AND
+  the full driver chain is preserved. Wrapped sites: the fire enqueue
+  and the boundary advance inside `tick` (schedule name + phase), and
+  the loop's retry-exhausted exit (tick phase).
+- **Exit-path leadership**: documentation only (the TTL-lapse posture
+  is the bar); no opportunistic release — keeping the exiting path
+  simple per the task's fine print.
+- **Per-slice soonest re-check not implemented** (the task's optional
+  arm): the module doc now states the honest posture (1 s slices only
+  renew the lease and check the stop token; new schedules noticed at
+  the current soonest deadline, or up to 60 s late when idle).
+- **No behavioral transient-error injection test**: no deterministic
+  arrangement exists that wouldn't depend on timing (server kills race
+  the retry window); the unit-level pin of count/backoff shape is the
+  bar and landed (`the_tick_retry_policy_is_pinned`, server-less).
+- **SQLite twin parity**: `alkstore-sqlite/src/scheduler.rs` retains
+  the original propagate-straight-out posture (its substrate tick has
+  the same re-parse-from-storage exposure). The task scoped the fix to
+  the pg engine; if the twins' error postures must agree at wave 5's
+  cross-engine suite, follow up there.
 
 ## Summary
 
-> To be filled on completion
+Landed the scheduler runner resilience fix in
+`alkstore-postgres/src/scheduler.rs` (review 002 Finding 5 + the
+scheduler bullet of Finding 6): (1) bad-row quarantine — a due row
+whose stored spec fails `@every` re-parse is logged with its name,
+has its boundary advanced strictly past now via the skip-forward
+helper at minimum interval, and the remaining due rows proceed (the
+tick tx still commits); (2) transient-error retry — the loop's tick
+(`tick_once` hoisted from the inline body) retries 3× with a short
+doubling backoff (250 ms → 1 s cap) before the loop exits `Err`;
+(3) error context — schedule-name/tick-phase context rides the exiting
+error's source chain via the `ErrorContext` wrapper (source chain
+preserved, unlike a plain message remap); (4) exit-path leadership
+honesty — the module docs state the TTL-lapse posture for the
+unreleased lock; (5) the false `scheduler.rs:46-53` doc claim
+corrected (slices renew the lease/check the stop token only; new
+schedules noticed at the current soonest deadline or up to 60 s late
+when idle). Tests: behavioral quarantine pin (`tampered_spec_row_
+is_quarantined_and_the_runner_survives` — tampered via direct SQL
+UPDATE, runner survives to a clean `Ok(())`, other schedule fires, bad
+row never fires, boundary advanced) and the server-less retry-policy
+pin (`the_tick_retry_policy_is_pinned`). Verified: full
+`cargo test -p alkstore-postgres` green against the harness server
+(121 lib + 10 contract_suite + 9 schema, including all pre-existing
+scheduler tests), workspace `cargo build`/`cargo test`,
+`cargo clippy --all-targets -- -D warnings`, `cargo fmt --check` all
+green server-less.