SQLite engine: scheduler + outbox — schedule/unschedule/run_schedules leader loop, outbox enqueue/run_once (task sqlite-engine-scheduler-outbox)

tasks/sqlite-engine-scheduler-outbox.md

@@ -1,7 +1,7 @@
 ---
 id: sqlite-engine-scheduler-outbox
 name: SQLite engine — scheduler (`schedule`/`unschedule`/`run_schedules`) and outbox helper
-status: pending
+status: completed
 depends_on: [sqlite-engine-queues, sqlite-engine-locks]
 scope: broad
 risk: high
@@ -95,8 +95,106 @@ their own long-running machinery:
 
 ## Notes
 
-> To be filled by implementation agent
+> Decisions of record made in implementation that the description
+> didn't pin:
+>
+> - **The leadership lock's owner token is per-runner-instance-unique**
+>   (pid + subsec-nanos, `scheduler::leader_owner`), not a shared
+>   constant: the substrate's `lock_acquire` matches by owner string,
+>   so two processes running the same constant would each see
+>   "self" holding and both would tick. The runner-unique token is
+>   what makes the renewal-refusal path meaningful across stores on
+>   the same db. TTL is engine-internal (10 s), renewed at the top of
+>   every loop iteration **and** on a 2 s cadence across the sleep —
+>   without the in-sleep renewals a boundary further out than the TTL
+>   (any `@every` ≥ ~10 s) would lapse its own lease by drift. A
+>   failed mid-sleep renewal is deliberately not acted on at the
+>   sleep site — the loop's top-of-iteration renew owns the loss
+>   decision (returns `Err(LeadershipLost)` before any tick).
+> - **Acquire-time refusal is also `Err(LeadershipLost)`**: a runner
+>   that fails to take the lock at start returns the same matchable
+>   loss a mid-run loser does (no runner-loop exists without the
+>   lock; the respawn recipe — ADR-009 §1 — is identical either way).
+> - **The tick runs under `BEGIN IMMEDIATE`** on the writer slot —
+>   fire enqueues + boundary advance commit together (ADR-009 §4's
+>   layer (b); writer serialization as the row lock; a rogue second
+>   ticker observing the same storage serializes behind the writer
+>   and cannot double-fire). The tick body is the substrate's
+>   `scheduler_tick` over `now_unix`, then `scheduler_soonest` for
+>   the sleep deadline — all inside the one transaction.
+> - **The idle sleep (soonest = 0) is 60 s.** Nothing pins it; a
+>   schedule registered while the runner idles is noticed at the
+>   next idle tick (≤ 60 s late) — acceptable engine detail; the
+>   boundary math itself is unaffected.
+> - **`schedule`'s `InvalidSpec` maps the substrate's
+>   `parse_every_interval` rejection** — `parse_every_interval` was
+>   made `pub(crate)` (one-line delta in `substrate/queue_ops.rs`)
+>   instead of duplicating the grammar engine-side; the substrate
+>   stays the one parser (the task's "the substrate's parser; map
+>   its rejection" wording, taken literally).
+> - **`Outbox::run_once`'s `Err` arm passes the delivery error's
+>   `Display` string as the retry's exhaust string** — the same
+>   posture `JobHandle::retry(Some(e), …)` carries (the string
+>   surfaces on a dead row only if the budget exhausts). The curve
+>   delay is the engine's one-owner `backoff_delay_s` (the queue
+>   handle's owner, reused — ADR-010 §3's single-owner rule).
+> - **`SqliteJobHandle` grew a `pub(crate) fn new`** so the outbox
+>   can rebuild the claim handle for the delivery callable
+>   (`run_once` claims through the same `claim_batch` machinery and
+>   rewraps the row); fields stay private.
+> - **Test flake fix**: the catch-up-cap test originally drove
+>   `@every 1s` from a −200 s horizon; under full-suite load the
+>   runner's *next* live boundary could race into the assertion
+>   window. Switched to `@every 1h` with a 200-boundary backlog —
+>   the skip-forward target lands ~1 h out, so nothing can fire
+>   in-window; the assertions are now exact (64 fires, none after).
+>
+> Verification: `cargo test` (workspace: 188 sqlite + 25 core + 3
+> contract-suite), `cargo clippy --all-targets -- -D warnings`,
+> `cargo fmt --check` — all clean. The scheduler tests were run
+> repeatedly (3×) to confirm the flake fix holds.
 
 ## Summary
 
-> To be filled on completion
+> What landed, verified how:
+>
+> - `alkstore-sqlite/src/scheduler.rs` — `schedule`/`unschedule`/
+>   `run_schedules` wired to the `Store` trait in `store.rs`
+>   (replacing the three placeholder stubs): entry-point validation on
+>   all three of name/spec/queue (InvalidSpec pre-storage via the
+>   substrate parser), the substrate upsert/unregister through the
+>   writer slot, and the leader loop over `__alkstore_scheduler` with
+>   per-instance owner, renew-before-tick, transactional tick
+>   (`BEGIN IMMEDIATE`), soonest-driven sliced sleep with in-sleep
+>   lease renewals, clean-stop `Ok(())` and loss
+>   `Err(LeadershipLost)`.
+> - `alkstore-sqlite/src/outbox.rs` — `outbox(name)` wired (validated
+>   constructor), `enqueue` stamping the 60/5/5 derived set with the
+>   `EnqueueOpts::max_attempts` override and full ADR-020 resolution,
+>   and `run_once(worker_id, delivery)` claiming via the ordinary
+>   queue machinery, acking on `Ok`, `retry(Some(e), None)` (curve
+>   delay, delivery error as exhaust string) on `Err`, `false` on
+>   empty, the boxed `SqliteJobHandle` handed to the caller with no
+>   engine-issued heartbeat.
+> - Small substrate/engine deltas: `parse_every_interval` made
+>   `pub(crate)`; `SqliteJobHandle::new` (pub(crate)) added;
+>   `jobs_from_json_page` made `pub(crate)`; `SCHEDULER_LEADER_LOCK`
+>   constant lives in `lock.rs` next to the machinery it reuses;
+>   `SqliteLockHandle::new_internal` (engine-internal constructor for
+>   the leadership lease).
+> - Tests: `store/scheduler_tests.rs` (6 acceptance tests —
+>   validation triad, upsert read-back, fire-into-queue with stamp
+>   pinning via `get_job`, leadership lock live-row probe, second-
+>   runner refusal + one-firer-per-boundary band, acquire-time loss
+>   with a zero-fire probe, catch-up cap 64 + skip-forward with no
+>   in-window refire) and `store/outbox_tests.rs` (5 — constructor
+>   validation, backing-queue unreachability by `enqueue_tx` +
+>   60/5/5 stamps + override, ok/err/false dispositions with the
+>   curve gap asserted, in-delivery heartbeat renewal, exhaustion to
+>   dead with the delivery error string). The commit-atomic tx half
+>   (`outbox_enqueue_tx`) was already covered by `tx_tests.rs`
+>   (`tx_stamps_resolve_max_attempts_and_outbox_set`,
+>   `outbox_backing_queue_name_is_reserved_and_derived`, the
+>   validation rows in `tx_entry_points_validate_names`).
+> - Gates: workspace build/test green (216 total), clippy `-D
+>   warnings` clean, fmt clean.