Postgres engine: scheduler + outbox — schedule (validation triad, the pg-owned @every-only parser, upsert over the schedule table), unschedule, run_schedules (leadership via the engine's lock machinery on __alkstore_scheduler with a per-instance owner token, in-sleep lease renewals, the row-locked FOR UPDATE tick in one pool tx — fire enqueues + boundary advance + soonest read commit together — the 64-boundary catch-up cap with skip-forward, clean stop / Err(LeadershipLost) arms), outbox (validated constructor, enqueue into the derived __alkstore_outbox:{name} with the 60/5/5 stamps + max_attempts override, run_once ack/retry-curve/false over the ordinary claim machinery, no engine-issued heartbeat) (task pg-engine-scheduler-outbox)

This commit is contained in:
glm-5.3-flash committed 2026-10-09 09:53:48 +00:00
1 parent c3591c2d44
commit 77619c5e93
9 files changed
+2547 -78

No files matched your search

+110 -3
View File
@@ -1,7 +1,7 @@
---
id: pg-engine-scheduler-outbox
name: Postgres engine — scheduler (`schedule`/`unschedule`/`run_schedules`) and outbox helper
status: pending
status: completed
depends_on: [pg-engine-queues, pg-engine-locks]
scope: broad
risk: high
@@ -147,8 +147,115 @@ queues task's claim machinery and the locks task's lock machinery:
## Notes
> To be filled by implementation agent
> Decisions of record made in implementation that the description
> didn't pin:
>
> - **Leadership acquire rides the lock module's `lock_acquire`
> directly, not `try_lock`** — `try_lock`'s entry-point shared-name
> validation rejects the reserved name
> (`__alkstore_scheduler`), so the engine's own holder builds the
> lease via `PgLockHandle::new_internal` (the SQLite twin's
> `SqliteLockHandle::new_internal` shape) after the raw acquire op.
> Same machinery, bypassing the consumer entry gate by construction.
> - **The wake-driven tick advance chose the bounded-wait interrupt**
> (the task's implementer's choice): no dedicated wake channel on
> the scheduler's side — the sleep is sliced at 1 s
> (`SLEEP_SLICE_S`) with the stop token checked and the lease
> renewal cadence (2 s) evaluated per slice, and the soonest/
> boundary state is re-read at the top of every loop iteration. A
> newly-registered schedule is therefore noticed at the next loop
> re-read after its boundary (and by the idle-tick's 60 s re-read
> at the latest) — the deadline always fires; the wake never gates
> correctness. The SQLite runner's idle-tick posture stands as the
> floor.
> - **The tick's fire-time `expires` rides `ScheduleOpts::expires`
> as a relative-seconds `EnqueueOpts::expires` through the shared
> `enqueue_row`** — the resolution (relative → absolute from the
> fire instant) is the resolution module's existing formula (one
> owner per formula, ADR-012 §2), not a duplicated scheduler-side
> arithmetic. The `enqueue_row` stamp override re-resolves
> `max_attempts` no-op-fully (the schedule row's stamps are already
> resolved and bind directly).
> - **An `EngineCtx` bundle (`pool`/`schema`/`closed`) threads the
> store wiring into the scheduler/outbox module's entry points**
> — `schedule`'s engine-side argument list stayed within the
> clippy arg guard and the wiring reads once at each call site.
> - **The outbox's `run_once` claim rebuilds the queues task's claim
> frame from the module's shared pieces** (`in_tx` +
> `sweep_exhausted_reclaimables_tx` + `claim_stmt_sql` +
> `job_from_row`, all made `pub(crate)`) with the backing queue
> name standing in for the handle's — one owner per formula, no
> duplicated claim SQL; the ok/err dispositions route through
> `PgJobHandle::ack`/`retry` (the `new` constructor made
> `pub(crate)` for the rebuild, mirroring the SQLite twin's
> `SqliteJobHandle::new`).
> - **The best-effort enqueue wake on the backing queue's channel**
> (the mechanism-name-is-the-channel realization) rides the outbox
> `enqueue` too — the queue enqueue's twin posture (durable row is
> the truth, the re-poll safety net covers a failed wake).
> - **The open-tests' stub-probe test retired** — all four
> `schedule`/`unschedule`/`run_schedules`/`outbox` methods are
> wired, so `store_trait_methods_are_wiring_stubs` (which asserted
> the `Err(Database(… wiring lands with …))` shape on them) now
> asserts the wired constructor/register/unregister surface
> instead; the `stub_err!` macro left with the removed stub calls.
> - **`tick` is `pub(crate)` (test-visible)** for the rogue-ticker
> row-lock probe — two concurrent pool connections driving the
> engine's own tick op directly (no leadership lock in either
> hand) fire exactly one boundary.
## Summary
> To be filled on completion
> What landed, verified how:
>
> - `alkstore-postgres/src/scheduler.rs` — the collapse shape's pg
> arm + the outbox helper: `schedule` (the validation triad
> pre-storage, the pg-owned `@every`-only grammar
> `parse_every_interval` — cron strings reject pre-storage — the
> upsert over the schema table with the plain-queue derived stamps
> + `ScheduleOpts` applied, `Schedule` read-back), `unschedule`
> (true/false), `run_schedules` (leadership via the engine's lock
> machinery on `__alkstore_scheduler` with a per-instance pid +
> subsec-nanos owner token, TTL 10 s engine-internal, renew at
> every loop top and on a 2 s cadence across the sliced sleep,
> acquire-time refusal = `Err(LeadershipLost)`, the row-locked
> `FOR UPDATE` tick in one pool-checked-out `BEGIN`/`COMMIT` frame
> — fire enqueues + boundary advance + soonest read commit
> together — the 64-boundary catch-up cap with skip-forward
> past-now, clean stop `Ok(())` releasing the lock, loss
> `Err(LeadershipLost)` before any tick); `outbox(name)` — the
> validated constructor, `enqueue` into the derived
> `__alkstore_outbox:{name}` (60/5/5 stamps, the `max_attempts`
> override, full ADR-020 resolution, best-effort wake) and
> `run_once(worker_id, delivery)` (the ordinary claim frame, ack
> on `Ok`, `retry(err-as-exhaust-string, curve-delay)` on `Err`,
> `false` on empty, no engine-issued heartbeat — the boxed
> handle's own `heartbeat` is the consumer's).
> - Store trait wiring (`store.rs`): the four stubs replaced; the
> `stub` fn and its last call sites removed. Queue module deltas:
> `in_tx`/`sweep_exhausted_reclaimables_tx`/`claim_stmt_sql`/
> `PgJobHandle::new` made `pub(crate)` (one-owned machinery
> reuse); lock module deltas: `lock_acquire` made `pub(crate)` +
> `PgLockHandle::new_internal` added.
> - Tests: `store/scheduler_tests.rs` (9 — the validation triad +
> exact parser surface, upsert/unregister read-backs, fire + stamp
> source via `get_job` + live leadership lock row with the
> pid-stamped owner + clean-stop release, second-runner refusal +
> the one-firer band, acquire-time loss with a zero-fire probe,
> in-sleep lease renewals holding across a 40 s boundary on a 10 s
> TTL, the 64-cap + skip-forward with no in-window refire from a
> 200-boundary far-out horizon, the rogue-ticker row-lock
> serialization, and the server-less parser unit pin) and
> `store/outbox_tests.rs` (5 — constructor validation,
> `enqueue_tx` unreachability + 60/5/5 stamps + override, ok/err/
> false dispositions with the curve gap asserted, in-delivery
> heartbeat renewal, exhaustion to dead with the delivery error
> string). `open_tests.rs`'s stub probe updated to the wired
> posture.
> - Verified: `cargo test -p alkstore-postgres` green against the
> harness server (111 + 9, six consecutive full runs — stable);
> workspace `cargo test` green server-less (the pg skips cleanly);
> `cargo clippy --all-targets -- -D warnings` and
> `cargo fmt --check` clean workspace-wide. Pre-existing flake
> (streams' wake-timing deadline race) untouched — none of the six
> runs tripped it.