alkstore/docs/architecture/decisions/009-scheduler-collapse.md

# ADR-009: Scheduler collapses into queues — `schedule()` + runner, `@every`-only v1

## Status

Accepted (2026-10-05, Phase 1 — OQ-09's resolution; resolves the
scheduler row transferred by [ADR-008](008-contract-v1-pinning.md) §7)

## Context

[ADR-002](002-feature-scope.md) took the scheduler in as
"documented-thin": the family-wide "who sweeps / renews / reaps, and
when" problem is real, but only just begins to have consumers lean on
it. OQ-09's decision rule (inherited from
[queues.md](../queues.md)'s framing of it): decide against the
inventory rows, not against the whole honker menu — the scheduler is a
first-class mechanism if consumers need inspectable/pausable schedule
*objects* (`add/pause/resume/update/list/remove`), and otherwise it
collapses into queues + a `schedule()` call with the tick machinery
absorbed.

The evidence for the decision:

- **alkblobs (sweep cadence)** — [documented:
  consumer-inventory.md](../../research/consumer-inventory.md) — needs
  "run the sweep every N ticks with leader election across a fleet."
  An interval, not a schedule object.
- **alkfs (orphan reaping)** — [recalled: alkfs OQ-FS-07,
  `/workspace/@alkdev/alkfs/docs/research/phase-0.md`] — a periodic
  orphan-cleanup pass. An interval.
- No consumer document (pinned, documented, or operator-authority)
  names any of: wall-clock cron expressions, per-schedule timezones,
  pausing/resuming schedules without unregistering, listing schedule
  objects, or updating a schedule's expression in place.

So the collapse question itself answers cleanly. Two secondary
questions ride along and this ADR pins them with the collapse:

- **Which schedule specs does v1 accept?** Honker's parser accepts
  5-field cron, seconds-field cron, and `@every <n><unit>` — but its
  cron boundary arithmetic is hard-wired to *host local time*
  (`chrono::Local`, honker cron.rs), which is brittleness a contract
  should not inherit: a server TZ change silently re-times every
  stored schedule, and the two engines would diverge if one inherited
  local time and the other pinned UTC.
- **What runs the tick?** Every consumer punting "the embedder owns
  cadence" is exactly the hand-rolling the substrate exists to kill —
  but an *ambient* runner violates the family's no-ambient-timers
  posture (alkblobs ADR-005). The line between them is opt-in.

## Decision

### 1. The scheduler collapses into queues

The contract surface is two registration methods on `Store` plus one
runner; there is no `Scheduler` handle, no schedule objects, no
inspection surface:

```text
store.schedule(name, spec, queue, payload, opts) -> Result<Schedule>
store.unschedule(name) -> bool
store.run_schedules(stop) -> Result<()>        // runs until `stop`
```

- `schedule` upserts by `name` (re-register replaces the whole row) —
  honker's register semantics, which makes update = re-register and
  pause = unregister + re-register. Both are honest v1 substitutes;
  nothing in the inventory needs finer control.
- `Schedule` is a value carrying the registered row (name, spec, queue,
  opts) — read-back for confirmation, not a mechanism handle.
- **Name validation**: schedule names follow the queue-name rules —
  non-empty (`InvalidName`) and reserved-prefix rejection
  (`ReservedName`) at the entry point; `schedule()`/`unschedule()` are
  name-bearing entry points in the
  [ADR-008](008-contract-v1-pinning.md) §4 sense (their consumers-side
  obligation extends the entry-point list: a reserved-prefix schedule
  name could collide with the machinery's derived names). No charset
  rule beyond that in v1. *(The **queue argument** of `schedule()`
  gets the same directly-supplied-queue-name validation, added
  2026-10-07 by [ADR-021](021-tx-reads-and-value-shape-fixes.md) §3 —
  without it, a schedule row could store a reserved name as its fire
  target and every boundary fire would enqueue into it, a third write
  path into the outbox's backing queue contradicting
  [ADR-014](014-outbox-tx-enqueue.md)'s stated guarantee.)*
- `run_schedules` is the tick: acquire the leadership lock, loop
  {renew lock — on loss, *return before ticking*; fire every due
  boundary; sleep until the next due boundary or `stop`}. Losing
  leadership ends the runner — the consumer's recipe is to respawn it
  (documented); the lock loss always precedes any stolen fire, so
  respawn never double-fires a boundary.
- **Stop and return semantics**: `stop` is a cancellation token — a
  core-crate cloneable `StopToken` (pinned by
  [ADR-019](019-mechanism-handle-surfaces.md) §4, 2026-10-06; the
  original "exact type shapes at implementation" deferral is
  superseded). A leadership
  loss returns `Err(LeadershipLost)` — a distinct, matchable outcome
  from the clean `Ok(())` a stop produces; the respawn recipe matches
  on it. (One taxonomy note: `LeadershipLost` is a *return-shape*
  variant of the runner, pinned by the same act-differently rule as
  ADR-008 §5 — the caller acts on it by respawning.)
  *(Annotated 2026-10-11 — review 004 item 9; amend-in-place per
  ADR-017 §2 class 1, no publish tag exists.)* The loss arm is the
  refused renewal (`renew` → `Ok(false)`): the lease expired and was
  re-acquired elsewhere. A *failed* renewal is not a loss and never
  returns `LeadershipLost` — the lock semantics were never reached:
  a closed store (respawning a leader against a dead store is
  misdirected; the engine-wide closed-store posture applies) and
  storage failure both surface the opaque `Database` fallback with
  the engine detail preserved via the source chain. Engines whose
  renew is infallible (`Ok(bool)` only) have no failure arm to
  classify; engines whose renew can fail must propagate the `Err`
  unchanged (the SQLite engine's pre-annotation flattening —
  `unwrap_or(false)` — was the finding).
- Schedules do not fire until a consumer runs `run_schedules` — the
  opt-in is explicit, the no-ambient-timers posture holds,
  [ADR-002](002-feature-scope.md)'s scheduler evidence is served, and
  consumers who never schedule never pay for the runner.

### 2. v1 spec grammar: `@every <n><unit>` only

`spec` accepts exactly `@every <n><unit>` with `n` a positive integer
and `<unit>` in `s | m | h | d` (honker's `@every` grammar,
honker cron.rs). Everything else — 5-field cron, seconds-field cron —
parses as `InvalidSpec { spec }` and is rejected.

- **Why not cron strings**: no consumer row names one (checked against
  the inventory, not against honker's menu), and accepting them forces
  a timezone answer now. `@every` durations are TZ-free — the TZ
  question dissolves instead of being answered.
- **Extension path** (when a consumer names wall-clock cron via a
  consumer-inventory row): cron-string support with an explicit TZ
  decision attached (UTC-pinned contract or a per-schedule TZ
  parameter — decided by that extension, not assumed here).
- This keeps the SQLite engine off honker's cron boundary machinery
  entirely (`@every` advance is `next = last + interval`) — the
  local-TZ brittleness is not inherited, and a dependency requirement
  is not created. *(Consumer-inventory rows change this only via the
  row-first discipline.)*

### 3. Enqueue semantics per boundary

At each due boundary the tick enqueues the schedule's `payload`
(`serde_json::Value`, uniform with all payloads) into the named queue
with `ScheduleOpts { priority, max_attempts, expires }` — the
[EnqueueOpts](008-contract-v1-pinning.md) fields minus
delay/run-at (a boundary fire is never delayed by its own config; the
schedule *is* the delay). The enqueued work is ordinary queue work: it
inherits the queue mechanism's delivery guarantees
([ADR-006](006-wake-and-delivery-contract.md) queues row). The
scheduler adds no delivery machinery of its own.

### 4. Boundary guarantee — the scheduler's row in the delivery table

Extends [ADR-006](006-wake-and-delivery-contract.md)'s table,
resolving the transfer from [ADR-008](008-contract-v1-pinning.md) §7:

| Mechanism | Durability | Replay | Atomicity | Guarantee |
|---|---|---|---|---|
| scheduler (`@every`) | schedule rows + boundary fires are durable rows | n/a (fire = enqueue; the work's durability is the queues row) | fire and boundary-advance commit atomically under a row lock on the schedule row (crash mid-tick rolls back both — the boundary refires; a rogue ticker cannot double-fire) | **at-least-once per elapsed boundary while a leader runs**, with bounded catch-up: downtime fires replay boundary-by-boundary up to a fixed contract-constant per-tick cap (64, honker parity), beyond which missed boundaries skip forward to the next future boundary |

- The **bounded-catch-up + skip-forward** semantics are pinned as the
  honest contract: a consumer who pauses runners for a week does not
  come back to 10,000 stacked sweep jobs. (For an `@every 60s` sweep
  under an outage, replays beyond the cap are useless anyway.) **The
  cap is a fixed contract constant (64 per task per tick, honker
  parity)** — not a knob; like the backoff cap, no consumer names one,
  and the skip-forward is documented behavior, not configuration.
- **One firer per boundary** is pinned engine-generically by two
  layers: (a) the leadership lock — one leader at a time, a leader
  that lost the lock returns before ticking; (b) **the boundary
  advance is a transactional claim on the schedule row**: the fire
  transaction takes a row lock on the schedule row, re-checks
  `next_fire_at <= now` under the lock, enqueues, and advances — so
  even a rogue second ticker (an operator bypassing the leadership
  lock) observes a boundary as already-fired and skips it, on either
  engine (Postgres: `SELECT … FOR UPDATE` on the row inside the fire
  tx; SQLite: the same guarantee via writer serialization —
  `BEGIN IMMEDIATE`). Layer (b) is the correctness floor, layer (a)
  the efficiency (one waiter instead of row-lock contention);
  crash mid-tick rolls back enqueue + advance together (the boundary
  refires).
- **Leader election** is a named lock under the reserved prefix —
  `__alkstore_scheduler` (engine-derived name in the lock namespace per
  [ADR-008](008-contract-v1-pinning.md) §4's rule; consumers can never
  collide with it because every entry point rejects the reserved
  prefix). Both engines: each engine's own lock machinery, engine-
  internal detail. One firer per boundary across all processes sharing
  the store.

### 5. Schedule storage is engine-internal

Schedule rows live in engine-owned storage (SQLite: the forked
substrate's `__alkstore_scheduler_tasks` table per
[ADR-011](011-sqlite-substrate-fork.md) — honker's scheduler-task
table re-owned and renamed, carrying `@every` specs in its spec
column, the cron machinery not ported; Postgres: the engine-owned
schema per [ADR-010](010-queue-semantics-depth.md) §8). Not consumer
namespace;
schedule *names* take the same validation as queue names
(§1 — non-empty, reserved-prefix rejected). No registry
validation against queue names — enqueuing into a queue nobody has
claimed yet is fine (queues are names, not registered objects).

### 6. Error taxonomy delta

Two variants added by this surface, per ADR-008 §5's act-differently
rule: `InvalidSpec { spec }` — a schedule spec that doesn't parse
under §2's grammar (the caller fixes the spec string; rejected before
storage, no round trip) — and `LeadershipLost` — the `run_schedules`
return when leadership is lost mid-run (the caller acts by
respawning; distinct from clean stop, §1).

## Consequences

**Positive**

- The contract gains the scheduler for two registrations + one runner;
  the "who sweeps" problem has a substrate answer that is one `@every`
  row and one spawned task.
- The tick machinery (leader election, boundary advance, catch-up cap)
  is engine-internal and mostly inherited (SQLite: honker's scheduler
  machinery with `@every` specs; Postgres: re-derived, small at this
  surface size — the row-locked fire tx is the one mechanism the
  contract pins engine-generically, §4).
- No TZ semantics, no cron parser in the contract, no schedule-object
  CRUD — the surface stays proportionate to the two inventory rows that
  exist.

**Negative**

- Honker's richer scheduler (`add/update/pause/resume/list`, cron
  with TZ) is deliberately not consumed at the contract level; consumers
  needing inspectable schedules need an inventory row + contract
  extension first.
- The opt-in runner is one more thing a deployer must remember: forgot
  `run_schedules` ⇒ schedules silently never fire. The mechanism's doc
  text must carry this (a silent-nothing deployment posture, the honest
  twin of "no ambient timers").
- The catch-up cap is a consumer-visible loss mode for long outages —
  documented (skip-forward), not silent: the schedule row's advance
  makes the gap inspectable.
- The scheduler surface (and this track's `QueueOpts` depth) are
  **post-v1 contract extensions** — the first exercises of the
  extension path [ADR-008](008-contract-v1-pinning.md) leaves open;
  their versioning discipline (how engine crates track the additions)
  is OQ-10's, still open. *(Resolved 2026-10-06 by
  [ADR-017](017-contract-versioning.md): nothing was released when
  these extensions were made, so they factually fall in ADR-017
  class 1 — pre-implementation amend-in-place — and ship inside the
  initial contract v1 text; the post-release extension *shape* this
  ADR exercised is ADR-017's class 2.)*

## References

- OQ-09 (`docs/architecture/open-questions.md`) — this ADR's
  resolution; owns ADR-008 §7's transferred scheduler guarantee row.
- `docs/research/consumer-inventory.md` §scheduler — the two rows the
  decision is made against.
- [ADR-002](002-feature-scope.md) — scheduler in-scope posture this
  ADR completes (documented-thin → pinned thin).
- [ADR-006](006-wake-and-delivery-contract.md) — the guarantee table
  §4 extends; the enqueued work inherits the queues row.
- [ADR-008](008-contract-v1-pinning.md) §1 (scheduler out of v1
  pending this decision), §4 (reserved-prefix rule the leadership lock
  name follows), §5 (act-differently rule the `InvalidSpec` variant
  follows), §7 (the guarantee-row transfer).
- [ADR-010](010-queue-semantics-depth.md) — the queue semantics depth
  this ADR's fire semantics compose with; the maintenance recipe (who
  sweeps) is recorded there with the collapse machinery.
- Honker reference checkout `/workspace/honker` @ f4e53c6 — scheduler
  surface (`packages/honker-rs/src/lib.rs`, `honker-core/src/honker_ops.rs`),
  `@every` grammar and local-TZ brittleness (`honker-core/src/cron.rs`),
  tick/leader mechanics.
- [ADR-011](011-sqlite-substrate-fork.md) — the SQLite-side schedule
  storage this §5 describes is the forked substrate's; cron machinery
  not ported.
- [core-contract.md](../core-contract.md) — the spec carrying this
  surface.