docs: resolve OQ-05 + OQ-09 — queue semantics depth (ADR-010) and scheduler collapse (ADR-009)
ADR-009: scheduler collapses into queues — schedule()/unschedule()/ run_schedules (opt-in, no ambient timers), @every-only v1 grammar (dissolves honker's local-TZ cron brittleness), boundary guarantee row (at-least-once per boundary, fixed 64-cap catch-up with skip-forward, row-locked fire tx as engine-generic no-double-fire floor), __alkstore_scheduler leadership lock, InvalidSpec + LeadershipLost taxonomy additions. ADR-010: queue depth pinned engine-uniformly — three-state machine (pending/processing/dead) with delete-on-ack, get_job sees dead rows, heartbeat = renewal with late-heartbeat refusal, reclaim-consumes- an-attempt stated as contract text, equal-jitter exponential backoff (range definitionally pinned, 1 h cap), QueueOpts stamped onto job rows at enqueue (no per-queue registry), move-to-dead dead-letter with retention-sweep support and no redrive API, sweep_expired carries the no-stranded-rows property (fixes honker's expired- processing zombie hole — SQLite-side realization rides OQ-06 as a concrete fork candidate), one engine-owned pg schema, queues are rows not tables, result-storage cut-flag stands. Also: full honker-machinery and pgboss-rs reference reads persisted (docs/research/reference-*.md — the honker defect list pre-stages the OQ-06 quality read), queues.md rewritten from design-space frame to resolved-depth spec, core-contract/engines/README/overview/deployment/ ADR-002 propagated.
This commit is contained in:
1 parent
7ad8ac56bc
commit
79a135c934
13 files changed
+1863
-198
No files matched your search
@@ -6,17 +6,16 @@ last_updated: 2026-10-05
|
||||
# alkstore — Open Questions
|
||||
|
||||
Centralized tracker. IDs `OQ-NN` are stable — never renumber; append.
|
||||
The Phase 0 register's questions (`OQ-ST-01..08` in
|
||||
`docs/research/phase-0.md`) are promoted here faithfully: **OQ-01..08
|
||||
mirror OQ-ST-01..08 one-to-one**, keeping their Phase 0 statuses
|
||||
(a resolved register question stays listed here as resolved, with the
|
||||
ADR that carries its decision). New Phase 1 questions append from
|
||||
OQ-09. Resolution order so far: **OQ-04 resolved** (2026-10-05,
|
||||
Promoted, one-to-one, from Phase 0's OQ-ST register. Resolution order
|
||||
so far: **OQ-04 resolved** (2026-10-05,
|
||||
[ADR-008](decisions/008-contract-v1-pinning.md) — the contract surface
|
||||
everything else hangs off). Next: OQ-05/OQ-09 (the queue semantics
|
||||
track; OQ-09 also owns the scheduler guarantee row), OQ-06
|
||||
(the SQLite dependency gate), OQ-08 (rides the now-pinned trait
|
||||
shape), OQ-10 (versioning discipline for contract extensions).
|
||||
everything else hangs off); **OQ-09 + OQ-05 resolved** (2026-10-05,
|
||||
[ADR-009](decisions/009-scheduler-collapse.md) and
|
||||
[ADR-010](decisions/010-queue-semantics-depth.md) — the queue semantics
|
||||
track and the scheduler collapse/guarantee row). Next: OQ-06
|
||||
(the SQLite dependency gate — now carrying two concrete fork
|
||||
candidates from the queue-depth design), OQ-08 (rides the now-pinned
|
||||
trait shape), OQ-10 (versioning discipline for contract extensions).
|
||||
|
||||
Resolved questions stay listed with their resolution; they are not
|
||||
deleted.
|
||||
@@ -117,43 +116,68 @@ narrowed to the pinning work its own record already scoped.)*
|
||||
|
||||
## Theme: Queues and scheduling
|
||||
|
||||
### OQ-05: Queue semantics depth — retry/backoff/dead-letter/sweep design *(== OQ-ST-05)*
|
||||
### OQ-05: Queue semantics depth — retry/backoff/dead-letter/sweep design *(== OQ-ST-05)* — **RESOLVED**
|
||||
|
||||
- **Origin**: [queues.md](queues.md)
|
||||
- **Status**: open (posture resolved: re-derived on both engines with
|
||||
the pg-boss schema family as design reference and honker's queue
|
||||
design as the SQLite-side one — ADR-004/ADR-005)
|
||||
- **Status**: resolved (2026-10-05, Phase 1 — [ADR-010](decisions/010-queue-semantics-depth.md))
|
||||
- **Priority**: high
|
||||
- **Resolution**: open. The transactional and claim properties (the
|
||||
hard driver-coupled part) are POC-pinned on both engines. Remaining:
|
||||
the semantics-depth design — retry policy shape (attempts, backoff
|
||||
curve), visibility-timeout/renewal mechanics, dead-letter
|
||||
move-vs-flag and retention, sweep/maintenance design
|
||||
(`sweep_expired` cadence and owner), and whether result-storage's
|
||||
cut-flag gets reconsidered as part of this surface (it is
|
||||
queue-adjacent tooling).
|
||||
- **Resolution**: Pinned by ADR-010: (1) job state machine — three
|
||||
states (`pending/processing/dead`), delete-on-ack, move-to-dead
|
||||
(honker's model, pg-boss's seven-state enum not inherited),
|
||||
`get_job` sees dead rows (`last_error`/`died_at` — post-mortem by
|
||||
API; `cancel` unconditional, not an interrupt). (2)
|
||||
Visibility/heartbeat — renewal semantics (absolute reset),
|
||||
late-heartbeat refusal (dual-execution window is contract-honest),
|
||||
**reclaim-consumes-an-attempt** stated as contract text. (3)
|
||||
Retry/backoff — `retry(err, None)` computes the queue's
|
||||
equal-jitter exponential curve (range definitionally pinned, not a
|
||||
labeled distribution; base `backoff_base_s`, cap 1 h, attempt index
|
||||
= row `attempts` at retry), `Some(d)` overrides; queue-level
|
||||
`QueueOpts {
|
||||
visibility_timeout_s, max_attempts, backoff_base_s,
|
||||
dead_letter_retention_s }` **stamped onto each job row at enqueue**
|
||||
(§3a — no per-queue registry; two handles with different opts for
|
||||
one queue name don't fight). (4) Dead-letter — move-to-table,
|
||||
retention forever by default, no redrive API (recipe: `get_job` +
|
||||
fresh `enqueue`). (5) `sweep_expired` — the no-stranded-rows
|
||||
property (every past-expiry row in any state moves to dead; honker's
|
||||
expired-processing zombie hole fails this — fixed at contract
|
||||
level, SQLite-side realization rides OQ-06). (6) Sweep cadence —
|
||||
no ambient sweeper; the recipe is the collapse machinery
|
||||
(schedule + runner). (7) Result-storage cut-flag reconsidered and
|
||||
**stands** (delete-on-ack; pg-boss's completed-row model is result
|
||||
storage under another name). (8) Table layout — pg engine-owned
|
||||
schema (`alkstore` default), queues are rows not tables; SQLite
|
||||
rides `_honker_*`. No new error variants (ADR-008 §5's rule).
|
||||
- **Cross-references**: OQ-09, OQ-06.
|
||||
|
||||
### OQ-09: Is the scheduler a first-class mechanism, or queues + `schedule()`?
|
||||
### OQ-09: Is the scheduler a first-class mechanism, or queues + `schedule()`? — **RESOLVED**
|
||||
|
||||
- **Origin**: [queues.md](queues.md) (spin-out of OQ-05's Phase 0
|
||||
framing, where the scheduler's boundary question lived)
|
||||
- **Status**: open
|
||||
- **Status**: resolved (2026-10-05, Phase 1 — [ADR-009](decisions/009-scheduler-collapse.md))
|
||||
- **Priority**: medium
|
||||
- **Resolution**: open. The inventory found the scheduler need real but
|
||||
thin (the family-wide "who sweeps/renews/reaps" problem), and
|
||||
mechanically scheduler = cron/`@every` enqueueing into named queues
|
||||
+ leader election + a tick. If design confirms the collapse, the
|
||||
contract surface is queues + `store.schedule(cron, queue)` with the
|
||||
scheduler mechanism absorbed; if consumers need
|
||||
inspectable/pausable schedule objects (`add/pause/resume/update/
|
||||
list/remove`), it stays a first-class mechanism (the full honker
|
||||
shape). Decided against the inventory rows, not the whole honker
|
||||
menu. **Also owns the delivery-guarantee table's scheduler row**
|
||||
(transferred by [ADR-008](decisions/008-contract-v1-pinning.md) §7:
|
||||
under collapse the scheduler inherits the queues row; otherwise a
|
||||
dedicated row is pinned with the surface decision). The scheduler
|
||||
surface is *not* part of contract v1 ([ADR-008](decisions/008-contract-v1-pinning.md) §1).
|
||||
- **Resolution**: **Collapse confirmed**, decided against the
|
||||
inventory rows (alkblobs sweep cadence, alkfs orphan reaping — both
|
||||
intervals; no row names schedule objects or wall-clock cron): the
|
||||
contract surface is `schedule(name, spec, queue, payload, opts)`
|
||||
(upsert by name), `unschedule(name)`, and the opt-in
|
||||
`run_schedules(stop)` runner — no `Scheduler` handle, no
|
||||
schedule objects; update = re-register, pause = unregister +
|
||||
re-register. **Spec grammar v1: `@every <n><unit>` only** — cron
|
||||
strings rejected (`InvalidSpec`), dissolving the timezone question
|
||||
honker's local-time cron brittleness poses (extension path requires
|
||||
a consumer-inventory row). Boundary guarantee row pinned (the
|
||||
transfer from [ADR-008](decisions/008-contract-v1-pinning.md) §7
|
||||
resolved): at-least-once per elapsed boundary while a leader runs,
|
||||
fire+advance commit atomically under a row lock on the schedule row
|
||||
(engine-generic no-double-fire floor; the leadership lock is the
|
||||
efficiency layer), bounded catch-up (**fixed contract-constant cap
|
||||
64**) with skip-forward; leadership lock is the reserved
|
||||
`__alkstore_scheduler` name; `run_schedules` returns
|
||||
`Err(LeadershipLost)` on lock loss, `Ok(())` on clean stop. Schedule
|
||||
storage engine-internal; schedules never fire without a runner (no
|
||||
ambient timers).
|
||||
- **Cross-references**: OQ-05, OQ-04.
|
||||
|
||||
## Theme: Engines and dependencies
|
||||
@@ -172,8 +196,10 @@ narrowed to the pinning work its own record already scoped.)*
|
||||
failure-handling, and schema-migration brittleness. Outcomes: posture
|
||||
holds (no ADR change), or a fork/patch need is named (fork is normal
|
||||
work per ADR-005).
|
||||
- **Cross-references**: ADR-003, ADR-005, OQ-05 (if the read forces a
|
||||
fork, queue-on-honker-machinery work changes shape).
|
||||
- **Cross-references**: ADR-003, ADR-005, OQ-05 (the queue-depth read
|
||||
handed it two concrete fork candidates: the expired-
|
||||
processing-row zombie fix and dead-row `get_job` visibility —
|
||||
complement-over-machinery vs fork is exactly its calculus).
|
||||
|
||||
## Theme: Deployment and capabilities
|
||||
|
||||
@@ -190,8 +216,9 @@ narrowed to the pinning work its own record already scoped.)*
|
||||
(`Store::capabilities()`), a documented deployment matrix only
|
||||
([deployment.md](deployment.md) carries the facts), or compile-time
|
||||
knowledge only
|
||||
(a consumer choosing the SQLite engine knows). Rides OQ-04: the
|
||||
trait's shape constrains where capability differences can surface.
|
||||
(a consumer choosing the SQLite engine knows). Rides the now-pinned
|
||||
contract shape ([ADR-008](decisions/008-contract-v1-pinning.md)):
|
||||
the trait constrains where capability differences can surface.
|
||||
- **Cross-references**: OQ-04, [ADR-006](decisions/006-wake-and-delivery-contract.md).
|
||||
|
||||
### OQ-10: How do engine crates track core-contract version changes?
|
||||
@@ -207,10 +234,12 @@ narrowed to the pinning work its own record already scoped.)*
|
||||
test suite the engines run against the core's trait definitions?
|
||||
What happens to a released engine crate when core makes a contract
|
||||
breaking change?
|
||||
- **Cross-references**: OQ-04 (the contract being versioned), OQ-02.
|
||||
- **Cross-references**: OQ-04 (the contract being versioned — now
|
||||
including its first post-v1 extensions, ADR-009/ADR-010), OQ-02.
|
||||
|
||||
## Deferred / Blocked
|
||||
|
||||
None currently. Every open OQ above is actionable Phase 1 architecture
|
||||
work (contract pinning, design, quality read) with its evidence base
|
||||
complete — no external arrivals are being waited on.
|
||||
work (quality read, capability-surface shape, versioning discipline)
|
||||
with its evidence base complete — no external arrivals are being
|
||||
waited on.
|
||||
Reference in new issue
Block a user