SQLite engine: queues — Queue/JobHandle over the writer slot, engine-side backoff curve, extent guard (task sqlite-engine-queues)

queue.rs: SqliteQueueHandle (QueueOpts-carrying, stamp-resolving
enqueue over the seam's shared resolution arithmetic), claim_one/
claim_batch through the writer slot with ADR-023 §2's extent guard
(n <= 0 -> empty Vec at the trait-impl entry), the full JobHandle impl
(one-shot ack/retry/fail as 'static boxed futures, repeatable absolute
reset heartbeat, substrate's uniform validity predicate), and the
engine-owned equal-jitter exponential backoff (std-only RandomState
hash jitter, integerized inclusive [ceil(half), cap], 1-hour cap;
no rand dep - documented).

ack_batch loses its substrate worker filter (register D-31 - ADR-019
§1's worker-less batch form); job_from_json decode + stamps_with_
override moved to their one owners (queue.rs / resolution.rs);
reader-pool helpers lifted from stream.rs into seam.rs. Store::queue
wired; 10 acceptance tests (lifecycle/stamps, extent guard,
exactly-once under concurrency, backoff range + cap + override,
validity predicate + reclaim, dead-letter defaults + get_job
visibility, cancel, ack_batch, sweep both-states + retention,
validation + closed-store). Workspace 25+3+171 green, clippy
-D warnings, fmt clean; sqlite suite 4x green.
This commit is contained in:
glm-5.3-flash committed 2026-10-08 12:59:57 +00:00
1 parent 4913302b47
commit 513df0b311
11 files changed
+1708 -195

No files matched your search

+95 -11
View File
@@ -1,7 +1,7 @@
---
id: sqlite-engine-queues
name: SQLite engine — queues (`Queue`, claims, job handles, backoff curve)
status: pending
status: completed
depends_on: [sqlite-engine-seam-tx]
scope: moderate
risk: medium
@@ -54,23 +54,23 @@ over the substrate's re-derived queue ops (`enqueue`, `claim_batch`,
## Acceptance Criteria
- [ ] Full `Queue` + `JobHandle` impls; enqueue→claim→ack lifecycle
- [x] Full `Queue` + `JobHandle` impls; enqueue→claim→ack lifecycle
works; stamps visible via `get_job` match the resolution rules
- [ ] Extent guard: `claim_batch(n <= 0)` claims nothing (empty
- [x] Extent guard: `claim_batch(n <= 0)` claims nothing (empty
`Vec`, no error, never the whole queue) — pinned per ADR-023
§2's backlog row
- [ ] Exactly-once handout under concurrent claims (two workers, one
- [x] Exactly-once handout under concurrent claims (two workers, one
row — the POC-pinned property, engine-tested here)
- [ ] Backoff curve: `retry(err, None)` delays land in the pinned
- [x] Backoff curve: `retry(err, None)` delays land in the pinned
range (statistical test over many retries, tolerance-bounded);
cap at 1 hour; `Some(d)` overrides
- [ ] Validity predicate: lapsed-deadline ops refuse with `Ok(false)`
- [x] Validity predicate: lapsed-deadline ops refuse with `Ok(false)`
(heartbeat, ack, retry, fail); reclaim consumes an attempt
- [ ] Dead-letter: exhaustion moves to dead with the pinned default
- [x] Dead-letter: exhaustion moves to dead with the pinned default
strings; `get_job` sees dead rows with `last_error`/`died_at`
- [ ] `sweep_expired` moves expired rows (both states) + applies
- [x] `sweep_expired` moves expired rows (both states) + applies
retention; returns the moved count
- [ ] `cargo test -p alkstore-sqlite`, clippy `-D warnings`, fmt clean
- [x] `cargo test -p alkstore-sqlite`, clippy `-D warnings`, fmt clean
## References
@@ -84,8 +84,92 @@ over the substrate's re-derived queue ops (`enqueue`, `claim_batch`,
## Notes
> To be filled by implementation agent
- **`queue.rs` is new** (`SqliteQueueHandle` + `SqliteJobHandle`);
`Store::queue` wires it (validation + closed-store check at the
constructor — construction stays free, queues are names not
registered objects). The handle carries its `QueueOpts` as the stamp
source; `enqueue` resolves over the handle's stamps via the seam
task's `resolve_enqueue_opts` + a moved `stamps_with_override`
(now in `resolution.rs` — one owner, the tx paths share it).
- **Backoff jitter posture (the flagged choice)**: std-only — a fresh
`RandomState` hasher (per-process-random seed) over
`(row_id, attempts, subsecond-nanos)` draws uniformly over the
integerized range. No `rand` dependency was added (the ADR-005
dependency posture; a randomness-source change later is engine-opts
scope, not contract). The integerization is second-precision
honest: the draw is uniform over `[ceil(cap/2), cap]` **inclusive**
(never below the pinned lower half, never above the cap; the
attempt exponent is clamped `u32::try_from`-safe at 30 so the cap
arithmetic saturates rather than overflows). Statistical acceptance
test pins range membership + spread over 400 draws per curve point;
cap test pins `[1800, 3600]` at a=40.
- **Substrate delta (register D-31)**: `ack_batch(ids_json)` dropped
its `worker_id` filter — the contract's batch ack carries no worker
identity (ADR-019 §1: the batch form of ack, per-id independent
outcomes); the single-row `ack(job_id, worker_id)` keeps its
conjunct. The engine layer was the only caller. Recorded in
`PROVENANCE.md` per ADR-012 §4's delta discipline.
- **Decode owner moved**: `job_from_json` moved from `tx.rs` to
`queue.rs` (pub(crate)) so the claim pages and the tx reads share
one decoder; `stream.rs`'s reader helpers (`with_reader`,
`blocking_read`, closed-error posture) lifted into `seam.rs` as the
engine-wide reader seam (the queue's `get_job` is a reader-pool
read; claims/writes stay writer-slot).
- **One-shot op shapes**: `ack`/`retry`/`fail` destructure
`Box<Self>` and move the writer `Arc` into the `'static` future
(the consumed-handle trait shape, mirroring `TxHandle::commit`);
`heartbeat` borrows `&self` (repeatable). Refusals are `Ok(false)`
mapped from the substrate's 0-rows (the substrate owns the uniform
validity predicate engine-blind).
- **`retry(err, None)`'s exhaust string**: `None` err means the pinned
`"max attempts exceeded"` (the exhaustion path's owned string wins
over the caller-None case, per ADR-019 §3's annotation); a
caller-supplied `Some(err)` rides the dead row verbatim on both the
reschedule refusals and the dead-letter move. `fail(None)` pins
`"failed"`.
- **`get_job` is a reader-pool read** (auto-commit pure read, ADR-021
§1's read posture at the auto-commit convenience shape); the tx
twin (`get_job_tx`) stays a writer-conn read as landed in the seam
task. Claims/ack-batch/cancel/sweep/enqueue ride the writer slot.
- **Tests** (`store/queue_tests.rs`, 10): lifecycle + stamps
resolution, extent guard (`-1`, `0`), two-workers-one-row +
6×40 concurrent drain exactly-once, backoff statistical + cap +
end-to-end `run_at` + `Some(d)` override, healthy renewal + lapsed
refusals per op (one claim per one-shot op — consumed handles) +
reclaim-consumes-an-attempt + ack-loses-to-reclaim,
dead-letter defaults (`retry(None)`-exhausted, `fail(None)`,
caller strings) + dead-row `get_job` visibility, cancel
unconditional + ack_batch mixed-worker count + empty-batch,
sweep both-states move + retention count, retention vs
forever-none, validation (empty/reserved queue names, empty
`worker_id`) + closed-store fail-closed on all queue ops. Tests
probe the db file with a bare `rusqlite::Connection::open` where
stamps need honest cross-connection reads. Suite stable 4× green.
## Summary
> To be filled on completion
Implemented the queue mechanism's SQLite arm: `Store::queue` (validated
constructor) → `SqliteQueueHandle` carrying its `QueueOpts` stamps,
`enqueue` resolving `EnqueueOpts` over the handle's opts through the
seam's shared resolution arithmetic (`stamps_with_override` lifted to
`resolution.rs`; one clock read per enqueue, delay-over-`run_at`,
relative-`expires`, `encode_payload` at the seam), `claim_one`/
`claim_batch` through the writer slot with the ADR-023 §2 extent guard
(`n <= 0` → empty `Vec` at the trait-impl entry, before the
contract-blind substrate), the full `JobHandle` impl (one-shot
`ack`/`retry`/`fail` as `'static` boxed futures over the consumed
handle, repeatable absolute-reset `heartbeat`; substrate's uniform
validity predicate, refusals `Ok(false)`), the engine-side backoff
curve (equal-jitter exponential from the row's stamped
`backoff_base_s`, integerized inclusive over `[ceil(half), cap]`,
capped 3600 s; std-only `RandomState` hash jitter, no `rand` dep —
documented), `ack_batch` over the worker-less per-id predicate
(substrate D-31 delta, register recorded), unconditional `cancel`,
dead-visible `get_job` via the shared `job_from_json` decode owner
(moved from `tx.rs`), and both-states+sweep with retention.
`reader-pool` helpers (`with_reader`/`blocking_read`/closed-error)
lifted from `stream.rs` into `seam.rs` as the engine-wide reader seam.
10 new acceptance tests + the substrate ack_batch test updated
(171 sqlite tests total). Verified: `cargo test` (workspace: 25 + 3 +
171 sqlite), `cargo clippy --all-targets -- -D warnings`,
`cargo fmt --check` all clean; sqlite suite 4× green.