alkstore/tasks/fork-rederive-queue-ops.md

id, name, status, depends_on, scope, risk, impact, level, tags

id	name	status	depends_on	scope	risk	impact	level	tags
fork-rederive-queue-ops	Fork re-derivation — queue ops on contract v1 (stamps, claim, sweep, get_job)	completed	fork-port-connection-watcher	broad	high	component	implementation	wave-2 substrate

Description

Re-derive the queue machinery in the substrate on contract v1 — the new-code half of the fork (ADR-011's re-derivation list; quality-read §6). This is the highest-risk task of wave 2: it is where ADR-010's semantics depth becomes SQL, and where the upstream defect class (the unreleased fix train's savepoint hardening, the .ok() error swallows) must be not inherited.

Re-derive (contract-derived names; the substrate API stays contract-blind — primitives in, primitives out, ADR-012 §2):

• Enqueue + stamping (ADR-010 §3a): job rows carry the QueueOpts stamps (visibility_timeout_s, max_attempts, backoff_base_s, dead_letter_retention_s) plus the EnqueueOpts resolution inputs. The resolution rules (delay-over-run_at, relative-expires, which stamps apply per call shape) are engine-layer work (wave 3) — the substrate's enqueue takes already-resolved primitive stamp values.
• Single-statement claim with per-row visibility from the job's stamps: exactly-once handout under concurrency (POC-pinned shape), attempts += 1 per claim, claim ordering priority DESC, ready-time ASC, enqueue order, claimant column stamped, claimed_at + claim_expires_at set (ADR-019 §3's Job fields; ADR-021 §2).
• Savepoint-guarded retry/fail/dead-letter (ADR-010 §1–§4): the DELETE→INSERT dead-letter moves are savepoint-hardened (the #133 defect class — a mid-flight error must strand the row in neither table); retry computes nothing itself (the engine passes the resolved delay in — the curve is engine-layer arithmetic per ADR-012 §2); budget exhaustion and fail move to dead with last_error/died_at; the pinned default strings ("failed", "max attempts exceeded") are engine-layer (contract-derived strings do not belong in the contract-blind substrate — pass them in).
• Both-states no-stranded-rows sweep_expired (ADR-010 §5): moves every past-expires_at row (pending and processing) to dead with last_error='expired', and enforces dead_letter_retention_s deletion. Single-statement atomic per queue.
• Dead-visible get_job (ADR-010 §1): reads dead rows with stamps, claimed_at, last_error, died_at — the zombie hole and dead-visibility gap are the fork's motivating fixes; both land here.
• Handle-op validity predicate support (ADR-010 §2): the ack/heartbeat/retry/fail transitions carry the uniform predicate (row processing + caller's claim deadline unexpired) — false, not error, on refusal; the ack-vs-reclaim race resolves atomically.
• Scheduler tick over the new enqueue with @every next-boundary math (numeric, a few lines — cron machinery not ported): the __alkstore_scheduler_tasks storage, boundary advance + fire enqueue + row-locked advance, the 64-cap catch-up (ADR-009 §4). The leadership-lock loop pattern is ported with the lock machinery; the boundary math is the re-derivation.

Schema: the __alkstore_* job/dead/scheduler table family with the stamp columns and claimed_at (ADR-012 §5 — the re-derivation adds them); partial indexes matching the claim hot path, dead rows outside it, single clock source (second-precision timestamps), per queues.md §Namespaces.

Tests: the contract-property floor — no-stranded-rows, dead-visible get_job, stamps immutability, claim exclusivity under concurrency, the validity predicate's refusal boundaries, savepoint hardening (mid-flight-error stranding test), scheduler boundary/catch-up math. Plus the inherited queue-machinery tests adapted where applicable.

Acceptance Criteria

• All re-derived ops above present with contract-blind primitive APIs (no contract types, no error-taxonomy types in the substrate)
• Savepoint hardening proven by test: a forced mid-flight error in the dead-letter move strands no row in either table
• No .ok()-style error swallows (the D-class defects); all substrate errors propagate typed
• Claim exclusivity under concurrent claims (multi-connection test); reclaim consumes an attempt; validity predicate refuses lapsed-deadline ops as values (false), not errors
• sweep_expired moves both states; retention deletion enforced; no-stranded-rows property test green
• Scheduler: boundary advance + fire atomic under row lock; 64-cap catch-up; @every math unit-tested (s|m|h|d)
• get_job sees dead rows with full stamp/error fields
• cargo test -p alkstore-sqlite, clippy -D warnings, fmt clean

References

• docs/architecture/decisions/010-queue-semantics-depth.md (§1–§6, §8)
• docs/architecture/decisions/009-scheduler-collapse.md §3–§4
• docs/architecture/decisions/012-forked-substrate-design.md §2, §5
• docs/research/quality-read-honker-core.md §3 (defect register), §6
• docs/research/reference-honker-machinery.md (the lineage mechanics, file/line cites)
• docs/architecture/queues.md (the semantics this realizes)

Notes

• Module shape: the re-derivation is one new substrate module, queue_ops.rs (beside the ported schema.rs / watcher.rs / ops.rs), with contract-derived names throughout per ADR-012 §3 — the re-derived half is new code, so no lineage-name fidelity obligation applies. It reuses the ported in_savepoint machinery (made pub(crate) for the new module — the only touch to the kept half besides schema).
• API shape: stamp values bundle into a Stamps { max_attempts, visibility_timeout_s, backoff_base_s, dead_letter_retention_s } struct (clippy arg-count floor of 7 forced the bundling; the bundle is also the honest shape — ADR-010 §3a's four stamps land together). Scheduler registration takes FireOpts { priority, expires_s } + Stamps. All resolution inputs (ready time as an absolute run_at, retry delay as a literal, the default error strings "max attempts exceeded" / "expired") are engine-layer — the substrate takes already-resolved primitives. Cron machinery absent: parse_every_interval is the whole spec grammar (ADR-009 §2), and a non-@every spec found in storage is rejected (pinned by test).
• Claim mechanics: single UPDATE … WHERE id IN (WITH picked AS select-ordered-limit) statement — per-row visibility via claim_expires_at = unixepoch() + visibility_timeout_s (the §3a bridge: the deadline computes from the row's own stamp, not a uniform call value). Ordering priority DESC, run_at ASC, id ASC in the picked subquery. The pre-claim dead-letter sweep for exhausted reclaimables is retained from the lineage (savepoint- guarded) with the engine-supplied exhaust string.
• Schema deltas: stamp columns on __alkstore_live (visibility_timeout_s, backoff_base_s, dead_letter_retention_s) alongside the ported claimed_at; the dead table gains the same stamps plus claimed_at/expires_at (ADR-019 §3's dead-row field list); the scheduler table carries the stamps too (schedule fires enqueue through the same stamping path); a __alkstore_dead(queue, died_at) index serves the retention sweep. All append-column migrations, so an existing __alkstore_* file migrates in place.
• retry shape: the retry branch is deliberately not savepoint-wrapped (it is a single UPDATE; there is no move), while the exhaustion branch is (DELETE→INSERT) — mirroring the lineage's post-#133 discipline, with the caller's resolved delay and the engine's exhaust string passed in.
• Boundary-advance transactionality: scheduler_tick keeps the lineage's shape — enqueue + advance in the caller's (writer) transaction, so crash mid-tick rolls both back; per-tick row-lock re-check rides SQLite's writer serialization exactly as ADR-009 §4 pins for this engine. The 64-cap (SCHEDULER_MAX_CATCHUP_FIRES) and skip-forward (next boundary strictly after now) are pinned by tests, including the never-doubles-fire re-tick check.
• Test floor: the inherited queue-op suites adapted (savepoint rollback paths through the Rust API; the scalar-function-path variants are not re-created — the substrate's SQL-function surface for queues gets attached at wave-3 wiring, where those adapters reproduce mechanically if needed), plus the contract-property suite (41 new tests): claim exclusivity under 6 concurrent connections × 120 jobs (zero double-handout, integrity intact), stamps immutability across claim/heartbeat, lapsed-deadline refusal as values for all four handle ops, reclaim-eats-attempt, ack-vs-reclaim race resolution, both-states sweep + retention TTL, the no-stranded-rows property including the zombie shape, savepoint hardening at every dead-letter site via a forced mid-flight error and a blocked dead INSERT, boundary/catch-up/skip-forward scheduler math, @every s/m/h/d unit math, get_job dead visibility with the full stamp list.
• Register: D-12 updated to the landed state (the only pre-declared entry this task owned); no new port-surfaced register entries arose — the deltas above are all within D-12's declared scope.

Summary

The queue half of the fork's re-derivation landed as alkstore-sqlite/src/substrate/queue_ops.rs (~560 lines of owned contract-v1 code + a 41-test property suite; 84 sqlite-crate tests green total): contract-blind enqueue with per-job stamping (Stamps), single-statement claim_batch with per-row visibility from the job's own stamps + the retained pre-claim dead-letter sweep, ack/ack_batch/heartbeat/retry/fail/cancel under the uniform validity predicate (refusals as values), savepoint-guarded dead-letter moves at every DELETE→INSERT site (forced-error and blocked-INSERT tests pin no-strand-in-either-table), the both-states no-stranded-rows sweep_expired with dead_letter_retention_s enforcement, dead-visible get_job carrying the full ADR-019 §3 field list (stamps, claimed_at, last_error, died_at), and the scheduler surface (register upsert, tick with @every boundary math, 64-cap catch-up + skip-forward, soonest, unregister) with fire+advance atomic under the writer transaction. Schema: stamp columns + claimed_at/expires_at on live and dead, stamps on the schedule table, retention index on dead — all append-column migrations. No contract or taxonomy types in the substrate; the default error strings are call-site parameters. PROVENANCE.md D-12 recorded as landed. Verified: cargo build, workspace cargo test (84 sqlite + 26 elsewhere), cargo clippy --all-targets -- -D warnings, cargo fmt --check all clean; zero tokio, no panics/unwrap/expect outside tests.