Implementation plan: wave-based decomposition; waves 1-2 decomposed (11 tasks)
docs/plans/implementation.md records the wave structure (core -> substrate fork || pg engine -> sqlite engine -> contract suite -> release), the decided points (contract-suite layout = option (a), engine-tests vs equivalence-suite split, no CI, mem-engine/fuzzing deferrals surfaced), and the review-gate rhythm. Wave 1 (foundations): workspace scaffold, core errors/validation, value types, trait surface, contract-suite scaffold (+ADR-022), review gate. Wave 2 (substrate fork): fork scaffold/provenance, connection+watcher port, queue-op re-derivation on contract v1, provenance/floor close, review gate.
This commit is contained in:
1 parent
83767e880b
commit
49743c690e
12 files changed
+1009
No files matched your search
@@ -0,0 +1,111 @@
|
||||
---
|
||||
id: fork-rederive-queue-ops
|
||||
name: Fork re-derivation — queue ops on contract v1 (stamps, claim, sweep, get_job)
|
||||
status: pending
|
||||
depends_on: [fork-port-connection-watcher]
|
||||
scope: broad
|
||||
risk: high
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-2, substrate]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Re-derive the queue machinery in the substrate on contract v1 — the
|
||||
new-code half of the fork (ADR-011's re-derivation list; quality-read
|
||||
§6). This is the highest-risk task of wave 2: it is where ADR-010's
|
||||
semantics depth becomes SQL, and where the upstream defect class (the
|
||||
unreleased fix train's savepoint hardening, the `.ok()` error
|
||||
swallows) must be *not* inherited.
|
||||
|
||||
**Re-derive (contract-derived names; the substrate API stays
|
||||
contract-blind — primitives in, primitives out, ADR-012 §2):**
|
||||
|
||||
- **Enqueue + stamping** (ADR-010 §3a): job rows carry the
|
||||
`QueueOpts` stamps (`visibility_timeout_s`, `max_attempts`,
|
||||
`backoff_base_s`, `dead_letter_retention_s`) plus the `EnqueueOpts`
|
||||
resolution inputs. The *resolution rules* (delay-over-run_at,
|
||||
relative-expires, which stamps apply per call shape) are engine-layer
|
||||
work (wave 3) — the substrate's enqueue takes already-resolved
|
||||
primitive stamp values.
|
||||
- **Single-statement claim** with per-row visibility from the job's
|
||||
stamps: exactly-once handout under concurrency (POC-pinned shape),
|
||||
`attempts += 1` per claim, claim ordering `priority DESC, ready-time
|
||||
ASC, enqueue order`, claimant column stamped, `claimed_at` +
|
||||
`claim_expires_at` set (ADR-019 §3's `Job` fields; ADR-021 §2).
|
||||
- **Savepoint-guarded retry/fail/dead-letter** (ADR-010 §1–§4): the
|
||||
DELETE→INSERT dead-letter moves are savepoint-hardened (the #133
|
||||
defect class — a mid-flight error must strand the row in *neither*
|
||||
table); retry computes nothing itself (the engine passes the
|
||||
resolved delay in — the curve is engine-layer arithmetic per
|
||||
ADR-012 §2); budget exhaustion and `fail` move to dead with
|
||||
`last_error`/`died_at`; the pinned default strings (`"failed"`,
|
||||
`"max attempts exceeded"`) are engine-layer (contract-derived
|
||||
strings do not belong in the contract-blind substrate — pass them
|
||||
in).
|
||||
- **Both-states no-stranded-rows `sweep_expired`** (ADR-010 §5): moves
|
||||
every past-`expires_at` row (pending and processing) to dead with
|
||||
`last_error='expired'`, and enforces `dead_letter_retention_s`
|
||||
deletion. Single-statement atomic per queue.
|
||||
- **Dead-visible `get_job`** (ADR-010 §1): reads dead rows with
|
||||
stamps, `claimed_at`, `last_error`, `died_at` — the zombie hole and
|
||||
dead-visibility gap are the fork's motivating fixes; both land here.
|
||||
- **Handle-op validity predicate support** (ADR-010 §2): the
|
||||
ack/heartbeat/retry/fail transitions carry the uniform predicate
|
||||
(row `processing` + caller's claim deadline unexpired) — false, not
|
||||
error, on refusal; the ack-vs-reclaim race resolves atomically.
|
||||
- **Scheduler tick over the new enqueue** with `@every` next-boundary
|
||||
math (numeric, a few lines — cron machinery not ported): the
|
||||
`__alkstore_scheduler_tasks` storage, boundary advance + fire
|
||||
enqueue + row-locked advance, the 64-cap catch-up (ADR-009 §4). The
|
||||
leadership-lock loop pattern is ported with the lock machinery; the
|
||||
boundary math is the re-derivation.
|
||||
|
||||
**Schema:** the `__alkstore_*` job/dead/scheduler table family with the
|
||||
stamp columns and `claimed_at` (ADR-012 §5 — the re-derivation adds
|
||||
them); partial indexes matching the claim hot path, dead rows outside
|
||||
it, single clock source (second-precision timestamps), per queues.md
|
||||
§Namespaces.
|
||||
|
||||
**Tests:** the contract-property floor — no-stranded-rows, dead-visible
|
||||
`get_job`, stamps immutability, claim exclusivity under concurrency,
|
||||
the validity predicate's refusal boundaries, savepoint hardening
|
||||
(mid-flight-error stranding test), scheduler boundary/catch-up math.
|
||||
Plus the inherited queue-machinery tests adapted where applicable.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] All re-derived ops above present with contract-blind primitive
|
||||
APIs (no contract types, no error-taxonomy types in the
|
||||
substrate)
|
||||
- [ ] Savepoint hardening proven by test: a forced mid-flight error in
|
||||
the dead-letter move strands no row in either table
|
||||
- [ ] No `.ok()`-style error swallows (the D-class defects); all
|
||||
substrate errors propagate typed
|
||||
- [ ] Claim exclusivity under concurrent claims (multi-connection
|
||||
test); reclaim consumes an attempt; validity predicate refuses
|
||||
lapsed-deadline ops as values (false), not errors
|
||||
- [ ] `sweep_expired` moves both states; retention deletion enforced;
|
||||
no-stranded-rows property test green
|
||||
- [ ] Scheduler: boundary advance + fire atomic under row lock;
|
||||
64-cap catch-up; `@every` math unit-tested (s|m|h|d)
|
||||
- [ ] `get_job` sees dead rows with full stamp/error fields
|
||||
- [ ] `cargo test -p alkstore-sqlite`, clippy `-D warnings`, fmt clean
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/decisions/010-queue-semantics-depth.md (§1–§6, §8)
|
||||
- docs/architecture/decisions/009-scheduler-collapse.md §3–§4
|
||||
- docs/architecture/decisions/012-forked-substrate-design.md §2, §5
|
||||
- docs/research/quality-read-honker-core.md §3 (defect register), §6
|
||||
- docs/research/reference-honker-machinery.md (the lineage mechanics, file/line cites)
|
||||
- docs/architecture/queues.md (the semantics this realizes)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
Reference in new issue
Block a user