alkdev/alkstore
6
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
49face898d0db24fbba061113494cf410da22abd
alkstore/docs/architecture/core-contract.md
T

46 KiB
Raw Blame History

status: draft last_updated: 2026-10-08 (ADR-023 — fourth review round: numeric-argument domains pinned by kind, encode_payload typed, plain-path SQLite open, watcher cadence posture)

Core contract

The unified, engine-agnostic surface a Store exposes. This document specifies WHAT the contract is; the per-engine specs map it onto their machinery; ADRs carry the WHY. The starting artifact is the honker-rs surface (the Phase 0 §Interface finding) scoped to the inventory-confirmed features — ADR-002 — and pinned against it. The base surface is pinned by ADR-008 (contract v1 partition, TxHandle representation, wake type, reserved strings, error taxonomy, config split — amended in place, pre-implementation, by ADR-014: outbox_enqueue_tx joins the TxHandle trait, and by ADR-015: streams depth — key semantics, StreamEvent shape, the ordering row, trim_to, and publish_with_key_tx); ADR-009/ADR-010 add the first extensions beyond ADR-008's original v1 partition (scheduler collapse surface, QueueOpts depth — factually pre-release additions per ADR-017's class 1, shipped inside the initial contract v1 text; versioning discipline for the surface's changes is ADR-017's, resolved: the core crate's semver is the contract version); ADR-016 decides the parked capability-surface question: none, by default ever — engine differences surface at compile time (engine-crate identity) and in the deployment matrix, never as a runtime descriptor; the obligations are this document.

Concepts

  • Store — what a consumer opens from a connection string or file path (ADR-001): one handle, the engine behind it chosen at open time. Consumer code never branches on engine type (guiding principle 4).
  • Mechanism — one of the surface's coordination families: notify, streams, queues (+outbox), locks, scheduler. Delivery guarantees are pinned per mechanism in ADR-006's table (notify/streams/queues), extended with the locks row by ADR-008, the scheduler row and collapse by ADR-009. (Guiding principles are defined in docs/research/phase-0.md §Vision and cited by number throughout this directory.)
  • Wake — the opaque "something changed, re-read" signal the wake contract delivers (ADR-006). The contract's central abstraction.
  • TxHandle — the caller-held transaction object whose *_tx methods make side effects commit-atomic with a business write (ADR-007). Contract shape: the *_tx methods live on the TxHandle trait itself — engines implement it for their concrete handle, no downcast (ADR-008 §2).

The seam

Per ADR-008 §6, the trait is constructed by engine crates (durability knobs, pool sizing, watcher cadence are engine options — deployment.md carries the facts); the contract is the trait surface it returns:

Store::begin_tx() -> Box<dyn TxHandle + Send>

trait TxHandle {
    enqueue_tx(name, opts, payload) -> job_id
    publish_tx(stream, payload) -> offset
    publish_with_key_tx(stream, key, payload) -> offset
    notify_tx(channel, payload)
    save_offset_tx(stream, consumer, offset)
    get_job_tx(queue, job_id) -> Option<Job>              // ADR-021
    get_offset_tx(stream, consumer) -> i64                // ADR-021
    read_since_tx(stream, offset, limit) -> Vec<StreamEvent>            // ADR-021
    read_from_consumer_tx(stream, consumer, limit) -> Vec<StreamEvent>  // ADR-021
    outbox_enqueue_tx(outbox, opts, payload) -> job_id
    commit(self: Box<Self>) -> Result<()>   // or rollback
}

The tx read methods (get_job_tx / get_offset_tx / read_since_tx / read_from_consumer_tx, ADR-021 §1) make read-your-own-writes possible inside the business transaction — they take the mechanism name (mirroring enqueue_tx/publish_tx's name-taking shape), read the caller's own transaction's rows, and are pure reads (no ops ride them). Reads and saves are tx-shaped; claim/maintenance ops never are (no claim_tx — a claim's visibility deadline must not be tied to a business transaction's lifetime; deliberate absence, not an omission). Dropping a handle without commit/rollback rolls it back (ADR-021 §4) — the no-ghosts property holds through RAII paths, not only the explicit ones.

outbox_enqueue_tx takes the outbox name and derives the backing queue engine-side (the reserved prefix makes the derived name unreachable by enqueue_tx by design — ADR-014). publish_with_key_tx validates the key like the non-tx form (a present key must be non-empty → InvalidName) and is publish_tx's keyed twin — commit-atomic keyed publishes are the point (ADR-015 §2).

Mechanism handles come off the store (or, for transactional variants, off the handle). Payload encoding (ADR-020 §4, signature typed by ADR-023 §1): the trait crosses serde_json::Value; engines serialize it with serde_json via core's encode_payload (fallible — Codec on serialization failure, typed at the helper; never a silent fallback form) and store exactly those bytes — payload_as<T> decodes them, Codec on failure; round-trip (publish → read → decode → Value equality) is what the contract suite pins, on both engines.

Numeric-argument domains (ADR-023 §2) — classified by argument kind, uniform on both engines (no engine may reinterpret a caller-supplied numeric argument via SQL-dialect semantics; validation happens at the trait-impl entry point, before the engine's storage layer):

  • Extents ("at most this many": claim_batch(n), read_since(limit), read_from_consumer(limit)) — non-positive values yield the empty value result (a value, not an error — the no-work vocabulary); the SQLite LIMIT -1 dialect artifact (a negative extent reaching the whole queue/stream) is dead by this rule.
  • Durations (try_lock(ttl), renew(ttl)) — non-positive values are rejections (Err, opaque Database, detail in the source chain); an exclusion mechanism must not silently hand back a lease the caller believes it holds.
  • Boundaries/offsets (trim_to(horizon), enqueue delay/ run_at/expires) — total: any i64 is well-defined (a negative horizon deletes nothing; a negative delay resolves to a ready time in the past, i.e. ready now).
store.notify(channel, payload)          handle.notify_tx(channel, payload)
store.stream(name) -> StreamHandle      handle.publish_tx / publish_with_key_tx
                                        handle.save_offset_tx
                                        (the StreamHandle also carries
                                        trim_to — stream-side, outside the
                                        tx seam)
store.queue(name, opts) -> Queue        handle.enqueue_tx
store.outbox(name) -> Outbox            handle.outbox_enqueue_tx(outbox, opts, payload)
store.try_lock(name, owner, ttl) -> Option<Lock>
store.listen(channel) -> Box<dyn WakeReceiver>
store.schedule(name, spec, queue, payload, opts) -> Result<Schedule>
store.unschedule(name) -> bool
store.run_schedules(stop: StopToken) -> Result<()>

The mechanism handles' full trait surfaces (Queue { enqueue, claim_one, claim_batch, ack_batch, cancel, get_job, sweep_expired }, StreamHandle { publish, publish_with_key, read_since, read_from_consumer, save_offset, get_offset, trim_to, subscribe }, Outbox { enqueue, run_once }, Lock { renew, release }, plus the boxed JobHandle a claim returns) are pinned by ADR-019; the StopToken parameter is a core-crate type (same ADR, §4).

Payloads cross the trait as core value types (non-generic trait methods — object safety of the boxed handles, ADR-008 §2); payload_as<T> decodes on concrete returned values (Job, StreamEvent). The with_tx closure wrapper ships over the handle shape (the POC-verified composition direction, ADR-007; signature pinned there 2026-10-07: with_tx(f) -> Result<()> with f: FnOnce(&mut dyn TxHandle) -> BoxedFuture<Result<()>> — Ok ⇒ commit, Err/drop/panic ⇒ rollback; non-generic for object safety, values return through captured state, no handle escapes the closure). Mechanism handles (Queue, StreamHandle, Outbox, Lock) are core-owned trait objects like the tx handle — engine types never appear in consumer signatures. Their trait surfaces are pinned by ADR-019 (placement mirroring the honker-rs starting artifact: claim/maintenance ops on the Queue handle, publish/read/trim on the StreamHandle, the claim ops on the boxed JobHandle, run_once on the Outbox); trait-method additions on these traits post-release ride ADR-017 class 2 like any other trait surface.

Mechanism contracts

notify / listen

Fire-and-forget signals, commit-atomic when sent in a transaction (ADR-007). No durability, no replay, no per-listener retry; a listener attached after a commit never sees it.

  • notify(channel, payload) — payload ≤ 8000 bytes on Postgres (client-side checked, typed PayloadTooLarge before the round trip, verified by POC #2; the measured quantity is the serde_json serialization of the payload Value — ADR-020 §4's stored-bytes form, so check and storage measure one thing); no limit on SQLite. The error variant is contract-wide (callers match it identically on both engines), the occurrence is the documented engine asymmetry (ADR-008 §5) — with no capability surface (ADR-016), this variant is the one runtime carriage of an engine asymmetry; its occurrence asymmetry is pinned in the verification backlog below.
  • listen(channel) -> Box<dyn WakeReceiver> — starts from "now"; delivers opaque Wake { channel } signals (ADR-006), never payloads or ids (ADR-008 §3). Wakes are best-effort hints, not per-commit delivery promises: possibly coalesced (SQLite) or per-notify (Postgres), possibly repeated after reconnect, possibly absent entirely (SQLite burst coalescing; the pg no-replay hole) — recovery is the consumer's re-read plus the engine's reconnect surface, never a guaranteed deliver. Consumers must be idempotent on wake.
  • Failure surfaces as channel events, never silence: watcher death closes the receiver (recv() -> None, SQLite); the synthetic reconnect-wake (a reserved channel (ADR-004)) covers Postgres connection gaps.
  • Channel name is the one piece of semantic content a wake carries — the invalidation key for caching subscribers.

streams

Durable pub/sub with per-consumer offsets (ADR-006). The durable cousin of notify: publish is commit-atomic; every committed event is readable by every consumer whose offset hasn't passed it; replay-on-attach is the default; offsets are explicit and transaction-aware (save_offset_tx gives exactly-once-within-a- business-tx shape). Depth pinned by ADR-015.

  • publish / publish_tx / publish_with_key / publish_with_key_tx — append to the stream's durable log; return the assigned offset. Key semantics (ADR-015 §1): the key is carried metadata — stored on the event row, round-tripped on every read, Option<String>-shaped on StreamEvent — with no engine-enforced behavioral role: the ordering guarantee stays global FIFO by offset within the stream; per-key in-order reading is the documented emergent pattern (filter by key, read in offset order), not a server promise. A present key must be non-empty (InvalidName). Server-enforced per-key ordering re-enters only via a consumer-inventory row naming it.
  • StreamEvent shape (ADR-015 §3): { offset: i64, stream: String, key: Option<String>, payload: Vec<u8>, created_at: i64 } — stream renames honker's topic (one term everywhere, the ADR-008 §4 kinds list); offset is engine-assigned, monotone per stream, immutable, never renumbered (gaps after a trim are legal); created_at is unix-seconds-at- publish, informational, not an ordering field (offset is the only one); payload_as<T> decodes (the error is Codec).
  • Ordering guarantee row (ADR-015 §4, extending ADR-006 §2's streams row): read_since/subscribe yield offset ASC — global FIFO per stream; committed-event visibility and replay-on-attach as already pinned; offsets immutable. The contract suite pins cross-engine equivalence against this row (same publish sequence → same offset sequence → same read order).
  • read_since / read_from_consumer(offset) — cursor-based reads; get_offset(consumer) — checkpoint inspection. A read from a trimmed-away offset region (below) resumes at the trim horizon's first remaining row.
  • save_offset / save_offset_tx — consumer checkpoint, explicit; the contract's save is always explicit (no auto-checkpoint cadence — honker's per-binding ambiguity is not inherited). The subscription handle exposes no save_every / auto-save-on-drop (ADR-008 §8). Saves are monotone — a save below the stored checkpoint is a silent no-op — so the direct form and the receiver's no-arg form (EventReceiver::save_offset(&mut self), which saves the last-yielded event's offset through the same op, ADR-019 §6) interleave freely and cannot regress one another. Both save forms' Err arm carries Database only (ADR-021 §5) — regression refusal is a silent no-op, so a save result errs only on storage failure. A saved offset below the trim horizon (ADR-015 §5) stays a valid position marker.
  • trim_to(horizon) -> count — delete events with offset <= horizon (exact-boundary; the horizon is a boundary argument, total per ADR-023 §2 — a negative horizon deletes nothing, idempotently) (ADR-015 §5; the return is the deleted-row count, ADR-019). The stream-side bounded-growth op, consumer-invoked, no engine-default retention and no ambient sweeper (the ADR-010 §6 posture; the replay-forever default is the mechanism's purpose — growth is documented with the tool in-contract to bound it). Trim emits no dedicated wake and no notify — on SQLite the data_version watcher may still fire (any committed write bumps it; a spurious hint, contract-legal); surviving events keep their offsets.
  • subscribe(consumer) -> Box<dyn EventReceiver> — durable consumption: attach, read to current tail, resume after restart from the stored offset; explicit save_offset on the receiver (shape in ADR-008 §8). Error and close arms pinned (ADR-021 §5): recv()'s Err arm carries Database only (payloads cross raw; Codec lives at payload_as<T>; Closed is the None arm); close is terminal and its cause is the documented engine asymmetry — SQLite's receiver closes on watcher death (POC-pinned), the pg receiver stays open across forwarder reconnects (reconnect wakes arrive on it) and closes only at engine shutdown. A closed receiver never reopens; recovery is a fresh subscribe(consumer) resuming from the saved offset. Consumption is wake-driven with table re-read — the same mechanism split as queues (durable row, LISTEN/watcher wake, ADR-006); the receiver's engine-side trigger is engine-internal, but events never require polling to become visible (the queues row's pg re-poll safety net is an admission of LISTEN loss, not the posture).

queues

Durable at-least-once work (ADR-002). Depth pinned by ADR-010; contract-level obligations:

  • enqueue / enqueue_tx — commit-atomic; EnqueueOpts { delay, run_at, priority, max_attempts, expires }; field types pinned (2026-10-07, third review round follow-through): delay, run_at, expires are Option<i64> (honker-rs parity, packages/honker-rs/src/lib.rs:439-447 — unset is distinguishable from zero, and the resolution rules below are the unset behavior), priority is a plain i64 with default 0 (the claim-order key; the row schema's DEFAULT 0), and max_attempts is Option<i64> with None = the queue-level stamp decides (the §3a override; no honker counterpart — our field). Every field has a default-resolved meaning, so all are constructor-settable with ..Default::default(). Queue-level QueueOpts { visibility_timeout_s, max_attempts, backoff_base_s, dead_letter_retention_s } (ADR-010 §3/§3a): plain i64 fields (dead_letter_retention_s: Option<i64>, None = forever) with a Default providing the pinned defaults 300/3/5/none (honker parities except retention, which is ours); ScheduleOpts { priority, max_attempts, expires } types identical to their EnqueueOpts twins. Option semantics (ADR-020): ready time resolves delay → now + delay (wins) over run_at → literal absolute over neither → now; expires is relative seconds from enqueue resolved to an absolute row expires_at (None = never); the resolved values — not the raw fields — are what get_job shows. The QueueOpts stamps land on each job row at enqueue (§3a), resolved per ADR-020 §3 by call shape: the auto-commit Queue::enqueue (the one handle-carrying shape) resolves over that handle's opts; enqueue_tx (a TxHandle carries no queue opts) and every other no-handle-open path — outbox backing queues, scheduler boundary fires — stamp the engine's derived defaults (plain 300/3/5/none; the outbox's 60/5/5 set).
  • claim_one / claim_batch — exactly-once handout under concurrency (POC-pinned on both engines); take a caller-supplied worker_id (claimant identity — consumer-local, no reserved rule, non-empty; ADR-019 §2); claim ordering: priority DESC, then ready-time, then enqueue order (FIFO under equal priority). Claims return boxed JobHandles — ops + row value (same ADR, §3). claim_batch(n)'s n is an extent (ADR-023 §2): n <= 0 claims nothing (the empty Vec, a value — never an error, never the whole queue).
  • Job handle: ack / retry / fail / heartbeat on the boxed JobHandle a claim returns (ADR-019 §3 — job(&self) -> &Job beside the ops; one-shot self: Box<Self> ops like the tx commit). The one-shot/repeatable split is deliberate: ack/retry/fail consume the handle (each is a terminal transition of the claim); heartbeat(self, extend) takes &self and is repeatable — renewal is inherently repeated within one claim; do not "fix" this asymmetry at implementation (ADR-019 §3). ack deletes the row; heartbeat(extend) is renewal — an absolute reset of the claim deadline (extend is the new full deadline from now, not additive to elapsed time); late heartbeat refused; a reclaim consumes an attempt; retry(err, None) computes the queue's equal-jitter exponential delay (range pinned by ADR-010 §3), retry(err, Some(d)) overrides; fail = immediate dead-letter. Handle-op validity (uniform predicate): an op succeeds only while the row is processing and the caller's claim deadline is unexpired — deadline lapse refuses all handle ops (heartbeats, acks included) and a reclaim does too; the dual-execution window's worker may still complete its work but its ack will not land (at-least-once, as documented); false, not error (ADR-010 §2). Dead letters: move-to-dead storage, get_job sees dead rows (with last_error/died_at), retention via dead_letter_retention_s (default forever), no redrive API (ADR-010 §1–§4). QueueOpts stamp onto the job row at enqueue (§3a) — queues are names, not config owners. The remaining v1-skeleton ops (ack_batch — the batch ack applies the uniform predicate per id, lapsed or non-claimed ids silently not counted; cancel — unconditional delete, not an interrupt) pin in ADR-010 §1.
  • sweep_expired() — moves every past-expiry row (any state) to dead + enforces dead-letter retention — the no-stranded-rows property (ADR-010 §5); the queue-scoped handle form (ADR-019 §1 — the name rides the handle); cadence recipe in queues.md (no ambient sweeper).

named locks

TTL-bounded coordination locks. Lock rows live in the same database as the caller's business data — coordination and data co-locate — but lock ops are not part of the tx seam: there is no lock_tx; a lock held across a business transaction is held by explicit acquisition before/inside it, not transaction-scoped (acquisition is a separate auto-commit op; rollback of the business tx does not release it — the TTL discipline governs).

  • try_lock(name, owner, ttl) — acquire or fail (Option<Lock> — no-work is a value, not an error); renew and release on the lock handle. Release on explicit unlock or TTL expiry. Re-acquirable after expiry (POC-pinned on Postgres; on SQLite it rests on the forked substrate's lock machinery — the SQLite-side pin is in the verification backlog below). ttl <= 0 is a rejection (duration kind, ADR-023 §2) — opaque Err, never a silent immediately-expired lease.
  • Guarantee row (ADR-008 §7): mutual exclusion bounded by TTL + renewal — after TTL expiry exclusion lapses silently (no revocation event); holders must renew within TTL; expiry is a loss of exclusivity, not an error.
  • Lock names are shared-namespace (reserved-prefix rules below, ADR-008 §4).

outbox

A helper over queues, not a separate mechanism: enqueue inside the business transaction + the delivery/consumption worker entry points. Same evidence base and guarantee as queues (ADR-002). Worker semantics (honker's run_once shape, inherited as the pinned posture; the delivery callable's shape pinned by ADR-019 §5): run_once(worker_id, delivery) -> bool is a pull op — claim one job, run the delivery closure, ack on Ok, retry(err, None) (the queue's curve) on Err; the consumer calls it in its own loop. No engine-issued heartbeat inside delivery (honker parity, inherited deliberately — the helper does not auto-heartbeat on the caller's behalf; the boxed JobHandle's own heartbeat remains available to the delivery closure, and renewal inside the deadline is the consumer's obligation): delivery slower than the backing queue's stamped visibility timeout (default 60 s for outbox-backed queues) can have its claim expire mid-delivery and be redelivered — the dual-execution window; idempotent delivery is the consumer's obligation, same as queues.

The transactional enqueue side is outbox_enqueue_tx(outbox, opts, payload) on the TxHandle trait (ADR-014): it takes the outbox name (validated like store.outbox(name) — empty → InvalidName, reserved-prefixed → ReservedName), derives the backing queue name engine-side, and lands the job row in the caller's transaction with EnqueueOpts stamped per ADR-010 §3a over the backing queue's derived QueueOpts. Commit makes the job visible to run_once exactly when the business write commits; rollback drops both (the no-ghosts property, ADR-007). The derived backing queue name is not expressible through enqueue_tx (the reserved prefix is rejected on directly-supplied names) — outbox_enqueue_tx is the only transactional path into it, which is itself the guarantee that the derivation cannot be collided with.

scheduler

Collapsed into queues per ADR-009: schedule(name, spec, queue, payload, opts) (upsert by name), unschedule(name), and the opt-in run_schedules(stop: StopToken) runner (StopToken is a core-crate cloneable cancel handle — ADR-019 §4; the leader-elected leadership lock is the reserved name __alkstore_scheduler). Both name-bearing arguments validate at the entry point: the schedule name like a queue name (non-empty, reserved-prefix rejected — ADR-009 §1) and, since ADR-021 §3, the queue argument identically (InvalidName/ReservedName) — a schedule cannot target a reserved name, so the outbox backing queue stays reachable only through the outbox surface (ADR-014). Schedules never fire without a runner (no ambient timers). v1 spec grammar: @every <n><unit> only (s|m|h|d); cron strings are rejected (InvalidSpec) pending a consumer-inventory row naming wall-clock cron. Each boundary fire enqueues ordinary work stamped with ScheduleOpts { priority, max_attempts, expires } (ADR-009 §3) — resolved over the target queue's derived engine defaults (visibility/backoff/ retention; no handle is open), per ADR-020 §3. Boundary guarantee: ADR-009 §4 (at-least-once per elapsed boundary, bounded catch-up with skip-forward past the cap).

Cross-cutting contracts

Delivery-guarantee table

The per-mechanism table in ADR-006 is the contract of record for notify/streams/queues; the locks row is ADR-008 §7's and the scheduler row is ADR-009 §4's. This spec inherits them and adds the consumer obligations:

  • wake idempotence (notify's best-effort, coalescing behavior);
  • explicit offset saves (streams);
  • visibility-timeout budgeting — heartbeat inside the deadline for long work; the dual-execution window and reclaim-eats-attempt rules are documented consumer obligations (ADR-010 §2);
  • running run_schedules for schedules to fire at all, and scheduling sweep cadences via the collapse recipe — timeliness is the consumer's, correctness-of-transition the engine's (ADR-010 §6);
  • *_tx operations are only durable after the caller's commit — rollback drops job rows, event rows (keyed and unkeyed alike), notifications, and offset saves together (the no-ghosts property, POC-pinned on both engines).

Errors

thiserror-typed, per the family standard. Pinned by ADR-008 §5: one top-level Error, with v1 variants PayloadTooLarge (universal; produced pg-side, contract-wide matchable), ReservedName, InvalidName, Closed, Codec, and the opaque Database fallback (engine detail preserved via the source chain). InvalidSpec (schedule spec grammar) and LeadershipLost (run_schedules return on leadership loss) — both from ADR-009 §6 — are part of contract v1's initial text (pre-release additions per ADR-017 class 1; the "post-v1 additions" framing an earlier draft used is retired). Pinning rule: a variant exists only when callers can act differently on it. Queue claim "no work" and job/lock boolean results are values, not errors; dead-letter is observable state (get_job), not an error.

Capability surface

None — decided by ADR-016 (2026-10-06): Store carries no capabilities accessor, in v1 and by default ever. The honest single-host/multi-host boundary lives at compile time (the engine-crate dependency is the deployment statement — ADR-001's single-driver binaries) and in deployment.md's matrix. Runtime carriage of the one caller-actionable engine asymmetry is the contract-wide matchable PayloadTooLarge variant (Errors above); re-entry requires a consumer-inventory row naming a runtime-adapt need.

Naming / reserved namespace

Consumer-visible names (channels, streams, queues, outboxes, locks, and schedule names) share engine-visible namespaces on Postgres (LISTEN channel names are server-global per database). The canonical kinds list is exactly these six (notify/stream/queue/outbox/lock/schedule; pinned by ADR-008 §4, outbox added by ADR-014, schedule names by ADR-009 §1):

  • Reserved prefix __alkstore_, engine-independent, applies across all name kinds; the one v1-reserved string is __alkstore_listener_reconnected__ (the Postgres reconnect-wake channel, ADR-004).
  • Reserved-prefix names are rejected at every entry point — the name-bearing methods and their *_tx counterparts — with the typed ReservedName error; empty names with InvalidName (ADR-008 §4 — enforced as non-empty-after-whitespace-exclusion per that ADR's wave-1 review annotation: whitespace-only names are InvalidName on both name classes; names are not trimmed). Stream-consumer names are consumer-local identifiers, not a reserved-namespace kind.
  • Engine-derived names in consumer namespaces carry the reserved prefix (the outbox's backing queue is derived under the prefix — honker's _outbox:{name} scheme is not inherited verbatim).
  • SQLite: the substrate's __alkstore_* internal table family is storage-internal (not consumer namespace) — re-owned from upstream's _honker_* by the fork (ADR-011, designed per ADR-012). Postgres: engine tables are schema-scoped — one engine-owned schema (ADR-010 §8) — the channel namespace is this section's.

Verification backlog

Contract properties POC-pinned on one engine only (or sketched rather than surface-verified) — the contract test suite must pin both engines before the engine specs are called stable:

  • Named-lock TTL/expiry re-acquisition on SQLite — pinned on Postgres (pg POC's lock probe); on SQLite it rests on the forked substrate's lock machinery (its lock_renew verified in the quality read; the re-acquire-does-not-refresh-TTL behavior is upstream's, inherited deliberately) — pin it in the contract suite.
  • Wake semantics under WakeReceiver shapes — POCs verified wake delivery/coalescing through their own probe types; the pinned Wake { channel } / recv forms (§3 of ADR-008) need the contract suite's own property tests on both engines.
  • save_offset_tx exactly-once-within-a-business-tx shape — both POCs verified it through their sketch implementations (SQLite's offset-save was a plain SQL upsert, not the forked substrate's full surface; the pg side likewise through its probe), so the property must be re-pinned against the real engines' save_offset_tx in the contract suite.
  • Concurrent try_lock loser/error behavior on SQLite (the pg side returns cleanly; the substrate's busy-path under lock contention is the thing to pin).
  • Scheduler + depth properties on Postgres — the SQLite side's tick/leader/catch-up machinery inherits the forked substrate's test-pinned implementation; the pg engine's re-derived tick (boundary advance, 64-cap skip-forward, leadership-loss discipline) and the ADR-010 depth properties (visibility reclaim consuming attempts, dead-letter moves, the no-stranded-rows sweep) pin in the contract suite at implementation.
  • Backoff-curve and stamp-resolution equivalence across engines — the equal-jitter curve (ADR-010 §3) and the opts-stamping resolution (§3a) are computed engine-side per ADR-012 §2's contract-blind boundary; the contract suite must pin both engines' arithmetic to identical outputs.
  • Job-handle validity predicate on both engines — the uniform processing-state + unexpired-deadline rule (ADR-010 §2): the late-heartbeat boundary (refused exactly when the deadline has lapsed), the post-lapse-ack-refusal (at-least-once), and the ack-vs-reclaim race (one wins atomically); ack_batch applies the predicate per id (lapsed or non-claimed ids silently not counted); retry's refusal is false with the row untouched; the engine-default strings land identically (fail(None) → "failed", exhaustion → "max attempts exceeded"); both engines' SQL pin identical outcomes in the contract suite.
  • outbox_enqueue_tx commit-atomicity on both engines (ADR-014) — rollback drops the backing-queue job row together with the business write (no ghost job); commit makes it claimable by run_once; reserved/empty outbox names rejected identically on the tx path (ReservedName/InvalidName); stamped opts visible via get_job equal for both engines.
  • Cross-engine stream equivalence (ADR-015 §3/§4) — publish sequence → identical (per-stream) offset sequence → identical read_since output order on both engines, direct and subscriber reads alike (offsets are global counters engine-side — pg bigserial, SQLite AUTOINCREMENT — so equivalence is per-stream relative order, never absolute offset values); keyed/unkeyed interleavings preserve global FIFO; key round-trips exactly (None vs Some), stream/created_at fields equal.
  • publish_with_key_tx commit-atomicity on both engines (ADR-015 §2) — rollback drops the keyed event row with the business write (no ghost event); commit makes it visible to read_since/subscribe; empty-Some-key InvalidName on both engines' tx paths.
  • PayloadTooLarge occurrence asymmetry (ADR-016 §5) — with no capability surface, PayloadTooLarge is the one runtime carriage of an engine asymmetry, so its matchability pins in the suite: an oversized notify/notify_tx payload is rejected client-side by the Postgres engine before any round trip (the limit in the variant), and the SQLite engine never produces the variant at any size — engine-agnostic code writes the same match on both engines and the non-occurring arm simply never fires.
  • trim_to semantics on both engines (ADR-015 §5) — exact-boundary trim (<=), surviving rows keep their offsets (gaps legal, never renumbered), reads resume at the trim horizon's first remaining row, saved offsets below the horizon stay valid, trim wakes nothing, and a concurrent subscriber never loses its place.
  • Save-offset monotone composition (ADR-019 §6) — direct-form and receiver-form saves interleave without regression on both engines; a lower-offset save is a silent no-op; receiver saves land the last-yielded event's offset exactly.
  • Payload encode/decode round-trip on both engines (ADR-020 §4) — same Value published/enqueued on both engines stores byte-identical rows and decodes back to Value equality (Codec only on non-serializable target types).
  • Enqueue-opts resolution equivalence (ADR-020 §1/§2/§3) — delay-over-run_at precedence, relative-expires resolution, and the scheduler-fired jobs' derived-default stamps (get_job-visible via the row) are identical on both engines; plain enqueue_tx stamps the same plain-queue derived defaults (300/3/5/none) on both engines — auto-commit-handle enqueues stamp the handle's opts, no-handle-open shapes stamp derived defaults (the outbox's 60/5/5 set being the one exception, ADR-014 §1).
  • In-tx read-your-own-writes on both engines (ADR-021 §1) — reads issued on the tx handle see the transaction's own writes (get_job_tx a row the same tx enqueued; read_since_tx an event the same tx published); another engine connection sees none of it pre-commit; post-rollback the reads' subjects are gone (no-ghosts for reads).
  • Drop = rollback on both engines (ADR-021 §4) — a handle dropped before commit leaves no job/event/notify/offset residue (no-ghosts via RAII); afterward the SQLite writer slot is reusable and the pg client re-poolable.
  • schedule() queue-argument validation on both engines (ADR-021 §3) — reserved-prefix and empty queue arguments rejected at registration with ReservedName/InvalidName; stored schedules fire only into non-reserved queue names.
  • Receiver error/close arms on both engines (ADR-021 §5) — recv()'s Err carries Database only; save_offset's Err carries Database only; regression saves are silent no-ops; SQLite's receiver closes on watcher death (terminal None); the pg receiver stays open across forwarder reconnects (reconnect-wakes arrive; no close) and closes only at engine shutdown.
  • Numeric-domain semantics on both engines (ADR-023 §2) — extents clamp to empty (claim_batch(n <= 0) claims nothing; stream reads with limit <= 0 read nothing, never the whole queue/stream), so the SQLite LIMIT -1 dialect artifact is dead on both engines; durations reject (try_lock/renew with ttl <= 0 are opaque Errs on both engines); boundaries are total (a negative trim_to horizon deletes nothing, idempotently).
  • encode_payload typed failure (ADR-023 §1) — the helper returns Result<Vec<u8>, Error::Codec>; enqueue/publish store and decode the exact serialization of the input Value (byte-identical rows cross-engine, the §4 round-trip unchanged), and the failure arm is typed at the helper — no silent empty-byte fallback exists.
  • Plain-path SQLite open (ADR-023 §3) — the SQLite engine's open path argument is a plain filesystem path: a string carrying a ?name=value suffix is treated as a filename, no URI interpretation, no connection- semantics mutation outside the constructor's opts.
  • Watcher cadence default + knob (ADR-023 §4) — the SQLite store's watcher runs at the inherited 1 ms default; SqliteOpts::poll_interval changes the configured cadence (config-surface accounting — wall-clock cadence measurements stay out of the contract suite's deterministic posture).

Design Decisions

ADR Decision Summary
001 Crate split core + per-engine crates; single-driver binaries
002 Feature scope inventory-confirmed features only; cut-flags explicit
006 Wake contract opaque wake + re-read; notify-vs-streams guarantee split
007 Tx seam caller-held handle, *_tx methods, native commit-atomicity
008 Contract v1 pinning surface partition, TxHandle trait shape, Wake type, reserved strings, error taxonomy, config split, locks guarantee row
009 Scheduler collapse (pre-release extension) queues + schedule()/run_schedules, @every-only v1, boundary guarantee row
010 Queue depth (pre-release extension) visibility/renewal, opts stamping, backoff curve, dead-letter, sweep, layout
011 Substrate fork SQLite substrate owned (__alkstore_* naming); queue ops re-derived on contract v1
012 Fork design contract-blind substrate boundary; engine-side formula arithmetic pinned equivalent by the contract suite
014 Outbox tx enqueue (amends 008) outbox_enqueue_tx on TxHandle; outbox-name validation; derived backing queue reached only through the outbox surface
015 Streams depth (amends 006/008) key = carried metadata, global-FIFO ordering row, StreamEvent shape, publish_with_key_tx, trim_to
016 Deployment honesty (decides 008's parked question) no runtime capability surface — compile-time engine identity + documented matrix; PayloadTooLarge occurrence asymmetry pinned
017 Contract versioning (governs this surface's changes) core crate's semver is the contract version; four change classes; amend-in-place ends at first release; pairing = manifest pin + version-stamped contract suite + docs
019 Mechanism handles (amends 008 §1/§2/§8) handle traits pinned (Queue/StreamHandle/Outbox/Lock/JobHandle); worker_id claimant identity; Job/Schedule struct shapes; core-owned StopToken; save-offset composition
020 Enqueue options + bridges (amends 008 §1/§2, 009 §3) delay wins over run_at; expires = relative seconds; scheduler stamps from derived queue defaults; serde_json byte-level payload encoding
021 Third review round (amends 008 §2/§8, 019 §1/§3, 009 §1) tx-side read methods on TxHandle (read-your-own-writes); Job.claimed_at restored; schedule() queue argument validated; drop = rollback; receiver close/error arms pinned
023 Fourth review round (amends 020 §4, 008 §5, 012 §4) numeric-argument domains by kind (extents clamp empty, durations reject, boundaries total); encode_payload typed (Codec); plain-path SQLite open (URI flag dropped); watcher cadence 1 ms default + SqliteOpts knob

Open Questions

Open questions are tracked in open-questions.md. Key questions affecting this document:

  • OQ-10: contract versioning discipline across engine crates — resolved (ADR-017): the core crate's semver is the contract version; change classes pinned; the verification backlog below matures into the version-stamped contract suite (the pairing instrument).

Resolved on this document's surface: OQ-09 (scheduler collapse — ADR-009), OQ-05 (queue semantics depth — ADR-010), OQ-13 (transactional outbox enqueue shape — ADR-014), OQ-12 (streams depth — ADR-015), OQ-08 (capability-surface shape — none, by default ever; ADR-016), and OQ-10 (the versioning discipline this document's surface is governed by — ADR-017), 2026-10-05/06. The second review round (2026-10-06) found no new open questions — its findings resolved directly as ADR-019 (handle surfaces, value shapes, stop token) and ADR-020 (enqueue-option semantics, scheduler stamp source, payload encoding). The third review round (2026-10-07, pre-decomposition) likewise found no new open questions — its five findings resolved directly as ADR-021 (tx-side reads, Job.claimed_at, schedule() queue-validation, drop = rollback, receiver close/error arms).

Reference in new issue View Git Blame Copy Permalink