docs: resolve OQ-04 — contract v1 pinned (ADR-008): surface partition, TxHandle-on-handle-trait, Wake/WakeReceiver, reserved __alkstore_ namespace, error taxonomy, engine-crate constructors, locks guarantee row

2026-10-05 02:23:15 +00:00
1 parent 4391f6e879
commit 7ad8ac56bc
@@ -0,0 +1,410 @@
+# ADR-008: Contract v1 surface pinning
+
+## Status
+
+Accepted (2026-10-05, Phase 1 — OQ-04's resolution; binds
+[core-contract.md](../core-contract.md); engine specs map it)
+
+## Context
+
+OQ-04 (promoted from OQ-ST-04) scoped the contract work: with both
+engines POC-verified ([ADR-003](003-sqlite-driver.md),
+[ADR-004](004-postgres-driver.md)) and the load-bearing decisions made
+([ADR-006](006-wake-and-delivery-contract.md) wake contract + guarantee
+table, [ADR-007](007-transactional-seam.md) caller-held tx seam), what
+remained was pinning the exact trait surface against the honker-rs
+starting artifact (`/workspace/honker`
+`packages/honker-rs/src/lib.rs` v0.5.0, the §Interface finding in
+phase-0.md). The unpinned items:
+
+- Which surface parts are contract v1 vs engine-extension.
+- `TxHandle` representation — the POC sketch used `dyn` + per-engine
+  `as_any_mut` downcast (two recorded frictions,
+  [ADR-007](007-transactional-seam.md)).
+- The reserved namespace's exact strings (POC used
+  `__listener_reconnected__` ad hoc).
+- The error taxonomy (e.g. is `PayloadTooLarge` universal when SQLite
+  has no 8000-byte limit?).
+- What `listen()` returns and where the honker-rs surface gets renamed.
+- The `Store::open` config split (contract vs engine-crate option).
+- Delivery-guarantee rows for locks and scheduler (ADR-006's table
+  covered notify/streams/queues only).
+- A verification backlog for properties POC-pinned on one engine only.
+
+All of this is paper over a complete evidence base: the honker-rs
+surface listing, both POC findings (including the trait sketches and
+their recorded frictions), the consumer inventory rows
+([ADR-002](002-feature-scope.md)), and the guiding principles
+(phase-0.md — engine-agnostic consumer API, no engine-type branching).
+
+## Decision
+
+### 1. Contract v1 partition
+
+**Contract v1** (the core crate's trait surface, both engines must
+implement identically):
+
+- notify / listen — `notify(channel, payload)`; `listen(channel)`.
+- streams — `publish`, `publish_with_key`, `read_since`,
+  `read_from_consumer`, `save_offset`, `get_offset`, `subscribe`
+  (durable log, explicit offsets — the auto-checkpoint ambiguity is
+  not inherited, per [ADR-006](006-wake-and-delivery-contract.md)).
+- queues v1 skeleton — `enqueue`, `claim_one`, `claim_batch`,
+  `ack_batch`, `cancel`, `get_job`, `sweep_expired`; job handle
+  `ack / retry / fail / heartbeat`; `EnqueueOpts { delay, run_at,
+  priority, max_attempts, expires }` — the honker-rs field set,
+  carried whole (the POC sketch's shape; `run_at` is the absolute-time
+  counterpart of `delay`, not dropped).
+- locks — `try_lock(name, owner, ttl) -> Option<Lock>` with `renew`
+  and `release`.
+- outbox helper — `outbox(name)` with `enqueue` + `run_once` delivery
+  worker.
+- the tx seam — `begin_tx` / commit / rollback and the `*_tx` methods
+  ([ADR-007](007-transactional-seam.md)).
+- `subscribe(consumer) -> Box<dyn EventReceiver>` — the durable
+  stream subscription handle, delivering `StreamEvent`s with an
+  explicit `save_offset` on the receiver (no auto-checkpoint;
+  §8 defines the shape).
+
+**Not in v1 — deliberately owned elsewhere:**
+
+- **Queue semantics depth** (retry/backoff curve, dead-letter
+  move-vs-flag, visibility-renewal mechanics, sweep cadence, `QueueOpts`
+  fields beyond the v1 skeleton) — OQ-05's design surface. v1 pins the
+  skeleton above; depth additions extend the contract later.
+- **Scheduler surface shape** (first-class mechanism vs queues +
+  `schedule()`) — OQ-09 decides; the scheduler surface (if any) is not
+  part of contract v1.
+- **Capability flags** — OQ-08 decides whether `Store` exposes them at
+  all; v1 has no capability surface.
+- **`claim_waker`** — dropped from the contract surface; wake-driven
+  claim is the engine's consumption posture
+  ([engine-postgres.md](../engine-postgres.md)'s LISTEN-driven claim),
+  engine-internal.
+- **Rate limits, result storage** — cut-flag rows
+  ([ADR-002](002-feature-scope.md)); re-enter only via a
+  consumer-inventory row.
+- **Notification payloads on wakes** — wake is opaque
+  ([ADR-006](006-wake-and-delivery-contract.md)); the payload carrying
+  honker's `Notification` type happens to provide is *not* surfaced in
+  the contract (see §3).
+
+### 2. `TxHandle` representation: the `*_tx` methods live on the handle trait
+
+The downcast friction dissolves by moving the ops, not by choosing a
+handle encoding. The POC sketches put `enqueue_tx` etc. on the
+engine/store trait taking `&mut dyn TxHandle`, forcing each impl to
+downcast the handle to its concrete type. The contract instead defines
+a **`TxHandle` trait carrying the `*_tx` methods directly**:
+
+```text
+trait TxHandle {
+    enqueue_tx(name, opts, payload) -> job_id
+    publish_tx(stream, payload) -> event_id
+    notify_tx(channel, payload)
+    save_offset_tx(stream, consumer, offset)
+    commit(self: Box<Self>) -> Result<()>   // or rollback
+}
+```
+
+*(Sketches elide `Result<>` wrappers on `*_tx` returns for brevity —
+every fallible op returns the taxonomy of §5; `notify_tx` can fail
+with `PayloadTooLarge`/`ReservedName` exactly like its auto-commit
+counterpart.)*
+
+- `Store::begin_tx() -> Box<dyn TxHandle + Send>` — callers hold the
+  boxed handle across await points.
+- Each engine implements `TxHandle` for its own concrete handle —
+  **no downcast, no enum, no generic parameters**. The friction the
+  POCs recorded disappears because there is no engine-store dispatch
+  of a handle argument; the handle *is* the object.
+- Object safety constraints that follow: no generic methods on
+  `TxHandle`, payload serialization is value-typed in core
+  (`serde_json::Value` — the type both POCs used; not generic over
+  `T: Serialize`); `commit`/`rollback` take `self: Box<Self>`
+  (dispatchable on a boxed trait object).
+- **Why not the enum** (one variant per engine): the enum's payload
+  types would make core depend on engine types — inverting ADR-001's
+  dependency direction (engines depend on core, never the reverse) —
+  or make the variants engine-opaque, which re-introduces downcast one
+  step further away. A new engine (a stated ADR-001 goal) must not be
+  a core-crate enum edit.
+- The generic-vs-dyn alternative buys static dispatch for a hot-path
+  cost the POCs measured as negligible against the seam itself (SQLite
+  hop 0.35 ms p50, pg 2.4 ms p50 — both dominated by commit costs);
+  it is the recorded optimization if ever needed (same posture as
+  ADR-003's dedicated-bridge optimization).
+- Async dispatch over a boxed handle uses the desugared boxed-future
+  form (the family-standard async trait posture); its per-op cost is
+  bounded by the same measurements.
+- Thread-affinity note retained verbatim: SQLite tx ops serialize
+  through the writer slot ([ADR-007](007-transactional-seam.md)) — a
+  documented engine note, not a contract item.
+
+### 3. Wake type and the `listen()` return
+
+`listen(channel)` returns a **wake receiver**, not honker's
+subscription:
+
+```text
+listen(channel) -> Box<dyn WakeReceiver>
+trait WakeReceiver {
+    recv() -> Option<Wake>          // None = source closed (watcher death)
+    try_recv() -> Result<Option<Wake>>
+    recv_timeout(d) -> Result<Option<Wake>>
+}
+
+struct Wake { channel: String }
+```
+
+- **`Wake` carries only the channel name** — the one piece of semantic
+  content [ADR-006](006-wake-and-delivery-contract.md) allows. Honker's
+  `Notification { id, channel, payload }` is not surfaced: the pg
+  engine would have to discard payloads to honor the opaque contract,
+  and honoring the contract uniformly is what makes consumer code
+  engine-agnostic. Consumers needing content use streams (the
+  guarantee split, ADR-006). The SQLite engine's payload transport is
+  a non-contract implementation detail.
+- Receiver close is the failure surface: SQLite watcher death closes
+  the receiver (`recv() -> None`,
+  [ADR-006](006-wake-and-delivery-contract.md)); the Postgres
+  reconnect-wake arrives as `Wake { channel:
+  Reserved::LISTENER_RECONNECTED }` on every subscriber's receiver
+  (a broadcast-to-all-subscribers fanout — engine behavior recorded in
+  [engine-postgres.md](../engine-postgres.md)).
+- `WakeReceiver` methods have no wake-count or id: wakes are hints,
+  their multiplicity is not part of the contract (coalescing on
+  SQLite, per-notify on Postgres — ADR-006). Close semantics across
+  the receiver's forms: `recv() -> None` means closed (source death);
+  `try_recv`/`recv_timeout` return `Err(Closed)` when the source is
+  closed, versus `Ok(None)` when merely no wake is pending right now
+  — closed and idle are distinguishable states.
+
+### 4. Reserved namespace — exact strings
+
+- **Reserved prefix: `__alkstore_`** (leading double underscore). The
+  namespace exists per [ADR-006](006-wake-and-delivery-contract.md);
+  this pins its string. Engine-independent — applies to channel,
+  stream, queue, and lock names on every engine.
+- **The one v1-reserved string:**
+  `__alkstore_listener_reconnected__` — the Postgres forwarder's
+  synthetic reconnect-wake channel ([ADR-004](004-postgres-driver.md)).
+  The POC's ad-hoc `__listener_reconnected__` string is renamed into
+  the namespace at implementation; nothing (consumer or contract text)
+  depended on the ad-hoc string.
+- Consumer names with the reserved prefix are **rejected at the entry
+  points** — the name-bearing surface methods (`notify`, `listen`,
+  `stream`, `queue`, `outbox`, `try_lock`) *and their `*_tx`
+  counterparts* (`notify_tx` at minimum; any `*_tx` op taking a name) —
+  with the typed `ReservedName` error (taxonomy, §5). Engine-side
+  validation obligation: reject before any engine round trip, on both
+  the auto-commit and tx paths (validation must not be bypassable by
+  the commit-atomic path). Stream-consumer names (`save_offset(_tx)`
+  consumer arguments, `subscribe(consumer)`) are *not* a reserved-
+  namespace kind — they are consumer-local identifiers, not shared
+  engine namespaces; no prefix rule applies (an engine may still quote
+  them).
+- Per-engine internal names:
+  - SQLite: honker's machinery owns two categories of internal names,
+    and the contract treats them differently. Its `_honker_*` *table*
+    family (`_honker_dead`, `_honker_locks`, …) is storage-internal —
+    not part of any consumer namespace; its fate rides the quality
+    read (OQ-06, [ADR-005](005-dependency-ownership.md)). Its
+    **consumer-namespace derived names** are a real collision surface:
+    honker-rs materializes an outbox's backing queue as
+    `_outbox:{name}` inside the queue-name namespace — a consumer
+    calling `queue("_outbox:foo")` directly could collide with
+    `outbox("foo")`. The contract's rule: **engine-derived names in
+    consumer namespaces carry the reserved prefix** — the SQLite
+    engine's outbox backing queue is `__alkstore_outbox:{name}`,
+    derived from the consumer's outbox name, and every entry point
+    rejects the reserved prefix for directly-supplied names (so a
+    consumer can never create or collide with it). Honker's
+    `_outbox:` scheme is not inherited verbatim.
+  - Postgres: queue/stream tables are schema-scoped to avoid
+    colliding with consumer tables (the co-tenancy precedent; layout
+    decision rides OQ-05's namespace bullet,
+    [queues.md](../queues.md)). The reserved *channel* namespace is
+    this section's; the outbox backing queue obeys the same
+    reserved-prefix rule as SQLite's.
+- Channel-name charset: the contract requires non-empty names and the
+  reserved-prefix rejection; identifier quoting for Postgres LISTEN /
+  `pg_notify` is the engine's obligation (POC #2 owned the quoting
+  pitfall — test-pinned).
+
+### 5. Error taxonomy
+
+One top-level `Error` in the core crate, `thiserror`-typed (family
+standard). **Pinning rule: an error gets its own matchable variant
+only when the caller can act differently on it** — engine-specific
+conditions that a caller cannot act on stay inside the opaque
+fallback with their detail in the source chain. This is the line that
+keeps the taxonomy small and keeps it from inheriting honker's
+per-binding sprawl.
+
+v1 variants (guaranteed-matchable on every engine):
+
+- `PayloadTooLarge { limit }` — **universal variant**, produced by the
+  Postgres engine's client-side check (8000 bytes, POC #2
+  verified-before-round-trip) and by contract never on SQLite (no
+  limit — the documented engine asymmetry). The variant lives in the
+  shared taxonomy so the limit asymmetry is visible to engine-agnostic
+  code — a caller can match it without knowing which engine is behind
+  the store, and the error itself documents the boundary. The
+  large-payload alternative (a table row with the id in the
+  notification — the outbox shape) is the documented workaround.
+- `ReservedName { name }` — rejected reserved-prefix names (§4).
+- `InvalidName { name }` — empty names and other entry-point
+  validation failures (§4).
+- `Closed` — an operation against a closed receiver/source (also the
+  `try_recv` / `recv_timeout` form of the wake close in §3).
+- `Codec` — payload serialization/deserialization failures
+  (`payload_as` on stream events and job payloads; wakes carry no
+  payload per §3).
+- `Database` — everything else: driver/connection/SQL errors, opaque
+  to the contract, engine detail preserved via the error source chain.
+  No engine-specific variants are minted for its contents.
+
+Explicitly *not* taxonomy items (callers act via value results, not
+errors): queue claim returning none (no work); job-handle ops
+(`ack`/`retry`/`fail`/`heartbeat`) returning "did the op land" booleans
+(`false` = the job was no longer claimable/owned — expired, acked
+elsewhere, cancelled); `try_lock` returning `Option<Lock>`; lock
+`release`'s boolean.
+
+### 6. `Store::open` config shape — engine options, not contract surface
+
+- The core crate defines the `Store` trait (the §1 surface plus
+  `begin_tx`). **Constructors and their option structs live in the
+  engine crates** (`alkstore_sqlite::open(path, SqliteOpts)` /
+  `alkstore_postgres::open(url, PgOpts)`): durability knobs, pool
+  sizing, pragmas, watcher cadence, listener posture are
+  engine-configuration concerns —
+  [deployment.md](../deployment.md) carries the knob
+  facts, engine-crate docs carry the options.
+- What is contract: the trait the constructor returns, and the
+  constructor's own obligation — a `Store` handed out by an engine
+  crate honors the full v1 surface.
+- No capability surface in v1 (OQ-08 owns whether one ever exists).
+- Consumer code stays engine-agnostic by depending on core and being
+  constructed by exactly one engine crate (ADR-001's single-driver
+  binaries) — the engine choice is a dependency-graph fact, not a
+  runtime branch.
+
+### 7. Delivery-guarantee rows — locks pinned, scheduler owned by OQ-09
+
+[ADR-006](006-wake-and-delivery-contract.md)'s table is extended by
+this decision:
+
+| Mechanism | Durability | Replay | Atomicity | Guarantee |
+|---|---|---|---|---|
+| **locks** | row-backed (machinery-defined) | n/a | acquire/release atomic | **mutual exclusion bounded by TTL + renewal discipline**: a lock excludes other holders while held (explicit release or within TTL); after TTL expiry exclusion lapses *silently* — no revocation event, exclusivity resumes with re-acquisition; a holder renewing inside its TTL keeps exclusion |
+
+- The consumer obligation this row carries: long held-lock work must
+  `renew` within TTL; expiry is not an error but a *loss of exclusivity*
+  (the honest TTL semantics the alkblobs fleet-sweeper ADR already
+  treats as ground).
+- The **scheduler guarantee row is not pinned here** — it is defined
+  by OQ-09's collapse decision (under collapse, the scheduler inherits
+  the queues row; the scheduler-tick/leader-election guarantees become
+  part of OQ-09's and OQ-05's resolution text). This is an explicit
+  transfer to OQ-09's scope, not an open residue of this ADR.
+
+### 8. Renames and grouping deltas from the honker-rs surface
+
+Pinned deltas (the starting artifact's names, where the contract
+differs):
+
+| honker-rs | contract v1 | Reason |
+|---|---|---|
+| `Subscription` (listen) | `WakeReceiver` / `Wake` | wake is opaque — no payload, no id; name must not suggest subscription semantics (no replay, ADR-006) |
+| `Notification { id, channel, payload }` | `Wake { channel }` | §3; payload is non-contract |
+| `Lock::heartbeat(ttl)` | `Lock::renew(ttl)` | renew describes the TTL discipline the guarantee row pins; heartbeat is the queue-side renewal term |
+| `Queue::claim_waker()` | (not surfaced) | engine-internal consumption wake (§1) |
+| `StreamSubscription` (`save_every`, auto-save on drop) | `subscribe(consumer) -> Box<dyn EventReceiver>`, explicit saves only | the explicit-offset obligation (core-contract.md streams; the auto-checkpoint ambiguity is not inherited, [ADR-006](006-wake-and-delivery-contract.md)) |
+| `update_events()` | (not surfaced) | superseded by the wake subscription — the pinned `WakeReceiver` *is* the reactive-event surface; honker's separate raw update-event stream has no contract role |
+| `prune_notifications` / `prune_notifications_keep_latest` | (not surfaced) | notifications-table maintenance tooling on the SQLite engine; disposition rides OQ-05's sweep/maintenance design (consumer-visible only if that design surfaces it) |
+| `Scheduler` surface | (out of v1, OQ-09 decides) | §1 |
+| `try_rate_limit` / `save_result` / `get_result` / `sweep_results` | (cut-flag rows) | [ADR-002](002-feature-scope.md) |
+
+Grouping stays honker-shaped otherwise: mechanism handles off the
+store (`stream(name)`, `queue(name, opts)`, `outbox(name)`), the tx
+handle off `begin_tx`, payloads serialized through core value types
+(non-generic trait methods, §2), `payload_as<T>` as the decode
+convenience on concrete returned values (`Job`, `StreamEvent`) — the
+deserialization error is `Codec`, not a per-type variant.
+
+The stream subscription handle's shape (defined here, since §1 pins
+the return type):
+
+```text
+trait EventReceiver {
+    recv() -> Option<Result<StreamEvent>>   // None = stream closed
+    try_recv() -> Result<Option<StreamEvent>>
+    read_since(offset, limit) -> Result<Vec<StreamEvent>>
+    save_offset(&mut self) -> Result<()>    // explicit only — the
+                                            // offset advances per read;
+                                            // the save is the consumer's
+                                            // checkpoint call
+    offset() -> i64                         // the receiver's current offset
+}
+```
+
+No `save_every`, no save-on-drop — checkpointing is the consumer's
+explicit call (the §1 obligation). The consumer's *name* is fixed at
+`subscribe(consumer)`; it is a consumer-local identifier (no reserved
+prefix, §4).
+
+## Consequences
+
+**Positive**
+
+- The contract v1 is a complete, implementable surface: every open
+  shape question from the OQ-04 list is pinned; no placeholder
+  semantics remain in [core-contract.md](../core-contract.md).
+- The tx-seam downcast friction is *dissolved*, not tolerated — the
+  trait shape makes each engine's handle self-contained, and a future
+  engine is purely additive (implement the traits; no core edits).
+- The taxonomy rule (variants only where callers act) is a durable
+  principle for later surface additions, keeping queue depth (OQ-05)
+  from accreting error variants.
+- The reserved namespace is exact (`__alkstore_`), testable, and
+  engine-independent.
+
+**Negative**
+
+- Dyn dispatch on `begin_tx`/mechanism returns means boxed handles and
+  boxed futures per op — bounded by POC measurement as negligible
+  against commit costs, but real; monomorphized fast paths are the
+  recorded optimization if profiles ever demand them.
+- Wake payloads are dropped from what the SQLite engine *could* carry
+  through — engine-side, honker's transport capability is unused by
+  the contract surface (the cost of an honest uniform contract).
+- `commit(self: Box<Self>)` shape means a consumed handle can't be
+  reused after commit (callers re-`begin_tx`) — matches the
+  caller-owned lifetime the POCs verified, but is stricter than
+  honker's re-usable `&Transaction` style.
+- OQ-05/OQ-09/OQ-08 remain open: this ADR deliberately does not pin
+  queue depth, the scheduler shape, or capability flags; the v1
+  additions to those surfaces will be later contract extensions.
+
+## References
+
+- OQ-04 (`docs/architecture/open-questions.md`) — this ADR's
+  resolution.
+- The honker-rs surface (`/workspace/honker`
+  `packages/honker-rs/src/lib.rs` v0.5.0) — the starting artifact;
+  `/workspace/honker` is a reference checkout
+  ([ADR-005](005-dependency-ownership.md) posture).
+- `docs/research/poc-sqlite-posture-findings.md` §Probe 1 (trait
+  friction), §"What feeds where"; `docs/research/poc-pg-posture-findings.md`
+  §Sub-module T (tx-seam shapes), §Sub-module W (wake/reconnect),
+  §Payload boundary (`PayloadTooLarge`).
+- [core-contract.md](../core-contract.md) — the spec this ADR's
+  §1–§5 pin into place.
+- [ADR-002](002-feature-scope.md) (scope),
+  [ADR-006](006-wake-and-delivery-contract.md) (wake contract,
+  namespace existence, guarantee table),
+  [ADR-007](007-transactional-seam.md) (tx seam mechanics).
+- OQ-09 (scheduler row transfer), OQ-08 (capability surface), OQ-10
+  (versioning discipline for future contract extensions).