alkstore/docs/architecture/open-questions.md

status, last_updated

status	last_updated
draft	2026-10-05

alkstore — Open Questions

Centralized tracker. IDs OQ-NN are stable — never renumber; append. Promoted, one-to-one, from Phase 0's OQ-ST register. Resolution order so far: OQ-04 resolved (2026-10-05, ADR-008 — the contract surface everything else hangs off); OQ-09 + OQ-05 resolved (2026-10-05, ADR-009 and ADR-010 — the queue semantics track and the scheduler collapse/guarantee row); OQ-06 resolved (2026-10-05, ADR-011 — the fork trigger fired on the published-artifact facts); the fork design follow-through (2026-10-05, ADR-012 — substrate boundaries, fidelity posture, port deltas; its substrate-side residue is OQ-11); the fork packaging folded (2026-10-05, ADR-013 — no alkstore-substrate crate; the forked machinery is a module subtree of alkstore-sqlite); OQ-13 resolved (2026-10-05, ADR-014 — outbox_enqueue_tx joins the TxHandle trait, the hole is closed). Next: OQ-12 (streams depth — a Phase 1 review find: key semantics, StreamEvent shape, ordering row, retention; evidence gathered; check consumer rows for per-key-ordering need before choosing among its options), OQ-08 (rides the now-pinned trait shape and the ADR-011 substrate fork), OQ-10 (versioning discipline for contract extensions; note ADR-011 changes its substrate-side facts for SQLite — the forked machinery lives in-tree inside the engine crate per ADR-013, so the engine/core contract pairing is what the discipline must track), OQ-11 (fork follow-through items — substrate-side, non-consumer-facing).

Resolved questions stay listed with their resolution; they are not deleted.

Theme: Scope

OQ-01: Scope boundary — which honker features are in-scope? (== OQ-ST-01)

• Origin: consumer-inventory.md, ADR-002
• Status: resolved (2026-10-04, Phase 0)
• Priority: medium
• Resolution: Answered by the consumer inventory — scope votes shrink to named rows, not the whole honker feature list. First-class: notify/listen (pinned), streams (operator-authority record). In scope: named locks, queues + outbox (documented), scheduler (documented-thin). Cut-flags: rate limits, result storage. Out: honker's exclusion lines. Decision recorded in ADR-002.
• Cross-references: OQ-07, OQ-04.

OQ-02: Crate scope — one store crate, or reactive-core + engines? (== OQ-ST-02)

• Origin: overview.md
• Status: resolved (2026-10-04, operator decision; Phase 0)
• Priority: medium
• Resolution: Reactive-core crate + per-engine engine crates; the split isolates the engines' asymmetry of work and keeps engine binaries single-driver (which the libsqlite3-sys collision effectively requires). Decision recorded in ADR-001 — including the reasoning that superseded the inventory's single-crate lean.
• Cross-references: OQ-03, OQ-10.

OQ-03: Driver story — sqlx, tokio-postgres, or per-engine drivers? (== OQ-ST-03)

• Origin: overview.md
• Status: resolved (2026-10-04, POC-backed both engines; Phase 0)
• Priority: medium
• Resolution: Per-engine drivers — rusqlite + honker-core 0.5.0 (SQLite; bridged seam), tokio-postgres 0.7.18 + deadpool-postgres 0.14.2 (Postgres), under OQ-02's crate split. Decisions recorded in ADR-003 and ADR-004; measurements in docs/research/poc-sqlite-posture-findings.md and docs/research/poc-pg-posture-findings.md.
• Cross-references: OQ-02, OQ-05, OQ-06.

OQ-07: SQLite-side scope — loadable extension? (== OQ-ST-07)

• Origin: ADR-002
• Status: resolved (cut-only, 2026-10-04; Phase 0)
• Priority: low
• Resolution: The loadable-extension surface is out: every identified consumer is in-process Rust attaching to its own connection (the honker-core attach_honker_functions shape). Folded into OQ-01's scope resolution and ADR-002 (explicitly, so the slot's disposition is visible — the Phase 0 register carried it as its own entry).
• Cross-references: OQ-01.

Theme: Core contract

OQ-04: Contract pinning — the exact trait surface and its semantics (== OQ-ST-04) — RESOLVED

(Retitled in promotion: OQ-ST-04's framing — "the reactive abstraction — what does the unified notify surface look like?" — narrowed to the pinning work its own record already scoped.)

• Origin: core-contract.md, ADR-006, ADR-007
• Status: resolved (2026-10-05, Phase 1 — ADR-008)
• Priority: high
• Resolution: Pinned as contract v1 by ADR-008: (1) the v1 surface partition — notify/streams/queue-skeleton/locks/outbox/tx seam are contract; queue depth rides OQ-05, scheduler shape rides OQ-09, capabilities ride OQ-08, claim_waker stays engine-internal, cut-flag rows stay out. (2) TxHandle: the *_tx methods live on the handle trait — boxed dyn handle, no downcast, no enum (the enum would invert ADR-001's dependency direction); the POCs' as_any_mut friction dissolves by construction. (3) listen() returns a WakeReceiver delivering opaque Wake { channel } — honker's payload transport is not surfaced (uniformity with the opaque contract, ADR-006). (4) Reserved prefix __alkstore_, with __alkstore_listener_reconnected__ as the one v1-reserved string (the POC's ad-hoc __listener_reconnected__ renamed at implementation). (5) Error taxonomy: PayloadTooLarge / ReservedName / InvalidName / Closed / Codec / Database, pinned by the act-differently rule. (6) Store::open is engine-crate surface; the contract is the trait it returns. (7) Locks guarantee row added to ADR-006's table (TTL-bounded mutual exclusion, silent expiry); the scheduler row is explicitly transferred to OQ-09's resolution. A verification backlog (core-contract.md §Verification backlog) tracks the one-engine-pinned properties.
• Cross-references: OQ-01, OQ-10, OQ-09, OQ-08; OQ-13 (the one gap the v1 pinning's own Phase 1 review found in it).

Theme: Queues and scheduling

OQ-05: Queue semantics depth — retry/backoff/dead-letter/sweep design (== OQ-ST-05) — RESOLVED

• Origin: queues.md
• Status: resolved (2026-10-05, Phase 1 — ADR-010)
• Priority: high
• Resolution: Pinned by ADR-010: (1) job state machine — three states (pending/processing/dead), delete-on-ack, move-to-dead (honker's model, pg-boss's seven-state enum not inherited), get_job sees dead rows (last_error/died_at — post-mortem by API; cancel unconditional, not an interrupt). (2) Visibility/heartbeat — renewal semantics (absolute reset), late-heartbeat refusal (dual-execution window is contract-honest), reclaim-consumes-an-attempt stated as contract text. (3) Retry/backoff — retry(err, None) computes the queue's equal-jitter exponential curve (range definitionally pinned, not a labeled distribution; base backoff_base_s, cap 1 h, attempt index = row attempts at retry), Some(d) overrides; queue-level QueueOpts { visibility_timeout_s, max_attempts, backoff_base_s, dead_letter_retention_s } stamped onto each job row at enqueue (§3a — no per-queue registry; two handles with different opts for one queue name don't fight). (4) Dead-letter — move-to-table, retention forever by default, no redrive API (recipe: get_job + fresh enqueue). (5) sweep_expired — the no-stranded-rows property (every past-expiry row in any state moves to dead; honker's expired-processing zombie hole fails this — fixed at contract level, SQLite-side realization rides OQ-06). (6) Sweep cadence — no ambient sweeper; the recipe is the collapse machinery (schedule + runner). (7) Result-storage cut-flag reconsidered and stands (delete-on-ack; pg-boss's completed-row model is result storage under another name). (8) Table layout — pg engine-owned schema (alkstore default), queues are rows not tables; SQLite rides _honker_*. (Item 8's SQLite half superseded 2026-10-05: the _honker_* family is re-owned as __alkstore_* by the fork — ADR-011, designed per ADR-012.) No new error variants (ADR-008 §5's rule).
• Cross-references: OQ-09, OQ-06.

OQ-09: Is the scheduler a first-class mechanism, or queues + schedule()? — RESOLVED

• Origin: queues.md (spin-out of OQ-05's Phase 0 framing, where the scheduler's boundary question lived)
• Status: resolved (2026-10-05, Phase 1 — ADR-009)
• Priority: medium
• Resolution: Collapse confirmed, decided against the inventory rows (alkblobs sweep cadence, alkfs orphan reaping — both intervals; no row names schedule objects or wall-clock cron): the contract surface is schedule(name, spec, queue, payload, opts) (upsert by name), unschedule(name), and the opt-in run_schedules(stop) runner — no Scheduler handle, no schedule objects; update = re-register, pause = unregister + re-register. Spec grammar v1: @every <n><unit> only — cron strings rejected (InvalidSpec), dissolving the timezone question honker's local-time cron brittleness poses (extension path requires a consumer-inventory row). Boundary guarantee row pinned (the transfer from ADR-008 §7 resolved): at-least-once per elapsed boundary while a leader runs, fire+advance commit atomically under a row lock on the schedule row (engine-generic no-double-fire floor; the leadership lock is the efficiency layer), bounded catch-up (fixed contract-constant cap 64) with skip-forward; leadership lock is the reserved __alkstore_scheduler name; run_schedules returns Err(LeadershipLost) on lock loss, Ok(()) on clean stop. Schedule storage engine-internal; schedules never fire without a runner (no ambient timers).
• Cross-references: OQ-05, OQ-04.

Theme: Engines and dependencies

OQ-06: honker-core quality read — does the default posture hold? (== OQ-ST-06) — RESOLVED

• Origin: ADR-005, engine-sqlite.md
• Status: resolved (2026-10-05, Phase 1 — ADR-011)
• Priority: high (fork trigger is a gate on the SQLite engine's dependency posture)
• Resolution: The fork trigger fires. Evidence: quality-read-honker-core.md. The read's original target is clean (Writer/Readers, the polling watcher's three-layer failure handling, WatcherDeathGuard — all verified in source, present in 0.5.0; schema migrations minor). But the published artifact is materially behind the reference revision: crates.io's honker-core 0.5.0 predates upstream's own fix train for its documented defect class (issue #133's savepoint hardening — silent job loss on mid-flight errors in the dead-letter paths; five .ok() error-swallows mapping every SQLite error to "no row"/"lock held"), stranded unreleased with no announced date. And contract v1's queue depth (ADR-010 §3a/§5) requires engine-owned enqueue/claim/sweep/get_job SQL in any posture — making a post-fix ride a minority-shape, dual-owning the schema. Resolution: fork honker-core at the reference revision into owned code, inherit the clean watcher/transactional core + test suites, re-derive the queue ops on contract v1, drop cron/experimental backends, rename _honker_* → __alkstore_*. Recorded in ADR-011; ADR-005's posture calculus applied, not renegotiated.

OQ-11: Forked-substrate follow-through — scaffold, provenance register, and cherry-pick discipline

• Origin: ADR-012 (the fork design; substrate-side residue), ADR-011, ADR-013
• Status: open (reshaped 2026-10-05 by ADR-013 — item (1) dissolved)
• Priority: medium (nothing consumer-facing rides on it; it resolves within the fork-scaffold task, which it does not gate)
• Resolution: open. ADR-012 fixed the design (contract-blind boundary, fidelity posture, port deltas, bootstrap machinery, upstream-tracking stance) and also pinned the provenance register's location, timing, and initial contents (§1, as carried in-tree by ADR-013). The residue is: (1) dissolved by the fold (ADR-013) — there is no separate substrate crate to wire; the engine crate carries one rusqlite dependency and no bundled-sqlite interplay question; (2) the provenance register's format and delta-list granularity (per-commit entries vs per-delta-class; whether cherry-pick records append at adoption time — the register lives in-tree beside the substrate subtree per ADR-013 §3; its timing and initial contents are ADR-012 §1's, not re-opened here); (3) the cherry-pick procedure in practice — how a candidate upstream fix is evaluated, applied, and recorded so ADR-012 §6's deliberate-work posture stays auditable. These resolve within the fork-scaffold task (its opening section), not before it and not as a gate on writing it.
• Cross-references: OQ-08 (the contract surface the engine maps the substrate under), OQ-10 (the engine's versioning discipline now carries the fold's provenance duties in-tree), ADR-011, ADR-012, ADR-013.

OQ-10: How do engine crates track core-contract version changes?

• Origin: overview.md, ADR-001
• Status: open
• Priority: medium
• Resolution: open. The core crate's trait surface is a contract the engine crates must track (ADR-001 negative consequence). What is the versioning/sync discipline — semver-bump-only-when- contract-changes, engines pin core ranges, a contract-compatibility test suite the engines run against the core's trait definitions? What happens to a released engine crate when core makes a contract breaking change?
• Cross-references: OQ-04 (the contract being versioned — now including its first post-v1 extensions, ADR-009/ADR-010), OQ-02.

Theme: Deployment and capabilities

OQ-08: Where does the honest single-host/multi-host boundary live in the trait surface? (== OQ-ST-08)

• Origin: deployment.md
• Status: open
• Priority: medium
• Resolution: open. The pg engine is natively multi-host (POC #2 verified — no single-host assumption to remove); SQLite is single-machine by nature (file-backed, NFS-two-writers unsupported — honker's honesty posture, inherited by the forked substrate). The unified surface must not pretend SQLite is multi-host. Options: per-engine capability flags (Store::capabilities()), a documented deployment matrix only (deployment.md carries the facts), or compile-time knowledge only (a consumer choosing the SQLite engine knows). Rides the now-pinned contract shape (ADR-008): the trait constrains where capability differences can surface.
• Cross-references: OQ-04, ADR-006, ADR-012 (the substrate inherits the honesty posture).

Theme: Core contract (Phase 1 review finds)

OQ-12: Streams depth — key semantics, StreamEvent shape, ordering row, retention

• Origin: core-contract.md streams section, ADR-008 §1 (Phase 1 architecture review, 2026-10-05)
• Status: open
• Priority: high (streams is a first-class ADR-002 mechanism; ADR-008's "no placeholder semantics remain" positive consequence is false for publish_with_key until this resolves — the contract suite also cannot pin cross-engine stream equivalence without an ordering row)
• Resolution: open. The method names are pinned (ADR-008 §1); the depth is not. Sub-questions, with the evidence already gathered:
  • Key semantics — honker's realization is a stored key column (_honker_stream(topic, key, payload), NULLable) with no per-key read path or ordering enforcement anywhere; the honker-rs doc comment says "used for per-key ordering downstream" (packages/honker-rs/src/lib.rs:894), i.e. per-key global-FIFO is an emergent property (a consumer reading offset ASC filtered by key), not server-enforced. Options: (a) pin exactly honker's shape — key is carried metadata, ordering stays global FIFO (document the emergent per-key pattern), cheapest and honest; (b) pin server-enforced per-key ordering (a read or delivery guarantee keyed on key) — real machinery on both engines, needs a consumer row naming the need; (c) cut publish_with_key from v1 (no consumer row names keys — the honker-rs doc comment is upstream's intent, not ours). Decision rule: consumer-inventory row first if (b).
  • StreamEvent shape — honker's: { offset: i64, topic: String, key: Option<String>, payload, created_at: i64 } with payload_as<T> (upstream lib.rs:1008). The contract needs its own pinned field list (stream name vs topic token, key in or out per the key decision, timestamp semantics) — probably inherited near-verbatim.
  • Ordering guarantee row — ADR-006's table has no streams ordering cell content beyond "readable"; pin FIFO-by-offset as the read ordering (ORDER BY offset ASC is already both engines' claim path) and whether any per-key strengthening rides the key decision.
  • Retention / bounded growth — the reference read flags _honker_stream as unbounded-growth unless the user sweeps (honker-machinery §8.5); no consumer row names stream retention (replay-forever is the purpose for the named need — subscription surfaces). Options: consumer-side trim API (trim_to(offset) — a new contract method), consumer-recipe only (new stream + advance consumers + unschedule/nothing — needs a deletion path that doesn't exist yet, so this option is really "document the gap"), or document nothing (honest but repeats the notifications-table mistake ADR-010 §6 just fixed). Rides the no-ambient-sweeper posture either way.
• Cross-references: OQ-04 (the pinning that left this depth open), OQ-01 (scope row), ADR-002, ADR-006 (the guarantee table the ordering row extends), ADR-008 §1.

OQ-13: Transactional outbox enqueue shape — the TxHandle surface cannot express it — RESOLVED

• Origin: core-contract.md (outbox section), ADR-008 §1 (Phase 1 architecture review, 2026-10-05)
• Status: resolved (2026-10-05, Phase 1 — ADR-014)
• Priority: high (the outbox's only reason to exist is the commit-atomic enqueue; as pinned, the v1 surface cannot perform it — a decomposition off the current text would produce an unimplementable task)
• Resolution: Pinned by ADR-014: one method, outbox_enqueue_tx(outbox, opts, payload) -> job_id, added to the TxHandle trait. The method takes the outbox name (validated like store.outbox(name) — InvalidName/ReservedName), derives the backing queue name __alkstore_outbox:{outbox} engine-side, and lands the job row in the caller's transaction with EnqueueOpts stamped per [ADR-010](decisions/010-queue-semantics- depth.md) §3a over the backing queue's derived QueueOpts — commit-atomic with the business write, rollback drops both (the no-ghosts property). The reserved-prefix rejection is unchanged: it governs directly-supplied names; the derivation is legitimate engine-internal naming (the same status run_once's consumption of the backing queue already had). Option (b) (an OutboxHandle off the tx handle) is rejected — a second boxed handle to deliver one op; option (c) (non-transactional only) is rejected — it drops the property the mechanism exists for. Honker's raw-Transaction seam is not inherited (no engine-neutral Transaction type exists; it would force a core→engine dependency or a downcast seam — the two things ADR-008 §2 dissolved). Framed as completing contract v1 in place, pre-implementation — no versioning event, OQ-10 untouched. Contract-suite row added (commit-atomicity on both engines).
• Cross-references: OQ-04 (the pinning that missed this), OQ-12 (the sibling depth gap found by the same review), ADR-002 (outbox scope row), ADR-007/ADR-008 (the seam design), ADR-010 §3 (the outbox's derived QueueOpts — unchanged by this).

Deferred / Blocked

None currently. Every open OQ above is actionable Phase 1 work (streams depth, capability-surface shape, versioning discipline, fork-scaffold follow-through) with its evidence base complete — no external arrivals are being waited on.