docs: open Phase 1 — architecture spec set over the Phase 0 evidence
docs/architecture/ now exists: README index, overview, five component specs (core-contract, engine-sqlite, engine-postgres, queues, deployment), ADR-001..007 carrying the Phase 0 resolved decisions (crate split, feature scope, per-engine drivers, dependency ownership, wake contract, tx seam), and the centralized open-questions tracker promotion: OQ-ST-01..08 mirror to OQ-01..08 one-to-one with statuses/resolutions carried; new Phase 1 questions append (OQ-09 scheduler collapse, OQ-10 contract versioning). Open Phase 1 work: OQ-04 contract pinning (high), OQ-05 queue semantics depth (high), OQ-06 honker-core quality read (high; fork-trigger gate), OQ-08 capability surface, OQ-09, OQ-10. Erratum fixed in phase-0 OQ-ST-04 (thread-affinity friction is SQLite-side, previously garbled as pg-side) and a stale scheduler- boundary pointer corrected in consumer-inventory.md. Two review passes run (findings: OQ-promotion numbering faithfulness, ADR back-reference sync) — all critical/warning findings resolved.
This commit is contained in:
1 parent
db73678090
commit
4391f6e879
17 files changed
+1812
-14
No files matched your search
@@ -0,0 +1,92 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-04
|
||||
---
|
||||
|
||||
# alkstore — Architecture
|
||||
|
||||
Architecture documentation for the alkstore project: one reactive
|
||||
store interface (notify, streams, queues, locks, scheduler, outbox)
|
||||
over SQLite and Postgres, with each engine native underneath
|
||||
(see [overview.md](overview.md)).
|
||||
|
||||
## Current State
|
||||
|
||||
**Phase 1 (Architecture) — in progress.** Phase 0 is complete
|
||||
(`docs/research/phase-0.md`): both POCs ran and passed, the scope
|
||||
inventory is confirmed, and the crate split, drivers, and ownership
|
||||
postures are decided. This directory carries the architecture spec
|
||||
build-out over that evidence base; all spec documents are `draft`
|
||||
pending architecture review and OQ resolution.
|
||||
|
||||
## Architecture Documents
|
||||
|
||||
| Doc | Status | Purpose | Key OQs |
|
||||
|---|---|---|---|
|
||||
| [overview.md](overview.md) | draft | Crate family, feature surface, non-goals, evidence base | — |
|
||||
| [core-contract.md](core-contract.md) | draft | The unified trait surface, delivery guarantees, tx seam | OQ-04, OQ-08, OQ-09, OQ-10 |
|
||||
| [engine-sqlite.md](engine-sqlite.md) | draft | SQLite engine: honker-core/rusqlite mapping | OQ-05, OQ-06, OQ-09 |
|
||||
| [engine-postgres.md](engine-postgres.md) | draft | Postgres engine: tokio-postgres/LISTEN mapping | OQ-05, OQ-08, OQ-09 |
|
||||
| [queues.md](queues.md) | draft | Queue/scheduler/outbox semantics depth frame | OQ-05, OQ-09 |
|
||||
| [deployment.md](deployment.md) | draft | Host semantics, connection budgets, knobs, matrix | OQ-04, OQ-08 |
|
||||
| [open-questions.md](open-questions.md) | draft | OQ tracker (promoted from OQ-ST register) | — |
|
||||
|
||||
## Architecture Decision Records
|
||||
|
||||
| ADR | Title | Status |
|
||||
|---|---|---|
|
||||
| [001](decisions/001-crate-split.md) | Reactive-core crate + per-engine crates | Accepted |
|
||||
| [002](decisions/002-feature-scope.md) | Feature scope — inventory-confirmed surface | Accepted |
|
||||
| [003](decisions/003-sqlite-driver.md) | SQLite engine — rusqlite + honker-core, bridged seam | Accepted |
|
||||
| [004](decisions/004-postgres-driver.md) | Postgres engine — tokio-postgres + deadpool, hand-rolled LISTEN | Accepted |
|
||||
| [005](decisions/005-dependency-ownership.md) | Published libraries by default, named fork triggers | Accepted |
|
||||
| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract — opaque wake + re-read; notify-vs-streams split | Accepted |
|
||||
| [007](decisions/007-transactional-seam.md) | Transactional seam — caller-held `TxHandle`, `*_tx` methods | Accepted |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Tracked in [open-questions.md](open-questions.md) (OQ-01..NN; the
|
||||
Phase 0 register's OQ-ST-01..08 promote one-to-one — OQ-NN mirrors
|
||||
OQ-ST-NN — with new Phase 1 questions appended after). Highlights,
|
||||
in suggested resolution order (OQ-04 first):
|
||||
|
||||
- **OQ-04** (high): contract pinning — exact trait shape, handle
|
||||
representation, error taxonomy, reserved strings, guarantee rows
|
||||
for locks/scheduler.
|
||||
- **OQ-05** (high): queue semantics depth — retry/backoff/dead-letter/
|
||||
sweep design.
|
||||
- **OQ-06** (high): honker-core quality read — fork-trigger gate.
|
||||
- **OQ-09** (medium): scheduler as first-class mechanism vs queues +
|
||||
`schedule()`.
|
||||
- **OQ-08** (medium): capability-surface shape.
|
||||
- **OQ-10** (medium): contract versioning across engine crates.
|
||||
|
||||
Resolved (Phase 0, kept with resolutions): OQ-01 (feature scope),
|
||||
OQ-02 (crate split), OQ-03 (drivers), OQ-07 (extension surface cut).
|
||||
|
||||
No deferred OQs: all open questions are actionable Phase 1 work with
|
||||
complete evidence bases.
|
||||
|
||||
## Document Lifecycle
|
||||
|
||||
| Status | Meaning | Transitions |
|
||||
|---|---|---|
|
||||
| `draft` | Under active development; may change significantly | → `reviewed` when the doc's OQs are resolved |
|
||||
| `reviewed` | Architecture final; implementation may begin | → `stable` when implementation verified |
|
||||
| `stable` | Locked; changes need review, may warrant an ADR | → `deprecated` when superseded |
|
||||
| `deprecated` | Superseded; kept for reference | Removed when unreferenced |
|
||||
|
||||
All spec documents carry YAML frontmatter (`status`, `last_updated`);
|
||||
ADRs carry a `## Status` section (Accepted/Proposed/Superseded).
|
||||
|
||||
## Provenance of decisions
|
||||
|
||||
Phase 1 inherits its decisions from Phase 0's evidence base — every
|
||||
Accepted ADR above cites its POC findings and register record. The
|
||||
research documents remain the deep background:
|
||||
|
||||
- `docs/research/phase-0.md` — vision, prior art, OQ-ST register,
|
||||
convergence.
|
||||
- `docs/research/consumer-inventory.md` — per-feature scope evidence.
|
||||
- `docs/research/poc-sqlite-posture-findings.md` /
|
||||
`docs/research/poc-pg-posture-findings.md` — measured ground.
|
||||
@@ -0,0 +1,204 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-04
|
||||
---
|
||||
|
||||
# Core contract
|
||||
|
||||
The unified, engine-agnostic surface a `Store` exposes. This document
|
||||
specifies WHAT the contract is; the per-engine specs map it onto their
|
||||
machinery; ADRs carry the WHY. The starting artifact is the honker-rs
|
||||
surface (the Phase 0 §Interface finding) scoped to the
|
||||
inventory-confirmed features — [ADR-002](decisions/002-feature-scope.md)
|
||||
— and pinned against it. Exact Rust shapes are OQ-04's pinning work;
|
||||
the *obligations* are this document.
|
||||
|
||||
## Concepts
|
||||
|
||||
- **Store** — what a consumer opens from a connection string or file
|
||||
path ([ADR-001](decisions/001-crate-split.md)): one handle, the engine
|
||||
behind it chosen at open time. Consumer code never branches on
|
||||
engine type (guiding principle 4).
|
||||
- **Mechanism** — one of the surface's coordination families: notify,
|
||||
streams, queues (+outbox), locks, scheduler. Delivery guarantees are
|
||||
pinned per mechanism in [ADR-006](decisions/006-wake-and-delivery-contract.md)'s
|
||||
table (notify/streams/queues); lock and scheduler guarantee rows are
|
||||
OQ-04/OQ-09 resolution items, not assumed.
|
||||
|
||||
*(Guiding principles are defined in `docs/research/phase-0.md`
|
||||
§Vision and cited by number throughout this directory.)*
|
||||
- **Wake** — the opaque "something changed, re-read" signal the wake
|
||||
contract delivers ([ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||||
The contract's central abstraction.
|
||||
- **TxHandle** — the caller-held transaction object whose `*_tx`
|
||||
methods make side effects commit-atomic with a business write
|
||||
([ADR-007](decisions/007-transactional-seam.md)).
|
||||
|
||||
## The seam
|
||||
|
||||
```text
|
||||
Store::open(config) -> Store
|
||||
Store::begin_tx() -> TxHandle
|
||||
```
|
||||
|
||||
Mechanism handles come off the store (or, for transactional variants,
|
||||
off the handle):
|
||||
|
||||
```text
|
||||
store.notify(channel, payload) handle.notify_tx(channel, payload)
|
||||
store.stream(name) -> Stream handle.publish_tx / save_offset_tx
|
||||
store.queue(name) -> Queue handle.enqueue_tx
|
||||
store.try_lock(name, ttl) -> Lock
|
||||
```
|
||||
|
||||
The exact shape of these handles (traits vs concrete types, `dyn` vs
|
||||
generic `TxHandle`) is OQ-04's pinning work.
|
||||
|
||||
## Mechanism contracts
|
||||
|
||||
### notify / listen
|
||||
|
||||
Fire-and-forget signals, commit-atomic when sent in a transaction
|
||||
([ADR-007](decisions/007-transactional-seam.md)). No durability, no
|
||||
replay, no per-listener retry; a listener attached after a commit
|
||||
never sees it.
|
||||
|
||||
- `notify(channel, payload)` — payload ≤ 8000 bytes on Postgres
|
||||
(client-side checked, typed error, verified by POC #2); no limit on
|
||||
SQLite. The asymmetry is a documented engine capability, see
|
||||
OQ-04/OQ-08.
|
||||
- `listen(channel)` — starts from "now"; delivers
|
||||
opaque wakes ([ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||||
Wakes are at-least-once, possibly coalesced (SQLite) or per-notify
|
||||
(Postgres), possibly repeated after reconnect. **Consumers must be
|
||||
idempotent on wake.** (The sketched return type — a wake receiver vs
|
||||
a subscription handle — is an OQ-04 pinning item, like all exact
|
||||
shapes here.)
|
||||
- Failure surfaces as channel events, never silence: watcher death
|
||||
closes the receiver (SQLite); the synthetic reconnect-wake (a
|
||||
reserved channel, [ADR-004](decisions/004-postgres-driver.md)) covers
|
||||
Postgres connection gaps.
|
||||
- Channel name is the one piece of semantic content a wake carries —
|
||||
the invalidation key for caching subscribers.
|
||||
|
||||
### streams
|
||||
|
||||
Durable pub/sub with per-consumer offsets
|
||||
([ADR-006](decisions/006-wake-and-delivery-contract.md)). The durable
|
||||
cousin of notify: publish is commit-atomic; every committed event is
|
||||
readable by every consumer whose offset hasn't passed it;
|
||||
replay-on-attach is the default; offsets are explicit and
|
||||
transaction-aware (`save_offset_tx` gives exactly-once-within-a-
|
||||
business-tx shape).
|
||||
|
||||
- `publish` / `publish_tx` / `publish_with_key` — append to the
|
||||
stream's durable log.
|
||||
- `read_since` / `read_from_consumer(offset)` — cursor-based reads.
|
||||
- `save_offset` / `save_offset_tx` — consumer checkpoint, explicit;
|
||||
**the contract's save is always explicit** (no auto-checkpoint
|
||||
cadence — honker's per-binding ambiguity is not inherited).
|
||||
- `subscribe(consumer)` — durable consumption: attach, read to
|
||||
current tail, resume after restart from the stored offset.
|
||||
|
||||
### queues
|
||||
|
||||
Durable at-least-once work
|
||||
([ADR-002](decisions/002-feature-scope.md)). The semantics-depth design
|
||||
is [queues.md](queues.md)'s (OQ-05); the contract-level obligations
|
||||
here:
|
||||
|
||||
- `enqueue` / `enqueue_tx` — commit-atomic; `EnqueueOpts { delay,
|
||||
priority, max_attempts, expires, ... }`.
|
||||
- `claim` / `claim_batch` — exactly-once handout under concurrency
|
||||
(POC-pinned on both engines).
|
||||
- Job handle: `ack / retry / fail / heartbeat`; visibility timeouts
|
||||
make at-least-once *safe* (work re-appears if a claimant dies).
|
||||
- `sweep_expired` — maintenance entry point (design in
|
||||
[queues.md](queues.md)).
|
||||
|
||||
### named locks
|
||||
|
||||
TTL-bounded coordination locks, transactional-friendly:
|
||||
|
||||
- `try_lock(name, ttl)` — acquire or fail; `renew`; release on
|
||||
explicit unlock or TTL expiry. Re-acquirable after expiry
|
||||
(POC-pinned on Postgres; on SQLite it rests on honker's machinery —
|
||||
the SQLite-side pin is in OQ-04's verification backlog).
|
||||
- Lock names are shared-namespace (contract text pins collision
|
||||
rules with the reserved namespace, OQ-04).
|
||||
|
||||
### outbox
|
||||
|
||||
A helper over queues, not a separate mechanism: enqueue inside the
|
||||
business transaction + the delivery/consumption worker entry points.
|
||||
Same evidence base and guarantee as queues
|
||||
([ADR-002](decisions/002-feature-scope.md)).
|
||||
|
||||
### scheduler
|
||||
|
||||
Cron/`@every` enqueueing into named queues, leader-elected where the
|
||||
engine has peers (SQLite: single-host, no election needed beyond
|
||||
documented posture; Postgres: advisory-lock election). Whether this is
|
||||
a first-class mechanism or queues + `schedule()` is OQ-09.
|
||||
|
||||
## Cross-cutting contracts
|
||||
|
||||
### Delivery-guarantee table
|
||||
|
||||
The per-mechanism table in
|
||||
[ADR-006](decisions/006-wake-and-delivery-contract.md) is the contract
|
||||
of record for notify/streams/queues; this spec inherits it and adds
|
||||
the consumer obligations:
|
||||
|
||||
- wake idempotence (notify's at-least-once, coalescing behavior);
|
||||
- explicit offset saves (streams);
|
||||
- visibility-timeout budgeting (queues);
|
||||
- `*_tx` operations are *only* durable after the caller's commit —
|
||||
rollback drops job rows, event rows, notifications, and offset saves
|
||||
together (the no-ghosts property, POC-pinned on both engines).
|
||||
|
||||
### Errors
|
||||
|
||||
`thiserror`-typed, per the family standard. The taxonomy (what
|
||||
callers match on per mechanism; engine-capability errors like
|
||||
`PayloadTooLarge` on pg vs no-limit on SQLite) is OQ-04's pinning
|
||||
work.
|
||||
|
||||
### Capability surface
|
||||
|
||||
Whether the `Store` exposes engine capabilities at all — and if so,
|
||||
which (payload limits, host semantics, wake-cadence knobs) — is
|
||||
OQ-08's decision ([deployment.md](deployment.md)).
|
||||
|
||||
### Naming / reserved namespace
|
||||
|
||||
Consumer-visible names (channels, streams, queues, locks) share
|
||||
engine-visible namespaces on Postgres (LISTEN channel names are
|
||||
server-global per database). Exact rules: OQ-04. Decided already: a
|
||||
**reserved/meta namespace exists** for engine-internal names — the
|
||||
Postgres reconnect-wake channel, any internal bookkeeping — and
|
||||
consumer names must not collide with it
|
||||
([ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|---|---|---|
|
||||
| [001](decisions/001-crate-split.md) | Crate split | core + per-engine crates; single-driver binaries |
|
||||
| [002](decisions/002-feature-scope.md) | Feature scope | inventory-confirmed features only; cut-flags explicit |
|
||||
| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract | opaque wake + re-read; notify-vs-streams guarantee split |
|
||||
| [007](decisions/007-transactional-seam.md) | Tx seam | caller-held handle, `*_tx` methods, native commit-atomicity |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in
|
||||
[open-questions.md](open-questions.md). Key
|
||||
questions affecting this document:
|
||||
|
||||
- **OQ-04**: exact trait-surface pinning — handle shape, error
|
||||
taxonomy, reserved strings, rename/grouping decisions (open,
|
||||
high priority)
|
||||
- **OQ-10**: contract versioning discipline across engine crates (open)
|
||||
- **OQ-08**: capability-surface shape (open)
|
||||
- **OQ-09**: scheduler as first-class mechanism vs queues + schedule
|
||||
(open)
|
||||
@@ -0,0 +1,88 @@
|
||||
# ADR-001: Reactive-core crate + per-engine crates
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The store must present one reactive interface (notify, streams, queues,
|
||||
locks, scheduler, outbox) over two backing engines, SQLite and Postgres.
|
||||
The engines differ sharply in how much of that surface is natively
|
||||
theirs:
|
||||
|
||||
- The **SQLite** engine rides honker's existing, battle-tested machinery
|
||||
(published `honker-core` — a port-and-adapt of a sync library to the
|
||||
family's async-facing-trait posture).
|
||||
- The **Postgres** engine is the build-heavy side: LISTEN/NOTIFY wiring
|
||||
and the pg-boss-family queue schema are built by this crate.
|
||||
|
||||
A single crate with feature-gated engines would force both engines'
|
||||
dependency graphs into every downstream binary that enables both
|
||||
features. That is not hypothetical: rusqlite 0.40's `libsqlite3-sys`
|
||||
link collision with sqlx's sqlite driver is a real constraint
|
||||
(see [ADR-003](003-sqlite-driver.md)); single-driver binaries are the
|
||||
only clean escape.
|
||||
|
||||
Scope evidence lives in `docs/research/consumer-inventory.md`; the
|
||||
operator decision was recorded 2026-10-04 against
|
||||
OQ-ST-02 (`docs/research/phase-0.md`).
|
||||
|
||||
## Decision
|
||||
|
||||
The project ships as a **family of crates**:
|
||||
|
||||
1. **`alkstore` (core)** — the crate carrying the unified trait
|
||||
surface, types, error model, capability flags, and the contract
|
||||
documentation. No driver dependencies. Compile-lean by construction:
|
||||
the base crate has no engine machinery to keep out.
|
||||
2. **`alkstore-sqlite`** — the SQLite engine implementing the core
|
||||
surface. Single driver: rusqlite + honker-core ([ADR-003]).
|
||||
3. **`alkstore-postgres`** — the Postgres engine implementing the core
|
||||
surface. Single driver: tokio-postgres + deadpool-postgres
|
||||
([ADR-004]).
|
||||
4. **A mem-shaped engine** (in-tree or separate test crate) may exist as
|
||||
a third implementation for tests and doctests, if the test story
|
||||
turns out to want it. Treated as an implementation convenience, not
|
||||
a contract artifact — decided at implementation time, not now.
|
||||
|
||||
Downstream consumers depend on `alkstore` plus exactly one engine
|
||||
crate. A consumer binary links at most one driver.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- The engines' asymmetry of work is structural: the SQLite engine has a
|
||||
small, mostly-wiring job; the Postgres engine carries the build-heavy
|
||||
LISTEN/queue work. Neither is burdened by the other's dependencies.
|
||||
- Any future engine (alkfs's in-tree needs, an ops-surface engine) is
|
||||
additive — a new crate implementing the core traits — rather than a
|
||||
feature-graph edit to one crate.
|
||||
- Core-lean is structural, not a feature-discipline to be enforced by
|
||||
review.
|
||||
- The libsqlite3-sys link-collision class of problems is eliminated by
|
||||
construction (single-driver binaries).
|
||||
- Per-engine compilation/test gates are independent.
|
||||
|
||||
**Negative**
|
||||
|
||||
- A version-coordination duty: the core's trait surface is a contract
|
||||
the engine crates must track. Version bumps in core must be adopted
|
||||
by engines in lockstep when the contract changes (minor/major
|
||||
discipline; no trait-default drift).
|
||||
- Slightly more crate plumbing; naming/publishing overhead.
|
||||
- Consumers who want *both* engines in one binary (rare, unsupported-by
|
||||
design) cannot get a both-features build today.
|
||||
|
||||
## References
|
||||
|
||||
- OQ-ST-02 (`docs/research/phase-0.md`) — the decision record with
|
||||
options considered.
|
||||
- `docs/research/consumer-inventory.md` — the inventory whose lean
|
||||
(single crate) was superseded, with the correction recorded there.
|
||||
- [ADR-003](003-sqlite-driver.md) — the SQLite driver choice whose
|
||||
link-collision constraint motivates this split.
|
||||
- [ADR-004](004-postgres-driver.md) — the Postgres driver choice.
|
||||
- OQ-10 (`docs/architecture/open-questions.md`) — trait-versioning
|
||||
duties created here.
|
||||
@@ -0,0 +1,99 @@
|
||||
# ADR-002: Feature scope — what the store surface includes
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
Honker's full surface (the interface prior art) covers nine feature
|
||||
families: notify/listen, streams, queues, scheduler, outbox, named
|
||||
locks, rate limits, result storage, and loadable-extension serving.
|
||||
The crate must decide which of these it ships, and the evidence-first
|
||||
answer (`docs/research/consumer-inventory.md`, 2026-10-04) grades each
|
||||
row by whether any consumer document actually names a need. This ADR
|
||||
fixes that scope so contract work isn't done against features no
|
||||
consumer wants.
|
||||
|
||||
## Decision
|
||||
|
||||
**In scope, first-class** (a named consumer is pinned or the operator
|
||||
has recorded the need):
|
||||
|
||||
- **notify / listen** — the transactional fire-and-forget signal layer.
|
||||
Pinned (alkfs path-tree invalidation; the `watch_fires_on_commit`
|
||||
POC evidence) and documented (alkblobs fleet visibility).
|
||||
- **streams** — durable pub/sub with per-consumer offsets and
|
||||
replay-on-attach. Operator-authority record (2026-10-04,
|
||||
`consumer-inventory.md`): type-filtered event watching / repo-change
|
||||
subscriptions that notify cannot honestly serve (no replay, no
|
||||
durability).
|
||||
|
||||
**In scope** (documented need, thinner):
|
||||
|
||||
- **named locks** — TTL coordination locks. Pinned (alkblobs fleet
|
||||
sweeper) and documented (alkfs writer coordination); alkgit's CAS has
|
||||
an alternative design a lock is allowed to replace but needn't.
|
||||
- **queues** — durable at-least-once work with the transactional
|
||||
enqueue shape. Documented (alkfs sync/fetch-on-miss outbox; alkblobs
|
||||
maintenance cadence).
|
||||
- **outbox helper** — a thin helper *over* queues (enqueue inside the
|
||||
business transaction + a delivery/consumption worker shape), not a
|
||||
separately-needed feature. Same evidence as queues.
|
||||
- **scheduler** — cron/`@every` enqueueing into named queues,
|
||||
leader-elected. Documented-thin; the family-wide
|
||||
"who sweeps/renews/reaps" need. May collapse into queues if design
|
||||
shows scheduler = queues + tick (tracked in OQ-09).
|
||||
|
||||
**Cut-flag** (no named consumer; not silently included):
|
||||
|
||||
- **rate limits** — alkgit enforces budgets in its own wire layer; a
|
||||
store-level rate limit is not the proven mechanism. The row stays
|
||||
listed with its flag visible; cut at implementation time rather than
|
||||
carried on momentum. If a distributed (cross-process) rate-limit need
|
||||
materializes, it gets a consumer-inventory row first.
|
||||
- **result storage** (`save_result`/`get_result`/`sweep_results`) —
|
||||
adjacent-tooling shape, useful but named by no consumer. Same
|
||||
cut-flag treatment.
|
||||
|
||||
**Out:**
|
||||
|
||||
- The **loadable-extension surface** — no consumer needs it; every
|
||||
identified consumer is in-process Rust attaching to its own
|
||||
connection (OQ-ST-07, resolved cut-only). Cut unless a consumer
|
||||
appears.
|
||||
- Honker's exclusion lines, inherited as out: workflow DAGs, task
|
||||
chains/chords, multi-writer replication, distributed (cross-machine)
|
||||
locking. They stay out unless a consumer document grows a row.
|
||||
|
||||
New consumers add a row to `docs/research/consumer-inventory.md`
|
||||
*before* being assumed into scope.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- The contract surface is small and every part of it has a nameable
|
||||
consumer — contract-pinning work isn't spent on speculative surface.
|
||||
- Scope honesty is mechanical: the inventory is the ledger, and the
|
||||
cut-flag rows keep their flags visible instead of being silently
|
||||
dropped or silently shipped.
|
||||
|
||||
**Negative**
|
||||
|
||||
- Rate limits and result storage, if ever needed, require the consumer
|
||||
inventory + contract process first — no ad-hoc additions.
|
||||
- Cutting the extension surface narrows honker compatibility; any
|
||||
future non-Rust consumer of these SQLite files cannot use honker's
|
||||
extension (acceptable: none exists).
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/consumer-inventory.md` — per-feature evidence grades.
|
||||
- OQ-ST-01, OQ-ST-07 (`docs/research/phase-0.md`) — resolved by this
|
||||
evidence; promoted as OQ-01 and OQ-07 in
|
||||
`docs/architecture/open-questions.md`.
|
||||
- [core-contract.md](../core-contract.md) — the trait surface this
|
||||
scope defines.
|
||||
- [ADR-007](007-transactional-seam.md) — the transactional property all
|
||||
in-scope enqueue/publish/notify features lean on.
|
||||
@@ -0,0 +1,104 @@
|
||||
# ADR-003: SQLite engine — rusqlite + published honker-core, bridged at the trait seam
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The SQLite engine's options were never a single "rusqlite vs sqlx"
|
||||
axis (OQ-ST-03). Three distinct postures existed:
|
||||
|
||||
1. **honker-core on our own rusqlite connection** — we own
|
||||
connection/schema/watcher wiring; honker supplies the SQL-function
|
||||
machinery (`attach_notify`, `attach_honker_functions`,
|
||||
`bootstrap_honker_schema`).
|
||||
2. **honker-rs as the SQLite substrate** (`Database::open` — honker
|
||||
holds its own connections; sync-only; transactions pin the
|
||||
connection mutex).
|
||||
3. **raw SQL over sqlx-sqlite with the honker loadable extension**
|
||||
(`.so` built from published source, loaded per pool connection).
|
||||
|
||||
POC #1 (`docs/research/poc-sqlite-posture-findings.md`, ran 2026-10-04;
|
||||
POC #2 is its pg twin) measured postures 1
|
||||
and 3 end-to-end and settled the choice. Constraint facts that outlive
|
||||
the comparison:
|
||||
|
||||
- rusqlite 0.40.x (what honker-core 0.5.0 pins) needs rustc ≥ 1.99
|
||||
(`cfg_select!`) — a deployment note for any binary linking it.
|
||||
- sqlx's `libsqlite3-sys` range collides with rusqlite 0.40's — mixed
|
||||
rusqlite+sqlx binaries need a vendored one-line patch today. The
|
||||
per-engine-crate split ([ADR-001]) keeps engine binaries
|
||||
single-driver, dissolving this.
|
||||
- The async-facing-trait + sync-bridge posture is family-standard
|
||||
(alktty REQ-TTY-01; alkblobs store-api.md): bridge-at-the-seam is a
|
||||
supported posture, not a workaround — which removes native-async's
|
||||
main differentiator before any measurement.
|
||||
|
||||
## Decision
|
||||
|
||||
The SQLite engine uses **posture 1**: published `honker-core = 0.5`
|
||||
as a library dependency over the crate's own rusqlite connections
|
||||
(`bundled-sqlite` for hermetic builds).
|
||||
|
||||
- **Driver**: rusqlite, riding honker-core's pinned version.
|
||||
- **Async seam**: bridged. The writer slot (`Writer`) +
|
||||
reader pool (`Readers`) + every engine call in `spawn_blocking`.
|
||||
A dedicated std-thread bridge (one thread owning the writer conn,
|
||||
ops over mpsc) is the recorded optimization path if seam throughput
|
||||
ever demands it — not the default.
|
||||
- **Wake**: honker-core's `SharedUpdateWatcher` inherited (p50 ≈ 1.4 ms
|
||||
at the default 1 ms `PRAGMA data_version` cadence, with
|
||||
battle-tested failure handling — subscriber senders close on watcher
|
||||
death, never silent-hang). A self-built thin watcher was benchmarked
|
||||
only to confirm honker-core's is tighter (p50 1.40 vs 2.15 ms; max
|
||||
29 vs 172 ms); no hybrid is warranted.
|
||||
- **Transactions**: the engine opens `BEGIN IMMEDIATE` on the writer
|
||||
slot and hands back a caller-held tx handle ([ADR-007]). The slot
|
||||
lease models WAL's single-writer honestly: a long transaction parks
|
||||
the writer — that is SQLite's shape, not an emulation artifact.
|
||||
- **No `.so` runtime artifact, no vendored patches** in the engine
|
||||
crate.
|
||||
|
||||
Rejected alternatives, briefly, for the record: posture 3 is *viable*
|
||||
but pays ~2× p50 seam cost, a runtime dlopen artifact, wider
|
||||
dependency tree, and sqlx 0.9 ergonomic warts (unsafe `extension()`,
|
||||
`SqlSafeStr`) for no compensating advantage. Posture 2 (honker-rs)
|
||||
was never measured because posture 1 dominates it on control with
|
||||
comparable reuse.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- ~0.35 ms p50 transactional seam (measured), ~1.4 ms wake latency,
|
||||
and honker-core's watcher failure-handling inherited for free.
|
||||
- Static linkage: "open a path, get a store" needs no runtime
|
||||
artifacts.
|
||||
- The engine's job shrinks to wiring + the SQL surfaces honker doesn't
|
||||
cover (queue semantics depth, OQ-ST-05's SQLite side rides honker's
|
||||
machinery directly) + the async bridge.
|
||||
|
||||
**Negative**
|
||||
|
||||
- honker-core pins rusqlite ^0.40.1; version movement in honker-core
|
||||
moves our rusqlite. A honker-core quality read is the recorded fork
|
||||
trigger ([ADR-005], OQ-06).
|
||||
- rustc ≥ 1.99 required by rusqlite 0.40.x — binaries linking this
|
||||
engine carry that toolchain floor (deployment-matrix row,
|
||||
[deployment.md](../deployment.md)).
|
||||
- The sync bridge means engine ops pay a `spawn_blocking` hop — mostly
|
||||
invisible next to SQLite write costs, but it shapes the tx-handle
|
||||
design ([ADR-007]).
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/poc-sqlite-posture-findings.md` — the measurements.
|
||||
- OQ-ST-03 (`docs/research/phase-0.md`) — resolution record.
|
||||
- [ADR-001](001-crate-split.md) — why the engine crate is
|
||||
single-driver.
|
||||
- [ADR-004](004-postgres-driver.md) — the Postgres counterpart.
|
||||
- [ADR-007](007-transactional-seam.md) — the tx-handle shape both
|
||||
engines implement.
|
||||
- [engine-sqlite.md](../engine-sqlite.md) — the engine spec this
|
||||
decision defines.
|
||||
@@ -0,0 +1,104 @@
|
||||
# ADR-004: Postgres engine — tokio-postgres + deadpool-postgres, hand-rolled LISTEN forwarder
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The Postgres engine's driver question (OQ-ST-03, Postgres half) had
|
||||
three candidate families: sqlx (single API across engines, and the
|
||||
driver pgboss-rs carries), tokio-postgres + deadpool-postgres (the
|
||||
alkblobs POC evidence base, natively async), and adopt/fork pgboss-rs
|
||||
(bringing sqlx where the queue lives). POC #2
|
||||
(`docs/research/poc-pg-posture-findings.md`, ran 2026-10-04) validated
|
||||
the tokio-postgres posture end-to-end and measured the pieces the
|
||||
contract depends on. The sqlite-arm comparison facts from
|
||||
[ADR-003] apply on this side too: pgboss-rs brings sqlx (a second
|
||||
driver per binary), has no LISTEN/NOTIFY at all (verified against the
|
||||
checkout — consumption is `fetch_job` polling), so the push-reactivity
|
||||
half is this crate's work regardless of fork-or-adopt.
|
||||
|
||||
## Decision
|
||||
|
||||
The Postgres engine uses:
|
||||
|
||||
- **tokio-postgres 0.7.x + deadpool-postgres 0.14.x** — pooled
|
||||
connections for queries/claims (exactly-once via
|
||||
`FOR UPDATE SKIP LOCKED`), natively async (client is `Send + Sync` —
|
||||
no bridge, no spawn_blocking, the tx handle holds the pooled object
|
||||
directly).
|
||||
|
||||
- **A hand-rolled LISTEN forwarder** (the POC's ~90-line shape) — a
|
||||
dedicated non-pooled listener connection per process, a poll-message
|
||||
loop fanning out to a bounded broadcast channel, immediate reconnect
|
||||
with exponential backoff (50 ms → 2 s cap), re-LISTEN from the
|
||||
channel list after every reconnect, and a **synthetic reconnect-wake
|
||||
on a reserved channel** closing the no-replay hole
|
||||
([ADR-006]). `postgres-notify` 0.3.8 was evaluated in-probe and
|
||||
passed over (derive-not-adopt: lazy reconnect, connect_script
|
||||
skipped at initial connect, unquoted-identifier LISTENs,
|
||||
single-maintainer posture; it stays a recorded fallback if upstream
|
||||
improves).
|
||||
|
||||
- **Queue machinery re-derived on this driver** with the pg-boss
|
||||
schema family as design reference — not adopted/forked as a
|
||||
dependency ([ADR-005]; semantics depth is [queues.md]'s work,
|
||||
OQ-ST-05). The minimal-queue ground is measured: enqueue/claim/ack
|
||||
with `SKIP LOCKED` is ~40 lines of SQL and passed the full property
|
||||
suite.
|
||||
|
||||
- **Default consumption posture: LISTEN-driven claim with a re-poll
|
||||
safety net**; poll-only remains the fallback (interval-tunable) when
|
||||
a LISTEN connection is unavailable or unwanted. Measured: claim
|
||||
latency p50 3–6 ms LISTEN-driven vs 32–50 ms at a 50 ms poll.
|
||||
|
||||
Structural constraints the engine owns (all test-pinned in POC #2):
|
||||
|
||||
- Pooled connections **cannot carry LISTEN** (deadpool#360 —
|
||||
registers server-side, never delivers). The listener connection is a
|
||||
per-process budget line outside the pool (`max_size + 1`).
|
||||
- `pg_notify` payloads are ≤ 8000 bytes; the engine checks the limit
|
||||
client-side and returns a typed error before the round-trip. Large
|
||||
payloads ride a table row with the id in the notification (the
|
||||
outbox shape).
|
||||
- A query-vs-poll starvation deadlock exists if the listener's poll
|
||||
loop isn't running before the first client query on the connection;
|
||||
and dropping the `Client` closes the server session even if the
|
||||
`Connection` task survives. Both are hand-rolled-forwarder failure
|
||||
modes, owned and test-pinned by the engine.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- Single-driver engine, natively async, zero version conflicts, no
|
||||
vendoring.
|
||||
- The push channel pgboss-rs lacks, this engine gets — measured at
|
||||
5–16× claim-latency improvement over even a 50 ms poll.
|
||||
- Multi-host is native (no single-host assumption anywhere; property
|
||||
tests ran all-through-network).
|
||||
- The forwarder's two deadlock pitfalls are *learned territory* with
|
||||
pinned tests, not unknowns.
|
||||
|
||||
**Negative**
|
||||
|
||||
- The listener forwarder is ours to maintain (~90 lines + the two
|
||||
pinned failure modes) — the cost of not adopting `postgres-notify`.
|
||||
- Per-process connection budget grows by one listener connection per
|
||||
process that listens (a deployment-matrix row; consumers sizing
|
||||
pools must account for it).
|
||||
- The pg queue schema is re-derived work; the pg-boss family is
|
||||
reference, not free.
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/poc-pg-posture-findings.md` — measurements and the
|
||||
pinned property suite.
|
||||
- OQ-ST-03, OQ-ST-05's posture half (`docs/research/phase-0.md`).
|
||||
- [ADR-001](001-crate-split.md) — single-driver binaries.
|
||||
- [ADR-005](005-dependency-ownership.md) — the ownership calculus this
|
||||
decision applies per-subsystem.
|
||||
- [ADR-006](006-wake-and-delivery-contract.md) — the wake contract the
|
||||
forwarder implements on this engine.
|
||||
- [engine-postgres.md](../engine-postgres.md) — the engine spec.
|
||||
@@ -0,0 +1,70 @@
|
||||
# ADR-005: Dependency ownership — published libraries by default, named fork triggers
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (posture); fork triggers are live assessments tracked in OQ-06
|
||||
|
||||
## Context
|
||||
|
||||
The crate's reactive machinery sits on third-party work: honker
|
||||
(SQLite side) and, conceptually, pgboss-rs (Postgres queue family).
|
||||
Neither is adopted-by-adjacency — per guiding principle 3, ownership of
|
||||
each subsystem is a deliberate per-question decision. The workspace
|
||||
precedent is the targeted fork (alksocks' fast-socks5 extraction), but
|
||||
a fork is expensive (especially when the forked code is sync and the
|
||||
family standard is tokio — a fork of honker-rs is already a serious
|
||||
port).
|
||||
|
||||
Two POCs (2026-10-04) measured the relevant postures per subsystem;
|
||||
each carries its own record. This ADR fixes the *default calculus* and
|
||||
the per-subsystem resolutions so Phase 1 design work starts from
|
||||
stable ground.
|
||||
|
||||
## Decision
|
||||
|
||||
**Default posture: consume published libraries.** Fork or vendor only
|
||||
when a named trigger fires. Per subsystem:
|
||||
|
||||
| Subsystem | Posture | Notes |
|
||||
|---|---|---|
|
||||
| honker-core (SQLite engine machinery) | published library, `honker-core = 0.5` | Fork trigger: the Phase 1 quality read of the watcher/transactional core finds a defect, OR a needed change upstream won't take. OQ-06 tracks the read. |
|
||||
| rusqlite | published, riding honker-core's pin | Pin movement is honker-core's; we ride it ([ADR-003]). |
|
||||
| tokio-postgres + deadpool-postgres | published library, as-is | Clean, zero conflicts, actively maintained (POC #2). |
|
||||
| postgres-notify | **not adopted** (derive-not-adopt) | Lazy reconnect, connect_script skipped at initial connect, unquoted-identifier LISTENs, single maintainer. Hand-rolled forwarder instead ([ADR-004]). Fallback if upstream improves materially. |
|
||||
| pg-boss queue machinery (pgboss-rs / node pg-boss) | **re-derived** on our driver; schema family as *design reference* only | pgboss-rs brings sqlx (second driver per binary) and has no LISTEN/NOTIFY (verified); the push half is ours either way, so queue-machinery reuse value is the honest comparison point and it loses on that arithmetic. Semantics depth: [queues.md], OQ-05. |
|
||||
| honker-rs interface shape | **design reference**, not a dependency | Sync-only (no tokio); the surface is prior art for contract pinning ([ADR-006]), not code to ship. |
|
||||
|
||||
If any fork fires, forking is normal work we own (license/provenance
|
||||
recorded per AGENTS.md §3 at adoption time) — it is not an exception
|
||||
to this posture.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- Zero vendored code in the tree by default; upgrades are
|
||||
`cargo update` work, not patch-management work.
|
||||
- The fork triggers are concrete and named *before* the quality read,
|
||||
so the read produces a decision, not a debate.
|
||||
- Per-subsystem votes are recorded with evidence, so no future
|
||||
discussion re-litigates from scratch.
|
||||
|
||||
**Negative**
|
||||
|
||||
- honker-core 0.5 remains alpha-quality software per its own README in
|
||||
the link graph; the quality read (OQ-06) is the mitigation gate.
|
||||
- The pg queue machinery is written by us — the pg-boss family's
|
||||
battle-tested edge cases must be re-earned by design + tests
|
||||
([queues.md]).
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/poc-sqlite-posture-findings.md` §"What feeds where"
|
||||
(posture votes), `docs/research/poc-pg-posture-findings.md`
|
||||
§"Dependency postures".
|
||||
- OQ-ST-05, OQ-ST-06 (`docs/research/phase-0.md`) — the resolution
|
||||
records.
|
||||
- [ADR-003](003-sqlite-driver.md), [ADR-004](004-postgres-driver.md) —
|
||||
the per-engine driver decisions applying this posture.
|
||||
- OQ-06 (`docs/architecture/open-questions.md`) — the quality-read
|
||||
tracker.
|
||||
@@ -0,0 +1,128 @@
|
||||
# ADR-006: Wake contract — opaque wake + re-read, and the notify-vs-streams delivery split
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (the contract skeleton the evidence supports); remaining
|
||||
surface pinning is OQ-04's contract work against this ADR
|
||||
|
||||
## Context
|
||||
|
||||
The two engines' wake mechanisms are structurally different — SQLite
|
||||
wakes by a watcher polling `PRAGMA data_version` (deliver-on-commit,
|
||||
no server push), Postgres by LISTEN/NOTIFY (server push,
|
||||
connection-bound, no replay). A naive unification either collapses to
|
||||
polling behavior on the strong side or promises semantics only one
|
||||
engine has. Both POCs (2026-10-04) measured the pieces and — the
|
||||
load-bearing finding — verified that **the two engines can share the
|
||||
same wake contract unchanged**: wake is opaque, delivery is not
|
||||
guaranteed to be precise, and consumers re-read state.
|
||||
|
||||
Honker's own processing-guarantees table is the cautionary prior art:
|
||||
per-binding auto-checkpoint-vs-manual-save ambiguity produces
|
||||
*different* guarantees under one function name. A single-crate version
|
||||
must pick one answer per mechanism, not inherit the table.
|
||||
|
||||
## Decision
|
||||
|
||||
The contract fixes three things:
|
||||
|
||||
### 1. The unified wake contract: opaque wake + re-read state
|
||||
|
||||
`listen()` (and stream/queue subscription wake) delivers an **opaque
|
||||
wake signal: "something changed; re-read the state you care about."**
|
||||
The contract never carries semantic content, ordering promises, or
|
||||
change descriptions. Consumers that need the actual data re-read it
|
||||
(the hot-path pattern the ecosystem already uses: a subscriber holding
|
||||
a cache invalidates on wake and re-reads, instead of re-polling).
|
||||
|
||||
- SQLite: `data_version` watcher fires on commit; wakes **coalesce**
|
||||
under bursts (overtriggering on purpose — "one indexed SELECT is
|
||||
cheap; a missed wake is a correctness bug").
|
||||
- Postgres: LISTEN delivers per-notification (no coalescing), and the
|
||||
forwarder's synthetic reconnect-wake covers connection gaps.
|
||||
- Both: consumer code must be correct if wakes repeat, coalesce, or
|
||||
arrive in any order, with at-least-once wake delivery. Exactly-once
|
||||
*processing* semantics belong to queues/streams (below), never to
|
||||
the wake layer.
|
||||
|
||||
Measured ground: wake latency p50 ≈ 1.1–2.2 ms on both engines at
|
||||
default cadence; per-payload burst delivery verified (30/30 on pg,
|
||||
coalesced-but-complete re-read on SQLite).
|
||||
|
||||
### 2. Delivery guarantees, pinned per mechanism (no per-binding table)
|
||||
|
||||
| Mechanism | Durability | Replay | Atomicity | Guarantee |
|
||||
|---|---|---|---|---|
|
||||
| `notify` / listen | none | never | commit-atomic (delivers at commit; rollback drops) | fire-and-forget, at-most-once per listener session |
|
||||
| streams | durable table row | yes, per-consumer offset, replay-on-attach default | publish is commit-atomic | every committed event readable by every consumer that hasn't passed its offset |
|
||||
| queues | durable row | claim/ack model | enqueue is commit-atomic | at-least-once *work* with visibility timeouts |
|
||||
|
||||
- The trait **does not promise replay under `listen()`** — on either
|
||||
engine. Durability+replay needs are what streams are for; this split
|
||||
is the contract's load-bearing line.
|
||||
- Listener sessions start "from now" (SQLite: the watcher's current
|
||||
state; Postgres: `MAX(id)`-equivalent at attach). History belongs to
|
||||
streams.
|
||||
- Wake *delivery* failures surface, not silence: watcher death closes
|
||||
subscriber channels (SQLite, honker-core's
|
||||
`WatcherDeathGuard` behavior); Postgres connection gaps surface as
|
||||
the synthetic reconnect-wake on a reserved channel
|
||||
([ADR-004], [engine-postgres.md]).
|
||||
|
||||
### 3. Reserved names are a contract surface
|
||||
|
||||
The Postgres forwarder's synthetic reconnect-wake needs a reserved
|
||||
channel namespace no consumer channel may collide with. The naming
|
||||
convention for reserved/meta channels (and any internal queue/stream
|
||||
names) is pinned in the core contract — exact strings are OQ-04
|
||||
contract work; the *existence of a reserved namespace* is decided
|
||||
here.
|
||||
|
||||
### The caching-subscriber pattern rides the same contract
|
||||
|
||||
A subscriber that also holds a cache gets invalidation from the
|
||||
opaque wake + re-read: the contract deliberately does not carry
|
||||
key-level payloads, so cache clients key their invalidation on their
|
||||
own read-set, informed by channel names (the channel name is the one
|
||||
piece of semantic content `listen()` carries). Key-payload design,
|
||||
if any consumer needs richer invalidation, is a
|
||||
consumer-inventory-row-gated extension — not assumed now.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- One wake contract, verified identically on both engines — the
|
||||
"how does a change become visible" question alkstore exists to
|
||||
answer, answered once.
|
||||
- The delivery-guarantee table is pinned per mechanism, killing the
|
||||
per-binding ambiguity class honker's docs exhibit.
|
||||
- Consumer code branches on *mechanism choice* (notify vs streams vs
|
||||
queues), never on engine type (guiding principle 4; the honest
|
||||
single/multi-host line is [deployment.md]'s and OQ-08's, not this
|
||||
contract's).
|
||||
|
||||
**Negative**
|
||||
|
||||
- The opaque wake is deliberately less informative than
|
||||
key-payload notify systems; consumers needing fine-grained
|
||||
invalidation pay a re-read or need streams.
|
||||
- Wake overtriggering on SQLite means consumers must be idempotent on
|
||||
wake (a documented consumer obligation, part of the contract text).
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/poc-sqlite-posture-findings.md` §Probe 2,
|
||||
§"What feeds where"; `docs/research/poc-pg-posture-findings.md`
|
||||
§Sub-module W.
|
||||
- OQ-ST-04 (`docs/research/phase-0.md`) — de-risked by these
|
||||
measurements; the contract remainder is OQ-04 +
|
||||
`core-contract.md`.
|
||||
- [ADR-002](002-feature-scope.md) — notify/listen and streams are
|
||||
first-class; the guarantee split is why both exist.
|
||||
- [ADR-004](004-postgres-driver.md) — the forwarder owning the
|
||||
Postgres side of this contract.
|
||||
- [ADR-007](007-transactional-seam.md) — commit-atomicity's mechanism
|
||||
(the `*_tx` seam).
|
||||
- [core-contract.md](../core-contract.md) — the spec carrying this
|
||||
contract's full surface.
|
||||
@@ -0,0 +1,102 @@
|
||||
# ADR-007: Transactional seam — caller-held tx handle with `*_tx` methods
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (the seam shape both POCs verified); handle-type ergonomics
|
||||
(generic vs downcast) is OQ-04 contract work
|
||||
|
||||
## Context
|
||||
|
||||
The load-bearing property of the whole store (guiding principle 2) is
|
||||
**transactional local-adjacency**: enqueue/publish/notify inside the
|
||||
same transaction as the caller's business write; rollback drops both.
|
||||
Serving it through an async trait over two transaction models —
|
||||
rusqlite's sync, thread-bound connection vs tokio-postgres's
|
||||
`Send + Sync` client — is the one genuinely driver-coupled design
|
||||
point (OQ-ST-03's framing). Both POCs (2026-10-04) built the candidates
|
||||
and measured them.
|
||||
|
||||
## Decision
|
||||
|
||||
The core contract's transactional seam is a **caller-held transaction
|
||||
handle**:
|
||||
|
||||
```text
|
||||
store.begin_tx() -> TxHandle
|
||||
handle.enqueue_tx(name, opts, payload) -> job id
|
||||
handle.publish_tx(name, payload) -> event id
|
||||
handle.notify_tx(channel, payload)
|
||||
handle.save_offset_tx(stream, consumer, offset)
|
||||
handle.commit() / handle.rollback()
|
||||
```
|
||||
|
||||
- The handle is owned by the caller across `await` points; every
|
||||
`*_tx` operation lands in *the caller's transaction*; commit/rollback
|
||||
are explicit and owned by the caller. Non-`_tx` operations are the
|
||||
auto-commit convenience counterparts, each atomic alone.
|
||||
- **Postgres** ([ADR-004]): the handle holds the pooled connection
|
||||
object directly (tokio-postgres `Client` is `Send + Sync` — verified
|
||||
by a compile-time probe). Straight `.await`s, no bridge, no hop
|
||||
cost. Commit/rollback return the object to the pool.
|
||||
- **SQLite** ([ADR-003]): the handle is a **writer-slot lease** —
|
||||
`begin_tx` acquires the `Writer` slot and opens `BEGIN IMMEDIATE`;
|
||||
every handle op round-trips `spawn_blocking` to that connection
|
||||
(rusqlite is not `Send`-across-await / not `Sync`). Holding the slot
|
||||
across `await` points *is* how the lease works — a long transaction
|
||||
parks the single WAL writer, which is SQLite's own shape.
|
||||
- **Closure-scoped transactions** (`with_tx`) are *available as a
|
||||
wrapper over* the handle shape, not instead of it (the POC verified
|
||||
the reverse composition doesn't work: a closure cannot outlive
|
||||
itself, so caching-subscriber state can't escape it).
|
||||
- Both engines deliver the property **natively** — no emulation:
|
||||
in-tx notify/Notify delivers only at commit; rollback drops job rows,
|
||||
business rows, and notifications together (the POC property tests on
|
||||
both engines green: no ghosts).
|
||||
|
||||
The POC's trait sketch carried two frictions, both now Phase 1
|
||||
contract-pinning input rather than open risk:
|
||||
|
||||
1. The `as_any_mut` downcast per engine for `*_tx` methods on a
|
||||
`dyn TxHandle` — small, but a generic or enum-favored handle may be
|
||||
cleaner. OQ-04 decides with the full contract.
|
||||
2. Thread-affinity of rusqlite tx ops (SQLite handle ops must
|
||||
round-trip the same blocking thread) — inherent to [ADR-003]'s
|
||||
bridge, documented as a contract note ("SQLite tx ops are
|
||||
serialized by the writer slot"), not a defect.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- The load-bearing property is one seam, both engines, POC-verified
|
||||
end-to-end — the dual-write problem is structurally prevented, not
|
||||
consumer-disciplined.
|
||||
- Long transactions compose naturally (checkout-then-work shape);
|
||||
nothing about the seam forbids multi-statement business
|
||||
transactions with interleaved reads.
|
||||
- Per-engine bridging differences are invisible to consumer code: the
|
||||
same `*_tx` calls, the same commit/rollback ownership.
|
||||
|
||||
**Negative**
|
||||
|
||||
- The SQLite handle parks the writer for its duration — a slow
|
||||
consumer transaction throttles all writers on the file. This is
|
||||
honest (WAL single-writer), but contract docs must say it so
|
||||
consumers budget transactions accordingly.
|
||||
- Per-op `spawn_blocking` hop on SQLite tx ops (measured fine at
|
||||
~0.35 ms p50; the dedicated-thread bridge is the recorded
|
||||
optimization).
|
||||
- `with_tx` wrapping is a convenience surface we must ship and test
|
||||
so it doesn't accrete ad-hoc in consumer code.
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/poc-sqlite-posture-findings.md` §Arm A, §Probe 1/3;
|
||||
`docs/research/poc-pg-posture-findings.md` §Sub-module T.
|
||||
- OQ-ST-04 (`docs/research/phase-0.md`), the `*_tx` seam bullet.
|
||||
- [ADR-003](003-sqlite-driver.md), [ADR-004](004-postgres-driver.md) —
|
||||
the per-engine handle mechanics.
|
||||
- [ADR-006](006-wake-and-delivery-contract.md) — commit-atomicity is
|
||||
the `*_tx` property this seam carries into the notify contract.
|
||||
- [core-contract.md](../core-contract.md) — the spec whose transaction
|
||||
section this ADR defines.
|
||||
@@ -0,0 +1,118 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-04
|
||||
---
|
||||
|
||||
# Deployment
|
||||
|
||||
What a deployer must know to size, run, and reason about alkstore
|
||||
engines: host semantics, connection budgets, durability knobs, and
|
||||
where engine differences may honestly surface in the contract. The
|
||||
capability-surface *decision* (how much of this the trait exposes) is
|
||||
OQ-08's; this document holds the facts and the decision's frame.
|
||||
|
||||
## Host semantics
|
||||
|
||||
| Engine | Host posture | Notes |
|
||||
|---|---|---|
|
||||
| SQLite | **single-machine**, file-backed | NFS two-writers unsupported (honker's honesty posture, inherited, [ADR-003](decisions/003-sqlite-driver.md)). Cross-process *on one host* is verified POC ground (`data_version` is cross-process by nature). |
|
||||
| Postgres | **multi-host native** | Nothing assumes a shared host; POC #2 ran all-through-network (docker bridge) with the same properties ([ADR-004](decisions/004-postgres-driver.md)). |
|
||||
|
||||
The unified trait must not pretend SQLite is multi-host — but whether
|
||||
that honesty lives as runtime capability flags, compile-time engine
|
||||
knowledge, or a documented matrix only is OQ-08
|
||||
([ADR-006](decisions/006-wake-and-delivery-contract.md) note: the
|
||||
trait's shape constrains where capability differences can surface).
|
||||
|
||||
Options for OQ-08, with their shape:
|
||||
|
||||
1. **Compile-time only** — a consumer chooses an engine crate at
|
||||
dependency time; the engine's docs carry its deployment facts.
|
||||
Smallest contract; nothing runtime to match on.
|
||||
2. **`Store::capabilities()`** — a runtime description
|
||||
(payload limits, wake cadence knobs, host semantics). Lets a
|
||||
consumer adapt (e.g., chunk large notify payloads) but adds a
|
||||
contract surface all engines must keep honest.
|
||||
3. **Deployment matrix only** (this document) — no API surface. The
|
||||
honest-middle choice; matches the ecosystem's doc-first posture
|
||||
but provides no programmatic guard.
|
||||
|
||||
## Connection budgets
|
||||
|
||||
### SQLite engine
|
||||
|
||||
- Connections are in-process (writer + reader pool + watcher thread
|
||||
owning a connection). No external budget lines; the file lock is
|
||||
the OS-level resource.
|
||||
|
||||
### Postgres engine
|
||||
|
||||
| Connection class | Count | Notes |
|
||||
|---|---|---|
|
||||
| Pool | `max_size` per process | claims/queries via deadpool |
|
||||
| Listener | **+1** per LISTEN-ing process | non-pooled, dedicated; pooled connections cannot carry LISTEN (deadpool#360, test-pinned — [ADR-004](decisions/004-postgres-driver.md)) |
|
||||
| — | — | Sizing rule: `max_size + 1` per process; verify end-to-end accounting (POC #2's pgdiag-st-4: pool 6 + 1 listener + probe conns = exact server-side count) |
|
||||
|
||||
Shared-server co-tenancy (the alkblobs ADR-008 precedent —
|
||||
`/workspace/@alkdev/alkblobs/docs/architecture/decisions/` — consumer
|
||||
tables co-tenant the pg instance) is supported and expected — the
|
||||
[naming / reserved namespace contract](core-contract.md#naming--reserved-namespace)
|
||||
protects reserved names; queue/stream/lock tables are schema-named to
|
||||
avoid collisions (layout decision in [queues.md](queues.md), OQ-05's
|
||||
namespace bullet).
|
||||
|
||||
## Durability knobs
|
||||
|
||||
| Engine | Knob | Shape |
|
||||
|---|---|---|
|
||||
| SQLite | `synchronous` | WAL + `NORMAL` shipped ([ADR-003](decisions/003-sqlite-driver.md)); FULL is available consumer-side for stricter durability; commit fsyncs land at WAL checkpoints (the ~1000-commit spike cadence, POC #1) |
|
||||
| Postgres | `synchronous_commit` | per-session knob; `on` is ship config (p50 2.40 ms seam); `off` trades max-tail (40.9 ms) for slightly better p50 — measured, honest trade ([ADR-004](decisions/004-postgres-driver.md)); session-level SET mechanics POC-verified |
|
||||
|
||||
These are engine-configuration concerns, *not* trait surface (a
|
||||
consumer may set them via their own engine config — what part of
|
||||
engine config is contract-level `Store::open` shape vs engine-crate
|
||||
docs is OQ-04's config-shape item).
|
||||
|
||||
## Toolchain / platform notes
|
||||
|
||||
| Note | Engine | Affects |
|
||||
|---|---|---|
|
||||
| rusqlite 0.40.x needs rustc ≥ 1.99 | SQLite | any binary linking `alkstore-sqlite` ([ADR-003](decisions/003-sqlite-driver.md)) |
|
||||
| `bundled-sqlite` adds a C build (~10 s dev, cacheable) | SQLite | build/CI time |
|
||||
| `libsqlite3-sys` collision with sqlx today | any mixed-driver binary | structurally avoided ([ADR-001](decisions/001-crate-split.md) single-driver rule) |
|
||||
| Listener `application_name` set for diagnosability (kill-targetable) | Postgres | ops runbooks ([ADR-004](decisions/004-postgres-driver.md)) |
|
||||
|
||||
## Consumer-facing latency profile (indicative, POC-measured)
|
||||
|
||||
From both POCs (single-box, relative shapes are the deliverable —
|
||||
[ADR-003](decisions/003-sqlite-driver.md) and
|
||||
[ADR-004](decisions/004-postgres-driver.md) carry the full tables):
|
||||
|
||||
- Tx seam: SQLite ~0.35 ms p50; Postgres ~2.4 ms p50 (ship config).
|
||||
- Wake: ~1.1–2.2 ms p50 both engines at default cadence.
|
||||
- Queue claim: LISTEN-driven 3–6 ms p50 (pg); poll-only interval-bound
|
||||
(32–50 ms at a 50 ms poll).
|
||||
- Absolute numbers will differ per hardware/network; they set
|
||||
*expectations of order*, not SLAs — contract docs must not bake
|
||||
them in ([ADR-007](decisions/007-transactional-seam.md)'s pg-POC
|
||||
note).
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|---|---|---|
|
||||
| [001](decisions/001-crate-split.md) | Crate split | single-driver binaries shape the matrix |
|
||||
| [003](decisions/003-sqlite-driver.md) | SQLite driver | bundling, toolchain floor |
|
||||
| [004](decisions/004-postgres-driver.md) | Postgres driver | listener budget line, forwarder posture |
|
||||
| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract | where capability differences may surface |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in
|
||||
[open-questions.md](open-questions.md). Key
|
||||
questions affecting this document:
|
||||
|
||||
- **OQ-08**: capability-surface shape — compile-time vs runtime flags
|
||||
vs matrix-only (open)
|
||||
- **OQ-04**: engine-config surface in the contract's `Store::open`
|
||||
shape (shared) (open)
|
||||
@@ -0,0 +1,113 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-04
|
||||
---
|
||||
|
||||
# Postgres engine
|
||||
|
||||
The `alkstore-postgres` engine implements
|
||||
[core-contract.md](core-contract.md) on tokio-postgres +
|
||||
deadpool-postgres. This spec records WHAT the engine is internally —
|
||||
connection architecture, the LISTEN forwarder, queue machinery
|
||||
re-derivation — not code-level HOW. Decisions live in ADRs; contract
|
||||
obligations live in the core spec.
|
||||
|
||||
## Identity and posture
|
||||
|
||||
- Single driver, natively async: tokio-postgres 0.7.x +
|
||||
deadpool-postgres 0.14.x ([ADR-004](decisions/004-postgres-driver.md)).
|
||||
No bridge, no `spawn_blocking`; the tx handle holds the pooled
|
||||
object directly (client is `Send + Sync` — POC #2 compile-probe
|
||||
verified).
|
||||
- Multi-host by nature: connections are per-process state; nothing
|
||||
assumes a shared host ([ADR-004](decisions/004-postgres-driver.md),
|
||||
verified all-through-network in POC #2). See
|
||||
[deployment.md](deployment.md).
|
||||
- Queue machinery re-derived on this driver with the pg-boss schema
|
||||
family as design reference
|
||||
([ADR-005](decisions/005-dependency-ownership.md)); semantics depth
|
||||
is [queues.md](queues.md)'s work (OQ-05).
|
||||
|
||||
## Connection architecture
|
||||
|
||||
- **Pool** (deadpool) — queries, claims, and all non-transactional
|
||||
work. Per-connection statement cache, `RecyclingMethod::Fast`
|
||||
(no `DISCARD ALL` recycling; claim SQL re-prepared implicitly with
|
||||
zero errors at POC scale).
|
||||
- **Listener connection** — one dedicated, **non-pooled** connection
|
||||
per process that listens. Pooled connections cannot carry LISTEN
|
||||
(deadpool#360 — registration succeeds, delivery is impossible; the
|
||||
client-wrapper exposes no notification surface, source-verified and
|
||||
test-pinned as `pooled_listen_registers_but_cannot_deliver`). The
|
||||
listener is therefore a per-process budget line *outside* the pool:
|
||||
`max_size + 1` per LISTEN-ing process
|
||||
([deployment.md](deployment.md)).
|
||||
- **Forwarder** — the listener's loop: `poll_message`
|
||||
fanning out into a bounded broadcast channel (lag surfaced, not
|
||||
silent), re-LISTEN from the channel list after every reconnect
|
||||
(exponential backoff 50 ms → 2 s cap), and the synthetic
|
||||
reconnect-wake on the reserved channel
|
||||
([ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||||
- One listener serves N channels and N subscribers; re-attach is a
|
||||
broadcast re-subscribe (no server round-trips); per-channel
|
||||
connections are never warranted at this scale (POC-verified).
|
||||
|
||||
## Mapping the contract
|
||||
|
||||
| Contract piece | Engine realization |
|
||||
|---|---|
|
||||
| notify / listen | `pg_notify(...)` inside the caller's tx (delivers at commit — native commit-atomicity, [ADR-007](decisions/007-transactional-seam.md)); `listen()` via LISTEN on the forwarder's connection, fanout to receivers |
|
||||
| streams | durable event table + per-consumer offset cursors; `pg_notify` as the wake trigger ([ADR-006](decisions/006-wake-and-delivery-contract.md) mechanism split: durable row, LISTEN wake — the pg-boss-family shape) |
|
||||
| queues | re-derived queue table + `FOR UPDATE SKIP LOCKED` claim + LISTEN-driven wake with re-poll safety net (default consumption posture, measured 5–16× vs 50 ms poll; poll-only fallback) |
|
||||
| named locks | advisory-lock-semantics TTL locks (pg-boss-family design reference; depth in OQ-05's design work) |
|
||||
| scheduler / outbox | pg-boss-family design reference, per [queues.md](queues.md) |
|
||||
| begin_tx | pool checkout + `BEGIN`, returning the caller-held handle ([ADR-007](decisions/007-transactional-seam.md)) |
|
||||
| handle ops | straight `.await`s through the held object; commit/rollback returns the object to the pool |
|
||||
|
||||
## Owned failure modes (all test-pinned in POC #2)
|
||||
|
||||
Hand-rolling the forwarder means owning its pitfalls — they are
|
||||
*learned territory*, pinned as passing tests:
|
||||
|
||||
1. **Query-vs-poll starvation deadlock** — the poll loop must be
|
||||
running before the first client query on the listener connection.
|
||||
2. **Client-drop closes the server session** — a long-lived listener
|
||||
keeps its `Client` alive for the listener's lifetime.
|
||||
3. **The no-replay hole** — commit during a connection gap is never
|
||||
re-delivered; recovery = reconnect + synthetic wake + consumer
|
||||
re-read ([ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||||
The `!saw_replay` test pins the honesty.
|
||||
4. **Payload boundary** — `pg_notify` ≤ 8000 bytes; client-side
|
||||
checked, typed error before the round-trip. Large payloads ride a
|
||||
table row with the id in the notification (outbox shape).
|
||||
5. **Read-your-writes** — `read committed` default verified in both
|
||||
directions (in-tx and post-commit).
|
||||
|
||||
Constraint also carried: mixed rusqlite+sqlx binaries would need a
|
||||
vendored patch today — excluded by construction in this engine
|
||||
(single driver, [ADR-001](decisions/001-crate-split.md)), noted for
|
||||
the record in [ADR-003](decisions/003-sqlite-driver.md).
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|---|---|---|
|
||||
| [001](decisions/001-crate-split.md) | Crate split | single-driver engine crate |
|
||||
| [002](decisions/002-feature-scope.md) | Feature scope | which rows this engine serves |
|
||||
| [004](decisions/004-postgres-driver.md) | Driver | tokio-postgres + deadpool; hand-rolled forwarder; re-derived queues |
|
||||
| [005](decisions/005-dependency-ownership.md) | Ownership | published libs as-is; `postgres-notify` derive-not-adopt |
|
||||
| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract | LISTEN push, no replay, synthetic reconnect-wake |
|
||||
| [007](decisions/007-transactional-seam.md) | Tx seam | direct pooled-object handle, no bridging |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in
|
||||
[open-questions.md](open-questions.md). Key
|
||||
questions affecting this document:
|
||||
|
||||
- **OQ-09**: scheduler collapse into queues (shared with
|
||||
[queues.md](queues.md)) (open)
|
||||
- **OQ-05**: queue semantics depth — the pg-boss-family design-input
|
||||
work (open)
|
||||
- **OQ-08**: capability surface (shared with
|
||||
[deployment.md](deployment.md)) (open)
|
||||
@@ -0,0 +1,114 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-04
|
||||
---
|
||||
|
||||
# SQLite engine
|
||||
|
||||
The `alkstore-sqlite` engine implements
|
||||
[core-contract.md](core-contract.md) on rusqlite + published
|
||||
honker-core. This spec records WHAT the engine is internally (its
|
||||
connection architecture, seam, and wake plumbing) — not code-level
|
||||
HOW. Decisions live in ADRs; per-contract obligations live in the core
|
||||
spec and are not restated here.
|
||||
|
||||
## Identity and posture
|
||||
|
||||
- Single driver: rusqlite (riding
|
||||
[honker-core's pin](decisions/003-sqlite-driver.md)),
|
||||
`bundled-sqlite` for hermetic builds. No `.so` runtime artifacts, no
|
||||
vendored patches
|
||||
([ADR-003](decisions/003-sqlite-driver.md),
|
||||
[ADR-001](decisions/001-crate-split.md)).
|
||||
- Sync machinery, async-facing trait: honker-core is sync (std
|
||||
threads, blocking iterators); the engine bridges at the trait seam
|
||||
per the family-standard posture (alktty REQ-TTY-01 precedent).
|
||||
Every engine call runs in `spawn_blocking`.
|
||||
- Single-host by nature: file-backed, one machine, NFS-two-writers
|
||||
unsupported (honker's honesty posture, inherited). See
|
||||
[deployment.md](deployment.md).
|
||||
|
||||
## Connection architecture
|
||||
|
||||
- **Writer** — one dedicated connection; the only one permitted to
|
||||
write. Serializes all mutations (WAL single-writer, modeled
|
||||
honestly, not fought).
|
||||
- **Readers** — a small pool of read connections for lookups and
|
||||
claim/ack work.
|
||||
- **Watcher** — honker-core's `SharedUpdateWatcher`: a dedicated
|
||||
thread polling `PRAGMA data_version` at the default 1 ms cadence,
|
||||
fanning out to listeners, overtriggering on purpose (waking all
|
||||
subscribers per poll tick, even when several commits coalesced
|
||||
inside one tick — wake is a hint; consumers re-read indexed state,
|
||||
[ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||||
- Each connection runs honker's bootstrap at open: pragmas
|
||||
(WAL, `synchronous=NORMAL`, busy timeout), `attach_notify`,
|
||||
`attach_honker_functions`, `bootstrap_honker_schema` (the
|
||||
alknet-filesystem POC's wiring shape).
|
||||
|
||||
## Mapping the contract
|
||||
|
||||
| Contract piece | Engine realization |
|
||||
|---|---|
|
||||
| notify / listen | honker's notify functions inside the caller's tx; `listen()` bridges the watcher's fanout into a tokio receiver (one `spawn_blocking` thread per subscription doing `blocking_send`) |
|
||||
| streams | honker's stream machinery; explicit offset saves through the tx seam |
|
||||
| queues | honker's queue functions ([ADR-002](decisions/002-feature-scope.md)); semantics depth design in [queues.md](queues.md) |
|
||||
| named locks | honker's lock machinery |
|
||||
| scheduler / outbox | honker's counterparts, per [queues.md](queues.md) |
|
||||
| begin_tx | acquires the writer slot, opens `BEGIN IMMEDIATE`, returns the caller-held handle ([ADR-007](decisions/007-transactional-seam.md)) |
|
||||
| handle ops | each `*_tx` op round-trips `spawn_blocking` to the same connection (thread-affinity note in [ADR-007](decisions/007-transactional-seam.md)) |
|
||||
| commit/rollback | releases the writer slot |
|
||||
|
||||
## Obligations and constraints
|
||||
|
||||
- **A long transaction parks the writer** (the slot lease is the
|
||||
honest model). Contract docs must surface this so consumers budget
|
||||
transactions ([ADR-007](decisions/007-transactional-seam.md) negative
|
||||
consequence).
|
||||
- Watcher failure handling is inherited: on watcher death, every
|
||||
subscriber's receiver closes (`WatcherDeathGuard` behavior) —
|
||||
consumers see the close, never a silent hang
|
||||
([ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||||
- Wake coalescing: bursts inside one poll tick produce one wake;
|
||||
correctness is preserved by the re-read contract
|
||||
(POC-pinned: missed-wake stress with correct post-burst re-reads).
|
||||
- **Deployment note**: honker-core 0.5 pins rusqlite ^0.40.1, whose
|
||||
rustc floor is ≥ 1.99; binaries linking this engine carry that
|
||||
requirement ([deployment.md](deployment.md) matrix).
|
||||
- The optimization path, if seam throughput ever demands it: a
|
||||
dedicated std-thread bridge (one thread owning the writer conn, ops
|
||||
over mpsc — measured ~2× the spawn_blocking shape at p50 in POC #1),
|
||||
or a raised watcher cadence for idle CPU. Neither is the default.
|
||||
|
||||
## What rides on the fork question
|
||||
|
||||
Nearly all of honker-core's surface is consumed (Writer/Readers/
|
||||
SharedUpdateWatcher/attach_*). The [quality read
|
||||
(OQ-06)](decisions/005-dependency-ownership.md) is this engine's only
|
||||
open dependency gate: if it names a defect or an upstream-unwon't
|
||||
change, the fork posture
|
||||
([ADR-005](decisions/005-dependency-ownership.md)) fires and this
|
||||
engine's substrate becomes owned code. Until then, published-library
|
||||
consumption stands.
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|---|---|---|
|
||||
| [001](decisions/001-crate-split.md) | Crate split | single-driver engine crate |
|
||||
| [002](decisions/002-feature-scope.md) | Feature scope | which rows this engine serves |
|
||||
| [003](decisions/003-sqlite-driver.md) | Driver | rusqlite + honker-core, bridged seam, inherited watcher |
|
||||
| [005](decisions/005-dependency-ownership.md) | Ownership | published honker-core; named fork triggers |
|
||||
| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract | data_version watcher, coalescing, death-closes-receivers |
|
||||
| [007](decisions/007-transactional-seam.md) | Tx seam | writer-slot lease, `BEGIN IMMEDIATE`, `spawn_blocking` round-trips |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in
|
||||
[open-questions.md](open-questions.md). Key
|
||||
questions affecting this document:
|
||||
|
||||
- **OQ-06**: honker-core quality read — fork-trigger assessment (open)
|
||||
- **OQ-09**: scheduler collapse into queues (shared with
|
||||
[queues.md](queues.md)) (open)
|
||||
- **OQ-05**: queue semantics depth on honker's machinery (open)
|
||||
@@ -0,0 +1,217 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-04
|
||||
---
|
||||
|
||||
# alkstore — Open Questions
|
||||
|
||||
Centralized tracker. IDs `OQ-NN` are stable — never renumber; append.
|
||||
The Phase 0 register's questions (`OQ-ST-01..08` in
|
||||
`docs/research/phase-0.md`) are promoted here faithfully: **OQ-01..08
|
||||
mirror OQ-ST-01..08 one-to-one**, keeping their Phase 0 statuses
|
||||
(a resolved register question stays listed here as resolved, with the
|
||||
ADR that carries its decision). New Phase 1 questions append from
|
||||
OQ-09. Suggested resolution order: **OQ-04 first** (the contract
|
||||
surface everything else hangs off), then OQ-09/OQ-10 (scoped to
|
||||
OQ-04's outcome), then OQ-05/OQ-08 (OQ-05 has an independent design
|
||||
track; OQ-08 depends only on the trait's shape, OQ-04).
|
||||
|
||||
Resolved questions stay listed with their resolution; they are not
|
||||
deleted.
|
||||
|
||||
## Theme: Scope
|
||||
|
||||
### OQ-01: Scope boundary — which honker features are in-scope? *(== OQ-ST-01)*
|
||||
|
||||
- **Origin**: [consumer-inventory.md](../research/consumer-inventory.md),
|
||||
[ADR-002](decisions/002-feature-scope.md)
|
||||
- **Status**: resolved (2026-10-04, Phase 0)
|
||||
- **Priority**: medium
|
||||
- **Resolution**: Answered by the consumer inventory — scope votes
|
||||
shrink to named rows, not the whole honker feature list. First-class:
|
||||
notify/listen (pinned), streams (operator-authority record). In
|
||||
scope: named locks, queues + outbox (documented), scheduler
|
||||
(documented-thin). Cut-flags: rate limits, result storage. Out:
|
||||
honker's exclusion lines. Decision recorded in
|
||||
[ADR-002](decisions/002-feature-scope.md).
|
||||
- **Cross-references**: OQ-07, OQ-04.
|
||||
|
||||
### OQ-02: Crate scope — one store crate, or reactive-core + engines? *(== OQ-ST-02)*
|
||||
|
||||
- **Origin**: [overview.md](overview.md)
|
||||
- **Status**: resolved (2026-10-04, operator decision; Phase 0)
|
||||
- **Priority**: medium
|
||||
- **Resolution**: Reactive-core crate + per-engine engine crates; the
|
||||
split isolates the engines' asymmetry of work and keeps engine
|
||||
binaries single-driver (which the libsqlite3-sys collision
|
||||
effectively requires). Decision recorded in
|
||||
[ADR-001](decisions/001-crate-split.md) — including the reasoning
|
||||
that superseded the inventory's single-crate lean.
|
||||
- **Cross-references**: OQ-03, OQ-10.
|
||||
|
||||
### OQ-03: Driver story — sqlx, tokio-postgres, or per-engine drivers? *(== OQ-ST-03)*
|
||||
|
||||
- **Origin**: [overview.md](overview.md)
|
||||
- **Status**: resolved (2026-10-04, POC-backed both engines; Phase 0)
|
||||
- **Priority**: medium
|
||||
- **Resolution**: Per-engine drivers — rusqlite + honker-core 0.5.0
|
||||
(SQLite; bridged seam), tokio-postgres 0.7.18 + deadpool-postgres
|
||||
0.14.2 (Postgres), under OQ-02's crate split. Decisions recorded in
|
||||
[ADR-003](decisions/003-sqlite-driver.md) and
|
||||
[ADR-004](decisions/004-postgres-driver.md); measurements in
|
||||
`docs/research/poc-sqlite-posture-findings.md` and
|
||||
`docs/research/poc-pg-posture-findings.md`.
|
||||
- **Cross-references**: OQ-02, OQ-05, OQ-06.
|
||||
|
||||
### OQ-07: SQLite-side scope — loadable extension? *(== OQ-ST-07)*
|
||||
|
||||
- **Origin**: [ADR-002](decisions/002-feature-scope.md)
|
||||
- **Status**: resolved (cut-only, 2026-10-04; Phase 0)
|
||||
- **Priority**: low
|
||||
- **Resolution**: The loadable-extension surface is out: every
|
||||
identified consumer is in-process Rust attaching to its own
|
||||
connection (the honker-core `attach_honker_functions` shape). Folded
|
||||
into OQ-01's scope resolution and [ADR-002](decisions/002-feature-scope.md)
|
||||
(explicitly, so the slot's disposition is visible — the Phase 0
|
||||
register carried it as its own entry).
|
||||
- **Cross-references**: OQ-01.
|
||||
|
||||
## Theme: Core contract
|
||||
|
||||
### OQ-04: Contract pinning — the exact trait surface and its semantics *(== OQ-ST-04)*
|
||||
|
||||
*(Retitled in promotion: OQ-ST-04's framing — "the reactive
|
||||
abstraction — what does the unified notify surface look like?" —
|
||||
narrowed to the pinning work its own record already scoped.)*
|
||||
|
||||
- **Origin**: [core-contract.md](core-contract.md),
|
||||
[ADR-006](decisions/006-wake-and-delivery-contract.md),
|
||||
[ADR-007](decisions/007-transactional-seam.md)
|
||||
- **Status**: open (de-risked; both engine sides POC-verified — this is
|
||||
paper work over a complete evidence base)
|
||||
- **Priority**: high
|
||||
- **Resolution**: open. The decisions that bound it are made
|
||||
([ADR-006](decisions/006-wake-and-delivery-contract.md): opaque-wake + re-read, the delivery-guarantee table, the
|
||||
reserved namespace's existence; [ADR-007](decisions/007-transactional-seam.md): caller-held `TxHandle`
|
||||
with `*_tx` methods). What remains to pin, per the honker-rs surface
|
||||
as starting artifact against the inventory rows:
|
||||
- Which surface parts are contract v1 and which are engine-extension
|
||||
(queue depth, scheduler surface, lock semantics details).
|
||||
- `TxHandle` representation: `dyn` + per-engine downcast (POC
|
||||
sketch, works, two recorded frictions) vs generic/enum handle.
|
||||
- The reserved/meta namespace: exact strings (Postgres
|
||||
reconnect-wake channel like `__listener_reconnected__`), and
|
||||
whether SQLite-side internal names carry an equivalent prefix.
|
||||
- Error taxonomy: what callers can match on per mechanism (e.g.
|
||||
`PayloadTooLarge` is POC-verified pg-side; SQLite's notify has no
|
||||
8000-byte limit — is the error universal?).
|
||||
- Naming/grouping of the honker-rs surface where renames clarify
|
||||
(e.g. what `listen()` returns — a wake receiver? a subscription
|
||||
handle?).
|
||||
- The `Store::open` config shape — what engine-configuration
|
||||
(durability knobs, pool sizing, listener posture) is consumer
|
||||
surface vs engine crate docs.
|
||||
- Verification backlog for properties the POCs pinned on one engine
|
||||
only — e.g. lock TTL/expiry re-acquisition is POC-pinned on
|
||||
Postgres but rests on honker's machinery on SQLite; the contract
|
||||
test suite must pin the SQLite side too.
|
||||
- Delivery-guarantee rows for locks and scheduler (the guarantee
|
||||
table in [ADR-006](decisions/006-wake-and-delivery-contract.md) covers notify/streams/queues; lock-vs-TTL-race
|
||||
and scheduler-tick guarantees need either rows there or an
|
||||
explicit "scheduler is queues" absorption via OQ-09).
|
||||
- **Cross-references**: OQ-01, OQ-10, OQ-09, OQ-08.
|
||||
|
||||
## Theme: Queues and scheduling
|
||||
|
||||
### OQ-05: Queue semantics depth — retry/backoff/dead-letter/sweep design *(== OQ-ST-05)*
|
||||
|
||||
- **Origin**: [queues.md](queues.md)
|
||||
- **Status**: open (posture resolved: re-derived on both engines with
|
||||
the pg-boss schema family as design reference and honker's queue
|
||||
design as the SQLite-side one — ADR-004/ADR-005)
|
||||
- **Priority**: high
|
||||
- **Resolution**: open. The transactional and claim properties (the
|
||||
hard driver-coupled part) are POC-pinned on both engines. Remaining:
|
||||
the semantics-depth design — retry policy shape (attempts, backoff
|
||||
curve), visibility-timeout/renewal mechanics, dead-letter
|
||||
move-vs-flag and retention, sweep/maintenance design
|
||||
(`sweep_expired` cadence and owner), and whether result-storage's
|
||||
cut-flag gets reconsidered as part of this surface (it is
|
||||
queue-adjacent tooling).
|
||||
- **Cross-references**: OQ-09, OQ-06.
|
||||
|
||||
### OQ-09: Is the scheduler a first-class mechanism, or queues + `schedule()`?
|
||||
|
||||
- **Origin**: [queues.md](queues.md) (spin-out of OQ-05's Phase 0
|
||||
framing, where the scheduler's boundary question lived)
|
||||
- **Status**: open
|
||||
- **Priority**: medium
|
||||
- **Resolution**: open. The inventory found the scheduler need real but
|
||||
thin (the family-wide "who sweeps/renews/reaps" problem), and
|
||||
mechanically scheduler = cron/`@every` enqueueing into named queues
|
||||
+ leader election + a tick. If design confirms the collapse, the
|
||||
contract surface is queues + `store.schedule(cron, queue)` with the
|
||||
scheduler mechanism absorbed; if consumers need
|
||||
inspectable/pausable schedule objects (`add/pause/resume/update/
|
||||
list/remove`), it stays a first-class mechanism (the full honker
|
||||
shape). Decided against the inventory rows, not the whole honker
|
||||
menu.
|
||||
- **Cross-references**: OQ-05, OQ-04.
|
||||
|
||||
## Theme: Engines and dependencies
|
||||
|
||||
### OQ-06: honker-core quality read — does the default posture hold? *(== OQ-ST-06)*
|
||||
|
||||
- **Origin**: [ADR-005](decisions/005-dependency-ownership.md),
|
||||
[engine-sqlite.md](engine-sqlite.md)
|
||||
- **Status**: open
|
||||
- **Priority**: high (fork trigger is a gate on the SQLite engine's
|
||||
dependency posture)
|
||||
- **Resolution**: open. ADR-005 fixed the calculus and the trigger: the
|
||||
Phase 1 quality read of honker-core 0.5's watcher/transactional core
|
||||
(Writer/Readers/SharedUpdateWatcher/attach_*) — looking for defects
|
||||
the POCs wouldn't surface, unsafe assumptions in the watcher
|
||||
failure-handling, and schema-migration brittleness. Outcomes: posture
|
||||
holds (no ADR change), or a fork/patch need is named (fork is normal
|
||||
work per ADR-005).
|
||||
- **Cross-references**: ADR-003, ADR-005, OQ-05 (if the read forces a
|
||||
fork, queue-on-honker-machinery work changes shape).
|
||||
|
||||
## Theme: Deployment and capabilities
|
||||
|
||||
### OQ-08: Where does the honest single-host/multi-host boundary live in the trait surface? *(== OQ-ST-08)*
|
||||
|
||||
- **Origin**: [deployment.md](deployment.md)
|
||||
- **Status**: open
|
||||
- **Priority**: medium
|
||||
- **Resolution**: open. The pg engine is natively multi-host (POC #2
|
||||
verified — no single-host assumption to remove); SQLite is
|
||||
single-machine by nature (file-backed, NFS-two-writers unsupported —
|
||||
honker's honesty posture, inherited). The unified surface must not
|
||||
pretend SQLite is multi-host. Options: per-engine capability flags
|
||||
(`Store::capabilities()`), a documented deployment matrix only
|
||||
([deployment.md] carries the facts), or compile-time knowledge only
|
||||
(a consumer choosing the SQLite engine knows). Rides OQ-04: the
|
||||
trait's shape constrains where capability differences can surface.
|
||||
- **Cross-references**: OQ-04, [ADR-006](decisions/006-wake-and-delivery-contract.md).
|
||||
|
||||
### OQ-10: How do engine crates track core-contract version changes?
|
||||
|
||||
- **Origin**: [overview.md](overview.md),
|
||||
[ADR-001](decisions/001-crate-split.md)
|
||||
- **Status**: open
|
||||
- **Priority**: medium
|
||||
- **Resolution**: open. The core crate's trait surface is a contract
|
||||
the engine crates must track ([ADR-001](decisions/001-crate-split.md) negative consequence). What
|
||||
is the versioning/sync discipline — semver-bump-only-when-
|
||||
contract-changes, engines pin core ranges, a contract-compatibility
|
||||
test suite the engines run against the core's trait definitions?
|
||||
What happens to a released engine crate when core makes a contract
|
||||
breaking change?
|
||||
- **Cross-references**: OQ-04 (the contract being versioned), OQ-02.
|
||||
|
||||
## Deferred / Blocked
|
||||
|
||||
None currently. Every open OQ above is actionable Phase 1 architecture
|
||||
work (contract pinning, design, quality read) with its evidence base
|
||||
complete — no external arrivals are being waited on.
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-04
|
||||
---
|
||||
|
||||
# alkstore — Overview
|
||||
|
||||
One reactive store interface over SQLite and Postgres: durable
|
||||
notify/subscribe signals, streams with per-consumer offsets, durable
|
||||
queues with the transactional enqueue property, named locks, a
|
||||
scheduler, and an outbox helper — with each engine using its own
|
||||
native machinery underneath (SQLite: honker's watcher design;
|
||||
Postgres: `pg_notify`/LISTEN and the pg-boss schema family).
|
||||
|
||||
## Why this crate exists
|
||||
|
||||
The alk* ecosystem's current shape is a repository pattern with a
|
||||
default in-memory adapter, cache-invalidation wiring in hot paths, and
|
||||
per-project approximations where a durable reactive substrate was
|
||||
needed. alkblobs paused partly on that fuzziness. alkstore is the
|
||||
substrate, made once: the answer to "how does a change in the database
|
||||
become visible to other processes/connections?" that downstream stores
|
||||
don't re-derive.
|
||||
|
||||
## Crate family
|
||||
|
||||
Per [ADR-001](decisions/001-crate-split.md):
|
||||
|
||||
| Crate | Contents | Driver dependencies |
|
||||
|---|---|---|
|
||||
| `alkstore` (core) | trait surface, types, error model, capability flags | none |
|
||||
| `alkstore-sqlite` | SQLite engine ([ADR-003](decisions/003-sqlite-driver.md)) | rusqlite, honker-core |
|
||||
| `alkstore-postgres` | Postgres engine ([ADR-004](decisions/004-postgres-driver.md)) | tokio-postgres, deadpool-postgres |
|
||||
| (mem engine, optional) | test convenience, decided at implementation ([ADR-001](decisions/001-crate-split.md)) | none |
|
||||
|
||||
Downstream consumers depend on core + exactly one engine. Family
|
||||
standards apply throughout: tokio async runtime, `thiserror` errors, no
|
||||
panics in library code, no `unwrap()`/`expect()` outside tests,
|
||||
lean base crate.
|
||||
|
||||
## Feature surface
|
||||
|
||||
Per [ADR-002](decisions/002-feature-scope.md):
|
||||
|
||||
- **First-class**: notify/listen; streams.
|
||||
- **In scope**: named locks; queues + outbox helper; scheduler.
|
||||
- **Cut-flag**: rate limits; result storage.
|
||||
- **Out**: loadable-extension surface; DAGs/task
|
||||
chains/chords/multi-writer replication/cross-machine locking.
|
||||
|
||||
## Document map
|
||||
|
||||
| Doc | Purpose |
|
||||
|---|---|
|
||||
| [core-contract.md](core-contract.md) | The unified trait surface: mechanisms, delivery guarantees, tx seam, naming |
|
||||
| [engine-sqlite.md](engine-sqlite.md) | SQLite engine: mapping the contract onto honker-core/rusqlite |
|
||||
| [engine-postgres.md](engine-postgres.md) | Postgres engine: mapping the contract onto tokio-postgres/LISTEN |
|
||||
| [queues.md](queues.md) | Queue/scheduler/outbox semantics depth (OQ-05/OQ-09) |
|
||||
| [deployment.md](deployment.md) | Host capabilities, connection budgets, deployment matrix (OQ-08) |
|
||||
| [open-questions.md](open-questions.md) | OQ-01..NN tracker |
|
||||
| [decisions/](decisions/) | ADRs |
|
||||
|
||||
## What is decided (ADR index)
|
||||
|
||||
| ADR | Decision | Status |
|
||||
|---|---|---|
|
||||
| [001](decisions/001-crate-split.md) | Reactive-core + per-engine crates | Accepted |
|
||||
| [002](decisions/002-feature-scope.md) | Feature scope (inventory-confirmed) | Accepted |
|
||||
| [003](decisions/003-sqlite-driver.md) | SQLite: rusqlite + honker-core, bridged seam | Accepted |
|
||||
| [004](decisions/004-postgres-driver.md) | Postgres: tokio-postgres + deadpool, hand-rolled LISTEN | Accepted |
|
||||
| [005](decisions/005-dependency-ownership.md) | Published libraries by default, named fork triggers | Accepted |
|
||||
| [006](decisions/006-wake-and-delivery-contract.md) | Opaque wake + re-read; notify-vs-streams guarantee split | Accepted |
|
||||
| [007](decisions/007-transactional-seam.md) | Caller-held `TxHandle` with `*_tx` methods | Accepted |
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Not a database abstraction/ORM: the store covers the reactive
|
||||
coordination surface, not general row storage (consumers keep their
|
||||
own engines/schema for business data, sharing the connection when
|
||||
appropriate — the outbox pattern).
|
||||
- Not a network service: no transport; any networked ops surface rides
|
||||
alkcall and is a separate future decision (store layer stays
|
||||
substrate-free, the alkblobs store-layer precedent).
|
||||
- Not multi-machine on SQLite: single-host honesty is inherited
|
||||
([deployment.md](deployment.md)).
|
||||
|
||||
## Evidence base
|
||||
|
||||
Phase 0 (`docs/research/phase-0.md`) is complete: two POCs
|
||||
(`poc-sqlite-posture-findings.md`, `poc-pg-posture-findings.md`) passed
|
||||
all gate conditions; the consumer inventory
|
||||
(`consumer-inventory.md`) graded every feature row. The Phase 1
|
||||
architecture work runs over that complete evidence base; open Phase 1
|
||||
work is tracked in [open-questions.md](open-questions.md).
|
||||
@@ -0,0 +1,138 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-04
|
||||
---
|
||||
|
||||
# Queues, scheduler, outbox — semantics depth
|
||||
|
||||
The queue family's *posture* is decided (re-derived on both engines;
|
||||
[ADR-004](decisions/004-postgres-driver.md) and
|
||||
[ADR-005](decisions/005-dependency-ownership.md)) and the hard
|
||||
driver-coupled properties are POC-pinned (transactional enqueue,
|
||||
exactly-once claim under concurrency). What is *not* yet designed is
|
||||
the semantics depth — this document's subject, and [OQ-05]'s home. It
|
||||
states the design space, the decided constraints any answer must fit,
|
||||
and the reference material; actual semantics decisions become ADRs
|
||||
when made.
|
||||
|
||||
## Decided constraints (inherited, not re-opened)
|
||||
|
||||
- **Mechanisms**: durable at-least-once queues; the outbox is a helper
|
||||
over queues; scheduler is cron/`@every` enqueueing into named
|
||||
queues ([ADR-002](decisions/002-feature-scope.md)) — pending
|
||||
OQ-09's collapse decision.
|
||||
- **Transactional enqueue**: commit-atomic with the caller's business
|
||||
write on both engines ([ADR-007](decisions/007-transactional-seam.md));
|
||||
rollback drops the job row with no ghosts.
|
||||
- **Exactly-once claim**: `FOR UPDATE SKIP LOCKED` (pg, POC-pinned
|
||||
under 4×4 concurrency) / honker's machinery (SQLite). At-least-once
|
||||
*work*; exactly-once *processing* needs the ack + visibility model.
|
||||
- **Wake-driven consumption**: the pg engine's default is LISTEN-driven
|
||||
claim with a re-poll safety net; SQLite's is watcher wake with
|
||||
per-subscription fanout ([ADR-006](decisions/006-wake-and-delivery-contract.md)).
|
||||
- **Job options** (the honker-rs surface, the contract starting
|
||||
artifact): `delay, priority, max_attempts, expires` at enqueue;
|
||||
`ack / retry / fail / heartbeat` on the job handle.
|
||||
- **Cut-flag context**: result storage is
|
||||
[ADR-002](decisions/002-feature-scope.md)'s cut-flag row; if
|
||||
OQ-05's design shows queue consumers provably need
|
||||
result-query-by-id, that is the channel to revisit it — with a
|
||||
consumer-inventory row, not silent inclusion.
|
||||
|
||||
## Design space (what OQ-05 must pin)
|
||||
|
||||
### Retry / backoff
|
||||
|
||||
- Trigger model: attempts counted from claim-completion; a job
|
||||
re-appears after visibility timeout OR explicit `retry`. Does
|
||||
`max_attempts` exhaustion move the job to dead-letter, flag it, or
|
||||
drop it?
|
||||
- Backoff curve between attempts: honker and the pg-boss family have
|
||||
their own shapes (fixed, exponential); which does the contract
|
||||
pin, what is configurable, what's the default?
|
||||
- `heartbeat` semantics: renewal vs progress signal (or both) — and
|
||||
what a missed heartbeat means (visibility re-expiry? nothing?).
|
||||
|
||||
### Visibility timeouts
|
||||
|
||||
- Where the renewal lives: explicit (`heartbeat`) only, or implicit
|
||||
renewal per op on the job handle? Honker's design is the SQLite-side
|
||||
answer; pg-boss's is the reference. One answer, contract-pinned.
|
||||
- Interaction with long business transactions (claims inside
|
||||
caller txs, [ADR-007](decisions/007-transactional-seam.md)).
|
||||
|
||||
### Dead-letter
|
||||
|
||||
- Move-to-table (honker's `_honker_dead`, pg-boss's dead-letter
|
||||
queue) vs flag-in-place. Inspection/requeue surface shape.
|
||||
- Retention: does a dead-lettered job expire (an `expires`-driven
|
||||
sweep — an OQ-05 sub-question) or live until explicitly cleared?
|
||||
|
||||
### Sweep / maintenance
|
||||
|
||||
- `sweep_expired` is on the queue's surface; cadence ownership is
|
||||
the open question. Honker's scheduler machinery (leader-elected,
|
||||
missed-boundary catch-up) can run it; a consumer's own scheduler row
|
||||
can too; the engine could ship a default. Decision shaped by OQ-09's
|
||||
scheduler collapse — same machinery either way.
|
||||
- Multi-process sweep safety on SQLite vs Postgres: named-lock
|
||||
coordination (the alkblobs fleet-sweeper pattern,
|
||||
[ADR-002](decisions/002-feature-scope.md)) vs native advisory locks.
|
||||
|
||||
### Scheduler surface
|
||||
|
||||
- OQ-09's question, stated crisply: is scheduler a first-class
|
||||
mechanism (own handle, methods, inspection) or queues + a
|
||||
`schedule(cron_expression, queue_name, payload)` call? The
|
||||
inventory's evidence is thin (documented-thin); a v1 that is queues
|
||||
+ a `schedule` call with leader-election behavior documented is the
|
||||
smaller contract; if a consumer needs inspectable/pausable schedule
|
||||
*objects* (`add/pause/resume/update/list/remove`), that's the
|
||||
full honker shape. Decide against the inventory rows, not against
|
||||
the whole honker menu.
|
||||
|
||||
### Namespaces / schema layout (pg side)
|
||||
|
||||
- Schema-scoped DDL (the pg-boss design) vs shared-schema table
|
||||
naming. Rides the reserved-namespace contract
|
||||
([ADR-006](decisions/006-wake-and-delivery-contract.md), OQ-04):
|
||||
queue/stream/lock tables and the reserved channel prefix
|
||||
must be collision-proof against consumer tables in the same
|
||||
database (the alkblobs co-tenant-tables precedent — ADR-008 in
|
||||
`/workspace/@alkdev/alkblobs/docs/architecture/decisions/`).
|
||||
|
||||
## Reference material
|
||||
|
||||
- **honker's queue design** — the SQLite-side incumbent
|
||||
(`/workspace/honker`, its honker-core machinery; the engine rides it
|
||||
directly, so the SQLite side's depth is largely "inherit + pin").
|
||||
- **pg-boss family** — `/workspace/pgboss-rs` @ 98f7d9e (queue states,
|
||||
maintenance/dead-letter behavior) and the node original (pg-boss, the
|
||||
upstream of record — compare semantics the port may have dropped).
|
||||
- **POC ground** — `poc-pg-posture-findings.md` (the minimal queue
|
||||
table + SKIP LOCKED claim measured; the property suite green);
|
||||
`poc-sqlite-posture-findings.md` (the SQLite twin).
|
||||
- **Consumers** — alkfs OQ-FS-14 (sync outbox), alkblobs
|
||||
ops-surface/gc docs (maintenance cadences), the family-wide
|
||||
"who sweeps" punt (inventory §scheduler).
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|---|---|---|
|
||||
| [002](decisions/002-feature-scope.md) | Feature scope | queues + outbox in; result storage cut-flag |
|
||||
| [004](decisions/004-postgres-driver.md) | pg driver | queue machinery re-derived, pg-boss as reference |
|
||||
| [005](decisions/005-dependency-ownership.md) | Ownership | design-reference postures for the queue family |
|
||||
| [006](decisions/006-wake-and-delivery-contract.md) | Wake contract | wake-driven consumption, guarantee table |
|
||||
| [007](decisions/007-transactional-seam.md) | Tx seam | enqueue_tx commit-atomicity |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in
|
||||
[open-questions.md](open-questions.md). Key
|
||||
questions affecting this document:
|
||||
|
||||
- **OQ-05**: the semantics-depth pinning this document frames (open,
|
||||
high priority)
|
||||
- **OQ-09**: scheduler as first-class mechanism vs queues + schedule
|
||||
(open)
|
||||
Reference in new issue
Block a user