alkdev/alkstore
6
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ADR-015: streams depth — carried-metadata keys, global-FIFO ordering, StreamEvent, trim_to (OQ-12 resolved)

Browse Source
This commit is contained in:
glm-5.3-flash committed 2026-10-05 14:15:28 +00:00
1 parent 04a04651dc
commit 8c4ec48f92
12 files changed
+562 -110
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/architecture/core-contract.md
+83 -18
View File
@@ -15,7 +15,10 @@ inventory-confirmed features — [ADR-002](decisions/002-feature-scope.md)
`TxHandle` representation, wake type, reserved strings, error
taxonomy, config split — amended in place, pre-implementation, by
[ADR-014](decisions/014-outbox-tx-enqueue.md): `outbox_enqueue_tx`
joins the `TxHandle` trait); ADR-009/ADR-010 add the first *post-v1
joins the `TxHandle` trait, and by
[ADR-015](decisions/015-streams-depth.md): streams depth — key
semantics, `StreamEvent` shape, the ordering row, `trim_to`,
and `publish_with_key_tx`); ADR-009/ADR-010 add the first *post-v1
contract extensions* (scheduler collapse surface, `QueueOpts` depth —
versioning discipline for such extensions is OQ-10's); the
*obligations* are this document.
@@ -56,7 +59,8 @@ Store::begin_tx() -> Box<dyn TxHandle + Send>
trait TxHandle {
enqueue_tx(name, opts, payload) -> job_id
publish_tx(stream, payload) -> event_id
publish_tx(stream, payload) -> offset
publish_with_key_tx(stream, key, payload) -> offset
notify_tx(channel, payload)
save_offset_tx(stream, consumer, offset)
outbox_enqueue_tx(outbox, opts, payload) -> job_id
@@ -68,13 +72,21 @@ trait TxHandle {
queue engine-side (the reserved prefix makes the derived name
unreachable by `enqueue_tx` by design —
[ADR-014](decisions/014-outbox-tx-enqueue.md)).
`publish_with_key_tx` validates the key like the non-tx form (a
present key must be non-empty → `InvalidName`) and is
`publish_tx`'s keyed twin — commit-atomic keyed publishes are the
point ([ADR-015](decisions/015-streams-depth.md) §2).
Mechanism handles come off the store (or, for transactional variants,
off the handle):
```text
store.notify(channel, payload) handle.notify_tx(channel, payload)
store.stream(name) -> Stream handle.publish_tx / save_offset_tx
store.stream(name) -> Stream handle.publish_tx / publish_with_key_tx
handle.save_offset_tx
(the Stream handle also carries
trim_to — stream-side, outside the
tx seam)
store.queue(name, opts) -> Queue handle.enqueue_tx
store.outbox(name) -> Outbox handle.outbox_enqueue_tx(outbox, opts, payload)
store.try_lock(name, owner, ttl) -> Option<Lock>
@@ -138,21 +150,56 @@ cousin of notify: publish is commit-atomic; every committed event is
readable by every consumer whose offset hasn't passed it;
replay-on-attach is the default; offsets are explicit and
transaction-aware (`save_offset_tx` gives exactly-once-within-a-
business-tx shape).
business-tx shape). Depth pinned by
[ADR-015](decisions/015-streams-depth.md).
- `publish` / `publish_tx` / `publish_with_key` — append to the
stream's durable log. *(Depth annotation: the key's semantics, the
`StreamEvent` shape, the ordering row, and log retention are
OQ-12's — method names pinned by
[ADR-008](decisions/008-contract-v1-pinning.md) §1, depth deferred
there; see [open-questions.md](open-questions.md).)*
- `publish` / `publish_tx` / `publish_with_key` / `publish_with_key_tx`
— append to the stream's durable log; return the assigned offset.
**Key semantics ([ADR-015](decisions/015-streams-depth.md) §1): the
key is carried metadata** — stored on the event row, round-tripped
on every read, `Option<String>`-shaped on `StreamEvent` — with no
engine-enforced behavioral role: **the ordering guarantee stays
global FIFO by offset within the stream**; per-key in-order reading
is the documented emergent pattern (filter by key, read in offset
order), not a server promise. A present key must be non-empty
(`InvalidName`). Server-enforced per-key ordering re-enters only
via a consumer-inventory row naming it.
- **`StreamEvent` shape** ([ADR-015](decisions/015-streams-depth.md)
§3): `{ offset: i64, stream: String, key: Option<String>, payload:
Vec<u8>, created_at: i64 }` — `stream` renames honker's `topic`
(one term everywhere, the ADR-008 §4 kinds list); `offset` is
engine-assigned, monotone per stream, **immutable, never renumbered**
(gaps after a trim are legal); `created_at` is unix-seconds-at-
publish, informational, **not an ordering field** (offset is the
only one); `payload_as<T>` decodes (the error is `Codec`).
- **Ordering guarantee row** ([ADR-015](decisions/015-streams-depth.md)
§4, extending [ADR-006](decisions/006-wake-and-delivery-contract.md)
§2's streams row): `read_since`/`subscribe` yield `offset ASC` —
global FIFO per stream; committed-event visibility and
replay-on-attach as already pinned; offsets immutable. The
contract suite pins cross-engine equivalence against this row
(same publish sequence → same offset sequence → same read order).
- `read_since` / `read_from_consumer(offset)` — cursor-based reads;
`get_offset(consumer)` — checkpoint inspection.
`get_offset(consumer)` — checkpoint inspection. A read from a
trimmed-away offset region (below) resumes at the trim horizon's
first remaining row.
- `save_offset` / `save_offset_tx` — consumer checkpoint, explicit;
**the contract's save is always explicit** (no auto-checkpoint
cadence — honker's per-binding ambiguity is not inherited). The
subscription handle exposes no `save_every` / auto-save-on-drop
([ADR-008](decisions/008-contract-v1-pinning.md) §8).
([ADR-008](decisions/008-contract-v1-pinning.md) §8). Saves are
monotone; a saved offset below the trim horizon
([ADR-015](decisions/015-streams-depth.md) §5) stays a valid
position marker.
- `trim_to(horizon)` — delete events with `offset <= horizon`
([ADR-015](decisions/015-streams-depth.md) §5). The stream-side
bounded-growth op, consumer-invoked, no engine-default
retention and no ambient sweeper (the ADR-010 §6 posture; the
replay-forever default is the mechanism's purpose — growth is
documented with the tool in-contract to bound it). Trim emits no
dedicated wake and no notify — on SQLite the `data_version`
watcher may still fire (any committed write bumps it; a spurious
hint, contract-legal); surviving events keep their offsets.
- `subscribe(consumer) -> Box<dyn EventReceiver>` — durable
consumption: attach, read to current tail, resume after restart
from the stored offset; explicit `save_offset` on the receiver
@@ -302,8 +349,9 @@ inherits them and adds the consumer obligations:
consumer's, correctness-of-transition the engine's
([ADR-010](decisions/010-queue-semantics-depth.md) §6);
- `*_tx` operations are *only* durable after the caller's commit —
rollback drops job rows, event rows, notifications, and offset saves
together (the no-ghosts property, POC-pinned on both engines).
rollback drops job rows, event rows (keyed and unkeyed alike),
notifications, and offset saves together (the no-ghosts property,
POC-pinned on both engines).
### Errors
@@ -412,6 +460,23 @@ before the engine specs are called `stable`:
outbox names rejected identically on the tx path
(`ReservedName`/`InvalidName`); stamped opts visible via `get_job`
equal for both engines.
- **Cross-engine stream equivalence** ([ADR-015](decisions/015-streams-depth.md)
§3/§4) — publish sequence → identical offset sequence → identical
`read_since` output order on both engines, direct and subscriber
reads alike; keyed/unkeyed interleavings preserve global FIFO;
`key` round-trips exactly (`None` vs `Some`), `stream`/`created_at`
fields equal.
- **`publish_with_key_tx` commit-atomicity on both engines**
([ADR-015](decisions/015-streams-depth.md) §2) — rollback drops the
keyed event row with the business write (no ghost event); commit
makes it visible to `read_since`/`subscribe`; empty-`Some`-key
`InvalidName` on both engines' tx paths.
- **`trim_to` semantics on both engines** ([ADR-015](decisions/015-streams-depth.md)
§5) — exact-boundary trim (`<=`), surviving rows keep their offsets
(gaps legal, never renumbered), reads resume at the trim horizon's
first remaining row, saved offsets below the horizon stay valid,
trim wakes nothing, and a concurrent subscriber never loses its
place.
## Design Decisions
@@ -427,6 +492,7 @@ before the engine specs are called `stable`:
| [011](decisions/011-sqlite-substrate-fork.md) | Substrate fork | SQLite substrate owned (`__alkstore_*` naming); queue ops re-derived on contract v1 |
| [012](decisions/012-forked-substrate-design.md) | Fork design | contract-blind substrate boundary; engine-side formula arithmetic pinned equivalent by the contract suite |
| [014](decisions/014-outbox-tx-enqueue.md) | Outbox tx enqueue (amends 008) | `outbox_enqueue_tx` on `TxHandle`; outbox-name validation; derived backing queue reached only through the outbox surface |
| [015](decisions/015-streams-depth.md) | Streams depth (amends 006/008) | key = carried metadata, global-FIFO ordering row, `StreamEvent` shape, `publish_with_key_tx`, `trim_to` |
## Open Questions
@@ -437,11 +503,10 @@ questions affecting this document:
- **OQ-10**: contract versioning discipline across engine crates
([open](open-questions.md))
- **OQ-08**: capability-surface shape ([open](open-questions.md))
- **OQ-12**: streams depth — key semantics, `StreamEvent` shape,
ordering row, retention ([open](open-questions.md))
Resolved on this document's surface: **OQ-09** (scheduler collapse —
[ADR-009](decisions/009-scheduler-collapse.md)), **OQ-05** (queue
semantics depth — [ADR-010](decisions/010-queue-semantics-depth.md)),
and **OQ-13** (transactional outbox enqueue shape —
[ADR-014](decisions/014-outbox-tx-enqueue.md)), 2026-10-05.
**OQ-13** (transactional outbox enqueue shape —
[ADR-014](decisions/014-outbox-tx-enqueue.md)), and **OQ-12** (streams
depth — [ADR-015](decisions/015-streams-depth.md)), 2026-10-05.