ADR-015: streams depth — carried-metadata keys, global-FIFO ordering, StreamEvent, trim_to (OQ-12 resolved)
This commit is contained in:
glm-5.3-flash committed
1 parent
04a04651dc
commit
8c4ec48f92
12 files changed
+562
-110
No files matched your search
@@ -23,11 +23,12 @@ is OQ-11); **the fork packaging folded** (2026-10-05,
|
||||
`alkstore-substrate` crate; the forked machinery is a module subtree of
|
||||
`alkstore-sqlite`); **OQ-13 resolved** (2026-10-05,
|
||||
[ADR-014](decisions/014-outbox-tx-enqueue.md) —
|
||||
`outbox_enqueue_tx` joins the `TxHandle` trait, the hole is closed).
|
||||
Next: **OQ-12** (streams depth — a Phase 1 review
|
||||
find: key semantics, `StreamEvent` shape, ordering row, retention;
|
||||
evidence gathered; check consumer rows for per-key-ordering need
|
||||
before choosing among its options), OQ-08 (rides the now-pinned trait
|
||||
`outbox_enqueue_tx` joins the `TxHandle` trait, the hole is closed);
|
||||
**OQ-12 resolved** (2026-10-05,
|
||||
[ADR-015](decisions/015-streams-depth.md) — streams depth pinned:
|
||||
carried-metadata keys, global-FIFO ordering row, `StreamEvent` shape,
|
||||
`publish_with_key_tx`, `trim_to`).
|
||||
Next: **OQ-08** (rides the now-pinned trait
|
||||
shape and the ADR-011 substrate fork), OQ-10 (versioning discipline
|
||||
for contract extensions; note ADR-011 changes its substrate-side facts
|
||||
for SQLite — the forked machinery lives in-tree inside the engine
|
||||
@@ -131,7 +132,8 @@ narrowed to the pinning work its own record already scoped.)*
|
||||
resolution. A verification backlog (core-contract.md §Verification
|
||||
backlog) tracks the one-engine-pinned properties.
|
||||
- **Cross-references**: OQ-01, OQ-10, OQ-09, OQ-08; OQ-13 (the one
|
||||
gap the v1 pinning's own Phase 1 review found in it).
|
||||
gap the v1 pinning's own Phase 1 review found in it), OQ-12 (the
|
||||
sibling depth gap — resolved, [ADR-015](decisions/015-streams-depth.md)).
|
||||
|
||||
## Theme: Queues and scheduling
|
||||
|
||||
@@ -308,59 +310,69 @@ narrowed to the pinning work its own record already scoped.)*
|
||||
|
||||
## Theme: Core contract (Phase 1 review finds)
|
||||
|
||||
### OQ-12: Streams depth — key semantics, `StreamEvent` shape, ordering row, retention
|
||||
### OQ-12: Streams depth — key semantics, `StreamEvent` shape, ordering row, retention — **RESOLVED**
|
||||
|
||||
- **Origin**: [core-contract.md](core-contract.md) streams section,
|
||||
[ADR-008](decisions/008-contract-v1-pinning.md) §1 (Phase 1
|
||||
architecture review, 2026-10-05)
|
||||
- **Status**: open
|
||||
- **Status**: resolved (2026-10-05, Phase 1 —
|
||||
[ADR-015](decisions/015-streams-depth.md))
|
||||
- **Priority**: high (streams is a first-class
|
||||
[ADR-002](decisions/002-feature-scope.md) mechanism; ADR-008's
|
||||
"no placeholder semantics remain" positive consequence is false for
|
||||
`publish_with_key` until this resolves — the contract suite also
|
||||
cannot pin cross-engine stream equivalence without an ordering row)
|
||||
- **Resolution**: open. The method *names* are pinned (ADR-008 §1);
|
||||
the depth is not. Sub-questions, with the evidence already gathered:
|
||||
- **Key semantics** — honker's realization is a stored `key` column
|
||||
(`_honker_stream(topic, key, payload)`, NULLable) with no
|
||||
per-key read path or ordering enforcement anywhere; the honker-rs
|
||||
doc comment says "used for per-key ordering downstream"
|
||||
(`packages/honker-rs/src/lib.rs:894`), i.e. per-key global-FIFO
|
||||
is an *emergent* property (a consumer reading `offset ASC`
|
||||
filtered by key), not server-enforced. Options: (a) pin exactly
|
||||
honker's shape — key is carried metadata, ordering stays global
|
||||
FIFO (document the emergent per-key pattern), cheapest and
|
||||
honest; (b) pin *server-enforced* per-key ordering (a read or
|
||||
delivery guarantee keyed on `key`) — real machinery on both
|
||||
engines, needs a consumer row naming the need; (c) cut
|
||||
`publish_with_key` from v1 (no consumer row names keys — the
|
||||
honker-rs doc comment is upstream's intent, not ours). Decision
|
||||
rule: consumer-inventory row first if (b).
|
||||
- **`StreamEvent` shape** — honker's: `{ offset: i64, topic: String,
|
||||
key: Option<String>, payload, created_at: i64 }` with
|
||||
`payload_as<T>` (upstream lib.rs:1008). The contract needs its own
|
||||
pinned field list (stream name vs topic token, key in or out per
|
||||
the key decision, timestamp semantics) — probably inherited
|
||||
near-verbatim.
|
||||
- **Ordering guarantee row** — ADR-006's table has no streams
|
||||
*ordering* cell content beyond "readable"; pin FIFO-by-offset as
|
||||
the read ordering (`ORDER BY offset ASC` is already both
|
||||
engines' claim path) and whether any per-key strengthening rides
|
||||
the key decision.
|
||||
- **Retention / bounded growth** — the reference read flags
|
||||
`_honker_stream` as unbounded-growth unless the user sweeps
|
||||
(honker-machinery §8.5); no consumer row names stream retention
|
||||
(replay-forever is the *purpose* for the named need —
|
||||
subscription surfaces). Options: consumer-side trim API
|
||||
(`trim_to(offset)` — a new contract method), consumer-recipe
|
||||
only (new stream + advance consumers + unschedule/nothing —
|
||||
needs a deletion path that doesn't exist yet, so this option is
|
||||
really "document the gap"), or document nothing (honest but
|
||||
repeats the notifications-table mistake ADR-010 §6 just fixed).
|
||||
Rides the no-ambient-sweeper posture either way.
|
||||
"no placeholder semantics remain" positive consequence was false
|
||||
for `publish_with_key` until this resolved — the contract suite
|
||||
also could not pin cross-engine stream equivalence without an
|
||||
ordering row)
|
||||
- **Resolution**: Pinned by
|
||||
[ADR-015](decisions/015-streams-depth.md). (1) **Key semantics —
|
||||
option (a)**: the key is carried metadata (honker's shape — a
|
||||
stored nullable column, round-tripped on every read, no
|
||||
engine-enforced behavioral role); the ordering guarantee stays
|
||||
**global FIFO by offset per stream**; per-key in-order reading is
|
||||
the documented emergent pattern, not a promise. Option (b)
|
||||
(server-enforced per-key ordering) rejected on its own decision
|
||||
rule — no consumer row names the need; the consumer inventory's
|
||||
streams row and the alkcall ecosystem-shape row (added for this
|
||||
decision — alkcall's Sub/Pub/fan-out vocabulary routes the
|
||||
topic/event-type dimension to *separate streams*, never per-key
|
||||
lanes) both name none. Re-entry gate: a consumer-inventory row
|
||||
naming server-enforced per-key ordering. Option (c) (cut
|
||||
`publish_with_key`) rejected — removing a pinned v1 method name
|
||||
would be a contract-breaking event for OQ-10, the cost is one
|
||||
nullable column the inherited realizations already have, and the
|
||||
key is a legitimately useful grouping token. (2) **A seam gap the
|
||||
key decision exposed, closed in the same ADR**:
|
||||
`publish_with_key_tx(stream, key, payload) -> offset` joins the
|
||||
`TxHandle` trait — keyed publishes were otherwise unreachable
|
||||
commit-atomically, the same hole shape OQ-13 exposed for the
|
||||
outbox (`publish_tx` = the `None`-key call, not a separate engine
|
||||
path; empty-`Some`-key → `InvalidName`, no new variants).
|
||||
(3) **`StreamEvent` shape** — honker's near-verbatim with one
|
||||
rename: `{ offset: i64, stream: String, key: Option<String>,
|
||||
payload: Vec<u8>, created_at: i64 }` (`topic` → `stream` — one
|
||||
term everywhere; the substrate's column name stays per ADR-012 §3
|
||||
fidelity); `offset` immutable, never renumbered (gaps after trim
|
||||
legal); `created_at` unix-seconds-at-publish, informational, not
|
||||
an ordering field. (4) **Ordering guarantee row** — ADR-006 §2's
|
||||
streams row extended: global FIFO by offset (read paths yield
|
||||
`offset ASC`), offsets immutable — the row the contract suite
|
||||
pins cross-engine equivalence against. (5) **Retention** —
|
||||
`trim_to(horizon)` added to the stream handle (delete
|
||||
`offset <= horizon`), consumer-invoked, no engine-default
|
||||
retention, no ambient sweeper (the ADR-010 §6 posture); the
|
||||
replay-forever default is documented with the in-contract tool to
|
||||
bound it; reads from a trimmed-away region resume at the horizon;
|
||||
saved offsets below the horizon stay valid. Framed as completing
|
||||
contract v1 in place, pre-implementation — no versioning event,
|
||||
OQ-10 untouched. Contract-suite rows added (cross-engine stream
|
||||
equivalence, keyed-tx commit-atomicity, `trim_to` semantics —
|
||||
both engines).
|
||||
- **Cross-references**: OQ-04 (the pinning that left this depth
|
||||
open), OQ-01 (scope row), ADR-002, ADR-006 (the guarantee table
|
||||
the ordering row extends), ADR-008 §1.
|
||||
open), OQ-13 (the sibling depth gap; the amend-in-place framing),
|
||||
OQ-01 (scope row), ADR-002, ADR-006 §2 (the guarantee table the
|
||||
ordering row extends), ADR-008 §1/§4/§5/§8, ADR-010 §6 (the
|
||||
no-ambient-sweeper posture the trim decision applies), ADR-012 §3
|
||||
(the fidelity posture the column-name keep rides).
|
||||
|
||||
### OQ-13: Transactional outbox enqueue shape — the `TxHandle` surface cannot express it — **RESOLVED**
|
||||
|
||||
@@ -396,13 +408,14 @@ narrowed to the pinning work its own record already scoped.)*
|
||||
place, pre-implementation — no versioning event, OQ-10 untouched.
|
||||
Contract-suite row added (commit-atomicity on both engines).
|
||||
- **Cross-references**: OQ-04 (the pinning that missed this), OQ-12
|
||||
(the sibling depth gap found by the same review), ADR-002
|
||||
(the sibling depth gap found by the same review — resolved the
|
||||
same day, [ADR-015](decisions/015-streams-depth.md)), ADR-002
|
||||
(outbox scope row), ADR-007/ADR-008 (the seam design), ADR-010 §3
|
||||
(the outbox's derived QueueOpts — unchanged by this).
|
||||
|
||||
## Deferred / Blocked
|
||||
|
||||
None currently. Every open OQ above is actionable Phase 1 work
|
||||
(streams depth, capability-surface shape, versioning discipline,
|
||||
fork-scaffold follow-through) with its evidence base complete — no
|
||||
(capability-surface shape, versioning discipline, fork-scaffold
|
||||
follow-through) with its evidence base complete — no
|
||||
external arrivals are being waited on.
|
||||
Reference in new issue
Block a user