The inventory graded streams absent because no paused consumer document names it; the operator correction: type-filtered event watching from several places (e.g. repo-change subscriptions in a git app at gitea/gitlab scale) is a basic reactivity requirement — and notify (fire-and-forget, no replay) cannot serve subscriptions honestly. The wanters are applications above the paused crates, which is why the docs don't carry the row. Inventory: streams row recorded on operator authority (the REQ-2 recording convention from alkblobs requirements.md), confidence system gains the operator-authority grade; rate-limits becomes the sole first-cut candidate. phase-0: OQ-ST-01 summary and OQ-ST-04's contract-candidates updated to match.
224 lines
15 KiB
Markdown
224 lines
15 KiB
Markdown
---
|
|
status: draft
|
|
last_updated: 2026-10-04 (streams corrected to documented/in-scope —
|
|
operator-authority record: type-filtered event watching from multiple
|
|
places, e.g. repo-change subscriptions in a git app. No consumer doc
|
|
carries the row yet; applications above the paused crates are the
|
|
wanters. rate-limits + result-storage remain the keep-with-flag rows.)
|
|
---
|
|
|
|
# alkstore — consumer-driven scope inventory
|
|
|
|
This document answers OQ-ST-01 (which honker features are in scope) the
|
|
only way that avoids circular hedging: by walking the concrete evidence
|
|
the paused consumers already left on disk. alkgit, alkfs, and alkblobs
|
|
are paused *processes*, but their *documents* are stable, reviewable
|
|
artifacts — requirements they impose on a reactive store, written before
|
|
alkstore existed. That is decisively different from deferring to "what
|
|
the consumers will say later," which was alkstore's original framing for
|
|
OQ-ST-01 and which never collapses: the consumer can't run until the
|
|
store exists and the store was waiting on the consumer — the classic
|
|
low-level/client deadlock, the exact shape the SDD process's hedging
|
|
rules name and the alkblobs open-questions tracker calls
|
|
"Schrödinger's code."
|
|
|
|
Method: for each feature on honker's surface (the interface finding in
|
|
`phase-0.md` §Interface finding), find where a consumer document names a
|
|
need for it, quote the need, and grade the evidence. Confidence grades:
|
|
|
|
- **pinned** — a consumer ADR or requirement doc names it and some
|
|
design leans on it already.
|
|
- **documented** — a consumer doc names the need concretely, but no
|
|
design has leaned on it yet.
|
|
- **recalled** — named in research/summary prose as intended, no
|
|
architecture written against it yet.
|
|
- **operator-authority** — the operator records a need the paused
|
|
documents don't carry yet (the REQ-2 recording convention from
|
|
alkblobs requirements.md), dated; expected where the wanter is an
|
|
*application above* the paused crates. Upgrades equivalent to
|
|
**documented**.
|
|
- **absent** — no consumer document names a need. Included by default
|
|
per the working posture, flagged for cut-at-implementation instead of
|
|
silently carried.
|
|
|
|
The inventory is per-feature need, not per-feature design: how a need is
|
|
served (honker-identical, pg-boss-shaped, or re-derived) is OQ-ST-05/06.
|
|
This document only decides *which features have a consumer*, and hence
|
|
which scope decisions OQ-ST-01 still has to take per feature.
|
|
|
|
## The consumers and their artifacts
|
|
|
|
- **alkfs** — `/workspace/@alkdev/alkfs/docs/research/phase-0.md`
|
|
(Phase 0, 2026-09-23; 17 OQs). The VFS/workspace crate; its ancestor
|
|
POC (`alknet-filesystem-poc`) used honker directly (honker-core 0.2.4
|
|
— notify-in-transaction, named locks identified as needed, queues as
|
|
the sync outbox). Findings:
|
|
`/workspace/@alkdev/alknet/docs/research/alknet-filesystem/poc-summary.md`.
|
|
- **alkgit** — `/workspace/@alkdev/alkgit/docs/architecture/` (Phase 1,
|
|
reviewed; 18 ADRs). The git protocol crate, paused on storage; its
|
|
object backend is being re-based on alkblobs (its OQ-07).
|
|
- **alkblobs** — `/workspace/@alkdev/alkblobs/docs/architecture/`
|
|
(12 ADRs + requirements REQ-1..4). The pooled CAS store; its `ops`
|
|
surface and fleet mode (REQ-2/ADR-008) are the consumers' reactive
|
|
needs at their sharpest.
|
|
- **alknet-filesystem POC** — the running evidence that the three-layer
|
|
shape (SQLite path tree + CAS + honker coordination) works, 8 passing
|
|
tests including notify-fires-on-commit inside the mutation txn
|
|
(`watch_fires_on_commit`).
|
|
|
|
## Per-feature inventory
|
|
|
|
### notify / listen (durable pub/sub signal)
|
|
|
|
| Consumer | Evidence | Need | Confidence |
|
|
|---|---|---|---|
|
|
| alkfs (path tree) | alknet-filesystem `poc-summary.md` (test `watch_fires_on_commit`, Layer 3 notes); alkfs phase-0 §"What is already settled" | `notify()` inside the same transaction as path-tree mutations, atomically; watcher wakes on commit — this is the load-bearing invalidation source for any client-side cache over the tree | **pinned** |
|
|
| alkblobs (fleet) | requirements.md REQ-2; ADR-008 (fleet pins, advisory-locked sweeper); decisions/007 §Decision 2 (consumer tables co-tenant the pg instance) | cross-instance visibility of pool/GC state changes among fleet instances over one shared postgres — currently solved by DB rows + polling-free advisory machinery in-ADR; a notify surface is the natural substrate when a fleet node must tell siblings "sweep/manifest changed" | **documented** |
|
|
| alkblobs (any deploy) | decisions/007 §Decision 1 (honker ships WAL+NORMAL as the shipped tier defaults — the engine posture already assumes co-tenant reactive tooling on the sqlite file) | notify as the family's invalidation substrate on the sqlite default engine | **documented** |
|
|
|
|
Verdict: **in scope, first-class.** This is the feature the crate is
|
|
named for — the substrate's answer to "how does a change in the database
|
|
become visible to other processes/connections?" (phase-0 §Vision
|
|
context). Transactional atomicity (notify inside the caller's
|
|
transaction, dropped on rollback) is the property every row leans on —
|
|
it is guiding principle 2 in phase-0.md.
|
|
|
|
### named locks
|
|
|
|
| Consumer | Evidence | Need | Confidence |
|
|
|---|---|---|---|
|
|
| alkfs (write path) | alkfs phase-0 OQ-FS-05 ("Concurrent writers to the same path need a real policy"); `poc-summary.md` ("honker named locks for writer coordination — mechanical", unknown #4: `honker_lock_acquire('path:<bucket>:<branch>:<path>', writer_id, timeout)` in `WriteSession::open`) | per-path writer coordination: prevent concurrent write sessions stomp-locking each other; last-close-wins is session-safe but not POSIX — the lock is the missing piece that makes write sessions honest | **documented** (mechanically designed, unwired) |
|
|
| alkblobs (fleet GC) | gc-and-namespaces.md ("advisory-locked single sweeper"); ADR-008 fleet GC; store-api.md (`SweeperLock` contention as a `GcAbortCause`) | one sweeper at a time across a fleet of store instances sharing one pool — the sweeper advisory lock has TTL semantics and is already ADR-shaped | **pinned** |
|
|
| alkgit (ref CAS) | backend.md `GitRefs::apply_updates` (ADR-018: "one CAS transaction per call — the atomic-correctness home") | CAS against expected ref values could be served by pure row-level compare-and-set; a named lock is one admissible mechanism, not a proven need | **recalled** (alternative design exists) |
|
|
|
|
Verdict: **in scope.** Two consumers with concrete, one already
|
|
ADR-shaped; the third has a competing design that a lock is *allowed* to
|
|
replace but needn't.
|
|
|
|
### queues
|
|
|
|
| Consumer | Evidence | Need | Confidence |
|
|
|---|---|---|---|
|
|
| alkfs (sync/replication) | alkfs phase-0 OQ-FS-14 ("the family pattern is composition — honker queues as the local outbox"); `poc-summary.md` ("background sync/replication kicks — honker queues; transactional outbox: enqueue in same txn as mutation"; "The honker queue is the coordination mechanism for background fetching" — the read-miss→async-fetch path) | durable background work whose trigger must be atomic with the mutation that created it (sync kicks, fetch-on-miss workers) — the write-session and deferred-fetch machinery of the FS | **documented** |
|
|
| alkblobs (ops/maintenance) | ops-surface.md (pin-token renewal is a TTL/TTL-3 cadence — a renew worker); gc-and-namespaces ("Explicit sweep; embedder owns cadence" — embedders that want interval sweeps add them above, per ADR-005 no-ambient-timers) | periodic maintenance (sweep cadence, pin renewal, orphaned-session reaping per alkfs OQ-FS-07) — every one of these has been *deliberately* left as "an embedder schedules it above"; a durable scheduled-job surface is the substrate answer those embeddings currently have to invent | **documented** |
|
|
| family-wide | phase-0 §Vision (queues as part of the one-sentence vision, from honker's default feature set) | the base reactive substrate: consumers shouldn't each hand-roll a poll loop over their own tables | **recalled** |
|
|
|
|
Verdict: **in scope.** The strongest need is the transactional-enqueue
|
|
half (outbox), which is the same property as notify (principle 2) —
|
|
queues extend it to *work* rather than *signal*.
|
|
|
|
### scheduler (cron / @every)
|
|
|
|
| Consumer | Evidence | Need | Confidence |
|
|
|---|---|---|---|
|
|
| alkblobs (sweep cadence) | ADR-005 ("no ambient timers"; embedder owns cadence) | embedding a sweep interval per node is exactly cron/`@every` — with leader election if the deployment is a fleet (honker's scheduler: leader-elected advisory lock + missed-boundary catch-up) | **documented** (the need is ADR-pinned even though the ADR assigns it elsewhere) |
|
|
| alkfs (session reaping) | OQ-FS-05/OQ-FS-07 (crash cleanup of orphaned write sessions) | periodic orphan reaping pass | **recalled** |
|
|
|
|
Verdict: **in scope, but the thinnest of the confirmed ones.** The need
|
|
is real (there is a family-wide "who sweeps/renews/reaps, and when"
|
|
problem that every consumer has punted to 'the embedder') but only just
|
|
begins to have consumers lean on it. Watch: if the scheduler turns out
|
|
to be queues + a tick, it may collapse into queues (OQ-ST-04's job).
|
|
|
|
### transactional outbox helper
|
|
|
|
| Consumer | Evidence | Need | Confidence |
|
|
|---|---|---|---|
|
|
| alkfs | OQ-FS-14, poc-summary (same rows as queues) | the enqueue-inside-business-tx shape, by name | **documented** |
|
|
|
|
Verdict: **in scope** (as a *helper over* queues, per honker's shape) —
|
|
same evidence as queues; kept separate in the inventory only to note
|
|
that the helper is cheap once queues exist, not a separately-needed
|
|
feature.
|
|
|
|
### streams (durable pub/sub with per-consumer offsets)
|
|
|
|
| Consumer | Evidence | Need | Confidence |
|
|
|---|---|---|---|
|
|
| family-wide (reactivity requirement) | user/planning record, 2026-10-04 (operator's authority — the REQ-2 recording convention) | watching for specific event *types* from several different places at once. Concrete example: a git app at gitea/gitlab scale — users and other apps subscribe to changes on a repo. Many cases along these lines. | **documented** (operator-authority record; no consumer doc has grown the section yet) |
|
|
|
|
Verdict: **in scope, first-class.** Corrected 2026-10-04 — the initial
|
|
draft (earlier the same day) graded this **absent** because no consumer
|
|
document names it; that was a fact about the paused documents, not
|
|
about the need, and the operator's correction supersedes it. The
|
|
reactivity reasoning: notify is fire-and-forget — no replay, no
|
|
durability (honker's own split: "streams are the durable cousin") — so
|
|
*subscriptions* cannot be built on it honestly. A repo-change
|
|
subscription must survive the subscriber being offline at event time
|
|
(durable log + per-consumer offset + replay-on-attach is exactly
|
|
honker's stream model). Why the paused docs missed it: the consumers
|
|
that want it (a git app's subscription surface, cross-app event
|
|
watching) are *applications above* the current paused crates, so
|
|
nothing written down yet carries the row — which is expected, not
|
|
suspicious. Design note for OQ-ST-04: on the Postgres side this is a
|
|
NOTIFY-triggered durable event table with per-consumer cursors (the
|
|
pg-boss-family shape); the per-consumer offset contract, not the
|
|
storage shape, is the seam to pin.
|
|
|
|
### rate limits
|
|
|
|
| Consumer | Evidence | Need | Confidence |
|
|
|---|---|---|---|
|
|
| alkgit (budgeted resources) | ADR-009 budget shape (max pack size, max concurrent blocking pipeline tasks); OQ-FS-16's OQ references it too | alkgit's budgets are enforced in its own wire layer (backend.md: "the wire layer enforces the pipeline-concurrency budget itself") — a store-level rate limit is *not* the proven mechanism; a distributed rate-limit (across processes) is a different thing alkgit has not asked for | **recalled** (alternative design exists, in-crate) |
|
|
|
|
Verdict: **weak candidate — first cut candidate.** No document needs
|
|
*store-level* rate limiting; alkgit's budgets are enforced above the
|
|
storage seam with its own mechanism. Keep listed because honker's
|
|
default set includes it and the machinery is trivial next to queues,
|
|
but it is now the least-needful row (streams outranked it as of the
|
|
2026-10-04 correction).
|
|
|
|
### result storage (`save_result` / `get_result` / `sweep_results`)
|
|
|
|
| Consumer | Evidence | Need | Confidence |
|
|
|---|---|---|---|
|
|
| (none pinned) | — | — | **absent** |
|
|
|
|
Verdict: **in scope per the working posture, flagged for
|
|
cut-at-implementation.** Adjacent-tooling shape (a queue job's outcome
|
|
queryable by id), useful to queues+scheduler consumers but named by
|
|
none of them yet.
|
|
|
|
## What this inventory changes
|
|
|
|
- **OQ-ST-01** stops being "blocked on a consumer-driven inventory
|
|
pass" — this *is* that pass, run against paused consumers' documents
|
|
instead of running consumers. Remaining per-feature scope votes
|
|
shrink to: streams, rate limits, result storage (all
|
|
keep-with-flag), and the exclusion lines (DAGs/chains/chords,
|
|
multi-writer replication, cross-machine locking — no consumer names
|
|
these either; they stay out unless a consumer document grows one).
|
|
- **OQ-ST-02** gains a data point: the confirmed features (notify,
|
|
locks, queues, outbox, scheduler) are the same feature family on
|
|
both engines, which argues for the single-crate-with-feature-gated-
|
|
engines shape rather than a reactive-core + per-engine crates split
|
|
(the split buys something only if the engines' feature sets diverge
|
|
sharply, and this inventory says they don't).
|
|
- **OQ-ST-07** sharpens: no consumer needs the *loadable extension*
|
|
surface (every consumer is in-process Rust attaching to its own
|
|
connection, the `honker-core`/`attach_honker_functions` shape per
|
|
the POC summary); the embedded-rusqlite-only shape is the whole
|
|
consumer story so far. The loadable-extension option is now a
|
|
cut-only decision, not an open design fork.
|
|
- **OQ-ST-04's** contract-pinning work keeps all of the confirmed
|
|
features as contract candidates; the ambiguity found in the
|
|
inventory (queues vs scheduler boundary; streams' absent consumer;
|
|
rate limits' alternative mechanism) is input to which parts of the
|
|
honker-rs surface become contract versus cut.
|
|
|
|
## The consumer list, honestly bounded
|
|
|
|
- The inventory covers artifacts existing as of 2026-10-04: alkfs
|
|
phase-0, alkgit architecture (reviewed), alkblobs architecture
|
|
(draft-but-ADR'd), alknet-filesystem POC summary. If a new consumer
|
|
is added to the family (alksftp, the alknet rewrite, the ops
|
|
platform), a row gets added here *before* it is assumed into a scope
|
|
decision — the same discipline as alkblobs' requirements.md ("when a
|
|
consumer fact changes, it changes here first").
|
|
- Absence of evidence is marked, not overruled: rate limits and
|
|
result storage stay on the list with their flags visible, per the
|
|
working posture, rather than being cut by this document alone.
|
|
Corrections run the other way too: an operator-authority record (the
|
|
2026-10-04 streams correction) upgrades a row without waiting for a
|
|
consumer document to grow one — recorded here first, per the
|
|
requirements-change convention. |