Files
alkstore/docs/research/consumer-inventory.md
T
glm-5.3-flash 4391f6e879 docs: open Phase 1 — architecture spec set over the Phase 0 evidence
docs/architecture/ now exists: README index, overview, five component
specs (core-contract, engine-sqlite, engine-postgres, queues,
deployment), ADR-001..007 carrying the Phase 0 resolved decisions
(crate split, feature scope, per-engine drivers, dependency
ownership, wake contract, tx seam), and the centralized
open-questions tracker promotion: OQ-ST-01..08 mirror to OQ-01..08
one-to-one with statuses/resolutions carried; new Phase 1 questions
append (OQ-09 scheduler collapse, OQ-10 contract versioning).
Open Phase 1 work: OQ-04 contract pinning (high), OQ-05 queue
semantics depth (high), OQ-06 honker-core quality read (high;
fork-trigger gate), OQ-08 capability surface, OQ-09, OQ-10.

Erratum fixed in phase-0 OQ-ST-04 (thread-affinity friction is
SQLite-side, previously garbled as pg-side) and a stale scheduler-
boundary pointer corrected in consumer-inventory.md. Two review
passes run (findings: OQ-promotion numbering faithfulness, ADR
back-reference sync) — all critical/warning findings resolved.
2026-10-04 18:13:10 +00:00

15 KiB


status: draft last_updated: 2026-10-04 (streams corrected to documented/in-scope — operator-authority record: type-filtered event watching from multiple places, e.g. repo-change subscriptions in a git app. No consumer doc carries the row yet; applications above the paused crates are the wanters. rate-limits + result-storage remain the keep-with-flag rows.)

alkstore — consumer-driven scope inventory

This document answers OQ-ST-01 (which honker features are in scope) the only way that avoids circular hedging: by walking the concrete evidence the paused consumers already left on disk. alkgit, alkfs, and alkblobs are paused processes, but their documents are stable, reviewable artifacts — requirements they impose on a reactive store, written before alkstore existed. That is decisively different from deferring to "what the consumers will say later," which was alkstore's original framing for OQ-ST-01 and which never collapses: the consumer can't run until the store exists and the store was waiting on the consumer — the classic low-level/client deadlock, the exact shape the SDD process's hedging rules name and the alkblobs open-questions tracker calls "Schrödinger's code."

Method: for each feature on honker's surface (the interface finding in phase-0.md §Interface finding), find where a consumer document names a need for it, quote the need, and grade the evidence. Confidence grades:

  • pinned — a consumer ADR or requirement doc names it and some design leans on it already.
  • documented — a consumer doc names the need concretely, but no design has leaned on it yet.
  • recalled — named in research/summary prose as intended, no architecture written against it yet.
  • operator-authority — the operator records a need the paused documents don't carry yet (the REQ-2 recording convention from alkblobs requirements.md), dated; expected where the wanter is an application above the paused crates. Upgrades equivalent to documented.
  • absent — no consumer document names a need. Included by default per the working posture, flagged for cut-at-implementation instead of silently carried.

The inventory is per-feature need, not per-feature design: how a need is served (honker-identical, pg-boss-shaped, or re-derived) is OQ-ST-05/06. This document only decides which features have a consumer, and hence which scope decisions OQ-ST-01 still has to take per feature.

The consumers and their artifacts

  • alkfs — /workspace/@alkdev/alkfs/docs/research/phase-0.md (Phase 0, 2026-09-23; 17 OQs). The VFS/workspace crate; its ancestor POC (alknet-filesystem-poc) used honker directly (honker-core 0.2.4 — notify-in-transaction, named locks identified as needed, queues as the sync outbox). Findings: /workspace/@alkdev/alknet/docs/research/alknet-filesystem/poc-summary.md.
  • alkgit — /workspace/@alkdev/alkgit/docs/architecture/ (Phase 1, reviewed; 18 ADRs). The git protocol crate, paused on storage; its object backend is being re-based on alkblobs (its OQ-07).
  • alkblobs — /workspace/@alkdev/alkblobs/docs/architecture/ (12 ADRs + requirements REQ-1..4). The pooled CAS store; its ops surface and fleet mode (REQ-2/ADR-008) are the consumers' reactive needs at their sharpest.
  • alknet-filesystem POC — the running evidence that the three-layer shape (SQLite path tree + CAS + honker coordination) works, 8 passing tests including notify-fires-on-commit inside the mutation txn (watch_fires_on_commit).

Per-feature inventory

notify / listen (durable pub/sub signal)

Consumer Evidence Need Confidence
alkfs (path tree) alknet-filesystem poc-summary.md (test watch_fires_on_commit, Layer 3 notes); alkfs phase-0 §"What is already settled" notify() inside the same transaction as path-tree mutations, atomically; watcher wakes on commit — this is the load-bearing invalidation source for any client-side cache over the tree pinned
alkblobs (fleet) requirements.md REQ-2; ADR-008 (fleet pins, advisory-locked sweeper); decisions/007 §Decision 2 (consumer tables co-tenant the pg instance) cross-instance visibility of pool/GC state changes among fleet instances over one shared postgres — currently solved by DB rows + polling-free advisory machinery in-ADR; a notify surface is the natural substrate when a fleet node must tell siblings "sweep/manifest changed" documented
alkblobs (any deploy) decisions/007 §Decision 1 (honker ships WAL+NORMAL as the shipped tier defaults — the engine posture already assumes co-tenant reactive tooling on the sqlite file) notify as the family's invalidation substrate on the sqlite default engine documented

Verdict: in scope, first-class. This is the feature the crate is named for — the substrate's answer to "how does a change in the database become visible to other processes/connections?" (phase-0 §Vision context). Transactional atomicity (notify inside the caller's transaction, dropped on rollback) is the property every row leans on — it is guiding principle 2 in phase-0.md.

named locks

Consumer Evidence Need Confidence
alkfs (write path) alkfs phase-0 OQ-FS-05 ("Concurrent writers to the same path need a real policy"); poc-summary.md ("honker named locks for writer coordination — mechanical", unknown #4: honker_lock_acquire('path:<bucket>:<branch>:<path>', writer_id, timeout) in WriteSession::open) per-path writer coordination: prevent concurrent write sessions stomp-locking each other; last-close-wins is session-safe but not POSIX — the lock is the missing piece that makes write sessions honest documented (mechanically designed, unwired)
alkblobs (fleet GC) gc-and-namespaces.md ("advisory-locked single sweeper"); ADR-008 fleet GC; store-api.md (SweeperLock contention as a GcAbortCause) one sweeper at a time across a fleet of store instances sharing one pool — the sweeper advisory lock has TTL semantics and is already ADR-shaped pinned
alkgit (ref CAS) backend.md GitRefs::apply_updates (ADR-018: "one CAS transaction per call — the atomic-correctness home") CAS against expected ref values could be served by pure row-level compare-and-set; a named lock is one admissible mechanism, not a proven need recalled (alternative design exists)

Verdict: in scope. Two consumers with concrete, one already ADR-shaped; the third has a competing design that a lock is allowed to replace but needn't.

queues

Consumer Evidence Need Confidence
alkfs (sync/replication) alkfs phase-0 OQ-FS-14 ("the family pattern is composition — honker queues as the local outbox"); poc-summary.md ("background sync/replication kicks — honker queues; transactional outbox: enqueue in same txn as mutation"; "The honker queue is the coordination mechanism for background fetching" — the read-miss→async-fetch path) durable background work whose trigger must be atomic with the mutation that created it (sync kicks, fetch-on-miss workers) — the write-session and deferred-fetch machinery of the FS documented
alkblobs (ops/maintenance) ops-surface.md (pin-token renewal is a TTL/TTL-3 cadence — a renew worker); gc-and-namespaces ("Explicit sweep; embedder owns cadence" — embedders that want interval sweeps add them above, per ADR-005 no-ambient-timers) periodic maintenance (sweep cadence, pin renewal, orphaned-session reaping per alkfs OQ-FS-07) — every one of these has been deliberately left as "an embedder schedules it above"; a durable scheduled-job surface is the substrate answer those embeddings currently have to invent documented
family-wide phase-0 §Vision (queues as part of the one-sentence vision, from honker's default feature set) the base reactive substrate: consumers shouldn't each hand-roll a poll loop over their own tables recalled

Verdict: in scope. The strongest need is the transactional-enqueue half (outbox), which is the same property as notify (principle 2) — queues extend it to work rather than signal.

scheduler (cron / @every)

Consumer Evidence Need Confidence
alkblobs (sweep cadence) ADR-005 ("no ambient timers"; embedder owns cadence) embedding a sweep interval per node is exactly cron/@every — with leader election if the deployment is a fleet (honker's scheduler: leader-elected advisory lock + missed-boundary catch-up) documented (the need is ADR-pinned even though the ADR assigns it elsewhere)
alkfs (session reaping) OQ-FS-05/OQ-FS-07 (crash cleanup of orphaned write sessions) periodic orphan reaping pass recalled

Verdict: in scope, but the thinnest of the confirmed ones. The need is real (there is a family-wide "who sweeps/renews/reaps, and when" problem that every consumer has punted to 'the embedder') but only just begins to have consumers lean on it. Watch: if the scheduler turns out to be queues + a tick, it may collapse into queues (OQ-ST-04's job).

transactional outbox helper

Consumer Evidence Need Confidence
alkfs OQ-FS-14, poc-summary (same rows as queues) the enqueue-inside-business-tx shape, by name documented

Verdict: in scope (as a helper over queues, per honker's shape) — same evidence as queues; kept separate in the inventory only to note that the helper is cheap once queues exist, not a separately-needed feature.

streams (durable pub/sub with per-consumer offsets)

Consumer Evidence Need Confidence
family-wide (reactivity requirement) user/planning record, 2026-10-04 (operator's authority — the REQ-2 recording convention) watching for specific event types from several different places at once. Concrete example: a git app at gitea/gitlab scale — users and other apps subscribe to changes on a repo. Many cases along these lines. documented (operator-authority record; no consumer doc has grown the section yet)

Verdict: in scope, first-class. Corrected 2026-10-04 — the initial draft (earlier the same day) graded this absent because no consumer document names it; that was a fact about the paused documents, not about the need, and the operator's correction supersedes it. The reactivity reasoning: notify is fire-and-forget — no replay, no durability (honker's own split: "streams are the durable cousin") — so subscriptions cannot be built on it honestly. A repo-change subscription must survive the subscriber being offline at event time (durable log + per-consumer offset + replay-on-attach is exactly honker's stream model). Why the paused docs missed it: the consumers that want it (a git app's subscription surface, cross-app event watching) are applications above the current paused crates, so nothing written down yet carries the row — which is expected, not suspicious. Design note for OQ-ST-04: on the Postgres side this is a NOTIFY-triggered durable event table with per-consumer cursors (the pg-boss-family shape); the per-consumer offset contract, not the storage shape, is the seam to pin.

rate limits

Consumer Evidence Need Confidence
alkgit (budgeted resources) ADR-009 budget shape (max pack size, max concurrent blocking pipeline tasks); OQ-FS-16's OQ references it too alkgit's budgets are enforced in its own wire layer (backend.md: "the wire layer enforces the pipeline-concurrency budget itself") — a store-level rate limit is not the proven mechanism; a distributed rate-limit (across processes) is a different thing alkgit has not asked for recalled (alternative design exists, in-crate)

Verdict: weak candidate — first cut candidate. No document needs store-level rate limiting; alkgit's budgets are enforced above the storage seam with its own mechanism. Keep listed because honker's default set includes it and the machinery is trivial next to queues, but it is now the least-needful row (streams outranked it as of the 2026-10-04 correction).

result storage (save_result / get_result / sweep_results)

Consumer Evidence Need Confidence
(none pinned) — — absent

Verdict: in scope per the working posture, flagged for cut-at-implementation. Adjacent-tooling shape (a queue job's outcome queryable by id), useful to queues+scheduler consumers but named by none of them yet.

What this inventory changes

  • OQ-ST-01 stops being "blocked on a consumer-driven inventory pass" — this is that pass, run against paused consumers' documents instead of running consumers. Remaining per-feature scope votes shrink to: streams, rate limits, result storage (all keep-with-flag), and the exclusion lines (DAGs/chains/chords, multi-writer replication, cross-machine locking — no consumer names these either; they stay out unless a consumer document grows one).
  • OQ-ST-02 gained a data point here, now superseded: the confirmed features are one feature family on both engines — read at inventory-draft time as arguing for a single-crate shape. Resolved for the reactive-core + engine-crate split on operator authority (2026-10-04, phase-0 OQ-ST-02): the split isolates the engines' real asymmetry of work (SQLite rides honker's existing machinery as the baseline; Postgres is the build-heavy side under the LISTEN/NOTIFY + pg-boss-family design brief) and makes any future engine additive rather than a feature-graph edit. The inventory's uniform-feature-family fact still holds — it now says the core contract can stay small (one family, both engines), not that the crates should merge.
  • OQ-ST-07 sharpens: no consumer needs the loadable extension surface (every consumer is in-process Rust attaching to its own connection, the honker-core/attach_honker_functions shape per the POC summary); the embedded-rusqlite-only shape is the whole consumer story so far. The loadable-extension option is now a cut-only decision, not an open design fork.
  • OQ-ST-04's contract-pinning work keeps all of the confirmed features as contract candidates; the ambiguity found in the inventory (queues vs scheduler boundary; streams' absent consumer; rate limits' alternative mechanism) is input to which parts of the honker-rs surface become contract versus cut. (Correction 2026-10-04, promotion pass: the scheduler-collapse question was OQ-ST-05/06 territory in this line's original framing and is now tracked as OQ-09 in docs/architecture/open-questions.md — not OQ-ST-04/02 as this line previously garbled.)

The consumer list, honestly bounded

  • The inventory covers artifacts existing as of 2026-10-04: alkfs phase-0, alkgit architecture (reviewed), alkblobs architecture (draft-but-ADR'd), alknet-filesystem POC summary. If a new consumer is added to the family (alksftp, the alknet rewrite, the ops platform), a row gets added here before it is assumed into a scope decision — the same discipline as alkblobs' requirements.md ("when a consumer fact changes, it changes here first").
  • Absence of evidence is marked, not overruled: rate limits and result storage stay on the list with their flags visible, per the working posture, rather than being cut by this document alone. Corrections run the other way too: an operator-authority record (the 2026-10-04 streams correction) upgrades a row without waiting for a consumer document to grow one — recorded here first, per the requirements-change convention.