alkgit/docs/research/pocr-findings.md

POC-R findings: composed registry + reactivity

Status: complete — proceed (the family adapter + the row-write/post-commit/reconcile posture landed; ADR inputs ready) Date: 2026-10-17 Mode: standalone (POC crate at /workspace/alkgit-pocr; no repo code touched — the ADR-018 trait family was consumed as shape documentation only). Deps: published-surface alkstore core + alkstore-sqlite/alkstore-postgres/alkstore-mem (path, the published family at 0.1), alkblobs main-branch published surface (at POC time the sibling carried the throwaway prototype branch; this POC used only blob APIs untouched by it — the git-object family has since landed per ADR-018 and the branch is gone), rusqlite + tokio-postgres for the registry's own co-tenant connections. Re-run note (2026-10-18): the suite ran green against the current sibling surfaces (12/12: reactive, two-process, maintenance, live pg) after the landed round — no behavioral drift, no new asks. Migration note (alkblobs 0.2.0 — 2026-10-19): the facade break (ADR-020: put → put_as(algorithm, …)) touched only the three sweep-demo blob puts (put_as(GitBlobSha256, Some(n), …) in demo.rs/maintenance.rs/reactive.rs) — the POC's own registry/refs/sweep machinery has no other alkblobs surface. Suite green the same day. Depends on: substrate-recomposition.md (the POC plan, DQ-3/7/8, the hot-path corollary), POC-O findings (refs-in-rows + the pin-bridge ingest posture this POC's receive-flow shapes lean on), alkcall's ownership adapter template (the family adapter pattern).

Hypothesis (restated)

The registry composes from the two substrate crates: record tables co-tenant on the substrate engines behind the family adapter pattern (sync read provider + async write store + in-memory default), reactive halves carried by alkstore (wake-driven cache invalidation, a change feed with per-consumer cursors, scheduler+queue background work), and the no-cross-store-atomicity write posture (durable row, post-commit effects, sequence + reconcile) lands cleanly in a receive-pack-shaped flow — with the DQ-8 data_version backstop probe deciding whether sqlite co-tenancy confers free invalidation.

Result

Confirmed on every gate, with one decisive structural answer (the sqlite backstop is deliverable, not internal) and one named artifact (the cached provider's coherence contract, stated below).

Gate	Result
Registry co-tenancy (relational adapter over the substrate engine instance)	one trait pair (RegistryRead sync / RegistryWrite async-trait), two engine twins (sqlite, postgres) implementing the identical shape, plus the in-memory default adapter; the sqlite co-tenant table + the alkstore machinery + alkblobs kv all on one file, no contention
Record write → reactive effects	post-commit publish (durable stream, keyed by repo id) + notify (wake hint) + enqueue (housekeeping) — each independent, each fail-soft, covered by the reconcile recipe
Wake-driven invalidation of a cached read provider	the snapshot-swap provider refreshed only from wakes + own writes + the reconcile tick (no polling); refresh count stable across idle windows; wake-coalesced bursts collapse
Cross-process invalidation wake	the two-process gate (tests/two_process.rs): writer and watcher processes over one sqlite file — the watcher's cache repopulated within wake latency of the writer's commits, wake-driven not polled
DQ-8 data_version backstop probe	deliverable, not internal: an alkstore listen receiver wakes on any commit to the file — including alkgit's own registry writes on their own connection, including from another process — measured ~3.2 ms after an unannounced co-tenant write (debug, quiet file); the demo ran the whole gate cross-process
Sequence/reconcile (the crash-window recipe)	registry_seq singleton bumps inside the row-write transaction (watermark transactionally exact); a durable commit with suppressed effects closed by one reconcile sweep (rows_since → snapshot install → watermark advance); second reconcile idempotently empty; the delete twin consumes a sequence identically
The receive-flow write posture (refs + registry + blobs)	the three durable surfaces in sequence each in its own transaction — blobs put under a pin bridge → (ref CAS, POC-O's one-txn home) → registry put → announced post-commit; the suppressed-effect variant still closes via reconcile; the pinned object survives the sweep, unreachable objects are reclaimed
Change feed cursor model (second consumer)	per-consumer durable checkpoints (monotone saves only), restart resumes after the saved offset, a fresh consumer replays from the log head, offset-anchored direct reads interleave freely; keyed publishes are carried metadata (not partitioning/dedup/order) — per-seq idempotency is the consumer's rule
Scheduled sweep + housekeeping chain	@every schedule → alkstore leader-elected scheduler → firing lands in the maintenance queue → worker claim → named-TTL leader lock (cross-process exclusion; the locked-out attempt is a skip + retry, a value not an error) → alkblobs::sweep() returns report data (stage → delete across the sweep window); the writer's orphaned blob reclaimed by the watcher's sweep cross-process
Postgres leg	the same composition code over alkstore-postgres machinery + the PgRegistry twin: announce, stream cursors, and reconcile all land; the DQ-8 asymmetry is real — a notify-free write wakes nothing on postgres (no file-level bump); LISTEN/NOTIFY is per-channel/per-notify, and the reconcile backstop closes the gap

Stack under test: alkstore-sqlite/postgres/mem (path, 0.1 family), alkblobs kv engines (mem_only, kv_only_sqlite) + sweep/pin/liveness surface, rusqlite + tokio-postgres registry connections.

The sqlite backstop: the deciding probe (DQ-8)

The open question was whether alkstore-sqlite's watcher (PRAGMA data_version poll) delivers its wake to listeners or stays alkstore-internal. Answered both ways before running anything: the source confirms the listen bridge subscribes to the watcher's fanout (alkstore-sqlite/src/notify.rs — every commit fans into the subscriber feed, coalesced), and the probe confirmed it live:

• an alkstore listen(WAKE_CHANNEL) receiver received a wake 3.2 ms after an unannounced co-tenant registry write (bare rusqlite connection, no alkstore call at all — the POC-O ref-table shape);
• the two-process demo ran the same thing across processes: the watcher's listener logged the wake-driven refresh carrying the writer's records.

Design consequence: on sqlite co-tenancy, the post-commit notify call is redundant — the co-tenant file itself is the invalidation bus, for any writer, at any layer (registry rows, blob puts, queue traffic). The notify stays in the sequence anyway: it is load-bearing on postgres (no file-level bump — LISTEN/NOTIFY needs the explicit post-commit notify) and it names the channel semantically for engines with per-channel delivery. This is the honest engine split the design must carry:

• sqlite: invalidation is free and file-wide; a lost hint cannot exist while any write lands; the reconcile backstop is for correctness, not latency;
• postgres: invalidation is per-channel and per-commit; a failed notify leaves the cache stale until the reconcile tick — the tick's cadence is the real staleness bound there.

Also measured (honest corollary): the file-wide fanout means the watcher's own background traffic wakes it — in the demo (scheduler

• queue worker + schedule firings at the default cadence) the listener refreshed ~5×/s. Harmless at this scale (refresh = one table read + snapshot swap), but the assembly tuning datum stands: raise the watcher poll interval, or accept the churn — a wake-filtered notification channel is not needed structurally, and is listed as assembly-level tuning, not an ask.

The coherence contract (the named deliverable)

The hot-path corollary asked for the cache's contract to be stated, not emergent. The POC pinned it on CachedRegistryProvider and it is the recommended ADR text:

• Cached set: the whole repo-record table, held as one immutable snapshot (Arc<HashMap>, swapped under a short lock). Whole-table is deliberate: record tables are small, snapshot swaps make coalesced refreshes trivially correct (refresh re-reads the table; no delta application to get wrong), and readers never block.
• Hot path: one lock read + Arc clone + map lookup. Zero I/O, zero round trips on the steady state.
• Invalidation triggers: (1) alkstore wakes — the post-commit notify hint, or sqlite's file-level bump from any writer (backstop); (2) the provider's own write path (write-through); (3) the reconcile tick (the correctness backstop; the only latency bound on postgres when a notify is lost).
• Staleness bound: mutation visibility ≤ one wake delivery + one snapshot refresh. On sqlite ≤ ~ms (3.2 ms measured for an unannounced write; a lost hint is not a real possibility while the file is written to); on postgres ≤ the reconcile cadence after a lost notify.
• Refresh coalescing: a burst of mutations within one wake collapses into one refresh, correctly.
• Failure posture: refresh failures leave the previous snapshot serving; the next wake, write, or reconcile tick retries. (The provider is the degenerate case of the family adapter's in-memory default — always-populated, always-correct, single process.)

The no-cross-store-atomicity posture, landed (DQ-8)

The classic outbox pattern requires the business write and the outbox enqueue to share one transaction — impossible here by contract (alkstore's TxHandle deliberately carries no consumer SQL). The posture that landed instead:

1. The durable write (row + its monotone mutation sequence, one transaction on the registry's own connection — the sequence is transactionally exact because the counter bumps in the same tx);
2. The post-commit effects (publish → durable change feed; notify → wake hint; enqueue → housekeeping), each independent and fail-soft;
3. The reconcile sweep: rows_since(applied_watermark) → snapshot install → watermark advance — idempotent by construction (pure reads, monotone watermark), the closure for any lost effect, driven by the scheduler tick (and usable on-demand).

In the receive-pack shape this composes as: blobs put under a pin bridge → ref CAS (POC-O's one-txn home) → registry put → announce post-commit. The crash window demonstrated (durable commit, effects suppressed) closed in one reconcile pass. The stream itself is the second correctness carrier — a consumer that missed a wake but got the event applies it by seq; a consumer that missed both is closed by the reconcile tick; the watermark makes every consumer eventually converge to the registry's own truth.

What this buys the design: no distributed-transaction machinery, no dual-write bookkeeping beyond the sequence column every registry table already wants for its own audit trail, and one background tick.

The family adapter pattern (DQ-3a), validated

• Trait shapes: RegistryRead (sync resolve, the OwnershipProvider posture — the advertisement path's hot read) + RegistryWrite (async put/delete/list/watermark/ rows_since, the OwnershipStore posture). The POC used #[async_trait] for the new traits (DQ-6 note: the alkgit-side trait posture question now has two data points — ADR-018's pinned async-trait family and this POC's new-trait choice — both the same side; alkstore's hand-boxed BoxedFuture posture stays confined to alkstore's own surface).
• Engine twins: the identical shape implemented for sqlite and postgres with per-engine clients; the reactive composition code (announce, waker, stream consumer, reconcile) is engine-generic over the alkstore Store trait and ran unchanged over all three engines (mem included, where notify/listen work in-process).
• Co-tenant tables: the registry owns its tables and its connection, next to the engine-owned schemas — the alkblobs ADR-013 §3 posture, now prototyped for alkgit's registry. The registry's registry_seq singleton (per-table monotone mutation counter) is the reconcile watermark's anchor and transactionally exact.
• registry-file's demotion is now evidence-backed: the relational adapter is less code than the file store's index/validity handling, and inherits engine properties (WAL, busy_timeout, cross-process sharing) the file store hand-rolled. The scrap decision stays an ADR conversation, but no counter-evidence appeared.

The cursor model for a second consumer (DQ-3c)

Validated as designed: subscribe under a consumer name → the engine's stored checkpoint is the durable cursor → saves are explicit and monotone → a re-subscribed consumer resumes after its checkpoint while a fresh consumer replays from the head → offset-anchored direct reads (read_since) interleave freely. Keyed publishes (the repo id) round-trip as metadata only — per the contract, not a partition or a dedup key; the consumer's per-seq idempotency rule is the dedup. The trim story stays as alkstore ships it (trim_to, gaps legal) — right for this feed's load, not exercised further here.

Maintenance: the scheduled sweep chain (gate)

The gate's chain ran end-to-end cross-process:

schedule("alkgit-sweep-schedule", "@every 2s")
  → alkstore scheduler (reserved-leadership lock; one leader)
  → firing lands in queue "alkgit-maintenance"
  → worker claim (visibility-timeout discipline)
  → try_lock("alkgit-sweeper", ttl) — cross-process GC exclusion;
      a locked-out sweep is a skip + short retry, a value
  → alkblobs Store::sweep() → SweepReport (data, never error)

Observed sweep-window semantics: a freed blob stages as a candidate (candidates_staged), executes on a later pass (deleted) — the two-pass windowed delete works as specced. The writer's orphaned blob was reclaimed by the watcher's sweep; a pinned/liveness-protected object survived (SweepReport.aborted: None). The pack-housekeeping queue job (enqueued post-commit by the writer) executed in the watcher. GC liveness stays an alkgit-owned composition (POC-O's refs walk); the POC's stand-in source exists only so sweeps don't abort with NoLivenessSources.

What this settles / does not

• DQ-3: the composition shape has its evidence — co-tenant relational registry with engine twin adapters, family adapter pattern, sequence+reconcile reactive halves, in-memory default. The ADR round (registry backing, option a + c shapes) has everything it needs; registry-file's removal decision rides it.
• DQ-8: answered on all three fronts — the backstop probe (deliverable on sqlite, absent on postgres), the post-commit + sequence/reconcile recipe (landed, idempotent, demonstrated through a live crash window), and the cross-store-atomicity posture (the pin bridge closes the object window; the watermark closes the event window).
• The hot-path corollary: the coherence contract is now named and prototyped (see above); the cached provider is the advertisement path's cache, invalidated by wake, bounded by reconcile.
• DQ-7 (assembly story): the multi-writer co-tenant assembly measured clean again; new data: the file-wide wake fanout's churn datum (~5 refreshes/s for a self-traffic-heavy watcher at default cadence) and the four-pool + registry-connection budget standing.
• Not settled / deferred: multi-node replication (POC-X; the cursor model is its substrate), a postgres two-process demo through the demo bin (LISTEN/NOTIFY is engine-native cross-process — the asymmetry facts don't change cross-process, and the pg leg demonstrated them cross-connection), registry schema migration stories (an ADR/impl concern), provider behavior under heavy multi-writer write bursts (single-writer push shape at POC scale; the coalesced-refresh posture should hold, unmeasured at burst scale), grants/keys shape in the record table (deliberately out of scope — pointer to ADR-011).

Upstream asks (updated)

None structural from this POC. Row-first notes (for alkstore's consumer-inventory.md when the re-baseline lands): stream enumeration and scheduled trim_to helpers (already listed); wake-filter / per-channel listener inventory — recorded here as assembly-level tuning only (poll cadence), since the file-wide sqlite fanout is contract-correct and the pg channel fanout is already per-channel. The alkblobs surface consumed (pin, sweep, LivenessSource, kv engines) needed nothing new; the &Key borrow nit (ask #4 in the main doc) still stands untouched.

POC art

• Crate: /workspace/alkgit-pocr (standalone; see its README; deps are the published alkstore/alkstore-mem/sqlite/postgres family + alkblobs at the sibling's current branch, published surface only).
• src/adapter.rs — RegistryRead/RegistryWrite + InMemoryRegistry
  • CachedRegistryProvider (the coherence contract in its docs).
• src/sqlite_registry.rs / src/pg_registry.rs — the co-tenant relational twins (row + sequence table in one tx; rows_since).
• src/reactor.rs — announce (publish + notify), wake runner, stream consumer (cursor + idempotent apply), AppliedWatermark, reconcile.
• src/maintenance.rs — schedule registration, scheduler runner, worker (leader TTL lock → sweep / housekeeping), WorkerEvent.
• src/bin/demo.rs — the two-process gate demo bins (writer, watcher).
• Tests: tests/reactive.rs (8 gates incl. the backstop probe + mem symmetry + the latency datum), tests/maintenance.rs (the sweep chain, leader-lock exclusion), tests/two_process.rs (the writer/watcher gate, four assertions on the watcher transcript), tests/pg_reactive.rs (the postgres leg + asymmetry probe; skipped without ALKPOCR_PG=1).
• Scratch: the co-tenancy probes (probe1–4 examples/) were deleted after use; their conclusion (plain cross-process rusqlite WAL visibility works — the demo's only hard requirement) is this bullet.