alkblobs/docs/architecture/ops-surface.md

status, last_updated, notes

status	last_updated	notes
stable	2026-10-17 (ADR-018 — the ops surface's digest wire form is blob-family-scoped in 0.1; a family-explicit wire form is a named-consumer + new-ADR door. The scoping record landed with the family's code — the ops module doc and the ingress doc carry it; behavior unchanged — review-002 C-1 complete)	ADR-017 — the authority model named (CAS global pool; capability-to-ask; separation = separate stores), renewal namespace parity with the mint record, and the remote-putter read+write role requirement. ADR-014 — put's data plane (zero call-plane bytes), one-response synthesis (ack-in-result), the alkcall amendment-4 gate. ADR-018 — the digest wire form's family posture (below)

Ops surface (alkcall-backed network operations)

What this is

The network-facing operation family over the store: have/need announcements, verified fetch, and streaming put — riding the alkcall substrate behind a feature-gated ops module in this crate (feature ops, default-off; ADR-001). The base crate stays lean and alkcall-free; enabling ops adds the alkcall dependency and registers the op family.

Why alkcall (ADR-001, the short form)

The op family is exactly alkcall's mixed-shape sweet spot (validated machinery, not parallel invention):

• JSON control plane — have/need announcements, stat-shaped probes, offer/carry lengths: alkcall OperationSpec + JSON Schema validation (alkcall ADR-016) + typed error schemas + from_call discovery + External/Internal visibility.
• Binary data plane — bulk bytes one way (fetch: Sub, alkcall ADR-021) or the other (put: Pub/Sink, alkcall ADR-046), allocated via the channel_open marker (alkcall ADR-047); established pump patterns (pump_bidi, alkcall ADR-050).
• ACL is solved there — AccessControl + ownership checks (alkcall ADR-011) under the alkcall ADR-017 privilege model map onto namespace gating directly; an in-band invented scheme would be a second, unreviewed authorization story in the family (AGENTS convention 6).

Producer/consumer vocabulary throughout; no transport enters either layer — the caller (an embedder, alkgit's replicator, a future alkfs sync) dials and hands an established Connection in (alkcall is a pure protocol crate; ADR-012 there).

Op family (WHAT is exposed)

Namespaces on the wire. Every ops payload carries an explicit namespace field — the consumer-scoped identifier the embedder's registry recognizes (syntax: non-empty UTF-8 string, embedder-validated; the minting of namespaces is the embedder's act — e.g. alkgit's registry maps repo ids to namespaces, alkfs maps workspace roots). The store core never sees this string (backends are namespace-blind, ADR-005); it selects the liveness source and the ACL resource. The ops module's job is to bind namespace → the embedder-registered liveness source for pin conversion, and to present the namespace as the alkcall ACL resource.

JSON control ops (Visibility per ADR-001 §Decision):

• blobs/stat {namespace, digest} → {len} — probe before offering; ACL-gated like fetch (read action on the namespace)
• blobs/have {namespace, digests[], token?} → {present[]} — the have half of have/need set diffing (hashes only — content never traverses this op); read-gated: an existence probe over arbitrary digests is a discovery surface, so it is gated exactly like fetch, not public. present[] corresponds to digests[] by index. The optional token is the renewal form (ADR-012 §2): when present, exactly one digest accompanies it — the handler verifies the token maps to that digest's pin row and that the presented namespace matches the mint record (renewal namespace parity, ADR-017), renews it (crate-internal pin_renew via the ops module's store-core access, ADR-010 §4), and answers with present[] as usual; an expired/foreign row answers present: false (the putter's backstop is re-put). Without token, have remains a pure existence probe (ADR-008's discipline).
• blobs/delete {namespace, digests[]} — operator machinery; Visibility::Internal, evaluated under the internal authority context per alkcall ADR-017 (internal calls switch authority context, never skip ACL): requires the embedder's operator/admin authority context, not a namespace grant. The namespace field exists for audit/triage scoping (which slice of the pool the deletion targets), not for gating.

Binary channel ops (registered via the channel_open marker):

The digest wire form's family posture ([ADR-018], 2026-10-17). The digest wire form is hex — the same form blobs/put's response returns — and the whole ops surface is blob-family-scoped in 0.1: the wire schema (^[0-9a-f]{40}$|^[0-9a-f]{64}$), the ingress (digest_ingress via Digest::from_git_oid_hex, whose contract is pinned the blob family only), the put handler's offer cross-check (it hashes the canonical blob derivation), and put responses all address the blob family and nothing else. This keeps the schema gate and the ingress in agreement by construction (the S-5 discipline). The git-object derivation family registered in ADR-018 is facade-only for now: a git-object oid presented on the wire answers the miss for its blob-family-shaped key — honest, because the family-explicit key bytes differ by tag — and git-object content moves directly through the store facade (an in-process consumer composes it; no named wire consumer of non-blob transfer exists yet). A family-explicit wire form (a typed or prefixed digest field) is a named-consumer-requirement door, decided by a new ADR — length-based family guessing is never the answer at any time.

• blobs/fetch (Sub — the server→client streaming op shape; open-op id channels/blobs/sub, the alk/blobs data-plane ALPN) — {namespace, digest, ranges?} in; verified bytes out: the consumer hash-checks the received whole against the carried digest (ADR-006's whole-blob verification — there are no per-chunk or per-range checks derivable under the canonical digest; ranged fetch responses carry the same ADR-006 slice-digest convention read_range returns). The establishment phase (ADR-049) carries the ack in the open reply's fields: {source, len?, slice_digests?} — source names the serving path (store/fanout); the whole form acks len from the store path (the hash-check's expected whole) and nothing from a fanout tap (a live put's tail; the hash-check runs at stream end); the ranged form acks one slice digest per range, in request order, over exactly the served bytes (over-reads clamp; the digest covers the clamp). The fanout consult (POC #3 finding A3's consumer side): a live broadcast → a subscriber arm taps it; no broadcast, or one that ended (a late joiner) → the normal post-commit get — identical bytes under CAS; a mid-put tap has no prefix, so the consumer's hash-check on a partial feed fails and the re-fetch is the degrade. Fetches never produce read-side broadcasts (recorded against the A3 text — the put handler's stream reader is the canonical producer; concurrent fetchers each ride their own facade read). The miss (absent digest, no broadcast) is the typed missing-error schema: the declared BLOB_MISSING contract, answered on the wire as the establishment refusal (channel:open_failed, reason unknown_resource, ADR-049's "the requested resource does not exist"), the digest in the refusal's message. The need announcement is the fetch miss — the have/need diff is caller-side; no separate need op exists (the wire carries only concrete fetch requests). Slow-subscriber policy (drop-and-late-join) is ops-layer policy, not a store concern.
• blobs/put (Sink — the client→server streaming op shape) — {namespace, digest?, len?} offer + byte stream in; server verifies against the canonical derivation before commit. Data plane (ADR-014 §1): bulk bytes ride the adopted channel's BiStream exclusively under ADR-040's bounded-buffer backpressure — the call plane carries the offer and the response, and nothing else; nonempty call.published on the marked op is rejected loudly, never consumed (no competing data path). Response: {token, digest} (ADR-012 §2) — the pin token (§Pin hand-over below) and the stored entry's canonical digest (hex), which an unknown-length putter needs for every later op. Refusal families (the op's declared error disclosure): VERIFICATION_REFUSED (the stream failed the canonical derivation — nothing committed, zero residue), PUT_REFUSED (a put-time store refusal — nothing committed), DIGEST_MISMATCH (the offered claim contradicted by the derivation — nothing handed over), and MINT_REFUSED (ADR-019 — the one refusal over a committed put: the stream committed and verified, but its hand-over's mint was refused at the token registry's admission cap; no token was issued and no pin custody was taken, the committed digest rides the refusal's details, and the CAS-idempotent re-put is the recovery). The establishment ack and the handler's result merge into the op's single response, emitted after the put handler completes — the put's ack is informative-in-result, not a gating ack (a putter cannot wait for the ack before streaming: sink ops respond once, at stream end; ADR-014 §2 pins the timing and the fetch/put asymmetry). A known-length offer is the encouraged path (one- pass); unknown-length rides the store's pre-threshold buffering path (ADR-003). Need half of have/need: a fetch miss is the need announcement — the consumer computes its need set by diffing blobs/have results and fetches the absent digests; no separate need op exists (the diff is caller-side; the wire carries only concrete fetch requests). Dependency gate (ADR-014 §3): put was not registered until alkcall's amendment-4 surface landed (Pub-typed marked opens, wrapper-awaits-handler, publish-chunk rejection, relay lifting, and the putter-facing stream-then-open client capability) — satisfied by alkcall 0.9.0 (2026-10-14); put unblocks for the put task, dependency floor at 0.9.

Pin hand-over for remote puts (the cross-wire contract). The server-side put handler owns the Pin guard; the offering side's reference registration happens in the remote process, invisible to the server. The hand-over contract:

1. the put lands pinned (ADR-005) — the entry cannot be swept while the handler holds the pin;
2. the blobs/put response returns a pin token in the specced response shape {token, digest} (ADR-012 §2 — opaque handle minted by the ops layer over the pin row the put committed; the token↔(key, owner) mapping is ops-layer state, not durable state — the durable state is the pin row itself, ADR-010 §4); the putter can later confirm its registration landed by observing the digest present via blobs/have;
3. the embedder's registered liveness source is the actual root of record: the contract on the embedder (documented on the ops module) is that it registers the namespace's roots such that a digest referenced by a namespace's manifest is either (a) already registered before the caller's next sweep could run on the serving node, or (b) protected by holding the pin token until its registration is confirmed (blobs/have returns present after registration). In short: the remote putter may not rely on an unconfirmed put surviving the next sweep — it holds its pin token until its registration shows up in blobs/have. A Pin conversion on the server side (token → registered liveness source) is the mechanism the embedder plugs its registry into.

Token renewal (ADR-010 §4, wired per ADR-012 §2 — the fleet-TTL race, resolved): the token is the pin — there is no second pin state. The server-side handler's in-process Pin guard drops when the blobs/put response completes (if it never dropped, the TTL machinery would be dead on the ops path — the pin row's expiry, not the guard's presence, is the fleet liveness signal). Under fleet engines the put's pin is an expiry-carrying row (ADR-008: TTL 60 s default, renewed every TTL/3). The remote putter holding a token is the row's owner and renews via the renewal form of blobs/have (ADR-012 §2: one digest, token alongside — the handler verifies the token maps to the digest's pin row and that the presented namespace matches the mint record — the mint records the namespace from the put offer, the same discipline confirm uses; ADR-017's parity ruling — and calls the crate-internal pin_renew via the ops module's store-core access — the ops module is in-crate and crosses the seam by construction, the same way the put handler holds the Pin guard; has itself remains a pure existence probe per ADR-008 — renewal is a token-authenticated act on the pin row, not a has semantic), or a re-put of the same digest (CAS dedup renews the row; the re-put carries the namespace itself, so parity holds there trivially). A putter may pause past the TTL/3 mark (20 s at the default TTL) only if it renews before expiry; a putter pausing past TTL lets the row expire — the renewal probe answers present: false, the sweeper reaps the row (ADR-008), and delete-then-recover (ADR-005) is the backstop: the putter re-offers the digest; CAS makes the re-put idempotent. The race resolves to a re-put, never to silent loss. Token validity domain: the minting serving node (ADR-012 §2) — the token↔(key, owner) mapping is ops-layer in-memory state in the node that minted it (process-lifetime, outliving the handler); renewal probes must reach that node, fleet peers refuse tokens they did not mint (present: false, not an error), and a minting node's death before renewal is ADR-005's in-flight-put loss case with the same re-put backstop — a documented embedder duty (route renewals to the serving node). For non-fleet serving nodes (any engine — sqlite, single-instance postgres, local, pg-lo; ADR-012 §2) the token is the in-process guard's handle — nothing expires; the process-crash case is ADR-005's in-flight-put loss with the same backstop.

The remote-putter role is read+write (ADR-017). The hand-over contract below already requires the putter to probe blobs/have to confirm its registration landed, and the renewal form rides blobs/have — both fleet pin duties ride read on the putter's own namespace. This pairing is a role-level duty, not a grant implication: the ACL matrix's write ⊄ read capability algebra is untouched, and an embedder granting put-without-read knowingly configures a principal that cannot complete fleet pin duties (minted pins expire through the TTL loop — the re-put backstop still prevents loss, but the pin never holds past TTL).

The admission posture (ADR-019 — the mint contract's typed answer to the unconfirming loop). The attack shape, on record (POC-L, deliberately unpatched in the consumer POC): a hostile client streaming conforming content and abandoning the confirm step each iteration — each minted token a held pin plus a registry entry, so on a non-fleet store the loop is unbounded registry growth plus custody GC can never reap (the content-mismatch variant rejects the put with no mint). The store's answer: the token registry admits up to its cap (OpsConfig::token_registry_max_entries, default 65_536; usize::MAX uncapped) and refuses mints beyond it — never evicting — with the put handler answering the already-committed put through the declared MINT_REFUSED error (above). The registry's cap bounds the registry's own resources (process-lifetime memory, never-expiring custody), not the loop's reach: an authenticated writer can still fill the pool with committed bytes between sweeps — that resource is the ACL write grant's story plus the GC cycle, not the registry's. The mint policy at the door is the embedder's responsibility (which identities mint; how many unconfirmed puts one principal may hold; whether a session confirms before its next mint); for a deployment whose door cannot prove confirm — an untrusted multi-tenant upload surface, the LFS shape — the fleet form is the safe posture (the pin row's TTL bounds every held pin with or without a confirm). Making the fleet TTL posture available non-fleet was rejected (ADR-019: the TTL is the fleet core's pin-row machinery, an expiry would break the confirm verb's contract, and the honest alternatives already exist); re-opening it is a new ADR.

Placement of the op registration (which embedders wire where they want them exposed) matches the alkgit ops pattern (its git/repo/* call ops): the module exports spec/handler pairs; the embedder assembles.

ACL mapping

Resources and actions, evaluated by alkcall's AccessControl machinery at op entry — this module never re-implements an authorization check (convention 6; ADR-001):

The authority model, named (ADR-017): the pool is a CAS global pool, and a namespace grant is capability-to-ask, not data partitioning. A read/put grant on a namespace confers the right to ask the store's ops on that namespace's behalf — it does not partition content. Content is addressed by digest; digest knowledge plus any namespace's read capability reaches the content, regardless of which namespace it was put under. A deployment whose threat model requires data isolation between namespaces runs separate store instances (separate pools — the ADR-003 topology ruling, named on this surface). This statement is the S-1/S-3 resolution's landed form; the model cannot be silent.

Resources and actions:

• Namespace = resource. resource_type: "blob-namespace", resource_id_path selecting the payload's namespace field; actions map onto read/put/manage:
  • blobs/fetch, blobs/stat, blobs/have — the read action; unlisted namespaces (no ACL entries) deny by default (OQ-09's resolution; a grant per namespace is the deferred cost, deliberately paid over accidental exposure)
  • blobs/put — the write action
  • blobs/delete — not namespace-gated at all; internal authority context only (alkcall ADR-017: internal switches context, never skips ACL)

Store-facing relationship

The ops module is a consumer of the store core (store-api.md), not a side channel: fetch handlers call get/read_range/stat; put handlers call put; both honor pins and delete windows. There is no ops-internal path around the store facade (ADR-001 §Consequences — this is what makes "re-homing the ops surface later loses nothing" true). One fleet topology additionally rides this module: re-routing (ADR-008) — a fleet whose nodes do not share large-tier media routes large-blob access through a single storage node via these ops, with client nodes' constructors in kv-only mode (ADR-011 §4) for pool content. The module supports this shape without change; no fleet-specific op exists.

Design Decisions

ADR	Decision	Summary
001	Substrate posture & placement	alkcall-backed, feature-gated ops module; conformance-not-deviation rationale
006	Verification	verified-fetch via carried digests; no chunk trees
008	Fleet re-routing topology	non-shared-media fleets route large-blob access through a storage node via these ops; no new ops
010	Pin-token renewal	the token is the pin; renewal rides have/re-put; TTL expiry falls to delete-then-recover
012	Wire-shape rulings	put response {token, digest}; have's token-renewal form; token validity domain = minting node; non-fleet token scope generalized
014	Marked-pub put contract	put's data plane = the adopted channel (zero call.published); one-response synthesis (ack-in-result, not ack-before-stream); the alkcall amendment-4 gate; no-stub posture
017	Authority model	CAS global pool — namespace grant is capability-to-ask, not data partitioning; renewal namespace parity; the remote-putter role is read+write
018	Digest wire form's family posture	the ops surface is blob-family-scoped in 0.1 (schema, ingress, put cross-check); a family-explicit wire form is a named-consumer + new-ADR door
011	Vocabulary + modes	instance/node/fleet split; kv-only mode named (the re-routing posture's client shape)

Open Questions

None owned by this document. OQ-09 (default namespace visibility) resolved closed-by-default — recorded in open-questions.md with its reopen condition (a named requirement for open-read-by-default, never mere existence of a deployment).

References

• docs/research/phase-0.md OQ-BL-01 (the substrate settlement + its justification, promoted verbatim into ADR-001 §Context)
• alkcall docs/architecture — ADR-011/012/016/017/021/046/047/050 (the machinery this module rides)
• docs/research/poc-largeblob-findings.md finding A3 (fanout seam)
• ADR-001; store-api.md; gc-and-namespaces.md (namespace-as-resource); ADR-012 §2 (the wire shapes this doc's op payloads carry); ADR-014 (the put's data plane, response synthesis, and the amendment-4 gate)