alkblobs/docs/architecture/ops-surface.md

status, last_updated

status	last_updated
draft	2026-10-03 (ADR-012 §2 — put response {token, digest}; have token-renewal form; fetch verification wording; non-fleet token scope)

Ops surface (alkcall-backed network operations)

What this is

The network-facing operation family over the store: have/need announcements, verified fetch, and streaming put — riding the alkcall substrate behind a feature-gated ops module in this crate (feature ops, default-off; ADR-001). The base crate stays lean and alkcall-free; enabling ops adds the alkcall dependency and registers the op family.

Why alkcall (ADR-001, the short form)

The op family is exactly alkcall's mixed-shape sweet spot (validated machinery, not parallel invention):

• JSON control plane — have/need announcements, stat-shaped probes, offer/carry lengths: alkcall OperationSpec + JSON Schema validation (alkcall ADR-016) + typed error schemas + from_call discovery + External/Internal visibility.
• Binary data plane — bulk bytes one way (fetch: Sub, alkcall ADR-021) or the other (put: Pub/Sink, alkcall ADR-046), allocated via the channel_open marker (alkcall ADR-047); established pump patterns (pump_bidi, alkcall ADR-050).
• ACL is solved there — AccessControl + ownership checks (alkcall ADR-011) under the ADR-017 privilege model map onto namespace gating directly; an in-band invented scheme would be a second, unreviewed authorization story in the family (AGENTS convention 6).

Producer/consumer vocabulary throughout; no transport enters either layer — the caller (an embedder, alkgit's replicator, a future alkfs sync) dials and hands an established Connection in (alkcall is a pure protocol crate; ADR-012 there).

Op family (WHAT is exposed)

Namespaces on the wire. Every ops payload carries an explicit namespace field — the consumer-scoped identifier the embedder's registry recognizes (syntax: non-empty UTF-8 string, embedder-validated; the minting of namespaces is the embedder's act — e.g. alkgit's registry maps repo ids to namespaces, alkfs maps workspace roots). The store core never sees this string (backends are namespace-blind, ADR-005); it selects the liveness source and the ACL resource. The ops module's job is to bind namespace → the embedder-registered liveness source for pin conversion, and to present the namespace as the alkcall ACL resource.

JSON control ops (Visibility per ADR-001 §Decision):

• blobs/stat {namespace, digest} → {len} — probe before offering; ACL-gated like fetch (read action on the namespace)
• blobs/have {namespace, digests[], token?} → {present[]} — the have half of have/need set diffing (hashes only — content never traverses this op); read-gated: an existence probe over arbitrary digests is a discovery surface, so it is gated exactly like fetch, not public. present[] corresponds to digests[] by index. The optional token is the renewal form (ADR-012 §2): when present, exactly one digest accompanies it — the handler verifies the token maps to that digest's pin row, renews it (crate-internal pin_renew via the ops module's store-core access, ADR-010 §4), and answers with present[] as usual; an expired/foreign row answers present: false (the putter's backstop is re-put). Without token, have remains a pure existence probe (ADR-008's discipline).
• blobs/delete {namespace, digests[]} — operator machinery; Visibility::Internal, evaluated under the internal authority context per alkcall ADR-017 (internal calls switch authority context, never skip ACL): requires the embedder's operator/admin authority context, not a namespace grant. The namespace field exists for audit/triage scoping (which slice of the pool the deletion targets), not for gating.

Binary channel ops (registered via the channel_open marker):

• blobs/fetch (Sub — the server→client streaming op shape) — {namespace, digest, ranges?} in; verified bytes out: the consumer hash-checks the received whole against the carried digest (ADR-006's whole-blob verification — there are no per-chunk or per-range checks derivable under the canonical digest; ranged fetch responses carry the same ADR-006 slice-digest convention read_range returns). Broadcast fanout above the store: one reader, store arm + subscriber arms (POC #3 finding A3); late joiners degrade to a normal post-commit get — identical bytes under CAS. Slow-subscriber policy (drop-and-late-join) is ops-layer policy, not a store concern.
• blobs/put (Sink — the client→server streaming op shape) — {namespace, digest?, len?} offer + byte stream in; server verifies against the canonical derivation before commit. Response: {token, digest} (ADR-012 §2) — the pin token (§Pin hand-over below) and the stored entry's canonical digest (hex), which an unknown-length putter needs for every later op. A known-length offer is the encouraged path (one- pass); unknown-length rides the store's pre-threshold buffering path (ADR-003). Need half of have/need: a fetch miss is the need announcement — the consumer computes its need set by diffing blobs/have results and fetches the absent digests; no separate need op exists (the diff is caller-side; the wire carries only concrete fetch requests).

Pin hand-over for remote puts (the cross-wire contract). The server-side put handler owns the Pin guard; the offering side's reference registration happens in the remote process, invisible to the server. The hand-over contract:

1. the put lands pinned (ADR-005) — the entry cannot be swept while the handler holds the pin;
2. the blobs/put response returns a pin token in the specced response shape {token, digest} (ADR-012 §2 — opaque handle minted by the ops layer over the pin row the put committed; the token↔(key, owner) mapping is ops-layer state, not durable state — the durable state is the pin row itself, ADR-010 §4); the putter can later confirm its registration landed by observing the digest present via blobs/have;
3. the embedder's registered liveness source is the actual root of record: the contract on the embedder (documented on the ops module) is that it registers the namespace's roots such that a digest referenced by a namespace's manifest is either (a) already registered before the caller's next sweep could run on the serving node, or (b) protected by holding the pin token until its registration is confirmed (blobs/have returns present after registration). In short: the remote putter may not rely on an unconfirmed put surviving the next sweep — it holds its pin token until its registration shows up in blobs/have. A Pin conversion on the server side (token → registered liveness source) is the mechanism the embedder plugs its registry into.

Token renewal (ADR-010 §4, wired per ADR-012 §2 — the fleet-TTL race, resolved): the token is the pin — there is no second pin state. The server-side handler's in-process Pin guard drops when the blobs/put response completes (if it never dropped, the TTL machinery would be dead on the ops path — the pin row's expiry, not the guard's presence, is the fleet liveness signal). Under fleet engines the put's pin is an expiry-carrying row (ADR-008: TTL 60 s default, renewed every TTL/3). The remote putter holding a token is the row's owner and renews via the renewal form of blobs/have (ADR-012 §2: one digest, token alongside — the handler verifies the token maps to the digest's pin row and calls the crate-internal pin_renew via the ops module's store-core access — the ops module is in-crate and crosses the seam by construction, the same way the put handler holds the Pin guard; has itself remains a pure existence probe per ADR-008 — renewal is a token-authenticated act on the pin row, not a has semantic), or a re-put of the same digest (CAS dedup renews the row). A putter may pause past the TTL/3 mark (20 s at the default TTL) only if it renews before expiry; a putter pausing past TTL lets the row expire — the renewal probe answers present: false, the sweeper reaps the row (ADR-008), and delete-then-recover (ADR-005) is the backstop: the putter re-offers the digest; CAS makes the re-put idempotent. The race resolves to a re-put, never to silent loss. Token validity domain: the minting serving node (ADR-012 §2) — the token↔(key, owner) mapping is ops-layer in-memory state in the node that minted it (process-lifetime, outliving the handler); renewal probes must reach that node, fleet peers refuse tokens they did not mint (present: false, not an error), and a minting node's death before renewal is ADR-005's in-flight-put loss case with the same re-put backstop — a documented embedder duty (route renewals to the serving node). For non-fleet serving nodes (any engine — sqlite, single-instance postgres, local, pg-lo; ADR-012 §2) the token is the in-process guard's handle — nothing expires; the process-crash case is ADR-005's in-flight-put loss with the same backstop.

Placement of the op registration (which embedders wire where they want them exposed) matches the alkgit ops pattern (its git/repo/* call ops): the module exports spec/handler pairs; the embedder assembles.

ACL mapping

Resources and actions, evaluated by alkcall's AccessControl machinery at op entry — this module never re-implements an authorization check (convention 6; ADR-001):

• Namespace = resource. resource_type: "blob-namespace", resource_id_path selecting the payload's namespace field; actions map onto read/put/manage:
  • blobs/fetch, blobs/stat, blobs/have — the read action; unlisted namespaces (no ACL entries) deny by default (OQ-09's resolution; a grant per namespace is the deferred cost, deliberately paid over accidental exposure)
  • blobs/put — the write action
  • blobs/delete — not namespace-gated at all; internal authority context only (alkcall ADR-017: internal switches context, never skips ACL)

Store-facing relationship

The ops module is a consumer of the store core (store-api.md), not a side channel: fetch handlers call get/read_range/stat; put handlers call put; both honor pins and delete windows. There is no ops-internal path around the store facade (ADR-001 §Consequences — this is what makes "re-homing the ops surface later loses nothing" true). One fleet topology additionally rides this module: re-routing (ADR-008) — a fleet whose nodes do not share large-tier media routes large-blob access through a single storage node via these ops, with client nodes' constructors in kv-only mode (ADR-011 §4) for pool content. The module supports this shape without change; no fleet-specific op exists.

Design Decisions

ADR	Decision	Summary
001	Substrate posture & placement	alkcall-backed, feature-gated ops module; conformance-not-deviation rationale
006	Verification	verified-fetch via carried digests; no chunk trees
008	Fleet re-routing topology	non-shared-media fleets route large-blob access through a storage node via these ops; no new ops
010	Pin-token renewal	the token is the pin; renewal rides have/re-put; TTL expiry falls to delete-then-recover
012	Wire-shape rulings	put response {token, digest}; have's token-renewal form; token validity domain = minting node; non-fleet token scope generalized
011	Vocabulary + modes	instance/node/fleet split; kv-only mode named (the re-routing posture's client shape)

Open Questions

None owned by this document. OQ-09 (default namespace visibility) resolved closed-by-default — recorded in open-questions.md with its reopen condition (a named requirement for open-read-by-default, never mere existence of a deployment).

References

• docs/research/phase-0.md OQ-BL-01 (the substrate settlement + its justification, promoted verbatim into ADR-001 §Context)
• alkcall docs/architecture — ADR-011/012/016/017/021/046/047/050 (the machinery this module rides)
• docs/research/poc-largeblob-findings.md finding A3 (fanout seam)
• ADR-001; store-api.md; gc-and-namespaces.md (namespace-as-resource); ADR-012 §2 (the wire shapes this doc's op payloads carry)