Files
alkblobs/tasks/ops-fetch.md
T
glm-5.3-flash d927da2ee5 feat(ops): blobs/fetch — the verified Sub op and the fanout seam's consumer side (wave 7)
- the open triple (fetch_openables): the channels/blobs/sub spec with
  the channel_open marker on the alk/blobs ALPN (read action,
  closed-by-default), the awaited establisher whose reply fields carry
  the ack {source, len?, slice_digests?}, and the pump handler over
  the channel's stream (bounded read windows; the fanout feed
  forwarded chunk-for-chunk)
- verified streaming per ADR-006: store-first consult, a live
  broadcast taps (the tap-time-set discipline documented; the
  re-fetch is the partial-feed degrade), the ranged form carries
  read_range's slice-digest convention over exactly the served bytes
- the miss maps onto ADR-049's vocabulary (channel:open_failed,
  reason unknown_resource, the digest in the message; the declared
  BLOB_MISSING contract disclosed on the spec); unverifiable reads
  refuse with handler_error, never stream
- the producer-side design point decided and recorded against the A3
  text: fetches never produce read-side broadcasts (module doc,
  op doc, ops-surface.md)
- five harness tests through the channel halves: the verified whole
  round-trip + consumer hash-check, ranged + ack digests, the
  live-broadcast tap served without a commit, the late joiner served
  by the store path, the typed miss, the closed-by-default denial

Verification: cargo test (default) + --features ops + --all-features
green; clippy -D warnings on all three shapes; fmt --check clean;
cargo doc --no-deps clean on both shapes.
2026-10-13 00:43:52 +00:00

194 lines
11 KiB
Markdown

---
id: ops-fetch
name: blobs/fetch — the verified Sub op and the fanout seam's consumer side
status: completed
depends_on: [ops-foundation, ops-call-harness]
scope: moderate
risk: medium
impact: component
level: implementation
tags: [wave-7, ops, fetch, fanout, verification]
---
## Description
`blobs/fetch` (ops-surface.md §Op family) — the server→client
streaming op: `{namespace, digest, ranges?}` in; **verified bytes
out** over the binary data channel.
- **The op shape**: `OperationType::Sub` (alkcall ADR-021's streaming
handler kind) with the **`channel_open` marker** (alkcall ADR-047)
— the binary data plane rides a `channel_open`-allocated channel;
the establishment phase (alkcall ADR-049) carries the fetch's ack
shape in the `Establishment` reply fields as the wire requires. The
**read** action (ACL exactly like stat/have — closed-by-default).
- **Verified streaming** (ADR-006; ops-surface.md): the *consumer*
hash-checks the received whole against the carried digest —
whole-blob verification; there are no per-chunk or per-range checks
derivable under the canonical digest. The handler streams from the
facade only: `Store::get` (the whole) or `Store::read_range` (the
ranged form). **Ranged fetch** (`ranges?`): the response carries
the same ADR-006 slice-digest convention `read_range` returns —
SHA-256 over exactly the returned bytes, out-of-band.
- **The fanout seam's consumer side** (ops-surface.md; POC #3 finding
A3): the fetch handler consults the foundation's fanout registry —
a live broadcast for the digest → a **subscriber arm** taps it; no
live broadcast, or one that ended (a **late joiner**) → the normal
post-commit verified `get` — identical bytes under CAS. The
slow-subscriber policy (drop-and-late-join) is ops-layer policy,
not a store concern. Named design point: **the producer side's
shape** — the put handler's stream reader as the canonical producer
(serve-while-persisting, A3's shape), and whether fetches also
produce read-side broadcasts (concurrent fetchers of one stored
digest sharing one store read) — decide and record against the A3
text.
- **The miss shape**: a fetch of an absent digest is the typed
missing-error schema (the need announcement *is* the fetch miss —
the have/need diff is caller-side; no separate need op exists).
- **Tests through the harness**: the whole-blob verified round-trip
(byte-exact; the client's hash-check passes); the ranged form
(bytes + slice digest matching `read_range`'s convention); the
fanout consumer side (a live broadcast tapped — the subscriber
receives while the producer runs; a late joiner degrades to the
get path — asserted by which path served); the miss shape; the ACL
denial (no read grant on the namespace → denied).
## Acceptance Criteria
- [ ] fetch exported as a Sub spec/handler pair with the
`channel_open` marker + ALPN; the establishment reply fields
carry the specced ack shape
- [ ] Whole-blob verification: the received whole hash-checks against
the carried digest (the consumer-side discipline documented on
the op; the server streams verified-store bytes)
- [ ] Ranged fetch carries the ADR-006 slice-digest convention
(out-of-band, over exactly the returned bytes)
- [ ] The fanout consumer side: a live broadcast → subscriber arm; a
late joiner → the verified get path; the producer-side design
point decided and recorded
- [ ] The miss shape is the typed missing error; the have/need diff
stays caller-side (no need op invented)
- [ ] ACL: the read action; closed-by-default verified by test
- [ ] Handlers touch only the store facade (no backend side-door)
- [ ] Harness tests green; `cargo test` (default) green;
`cargo test --features ops` green; clippy/fmt green on both
shapes; `cargo doc --no-deps` clean
## References
- docs/architecture/ops-surface.md (§Op family — the fetch bullet +
the fanout text; the need-half note)
- docs/architecture/decisions/006-verification-posture-and-transfer-encoding.md
(whole-blob verification; the slice-digest convention)
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
§3 (the re-routing topology this op serves — large-blob access
through a storage node)
- docs/research/poc-largeblob-findings.md finding A3 (the fanout
seam: one reader, store arm + subscriber arms; late joiners
degrade to get-after-commit)
- alkcall (upstream checkout): ADR-021 (streaming handler for
subscriptions), ADR-046/047/049 (the sink kind, the channel_open
marker, the establishment phase), `src/channels/client.rs`
- tasks/ops-foundation.md (the fanout registry's mechanics),
tasks/ops-call-harness.md (the harness)
## Notes
Implementation agent, 2026-10-13. The op landed as `fetch_openables(ctx)
-> (OperationSpec, OpenEstablisher, OpenHandler)` in `src/ops.rs` —
the `Sub` open triple the embedder assembles through
`ChannelCore::register_openable_with_establisher` (spec
`channels/blobs/sub`, `OperationType::Sub`, `channel_open` marker on
the `alk/blobs` ALPN `BLOBS_CHANNEL_ALPN`; the standard shape, so the
ALPN derives from the name and no explicit `channel_open_alpn` rides).
Exports: `fetch_openables`, `BLOBS_CHANNEL_ALPN`, `FETCH_OP_ID`.
Design points decided and recorded (op doc + the module doc's fetch
section + ops-surface.md's fetch bullet):
- **The ack rides ADR-049 amendment 3's establishment reply fields**:
the open reply merges `{channel_id, source, len?, slice_digests?}`.
`source` names the serving path (`store`/`fanout` — the test
criterion "asserted by which path served" is wire-visible, and a
consumer knows whether the stream is a live put's tail); the whole
form acks `{source: "store", len}` or `{source: "fanout"}` (no
length on a tapped broadcast — the hash-check runs at stream end);
the ranged form acks `{source: "store", slice_digests[]}` —
read_range's convention verbatim, one per range in request order,
over exactly the served bytes (over-reads clamp; the digest covers
the clamp).
- **The establisher's consult order**: store `get` first (a commit
that raced the broadcast's end is the same bytes under CAS; the
Blob opens into the plan at establishment, closing the
establish→pump race window), then the fanout subscribe (a live
broadcast taps), then the miss. The ranged form serves
`read_range` on the committed entry only — a not-yet-committed
digest's ranged fetch is the miss (ranges are the committed entry's
shapes; no tap-window feed serves them). The ranged form's reads
happen at establishment because the slice digests must ride the
ack, which lands before the pump runs.
- **The miss maps onto ADR-049's established vocabulary** — the only
op-plane failure surface an open op has: the establishment phase
refuses with reason `unknown_resource` ("the requested resource
does not exist"), the digest in the refusal's message; the
declared `BLOB_MISSING` ErrorDefinition (same code + digest-details
schema the JSON ops declare) is the op's disclosed missing-error
contract. `channel:open_failed` + `details.reason` is what a
consumer branches on; a phantom channel with an in-band error frame
(the ADR-049 antipattern) is never reachable. The same closure
covers the refusal-to-serve-unverifiable shape: a store-core error
mid-establishment (including a verification-failed read) refuses
with reason `handler_error` rather than streaming unverified bytes.
- **The producer-side design point (decided, against the A3 text)**:
fetches do NOT produce read-side broadcasts — the put handler's
stream reader is the canonical producer, concurrent fetchers of one
stored digest each ride their own facade read. Recorded in the
module doc's fanout section (a named design point paragraph), the
op doc, and ops-surface.md's fetch bullet. Rationale: A3's fanout
is the put stream's seam (its late-joiner note degrades to get,
never get-then-rebroadcast); a read-side broadcast adds a second
producer class with no lifecycle owner (its end would be an
arbitrary policy, its arms hold memory where the large tier exists
to avoid materializing), for a concurrency case bounded by the put
window the put-side broadcast already covers; under CAS every
per-fetcher read returns identical bytes.
- **Slow-subscriber policy stays producer-owned** (per-broadcast, set
by the put at `begin` — the foundation's registry semantics); a
fetch tap rides the broadcast's policy. The tap-time-set discipline
(a mid-put tap has no prefix) is documented on the op + registry:
the consumer's hash-check on a partial feed fails and the re-fetch
is the degrade, identical bytes post-commit.
- **A mid-stream note for the whole form**: between a store-hit
establishment and the pump, the entry cannot vanish (the Blob read
is already open; committed entries are immutable), so the ack's
`len` is trustworthy; a mid-stream store/transport failure ends the
pump (teardown closes the stream), the consumer's hash-check fails,
and the re-fetch is the degrade.
Verification: `cargo test` (default) green, `cargo test --features ops`
green (the five harness tests: verified whole round-trip + consumer
hash-check; ranged + ack slice digests vs `read_range`; live-broadcast
tap served without a commit; late joiner served by the store path; the
typed miss; the closed-by-default read denial), `--all-features` green,
clippy `-D warnings` on all three shapes, `cargo fmt --check` green,
`cargo doc --no-deps` clean on both shapes (the private
`FETCH_WINDOW` doc link de-linked, the PGLO-1 class). The pure
`--no-default-features --features ops` build fails on main already
(engine-typed match arms upstream of this task, unreachable without an
engine feature) — pre-existing, not this task's regression.
## Summary
`blobs/fetch` landed complete: the verified `Sub` op as the
`fetch_openables` open triple (`channels/blobs/sub` + the
`channel_open` marker on `alk/blobs`; read-gated, closed-by-default),
the establishment ack in ADR-049's reply fields
(`{source, len?, slice_digests?}`), whole-blob consumer-side
verification per ADR-006, the ranged form carrying `read_range`'s
slice-digest convention, the fanout seam's consumer side (live tap /
late-joiner store degrade, the path named in the ack), the miss as the
typed missing-error refusal (BLOB_MISSING disclosed; the wire shape is
ADR-049's `unknown_resource`), and the producer-side design point
decided and recorded against the A3 text (fetches never produce
read-side broadcasts). All acceptance criteria verified; harness tests
green; all gates green on both feature shapes.