alkblobs/docs/architecture/decisions/006-verification-posture-and-transfer-encoding.md

ADR-006: Verification posture — whole-blob checks; transfer encodings excluded by scoping

Status

Accepted

Context

OQ-BL-04's settled Phase 0 posture: whole-file CAS as the default granularity; per-range verification impossible under the canonical digest (the preamble hashes the length — POC #3 finding A2, independently reproduced in the iroh-blobs eval; no slice has any hash relationship to the whole); the chunk-tree encoding framed as "a transfer-layer conditional — deferred until a consumer exists".

That last framing is a deferral this crate cannot make under its own deferral policy: "the consumer that would want networked verified ranged fetch" (p2p git's replicator) is paused because this crate is being built. A condition gated on it is Schrödinger's code — it never collapses. The decision must stand on the evidence in hand, which is complete for the scoping question even though nothing measures a future consumer:

• What the consumers need today is verifiable whole-blob transfer: fetch bytes, hash-check against the canonical digest the caller carries. That is exactly verified-fetch (the Sub op with the digest in the offer), works at any size, and needs no encoding — POC #3's streaming paths are validated byte-exact against git.
• What would need a chunk tree is per-chunk integrity during ranged transfer of one large blob — a property nothing in the current consumer set requires, and which cannot be retrofitted onto the canonical digest anyway (impossible under the preamble, finding A2; iroh gets it from the bao tree, not BLAKE3 the algorithm).
• Storing chunk trees would add a second stored artifact per blob (outboard encoding), a second write path, new GC surface, and a second hash domain — speculative cost before any consumer names the need.

Also settled here: local range reads need only transport-integrity slice digests (out-of-band, carried by the caller — the store returns one from read_range); git objects self-verify under git's own model when read whole; small-tier range reads slice whole values in memory.

Decision

1. Verification is whole-blob. Put and get paths hash-check against the canonical derivation (ADR-002). stat probes length without reading content.
2. Range reads return (slice, slice_digest) — the slice digest is a transport-integrity check carried out-of-band by the caller (e.g. the offering side of a fetch). The store performs no per-range verification because none is derivable under the canonical digest; local serving (packfiles, large-file reads) is sound on honest media (finding A2).
3. No chunk-tree, CDC, or otherwise partial-verification encoding is adopted in this crate — by scoping, not by deferral. The crate's domain is the store; a transfer encoding is a network-layer artifact and has no owner here. If a future networked consumer (a p2p replicator above an un-paused alkgit, alkfs sync, a future ops extension) names a requirement for verified ranged fetch of one large blob, that requirement is met by a new decision at a new layer: an optional encoding module (BLAKE3-confined per ADR-002's tier 3 if a tree encoding is chosen; content-defined chunking if edit-resistance is the goal) whose verified whole-contents register back into the pool as ordinary canonical entries — dedup between chunked and whole-file paths is preserved by registration, not by shared structure.
4. The verified-fetch op (ADR-001) carries whole-blob verification as its integrity story: digest in the offer, hash-check on receipt, fanout above the store for concurrent subscribers (POC #3 finding A3); late joiners degrade to post-commit verified get.

Consequences

Positive

• The store stays single-granularity, single-hash-domain, single write-path family — the simplest shape consistent with all consumers' current needs; no speculative outboard/GC surface.
• Verified transfer exists at every blob size with zero extra machinery (whole-blob digest check), which is what the confirmed ops family serves.

Negative

• A future large-blob verified-ranged-fetch consumer pays a new module and a second decision — deliberately deferred cost, not unmade decision (the distinction this ADR records).
• Callers wanting mid-transfer corruption localization on very large blobs get error-at-EOF, not error-at-chunk, without an encoding layer (acceptable; named above).

Neutral

• BLAKE3's "large-blob encoding consideration" stays conditional and out-of-crate; nothing here allocates an API slot for it.

References

• docs/research/phase-0.md OQ-BL-04 (posture + surface addendum promoted here and to ADR-003/store-api)
• docs/research/poc-largeblob-findings.md findings A2/A3
• docs/research/iroh-blobs-eval.md §Verification
• ADR-001 (ops surface; verified-fetch op), ADR-002 (digest + preamble consequence), ADR-005 (range reads in packfile serving)
• store-api.md; ops-surface.md