- Public `(len, stream)` shape: `Store::get` returns
`Option<(u64, Blob)>`; `Blob` (src/facade.rs) is a bounded, re-entrant
windowed reader over the new crate-internal `AnyCursor` tier dispatch
(mem materializing / local pread body families) — engine cursors
never leak onto the surface; no chunk-iterator machinery.
- Verification-on-get pinned against ADR-006: kv-tier materialized
reads whole-blob hash-check at get time (mismatch = Err(Verification),
never a further fall-through); the large tier's streaming open is
metadata + handle only (put-path verification + slice digests are its
story); `read_range` carries slice digests only, on both tiers.
- `stat` rides the trait's `size` probe (never a content load) into
`EntryMeta { len, algorithm }`; `has` is the pure existence probe;
both fall through kv -> large like every key-addressed op (ADR-012
§6.1) — consult order, not routing (never consults `tier_for`).
- Miss shapes enforced (the error module's behavioral note): both-tier
misses are Ok(None) on get/stat, false on has, Err(Missing) on
read_range; `src/error.rs` wave-2 deferral notes updated to enforced
state.
- `list` decided Vec<Key>: materialized, sorted total-order union of
both tiers (the sweep consumes the engines' enumeration directly);
malformed pool bytes are fatal `Err(KeyInvalid)`, never a silent
skip.
- `KeyInvalid` at the facade boundary: ops take opaque key bytes and
convert via `Key::from_bytes` once (`boundary_key`); engines never
see typed keys.
- Twelve read-path tests: virgin-store shapes per mode (dual-tier,
mem-only, kv-only), the `(len, stream)` windows, fall-through,
kv-first (decoy content in the large tier), both verification
postures, list union/sort/pool-fatality, and malformed-key
rejection on every op.
Task: tasks/store-read-paths.md (design points recorded in Notes for
the wave-2 review gate).
Verification: cargo test (default), cargo test --features mem, cargo
test --all-features green; cargo clippy --all-targets -- -D warnings
green in all three feature shapes; cargo fmt --check green; cargo doc
--no-deps clean beyond the pre-existing private-intra-doc-links class.
180 lines
9.9 KiB
Markdown
180 lines
9.9 KiB
Markdown
---
|
||
id: store-read-paths
|
||
name: Store read paths — get/stat/read_range/has/list with dispatch fall-through
|
||
status: completed
|
||
depends_on: [store-construction]
|
||
scope: moderate
|
||
risk: medium
|
||
impact: component
|
||
level: implementation
|
||
tags: [wave-2, facade, read-path]
|
||
---
|
||
|
||
## Description
|
||
|
||
Implement the facade's read-side ops on the `Store` type (store-api.md
|
||
§Read path, §Lifecycle), behind the dispatch seam from
|
||
`store-construction`. Direct `delete` is *not* here — it lands in
|
||
`gc-protocol` with its arbitration.
|
||
|
||
- **The fall-through general rule** (ADR-012 §6.1; POC #1 finding 6):
|
||
every key-addressed op consults the kv tier first, then the large
|
||
tier. `get`/`stat`/`has`/`read_range`/`list` all follow it.
|
||
- **`get(key) → Option<(len, stream)>`**: small tier returns a bounded
|
||
read; large tier streams. The per-engine cursor shapes stay hidden
|
||
behind the crate-internal `ReadCursor` the store core drives
|
||
(ADR-010 §1) — **named design point**: the public stream type that
|
||
wraps the cursor (the facade's `(len, stream)` shape is uniform
|
||
across tiers; the cursor is never exposed). A miss on both tiers is
|
||
`None` — `get` never carries `Error::Missing` (ADR-012 §6.5 ruling 5;
|
||
the error module's behavioral-enforcement note lands here).
|
||
- **`stat(key) → Option<EntryMeta>`**: the cheap probe riding the
|
||
trait's `size` method — length plus algorithm, never a content load
|
||
(ADR-008's size discipline). Fall-through per the general rule.
|
||
- **`read_range(key, range) → (slice, slice_digest)`**: slice plus the
|
||
out-of-band SHA-256 slice digest (ADR-006, fixed by convention —
|
||
composed through the cursor's `read_range` seam). `Err(Missing)` on a
|
||
both-tier miss. Range reads fall through tiers exactly as `get` does.
|
||
- **`has(key) → bool`**: the pure existence probe (ADR-008's `has`
|
||
discipline) — a size-probe under the fall-through rule; absent on
|
||
both tiers is `false`; virgin-store reads are no-ops.
|
||
- **`list() → stream of keys`**: whole-pool enumeration, complete by
|
||
contract. **Named design point**: the return shape — store-api.md
|
||
says "stream of keys"; decide Vec vs stream (the sweep consumes the
|
||
same enumeration crate-internally) and record it.
|
||
- **Verification posture on get** (named design point): ADR-006's
|
||
whole-blob verification vs the slice-digest-only reading — where the
|
||
store core's hash-check actually fires on the read path (kv
|
||
materialized reads are verifiable cheaply; large streaming reads ride
|
||
per-range slice digests). Pin the reading against ADR-006's text and
|
||
record it for the review gate.
|
||
- **`KeyInvalid` at the boundary**: typed `Key` conversion happens at
|
||
the store boundary only (ADR-002/003) — malformed key bytes on any
|
||
read op reject with `Error::KeyInvalid`, never a guess; engines never
|
||
see typed keys.
|
||
|
||
## Acceptance Criteria
|
||
|
||
- [x] `get`/`stat`/`read_range`/`has`/`list` implemented with the
|
||
fall-through general rule (kv first, then large), unit-tested
|
||
including the both-tier-miss shapes (`None` vs `Err(Missing)` per
|
||
op)
|
||
- [x] The public stream type decided and recorded; engine cursors never
|
||
leak into the public surface
|
||
- [x] `stat` never fetches content (rides `size`); `EntryMeta` carries
|
||
len + algorithm
|
||
- [x] `read_range` returns slice + SHA-256 slice digest per the ADR-006
|
||
convention; verification-on-get posture decided and recorded
|
||
- [x] Malformed key bytes reject with `KeyInvalid` at the facade
|
||
boundary on every key-addressed op
|
||
- [x] `cargo test` green; `cargo test --features mem` and `cargo test
|
||
--all-features` green; clippy/fmt gates green
|
||
|
||
## References
|
||
|
||
- docs/architecture/store-api.md (§Read path, §Lifecycle, §Error model)
|
||
- docs/architecture/backends-and-dispatch.md (§Size-threshold dispatch — get fall-through)
|
||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §6.1/§6.5
|
||
- docs/architecture/decisions/006-verification-posture-and-transfer-encoding.md
|
||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md (size discipline)
|
||
- src/backend.rs (the ReadCursor seam this drives)
|
||
|
||
## Notes
|
||
|
||
- **The public stream type decided: `Blob` (in `src/facade.rs`) — a
|
||
bounded, re-entrant windowed reader; `get` returns the pinned
|
||
`(len, stream)` shape as `Option<(u64, Blob)>`.** The engine cursor
|
||
bodies (mem's materializing body, `local`'s pread body) stay behind
|
||
a crate-internal tier-dispatch enum `AnyCursor` (`src/store.rs`)
|
||
implementing `ReadCursor`; `Blob` exposes `len()`/`is_empty()`,
|
||
`read_at(offset, len)` (pread-style clamped windows), and
|
||
`read_range(offset, len)` (window + ADR-006 slice digest), constructed
|
||
only by the store core. No chunk-iterator machinery was adopted: the
|
||
cursor bodies *are* the streaming abstraction (large tier reads
|
||
window-by-window over the committed file without ever materializing
|
||
the whole; kv is bounded by materialization policy) — a per-chunk
|
||
`Stream` wrapper would re-shape them without changing the underlying
|
||
reads. Recorded in `src/store.rs` module docs + `Blob`'s doc.
|
||
- **Verification-on-get posture (pinned against ADR-006's text):** the
|
||
whole-blob hash-check fires where the whole content is already in
|
||
hand — the kv tier's materializing reads (ADR-006 decision 1: "put
|
||
and get paths hash-check"): kv get materializes, re-derives the
|
||
canonical digest over the stored bytes, compares against the key;
|
||
mismatch is `Err(Verification)` and never falls through further (a
|
||
verified mismatch is a pool-integrity fact; consulting another tier
|
||
past it would be a guess). The large tier's streaming get does *not*
|
||
re-derive at open (metadata + handle only) — its integrity story is
|
||
the put path's whole-blob verification pass plus the per-range slice
|
||
digests (ADR-006 decisions 1–2; §Context's "git objects self-verify
|
||
under git's own model when read whole"). `read_range` carries no
|
||
canonical check on either tier (decision 2: no per-range verification
|
||
is derivable). Pinned by tests on both tiers
|
||
(`kv_get_whole_blob_verifies_and_never_falls_through_past_mismatch`,
|
||
`large_streaming_get_does_not_re_derive_at_open`).
|
||
- **`list` decided: `Vec<Key>`** — whole-pool enumeration returned
|
||
materialized and sorted in the keys' total order (deterministic,
|
||
typed, both algorithms). The sweep consumes the same crate-internal
|
||
enumeration directly from the engines (its live-set diffs sort
|
||
independently), so the facade shape serves consumers, not internal
|
||
needs; a lazy stream was rejected because every shipped engine
|
||
enumerates fully and eagerly (the completeness contract *is* the
|
||
GC-safety property) — a pseudo-lazy wrapper adds machinery without
|
||
changing the underlying walk. Listed bytes failing `Key::from_bytes`
|
||
reject `Err(KeyInvalid)`: a malformed pool list is fatal, never a
|
||
silent skip (ADR-003's malformed-list ruling applied at the facade
|
||
boundary).
|
||
- **`KeyInvalid` at the boundary:** the read ops take opaque key
|
||
*bytes* (`&[u8]`) and convert via `Key::from_bytes` at the store
|
||
boundary (`boundary_key`, one seam in `src/store.rs`) — malformed
|
||
bytes reject `Error::KeyInvalid` on get/stat/read_range/has before
|
||
any tier is consulted and never as a `None`/`false` guess; engines
|
||
receive only the physical encoding. `list()`'s pool decode is the
|
||
same one-way door's read face (above).
|
||
- **Fall-through is consult order, not routing:** the read path never
|
||
consults `tier_for`/`route` — a sub-threshold entry planted only in
|
||
the large tier is found (test
|
||
`read_ops_fall_through_kv_miss_to_large` asserts the *routing*
|
||
disagrees while the read succeeds to make the distinction
|
||
observable). kv-first is proven behaviorally by planting different
|
||
content under the same key bytes in both tiers
|
||
(`kv_tier_is_the_first_consult_on_every_key_addressed_op`).
|
||
- **Miss shapes enforced (the error module's behavioral note):**
|
||
both-tier misses are `Ok(None)` on get/stat, `false` on has,
|
||
`Err(Missing)` on read_range — pinned per mode (dual-tier/mem-only/
|
||
kv-only) via `assert_virgin_read_shapes`; `src/error.rs`'s
|
||
wave-2 deferral notes updated to the in-place state.
|
||
- **Tier dispatch plumbing:** `KvEngine`/`LargeEngine` grew
|
||
crate-internal `get`/`has`/`size`/`list` dispatch methods returning
|
||
engine-raw shapes (cursor enum / `Option<u64>` / `bool` /
|
||
`Vec<Vec<u8>>`); adding a variant per enum (waves 3–5) is slotting
|
||
in. `get`'s kv arm (and its verification) is
|
||
`#[cfg(feature = "mem")]` — in default builds (no kv engine until
|
||
wave 3's sqlite) the large/None arms fully cover the unconstructible
|
||
facade.
|
||
|
||
## Summary
|
||
|
||
Implemented the facade's read-side ops on `Store` (`src/store.rs`):
|
||
`get(key) → Result<Option<(u64, Blob)>>` with the pinned `(len, stream)`
|
||
shape, `stat(key) → Result<Option<EntryMeta>>` riding the trait's
|
||
`size` probe, `read_range(key, offset, len) → Result<(Vec<u8>, [u8;32])>`
|
||
composed through the cursor's `read_range` slice-digest seam, `has(key)`
|
||
as the pure existence probe, and `list() → Result<Vec<Key>>` — all on
|
||
the kv-first-then-large fall-through rule with typed key conversion at
|
||
the facade boundary. Added the public stream type `Blob`
|
||
(`src/facade.rs`) over the new crate-internal `AnyCursor` cursor
|
||
dispatch and recorded the four named design points (stream shape,
|
||
verification-on-get, list return shape, `KeyInvalid` boundary) in
|
||
`src/store.rs` module docs; updated the wave-2 deferral notes in
|
||
`src/error.rs` to their enforced state and the scope notes in
|
||
`src/mem.rs`/`src/local.rs`/`src/dispatch.rs` now that the read path
|
||
drives them. Twelve read-path tests pin the miss shapes, the
|
||
fall-through order, both verification postures, the list union
|
||
(including malformed-pool fatality), and `KeyInvalid` on every
|
||
key-addressed op.
|
||
|
||
Verification: `cargo test` (default), `cargo test --features mem`,
|
||
`cargo test --all-features` all green (66 lib tests green under mem);
|
||
`cargo clippy --all-targets -- -D warnings` green in all three feature
|
||
shapes; `cargo fmt --check` green; `cargo doc --no-deps` clean of new
|
||
warning classes beyond the pre-existing private-intra-doc-links class. |