feat(store): the read paths — get/stat/read_range/has/list on the fall-through rule
- Public `(len, stream)` shape: `Store::get` returns
`Option<(u64, Blob)>`; `Blob` (src/facade.rs) is a bounded, re-entrant
windowed reader over the new crate-internal `AnyCursor` tier dispatch
(mem materializing / local pread body families) — engine cursors
never leak onto the surface; no chunk-iterator machinery.
- Verification-on-get pinned against ADR-006: kv-tier materialized
reads whole-blob hash-check at get time (mismatch = Err(Verification),
never a further fall-through); the large tier's streaming open is
metadata + handle only (put-path verification + slice digests are its
story); `read_range` carries slice digests only, on both tiers.
- `stat` rides the trait's `size` probe (never a content load) into
`EntryMeta { len, algorithm }`; `has` is the pure existence probe;
both fall through kv -> large like every key-addressed op (ADR-012
§6.1) — consult order, not routing (never consults `tier_for`).
- Miss shapes enforced (the error module's behavioral note): both-tier
misses are Ok(None) on get/stat, false on has, Err(Missing) on
read_range; `src/error.rs` wave-2 deferral notes updated to enforced
state.
- `list` decided Vec<Key>: materialized, sorted total-order union of
both tiers (the sweep consumes the engines' enumeration directly);
malformed pool bytes are fatal `Err(KeyInvalid)`, never a silent
skip.
- `KeyInvalid` at the facade boundary: ops take opaque key bytes and
convert via `Key::from_bytes` once (`boundary_key`); engines never
see typed keys.
- Twelve read-path tests: virgin-store shapes per mode (dual-tier,
mem-only, kv-only), the `(len, stream)` windows, fall-through,
kv-first (decoy content in the large tier), both verification
postures, list union/sort/pool-fatality, and malformed-key
rejection on every op.
Task: tasks/store-read-paths.md (design points recorded in Notes for
the wave-2 review gate).
Verification: cargo test (default), cargo test --features mem, cargo
test --all-features green; cargo clippy --all-targets -- -D warnings
green in all three feature shapes; cargo fmt --check green; cargo doc
--no-deps clean beyond the pre-existing private-intra-doc-links class.
This commit is contained in:
1 parent
532adfa707
commit
f1f505a910
7 files changed
+935
-46
No files matched your search
+5
-2
@@ -21,8 +21,11 @@
|
||||
//! own modes.
|
||||
//!
|
||||
//! The `#![allow(dead_code)]` below marks the seam's scope: `route` is
|
||||
//! driven by the facade's write path (store-put-paths) and consulted by
|
||||
//! the facade's read paths; the allow shrinks as those land.
|
||||
//! driven by the facade's write-side tier selection (`Store::tier_for`),
|
||||
//! whose own caller is the put seam (store-put-paths); the allow shrinks
|
||||
//! as those land. Read-side ops do not consult `route` — the
|
||||
//! fall-through consult order is the facade's (kv first, then large;
|
||||
//! ADR-012 §6.1).
|
||||
|
||||
#![allow(dead_code)]
|
||||
|
||||
|
||||
+11
-9
@@ -26,9 +26,10 @@ pub enum Error {
|
||||
/// store-api.md §Error model).
|
||||
///
|
||||
/// A miss on `get` and `stat` is a value, not an error — they
|
||||
/// signal absence with `None` and never carry `Missing`
|
||||
/// (behavioral enforcement lands with the facade in wave 2; this
|
||||
/// doc comment states the contract here).
|
||||
/// signal absence with `None` and never carry `Missing`. The
|
||||
/// facade's read paths enforce this behaviorally: both-tier misses
|
||||
/// resolve `Ok(None)` on get/stat (see `src/store.rs`'s read-path
|
||||
/// tests; ADR-012 §6.5 ruling 5; store-api.md §Error model).
|
||||
#[error("key missing (read_range/delete: get/stat signal absence with None)")]
|
||||
Missing,
|
||||
|
||||
@@ -96,8 +97,9 @@ mod tests {
|
||||
/// both altitudes — the trait boundary's effective members plus
|
||||
/// the facade re-exposure with `GcRefuse` added, carried by doc
|
||||
/// comments and construction discipline rather than two types.
|
||||
/// Facade behavioral enforcement (`get`/`stat` never carry
|
||||
/// `Missing`; `Missing`/`KeyInvalid` exclusions) lands in wave 2.
|
||||
/// Facade behavioral enforcement is in place on the read paths
|
||||
/// (`get`/`stat` never carry `Missing` — their both-tier misses
|
||||
/// are `Ok(None)`, pinned by the store.rs read-path tests).
|
||||
#[test]
|
||||
fn one_enum_both_altitudes() {
|
||||
let trait_effective: [Error; 4] = [
|
||||
@@ -118,10 +120,10 @@ mod tests {
|
||||
}
|
||||
|
||||
/// `get`/`stat` signal absence with `None` and never carry
|
||||
/// `Missing` (ADR-012 §6.5 ruling 5; store-api.md §Error model).
|
||||
/// Behavioral enforcement lands with the facade in wave 2; the
|
||||
/// doc contract on [`Error::Missing`] is stated with this test as
|
||||
/// its marker.
|
||||
/// `Missing` (ADR-012 §6.5 ruling 5; store-api.md §Error model)
|
||||
/// — enforced behaviorally by the facade's `get`/`stat` returning
|
||||
/// `Option` (the store.rs read-path tests pin the both-tier-miss
|
||||
/// shapes); this test is the doc contract's in-module marker.
|
||||
#[test]
|
||||
fn missing_is_exclusive_to_err_shaped_ops() {
|
||||
let absence_on_get: Option<String> = None;
|
||||
|
||||
@@ -10,10 +10,80 @@
|
||||
//! semantics never change shape — wave 2 wires the behavior, not the
|
||||
//! surface.
|
||||
|
||||
use std::fmt;
|
||||
use std::time::SystemTime;
|
||||
|
||||
use crate::backend::ReadCursor;
|
||||
use crate::hashing::Algorithm;
|
||||
use crate::keys::Key;
|
||||
use crate::store::AnyCursor;
|
||||
|
||||
/// The `get` seam's public stream (`(len, stream)` — store-api.md §Read
|
||||
/// path): a bounded, re-entrant **windowed reader** over a stored entry,
|
||||
/// uniform across both tiers with the crate-internal read cursor
|
||||
/// (ADR-010 §1) staying behind it. The kv tier's windows slice
|
||||
/// materialized content (bounded by tier policy); the large tier's read
|
||||
/// window-by-window over the committed file — streaming without ever
|
||||
/// materializing the whole value. `read_at` is a pread-style bounded
|
||||
/// window (over-reads clamp at the entry's end; a read at EOF is empty,
|
||||
/// never an error); `read_range` is the same window plus the ADR-006
|
||||
/// slice digest — plain SHA-256 over exactly the returned bytes, fixed
|
||||
/// by convention.
|
||||
///
|
||||
/// Constructed by the store core only (via `get`); engine cursors never
|
||||
/// leak onto this surface. A `Blob` stays valid across concurrent pool
|
||||
/// activity because committed entries are immutable. No verification
|
||||
/// rides the windows themselves: canonical verification fires at get
|
||||
/// time on the kv tier's materialized reads (and at put time), per the
|
||||
/// store core's recorded verification-on-get posture (ADR-006).
|
||||
pub struct Blob {
|
||||
cursor: AnyCursor,
|
||||
}
|
||||
|
||||
impl fmt::Debug for Blob {
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
f.debug_struct("Blob")
|
||||
.field("len", &self.cursor.len())
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
impl Blob {
|
||||
/// Store-core constructor: a `Blob` wraps a cursor-dispatched
|
||||
/// engine body; consumers obtain one only through the read seam.
|
||||
pub(crate) fn new(cursor: AnyCursor) -> Blob {
|
||||
Blob { cursor }
|
||||
}
|
||||
|
||||
/// The entry's length in bytes — the metadata-only length the
|
||||
/// cursor body carries (the same probe `stat` rides).
|
||||
pub fn len(&self) -> u64 {
|
||||
self.cursor.len()
|
||||
}
|
||||
|
||||
/// Whether the entry holds zero bytes.
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.cursor.len() == 0
|
||||
}
|
||||
|
||||
/// The bytes at `[offset, offset + len)` — clamped at the entry's
|
||||
/// end (pread-style short read); `offset >= len()` (EOF) is empty,
|
||||
/// never an error.
|
||||
pub async fn read_at(&self, offset: u64, len: u64) -> Result<Vec<u8>, crate::Error> {
|
||||
self.cursor.read_at(offset, len).await
|
||||
}
|
||||
|
||||
/// The bytes at `[offset, offset + len)` plus the ADR-006 slice
|
||||
/// digest — SHA-256 over exactly the returned bytes, for the caller
|
||||
/// to carry out-of-band.
|
||||
pub async fn read_range(
|
||||
&self,
|
||||
offset: u64,
|
||||
len: u64,
|
||||
) -> Result<(Vec<u8>, [u8; 32]), crate::Error> {
|
||||
self.cursor.read_range(offset, len).await
|
||||
}
|
||||
}
|
||||
|
||||
/// The cheap stat probe's result (store-api.md §Read path): the entry's
|
||||
/// length and its hash algorithm — the "type" at this layer is the hash
|
||||
|
||||
+4
-3
@@ -41,9 +41,10 @@
|
||||
//! a key is never overwritten — a losing race or a re-put is dedup,
|
||||
//! landing as `Ok` with the stage discarded.
|
||||
//!
|
||||
//! The `#![allow(dead_code)]` below marks the engine's scope: it is a
|
||||
//! wave-2 deliverable defined ahead of its dispatch driver. The allow
|
||||
//! shrinks when dispatch names the `local` engine.
|
||||
//! The `#![allow(dead_code)]` below marks the narrow remaining scope:
|
||||
//! in default builds (no `mem`) nothing constructs the engine, so
|
||||
//! [`LocalBackend::new`]/the staged-put bodies have no caller until
|
||||
//! wave 3's facade constructors and `store-put-paths` name them.
|
||||
|
||||
#![allow(dead_code)]
|
||||
|
||||
|
||||
@@ -16,13 +16,6 @@
|
||||
//! (no media can fail here); the test-only hook noted on [`MemStaging`]
|
||||
//! exercises the failure form's self-discard hygiene, standing in for the
|
||||
//! media-failure injection later engines need.
|
||||
//!
|
||||
//! The `#![allow(dead_code)]` below marks the engine's scope: it is a
|
||||
//! wave-2 deliverable defined ahead of its dispatch driver (no store-core
|
||||
//! or facade wiring reaches it yet). The allow shrinks when dispatch
|
||||
//! names the mem engine.
|
||||
|
||||
#![allow(dead_code)]
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
+744
-16
@@ -1,18 +1,20 @@
|
||||
//! The `Store` facade type (store-api.md §What this is): the public
|
||||
//! surface consumers program against — constructor modes and the
|
||||
//! size-threshold dispatch seam. **Skeleton and routing only**: the ops
|
||||
//! methods land against this type in `store-read-paths` (get/stat/
|
||||
//! read_range/has/list with the fall-through general rule),
|
||||
//! `gc-protocol` (sweep, direct delete, the pin/liveness machinery),
|
||||
//! and `store-put-paths` (put, the kv-only rejection behavior). This
|
||||
//! module owns the engine representation and the constructor-mode
|
||||
//! machinery; the pure routing function lives in [`crate::dispatch`].
|
||||
//! surface consumers program against — constructor modes, the
|
||||
//! size-threshold dispatch seam, and the read-path ops (`get`/`stat`/
|
||||
//! `read_range`/`has`/`list` with the fall-through general rule,
|
||||
//! implemented below). The remaining ops land against this type in
|
||||
//! `store-put-paths` (put, the kv-only rejection behavior) and
|
||||
//! `gc-protocol` (sweep, direct delete, the pin/liveness machinery).
|
||||
//! Direct `delete` is *not* a read path — it lands with the GC
|
||||
//! protocol's arbitration. This module owns the engine representation
|
||||
//! and the constructor-mode machinery; the pure routing function lives
|
||||
//! in [`crate::dispatch`].
|
||||
//!
|
||||
//! **Named design point — engine representation: concrete `Store`,
|
||||
//! crate-internal enum dispatch.** A `Store<Kv: Backend, Large: Backend>`
|
||||
//! generic was the alternative: rejected. The engines are statically
|
||||
//! known and in-crate (no `dyn` anywhere — wave-1 design point 3), and
|
||||
//! they are `pub(crate)` behind the crate-internal [`Backend`][crate::backend::Backend]
|
||||
//! they are `pub(crate)` behind the crate-internal [`Backend`][crate::backend]
|
||||
//! trait — a generic facade parameterized over `Backend` implementers
|
||||
//! would leak engine types through the public surface, force consumers
|
||||
//! to name engines they must never see (the facade is **engine-blind**,
|
||||
@@ -52,6 +54,68 @@
|
||||
//! feature `mem` and constructs the mem engine — and every instance
|
||||
//! carries its mode for introspection ([`Store::mode`]).
|
||||
//!
|
||||
//! **Read-path design points (recorded for the review gate):**
|
||||
//!
|
||||
//! - **The fall-through general rule (ADR-012 §6.1; POC #1 finding
|
||||
//! 6).** Every key-addressed op consults [`KvEngine`] first, then
|
||||
//! [`LargeEngine`] — there is no length routing on the read side and
|
||||
//! [`Store::tier_for`] is never consulted by it (the dispatch seam's
|
||||
//! write-half routing is the put path's; `route` feeds nothing on
|
||||
//! reads). A miss on both tiers resolves per op's shape: `Ok(None)`
|
||||
//! on get/stat, `false` on has, `Err(Missing)` on read_range.
|
||||
//! - **The public stream type (named design point): `Blob`.** `get`
|
||||
//! returns the pinned `(len, stream)` shape as
|
||||
//! `Option<(u64, Blob)>` (store-api.md §Read path) — [`Blob`] is a
|
||||
//! bounded, re-entrant windowed reader over the crate-internal
|
||||
//! [`AnyCursor`], which dispatches the engine cursor bodies by tier
|
||||
//! (mem's materializing body for kv, `local`'s pread body for
|
||||
//! large). Windowed reads are the uniform stream shape across
|
||||
//! tiers; no chunk-iterator machinery was adopted — the cursor
|
||||
//! bodies *are* the streaming abstraction, and per-chunk `Stream`-
|
||||
//! wrapping would re-shape them without changing the underlying
|
||||
//! reads. Engine cursors never leak onto the public surface:
|
||||
//! [`AnyCursor`] is `pub(crate)`, `Blob` is opaque but drivable
|
||||
//! (constructed only by the store core, ADR-010 §1).
|
||||
//! - **Verification-on-get (named design point, pinned against
|
||||
//! ADR-006's text).** The whole-blob hash-check fires where the
|
||||
//! whole content is already in hand — the kv tier's materializing
|
||||
//! reads. A kv-tier `get` materializes the stored value and
|
||||
//! re-derives the canonical digest over it, comparing against the
|
||||
//! key (ADR-006 decision 1: "put and get paths hash-check");
|
||||
//! mismatch is `Err(Error::Verification)` and **never falls
|
||||
//! through further** — a verified mismatch is a pool-integrity
|
||||
//! fact, and consulting another tier past it would be a guess.
|
||||
//! The large tier's streaming `get` does *not* re-derive at open:
|
||||
//! opening is metadata + handle only, and the tier's integrity
|
||||
//! story is the put path's whole-blob verification pass (the
|
||||
//! complete staged artifact was checked before commit) plus the
|
||||
//! per-range slice digests `read_range` carries out-of-band
|
||||
//! (ADR-006 decisions 1–2; the §Context line "git objects
|
||||
//! self-verify under git's own model when read whole"). `read_range`
|
||||
//! carries no canonical check on either tier — the slice digest is
|
||||
//! the convention's whole answer (ADR-006 decision 2). stat/has/
|
||||
//! list carry no verification (metadata and enumeration only).
|
||||
//! - **`list` returns `Vec<Key>` (named design point: Vec vs
|
||||
//! stream).** Whole-pool enumeration, complete by contract
|
||||
//! (ADR-003 — load-bearing for GC), returned **materialized and
|
||||
//! sorted in the keys' total order** (deterministic output). The
|
||||
//! sweep consumes the same crate-internal enumeration directly from
|
||||
//! the engines (its live-set diffs sort independently), so the
|
||||
//! facade shape serves consumers, not internal needs. A lazy stream
|
||||
//! was rejected: every shipped engine enumerates fully and eagerly
|
||||
//! (the completeness contract *is* the GC-safety property), so a
|
||||
//! pseudo-lazy wrapper adds machinery without changing the
|
||||
//! underlying walk. Listed bytes failing [`Key::from_bytes`]
|
||||
//! reject with `Error::KeyInvalid` — a malformed pool list is
|
||||
//! fatal, never a silent skip (ADR-003's malformed-list ruling
|
||||
//! applied at the facade boundary).
|
||||
//! - **`KeyInvalid` at the facade boundary (ADR-002/003).** Read ops
|
||||
//! take opaque key *bytes* and convert to typed `Key` at the
|
||||
//! boundary via [`Key::from_bytes`]: malformed bytes reject with
|
||||
//! `Error::KeyInvalid` on every key-addressed op, never a guess,
|
||||
//! and the engines receive only the physical byte encoding — they
|
||||
//! never see typed keys.
|
||||
//!
|
||||
//! **The fleet declaration defers to the engine waves.** ADR-012 §3
|
||||
//! rules fleet mode a constructor declaration (`fleet: true`), never
|
||||
//! inferred — but it is a constructor *error* for every non-fleet-valid
|
||||
@@ -70,21 +134,24 @@
|
||||
//! [`Store::large`]), and the GC state home ([`crate::gc::GcHome`])
|
||||
//! the gc-protocol task fills.
|
||||
//!
|
||||
//! The `#![allow(dead_code)]` below marks the skeleton's scope: the
|
||||
//! accessors and engine enums exist for the ops tasks' drivers, and the
|
||||
//! kv engine set is empty in default builds (sqlite is wave 3). The
|
||||
//! allow shrinks as the ops tasks land.
|
||||
//! The `#![allow(dead_code)]` below marks the work still ahead, not the
|
||||
//! read path (which is driven below): write-side `tier_for`/`threshold`
|
||||
//! and the GC home wait on `store-put-paths`/`gc-protocol`, and the kv
|
||||
//! engine set is empty in default builds (mem is feature-gated; sqlite
|
||||
//! is wave 3). The allow shrinks as those land.
|
||||
|
||||
#![allow(dead_code)]
|
||||
|
||||
use crate::backend::{Backend, ReadCursor};
|
||||
use crate::dispatch::{route, Tier};
|
||||
use crate::facade::{Blob, EntryMeta};
|
||||
use crate::gc::GcHome;
|
||||
use crate::local::LocalBackend;
|
||||
use crate::keys::Key;
|
||||
use crate::local::{LocalBackend, LocalCursor};
|
||||
|
||||
#[cfg(feature = "mem")]
|
||||
use crate::mem::MemBackend;
|
||||
use crate::mem::{MemBackend, MemCursor};
|
||||
|
||||
#[cfg(feature = "mem")]
|
||||
use crate::Error;
|
||||
|
||||
#[cfg(feature = "mem")]
|
||||
@@ -150,6 +217,115 @@ pub(crate) enum LargeEngine {
|
||||
Local(LocalBackend),
|
||||
}
|
||||
|
||||
/// The read path's cursor-dispatch enum (ADR-010 §1's "store core
|
||||
/// programs the abstraction"): tier dispatch over the engine cursor
|
||||
/// bodies — the kv tier's materializing body and the large tier's
|
||||
/// streaming body — behind one crate-internal value that [`Blob`] holds
|
||||
/// and drives. Crate-internal: engine cursors never appear on the
|
||||
/// public surface; the facade's stream shape is this enum's uniform
|
||||
/// [`ReadCursor`] view (`Blob`).
|
||||
pub(crate) enum AnyCursor {
|
||||
/// The kv body family: the whole value materialized in memory
|
||||
/// (bounded by tier policy).
|
||||
#[cfg(feature = "mem")]
|
||||
Kv(MemCursor),
|
||||
/// The large body family: a pread cursor over the committed file.
|
||||
Large(LocalCursor),
|
||||
}
|
||||
|
||||
impl ReadCursor for AnyCursor {
|
||||
fn len(&self) -> u64 {
|
||||
match self {
|
||||
#[cfg(feature = "mem")]
|
||||
Self::Kv(cursor) => cursor.len(),
|
||||
Self::Large(cursor) => cursor.len(),
|
||||
}
|
||||
}
|
||||
|
||||
async fn read_at(&self, offset: u64, len: u64) -> Result<Vec<u8>, Error> {
|
||||
match self {
|
||||
#[cfg(feature = "mem")]
|
||||
Self::Kv(cursor) => cursor.read_at(offset, len).await,
|
||||
Self::Large(cursor) => cursor.read_at(offset, len).await,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl KvEngine {
|
||||
#[allow(unused_variables)] // kv engine set empty until wave 3 (sqlite): no-mem builds never reference `key`
|
||||
async fn get(&self, key: &[u8]) -> Result<Option<AnyCursor>, Error> {
|
||||
match *self {
|
||||
#[cfg(feature = "mem")]
|
||||
KvEngine::Mem(ref engine) => Ok(engine.get(key).await?.map(AnyCursor::Kv)),
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(unused_variables)] // same empty-set scope as `get`
|
||||
async fn has(&self, key: &[u8]) -> Result<bool, Error> {
|
||||
match *self {
|
||||
#[cfg(feature = "mem")]
|
||||
KvEngine::Mem(ref engine) => engine.has(key).await,
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(unused_variables)] // same empty-set scope as `get`
|
||||
async fn size(&self, key: &[u8]) -> Result<Option<u64>, Error> {
|
||||
match *self {
|
||||
#[cfg(feature = "mem")]
|
||||
KvEngine::Mem(ref engine) => engine.size(key).await,
|
||||
}
|
||||
}
|
||||
|
||||
async fn list(&self) -> Result<Vec<Vec<u8>>, Error> {
|
||||
match *self {
|
||||
#[cfg(feature = "mem")]
|
||||
KvEngine::Mem(ref engine) => engine.list().await,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl LargeEngine {
|
||||
async fn get(&self, key: &[u8]) -> Result<Option<AnyCursor>, Error> {
|
||||
match self {
|
||||
LargeEngine::Local(engine) => Ok(engine.get(key).await?.map(AnyCursor::Large)),
|
||||
}
|
||||
}
|
||||
|
||||
async fn has(&self, key: &[u8]) -> Result<bool, Error> {
|
||||
match self {
|
||||
LargeEngine::Local(engine) => engine.has(key).await,
|
||||
}
|
||||
}
|
||||
|
||||
async fn size(&self, key: &[u8]) -> Result<Option<u64>, Error> {
|
||||
match self {
|
||||
LargeEngine::Local(engine) => engine.size(key).await,
|
||||
}
|
||||
}
|
||||
|
||||
async fn list(&self) -> Result<Vec<Vec<u8>>, Error> {
|
||||
match self {
|
||||
LargeEngine::Local(engine) => engine.list().await,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The facade's key-bytes decode: typed `Key` conversion at the store
|
||||
/// boundary (ADR-002/003) — the one place on the read path where key
|
||||
/// bytes are interpreted; anything malformed is `Error::KeyInvalid`,
|
||||
/// never a guess, and never reaches an engine typed.
|
||||
fn boundary_key(key: &[u8]) -> Result<Key, Error> {
|
||||
Key::from_bytes(key)
|
||||
}
|
||||
|
||||
/// Constitute the facade's typed keys from an engine's opaque list
|
||||
/// rows: a failed decode is fatal (`Error::KeyInvalid`) — a malformed
|
||||
/// pool list is as deadly as an incomplete one and is never silently
|
||||
/// skipped (ADR-003).
|
||||
fn typed_keys(raws: Vec<Vec<u8>>) -> Result<Vec<Key>, Error> {
|
||||
raws.into_iter().map(|raw| boundary_key(&raw)).collect()
|
||||
}
|
||||
|
||||
impl Store {
|
||||
/// **mem-only mode** (ADR-011 §4): the mem engine alone, no
|
||||
/// dispatch threshold in effect — a testing/embedder-ephemeral
|
||||
@@ -268,10 +444,144 @@ impl Store {
|
||||
}
|
||||
}
|
||||
|
||||
/// The read path (store-api.md §Read path): `get`, `stat`,
|
||||
/// `read_range`, `has`, `list` — every key-addressed op falling through
|
||||
/// kv first, then large (ADR-012 §6.1's general rule), typed key
|
||||
/// conversion at the facade boundary, and the module-doc design points
|
||||
/// (the `Blob` stream shape, verification-on-get, the `Vec<Key>` list
|
||||
/// shape) carried in their implementations.
|
||||
impl Store {
|
||||
/// `get(key) → Option<(len, stream)>`. Falls through kv → large;
|
||||
/// a both-tier miss is `Ok(None)` — never `Error::Missing`
|
||||
/// (ADR-012 §6.5 ruling 5). On a kv hit the materialized read is
|
||||
/// whole-blob-verified against the key at get time (mismatch:
|
||||
/// `Err(Error::Verification)`, never a further fall-through) and
|
||||
/// the returned length is the verified materialized byte length;
|
||||
/// the large tier's hit opens streaming (metadata + handle, no
|
||||
/// canonical re-derivation — see the module doc), its length the
|
||||
/// cursor's metadata-only probe. Both hits return the same
|
||||
/// `(len, stream)` shape: the stream is [`Blob`] over the tier's
|
||||
/// cursor body.
|
||||
pub async fn get(&self, key: &[u8]) -> Result<Option<(u64, Blob)>, Error> {
|
||||
let key = boundary_key(key)?;
|
||||
let bytes = key.as_bytes();
|
||||
|
||||
let cursor = if let Some(cursor) = self.kv().get(&bytes).await? {
|
||||
cursor
|
||||
} else if let Some(cursor) = self.large_get(&bytes).await? {
|
||||
cursor
|
||||
} else {
|
||||
return Ok(None);
|
||||
};
|
||||
|
||||
match cursor {
|
||||
#[cfg(feature = "mem")]
|
||||
AnyCursor::Kv(kv) => {
|
||||
let value = kv.read_at(0, kv.len()).await?;
|
||||
let derived = key.algorithm().derive(&value, value.len() as u64);
|
||||
if &derived != key.digest() {
|
||||
return Err(Error::Verification);
|
||||
}
|
||||
Ok(Some((value.len() as u64, Blob::new(AnyCursor::Kv(kv)))))
|
||||
}
|
||||
AnyCursor::Large(large) => Ok(Some((large.len(), Blob::new(AnyCursor::Large(large))))),
|
||||
}
|
||||
}
|
||||
|
||||
/// The large tier's consult — `None` where the mode has no large
|
||||
/// tier (single-tier modes cannot hold large-tier content).
|
||||
async fn large_get(&self, bytes: &[u8]) -> Result<Option<AnyCursor>, Error> {
|
||||
match self.large() {
|
||||
Some(large) => large.get(bytes).await,
|
||||
None => Ok(None),
|
||||
}
|
||||
}
|
||||
|
||||
/// `stat(key) → Option<EntryMeta>` — the cheap probe riding the
|
||||
/// trait's `size` method (ADR-008): length plus the key's hash
|
||||
/// algorithm, never a content load, falling through kv → large.
|
||||
/// A both-tier miss is `Ok(None)`.
|
||||
pub async fn stat(&self, key: &[u8]) -> Result<Option<EntryMeta>, Error> {
|
||||
let key = boundary_key(key)?;
|
||||
let bytes = key.as_bytes();
|
||||
|
||||
let len = match self.kv().size(&bytes).await? {
|
||||
Some(len) => Some(len),
|
||||
None => match self.large() {
|
||||
Some(large) => large.size(&bytes).await?,
|
||||
None => None,
|
||||
},
|
||||
};
|
||||
Ok(len.map(|len| EntryMeta {
|
||||
len,
|
||||
algorithm: key.algorithm(),
|
||||
}))
|
||||
}
|
||||
|
||||
/// `read_range(key, offset, len) → (slice, slice_digest)` — the
|
||||
/// slice plus the out-of-band SHA-256 slice digest (ADR-006, fixed
|
||||
/// by convention), composed through the cursor's `read_range` seam.
|
||||
/// Falls through kv → large exactly as [`Store::get`] does; no
|
||||
/// canonical verification rides a range (the slice digest is the
|
||||
/// whole answer, ADR-006 decision 2). A both-tier miss is
|
||||
/// `Err(Error::Missing)` — the pinned shape difference vs `get`'s
|
||||
/// `None` (ADR-012 §6.5 ruling 5).
|
||||
pub async fn read_range(
|
||||
&self,
|
||||
key: &[u8],
|
||||
offset: u64,
|
||||
len: u64,
|
||||
) -> Result<(Vec<u8>, [u8; 32]), Error> {
|
||||
let key = boundary_key(key)?;
|
||||
let bytes = key.as_bytes();
|
||||
|
||||
if let Some(cursor) = self.kv().get(&bytes).await? {
|
||||
return cursor.read_range(offset, len).await;
|
||||
}
|
||||
match self.large_get(&bytes).await? {
|
||||
Some(cursor) => cursor.read_range(offset, len).await,
|
||||
None => Err(Error::Missing),
|
||||
}
|
||||
}
|
||||
|
||||
/// `has(key) → bool` — the pure existence probe (ADR-008's `has`
|
||||
/// discipline: an existence probe per tier, `local`'s body being
|
||||
/// the literal size-probe), falling through kv → large. Absent on
|
||||
/// both tiers: `false`. Virgin-store reads are no-ops.
|
||||
pub async fn has(&self, key: &[u8]) -> Result<bool, Error> {
|
||||
let key = boundary_key(key)?;
|
||||
let bytes = key.as_bytes();
|
||||
|
||||
if self.kv().has(&bytes).await? {
|
||||
return Ok(true);
|
||||
}
|
||||
match self.large() {
|
||||
Some(large) => large.has(&bytes).await,
|
||||
None => Ok(false),
|
||||
}
|
||||
}
|
||||
|
||||
/// `list() → Vec<Key>` — whole-pool enumeration of every tier,
|
||||
/// complete by contract (ADR-003), returned materialized and
|
||||
/// sorted in the keys' total order (the recorded Vec-vs-stream
|
||||
/// design point; see the module doc). Malformed pool bytes are
|
||||
/// fatal: `Err(Error::KeyInvalid)`, never a silent skip.
|
||||
pub async fn list(&self) -> Result<Vec<Key>, Error> {
|
||||
let mut keys = typed_keys(self.kv().list().await?)?;
|
||||
if let Some(large) = self.large() {
|
||||
keys.extend(typed_keys(large.list().await?)?);
|
||||
}
|
||||
keys.sort();
|
||||
Ok(keys)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(all(test, feature = "mem"))]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::backend::{slice_digest, Backend};
|
||||
use crate::dispatch::DEFAULT_DISPATCH_THRESHOLD;
|
||||
use crate::hashing::Algorithm;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
@@ -416,4 +726,422 @@ mod tests {
|
||||
let store = Store::mem_only();
|
||||
let _ = store.gc();
|
||||
}
|
||||
|
||||
/// Entry planters straight into the tier engines (bypassing the
|
||||
/// put seam, which is a later task): opaque key bytes + value,
|
||||
/// exactly what the engines' CAS tables accept. Planting
|
||||
/// mismatching content under a key is how the read path's
|
||||
/// verification postures are observed.
|
||||
async fn plant_kv(store: &Store, key: &[u8], value: &[u8]) {
|
||||
match store.kv() {
|
||||
KvEngine::Mem(engine) => engine.put(key, value).await.expect("kv plant"),
|
||||
}
|
||||
}
|
||||
|
||||
async fn plant_large(store: &Store, key: &[u8], value: &[u8]) {
|
||||
match store.large() {
|
||||
Some(LargeEngine::Local(engine)) => engine.put(key, value).await.expect("large plant"),
|
||||
None => panic!("planting into a mode with no large tier"),
|
||||
}
|
||||
}
|
||||
|
||||
fn key_for(content: &[u8]) -> Key {
|
||||
Key::derive(content, content.len() as u64)
|
||||
}
|
||||
|
||||
fn sha1_key_for(content: &[u8]) -> Key {
|
||||
Key::from(Algorithm::GitBlobSha1.derive(content, content.len() as u64))
|
||||
}
|
||||
|
||||
/// The both-tier-miss shapes every read op must resolve to
|
||||
/// (store-api.md §Error model; ADR-012 §6.5 ruling 5) — pinned on
|
||||
/// all three constructible modes, virgin-store reads as no-ops.
|
||||
async fn assert_virgin_read_shapes(store: &Store) {
|
||||
let key = key_for(b"nothing ever landed").as_bytes();
|
||||
|
||||
assert!(
|
||||
store.get(&key).await.unwrap().is_none(),
|
||||
"get's both-tier miss is Ok(None)"
|
||||
);
|
||||
assert_eq!(store.stat(&key).await.unwrap(), None, "stat misses as None");
|
||||
assert_eq!(
|
||||
store.read_range(&key, 0, 16).await.unwrap_err(),
|
||||
Error::Missing,
|
||||
"read_range's both-tier miss is Err(Missing) — the pinned shape difference vs get"
|
||||
);
|
||||
assert!(
|
||||
!store.has(&key).await.unwrap(),
|
||||
"has's both-tier miss is false"
|
||||
);
|
||||
assert_eq!(
|
||||
store.list().await.unwrap(),
|
||||
Vec::<Key>::new(),
|
||||
"virgin-store list is empty"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn virgin_store_reads_are_no_ops_dual_tier() {
|
||||
let root = temp_root("read-virgin-dual");
|
||||
let store = Store::dual_tier_mem_local(root.path(), 1024)
|
||||
.await
|
||||
.expect("dual-tier constructs");
|
||||
assert_virgin_read_shapes(&store).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn virgin_store_reads_are_no_ops_mem_only() {
|
||||
let store = Store::mem_only();
|
||||
assert_virgin_read_shapes(&store).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn virgin_store_reads_are_no_ops_kv_only() {
|
||||
let store = Store::kv_only_mem(64);
|
||||
assert_virgin_read_shapes(&store).await;
|
||||
}
|
||||
|
||||
/// `get` returns the pinned `(len, stream)` shape from the kv tier:
|
||||
/// the tuple carries the metadata length, the `Blob` serves
|
||||
/// bounded windows (clamped at EOF) and the ADR-006 range seam.
|
||||
#[tokio::test]
|
||||
async fn get_returns_the_pinned_len_stream_shape_from_kv() {
|
||||
let store = Store::mem_only();
|
||||
let content = b"small-tier content";
|
||||
let key = key_for(content);
|
||||
plant_kv(&store, &key.as_bytes(), content).await;
|
||||
|
||||
let (len, blob) = store
|
||||
.get(&key.as_bytes())
|
||||
.await
|
||||
.expect("get succeeds")
|
||||
.expect("entry present");
|
||||
assert_eq!(len, content.len() as u64, "tuple len is the entry length");
|
||||
assert_eq!(blob.len(), len);
|
||||
assert!(!blob.is_empty());
|
||||
|
||||
let whole = blob.read_at(0, len).await.expect("full window");
|
||||
assert_eq!(whole, content, "full window matches");
|
||||
assert_eq!(
|
||||
blob.read_at(0, len + 17).await.expect("over-read"),
|
||||
content,
|
||||
"over-read clamps at the entry's end"
|
||||
);
|
||||
|
||||
let (slice, digest) = blob.read_range(6, 6).await.expect("ranged window");
|
||||
assert_eq!(slice, &content[6..12]);
|
||||
assert_eq!(
|
||||
digest,
|
||||
slice_digest(&content[6..12]),
|
||||
"slice digest is SHA-256 over exactly the returned bytes"
|
||||
);
|
||||
|
||||
let (full, full_digest) = blob.read_range(0, len).await.expect("full range");
|
||||
assert_eq!(full, content);
|
||||
assert_eq!(full_digest, slice_digest(content));
|
||||
}
|
||||
|
||||
/// The zero-length degenerate: a kv-tier empty entry get's as
|
||||
/// `Some((0, blob))`, its windows read empty-never-error, and
|
||||
/// `stat` carries exact zero.
|
||||
#[tokio::test]
|
||||
async fn zero_length_kv_entry_reads_as_empty_blob() {
|
||||
let store = Store::mem_only();
|
||||
let key = key_for(b"");
|
||||
plant_kv(&store, &key.as_bytes(), b"").await;
|
||||
|
||||
let (len, blob) = store.get(&key.as_bytes()).await.unwrap().expect("present");
|
||||
assert_eq!(len, 0);
|
||||
assert!(blob.is_empty());
|
||||
assert_eq!(blob.read_at(0, 8).await.unwrap(), Vec::<u8>::new());
|
||||
|
||||
assert_eq!(
|
||||
store.stat(&key.as_bytes()).await.unwrap(),
|
||||
Some(EntryMeta {
|
||||
len: 0,
|
||||
algorithm: key.algorithm()
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
/// `get`/`stat`/`read_range`/`has` fall through to the large tier
|
||||
/// on a kv miss — consult order, not length routing: a
|
||||
/// sub-threshold entry planted only in the large tier is still
|
||||
/// found even though `tier_for` routes its length to kv.
|
||||
#[tokio::test]
|
||||
async fn read_ops_fall_through_kv_miss_to_large() {
|
||||
let root = temp_root("read-fall-through");
|
||||
let store = Store::dual_tier_mem_local(root.path(), 1024)
|
||||
.await
|
||||
.expect("dual-tier constructs");
|
||||
let content = b"a sub-threshold entry planted directly in the large tier";
|
||||
let key = key_for(content);
|
||||
|
||||
assert_eq!(
|
||||
store.tier_for(content.len() as u64),
|
||||
Tier::Kv,
|
||||
"the length *routes* kv — the read still lands via the large tier"
|
||||
);
|
||||
plant_large(&store, &key.as_bytes(), content).await;
|
||||
|
||||
let (len, blob) = store
|
||||
.get(&key.as_bytes())
|
||||
.await
|
||||
.expect("fall-through get succeeds")
|
||||
.expect("large-tier entry found via fall-through");
|
||||
assert_eq!(len, content.len() as u64);
|
||||
assert_eq!(blob.read_at(0, len).await.unwrap(), content);
|
||||
|
||||
assert_eq!(
|
||||
store.stat(&key.as_bytes()).await.unwrap(),
|
||||
Some(EntryMeta {
|
||||
len: content.len() as u64,
|
||||
algorithm: key.algorithm()
|
||||
}),
|
||||
"stat falls through too"
|
||||
);
|
||||
assert!(
|
||||
store.has(&key.as_bytes()).await.unwrap(),
|
||||
"has falls through"
|
||||
);
|
||||
|
||||
let (slice, digest) = store
|
||||
.read_range(&key.as_bytes(), 4, 8)
|
||||
.await
|
||||
.expect("read_range falls through");
|
||||
assert_eq!(slice, &content[4..12]);
|
||||
assert_eq!(digest, slice_digest(&slice));
|
||||
}
|
||||
|
||||
/// The kv tier is the *first* consult on every key-addressed op:
|
||||
/// the same key bytes planted with different (mismatching) content
|
||||
/// in both tiers — get/read_range/stat/has must all answer from
|
||||
/// the kv tier's entry.
|
||||
#[tokio::test]
|
||||
async fn kv_tier_is_the_first_consult_on_every_key_addressed_op() {
|
||||
let root = temp_root("read-kv-first");
|
||||
let store = Store::dual_tier_mem_local(root.path(), 1024)
|
||||
.await
|
||||
.expect("dual-tier constructs");
|
||||
const KV_CONTENT: &[u8] = b"the kv tier's entry";
|
||||
let key = key_for(KV_CONTENT);
|
||||
plant_kv(&store, &key.as_bytes(), KV_CONTENT).await;
|
||||
plant_large(&store, &key.as_bytes(), b"decoy content in the large tier").await;
|
||||
|
||||
let (len, blob) = store
|
||||
.get(&key.as_bytes())
|
||||
.await
|
||||
.expect("get succeeds")
|
||||
.expect("kv hit wins");
|
||||
assert_eq!(
|
||||
blob.read_at(0, len).await.unwrap(),
|
||||
KV_CONTENT,
|
||||
"kv tier first — and its content verifies against the key"
|
||||
);
|
||||
|
||||
let (slice, _) = store
|
||||
.read_range(&key.as_bytes(), 0, 4)
|
||||
.await
|
||||
.expect("ranged");
|
||||
assert_eq!(
|
||||
slice,
|
||||
&KV_CONTENT[0..4],
|
||||
"read_range also consults kv first"
|
||||
);
|
||||
|
||||
assert!(store.has(&key.as_bytes()).await.unwrap());
|
||||
assert_eq!(
|
||||
store.stat(&key.as_bytes()).await.unwrap(),
|
||||
Some(EntryMeta {
|
||||
len: KV_CONTENT.len() as u64,
|
||||
algorithm: key.algorithm()
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
/// Verification-on-get, kv tier (ADR-006 decision 1): a
|
||||
/// materialized get hash-checks against the key; mismatch is
|
||||
/// `Err(Verification)` and never falls through further. `read_range`
|
||||
/// by contrast carries no canonical check — the slice digest is its
|
||||
/// whole verification story (ADR-006 decision 2).
|
||||
#[tokio::test]
|
||||
async fn kv_get_whole_blob_verifies_and_never_falls_through_past_mismatch() {
|
||||
let store = Store::mem_only();
|
||||
let key = key_for(b"the derived address");
|
||||
let bytes = key.as_bytes();
|
||||
plant_kv(&store, &bytes, b"content that is not the key").await;
|
||||
|
||||
assert_eq!(
|
||||
store.get(&bytes).await.unwrap_err(),
|
||||
Error::Verification,
|
||||
"kv materialized reads hash-check at get time"
|
||||
);
|
||||
assert_eq!(
|
||||
store.read_range(&bytes, 0, 8).await.unwrap(),
|
||||
(b"content ".to_vec(), slice_digest(b"content ")),
|
||||
"read_range carries no canonical check — the slice-digest posture"
|
||||
);
|
||||
assert!(
|
||||
store.stat(&bytes).await.unwrap().is_some(),
|
||||
"stat is a metadata probe, not a verifier"
|
||||
);
|
||||
}
|
||||
|
||||
/// Verification-on-get, large tier (the recorded posture's other
|
||||
/// half): a streaming get opens metadata + handle only — it does
|
||||
/// not re-derive the canonical digest at open. The tier's
|
||||
/// integrity story is the put path's whole-blob verification pass
|
||||
/// plus the per-range slice digests (ADR-006 decisions 1–2).
|
||||
#[tokio::test]
|
||||
async fn large_streaming_get_does_not_re_derive_at_open() {
|
||||
let root = temp_root("read-large-no-reverify");
|
||||
let store = Store::dual_tier_mem_local(root.path(), 8)
|
||||
.await
|
||||
.expect("dual-tier constructs");
|
||||
let key = key_for(b"the derived address");
|
||||
plant_large(&store, &key.as_bytes(), b"stored bytes").await;
|
||||
|
||||
assert!(
|
||||
store.get(&key.as_bytes()).await.unwrap().is_some(),
|
||||
"streaming open never re-derives (mismatching content and all)"
|
||||
);
|
||||
}
|
||||
|
||||
/// `stat` carries len + algorithm for both algorithms (the
|
||||
/// "type" at this layer is the hash algorithm, ADR-002) and is a
|
||||
/// `size` ride — the fall-through consult, `None` when absent.
|
||||
#[tokio::test]
|
||||
async fn stat_carries_len_and_algorithm_from_size_probes() {
|
||||
let root = temp_root("read-stat");
|
||||
let store = Store::dual_tier_mem_local(root.path(), 1024)
|
||||
.await
|
||||
.expect("dual-tier constructs");
|
||||
|
||||
let sha256_key = key_for(b"canonical");
|
||||
plant_kv(&store, &sha256_key.as_bytes(), b"canonical").await;
|
||||
let legacy_key = sha1_key_for(b"legacy");
|
||||
plant_large(&store, &legacy_key.as_bytes(), b"legacy").await;
|
||||
|
||||
assert_eq!(
|
||||
store.stat(&sha256_key.as_bytes()).await.unwrap(),
|
||||
Some(EntryMeta {
|
||||
len: 9,
|
||||
algorithm: Algorithm::GitBlobSha256
|
||||
})
|
||||
);
|
||||
assert_eq!(
|
||||
store
|
||||
.stat(&legacy_key.as_bytes())
|
||||
.await
|
||||
.unwrap()
|
||||
.expect("legacy entry present")
|
||||
.algorithm,
|
||||
Algorithm::GitBlobSha1,
|
||||
"the algorithm comes from the key"
|
||||
);
|
||||
assert_eq!(
|
||||
store.stat(&sha256_key.as_bytes()).await.unwrap(),
|
||||
store.stat(&sha256_key.as_bytes()).await.unwrap(),
|
||||
"stat is a pure probe"
|
||||
);
|
||||
}
|
||||
|
||||
/// `list` returns the sorted, typed union of both tiers'
|
||||
/// enumerations — both algorithms, dedup-free by key identity.
|
||||
#[tokio::test]
|
||||
async fn list_returns_the_sorted_typed_union_of_both_tiers() {
|
||||
let root = temp_root("read-list");
|
||||
let store = Store::dual_tier_mem_local(root.path(), 1024)
|
||||
.await
|
||||
.expect("dual-tier constructs");
|
||||
|
||||
let kv_keys = [
|
||||
key_for(b"kv entry one"),
|
||||
sha1_key_for(b"kv legacy"),
|
||||
key_for(b"kv entry two"),
|
||||
];
|
||||
for key in &kv_keys {
|
||||
plant_kv(&store, &key.as_bytes(), b"kv bytes").await;
|
||||
}
|
||||
let large_keys = [key_for(b"large entry one"), key_for(b"large entry two")];
|
||||
for key in &large_keys {
|
||||
plant_large(&store, &key.as_bytes(), b"large bytes").await;
|
||||
}
|
||||
|
||||
let mut expected: Vec<Key> = kv_keys.into_iter().chain(large_keys).collect();
|
||||
expected.sort();
|
||||
|
||||
let listed = store.list().await.expect("list succeeds");
|
||||
assert_eq!(listed, expected, "complete, typed, both tiers");
|
||||
for window in listed.windows(2) {
|
||||
assert!(
|
||||
window[0] < window[1],
|
||||
"output is sorted in the keys' total order"
|
||||
);
|
||||
}
|
||||
for key in &listed {
|
||||
assert!(
|
||||
store.has(&key.as_bytes()).await.unwrap(),
|
||||
"every listed key addresses a real entry"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// A malformed pool list is fatal (`Error::KeyInvalid`), never a
|
||||
/// silent skip — ADR-003's malformed-list ruling applied at the
|
||||
/// facade boundary; the engines own their own body-level checks.
|
||||
#[tokio::test]
|
||||
async fn list_of_malformed_pool_keys_is_fatal() {
|
||||
let store = Store::mem_only();
|
||||
plant_kv(&store, &[0x07; 33], b"stray in the table").await;
|
||||
|
||||
assert_eq!(
|
||||
store.list().await.unwrap_err(),
|
||||
Error::KeyInvalid,
|
||||
"malformed pool bytes are fatal on list"
|
||||
);
|
||||
}
|
||||
|
||||
/// `KeyInvalid` at the facade boundary on every key-addressed
|
||||
/// op (typed conversion happens there and only there, ADR-002/003)
|
||||
/// — malformed bytes reject on hit paths (a planted entry) and
|
||||
/// miss paths alike, before any tier is consulted and never as a
|
||||
/// `None`/`false` guess.
|
||||
#[tokio::test]
|
||||
async fn malformed_key_bytes_reject_keyinvalid_on_every_op() {
|
||||
let store = Store::mem_only();
|
||||
let anchor = key_for(b"an anchor entry");
|
||||
plant_kv(&store, &anchor.as_bytes(), b"anchor").await;
|
||||
|
||||
let key_bytes = anchor.as_bytes();
|
||||
let malformed = [
|
||||
Vec::new(), // empty
|
||||
key_bytes[..key_bytes.len() - 1].to_vec(), // truncated digest
|
||||
[0x00u8; 33].to_vec(), // reserved tag, full width
|
||||
[0x03u8; 33].to_vec(), // unknown tag
|
||||
[0x01u8; 21].to_vec(), // right tag, wrong digest length
|
||||
];
|
||||
for raw in &malformed {
|
||||
assert_eq!(
|
||||
store.get(raw).await.unwrap_err(),
|
||||
Error::KeyInvalid,
|
||||
"get rejects malformed key bytes: {raw:?}"
|
||||
);
|
||||
assert_eq!(
|
||||
store.stat(raw).await.unwrap_err(),
|
||||
Error::KeyInvalid,
|
||||
"stat rejects malformed key bytes: {raw:?}"
|
||||
);
|
||||
assert_eq!(
|
||||
store.read_range(raw, 0, 1).await.unwrap_err(),
|
||||
Error::KeyInvalid,
|
||||
"read_range rejects malformed key bytes: {raw:?}"
|
||||
);
|
||||
assert_eq!(
|
||||
store.has(raw).await.unwrap_err(),
|
||||
Error::KeyInvalid,
|
||||
"has rejects malformed key bytes: {raw:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
+101
-9
@@ -1,7 +1,7 @@
|
||||
---
|
||||
id: store-read-paths
|
||||
name: Store read paths — get/stat/read_range/has/list with dispatch fall-through
|
||||
status: pending
|
||||
status: completed
|
||||
depends_on: [store-construction]
|
||||
scope: moderate
|
||||
risk: medium
|
||||
@@ -55,19 +55,19 @@ Implement the facade's read-side ops on the `Store` type (store-api.md
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `get`/`stat`/`read_range`/`has`/`list` implemented with the
|
||||
- [x] `get`/`stat`/`read_range`/`has`/`list` implemented with the
|
||||
fall-through general rule (kv first, then large), unit-tested
|
||||
including the both-tier-miss shapes (`None` vs `Err(Missing)` per
|
||||
op)
|
||||
- [ ] The public stream type decided and recorded; engine cursors never
|
||||
- [x] The public stream type decided and recorded; engine cursors never
|
||||
leak into the public surface
|
||||
- [ ] `stat` never fetches content (rides `size`); `EntryMeta` carries
|
||||
- [x] `stat` never fetches content (rides `size`); `EntryMeta` carries
|
||||
len + algorithm
|
||||
- [ ] `read_range` returns slice + SHA-256 slice digest per the ADR-006
|
||||
- [x] `read_range` returns slice + SHA-256 slice digest per the ADR-006
|
||||
convention; verification-on-get posture decided and recorded
|
||||
- [ ] Malformed key bytes reject with `KeyInvalid` at the facade
|
||||
- [x] Malformed key bytes reject with `KeyInvalid` at the facade
|
||||
boundary on every key-addressed op
|
||||
- [ ] `cargo test` green; `cargo test --features mem` and `cargo test
|
||||
- [x] `cargo test` green; `cargo test --features mem` and `cargo test
|
||||
--all-features` green; clippy/fmt gates green
|
||||
|
||||
## References
|
||||
@@ -81,8 +81,100 @@ Implement the facade's read-side ops on the `Store` type (store-api.md
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
- **The public stream type decided: `Blob` (in `src/facade.rs`) — a
|
||||
bounded, re-entrant windowed reader; `get` returns the pinned
|
||||
`(len, stream)` shape as `Option<(u64, Blob)>`.** The engine cursor
|
||||
bodies (mem's materializing body, `local`'s pread body) stay behind
|
||||
a crate-internal tier-dispatch enum `AnyCursor` (`src/store.rs`)
|
||||
implementing `ReadCursor`; `Blob` exposes `len()`/`is_empty()`,
|
||||
`read_at(offset, len)` (pread-style clamped windows), and
|
||||
`read_range(offset, len)` (window + ADR-006 slice digest), constructed
|
||||
only by the store core. No chunk-iterator machinery was adopted: the
|
||||
cursor bodies *are* the streaming abstraction (large tier reads
|
||||
window-by-window over the committed file without ever materializing
|
||||
the whole; kv is bounded by materialization policy) — a per-chunk
|
||||
`Stream` wrapper would re-shape them without changing the underlying
|
||||
reads. Recorded in `src/store.rs` module docs + `Blob`'s doc.
|
||||
- **Verification-on-get posture (pinned against ADR-006's text):** the
|
||||
whole-blob hash-check fires where the whole content is already in
|
||||
hand — the kv tier's materializing reads (ADR-006 decision 1: "put
|
||||
and get paths hash-check"): kv get materializes, re-derives the
|
||||
canonical digest over the stored bytes, compares against the key;
|
||||
mismatch is `Err(Verification)` and never falls through further (a
|
||||
verified mismatch is a pool-integrity fact; consulting another tier
|
||||
past it would be a guess). The large tier's streaming get does *not*
|
||||
re-derive at open (metadata + handle only) — its integrity story is
|
||||
the put path's whole-blob verification pass plus the per-range slice
|
||||
digests (ADR-006 decisions 1–2; §Context's "git objects self-verify
|
||||
under git's own model when read whole"). `read_range` carries no
|
||||
canonical check on either tier (decision 2: no per-range verification
|
||||
is derivable). Pinned by tests on both tiers
|
||||
(`kv_get_whole_blob_verifies_and_never_falls_through_past_mismatch`,
|
||||
`large_streaming_get_does_not_re_derive_at_open`).
|
||||
- **`list` decided: `Vec<Key>`** — whole-pool enumeration returned
|
||||
materialized and sorted in the keys' total order (deterministic,
|
||||
typed, both algorithms). The sweep consumes the same crate-internal
|
||||
enumeration directly from the engines (its live-set diffs sort
|
||||
independently), so the facade shape serves consumers, not internal
|
||||
needs; a lazy stream was rejected because every shipped engine
|
||||
enumerates fully and eagerly (the completeness contract *is* the
|
||||
GC-safety property) — a pseudo-lazy wrapper adds machinery without
|
||||
changing the underlying walk. Listed bytes failing `Key::from_bytes`
|
||||
reject `Err(KeyInvalid)`: a malformed pool list is fatal, never a
|
||||
silent skip (ADR-003's malformed-list ruling applied at the facade
|
||||
boundary).
|
||||
- **`KeyInvalid` at the boundary:** the read ops take opaque key
|
||||
*bytes* (`&[u8]`) and convert via `Key::from_bytes` at the store
|
||||
boundary (`boundary_key`, one seam in `src/store.rs`) — malformed
|
||||
bytes reject `Error::KeyInvalid` on get/stat/read_range/has before
|
||||
any tier is consulted and never as a `None`/`false` guess; engines
|
||||
receive only the physical encoding. `list()`'s pool decode is the
|
||||
same one-way door's read face (above).
|
||||
- **Fall-through is consult order, not routing:** the read path never
|
||||
consults `tier_for`/`route` — a sub-threshold entry planted only in
|
||||
the large tier is found (test
|
||||
`read_ops_fall_through_kv_miss_to_large` asserts the *routing*
|
||||
disagrees while the read succeeds to make the distinction
|
||||
observable). kv-first is proven behaviorally by planting different
|
||||
content under the same key bytes in both tiers
|
||||
(`kv_tier_is_the_first_consult_on_every_key_addressed_op`).
|
||||
- **Miss shapes enforced (the error module's behavioral note):**
|
||||
both-tier misses are `Ok(None)` on get/stat, `false` on has,
|
||||
`Err(Missing)` on read_range — pinned per mode (dual-tier/mem-only/
|
||||
kv-only) via `assert_virgin_read_shapes`; `src/error.rs`'s
|
||||
wave-2 deferral notes updated to the in-place state.
|
||||
- **Tier dispatch plumbing:** `KvEngine`/`LargeEngine` grew
|
||||
crate-internal `get`/`has`/`size`/`list` dispatch methods returning
|
||||
engine-raw shapes (cursor enum / `Option<u64>` / `bool` /
|
||||
`Vec<Vec<u8>>`); adding a variant per enum (waves 3–5) is slotting
|
||||
in. `get`'s kv arm (and its verification) is
|
||||
`#[cfg(feature = "mem")]` — in default builds (no kv engine until
|
||||
wave 3's sqlite) the large/None arms fully cover the unconstructible
|
||||
facade.
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
Implemented the facade's read-side ops on `Store` (`src/store.rs`):
|
||||
`get(key) → Result<Option<(u64, Blob)>>` with the pinned `(len, stream)`
|
||||
shape, `stat(key) → Result<Option<EntryMeta>>` riding the trait's
|
||||
`size` probe, `read_range(key, offset, len) → Result<(Vec<u8>, [u8;32])>`
|
||||
composed through the cursor's `read_range` slice-digest seam, `has(key)`
|
||||
as the pure existence probe, and `list() → Result<Vec<Key>>` — all on
|
||||
the kv-first-then-large fall-through rule with typed key conversion at
|
||||
the facade boundary. Added the public stream type `Blob`
|
||||
(`src/facade.rs`) over the new crate-internal `AnyCursor` cursor
|
||||
dispatch and recorded the four named design points (stream shape,
|
||||
verification-on-get, list return shape, `KeyInvalid` boundary) in
|
||||
`src/store.rs` module docs; updated the wave-2 deferral notes in
|
||||
`src/error.rs` to their enforced state and the scope notes in
|
||||
`src/mem.rs`/`src/local.rs`/`src/dispatch.rs` now that the read path
|
||||
drives them. Twelve read-path tests pin the miss shapes, the
|
||||
fall-through order, both verification postures, the list union
|
||||
(including malformed-pool fatality), and `KeyInvalid` on every
|
||||
key-addressed op.
|
||||
|
||||
Verification: `cargo test` (default), `cargo test --features mem`,
|
||||
`cargo test --all-features` all green (66 lib tests green under mem);
|
||||
`cargo clippy --all-targets -- -D warnings` green in all three feature
|
||||
shapes; `cargo fmt --check` green; `cargo doc --no-deps` clean of new
|
||||
warning classes beyond the pre-existing private-intra-doc-links class.
|
||||
Reference in new issue
Block a user