feat(store): the sqlite constructor wiring — the default-build store shapes

- KvEngine::Sqlite slots in; all six kv ops dispatch; the kv cursor
  body dispatches through a new KvCursor sub-enum behind AnyCursor's
  one kv arm (the kv-arm-over-multiple-kv-engines design point,
  recorded)
- dual_tier_sqlite_local (kv = sqlite + large = local, the
  default-build production shape) and kv_only_sqlite constructors —
  both feature-gated on sqlite (default-on), both taking the
  threshold and the delete-window delay, fallible where I/O is
  fallible (Err(Error::Io) per the dual_tier_mem_local pattern)
- ensure_gc_schema runs at construction via the engine-neutral
  KvEngine call (sqlite's body creates gc_window only); the
  window-delay shipped default is the pinned
  DEFAULT_DELETE_WINDOW_DELAY (5 min, ADR-008 §2), constructor-
  tunable, Duration::ZERO legal — the store carries the value for
  the sweep's durable form to consume
- the one-window-host posture recorded in the module doc (the
  durable window hosts on the kv engine when it is SQL-backed; wave
  5's pg-lo dormancy slots in without reshaping)
- stale empty-set scaffolding updated to the landed state: the
  #[allow(unused_variables)] attrs off the KvEngine methods; the
  scope notes in store.rs, backend.rs, sqlite.rs, gc.rs
- constructor unit tests: mode/threshold/window-delay introspection,
  the GC schema observed in the fresh database (window table present,
  gc_pins never), kv-only rejection wiring unchanged, and the
  dual-tier shape round-tripping end-to-end through both tiers

Verification: cargo test 62 passed (default; 5 new tests), cargo test
--all-features 117, cargo test --features mem 117, --no-default-
features --features mem 97; cargo clippy --all-targets -- -D warnings
clean (default + all-features); cargo fmt --check clean; cargo doc
--no-deps at the pre-existing warning baseline.
This commit is contained in:
glm-5.3-flash committed 2026-10-12 04:50:22 +00:00
1 parent 51d8f6525f
commit 2993df0e36
6 files changed
+673 -64

No files matched your search

+7 -3
View File
@@ -31,9 +31,13 @@
//! the crate-internal engine-state companion seam lives in [`crate::gc`]. //! the crate-internal engine-state companion seam lives in [`crate::gc`].
//! //!
//! The `#![allow(dead_code)]` below still has real scope: `Backend` //! The `#![allow(dead_code)]` below still has real scope: `Backend`
//! trait items with no in-tree caller yet (reporting names, the kv //! trait items with no in-tree caller yet (the engines' reporting
//! engines' whole-value `put` before wave 3) plus the wave-1 seam //! names — the dispatch/conformance identity the reporting and fleet
//! tests' doubles. It shrinks as the waves land. //! waves consume) plus the wave-1 seam tests' doubles. The kv engines'
//! whole-value `put` gained its in-tree callers in wave 3 (sqlite is
//! the default build's kv engine and the facade's put seam drives the
//! kv tier through it); the allow shrinks as the remaining callers
//! land.
#![allow(dead_code)] #![allow(dead_code)]
+5 -2
View File
@@ -65,8 +65,11 @@
//! report data (`Ok(SweepReport)` — ADR-012 §1), nothing deleted. //! report data (`Ok(SweepReport)` — ADR-012 §1), nothing deleted.
//! //!
//! The `#![allow(dead_code)]` below covers the companion-seam scope: //! The `#![allow(dead_code)]` below covers the companion-seam scope:
//! the engine-state clauses are implemented by the SQL engines of //! the window family has its first in-tree caller in wave 3 (store
//! waves 3–5 and are not called yet. It shrinks as those waves land. //! construction runs `ensure_gc_schema` on the SQL-backed kv engine),
//! while `window_stage`/`window_take` wait for the sweep's durable
//! form and the fleet family waits for postgres's wave. It shrinks as
//! those land.
#![allow(dead_code)] #![allow(dead_code)]
+1 -1
View File
@@ -16,7 +16,7 @@ pub use facade::{
}; };
pub use hashing::{Algorithm, Digest}; pub use hashing::{Algorithm, Digest};
pub use keys::Key; pub use keys::Key;
pub use store::{Store, StoreMode}; pub use store::{Store, StoreMode, DEFAULT_DELETE_WINDOW_DELAY};
#[cfg(feature = "mem")] #[cfg(feature = "mem")]
pub mod mem; pub mod mem;
+33 -18
View File
@@ -44,8 +44,15 @@
//! databases included): the durability promise is verified at open, not //! databases included): the durability promise is verified at open, not
//! assumed. //! assumed.
//! //!
//! The `#![allow(dead_code)]` below has real scope: nothing constructs //! The engine is constructed by the store facade —
//! the engine until wave 3's constructor wiring (`store-sqlite-construction`). //! `dual_tier_sqlite_local` / `kv_only_sqlite` (the wave-3 constructor
//! wiring, landed) — which also runs the GC schema's engine-neutral
//! call (`ensure_gc_schema`, creating `gc_window`). The
//! `#![allow(dead_code)]` below has real scope only through that
//! wiring's cfg-gates: a build with the feature off never compiles this
//! module, and the test-only instruments (the tableless constructor,
//! the commit-failure injection, the probe helpers) sit behind
//! `#[cfg(test)]`.
#![allow(dead_code)] #![allow(dead_code)]
@@ -182,6 +189,27 @@ impl SqliteBackend {
}) })
} }
/// The sqlite_master table-existence probe — test-only: the store
/// construction test's instrument (covers every schema object,
/// not just probed reads) and the local tests' helper body.
#[cfg(test)]
pub(crate) async fn table_exists(&self, name: &str) -> Result<bool, Error> {
let conn = Arc::clone(&self.conn);
let name = name.to_owned();
blocking(move || {
let guard = conn.lock();
let probed: rusqlite::Result<bool> = guard
.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = ?1",
params![name],
|row| row.get::<_, i64>(0),
)
.map(|count| count != 0);
probed.map_err(db_err)
})
.await
}
/// The connection's live pragma value — the posture test's probe /// The connection's live pragma value — the posture test's probe
/// (`synchronous` is per-connection, so the backend's own connection /// (`synchronous` is per-connection, so the backend's own connection
/// must be asked). /// must be asked).
@@ -1054,23 +1082,10 @@ mod tests {
} }
/// Table existence probed through sqlite_master — the schema test's /// Table existence probed through sqlite_master — the schema test's
/// instrument (covers every schema object, not just probed reads). /// instrument (covers every schema object, not just probed reads); a
/// thin unwrap-wrap over the backend's helper.
async fn table_exists(backend: &SqliteBackend, name: &str) -> bool { async fn table_exists(backend: &SqliteBackend, name: &str) -> bool {
let conn = Arc::clone(&backend.conn); backend.table_exists(name).await.unwrap()
let name = name.to_owned();
blocking(move || {
let guard = conn.lock();
Ok(guard
.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = ?1",
params![name],
|row| row.get::<_, i64>(0),
)
.unwrap()
!= 0)
})
.await
.unwrap()
} }
fn stages(key: &[u8], visible_after: SystemTime) -> WindowCandidate { fn stages(key: &[u8], visible_after: SystemTime) -> WindowCandidate {
+556 -31
View File
@@ -36,13 +36,17 @@
//! - **dual-tier** (the default): kv + large, size-threshold dispatch //! - **dual-tier** (the default): kv + large, size-threshold dispatch
//! via [`crate::dispatch::route`]. The wave-2 constructible shape is //! via [`crate::dispatch::route`]. The wave-2 constructible shape is
//! `kv = mem` + `large = local` — the dispatch-coverage test shape //! `kv = mem` + `large = local` — the dispatch-coverage test shape
//! ADR-012 §6.3 names (both tiers load-bearing). Over-threshold //! ADR-012 §6.3 names (both tiers load-bearing); the wave-3
//! content routes to the large tier; at-or-below to kv. //! constructible shape is `kv = sqlite` + `large = local` — the
//! **default-build production shape** (feature `sqlite` is
//! default-on, ADR-013 §4). Over-threshold content routes to the
//! large tier; at-or-below to kv.
//! - **kv-only**: no large tier; over-threshold puts reject **at put //! - **kv-only**: no large tier; over-threshold puts reject **at put
//! time** (ADR-012 §6.2 — known-length immediately; unknown-length at //! time** (ADR-012 §6.2 — known-length immediately; unknown-length at
//! mid-stream threshold overflow). The rejection behavior itself //! mid-stream threshold overflow). The rejection behavior itself
//! lands with the put seam; the threshold this mode carries is the //! lands with the put seam; the threshold this mode carries is the
//! rejection point's value. //! rejection point's value. The wave-2 engine is mem (testing);
//! the wave-3 engine is sqlite (the durable single-node medium).
//! - **mem-only**: the mem engine alone, **no dispatch threshold in //! - **mem-only**: the mem engine alone, **no dispatch threshold in
//! effect** (everything is local and small) — a testing/embedder- //! effect** (everything is local and small) — a testing/embedder-
//! ephemeral posture, never production. Structurally distinct from //! ephemeral posture, never production. Structurally distinct from
@@ -193,16 +197,38 @@
//! [`Store::large`]), and the GC state home ([`crate::gc::GcHome`]) //! [`Store::large`]), and the GC state home ([`crate::gc::GcHome`])
//! the gc-protocol task fills. //! the gc-protocol task fills.
//! //!
//! **The durable delete window hosts on the kv engine when it is
//! SQL-backed (ADR-012 §5's one-window-host rule).** Sqlite — the
//! default build's kv engine — hosts: its constructors run the GC
//! schema's engine-neutral call (`ensure_gc_schema` on the
//! `EngineWindowState` seam) at construction, dispatched on the
//! kv-engine enum — a mem arm is structurally nothing, the in-process
//! engines execute the ADR-005 window protocol inline, so there is no
//! schema to ensure — and the SQL-backed sweep form (wave 3's
//! sweep-durable-window task) stages its candidates on the kv engine
//! the constructor built. The host branch keys off the engine
//! representation — one more kv-engine-enum match arm — so wave 5's
//! pg-lo dormancy (the dual-SQL instance's large-engine window clauses
//! are dormant when the kv engine already hosts) slots in without
//! reshaping. Both sqlite constructors carry the delete-window delay
//! (ADR-008 §2: default 5 min, constructor-tunable per deployment —
//! [`DEFAULT_DELETE_WINDOW_DELAY`]; `0` is a legal degenerate tuning,
//! execute-on-next-sweep) as the cadence knob the durable sweep form
//! consumes; the mem/local constructors' inline form has no delay in
//! effect.
//!
//! The `#![allow(dead_code)]` below marks the work still ahead, not //! The `#![allow(dead_code)]` below marks the work still ahead, not
//! the put path (which is driven below): the GC home's fleet clauses //! the put path (which is driven below): the GC home's fleet clauses
//! and the kv engine set are empty in default builds (mem is //! and the durable window's cadence knob (until the sweep's durable
//! feature-gated; sqlite is wave 3). The allow shrinks as those land. //! form consumes it) hold the real scope. The kv engine set is no
//! longer empty in default builds — sqlite landed in wave 3 — and the
//! empty-set `allow`s came off with it.
#![allow(dead_code)] #![allow(dead_code)]
use std::collections::BTreeSet; use std::collections::BTreeSet;
use std::sync::Arc; use std::sync::Arc;
use std::time::SystemTime; use std::time::{Duration, SystemTime};
use tokio::io::{AsyncRead, AsyncReadExt}; use tokio::io::{AsyncRead, AsyncReadExt};
@@ -212,6 +238,8 @@ use crate::facade::{
Batch, Blob, EntryMeta, GcAbortCause, LivenessSource, Pin, ProtectCallback, Registration, Batch, Blob, EntryMeta, GcAbortCause, LivenessSource, Pin, ProtectCallback, Registration,
SweepReport, SweepReport,
}; };
#[cfg(feature = "sqlite")]
use crate::gc::EngineWindowState;
use crate::gc::{Arbitration, GcHome}; use crate::gc::{Arbitration, GcHome};
use crate::hashing::{Algorithm, BlobHasher}; use crate::hashing::{Algorithm, BlobHasher};
use crate::keys::Key; use crate::keys::Key;
@@ -219,14 +247,26 @@ use crate::local::{LocalBackend, LocalCursor, LocalStaging};
#[cfg(feature = "mem")] #[cfg(feature = "mem")]
use crate::mem::{MemBackend, MemCursor}; use crate::mem::{MemBackend, MemCursor};
#[cfg(feature = "sqlite")]
use crate::sqlite::{SqliteBackend, SqliteCursor};
use crate::Error; use crate::Error;
#[cfg(feature = "mem")] #[cfg(any(feature = "mem", feature = "sqlite"))]
use std::path::PathBuf; use std::path::PathBuf;
use self::StoreMode::{DualTier, KvOnly, MemOnly}; use self::StoreMode::{DualTier, KvOnly, MemOnly};
/// The shipped default **delete-window delay** (ADR-008 §2): the
/// staged candidates in the durable window become visible for
/// execution this long after the sweep staged them — 5 minutes,
/// constructor-tunable per deployment. `Duration::ZERO` is a legal
/// degenerate tuning (execute-on-next-sweep; the test tuning). The
/// sqlite constructors take the delay explicitly and their docs point
/// here as the shipped default; the mem/local inline form has no
/// delay in effect and the constant is irrelevant to it.
pub const DEFAULT_DELETE_WINDOW_DELAY: Duration = Duration::from_secs(5 * 60);
/// The store instance — the public facade above the tier engines /// The store instance — the public facade above the tier engines
/// (ADR-011 §1's vocabulary: *one running instantiation of the store /// (ADR-011 §1's vocabulary: *one running instantiation of the store
/// facade — one engine set per the constructor mode, plus dispatch and /// facade — one engine set per the constructor mode, plus dispatch and
@@ -248,6 +288,13 @@ pub struct Store {
/// threshold in effect" is the mode's ruling (ADR-011 §4), not a /// threshold in effect" is the mode's ruling (ADR-011 §4), not a
/// value absence. /// value absence.
threshold: u64, threshold: u64,
/// The delete-window delay the constructor took (ADR-008 §2) — the
/// durable window's cadence knob the sweep's SQL-backed form
/// consumes (`window_stage`'s `visible_after = mark_end + window`
/// on the `EngineWindowState` seam). The mem/local inline form
/// has no delay in effect: those constructors carry
/// [`DEFAULT_DELETE_WINDOW_DELAY`], never consulted.
window_delay: Duration,
/// The constructor mode; the dispatch/put/read/gc paths branch on it. /// The constructor mode; the dispatch/put/read/gc paths branch on it.
mode: StoreMode, mode: StoreMode,
/// The GC state home (ADR-010 §2: the store core owns GC state) /// The GC state home (ADR-010 §2: the store core owns GC state)
@@ -271,11 +318,18 @@ pub enum StoreMode {
/// The kv-tier engine set — the closed per-tier engine enum of the /// The kv-tier engine set — the closed per-tier engine enum of the
/// engine-representation design point. Feature-gated variants slot in /// engine-representation design point. Feature-gated variants slot in
/// as their engine waves land: sqlite (wave 3), postgres (wave 4). /// as their engine waves land: sqlite (wave 3 — landed, the default),
/// postgres (wave 4).
pub(crate) enum KvEngine { pub(crate) enum KvEngine {
/// The contract-reference ephemeral engine (ADR-011 §2). /// The contract-reference ephemeral engine (ADR-011 §2).
#[cfg(feature = "mem")] #[cfg(feature = "mem")]
Mem(MemBackend), Mem(MemBackend),
/// The default build's kv engine — the first SQL-backed engine,
/// the durable delete window's host (ADR-012 §5's one-window-host
/// rule: the window is the kv engine's when the kv engine is
/// SQL-backed).
#[cfg(feature = "sqlite")]
Sqlite(SqliteBackend),
} }
/// The large-tier engine set. `local` is ungated (dependency-free, /// The large-tier engine set. `local` is ungated (dependency-free,
@@ -285,18 +339,56 @@ pub(crate) enum LargeEngine {
Local(LocalBackend), Local(LocalBackend),
} }
/// **The kv-arm shape over multiple kv engines (named design point):
/// a `KvCursor` sub-enum behind [`AnyCursor`]'s one kv arm.** The
/// alternative — one `AnyCursor` variant per kv engine — would copy
/// the kv materializing arm (and its consumers: the verification-on-
/// get body, `Blob`'s drive) per engine without adding a behavior;
/// the sub-enum keeps [`AnyCursor`] a two-arm tier dispatch (kv /
/// large) while kv engines multiply inside this one. Recorded for the
/// review gate: postgres (wave 4) slots one arm in here.
pub(crate) enum KvCursor {
/// The contract-reference ephemeral body (ADR-011 §2).
#[cfg(feature = "mem")]
Mem(MemCursor),
/// The default build's materializing body: the row's value loaded
/// once, reads in-memory slices (the sqlite engine's record).
#[cfg(feature = "sqlite")]
Sqlite(SqliteCursor),
}
impl ReadCursor for KvCursor {
fn len(&self) -> u64 {
match self {
#[cfg(feature = "mem")]
Self::Mem(cursor) => cursor.len(),
#[cfg(feature = "sqlite")]
Self::Sqlite(cursor) => cursor.len(),
}
}
async fn read_at(&self, offset: u64, len: u64) -> Result<Vec<u8>, Error> {
match self {
#[cfg(feature = "mem")]
Self::Mem(cursor) => cursor.read_at(offset, len).await,
#[cfg(feature = "sqlite")]
Self::Sqlite(cursor) => cursor.read_at(offset, len).await,
}
}
}
/// The read path's cursor-dispatch enum (ADR-010 §1's "store core /// The read path's cursor-dispatch enum (ADR-010 §1's "store core
/// programs the abstraction"): tier dispatch over the engine cursor /// programs the abstraction"): tier dispatch over the engine cursor
/// bodies — the kv tier's materializing body and the large tier's /// bodies — the kv tier's materializing body and the large tier's
/// streaming body — behind one crate-internal value that [`Blob`] holds /// streaming body — behind one crate-internal value that [`Blob`] holds
/// and drives. Crate-internal: engine cursors never appear on the /// and drives. Crate-internal: engine cursors never appear on the
/// public surface; the facade's stream shape is this enum's uniform /// public surface; the facade's stream shape is this enum's uniform
/// [`ReadCursor`] view (`Blob`). /// [`ReadCursor`] view (`Blob`). The kv bodies live behind
/// [`KvCursor`] (the recorded design point above).
pub(crate) enum AnyCursor { pub(crate) enum AnyCursor {
/// The kv body family: the whole value materialized in memory /// The kv body family: the whole value materialized in memory
/// (bounded by tier policy). /// (bounded by tier policy).
#[cfg(feature = "mem")] Kv(KvCursor),
Kv(MemCursor),
/// The large body family: a pread cursor over the committed file. /// The large body family: a pread cursor over the committed file.
Large(LocalCursor), Large(LocalCursor),
} }
@@ -304,7 +396,6 @@ pub(crate) enum AnyCursor {
impl ReadCursor for AnyCursor { impl ReadCursor for AnyCursor {
fn len(&self) -> u64 { fn len(&self) -> u64 {
match self { match self {
#[cfg(feature = "mem")]
Self::Kv(cursor) => cursor.len(), Self::Kv(cursor) => cursor.len(),
Self::Large(cursor) => cursor.len(), Self::Large(cursor) => cursor.len(),
} }
@@ -312,7 +403,6 @@ impl ReadCursor for AnyCursor {
async fn read_at(&self, offset: u64, len: u64) -> Result<Vec<u8>, Error> { async fn read_at(&self, offset: u64, len: u64) -> Result<Vec<u8>, Error> {
match self { match self {
#[cfg(feature = "mem")]
Self::Kv(cursor) => cursor.read_at(offset, len).await, Self::Kv(cursor) => cursor.read_at(offset, len).await,
Self::Large(cursor) => cursor.read_at(offset, len).await, Self::Large(cursor) => cursor.read_at(offset, len).await,
} }
@@ -367,50 +457,79 @@ impl StagingCursor for AnyStaging {
} }
impl KvEngine { impl KvEngine {
#[allow(unused_variables)] // kv engine set empty until wave 3 (sqlite): no-mem builds never reference `key`/`value`
async fn put(&self, key: &[u8], value: &[u8]) -> Result<(), Error> { async fn put(&self, key: &[u8], value: &[u8]) -> Result<(), Error> {
match *self { match self {
#[cfg(feature = "mem")] #[cfg(feature = "mem")]
KvEngine::Mem(ref engine) => engine.put(key, value).await, KvEngine::Mem(engine) => engine.put(key, value).await,
#[cfg(feature = "sqlite")]
KvEngine::Sqlite(engine) => engine.put(key, value).await,
} }
} }
#[allow(unused_variables)] // kv engine set empty until wave 3 (sqlite): no-mem builds never reference `key`
async fn get(&self, key: &[u8]) -> Result<Option<AnyCursor>, Error> { async fn get(&self, key: &[u8]) -> Result<Option<AnyCursor>, Error> {
match *self { match self {
#[cfg(feature = "mem")] #[cfg(feature = "mem")]
KvEngine::Mem(ref engine) => Ok(engine.get(key).await?.map(AnyCursor::Kv)), KvEngine::Mem(engine) => Ok(engine
.get(key)
.await?
.map(|c| AnyCursor::Kv(KvCursor::Mem(c)))),
#[cfg(feature = "sqlite")]
KvEngine::Sqlite(engine) => Ok(engine
.get(key)
.await?
.map(|c| AnyCursor::Kv(KvCursor::Sqlite(c)))),
}
}
/// The GC schema's engine-neutral store-core call (ADR-010 §2's
/// companion seam; ADR-012 §5's applicability table), run at
/// construction: a SQL-backed kv engine creates its window
/// table(s) — sqlite's body creates `gc_window` only — and an
/// in-process kv engine is structurally nothing (the
/// local/mem row of the table: those engines execute the window
/// protocol inline, ADR-005, and carry no GC schema).
async fn ensure_gc_schema(&self) -> Result<(), Error> {
match self {
#[cfg(feature = "mem")]
KvEngine::Mem(_) => Ok(()),
#[cfg(feature = "sqlite")]
KvEngine::Sqlite(engine) => engine.ensure_gc_schema().await,
} }
} }
#[allow(unused_variables)] // same empty-set scope as `get`
async fn has(&self, key: &[u8]) -> Result<bool, Error> { async fn has(&self, key: &[u8]) -> Result<bool, Error> {
match *self { match self {
#[cfg(feature = "mem")] #[cfg(feature = "mem")]
KvEngine::Mem(ref engine) => engine.has(key).await, KvEngine::Mem(engine) => engine.has(key).await,
#[cfg(feature = "sqlite")]
KvEngine::Sqlite(engine) => engine.has(key).await,
} }
} }
#[allow(unused_variables)] // same empty-set scope as `get`
async fn size(&self, key: &[u8]) -> Result<Option<u64>, Error> { async fn size(&self, key: &[u8]) -> Result<Option<u64>, Error> {
match *self { match self {
#[cfg(feature = "mem")] #[cfg(feature = "mem")]
KvEngine::Mem(ref engine) => engine.size(key).await, KvEngine::Mem(engine) => engine.size(key).await,
#[cfg(feature = "sqlite")]
KvEngine::Sqlite(engine) => engine.size(key).await,
} }
} }
#[allow(unused_variables)] // same empty-set scope as `get`
async fn delete(&self, key: &[u8]) -> Result<(), Error> { async fn delete(&self, key: &[u8]) -> Result<(), Error> {
match *self { match self {
#[cfg(feature = "mem")] #[cfg(feature = "mem")]
KvEngine::Mem(ref engine) => engine.delete(key).await, KvEngine::Mem(engine) => engine.delete(key).await,
#[cfg(feature = "sqlite")]
KvEngine::Sqlite(engine) => engine.delete(key).await,
} }
} }
async fn list(&self) -> Result<Vec<Vec<u8>>, Error> { async fn list(&self) -> Result<Vec<Vec<u8>>, Error> {
match *self { match self {
#[cfg(feature = "mem")] #[cfg(feature = "mem")]
KvEngine::Mem(ref engine) => engine.list().await, KvEngine::Mem(engine) => engine.list().await,
#[cfg(feature = "sqlite")]
KvEngine::Sqlite(engine) => engine.list().await,
} }
} }
} }
@@ -483,6 +602,7 @@ impl Store {
kv: KvEngine::Mem(MemBackend::default()), kv: KvEngine::Mem(MemBackend::default()),
large: None, large: None,
threshold: crate::dispatch::DEFAULT_DISPATCH_THRESHOLD, threshold: crate::dispatch::DEFAULT_DISPATCH_THRESHOLD,
window_delay: DEFAULT_DELETE_WINDOW_DELAY,
mode: MemOnly, mode: MemOnly,
gc: GcHome::default(), gc: GcHome::default(),
} }
@@ -500,6 +620,7 @@ impl Store {
kv: KvEngine::Mem(MemBackend::default()), kv: KvEngine::Mem(MemBackend::default()),
large: None, large: None,
threshold, threshold,
window_delay: DEFAULT_DELETE_WINDOW_DELAY,
mode: KvOnly, mode: KvOnly,
gc: GcHome::default(), gc: GcHome::default(),
} }
@@ -526,6 +647,82 @@ impl Store {
kv: KvEngine::Mem(MemBackend::default()), kv: KvEngine::Mem(MemBackend::default()),
large: Some(LargeEngine::Local(LocalBackend::new(root).await?)), large: Some(LargeEngine::Local(LocalBackend::new(root).await?)),
threshold, threshold,
window_delay: DEFAULT_DELETE_WINDOW_DELAY,
mode: DualTier,
gc: GcHome::default(),
})
}
/// **kv-only mode** (ADR-011 §4; ADR-012 §6.2) with the sqlite kv
/// engine — the durable single-node kv-only shape: no large tier;
/// `threshold` is the put-time rejection point (known-length puts
/// over it reject immediately, unknown-length puts at mid-stream
/// threshold overflow — the put seam's behavior, identical to the
/// mem kv-only wiring; zero residue on either rejection). Routing
/// in this mode is kv for every length.
///
/// Opening (or creating) the database file is fallible I/O; the
/// constructor also runs the GC schema's engine-neutral call
/// (`ensure_gc_schema` — sqlite's body creates `gc_window` only,
/// ADR-012 §5), so a construction failure is `Err(Error::Io)` per
/// the fallible-constructor pattern `dual_tier_mem_local` set.
///
/// `window_delay` (ADR-008 §2) is the durable window's cadence
/// knob — [`DEFAULT_DELETE_WINDOW_DELAY`] is the shipped default,
/// `Duration::ZERO` the legal degenerate tuning. Not fleet-
/// anything: a sqlite fleet is a recorded non-option (ADR-010 §3).
#[cfg(feature = "sqlite")]
pub async fn kv_only_sqlite(
database: impl Into<PathBuf>,
threshold: u64,
window_delay: Duration,
) -> Result<Store, Error> {
let kv = KvEngine::Sqlite(SqliteBackend::new(database).await?);
kv.ensure_gc_schema().await?;
Ok(Store {
kv,
large: None,
threshold,
window_delay,
mode: KvOnly,
gc: GcHome::default(),
})
}
/// **dual-tier mode** (the default, ADR-011 §4) in the default-build
/// production shape — `kv = sqlite` + `large = local` (feature
/// `sqlite` is default-on, ADR-013 §4): size-threshold dispatch at
/// `threshold` (constructor-tunable;
/// [`crate::dispatch::DEFAULT_DISPATCH_THRESHOLD`] is the shipped
/// default), at-or-below on the kv tier, over-threshold staged into
/// the large tier.
///
/// Fallible on two accounts (file I/O): opening/creating the pool
/// database — which also runs the GC schema's engine-neutral call
/// (`ensure_gc_schema`; sqlite's body creates `gc_window` only,
/// ADR-012 §5's one-window-host rule: the durable window is the kv
/// engine's when the kv engine is SQL-backed) — and creating the
/// local root. A construction failure is `Err(Error::Io)` per the
/// fallible-constructor pattern `dual_tier_mem_local` set.
///
/// `window_delay` (ADR-008 §2) is the durable window's cadence
/// knob — [`DEFAULT_DELETE_WINDOW_DELAY`] is the shipped default,
/// `Duration::ZERO` the legal degenerate tuning. Not fleet-
/// anything: a sqlite fleet is a recorded non-option (ADR-010 §3).
#[cfg(feature = "sqlite")]
pub async fn dual_tier_sqlite_local(
database: impl Into<PathBuf>,
root: impl Into<PathBuf>,
threshold: u64,
window_delay: Duration,
) -> Result<Store, Error> {
let kv = KvEngine::Sqlite(SqliteBackend::new(database).await?);
kv.ensure_gc_schema().await?;
Ok(Store {
kv,
large: Some(LargeEngine::Local(LocalBackend::new(root).await?)),
threshold,
window_delay,
mode: DualTier, mode: DualTier,
gc: GcHome::default(), gc: GcHome::default(),
}) })
@@ -548,6 +745,16 @@ impl Store {
} }
} }
/// The delete-window delay the constructor took (ADR-008 §2) —
/// crate-internal: the sweep's SQL-backed form consumes it when
/// staging candidates (`visible_after = mark_end + window`);
/// introspection for the construction tests. `Duration::ZERO`
/// (a constructor-passed value) is the legal degenerate tuning —
/// execute-on-next-sweep.
pub(crate) fn window_delay(&self) -> Duration {
self.window_delay
}
/// **Write-side tier selection for a given content length** — the /// **Write-side tier selection for a given content length** — the
/// dispatch seam's routing half (the put seam programs through /// dispatch seam's routing half (the put seam programs through
/// this; read paths do not consult it). Dual-tier: /// this; read paths do not consult it). Dual-tier:
@@ -621,7 +828,6 @@ impl Store {
}; };
match cursor { match cursor {
#[cfg(feature = "mem")]
AnyCursor::Kv(kv) => { AnyCursor::Kv(kv) => {
let value = kv.read_at(0, kv.len()).await?; let value = kv.read_at(0, kv.len()).await?;
let derived = key.algorithm().derive(&value, value.len() as u64); let derived = key.algorithm().derive(&value, value.len() as u64);
@@ -1519,7 +1725,10 @@ mod tests {
/// verification postures are observed. /// verification postures are observed.
async fn plant_kv(store: &Store, key: &[u8], value: &[u8]) { async fn plant_kv(store: &Store, key: &[u8], value: &[u8]) {
match store.kv() { match store.kv() {
#[cfg(feature = "mem")]
KvEngine::Mem(engine) => engine.put(key, value).await.expect("kv plant"), KvEngine::Mem(engine) => engine.put(key, value).await.expect("kv plant"),
#[cfg(feature = "sqlite")]
KvEngine::Sqlite(engine) => engine.put(key, value).await.expect("kv plant"),
} }
} }
@@ -3486,3 +3695,319 @@ mod tests {
assert_eq!(blob.read_at(0, 8).await.unwrap(), Vec::<u8>::new()); assert_eq!(blob.read_at(0, 8).await.unwrap(), Vec::<u8>::new());
} }
} }
/// The wave-3 constructor wiring's tests (feature `sqlite`): the
/// sqlite modes construct and introspect correctly, the GC schema ran
/// at construction, the window-delay default is the pinned constant
/// and is constructor-tunable, the kv-only rejection wiring is the
/// mode's put seam unchanged, and the default-build production shape
/// (kv = sqlite + large = local) round-trips end-to-end through both
/// tiers. All-features builds share this module (mem on alongside):
/// the engine accessors carry cfg arms for that.
#[cfg(all(test, feature = "sqlite"))]
mod sqlite_construction_tests {
use super::*;
use std::fs;
use std::io::Cursor;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::{SystemTime, UNIX_EPOCH};
/// A fresh temp database file (and sibling stage/root dir) per
/// test; Drop removes the file tree and the WAL sidecars
/// idempotently (the sqlite engine's test helper's shape).
struct TempSite {
database: PathBuf,
root: PathBuf,
}
impl TempSite {
fn path(&self) -> &Path {
&self.root
}
fn drop_sidecar(&self, suffix: &str) {
let base = self.database.to_string_lossy().into_owned();
let _ = fs::remove_file(format!("{base}{suffix}"));
}
}
impl Drop for TempSite {
fn drop(&mut self) {
self.drop_sidecar("-wal");
self.drop_sidecar("-shm");
let _ = fs::remove_file(&self.database);
let _ = fs::remove_dir_all(&self.root);
}
}
fn temp_site(tag: &str) -> TempSite {
static SEQ: AtomicU64 = AtomicU64::new(0);
let seq = SEQ.fetch_add(1, Ordering::Relaxed);
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0);
let base = std::env::temp_dir().join(format!(
"alkblobs-store-sqlite-{tag}-{}-{nanos:x}-{seq}",
std::process::id()
));
fs::create_dir_all(&base).expect("temp site created");
TempSite {
database: base.join("pool.db"),
root: base.join("local-root"),
}
}
fn key_for(content: &[u8]) -> Key {
Key::derive(content, content.len() as u64)
}
/// The sqlite kv engine behind the constructed store — the schema
/// observations' instrument.
fn sqlite_engine(store: &Store) -> &SqliteBackend {
match store.kv() {
#[cfg(feature = "sqlite")]
KvEngine::Sqlite(engine) => engine,
#[cfg(feature = "mem")]
KvEngine::Mem(_) => panic!("these tests construct sqlite stores"),
}
}
/// The engine-neutral GC-schema call ran at construction: the pool
/// table exists (the engine's own open), `gc_window` exists (the
/// store-core call), `gc_pins` never (sqlite's applicability row,
/// ADR-012 §5's one-window-host rule — the kv engine hosts the
/// window and nothing else).
async fn assert_gc_schema_ran(store: &Store) {
let engine = sqlite_engine(store);
assert!(
engine.table_exists("blobs").await.unwrap(),
"the pool table exists at construction"
);
assert!(
engine.table_exists("gc_window").await.unwrap(),
"ensure_gc_schema ran at construction: the window table exists in the fresh database"
);
assert!(
!engine.table_exists("gc_pins").await.unwrap(),
"gc_pins is never created on sqlite (ADR-012 §5)"
);
}
/// The dual-tier construction: both tiers present, mode/threshold
/// introspection exact, boundary routing deterministic, and the
/// GC schema ran.
#[tokio::test]
async fn dual_tier_sqlite_local_constructs_and_routes() {
let site = temp_site("dual-tier");
let store = Store::dual_tier_sqlite_local(
&site.database,
site.path(),
1024,
DEFAULT_DELETE_WINDOW_DELAY,
)
.await
.expect("dual-tier sqlite+local constructs");
assert_eq!(store.mode(), DualTier);
assert_eq!(store.threshold(), Some(1024));
assert_eq!(store.window_delay(), DEFAULT_DELETE_WINDOW_DELAY);
assert!(store.large().is_some(), "both tiers present");
for (len, tier) in [
(0u64, Tier::Kv),
(1023, Tier::Kv),
(1024, Tier::Kv),
(1025, Tier::Large),
(u64::MAX, Tier::Large),
] {
assert_eq!(
store.tier_for(len),
tier,
"length {len} routes deterministically"
);
}
assert_gc_schema_ran(&store).await;
}
/// The kv-only construction: no large tier, the threshold is the
/// rejection point's carrier, and the rejection wiring is the put
/// seam's unchanged — known-length over-threshold rejects
/// immediately (zero residue), unknown-length at mid-stream
/// overflow, and exactly-at-threshold puts land.
#[tokio::test]
async fn kv_only_sqlite_constructs_and_rejects_over_threshold_puts() {
let site = temp_site("kv-only");
let store = Store::kv_only_sqlite(&site.database, 64, DEFAULT_DELETE_WINDOW_DELAY)
.await
.expect("kv-only sqlite constructs");
assert_eq!(store.mode(), KvOnly);
assert_eq!(store.threshold(), Some(64));
assert_eq!(store.window_delay(), DEFAULT_DELETE_WINDOW_DELAY);
assert!(store.large().is_none(), "kv-only has no large tier");
assert_gc_schema_ran(&store).await;
for len in [0u64, 63, 64, 65, u64::MAX] {
assert_eq!(
store.tier_for(len),
Tier::Kv,
"kv-only routes every length kv"
);
}
let err = store
.put(Some(65), Cursor::new(&b"never read"[..]))
.await
.unwrap_err();
assert!(
err.to_string().contains("kv-only mode rejects"),
"the immediate rejection is the mode's: {err}"
);
assert_eq!(
store.list().await.unwrap(),
Vec::<Key>::new(),
"zero residue"
);
let streamed = [3u8; 100];
let err = store
.put(None, Cursor::new(&streamed[..]))
.await
.unwrap_err();
assert!(
err.to_string()
.contains("kv-only mode rejects an over-threshold put"),
"the overflow rejection is the mode's, not the stream's: {err}"
);
assert_eq!(
store.list().await.unwrap(),
Vec::<Key>::new(),
"zero residue"
);
let at_threshold = vec![9u8; 64];
let (key, pin) = store
.put(Some(64), Cursor::new(&at_threshold[..]))
.await
.expect("exactly-at-threshold known put lands");
assert_eq!(key, key_for(&at_threshold));
assert!(store.has(&key.as_bytes()).await.unwrap());
drop(pin);
}
/// The window-delay parameter: the shipped default is the pinned
/// 5-minute constant, the constructor's value passes through the
/// store's introspection, and `Duration::ZERO` is a legal
/// degenerate tuning (execute-on-next-sweep).
#[tokio::test]
async fn window_delay_default_is_pinned_and_constructor_tunable() {
assert_eq!(
DEFAULT_DELETE_WINDOW_DELAY,
Duration::from_secs(5 * 60),
"the shipped default is 5 minutes (ADR-008 §2)"
);
let site = temp_site("delay-default");
let store = Store::dual_tier_sqlite_local(
&site.database,
site.path(),
64,
DEFAULT_DELETE_WINDOW_DELAY,
)
.await
.expect("constructs at the shipped default");
assert_eq!(store.window_delay(), Duration::from_secs(300));
let site = temp_site("delay-tuned");
let store = Store::kv_only_sqlite(&site.database, 64, Duration::from_secs(90))
.await
.expect("constructs at a custom tuning");
assert_eq!(store.window_delay(), Duration::from_secs(90));
let site = temp_site("delay-zero");
let store = Store::dual_tier_sqlite_local(&site.database, site.path(), 64, Duration::ZERO)
.await
.expect("the degenerate zero tuning is legal");
assert_eq!(store.window_delay(), Duration::ZERO);
let site = temp_site("delay-long");
Store::kv_only_sqlite(&site.database, 64, Duration::from_secs(6000))
.await
.expect("a long delay is a legal tuning too");
}
/// The default-build production shape round-trips end-to-end: a
/// sub-threshold put lands on the kv tier (sqlite), an
/// over-threshold put stages into the large tier (local), both
/// get back byte-identical, and the pool list is the sorted union
/// of both tiers.
#[tokio::test]
async fn dual_tier_sqlite_shape_round_trips_end_to_end_through_both_tiers() {
let site = temp_site("round-trip");
let store = Store::dual_tier_sqlite_local(&site.database, site.path(), 64, Duration::ZERO)
.await
.expect("dual-tier constructs");
let small = b"a sub-threshold blob";
let (key, pin) = store
.put(Some(small.len() as u64), Cursor::new(&small[..]))
.await
.expect("kv put publishes");
assert_eq!(key, key_for(small));
assert!(
store.kv().has(&key.as_bytes()).await.unwrap(),
"landed on the kv tier (sqlite)"
);
assert!(
!store.large().unwrap().has(&key.as_bytes()).await.unwrap(),
"no large-tier copy on the kv route"
);
let (len, blob) = store
.get(&key.as_bytes())
.await
.unwrap()
.expect("entry present");
assert_eq!(len, small.len() as u64);
assert_eq!(blob.read_at(0, len).await.unwrap(), &small[..]);
drop(pin);
let big = vec![42u8; 200];
let (key, pin) = store
.put(Some(200), Cursor::new(&big[..]))
.await
.expect("large staged put publishes");
assert_eq!(key, key_for(&big));
assert!(
store.large().unwrap().has(&key.as_bytes()).await.unwrap(),
"landed on the large tier (commit-rename)"
);
assert!(
!store.kv().has(&key.as_bytes()).await.unwrap(),
"no kv copy on the large route"
);
let (len, blob) = store
.get(&key.as_bytes())
.await
.unwrap()
.expect("entry present");
assert_eq!(len, 200);
assert_eq!(blob.read_at(0, len).await.unwrap(), big);
drop(pin);
let listed = store.list().await.expect("list succeeds");
assert_eq!(listed.len(), 2, "one entry per tier, both enumerated");
for window in listed.windows(2) {
assert!(window[0] < window[1], "list is sorted");
}
// A re-put dedups into the existing entry (CAS first-writer-
// wins, through the facade's put).
let (_, pin) = store
.put(Some(small.len() as u64), Cursor::new(&small[..]))
.await
.expect("dedup re-put");
assert_eq!(store.list().await.unwrap().len(), 2, "still two entries");
drop(pin);
}
}
+71 -9
View File
@@ -1,7 +1,7 @@
--- ---
id: store-sqlite-construction id: store-sqlite-construction
name: sqlite constructor wiring — the default-build store shapes name: sqlite constructor wiring — the default-build store shapes
status: pending status: completed
depends_on: [engine-sqlite, engine-sqlite-window] depends_on: [engine-sqlite, engine-sqlite-window]
scope: narrow scope: narrow
risk: low risk: low
@@ -61,19 +61,19 @@ a named constructor is slotting in"):
## Acceptance Criteria ## Acceptance Criteria
- [ ] `KvEngine::Sqlite` slots in; all six kv ops dispatch; the kv - [x] `KvEngine::Sqlite` slots in; all six kv ops dispatch; the kv
cursor body dispatches (design point recorded) cursor body dispatches (design point recorded)
- [ ] `dual_tier_sqlite_local` + `kv_only_sqlite` constructors exist - [x] `dual_tier_sqlite_local` + `kv_only_sqlite` constructors exist
behind feature `sqlite`, take threshold + window delay, fallible behind feature `sqlite`, take threshold + window delay, fallible
where I/O is fallible where I/O is fallible
- [ ] `ensure_gc_schema` runs at construction; the window-delay - [x] `ensure_gc_schema` runs at construction; the window-delay
shipped default is 5 min (pinned constant) and shipped default is 5 min (pinned constant) and
constructor-tunable; `0` legal constructor-tunable; `0` legal
- [ ] The one-window-host posture recorded (sqlite hosts when the kv - [x] The one-window-host posture recorded (sqlite hosts when the kv
engine is SQL-backed) engine is SQL-backed)
- [ ] The stale empty-set scaffolding/allow notes updated to the - [x] The stale empty-set scaffolding/allow notes updated to the
landed state landed state
- [ ] `cargo test` (default) green — the default build constructs and - [x] `cargo test` (default) green — the default build constructs and
exercises sqlite stores; `cargo clippy --all-targets -- -D exercises sqlite stores; `cargo clippy --all-targets -- -D
warnings` and `cargo fmt --check` green warnings` and `cargo fmt --check` green
@@ -88,8 +88,70 @@ a named constructor is slotting in"):
## Notes ## Notes
> To be filled by implementation agent - **KvCursor sub-enum (the recorded design point).** The kv cursor
dispatch went with a `KvCursor` sub-enum behind `AnyCursor`'s one
`Kv` arm (rather than a per-engine `AnyCursor::Kv*` variant pair):
per-engine variants would copy the kv materializing arm and its
consumers (the verification-on-get body, `Blob`'s drive) per engine
without adding a behavior; the sub-enum keeps `AnyCursor` a two-arm
tier dispatch. Postgres (wave 4) slots one arm into `KvCursor`.
- **`KvEngine::ensure_gc_schema` — the engine-neutral store-core
call** lives on the kv-tier enum itself: the sqlite arm forwards to
the `EngineWindowState` clause; the mem arm is structurally nothing
(the in-process engines execute the ADR-005 window protocol inline
and carry no GC schema). Both sqlite constructors dispatch through
it — the store core calls the kv engine, never a typed sqlite path,
which is the recording shape the one-window-host posture wanted.
- **The window delay** is a named shipped constant
(`DEFAULT_DELETE_WINDOW_DELAY`, 5 min, ADR-008 §2) re-exported from
`lib.rs`; both sqlite constructors take it explicitly
(`Duration`, `Duration::ZERO` the legal degenerate tuning) and the
store carries it as a `window_delay` field with a crate-internal
accessor (`Store::window_delay`) for the durable sweep form
(`store-sweep-durable-window`) to consume. The mem/local
constructors are signature-unchanged; they carry the constant,
never consulted (the inline form has no delay in effect — same
posture as the mem-only threshold field).
- **Fallibility split**: `dual_tier_sqlite_local(database, root,
threshold, window_delay)` is fallible on the database file I/O,
`ensure_gc_schema`, *and* the local root creation (all
`Err(Error::Io)` via `?`); `kv_only_sqlite(database, threshold,
window_delay)` on the first two. WAL engagement failure (e.g.
`:memory:`) is a construction error, per the engine's recorded
posture.
- `SqliteBackend` gained a `#[cfg(test)] table_exists` probe (the
sqlite_master instrument) so the store tests can observe the
schema from the facade side; sqlite.rs's own test helper now
delegates to it.
- The stale empty-set scaffolding is updated to the landed state:
the `#[allow(unused_variables)]` attrs came off the `KvEngine`
methods; the scope notes in store.rs, backend.rs
(whole-value `put` now has its in-tree callers), sqlite.rs
(nothing-constructs-the-engine), and gc.rs (the window family's
first caller) reflect the landed state. `--no-default-features`
builds (kv set genuinely empty again) still compile under their
own cfg arithmetic — verified along with the default, mem-only,
and all-features builds.
## Summary ## Summary
> To be filled on completion Sqlite slotted into the facade's engine representation (wave 3
construction; commit pending at write time): `KvEngine::Sqlite` with
all six kv-op dispatch arms, the `KvCursor` sub-enum carrying the
sqlite materializing body behind `AnyCursor`'s kv arm (design point
recorded), the two named constructors (`dual_tier_sqlite_local` — the
default-build production shape, and `kv_only_sqlite`) taking
threshold + the constructor-tunable delete-window delay
(`DEFAULT_DELETE_WINDOW_DELAY` pinned at 5 min, `0` legal),
`ensure_gc_schema` running at construction through the engine-neutral
`KvEngine` call, the one-window-host posture recorded in the module
doc (the durable window is the kv engine's when the kv engine is
SQL-backed — wave 5's pg-lo dormancy slots in without reshaping),
and all stale empty-set allow/scope notes updated to the landed
state. Verification: `cargo test` 62 passing (default build — the
5 new sqlite-construction tests construct, introspect, schema-probe,
rejection-check, and end-to-end round-trip both tiers),
`cargo test --all-features` 117, `cargo test --features mem` 117,
`--no-default-features --features mem` 97, clippy `-D warnings`
(default + all-features) clean, `cargo fmt --check` clean, and
`cargo doc --no-deps` at the pre-existing warning baseline.