feat(store): the sqlite constructor wiring — the default-build store shapes
- KvEngine::Sqlite slots in; all six kv ops dispatch; the kv cursor body dispatches through a new KvCursor sub-enum behind AnyCursor's one kv arm (the kv-arm-over-multiple-kv-engines design point, recorded) - dual_tier_sqlite_local (kv = sqlite + large = local, the default-build production shape) and kv_only_sqlite constructors — both feature-gated on sqlite (default-on), both taking the threshold and the delete-window delay, fallible where I/O is fallible (Err(Error::Io) per the dual_tier_mem_local pattern) - ensure_gc_schema runs at construction via the engine-neutral KvEngine call (sqlite's body creates gc_window only); the window-delay shipped default is the pinned DEFAULT_DELETE_WINDOW_DELAY (5 min, ADR-008 §2), constructor- tunable, Duration::ZERO legal — the store carries the value for the sweep's durable form to consume - the one-window-host posture recorded in the module doc (the durable window hosts on the kv engine when it is SQL-backed; wave 5's pg-lo dormancy slots in without reshaping) - stale empty-set scaffolding updated to the landed state: the #[allow(unused_variables)] attrs off the KvEngine methods; the scope notes in store.rs, backend.rs, sqlite.rs, gc.rs - constructor unit tests: mode/threshold/window-delay introspection, the GC schema observed in the fresh database (window table present, gc_pins never), kv-only rejection wiring unchanged, and the dual-tier shape round-tripping end-to-end through both tiers Verification: cargo test 62 passed (default; 5 new tests), cargo test --all-features 117, cargo test --features mem 117, --no-default- features --features mem 97; cargo clippy --all-targets -- -D warnings clean (default + all-features); cargo fmt --check clean; cargo doc --no-deps at the pre-existing warning baseline.
This commit is contained in:
1 parent
51d8f6525f
commit
2993df0e36
6 files changed
+673
-64
No files matched your search
+7
-3
@@ -31,9 +31,13 @@
|
|||||||
//! the crate-internal engine-state companion seam lives in [`crate::gc`].
|
//! the crate-internal engine-state companion seam lives in [`crate::gc`].
|
||||||
//!
|
//!
|
||||||
//! The `#![allow(dead_code)]` below still has real scope: `Backend`
|
//! The `#![allow(dead_code)]` below still has real scope: `Backend`
|
||||||
//! trait items with no in-tree caller yet (reporting names, the kv
|
//! trait items with no in-tree caller yet (the engines' reporting
|
||||||
//! engines' whole-value `put` before wave 3) plus the wave-1 seam
|
//! names — the dispatch/conformance identity the reporting and fleet
|
||||||
//! tests' doubles. It shrinks as the waves land.
|
//! waves consume) plus the wave-1 seam tests' doubles. The kv engines'
|
||||||
|
//! whole-value `put` gained its in-tree callers in wave 3 (sqlite is
|
||||||
|
//! the default build's kv engine and the facade's put seam drives the
|
||||||
|
//! kv tier through it); the allow shrinks as the remaining callers
|
||||||
|
//! land.
|
||||||
|
|
||||||
#![allow(dead_code)]
|
#![allow(dead_code)]
|
||||||
|
|
||||||
|
|||||||
@@ -65,8 +65,11 @@
|
|||||||
//! report data (`Ok(SweepReport)` — ADR-012 §1), nothing deleted.
|
//! report data (`Ok(SweepReport)` — ADR-012 §1), nothing deleted.
|
||||||
//!
|
//!
|
||||||
//! The `#![allow(dead_code)]` below covers the companion-seam scope:
|
//! The `#![allow(dead_code)]` below covers the companion-seam scope:
|
||||||
//! the engine-state clauses are implemented by the SQL engines of
|
//! the window family has its first in-tree caller in wave 3 (store
|
||||||
//! waves 3–5 and are not called yet. It shrinks as those waves land.
|
//! construction runs `ensure_gc_schema` on the SQL-backed kv engine),
|
||||||
|
//! while `window_stage`/`window_take` wait for the sweep's durable
|
||||||
|
//! form and the fleet family waits for postgres's wave. It shrinks as
|
||||||
|
//! those land.
|
||||||
|
|
||||||
#![allow(dead_code)]
|
#![allow(dead_code)]
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -16,7 +16,7 @@ pub use facade::{
|
|||||||
};
|
};
|
||||||
pub use hashing::{Algorithm, Digest};
|
pub use hashing::{Algorithm, Digest};
|
||||||
pub use keys::Key;
|
pub use keys::Key;
|
||||||
pub use store::{Store, StoreMode};
|
pub use store::{Store, StoreMode, DEFAULT_DELETE_WINDOW_DELAY};
|
||||||
|
|
||||||
#[cfg(feature = "mem")]
|
#[cfg(feature = "mem")]
|
||||||
pub mod mem;
|
pub mod mem;
|
||||||
|
|||||||
+33
-18
@@ -44,8 +44,15 @@
|
|||||||
//! databases included): the durability promise is verified at open, not
|
//! databases included): the durability promise is verified at open, not
|
||||||
//! assumed.
|
//! assumed.
|
||||||
//!
|
//!
|
||||||
//! The `#![allow(dead_code)]` below has real scope: nothing constructs
|
//! The engine is constructed by the store facade —
|
||||||
//! the engine until wave 3's constructor wiring (`store-sqlite-construction`).
|
//! `dual_tier_sqlite_local` / `kv_only_sqlite` (the wave-3 constructor
|
||||||
|
//! wiring, landed) — which also runs the GC schema's engine-neutral
|
||||||
|
//! call (`ensure_gc_schema`, creating `gc_window`). The
|
||||||
|
//! `#![allow(dead_code)]` below has real scope only through that
|
||||||
|
//! wiring's cfg-gates: a build with the feature off never compiles this
|
||||||
|
//! module, and the test-only instruments (the tableless constructor,
|
||||||
|
//! the commit-failure injection, the probe helpers) sit behind
|
||||||
|
//! `#[cfg(test)]`.
|
||||||
|
|
||||||
#![allow(dead_code)]
|
#![allow(dead_code)]
|
||||||
|
|
||||||
@@ -182,6 +189,27 @@ impl SqliteBackend {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The sqlite_master table-existence probe — test-only: the store
|
||||||
|
/// construction test's instrument (covers every schema object,
|
||||||
|
/// not just probed reads) and the local tests' helper body.
|
||||||
|
#[cfg(test)]
|
||||||
|
pub(crate) async fn table_exists(&self, name: &str) -> Result<bool, Error> {
|
||||||
|
let conn = Arc::clone(&self.conn);
|
||||||
|
let name = name.to_owned();
|
||||||
|
blocking(move || {
|
||||||
|
let guard = conn.lock();
|
||||||
|
let probed: rusqlite::Result<bool> = guard
|
||||||
|
.query_row(
|
||||||
|
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = ?1",
|
||||||
|
params![name],
|
||||||
|
|row| row.get::<_, i64>(0),
|
||||||
|
)
|
||||||
|
.map(|count| count != 0);
|
||||||
|
probed.map_err(db_err)
|
||||||
|
})
|
||||||
|
.await
|
||||||
|
}
|
||||||
|
|
||||||
/// The connection's live pragma value — the posture test's probe
|
/// The connection's live pragma value — the posture test's probe
|
||||||
/// (`synchronous` is per-connection, so the backend's own connection
|
/// (`synchronous` is per-connection, so the backend's own connection
|
||||||
/// must be asked).
|
/// must be asked).
|
||||||
@@ -1054,23 +1082,10 @@ mod tests {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// Table existence probed through sqlite_master — the schema test's
|
/// Table existence probed through sqlite_master — the schema test's
|
||||||
/// instrument (covers every schema object, not just probed reads).
|
/// instrument (covers every schema object, not just probed reads); a
|
||||||
|
/// thin unwrap-wrap over the backend's helper.
|
||||||
async fn table_exists(backend: &SqliteBackend, name: &str) -> bool {
|
async fn table_exists(backend: &SqliteBackend, name: &str) -> bool {
|
||||||
let conn = Arc::clone(&backend.conn);
|
backend.table_exists(name).await.unwrap()
|
||||||
let name = name.to_owned();
|
|
||||||
blocking(move || {
|
|
||||||
let guard = conn.lock();
|
|
||||||
Ok(guard
|
|
||||||
.query_row(
|
|
||||||
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = ?1",
|
|
||||||
params![name],
|
|
||||||
|row| row.get::<_, i64>(0),
|
|
||||||
)
|
|
||||||
.unwrap()
|
|
||||||
!= 0)
|
|
||||||
})
|
|
||||||
.await
|
|
||||||
.unwrap()
|
|
||||||
}
|
}
|
||||||
|
|
||||||
fn stages(key: &[u8], visible_after: SystemTime) -> WindowCandidate {
|
fn stages(key: &[u8], visible_after: SystemTime) -> WindowCandidate {
|
||||||
|
|||||||
+556
-31
@@ -36,13 +36,17 @@
|
|||||||
//! - **dual-tier** (the default): kv + large, size-threshold dispatch
|
//! - **dual-tier** (the default): kv + large, size-threshold dispatch
|
||||||
//! via [`crate::dispatch::route`]. The wave-2 constructible shape is
|
//! via [`crate::dispatch::route`]. The wave-2 constructible shape is
|
||||||
//! `kv = mem` + `large = local` — the dispatch-coverage test shape
|
//! `kv = mem` + `large = local` — the dispatch-coverage test shape
|
||||||
//! ADR-012 §6.3 names (both tiers load-bearing). Over-threshold
|
//! ADR-012 §6.3 names (both tiers load-bearing); the wave-3
|
||||||
//! content routes to the large tier; at-or-below to kv.
|
//! constructible shape is `kv = sqlite` + `large = local` — the
|
||||||
|
//! **default-build production shape** (feature `sqlite` is
|
||||||
|
//! default-on, ADR-013 §4). Over-threshold content routes to the
|
||||||
|
//! large tier; at-or-below to kv.
|
||||||
//! - **kv-only**: no large tier; over-threshold puts reject **at put
|
//! - **kv-only**: no large tier; over-threshold puts reject **at put
|
||||||
//! time** (ADR-012 §6.2 — known-length immediately; unknown-length at
|
//! time** (ADR-012 §6.2 — known-length immediately; unknown-length at
|
||||||
//! mid-stream threshold overflow). The rejection behavior itself
|
//! mid-stream threshold overflow). The rejection behavior itself
|
||||||
//! lands with the put seam; the threshold this mode carries is the
|
//! lands with the put seam; the threshold this mode carries is the
|
||||||
//! rejection point's value.
|
//! rejection point's value. The wave-2 engine is mem (testing);
|
||||||
|
//! the wave-3 engine is sqlite (the durable single-node medium).
|
||||||
//! - **mem-only**: the mem engine alone, **no dispatch threshold in
|
//! - **mem-only**: the mem engine alone, **no dispatch threshold in
|
||||||
//! effect** (everything is local and small) — a testing/embedder-
|
//! effect** (everything is local and small) — a testing/embedder-
|
||||||
//! ephemeral posture, never production. Structurally distinct from
|
//! ephemeral posture, never production. Structurally distinct from
|
||||||
@@ -193,16 +197,38 @@
|
|||||||
//! [`Store::large`]), and the GC state home ([`crate::gc::GcHome`])
|
//! [`Store::large`]), and the GC state home ([`crate::gc::GcHome`])
|
||||||
//! the gc-protocol task fills.
|
//! the gc-protocol task fills.
|
||||||
//!
|
//!
|
||||||
|
//! **The durable delete window hosts on the kv engine when it is
|
||||||
|
//! SQL-backed (ADR-012 §5's one-window-host rule).** Sqlite — the
|
||||||
|
//! default build's kv engine — hosts: its constructors run the GC
|
||||||
|
//! schema's engine-neutral call (`ensure_gc_schema` on the
|
||||||
|
//! `EngineWindowState` seam) at construction, dispatched on the
|
||||||
|
//! kv-engine enum — a mem arm is structurally nothing, the in-process
|
||||||
|
//! engines execute the ADR-005 window protocol inline, so there is no
|
||||||
|
//! schema to ensure — and the SQL-backed sweep form (wave 3's
|
||||||
|
//! sweep-durable-window task) stages its candidates on the kv engine
|
||||||
|
//! the constructor built. The host branch keys off the engine
|
||||||
|
//! representation — one more kv-engine-enum match arm — so wave 5's
|
||||||
|
//! pg-lo dormancy (the dual-SQL instance's large-engine window clauses
|
||||||
|
//! are dormant when the kv engine already hosts) slots in without
|
||||||
|
//! reshaping. Both sqlite constructors carry the delete-window delay
|
||||||
|
//! (ADR-008 §2: default 5 min, constructor-tunable per deployment —
|
||||||
|
//! [`DEFAULT_DELETE_WINDOW_DELAY`]; `0` is a legal degenerate tuning,
|
||||||
|
//! execute-on-next-sweep) as the cadence knob the durable sweep form
|
||||||
|
//! consumes; the mem/local constructors' inline form has no delay in
|
||||||
|
//! effect.
|
||||||
|
//!
|
||||||
//! The `#![allow(dead_code)]` below marks the work still ahead, not
|
//! The `#![allow(dead_code)]` below marks the work still ahead, not
|
||||||
//! the put path (which is driven below): the GC home's fleet clauses
|
//! the put path (which is driven below): the GC home's fleet clauses
|
||||||
//! and the kv engine set are empty in default builds (mem is
|
//! and the durable window's cadence knob (until the sweep's durable
|
||||||
//! feature-gated; sqlite is wave 3). The allow shrinks as those land.
|
//! form consumes it) hold the real scope. The kv engine set is no
|
||||||
|
//! longer empty in default builds — sqlite landed in wave 3 — and the
|
||||||
|
//! empty-set `allow`s came off with it.
|
||||||
|
|
||||||
#![allow(dead_code)]
|
#![allow(dead_code)]
|
||||||
|
|
||||||
use std::collections::BTreeSet;
|
use std::collections::BTreeSet;
|
||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
use std::time::SystemTime;
|
use std::time::{Duration, SystemTime};
|
||||||
|
|
||||||
use tokio::io::{AsyncRead, AsyncReadExt};
|
use tokio::io::{AsyncRead, AsyncReadExt};
|
||||||
|
|
||||||
@@ -212,6 +238,8 @@ use crate::facade::{
|
|||||||
Batch, Blob, EntryMeta, GcAbortCause, LivenessSource, Pin, ProtectCallback, Registration,
|
Batch, Blob, EntryMeta, GcAbortCause, LivenessSource, Pin, ProtectCallback, Registration,
|
||||||
SweepReport,
|
SweepReport,
|
||||||
};
|
};
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
use crate::gc::EngineWindowState;
|
||||||
use crate::gc::{Arbitration, GcHome};
|
use crate::gc::{Arbitration, GcHome};
|
||||||
use crate::hashing::{Algorithm, BlobHasher};
|
use crate::hashing::{Algorithm, BlobHasher};
|
||||||
use crate::keys::Key;
|
use crate::keys::Key;
|
||||||
@@ -219,14 +247,26 @@ use crate::local::{LocalBackend, LocalCursor, LocalStaging};
|
|||||||
|
|
||||||
#[cfg(feature = "mem")]
|
#[cfg(feature = "mem")]
|
||||||
use crate::mem::{MemBackend, MemCursor};
|
use crate::mem::{MemBackend, MemCursor};
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
use crate::sqlite::{SqliteBackend, SqliteCursor};
|
||||||
|
|
||||||
use crate::Error;
|
use crate::Error;
|
||||||
|
|
||||||
#[cfg(feature = "mem")]
|
#[cfg(any(feature = "mem", feature = "sqlite"))]
|
||||||
use std::path::PathBuf;
|
use std::path::PathBuf;
|
||||||
|
|
||||||
use self::StoreMode::{DualTier, KvOnly, MemOnly};
|
use self::StoreMode::{DualTier, KvOnly, MemOnly};
|
||||||
|
|
||||||
|
/// The shipped default **delete-window delay** (ADR-008 §2): the
|
||||||
|
/// staged candidates in the durable window become visible for
|
||||||
|
/// execution this long after the sweep staged them — 5 minutes,
|
||||||
|
/// constructor-tunable per deployment. `Duration::ZERO` is a legal
|
||||||
|
/// degenerate tuning (execute-on-next-sweep; the test tuning). The
|
||||||
|
/// sqlite constructors take the delay explicitly and their docs point
|
||||||
|
/// here as the shipped default; the mem/local inline form has no
|
||||||
|
/// delay in effect and the constant is irrelevant to it.
|
||||||
|
pub const DEFAULT_DELETE_WINDOW_DELAY: Duration = Duration::from_secs(5 * 60);
|
||||||
|
|
||||||
/// The store instance — the public facade above the tier engines
|
/// The store instance — the public facade above the tier engines
|
||||||
/// (ADR-011 §1's vocabulary: *one running instantiation of the store
|
/// (ADR-011 §1's vocabulary: *one running instantiation of the store
|
||||||
/// facade — one engine set per the constructor mode, plus dispatch and
|
/// facade — one engine set per the constructor mode, plus dispatch and
|
||||||
@@ -248,6 +288,13 @@ pub struct Store {
|
|||||||
/// threshold in effect" is the mode's ruling (ADR-011 §4), not a
|
/// threshold in effect" is the mode's ruling (ADR-011 §4), not a
|
||||||
/// value absence.
|
/// value absence.
|
||||||
threshold: u64,
|
threshold: u64,
|
||||||
|
/// The delete-window delay the constructor took (ADR-008 §2) — the
|
||||||
|
/// durable window's cadence knob the sweep's SQL-backed form
|
||||||
|
/// consumes (`window_stage`'s `visible_after = mark_end + window`
|
||||||
|
/// on the `EngineWindowState` seam). The mem/local inline form
|
||||||
|
/// has no delay in effect: those constructors carry
|
||||||
|
/// [`DEFAULT_DELETE_WINDOW_DELAY`], never consulted.
|
||||||
|
window_delay: Duration,
|
||||||
/// The constructor mode; the dispatch/put/read/gc paths branch on it.
|
/// The constructor mode; the dispatch/put/read/gc paths branch on it.
|
||||||
mode: StoreMode,
|
mode: StoreMode,
|
||||||
/// The GC state home (ADR-010 §2: the store core owns GC state)
|
/// The GC state home (ADR-010 §2: the store core owns GC state)
|
||||||
@@ -271,11 +318,18 @@ pub enum StoreMode {
|
|||||||
|
|
||||||
/// The kv-tier engine set — the closed per-tier engine enum of the
|
/// The kv-tier engine set — the closed per-tier engine enum of the
|
||||||
/// engine-representation design point. Feature-gated variants slot in
|
/// engine-representation design point. Feature-gated variants slot in
|
||||||
/// as their engine waves land: sqlite (wave 3), postgres (wave 4).
|
/// as their engine waves land: sqlite (wave 3 — landed, the default),
|
||||||
|
/// postgres (wave 4).
|
||||||
pub(crate) enum KvEngine {
|
pub(crate) enum KvEngine {
|
||||||
/// The contract-reference ephemeral engine (ADR-011 §2).
|
/// The contract-reference ephemeral engine (ADR-011 §2).
|
||||||
#[cfg(feature = "mem")]
|
#[cfg(feature = "mem")]
|
||||||
Mem(MemBackend),
|
Mem(MemBackend),
|
||||||
|
/// The default build's kv engine — the first SQL-backed engine,
|
||||||
|
/// the durable delete window's host (ADR-012 §5's one-window-host
|
||||||
|
/// rule: the window is the kv engine's when the kv engine is
|
||||||
|
/// SQL-backed).
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
Sqlite(SqliteBackend),
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The large-tier engine set. `local` is ungated (dependency-free,
|
/// The large-tier engine set. `local` is ungated (dependency-free,
|
||||||
@@ -285,18 +339,56 @@ pub(crate) enum LargeEngine {
|
|||||||
Local(LocalBackend),
|
Local(LocalBackend),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// **The kv-arm shape over multiple kv engines (named design point):
|
||||||
|
/// a `KvCursor` sub-enum behind [`AnyCursor`]'s one kv arm.** The
|
||||||
|
/// alternative — one `AnyCursor` variant per kv engine — would copy
|
||||||
|
/// the kv materializing arm (and its consumers: the verification-on-
|
||||||
|
/// get body, `Blob`'s drive) per engine without adding a behavior;
|
||||||
|
/// the sub-enum keeps [`AnyCursor`] a two-arm tier dispatch (kv /
|
||||||
|
/// large) while kv engines multiply inside this one. Recorded for the
|
||||||
|
/// review gate: postgres (wave 4) slots one arm in here.
|
||||||
|
pub(crate) enum KvCursor {
|
||||||
|
/// The contract-reference ephemeral body (ADR-011 §2).
|
||||||
|
#[cfg(feature = "mem")]
|
||||||
|
Mem(MemCursor),
|
||||||
|
/// The default build's materializing body: the row's value loaded
|
||||||
|
/// once, reads in-memory slices (the sqlite engine's record).
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
Sqlite(SqliteCursor),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ReadCursor for KvCursor {
|
||||||
|
fn len(&self) -> u64 {
|
||||||
|
match self {
|
||||||
|
#[cfg(feature = "mem")]
|
||||||
|
Self::Mem(cursor) => cursor.len(),
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
Self::Sqlite(cursor) => cursor.len(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn read_at(&self, offset: u64, len: u64) -> Result<Vec<u8>, Error> {
|
||||||
|
match self {
|
||||||
|
#[cfg(feature = "mem")]
|
||||||
|
Self::Mem(cursor) => cursor.read_at(offset, len).await,
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
Self::Sqlite(cursor) => cursor.read_at(offset, len).await,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// The read path's cursor-dispatch enum (ADR-010 §1's "store core
|
/// The read path's cursor-dispatch enum (ADR-010 §1's "store core
|
||||||
/// programs the abstraction"): tier dispatch over the engine cursor
|
/// programs the abstraction"): tier dispatch over the engine cursor
|
||||||
/// bodies — the kv tier's materializing body and the large tier's
|
/// bodies — the kv tier's materializing body and the large tier's
|
||||||
/// streaming body — behind one crate-internal value that [`Blob`] holds
|
/// streaming body — behind one crate-internal value that [`Blob`] holds
|
||||||
/// and drives. Crate-internal: engine cursors never appear on the
|
/// and drives. Crate-internal: engine cursors never appear on the
|
||||||
/// public surface; the facade's stream shape is this enum's uniform
|
/// public surface; the facade's stream shape is this enum's uniform
|
||||||
/// [`ReadCursor`] view (`Blob`).
|
/// [`ReadCursor`] view (`Blob`). The kv bodies live behind
|
||||||
|
/// [`KvCursor`] (the recorded design point above).
|
||||||
pub(crate) enum AnyCursor {
|
pub(crate) enum AnyCursor {
|
||||||
/// The kv body family: the whole value materialized in memory
|
/// The kv body family: the whole value materialized in memory
|
||||||
/// (bounded by tier policy).
|
/// (bounded by tier policy).
|
||||||
#[cfg(feature = "mem")]
|
Kv(KvCursor),
|
||||||
Kv(MemCursor),
|
|
||||||
/// The large body family: a pread cursor over the committed file.
|
/// The large body family: a pread cursor over the committed file.
|
||||||
Large(LocalCursor),
|
Large(LocalCursor),
|
||||||
}
|
}
|
||||||
@@ -304,7 +396,6 @@ pub(crate) enum AnyCursor {
|
|||||||
impl ReadCursor for AnyCursor {
|
impl ReadCursor for AnyCursor {
|
||||||
fn len(&self) -> u64 {
|
fn len(&self) -> u64 {
|
||||||
match self {
|
match self {
|
||||||
#[cfg(feature = "mem")]
|
|
||||||
Self::Kv(cursor) => cursor.len(),
|
Self::Kv(cursor) => cursor.len(),
|
||||||
Self::Large(cursor) => cursor.len(),
|
Self::Large(cursor) => cursor.len(),
|
||||||
}
|
}
|
||||||
@@ -312,7 +403,6 @@ impl ReadCursor for AnyCursor {
|
|||||||
|
|
||||||
async fn read_at(&self, offset: u64, len: u64) -> Result<Vec<u8>, Error> {
|
async fn read_at(&self, offset: u64, len: u64) -> Result<Vec<u8>, Error> {
|
||||||
match self {
|
match self {
|
||||||
#[cfg(feature = "mem")]
|
|
||||||
Self::Kv(cursor) => cursor.read_at(offset, len).await,
|
Self::Kv(cursor) => cursor.read_at(offset, len).await,
|
||||||
Self::Large(cursor) => cursor.read_at(offset, len).await,
|
Self::Large(cursor) => cursor.read_at(offset, len).await,
|
||||||
}
|
}
|
||||||
@@ -367,50 +457,79 @@ impl StagingCursor for AnyStaging {
|
|||||||
}
|
}
|
||||||
|
|
||||||
impl KvEngine {
|
impl KvEngine {
|
||||||
#[allow(unused_variables)] // kv engine set empty until wave 3 (sqlite): no-mem builds never reference `key`/`value`
|
|
||||||
async fn put(&self, key: &[u8], value: &[u8]) -> Result<(), Error> {
|
async fn put(&self, key: &[u8], value: &[u8]) -> Result<(), Error> {
|
||||||
match *self {
|
match self {
|
||||||
#[cfg(feature = "mem")]
|
#[cfg(feature = "mem")]
|
||||||
KvEngine::Mem(ref engine) => engine.put(key, value).await,
|
KvEngine::Mem(engine) => engine.put(key, value).await,
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
KvEngine::Sqlite(engine) => engine.put(key, value).await,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[allow(unused_variables)] // kv engine set empty until wave 3 (sqlite): no-mem builds never reference `key`
|
|
||||||
async fn get(&self, key: &[u8]) -> Result<Option<AnyCursor>, Error> {
|
async fn get(&self, key: &[u8]) -> Result<Option<AnyCursor>, Error> {
|
||||||
match *self {
|
match self {
|
||||||
#[cfg(feature = "mem")]
|
#[cfg(feature = "mem")]
|
||||||
KvEngine::Mem(ref engine) => Ok(engine.get(key).await?.map(AnyCursor::Kv)),
|
KvEngine::Mem(engine) => Ok(engine
|
||||||
|
.get(key)
|
||||||
|
.await?
|
||||||
|
.map(|c| AnyCursor::Kv(KvCursor::Mem(c)))),
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
KvEngine::Sqlite(engine) => Ok(engine
|
||||||
|
.get(key)
|
||||||
|
.await?
|
||||||
|
.map(|c| AnyCursor::Kv(KvCursor::Sqlite(c)))),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The GC schema's engine-neutral store-core call (ADR-010 §2's
|
||||||
|
/// companion seam; ADR-012 §5's applicability table), run at
|
||||||
|
/// construction: a SQL-backed kv engine creates its window
|
||||||
|
/// table(s) — sqlite's body creates `gc_window` only — and an
|
||||||
|
/// in-process kv engine is structurally nothing (the
|
||||||
|
/// local/mem row of the table: those engines execute the window
|
||||||
|
/// protocol inline, ADR-005, and carry no GC schema).
|
||||||
|
async fn ensure_gc_schema(&self) -> Result<(), Error> {
|
||||||
|
match self {
|
||||||
|
#[cfg(feature = "mem")]
|
||||||
|
KvEngine::Mem(_) => Ok(()),
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
KvEngine::Sqlite(engine) => engine.ensure_gc_schema().await,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[allow(unused_variables)] // same empty-set scope as `get`
|
|
||||||
async fn has(&self, key: &[u8]) -> Result<bool, Error> {
|
async fn has(&self, key: &[u8]) -> Result<bool, Error> {
|
||||||
match *self {
|
match self {
|
||||||
#[cfg(feature = "mem")]
|
#[cfg(feature = "mem")]
|
||||||
KvEngine::Mem(ref engine) => engine.has(key).await,
|
KvEngine::Mem(engine) => engine.has(key).await,
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
KvEngine::Sqlite(engine) => engine.has(key).await,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[allow(unused_variables)] // same empty-set scope as `get`
|
|
||||||
async fn size(&self, key: &[u8]) -> Result<Option<u64>, Error> {
|
async fn size(&self, key: &[u8]) -> Result<Option<u64>, Error> {
|
||||||
match *self {
|
match self {
|
||||||
#[cfg(feature = "mem")]
|
#[cfg(feature = "mem")]
|
||||||
KvEngine::Mem(ref engine) => engine.size(key).await,
|
KvEngine::Mem(engine) => engine.size(key).await,
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
KvEngine::Sqlite(engine) => engine.size(key).await,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[allow(unused_variables)] // same empty-set scope as `get`
|
|
||||||
async fn delete(&self, key: &[u8]) -> Result<(), Error> {
|
async fn delete(&self, key: &[u8]) -> Result<(), Error> {
|
||||||
match *self {
|
match self {
|
||||||
#[cfg(feature = "mem")]
|
#[cfg(feature = "mem")]
|
||||||
KvEngine::Mem(ref engine) => engine.delete(key).await,
|
KvEngine::Mem(engine) => engine.delete(key).await,
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
KvEngine::Sqlite(engine) => engine.delete(key).await,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn list(&self) -> Result<Vec<Vec<u8>>, Error> {
|
async fn list(&self) -> Result<Vec<Vec<u8>>, Error> {
|
||||||
match *self {
|
match self {
|
||||||
#[cfg(feature = "mem")]
|
#[cfg(feature = "mem")]
|
||||||
KvEngine::Mem(ref engine) => engine.list().await,
|
KvEngine::Mem(engine) => engine.list().await,
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
KvEngine::Sqlite(engine) => engine.list().await,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -483,6 +602,7 @@ impl Store {
|
|||||||
kv: KvEngine::Mem(MemBackend::default()),
|
kv: KvEngine::Mem(MemBackend::default()),
|
||||||
large: None,
|
large: None,
|
||||||
threshold: crate::dispatch::DEFAULT_DISPATCH_THRESHOLD,
|
threshold: crate::dispatch::DEFAULT_DISPATCH_THRESHOLD,
|
||||||
|
window_delay: DEFAULT_DELETE_WINDOW_DELAY,
|
||||||
mode: MemOnly,
|
mode: MemOnly,
|
||||||
gc: GcHome::default(),
|
gc: GcHome::default(),
|
||||||
}
|
}
|
||||||
@@ -500,6 +620,7 @@ impl Store {
|
|||||||
kv: KvEngine::Mem(MemBackend::default()),
|
kv: KvEngine::Mem(MemBackend::default()),
|
||||||
large: None,
|
large: None,
|
||||||
threshold,
|
threshold,
|
||||||
|
window_delay: DEFAULT_DELETE_WINDOW_DELAY,
|
||||||
mode: KvOnly,
|
mode: KvOnly,
|
||||||
gc: GcHome::default(),
|
gc: GcHome::default(),
|
||||||
}
|
}
|
||||||
@@ -526,6 +647,82 @@ impl Store {
|
|||||||
kv: KvEngine::Mem(MemBackend::default()),
|
kv: KvEngine::Mem(MemBackend::default()),
|
||||||
large: Some(LargeEngine::Local(LocalBackend::new(root).await?)),
|
large: Some(LargeEngine::Local(LocalBackend::new(root).await?)),
|
||||||
threshold,
|
threshold,
|
||||||
|
window_delay: DEFAULT_DELETE_WINDOW_DELAY,
|
||||||
|
mode: DualTier,
|
||||||
|
gc: GcHome::default(),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// **kv-only mode** (ADR-011 §4; ADR-012 §6.2) with the sqlite kv
|
||||||
|
/// engine — the durable single-node kv-only shape: no large tier;
|
||||||
|
/// `threshold` is the put-time rejection point (known-length puts
|
||||||
|
/// over it reject immediately, unknown-length puts at mid-stream
|
||||||
|
/// threshold overflow — the put seam's behavior, identical to the
|
||||||
|
/// mem kv-only wiring; zero residue on either rejection). Routing
|
||||||
|
/// in this mode is kv for every length.
|
||||||
|
///
|
||||||
|
/// Opening (or creating) the database file is fallible I/O; the
|
||||||
|
/// constructor also runs the GC schema's engine-neutral call
|
||||||
|
/// (`ensure_gc_schema` — sqlite's body creates `gc_window` only,
|
||||||
|
/// ADR-012 §5), so a construction failure is `Err(Error::Io)` per
|
||||||
|
/// the fallible-constructor pattern `dual_tier_mem_local` set.
|
||||||
|
///
|
||||||
|
/// `window_delay` (ADR-008 §2) is the durable window's cadence
|
||||||
|
/// knob — [`DEFAULT_DELETE_WINDOW_DELAY`] is the shipped default,
|
||||||
|
/// `Duration::ZERO` the legal degenerate tuning. Not fleet-
|
||||||
|
/// anything: a sqlite fleet is a recorded non-option (ADR-010 §3).
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
pub async fn kv_only_sqlite(
|
||||||
|
database: impl Into<PathBuf>,
|
||||||
|
threshold: u64,
|
||||||
|
window_delay: Duration,
|
||||||
|
) -> Result<Store, Error> {
|
||||||
|
let kv = KvEngine::Sqlite(SqliteBackend::new(database).await?);
|
||||||
|
kv.ensure_gc_schema().await?;
|
||||||
|
Ok(Store {
|
||||||
|
kv,
|
||||||
|
large: None,
|
||||||
|
threshold,
|
||||||
|
window_delay,
|
||||||
|
mode: KvOnly,
|
||||||
|
gc: GcHome::default(),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// **dual-tier mode** (the default, ADR-011 §4) in the default-build
|
||||||
|
/// production shape — `kv = sqlite` + `large = local` (feature
|
||||||
|
/// `sqlite` is default-on, ADR-013 §4): size-threshold dispatch at
|
||||||
|
/// `threshold` (constructor-tunable;
|
||||||
|
/// [`crate::dispatch::DEFAULT_DISPATCH_THRESHOLD`] is the shipped
|
||||||
|
/// default), at-or-below on the kv tier, over-threshold staged into
|
||||||
|
/// the large tier.
|
||||||
|
///
|
||||||
|
/// Fallible on two accounts (file I/O): opening/creating the pool
|
||||||
|
/// database — which also runs the GC schema's engine-neutral call
|
||||||
|
/// (`ensure_gc_schema`; sqlite's body creates `gc_window` only,
|
||||||
|
/// ADR-012 §5's one-window-host rule: the durable window is the kv
|
||||||
|
/// engine's when the kv engine is SQL-backed) — and creating the
|
||||||
|
/// local root. A construction failure is `Err(Error::Io)` per the
|
||||||
|
/// fallible-constructor pattern `dual_tier_mem_local` set.
|
||||||
|
///
|
||||||
|
/// `window_delay` (ADR-008 §2) is the durable window's cadence
|
||||||
|
/// knob — [`DEFAULT_DELETE_WINDOW_DELAY`] is the shipped default,
|
||||||
|
/// `Duration::ZERO` the legal degenerate tuning. Not fleet-
|
||||||
|
/// anything: a sqlite fleet is a recorded non-option (ADR-010 §3).
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
pub async fn dual_tier_sqlite_local(
|
||||||
|
database: impl Into<PathBuf>,
|
||||||
|
root: impl Into<PathBuf>,
|
||||||
|
threshold: u64,
|
||||||
|
window_delay: Duration,
|
||||||
|
) -> Result<Store, Error> {
|
||||||
|
let kv = KvEngine::Sqlite(SqliteBackend::new(database).await?);
|
||||||
|
kv.ensure_gc_schema().await?;
|
||||||
|
Ok(Store {
|
||||||
|
kv,
|
||||||
|
large: Some(LargeEngine::Local(LocalBackend::new(root).await?)),
|
||||||
|
threshold,
|
||||||
|
window_delay,
|
||||||
mode: DualTier,
|
mode: DualTier,
|
||||||
gc: GcHome::default(),
|
gc: GcHome::default(),
|
||||||
})
|
})
|
||||||
@@ -548,6 +745,16 @@ impl Store {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The delete-window delay the constructor took (ADR-008 §2) —
|
||||||
|
/// crate-internal: the sweep's SQL-backed form consumes it when
|
||||||
|
/// staging candidates (`visible_after = mark_end + window`);
|
||||||
|
/// introspection for the construction tests. `Duration::ZERO`
|
||||||
|
/// (a constructor-passed value) is the legal degenerate tuning —
|
||||||
|
/// execute-on-next-sweep.
|
||||||
|
pub(crate) fn window_delay(&self) -> Duration {
|
||||||
|
self.window_delay
|
||||||
|
}
|
||||||
|
|
||||||
/// **Write-side tier selection for a given content length** — the
|
/// **Write-side tier selection for a given content length** — the
|
||||||
/// dispatch seam's routing half (the put seam programs through
|
/// dispatch seam's routing half (the put seam programs through
|
||||||
/// this; read paths do not consult it). Dual-tier:
|
/// this; read paths do not consult it). Dual-tier:
|
||||||
@@ -621,7 +828,6 @@ impl Store {
|
|||||||
};
|
};
|
||||||
|
|
||||||
match cursor {
|
match cursor {
|
||||||
#[cfg(feature = "mem")]
|
|
||||||
AnyCursor::Kv(kv) => {
|
AnyCursor::Kv(kv) => {
|
||||||
let value = kv.read_at(0, kv.len()).await?;
|
let value = kv.read_at(0, kv.len()).await?;
|
||||||
let derived = key.algorithm().derive(&value, value.len() as u64);
|
let derived = key.algorithm().derive(&value, value.len() as u64);
|
||||||
@@ -1519,7 +1725,10 @@ mod tests {
|
|||||||
/// verification postures are observed.
|
/// verification postures are observed.
|
||||||
async fn plant_kv(store: &Store, key: &[u8], value: &[u8]) {
|
async fn plant_kv(store: &Store, key: &[u8], value: &[u8]) {
|
||||||
match store.kv() {
|
match store.kv() {
|
||||||
|
#[cfg(feature = "mem")]
|
||||||
KvEngine::Mem(engine) => engine.put(key, value).await.expect("kv plant"),
|
KvEngine::Mem(engine) => engine.put(key, value).await.expect("kv plant"),
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
KvEngine::Sqlite(engine) => engine.put(key, value).await.expect("kv plant"),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -3486,3 +3695,319 @@ mod tests {
|
|||||||
assert_eq!(blob.read_at(0, 8).await.unwrap(), Vec::<u8>::new());
|
assert_eq!(blob.read_at(0, 8).await.unwrap(), Vec::<u8>::new());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The wave-3 constructor wiring's tests (feature `sqlite`): the
|
||||||
|
/// sqlite modes construct and introspect correctly, the GC schema ran
|
||||||
|
/// at construction, the window-delay default is the pinned constant
|
||||||
|
/// and is constructor-tunable, the kv-only rejection wiring is the
|
||||||
|
/// mode's put seam unchanged, and the default-build production shape
|
||||||
|
/// (kv = sqlite + large = local) round-trips end-to-end through both
|
||||||
|
/// tiers. All-features builds share this module (mem on alongside):
|
||||||
|
/// the engine accessors carry cfg arms for that.
|
||||||
|
#[cfg(all(test, feature = "sqlite"))]
|
||||||
|
mod sqlite_construction_tests {
|
||||||
|
use super::*;
|
||||||
|
use std::fs;
|
||||||
|
use std::io::Cursor;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::sync::atomic::{AtomicU64, Ordering};
|
||||||
|
use std::time::{SystemTime, UNIX_EPOCH};
|
||||||
|
|
||||||
|
/// A fresh temp database file (and sibling stage/root dir) per
|
||||||
|
/// test; Drop removes the file tree and the WAL sidecars
|
||||||
|
/// idempotently (the sqlite engine's test helper's shape).
|
||||||
|
struct TempSite {
|
||||||
|
database: PathBuf,
|
||||||
|
root: PathBuf,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TempSite {
|
||||||
|
fn path(&self) -> &Path {
|
||||||
|
&self.root
|
||||||
|
}
|
||||||
|
|
||||||
|
fn drop_sidecar(&self, suffix: &str) {
|
||||||
|
let base = self.database.to_string_lossy().into_owned();
|
||||||
|
let _ = fs::remove_file(format!("{base}{suffix}"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for TempSite {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
self.drop_sidecar("-wal");
|
||||||
|
self.drop_sidecar("-shm");
|
||||||
|
let _ = fs::remove_file(&self.database);
|
||||||
|
let _ = fs::remove_dir_all(&self.root);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn temp_site(tag: &str) -> TempSite {
|
||||||
|
static SEQ: AtomicU64 = AtomicU64::new(0);
|
||||||
|
let seq = SEQ.fetch_add(1, Ordering::Relaxed);
|
||||||
|
let nanos = SystemTime::now()
|
||||||
|
.duration_since(UNIX_EPOCH)
|
||||||
|
.map(|d| d.as_nanos())
|
||||||
|
.unwrap_or(0);
|
||||||
|
let base = std::env::temp_dir().join(format!(
|
||||||
|
"alkblobs-store-sqlite-{tag}-{}-{nanos:x}-{seq}",
|
||||||
|
std::process::id()
|
||||||
|
));
|
||||||
|
fs::create_dir_all(&base).expect("temp site created");
|
||||||
|
TempSite {
|
||||||
|
database: base.join("pool.db"),
|
||||||
|
root: base.join("local-root"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn key_for(content: &[u8]) -> Key {
|
||||||
|
Key::derive(content, content.len() as u64)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The sqlite kv engine behind the constructed store — the schema
|
||||||
|
/// observations' instrument.
|
||||||
|
fn sqlite_engine(store: &Store) -> &SqliteBackend {
|
||||||
|
match store.kv() {
|
||||||
|
#[cfg(feature = "sqlite")]
|
||||||
|
KvEngine::Sqlite(engine) => engine,
|
||||||
|
#[cfg(feature = "mem")]
|
||||||
|
KvEngine::Mem(_) => panic!("these tests construct sqlite stores"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The engine-neutral GC-schema call ran at construction: the pool
|
||||||
|
/// table exists (the engine's own open), `gc_window` exists (the
|
||||||
|
/// store-core call), `gc_pins` never (sqlite's applicability row,
|
||||||
|
/// ADR-012 §5's one-window-host rule — the kv engine hosts the
|
||||||
|
/// window and nothing else).
|
||||||
|
async fn assert_gc_schema_ran(store: &Store) {
|
||||||
|
let engine = sqlite_engine(store);
|
||||||
|
assert!(
|
||||||
|
engine.table_exists("blobs").await.unwrap(),
|
||||||
|
"the pool table exists at construction"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
engine.table_exists("gc_window").await.unwrap(),
|
||||||
|
"ensure_gc_schema ran at construction: the window table exists in the fresh database"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
!engine.table_exists("gc_pins").await.unwrap(),
|
||||||
|
"gc_pins is never created on sqlite (ADR-012 §5)"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The dual-tier construction: both tiers present, mode/threshold
|
||||||
|
/// introspection exact, boundary routing deterministic, and the
|
||||||
|
/// GC schema ran.
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dual_tier_sqlite_local_constructs_and_routes() {
|
||||||
|
let site = temp_site("dual-tier");
|
||||||
|
let store = Store::dual_tier_sqlite_local(
|
||||||
|
&site.database,
|
||||||
|
site.path(),
|
||||||
|
1024,
|
||||||
|
DEFAULT_DELETE_WINDOW_DELAY,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect("dual-tier sqlite+local constructs");
|
||||||
|
assert_eq!(store.mode(), DualTier);
|
||||||
|
assert_eq!(store.threshold(), Some(1024));
|
||||||
|
assert_eq!(store.window_delay(), DEFAULT_DELETE_WINDOW_DELAY);
|
||||||
|
assert!(store.large().is_some(), "both tiers present");
|
||||||
|
|
||||||
|
for (len, tier) in [
|
||||||
|
(0u64, Tier::Kv),
|
||||||
|
(1023, Tier::Kv),
|
||||||
|
(1024, Tier::Kv),
|
||||||
|
(1025, Tier::Large),
|
||||||
|
(u64::MAX, Tier::Large),
|
||||||
|
] {
|
||||||
|
assert_eq!(
|
||||||
|
store.tier_for(len),
|
||||||
|
tier,
|
||||||
|
"length {len} routes deterministically"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_gc_schema_ran(&store).await;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The kv-only construction: no large tier, the threshold is the
|
||||||
|
/// rejection point's carrier, and the rejection wiring is the put
|
||||||
|
/// seam's unchanged — known-length over-threshold rejects
|
||||||
|
/// immediately (zero residue), unknown-length at mid-stream
|
||||||
|
/// overflow, and exactly-at-threshold puts land.
|
||||||
|
#[tokio::test]
|
||||||
|
async fn kv_only_sqlite_constructs_and_rejects_over_threshold_puts() {
|
||||||
|
let site = temp_site("kv-only");
|
||||||
|
let store = Store::kv_only_sqlite(&site.database, 64, DEFAULT_DELETE_WINDOW_DELAY)
|
||||||
|
.await
|
||||||
|
.expect("kv-only sqlite constructs");
|
||||||
|
assert_eq!(store.mode(), KvOnly);
|
||||||
|
assert_eq!(store.threshold(), Some(64));
|
||||||
|
assert_eq!(store.window_delay(), DEFAULT_DELETE_WINDOW_DELAY);
|
||||||
|
assert!(store.large().is_none(), "kv-only has no large tier");
|
||||||
|
assert_gc_schema_ran(&store).await;
|
||||||
|
|
||||||
|
for len in [0u64, 63, 64, 65, u64::MAX] {
|
||||||
|
assert_eq!(
|
||||||
|
store.tier_for(len),
|
||||||
|
Tier::Kv,
|
||||||
|
"kv-only routes every length kv"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
let err = store
|
||||||
|
.put(Some(65), Cursor::new(&b"never read"[..]))
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(
|
||||||
|
err.to_string().contains("kv-only mode rejects"),
|
||||||
|
"the immediate rejection is the mode's: {err}"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
store.list().await.unwrap(),
|
||||||
|
Vec::<Key>::new(),
|
||||||
|
"zero residue"
|
||||||
|
);
|
||||||
|
|
||||||
|
let streamed = [3u8; 100];
|
||||||
|
let err = store
|
||||||
|
.put(None, Cursor::new(&streamed[..]))
|
||||||
|
.await
|
||||||
|
.unwrap_err();
|
||||||
|
assert!(
|
||||||
|
err.to_string()
|
||||||
|
.contains("kv-only mode rejects an over-threshold put"),
|
||||||
|
"the overflow rejection is the mode's, not the stream's: {err}"
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
store.list().await.unwrap(),
|
||||||
|
Vec::<Key>::new(),
|
||||||
|
"zero residue"
|
||||||
|
);
|
||||||
|
|
||||||
|
let at_threshold = vec![9u8; 64];
|
||||||
|
let (key, pin) = store
|
||||||
|
.put(Some(64), Cursor::new(&at_threshold[..]))
|
||||||
|
.await
|
||||||
|
.expect("exactly-at-threshold known put lands");
|
||||||
|
assert_eq!(key, key_for(&at_threshold));
|
||||||
|
assert!(store.has(&key.as_bytes()).await.unwrap());
|
||||||
|
drop(pin);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The window-delay parameter: the shipped default is the pinned
|
||||||
|
/// 5-minute constant, the constructor's value passes through the
|
||||||
|
/// store's introspection, and `Duration::ZERO` is a legal
|
||||||
|
/// degenerate tuning (execute-on-next-sweep).
|
||||||
|
#[tokio::test]
|
||||||
|
async fn window_delay_default_is_pinned_and_constructor_tunable() {
|
||||||
|
assert_eq!(
|
||||||
|
DEFAULT_DELETE_WINDOW_DELAY,
|
||||||
|
Duration::from_secs(5 * 60),
|
||||||
|
"the shipped default is 5 minutes (ADR-008 §2)"
|
||||||
|
);
|
||||||
|
|
||||||
|
let site = temp_site("delay-default");
|
||||||
|
let store = Store::dual_tier_sqlite_local(
|
||||||
|
&site.database,
|
||||||
|
site.path(),
|
||||||
|
64,
|
||||||
|
DEFAULT_DELETE_WINDOW_DELAY,
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.expect("constructs at the shipped default");
|
||||||
|
assert_eq!(store.window_delay(), Duration::from_secs(300));
|
||||||
|
|
||||||
|
let site = temp_site("delay-tuned");
|
||||||
|
let store = Store::kv_only_sqlite(&site.database, 64, Duration::from_secs(90))
|
||||||
|
.await
|
||||||
|
.expect("constructs at a custom tuning");
|
||||||
|
assert_eq!(store.window_delay(), Duration::from_secs(90));
|
||||||
|
|
||||||
|
let site = temp_site("delay-zero");
|
||||||
|
let store = Store::dual_tier_sqlite_local(&site.database, site.path(), 64, Duration::ZERO)
|
||||||
|
.await
|
||||||
|
.expect("the degenerate zero tuning is legal");
|
||||||
|
assert_eq!(store.window_delay(), Duration::ZERO);
|
||||||
|
|
||||||
|
let site = temp_site("delay-long");
|
||||||
|
Store::kv_only_sqlite(&site.database, 64, Duration::from_secs(6000))
|
||||||
|
.await
|
||||||
|
.expect("a long delay is a legal tuning too");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The default-build production shape round-trips end-to-end: a
|
||||||
|
/// sub-threshold put lands on the kv tier (sqlite), an
|
||||||
|
/// over-threshold put stages into the large tier (local), both
|
||||||
|
/// get back byte-identical, and the pool list is the sorted union
|
||||||
|
/// of both tiers.
|
||||||
|
#[tokio::test]
|
||||||
|
async fn dual_tier_sqlite_shape_round_trips_end_to_end_through_both_tiers() {
|
||||||
|
let site = temp_site("round-trip");
|
||||||
|
let store = Store::dual_tier_sqlite_local(&site.database, site.path(), 64, Duration::ZERO)
|
||||||
|
.await
|
||||||
|
.expect("dual-tier constructs");
|
||||||
|
|
||||||
|
let small = b"a sub-threshold blob";
|
||||||
|
let (key, pin) = store
|
||||||
|
.put(Some(small.len() as u64), Cursor::new(&small[..]))
|
||||||
|
.await
|
||||||
|
.expect("kv put publishes");
|
||||||
|
assert_eq!(key, key_for(small));
|
||||||
|
assert!(
|
||||||
|
store.kv().has(&key.as_bytes()).await.unwrap(),
|
||||||
|
"landed on the kv tier (sqlite)"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
!store.large().unwrap().has(&key.as_bytes()).await.unwrap(),
|
||||||
|
"no large-tier copy on the kv route"
|
||||||
|
);
|
||||||
|
let (len, blob) = store
|
||||||
|
.get(&key.as_bytes())
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.expect("entry present");
|
||||||
|
assert_eq!(len, small.len() as u64);
|
||||||
|
assert_eq!(blob.read_at(0, len).await.unwrap(), &small[..]);
|
||||||
|
drop(pin);
|
||||||
|
|
||||||
|
let big = vec![42u8; 200];
|
||||||
|
let (key, pin) = store
|
||||||
|
.put(Some(200), Cursor::new(&big[..]))
|
||||||
|
.await
|
||||||
|
.expect("large staged put publishes");
|
||||||
|
assert_eq!(key, key_for(&big));
|
||||||
|
assert!(
|
||||||
|
store.large().unwrap().has(&key.as_bytes()).await.unwrap(),
|
||||||
|
"landed on the large tier (commit-rename)"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
!store.kv().has(&key.as_bytes()).await.unwrap(),
|
||||||
|
"no kv copy on the large route"
|
||||||
|
);
|
||||||
|
let (len, blob) = store
|
||||||
|
.get(&key.as_bytes())
|
||||||
|
.await
|
||||||
|
.unwrap()
|
||||||
|
.expect("entry present");
|
||||||
|
assert_eq!(len, 200);
|
||||||
|
assert_eq!(blob.read_at(0, len).await.unwrap(), big);
|
||||||
|
drop(pin);
|
||||||
|
|
||||||
|
let listed = store.list().await.expect("list succeeds");
|
||||||
|
assert_eq!(listed.len(), 2, "one entry per tier, both enumerated");
|
||||||
|
for window in listed.windows(2) {
|
||||||
|
assert!(window[0] < window[1], "list is sorted");
|
||||||
|
}
|
||||||
|
|
||||||
|
// A re-put dedups into the existing entry (CAS first-writer-
|
||||||
|
// wins, through the facade's put).
|
||||||
|
let (_, pin) = store
|
||||||
|
.put(Some(small.len() as u64), Cursor::new(&small[..]))
|
||||||
|
.await
|
||||||
|
.expect("dedup re-put");
|
||||||
|
assert_eq!(store.list().await.unwrap().len(), 2, "still two entries");
|
||||||
|
drop(pin);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
---
|
---
|
||||||
id: store-sqlite-construction
|
id: store-sqlite-construction
|
||||||
name: sqlite constructor wiring — the default-build store shapes
|
name: sqlite constructor wiring — the default-build store shapes
|
||||||
status: pending
|
status: completed
|
||||||
depends_on: [engine-sqlite, engine-sqlite-window]
|
depends_on: [engine-sqlite, engine-sqlite-window]
|
||||||
scope: narrow
|
scope: narrow
|
||||||
risk: low
|
risk: low
|
||||||
@@ -61,19 +61,19 @@ a named constructor is slotting in"):
|
|||||||
|
|
||||||
## Acceptance Criteria
|
## Acceptance Criteria
|
||||||
|
|
||||||
- [ ] `KvEngine::Sqlite` slots in; all six kv ops dispatch; the kv
|
- [x] `KvEngine::Sqlite` slots in; all six kv ops dispatch; the kv
|
||||||
cursor body dispatches (design point recorded)
|
cursor body dispatches (design point recorded)
|
||||||
- [ ] `dual_tier_sqlite_local` + `kv_only_sqlite` constructors exist
|
- [x] `dual_tier_sqlite_local` + `kv_only_sqlite` constructors exist
|
||||||
behind feature `sqlite`, take threshold + window delay, fallible
|
behind feature `sqlite`, take threshold + window delay, fallible
|
||||||
where I/O is fallible
|
where I/O is fallible
|
||||||
- [ ] `ensure_gc_schema` runs at construction; the window-delay
|
- [x] `ensure_gc_schema` runs at construction; the window-delay
|
||||||
shipped default is 5 min (pinned constant) and
|
shipped default is 5 min (pinned constant) and
|
||||||
constructor-tunable; `0` legal
|
constructor-tunable; `0` legal
|
||||||
- [ ] The one-window-host posture recorded (sqlite hosts when the kv
|
- [x] The one-window-host posture recorded (sqlite hosts when the kv
|
||||||
engine is SQL-backed)
|
engine is SQL-backed)
|
||||||
- [ ] The stale empty-set scaffolding/allow notes updated to the
|
- [x] The stale empty-set scaffolding/allow notes updated to the
|
||||||
landed state
|
landed state
|
||||||
- [ ] `cargo test` (default) green — the default build constructs and
|
- [x] `cargo test` (default) green — the default build constructs and
|
||||||
exercises sqlite stores; `cargo clippy --all-targets -- -D
|
exercises sqlite stores; `cargo clippy --all-targets -- -D
|
||||||
warnings` and `cargo fmt --check` green
|
warnings` and `cargo fmt --check` green
|
||||||
|
|
||||||
@@ -88,8 +88,70 @@ a named constructor is slotting in"):
|
|||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
|
|
||||||
> To be filled by implementation agent
|
- **KvCursor sub-enum (the recorded design point).** The kv cursor
|
||||||
|
dispatch went with a `KvCursor` sub-enum behind `AnyCursor`'s one
|
||||||
|
`Kv` arm (rather than a per-engine `AnyCursor::Kv*` variant pair):
|
||||||
|
per-engine variants would copy the kv materializing arm and its
|
||||||
|
consumers (the verification-on-get body, `Blob`'s drive) per engine
|
||||||
|
without adding a behavior; the sub-enum keeps `AnyCursor` a two-arm
|
||||||
|
tier dispatch. Postgres (wave 4) slots one arm into `KvCursor`.
|
||||||
|
- **`KvEngine::ensure_gc_schema` — the engine-neutral store-core
|
||||||
|
call** lives on the kv-tier enum itself: the sqlite arm forwards to
|
||||||
|
the `EngineWindowState` clause; the mem arm is structurally nothing
|
||||||
|
(the in-process engines execute the ADR-005 window protocol inline
|
||||||
|
and carry no GC schema). Both sqlite constructors dispatch through
|
||||||
|
it — the store core calls the kv engine, never a typed sqlite path,
|
||||||
|
which is the recording shape the one-window-host posture wanted.
|
||||||
|
- **The window delay** is a named shipped constant
|
||||||
|
(`DEFAULT_DELETE_WINDOW_DELAY`, 5 min, ADR-008 §2) re-exported from
|
||||||
|
`lib.rs`; both sqlite constructors take it explicitly
|
||||||
|
(`Duration`, `Duration::ZERO` the legal degenerate tuning) and the
|
||||||
|
store carries it as a `window_delay` field with a crate-internal
|
||||||
|
accessor (`Store::window_delay`) for the durable sweep form
|
||||||
|
(`store-sweep-durable-window`) to consume. The mem/local
|
||||||
|
constructors are signature-unchanged; they carry the constant,
|
||||||
|
never consulted (the inline form has no delay in effect — same
|
||||||
|
posture as the mem-only threshold field).
|
||||||
|
- **Fallibility split**: `dual_tier_sqlite_local(database, root,
|
||||||
|
threshold, window_delay)` is fallible on the database file I/O,
|
||||||
|
`ensure_gc_schema`, *and* the local root creation (all
|
||||||
|
`Err(Error::Io)` via `?`); `kv_only_sqlite(database, threshold,
|
||||||
|
window_delay)` on the first two. WAL engagement failure (e.g.
|
||||||
|
`:memory:`) is a construction error, per the engine's recorded
|
||||||
|
posture.
|
||||||
|
- `SqliteBackend` gained a `#[cfg(test)] table_exists` probe (the
|
||||||
|
sqlite_master instrument) so the store tests can observe the
|
||||||
|
schema from the facade side; sqlite.rs's own test helper now
|
||||||
|
delegates to it.
|
||||||
|
- The stale empty-set scaffolding is updated to the landed state:
|
||||||
|
the `#[allow(unused_variables)]` attrs came off the `KvEngine`
|
||||||
|
methods; the scope notes in store.rs, backend.rs
|
||||||
|
(whole-value `put` now has its in-tree callers), sqlite.rs
|
||||||
|
(nothing-constructs-the-engine), and gc.rs (the window family's
|
||||||
|
first caller) reflect the landed state. `--no-default-features`
|
||||||
|
builds (kv set genuinely empty again) still compile under their
|
||||||
|
own cfg arithmetic — verified along with the default, mem-only,
|
||||||
|
and all-features builds.
|
||||||
|
|
||||||
## Summary
|
## Summary
|
||||||
|
|
||||||
> To be filled on completion
|
Sqlite slotted into the facade's engine representation (wave 3
|
||||||
|
construction; commit pending at write time): `KvEngine::Sqlite` with
|
||||||
|
all six kv-op dispatch arms, the `KvCursor` sub-enum carrying the
|
||||||
|
sqlite materializing body behind `AnyCursor`'s kv arm (design point
|
||||||
|
recorded), the two named constructors (`dual_tier_sqlite_local` — the
|
||||||
|
default-build production shape, and `kv_only_sqlite`) taking
|
||||||
|
threshold + the constructor-tunable delete-window delay
|
||||||
|
(`DEFAULT_DELETE_WINDOW_DELAY` pinned at 5 min, `0` legal),
|
||||||
|
`ensure_gc_schema` running at construction through the engine-neutral
|
||||||
|
`KvEngine` call, the one-window-host posture recorded in the module
|
||||||
|
doc (the durable window is the kv engine's when the kv engine is
|
||||||
|
SQL-backed — wave 5's pg-lo dormancy slots in without reshaping),
|
||||||
|
and all stale empty-set allow/scope notes updated to the landed
|
||||||
|
state. Verification: `cargo test` 62 passing (default build — the
|
||||||
|
5 new sqlite-construction tests construct, introspect, schema-probe,
|
||||||
|
rejection-check, and end-to-end round-trip both tiers),
|
||||||
|
`cargo test --all-features` 117, `cargo test --features mem` 117,
|
||||||
|
`--no-default-features --features mem` 97, clippy `-D warnings`
|
||||||
|
(default + all-features) clean, `cargo fmt --check` clean, and
|
||||||
|
`cargo doc --no-deps` at the pre-existing warning baseline.
|
||||||
Reference in new issue
Block a user