glm-5.3-flash
abf8c5ae9c
ADR-010 (new): the Backend trait's I/O seams pinned before any code — 'get' returns a crate-internal read cursor (kv engines materialize; 'local' pread loop; pg-lo lo_get windows), 'put' has two named forms (whole-value kv / staged-put large); GC state is store-core-owned via a crate-internal contract-tested engine-state seam on the SQL-backed engines (backends never learn liveness — ADR-005 verbatim); the store core holds the joint entry+pin transaction (corrects ADR-009 §2's ownership statement); kv-tier fleet-validity rule added (mirror of ADR-008 §3); ops pin-token TTL race resolved (the token IS the pin; renewal rides have/re-put; pause-past-TTL falls to delete-then-recover); feature graph pinned (pg-lo does not imply postgres). ADR-011 (new): vocabulary pinned — tier (contract, exactly two) vs engine (concrete impl, exactly five); store instance / node / fleet split (fleet = pool-sharing, not node count — fixes requirements.md's self-contradiction with REQ-2); mem demoted from 'backend'/'testing tier' to the kv tier's third engine (contract-reference engine); fs tier renamed 'large' including the feature name (last free moment before code exists); constructor modes pinned (dual-tier default, kv-only, mem-only) plus the composite fleet-validity predicate. Consistency round across all specs and ADR-003/004/007/008/009: stale pg-lo passages resolved per ADR-009's supersession; ADR-008 owner 'node id' → store-instance id; threshold wording corrected (conservative edge, not midpoint, of the 128-256 KiB crossover zone); SweepReport shape specced; mem classification unified; 008/009 ADR files renamed to match the tier rename; README deferral-policy recap aligned (third category = decided-but-sequenced work, not a parking kind); POC crate list completed. Verification: two independent architecture review rounds (the first found 4 criticals — unpinned trait I/O shapes, missing fleet-state seam, node/fleet self-contradiction, pin-token/TTL conflict — all resolved; final round: zero criticals); all markdown links resolve; ADR tables complete (11 ADRs).
208 lines
11 KiB
Markdown
208 lines
11 KiB
Markdown
---
|
||
status: draft
|
||
last_updated: 2026-10-03 (ADR-010 I/O seams + ADR-011 rename — large tier → large)
|
||
---
|
||
|
||
# Store API
|
||
|
||
## What this is
|
||
|
||
The store core's public surface: the typed facade above the tier
|
||
engines that consumers (alkgit, alkfs, the ops module) program against.
|
||
It owns typed keys and hashing (ADR-002), size-threshold dispatch
|
||
(ADR-003), the put-pinning seam, and the liveness/sweep seam (ADR-005)
|
||
— and nothing else: no paths, no manifests, no wire (ADR-004, ADR-001).
|
||
The facade is **engine-blind** (ADR-007/008/010): which engines a node
|
||
runs changes nothing here except fleet-GC mechanism wiring (ADR-008's
|
||
mechanism lives in the engine-state seam, ADR-010 §2).
|
||
|
||
## Public surface
|
||
|
||
### Write path
|
||
|
||
- **`put(len: Option<u64>, stream)` → `(Key, Pin)`** — the streaming
|
||
put seam (POC #3 finding A1). Two internal paths behind one signature:
|
||
- *Known length* (encouraged, documented as such on the API): the
|
||
preamble hashes before content flows; one pass, no staging needed
|
||
for hashing. Covers git objects, pre-staged files (`stat`-derived
|
||
length), network offers, `Content-Length`-bearing uploads.
|
||
- *Unknown length*: content buffers in memory up to the dispatch
|
||
threshold; on commit the buffer is the whole stored artifact
|
||
(kv tier) — or, on mid-stream threshold overflow, the buffer
|
||
flushes into a stage file and the stream continues into it
|
||
(large tier — the streaming pass cannot include the preamble, so
|
||
the derivation pass restarts over preamble + staged bytes; forced
|
||
by the preamble being inside the hashed input, not a design
|
||
choice). Verification runs over the eventual stored artifact as a
|
||
whole; a stream error on either path leaves no entry and no stage
|
||
file. Eliminates the A6 dispatch asymmetry by construction:
|
||
routing is a pure function of content length (ADR-003). The large
|
||
tier's stage mechanics are the **staged-put contract** (ADR-010
|
||
§1): the engine hands out a staging cursor (`local`: the stage file;
|
||
pg-lo: chunked `lowrite` statements inside its tx), commit
|
||
publishes, every failure path discards.
|
||
- **Stage-file hygiene invariant** (POC #3 finding A5, codified): every
|
||
failure path converges on stage-discard; every success path converges
|
||
on commit-rename. No early returns that bypass cleanup; this is an
|
||
asserted invariant with dedicated exact-count tests, not review
|
||
hygiene.
|
||
- **`Pin` / `Batch`** — RAII guard on the put path (ADR-005): the
|
||
entry is liveness-protected until the guard drops or the caller
|
||
converts it into a consumer-side reference (registered liveness, a
|
||
root tag). **`Batch`** groups multi-put writes under one batch-scoped
|
||
pin (manifest writes are exactly this); a batch's pins drop together
|
||
on batch commit or drop (ADR-005 §Decision).
|
||
|
||
### Read path
|
||
|
||
- **`get(key) → Option<(len, stream)>`** — small tier returns a
|
||
bounded read; large tier streams (`local`: a pread loop; pg-lo: `lo_get`
|
||
windows). The per-engine shapes are hidden behind the crate-internal
|
||
read cursor the store core drives (ADR-010 §1 — engines construct
|
||
their own cursor bodies; the facade's shape is uniform). Dispatch
|
||
fall-through: a small-tier miss queries the large tier (POC #1
|
||
finding 6).
|
||
- **`stat(key) → Option<EntryMeta>`** — cheap length/type probe
|
||
(phase-0 OQ-BL-04 addendum; gix's header-only read is git's cheapest
|
||
primitive and the packfile-serving surface rides this). `EntryMeta`
|
||
carries `len: u64` and the key's algorithm (the "type" from the
|
||
store's perspective is the hash algorithm — there is no other type at
|
||
this layer). `stat` rides the `Backend` trait's `size` method
|
||
(ADR-008): an indexed-column probe on kv engines, a file stat on
|
||
`local`, a companion-row probe on pg-lo — never a value/content
|
||
load.
|
||
- **`read_range(key, range) → (slice, slice_digest)`** — slice plus an
|
||
out-of-band slice digest: SHA-256 over the slice bytes, fixed by
|
||
convention so callers and ops handlers agree without negotiation
|
||
(per-range verification against the canonical digest is impossible —
|
||
ADR-006). Range reads fall through tiers exactly as `get` does
|
||
(small-tier miss slices the large tier's value). Local range serving
|
||
(packfiles) is the consumer of this.
|
||
- **`get_stream` / fanout** — the ops module's fetch handler is a
|
||
broadcast above the store: one reader, store arm + subscriber
|
||
arms (POC #3 finding A3); late joiners degrade to normal verified
|
||
`get` after commit. The store exposes the seam; the fanout policy
|
||
lives in the ops layer (ADR-001).
|
||
|
||
### Lifecycle
|
||
|
||
- **`has(key)`, `delete(key)`** — deletion participates in ADR-005's
|
||
delete windows; direct deletes are permitted but refuse protected
|
||
keys — pinned or re-observed live via registered sources (the same
|
||
per-key arbitration sweeps apply, typed error) — and are unusual by
|
||
posture; most deletion flows through sweeps.
|
||
- **`list() → stream of keys`** — whole-pool enumeration; complete by
|
||
contract (see backends doc for why list correctness is load-bearing).
|
||
- **`register_liveness_source(...)`** — the live-shared GC seam
|
||
(ADR-005, §Decision; the POC's "install implies copy" failure
|
||
(finding 7) is why the verb/name is pinned here).
|
||
- **`sweep() → SweepReport`** — explicit mark-and-sweep; embedders own
|
||
the cadence (ADR-005, no ambient timers). A sweep with no registered
|
||
liveness sources aborts without deleting — the safe default.
|
||
`SweepReport`'s shape (specced here, resolving its earlier
|
||
name-only status): `candidates_staged`, `deleted`,
|
||
`cancelled_by_arbitration` (u64 counts), `aborted: Option<GcAbortCause>`
|
||
(the typed cause — protection-source failure (ADR-005) or fleet
|
||
sweeper-lock contention (ADR-008) — present iff the sweep deleted
|
||
nothing for an abort reason; named `GcAbortCause` to keep it a data
|
||
enum, distinct from the `GcAborted` error which carries the
|
||
direct-delete refusal path), and `started_at`/`finished_at`. The
|
||
report is data, not an error carrier: an aborted sweep returns
|
||
`Ok(report)` with `aborted: Some(..)` — the typed error
|
||
`GcAborted` is reserved for the direct-delete refusal path.
|
||
|
||
Under a fleet (multiple store instances over one pool —
|
||
requirements.md REQ-2), sweep/put/delete semantics are
|
||
mechanism-extended by ADR-008: pins are DB-backed rows committed
|
||
atomically with entries (the store core holds the joint tx,
|
||
ADR-010 §2), one sweeper holds the advisory lock, and delete
|
||
arbitration is SQL-level — the facade's invariant text is
|
||
unchanged.
|
||
|
||
## Error model
|
||
|
||
`thiserror`; no panics in library code; no `unwrap`/`expect` outside
|
||
tests (AGENTS convention 2). Distinguished failure families:
|
||
|
||
- `Missing` — key absent (get/stat miss)
|
||
- `Verification` — put/get hash-check failure (content ≠ key)
|
||
- `Io(String)` — backend media failure (stringly because
|
||
`std::io::Error` is not stable across versions)
|
||
- `GcAborted` — nothing deleted; sweep not run. Two cause classes
|
||
(ADR-005: a protection source failed; ADR-008: another node holds
|
||
the fleet sweeper lock) — the typed error distinguishes them.
|
||
- `KeyInvalid` — malformed key bytes at the boundary (hashing doc)
|
||
|
||
Virgin-store semantics: read paths on a fresh store see absent/empty,
|
||
never "table does not exist" errors (POC #1 finding 2 — the redb
|
||
lesson generalizes to any kv engine).
|
||
|
||
## Concurrency posture
|
||
|
||
- Async I/O throughout; `tokio::sync` for lifecycle correlation;
|
||
`parking_lot` for short-held internal locks (AGENTS convention 3);
|
||
poisoned locks degrade via `unwrap_or_else(|e| e.into_inner())`.
|
||
- Blocking file work lives in `spawn_blocking` inside backend impls
|
||
(the alkgit trait-execution pattern); the store never blocks the
|
||
executor.
|
||
- The sweep-vs-put window is an architectural mechanism — delete
|
||
windows — specified in ADR-005, not an implementation note.
|
||
|
||
## Invariants (the test gate)
|
||
|
||
1. Whole-blob put/get round-trips byte-identical under the canonical
|
||
derivation, both tiers, both put paths, **and every shipped engine
|
||
of both tiers** (kv: sqlite + postgres + mem; large: `local` +
|
||
pg-lo; ADR-007/008/009/011)
|
||
(interop with real git remains the source of truth, per POC #1's
|
||
lesson about hardcoded vectors).
|
||
2. Dedup: putting identical content twice (same or different path) is
|
||
one pool entry.
|
||
3. Stage hygiene: any failure mid-put leaves zero stage files; any
|
||
success leaves exactly one committed entry.
|
||
4. Sweep safety: with correct liveness registered, sweep counts are
|
||
exact; with aborting sources, sweep deletes nothing. **Engine-gated
|
||
form (ADR-008):** single-node postures assert this under the
|
||
in-process protocol; fleet postures (shared-pool engines) assert it
|
||
under DB-backed pins + the advisory-locked sweep, with the
|
||
exact-count sweep-outcome shape run against dockerized postgres
|
||
under `--all-features`.
|
||
5. `list()` correctness is observable only through GC — list-related
|
||
tests assert through sweep outcomes (POC #1 finding 2's lesson
|
||
codified).
|
||
6. `size`/`stat` on any engine returns the entry length without
|
||
fetching content (asserted per engine: kv row probe, `local` stat,
|
||
pg-lo companion-row probe — ADR-008/009).
|
||
7. **Engine-conformance uniformity (ADR-010 §1):** every engine's
|
||
cursor/stage/list bodies pass the same contract suite mem
|
||
(the reference engine) passes — exact-count sweep outcomes, stage
|
||
hygiene, virgin-store no-ops, CAS re-put, ranged reads (the POC #7
|
||
gate is the norm; per-engine suite runs in CI, `--all-features` for
|
||
the feature-gated engines).
|
||
|
||
## Design Decisions
|
||
|
||
| ADR | Decision | Summary |
|
||
|---|---|---|
|
||
| [002](decisions/002-canonical-hash-and-key-encoding.md) | Hashing & keys | derivation + encoding this surface is built on |
|
||
| [003](decisions/003-backend-contract-and-dispatch.md) | Dispatch | routing is a pure function of length; pre-threshold buffering |
|
||
| [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Pools & GC | pins, liveness seams, delete windows, sweep semantics |
|
||
| [006](decisions/006-verification-posture-and-transfer-encoding.md) | Verification | whole-blob checks; slice digests out-of-band |
|
||
| [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | Trait `size` + fleet GC | stat/accounting ride the trait's `size` probe; fleet pins/sweeps are engine-backed |
|
||
| [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | I/O seams + GC-state home | read cursor / staged put behind the facade; the store core holds the joint entry+pin tx |
|
||
|
||
## Open Questions
|
||
|
||
None owned by this document beyond the cross-references above.
|
||
|
||
## References
|
||
|
||
- `docs/research/poc-largeblob-findings.md` findings A1/A5/A6,
|
||
benchmark table
|
||
- `docs/research/poc-trait-dispatch-findings.md` findings 1–3, 7
|
||
- ADR-003 (put-path buffering), ADR-005 (pin/Batch/Pin lifecycle),
|
||
ADR-006 (range-read semantics), ADR-002 (keys)
|
||
- [ops-surface.md](ops-surface.md) — the fanout/fetch consumer of the
|
||
read path
|
||
- [gc-and-namespaces.md](gc-and-namespaces.md) — the lifecycle seam
|
||
details |