docs(architecture): Phase 1 spec tree — 7 specs, ADRs 001-006, OQ tracker
- overview/hashing/store-api/backends/gc/ops specs + README index, all draft status with YAML frontmatter and cross-referenced ADR/OQ - ADR-001: substrate posture (store layer alkcall-free = family conformance, not deviation) + ops placement decided (feature-gated module, no consumer-waited deferral) - ADR-002: canonical git-blob-sha-256 + key encoding — the declared wire-format ADR (one-way door) - ADR-003: backend contract, sqlite/fs/mem tiers, pure-function dispatch, no migration, unknown-length buffering semantics pinned - ADR-004: exactly two production backends; manifest/path-tree layers are consumer-side (the corrected two-vs-three-backends framing) - ADR-005: pooled CAS, three liveness sources, delete windows with pin-before-publish + delete-time arbitration (race closed), no ambient scheduling - ADR-006: whole-blob verification; chunk-tree encodings excluded by scoping, not deferred on a paused consumer - open-questions.md: OQ-BL-01..06 resolved with ADR cross-refs; OQ-07 /08 externally-owned, OQ-09 deferred(scope) with SDD tracker task - AGENTS.md: status updated to Phase 1, gix-odb path corrected Verification: cargo test / clippy -D warnings / fmt --check clean
This commit is contained in:
glm-5.3-flash committed
1 parent
7d9ac8a6ec
commit
4b5009d85a
16 files changed
+1953
-8
No files matched your search
@@ -0,0 +1,156 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-01
|
||||
---
|
||||
|
||||
# Store API
|
||||
|
||||
## What this is
|
||||
|
||||
The store core's public surface: the typed facade above the backends
|
||||
that consumers (alkgit, alkfs, the ops module) program against. It owns
|
||||
typed keys and hashing (ADR-002), size-threshold dispatch (ADR-003),
|
||||
the put-pinning seam, and the liveness/sweep seam (ADR-005) — and
|
||||
nothing else: no paths, no manifests, no wire (ADR-004, ADR-001).
|
||||
|
||||
## Public surface
|
||||
|
||||
### Write path
|
||||
|
||||
- **`put(len: Option<u64>, stream)` → `(Key, Pin)`** — the streaming
|
||||
put seam (POC #3 finding A1). Two internal paths behind one signature:
|
||||
- *Known length* (encouraged, documented as such on the API): the
|
||||
preamble hashes before content flows; one pass, no staging needed
|
||||
for hashing. Covers git objects, pre-staged files (`stat`-derived
|
||||
length), network offers, `Content-Length`-bearing uploads.
|
||||
- *Unknown length*: content buffers in memory up to the dispatch
|
||||
threshold; on commit the buffer is the whole stored artifact
|
||||
(kv tier) — or, on mid-stream threshold overflow, the buffer
|
||||
flushes into a stage file and the stream continues into it
|
||||
(fs tier — the streaming pass cannot include the preamble, so the
|
||||
derivation pass restarts over preamble + staged bytes; forced by
|
||||
the preamble being inside the hashed input, not a design choice).
|
||||
Verification runs over the eventual stored artifact as a whole;
|
||||
a stream error on either path leaves no entry and no stage file.
|
||||
Eliminates the A6 dispatch asymmetry by construction: routing is
|
||||
a pure function of content length (ADR-003).
|
||||
- **Stage-file hygiene invariant** (POC #3 finding A5, codified): every
|
||||
failure path converges on stage-discard; every success path converges
|
||||
on commit-rename. No early returns that bypass cleanup; this is an
|
||||
asserted invariant with dedicated exact-count tests, not review
|
||||
hygiene.
|
||||
- **`Pin` / `Batch`** — RAII guard on the put path (ADR-005): the
|
||||
entry is liveness-protected until the guard drops or the caller
|
||||
converts it into a consumer-side reference (registered liveness, a
|
||||
root tag). **`Batch`** groups multi-put writes under one batch-scoped
|
||||
pin (manifest writes are exactly this); a batch's pins drop together
|
||||
on batch commit or drop (ADR-005 §Decision).
|
||||
|
||||
### Read path
|
||||
|
||||
- **`get(key) → Option<(len, stream)>`** — small tier returns a
|
||||
bounded read; large tier streams a file handle. Dispatch
|
||||
fall-through: a small-tier miss queries the large tier (POC #1
|
||||
finding 6).
|
||||
- **`stat(key) → Option<EntryMeta>`** — cheap length/type probe
|
||||
(phase-0 OQ-BL-04 addendum; gix's header-only read is git's cheapest
|
||||
primitive and the packfile-serving surface rides this). `EntryMeta`
|
||||
carries `len: u64` and the key's algorithm (the "type" from the
|
||||
store's perspective is the hash algorithm — there is no other type at
|
||||
this layer).
|
||||
- **`read_range(key, range) → (slice, slice_digest)`** — slice plus an
|
||||
out-of-band slice digest: SHA-256 over the slice bytes, fixed by
|
||||
convention so callers and ops handlers agree without negotiation
|
||||
(per-range verification against the canonical digest is impossible —
|
||||
ADR-006). Range reads fall through tiers exactly as `get` does
|
||||
(small-tier miss slices the large tier's value). Local range serving
|
||||
(packfiles) is the consumer of this.
|
||||
- **`get_stream` / fanout** — the ops module's fetch handler is a
|
||||
broadcast above the store: one reader, store arm + subscriber
|
||||
arms (POC #3 finding A3); late joiners degrade to normal verified
|
||||
`get` after commit. The store exposes the seam; the fanout policy
|
||||
lives in the ops layer (ADR-001).
|
||||
|
||||
### Lifecycle
|
||||
|
||||
- **`has(key)`, `delete(key)`** — deletion participates in ADR-005's
|
||||
delete windows; direct deletes are permitted but refuse protected
|
||||
keys — pinned or re-observed live via registered sources (the same
|
||||
per-key arbitration sweeps apply, typed error) — and are unusual by
|
||||
posture; most deletion flows through sweeps.
|
||||
- **`list() → stream of keys`** — whole-pool enumeration; complete by
|
||||
contract (see backends doc for why list correctness is load-bearing).
|
||||
- **`register_liveness_source(...)`** — the live-shared GC seam
|
||||
(ADR-005, §Decision; the POC's "install implies copy" failure
|
||||
(finding 7) is why the verb/name is pinned here).
|
||||
- **`sweep() → SweepReport`** — explicit mark-and-sweep; embedders own
|
||||
the cadence (ADR-005, no ambient timers). A sweep with no registered
|
||||
liveness sources aborts without deleting — the safe default.
|
||||
|
||||
## Error model
|
||||
|
||||
`thiserror`; no panics in library code; no `unwrap`/`expect` outside
|
||||
tests (AGENTS convention 2). Distinguished failure families:
|
||||
|
||||
- `Missing` — key absent (get/stat miss)
|
||||
- `Verification` — put/get hash-check failure (content ≠ key)
|
||||
- `Io(String)` — backend media failure (stringly because
|
||||
`std::io::Error` is not stable across versions)
|
||||
- `GcAborted` — a protection source failed; nothing deleted (ADR-005)
|
||||
- `KeyInvalid` — malformed key bytes at the boundary (hashing doc)
|
||||
|
||||
Virgin-store semantics: read paths on a fresh store see absent/empty,
|
||||
never "table does not exist" errors (POC #1 finding 2 — the redb
|
||||
lesson generalizes to any kv engine).
|
||||
|
||||
## Concurrency posture
|
||||
|
||||
- Async I/O throughout; `tokio::sync` for lifecycle correlation;
|
||||
`parking_lot` for short-held internal locks (AGENTS convention 3);
|
||||
poisoned locks degrade via `unwrap_or_else(|e| e.into_inner())`.
|
||||
- Blocking file work lives in `spawn_blocking` inside backend impls
|
||||
(the alkgit trait-execution pattern); the store never blocks the
|
||||
executor.
|
||||
- The sweep-vs-put window is an architectural mechanism — delete
|
||||
windows — specified in ADR-005, not an implementation note.
|
||||
|
||||
## Invariants (the test gate)
|
||||
|
||||
1. Whole-blob put/get round-trips byte-identical under the canonical
|
||||
derivation, both tiers, both put paths (interop with real git
|
||||
remains the source of truth, per POC #1's lesson about hardcoded
|
||||
vectors).
|
||||
2. Dedup: putting identical content twice (same or different path) is
|
||||
one pool entry.
|
||||
3. Stage hygiene: any failure mid-put leaves zero stage files; any
|
||||
success leaves exactly one committed entry.
|
||||
4. Sweep safety: with correct liveness registered, sweep counts are
|
||||
exact; with aborting sources, sweep deletes nothing.
|
||||
5. `list()` correctness is observable only through GC — list-related
|
||||
tests assert through sweep outcomes (POC #1 finding 2's lesson
|
||||
codified).
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|---|---|---|
|
||||
| [002](decisions/002-canonical-hash-and-key-encoding.md) | Hashing & keys | derivation + encoding this surface is built on |
|
||||
| [003](decisions/003-backend-contract-and-dispatch.md) | Dispatch | routing is a pure function of length; pre-threshold buffering |
|
||||
| [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Pools & GC | pins, liveness seams, delete windows, sweep semantics |
|
||||
| [006](decisions/006-verification-posture-and-transfer-encoding.md) | Verification | whole-blob checks; slice digests out-of-band |
|
||||
|
||||
## Open Questions
|
||||
|
||||
None owned by this document beyond the cross-references above.
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/poc-largeblob-findings.md` findings A1/A5/A6,
|
||||
benchmark table
|
||||
- `docs/research/poc-trait-dispatch-findings.md` findings 1–3, 7
|
||||
- ADR-003 (put-path buffering), ADR-005 (pin/Batch/Pin lifecycle),
|
||||
ADR-006 (range-read semantics), ADR-002 (keys)
|
||||
- [ops-surface.md](ops-surface.md) — the fanout/fetch consumer of the
|
||||
read path
|
||||
- [gc-and-namespaces.md](gc-and-namespaces.md) — the lifecycle seam
|
||||
details
|
||||
Reference in new issue
Block a user