--- status: draft last_updated: 2026-10-02 --- # Overview ## What this crate is `alkblobs` is content-addressed blob storage in the alk* family: a put/get/verify store keyed under one canonical content hash (the git-family derivation; legacy SHA-1-tolerant), with pluggable backends (small blobs in a kv store, large blobs on a filesystem fallback), split from any wire/ops-protocol layer. Network ops ride the alkcall substrate behind a feature-gated `ops` module (ADR-001). ## Why it exists — the extracted shared core Two downstream projects were converging on the same storage stack, so it is being built once, here: - **alkgit** was paused mid-Phase-1 (its architecture is reviewed, `docs/architecture/` in the alkgit repo) to stop duplicated storage-layer work. Its object backend needs git's own oid derivation, a better small-object backend than its proposed default file layout, and a git-lfs-shaped large-blob story. Under the canonical-hash resolution (ADR-002) git objects are first-class pool entries with zero indirection. - **alkfs** (Phase 0 pending) needs the appfile/workspace shape: small content in a fast kv store, large content on the filesystem, path→hash mapping above. The path-tree/manifest layer is alkgit's and alkfs's, not this crate's (ADR-004). Both demanding consumers (p2p git replicators, agent workspaces) reduce to the same shape: **a pooled CAS per node**, where repos and workspaces are *sets of hash references*, not walled-off object stores — cross-repo dedup by construction (ADR-005). That pooling property is why the storage layer was pulled out of both consumers rather than built twice or three times. ## Layer map ``` ┌────────────────────────────────────────────────────────────┐ │ consumers (above this crate) │ │ alkgit object storage · alkfs path trees/workspaces │ │ refs, manifests, path→hash maps, git semantics │ └────────────┬───────────────────────────────────────────────┘ ▼ ┌────────────────────────────────────────────────────────────┐ │ ops surface (feature "ops"; ADR-001) │ │ have/need + verified fetch + put; JSON control ops, │ │ binary data channels; ACL via alkcall AccessControl │ └────────────┬───────────────────────────────────────────────┘ ▼ ┌────────────────────────────────────────────────────────────┐ │ store core (always on) │ │ hashing & keys (ADR-002) · put/get/stat/range/pin seams │ │ mark-and-sweep GC + liveness registration (ADR-005) │ │ size-threshold dispatch (ADR-003) │ └───────┬───────────────────┬───────────────────────────────┘ ▼ ▼ kv backend fs backend (ADR-003: exactly two) transport: none, anywhere. alkgit/alkfs or the embedder dials; an alkcall Connection is handed in at the ops boundary. ``` ## Dependency posture | Layer | Dependencies | Notes | |---|---|---| | base crate (store core + backends) | `tokio`, `thiserror`, `parking_lot`, plus `rusqlite` behind the `kv` feature and nothing behind `fs`/`mem` | no alkcall, no serialization frameworks | | `ops` feature | adds `alkcall` (feature-gated, default-off) | JSON op specs + binary channels + `AccessControl`; ADR-001 | | consumer crates | depend on alkblobs (and on alkcall directly when they speak ops) | alkgit, alkfs | Backends are feature-gated; both `kv` and `fs` ship default-on (batteries-included, the alkgit feature-model pattern); the `ops` feature is default-off. Wasm: the backends are not a wasm story; a wasm client is a consumer of the `ops` surface (which rides alkcall, itself wasm-clean), never an in-process embedder (ADR-001 §Consequences). ## Non-goals (each enforced by an ADR) - No wire framing, op dispatch, ACL, or transport inside the store layer (ADR-001). - No adopted iroh-blobs wire surface — no tickets, no postcard, no provider protocol (AGENTS convention 7; ADR-001 §Context). - No namespaces-as-physical-storage; the pool is flat, namespaces are consumer-level reference sets (ADR-005). - No manifest/path-tree/vfs tables; consumers never implement a Backend to model paths (ADR-004). - No chunk-tree/CDC storage encoding; whole-file CAS is the store's only granularity (ADR-006). - No per-repo or per-namespace walled stores — one pooled CAS (ADR-005). ## Design Decisions | ADR | Decision | Summary | |---|---|---| | [001](decisions/001-substrate-posture-and-ops-placement.md) | Substrate posture & ops placement | store stays substrate-free; ops ride alkcall feature-gated | | [002](decisions/002-canonical-hash-and-key-encoding.md) | Canonical hash + key encoding | git-blob-sha-256 canonical; algorithm in-key; one-way door | | [003](decisions/003-backend-contract-and-dispatch.md) | Backend contract & dispatch | two backends, size-threshold routing, no migration | | [004](decisions/004-two-backends-no-third.md) | Two backends, no third | manifest layers are consumer-side | | [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Pooled CAS & GC | flat pool, registered liveness, delete windows | | [006](decisions/006-verification-posture-and-transfer-encoding.md) | Verification posture | whole-blob verification; chunk trees excluded by scoping | ## Open Questions Tracked in [open-questions.md](open-questions.md). Ones affecting the crate level: - **OQ-07**: how alkgit's object-storage seam consumes alkblobs (externally-owned: alkgit's architecture process) - **OQ-08**: alkfs requirement intake (externally-owned: alkfs Phase 0) - **OQ-09**: default namespace visibility in network ops — resolved closed-by-default (deny-unlisted; reopen only on a named open-read requirement) ## References - `docs/research/phase-0.md` — the converged Phase 0 record; inputs to every ADR above - `docs/research/poc-trait-dispatch-findings.md`, `docs/research/poc-largeblob-findings.md`, `docs/research/iroh-blobs-eval.md` — the evidence trail - `docs/sdd_process.md` — the process governing this tree - `/workspace/@alkdev/alkgit/docs/architecture/backend.md` — the reviewed consumer-side seam that OQ-07 maps onto alkblobs - AGENTS.md — family conventions (conventions 4–8 gate the decisions)