--- status: draft last_updated: "2026-10-12 (wave 4 decomposed into tasks/)" --- # alkblobs — Implementation plan Wave-based decomposition of the architecture (`docs/architecture/`) into units of implementable work. This is a deliberate deviation from the SDD process's decompose-everything-upfront step, adopting the sibling pattern proven on alkstore: the architecture is large, early waves change the shape of later ones (the seam shapes wave 1 pins are what every engine task implements against; the conformance suite's factory contract feeds every engine's verification work), and decomposing only the next wave at a time keeps each session's task set reviewable and lets later decompositions absorb earlier waves' course corrections. Rhythm: decompose a wave → implement it → review gate → decompose the next wave. Task files live in `tasks/` (frontmatter carries the categorical estimates). Wave boundaries are also review boundaries. Scope authority: **v0.1.0 ships the full surface** — all five engines (kv: sqlite, postgres, mem; large: local, pg-lo), fleet GC, and the feature-gated `ops` module — per operator decision (2026-10-11): all downstream consumers need all of it, and the base crate must be fully usable and composable across mem/sqlite/postgres from the default build onward. ## The waves Dependency logic in one line: core contract → (mem ∥ local) → sqlite → postgres → pg-lo → fleet GC → ops → fuzzing → release readiness. The two dependency-free engines (mem, local) prove the entire facade + dispatch + in-process GC story before any SQL driver exists; sqlite and postgres are structurally independent of each other (both need only wave 2's trait + suite) and could run in either order — postgres is placed first of the two SQL engines because its wave establishes the full engine-state seam clause families that pg-lo (window family) and the fleet wave (the host) build on. pg-lo shares postgres's driver cluster and seam pattern. The fleet wave needs both SQL tiers landed: its canonical testable shape is kv=postgres + large=pg-lo on one instance (the composite fleet-validity predicate's joint entry+pin tx is expressible exactly there). The ops wave rides alkcall (published, 0.8.1) behind the default-off feature. Fuzzing follows the ops wave so its targets cover the key encoding *and* the ops payload surfaces; release readiness is last. | Wave | Contents | Depends on | Status | |---|---|---|---| | [1](#wave-1--foundations) | Crate/feature scaffold; hashing & keys (the one-way door); error model; Backend trait + I/O seams + engine-state seam shape; facade value types; conformance-suite convention | — | decomposed (2026-10-11) | | [2](#wave-2--the-store-works-dependency-free-mem--local) | mem + local engines; facade put/read paths + dispatch + constructor modes; in-process GC protocol; suite's first columns | wave 1 | decomposed (2026-10-11) | | [3](#wave-3--sqlite-engine) | sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column | wave 2 | decomposed (2026-10-11) | | [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | decomposed (2026-10-12) | | [5](#wave-5--pg-lo-large-engine) | pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column | wave 4 | pending | | [6](#wave-6--fleet-gc) | Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations | waves 4 + 5 | pending | | [7](#wave-7--ops-surface) | `ops` feature: alkcall-backed op family, ACL mapping, pin tokens + renewal, fanout seam | wave 6 | pending | | [8](#wave-8--fuzzing) | Fuzzing: family pattern targets (key encoding, ops payloads), corpus-replay gate | wave 7 | pending | | [9](#wave-9--release-readiness) | Crate docs, feature-graph verification, README, publish prep | wave 8 | pending | ## Wave 1 — Foundations The contract surface in code, engine-blind: everything later waves implement *against*, and nothing that needs an engine to test. - **Crate/feature scaffold** — the ADR-013 §4 feature graph declared in the manifest from the first commit: `sqlite` default-on (rusqlite), `postgres` (tokio-postgres + deadpool-postgres), `mem` (dependency-free), `pg-lo` (the shared pg cluster; does **not** imply `postgres` — ADR-010 §1's consequence), `ops` (alkcall, default-off); `local` ungated. Engine modules ride `#[cfg(feature)]` from the start. - **Hashing & keys** (ADR-002 — the one-way door): `Digest`/`Key` types, the canonical `git-blob-sha-256` derivation (`H("blob \0" + content)`), tolerated `git-blob-sha-1`, the algorithm-tagged byte encoding (1 alg byte + digest; 0x01/0x02 assigned, 0x00 reserved), rejects-never-guesses validation, and the byte-exact `git hash-object` interop test (POC #1 findings 4/5's standard — validated against the git CLI, not just vectors). - **Error model** (store-api.md; ADR-012 §6.5): the one-enum error family with its altitude discipline — the trait's effective members (`Missing`/`Verification`/`Io`/`KeyInvalid`) and the facade's `GcRefuse` addition. - **Backend trait + I/O seams** (ADR-003/010): the trait method set (`has`/`get`/`put`/`delete`/`list`/`name`/`size`) with opaque byte keys; the read-cursor abstraction (kv materializing vs large streaming bodies; serves `read_range`'s slice + slice-digest contract at one seam); the staged-put contract as a named per-engine shape (staging cursor handout, commit publishes, every failure path discards); and the crate-internal engine-state companion trait with its clause families and per-engine applicability (ADR-010 §2 as reduced by ADR-012 §5) — SQL engines implement it in waves 3–5; `local`/`mem` are excluded by construction. - **Facade value types**: `EntryMeta`, `Pin`/`Batch` (types + pinned lifecycle contract; the arbitration machinery is wave 2), `SweepReport`/`GcAbortCause` (abort-as-data, exactly three causes), `PinState`. - **Conformance-suite convention** — `tests/conformance/` established with the row/stamp convention (ADR-§-stamped rows, the alkstore ADR-017 pattern adapted) and the factory contract (isolation + idempotent teardown); the invariant backlog from store-api.md registered as rows-to-grow. The compile-able harness lands with wave 2's first column (mem — the contract-reference engine). The wave-1 review gate is the cheap-place gate: the key encoding is the declared one-way door (AGENTS convention 5) — a shape error found at this gate costs a task revision; found after an engine ships data it costs a migration ADR. The gate reads the encoding, the trait, and the seam shapes against the pinned ADR text line by line. ## Wave 2 — The store works dependency-free (mem + local) The first engines and the facade behavior, chosen so the whole store story is provable with zero driver dependencies: - **mem kv engine** (feature `mem`) — the contract-reference engine (ADR-011 §2): a `BTreeMap`-shaped implementation of the kv tier's contract including `size`; every other engine's conformance rows mirror its column. - **local large engine** (ungated) — the ADR-003 backend as specced: flat `{hex-prefix}/{hex-prefix}/{hash}` sharding, stage-then- commit-rename (the staged-put contract realized with its exact-count hygiene tests, POC #3 A5's shape), pread-loop read cursor, pread range reads. - **Facade behavior** — the put seam's two internal paths behind one signature (known-length one-pass; unknown-length pre-threshold buffering with the mid-stream stage flush and the restart-over- staged-bytes derivation pass, ADR-003's option (d) semantics pinned in three numbered rules); `get`/`stat`/`read_range`/`has`/`delete`/ `list` with the dispatch fall-through general rule (ADR-012 §6.1); size-threshold dispatch (pure function of length, default 128 KiB, constructor-tunable); constructor modes (dual-tier / kv-only with put-time rejection per ADR-012 §6.2 / mem-only) — `kv=mem + large=local` is the dispatch-coverage test shape ADR-012 §6.3 names. - **In-process GC protocol** (ADR-005) — pins with pin-before-publish ordering, registered liveness sources (live-shared), the protect callback with abort-as-data, delete windows (mark → arbitrate-at- delete-time → batched commit), `sweep()` returning `Ok(SweepReport)` always, direct-delete refusal (`GcRefuse`, the full protection set per ADR-012 §6.4), `local`/`mem` staging window candidates in-process and executing them inline in the same sweep (ADR-012 §5). - **Suite columns** — the harness compile-able and green on the mem and local columns: round-trips, dedup, stage hygiene, sweep-safety exact counts, virgin-store no-ops, `stat`-without-content. Wave 2 decomposes after the wave-1 review gate, absorbing whatever the gate corrects. ## Wave 3 — sqlite engine The default kv engine (feature `sqlite`, default-on; REQ-1): WAL + `synchronous=NORMAL` as the shipped durability tier, prepared statements, `has` as an EXISTS probe, the indexed `size` column probe (POC #5 B5's row shape), virgin-store no-tables-yet tolerance, and the engine-state seam's **window family only** (ADR-012 §5's reduction — durable single-node window staging; no pin rows, no sweep-lock body). Suite column green server-less. The single-writer ceiling is a documented posture, not a defect (REQ-1's economics). ## Wave 4 — postgres kv engine The fleet-capable kv engine (feature `postgres`; REQ-2/REQ-3): schema bootstrap, pooled connections (tokio-postgres + deadpool-postgres) with per-connection prepared-statement discipline, `ON CONFLICT DO NOTHING` CAS rows, complete `list()` as a stable cursor over a vacuuming table, batch-delete via `key = ANY($1)`, the `synchronous_commit` parity knob, and the engine-state seam's **full clause families** — postgres is the fleet host (ADR-012 §4): `ensure_gc_schema` (pins + window), `pin_commit`, `pin_renew`, `pin_check`/`arbitrate`, `window_stage`/`window_take`, `sweep_lock_acquire`/`release` (the pg advisory lock, the sweep lease's shipped provider). Suite column green against dockerized postgres (the alkstore harness convention: connection settings ride environment, tests skip cleanly server-less, acceptance requires green against the server). This wave establishes the seam implementation pattern pg-lo reuses. ## Wave 5 — pg-lo large engine The large tier's second engine (feature `pg-lo`; ADR-009, admitted on POC #7): Large Objects as storage — staged put as chunked `lowrite` statements (512 KiB per the POC's measured knee) inside a tx the store core holds, descriptorless `lo_get(oid, off, len)` window reads over the companion table (the C4 posture), companion-row `size` probes, transactional LO creation (zero crash orphans), and the seam's window family (durable window staging where the kv engine is not SQL-backed — the one-window-host rule). Suite column green against dockerized postgres. Named deltas documented (catalog space never returned; `VACUUM FULL` is the operator's shrink path). ## Wave 6 — Fleet GC The mechanism wave (ADR-008/010/012/013; REQ-2): DB-backed pins as expiry-carrying rows committed atomically with entries — the store core holds the joint entry+pin tx and engines contribute mechanics (`pin_commit`); the named exclusive sweep lease (one sweeper per pool, atomic test-and-set, holdership ends at release/owner death/session loss; contention is the `SweeperLock` report abort); SQL-level delete arbitration (`SELECT … FOR UPDATE` finger-lock at delete time); TTL/3 pin renewal on the guard; the composite fleet-validity predicate and the explicit `fleet: true` constructor declaration (never inferred); kv-only re-routing posture; the one-window-host rule under dual-SQL instances. Exact-count fleet sweep outcomes against live postgres — the wave's gate is the architecture's most intricate state machine proven under concurrency. ## Wave 7 — Ops surface The `ops` feature (ADR-001; alkcall 0.8.1): the op family — `blobs/stat`, `blobs/have` (with the token-renewal form), `blobs/delete` (internal authority), `blobs/fetch` (Sub), `blobs/put` (Sink, response `{token, digest}`) — JSON control plane via alkcall `OperationSpec` + JSON Schema validation, binary data plane via `channel_open`-allocated Sub/Sink channels; namespace-as-resource ACL mapping (closed-by-default per OQ-09); the pin hand-over + token renewal contract (the token is the pin; TTL/3 renewal via have/re-put; validity domain = minting node); the fetch fanout seam (one reader, store arm + subscriber arms; late joiners degrade to verified `get`). The module is a store-facade consumer with no backend side-door. ## Wave 8 — Fuzzing The family pattern (alkcall/alktunnels/alksocks/alktty/alkstore), per operator standing decision that anything worth fuzzing gets fuzzed: the `fuzz/` sidecar workspace (nightly pinned inside `fuzz/` only, the stable `fuzz/shared` corpus-replay gate, the detached runner), targets over the key-encoding parser (`Key::from_bytes` — the one-way door's reject-never-guess discipline), the ops payload surfaces (JSON specs + binary channel framing, if wave 7's shapes warrant it), and any put-path staging logic the surface review names. Committed crash-input seeds; grown corpora gitignored; the corpus-replay gate joins the verification list when the wave lands. Targets are chosen at decomposition time against the then-current surface — the wave exists in the plan as a commitment, not yet a target inventory. ## Wave 9 — Release readiness Crate docs (public-API rustdoc warning-free; the facade's docs are contract text), the feature-graph verification pass (every coherent engine-feature combination compiles and passes its gates — pg-pure builds, `pg-lo` without `postgres`, `--all-features`), the root README (written last, honestly), publish prep (package metadata, `cargo package` verification per the exclude list, path-dep/version bounds), and the pre-release review gate flipping the release-facing specs to `stable`. ## Decided points - **Wave-based decomposition** (operator-endorsed, 2026-10-11): the SDD decompose-everything-upfront step is replaced by the alkstore-proven rhythm — decompose the next wave after the previous wave's review gate; later decompositions absorb earlier outcomes. - **v0.1.0 scope — full surface** (operator, 2026-10-11): all five engines, fleet GC, and the `ops` module ship in the first release; the base crate is fully usable and composable across mem/sqlite/postgres by default. - **Engine order**: mem + local first (the contract-reference engine plus the ungated engine — the dependency-free pair proves facade, dispatch, and in-process GC end-to-end, and `kv=mem + large=local` is the ADR-012 §6.3 dispatch-coverage shape); then sqlite (default), postgres (establishes the full seam), pg-lo (shares the pg cluster + seam pattern), fleet GC last of the storage waves (its canonical testable shape needs both SQL tiers). - **Contract-suite home**: in-crate `tests/conformance/` (operator, 2026-10-11) — engines are features of one crate, not sibling crates, so a separate suite crate buys nothing; the harness is factory-parameterized over engine constructors, rows ADR-stamped (the alkstore ADR-017 convention adapted), mem the reference column. - **Fuzzing**: adopted family-wide as wave 8, after ops and before release (operator standing rule: fuzz anything worth fuzzing; it almost always pays for itself). Target inventory is fixed at that wave's decomposition against the matured surface. - **Single crate, engine features** (ADR-013 §4): no engine-crate workspace split; the feature graph is declared in the manifest from wave 1; feature-gated engines' gates run under `--all-features`. - **CI**: none, deliberately (family posture): CI and publishing run manually; the merge gates (`cargo test`, `cargo clippy --all-targets -- -D warnings`, `cargo fmt --check`, plus `--all-features` variants once features carry code) are run coordinator-side. ## Review gates Each wave ends in a review task (`review-wave-N`) before the next wave decomposes. Specific gates: - **Wave 1 review** — the one-way-door gate: the key encoding read against ADR-002 line by line (a shape error here is a task revision; after engines ship data it is a migration ADR); the trait + seam shapes against ADR-003/010; the error model's altitude split against store-api.md/ADR-012 §6.5; the feature graph against ADR-013 §4; the named design points (single-enum error reading, Backend visibility, trait put-form shapes, oid hex policy, async-trait shape) audited as resolved. - **Wave 2 review** — facade + dispatch + GC vs ADR-003/005/012 (the three numbered unknown-length rules, the delete-window ordering invariants, abort-as-data); stage-hygiene exact-count tests; the suite's first columns green on mem + local; the dispatch-coverage shape exercised. - **Engine wave reviews (3/4/5)** — engine-vs-contract conformance: the suite column green (server-less for sqlite; vs dockerized postgres for the pg engines), the engine-state clause families contract-tested per applicability, virgin-store semantics, `size` metadata-only discipline. - **Wave 6 review** — the fleet gate: exact-count sweep outcomes under concurrency vs live postgres, the joint entry+pin tx atomicity, the sweep lease semantics vs ADR-013 §2, the composite fleet-validity predicate enforced at the constructor, kv-only put-time rejection. - **Wave 7 review** — ops-vs-alkcall conformance (the machinery ridden, not re-invented); ACL mapping vs OQ-09's closed-by-default; the pin token contract vs ADR-010 §4/ADR-012 §2; no backend side-door. - **Wave 8 review** — campaigns run clean (or findings fixed + promoted to seeds), the corpus-replay gate green on stable, seeds committed per the corpus policy. - **Wave 9 review** — the pre-release gate: docs honesty audit, the feature-graph verification re-run, all gates green (including the fuzz replay), release-facing specs flipped `stable`.