alkblobs/docs/plans/implementation.md

status, last_updated

status	last_updated
draft	2026-10-12 (wave 4 decomposed into tasks/)

alkblobs — Implementation plan

Wave-based decomposition of the architecture (docs/architecture/) into units of implementable work. This is a deliberate deviation from the SDD process's decompose-everything-upfront step, adopting the sibling pattern proven on alkstore: the architecture is large, early waves change the shape of later ones (the seam shapes wave 1 pins are what every engine task implements against; the conformance suite's factory contract feeds every engine's verification work), and decomposing only the next wave at a time keeps each session's task set reviewable and lets later decompositions absorb earlier waves' course corrections.

Rhythm: decompose a wave → implement it → review gate → decompose the next wave. Task files live in tasks/ (frontmatter carries the categorical estimates). Wave boundaries are also review boundaries.

Scope authority: v0.1.0 ships the full surface — all five engines (kv: sqlite, postgres, mem; large: local, pg-lo), fleet GC, and the feature-gated ops module — per operator decision (2026-10-11): all downstream consumers need all of it, and the base crate must be fully usable and composable across mem/sqlite/postgres from the default build onward.

The waves

Dependency logic in one line: core contract → (mem ∥ local) → sqlite → postgres → pg-lo → fleet GC → ops → fuzzing → release readiness. The two dependency-free engines (mem, local) prove the entire facade + dispatch + in-process GC story before any SQL driver exists; sqlite and postgres are structurally independent of each other (both need only wave 2's trait + suite) and could run in either order — postgres is placed first of the two SQL engines because its wave establishes the full engine-state seam clause families that pg-lo (window family) and the fleet wave (the host) build on. pg-lo shares postgres's driver cluster and seam pattern. The fleet wave needs both SQL tiers landed: its canonical testable shape is kv=postgres + large=pg-lo on one instance (the composite fleet-validity predicate's joint entry+pin tx is expressible exactly there). The ops wave rides alkcall (published, 0.8.1) behind the default-off feature. Fuzzing follows the ops wave so its targets cover the key encoding and the ops payload surfaces; release readiness is last.

Wave	Contents	Depends on	Status
1	Crate/feature scaffold; hashing & keys (the one-way door); error model; Backend trait + I/O seams + engine-state seam shape; facade value types; conformance-suite convention	—	decomposed (2026-10-11)
2	mem + local engines; facade put/read paths + dispatch + constructor modes; in-process GC protocol; suite's first columns	wave 1	decomposed (2026-10-11)
3	sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column	wave 2	decomposed (2026-10-11)
4	postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column	wave 2	decomposed (2026-10-12)
5	pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column	wave 4	pending
6	Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations	waves 4 + 5	pending
7	ops feature: alkcall-backed op family, ACL mapping, pin tokens + renewal, fanout seam	wave 6	pending
8	Fuzzing: family pattern targets (key encoding, ops payloads), corpus-replay gate	wave 7	pending
9	Crate docs, feature-graph verification, README, publish prep	wave 8	pending

Wave 1 — Foundations

The contract surface in code, engine-blind: everything later waves implement against, and nothing that needs an engine to test.

• Crate/feature scaffold — the ADR-013 §4 feature graph declared in the manifest from the first commit: sqlite default-on (rusqlite), postgres (tokio-postgres + deadpool-postgres), mem (dependency-free), pg-lo (the shared pg cluster; does not imply postgres — ADR-010 §1's consequence), ops (alkcall, default-off); local ungated. Engine modules ride #[cfg(feature)] from the start.
• Hashing & keys (ADR-002 — the one-way door): Digest/Key types, the canonical git-blob-sha-256 derivation (H("blob <len>\0" + content)), tolerated git-blob-sha-1, the algorithm-tagged byte encoding (1 alg byte + digest; 0x01/0x02 assigned, 0x00 reserved), rejects-never-guesses validation, and the byte-exact git hash-object interop test (POC #1 findings 4/5's standard — validated against the git CLI, not just vectors).
• Error model (store-api.md; ADR-012 §6.5): the one-enum error family with its altitude discipline — the trait's effective members (Missing/Verification/Io/KeyInvalid) and the facade's GcRefuse addition.
• Backend trait + I/O seams (ADR-003/010): the trait method set (has/get/put/delete/list/name/size) with opaque byte keys; the read-cursor abstraction (kv materializing vs large streaming bodies; serves read_range's slice + slice-digest contract at one seam); the staged-put contract as a named per-engine shape (staging cursor handout, commit publishes, every failure path discards); and the crate-internal engine-state companion trait with its clause families and per-engine applicability (ADR-010 §2 as reduced by ADR-012 §5) — SQL engines implement it in waves 3–5; local/mem are excluded by construction.
• Facade value types: EntryMeta, Pin/Batch (types + pinned lifecycle contract; the arbitration machinery is wave 2), SweepReport/GcAbortCause (abort-as-data, exactly three causes), PinState.
• Conformance-suite convention — tests/conformance/ established with the row/stamp convention (ADR-§-stamped rows, the alkstore ADR-017 pattern adapted) and the factory contract (isolation + idempotent teardown); the invariant backlog from store-api.md registered as rows-to-grow. The compile-able harness lands with wave 2's first column (mem — the contract-reference engine).

The wave-1 review gate is the cheap-place gate: the key encoding is the declared one-way door (AGENTS convention 5) — a shape error found at this gate costs a task revision; found after an engine ships data it costs a migration ADR. The gate reads the encoding, the trait, and the seam shapes against the pinned ADR text line by line.

Wave 2 — The store works dependency-free (mem + local)

The first engines and the facade behavior, chosen so the whole store story is provable with zero driver dependencies:

• mem kv engine (feature mem) — the contract-reference engine (ADR-011 §2): a BTreeMap-shaped implementation of the kv tier's contract including size; every other engine's conformance rows mirror its column.
• local large engine (ungated) — the ADR-003 backend as specced: flat {hex-prefix}/{hex-prefix}/{hash} sharding, stage-then- commit-rename (the staged-put contract realized with its exact-count hygiene tests, POC #3 A5's shape), pread-loop read cursor, pread range reads.
• Facade behavior — the put seam's two internal paths behind one signature (known-length one-pass; unknown-length pre-threshold buffering with the mid-stream stage flush and the restart-over- staged-bytes derivation pass, ADR-003's option (d) semantics pinned in three numbered rules); get/stat/read_range/has/delete/ list with the dispatch fall-through general rule (ADR-012 §6.1); size-threshold dispatch (pure function of length, default 128 KiB, constructor-tunable); constructor modes (dual-tier / kv-only with put-time rejection per ADR-012 §6.2 / mem-only) — kv=mem + large=local is the dispatch-coverage test shape ADR-012 §6.3 names.
• In-process GC protocol (ADR-005) — pins with pin-before-publish ordering, registered liveness sources (live-shared), the protect callback with abort-as-data, delete windows (mark → arbitrate-at- delete-time → batched commit), sweep() returning Ok(SweepReport) always, direct-delete refusal (GcRefuse, the full protection set per ADR-012 §6.4), local/mem staging window candidates in-process and executing them inline in the same sweep (ADR-012 §5).
• Suite columns — the harness compile-able and green on the mem and local columns: round-trips, dedup, stage hygiene, sweep-safety exact counts, virgin-store no-ops, stat-without-content.

Wave 2 decomposes after the wave-1 review gate, absorbing whatever the gate corrects.

Wave 3 — sqlite engine

The default kv engine (feature sqlite, default-on; REQ-1): WAL + synchronous=NORMAL as the shipped durability tier, prepared statements, has as an EXISTS probe, the indexed size column probe (POC #5 B5's row shape), virgin-store no-tables-yet tolerance, and the engine-state seam's window family only (ADR-012 §5's reduction — durable single-node window staging; no pin rows, no sweep-lock body). Suite column green server-less. The single-writer ceiling is a documented posture, not a defect (REQ-1's economics).

Wave 4 — postgres kv engine

The fleet-capable kv engine (feature postgres; REQ-2/REQ-3): schema bootstrap, pooled connections (tokio-postgres + deadpool-postgres) with per-connection prepared-statement discipline, ON CONFLICT DO NOTHING CAS rows, complete list() as a stable cursor over a vacuuming table, batch-delete via key = ANY($1), the synchronous_commit parity knob, and the engine-state seam's full clause families — postgres is the fleet host (ADR-012 §4): ensure_gc_schema (pins + window), pin_commit, pin_renew, pin_check/arbitrate, window_stage/window_take, sweep_lock_acquire/release (the pg advisory lock, the sweep lease's shipped provider). Suite column green against dockerized postgres (the alkstore harness convention: connection settings ride environment, tests skip cleanly server-less, acceptance requires green against the server). This wave establishes the seam implementation pattern pg-lo reuses.

Wave 5 — pg-lo large engine

The large tier's second engine (feature pg-lo; ADR-009, admitted on POC #7): Large Objects as storage — staged put as chunked lowrite statements (512 KiB per the POC's measured knee) inside a tx the store core holds, descriptorless lo_get(oid, off, len) window reads over the companion table (the C4 posture), companion-row size probes, transactional LO creation (zero crash orphans), and the seam's window family (durable window staging where the kv engine is not SQL-backed — the one-window-host rule). Suite column green against dockerized postgres. Named deltas documented (catalog space never returned; VACUUM FULL is the operator's shrink path).

Wave 6 — Fleet GC

The mechanism wave (ADR-008/010/012/013; REQ-2): DB-backed pins as expiry-carrying rows committed atomically with entries — the store core holds the joint entry+pin tx and engines contribute mechanics (pin_commit); the named exclusive sweep lease (one sweeper per pool, atomic test-and-set, holdership ends at release/owner death/session loss; contention is the SweeperLock report abort); SQL-level delete arbitration (SELECT … FOR UPDATE finger-lock at delete time); TTL/3 pin renewal on the guard; the composite fleet-validity predicate and the explicit fleet: true constructor declaration (never inferred); kv-only re-routing posture; the one-window-host rule under dual-SQL instances. Exact-count fleet sweep outcomes against live postgres — the wave's gate is the architecture's most intricate state machine proven under concurrency.

Wave 7 — Ops surface

The ops feature (ADR-001; alkcall 0.8.1): the op family — blobs/stat, blobs/have (with the token-renewal form), blobs/delete (internal authority), blobs/fetch (Sub), blobs/put (Sink, response {token, digest}) — JSON control plane via alkcall OperationSpec + JSON Schema validation, binary data plane via channel_open-allocated Sub/Sink channels; namespace-as-resource ACL mapping (closed-by-default per OQ-09); the pin hand-over + token renewal contract (the token is the pin; TTL/3 renewal via have/re-put; validity domain = minting node); the fetch fanout seam (one reader, store arm + subscriber arms; late joiners degrade to verified get). The module is a store-facade consumer with no backend side-door.

Wave 8 — Fuzzing

The family pattern (alkcall/alktunnels/alksocks/alktty/alkstore), per operator standing decision that anything worth fuzzing gets fuzzed: the fuzz/ sidecar workspace (nightly pinned inside fuzz/ only, the stable fuzz/shared corpus-replay gate, the detached runner), targets over the key-encoding parser (Key::from_bytes — the one-way door's reject-never-guess discipline), the ops payload surfaces (JSON specs + binary channel framing, if wave 7's shapes warrant it), and any put-path staging logic the surface review names. Committed crash-input seeds; grown corpora gitignored; the corpus-replay gate joins the verification list when the wave lands. Targets are chosen at decomposition time against the then-current surface — the wave exists in the plan as a commitment, not yet a target inventory.

Wave 9 — Release readiness

Crate docs (public-API rustdoc warning-free; the facade's docs are contract text), the feature-graph verification pass (every coherent engine-feature combination compiles and passes its gates — pg-pure builds, pg-lo without postgres, --all-features), the root README (written last, honestly), publish prep (package metadata, cargo package verification per the exclude list, path-dep/version bounds), and the pre-release review gate flipping the release-facing specs to stable.

Decided points

• Wave-based decomposition (operator-endorsed, 2026-10-11): the SDD decompose-everything-upfront step is replaced by the alkstore-proven rhythm — decompose the next wave after the previous wave's review gate; later decompositions absorb earlier outcomes.
• v0.1.0 scope — full surface (operator, 2026-10-11): all five engines, fleet GC, and the ops module ship in the first release; the base crate is fully usable and composable across mem/sqlite/postgres by default.
• Engine order: mem + local first (the contract-reference engine plus the ungated engine — the dependency-free pair proves facade, dispatch, and in-process GC end-to-end, and kv=mem + large=local is the ADR-012 §6.3 dispatch-coverage shape); then sqlite (default), postgres (establishes the full seam), pg-lo (shares the pg cluster
  • seam pattern), fleet GC last of the storage waves (its canonical testable shape needs both SQL tiers).
• Contract-suite home: in-crate tests/conformance/ (operator, 2026-10-11) — engines are features of one crate, not sibling crates, so a separate suite crate buys nothing; the harness is factory-parameterized over engine constructors, rows ADR-stamped (the alkstore ADR-017 convention adapted), mem the reference column.
• Fuzzing: adopted family-wide as wave 8, after ops and before release (operator standing rule: fuzz anything worth fuzzing; it almost always pays for itself). Target inventory is fixed at that wave's decomposition against the matured surface.
• Single crate, engine features (ADR-013 §4): no engine-crate workspace split; the feature graph is declared in the manifest from wave 1; feature-gated engines' gates run under --all-features.
• CI: none, deliberately (family posture): CI and publishing run manually; the merge gates (cargo test, cargo clippy --all-targets -- -D warnings, cargo fmt --check, plus --all-features variants once features carry code) are run coordinator-side.

Review gates

Each wave ends in a review task (review-wave-N) before the next wave decomposes. Specific gates:

• Wave 1 review — the one-way-door gate: the key encoding read against ADR-002 line by line (a shape error here is a task revision; after engines ship data it is a migration ADR); the trait + seam shapes against ADR-003/010; the error model's altitude split against store-api.md/ADR-012 §6.5; the feature graph against ADR-013 §4; the named design points (single-enum error reading, Backend visibility, trait put-form shapes, oid hex policy, async-trait shape) audited as resolved.
• Wave 2 review — facade + dispatch + GC vs ADR-003/005/012 (the three numbered unknown-length rules, the delete-window ordering invariants, abort-as-data); stage-hygiene exact-count tests; the suite's first columns green on mem + local; the dispatch-coverage shape exercised.
• Engine wave reviews (3/4/5) — engine-vs-contract conformance: the suite column green (server-less for sqlite; vs dockerized postgres for the pg engines), the engine-state clause families contract-tested per applicability, virgin-store semantics, size metadata-only discipline.
• Wave 6 review — the fleet gate: exact-count sweep outcomes under concurrency vs live postgres, the joint entry+pin tx atomicity, the sweep lease semantics vs ADR-013 §2, the composite fleet-validity predicate enforced at the constructor, kv-only put-time rejection.
• Wave 7 review — ops-vs-alkcall conformance (the machinery ridden, not re-invented); ACL mapping vs OQ-09's closed-by-default; the pin token contract vs ADR-010 §4/ADR-012 §2; no backend side-door.
• Wave 8 review — campaigns run clean (or findings fixed + promoted to seeds), the corpus-replay gate green on stable, seeds committed per the corpus policy.
• Wave 9 review — the pre-release gate: docs honesty audit, the feature-graph verification re-run, all gates green (including the fuzz replay), release-facing specs flipped stable.