diff --git a/docs/plans/implementation.md b/docs/plans/implementation.md new file mode 100644 index 0000000..c5255f1 --- /dev/null +++ b/docs/plans/implementation.md @@ -0,0 +1,316 @@ +--- +status: draft +last_updated: "2026-10-11 (plan created; wave 1 decomposed into tasks/)" +--- + +# alkblobs — Implementation plan + +Wave-based decomposition of the architecture (`docs/architecture/`) +into units of implementable work. This is a deliberate deviation from +the SDD process's decompose-everything-upfront step, adopting the +sibling pattern proven on alkstore: the architecture is large, early +waves change the shape of later ones (the seam shapes wave 1 pins are +what every engine task implements against; the conformance suite's +factory contract feeds every engine's verification work), and +decomposing only the next wave at a time keeps each session's task set +reviewable and lets later decompositions absorb earlier waves' course +corrections. + +Rhythm: decompose a wave → implement it → review gate → decompose the +next wave. Task files live in `tasks/` (frontmatter carries the +categorical estimates). Wave boundaries are also review boundaries. + +Scope authority: **v0.1.0 ships the full surface** — all five engines +(kv: sqlite, postgres, mem; large: local, pg-lo), fleet GC, and the +feature-gated `ops` module — per operator decision (2026-10-11): all +downstream consumers need all of it, and the base crate must be fully +usable and composable across mem/sqlite/postgres from the default +build onward. + +## The waves + +Dependency logic in one line: core contract → (mem ∥ local) → sqlite +→ postgres → pg-lo → fleet GC → ops → fuzzing → release readiness. +The two dependency-free engines (mem, local) prove the entire facade + +dispatch + in-process GC story before any SQL driver exists; sqlite +and postgres are structurally independent of each other (both need +only wave 2's trait + suite) and could run in either order — postgres +is placed first of the two SQL engines because its wave establishes +the full engine-state seam clause families that pg-lo (window family) +and the fleet wave (the host) build on. pg-lo shares postgres's +driver cluster and seam pattern. The fleet wave needs both SQL tiers +landed: its canonical testable shape is kv=postgres + large=pg-lo on +one instance (the composite fleet-validity predicate's joint +entry+pin tx is expressible exactly there). The ops wave rides +alkcall (published, 0.8.1) behind the default-off feature. Fuzzing +follows the ops wave so its targets cover the key encoding *and* the +ops payload surfaces; release readiness is last. + +| Wave | Contents | Depends on | Status | +|---|---|---|---| +| [1](#wave-1--foundations) | Crate/feature scaffold; hashing & keys (the one-way door); error model; Backend trait + I/O seams + engine-state seam shape; facade value types; conformance-suite convention | — | decomposed (2026-10-11) | +| [2](#wave-2--the-store-works-dependency-free-mem--local) | mem + local engines; facade put/read paths + dispatch + constructor modes; in-process GC protocol; suite's first columns | wave 1 | pending | +| [3](#wave-3--sqlite-engine) | sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column | wave 2 | pending | +| [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | pending | +| [5](#wave-5--pg-lo-large-engine) | pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column | wave 4 | pending | +| [6](#wave-6--fleet-gc) | Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations | waves 4 + 5 | pending | +| [7](#wave-7--ops-surface) | `ops` feature: alkcall-backed op family, ACL mapping, pin tokens + renewal, fanout seam | wave 6 | pending | +| [8](#wave-8--fuzzing) | Fuzzing: family pattern targets (key encoding, ops payloads), corpus-replay gate | wave 7 | pending | +| [9](#wave-9--release-readiness) | Crate docs, feature-graph verification, README, publish prep | wave 8 | pending | + +## Wave 1 — Foundations + +The contract surface in code, engine-blind: everything later waves +implement *against*, and nothing that needs an engine to test. + +- **Crate/feature scaffold** — the ADR-013 §4 feature graph declared + in the manifest from the first commit: `sqlite` default-on + (rusqlite), `postgres` (tokio-postgres + deadpool-postgres), `mem` + (dependency-free), `pg-lo` (the shared pg cluster; does **not** + imply `postgres` — ADR-010 §1's consequence), `ops` (alkcall, + default-off); `local` ungated. Engine modules ride `#[cfg(feature)]` + from the start. +- **Hashing & keys** (ADR-002 — the one-way door): `Digest`/`Key` + types, the canonical `git-blob-sha-256` derivation + (`H("blob \0" + content)`), tolerated `git-blob-sha-1`, the + algorithm-tagged byte encoding (1 alg byte + digest; 0x01/0x02 + assigned, 0x00 reserved), rejects-never-guesses validation, and the + byte-exact `git hash-object` interop test (POC #1 findings 4/5's + standard — validated against the git CLI, not just vectors). +- **Error model** (store-api.md; ADR-012 §6.5): the one-enum error + family with its altitude discipline — the trait's effective members + (`Missing`/`Verification`/`Io`/`KeyInvalid`) and the facade's + `GcRefuse` addition. +- **Backend trait + I/O seams** (ADR-003/010): the trait method set + (`has`/`get`/`put`/`delete`/`list`/`name`/`size`) with opaque byte + keys; the read-cursor abstraction (kv materializing vs large + streaming bodies; serves `read_range`'s slice + slice-digest + contract at one seam); the staged-put contract as a named per-engine + shape (staging cursor handout, commit publishes, every failure path + discards); and the crate-internal engine-state companion trait with + its clause families and per-engine applicability (ADR-010 §2 as + reduced by ADR-012 §5) — SQL engines implement it in waves 3–5; + `local`/`mem` are excluded by construction. +- **Facade value types**: `EntryMeta`, `Pin`/`Batch` (types + pinned + lifecycle contract; the arbitration machinery is wave 2), + `SweepReport`/`GcAbortCause` (abort-as-data, exactly three causes), + `PinState`. +- **Conformance-suite convention** — `tests/conformance/` established + with the row/stamp convention (ADR-§-stamped rows, the alkstore + ADR-017 pattern adapted) and the factory contract (isolation + + idempotent teardown); the invariant backlog from store-api.md + registered as rows-to-grow. The compile-able harness lands with + wave 2's first column (mem — the contract-reference engine). + +The wave-1 review gate is the cheap-place gate: the key encoding is +the declared one-way door (AGENTS convention 5) — a shape error found +at this gate costs a task revision; found after an engine ships data +it costs a migration ADR. The gate reads the encoding, the trait, and +the seam shapes against the pinned ADR text line by line. + +## Wave 2 — The store works dependency-free (mem + local) + +The first engines and the facade behavior, chosen so the whole store +story is provable with zero driver dependencies: + +- **mem kv engine** (feature `mem`) — the contract-reference engine + (ADR-011 §2): a `BTreeMap`-shaped implementation of the kv tier's + contract including `size`; every other engine's conformance rows + mirror its column. +- **local large engine** (ungated) — the ADR-003 backend as specced: + flat `{hex-prefix}/{hex-prefix}/{hash}` sharding, stage-then- + commit-rename (the staged-put contract realized with its exact-count + hygiene tests, POC #3 A5's shape), pread-loop read cursor, pread + range reads. +- **Facade behavior** — the put seam's two internal paths behind one + signature (known-length one-pass; unknown-length pre-threshold + buffering with the mid-stream stage flush and the restart-over- + staged-bytes derivation pass, ADR-003's option (d) semantics pinned + in three numbered rules); `get`/`stat`/`read_range`/`has`/`delete`/ + `list` with the dispatch fall-through general rule (ADR-012 §6.1); + size-threshold dispatch (pure function of length, default 128 KiB, + constructor-tunable); constructor modes (dual-tier / kv-only with + put-time rejection per ADR-012 §6.2 / mem-only) — `kv=mem + + large=local` is the dispatch-coverage test shape ADR-012 §6.3 names. +- **In-process GC protocol** (ADR-005) — pins with pin-before-publish + ordering, registered liveness sources (live-shared), the protect + callback with abort-as-data, delete windows (mark → arbitrate-at- + delete-time → batched commit), `sweep()` returning `Ok(SweepReport)` + always, direct-delete refusal (`GcRefuse`, the full protection set + per ADR-012 §6.4), `local`/`mem` staging window candidates + in-process and executing them inline in the same sweep (ADR-012 §5). +- **Suite columns** — the harness compile-able and green on the mem + and local columns: round-trips, dedup, stage hygiene, sweep-safety + exact counts, virgin-store no-ops, `stat`-without-content. + +Wave 2 decomposes after the wave-1 review gate, absorbing whatever the +gate corrects. + +## Wave 3 — sqlite engine + +The default kv engine (feature `sqlite`, default-on; REQ-1): WAL + +`synchronous=NORMAL` as the shipped durability tier, prepared +statements, `has` as an EXISTS probe, the indexed `size` column probe +(POC #5 B5's row shape), virgin-store no-tables-yet tolerance, and the +engine-state seam's **window family only** (ADR-012 §5's reduction — +durable single-node window staging; no pin rows, no sweep-lock body). +Suite column green server-less. The single-writer ceiling is a +documented posture, not a defect (REQ-1's economics). + +## Wave 4 — postgres kv engine + +The fleet-capable kv engine (feature `postgres`; REQ-2/REQ-3): schema +bootstrap, pooled connections (tokio-postgres + deadpool-postgres) +with per-connection prepared-statement discipline, `ON CONFLICT DO +NOTHING` CAS rows, complete `list()` as a stable cursor over a +vacuuming table, batch-delete via `key = ANY($1)`, the +`synchronous_commit` parity knob, and the engine-state seam's **full +clause families** — postgres is the fleet host (ADR-012 §4): +`ensure_gc_schema` (pins + window), `pin_commit`, `pin_renew`, +`pin_check`/`arbitrate`, `window_stage`/`window_take`, +`sweep_lock_acquire`/`release` (the pg advisory lock, the sweep +lease's shipped provider). Suite column green against dockerized +postgres (the alkstore harness convention: connection settings ride +environment, tests skip cleanly server-less, acceptance requires green +against the server). This wave establishes the seam implementation +pattern pg-lo reuses. + +## Wave 5 — pg-lo large engine + +The large tier's second engine (feature `pg-lo`; ADR-009, admitted on +POC #7): Large Objects as storage — staged put as chunked `lowrite` +statements (512 KiB per the POC's measured knee) inside a tx the store +core holds, descriptorless `lo_get(oid, off, len)` window reads over +the companion table (the C4 posture), companion-row `size` probes, +transactional LO creation (zero crash orphans), and the seam's window +family (durable window staging where the kv engine is not SQL-backed — +the one-window-host rule). Suite column green against dockerized +postgres. Named deltas documented (catalog space never returned; +`VACUUM FULL` is the operator's shrink path). + +## Wave 6 — Fleet GC + +The mechanism wave (ADR-008/010/012/013; REQ-2): DB-backed pins as +expiry-carrying rows committed atomically with entries — the store +core holds the joint entry+pin tx and engines contribute mechanics +(`pin_commit`); the named exclusive sweep lease (one sweeper per pool, +atomic test-and-set, holdership ends at release/owner death/session +loss; contention is the `SweeperLock` report abort); SQL-level delete +arbitration (`SELECT … FOR UPDATE` finger-lock at delete time); TTL/3 +pin renewal on the guard; the composite fleet-validity predicate and +the explicit `fleet: true` constructor declaration (never inferred); +kv-only re-routing posture; the one-window-host rule under dual-SQL +instances. Exact-count fleet sweep outcomes against live postgres — +the wave's gate is the architecture's most intricate state machine +proven under concurrency. + +## Wave 7 — Ops surface + +The `ops` feature (ADR-001; alkcall 0.8.1): the op family — +`blobs/stat`, `blobs/have` (with the token-renewal form), +`blobs/delete` (internal authority), `blobs/fetch` (Sub), `blobs/put` +(Sink, response `{token, digest}`) — JSON control plane via alkcall +`OperationSpec` + JSON Schema validation, binary data plane via +`channel_open`-allocated Sub/Sink channels; namespace-as-resource ACL +mapping (closed-by-default per OQ-09); the pin hand-over + token +renewal contract (the token is the pin; TTL/3 renewal via have/re-put; +validity domain = minting node); the fetch fanout seam (one reader, +store arm + subscriber arms; late joiners degrade to verified `get`). +The module is a store-facade consumer with no backend side-door. + +## Wave 8 — Fuzzing + +The family pattern (alkcall/alktunnels/alksocks/alktty/alkstore), per +operator standing decision that anything worth fuzzing gets fuzzed: +the `fuzz/` sidecar workspace (nightly pinned inside `fuzz/` only, the +stable `fuzz/shared` corpus-replay gate, the detached runner), targets +over the key-encoding parser (`Key::from_bytes` — the one-way door's +reject-never-guess discipline), the ops payload surfaces (JSON specs + +binary channel framing, if wave 7's shapes warrant it), and any +put-path staging logic the surface review names. Committed crash-input +seeds; grown corpora gitignored; the corpus-replay gate joins the +verification list when the wave lands. Targets are chosen at +decomposition time against the then-current surface — the wave exists +in the plan as a commitment, not yet a target inventory. + +## Wave 9 — Release readiness + +Crate docs (public-API rustdoc warning-free; the facade's docs are +contract text), the feature-graph verification pass (every coherent +engine-feature combination compiles and passes its gates — pg-pure +builds, `pg-lo` without `postgres`, `--all-features`), the root README +(written last, honestly), publish prep (package metadata, `cargo +package` verification per the exclude list, path-dep/version bounds), +and the pre-release review gate flipping the release-facing specs to +`stable`. + +## Decided points + +- **Wave-based decomposition** (operator-endorsed, 2026-10-11): the + SDD decompose-everything-upfront step is replaced by the + alkstore-proven rhythm — decompose the next wave after the previous + wave's review gate; later decompositions absorb earlier outcomes. +- **v0.1.0 scope — full surface** (operator, 2026-10-11): all five + engines, fleet GC, and the `ops` module ship in the first release; + the base crate is fully usable and composable across + mem/sqlite/postgres by default. +- **Engine order**: mem + local first (the contract-reference engine + plus the ungated engine — the dependency-free pair proves facade, + dispatch, and in-process GC end-to-end, and `kv=mem + large=local` + is the ADR-012 §6.3 dispatch-coverage shape); then sqlite (default), + postgres (establishes the full seam), pg-lo (shares the pg cluster + + seam pattern), fleet GC last of the storage waves (its canonical + testable shape needs both SQL tiers). +- **Contract-suite home**: in-crate `tests/conformance/` (operator, + 2026-10-11) — engines are features of one crate, not sibling crates, + so a separate suite crate buys nothing; the harness is + factory-parameterized over engine constructors, rows ADR-stamped + (the alkstore ADR-017 convention adapted), mem the reference column. +- **Fuzzing**: adopted family-wide as wave 8, after ops and before + release (operator standing rule: fuzz anything worth fuzzing; it + almost always pays for itself). Target inventory is fixed at that + wave's decomposition against the matured surface. +- **Single crate, engine features** (ADR-013 §4): no engine-crate + workspace split; the feature graph is declared in the manifest from + wave 1; feature-gated engines' gates run under `--all-features`. +- **CI**: none, deliberately (family posture): CI and publishing run + manually; the merge gates (`cargo test`, `cargo clippy --all-targets + -- -D warnings`, `cargo fmt --check`, plus `--all-features` variants + once features carry code) are run coordinator-side. + +## Review gates + +Each wave ends in a review task (`review-wave-N`) before the next wave +decomposes. Specific gates: + +- **Wave 1 review** — the one-way-door gate: the key encoding read + against ADR-002 line by line (a shape error here is a task revision; + after engines ship data it is a migration ADR); the trait + seam + shapes against ADR-003/010; the error model's altitude split against + store-api.md/ADR-012 §6.5; the feature graph against ADR-013 §4; + the named design points (single-enum error reading, Backend + visibility, trait put-form shapes, oid hex policy, async-trait + shape) audited as resolved. +- **Wave 2 review** — facade + dispatch + GC vs ADR-003/005/012 (the + three numbered unknown-length rules, the delete-window ordering + invariants, abort-as-data); stage-hygiene exact-count tests; the + suite's first columns green on mem + local; the dispatch-coverage + shape exercised. +- **Engine wave reviews (3/4/5)** — engine-vs-contract conformance: + the suite column green (server-less for sqlite; vs dockerized + postgres for the pg engines), the engine-state clause families + contract-tested per applicability, virgin-store semantics, `size` + metadata-only discipline. +- **Wave 6 review** — the fleet gate: exact-count sweep outcomes under + concurrency vs live postgres, the joint entry+pin tx atomicity, the + sweep lease semantics vs ADR-013 §2, the composite fleet-validity + predicate enforced at the constructor, kv-only put-time rejection. +- **Wave 7 review** — ops-vs-alkcall conformance (the machinery ridden, + not re-invented); ACL mapping vs OQ-09's closed-by-default; the pin + token contract vs ADR-010 §4/ADR-012 §2; no backend side-door. +- **Wave 8 review** — campaigns run clean (or findings fixed + promoted + to seeds), the corpus-replay gate green on stable, seeds committed + per the corpus policy. +- **Wave 9 review** — the pre-release gate: docs honesty audit, the + feature-graph verification re-run, all gates green (including the + fuzz replay), release-facing specs flipped `stable`. \ No newline at end of file diff --git a/tasks/core-error-model.md b/tasks/core-error-model.md new file mode 100644 index 0000000..66b012c --- /dev/null +++ b/tasks/core-error-model.md @@ -0,0 +1,74 @@ +--- +id: core-error-model +name: Error model — the one-enum family with altitude discipline +status: pending +depends_on: [scaffold-crate-features] +scope: single +risk: low +impact: component +level: implementation +tags: [wave-1, errors] +--- + +## Description + +Implement the crate's error family (`src/error.rs`, re-exported from +`lib.rs`) per store-api.md's error model and ADR-012 §6.5: + +- One `Error` enum (thiserror) with exactly the five distinguished + variants: `Missing`, `Verification`, `Io(String)`, `GcRefuse`, + `KeyInvalid`. +- **The altitude discipline in doc comments** (store-api.md's "one + enum, both altitudes"): the `Backend` trait's methods return this + family but its *effective* members are + `Missing`/`Verification`/`Io`/`KeyInvalid` — `KeyInvalid` never + fires at the trait boundary (keys arrive opaque there) and + `GcRefuse` never fires below the facade (arbitration answers flow up + as data; the facade decides and owns the refusal). The facade's + public surface re-exposes the family with `GcRefuse` added. +- Semantics per variant: `Missing` — key absent on ops that signal + absence via `Err` (`read_range`, `delete`); `get`/`stat` signal + absence with `None` and never carry `Missing`. `Verification` — + put/get hash-check failure (content ≠ key). `Io(String)` — backend + media failure, stringly because `std::io::Error` is not stable + across versions. `GcRefuse` — direct delete refused (protected key: + pinned, registered-source-live, or protect-callback-named). + `KeyInvalid` — malformed key bytes at the boundary (truncated, + unknown algorithm — rejects, never guesses). +- **Named design point for the review gate**: store-api.md's wording + ("one enum, both altitudes… the trait's effective members… the + facade re-exposes them with `GcRefuse` added") is implemented as a + *single* enum with the altitude discipline carried by doc comments + and construction discipline (not two types). The gate audits this + reading against the spec text; if the gate reads it as requiring two + types, that is a cheap fix now. + +No panics; no `unwrap`/`expect` outside tests; `#[non_exhaustive]` is +**not** applied unless the spec requires it (the variant set is +ADR-pinned; new failure families are new ADRs — match the family's +posture on alkstore's error taxonomy). + +## Acceptance Criteria + +- [ ] `Error` enum with the five variants, thiserror-derived + Display/Error impls +- [ ] Doc comments on the enum and each variant stating the semantics + and the altitude discipline verbatim-traceable to store-api.md +- [ ] Unit tests: each variant constructs; `Missing` documented + get/stat exclusion stated in docs (behavioral enforcement lands + with the facade in wave 2) +- [ ] `cargo test`, `cargo clippy --all-targets -- -D warnings`, + `cargo fmt --check` green + +## References + +- docs/architecture/store-api.md (§Error model) +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §6.5, §1 + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/core-hashing-keys.md b/tasks/core-hashing-keys.md new file mode 100644 index 0000000..aeae969 --- /dev/null +++ b/tasks/core-hashing-keys.md @@ -0,0 +1,92 @@ +--- +id: core-hashing-keys +name: Hashing & keys — the canonical derivation and the one-way-door encoding +status: pending +depends_on: [core-error-model] +scope: narrow +risk: medium +impact: project +level: implementation +tags: [wave-1, keys] +--- + +## Description + +Implement the content-address layer (`src/keys.rs` or `keys/` + +`hashing`, one module per file per convention) per ADR-002 and +hashing-and-keys.md. This is the crate's declared one-way door — key +bytes persist in backend rows and filenames with no migration story — +so the encoding is implemented exactly to the ADR text and reviewed +line-by-line at the wave gate. + +- **`Digest`** — algorithm + digest pair. Algorithm enum with exactly + two members: `GitBlobSha256` (canonical), `GitBlobSha1` (tolerated + legacy). BLAKE3 is absent from the crate entirely (ADR-002 §Decision + class 3). +- **Canonical derivation**: `H("blob \0" + content)` — SHA-256 + canonical, SHA-1 same derivation. The preamble is the algorithm's + domain separation, not a special case (ADR-002 §The preamble as + abstraction). Known-length derivation is the primary constructor; + the unknown-length path's derivation-over-staged-bytes restart is + wave 2 facade logic, not here. +- **Interop constructor**: `from_git_oid_hex` shape (POC #1 finding 4) + — a pre-computed pair from an external git oid, both algorithms. +- **`Key`** — the typed wrapper; `as_bytes()` / `from_bytes()` are the + only conversions across the backend boundary. Physical encoding per + ADR-002: `1 algorithm byte + digest bytes` — `0x01` = + git-blob-sha-256 (32-byte digest; total 33), `0x02` = git-blob-sha-1 + (20-byte digest; total 21). `0x00` is reserved and never assigned. +- **Validation**: truncated keys and unknown algorithm bytes are + rejects (`Error::KeyInvalid`), never guesses (the POC's + length-checking test is the standard). +- **Sort stability**: fixed-length enum-tagged keys sort in total + order (required for the sweep's live-set diffs) — asserted by test. +- **Registry constants**: the algorithm bytes are the ADR-002 + registry; new algorithms require a new ADR (doc comment states + this). +- **Interop test (the load-bearing test)**: byte-exact derivation + against the `git hash-object` CLI for both algorithms (POC #1 + findings 4/5's standard — validated against git, not just hardcoded + vectors), plus the external-oid round-trip: `from_git_oid_hex` → + `as_bytes` → `from_bytes` → equal, and tagged-key coexistence of + both algorithms in one flat collection. +- **Named design point for the review gate**: oid hex parsing policy — + git oids are lowercase hex; pin lowercase-only acceptance (reject + uppercase as malformed) or case-insensitive acceptance, with the + choice recorded. Lean lowercase-only (git's own output is always + lowercase; strictness at the boundary matches rejects-never-guesses) + unless implementation finds a concrete interop reason otherwise. +- Deps: `sha2`, `sha1` (RustCrypto), `hex` — added to the manifest by + this task. + +## Acceptance Criteria + +- [ ] `Digest`/`Key`/algorithm enum per ADR-002; encoding byte-exact + (33/21 total lengths; 0x01/0x02 tags; 0x00 rejected) +- [ ] Canonical derivation matches `git hash-object` byte-exact for + both algorithms (CLI-driven test, both a small and a multi-KiB + content case) +- [ ] `from_git_oid_hex` round-trips external oids into pool keys +- [ ] Rejects: truncated keys, unknown algorithm bytes (incl. 0x00, + 0xFF), wrong digest lengths → `Error::KeyInvalid`, never guesses +- [ ] Total-order sort stability asserted (mixed-algorithm collection + sorts deterministically) +- [ ] Doc comments carry the one-way-door warning and the registry + note (new algorithm = new ADR) +- [ ] `cargo test`, `cargo clippy --all-targets -- -D warnings`, + `cargo fmt --check` green + +## References + +- docs/architecture/hashing-and-keys.md +- docs/architecture/decisions/002-canonical-hash-and-key-encoding.md +- docs/research/poc-trait-dispatch-findings.md findings 4/5 +- AGENTS.md convention 5 (the one-way door) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/core-trait-and-types.md b/tasks/core-trait-and-types.md new file mode 100644 index 0000000..2e59d3e --- /dev/null +++ b/tasks/core-trait-and-types.md @@ -0,0 +1,119 @@ +--- +id: core-trait-and-types +name: Backend trait, I/O seams, engine-state seam shape, and facade value types +status: pending +depends_on: [core-error-model, core-hashing-keys] +scope: moderate +risk: medium +impact: project +level: implementation +tags: [wave-1, backend, traits] +--- + +## Description + +Implement the crate's second stable seam and the facade's value types +— everything engine-blind that waves 2–6 implement *against*. No +engine code, no facade behavior logic (those are wave 2); this task is +the contract surface in code. + +- **`Backend` trait** (ADR-003 + ADR-008's `size`): methods + `has`/`get`/`put`/`delete`/`list`/`name`/`size` over **opaque byte + keys** (`&[u8]`) and opaque byte-valued entries. Typed `Key` + converts at the store boundary only. Namespace-blind; virgin-store + reads are no-ops (engine posture, documented on the trait). + **Named design point for the review gate**: trait visibility — all + engines ship in-crate as features and consumers never implement a + Backend (ADR-004), so the trait is `pub(crate)` with its + stable-seam status documented, unless the gate reads ADR-003's + "second stable seam" as requiring public export. +- **Read cursor abstraction** (ADR-010 §1, crate-internal): the type + `get` returns; the store core drives it, never the engine shape. + Two body families documented: kv engines materialize the whole + value (bounded by tier policy; `len()` from the row's size); large + engines stream (`local` pread loop, pg-lo `lo_get` windows — wave + 4/5 bodies). Both serve `read_range` → slice + slice digest at one + seam (ADR-006: SHA-256 over the slice bytes, fixed by convention — + the digest convention is implemented here and unit-tested). The + cursor is not public API. +- **Staged put** (ADR-010 §1, the named per-engine contract): the + trait's put side has two forms selected by the store core before + content flows — the whole-value put (kv tier's settled row write) + and the staged put (the engine hands out a *staging cursor*; commit + publishes; every failure path converges on discard with zero + residue). **Named design point for the review gate**: the exact + trait method shapes for the two forms (e.g. `put` + + `put_staged() -> StagingCursor` with commit/discard on the cursor, + or an equivalent) — pinned here so engine tasks never guess + signatures. +- **Engine-state companion trait** (ADR-010 §2, crate-internal, never + exported): the clause families with per-engine applicability + documented on the trait (ADR-012 §5's table): `ensure_gc_schema()`, + `pin_commit(joint_tx, entries)`, `pin_renew(key, owner)`, + `pin_check(key) -> PinState` / `arbitrate(key) -> bool`, + `window_stage(candidates)` / `window_take(now)`, + `sweep_lock_acquire()`/`release()`. Applicability: postgres — all + families (the fleet host); sqlite and pg-lo — window family only; + `local`/`mem` — excluded by construction (never fleet engines). + The trait shape is defined now; SQL engines implement it in waves + 3–5. The joint-tx holder type (`joint_tx`) is a declared + crate-internal type whose concrete per-engine bodies are engine + tasks — the trait pins the ownership rule (store holds the tx; + engine contributes mechanics) in its signature and docs. +- **Facade value types** (`src/facade.rs` types module or equivalent): + `EntryMeta { len: u64, algorithm }`; `Pin` — RAII guard type with + its lifecycle contract documented (drop or convert into a + registered reference; the conversion-ordering rule per ADR-005; + the arbitration machinery itself is wave 2); `Batch` — batch-scoped + pin group (drops together on commit or drop); `SweepReport { + candidates_staged, deleted, cancelled_by_arbitration, aborted: + Option, started_at, finished_at }`; `GcAbortCause` — + data enum with exactly three variants (`ProtectFailed`, + `SweeperLock`, `NoLivenessSources`), callers match + non-exhaustively, new causes are new ADRs; `PinState` — + `Protected`/`Unprotected`. +- **Async shape** (named design point for the review gate): async fn + in trait (AFIT) with the engines statically known (no dyn + requirement), vs `async_trait` — pin the choice; the family's tokio + posture and the in-crate engine set favor AFIT unless a dyn need + appears. +- Unit tests: the slice-digest convention (SHA-256 over slice bytes), + cursor contract against a test-double body, value-type invariants + (GcAbortCause exhaustiveness shape, SweepReport field semantics). + +## Acceptance Criteria + +- [ ] `Backend` trait with the seven methods, opaque byte keys, + `size` metadata-only discipline documented +- [ ] Read cursor + staging cursor abstractions compile and are + unit-tested against a test-double body; slice-digest convention + tested +- [ ] Engine-state companion trait with all clause families and the + per-engine applicability table in docs; crate-internal (not + exported from lib.rs) +- [ ] Facade value types complete (`EntryMeta`, `Pin`, `Batch`, + `SweepReport`, `GcAbortCause` ×3, `PinState`) with lifecycle + contracts documented +- [ ] The three named design points (trait visibility, put-form + shapes, async shape) decided and recorded in the task's Notes + for the review gate +- [ ] `cargo test`, `cargo clippy --all-targets -- -D warnings`, + `cargo fmt --check` green + +## References + +- docs/architecture/backends-and-dispatch.md (§The Backend trait contract) +- docs/architecture/store-api.md (§Public surface, §Error model, value shapes) +- docs/architecture/decisions/003-backend-contract-and-dispatch.md +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §1–§2 +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §5 (clause applicability) +- docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md (Pin/Batch lifecycle contract) +- docs/architecture/decisions/006-verification-posture-and-transfer-encoding.md (slice digest) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/review-wave-1.md b/tasks/review-wave-1.md new file mode 100644 index 0000000..f35659a --- /dev/null +++ b/tasks/review-wave-1.md @@ -0,0 +1,83 @@ +--- +id: review-wave-1 +name: Wave 1 review gate — the one-way-door gate +status: pending +depends_on: [core-hashing-keys, core-error-model, core-trait-and-types, suite-scaffold] +scope: narrow +risk: low +impact: phase +level: review +tags: [wave-1, review] +--- + +## Description + +The wave-1 review gate, before wave 2 decomposes. Wave 1 is the +contract surface every later wave implements against, and it carries +the crate's declared one-way door — a shape error found here is a task +revision; found after an engine ships data it is a migration ADR. The +gate reads the code against the pinned ADR text line by line. + +Gate checks: + +- **Key encoding vs ADR-002, line by line** (the one-way door): byte + layout (`1 alg byte + digest`, 0x01/0x02, 0x00 reserved, 33/21 + lengths), rejects-never-guesses, sort stability, the preamble-as- + abstraction derivation, the interop test's fidelity (git CLI, not + just vectors), the oid hex policy decision recorded. +- **Backend trait + seams vs ADR-003/010**: the seven-method set with + `size` metadata-only; opaque byte keys; the read-cursor abstraction + (two body families, range contract at one seam, not public API); + the staged-put contract (staging cursor, commit publishes, discard + converges); the engine-state companion trait's clause families and + per-engine applicability vs ADR-012 §5's table. +- **Error model vs store-api.md / ADR-012 §6.5**: the five variants; + the altitude discipline; the single-enum reading audited against + the spec text (the named design point from `core-error-model`). +- **Feature graph vs ADR-013 §4**: the manifest's feature set, the + pg-lo-no-implies-postgres rule, `sqlite` default-on, `local` + ungated. +- **The named design points audited as resolved** (each recorded in + its task's Notes): single-enum error reading; Backend trait + visibility; trait put-form method shapes; oid hex policy; async + trait shape (AFIT vs async_trait). +- **Suite convention**: the factory contract, stamp format, and + backlog traceable to store-api.md's invariants. +- **Family conventions**: no comments in code (doc comments on public + API fine), thiserror, tokio-only async, no unwrap/expect outside + tests, naming, module structure. +- **Gates green**: `cargo test`, `cargo clippy --all-targets -- -D + warnings`, `cargo fmt --check`, `cargo check --all-features`. + +Findings are fixed inline where cheap, or recorded as wave-2 +amendments where they shape the next decomposition. The gate's +completion is what unblocks wave 2's decomposition. + +## Acceptance Criteria + +- [ ] Every gate check above read and verified against the cited ADR + text (not from memory of it) +- [ ] Named design points each decided, recorded, and consistent with + the specs +- [ ] Findings fixed inline or recorded as wave-2 amendments with + owners +- [ ] All gates re-run green after any fixes +- [ ] Wave 2 decomposition unblocked (recorded in this task's Summary) + +## References + +- docs/plans/implementation.md (§Wave 1, §Review gates) +- docs/architecture/decisions/002-canonical-hash-and-key-encoding.md +- docs/architecture/decisions/003-backend-contract-and-dispatch.md +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md +- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §4 +- docs/architecture/store-api.md + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/scaffold-crate-features.md b/tasks/scaffold-crate-features.md new file mode 100644 index 0000000..53f1b55 --- /dev/null +++ b/tasks/scaffold-crate-features.md @@ -0,0 +1,72 @@ +--- +id: scaffold-crate-features +name: Crate scaffold — ADR-013 §4 feature graph and module skeleton +status: pending +depends_on: [] +scope: narrow +risk: low +impact: project +level: implementation +tags: [wave-1, workspace] +--- + +## Description + +Replace the placeholder crate body with the real skeleton, so every +later task lands in an existing, compiling structure with the decided +feature graph present from the first commit: + +- **Feature graph per ADR-013 §4** (backends-and-dispatch.md's table + is the spec home): `sqlite` (default-on, carries rusqlite with the + `bundled` feature — match alkstore's engine manifests as the family + baseline, verifying current compatible versions at implementation), + `postgres` (tokio-postgres + deadpool-postgres), `mem` (no driver — + dependency-free), `pg-lo` (the shared pg cluster: tokio-postgres + + deadpool-postgres; **must not** imply `postgres` — ADR-010 §1's + feature-graph consequence), `ops` (alkcall, default-off — published + crate, version dep per the sibling posture, `alkcall = "0.8.1"`). + `local` is ungated (no feature, no dependency). `default = ["sqlite"]`. +- **Module skeleton** under `src/`: `lib.rs` re-export shape with + module stubs for the planned tree (keys/hashing, error, backend, + facade, dispatch, gc; engine modules behind `#[cfg(feature)]`). + Remove `pub fn placeholder()`. Modules are one per file, re-exported + from `lib.rs` (AGENTS convention 10); engine module files exist + gated and empty. +- Base dependencies stay as-is (tokio, thiserror, parking_lot). Driver + deps declared now with no consuming code — unused optional deps are + inert, and the graph being a decided artifact (ADR-013 §4) is why it + lands whole. + +Family standards apply from this commit onward: tokio async runtime, +`thiserror` errors, no panics in library code, no `unwrap()`/`expect()` +outside tests, no comments in code (doc comments on public API fine). + +## Acceptance Criteria + +- [ ] `cargo test` green with the default build (sqlite feature on, + no engine code yet) +- [ ] `cargo check --all-features` resolves the full feature graph + (sqlite, postgres, mem, pg-lo, ops) with their driver clusters +- [ ] `pg-lo` alone (`cargo check --no-default-features --features pg-lo`) + resolves without pulling postgres-only machinery (the + no-implication rule, ADR-010 §1) +- [ ] `cargo clippy --all-targets -- -D warnings` and + `cargo fmt --check` green +- [ ] `lib.rs` carries the module re-export shape; no placeholder + function remains + +## References + +- docs/architecture/overview.md (dependency posture table) +- docs/architecture/backends-and-dispatch.md (the engine/feature table) +- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §4 +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §1 (pg-lo no-implication) +- /workspace/@alkdev/alkstore/tasks/scaffold-workspace.md (the family scaffold precedent) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/suite-scaffold.md b/tasks/suite-scaffold.md new file mode 100644 index 0000000..cd89be2 --- /dev/null +++ b/tasks/suite-scaffold.md @@ -0,0 +1,73 @@ +--- +id: suite-scaffold +name: Conformance suite — home, convention, and invariant backlog +status: pending +depends_on: [scaffold-crate-features] +scope: narrow +risk: low +impact: project +level: implementation +tags: [wave-1, suite] +--- + +## Description + +Establish `tests/conformance/` — the in-crate conformance harness home +(decided point: engines are features of one crate, so the suite lives +in-crate, factory-parameterized over engine constructors; the +alkstore ADR-017/022 pattern adapted) — with the convention that waves +2–6 grow columns against: + +- **The convention document** (a `tests/conformance/README.md` or + module doc): row naming, the ADR-stamp format (every row cites the + ADR § / spec § that pins it — the alkstore version-stamping + convention), the one-normative-owner-per-property rule, and the + factory contract: each factory opens a **fresh, isolated** store + instance (no shared state between rows) with **idempotent + teardown**; engine factories for SQL engines skip cleanly when no + server is reachable (the alkstore SKIP posture) while acceptance + requires green against the server. +- **The invariant backlog**: store-api.md's seven invariants + registered as the rows-to-grow table (round-trips per engine/tier/ + put-path; dedup; stage hygiene; sweep safety incl. the engine-gated + fleet form; list-correctness-through-sweep-outcomes; `stat`-without- + content; engine-conformance uniformity) — each row's first column + lands with the mem engine in wave 2 (mem is the contract-reference + engine; every other engine's rows mirror its column). +- **The harness skeleton**: the factory trait shape and the row + registration structure, compile-able as a stub (the `Store` facade + type arrives in wave 2 — the skeleton pins the shape, wave 2's + first column binds it; if binding reveals a shape correction, that + is wave 2's to make and record). + +This task is deliberately convention-first: the harness shape is +easier to grow row by row as engines land than to retrofit onto five +finished engines (the alkstore wave-1 rationale, which applies +identically here). + +## Acceptance Criteria + +- [ ] `tests/conformance/` exists with the convention doc (row naming, + ADR-stamp format, one-owner rule, factory isolation + teardown + contract, SKIP posture) +- [ ] The invariant backlog table registered, traceable to + store-api.md's seven invariants, with the mem-first column plan + noted +- [ ] The harness skeleton compiles (`cargo test` green — a stub with + no rows yet is acceptable and expected) +- [ ] No engine-specific code; the task adds no dependencies + +## References + +- docs/architecture/store-api.md (§Invariants — the test gate) +- docs/architecture/backends-and-dispatch.md (§Per-engine conformance; POC #7 gate as the norm) +- /workspace/@alkdev/alkstore/docs/plans/implementation.md (wave 1 + wave 5 sections — the convention's source pattern) +- docs/plans/implementation.md (Decided points — suite home) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file