docs(tasks): decompose wave 2 — mem + local engines, facade, in-process GC, first suite columns

Nine tasks against the wave-1 contract surface as reviewed (no
amendments from the gate):

- engine-mem / engine-local (parallel, dependency-free): the two
  engines; local owns the stage-hygiene exact-count tests and the
  Drop-cancellation story the wave-1 review recorded
- store-construction: the Store type, constructor modes, size-threshold
  dispatch; binds the conformance factory's Store associated type
- store-read-paths ∥ gc-protocol: read ops with the ADR-012 §6.1
  fall-through rule; the ADR-005 in-process protocol (pins, liveness,
  delete windows, abort-as-data, GcRefuse with the full protection set)
- store-put-paths: the two put paths behind one signature, ADR-003's
  three numbered rules, pin-before-publish, Batch, kv-only rejection
- suite-column-mem → suite-column-local: the reference column, then the
  dual-tier dispatch-coverage shape mirroring it
- review-wave-2: the gate (facade/dispatch/GC vs ADR-003/005/012)

Verification: dependency graph validated (all ids resolve, no cycles,
topological order clean); docs-only change.

Refs: docs/plans/implementation.md §Wave 2
This commit is contained in:
glm-5.3-flash committed 2026-10-11 21:39:28 +00:00
1 parent c1d9494c11
commit 6bce301c55
10 files changed
+834 -2

No files matched your search

+95
View File
@@ -0,0 +1,95 @@
---
id: engine-local
name: local large engine — sharded files, stage-then-commit-rename, pread cursors
status: pending
depends_on: []
scope: moderate
risk: medium
impact: component
level: implementation
tags: [wave-2, engines, local]
---
## Description
Implement the `local` large-tier engine (ungated — dependency-free,
always present; ADR-003/008) in `src/local.rs`: the ADR-003 backend as
specced.
- **Constructor**: takes a root directory (tests use fresh temp dirs).
Layout: flat `{hex-prefix}/{hex-prefix}/{hash}` sharding (iroh's
surviving conclusion — limits directory size on huge pools).
**Named design point**: the filename encoding — lowercase hex of the
full key bytes (`Key::as_bytes`, alg-tagged and self-describing per
ADR-002's persistence-boundary ruling), prefixes taken from the hex
string; record the decision (and the list()-decode round-trip) for
the review gate.
- **Staged put** (the staged-put contract realized, ADR-010 §1):
stage-then-commit-rename. `put_staged` hands out a staging cursor
backed by a stage file in a staging area (e.g. `{root}/staging/`,
unique per stage); `append` writes through; `commit` renames the
stage file into its sharded committed path (creating parent dirs) —
publish is the rename; **every failure path converges on discard with
zero residue** (commit failure self-discards; there is no
post-failure discard to drive). `Drop` is best-effort residue cleanup
— and per the wave-1 review's recorded observation, this engine's
hygiene tests must cover its own cancellation story (a staging cursor
dropped without commit/discard leaves no residue).
- **Read cursor**: the **streaming family** — a pread loop over the
committed file (immutable once committed), `read_at` clamping at EOF
pread-style, the provided `read_range` composing the slice-digest
convention. **Blocking file work lives in `spawn_blocking`** inside
the impl (store-api.md §Concurrency posture; the store never blocks
the executor).
- **Whole-value `put`**: the large tier's settled form for direct
whole-value writes — write-then-rename into the sharded path, CAS
first-writer-wins (an existing file for the key is never overwritten).
- **`size`**: file stat — metadata-only, never a content load.
- **`list`**: walk the sharded tree, decode filenames to key bytes —
complete and well-formed by contract (a malformed list is as deadly
as an incomplete one, ADR-003).
- **`delete`**: remove the committed file; `Err(Error::Missing)` when
absent. Virgin store (fresh root): reads are no-ops, `list` empty.
- **Exact-count stage-hygiene tests** (POC #3 finding A5's shape,
store-api.md invariant 3): N failed puts leave zero stage files and
zero committed entries; N successful puts leave exactly N committed
entries and zero stage files — over both failure modes (explicit
discard, failed commit) plus the cancellation/Drop form.
The conformance column itself is `suite-column-local` (separate task);
this task ships the engine and its in-module unit tests only.
## Acceptance Criteria
- [ ] `src/local.rs` implements `Backend` completely (ungated), with
the sharded layout, the streaming pread cursor, and the
stage-then-commit-rename staging cursor
- [ ] Filename encoding decided and recorded (self-describing key
bytes; list()-decode round-trips)
- [ ] All file I/O off the async path (`spawn_blocking`); no blocking
on the executor
- [ ] Exact-count stage-hygiene tests green over discard, failed
commit, and Drop-cancellation forms — zero residue on every
failure path, exactly one committed entry per success
- [ ] Virgin-store reads are no-ops on a fresh root; `size` is a stat,
never a content load; `list` is complete and well-formed
- [ ] `cargo test` green (local is ungated — in the default build);
`cargo clippy --all-targets -- -D warnings`, `cargo fmt --check`
green
## References
- docs/architecture/backends-and-dispatch.md (§large tier — local engine)
- docs/architecture/decisions/003-backend-contract-and-dispatch.md (stage-then-commit-rename, sharding)
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §1 (the staged-put contract)
- docs/architecture/store-api.md (§Concurrency posture; §Invariants 3)
- docs/research/poc-largeblob-findings.md (findings A2/A5 — pread soundness, stage hygiene)
- tasks/review-wave-1.md (Notes — the cancellation-residue observation this task owns)
## Notes
> To be filled by implementation agent
## Summary
> To be filled on completion