docs(tasks): decompose wave 2 — mem + local engines, facade, in-process GC, first suite columns
Nine tasks against the wave-1 contract surface as reviewed (no amendments from the gate): - engine-mem / engine-local (parallel, dependency-free): the two engines; local owns the stage-hygiene exact-count tests and the Drop-cancellation story the wave-1 review recorded - store-construction: the Store type, constructor modes, size-threshold dispatch; binds the conformance factory's Store associated type - store-read-paths ∥ gc-protocol: read ops with the ADR-012 §6.1 fall-through rule; the ADR-005 in-process protocol (pins, liveness, delete windows, abort-as-data, GcRefuse with the full protection set) - store-put-paths: the two put paths behind one signature, ADR-003's three numbered rules, pin-before-publish, Batch, kv-only rejection - suite-column-mem → suite-column-local: the reference column, then the dual-tier dispatch-coverage shape mirroring it - review-wave-2: the gate (facade/dispatch/GC vs ADR-003/005/012) Verification: dependency graph validated (all ids resolve, no cycles, topological order clean); docs-only change. Refs: docs/plans/implementation.md §Wave 2
This commit is contained in:
1 parent
c1d9494c11
commit
6bce301c55
10 files changed
+834
-2
No files matched your search
@@ -0,0 +1,95 @@
|
||||
---
|
||||
id: engine-local
|
||||
name: local large engine — sharded files, stage-then-commit-rename, pread cursors
|
||||
status: pending
|
||||
depends_on: []
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-2, engines, local]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Implement the `local` large-tier engine (ungated — dependency-free,
|
||||
always present; ADR-003/008) in `src/local.rs`: the ADR-003 backend as
|
||||
specced.
|
||||
|
||||
- **Constructor**: takes a root directory (tests use fresh temp dirs).
|
||||
Layout: flat `{hex-prefix}/{hex-prefix}/{hash}` sharding (iroh's
|
||||
surviving conclusion — limits directory size on huge pools).
|
||||
**Named design point**: the filename encoding — lowercase hex of the
|
||||
full key bytes (`Key::as_bytes`, alg-tagged and self-describing per
|
||||
ADR-002's persistence-boundary ruling), prefixes taken from the hex
|
||||
string; record the decision (and the list()-decode round-trip) for
|
||||
the review gate.
|
||||
- **Staged put** (the staged-put contract realized, ADR-010 §1):
|
||||
stage-then-commit-rename. `put_staged` hands out a staging cursor
|
||||
backed by a stage file in a staging area (e.g. `{root}/staging/`,
|
||||
unique per stage); `append` writes through; `commit` renames the
|
||||
stage file into its sharded committed path (creating parent dirs) —
|
||||
publish is the rename; **every failure path converges on discard with
|
||||
zero residue** (commit failure self-discards; there is no
|
||||
post-failure discard to drive). `Drop` is best-effort residue cleanup
|
||||
— and per the wave-1 review's recorded observation, this engine's
|
||||
hygiene tests must cover its own cancellation story (a staging cursor
|
||||
dropped without commit/discard leaves no residue).
|
||||
- **Read cursor**: the **streaming family** — a pread loop over the
|
||||
committed file (immutable once committed), `read_at` clamping at EOF
|
||||
pread-style, the provided `read_range` composing the slice-digest
|
||||
convention. **Blocking file work lives in `spawn_blocking`** inside
|
||||
the impl (store-api.md §Concurrency posture; the store never blocks
|
||||
the executor).
|
||||
- **Whole-value `put`**: the large tier's settled form for direct
|
||||
whole-value writes — write-then-rename into the sharded path, CAS
|
||||
first-writer-wins (an existing file for the key is never overwritten).
|
||||
- **`size`**: file stat — metadata-only, never a content load.
|
||||
- **`list`**: walk the sharded tree, decode filenames to key bytes —
|
||||
complete and well-formed by contract (a malformed list is as deadly
|
||||
as an incomplete one, ADR-003).
|
||||
- **`delete`**: remove the committed file; `Err(Error::Missing)` when
|
||||
absent. Virgin store (fresh root): reads are no-ops, `list` empty.
|
||||
- **Exact-count stage-hygiene tests** (POC #3 finding A5's shape,
|
||||
store-api.md invariant 3): N failed puts leave zero stage files and
|
||||
zero committed entries; N successful puts leave exactly N committed
|
||||
entries and zero stage files — over both failure modes (explicit
|
||||
discard, failed commit) plus the cancellation/Drop form.
|
||||
|
||||
The conformance column itself is `suite-column-local` (separate task);
|
||||
this task ships the engine and its in-module unit tests only.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `src/local.rs` implements `Backend` completely (ungated), with
|
||||
the sharded layout, the streaming pread cursor, and the
|
||||
stage-then-commit-rename staging cursor
|
||||
- [ ] Filename encoding decided and recorded (self-describing key
|
||||
bytes; list()-decode round-trips)
|
||||
- [ ] All file I/O off the async path (`spawn_blocking`); no blocking
|
||||
on the executor
|
||||
- [ ] Exact-count stage-hygiene tests green over discard, failed
|
||||
commit, and Drop-cancellation forms — zero residue on every
|
||||
failure path, exactly one committed entry per success
|
||||
- [ ] Virgin-store reads are no-ops on a fresh root; `size` is a stat,
|
||||
never a content load; `list` is complete and well-formed
|
||||
- [ ] `cargo test` green (local is ungated — in the default build);
|
||||
`cargo clippy --all-targets -- -D warnings`, `cargo fmt --check`
|
||||
green
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/backends-and-dispatch.md (§large tier — local engine)
|
||||
- docs/architecture/decisions/003-backend-contract-and-dispatch.md (stage-then-commit-rename, sharding)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §1 (the staged-put contract)
|
||||
- docs/architecture/store-api.md (§Concurrency posture; §Invariants 3)
|
||||
- docs/research/poc-largeblob-findings.md (findings A2/A5 — pread soundness, stage hygiene)
|
||||
- tasks/review-wave-1.md (Notes — the cancellation-residue observation this task owns)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
Reference in new issue
Block a user