--- id: store-put-paths name: Store put seam — two internal paths, the three numbered rules, Batch, kv-only rejection status: completed depends_on: [gc-protocol] scope: moderate risk: medium impact: component level: implementation tags: [wave-2, facade, write-path] --- ## Description Implement the facade's put seam on the `Store` type (store-api.md §Write path): **`put(len: Option, stream) → (Key, Pin)`** — two internal paths behind one signature, with pin-before-publish from the `gc-protocol` registry. - **Known-length path** (encouraged, documented as such on the API): the preamble (`"blob \0"`) is known before content flows, so the hash initializes with it and one pass suffices — no staging needed for hashing. Routing by the dispatch seam: ≤ threshold → the kv tier's whole-value put; > threshold → the large tier's staged put (content streams into the staging cursor while hashing; the derivation completes at stream end; commit-rename publishes). - **Unknown-length path**: content buffers in memory up to the dispatch threshold. ADR-003's **three numbered rules** (option (d)), each a test-asserted behavior: 1. **Threshold exceeded mid-stream** (non-restartable sources are normal): the buffered prefix flushes *into a fresh stage file* and the stream continues appending into it — there is no "restart the stream" requirement anywhere. 2. **Verification runs over the eventual stored artifact as a whole**: the memory buffer for the sub-threshold kv commit; the complete staged bytes for the large commit — the derivation pass restarts over preamble + staged bytes (forced by the preamble being inside the hashed input, ADR-002), reading the staged artifact back through the `StagingCursor: ReadCursor` seam (the wave-1 design-led extension). There is no stitched-input case left ambiguous. 3. **Stream error before commit**: no entry lands and no stage file remains — buffer discard on the failure path, exactly like stage discard (the stage-hygiene invariant extends to the buffered case). - **Pin-before-publish** (ADR-005): the pin refcount is acquired under the arbitration lock *before* the entry becomes visible — publish and protection are one step; an entry cannot appear in a sweep's `list()` without already carrying its pin. - **Dedup**: a re-put of existing content acquires no new pool entry — the returned `Pin` refcounts the existing one under the same lock. - **`Batch`** (store-api.md §Write path): the batching put seam — multi-put writes group under one batch-scoped pin; member puts publish pin-before-publish exactly as lone puts do; the batch's pins drop together on commit or drop (the `gc-protocol` machinery drives). - **kv-only mode rejection** (ADR-012 §6.2): known-length puts exceeding the threshold reject **immediately**; unknown-length puts reject **at mid-stream threshold overflow** — lengths are not known at construction, so put time is the only possible rejection point. Zero residue on either rejection. - **Named design points**: (a) the stream parameter shape (`impl AsyncRead + Unpin + Send` vs a boxed trait object — the family's no-dyn posture vs ergonomics); (b) the declared-len mismatch case (a known-length stream that yields fewer/more bytes than `len` — which error family member carries it, and zero residue either way). Record both for the review gate. ## Acceptance Criteria - [x] `put(len, stream) → (Key, Pin)` with both internal paths behind the one signature; routing is the pure length function (no content ever migrates tiers) - [x] The three numbered rules each behavior-tested: mid-stream overflow flushes into a fresh stage and the stream continues; the derivation pass runs over the eventual stored artifact as a whole (buffer form and staged-bytes form); stream error leaves no entry and no stage file - [x] Pin-before-publish holds on both paths and both tiers; dedup re-puts refcount the existing entry - [x] `Batch` groups multi-put writes; member pins drop together - [x] kv-only mode: known-length over-threshold rejects immediately; unknown-length rejects at mid-stream overflow; zero residue on both - [x] Stream-shape and len-mismatch design points decided and recorded - [x] `cargo test` green; `cargo test --features mem` and `cargo test --all-features` green; clippy/fmt gates green ## References - docs/architecture/store-api.md (§Write path — the put seam, stage hygiene invariant) - docs/architecture/decisions/003-backend-contract-and-dispatch.md (§Dispatch policy — the three numbered rules, option (d)) - docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §6.2 (kv-only put-time rejection) - docs/architecture/decisions/005-pooled-cas-and-gc-mechanism.md (pin-before-publish, dedup refcount) - docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §1 (the staged-put contract) - src/backend.rs (StagingCursor: ReadCursor — the verification-pass seam) ## Notes - **The staged-put seam was amended: the key resolves at commit, not at handout.** Implementing the two large-tier flows surfaced a contradiction in the wave-1 pin: `put_staged(key)` hands the staging cursor out with the key, but both large-tier put paths only know the canonical key when the derivation completes — at stream end (the preamble covers a length, and the digest covers content that has not streamed yet). Content streams once (non-restartable sources are the normal case), so `commit(self)` publishing under the handout key cannot express the specced flow ("the derivation completes at stream end; commit-rename publishes"). Amended with operator sign-off: `put_staged()` is a keyless handout and `StagingCursor::commit(self, key)` publishes under the store-core-resolved key (CAS first-writer-wins as the whole-value put). No ADR text pins the key's arrival point (ADR-003/010 pin handout/commit/discard mechanics only); the wave-1 task notes' `put_staged(key)` shape was pre-flow realism. Recorded for review-wave-2. Engines updated in place (mem, local, the wave-1 test double) with their hygiene suites re-run green. - **Put-path named design points (recorded in `src/store.rs` module docs, for review-wave-2):** - *Stream parameter shape:* `put(len, stream: impl AsyncRead + Unpin + Send)` — generic, no boxing; the family's no-`dyn` posture carried to the write seam (a boxed reader is itself `AsyncRead + Unpin + Send` under tokio's blanket impl, so callers needing erasure box at their call site). `Send` keeps spawn-site bounds resolvable for the ops wave; no `'static` (nothing spawns). - *Declared-len mismatch:* a known-length stream yielding fewer or more bytes than `len` rejects `Error::Verification` on either tier, zero residue either side — the preamble carries the *declared* length, so a mismatched artifact's canonical address can never equal the stored key (storing it would break pool integrity by construction; `Verification` is the put-path content-≠-key family member, ADR-006's whole-blob posture applied at put time). - *kv-only rejection carrier (a named decision beyond the two the task listed):* `Error::Io(String)` with a descriptive message — the ADR-pinned family (ADR-012 §6.5) holds no dedicated variant and the rejection is an operational/capability fact about the medium posture, the bucket `Io(String)` covers. A dedicated variant for typed matchability would be a new failure family — a new ADR. - **Flows as implemented** (`src/store.rs`, write-path `impl Store`): known-length initializes the derivation with the preamble before content flows and streams once (kv: buffer + hash in one pass → mint pin → whole-value put; large: stream into the staging cursor while hashing → mint pin → `commit(key)`). Unknown-length buffers to the threshold; a mid-stream overflow (strictly past it — exactly-at stays kv) flushes the buffered prefix into a fresh stage and the stream continues into it; at EOF the derivation pass runs over the eventual stored artifact as a whole — the buffer (kv form) or preamble + staged bytes read back through the `StagingCursor: ReadCursor` seam (staged form) — then pin, then commit. Every failure path (read error, mismatch, under-read) discards: no entry, no stage file. - **API shapes:** `Store::put(&self, len: Option, stream) → Result<(Key, Pin), Error>`; the batching seam is `Store::batch() → Batch` + `Store::put_in_batch(&mut Batch, len, stream) → Result` (members run the lone-put flow; the minted guard is contributed to the group via crate-internal `Batch::add`); a public `Batch::commit()` was added consuming the batch with the same one-step release as drop (store-api.md's "pins drop together on batch commit or drop", as a named verb). `hashing.rs` gained crate-internal `BlobHasher` — the same derivation driven incrementally, pinned byte-equal to `Algorithm::derive` by tests (chunk-boundary invariance, both algorithms, the git vectors). - **git-interop leg added** (`tests/git_interop.rs`): the put seam keys entries git-identically through both put paths on both tiers against live `git hash-object` (INV-1's source-of-truth posture extended from the derivation constructor to the facade seam). - **Wave-1 note updates:** `put_staged`/`commit` doc text in `backend.rs`, `mem.rs`, `local.rs` now carries the key-at-commit design point; the store.rs module doc's remaining-deferral note shrunk to the actual remaining gaps (fleet clauses, wave-3 kv engine). ## Summary Implemented the facade's put seam (`src/store.rs`): `put(len: Option, stream: impl AsyncRead + Unpin + Send) → Result<(Key, Pin), Error>` with the two internal paths behind the one signature — the known-length one-pass flow and the unknown-length buffer-then-maybe-flush flow implementing ADR-003's option (d) three numbered rules — plus `batch()`/`put_in_batch()` (ADR-005 §Marks 2's batch-scoped pin group) and kv-only's put-time rejection at its two pinned points (ADR-012 §6.2). Pin-before-publish rides `gc().acquire` under the arbitration lock before both tiers' publishes; dedup re-puts refcount the existing entry; every failure path converges on discard with zero stage files. The staged-put seam was amended (keyless handout; `commit(self, key)`) after operator sign-off — the one blocking finding of the task. Supporting changes: staging-cursor dispatch enum `AnyStaging` beside the read path's `AnyCursor`; incremental `BlobHasher` in `hashing.rs`; public `Batch::commit`; a `Pin`/`Debug` impl; the git-interop put-seam leg. Eleven put-path tests in `src/store.rs` (plus four fixtures) pin the three rules, both-tier round-trips, the threshold boundary, pin- before-publish under a racing sweep, dedup refcounting across paths and tiers, batch grouping/commit/failure, kv-only's two rejection points, len-mismatch zero-residue, mem-only's threshold-free posture, and the empty-content degenerate. Verification: `cargo test` (default), `cargo test --features mem`, `cargo test --all-features` all green (97 lib tests under mem plus conformance and git-interop); `cargo clippy --all-targets -- -D warnings` green in all three feature shapes; `cargo fmt --check` green; `cargo doc --no-deps` at the pre-existing 24-warning baseline.