feat(store): the put seam — two internal paths, the three numbered rules, Batch, kv-only rejection
- `put(len: Option<u64>, stream: impl AsyncRead + Unpin + Send) -> (Key, Pin)`: known-length one-pass flow; unknown-length buffer-to-threshold flow with the mid-stream fresh-stage flush (ADR-003 option (d), three numbered rules: flush-and-continue, derivation over the eventual stored artifact as a whole via buffer form and StagingCursor read-back form, zero residue on stream error) - staged-put seam amended (operator sign-off): `put_staged()` keyless handout, `StagingCursor::commit(self, key)` publishes under the store-core-resolved key — the canonical key completes at stream end and the wave-1 handout shape could not express the commit-rename flow - pin-before-publish rides the arbitration lock on both paths and both tiers; dedup re-puts refcount the existing entry - Batch seam: `Store::batch()` + `put_in_batch()`, members publish pin- before-publish, group pins drop together on `Batch::commit` (new public verb) or drop - kv-only rejection (ADR-012 §6.2): known-length over-threshold rejects immediately (stream never read), unknown-length at mid-stream overflow; zero residue; error carrier `Io(String)` (named design point) - declared-len mismatch rejects `Verification` zero-residue (named design point); stream shape generic-no-dyn (named design point) - incremental BlobHasher (pinned byte-equal to Algorithm::derive); AnyStaging cursor dispatch; git-interop put-path leg against git hash-object; eleven put-path behavior tests Verification: cargo test (default/--features mem/--all-features) green; clippy --all-targets -D warnings green in all three feature shapes; cargo fmt --check green; cargo doc --no-deps at the 24-warning baseline.
This commit is contained in:
1 parent
889be4cada
commit
539dabcbf7
8 files changed
+1723
-105
No files matched your search
+110
-10
@@ -1,7 +1,7 @@
|
||||
---
|
||||
id: store-put-paths
|
||||
name: Store put seam — two internal paths, the three numbered rules, Batch, kv-only rejection
|
||||
status: pending
|
||||
status: completed
|
||||
depends_on: [gc-protocol]
|
||||
scope: moderate
|
||||
risk: medium
|
||||
@@ -67,22 +67,22 @@ internal paths behind one signature, with pin-before-publish from the
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `put(len, stream) → (Key, Pin)` with both internal paths behind
|
||||
- [x] `put(len, stream) → (Key, Pin)` with both internal paths behind
|
||||
the one signature; routing is the pure length function (no
|
||||
content ever migrates tiers)
|
||||
- [ ] The three numbered rules each behavior-tested: mid-stream
|
||||
- [x] The three numbered rules each behavior-tested: mid-stream
|
||||
overflow flushes into a fresh stage and the stream continues;
|
||||
the derivation pass runs over the eventual stored artifact as a
|
||||
whole (buffer form and staged-bytes form); stream error leaves
|
||||
no entry and no stage file
|
||||
- [ ] Pin-before-publish holds on both paths and both tiers; dedup
|
||||
- [x] Pin-before-publish holds on both paths and both tiers; dedup
|
||||
re-puts refcount the existing entry
|
||||
- [ ] `Batch` groups multi-put writes; member pins drop together
|
||||
- [ ] kv-only mode: known-length over-threshold rejects immediately;
|
||||
- [x] `Batch` groups multi-put writes; member pins drop together
|
||||
- [x] kv-only mode: known-length over-threshold rejects immediately;
|
||||
unknown-length rejects at mid-stream overflow; zero residue on
|
||||
both
|
||||
- [ ] Stream-shape and len-mismatch design points decided and recorded
|
||||
- [ ] `cargo test` green; `cargo test --features mem` and `cargo test
|
||||
- [x] Stream-shape and len-mismatch design points decided and recorded
|
||||
- [x] `cargo test` green; `cargo test --features mem` and `cargo test
|
||||
--all-features` green; clippy/fmt gates green
|
||||
|
||||
## References
|
||||
@@ -96,8 +96,108 @@ internal paths behind one signature, with pin-before-publish from the
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
- **The staged-put seam was amended: the key resolves at commit, not
|
||||
at handout.** Implementing the two large-tier flows surfaced a
|
||||
contradiction in the wave-1 pin: `put_staged(key)` hands the staging
|
||||
cursor out with the key, but both large-tier put paths only know the
|
||||
canonical key when the derivation completes — at stream end (the
|
||||
preamble covers a length, and the digest covers content that has
|
||||
not streamed yet). Content streams once (non-restartable sources are
|
||||
the normal case), so `commit(self)` publishing under the handout
|
||||
key cannot express the specced flow ("the derivation completes at
|
||||
stream end; commit-rename publishes"). Amended with operator
|
||||
sign-off: `put_staged()` is a keyless handout and
|
||||
`StagingCursor::commit(self, key)` publishes under the
|
||||
store-core-resolved key (CAS first-writer-wins as the whole-value
|
||||
put). No ADR text pins the key's arrival point (ADR-003/010 pin
|
||||
handout/commit/discard mechanics only); the wave-1 task notes'
|
||||
`put_staged(key)` shape was pre-flow realism. Recorded for
|
||||
review-wave-2. Engines updated in place (mem, local, the wave-1
|
||||
test double) with their hygiene suites re-run green.
|
||||
- **Put-path named design points (recorded in `src/store.rs` module
|
||||
docs, for review-wave-2):**
|
||||
- *Stream parameter shape:* `put(len, stream: impl AsyncRead + Unpin
|
||||
+ Send)` — generic, no boxing; the family's no-`dyn` posture
|
||||
carried to the write seam (a boxed reader is itself `AsyncRead +
|
||||
Unpin + Send` under tokio's blanket impl, so callers needing
|
||||
erasure box at their call site). `Send` keeps spawn-site bounds
|
||||
resolvable for the ops wave; no `'static` (nothing spawns).
|
||||
- *Declared-len mismatch:* a known-length stream yielding fewer or
|
||||
more bytes than `len` rejects `Error::Verification` on either
|
||||
tier, zero residue either side — the preamble carries the
|
||||
*declared* length, so a mismatched artifact's canonical address
|
||||
can never equal the stored key (storing it would break pool
|
||||
integrity by construction; `Verification` is the put-path
|
||||
content-≠-key family member, ADR-006's whole-blob posture applied
|
||||
at put time).
|
||||
- *kv-only rejection carrier (a named decision beyond the two the
|
||||
task listed):* `Error::Io(String)` with a descriptive message —
|
||||
the ADR-pinned family (ADR-012 §6.5) holds no dedicated variant
|
||||
and the rejection is an operational/capability fact about the
|
||||
medium posture, the bucket `Io(String)` covers. A dedicated
|
||||
variant for typed matchability would be a new failure family —
|
||||
a new ADR.
|
||||
- **Flows as implemented** (`src/store.rs`, write-path `impl Store`):
|
||||
known-length initializes the derivation with the preamble before
|
||||
content flows and streams once (kv: buffer + hash in one pass →
|
||||
mint pin → whole-value put; large: stream into the staging cursor
|
||||
while hashing → mint pin → `commit(key)`). Unknown-length buffers
|
||||
to the threshold; a mid-stream overflow (strictly past it —
|
||||
exactly-at stays kv) flushes the buffered prefix into a fresh stage
|
||||
and the stream continues into it; at EOF the derivation pass runs
|
||||
over the eventual stored artifact as a whole — the buffer (kv form)
|
||||
or preamble + staged bytes read back through the
|
||||
`StagingCursor: ReadCursor` seam (staged form) — then pin, then
|
||||
commit. Every failure path (read error, mismatch, under-read)
|
||||
discards: no entry, no stage file.
|
||||
- **API shapes:** `Store::put(&self, len: Option<u64>, stream) →
|
||||
Result<(Key, Pin), Error>`; the batching seam is `Store::batch() →
|
||||
Batch` + `Store::put_in_batch(&mut Batch, len, stream) →
|
||||
Result<Key, Error>` (members run the lone-put flow; the minted guard
|
||||
is contributed to the group via crate-internal `Batch::add`); a
|
||||
public `Batch::commit()` was added consuming the batch with the
|
||||
same one-step release as drop (store-api.md's "pins drop together
|
||||
on batch commit or drop", as a named verb). `hashing.rs` gained
|
||||
crate-internal `BlobHasher` — the same derivation driven
|
||||
incrementally, pinned byte-equal to `Algorithm::derive` by tests
|
||||
(chunk-boundary invariance, both algorithms, the git vectors).
|
||||
- **git-interop leg added** (`tests/git_interop.rs`): the put seam
|
||||
keys entries git-identically through both put paths on both tiers
|
||||
against live `git hash-object` (INV-1's source-of-truth posture
|
||||
extended from the derivation constructor to the facade seam).
|
||||
- **Wave-1 note updates:** `put_staged`/`commit` doc text in
|
||||
`backend.rs`, `mem.rs`, `local.rs` now carries the key-at-commit
|
||||
design point; the store.rs module doc's remaining-deferral note
|
||||
shrunk to the actual remaining gaps (fleet clauses, wave-3 kv
|
||||
engine).
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
Implemented the facade's put seam (`src/store.rs`): `put(len:
|
||||
Option<u64>, stream: impl AsyncRead + Unpin + Send) →
|
||||
Result<(Key, Pin), Error>` with the two internal paths behind the one
|
||||
signature — the known-length one-pass flow and the unknown-length
|
||||
buffer-then-maybe-flush flow implementing ADR-003's option (d) three
|
||||
numbered rules — plus `batch()`/`put_in_batch()` (ADR-005 §Marks 2's
|
||||
batch-scoped pin group) and kv-only's put-time rejection at its two
|
||||
pinned points (ADR-012 §6.2). Pin-before-publish rides
|
||||
`gc().acquire` under the arbitration lock before both tiers'
|
||||
publishes; dedup re-puts refcount the existing entry; every failure
|
||||
path converges on discard with zero stage files. The staged-put seam
|
||||
was amended (keyless handout; `commit(self, key)`) after operator
|
||||
sign-off — the one blocking finding of the task. Supporting changes:
|
||||
staging-cursor dispatch enum `AnyStaging` beside the read path's
|
||||
`AnyCursor`; incremental `BlobHasher` in `hashing.rs`; public
|
||||
`Batch::commit`; a `Pin`/`Debug` impl; the git-interop put-seam leg.
|
||||
Eleven put-path tests in `src/store.rs` (plus four fixtures) pin the
|
||||
three rules, both-tier round-trips, the threshold boundary, pin-
|
||||
before-publish under a racing sweep, dedup refcounting across paths
|
||||
and tiers, batch grouping/commit/failure, kv-only's two rejection
|
||||
points, len-mismatch zero-residue, mem-only's threshold-free posture,
|
||||
and the empty-content degenerate.
|
||||
|
||||
Verification: `cargo test` (default), `cargo test --features mem`,
|
||||
`cargo test --all-features` all green (97 lib tests under mem plus
|
||||
conformance and git-interop); `cargo clippy --all-targets --
|
||||
-D warnings` green in all three feature shapes; `cargo fmt --check`
|
||||
green; `cargo doc --no-deps` at the pre-existing 24-warning baseline.
|
||||
Reference in new issue
Block a user