diff --git a/docs/plans/implementation.md b/docs/plans/implementation.md index f119539..2124e8f 100644 --- a/docs/plans/implementation.md +++ b/docs/plans/implementation.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: "2026-10-12 (wave 6 decomposed into tasks/)" +last_updated: "2026-10-12 (wave 7 decomposed into tasks/)" --- # alkblobs — Implementation plan @@ -54,7 +54,7 @@ ops payload surfaces; release readiness is last. | [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | decomposed (2026-10-12) | | [5](#wave-5--pg-lo-large-engine) | pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column | wave 4 | decomposed (2026-10-12) | | [6](#wave-6--fleet-gc) | Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations | waves 4 + 5 | decomposed (2026-10-12) | -| [7](#wave-7--ops-surface) | `ops` feature: alkcall-backed op family, ACL mapping, pin tokens + renewal, fanout seam | wave 6 | pending | +| [7](#wave-7--ops-surface) | `ops` feature: alkcall-backed op family, ACL mapping, pin tokens + renewal, fanout seam | wave 6 | decomposed (2026-10-12) | | [8](#wave-8--fuzzing) | Fuzzing: family pattern targets (key encoding, ops payloads), corpus-replay gate | wave 7 | pending | | [9](#wave-9--release-readiness) | Crate docs, feature-graph verification, README, publish prep | wave 8 | pending | diff --git a/tasks/ops-call-harness.md b/tasks/ops-call-harness.md new file mode 100644 index 0000000..b391ec5 --- /dev/null +++ b/tasks/ops-call-harness.md @@ -0,0 +1,85 @@ +--- +id: ops-call-harness +name: the in-process alkcall harness — the serving assembly and the client side over one duplex pair +status: pending +depends_on: [ops-foundation] +scope: narrow +risk: low +impact: component +level: implementation +tags: [wave-7, ops, testing, harness, alkcall] +--- + +## Description + +The test harness every ops handler task exercises its handlers +through: an in-process alkcall **serving side + client side over one +`tokio::io::duplex` pair** — the embedder's "dials and hands an +established `Connection` in" shape (ops-surface.md §Why alkcall; +alkcall is a pure protocol crate), realized in-process so the ops +surface is testable server-less. + +- **The serving assembly** (the embedder-assembly pattern realized in + test): `OperationRegistryBuilder` receiving the ops family's + spec/handler pairs → the dispatch serving stack (`CallAdapter` / + `GatewayDispatch`, plus `ChannelsAdapter` for the binary ops' + `channel_open` path) → the dispatch run loop over a + `Connection::from_bidi` half (alkcall ADR-009's public stream + constructor). The assembly's shape is the module's documented + embedder entry — the "module exports spec/handler pairs; the + embedder assembles" placement (ops-surface.md's placement note, + the alkgit `git/repo/*` pattern). +- **The client side**: `CallClient` + `spawn_dispatch` over the + duplex pair's other half; `ChannelClient` (`from_connection`, + `open_channel`/`call_open_op`) for the channel ops. +- **The identity/ACL wiring**: a test `IdentityProvider` + + `OwnershipProvider` — the embedder-registry stand-in (namespace → + identities → granted actions) — so ACL behavior is exercisable from + the first handler task onward, including the closed-by-default + denial path (OQ-09). +- **Proven with a stub op**: a trivial stub op registered through the + same path round-trips (call → response envelope), and the channel + path opens with bytes flowing — the pipe proven before any real op + rides it. The stub lives in the test tree; it registers nothing + into the shipped module surface. + +## Acceptance Criteria + +- [ ] The harness lives in the ops module's test tree + (`#[cfg(test)]` under feature `ops`), parameterized over the + assembled registry (ops register; the harness serves) +- [ ] Server + client halves over one duplex pair; the dispatch run + loop serves; the client calls and receives envelopes +- [ ] The identity/ownership wiring exercisable: a stub op carrying + an `AccessControl` denies an ungranted identity and admits a + granted one (the ACL path proven end-to-end pre-handler) +- [ ] The channel path proven: a stub `channel_open` op establishes + (the `Establishment` phase) and bytes flow both directions +- [ ] `cargo test --features ops` green; the default build unaffected + (the harness is feature-gated test code); + `cargo clippy --all-targets -- -D warnings` and + `cargo fmt --check` green on both shapes + +## References + +- docs/architecture/ops-surface.md (§Why alkcall — the Connection + hand-in; the placement note) +- docs/architecture/decisions/001-substrate-posture-and-ops-placement.md + §2 (the machinery inventory this harness assembles) +- alkcall (upstream checkout `/workspace/@alkdev/alkcall`): + `src/registry/registration.rs` (`OperationRegistryBuilder`, the + handler kinds), `src/protocol/adapter.rs` (`CallAdapter`), + `src/protocol/connection.rs` (`Connection::from_bidi`), + `src/channels/adapter.rs` (`ChannelsAdapter`), + `src/channels/client.rs` (`ChannelClient`), ADR-047/049 (the + channel_open marker + establishment) +- tasks/ops-foundation.md (the context the harness's assembled + handlers capture) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/ops-e2e-flows.md b/tasks/ops-e2e-flows.md new file mode 100644 index 0000000..7f3b095 --- /dev/null +++ b/tasks/ops-e2e-flows.md @@ -0,0 +1,115 @@ +--- +id: ops-e2e-flows +name: the ops wire-level lifecycle proofs — have/need diffing, the token lifecycle, and the ACL matrix end-to-end +status: pending +depends_on: [ops-json-ops, ops-fetch, ops-put] +scope: broad +risk: medium +impact: phase +level: implementation +tags: [wave-7, ops, e2e, tokens, acl, fleet] +--- + +## Description + +The capstone proofs through the harness (`ops-call-harness`) — the +full lifecycle flows the individual handler tests exercise only in +pieces. This is wave 7's equivalent of the engine waves' suite +columns: the surface proven as a whole, on the wire. + +- **The have/need flow**: a consumer diffs `blobs/have` over its + digest set → fetches the absent digests (the need announcement *is* + the fetch miss — the diff is caller-side; the wire carries only + concrete fetch requests). Proven end-to-end: seed a store via the + facade, diff from the client side, fetch exactly the absent set, + re-diff to all-present. +- **The fleet token lifecycle end-to-end** (against the harness + server — env-driven, skipping cleanly server-less; the + pglo-poc convention): put → token → renew via have's renewal form + before expiry (`present: true`; the row's expiry observed moving) → + the embedder's registration confirms → the row's protection hands + to the registered liveness source (the namespace binding's + converter) → the row expires naturally and the entry survives on + the registered source's protection. Then the failure half: pause + past TTL → the renewal probe answers `present: false` → the + sweeper reaps the row → the **re-put backstop** (delete-then- + recover; CAS makes the re-put idempotent) → **the race resolves to + a re-put, never to silent loss** (ADR-010 §4's expiry semantics + verbatim, on the wire). +- **The foreign-token refusal**: a token minted by node A presented + to node B (a second fleet instance's ops module over the same + pool) → `present: false`, not an error — the validity domain is + the minting node (ADR-012 §2). +- **The non-fleet token lifecycle**: put → token (the registry holds + the guard) → have-present answers from the custody → the embedder's + registration confirms → the guard converts (the namespace + binding's converter) → the token retires. Proven on a non-fleet + store (sqlite or mem — server-less). +- **The ACL matrix end-to-end**: an unlisted namespace denies read + AND write (closed-by-default — OQ-09); a read-granted identity + fetches/stats/haves but cannot put; a write-granted identity puts + but a read-only peer cannot fetch its namespace; delete only under + the internal authority context (never a namespace grant). +- **The re-routing posture's serving half** (ADR-008 §3): a large + (pg-lo) entry served through `blobs/fetch` from the storage node's + dual-tier fleet store — the shape client nodes in kv-only mode + consume; no fleet-specific op exists (the module supports the shape + without change). +- **The no-side-door audit** (structural, ADR-001 §4): the ops + module's store access enumerated — the public facade plus the + foundation's crate-internal renewal seam only; no `Backend` touch, + no engine types, no path around the facade. Recorded as the audit + the wave-7 review gate re-checks. +- **The renewal-cadence boundary honored** (ADR-013 §1): the tests + drive renewal explicitly (the embedder's cadence role played by the + test); no in-crate renewal worker exists. + +## Acceptance Criteria + +- [ ] The have/need flow proven end-to-end (diff caller-side; the + wire carries only concrete fetches) +- [ ] The fleet token lifecycle end-to-end vs the harness server: + renew-before-expiry (the row's expiry moves), the + registration-handover, pause-past-TTL → reap → the re-put + backstop — never silent loss (env-driven; skipping cleanly + server-less; acceptance requires green against the server) +- [ ] The foreign-token refusal: `present: false`, not an error +- [ ] The non-fleet lifecycle: custody → confirmation → conversion → + token retirement (server-less) +- [ ] The ACL matrix end-to-end (closed-by-default read+write denial; + the read/write split; delete internal-only) +- [ ] The re-routing posture's serving half: a large entry fetched + through the ops surface from the dual-tier fleet store +- [ ] The no-side-door audit recorded (the store-access enumeration) +- [ ] Gates green: `cargo test` (default), `cargo test --features + ops`, `cargo test --all-features` (the fleet rows live at + acceptance), clippy/fmt across the variants, `cargo doc + --no-deps` clean + +## References + +- docs/architecture/ops-surface.md (§Op family — the hand-over + contract; the need-half note; §Store-facing relationship — the + re-routing posture) +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md + §4 (the expiry semantics; the re-put backstop) +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md + §2 (the validity domain; the non-fleet generalization) +- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md + §2/§3 (the TTL/3 cadence; the re-routing topology) +- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md + §1 (the cadence is embedder composition) +- docs/architecture/open-questions.md OQ-09 (closed-by-default) +- tasks/ops-foundation.md, tasks/ops-call-harness.md, + tasks/ops-json-ops.md, tasks/ops-fetch.md, tasks/ops-put.md (the + surface this proves as a whole) +- tasks/suite-column-fleet.md (the env-driven harness-server + convention this task's fleet rows follow) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/ops-fetch.md b/tasks/ops-fetch.md new file mode 100644 index 0000000..a024fd7 --- /dev/null +++ b/tasks/ops-fetch.md @@ -0,0 +1,101 @@ +--- +id: ops-fetch +name: blobs/fetch — the verified Sub op and the fanout seam's consumer side +status: pending +depends_on: [ops-foundation, ops-call-harness] +scope: moderate +risk: medium +impact: component +level: implementation +tags: [wave-7, ops, fetch, fanout, verification] +--- + +## Description + +`blobs/fetch` (ops-surface.md §Op family) — the server→client +streaming op: `{namespace, digest, ranges?}` in; **verified bytes +out** over the binary data channel. + +- **The op shape**: `OperationType::Sub` (alkcall ADR-021's streaming + handler kind) with the **`channel_open` marker** (alkcall ADR-047) + — the binary data plane rides a `channel_open`-allocated channel; + the establishment phase (alkcall ADR-049) carries the fetch's ack + shape in the `Establishment` reply fields as the wire requires. The + **read** action (ACL exactly like stat/have — closed-by-default). +- **Verified streaming** (ADR-006; ops-surface.md): the *consumer* + hash-checks the received whole against the carried digest — + whole-blob verification; there are no per-chunk or per-range checks + derivable under the canonical digest. The handler streams from the + facade only: `Store::get` (the whole) or `Store::read_range` (the + ranged form). **Ranged fetch** (`ranges?`): the response carries + the same ADR-006 slice-digest convention `read_range` returns — + SHA-256 over exactly the returned bytes, out-of-band. +- **The fanout seam's consumer side** (ops-surface.md; POC #3 finding + A3): the fetch handler consults the foundation's fanout registry — + a live broadcast for the digest → a **subscriber arm** taps it; no + live broadcast, or one that ended (a **late joiner**) → the normal + post-commit verified `get` — identical bytes under CAS. The + slow-subscriber policy (drop-and-late-join) is ops-layer policy, + not a store concern. Named design point: **the producer side's + shape** — the put handler's stream reader as the canonical producer + (serve-while-persisting, A3's shape), and whether fetches also + produce read-side broadcasts (concurrent fetchers of one stored + digest sharing one store read) — decide and record against the A3 + text. +- **The miss shape**: a fetch of an absent digest is the typed + missing-error schema (the need announcement *is* the fetch miss — + the have/need diff is caller-side; no separate need op exists). +- **Tests through the harness**: the whole-blob verified round-trip + (byte-exact; the client's hash-check passes); the ranged form + (bytes + slice digest matching `read_range`'s convention); the + fanout consumer side (a live broadcast tapped — the subscriber + receives while the producer runs; a late joiner degrades to the + get path — asserted by which path served); the miss shape; the ACL + denial (no read grant on the namespace → denied). + +## Acceptance Criteria + +- [ ] fetch exported as a Sub spec/handler pair with the + `channel_open` marker + ALPN; the establishment reply fields + carry the specced ack shape +- [ ] Whole-blob verification: the received whole hash-checks against + the carried digest (the consumer-side discipline documented on + the op; the server streams verified-store bytes) +- [ ] Ranged fetch carries the ADR-006 slice-digest convention + (out-of-band, over exactly the returned bytes) +- [ ] The fanout consumer side: a live broadcast → subscriber arm; a + late joiner → the verified get path; the producer-side design + point decided and recorded +- [ ] The miss shape is the typed missing error; the have/need diff + stays caller-side (no need op invented) +- [ ] ACL: the read action; closed-by-default verified by test +- [ ] Handlers touch only the store facade (no backend side-door) +- [ ] Harness tests green; `cargo test` (default) green; + `cargo test --features ops` green; clippy/fmt green on both + shapes; `cargo doc --no-deps` clean + +## References + +- docs/architecture/ops-surface.md (§Op family — the fetch bullet + + the fanout text; the need-half note) +- docs/architecture/decisions/006-verification-posture-and-transfer-encoding.md + (whole-blob verification; the slice-digest convention) +- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md + §3 (the re-routing topology this op serves — large-blob access + through a storage node) +- docs/research/poc-largeblob-findings.md finding A3 (the fanout + seam: one reader, store arm + subscriber arms; late joiners + degrade to get-after-commit) +- alkcall (upstream checkout): ADR-021 (streaming handler for + subscriptions), ADR-046/047/049 (the sink kind, the channel_open + marker, the establishment phase), `src/channels/client.rs` +- tasks/ops-foundation.md (the fanout registry's mechanics), + tasks/ops-call-harness.md (the harness) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/ops-foundation.md b/tasks/ops-foundation.md new file mode 100644 index 0000000..8bc7032 --- /dev/null +++ b/tasks/ops-foundation.md @@ -0,0 +1,160 @@ +--- +id: ops-foundation +name: the ops module's substrate — the token layer, the namespace binding, and the fanout registry +status: pending +depends_on: [] +scope: moderate +risk: medium +impact: component +level: implementation +tags: [wave-7, ops, tokens, fanout, namespaces] +--- + +## Description + +The `ops` feature's state substrate in `src/ops.rs` (currently the +ADR-001 stub doc comment) — everything the five op handlers ride, and +none of the handlers themselves (they land in the following tasks). +The module is a **store-facade consumer** (ADR-001 §4): its store +access is the public facade plus the crate-internal renewal seam named +below — no backend side-door, no engine types. + +- **Module structure + the ops context**: the feature-gated module + shape (`#[cfg(feature = "ops")]`, re-exported from `lib.rs` per the + one-module-per-file convention); the ops-layer context type binding + the `Arc`, the module's state (the token registry, the + namespace bindings, the fanout registry), and the module config. + This context is what the handler pairs (later tasks) capture. +- **The pin-token layer** (ADR-010 §4; ADR-012 §2): opaque token + minting over a landed put's pin; the **token↔(key, owner) in-memory + mapping with process lifetime** (outlives the minting handler — + ADR-012 §2's correction of the original handler-lifetime wording); + the **validity domain is the minting node** — the registry is + per-process state, there is no cross-node form, and fleet peers + refuse tokens they did not mint (`present: false`, not an error). + Named design points: **the token's minting form** (a CSPRNG id — + e.g. `getrandom` gated behind `ops`; a counter or derivable form is + rejected as a guessable capability — decide and record), and **the + mapping's retention discipline** (when a mapping retires: + conversion-confirmed or process end; the mapping carries no expiry + machinery of its own — the pin row's expiry is the fleet liveness + signal, and nothing expires non-fleet). +- **The guard-custody design** (the fleet/non-fleet split ADR-010 §4 + pins): under fleet, the put handler's `Pin` guard **drops at + response completion** — the pin row's expiry is the only protection + from that moment, which is exactly why the TTL/3 renewal duty + exists (a guard held for the token's whole life would make the TTL + machinery dead code on the ops path); under non-fleet, the token + registry **holds the guard** — the token is the in-process guard's + handle (ADR-012 §2's generalization: sqlite, single-instance + postgres, pg-lo, `local`, `mem` — nothing expires outside fleet + mode) until the embedder's registration confirms and the guard + converts. The custody split is the token layer's core branch. +- **The namespace→liveness-source binding** (ops-surface.md §Op + family, hand-over contract step 3): the embedder's converter seam — + per-namespace registration of the converter the embedder plugs its + registry into ("A `Pin` conversion on the server side (token → + registered liveness source) is the mechanism the embedder plugs its + registry into"). The conversion rides the crate-internal + `RegistryCore::convert` machinery (wave 2). Named design point: ** + the conversion trigger** (an embedder-called confirm verb vs a + probe-triggered conversion — decide and record; the hand-over + contract's duty text is the embedder's, the mechanism is this + module's). +- **The crate-internal store-core renewal access** (ADR-010 §4: "the + ops module is in-crate and crosses the seam by construction, the + same way the put handler holds the `Pin` guard"): the renewal + probe's access to `pin_renew(key, owner)` and `pin_check(key)` + through the fleet host — the `pin_check` deliberately-unconsumed + note from the wave-6 review (W6-3) names this consumer. The exact + probe composition (verify-the-mapping-then-renew vs + renew-answers-the-verification) is a named design point against + ADR-012 §2's three-step handler duty. What never happens: a * + consumer* reaching the seam, or a public-API widening for this. +- **The fanout registry** (ops-surface.md's fetch fanout seam; POC #3 + finding A3): per-digest broadcast state — one reader, store arm + + subscriber arms; bounded subscriber channels (the backpressure + discipline: a slow subscriber stalls the broadcast, which is + correct for a put — subscriber count bounded, slow-subscriber + handling is policy); the **late-joiner signal** (broadcast ended → + the degrade-to-verified-get path); the **drop-and-late-join** + slow-subscriber policy as ops-layer policy. The producer wiring + lands with `ops-put`, the consumer wiring with `ops-fetch`; the + registry's mechanics land here, unit-tested standalone. +- **The renewal-cadence boundary** (ADR-013 §1): the module ships the + renewal *mechanism* (the have form's handler); the TTL/3 *cadence* + — the putter's renewal worker — is embedder composition above the + facade (scheduler/queues), never in-crate machinery. Documented on + the module, not implemented. +- **Dependency posture**: `serde`/`serde_json` (payload handling) and + the token CSPRNG enter as `ops`-gated dependencies — the base crate + stays lean (convention 8; ADR-013 §4's engine table unchanged — + `ops` pulls alkcall plus these, default-off). +- **Unit tests**: token mint/lookup/retirement and the mapping's + process-lifetime semantics; the custody split (the fleet and + non-fleet forms branch as designed); the namespace binding's + converter invocation; the fanout registry's subscribe / + late-joiner-degrade / slow-subscriber-drop mechanics. + +## Acceptance Criteria + +- [ ] `src/ops.rs` carries the module structure under feature `ops`: + the ops context, the token layer, the namespace binding, the + fanout registry — no op handlers yet (they land in + ops-json-ops/ops-fetch/ops-put) +- [ ] The token layer: opaque, unguessable minting (the CSPRNG design + point decided and recorded); the token↔(key, owner) mapping with + process lifetime; validity domain = the minting node (a + per-process registry; no cross-node form exists) +- [ ] The guard-custody design decided and recorded: fleet — the + guard drops at response completion (the row's expiry is the + liveness signal); non-fleet — the registry holds the guard + (nothing expires) until conversion +- [ ] The namespace→liveness-source binding: per-namespace converter + registration riding `RegistryCore::convert`; the conversion + trigger design decided and recorded +- [ ] The crate-internal renewal access: the ops module reaches + `pin_renew`/`pin_check` through the fleet host (in-crate seam + crossing; no public-API widening; no backend side-door) +- [ ] The fanout registry: per-digest broadcast, bounded subscriber + channels, the late-joiner degrade signal, the + drop-and-late-join policy point recorded +- [ ] New dependencies (`serde`, `serde_json`, the CSPRNG) are + `ops`-gated; the default build's dependency tree is unchanged +- [ ] Unit tests green; `cargo test` (default) green with the module + compiled out; `cargo test --features ops` green; + `cargo clippy --all-targets -- -D warnings` and + `cargo fmt --check` green on both shapes; `cargo doc --no-deps` + clean + +## References + +- docs/architecture/ops-surface.md (§Op family — the hand-over + contract; §ACL mapping; §Store-facing relationship) +- docs/architecture/decisions/001-substrate-posture-and-ops-placement.md + §3/§4 (placement; the facade-consumer rule) +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md + §4 (the token contract: minting, custody, validity domain, the + in-crate seam crossing) +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md + §2 (the wire shapes: {token, digest}; the have renewal form; the + validity domain; the non-fleet generalization) +- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md + §1 (the renewal cadence is embedder composition), §4 (the feature + graph) +- docs/research/poc-largeblob-findings.md finding A3 (the fanout seam + shape) +- src/ops.rs (the ADR-001 stub this task fills) +- src/gc.rs (`EngineFleetState::pin_renew`/`pin_check` — the seam the + renewal access crosses; `RegistryCore::convert`) +- src/store.rs (`Store::gc()` — the crate-internal access path) +- tasks/review-wave-6.md (W6-3: `pin_check` deliberately unconsumed — + this wave's renewal probe is the named consumer) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/ops-json-ops.md b/tasks/ops-json-ops.md new file mode 100644 index 0000000..d2ccf30 --- /dev/null +++ b/tasks/ops-json-ops.md @@ -0,0 +1,119 @@ +--- +id: ops-json-ops +name: the JSON control plane — blobs/stat, blobs/have (+ the token-renewal form), blobs/delete +status: pending +depends_on: [ops-foundation, ops-call-harness] +scope: moderate +risk: medium +impact: component +level: implementation +tags: [wave-7, ops, json, acl, tokens] +--- + +## Description + +The three JSON control ops (ops-surface.md §Op family), each exported +as a spec/handler pair the embedder assembles — riding alkcall's +`OperationSpec` + JSON Schema validation (alkcall ADR-016) and its ACL +machinery at op entry (convention 6: this module never re-implements +an authorization check): + +- **`blobs/stat`** `{namespace, digest}` → `{len}` — Query; the + **read** action; the handler calls `Store::stat` (the fall-through + general rule, ADR-012 §6.1); a miss is the typed missing-error + schema (not a `{len: null}`). +- **`blobs/have`** `{namespace, digests[], token?}` → `{present[]}` — + Query; **read-gated** (an existence probe over arbitrary digests is + a discovery surface — gated exactly like fetch, not public); + `present[]` corresponds to `digests[]` by index; content never + traverses this op (hashes only). **The renewal form** (ADR-012 §2): + when `token` is present, exactly one digest accompanies it — the + handler verifies the token maps to that digest's pin row (the + foundation's token layer), calls the crate-internal `pin_renew(key, + owner)` via the foundation's store-core access, and answers + `present[]` as usual; an expired or foreign row answers + `present: false` and no renewal occurs — the putter's backstop is + re-put. Without `token`, the pure existence probe (ADR-008's `has` + discipline) over the facade's `has`. **The non-fleet renewal form**: + the token is the held guard's handle — the probe answers present + from the custody (nothing expires; ADR-012 §2's generalization); + the custody/conversion lifecycle is the foundation's. +- **`blobs/delete`** `{namespace, digests[]}` — Mutation; + **`Visibility::Internal`**, evaluated under the internal authority + context per alkcall ADR-017 (internal calls switch authority + context, never skip ACL): requires the embedder's operator/admin + authority context, **not a namespace grant**; the `namespace` field + exists for audit/triage scoping, not gating. The handler calls + `Store::delete` (the arbitration-then-delete path; a `GcRefuse` + refusal surfaces as the typed error schema). External discovery of + an Internal op returns NOT_FOUND (alkcall ADR-017's visibility + discipline). +- **The ACL mapping** (ops-surface.md §ACL mapping; OQ-09's + resolution): **namespace = resource** — `resource_type: + "blob-namespace"`, `resource_id_path` selecting the payload's + `namespace` field; stat/have = the **read** action; **unlisted + namespaces deny by default** (closed-by-default: no ACL entry ⇒ no + read, no write — a grant per namespace is the deferred cost, + deliberately paid over accidental exposure). +- **Schemas** (alkcall ADR-016): input/output JSON Schema per op + + typed error schemas; the namespace field's syntax (non-empty UTF-8 + string, embedder-validated — the minting of namespaces is the + embedder's act); the digest wire form is **hex** (the canonical + digest hex — the same form `blobs/put`'s response returns; parsed + via the key layer's hex ingress, malformed → the typed invalid + error, rejects-never-guesses). +- **Tests through the harness** (`ops-call-harness`): stat round-trip + + the miss shape; have's index correspondence (present/absent + mixed); the renewal form's three answers (live row renews → + present: true; expired row → present: false, no renewal; foreign + token → present: false); delete under the internal authority + context (an external caller gets NOT_FOUND; an internal + operator-context call deletes); the closed-by-default ACL matrix + (an unlisted namespace denies read; a granted identity reads). + +## Acceptance Criteria + +- [ ] Three spec/handler pairs exported (stat, have, delete); input/ + output schemas + typed error schemas per alkcall ADR-016 +- [ ] The ACL mapping: namespace-as-resource with the + resource_id_path; the read action on stat/have; + closed-by-default verified by test (unlisted namespace denies) +- [ ] delete: `Visibility::Internal`, the internal-authority-context + requirement (not namespace-gated), external discovery NOT_FOUND +- [ ] have's renewal form: one digest + token; verify-mapping → + `pin_renew`; expired/foreign → `present: false` with no + renewal; the non-fleet form answers from the held custody +- [ ] The handlers touch only the store facade + the foundation's + crate-internal renewal access (no backend side-door — the + ADR-001 §4 rule) +- [ ] Harness tests green; `cargo test` (default) green; + `cargo test --features ops` green; clippy/fmt green on both + shapes; `cargo doc --no-deps` clean + +## References + +- docs/architecture/ops-surface.md (§Op family — the three JSON ops; + §ACL mapping) +- docs/architecture/decisions/001-substrate-posture-and-ops-placement.md + §2/§4 +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md + §2 (the have payload shape + the renewal form's three-step handler + duty) +- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md + (the `has` discipline the non-token form preserves) +- docs/architecture/open-questions.md OQ-09 (closed-by-default, the + risk-asymmetry rationale) +- alkcall (upstream checkout): ADR-016 (operation error schemas), + ADR-017 (the privilege model — internal switches context, never + skips ACL), `src/registry/spec.rs` (`AccessControl`, + `resource_id_path`) +- tasks/ops-foundation.md (the token layer + the renewal access the + have handler rides), tasks/ops-call-harness.md (the harness) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/ops-put.md b/tasks/ops-put.md new file mode 100644 index 0000000..33be5eb --- /dev/null +++ b/tasks/ops-put.md @@ -0,0 +1,128 @@ +--- +id: ops-put +name: blobs/put — the verified Sink op, the pin hand-over, and the {token, digest} response +status: pending +depends_on: [ops-foundation, ops-call-harness] +scope: moderate +risk: high +impact: component +level: implementation +tags: [wave-7, ops, put, tokens, hand-over, verification] +--- + +## Description + +`blobs/put` (ops-surface.md §Op family) — the client→server streaming +op: `{namespace, digest?, len?}` offer + the byte stream in; the +server verifies against the canonical derivation **before commit**; +the response is **`{token, digest}`** (ADR-012 §2). + +- **The op shape**: the Sink handler kind (alkcall ADR-046's + `Pub`/Sink — the client→server streaming shape) with the + **`channel_open` marker** (alkcall ADR-047); the establishment + phase carries the offer ack. The **write** action (ACL; + closed-by-default). +- **Verification before commit** (ADR-006; ops-surface.md): the + server hash-checks the received stream against the canonical + derivation (`H("blob \0" + content)`) before commit. A + **known-length offer** (`len` present) is the encouraged one-pass + path; an **unknown-length offer** rides the store's pre-threshold + buffering path (ADR-003's option (d) semantics — the facade's + `put(None, …)` form, the mid-stream stage flush and the + restart-over-staged-bytes derivation pass riding unchanged). When + the offer carries `digest?`, the derived key is checked against it + — a mismatch rejects before commit (the offered digest is a + claim; the derivation is the authority). +- **The pin hand-over** (ops-surface.md §Pin hand-over; ADR-010 §4): + the put lands pinned (pin-before-publish — the facade's put + returns the `Pin` guard); the handler owns the guard; the custody + split is the foundation's design, realized here: + - **fleet**: the guard **drops at response completion** — the pin + row's expiry is the only protection from that moment (the TTL + machinery stays live on the ops path); the token mints over the + committed row (the token↔(key, owner) mapping, owner = the + serving instance's id); + - **non-fleet**: the token registry **takes custody** of the guard + — the token is the in-process guard's handle; nothing expires; + the guard's fate is the custody lifecycle (conversion on the + embedder's confirmation, or process-lifetime hold). + The response returns `{token, digest}` — **both fields always + present** (ADR-012 §2): `digest` is the stored entry's canonical + digest hex (the unknown-length putter cannot know it beforehand and + needs it for every later op); `token` is the minted opaque handle. + Known-length putters receive the same shape. +- **The fanout seam's producer side** (POC #3 finding A3): the put's + stream reader is the canonical fanout producer — **one read** of + the incoming stream (never polled twice), the **store arm** + persists, **subscriber arms** serve concurrent fetchers of that + digest while the put lands (serve-while-persisting); the wiring + into the foundation's fanout registry. Backpressure: bounded + subscriber channels — a slow subscriber stalls the broadcast, + which is correct for a put (the put must land); the + drop-and-late-join escape is the fetch side's policy. +- **The dedup re-put**: CAS dedup means no new entry — the re-put + renews the pin row (fleet: the joint tx's upsert-renewal of + `(key, owner)`) or re-mints the custody guard (non-fleet); the + re-put backstop's mechanics ride the facade's CAS unchanged. +- **Tests through the harness**: the known-length round-trip (the + response's digest matches the git derivation — the byte-exact + standard); the unknown-length round-trip (the buffering path + through the wire); the verification-failure shape (a + byte-corrupted stream → no commit, the typed error, zero residue + on both tiers); the offer-digest mismatch rejection; the hand-over + custody split (fleet: the guard dropped at response — observable + via the row's expiry path; non-fleet: the registry holds the guard + — observable via have-present); the minted token renews via + have's renewal form (the cross-task contract); the dedup re-put; + the ACL denial (no write grant → denied). + +## Acceptance Criteria + +- [ ] put exported as a Sink spec/handler pair with the + `channel_open` marker; the establishment reply fields carry the + offer ack +- [ ] Server-side verification before commit; an offer-digest + mismatch rejects before commit; a verification failure leaves + zero residue on both tiers +- [ ] The response is `{token, digest}`, both fields always present; + `digest` is the canonical hex (the unknown-length putter's + later-op handle) +- [ ] The hand-over custody split implemented per ADR-010 §4 (fleet: + the guard drops at response completion — the row is the + signal; non-fleet: the registry holds the guard — nothing + expires) +- [ ] The fanout producer side wired: one read of the incoming + stream, the store arm + subscriber arms (serve-while- + persisting) +- [ ] The dedup re-put renews (row upsert fleet-form / custody + re-mint non-fleet) +- [ ] Harness tests green; `cargo test` (default) green; + `cargo test --features ops` green; clippy/fmt green on both + shapes; `cargo doc --no-deps` clean + +## References + +- docs/architecture/ops-surface.md (§Op family — the put bullet; §Pin + hand-over for remote puts — the cross-wire contract) +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md + §2 (the {token, digest} response; both fields always present) +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md + §4 (the guard drops at response time — the TTL-machinery-alive + rationale; the non-fleet custody) +- docs/architecture/decisions/006-verification-posture-and-transfer-encoding.md + (verification before commit) +- docs/architecture/decisions/003-backend-contract-and-dispatch.md + (the staged-put contract; the unknown-length path the wire rides) +- docs/research/poc-largeblob-findings.md finding A3 (the fanout + producer shape) +- tasks/ops-foundation.md (the token layer, the custody design, the + fanout registry), tasks/ops-call-harness.md (the harness), + tasks/ops-json-ops.md (the renewal form the minted token rides) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/review-wave-7.md b/tasks/review-wave-7.md new file mode 100644 index 0000000..d506eb7 --- /dev/null +++ b/tasks/review-wave-7.md @@ -0,0 +1,142 @@ +--- +id: review-wave-7 +name: Wave 7 review gate — the ops-vs-alkcall conformance gate +status: pending +depends_on: [ops-e2e-flows] +scope: narrow +risk: low +impact: phase +level: review +tags: [wave-7, review] +--- + +## Description + +The wave-7 review gate (docs/plans/implementation.md §Review gates: +"ops-vs-alkcall conformance (the machinery ridden, not re-invented); +ACL mapping vs OQ-09's closed-by-default; the pin token contract vs +ADR-010 §4/ADR-012 §2; no backend side-door"), before wave 8 +decomposes. The gate reads the implementation against the pinned ADR +text line by line (the wave-1..6 gates' method) — the documents, not +memory. + +Gate checks: + +- **ops-vs-alkcall conformance — the machinery ridden, not + re-invented**: the JSON control plane rides `OperationSpec` + JSON + Schema validation + typed error schemas (alkcall ADR-016); the + binary data plane rides the `channel_open` marker + establishment + (alkcall ADR-047/049) with the Sub/Sink handler kinds (ADR-021/ + 046); ACL is alkcall's `AccessControl` at op entry (ADR-011) under + the ADR-017 privilege model; producer/consumer vocabulary + throughout; no parallel framing, validation, or authorization + invented anywhere in the module. +- **ACL mapping vs OQ-09's closed-by-default**: namespace-as-resource + (`resource_type: "blob-namespace"`, the `resource_id_path` + selecting the payload's `namespace` field); the read action on + fetch/stat/have, the write action on put; unlisted namespaces deny + by default (no ACL entry ⇒ no read, no write — verified by test, + not by reading the code); delete not namespace-gated at all — + internal authority context only (alkcall ADR-017: internal switches + context, never skips ACL). +- **The pin token contract vs ADR-010 §4/ADR-012 §2**: the token *is* + the pin (no second pin state); the `blobs/put` response + `{token, digest}` with both fields always present; the have + renewal form (one digest + token; verify-mapping → the crate- + internal `pin_renew`; expired/foreign → `present: false`, no + renewal); the token's validity domain = the minting node (foreign + tokens refused `present: false`, not an error); the mapping's + process lifetime (outlives the handler); the guard-custody split + (fleet: the guard drops at response completion — the row's expiry + is the liveness signal; non-fleet: the registry holds the guard — + nothing expires); the TTL/3 renewal cadence is embedder + composition (ADR-013 §1) — no in-crate renewal worker. +- **No backend side-door** (ADR-001 §4): the ops module's store + access enumerated — the public facade plus the crate-internal + renewal seam only; no `Backend` touch, no engine types, no + ops-internal path around the store facade; the e2e audit's + enumeration re-verified against the code. +- **The fanout seam vs POC #3 finding A3**: one reader, store arm + + subscriber arms; late joiners degrade to a verified post-commit + `get` (identical bytes under CAS); the slow-subscriber + drop-and-late-join policy is ops-layer; the producer-side design + point consistent with the A3 text. +- **The base crate stays lean**: the `ops` feature default-off; the + alkcall + serde + CSPRNG deps ride the gate; the default build's + dependency tree unchanged (convention 8; ADR-013 §4's table). +- **Code-smell / hygiene sweep** (the gate's standing hygiene block): + - **`cargo doc --no-deps` zero warnings** (the PGLO-1 class — + public-module doc links resolving to crate-internal items — + de-link rather than export). + - **The `#![allow(dead_code)]` scope notes match their remaining + scope** — the ops module's notes audited; `pin_check`'s + wave-6 "deliberately unconsumed" note updated to its landed + state. + - **Stale forward references updated** — every "wave 7" note in + the module docs reads the landed state. + - **Family conventions**: no inline comments in code (doc comments + on public API fine), thiserror, tokio-only async, no + `unwrap`/`expect` outside tests, naming, one-module-per-file + re-export shape, no blocking I/O on the async path. + - **Task-description correctness**: each wave-7 task's Summary + matches what actually landed (the gate reads the diffs against + the task texts, not the Notes' claims alone). +- **Gates green**: `cargo test` (default), `cargo test --features + ops`, `cargo test --all-features` (the fleet rows live at + acceptance), the pure builds (`cargo check --no-default-features + --features postgres` / `--features "postgres,pg-lo"`), + `cargo clippy --all-targets -- -D warnings` across the feature + variants (default, ops, all-features, the pg-pure builds), + `cargo fmt --check`, `cargo doc --no-deps`. + +Findings are fixed inline where cheap, or recorded as wave-8 +amendments where they shape the next decomposition (the fuzzing +wave's target inventory reads this wave's surfaces — a payload-shape +finding lands beside it). The gate's completion is what unblocks +wave 8's decomposition. + +## Acceptance Criteria + +- [ ] Every gate check above read and verified against the cited ADR + text (not from memory of it) +- [ ] Named design points each decided, recorded, and consistent with + the specs +- [ ] Findings fixed inline or recorded as wave-8 amendments with + owners +- [ ] All gates re-run green after any fixes — including + `cargo doc --no-deps` explicitly (the PGLO-1 class) +- [ ] Wave 8 decomposition unblocked (recorded in this task's + Summary) + +## References + +- docs/plans/implementation.md (§Wave 7, §Review gates — the + ops-vs-alkcall conformance gate) +- docs/architecture/ops-surface.md (the op family, the ACL mapping, + the hand-over contract) +- docs/architecture/decisions/001-substrate-posture-and-ops-placement.md + §2/§3/§4 +- docs/architecture/decisions/006-verification-posture-and-transfer-encoding.md +- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md + §2/§3 +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md + §4 +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md + §2 +- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md + §1/§4 +- docs/architecture/open-questions.md OQ-09 +- docs/research/poc-largeblob-findings.md finding A3 +- tasks/review-wave-6.md (the gate method; the W6-3 note this gate + audits closed) +- tasks/ops-foundation.md, tasks/ops-call-harness.md, + tasks/ops-json-ops.md, tasks/ops-fetch.md, tasks/ops-put.md, + tasks/ops-e2e-flows.md (the wave's tasks) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file