docs(tasks): decompose wave 7 — the ops surface, the alkcall-backed op family behind the default-off feature
This commit is contained in:
1 parent
e84a874715
commit
c9829836b1
8 files changed
+852
-2
No files matched your search
@@ -0,0 +1,85 @@
|
||||
---
|
||||
id: ops-call-harness
|
||||
name: the in-process alkcall harness — the serving assembly and the client side over one duplex pair
|
||||
status: pending
|
||||
depends_on: [ops-foundation]
|
||||
scope: narrow
|
||||
risk: low
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-7, ops, testing, harness, alkcall]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
The test harness every ops handler task exercises its handlers
|
||||
through: an in-process alkcall **serving side + client side over one
|
||||
`tokio::io::duplex` pair** — the embedder's "dials and hands an
|
||||
established `Connection` in" shape (ops-surface.md §Why alkcall;
|
||||
alkcall is a pure protocol crate), realized in-process so the ops
|
||||
surface is testable server-less.
|
||||
|
||||
- **The serving assembly** (the embedder-assembly pattern realized in
|
||||
test): `OperationRegistryBuilder` receiving the ops family's
|
||||
spec/handler pairs → the dispatch serving stack (`CallAdapter` /
|
||||
`GatewayDispatch`, plus `ChannelsAdapter` for the binary ops'
|
||||
`channel_open` path) → the dispatch run loop over a
|
||||
`Connection::from_bidi` half (alkcall ADR-009's public stream
|
||||
constructor). The assembly's shape is the module's documented
|
||||
embedder entry — the "module exports spec/handler pairs; the
|
||||
embedder assembles" placement (ops-surface.md's placement note,
|
||||
the alkgit `git/repo/*` pattern).
|
||||
- **The client side**: `CallClient` + `spawn_dispatch` over the
|
||||
duplex pair's other half; `ChannelClient` (`from_connection`,
|
||||
`open_channel`/`call_open_op`) for the channel ops.
|
||||
- **The identity/ACL wiring**: a test `IdentityProvider` +
|
||||
`OwnershipProvider` — the embedder-registry stand-in (namespace →
|
||||
identities → granted actions) — so ACL behavior is exercisable from
|
||||
the first handler task onward, including the closed-by-default
|
||||
denial path (OQ-09).
|
||||
- **Proven with a stub op**: a trivial stub op registered through the
|
||||
same path round-trips (call → response envelope), and the channel
|
||||
path opens with bytes flowing — the pipe proven before any real op
|
||||
rides it. The stub lives in the test tree; it registers nothing
|
||||
into the shipped module surface.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The harness lives in the ops module's test tree
|
||||
(`#[cfg(test)]` under feature `ops`), parameterized over the
|
||||
assembled registry (ops register; the harness serves)
|
||||
- [ ] Server + client halves over one duplex pair; the dispatch run
|
||||
loop serves; the client calls and receives envelopes
|
||||
- [ ] The identity/ownership wiring exercisable: a stub op carrying
|
||||
an `AccessControl` denies an ungranted identity and admits a
|
||||
granted one (the ACL path proven end-to-end pre-handler)
|
||||
- [ ] The channel path proven: a stub `channel_open` op establishes
|
||||
(the `Establishment` phase) and bytes flow both directions
|
||||
- [ ] `cargo test --features ops` green; the default build unaffected
|
||||
(the harness is feature-gated test code);
|
||||
`cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green on both shapes
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/ops-surface.md (§Why alkcall — the Connection
|
||||
hand-in; the placement note)
|
||||
- docs/architecture/decisions/001-substrate-posture-and-ops-placement.md
|
||||
§2 (the machinery inventory this harness assembles)
|
||||
- alkcall (upstream checkout `/workspace/@alkdev/alkcall`):
|
||||
`src/registry/registration.rs` (`OperationRegistryBuilder`, the
|
||||
handler kinds), `src/protocol/adapter.rs` (`CallAdapter`),
|
||||
`src/protocol/connection.rs` (`Connection::from_bidi`),
|
||||
`src/channels/adapter.rs` (`ChannelsAdapter`),
|
||||
`src/channels/client.rs` (`ChannelClient`), ADR-047/049 (the
|
||||
channel_open marker + establishment)
|
||||
- tasks/ops-foundation.md (the context the harness's assembled
|
||||
handlers capture)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,115 @@
|
||||
---
|
||||
id: ops-e2e-flows
|
||||
name: the ops wire-level lifecycle proofs — have/need diffing, the token lifecycle, and the ACL matrix end-to-end
|
||||
status: pending
|
||||
depends_on: [ops-json-ops, ops-fetch, ops-put]
|
||||
scope: broad
|
||||
risk: medium
|
||||
impact: phase
|
||||
level: implementation
|
||||
tags: [wave-7, ops, e2e, tokens, acl, fleet]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
The capstone proofs through the harness (`ops-call-harness`) — the
|
||||
full lifecycle flows the individual handler tests exercise only in
|
||||
pieces. This is wave 7's equivalent of the engine waves' suite
|
||||
columns: the surface proven as a whole, on the wire.
|
||||
|
||||
- **The have/need flow**: a consumer diffs `blobs/have` over its
|
||||
digest set → fetches the absent digests (the need announcement *is*
|
||||
the fetch miss — the diff is caller-side; the wire carries only
|
||||
concrete fetch requests). Proven end-to-end: seed a store via the
|
||||
facade, diff from the client side, fetch exactly the absent set,
|
||||
re-diff to all-present.
|
||||
- **The fleet token lifecycle end-to-end** (against the harness
|
||||
server — env-driven, skipping cleanly server-less; the
|
||||
pglo-poc convention): put → token → renew via have's renewal form
|
||||
before expiry (`present: true`; the row's expiry observed moving) →
|
||||
the embedder's registration confirms → the row's protection hands
|
||||
to the registered liveness source (the namespace binding's
|
||||
converter) → the row expires naturally and the entry survives on
|
||||
the registered source's protection. Then the failure half: pause
|
||||
past TTL → the renewal probe answers `present: false` → the
|
||||
sweeper reaps the row → the **re-put backstop** (delete-then-
|
||||
recover; CAS makes the re-put idempotent) → **the race resolves to
|
||||
a re-put, never to silent loss** (ADR-010 §4's expiry semantics
|
||||
verbatim, on the wire).
|
||||
- **The foreign-token refusal**: a token minted by node A presented
|
||||
to node B (a second fleet instance's ops module over the same
|
||||
pool) → `present: false`, not an error — the validity domain is
|
||||
the minting node (ADR-012 §2).
|
||||
- **The non-fleet token lifecycle**: put → token (the registry holds
|
||||
the guard) → have-present answers from the custody → the embedder's
|
||||
registration confirms → the guard converts (the namespace
|
||||
binding's converter) → the token retires. Proven on a non-fleet
|
||||
store (sqlite or mem — server-less).
|
||||
- **The ACL matrix end-to-end**: an unlisted namespace denies read
|
||||
AND write (closed-by-default — OQ-09); a read-granted identity
|
||||
fetches/stats/haves but cannot put; a write-granted identity puts
|
||||
but a read-only peer cannot fetch its namespace; delete only under
|
||||
the internal authority context (never a namespace grant).
|
||||
- **The re-routing posture's serving half** (ADR-008 §3): a large
|
||||
(pg-lo) entry served through `blobs/fetch` from the storage node's
|
||||
dual-tier fleet store — the shape client nodes in kv-only mode
|
||||
consume; no fleet-specific op exists (the module supports the shape
|
||||
without change).
|
||||
- **The no-side-door audit** (structural, ADR-001 §4): the ops
|
||||
module's store access enumerated — the public facade plus the
|
||||
foundation's crate-internal renewal seam only; no `Backend` touch,
|
||||
no engine types, no path around the facade. Recorded as the audit
|
||||
the wave-7 review gate re-checks.
|
||||
- **The renewal-cadence boundary honored** (ADR-013 §1): the tests
|
||||
drive renewal explicitly (the embedder's cadence role played by the
|
||||
test); no in-crate renewal worker exists.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The have/need flow proven end-to-end (diff caller-side; the
|
||||
wire carries only concrete fetches)
|
||||
- [ ] The fleet token lifecycle end-to-end vs the harness server:
|
||||
renew-before-expiry (the row's expiry moves), the
|
||||
registration-handover, pause-past-TTL → reap → the re-put
|
||||
backstop — never silent loss (env-driven; skipping cleanly
|
||||
server-less; acceptance requires green against the server)
|
||||
- [ ] The foreign-token refusal: `present: false`, not an error
|
||||
- [ ] The non-fleet lifecycle: custody → confirmation → conversion →
|
||||
token retirement (server-less)
|
||||
- [ ] The ACL matrix end-to-end (closed-by-default read+write denial;
|
||||
the read/write split; delete internal-only)
|
||||
- [ ] The re-routing posture's serving half: a large entry fetched
|
||||
through the ops surface from the dual-tier fleet store
|
||||
- [ ] The no-side-door audit recorded (the store-access enumeration)
|
||||
- [ ] Gates green: `cargo test` (default), `cargo test --features
|
||||
ops`, `cargo test --all-features` (the fleet rows live at
|
||||
acceptance), clippy/fmt across the variants, `cargo doc
|
||||
--no-deps` clean
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/ops-surface.md (§Op family — the hand-over
|
||||
contract; the need-half note; §Store-facing relationship — the
|
||||
re-routing posture)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
|
||||
§4 (the expiry semantics; the re-put backstop)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§2 (the validity domain; the non-fleet generalization)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
§2/§3 (the TTL/3 cadence; the re-routing topology)
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
|
||||
§1 (the cadence is embedder composition)
|
||||
- docs/architecture/open-questions.md OQ-09 (closed-by-default)
|
||||
- tasks/ops-foundation.md, tasks/ops-call-harness.md,
|
||||
tasks/ops-json-ops.md, tasks/ops-fetch.md, tasks/ops-put.md (the
|
||||
surface this proves as a whole)
|
||||
- tasks/suite-column-fleet.md (the env-driven harness-server
|
||||
convention this task's fleet rows follow)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
id: ops-fetch
|
||||
name: blobs/fetch — the verified Sub op and the fanout seam's consumer side
|
||||
status: pending
|
||||
depends_on: [ops-foundation, ops-call-harness]
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-7, ops, fetch, fanout, verification]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
`blobs/fetch` (ops-surface.md §Op family) — the server→client
|
||||
streaming op: `{namespace, digest, ranges?}` in; **verified bytes
|
||||
out** over the binary data channel.
|
||||
|
||||
- **The op shape**: `OperationType::Sub` (alkcall ADR-021's streaming
|
||||
handler kind) with the **`channel_open` marker** (alkcall ADR-047)
|
||||
— the binary data plane rides a `channel_open`-allocated channel;
|
||||
the establishment phase (alkcall ADR-049) carries the fetch's ack
|
||||
shape in the `Establishment` reply fields as the wire requires. The
|
||||
**read** action (ACL exactly like stat/have — closed-by-default).
|
||||
- **Verified streaming** (ADR-006; ops-surface.md): the *consumer*
|
||||
hash-checks the received whole against the carried digest —
|
||||
whole-blob verification; there are no per-chunk or per-range checks
|
||||
derivable under the canonical digest. The handler streams from the
|
||||
facade only: `Store::get` (the whole) or `Store::read_range` (the
|
||||
ranged form). **Ranged fetch** (`ranges?`): the response carries
|
||||
the same ADR-006 slice-digest convention `read_range` returns —
|
||||
SHA-256 over exactly the returned bytes, out-of-band.
|
||||
- **The fanout seam's consumer side** (ops-surface.md; POC #3 finding
|
||||
A3): the fetch handler consults the foundation's fanout registry —
|
||||
a live broadcast for the digest → a **subscriber arm** taps it; no
|
||||
live broadcast, or one that ended (a **late joiner**) → the normal
|
||||
post-commit verified `get` — identical bytes under CAS. The
|
||||
slow-subscriber policy (drop-and-late-join) is ops-layer policy,
|
||||
not a store concern. Named design point: **the producer side's
|
||||
shape** — the put handler's stream reader as the canonical producer
|
||||
(serve-while-persisting, A3's shape), and whether fetches also
|
||||
produce read-side broadcasts (concurrent fetchers of one stored
|
||||
digest sharing one store read) — decide and record against the A3
|
||||
text.
|
||||
- **The miss shape**: a fetch of an absent digest is the typed
|
||||
missing-error schema (the need announcement *is* the fetch miss —
|
||||
the have/need diff is caller-side; no separate need op exists).
|
||||
- **Tests through the harness**: the whole-blob verified round-trip
|
||||
(byte-exact; the client's hash-check passes); the ranged form
|
||||
(bytes + slice digest matching `read_range`'s convention); the
|
||||
fanout consumer side (a live broadcast tapped — the subscriber
|
||||
receives while the producer runs; a late joiner degrades to the
|
||||
get path — asserted by which path served); the miss shape; the ACL
|
||||
denial (no read grant on the namespace → denied).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] fetch exported as a Sub spec/handler pair with the
|
||||
`channel_open` marker + ALPN; the establishment reply fields
|
||||
carry the specced ack shape
|
||||
- [ ] Whole-blob verification: the received whole hash-checks against
|
||||
the carried digest (the consumer-side discipline documented on
|
||||
the op; the server streams verified-store bytes)
|
||||
- [ ] Ranged fetch carries the ADR-006 slice-digest convention
|
||||
(out-of-band, over exactly the returned bytes)
|
||||
- [ ] The fanout consumer side: a live broadcast → subscriber arm; a
|
||||
late joiner → the verified get path; the producer-side design
|
||||
point decided and recorded
|
||||
- [ ] The miss shape is the typed missing error; the have/need diff
|
||||
stays caller-side (no need op invented)
|
||||
- [ ] ACL: the read action; closed-by-default verified by test
|
||||
- [ ] Handlers touch only the store facade (no backend side-door)
|
||||
- [ ] Harness tests green; `cargo test` (default) green;
|
||||
`cargo test --features ops` green; clippy/fmt green on both
|
||||
shapes; `cargo doc --no-deps` clean
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/ops-surface.md (§Op family — the fetch bullet +
|
||||
the fanout text; the need-half note)
|
||||
- docs/architecture/decisions/006-verification-posture-and-transfer-encoding.md
|
||||
(whole-blob verification; the slice-digest convention)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
§3 (the re-routing topology this op serves — large-blob access
|
||||
through a storage node)
|
||||
- docs/research/poc-largeblob-findings.md finding A3 (the fanout
|
||||
seam: one reader, store arm + subscriber arms; late joiners
|
||||
degrade to get-after-commit)
|
||||
- alkcall (upstream checkout): ADR-021 (streaming handler for
|
||||
subscriptions), ADR-046/047/049 (the sink kind, the channel_open
|
||||
marker, the establishment phase), `src/channels/client.rs`
|
||||
- tasks/ops-foundation.md (the fanout registry's mechanics),
|
||||
tasks/ops-call-harness.md (the harness)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,160 @@
|
||||
---
|
||||
id: ops-foundation
|
||||
name: the ops module's substrate — the token layer, the namespace binding, and the fanout registry
|
||||
status: pending
|
||||
depends_on: []
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-7, ops, tokens, fanout, namespaces]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
The `ops` feature's state substrate in `src/ops.rs` (currently the
|
||||
ADR-001 stub doc comment) — everything the five op handlers ride, and
|
||||
none of the handlers themselves (they land in the following tasks).
|
||||
The module is a **store-facade consumer** (ADR-001 §4): its store
|
||||
access is the public facade plus the crate-internal renewal seam named
|
||||
below — no backend side-door, no engine types.
|
||||
|
||||
- **Module structure + the ops context**: the feature-gated module
|
||||
shape (`#[cfg(feature = "ops")]`, re-exported from `lib.rs` per the
|
||||
one-module-per-file convention); the ops-layer context type binding
|
||||
the `Arc<Store>`, the module's state (the token registry, the
|
||||
namespace bindings, the fanout registry), and the module config.
|
||||
This context is what the handler pairs (later tasks) capture.
|
||||
- **The pin-token layer** (ADR-010 §4; ADR-012 §2): opaque token
|
||||
minting over a landed put's pin; the **token↔(key, owner) in-memory
|
||||
mapping with process lifetime** (outlives the minting handler —
|
||||
ADR-012 §2's correction of the original handler-lifetime wording);
|
||||
the **validity domain is the minting node** — the registry is
|
||||
per-process state, there is no cross-node form, and fleet peers
|
||||
refuse tokens they did not mint (`present: false`, not an error).
|
||||
Named design points: **the token's minting form** (a CSPRNG id —
|
||||
e.g. `getrandom` gated behind `ops`; a counter or derivable form is
|
||||
rejected as a guessable capability — decide and record), and **the
|
||||
mapping's retention discipline** (when a mapping retires:
|
||||
conversion-confirmed or process end; the mapping carries no expiry
|
||||
machinery of its own — the pin row's expiry is the fleet liveness
|
||||
signal, and nothing expires non-fleet).
|
||||
- **The guard-custody design** (the fleet/non-fleet split ADR-010 §4
|
||||
pins): under fleet, the put handler's `Pin` guard **drops at
|
||||
response completion** — the pin row's expiry is the only protection
|
||||
from that moment, which is exactly why the TTL/3 renewal duty
|
||||
exists (a guard held for the token's whole life would make the TTL
|
||||
machinery dead code on the ops path); under non-fleet, the token
|
||||
registry **holds the guard** — the token is the in-process guard's
|
||||
handle (ADR-012 §2's generalization: sqlite, single-instance
|
||||
postgres, pg-lo, `local`, `mem` — nothing expires outside fleet
|
||||
mode) until the embedder's registration confirms and the guard
|
||||
converts. The custody split is the token layer's core branch.
|
||||
- **The namespace→liveness-source binding** (ops-surface.md §Op
|
||||
family, hand-over contract step 3): the embedder's converter seam —
|
||||
per-namespace registration of the converter the embedder plugs its
|
||||
registry into ("A `Pin` conversion on the server side (token →
|
||||
registered liveness source) is the mechanism the embedder plugs its
|
||||
registry into"). The conversion rides the crate-internal
|
||||
`RegistryCore::convert` machinery (wave 2). Named design point: **
|
||||
the conversion trigger** (an embedder-called confirm verb vs a
|
||||
probe-triggered conversion — decide and record; the hand-over
|
||||
contract's duty text is the embedder's, the mechanism is this
|
||||
module's).
|
||||
- **The crate-internal store-core renewal access** (ADR-010 §4: "the
|
||||
ops module is in-crate and crosses the seam by construction, the
|
||||
same way the put handler holds the `Pin` guard"): the renewal
|
||||
probe's access to `pin_renew(key, owner)` and `pin_check(key)`
|
||||
through the fleet host — the `pin_check` deliberately-unconsumed
|
||||
note from the wave-6 review (W6-3) names this consumer. The exact
|
||||
probe composition (verify-the-mapping-then-renew vs
|
||||
renew-answers-the-verification) is a named design point against
|
||||
ADR-012 §2's three-step handler duty. What never happens: a *
|
||||
consumer* reaching the seam, or a public-API widening for this.
|
||||
- **The fanout registry** (ops-surface.md's fetch fanout seam; POC #3
|
||||
finding A3): per-digest broadcast state — one reader, store arm +
|
||||
subscriber arms; bounded subscriber channels (the backpressure
|
||||
discipline: a slow subscriber stalls the broadcast, which is
|
||||
correct for a put — subscriber count bounded, slow-subscriber
|
||||
handling is policy); the **late-joiner signal** (broadcast ended →
|
||||
the degrade-to-verified-get path); the **drop-and-late-join**
|
||||
slow-subscriber policy as ops-layer policy. The producer wiring
|
||||
lands with `ops-put`, the consumer wiring with `ops-fetch`; the
|
||||
registry's mechanics land here, unit-tested standalone.
|
||||
- **The renewal-cadence boundary** (ADR-013 §1): the module ships the
|
||||
renewal *mechanism* (the have form's handler); the TTL/3 *cadence*
|
||||
— the putter's renewal worker — is embedder composition above the
|
||||
facade (scheduler/queues), never in-crate machinery. Documented on
|
||||
the module, not implemented.
|
||||
- **Dependency posture**: `serde`/`serde_json` (payload handling) and
|
||||
the token CSPRNG enter as `ops`-gated dependencies — the base crate
|
||||
stays lean (convention 8; ADR-013 §4's engine table unchanged —
|
||||
`ops` pulls alkcall plus these, default-off).
|
||||
- **Unit tests**: token mint/lookup/retirement and the mapping's
|
||||
process-lifetime semantics; the custody split (the fleet and
|
||||
non-fleet forms branch as designed); the namespace binding's
|
||||
converter invocation; the fanout registry's subscribe /
|
||||
late-joiner-degrade / slow-subscriber-drop mechanics.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `src/ops.rs` carries the module structure under feature `ops`:
|
||||
the ops context, the token layer, the namespace binding, the
|
||||
fanout registry — no op handlers yet (they land in
|
||||
ops-json-ops/ops-fetch/ops-put)
|
||||
- [ ] The token layer: opaque, unguessable minting (the CSPRNG design
|
||||
point decided and recorded); the token↔(key, owner) mapping with
|
||||
process lifetime; validity domain = the minting node (a
|
||||
per-process registry; no cross-node form exists)
|
||||
- [ ] The guard-custody design decided and recorded: fleet — the
|
||||
guard drops at response completion (the row's expiry is the
|
||||
liveness signal); non-fleet — the registry holds the guard
|
||||
(nothing expires) until conversion
|
||||
- [ ] The namespace→liveness-source binding: per-namespace converter
|
||||
registration riding `RegistryCore::convert`; the conversion
|
||||
trigger design decided and recorded
|
||||
- [ ] The crate-internal renewal access: the ops module reaches
|
||||
`pin_renew`/`pin_check` through the fleet host (in-crate seam
|
||||
crossing; no public-API widening; no backend side-door)
|
||||
- [ ] The fanout registry: per-digest broadcast, bounded subscriber
|
||||
channels, the late-joiner degrade signal, the
|
||||
drop-and-late-join policy point recorded
|
||||
- [ ] New dependencies (`serde`, `serde_json`, the CSPRNG) are
|
||||
`ops`-gated; the default build's dependency tree is unchanged
|
||||
- [ ] Unit tests green; `cargo test` (default) green with the module
|
||||
compiled out; `cargo test --features ops` green;
|
||||
`cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green on both shapes; `cargo doc --no-deps`
|
||||
clean
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/ops-surface.md (§Op family — the hand-over
|
||||
contract; §ACL mapping; §Store-facing relationship)
|
||||
- docs/architecture/decisions/001-substrate-posture-and-ops-placement.md
|
||||
§3/§4 (placement; the facade-consumer rule)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
|
||||
§4 (the token contract: minting, custody, validity domain, the
|
||||
in-crate seam crossing)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§2 (the wire shapes: {token, digest}; the have renewal form; the
|
||||
validity domain; the non-fleet generalization)
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
|
||||
§1 (the renewal cadence is embedder composition), §4 (the feature
|
||||
graph)
|
||||
- docs/research/poc-largeblob-findings.md finding A3 (the fanout seam
|
||||
shape)
|
||||
- src/ops.rs (the ADR-001 stub this task fills)
|
||||
- src/gc.rs (`EngineFleetState::pin_renew`/`pin_check` — the seam the
|
||||
renewal access crosses; `RegistryCore::convert`)
|
||||
- src/store.rs (`Store::gc()` — the crate-internal access path)
|
||||
- tasks/review-wave-6.md (W6-3: `pin_check` deliberately unconsumed —
|
||||
this wave's renewal probe is the named consumer)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,119 @@
|
||||
---
|
||||
id: ops-json-ops
|
||||
name: the JSON control plane — blobs/stat, blobs/have (+ the token-renewal form), blobs/delete
|
||||
status: pending
|
||||
depends_on: [ops-foundation, ops-call-harness]
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-7, ops, json, acl, tokens]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
The three JSON control ops (ops-surface.md §Op family), each exported
|
||||
as a spec/handler pair the embedder assembles — riding alkcall's
|
||||
`OperationSpec` + JSON Schema validation (alkcall ADR-016) and its ACL
|
||||
machinery at op entry (convention 6: this module never re-implements
|
||||
an authorization check):
|
||||
|
||||
- **`blobs/stat`** `{namespace, digest}` → `{len}` — Query; the
|
||||
**read** action; the handler calls `Store::stat` (the fall-through
|
||||
general rule, ADR-012 §6.1); a miss is the typed missing-error
|
||||
schema (not a `{len: null}`).
|
||||
- **`blobs/have`** `{namespace, digests[], token?}` → `{present[]}` —
|
||||
Query; **read-gated** (an existence probe over arbitrary digests is
|
||||
a discovery surface — gated exactly like fetch, not public);
|
||||
`present[]` corresponds to `digests[]` by index; content never
|
||||
traverses this op (hashes only). **The renewal form** (ADR-012 §2):
|
||||
when `token` is present, exactly one digest accompanies it — the
|
||||
handler verifies the token maps to that digest's pin row (the
|
||||
foundation's token layer), calls the crate-internal `pin_renew(key,
|
||||
owner)` via the foundation's store-core access, and answers
|
||||
`present[]` as usual; an expired or foreign row answers
|
||||
`present: false` and no renewal occurs — the putter's backstop is
|
||||
re-put. Without `token`, the pure existence probe (ADR-008's `has`
|
||||
discipline) over the facade's `has`. **The non-fleet renewal form**:
|
||||
the token is the held guard's handle — the probe answers present
|
||||
from the custody (nothing expires; ADR-012 §2's generalization);
|
||||
the custody/conversion lifecycle is the foundation's.
|
||||
- **`blobs/delete`** `{namespace, digests[]}` — Mutation;
|
||||
**`Visibility::Internal`**, evaluated under the internal authority
|
||||
context per alkcall ADR-017 (internal calls switch authority
|
||||
context, never skip ACL): requires the embedder's operator/admin
|
||||
authority context, **not a namespace grant**; the `namespace` field
|
||||
exists for audit/triage scoping, not gating. The handler calls
|
||||
`Store::delete` (the arbitration-then-delete path; a `GcRefuse`
|
||||
refusal surfaces as the typed error schema). External discovery of
|
||||
an Internal op returns NOT_FOUND (alkcall ADR-017's visibility
|
||||
discipline).
|
||||
- **The ACL mapping** (ops-surface.md §ACL mapping; OQ-09's
|
||||
resolution): **namespace = resource** — `resource_type:
|
||||
"blob-namespace"`, `resource_id_path` selecting the payload's
|
||||
`namespace` field; stat/have = the **read** action; **unlisted
|
||||
namespaces deny by default** (closed-by-default: no ACL entry ⇒ no
|
||||
read, no write — a grant per namespace is the deferred cost,
|
||||
deliberately paid over accidental exposure).
|
||||
- **Schemas** (alkcall ADR-016): input/output JSON Schema per op +
|
||||
typed error schemas; the namespace field's syntax (non-empty UTF-8
|
||||
string, embedder-validated — the minting of namespaces is the
|
||||
embedder's act); the digest wire form is **hex** (the canonical
|
||||
digest hex — the same form `blobs/put`'s response returns; parsed
|
||||
via the key layer's hex ingress, malformed → the typed invalid
|
||||
error, rejects-never-guesses).
|
||||
- **Tests through the harness** (`ops-call-harness`): stat round-trip
|
||||
+ the miss shape; have's index correspondence (present/absent
|
||||
mixed); the renewal form's three answers (live row renews →
|
||||
present: true; expired row → present: false, no renewal; foreign
|
||||
token → present: false); delete under the internal authority
|
||||
context (an external caller gets NOT_FOUND; an internal
|
||||
operator-context call deletes); the closed-by-default ACL matrix
|
||||
(an unlisted namespace denies read; a granted identity reads).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Three spec/handler pairs exported (stat, have, delete); input/
|
||||
output schemas + typed error schemas per alkcall ADR-016
|
||||
- [ ] The ACL mapping: namespace-as-resource with the
|
||||
resource_id_path; the read action on stat/have;
|
||||
closed-by-default verified by test (unlisted namespace denies)
|
||||
- [ ] delete: `Visibility::Internal`, the internal-authority-context
|
||||
requirement (not namespace-gated), external discovery NOT_FOUND
|
||||
- [ ] have's renewal form: one digest + token; verify-mapping →
|
||||
`pin_renew`; expired/foreign → `present: false` with no
|
||||
renewal; the non-fleet form answers from the held custody
|
||||
- [ ] The handlers touch only the store facade + the foundation's
|
||||
crate-internal renewal access (no backend side-door — the
|
||||
ADR-001 §4 rule)
|
||||
- [ ] Harness tests green; `cargo test` (default) green;
|
||||
`cargo test --features ops` green; clippy/fmt green on both
|
||||
shapes; `cargo doc --no-deps` clean
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/ops-surface.md (§Op family — the three JSON ops;
|
||||
§ACL mapping)
|
||||
- docs/architecture/decisions/001-substrate-posture-and-ops-placement.md
|
||||
§2/§4
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§2 (the have payload shape + the renewal form's three-step handler
|
||||
duty)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
(the `has` discipline the non-token form preserves)
|
||||
- docs/architecture/open-questions.md OQ-09 (closed-by-default, the
|
||||
risk-asymmetry rationale)
|
||||
- alkcall (upstream checkout): ADR-016 (operation error schemas),
|
||||
ADR-017 (the privilege model — internal switches context, never
|
||||
skips ACL), `src/registry/spec.rs` (`AccessControl`,
|
||||
`resource_id_path`)
|
||||
- tasks/ops-foundation.md (the token layer + the renewal access the
|
||||
have handler rides), tasks/ops-call-harness.md (the harness)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,128 @@
|
||||
---
|
||||
id: ops-put
|
||||
name: blobs/put — the verified Sink op, the pin hand-over, and the {token, digest} response
|
||||
status: pending
|
||||
depends_on: [ops-foundation, ops-call-harness]
|
||||
scope: moderate
|
||||
risk: high
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-7, ops, put, tokens, hand-over, verification]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
`blobs/put` (ops-surface.md §Op family) — the client→server streaming
|
||||
op: `{namespace, digest?, len?}` offer + the byte stream in; the
|
||||
server verifies against the canonical derivation **before commit**;
|
||||
the response is **`{token, digest}`** (ADR-012 §2).
|
||||
|
||||
- **The op shape**: the Sink handler kind (alkcall ADR-046's
|
||||
`Pub`/Sink — the client→server streaming shape) with the
|
||||
**`channel_open` marker** (alkcall ADR-047); the establishment
|
||||
phase carries the offer ack. The **write** action (ACL;
|
||||
closed-by-default).
|
||||
- **Verification before commit** (ADR-006; ops-surface.md): the
|
||||
server hash-checks the received stream against the canonical
|
||||
derivation (`H("blob <len>\0" + content)`) before commit. A
|
||||
**known-length offer** (`len` present) is the encouraged one-pass
|
||||
path; an **unknown-length offer** rides the store's pre-threshold
|
||||
buffering path (ADR-003's option (d) semantics — the facade's
|
||||
`put(None, …)` form, the mid-stream stage flush and the
|
||||
restart-over-staged-bytes derivation pass riding unchanged). When
|
||||
the offer carries `digest?`, the derived key is checked against it
|
||||
— a mismatch rejects before commit (the offered digest is a
|
||||
claim; the derivation is the authority).
|
||||
- **The pin hand-over** (ops-surface.md §Pin hand-over; ADR-010 §4):
|
||||
the put lands pinned (pin-before-publish — the facade's put
|
||||
returns the `Pin` guard); the handler owns the guard; the custody
|
||||
split is the foundation's design, realized here:
|
||||
- **fleet**: the guard **drops at response completion** — the pin
|
||||
row's expiry is the only protection from that moment (the TTL
|
||||
machinery stays live on the ops path); the token mints over the
|
||||
committed row (the token↔(key, owner) mapping, owner = the
|
||||
serving instance's id);
|
||||
- **non-fleet**: the token registry **takes custody** of the guard
|
||||
— the token is the in-process guard's handle; nothing expires;
|
||||
the guard's fate is the custody lifecycle (conversion on the
|
||||
embedder's confirmation, or process-lifetime hold).
|
||||
The response returns `{token, digest}` — **both fields always
|
||||
present** (ADR-012 §2): `digest` is the stored entry's canonical
|
||||
digest hex (the unknown-length putter cannot know it beforehand and
|
||||
needs it for every later op); `token` is the minted opaque handle.
|
||||
Known-length putters receive the same shape.
|
||||
- **The fanout seam's producer side** (POC #3 finding A3): the put's
|
||||
stream reader is the canonical fanout producer — **one read** of
|
||||
the incoming stream (never polled twice), the **store arm**
|
||||
persists, **subscriber arms** serve concurrent fetchers of that
|
||||
digest while the put lands (serve-while-persisting); the wiring
|
||||
into the foundation's fanout registry. Backpressure: bounded
|
||||
subscriber channels — a slow subscriber stalls the broadcast,
|
||||
which is correct for a put (the put must land); the
|
||||
drop-and-late-join escape is the fetch side's policy.
|
||||
- **The dedup re-put**: CAS dedup means no new entry — the re-put
|
||||
renews the pin row (fleet: the joint tx's upsert-renewal of
|
||||
`(key, owner)`) or re-mints the custody guard (non-fleet); the
|
||||
re-put backstop's mechanics ride the facade's CAS unchanged.
|
||||
- **Tests through the harness**: the known-length round-trip (the
|
||||
response's digest matches the git derivation — the byte-exact
|
||||
standard); the unknown-length round-trip (the buffering path
|
||||
through the wire); the verification-failure shape (a
|
||||
byte-corrupted stream → no commit, the typed error, zero residue
|
||||
on both tiers); the offer-digest mismatch rejection; the hand-over
|
||||
custody split (fleet: the guard dropped at response — observable
|
||||
via the row's expiry path; non-fleet: the registry holds the guard
|
||||
— observable via have-present); the minted token renews via
|
||||
have's renewal form (the cross-task contract); the dedup re-put;
|
||||
the ACL denial (no write grant → denied).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] put exported as a Sink spec/handler pair with the
|
||||
`channel_open` marker; the establishment reply fields carry the
|
||||
offer ack
|
||||
- [ ] Server-side verification before commit; an offer-digest
|
||||
mismatch rejects before commit; a verification failure leaves
|
||||
zero residue on both tiers
|
||||
- [ ] The response is `{token, digest}`, both fields always present;
|
||||
`digest` is the canonical hex (the unknown-length putter's
|
||||
later-op handle)
|
||||
- [ ] The hand-over custody split implemented per ADR-010 §4 (fleet:
|
||||
the guard drops at response completion — the row is the
|
||||
signal; non-fleet: the registry holds the guard — nothing
|
||||
expires)
|
||||
- [ ] The fanout producer side wired: one read of the incoming
|
||||
stream, the store arm + subscriber arms (serve-while-
|
||||
persisting)
|
||||
- [ ] The dedup re-put renews (row upsert fleet-form / custody
|
||||
re-mint non-fleet)
|
||||
- [ ] Harness tests green; `cargo test` (default) green;
|
||||
`cargo test --features ops` green; clippy/fmt green on both
|
||||
shapes; `cargo doc --no-deps` clean
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/ops-surface.md (§Op family — the put bullet; §Pin
|
||||
hand-over for remote puts — the cross-wire contract)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§2 (the {token, digest} response; both fields always present)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
|
||||
§4 (the guard drops at response time — the TTL-machinery-alive
|
||||
rationale; the non-fleet custody)
|
||||
- docs/architecture/decisions/006-verification-posture-and-transfer-encoding.md
|
||||
(verification before commit)
|
||||
- docs/architecture/decisions/003-backend-contract-and-dispatch.md
|
||||
(the staged-put contract; the unknown-length path the wire rides)
|
||||
- docs/research/poc-largeblob-findings.md finding A3 (the fanout
|
||||
producer shape)
|
||||
- tasks/ops-foundation.md (the token layer, the custody design, the
|
||||
fanout registry), tasks/ops-call-harness.md (the harness),
|
||||
tasks/ops-json-ops.md (the renewal form the minted token rides)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,142 @@
|
||||
---
|
||||
id: review-wave-7
|
||||
name: Wave 7 review gate — the ops-vs-alkcall conformance gate
|
||||
status: pending
|
||||
depends_on: [ops-e2e-flows]
|
||||
scope: narrow
|
||||
risk: low
|
||||
impact: phase
|
||||
level: review
|
||||
tags: [wave-7, review]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
The wave-7 review gate (docs/plans/implementation.md §Review gates:
|
||||
"ops-vs-alkcall conformance (the machinery ridden, not re-invented);
|
||||
ACL mapping vs OQ-09's closed-by-default; the pin token contract vs
|
||||
ADR-010 §4/ADR-012 §2; no backend side-door"), before wave 8
|
||||
decomposes. The gate reads the implementation against the pinned ADR
|
||||
text line by line (the wave-1..6 gates' method) — the documents, not
|
||||
memory.
|
||||
|
||||
Gate checks:
|
||||
|
||||
- **ops-vs-alkcall conformance — the machinery ridden, not
|
||||
re-invented**: the JSON control plane rides `OperationSpec` + JSON
|
||||
Schema validation + typed error schemas (alkcall ADR-016); the
|
||||
binary data plane rides the `channel_open` marker + establishment
|
||||
(alkcall ADR-047/049) with the Sub/Sink handler kinds (ADR-021/
|
||||
046); ACL is alkcall's `AccessControl` at op entry (ADR-011) under
|
||||
the ADR-017 privilege model; producer/consumer vocabulary
|
||||
throughout; no parallel framing, validation, or authorization
|
||||
invented anywhere in the module.
|
||||
- **ACL mapping vs OQ-09's closed-by-default**: namespace-as-resource
|
||||
(`resource_type: "blob-namespace"`, the `resource_id_path`
|
||||
selecting the payload's `namespace` field); the read action on
|
||||
fetch/stat/have, the write action on put; unlisted namespaces deny
|
||||
by default (no ACL entry ⇒ no read, no write — verified by test,
|
||||
not by reading the code); delete not namespace-gated at all —
|
||||
internal authority context only (alkcall ADR-017: internal switches
|
||||
context, never skips ACL).
|
||||
- **The pin token contract vs ADR-010 §4/ADR-012 §2**: the token *is*
|
||||
the pin (no second pin state); the `blobs/put` response
|
||||
`{token, digest}` with both fields always present; the have
|
||||
renewal form (one digest + token; verify-mapping → the crate-
|
||||
internal `pin_renew`; expired/foreign → `present: false`, no
|
||||
renewal); the token's validity domain = the minting node (foreign
|
||||
tokens refused `present: false`, not an error); the mapping's
|
||||
process lifetime (outlives the handler); the guard-custody split
|
||||
(fleet: the guard drops at response completion — the row's expiry
|
||||
is the liveness signal; non-fleet: the registry holds the guard —
|
||||
nothing expires); the TTL/3 renewal cadence is embedder
|
||||
composition (ADR-013 §1) — no in-crate renewal worker.
|
||||
- **No backend side-door** (ADR-001 §4): the ops module's store
|
||||
access enumerated — the public facade plus the crate-internal
|
||||
renewal seam only; no `Backend` touch, no engine types, no
|
||||
ops-internal path around the store facade; the e2e audit's
|
||||
enumeration re-verified against the code.
|
||||
- **The fanout seam vs POC #3 finding A3**: one reader, store arm +
|
||||
subscriber arms; late joiners degrade to a verified post-commit
|
||||
`get` (identical bytes under CAS); the slow-subscriber
|
||||
drop-and-late-join policy is ops-layer; the producer-side design
|
||||
point consistent with the A3 text.
|
||||
- **The base crate stays lean**: the `ops` feature default-off; the
|
||||
alkcall + serde + CSPRNG deps ride the gate; the default build's
|
||||
dependency tree unchanged (convention 8; ADR-013 §4's table).
|
||||
- **Code-smell / hygiene sweep** (the gate's standing hygiene block):
|
||||
- **`cargo doc --no-deps` zero warnings** (the PGLO-1 class —
|
||||
public-module doc links resolving to crate-internal items —
|
||||
de-link rather than export).
|
||||
- **The `#![allow(dead_code)]` scope notes match their remaining
|
||||
scope** — the ops module's notes audited; `pin_check`'s
|
||||
wave-6 "deliberately unconsumed" note updated to its landed
|
||||
state.
|
||||
- **Stale forward references updated** — every "wave 7" note in
|
||||
the module docs reads the landed state.
|
||||
- **Family conventions**: no inline comments in code (doc comments
|
||||
on public API fine), thiserror, tokio-only async, no
|
||||
`unwrap`/`expect` outside tests, naming, one-module-per-file
|
||||
re-export shape, no blocking I/O on the async path.
|
||||
- **Task-description correctness**: each wave-7 task's Summary
|
||||
matches what actually landed (the gate reads the diffs against
|
||||
the task texts, not the Notes' claims alone).
|
||||
- **Gates green**: `cargo test` (default), `cargo test --features
|
||||
ops`, `cargo test --all-features` (the fleet rows live at
|
||||
acceptance), the pure builds (`cargo check --no-default-features
|
||||
--features postgres` / `--features "postgres,pg-lo"`),
|
||||
`cargo clippy --all-targets -- -D warnings` across the feature
|
||||
variants (default, ops, all-features, the pg-pure builds),
|
||||
`cargo fmt --check`, `cargo doc --no-deps`.
|
||||
|
||||
Findings are fixed inline where cheap, or recorded as wave-8
|
||||
amendments where they shape the next decomposition (the fuzzing
|
||||
wave's target inventory reads this wave's surfaces — a payload-shape
|
||||
finding lands beside it). The gate's completion is what unblocks
|
||||
wave 8's decomposition.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Every gate check above read and verified against the cited ADR
|
||||
text (not from memory of it)
|
||||
- [ ] Named design points each decided, recorded, and consistent with
|
||||
the specs
|
||||
- [ ] Findings fixed inline or recorded as wave-8 amendments with
|
||||
owners
|
||||
- [ ] All gates re-run green after any fixes — including
|
||||
`cargo doc --no-deps` explicitly (the PGLO-1 class)
|
||||
- [ ] Wave 8 decomposition unblocked (recorded in this task's
|
||||
Summary)
|
||||
|
||||
## References
|
||||
|
||||
- docs/plans/implementation.md (§Wave 7, §Review gates — the
|
||||
ops-vs-alkcall conformance gate)
|
||||
- docs/architecture/ops-surface.md (the op family, the ACL mapping,
|
||||
the hand-over contract)
|
||||
- docs/architecture/decisions/001-substrate-posture-and-ops-placement.md
|
||||
§2/§3/§4
|
||||
- docs/architecture/decisions/006-verification-posture-and-transfer-encoding.md
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
§2/§3
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
|
||||
§4
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§2
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
|
||||
§1/§4
|
||||
- docs/architecture/open-questions.md OQ-09
|
||||
- docs/research/poc-largeblob-findings.md finding A3
|
||||
- tasks/review-wave-6.md (the gate method; the W6-3 note this gate
|
||||
audits closed)
|
||||
- tasks/ops-foundation.md, tasks/ops-call-harness.md,
|
||||
tasks/ops-json-ops.md, tasks/ops-fetch.md, tasks/ops-put.md,
|
||||
tasks/ops-e2e-flows.md (the wave's tasks)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
Reference in new issue
Block a user