docs(arch): amend call specs for ADR-029/030/032/034 — peer-keyed routing, PeerEntry, forwarded-for, three roles
Sync the call crate specs to the accepted ADRs 029-034: - operation-registry: PeerCompositeEnv (peer-keyed overlays), invoke_peer/ PeerRef routing, retire remote_safe/trusted_peer, AccessControl-based peer auth, forwarded_for on OperationContext (ADR-029/030/032) - call-protocol: peer-keyed compose_root_env, forwarded_for in call.requested payload, build_root_context forwarded_for parameter (ADR-029/032) - client-and-adapters: CallClient verifier selection by PeerEntry presence, remote_identity: None load-bearing, three remote roles (ADR-034) - README: ADR-029/030/032/034 in applicable ADRs table
This commit is contained in:
@@ -7,7 +7,7 @@ last_updated: 2026-06-28
|
||||
|
||||
## Current State
|
||||
|
||||
**Pre-implementation of the storage/repo pattern.** The project has completed a pivot from a three-layer model to an ALPN-as-service model. The greenfield workspace contains `alknet-vault` (stable — implementation complete and verified, local-only by construction per ADR-025, HD-derivation key model per ADR-026) and research/reference material. Foundational ADRs (001–029) are in place, with ADR-029 (peer-graph routing) Proposed and the call crate implemented and reviewed.
|
||||
**Pre-implementation of the storage/repo pattern.** The project has completed a pivot from a three-layer model to an ALPN-as-service model. The greenfield workspace contains `alknet-vault` (stable — implementation complete and verified, local-only by construction per ADR-025, HD-derivation key model per ADR-026) and research/reference material. Foundational ADRs (001–035) are in place, with the call crate implemented and reviewed.
|
||||
|
||||
The storage and auth strategy research (`docs/research/alknet-storage-strategy/findings.md`) surfaced the repo/adapter pattern as the answer to cross-node state (peer identity, credentials). This has now landed as four ADRs:
|
||||
|
||||
|
||||
@@ -87,12 +87,17 @@ pub trait SessionOverlaySource: Send + Sync {
|
||||
/// The agent crate determines how to map a call to its session.
|
||||
fn overlay_for(&self, context: &OperationContext) -> Option<Arc<dyn OperationEnv + Send + Sync>>;
|
||||
}
|
||||
```
|
||||
|
||||
The `CallAdapter` holds the static curated registry and an optional
|
||||
session-overlay source. Per-connection imported-ops overlays (Layer 2,
|
||||
ADR-024) are held with the connection and composed into the root
|
||||
`OperationContext.env` per incoming call. See ADR-024 for the layering
|
||||
model and `compose_root_env` below.
|
||||
`OperationContext.env` per incoming call. The composition env is peer-keyed
|
||||
(`PeerCompositeEnv`, ADR-029 §1) to handle head→N-workers routing — a head
|
||||
node with multiple worker connections holds a peer-keyed
|
||||
`HashMap<PeerId, connection_overlay>`, not one overlay. See ADR-024 for the
|
||||
layering model, ADR-029 for the peer-keyed extension, and `compose_root_env`
|
||||
below.
|
||||
|
||||
### CallConnection
|
||||
|
||||
@@ -237,13 +242,19 @@ The `payload` of a `call.requested` event has this shape:
|
||||
{
|
||||
"operationId": "/fs/readFile",
|
||||
"input": { ... },
|
||||
"auth_token": "alk_..." // optional — see Identity Resolution below
|
||||
"auth_token": "alk_...", // optional — see Identity Resolution below
|
||||
"forwarded_for": { // optional (ADR-032) — present when a hub forwards a call
|
||||
"id": "alice",
|
||||
"scopes": ["fs:read", "docker:start"],
|
||||
"resources": {}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `operationId` — the operation to invoke, **with a leading slash** on the wire (e.g., `/fs/readFile`, `/agent/chat`, `/services/list`). This is the display form of the operation name. The registry stores names without the leading slash (`fs/readFile` — see [operation-registry.md](operation-registry.md#operationspec)); the wire format adds it. The `CallAdapter` strips the leading slash before registry lookup.
|
||||
- `input` — the operation input, matching the operation's `input_schema` (JSON Schema). Always a `serde_json::Value`.
|
||||
- `auth_token` — optional. If present, the `CallAdapter` resolves it via `IdentityProvider::resolve_from_token()` and the resulting `Identity` takes precedence over the connection-level identity for this request. See [Identity Resolution](#authcontext-and-identity-resolution) below.
|
||||
- `forwarded_for` — optional (ADR-032). Present when a `from_call` forwarding handler propagates the originator's identity to a spoke. Carries a serialized `Identity` (id, scopes, resources) — the end user the hub authenticated. **Metadata only** — `AccessControl::check` never reads it; the spoke authorizes the hub (its direct caller), not the end user. The hub may set `forwarded_for: None` if it doesn't want to disclose the originator. See [ADR-032](../../decisions/032-forwarded-for-identity.md).
|
||||
|
||||
The `call.requested` payload does **not** carry an abort policy field. The abort policy (`abort-dependents` vs `continue-running`, ADR-016) is set on `OperationContext` and propagated through `OperationEnv::invoke()` — the composing handler decides the child's policy, not the wire caller. See [Abort Cascade and Nested Calls](#abort-cascade-and-nested-calls) below.
|
||||
|
||||
@@ -283,7 +294,7 @@ The `payload` field of `EventEnvelope` has a different shape per event type:
|
||||
|
||||
| Event | `payload` shape |
|
||||
|-------|----------------|
|
||||
| `call.requested` | `{ "operationId": "/fs/readFile", "input": {...}, "auth_token": "alk_..." (optional) }` |
|
||||
| `call.requested` | `{ "operationId": "/fs/readFile", "input": {...}, "auth_token": "alk_..." (optional), "forwarded_for": { "id": "...", "scopes": [...], "resources": {} } (optional, ADR-032) }` |
|
||||
| `call.responded` | `{ "output": <Value> }` — the operation's output, matching `output_schema` |
|
||||
| `call.completed` | `{}` — empty object (subscription stream end signal) |
|
||||
| `call.aborted` | `{}` — empty object (cancellation signal; the `id` identifies which request) |
|
||||
@@ -429,6 +440,7 @@ fn build_root_context(
|
||||
request_id: String,
|
||||
operation_name: &str, // looked up in registry for the registration bundle
|
||||
identity: Option<Identity>, // resolved per-request above (caller's identity)
|
||||
forwarded_for: Option<Identity>, // from call.requested.forwarded_for (ADR-032)
|
||||
) -> OperationContext {
|
||||
let registration = self.registry.registration(operation_name);
|
||||
OperationContext {
|
||||
@@ -440,18 +452,27 @@ fn build_root_context(
|
||||
// This is on the context for PROPAGATION to children via invoke(),
|
||||
// not for the root's own ACL (which uses identity above).
|
||||
handler_identity: registration.composition_authority.clone(),
|
||||
// Forwarded-for identity (ADR-032): the originator when this call was
|
||||
// forwarded by a from_call handler. Metadata only — AccessControl::check
|
||||
// never reads it; ACL always authorizes `identity` (the direct caller).
|
||||
// None when the call wasn't forwarded or the forwarder chose not to
|
||||
// propagate it. Populated from the wire call.requested.forwarded_for
|
||||
// field; NOT inherited by composed children (wire-ingress only).
|
||||
forwarded_for,
|
||||
capabilities: registration.capabilities.clone(), // from the registration bundle
|
||||
metadata: HashMap::new(), // fresh per request
|
||||
deadline: Some(Instant::now() + self.default_timeout), // root deadline (W7)
|
||||
scoped_env: registration.scoped_env.clone()
|
||||
.unwrap_or_else(ScopedOperationEnv::empty), // from the bundle, empty for leaves
|
||||
// Per-call env composition (ADR-024): the root env is a composite
|
||||
// of the curated base + this connection's imported-ops overlay +
|
||||
// the active session overlay (if any). The CallAdapter builds this
|
||||
// Per-call env composition (ADR-024 + ADR-029): the root env is a
|
||||
// PeerCompositeEnv — the curated base + this connection's imported-
|
||||
// ops overlay (peer-keyed in the head's aggregation env, ADR-029 §1)
|
||||
// + the active session overlay (if any). The CallAdapter builds this
|
||||
// composite per incoming call — same shape as per-call identity
|
||||
// resolution via IdentityProvider. Handlers call env.invoke();
|
||||
// resolution via IdentityProvider. Handlers call env.invoke() (peer-
|
||||
// agnostic) or env.invoke_peer(peer, ...) (peer-specific, ADR-029 §2);
|
||||
// the composite routes to the right overlay.
|
||||
env: self.compose_root_env(/* connection, session */),
|
||||
env: self.compose_root_env(/* peer_id, connection_overlay, session */),
|
||||
abort_policy: AbortPolicy::default(), // abort-dependents (ADR-016 Decision 6)
|
||||
internal: false, // external call — ACL against caller identity
|
||||
}
|
||||
@@ -460,7 +481,7 @@ fn build_root_context(
|
||||
|
||||
The `internal: false` here is what makes a wire call a wire call — ACL checks against the caller's resolved `identity`. When a handler subsequently calls `context.env.invoke(...)`, the `OperationEnv::invoke()` path (see [operation-registry.md](operation-registry.md#operationenv)) constructs a nested `OperationContext` with `internal: true`, switching authority to `handler_identity`. The two construction paths — `CallAdapter` for wire-originated, `OperationEnv::invoke()` for composition-originated — are the only places `internal` is set. Handlers cannot set it themselves (the field is module-private for writes — see [operation-registry.md](operation-registry.md#operationcontext) and ADR-015).
|
||||
|
||||
The per-call `env` composition (ADR-024) is the operation-dispatch analogue of the per-call identity resolution the CallAdapter already does via `IdentityProvider`. Both are integration-point patterns: the trait object owns the routing, the CallAdapter supplies the right sources per call. A connection's imported-ops overlay is part of the root env only for calls arriving on that connection; a session overlay is part of the root env only when a session is active. See ADR-024.
|
||||
The per-call `env` composition (ADR-024 + ADR-029) is the operation-dispatch analogue of the per-call identity resolution the CallAdapter already does via `IdentityProvider`. Both are integration-point patterns: the trait object owns the routing, the CallAdapter supplies the right sources per call. A connection's imported-ops overlay is part of the root env only for calls arriving on that connection — and on a head node with multiple worker connections, the overlays are peer-keyed (`PeerCompositeEnv`, ADR-029 §1); a session overlay is part of the root env only when a session is active. See ADR-024, ADR-029, and the `PeerCompositeEnv` sketch in [operation-registry.md](operation-registry.md#operationenv).
|
||||
|
||||
### ResponseEnvelope
|
||||
|
||||
@@ -539,6 +560,7 @@ Handlers clean up resources when their call is cancelled (in Rust, the future is
|
||||
| Call protocol client and adapter contract | [ADR-017](../../decisions/017-call-protocol-client-and-adapter-contract.md) | `CallClient` opens connections; `from_call` imports remote ops; connection direction independent of call direction. Client/adapter surface specced in [client-and-adapters.md](client-and-adapters.md) |
|
||||
| Handler registration, provenance, and composition authority | [ADR-022](../../decisions/022-handler-registration-provenance-and-composition-authority.md) | Registration bundle carries provenance, composition authority, scoped env, capabilities; dispatch path reads from bundle |
|
||||
| Peer-graph routing model (supersedes ADR-028) | [ADR-029](../../decisions/029-peer-graph-routing-model.md) | Peer-keyed overlays + `PeerRef` routing; `AccessControl`-based peer authorization; retires `remote_safe`/`trusted_peer` |
|
||||
| Forwarded-for identity | [ADR-032](../../decisions/032-forwarded-for-identity.md) | `forwarded_for` field on `call.requested` and `OperationContext`; metadata only — `AccessControl::check` never reads it; the `from_call` handler populates it |
|
||||
| Operation error schemas | [ADR-023](../../decisions/023-operation-error-schemas.md) | Operations declare domain errors; `call.error` carries typed `details` |
|
||||
|
||||
## Open Questions
|
||||
@@ -552,16 +574,45 @@ See [open-questions.md](../../open-questions.md) for full details.
|
||||
- **OQ-25** (dissolved by ADR-029): `remote_safe` marking shape — moot;
|
||||
`remote_safe`/`trusted_peer` retired; peer authorization is
|
||||
`AccessControl::check(peer_identity)`.
|
||||
- **OQ-26..29** (OQ-26/27/29 open two-way; OQ-28 cross-peer dissolved / same-peer stays):
|
||||
`OperationAdapter` error type, `from_call` re-import trigger, `from_call`
|
||||
namespace collision, `CallClient` TLS client-auth. See
|
||||
[client-and-adapters.md](client-and-adapters.md) and ADR-029.
|
||||
- **OQ-30..32** (open): `PeerRef::Any` routing policy, `services/list-peers`
|
||||
re-export semantics, multi-hop federation. See ADR-029.
|
||||
- **OQ-26** (resolved): `OperationAdapter` error type — `AdapterError`
|
||||
variants (`DiscoveryFailed`, `SchemaParse`, `Transport`, `Unauthorized`,
|
||||
`SamePeerCollision`); `#[non_exhaustive]`. See
|
||||
[client-and-adapters.md](client-and-adapters.md).
|
||||
- **OQ-27** (resolved): `from_call` re-import trigger — auto-re-import on
|
||||
connection establishment. See [client-and-adapters.md](client-and-adapters.md).
|
||||
- **OQ-28** (resolved): `from_call` namespace collision — same-peer collision
|
||||
= error; cross-peer dissolved by ADR-029 (separate sub-overlays). See
|
||||
[client-and-adapters.md](client-and-adapters.md).
|
||||
- **OQ-29** (resolved): `CallClient` TLS client-auth — wire quinn client-auth
|
||||
(present Ed25519 key as raw public key client cert); key-type-aware server
|
||||
cert verification; fingerprint normalization. See
|
||||
[client-and-adapters.md](client-and-adapters.md).
|
||||
- **OQ-30** (resolved): `PeerRef::Any` routing policy — insertion-order
|
||||
first-match. See [client-and-adapters.md](client-and-adapters.md).
|
||||
- **OQ-31** (resolved): `services/list-peers` re-export semantics — opt-in;
|
||||
`services/list` is "own ops only." See [client-and-adapters.md](client-and-adapters.md).
|
||||
- **OQ-32** (open, feature extension): Multi-hop federation — the one-hop
|
||||
model is the architectural commitment; multi-hop is a feature extension
|
||||
that doesn't break downstream. See [client-and-adapters.md](client-and-adapters.md).
|
||||
- **OQ-33** (resolved by ADR-030): `PeerId` source — `Identity.id` from
|
||||
`IdentityProvider` resolution (= `PeerEntry.peer_id`, stable across key
|
||||
rotation), not a connection-assigned UUID.
|
||||
- **OQ-34** (resolved by ADR-030 + ADR-033): Persistent peer registry —
|
||||
the storage boundary is `core trait + in-memory default`; persistence
|
||||
adapters are separate crates.
|
||||
- **OQ-37** (resolved by ADR-034): X.509 outgoing-only case — three remote
|
||||
roles named (public X.509 endpoint, transport relay, hub); pure-client
|
||||
X.509 connections are not in the peer graph on the client side. See
|
||||
[client-and-adapters.md](client-and-adapters.md).
|
||||
|
||||
## References
|
||||
|
||||
- [operation-registry.md](operation-registry.md) — OperationSpec, Handler, AccessControl, service discovery
|
||||
- [client-and-adapters.md](client-and-adapters.md) — CallClient, from_call, OperationAdapter, peer-keyed composition env
|
||||
- ADR-005: irpc as call protocol foundation
|
||||
- ADR-012: Call protocol stream model
|
||||
- ADR-029: Peer-graph routing model (peer-keyed overlays + `PeerRef` routing)
|
||||
- ADR-030: PeerEntry and Identity.id decoupling (`PeerId` source)
|
||||
- ADR-032: Forwarded-for identity (`forwarded_for` on `call.requested` and `OperationContext`)
|
||||
- ADR-034: Outgoing-only X.509 and the three peer roles
|
||||
- Reference implementation: `/workspace/@alkdev/alknet-main/crates/alknet-core/src/call/`
|
||||
@@ -245,21 +245,32 @@ invariant (below). The concrete shapes of `TlsIdentity`, `AuthToken`, and
|
||||
`RemoteIdentity` are implementation-detail two-way doors; the one-way
|
||||
constraints are that they come from `Capabilities`, not env vars (ADR-014).
|
||||
|
||||
**v1 TLS client-auth gap** (OQ-29): v1 `connect()` builds the quinn client
|
||||
config with `with_no_client_auth()` and an `AcceptAnyServerCertVerifier` — the
|
||||
client does not present its TLS identity as a client cert, and does not pin the
|
||||
remote's expected identity from `credentials.remote_identity`. This is a
|
||||
two-way-door remainder: wiring the local node's RawKey/X509 identity as a
|
||||
rustls client-auth cert (for servers that verify client identity) and
|
||||
plugging `credentials.remote_identity` into a real `ServerCertVerifier` is
|
||||
additive. The one-way constraint (credentials from `Capabilities`, not env
|
||||
vars, ADR-014) is unaffected — the `auth_token` dimension flows through the
|
||||
call-protocol `auth_token` payload field, not TLS, so the no-env-vars
|
||||
invariant holds independently of this gap.
|
||||
**TLS client-auth presentation** (OQ-29 #1, wired): the client presents
|
||||
its Ed25519 key as an RFC 7250 raw public key client cert — the client-side
|
||||
equivalent of the server's `RawKeyCertResolver`. This is **wired now**, not
|
||||
additive: it is what activates the `PeerEntry` fingerprint → `peer_id`
|
||||
resolution path on quinn connections (ADR-030 §5). Without it, the ADR-029
|
||||
peer graph doesn't populate for quinn connections — `PeerId` resolution
|
||||
fails because the server has no client cert to extract a fingerprint from.
|
||||
The iroh path already works (iroh uses RFC 7250 raw keys and exchanges
|
||||
Ed25519 public keys during the TLS handshake automatically); the gap was
|
||||
quinn-only, and OQ-29 #1 resolves it by replacing `with_no_client_auth()`
|
||||
with presenting the key. The one-way constraint (credentials from
|
||||
`Capabilities`, not env vars, ADR-014) is unaffected — the `auth_token`
|
||||
dimension flows through the call-protocol `auth_token` payload field, not
|
||||
TLS, so the no-env-vars invariant holds independently of the TLS layer.
|
||||
|
||||
**Outgoing X.509 and the peer model** (ADR-034): the client-side
|
||||
`ServerCertVerifier` is selected by whether the local node has a
|
||||
`PeerEntry` for the remote, not by key type alone. A pure-client
|
||||
**Remote-identity verification** (OQ-29 #2, additive): verifying the
|
||||
server's fingerprint against an expected value (`credentials.remote_identity`)
|
||||
is **additive** — the server-side fingerprint extraction is what matters for
|
||||
`PeerId`, not the client-side verification. The verifier for raw keys can
|
||||
start as "accept any, extract fingerprint" and add fingerprint-pinning later.
|
||||
This is a two-way-door remainder; the one-way constraint (credentials from
|
||||
`Capabilities`, not env vars) is unaffected.
|
||||
|
||||
**Server cert verifier selection** (OQ-29 #2 + ADR-034 §3): the client-side
|
||||
`ServerCertVerifier` is selected by whether the local node has a `PeerEntry`
|
||||
for the remote, not by key type alone. A pure-client
|
||||
connection to a **public X.509 endpoint** (no `PeerEntry` on the local
|
||||
side — e.g., dialing `api.alk.dev` or a third-party API) uses
|
||||
`WebPkiServerVerifier` (CA verification), gets **no `PeerId`** on the
|
||||
@@ -269,7 +280,9 @@ on such a connection land in the connection's Layer 2 overlay
|
||||
(ADR-024) and are invoked through the `CallConnection` handle directly,
|
||||
not via `PeerRef::Specific`. A connection to a **hub** (a `PeerEntry`
|
||||
with mixed Ed25519 + X.509 fingerprints) uses fingerprint pinning on
|
||||
both cert paths and does enter the peer graph. See
|
||||
both cert paths and does enter the peer graph. An unknown Ed25519
|
||||
raw-key remote fails closed (no CA to fall back to — raw-key remotes
|
||||
are always known peers). See
|
||||
[ADR-034](../../decisions/034-outgoing-only-x509-and-three-peer-roles.md)
|
||||
for the verifier selection rule and the three-role naming.
|
||||
|
||||
|
||||
@@ -361,24 +361,68 @@ pub trait OperationEnv: Send + Sync {
|
||||
) -> ResponseEnvelope;
|
||||
|
||||
/// Does this env contain the named operation? Used by
|
||||
/// `CompositeOperationEnv` to probe overlays before dispatching
|
||||
/// (ADR-024). The composite checks `session.contains()` →
|
||||
/// `connection.contains()` → base, dispatching to the first overlay
|
||||
/// that contains the op. Default impl returns `true` (a single-layer
|
||||
/// env like `LocalOperationEnv` contains everything it can dispatch).
|
||||
/// `PeerCompositeEnv` to probe overlays before dispatching
|
||||
/// (ADR-024 + ADR-029). The composite checks `session.contains()` →
|
||||
/// each peer's sub-overlay (in `connection_order`) → base,
|
||||
/// dispatching to the first overlay that contains the op. Default
|
||||
/// impl returns `true` (a single-layer env like `LocalOperationEnv`
|
||||
/// contains everything it can dispatch).
|
||||
fn contains(&self, name: &str) -> bool { true }
|
||||
|
||||
/// Peer-routing composition (ADR-029 §2). Routes to a specific peer
|
||||
/// (`PeerRef::Specific`) or to the first peer that serves the op
|
||||
/// (`PeerRef::Any`). The default impl ignores the peer selector and
|
||||
/// delegates to `invoke_with_policy`, preserving back-compat for
|
||||
/// single-layer envs (`LocalOperationEnv`, `OverlayOperationEnv`)
|
||||
/// that don't override it. `PeerCompositeEnv` overrides with real
|
||||
/// peer-keyed routing.
|
||||
async fn invoke_peer(
|
||||
&self,
|
||||
peer: &PeerRef,
|
||||
namespace: &str,
|
||||
operation: &str,
|
||||
input: Value,
|
||||
parent: &OperationContext,
|
||||
policy: AbortPolicy,
|
||||
) -> ResponseEnvelope {
|
||||
// default: ignore peer selector, dispatch via invoke_with_policy
|
||||
let _ = peer; // unused — single-layer envs don't route by peer
|
||||
self.invoke_with_policy(namespace, operation, input, parent, policy).await
|
||||
}
|
||||
|
||||
/// Does this env contain the named op *on the named peer*? Used by
|
||||
/// `PeerCompositeEnv` to probe a specific peer's sub-overlay before
|
||||
/// dispatching via `invoke_peer` with `PeerRef::Specific`. Default
|
||||
/// impl delegates to `contains` (single-layer envs ignore the peer
|
||||
/// dimension). `PeerCompositeEnv` overrides to check the specific
|
||||
/// peer's sub-overlay.
|
||||
fn peer_contains(&self, _peer: &PeerId, name: &str) -> bool { self.contains(name) }
|
||||
}
|
||||
```
|
||||
|
||||
The `parent` parameter propagates the calling context: the nested call gets `parent_request_id: Some(parent.request_id)`, inherits `parent.handler_identity` as the caller identity, and is marked `internal: true`.
|
||||
|
||||
The `invoke_peer` / `peer_contains` methods (ADR-029 §2) take a `PeerRef`
|
||||
selector and a `PeerId`. These types are defined alongside the
|
||||
`PeerCompositeEnv` struct — see [client-and-adapters.md](client-and-adapters.md#peer-keyed-composition-env-adr-029)
|
||||
and [ADR-029](../../decisions/029-peer-graph-routing-model.md) §2:
|
||||
|
||||
```rust
|
||||
pub enum PeerRef {
|
||||
Specific(PeerId), // route to this peer; NOT_FOUND if it doesn't serve the op
|
||||
Any, // first peer (insertion order) that serves it
|
||||
}
|
||||
pub type PeerId = String; // = Identity.id from IdentityProvider resolution
|
||||
// = PeerEntry.peer_id (stable, not crypto material — ADR-030)
|
||||
```
|
||||
|
||||
**Metadata does not propagate through composition.** Nested calls get fresh metadata (`HashMap::new()`), not the parent's metadata bag. This is a security constraint (ADR-014): `metadata: HashMap<String, Value>` accepts any `serde_json::Value`, including secret material. If metadata propagated through `env.invoke()`, a handler that accidentally placed a secret in metadata would leak it to every child operation — and if a child is a `from_call` operation (ADR-017), the metadata would cross the wire to the remote node. The tracing link between parent and child is `parent_request_id`, not metadata propagation. Anything a handler needs to pass to a child goes in the call `input`, not in ambient context.
|
||||
|
||||
**Local dispatch only.** The initial `OperationEnv` implementation for the
|
||||
curated layer (Layer 0) dispatches directly through the local
|
||||
`OperationRegistry`. The composite env (curated + session + connection
|
||||
overlays) is a separate type built by the `CallAdapter` per call — see
|
||||
ADR-024 and the `CompositeOperationEnv` sketch below.
|
||||
`OperationRegistry`. The composite env (curated + session + peer-keyed
|
||||
connection overlays) is a separate type built by the `CallAdapter` per call —
|
||||
see ADR-024, ADR-029, and the `PeerCompositeEnv` sketch below.
|
||||
|
||||
```rust
|
||||
/// Layer 0 dispatch — the curated registry. This is the base env that
|
||||
@@ -446,53 +490,102 @@ impl OperationEnv for LocalOperationEnv {
|
||||
```
|
||||
|
||||
The composite env (built by the `CallAdapter` per incoming call) wraps the
|
||||
curated base and any active overlays:
|
||||
curated base and any active overlays. Per ADR-029, the connection overlay is
|
||||
**peer-keyed** — a head node with N worker connections holds a
|
||||
`HashMap<PeerId, connection_overlay>`, not one overlay. The singular-connection
|
||||
case (one peer) is the degenerate case with a single-entry map.
|
||||
|
||||
```rust
|
||||
/// Per-call composite env (ADR-024). Built by the CallAdapter in
|
||||
/// Per-call composite env (ADR-024 + ADR-029). Built by the CallAdapter in
|
||||
/// build_root_context from the active layers. The child inherits this by
|
||||
/// Arc::clone through invoke().
|
||||
pub struct CompositeOperationEnv {
|
||||
session: Option<Arc<dyn OperationEnv + Send + Sync>>, // Layer 1 — active session, if any
|
||||
connection: Option<Arc<dyn OperationEnv + Send + Sync>>, // Layer 2 — this connection's imported ops
|
||||
base: Arc<dyn OperationEnv + Send + Sync>, // Layer 0 — curated registry (LocalOperationEnv)
|
||||
/// Arc::clone through invoke(). The connection overlay is peer-keyed
|
||||
/// (ADR-029 §1) to handle head→N-workers routing.
|
||||
pub struct PeerCompositeEnv {
|
||||
pub base: Arc<dyn OperationEnv + Send + Sync>, // Layer 0 curated
|
||||
pub session: Option<Arc<dyn OperationEnv + Send + Sync>>, // Layer 1
|
||||
pub connections: HashMap<PeerId, Arc<dyn OperationEnv + Send + Sync>>, // Layer 2, peer-keyed
|
||||
connection_order: Vec<PeerId>, // insertion order for PeerRef::Any first-match
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl OperationEnv for CompositeOperationEnv {
|
||||
impl OperationEnv for PeerCompositeEnv {
|
||||
// `invoke` uses the default impl (delegates to `invoke_with_policy`
|
||||
// with `parent.abort_policy.clone()`).
|
||||
|
||||
async fn invoke_with_policy(&self, namespace: &str, operation: &str, input: Value, parent: &OperationContext, policy: AbortPolicy) -> ResponseEnvelope {
|
||||
// PeerRef::Any routing (ADR-029 §2): session → peers in insertion
|
||||
// order → curated base. First overlay that *contains* the op wins.
|
||||
let name = format!("{namespace}/{operation}");
|
||||
// Reachability check against parent.scoped_env (same as LocalOperationEnv).
|
||||
if !parent.scoped_env.allows(&name) {
|
||||
return ResponseEnvelope::not_found(name);
|
||||
}
|
||||
// Dispatch in overlay order: session → connection → curated base.
|
||||
// First overlay that *contains* the op wins. `contains()` (ADR-024)
|
||||
// is the probe — it avoids the sentinel-return ambiguity and ensures
|
||||
// cross-impl interop: any OperationEnv impl that correctly reports
|
||||
// `contains` works with this composite.
|
||||
if let Some(session) = &self.session {
|
||||
if session.contains(&name) {
|
||||
return session.invoke_with_policy(namespace, operation, input, parent, policy).await;
|
||||
}
|
||||
}
|
||||
if let Some(connection) = &self.connection {
|
||||
if connection.contains(&name) {
|
||||
return connection.invoke_with_policy(namespace, operation, input, parent, policy).await;
|
||||
// Peer-keyed overlay: iterate peers in insertion order, dispatch to
|
||||
// the first peer whose sub-overlay contains the op. This is the
|
||||
// head→N-workers fan-out primitive (ADR-029 §2, OQ-30: insertion-
|
||||
// order first-match).
|
||||
for peer_id in &self.connection_order {
|
||||
if let Some(conn_env) = self.connections.get(peer_id) {
|
||||
if conn_env.contains(&name) {
|
||||
return conn_env.invoke_with_policy(namespace, operation, input, parent, policy).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
self.base.invoke_with_policy(namespace, operation, input, parent, policy).await
|
||||
}
|
||||
|
||||
// `invoke_peer` overrides the default impl with real peer-keyed
|
||||
// routing (ADR-029 §2). `PeerRef::Specific` routes to the named peer's
|
||||
// sub-overlay only (no fallthrough — NOT_FOUND if that peer doesn't
|
||||
// serve the op). `PeerRef::Any` reuses `invoke_with_policy` (the
|
||||
// insertion-order fan-out above).
|
||||
async fn invoke_peer(
|
||||
&self,
|
||||
peer: &PeerRef,
|
||||
namespace: &str,
|
||||
operation: &str,
|
||||
input: Value,
|
||||
parent: &OperationContext,
|
||||
policy: AbortPolicy,
|
||||
) -> ResponseEnvelope {
|
||||
let name = format!("{namespace}/{operation}");
|
||||
if !parent.scoped_env.allows(&name) {
|
||||
return ResponseEnvelope::not_found(name);
|
||||
}
|
||||
match peer {
|
||||
PeerRef::Specific(peer_id) => {
|
||||
// Route to this peer's sub-overlay only. No fallthrough —
|
||||
// explicit routing must be honored or fail loudly (ADR-029 §2).
|
||||
match self.connections.get(peer_id) {
|
||||
Some(conn_env) if conn_env.contains(&name) => {
|
||||
conn_env.invoke_with_policy(namespace, operation, input, parent, policy).await
|
||||
}
|
||||
_ => ResponseEnvelope::not_found(name),
|
||||
}
|
||||
}
|
||||
PeerRef::Any => {
|
||||
// Same as invoke_with_policy: session → peers in order → base.
|
||||
self.invoke_with_policy(namespace, operation, input, parent, policy).await
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn contains(&self, name: &str) -> bool {
|
||||
// The composite contains the op if any layer does.
|
||||
// The composite contains the op if any layer does (peer-agnostic).
|
||||
self.session.as_ref().map_or(false, |s| s.contains(name))
|
||||
|| self.connection.as_ref().map_or(false, |c| c.contains(name))
|
||||
|| self.connections.values().any(|c| c.contains(name))
|
||||
|| self.base.contains(name)
|
||||
}
|
||||
|
||||
fn peer_contains(&self, peer: &PeerId, name: &str) -> bool {
|
||||
// Does the named peer's sub-overlay contain the op?
|
||||
self.connections.get(peer).map_or(false, |c| c.contains(name))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -500,8 +593,10 @@ The `contains()` method (review #003 C9) is the overlay-dispatch contract.
|
||||
It replaces the previous "sentinel or contains check — two-way door" framing,
|
||||
which was ambiguous enough to produce non-interoperable `OperationEnv` impls.
|
||||
The structural decision (composite trait object, overlay order, `Arc::clone`
|
||||
inheritance) is locked by ADR-024; the dispatch contract (`contains` probe
|
||||
before `invoke_with_policy`) is now locked too.
|
||||
inheritance) is locked by ADR-024; the peer-keyed overlay extension
|
||||
(`PeerCompositeEnv`, `invoke_peer`, `peer_contains`) is locked by ADR-029; the
|
||||
dispatch contract (`contains` probe before `invoke_with_policy`) is locked by
|
||||
both.
|
||||
|
||||
Two things happen in `invoke()`:
|
||||
|
||||
@@ -510,7 +605,7 @@ Two things happen in `invoke()`:
|
||||
|
||||
Future work may add irpc service dispatch and remote call protocol dispatch as additional backends. The handler-facing API stays the same.
|
||||
|
||||
**`OperationEnv` must remain a trait.** This is a constraint, not a suggestion. The trait-based design enables registry layering (ADR-024): the CallAdapter composes the root env per call from the curated base + active connection/session overlays, and overlays wrap the base via trait layering. Session-scoped registries (OQ-19) and connection-scoped remote imports (ADR-017 `from_call`) are both overlays on the same base, using the same mechanism. Making `OperationEnv` concrete or hardcoding the global registry into the dispatch path would close both the session-overlay and connection-overlay patterns. This is the same integration-point pattern as `IdentityProvider` (ADR-004). See OQ-19 and ADR-024.
|
||||
**`OperationEnv` must remain a trait.** This is a constraint, not a suggestion. The trait-based design enables registry layering (ADR-024): the CallAdapter composes the root env per call from the curated base + active peer-keyed connection overlays + session overlay, and overlays wrap the base via trait layering. Session-scoped registries (OQ-19) and connection-scoped remote imports (ADR-017 `from_call`) are both overlays on the same base, using the same mechanism. The peer-keyed extension (`PeerCompositeEnv`, `invoke_peer`, ADR-029) composes on top of the same trait — it overrides the new peer-routing methods, not the base dispatch. Making `OperationEnv` concrete or hardcoding the global registry into the dispatch path would close both the session-overlay and connection-overlay patterns, and would prevent the peer-keyed routing model from composing. This is the same integration-point pattern as `IdentityProvider` (ADR-004). See OQ-19, ADR-024, and ADR-029.
|
||||
|
||||
### Service Discovery
|
||||
|
||||
@@ -686,10 +781,27 @@ See [open-questions.md](../../open-questions.md) for full details.
|
||||
`remote_safe`/`trusted_peer` are retired; peer authorization is
|
||||
`AccessControl::check(peer_identity)`, the existing mechanism. See
|
||||
[client-and-adapters.md](client-and-adapters.md) and ADR-029 §3.
|
||||
- **OQ-26..28** (OQ-26/27 stay two-way; OQ-28 cross-peer dissolved by ADR-029 /
|
||||
same-peer stays): `OperationAdapter` error type, `from_call` re-import
|
||||
trigger, `from_call` namespace collision. v1 defaults recorded in
|
||||
- **OQ-26** (resolved): `OperationAdapter` error type — `AdapterError`
|
||||
variants: `DiscoveryFailed`, `SchemaParse`, `Transport`, `Unauthorized`,
|
||||
`SamePeerCollision` (replaces flat `Conflict`). `#[non_exhaustive]`. See
|
||||
[client-and-adapters.md](client-and-adapters.md).
|
||||
- **OQ-27** (resolved): `from_call` re-import trigger — auto-re-import on
|
||||
connection establishment. `CallConnection::refresh()` is a feature
|
||||
addition, not an unmade decision. See [client-and-adapters.md](client-and-adapters.md).
|
||||
- **OQ-28** (resolved): `from_call` namespace collision — same-peer
|
||||
collision = error; cross-peer dissolved by ADR-029 (separate sub-overlays).
|
||||
`namespace_prefix` is optional local-naming sugar. See
|
||||
[client-and-adapters.md](client-and-adapters.md).
|
||||
- **OQ-29..37** are tracked in [client-and-adapters.md](client-and-adapters.md)
|
||||
(they concern the `CallClient` / adapter surface and peer-graph routing,
|
||||
not the registry layering this document covers). In brief: OQ-29
|
||||
(CallClient TLS client-auth) resolved; OQ-30 (`PeerRef::Any` routing
|
||||
policy) resolved; OQ-31 (`services/list-peers` re-export semantics)
|
||||
resolved; OQ-32 (multi-hop federation) open (feature extension);
|
||||
OQ-33 (PeerId source) resolved by ADR-030; OQ-34 (persistent peer
|
||||
registry) resolved by ADR-030+033; OQ-35 (API key asymmetry) dissolved;
|
||||
OQ-36 (concrete persistence adapter shapes) resolved by ADR-035;
|
||||
OQ-37 (X.509 outgoing-only) resolved by ADR-034.
|
||||
|
||||
## References
|
||||
|
||||
@@ -699,4 +811,7 @@ See [open-questions.md](../../open-questions.md) for full details.
|
||||
- ADR-010: ALPN router and endpoint (static registration — applies to the `HandlerRegistry`, not the `OperationRegistry`; see ADR-024 for the distinction)
|
||||
- ADR-012: Call protocol stream model
|
||||
- ADR-024: Operation registry layering (curated + session/connection overlays; `OperationEnv` as trait-object integration point)
|
||||
- ADR-029: Peer-graph routing model (peer-keyed overlays + `PeerRef` routing; `PeerCompositeEnv` supersedes the singular-connection `CompositeOperationEnv`)
|
||||
- ADR-030: PeerEntry and Identity.id decoupling (`PeerId` source = `Identity.id` = `PeerEntry.peer_id`)
|
||||
- ADR-032: Forwarded-for identity (`forwarded_for` on `OperationContext` and `call.requested`; metadata only)
|
||||
- Reference implementation: `/workspace/@alkdev/alknet-main/crates/alknet-core/src/call/`
|
||||
Reference in New Issue
Block a user