alkdev/alknet
6
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(arch): sync specs to transport generalization sweep (ADR-064, ADR-065)

Browse Source 
Three code commits landed a clean sweep discovered when building an
external app against the crates. This syncs the architecture specs to
match the codebase and amends the affected decisions.

ADR-064 (supersedes ADR-005): irpc was never integrated — no .rs file in
the workspace ever imported it. The wire protocol (wire.rs) is hand-rolled
length-prefixed JSON; the EventEnvelope shape was derived from the
@alkdev/pubsub TypeScript prior art (ADR-013), not from irpc. ADR-005's
premise ('irpc as the call protocol foundation') was never implemented as
stated. Superseded with a clear header; the body is kept as historical
record.

ADR-065: Connection::from_stream / from_bidi — Connection now accepts any
AsyncRead + AsyncWrite pair via ConnectionKind::Stream (yield-once
accept_bi contract: QUIC yields many streams, everything else yields one
then ConnectionClosed). Unblocks TCP+TLS, SSH channel dispatch,
WebTransport streams, and wasm streams through the same HandlerRegistry
as QUIC connections, with zero handler code changes. MockConnection /
ConnectionKind::Mock removed (tests use from_stream with sink/empty).
Stream-level Mock variants renamed to Stream (they were already generic).

Amended ADRs: ADR-003 (irpc removed from dep table), ADR-007 (from_stream
opens the server-side door; MockConnection removed), ADR-010 (TCP+TLS can
now dispatch through the registry via from_bidi; iroh 1.0 migration noted).

Updated specs: core-types.md (Connection/SendStream/RecvStream sections),
endpoint.md (TCP section, iroh note), call-protocol.md, operation-registry
(irpc Integration section replaced), overview.md, http-server.md,
webtransport.md, call README. OQ-09 (WASM) resolution amended: the
Connection door is now open via from_stream; the accept-loop runtime door
remains closed (tokio doesn't run on WASM).

See docs/research/transport-generalization/findings.md for the full trace.
This commit is contained in:
glm-5.2 committed 2026-07-09 07:40:51 +00:00
1 parent 865fef6210
commit 2aa6363e57
20 files changed
+782 -87
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/architecture/crates/call/README.md
+10 -8
View File
@@ -1,19 +1,19 @@
---
status: draft
last_updated: 2026-06-27
review: call/review-call passed 2026-06-23 — registry, protocol, ADR (005/012/014/015/016/017/022/023/024), security, and pattern-consistency checks all conformant; 159 unit/integration tests green; `cargo build`, `cargo clippy -- -D warnings`, `cargo fmt --check`, `cargo test` clean. Call-completion gap (ADR-017 client/adapter surface) addressed 2026-06-26; ADR-029 migration pending.
last_updated: 2026-07-09
review: call/review-call passed 2026-06-23 — registry, protocol, ADR (005/012/014/015/016/017/022/023/024), security, and pattern-consistency checks all conformant; 159 unit/integration tests green; `cargo build`, `cargo clippy -- -D warnings`, `cargo fmt --check`, `cargo test` clean. Call-completion gap (ADR-017 client/adapter surface) addressed 2026-06-26; ADR-029 migration pending. Transport generalization sweep (ADR-064 supersedes ADR-005; ADR-065 `from_stream`) synced 2026-07-09.
---
# alknet-call
Structured RPC over QUIC: operations, request/response, streaming subscriptions, and service discovery. Implements `ProtocolHandler` on ALPN `alknet/call`.
Structured RPC: operations, request/response, streaming subscriptions, and service discovery. Implements `ProtocolHandler` on ALPN `alknet/call`. Runs over QUIC (quinn/iroh) and, via `Connection::from_stream` (ADR-065), over any `AsyncRead + AsyncWrite` transport.
## Documents
| Document | Status | Description |
|----------|--------|-------------|
| [call-protocol.md](call-protocol.md) | draft | CallAdapter, EventEnvelope framing, stream model, PendingRequestMap, bidirectional calls |
| [operation-registry.md](operation-registry.md) | draft | OperationSpec, Handler, OperationRegistry, AccessControl, service discovery, irpc integration |
| [call-protocol.md](call-protocol.md) | draft | CallAdapter, hand-rolled EventEnvelope framing (no irpc — ADR-064), stream model, PendingRequestMap, bidirectional calls |
| [operation-registry.md](operation-registry.md) | draft | OperationSpec, Handler, OperationRegistry, AccessControl, service discovery, hand-rolled framing (no irpc — ADR-064) |
| [client-and-adapters.md](client-and-adapters.md) | draft | CallClient (outbound connection opener), from_call / from_jsonschema, OperationAdapter trait, adapter location map, no-env-vars invariant, exchange-of-operations pattern |
## Applicable ADRs
@@ -22,10 +22,12 @@ Structured RPC over QUIC: operations, request/response, streaming subscriptions,
|-----|-------|-----------|
| [001](../../decisions/001-alpn-protocol-dispatch.md) | ALPN-Based Protocol Dispatch | CallAdapter registers on ALPN `alknet/call` |
| [002](../../decisions/002-protocol-handler-trait.md) | ProtocolHandler Trait | CallAdapter implements ProtocolHandler |
| [003](../../decisions/003-crate-decomposition.md) | Crate Decomposition | alknet-call depends on alknet-core and irpc |
| [003](../../decisions/003-crate-decomposition.md) | Crate Decomposition | alknet-call depends on alknet-core (no irpc — ADR-064) |
| [013](../../decisions/013-rust-canonical-implementation.md) | Rust as Canonical Implementation Language | Adapter traits defined in Rust; TS is reference/browser adaptation |
| [004](../../decisions/004-auth-as-shared-core.md) | Auth as Shared Core | AuthContext passed to call handlers |
| [005](../../decisions/005-irpc-as-call-protocol-foundation.md) | irpc as Call Protocol Foundation | irpc provides framing and service dispatch |
| [005](../../decisions/005-irpc-as-call-protocol-foundation.md) | ~~irpc as Call Protocol Foundation~~ | ~~Accepted~~ → **Superseded** by [ADR-064](../../decisions/064-irpc-never-integrated-hand-rolled-framing.md) (irpc was never integrated; framing is hand-rolled) |
| [064](../../decisions/064-irpc-never-integrated-hand-rolled-framing.md) | Hand-Rolled EventEnvelope Framing | Wire format, registry, dispatch are hand-rolled in alknet-call; supersedes ADR-005 |
| [065](../../decisions/065-connection-from-stream-generic-single-stream.md) | `Connection::from_stream` | Generic single-stream connections; unblocks TCP+TLS/SSH/WT/wasm dispatch |
| [006](../../decisions/006-alpn-convention-and-connection-model.md) | ALPN String Convention | `alknet/call` ALPN, one ALPN per connection |
| [007](../../decisions/007-bistream-type-definition.md) | BiStream Type Definition | CallAdapter receives Connection, not BiStream |
| [008](../../decisions/008-secret-service-integration.md) | Vault Integration Point | Vault accessed at assembly layer, not on the wire |
@@ -72,7 +74,7 @@ Structured RPC over QUIC: operations, request/response, streaming subscriptions,
2. **Protocol is symmetric**: Both sides can initiate calls. The server calling a client uses the same EventEnvelope format and correlation.
3. **Stream-agnostic correlation**: PendingRequestMap correlates by request ID, not by stream. The protocol works with any stream arrangement.
4. **Operation registry is layered**: The curated layer (`Local` provenance) is static — registered at startup by the CLI binary, immutable for the process lifetime. Session (`Session`) and imported (`FromCall` etc.) ops are dynamic overlays at their respective scopes (per-session, per-connection). The registry supports JSON Schema discovery. See ADR-024.
5. **irpc is one dispatch backend**: Local operations dispatch directly. irpc service calls (in-process, type-safe) are internal. The call protocol is the external interface.
5. **Hand-rolled dispatch (no irpc)**: Operations dispatch through the hand-rolled `OperationRegistry` (ADR-064). The call protocol is the external interface; internal handler dispatch uses `Handler`/`StreamingHandler` trait objects (ADR-049), not an irpc service.
6. **Local dispatch only**: The operation registry dispatches to local handlers. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer, not a modification to alknet-call's path format.
7. **No secret material on the wire**: The call protocol carries no private keys, API keys, mnemonics, or decrypted credentials. Handlers receive outbound credentials through `OperationContext.capabilities`, injected at the assembly layer. See ADR-014.
8. **Abort cascades to descendants**: `call.aborted` for a parent request cascades to all non-terminal descendants. Default `abort-dependents`; `continue-running` opt-in. See ADR-016.
docs/architecture/crates/call/call-protocol.md
+12 -6
View File
@@ -1,6 +1,6 @@
---
status: draft
last_updated: 2026-07-02
last_updated: 2026-07-09
---
# Call Protocol
@@ -23,7 +23,10 @@ The protocol must be:
- **Stream-agnostic**: QUIC provides stream multiplexing; the protocol shouldn't impose additional constraints
- **Discoverable**: Clients can query what operations exist and their schemas
See ADR-005 for the decision to use irpc as the call protocol's foundation and ADR-012 for the stream model decision.
See ADR-064 for the decision that the call protocol uses hand-rolled
`EventEnvelope` framing (irpc was never integrated — ADR-005, which
accepted "irpc as the call protocol foundation," is superseded) and ADR-012
for the stream model decision.
## Architecture
@@ -210,7 +213,10 @@ The `Value` type is `serde_json::Value`. The envelope is JSON because it must be
Binary payloads (postcard, protobuf) are base64-encoded as a JSON string within the `payload` field. The convention is: if an operation's output schema specifies a binary field, the handler encodes it as a base64 string and the client decodes it. The `EventEnvelope` structure is not aware of this convention — it carries a `serde_json::Value` and does not interpret the payload. This is a handler-level concern, not a protocol-level concern.
This is the same framing used by irpc. The Rust implementation in alknet-call is canonical — the `@alkdev/pubsub` TypeScript adapters serve as a reference and browser adaptation, not a parallel implementation (see ADR-013).
This is hand-rolled length-prefixed JSON framing (ADR-064), coincidentally
the same shape irpc uses. The Rust implementation in alknet-call is
canonical — the `@alkdev/pubsub` TypeScript adapters serve as a reference
and browser adaptation, not a parallel implementation (see ADR-013).
### Event Types
@@ -546,7 +552,7 @@ Handlers clean up resources when their call is cancelled (in Rust, the future is
- Operation specs use JSON Schema. The envelope is always JSON. Binary payloads may be base64-encoded in the `payload` field.
- Batch is not a protocol primitive — multiple `call.requested` events with correlated IDs provide equivalent semantics. See OQ-14.
- The call protocol is transport-agnostic at the envelope level. The `EventEnvelope` framing can run over QUIC streams, WebSocket frames, or Worker `postMessage`. The `CallAdapter` is the QUIC-specific implementation. **The `EventEnvelope` shape (`{ type, id, payload }`) was derived from the `@alkdev/pubsub` `EventEnvelope` (`/workspace/@alkdev/pubsub/src/types.ts`), which already has a working WebSocket client/server implementation (`event-target-websocket-client.ts` / `event-target-websocket-server.ts`) and a generalized "event target" abstraction. The call protocol refined the envelope with typed event names (`call.requested`, `call.responded`, etc.) and structured payloads; the delta is small and well-defined, making a browser (and Node) WebSocket client straightforward to derive from the pubsub prior art. See ADR-044, [ADR-048](../../decisions/048-websocket-native-session-not-gateway.md), and [websocket.md](../http/websocket.md).
- `OperationEnv::invoke()` dispatches through the local registry. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer. See ADR-005 and OQ-13.
- `OperationEnv::invoke()` dispatches through the local registry. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer. See ADR-064 and OQ-13.
- **The call protocol carries no secret material.** Secret material (private keys, API keys, mnemonics, decrypted credentials, raw tokens) must not appear in `call.requested` payloads, `call.responded` payloads, or `OperationContext.metadata`. The wire format carries `serde_json::Value` and cannot enforce this at the type level — the constraint is architectural, enforced by the operation registry and by convention. Operations that need to share public key material use a dedicated operation that returns only the public component. See ADR-014.
- **Abort cascades to descendants.** `call.aborted` for a parent request cascades to all non-terminal descendants in the call tree. Default policy is `abort-dependents`; `continue-running` is an opt-in. See ADR-016.
@@ -554,7 +560,7 @@ Handlers clean up resources when their call is cancelled (in Rust, the future is
| Decision | ADR | Summary |
|----------|-----|---------|
| irpc as call protocol foundation | [ADR-005](../../decisions/005-irpc-as-call-protocol-foundation.md) | irpc provides framing and service dispatch |
| Hand-rolled EventEnvelope framing (irpc never integrated) | [ADR-064](../../decisions/064-irpc-never-integrated-hand-rolled-framing.md) | Hand-rolled length-prefixed JSON framing, operation registry, dispatch; supersedes ADR-005 (irpc was never imported) |
| Call protocol stream model | [ADR-012](../../decisions/012-call-protocol-stream-model.md) | Bidirectional streams, EventEnvelope, ID-based correlation |
| ALPN per connection | [ADR-006](../../decisions/006-alpn-convention-and-connection-model.md) | `alknet/call` is a distinct ALPN, one connection per ALPN |
| ProtocolHandler receives Connection | [ADR-007](../../decisions/007-bistream-type-definition.md) | CallAdapter gets Connection, can accept/open multiple streams |
@@ -615,7 +621,7 @@ See [open-questions.md](../../open-questions.md) for full details.
- [operation-registry.md](operation-registry.md) — OperationSpec, Handler, AccessControl, service discovery
- [client-and-adapters.md](client-and-adapters.md) — CallClient, from_call, OperationAdapter, peer-keyed composition env
- ADR-005: irpc as call protocol foundation
- ADR-064: Hand-rolled EventEnvelope framing (irpc never integrated; supersedes ADR-005)
- ADR-012: Call protocol stream model
- ADR-029: Peer-graph routing model (peer-keyed overlays + `PeerRef` routing)
- ADR-030: PeerEntry and Identity.id decoupling (`PeerId` source)
docs/architecture/crates/call/operation-registry.md
+25 -18
View File
@@ -5,7 +5,7 @@ last_updated: 2026-07-05
# Operation Registry
OperationSpec, Handler, OperationRegistry, AccessControl, service discovery, and irpc integration.
OperationSpec, Handler, OperationRegistry, AccessControl, service discovery, and the hand-rolled framing (no irpc — ADR-064).
## What
@@ -185,7 +185,11 @@ pub type ResponseStream = Pin<Box<dyn Stream<Item = ResponseEnvelope> + Send>>;
```
Both handlers are async — many operations (file I/O, HTTP service calls,
irpc service calls, LLM streaming) are inherently asynchronous. A handler
LLM streaming) are inherently asynchronous. A handler (whether it wraps a
local function, an HTTP-backed OpenAPI operation, an LLM stream, or a
`from_call` remote) is a `Future` (for `Query`/`Mutation`) or a `Stream`
(for `Subscription`, ADR-049). The registry's `Handler` /
`StreamingHandler` trait objects (ADR-049) abstract over this. A handler
receives:
- `input: Value` — the deserialized `payload` from the `call.requested` event
@@ -447,7 +451,7 @@ The CLI binary (or assembly layer) constructs the registry and passes it to the
### OperationEnv
The `OperationEnv` trait is the universal composition mechanism. A handler calls `context.env.invoke("fs", "readFile", input, &context)` and gets a `ResponseEnvelope` back — regardless of whether the operation runs locally, via an irpc service, or on a remote node.
The `OperationEnv` trait is the universal composition mechanism. A handler calls `context.env.invoke("fs", "readFile", input, &context)` and gets a `ResponseEnvelope` back — regardless of whether the operation runs locally or on a remote node.
**`OperationEnv` is request/response-only** (ADR-049). It returns a single `ResponseEnvelope` — no streaming variant exists. Calling `invoke()` on a `Subscription` op produces `CallError { code: "INVALID_OPERATION_TYPE", ... }` — composition cannot truncate a stream to its first value. Stream composition (filter, map, combine, window, dedupe) is a handler-level concern, not a protocol composition concern; see ADR-049 for the rationale and the `@alkdev/pubsub` `operators.ts` prior art.
@@ -736,7 +740,7 @@ Two things happen in `invoke()`:
1. **Reachability check**: before constructing the child context, `invoke()` checks whether the requested op is in the parent's scoped env. If not, `NOT_FOUND`. This is the reachability control — a handler can only compose declared ops.
2. **Authority propagation**: the child's `identity` is the parent's `handler_identity` (the parent's composition authority becomes the caller). The child's `handler_identity` is the child's own registration's `composition_authority` — so if the child itself composes further, its children inherit the child's authority. This is the principal/agent chain from ADR-015, now wired via ADR-022.
Future work may add irpc service dispatch and remote call protocol dispatch as additional backends. The handler-facing API stays the same.
Future work may add remote call protocol dispatch as an additional backend. The handler-facing API stays the same.
**`OperationEnv` must remain a trait.** This is a constraint, not a suggestion. The trait-based design enables registry layering (ADR-024): the CallAdapter composes the root env per call from the curated base + active peer-keyed connection overlays + session overlay, and overlays wrap the base via trait layering. Session-scoped registries (OQ-19) and connection-scoped remote imports (ADR-017 `from_call`) are both overlays on the same base, using the same mechanism. The peer-keyed extension (`PeerCompositeEnv`, `invoke_peer`, ADR-029) composes on top of the same trait — it overrides the new peer-routing methods, not the base dispatch. Making `OperationEnv` concrete or hardcoding the global registry into the dispatch path would close both the session-overlay and connection-overlay patterns, and would prevent the peer-keyed routing model from composing. This is the same integration-point pattern as `IdentityProvider` (ADR-004). See OQ-19, ADR-024, and ADR-029.
@@ -774,18 +778,21 @@ from wire `operationId`s before lookup, so `services/schema` accepts both
client reading the schema can produce typed error enums instead of generic
error handling.
### irpc Integration
### Operation Registry (hand-rolled, no irpc)
irpc and the operation registry serve different scopes:
The operation registry is hand-rolled in alknet-call. ADR-005 accepted
"irpc as the call protocol foundation," but no `.rs` file in the workspace
ever imported irpc — the wire format (`wire.rs`), the operation registry,
and the dispatch are all hand-rolled. ADR-064 supersedes ADR-005 and
records the actual state. The table that previously contrasted "call
protocol (external, JSON)" with "irpc services (internal, postcard)" is
moot — there is no irpc layer.
| Layer | Mechanism | Serialization | Scope |
|-------|-----------|---------------|-------|
| Call protocol (external) | `EventEnvelope` over QUIC streams | JSON | Cross-language, cross-node |
| irpc services (internal) | `#[rpc_requests]` derive macro, `Service` trait | postcard (binary) | Rust-to-Rust, in-process or in-cluster |
irpc services are an internal dispatch mechanism — they are not directly exposed on the call protocol. alknet-call itself uses irpc for its call-protocol framing (ADR-005); the vault no longer uses irpc (ADR-025 — direct method calls on `VaultServiceHandle`). The vault is accessed by the assembly layer (CLI binary) at startup, not by handlers at call time. See ADR-008 and ADR-014.
If a handler internally uses an irpc-based service, the handler bridges the two: it receives JSON input from the call protocol, calls the irpc service in-process (postcard, type-safe), and serializes the result back to JSON for the call protocol response. This layering preserves irpc's type safety for internal calls while keeping the external interface cross-language.
If a handler internally uses a postcard/binary RPC for in-process calls,
that's a handler-internal choice, not an alknet-call integration. The
operation registry's external interface is always JSON (the `EventEnvelope`
wire format); the internal handler dispatch is a `Handler` /
`StreamingHandler` trait object (ADR-049), not an irpc `Service`.
### Operation Registration at Startup
@@ -874,8 +881,8 @@ The `Capabilities` type holds non-serializable, zeroized secret material. It doe
## Constraints
- The registry is **layered by trust boundary** (ADR-024). The curated layer (`Local` provenance) is immutable after construction — adding a `Local` op requires restarting the process, which re-enters the startup trust boundary. Session (`Session`) and imported (`FromCall` etc.) ops are dynamic at their respective scopes (per-session, per-connection). The pre-ADR-024 blanket immutability claim was inherited by analogy from ADR-010's `HandlerRegistry` (ALPN-level) and did not apply to the operation registry — the TLS-config argument that justifies `HandlerRegistry` immutability does not touch the operation registry, which lives behind the single ALPN `alknet/call`.
- Operation specs use JSON Schema. The call protocol's external interface is always JSON. irpc's postcard serialization is internal only.
- `OperationEnv::invoke()` dispatches through the local registry. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer — not a prefix added to operation paths. irpc service dispatch is contracted but not built.
- Operation specs use JSON Schema. The call protocol's external interface is always JSON. Internal handler dispatch is via `Handler` / `StreamingHandler` trait objects (ADR-049), not a binary RPC framework.
- `OperationEnv::invoke()` dispatches through the local registry. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer — not a prefix added to operation paths.
- The call protocol does not depend on any database. Operation specs are in-memory, populated at startup.
- `OperationContext.internal` is set by `OperationEnv`, not by callers. A handler cannot mark its own call as internal. The `internal` flag switches authority context (composition authority for ACL), it does not skip ACL — see ADR-015, ADR-022.
- **Operations have External/Internal visibility.** `Internal` operations return `NOT_FOUND` when called from the wire and are excluded from `services/list`. The assembly layer declares visibility at registration. See ADR-015.
@@ -890,7 +897,7 @@ The `Capabilities` type holds non-serializable, zeroized secret material. It doe
| Decision | ADR | Summary |
|----------|-----|---------|
| irpc as call protocol foundation | [ADR-005](../../decisions/005-irpc-as-call-protocol-foundation.md) | irpc provides framing and service dispatch |
| Hand-rolled EventEnvelope framing (irpc never integrated) | [ADR-064](../../decisions/064-irpc-never-integrated-hand-rolled-framing.md) | Hand-rolled framing, registry, dispatch; supersedes ADR-005 |
| Call protocol stream model | [ADR-012](../../decisions/012-call-protocol-stream-model.md) | Bidirectional streams, EventEnvelope, ID-based correlation |
| Static handler registration | [ADR-010](../../decisions/010-alpn-router-and-endpoint.md) | `HandlerRegistry` (ALPN-level) immutable after construction; `OperationRegistry` layered by ADR-024 (curated immutable, session/imported dynamic) |
| Vault integration via assembly layer | [ADR-008](../../decisions/008-secret-service-integration.md) | Vault is a capability source, accessed at assembly time |
@@ -949,7 +956,7 @@ See [open-questions.md](../../open-questions.md) for full details.
## References
- [call-protocol.md](call-protocol.md) — CallAdapter, EventEnvelope, stream model, PendingRequestMap
- ADR-005: irpc as call protocol foundation
- ADR-064: Hand-rolled EventEnvelope framing (irpc never integrated; supersedes ADR-005)
- ADR-008: Vault integration point
- ADR-010: ALPN router and endpoint (static registration — applies to the `HandlerRegistry`, not the `OperationRegistry`; see ADR-024 for the distinction)
- ADR-012: Call protocol stream model