alkdev/alkcall
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: port architecture specs + 45 ADRs from alknet, renumbered

Browse Source 
Port the call + channels architecture documentation from the alknet
mono-repo into docs/architecture/, renumbered as alkcall ADR-001..045.

Renumbering map (alknet -> alkcall):
  Core:        001,002,004,006,007,011,065,070,092,014,050,091 -> 001-012
  Call:        005,064,012,023,015,022,024,016,049,017,028,029,030,032,066,069,067,068 -> 013-030
  Shared:      003,009,013 -> 031-033
  Channels:    071,093,072,073,074,075,076,094,079,080,081,089 -> 034-045

3 superseded/reversed ADRs kept for historical trail:
  - ADR-013 (irpc foundation, superseded by ADR-014)
  - ADR-023 (peer-scoped filtering, superseded by ADR-024)
  - ADR-077 (TTY inside channels, reversed by ADR-035 — not ported, TTY-only)

Ported docs (11 spec files + README + open-questions):
  - call-README.md, call-protocol.md, operation-registry.md, client-and-adapters.md
  - channels-README.md, channels-overview.md, channels-wire.md, channels-connection.md, channels-adapter.md, channel-operations.md, channel-client.md
  - README.md (index with doc table, ADR table grouped by category, key principles)
  - open-questions.md (lean — 30 OQs, renumbered OQ-01..030; includes new OQ-22 for the pub/sub gap)

Cross-reference rewriting:
  - All ADR-NNN references rewritten single-pass (no chaining bug)
  - Markdown link paths fixed
  - Title lines aligned with filenames
  - Non-ported ADR refs (052, 082, 086, etc.) left as-is with README note

The open-questions.md includes OQ-22 (new): the call protocol pub/sub
gap — subscribe exists but pub does not, needed for channels
channel/resources/subscribe fan-out. This is the next ADR to write
(alkcall ADR-046).
This commit is contained in:
glm-5.2
2026-08-12 07:06:57 +00:00
parent 1ceb9b785d
commit cc470a363a
58 changed files with 16729 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

161
docs/architecture/README.md Normal file
View File

@@ -0,0 +1,161 @@
---
status: draft
last_updated: 2026-08-12
---
# alkcall
The call + channels RPC crate. Structured JSON RPC (operations, streaming
subscriptions, service discovery) and N-channel multiplexing over one
transport stream (channel 0 pre-negotiated as `alknet/call`).
This crate unifies `alknet-call` and `alknet-channels` from the alknet
mono-repo, plus the vendored core types formerly in `alknet-core`. The
source architecture docs were ported from
`/workspace/@alkdev/alknet/docs/architecture/` and renumbered as alkcall
ADRs (ADR-001..045). The ALPN strings (`alknet/call`, `alknet/channels`)
are wire-stable and unchanged — see ADR-004.
## Documents
| Document | Status | Description |
|----------|--------|-------------|
| [call-README.md](call-README.md) | draft | Call protocol index — adapter, stream model, registry, client (ported from alknet call/README.md) |
| [call-protocol.md](call-protocol.md) | draft | CallAdapter, hand-rolled EventEnvelope framing (ADR-014), stream model, PendingRequestMap, bidirectional calls |
| [operation-registry.md](operation-registry.md) | draft | OperationSpec, Handler, OperationRegistry, AccessControl, service discovery |
| [client-and-adapters.md](client-and-adapters.md) | draft | CallClient (transport-agnostic spawn_dispatch), from_call, OperationAdapter trait, no-env-vars invariant |
| [channels-README.md](channels-README.md) | draft | Channels protocol index — wire format, adapter, lifecycle, client (ported from alknet channels/README.md) |
| [channels-overview.md](channels-overview.md) | draft | The multiplexing collapse, crate dependencies, transport agnosticism, WASM |
| [channels-wire.md](channels-wire.md) | draft | The 8-byte chunk format, sentinels, wire-level invariants (REQ-CH-01..05) |
| [channels-connection.md](channels-connection.md) | draft | ChannelBidiStreamSource, accept_bi yields one BiStream per channel |
| [channels-adapter.md](channels-adapter.md) | draft | ChannelsAdapter, ChannelManager, demux/mux contracts |
| [channel-operations.md](channel-operations.md) | draft | channel/open, channel/close, channel/control, channel/resources/subscribe |
| [channel-client.md](channel-client.md) | draft | ChannelClient — transport-agnostic from_connection primary |
## Applicable ADRs
### Core (vendored types) — ADR-001..012
| ADR | Title | Relevance |
|-----|-------|-----------|
| [001](decisions/001-alpn-protocol-dispatch.md) | ALPN-Based Protocol Dispatch | HandlerRegistry, ALPN routing |
| [002](decisions/002-protocol-handler-trait.md) | ProtocolHandler Trait | The trait every handler implements |
| [003](decisions/003-auth-as-shared-core.md) | Auth as Shared Core | IdentityProvider, Identity, AuthToken |
| [004](decisions/004-alpn-convention-and-connection-model.md) | ALPN String Convention | `alknet/` prefix, one ALPN per connection |
| [005](decisions/005-bistream-type-definition.md) | BiStream Type Definition | BiStream, handlers receive Connection |
| [006](decisions/006-authcontext-structure.md) | AuthContext Structure | AuthContext fields, hybrid resolution |
| [007](decisions/007-connection-from-stream-generic-single-stream.md) | Connection::from_stream | Generic single-stream connections |
| [008](decisions/008-bidistreamsource-trait.md) | BidiStreamSource Trait | Connection extension point |
| [009](decisions/009-bistream-as-the-handler-leaf.md) | BiStream as the Handler Leaf | accept_bi returns BiStream (concrete) |
| [010](decisions/010-secret-material-flow-and-capability-injection.md) | Secret Material Flow | No secrets on wire; Capabilities |
| [011](decisions/011-dynamic-resource-ownership-for-runtime-spawned-resources.md) | Dynamic Resource Ownership | OwnershipProvider, resource_id_path |
| [012](decisions/012-connectioncredentials-decouple-dial-from-call.md) | ConnectionCredentials | Transport-level credentials, auth_token is per-request |
### Call protocol — ADR-013..030
| ADR | Title | Relevance |
|-----|-------|-----------|
| [013](decisions/013-irpc-as-call-protocol-foundation.md) | ~~irpc as Call Protocol Foundation~~ | **Superseded** by ADR-014 |
| [014](decisions/014-irpc-never-integrated-hand-rolled-framing.md) | Hand-Rolled EventEnvelope Framing | The call wire format ADR; supersedes ADR-013 |
| [015](decisions/015-call-protocol-stream-model.md) | Call Protocol Stream Model | Bidi streams, EventEnvelope, ID correlation |
| [016](decisions/016-operation-error-schemas.md) | Operation Error Schemas | call.error with typed details |
| [017](decisions/017-privilege-model-and-authority-context.md) | Privilege Model | internal = authority switch; Visibility |
| [018](decisions/018-handler-registration-provenance-and-composition-authority.md) | Handler Registration | Registration bundle, provenance, composition authority |
| [019](decisions/019-operation-registry-layering.md) | Operation Registry Layering | Curated + session + connection overlays; OperationEnv trait |
| [020](decisions/020-abort-cascade-for-nested-calls.md) | Abort Cascade | call.aborted cascades; abort-dependents default |
| [021](decisions/021-streaming-handler-for-subscriptions.md) | Streaming Handler | StreamingHandler, invoke_streaming() |
| [022](decisions/022-call-protocol-client-and-adapter-contract.md) | Client and Adapter Contract | CallClient, from_call, OperationAdapter |
| [023](decisions/023-callclient-peer-scoped-registry-filtering.md) | ~~Peer-Scoped Registry Filtering~~ | **Superseded** by ADR-024 |
| [024](decisions/024-peer-graph-routing-model.md) | Peer-Graph Routing Model | PeerCompositeEnv, PeerRef, AccessControl peer auth |
| [025](decisions/025-peerentry-and-identity-id-decoupling.md) | PeerEntry and Identity.id Decoupling | PeerId = Identity.id (stable) |
| [026](decisions/026-forwarded-for-identity.md) | Forwarded-For Identity | Metadata only, never used by ACL |
| [027](decisions/027-from-jsonschema-as-http-adapter.md) | from_jsonschema as HTTP Adapter | FromJsonSchema provenance stays; impl in alknet-http |
| [028](decisions/028-from-call-manual-free-function.md) | from_call Is a Manual Free Function | Assembly layer calls it after dial |
| [029](decisions/029-aggregated-peer-env-wiring.md) | Aggregated Peer-Environment Wiring | Dispatcher hub wiring |
| [030](decisions/030-peer-composite-env-peer-operations.md) | PeerCompositeEnv::peer_operations | OperationEnv::peer_operations override |
### Shared — ADR-031..033
| ADR | Title | Relevance |
|-----|-------|-----------|
| [031](decisions/031-crate-decomposition.md) | Crate Decomposition | alkcall unifies core+call+channels |
| [032](decisions/032-one-way-door-decision-framework.md) | One-Way Door Decision Framework | Reversal cost classification |
| [033](decisions/033-rust-canonical-implementation.md) | Rust as Canonical Implementation Language | Rust canonical, TS reference |
### Channels — ADR-034..045
| ADR | Title | Relevance |
|-----|-------|-----------|
| [034](decisions/034-channels-wire-format.md) | Channels Wire Format | 8-byte chunk header; one-way door |
| [035](decisions/035-channels-pure-channel-multiplexing.md) | Pure Channel Multiplexing | No stream_type; BiStream-only; handler owns sub-mux |
| [036](decisions/036-channel-0-pre-negotiated-call.md) | Channel 0 Pre-Negotiated | Channel 0 = alknet/call |
| [037](decisions/037-channel-lifecycle-operations.md) | Channel Lifecycle Operations | channel/open, close, control, resources/subscribe |
| [038](decisions/038-channelconnection-bidistreamsource.md) | ChannelConnection | Per-channel BidiStreamSource; yield-once accept_bi |
| [039](decisions/039-channelsadapter-and-channelmanager.md) | ChannelsAdapter and ChannelManager | Demux/mux; ALPN-blind, auth-blind |
| [040](decisions/040-backpressure-channel-limits-id-reuse.md) | Backpressure, Limits, ID Reuse | Bounded-buffer; 256-channel memory bound |
| [041](decisions/041-per-identity-channel-cap.md) | Per-Identity Channel Cap | ChannelLifecyclePolicy; 256 per PeerId |
| [042](decisions/042-hub-relay-translate-not-forward.md) | Hub Relay | Translate channel 0, byte-forward data channels |
| [043](decisions/043-channelclient.md) | ChannelClient | Transport-agnostic from_connection |
| [044](decisions/044-channels-subcrate-decomposition.md) | Channels Sub-Crate Decomposition | channels-core / channels-call (modules in alkcall) |
| [045](decisions/045-alknetclient-native-dial-seam.md) | AlknetClient Dial Seam | spawn_dispatch / from_connection take-over; dial in consumer |
## Relevant Open Questions
See [open-questions.md](open-questions.md) for the full tracker. Key
questions affecting this crate:
- **OQ-01**: Call protocol pub/sub primitive (open) — the call protocol
has `subscribe` but no `pub`; needed for channels `channel/resources/
subscribe` fan-out. This is the next ADR to write.
- **OQ-02**: Full channel-level flow-control windowing (deferred(scope))
— bounded-buffer is decided (ADR-040); full windowing blocked on a real
HOL-blocking deployment observation.
- **OQ-03**: Channels add/strip API shape (open) — whether the 8-byte
header add/strip is built into the read/write path or a standalone
utility.
## Key Design Principles
1. **One connection, full access**: An `alknet/call` connection gives
access to the entire operation registry.
2. **Protocol is symmetric**: Both sides can initiate calls. Producer/
consumer, not server/client.
3. **Hand-rolled framing (no irpc)**: EventEnvelope is hand-rolled
length-prefixed JSON. See ADR-014.
4. **Operation registry is layered**: Curated (static) + session +
connection overlays. OperationEnv is a trait. See ADR-019.
5. **No secret material on the wire**: Capabilities injected at assembly
layer. See ADR-010.
6. **Abort cascades to descendants**: Default abort-dependents.
See ADR-020.
7. **Peer authorization via AccessControl**: No remote_safe flag.
See ADR-024.
8. **Streams are streams**: Every channel is a BiStream. The handler
owns its sub-stream multiplexing. See ADR-035.
9. **Channel 0 is alknet/call**: Channel lifecycle is call operations on
channel 0. See ADR-036, ADR-037.
10. **Wire formats are stable**: EventEnvelope shape and the 8-byte chunk
header are one-way doors. See ADR-014, ADR-034.
## References
- `@alkdev/alknet: docs/architecture/` — the source architecture docs
these were ported from (renumbered from alknet ADR-001..094 to alkcall
ADR-001..045)
- `@alkdev/alktype` — the binary struct engine, used for channels chunk
header layout and JSON payload schema validation
- `@alkdev/pubsub` — the TypeScript EventEnvelope prior art the call
wire format was derived from
> **Note**: The source ADRs and spec docs were ported from the parent
> `@alkdev/alknet` workspace where this crate originated. They are
> preserved here as the authoritative spec for alkcall; the alknet
> mono-repo will consume alkcall's versions when it is reworked.
>
> **Cross-references to non-ported ADRs**: Some spec docs and ADRs
> reference alknet ADRs by their original numbers (e.g., ADR-052 for
> TTY's wire format, ADR-082 for alknet-tls, ADR-086 for endpoint types).
> These are ADRs for sibling crates that are not part of alkcall. They
> retain their alknet numbering (052, 082, 086, etc.) — any ADR number
> outside the alkcall range 001..045 is an alknet source ADR, found at
> `/workspace/@alkdev/alknet/docs/architecture/decisions/`.

89
docs/architecture/call-README.md Normal file
View File

@@ -0,0 +1,89 @@
---
status: draft
last_updated: 2026-07-17
review: call/review-call passed 2026-06-23 — registry, protocol, ADR (005/012/014/015/016/017/022/023/024), security, and pattern-consistency checks all conformant; 159 unit/integration tests green; `cargo build`, `cargo clippy -- -D warnings`, `cargo fmt --check`, `cargo test` clean. Call-completion gap (ADR-022 client/adapter surface) addressed 2026-06-26; ADR-024 migration landed. Transport generalization sweep (ADR-014 supersedes ADR-013; ADR-007 `from_stream`) synced 2026-07-09. Crate-extraction sweep (phases 0–5) landed 2026-07-17: `ConnectionCredentials`/`RemoteIdentity` in `alknet-core` (ADR-012); TLS helpers in `alknet-tls` (ADR-045 §5); dial in `alknet-client` (ADR-045); `alknet-call` is a pure protocol crate with no TLS/transport deps.
---
# alknet-call
Structured RPC: operations, request/response, streaming subscriptions, and service discovery. Implements `ProtocolHandler` on ALPN `alknet/call`. Runs over QUIC (quinn/iroh) and, via `Connection::from_stream` (ADR-007), over any `AsyncRead + AsyncWrite` transport. A pure protocol crate — no TLS or transport deps (the dial is in `alknet-client`, the TLS config is in `alknet-tls`).
## Documents
| Document | Status | Description |
|----------|--------|-------------|
| [call-protocol.md](call-protocol.md) | draft | CallAdapter, hand-rolled EventEnvelope framing (no irpc — ADR-014), stream model, PendingRequestMap, bidirectional calls |
| [operation-registry.md](operation-registry.md) | draft | OperationSpec, Handler, OperationRegistry, AccessControl, service discovery, hand-rolled framing (no irpc — ADR-014) |
| [client-and-adapters.md](client-and-adapters.md) | draft | CallClient (transport-agnostic `spawn_dispatch` primary; dial lives in `AlknetClient` per ADR-045), from_call, OperationAdapter trait, adapter location map, no-env-vars invariant, exchange-of-operations pattern (`from_jsonschema` in alknet-http per ADR-027) |
## Applicable ADRs
| ADR | Title | Relevance |
|-----|-------|-----------|
| [001](decisions/001-alpn-protocol-dispatch.md) | ALPN-Based Protocol Dispatch | CallAdapter registers on ALPN `alknet/call` |
| [002](decisions/002-protocol-handler-trait.md) | ProtocolHandler Trait | CallAdapter implements ProtocolHandler |
| [003](decisions/003-crate-decomposition.md) | Crate Decomposition | alknet-call depends on alknet-core (no irpc — ADR-014) |
| [013](decisions/013-rust-canonical-implementation.md) | Rust as Canonical Implementation Language | Adapter traits defined in Rust; TS is reference/browser adaptation |
| [004](decisions/004-auth-as-shared-core.md) | Auth as Shared Core | AuthContext passed to call handlers |
| [005](decisions/005-irpc-as-call-protocol-foundation.md) | ~~irpc as Call Protocol Foundation~~ | ~~Accepted~~ → **Superseded** by [ADR-014](decisions/064-irpc-never-integrated-hand-rolled-framing.md) (irpc was never integrated; framing is hand-rolled) |
| [064](decisions/064-irpc-never-integrated-hand-rolled-framing.md) | Hand-Rolled EventEnvelope Framing | Wire format, registry, dispatch are hand-rolled in alknet-call; supersedes ADR-013 |
| [065](decisions/065-connection-from-stream-generic-single-stream.md) | `Connection::from_stream` | Generic single-stream connections; unblocks TCP+TLS/SSH/WT/wasm dispatch |
| [006](decisions/006-alpn-convention-and-connection-model.md) | ALPN String Convention | `alknet/call` ALPN, one ALPN per connection |
| [007](decisions/007-bistream-type-definition.md) | BiStream Type Definition | CallAdapter receives Connection, not BiStream |
| [008](decisions/008-secret-service-integration.md) | Vault Integration Point | Vault accessed at assembly layer, not on the wire |
| [010](decisions/010-alpn-router-and-endpoint.md) | ALPN Router and Endpoint | Static handler registration |
| [012](decisions/012-call-protocol-stream-model.md) | Call Protocol Stream Model | Bidirectional streams, EventEnvelope, ID-based correlation |
| [014](decisions/014-secret-material-flow-and-capability-injection.md) | Secret Material Flow and Capability Injection | Call protocol carries no secret material; capabilities injected at assembly layer |
| [015](decisions/015-privilege-model-and-authority-context.md) | Privilege Model and Authority Context | `internal` = authority switch not ACL skip; External/Internal visibility; handler identity + scoped env |
| [016](decisions/016-abort-cascade-for-nested-calls.md) | Abort Cascade for Nested Calls | `call.aborted` cascades to descendants; default `abort-dependents`, `continue-running` opt-in |
| [017](decisions/017-call-protocol-client-and-adapter-contract.md) | Call Protocol Client and Adapter Contract | `CallClient` opens connections; `from_call` imports remote ops; connection direction independent of call direction |
| [066](decisions/066-from-jsonschema-as-http-adapter.md) | `from_jsonschema` as HTTP-Backed Single-Endpoint Adapter in alknet-http | Moved `from_jsonschema` from `alknet-call` (broken schema-only placeholder) to `alknet-http` as a real reqwest-backed single-endpoint adapter; `FromJsonSchema` provenance stays in `alknet-call` as a leaf |
| [022](decisions/022-handler-registration-provenance-and-composition-authority.md) | Handler Registration, Provenance, and Composition Authority | Registration bundle carries provenance, composition authority, scoped env, capabilities |
| [023](decisions/023-operation-error-schemas.md) | Operation Error Schemas | Operations declare domain errors; `call.error` carries typed `details`; adapter fidelity |
| [024](decisions/024-operation-registry-layering.md) | Operation Registry Layering | Curated (static) + session/connection overlays (dynamic); `OperationEnv` as trait-object integration point; `OperationContext.env` split into `scoped_env` (data) and `env` (dispatch trait) |
| [028](decisions/028-callclient-peer-scoped-registry-filtering.md) | ~~Peer-Scoped Registry Filtering~~ | ~~Accepted~~ → **Superseded** by ADR-024 (flat-namespace single-peer model couldn't express head→N-workers; parallel auth system duplicated `AccessControl`) |
| [029](decisions/029-peer-graph-routing-model.md) | Peer-Graph Routing Model | Peer-keyed overlays + `PeerRef` routing; `AccessControl`-based peer authorization; retires `remote_safe`/`trusted_peer` |
| [030](decisions/030-peerentry-and-identity-id-decoupling.md) | PeerEntry and Identity.id Decoupling | `PeerId` source = `Identity.id` = `PeerEntry.peer_id` (stable); supersedes ADR-024's UUID source |
| [032](decisions/032-forwarded-for-identity.md) | Forwarded-For Identity | `forwarded_for` on `OperationContext` and `call.requested`; metadata only, never used by `AccessControl::check` |
| [033](decisions/033-storage-boundary-and-repo-adapter-pattern.md) | Storage Boundary and Repo/Adapter Pattern | Core defines repo traits + in-memory defaults; persistence adapters are separate crates |
| [089](decisions/089-alknetclient-native-dial-seam.md) | AlknetClient — Native Client Dial Seam | The dial is in `alknet-client`; `CallClient` is `spawn_dispatch` only; `alknet-call` is a pure protocol crate with no TLS/transport deps |
| [091](decisions/091-connectioncredentials-decouple-dial-from-call.md) | `ConnectionCredentials` — Decouple Dial from Call Protocol | `ConnectionCredentials`/`RemoteIdentity` in `alknet-core` (not `alknet-call`); `auth_token` is a per-request payload field |
## Relevant Open Questions
| OQ | Title | Status | Relevance |
|----|-------|--------|-----------|
| OQ-07 | Call protocol scope within a connection | resolved (ADR-015) | Stream model, multiplexing, scope |
| OQ-13 | Operation path format and routing scope | resolved | `/{service}/{op}` is the correct design; remote dispatch is a separate layer |
| OQ-14 | Batch operation semantics | resolved | Correlated `call.requested` events is the correct protocol design |
| OQ-16 | Safe vault operations for call protocol exposure | resolved (ADR-010) | None exposed for now |
| OQ-19 | Session-scoped operation registries | resolved | Agent-written operations overlaid on curated registry via `OperationEnv` trait layering. Protocol doesn't need changes; `OperationEnv` must remain a trait. Generalized by ADR-019 to cover connection-scoped overlays. |
| OQ-25 | ~~Remote-safe marking shape~~ | **dissolved** (ADR-024) | `remote_safe`/`trusted_peer` retired; peer authorization is `AccessControl::check(peer_identity)` |
| OQ-26 | OperationAdapter error type (AdapterError variants) | **resolved** | `DiscoveryFailed`, `SchemaParse`, `Transport`, `Unauthorized`, `SamePeerCollision`; `#[non_exhaustive]` |
| OQ-27 | from_call re-import trigger | **resolved** | `from_call` is a manual free function; the assembly layer calls it after the dial (in `AlknetClient`). `refresh()` is a genuine feature addition. See ADR-028. |
| OQ-28 | from_call namespace collision | **resolved** | Same-peer collision = error; cross-peer dissolved by ADR-024 (separate sub-overlays) |
| OQ-29 | CallClient TLS client-auth | **resolved** | Wire quinn client-auth; key-type-aware server cert verification; fingerprint normalization |
| OQ-30 | `PeerRef::Any` routing policy | **resolved** | Insertion-order first-match; richer routing is a feature extension |
| OQ-31 | `services/list-peers` re-export semantics | **resolved** | Opt-in `services/list-peers`; `services/list` is "own ops only" |
| OQ-32 | Multi-hop federation | open (feature extension) | One-hop model is the commitment; multi-hop is a feature extension, not a deferral |
| OQ-33 | PeerId — crypto identity vs stable logical id | **resolved** (ADR-025) | `PeerId = Identity.id = PeerEntry.peer_id` (stable across key rotation) |
| OQ-34 | Persistent peer registry | **resolved** (ADR-025+033) | Core trait + in-memory default; persistence adapters are separate crates |
| OQ-35 | ~~API key asymmetry~~ | **dissolved** | `PeerEntry` supports multiple credential paths; `ApiKeyEntry` is for tokens that ARE the identity |
| OQ-37 | X.509 outgoing-only case | **resolved** (ADR-034) | Three remote roles (public X.509 endpoint, transport relay, hub); `PeerEntry` asymmetry correct; verifier by `PeerEntry` presence |
## Key Design Principles
1. **One connection, full access**: An `alknet/call` connection gives access to the entire operation registry — calls, subscriptions, batch, schema.
2. **Protocol is symmetric**: Both sides can initiate calls. The server calling a client uses the same EventEnvelope format and correlation.
3. **Stream-agnostic correlation**: PendingRequestMap correlates by request ID, not by stream. The protocol works with any stream arrangement.
4. **Operation registry is layered**: The curated layer (`Local` provenance) is static — registered at startup by the CLI binary, immutable for the process lifetime. Session (`Session`) and imported (`FromCall` etc.) ops are dynamic overlays at their respective scopes (per-session, per-connection). The registry supports JSON Schema discovery. See ADR-019.
5. **Hand-rolled dispatch (no irpc)**: Operations dispatch through the hand-rolled `OperationRegistry` (ADR-014). The call protocol is the external interface; internal handler dispatch uses `Handler`/`StreamingHandler` trait objects (ADR-021), not an irpc service.
6. **Local dispatch only**: The operation registry dispatches to local handlers. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer, not a modification to alknet-call's path format.
7. **No secret material on the wire**: The call protocol carries no private keys, API keys, mnemonics, or decrypted credentials. Handlers receive outbound credentials through `OperationContext.capabilities`, injected at the assembly layer. See ADR-010.
8. **Abort cascades to descendants**: `call.aborted` for a parent request cascades to all non-terminal descendants. Default `abort-dependents`; `continue-running` opt-in. See ADR-020.
9. **Internal calls switch authority context, not skip ACL**: The `internal` flag marks composition-originated calls. ACL runs against the handler's composition authority, not the caller's and not as a blanket skip. Operations have External/Internal visibility. Scoped composition env bounds reachability. See ADR-017, ADR-018.
10. **Provenance determines composition capability**: Only `Local` and `Session` ops can compose. Leaves (`FromOpenAPI`, `FromMCP`, `FromCall`, `FromJsonSchema`) are forwarding stubs — they don't get composition authority or a scoped env. The assembly layer is the sole grantor of composition authority. See ADR-018. (`FromJsonSchema` is now a real HTTP-forwarding leaf per ADR-027, not a schema-only placeholder.)
11. **Connection direction is independent of call direction**: Who opens the connection is a connection-layer concern, not a protocol-layer concern. Both sides can call each other once connected. The `CallAdapter` accepts connections; the `CallClient` takes them over (`spawn_dispatch` primary; dial in `AlknetClient` per ADR-045); both produce the same `CallConnection` and dispatch through the same loop. See ADR-022, [client-and-adapters.md](client-and-adapters.md).
12. **Peer authorization via `AccessControl`**: A remote peer's call is authorized by `AccessControl::check(peer_identity)` against the op's `AccessControl` — the same mechanism that gates every other call. No `remote_safe` flag, no `trusted_peer` bypass. An op with `AccessControl::default()` is callable by any peer; an op with `required_scopes` is callable only by peers whose `Identity.scopes` satisfy them; an op with `Visibility::Internal` is never callable from the wire. See ADR-024.
13. **Adapter trait lives with the types; implementations live with their transport**: `OperationAdapter` is in `alknet-call`; `from_call` is in `alknet-call` (QUIC); `from_jsonschema`/`from_openapi`/`from_mcp`/`to_openapi`/`to_mcp` are in `alknet-http` (reqwest / axum). `alknet-call` stays lean — no HTTP client, no HTTP server. (`from_jsonschema` was originally in `alknet-call` as a schema-only placeholder; ADR-027 moved it to `alknet-http` as a real HTTP-backed adapter.) See [client-and-adapters.md](client-and-adapters.md).
14. **No handler reads outbound credentials from any source other than `OperationContext.capabilities`** (no-env-vars invariant): the credential injection path is vault → assembly layer → `Capabilities` → `HandlerRegistration.capabilities` → `OperationContext.capabilities` → handler. Downstream consumers' `std::env::var` reads are unreachable because the assembly layer never calls `Default::default()`. See ADR-010, [client-and-adapters.md](client-and-adapters.md).

637
docs/architecture/call-protocol.md Normal file
View File

@@ -0,0 +1,637 @@
---
status: draft
last_updated: 2026-07-09
---
# Call Protocol
The wire protocol, stream model, framing, and adapter that alknet-call implements on ALPN `alknet/call`.
## What
The call protocol is a bidirectional, transport-agnostic RPC protocol that runs over any ordered, reliable bidirectional stream within a single `alknet/call` connection. It supports request/response calls, streaming subscriptions, batch operations, and service discovery — all using the same EventEnvelope wire format.
The `CallAdapter` implements `ProtocolHandler` for ALPN `alknet/call`. It receives a `Connection` from the endpoint (QUIC-native, or TCP+TLS/WebTransport/SSH via `Connection::from_stream` — ADR-007), accepts bidirectional streams, and dispatches incoming `EventEnvelope` messages to the operation registry.
## Why
The call protocol is the primary programmatic interface to an alknet node. While SSH provides interactive shell access and HTTP provides REST APIs, the call protocol provides structured, discoverable RPC — the same interface that NAPI clients, MCP tools, and other automation consumers use.
The protocol must be:
- **Cross-language**: JSON wire format consumable from TypeScript, Python, any language
- **Bidirectional**: Both sides can initiate calls (server-to-client is as natural as client-to-server)
- **Stream-agnostic**: the protocol runs over any `AsyncRead + AsyncWrite` pair (QUIC streams, TCP+TLS, WebTransport, SSH channels, WebSocket — ADR-007); the transport provides the stream, the protocol provides the framing
- **Discoverable**: Clients can query what operations exist and their schemas
See ADR-014 for the decision that the call protocol uses hand-rolled
`EventEnvelope` framing (irpc was never integrated — ADR-013, which
accepted "irpc as the call protocol foundation," is superseded) and ADR-015
for the stream model decision.
## Architecture
### CallAdapter
The `CallAdapter` implements `ProtocolHandler`:
```rust
pub struct CallAdapter {
/// Layer 0 — the curated operation registry. Immutable after startup.
registry: Arc<OperationRegistry>,
identity_provider: Arc<dyn IdentityProvider>,
/// Layer 1 — optional session-overlay source (agent crate supplies this;
/// None for non-agent deployments). See ADR-019, OQ-19.
session_source: Option<Arc<dyn SessionOverlaySource + Send + Sync>>,
/// Default timeout for wire calls (30s). Composed calls inherit the
/// parent's remaining deadline via `OperationContext.deadline`.
default_timeout: Duration,
}
impl CallAdapter {
/// Non-agent deployment: no session overlay, default timeout.
pub fn new(
registry: Arc<OperationRegistry>,
identity_provider: Arc<dyn IdentityProvider>,
) -> Self {
Self { registry, identity_provider, session_source: None,
default_timeout: Duration::from_secs(30) }
}
/// Agent deployment: supply a session-overlay source. The agent crate
/// implements `SessionOverlaySource`; alknet-call defines the trait.
pub fn with_session_source(mut self, source: Arc<dyn SessionOverlaySource + Send + Sync>) -> Self {
self.session_source = Some(source);
self
}
/// Override the default timeout.
pub fn with_timeout(mut self, timeout: Duration) -> Self {
self.default_timeout = timeout;
self
}
}
/// Session overlay integration point (ADR-019). Defined in alknet-call
/// because `CallAdapter` must name the type — alknet-call cannot depend on
/// alknet-agent (agent depends on call, not reverse). The agent crate
/// implements this trait; alknet-call defines it. This is the same pattern
/// as `IdentityProvider` (ADR-003: core defines the trait, handlers impl it).
///
/// The session overlay is an `OperationEnv` impl that wraps the curated base
/// (Layer 0). The `CallAdapter` composes it into the root
/// `OperationContext.env` per incoming call when a session is active. The
/// lookup mechanism (session ID in metadata, payload field, connection-bound
/// session state) belongs to the agent crate — this trait is the integration
/// point, not the lookup policy.
pub trait SessionOverlaySource: Send + Sync {
/// Returns the session overlay env for the given call, if a session is
/// active. `None` means no session is active for this call — the root
/// env is `curated base + connection overlay` (no session layer).
/// The agent crate determines how to map a call to its session.
fn overlay_for(&self, context: &OperationContext) -> Option<Arc<dyn OperationEnv + Send + Sync>>;
}
```
The `CallAdapter` holds the static curated registry and an optional
session-overlay source. Per-connection imported-ops overlays (Layer 2,
ADR-019) are held with the connection and composed into the root
`OperationContext.env` per incoming call. The composition env is peer-keyed
(`PeerCompositeEnv`, ADR-024 §1) to handle head→N-workers routing — a head
node with multiple worker connections holds a peer-keyed
`HashMap<PeerId, connection_overlay>`, not one overlay. See ADR-019 for the
layering model, ADR-024 for the peer-keyed extension, and `compose_root_env`
below.
### CallConnection
A `CallConnection` represents an established `alknet/call` connection,
regardless of which side opened it (ADR-022). It holds the connection's
imported-ops overlay (Layer 2, ADR-019) — the set of `from_call`-imported
operations discovered when the connection was established.
```rust
/// An established alknet/call connection (either direction — accepted or
/// opened). Holds the connection's Layer 2 overlay (imported ops).
pub struct CallConnection {
/// The underlying transport Connection (from endpoint.accept,
/// CallClient::spawn_dispatch, or AlknetClient::dial_*). May be QUIC,
/// TCP+TLS, WebTransport, SSH, or any Connection::from_stream source
/// (ADR-007).
connection: Connection,
/// Layer 2 — this connection's imported-ops overlay. Populated by
/// `from_call` discovery when the connection is established. Each
/// imported op is a `HandlerRegistration` with `provenance: FromCall`.
/// This overlay is an `OperationEnv` impl that the `CallAdapter`
/// composes into the root `OperationContext.env` per incoming call.
imported_operations: Arc<RwLock<HashMap<String, HandlerRegistration>>>,
}
impl CallConnection {
/// Register an imported operation into this connection's overlay
/// (Layer 2, ADR-019). Called by `from_call` after discovery.
pub fn register_imported(&self, registration: HandlerRegistration) {
let name = registration.spec.name.clone();
self.imported_operations.write().insert(name, registration);
}
/// Register multiple imported operations (bulk variant for `from_call`).
pub fn register_imported_all(&self, registrations: Vec<HandlerRegistration>) {
let mut overlay = self.imported_operations.write();
for reg in registrations {
overlay.insert(reg.spec.name.clone(), reg);
}
}
/// Build an `OperationEnv` impl for this connection's overlay. Used by
/// the `CallAdapter` when composing the root `OperationContext.env`.
/// Returns an `OperationEnv` that dispatches to this connection's
/// imported ops (and reports `contains` only for ops in the overlay).
pub fn overlay_env(&self) -> Arc<dyn OperationEnv + Send + Sync>;
/// Call an operation on the remote peer (sends `call.requested`).
pub async fn call(&self, operation_id: &str, input: Value) -> ResponseEnvelope;
/// Subscribe to a streaming operation on the remote peer.
pub async fn subscribe(&self, operation_id: &str, input: Value) -> impl Stream<Item = ResponseEnvelope>;
/// Abort an in-flight request (sends `call.aborted`, cascades per ADR-020).
pub async fn abort(&self, request_id: &str);
}
```
**Layer 0 vs Layer 2 registration API** (ADR-019): `OperationRegistryBuilder`
builds Layer 0 (curated, immutable after startup) via `.with_local()` /
`.with_leaf()` / `.with()`. Layer 2 (per-connection) registration uses
`CallConnection::register_imported()` at runtime — the builder is
Layer-0-only; runtime overlay registration uses `CallConnection` methods.
When the connection drops, the overlay (and all imported ops) is dropped —
no explicit deregistration needed.
The adapter:
1. Accepts bidirectional streams on the connection
2. Reads length-prefixed JSON `EventEnvelope` frames from each stream
3. Resolves the peer's identity using `AuthContext` and `IdentityProvider`
4. Dispatches `call.requested` events to the operation registry
5. Writes response `EventEnvelope` frames back to the appropriate stream
6. Manages the `PendingRequestMap` for outgoing calls
The dispatch loop is **shared** with `CallClient` (ADR-022 §1): both
`CallAdapter::handle` (accept path) and `CallClient::spawn_dispatch`
(connect path — the dial is `AlknetClient::dial_*` per ADR-045) construct
a `Dispatcher` (`protocol/dispatch.rs`) and call `run_loop` — the
dispatch half is one implementation, the connection-establishment half
differs (accept vs dial). Peer authorization flows through the existing
`AccessControl::check(peer_identity)` — no `RemoteFilter`/`remote_safe` gate
(ADR-024 §3). The composition env is peer-keyed (`PeerCompositeEnv`,
ADR-024 §1) to handle head→N-workers routing. See
[client-and-adapters.md](client-and-adapters.md) for the `Dispatcher` mechanism
and [ADR-024](decisions/029-peer-graph-routing-model.md) for the
peer-graph routing model.
### Stream Model
See ADR-015 for the full rationale.
The call protocol uses bidirectional streams with EventEnvelope framing (transport-agnostic — QUIC streams, TCP+TLS, WebTransport, SSH channels, WebSocket — ADR-007). Key properties:
- **Either side can open streams**: The client opens a stream to call a server operation. The server opens a stream to call a client operation. Both use `open_bi()` and `accept_bi()`.
- **Correlation by request ID**: The `id` field in `EventEnvelope` correlates requests with responses. A response arriving on stream N can fulfill a request sent on stream M. The `PendingRequestMap` is keyed by ID, not by stream.
- **Stream usage is the client's choice**: A client may open one stream per operation, one stream for all operations, or any mix. The server processes EventEnvelopes regardless of stream origin.
- **One connection, full access**: A single `alknet/call` connection provides access to all operations (call, subscribe, batch, schema). No need for multiple connections or multiple ALPNs.
### Wire Format: EventEnvelope
Every message on the wire is a length-prefixed JSON `EventEnvelope`:
```rust
pub struct EventEnvelope {
pub r#type: String, // Event type
pub id: String, // Correlation key (request ID, subscription ID)
pub payload: Value, // serde_json::Value — schema depends on event type
}
// Frame: 4-byte big-endian length prefix + UTF-8 JSON body
```
The `Value` type is `serde_json::Value`. The envelope is JSON because it must be consumable from JavaScript, Python, and any language. The envelope itself stays JSON for cross-language compatibility.
Binary payloads (postcard, protobuf) are base64-encoded as a JSON string within the `payload` field. The convention is: if an operation's output schema specifies a binary field, the handler encodes it as a base64 string and the client decodes it. The `EventEnvelope` structure is not aware of this convention — it carries a `serde_json::Value` and does not interpret the payload. This is a handler-level concern, not a protocol-level concern.
This is hand-rolled length-prefixed JSON framing (ADR-014), coincidentally
the same shape irpc uses. The Rust implementation in alknet-call is
canonical — the `@alkdev/pubsub` TypeScript adapters serve as a reference
and browser adaptation, not a parallel implementation (see ADR-033).
### Event Types
Five event types carry request/response and subscription semantics:
| Event | Direction | Purpose |
|-------|-----------|---------|
| `call.requested` | Caller → Handler | Initiate a call or subscription |
| `call.responded` | Handler → Caller | Deliver a result (one for calls, many for subscriptions) |
| `call.completed` | Handler → Caller | Signal end of subscription stream |
| `call.aborted` | Either side | Cancel the call/subscription |
| `call.error` | Handler → Caller | Signal an error |
**A call is a subscribe that resolves after one event.** Both `call()` and `subscribe()` send the same `call.requested` event. The difference is consumption pattern:
- **call()**: Sends `call.requested`, resolves on first `call.responded`
- **subscribe()**: Sends `call.requested`, yields each `call.responded` until `call.completed` or `call.aborted`
The `id` field carries the `requestId` for correlation.
`call.completed` is sent only for subscriptions. A plain `call()` (request/response)
is complete after its single `call.responded`; no `call.completed` follows. The
`PendingRequestMap` entry for a `Call` is deleted on the first `call.responded`.
### `call.requested` Payload
The `payload` of a `call.requested` event has this shape:
```json
{
"operationId": "/fs/readFile",
"input": { ... },
"auth_token": "alk_...", // optional — see Identity Resolution below
"forwarded_for": { // optional (ADR-026) — present when a hub forwards a call
"id": "alice",
"scopes": ["fs:read", "docker:start"],
"resources": {}
}
}
```
- `operationId` — the operation to invoke, **with a leading slash** on the wire (e.g., `/fs/readFile`, `/agent/chat`, `/services/list`). This is the display form of the operation name. The registry stores names without the leading slash (`fs/readFile` — see [operation-registry.md](operation-registry.md#operationspec)); the wire format adds it. The `CallAdapter` strips the leading slash before registry lookup.
- `input` — the operation input, matching the operation's `input_schema` (JSON Schema). Always a `serde_json::Value`.
- `auth_token` — optional. If present, the `CallAdapter` resolves it via `IdentityProvider::resolve_from_token()` and the resulting `Identity` takes precedence over the connection-level identity for this request. See [Identity Resolution](#authcontext-and-identity-resolution) below.
- `forwarded_for` — optional (ADR-026). Present when a `from_call` forwarding handler propagates the originator's identity to a spoke. Carries a serialized `Identity` (id, scopes, resources) — the end user the hub authenticated. **Metadata only** — `AccessControl::check` never reads it; the spoke authorizes the hub (its direct caller), not the end user. The hub may set `forwarded_for: None` if it doesn't want to disclose the originator. See [ADR-026](decisions/032-forwarded-for-identity.md).
The `call.requested` payload does **not** carry an abort policy field. The abort policy (`abort-dependents` vs `continue-running`, ADR-020) is set on `OperationContext` and propagated through `OperationEnv::invoke()` — the composing handler decides the child's policy, not the wire caller. See [Abort Cascade and Nested Calls](#abort-cascade-and-nested-calls) below.
**Leading-slash convention**: `operationId` on the wire always has a leading slash (`/fs/readFile`). `OperationSpec.name` in the registry and in `services/list` responses never has a leading slash (`fs/readFile`). `OperationSpec.path()` produces the wire form (`/fs/readFile`). This is a single rule applied consistently — do not mix the two forms.
### `call.error` Payload
```json
{
"code": "FILE_NOT_FOUND",
"message": "file not found: /etc/nonexistent",
"retryable": false,
"details": { "path": "/etc/nonexistent", "errno": 2 }
}
```
Error codes use an extensible string enum. The protocol defines the following **protocol-level codes** (emitted by the dispatch machinery, not by handlers):
- `NOT_FOUND` — operation not in registry (or Internal op called from wire)
- `FORBIDDEN` — access denied (insufficient scopes or unauthenticated)
- `INVALID_INPUT` — input doesn't match the operation's JSON Schema
- `INVALID_OPERATION_TYPE` — wrong dispatch path for the operation's type (`invoke()` called on a `Subscription`, or `invoke_streaming()` on a `Query`/`Mutation`, or `OperationEnv::invoke()` on a `Subscription` during composition — ADR-021)
- `INTERNAL` — handler error, panic, connection failure
- `TIMEOUT` — request timed out (retryable: true)
Operations may also declare **operation-level domain codes** in their `error_schemas` (ADR-016) — e.g., `FILE_NOT_FOUND`, `RATE_LIMITED`, `INSUFFICIENT_CREDITS`. These are emitted by handlers and carry a `details` payload conforming to the declared `ErrorDefinition.schema`. Protocol-level errors omit `details` or carry protocol-specific context (e.g., the operation name for `NOT_FOUND`).
Fields:
- `code` — the error code (protocol-level or operation-level)
- `message` — human-readable error message. For logging and debugging, not for programmatic handling. Clients should switch on `code`, not parse `message`.
- `retryable` — whether the caller should retry. `true` for transient failures, `false` for permanent ones.
- `details` — optional. When the code matches a declared `ErrorDefinition`, `details` conforms to that definition's schema. This is the typed error payload — it makes errors structured instead of string-matched. See ADR-016.
New error codes may be added in future versions. Clients should treat unknown error codes as `INTERNAL` with `retryable: false`.
### Wire Payload Schemas
The `payload` field of `EventEnvelope` has a different shape per event type:
| Event | `payload` shape |
|-------|----------------|
| `call.requested` | `{ "operationId": "/fs/readFile", "input": {...}, "auth_token": "alk_..." (optional), "forwarded_for": { "id": "...", "scopes": [...], "resources": {} } (optional, ADR-026) }` |
| `call.responded` | `{ "output": <Value> }` — the operation's output, matching `output_schema` |
| `call.completed` | `{}` — empty object (subscription stream end signal) |
| `call.aborted` | `{}` — empty object (cancellation signal; the `id` identifies which request) |
| `call.error` | `{ "code": "...", "message": "...", "retryable": bool, "details": {...} (optional) }` |
### `ResponseEnvelope` → `EventEnvelope` Conversion
Local dispatch produces `ResponseEnvelope { request_id, result: Result<Value, CallError> }`. The `CallAdapter` converts it to `EventEnvelope` for the wire:
| `ResponseEnvelope` | `EventEnvelope` |
|--------------------|-----------------|
| `Ok(value)` | `{ type: "call.responded", id: request_id, payload: { output: value } }` |
| `Err(call_error)` | `{ type: "call.error", id: request_id, payload: <serialized CallError> }` |
The `request_id` becomes the `id` field. For subscriptions, each `call.responded` is a separate `EventEnvelope` with the same `id`; `call.completed` is `{ type: "call.completed", id, payload: {} }`. The streaming dispatch path (`invoke_streaming()` → write each → write `call.completed`) produces these frames from a `StreamingHandler`'s stream; the single-response path (`invoke()` → write one) produces them from a `Handler`'s future. See ADR-021 and [operation-registry.md](operation-registry.md#handler).
### Protocol Operations
The call protocol defines four top-level operations, expressed through event types and operation names:
| Operation | Event Pattern | Description |
|-----------|--------------|-------------|
| **call** | `call.requested` → `call.responded` or `call.error` | Request/response — one result |
| **subscribe** | `call.requested` → many `call.responded` → `call.completed` or `call.aborted` | Streaming — zero or more results |
| **batch** | multiple `call.requested` (different IDs) → multiple `call.responded` | Multiple operations in one round |
| **schema** | `call.requested` name `services/list` or `services/schema` → `call.responded` | Discover available operations |
Batch is not a separate event type — it's multiple `call.requested` events with different request IDs. The client sends them (on one or many streams) and correlates the responses by ID. See OQ-14.
### Bidirectional Calls
Both sides of the connection can initiate calls. The server can call operations on the client just as the client calls operations on the server.
```
Client Server
│ │
│── open_bi() → stream ─────────────────────────▶│
│── call.requested { id: "c1", ... } ────────────▶│ (client calls server)
│◀─ call.responded { id: "c1", ... } ───────────│
│ │
│◀─ open_bi() ← stream ──────────────────────────│
│◀─ call.requested { id: "s1", ... } ────────────│ (server calls client)
│── call.responded { id: "s1", ... } ───────────▶│
│ │
```
The server calls client operations using the same `PendingRequestMap` and the same `EventEnvelope` format. The operation registry on the client side dispatches `call.requested` events just like the server side.
This enables patterns where the server pushes notifications, requests configuration from the client, or orchestrates workflows that require the client to perform operations.
### Streaming Subscribe Example: LLM Chat
The subscribe operation pattern maps naturally to LLM streaming. An agent handler exposing `/agent/chat` as a subscription receives a `call.requested` event and streams `call.responded` events back as the LLM generates tokens. The output payloads use a normalized streaming UI format (e.g., Vercel AI SDK UI chunks — text-delta, tool-input-delta, etc.):
```
Client Server (agent handler)
│ │
│── open_bi() → stream ──────────────────────────────▶│
│── call.requested { id: "c1", │
│ operationId: "/agent/chat", │
│ input: { messages, model } } │
│ │ handler reads capabilities (API key)
│ │ handler makes HTTP request to LLM provider
│ │ handler normalizes provider SSE → UI chunks
│←─ call.responded { id: "c1", output: { type: "text-start", ... } } │
│←─ call.responded { id: "c1", output: { type: "text-delta", delta: "Hel" } }│
│←─ call.responded { id: "c1", output: { type: "text-delta", delta: "lo" } } │
│←─ call.responded { id: "c1", output: { type: "text-end", ... } } │
│←─ call.completed { id: "c1" } │
```
The API key used for the outbound LLM HTTP request comes from `OperationContext.capabilities`, not from the call protocol input and not from environment variables. See ADR-010 and [operation-registry.md → Capability Injection](operation-registry.md#capability-injection).
### PendingRequestMap
Manages in-flight calls and subscriptions. Correlates `call.responded` events back to the original `call.requested`:
```rust
pub struct PendingRequestMap {
pending: HashMap<String, PendingEntry>,
}
enum PendingEntry {
Call {
tx: oneshot::Sender<Result<Value, CallError>>,
timeout: Instant,
},
Subscribe {
tx: mpsc::Sender<Result<Value, CallError>>,
timeout: Option<Instant>,
},
}
```
When a `call.responded` event arrives:
- If `PendingEntry::Call` → resolve the oneshot, delete entry
- If `PendingEntry::Subscribe` → push to the mpsc channel, keep entry alive
When `call.completed` arrives on a subscription → close the mpsc channel, delete entry.
When `call.aborted` arrives → cancel/drop whichever side initiated it.
A `call.aborted` for an unknown `requestId` is silently discarded.
Timeouts prevent dangling entries. A background task sweeps expired entries periodically.
### CallAdapter Stream Handling
The `CallAdapter::handle()` method:
1. Spawns a task that continuously calls `connection.accept_bi()` to receive incoming streams
2. For each accepted stream, reads `EventEnvelope` frames using `FrameFramedReader`
3. Dispatches `call.requested` events to the operation registry, **branching on `op_type`** (ADR-021):
- **`Query` / `Mutation`** → `OperationRegistry::invoke()` → write one `call.responded` (or `call.error`) `EventEnvelope` frame
- **`Subscription`** → `OperationRegistry::invoke_streaming()` → write each `call.responded` `EventEnvelope` as the stream yields → write `call.completed` on natural stream end (or `call.error` if the stream yields an `Err`). `deadline: None` for subscriptions (unbounded — see Timeouts below). Abort (`call.aborted` arriving for the request ID, or the stream being dropped) cascades per ADR-020: the stream future is dropped, `Drop` guards release the handler's resources, and descendants are aborted.
4. Writes response `EventEnvelope` frames using `FrameFramedWriter`
5. Manages `PendingRequestMap` for outgoing calls initiated by the server
The streaming branch is the server-side path that makes `Subscription` operations work end-to-end. Without it, a `Subscription` op registered with a `StreamingHandler` had no server-side dispatch path — the handler produced a stream but the dispatcher only read one `ResponseEnvelope` and closed. ADR-021 adds the `StreamingHandler` type and the `invoke_streaming()` dispatch path; this section wires them into the accept loop. See [operation-registry.md](operation-registry.md#handler) for the `Handler` / `StreamingHandler` / `HandlerKind` types.
For outgoing calls (server → client), the adapter:
1. Opens a bidirectional stream with `connection.open_bi()`
2. Sends `call.requested` on that stream
3. Adds the request ID to the `PendingRequestMap`
4. Reads responses from any stream, correlates by ID
### AuthContext and Identity Resolution
The `CallAdapter` receives an `AuthContext` from the endpoint. The call protocol resolves identity per-request, not per-connection:
**Resolution flow**:
1. The endpoint provides `AuthContext` with whatever identity it resolved at the TLS layer (e.g., client certificate fingerprint). This may be `None` — the `AuthContext.identity` field is `Option<Identity>`.
2. When a `call.requested` event arrives, the `CallAdapter` constructs an `OperationContext` with the connection-level `AuthContext.identity`.
3. If the `call.requested` payload includes an `auth_token` field, the `CallAdapter` resolves it using `IdentityProvider::resolve_from_token()`. If resolution succeeds, the resulting `Identity` replaces the connection-level identity in the `OperationContext`. If resolution fails, the request proceeds with the connection-level identity (which may be `None`).
4. The `OperationContext.identity` is passed to the `OperationRegistry` for ACL checking.
5. If `identity` is `None` and the operation's `AccessControl` has restrictions, the registry returns `FORBIDDEN` with message `"authentication required"`.
**Key point**: Identity is resolved per-request, not per-connection. This allows a single connection to upgrade authentication mid-session (e.g., after an `auth/login` operation returns a token), and allows different operations on the same connection to have different identity levels.
### Root OperationContext Construction
When a `call.requested` arrives from the wire, the `CallAdapter` constructs the root `OperationContext` — the entry point of the call tree. This is the counterpart to `OperationEnv::invoke()` (which constructs nested contexts with `internal: true`): the wire path sets `internal: false`, meaning ACL runs against the caller's `identity`, not a handler's composition authority (ADR-017, ADR-018).
```rust
// CallAdapter dispatch path — root context for an incoming wire request
fn build_root_context(
&self,
request_id: String,
operation_name: &str, // looked up in registry for the registration bundle
identity: Option<Identity>, // resolved per-request above (caller's identity)
forwarded_for: Option<Identity>, // from call.requested.forwarded_for (ADR-026)
) -> OperationContext {
let registration = self.registry.registration(operation_name);
OperationContext {
request_id,
parent_request_id: None, // wire request — top of the call tree
identity: identity.clone(), // caller's identity (inbound — gate credential)
// Composition authority from the registration bundle (ADR-018).
// None for leaves (FromOpenAPI/FromMCP/FromCall); Some for Local/Session.
// This is on the context for PROPAGATION to children via invoke(),
// not for the root's own ACL (which uses identity above).
handler_identity: registration.composition_authority.clone(),
// Forwarded-for identity (ADR-026): the originator when this call was
// forwarded by a from_call handler. Metadata only — AccessControl::check
// never reads it; ACL always authorizes `identity` (the direct caller).
// None when the call wasn't forwarded or the forwarder chose not to
// propagate it. Populated from the wire call.requested.forwarded_for
// field; NOT inherited by composed children (wire-ingress only).
forwarded_for,
capabilities: registration.capabilities.clone(), // from the registration bundle
metadata: HashMap::new(), // fresh per request
deadline: Some(Instant::now() + self.default_timeout), // root deadline (W7)
scoped_env: registration.scoped_env.clone()
.unwrap_or_else(ScopedPeerEnv::empty), // from the bundle, empty for leaves
// Per-call env composition (ADR-019 + ADR-024): the root env is a
// PeerCompositeEnv — the curated base + this connection's imported-
// ops overlay (peer-keyed in the head's aggregation env, ADR-024 §1)
// + the active session overlay (if any). The CallAdapter builds this
// composite per incoming call — same shape as per-call identity
// resolution via IdentityProvider. Handlers call env.invoke() (peer-
// agnostic) or env.invoke_peer(peer, ...) (peer-specific, ADR-024 §2);
// the composite routes to the right overlay.
env: self.compose_root_env(/* peer_id, connection_overlay, session */),
abort_policy: AbortPolicy::default(), // abort-dependents (ADR-020 Decision 6)
internal: false, // external call — ACL against caller identity
}
}
```
The `internal: false` here is what makes a wire call a wire call — ACL checks against the caller's resolved `identity`. When a handler subsequently calls `context.env.invoke(...)`, the `OperationEnv::invoke()` path (see [operation-registry.md](operation-registry.md#operationenv)) constructs a nested `OperationContext` with `internal: true`, switching authority to `handler_identity`. The two construction paths — `CallAdapter` for wire-originated, `OperationEnv::invoke()` for composition-originated — are the only places `internal` is set. Handlers cannot set it themselves (the field is module-private for writes — see [operation-registry.md](operation-registry.md#operationcontext) and ADR-017).
The per-call `env` composition (ADR-019 + ADR-024) is the operation-dispatch analogue of the per-call identity resolution the CallAdapter already does via `IdentityProvider`. Both are integration-point patterns: the trait object owns the routing, the CallAdapter supplies the right sources per call. A connection's imported-ops overlay is part of the root env only for calls arriving on that connection — and on a head node with multiple worker connections, the overlays are peer-keyed (`PeerCompositeEnv`, ADR-024 §1); a session overlay is part of the root env only when a session is active. See ADR-019, ADR-024, and the `PeerCompositeEnv` sketch in [operation-registry.md](operation-registry.md#operationenv).
### ResponseEnvelope
The universal return type from all operation invocations:
```rust
pub struct ResponseEnvelope {
pub request_id: String,
pub result: Result<Value, CallError>,
}
pub struct CallError {
pub code: String, // protocol-level (NOT_FOUND, FORBIDDEN, ...) or operation-level (ADR-016)
pub message: String, // human-readable, for logging — not for programmatic handling
pub retryable: bool,
pub details: Option<Value>, // typed error payload, conforms to ErrorDefinition.schema (ADR-016)
}
```
Local dispatch produces `ResponseEnvelope` with no serialization overhead. The `CallAdapter` converts `ResponseEnvelope` to `EventEnvelope` for the wire. When a handler returns a `CallError` whose `code` matches a declared `ErrorDefinition`, the `details` field carries the typed error payload. See ADR-016.
### Connection and Stream Lifecycle
**Connection drop**: When the transport connection closes (QUIC close, TCP FIN, WebSocket close — the `Connection` reports `ConnectionClosed`), all pending requests in the `PendingRequestMap` are failed with `call.error` code `INTERNAL` and message `"connection closed"`. All subscription channels are closed. The `CallAdapter::handle()` method returns `Ok(())` (clean shutdown) or `Err(HandlerError::ConnectionClosed)` (unexpected).
**Stream reset**: When a stream is reset mid-operation (QUIC stream reset, TCP connection drop, or any transport-level error — the `FrameFramedReader` returns an error), the `PendingRequestMap` entry is removed and the mpsc channel is closed. If the stream was carrying a call, the oneshot is resolved with an error. No `call.aborted` is sent — the stream is gone.
**Timeouts**: Default timeout for wire calls is 30 seconds, configurable via
`CallAdapter::with_timeout()`. The `build_root_context` sets
`OperationContext.deadline` to `now + default_timeout`. Composed calls
inherit the parent's deadline (children do **not** get a fresh 30s — the
root call's deadline bounds the entire call tree, preventing a depth-5
composition from running 150s). A composed call that exceeds the deadline
is cancelled (future dropped, `Drop` guards release resources) and returns
`CallError { code: "TIMEOUT", retryable: true }`. Subscriptions default to
no deadline (`deadline: None` — unbounded); the client can specify a
timeout in the `call.requested` payload. The `PendingRequestMap` sweeper
runs every 10 seconds and removes expired wire entries.
**Error handling in `CallAdapter::handle()`**: If a handler panics, the stream is closed and the `PendingRequestMap` entry (if any) is cleaned up by the next sweeper pass. Other streams and the connection are unaffected.
### Abort Cascade and Nested Calls
When a handler composes other operations via `OperationEnv::invoke()`, it creates a call tree: a parent request (r1) spawns children (r1-a, r1-b), which may spawn their own children. The `parent_request_id` field on `OperationContext` records this tree — it is the agency chain (ADR-017).
When `call.aborted` arrives for a parent request, the protocol cascades the abort to all non-terminal descendants in the tree. The CallAdapter walks the tree (indexed by `parent_request_id` in `PendingRequestMap`) and sends `call.aborted` for each descendant. The default policy is **`abort-dependents`**: aborting a request aborts everything downstream, regardless of branch. This is the correct default because aborted parent work has no consumer waiting for results — continuing is wasted work at best and unwanted side effects at worst (e.g., a `bash/exec` that keeps running after the caller stopped caring).
An opt-in **`continue-running`** policy is available for cases where long-running work should survive a parent's abort (e.g., a subscription that should keep streaming). Under `continue-running`, descendants that have already started continue to completion; descendants that haven't started yet are aborted; no new descendants start.
The abort policy is set on `OperationContext` and propagated through `OperationEnv::invoke()` — the composing handler decides the child's policy, not the wire caller. The `call.requested` payload does not carry an abort policy field (the wire caller doesn't know the composition tree). The root context gets the default (`abort-dependents`); a handler can opt a child into `continue-running` at `invoke()` time. See ADR-020 Decision 6.
Handlers clean up resources when their call is cancelled (in Rust, the future is dropped and `Drop` guards release resources — HTTP streams, file handles, locks). This is a handler-level concern; the protocol's job is to cascade the abort. See ADR-020.
## Constraints
- The call protocol does not depend on any database. `PendingRequestMap` is in-memory. Durable session storage is a consumer concern.
- Operation specs use JSON Schema. The envelope is always JSON. Binary payloads may be base64-encoded in the `payload` field.
- Batch is not a protocol primitive — multiple `call.requested` events with correlated IDs provide equivalent semantics. See OQ-14.
- The call protocol is transport-agnostic at the envelope level. The `EventEnvelope` framing can run over QUIC streams, WebSocket frames, or Worker `postMessage`. The `CallAdapter` is the `ProtocolHandler` implementation that receives a `Connection` (any transport — ADR-007) and dispatches `EventEnvelope` frames. **The `EventEnvelope` shape (`{ type, id, payload }`) was derived from the `@alkdev/pubsub` `EventEnvelope` (`/workspace/@alkdev/pubsub/src/types.ts`), which already has a working WebSocket client/server implementation (`event-target-websocket-client.ts` / `event-target-websocket-server.ts`) and a generalized "event target" abstraction. The call protocol refined the envelope with typed event names (`call.requested`, `call.responded`, etc.) and structured payloads; the delta is small and well-defined, making a browser (and Node) WebSocket client straightforward to derive from the pubsub prior art. See ADR-044, [ADR-048](decisions/048-websocket-native-session-not-gateway.md), and [websocket.md](../http/websocket.md).
- `OperationEnv::invoke()` dispatches through the local registry. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer. See ADR-014 and OQ-13.
- **The call protocol carries no secret material.** Secret material (private keys, API keys, mnemonics, decrypted credentials, raw tokens) must not appear in `call.requested` payloads, `call.responded` payloads, or `OperationContext.metadata`. The wire format carries `serde_json::Value` and cannot enforce this at the type level — the constraint is architectural, enforced by the operation registry and by convention. Operations that need to share public key material use a dedicated operation that returns only the public component. See ADR-010.
- **Abort cascades to descendants.** `call.aborted` for a parent request cascades to all non-terminal descendants in the call tree. Default policy is `abort-dependents`; `continue-running` is an opt-in. See ADR-020.
## Design Decisions
| Decision | ADR | Summary |
|----------|-----|---------|
| Hand-rolled EventEnvelope framing (irpc never integrated) | [ADR-014](decisions/064-irpc-never-integrated-hand-rolled-framing.md) | Hand-rolled length-prefixed JSON framing, operation registry, dispatch; supersedes ADR-013 (irpc was never imported) |
| Call protocol stream model | [ADR-015](decisions/012-call-protocol-stream-model.md) | Bidirectional streams, EventEnvelope, ID-based correlation |
| ALPN per connection | [ADR-004](decisions/006-alpn-convention-and-connection-model.md) | `alknet/call` is a distinct ALPN, one connection per ALPN |
| ProtocolHandler receives Connection | [ADR-005](decisions/007-bistream-type-definition.md) | CallAdapter gets Connection, can accept/open multiple streams |
| Vault integration point | [ADR-008](decisions/008-secret-service-integration.md) | Vault is a capability source, accessed at assembly time |
| Secret material flow | [ADR-010](decisions/014-secret-material-flow-and-capability-injection.md) | Call protocol carries no secret material; capabilities injected at assembly layer |
| Privilege model and authority context | [ADR-017](decisions/015-privilege-model-and-authority-context.md) | `internal` = authority switch not ACL skip; External/Internal visibility; handler identity + scoped env |
| Abort cascade for nested calls | [ADR-020](decisions/016-abort-cascade-for-nested-calls.md) | `call.aborted` cascades to descendants; default `abort-dependents`, `continue-running` opt-in |
| Call protocol client and adapter contract | [ADR-022](decisions/017-call-protocol-client-and-adapter-contract.md) | `CallClient` opens connections; `from_call` imports remote ops; connection direction independent of call direction. Client/adapter surface specced in [client-and-adapters.md](client-and-adapters.md) |
| Handler registration, provenance, and composition authority | [ADR-018](decisions/022-handler-registration-provenance-and-composition-authority.md) | Registration bundle carries provenance, composition authority, scoped env, capabilities; dispatch path reads from bundle |
| Peer-graph routing model (supersedes ADR-023) | [ADR-024](decisions/029-peer-graph-routing-model.md) | Peer-keyed overlays + `PeerRef` routing; `AccessControl`-based peer authorization; retires `remote_safe`/`trusted_peer` |
| Forwarded-for identity | [ADR-026](decisions/032-forwarded-for-identity.md) | `forwarded_for` field on `call.requested` and `OperationContext`; metadata only — `AccessControl::check` never reads it; the `from_call` handler populates it |
| Operation error schemas | [ADR-016](decisions/023-operation-error-schemas.md) | Operations declare domain errors; `call.error` carries typed `details` |
| Streaming handler for subscriptions | [ADR-021](decisions/049-streaming-handler-for-subscriptions.md) | `StreamingHandler` type, `invoke_streaming()` dispatch path, `INVALID_OPERATION_TYPE` protocol code; the server-side streaming branch in `handle_stream` |
## Open Questions
See [open-questions.md](open-questions.md) for full details.
- **OQ-13** (resolved): Operation path format is `/{service}/{op}`. Remote dispatch is a separate mechanism, not a path prefix.
- **OQ-14** (resolved): Batch is a client-side pattern of correlated `call.requested` events, not a protocol primitive.
- **OQ-16** (resolved by ADR-010): No vault operations are exposed over the call protocol for now.
- **OQ-19** (resolved): Session-scoped operation registries — agent-written operations overlaid on global registry via `OperationEnv` trait layering. Protocol doesn't need changes; `OperationEnv` must remain a trait.
- **OQ-25** (dissolved by ADR-024): `remote_safe` marking shape — moot;
`remote_safe`/`trusted_peer` retired; peer authorization is
`AccessControl::check(peer_identity)`.
- **OQ-26** (resolved): `OperationAdapter` error type — `AdapterError`
variants (`DiscoveryFailed`, `SchemaParse`, `Transport`, `Unauthorized`,
`SamePeerCollision`); `#[non_exhaustive]`. See
[client-and-adapters.md](client-and-adapters.md).
- **OQ-27** (resolved): `from_call` re-import trigger — `from_call` is a
manual free function; the assembly layer calls it after the dial (in
`AlknetClient`). See
[ADR-028](decisions/069-from-call-manual-free-function.md).
- **OQ-28** (resolved): `from_call` namespace collision — same-peer collision
= error; cross-peer dissolved by ADR-024 (separate sub-overlays). See
[client-and-adapters.md](client-and-adapters.md).
- **OQ-29** (resolved): `CallClient` TLS client-auth — wire quinn client-auth
(present Ed25519 key as raw public key client cert); key-type-aware server
cert verification; fingerprint normalization. See
[client-and-adapters.md](client-and-adapters.md).
- **OQ-30** (resolved): `PeerRef::Any` routing policy — insertion-order
first-match. See [client-and-adapters.md](client-and-adapters.md).
- **OQ-31** (resolved): `services/list-peers` re-export semantics — opt-in;
`services/list` is "own ops only." See [client-and-adapters.md](client-and-adapters.md).
- **OQ-32** (open, feature extension): Multi-hop federation — the one-hop
model is the architectural commitment; multi-hop is a feature extension
that doesn't break downstream. See [client-and-adapters.md](client-and-adapters.md).
- **OQ-33** (resolved by ADR-025): `PeerId` source — `Identity.id` from
`IdentityProvider` resolution (= `PeerEntry.peer_id`, stable across key
rotation), not a connection-assigned UUID.
- **OQ-34** (resolved by ADR-025 + ADR-033): Persistent peer registry —
the storage boundary is `core trait + in-memory default`; persistence
adapters are separate crates.
- **OQ-37** (resolved by ADR-034): X.509 outgoing-only case — three remote
roles named (public X.509 endpoint, transport relay, hub); pure-client
X.509 connections are not in the peer graph on the client side. See
[client-and-adapters.md](client-and-adapters.md).
## References
- [operation-registry.md](operation-registry.md) — OperationSpec, Handler, AccessControl, service discovery
- [client-and-adapters.md](client-and-adapters.md) — CallClient, from_call, OperationAdapter, peer-keyed composition env
- ADR-014: Hand-rolled EventEnvelope framing (irpc never integrated; supersedes ADR-013)
- ADR-015: Call protocol stream model
- ADR-024: Peer-graph routing model (peer-keyed overlays + `PeerRef` routing)
- ADR-025: PeerEntry and Identity.id decoupling (`PeerId` source)
- ADR-026: Forwarded-for identity (`forwarded_for` on `call.requested` and `OperationContext`)
- ADR-034: Outgoing-only X.509 and the three peer roles
- ADR-021: Streaming handler for subscriptions (server-side streaming dispatch path)
- Reference implementation: `/workspace/@alkdev/alknet-main/crates/alknet-core/src/call/`

190
docs/architecture/channel-client.md Normal file
View File

@@ -0,0 +1,190 @@
---
status: draft
last_updated: 2026-07-18
---
# channel-client.md — ChannelClient
The client side of a channels connection. ADR-043 is the decision; this doc
specifies the API.
## What
`ChannelClient` is the symmetric counterpart to `ChannelsAdapter` (ADR-039).
The server side is a `ProtocolHandler` (`ChannelsAdapter::handle`); the
client side takes over an established transport `Connection`, runs the
demux/mux, and exposes `open_channel(alpn, params) -> Channel` to the
application.
This is the channels analogue of `CallClient` (server: `CallAdapter`;
client: `CallClient`) in the call protocol.
## API
```rust
pub struct ChannelClient {
manager: ChannelManager,
// The transport-side demux/mux, running in a background task.
...
}
impl ChannelClient {
/// Construct a `ChannelClient` over a pre-established transport
/// `Connection` on ALPN `alknet/channels`. This is the
/// transport-agnostic primary constructor: the caller (or a
/// transport-specific dial helper) produces the `Connection` —
/// via `Connection::from_bidi` (TCP+TLS, WebTransport, SSH
/// `direct-tcpip`), a quinn connection, or any other `AsyncRead +
/// AsyncWrite` source — and this method takes over: installs
/// channel 0 (`alknet/call`), spawns the demux/mux, and returns
/// the client. Mirrors the server side's transport-agnostic
/// `ChannelsAdapter::handle(Connection)` and
/// `CallClient::spawn_dispatch(Connection)`.
///
/// This is the one-way-door API surface (ADR-043). It must not be
/// coupled to a transport — the channels protocol is
/// transport-agnostic (ADR-034, as amended by ADR-035; ADR-007,
/// ADR-009), and the client side is half of that protocol.
pub async fn from_connection(connection: Connection)
-> Result<Self, ChannelError>;
/// Open a data channel with the given ALPN and params. Sends
/// `channel/open` on channel 0, waits for the response, and returns
/// the channel.
pub async fn open_channel(
&self,
alpn: &str,
params: Value,
direction: ChannelDirection,
) -> Result<Channel, ChannelError>;
/// Subscribe to the peer's resource updates. Returns a stream of
/// resource-set events (ADR-037 channel/resources/subscribe). Each
/// event carries the JSON `output.resources` array from ADR-037's
/// `channel/resources/subscribe` response shape.
pub async fn subscribe_resources(&self)
-> Result<BoxStream<ResourceEvent>, ChannelError>;
/// The call-protocol connection on channel 0, for invoking channel
/// lifecycle operations and any other call ops the peer exposes.
pub fn call(&self) -> &CallConnection;
}
pub enum ChannelDirection {
InitiatorToResponder,
ResponderToInitiator,
}
pub struct Channel {
pub channel_id: u32,
/// The channel's BiStream, accessible via the BidiStreamSource
/// (accept_bi — ADR-038 as amended by ADR-035).
pub source: ChannelBidiStreamSource,
}
/// One event from `channel/resources/subscribe`. Wraps the JSON `output`
/// object from ADR-037's subscribe response — the `resources` array
/// describing what ALPNs the peer exposes and with what `access` preview.
/// The channels crate maps the JSON to this typed struct; the fields mirror
/// ADR-037's response shape.
pub struct ResourceEvent {
pub resources: Vec<ResourceEntry>,
}
pub struct ResourceEntry {
pub alpn: String,
pub backends_or_targets: Vec<String>, // ALPN-specific enumeration
pub access: Value, // preview of AccessControl (advisory)
}
```
## Transport-agnostic by construction
`ChannelClient` is the client side of the channels protocol. The channels
protocol is transport-agnostic (ADR-034 substrate modes, as amended by
ADR-035; `Connection::from_bidi`/`from_source` from ADR-007/070/092 take
any `AsyncRead + AsyncWrite`). The client side must not be welded to a
transport — that would repeat the server-side welding ADR-007 explicitly
unwound.
`from_connection(connection: Connection)` is the primary constructor and
the one-way-door API surface. It takes a pre-established `Connection` and
takes over channels establishment. The transport is the caller's concern:
`Connection::from_bidi(tls_stream, ...)` for TCP+TLS, a quinn `Connection`,
a WebTransport `BiStream`, an SSH `direct-tcpip` channel wrapped via
`from_bidi`, a WebSocket carrying `alknet/channels` (the browser path per
ADR-044) — all produce a `Connection` that `from_connection` accepts
unchanged. This mirrors the server side's `ChannelsAdapter::handle(Connection)`, which is substrate-agnostic by the same mechanism.
The dial (QUIC, TCP+TLS, iroh) lives in `AlknetClient` (`alknet-client`,
ADR-045), not on `ChannelClient`. Callers compose
`AlknetClient::dial_quic(...).await?` + `ChannelClient::from_connection(conn).await?`
— two lines, the dial then the take-over. Keeping the dial off
`ChannelClient` avoids `alknet-channels-call` depending on `alknet-client`;
the protocol crates are parallel to the dial, not downstream of it.
The credential/verifier-selection rule (ADR-034) lives in the dial
(`AlknetClient`), not in `from_connection` — `from_connection` receives
an already-established, already-authenticated `Connection`, exactly as
`ChannelsAdapter::handle` does on the server side.
## Bidirectionality preserved
The channels protocol is bidirectional — either side can open a channel
(ADR-037 §direction semantics). `ChannelClient::open_channel` supports both
`ChannelDirection::InitiatorToResponder` and
`ChannelDirection::ResponderToInitiator`. The client is not "the client
side" in the request/response sense — it can also receive `channel/open`
requests from the peer (the peer initiates, the client's `ChannelManager`
responds). This mirrors the call protocol's operation overlay (each side
populates what operations they expose).
`ChannelClient` is one endpoint of a bidirectional channels connection. The
name follows the `CallClient` convention (the side that dialed), not a
request/response role.
## Relationship to `AlknetClient`
`ChannelClient`'s *API* is transport-agnostic — `from_connection` takes a
pre-established `Connection`. The shared *dial+TLS* seam
(`AlknetClient`, OQ-55) is [`alknet-client`](../client/README.md), which
provides `AlknetClient` with three dial methods (`dial_quic` /
`dial_tcp_tls` / `dial_iroh`), each producing a `Connection` that
`from_connection` consumes. The dial is transport-specific (QUIC,
TCP+TLS, iroh); the take-over (`from_connection`) is
transport-agnostic. The two concerns are separated.
`AlknetClient::dial_quic` is the dial that feeds `from_connection`. A
caller that needs transport selection (QUIC with TCP+TLS fallback) uses
`AlknetClient` directly; the fallback policy is a caller concern. See
[ADR-045](decisions/089-alknetclient-native-dial-seam.md) for the
full decision and [OQ-55](../../questions/055-alknetclient-establishment-extraction.md)
(resolved).
## Design Decisions
All design decisions are documented as ADRs in [decisions/](decisions/).
| ADR | Decision | Summary |
|-----|----------|---------|
| [080](decisions/080-channelclient.md) | ChannelClient | Client side; transport-agnostic `from_connection` primary; dial lives in `AlknetClient` (ADR-045, resolves OQ-55) |
| [093](decisions/093-channels-pure-channel-multiplexing.md) | channels Pure Channel Multiplexing | No `stream_types` on `open_channel`/`Channel`; handler owns sub-stream multiplexing |
## Open Questions
- **OQ-55** (resolved by ADR-045): `AlknetClient` core **dial+TLS seam**
— `alknet-client` with three dial methods. `ChannelClient`'s API is
transport-agnostic (`from_connection`); the dial is the shared seam.
See [ADR-045](decisions/089-alknetclient-native-dial-seam.md).
## References
- ADR-043: ChannelClient (the decision)
- ADR-035: channels pure channel multiplexing (no `stream_types`)
- ADR-037: channel lifecycle operations (`open_channel` sends `channel/open`)
- ADR-038: ChannelBidiStreamSource (what `Channel.source` wraps, as
amended by ADR-035 — `accept_bi` yields a `BiStream`)
- ADR-039: ChannelManager (the shared state `ChannelClient` holds)
- OQ-55: AlknetClient / client establishment extraction
- `docs/architecture/crates/call/client-and-adapters.md` — `CallClient` (the
shape `ChannelClient` mirrors)

427
docs/architecture/channel-operations.md Normal file
View File

@@ -0,0 +1,427 @@
---
status: draft
last_updated: 2026-07-18
---
# channel-operations.md — Channel Lifecycle on the Call Protocol
Channel lifecycle is orchestrated by the call protocol on channel 0
(ADR-036). Four operations on channel 0's `OperationRegistry` (ADR-037)
handle open, close, control, and resource discovery. All four go through
the existing `OperationContext` / `AccessControl::check` path — no new auth
machinery, no new framing.
## The four operations
### `channel/open` — open a data channel
Request (on channel 0):
```json
{
"operation": "channel/open",
"input": {
"alpn": "alknet/tty",
"params": { "backend": "docker", "cmd": ["bash"], "container": "abc123" },
"direction": "initiator-to-responder"
}
}
```
| field | type | meaning |
|-------|------|---------|
| `alpn` | string | The ALPN the channel will carry. Responder looks this up in its `HandlerRegistry`. |
| `params` | object | ALPN-specific parameters. For `alknet/tty` this is `NegotiateRequest`. For `alknet/tunnel` this is the target resource. The channels layer does not interpret `params`. |
| `direction` | string | `initiator-to-responder` or `responder-to-initiator`. See "Direction semantics" below. |
Response:
```json
{
"output": {
"channel_id": 7
}
}
```
| field | type | meaning |
|-------|------|---------|
| `channel_id` | u32 | Server-assigned (DP-1). The responder allocates via monotonic `AtomicU32`. |
**Channel ID allocation: server-assigned (DP-1).** One round-trip before
data flows — the same round-trip the call protocol makes for every
operation. All current channel types (TTY, tunnel, SSH) already require a
negotiation round-trip, so the open round-trip is not additive latency.
**Error codes** (new `CallError.code` strings, not new framing):
| code | meaning | retryable |
|------|---------|-----------|
| `channel:unknown_alpn` | ALPN not in responder's `HandlerRegistry` | false |
| `channel:forbidden` | `AccessControl::check` denied the open | false |
| `channel:allocation_failed` | Handler allocate failed | true (often transient) |
| `channel:invalid_params` | `params` JSON didn't satisfy the ALPN's expectations | false |
| `channel:too_many_channels` | Per-connection channel limit hit (ADR-040) | false |
### `channel/close` — tear down a channel
```json
{
"operation": "channel/close",
"input": { "channel_id": 7, "reason": "exit" }
}
```
The responder (the side that didn't send the close) drains its reassembled
stream for `channel_id`, signals EOF to the handler, and returns
`{ "closed": true }`. The `channel_id` is eligible for reuse after the drain
completes (ADR-040 — monotonic IDs with wrap-around, not a free-list).
`reason` is free-form for observability — not semantically required.
**REQ-CH-06: exit-chunk-before-close ordering.** The channel's data chunks
MUST be written and flushed before the `channel/close` operation is sent on
channel 0. The side closing must observe the data-channel pump complete
before issuing the call operation. For TTY this is the exit-chunk-is-last
invariant (ADR-055) carried forward — the exit control message rides on
TTY's `STREAM_CTRL_OUT` (stream_type 4, inside TTY's 5-byte payload
format); for tunnels it is the last data byte before close. This invariant
crosses two channels (the data channel and channel 0), so the channels
layer owns the ordering guarantee.
### `channel/control` — out-of-band control on channel 0
For control that doesn't need ordering relative to data (resize, signal,
keepalive):
```json
{
"operation": "channel/control",
"input": {
"channel_id": 7,
"message": { "type": "resize", "cols": 80, "rows": 24 }
}
}
```
The channels layer routes `message` to the handler's control handle for
`channel_id`. The `message` JSON is ALPN-specific; the channels layer does
not interpret it.
### `channel/resources/subscribe` — live resource discovery
**This is a `Subscription` operation (ADR-021), not a polled Query.** The
call protocol has `StreamingHandler` / `invoke_streaming` (implemented and
tested). The first consumer (the hub aggregating worker resources) needs
live updates when workers connect/disconnect or containers start/stop.
```json
{
"operation": "channel/resources/subscribe",
"input": {}
}
```
The responder registers a `StreamingHandler` that emits a `ResponseEnvelope`
whenever the resource set changes. Each event:
```json
{
"output": {
"resources": [
{
"alpn": "alknet/tty",
"backends": ["docker", "local"],
"access": { "required_scopes": ["tty:open"] }
},
{
"alpn": "alknet/tunnel",
"targets": ["container:*", "service:postgres"],
"access": { "required_scopes_any": ["tunnel:open", "admin"] }
}
]
}
}
```
| field | type | meaning |
|-------|------|---------|
| `alpn` | string | The ALPN this side accepts `channel/open` for. |
| `backends` / `targets` | `[string]` | ALPN-specific enumeration of what's available. The channels layer doesn't interpret these. |
| `access` | object | A preview of the `AccessControl` that `channel/open` will check. Advisory — lets the initiator fail fast. The real check happens on `channel/open`. |
The stream emits an initial snapshot immediately, then subsequent events on
any change. The stream is long-lived; the subscriber cancels by dropping the
subscription (ADR-020 abort cascade applies).
A `channel/resources` (non-subscribe, `Query`) operation is NOT provided.
The subscription's initial snapshot serves the poll use case (subscribe,
read the first event, cancel). Providing both would be redundant and would
pressure consumers toward the stale-poll path.
## Direction semantics (OQ-CH-09 — pinned)
Channel open is **bidirectional** — either side can initiate. The
`direction` field determines who is the ALPN-server (allocates the handler,
writes the negotiation response) vs the ALPN-client (writes the first
request).
| `direction` | Initiator role | Responder role | Who writes first |
|-------------|----------------|----------------|-------------------|
| `initiator-to-responder` | ALPN-client | ALPN-server | Initiator writes first (the request data); responder's handler is the server side. The common case: "open me a TTY on your docker container." |
| `responder-to-initiator` | ALPN-server | ALPN-client | Responder writes first (the negotiation response); initiator's handler is the client side. The "worker exposes, hub consumes" case: the worker initiates the open to make itself available; the hub is the client. |
**The channels layer does not enforce write order.** Write order is
ALPN-specific, determined by which side is the ALPN-server. The channels
layer routes chunks; the handlers negotiate who writes first via their
ALPN's `params` contract.
**`channel_id` allocation is always by the responder** (DP-1), regardless of
`direction`. The responder is the side that receives the `channel/open` call
operation; it allocates the ID and returns it. In the `responder-to-
initiator` case, the initiator (worker) sends the `channel/open`, so the
responder (hub) allocates the ID — even though the worker is the ALPN-server
for the channel's data. This keeps ID allocation in one place and avoids the
collision-prone client-assigned alternative.
## Control-message division (DP-4 — pinned)
| Control path | When | Examples |
|--------------|------|----------|
| Call operations on channel 0 (`channel/control`, `channel/close`) | Control that doesn't need ordering relative to data, or lifecycle events | resize, signal, keepalive, close |
| Data-ordered bytes on the data channel's `BiStream` (handler-internal framing) | Control that MUST be ordered relative to data | EOF before exit, flush before close |
The TTY crate's exit-chunk-is-last invariant (ADR-055) is the canonical
example of data-ordered control — it rides on TTY's `STREAM_CTRL_OUT`
(stream_type 4, inside TTY's 5-byte payload format) because it must arrive
after the last data on TTY's stdout stream_type, guaranteed by TTY's
per-stream_type chunk ordering within its own 5-byte format, not by a
call-protocol round-trip. The `channel/close` operation that follows is
on channel 0 and is ordered after the data pump completes (REQ-CH-06).
**The control-message division is handler-internal.** Under ADR-035, the
channels layer has no `stream_type` concept — it carries the handler's
framing transparently in the payload. TTY's `STREAM_CTRL_IN` (stream_type
3) and `STREAM_CTRL_OUT` (stream_type 4) are stream_types in TTY's 5-byte
format (ADR-052, amended by Phase 7), not channels-layer concepts. The
channels layer routes by `channel_id` only; the handler owns its
sub-stream multiplexing on the `BiStream` it receives. The
"bidirectional control channel" property is a TTY-layer concern, fixed
at the TTY layer by Phase 7's split — the channels layer doesn't know
about it.
## ACL flow (end-to-end)
A browser opening a TTY channel to a spoke through a hub (ADR-042):
1. Browser's channel 0 → hub's channel 0: `channel/open`
`{ alpn: "alknet/tty", params: { backend: "docker", cmd: ["bash"], container: "abc123" } }`.
The browser's identity is a bearer token (ADR-034).
2. Hub's `CallAdapter` runs `AccessControl::check` on `channel/open` with
the browser's identity. If denied → `channel:forbidden`.
3. Hub forwards to spoke via `from_call`: the hub's `forwarded_for` handler
constructs a `call.requested` with the hub as caller and the browser as
`forwarded_for` (ADR-026 §3). The spoke receives `channel/open` with
`caller = hub`, `forwarded_for = browser`.
4. Spoke's `CallAdapter` runs `AccessControl::check` with the hub as caller
(the spoke authorizes the hub — ADR-011). The spoke's ownership store
verifies the hub (or the `forwarded_for` browser, per policy) owns
`container:abc123`.
5. Spoke allocates the channel via `TtyAdapter` / `DockerTtyBackend`,
returns `channel_id`.
6. Hub opens a matching channel on the browser's side and bridges them
(byte-forward with `channel_id` rewrite — ADR-042).
The hub ran **zero** protocol-specific auth. It ran `channel/open`'s
`AccessControl::check` (call-protocol machinery) and forwarded. The channels
layer inherited the auth model by being a call-protocol operation.
## Per-identity channel cap (ADR-041)
A channel slot is a resource. The cap on how many channels an identity
may hold open is a quota check on that resource — parallel to
`OwnershipProvider::owns` (ADR-011) for spawned resources. Same
primitive, different resource. The cap is a **peer concern**, not a
hub-specific concern: any accepting peer (worker or hub) enforces the
cap on its inbound channels, just as it enforces `AccessControl::check`
on `channel/open`. The cap is also **symmetric** — both sides of a
channels connection enforce their cap on the other's channels.
### Why the cap is not in the channels layer
`ChannelManager` (ADR-039) is auth-blind by design — no auth state, no
identity, no scopes. That decision is load-bearing (it is what makes
the channels layer WASM-compatible, transport-agnostic, and
ALPN-blind). So the per-identity cap lives in `channels-call`, where
the identity is already on `OperationContext` (the same place
`AccessControl::check` runs). The channels layer (`channels-core`) is
unchanged. See ADR-041 §"Why the channels layer cannot hold the cap".
The channels-layer per-connection `max_channels = 256` (ADR-040) is
a **per-connection memory bound** (limits one connection's
reassembly-buffer cost), not a DoS defense. A peer can open an
unbounded number of transport connections, so a per-connection cap is
not a per-peer DoS defense. The per-identity DoS defense is the cap
documented here; see ADR-041 for the corrected DoS-defense framing.
### The `ChannelLifecyclePolicy` trait
```rust
/// Per-identity channel lifecycle policy. Consulted by the
/// `channel/open` handler (after `AccessControl::check`, before
/// allocation) and the `channel/close` handler (after deallocation).
/// Both handlers have the identity via `OperationContext`.
pub trait ChannelLifecyclePolicy: Send + Sync + 'static {
/// Before channel allocation. Deny with `channel:too_many_channels`
/// (ADR-037) when the identity is over its cap. The identity is
/// the direct caller (the peer that opened this channels
/// connection); `forwarded_for` is metadata and is NOT consulted
/// (ADR-026).
fn check_open(&self, identity: &Identity) -> Result<(), ChannelError>;
/// After channel deallocation. Decrement the per-identity count.
/// Called by the `channel/close` handler after the drain completes
/// (ADR-040 §channel-id-reuse).
fn on_close(&self, identity: &Identity);
}
```
### Default: `PerIdentityChannelPolicy::new(256)`
The default constructor enforces 256 per identity out of the box — no
"NoOp default + wire it later." A channels-accepting peer that
constructs `ChannelOperations::new(manager)` with no policy argument
gets `PerIdentityChannelPolicy::new(256)`. The default is secure;
opt-outs are explicit:
- `PerIdentityChannelPolicy::new(cap)` — shared per-identity state
(`HashMap<PeerId, usize>` + cap), constructed **once per accepting
peer** and shared (via `Arc`) across every channels connection that
peer accepts. The sharing is what makes the cap per-identity, not
per-connection.
- `PerIdentityChannelPolicy::with_per_identity_caps(mapping)` —
per-peer-role variant: `HashMap<PeerId, usize>` overrides the
default cap for specific peers. Used by a spoke that serves a
high-fan-out hub (the hub peer's cap is set higher than a worker
peer's cap — see "Relay consequence" below).
- `NoCap` — no cap. Explicit opt-out for tests, POCs, and trusted
single-peer deployments. Not the default.
The policy is constructed once and passed to `ChannelOperations` at
registration time:
```rust
let policy = Arc::new(PerIdentityChannelPolicy::new(256));
let channel_ops = ChannelOperations::new(manager, policy);
channel_ops.register_on(&mut call_registry)?;
```
### Enforcement point: between `AccessControl::check` and allocation
The `channel/open` handler (above) gains the policy check after ACL
and before `next_id.fetch_add`:
1. ACL is already checked by `OperationRegistry::invoke` (the existing
`AccessControl::check` path — unchanged).
2. **NEW:** `policy.check_open(&op_ctx.identity)?` — deny with
`channel:too_many_channels` if over cap.
3. Allocate the `channel_id` via `next_id.fetch_add(1, Relaxed)`
(DP-1: server-assigned — unchanged).
4. Construct the `ChannelBidiStreamSource`, spawn the handler, record
the `ChannelState` (unchanged).
5. Return the `channel_id`.
The `channel/close` handler gains the decrement after the drain
completes (the same point ADR-040 marks the `channel_id` as eligible
for reuse):
1. Drain the reassembly buffer for `channel_id` (existing — ADR-040
§channel-id-reuse).
2. **NEW:** `policy.on_close(&op_ctx.identity)` — decrement the
per-identity count.
3. Return `{ "closed": true }` (unchanged).
### Relay consequence: the spoke caps the hub, not the browser
When the hub relays a browser's channel to a spoke (ADR-042), the
spoke sees the hub as the direct caller. `forwarded_for` carries the
browser's identity as metadata (ADR-026 — `forwarded_for` is not
authority; `AccessControl::check` never reads it). The channel cap
follows the same shape: the spoke's `ChannelLifecyclePolicy` is
consulted with the **hub's** identity, not the browser's. The spoke
asks "does the hub have access to open another channel?" and the
hub's quota on the spoke reflects the aggregate of all relayed
channels. The hub's per-browser caps are the hub's own concern
(enforced on the browser leg by the hub's own policy), not the
spoke's.
This is correct and consistent — the spoke authorizes the hub for
container access the same way it authorizes any peer, and the hub's
browser-relay ACL is the hub's own layer. The channel cap follows the
same pattern as any other resource ACL.
**Deployment consequence:** a spoke that serves a hub relaying for
many browsers must set the hub peer's cap higher than a worker peer's
cap, or the spoke denies legitimate relayed channels when the hub's
aggregate count exceeds a worker-sized cap. This is a per-peer-role
policy, set by the spoke via `with_per_identity_caps`. The
architecture provides the mechanism; the deployment sets the numbers.
This is not a flaw — it is the same shape as any per-peer ACL (a
spoke may authorize one peer for 1000 containers and another for 10;
the channel cap is the same kind of per-peer policy).
### Recursive channels do not bypass the cap
A recursive `alknet/channels`-inside-`alknet/channels` channel runs a
new `ChannelsAdapter` with a new `ChannelManager`. If the same
`ChannelLifecyclePolicy` is wired into the inner `ChannelOperations`,
the inner channels are counted against the same identity. Recursion
is not a bypass; the 13-byte-per-chunk overhead is the documented
cost (ADR-035), and the cap behavior is unchanged. Recursive channels
are an edge case for edge cases and not specced further.
## Hub relay contract (ADR-042 — summary)
The hub **translates**, not transparently forwards:
1. **Call-protocol layer (channel 0): translate.** The hub terminates
channel 0 on both legs. `channel/open` from the browser → hub's
`AccessControl::check` → hub re-issues `channel/open` on the spoke leg
with `forwarded_for` → spoke returns its `channel_id` → hub maps
browser-id ↔ spoke-id.
2. **Data-channel layer: byte-forward with `channel_id` rewrite.** The
relay reads chunks for `browser_id`, rewrites the `channel_id` field to
`spoke_id`, writes onto the spoke's channels connection — and vice versa.
The relay does not parse the payload.
`channel/control` operations on channel 0 carry `channel_id` in their JSON
payload; the hub's `CallAdapter` translates these too (rewrites
`channel_id` in the payload). The relay does not touch `channel/control` —
it's a call operation, translated, not byte-forwarded.
The hub never runs a handler for `alknet/tty`, `alknet/ssh`, or
`alknet/tunnel`. It runs `alknet/channels` (the relay) and `alknet/call`
(for its own hub-level operations + translation).
## Design Decisions
All design decisions are documented as ADRs in [decisions/](decisions/).
| ADR | Decision | Summary |
|-----|----------|---------|
| [073](decisions/073-channel-lifecycle-operations.md) | Channel Lifecycle Operations | The four ops; `direction` pinned; subscribe not poll |
| [072](decisions/072-channel-0-pre-negotiated-call.md) | Channel 0 Pre-Negotiated | Channel 0 = `alknet/call` |
| [079](decisions/079-hub-relay-translate-not-forward.md) | Hub Relay | Translate channel 0, byte-forward data channels |
| [094](decisions/094-per-identity-channel-cap.md) | Per-Identity Channel Cap | 256 per `PeerId`, enforced via `ChannelLifecyclePolicy` in `channels-call`; per-connection `max_channels` reframed as a memory bound |
| [093](decisions/093-channels-pure-channel-multiplexing.md) | channels Pure Channel Multiplexing | No `stream_types` on `channel/open`; no `stream_type` on `channel/control`; handler owns sub-stream multiplexing |
| [049](decisions/049-streaming-handler-for-subscriptions.md) | StreamingHandler | The machinery `channel/resources/subscribe` uses |
| [032](decisions/032-forwarded-for-identity.md) | Forwarded-For Identity | The auth chain for hub-relayed opens (and why the cap is per direct-caller, not per `forwarded_for`) |
| [050](decisions/050-dynamic-resource-ownership-for-runtime-spawned-resources.md) | Dynamic Resource Ownership | The parallel — a channel slot is a resource, the cap is a quota check |
## References
- ADR-037: channel lifecycle operations (the decision)
- ADR-041: per-identity channel cap (the cap, the trait, the relay
consequence)
- ADR-042: hub relay (the translate contract)
- `docs/research/alknet-channels/phase-0-findings.md` §Channel Open
Negotiation, §ACL and Security Model

153
docs/architecture/channels-README.md Normal file
View File

@@ -0,0 +1,153 @@
---
status: draft
last_updated: 2026-07-18
---
# alknet-channels
A multiplexing proxy: a `ProtocolHandler` on `alknet/channels` that
decomposes a single bidirectional transport stream into N logical channels,
each carrying a different ALPN. Channel 0 is pre-negotiated as `alknet/call`
(ADR-036); every other channel is opened dynamically via call operations on
channel 0 and routed through the same `HandlerRegistry` as top-level
connections. The channels layer is a re-framing proxy — it converts between
"one transport stream carrying N channels" (the wire) and "N independent
`BiStream` handles" (what handlers see) — and it does no protocol work
itself. The channels layer has no `stream_type` concept (ADR-035); the
handler owns its sub-stream multiplexing on the `BiStream` it receives.
## Documents
| Document | Status | Description |
|----------|--------|-------------|
| [overview.md](overview.md) | draft | Crate purpose, the multiplexing collapse, dependencies, ALPN, transport agnosticism, WASM, relationship to existing crates |
| [channels-wire.md](channels-wire.md) | draft | The 8-byte chunk format (`[channel_id:u32 be][length:u32 be][payload]`), the add/strip composition, sentinels, framing disambiguation, wire-level invariants (REQ-CH-01..05) |
| [channels-connection.md](channels-connection.md) | draft | `ChannelBidiStreamSource` (implements `BidiStreamSource` — ADR-008/074, as amended by ADR-035), `accept_bi` yields one `BiStream` per channel, recursive composition |
| [channels-adapter.md](channels-adapter.md) | draft | `ChannelsAdapter` (`ProtocolHandler` on `alknet/channels`), `ChannelManager`, demux/mux contracts (REQ-CH-01..04), the two-pump pattern (ADR-078) |
| [channel-operations.md](channel-operations.md) | draft | `channel/open`, `channel/close`, `channel/control`, `channel/resources/subscribe` — call-protocol operations on channel 0, ACL flow, `direction` semantics, the hub relay contract (ADR-042) |
| [channel-client.md](channel-client.md) | draft | `ChannelClient` — the client side of a channels connection; transport-agnostic `from_connection` primary; dial lives in `AlknetClient` (ADR-045); bidirectionality preserved |
## Applicable ADRs
| ADR | Title | Relevance |
|-----|-------|-----------|
| [071](decisions/071-channels-wire-format.md) | channels Wire Format — 8-Byte Chunk Header | The chunk format; channels layer has no `stream_type` concept (amended by ADR-035); substrate-agnostic; one-way door |
| [093](decisions/093-channels-pure-channel-multiplexing.md) | channels Pure Channel Multiplexing | The umbrella decision: 8-byte header, no `stream_type`, `into_sub_streams` removed, `BiStream`-only, TTY always 5-byte |
| [072](decisions/072-channel-0-pre-negotiated-call.md) | Channel 0 Is Pre-Negotiated `alknet/call` | Channel 0 = call protocol; no special control plane |
| [073](decisions/073-channel-lifecycle-operations.md) | Channel Lifecycle Operations on the Call Protocol | `channel/open`/`close`/`control`/`resources/subscribe`; `direction` semantics; subscribe not poll |
| [074](decisions/074-channelconnection-bidistreamsource.md) | ChannelConnection — BidiStreamSource over Chunk Reassembly | Per-channel `BidiStreamSource` impl; `accept_bi` yields `BiStream` (amended by ADR-035 — `into_sub_streams` removed) |
| [075](decisions/075-channelsadapter-and-channelmanager.md) | ChannelsAdapter and ChannelManager | Substrate-agnostic demux loop; REQ-CH-01..04 contracts |
| [076](decisions/076-backpressure-channel-limits-id-reuse.md) | Backpressure, Channel Limits, and ID Reuse | Bounded-buffer (1 MiB default), 256-channel per-connection memory bound, monotonic IDs with wrap (DoS defense reframed by ADR-041) |
| [094](decisions/094-per-identity-channel-cap.md) | Per-Identity Channel Cap | 256 per `PeerId`, enforced via `ChannelLifecyclePolicy` in `channels-call`; per-connection `max_channels` reframed as a memory bound; symmetric (both sides enforce); spoke caps hub (direct caller), not browser (forwarded_for is metadata) |
| [077](decisions/077-tty-inside-channels.md) | TTY Inside Channels — Sub-Streams, Not Wire Format | TTY's two modes (direct vs channels); TTY always uses its 5-byte format, carried transparently in the channels payload |
| [078](decisions/078-two-pump-shutdown-on-completion.md) | Two-Pump Shutdown-on-Completion Pattern | The two-pump deadlock contract; handler-level, not channels-layer |
| [079](decisions/079-hub-relay-translate-not-forward.md) | Hub Relay — Translate, Not Transparently Forward | The hub translates channel 0, byte-forwards data channels with ID rewrite |
| [080](decisions/080-channelclient.md) | ChannelClient — the Client Side of a Channels Connection | `ChannelClient`, transport-agnostic `from_connection` primary; dial lives in `AlknetClient` (ADR-045, resolves OQ-55) |
| [081](decisions/081-channels-subcrate-decomposition.md) | channels Sub-Crate Decomposition | `channels-core` (pure multiplexer) / `channels-call` (call coupling + ChannelClient); hub and worker are consumers, not sub-crates |
| [070](decisions/070-bidistreamsource-trait.md) | BidiStreamSource Trait | The `Connection` extension point `ChannelBidiStreamSource` implements |
| [092](decisions/092-bistream-as-the-handler-leaf.md) | `BiStream` as the Handler Leaf | `accept_bi` returns `BiStream`; the transport-leaf decision ADR-035 builds on |
| [065](decisions/065-connection-from-stream-generic-single-stream.md) | `Connection::from_stream` | The transport-agnostic `Connection` the channels layer rides on |
| [052](decisions/052-alknet-tty-wire-format-and-two-carriage.md) | alknet-tty Wire Format | The 5-byte format carried transparently in the channels payload (control bidirectional via `STREAM_CTRL_IN`/`OUT` — Phase 7 amendment) |
| [049](decisions/049-streaming-handler-for-subscriptions.md) | StreamingHandler for Subscriptions | The machinery `channel/resources/subscribe` uses |
| [032](decisions/032-forwarded-for-identity.md) | Forwarded-For Identity | The auth chain for hub-relayed channel opens |
| [003](decisions/003-crate-decomposition.md) | Crate Decomposition | alknet-channels depends on alknet-core only; no handler-depends-on-handler |
## Relevant Open Questions
| OQ | Title | Status | Relevance |
|----|-------|--------|-----------|
| OQ-55 | AlknetClient / Client Establishment Extraction | resolved (ADR-045) | `ChannelClient`'s API is decided (ADR-043): transport-agnostic `from_connection` primary; dial lives in `AlknetClient` (`alknet-client`, ADR-045) |
| OQ-56 | Full channel-level flow-control windowing | deferred(scope) | Bounded-buffer is decided (ADR-040); full windowing is an extension blocked on "a real deployment observes HOL blocking on a saturated channel where bounded buffer is insufficient" |
| OQ-57 | Two-pump helper extraction to alknet-core | deferred(scope) | The shutdown-on-completion *contract* is decided (ADR-078); the *helper* extraction is blocked on a second two-pump handler existing (shape convergence) |
| OQ-68 | Add/strip API shape (built-in vs utility) | open | Whether the 8-byte header add/strip is built into the channels read/write path or exposed as a standalone utility. The *contract* is decided (ADR-035); the *function surface* is not |
## Key Design Principles
1. **Streams are streams.** A TTY session, an SSH channel, a forwarded TCP
connection, a QUIC bidi stream — they're all `BiStream` (a concrete
`AsyncRead + AsyncWrite` newtype, per ADR-009). The differences are only
in how they're *opened* (negotiation via `channel/open` on channel 0)
and what *multiplexing layer* carries them (the 8-byte chunk format).
Once normalized, every channel is an ALPN routed through the same
`HandlerRegistry`. See [overview.md](overview.md) and ADR-034 (as
amended by ADR-035).
2. **Channel 0 is `alknet/call` pre-negotiated, not a special control
plane.** The call protocol runs on channel 0 exactly as on a top-level
`alknet/call` connection. Channel lifecycle operations
(`channel/open`, `channel/close`, `channel/control`,
`channel/resources/subscribe`) are call operations on channel 0's
`OperationRegistry`, gated by the existing `AccessControl::check`. No
new auth machinery, no new framing. See ADR-036, ADR-037.
3. **The channels layer is a re-framing proxy, not a protocol engine.** It
converts between "one transport stream carrying N channels" (the wire)
and "N independent `BiStream` handles" (what handlers see). It does no
ALPN-specific parsing, no auth, no transport coupling, and carries no
`stream_type` concept (ADR-035). This makes it WASM-compatible and
transport-agnostic by construction. See [channels-adapter.md](channels-adapter.md)
and ADR-039.
4. **The handler owns its sub-stream multiplexing.** The channels layer
yields one `BiStream` per channel; the handler sub-multiplexes it
however it wants (TTY's 5-byte format, call's length-prefixed JSON,
tunnel's raw bytes, SSH's channel protocol). The channels layer carries
the bytes transparently. See [channels-connection.md](channels-connection.md)
and ADR-035.
5. **`channel/resources/subscribe` is a `Subscription`, not a polled
`Query`.** The call protocol has `StreamingHandler` / `invoke_streaming`
(ADR-021, implemented and tested). The first consumer (the hub
aggregating worker resources) needs live updates. Polling would be built
and immediately reworked. See ADR-037.
6. **Bidirectional open.** Either side can open a channel to the other,
just like the call protocol's operation overlay. The `direction` field
on `channel/open` pins who is the ALPN-server vs ALPN-client. See
ADR-037 §Direction semantics.
7. **Wire-level invariants are contracts, not implementation details.**
The POC surfaced five invariants (REQ-CH-01..04, plus REQ-CH-06 for
close ordering) that hang channels silently if underspecified: shutdown
emits a zero-length sentinel; transport close drops all senders; the mux
supports dynamic registration; unknown `channel_id` is lenient-dropped;
bounded-buffer backpressure doesn't deadlock; data chunks flush before
`channel/close`. See [channels-wire.md](channels-wire.md) and
[channels-adapter.md](channels-adapter.md).
8. **The hub translates, not transparently forwards.** The hub terminates
channel 0 on both legs, runs `AccessControl::check`, and re-issues
`channel/open` on the spoke leg with `forwarded_for` (ADR-026). Data
channels are byte-forwarded with `channel_id` rewrite (a 4-byte rewrite
within the 8-byte header). This preserves the auth model. See ADR-042.
9. **The channel cap is per-identity, not per-connection.** A channel
slot is a resource; the cap on how many an identity may hold open is
a quota check, parallel to `OwnershipProvider::owns` (ADR-011) for
spawned resources. The cap lives in `channels-call` (the channels
layer is auth-blind by ADR-039 — no identity, no scopes), consulted
by the `channel/open` and `channel/close` handlers after
`AccessControl::check`. The default is `PerIdentityChannelPolicy::
new(256)` — 256 per `PeerId` across all the peer's connections. The
per-connection `max_channels` (ADR-040) is a memory bound, not a
DoS defense. The cap is symmetric (both sides enforce); the spoke
caps the hub as direct caller, not the browser as `forwarded_for`
(metadata, not authority — ADR-026). See ADR-041.
## References
- `docs/research/alknet-channels/phase-0-findings.md` — Phase 0 research
(vision, hub motivation, wire format, negotiation, internals, DPs, OQs)
- `docs/research/alknet-channels/poc-summary.md` — the de-risk POC (28
tests, three validated targets, REQ-CH-01..06 wire-level invariants
surfaced; REQ-CH-07 is a cosmetic clippy item, not a wire invariant)
- `docs/research/alknet-channels/poc-plan.md` — the POC plan
- `docs/research/stream-unification/findings.md` — the research that
surfaced the pure-channel-multiplexing resolution (ADR-035)
- `/workspace/alknet-channels-poc/` — the POC codebase
- `docs/research/alknet-tty/phase-0-findings.md` — the TTY crate's chunk
format (the seed of the channels generalization)
- `docs/research/alknet-ssh/phase-0-findings.md` — SSH's channel
multiplexer (the prior art for N-channel multiplexing)
- `docs/architecture/crates/hub/README.md` — the hub crate (the primary
consumer; the relay implementation's home)

318
docs/architecture/channels-adapter.md Normal file
View File

@@ -0,0 +1,318 @@
---
status: draft
last_updated: 2026-07-18
---
# channels-adapter.md — ChannelsAdapter and ChannelManager
The two internal components of the channels crate: the read/demux half
(`ChannelsAdapter`) and the reassemble/allocate half (`ChannelManager`).
ADR-039 is the decision; this doc specifies the contracts and the demux/mux
invariants. The channels layer has no `stream_type` concept (ADR-035) —
the demux routes by `channel_id` only, and the reassembly buffer is one
per channel (not per `(channel_id, stream_type)`).
## The split
| Component | Role | What it knows |
|-----------|------|---------------|
| `ChannelsAdapter` | `ProtocolHandler` on `alknet/channels`; reads 8-byte chunk headers off every bidi stream the transport yields and routes to `ChannelManager`. Substrate-agnostic (ADR-034 §substrate modes, as amended by ADR-035). | The transport stream(s); the `ChannelManager` handle. ALPN-blind. |
| `ChannelManager` | Shared state; holds `channel_id → ChannelState`, `HandlerRegistry`. Constructs `ChannelBidiStreamSource` per channel. What `channel/open` closes over (in `channels-call`). | The channel map; the handler registry for ALPN lookup. ALPN-blind (looks up ALPNs, doesn't parse their protocols). |
The split mirrors the TTY crate's `ChunkReader`/`ChunkWriter` + adapter
pattern, generalized to N channels: the adapter drives N channels, and
channel 0 is special only in that it's pre-allocated (by `channels-call`).
## `ChannelsAdapter::handle` (substrate-agnostic)
```rust
#[async_trait]
impl ProtocolHandler for ChannelsAdapter {
fn alpn(&self) -> &'static [u8] { b"alknet/channels" }
async fn handle(&self, connection: Connection, auth: &AuthContext)
-> Result<(), HandlerError>
{
// 1. Channel 0 is pre-negotiated (ADR-036). The first bidi stream
// the transport yields is channel 0. The consumer (channels-call)
// installs the CallAdapter on it.
let bidi = connection.accept_bi().await?;
self.manager.preinstall_channel_0(bidi, auth).await?;
// 2. Accept remaining bidi streams and read 8-byte headers off each.
// On an in-line transport, accept_bi() yields once and the header
// demuxes N channels from that stream. On QUIC native, accept_bi()
// yields repeatedly — each stream carries one logical channel.
// Same code path, same wire format (ADR-034 §substrate modes,
// as amended by ADR-035).
self.manager.run_demux_loop(connection).await
}
}
```
The `preinstall_channel_0` step (provided by `channels-call`, ADR-044)
constructs the reassembly buffer for `channel_id = 0`, wraps it as a
`Connection` via `Connection::from_source` with a
`ChannelBidiStreamSource` (ADR-038, as amended by ADR-035 — `accept_bi`
yields a `BiStream`), and hands that `Connection` to the `CallAdapter`.
The `ChannelsAdapter` in `channels-core` exposes the hook; `channels-call`
provides the implementation.
`run_demux_loop` continues accepting bidi streams from the transport. For
each stream, it reads 8-byte headers and routes payloads to the matching
`channel_id`'s reassembly buffer. On an in-line transport, there is only
one stream (channel 0 rides inside it via the header); the header demuxes
all channels. On QUIC, each subsequent stream is a new channel; the
header's `channel_id` correlates it. The loop is the same; only the
transport's stream count differs.
## `ChannelManager`
```rust
// In alknet-channels-core:
pub struct ChannelManager {
channels: Mutex<HashMap<u32, ChannelState>>,
handlers: Arc<HandlerRegistry>,
// Note: no call_ops field — the call-protocol coupling lives in
// channels-call (ADR-044). The ChannelManager is ALPN-blind and
// call-protocol-blind.
next_id: AtomicU32, // monotonic; wraps at u32::MAX
buffer_cap: usize, // default 1 MiB (ADR-040)
max_channels: usize, // default 256 (ADR-040) — per-connection
// memory bound, NOT a DoS defense. The
// per-identity DoS defense is the
// ChannelLifecyclePolicy consulted by the
// channel/open handler in channels-call
// (ADR-041). The auth-blindness that forces
// the cap out of this struct is ADR-039's
// "no auth state" rule.
}
struct ChannelState {
alpn: String,
/// One reassembly buffer per channel (not per (channel_id, stream_type) —
/// the channels layer has no stream_type concept per ADR-035). Yields
/// a BiStream to the handler.
reassembly: ReassemblyBuffer,
handler_task: JoinHandle<()>,
}
```
`ChannelManager` is `Clone` (cheap — `Arc` internally) so the
`ChannelsAdapter`, the `channel/open` operation handler, and relay logic can
all hold a handle.
> **Type-name convention:** `ChannelManager`, `ChannelsAdapter`,
> `ChannelBidiStreamSource`, and `ChannelClient` are the public API
> surface (contract). `ReassemblyBuffer`, `Demux`, `MuxHandle`/`MuxRunner`,
> `MpscSendStream`/`MpscRecvStream`, and `ChannelOperations` are
> illustrative internal type names — the channels crate's implementation
> may name them differently. The contracts are the invariants
> (REQ-CH-01..04, 06) and the public API; the internal names are not
> contractual.
### `ChannelManager` is ALPN-blind and auth-blind
The `ChannelManager` deliberately does **not** hold:
- **No `ProtocolHandler` implementations.** It holds a `HandlerRegistry`
reference for ALPN lookup, but it doesn't *be* a handler. Handlers live in
their crates and register on the same registry.
- **No ALPN-specific parsing.** It does not parse `NegotiateRequest` JSON,
SSH frames, or tunnel target strings. It hands `params` JSON to the
handler and gets back a handler task. The channels layer carries the
handler's framing transparently in the payload — it does not interpret
the payload bytes.
- **No auth state.** Auth lives in the `OperationContext` that the call
protocol passes to `channel/open`. The `ChannelManager` doesn't check
scopes or ownership — that's `AccessControl::check` in
`OperationRegistry::invoke`, run before the `channel/open` handler.
- **No transport coupling.** It talks to the transport only through the
`ChannelsAdapter`'s read loop and the per-channel write pumps, both of
which use `AsyncRead + AsyncWrite`.
- **No `stream_type` concept.** Per ADR-035, the channels layer routes by
`channel_id` only. There is one reassembly buffer per channel (yielding
a `BiStream`), not one per `(channel_id, stream_type)`. The handler
owns its sub-stream multiplexing on the `BiStream` it receives.
This is what makes the channels layer WASM-compatible and transport-agnostic
— the `ChannelManager` is pure byte routing with no platform or protocol
dependencies.
## The `channel/open` handler
The `channel/open` (and `channel/close`, `channel/control`,
`channel/resources/subscribe`) operations are registered on the call
protocol's `OperationRegistry` at registration time. The
`ChannelOperations` constructor takes a `ChannelLifecyclePolicy`
(ADR-041) — the default is `PerIdentityChannelPolicy::new(256)` (a
real per-identity cap, not NoOp):
```rust
let policy = Arc::new(PerIdentityChannelPolicy::new(256));
let channel_ops = ChannelOperations::new(manager.clone(), policy);
channel_ops.register_on(&mut call_registry)?;
```
The same `Arc<PerIdentityChannelPolicy>` is shared across every
channels connection this peer accepts — that is what makes the cap
per-identity, not per-connection. A hub constructs one policy and
shares it across all worker and browser legs; a worker accepting
direct channels constructs one policy and shares it across whatever
connections it accepts. See ADR-041 for the policy trait and the
default/opt-out variants.
The `channel/open` handler (ADR-037):
1. ACL is already checked by `OperationRegistry::invoke` before this handler
runs.
2. Looks up the ALPN in `HandlerRegistry` → `channel:unknown_alpn` if
missing.
3. **Per-identity cap check (ADR-041):**
`policy.check_open(&op_ctx.identity)?` — deny with
`channel:too_many_channels` if the identity is over its cap. The
identity is the direct caller (the peer on this channels
connection); `forwarded_for` is metadata and is NOT consulted
(ADR-026). For the hub-relay path, the spoke sees the hub as the
direct caller — the hub's quota on the spoke reflects the aggregate
of all relayed channels (ADR-041 §5).
4. Allocates the `channel_id` via `next_id.fetch_add(1, Relaxed)` (DP-1:
server-assigned). The per-connection `max_channels` (ADR-040) is
checked here too — the per-connection memory bound; if hit, the same
`channel:too_many_channels` error is returned (which cap fired first
is an implementation detail — ADR-041 §4).
5. Constructs the `ChannelBidiStreamSource` (ADR-038, as amended by
ADR-035) — one reassembly buffer, yielding a `BiStream`.
6. Spawns the handler task — `tokio::spawn(handler.handle(conn, &auth))`.
Identical to what `TtyAdapter::handle` does today, but on a
channels-backed `Connection`.
7. Records the `ChannelState`.
8. Returns the `channel_id`.
The `channel/close` handler (ADR-037) gains a symmetric
`policy.on_close(&op_ctx.identity)` call after the drain completes
(the same point ADR-040 marks the `channel_id` as eligible for reuse)
— decrementing the per-identity count.
## Demux invariants (REQ-CH-02, 04)
### REQ-CH-02: transport close → all channel senders drop → all handlers see EOF
On transport EOF, `run_demux_loop` clears the `channels` map, dropping all
`ReassemblyBuffer` senders. Every handler's reassembled `BiStream` sees
EOF even without an explicit zero-length sentinel on the wire. Without this,
`read_to_end` / `tokio::io::copy` in handlers hangs forever waiting for a
sender that never drops. This is a teardown invariant of the
`ChannelsAdapter::handle` contract.
### REQ-CH-04: lenient unknown-`channel_id` handling
A chunk with an unallocated `channel_id` is dropped with a debug log and
an error counter (exposed via `Demux::stats()`), and the demux continues.
This matches SSH's behavior and survives transient mis-ordering during
teardown. Validated by the POC (`demux_unknown_channel_drops_lenient`).
## Mux invariants (REQ-CH-03)
### REQ-CH-03: dynamic registration (handle/runner split)
The mux frames per-channel bytes back onto the transport. The POC surfaced
that `Mux::run(self, transport)` (consume, run pre-registered pumps) does
not compose with the dynamic `channel/open` model — channels are opened
after the run loop starts.
The mux is split into:
- **`MuxHandle`** — clone-able, `register(channel_id) -> Sender<Bytes>`
callable at any time after the runner starts.
- **`MuxRunner`** — owns the transport, `select!`s on new-pump registrations
and per-channel write pumps.
The runner's `select!` loop exits when all `MuxHandle` clones drop (the
`new_pumps` sender closes) — the natural shutdown signal. This matches the
dynamic `channel/open` model.
## The two-pump pattern (ADR-078 — documented here for handler authors)
Handlers with a two-pump shape (two `tokio::io::copy` pumps, one per
direction — tunnel, SSH `direct-tcpip`) MUST shut down the opposite sink
when one pump completes. `tokio::try_join!` alone deadlocks: each pump
waits for the other's EOF, which only comes after the opposite pump shuts
down its sink.
```rust
let c2t = async {
tokio::io::copy(&mut recv, &mut tcp_write).await?;
tcp_write.shutdown().await.ok(); // shut down the peer's sink
Ok::<_, std::io::Error>(())
};
let t2c = async {
tokio::io::copy(&mut tcp_read, &mut send).await?;
send.shutdown().await.ok(); // shut down the peer's sink (emits sentinel — REQ-CH-01)
Ok::<_, std::io::Error>(())
};
tokio::try_join!(c2t, t2c)?;
```
The three-pump pattern (TTY's `pump_session`, coordinating via the
`exit_code` future) does not have this deadlock — the `exit_code` future is
the third signal. The two-pump pattern is documented in ADR-078; the
shutdown-on-completion contract is a handler-level concern, not a
channels-layer one.
## The hub relay interface
The hub relay (ADR-042) uses the `ChannelManager`'s interface to bridge two
channels connections:
```rust
// For channel_id=7 on browser side, channel_id=12 on spoke side:
tokio::spawn(async move {
let mut b_bidi = browser_mgr.open_channel_stream(7).await;
let mut s_bidi = spoke_mgr.open_channel_stream(12).await;
tokio::join!(
pump(&mut b_bidi, &mut s_bidi), // browser → spoke (with channel_id rewrite)
pump(&mut s_bidi, &mut b_bidi), // spoke → browser (with channel_id rewrite)
);
});
```
The relay reads opaque bytes off one `ChannelManager`'s reassembled
`BiStream` and writes them onto the other's write-half, which re-chunks
them with the other leg's `channel_id` (a 4-byte rewrite within the
8-byte header). The relay does not parse the payload — it doesn't know if
the bytes are TTY chunks, SSH frames, or tunnel data. The hub translates
`channel/open` on channel 0 (re-issues on the spoke leg with
`forwarded_for`); data channels are byte-forwarded with `channel_id`
rewrite. See ADR-042 for the full relay contract.
## Design Decisions
All design decisions are documented as ADRs in [decisions/](decisions/).
| ADR | Decision | Summary |
|-----|----------|---------|
| [075](decisions/075-channelsadapter-and-channelmanager.md) | ChannelsAdapter and ChannelManager | The split; the contracts |
| [093](decisions/093-channels-pure-channel-multiplexing.md) | channels Pure Channel Multiplexing | The umbrella decision: 8-byte header, no `stream_type`, one reassembly buffer per channel |
| [076](decisions/076-backpressure-channel-limits-id-reuse.md) | Backpressure, Limits, ID Reuse | Bounded-buffer, 256-channel per-connection memory bound, monotonic IDs (DoS defense reframed by ADR-041) |
| [094](decisions/094-per-identity-channel-cap.md) | Per-Identity Channel Cap | 256 per `PeerId`, enforced via `ChannelLifecyclePolicy` in `channels-call`; per-connection `max_channels` reframed as a memory bound |
| [078](decisions/078-two-pump-shutdown-on-completion.md) | Two-Pump Pattern | Shutdown-on-completion contract |
| [079](decisions/079-hub-relay-translate-not-forward.md) | Hub Relay | Translate channel 0, byte-forward data channels |
## References
- ADR-039: ChannelsAdapter and ChannelManager (the decision)
- ADR-035: channels pure channel multiplexing (the umbrella decision that
amends ADR-034/074/077)
- ADR-036: channel 0 pre-negotiated (the `preinstall_channel_0` step)
- ADR-037: channel lifecycle operations (the ops registered on `call_ops`)
- ADR-038: ChannelBidiStreamSource (what the manager constructs per
channel, as amended by ADR-035)
- ADR-040: backpressure and limits (`buffer_cap`, `max_channels` — the
per-connection memory bound)
- ADR-041: per-identity channel cap (the `ChannelLifecyclePolicy`
consulted by the `channel/open` handler; the relay consequence for
hub-relayed channels)
- `docs/research/alknet-channels/poc-summary.md` §Issues Surfaced #4-#7
(REQ-CH-01..04, the two-pump deadlock)
- `docs/research/stream-unification/findings.md` — the research that
surfaced the pure-multiplexing resolution

179
docs/architecture/channels-connection.md Normal file
View File

@@ -0,0 +1,179 @@
---
status: draft
last_updated: 2026-07-18
---
# channels-connection.md — ChannelBidiStreamSource and `BiStream` Access
How a reassembled channel is presented to its handler as a `Connection`.
ADR-038 (amended by ADR-035) is the decision; this doc specifies the API
shape — one accessor, one `BiStream` per channel.
## What
Each channel is reassembled into a `BiStream` — a single duplex
(`AsyncRead + AsyncWrite`) byte stream. The channels layer strips its
8-byte header (`channel_id` + `length`) on read, hands the payload to the
reassembled `BiStream`, and the handler parses its own framing from the
payload. The handler sub-multiplexes its `BiStream` however it wants —
TTY sub-demuxes `stream_type` from its `BiStream` via its 5-byte format,
tunnel uses the `BiStream` as raw bytes, call length-prefixes JSON, SSH
runs its own channel protocol.
The `BiStream` is wrapped in a `ChannelBidiStreamSource` that implements
`alknet-core`'s `BidiStreamSource` trait (ADR-008), and a `Connection` is
constructed from it via `Connection::from_source(source, alpn)`. The
handler receives a `Connection`, calls `accept_bi()` once (yield-once per
channel), gets a `BiStream`, and drives its session — identical to how it
works on a top-level QUIC connection.
## `ChannelBidiStreamSource`
```rust
// In alknet-channels:
pub struct ChannelBidiStreamSource {
// The reassembly buffer for this channel's payload bytes (one per
// channel_id, not per (channel_id, stream_type) — the channels layer
// has no stream_type concept), plus the mux handle for writing back
// onto the transport. Constructed by ChannelManager::build_channel_connection
// (ADR-039).
...
}
#[async_trait]
impl BidiStreamSource for ChannelBidiStreamSource {
async fn accept_bi(&self)
-> Result<BiStream, StreamError>
{
// Yields the channel's BiStream on first call,
// ConnectionClosed on subsequent calls. Yield-once per channel,
// matching the POC's validated shape.
}
async fn open_bi(&self)
-> Result<BiStream, StreamError>
{
// StreamClosed — a single channel cannot open new application
// streams (same as ADR-007's Stream backend). The handler owns
// its sub-stream multiplexing on the BiStream it received.
}
fn remote_addr(&self) -> Option<SocketAddr> { ... }
fn close(&self, _code: u32, _reason: &str) { ... }
}
```
One `ChannelBidiStreamSource` instance represents **one channel** (not the
whole channels connection). The `ChannelManager` (ADR-039) constructs one
per channel at `channel/open` time and wraps it in a `Connection` via
`from_source`.
## The single path: `accept_bi()`
Every handler — TTY, tunnel, SSH, call — receives a `Connection`, calls
`accept_bi()` once, gets a `BiStream`, and sub-multiplexes it however it
wants. There is one accessor.
```rust
// Tunnel handler — ~15 lines, zero channels-layer awareness
async fn handle(&self, connection: Connection, _auth: &AuthContext)
-> Result<(), HandlerError>
{
let mut bidi = connection.accept_bi().await?;
let mut tcp = TcpStream::connect(target).await?;
let (mut tcp_read, mut tcp_write) = tcp.into_split();
let (mut recv, mut send) = tokio::io::split(&mut bidi);
// Two-pump with shutdown-on-completion (ADR-078)
let c2t = async {
tokio::io::copy(&mut recv, &mut tcp_write).await?;
tcp_write.shutdown().await.ok();
Ok::<_, std::io::Error>(())
};
let t2c = async {
tokio::io::copy(&mut tcp_read, &mut send).await?;
send.shutdown().await.ok(); // emits zero-length sentinel (REQ-CH-01)
Ok::<_, std::io::Error>(())
};
tokio::try_join!(c2t, t2c)?;
Ok(())
}
```
```rust
// TTY handler (inside-channels mode) — the SAME code as direct
// mode, just a different BiStream source.
async fn handle(&self, connection: Connection, _auth: &AuthContext)
-> Result<(), HandlerError>
{
let mut bidi = connection.accept_bi().await?;
// drive_session reads the 5-byte TTY chunks off `bidi` — the same
// code as direct mode. The channels layer stripped its 8-byte
// header; TTY's 5-byte format is the payload.
drive_session(bidi, backends, ownership, identity).await
}
```
The handler calls `accept_bi()` once, gets a `BiStream`, and pumps. It
does not know it's inside a channels connection — the `Connection` looks
like any other. This is the path the POC's `EchoHandler` and
`TunnelHandler` validated.
`accept_bi()` is yield-once: the first call returns the `BiStream`;
subsequent calls return `ConnectionClosed`. This matches the POC's
validated shape and the `StreamBidiStreamSource` yield-once contract
(ADR-008, ADR-009).
## Recursive composition
A `ChannelBidiStreamSource` is a `BidiStreamSource`, and
`Connection::from_source` wraps it. A handler that is itself
`alknet/channels` can open a sub-channels connection on a data channel —
`alknet/channels` inside `alknet/channels`. The outer layer strips its
8-byte header; the inner layer parses its own 8-byte header from the
payload. Each level is the same shape: `BiStream → accept_bi → N
BiStreams`. The recursion is unbounded and uniform at every level.
This is a property, not a feature. The primary use case is one level of
multiplexing. But the add/strip composition makes it cleaner than
ADR-034's group framing did — the recursion is the same operation
(strip an 8-byte header) at every level, not a different framing per
level.
## What does NOT change
- **`ProtocolHandler` trait** (ADR-002) — handlers still receive a
`Connection` and call `accept_bi()`. The `ChannelBidiStreamSource` is
internal to the channels crate; handlers see a `Connection`.
- **`BiStream`** (ADR-009) — the leaf type `accept_bi` returns. The
channels layer yields `BiStream`s; handlers parse them per their ALPN.
- **`HandlerRegistry`** — unchanged. The channels layer looks up ALPNs in
the same registry as top-level connections.
## Design Decisions
All design decisions are documented as ADRs in [decisions/](decisions/).
| ADR | Decision | Summary |
|-----|----------|---------|
| [074](decisions/074-channelconnection-bidistreamsource.md) | ChannelConnection | Per-channel `BidiStreamSource`; yield-once `accept_bi` is the only accessor |
| [093](decisions/093-channels-pure-channel-multiplexing.md) | channels Pure Channel Multiplexing | The umbrella decision: 8-byte header, no `stream_type`, `BiStream`-only |
| [070](decisions/070-bidistreamsource-trait.md) | BidiStreamSource Trait | The extension point `ChannelBidiStreamSource` implements |
| [092](decisions/092-bistream-as-the-handler-leaf.md) | `BiStream` as the Handler Leaf | `accept_bi` returns `BiStream` (the transport-leaf decision this doc builds on) |
| [065](decisions/065-connection-from-stream-generic-single-stream.md) | `Connection::from_stream` | The yield-once path generalized for channels |
## References
- ADR-038: ChannelConnection (the decision)
- ADR-035: channels pure channel multiplexing (the umbrella decision)
- ADR-008: BidiStreamSource trait
- ADR-009: `BiStream` as the handler leaf
- ADR-007: `Connection::from_stream` (the yield-once path generalized)
- ADR-077: TTY inside channels (TTY always uses its 5-byte format,
carried transparently in the channels payload)
- `docs/research/alknet-channels/poc-summary.md` §POC Target 2 (the
yield-once `Connection::from_stream` validation)
- `docs/research/stream-unification/findings.md` — the research that
surfaced the single-accessor resolution

307
docs/architecture/channels-overview.md Normal file
View File

@@ -0,0 +1,307 @@
---
status: draft
last_updated: 2026-07-18
---
# alknet-channels — Overview
## What
`alknet-channels` is a multiplexing proxy crate. It implements
`ProtocolHandler` for the `alknet/channels` ALPN: it receives one
bidirectional transport stream, reads 8-byte chunk headers, and routes each
chunk's payload to the right logical channel. Each channel is reassembled
into a `BiStream` (a concrete `AsyncRead + AsyncWrite` newtype, per
ADR-009) and presented to its handler as a `Connection` — the handler
doesn't know it's inside a channels connection.
Channel 0 is pre-negotiated as `alknet/call` (ADR-036). Every other channel
is opened dynamically via `channel/open` on channel 0 (ADR-037) and routed
through the same `HandlerRegistry` as top-level connections. The channels
layer does no protocol work itself — it is a re-framing proxy that converts
between "one transport stream carrying N channels" (the wire) and "N
independent `BiStream` handles" (what handlers see). The channels layer has
no `stream_type` concept (ADR-035) — the handler owns its sub-stream
multiplexing on the `BiStream` it receives.
## Why
### The problem: three multiplexing models that don't compose
Before channels, alknet had three multiplexing models:
| Model | Where | Mechanism |
|-------|-------|-----------|
| Connection-level | ALPN router | One ALPN per QUIC connection |
| Stream-level | QUIC native | Many bidi streams per connection |
| Sub-stream-level | TTY chunk format | 4 logical channels within one bidi stream |
A docker client needing both JSON call operations and raw TTY sessions
required **two separate QUIC connections** with different ALPNs. The call
protocol can't say "for this operation, open a TTY stream." The hub,
bridging browsers and spokes over multiple transports, faced an
O(protocols × transports × spokes) matrix of per-protocol framing parsers
and per-ALPN connection management.
### The collapse: one multiplexing model, one connection per leg
With `alknet/channels`, one connection carries everything:
```
Browser ──WebTransport──► Hub ──QUIC──► Spoke
alknet/channels alknet/channels
┌─────────────┐ ┌─────────────┐
│ ch0: call │ │ ch0: call │
│ ch1: tty │ relay │ ch1: tty │
│ ch2: ssh │ ◄─────► │ ch2: ssh │
│ ch3: tunnel │ │ ch3: tunnel │
└─────────────┘ └─────────────┘
```
The hub's relay is channel-by-channel byte forwarding (with `channel_id`
rewrite — ADR-042), not per-protocol framing parsers. The hub's complexity
collapses from O(protocols × transports × spokes) to O(channels).
The collapse is at three levels:
1. **One connection per leg, not one per protocol.** All needs (call, TTY,
SSH, tunnel) ride as channels on one connection per leg.
2. **One multiplexing model, not three.** Connection-level, stream-level,
and sub-stream-level all become channels chunks.
3. **The call protocol orchestrates from inside.** Channel 0 is
`alknet/call` on both legs. The call protocol's `OperationRegistry`,
`AccessControl`, and `forwarded_for` machinery govern channel lifecycle
with no new auth.
### The separation: channels layer is pure channel multiplexing
The channels layer's job is "one connection carries N channels, routed by
`channel_id`." It does not know about TTY's sub-streams, SSH's channel
protocol, or how call frames its JSON. Handlers own their sub-multiplexing
on the `BiStream` the channels layer gives them (ADR-035).
- **Every channel is a `BiStream`.** `accept_bi()` yields one `BiStream`
per channel (per ADR-009). The handler sub-multiplexes it however it
wants — TTY's 5-byte format, call's length-prefixed JSON, tunnel's raw
bytes, SSH's channel protocol.
- **The channels layer has no `stream_type` concept.** Not in its 8-byte
header, not in its code, not in its mental model. `stream_type` is the
inner layer's framing byte, carried transparently in the payload.
- **The control channel is handler-internal.** TTY sub-demuxes control
from its io `BiStream` using its 5-byte format (`STREAM_CTRL_IN` /
`STREAM_CTRL_OUT` — ADR-052 amended by Phase 7). The channels layer
doesn't carry control.
- **Recursive composition is literal.** A channel with ALPN
`alknet/channels` runs another channels demux on its `BiStream`. The
outer layer strips its 8-byte header; the inner layer parses its own
8-byte header from the payload.
## Architecture
The crate has two internal components (ADR-039):
- **`ChannelsAdapter`** — implements `ProtocolHandler` for
`alknet/channels`. Its `handle()` receives one `Connection`, reads 8-byte
chunk headers, and routes chunks to the `ChannelManager`. The read/demux
half.
- **`ChannelManager`** — the shared state. Holds `channel_id →
ChannelState`, the `HandlerRegistry` reference, and the
`OperationRegistry` reference. The reassemble/allocate half. What the
`channel/open` operation handler closes over.
Each channel is presented to its handler as a `Connection` constructed via
`Connection::from_source(ChannelBidiStreamSource, alpn)` (ADR-008/074, as
amended by ADR-035). The handler calls `accept_bi()` once (yield-once per
channel) and gets a `BiStream` — identical to how it works on a top-level
QUIC connection.
See [channels-adapter.md](channels-adapter.md) for the full adapter/manager
design.
## Crate dependencies
```
alknet-channels-core
├── alknet-core (ProtocolHandler, Connection, HandlerRegistry,
│ BidiStreamSource, BiStream, AuthContext)
├── tokio (spawn, mpsc, io)
├── bytes (Bytes for chunk payloads)
├── async-trait
├── thiserror
└── tracing
alknet-channels-call
├── alknet-channels-core (ChannelManager, ChannelsAdapter,
│ ChannelBidiStreamSource, ChannelClient)
├── alknet-call (OperationRegistry, HandlerKind, make_handler,
│ make_streaming_handler, CallError, ResponseEnvelope)
└── tokio
alknet-hub (the existing hub crate — consumes channels)
├── alknet-channels-call
└── alknet-call (from_call, CallAdapter, forwarded_for — ADR-042)
worker crates (any crate that dials a hub — consumes channels)
└── alknet-channels-call (ChannelClient — ADR-043)
```
`alknet-channels-core` is the pure multiplexer — wire format, demux/mux,
`ChannelBidiStreamSource`, `ChannelManager`. It depends on `alknet-core`
only. No `alknet-call` dependency. ALPN-blind, call-protocol-blind,
transport-blind. This is where the "streams are streams" insight lives.
`alknet-channels-call` is the call-protocol coupling — channel 0
pre-negotiation as `alknet/call` (ADR-036), the four lifecycle operations
(ADR-037) registered on the call protocol's `OperationRegistry`, and
`ChannelClient` (ADR-043). This is where the call-protocol coupling lives,
isolated from the pure multiplexer.
The hub and worker are **consumers**, not sub-crates. The existing
`alknet-hub` crate IS the channels hub — it depends on `channels-call` and
uses channels as its substrate, with the relay logic (ADR-042) living in
`alknet-hub` alongside its existing peer lifecycle and service discovery
responsibilities. A worker is any crate that uses `ChannelClient` to dial.
There are no `channels-hub` or `channels-worker` sub-crates.
See ADR-044 for the full decomposition rationale.
## ALPN
`alknet/channels` — the ALPN the `ChannelsAdapter` registers on. One ALPN
per channels connection; the connection carries N logical channels, each
with its own ALPN (negotiated via `channel/open`).
## Transport agnosticism
The channels wire format works over any ordered, reliable bidirectional byte
stream:
| Transport | How |
|-----------|-----|
| QUIC bidi stream | `alknet/channels` ALPN on a QUIC connection; one bidi stream carries all channels |
| TCP+TLS | `alknet/channels` ALPN on a TLS connection; the TCP stream carries all channels |
| WebTransport | `alknet/channels` session (deferred per ADR-044; the browser path uses WebSocket carrying `alknet/channels`) |
| SSH channel | channels connection riding inside an SSH `direct-tcpip` channel (channels-over-SSH) |
| Another channels connection | recursive composition (channel type `alknet/channels` inside `alknet/channels`) |
The same wire format, the same chunk reassembly, the same `Connection`
abstraction. The transport is a parameter, not a design constraint.
`Connection::from_bidi` / `from_source` (ADR-007/070/092) handles the
transport-agnostic `Connection` construction.
## WASM compatibility
The wire format's core is pure byte manipulation — `parse_header` /
`write_header` are pure functions with no platform dependencies. The de-risk
POC validated the sync core compiles under `wasm32-unknown-unknown`. The
async shell (demux/mux) wraps this core with `read_exact`/`write_all` and
`mpsc` routing.
The `ChannelManager` is ALPN-blind, auth-blind, and transport-blind (ADR-
075) — pure byte routing with no platform or protocol dependencies. A WASM
build can read chunks from a WebTransport `BiStream`, reassemble them, and
present `AsyncRead + AsyncWrite` handles to WASM-compatible handlers. The
handlers themselves may or may not be WASM-compatible (russh's client is;
`portable_pty` is not), but the channels layer is WASM-compatible by
construction.
The async shell and `alknet-core` dep graph are not fully WASM-clean yet
(transitive `getrandom`/`rand` deps) — this is an implementation concern,
not an architecture concern. The sync core's WASM compatibility is validated.
## Relationship to existing crates
### alknet-call
Unchanged. The call protocol remains JSON-only, `EventEnvelope`-based. It
runs on channel 0 exactly as on a top-level `alknet/call` connection. The
`CallAdapter` receives a `Connection` backed by channel-0 chunk reassembly
and dispatches operations — it doesn't know it's inside channels. The call
protocol's `EventEnvelope` framing (ADR-014) is the channels payload; the
channels layer carries it transparently.
What changes: the call protocol gains a new class of operations — channel
lifecycle (ADR-037). These are registered on the `OperationRegistry` at
assembly time and dispatched through the existing `OperationContext` /
`AccessControl::check` path.
### alknet-tty
The TTY crate gains a `channels` feature that enables inside-channels
mode. In both direct mode (`alknet/tty` ALPN on a top-level connection) and
inside-channels mode (`channel/open` with ALPN `alknet/tty`), the TTY
adapter uses its own 5-byte wire format (ADR-052). The two modes differ
only in *where the `BiStream` comes from* — a top-level connection vs a
channels-backed `Connection`. The same `wire.rs` code runs in both modes
(ADR-077): the channels layer strips its 8-byte header and hands TTY the
payload bytes; TTY parses its 5-byte header from the payload. The
`TtyBackend` trait and `TtyHandle` are unchanged; backends don't know
which mode the adapter is in.
### alknet-ssh (future)
SSH as a channel type: an `alknet/ssh` channel carries the SSH binary
protocol on its `BiStream`. The channels layer hands the reassembled
`BiStream` to `SshAdapter`, which feeds it to russh. SSH as a channels
transport: an SSH `direct-tcpip` channel could carry a channels connection
(channels-over-SSH). The SSH crate doesn't need to know about channels —
it implements `ProtocolHandler` for `alknet/ssh` and accepts a
`Connection`. SSH multiplexes internally (its own channel protocol rides
the channels payload transparently).
### alknet-docker
Docker lifecycle operations are call operations on channel 0 (unchanged
from ADR-058). Interactive exec/attach opens a TTY channel via
`channel/open` with ALPN `alknet/tty` and backend `docker`. No separate
`alknet/tty` connection needed — one `alknet/channels` connection handles
both JSON operations and raw TTY sessions.
### alknet-hub
The hub is the primary consumer. With channels, the hub holds one channels
connection per leg (browser↔hub, hub↔spoke) and relays channels between
them. The hub translates `channel/open` on channel 0 (re-issues on the
spoke leg with `forwarded_for` — ADR-042) and byte-forwards data channels
with `channel_id` rewrite. The hub's complexity collapses from
O(protocols × transports × spokes) to O(channels).
## Design Decisions
All design decisions are documented as ADRs in [decisions/](decisions/).
| ADR | Decision | Summary |
|-----|----------|---------|
| [071](decisions/071-channels-wire-format.md) | channels Wire Format | 8-byte chunk header (amended by ADR-035); channels layer has no `stream_type` concept; one-way door |
| [093](decisions/093-channels-pure-channel-multiplexing.md) | channels Pure Channel Multiplexing | The umbrella decision: 8-byte header, no `stream_type`, `into_sub_streams` removed, `BiStream`-only, TTY always 5-byte |
| [072](decisions/072-channel-0-pre-negotiated-call.md) | Channel 0 Pre-Negotiated | Channel 0 = `alknet/call` |
| [073](decisions/073-channel-lifecycle-operations.md) | Channel Lifecycle Operations | `channel/open`/`close`/`control`/`resources/subscribe`; subscribe not poll; `direction` pinned |
| [074](decisions/074-channelconnection-bidistreamsource.md) | ChannelConnection | Per-channel `BidiStreamSource`; yield-once `accept_bi` (amended by ADR-035 — `into_sub_streams` removed) |
| [075](decisions/075-channelsadapter-and-channelmanager.md) | ChannelsAdapter and ChannelManager | Substrate-agnostic demux loop; REQ-CH-01..04 |
| [076](decisions/076-backpressure-channel-limits-id-reuse.md) | Backpressure, Limits, ID Reuse | Bounded-buffer (1 MiB), 256-channel cap, monotonic IDs |
| [077](decisions/077-tty-inside-channels.md) | TTY Inside Channels | TTY's two modes (direct vs channels); TTY always uses its 5-byte format, carried transparently in the channels payload |
| [078](decisions/078-two-pump-shutdown-on-completion.md) | Two-Pump Pattern | Shutdown-on-completion contract; handler-level |
| [079](decisions/079-hub-relay-translate-not-forward.md) | Hub Relay | Translate channel 0, byte-forward data channels with ID rewrite |
| [080](decisions/080-channelclient.md) | ChannelClient | Client side; transport-agnostic `from_connection` primary; dial lives in `AlknetClient` (ADR-045, resolves OQ-55) |
| [081](decisions/081-channels-subcrate-decomposition.md) | Sub-Crate Decomposition | `channels-core` (pure multiplexer) / `channels-call` (call coupling + ChannelClient); hub and worker are consumers |
## Open Questions
Open questions are tracked in [open-questions.md](open-questions.md).
Key questions affecting this crate:
- **OQ-55** (resolved by ADR-045): `AlknetClient` core **dial+TLS seam**
— extracted as `alknet-client` with three dial methods.
`ChannelClient`'s API is transport-agnostic (`from_connection`); the
dial is the shared seam, now extracted. See
[ADR-045](decisions/089-alknetclient-native-dial-seam.md).
- **OQ-56** (deferred(scope)): Full channel-level flow-control windowing —
bounded-buffer is decided (ADR-040); full windowing is an extension
blocked on a real HOL-blocking deployment observation.
- **OQ-57** (deferred(scope)): Two-pump helper extraction to alknet-core —
the *contract* is decided (ADR-078); the *helper* is blocked on a second
two-pump handler existing.
- **OQ-68** (open): Add/strip API shape — whether the 8-byte header
add/strip is built into the channels read/write path or exposed as a
standalone utility. The *contract* (channels strips, handler parses
payload) is decided (ADR-035); the *function surface* is not.

302
docs/architecture/channels-wire.md Normal file
View File

@@ -0,0 +1,302 @@
---
status: draft
last_updated: 2026-07-18
---
# channels-wire.md — The 8-Byte Chunk Format
The wire format for `alknet/channels`: an 8-byte chunk header that
multiplexes N logical channels over a single ordered, reliable
bidirectional transport stream. ADR-034 (amended by ADR-035) is the
decision; this doc specifies the format and the wire-level invariants.
The channels layer has no `stream_type` concept — not in its header, not
in its code, not in its mental model. The handler owns its sub-stream
multiplexing on the `BiStream` the channels layer gives it.
## Chunk header
```
[channel_id: u32 BE][length: u32 BE][payload bytes]
```
8 bytes of header, followed by `length` bytes of opaque payload.
| field | offset | width | meaning |
|-------|--------|-------|---------|
| `channel_id` | 0 | 4 (BE) | The logical channel this chunk belongs to. Channel 0 is pre-negotiated as `alknet/call` (ADR-036). Channels 1..N are opened dynamically via `channel/open` (ADR-037). |
| `length` | 4 | 4 (BE) | The payload length in bytes. 0 = EOF sentinel. Max `MAX_CHUNK_LEN`. |
The payload is opaque to the channels layer. The handler parses its own
framing from the payload — TTY's `[stream_type:u8][length:u32][payload]`
(5-byte format, ADR-052), call's length-prefixed JSON (`EventEnvelope`
framing, ADR-014), tunnel's raw bytes, SSH's channel protocol. The
channels layer carries the bytes transparently.
### How the wire formats compose
The channels 8-byte header and the handler's framing compose by layering:
```
channels: [channel_id:u32 BE][length:u32 BE][payload]
= 8-byte header + opaque payload
8 bytes
TTY inside channels:
[channel_id:u32][ch_len:u32][stream_type:u8][tty_len:u32][payload]
4 bytes 4 bytes 1 byte 4 bytes N bytes
\_________ __________/ \_________ _____________/
| |
channels header TTY chunk (5+N bytes)
(8 bytes) carried as channels payload
```
The channels layer reads its 8-byte header (`channel_id` + `length`),
reads `length` bytes of payload, and hands the payload to the handler.
The handler parses its own framing from the payload — TTY reads its
5-byte header (`stream_type` + `length`) from the payload bytes.
The two length fields are close but not identical: `ch_len = tty_len + 5`.
This is a small amount of waste per chunk (the channels `length` is always
5 bytes more than TTY's `length`), but the trade-off is clean separation
of concerns: the channels layer has no `stream_type` concept — not in
its header, not in its code, not in its mental model. The handler owns
its framing entirely. See ADR-035 for the full cost/benefit analysis.
## `MAX_CHUNK_LEN`
`16 * 1024 * 1024` (16 MiB), matching TTY's cap (ADR-052 §5). A chunk with
`length > MAX_CHUNK_LEN` returns `ChunkTooLarge` and does not corrupt the
stream — the demux drops the chunk and continues. The header is always
exactly 8 bytes, so the demux can always resync by reading the next
8-byte header.
## Channel 0 — pre-negotiated `alknet/call`
Channel 0 is not a special "control plane" with its own framing. It is
`alknet/call` pre-negotiated (ADR-036): both sides know `channel_id = 0`
is routed to the `CallAdapter` without an explicit `channel/open`
exchange.
Channel 0's chunks have `channel_id = 0` in the 8-byte header — same
format as every other channel. The call protocol's `EventEnvelope` JSON
framing (ADR-014) is the payload; the channels layer carries it
transparently. Disambiguation between channel 0 and data channels is by
`channel_id`, not by a special first-byte trick.
## Framing disambiguation
The 8-byte header is always exactly 8 bytes. `length` is bounded by
`MAX_CHUNK_LEN`. The demux reads 8 bytes, parses the header, reads
`length` bytes of payload, and routes. If a chunk is dropped (e.g.,
`ChunkTooLarge`), the demux resyncs by reading the next 8-byte header —
the format is self-synchronizing.
There is no channels-layer framing-disambiguation trick beyond the fixed
8-byte header. The channels layer does not interpret the payload — it
doesn't know if the payload is TTY chunks, call frames, or tunnel bytes.
Any framing disambiguation within the payload is the handler's concern
(see `tty-wire.md` §"Framing disambiguation" for TTY's first-byte trick,
which is internal to TTY's 5-byte format).
## Zero-length sentinel = EOF
A zero-length chunk (`length = 0`) is delivered as an empty payload,
which the reassembled stream interprets as EOF. This is the clean-shutdown
signal for a `channel_id` — the same convention as TTY (ADR-052
§Sentinels), now at the channels layer (one sentinel per channel, not
per `(channel_id, stream_type)`).
The sentinel is emitted by the write side's `AsyncWrite::shutdown` (see
REQ-CH-01 below) and consumed by the read side's `AsyncRead::poll_read` as
EOF.
## Substrate modes — same wire format, different stream counts
The 8-byte header is used in all substrates, on every bidi stream. The
difference between substrates is only **how many bidi streams the
transport yields**:
| Substrate | Transport | Streams | Header role |
|-----------|-----------|---------|--------------|
| In-line | TCP+TLS, WebTransport session, SSH `direct-tcpip` | 1 | Header demuxes N channels from that 1 stream |
| Native | QUIC (quinn/iroh) | N | Each stream carries 1 logical channel; header provides `channel_id` correlation |
| Multi-connection | Any, N connections | N × M | Each connection is self-contained (own channel 0, own demux); header is per-connection |
The `ChannelsAdapter::handle` loop: `accept_bi()` → for each stream, read
the 8-byte header → route by `channel_id` → reassemble into a `BiStream`.
On an in-line transport, `accept_bi()` yields once then
`ConnectionClosed` — the header does all the demux. On QUIC, `accept_bi()`
yields repeatedly — each stream is a channel, and the header provides
`channel_id` correlation. Same code path, same wire format, same handler
experience. See ADR-034 §substrate modes (as amended by ADR-035), ADR-039.
## Wire-level invariants (REQ-CH-01, 02, 04, 05)
The de-risk POC (`docs/research/alknet-channels/poc-summary.md` §Issues
Surfaced) surfaced invariants that hang channels silently if underspecified.
These are **contracts**, not implementation details — both sides must agree.
### REQ-CH-01: `AsyncWrite::shutdown` emits a zero-length sentinel
The reassembled stream's write half (`MpscSendStream` or equivalent) MUST
send an empty payload (the EOF sentinel) before dropping the sender on
`AsyncWrite::shutdown`. Without this, the demux never sees EOF on the
channel, and `tokio::io::copy` in the handler never
completes — the session hangs.
The TTY crate's `pump_session` emits the zero-length stdout sentinel
explicitly via its own 5-byte format's zero-length chunk; the channels
layer's per-channel write pump does NOT forward a sentinel on
sender-drop, so the send adapter must. Both sides must agree on this
convention, or channels hang on clean shutdown.
### REQ-CH-02: transport close → all channel senders drop → all handlers see EOF
The demux loop MUST clear its `channels` map on transport EOF, dropping
all `ReassemblyBuffer` senders. Every handler's reassembled `BiStream`
sees EOF even without an explicit zero-length sentinel arriving on the
wire.
Without this, `read_to_end` / `tokio::io::copy` in handlers hangs forever
waiting for a sender that never drops because the demux task is holding the
map. This is a teardown invariant of the `ChannelsAdapter::handle` contract.
### REQ-CH-04: lenient unknown-`channel_id` handling with error counter
A chunk with an unallocated `channel_id` is dropped with a debug log and
an error counter (exposed via `Demux::stats()`), and the demux continues.
This matches SSH's behavior and survives transient mis-ordering during
teardown (a chunk for a channel that was just closed may arrive after
the close is processed).
The alternative (strict — close the transport on unknown `channel_id`) is
fragile during teardown and catches bugs at the cost of reliability. The
lenient approach with an error counter provides observability without
fragility.
### REQ-CH-05: bounded-buffer backpressure does not deadlock
Each `channel_id` has an independent bounded `mpsc` buffer (default 1 MiB
— ADR-040). A slow reader on one channel does not block another channel's
reads — the demux's per-chunk route awaits the matching sender without
holding a global lock.
The 1 MiB `tunnel_large_payload` POC test exercised this end-to-end: a
channel writer faster than the TCP echo server consumer, with no deadlock
and no cross-channel blocking. This invariant must hold for all transport
shapes — the bounded-buffer approach is the decision (ADR-040).
## Sync core / async shell split
The wire format's core is pure byte manipulation:
```rust
// wire.rs — sync core, no async, no platform deps, WASM-clean
const CHUNK_HEADER_LEN: usize = 8;
const MAX_CHUNK_LEN: u32 = 16 * 1024 * 1024;
pub struct ChunkHeader {
pub channel_id: u32,
pub length: u32,
}
pub fn parse_header(buf: &[u8; 8]) -> Result<ChunkHeader, ChunkError> { ... }
pub fn write_header(channel_id: u32, length: u32, out: &mut [u8; 8]) { ... }
```
The async shell (demux/mux — see [channels-adapter.md](channels-adapter.md))
wraps this core with `read_exact` / `write_all` on the transport and `mpsc`
routing. The split keeps the WASM-compatible core separate from the
tokio-dependent shell. The POC validated the sync core compiles under
`wasm32-unknown-unknown`.
## The add/strip composition
Each layer has its own add/strip pair. The channels layer:
`add_channel_id(channel_id, payload_bytes) -> chunk` on write (prepends
the 8-byte header); `strip_channel_id(chunk) -> (channel_id,
payload_bytes)` on read (strips the 8-byte header, returns the payload).
The handler layer (e.g. TTY) parses its own framing from the payload
bytes per its existing `wire.rs`. The handler doesn't know or care that
a `channel_id` was stripped before it saw the bytes.
The composition is uniform — the same shape at every level. This is SSH's
model (layered headers, each layer strips its own at its boundary),
applied to channels. A `alknet/channels`-inside-`alknet/channels`
recursive composition is the outer layer stripping its 8-byte header, the
inner layer parsing its own 8-byte header from the payload — same code,
same shape, each level.
The exact API shape of the add/strip pair (built into the read/write path
vs. a standalone utility) is an implementation detail for the channels
crate, tracked as OQ-68. The *contract* — the channels layer strips its
8-byte header on read and the handler parses its own framing from the
payload — is decided; the *function surface* is not.
## Channel lifecycle (summary)
| Phase | Mechanism | Reference |
|-------|-----------|-----------|
| Open | `channel/open` call operation on channel 0; responder allocates `channel_id`, returns it | ADR-037 |
| Data | chunks with `channel_id` routed to reassembly buffers; handler sees a `BiStream` | this doc, [channels-connection.md](channels-connection.md) |
| Control (out-of-band) | `channel/control` call operation on channel 0 | ADR-037 |
| Close | `channel/close` call operation on channel 0; data chunks flushed before close | ADR-037, REQ-CH-06 |
### REQ-CH-06: exit-chunk-before-close ordering (generalizes ADR-055)
The channel's data chunks MUST be written and flushed before the
`channel/close` operation is sent on channel 0. This is a wire-level
invariant: the side closing must observe the data-channel pump complete
before issuing the call operation.
For TTY this is the exit-chunk-is-last invariant (ADR-055) carried
forward: the exit control message (on TTY's `STREAM_CTRL_OUT` stream_type
4, inside TTY's 5-byte payload) is the last data before `channel/close`.
For tunnels it is the last data byte before close. The channels layer's
close handler observes the pump completion; the call operation is issued
after.
This invariant crosses two channels (the data channel and channel 0), so
the channels layer owns the ordering guarantee — it is not a handler
concern. The control-message division (data-ordered control vs
out-of-band control) is now entirely handler-internal: TTY's
`STREAM_CTRL_IN` / `STREAM_CTRL_OUT` are stream_types in TTY's 5-byte
payload format, not channels-layer concepts.
## Design Decisions
All design decisions are documented as ADRs in [decisions/](decisions/).
| ADR | Decision | Summary |
|-----|----------|---------|
| [071](decisions/071-channels-wire-format.md) | channels Wire Format | 8-byte chunk header (amended by ADR-035); channels layer has no `stream_type` concept; one-way door |
| [093](decisions/093-channels-pure-channel-multiplexing.md) | channels Pure Channel Multiplexing | The umbrella decision: 8-byte header, no `stream_type`, `into_sub_streams` removed, `BiStream`-only, TTY always 5-byte |
## Open Questions
Open questions are tracked in [open-questions.md](open-questions.md).
Key questions affecting this doc:
- **OQ-68** (open): Add/strip API shape — whether the 8-byte header
add/strip is built into the channels read/write path or exposed as a
standalone utility. The *contract* (channels strips, handler parses
payload) is decided; the *function surface* is not.
## References
- ADR-034: channels wire format (the decision, amended by ADR-035 — 8-byte
header, no `stream_type`)
- ADR-035: channels pure channel multiplexing (the umbrella decision that
amends ADR-034/074/077)
- ADR-052: alknet-tty wire format (the 5-byte format carried
transparently in the channels payload)
- ADR-036: channel 0 pre-negotiated
- ADR-037: channel lifecycle operations
- ADR-040: backpressure, channel limits, ID reuse
- `docs/research/alknet-channels/poc-summary.md` §POC Target 1, §Issues
Surfaced #4-#6 (REQ-CH-01, 02, 04)
- `docs/research/stream-unification/findings.md` — the research that
surfaced the 8-byte format decision
- `crates/alknet-tty/src/wire.rs` — the 5-byte format implementation
(carried transparently in the channels payload)

855
docs/architecture/client-and-adapters.md Normal file
View File

@@ -0,0 +1,855 @@
---
status: draft
last_updated: 2026-07-17
---
# alknet-call — Client and Adapters
The outbound half of the call protocol: opening connections, importing remote
operations, and the adapter contract that ties import-style adapters together.
This document covers what ADR-022 specced but the server-side implementation
(`call-protocol.md`, `operation-registry.md`) did not include — the `CallClient`
that *opens* a connection, the `from_call` adapter, and the
`OperationAdapter` trait. (`from_jsonschema` was originally specced here
too, but ADR-027 moved it to `alknet-http` — see §"from_jsonschema" below.)
The server-side `CallAdapter` and `CallConnection`
dispatch loop are covered in `call-protocol.md`; this document covers the
client-side connection-establishment half and the adapter surface.
## What
This document specifies three components, all in `alknet-call`:
1. **`CallClient`** — takes over an established transport `Connection`
on ALPN `alknet/call`, spawns the shared dispatch loop, and produces
a `CallConnection`. Transport-agnostic (`spawn_dispatch` primary;
dial lives in `AlknetClient` per ADR-045); the dispatch loop is
shared with the server-side `CallAdapter`
(ADR-022 §1); `CallClient` is the connection-take-over half, not a
parallel protocol implementation.
2. **`from_call`** — discovers operations on a remote call-protocol endpoint
via `services/list` + `services/schema` (already implemented in
`registry/discovery.rs`) and registers them in the connection's Layer 2
overlay as `FromCall`-provenance leaves with forwarding handlers.
3. **`OperationAdapter` trait** — the async trait that `from_call`,
`from_openapi`, `from_mcp`, and `from_jsonschema` all implement.
> **`from_jsonschema` moved.** ADR-027 moved `from_jsonschema` from
> `alknet-call` to `alknet-http` and gave it a real reqwest-backed
> forwarding handler (it was a broken schema-only placeholder before).
> It is now an HTTP-backed single-endpoint adapter for non-standard /
> non-OpenAPI / basic REST endpoints, functionally similar to
> `from_openapi` but one endpoint at a time. See
> [`crates/http/http-adapters.md`](../http/http-adapters.md) §"from_jsonschema".
> The `FromJsonSchema` provenance variant stays in `alknet-call`
> (`OperationProvenance`); only the adapter implementation moved.
It also records two cross-cutting architectural mechanisms that the adapter
surface rests on:
- The **adapter location map** — which adapters live in `alknet-call` vs
`alknet-http`, and why.
- The **no-env-vars invariant** — the architectural mechanism by which
downstream consumers' `std::env::var` credential reads are made unreachable.
And one downstream pattern this completion unblocks:
- The **exchange-of-operations pattern** (runner / container service) — the
canonical bilateral composition this client surface enables.
## Why
The server-side `CallAdapter` (accept path) and `CallConnection` (dispatch
loop) are implemented and tested. The client side is the #1 gap blocking every
downstream consumer: the runner pattern (a process that connects outward to a
hub and exposes local ops), the container-service rewrite, the bilateral
exchange, the NAPI projection, and the agent's cross-node tool dispatch all
require a `CallClient`. `from_call` is the #2 gap; the `OperationAdapter`
trait is the enabling gap for `alknet-http`'s `from_openapi`/`from_mcp`.
ADR-022 specced this surface. This document is the spec that operationally
fills the gap ADR-022 left to implementation: the `CallClient` API, the
`from_call` flow, the trait signature, the adapter location, the credential
invariant, and the bilateral pattern. The gap
analysis (`docs/research/alknet-call-completion/gap-analysis.md`) identified
four decisions (DC-1..4) needed before implementation. DC-1 was initially
resolved by ADR-023 (`remote_safe`/`trusted_peer`), but a subsequent research
pass (`docs/research/alknet-call-peer-routing/findings.md`) found that
ADR-023's model was structurally broken for the head→N-workers pattern (the
primary use case) and that its parallel `remote_safe`/`trusted_peer`
authorization system duplicated the existing `AccessControl`/`Identity`
machinery. **ADR-024 supersedes ADR-023**: peer-keyed overlays + `PeerRef`
routing, and peer authorization through the existing `AccessControl::check(peer_identity)`.
DC-2/3/4 are two-way-door defaults recorded here (DC-2→OQ-27, DC-3→OQ-28
cross-peer dissolved / same-peer stays, DC-4→OQ-26).
## Architecture
### CallClient
`CallClient` takes over an established transport `Connection` on ALPN
`alknet/call`, spawns the shared dispatch loop, and produces a
`CallConnection`. The `CallConnection` type is already implemented
(`call-protocol.md` §"CallConnection") — it wraps an established
`Connection` and holds the Layer 2 imported-ops overlay. `CallClient`
is the producer on the outbound side; `CallAdapter`'s accept path is
the producer on the inbound side. Both produce the same
`CallConnection` and hand it to the same shared dispatch loop.
`CallClient` is transport-agnostic. The call protocol runs over any
ordered, reliable bidirectional stream — QUIC, TCP+TLS, WebTransport,
SSH `direct-tcpip`, a WebSocket (ADR-007 `Connection::from_stream` /
`from_bidi`). The primary constructor (`spawn_dispatch`) takes a
pre-established `Connection` from any transport; the dial lives in
`AlknetClient` (`alknet-client`, ADR-045). This mirrors
`ChannelClient::from_connection` (ADR-043) and is the client-side
analogue of the server-side generalization ADR-007 made.
```rust
pub struct CallClient {
registry: Arc<OperationRegistry>,
identity_provider: Arc<dyn IdentityProvider>,
}
impl CallClient {
pub fn new(registry: Arc<OperationRegistry>, idp: Arc<dyn IdentityProvider>) -> Self;
/// Transport-agnostic primary constructor. Takes a pre-established
/// `Connection` on ALPN `alknet/call` (any transport — QUIC via
/// `from_quinn`, TCP+TLS via `from_bidi`, WebTransport, SSH
/// `direct-tcpip`, a WebSocket), spawns the shared dispatch loop,
/// and returns a live `CallConnection`. Mirrors the server-side
/// `CallAdapter::handle(Connection)`. This is the one-way-door
/// API surface (ADR-022 Am. 2026-07-13) — it must not be coupled to
/// a transport.
pub fn spawn_dispatch(&self, connection: Connection) -> CallConnection;
}
```
Peer authorization flows through the existing `AccessControl::check` against
the peer's resolved `Identity` (ADR-024 §3) — there is no `trusted_peer` flag
and no `remote_safe` marking. When a remote peer calls an op, the dispatch
path resolves the peer's `Identity` (from the connection's TLS fingerprint or
the `auth_token` payload, via the existing `IdentityProvider`) and runs
`AccessControl::check(peer_identity)` against the op's `AccessControl`. If
the op's required scopes/resources are satisfied, the call dispatches; if not,
`FORBIDDEN` before the handler runs (capabilities never populated — the
security property). An op that should never be callable from the wire uses
`Visibility::Internal` (existing mechanism, `NOT_FOUND` before ACL). See
[ADR-024](decisions/029-peer-graph-routing-model.md) §3 for the full
mapping of the three `remote_safe` cases to `AccessControl`/`Visibility`.
The connection is symmetric after establishment (ADR-022 §2): both sides can
send and receive `call.requested`. Connection direction (who opened it) is
independent of call direction (who calls whom). The `CallClient` is therefore
both a caller and a callee — it dispatches incoming calls from the remote
peer through the same `AccessControl`-gated path, and it initiates outgoing
calls through the `CallConnection::call()` / `subscribe()` / `abort()` API.
#### Shared Dispatcher
The shared dispatch loop lives in `protocol/dispatch.rs` as the `Dispatcher`
struct. This is the architectural mechanism that keeps `CallClient` from
becoming a parallel protocol implementation (ADR-022 §1): both `CallAdapter`'s
accept path and `CallClient`'s connect path construct a `Dispatcher` and call
`run_loop` — the dispatch half is one implementation, the
connection-establishment half differs (accept vs dial).
```rust
/// Shared dispatcher for an established CallConnection. Constructed by both
/// CallAdapter (accept path) and CallClient (connect path). Holds no
/// per-connection state; the CallConnection is passed into run_loop.
pub struct Dispatcher {
pub registry: Arc<OperationRegistry>,
pub identity_provider: Arc<dyn IdentityProvider>,
pub session_source: Option<Arc<dyn SessionOverlaySource + Send + Sync>>,
pub default_timeout: Duration,
}
```
The dispatch path resolves the peer's `Identity`, runs `AccessControl::check`
against the op's `AccessControl`, and dispatches if allowed — the same
authorization machinery that gates every other call. No `RemoteFilter`, no
`remote_safe` gate (ADR-024 §3 retires these).
`CallClient::spawn_dispatch(connection)` is the transport-agnostic
primary constructor — it takes a pre-established `Connection`,
constructs a `CallConnection`, builds a `Dispatcher`, spawns the
dispatch task, and returns the live `CallConnection`. The dial lives in
`AlknetClient` (`alknet-client`, ADR-045): keeping a QUIC convenience
constructor on `CallClient` would make `alknet-call` depend on
`alknet-client`, contradicting the dep graph (the protocol crates are
parallel to the dial, not downstream of it). Callers compose
`AlknetClient::dial_quic` + `spawn_dispatch` — two lines, the dial then
the take-over. Tests use `spawn_dispatch` directly to wire mock/loopback
connections. The one-way-door surface is `spawn_dispatch`; the dial
lives in `alknet-client`.
This mirrors `ChannelClient::from_connection` (ADR-043) and is the
client-side analogue of the server-side generalization ADR-007 made.
The call protocol, like the channels protocol, is transport-agnostic —
`Connection::from_stream` / `from_bidi` (ADR-007) accept any
`AsyncRead + AsyncWrite`, and `spawn_dispatch` takes the resulting
`Connection` unchanged.
#### Peer-keyed composition env (ADR-024)
The composition env that aggregates multiple connections is **peer-keyed**
(ADR-024 §1). `CompositeOperationEnv`'s singular
`connection: Option<Arc<dyn OperationEnv>>` is replaced by `PeerCompositeEnv`
with peer-keyed connections:
```rust
pub struct PeerCompositeEnv {
pub base: Arc<dyn OperationEnv + Send + Sync>, // Layer 0 curated
pub session: Option<Arc<dyn OperationEnv + Send + Sync>>, // Layer 1
pub connections: HashMap<PeerId, Arc<dyn OperationEnv + Send + Sync>>, // Layer 2, peer-keyed
connection_order: Vec<PeerId>, // insertion order for PeerRef::Any first-match
}
pub type PeerId = String; // = Identity.id from IdentityProvider resolution
// = PeerEntry.peer_id (stable, not crypto material — ADR-025)
```
`OperationEnv` gains a peer-routing method with a `PeerRef` selector
(`Specific(PeerId)` / `Any`), default-impl for back-compat. See
[ADR-024](decisions/029-peer-graph-routing-model.md) §2 for the full
`invoke_peer` signature and `ScopedPeerEnv` peer-qualified reachability. The
per-`CallConnection` overlay stays flat (one connection = one peer); the
peer-keying is at the aggregation layer (the head node's composition env).
#### services/list
`services/list` filters by `AccessControl::check(calling_peer_identity)` —
the calling peer sees only ops it is authorized to call. There is a
single `AccessControl`-filtered handler (no `peer_scoped` variant, no
`remote_safe` filter — both retired by ADR-024). `services/list-peers`
is the opt-in for peer-attributed re-export listing (each peer's
sub-overlay listed with attribution, filtered by the calling peer's
authorization). See [ADR-024](decisions/029-peer-graph-routing-model.md) §6.
### Credential sources for connections
The credential dimensions are split across two layers (ADR-012, amended
2026-07-17):
- **`ConnectionCredentials`** (in `alknet-core`, per ADR-012) — the
**transport-level** credential bundle, consumed by the dial
(`AlknetClient`). Carries the two transport-identity dimensions:
`local_identity` (the local node's `TlsIdentity`) and `remote_identity`
(the expected fingerprint). The dial does not depend on the call
protocol for this type.
- **`auth_token`** — a **per-request payload field**, not a
call-protocol credential bundle. `Dispatcher::resolve_identity`
reads `payload.get("auth_token")` on each `call.requested` payload.
Browsers send it directly in the WebSocket call payload; the HTTP
gateway resolves the bearer token to an `Identity` at its boundary
(the call layer sees the identity, not the token). See ADR-012 for
the credential-bundle decoupling.
Credentials come from `Capabilities` (ADR-010), never from environment
variables. The transport-identity dimensions (ADR-022 §7):
```rust
// Transport-level (alknet-core, consumed by the dial — ADR-012)
pub struct ConnectionCredentials {
pub local_identity: Option<TlsIdentity>, // RFC 7250 raw key or X.509
pub remote_identity: Option<RemoteIdentity>, // expected fingerprint (None = CA path / fail-closed)
}
// auth_token is a per-request payload field, not a credential struct.
// Browsers send it in the WebSocket call payload; the HTTP gateway
// resolves bearer → Identity at its boundary.
// Dispatcher::resolve_identity reads payload.get("auth_token").
```
`RemoteIdentity` (ADR-022 §7, extended by ADR-034 §2) carries a
fingerprint string the assembly layer derives from `Capabilities` when
the local node has a `PeerEntry` for the remote (the known-peer case →
fingerprint pin). `remote_identity: None` is the **public X.509
endpoint** case: the local node has no `PeerEntry` for the remote, so
there is no fingerprint to pin. Combined with an X.509 transport, `None`
selects CA verification (`WebPkiServerVerifier`) per the
verifier-selection rule in ADR-034 §3. Combined with an Ed25519
raw-key transport, `None` fails closed (raw-key remotes are always
known peers — no CA to fall back to). The `Option` is load-bearing, not
cosmetic: `Some(fingerprint)` means "pin this" (known peer), `None`
means "trust the CA or fail" (unknown remote). An implementer must not
default `remote_identity` to a placeholder value to "satisfy" the field
— `None` is a real state that drives verifier selection.
```rust
pub struct RemoteIdentity { pub fingerprint: String }
```
There is no call-protocol credential bundle. The transport dimensions
(`local_identity`, `remote_identity`) are in `ConnectionCredentials` in
`alknet-core` per ADR-012.
- **TLS identity** — the local node's Ed25519 raw key (RFC 7250) or X.509 cert,
derived from the vault at startup (ADR-020, ADR-026, ADR-027).
- **Auth token** — an opaque call-protocol-level token, decrypted from the
vault or derived from a shared secret.
- **Remote identity verification** — the expected fingerprint/cert of the
remote node, stored as a capability. `Some` → fingerprint pin (known
peer with a `PeerEntry`); `None` → CA verification for X.509 remotes,
fail-closed for Ed25519 raw-key remotes (ADR-034 §2/§3). The `None`
case is the public-X.509-endpoint path, not a missing field.
These are populated by the assembly layer at `CallClient` construction time
from vault-derived `Capabilities`. The credential path is the no-env-vars
invariant (below). The concrete shapes of `TlsIdentity`, `AuthToken`, and
`RemoteIdentity` are implementation-detail two-way doors; the one-way
constraints are that they come from `Capabilities`, not env vars (ADR-010).
**TLS client-auth presentation** (OQ-29 #1, wired): the client presents
its Ed25519 key as an RFC 7250 raw public key client cert — the client-side
equivalent of the server's `RawKeyCertResolver`. This is **wired now**, not
additive: it is what activates the `PeerEntry` fingerprint → `peer_id`
resolution path on quinn connections (ADR-025 §5). Without it, the ADR-024
peer graph doesn't populate for quinn connections — `PeerId` resolution
fails because the server has no client cert to extract a fingerprint from.
The iroh path already works (iroh uses RFC 7250 raw keys and exchanges
Ed25519 public keys during the TLS handshake automatically); the gap was
quinn-only, and OQ-29 #1 resolves it by replacing `with_no_client_auth()`
with presenting the key. The one-way constraint (credentials from
`Capabilities`, not env vars, ADR-010) is unaffected — the `auth_token`
dimension flows through the call-protocol `auth_token` payload field, not
TLS, so the no-env-vars invariant holds independently of the TLS layer.
**Remote-identity verification** (OQ-29 #2, additive): verifying the
server's fingerprint against an expected value (`credentials.remote_identity`)
is **additive** — the server-side fingerprint extraction is what matters for
`PeerId`, not the client-side verification. The verifier for raw keys can
start as "accept any, extract fingerprint" and add fingerprint-pinning later.
This is a two-way-door remainder; the one-way constraint (credentials from
`Capabilities`, not env vars) is unaffected.
**Server cert verifier selection** (OQ-29 #2 + ADR-034 §3): the client-side
`ServerCertVerifier` is selected by whether the local node has a `PeerEntry`
for the remote, not by key type alone. A pure-client
connection to a **public X.509 endpoint** (no `PeerEntry` on the local
side — e.g., dialing `api.alk.dev` or a third-party API) uses
`WebPkiServerVerifier` (CA verification), gets **no `PeerId`** on the
client side, and is **not added to `PeerCompositeEnv`** — it is not in
the call-protocol peer graph (ADR-024). Ops discovered via `from_call`
on such a connection land in the connection's Layer 2 overlay
(ADR-019) and are invoked through the `CallConnection` handle directly,
not via `PeerRef::Specific`. A connection to a **hub** (a `PeerEntry`
with mixed Ed25519 + X.509 fingerprints) uses fingerprint pinning on
both cert paths and does enter the peer graph. An unknown Ed25519
raw-key remote fails closed (no CA to fall back to — raw-key remotes
are always known peers). See
[ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md)
for the verifier selection rule and the three-role naming.
### from_call
`from_call` discovers the remote peer's `External` operations and registers
them in the connection's Layer 2 overlay as `FromCall`-provenance leaves with
forwarding handlers. The discovery mechanism (`services/list` +
`services/schema`) is already implemented in `registry/discovery.rs`;
`from_call` is the client-side consumer of that API.
```rust
pub struct FromCallConfig {
/// Namespace prefix applied to imported operation names. Optional —
/// default no prefix. Collision on import is an error (DC-3, OQ-28),
/// not last-wins.
pub namespace_prefix: Option<String>,
/// Optional filter — import only operations whose names match. None
/// imports all External ops discovered via services/list.
pub operation_filter: Option<HashSet<String>>,
}
/// Discover the remote peer's External ops and construct HandlerRegistration
/// bundles with FromCall provenance and forwarding handlers. The caller
/// registers the bundles in the connection's overlay via
/// CallConnection::register_imported_all().
pub async fn from_call(
connection: &CallConnection,
config: FromCallConfig,
) -> Result<Vec<HandlerRegistration>, AdapterError>;
```
The flow (ADR-022 §3):
1. Call `services/list` on the remote → list of `External` operations.
2. Call `services/schema` for each → input/output JSON Schemas and declared
`error_schemas` (ADR-016).
3. For each discovered op, construct a `HandlerRegistration`:
- `spec` mirrors the remote op's name (with optional prefix), namespace,
type, schemas, access control.
- `handler` is a forwarding handler, **branched on `op_type`** (ADR-021):
- `Query` / `Mutation` → a `Handler` (registered as `HandlerKind::Once`):
sends `call.requested` via `CallConnection::call_with_payload()`, awaits
the single `call.responded` (or `call.error`), returns the
`ResponseEnvelope`.
- `Subscription` → a `StreamingHandler` (registered as
`HandlerKind::Stream`): calls `CallConnection::subscribe()`, which
returns `impl Stream<Item = ResponseEnvelope>` (the client-side
streaming path, already implemented), maps it to a
`BoxStream<ResponseEnvelope>`. The remote stream flows end-to-end:
each `call.responded` the remote sends becomes a stream item; the
remote's `call.completed` ends the stream (→ wire `call.completed`);
`call.aborted` drops the stream (cascade per ADR-020). No truncation,
no first-value fallback — a `from_call`-imported subscription forwards
the full remote stream.
- `provenance: FromCall`, `composition_authority: None`, `scoped_env: None`
(leaf — ADR-018).
4. The caller registers the bundles via
`CallConnection::register_imported_all()`.
**Re-import on reconnection** (DC-2, OQ-27): `from_call` is a free function;
the assembly layer calls it after the dial (in `AlknetClient`). The overlay
is per-connection (Layer 2, ADR-019), so a stale overlay dies with the
connection; re-import on reconnect is naturally scoped to the new
connection. A `CallConnection::refresh()` method for mid-connection
re-discovery is a genuine feature addition — non-breaking, additive — if a
deployment needs manual re-discovery without drop-and-reconnect. See
[ADR-028](decisions/069-from-call-manual-free-function.md).
**Namespace collision** (DC-3, OQ-28): under the peer-graph model (ADR-024),
cross-peer collision dissolves — same name on different peers is fine (they
live in separate peer sub-overlays, no prefix needed). Same-peer collision
stays an error (a peer shouldn't expose two ops with the same name).
`FromCallConfig::namespace_prefix` is optional local-naming sugar for when
the importing node wants to expose a peer's ops under a different name
*locally* — a local-naming concern, not a disambiguation concern. It defaults
to `None`.
**Trust is transitive** (recorded in `operation-registry.md`): a
`from_call`-imported operation executes the remote node's code, not yours.
The scoped env (ADR-017) bounds *which* operations are reachable, not *what*
they do. `from_call` means "I trust the remote node as much as my own
handlers." The abort cascade (ADR-020) crosses the node boundary transparently
through the forwarding handler's `parent_request_id`.
**Forwarded-for identity** (ADR-026): the `from_call` forwarding handler
populates `forwarded_for` on the `call.requested` payload it constructs to
send to the spoke. The hub reads its own `OperationContext.identity` (the
end user it authenticated) and sets `forwarded_for` to that identity when
forwarding. The spoke receives it as metadata on its `OperationContext` —
available for logging, auditing, per-user rate limiting, but never used by
`AccessControl::check` (the spoke authorizes the hub, its direct caller,
not the end user). The hub may set `forwarded_for: None` if it doesn't
want to disclose the originator. See [ADR-026](decisions/032-forwarded-for-identity.md).
### from_jsonschema
`from_jsonschema` was originally specified here (ADR-022 §5) as a
schema-only adapter in `alknet-call` — a placeholder handler returning
`NOT_FOUND`. That was broken: an op in the registry needs a real handler,
and the "schema-only, no handler" concept conflated schema validation
(a planning activity that doesn't need a registry entry) with operation
registration (which always needs a handler).
[ADR-027](decisions/066-from-jsonschema-as-http-adapter.md) moved
`from_jsonschema` to `alknet-http` as an HTTP-backed single-endpoint
adapter: the caller supplies an `OperationSpec` + `HttpServiceConfig` +
path template + method, and the adapter builds one
`HandlerRegistration` with a real reqwest forwarding handler and
`FromJsonSchema` provenance. It is functionally similar to `from_openapi`
but one endpoint at a time, for non-standard / non-OpenAPI / basic REST
endpoints that don't have a full OpenAPI document. See
[`crates/http/http-adapters.md`](../http/http-adapters.md) §"from_jsonschema".
The schema-validation-without-a-handler use case (the original stated
purpose) is served by consuming `OperationSpec` directly — the spec
already carries the input/output JSON Schemas. No adapter, no registry
entry, no handler is needed for that.
The `FromJsonSchema` provenance variant stays in `alknet-call`
(`OperationProvenance` in `registry/registration.rs`); only the adapter
implementation moved.
### OperationAdapter trait
The shared shape across import-style adapters. The trait lives in
`alknet-call` (where the types live); the implementations live where their
transport dependencies live (see "Adapter Location Map" below).
```rust
#[async_trait]
pub trait OperationAdapter: Send + Sync {
async fn import(&self) -> Result<Vec<HandlerRegistration>, AdapterError>;
}
```
The trait is **async** because `from_call` requires async discovery
(`services/list` + `services/schema` over a call-protocol connection,
which may be QUIC, TCP+TLS, or any other transport). Sync adapters
(`from_openapi`, `from_mcp` reading a static spec) trivially satisfy an async
trait — their `import()` bodies contain no `.await` points. This is locked by
ADR-022 §5.
The **error type** (DC-4, OQ-26) is `Result<Vec<HandlerRegistration>,
AdapterError>` where `AdapterError` is a crate-level enum covering the
failure modes real implementations hit: discovery transport failure
(`from_call` remote unreachable), schema parse failure (`from_openapi`,
`from_jsonschema`), unauthorized (HTTP 401 for `from_openapi`,
`from_mcp`). The exact `AdapterError` variants are the two-way-door
remainder; the *presence* of an error type is filled in here. ADR-022 §5
showed `async fn import(&self) -> Vec<HandlerRegistration>` with no error
type; the spec omitted the error type as an implementation-detail two-way
door, recorded here.
Implementations:
- `FromCall` — call-protocol-backed, transport-agnostic (in
`alknet-call`). `from_call` discovers ops over a `CallConnection`,
which may be QUIC, TCP+TLS, or any transport `Connection::from_stream`
supports (ADR-007).
- `FromOpenAPI` — HTTP-backed (in `alknet-http`).
- `FromJsonSchema` — HTTP-backed, single-endpoint (in `alknet-http` per
ADR-027; was a broken schema-only placeholder in `alknet-call`).
- `FromMCP` — MCP streamable-HTTP-backed (in `alknet-http`, feature-gated).
The `to_*` adapters (`to_openapi`, `to_mcp`) are outbound projections, not
`OperationAdapter` implementations — they consume the registry, they don't
produce entries for it (ADR-022 §5).
### Adapter Location Map
The decomposition principle: **the adapter trait lives where the types live
(`alknet-call`); the adapter implementations live where their transport
dependencies live.**
```
alknet-call (lean — no HTTP client, no HTTP server)
├── OperationAdapter trait (the contract — async, per ADR-022 §5)
├── from_call (transport-agnostic — discovers remote ops via
│ call protocol over any Connection)
└── CallClient (outbound connection take-over —
spawn_dispatch, transport-agnostic; dial in AlknetClient)
alknet-http (owns HTTP server + HTTP client — separate crate, separate Phase 0)
├── ProtocolHandler for h2/http1.1/h3 (axum server — inbound HTTP)
├── from_openapi (parse OpenAPI doc + reqwest forwarding handler)
├── from_jsonschema (single-endpoint reqwest forwarding handler — ADR-027)
├── to_openapi (generate OpenAPI doc from local registry)
├── from_mcp (feature-gated) (import remote MCP tools over streamable HTTP — reqwest)
└── to_mcp (feature-gated) (expose local ops as MCP tools over streamable HTTP — axum)
Not built: MCP stdio transport
— stdio = spawn arbitrary executable = built-in RCE ("download untrusted MCP servers")
— streamable HTTP is the only supported MCP transport in alknet
— recorded as an explicit security position, not a feature gap
```
`alknet-call` never sees the HTTP client. The `from_openapi`/`from_mcp`
forwarding handlers are opaque `Arc<dyn Handler>` from the registry's
perspective — constructed by `alknet_http::from_openapi()` at registration
time, stored in `HandlerRegistration`, dispatched by the `CallAdapter` which
doesn't know reqwest is involved. `alknet-call` stays lean (no reqwest, no
axum); `alknet-http` owns both HTTP directions.
**ADR-031 dependency note**: `alknet-http` implementing `from_openapi`/
`from_mcp` means `alknet-http` depends on `alknet-call` (for `OperationSpec`,
`Handler`, `HandlerRegistration`, `OperationAdapter`). ADR-031's rule is "no
handler crate depends on another handler crate" — but `alknet-call` is both
a handler *and* the protocol foundation that `alknet-agent` and `alknet-napi`
already consume. `alknet-http` depending on `alknet-call` is "HTTP uses the
call protocol types," not "HTTP depends on SSH." This is within the spirit of
ADR-031 (`alknet-call` is protocol-foundation, not a peer handler). The
`alknet-http` spec should note this explicitly; a one-line amendment to
ADR-031 clarifying that `alknet-call` is a protocol-foundation crate is
deferred to the `alknet-http` Phase 0.
### No-Env-Vars Invariant
The architectural mechanism for the env-var problem in downstream consumers
(the Rust port of Vercel's AI SDK at `/workspace/aisdk/`, whose providers all
read `std::env::var("OPENAI_API_KEY")` in their `Default` impls). The fix is
**not** to modify those consumers — it's that the env-var path is never taken
because the assembly layer never calls `Default::default()`.
The credential injection path:
```
vault (seed)
→ assembly layer (derive + decrypt at startup, per ADR-010/019/025)
→ Capabilities (non-serializable, zeroized, immutable — ADR-010)
→ HandlerRegistration.capabilities (ADR-018, the registration bundle)
→ OperationContext.capabilities (per-request, populated by dispatch
path from the bundle — ADR-018 §6)
→ from_openapi handler reads context.capabilities.get("openai")
→ injects into HTTP Authorization header
→ reqwest request goes out with vault-derived credential
```
The `from_openapi`/`from_mcp` forwarding handlers (in `alknet-http`) are the
credential injection point. They read from `context.capabilities`, not from
`std::env::var`. The downstream consumers' `Default` impls reading env vars
are simply never called — the assembly layer constructs providers with
vault-derived credentials through the builder API, or the provider's HTTP
calls are routed through `from_openapi` operations that carry the credential
in `Capabilities`.
**This is a spec-level invariant in `alknet-call`, not a runtime convention.**
The dispatch path (`build_root_context` and `OperationEnv::invoke()` per
ADR-018 §6) populates `OperationContext.capabilities` from the registration
bundle. The invariant is: *no handler reads outbound credentials from any
source other than `OperationContext.capabilities`.* This is already the
architectural intent of ADR-010; this document records it as an explicit
invariant that the `from_openapi`/`from_mcp` handler implementations (in
`alknet-http`) are verified against.
### Exchange-of-Operations Pattern (Runner / Container Service)
The canonical downstream pattern this completion unblocks, recorded here so
Phase 1 specs can reference it. Concrete example: the container service at
`/workspace/@alkdev/dispatch` (axum + russh SSH client for "reverse git
runner" over Docker/vast.ai) gets rewritten as a call-protocol service.
**Bilateral exchange**:
```
Container service (runs on a vast.ai/docker instance):
Defines Local ops: /container/exec, /container/list, /container/logs...
(real handlers — calls bollard or vast.ai API)
Connects to hub as a CallClient (outbound connection — runner pattern)
Hub (central server):
Runs CallAdapter (server) on alknet/call (already implemented)
When the container service connects:
hub runs from_call → discovers /container/* via services/list + services/schema
registers them as FromCall provenance (leaf, forwarding handlers) in the
connection's Layer 2 overlay (ADR-019)
Now the hub (or anything connected to the hub) can call /container/exec
The from_call handler forwards over the connection back to the container service
Bilateral: the container service ALSO runs from_call against the hub,
discovers the hub's External ops, and can call them.
Connection direction (container → hub) is independent of call direction
(both can call each other) per ADR-022 §2.
```
**What this requires**:
1. `CallClient` — the container service uses it to open the outbound
connection to the hub. The #1 gap.
2. `from_call` — both sides run it to populate their Layer 2 overlays with
the other side's `External` ops. The #2 gap.
3. `OperationAdapter` trait — `from_call` implements it. The #3 gap (enabling,
not blocking — `from_call` can be built as a free function before the trait
exists, but the trait is needed for `alknet-http`'s adapters).
**Why the container service doesn't need alknet-ssh**: under the call
protocol, the container service is a `CallClient` that dials the hub's
`alknet/call` ALPN (over QUIC, TCP+TLS, or any transport) — no SSH in
the loop. SSH port
forwarding becomes the *transitional* mechanism for targets that can't run a
call-protocol client (the `alknet-ssh` phase-0 findings document this
transition). Once the container service runs a `CallClient`, SSH is out of
the path entirely.
This is the "dev runner" pattern: a call-protocol client that connects back
to a hub and exposes core dev tools (bash, fs, etc.) as operations. The agent
service (`alknet-agent`, downstream) is the consumer that orchestrates these
via `env.invoke()`.
## Implementation Priority Order
Based on the gap analysis and the downstream unblock chain:
1. **`CallClient`** (critical) — outbound connection opener. Without it, no
runner, no container service, no bilateral exchange. Reuses the existing
`CallConnection` for the dispatch loop; adds only the
connection-establishment + credential-handling half. The single
highest-value piece of work in the entire `alknet-call` completion.
2. **`from_call`** (critical, depends on `CallClient`) — consumes the
already-implemented `services/list` + `services/schema` discovery API.
3. **`OperationAdapter` trait** (enabling) — the async trait. Small,
standalone, unblocks `alknet-http` Phase 1 (including `from_jsonschema`
per ADR-027).
4. **DC-1 resolution** (peer-graph routing model, ADR-024) — the
peer-keyed overlay + `AccessControl`-based peer authorization model that
replaces ADR-023's `remote_safe`/`trusted_peer`. This is a structural
change to `CompositeOperationEnv` (→ `PeerCompositeEnv`), the dispatch
path (retire `RemoteFilter`), and `OperationEnv` (gain `invoke_peer`).
See ADR-024 for the migration; the POC shapes in the research doc are the
reference.
## What This Completion Unblocks
| Downstream crate | What it needs from alknet-call | Status without completion |
|-------------------|-------------------------------|--------------------------|
| alknet-http | `OperationAdapter` trait (to implement `from_openapi`/`from_mcp`) | Blocked — can't define HTTP-backed adapters without the trait |
| alknet-ssh | Stable alknet-call types (no adapter dependency) | Not blocked — ssh depends on alknet-core, not alknet-call's adapters. Proceeds in parallel. |
| alknet-agent | `CallClient` (tool dispatch), `from_call` (remote tool import), `OperationAdapter` (provider adapters) | Blocked on `CallClient` + `from_call` |
| Container service (dispatch rewrite) | `CallClient` + `from_call` | Blocked — this is the primary consumer |
| Runner pattern (dev runner, opencode runner) | `CallClient` + `from_call` | Blocked — the runner IS a `CallClient` |
| alknet-napi | `CallClient` (Node.js calls remote ops) | Blocked — NAPI projects `CallClient` to JS |
## Constraints
- **No HTTP in alknet-call.** `from_openapi`/`from_mcp`/`from_jsonschema`/
`to_openapi`/`to_mcp` live in `alknet-http`. The `OperationAdapter`
trait and the call-protocol-backed adapter (`from_call`, transport-
agnostic) live in `alknet-call`. `from_jsonschema` was originally
(mis)placed in `alknet-call` as a schema-only placeholder; ADR-027
moved it to `alknet-http` as a real HTTP-backed adapter. See Adapter
Location Map.
- **No secret material on the wire.** `ConnectionCredentials` carries vault-derived
material for the *outbound* connection (TLS identity); `auth_token` is a
per-request payload field (browsers send it in the WebSocket call payload;
the HTTP gateway resolves bearer → `Identity` at its boundary). The
call protocol's wire format carries no private keys, API keys, or decrypted
credentials (ADR-010). The no-env-vars invariant (above) is the dispatch-side
corollary.
- **Peer authorization via `AccessControl`.** A remote peer's call is
authorized by `AccessControl::check(peer_identity)` against the op's
`AccessControl` — the same mechanism that gates every other call. No
`remote_safe` flag, no `trusted_peer` bypass (ADR-024 §3). An op with
`AccessControl::default()` is callable by any peer; an op with
`required_scopes` is callable only by peers whose `Identity.scopes` satisfy
them; an op with `Visibility::Internal` is never callable from the wire.
- **Composition env is peer-keyed.** A head node with N worker connections
holds a `PeerCompositeEnv` with `connections: HashMap<PeerId, Arc<dyn OperationEnv>>`,
not a singular connection overlay. `invoke_peer()` routes to the right peer
via `PeerRef::Specific` / `PeerRef::Any` (ADR-024 §1-2).
- **`from_call` is a manual free function.** The assembly layer calls it
after the dial (in `AlknetClient`). The overlay is per-connection so
re-import on reconnect is naturally scoped (DC-2, OQ-27). See
[ADR-028](decisions/069-from-call-manual-free-function.md).
- **`from_call` namespace collision is same-peer only.** Cross-peer collision
dissolves (same name on different peers is fine — separate sub-overlays,
ADR-024 §5). Same-peer collision stays an error. `namespace_prefix` is
optional local-naming sugar, not the disambiguation mechanism (DC-3, OQ-28).
- **`OperationAdapter::import()` returns `Result`.** Failures surface as
`AdapterError` (DC-4, OQ-26).
- **MCP stdio transport is not built.** Streamable HTTP is the only supported
MCP transport in alknet. stdio = spawn arbitrary executable = built-in RCE.
Recorded as an explicit security position, not a feature gap.
- **Pure-client X.509 connections are not in the peer graph on the client
side.** A `CallClient` connection to a public X.509 endpoint with no
local `PeerEntry` for the remote gets no `PeerId`, is not added to
`PeerCompositeEnv`, and is not addressable via `PeerRef::Specific`.
Ops discovered on it live in the connection's Layer 2 overlay and are
invoked through the `CallConnection` handle. The client-side
`ServerCertVerifier` uses CA verification (`WebPkiServerVerifier`) for
such remotes; known peers (hub with `PeerEntry`) use fingerprint
pinning. See [ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md).
- **`ConnectionCredentials.remote_identity: None` is load-bearing.** `None`
means "no `PeerEntry` for this remote → use CA verification (X.509)
or fail closed (Ed25519 raw key)" per the ADR-034 §3 verifier rule.
The implementation must not default `remote_identity` to a placeholder
to satisfy the field, and must not treat `None` as "skip verification"
— `None` + X.509 is CA verification, `None` + raw key is a hard
failure. `Some(fingerprint)` is the known-peer pin path.
## Design Decisions
| Decision | ADR | Summary |
|----------|-----|---------|
| Call protocol client and adapter contract | [ADR-022](decisions/017-call-protocol-client-and-adapter-contract.md) | `CallClient` opens connections; `from_call` imports remote ops; connection direction independent of call direction; trait is async; adapters produce `HandlerRegistration` bundles |
| `from_jsonschema` as HTTP-backed single-endpoint adapter in alknet-http | [ADR-027](decisions/066-from-jsonschema-as-http-adapter.md) | Moved `from_jsonschema` from `alknet-call` (broken schema-only placeholder) to `alknet-http` as a real reqwest-backed single-endpoint adapter; `FromJsonSchema` provenance stays in `alknet-call` as a leaf |
| Peer-graph routing model (DC-1, supersedes ADR-023) | [ADR-024](decisions/029-peer-graph-routing-model.md) | Peer-keyed overlays + `PeerRef` routing; peer authorization via existing `AccessControl::check(peer_identity)`; retires `remote_safe`/`trusted_peer` |
| PeerEntry and Identity.id decoupling | [ADR-025](decisions/030-peerentry-and-identity-id-decoupling.md) | `PeerId` source changes from UUID to `Identity.id` (= `PeerEntry.peer_id`, stable across key rotation); `Identity.id` decoupled from crypto material on the fingerprint path |
| Forwarded-for identity | [ADR-026](decisions/032-forwarded-for-identity.md) | `forwarded_for` field on `call.requested` and `OperationContext`; the `from_call` handler populates it; metadata only, never used by `AccessControl::check` |
| Storage boundary and repo/adapter pattern | [ADR-033](decisions/033-storage-boundary-and-repo-adapter-pattern.md) | Core defines repo traits + in-memory defaults; persistence adapters are separate crates |
| Secret material flow and capability injection | [ADR-010](decisions/014-secret-material-flow-and-capability-injection.md) | The no-env-vars invariant's foundation; capabilities injected at assembly layer |
| Handler registration, provenance, and composition authority | [ADR-018](decisions/022-handler-registration-provenance-and-composition-authority.md) | The registration bundle adapters produce; `composition_authority: None` for leaves |
| Operation registry layering | [ADR-019](decisions/024-operation-registry-layering.md) | Layer 2 per-connection overlay where `from_call` imports land |
| Privilege model and authority context | [ADR-017](decisions/015-privilege-model-and-authority-context.md) | Adapter-registered ops are `Internal` by default; default-deny posture |
| Abort cascade for nested calls | [ADR-020](decisions/016-abort-cascade-for-nested-calls.md) | Cross-node abort through `from_call` forwarding handler's `parent_request_id` |
| Operation error schemas | [ADR-016](decisions/023-operation-error-schemas.md) | `error_schemas` mirrored by `from_call` from remote op's spec |
| Streaming handler for subscriptions | [ADR-021](decisions/049-streaming-handler-for-subscriptions.md) | `from_call` `Subscription` ops register a `StreamingHandler` (`HandlerKind::Stream`) that calls `CallConnection::subscribe()` and forwards the remote stream; `Query`/`Mutation` stay `HandlerKind::Once` |
| TLS identity redesign | [ADR-027](decisions/027-tls-identity-redesign-acme-rawkey-decoupling.md) | RFC 7250 raw key / X.509 cert dimensions of the local `TlsIdentity` (now carried by `ConnectionCredentials.local_identity`) |
| Outgoing-only X.509 and three peer roles | [ADR-034](decisions/034-outgoing-only-x509-and-three-peer-roles.md) | Public X.509 endpoint is not a `PeerEntry` on the client side (no `PeerId`, not in peer graph); client-side verifier by `PeerEntry` presence (CA vs fingerprint pin); hub = mixed-fingerprint `PeerEntry` |
| HD derivation for encryption keys | [ADR-020](decisions/020-hd-derivation-for-encryption-keys.md) | Vault-derived TLS identity material |
| Vault key model | [ADR-026](decisions/026-vault-key-model-hd-derivation.md) | Vault-derived TLS identity material |
| Vault local-only dispatch | [ADR-025](decisions/025-vault-local-only-dispatch.md) | Vault access at assembly layer only; the credential injection path's first hop |
| Crate decomposition | [ADR-031](decisions/003-crate-decomposition.md) | `alknet-http` depends on `alknet-call` (protocol-foundation exception, noted in Adapter Location Map) |
| One-way door decision framework | [ADR-032](decisions/009-one-way-door-decision-framework.md) | Door-type classification for DC-1..4 |
## Open Questions
See [open-questions.md](open-questions.md) for full details.
- **OQ-25** (dissolved by ADR-024): `remote_safe` marking shape — moot.
`remote_safe`/`trusted_peer` are retired; peer authorization is
`AccessControl::check(peer_identity)`. No marking to shape.
- **OQ-26** (resolved): `AdapterError` variants — `DiscoveryFailed`,
`SchemaParse`, `Transport`, `Unauthorized`, `SamePeerCollision`
(replaces flat `Conflict`). `#[non_exhaustive]`.
- **OQ-27** (resolved): `from_call` re-import trigger — `from_call` is a
manual free function; the assembly layer calls it after the dial (in
`AlknetClient`). A `CallConnection::refresh()` method is a genuine
feature addition — non-breaking, additive. See
[ADR-028](decisions/069-from-call-manual-free-function.md).
- **OQ-28** (resolved): `from_call` namespace collision — same-peer
collision = error; cross-peer dissolved by ADR-024 (separate sub-overlays).
`namespace_prefix` is optional local-naming sugar.
- **OQ-29** (resolved): `CallClient` TLS client-auth — wire quinn
client-auth (present Ed25519 key as raw public key client cert);
key-type-aware server cert verification (raw key = fingerprint match,
X.509 = CA verification); fingerprint normalization (`ed25519:` across
quinn/iroh). The iroh path already works; the gap was quinn-only.
See OQ-29 in open-questions.md.
- **OQ-30** (resolved): `PeerRef::Any` routing policy — insertion-order
first-match. A richer `RoutingPolicy` is a feature extension.
- **OQ-31** (resolved): `services/list-peers` — opt-in; `services/list`
is "own ops only."
- **OQ-32** (open, feature extension): Multi-hop federation — the one-hop
model is the architectural commitment; multi-hop is a feature extension
that doesn't break downstream. The peer-keyed model extends to multi-hop
without redesign; petgraph is the candidate if path-finding becomes real
(ADR-024 §3.7).
- **OQ-33** (resolved by ADR-025): `PeerId` is a logical id. Source is
`Identity.id` from `IdentityProvider` resolution (= `PeerEntry.peer_id`,
stable across key rotation). See OQ-33 in open-questions.md.
- **OQ-34** (resolved by ADR-025 + ADR-033): Persistent peer registry —
the storage boundary is `core trait + in-memory default` (config-backed
`ConfigIdentityProvider` now; persistence adapters additive in separate
crates). See OQ-34 in open-questions.md.
- **OQ-35** (dissolved): the "API key asymmetry" framing was wrong;
`PeerEntry` supports multiple credential paths (fingerprints +
auth_token_hash), `ApiKeyEntry` is for tokens that ARE the identity.
See OQ-35 in open-questions.md.
- **OQ-36** (resolved by ADR-035): Concrete persistence adapter shapes —
read-sync / write-async split (`IdentityStore` async write trait
extends the sync `IdentityProvider` read trait); SQLite adapter caches
in memory and uses honker NOTIFY/LISTEN for no-restart cache
invalidation; `alknet-store-sqlite` crate implements both
`IdentityStore` and `CredentialStore`. See ADR-035 and OQ-36 in
open-questions.md.
- **OQ-37** (resolved by ADR-034): X.509 outgoing-only case — three
remote roles named (public X.509 endpoint, transport relay, hub).
`PeerEntry` asymmetry is correct: a pure-client connection to a public
X.509 endpoint is **not** in the call-protocol peer graph on the
client side — no `PeerEntry`, no `PeerId`, no `PeerRef::Specific`
routing. Ops discovered via `from_call`/`from_openapi`/`from_mcp`
land in the connection's Layer 2 overlay and are invoked through the
connection handle. The client-side `ServerCertVerifier` is selected
by `PeerEntry` presence: known peer → fingerprint pin; unknown X.509
remote → CA verification (`WebPkiServerVerifier`). See ADR-034 and
OQ-37 in open-questions.md.
## References
- ADR-022: Call Protocol Client and Adapter Contract (the spec this document
operationally fills)
- ADR-024: Peer-Graph Routing Model (resolves DC-1 with peer-keyed overlays
+ `AccessControl`-based peer authorization)
- `call-protocol.md` — `CallAdapter`, `CallConnection`, dispatch loop, stream
model (the server-side complement to this document)
- `operation-registry.md` — `HandlerRegistration`, provenance, capability
injection, service discovery (the discovery API `from_call` consumes)
- `docs/research/alknet-call-completion/gap-analysis.md` — DC-1..4, the
implementation-state audit, the downstream unblock chain
- `docs/research/alknet-call-peer-routing/findings.md` — the peer-graph
routing research that identified ADR-023's structural gap and validated
the ADR-024 design via POC
- `/workspace/@alkdev/operations/` — TypeScript prior art (`from_openapi.ts`,
`from_mcp.ts`, `from_schema.ts`, `scanner.ts`)
- `/workspace/@alkdev/dispatch/` — concrete downstream consumer (container
service / "reverse git runner") this completion unblocks
- `/workspace/aisdk/` — downstream consumer (Rust port of Vercel AI SDK); the
no-env-vars invariant makes its `std::env::var` reads unreachable
- `/workspace/rust-sdk/` — MCP Rust SDK (rmcp); streamable HTTP transport for
`alknet-http`'s `from_mcp`/`to_mcp` (separate crate, separate Phase 0)
- `docs/research/alknet-ssh/phase-0-findings.md` — alknet-ssh Phase 0;
confirms ssh depends on alknet-core not alknet-call's adapters, so it
proceeds in parallel with this completion

46
docs/architecture/decisions/001-alpn-protocol-dispatch.md Normal file
View File

@@ -0,0 +1,46 @@
# ADR-001: ALPN-Based Protocol Dispatch
## Status
Accepted
## Context
The previous architecture used a three-layer model: transports produced byte streams, interfaces defined how to interpret those streams (StreamInterface, MessageInterface), and OperationEnv dispatched operations through local, irpc, or remote paths. This required a ListenerConfig enum with three variants (Stream, Http, Dns), a server accept loop handling three different listener types, and a complex dispatch model that mixed concerns across layers.
Protocol detection was done by byte-peeking — the server read the first bytes of an incoming connection and guessed which protocol the client was speaking. This is fragile, limits protocol extensibility, and cannot work with encrypted transports where the payload is opaque.
ALPN (Application-Layer Protocol Negotiation) is a TLS extension where the client advertises supported protocols during the handshake and the server selects one. QUIC builds on this natively — every QUIC connection has an ALPN. This is the same pattern iroh uses: `Router` dispatches incoming QUIC connections to `ProtocolHandler` implementations based on the ALPN string. Hickory DNS registers ALPN protocols (`dot`, `doq`, `h2`, `h3`). The reverse-proxy project at `@alkdev/reverse-proxy` uses the same pattern for TLS.
The core insight: **a service IS an ALPN**. Every protocol handler registers an ALPN string on a shared QUIC+TLS endpoint. The ALPN negotiation during the handshake routes the connection to the correct handler before any application bytes are read.
## Decision
All protocol dispatch in alknet is ALPN-based. A single QUIC+TLS endpoint accepts connections, and the ALPN string selected during the handshake determines which `ProtocolHandler` receives the connection. There is no byte-peeking, no ListenerConfig enum, and no three-layer dispatch model.
The endpoint advertises the union of all registered handlers' ALPN strings. When a client connects, the TLS/QUIC handshake negotiates the ALPN. If the client's offered ALPNs and the server's advertised ALPNs have no intersection, the handshake fails — this is the correct behavior, not an error to work around.
## Consequences
**Positive:**
- Single dispatch mechanism replaces three separate listener types
- Protocol detection happens at the TLS layer, not application layer — no byte-peeking
- Adding a new protocol is registering a new ALPN string — no server code changes
- Each handler owns its entire wire format — no shared framing layer
- QUIC connections are cheap — a client that needs multiple protocols opens one connection per ALPN, all multiplexed over the same UDP flow
- Stealth mode (byte-peek protocol detection on port 443) is unnecessary — ALPN negotiation handles this cleanly
- WASM story is clean: handlers receive byte streams, protocol parsers that operate on bytes compile to WASM
**Negative:**
- ALPN is negotiated per-connection, not per-stream — a client that wants to use multiple ALPNs (e.g., SSH and call protocol) opens separate QUIC connections for each. QUIC connections are cheap (multiplexed over the same UDP flow), so this is acceptable, but it means `alknet/call` cannot serve as a multiplexer for other ALPNs within a single connection unless explicitly designed to do so (see ADR-004).
- All protocols must be registered at endpoint creation time (or use hot-reload via ArcSwap for dynamic addition)
- Custom protocols require reserving ALPN strings — we own the `alknet/` namespace
- Debugging requires knowing which ALPN was negotiated (mitigated by logging at the endpoint level)
## References
- Pivot proposal: `docs/research/pivot/alpn-service-architecture.md`
- ADR-002: ProtocolHandler trait
- ADR-031: Crate decomposition
- iroh reference: `docs/research/references/iroh/` (ALPN dispatch, ProtocolHandler pattern)
- Replaces the old three-layer model (StreamInterface/MessageInterface/OperationEnv)

66
docs/architecture/decisions/002-protocol-handler-trait.md Normal file
View File

@@ -0,0 +1,66 @@
# ADR-002: ProtocolHandler Trait
## Status
Accepted
## Context
The previous architecture had two separate interface traits: `StreamInterface` (for byte-stream protocols like SSH, raw TCP) and `MessageInterface` (for message-based protocols like DNS, HTTP). This split created complexity — each interface type needed its own listener configuration, its own dispatch path, and its own framing assumptions. The `ListenerConfig` enum had three variants. The server accept loop handled three different listener types.
In practice, the distinction between "stream" and "message" protocols is artificial at the handler level. SSH starts as a byte stream but internally multiplexes channels and messages. DNS over QUIC is message-based but arrives as a stream of frames. HTTP/2 is both — bidirectional streams with message semantics. Every protocol can be modeled as "receive a byte stream, manage your own wire format."
iroh's `ProtocolHandler` trait demonstrates this: it takes a bidirectional QUIC stream and the handler is responsible for its own protocol. One trait, one dispatch point.
## Decision
A single `ProtocolHandler` trait replaces both `StreamInterface` and `MessageInterface`:
> **Note**: The signature below was revised by ADR-005. The `handle()` method
> now receives a `Connection` (not a `BiStream`) — see ADR-005 for the
> current authoritative signature. The original signature is retained here
> for historical context.
```rust
#[async_trait]
pub trait ProtocolHandler: Send + Sync + 'static {
/// The ALPN string this handler claims (e.g. b"alknet/ssh")
fn alpn(&self) -> &'static [u8];
/// Handle an incoming connection (revised by ADR-005 to receive
/// `Connection` instead of `BiStream`)
async fn handle(&self, connection: Connection, auth: &AuthContext) -> Result<(), HandlerError>;
}
```
- `alpn()` returns a static byte string — the handler's ALPN identifier
- `handle()` receives a `Connection` (revised by ADR-005 from the original
`BiStream`) and an `AuthContext` carrying the authenticated identity, and
returns `HandlerError` on failure
- Every handler manages its own wire format — no shared framing, no StreamInterface/MessageInterface split
- The `ListenerConfig` enum is eliminated — ALPN advertisement configuration replaces it
**AuthContext resolution is hybrid** (see ADR-003, OQ-02 resolution): the endpoint resolves what it can before calling `handle()` (e.g., TLS client certificate fingerprint), and the handler resolves what it must inside `handle()` (e.g., AuthToken in the first frame of a call stream). The `AuthContext` passed to `handle()` may contain partial identity information — the handler is responsible for completing authentication if the endpoint didn't have enough information.
## Consequences
**Positive:**
- One trait, one dispatch point — eliminates the StreamInterface/MessageInterface split and ListenerConfig enum
- Each handler owns its wire format — no shared framing assumptions that constrain protocol design
- Adding a new protocol is implementing one trait with two methods
- Testable in isolation — give a handler a mock BiStream and AuthContext
- WASM-compatible in principle — handlers that don't need tokio runtime features compile to WASM
**Negative:**
- Every handler must implement its own framing — no shared "read a length-prefixed message" utility (mitigated: common utilities can live in alknet-core without mandating their use)
- Handlers that want message semantics must build them (mitigated: alknet-call provides this as a handler, not a mandatory layer)
- AuthContext resolution is hybrid — the endpoint resolves what it can (TLS-level auth), but handlers that need protocol-level credential extraction must do so inside handle(). This means AuthContext may be partial when handle() is called. Handlers must not assume AuthContext is fully resolved.
## References
- Pivot proposal: `docs/research/pivot/alpn-service-architecture.md`
- ADR-001: ALPN-based protocol dispatch
- ADR-003: Auth as shared core (IdentityProvider)
- ADR-005: BiStream type definition (revised this ADR's signature from BiStream to Connection)
- iroh ProtocolHandler pattern: `docs/research/references/iroh/`
- Replaces StreamInterface, MessageInterface, and ListenerConfig

76
docs/architecture/decisions/003-auth-as-shared-core.md Normal file
View File

@@ -0,0 +1,76 @@
# ADR-003: Auth as Shared Core (IdentityProvider)
## Status
Accepted
## Context
The previous architecture had authentication spread across multiple layers: `CredentialProvider` with four phases (A–D), `AuthProtocol` as an irpc service, `server_auth` and `client_auth` as separate modules, and `IdentityProvider` as a trait in alknet-core. Different interface types presented credentials differently — SSH used key fingerprints, HTTP used Bearer tokens, DNS used query labels — but the resolution was ad-hoc and tied to the three-layer model.
The ALPN dispatch model simplifies this: every handler receives the same `AuthContext`, but the credential extraction (how a handler learns who the peer is) differs per ALPN. The resolution (turning a credential into an `Identity`) should be shared across all handlers.
## Decision
> **Note**: The original text of this decision described the handler
> "enriching or replacing" the `AuthContext`. This was superseded by
> ADR-006, which made `AuthContext` immutable in `handle()` (passed as
> `&AuthContext`). Handlers resolve identity into a local variable and
> store it on `Connection` via `set_identity()`. The text below has been
> updated to reflect the ADR-006 model.
Authentication and identity resolution live in `alknet-core` as shared infrastructure. Each handler presents credentials differently, but all resolve through the same `IdentityProvider`:
```rust
pub trait IdentityProvider: Send + Sync + 'static {
fn resolve_from_fingerprint(&self, fingerprint: &str) -> Option<Identity>;
fn resolve_from_token(&self, token: &AuthToken) -> Option<Identity>;
}
```
Credential presentation per handler:
| Handler | Credential presentation | Resolves via |
|---------|------------------------|-------------|
| SshAdapter | SSH public key handshake | `resolve_from_fingerprint()` |
| CallAdapter | AuthToken in first frame | `resolve_from_token()` |
| HttpAdapter | `Authorization: Bearer` header | `resolve_from_token()` |
| DnsAdapter | AuthToken in query labels | `resolve_from_token()` |
| WebTransportAdapter | AuthToken in CONNECT headers | `resolve_from_token()` |
| GitAdapter | Signed push certificate | `resolve_from_fingerprint()` |
Auth resolution is **hybrid** — the endpoint resolves what it can, and handlers resolve what they must:
1. **Endpoint-level resolution** (before `handle()` is called): If the TLS handshake provides a client certificate, the endpoint resolves the fingerprint to an `Identity` and passes it in `AuthContext`. This is the case for SSH (where the key exchange happens at the protocol level, but the TLS layer may also provide information).
2. **Handler-level resolution** (inside `handle()`): For protocols that carry credentials in application frames (AuthToken in the first call frame, Bearer header in HTTP), the handler extracts the credential from the stream and calls `IdentityProvider` to resolve it. The handler then resolves the `Identity` into a local variable and stores it on the `Connection` via `set_identity()` for observability — it does **not** mutate the `AuthContext` (which is passed as `&AuthContext`, an immutable reference — see ADR-006). The per-request identity (for ACL) is resolved separately by the `CallAdapter` at `call.requested` time.
The `AuthContext` passed to `handle()` may be partial — containing only transport-level information if no TLS client certificate was provided. Handlers must not assume `AuthContext` contains a fully resolved `Identity`. Each handler knows its own credential extraction protocol and is responsible for completing authentication.
The `CredentialProvider` concept from the previous architecture is simplified: there is no phase progression (A–D). The `IdentityProvider` has two resolution paths — fingerprint and token — and a `ConfigIdentityProvider` implementation that draws from static and dynamic config.
`alknet-vault` stays standalone. It does not depend on `alknet-core` or `IdentityProvider`. The vault provides derived keys on request; identity resolution is a separate concern.
## Consequences
**Positive:**
- Unified identity model — every handler resolves identities the same way through `IdentityProvider`
- Handlers own their credential extraction — SSH reads key fingerprints, call reads AuthTokens, HTTP reads Bearer headers
- Endpoint provides what it can for free (TLS-level auth), handlers complete what they need
- Adding a new credential type is adding a method to `IdentityProvider`, not a new phase
- alknet-secret stays standalone — no coupling between key derivation and identity resolution
- `AuthContext` is a value type — easy to construct in tests, can be partial for handler-level testing
**Negative:**
- `IdentityProvider` is in alknet-core — any change to it recompiles all handlers (mitigated: the trait should be stable; implementation changes don't force recompiles)
- Two resolution paths (fingerprint, token) may not cover all future auth schemes (mitigated: the trait can be extended, or a handler can do custom resolution after the initial AuthContext)
- Handlers must handle partial AuthContext — the endpoint may not have resolved an Identity, so handlers must be prepared to do credential extraction themselves
- WebTransport and browser-based auth needs careful design — AuthToken in CONNECT headers requires the token to be available before the stream is established
## References
- Pivot proposal: `docs/research/pivot/alpn-service-architecture.md`
- ADR-002: ProtocolHandler trait
- ADR-031: Crate decomposition
- ADR-013: irpc as call protocol foundation
- The previous architecture had equivalent decisions in ADR-016 (unified auth) and ADR-024 (identity as core type), which are archived in the reference implementation at `/workspace/@alkdev/alknet-main/`.

71
docs/architecture/decisions/004-alpn-convention-and-connection-model.md Normal file
View File

@@ -0,0 +1,71 @@
# ADR-004: ALPN String Convention and Connection Model
## Status
Accepted
## Context
ADR-001 establishes ALPN-based protocol dispatch. Two questions arise:
1. **ALPN string naming**: What format do custom ALPN strings follow? Should they include version numbers? How do standard ALPNs (`h2`, `http/1.1`, `h3`) coexist with custom ones?
2. **Connection model**: ALPN is negotiated per-connection in QUIC/TLS, not per-stream. A client that wants to speak both SSH and call protocol must open two separate QUIC connections, each with its own ALPN. This is different from the claim in earlier drafts that "a single connection can carry multiple protocols via additional streams" — it cannot. However, QUIC connections are cheap (multiplexed over the same UDP flow), so opening multiple connections is acceptable.
The iroh reference project uses the same model: each `ProtocolHandler` claims an ALPN, and each incoming connection is dispatched to exactly one handler based on the negotiated ALPN.
## Decision
### ALPN String Convention
Custom ALPN strings use the `alknet/` prefix:
| ALPN | Handler | Type |
|------|---------|------|
| `alknet/ssh` | SshAdapter | Custom |
| `alknet/call` | CallAdapter | Custom |
| `alknet/git` | GitAdapter | Custom |
| `alknet/sftp` | SftpAdapter | Custom |
| `alknet/msg` | MessageAdapter | Custom |
| `alknet/http` | HttpAdapter | Custom |
| `alknet/dns` | DnsAdapter | Custom |
| `h3` | WebTransport → alknet/http | Standard (IANA) |
| `h2` | HTTP/2 → alknet/http | Standard (IANA) |
| `http/1.1` | HTTP/1.1 → alknet/http | Standard (IANA) |
Rules:
- Custom ALPNs use the format `alknet/<name>` — lowercase, no version number
- Standard ALPNs (`h2`, `http/1.1`, `h3`) use their IANA-registered strings and are handled by the HTTP adapter
- No version numbers in ALPN strings initially. If protocol compatibility breaks, a new ALPN string is registered (e.g., `alknet/call/v2`). This is simpler than version negotiation and follows the QUIC convention that ALPN mismatch means connection failure
- ALPN strings are compile-time constants in each handler's `alpn()` method — no runtime registration of new ALPN strings
### Connection Model
**One ALPN per connection.** A client that wants to use multiple ALPNs opens one QUIC connection per ALPN. All connections from the same client are multiplexed over the same UDP flow (QUIC's natural connection multiplexing), so the overhead is minimal.
This means:
- `alknet/call` is a distinct ALPN with its own connection — not a multiplexer for other ALPNs
- A client interacting with both SSH and call protocol has two QUIC connections
- Within an `alknet/call` connection, multiple QUIC streams can carry independent operations (see ADR-013)
- The endpoint logs the negotiated ALPN for each connection for observability
## Consequences
**Positive:**
- Simple model: one connection, one protocol — no multiplexing layer needed inside a connection
- ALPN strings are predictable and discoverable — `alknet/<name>` is a clear namespace
- No version negotiation complexity — incompatible versions get new ALPN strings
- QUIC connection multiplexing means multiple ALPN connections share the same UDP flow
**Negative:**
- Multiple ALPNs require multiple connections — a full-featured client might have 3-5 QUIC connections open simultaneously
- No version negotiation — an incompatible change requires a new ALPN string, which means old and new clients can coexist only if the server registers both ALPNs
- The `alknet/` namespace is owned by this project — third-party extensions need their own prefix
## References
- ADR-001: ALPN-based protocol dispatch
- ADR-002: ProtocolHandler trait
- OQ-03: ALPN string naming convention (resolved by this ADR)
- OQ-06: Server-side ALPN vs client-side ALPN (resolved by this ADR)
- iroh reference: `docs/research/references/iroh/`

169
docs/architecture/decisions/005-bistream-type-definition.md Normal file
View File

@@ -0,0 +1,169 @@
# ADR-005: BiStream Type Definition
## Status
Accepted
## Context
OQ-01 asked whether BiStream should be a concrete type wrapping quinn's `SendStream` + `RecvStream`, a trait `BiStream: AsyncRead + AsyncWrite + Send + Unpin`, or a type alias/newtype. This is a one-way door decision: if BiStream is a concrete type bound to a specific QUIC library, WASM targets and alternative transports cannot implement it. If it's a trait, the door stays open.
### iroh's pattern
iroh's `ProtocolHandler::accept` receives a `Connection`, not a stream. The handler calls `connection.accept_bi()` to get `(SendStream, RecvStream)` pairs. This means iroh's handlers own the entire connection lifecycle and can open/accept multiple streams.
### Alknet's pattern differs
Alknet's handlers are different from iroh's for two reasons:
1. **One ALPN per connection** (ADR-004). An incoming connection is already dispatched to exactly one handler by ALPN. The handler receives the connection and can manage streams however it wants.
2. **Some handlers need connection-level ownership**. SSH multiplexes channels over multiple streams within a single connection. The call protocol opens a new stream per operation. These handlers need the connection, not just a single stream.
### WASM constraint
If alknet-core defines BiStream as `quinn::SendStream + quinn::RecvStream` joined via `tokio::io::join`, then:
- WASM targets cannot implement it (quinn doesn't compile to WASM)
- WebTransport clients in browsers cannot participate as full peers
- The cost of making BiStream a trait later would require changing every handler's signature
If BiStream is a trait, WASM targets implement it over WebTransport streams. Native targets implement it over quinn streams. The cost is minimal — a trait vs a concrete type adds a small amount of indirection and trait object overhead that is negligible compared to I/O latency.
### Testing constraint
A BiStream trait allows test implementations (in-memory channels, mock streams) without requiring a running QUIC connection. A concrete quinn type requires mocking at a higher level (connection mocking) which is more complex.
## Decision
### BiStream is a trait
```rust
pub trait BiStream: AsyncRead + AsyncWrite + Send + Unpin {}
```
Handlers receive a `Connection` (not a single BiStream) in their `handle` method. This differs from the original ADR-002 signature and aligns with iroh's proven pattern.
### Revised ProtocolHandler signature
```rust
#[async_trait]
pub trait ProtocolHandler: Send + Sync + 'static {
fn alpn(&self) -> &'static [u8];
async fn handle(&self, connection: Connection, auth: &AuthContext) -> Result<(), HandlerError>;
}
```
Where `Connection` wraps a QUIC connection (or, in test contexts, a mock) and provides:
```rust
pub struct Connection {
// Private: wraps the underlying QUIC connection or test mock
}
impl Connection {
pub async fn accept_bi(&self) -> Result<(SendStream, RecvStream), StreamError>;
pub async fn open_bi(&self) -> Result<(SendStream, RecvStream), StreamError>;
pub fn remote_alpn(&self) -> &[u8];
// Additional methods as needed: close, remote_addr, etc.
}
```
`SendStream` and `RecvStream` are concrete types that implement `AsyncWrite` and `AsyncRead` respectively. They wrap the underlying QUIC stream types.
### Why Connection, not BiStream, as the handler parameter
The original ADR-002 specified `handle(&self, stream: BiStream, auth: &AuthContext)`. This was modeled on the idea that a handler receives a single bidirectional stream. But:
- **SSH** needs to open/accept multiple streams (channels) on one connection
- **Call protocol** opens a new stream per operation
- **HTTP** maps requests to streams within an HTTP/2 or HTTP/3 connection
- **iroh** already uses this pattern successfully
Passing a single BiStream would force handlers that need multiple streams to somehow obtain the Connection through other means, which is awkward. Passing the Connection directly is simpler and more flexible.
Handlers that only need a single stream (simple protocols) call `connection.accept_bi().await` once and work with that stream. Handlers that need multiple streams (SSH, call) use the Connection to open/accept as needed.
### Why BiStream is still defined as a trait
Even though handlers receive a `Connection` rather than a single `BiStream`, the BiStream trait is still useful:
1. **Client-side**: A client connecting to an alknet endpoint needs a way to represent "I have a bidirectional stream to speak my protocol on." That stream should be implementable over WebTransport in WASM.
2. **Testing**: Mock BiStream implementations for unit tests.
3. **Portability**: If alknet later supports transports other than QUIC (raw TCP, iroh P2P), those transports need to produce BiStream-compatible streams.
The BiStream trait is a thin convenience — `AsyncRead + AsyncWrite + Send + Unpin` — that can be implemented by any byte transport. It does not mandate tokio or quinn.
## Consequences
**Positive:**
- WASM door stays open: browser clients can implement BiStream over WebTransport streams
- Testing is straightforward: mock BiStream implementations without QUIC infrastructure
- Handlers that need multiple streams (SSH, call) have direct access to the Connection
- Handlers that need a single stream call `accept_bi()` once — simple case stays simple
- Aligns with iroh's proven ProtocolHandler pattern
- Alternative transports (TCP, iroh P2P) can implement Connection and BiStream traits
**Negative:**
- Slight runtime overhead from trait dispatch vs concrete types (negligible compared to I/O)
- Two concepts (Connection and BiStream) instead of one (BiStream alone) — more types in alknet-core
- ADR-002's `handle` signature changes from `(BiStream, AuthContext)` to `(Connection, AuthContext)` — this is a revision to the original trait signature
- Handlers must call `accept_bi()` explicitly even for simple protocols — one additional line of code per handler
## References
- ADR-002: ProtocolHandler trait (signature revised by this ADR)
- ADR-031: Crate decomposition
- ADR-004: ALPN string convention and connection model
- OQ-01: BiStream type definition (resolved by this ADR)
- iroh ProtocolHandler pattern: `docs/research/references/iroh/iroh/`
- Pivot proposal: `docs/research/pivot/alpn-service-architecture.md`
## Amendments
### Amendment 1 (2026-07-09): `Connection::from_stream` opens the server-side door
This ADR's "WASM constraint" section argued that if `Connection` (or
BiStream) were bound to a QUIC library, WASM targets and alternative
transports couldn't implement it — and that a trait-based `BiStream`
preserves the *client-side* door. That argument was correct for `BiStream`
(the trait) but incomplete for `Connection`: until ADR-007, `Connection`
was a concrete type with only QUIC variants (`ConnectionKind::Quinn` /
`ConnectionKind::Iroh`), plus a `ConnectionKind::Mock` test stub. There was
no way to construct a `Connection` from a non-QUIC stream, which meant
TCP+TLS, SSH channels, WebTransport streams, and wasm streams could not
be dispatched through the `HandlerRegistry` — the *server-side* dispatch
door was closed.
**Resolved by [ADR-007](007-connection-from-stream-generic-single-stream.md):**
`Connection::from_stream(send, recv, alpn, remote_addr)` and
`Connection::from_bidi(stream, alpn, remote_addr)` construct a `Connection`
from any `AsyncRead + AsyncWrite` pair. A new `ConnectionKind::Stream`
variant holds a single read/write pair behind a `Mutex<Option<...>>` with a
yield-once `accept_bi` contract (QUIC yields many streams; everything else
yields one, then `ConnectionClosed`). Every existing `ProtocolHandler`
works over the new kind **unchanged** — handlers that loop `accept_bi`
(TtyAdapter) get one iteration; handlers that call once (HttpAdapter) get
the stream directly. Both correct, no branching on transport.
The stream-level `SendStreamKind::Mock` / `RecvStreamKind::Mock` variants
(already generic `Box<dyn AsyncRead/Write>` — the name was wrong) are
renamed to `Stream` and made load-bearing: `from_stream` calls
`SendStream::from_stream` / `RecvStream::from_stream`.
### Amendment 2 (2026-07-09): `MockConnection` / `ConnectionKind::Mock` removed
This ADR's Decision section and the `Connection` sketch referenced "test
mock" as one of the things `Connection` wraps. The implementation had a
`MockConnection` trait and `ConnectionKind::Mock` variant for test-only
full-connection mocks. ADR-007 removed both entirely: test stubs now use
`Connection::from_stream(tokio::io::sink(), tokio::io::empty(), alpn,
addr)` — `tokio::io::empty()` yields immediate EOF on read (handler exits
cleanly), and `accept_bi` returns `ConnectionClosed` after the first take
(run loop exits). One connection kind for production and tests, not two.
The "test mock" concept this ADR references is now subsumed by
`from_stream` — a test connection is just a single-stream connection with
EOF-on-read.
The server-side WASM door (OQ-09) is no longer closed by `Connection` being
QUIC-bound — `from_stream` accepts any `AsyncRead + AsyncWrite`, including
wasm-compatible streams. See OQ-09 for the updated resolution.

156
docs/architecture/decisions/006-authcontext-structure.md Normal file
View File

@@ -0,0 +1,156 @@
# ADR-006: AuthContext Structure and Resolution Flow
## Status
Accepted
## Context
ADR-003 establishes the hybrid auth model: the endpoint resolves what it can (TLS client certificate fingerprint), handlers resolve what they must (AuthToken in the first frame, Bearer header, SSH key fingerprint). The `AuthContext` passed to `handle()` may be partial.
The reference implementation's `Identity` struct is:
```rust
pub struct Identity {
pub id: String,
pub scopes: Vec<String>,
pub resources: HashMap<String, Vec<String>>,
}
```
And `ConfigIdentityProvider` resolves fingerprints and API keys to `Identity`. This works well and carries forward.
But the reference implementation has no `AuthContext` type — auth resolution happens inside the SSH handler before calling `IdentityProvider`. The new model needs a type that represents "what the endpoint knows about this connection's identity before the handler starts," plus a way for handlers to enrich it.
This is a one-way door: once handlers depend on `AuthContext`'s structure, changing it affects every handler. The structure must be right.
### Design considerations
1. **Handlers need identity information to make authorization decisions.** A handler that requires authentication needs to know: is the peer authenticated? Who are they? What scopes do they have?
2. **The endpoint may have zero, partial, or complete identity information.** A plain QUIC connection with no TLS client cert gives the endpoint nothing. A TLS connection with a client cert gives the endpoint a fingerprint that may resolve to an Identity. A handler that extracts an AuthToken from the first frame can complete the resolution.
3. **AuthContext must not be SSH-specific.** The reference implementation's auth types are tangled with russh (SSH key fingerprints, certificate authorities). The new model needs to be ALPN-agnostic.
4. **AuthContext is constructed by the endpoint and enriched by handlers.** The endpoint creates it from TLS-level information. The handler mutates or replaces it with protocol-level information.
5. **AuthContext must be cheap to construct.** Every incoming connection gets one, even if authentication ultimately fails.
## Decision
### AuthContext is a struct with optional fields
```rust
pub struct AuthContext {
/// The peer's authenticated identity, if resolved.
/// None means the endpoint has no identity information for this connection.
/// Some(Identity) means the endpoint resolved the peer's identity.
pub identity: Option<Identity>,
/// The negotiated ALPN for this connection.
/// Always present — the endpoint sets this from the TLS handshake.
pub alpn: Vec<u8>,
/// The peer's remote address, if available.
pub remote_addr: Option<SocketAddr>,
/// TLS client certificate fingerprint, if the client presented a certificate.
/// Set by the endpoint during TLS handshake. Handlers may use this for
/// SSH host key verification or other fingerprint-based auth.
pub tls_client_fingerprint: Option<String>,
}
```
Key design points:
- `identity: Option<Identity>` — not `Identity` with optional fields, not a separate `PartialAuthContext`. The endpoint sets it to `None` if it has no identity information, or `Some(identity)` if it resolved one. Handlers that need to complete auth call `IdentityProvider` themselves and store the resolved identity in a local variable — they do NOT mutate AuthContext (see immutability section below).
- `alpn` is always present — every connection has a negotiated ALPN.
- `remote_addr` is informational. It's available from the QUIC connection and useful for logging and rate limiting, but it's not authoritative (clients can be behind NATs/proxies).
- `tls_client_fingerprint` captures the TLS-level credential. If present, it's the SHA-256 fingerprint of the client's TLS certificate. This is separate from `identity` because a handler might need the fingerprint even when `IdentityProvider::resolve_from_fingerprint()` returns `None` (e.g., unknown cert, but the handler wants to log it).
### AuthContext is Clone
`AuthContext` derives `Clone`. Handlers can clone it for per-stream or per-channel contexts within a connection. The `Identity` inside is also `Clone`.
### Handler-level auth enrichment pattern
Handlers that need to complete authentication do so inside `handle()`:
```rust
async fn handle(&self, connection: Connection, auth: &AuthContext) -> Result<(), HandlerError> {
let identity = if let Some(id) = &auth.identity {
id.clone() // Endpoint already resolved identity
} else {
// Extract credentials from the protocol, resolve via IdentityProvider
let token = self.extract_auth_token(&connection).await?;
self.identity_provider.resolve_from_token(&token)
.ok_or(HandlerError::AuthRequired)?
};
// ... proceed with authenticated identity
}
```
Handlers that don't need authentication (e.g., DNS resolver, health check) can ignore `auth.identity` entirely.
### Identity carries over from reference implementation
```rust
pub struct Identity {
pub id: String,
pub scopes: Vec<String>,
pub resources: HashMap<String, Vec<String>>,
}
```
This is the same structure from the reference implementation, minus the russh dependency. It's ALPN-agnostic:
- `id`: A unique identifier string. For SSH key auth, this is the SHA-256 fingerprint. For API key auth, this is the key prefix. For certificate auth, this is the principal name.
- `scopes`: Authorization scopes. `["relay:connect", "secrets:derive"]` etc.
- `resources`: Named resource lists. `{"service": ["gitea", "registry"]}` etc.
### AuthToken carries raw bytes
```rust
pub struct AuthToken {
pub raw: Vec<u8>,
}
```
Unchanged from the reference implementation. Opaque bytes — the handler that extracted it knows its encoding.
### IdentityProvider carries over with minor adaptation
```rust
pub trait IdentityProvider: Send + Sync + 'static {
fn resolve_from_fingerprint(&self, fingerprint: &str) -> Option<Identity>;
fn resolve_from_token(&self, token: &AuthToken) -> Option<Identity>;
}
```
The implementation (`ConfigIdentityProvider`) changes from the reference: it no longer depends on russh types for key storage. Instead, it stores fingerprint strings and API key entries, drawing from `DynamicConfig` via `ArcSwap`.
### AuthContext is NOT mutable inside handle()
The `handle()` signature passes `&AuthContext` (immutable reference). Handlers that resolve identity create a local variable with the resolved identity — they don't mutate the AuthContext. This prevents accidental cross-contamination between streams on the same connection.
## Consequences
**Positive:**
- `AuthContext` is a value type — cheap to construct, clone, and pass around
- Handlers that don't need auth can ignore it entirely
- The endpoint provides what it can for free (TLS client cert fingerprint), handlers complete what they need
- No russh dependency in AuthContext — it's ALPN-agnostic
- `Option<Identity>` is explicit — there's no "partially authenticated" state that handlers have to interpret
- Handlers that need to enrich auth create local variables, not mutation — clean data flow
**Negative:**
- Handlers that need auth must call `IdentityProvider` themselves — this is intentional (ADR-003 hybrid model) but means each handler has its own auth extraction logic
- `tls_client_fingerprint` is separate from `identity` — a handler might wonder "why do I have a fingerprint but no identity?" This happens when the client presents a cert that's not in the authorized keys. The handler can log the fingerprint for debugging.
- `AuthContext` doesn't carry protocol-specific auth state (e.g., SSH auth method, HTTP auth scheme). This is by design — protocol-specific details belong inside the handler, not in the shared auth context.
## References
- ADR-002: ProtocolHandler trait
- ADR-003: Auth as shared core (IdentityProvider, hybrid auth model)
- ADR-005: BiStream type definition (Connection parameter)
- ADR-010: ALPN router and endpoint (where AuthContext is created)
- Reference implementation: `alknet-main/crates/alknet-core/src/auth/identity.rs`

236
docs/architecture/decisions/007-connection-from-stream-generic-single-stream.md Normal file
View File

@@ -0,0 +1,236 @@
# ADR-007: Connection::from_stream — Generic Single-Stream Connections
## Status
Accepted
## Context
ADR-005 defines `Connection` as a concrete type wrapping a QUIC connection
(quinn or iroh). ADR-010 establishes the endpoint as a multi-connectivity
QUIC acceptor — quinn and iroh, both producing QUIC connections dispatched by
ALPN. The `ProtocolHandler` trait (ADR-002) receives a `Connection`, and
handlers call `accept_bi()` / `open_bi()` to get bidirectional streams.
This design is **welded to QUIC**. Both real `ConnectionKind` variants
(`Quinn`, `Iroh`) are QUIC. The `HttpAdapter::handle` method calls
`connection.accept_bi().await` to get a bidi stream and serves HTTP over it
— "HTTP over QUIC," not "HTTP over TCP+TLS." There is no way to serve the
standard HTTP interface that `api.alk.dev` (an external app being built
against the crates) requires without either bypassing the
`HandlerRegistry` (a parallel listener, defeating the ALPN-router design)
or generalizing `Connection` to accept a non-QUIC stream.
The same welding blocks `alknet-ssh` (needs to dispatch SSH channels —
each channel is a read/write pair — through the same `HandlerRegistry` as
QUIC connections) and WebTransport stream dispatch (each WT stream is a
read/write pair). The `TtyAdapter` and `CallAdapter` dispatch loops are
already transport-agnostic in their inner logic — only the
`connection.accept_bi()` call is QUIC-coupled, because `accept_bi` only
works when `Connection` is QUIC-kind.
### The yield-once contract composes
QUIC's `accept_bi` returns a new bidi stream per call (many). A generic
single-stream connection's `accept_bi` returns the underlying stream on the
first call, then `ConnectionClosed` on all subsequent calls. This is the
contract that makes the abstraction compose:
- Handlers that loop `accept_bi` (TtyAdapter) get one session per
single-stream connection — the loop body runs once, then
`ConnectionClosed` breaks the loop. Correct.
- Handlers that call `accept_bi` once (HttpAdapter) get the stream
directly. Correct.
No branching on transport. The handler code is unchanged across QUIC
(many streams) and TCP+TLS / SSH channels / WebTransport streams (one
stream). The `ProtocolHandler` trait shape is not touched — this is an
additive change to `Connection`, not a trait revision.
### The stream-level Mock variants were already generic
`SendStreamKind::Mock(Box<dyn AsyncWrite>)` and `RecvStreamKind::Mock(Box<dyn
AsyncRead>)` were already generic stream holders — the name was wrong
(carried over from a test-only context). The generalization renames them
to `Stream` and makes them load-bearing: `Connection::from_stream` calls
`SendStream::from_stream` / `RecvStream::from_stream` to wrap the halves of
the single stream.
### The connection-level Mock is removed
The findings doc (`docs/research/transport-generalization/findings.md`)
proposed keeping `ConnectionKind::Mock` / `MockConnection` for test-only
full-connection mocks. The implementation went further: `MockConnection`
and `ConnectionKind::Mock` are removed entirely. Test stubs that
previously used `Connection::from_mock(Arc<StubConnection>)` now use
`Connection::from_stream(tokio::io::sink(), tokio::io::empty(), alpn,
addr)` — `tokio::io::empty()` yields immediate EOF on the read side,
causing the handler's `handle_stream` to exit cleanly, and `accept_bi`
returns `ConnectionClosed` after the first take (driving the run loop to
exit). This is simpler (one connection kind, not two) and the test stubs
are shorter. `from_stream` subsumes the test-mock use case because a test
connection is just a single-stream connection with EOF-on-read.
### Why not change the ProtocolHandler trait
An earlier analysis proposed changing `ProtocolHandler::handle` to take a
single `Channel` instead of a `Connection`, moving the multiplexing loop
from the handler to the endpoint. This ADR does **not** do that:
1. **TtyAdapter already establishes the pattern.** The handler loops
`accept_bi` and dispatches each stream internally. SSH does the same —
parse channels, dispatch each. The multiplexing loop belongs in the
handler, not the endpoint.
2. **The trait shape is a one-way door (ADR-032).** Changing
`handle(Connection)` → `handle(Channel)` would require migrating every
handler and would lock in a specific multiplexing model. `from_stream`
is additive — it extends `Connection` without touching the trait. If a
trait change is ever warranted, it can come later; `from_stream` doesn't
preclude it.
See `docs/research/transport-generalization/findings.md` §6 for the full
argument against the trait shape change.
## Decision
### Add `ConnectionKind::Stream`
A new variant holding a single read/write pair behind a
`Mutex<Option<(SendStream, RecvStream)>>` — the yield-once semantic. No
feature gate (generic, no transport deps). `StreamConn` is always
available; the quinn/iroh variants remain feature-gated.
### Add `Connection::from_stream` and `Connection::from_bidi`
```rust
/// Construct a Connection from a pre-split read/write pair.
/// `accept_bi()` yields this pair once, then returns `ConnectionClosed`.
/// `open_bi()` returns `StreamClosed` (a single stream can't open new streams).
pub fn from_stream(
send: impl AsyncWrite + Send + Unpin + 'static,
recv: impl AsyncRead + Send + Unpin + 'static,
alpn: Vec<u8>,
remote_addr: Option<SocketAddr>,
) -> Self;
/// Convenience for a single bidirectional stream (e.g. TlsStream<TcpStream>).
/// Splits internally via tokio::io::split.
pub fn from_bidi(
stream: impl AsyncRead + AsyncWrite + Send + Unpin + 'static,
alpn: Vec<u8>,
remote_addr: Option<SocketAddr>,
) -> Self;
```
### Make `accept_bi`'s yield-once contract explicit
The `accept_bi` doc comment now states the transport semantics: QUIC yields
many streams, single-stream yields once then `ConnectionClosed`. This is
the contract that makes the abstraction compose — handlers don't branch
on transport.
### Rename stream-level `Mock` → `Stream`
`SendStreamKind::Mock` → `SendStreamKind::Stream`,
`RecvStreamKind::Mock` → `RecvStreamKind::Stream`.
`SendStream::from_mock` → `from_stream`, `RecvStream::from_mock` →
`from_stream`. The variants were already generic stream holders; the name
was wrong. Drop the `#[allow(dead_code)]` — `from_stream` is now
load-bearing.
### Remove `MockConnection` / `ConnectionKind::Mock`
The connection-level test mock trait and variant are removed. Test stubs
use `Connection::from_stream` with `tokio::io::sink()` / `tokio::io::empty()`
(immediate EOF on read → handler exits cleanly → `accept_bi` returns
`ConnectionClosed` → run loop exits). One connection kind for both
production and tests, not two.
### `open_bi` on `Stream` returns `StreamClosed`
A single stream cannot open new application streams. `open_bi` on
`ConnectionKind::Stream` returns `StreamError::StreamClosed`. Handlers that
call `open_bi` (the call protocol's server→client direction) work over
QUIC but not over a single-stream connection — this is inherent to the
transport, not a flaw. A handler that needs `open_bi` should not be
dispatched over a single-stream connection (or should multiplex its own
sub-streams within the one stream, as the call protocol does over a single
WebTransport stream).
### What does NOT change
- `ProtocolHandler` trait shape — `handle(&self, connection: Connection,
auth: &AuthContext)` stays. This is an additive change to `Connection`,
not a trait revision (ADR-032: the trait is a one-way door).
- `HandlerRegistry` — unchanged.
- All handler code (`HttpAdapter`, `TtyAdapter`, `CallAdapter`) —
unchanged. `HttpAdapter` is one `accept_bi` call away from
transport-agnostic (it already is — the call works over `from_stream`).
- `BiStream` trait — unchanged (ADR-005). `from_stream` is a server-side
connection constructor; `BiStream` is a client-side/test convenience
trait. They're complementary, not competing.
- The endpoint's accept loops (quinn/iroh) — unchanged. The TCP+TLS accept
loop that *uses* `from_stream` is a follow-up, not this ADR.
## Consequences
**Positive:**
- Every existing `ProtocolHandler` works over TCP+TLS, SSH channels,
WebTransport streams, and wasm streams **unchanged** — dispatch through
the same `HandlerRegistry` by ALPN string, no handler code changes.
- `api.alk.dev`'s HTTP blocker is resolved: a TCP+TLS accept loop can call
`Connection::from_bidi(tls_stream, alpn, remote_addr)` and dispatch
through `HandlerRegistry` — `HttpAdapter` works unchanged over the
single stream. (The accept loop itself is a follow-up commit; the
primitive it needs is now in place.)
- `alknet-ssh` is unblocked: the SSH handler wraps each russh channel via
`from_stream` and dispatches by channel-type (treated as the ALPN string)
through `HandlerRegistry`. One SSH connection carries heterogeneous
channels (`alknet/tty`, `alknet/call`, `h2`, ...) — a multiplexing power
QUIC's per-connection ALPN doesn't give natively.
- WebTransport stream dispatch is unblocked: the WT handler wraps each WT
stream via `from_stream` and dispatches through `HandlerRegistry` (the
primitive exists; WT itself is parked per ADR-044).
- The server-side WASM door (OQ-09) is no longer closed by `Connection`
being QUIC-bound — `from_stream` accepts any `AsyncRead + AsyncWrite`,
including wasm-compatible streams. (The accept-loop runtime remains
tokio-bound; the *connection* door is now open.)
- One connection kind for production and tests (no `MockConnection`
trait) — simpler type, shorter test stubs.
- No new deps, no `Cargo.toml` change — `tokio::io::split` is already
available via the existing tokio dep.
**Negative:**
- `open_bi` on a single-stream connection returns `StreamClosed` —
handlers that need server→client stream initiation (the call protocol's
bidirectional call direction) don't work over a single-stream
connection. This is inherent to the transport, not a design flaw: a
single TCP+TLS stream is not a multiplexed transport. Handlers that need
`open_bi` should run over QUIC, or multiplex their own sub-streams within
the one stream (as the call protocol does over a single WebTransport
stream — the `EventEnvelope` framing is stream-agnostic, ADR-015).
- The `close()` method's `code`/`reason` args are QUIC-specific
(application-level close codes). For a raw stream they're ignored — the
drop is the close. This is the same best-effort semantic `close` already
had for the removed `Mock` variant.
- A `Mutex` on the `StreamConn` — a single lock per `accept_bi` / `close`
call. Negligible cost (one `take()`), but it is a lock where the QUIC
variants have none.
## References
- ADR-002: ProtocolHandler trait (unchanged by this ADR)
- ADR-005: BiStream type definition (amended by this ADR — `Connection` is
no longer QUIC-only; the server-side WASM door is open)
- ADR-032: One-way door decision framework (why the trait shape is not
changed — `from_stream` is additive)
- ADR-010: ALPN router and endpoint (amended by this ADR — "TCP is not an
endpoint concern" is revised; `from_stream` lets TCP+TLS participate in
ALPN dispatch via a handler-internal accept loop)
- ADR-015: Call protocol stream model (the `EventEnvelope` framing is
stream-agnostic — composes over `from_stream`)
- OQ-09: WASM target boundaries (resolution amended — the server-side
dispatch door is no longer closed by `Connection` being QUIC-bound)
- Transport generalization findings:
[`docs/research/transport-generalization/findings.md`](../../research/transport-generalization/findings.md)
- Implementation commit: `865fef6` (2026-07-09)

302
docs/architecture/decisions/008-bidistreamsource-trait.md Normal file
View File

@@ -0,0 +1,302 @@
# ADR-008: BidiStreamSource Trait — Open Connection for Extension
## Status
Accepted
## Context
ADR-007 generalized `Connection` beyond QUIC by adding
`ConnectionKind::Stream` (a yield-once read/write pair) and the
`Connection::from_stream` / `from_bidi` constructors. That closed the
server-side "QUIC-only" gap: TCP+TLS, SSH channels, WebTransport streams,
and wasm streams now dispatch through the same `HandlerRegistry` as QUIC
connections, unchanged.
What ADR-007 did **not** change is the *shape* of `Connection` itself. It
remains a closed enum:
```rust
enum ConnectionKind {
#[cfg(feature = "quinn")] Quinn(quinn::Connection),
#[cfg(feature = "iroh")] Iroh(iroh::endpoint::Connection),
Stream(StreamConn), // yield-once — ADR-007
}
```
Adding a new connection type today requires editing `alknet-core` — adding a
variant to `ConnectionKind`, adding match arms to `accept_bi` / `open_bi` /
`remote_addr` / `close`. Every downstream crate that introduces a new
connection shape (channels, a future transport, a test double beyond the
`from_stream` case) forces a core change. `Connection` is closed for
extension.
### The channels crate is the first crate that needs to extend it
The `alknet-channels` POC (`docs/research/alknet-channels/poc-summary.md`)
validated the channels multiplexer and surfaced the concrete blocker. A
channels connection carries N logical channels over one transport stream;
each channel is a bidirectional byte stream presented to a `ProtocolHandler`
as a `Connection`. With the ADR-007 shape, each channel becomes a fresh
yield-once `Connection::from_stream`, and the channels endpoint holds a *bag*
of these connections (one per channel) rather than one `ChannelConnection`
that yields N streams.
The POC confirmed this is *sufficient* (the yield-once path works — handlers
run unchanged) but *awkward*: the channels layer wants to expose a single
`ChannelConnection` that is a first-class peer of QUIC (many bidi streams),
not a collection of yield-once `Connection`s. The clean shape is for the
channels crate to implement the stream-yield interface itself, in its own
crate, without a core edit.
### The extension point is narrow and already implied by ADR-007
`Connection`'s public surface is four operations: `accept_bi`, `open_bi`,
`remote_addr`, `close`. `remote_alpn` / `set_identity` / `identity` are
`Connection`-level (not transport-level) and stay on `Connection` itself.
The four transport-level operations are the seam. Extracting them into a
trait that downstream crates can implement turns `Connection` from a closed
enum into an open trait object — the same extensibility `ProtocolHandler`
already gives handlers, applied to the connection.
### What the POC de-risked
The channels POC (28 tests passing) validated that:
1. The yield-once `Connection::from_stream` path is sufficient for per-channel
presentation — an echo `ProtocolHandler` runs through the full
demux→Connection→handler→mux path with zero channels-layer awareness
(`poc-summary.md` §"POC Target 2").
2. The `BidiStreamSource` trait is **additive** — existing callers keep working
via a `from_stream`-backed implementation of the trait, and the trait
cleanly supports a `ChannelConnection` that yields N streams
(`poc-summary.md` §"Issues Surfaced" #1).
3. The trait does not touch the `ProtocolHandler` trait shape (ADR-002) —
handlers continue to receive a `Connection` and call `accept_bi` /
`open_bi` on it. This is a `Connection` internal refactor, not a handler
API change (`poc-summary.md` §"POC Target 2").
The remaining unknowns are spec-scope (the channels crate's API), not
feasibility. This ADR makes the core-side extension point available so the
channels spec can build on it.
## Decision
### Extract `BidiStreamSource` trait
```rust
#[async_trait]
pub trait BidiStreamSource: Send + Sync + 'static {
/// Yield the next bidirectional stream this connection provides.
///
/// Transport semantics (carried from ADR-007):
/// - QUIC (quinn/iroh): returns a new bidi stream on each call,
/// `ConnectionClosed` when the underlying connection closes.
/// - Single-stream (TCP+TLS, SSH channel, WebTransport stream, wasm):
/// yields the underlying stream on the first call, then
/// `ConnectionClosed` on all subsequent calls.
/// - Channels: yields one bidi stream per channel, `ConnectionClosed`
/// when the channels connection closes.
async fn accept_bi(&self) -> Result<(SendStream, RecvStream), StreamError>;
/// Open a bidirectional stream to the peer.
///
/// Single-stream sources return `StreamClosed` (a single stream cannot
/// open new application streams — ADR-007). QUIC and channels sources
/// open new streams.
async fn open_bi(&self) -> Result<(SendStream, RecvStream), StreamError>;
/// The peer's address, if available. Informational (NAT/proxy).
fn remote_addr(&self) -> Option<SocketAddr>;
/// Close the connection. The `code`/`reason` args are QUIC application-
/// level close codes; non-QUIC sources ignore them (the drop is the
/// close — ADR-007 §"Negative"). See REQ-CORE-02 below for the
/// rationale for keeping the QUIC-shaped signature on the trait.
fn close(&self, code: u32, reason: &str);
}
```
### `Connection` holds `Box<dyn BidiStreamSource>`
```rust
pub struct Connection {
source: Box<dyn BidiStreamSource>,
alpn: Vec<u8>,
identity: OnceLock<Identity>,
}
```
`ConnectionKind` (the private enum) is replaced by the trait object. The
public `Connection` API (`accept_bi`, `open_bi`, `remote_alpn`, `remote_addr`,
`close`, `set_identity`, `identity`) is preserved verbatim — each method
delegates to `self.source`. `remote_alpn` reads `self.alpn` (unchanged).
`set_identity` / `identity` read/write `self.identity` (unchanged).
### Constructors stay; each wraps a `BidiStreamSource` impl
| Constructor | Wraps |
|-------------|-------|
| `from_quinn` / `from_quinn_with_alpn` (feature `quinn`) | `QuinnBidiStreamSource` (crate-private) |
| `from_iroh` (feature `iroh`) | `IrohBidiStreamSource` (crate-private) |
| `from_stream` / `from_bidi` (no feature gate) | `StreamBidiStreamSource` (crate-private, yield-once) |
| `from_source` (no feature gate) | caller-supplied `impl BidiStreamSource` — the extension point for downstream crates |
`from_source(source: impl BidiStreamSource, alpn: Vec<u8>) -> Self` is the
constructor that makes the trait the extension point. A downstream crate
implements `BidiStreamSource` (e.g. the channels crate's
`ChannelBidiStreamSource`) and constructs a `Connection` from it via
`from_source` — no core edit. The built-in impls (`QuinnBidiStreamSource`,
`IrohBidiStreamSource`, `StreamBidiStreamSource`) are crate-private;
`from_source` is the only path a downstream crate uses to wrap its own
impl. The `from_quinn` / `from_iroh` / `from_stream` / `from_bidi`
constructors are convenience wrappers for the three built-in impls.
The `Stream`-backend implementations are crate-private; downstream crates
do not implement `BidiStreamSource` by wrapping `from_stream`. They
implement the trait directly (channels: `ChannelBidiStreamSource`) and
construct the `Connection` via `from_source`.
### `from_stream`-backed default impl is the compatibility path
The yield-once `StreamBidiStreamSource` is the implementation that keeps
existing callers working: `Connection::from_stream(send, recv, alpn, addr)`
constructs a `Connection` backed by a `StreamBidiStreamSource` whose
`accept_bi` yields once then returns `ConnectionClosed`, whose `open_bi`
returns `StreamClosed`, whose `close` drops the stream. Behaviorally identical
to the ADR-007 `ConnectionKind::Stream` variant. No caller change.
### REQ-CORE-02: `close()` keeps the QUIC-shaped signature on the trait
The `close(&self, code: u32, reason: &str)` signature is preserved on the
trait, rather than being split into transport-specific close methods. This
resolves the ADR-007 leftover: the `Stream` backend's `close(code, reason)`
currently takes both args and uses neither, which clippy flags under
`--no-default-features` (the channels POC's build mode) as two unused
variable warnings on `crates/alknet-core/src/types.rs:500`.
Two options were considered:
- **(a) Split `close`**: `trait BidiStreamSource { fn close(&self); }` plus a
separate `fn close_with_code(&self, code: u32, reason: &str)` default-
implemented to call `close()`. Non-QUIC impls implement only `close()`;
QUIC impls override `close_with_code`. This moves the QUIC-shaped args off
the common method.
- **(b) Keep the QUIC-shaped signature on the trait**: `fn close(&self, code:
u32, reason: &str)`. Non-QUIC impls prefix the args with `_` and document
why they're ignored (the drop is the close — ADR-007). The trait method
matches the existing public `Connection::close` signature verbatim — no
caller change, no `Connection` API split.
**Decision: (b).** Rationale:
1. **No caller breakage.** `Connection::close(code, reason)` is the existing
public signature; every caller passes both args. Option (a) would force
either a `Connection::close` that *always* takes `code`/`reason` and
dispatches to the right trait method (which means the trait still has the
QUIC-shaped method, just renamed — no actual improvement), or a
`Connection::close` that drops the args (which breaks every caller).
2. **The args are not QUIC-only in principle.** WebTransport has
application-level close codes; a future transport may as well. The
signature `close(code, reason)` is a reasonable "close with diagnostic"
shape that multiple transports can use. Only raw-stream backends (the
ADR-007 `Stream` case) have nothing to do with the args, and they're the
degenerate case.
3. **The clippy warning is fixed by the trait, not by renaming.** Under the
trait, the `StreamBidiStreamSource::close` impl prefixes the args with
`_code`/`_reason` and carries a doc comment stating they're ignored
because the drop is the close. The warning disappears; the signature
matches the public API.
The trait method's doc comment carries the "QUIC application-level close
codes; non-QUIC sources ignore them" note from ADR-007, so implementers know
the args are optional for their transport.
### What does NOT change
- **`ProtocolHandler` trait shape** — `handle(&self, connection: Connection,
auth: &AuthContext)` stays. This is an internal `Connection` refactor, not
a handler API change (ADR-032: the handler trait is a one-way door).
- **`HandlerRegistry`** — unchanged.
- **All handler code** (`HttpAdapter`, `TtyAdapter`, `CallAdapter`,
`ChannelsAdapter`) — unchanged. They receive a `Connection` and call
`accept_bi` / `open_bi` on it. The dispatch through `Box<dyn
BidiStreamSource>` is transparent to them.
- **`SendStream` / `RecvStream`** — unchanged. They continue to wrap
quinn/iroh/generic-stream sources via their own internal enum dispatch.
`BidiStreamSource` implementations construct `SendStream` / `RecvStream`
via the existing `from_quinn` / `from_iroh` / `from_stream` constructors.
- **`BiStream` trait** — unchanged (ADR-005). `BidiStreamSource` is the
server-side / connection-level seam; `BiStream` is a client-side / test
convenience trait. Complementary, not competing.
- **The endpoint's accept loops** (quinn/iroh) — unchanged. They construct
`Connection::from_quinn` / `from_iroh`, which now internally wrap a
`QuinnBidiStreamSource` / `IrohBidiStreamSource`. The accept loops
themselves don't touch the trait.
- **`Connection::remote_alpn` / `set_identity` / `identity`** — unchanged.
These are `Connection`-level (the `alpn` field and the `identity` OnceLock),
not transport-level. They stay on `Connection` and do not appear on
`BidiStreamSource`.
## Consequences
**Positive:**
- `Connection` is open for extension. The channels crate implements
`ChannelBidiStreamSource` in its own crate and constructs `Connection`
from it via `from_source` — no core edit. A future transport, test
double, or relay connection follows the same path. This is the
structural payoff: the connection type is no longer a closed enum that
every new connection shape must edit.
- A channels connection is a first-class peer of QUIC: one
`ChannelConnection` that yields N streams, rather than a bag of yield-
once `Connection`s. The channels layer's API matches its actual shape.
- The ADR-007 leftover clippy warning (unused `code`/`reason` on the
`Stream` backend under `--no-default-features`) is resolved — the
`StreamBidiStreamSource::close` impl documents why the args are ignored,
and the `_` prefix is intentional, not a missing fix.
- Existing callers, handlers, and tests are unchanged. The public
`Connection` API is preserved verbatim; the refactor is internal.
- No new deps. `async_trait` is already a core dep (used by
`ProtocolHandler`).
**Negative:**
- One dyn-dispatch indirection per `accept_bi` / `open_bi` / `close` /
`remote_addr` call. The previous enum match was also a branch, so the cost
is roughly one `Box<dyn>` method call per stream operation — negligible
next to the async I/O those operations perform. The `alpn` / `identity`
fields stay on `Connection` (not behind the dyn), so `remote_alpn` /
`set_identity` / `identity` have no new indirection.
- `BidiStreamSource: Send + Sync + 'static` is object-safe. This constrains
implementations to `Send + Sync + 'static`, matching `ProtocolHandler` —
consistent with the existing handler model.
- The `Box<dyn BidiStreamSource>` is one allocation per `Connection`. The
enum was stack-allocated (except the `StreamConn`'s inner `Mutex`).
Negligible per-connection cost; only matters if connections are
constructed in a hot loop, which they are not.
## References
- ADR-002: ProtocolHandler trait (unchanged by this ADR)
- ADR-005: BiStream type definition (amended by ADR-007; this ADR does not
touch `BiStream`; amended by ADR-009 — `BiStream` is the concrete handler
leaf, not a bare trait)
- ADR-009: `BiStream` as the handler leaf (amends this ADR's `accept_bi`
return type — `(SendStream, RecvStream)` → `BiStream`; the trait shape
and the `from_source` extension point are preserved)
- ADR-032: One-way door decision framework (why `ProtocolHandler` is not
changed — this ADR is additive to `Connection`, not a trait revision)
- ADR-010: ALPN router and endpoint (the endpoint constructs `Connection`s
via `from_quinn` / `from_iroh`; those now wrap a `BidiStreamSource` impl,
transparently)
- ADR-007: `Connection::from_stream` — generic single-stream (this ADR
generalizes `Connection` to hold a trait object; the `from_stream` /
`from_bidi` constructors and the yield-once contract are preserved via
`StreamBidiStreamSource`)
- Channels POC summary:
[`docs/research/alknet-channels/poc-summary.md`](../../research/alknet-channels/poc-summary.md)
§"Issues Surfaced" #1 (OQ-CH-13 confirmed +EV), #2 (REQ-CORE-02)
- Channels Phase 0 findings:
[`docs/research/alknet-channels/phase-0-findings.md`](../../research/alknet-channels/phase-0-findings.md)
§POC-Validated Requirements — REQ-CORE-01, REQ-CORE-02

626
docs/architecture/decisions/009-bistream-as-the-handler-leaf.md Normal file
View File

@@ -0,0 +1,626 @@
# ADR-009: `BiStream` as the Handler Leaf — Unify the Split-Pair `accept_bi`
## Status
Proposed (amends ADR-008's `BidiStreamSource::accept_bi` return type;
amends ADR-007's `from_stream` / `from_bidi` constructors; amends
ADR-038's `ChannelBidiStreamSource::accept_bi` return type;
resurrects ADR-005's `BiStream` trait as the handler-facing leaf type;
supersedes the "two Phase 6 issues" framing in
`docs/research/alknet-crate-extraction/findings.md`; **`into_sub_streams()`
preservation subsequently reversed by ADR-035 (2026-07-18) — see the
note at the bottom of this ADR**)
> **Note on `into_sub_streams()` (added 2026-07-18, ADR-035):** This ADR's
> body states `into_sub_streams()` (ADR-038) is "preserved" as the
> second accessor alongside `accept_bi`, because TTY's named
> unidirectional sub-streams are the case that justifies keeping
> `SendStream` / `RecvStream`. ADR-035 reverses that preservation: the
> channels layer has no `stream_type` concept, `into_sub_streams()` is
> removed, and TTY sub-demuxes its `BiStream` via its own 5-byte format
> (the same code TTY runs in direct mode). `SendStream` / `RecvStream`
> collapse to thin newtypes over `Box<dyn Async* + Send + Unpin>` as
> this ADR specifies, but their only consumer is the channels
> reassembly path's internal join (constructing a `BiStream` from split
> halves), not `into_sub_streams()`. See ADR-035 for the resolution
> rationale (the channels layer is pure channel multiplexing; the
> handler owns its sub-stream multiplexing on the `BiStream`).
## Context
The crate-extraction findings doc
(`docs/research/alknet-crate-extraction/findings.md` Phase 6) deferred
the `alknet-http` rework on the grounds that the `QuicStream` wrapper
(44 lines, `crates/alknet-http/src/server/adapter.rs:271-314`) is a
*necessary* adapter — `accept_bi()` returns a split
`(SendStream, RecvStream)` pair, `SendStream` implements only
`AsyncWrite`, `RecvStream` implements only `AsyncRead`, and
`HttpAdapter::serve_io` needs a single `AsyncRead + AsyncWrite`. The
finding was correct about the symptom and wrong about the cause. This
ADR untangles the cause.
### The tangle: five abstractions for "a bidirectional byte stream"
Today the codebase has five abstractions for the same concept, and every
handler picks a joining strategy per-handler:
| # | Abstraction | Where | Notes |
|---|-------------|-------|-------|
| 1 | `BiStream` trait (`AsyncRead + AsyncWrite + Send + Unpin`) | `crates/alknet-core/src/types.rs:226` | **Vestigial in code.** Declared per ADR-005, named in ADR-008 as "a client-side / test convenience trait," but grep across the workspace finds **zero** consumers — no `impl BiStream`, no `dyn BiStream`, no `Box<dyn BiStream>`. The trait is the ecosystem convention (`tokio::net::TcpStream`, `TlsStream<TcpStream>`, `russh::Channel::into_stream()` all satisfy it natively) but it was never wired in. |
| 2 | `Connection` (yields `(SendStream, RecvStream)` via `accept_bi`) | `crates/alknet-core/src/types.rs:507` | The handler-facing abstraction. Leaf is split. |
| 3 | `SendStream` (AsyncWrite-only) + `RecvStream` (AsyncRead-only) | `crates/alknet-core/src/types.rs:228-294` | The actual leaves handlers receive. Each carries a quinn/iroh/generic enum (`SendStreamKind` / `RecvStreamKind`) and dispatches per-call. |
| 4 | `WsStream` trait (recv/send `axum::ws::Message`) | `crates/alknet-http/src/websocket/upgrade.rs:44` | Bypasses `Connection` entirely. The WS session runs its own dispatch loop directly over `axum::extract::ws::WebSocket`; `CallConnection::new_overlay_only` is used instead of `Connection::from_bidi`. ADR-044/048 already say "a WS message stream is another `BiStream`-satisfying transport" — the code does not. |
| 5 | `MpscSendStream` / `MpscRecvStream` (channels POC) | `/workspace/alknet-channels-poc/src/mpsc_stream.rs` | Split mpsc-backed halves fed to `Connection::from_stream`. The channels POC's `TunnelHandler` consumes them directly as two `tokio::io::copy` pumps — the split shape is right for the tunnel, wrong for HTTP. |
The two Phase 6 issues are symptoms of one root: **the leaf type is
split, so every consumer either re-joins it (HTTP's `QuicStream`,
`QuicStreamDuplex` test helper) or bypasses `Connection` entirely
(WS's `WsStream` + bespoke dispatch loop).**
### What ADR-008 left half-finished
ADR-008 extracted `BidiStreamSource` as the connection-level extension
point and kept `accept_bi` returning `(SendStream, RecvStream)`:
```rust
async fn accept_bi(&self) -> Result<(SendStream, RecvStream), StreamError>;
```
This preserved the existing `Connection` API verbatim (the right call
for ADR-008's scope — the trait extraction was the one-way door; the
return shape was a known leftover). But it left the join *per-handler*:
every handler that wants a single duplex stream re-implements the same
`AsyncRead + AsyncWrite` wrapper. The wrapper is small (44 lines) and
correct, but it is duplicated per-handler, and the duplication is what
forces the WS path into a bespoke `WsStream` trait instead of running
through `Connection::from_bidi` like every other transport.
### What ADR-005 already specified
ADR-005 defined `BiStream: AsyncRead + AsyncWrite + Send + Unpin` as
the leaf, and the ADR's "Why BiStream is still defined as a trait"
section (lines 86-94) lists three uses: WASM door, testing,
portability. The trait was placed in `alknet-core` and then not used
as the handler leaf — ADR-002's `handle` signature takes `Connection`
(correctly, for multi-stream handlers like TTY that loop `accept_bi`),
and `Connection::accept_bi` returns the split pair. `BiStream` became
"the trait that would have been the leaf if handlers received a single
stream." This ADR makes it the actual leaf — not by changing the
handler signature (still `Connection`), but by changing what
`accept_bi` yields.
### The ecosystem convention
`AsyncRead + AsyncWrite + Send + Unpin` (or close variants) is the
Rust ecosystem's standard "bidirectional byte stream" shape:
- `tokio::net::TcpStream`, `tokio::net::UdpSocket`
- `tokio_rustls::server::TlsStream<TcpStream>`
- `russh::Channel::into_stream()` — "Consume the Channel to produce a
bidirectional stream, sending and receiving `ChannelMsg::Data` as
`AsyncRead + AsyncWrite`"
- `tokio::io::DuplexStream`
- A WS-message adapter (the one place real adapter work is required)
All satisfy `BiStream` natively. Making `BiStream` the handler leaf
aligns alknet with the convention: `Connection::from_bidi(stream)`
accepts any of these directly, no per-handler wrapper.
### The two-pump shape is unaffected
The tunnel handler (ADR-078) and the SSH `direct-tcpip` handler
(future) use the split shape — two `tokio::io::copy` pumps, one per
direction. With `BiStream` as the leaf, these handlers call
`tokio::io::split(bidi)` to get `(ReadHalf, WriteHalf)` — the same
stdlib idiom `tokio::io::split` already provides for `TcpStream` and
`TlsStream<TcpStream>`. The split is a stdlib call at the handler
boundary, not a per-handler trait wrapper. ADR-078's
shutdown-on-completion contract applies to the `ReadHalf`/`WriteHalf`
unchanged.
### The TTY named-sub-streams case (ADR-038, ADR-077)
ADR-038 specifies `into_sub_streams()` returning
`Vec<(u8, SubStreamHandle)>` where `SubStreamHandle` is
`Send(SendStream) | Recv(RecvStream)`. ADR-077's TTY-inside-channels
mode destructures into five named handles (`stdin`, `stdout`,
`stderr`, `ctrl_in`, `ctrl_out`). **Every stream_type is
unidirectional** (ADR-034) — the typed-sub-stream leaves are
unidirectional by design, and the join is wrong for them.
This means `SendStream` and `RecvStream` cannot fully go away. They
remain as the typed-sub-stream leaves for the channels-inside-TTY
case (and any future handler that destructures a `ChannelSubStreams`).
What goes away is the *quinn-welding* in them: today `SendStreamKind`
/ `RecvStreamKind` are enums with `Quinn` / `Iroh` / `Stream` variants
that dispatch per-call. Once `accept_bi` returns a joined `BiStream`,
the quinn/iroh `accept_bi` impls do the join *once* (via
`tokio::io::join`) and yield a `BiStream`. The `SendStream` /
`RecvStream` types collapse to thin newtypes over
`Box<dyn AsyncWrite + Send + Unpin>` / `Box<dyn AsyncRead + Send +
Unpin>` — used only by `into_sub_streams()` and the channels reassembly
path, never by a top-level handler's `accept_bi` call.
## Decision
### `accept_bi` returns `BiStream`
`BidiStreamSource::accept_bi` returns a single `BiStream`, not a split
pair:
```rust
#[async_trait]
pub trait BidiStreamSource: Send + Sync + 'static {
async fn accept_bi(&self) -> Result<BidiStream, StreamError>;
async fn open_bi(&self) -> Result<BidiStream, StreamError>;
fn remote_addr(&self) -> Option<SocketAddr>;
fn close(&self, code: u32, reason: &str);
}
```
`Connection::accept_bi` / `open_bi` delegate verbatim. The public
`Connection` API is preserved except for the return type — which is a
type change every caller sees, addressed below.
### `BiStream` is a concrete newtype, not a bare trait
A bare `dyn BiStream` won't work: `AsyncRead` / `AsyncWrite` methods
take `Pin<&mut Self>`, and trait objects need `Pin<Box<dyn ...>>` or a
newtype that owns the inner stream and re-projects. The clean shape is
a concrete struct that boxes the inner joined stream:
```rust
pub struct BiStream {
inner: Box<dyn AsyncReadWrite + Send + Unpin>,
}
// Internal helper trait — the union of AsyncRead + AsyncWrite + Send +
// Unpin. Not public; exists only to give BiStream a single boxed field.
trait AsyncReadWrite: AsyncRead + AsyncWrite {}
impl<T: AsyncRead + AsyncWrite> AsyncReadWrite for T {}
impl AsyncRead for BiStream { /* delegate to self.inner */ }
impl AsyncWrite for BiStream { /* delegate to self.inner */ }
```
`BiStream: AsyncRead + AsyncWrite + Send + Unpin` by construction. The
old `pub trait BiStream: AsyncRead + AsyncWrite + Send + Unpin {}`
(ADR-005, `types.rs:226`) is removed — the trait was never consumed,
and the concrete struct carries the same trait bounds forward as
implied bounds, not a marker trait. This is the ADR-005 resurrection:
the name and the bounds survive, the shape becomes a concrete leaf.
### The join moves into core's quinn/iroh impls (once)
```rust
#[cfg(feature = "quinn")]
async fn accept_bi(&self) -> Result<BidiStream, StreamError> {
let (send, recv) = self.conn.accept_bi().await
.map_err(map_quinn_connection_error)?;
Ok(BiStream::from_joined(send, recv)) // tokio::io::join internally
}
```
The `QuicStream` wrapper (`adapter.rs:271-314`, 44 lines) becomes
`BiStream::from_joined(send, recv)` — one line, in core, invisible to
handlers. The same applies to iroh. The join is no longer per-handler.
### `Connection::from_bidi` is the only public stream constructor;
`from_stream` is removed
Today `from_bidi` is a convenience wrapper that calls
`tokio::io::split(stream)` then `from_stream(send, recv)`, and
`from_stream` bakes the split into the constructor API — the same
split-leaf shape pushed one step earlier. With `BiStream` as the leaf,
`from_bidi` is the only public constructor that takes a joined stream.
`Connection::from_stream(send, recv, ...)` is **removed**.
The rule this normalizes: **the split never crosses a crate boundary
as part of a constructor.** A crate that produces split halves
naturally (the channels reassembly path, which produces
`MpscSendStream` / `MpscRecvStream` as distinct async types) joins
them *itself* via `tokio::io::join(send, recv)` (one line) and calls
`from_bidi`. A crate that has a joined stream (`TcpStream`,
`TlsStream<TcpStream>`, `russh::Channel::into_stream()`,
`WsBidiStream`, even a test `DuplexStream`) calls `from_bidi` directly.
`Connection` only ever holds a `BiStream`. The split is a crate-internal
concern of wherever it naturally arises.
The existing `from_stream` call sites update mechanically:
- `crates/alknet-client/src/dial/tcp_tls.rs` already uses `from_bidi`
(no change).
- `crates/alknet-endpoint/src/accept/tcp_tls.rs` already uses
`from_bidi` (no change).
- The call crate's test stubs
(`call_client.rs:91`, `protocol/connection.rs:465`,
`protocol/dispatch.rs:465`, `protocol/adapter.rs:294`,
`client/from_call.rs:428`) today do
`tokio::io::split(x)` then `from_stream(send, recv, ...)` — they
become `from_bidi(x, ...)` directly, one call, no split.
- The channels reassembly path (per ADR-038, the future
`ChannelBidiStreamSource::accept_bi` impl) joins its
`MpscSendStream` / `MpscRecvStream` via `tokio::io::join` and calls
`from_bidi` — the join is in the channels crate (where the split
exists), not in the core constructor API.
- The core test at `types.rs:768` and the `from_source_tests` helper
become `from_bidi` calls (or construct `BiStream` directly via
`BiStream::from_joined`).
`SendStream::from_stream` / `RecvStream::from_stream` (the per-half
constructors, `types.rs:267` / `types.rs:289`) are **retained** — they
are the per-half boxing for `into_sub_streams()` (ADR-038) and the
channels reassembly path's `SubStreamHandle` leaves, not constructors
that feed `Connection`. The split lives where it is natural (channels
reassembly → `SubStreamHandle`), doesn't leak into `Connection`'s API.
### `SendStream` / `RecvStream` collapse to thin newtypes
```rust
pub struct SendStream { inner: Box<dyn AsyncWrite + Send + Unpin> }
pub struct RecvStream { inner: Box<dyn AsyncRead + Send + Unpin> }
```
Used by `into_sub_streams()` (ADR-038) and the channels reassembly
path. No `SendStreamKind` / `RecvStreamKind` enum — the quinn/iroh
dispatch is gone, the join happens once in the `BidiStreamSource` impl.
`SendStream::from_quinn` / `from_iroh` (crate-private) become the
thin-boxing constructors used only by the channels reassembly path
when it needs to expose unidirectional sub-streams. The
`from_stream(impl AsyncWrite + Send + Unpin)` / `from_stream(impl
AsyncRead + Send + Unpin)` public constructors are retained.
### `HttpAdapter` drops `QuicStream`
```rust
async fn handle(&self, connection: Connection, auth: &AuthContext)
-> Result<(), HandlerError>
{
if let Some(identity) = auth.identity.clone() {
let _ = connection.set_identity(identity);
}
let stream = connection.accept_bi().await
.map_err(stream_error_to_handler)?;
self.serve_io(stream).await // BiStream: AsyncRead + AsyncWrite + Unpin
}
```
`QuicStream` (44 lines) and `QuicStreamDuplex` (test helper, 38 lines)
are removed. `serve_io<I: AsyncRead + AsyncWrite + Send + Unpin>` is
unchanged — `BiStream` satisfies the bounds by construction.
### WebSocket runs through `Connection::from_bidi`
`WsBidiStream` (new, ~50-80 lines) implements `AsyncRead` / `AsyncWrite`
over `axum::extract::ws::WebSocket` binary messages: `AsyncRead`
consumes `Message::Binary` payloads (text messages close with a
protocol error, matching the current `drive_ws_session` behavior);
`AsyncWrite` frames each write as a `Message::Binary`; `poll_shutdown`
emits `Message::Close`. The WS session then runs through
`Connection::from_bidi(WsBidiStream::new(socket), alpn, addr)` +
`CallAdapter::handle` (or whatever the call-protocol's
`ProtocolHandler` is at the assembly layer) — the same path as any
other transport.
The `WsStream` trait (`upgrade.rs:44-49`), the bespoke `drive_ws_session`
loop, the `handle_inbound_envelope` / `dispatch_envelope_to_pending`
helpers, and the `run_ws_session` glue are removed. The session's
wire-level invariants (binary-only, protocol-level close on text,
`fail_all` pending on disconnect, ADR-048's `EventEnvelope` framing)
move into `WsBidiStream`'s `AsyncRead` / `AsyncWrite` / `poll_shutdown`
impls and the standard call-protocol dispatch path.
`CallConnection::new_overlay_only` stays — it's the
connection-local-overlay construction for non-peer clients
(ADR-034 §4, ADR-044 §5), orthogonal to the transport seam. What
changes is that the WS session feeds it through a `Connection` rather
than a parallel `WsStream` trait.
### Channels spec updates
ADR-038's `ChannelBidiStreamSource::accept_bi` returns `BiStream`:
```rust
async fn accept_bi(&self) -> Result<BiStream, StreamError> {
// Yields the joined (stream_type 0, stream_type 1) pair on first
// call, ConnectionClosed on subsequent calls.
}
```
`into_sub_streams()` is unchanged in shape — it still returns
`Vec<(u8, SubStreamHandle)>` with `SubStreamHandle::Send(SendStream) |
Recv(RecvStream)`, because TTY's named sub-streams are unidirectional
(ADR-034, ADR-077). The two paths (`accept_bi` for handlers that want
the joined pair, `into_sub_streams` for handlers that want the typed
unidirectional sub-streams) are preserved per ADR-038.
The channels POC's `MpscSendStream` / `MpscRecvStream` feed
`BiStream::from_joined(send, recv)` (or `from_stream` if the channels
crate prefers to construct the joined leaf directly from the mux
handle) — the `ChannelBidiStreamSource::accept_bi` impl does the join
once, and the per-channel `Connection::accept_bi` yields a `BiStream`.
The POC's `TunnelHandler` calls `tokio::io::split(bidi)` to get its two
pump halves, the same idiom it would use over `TcpStream`.
### `BiStream` over WebSocket enables "VPN-like without being a VPN" in v1
The `webtransport.md` spec describes the "VPN-like without being a VPN"
path: a browser opens a WebTransport session to `/alknet/ssh`, the h3
handler hands each bidi stream to `SshAdapter::handle` as a
`Connection`, the browser's WASM SSH parser speaks SSH over the
stream. WebTransport is deferred per ADR-044.
With `BiStream` as the leaf, the same path exists over WebSocket in
v1: a browser opens a WS connection, `WsBidiStream` presents it as a
`BiStream`, `Connection::from_bidi` wraps it, `ChannelsAdapter::handle`
runs the channels demux over it, each channel's `accept_bi` yields a
`BiStream` that `SshAdapter::handle` receives. The WASM SSH parser
runs over a `BiStream`-over-WS-message adapter on the browser side
(the same `WsBidiStream` shape, browser-implemented). ADR-044/048's
"WS message stream is another `BiStream`-satisfying transport" becomes
literal — the code does what the spec said.
The channels POC's sync core already compiles under
`wasm32-unknown-unknown`; a `BiStream`-over-WS adapter would too. The
WASM-clean property is preserved by the unification, not blocked by
it.
### WebTransport is a channels concern, not an alknet-http concern
The `h3` handler as specified in `webtransport.md` does exactly the
channels shape: one connection, N bidi streams inside, each routed to
an ALPN by the CONNECT path. That's `ChannelsAdapter::handle` with a
different wire format (HTTP/3 extended CONNECT vs the 9-byte chunk
header). When WebTransport revives, the h3 multi-stream demux leaves
`alknet-http` and becomes a channels-variant ALPN — the `alknet-http`
h3 path becomes "register an ALPN handler that gets one `BiStream` and
serves it as HTTP/3," same as `h2`/`http/1.1`. The
ALPN-stream-proxy (ADR-040) is the channels-over-WebTransport shape,
not an `alknet-http` shape.
This is out of scope for this ADR (WebTransport is deferred per
ADR-044). It is recorded here because the unification is what makes
the future extraction clean: once `accept_bi` returns `BiStream`, the
`h3` handler's "accept a WebTransport session, yield each stream as a
`BiStream` to the ALPN handler" shape is the same code as
`ChannelsAdapter::handle`, and the extraction is a move, not a
redesign.
## What does NOT change
- **`ProtocolHandler` trait shape** — `handle(&self, connection:
Connection, auth: &AuthContext)` stays. This is an internal
`Connection` refactor; the handler trait is the ADR-032 one-way door.
- **`HandlerRegistry`** — unchanged.
- **`Connection::remote_alpn` / `set_identity` / `identity` / `close`**
— unchanged. These are `Connection`-level, not transport-level.
- **`BidiStreamSource` trait** (ADR-008) — preserved. Three signatures
change return type (`accept_bi`, `open_bi`, and the implied
`Connection::accept_bi` / `open_bi`); the trait shape and the
extension-point model are preserved.
- **`from_source` constructor** (ADR-008) — preserved. Downstream
crates implement `BidiStreamSource` and construct via `from_source`;
their `accept_bi` impls return `BiStream`.
- **`into_sub_streams()`** (ADR-038) — preserved. TTY's named
unidirectional sub-streams are the case that justifies keeping
`SendStream` / `RecvStream` (as thin newtypes, not quinn-welded
enums).
- **The two-pump pattern** (ADR-078) — preserved. Tunnel/SSH handlers
call `tokio::io::split(bidi)` for their two pump halves; the
shutdown-on-completion contract applies to the `ReadHalf` /
`WriteHalf` unchanged.
- **Yield-once contract** (ADR-007) — preserved.
`StreamBidiStreamSource::accept_bi` yields the `BiStream` once then
returns `ConnectionClosed`. The contract is about *how many times*
`accept_bi` yields, not *what shape* it yields.
- **`Connection::from_quinn` / `from_iroh`** — preserved as
convenience wrappers; internally wrap the `QuinnBidiStreamSource` /
`IrohBidiStreamSource` whose `accept_bi` does the join.
- **`Connection::from_bidi`** — promoted to the only public stream
constructor. `Connection::from_stream` is removed (the split no
longer crosses a crate boundary as part of a constructor).
- **`SendStream::from_stream` / `RecvStream::from_stream`** (per-half
constructors) — retained, but only as the boxing for
`into_sub_streams()` and the channels reassembly path's
`SubStreamHandle` leaves. Not constructors that feed `Connection`.
- **The endpoint's accept loops** (quinn/iroh) — unchanged.
## Consequences
**Positive:**
- The `QuicStream` wrapper (44 lines) and `QuicStreamDuplex` test
helper (38 lines) are removed from `alknet-http`. `HttpAdapter::handle`
becomes 4 lines. `serve_io`'s signature is unchanged.
- The `WsStream` trait, the bespoke `drive_ws_session` loop, and ~150
lines of WS-specific dispatch glue are removed from
`alknet-http/websocket/upgrade.rs`. The WS session runs through
`Connection::from_bidi` + the call-protocol handler like any other
transport. ADR-044/048's "WS message stream is `BiStream`-satisfying"
becomes literal.
- One abstraction (`BiStream`) replaces five. The leaf type matches the
ecosystem convention (`russh::Channel::into_stream()`, `TcpStream`,
`TlsStream<TcpStream>`, `DuplexStream`).
- "VPN-like without being a VPN" over WS in v1 becomes real: the same
path `webtransport.md` specified, over WS, now. The browser's WASM
parser implements `BiStream` over a WS-message adapter; the server
wraps it via `Connection::from_bidi`; `ChannelsAdapter::handle` runs
the demux; each channel's `BiStream` reaches `SshAdapter::handle`
unchanged.
- The quinn-welding in `SendStream` / `RecvStream` (the
`SendStreamKind` / `RecvStreamKind` enums and their per-call
dispatch) is gone. `SendStream` / `RecvStream` become thin newtypes
used only by the channels reassembly path and `into_sub_streams()`.
- The future WebTransport extraction is a move (h3 demux → a
channels-variant ALPN), not a redesign. The unification is what
makes it clean.
- ADR-005's `BiStream` is resurrected as the actual leaf, matching the
original intent the code never delivered.
**Negative:**
- Every `accept_bi().await` caller sees a return-type change from
`(SendStream, RecvStream)` to `BiStream`. Callers that want the
split pair call `tokio::io::split(bidi)`. The call-site change is
mechanical (`let (send, recv) = ...` → `let bidi = ...; let (recv,
send) = tokio::io::split(bidi)`), but it touches every handler. This
is the one-time cost of the unification; the alternative is
per-handler wrappers forever.
- Every `Connection::from_stream(send, recv, ...)` call site is
removed. The call crate's test stubs (5 sites) become `from_bidi`
calls. The channels reassembly path gains a one-line
`tokio::io::join` before `from_bidi`. No caller outside core and
the channels reassembly path was ever doing anything other than
`tokio::io::split` then `from_stream` — the split was always
gratuitous at the call site.
- ADR-008's `accept_bi` return shape is amended. ADR-008 explicitly
preserved the split-pair shape to keep the `Connection` API verbatim;
this ADR reverses that preservation. The trade is: one type change
across the codebase now, vs. one wrapper per handler forever.
ADR-008's trait-extraction (the one-way door) is preserved; the
return-shape is the amended part.
- ADR-038's `ChannelBidiStreamSource::accept_bi` return shape is
amended (same change, same rationale). `into_sub_streams()` is
unchanged.
- `BiStream` becomes a concrete struct (with an internal boxed
`dyn AsyncReadWrite`), not a bare trait object. This is the
`Pin<&mut Self>` projection requirement — a bare `dyn BiStream` is
not ergonomic for `AsyncRead` / `AsyncWrite` impls. The ADR-005
trait is removed; the bounds survive as implied bounds on the
concrete struct. The name and the convention are preserved; the
shape becomes a concrete leaf.
- `WsBidiStream` is real new code (~50-80 lines). The WS-message ↔
byte-stream adapter is the one place the unification requires
non-trivial work — WS messages are framed, not a byte stream, so
the adapter owns the framing. This is the same work the current
`drive_ws_session` loop does, just relocated from a bespoke loop
into the `AsyncRead` / `AsyncWrite` impls.
- The `alknet-http` crate gains a dependency on whatever crate
owns `WsBidiStream` (likely `alknet-http` itself, or a small
`alknet-ws` crate if WASM-targetability is a goal — the browser side
needs the same adapter). This is a packaging decision, not a
design one — recorded as an open question below.
## Door type
**One-way.** The `accept_bi` return shape is the handler-facing API
surface. Once handlers are written against `BiStream`, reversing to
the split-pair shape is a rewrite of every handler's call site. The
trade is one type change across the codebase now vs. one wrapper per
handler forever — this ADR takes the one-time cost.
The `BiStream` concrete-struct shape (internal `Box<dyn
AsyncReadWrite>`, `Pin` projection) is a two-way-door implementation
detail — the internal representation can change without breaking the
public `AsyncRead + AsyncWrite + Send + Unpin` bounds.
## Migration
The migration is mechanical and can be ordered to keep the workspace
compilable:
1. **Core: introduce `BiStream` as the concrete leaf.** Add the
struct, the `AsyncRead` / `AsyncWrite` impls, the `from_joined`
constructor. Change `BidiStreamSource::accept_bi` / `open_bi` return
types to `BiStream`. Update `QuinnBidiStreamSource` /
`IrohBidiStreamSource` / `StreamBidiStreamSource` impls to do the
join. `Connection::accept_bi` / `open_bi` delegate verbatim. Remove
`Connection::from_stream` (the split-pair constructor); promote
`Connection::from_bidi` to the only public stream constructor.
This is a single-crate change; every `accept_bi` and `from_stream`
caller breaks mechanically.
2. **Update every handler's `accept_bi` call sites and every
`from_stream` call site.** `HttpAdapter::handle` becomes 4 lines
(drop `QuicStream`). `TtyAdapter::handle` calls
`tokio::io::split(bidi)` for its pump halves (or uses
`into_sub_streams()` in channels mode — unchanged). The channels
POC's `TunnelHandler` and `EchoHandler` get the same
`tokio::io::split` treatment. `CallAdapter::handle` (wherever it
consumes `accept_bi`) gets the same. The call crate's test stubs
(5 `from_stream` sites) become `from_bidi` calls (drop the
`tokio::io::split` they were doing immediately before). The channels
reassembly path gains a one-line `tokio::io::join` before `from_bidi`.
3. **Collapse `SendStream` / `RecvStream` to thin newtypes.** Remove
`SendStreamKind` / `RecvStreamKind` enums; the quinn/iroh
constructors become thin-boxing. Used only by the channels
reassembly path and `into_sub_streams()`.
4. **`alknet-http`: rewrite WS through `Connection::from_bidi`.** Add
`WsBidiStream`; remove `WsStream` trait, `drive_ws_session` loop,
and the dispatch glue. The WS session runs through
`Connection::from_bidi` + the call-protocol handler. This is the
largest single change and can land after (1)-(3) — the WS path is
independent of the handler call-site updates.
5. **Update ADR-007, ADR-008, ADR-038, ADR-077** to reflect the
`BiStream` return shape. ADR-007's `from_stream` constructor is
removed; `from_bidi` is the only public stream constructor (the
rule: the split never crosses a crate boundary as part of a
constructor). ADR-008's `accept_bi` return type is amended. ADR-038's
`ChannelBidiStreamSource::accept_bi` return type is amended;
`into_sub_streams()` is unchanged. ADR-077's two-mode TTY design is
unchanged (the modes differ in *how* the adapter gets sub-streams,
not in the leaf type).
6. **Update `findings.md` Phase 6.** The "deferred" status is
replaced: the `QuicStream` wrapper is removed (not because
`accept_bi` returns streams that are already duplex, but because
`accept_bi` now returns a `BiStream`); the WS path is unified; the
h3/WebTransport extraction is recorded as a future channels-variant
move enabled by this ADR.
The Phase 6 deferral in `findings.md` is resolved by this ADR — not by
the original plan (drop the wrapper as redundant) but by the actual
fix (unify the leaf so the wrapper moves into core).
## Open questions
- **Where does `WsBidiStream` live?** If WASM-targetability is a goal
(the browser side needs the same adapter), it may want to live
somewhere a WASM client can reach — `alknet-core` (no, HTTP deps
don't belong in core), a small `alknet-ws` crate, or
`alknet-http` with the browser-side adapter extracted separately.
Default: `alknet-http` owns the server-side `WsBidiStream`; the
browser-side adapter is a separate concern (the WASM SDK, not
alknet-http). Resolved at implementation time.
- **`SendStream` / `RecvStream` long-term home.** With the quinn
enums gone, these are thin newtypes over
`Box<dyn Async* + Send + Unpin>`. They could move out of
`alknet-core` into `alknet-channels-core` (their only consumer is
`into_sub_streams()`). Default: stay in `alknet-core` for now (the
channels crate is not yet extracted); revisit at the channels
extraction.
## References
- ADR-005: `BiStream` type definition (resurrected by this ADR — the
trait is removed, the bounds survive as implied bounds on the
concrete struct)
- ADR-007: `Connection::from_stream` / `from_bidi` (amended —
`from_stream` is removed; `from_bidi` is the only public stream
constructor; the split never crosses a crate boundary as part of a
constructor)
- ADR-008: `BidiStreamSource` trait (amended — `accept_bi` / `open_bi`
return `BiStream`, not the split pair; the trait shape and the
`from_source` extension point are preserved)
- ADR-038: `ChannelBidiStreamSource` (amended — `accept_bi` returns
`BiStream`; `into_sub_streams()` is unchanged)
- ADR-077: TTY inside channels (unchanged — the two-mode design is
preserved; the modes differ in how the adapter gets sub-streams,
not in the leaf type)
- ADR-078: two-pump shutdown-on-completion (unchanged — the contract
applies to `tokio::io::split(bidi)` halves)
- ADR-044, ADR-048: WebSocket is the v1 browser bidirectional path
(this ADR makes the "WS message stream is `BiStream`-satisfying"
claim literal)
- `docs/research/alknet-crate-extraction/findings.md` Phase 6 — the
deferred `alknet-http` rework; this ADR resolves the deferral by
unifying the leaf rather than by dropping the wrapper as redundant
- `docs/architecture/crates/http/webtransport.md` — the deferred h3
handler; this ADR records the future extraction as a
channels-variant move, enabled by the unification
- `crates/alknet-core/src/types.rs:226` — the vestigial `BiStream`
trait this ADR resurrects as the concrete leaf
- `crates/alknet-http/src/server/adapter.rs:271-314` — the `QuicStream`
wrapper this ADR removes
- `crates/alknet-http/src/websocket/upgrade.rs:44-49` — the `WsStream`
trait this ADR removes
- `russh::Channel::into_stream()` — the ecosystem convention this ADR
aligns with

203
docs/architecture/decisions/010-secret-material-flow-and-capability-injection.md Normal file
View File

@@ -0,0 +1,203 @@
# ADR-010: Secret Material Flow and Capability Injection
## Status
Accepted
## Context
alknet-vault holds the master seed and can derive keys and encrypt/decrypt
arbitrary data. ADR-008 established that the vault is a **capability source**:
"derived keys and decrypted credentials are injected into operation contexts
at the assembly layer, not passed as vault references to handlers." That
prose was correct but the mechanism was never specified.
The result was a contradiction in the spec documents. ADR-008 said the master
seed never crosses the network, but `operation-registry.md` showed
`vault/derive`, `vault/unlock`, and `vault/decrypt` registered as call protocol
operations — directly on the wire. Those two statements cannot both be true.
The contradiction arose because no injection mechanism existed in the
architecture, so the only way the docs could show a handler obtaining a key was
to expose vault operations over the call protocol.
This is a one-way door. Once secret material crosses the wire as a call
protocol operation, the attack surface is permanent:
- `vault/unlock` accepts a BIP39 mnemonic — the root of trust — over QUIC. A
compromised peer, a logging accident, a tracing span, and the seed is gone.
- `vault/derive` returns a `DerivedKey`. The type redacts the private key in
JSON today, but the operation's existence means a serialization change, a
binary codec addition, or a wrapper change would leak it. The surface is
the risk, not the current implementation.
- `vault/decrypt` accepts an encrypted blob and returns plaintext. Any
authorized caller can decrypt any blob they possess.
The broader problem this decision addresses is structural: the industry
default for storing LLM provider keys, API tokens, and other credentials is
plaintext config files and environment variables (e.g., the aisdk Rust port
reads `std::env::var("GOOGLE_API_KEY")` and the example backend calls
`dotenv::dotenv()`). alknet replaces that with a vault. But the vault only
solves the storage problem; the flow problem — how decrypted material reaches
the code that needs it without crossing the network — requires its own
decision.
There is a separate, second axis that the current `OperationContext` conflates
with the secret-flow problem. A handler has two orthogonal credential concerns:
- **Identity (inbound)**: who is calling me? Resolved per-request from
`AuthContext` (TLS client cert, auth token). Already in `OperationContext`.
- **Capabilities (outbound)**: what secrets can I use for outbound calls? This
is the missing axis. A handler calling Google's API needs a decrypted Google
API key. That is not the caller's identity — it is the handler's own outbound
credential, provisioned by the assembly layer.
Mixing these two into one channel (e.g., stuffing secrets into
`OperationContext.metadata: HashMap<String, Value>`) is a leak risk: metadata
propagates through nested calls via `OperationEnv::invoke()`, so a secret
placed there by one handler would flow to every downstream operation.
## Decision
**1. The vault is assembly-layer only.**
The CLI binary (the `alknet` crate, or an embedded assembly layer) is the sole
component that talks to `VaultServiceHandle` directly. It unlocks the vault at
startup, derives and decrypts what each handler needs, and constructs handlers
with the results. No vault operation (`derive`, `decrypt`, `unlock`, `lock`)
is registered as a call protocol operation. The vault has no ALPN. The master
seed and derived private keys never enter the call protocol.
**2. Capabilities are the injection mechanism.**
A `Capabilities` type carries outbound secret material from the assembly layer
into handlers. Capabilities are distinct from identity (inbound auth) and
distinct from per-request metadata. The concrete shape of the `Capabilities`
type is a two-way door — to be decided during implementation of the
`alknet-call` crate. The one-way constraint is:
- Capabilities hold non-serializable, zeroized secret material. They cannot
cross the call protocol wire even by accident — they are not
`serde_json::Value`, they do not implement `Serialize`, and they do not
appear in `EventEnvelope` payloads.
- Capabilities are injected at handler construction (the common case: a static
decrypted API key held for the handler's lifetime) or scoped per-request for
internal-only flows. They are never populated from call protocol
inputs.
**3. The call protocol carries no secret material.**
This is a wire-level constraint on the call protocol, not a handler-level
convention. Secret material (private keys, API keys, mnemonics, decrypted
credentials, raw tokens) must not appear in:
- `call.requested` payloads (inputs)
- `call.responded` payloads (outputs)
- `OperationContext.metadata`
The wire format does not enforce this — it carries `serde_json::Value` — so the
constraint is architectural, enforced by the operation registry and by
convention. Operations that need to share public key material (e.g., for
identity verification) use a dedicated operation that returns only the public
component, never the private key.
**4. Adapters take credential sources, not static tokens.**
The `from_openapi` and `from_jsonschema` adapter patterns (defined in Rust in
alknet-call per ADR-033) register HTTP-backed operations. The TypeScript
`@alkdev/operations` `from_openapi` takes `config.auth: { token: "..." }` — a
static string. The Rust adapters take a credential source wired to the assembly
layer (a resolver, a capability handle, or an injected secret), not a literal
token. This is the integration point where the vault feeds credentials into
HTTP-backed operations: the assembly layer decrypts the token at startup and
provides it to the adapter at registration time.
**5. Handlers that need per-request vault access receive a scoped capability.**
The common case (a static decrypted API key) is covered by construction-time
injection. A narrower case — a handler that derives a child key for a specific
operation (e.g., signing for GitHub authentication) — receives a
scoped capability that can only derive at a restricted path set. This is still
not a vault reference: it is a restricted handle that performs a specific
derivation and returns the result to the handler, in-process. The handler
never sees the master seed. Whether this scoped capability is a distinct type
or modeled as a pre-derived key injected at construction is a two-way door
left to the `alknet-call` and `alknet-agent` crate specs.
## Consequences
**Positive:**
- The master seed and derived private keys never cross the network. The attack
surface for the root of trust is local-only.
- The `OperationContext` gains a clean second axis (capabilities) instead of
overloading `metadata` for secrets, preventing accidental propagation of
secret material through nested calls.
- Handlers that need outbound credentials (the agent handler calling an LLM
provider) receive them directly — no indirection through a `vault/derive`
call, no latency, no failure mode where the vault must be reachable at call
time.
- The adapter contract (OQ-15) gains a concrete shape: adapters take a
credential source from the assembly layer, not a static token. This makes
the `from_openapi` / `from_jsonschema` / `from_call` patterns safe by
construction.
- The model is structurally incompatible with the env-var / plaintext-config
default. There is no `std::env::var("API_KEY")` path — the only way a handler
gets a credential is through a capability, and the only way a capability is
populated is through the assembly layer from the vault.
**Negative:**
- The assembly layer (CLI binary) has more construction-time responsibility: it
must know which handlers need which credentials and wire them. This is
expected — the CLI assembles everything (ADR-008).
- Adding a new handler that needs a new credential requires updating the
assembly layer, not just registering an operation. This is a feature, not a
bug: it forces an explicit decision about what secret material a handler
needs.
- Remote vault administration (unlock a running node's vault over the network)
is not supported by this decision. If that capability is needed in the
future, it would require a separate, heavily restricted mechanism (admin
scope, mTLS-only, never expose the mnemonic over an unauthenticated channel)
and its own ADR. This decision does not close that door; it simply does not
open it.
- The `Capabilities` type shape is not fully specified here. The one-way
constraint (non-serializable, zeroized, injection-only) is fixed; the
concrete API is a two-way door for the `alknet-call` spec.
## Assumptions
These are the load-bearing assumptions. If any of them breaks, the decision
should be revisited:
1. **Handlers need credentials at construction time or at call time, not
dynamically discovered at call time.** If a handler needs to derive a key
at an unpredictable path determined by call input, the scoped-capability
model still covers it (the handler holds a scoped vault access), but the
surface area is larger. The assumption is that this case is rare.
2. **The call protocol's threat model excludes the assembly layer.** The CLI
binary is trusted to hold the vault handle and inject capabilities. If the
assembly layer is compromised, all handlers' capabilities are compromised.
This is the same trust boundary as ADR-008.
3. **No legitimate use case requires returning a private key over the wire.**
Public key sharing (identity verification, encryption to a recipient) is
the only cross-node key material flow. If a use case for returning a
private key emerges (e.g., a key-escrow service), it needs its own ADR and a
very different threat model.
4. **Adapters are registered at startup, not at call time.** The credential
source is wired to the adapter when the operation is registered, not when
the operation is invoked. This is consistent with OQ-04 (static
registration at startup).
## References
- ADR-031: Crate decomposition (alknet-vault is standalone)
- ADR-013: irpc as call protocol foundation
- ADR-008: Vault integration point (capability source — this ADR specifies the
mechanism that ADR-008 described in prose)
- ADR-032: One-way door decision framework
- ADR-033: Rust as canonical implementation language
- OQ-15: Call protocol client and adapter contract (this ADR constrains the
adapter contract: adapters take credential sources, not static tokens)
- OQ-16: Safe vault operations for call protocol exposure (resolved by this
ADR: none, for now)
- alknet-vault implementation: `crates/alknet-vault/`

633
docs/architecture/decisions/011-dynamic-resource-ownership-for-runtime-spawned-resources.md Normal file
View File

@@ -0,0 +1,633 @@
# ADR-011: Dynamic Resource Ownership for Runtime-Spawned Resources
## Status
Accepted (resolves OQ-42; amends the `AccessControl::check` signature and
adds `OperationSpec.resource_id_path`. Does not amend ADR-017 or ADR-018 —
specific #4 confirms the composition authority stays static. Blocks lifted:
the alknet-docker, alknet-tty, opencode-runner wrapper, and
`alknet-container` (fleet normalization) crate specs can declare their
`AccessControl` shapes against this model.)
## Context
The alknet-docker POC (`docs/research/alknet-docker/poc-summary.md`)
surfaced a class of resource that the existing auth model doesn't handle:
**runtime-spawned resources with derived ownership**. A coordinator starts
a container and exposes docker operations over the call protocol; the
question "is this peer allowed to `docker/container/exec` against container
C?" is not answerable by the static `Identity.resources` model —
`Identity.resources` is config-sourced (from `PeerEntry` on the fingerprint
path, from `CompositionAuthority` on the composition path) and set at
registration or connection time. The container didn't exist then. Its
ownership was derived at spawn time: whoever started it owns it.
This generalizes beyond docker. Every "spawn a thing at runtime and expose
it over the call protocol" crate has the same shape:
- **alknet-docker** — containers as `AccessControl` resources.
- **alknet-tty** — terminal sessions as resources (who owns this TTY?).
- **opencode-runner wrapper** — workspace containers / processes as
resources.
- **`alknet-container`** (fleet normalization) — the fleet layer that
normalizes container operations across hosts.
None of those crate specs can declare their `AccessControl` shapes until
the core model answers: **how does `AccessControl::check` learn whether
identity X owns runtime-spawned resource R?**
OQ-42 tracked this. The OQ resolved five sub-questions:
1. Storage shape — reuse the repo/adapter pattern (ADR-033).
2. Integration point — `AccessControl::check` consults an ownership
provider directly (Option 2), with `OperationSpec` gaining a
`resource_id_path` JSON pointer.
3. Access pattern — proxy-only (spawner owns, proxy to share, teardown
revokes; no grant mechanism in core).
4. Four edge specifics — the `list` case, teardown coupling, fleet
representation, composition interaction.
This ADR writes those decisions into ADR format.
### Why the static model breaks
The current `AccessControl::check` is a pure function of `(ACL,
Identity)`:
```rust
// operation-registry.md (current)
pub struct AccessControl {
pub required_scopes: Vec<String>,
pub required_scopes_any: Option<Vec<String>>,
pub resource_type: Option<String>, // e.g., "service"
pub resource_action: Option<String>, // e.g., "read"
}
impl AccessControl {
pub fn check(&self, identity: Option<&Identity>) -> bool {
// scope check: identity.scopes ⊇ required_scopes
// resource check: identity.resources[resource_type] ∋ resource_action
}
}
```
`Identity.resources` is a `HashMap<String, Vec<String>>` — named resource
lists, populated from `PeerEntry.resources` (fingerprint path, ADR-025) or
`CompositionAuthority.resources` (composition path, ADR-018). Both are
**static**: `PeerEntry` is config-sourced; `CompositionAuthority` is set
at registration. Neither grows at runtime.
For a static resource set ("alice can access services `gitea` and
`registry`"), this works: the config lists the resources, the identity
carries them, `check` matches them. For a runtime-spawned resource
("alice can exec into container `C` which didn't exist when the config was
written"), there's nowhere to record that alice owns C. The identity was
resolved at connection time, before C was spawned. The ownership exists —
the coordinator that started C knows alice asked for it — but the auth
model has no path from that knowledge to `check`.
### The options considered
Three integration points were considered (see OQ-42 for the full
reasoning):
- **Option 1 — augment `Identity.resources` with a per-request snapshot.**
The dispatcher would pull owned resources into a per-request identity
snapshot before calling `check`, so `check` *looks* unchanged while
reading state that was never part of the static identity. **Rejected:**
the purity was always theatrical (the question "can X exec into C" was
never purely a function of identity; it just looked that way because the
resource set was static). Option 1 hides the impurity in a snapshot
pretending to be static identity.
- **Option 2 — `check` consults the ownership provider directly.**
`AccessControl::check` grows a parameter (or reads one from
`OperationContext`) for the ownership provider, and consults it for
`resource_type`/`resource_action` checks against runtime-spawned
resources. **Accepted.** This makes `check`'s signature honest about
what ACL checking *is* in the presence of dynamic resources: a function
of (ACL, Identity, current-ownership-state). The impurity is real either
way; Option 2 puts it in the signature where it's visible.
- **Option 3 — handler-level ownership check, `AccessControl` gates only
scope.** Some resources statically checked, some handler-checked.
**Rejected:** it splits the ACL story — the kind of inconsistency that
creates the "figure out how it fits with what is there" cleanup this ADR
exists to prevent.
### The two access patterns
Walking through the concrete use cases (the agent-workspace case in
particular) surfaced two patterns for how a downstream consumer reaches a
runtime-spawned resource:
- **Proxy pattern (the common case).** A coordinator starts a container
and manages its lifecycle; the end user never talks to docker directly.
The coordinator re-exports the docker operations it wants to expose (via
`from_call` — the adapter that imports a peer's operations and
re-registers them locally, ADR-022 — or by composing them in its own
handlers), and when the end user invokes one, the coordinator is the
*direct caller* to the docker endpoint. Docker's ownership store sees
the coordinator as the owner and as the caller — the check passes. The
end user's identity rides as `forwarded_for` metadata (ADR-026), and the
coordinator does whatever end-user-level ACL it wants at its own layer.
This is the kernel/user-land + forwarded-for model: the hub's authority
is used, `forwarded_for` is metadata, the hub handles its own ACL.
- **Grant pattern ("poking holes").** A downstream app wants to give an
end user *direct* call-protocol access to the docker endpoint for
specific containers — the end user calls `docker/container/exec`
themselves, not through a proxy. Docker's ownership store would need a
record that the end user has access to that container, even though the
downstream app spawned it.
The agent-workspace case — the concrete one — is entirely the proxy
pattern. The coordinator starts the workspace container; the agent
interacts with what's *inside* the container (a TTY, an opencode
instance's API surface), not with docker operations on the container.
Docker-level operations (stop, remove, inspect) are the coordinator's
job. No described use case requires the grant pattern. This ADR commits to
proxy-only (Decision 3).
## Decision
### 1. Storage: reuse the repo/adapter pattern (fourth instance)
The ownership store is a fourth instance of the established repo/adapter
pattern (ADR-033), alongside `IdentityProvider` (ADR-003), `IdentityStore`
(ADR-035), and `CredentialStore` (ADR-031). A trait in `alknet-core` with
an in-memory default adapter:
```rust
// alknet-core
/// Read side: consulted by AccessControl::check on the dispatch hot path.
/// Sync — called in the accept/dispatch loop, no .await.
pub trait OwnershipProvider: Send + Sync + 'static {
/// Does `identity` own `resource_type/resource_id` with `action`?
/// Called when AccessControl has resource_type + resource_action set
/// and the dispatcher has extracted resource_id from the input via
/// OperationSpec.resource_id_path.
fn owns(
&self,
identity: &Identity,
resource_type: &str,
resource_id: &str,
action: &str,
) -> bool;
/// What resources of `resource_type` does `identity` own?
/// Called for the `list` case (resource_type set, resource_id_path
/// absent) — the result-filter path. Returns the set of resource IDs
/// the caller owns, for the handler to filter against.
fn owned_resources(
&self,
identity: &Identity,
resource_type: &str,
) -> Vec<String>;
/// Does `identity` own *any* resource of `resource_type`?
/// Called for the `list` case — the scope-gate path. Cheap boolean
/// for the "allow if scoped" default.
fn owns_any(
&self,
identity: &Identity,
resource_type: &str,
) -> bool;
}
/// Write side: called by the handler that manages the resource lifecycle.
/// Async — not on the dispatch hot path.
#[async_trait]
pub trait OwnershipStore: Send + Sync + 'static {
/// Record that `identity` spawned `resource_type/resource_id`.
/// Called by the docker handler after `docker/container/create`
/// succeeds.
async fn record(
&mut self,
identity: &Identity,
resource_type: &str,
resource_id: &str,
) -> Result<(), OwnershipError>;
/// Revoke ownership of `resource_type/resource_id`.
/// Called by the docker handler on container exit / removal
/// (specific #2 — handler-driven teardown).
async fn revoke(
&mut self,
resource_type: &str,
resource_id: &str,
) -> Result<(), OwnershipError>;
}
/// In-memory default adapter. Carries the docker/runner cases with no
/// backend dependency — ownership is runtime state, meaningless across
/// restarts (a container ID from a previous process doesn't exist).
pub struct InMemoryOwnershipStore { /* HashMap<...> */ }
```
The read trait (`OwnershipProvider`) is sync — called from
`AccessControl::check` on the dispatch hot path, no `.await`. The write
trait (`OwnershipStore`) is async — called by handlers off the hot path.
A persistence adapter (e.g., sqlite/honker-backed, for a hub that wants
fleet ownership to survive restarts) is separable and built when a
concrete use case forces it — same as `alknet-store-sqlite` for
peer/credential persistence (ADR-035). The in-memory default carries no
persistence; ownership is runtime state. A persistence adapter would cache
in memory and use honker `NOTIFY` for invalidation — same
`ArcSwap`-backed full-reload pattern as `ConfigIdentityProvider`
(ADR-035).
**The read/write split mirrors ADR-035.** `OwnershipProvider` (read,
sync) is the trait the dispatch path depends on. `OwnershipStore` (write,
async) is the trait the handler lifecycle calls. The in-memory default
implements both. A persistence adapter would implement both with an
in-memory read cache backed by SQLite, same as `SqliteIdentityProvider`
implements `IdentityProvider` (sync, cached) + `IdentityStore` (async
write).
### 2. Integration: `AccessControl::check` consults the ownership provider
`AccessControl::check` grows a parameter for the ownership provider. The
provider is carried on `OperationContext` (populated by the dispatch path
from the registry's wiring), not threaded through every call site
manually:
```rust
pub struct AccessControl {
pub required_scopes: Vec<String>,
pub required_scopes_any: Option<Vec<String>>,
pub resource_type: Option<String>,
pub resource_action: Option<String>,
}
impl AccessControl {
/// `ownership` is None when the operation has no resource_type
/// (pure scope check) or when no ownership provider is wired
/// (the static `Identity.resources` path — backward compatible).
/// `resource_id` is None for the `list` case (resource_type set,
/// resource_id_path absent — specific #4a).
pub fn check(
&self,
identity: Option<&Identity>,
resource_id: Option<&str>,
ownership: Option<&dyn OwnershipProvider>,
) -> bool {
// 1. Scope check (unchanged): identity.scopes ⊇ required_scopes.
// If identity is None and scopes are required, deny here.
// 2. Resource check (only if self.resource_type is Some):
// a. If resource_id is Some(id) and ownership is Some(p):
// → identity must be Some (owns takes &Identity, not
// Option); if identity is None, deny. Otherwise
// → p.owns(identity.unwrap(), resource_type, id, resource_action)
// b. If resource_id is None (the `list` case) and ownership is Some(p):
// → if identity is None, deny; otherwise
// → p.owns_any(identity.unwrap(), resource_type) [scope-gate; see #4a]
// c. If ownership is None → fall back to static
// identity.resources[resource_type] ∋ resource_action
// (backward compat for non-runtime resources; identity
// may be None here — empty resources → deny if action required)
}
}
```
The `resource_id` parameter is extracted by the dispatcher from the
operation input using `OperationSpec.resource_id_path` (Decision 2a
below). When the spec has no `resource_id_path` (the `list` case), the
dispatcher passes `resource_id: None`, and `check` takes the scope-gate
path (specific #1).
**Backward compatibility.** When `ownership` is `None`, `check` falls
back to the static `Identity.resources` path. This means existing
operations with static resource sets (no runtime spawning) work unchanged
— the ownership provider is an additional check, not a replacement. The
signature change is a one-way door (every call site and test updates), but
the semantic change is additive: operations that don't wire an ownership
provider behave exactly as before.
### 2a. `OperationSpec` gains `resource_id_path`
```rust
pub struct OperationSpec {
pub name: String,
pub namespace: String,
pub op_type: OperationType,
pub visibility: Visibility,
pub input_schema: Value,
pub output_schema: Value,
pub error_schemas: Vec<ErrorDefinition>,
pub access_control: AccessControl,
/// JSON pointer into the input for the resource ID, when
/// `access_control.resource_type` is set and the operation targets a
/// specific runtime-spawned resource. e.g., `"$.containerId"` for
/// `docker/container/exec`. Absent for no-specific-resource operations
/// (the `list` case — specific #1). The dispatcher extracts the
/// resource ID from the input using this path and passes it to
/// `AccessControl::check`.
pub resource_id_path: Option<String>,
}
```
The fit with JSON Schema is load-bearing, not incidental: `input_schema`
is already a JSON Schema, so `resource_id_path` is a pointer *within* an
existing schema on the same spec. The `OperationSpec` becomes fully
self-describing for authorization — what resource type, what action, and
*which input field* drives the resource lookup. No per-namespace
conventions, no handler-level knowledge, no "the dispatcher just knows."
The contract is on the spec, where it belongs.
### 3. Access pattern: proxy-only
The base model is **"spawner owns, proxy to share, teardown revokes"** —
with no grant/transfer mechanism in the core ownership store.
When a coordinator spawns a container and wants to expose docker
operations on it to an end user, the coordinator re-exports those
operations via `from_call` (or composes them in its own handlers). The end
user invokes the re-exported operation; the coordinator is the direct
caller to the docker endpoint; docker's ownership store sees the
coordinator as owner and caller; the check passes. The end user's identity
rides as `forwarded_for` metadata (ADR-026), and the coordinator handles
its own end-user-level ACL at its own layer.
**"Poking holes" (the grant pattern) is a downstream-app concern, not a
core-model concern.** The app that owns the resources re-exports the
operations it wants to share via `from_call` with its own ACL layer,
rather than the core ownership store growing a grant API. The ADR commits
to proxy-only and explicitly states that "poking holes" is a downstream
app's job.
**A future grant mechanism is additive, not a one-way door closure.** If
a use case forces the grant pattern, it's a new method on the ownership
store trait (`grant(identity, resource)` / `revoke_grant(...)`).
`AccessControl::check` already consults the ownership provider; a
grant-aware provider would answer "yes" for grantees in addition to
owners, without a trait-shape change. The two-way-door classification
(additive) is stated here as reversal-cost classification, not as a reason
to defer the decision — the decision is made (proxy-only), and the cost of
reversing it if a future use case forces it is low. If the grant pattern
is later admitted, specifics #3 and #4 are revisited: cross-node ownership
propagation returns to the table (#3), and composition under a grant would
need `CompositionAuthority` to grow a dynamic path, amending ADR-017/022
(#4).
### 4. The four edge specifics
#### 4a. The `list` case: scope-gate + result-filter, composing
Operations with `resource_type` set but `resource_id_path` absent — e.g.
`docker/container/list` — don't reference a specific container. When a
coordinator lists containers it owns, it should see only its own — not
every container on the host. That's not just scope-gating ("can you call
`container/list` at all?") and not just result-filtering ("return only
owned") — it's **both**:
1. **Scope-gate** (the call): does the peer have the `container:list`
scope? This is the static `required_scopes` check — unchanged. If the
peer doesn't have the scope, the call is denied before the ownership
provider is consulted.
2. **Result-filter** (the response): the handler calls
`OwnershipProvider::owned_resources(identity, "container")` and filters
the result to only the containers the caller owns. The default is
"allow if scoped, filter to owned."
The scope-gate is `AccessControl::check`'s scope path (static, unchanged).
The result-filter is a handler-level concern — the handler calls the
ownership provider's `owned_resources` method and filters. The ADR states
the default ("allow if scoped, filter to owned") and the composition
(scope-gate the call, then filter the result). A spec declares it wants
this by setting `resource_type` without `resource_id_path`.
`exec`/`inspect`/`stop` against a specific container are the clean case:
`resource_id_path: Some("$.containerId")`, the dispatcher extracts the ID,
`check` calls `owns(identity, "container", id, "exec")` — a single
targeted lookup.
#### 4b. Teardown coupling: automatic, handler-driven
The ownership store's write path (revoke on teardown) is coupled to the
spawned resource's lifecycle. The "burn it and start over" capability
depends on ownership state tracking the lifecycle correctly. When a
container dies or is destroyed, the ownership entry is revoked **by the
handler that managed the lifecycle** (the docker handler calls
`OwnershipStore::revoke` on container exit), not by an operator workflow
or a background reaper.
The burn-and-start-over pattern is:
1. Destroy container → handler calls `revoke("container", id)` → ownership
revoked automatically.
2. Spawn new container → handler calls `record(identity, "container", new_id)`
→ new ownership recorded.
If teardown weren't automatic, stale ownership entries would accumulate
and the "burn" path would leave dangling ACL state — an ACL check could
reference a resource that no longer exists, and a reused container ID
could grant access to the wrong caller.
The architectural commitment is: **handler-driven revoke on lifecycle
end, not a reaper.** The coupling mechanism (explicit handler call vs. a
lifecycle-hook abstraction the handler framework provides) is two-way-door
implementation work — the docker handler calling `revoke` directly is the
initial mechanism (explicit handler call); a lifecycle-hook abstraction is
a refinement if multiple resource-spawning crates share the pattern.
#### 4c. Fleet representation: per-node ownership, downstream app tracks "who is this for"
Under the proxy pattern (Decision 3), the docker node records "coordinator
owns C" in its local ownership store. The coordinator's "I started C for
agent Y" mapping lives in the coordinator's own downstream-app state, not
in the core ownership store.
The ownership store is **per-node** — each docker node records its local
ownership. The hub's agent-to-workspace mapping is app state. There is
**no cross-node ownership propagation in the base model** — the spoke
sees the hub as the owner (the hub's `Identity` is what the spoke's
ownership store records), and the hub's "who is this for" is its own
concern, tracked in the hub's app state, carried as `forwarded_for`
metadata on the wire (ADR-026).
This simplifies fleet representation: the proxy pattern keeps ownership
local. The spoke doesn't need to know about the end user; the hub doesn't
need to push ownership records to the spoke. The hub authenticates as
itself (its own `auth_token`), the spoke records the hub as the owner, and
the hub's end-user ACL is its own layer.
#### 4d. Composition interaction: two separate checks, no change to `CompositionAuthority`
In the proxy pattern, the coordinator composes `docker/container/exec` on
behalf of an agent. Two checks must pass:
1. **Static scope check** (ADR-017/022, unchanged): the coordinator's
`CompositionAuthority` has the `container:exec` scope. This is the
existing `CompositionAuthority.scopes` check — static, set at
registration, no dynamic path.
2. **Dynamic ownership check** (this ADR): the coordinator owns this
specific container. This is the new `OwnershipProvider::owns` check —
dynamic, consults the ownership store.
The composition authority stays static — it doesn't grow a dynamic path.
The ownership store handles the dynamic resource-level check. Both must
pass; they're orthogonal. **ADR-017 and ADR-018 do not need amendment.**
The `CompositionAuthority.resources` field (ADR-018, line 180:
`resources: HashMap<String, Vec<String>>`) continues to serve its existing
purpose: static resource lists for composition authority (e.g.,
`{"service": ["vastai", "github"]}` bounds which services the handler can
reach in composition). It is not involved in the dynamic ownership check —
that's the ownership provider's job. The two are separate:
- `CompositionAuthority.resources` — static, "what services can this
handler compose," checked against the composition authority's declared
resource lists.
- `OwnershipProvider::owns` — dynamic, "does this identity own this
specific runtime-spawned resource," checked against the ownership store.
A handler composing `docker/container/exec` passes both: its composition
authority has `container:exec` in its scopes (static), and the ownership
provider confirms it owns the container (dynamic).
## Consequences
**Positive:**
- The alknet-docker, alknet-tty, opencode-runner wrapper, and
`alknet-container` crate specs can declare their `AccessControl` shapes
against a single coherent model. The block on those specs is lifted.
- The ownership store reuses the established repo/adapter pattern
(ADR-033) — no new shape invented on the storage side. The in-memory
default carries the docker/runner cases with no backend dependency; a
persistence adapter is additive when a use case forces it.
- `OperationSpec` is fully self-describing for authorization: resource
type, action, and which input field drives the resource lookup are all
on the spec. No per-namespace conventions, no handler-level knowledge.
- The proxy-only model keeps the base model simple: spawner owns, proxy
to share, teardown revokes. The `forwarded_for` metadata (ADR-026) is
the end-user-identity carrier; the coordinator handles its own ACL. No
grant API in the core ownership store.
- ADR-017/022 are unchanged. The composition authority stays static; the
ownership store is an additional check, not a modification to the
existing one. The privilege model stays coherent with the ownership
model.
- The `list` case has a clean default ("allow if scoped, filter to owned")
that composes scope-gating and result-filtering without conflating them.
- Teardown is automatic and handler-driven, so the "burn it and start
over" pattern leaves no dangling ACL state.
**Negative:**
- `AccessControl::check` gains a parameter. This is a one-way door —
every call site and test updates. Per the project's decision principle
(implementation workload is a non-issue relative to semantic
correctness and long-term clarity), this is implementation cost, not
semantic cost.
- `OperationSpec` gains a field (`resource_id_path`). Spec-constructing
code (tests, adapter registrations) must add the field. The field is
`Option<String>` — `None` for operations without runtime-spawned
resources, so existing specs are unchanged in shape (the field defaults
to `None`).
- `alknet-core` gains two new traits (`OwnershipProvider`,
`OwnershipStore`) and an in-memory default adapter. Each trait is a
contract downstream crates depend on. The trait shapes are the one-way
doors; the adapter shapes are two-way.
- The handler that manages a resource's lifecycle has an additional
responsibility: calling `record` on spawn and `revoke` on teardown. If
a handler forgets to call `revoke`, stale ownership entries accumulate.
This is the coupling requirement (specific #2) — it's the handler's
job, not the framework's, and the handler framework can provide a
lifecycle-hook abstraction to reduce boilerplate (two-way-door
mechanism work).
- The ownership provider is consulted on every resource-typed
`AccessControl::check`. The in-memory default is a `HashMap` lookup —
negligible. A persistence adapter caches in memory (sync read from
cache, same `ArcSwap` pattern as `ConfigIdentityProvider`), so the hot
path stays sync and fast.
- The proxy-only decision means a downstream app that wants to give an
end user direct access to a runtime-spawned resource must build its own
re-export + ACL layer, rather than using a core grant mechanism. This
is the intended trade — "poking holes" is the app's job, not the core
model's. If a future use case forces the grant pattern, it's additive
(a new trait method), not a redesign.
## Assumptions
1. **The ownership store is per-node.** Each node records its local
ownership. There is no cross-node ownership propagation in the base
model. The hub's "who is this for" mapping is app state, not core
ownership state. (Specific #4c.)
2. **The proxy pattern is sufficient for all current use cases.** The
agent-workspace case, the docker coordinator case, and the runner
wrapper case are all proxy-pattern. No described use case requires the
grant pattern. If one emerges, the grant mechanism is additive (a new
method on the ownership store trait), not a redesign.
3. **The ownership store's read trait is sync.** It is called from
`AccessControl::check` on the dispatch hot path, no `.await`. A
persistence adapter caches in memory and uses honker `NOTIFY` for
invalidation — same `ArcSwap`-backed full-reload pattern as
`ConfigIdentityProvider` (ADR-035).
4. **Ownership is runtime state, meaningless across restarts.** A
container ID from a previous process doesn't exist. The in-memory
default carries no persistence; a persistence adapter is built when a
concrete use case forces it (e.g., a hub that wants fleet ownership to
survive restarts).
5. **The handler that manages a resource's lifecycle is responsible for
calling `record` and `revoke`.** This is the coupling requirement
(specific #4b). The framework can provide a lifecycle-hook abstraction
to reduce boilerplate, but the responsibility is the handler's, not
the framework's.
6. **`CompositionAuthority.resources` (ADR-018) is not involved in the
dynamic ownership check.** It serves its existing purpose (static
resource lists for composition). The dynamic ownership check is the
ownership provider's job. The two are separate and orthogonal.
7. **`AccessControl::check`'s new `ownership` parameter is `None` for
operations without runtime-spawned resources.** This preserves
backward compatibility — operations with static resource sets work
unchanged via the `Identity.resources` fallback path.
## References
- OQ-42: Dynamic Resource Ownership for Runtime-Spawned Resources
(resolved by this ADR — the five sub-questions this ADR writes into
decision text)
- ADR-003: Auth as Shared Core (`IdentityProvider` — the first instance
of the repo/adapter pattern the ownership store reuses; ADR-033 makes
the pattern explicit, ADR-003 is the concrete first instance)
- ADR-032: One-Way Door Decision Framework (the door-type-as-deferral
anti-pattern this ADR's proxy-only decision avoids; the reversal-cost
classification of the grant pattern's additive nature)
- ADR-017: Privilege Model and Authority Context (the static
composition-authority model; this ADR adds an **orthogonal** dynamic
ownership check alongside it — ADR-017's text is **unchanged** per
specific #4d; the system gains a second check, not a modification to
the first)
- ADR-018: Handler Registration, Provenance, and Composition Authority
(`CompositionAuthority.resources` — the static resource list field this
ADR confirms is not involved in the dynamic ownership check —
**unchanged** per specific #4d)
- ADR-025: PeerEntry and Identity.id Decoupling (`Identity.resources` —
the static resource path this ADR's ownership provider extends for
runtime-spawned resources)
- ADR-026: Forwarded-For Identity (Metadata, Not Authority) (`forwarded_for`
— the proxy pattern's end-user-identity carrier; the proxy-only model
relies on this)
- ADR-033: Storage Boundary and Repo/Adapter Pattern (the pattern this ADR
reuses for the ownership store — fourth instance alongside
`IdentityProvider`/`IdentityStore`/`CredentialStore`)
- ADR-035: Concrete Persistence Adapter Shapes (the sync-read + ArcSwap +
honker-NOTIFY shape this ADR's persistence adapter would follow, if
built; `IdentityStore` is the write-trait analogue)
- ADR-022: Call Protocol Client and Adapter Contract (`from_call` — the
adapter that imports a peer's operations and re-registers them locally;
the proxy pattern's re-export mechanism)
- [auth.md](../crates/core/auth.md) (`Identity.resources`,
`AccessControl::check` interaction — both under edit by this decision)
- [operation-registry.md](../crates/call/operation-registry.md)
(`AccessControl`, `OperationSpec` — `resource_id_path` addition)
- [alknet-docker POC summary](../../research/alknet-docker/poc-summary.md)
§"Open Unknowns" #3 (the research finding that surfaced this question)

424
docs/architecture/decisions/012-connectioncredentials-decouple-dial-from-call.md Normal file
View File

@@ -0,0 +1,424 @@
# ADR-012: `ConnectionCredentials` — Decouple the Dial Credentials from the Call Protocol
## Status
Accepted (amends ADR-045 §3 and §5; amends ADR-087's `TlsClientConfig::new`
input framing; amended 2026-07-17 — `CallCredentials` is removed, not
retained in `alknet-call`; `from_call`'s `credentials_auth_token` dead
path removed; `auth_token` is a per-request payload field, not a
call-protocol credential)
## Context
ADR-045 extracted the dial into `AlknetClient` and moved `CallCredentials`
from `alknet-call` to `alknet-core` so the dial would not depend on the
call protocol. The three dial signatures were:
```rust
dial_quic(addr, server_name, alpn, credentials: &CallCredentials) -> Connection
dial_tcp_tls(host, addr, alpn, credentials: &CallCredentials) -> Connection
dial_iroh(node_id: iroh::NodeId, alpn, local_key: &Ed25519SecretKey) -> Connection
```
Two problems surfaced on review:
### Problem 1: the iroh dial signature is asymmetric
`dial_quic` and `dial_tcp_tls` take `&CallCredentials`; `dial_iroh` takes
a bare `&Ed25519SecretKey` + a separate `node_id: iroh::NodeId`. The
asymmetry exists because iroh has its own TLS (it shares the key, not
the rustls config — ADR-087 §3), so the iroh dial bypasses
`TlsClientConfig` and reads the key directly. But the asymmetry forces
the caller to know which dimension of the credential bundle each
transport consumes, and it leaves no path for the iroh dial to receive
the same inputs as the rustls dials — even though all three consume the
same two things: a local identity (key/cert) and an expected remote
identity (fingerprint).
### Problem 2: `CallCredentials` couples the dial to the call protocol
`CallCredentials` carries three dimensions (ADR-022 §7):
1. `tls_identity: Option<TlsIdentity>` — the local node's key/cert
2. `auth_token: Option<AuthToken>` — a call-protocol-level bearer token
3. `remote_identity: Option<RemoteIdentity>` — the expected remote fingerprint
The dial uses only dimensions 1 and 3 (the transport-identity layer).
Dimension 2 (`auth_token`) is a **call-protocol** concept: it correlates
a token to an identity via `IdentityProvider::resolve_from_token` — a
mechanism that exists for two hub-dependent cases where TLS-fingerprint
identity is unavailable:
- **Browsers** — no raw-key support, no client cert the hub can
fingerprint; the browser authenticates via a bearer token over
HTTP/WebSocket, and the hub's `IdentityProvider` resolves it.
- **`alknet/register`** — a native worker that hasn't been enrolled dials
in with no prior peer relationship; a registration token (or open
registration) establishes identity, not a TLS fingerprint.
Both depend on a **hub** running `IdentityProvider` with token-to-identity
mapping. A pure P2P connection (two nodes with raw-key identities) never
needs `auth_token` — the TLS fingerprint IS the identity.
`auth_token` is not a transport credential. It is a per-request field on
`call.requested` payloads (`Dispatcher::resolve_identity` reads
`payload.get("auth_token")`; the `from_call` forwarding handler sets it
via `build_forwarded_payload`). The dial never delivers it to the
protocol take-over — `spawn_dispatch(&self, connection: Connection)`
takes no credentials, and `Connection` (a `Box<dyn BidiStreamSource>`)
carries no `auth_token` field. The `auth_token` in `CallCredentials` is
unused by the dial and dropped after `connect()` in the current code.
By moving `CallCredentials` (with `auth_token` in it) to `alknet-core`
for the dial's benefit, ADR-045 §5 would drag a call-protocol concept
into the shared-types crate *for the dial's benefit* — when the dial
doesn't use it. The dial should consume a transport-level credential
bundle, not a call-protocol one.
### The two identity models
Underneath the three transports, there are two identity-consumption
models, both consuming the same two dimensions:
| Model | Transports | Consumes | What the transport does |
|-------|-----------|----------|------------------------|
| **rustls config** | QUIC (quinn), TCP+TLS (tokio-rustls) | `local_identity` → `TlsClientConfig` (client cert); `remote_identity` → verifier (`FingerprintPinVerifier` / `WebPkiServerVerifier`) | Builds `rustls::ClientConfig`, hands to transport connector |
| **key-native** | iroh, SSH (future — `docs/research/references/ssh/russh/06-usage-patterns.md`) | `local_identity` → `Ed25519SecretKey` → transport's key type (`iroh::SecretKey`, russh key); `remote_identity` → fingerprint → transport's verifier (`NodeId` match, known_hosts) | Reads the key directly; transport handles identity internally |
The difference is *how* each model consumes the dimensions, not *what*
they are. A unified credential bundle carrying just those two dimensions
lets every dial extract what its transport's identity layer needs,
without call-protocol coupling.
## Decision
### `ConnectionCredentials` — the dial's credential bundle
A new type in `alknet-core`, carrying the two transport-identity
dimensions every dial consumes:
```rust
/// Transport-level credentials for an outbound dial. Consumed by
/// `AlknetClient`'s dial methods and (for the server side) by the
/// assembly layer when building transports. Carries only the dimensions
/// the transport's identity layer needs — the local identity (key/cert
/// presented to the transport) and the expected remote identity
/// (fingerprint, driving verifier selection per ADR-034).
///
/// This is NOT the call-protocol credential bundle. The call-protocol
/// `auth_token` (hub-correlated bearer for browsers / `alknet/register`)
/// is a per-request field on `call.requested` payloads, not a
/// transport credential. It stays in the call-protocol layer.
pub struct ConnectionCredentials {
/// The local node's identity (RFC 7250 raw key or X.509), presented
/// to the transport's identity layer. rustls dials → `TlsClientConfig`
/// (client cert via `RawKeyClientCertResolver`); iroh/SSH dials →
/// key directly (`iroh::SecretKey::from_bytes`, russh key).
pub local_identity: Option<TlsIdentity>,
/// Expected identity of the remote node. `Some(fingerprint)` → pin
/// (known peer); `None` → CA verification for X.509 remotes or
/// fail-closed for Ed25519 raw-key remotes (ADR-034 §2/§3). `None`
/// is the public-X.509-endpoint state, not a missing field.
pub remote_identity: Option<RemoteIdentity>,
}
```
`RemoteIdentity` moves with `ConnectionCredentials` to `alknet-core`
(both are transport-level types; the dial and the server-side transport
construction both consume them).
### Unified dial signatures
All three dials take `&ConnectionCredentials`:
```rust
impl AlknetClient {
#[cfg(feature = "quinn")]
pub async fn dial_quic(
&self,
addr: SocketAddr,
server_name: &str,
alpn: &[u8],
creds: &ConnectionCredentials,
) -> Result<Connection, ClientDialError>;
#[cfg(feature = "tcp")]
pub async fn dial_tcp_tls(
&self,
host: &str,
addr: SocketAddr,
alpn: &[u8],
creds: &ConnectionCredentials,
) -> Result<Connection, ClientDialError>;
#[cfg(feature = "iroh")]
pub async fn dial_iroh(
&self,
alpn: &[u8],
creds: &ConnectionCredentials,
) -> Result<Connection, ClientDialError>;
}
```
The `node_id: iroh::NodeId` parameter on `dial_iroh` is removed — it is
derived from `creds.remote_identity.fingerprint` (`ed25519:<hex>` →
`NodeId::from_bytes`), the same way the rustls dials derive their
verifier from `remote_identity`. The consistency is now in both the rule
(ADR-034) and the type.
Each dial extracts what its transport's identity layer needs:
- **rustls dials** (`dial_quic`, `dial_tcp_tls`): `creds.local_identity`
→ `TlsClientConfig::new` (client cert); `creds.remote_identity` →
`ClientVerifierContext` (verifier selection).
- **iroh dial** (`dial_iroh`): `creds.local_identity` →
`Ed25519SecretKey` → `iroh::SecretKey::from_bytes`;
`creds.remote_identity.fingerprint` → `NodeId` (verifier).
### `CallCredentials` is removed (amendment 2026-07-17)
> **This section supersedes the original "CallCredentials stays in
> `alknet-call`" decision.** The original rationale rested on a code
> path that does not exist. The trace below is the correction.
`CallCredentials` is **removed**, not retained. Once the transport
dimensions (`local_identity`, `remote_identity`) move to
`ConnectionCredentials` in `alknet-core`, `CallCredentials` would
reduce to a one-field struct `{ auth_token: Option<AuthToken> }` — and
that field has **no reader**.
**The trace (why the original rationale was wrong).** The original
section claimed the call protocol uses `CallCredentials.auth_token`
because "the `from_call` forwarding handler populates `auth_token` on
outgoing `call.requested` payloads." That chain does not connect:
- `from_call`'s signature is `from_call(connection: &CallConnection,
config: FromCallConfig)` — no `CallCredentials` parameter.
`FromCallConfig` has no credential field.
- The `auth_token` the `from_call` forwarding handlers *can* set on
payloads is sourced from `OpSummary.credentials_auth_token`, an
`Option<String>` that is **hardcoded to `None` at every construction
site** (`from_call.rs:185, 748, 757`). It is not read from
`CallCredentials.auth_token`, and it is a different type
(`Option<String>` vs `Option<AuthToken>`). The two were never
connected, even in intent.
- The consuming side — `Dispatcher::resolve_identity`
(`dispatch.rs:119`) — reads `payload.get("auth_token").as_str()` from
the per-request call payload. It does not read `CallCredentials`.
**Where `auth_token` actually originates.** It is a per-request payload
field, populated by two real paths, neither of which touches
`CallCredentials`:
- **Browsers over WebSocket** — the browser sends `auth_token` directly
in the `call.requested` JSON payload (`websocket/mod.rs:202–206`); the
WS layer (`upgrade.rs:178–181`) passes `envelope.payload` straight to
`dispatch_requested`. The browser is the originator; the WS layer is
a transparent passthrough.
- **HTTP gateway (bearer)** — `gateway/dispatch.rs` resolves the
`Authorization: Bearer` header to an `Identity` at the HTTP boundary
(`resolve_bearer`, line 58) and passes the `Identity` into
`build_root_context`. The call protocol sees the resolved `Identity`,
not the token. `auth_token` does not enter the call payload on this
path.
So `CallCredentials.auth_token` is a write-only field (it has a setter,
`with_auth_token`, and zero readers). `connect()` — `CallCredentials`'s
only consumer — is removed in Phase 5 of the migration. With `connect`
gone, nothing constructs or reads `CallCredentials` except the tests.
**`auth_token`'s two real use cases (confirming no call-protocol
credential bundle is needed):**
1. **HTTP auth** — the inbound case. The HTTP gateway resolves the
bearer token to an `Identity` via `IdentityProvider::resolve_from_token`
at the HTTP boundary. The call protocol receives the `Identity`, not
the token.
2. **Registration** (`alknet/register` native ALPN, `/register` HTTP
endpoint) — a client not yet associated with a hub presents a
one-time registration token; the hub creates a `PeerEntry` (a new
identity based on the fingerprint). Outbound, the vault manages the
token on the client side; inbound, the hub's registration handler
consumes it. Neither path involves `CallCredentials`.
A hub does not "forward with its own token" in the way the original
rationale assumed. Where the hub authenticates to an outside service
(another hub's HTTP interface, an external API), the vault manages that
outbound token — it is not a call-protocol credential. The
`from_call` `credentials_auth_token` path was a future hatch for a
use case that dissolved once `IdentityProvider::resolve_from_token`
solved the inbound identity problem: the hub authenticates as itself
(its `Identity` is on the connection), and the spoke authorizes the hub
as the direct caller. No per-forwarded-call token is needed.
**`from_call`'s `credentials_auth_token` is removed too.** It is the
same family of dead code — an always-`None` field of a different type
than `CallCredentials.auth_token`, never connected to anything. The
`credentials_auth_token` field on `OpSummary`, the `credentials_auth_token`
parameters on `make_forwarding_handler` / `make_streaming_forwarding_handler`,
and the `auth_token` parameter on `build_forwarded_payload` are removed.
The forwarding handlers stop emitting `auth_token` in payloads (which
they never did in practice — the source was always `None`). The two
`from_call` tests asserting the `Some` path
(`build_forwarded_payload_sets_auth_token_when_provided`,
`streaming_forwarding_handler_sets_auth_token_when_provided`) are
removed — they test a code path never exercised in production. If a
future hub needs its own token on forwarded payloads, that is a fresh,
end-to-end-wired feature, not a vestigial path.
**What does NOT move to `alknet-core`:** `ConnectionCredentials` and
`RemoteIdentity` move (the original decision). `CallCredentials` does
not move — it is removed. ADR-045 §5's move of `CallCredentials` to
core is superseded twice over: first by the original ADR-012 (move
`ConnectionCredentials` instead), and now by this amendment (remove
`CallCredentials` entirely). There is no call-protocol credential
bundle; `auth_token` is a per-request payload field, full stop.
### `TlsClientConfig::new` input framing
`TlsClientConfig::new` (ADR-087) takes a `ClientVerifierContext` derived
from the credential bundle's `remote_identity`. The rustls dials extract
`creds.local_identity` and `creds.remote_identity` from
`ConnectionCredentials` and build `ClientVerifierContext` from the latter
— the same extraction ADR-087 described, just from
`ConnectionCredentials` instead of `CallCredentials`. The `auth_token`
dimension is simply not present in `ConnectionCredentials`, so the
"stripped at the TLS boundary" framing (ADR-045's claim that the token
"travels with the Connection") is no longer needed — the token was never
in the dial's credential bundle to strip.
### Future `dial_ssh` validates the shape
An SSH dial (`docs/research/references/ssh/russh/06-usage-patterns.md`)
consumes the same two dimensions:
- `check_server_key(&mut self, key: &ssh_key::PublicKey)` — the verifier
(fingerprint pin against known_hosts = `remote_identity`)
- `authenticate_publickey("user", PrivateKeyWithHashAlg::new(...))` —
local identity (the Ed25519 key = `local_identity`)
- `channel_open_session()` → `Connection::from_bidi` (ADR-007)
`dial_ssh(addr, alpn, creds: &ConnectionCredentials)` fits the same
signature. The SSH host-key verification is fingerprint-pinning
(known_hosts), which is what `remote_identity` carries. The local SSH
key is the same Ed25519 key iroh and raw-key quinn use. The pattern is
general — `ConnectionCredentials` covers it without call-protocol
coupling. SSH itself is unspecced (not yet specced — comes after
channels, tunnels, TTY rework), but the russh usage patterns confirm
the credential dimensions.
## Consequences
**Positive:**
- **The dial is fully decoupled from the call protocol.**
`ConnectionCredentials` carries only transport-identity dimensions;
`alknet-client` has no call-protocol coupling in its credential type.
There is no call-protocol credential bundle — `auth_token` is a
per-request payload field, not a credential.
- **All three dial signatures are unified.** A caller no longer needs to
know that iroh takes a bare key while quinn/tcp take a credential
bundle — all take `&ConnectionCredentials`. The `node_id` parameter on
`dial_iroh` is derived from `remote_identity`, the same extraction
pattern the rustls dials use for the verifier.
- **The `auth_token` spec inaccuracy is fixed.** ADR-045 claimed the
`auth_token` "travels with the `Connection` into the protocol
take-over, where it is sent as the first call-protocol frame." This
was aspirational — `Connection` carries no `auth_token`, and
`spawn_dispatch` takes no credentials. With `CallCredentials` removed,
the claim is not merely unneeded; the field it described was never
read. `auth_token` is a per-request field on `call.requested` payloads,
set by browsers (in the WS payload) or resolved by the HTTP gateway at
its boundary (bearer → `Identity`).
- **`dial_ssh` fits the same shape when it arrives.** The credential
dimensions SSH needs (local key + expected host key) are exactly what
`ConnectionCredentials` carries. No future ADR needed for the SSH dial
signature.
- **A dead credential type and a dead forwarding-token path are removed
(amendment 2026-07-17).** `CallCredentials` is removed (its
`auth_token` field had no reader). `from_call`'s
`credentials_auth_token` is removed (always `None`, different type
than `CallCredentials.auth_token`, never connected). Both were future
hatches from the era before `IdentityProvider::resolve_from_token`
solved the inbound identity problem; the hatches dissolved once it
did. See the amended §"`CallCredentials` is removed" above for the
trace.
**Negative:**
- **`CallCredentials` is removed (a public type).** Callers that
constructed `CallCredentials` (the integration test; any future
assembly-layer code) switch to `ConnectionCredentials` for the dial.
`auth_token`, where needed, is a per-request payload field (browsers
send it in the WS payload; the HTTP gateway resolves bearer →
`Identity` at its boundary). This is expected — `connect()` was
`CallCredentials`'s only consumer and is removed in the same
migration. There are no external consumers (develop branch is a total
rewrite).
- **The assembly layer builds one credential bundle, not two.** Where
ADR-045 had the assembly layer build one `CallCredentials` (and the
original ADR-012 reframed it as two — `ConnectionCredentials` for the
dial + a per-request `auth_token`), the assembly layer now builds
`ConnectionCredentials` for the dial only. `auth_token` is not a
credential the assembly layer constructs; it is a per-request payload
field the browser (or the HTTP gateway's bearer resolution) supplies.
This is fewer types at the assembly site, not more.
- **ADR-045 §5's "CallCredentials moves to core" is superseded twice.**
The original ADR-012 reframed the move target as `ConnectionCredentials`
(not `CallCredentials`); this amendment removes `CallCredentials`
entirely. What moves to `alknet-core`: `ConnectionCredentials` +
`RemoteIdentity`. What does not move: `CallCredentials` (removed, not
relocated). This affects the extraction plan's Phase 0 (additive
credentials move) and Phase 5 (the call prune now removes
`CallCredentials` and the `from_call` dead path, not just `connect`
and the TLS helpers).
## Door type
**One-way.** The dial signatures (`dial_quic` / `dial_tcp_tls` /
`dial_iroh` all taking `&ConnectionCredentials`) are the public API
surface of `alknet-client`. The credential-type decoupling
(`ConnectionCredentials` in core, no call-protocol credential bundle)
determines the dep graph (`alknet-client` depends on `alknet-core` for
`ConnectionCredentials`, not on `alknet-call`). Reversing would mean
re-coupling the dial to the call protocol's credential type and
re-asymmetrizing the iroh dial. The `CallCredentials` removal
(amendment 2026-07-17) is the same door — removing a public type whose
only consumer (`connect`) is removed in the same migration. The crate
is greenfield (Phase 3 of the extraction plan), so the door is still
open now — this ADR records the decisions before implementation.
## References
- ADR-045 — `AlknetClient` native dial seam (§3 dial signatures amended
— all take `&ConnectionCredentials`; §5 move amended —
`ConnectionCredentials`/`RemoteIdentity` move to core, not
`CallCredentials`; §5 further amended 2026-07-17 — `CallCredentials`
removed, not retained in `alknet-call`)
- ADR-087 — `TlsClientConfig` not blocked on dial (input framing
amended — `ClientVerifierContext` derived from
`ConnectionCredentials.remote_identity`, not `CallCredentials`)
- ADR-034 — client-side verifier selection (the rule
`ConnectionCredentials.remote_identity` drives — unchanged)
- ADR-022 §7 — the three credential dimensions (the historical source of
`CallCredentials`'s three fields; the transport dimensions moved to
`ConnectionCredentials`, the `auth_token` dimension is a per-request
payload field, and `CallCredentials` itself is removed)
- `crates/alknet-call/src/protocol/dispatch.rs` —
`Dispatcher::resolve_identity` reads `payload.get("auth_token")`
(per-request, not connection-level — the consumer of `auth_token`)
- `crates/alknet-call/src/client/from_call.rs` — the
`credentials_auth_token` field on `OpSummary` and the
`auth_token` parameter on `build_forwarded_payload` (the removed dead
path; always `None`, different type than `CallCredentials.auth_token`,
never connected)
- `crates/alknet-http/src/gateway/dispatch.rs` — `resolve_bearer` (the
HTTP path: bearer → `Identity` at the boundary; the call layer sees
the identity, not the token)
- `crates/alknet-http/src/websocket/mod.rs` — the WS path:
`auth_token` in the browser's call payload, passed through to
`dispatch_requested` unchanged
- `docs/research/references/ssh/russh/06-usage-patterns.md` — the SSH
client usage patterns (check_server_key + authenticate_publickey)
validating the `ConnectionCredentials` shape for a future `dial_ssh`

78
docs/architecture/decisions/013-irpc-as-call-protocol-foundation.md Normal file
View File

@@ -0,0 +1,78 @@
# ADR-013: irpc as Call Protocol Foundation
## Status
~~Accepted~~ → **Superseded** by [ADR-014](014-irpc-never-integrated-hand-rolled-framing.md)
> **Superseded 2026-07-09.** This ADR accepted "irpc as the call protocol
> foundation" based on the previous architecture's use of irpc. When the
> call protocol was implemented, it turned out that **no `.rs` file in the
> workspace ever imported irpc** — the `irpc` / `irpc-derive` workspace deps
> were a Cargo.toml entry with no corresponding import. The wire protocol
> (`crates/alknet-call/src/protocol/wire.rs`) is hand-rolled length-prefixed
> JSON; the `EventEnvelope` shape was derived from the `@alkdev/pubsub`
> TypeScript prior art (ADR-033), not from irpc. ADR-014 supersedes this
> ADR and records the actual state: hand-rolled framing, no irpc
> integration. The architectural properties this ADR sought (proven
> length-prefixed JSON framing, cross-language JSON wire format, streaming)
> are preserved by the hand-rolled implementation. The text below is kept
> as the historical record of the decision that was made (and never
> implemented as stated).
## Context
The call protocol (alknet-call) provides structured RPC — operations, request/response, streaming subscriptions, and pub/sub. This is the primary interface for programmatic interaction with an alknet node. It needs to work across platforms: Rust clients, TypeScript/JavaScript clients (via NAPI), WASM targets, and any language that can speak the wire format.
The previous implementation used `irpc` for the call protocol's operation registry, framing, and service patterns. irpc provides:
- An operation registry with schema-based discovery
- Length-prefixed JSON framing (EventEnvelope)
- Request/response and streaming patterns
- Type-safe operation definitions via derive macros
The call protocol is derived from a TypeScript implementation (`@alkdev/operations`, `@alkdev/pubsub`) that informed the design of the operation registry, EventEnvelope framing, and adapter patterns (from_openapi, from_mcp, from_call). This bidirectional composition capability is strategically important. The TypeScript code is a reference that informed the Rust design — it is not a parallel implementation (see ADR-033).
## Decision
alknet-call uses irpc as its foundation. The `CallAdapter` implements `ProtocolHandler` on ALPN `alknet/call` and delegates to irpc's operation registry, framing, and dispatch.
irpc is not replaced or wrapped in an abstraction layer — it IS the call protocol's core. The relationship is:
- irpc provides: operation registry, schema discovery, frame encoding/decoding, request/response routing, streaming
- alknet-call provides: the ProtocolHandler adapter (BiStream → irpc), AuthContext integration, access control checks, the ALPN registration
This means:
- The wire format is irpc's EventEnvelope framing — length-prefixed JSON
- Operation schemas follow irpc's schema model — JSON Schema compatible
- The TypeScript operation and pub/sub patterns that can import OpenAPI schemas, wrap MCP servers, and expose operations as endpoints are supported at the protocol level — the adapter contract (from_*, to_*) is defined in Rust (see ADR-033)
- Future NAPI and WASM clients speak the same wire format — alknet-napi projects the Rust call protocol client to Node.js; a browser SDK can be adapted from the existing TypeScript code
The `VaultProtocol` in alknet-vault previously used irpc as its service
protocol. ADR-025 dropped irpc from the vault — the vault uses direct method
calls on `VaultServiceHandle`, not irpc dispatch. irpc remains the
foundation for alknet-*call* (the call protocol), not for alknet-*vault*.
See ADR-025 for the rationale (security default inversion: the vault is
local-only by construction, not remote-capable by default).
## Consequences
**Positive:**
- Proven operation registry and framing — irpc is already tested in production (iroh uses it)
- JSON Schema compatible — OpenAPI import, MCP tool exposure, cross-language client generation
- No need to design a custom RPC wire format — irpc's is already battle-tested
- The call protocol inherits irpc's streaming and subscription patterns
**Negative:**
- alknet-call depends on irpc — if irpc has limitations or bugs, we're affected (mitigated: irpc is lightweight and we can fork if needed)
- JSON framing is not the most compact binary format — for high-throughput scenarios, a binary codec could be added later as an irpc extension
- irpc's derive macros add a compilation dependency — but this is standard for Rust RPC frameworks
- The call protocol's cross-language story depends on irpc's wire format being documented and stable (mitigated: it's length-prefixed JSON, which is inherently cross-language)
## References
- **Superseding ADR**: [ADR-014](014-irpc-never-integrated-hand-rolled-framing.md) — irpc was never integrated; hand-rolled framing is the actual state
- ADR-033: Rust as canonical implementation (the `@alkdev/pubsub` prior art the `EventEnvelope` shape was actually derived from)
- ADR-025: Vault local-only dispatch (dropped irpc from the vault; ADR-014 confirms irpc was never in alknet-call either)
- Pivot proposal: `docs/research/pivot/alpn-service-architecture.md`
- ADR-031: Crate decomposition
- ADR-003: Auth as shared core (IdentityProvider)
- Call protocol wire format (actual): `crates/alknet-call/src/protocol/wire.rs`
- The previous architecture had an equivalent decision in ADR-019 (bidirectional call protocol with EventEnvelope framing), which is archived in the reference implementation at `/workspace/@alkdev/alknet-main/`.

164
docs/architecture/decisions/014-irpc-never-integrated-hand-rolled-framing.md Normal file
View File

@@ -0,0 +1,164 @@
# ADR-014: irpc Was Never Integrated — Hand-Rolled EventEnvelope Framing
## Status
Accepted
## Context
ADR-013 accepted "irpc as the call protocol foundation" based on the
previous architecture's use of irpc. When the call protocol was implemented,
it turned out that **no `.rs` file in the workspace ever imported irpc**.
The workspace `Cargo.toml` declared `irpc = "0.16"` / `irpc-derive = "0.16"`
as workspace dependencies, and `crates/alknet-call/Cargo.toml` declared
`irpc = { workspace = true }`, but the import was never written. The wire
protocol (`crates/alknet-call/src/protocol/wire.rs`) is hand-rolled
length-prefixed JSON — 4-byte big-endian length prefix + UTF-8 JSON body —
not an irpc service.
This was discovered when building an external app against the crates: the
`irpc 0.16` workspace dep was a version-gap blocker for `alknet-blobs` (which
pulls `irpc 0.17` transitively via `iroh-blobs 0.103`). A grep for any `irpc`
import in the workspace found zero hits — the dep was dead weight carried
over from the previous architecture without verification.
The framing, operation registry, dispatch, and subscription patterns that
ADR-013 attributed to irpc are all hand-rolled in alknet-call:
- **Framing**: `FrameFramedReader` / `FrameFramedWriter` in `wire.rs` —
length-prefixed JSON, hand-written against `tokio::io::AsyncRead`/
`AsyncWrite`. Not an irpc service.
- **Operation registry**: `OperationSpec`, `Handler`, `OperationRegistry`,
`AccessControl` — hand-rolled in alknet-call, not irpc's `Service` trait.
- **Event types**: `call.requested`, `call.responded`, `call.completed`,
`call.aborted`, `call.error` — the alknet call protocol's own event
vocabulary, not irpc's.
- **Subscription/streaming**: `StreamingHandler` / `invoke_streaming()`
(ADR-021) — hand-rolled, not irpc's streaming patterns.
The `EventEnvelope { type, id, payload }` shape was derived from the
`@alkdev/pubsub` TypeScript `EventEnvelope` (`/workspace/@alkdev/pubsub/src/
types.ts`), not from irpc. ADR-013's claim that "the wire format is irpc's
EventEnvelope framing" was wrong — irpc was never imported, and the envelope
shape has a different origin (the pubsub prior art, ADR-033). The framing
coincidentally resembles irpc's (both are length-prefixed JSON), which is
how the misattribution went unnoticed.
### What ADR-013 got right
Despite the irpc misattribution, ADR-013's *architectural* decisions are
correct and stand unchanged:
- The call protocol uses length-prefixed JSON `EventEnvelope` framing
(hand-rolled, not irpc-supplied).
- The wire format is cross-language and consumable from TypeScript, Python,
any language (JSON is inherently cross-language — ADR-013's "mitigated:
it's length-prefixed JSON" note was the load-bearing point, not the irpc
attribution).
- Operations use JSON Schema discovery. The `OperationSpec` shape is
hand-rolled, JSON-Schema-compatible — the same property ADR-013 attributed
to irpc, achieved without irpc.
### Why a new ADR rather than an amendment
ADR-013's Decision and Consequences are built on the premise "alknet-call
uses irpc as its foundation — irpc IS the call protocol's core." That
premise is false. Amending ADR-013 to say "actually it's hand-rolled" would
leave an ADR whose Context, Decision, and Consequences sections all argue
for a choice that was never made. The correct record is: ADR-013 is
superseded; the call protocol uses hand-rolled framing (this ADR-014); the
architectural properties ADR-013 sought (proven framing, cross-language
JSON, streaming) are preserved, but the mechanism is hand-rolled, not
irpc-sourced.
### The dead dep removal
The `irpc` / `irpc-derive` workspace deps and the `alknet-call` consumer dep
were removed in commit `668d777` (2026-07-09). `irpc` may be re-added as
`0.17` when `alknet-blobs` lands (it pulls `irpc 0.17` transitively via
`iroh-blobs 0.103`), but that would be a *transitive* dependency of
`alknet-blobs`, not a direct dependency of `alknet-call` — alknet-call does
not import irpc and has no plans to. See
[`docs/research/transport-generalization/findings.md`](../../research/transport-generalization/findings.md)
§3.1 for the removal trace.
## Decision
1. **ADR-013 is superseded.** The call protocol does not use irpc. irpc was
never imported by any `.rs` file in the workspace. The dead `irpc` /
`irpc-derive` workspace and crate deps are removed.
2. **The call protocol uses hand-rolled `EventEnvelope` framing.** The wire
format is length-prefixed JSON (4-byte big-endian length + UTF-8 JSON
body), implemented in `crates/alknet-call/src/protocol/wire.rs`. The
`EventEnvelope { type, id, payload }` shape was derived from the
`@alkdev/pubsub` TypeScript prior art (ADR-033), not from irpc. The
framing, operation registry, dispatch, and streaming patterns are all
hand-rolled in alknet-call.
3. **The architectural properties ADR-013 sought are preserved by the
hand-rolled implementation:**
- Proven framing — length-prefixed JSON is a well-understood,
battle-tested pattern; the hand-rolled implementation is tested (207
lib + 2 integration tests passing).
- Cross-language — JSON is inherently consumable from any language;
NAPI, WASM, and browser clients speak the same wire format.
- Streaming — `StreamingHandler` / `invoke_streaming()` (ADR-021) provide
the subscription/streaming patterns ADR-013 attributed to irpc,
hand-rolled.
4. **irpc is not a planned dependency for alknet-call.** If `alknet-blobs`
pulls irpc transitively, it will be a transitive dependency of that
crate, not a direct dependency of alknet-call. alknet-call's framing,
registry, and dispatch are hand-rolled and will remain so. The "mitigated:
irpc is lightweight and we can fork if needed" caveat in ADR-013 is moot
— there is nothing to fork because nothing was integrated.
5. **The vault's irpc drop (ADR-025) stands.** ADR-025 dropped irpc from
alknet-vault. With this ADR, irpc is also confirmed absent from
alknet-call. The vault and call decisions are now consistent: neither
crate uses irpc. The only difference is that ADR-025 *removed* a real
(but unused-for-its-primary-path) irpc dependency from the vault, while
this ADR records that alknet-call's irpc dependency was never integrated
at all — it was a Cargo.toml entry with no corresponding import.
## Consequences
**Positive:**
- The spec matches the code. ADR-013's irpc claims were a spec/code
divergence that surfaced only when the `irpc 0.16` version gap blocked
`alknet-blobs`. This ADR closes the divergence.
- `alknet-blobs` is unblocked — the dead `irpc 0.16` workspace dep is
gone; `iroh-blobs 0.103` (which pulls `irpc 0.17` transitively) no longer
conflicts with a workspace-pinned older irpc.
- The call protocol's framing, registry, and dispatch are documented
accurately as hand-rolled — readers of the spec aren't sent looking for
an irpc integration that doesn't exist.
- The cross-language story is unchanged: JSON wire format, JSON Schema
discovery. The mechanism changed (hand-rolled vs irpc), but the property
ADR-013 sought is preserved.
**Negative:**
- The call protocol does not inherit irpc's testing or production pedigree
for its framing. Mitigation: length-prefixed JSON is a trivial,
well-understood pattern; the hand-rolled implementation is tested; and
the framing is small enough to audit completely (~30 lines in `wire.rs`).
- ADR-013's claim that "the call protocol inherits irpc's streaming and
subscription patterns" was wrong — those patterns are hand-rolled
(ADR-021). The streaming implementation is younger and less battle-tested
than irpc's would have been, but it is also simpler and fully owned.
## References
- ADR-013: irpc as call protocol foundation (superseded by this ADR)
- ADR-025: Vault local-only dispatch (dropped irpc from the vault; this ADR
records irpc was never integrated into alknet-call either)
- ADR-033: Rust as canonical implementation (the `@alkdev/pubsub` prior art
that the `EventEnvelope` shape was actually derived from)
- ADR-021: Streaming handler for subscriptions (the hand-rolled streaming
dispatch path)
- Call protocol wire format: `crates/alknet-call/src/protocol/wire.rs`
- Transport generalization findings:
[`docs/research/transport-generalization/findings.md`](../../research/transport-generalization/findings.md)
§3.1 (dead `irpc` dep removal)
- Removal commit: `668d777` (2026-07-09)

56
docs/architecture/decisions/015-call-protocol-stream-model.md Normal file
View File

@@ -0,0 +1,56 @@
# ADR-015: Call Protocol Stream Model
## Status
Accepted
## Context
The call protocol (alknet-call) operates on a QUIC connection with ALPN `alknet/call`. Within that connection, QUIC provides bidirectional streams. The question is how the call protocol uses those streams and how it correlates requests with responses — especially when both sides can initiate calls.
The reference implementation used `EventEnvelope` framing with a `PendingRequestMap` that correlates `call.requested` events to `call.responded` events by request ID, regardless of which stream carries them. This works well but the relationship between streams and operations was underspecified.
OQ-07 asked: "What is the scope of the call protocol within a connection? Should operations be multiplexed within a single stream, or should each operation get its own stream?"
## Decision
The call protocol uses **bidirectional QUIC streams with EventEnvelope framing and ID-based correlation**. The protocol does not prescribe a stream usage pattern — it works with any arrangement:
1. **EventEnvelope on every stream** — every bidirectional stream opened on the `alknet/call` connection carries length-prefixed JSON `EventEnvelope` messages. The five event types (`call.requested`, `call.responded`, `call.completed`, `call.aborted`, `call.error`) are the protocol primitives.
2. **PendingRequestMap correlates by ID, not by stream** — the `id` field in `EventEnvelope` correlates requests with responses. A response on stream 5 can fulfill a request sent on stream 3. The PendingRequestMap is keyed by request ID.
3. **Protocol is symmetric** — both sides of the connection can `open_bi()` to initiate calls and `accept_bi()` to receive them. The server calling a client operation uses the same EventEnvelope format and the same correlation mechanism.
4. **Top-level protocol operations** — the call protocol defines four operations that map to EventEnvelope event patterns:
- **call**: `call.requested` → `call.responded` (one response) or `call.error`
- **subscribe**: `call.requested` → one or more `call.responded` → `call.completed` or `call.aborted`
- **batch**: multiple `call.requested` events (with correlated IDs) → multiple `call.responded` events
- **schema**: `call.requested` (name `/services/list` or `/services/schema`) → `call.responded`
5. **Stream usage is the client's choice** — a client may open one stream per operation, one stream for all operations, or any mix. The protocol is stream-agnostic. The server accepts streams and processes EventEnvelopes regardless of which stream they arrive on.
This resolves OQ-07: the call protocol's scope within a connection is the full operation registry. One `alknet/call` connection gives access to all operations (call, subscribe, batch, schema). QUIC's built-in stream multiplexing handles concurrency — the protocol doesn't need to impose additional multiplexing.
## Consequences
**Positive:**
- Simple mental model: one connection, full access, stream-agnostic correlation
- The protocol works the same way regardless of stream usage — no "right" way to use streams
- Bidirectional calls are natural — either side can open a stream and send `call.requested`
- PendingRequestMap from the reference implementation carries forward without modification
- QUIC's stream multiplexing provides natural flow control and head-of-line blocking avoidance
- The top-level operations (call, subscribe, batch, schema) are protocol primitives, not separate ALPNs
**Negative:**
- Clients that multiplex many operations on one stream must manage request IDs carefully — but this is standard RPC practice
- The PendingRequestMap requires timeout-based cleanup to prevent memory leaks from abandoned requests — but this is already implemented and tested in the reference
- No built-in stream-level backpressure per operation when multiple operations share a stream — but QUIC provides connection-level and stream-level flow control
## References
- ADR-013: irpc as call protocol foundation
- ADR-004: ALPN string convention and connection model
- ADR-005: BiStream type definition
- OQ-07: Call protocol scope within a connection (resolved by this ADR)
- Reference implementation: `/workspace/@alkdev/alknet-main/crates/alknet-core/src/call/`

417
docs/architecture/decisions/016-operation-error-schemas.md Normal file
View File

@@ -0,0 +1,417 @@
# ADR-016: Operation Error Schemas
## Status
Accepted (amended by ADR-021 — protocol-level code list extended to six)
## Context
The `OperationSpec` in alknet-call has `input_schema` and `output_schema` but
no `error_schemas`. The `call.error` payload (call-protocol.md L128–134)
carries a `code` and `message`, where `code` is one of six infrastructure
codes: `NOT_FOUND`, `FORBIDDEN`, `INVALID_INPUT`, `INVALID_OPERATION_TYPE`,
`INTERNAL`, `TIMEOUT`.
These six codes cover **protocol-level failures** — the call protocol
itself can always fail to find an operation, deny access, reject bad input,
reject the wrong dispatch method for the operation type, time out, or hit
an internal error. They are emitted by the dispatch machinery (the registry,
the adapter), not by operation handlers. `INVALID_OPERATION_TYPE` was added
by ADR-021 (streaming handler for subscriptions — `invoke()` called on a
`Subscription`, or `invoke_streaming()` on a `Query`/`Mutation`).
But operations also have **domain-level failures** that are not covered:
- `/fs/readFile` can fail because the file doesn't exist, the path is
invalid, or the caller lacks OS-level read permission. These are
operation-specific failures distinct from the protocol-level
`INVALID_INPUT` (schema mismatch) or `FORBIDDEN` (scope mismatch).
- `/vastai/createMachine` can fail because the account has insufficient
credits, the machine type is unavailable in the requested region, or the
upstream API rate-limited the request.
- `/agent/chat` can fail because the LLM provider returned an error, the
context window overflowed, or the model refused the request.
Today, these failures collapse into `INTERNAL` with a `message` string.
A client calling `/fs/readFile` has no way to know from the schema that it
might return `FILE_NOT_FOUND` vs `PERMISSION_DENIED` vs `INVALID_PATH`. The
caller has to parse `message` strings — the exact anti-pattern that typed
RPC is meant to avoid. This is a **type safety gap**: inputs and outputs are
typed, but errors are untyped strings.
### Why this matters for adapters
OpenAPI specs naturally include error information — response status codes
with schemas (e.g., `404: { schema: NotFoundError }`, `422: { schema:
ValidationError }`). MCP tool definitions carry error descriptions. The
`from_openapi` adapter (ADR-022 L113–124) imports operations and mirrors
"the remote operation's name, namespace, type, schemas, and access control"
— but with no error schema field, error responses from the OpenAPI source
are dropped on import. `to_openapi` has nowhere to project error information
to. The same gap applies to `from_mcp`/`to_mcp`.
An OpenAPI operation that declares:
```yaml
responses:
'200': { schema: MachineList }
'401': { schema: AuthError }
'429': { schema: RateLimitError }
```
cannot be faithfully represented in alknet's `OperationSpec` today. The
adapter would import the `200` output schema and drop the error schemas —
a lossy import that silently discards the operation's failure contract.
### Prior art
The TypeScript reference (`/workspace/@alkdev/operations/src/types.ts`
L38–47, L94, L112) defines `ErrorDefinitionSchema` and an optional
`errorSchemas?: ErrorDefinition[]` on `OperationSpec`:
```typescript
export const ErrorDefinitionSchema = Type.Object({
code: Type.String({ description: "Error Code e.g., INVALID_INPUT, NOT_FOUND, UNAUTHORIZED" }),
description: Type.String(),
schema: Type.Unknown(),
httpStatus: Type.Optional(Type.Number()),
});
```
The `mapError()` function (`error.ts` L25–51) matches thrown errors against
the declared error schemas by code prefix — if a handler throws an error
whose message starts with a declared code, `mapError` rewrites it to a
typed `CallError` with that code. This is a proven pattern: operations
declare their error contract, the dispatch machinery maps runtime failures
to the declared codes, and clients get typed errors instead of string
parsing.
The translator agent omitted `errorSchemas` from the Rust spec, likely
because it's `Optional` in the TS schema (so dropping it doesn't break the
happy path) and because error schemas are semantically different from
input/output schemas (an operation returns one output but could return any
of several errors). That's a reasonable judgment call for a first
translation pass, but it leaves a real gap for adapters and clients.
### The general principle
This is the same principle as the Safe Exit protocol in the SDD process
(docs/sdd_process.md L19, L423): **make failure a typed, declared thing
rather than an untyped exception that crashes into whoever's listening.**
An operation that declares "I can fail with `FILE_NOT_FOUND`" is the same
shape as an agent that declares "I can fail with `TASK_AMBIGUOUS`" — both
turn an unknown unknown into a known known that the caller can handle
deliberately.
Complex systems survive not because every component is reliable, but
because failure is expected and typed. Cells have apoptosis (a declared
failure mode that protects the organism). Operations have error schemas (a
declared failure mode that lets the caller handle it). The alternative —
components that fail with untyped strings — is how you get brittle clients
that string-match error messages and break when the message wording
changes.
## Decision
### 1. `OperationSpec` gains an optional `error_schemas` field
```rust
pub struct OperationSpec {
pub name: String,
pub namespace: String,
pub op_type: OperationType,
pub visibility: Visibility,
pub input_schema: Value,
pub output_schema: Value,
pub access_control: AccessControl,
pub error_schemas: Vec<ErrorDefinition>, // NEW — empty vec = no declared errors
}
pub struct ErrorDefinition {
/// Machine-readable error code. e.g., "FILE_NOT_FOUND", "RATE_LIMITED",
/// "INSUFFICIENT_CREDITS". Distinct from the protocol-level codes
/// (NOT_FOUND, FORBIDDEN, etc.) — these are operation-level domain codes.
pub code: String,
/// Human-readable description of when this error occurs.
pub description: String,
/// JSON Schema for the error detail payload. The `call.error` event's
/// `details` field conforms to this schema when this error code is
/// returned. `Value` (serde_json::Value) carrying a JSON Schema, same
/// as input_schema/output_schema.
pub schema: Value,
/// HTTP status code for adapter projection. `from_openapi` maps OpenAPI
/// response status codes to error definitions; `to_openapi` projects
/// error definitions back to response status codes. Optional — not all
/// error sources are HTTP-backed.
pub http_status: Option<u16>,
}
```
`error_schemas` is a `Vec<ErrorDefinition>`, not `Option<Vec<...>>`. An
empty vec means "this operation declares no specific domain errors" (it may
still fail with protocol-level codes like `INTERNAL`). This avoids the
`None` vs `Some([])` ambiguity and matches the TypeScript reference's
optional-array convention.
### 2. The `call.error` payload gains an optional `details` field
```json
{
"code": "FILE_NOT_FOUND",
"message": "file not found: /etc/nonexistent",
"retryable": false,
"details": { "path": "/etc/nonexistent", "errno": 2 }
}
```
- `code` — the error code. Either a protocol-level code (`NOT_FOUND`,
`FORBIDDEN`, `INVALID_INPUT`, `INVALID_OPERATION_TYPE`, `INTERNAL`,
`TIMEOUT`) or an operation-level domain code from `error_schemas` (e.g.,
`FILE_NOT_FOUND`, `RATE_LIMITED`).
- `message` — human-readable error message. Unstructured — for logging and
debugging, not for programmatic handling. Clients should switch on
`code`, not parse `message`.
- `retryable` — whether the caller should retry. `true` for transient
failures (`TIMEOUT`, `RATE_LIMITED`), `false` for permanent ones
(`NOT_FOUND`, `FORBIDDEN`, `FILE_NOT_FOUND`).
- `details` — optional. When the error code matches a declared
`ErrorDefinition`, `details` conforms to that definition's `schema`. When
the error is protocol-level (`NOT_FOUND`, `FORBIDDEN`, etc.), `details`
is absent or carries protocol-specific context (e.g., the operation name
for `NOT_FOUND`). This field is the typed error payload — it's what
makes errors structured instead of string-matched.
### 3. Protocol-level vs operation-level error codes
The six existing codes are **protocol-level** — emitted by the dispatch
machinery, not by handlers:
| Code | Emitted by | Meaning |
|------|-----------|---------|
| `NOT_FOUND` | Registry | Operation not registered (or Internal op called from wire) |
| `FORBIDDEN` | Registry / ACL | Caller lacks required scopes, or unauthenticated |
| `INVALID_INPUT` | Registry | Input doesn't match `input_schema` |
| `INVALID_OPERATION_TYPE` | Registry / `OperationEnv` | Wrong dispatch path for the operation's type (`invoke()` on a `Subscription`, `invoke_streaming()` on a `Query`/`Mutation`, or `OperationEnv::invoke()` on a `Subscription` during composition — ADR-021) |
| `INTERNAL` | Registry / Adapter | Handler panic, unhandled error, connection failure |
| `TIMEOUT` | Adapter | Request timed out |
Operation-level domain codes are emitted by **handlers** — the operation's
own logic determines what went wrong. They are declared in `error_schemas`
and appear in the `code` field of `call.error`. Examples: `FILE_NOT_FOUND`,
`PERMISSION_DENIED`, `RATE_LIMITED`, `INSUFFICIENT_CREDITS`,
`CONTEXT_OVERFLOW`.
The two namespaces are distinct but share the `code` field. Clients
should handle protocol-level codes uniformly (they mean the same thing
regardless of operation) and operation-level codes per-operation (they
mean what the operation's `error_schemas` says they mean). Unknown codes
— whether a future protocol code or an undeclared operation code — should
be treated as `INTERNAL` with `retryable: false` (same as the current
guidance in call-protocol.md L143).
### 4. Handler error mapping
When a handler returns an error, the dispatch machinery maps it to a
`call.error` event. The mapping:
1. If the handler returns a structured error with a `code` that matches a
declared `ErrorDefinition.code`, the `call.error` carries that code and
the error's detail payload (validated against the definition's `schema`).
2. If the handler returns a structured error with a `code` that doesn't
match any declared `ErrorDefinition`, the `call.error` carries
`INTERNAL` with the original code in `details`. This is an undeclared
error — the handler returned a typed error but didn't declare it.
3. If the handler returns an unstructured error (a string, a generic
`Error`, a panic), the `call.error` carries `INTERNAL` with
`retryable: false`. This is the current behavior for all handler
errors.
The TypeScript `mapError()` function (error.ts L25–51) implements case 2
and 3 by matching error messages against declared codes. The Rust
implementation can use a typed error return from the handler (`Result<Value,
CallError>` where `CallError` carries a `code`), which is cleaner than
message-string matching — the handler returns a typed error, the registry
checks whether the code is declared, and the `call.error` is constructed
accordingly.
### 5. `from_openapi` and `to_openapi` error fidelity
`from_openapi` maps OpenAPI response status codes to `ErrorDefinition`s:
```rust
// OpenAPI: 404: { schema: NotFoundError }
// → ErrorDefinition { code: "HTTP_404", http_status: Some(404), schema: NotFoundError }
```
**Normative rule (review #002 W20)**: `from_openapi` must not produce error
codes that collide with the six protocol-level codes (`NOT_FOUND`,
`FORBIDDEN`, `INVALID_INPUT`, `INVALID_OPERATION_TYPE`, `INTERNAL`,
`TIMEOUT`). The adapter prefixes
imported error codes with `HTTP_` and the status number (e.g., `HTTP_404`,
`HTTP_429`) to avoid collision. This is a requirement for the adapter, not
a naming convention — the `from_openapi` example above was previously shown
producing `NOT_FOUND` from a 404, which collided with the protocol-level
`NOT_FOUND` (operation not registered). The `details` field disambiguates
in practice (present for operation-level, absent for protocol-level), but
ADR-016 says "clients should switch on `code`, not parse `message`" — so
the `code` alone must be unambiguous. Operations that hand-write their own
`ErrorDefinition`s should use domain-specific codes (`FILE_NOT_FOUND`,
`RATE_LIMITED`) rather than reusing protocol codes.
The adapter maps the OpenAPI error schema to alknet's JSON Schema format
(same conversion as input/output schemas). The `http_status` field records
the original status code so `to_openapi` can project it back.
`to_openapi` projects `error_schemas` back to OpenAPI response definitions:
```yaml
responses:
'200': { schema: <output_schema> }
'404': { schema: <error_schemas[0].schema> } # where http_status = 404
'429': { schema: <error_schemas[1].schema> } # where http_status = 429
```
This makes the adapter contract from ADR-022 faithful on the error axis —
no silent dropping of error contracts.
`from_mcp` and `to_mcp` follow the same pattern: MCP tool definitions carry
error descriptions, and the adapters map them to/from `ErrorDefinition`s.
### 6. `services/schema` exposes error schemas
`services/schema` returns the full `OperationSpec` including `error_schemas`.
A client querying `/services/schema` for `/fs/readFile` gets:
```json
{
"name": "fs/readFile",
"namespace": "fs",
"op_type": "query",
"input_schema": { ... },
"output_schema": { ... },
"error_schemas": [
{ "code": "FILE_NOT_FOUND", "description": "The file does not exist",
"schema": { "type": "object", "properties": { "path": { "type": "string" } } },
"http_status": null },
{ "code": "PERMISSION_DENIED", "description": "OS-level read permission denied",
"schema": { "type": "object", "properties": { "path": { "type": "string" }, "errno": { "type": "integer" } } },
"http_status": null }
]
}
```
This enables client code generation: a TypeScript or Rust client generator
reading the schema can produce a typed `Result<Output, FsReadFileError>`
enum instead of a generic `Result<Output, string>`.
## Consequences
**Positive:**
- Operations declare their failure modes. Clients get typed errors instead
of string-matched messages. This is the same type-safety property that
`input_schema` and `output_schema` provide, extended to the error axis.
- `from_openapi` and `to_openapi` are faithful on the error axis. An
OpenAPI operation's error contract is no longer silently dropped on
import or absent on export. The adapter contract from ADR-022 is now
complete.
- Client code generation can produce typed error enums. A client calling
`/fs/readFile` can match on `FILE_NOT_FOUND` vs `PERMISSION_DENIED`
instead of parsing `message` strings.
- The protocol-level vs operation-level distinction is explicit. Protocol
codes (`NOT_FOUND`, `FORBIDDEN`, etc.) mean the same thing regardless of
operation. Operation codes (`FILE_NOT_FOUND`, `RATE_LIMITED`) mean what
the operation declares. No conflation.
- The `details` field carries structured error context that conforms to a
schema — the error payload is typed, not a bare string. This enables
programmatic error handling (retry logic, user-facing error messages,
logging) without string parsing.
- The principle generalizes: making failure a typed, declared thing is the
same pattern as the SDD process's Safe Exit protocol (typed agent
failure) and the same pattern complex biological systems use (apoptosis
as a declared cell failure mode). The more components declare their
failure modes, the more robust the system.
**Negative:**
- `OperationSpec` gains a field. Operations that don't declare errors
(empty `error_schemas` vec) still work — the field is additive. But
operations that *should* declare errors and don't will produce `INTERNAL`
with `retryable: false`, same as today. The gap is visible but not
enforced — an operation can ship without error schemas and clients get
untyped errors for it. This is a documentation/guidance issue, not a
type-system issue.
- The `call.error` payload gains a `details` field. This is a wire-format
addition. Existing clients that only read `code` and `message` are
unaffected (they ignore `details`). New clients can read `details` for
structured error context. This is backward-compatible — `details` is
optional and absent for protocol-level errors.
- Handler error mapping adds a step to the dispatch path: the registry
checks whether the handler's error code matches a declared
`ErrorDefinition`. This is a `HashMap` lookup by code — negligible cost.
- The `http_status` field on `ErrorDefinition` is HTTP-specific. Operations
that aren't HTTP-backed (local, session, from_mcp) leave it as `None`.
This is a pragmatic choice: `from_openapi`/`to_openapi` need it, and it's
optional for everything else. A future non-HTTP adapter that needs a
different error projection field would add it — but `http_status` covers
the immediate use case.
- The TypeScript `mapError()` uses message-string matching to map thrown
errors to codes. The Rust implementation can do better (typed `CallError`
return from handlers), but this means the `Handler` type's return is
`Result<Value, CallError>` rather than `Result<Value, Box<dyn Error>>`.
This is a cleaner API but a slight constraint on handler authors — they
return typed errors, not generic ones. Mitigated: `CallError::internal()`
is available for errors that don't fit a declared code.
## Assumptions
1. **Operations can enumerate their meaningful failure modes at
registration time.** If an operation has failure modes that are only
discoverable at runtime (e.g., a dynamic API that returns novel error
codes), those would be `INTERNAL` with `details` carrying the upstream
error. The assumption is that most operations have a knowable set of
domain errors.
2. **Error codes are stable per operation.** Once an operation declares
`FILE_NOT_FOUND`, clients depend on that code. Changing it (renaming to
`NOT_FOUND_FILE`) is a breaking change for clients that match on it.
This is the same stability property as `input_schema` and
`output_schema` — the operation's interface is its contract. Adding new
error codes is additive (clients that don't know the new code treat it
as `INTERNAL`); removing or renaming codes is breaking.
3. **Protocol-level codes are distinct from operation-level codes.** If an
operation declares a code that collides with a protocol code (e.g., an
operation declares `NOT_FOUND` as a domain error), the protocol code
takes precedence in the dispatch machinery (the registry's `NOT_FOUND`
for "operation not registered" is emitted before the handler runs). The
assumption is that operations use domain-specific codes (`FILE_NOT_FOUND`)
rather than reusing protocol codes (`NOT_FOUND`). This is a naming
convention, not a type-system enforcement.
4. **`details` is optional and backward-compatible.** Existing clients that
ignore `details` continue to work. New clients read `details` for
structured context. The wire format addition is additive.
## References
- ADR-022: Call protocol client and adapter contract (adapter fidelity —
this ADR makes `from_openapi`/`to_openapi` faithful on the error axis)
- ADR-010: Secret material flow (the `details` field must not carry secret
material — same constraint as `metadata`)
- ADR-017: Privilege model (the `FORBIDDEN` protocol code covers ACL
denial; operation-level `PERMISSION_DENIED` is a distinct domain error
for OS-level permission issues)
- docs/reviews/001-pre-implementation-architecture-sanity-check.md
(finding C5, which this ADR resolves)
- ADR-021: Streaming handler for subscriptions (amends this ADR's
protocol-level code list — `INVALID_OPERATION_TYPE` added as the sixth
protocol-level code)
- docs/sdd_process.md L19, L423 (Safe Exit protocol — the general principle
of making failure typed and declared)
- TypeScript reference: `/workspace/@alkdev/operations/src/types.ts`
L38–47 (`ErrorDefinitionSchema`), L94, L112 (`errorSchemas` on
`OperationSpec`), `error.ts` L25–51 (`mapError`)

310
docs/architecture/decisions/017-privilege-model-and-authority-context.md Normal file
View File

@@ -0,0 +1,310 @@
# ADR-017: Privilege Model and Authority Context
## Status
Accepted
## Context
The call protocol allows handlers to compose other operations through
`OperationEnv::invoke()`. This creates a call tree: a parent request spawns
children, which may spawn their own children. The `parent_request_id` field
records this tree.
The previous design had a `trusted: bool` flag on `OperationContext`. When a
handler invoked another operation through `OperationEnv`, the nested call was
marked `trusted: true` and **all ACL checks were skipped**. The intent was to
avoid double-checking: if `/agent/chat` is allowed and it internally calls
`/auth/verify`, the auth check is "trusted" because the caller already passed
ACL on `/agent/chat`.
This is a privilege escalation vector. Two concrete attacks:
**Buggy handler**: a handler accidentally calls an operation it shouldn't. With
`trusted: true`, ACL is skipped entirely. A handler with `read` scope that
accidentally calls an operation requiring `admin` succeeds — the caller's `read`
scope effectively triggered an `admin` operation.
**Parameterized dispatch**: a handler takes caller input that determines which
internal operation to call. This is the core agent use case — an LLM picks which
tool to invoke based on the user's prompt. With `trusted: true`, the LLM (and
therefore the user) can invoke any registered operation without ACL checks,
regardless of the caller's scopes. A caller with `chat` scope can invoke
operations requiring `admin` by choosing the right tool name.
The call protocol is a general-purpose cross-boundary RPC mechanism. Every
consumer — NAPI adapter, Python adapter, agent service, future services —
inherits whatever privilege model the protocol defines. The privilege boundary
between external and internal calls, and the authority context switch for
composition, are core protocol semantics. This is not a feature of any single
consumer; it is the protocol's security model.
The agent service is a useful test case because it exercises every edge case
(parameterized dispatch, deep composition, dynamic operations, role-based
escalation), but the decision belongs to the call protocol.
## Mental Models
Two analogies clarify the model:
**Kernel/user mode**: external operations are syscalls — curated entry points
where an unprivileged caller can enter the kernel. Internal operations are
kernel functions — callable only from composition, not from userspace. The
`internal` flag means "this call is in kernel mode." Kernel mode has access
controls — it runs under a different principal, not with no principal.
**Domain/integration events**: external operations are integration events —
they cross a boundary and are visible to external systems. Internal operations
are domain events — they stay within the bounded context. `services/list` is
the integration contract; it only exposes integration events.
**Principal/agent (legal contracting)**: the caller is the principal; the
handler is the agent. The principal delegates scoped authority to the agent.
The agent acts under its own identity (for attribution) but with the principal's
delegated authority (for scope). Liabilities flow upstream (traceable through
`parent_request_id`); privileges flow downstream (the agent gets a subset of the
principal's authority). Role-based escalation: a lower-privileged role can
escalate through a chain of command (agent requests promotion, architect
performs it), not through direct authority.
## Decision
### 1. The `internal` flag switches authority context, not skips ACL
The `internal` flag on `OperationContext` marks calls that originated from
composition (a handler calling another operation via `OperationEnv`), as opposed
to external calls that arrived as `call.requested` from a wire client.
When `internal: true`:
- The ACL check runs against the **handler's identity** (set at registration by
the assembly layer), not the caller's identity and not as a blanket skip.
- The handler's identity has scopes scoped to its composition needs (least
privilege), not blanket root and not the caller's scopes.
When `internal: false` (external call from the wire):
- The ACL check runs against the **caller's identity** (from `AuthContext`,
resolved per-request).
The `internal` flag is set by `OperationEnv`, not by callers. A handler cannot
mark its own call as internal. The field uses module-private construction; only
`pub fn is_internal(&self) -> bool` is exposed for reads.
### 2. Operations have External/Internal visibility
`OperationSpec` has a `visibility: Visibility` field:
```rust
pub enum Visibility {
External, // Callable from the wire (call.requested from a client)
Internal, // Composition-only (env.invoke from a handler)
}
```
The assembly layer declares visibility when registering operations.
When a `call.requested` arrives from a wire client:
- An `Internal` operation returns `call.error` with code `NOT_FOUND` (not
`FORBIDDEN`). This does not leak that the operation exists.
- An `External` operation proceeds to ACL checking.
`services/list` only returns `External` operations to remote callers. Internal
operations are not part of the wire-facing API surface. A remote client cannot
enumerate the internal call tree.
### 3. Handler identity is carried on OperationContext
> **Note**: This decision's `handler_identity: Option<Identity>` type was
> superseded by ADR-018, which replaced `Identity` with
> `CompositionAuthority` — a declared authority bundle that is not a peer
> identity and is not resolvable through `IdentityProvider`. The core
> decision (authority switch, not ACL skip) holds unchanged. See ADR-018
> Decision 2 for the current type.
`OperationContext` carries both the caller's identity (who invoked me) and
the handler's identity (who am I acting as):
```rust
pub struct OperationContext {
pub request_id: String,
pub parent_request_id: Option<String>,
pub identity: Option<Identity>, // Caller's identity (inbound)
// Type changed to Option<CompositionAuthority> by ADR-018:
pub handler_identity: Option<CompositionAuthority>, // Handler's composition authority
pub capabilities: Capabilities,
pub metadata: HashMap<String, Value>,
// env/scoped_env split by ADR-019:
pub scoped_env: ScopedOperationEnv, // Reachability data (ADR-018, ADR-019)
pub env: Arc<dyn OperationEnv + Send + Sync>, // Dispatch trait (ADR-019)
/// Module-private for writes; read via `is_internal()`. Set only by
/// `OperationEnv::invoke()` (true) or `CallAdapter` dispatch (false).
pub(crate) internal: bool,
}
impl OperationContext {
pub fn is_internal(&self) -> bool { self.internal }
}
```
- `identity`: the authenticated caller (from `AuthContext`). For external calls,
this is who sent the `call.requested`. For internal calls, this is the
*parent handler's* identity (propagated through `OperationEnv::invoke()`).
- `handler_identity`: the identity of the handler processing this call. Set at
registration by the assembly layer. For external calls, this is the handler's
own identity. For internal calls, the ACL check runs against this identity.
The distinction is the principal/agent model: `identity` is the principal (who
delegated), `handler_identity` is the agent (who is acting). Attribution traces
through both — any action can be attributed to the handler that performed it and
the caller that initiated the chain.
### 4. Scoped composition env
The `OperationEnv` given to a handler is scoped — it can only invoke a declared
set of operations. This bounds the parameterized-dispatch attack surface: a
caller (or an LLM) picking which operation to invoke picks from the declared
set, not from the entire registry.
Scoping happens at two levels:
**Static scoping at registration**: the assembly layer declares which operations
a handler may compose. The `OperationEnv` given to that handler is pre-filtered
— `invoke("fs", "readFile", ...)` works, `invoke("admin", "deleteUser", ...)`
returns `NOT_FOUND`. This is the reachability control.
**Dynamic scoping at sandbox creation**: when a handler spawns a sandbox
(quickjs), it passes a *further scoped* env to the sandbox — a subset of what
the handler itself can reach. The handler might have `fs:read` and `bash:exec`,
but it only gives the sandbox `fs:read` (not `bash:exec`), because the sandbox
runs untrusted LLM-generated code. This is the "privileges flow downstream"
principle: the principal delegates a subset.
The specific API for declaring the scoped operation set is specified in
ADR-018: `ScopedOperationEnv { allowed_operations: HashSet<String> }`,
operation-level granularity (not just namespace-level). This is finer-grained
than the TypeScript `@alkdev/operations` `buildEnv()` which used
`allowedNamespaces` — operation-level scoping is safer for the
parameterized-dispatch use case.
### 5. The three controls together
The three controls are independent and all are needed:
| Control | What it gates | Without it |
|---------|--------------|-----------|
| Operation visibility | Whether an operation is callable from the wire | Internal operations exposed to external callers |
| Handler identity | What authority composition runs under | ACL skipped or caller's scopes propagated (escalation) |
| Scoped composition env | What operations a handler can reach | Handler can call anything in the registry |
- Visibility alone: internal operations are hidden from the wire, but
composition skips ACL (escalation through buggy handler).
- Handler identity alone: ACL checks against handler scopes, but the handler can
reach any operation (parameterized dispatch unbounded).
- Scoped env alone: handler can only reach declared operations, but ACL is
skipped (if a declared operation requires a scope the handler doesn't have, it
still runs).
All three together: the handler can only reach declared operations (scoped env),
those operations are ACL-checked against the handler's scoped identity (handler
identity), and internal operations are never exposed to the wire (visibility).
Principle of least privilege.
## Consequences
**Positive:**
- No privilege escalation through composition. A handler can only compose
operations its own identity is authorized for, and only from its declared
scope.
- Parameterized dispatch is safe. The agent/LLM tool selection case is bounded
by the scoped env — the LLM picks from the declared tool set, not from the
entire registry. The ACL checks against the handler's identity, not the
caller's.
- Buggy handlers can't accidentally escalate. A handler that tries to call an
operation outside its scoped env gets `NOT_FOUND`; one that calls an operation
its identity lacks scopes for gets `FORBIDDEN`.
- Attribution is complete. Every call carries both the caller's identity (who
initiated the chain) and the handler's identity (who is acting). The
`parent_request_id` chain traces the full agency chain. This supports the
gitea-per-agent pattern where each agent (human or LLM) has its own account.
- Session-scoped operations (OQ-19) are safe by construction. They're always
`Internal`, run under the handler's identity, through the scoped env, in a
locked-down sandbox. The self-improving workflow (agents writing tools) is
bounded.
- Role-based escalation is explicit. An agent requesting promotion (session →
core) is a lower-privileged role asking a higher-privileged role (architect
with `promote` scope) to perform an action. The escalation goes through the
chain of command, not through direct authority.
**Negative:**
- `OperationContext` has two identity fields (`identity` and
`handler_identity`), which is more complex than a single identity. This is
necessary — the principal/agent distinction is real and both are needed for
attribution and ACL.
- The assembly layer has more responsibility: it must declare each handler's
identity (scopes), its scoped composition env (which operations it may
compose), and operation visibility. This is expected — the assembly layer
assembles everything (ADR-008), and forcing explicit declaration of privilege
is a feature, not a bug.
- Adding a new composition to a handler requires updating the assembly layer
(declare the new operation in the scoped env), not just the handler code.
This prevents accidental composition of unauthorized operations.
- The scoped env API is not fully specified here. The one-way constraint
(scoped env exists, is declared at registration, can be further scoped at
runtime) is fixed; the concrete API is a two-way door for implementation.
## Assumptions
1. **Internal calls should run under a different authority than external calls,
not skip ACL entirely.** If internal calls should skip ACL (the old `trusted`
model), this entire ADR is wrong. The assumption is that the escalation
vectors (buggy handler, parameterized dispatch) are real and must be
prevented.
2. **Handler identity is set at registration by the assembly layer.** The
assembly layer is the trust boundary (ADR-008, ADR-010). If the assembly
layer is compromised, all handler identities are compromised. This is the
same trust boundary as capabilities.
3. **The scoped env is declared at registration (static) and can be further
scoped at runtime (dynamic, for sandbox creation).** The static scoping is
the reachability control; the dynamic scoping is the sandbox boundary. If a
use case requires fully dynamic scoping (handler discovers at call time what
it can compose), the model needs extension — but the assumption is that
composition reachability is knowable at registration time.
4. **`services/list` hides internal operations.** If internal operations should
be discoverable by remote callers (e.g., for debugging), the visibility model
needs a third state. The assumption is that internal operations are
implementation details, not part of the external API surface.
5. **Internal operations return `NOT_FOUND`, not `FORBIDDEN`.** This prevents
existence leakage. If a use case requires distinguishing "you can't call
this" from "this doesn't exist" (e.g., for debugging), the error model needs
refinement. The assumption is that not leaking internal operation existence
is more important than debuggability from the wire.
6. **The handler identity is a full `Identity` (with scopes), not a special
principal type.** ~~This reuses the existing `Identity` type and
`IdentityProvider` infrastructure (ADR-003).~~ **Superseded by ADR-018
Decision 2**: composition authority is a declared authority bundle
(`CompositionAuthority`), not a peer `Identity`. It is not resolvable
through `IdentityProvider` and does not represent an inbound caller. The
distinction is necessary because a handler is not a network peer — its
authority is declared by the assembly layer at registration, not resolved
from credentials.
## References
- ADR-003: Auth as shared core (`IdentityProvider`, `Identity`)
- ADR-008: Vault integration (assembly layer is the trust boundary)
- ADR-010: Secret material flow and capability injection (capabilities are
orthogonal — both are set at registration by the assembly layer)
- OQ-15: Call protocol client and adapter contract (adapters produce scoped envs)
- OQ-17: Abort cascade (the call tree is the agency chain — `parent_request_id`
traces principal → agent)
- OQ-19: Session-scoped registries (session operations are always `Internal`)
- [operation-registry.md](../crates/call/operation-registry.md)
- [call-protocol.md](../crates/call/call-protocol.md)
- TypeScript `@alkdev/operations` `buildEnv()` with `allowedNamespaces` — prior
art for scoped composition env
- POC at `/workspace/toolEnv` — demonstrated the sandbox-to-registry bridge with
the full-registry exposure gap

655
docs/architecture/decisions/018-handler-registration-provenance-and-composition-authority.md Normal file
View File

@@ -0,0 +1,655 @@
# ADR-018: Handler Registration, Provenance, and Composition Authority
## Status
Accepted
## Context
ADR-017 established the privilege model: the `internal` flag marks
composition-originated calls and switches the ACL from the caller's identity
to the handler's identity. This replaces the old `trusted: bool` flag, which
skipped ACL entirely — a privilege escalation vector. The core decision in
ADR-017 is sound: internal calls switch authority, they don't skip ACL.
However, ADR-017 left three things unspecified, which the pre-implementation
review (docs/reviews/001-pre-implementation-architecture-sanity-check.md,
findings C1–C4) identified as critical gaps:
1. **`handler_identity` has no registration path.** ADR-017 says the handler's
identity is "set at registration by the assembly layer" (Assumption 2) and
that "ACL check runs against the handler's identity (set at registration)"
(Decision 1). But the registration API shown in operation-registry.md —
`register(spec, handler)` and `OperationRegistryBuilder::with(spec,
handler)` — accepts no identity. Tracing the dispatch path reveals that
`build_root_context` sets `handler_identity: None` for wire calls (correct
for the root), and `OperationEnv::invoke()` propagates
`parent.handler_identity.clone()` to children. Since the root's
`handler_identity` is `None`, every internal call gets `handler_identity:
None` — meaning ADR-017's "ACL runs against `handler_identity` for internal
calls" checks against `None`, which is the privilege-escalation gap ADR-017
was written to close.
2. **The scoped composition env has no registration/construction path.**
ADR-017 says the `OperationEnv` given to a handler is "scoped — it can
only invoke a declared set of operations, set at registration by the
assembly layer" (Decision 4, Assumption 3). But `register(spec, handler)`
takes no scoped-env declaration, `OperationSpec` has no field for it, and
the only `OperationEnv` implementation shown is `LocalOperationEnv` wrapping
the *full* registry — no scoping layer exists.
3. **`Capabilities` lives in two unconnected models.** ADR-010 and
operation-registry.md show two models for how a handler gets outbound
credentials: construction-time capture in the handler closure (Model A) and
per-request on `OperationContext.capabilities` propagated through
composition (Model B). The two don't connect: if the handler closure
captured capabilities at construction, `OperationContext.capabilities` is
either redundant or must be populated from the closure — but the closure
receives the context, it isn't passed it. An implementer would have to
invent the bridge, and the consuming crates (call, agent, napi) could
diverge.
Beyond these wiring gaps, there is a deeper issue with ADR-017's Assumption 6:
"the handler identity is a full `Identity` (with scopes), not a special
principal type." `Identity` was designed for **inbound peer identity** — who
is calling me from the network. A handler is not a peer. Its `id` field would
be something like `"agent-chat-handler"` — a label, not something resolvable
through `IdentityProvider`. Calling it an `Identity` implies it's a peer,
which it isn't. It's an authority bundle.
### The kernel/user analogy
This is structurally the same problem an operating system solves with
kernel/user mode:
- User calls `getaddrinfo()` — the syscall gate (an **External** op). The
kernel checks the user's capabilities at entry.
- `getaddrinfo` internally makes DNS queries, allocates sockets, reads
`/etc/hosts` — **Internal** kernel functions. They don't check the user's
`CAP_NET_RAW`. They run under **kernel authority**.
- The user does NOT need `CAP_NET_RAW` to resolve DNS. The kernel does network
access on the user's behalf, under the kernel's own authority.
The key principle: **the user's authority is checked once at the gate. Inside,
the handler runs under its own authority. The user's authority does not
propagate into internal calls.**
This is exactly what ADR-017 specifies. The `internal` flag is the boundary
crossing. When `internal: true`, ACL switches from the caller's identity to
the handler's composition authority. The user's `[chat]` scope got them through
`/agent/chat`'s External ACL. Once inside, it's `/agent/chat`'s composition
authority that authorizes composing `/vastai/listMachines` — not the user's.
### The graph framing
Call trees and operation registries are graph-shaped. The TypeScript
`@alkdev/flowgraph` package models this explicitly with three graphs:
1. **Operation Graph** (static) — nodes are registered operations, edges are
type-compatibility relationships. Built from `OperationSpec`s at startup.
2. **Call Graph** (dynamic) — nodes are call invocations (request IDs), edges
are parent-child relationships (`parent_request_id`). Built from call
protocol events at runtime.
3. **Scoped Operation Subgraph** (per-handler, static) — the declared subset
of the operation graph that a handler may reach. This is what ADR-017 calls
the "scoped env," framed as a subgraph rather than a list of names.
This ADR uses the graph *model* as structural framing but does not mandate a
graph *library*. For v1, the operation graph can be implicit (a
`HashMap<String, OperationNode>`), the call graph can be implicit (the
`PendingRequestMap` indexed by `parent_request_id` *is* a call graph), and the
scoped env can be a `HashSet<String>` of reachable operation names. A
dedicated `alknet-flowgraph` crate (or folding graph structures into
`alknet-call`) is a future enhancement for workflow templates, type
compatibility validation, and call-graph observability — not a prerequisite
for the security model.
## Decision
### 1. Provenance is the primary registration axis
Every registered operation carries a provenance tag that classifies where it
came from. Provenance determines whether the operation can compose, whether it
has composition authority, its default visibility, and its trust model.
```rust
pub enum OperationProvenance {
/// Assembly-written, trusted code, can compose.
Local,
/// HTTP forwarding stub (from_openapi), leaf — cannot compose.
FromOpenAPI,
/// MCP forwarding stub (from_mcp), leaf — cannot compose.
FromMCP,
/// QUIC forwarding stub (from_call). Leaf in the local registry —
/// forwards calls to a remote node; cannot compose locally.
FromCall,
/// HTTP forwarding stub (from_jsonschema, single endpoint), leaf —
/// cannot compose. (ADR-027: was "no handler — schema only"; now a
/// real reqwest-backed forwarding handler in alknet-http.)
FromJsonSchema,
/// Agent-written, sandboxed, can compose within sandbox bounds.
Session,
}
```
| Provenance | Can compose? | Has composition authority? | Default visibility | Trust model |
|-----------|-------------|---------------------------|-------------------|-------------|
| `Local` | Yes | Yes — scopes set by assembly layer | External or Internal (assembly declares) | Trusted code |
| `FromOpenAPI` | No (leaf) | No | Internal | HTTP endpoint trusted; handler is a forwarding stub |
| `FromMCP` | No (leaf) | No | Internal | MCP server trusted; handler is a forwarding stub |
| `FromCall` | No (leaf in local registry) | No | Internal | Remote node trusted; handler is a forwarding stub |
| `FromJsonSchema` | No (leaf) | No | Internal | HTTP endpoint trusted; handler is a forwarding stub (ADR-027) |
| `Session` | Yes (within sandbox) | Yes — scopes set by assembly layer at sandbox creation | Internal always | Untrusted code in sandbox |
> **ADR-027 amendment (2026-07-09).** The `FromJsonSchema` row
> previously read "N/A (no handler) / N/A / N/A" — `from_jsonschema`
> was a schema-only placeholder in `alknet-call` with a
> `NOT_FOUND`-returning handler.
> [ADR-027](027-from-jsonschema-as-http-adapter.md) moved the adapter
> to `alknet-http` as a real HTTP-backed single-endpoint adapter with a
> reqwest forwarding handler. `FromJsonSchema` is now a leaf, same
> trust model as `FromOpenAPI` (HTTP endpoint trusted; handler is a
> forwarding stub). The "schema-only, no handler" concept is removed.
Only `Local` and `Session` ops get composition authority. Leaves
(`FromOpenAPI`, `FromMCP`, `FromCall`, `FromJsonSchema`) don't compose, so
they don't get one. The assembly layer does not invent identities for leaves.
### 2. Composition authority replaces `handler_identity: Identity`
ADR-017's Assumption 6 said "the handler identity is a full `Identity` (with
scopes), not a special principal type." This ADR refines that: composition
authority is a declared authority bundle, not a peer `Identity`. It's only set
for ops that can compose (`Local`, `Session`). Leaves don't have one.
```rust
/// Authority under which a handler composes child operations.
///
/// This is NOT a peer `Identity` — it's not resolvable through
/// `IdentityProvider` and doesn't represent an inbound caller. It's the
/// declared authority (scopes + resources + label) that the assembly layer
/// grants a handler for composition. When the handler composes children via
/// `OperationEnv::invoke()`, the child's ACL runs against this authority,
/// not the caller's identity and not as a blanket skip.
///
/// Only ops that can compose (`Local`, `Session`) have one. Leaves
/// (`FromOpenAPI`, `FromMCP`, `FromCall`) have `None`.
pub struct CompositionAuthority {
/// Human-readable label for attribution and logging
/// (e.g., "agent-chat", "fs-handler"). Not a peer id — not resolvable
/// through IdentityProvider.
pub label: String,
/// Scopes the handler operates under for composition. When the handler
/// composes a child via `env.invoke()`, the child's ACL checks against
/// these scopes. Least privilege: the assembly layer grants only the
/// scopes the handler needs for its declared composition.
pub scopes: Vec<String>,
/// Named resource lists, same shape as `Identity.resources`. Optional.
/// e.g., {"service": ["vastai", "github"]} bounds which services the
/// handler can reach in composition.
pub resources: HashMap<String, Vec<String>>,
}
impl CompositionAuthority {
/// `None` — for leaves that don't compose (convenience for
/// `composition_authority: CompositionAuthority::none()`).
pub fn none() -> Option<Self> { None }
/// Construct a composition authority with the given label and scopes.
pub fn new(
label: &str,
scopes: impl IntoIterator<Item = String>,
) -> Self {
Self {
label: label.to_string(),
scopes: scopes.into_iter().collect(),
resources: HashMap::new(),
}
}
/// Convert to a synthetic `Identity` for ACL matching on child calls.
///
/// When a handler composes a child via `env.invoke()`, the child's
/// `identity` (the caller identity for ACL) is set to the parent's
/// composition authority converted to an `Identity`. This constructs
/// a synthetic `Identity { id: label, scopes, resources }` that is
/// **not** resolvable via `IdentityProvider` — it's not a peer
/// identity, it's a declared authority bundle used directly for ACL
/// matching. This creates a second `Identity` construction path (the
/// first is `IdentityProvider::resolve_*`), which is acknowledged and
/// intentional: the composition authority is a declared authority, not
/// a resolved credential.
///
/// Returns `None` when the authority is `None` (leaf case — leaves
/// don't compose, so `as_identity()` is never called on them in
/// practice, but the `Option` makes the types line up).
pub fn as_identity(&self) -> Option<Identity> {
Some(Identity {
id: self.label.clone(),
scopes: self.scopes.clone(),
resources: self.resources.clone(),
})
}
}
```
This supersedes ADR-017's Assumption 6. ADR-017's core decision (authority
switch, not ACL skip) holds unchanged — the only change is *what* the
authority is and which ops have it.
### 3. The scoped env is a declared subgraph (reachability control)
The scoped composition env from ADR-017 is the **reachability control**: it
bounds which operations a handler can reach via `env.invoke()`. ADR-017
specifies it as "a declared set of operations, set at registration by the
assembly layer." This ADR makes the registration path explicit and frames it
as a subgraph of the operation graph.
```rust
/// The set of operations a handler may reach via `env.invoke()`.
///
/// This is the reachability control from ADR-017: a handler (or an LLM
/// picking tools, or a quickjs sandbox) can only compose declared operations,
/// not the entire registry. Set at registration by the assembly layer for
/// composing ops (`Local`, `Session`). `None` for leaves — they don't
/// compose, so they get an empty/no-op env.
///
/// Conceptually a subgraph of the operation graph. For v1, implemented as a
/// set of operation names — the *model* is a subgraph (which nodes this
/// handler can reach), but type-compatibility edges between those nodes are
/// a future enhancement for static validation, not a v1 requirement.
///
/// The `allowed_operations` field is **private** (not `pub`). Construction
/// is via `ScopedOperationEnv::new(ops)` or `ScopedOperationEnv::empty()`.
/// Reachability is queried via `allows(&name)`. This encapsulation makes the
/// future subgraph refactor (from `HashSet<String>` to a typed subgraph) a
/// non-breaking change to construction sites (review #002 W21). The
/// `HashSet<String>` representation does not support type-compatibility
/// validation — session-scoped ops (OQ-19, untrusted code) compose without
/// static type checking until a flowgraph crate is built.
pub struct ScopedOperationEnv {
allowed_operations: HashSet<String>,
}
impl ScopedOperationEnv {
/// Empty set — for leaves that don't compose (no reachable operations).
pub fn empty() -> Self {
Self { allowed_operations: HashSet::new() }
}
/// Construct from an iterable of operation names.
pub fn new(ops: impl IntoIterator<Item = String>) -> Self {
Self { allowed_operations: ops.into_iter().collect() }
}
/// Returns true if the given operation name is reachable.
pub fn allows(&self, name: &str) -> bool {
self.allowed_operations.contains(name)
}
}
```
### 4. The registration bundle carries all three
The three controls from ADR-017 (visibility, composition authority, scoped
env) plus the capability injection from ADR-010 all enter the system at the
same boundary: the assembly layer hands the registry a `(spec, handler)` pair
*plus* the handler's runtime context material. This ADR makes that explicit
as a registration bundle.
```rust
pub struct HandlerRegistration {
pub spec: OperationSpec,
pub handler: Handler,
pub provenance: OperationProvenance,
/// Composition authority for this handler. `None` for leaves
/// (`FromOpenAPI`, `FromMCP`, `FromCall`) — they don't compose.
/// `Some(...)` for `Local` and `Session` ops that can compose children.
pub composition_authority: Option<CompositionAuthority>,
/// Scoped composition env. `None` for leaves — they get an empty
/// no-op env. `Some(...)` for composing ops.
pub scoped_env: Option<ScopedOperationEnv>,
/// Outbound credentials the handler may use (decrypted API keys, signing
/// keys, HTTP tokens). Populated by the assembly layer from the vault
/// at handler construction. See ADR-010.
pub capabilities: Capabilities,
}
```
The registry's `register` and builder's `with` accept a `HandlerRegistration`,
not a bare `(OperationSpec, Handler)` pair:
```rust
impl OperationRegistry {
pub fn register(&mut self, registration: HandlerRegistration);
}
impl OperationRegistryBuilder {
pub fn with(mut self, registration: HandlerRegistration) -> Self;
}
```
Adapter convenience methods (`from_openapi`, `from_mcp`, `from_call`)
construct `HandlerRegistration` with `composition_authority: None` and
`scoped_env: None` for the leaf ops they produce — the adapter doesn't grant
composition authority, and the assembly layer doesn't have to invent values
for leaves.
### 5. The dispatch path reads from the registration bundle
The CallAdapter's `build_root_context` and `OperationEnv::invoke()` read
composition authority, scoped env, and capabilities from the registration
bundle, looked up by operation name.
**`build_root_context` (wire-originated call, `internal: false`):**
```rust
fn build_root_context(
&self,
request_id: String,
operation_name: &str, // looked up in registry
identity: Option<Identity>, // resolved per-request from AuthContext/auth_token
) -> OperationContext {
let registration = self.registry.registration(operation_name);
OperationContext {
request_id,
parent_request_id: None,
identity, // caller's identity (inbound — gate credential)
handler_identity: registration.composition_authority, // C1: from bundle, None for leaves
capabilities: registration.capabilities.clone(), // C3: from bundle
metadata: HashMap::new(),
abort_policy: AbortPolicy::default(), // abort-dependents (ADR-020 Decision 6)
// env/scoped_env split by ADR-019: scoped_env is the reachability
// data (from the bundle), env is the dispatch trait object (composed
// per-call by the CallAdapter from active overlays).
scoped_env: registration.scoped_env.clone()
.unwrap_or_else(ScopedOperationEnv::empty), // C2: from bundle, empty for leaves
env: self.compose_root_env(/* connection, session */), // Arc<dyn OperationEnv + Send + Sync> — see ADR-019
internal: false, // wire call — ACL against caller identity
}
}
```
ACL for the root checks against `identity` (the caller's identity, resolved
per-request). `handler_identity` is on the context for *propagation* to
children, not for the root's own ACL.
**`OperationEnv::invoke()` (composition-originated call, `internal: true`):**
```rust
async fn invoke(&self, namespace: &str, operation: &str, input: Value,
parent: &OperationContext) -> ResponseEnvelope {
let name = format!("{namespace}/{operation}");
// Reachability check (C2): is this op in the parent's scoped env?
// If not, return NOT_FOUND. This is the reachability control.
// (ADR-019: the reachability check consults parent.scoped_env, not
// parent.env — env is now the dispatch trait, scoped_env is the data.)
if !parent.scoped_env.allows(&name) {
return ResponseEnvelope::not_found(name);
}
let registration = self.registry.registration(&name);
let context = OperationContext {
request_id: generate_request_id(),
parent_request_id: Some(parent.request_id.clone()),
identity: parent.handler_identity.as_identity(), // parent's authority becomes the caller
handler_identity: registration.composition_authority.clone(), // C1: child's own authority
capabilities: parent.capabilities.clone(), // C3: propagate through composition
metadata: HashMap::new(), // fresh — does NOT propagate (ADR-010)
abort_policy: parent.abort_policy.clone(), // inherit parent's policy (ADR-020 Decision 6, W19)
// env/scoped_env split by ADR-019:
scoped_env: registration.scoped_env.clone()
.unwrap_or_else(ScopedOperationEnv::empty), // C2: child's own scoped env
env: parent.env.clone(), // child inherits parent's composite env (Arc::clone)
internal: true, // composition — ACL against handler_identity
};
self.registry.invoke(&name, input, context).await
}
```
Two things happen here:
1. **Reachability check**: before constructing the child context, `invoke()`
checks whether the requested op is in the parent's scoped env. If not,
`NOT_FOUND`. This bounds the parameterized-dispatch attack surface — a
handler (or an LLM picking tools) can only reach declared ops.
2. **Authority propagation**: the child's `identity` is the parent's
`handler_identity` (the parent's composition authority becomes the caller
for the child). The child's `handler_identity` is the *child's own*
registration's `composition_authority` — so if the child itself composes
further, its children inherit the child's authority. This is the
principal/agent chain from ADR-017, now wired.
ACL for the child checks against `handler_identity` (the child's composition
authority). For leaves, `handler_identity` is `None` — but leaves don't
compose, so their `handler_identity` is never used for ACL on a grandchild.
Leaves only have ACL checked against *themselves* (as the target of
composition), where the check is: does the parent's composition authority
satisfy the leaf's `AccessControl`?
### 6. Capabilities are per-request, populated from the bundle (Model A reconciled)
This ADR resolves the C3 ambiguity by adopting option (a) from the review:
capabilities are only per-request on `OperationContext`, populated by the
dispatch path from the per-handler capabilities in the registration bundle.
The construction-time "baking" described in ADR-010 L82 populates the
registration bundle's `capabilities` field — the handler closure does not
capture capabilities.
```rust
// Assembly layer: construct registration with capabilities from vault
let google_api_key = vault.decrypt(&google_key_blob)?;
let agent_registration = HandlerRegistration {
spec: agent_chat_spec(),
handler: Arc::new(agent_chat_handler), // closure captures nothing
provenance: OperationProvenance::Local,
composition_authority: Some(CompositionAuthority {
label: "agent-chat".into(),
scopes: vec!["llm:call".into(), "fs:read".into(), "vastai:query".into()],
resources: HashMap::new(),
}),
scoped_env: Some(ScopedOperationEnv::new(
["fs/readFile", "vastai/listMachines", "llm/generate"])),
capabilities: Capabilities::new()
.with_api_key("google", google_api_key), // C3: in the bundle, not the closure
};
```
The handler reads `context.capabilities` at call time. The dispatch path
populates it from `registration.capabilities`. Composition propagates it via
`parent.capabilities.clone()` in `invoke()`. No circular dependency, no
redundant models.
### 7. The three controls together (ADR-017's model, now wired)
| Control | What it gates | Where it's set | Without it |
|---------|--------------|----------------|-----------|
| Visibility (External/Internal) | Whether the op is callable from the wire | `OperationSpec.visibility` | Internal ops exposed to external callers |
| Composition authority | What authority internal calls run under | `HandlerRegistration.composition_authority` | ACL skipped or caller's scopes propagated (escalation) |
| Scoped env | What ops a handler can reach | `HandlerRegistration.scoped_env` | Handler can call anything in the registry (confused deputy) |
All three enter at registration. All three reach the dispatch path via the
registration bundle. The user's identity is the **gate credential** — checked
once at the External boundary. The composition authority is the **internal
credential** — used for all composition inside. The scoped env is the
**reachability boundary** — what the handler can even attempt to compose.
### 8. No intersection semantics
The user's authority does NOT limit internal calls. If the user has `chat` but
not `vastai:query`, `/agent/chat` composing `/vastai/listMachines` is NOT
denied because the user lacks `vastai:query`. The user's authority was
checked at the gate (`/agent/chat` requires `chat`, user has `chat`). Inside,
the handler runs under its own composition authority. The user's authority
does not propagate into internal calls.
This is the kernel/user model: `getaddrinfo` doesn't require the caller to
have `CAP_NET_RAW` to make DNS queries. The curated entry point exists
*because* it does things the user can't, on the user's behalf, under its own
authority.
If a handler *wants* to act on behalf of the user (e.g., a database proxy
that runs queries under the user's DB identity), that's a **handler-level
decision** — it reads `context.identity` and explicitly narrows its
behavior. That's delegated access, not automatic intersection. The system
shouldn't silently intersect; the handler should explicitly delegate.
## Consequences
**Positive:**
- The privilege model in ADR-017 is now implementable as specified. The
composition authority, scoped env, and capabilities all have registration
paths and dispatch-path wiring. No implementer has to invent the bridge.
- Leaves (`from_openapi`, `from_mcp`, `from_call`) don't get fake identities.
The assembly layer doesn't have to invent `Identity { id:
"vastai-listmachines-handler", scopes: [], resources: {} }` for forwarding
stubs that will never compose. `composition_authority: None` is natural for
leaves, not an oversight.
- External services can't self-grant composition authority. The OpenAPI spec
defines the operation interface (name, schemas, access control). The
*provenance* is set by the assembly layer when it runs `from_openapi`. The
*composition authority* is `None` for imported ops — the external service
can't grant itself scopes to compose into your registry. The assembly layer
is the sole grantor, and only for `Local` and `Session` ops.
- Capabilities have one model: per-request on `OperationContext`, populated
from the registration bundle. No closure-capture vs context duplication
ambiguity. The three consuming crates (call, agent, napi) can't diverge
because there's one wiring path.
- The graph model provides a precise structural framing without mandating a
graph library for v1. The operation graph, scoped subgraph, and call graph
are concepts that guide the API shape; HashMaps and HashSets are the v1
implementation. A future `alknet-flowgraph` crate can reify these as
petgraph structures when workflow templates and type-compatibility
validation are needed.
- The kernel/user analogy makes the security model legible. The user's
authority is the gate credential (checked once at External entry). The
composition authority is the internal credential (used for all
composition inside). The scoped env is the reachability boundary (what the
handler can attempt to compose). This is the same model every OS uses, and
it's been battle-tested.
**Negative:**
- The registration API changes from `register(spec, handler)` to
`register(HandlerRegistration)`. This is a breaking change to the API
surface shown in operation-registry.md, but since no implementation exists
yet, it's a spec edit, not a migration.
- `CompositionAuthority` is a new type, distinct from `Identity`. This adds a
type to alknet-call. It's not a peer identity — it's a declared authority
bundle. The distinction from `Identity` is intentional and necessary (a
handler is not a network peer), but it means the codebase has two
scope-bearing types. Mitigated: they serve different roles and don't
converge — `Identity` is inbound (resolved from credentials via
`IdentityProvider`), `CompositionAuthority` is declared (set by the
assembly layer at registration).
- The assembly layer has more registration-time responsibility: it must
declare each handler's provenance, composition authority, and scoped env.
This is expected — the assembly layer assembles everything (ADR-008), and
forcing explicit declaration of privilege is a feature, not a bug. An
`OperationRegistryBuilder` convenience API can reduce boilerplate for
common cases (e.g., `.with_local(spec, handler, authority, env,
capabilities)` vs `.with_leaf(spec, handler, capabilities)`).
- The dispatch path does a registry lookup per call (to fetch the
registration bundle's composition authority, scoped env, and capabilities).
This is a `HashMap` lookup — negligible cost. The alternative (baking
everything into the handler closure) creates the C3 ambiguity. The lookup
is the right trade.
**Validation strategy:**
The security model should be validated by fuzzing. A fuzzer that generates
call trees (valid and invalid compositions, different provenance mixes, edge
cases around the gate) and asserts "no path through the call graph lets a
user with scope X reach an operation requiring Y without going through a gate
that checks X" would catch the class of privilege-escalation bug this ADR is
designed to prevent. The typebox-rs fake data generator can produce valid and
invalid inputs from JSON Schemas; with minor edits it can output invalid
inputs or a mix of valid/invalid, enabling property-based testing of the ACL
model. This is a downstream concern — the spec needs to be right first, then
the fuzzer validates the implementation against the spec.
## Assumptions
1. **Internal calls should run under a different authority than external
calls, not skip ACL entirely.** Inherited from ADR-017. The escalation
vectors (buggy handler, parameterized dispatch) are real and must be
prevented.
2. **Provenance is knowable at registration time.** The assembly layer knows
whether an op is `Local`, `FromOpenAPI`, `FromMCP`, `FromCall`, or
`Session` when it registers the op — the adapter that produced the
`(OperationSpec, Handler)` pair knows its own type. If a future use case
requires provenance to be discovered at call time, the model needs
extension.
3. **Composition reachability is knowable at registration time.** The
assembly layer can declare which operations a handler may compose when it
registers the handler. If a use case requires fully dynamic scoping
(handler discovers at call time what it can compose), the model needs
extension — but the assumption is that composition reachability is
knowable at registration time for `Local` ops, and at sandbox creation
time for `Session` ops.
4. **The assembly layer is the trust boundary.** The assembly layer declares
provenance, composition authority, and scoped env. If the assembly layer
is compromised, all handler authority is compromised. This is the same
trust boundary as ADR-008 and ADR-010.
5. **Leaves don't compose.** `FromOpenAPI`, `FromMCP`, and `FromCall` ops are
forwarding stubs — they take input, forward it (over HTTP, MCP, or QUIC),
and return output. They don't call `env.invoke()`. If a future use case
requires an imported op to compose (e.g., a `from_call` op that locally
composes other ops before forwarding), its provenance would need to change
to `Local` (it's no longer a pure forwarding stub), or the model needs a
hybrid provenance.
6. **`Session` ops compose under restricted authority.** Session ops
(agent-written, OQ-19) get composition authority scoped down by the parent
handler at sandbox creation (ADR-017's "dynamic scoping at sandbox
creation"). The assembly layer grants the sandbox's parent handler a
composition authority; the parent handler scopes it down further when
creating the sandbox. The session op's composition authority is a subset
of the parent's.
## References
- ADR-010: Secret material flow and capability injection (capabilities are
orthogonal to identity — both set at registration; this ADR specifies the
registration path ADR-010 left as a two-way door)
- ADR-017: Privilege model and authority context (this ADR refines
Assumption 6 — composition authority is not a peer `Identity`; and wires
the three controls that ADR-017 specified but left without registration
paths)
- ADR-020: Abort cascade for nested calls (the call graph is the abort
cascade tree; `parent_request_id` indexes it)
- ADR-022: Call protocol client and adapter contract (adapter-registered
ops are `Internal` by default; this ADR's provenance makes that explicit)
- ADR-019: Operation registry layering (amends this ADR's Decision 5: the
`env` field shown in `build_root_context` and `invoke()` is split into
`scoped_env: ScopedOperationEnv` (reachability data, populated from the
bundle's `scoped_env`) and `env: Arc<dyn OperationEnv + Send + Sync>`
(dispatch trait object). The split is required by ADR-019's overlay model
— the trait-object design is what enables connection and session overlays
to compose. The `HandlerRegistration` bundle shape, provenance model,
composition authority, and capability injection specified by this ADR
are unchanged.)
- ADR-008: Vault integration point (assembly layer is the trust boundary)
- OQ-19: Session-scoped operation registries (session ops are `Session`
provenance, always `Internal`, compose under restricted authority)
- docs/reviews/001-pre-implementation-architecture-sanity-check.md (findings
C1–C4, which this ADR resolves)
- docs/reviews/002-pre-implementation-architecture-sanity-check.md (finding
C6, resolved by ADR-019's `env`/`scoped_env` split)
- `/workspace/@alkdev/flowgraph/README.md` — operation graph, call graph, and
scoped subgraph concepts (the graph model this ADR uses as framing)
- `/workspace/@alkdev/alknet-main/docs/architecture/flowgraph.md` — prior
Rust speccing of flowgraph (incomplete; this ADR uses the model, not the
crate)
- Kernel/user mode analogy: `getaddrinfo` runs under kernel authority, not
the caller's `CAP_NET_RAW`; the curated entry point exists to do things
the user can't, on the user's behalf

483
docs/architecture/decisions/019-operation-registry-layering.md Normal file
View File

@@ -0,0 +1,483 @@
# ADR-019: Operation Registry Layering
## Status
Accepted
## Context
The architecture has two registries that the spec documents previously treated
as sharing one immutability argument:
1. **The endpoint's `HandlerRegistry`** (ALPN string → `ProtocolHandler`).
This is what ADR-010 and OQ-04 are about. Its immutability is load-bearing:
ALPN strings are baked into the TLS `ServerConfig` at startup, so adding a
protocol handler at runtime requires rebuilding the TLS config. This is a
genuine one-way door and the rationale is correct.
2. **The call protocol's `OperationRegistry`** (operation name →
`HandlerRegistration`). This lives *inside* the `CallAdapter`, which is one
`ProtocolHandler` behind the single ALPN `alknet/call`. Adding an operation
to the `OperationRegistry` does **not** touch the TLS `ServerConfig` — the
ALPN is already `alknet/call`, registered once at startup.
`operation-registry.md` stated the operation registry "is immutable after
construction… consistent with OQ-04 and ADR-010." That inheritance was by
analogy, not by shared rationale. The TLS argument that justifies
`HandlerRegistry` immutability does not apply to the `OperationRegistry`. The
operation registry's mutability profile is a separate question, and it has been
answered incorrectly by inheriting a constraint that belongs to a different
registry.
### Why `from_call` breaks the inherited constraint
The import adapters have different lifecycle requirements:
- **`from_openapi` / `from_mcp`** can run at startup — the assembly layer reads
a static spec file or queries a known service before the registry is frozen.
Static import, fits immutability.
- **`from_call`** requires a **live connection** to discover operations
(`services/list` + `services/schema`). Connections happen at runtime.
Workers join and leave dynamically in the machine→worker topology. You
cannot pre-freeze a set you discover over a connection you haven't opened
yet.
So `from_call` is structurally incompatible with "frozen at startup, never
touched again." The pre-ADR-019 spec held two contradictory positions: the
registry is immutable (operation-registry.md), and `from_call` imports remote
operations at connection time (ADR-022). An implementer would have to resolve
the contradiction by guessing — likely by either forcing all `from_call`
imports to happen at startup (awkward, doesn't fit worker topologies) or
quietly making the registry mutable (undermining the stated constraint without
acknowledging it).
### Why immutability is not the load-bearing security control for imported ops
Imported operations (`FromOpenAPI`, `FromMCP`, `FromCall`) are leaves — they
cannot compose (ADR-018 Assumption 5). They have no composition authority, no
scoped env, `Internal` visibility by default, and their trust model is "the
remote endpoint is trusted as much as my own handlers" (ADR-022). Their
reachability from a composing handler is bounded by the *parent handler's*
scoped env, not by their registration timing.
The security controls on imported ops are **provenance** and **composition
authority** — both set at registration, both checked at dispatch. Immutability
is redundant here. An imported op registered at runtime is no more or less
privileged than one registered at startup; it's a forwarding stub either way,
and its capacity to do harm is bounded by what the *composing parent*'s
authority and scoped env permit.
Immutability *is* load-bearing for **curated** operations — the `Local` ops
the assembly layer writes at startup, which *can* compose and therefore *can*
escalate privilege under their own authority. For those, the trust boundary is
"the assembly layer declared them at startup," and immutability is what locks
that declaration. But that's a constraint on `Local` provenance specifically,
not on the registry as a whole.
### The trust-boundary principle
The right axis is not visibility (`Internal` vs `External`) or wire-vs-local —
it is **provenance combined with import timing**, which maps to where each
operation's trust decision is made:
| Provenance | Import timing | Trust boundary | Layer | Lifetime |
|-----------|---------------|----------------|-------|----------|
| `Local` | Startup | Assembly layer at startup | 0 (curated) | Process — immutable |
| `Session` | Sandbox creation | Composing handler at sandbox creation | 1 (session) | Session — dynamic |
| `FromCall` | Connection (runtime) | Remote node at connection time | 2 (connection) | Connection — dynamic |
| `FromOpenAPI` / `FromMCP` | Startup | External endpoint, discovered at startup | 0 (curated) | Process — immutable |
| `FromOpenAPI` / `FromMCP` | Runtime (rare) | External endpoint, discovered at runtime | 2 (discovery) | Discovery-scoped — dynamic |
`FromOpenAPI` / `FromMCP` provenance is **layer-polymorphic**: the same
provenance lands in Layer 0 (immutable) or Layer 2 (dynamic) depending on
when the import happens. The common case is startup import into Layer 0
(Decision 6); runtime import into Layer 2 is permitted but rare.
**Immutability follows the trust boundary.** Operations are mutable at the
scope where their trust decision is made. `Local` ops (and startup-imported
`FromOpenAPI`/`FromMCP`) are trusted at startup → immutable. Session ops
are trusted at sandbox creation → session-scoped dynamic. `FromCall` ops
(and runtime-imported `FromOpenAPI`/`FromMCP`) are trusted at
connection/discovery time → connection/runtime dynamic.
Session ops are the edge case that proves the rule: they are `Internal`
visibility and can compose, but their trust boundary is per-session (the
parent handler grants them restricted authority at sandbox creation, per
ADR-018 Assumption 6), not per-startup. Visibility alone would misclassify
them; provenance correctly identifies them as dynamic.
### The precedent: `IdentityProvider`
The structural problem — *N consumers need to resolve something from M
sources, don't globalize the sources into one pot, don't make each consumer
know about all sources* — is the same problem `IdentityProvider` solves for
auth (ADR-003). An `IdentityProvider` is a trait (`Arc<dyn IdentityProvider>`)
that centralizes resolution policy behind a stable interface; source
composition is an impl detail. Handlers consume the result; the trait owns the
routing.
`OperationEnv` is the same problem one layer over: *N handlers need to
dispatch to operations, operations come from M sources (curated local, this
session, this peer connection, that peer connection), don't globalize all
sources into one mutable pot, don't make each handler know about all sources
and pick the right registry.* The solution is the same shape: a trait —
`Arc<dyn OperationEnv>` — that centralizes dispatch routing behind a stable
interface, with overlay composition as an impl detail.
The alternative — a single global `ArcSwap<OperationRegistry>` into which all
imported ops merge with namespace prefixes — is the registry equivalent of
"every handler reads identity from a global env var." It works at one
connection. At many connections it produces: an unbounded pot, namespace
collisions scaling with connection count, disconnect cleanup requiring a
reverse index (op → owning connection), zero source isolation, and
routing-by-naming-convention instead of routing-by-structure. That is the
failure mode the `IdentityProvider` pattern exists to prevent.
## Decision
### 1. The operation registry is layered by trust boundary
The `OperationRegistry` is not a single flat map. It is a layered structure
where each layer corresponds to a trust boundary:
```
Layer 0 — Curated (static, immutable, startup trust boundary)
Local provenance operations from the assembly layer.
Registered once at startup, never mutated for the process lifetime.
This is where immutability is load-bearing: these ops can compose,
therefore can escalate privilege under their own authority. The
startup trust boundary + immutability is the security control.
Layer 1 — Session (dynamic, per-session, sandbox-creation trust boundary)
Session provenance operations, agent-written, sandboxed.
Created and destroyed with each session.
Already specified by OQ-19 as an overlay on Layer 0.
Layer 2 — Imported (dynamic, per-connection, peer trust boundary)
FromCall operations discovered when a peer connects.
FromOpenAPI / FromMCP operations when imported at runtime (rare;
usually at startup into Layer 0, but runtime import is permitted).
Created and destroyed with the connection / discovery event.
```
Layers 1 and 2 are the same shape: **per-scope dynamic overlays on the static
curated base.** The scope is "session" for Layer 1 and "connection" (or
"discovery event") for Layer 2. OQ-19 already specified the overlay mechanism
for Layer 1 (session env wraps global env via `OperationEnv` trait layering).
This ADR generalizes the same mechanism to Layer 2.
### 2. The `OperationEnv` trait is the integration point
`OperationContext.env` is `Arc<dyn OperationEnv + Send + Sync>` — a trait
object, not a concrete struct. This is required by the overlay model: a
composite env (curated base + connection overlay + session overlay) is built
by composing `OperationEnv` impls, not by merging registries.
This resolves review #002 finding C6 (`OperationContext.env` type identity
crisis). The pre-ADR-019 spec had `env: OperationEnv` (a trait, which can't
be a field without `dyn`) and used the same field as both a reachability set
(`parent.env.allows()`) and a dispatch trait (`context.env.invoke()`). One
field cannot be both. The split:
- `scoped_env: ScopedOperationEnv` — reachability data. Populated from the
registration bundle's `scoped_env` (ADR-018). The reachability check in
`invoke()` consults `parent.scoped_env.allows(&name)`.
- `env: Arc<dyn OperationEnv + Send + Sync>` — dispatch trait. The handler
calls `context.env.invoke(...)`; the trait impl routes to the right
overlay.
This is the `IdentityProvider`-shaped integration point: handlers consume
the trait; source composition is an impl detail.
### 3. The `CallAdapter` composes the root env per incoming call
When a `call.requested` arrives over connection C, the `CallAdapter` does
not look up the operation in a single global registry. It composes the root
`OperationContext.env` from the layers active for this call:
```
root env = CompositeOperationEnv {
base: curated_registry_env, // Layer 0 — static
connection: C.imported_operations, // Layer 2 — this connection's overlay
session: active_session_overlay, // Layer 1 — if a session is active
}
```
The composite impl checks overlays in order (session first, then connection,
then curated base) and dispatches to the first match. This is structural
source binding: a handler composing `worker/exec` reaches it via the
connection overlay that contains it, not via a naming convention in a
global pot.
**Env inheritance through composition**: the child's `env` is
`parent.env.clone()` — an `Arc::clone`, not a re-composition. Overlay
composition happens once at the root (in `build_root_context`) and
propagates by `Arc` through the composition tree. A child handler sees the
same active overlays its parent saw. This is deliberate: re-composing per
`invoke()` would re-resolve overlays on every dispatch and would break the
session-overlay case (a session that was active when the parent ran must
still be active for the child, even if the session ended mid-composition —
the child is part of the same call tree the parent started). The root env
is composed per incoming call; nested calls inherit it by `Arc::clone`.
When connection C disconnects, its overlay is dropped. Operations imported
from C vanish from the reachable set with no global mutation and no reverse
index. Handlers that try to compose a now-gone op receive `NOT_FOUND` (if
the overlay was already dropped when `invoke()` runs the reachability
check) or a connection error with code `INTERNAL` (if the call was
dispatched to the forwarding handler and the connection drops mid-flight).
Both cases are clean failures — no stale-handler-binds-to-dead-connection
hazard.
### 4. Curated operations remain immutable; imported and session ops are dynamic
The blanket immutability claim in `operation-registry.md` is replaced by:
- **Layer 0 (curated, `Local`)**: immutable after startup. The
`OperationRegistry` holding curated ops is constructed once by the
assembly layer and never mutated. This is where the security argument for
immutability applies: composing ops are privileged, the startup trust
boundary is where that privilege is granted, immutability locks it.
- **Layer 1 (session, `Session`)**: dynamic, per-session. Created at sandbox
creation, destroyed at session end. Already specified by OQ-19.
- **Layer 2 (imported, `FromCall` etc.)**: dynamic, per-connection. Created
when a peer connection completes `from_call` discovery, destroyed when the
connection closes.
Adding a `Local` op at runtime is not supported — it would require re-entering
the startup trust boundary, which is a deployment (restart), not a runtime
operation. This preserves the security property ADR-010/OQ-04 were concerned
with, scoped to where it actually applies.
### 5. `from_call` imports into the connection's overlay, not the global registry
The `from_call` adapter (ADR-022) discovers operations on a remote peer and
produces `HandlerRegistration` bundles. Under ADR-019, those bundles are
registered into the **connection's overlay**, not a global mutable registry.
```rust
// On CallConnection establishment:
let imported = from_call(&connection, config).await;
connection.imported_operations.extend(imported);
// The connection's env now includes these ops.
```
The handler closures produced by `from_call` capture the `CallConnection` —
when the connection drops, the handlers become unreachable (their env is
dropped), and any in-flight calls to them return connection errors. This is
the natural lifecycle; no explicit deregistration is needed.
### 6. `from_openapi` and `from_mcp` default to startup import into Layer 0
For the common case — the assembly layer imports a static OpenAPI spec or
connects to a known MCP server at startup — `from_openapi` / `from_mcp`
register into the curated (Layer 0) registry, which is then frozen. This
preserves the pre-ADR-019 behavior for the case where it was correct.
Runtime `from_openapi` / `from_mcp` import (e.g., discovering an MCP server
at connection time) is permitted and follows the Layer 2 model — the imported
ops live in a connection/discovery-scoped overlay. This is additive and
does not affect the startup-import path.
### 7. OQ-04 scope clarification and OQ-19 generalization
This ADR amends OQ-04 to scope its immutability claim to the
**`HandlerRegistry`** (ALPN-level, ADR-010). The `OperationRegistry`'s
mutability profile is now governed by this ADR: curated (Layer 0) is
immutable; session and imported layers are dynamic at their trust-boundary
scopes. See the OQ-04 amendment in `open-questions.md`.
This ADR generalizes OQ-19's session-overlay mechanism to also cover
connection-scoped remote imports. Both are per-scope dynamic overlays on the
static curated base, composed into the per-call `OperationContext.env` by
the `CallAdapter`. `OperationEnv` being a trait object is what enables
both. See the OQ-19 resolution update in `open-questions.md`.
## Consequences
**Positive:**
- `from_call` has a coherent home. Imported ops live with the connection
that produced them, appear when the connection is established, and
disappear when it closes. No contradiction with immutability, no awkward
"import everything at startup" workaround.
- The immutability argument is now correctly scoped. Layer 0 (curated,
composing ops) is immutable because that's where the security control
applies. Layers 1 and 2 are dynamic because their trust boundaries are
per-scope. An implementer reading the spec sees the right constraint in
the right place, instead of a blanket claim that doesn't fit all cases.
- The `OperationEnv`-as-trait constraint (OQ-19) is now required by the
overlay model, not just by the session-overlay pattern. The same
mechanism (trait layering) supports both session overlays and connection
overlays — one pattern, two scopes. This makes C6's resolution
(`env: Arc<dyn OperationEnv>`) structurally motivated, not just a
type-system cleanup.
- Disconnect handling is structural. A connection drops → its overlay drops
→ its ops vanish from the reachable set. No `ArcSwap` coordination, no
reverse index from op to owning connection, no stale handlers bound to a
dead connection. This is the same lifecycle property session overlays
already have (session ends → session overlay drops).
- Source isolation is structural. Imported ops from peer X are only
reachable from handlers whose `OperationEnv` is wired to X's overlay.
They are not globally callable. A handler that shouldn't be able to
reach peer X's ops simply doesn't have X's overlay in its env. This is
better hygiene than a global registry with namespace prefixes, where
every handler sees every imported op and isolation is a naming
convention.
- The `IdentityProvider` precedent makes the design legible. A future
reader sees "trait-object integration point, source composition as impl
detail" and recognizes the pattern; they don't have to re-derive why
trait-composed overlays were chosen over a global mutable registry.
**Negative:**
- The dispatch path is a composite lookup (session → connection → curated)
rather than a single `HashMap` lookup. This is a small constant cost —
three hash lookups in the worst case instead of one — and the overlays are
small (a session's ops, a connection's imported ops). The common case
(composing a curated op) hits Layer 0 after two empty-overlay misses, which
is a predictable and cache-friendly path. The cost is justified by the
source isolation and lifecycle properties it buys.
- `OperationContext.env` is now `Arc<dyn OperationEnv + Send + Sync>`, which
is a trait object with dynamic dispatch. This is the same cost as
`Arc<dyn IdentityProvider>` — a vtable call per `invoke()`. Negligible
relative to the work an operation does, and the same pattern the codebase
already uses for auth.
- The `CallAdapter` has more responsibility: it composes the root env per
call from the active layers, rather than handing every call the same
global registry. This is expected — the CallAdapter is the integration
point for the call protocol, and per-call env composition is the same
shape as per-call identity resolution (which the CallAdapter already does
via `IdentityProvider`).
- Naming across overlays: if two connections import ops with the same name
(e.g., both peers expose `worker/exec`), the composite env dispatches to
the first overlay that contains the name. This is the same ambiguity
`FromCallConfig`'s namespace prefix (ADR-022) was designed to address —
the caller disambiguates with a prefix at import time. ADR-019 does not
change this; it makes the disambiguation structural (which overlay is in
the env) rather than nominal (which prefix is in the name).
- The blanket immutability claim in `operation-registry.md` and the
cross-references that inherit it (the "Two-way door —
`ArcSwap<OperationRegistry>` can be added later" note, OQ-04's framing)
must be updated. This is a spec edit, not a migration — no implementation
exists yet.
**On review #002 findings resolved by this ADR:**
- **C6** (`OperationContext.env` type identity crisis): resolved by Decision 2.
The field is split into `scoped_env` (reachability data) and `env` (dispatch
trait object). The split is structurally motivated by the overlay model,
not just a type-system cleanup.
- **W4** (hot-swap ↔ registry mutability coupling): localized to the
connection scope. There is no global mutable registry to hot-swap.
Overlays are per-scope and replace naturally with connect/disconnect and
session start/end. The schema-drift hazard (a peer re-runs
`services/list` on reconnect and re-imports with a changed schema) moves
from global to per-connection — it does not vanish. A handler
mid-composition whose peer reconnects with a changed schema sees the old
schema until the overlay is rebuilt. This is a per-connection concern,
not a global one; the guard clause the review asked for becomes a note on
overlay rebuild semantics rather than a global hot-swap protocol.
- **W3** (CallClient registry security dimension): partially addressed. The
*registry-shape* sub-question is resolved by the overlay model — a
`CallClient`'s incoming-call dispatch uses the same overlay composition,
and sharing the curated base with a remote peer is fine (curated ops are
trusted). The *capability-exposure* sub-question (a remote peer calling
`/llm/generate` uses the local node's API key) is **not resolved by this
ADR** — it is a separate concern about what capabilities a remote peer
can trigger, and it is unaffected by the registry shape. That sub-question
remains open for ADR-022 (a guard-clause note: a peer-scoped subset must
filter by capability remote-safety, not just operation name). ADR-019
resolves the dispatch shape; ADR-022 retains the capability-exposure
decision.
## Assumptions
1. **Provenance is knowable at registration time and stable for the
registration's lifetime.** A `Local` op does not become `FromCall` later;
a `FromCall` op does not become `Local`. If a remote-imported op is later
"promoted" to curated, that's a re-registration at the next startup
(deployment), not a runtime mutation. Inherited from ADR-018 Assumption 2.
2. **Layer 0 immutability is the security control for composing ops.** The
pre-ADR-019 blanket immutability claim was overbroad but not wrong about
`Local` ops. Curated composing ops must be immutable because the startup
trust boundary is where their authority is granted. This ADR narrows the
claim, it does not remove it.
3. **Imported and session ops do not need immutability as a security
control for privilege escalation.** Their security against privilege
escalation is bounded by provenance (no composition authority → no
privilege escalation) and by the parent handler's scoped env
(reachability control). This is the central argument; if it's wrong —
if a `from_call` op can escalate in some way provenance + scoped env
don't bound — the model needs revisiting. **Immutability is not the
control for non-escalation threats** (availability, schema drift):
availability is bounded by per-handler timeouts (ADR-020) and the
connection's overlay being drop-on-disconnect; schema drift on
reconnect is a per-connection overlay-rebuild concern (see W4 in
Consequences), not a global-registry-mutation concern. The point of
scoping immutability to Layer 0 is that immutability is the right
control *for composing ops* and the wrong control *for non-composing
ops*; it is not a claim that non-composing ops face no threats.
4. **A connection's overlay is the right scope for `from_call` imports.**
Operations discovered from peer X are reachable from handlers whose env
includes X's overlay. If a use case requires imported ops to be globally
reachable (every handler sees every peer's ops), the composite env can be
built to include all active connection overlays — but the default is
per-connection scoping for isolation.
5. **Disconnect → overlay drop → op vanishes is acceptable behavior.** A
handler composing an op whose peer has disconnected receives `NOT_FOUND`
(or a connection error if the in-flight call was mid-dispatch). This is
the same behavior as a peer that never exposed the op. If a use case
requires disconnected-peer ops to remain reachable (e.g., cached results),
that's a handler-level caching concern, not a registry concern.
6. **The root env is composed per incoming call, not cached per
connection.** The active session overlay can change during a connection's
lifetime (a session starts or ends mid-connection), so the env cannot be
composed once at connection establishment and reused. `build_root_context`
runs per `call.requested` and composes the env from the layers active at
that moment. The cost (constructing an `Arc<CompositeOperationEnv>` per
call) is negligible — it's three `Arc::clone`s, not three registry
traversals.
7. **Session-overlay attachment is an agent-crate concern.** ADR-019
generalizes OQ-19's session overlay to also cover connection overlays,
but the mechanism by which a session overlay attaches to a given wire
call (session ID in metadata, payload field, connection-bound session
state, etc.) is not specified here. The `CallAdapter` is wired with an
optional session-overlay source by the assembly layer; the lookup
mechanism belongs to the agent crate spec (OQ-19: "the agent-specific
mechanism belongs to the agent crate spec"). If a wire call has no
active session, the root env is `curated base + connection overlay`
(no session layer).
## References
- ADR-010: ALPN router and endpoint (the `HandlerRegistry` immutability
argument — this ADR clarifies that it applies to the ALPN registry, not
the operation registry)
- ADR-010: Secret material flow and capability injection (capabilities are
per-`HandlerRegistration` bundle, not per-registry — the overlay model
doesn't change how capabilities flow; an imported op's capabilities come
from its bundle, which for `from_call` is whatever the assembly layer
granted the import)
- ADR-022: Call protocol client and adapter contract (`from_call` adapter;
the `FromCallConfig` namespace prefix is the disambiguation mechanism this
ADR's overlay model uses structurally)
- ADR-018: Handler registration, provenance, and composition authority
(provenance is the axis this ADR's layering is based on; the
`HandlerRegistration` bundle shape is unchanged)
- ADR-003: Auth as shared core (`IdentityProvider` — the precedent for the
trait-object integration point pattern this ADR applies to `OperationEnv`)
- OQ-04: Dynamic handler registration (this ADR amends OQ-04 to scope it to
the `HandlerRegistry`; the operation registry's mutability is now governed
by ADR-019)
- OQ-19: Session-scoped operation registries (this ADR generalizes the
session-overlay mechanism to connection overlays — same pattern, two
scopes)
- docs/reviews/002-pre-implementation-architecture-sanity-check.md
(findings C6, W3, W4 — resolved by this ADR)

257
docs/architecture/decisions/020-abort-cascade-for-nested-calls.md Normal file
View File

@@ -0,0 +1,257 @@
# ADR-020: Abort Cascade for Nested Calls
## Status
Accepted
## Context
The call protocol allows handlers to compose other operations through
`OperationEnv::invoke()`. This creates a call tree: a parent request spawns
children (via `parent_request_id`), which may spawn their own children. The
tree is the agency chain (ADR-017) — principal delegates to agent, agent may
delegate to sub-agent.
When `call.aborted` arrives for a parent request, the current `PendingRequestMap`
removes only that single entry. The children are unaware — they continue running,
consuming resources, and potentially producing side effects. This is the nested
abort problem:
```
Client calls /agent/chat (r1)
agent handler calls /fs/readFile via env.invoke (r1-a)
fs handler calls /db/query via env.invoke (r1-a-1)
agent handler calls /bash/exec via env.invoke (r1-b)
Client aborts r1 (call.aborted { id: "r1" })
→ r1 removed from PendingRequestMap
→ r1-a, r1-a-1, r1-b continue running (ghost work)
→ bash/exec keeps executing (unwanted side effect)
→ db/query keeps running (wasted resources)
→ results produced that nobody consumes
```
The `@alkdev/flowgraph` TypeScript package solved this with a directed graph
that tracks the call tree and a `FailurePolicy` enum:
- `"abort-dependents"`: aborting a node cascades to all non-terminal descendants.
This is the "whole tree should abort" behavior.
- `"continue-running"`: only idle/waiting dependents are aborted; started ones
keep going. New ones don't start because their predecessors failed/aborted.
The agent use case makes this concrete and urgent: an LLM composes deep, dynamic
call trees (parallel tools, sequential tools, sub-agents calling sub-tools).
Aborting a chat should tear down the entire tree — the LLM HTTP stream, all tool
calls, all sub-calls. But this is a protocol-level concern, not an agent feature:
every consumer (NAPI adapter, Python adapter, any service speaking EventEnvelope)
inherits whatever abort model the protocol defines. The call protocol is a
general-purpose cross-boundary RPC mechanism; nested composition is a core
protocol feature, and abort semantics for that composition are protocol semantics.
## Decision
### 1. `call.aborted` cascades to descendants
When `call.aborted` arrives for a request, the protocol cascades the abort to
all non-terminal descendants in the call tree (identified via `parent_request_id`).
Each descendant receives a `call.aborted` event. The `PendingRequestMap` removes
all affected entries.
The cascade is protocol-level: the event schema carries cascade semantics. A
`call.aborted` for a parent implies abort of all descendants. This is not a
client-side convention — the server (CallAdapter) is responsible for discovering
descendants and propagating the abort.
### 2. Default policy: `abort-dependents`
The default policy is `abort-dependents`: aborting a request aborts everything
downstream, regardless of branch. This is the correct default because aborted
parent work has no consumer waiting for results — continuing is wasted work at
best and unwanted side effects at worst (e.g., a `bash/exec` that keeps running
after the caller stopped caring, a DB mutation that completes after the
transaction was aborted).
### 3. Opt-in policy: `continue-running`
An opt-in `continue-running` policy is available for cases where long-running
work should survive a parent's abort. Under `continue-running`:
- Descendants that have already started (status: running) continue to completion.
- Descendants that haven't started yet (status: pending/waiting) are aborted
(their predecessors failed, so they can't proceed).
- No new descendants start (the parent is gone).
Use cases for `continue-running`: a long-running subscription that should keep
streaming after its parent's sibling failed; a background task that was spawned
by a handler and should survive the handler's abort.
The caller or handler specifies the policy at call time. The policy is set
on the `OperationContext` and propagated to children via `OperationEnv::invoke()`
— see Decision 6 below. The default is `abort-dependents`; `continue-running`
is an opt-in for long-running work that should survive a parent's abort.
### 4. Cleanup hooks
When a call is aborted, handlers need a mechanism to clean up resources: cancel
an HTTP stream, cancel a honker queue job, close a file handle, release a lock.
The protocol provides this through the call lifecycle — when a call is aborted,
the handler's task is cancelled (in Rust, the future is dropped). Cleanup is
handled by `Drop` implementations on resource guards, or by explicit
cancellation callbacks if the handler registers them.
This is a handler-level concern, not a protocol-level one. The protocol's job is
to cascade the abort; the handler's job is to clean up when cancelled. The
mechanism (tokio `CancellationToken`, `Drop` guards, explicit callbacks) is a
two-way door for implementation.
### 5. The call tree is tracked via `parent_request_id`
The call tree is already recorded: `OperationContext.parent_request_id` links
each call to its parent. The cascade mechanism walks this tree to find
descendants. No separate graph structure is required at the protocol level —
the `PendingRequestMap` can index entries by `parent_request_id` to enable
efficient descendant lookup.
The `@alkdev/flowgraph` package (directed graph with `descendants()`,
reactive status propagation, `FailurePolicy`) is prior art and may be adapted
as a separate Rust crate for consumers that need richer call-tree visualization
or reactive status tracking. It is not required for the protocol-level cascade
— a parent-indexed map suffices.
### 6. The abort policy is set on `OperationContext`, not on the wire payload
The abort policy (`abort-dependents` vs `continue-running`) is set on
`OperationContext` and propagated to children via `OperationEnv::invoke()`.
It is NOT a field in the `call.requested` wire payload, and it is NOT a
per-operation declaration on `OperationSpec`.
**Why not the wire payload**: the wire caller doesn't know the composition
tree. The caller of `/agent/chat` cannot meaningfully decide whether
`/fs/readFile` (composed internally by the agent handler) should survive an
abort — the handler that composes the child knows that, not the wire caller.
Putting the policy on the wire payload would give the wire caller control
over internal composition behavior it can't see.
**Why not per-operation declaration**: Assumption 5 says the policy
is per-call, not per-operation. The same operation may need
`abort-dependents` in one composition context and `continue-running` in
another. A static property on `OperationSpec` can't express that.
**How it works on `OperationContext`**: the root context
(`build_root_context` in the CallAdapter) gets the default policy
(`abort-dependents`). When a handler composes a child via
`env.invoke()`, it can specify the policy for that child:
```rust
// Default: abort-dependents (child aborts if parent aborts)
context.env.invoke("fs", "readFile", input, &context).await
// Opt-in: continue-running (child survives parent's abort)
context.env.invoke_with_policy(
"fs", "readFile", input, &context, AbortPolicy::ContinueRunning
).await
```
The child's `OperationContext` carries the policy. If the child itself
composes grandchildren, the policy **propagates by inheritance** — the
grandchild inherits the child's policy (which was the parent's policy,
unless the parent overrode it for the child via `invoke_with_policy`).
`ContinueRunning` does auto-propagate to grandchildren: if a parent opts
its child into `ContinueRunning`, and the child composes grandchildren
without explicitly overriding, the grandchildren also get
`ContinueRunning`. This is consistent with the composition authority and
scoped env propagation in ADR-018 — the parent handler decides the
child's runtime context, including abort policy, and that decision
propagates through the composition tree by default.
**Review #002 W19 resolution**: `invoke()` with no explicit policy
argument inherits the parent's current policy (option a). It does **not**
reset to `AbortDependents`. A handler that wants a child to reset to the
default must explicitly call `invoke_with_policy(...,
AbortPolicy::AbortDependents)`. This makes the propagation predictable:
the policy I set for my child applies to my child's children unless they
re-decide. The `invoke()` default in operation-registry.md
(`abort_policy: parent.abort_policy.clone()`) is correct.
The `OperationEnv` trait gains an optional policy parameter. The specific
API shape (a separate `invoke_with_policy` method, a policy field on an
`InvokeOptions` struct, or a builder pattern) is a two-way door for
implementation — but the policy enters through `OperationEnv::invoke()`,
not through the wire and not through `OperationSpec`.
## Consequences
**Positive:**
- No ghost work. Aborting a parent call tears down the entire tree. Resources
are released, side effects are halted, no results are produced for absent
consumers.
- The default (`abort-dependents`) matches the intuitive expectation: if I
stop caring about the parent, I stop caring about everything it spawned.
- The opt-in (`continue-running`) covers the legitimate exception (long-running
work that should survive) without making it the default.
- The protocol carries cascade semantics, so every consumer inherits the
correct behavior — no consumer needs to implement its own abort propagation.
- The `parent_request_id` chain already exists; the cascade mechanism is an
index on it, not a new data structure.
- Cleanup hooks are handled by Rust's async drop semantics — dropping the
handler's future cancels it, and `Drop` guards release resources. This is
idiomatic Rust, not a custom mechanism.
**Negative:**
- The `PendingRequestMap` needs a parent-indexed lookup (a `HashMap<String,
Vec<String>>` from parent_request_id to child request_ids, or a scan). This
is a minor implementation cost, not a protocol change.
- The `call.aborted` event schema carries cascade semantics — clients that
don't understand cascade (future versions, other implementations) would
need to handle it. Mitigated: cascade is server-side (the CallAdapter walks
the tree and sends `call.aborted` per descendant), so clients see individual
abort events regardless of whether they understand the cascade concept.
- The `continue-running` policy adds a parameter to the call lifecycle. The
specific location (payload field, context field, per-operation declaration)
is a two-way door, but the existence of the policy is a one-way commitment.
## Assumptions
1. **Aborting a parent should abort descendants by default.** If the default
should be `continue-running` (descendants survive), this ADR is wrong. The
assumption is that ghost work is worse than premature cancellation — a
cancelled descendant can be retried, but a ghost process consuming
resources and producing unwanted side effects is harder to recover from.
2. **The server (CallAdapter) is responsible for cascade.** The client sends
`call.aborted` for one request ID; the server discovers descendants and
propagates. If the client were responsible for cascading, it would need to
know the full tree — which it may not (server-side composition creates
children the client never saw).
3. **`parent_request_id` is sufficient to discover descendants.** The call tree
is a tree (acyclic, single parent per node). If future composition patterns
create multi-parent relationships (e.g., a shared subcall invoked by two
parents), the cascade model needs extension. The assumption is that
composition creates a tree, not a DAG.
4. **Dropping the handler's future is sufficient for cleanup.** Rust's async
drop semantics cancel the future and run `Drop` guards. If a use case
requires explicit cleanup callbacks (e.g., external systems that need a
signal), the mechanism needs extension. The assumption is that `Drop`
guards cover the common cases (HTTP stream cancellation, file handle
release, lock release).
5. **`continue-running` is per-call, not per-operation.** The policy is
specified at call time via `OperationEnv::invoke()`, not declared at
registration on `OperationSpec` and not set by the wire caller. The
composing handler decides the child's policy based on the specific
context. See Decision 6.
## References
- ADR-015: Call protocol stream model (bidirectional streams, EventEnvelope,
ID-based correlation)
- ADR-017: Privilege model (the call tree is the agency chain —
`parent_request_id` traces principal → agent)
- OQ-17: Abort cascade semantics (resolved by this ADR)
- OQ-19: Session-scoped registries (session-scoped operations are in the call
tree and participate in cascade)
- `@alkdev/flowgraph` TypeScript package — prior art for call-graph tracking
with `descendants()`, `FailurePolicy`, reactive status propagation
- [call-protocol.md](../crates/call/call-protocol.md)
- [operation-registry.md](../crates/call/operation-registry.md)

337
docs/architecture/decisions/021-streaming-handler-for-subscriptions.md Normal file
View File

@@ -0,0 +1,337 @@
# ADR-021: Streaming Handler for Subscription Operations
## Status
Accepted
## Context
The call protocol defines `Subscription` as a first-class operation type
(ADR-015 lists `subscribe` as one of four top-level protocol operations;
`OperationSpec.op_type` includes `Subscription`). The wire protocol supports
streaming: five event types (`call.requested`, `call.responded`,
`call.completed`, `call.aborted`, `call.error`), `PendingRequestMap::Subscribe`
with an mpsc channel, `CallConnection::subscribe()` returning
`impl Stream<Item = ResponseEnvelope>`, and a full streaming-subscribe example
in `call-protocol.md`. The **client side** works — a client can subscribe to a
remote stream and consume `call.responded` events until `call.completed`.
The **server side does not.** The `Handler` type in `alknet-call` is:
```rust
pub type Handler = Arc<
dyn Fn(Value, OperationContext) -> Pin<Box<dyn Future<Output = ResponseEnvelope> + Send>>
+ Send + Sync,
>;
```
It returns a single `ResponseEnvelope`. `OperationRegistry::invoke()` returns
one `ResponseEnvelope` and closes. `Dispatcher::handle_stream` calls
`dispatch_requested` → `registry.invoke()` → writes one `EventEnvelope` frame →
loops. A `Subscription` operation that should produce a *stream* of
`call.responded` events followed by `call.completed` has no way to express that
through this handler signature.
This is a **spec gap that should not have shipped.** The TypeScript predecessor
(`@alkdev/operations`, from which the Rust port was derived) had two distinct
handler types:
```typescript
type OperationHandler<I, O, C> = (input: I, context: C) => Promise<O> | O;
type SubscriptionHandler<I, O, C> = (input: I, context: C) => AsyncGenerator<O, void, unknown>;
```
The TS registry (`registry.ts:21`) stored them as a union, validated at
registration that `SUBSCRIPTION` ops get an `AsyncGeneratorFunction`, and the
server-side dispatch (`call.ts:341-349`, `buildCallHandler`) branched on
`op_type`: `SUBSCRIPTION` → iterate the async generator, `respond()` for each,
then `complete()`; else → `execute()`, single `respond()`. The Rust port
collapsed the union into a single `Handler` returning one `ResponseEnvelope`,
losing the streaming path. The fix is to restore it.
The downstream consequences of the gap:
- **`/subscribe` HTTP endpoint** (`GatewayDispatch::invoke()` →
`subscribe_handler`) wraps a single `ResponseEnvelope` in a one-event SSE
stream. A real `Subscription` operation (e.g., `agent/chat` streaming LLM
tokens) cannot stream through it.
- **`from_call` forwarding** for a `Subscription` op calls
`CallConnection::call_with_payload()` (single response), not
`CallConnection::subscribe()` (stream). A `from_call`-imported subscription
truncates to the first value.
- **`from_openapi` forwarding** for a `text/event-stream` response returns one
`ResponseEnvelope` instead of streaming the SSE chunks.
All three are symptoms of the same root cause: the `Handler` type cannot
produce a stream.
## Decision
### 1. `StreamingHandler` type (alongside `Handler`)
Add a streaming handler type that returns a stream of `ResponseEnvelope`s,
mirroring the TS `SubscriptionHandler` / `OperationHandler` split:
```rust
pub type StreamingHandler = Arc<
dyn Fn(Value, OperationContext)
-> Pin<Box<dyn Stream<Item = ResponseEnvelope> + Send>>
+ Send + Sync,
>;
```
Each `Ok(value)` in the stream becomes a `call.responded` event. An `Err`
becomes a `call.error` event (terminal — the stream ends). Natural stream end
becomes `call.completed`. The dispatch path converts each `ResponseEnvelope` to
`EventEnvelope` exactly as it does today for the single-response case — no new
wire-format concept is introduced.
A `make_streaming_handler()` helper (analogue of `make_handler()`) wraps an
async generator / stream-producing closure into a `StreamingHandler`.
### 2. `HandlerKind` enum on `HandlerRegistration`
```rust
pub enum HandlerKind {
Once(Handler),
Stream(StreamingHandler),
}
pub struct HandlerRegistration {
pub spec: OperationSpec,
pub handler: HandlerKind, // validated against spec.op_type at registration
pub provenance: OperationProvenance,
pub composition_authority: Option<CompositionAuthority>,
pub scoped_env: Option<ScopedPeerEnv>,
pub capabilities: Capabilities,
}
```
Registration validates: `Query` / `Mutation` → `HandlerKind::Once`;
`Subscription` → `HandlerKind::Stream`. Mismatch is a startup error (same as
the TS `validateSubscriptionHandler`). The enum makes the "one or the other,
matching `op_type`" invariant type-level rather than two `Option`s validated at
runtime.
### 3. `OperationRegistry::invoke_streaming()`
```rust
impl OperationRegistry {
/// Dispatch a Subscription operation. Returns a stream of
/// ResponseEnvelopes. Errors (not-found, forbidden, invalid operation
/// type) yield a single ResponseEnvelope::error and end the stream.
pub fn invoke_streaming(
&self,
name: &str,
input: Value,
context: OperationContext,
) -> BoxStream<ResponseEnvelope>;
}
```
`invoke_streaming()` performs the same visibility + ACL checks as `invoke()`,
then dispatches to the `StreamingHandler`. Pre-handler errors (not-found,
forbidden) produce a single error `ResponseEnvelope` and end the stream
(matching the single-response path's behavior, just on a stream).
### 4. `OperationRegistry::invoke()` errors on `Subscription`
`invoke()` is the request/response dispatch path. Calling it on a
`Subscription` op is a type mismatch — a streaming operation dispatched through
the request/response path. It returns a `ResponseEnvelope` carrying
`CallError { code: "INVALID_OPERATION_TYPE", ... }` (a new protocol-level
code):
```
INVALID_OPERATION_TYPE
```
(`retryable: false`, `details: None`). This is the wire-format addition: a
sixth protocol-level error code. It signals "you called the wrong dispatch
method for this operation's type" — distinct from `INVALID_INPUT` (schema
mismatch) and `INTERNAL` (handler failure). Clients should treat unknown codes
as `INTERNAL` with `retryable: false` (the existing rule); `INVALID_OPERATION_
TYPE` is a permanent client-side programming error, not a transient failure.
### 5. `OperationEnv::invoke()` errors on `Subscription`
`OperationEnv::invoke()` (composition) stays request/response-only. It returns
a single `ResponseEnvelope`. Calling it on a `Subscription` op produces the
same `INVALID_OPERATION_TYPE` error — composition cannot truncate a stream to
its first value. This is a clean architectural boundary, not a deferral:
- **`OperationEnv` composition** is "call a child operation, get a result"
(the `OperationHandler` model). It is request/response by construction.
- **Stream composition** (filter, map, combine, window, dedupe) is a
handler-level concern. A handler that produces a stream transforms it with
stream operators at the handler level, not through `OperationEnv`. The
`@alkdev/pubsub` `operators.ts` is the prior art for this model: 13 operators
(`filter`, `map`, `take`, `batch`, `dedupe`, `window`, `chain`, `join`, etc.)
that operate on `AsyncIterable<T>`, distinct from the request/response
composition. In Rust, the analogues operate on `BoxStream<T>`.
- No `invoke_streaming()` is added to `OperationEnv`. The protocol composition
surface is request/response; stream manipulation is handler-internal.
### 6. Server-side dispatch branches on `op_type`
`Dispatcher::handle_stream` / `dispatch_requested` gains a branch on
`op_type`:
- `Subscription` → `registry.invoke_streaming()` → for each `ResponseEnvelope`
in the stream, write `EventEnvelope` to the wire → write `call.completed` on
stream end.
- `Query` / `Mutation` → `registry.invoke()` → write one `EventEnvelope`
(existing path, unchanged).
The streaming branch sets `deadline: None` for subscriptions (unbounded —
already specced in `call-protocol.md` Timeouts) and wires abort cascade
(ADR-020): if `call.aborted` arrives for a streaming request, the stream is
dropped (Rust `Drop` releases the handler's resources).
### 7. `GatewayDispatch::invoke_streaming()` (alknet-http)
The shared dispatch spine gains a streaming variant:
```rust
impl GatewayDispatch {
pub async fn invoke_streaming(
&self,
identity: Option<Identity>,
op: &str,
input: Value,
) -> BoxStream<ResponseEnvelope>;
}
```
`invoke_streaming()` builds the root `OperationContext` identically to
`invoke()` (same security invariants: `internal: false`, `forwarded_for:
None`, same capabilities, same `scoped_env`), then calls
`registry.invoke_streaming()`. The two gateways (`to_openapi`, `to_mcp`)
diverge only on wire-framing; the security axis is provably identical between
`invoke()` and `invoke_streaming()`.
The HTTP `/subscribe` handler calls `invoke_streaming()` and pipes the
`BoxStream<ResponseEnvelope>` to SSE: each `Ok(value)` → SSE `data:` frame,
`Err` → SSE error event + close, stream end → close. This replaces the current
one-event `subscribe_stream_from_envelope` with the real streaming path.
### 8. `from_call` stream forwarding
The `from_call` forwarding handler construction branches on `op_type` during
discovery:
- `Query` / `Mutation` → existing `make_forwarding_handler()` (calls
`CallConnection::call_with_payload()`, returns single `ResponseEnvelope`),
registered as `HandlerKind::Once`.
- `Subscription` → new `make_streaming_forwarding_handler()` (calls
`CallConnection::subscribe()`, returns `impl Stream<Item =
ResponseEnvelope>`, maps to `BoxStream<ResponseEnvelope>`), registered as
`HandlerKind::Stream`.
A `from_call`-imported `Subscription` op forwards the remote stream end-to-end:
the client-side `CallConnection::subscribe()` (already working) feeds a
`StreamingHandler` that produces the stream. No truncation, no first-value
fallback.
### 9. `from_openapi` SSE forwarding
The `from_openapi` forwarding handler construction branches on `op_type`
(determined by `detectOperationType` — `text/event-stream` response →
`Subscription`):
- `Query` / `Mutation` → existing forwarding handler (single HTTP request →
single `ResponseEnvelope`), `HandlerKind::Once`.
- `Subscription` → streaming forwarding handler (HTTP request → SSE response
stream → parse SSE chunks → `BoxStream<ResponseEnvelope>`), `HandlerKind::
Stream`.
The SSE parsing reuses the TS `parseSSEFrames` pattern: each SSE `data:` frame
becomes a `ResponseEnvelope::ok()`, SSE stream end becomes stream end (→
`call.completed`).
## Consequences
**Positive:**
- `Subscription` operations work end-to-end: server-side handler →
server-side dispatch → wire → HTTP `/subscribe` SSE → `from_call`
forwarding → `from_openapi` SSE forwarding. No truncation, no broken paths.
- The `Handler` / `StreamingHandler` split mirrors the TS prior art exactly,
making the Rust port faithful to its source.
- `HandlerKind` makes the "one or the other, matching `op_type`" invariant
type-level (a `Once` variant for `Query`/`Mutation`, a `Stream` variant for
`Subscription`) rather than a runtime check on two `Option`s.
- Existing handlers (echo, discovery, from_openapi Query/Mutation, from_mcp,
from_call Query/Mutation) are unchanged — they return a single
`ResponseEnvelope` and register as `HandlerKind::Once`. The streaming path
is additive to the existing handler surface.
- `OperationEnv` composition stays request/response, preserving the
composition model's simplicity. Stream composition is a handler-level
concern, cleanly separated.
- The new `INVALID_OPERATION_TYPE` protocol code catches dispatch-path misuse
(calling `invoke()` on a `Subscription`) at the protocol level instead of
silently producing wrong behavior.
**Negative:**
- `HandlerRegistration.handler` changes type from `Handler` to `HandlerKind`.
Existing code constructing `HandlerRegistration` bundles must wrap in
`HandlerKind::Once(...)`. This is a mechanical change across handler
construction sites (the builder's `.with_local()` / `.with_leaf()` /
`.with()` methods absorb the wrapping internally, so most assembly-layer
code is unaffected; direct `HandlerRegistration::new()` calls need the
wrap).
- A new protocol-level error code (`INVALID_OPERATION_TYPE`) is a wire-format
addition. Existing clients that treat unknown codes as `INTERNAL` with
`retryable: false` (the existing rule) handle it correctly — they just
don't distinguish it from `INTERNAL` until updated. The code is distinct
from all existing codes and from operation-level domain codes (no
`HTTP_` prefix, no collision with the five existing protocol codes).
- The `Dispatcher::handle_stream` streaming branch adds a stream-to-wire
pump (read stream → write frames → write `call.completed`). This is new
code in the hot dispatch path, but it is a straightforward `while let
Some(envelope) = stream.next().await` loop, not a complex abstraction.
## Door type
**One-way.** The `Handler` / `StreamingHandler` / `HandlerKind` API surface
is what handlers are written against across crates (`alknet-call`,
`alknet-http`, downstream consumers). Changing it after handlers exist is a
rewrite. The `INVALID_OPERATION_TYPE` wire code is also one-way — once
emitted, clients may handle it, and removing it would break those handlers.
The `HandlerKind` enum shape (`Once(Handler) | Stream(StreamingHandler)`) is
the one-way commitment: two handler variants, validated against `op_type`.
The concrete `BoxStream` library choice (`futures::stream::BoxStream` vs a
custom type) is a two-way-door implementation detail within the one-way
decision.
## References
- ADR-015: Call Protocol Stream Model (defines `subscribe` as a top-level
protocol operation; the streaming path this ADR implements)
- ADR-022: Call Protocol Client and Adapter Contract (the adapter contract
this ADR extends with the `StreamingHandler` variant)
- ADR-018: Handler Registration, Provenance, and Composition Authority
(`HandlerRegistration` gains `HandlerKind`)
- ADR-017: Privilege Model and Authority Context (visibility/ACL checks run
identically in `invoke_streaming()` as in `invoke()`)
- ADR-020: Abort Cascade for Nested Calls (stream drop on abort; cascade
through streaming handlers)
- ADR-016: Operation Error Schemas (`INVALID_OPERATION_TYPE` is a
protocol-level code, distinct from operation-level domain codes)
- ADR-024: Peer-Graph Routing Model (`from_call` forwarding handlers gain the
streaming variant)
- ADR-032: One-Way Door Decision Framework (the `Handler` / `StreamingHandler`
split is a one-way door — handler API surface)
- `@alkdev/operations/src/types.ts:62-78` — TS prior art
(`OperationHandler` / `SubscriptionHandler` split)
- `@alkdev/operations/src/registry.ts:65-75` — TS prior art
(`validateSubscriptionHandler` — runtime validation against `op_type`)
- `@alkdev/operations/src/call.ts:341-349` — TS prior art (`buildCallHandler`
branches on `op_type`: `SUBSCRIPTION` → iterate + complete; else → execute)
- `@alkdev/pubsub/src/operators.ts` — stream operators prior art (filter,
map, batch, dedupe, window, chain, join — handler-level stream
composition, distinct from `OperationEnv` request/response composition)
- Spec documents amended: `operation-registry.md`, `call-protocol.md`,
`http-server.md`, `http-adapters.md`, `http-mcp.md`, `client-and-adapters.md`

488
docs/architecture/decisions/022-call-protocol-client-and-adapter-contract.md Normal file
View File

@@ -0,0 +1,488 @@
# ADR-022: Call Protocol Client and Adapter Contract
## Status
Accepted (amended 2026-06-26, 2026-07-13, and 2026-07-16 — see "Amendments" below; the 2026-07-16 amendment per ADR-045 §5 removes `CallClient::connect`)
## Context
The call protocol spec (ADR-015) defined the stream model as bidirectional —
"both sides can initiate calls." But the spec only described the server side:
`CallAdapter` implements `ProtocolHandler`, accepts incoming QUIC connections,
and dispatches to the operation registry. The client side — who opens the
connection, how calls are sent, how remote operations are discovered and
imported — was left as OQ-15.
The need for the client side is concrete and immediate:
- **Head/worker dispatch**: a head node manages worker nodes (Vast.ai, RunPod,
local Docker). The head needs to call operations on workers (exec, sync,
status) and workers need to call back (report status, request work). The
POC at `/workspace/@alkdev/dispatch` demonstrated this over SSH+axum; under
the call protocol, it's cross-node composition.
- **NAPI/Python adapters**: Node.js and Python clients need to call operations
on an alknet node. They speak the EventEnvelope wire format over a QUIC
connection.
- **Agent tool dispatch**: an agent handler needs to call operations on remote
nodes (tools, services) the same way it calls local operations — through
`OperationEnv::invoke()`. The `from_call` adapter makes remote operations
appear in the local registry.
- **Cross-protocol interop**: external systems (HTTP APIs, MCP servers) are
imported via `from_openapi` and `from_mcp`. The reverse direction —
exposing local operations to external systems — needs `to_openapi` and
`to_mcp`.
The `@alkdev/operations` TypeScript package demonstrated the adapter patterns
(`from_openapi`, `from_mcp`) and the `buildEnv` composition mechanism. The Rust
implementation defines the canonical traits (ADR-033).
OQ-15 was constrained by ADR-010 (adapters take credential sources, not static
tokens) and ADR-017 (adapter-registered operations are `Internal` by default).
This ADR locks the remaining one-way door: the client/adapter contract
architecture.
## Decision
### 1. `CallClient` opens connections and shares the dispatch loop
`CallClient` opens a QUIC connection to a remote node with ALPN `alknet/call`.
Once connected, the connection is symmetric — both sides can send and receive
`call.requested`. The `CallClient` is not just a caller; it is also a callee.
It has its own operation registry to dispatch incoming calls from the remote
side.
```rust
pub struct CallClient {
registry: Arc<OperationRegistry>,
identity_provider: Arc<dyn IdentityProvider>,
}
impl CallClient {
pub async fn connect(&self, addr: SocketAddr, credentials: CallCredentials) -> Result<CallConnection>;
}
```
The dispatch loop is shared between `CallAdapter` and `CallClient`. Once a
connection is established (whether accepted by the adapter or opened by the
client), the same logic applies: read `EventEnvelope` frames, dispatch to the
operation registry, write responses, and send outgoing `call.requested` events
for calls initiated on this side. The only difference is who opened the
connection.
`CallConnection` provides:
- `call(operation_id, input) -> ResponseEnvelope` — send `call.requested`,
await `call.responded` (one result)
- `subscribe(operation_id, input) -> Stream<ResponseEnvelope>` — send
`call.requested`, yield each `call.responded` until `call.completed` or
`call.aborted`
- `abort(request_id)` — send `call.aborted`, cascade to descendants (ADR-020)
- `services_list() -> Vec<OperationSpec>` — call `services/list`
- `services_schema(name) -> OperationSpec` — call `services/schema`
### 2. Connection direction is independent of call direction
Who opens the QUIC connection (who has the public IP, who uses a relay, who
connects out reverse-runner style) is a connection-layer concern, not a
protocol-layer concern. Once connected, both sides can call each other.
| Topology | Who advertises | Who opens connection | Who can call whom |
|----------|---------------|----------------------|-------------------|
| Public service | Server (public IP/domain) | Client | Both directions |
| P2P (iroh relay) | Both (relay-assisted) | Either | Both directions |
| Reverse (runner pattern) | Head (public IP) | Worker connects out | Both directions |
| Reverse (dispatch pattern) | Worker (public SSH port) | Head connects out | Both directions |
The protocol does not distinguish "server" and "client" after connection
establishment. The `CallAdapter` accepts connections; the `CallClient` opens
connections. Both dispatch incoming and outgoing calls through the same
mechanism.
### 3. `from_call` adapter imports remote operations
`from_call` does for call protocol endpoints what `from_openapi` does for HTTP
APIs: discovers operations and registers them in the local registry with
forwarding handlers.
```rust
pub async fn from_call(
connection: &CallConnection,
config: FromCallConfig,
) -> Vec<HandlerRegistration>
```
The adapter:
1. Calls `services/list` on the remote node → gets the list of `External`
operations
2. Calls `services/schema` for each → gets the input/output JSON Schemas and
declared error_schemas (ADR-016)
3. For each discovered operation, constructs a `HandlerRegistration` bundle:
- The spec mirrors the remote operation's name, namespace, type, schemas
(input, output, and error_schemas — ADR-016), and access control
- The handler sends `call.requested` through the `CallConnection` and awaits
`call.responded` (or streams for subscriptions)
- `provenance: FromCall`, `composition_authority: None`, `scoped_env: None`
(leaves — ADR-018)
4. The caller registers these bundles in their local registry (into the
connection's overlay — ADR-019)
`from_call`-registered operations are `Internal` by default (ADR-017) — they
are composition material, not directly callable from the wire. The handler
that composes them is `External`.
The `FromCallConfig` includes:
- The credential source for the outbound connection (ADR-010) — TLS identity,
auth token, or capability-provided credentials
- An optional namespace prefix (to avoid collisions when importing from
multiple remote nodes)
- An optional operation filter (to import only specific operations)
### 4. `to_openapi` and `to_mcp` adapters export local operations
The reverse direction — exposing local operations to external systems:
- **`to_openapi`**: generates an OpenAPI spec from the local registry's
`External` operations. External systems (HTTP clients, API gateways) can
discover and call alknet operations through a standard HTTP interface.
- **`to_mcp`**: exposes local operations as MCP tools. MCP clients (editors,
AI tools) can discover and call alknet operations through the MCP protocol.
These adapters are outbound bridges — they translate the call protocol's
operation model into external protocol formats. They do not modify the local
registry; they project it.
### 5. The adapter contract trait
The adapter patterns share a common shape: they produce
`HandlerRegistration` bundles that register in the local registry. The
trait:
```rust
#[async_trait]
pub trait OperationAdapter: Send + Sync {
async fn import(&self) -> Vec<HandlerRegistration>;
}
```
The return type is `Vec<HandlerRegistration>` (not `(OperationSpec,
Handler)` pairs) — ADR-018 changed the registration API to the bundle
shape, and adapters must produce bundles. Adapter convenience methods
construct bundles with `composition_authority: None` and `scoped_env: None`
for the leaf ops they produce.
The trait is **async** because `from_call` requires async discovery
(`services/list` + `services/schema` over a QUIC connection). A synchronous
trait cannot accommodate `from_call` without a separate async pre-step that
populates a cache. The sync adapters (`from_openapi`, `from_mcp` reading a
static spec) trivially satisfy an async trait — their `import()` bodies
contain no `.await` points. The async/sync question is decided: the trait
is async.
Implementations:
- `FromOpenAPI` — imports from an OpenAPI spec (HTTP-backed handlers)
- `FromMCP` — imports from an MCP server (MCP-backed handlers)
- `FromCall` — imports from a remote call protocol endpoint
(call-protocol-backed handlers)
- ~~`FromJsonSchema` — imports from a JSON Schema definition (schema-only, no
handler — used for validation or client generation)~~ — **superseded by
[ADR-027](027-from-jsonschema-as-http-adapter.md)**: `from_jsonschema` is
now an HTTP-backed single-endpoint adapter in `alknet-http` (reqwest
forwarding handler, not a schema-only placeholder); `FromJsonSchema`
provenance stays in `alknet-call` as a handler-bearing leaf.
The `to_*` adapters are outbound projections, not `OperationAdapter`
implementations — they consume the registry, they don't produce entries for it.
The specific trait signatures (error types, configuration parameters) are
two-way doors for implementation. The one-way doors are the architectural
commitments: adapters produce `HandlerRegistration` bundles (ADR-018), the
trait is async (required by `from_call`), and the adapter *trait* lives in
`alknet-call` while adapter *implementations* live with their transport
(HTTP-backed adapters in `alknet-http` per ADR-027; QUIC-backed `from_call`
in `alknet-call`). See `client-and-adapters.md` §"Adapter Location Map."
### 6. Cross-node call tree and abort cascade
When a `from_call` handler sends `call.requested` to a remote node, the call
participates in the local call tree via `parent_request_id`. If the parent is
aborted, the cascade (ADR-020) reaches the `from_call` handler, which sends
`call.aborted` to the remote node. The remote node cascades to its own
descendants. The abort crosses the node boundary transparently.
```
Head node Worker node
r1: /dispatch/run_training
r1-a: worker/exec (from_call handler)
→ call.requested { id: r1-a } ────────→ receives, dispatches to exec
r1-a-1: exec spawns child
user aborts r1
cascade to r1-a
from_call handler sends:
call.aborted { id: r1-a } ───────────→ receives, cascades to r1-a-1
aborts exec and children
```
### 7. Credential sources for connections
The `CallClient` needs credentials to authenticate to the remote node. These
come from capabilities (ADR-010), not environment variables. The credential
types:
- **TLS identity**: the local node's Ed25519 key (RFC 7250 raw key) or X.509
cert, derived from the vault at startup
- **Auth token**: an opaque token for call-protocol-level authentication,
decrypted from the vault or derived from a shared secret
- **Remote identity verification**: the expected fingerprint or cert of the
remote node, stored as a capability (not an env var or config file)
The `from_call` adapter receives these credentials at registration time,
same as `from_openapi` receives HTTP credentials.
## Consequences
**Positive:**
- Cross-node composition works the same as local composition. A handler calls
`env.invoke("worker", "exec", ...)` and doesn't know (or care) whether
`worker/exec` is a local operation or a `from_call`-imported remote
operation. The composition is transparent.
- The head/worker pattern (dispatch, runners) is a connection topology, not a
protocol feature. Workers can connect to heads (runner pattern) or heads can
connect to workers (dispatch pattern) — the protocol handles both.
- `from_call` is the same pattern as `from_openapi` and `from_mcp`: discover,
register, forward. The adapter contract is unified.
- `to_openapi` and `to_mcp` enable interop with non-alknet systems without
those systems needing to speak EventEnvelope.
- The abort cascade (ADR-020) crosses node boundaries transparently. No
consumer needs to implement cross-node abort propagation.
- The NAPI and Python adapters can use `CallClient` directly to call remote
operations — they don't need a separate client implementation.
**Negative:**
- `CallClient` has its own operation registry (for dispatching incoming calls
from the remote side). This is a second registry instance, not the global
one — it needs to be populated with the operations this node wants to expose
to that specific remote peer. The specific mechanism (sharing the global
registry, a peer-scoped subset, or a separate registry) is a two-way door.
- `from_call`-registered operations have a latency cost: each invocation sends
a `call.requested` over QUIC and awaits a `call.responded`. This is
inherent to remote calls and not specific to the adapter pattern. Caching
or batching strategies are consumer concerns.
- The `to_*` adapters need to translate the call protocol's operation model
(JSON Schema, EventEnvelope, subscribe/stream) into external formats
(OpenAPI paths, MCP tools). Some semantics don't map cleanly (e.g.,
subscriptions in OpenAPI, bidirectional calls in MCP). The adapters handle
these with best-effort mappings and document the gaps.
- **Published `to_*` specs are compatibility contracts.** The "best-effort"
mapping label is internal framing. Once a generated spec is published and
external clients build against it, the mapping semantics (e.g.,
subscriptions → SSE long-poll) become a de facto contract. Changing the
mapping later breaks every client. `to_*` mapping choices are two-way
*before* first publication but one-way *after*. Version the generated
specs (e.g., OpenAPI spec version tied to the registry's External
operation set version) and emit a spec version marker so consumers can
detect mapping changes. This is the "published artifact is a contract"
blind spot in ADR-032's framework: it classifies doors by reversal cost
in the codebase, not by compatibility cost for external consumers.
- **Sharing the global registry with a `CallClient` exposes local
capabilities to the remote peer.** Each `HandlerRegistration` carries
`Capabilities` with secret material. If the `CallClient` shares the
global registry, a remote peer calling an External operation triggers
dispatch that populates `OperationContext.capabilities` from the local
registration bundle — meaning the local node's API keys and signing keys
are used for the remote peer's call. A peer-scoped subset must filter by
capability remote-safety (is this operation's capability safe to expose
to this peer?), not just operation name. The registry-mechanism choice
(share global vs subset vs separate) is two-way mechanically but has a
security dimension post-ADR-018: the "share global" option is a
capability-exposure decision, not just a dispatch decision.
- The `CallConnection` abstraction adds a layer between the handler and the
raw QUIC stream. This is necessary for the `from_call` handler to be
transparent — it shouldn't know about QUIC streams, only about call/request
semantics.
## Assumptions
1. **The connection is symmetric after establishment.** Both sides can send
and receive `call.requested`. If a future use case requires one-directional
connections (e.g., a fire-and-forget notification where the receiver can't
call back), the model needs extension. The assumption is that bidirectional
is the correct default.
2. **`services/list` and `services/schema` are the discovery mechanism for
`from_call`.** The remote node exposes its `External` operations through
these built-in operations. If a remote node doesn't support service
discovery (e.g., a minimal worker that only accepts specific calls),
`from_call` needs an alternative discovery mechanism (static config, manual
spec). The assumption is that nodes participating in cross-node composition
support service discovery.
3. **The `from_call` handler is transparent to composition.** A handler that
calls `env.invoke("worker", "exec", ...)` doesn't know it's a remote call.
If the remote node is unreachable or the connection drops, the handler gets
a `call.error` (same as a local handler error). The assumption is that
remote call failures are handled the same as local handler failures.
4. **`from_call`-registered operations mirror the remote spec.** The imported
`OperationSpec` has the same name, namespace, type, schemas (input, output,
and error_schemas per ADR-016), and access control as the remote operation. If the remote operation changes (new
schema, renamed), the imported spec is stale until re-import. The
assumption is that re-import happens on reconnection or is triggered
explicitly. Hot-swapping imported specs is a two-way door.
5. **The `to_*` adapters are projections, not live bridges.** `to_openapi`
generates a spec; it doesn't proxy HTTP requests. An external HTTP client
calling the generated OpenAPI endpoints needs an HTTP handler (alknet-http)
that translates HTTP requests into call protocol operations. The assumption
is that `to_*` generates specs/tools, and a separate HTTP/MCP handler
bridges the actual traffic.
## References
- ADR-013: irpc as call protocol foundation
- ADR-015: Call protocol stream model (bidirectional streams)
- ADR-033: Rust as canonical implementation language (adapter traits in Rust)
- ADR-010: Secret material flow (credential sources, not static tokens)
- ADR-017: Privilege model (adapter ops are Internal by default)
- ADR-020: Abort cascade (cross-node abort propagation)
- ADR-023: Peer-Scoped Registry Filtering for CallClient Inbound Dispatch
(resolves the §1 Consequences security dimension flagged as a two-way door)
- OQ-15: Call protocol client and adapter contract (resolved by this ADR)
- OQ-25..28: Two-way-door remainders from the call-completion gap analysis
(DC-1 shape, DC-4 error type, DC-2 re-import trigger, DC-3 namespace
collision — see [open-questions.md](../open-questions.md))
- [call-protocol.md](../crates/call/call-protocol.md)
- [operation-registry.md](../crates/call/operation-registry.md)
- [client-and-adapters.md](../crates/call/client-and-adapters.md) — the spec
that operationally fills the gap this ADR left to implementation
- `docs/research/alknet-call-completion/gap-analysis.md` — DC-1..4, the
decisions that needed resolution before implementation
- TypeScript `@alkdev/operations` — `from_openapi`, `from_mcp`, `buildEnv`
prior art
- POC at `/workspace/@alkdev/dispatch` — head/worker dispatch over SSH+axum
## Amendments (2026-06-26)
This ADR left four decisions as two-way doors (§1 Consequences flagged DC-1's
security dimension; §5 noted trait signatures are two-way doors; Assumption 4
noted re-import hot-swap is a two-way door; §3 mentioned the namespace prefix).
The call-completion gap analysis (`docs/research/alknet-call-completion/gap-analysis.md`
DC-1..4) resolved them. The resolutions:
### DC-1 — CallClient registry scope: resolved by ADR-023, superseded by ADR-024
The §1 Consequences security dimension was originally resolved by ADR-023
(default-deny `remote_safe: bool` + `trusted_peer` opt-in). **ADR-023 is now
superseded by [ADR-024](024-peer-graph-routing-model.md)** (2026-06-27):
the flat-namespace single-peer model ADR-023 built on cannot express the
head→N-workers pattern, and the `remote_safe`/`trusted_peer` gate duplicates
the existing `AccessControl`/`Identity` machinery while reintroducing the
blanket-bypass anti-pattern ADR-017 killed. ADR-024 replaces the flat overlay
with peer-keyed overlays + `PeerRef` routing, and retires `remote_safe`/
`trusted_peer` in favor of `AccessControl::check(peer_identity)` — the
existing authorization path that was already in the dispatch path. The peer-
scoping question this section flagged is now answered structurally (peer-keyed
overlays), not by a parallel boolean gate.
### DC-4 — OperationAdapter trait error type: resolved
§5 showed `async fn import(&self) -> Vec<HandlerRegistration>` with no error
type. The trait returns `Result<Vec<HandlerRegistration>, AdapterError>`
where `AdapterError` is a crate-level enum. The *presence* of the error type
is recorded in [client-and-adapters.md](../crates/call/client-and-adapters.md);
the exact variants are the two-way-door remainder, tracked as OQ-26.
### DC-2 — from_call re-import on reconnection: manual free function
Assumption 4 noted re-import "happens on reconnection or is triggered
explicitly." The decision is **manual**: `from_call` is a free function; the
assembly layer calls it after `connect()`. The overlay is per-connection
(Layer 2, ADR-019), so re-import on reconnect is naturally scoped; a stale
overlay dies with the connection. A `CallConnection::refresh()` method for
mid-connection re-discovery is a genuine feature addition — non-breaking,
additive — if a deployment needs manual re-discovery without
drop-and-reconnect. Two-way door; recorded in
[client-and-adapters.md](../crates/call/client-and-adapters.md); tracked as
OQ-27. See [ADR-028](028-from-call-manual-free-function.md) for the full
rationale.
### DC-3 — from_call namespace collision: default set
§3's `FromCallConfig` namespace prefix is **optional, default no prefix,
collision = error**. A node importing from two remotes that both expose the
same unprefixed op name should fail loudly. The operator adds prefixes when
importing from multiple sources. Two-way door; recorded in
[client-and-adapters.md](../crates/call/client-and-adapters.md); tracked as
OQ-28.
### Operational spec
The gap this ADR left to implementation — the `CallClient` API, the
`from_call` flow, the trait signature, the adapter location map, the
no-env-vars invariant, and the exchange-of-operations pattern — is
specified in
[client-and-adapters.md](../crates/call/client-and-adapters.md). That document
is the operational complement to this ADR; this ADR remains the architectural
authority.
## Amendments (2026-07-09)
### `from_jsonschema` clause superseded by ADR-027
The §5 `FromJsonSchema` implementation listing ("schema-only, no handler")
is **superseded by [ADR-027](027-from-jsonschema-as-http-adapter.md)**.
`from_jsonschema` is now an HTTP-backed single-endpoint adapter in
`alknet-http` (reqwest forwarding handler, same shape as `from_openapi`),
not a schema-only placeholder in `alknet-call`. The `FromJsonSchema`
provenance variant stays in `alknet-call` (`OperationProvenance`) but is
now a handler-bearing leaf, not a "no handler" entry. The "schema-only,
no handler" concept is removed — schema validation without a handler is
served by consuming `OperationSpec` directly. The §5 "adapters live in
alknet-call" one-way-door statement is corrected above to "the adapter
trait lives in `alknet-call`; implementations live with their transport."
See [ADR-027](027-from-jsonschema-as-http-adapter.md) and
[client-and-adapters.md](../crates/call/client-and-adapters.md) §"from_jsonschema".
## Amendments (2026-07-13)
### `CallClient` transport-agnostic API (mirrors ADR-043's amendment)
The §1 Decision framed `CallClient::connect(addr: SocketAddr,
credentials)` as the primary constructor and described it as "opens a
QUIC connection." The operational spec
([client-and-adapters.md](../crates/call/client-and-adapters.md))
framed `spawn_dispatch(connection)` as the "lower-level API" that
`connect()` uses after the QUIC dial. That framing welded the
client-side one-way-door API to QUIC — the same welding ADR-007
unwound on the server side and ADR-043 corrected for `ChannelClient`.
The call protocol is transport-agnostic (ADR-015 EventEnvelope framing;
ADR-007 `Connection::from_stream`/`from_bidi` accept any
`AsyncRead + AsyncWrite`). The client side is half of that protocol
and must not be coupled to a transport. This amendment reframes the
existing code (which already has the right structure —
`spawn_dispatch` is not feature-gated, `connect` is
`#[cfg(feature = "quinn")]`):
- **`CallClient::spawn_dispatch(connection: Connection)`** — the
transport-agnostic primary constructor and the one-way-door API.
Takes a pre-established `Connection` (any transport), spawns the
shared dispatch loop, returns a live `CallConnection`. Mirrors the
server-side `CallAdapter::handle(Connection)` and
`ChannelClient::from_connection` (ADR-043).
- **`CallClient::connect(addr, credentials)`** — ~~a QUIC convenience
constructor~~ **REMOVED per ADR-045 §5 (2026-07-16)**. The dial is
centralized in `AlknetClient` (`alknet-client`); `connect` is
deleted, not retained as a two-way-door convenience, to avoid
`alknet-call` depending on `alknet-client` and to let `alknet-call`
shed its TLS/transport deps. Callers compose
`AlknetClient::dial_quic(...).await?` +
`CallClient::new(...).spawn_dispatch(conn)`.
The door-type classification is updated: `spawn_dispatch` is one-way
(the handler-facing surface); ~~`connect` is two-way (additive
convenience)~~ `connect` is **removed** (ADR-045 §5). The
`AlknetClient` extraction (OQ-55) is **resolved** by ADR-045 — the
shared dial seam is `alknet-client`; `spawn_dispatch` is the
protocol-crate take-over that consumes the dial's `Connection`.
See [client-and-adapters.md](../crates/call/client-and-adapters.md)
§"CallClient" for the reframed operational spec.

228
docs/architecture/decisions/023-callclient-peer-scoped-registry-filtering.md Normal file
View File

@@ -0,0 +1,228 @@
# ADR-023: Peer-Scoped Registry Filtering for CallClient Inbound Dispatch
## Status
**Superseded** by [ADR-024](024-peer-graph-routing-model.md) (2026-06-27).
ADR-023 introduced `remote_safe: bool` and `trusted_peer: bool` as a parallel
authorization system for peer-scoped dispatch. This was a structural miss: the
flat-namespace single-peer model it built on cannot express the head→N-workers
pattern (the primary use case), and the parallel `remote_safe`/`trusted_peer`
gate duplicates the existing `AccessControl`/`Identity` machinery (which
already authorizes peer calls) while reintroducing the blanket-bypass
anti-pattern ADR-017 was written to kill. ADR-024 replaces the flat overlay
with peer-keyed overlays + `PeerRef` routing, and retires `remote_safe`/
`trusted_peer` in favor of the existing `AccessControl::check(peer_identity)`.
See ADR-024 for the design that replaces this one; see
`docs/research/alknet-call-peer-routing/findings.md` for the research that
identified the gap.
## Context
ADR-022 §1 established that a `CallClient` — which opens an outbound
`alknet/call` connection — "has its own operation registry to dispatch incoming
calls from the remote side." The ADR left the *registry scope* as an explicit
two-way door in its Consequences:
> Sharing the global registry with a `CallClient` exposes local capabilities to
> the remote peer… A peer-scoped subset must filter by capability
> remote-safety, not just operation name. The registry-mechanism choice
> (share global vs subset vs separate) is two-way mechanically but has a
> security dimension post-ADR-018: the "share global" option is a
> capability-exposure decision, not just a dispatch decision.
This is the one decision identified in
`docs/research/alknet-call-completion/gap-analysis.md` (DC-1) that must be
locked before `CallClient` can be implemented correctly. It is a **one-way door
on the security dimension**: the choice of default determines what a remote peer
can reach, and a wrong default silently exposes outbound credentials.
### Why this is a one-way door, not a two-way door
The gap analysis framed the *mechanism* (share-global vs subset vs separate
registry instance) as a two-way door, and that framing holds. But the
**existence of peer-scoped filtering as the v1 default** is one-way, because:
1. Once a downstream consumer (the runner pattern, the container service, the
NAPI projection) is written against the "remote peer can call any
`External` op and the local node's capabilities will be populated for it"
semantics, switching the default to default-deny is a breaking change for
every consumer. The container-service rewrite at `/workspace/@alkdev/dispatch`
and the dev/runner patterns are the first consumers; the default is set
before they're written, so it's still cheap to set correctly — but only now.
2. The security dimension is asymmetric in ADR-032 terms. "Share global" leaks
silently: there is no error, no log line, no test that fails — the remote
peer simply receives a populated `OperationContext.capabilities` drawn from
the local `HandlerRegistration.capabilities`, and the local node's API keys
get used for the remote peer's call. The reversal cost is "discover which
consumers quietly depend on the leak and re-audit." Default-deny fails
loudly (the remote peer's call to an unexposed op returns `NOT_FOUND`),
which is the cheaper failure mode.
3. ADR-010's invariant — "no handler reads outbound credentials from any
source other than `OperationContext.capabilities`" — combined with
ADR-018's dispatch path (which populates `capabilities` from the
registration bundle) means the registration bundle *is* the exposure
boundary. Whatever the `CallClient` dispatches determines which
`Capabilities` objects cross to the remote peer's call context. Filtering
the registry is filtering capability exposure.
### The runner/dispatch pattern is the primary use case, and it is semi-trusted
The canonical consumer (gap analysis §"Exchange of Operations"): a container
service / dev runner connects *outward* to a hub and exposes `/container/exec`,
`/container/list`, etc. The hub then calls back into the runner. Both sides
are semi-trusted peers, not extensions of self. Exposing every `External`
operation on the runner — including any operation that carries an outbound
API key the runner happens to hold — is wrong by default. The operator who
*does* want full bilateral sharing is making an explicit trust decision.
## Decision
### 1. Default-deny: a CallClient exposes no operations to the remote peer unless explicitly marked remote-safe
The `CallClient` does not share the global `OperationRegistry` by default. It
holds a **peer-scoped subset**: a filtered view containing only
`HandlerRegistration`s that are explicitly marked as remote-safe for this peer.
The *existence* of filtering is the one-way door; this ADR locks it.
### 2. The remote-safe marking lives on the registration bundle, not on capabilities
The marking is added to `HandlerRegistration` (per ADR-018, the registration
bundle) as a peer-exposure field. It is not placed on `Capabilities` entries,
because:
- `Capabilities` is a flat credential bag; marking individual entries
remote-safe conflates "this credential is safe to send over the wire" with
"this operation may be dispatched on behalf of a remote peer." Those are
different questions — an operation may be remote-safe while using a
credential that must never leave the node, and the dispatch path already
keeps `Capabilities` off the wire (ADR-010). The exposure question is about
*which ops dispatch*, not *which credentials are serializable*.
- The registration bundle is already the integration point for provenance,
composition authority, scoped env, and visibility (ADR-018). Peer-exposure is
a property of the same shape: a dispatch-path concern set at registration.
The exact shape of the marking (a boolean, a per-peer allowlist, a
capability-class tag) is the two-way-door remainder — tracked as OQ-25, not
decided here. v1 uses the simplest shape that supports default-deny: a boolean
`remote_safe: bool` on `HandlerRegistration`, defaulting to `false`.
### 3. "Share the global registry" remains available as an explicit opt-in
A `CallClient` may be constructed in "trusted-peer" mode that exposes all
`External` operations from the global registry regardless of the remote-safe
marking. This is the explicit-allow path for operators who have made the trust
decision (e.g., two nodes under single administrative control, a test harness).
It is opt-in, never the default.
### 4. Provenance-based defaults
The remote-safe marking has a provenance-aware default at registration time,
before the operator's explicit choice:
| Provenance | Default `remote_safe` |
|-----------|----------------------|
| `Local` | `false` — assembly-written ops are not remote-callable unless the operator says so |
| `Session` | `false` — agent-written ops are sandboxed (ADR-017); exposing them to a remote peer would widen the sandbox |
| `FromOpenAPI`, `FromMCP`, `FromCall`, `FromJsonSchema` | `false` — leaves are composition material, not wire-callable (ADR-017) |
`false` across the board as the default. The operator flips specific
operations to `true` when they want this peer to reach them. This is the same
default-deny posture as ADR-017's visibility (`Internal` by default) and
ADR-018's composition authority (`None` for leaves by default).
### 5. The filtering is a dispatch-time read, not a copy
The `CallClient`'s peer-scoped view is not a second copy of the registry. It
is a dispatch-time read against the global registry, gated by the remote-safe
marking (and the trusted-peer flag). This keeps the curated layer (Layer 0,
ADR-019) single-source — the global registry is still the one Layer-0 store
built by the assembly layer at startup. Only the *visibility* to the remote
peer is filtered.
This avoids a third registry instance (the "separate registry per CallClient"
option from DC-1) and avoids the staleness problem a copied subset would
introduce: if the assembly layer reloads a curated op's spec, the peer-scoped
view reflects it on the next dispatch, not on the next copy.
## Consequences
**Positive:**
- The default is safe-by-construction for the runner/dispatch pattern. A
container service that connects outward to a hub cannot accidentally expose
its local vault-derived API keys to the hub's calls.
- The one-way security door is locked before any consumer is written against
the leaky default. The container-service rewrite and the dev/runner patterns
implement against default-deny from day one.
- Failure mode is loud: a remote peer calling an unexposed op gets
`NOT_FOUND`, not silent credential exposure.
- The mechanism is additive. Trusted-peer opt-in preserves the "share global"
path for operators who want it, without making it the default.
- Single-source Layer 0: no copied registry, no staleness.
**Negative:**
- Adds one field (`remote_safe: bool`) to `HandlerRegistration` (ADR-018).
The registration bundle grows. This is the smallest shape that supports
default-deny; OQ-25 may replace it with a richer mechanism (per-peer
allowlists, capability-class tags).
- Operators must explicitly mark operations remote-safe for bilateral
exchange. This is friction, deliberately: the bilateral container-service
pattern requires the operator to declare which of the runner's ops the hub
may call back into.
- The remote-safe marking is a v1 mechanism and may be superseded. OQ-25
tracks the shape; a future ADR may amend or supersede this one without
revisiting the *existence* of filtering.
- The trusted-peer opt-in is a sharp tool. An operator who enables it for the
wrong peer gets the "share global" exposure this ADR exists to prevent.
The opt-in is documented as a trust decision, not as a convenience.
## Assumptions
1. **The remote-safe marking is set at registration time, not at connection
time.** The marking is a property of the operation (per-peer in a richer
shape, but at least a boolean in v1), set by the assembly layer when it
builds the registry. Per-connection overrides are not part of v1; if a
deployment needs different exposure per peer, it uses the richer shape
(OQ-25) or multiple `CallClient`s with different filtered views.
2. **The peer-scoped view filters dispatch, not `services/list` semantics.**
The remote peer discovers operations via `services/list` (ADR-022 §3),
which already filters by `Visibility::External` (ADR-017). The remote-safe
marking is an *additional* filter for the dispatch path: an op may be
`External` yet not remote-safe. In v1, `services/list` served to a
`CallClient` peer **hides** non-remote-safe ops — a peer should not see
ops it cannot call, so discovery and dispatch filters agree. (The
pre-filter mental model — "`External` appears in `services/list`, then
the dispatch path returns `NOT_FOUND` for non-remote-safe" — is *not* the
v1 behavior; v1 hides them from listing too.) Whether a richer shape
(OQ-25) should expose-but-deny instead of hide is a two-way-door detail
tracked in OQ-25.
3. **Filtering is per-`CallClient`, not global.** A node with multiple
outbound connections may expose different subsets to different peers. The
v1 boolean marking limits this to "remote-safe for any peer" vs "not"; the
richer OQ-25 shape is what enables per-peer differentiation. v1's
limitation is acceptable because the runner/dispatch pattern has one
remote peer per `CallClient`.
## References
- ADR-032: One-Way Door Decision Framework (the door-type framing this ADR
relies on)
- ADR-010: Secret Material Flow and Capability Injection (the no-env-vars
invariant this ADR's security argument rests on)
- ADR-017: Privilege Model and Authority Context (the default-`Internal`,
default-deny posture this ADR mirrors)
- ADR-022: Call Protocol Client and Adapter Contract (§1 Consequences flagged
this decision; §1 is amended by this ADR)
- ADR-018: Handler Registration, Provenance, and Composition Authority (the
registration bundle this ADR adds a field to)
- ADR-019: Operation Registry Layering (Layer 0 single-source; the peer-scoped
view is a dispatch-time read, not a copy)
- OQ-25: Remote-safe marking shape (the two-way-door remainder)
- `docs/research/alknet-call-completion/gap-analysis.md` — DC-1
- `docs/architecture/crates/call/client-and-adapters.md` — the spec this ADR
informs

311
docs/architecture/decisions/024-peer-graph-routing-model.md Normal file
View File

@@ -0,0 +1,311 @@
# ADR-024: Peer-Graph Routing Model for alknet-call Composition
## Status
Accepted (supersedes ADR-023; Assumption 1's `PeerId` source is superseded
by ADR-025 on the source dimension — the one-way door is preserved)
## Context
The call protocol's composition model is **flat per overlay and single-peer**.
`CompositeOperationEnv` holds one `connection: Option<Arc<dyn OperationEnv>>`
overlay; the Layer 2 imported-ops overlay on `CallConnection` is a flat
`HashMap<String, HandlerRegistration>` keyed by operation name. This works for
one remote peer. The head→many-workers / hub→spoke pattern (the ray.io model,
and the primary downstream use case — the container-service rewrite this
completion was supposed to unblock) cannot be expressed:
1. **Overlay collision.** A head importing from worker A and worker B, both
exposing `/container/exec`, has no way to route
`invoke("container", "exec")` to the right peer. The composite env holds
one connection overlay; even with two, `contains("container/exec")` is
true for both with no disambiguation.
2. **`from_call` namespace prefix is a naming-convention hack.** DC-3 / OQ-28
made `FromCallConfig::namespace_prefix` the disambiguation mechanism — the
operator prefixes imported op names so two peers' ops don't collide in a
flat map. This pushes disambiguation to the caller and into the
`ScopedOperationEnv { allowed: HashSet<String> }` reachability list. It is
bolted onto a flat map instead of being structural routing.
3. **ADR-023's `remote_safe: bool` + `trusted_peer: bool` is a second,
parallel, weaker authorization system.** ADR-023 introduced a
`RemoteFilter { trusted_peer: bool }` gate in `protocol/dispatch.rs` that
runs *before* the existing `AccessControl::check`.
`trusted_peer: true` is a blanket security-bypass flag — the exact
anti-pattern ADR-017 was written to kill (it replaced `trusted: true` with
the authority-switch model). ADR-023 reintroduced it at the peer boundary.
The existing authorization machinery in core (`Identity` with scopes and
resources, `IdentityProvider`, `AccessControl::check`) is real, grounded,
and already wired into the dispatch path — ADR-023 should have *used* it for
peer authorization, not invented a parallel system.
This is a blocking structural fix, not a "v1/later" refinement. The research
at `docs/research/alknet-call-peer-routing/findings.md` validates the design
through a POC that type-checks against the real types (since removed; the
shapes are recorded in the research doc). ADR-023 is superseded by this ADR.
## Decision
### 1. Peer-keyed overlays
The Layer 2 overlay becomes peer-keyed at the composition-env level.
`CompositeOperationEnv`'s singular `connection: Option<Arc<dyn OperationEnv>>`
is replaced by `PeerCompositeEnv` with peer-keyed connections:
```rust
pub struct PeerCompositeEnv {
pub base: Arc<dyn OperationEnv + Send + Sync>, // Layer 0 curated
pub session: Option<Arc<dyn OperationEnv + Send + Sync>>, // Layer 1
pub connections: HashMap<PeerId, Arc<dyn OperationEnv + Send + Sync>>, // Layer 2, peer-keyed
connection_order: Vec<PeerId>, // insertion order for PeerRef::Any first-match
}
```
The per-`CallConnection` overlay stays flat (one connection = one peer — a
flat `HashMap<String, HandlerRegistration>` per connection is correct). The
peer-keying is at the *aggregation* layer: the head node's composition env
holds a `HashMap<PeerId, connection_overlay>`, not one overlay. `PeerId` is
the peer's `Identity.id` — the same field `Connection::identity()` already
exposes, already resolved in the dispatch path, and already unique per peer.
### 2. `PeerRef` routing selector
`OperationEnv` gains a peer-routing method with a `PeerRef` selector. The
default-impl preserves back-compat (existing impls that don't override it
delegate to `invoke_with_policy`, preserving current behavior):
```rust
pub enum PeerRef {
Specific(PeerId), // route to this peer; NOT_FOUND if it doesn't serve the op
Any, // first peer (insertion order) that serves it
}
pub type PeerId = String; // logical id, NOT Identity.id — see OQ-33
async fn invoke_peer(&self, peer: &PeerRef, namespace: &str, operation: &str,
input: Value, parent: &OperationContext, policy: AbortPolicy) -> ResponseEnvelope {
// default: ignore peer selector, dispatch via invoke_with_policy
self.invoke_with_policy(namespace, operation, input, parent, policy).await
}
fn peer_contains(&self, _peer: &PeerId, name: &str) -> bool { self.contains(name) }
```
`PeerRef::Specific(PeerId)` routes to the named peer's overlay; if that peer
doesn't serve the op, `NOT_FOUND` (no silent fallthrough — explicit routing
must be honored or fail loudly). `PeerRef::Any` routes to the first peer
(insertion order) whose overlay contains the op — the "any worker that serves
this name" fan-out primitive. A richer `RoutingPolicy` (round-robin,
least-loaded) is the two-way-door remainder tracked as OQ-30; the `PeerRef`
enum is designed to compose with it without breaking the signature.
The existing `invoke()` / `invoke_with_policy()` methods stay as the
`PeerRef::Any` equivalent for code that doesn't care about peer selection.
### 3. `AccessControl`-based peer authorization; retire `remote_safe`/`trusted_peer`
`RemoteFilter`, `HandlerRegistration::remote_safe`,
`CallClient::trusted_peer`, `OperationRegistry::list_operations_peer_scoped`,
and `services_list_handler_peer_scoped` are **removed**. Peer authorization
flows through the existing `AccessControl::check` against the peer's resolved
`Identity`:
- A remote peer's call arrives → `dispatch_requested` resolves the peer's
`Identity` (already does, from the connection's TLS fingerprint or the
`auth_token` payload) → `OperationRegistry::invoke` runs
`AccessControl::check(peer_identity)`.
- If the op's `AccessControl` is satisfied → dispatch (capabilities populated
from the bundle, same as today).
- If not → `FORBIDDEN` (capabilities never populated — the security property
ADR-023 wanted, achieved by the existing ACL, not a parallel gate).
- If the op is `Visibility::Internal` → `NOT_FOUND` before ACL (existing
behavior). This is the "never callable from wire" case.
The three cases `remote_safe` was meant to handle map to existing mechanisms:
| `remote_safe` case | Replacement |
|---|---|
| Op callable by any peer (was `remote_safe: true`) | `AccessControl::default()` — no restrictions; implicitly "remote-safe" because it requires no privileged scope. |
| Op callable only by some peers | `AccessControl { required_scopes: [...] }` — only peers whose `Identity.scopes` satisfy the AND-gate may call. Per-peer differentiation via `IdentityProvider` config. |
| Op never callable from wire | `Visibility::Internal` — `NOT_FOUND` before ACL. Existing mechanism, unchanged. |
**The op's `AccessControl` *is* the peer-authorization policy.** There is no
separate exposure decision. If the peer's `Identity` satisfies the op's
`AccessControl`, the op dispatches and capabilities populate (same as for any
authorized caller). If not, `FORBIDDEN` before the handler — capabilities
never populate. The exposure decision and the authorization decision are the
same decision, made through one mechanism, not two.
### 4. Peer-qualified reachability (`ScopedPeerEnv`)
`ScopedOperationEnv { allowed: HashSet<String> }` is extended with an optional
peer-pinned allowlist. Unqualified reachability (peer-agnostic composition —
"I want to call `container/exec` on whichever worker serves it") stays the
common case; peer-pinning is opt-in for the disambiguation case that replaces
`FromCallConfig::namespace_prefix`:
```rust
pub struct ScopedPeerEnv {
pub allowed_ops: HashSet<String>, // peer-agnostic — reachable via PeerRef::Any
pub peer_pinned: HashSet<String>, // "peer-id/op-name" — reachable only via PeerRef::Specific(that peer)
}
```
Instead of prefixing the *op name* (the flat-namespace hack), you pin the
*peer* in the reachability set. The existing `ScopedOperationEnv.allowed`
becomes the `allowed_ops` field; peer-pinning is additive.
### 5. `from_call` peer-keyed registration; collision rule change
`from_call` registers into the specific peer's sub-overlay, not a flat
overlay. Cross-peer collision dissolves: same name on different peers is fine
(separate sub-overlays, no collision, no prefix needed). Same-peer collision
stays an error (a peer shouldn't expose two ops with the same name).
`FromCallConfig::namespace_prefix` becomes optional local-naming sugar for
the case where the importing node wants to expose a peer's ops under a
different name *locally* — a local-naming concern, not a disambiguation
concern. It defaults to `None`.
### 6. `services/list` `AccessControl`-filtered; `services/list-peers` opt-in
`services/list` filters by `AccessControl::check(calling_peer_identity)` — the
calling peer sees only ops it is authorized to call. The
`services_list_handler` / `services_list_handler_peer_scoped` split collapses
to a single `AccessControl`-filtered handler. `services/list-peers` is the
opt-in for peer-attributed re-export listing (each peer's sub-overlay listed
with attribution, filtered by the calling peer's authorization).
## Consequences
**Positive:**
- The head→N-workers pattern works. A head with multiple worker connections
routes `invoke()` to the right peer via `PeerRef`. This is the primary use
case the previous model couldn't express.
- One authorization system, not two. Peer authorization flows through the
existing `AccessControl`/`Identity` machinery — the same mechanism that
gates every other call. No parallel `remote_safe` gate, no blanket-bypass
`trusted_peer` flag. Per-peer differentiation is via `IdentityProvider`
config (different peers get different scopes), which is a real
authorization decision, not a boolean.
- Structural disconnect cleanup. When a peer disconnects, its sub-overlay
drops (the `PeerId` key is removed from `connections`). No stale overlay,
no explicit deregistration. An in-flight `PeerRef::Specific(that_peer)` gets
`NOT_FOUND` — the correct failure mode.
- `from_call` collision dissolves across peers. Two workers exposing
`/container/exec` coexist; the prefix is no longer the disambiguation
mechanism.
- The `OperationEnv` trait gains a method with a default-impl, preserving
back-compat. Existing impls (`LocalOperationEnv`, `OverlayOperationEnv`)
work unchanged; `PeerCompositeEnv` overrides with real peer routing.
- The peer-keyed overlay model extends naturally to multi-hop federation (a
chain of `PeerRef::Specific` routing decisions) without redesign. Petgraph
is not needed for v1 (one-hop, shallow); it pays off if multi-hop
path-finding becomes real (OQ-32).
**Negative:**
- `CompositeOperationEnv` → `PeerCompositeEnv` is a migration. Existing call
sites that construct `CompositeOperationEnv::new(base, Some(conn), session)`
migrate to `PeerCompositeEnv::new(base).with_session(session).attach_peer(peer_id, conn)`.
The singular-connection case (one peer) is the degenerate case
(`connections` with one entry).
- `OperationEnv` trait gains a method. The default-impl preserves back-compat,
but it's a trait surface change; downstream impls (`alknet-http`,
`alknet-agent`) gain the method with the default delegation.
- `services/list` semantics change: the filter is `AccessControl`-based, not
`remote_safe`-based. An op with `AccessControl::default()` (no restrictions)
is now listed to any peer — this is correct (it's implicitly callable by
any authenticated peer), but operators who relied on `remote_safe: false` to
hide ops from peers must instead set `required_scopes` or `Visibility::Internal`.
- ADR-023 is superseded. The `remote_safe` field, `trusted_peer` flag,
`RemoteFilter`, `list_operations_peer_scoped`, and
`services_list_handler_peer_scoped` are removed. Code that references them
(the `CallClient`, `Dispatcher`, `HandlerRegistration`, `discovery.rs`)
changes. This is the cost of fixing a one-way-door miss — the previous model
shipped and was reviewed before the structural gap was caught.
- `PeerId` is a logical identifier, **not** `Identity.id` (the fingerprint or
API-key prefix). Coupling `PeerId` to the crypto material would break every
in-flight `PeerRef::Specific` and every ACL entry referencing that peer on
key rotation. v1 uses a connection-assigned UUID; a configured node name is
the future shape. See OQ-33 for the full decision and the key-rotation/ACL
rationale.
## Assumptions
1. **`PeerId` is a logical identifier, not `Identity.id`.** v1 source is a
connection-assigned UUID (v4) — stable for the connection's lifetime,
changes on reconnect. This is a no-storage workaround: the core crates are
deliberately DB-free (smaller, fewer deps), which works for local-only
state but not for cross-node peer identity that wants to persist across
restarts and key rotations. An in-flight `PeerRef::Specific(stale_uuid)`
gets `NOT_FOUND` on reconnect — the correct failure mode (the peer is
gone); re-`from_call` produces a fresh `PeerRef`. The real solution (a
persistent peer registry that maps a stable logical name to current crypto
material, surviving key rotation) is tracked as OQ-34, not a v1 blocker.
The one-way door: `PeerId` is logical, not crypto — this determines the
`PeerCompositeEnv` key type and `PeerRef::Specific` payload. See OQ-33.
> **Superseded by ADR-025 on the `PeerId` source dimension.** The
> one-way door (`PeerId` is logical, not crypto) is preserved. The v1
> UUID source is replaced by `Identity.id` from `PeerEntry.peer_id`
> (stable across key rotation). The "no-storage workaround" framing is
> no longer accurate — the storage boundary is now `config + in-memory
> adapter` (ADR-025 + ADR-033), with persistence adapters additive. See
> ADR-025 and OQ-33 (resolved).
2. **`PeerRef::Any` = insertion-order first-match.** Deterministic but
order-dependent (worker A connects before worker B → `Any` routes to A
until A disconnects). This is the simplest routing policy and is correct for
the immediate use case (the head picks the first worker that serves the
op). A richer `RoutingPolicy` (round-robin, least-loaded, affinity) is OQ-30;
the `PeerRef` enum composes with it without breaking the signature.
3. **`services/list` defaults to "own ops only" (unchanged from today).**
Re-exported peer ops are not listed unless the calling peer invokes
`services/list-peers` (the opt-in). The re-export policy (which peers' ops a
given peer sees) is an `AccessControl` decision on the listing op.
4. **Capability exposure under `PeerRef::Any`.** When a handler composes via
`Any` and routing picks worker A, the handler's `Capabilities` propagate to
worker A's call (same as today's `from_call` forwarding). This is correct:
the handler declared the op in its scoped env, so it authorized the
composition; the peer selection is a routing detail. If a handler needs
per-peer capability scoping, it uses `PeerRef::Specific` and peer-pinned
reachability.
5. **Multi-hop federation is out of scope for v1.** Worker A does not
transitively see worker B's ops through the head unless the head explicitly
re-exports them. The peer-keyed overlay model extends to multi-hop without
redesign (a chain of `PeerRef::Specific` decisions), but path-finding
(which peer reaches which op transitively) is where petgraph would pay off
(OQ-32, not designed).
## References
- ADR-017: Privilege Model and Authority Context (the authority-switch pattern
ADR-023 violated by reintroducing a blanket-bypass flag)
- ADR-022: Call Protocol Client and Adapter Contract (amended: `CallClient`
no longer has `trusted_peer`; the client/adapter spec updates)
- ADR-018: Handler Registration, Provenance, and Composition Authority
(`remote_safe` field removed from the registration bundle)
- ADR-019: Operation Registry Layering (Layer 2 becomes peer-keyed at the
composition-env aggregation level)
- ADR-023: Peer-Scoped Registry Filtering for CallClient Inbound Dispatch
(superseded)
- OQ-25: dissolved (no `remote_safe` marking — `AccessControl` is the policy)
- OQ-26: resolved (`AdapterError` variants — `SamePeerCollision` replaces
the flat `Conflict` variant; `#[non_exhaustive]`)
- OQ-27: stays (re-import trigger — unchanged; the overlay is now peer-scoped)
- OQ-28: dissolved cross-peer (same name on different peers is fine); stays
same-peer
- OQ-29: stays (TLS client-auth — orthogonal to the routing model)
- OQ-30: `PeerRef::Any` routing policy (new — round-robin/least-loaded)
- OQ-31: `services/list-peers` re-export semantics (new)
- OQ-32: Multi-hop federation (new — petgraph candidate)
- OQ-33: resolved — `PeerId` is a logical id (UUID v1), not `Identity.id`;
decoupling from crypto material keeps the door open for key-rotation-safe ACLs
- OQ-34: persistent peer registry (new — the storage dimension OQ-33 surfaced;
not a v1 blocker, tracked so the no-DB posture's limit is deliberate)
- Research: `docs/research/alknet-call-peer-routing/findings.md`
- Prior art: Ray.io actors (`ActorHandle` = `PeerRef::Specific`), Dapr service
invocation (app-ID routing = `PeerRef::Specific`, access-control allowlist =
`AccessControl`-based peer authorization)

435
docs/architecture/decisions/025-peerentry-and-identity-id-decoupling.md Normal file
View File

@@ -0,0 +1,435 @@
# ADR-025: PeerEntry and Identity.id Decoupling
## Status
Accepted (supersedes the "v1 UUID" source in ADR-024 Assumption 1; resolves
the "real solution" half of OQ-33 and the storage-boundary half of OQ-34)
## Context
`Identity.id` is the string that keys authorization decisions across the
alknet crate graph. Today it is **coupled to the cryptographic material**:
```rust
// crates/alknet-core/src/config.rs — current implementation
pub struct AuthPolicy {
pub authorized_fingerprints: HashSet<String>, // just strings, no stable id
pub api_keys: Vec<ApiKeyEntry>,
}
impl AuthPolicy {
pub fn resolve_identity_from_fingerprint(&self, fingerprint: &str) -> Option<Identity> {
if self.authorized_fingerprints.contains(fingerprint) {
Some(Identity {
id: fingerprint.to_string(), // ← identity IS the crypto material
scopes: vec!["relay:connect".to_string()],
...
})
}
}
}
```
This coupling is a latent bug for any cross-node authorization decision:
- A TLS fingerprint or raw-key identity changes when the node rotates its key.
- When it changes, every ACL entry that references the old fingerprint stops
matching — the peer "disappears" from the authorization system even though
it is the same logical node.
- `PeerRef::Specific(PeerId)` (ADR-024) routes by `Identity.id`; a key
rotation would break in-flight routing references the same way.
- The hub's `authorized_fingerprints` set has to be manually updated on every
rotation on the *remote* side, which is exactly the operational pain the
vault's local key rotation (ADR-021) was meant to remove.
ADR-024 §1 set `PeerId = Identity.id` and made `PeerId` a logical identifier
"NOT `Identity.id` (the fingerprint)" — but left the *source* of that logical
identifier as a connection-assigned UUID (OQ-33's v1 workaround). The UUID
is ephemeral: it survives only for the connection's lifetime, changes on
reconnect, and cannot persist across restarts or key rotations. It is a
no-storage workaround, not a real identity.
The research at `docs/research/alknet-storage-strategy/findings.md` §4
established the real fix: introduce a `PeerEntry` config model that maps a
**stable logical peer id** to its current cryptographic material and
authorization scopes, and have `ConfigIdentityProvider` resolve
fingerprint → `PeerEntry` → `Identity { id: peer_entry.peer_id, scopes:
peer_entry.scopes, ... }`. The `Identity.id` becomes the stable `peer_id`,
decoupled from the fingerprint. Key rotation is a single field update in the
peer entry; the `peer_id` and every ACL / routing reference to it stay
stable.
This is the storage-boundary question OQ-34 tracks. With ADR-033 (the
repo/adapter pattern) establishing that core defines repo traits and the
default in-memory adapter lives alongside the trait, the answer is: core
gets the `PeerEntry` config model and the
`ConfigIdentityProvider::resolve_from_fingerprint → Identity { id: peer_id
}` resolution path now, with no SQLite dependency in core. A future
`alknet-peer-store-sqlite` adapter that persists `PeerEntry` records is
additive — it implements the same `IdentityProvider` trait against a `peers`
table instead of config. The trait is the one-way door; the adapter is the
two-way door.
## Decision
### 1. Add `PeerEntry` to `AuthPolicy`, replacing `authorized_fingerprints`
```rust
pub struct PeerEntry {
/// Stable logical peer id ("worker-a", "alice"). Does NOT change on
/// key rotation. This becomes Identity.id on resolution, regardless of
/// which credential path resolved the identity.
pub peer_id: String,
/// TLS fingerprints for this peer — one or more. A peer may have
/// multiple keys (e.g., an Ed25519 raw key for P2P and an X.509 cert
/// for domain-facing). Resolution matches against any entry.
/// Format: "ed25519:<hex of 32-byte pub key>" for RFC 7250 raw keys
/// (normalized across quinn and iroh — see §6), "SHA256:<hex>" for
/// X.509 certs (DER hash). Changes on key rotation.
pub fingerprints: Vec<String>,
/// Optional: bearer-token authentication for this peer. A peer that
/// also authenticates via auth_token (e.g., HTTP clients that can't
/// do TLS client-auth) stores the SHA-256 hash of the token here.
/// Resolution via resolve_from_token matches this field and returns
/// the same Identity { id: peer_id, ... } as the fingerprint path.
pub auth_token_hash: Option<String>,
/// Authorization scopes granted to this peer. Resolved into
/// Identity.scopes.
pub scopes: Vec<String>,
/// Named resource lists granted to this peer. Resolved into
/// Identity.resources.
pub resources: HashMap<String, Vec<String>>,
/// Human-readable display name for logs / UIs. Optional.
pub display_name: Option<String>,
/// Whether this peer is authorized at all. false = recognized but
/// disabled (revoked). Resolution returns None.
pub enabled: bool,
}
pub struct AuthPolicy {
/// Replaces authorized_fingerprints: HashSet<String>. Each entry maps
/// a stable logical peer_id to its credential paths (fingerprints,
/// optional auth_token_hash) + scopes + resources. The list is keyed
/// by peer_id; resolution looks up by fingerprint OR auth_token.
pub peers: Vec<PeerEntry>,
/// API keys for bearer-token auth where the token IS the identity
/// (rotation = new identity). Peers that need a stable logical id
/// across credential rotation use PeerEntry.auth_token_hash instead.
/// See "Bearer tokens" below.
pub api_keys: Vec<ApiKeyEntry>,
}
```
### 2. `Identity.id` becomes `PeerEntry.peer_id` on resolution (any credential path)
`ConfigIdentityProvider::resolve_from_fingerprint` resolves fingerprint →
matching `PeerEntry` (by any entry in `fingerprints`) → `Identity { id:
peer_entry.peer_id, ... }`. `ConfigIdentityProvider::resolve_from_token`
resolves token → matching `PeerEntry` (by `auth_token_hash`) → the same
`Identity { id: peer_entry.peer_id, ... }`. Both paths produce the same
`Identity` — the `peer_id` is the stable logical id regardless of how the
peer authenticated.
```rust
impl AuthPolicy {
pub fn resolve_identity_from_fingerprint(&self, fingerprint: &str) -> Option<Identity> {
self.peers.iter()
.find(|p| p.enabled && p.fingerprints.iter().any(|f| f == fingerprint))
.map(|p| Identity {
id: p.peer_id.clone(),
scopes: p.scopes.clone(),
resources: p.resources.clone(),
})
}
pub fn resolve_identity_from_token(&self, token: &str) -> Option<Identity> {
let token_hash = sha256(token);
self.peers.iter()
.find(|p| p.enabled && p.auth_token_hash.as_deref() == Some(&token_hash))
.map(|p| Identity {
id: p.peer_id.clone(),
scopes: p.scopes.clone(),
resources: p.resources.clone(),
})
.or_else(|| self.resolve_api_key(token)) // fall through to ApiKeyEntry
}
}
```
If the token doesn't match any `PeerEntry.auth_token_hash`, resolution falls
through to `resolve_api_key` (the `ApiKeyEntry` path, where `Identity.id =
prefix`). This preserves the existing API-key path for bearer tokens that
ARE the identity, while adding the `PeerEntry` token path for tokens that
are one credential path among several for a stable logical peer.
This removes the pre-ADR-025 limitation in `auth.md`
§"Resource-scoped ACLs and external identities" — resolved identities now
carry `resources` from the `PeerEntry`, not just from the composition path.
### 3. Key rotation is a `PeerEntry` field update (no `peer_id` change)
Rotating a peer's TLS key:
- The vault derives the new key locally (ADR-020/021).
- The remote side's config updates the `PeerEntry.fingerprints` entry for
that `peer_id`. The `peer_id`, `scopes`, `resources`, ACL entries, and
any `PeerRef::Specific(peer_id)` references stay stable.
- A config reload (`ConfigReloadHandle::reload`) makes the change live.
Rotating a peer's auth token:
- Update `PeerEntry.auth_token_hash` for that `peer_id`. The `peer_id`
and everything that references it stays stable.
No ACL update, no routing reference invalidation, no peer "disappears."
The vault's local rotation + a remote-side config edit is the full key
rotation story across nodes, for any credential path.
### 4. `PeerId` source changes from UUID to `Identity.id` from `PeerEntry`
ADR-024 Assumption 1 said `PeerId` is a connection-assigned UUID (v4). With
`Identity.id` now stable (`peer_id`), the UUID workaround is no longer
needed: `PeerId = Identity.id` from `IdentityProvider` resolution. This is
the one-way-door tightening — `PeerId` was always specified as logical-not-
crypto (ADR-024), the UUID was the *source*; the source now becomes the
auth system.
```rust
// ADR-024 §1, updated by this ADR:
pub type PeerId = String; // = Identity.id from IdentityProvider resolution
// = PeerEntry.peer_id (stable, not crypto material)
```
ADR-024 §2's `invoke_peer` / `PeerRef::Specific(PeerId)` signatures are
unchanged. The `PeerId` payload is now stable across reconnects and key
rotations, instead of ephemeral. An in-flight `PeerRef::Specific` that
survives a reconnect now keeps resolving (the `peer_id` is unchanged), which
is the property the UUID workaround could not provide.
### 5. The `PeerId` for a connection comes from `IdentityProvider` resolution
The dispatch path that builds a `CallConnection` and assigns a `PeerId` to
the peer-keyed overlay (`PeerCompositeEnv::attach_peer`) reads
`connection.identity().id` — the resolved `Identity.id` from the
`IdentityProvider`. If identity resolution returns `None` (no client cert,
unrecognized fingerprint), the peer has no `PeerId` and the connection
cannot be added to the peer-keyed overlay. The handler either rejects the
connection or falls back to a connection-without-peer-identity path (the
caller-id-is-the-connection case, e.g., anonymous dial-in).
The UUID fallback is removed. A connection with no resolved identity has no
`PeerId`, not a random one.
### 6. Fingerprint format normalization: `ed25519:` for raw keys
Ed25519 raw keys (RFC 7250) produce different fingerprint formats depending
on the transport:
- **iroh** (direct or relay): `ed25519:<hex of 32-byte public key>` —
extracted from `connection.remote_node_id()`, which returns the NodeId
(the raw Ed25519 public key). Already implemented.
- **quinn RawKey**: currently `SHA256:<hex of cert DER>` — because
`fingerprint_from_cert_der` hashes the SPKI DER bytes. The DER encoding
of the SPKI is not the raw 32-byte public key; it's an ASN.1 wrapper.
So the same Ed25519 key produces `ed25519:abc...` on iroh and
`SHA256:def...` on quinn — two different fingerprints for the same key.
This is normalized: the quinn path extracts the Ed25519 public key from the
cert DER (the `RawKeyCertResolver` already has the raw key bytes via
`Ed25519SecretKey::public()`) and formats it as `ed25519:<hex>`, matching
iroh. A peer that connects via quinn direct and via iroh has the same
fingerprint in `PeerEntry.fingerprints` — one entry, both transports.
The normalization is in `extract_quinn_client_fingerprint`: when the
presented cert is an RFC 7250 raw public key (SPKI with Ed25519 algorithm
identifier), extract the raw 32-byte public key and format as
`ed25519:<hex>`. When the cert is X.509, keep the `SHA256:<hex of DER>`
format (X.509 certs don't have a "raw public key" form — the DER hash is
the fingerprint).
This also simplifies the coming WebTransport relay work: a WebTransport
relay acts as a proxy, and the proxied connection's Ed25519 identity
should be the same `ed25519:<hex>` whether the client connected directly
or through the relay. Normalizing on the iroh pattern means the relay
doesn't need a separate fingerprint format.
## Bearer tokens
There are three credential types in the alknet auth model:
1. **Ed25519 raw key** (RFC 7250) — the most common. Same key type as SSH
keys, native to iroh's `NodeId`. Fingerprint format: `ed25519:<hex>`.
Used for direct quinn, iroh direct, and iroh relay connections. The
fingerprint IS the trust anchor (no CA needed).
2. **X.509 cert** — for domain-facing endpoints (`api.alk.dev`, relays,
ACME/Let's Encrypt). Fingerprint format: `SHA256:<hex of DER>`. Requires
CA verification on the client side. The outgoing-only case (a client
connects to a public X.509 endpoint) is tracked as OQ-37.
3. **Bearer token** (auth_token) — for HTTP clients that can't do TLS
client-auth (browsers, curl), or as a secondary credential path. Carried
in the call-protocol `auth_token` payload field.
A `PeerEntry` can have any combination of these: `fingerprints: Vec<String>`
for one or more TLS keys (Ed25519 and/or X.509), `auth_token_hash:
Option<String>` for an optional bearer-token path. All resolve to the same
`peer_id`. A peer that authenticates via Ed25519 today and via auth_token
tomorrow gets the same `PeerId` — the logical identity is stable across
credential paths.
`ApiKeyEntry` stays as a separate path for bearer tokens where the token IS
the identity (rotation = new identity, no stable logical id needed). When a
bearer token is one credential path among several for a stable peer, it
goes in `PeerEntry.auth_token_hash`. The distinction is not "peer bearer vs
auth bearer" — it's whether the token needs a stable logical id across
rotation (`PeerEntry`) or not (`ApiKeyEntry`).
| Credential type | `PeerEntry` field | `Identity.id` | Rotation |
|-----------------|-------------------|---------------|----------|
| Ed25519 raw key | `fingerprints[i]` (`ed25519:...`) | `peer_id` (stable) | Update `fingerprints` entry |
| X.509 cert | `fingerprints[i]` (`SHA256:...`) | `peer_id` (stable) | Update `fingerprints` entry |
| Bearer token (peer) | `auth_token_hash` | `peer_id` (stable) | Update `auth_token_hash` |
| Bearer token (identity) | `ApiKeyEntry` (separate) | `prefix` (changes with key) | New `ApiKeyEntry` |
## What this does NOT change
- **`Identity` struct shape** — `id: String`, `scopes: Vec<String>`,
`resources: HashMap<String, Vec<String>>` are unchanged. Only the
*meaning* of `id` on the fingerprint path changes (fingerprint →
peer_id).
- **`IdentityProvider` trait** — unchanged. The adapter's resolution
semantics change, not the trait.
- **`AccessControl::check`** — unchanged. Still a flat scope/resource match
against `Identity`. The `Identity` it checks now has a stable `id` on the
fingerprint path, but `check` doesn't key on `id` (it checks scopes and
resources).
- **`AuthToken`, `AuthContext`** — unchanged.
- **`PeerRef::Specific(PeerId)` signature** — unchanged. The payload is now
stable.
- **`CompositeOperationEnv` → `PeerCompositeEnv` migration** — unchanged.
This ADR provides the stable `PeerId` source; ADR-024 still owns the
overlay-keying model.
## Consequences
**Positive:**
- Key rotation no longer breaks ACL entries or routing references on the
remote side — for any credential path (TLS key or auth token). The
vault's local rotation story (ADR-021) is now the complete story.
- `PeerRef::Specific` survives reconnects. An in-flight routing reference
to "worker-a" keeps resolving after worker-a's TLS key rotates and after
worker-a reconnects.
- OQ-33's UUID workaround is removed — the stable logical id is the real
thing, not an ephemeral stand-in.
- OQ-34's storage-boundary question is resolved: core has the config model
(`PeerEntry`) + the in-memory adapter (`ConfigIdentityProvider`); a
future `alknet-peer-store-sqlite` adapter that persists `PeerEntry`
records is additive, implementing the same `IdentityProvider` trait
against a `peers` table. See ADR-033.
- Resolved identities now carry `resources` (the pre-ADR-025 limitation is
lifted) — `AccessControl::check` against `resource_type`/
`resource_action` works for external authenticated callers when
configured, regardless of credential path.
- A peer can authenticate via Ed25519 today and via auth_token tomorrow,
getting the same `PeerId` — the logical identity is stable across
credential paths.
- Fingerprint normalization (`ed25519:<hex>` for raw keys across quinn and
iroh) means the same key has the same fingerprint regardless of transport.
This also simplifies the coming WebTransport relay work.
**Negative:**
- `AuthPolicy.authorized_fingerprints: HashSet<String>` is replaced with
`AuthPolicy.peers: Vec<PeerEntry>`. This is a breaking config change —
existing config files with `authorized_fingerprints` migrate to `peers`
entries. The migration is mechanical (each fingerprint becomes a
`PeerEntry { peer_id: <chosen name>, fingerprints: vec![<old value>], ... }`),
and operators must choose a `peer_id` per peer, but it is a config break.
- `Identity.id` for resolved identities changes from the fingerprint to
the `peer_id`. Code that logs or compares `Identity.id` and assumed it
was the fingerprint string will see the `peer_id` instead. This is the
correct behavior (logs should show the logical name, not the rotating
crypto material), but it's a behavior change in log output.
- The quinn fingerprint extraction changes from `SHA256:<hex of DER>` to
`ed25519:<hex of raw key>` for raw-key certs. Existing configs with
`SHA256:` fingerprints for Ed25519 keys migrate to `ed25519:` format.
X.509 fingerprints stay as `SHA256:<hex of DER>`.
- ADR-024 Assumption 1 is superseded on the `PeerId` source dimension:
the one-way door (`PeerId` is logical, not crypto) is preserved, but the
UUID source is replaced by `Identity.id` from `PeerEntry`. The
Assumption's framing of "no-storage workaround" is no longer accurate —
the storage boundary is now explicitly `config + in-memory adapter`
(this ADR + ADR-033), with the SQLite adapter additive.
## Assumptions
1. **The dispatch path can require identity resolution for peer-keyed
overlay membership.** A connection that fails `IdentityProvider`
resolution has no `PeerId` and is not added to `PeerCompositeEnv`. The
caller either authenticates with a recognized fingerprint (and gets a
`peer_id`) or is rejected / falls back to a no-peer-identity path. The
v1 UUID fallback is removed deliberately — anonymous dial-in to a
peer-keyed composition env is a contradiction.
2. **`PeerEntry.peer_id` is operator-chosen and unique within a config.**
Config validation enforces uniqueness; duplicate `peer_id` values in
`AuthPolicy.peers` are a config error.
3. **Bearer tokens have two paths.** `PeerEntry.auth_token_hash` is for
tokens that are one credential path among several for a stable logical
peer (the token rotates, the `peer_id` stays). `ApiKeyEntry` is for
tokens that ARE the identity (rotation = new identity, no stable
logical id needed). See "Bearer tokens" above. The distinction is not
"peer bearer vs auth bearer" — it's whether the token needs a stable
logical id across rotation.
4. **The `peers` list resolution is O(peers) per fingerprint lookup.** The
expected peer count per node is small (10s–100s); a linear scan with a
side index is fine. A `HashMap<fingerprint, &PeerEntry>` index is an
implementation-detail two-way door.
5. **Adapter crates that persist `PeerEntry` records are additive and not
specified here.** ADR-033 establishes the pattern (core trait + in-memory
default; persistence adapters are separate crates); the concrete adapter
shapes are deferred for exploration per the user's note. This ADR's
commitment is to the `PeerEntry` config model + the resolution
semantics + the `PeerId` source, not to any specific backend.
## References
- ADR-003: Auth as Shared Core (`IdentityProvider` in core)
- ADR-017: Privilege Model and Authority Context (`AccessControl::check`
against `Identity`)
- ADR-021: Key Rotation via Version-Indexed Paths (the local rotation half
this ADR completes across nodes)
- ADR-018: Handler Registration, Provenance, and Composition Authority
(the registration bundle's `composition_authority` path produces its own
`Identity`; this ADR's `PeerEntry.resources` populates the external-auth
path's `Identity.resources`)
- ADR-024: Peer-Graph Routing Model (the `PeerId = Identity.id` model;
Assumption 1's UUID source is superseded by this ADR's `PeerEntry.peer_id`
source — the one-way door is preserved)
- ADR-033: Storage Boundary and Repo/Adapter Pattern (the overarching pattern
this ADR's `PeerEntry` + `ConfigIdentityProvider` follows)
- OQ-33: PeerId — Cryptographic Identity vs Stable Logical Identifier
(resolved by this ADR — the "real solution" half, replacing the UUID
workaround)
- OQ-34: Persistent Peer Registry (resolved by this ADR + ADR-033 — the
storage boundary is `config + in-memory adapter` now, SQLite adapter
additive)
- ~~OQ-35: API Key Identity vs Peer Identity~~ (dissolved — the
"asymmetry" framing was wrong; `PeerEntry` supports multiple credential
paths, and `ApiKeyEntry` is for tokens that ARE the identity)
- OQ-29: CallClient TLS Client-Auth (resolved by this ADR's §6 fingerprint
normalization + the client-auth wiring decision recorded in OQ-29)
- OQ-37: X.509 outgoing-only case (the three auth types and how X.509
server identity fits — see OQ-37 in open-questions.md)
- `docs/research/alknet-storage-strategy/findings.md` §4 (the `PeerEntry`
model and resolution path)
- `docs/architecture/crates/core/auth.md` (the spec amended by this ADR)
- `docs/architecture/crates/core/config.md` (the `AuthPolicy` change)

221
docs/architecture/decisions/026-forwarded-for-identity.md Normal file
View File

@@ -0,0 +1,221 @@
# ADR-026: Forwarded-For Identity (Metadata, Not Authority)
## Status
Accepted (adds a wire-format field and an `OperationContext` field;
included with the ADR-024 migration or a companion task immediately after,
since `OperationContext` and the `from_call` handler are being rewritten)
## Context
When a hub forwards a call to a spoke (via `from_call`, ADR-022), the spoke
authenticates the hub (resolves the hub's identity from the connection)
and checks its ACL: "is the hub authorized to call this operation?" The
spoke's ACL answers yes/no based on the hub's identity. This is per-node
ACL (ADR-024 §3) — the correct authorization model, no "trusted" bypass.
But the spoke is **blind to the originator**. It knows "the hub called me"
but not "alice asked the hub to call me." The hub's `OperationContext.identity`
holds alice's identity (the hub authenticated alice), but the `from_call`
forwarding handler authenticates as the hub (its own `auth_token`), so the
spoke sees the hub's identity, not alice's. The originator information is
at the hub, not at the spoke.
This matters for three use cases the research at
`docs/research/alknet-storage-strategy/findings.md` §6 identified:
1. **Audit trail.** A cross-node call chain is untraceable at the leaf
without the originator. The spoke logs "the hub called `/docker/start`"
but can't log "alice asked the hub to call `/docker/start`." For
debugging, billing, and abuse investigation, the originator matters.
2. **Per-user rate limiting at the leaf.** If the spoke wants to rate-limit
per-user (not per-hub), or apply per-user quotas, it can't — it only
sees the hub. The hub would have to proxy and track everything, which
defeats the point of direct service composition.
3. **Handler context.** A handler may want the originator's identity for
application logic (per-user views, per-user data isolation, attribution
in logs).
The question is whether to include the originator's identity in the
forwarded call. The wire format is the constraint: a field is either in the
`call.requested` payload or it isn't — it can't be bolted on later without
a protocol change. This is a wire-format one-way door.
## Decision
### 1. Add `forwarded_for` to the `call.requested` payload
```json
{
"operationId": "/docker/start",
"input": { ... },
"auth_token": "alk_...", // the direct caller's token (the hub's)
"forwarded_for": { // the original caller (the end user's)
"id": "alice",
"scopes": ["fs:read", "docker:start"],
"resources": {}
}
}
```
`forwarded_for` is optional (`None` when the call is not forwarded, or
when the forwarder chooses not to propagate it). It carries a serialized
`Identity` (id, scopes, resources) — the originator's resolved identity at
the forwarding node.
### 2. Add `forwarded_for` to `OperationContext`
```rust
pub struct OperationContext {
// ... existing fields ...
/// The original caller when this call was forwarded (ADR-026).
/// Metadata only — NOT used by `AccessControl::check`. The dispatch
/// path populates it from the `call.requested.forwarded_for` field;
/// the `from_call` handler sets it when constructing the forwarded
/// payload (see §3). Handlers may read it for logging, auditing,
/// per-user rate limiting, or application context. The ACL check
/// always runs against `identity` (the direct caller), never against
/// `forwarded_for`.
pub forwarded_for: Option<Identity>,
}
```
`identity` is the direct caller (authorized by ACL). `forwarded_for` is
the original caller (metadata only). The ACL check signature is
`AccessControl::check(identity.as_ref())` — unchanged. The
`forwarded_for` field is a **separate** field, not a parameter to `check`.
### 3. The `from_call` handler populates `forwarded_for`
The hub's `from_call` forwarding handler constructs the `call.requested`
payload to send to the spoke. It populates `forwarded_for` with the end
user's identity — read from the hub's `OperationContext.identity` (the
caller the hub authenticated) when the hub forwards the call. The hub
authenticates as itself (its own `auth_token`); the `forwarded_for` field
carries the originator's identity as context.
This is the hub's responsibility, not the protocol's. The protocol carries
the field; the `from_call` handler chooses to populate it. A forwarder that
doesn't want to disclose the originator can set `forwarded_for: None` (the
spoke sees only the hub). A forwarder that wants to propagate it sets it.
### 4. `AccessControl::check` never reads `forwarded_for`
The security property: `forwarded_for` is metadata, not authority. The
spoke's dispatch path makes it available on `OperationContext` for handlers,
but `AccessControl::check(identity.as_ref())` — the ACL check — always
authorizes the **direct caller's** identity, never the forwarded-for
identity. There is no path through which `forwarded_for` becomes an
authorization input.
This is enforced structurally, not by convention: `AccessControl::check`
takes `Option<&Identity>` (the direct caller's identity). The
`forwarded_for` field is `Option<Identity>` on `OperationContext`, but
the check signature doesn't accept it. If someone wants to ACL on the
forwarded-for identity, they would have to change the
`AccessControl::check` signature — a visible, reviewable change, not a
quiet flag flip. The type system prevents accidental misuse.
## Why include it now
The window is the ADR-024 migration. The `from_call` handler is being
rewritten (peer-keyed overlays, `AccessControl`-based peer authorization,
removal of `remote_safe`/`trusted_peer`), and `OperationContext` is being
touched (the `PeerCompositeEnv` aggregation changes how the context is
built). Adding a field to the `call.requested` payload and to
`OperationContext` now is the cheapest point — the structures are already
under edit. After the protocol ships without it, adding it is a breaking
wire-format change (every client and server must learn the new field) and
an `OperationContext` break (every handler that pattern-matches the struct
must update).
## Consequences
**Positive:**
- The spoke can audit cross-node call chains. The leaf knows who actually
initiated the call, not just who forwarded it.
- Per-user rate limiting at the leaf becomes possible. The spoke can key
rate-limit state on `forwarded_for.id` instead of only on the hub's
identity.
- Handler application logic can use the originator's identity for per-user
views, per-user data isolation, or attribution.
- The security model is unchanged: the spoke authorizes the hub (its
direct caller), not the end user. The `forwarded_for` field is metadata,
not authority. The type-system separation (`check` takes `identity`, not
`forwarded_for`) prevents misuse.
- The forwarder decides. A hub that doesn't want to disclose the
originator (e.g., for privacy, or because the originator's identity is
not meaningful to the spoke) sets `forwarded_for: None`. The field is
opt-in by the forwarder, not mandatory.
**Negative:**
- The `call.requested` payload gains a field. Wire-format addition — old
servers that don't recognize `forwarded_for` ignore it (JSON
deserialization into a struct without the field drops it); old clients
that don't send it produce `forwarded_for: None` on the server. This is
forward-compatible, but a server that wants to *use* `forwarded_for`
must be new enough to deserialize it.
- `OperationContext` gains a field. Handlers that construct
`OperationContext` literals (tests, custom dispatch paths) must add the
field. The composition path (`OperationEnv::invoke`) sets it to `None`
for composed children — `forwarded_for` is a wire-ingress field, not a
composition-ingress field.
- The `Identity` in `forwarded_for` is a serialized value on the wire,
not a server-resolved identity. The spoke receives the hub's *claim*
about the originator's identity. A malicious hub could lie — set
`forwarded_for` to a fake identity. The spoke must not treat
`forwarded_for` as authoritative for anything security-relevant; it's
the hub's assertion, useful for audit/attribution when the hub is
trusted, but not a verified identity. This is the inherent property of
forwarded-for metadata (same as HTTP `X-Forwarded-For` — it's a claim by
the forwarder, not a verified value).
- One more field for the `from_call` handler to populate correctly. The
handler must read the hub's `OperationContext.identity` and decide
whether to propagate it. This is a small implementation cost, but it's a
handler-responsibility increase.
## Assumptions
1. **`forwarded_for` is a claim by the forwarder, not a verified
identity.** The spoke receives the hub's assertion about the
originator. A malicious hub can lie. The spoke must not use
`forwarded_for` as authoritative for security decisions — only for
audit, logging, and application-context purposes when the hub is
trusted. This is the same property as HTTP `X-Forwarded-For`.
2. **`AccessControl::check` never reads `forwarded_for`.** The security
property is structural (the check signature doesn't accept it), not
conventional. Adding `forwarded_for` to the ACL path would require a
signature change to `AccessControl::check` — a visible, reviewable
change.
3. **`forwarded_for` is wire-ingress only.** Composed children (calls via
`OperationEnv::invoke`) do not inherit `forwarded_for` — they get
`None`. The field is populated from `call.requested.forwarded_for` by
the dispatch path, and the `from_call` forwarding handler sets it when
constructing the forwarded payload. Composition-propagation of
`forwarded_for` would be a separate decision (not in this ADR).
4. **The `Identity` shape in `forwarded_for` is the same as `Identity`
on `OperationContext`.** Both carry `id`, `scopes`, `resources`. The
`forwarded_for` value is a serialized `Identity` from the forwarding
node's resolution — the same `Identity` the hub resolved for the end
user, just passed along as metadata.
## References
- ADR-010: Secret Material Flow and Capability Injection (`forwarded_for`
carries an `Identity` with scopes/resources, not secret material — the
no-secret-material-on-the-wire invariant is preserved)
- ADR-017: Privilege Model and Authority Context (the authority-switch
model — `forwarded_for` does not participate; the direct caller's
identity is the authority)
- ADR-022: Call Protocol Client and Adapter Contract (the `from_call`
forwarding handler that populates `forwarded_for`)
- ADR-024: Peer-Graph Routing Model (the migration window —
`OperationContext` and the `from_call` handler are being rewritten)
- `docs/research/alknet-storage-strategy/findings.md` §6 (the
forwarded-for identity decision and rationale)

180
docs/architecture/decisions/027-from-jsonschema-as-http-adapter.md Normal file
View File

@@ -0,0 +1,180 @@
# ADR-027: `from_jsonschema` as an HTTP-Backed Single-Endpoint Adapter in alknet-http
## Status
Accepted (supersedes the `from_jsonschema` clause of ADR-022 §5 and the
`FromJsonSchema` provenance row of ADR-018 — both described a schema-only,
no-handler adapter in `alknet-call`)
## Context
`from_jsonschema` was originally specified (ADR-022 §5) as a schema-only
adapter living in `alknet-call`: it produced `HandlerRegistration` bundles
with a `NOT_FOUND`-returning placeholder handler and `FromJsonSchema`
provenance. The stated use case was validation, discovery, and
composition-graph construction without a runtime — type-checking a
composition plan without executing it, building a UI of available
operations without standing up the transports.
This is broken. An operation in the `OperationRegistry` needs a real
handler. A placeholder that returns `NOT_FOUND` does not work with how
the registry is supposed to function: an `Internal` op registered with
a dead handler is a trap, not a feature. The "schema-only, no handler"
concept conflated two things — schema *validation* (a compile-time /
planning activity that doesn't need a registry entry at all) and
operation *registration* (which always needs a handler). Validation
against a JSON Schema does not require a `HandlerRegistration`; it
requires the schema and a validator. Registering an operation requires
a handler. The old `from_jsonschema` tried to do the former by abusing
the latter, and produced something that works for neither.
The misplacement was compounded by a location error: the adapter lived
in `alknet-call` (which is supposed to stay lean — no HTTP client), but
a `from_jsonschema` that is actually useful for calling non-standard
endpoints needs reqwest, exactly like `from_openapi` and `from_mcp`.
The adapter location map in ADR-022 / `client-and-adapters.md` already
establishes that HTTP-backed adapters live in `alknet-http`; the old
`from_jsonschema` violated its own stated principle by living in
`alknet-call`.
A concrete use case now forces the decision: composing a non-standard,
non-OpenAPI, basic REST endpoint that does not have a full OpenAPI
document. The endpoint has a method, a URL, an input/output JSON Schema,
and an auth scheme — but no `paths` object, no `operationId`, no
`components`. `from_openapi` requires an OpenAPI document; this endpoint
doesn't have one. The gap is: register a single HTTP endpoint as a
call-protocol operation, one at a time, with the caller supplying the
schema directly.
## Decision
`from_jsonschema` becomes an HTTP-backed single-endpoint adapter in
`alknet-http`, functionally similar to `from_openapi` but registering
one endpoint at a time instead of parsing a full OpenAPI document:
1. **Move the adapter implementation to `alknet-http`**
(`crates/alknet-http/src/adapters/from_jsonschema.rs`). The
forwarding handler uses the same reqwest-backed `SharedHttpClient`
and the same no-env-vars credential injection as `from_openapi`. The
adapter implements `OperationAdapter` (the trait from `alknet-call`,
ADR-022 §5 — unchanged).
2. **Give it a real forwarding handler.** A `from_jsonschema`-imported
operation is a leaf with a reqwest forwarding handler, identical in
shape to a `from_openapi`-imported operation — it builds an HTTP
request from the input (path/query/body split per a path template),
injects credentials from `context.capabilities`, sends via the shared
HTTP client, and parses the response (JSON, text, or binary — same
content-type branching as `from_openapi`). For a `Subscription`
op type with `text/event-stream` response, it registers a
`StreamingHandler` (ADR-021), same as `from_openapi`.
3. **Single-endpoint registration.** The caller supplies:
- An `OperationSpec` (name, op type, input/output JSON Schema,
`error_schemas`, `access_control`, `visibility`).
- An `HttpServiceConfig` (base URL, auth scheme, default headers —
the same config type `from_openapi` uses).
- A path template + HTTP method (the one endpoint).
The adapter builds one `HandlerRegistration` with `FromJsonSchema`
provenance and a real forwarding handler. The caller registers it in
the `OperationRegistry`. This is the "one endpoint at a time" shape:
no `paths` object to iterate, no `operationId` to normalize.
4. **`FromJsonSchema` provenance stays in `alknet-call`** (in the
`OperationProvenance` enum, `registration.rs`). The provenance type
lives where the registry types live; only the adapter implementation
moves. `FromJsonSchema` is now a leaf provenance — it has a handler
(a reqwest forwarding handler), same trust model as `FromOpenAPI`
(HTTP endpoint trusted; handler is a forwarding stub).
5. **Remove the "schema-only, no handler" concept.** The placeholder
handler and the "schema-only ops are `Internal`, so dispatch should
never reach them" rationale are removed. An op registered with
`FromJsonSchema` provenance is a real, callable, HTTP-forwarding
operation — `Internal` by default (adapter-registered ops are
composition material, ADR-017), but it actually forwards if invoked.
The schema-validation-without-a-handler use case (type-checking a
composition plan, building a UI) does not require a
`HandlerRegistration` at all. That use case is served by consuming
the `OperationSpec` directly (the spec already carries the input/
output JSON Schemas); no adapter, no registry entry, no handler is
needed. If a future use case requires registering a schema-only op
for discovery purposes, that is a separate feature and would warrant
its own ADR — it is not what `from_jsonschema` is.
### Relationship to `from_openapi`
| | `from_openapi` | `from_jsonschema` |
|---|---|---|
| Input | A full OpenAPI 3.x document (JSON or YAML) | A single endpoint: `OperationSpec` + `HttpServiceConfig` + path template + method |
| Granularity | One `HandlerRegistration` per `(path, method)` in the doc | One `HandlerRegistration` per call |
| Schema source | Parsed from the OpenAPI doc (parameters, request body, responses) | Supplied directly by the caller |
| Handler | reqwest forwarding handler (shared HTTP client) | Same reqwest forwarding handler |
| Provenance | `FromOpenAPI` | `FromJsonSchema` |
| Location | `alknet-http` | `alknet-http` |
| Use case | Standard OpenAPI APIs (GitHub, OpenAI, Anthropic) | Non-standard, non-OpenAPI, or basic REST endpoints without a full spec |
The two adapters share the forwarding-handler implementation, the
credential injection path, the error-fidelity rule (`HTTP_<status>`
prefix, ADR-016), and the no-env-vars invariant (ADR-010). The
difference is purely the input shape: a full document vs. a single
endpoint.
## Consequences
**Positive**:
- `from_jsonschema` actually works — it has a real handler, not a
placeholder. A concrete use case (non-standard REST endpoints) is
served.
- The adapter location is consistent: all HTTP-backed adapters
(`from_openapi`, `from_mcp`, `from_jsonschema`) live in `alknet-http`,
where reqwest is. `alknet-call` stays lean.
- The "schema-only, no handler" trap is removed. An op in the registry
is always callable.
- `FromJsonSchema` provenance becomes a real leaf, consistent with
`FromOpenAPI`/`FromMCP`/`FromCall`.
**Negative**:
- The schema-validation-without-a-handler use case (the original stated
purpose) is no longer served by `from_jsonschema`. That use case is
served by consuming `OperationSpec` directly, but any code that relied
on the placeholder handler returning `NOT_FOUND` breaks. The only
existing consumer is the call crate's own tests; no downstream consumer
depended on this — the placeholder was a trap, not a contract.
- `alknet-call` loses a public export (`from_jsonschema`, `FromJsonSchema`
the adapter struct). The `FromJsonSchema` provenance variant stays;
the adapter struct moves. Downstream consumers that referenced the
adapter (none currently) would need to use `alknet-http`'s re-export.
**Neutral**:
- `FromJsonSchema` provenance is now a leaf (handler-bearing), not a
"no handler" provenance. The ADR-018 table row updates: it can compose?
No. Has composition authority? No. Default visibility? Internal. Trust
model? HTTP endpoint trusted; handler is a forwarding stub. This
aligns with the other leaves. ADR-022 §5 and ADR-018's provenance
table/enum-doc are amended (2026-07-09) to point here — the
supersession is recorded in the superseded ADRs, not only in this one.
## References
- Supersedes the `from_jsonschema` clause of
[ADR-022](022-call-protocol-client-and-adapter-contract.md) §5
("`FromJsonSchema` — imports from a JSON Schema definition (schema-only,
no handler)") and the operational spec in
`docs/architecture/crates/call/client-and-adapters.md` §"from_jsonschema".
- Supersedes the `FromJsonSchema` row of
[ADR-018](018-handler-registration-provenance-and-composition-authority.md)
(the "no handler — schema only" framing).
- Aligns with the adapter location principle in
[ADR-022](022-call-protocol-client-and-adapter-contract.md) §5 and
`client-and-adapters.md` §"Adapter Location Map": HTTP-backed adapters
live in `alknet-http`.
- Reuses the forwarding handler, credential injection, error fidelity
(`HTTP_<status>` prefix, [ADR-016](016-operation-error-schemas.md)),
streaming shape ([ADR-021](021-streaming-handler-for-subscriptions.md)),
and no-env-vars invariant ([ADR-010](010-secret-material-flow-and-capability-injection.md))
established by `from_openapi`.
- Reuses `HttpServiceConfig` and `SharedHttpClient` from
`from_openapi` (in `alknet-http`).

126
docs/architecture/decisions/028-from-call-manual-free-function.md Normal file
View File

@@ -0,0 +1,126 @@
# ADR-028: from_call Is a Manual Free Function, Not Auto-Wired
## Status
Proposed (amended 2026-07-16 by ADR-045 §5: `CallClient::connect()` is
removed; `from_call` is now called by the assembly layer after
`AlknetClient::dial_*` + `CallClient::new(...).spawn_dispatch(conn)`,
not after `connect()`. The manual-free-function decision stands; the
`connect()` references in the body are the pre-ADR-045 shape.)
## Context
OQ-27 resolved (2026-06-27): "The decision is **auto-re-import on connection
establishment**. The overlay is per-connection (Layer 2, ADR-019), so a stale
overlay dies with the connection; re-import on reconnect is naturally scoped to
the new connection."
The spec in `client-and-adapters.md` §"from_call" (line 358) states: "This is
the v1 default; explicit re-import via a future `CallConnection::refresh()` is
additive."
The implementation does not match. `from_call` is a standalone free function
(`client/from_call.rs:80`). `CallClient::connect()` does not call it. The
assembly layer must call `from_call()` + `register_imported_all()` explicitly
after every `connect()`. There is no `CallConnection::refresh()`.
The "v1 default" language is hedging — it makes a committed-but-not-implemented
feature sound like a deliberate phase. The spec says "auto-re-import on
connection establishment" but the code says "the assembly layer calls
`from_call` immediately after `connect()`" (the doc comment on `from_call`,
line 76). These are different things: auto-wiring means `connect()` calls
`from_call()` internally; manual means the caller does it.
The alkapi project identified this as gap G.4: the hedging language in the
spec, and the question of whether `from_call` should be auto-wired into
`connect()`.
## Decision
**`from_call` is a manual free function. The assembly layer calls it after
`connect()`. It is not auto-wired into `CallClient::connect()`.**
### Why manual is correct
1. **The hub controls discovery timing.** A hub may want to verify the
connection, resolve the peer's identity, check authorization, and *then*
discover operations. Auto-wiring `from_call` into `connect()` would run
discovery before the assembly layer has a chance to inspect the connection.
2. **Discovery is not always wanted.** A pure-client connection to a public
X.509 endpoint (ADR-034) has no `PeerEntry` and no `PeerId` — the remote
is not in the peer graph. Auto-discovering ops on such a connection would
register them in a connection overlay that has no peer key, making them
unreachable via `PeerRef`. The assembly layer decides whether to run
`from_call` based on whether the remote is a known peer.
3. **The `from_call` function is already the right API.** It takes a
`&CallConnection` and a `FromCallConfig`, returns
`Result<Vec<HandlerRegistration>, AdapterError>`, and the caller registers
the bundles. This is a clean separation: connect, discover, register. Each
step is independently testable and independently controllable.
4. **Auto-wiring would require `from_call` to know about the registry.**
`CallClient` holds an `Arc<OperationRegistry>`, but `from_call` produces
`HandlerRegistration` bundles that the caller registers — the caller
decides *where* to register them (the connection's overlay, a session
overlay, or not at all). Auto-wiring would hardcode the registration
target.
### What changes in the spec
The "v1 default" language in `client-and-adapters.md` and ADR-022 is replaced
with an honest statement: `from_call` is a free function; the assembly layer
calls it after `connect()`; there is no `CallConnection::refresh()` for
mid-connection re-discovery. A `CallConnection::refresh()` method is a
genuine feature addition — non-breaking, additive — if a deployment needs
manual re-discovery without drop-and-reconnect.
OQ-27 is updated: the resolution changes from "auto-re-import on connection
establishment" to "manual — the assembly layer calls `from_call` after
`connect()`." The door type remains two-way (auto-wiring is additive).
### What does NOT change
- The `from_call` function signature, behavior, and tests are unchanged.
- `CallClient::connect()` is unchanged.
- The re-import-on-reconnect pattern is unchanged: the assembly layer's
supervision loop calls `from_call` after each `connect()`. The overlay is
per-connection, so a stale overlay dies with the connection; re-import on
reconnect is naturally scoped. This is the correct behavior — it just
isn't automatic.
## Consequences
**Positive:**
- The spec matches the implementation. No hedging language.
- The assembly layer has full control over discovery timing and registration
target.
- The separation of concerns (connect / discover / register) is clean and
testable.
- No code changes needed — this is a spec correction, not an implementation
change.
**Negative:**
- The assembly layer must remember to call `from_call` after `connect()`.
This is a documentation concern, not a correctness concern — forgetting to
call `from_call` means the peer's ops are not imported, which is immediately
visible (calls to those ops return `NOT_FOUND`).
- The "auto-re-import on connection establishment" resolution of OQ-27 was
aspirational and is now corrected. The resolution was written before the
implementation existed; the implementation made the right call (manual),
and the spec is catching up.
## References
- ADR-022 §3: `from_call` adapter specification
- ADR-022 Amendments (DC-2): the amended `from_call` re-import resolution
(manual free function)
- ADR-029: Aggregated Peer-Environment Wiring (sibling hub-wiring decision)
- ADR-030: PeerCompositeEnv::peer_operations Override (sibling hub-wiring decision)
- OQ-27: from_call re-import trigger (amended 2026-07-09)
- `client-and-adapters.md` §"from_call" (updated)
- `crates/alknet-call/src/client/from_call.rs:80` — `from_call` free function
- `crates/alknet-call/src/client/call_client.rs:142-168` — `connect()` does
not call `from_call`
- alkapi gap G.4: `from_call` wiring + "v1 default" hedging cleanup

213
docs/architecture/decisions/029-aggregated-peer-env-wiring.md Normal file
View File

@@ -0,0 +1,213 @@
# ADR-029: Aggregated Peer-Environment Wiring for Hub Deployments
## Status
Proposed
## Context
`Dispatcher::compose_root_env` (`protocol/dispatch.rs:134-154`) constructs a
**fresh** `PeerCompositeEnv` per call and attaches **only the current call's
own connection** as a peer overlay. It does not aggregate the hub's other live
worker connections into that call's environment.
Consequence: on a hub with N connected workers, a handler composing
`env.invoke_peer(&PeerRef::Specific("dev1"), "docker", "container/exec",
input, &ctx, policy)` from a call that arrived on an HTTP connection (or on a
*different* worker's connection) will **not** find dev1's overlay — the routing
falls through to the curated base and returns `NOT_FOUND`.
The `PeerCompositeEnv` *type* and the `invoke_peer` routing logic are built for
multi-peer aggregation (`attach_peer`/`detach_peer` with insertion-order
preservation, `PeerRef::Specific`/`Any` routing — `registry/env.rs:155-301`).
The per-call `compose_root_env` does not use that capability. The ADR-024
*model* is committed; the implementation is incomplete for the head→N-workers
case.
This is the single highest-impact gap a first hub consumer (alkapi) surfaces.
A hub is *defined* by composing ops across its connected workers. Without an
aggregated env shared across all calls, the hub pattern does not work: a call
arriving on one transport cannot reach a worker connected on another.
The alkapi project identified this as OQ-08 and committed to the aggregation
decision in their ADR-006. The decision to aggregate is made; the question is
where the wiring lives — alknet (reusable by any hub) or a hub-side wrapper
(alkapi-only). This ADR resolves that question: the wiring lives in alknet.
## Decision
### 1. `Dispatcher` gains a `with_aggregated_env` builder method
A new optional field on `Dispatcher` holds a shared aggregated
`PeerCompositeEnv`:
```rust
pub struct Dispatcher {
pub registry: Arc<OperationRegistry>,
pub identity_provider: Arc<dyn IdentityProvider>,
pub session_source: Option<Arc<dyn SessionOverlaySource + Send + Sync>>,
pub ownership_provider: Option<Arc<dyn OwnershipProvider>>,
pub aggregated_env: Option<Arc<std::sync::RwLock<PeerCompositeEnv>>>,
pub default_timeout: Duration,
}
impl Dispatcher {
pub fn with_aggregated_env(
mut self,
env: Arc<std::sync::RwLock<PeerCompositeEnv>>,
) -> Self {
self.aggregated_env = Some(env);
self
}
}
```
The builder method mirrors `with_session_source` and `with_ownership_provider`
— an optional hook the assembly layer wires at construction time. A deployment
that does not set an aggregated env gets today's `compose_root_env` behavior
unchanged.
### 2. `CallAdapter` gains a matching `with_aggregated_env` builder method
`CallAdapter` delegates to `Dispatcher`:
```rust
impl CallAdapter {
pub fn with_aggregated_env(
mut self,
env: Arc<std::sync::RwLock<PeerCompositeEnv>>,
) -> Self {
self.dispatcher = self.dispatcher.with_aggregated_env(env);
self
}
}
```
### 3. `compose_root_env` reads the aggregated env when set
When `aggregated_env` is `Some`, `compose_root_env` reads the shared env,
attaches the current connection's overlay as an override for the current call
only, and returns the result. When `None`, the existing per-call behavior is
preserved:
```rust
pub fn compose_root_env(
&self,
connection: &CallConnection,
context: &OperationContext,
) -> Arc<dyn OperationEnv + Send + Sync> {
let base: Arc<dyn OperationEnv + Send + Sync> =
Arc::new(LocalOperationEnv::new(Arc::clone(&self.registry)));
let session = self
.session_source
.as_ref()
.and_then(|s| s.overlay_for(context));
if let Some(aggregated) = &self.aggregated_env {
// Acquire read lock on the shared aggregated env, clone it (cheap —
// all fields are Arc), and release the lock. The clone is the
// per-call snapshot; the lock is not held for the call duration.
let mut env = aggregated
.read()
.expect("aggregated env lock poisoned")
.clone();
// Attach the current connection's overlay as an override for this
// call only. The current connection's overlay is the authoritative
// view of *that* peer; the aggregated env is the authoritative view
// of *all other* peers. This avoids a race where the aggregated env
// has not yet picked up a new op the current peer just registered.
if let Some(peer_id) = connection.identity().map(|identity| identity.id.clone()) {
env.attach_peer(peer_id, connection.overlay_env());
}
Arc::new(env)
} else {
let mut env = PeerCompositeEnv::new(base);
if let Some(session) = session {
env = env.with_session(session);
}
if let Some(peer_id) = connection.identity().map(|identity| identity.id.clone()) {
env.attach_peer(peer_id, connection.overlay_env());
}
Arc::new(env)
}
}
```
The clone of the aggregated env is cheap: `PeerCompositeEnv`'s fields are all
`Arc` (the `HashMap` values are `Arc<dyn OperationEnv>`, the `Vec` is
`Vec<PeerId>` which is a `String` clone). The `RwLock::read()` is held only
for the clone, not for the duration of the call.
### 4. The hub owns the aggregated env lifecycle
The hub (assembly layer) constructs the aggregated env once at startup:
```rust
let base: Arc<dyn OperationEnv + Send + Sync> =
Arc::new(LocalOperationEnv::new(Arc::clone(&registry)));
let aggregated = Arc::new(RwLock::new(PeerCompositeEnv::new(base)));
let adapter = CallAdapter::new(registry, identity_provider)
.with_aggregated_env(Arc::clone(&aggregated));
```
The hub calls `attach_peer(peer_id, overlay)` on the aggregated env on every
worker connection-establish (after `from_call` populates the overlay) and
`detach_peer(&peer_id)` on every disconnect. The write lock is held only for
the `HashMap` insert/remove — connection-rate, not call-rate.
### 5. `PeerCompositeEnv` gains `Clone`
`PeerCompositeEnv` is made `Clone` (all fields are `Arc` or `Clone` already).
This is a one-line derive addition.
## Consequences
**Positive:**
- The hub pattern works. A call arriving on any transport can reach any
connected worker's ops via `PeerRef::Specific` or `PeerRef::Any`.
- The existing single-connection behavior is preserved. A deployment that does
not set an aggregated env gets today's `compose_root_env` unchanged.
- The hook is additive — a new optional field, a new builder method, a branch
in `compose_root_env`. No existing code path changes.
- The capability is reusable by any future hub, not just alkapi. The alkapi
project's ADR-006 fallback (hub-side wrapper) is no longer needed.
**Negative:**
- A `RwLock<PeerCompositeEnv>` on the read hot path of every dispatch. The
lock is held only for a clone (all `Arc` fields — cheap). An `ArcSwap`
copy-on-write variant could avoid the lock on reads entirely, at the cost
of a clone on `attach_peer`/`detach_peer` (infrequent). The `RwLock` is the
simpler starting point; `ArcSwap` is an additive optimization.
- `PeerCompositeEnv` gains `Clone`. The derive is mechanical; all fields are
already `Clone`.
- The hub must manage the aggregated env lifecycle (`attach_peer`/`detach_peer`
on connection events). This is assembly-layer code, not alknet-call code.
The hooks exist; the hub wires them.
## Assumptions
1. **`PeerCompositeEnv` clone is cheap.** All fields are `Arc` or `Clone` of
small types (`String`, `Vec<String>`). The clone does not copy the
operation registries or the connection overlays — it copies `Arc` pointers.
2. **The current connection's overlay is authoritative for that peer.** A call
arriving on worker-a uses worker-a's live overlay as the view of worker-a
(not the aggregated env's possibly-stale snapshot), and the aggregated env
for all other peers. This avoids a race where the aggregated env has not
yet picked up a new op worker-a just registered.
3. **The `RwLock` is not a contention point.** Reads (clones) are call-rate
but the lock is held only for the clone duration (microseconds). Writes
(`attach_peer`/`detach_peer`) are connection-rate (seconds to minutes). If
profiling shows contention, `ArcSwap` is the additive optimization.
## References
- ADR-024: Peer-Graph Routing Model (the model this wiring completes)
- ADR-025: PeerEntry and Identity.id Decoupling (the `PeerId` source)
- ADR-030: PeerCompositeEnv::peer_operations Override (sibling hub-wiring decision)
- ADR-028: from_call Is a Manual Free Function (sibling hub-wiring decision)
- alkapi ADR-006: Aggregated Peer Environment (the downstream commitment)
- alkapi OQ-08: alknet aggregated peer-env wiring (the blocking question)
- `crates/alknet-call/src/protocol/dispatch.rs:134-154` — current
`compose_root_env`
- `crates/alknet-call/src/registry/env.rs:155-301` — `PeerCompositeEnv`

154
docs/architecture/decisions/030-peer-composite-env-peer-operations.md Normal file
View File

@@ -0,0 +1,154 @@
# ADR-030: PeerCompositeEnv::peer_operations Override
## Status
Proposed
## Context
`OperationEnv::peer_operations` (defined in `registry/env.rs:63-65`) has a
default implementation returning `Vec::new()`. `PeerCompositeEnv` overrides
`invoke_with_policy`, `contains`, `invoke_peer`, `peer_contains`, and
`peer_ids` — but does **not** override `peer_operations`. This means
`peer_operations` on a `PeerCompositeEnv` always returns an empty `Vec`.
The `services/list-peers` handler (`registry/discovery.rs:245-296`) calls
`ctx.env.peer_operations(&peer_id)` to discover what operations each peer
serves. Since `PeerCompositeEnv` does not override this, non-local peers
always show empty operation lists in the `list-peers` response. The
`peer_ids()` method correctly returns the peer IDs, but the operations for
each peer are always empty.
This is a pure gap — the `services/list-peers` handler is specced to enumerate
each peer's operations (ADR-024 §6), and the `PeerCompositeEnv` type has all
the data needed to implement it (each peer's `OverlayOperationEnv` holds a
`HashMap<String, HandlerRegistration>`). The override is one method collecting
each peer overlay's registered op names.
The alkapi project identified this as gap G.6: a hub consumer calling
`services/list-peers` gets `peers: [{peer_id: "dev1", operations: []}]` until
this is fixed.
## Decision
`PeerCompositeEnv` overrides `peer_operations` to collect the operation names
from each peer's connection overlay:
```rust
fn peer_operations(&self, peer: &PeerId) -> Vec<String> {
match self.connections.get(peer) {
Some(overlay) => {
// The overlay is an OverlayOperationEnv wrapping a
// HashMap<String, HandlerRegistration>. We need the op names.
// Rather than adding a method to OperationEnv (which would
// require every impl to add it), we use the existing `contains`
// method — but that requires knowing the name to check.
//
// The correct approach: iterate the overlay's known names.
// OverlayOperationEnv already has the data (the HashMap keys).
// We add a `list_operation_names(&self) -> Vec<String>` method
// to OperationEnv with a default returning Vec::new(), and
// OverlayOperationEnv overrides it to return the keys.
overlay.list_operation_names()
}
None => Vec::new(),
}
}
```
### 1. `OperationEnv` gains `list_operation_names` with a default impl
```rust
fn list_operation_names(&self) -> Vec<String> {
Vec::new()
}
```
The default returns empty — existing impls (`LocalOperationEnv`, test-only
envs) don't need to change. Only `OverlayOperationEnv` overrides it.
### 2. `OverlayOperationEnv` overrides `list_operation_names`
```rust
impl OperationEnv for OverlayOperationEnv {
fn list_operation_names(&self) -> Vec<String> {
self.overlay.read().keys().cloned().collect()
}
// ... existing impl unchanged
}
```
### 3. `PeerCompositeEnv::peer_operations` uses `list_operation_names`
The override delegates to each peer's overlay:
```rust
fn peer_operations(&self, peer: &PeerId) -> Vec<String> {
self.connections
.get(peer)
.map(|overlay| overlay.list_operation_names())
.unwrap_or_default()
}
```
### Why a new trait method instead of a different approach
Alternatives considered:
- **Add `fn operations(&self) -> Vec<String>` to `OperationEnv`**: Same
concept, different name. `list_operation_names` is chosen to match the
existing `list_operations` naming on `OperationRegistry`.
- **Make `peer_operations` on `PeerCompositeEnv` reach into
`OverlayOperationEnv`'s internals**: Requires `OverlayOperationEnv` to
expose its `HashMap` or a method. The trait method is cleaner — it keeps
the abstraction boundary intact.
- **Have `services/list-peers` iterate `ctx.env.peer_ids()` and call
`contains` for every known op name**: Requires knowing all possible op
names (from the registry), which is a cross-layer coupling. The trait
method keeps the data where it lives.
The trait method is the smallest surface change: one new method with a
default impl, one override on `OverlayOperationEnv`, one override on
`PeerCompositeEnv`. No existing code changes.
## Consequences
**Positive:**
- `services/list-peers` returns actual operation lists for each peer. A hub
consumer calling `services/list-peers` gets `peers: [{peer_id: "dev1",
operations: [{name: "docker/container/exec", ...}, ...]}]` — the specced
behavior.
- The fix is small: one trait method, two overrides. No existing code paths
change.
- The `list_operation_names` method is generally useful — any future code
that needs to enumerate an env's operations can use it.
**Negative:**
- `OperationEnv` gains a method. The default impl preserves back-compat for
all existing implementors. Only `OverlayOperationEnv` and
`PeerCompositeEnv` override it.
- The `OverlayOperationEnv` override holds the `RwLock` read for the
duration of the `keys().cloned().collect()`. This is a `Vec<String>`
allocation — cheap for typical peer operation counts (tens, not thousands).
## Assumptions
1. **`OverlayOperationEnv`'s `RwLock<HashMap<String, HandlerRegistration>>`
read is cheap.** The lock is held only for the `keys()` iteration and
`collect()`. Typical peer operation counts are small (tens of ops).
2. **`list_operation_names` is the right name.** It matches the existing
`list_operations` naming on `OperationRegistry` and avoids confusion with
`peer_operations` (which takes a `PeerId` parameter).
## References
- ADR-024 §6: `services/list-peers` opt-in peer-attributed re-export listing
- ADR-029: Aggregated Peer-Environment Wiring (sibling hub-wiring decision)
- ADR-028: from_call Is a Manual Free Function (sibling hub-wiring decision)
- `crates/alknet-call/src/registry/env.rs:63-65` — default `peer_operations`
- `crates/alknet-call/src/registry/env.rs:155-301` — `PeerCompositeEnv`
- `crates/alknet-call/src/protocol/connection.rs:305-397` —
`OverlayOperationEnv`
- `crates/alknet-call/src/registry/discovery.rs:245-296` —
`services_list_peers_handler`
- alkapi gap G.6: `PeerCompositeEnv::peer_operations` unimplemented

143
docs/architecture/decisions/031-crate-decomposition.md Normal file
View File

@@ -0,0 +1,143 @@
# ADR-031: Crate Decomposition
## Status
Accepted
## Context
The previous alknet-core crate was a monolith containing transport, interface, server, client, call, auth, config, socks5, credentials, and HTTP — all in one crate with interdependent modules. This created coupling (interface types depended on auth, server depended on call, everything depended on config) and made it impossible to use individual components independently.
The new ALPN dispatch model eliminates the need for a shared interface layer. Each handler is self-contained — it receives a byte stream and manages its own protocol. This naturally decomposes into separate crates.
Key constraints:
- Protocol crates must depend on alknet-core for auth/identity/config — but not on each other
- alknet-vault is already standalone (no alknet-core dependency) and must remain so (see ADR-008)
- The CLI binary assembles everything — it's the only crate that depends on all handler crates
- Handlers with protocol-agnostic cores (SFTP, call protocol) preserve the WASM door — browser clients can implement the wire format over WebTransport (see ADR-032, ADR-033)
- alknet-call includes the call protocol client and adapter traits, not just the server side — this enables alknet-agent and alknet-napi to use it for remote invocation
- Rust is the canonical implementation language. TypeScript is a reference/browser adaptation, not a parallel implementation (see ADR-033)
## Decision
The workspace decomposes into the following crates:
| Crate | Responsibility | Depends on |
|-------|---------------|------------|
| `alknet-core` | ProtocolHandler trait, ALPN router, endpoint, BiStream, AuthContext, IdentityProvider, config, ArcSwap dynamic config | tokio, quinn, rustls, iroh (feature-gated, added by ADR-010) |
| `alknet-vault` | Local key vault: BIP39/SLIP-0010/AES-GCM key derivation, encryption | (standalone, no alknet-core) |
| `alknet-ssh` | SshAdapter (russh, SOCKS5, port forwarding) | alknet-core, russh |
| `alknet-call` | CallAdapter (JSON-RPC via hand-rolled EventEnvelope framing, operation registry, pub/sub, access control, call protocol client, adapter traits) | alknet-core |
| `alknet-agent` | Agent service: LLM execution loop (forked aisdk), tool dispatch via call protocol, provider key retrieval via vault | alknet-call |
| `alknet-git` | GitAdapter (gix, pkt-line protocol) | alknet-core, gix |
| `alknet-sftp` | SftpAdapter (russh-sftp protocol core) | alknet-core, russh-sftp |
| `alknet-msg` | MessageAdapter (E2E encryption, mixnet) | alknet-core |
| `alknet-http` | HttpAdapter (axum, REST API, MCP endpoint) | alknet-core, axum |
| `alknet-dns` | DnsAdapter (hickory-proto, pkarr, service discovery) | alknet-core, hickory-proto |
| `alknet-napi` | Node.js native addon — thin NAPI projection of the call protocol client | alknet-call, napi-rs |
| `alknet` | CLI binary — registers handlers, starts endpoint | all handler crates, alknet-vault |
Dependency flow:
```
alknet-vault (standalone)
alknet-core ← all handler crates ← alknet (CLI)
alknet-call ← alknet-agent
alknet-call ← alknet-napi
```
No handler crate depends on another handler crate. Cross-handler communication goes through the call protocol (alknet-call) or through alknet-core's endpoint.
alknet-agent depends on alknet-call (not alknet-core directly) because it uses the call protocol client for tool dispatch and the operation registry for tool registration. It receives LLM provider keys through capabilities injected at the assembly layer (from alknet-vault), never from environment variables and never over the call protocol. See ADR-008 and ADR-010.
alknet-napi is a thin projection layer — it exposes the Rust call protocol client to Node.js via NAPI. It does not contain business logic or adapter implementations. See ADR-033.
## Consequences
**Positive:**
- Each handler can be developed, tested, and versioned independently
- WASM-compatible handlers (sftp, call) don't pull in heavy dependencies (russh, axum)
- alknet-vault remains standalone — no circular dependency risk
- New handlers are added by creating a crate and registering it with the endpoint
- Clean separation of concerns — each crate has one job
**Negative:**
- More crates to manage in the workspace — workspace Cargo.toml and version coordination
- Shared types (AuthContext, BiStream) must live in alknet-core — if they change, all handlers recompile
- The CLI binary has a large dependency tree (all handlers) — but this is expected for a binary that assembles everything
- Testing cross-handler behavior requires integration tests in the CLI or a test utility crate
## References
- Pivot proposal: `docs/research/pivot/alpn-service-architecture.md`
- ADR-001: ALPN-based protocol dispatch
- ADR-002: ProtocolHandler trait
- ADR-003: Auth as shared core (IdentityProvider)
- ADR-013: irpc as call protocol foundation (superseded by ADR-014)
## Amendments
### Amendment 1 (2026-06-29): `alknet-call` is a protocol-foundation crate
The Decision table lists `alknet-call` as a handler crate that "depends
on alknet-core, irpc." The dependency-flow diagram and the "No handler
crate depends on another handler crate" rule were written before
`alknet-http` (which implements `from_openapi`/`from_mcp`/`to_openapi`/
`to_mcp` and therefore needs `alknet-call`'s `OperationSpec`, `Handler`,
`HandlerRegistration`, and `OperationAdapter` trait) was specced.
**Clarification:** `alknet-call` is both a handler crate (it implements
`ProtocolHandler` on ALPN `alknet/call`) *and* the protocol-foundation
crate that `alknet-agent`, `alknet-napi`, and `alknet-http` consume for
the operation registry, adapter contract, and call client. The "no
handler crate depends on another handler crate" rule applies to peer
handler crates (e.g., `alknet-http` does not depend on `alknet-ssh`);
`alknet-call` is a protocol-foundation crate in the same spirit that
`alknet-core` is, just at a different layer (operations/RPC vs.
transport/auth/config).
`alknet-http` depending on `alknet-call` is "HTTP uses the call protocol
types," not "HTTP depends on SSH." This is within the spirit of this
ADR's decomposition. The `alknet-call` → `alknet-http` edge is recorded
in the `alknet-http` spec (`crates/http/overview.md`) and in the adapter
location map (`crates/call/client-and-adapters.md`).
### Amendment 2 (2026-07-07): alknet-tty does not depend on alknet-call
Amendment 1's protocol-foundation framing was extended to alknet-tty in
an earlier draft ("alknet-tty depends on alknet-call for the
`FrameFramedReader`/`FrameFramedWriter` framing utility"). A
pre-implementation sanity check found this was unsound:
`FrameFramedReader::read_frame()` is hardcoded to deserialize
`EventEnvelope` — the length-prefix read and the type-specific
deserialize are one entangled call, not a separable "framing utility."
alknet-tty's negotiation frame is a `NegotiateRequest`, not an
`EventEnvelope`, so `read_frame()` cannot return what alknet-tty needs;
the claimed reuse did not exist in a usable form.
**Clarification:** alknet-tty does **not** depend on alknet-call.
alknet-tty implements its own length-prefixed framing (~30 lines: 4-byte
big-endian length + UTF-8 JSON body) directly on tokio's
`AsyncRead`/`AsyncWrite`. The format coincides with alknet-call's
framing by convention (both are length-prefixed JSON); the
implementations are independent. The Amendment 1 protocol-foundation
exception remains for alknet-http/agent/napi (which use alknet-call's
`OperationSpec`/`Handler`/`OperationAdapter` types — actual type reuse,
not framing glue); it no longer covers alknet-tty. See
[ADR-057](057-alknet-tty-no-alknet-call-dep.md) for the full decision
and the three options considered (duplicate / promote to core / use
alknet-call).
### Amendment 3 (2026-07-09): irpc is not a dependency of any crate
The Decision table listed `irpc` as a dependency of `alknet-core` ("tokio,
quinn, rustls, irpc, iroh") and `alknet-call` ("alknet-core, irpc"). This
was carried over from the previous architecture and never verified against
the implementation: **no `.rs` file in the workspace ever imported irpc**.
The call protocol's wire format (`crates/alknet-call/src/protocol/wire.rs`)
is hand-rolled length-prefixed JSON; the `EventEnvelope` shape was derived
from the `@alkdev/pubsub` TypeScript prior art (ADR-033), not from irpc.
The dead `irpc` / `irpc-derive` workspace deps and the `alknet-call` consumer
dep were removed in commit `668d777`. See
[ADR-014](014-irpc-never-integrated-hand-rolled-framing.md) for the full
record (ADR-013, which accepted "irpc as the call protocol foundation," is
superseded).

72
docs/architecture/decisions/032-one-way-door-decision-framework.md Normal file
View File

@@ -0,0 +1,72 @@
# ADR-032: One-Way Door Decision Framework
## Status
Accepted
## Context
Not all architectural decisions carry the same reversal cost. Some decisions are easy to change later — if you pick the wrong data structure, you refactor. Other decisions are nearly impossible to reverse — if you build a type hierarchy that forecloses WASM compatibility, every handler written against that hierarchy must be rewritten.
This distinction matters especially during Phase 0 (exploration) and early Phase 1 (architecture). The project is post-pivot with foundational ADRs in place but no implementation code yet (except alknet-vault). Decisions made now shape the API surface that every handler depends on.
Without an explicit framework, one-way doors can be treated as casually as two-way doors, leading to costly rework. Or conversely, two-way doors can be over-analyzed, blocking progress on decisions that are cheap to reverse.
## Decision
### Classification
Every architectural decision is classified by **reversal cost** — how expensive it is to undo if you got it wrong:
**One-way door** — Reversing this decision requires rewriting significant code across multiple crates or permanently closes a capability door. Getting it wrong is expensive. Examples:
- BiStream as a concrete quinn type (closes WASM door permanently)
- alknet-vault pulled into alknet-core as a dependency (loses standalone property permanently)
- ProtocolHandler signature changes (every handler must be rewritten)
**Two-way door** — Reversing this decision is cheap or additive. Getting it wrong is recoverable. Examples:
- Static vs dynamic handler registration (can add ArcSwap later)
- Single transport vs multi-transport endpoint (can add transport trait later)
- Call protocol stream model (can add multiplexing later)
### Process
- **One-way doors** require an ADR before implementation. If the right choice is unclear, validate with a POC before writing the ADR. If a POC can't resolve the uncertainty within a reasonable timebox, default to the option that keeps more doors open. One-way doors get the deliberation they deserve because getting them wrong is expensive.
- **Two-way doors** still require a decision — pick the simplest option that works, implement it, and move on. If it turns out wrong, revert and try the alternative. The decision is made; what's cheap is the reversal. Note the decision in a commit message or a brief ADR if the context is worth capturing, but don't block on it.
- When in doubt about which classification applies, classify up. If it's unclear whether a door is one-way or two-way, treat it as one-way until proven otherwise.
### What this framework is NOT
This framework classifies decisions by **reversal cost**, not by **urgency**. It does not say "two-way doors can be deferred." A two-way door is a decision you make now and can revert later if needed — it's not a license to leave the decision unmade.
- **Deferral** is a separate concept: sometimes a decision genuinely doesn't need to be made yet because the use case isn't concrete (scope management). That's valid, but it's a scoping judgment, not a door-type classification.
- **Conflating the two** — using "it's a two-way door" as a reason to defer an architectural decision — leads to decisions that compound into a mess. The decision gets made by default (the implementation picks something), and downstream code builds on it, making the "cheap reversal" expensive.
- **The architect's role**: architecture decisions (one-way OR two-way) are for the architect to make, not the implementation agent. The implementation agent makes implementation decisions (variable names, loop order, which library to use for a parsed task). If a decision affects the system's structure, constraints, or API surface, it's an architecture decision regardless of its door type.
### WASM as a design constraint
WASM compatibility is not an immediate implementation goal, but it is a **design constraint on one-way doors**. Decisions that would permanently prevent WASM targets from participating as peers require explicit justification. This means:
- Core types (BiStream, ProtocolHandler, AuthContext) must not assume tokio or quinn
- Protocol parsers that are pure data transformations should remain transport-agnostic
- The cost of keeping the WASM door open is low (trait vs concrete type, abstracted I/O) and the cost of closing it is high (impossible to reverse without rewriting every handler)
This is not "WASM support now." It's "don't close the WASM door accidentally."
## Consequences
**Positive:**
- One-way doors get the deliberation they deserve — ADRs, POCs, explicit justification — because getting them wrong is expensive
- Two-way doors don't block progress — decide, implement, revert if needed — because getting them wrong is recoverable
- WASM compatibility is preserved as a constraint, not treated as an active deliverable
- The framework creates a shared vocabulary for discussing reversal cost ("is this a one-way door?")
**Negative:**
- Classification requires judgment — some decisions are genuinely ambiguous (mitigated: classify up when in doubt)
- POC timeboxing can feel constraining on genuine hard problems (mitigated: the timebox is "reasonable," not "arbitrary")
- The framework adds a step to every architectural discussion ("is this one-way or two-way?") — but this step is fast and prevents expensive mistakes
## References
- ADR-005: BiStream type definition (one-way door: WASM compatibility)
- ADR-008: Secret service integration point (one-way door: standalone crate independence)
- SDD process: `docs/sdd_process.md` (Phase 0 exploration, POC specialist)
- Pivot proposal: `docs/research/pivot/alpn-service-architecture.md`

72
docs/architecture/decisions/033-rust-canonical-implementation.md Normal file
View File

@@ -0,0 +1,72 @@
# ADR-033: Rust as Canonical Implementation Language
## Status
Accepted
## Context
alknet's core crates (alknet-core, alknet-call, alknet-vault) and all handler crates are implemented in Rust. A previous TypeScript implementation (`@alkdev/operations`, `@alkdev/pubsub`) informed the design of the call protocol — its operation registry, EventEnvelope framing, adapter patterns (from_openapi, from_mcp, from_call), and bidirectional composition.
The question is: what is the relationship between the TypeScript implementation and the Rust implementation? Is TypeScript a parallel implementation that must be maintained in lockstep, or is Rust the canonical implementation with TypeScript serving a specific role?
Five factors make Rust the canonical choice:
1. **Memory safety eliminates an entire vulnerability class.** Rust's ownership model prevents buffer overflows, use-after-free, and other memory corruption bugs that are endemic in C/C++ and impossible to audit away in JavaScript runtimes.
2. **LLM code generation quality is comparable across Rust and TypeScript.** Agents "grok" both languages roughly equally, so there is no productivity argument for TypeScript.
3. **NPM supply chain attacks are growing rapidly.** The JavaScript ecosystem's dependency density makes supply chain attacks a persistent and increasing risk. NPM is dropping features like post-install scripts in response. This trend makes JavaScript an unreliable foundation for security-critical infrastructure.
4. **Rust is significantly faster.** For networking, encryption, and protocol handling, the performance difference is material — not marginal.
5. **The only legitimate JavaScript use case is the browser.** WASM/WebTransport clients need a JavaScript SDK, and the existing `@alkdev/operations` TypeScript code can be adapted for browser use cases where users want to expose operations to web applications. This is a consumer SDK, not a parallel implementation.
## Decision
**Rust is the canonical implementation language.** All alknet crates are implemented in Rust. The TypeScript `@alkdev/operations` and `@alkdev/pubsub` libraries are reference implementations that informed the design; they are not maintained as parallel implementations.
The relationship between the TypeScript and Rust implementations:
| Aspect | Rust (canonical) | TypeScript (reference/browser) |
|--------|-----------------|-------------------------------|
| OperationSpec, OperationRegistry | alknet-call owns canonical types | `@alkdev/operations` projects canonical types into TS |
| Wire protocol (EventEnvelope) | alknet-call owns canonical framing | `@alkdev/pubsub` implements the same wire format for browser |
| Adapter patterns (from_*, to_*) | alknet-call defines adapter traits and Rust implementations | Browser-adapted implementations where needed |
| Call protocol client | alknet-call (QUIC) | alknet-napi (QUIC via NAPI) or browser SDK (WebTransport) |
| LLM provider integration | alknet-agent (forked aisdk, simplified) | Not applicable |
| Provider key management | alknet-vault via assembly-layer capabilities (no env vars) | Not applicable |
**The adapter contract (from_openapi, from_mcp, from_call, to_openapi, to_mcp) lives in Rust.** These patterns convert external specifications or protocols into `OperationSpec + Handler` pairs that register in the local `OperationRegistry`. The TypeScript implementations serve as reference for browser adaptations, not as the source of truth.
**alknet-napi is a thin projection layer.** It exposes the Rust call protocol client to Node.js via NAPI. It does not contain business logic or adapter implementations. TypeScript consumers who want to use alknet from Node.js use alknet-napi to access the Rust implementation.
**The browser SDK is a future adaptation.** When WASM/WebTransport support is needed, the existing TypeScript code can be adapted to run in browsers, speaking the same EventEnvelope wire format over WebTransport streams. This preserves the WASM door (ADR-032) without requiring Rust-to-WASM compilation of the full stack.
## Consequences
**Positive:**
- Single implementation to maintain, test, and secure
- Memory safety eliminates a whole class of vulnerabilities
- Provider key management through alknet-vault (call protocol) instead of env vars
- No NPM dependency chain for security-critical infrastructure
- The existing TypeScript code informs the Rust design — its patterns are preserved, not its implementation
- Browser clients get a thin, adapted SDK rather than the full operations library
**Negative:**
- Browser support requires a separate JavaScript SDK (adapted from existing TS code) rather than a shared implementation
- Contributors who only know JavaScript cannot contribute to core alknet crates
- The `@alkdev/operations` TypeScript library may drift from the canonical Rust types if not kept in sync during the transition period
**Risks mitigated:**
- WASM door preserved: The `@alkdev/operations` TypeScript code can be adapted for browser use without recompiling Rust to WASM. The wire format is JSON, which any runtime can produce and consume.
- NAPI consumers: alknet-napi provides the call protocol client to Node.js without reimplementing in JavaScript.
## References
- ADR-031: Crate decomposition
- ADR-013: irpc as call protocol foundation
- ADR-032: One-way door decision framework (WASM door)
- Reference TypeScript implementation: `/workspace/@alkdev/operations`
- Reference TypeScript pubsub: `/workspace/@alkdev/pubsub`
- aisdk (Rust port to be forked): `/workspace/aisdk`

311
docs/architecture/decisions/034-channels-wire-format.md Normal file
View File

@@ -0,0 +1,311 @@
# ADR-034: alknet-channels Wire Format — 8-Byte Chunk Header
## Status
Accepted (revised 2026-07-12: substrate simplification + stream_type
decomposition; **amended 2026-07-18 by ADR-035: wire format is 8 bytes,
not 9; `stream_type` removed from the channels header — see "Amendment
(ADR-035, 2026-07-18)" below**)
## Amendment (ADR-035, 2026-07-18)
The 9-byte chunk header is **amended to 8 bytes**:
`[channel_id:u32 BE][length:u32 BE][payload]`. The `stream_type` byte is
**removed** from the channels header — the channels layer has no
`stream_type` concept, not in its header, not in its code, not in its
mental model. The handler owns its sub-stream multiplexing on the
`BiStream` the channels layer gives it (per ADR-035, the channels-layer
consequence of ADR-009's `BiStream` handler leaf). What was the channels
header's `stream_type` byte is now the first byte of the payload, owned
by the handler's framing (TTY's 5-byte format, call's length-prefixed
JSON, tunnel's raw bytes, SSH's channel protocol).
The stream_type decomposition (unidirectional halves, mod 3 formula, 85
groups) is **removed from the channels layer**. The stream_type concept
survives in TTY's 5-byte format (ADR-052, amended by Phase 7), which the
channels layer carries transparently in its payload. The total header
for a TTY chunk inside channels is 13 bytes (8 channels + 5 TTY), not
9; the two length fields are close but not identical
(`ch_len = tty_len + 5`). This is the documented cost of clean
separation of concerns — see ADR-035 §"Consequences" for the full
cost/benefit.
The body below describes the **current** (9-byte) shape; the amendment
above is the operative decision. The 9-byte description is kept as the
historical context for the amendment. See ADR-035 for the resolution
rationale and the cross-ADR impacts.
## Context
`alknet-channels` is a multiplexing proxy: a `ProtocolHandler` on
`alknet/channels` that carries N logical channels, each with a different
ALPN, over transport stream(s). The wire format is the substrate that makes
this work.
Two prior formats inform this design:
1. **SSH's channel multiplexer (RFC 4254)** — `ChannelId(u32)` with
string-named types negotiated per channel, all traffic interleaved on one
encrypted transport stream.
2. **alknet-tty's chunk format (ADR-052)** — `[stream_type: u8][length: u32 be]
[payload]`, a fixed set of four sub-streams (stdin/stdout/stderr/control)
within one bidi stream. Validated by two POCs and in production code.
The channels format is the generalization: add a `channel_id: u32` prefix to
TTY's 5-byte header, turning a fixed 4-channel multiplexer into an arbitrary
N-channel multiplexer. The de-risk POC (28 tests) validated the core
mechanics.
### The substrate simplification
Different transports have different native multiplexing capabilities. QUIC
has native bidi streams; TCP+TLS does not. The initial framing of this ADR
treated the 9-byte header as "the in-line substrate format" — used when the
transport has no native multiplexing, with a separate substrate for QUIC
native streams. That framing adds complexity (two substrates, a
`channel_id`↔stream-ID mapping) for no benefit.
The simplification: **the 9-byte header is used in all substrates, on every
bidi stream that carries channels.** The `channel_id` in the header is the
logical correlation key. The transport's native multiplexing (when present)
is a performance optimization (independent flow-control windows per
stream), not a protocol change. The `ChannelsAdapter` reads the 9-byte
header off every bidi stream it accepts — on a transport with one stream
(in-line), the header demuxes N channels from that stream; on a transport
with N streams (QUIC native), each stream carries one logical channel and
the header provides `stream_type` and `channel_id` correlation. Same code
path, same wire format, same handler experience.
### The stream_type decomposition
The initial framing had `stream_type` 3 as "bidirectional" (control
messages). This is a design flaw: one stream_type both sides write to is not
properly multiplexed — it loses independent flow control, independent EOF,
and clean separation of concerns. The TTY crate's control channel already
exhibits this problem ("the control channel isn't actually bidirectional…
the adapter ignores Exit from the client" — phase-0 findings §Less
Straightforward Parts).
The fix: **every stream_type is unidirectional.** Bidirectionality is
achieved by having two stream_types (one write, one read), the same way
QUIC bidi streams are two unidirectional halves. Control becomes 3 (write,
client→server) and 4 (read, server→client), not one "bidirectional" 3.
## Decision
### Chunk header
```
[channel_id: u32 be][stream_type: u8][length: u32 be][payload bytes]
```
9 bytes of header. The `channel_id` is the addition over TTY's 5-byte
format; `stream_type` and `length` are identical to TTY's fields (ADR-052),
preserving the framing-disambiguation soundness property.
| field | width | meaning |
|-------|-------|---------|
| `channel_id` | u32 BE | The logical channel this chunk belongs to. Channel 0 is pre-negotiated as `alknet/call` (ADR-036). Channels 1..N are opened dynamically via `channel/open`. |
| `stream_type` | u8 | The unidirectional sub-stream within the channel. See "Stream types" below. |
| `length` | u32 BE | The payload length in bytes. 0 = EOF sentinel (same convention as TTY — ADR-052 §Sentinels). |
### `MAX_CHUNK_LEN`
`16 * 1024 * 1024` (16 MiB), matching TTY's cap (ADR-052 §5). A chunk with
`length > MAX_CHUNK_LEN` returns `ChunkTooLarge` and does not corrupt the
stream — the demux drops the chunk and continues. The header is always
exactly 9 bytes, so the demux can always resync by reading the next 9-byte
header.
### Stream types — unidirectional, grouped in threes
**Every stream_type is unidirectional.** Bidirectionality is two
stream_types (write + read), not one "bidirectional" stream_type. The
stream_types are grouped in threes:
| Group | stream_type | direction | purpose |
|-------|-------------|-----------|---------|
| Data | 0 | write (client→server) | data in (stdin equivalent) |
| | 1 | read (server→client) | data out (stdout equivalent) |
| | 2 | read (server→client) | data err (stderr equivalent, optional) |
| Control | 3 | write (client→server) | control in (ALPN-specific format) |
| | 4 | read (server→client) | control out (ALPN-specific format) |
| | 5 | read (server→client) | control err (optional) |
| Future | 6/7/8 | write/read/read | next group, same pattern |
| | 9/10/11 | write/read/read | next group |
| | ... | | |
**Formula:** `stream_type % 3 == 0` → write half (in), `stream_type % 3 ==
1` → read half (out), `stream_type % 3 == 2` → diagnostic read half (err).
256 values / 3 = 85 groups. The `u32` channel_id space (~4 billion channels
before wrap, ADR-040) combined with 85 stream_type groups is effectively
unlimited for the intended use cases.
**Why unidirectional:** each stream_type gets its own reassembly buffer, its
own flow control, its own EOF. The TTY control channel becomes *actually*
bidirectional because there are two unidirectional streams (3 in, 4 out),
not one stream both sides write to. This resolves the "control channel
isn't actually bidirectional" problem the TTY crate has today. The same
principle applies to any future channel type — control is two halves, not
one shared stream.
**Control payload format is ALPN-specific, not channels-enforced.** The
channels layer is blind to what stream_types 3/4/5 carry — it reassembles
bytes and delivers them to the handler. The TTY crate happens to use JSON
for its control channel (resize, signal, eof, exit) because its control
messages map cleanly to JSON; another ALPN might use a binary control
format. The channels layer does not mandate JSON on control stream_types.
This is the same ALPN-blindness principle that applies to the data
stream_types: the channels layer routes bytes, the handler interprets them.
### Per-ALPN stream_type sets
| ALPN | Active stream_types | Why |
|------|---------------------|-----|
| `alknet/call` (channel 0) | [0, 1] | call frames bidirectional via 0=in, 1=out |
| `alknet/tty` | [0, 1, 2, 3, 4] | data in/out/err + control in/out |
| `alknet/tunnel` | [0, 1] | data in/out only (no channels-layer control needed) |
| `alknet/ssh` | [0, 1] | SSH multiplexes internally, including its own control |
The active set is declared at `channel/open` time (ADR-037 `stream_types`
field) and fixed for the channel's lifetime. A tunnel that wants keepalive
could declare [0, 1, 3, 4].
### Substrate modes — same wire format, different stream counts
The 9-byte header is used in all substrates, on every bidi stream. The
difference between substrates is only **how many bidi streams the transport
yields**:
| Substrate | Transport | Streams | Header role |
|-----------|-----------|---------|--------------|
| In-line | TCP+TLS, WebTransport session, SSH `direct-tcpip` | 1 | Header demuxes N channels from that 1 stream |
| Native | QUIC (quinn/iroh) | N | Each stream carries 1 logical channel; header provides `stream_type` + `channel_id` correlation |
| Multi-connection | Any, N connections | N × M | Each connection is self-contained (own channel 0, own demux); header is per-connection |
The `ChannelsAdapter::handle` loop: `accept_bi()` → for each stream, read
the 9-byte header → route by `(channel_id, stream_type)` → reassemble. On
an in-line transport, `accept_bi()` yields once then `ConnectionClosed` —
the header does all the demux. On QUIC, `accept_bi()` yields repeatedly —
each stream is a channel, and the header provides `stream_type` and
`channel_id` correlation. Same code path, same wire format, same handler
experience.
**Why keep the header on QUIC when the stream already separates channels:**
- **`stream_type` decomposition.** TTY needs 5 sub-streams. On QUIC, one
stream per channel + the header carrying `stream_type` is simpler than 5
streams per channel and matches the in-line case's shape. Per-stream-type
flow control is handled at the reassembly buffer level (ADR-040)
regardless of substrate.
- **`channel_id` correlation.** The hub relay forwards a channel from the
browser leg to the spoke leg, mapping `browser_id ↔ spoke_id`. If
`channel_id` is in the header on every stream, the relay correlates by
reading the header — regardless of substrate. If `channel_id` lived only
in the `channel/open` response, the relay would need a stream-ID-to-
`channel_id` mapping per leg.
- **Uniform handler experience.** The handler receives a `Connection` and
calls `accept_bi()` or `into_sub_streams()`. It doesn't know or care
whether the substrate is in-line or native — the
`ChannelBidiStreamSource` wraps the reassembled stream either way.
The "bloat" (9 bytes/chunk on transports that have native multiplexing) is
the cost of uniformity. For the intended use cases (TTY, SSH, tunnels, call
operations), this is noise. The high-throughput escape hatch is
multi-connection, not stripping the header.
### Framing disambiguation (carried from ADR-052 §5)
Channel 0 is just another channel — its chunks have `channel_id=0` in the
header. Disambiguation between channel 0 (call protocol) and data channels
is by `channel_id`, not by a special first-byte trick. Within a channel,
`stream_type` 0 (write half) from the server is invalid, so `0x00` as the
first byte of a chunk payload from the server is unambiguous.
### Zero-length sentinel = EOF
A zero-length chunk is delivered as an empty `Bytes`, which the reassembled
stream interprets as EOF (same convention as TTY — ADR-052 §Sentinels). This
is the clean-shutdown signal for a `(channel_id, stream_type)` pair.
### Sync core / async shell split
The wire format's core is pure byte manipulation — `parse_header(&[u8; 9])
-> ChunkHeader` and `write_header(channel_id, stream_type, length, &mut
[u8; 9])`. No async, no platform dependencies. Compiles under
`wasm32-unknown-unknown` (validated by the POC). The async shell (demux/mux)
wraps this core with `read_exact`/`write_all` on the transport and `mpsc`
routing. This split keeps the WASM-compatible core separate from the
tokio-dependent shell.
## Consequences
**Positive:**
- One wire format across all substrates. The `ChannelsAdapter` code path is
the same regardless of transport; the transport's native multiplexing is a
performance optimization, not a protocol change.
- Every stream_type is unidirectional — proper multiplexing with independent
flow control and EOF per half. The TTY control channel becomes actually
bidirectional (3 in, 4 out), resolving the "not actually bidirectional"
flaw.
- The hub relay correlates by `channel_id` in the header, uniformly across
substrates — no stream-ID-to-`channel_id` mapping per leg.
- WASM-compatible by construction — the pure core has no platform deps.
- The framing-disambiguation property from ADR-052 carries forward unchanged.
**Negative:**
- 9 bytes per chunk on all substrates, including QUIC where the transport
already separates streams. This is 4 bytes more than using the QUIC stream
ID directly as the `channel_id`. For the intended use cases this is noise;
for high-throughput bulk transfer, the answer is multi-connection, not
stripping the header.
- All channels on one in-line connection share one transport stream's
flow-control window. A slow consumer on one channel can backpressure
others. Mitigated by bounded-buffer backpressure (ADR-040), not
eliminated. The native substrate (QUIC streams) avoids this — each
channel gets its own flow-control window. For high-throughput, use native
or multi-connection.
## Door type
**One-way.** The chunk header layout (`channel_id:u32 + stream_type:u8 +
length:u32`, 9 bytes) and the stream_type group assignments (0/1/2 = data, 3/4/5 =
control, `% 3` formula) are wire-format commitments. Changing them after
deployments exist requires a version migration.
**Amended by ADR-035 (2026-07-18):** the header layout is now
`channel_id:u32 + length:u32` (8 bytes); the `stream_type` byte and its
decomposition are removed from the channels layer. The one-way door is
re-cast (the channels crate is not yet implemented, so this is the right
time to cast it). See ADR-035 for the amended door-type discussion.
The `MAX_CHUNK_LEN` value (16 MiB) is a two-way-door implementation detail
within the one-way format.
## References
- **ADR-035**: channels pure channel multiplexing (amends this ADR —
wire format is 8 bytes, not 9; `stream_type` removed from the channels
header; the stream_type decomposition is removed from the channels
layer; the handler owns its sub-stream multiplexing on the `BiStream`)
- ADR-052: alknet-tty wire format (the 5-byte format this generalizes;
amended by ADR-077 — scoped to direct TTY; **re-amended by ADR-035 —
TTY always uses its 5-byte format, carried transparently in the
channels payload**)
- ADR-007: `Connection::from_stream` (the transport-agnostic Connection)
- ADR-008: `BidiStreamSource` trait (the extension point the channels
connection implements; its docstring already anticipated per-channel
streams)
- ADR-036: channel 0 pre-negotiated (now uses stream_types [0, 1])
- ADR-037: channel lifecycle operations (stream_types field examples
updated)
- ADR-038: ChannelBidiStreamSource (into_sub_streams returns unidirectional
handles)
- ADR-040: backpressure (bounded-buffer applies at reassembly regardless of
substrate)
- ADR-077: TTY inside channels (5 sub-streams; control properly
bidirectional via 3/4)
- `docs/research/alknet-channels/poc-summary.md` — the POC that validated
the format (28 tests, WASM compile check)
- `docs/research/alknet-channels/phase-0-findings.md` §The Wire Format, §Less
Straightforward Parts (the control-channel bidirectionality problem)

485
docs/architecture/decisions/035-channels-pure-channel-multiplexing.md Normal file
View File

@@ -0,0 +1,485 @@
# ADR-035: alknet-channels — Pure Channel Multiplexing (8-Byte Header, No `stream_type`)
## Status
Accepted (amends ADR-034 — wire format is 8 bytes, not 9, and the channels
layer has no `stream_type` concept; amends ADR-038 — `into_sub_streams()`
removed, `accept_bi` is the only accessor and yields one `BiStream` per
channel; reverses ADR-077 — TTY always uses its 5-byte format, the channels
layer carries it transparently in the payload)
## Context
ADR-034 committed the channels wire format as a 9-byte chunk header
(`[channel_id:u32][stream_type:u8][length:u32]`) — a 4-byte extension of
TTY's 5-byte format, with `stream_type` carried in the channels header
and decomposed into unidirectional halves (0/1/2 = data write/read/err,
3/4/5 = control write/read/err, `% 3` formula). ADR-038 added a second
accessor (`into_sub_streams()`) alongside `accept_bi` for handlers that
need typed sub-streams (TTY's stdin/stdout/stderr/ctrl-in/ctrl-out).
ADR-077 split TTY's wire format into two modes — direct (5-byte) and
inside-channels (the channels layer de-chunks and the adapter destructures
via `into_sub_streams()`).
The stream-unification research
(`docs/research/stream-unification/findings.md`, 2026-07-18) surfaced that
these three decisions share one root: the channels layer carries a
concept (`stream_type`) it doesn't own. The 9-byte header bakes TTY's
sub-stream multiplexing into the channels wire format. The
`into_sub_streams()` accessor exists because the channels layer reassembles
per-`stream_type` and needs to expose the result. The two-mode TTY design
exists because the channels layer's `stream_type` overlaps with TTY's own
`stream_type`. The mod 2/mod 3/mod 4 numbering question (settled as mod 3
in ADR-034 revised) was a symptom of this overlap — a numbering convention
for a concept the channels layer shouldn't carry.
### The structural question
The channels layer has two objectives in tension:
1. **"Pass a stream to/from any ALPN"** — every channel is a `BiStream`;
any handler gets `accept_bi()` and treats the channel as a duplex
stream. Uniform, transport-agnostic, recursive-composition-friendly.
2. **"Channels carry N sub-streams"** — a TTY channel carries
stdin/stdout/stderr/control; the handler destructures via
`into_sub_streams()`. Carries what the source produces.
The tension is real when a sub-stream is *unidirectional* (stderr). You
can't represent stderr as a `BiStream` without wasting the write half;
you can't make it a "third half" (mod 3) without breaking pair symmetry;
you can't make the channel a single `BiStream` without losing the
stdout/stderr distinction.
ADR-038's two-accessor design resolves this by making the "pass a stream
to/from any ALPN" objective *qualified* — it applies to single-stream
channels (tunnel, SSH, call), not multi-stream channels (TTY). The mod
2/mod 3/mod 4 numbering was a symptom of that qualified design.
### The resolution: channels layer is pure channel multiplexing
The channels layer's job is "one connection carries N channels, routed
by `channel_id`." It does not know about TTY's sub-streams, SSH's channel
protocol, or how call frames its JSON. Handlers own their sub-multiplexing
on the `BiStream` the channels layer gives them.
- **Every channel is a `BiStream`.** `accept_bi()` yields one `BiStream`
per channel (per ADR-009, already landed). No `into_sub_streams()`, no
second-class accessor.
- **Handlers sub-multiplex their `BiStream` however they want.** TTY
sub-demuxes `stream_type` from its `BiStream` (its 5-byte format). Tunnel
uses the `BiStream` as raw bytes. Call length-prefixes JSON. SSH runs
its own channel protocol. The channels layer carries the bytes
transparently.
- **The mod 2/mod 3/mod 4 question dissolves at the channels layer.** The
channels layer has no `stream_type` concept — not in its header, not in
its code, not in its mental model. `stream_type` is the inner layer's
framing byte, carried transparently.
- **The control channel is handler-internal.** TTY sub-demuxes control
from its io `BiStream` using its 5-byte format (`STREAM_CTRL_IN = 3`,
`STREAM_CTRL_OUT = 4` — ADR-052 amended by Phase 7). The channels layer
doesn't carry control. The "control isn't actually bidirectional" flaw
is fixed at the TTY layer, not the channels layer.
- **Recursive composition is literal.** A channel with ALPN
`alknet/channels` runs another channels demux on its `BiStream`. The
outer layer strips its 8-byte header; the inner layer parses its own
8-byte header from the payload. Each level is the same shape —
`BiStream → accept_bi → N BiStreams`.
### The wire format decision: 8 bytes
The channels wire format is **8 bytes**: `[channel_id:u32 BE][length:u32
BE]` followed by an opaque payload. The channels layer owns `channel_id`
and `length`; the payload is the handler's framing, carried transparently.
The 9-byte alternative (`[channel_id:u32][stream_type:u8][length:u32]`)
was considered and rejected. The 9-byte format puts `stream_type` in the
channels header, which means the channels layer carries a concept it
doesn't own. For TTY this composes cleanly (the 9-byte header is TTY's
5-byte header with `channel_id` prepended), but for non-TTY handlers
(tunnel, call, SSH) the `stream_type` byte is dead weight — the channels
layer carries a byte it doesn't understand, and the handler ignores a
byte in a header it doesn't control.
The 8-byte format is uniform across all handlers: the channels layer
carries `channel_id` + `length` + opaque payload. Every handler parses
its own framing from the payload. The cost is that TTY's `wire.rs` is
called from a payload buffer rather than directly from the wire, and the
total header for a TTY chunk is 13 bytes (8 channels + 5 TTY) instead of
9. The two length fields are close but not identical (`ch_len = tty_len +
5`); for typical TTY chunks (4 KiB+), the 5-byte overhead is ~0.1%, and
the trade is clean separation of concerns. See "Consequences" for the
full cost/benefit.
### The add/strip composition
Each layer has its own add/strip pair. The channels layer:
`add_channel_id(channel_id, payload_bytes) -> chunk` on write (prepends
the 8-byte header); `strip_channel_id(chunk) -> (channel_id,
payload_bytes)` on read (strips the 8-byte header, returns the payload).
The handler layer (e.g. TTY) parses its own framing from the payload
bytes per its existing `wire.rs`. The handler doesn't know or care that
a `channel_id` was stripped before it saw the bytes.
The composition is uniform — the same shape at every level. This is SSH's
model (layered headers, each layer strips its own at its boundary),
applied to channels. A `alknet/channels`-inside-`alknet/channels`
recursive composition is the outer layer stripping its 8-byte header, the
inner layer parsing its own 8-byte header from the payload — same code,
same shape, each level.
### Why this can land now
Three things changed since ADR-034/074/077 were accepted:
1. **ADR-009 landed `BiStream` as the handler leaf.** `accept_bi()`
returns a `BiStream` (a concrete `AsyncRead + AsyncWrite` newtype), not
a split `(SendStream, RecvStream)` pair. The join moves into core's
quinn/iroh/stream impls (once per source, invisible to handlers). This
ADR's "every channel is a `BiStream`" is the channels-layer
consequence of ADR-009's handler-leaf decision — the research-then-sync
pattern applied: ADR-009 settled the transport leaf, this ADR settles
the multiplexing layer above it.
2. **Phase 7 fixed the TTY control channel at the TTY layer.** The
`STREAM_CONTROL = 3` "bidirectional" flaw is fixed by splitting it into
`STREAM_CTRL_IN = 3` / `STREAM_CTRL_OUT = 4` — *inside TTY's 5-byte
format*, not at the channels layer. This removed the load-bearing
reason for the channels layer to carry `stream_type`: the control
bidirectionality fix is a TTY-internal concern, not a channels-layer
concern. ADR-077's two-mode TTY design was motivated by the channels
layer carrying control; with control moved inside TTY, the motivation
dissolves.
3. **No production constraint.** The develop branch is a rewrite of main
(pre-alpha). The channels crate doesn't exist yet (per ADR-044, it's
planned as `alknet-channels-core` + `alknet-channels-call`). The
decision is purely "what's cleanest," not "what's least disruptive."
The 9-byte POC validated the per-`channel_id`/`stream_type` routing
mechanism; the 8-byte spec update changes the header before
implementation begins.
### What this ADR does NOT decide
- **The add/strip API shape** (built into read/write vs. a separate
utility): the stream-unification research proposed `add_channel_id` /
`strip_channel_id` as standalone functions. Ideally the header is
built into the read/write path so the utility isn't needed at the
handler boundary — but there may be a generalized reason to expose it
(recursive composition, test helpers, the hub relay's `channel_id`
rewrite). The exact API shape is an implementation detail for the
channels crate, tracked as OQ-68. The *contract* — the channels layer
strips its 8-byte header on read and the handler parses its own framing
from the payload — is decided here; the *function surface* is not.
- **TTY's `wire.rs` adaptation:** TTY's `ChunkReader` currently reads from
an `AsyncRead`. Adapting it to read from a payload buffer (`&[u8]` or
`Cursor<Bytes>`) is a small, well-scoped change (the framing logic —
stream_type constants, length validation, control message parsing — is
unchanged). This is an implementation concern for the channels + TTY
integration, not an architecture decision.
- **Full channel-level flow-control windowing (OQ-56):** unchanged. The
bounded-buffer backpressure (ADR-040) is the v1 mechanism; full
windowing is an additive extension that doesn't change the wire
format. OQ-56 stays deferred(scope).
## Decision
### 1. The channels wire format is 8 bytes
```
[channel_id: u32 BE][length: u32 BE][payload bytes]
```
8 bytes of header, followed by `length` bytes of opaque payload. The
channels layer owns `channel_id` and `length`; the payload is the
handler's framing, carried transparently.
| field | offset | width | meaning |
|-------|--------|-------|---------|
| `channel_id` | 0 | 4 (BE) | The logical channel this chunk belongs to. Channel 0 is pre-negotiated as `alknet/call` (ADR-036). Channels 1..N are opened dynamically via `channel/open` (ADR-037). |
| `length` | 4 | 4 (BE) | The payload length in bytes. 0 = EOF sentinel. Max `MAX_CHUNK_LEN` (16 MiB, matching TTY's cap — ADR-052 §5). |
The `stream_type` byte is **removed** from the channels header. The
channels layer has no `stream_type` concept — not in its header, not in
its code, not in its mental model. What was the channels header's
`stream_type` byte is now the first byte of the payload, owned by the
handler's framing (TTY's 5-byte format, call's length-prefixed JSON,
tunnel's raw bytes, SSH's channel protocol).
This amends ADR-034: the wire format is 8 bytes, not 9; the
`stream_type` decomposition (mod 3, unidirectional halves, 85 groups) is
removed from the channels layer. The stream_type concept survives in
TTY's 5-byte format (ADR-052, amended by Phase 7), which the channels
layer carries transparently.
### 2. `into_sub_streams()` is removed; `accept_bi` is the only accessor
ADR-038's `into_sub_streams()` / `ChannelSubStreams` / `SubStreamHandle`
are removed. The channels layer exposes one accessor: `accept_bi()`,
which yields one `BiStream` per channel (per ADR-009). Every handler —
TTY, tunnel, SSH, call — receives a `Connection`, calls `accept_bi()`
once, gets a `BiStream`, and sub-multiplexes it however it wants.
This amends ADR-038: the two-accessor design (`accept_bi` for generic
handlers, `into_sub_streams` for typed handlers) collapses to one
accessor. The "typed handler path" (ADR-038's motivating case for TTY) is
replaced by TTY sub-demuxing its `BiStream` via its own 5-byte format —
the same code TTY runs in direct mode. ADR-038's yield-once `accept_bi`
contract is preserved; the `into_sub_streams()` accessor is the amended
part.
### 3. TTY always uses its 5-byte format; the channels layer carries it transparently
ADR-077's two-mode TTY design (direct vs inside-channels) is reversed.
TTY's 5-byte format (`[stream_type:u8][length:u32][payload]`, ADR-052) is
TTY's internal format, used in *both* direct mode and inside-channels
mode. The two modes differ only in *where the `BiStream` comes from*
(a top-level `alknet/tty` connection vs a `channel/open` with ALPN
`alknet/tty`), not in *how TTY parses it*. The same `wire.rs` code runs
in both modes.
When TTY is inside channels, the channels layer strips its 8-byte header
and hands TTY the payload bytes. TTY parses its 5-byte header from the
payload. The channels layer carries TTY's 5-byte chunks transparently
in its payload — no shared fields, no leaked abstraction, no
double-chunking concern (the 13-byte total header is 8 channels + 5
TTY, not 8 + 9; the channels `length` is always `tty_len + 5`).
This reverses ADR-077: the 5-byte format is NOT scoped to direct — it's
TTY's internal format, carried transparently in the channels payload.
The `channels` feature on `alknet-tty` becomes "run TTY's sub-demux on a
channels-backed `BiStream`" — the same code as direct mode, different
`BiStream` source. The control channel split (`STREAM_CTRL_IN` /
`STREAM_CTRL_OUT`, Phase 7) is TTY-internal; the channels layer doesn't
know about it.
### 4. The add/strip composition
The channels layer's read path strips the 8-byte header and hands the
payload to the handler. The write path prepends the 8-byte header
(`add_channel_id`) onto the handler's output. The handler never sees
the `channel_id`; it sees only its own framing (the payload bytes).
```
channels: [channel_id:u32 BE][length:u32 BE][payload]
= 8-byte header + opaque payload
8 bytes
TTY inside channels:
[channel_id:u32][ch_len:u32][stream_type:u8][tty_len:u32][payload]
4 bytes 4 bytes 1 byte 4 bytes N bytes
\_________ __________/ \_________ _____________/
| |
channels header TTY chunk (5+N bytes)
(8 bytes) carried as channels payload
```
The composition is uniform — the same shape at every level. A
`alknet/channels`-inside-`alknet/channels` recursive composition is the
outer layer stripping its 8-byte header, the inner layer parsing its own
8-byte header from the payload — same code, same shape, each level.
### 5. What does NOT change
- **ADR-009's `BiStream` leaf** — unchanged. This ADR is the
channels-layer consequence of ADR-009: `accept_bi` yields a `BiStream`,
handlers sub-multiplex it. The two ADRs compose (ADR-009 settles the
transport leaf; this ADR settles the multiplexing layer above it).
- **`ProtocolHandler` trait shape** (ADR-002) — unchanged. Handlers
receive a `Connection` and call `accept_bi()`.
- **Channel 0 pre-negotiated as `alknet/call`** (ADR-036) — unchanged.
Channel 0's chunks have `channel_id = 0` in the 8-byte header. The call
protocol's `EventEnvelope` framing is the payload; the channels layer
carries it transparently.
- **Channel lifecycle operations** (ADR-037) — unchanged. The four
operations (`channel/open`/`close`/`control`/`resources/subscribe`) and
their `direction` semantics are call-protocol operations on channel 0,
not channels-wire-format concerns.
- **`ChannelsAdapter` / `ChannelManager` split** (ADR-039) —
structurally unchanged. The demux loop reads 8-byte headers (not
9-byte); the `ChannelManager` is ALPN-blind, auth-blind,
transport-blind. The `stream_types` field on `channel/open` and
`ChannelState` is removed (the channels layer doesn't track
per-stream-type reassembly buffers; it tracks one reassembly buffer
per `channel_id`, yielding a `BiStream`).
- **Backpressure, channel limits, ID reuse** (ADR-040) — unchanged. The
bounded-buffer backpressure is per-`channel_id` (was per-
`(channel_id, stream_type)`; now per-`channel_id` since there's one
reassembly buffer per channel). The 256-channel cap, 1 MiB default,
and monotonic-ID-with-wrap strategy are unchanged.
- **Two-pump shutdown-on-completion** (ADR-078) — unchanged. Tunnel/SSH
handlers call `tokio::io::split(bidi)` for their two pump halves; the
shutdown-on-completion contract applies to the `ReadHalf` /
`WriteHalf` unchanged.
- **Hub relay** (ADR-042) — unchanged in contract. The hub translates
`channel/open` on channel 0 and byte-forwards data channels with
`channel_id` rewrite. The relay reads 8-byte headers (not 9-byte) and
rewrites the `channel_id` field (a 4-byte rewrite within the 8-byte
header, not a 9-byte header). The relay does not parse the payload.
- **`ChannelClient`** (ADR-043) — unchanged in API. `from_connection`
primary, `open_channel` returns a `Channel`. The `stream_types` field
on `open_channel` and `Channel` is removed (the channels layer doesn't
negotiate per-stream-type sets; the handler owns its sub-stream
multiplexing). The `channel:stream_type_unavailable` error code is
removed (the channels layer can't refuse a `stream_type` it doesn't
know about).
- **Sub-crate decomposition** (ADR-044) — unchanged. `channels-core`
(pure multiplexer, depends on `alknet-core` only) / `channels-call`
(channel 0 pre-negotiation + lifecycle op registrations, depends on
`channels-core` + `alknet-call`). The 8-byte wire format, demux/mux,
and `ChannelBidiStreamSource` are in `channels-core`; the call-protocol
coupling is in `channels-call`.
- **`BidiStreamSource` trait** (ADR-008) — unchanged in shape.
`ChannelBidiStreamSource` implements it; `accept_bi` yields a
`BiStream` (per ADR-009, already landed).
## Consequences
**Positive:**
- **Clean separation of concerns.** The channels layer has no
`stream_type` concept — not in its header, not in its code, not in its
mental model. The handler owns its framing entirely. This dissolves
the mod 2/mod 3/mod 4 question at the channels layer (there's nothing
to decompose) and fixes the "control isn't actually bidirectional" TTY
flaw at the TTY layer (where it lives, not the channels layer).
- **Uniform across all handlers.** Tunnel, call, SSH, and TTY all
receive the same shape: a `BiStream`. No handler gets a `stream_type`
byte it doesn't use; no handler needs a second accessor
(`into_sub_streams`) to reach its sub-streams. The channels layer's
API surface is `accept_bi -> BiStream`, period.
- **Recursive composition is literal.** A `alknet/channels` channel runs
another channels demux on its `BiStream`. The outer layer strips its
8-byte header; the inner layer parses its own 8-byte header from the
payload. Same code, same shape, each level. This is a property, not a
feature — the primary use case is one level of multiplexing, but the
add/strip composition makes the recursion cleaner than ADR-034's
group framing did.
- **The `into_sub_streams()` accessor and its consuming handler code are
removed.** This is a net simplification: one accessor, one handler
path, no downcast / extension trait / "two paths" ergonomics question
(which ADR-038 left as an implementation detail). The handler crate
destructures its `BiStream` via its own framing (TTY's 5-byte format),
not via a channels-crate-provided typed accessor.
- **TTY's `wire.rs` runs unchanged in both modes.** Direct mode and
inside-channels mode use the same code; only the `BiStream` source
differs. ADR-077's `drive_session_direct` / `drive_session_channels`
split collapses to one `drive_session` function. The `channels` feature
on `alknet-tty` becomes a thin wrapper that gets the `BiStream` from a
channels-backed `Connection` instead of a top-level one.
- **The channels layer is WASM-compatible by construction.** The 8-byte
header's core is pure byte manipulation (the sync core compiles under
`wasm32-unknown-unknown`, validated by the POC). The 8-byte format is
simpler than the 9-byte (one fewer field to parse), strengthening the
WASM-clean property.
**Negative:**
- **5 extra bytes per TTY chunk.** The total header for a TTY chunk
inside channels is 13 bytes (8 channels + 5 TTY), not 9. The two length
fields are close but not identical (`ch_len = tty_len + 5`). For
typical TTY chunks (4 KiB+), this is ~0.1% overhead. For extreme
multiplexing scenarios, the clean separation is worth the trade-off;
for high-throughput bulk transfer, the escape hatch is multi-connection
(one channels connection per leg), not stripping the header. This is
the documented cost of the clean separation; the alternative (9-byte
header with `stream_type` in the channels layer) carries a concept the
channels layer doesn't own, which is the root cause this ADR addresses.
- **TTY's `wire.rs` needs a small adaptation.** `ChunkReader` currently
reads from an `AsyncRead` (the transport stream). Inside channels, it
reads from a payload buffer (`&[u8]` or `Cursor<Bytes>`) — the bytes
the channels layer handed it after stripping its 8-byte header. The
framing logic (stream_type constants, length validation, control
message parsing) is unchanged. This is a bounded, well-scoped
implementation change, not an architecture change. The same adaptation
applies to any handler that parses its own framing from a payload
buffer (call's `EventEnvelope` framing already reads from a buffer;
tunnel and SSH don't parse the payload, so no adaptation).
- **`channel/open` loses the `stream_types` field.** ADR-037's
`channel/open` input included `stream_types: [u8]` (the active sub-stream
set) and the response echoed the negotiated set. Under this ADR, the
channels layer doesn't negotiate sub-stream sets — the handler owns
its sub-stream multiplexing. The `stream_types` field is removed from
`channel/open` (and from the `channel:stream_type_unavailable` error
code). The `alpn` and `params` fields remain; the handler's sub-stream
set is implicit in its ALPN's wire format. This is a small wire-format
change to `channel/open` (one field removed); since the channels crate
isn't implemented yet, there's no migration cost.
- **`ChannelState.streams: HashMap<u8, ReassemblyBuffer>` becomes
`ChannelState.reassembly: ReassemblyBuffer` (one per channel, not per
`(channel_id, stream_type)`).** This is an internal simplification
(fewer reassembly buffers, simpler drain logic) but is an
implementation change, not an architecture one. The bounded-buffer
backpressure (ADR-040) is per-`channel_id` now, not per-
`(channel_id, stream_type)` — the 1 MiB default and the 256-channel cap
are unchanged; the per-channel memory ceiling is 1 MiB (was up to 5 MiB
for a TTY channel with 5 active stream_types). This is a net
improvement (lower memory ceiling per channel), not a regression.
## Door type
**One-way (wire format, accessor removal, two-mode reversal).** The 8-byte
chunk header layout (`channel_id:u32 + length:u32`), the removal of
`stream_type` from the channels header, and the removal of
`into_sub_streams()` are wire-format and API commitments. Changing them
after the channels crate is implemented and handlers are written against
them requires a version migration. Since the channels crate doesn't exist
yet, the one-way door is being cast now, before implementation — the
right time to cast a one-way door.
The reversal of ADR-077 (TTY always uses its 5-byte format) is one-way in
the same sense: once TTY's `wire.rs` runs in both modes (direct and
inside-channels), re-introducing a separate inside-channels mode would be
a rewrite of TTY's session driver. The trade is one unified session
driver now vs. two-mode maintenance forever.
The add/strip API shape (OQ-68) is a **two-way door** — whether the
header add/strip is built into the read/write path or exposed as a
standalone utility is an implementation detail that can change without
breaking the wire format or the handler contract.
## References
- ADR-034: channels wire format (amended — wire format is 8 bytes, not
9; `stream_type` removed from the channels header; the stream_type
decomposition is removed from the channels layer)
- ADR-038: ChannelBidiStreamSource (amended — `into_sub_streams()`
removed; `accept_bi` is the only accessor, yields one `BiStream` per
channel)
- ADR-077: TTY inside channels (reversed — TTY always uses its 5-byte
format; the channels layer carries it transparently in the payload;
the two-mode design is preserved but differs only in `BiStream`
source, not in parsing)
- ADR-009: `BiStream` as the handler leaf (the transport-leaf layer this
ADR builds on — `accept_bi` returns `BiStream`; `from_bidi` is the only
public stream constructor)
- ADR-008: `BidiStreamSource` trait (the extension point
`ChannelBidiStreamSource` implements; `accept_bi` yields `BiStream`)
- ADR-036: channel 0 pre-negotiated `alknet/call` (unchanged — channel 0's
chunks have `channel_id = 0` in the 8-byte header; the call protocol's
framing is the payload)
- ADR-037: channel lifecycle operations (amended — `stream_types` field
removed from `channel/open`; `channel:stream_type_unavailable` error
code removed)
- ADR-039: `ChannelsAdapter` and `ChannelManager` (structurally
unchanged — demux reads 8-byte headers; one reassembly buffer per
channel)
- ADR-040: backpressure, channel limits, ID reuse (unchanged —
bounded-buffer is per-`channel_id`; 256-channel cap, 1 MiB default,
monotonic IDs)
- ADR-078: two-pump shutdown-on-completion (unchanged — the contract
applies to `tokio::io::split(bidi)` halves)
- ADR-042: hub relay (unchanged in contract — 8-byte header, 4-byte
`channel_id` rewrite, payload byte-forwarded)
- ADR-043: `ChannelClient` (amended — `stream_types` field removed from
`open_channel` and `Channel`)
- ADR-044: sub-crate decomposition (unchanged — 8-byte wire format in
`channels-core`; call-protocol coupling in `channels-call`)
- ADR-052: alknet-tty wire format (the 5-byte format carried
transparently in the channels payload; the control channel split
from Phase 7 is TTY-internal)
- `docs/research/stream-unification/findings.md` — the research that
surfaced the structural question and the resolution this ADR commits
- `docs/research/alknet-crate-extraction/findings.md` Phase 8 — the
spec-cleanup phase this ADR is the substance of
- `/workspace/alknet-channels-poc/` — the POC that validated the
per-`channel_id`/`stream_type` routing mechanism (the mechanism
supports any convention; this ADR says the channels layer doesn't have
a convention, the handler does)

156
docs/architecture/decisions/036-channel-0-pre-negotiated-call.md Normal file
View File

@@ -0,0 +1,156 @@
# ADR-036: Channel 0 Is Pre-Negotiated `alknet/call`
## Status
Accepted (amended 2026-07-18 by ADR-035 — channel 0's `stream_types`
field is removed; the channels layer has no `stream_type` concept; the
call protocol's `EventEnvelope` framing is the channels payload, carried
transparently — see "Amendment (ADR-035, 2026-07-18)" below)
## Amendment (ADR-035, 2026-07-18)
Channel 0's `stream_types` field (the `[0, 1]` active set) is **removed**.
The channels layer has no `stream_type` concept (ADR-035) — it carries the
call protocol's `EventEnvelope` framing (ADR-014) transparently in the
8-byte header's payload. The call protocol's bidirectionality (client
writes requests, server writes responses) is a call-protocol concern,
not a channels-layer concern; the channels layer routes by `channel_id`
only and yields a `BiStream` to the `CallAdapter`. The `CallAdapter`'s
`accept_bi()` returns one `BiStream` (per ADR-009); the call protocol
reads/writes `EventEnvelope` frames on it, exactly as on a top-level
`alknet/call` connection.
The body below describes the **original** (with `stream_types`) shape;
the amendment above is the operative decision. See ADR-035 for the
resolution rationale and the cross-ADR impacts.
## Context
A channels connection carries N logical channels. One of them must carry the
call protocol — the JSON-RPC layer that orchestrates channel lifecycle
(`channel/open`, `channel/close`, `channel/control`, `channel/resources`).
The question is how channel 0 relates to the call protocol: is it a special
"control plane" with its own framing, or is it just `alknet/call` pre-
negotiated?
The phase-0 research (`docs/research/alknet-channels/phase-0-findings.md`
§DP-2) recommends channel 0 is `alknet/call` pre-negotiated — no special
framing, no separate control-plane wire format. The call protocol runs on
channel 0 exactly as it runs on a top-level `alknet/call` QUIC connection.
This matters because the alternative (a special control plane) would mean
the channels layer has its own JSON protocol for channel lifecycle, parallel
to and duplicating the call protocol's `OperationRegistry`, `AccessControl`,
`OperationContext`, and `forwarded_for` machinery. That duplication is the
"re-implement every protocol's framing per transport" problem the hub
motivation (§Hub Motivation) identifies as the thing channels exists to
collapse.
## Decision
**Channel 0 is `alknet/call`, pre-negotiated.** Both sides of a channels
connection know that `channel_id = 0` is routed to the `CallAdapter` without
an explicit `channel/open` exchange. The `CallAdapter` receives a
`Connection` backed by channel-0 chunk reassembly and dispatches operations
exactly as it does on a top-level `alknet/call` connection.
### What this means concretely
1. **Channel 0 uses the same 9-byte chunk format as every other channel**
(ADR-034). Its chunks have `channel_id = 0` in the header. No special
first-byte trick, no separate framing.
2. **The `CallAdapter` is unchanged.** It receives a `Connection`, calls
`accept_bi()`, gets one bidi stream (the channel-0 reassembled stream),
and runs its dispatch loop. `EventEnvelope` frames ride on `stream_type =
0` of channel 0. The `CallAdapter` does not know it is inside a channels
connection.
3. **Channel lifecycle operations are call operations.** `channel/open`,
`channel/close`, `channel/control`, `channel/resources` are registered on
the call protocol's `OperationRegistry` at assembly time (ADR-037). They
are dispatched through the existing `OperationContext` (identity, scopes,
capabilities, ownership, `forwarded_for`), gated by the existing
`AccessControl::check`. No new auth machinery, no new framing, no
protocol version bump.
4. **Channel 0 is allocated at `ChannelsAdapter::handle` entry.** The
`ChannelsAdapter` constructs channel 0's reassembly buffers, wraps them
as a `Connection` (via `Connection::from_source` with a
`ChannelBidiStreamSource` — ADR-008/074), and hands that `Connection` to
the `CallAdapter` — exactly as if `alknet/call` had been the top-level
ALPN. The `CallAdapter` is looked up in the same `HandlerRegistry` as
every other ALPN.
### Channel 0's stream_type usage
| stream_type | direction | purpose |
|-------------|-----------|---------|
| 0 | write (client→server) | `EventEnvelope` frames from the client (call.requested, call.aborted) |
| 1 | read (server→client) | `EventEnvelope` frames from the server (call.responded, call.completed, call.error) |
Channel 0 uses stream_types [0, 1] — the call protocol is bidirectional via
two unidirectional halves, the same way every channel type works
(ADR-034 §stream_type decomposition). The call protocol's `(SendStream,
RecvStream)` pair maps directly: `SendStream` backed by stream_type 0,
`RecvStream` backed by stream_type 1. Both sides write to their write half
and read from their read half — no shared stream_type both sides write to.
The call protocol is JSON-only and single-stream by design (ADR-014).
stream_types 2-255 on channel 0 are reserved for future call-protocol
sub-streams.
## Consequences
**Positive:**
- No control-plane duplication. The channels layer reuses the call protocol's
`OperationRegistry`, `AccessControl`, `OperationContext`, `forwarded_for`,
and `StreamingHandler` (ADR-021) machinery verbatim. Channel lifecycle is
just another class of call operations.
- The `CallAdapter` is transport-agnostic by construction — it works
identically whether the `Connection` is a top-level QUIC stream or a
channels-reassembled channel-0 stream. This is the "streams are streams"
insight made concrete.
- `channel/resources/subscribe` (ADR-037) is a `Subscription` operation on
channel 0, using the already-implemented `StreamingHandler` /
`invoke_streaming` path (ADR-021). The resource registry is a live view,
not a polled snapshot.
- Auth is inherited: `channel/open` goes through `AccessControl::check`
exactly like any other call operation. The channels layer does not re-
implement auth.
**Negative:**
- Channel 0 is a single point of orchestration. If channel 0's `CallAdapter`
hangs, no new channels can be opened. This is the same property as the call
protocol today (one dispatch loop per connection) and is not a new
vulnerability.
- The call protocol's JSON-only nature means channel lifecycle operations
are JSON. For high-frequency control (e.g., per-keystroke resize), this is
more overhead than a binary control frame. The division (ADR-037 §DP-4)
handles this: `stream_type 3` on the data channel for data-ordered control,
call operations for lifecycle and infrequent control.
## Door type
**One-way.** Channel 0's role as `alknet/call` pre-negotiated is a wire-
format and protocol-structure commitment. Changing it after deployments
exist (e.g., to a special control plane) requires a version migration and
re-architecting the channel lifecycle operations. The reservation of
`stream_type` 1-255 on channel 0 is a two-way-door detail (they're currently
unused; assigning them is additive).
## References
- ADR-034: channels wire format (the 8-byte chunk header channel 0 uses,
as amended by ADR-035)
- ADR-035: channels pure channel multiplexing (amends this ADR —
channel 0's `stream_types` field removed; the call protocol's framing
is the channels payload, carried transparently)
- ADR-037: channel lifecycle operations (registered on channel 0's
`OperationRegistry`)
- ADR-014: irpc never integrated — hand-rolled EventEnvelope framing (the
call protocol channel 0 carries)
- ADR-021: StreamingHandler for subscriptions (the machinery
`channel/resources/subscribe` uses)
- ADR-008: BidiStreamSource trait (the `Connection` extension point)
- `docs/research/alknet-channels/phase-0-findings.md` §DP-2, §Channel 0

326
docs/architecture/decisions/037-channel-lifecycle-operations.md Normal file
View File

@@ -0,0 +1,326 @@
# ADR-037: Channel Lifecycle Operations on the Call Protocol
## Status
Accepted (amended 2026-07-18 by ADR-035 — `stream_types` field removed
from `channel/open`; `stream_type` field removed from `channel/control`;
`channel:stream_type_unavailable` error code removed; the channels layer
has no `stream_type` concept — see "Amendment (ADR-035, 2026-07-18)"
below)
## Amendment (ADR-035, 2026-07-18)
The `stream_types` field is **removed** from `channel/open`'s input and
output. The `stream_type` field is **removed** from `channel/control`'s
input. The `channel:stream_type_unavailable` error code is **removed**.
The channels layer has no `stream_type` concept (ADR-035) — the handler
owns its sub-stream multiplexing on the `BiStream` it receives. The
handler's sub-stream set is implicit in its ALPN's wire format (e.g.,
TTY's 5-byte format declares its own `stream_type` set internally; the
channels layer carries the bytes transparently).
The body below describes the **original** (with `stream_types`) shape;
the amendment above is the operative decision. See ADR-035 for the
resolution rationale and the cross-ADR impacts.
## Context
Channel lifecycle — open, close, control, resource discovery — must be
orchestrated somehow. The phase-0 research (`docs/research/alknet-channels/
phase-0-findings.md` §Channel Open Negotiation, §DP-4) established that
channel lifecycle is orchestrated by the call protocol on channel 0
(ADR-036). This ADR pins the exact operation shapes, the `direction` field
semantics, the control-message division, and the resource-discovery model.
Three things from the research needed real decisions, not hedges:
1. **Resource discovery: poll vs subscribe (OQ-CH-08).** The research
recommended "poll for v1, add subscription if staleness bites." This is a
hedge: the call protocol already has `StreamingHandler` /
`invoke_streaming` (ADR-021, implemented and tested), and the first
consumer (the hub aggregating worker resources) needs live updates. Polling
would be built, immediately found insufficient, and reworked. This ADR
commits to subscribe from day one.
2. **The `direction` field and who writes first (OQ-CH-09).** The research
said "ALPN-specific and probably doesn't need a channels-layer rule…
needs to be pinned down." That IS the rule: the channels layer declares
write-order is ALPN-specific (determined by who is the ALPN-server), not
channels-enforced. This ADR pins which side is the ALPN-server for each
`direction` value.
3. **Control messages: call ops vs stream_type 3 (DP-4).** The research
recommended "both, with clear division." This ADR pins the division.
## Decision
### Four operations on channel 0's `OperationRegistry`
Registered at assembly time by the channels crate (via `ChannelOperations::
register_on(&mut call_registry)`). All four go through the existing
`OperationContext` / `AccessControl::check` path — no new auth machinery.
#### `channel/open` — open a data channel
Request (`call.requested` on channel 0):
```json
{
"operation": "channel/open",
"input": {
"alpn": "alknet/tty",
"stream_types": [0, 1, 2, 3, 4],
"params": { "backend": "docker", "cmd": ["bash"], "container": "abc123" },
"direction": "initiator-to-responder"
}
}
```
| field | type | meaning |
|-------|------|---------|
| `alpn` | string | The ALPN the channel will carry. The responder looks this up in its `HandlerRegistry`. |
| `stream_types` | `[u8]` | Which sub-stream types this channel will use. E.g. `[0,1,2,3,4]` for TTY (data in/out/err + control in/out), `[0,1]` for a tunnel, `[0,1]` for channel 0 (call frames). See ADR-034 §stream_type decomposition. |
| `params` | object | ALPN-specific parameters. For `alknet/tty` this is the `NegotiateRequest`. For `alknet/tunnel` this is the target resource. The channels layer does not interpret `params`. |
| `direction` | string | `initiator-to-responder` or `responder-to-initiator`. See "Direction semantics" below. |
Response (`call.responded`):
```json
{
"output": {
"channel_id": 7,
"stream_types": [0, 1, 2, 3, 4]
}
}
```
| field | type | meaning |
|-------|------|---------|
| `channel_id` | u32 | The server-assigned channel ID (DP-1: server-assigned). Both sides route chunks with this ID to the new channel. |
| `stream_types` | `[u8]` | The *negotiated* set — the responder may narrow the initiator's requested set (e.g., refuse stderr). The intersection of requested and supported. |
**Channel ID allocation (DP-1): server-assigned.** The responder allocates
the `channel_id` via a monotonic `AtomicU32` (`next_id.fetch_add(1, Relaxed)`)
and returns it in the response. One round-trip before data flows — the same
round-trip the call protocol makes for every operation. All current channel
types (TTY, tunnel, SSH) already require a negotiation round-trip, so the
open round-trip is not additive latency.
**Error codes** (new `CallError.code` strings, not new framing):
| code | meaning | retryable |
|------|---------|-----------|
| `channel:unknown_alpn` | ALPN not in responder's `HandlerRegistry` | false |
| `channel:forbidden` | `AccessControl::check` denied the open | false |
| `channel:allocation_failed` | Handler allocate failed (e.g., backend couldn't start) | true (often transient) |
| `channel:invalid_params` | `params` JSON didn't satisfy the ALPN's expectations | false |
| `channel:too_many_channels` | Per-connection channel limit hit (ADR-040) | false |
| `channel:stream_type_unavailable` | Responder can't provide a requested `stream_type` | false |
#### `channel/close` — tear down a channel
```json
{
"operation": "channel/close",
"input": { "channel_id": 7, "reason": "exit" }
}
```
The responder (the side that didn't send the close) drains its reassembled
streams for `channel_id`, signals EOF to the handler, and returns
`{ "closed": true }`. The `channel_id` is now eligible for reuse after the
drain completes (ADR-040 §channel-id-reuse). `reason` is free-form for
observability — not semantically required.
**Exit-chunk-before-close ordering (generalizes ADR-055):** the channel's
data chunks must be written and flushed before the `channel/close` operation
is sent on channel 0. This is a wire-level invariant: the side closing must
observe the data-channel pump complete before issuing the call operation.
For TTY this is the exit-chunk-is-last invariant (ADR-055) carried forward;
for tunnels it is the last data byte before close. The channels layer's
close handler observes the pump completion; the call operation is issued
after. This is REQ-CH-06.
#### `channel/control` — out-of-band control on channel 0
For control that doesn't need ordering relative to data (resize, signal,
keepalive):
```json
{
"operation": "channel/control",
"input": {
"channel_id": 7,
"stream_type": 3,
"message": { "type": "resize", "cols": 80, "rows": 24 }
}
}
```
The channels layer routes `message` to the handler's control handle for
`channel_id`. The `message` JSON is ALPN-specific; the channels layer does
not interpret it.
#### `channel/resources/subscribe` — live resource discovery
**This is a `Subscription` operation (ADR-021), not a polled Query.** The
research's "poll for v1, add subscription if staleness bites" is a hedge that
would cause rework — the `StreamingHandler` / `invoke_streaming` machinery
exists and is tested, and the hub consumer needs live updates when workers
connect/disconnect or containers start/stop.
```json
{
"operation": "channel/resources/subscribe",
"input": {}
}
```
The responder registers a `StreamingHandler` that emits a
`ResponseEnvelope` whenever the resource set changes. Each event:
```json
{
"output": {
"resources": [
{
"alpn": "alknet/tty",
"backends": ["docker", "local"],
"access": { "required_scopes": ["tty:open"] }
},
{
"alpn": "alknet/tunnel",
"targets": ["container:*", "service:postgres"],
"access": { "required_scopes_any": ["tunnel:open", "admin"] }
}
]
}
}
```
| field | type | meaning |
|-------|------|---------|
| `alpn` | string | The ALPN this side accepts `channel/open` for. |
| `backends` / `targets` | `[string]` | ALPN-specific enumeration of what's available. The channels layer doesn't interpret these; they're for the initiator to know what `params` to send. |
| `access` | object | A preview of the `AccessControl` that `channel/open` will check. Advisory — lets the initiator fail fast. The real check happens on `channel/open`. |
The stream emits an initial snapshot immediately, then subsequent events on
any change (worker connects/disconnects, container starts/stops, resource
exposed/withdrawn). The stream is long-lived; the subscriber cancels by
dropping the subscription (ADR-020 abort cascade applies). This is the
resource-discovery analogue of `services/list`, but live — matching the
bidirectional symmetry of the operation overlay.
A `channel/resources` (non-subscribe, Query) operation is NOT provided. The
subscription's initial snapshot serves the poll use case (subscribe, read
the first event, cancel). Providing both would be redundant and would
pressure consumers toward the stale-poll path.
### Direction semantics (OQ-CH-09 — pinned)
Channel open is **bidirectional** — either side can initiate. The
`direction` field determines who is the ALPN-server (allocates the handler,
writes the negotiation response) vs the ALPN-client (writes the first
request).
| `direction` | Initiator role | Responder role | Who writes first |
|-------------|----------------|----------------|-------------------|
| `initiator-to-responder` | ALPN-client | ALPN-server | Initiator writes first (the request data); responder's handler is the server side of the ALPN. The common case: "open me a TTY on your docker container." |
| `responder-to-initiator` | ALPN-server | ALPN-client | Responder writes first (the negotiation response / server greeting); initiator's handler is the client side. The "worker exposes, hub consumes" case: the worker initiates the open to make itself available; the hub is the client that connects to the exposed resource. |
**The channels layer does not enforce write order.** Write order is
ALPN-specific, determined by which side is the ALPN-server (per the table
above). The channels layer's job is to route chunks; the handlers negotiate
who writes first via their ALPN's `params` contract. This is the rule the
research asked for: "the channels layer declares write-order is ALPN-
specific, not channels-enforced."
**`channel_id` allocation is always by the responder** (DP-1), regardless of
`direction`. The responder is the side that receives the `channel/open` call
operation; it allocates the ID and returns it. In the `responder-to-
initiator` case, the initiator (worker) sends the `channel/open`, so the
responder (hub) allocates the ID — even though the worker is the ALPN-
server for the channel's data. This keeps ID allocation in one place (the
`channel/open` responder) and avoids the collision-prone client-assigned
alternative.
### Control-message division (DP-4 — pinned)
| Control path | When | Examples |
|--------------|------|----------|
| Call operations on channel 0 (`channel/control`, `channel/close`) | Control that doesn't need ordering relative to data, or lifecycle events | resize, signal, keepalive, close |
| `stream_type 3` (write, client→server) and `stream_type 4` (read, server→client) chunks on the data channel | Control that MUST be ordered relative to data | EOF before exit, flush before close |
The TTY crate's exit-chunk-is-last invariant (ADR-055) is the canonical
example of data-ordered control — the exit message rides on `stream_type 4`
(read, server→client) because it must arrive after the last data on
`stream_type 1`, guaranteed by per-stream_type ordering. The client's EOF
signal rides on `stream_type 3` (write, client→server), ordered after the
last data on `stream_type 0`. The `channel/close` operation that follows is
on channel 0 and is ordered after the data pump completes (REQ-CH-06).
**Every stream_type is unidirectional** (ADR-034 §stream_type decomposition).
Control is bidirectional via two halves (3 in, 4 out), not one shared
stream_type both sides write to. This resolves the TTY control channel's
"not actually bidirectional" flaw.
## Consequences
**Positive:**
- Channel lifecycle reuses the call protocol's `OperationRegistry`,
`AccessControl`, `OperationContext`, `forwarded_for`, and
`StreamingHandler` verbatim. Zero new auth, zero new framing.
- `channel/resources/subscribe` gives the hub a live view of worker
resources — no polling, no staleness, no rework when the first consumer
needs subscriptions.
- The `direction` field makes bidirectional open explicit and pins who is
the ALPN-server, resolving the "who writes first" ambiguity without
channels-layer write-order enforcement.
- The control-message division (call ops vs stream_type 3) handles both
lifecycle control (infrequent, benefits from auth/observability) and
data-ordered control (frequent, needs ordering) without duplicating
machinery.
**Negative:**
- Four new operation names in the `OperationRegistry`. The registry already
handles namespaced operations (`docker/container/list`, etc.); these are
in the `channel/` namespace. No registry changes needed.
- `channel/resources/subscribe` is a long-lived `Subscription` stream per
interested peer. This is the same cost as any other subscription (ADR-021);
the hub holds one per connected peer. Acceptable.
- The `direction` field adds one field to the `channel/open` input. It is
required (no default) — the initiator must state its intent. This is a
one-way-door wire-format field (removing it would break the bidirectional
open contract).
## Door type
**One-way.** The four operation names (`channel/open`, `channel/close`,
`channel/control`, `channel/resources/subscribe`), their input/output
schemas, and the `direction` field's semantics are wire-format commitments.
Changing them after deployments exist requires a protocol version migration.
The `reason` field on `channel/close` (free-form, observability-only) is a
two-way-door detail.
The decision to use `Subscription` for resource discovery (not `Query`) is
one-way: consumers will depend on the live stream, and the Decision section
committed to Subscribe-only (a `Query` variant is NOT provided — the
subscription's initial snapshot serves the poll use case). The
`Handler` / `StreamingHandler` / `HandlerKind` API surface (ADR-021) is
the underlying one-way commitment.
## References
- ADR-034: channels wire format (amended by ADR-035 — 8-byte header, no
`stream_type`)
- ADR-035: channels pure channel multiplexing (amends this ADR —
`stream_types` field removed from `channel/open`; `stream_type` field
removed from `channel/control`; handler owns sub-stream multiplexing)
- ADR-036: channel 0 is pre-negotiated `alknet/call`
- ADR-021: StreamingHandler for subscriptions (the machinery
`channel/resources/subscribe` uses — implemented and tested)
- ADR-020: abort cascade (subscription cancellation)
- ADR-026: forwarded-for identity (the auth chain for hub-relayed opens)
- ADR-055: exit-chunk-is-last (the TTY invariant generalized by REQ-CH-06)
- `docs/research/alknet-channels/phase-0-findings.md` §Channel Open
Negotiation, §DP-4, §OQ-CH-08, §OQ-CH-09

245
docs/architecture/decisions/038-channelconnection-bidistreamsource.md Normal file
View File

@@ -0,0 +1,245 @@
# ADR-038: ChannelConnection — BidiStreamSource over Chunk Reassembly
## Status
Accepted (**amended 2026-07-18 by ADR-035: `into_sub_streams()` removed;
`accept_bi` is the only accessor, yields one `BiStream` per channel —
see "Amendment (ADR-035, 2026-07-18)" below**)
## Amendment (ADR-035, 2026-07-18)
`into_sub_streams()`, `ChannelSubStreams`, and `SubStreamHandle` are
**removed**. The channels layer exposes one accessor: `accept_bi()`,
which yields one `BiStream` per channel (per ADR-009, already landed).
Every handler — TTY, tunnel, SSH, call — receives a `Connection`, calls
`accept_bi()` once, gets a `BiStream`, and sub-multiplexes it however it
wants. The "typed handler path" (this ADR's motivating case for TTY) is
replaced by TTY sub-demuxing its `BiStream` via its own 5-byte format
(ADR-052) — the same code TTY runs in direct mode. The two-accessor
design (`accept_bi` for generic handlers, `into_sub_streams` for typed
handlers) collapses to one accessor.
The body below describes the **original** (two-accessor) shape; the
amendment above is the operative decision. The two-accessor description
is kept as the historical context for the amendment. See ADR-035 for
the resolution rationale (the channels layer has no `stream_type`
concept; the handler owns its sub-stream multiplexing) and the
cross-ADR impacts.
## Context
ADR-008 landed the `BidiStreamSource` trait and `Connection::from_source`
extension point so downstream crates can implement their own connection
shapes without a core edit. The channels crate is the first downstream
consumer: a channels connection carries N logical channels, each a
bidirectional byte stream presented to a `ProtocolHandler` as a `Connection`.
The phase-0 research (`docs/research/alknet-channels/phase-0-findings.md`
§The Channel Connection Abstraction, §OQ-CH-10) proposed that
`ChannelConnection` *implements* the `Connection` interface (for recursion
and generic handlers) **and** can be destructured into typed sub-stream
handles (`TtyChannel { stdin, stdout, stderr, control }`). The research
recommended "the TTY crate destructures; channels exposes `(channel_id,
stream_type) → (SendStream, RecvStream)` accessors" but did not pin the
exact API shape. This ADR pins it.
The de-risk POC (`docs/research/alknet-channels/poc-summary.md` §POC Target
2) validated that `Connection::from_stream` (the yield-once path) is
sufficient — an echo `ProtocolHandler` runs through the full
demux→Connection→handler→mux path with zero channels-layer awareness. But
the POC deliberately used the yield-once path (one `Connection` per channel)
rather than the N-stream `ChannelBidiStreamSource` shape. This ADR commits
to the N-stream shape that ADR-008 unblocked.
## Decision
### `ChannelBidiStreamSource` implements `BidiStreamSource`
The channels crate defines a `ChannelBidiStreamSource` that implements
`alknet-core`'s `BidiStreamSource` trait (ADR-008). One
`ChannelBidiStreamSource` instance represents **one channel** (not the
whole channels connection). Its `accept_bi()` yields one bidi stream — the
`(stream_type 0, stream_type 1)` pair for that channel — then returns
`ConnectionClosed` on subsequent calls (yield-once per channel, matching
the POC's validated shape).
```rust
// In alknet-channels:
pub struct ChannelBidiStreamSource {
// The reassembly buffers for this channel's active stream_types,
// plus the mux handle for writing back onto the transport.
// Constructed by ChannelManager::build_channel_connection (ADR-039).
...
}
#[async_trait]
impl BidiStreamSource for ChannelBidiStreamSource {
async fn accept_bi(&self) -> Result<(SendStream, RecvStream), StreamError> {
// Yields the (stream_type 0, stream_type 1) pair on first call,
// ConnectionClosed on subsequent calls. This is the yield-once
// contract per channel, matching the POC's validated shape.
}
async fn open_bi(&self) -> Result<(SendStream, RecvStream), StreamError> {
// StreamClosed — a single channel cannot open new application
// streams (same as ADR-007's Stream backend). Additional sub-streams
// (stream_type 2, 3) are accessed via sub_streams(), not open_bi().
}
fn remote_addr(&self) -> Option<SocketAddr> { ... }
fn close(&self, _code: u32, _reason: &str) { ... }
}
```
Each channel is presented to its handler as a `Connection` constructed via
`Connection::from_source(ChannelBidiStreamSource::new(...), alpn)`. The
handler calls `accept_bi()` once, gets the main data pair, and drives its
session — exactly as the POC's `EchoHandler` and `TtyAdapter` do today.
### Sub-stream accessor for typed destructure (OQ-CH-10)
Some handlers need access to `stream_type` 2 (stderr), 3 (control in), and
4 (control out) in addition to the main 0/1 pair. The `Connection`
interface alone (accept_bi) only exposes the 0/1 pair. The channels crate
provides a typed-accessor extension:
```rust
// In alknet-channels:
pub struct ChannelSubStreams {
/// (stream_type, handle) for each active stream_type. Each handle is
/// unidirectional: write stream_types (0, 3, 6, ...) carry a SendStream;
/// read stream_types (1, 2, 4, 5, 7, ...) carry a RecvStream.
/// See ADR-034 §stream_type decomposition.
pub streams: Vec<(u8, SubStreamHandle)>,
}
pub enum SubStreamHandle {
Send(SendStream), // write half (stream_type % 3 == 0)
Recv(RecvStream), // read half (stream_type % 3 == 1 or 2)
}
impl ChannelBidiStreamSource {
/// Returns the typed sub-streams for this channel, keyed by stream_type.
/// Consumes the source — call this instead of accept_bi() if the handler
/// needs direct access to stream_types 2/3/4. For handlers that only need
/// the main 0/1 pair, accept_bi() is the path (and sub_streams() is not
/// called).
pub fn into_sub_streams(self) -> ChannelSubStreams { ... }
}
```
The handler crate (e.g., `alknet-tty`) destructures `ChannelSubStreams` into
its typed names:
```rust
// In alknet-tty (inside-channels mode, ADR-077):
let sub = channel_source.into_sub_streams();
let stdin = sub.get_send(0).unwrap(); // SendStream (write, client→server)
let stdout = sub.get_recv(1).unwrap(); // RecvStream (read, server→client)
let stderr = sub.get_recv(2); // Option<RecvStream> (read, optional)
let ctrl_in = sub.get_send(3).unwrap(); // SendStream (write, client→server)
let ctrl_out = sub.get_recv(4).unwrap();// RecvStream (read, server→client)
```
**Every stream_type is unidirectional** (ADR-034). Write stream_types
(`% 3 == 0`) carry a `SendStream`; read stream_types (`% 3 == 1 or 2`) carry
a `RecvStream`. There is no "bidirectional" stream_type — bidirectionality
is two halves (e.g., control is 3 write + 4 read). This resolves the TTY
control channel's "not actually bidirectional" flaw: the TTY adapter reads
exit/keepalive from `ctrl_out` (stream_type 4) and writes resize/signal/eof
to `ctrl_in` (stream_type 3), each with its own flow control and EOF.
**The channels crate does not know about TTY's `stream_type` semantics.**
It exposes `(stream_type, SubStreamHandle)` tuples. The handler crate maps
stream_types to its typed names. This preserves ADR-031's
no-handler-depends-on-another-handler rule and keeps the channels crate
ALPN-blind.
### When to use `accept_bi` vs `into_sub_streams`
| Handler shape | Path | Example |
|---------------|------|---------|
| Main data pair only (0/1) | `accept_bi()` | tunnel handler, SSH handler (SSH multiplexes internally) |
| Needs stderr/control (2/3) | `into_sub_streams()` | TTY handler (stdin/stdout/stderr/control) |
The handler chooses at construction time based on its ALPN's `stream_type`
set (declared at `channel/open` time, ADR-037). The `ChannelsAdapter` passes
the handler a `Connection` (via `from_source`); handlers that need sub-
streams downcast or receive the `ChannelBidiStreamSource` directly via a
channels-crate extension trait. The exact ergonomics (downcast vs. a
channels-crate constructor that hands the source directly to handlers that
opt in) are an implementation detail for the channels crate; the contract is
that both paths are available and the handler crate chooses.
### Recursive composition
A `ChannelBidiStreamSource` is a `BidiStreamSource`, and `Connection::
from_source` wraps it. A handler that is itself `alknet/channels` can open a
sub-channels connection on a data channel. This is recursive composition:
`alknet/channels` inside `alknet/channels`. It is allowed (the `Connection`
abstraction permits it) but not a feature designed for — the primary use
case is one level of multiplexing. Recursive composition is a natural
consequence of the abstraction, not a goal.
## Consequences
**Positive:**
- `ChannelConnection` is a first-class peer of QUIC: one
`BidiStreamSource` impl per channel, constructed via `from_source` — no
core edit (the ADR-008 extension point).
- Handlers that only need the main data pair use `accept_bi()` — identical
to how they work on top-level QUIC connections. Zero handler changes for
the tunnel/SSH shape.
- Handlers that need typed sub-streams (TTY) use `into_sub_streams()` — the
channels crate provides the accessor, the handler crate maps to typed
names. No channels-crate knowledge of TTY semantics.
- The POC's validated yield-once shape is preserved per-channel; the N-stream
generalization is at the connection level (one channels connection = N
channels = N `ChannelBidiStreamSource` instances), not per-channel.
**Negative:**
- Two paths to access channel data (`accept_bi` vs `into_sub_streams`). This
is a necessary divergence: the `Connection` interface alone can't express
"give me four named sub-streams" without four `accept_bi` calls (which
would violate the yield-once contract). The two-path design is the
minimum-complexity solution; the alternative (a new `Connection` variant
with multi-stream semantics) would touch `alknet-core` and break the
ADR-008 extension-point model.
- `into_sub_streams()` consumes the source, so a handler can't call both
`accept_bi()` and `into_sub_streams()`. This is by design — the sub-
streams include the 0/1 pair, so `into_sub_streams()` is the superset.
## Door type
**One-way.** The `ChannelBidiStreamSource` shape (one source per channel,
yield-once `accept_bi`, `into_sub_streams` accessor) is the handler-facing
API surface. Changing it after handlers exist (TTY, tunnel, SSH) is a
rewrite of those handlers' integration code. The trait impl is in the
channels crate (not core), so the one-way door is the channels crate's API,
not a core type.
**Amended by ADR-035 (2026-07-18):** `into_sub_streams()` is removed;
`accept_bi` is the only accessor. The one-way door is re-cast (the
channels crate is not yet implemented, so this is the right time). See
ADR-035 for the amended door-type discussion.
The choice of `into_sub_streams()` returning `Vec<(u8, SendStream,
RecvStream)>` (vs a typed struct, vs a map) is a two-way-door implementation
detail — the return type can change without breaking the contract as long
as the handler crate's destructure code updates.
## References
- **ADR-035**: channels pure channel multiplexing (amends this ADR —
`into_sub_streams()` removed; `accept_bi` is the only accessor, yields
one `BiStream` per channel; the handler owns its sub-stream
multiplexing)
- ADR-009: `BiStream` as the handler leaf (the transport-leaf decision
this ADR's amendment builds on — `accept_bi` returns `BiStream`)
- ADR-008: BidiStreamSource trait (the extension point this implements)
- ADR-007: Connection::from_stream (the yield-once path this generalizes for
channels)
- ADR-034: channels wire format (the chunks this reassembles)
- ADR-039: ChannelsAdapter and ChannelManager (the components that construct
`ChannelBidiStreamSource` instances)
- ADR-077: TTY inside channels (the primary consumer of `into_sub_streams`)
- `docs/research/alknet-channels/poc-summary.md` §POC Target 2, §Issues
Surfaced #1

257
docs/architecture/decisions/039-channelsadapter-and-channelmanager.md Normal file
View File

@@ -0,0 +1,257 @@
# ADR-039: ChannelsAdapter and ChannelManager
## Status
Accepted (amended 2026-07-18 by ADR-035 — demux reads 8-byte headers, not
9-byte; one reassembly buffer per `channel_id` (not per
`(channel_id, stream_type)`); `ChannelState.stream_types` removed; the
channels layer has no `stream_type` concept — see "Amendment (ADR-035,
2026-07-18)" below)
## Amendment (ADR-035, 2026-07-18)
The demux loop reads **8-byte headers** (not 9-byte). `ChannelState` has
**one reassembly buffer per `channel_id`** (not per
`(channel_id, stream_type)`), yielding a `BiStream` to the handler. The
`stream_types: Vec<u8>` field on `ChannelState` is **removed**. The
`ChannelManager` has no `stream_type` concept — it routes by `channel_id`
only, and the handler owns its sub-stream multiplexing on the `BiStream`
it receives (per ADR-035, the channels-layer consequence of ADR-009's
`BiStream` handler leaf).
The body below describes the **original** (9-byte, per-stream_type) shape;
the amendment above is the operative decision. See ADR-035 for the
resolution rationale and the cross-ADR impacts.
## Context
The channels crate has two internal components, split by responsibility
(`docs/research/alknet-channels/phase-0-findings.md` §Channel Manager and
Connection Internals):
1. **`ChannelsAdapter`** — implements `ProtocolHandler` for
`alknet/channels`. Its `handle()` receives one `Connection` (the
transport), reads 9-byte chunk headers, and routes each chunk. It is the
read/demux half.
2. **`ChannelManager`** — the shared state both halves touch. It holds the
map of `channel_id → ChannelState`, the `HandlerRegistry` reference, and
the `OperationRegistry` reference. It is the reassemble/allocate half.
It is what the `channel/open` operation handler closes over.
The de-risk POC (`docs/research/alknet-channels/poc-summary.md` §Issues
Surfaced) surfaced three invariants the spec must pin: the mux needs dynamic
registration (handle/runner split — REQ-CH-03), the demux must drop all
channel senders on transport EOF (REQ-CH-02), and the `AsyncWrite::shutdown`
must emit a zero-length sentinel (REQ-CH-01). This ADR pins these as
contracts.
## Decision
### `ChannelsAdapter` — the read/demux half
```rust
#[async_trait]
impl ProtocolHandler for ChannelsAdapter {
fn alpn(&self) -> &'static [u8] { b"alknet/channels" }
async fn handle(&self, connection: Connection, auth: &AuthContext)
-> Result<(), HandlerError>
{
// 1. Channel 0 is pre-negotiated as alknet/call (ADR-036).
// The first bidi stream the transport yields is channel 0.
let (send, recv) = connection.accept_bi().await?;
self.manager.preinstall_channel_0(send, recv, auth).await?;
// 2. Accept remaining bidi streams and read 9-byte headers off each.
// On an in-line transport (TCP+TLS, WebTransport), accept_bi()
// yields once and the header demuxes N channels from that stream.
// On QUIC native, accept_bi() yields repeatedly — each stream
// carries one logical channel, and the header provides
// stream_type + channel_id correlation. Same code path, same
// wire format (ADR-034 §substrate modes).
self.manager.run_demux_loop(connection).await
}
}
```
The `preinstall_channel_0` step constructs the reassembly buffers for
`channel_id = 0` using stream_types [0, 1] (ADR-036), wraps them as a
`Connection` (via `Connection::from_source` with a `ChannelBidiStreamSource`
— ADR-038), and hands that `Connection` to the `CallAdapter` — exactly as if
`alknet/call` had been the top-level ALPN. The `CallAdapter` is looked up in
the same `HandlerRegistry` as every other ALPN.
`run_demux_loop` continues accepting bidi streams from the transport. For
each stream, it reads 9-byte headers and routes payloads to the matching
`(channel_id, stream_type)` reassembly buffer. On an in-line transport,
there is only one stream (channel 0 rides inside it via the header); the
header demuxes all channels. On QUIC, each subsequent stream is a new
channel; the header's `channel_id` correlates it. The loop is the same;
only the transport's stream count differs.
### `ChannelManager` — the shared state
```rust
pub struct ChannelManager {
/// channel_id → per-channel state. Channel 0 is pre-inserted at
/// construction by preinstall_channel_0.
channels: Mutex<HashMap<u32, ChannelState>>,
/// The handler registry for looking up ALPNs on channel/open.
handlers: Arc<HandlerRegistry>,
/// The call protocol's operation registry, so channel/open etc. can be
/// registered at assembly time.
call_ops: Arc<OperationRegistry>,
/// Next server-assigned channel_id. Monotonic; wraps at u32::MAX.
next_id: AtomicU32,
/// Per-channel reassembly buffer cap (ADR-040). Default 1 MiB.
buffer_cap: usize,
/// Per-connection channel limit (ADR-040). Default 256.
max_channels: usize,
}
struct ChannelState {
/// The ALPN this channel carries, for routing and observability.
alpn: String,
/// Reassembly buffers per active stream_type.
streams: HashMap<u8, ReassemblyBuffer>,
/// The handler task driving this channel. Dropping this aborts it.
handler_task: JoinHandle<()>,
/// Which stream_types are active (from the open negotiation).
stream_types: Vec<u8>,
}
```
`ChannelManager` is `Clone` (cheap — `Arc` internally) so the
`ChannelsAdapter`, the `channel/open` operation handler, and relay logic
can all hold a handle.
### The demux loop — REQ-CH-02 and REQ-CH-04
`run_demux_loop` reads 9-byte headers, looks up `channel_id` in `channels`,
and pushes the payload into the right `ReassemblyBuffer` for `(channel_id,
stream_type)`.
**REQ-CH-04 (lenient unknown-channel_id):** a chunk with an unallocated
`channel_id` (or `stream_type`) is dropped with a debug log and an error
counter (exposed via `Demux::stats()`), and the demux continues. This
matches SSH's behavior and survives transient mis-ordering during teardown.
Validated by the POC (`demux_unknown_channel_drops_lenient`).
**REQ-CH-02 (transport close → all handlers see EOF):** on transport EOF,
the demux loop clears its `channels` map, dropping all `ReassemblyBuffer`
senders. Every handler's reassembled `RecvStream` sees EOF even without an
explicit zero-length sentinel on the wire. Without this, `read_to_end` /
`tokio::io::copy` in handlers hangs forever waiting for a sender that never
drops. This is a teardown invariant of the `ChannelsAdapter::handle`
contract. Validated by the POC.
### The mux — REQ-CH-03 (handle/runner split)
The mux frames per-channel bytes back onto the transport. The POC surfaced
that the plan's `Mux::run(self, transport)` shape (consume the mux, run
pumps for pre-registered channels) does not compose with the dynamic
`channel/open` model — channels are opened after the run loop starts.
**REQ-CH-03 (dynamic registration):** the mux is split into:
- **`MuxHandle`** — clone-able, `register(channel_id, stream_type) ->
Sender<Bytes>` callable at any time (after the runner has started).
- **`MuxRunner`** — owns the transport, `select!`s on new-pump registrations
and per-channel write pumps.
The runner's `select!` loop exits when all `MuxHandle` clones drop (the
`new_pumps` sender closes), which is the natural shutdown signal. This
matches the dynamic `channel/open` model. The split adds one
`mpsc::UnboundedSender` + `Arc<Mutex<HashMap>>` per mux — cheap. Validated
by the POC.
### `ChannelManager` is ALPN-blind and auth-blind
The `ChannelManager` deliberately does **not** hold:
- **No `ProtocolHandler` implementations.** It holds a `HandlerRegistry`
reference for ALPN lookup, but it doesn't *be* a handler. Handlers live in
their crates and register on the same registry.
- **No ALPN-specific parsing.** It does not parse `NegotiateRequest` JSON,
SSH frames, or tunnel target strings. It hands `params` JSON to the
handler and gets back a handler task; it hands `stream_type 3` JSON to the
handler's control handle.
- **No auth state.** Auth lives in the `OperationContext` that the call
protocol passes to `channel/open`. The `ChannelManager` doesn't check
scopes or ownership — that's `AccessControl::check` in
`OperationRegistry::invoke`, run before the `channel/open` handler.
- **No transport coupling.** It talks to the transport only through the
`ChannelsAdapter`'s read loop and the per-channel write pumps, both of
which use `AsyncRead + AsyncWrite`.
This is what makes the channels layer WASM-compatible and transport-agnostic
— the `ChannelManager` is pure byte routing with no platform or protocol
dependencies.
### The `channel/open` handler — threading into `OperationRegistry`
The `channel/open` (and `channel/close`, `channel/control`,
`channel/resources/subscribe`) operations are registered on the call
protocol's `OperationRegistry` at assembly time. The handler closures close
over a `ChannelManager` clone:
```rust
let channel_ops = ChannelOperations::new(manager.clone());
channel_ops.register_on(&mut call_registry)?;
```
The `channel/open` handler (ADR-037) looks up the ALPN in `HandlerRegistry`,
allocates the `channel_id` via `next_id.fetch_add(1, Relaxed)`, constructs
the `ChannelBidiStreamSource` (ADR-038), spawns the handler task, and
records the `ChannelState`. The key insight: spawning the handler task is
identical to what `TtyAdapter::handle` does today — `tokio::spawn` a
session-driving task. The only difference is the `Connection` passed in is
backed by chunk reassembly rather than a quinn connection.
## Consequences
**Positive:**
- The ChannelsAdapter/ChannelManager split mirrors the TTY crate's
ChunkReader/ChunkWriter + adapter pattern, generalized to N channels.
- The demux/mux contracts (REQ-CH-01..04) are pinned as wire-level
invariants, not implementation details. Both sides must agree, or channels
hang on clean shutdown.
- The `ChannelManager` is ALPN-blind, auth-blind, and transport-blind — the
channels layer is a re-framing proxy, not a protocol engine. This is what
makes it reusable across TTY, SSH, tunnel, and future ALPNs.
**Negative:**
- The mux handle/runner split (REQ-CH-03) adds one `mpsc::UnboundedSender` +
`Arc<Mutex<HashMap>>` per mux. Cheap, but more moving parts than the
pre-register-all-then-run alternative. The alternative doesn't match the
dynamic `channel/open` model, so the split is necessary, not optional.
- The demux loop is one task per transport. If the demux task panics, all
channels on that transport lose their read side. The teardown invariant
(REQ-CH-02) ensures handlers see EOF, not a hang — but a panic in the
demux is still a transport-wide failure. This is the same property as any
single-task read loop (including the call protocol's dispatch loop).
## Door type
**One-way (contracts) + two-way (internals).** The wire-level invariants
(REQ-CH-01..04) are one-way — both sides must agree, and changing them
after deployments exist is a protocol migration. The `ChannelManager`'s
internal structure (fields, `Arc<Mutex<HashMap>>` vs a concurrent map, etc.)
is two-way — implementation details that can change without breaking the
contract.
## References
- ADR-034: channels wire format (the chunks the demux reads, as amended
by ADR-035 — 8-byte header)
- ADR-035: channels pure channel multiplexing (amends this ADR — 8-byte
header, one reassembly buffer per channel, no `stream_type` concept)
- ADR-036: channel 0 pre-negotiated (the `preinstall_channel_0` step)
- ADR-037: channel lifecycle operations (the ops registered on `call_ops`)
- ADR-038: ChannelBidiStreamSource (the per-channel source the manager
constructs, as amended by ADR-035 — `accept_bi` yields a `BiStream`)
- ADR-040: backpressure, channel limits, ID reuse (the `buffer_cap` /
`max_channels` / reuse invariants)
- `docs/research/alknet-channels/poc-summary.md` §Issues Surfaced #4-#6
(REQ-CH-01, 02, 03)

237
docs/architecture/decisions/040-backpressure-channel-limits-id-reuse.md Normal file
View File

@@ -0,0 +1,237 @@
# ADR-040: Backpressure, Channel Limits, and ID Reuse
## Status
Accepted (amended 2026-07-18 by ADR-035 — backpressure is per-`channel_id`,
not per-`(channel_id, stream_type)`; the channels layer has one reassembly
buffer per channel, yielding a `BiStream` — see "Amendment (ADR-035,
2026-07-18)" below; **amended 2026-07-19 by ADR-041 — the per-connection
`max_channels = 256` is reframed as a per-connection memory bound, not a
DoS defense; the per-identity DoS defense lives in `channels-call` via
`ChannelLifecyclePolicy` — see "Amendment (ADR-041, 2026-07-19)" below**)
## Amendment (ADR-041, 2026-07-19)
The per-connection `max_channels = 256` cap is **reframed as a
per-connection memory bound**, not a DoS defense. A single peer can
open an unbounded number of transport connections, so a per-connection
cap is not a per-peer DoS defense — it is a bound on one connection's
reassembly-buffer cost. The per-identity DoS defense (256 per
`PeerId`, enforced in `channels-call` via `ChannelLifecyclePolicy`)
is documented in [ADR-041](041-per-identity-channel-cap.md).
What changes in this ADR:
1. **§"Maximum channels per connection: 256 default"** — the cap stays
at 256, but its role is reframed. It is a per-connection memory
bound (limits one connection's reassembly-buffer cost regardless of
policy), not the DoS defense against an authenticated peer. The
per-identity DoS defense is the `ChannelLifecyclePolicy`
consultation in the `channel/open` handler (ADR-041).
2. **§"DoS defense summary"** — the table is **removed**. It framed
the per-connection cap as the DoS defense, which it is not. ADR-041
§2 contains the corrected per-identity DoS defense summary.
3. **The "per-connection, not per-peer — a peer can open more channels
on a second connection" line** — this was the channels layer
confessing a hole and hoping the layer above it would fill it. The
line is **corrected** to state that the per-connection cap is a
memory bound, and that the per-identity cap is the DoS defense
(ADR-041). A peer that opens a second connection gets a second
per-connection memory bound; it does **not** get a second
per-identity quota — the `ChannelLifecyclePolicy` is shared across
connections.
What stays:
- The 256 default and the `max_channels` field on `ChannelManager`
(still returns `channel:too_many_channels` when hit — the
per-identity policy returns the same error code, so an over-cap
peer sees the same error either way).
- The bounded-buffer backpressure decision (DP-5) — unchanged.
- The channel-ID reuse decision (monotonic `next_id` with
wrap-around) — unchanged.
- The drain-before-reuse invariant — unchanged, and the
`channel/close` handler now also calls
`ChannelLifecyclePolicy::on_close` at this point (ADR-041 §3).
## Amendment (ADR-035, 2026-07-18)
The bounded-buffer backpressure is per-`channel_id` (not per
`(channel_id, stream_type)`). The channels layer has one reassembly
buffer per channel (yielding a `BiStream`), not one per
`(channel_id, stream_type)`. The 1 MiB default and the 256-channel cap are
unchanged; the per-channel memory ceiling is 1 MiB (was up to 5 MiB for a
TTY channel with 5 active stream_types under the per-stream_type model).
This is a net improvement (lower memory ceiling per channel), not a
regression. The bounded-buffer *approach* is unchanged; only the
buffer granularity changes (per-channel, not per-stream_type).
The body below describes the **original** (per-stream_type) shape; the
amendment above is the operative decision. See ADR-035 for the resolution
rationale.
## Context
The phase-0 research (`docs/research/alknet-channels/phase-0-findings.md`
§DP-5, §OQ-CH-03/04/05/06) raised four operational questions about the
channels layer:
1. **Flow control (DP-5, OQ-CH-03):** if one data channel's consumer is
slow, could it block all other channels on the same transport
(head-of-line blocking)? The research recommended "bounded-buffer
backpressure (option c)… if head-of-line blocking becomes a real problem,
full windowing can be added." The "if it becomes a problem" is a hedge —
the POC validated bounded-buffer with a 1 MiB test and no deadlock. The
decision is bounded-buffer.
2. **Channel ID reuse (OQ-CH-04):** after a channel is closed, can its ID be
reused?
3. **Maximum channels per connection (OQ-CH-05):** is there a limit?
4. **Channel open DoS (OQ-CH-06):** an authenticated peer could open many
channels and never read from them, exhausting memory.
The de-risk POC (`docs/research/alknet-channels/poc-summary.md` §POC Target
1, §POC Target 3) validated the bounded-buffer backpressure path: the 1 MiB
`tunnel_large_payload` test exercises a channel writer faster than the TCP
echo server consumer, with no deadlock and no cross-channel blocking.
## Decision
### Backpressure: bounded-buffer, 1 MiB default (DP-5)
Each `(channel_id, stream_type)` pair has an independent bounded `mpsc`
buffer. When a channel's buffer is full, the demux stops reading chunks for
that `channel_id` until the consumer drains it. Other channels keep flowing
— the demux's per-chunk route awaits the matching sender without holding a
global lock.
**Default buffer cap: 1 MiB per `(channel_id, stream_type)`.** Configurable
per `ChannelManager` (`buffer_cap` field). This prevents memory exhaustion
without the complexity of SSH's sliding-window protocol.
Full channel-level windowing (SSH-style sliding-window per channel) is a
deferred extension, tracked as [OQ-56](../questions/056-full-channel-level-flow-control-windowing.md)
(deferred(scope)). It is blocked on a real deployment observing head-of-
line blocking where the bounded-buffer mitigation is insufficient. The
bounded-buffer decision is made; the extension is not.
### Channel ID reuse: yes, after drain (OQ-CH-04)
After a channel is closed (`channel/close` acknowledged), its `channel_id`
is eligible for reuse. The reassembly buffers must be fully drained before
reuse to prevent data from the old channel leaking into the new one.
**Drain-before-reuse invariant:** the `ChannelManager` marks a closed
channel's ID as "draining" (not in the `channels` map, but not yet returned
to the free pool). The ID returns to the free pool only after:
1. The `channel/close` response is sent (the close is acknowledged).
2. All reassembly buffers for that `channel_id` are empty (the handler has
consumed all data).
The `next_id: AtomicU32` is monotonic (not a free-list) — IDs are not
immediately reused; the monotonic counter wraps at `u32::MAX`. This is
simpler than a free-list and avoids the drain-tracking complexity. With a
default `max_channels` of 256, the `u32` space is effectively unlimited
(~16.7 million channels before wrap). Reuse happens naturally on wrap, by
which time old channels are long drained. **The "reuse" in OQ-CH-04 is
satisfied by the wrap-around, not by a free-list.**
### Maximum channels per connection: 256 default (OQ-CH-05/06 — memory bound)
The `channel_id` is `u32` — the wire format supports ~4 billion channels.
The practical limit is memory (reassembly buffers per channel) and the
transport's flow control.
**Default per-connection channel limit: 256** (`max_channels` field on
`ChannelManager`, configurable). This is a **per-connection memory
bound**: it limits one connection's reassembly-buffer cost (256 × 1 MiB
= 256 MiB worst case per connection) regardless of policy. It composes
with the per-identity DoS defense (ADR-041) but is not itself a DoS
defense — a peer can open an unbounded number of transport connections,
so a per-connection cap cannot bound a peer's total channels. The
per-identity DoS defense (256 per `PeerId`, enforced in `channels-call`
via `ChannelLifecyclePolicy`) is documented in
[ADR-041](041-per-identity-channel-cap.md).
Exceeding the per-connection limit returns `channel:too_many_channels`
(ADR-037 error codes) — the same error code the per-identity policy
returns when the per-identity cap is hit. An over-cap peer sees the
same error either way; which cap fired first is an implementation
detail. The limit is per-connection as a memory bound; the per-identity
cap (ADR-041) is what bounds a peer's total channels across all its
connections.
### DoS defense summary (OQ-CH-06)
The DoS defense against an authenticated peer opening many channels is
the **per-identity cap** enforced in `channels-call` via
`ChannelLifecyclePolicy` — documented in
[ADR-041](041-per-identity-channel-cap.md). A per-connection cap
cannot be the DoS defense because a peer can open an unbounded number
of transport connections; the unit that must be bounded is the
identity, not the connection.
The per-connection `max_channels = 256` (this ADR) is a **memory
bound** that limits one connection's reassembly-buffer cost. It
composes with the per-identity cap as defense-in-depth (the
`NoCap` policy path still has the per-connection memory bound), but
it is not the security boundary. See ADR-041 §2 for the corrected
DoS defense summary.
## Consequences
**Positive:**
- Bounded-buffer backpressure is validated by the POC (1 MiB test, no
deadlock, no cross-channel blocking). The decision is made, not hedged.
- The 256-channel default cap with 1 MiB buffers gives a bounded 256 MiB
worst-case memory per connection — a clear per-connection memory
ceiling, not an open-ended one. The per-identity DoS ceiling (256 per
`PeerId` across all the peer's connections) is documented in ADR-041.
- Monotonic `next_id` with wrap-around avoids free-list drain-tracking
complexity while still satisfying ID reuse (on wrap, after ~16.7M
channels).
**Negative:**
- The 256-channel per-connection cap may be too low for a hub with many
concurrent browser sessions each opening multiple channels. The cap
is configurable per `ChannelManager`; the hub deployment may set it
higher for deployments with many concurrent sessions. This is a
deployment-time decision, not an architecture decision. (The
per-identity cap in ADR-041 is the DoS-relevant bound; the
per-connection cap is a memory backstop.)
- Bounded-buffer backpressure does not eliminate head-of-line blocking — it
bounds the memory cost. A slow consumer still stalls its own channel's
demux reads. For the intended use cases (TTY, SSH, tunnels) this is
acceptable; full windowing is tracked as OQ-56 (deferred(scope)).
## Door type
**Two-way.** The buffer cap (1 MiB), the channel limit (256), and the
monotonic-ID-with-wrap strategy are all configurable / changeable without a
wire-format change. The bounded-buffer *approach* (vs full windowing) is
one-way in the sense that the demux/mux code is written around it — but
full windowing is an additive extension (per-channel window tracking) that
doesn't change the wire format, so even that reversal is feasible.
## References
- ADR-034: channels wire format (the chunks the buffers hold, as amended
by ADR-035)
- ADR-035: channels pure channel multiplexing (amends this ADR —
per-channel reassembly buffer, not per-`(channel_id, stream_type)`)
- ADR-041: per-identity channel cap as DoS defense (amends this ADR —
the per-connection `max_channels = 256` is reframed as a per-connection
memory bound, not a DoS defense; the per-identity DoS defense lives in
`channels-call` via `ChannelLifecyclePolicy`)
- ADR-037: channel lifecycle operations (`channel:too_many_channels`
error; the `channel/open` and `channel/close` handlers that gain the
`ChannelLifecyclePolicy` consultation)
- ADR-039: ChannelManager (`buffer_cap`, `max_channels`, `next_id`
fields; the auth-blindness that forces the per-identity cap into
`channels-call`, not `channels-core`)
- ADR-026: forwarded-for identity (why the spoke caps the hub, not the
browser — `forwarded_for` is metadata, not authority, for the cap as
for `AccessControl::check`)
- `docs/research/alknet-channels/poc-summary.md` §POC Target 1 (backpressure
validation), §POC Target 3 (1 MiB tunnel test)
- `docs/research/alknet-channels/phase-0-findings.md` §DP-5, §OQ-CH-03/04/
05/06

324
docs/architecture/decisions/041-per-identity-channel-cap.md Normal file
View File

@@ -0,0 +1,324 @@
# ADR-041: Per-Identity Channel Cap as DoS Defense
## Status
Accepted (amends ADR-040's DoS-defense framing — the per-connection
`max_channels = 256` is reframed as a per-connection memory bound, not a
DoS defense)
## Context
ADR-040 set the channels-layer channel limit at 256 **per connection**
and framed that cap as the DoS defense against an authenticated peer
opening many channels and never reading from them ("DoS defense
summary" table, "Per-connection channel count cap → 256 channels"). On
review, the per-connection cap is not a DoS defense at all. A single
peer can open an unbounded number of transport connections, and across
those connections, across substrates, the peer gets 256 × N × (substrate
multiplier) channels:
| Substrate a peer can use | Channels per connection |
|--------------------------|--------------------------|
| In-line (TCP+TLS, WebTransport, SSH `direct-tcpip`) | 256 (one stream, header-demuxed) |
| Native (QUIC substreams) | 256 (per-connection demux; each substream carries one channel) |
| Multi-connection (N transport connections) | 256 × N |
A peer that opens 10 transport connections to the same accepting peer
gets 2,560 channels. A peer that opens 100 gets 25,600. There is no
bound on the number of transport connections a peer can open. The
"per-connection, not per-peer — a peer can open more channels on a
second connection" line in ADR-040 was, in retrospect, the channels
layer confessing a hole and hoping the layer above it would fill it.
That is not a DoS defense; it is a per-connection memory bound
(reassembly-buffer cost per connection) labeled as a DoS defense.
The only coherent unit for a channel DoS defense is the **identity**.
The peer, not the connection, is what an authenticated-DoS defense
must bound. This is the same primitive as any other resource ACL:
`OwnershipProvider` (ADR-011) checks "does identity X own resource
Y?"; the channel cap checks "has identity X exceeded their channel
quota?" Same shape, different resource.
### Why the channels layer cannot hold the cap
`ChannelManager` (ADR-039) is auth-blind by design: "No auth state.
Auth lives in the `OperationContext` that the call protocol passes to
`channel/open`." That decision is load-bearing — it is what makes the
channels layer WASM-compatible, transport-agnostic, and ALPN-blind
(ADR-039, ADR-035). Putting per-identity tracking in the channels
layer would reverse ADR-039.
So the per-identity cap lives **one layer up**, in `channels-call`,
where the identity is already on `OperationContext` (the same place
`AccessControl::check` runs). The `channel/open` and `channel/close`
handlers (ADR-037) are in `channels-call` already; they gain a policy
consultation. The channels layer (`channels-core`) is unchanged —
still auth-blind, still WASM-clean.
### This is not a hub-specific concern
The cap is a **channels-accepting-peer concern**. A worker accepting a
direct channels connection from a peer needs the cap just as much as a
hub does. The call protocol does not need a hub to enforce "does this
peer have access to this resource?" (ADR-037: `AccessControl::check` on
`channel/open`), and neither should channels. Framing the cap as
hub-specific would be the "assembly layer" hedging pattern — putting
the hard question off on a fictional "later" that, when it arrives,
turns out to be exactly the same problem. The cap is a peer concern;
the hub is one peer that happens to aggregate others.
The cap is also **symmetric**, like the call protocol. Peer A accepts
a channels connection from Peer B; A enforces its cap on B's channels;
B enforces its cap on A's channels. Both sides have the cap, both
sides check it, same as `AccessControl::check` on any operation.
## Decision
### 1. A `ChannelLifecyclePolicy` trait in `channels-call`
```rust
/// Per-identity channel lifecycle policy. Consulted by the
/// `channel/open` handler (after `AccessControl::check`, before
/// allocation) and the `channel/close` handler (after deallocation).
/// Both handlers have the identity via `OperationContext`.
///
/// A channel slot is a resource; the cap is a quota check on that
/// resource — parallel to `OwnershipProvider::owns` (ADR-011) for
/// spawned resources. Same primitive, different resource.
pub trait ChannelLifecyclePolicy: Send + Sync + 'static {
/// Before channel allocation. Deny with `channel:too_many_channels`
/// (ADR-037) when the identity is over its cap. The identity is
/// the direct caller (the peer that opened this channels
/// connection); `forwarded_for` is metadata and is NOT consulted
/// (ADR-026).
fn check_open(&self, identity: &Identity) -> Result<(), ChannelError>;
/// After channel deallocation. Decrement the per-identity count.
/// Called by the `channel/close` handler after the drain completes
/// (ADR-040 §channel-id-reuse).
fn on_close(&self, identity: &Identity);
}
```
### 2. Default: `PerIdentityChannelPolicy::new(256)`
The default constructor enforces 256 per identity out of the box — no
hedging, no "NoOp default + wire it in the assembly layer." A channels
accepting peer that constructs `ChannelOperations::new(manager)` with
no policy argument gets `PerIdentityChannelPolicy::new(256)`. The
default is secure; opt-outs are explicit:
- `PerIdentityChannelPolicy::new(cap)` — shared per-identity state
(`HashMap<PeerId, usize>` + cap), constructed **once per accepting
peer** and shared (via `Arc`) across every channels connection that
peer accepts. For a hub, that's one `Arc<PerIdentityChannelPolicy>`
on the `Hub`, shared across all worker and browser legs. For a
worker accepting direct channels, that's one `Arc` on the worker's
own state, shared across whatever connections it accepts. For tests
and POCs, the default constructor.
- `PerIdentityChannelPolicy::with_per_identity_caps(mapping)` — a
per-peer-role variant: `HashMap<PeerId, usize>` overrides the
default cap for specific peers. Used by a spoke that serves a
high-fan-out hub (the hub peer's cap is set higher than a worker
peer's cap — see "Relay consequence" below).
- `NoCap` — no cap (for tests, POCs, and trusted single-peer
deployments). Explicit opt-out, not the default.
The policy is constructed once and passed to `ChannelOperations` at
registration time:
```rust
let policy = Arc::new(PerIdentityChannelPolicy::new(256));
let channel_ops = ChannelOperations::new(manager, policy);
channel_ops.register_on(&mut call_registry)?;
```
The same `Arc<PerIdentityChannelPolicy>` is shared across every
channels connection that peer accepts — that is what makes the cap
per-identity, not per-connection.
### 3. Enforcement point: between `AccessControl::check` and allocation
The `channel/open` handler (ADR-037) gains the policy check after
ACL and before `next_id.fetch_add`:
1. ACL is already checked by `OperationRegistry::invoke` (the existing
`AccessControl::check` path — unchanged).
2. **NEW:** `policy.check_open(&op_ctx.identity)?` — deny with
`channel:too_many_channels` if over cap.
3. Allocate the `channel_id` via `next_id.fetch_add(1, Relaxed)` (DP-1:
server-assigned — unchanged).
4. Construct the `ChannelBidiStreamSource`, spawn the handler, record
the `ChannelState` (unchanged).
5. Return the `channel_id`.
The `channel/close` handler (ADR-037) gains the decrement after the
drain completes (the same point ADR-040 marks the `channel_id` as
eligible for reuse):
1. Drain the reassembly buffer for `channel_id` (existing — ADR-040
§channel-id-reuse).
2. **NEW:** `policy.on_close(&op_ctx.identity)` — decrement the
per-identity count.
3. Return `{ "closed": true }` (unchanged).
### 4. `ChannelManager.max_channels = 256` stays as a per-connection memory bound
The per-connection cap (ADR-040) stays, but is reframed. It is no
longer the DoS defense — it is a per-connection **memory bound** that
limits one connection's reassembly-buffer cost regardless of policy.
It composes with the per-identity cap but is not the security
boundary. It still returns `channel:too_many_channels` when hit; the
per-identity policy returns the same error when the per-identity cap
is hit. An over-cap peer sees the same error either way; which cap
fired first is an implementation detail.
Keeping the per-connection bound as a backstop covers deployments that
use `NoCap` (tests, trusted single-peer) and bounds the damage if a
custom policy is buggy. Removing it would leave the channels layer
unbounded in the no-policy case. The cost of keeping it is zero (the
cap is already implemented in the POC); the cost of removing it is a
real hole in the `NoCap` path.
### 5. Relay consequence: the spoke caps the hub, not the browser
When the hub relays a browser's channel to a spoke (ADR-042), the
spoke sees the hub as the direct caller. `forwarded_for` carries the
browser's identity as metadata (ADR-026 — `forwarded_for` is not
authority; `AccessControl::check` never reads it). The channel cap
follows the same shape: the spoke's `ChannelLifecyclePolicy` is
consulted with the **hub's** identity, not the browser's. The spoke
asks "does the hub have access to open another channel?" and the
hub's quota on the spoke reflects the aggregate of all relayed
channels. The hub's per-browser caps are the hub's own concern
(enforced on the browser leg by the hub's own policy), not the
spoke's.
This is correct and consistent — the spoke authorizes the hub for
container access the same way it authorizes any peer, and the hub's
browser-relay ACL is the hub's own layer. The channel cap follows the
same pattern as any other resource ACL.
**Deployment consequence:** a spoke that serves a hub relaying for
many browsers must set the hub peer's cap higher than a worker peer's
cap, or the spoke denies legitimate relayed channels when the hub's
aggregate count exceeds a worker-sized cap. This is a per-peer-role
policy, set by the spoke via `with_per_identity_caps`. The
architecture provides the mechanism (`PerIdentityChannelPolicy::with_per_identity_caps`);
the deployment sets the numbers. This is not a flaw — it is the same
shape as any per-peer ACL (a spoke may authorize one peer for 1000
containers and another for 10; the channel cap is the same kind of
per-peer policy).
### 6. Recursive channels do not bypass the cap
A recursive `alknet/channels`-inside-`alknet/channels` channel runs a
new `ChannelsAdapter` with a new `ChannelManager`. If the same
`ChannelLifecyclePolicy` is wired into the inner `ChannelOperations`,
the inner channels are counted against the same identity. If a
different policy is wired, the inner channels are counted against
that policy's identity (which may be a different identity, if the
inner channels connection is authenticated separately). Either way,
the cap applies; recursion is not a bypass. The 13-byte-per-chunk
overhead of recursion is the documented cost (ADR-035); the cap
behavior is unchanged. Recursive channels are an edge case for edge
cases and not specced further.
## Consequences
**Positive:**
- A real per-identity DoS defense. A peer with N transport connections
to the same accepting peer is bounded by 256 (or the configured
per-identity cap), not 256 × N × (substrate multiplier). The cap
composes correctly across substrates because the unit is the
identity, not the connection.
- The cap is symmetric, like the call protocol. Both sides of a
channels connection enforce their cap; the cap is a peer concern,
not a hub-specific concern.
- The cap lives in `channels-call`, where the identity is already on
`OperationContext`. The channels layer (`channels-core`) is
unchanged — still auth-blind, still WASM-clean, still
transport-agnostic. ADR-039's auth-blindness is preserved.
- The default is secure. `PerIdentityChannelPolicy::new(256)` is the
out-of-the-box behavior; opt-outs (`NoCap`) are explicit. A
deployment that forgets to wire a policy still gets a per-identity
cap.
- The cap is the same primitive as any other resource ACL
(`OwnershipProvider` for spawned resources, `AccessControl::check`
for operations). The mental model is uniform: a channel slot is a
resource, the cap is a quota check on that resource.
**Negative:**
- One new trait (`ChannelLifecyclePolicy`) and one new constructor
argument on `ChannelOperations`. The `channel/open` and
`channel/close` handlers gain a policy call. Small implementation
cost; the policy is a single trait method per direction.
- Per-identity state is shared across connections
(`HashMap<PeerId, usize>` on the policy, guarded by a `Mutex`). The
state is touched on `channel/open` and `channel/close` only — not
on every chunk. The contention is per-identity, not per-chunk;
acceptable for the intended use cases.
- A spoke serving a high-fan-out hub must set the hub peer's cap
higher than the default, or legitimate relayed channels are denied.
This is a deployment-time policy decision, surfaced explicitly by
`with_per_identity_caps`. Not a flaw; the same shape as any
per-peer ACL.
- The cap is per direct-caller identity (ADR-026), not per
`forwarded_for` originator. A hub relaying for 100 browsers
consumes one channel slot per relayed channel against the hub's
quota on the spoke, not 100 slots against 100 browser quotas. A
spoke that wants per-browser capping would need to read
`forwarded_for` for authority, which ADR-026 explicitly forbids.
This is the correct trade-off: capping against `forwarded_for`
would reverse ADR-026's "forwarded_for is metadata, not authority"
and is a much bigger change. The hub enforces per-browser caps on
the browser leg; the spoke enforces per-hub caps on the spoke leg.
## Door type
**One-way.** The `ChannelLifecyclePolicy` trait surface
(`check_open(&Identity) -> Result<(), ChannelError>` and
`on_close(&Identity)`) is a one-way-door API commitment — the
`channels-call` `channel/open` and `channel/close` handlers depend on
it, and consumers (`Hub`, worker crates) construct implementations.
Removing the trait or changing the signatures after deployments exist
is a breaking change.
The **default cap value (256)** is a two-way-door implementation
detail within the one-way trait surface — changing the default is
additive (a new constructor or a default-override), not a wire-format
change.
The **reframing of ADR-040's per-connection cap** (from DoS defense
to memory bound) is two-way — it's a documentation change, not a
behavior change. The per-connection cap still exists, still returns
`channel:too_many_channels`, and still bounds one connection's
reassembly-buffer cost.
## References
- **ADR-040**: Backpressure, Channel Limits, and ID Reuse (amended by
this ADR — the per-connection `max_channels = 256` is reframed as a
per-connection memory bound, not a DoS defense; the "DoS defense
summary" table is removed; the "per-connection, not per-peer" line
is corrected)
- **ADR-039**: ChannelsAdapter and ChannelManager (the auth-blindness
this ADR preserves — the cap lives in `channels-call`, not
`channels-core`)
- **ADR-037**: Channel Lifecycle Operations (the `channel/open` and
`channel/close` handlers that gain the policy check; the
`channel:too_many_channels` error code)
- **ADR-035**: channels Pure Channel Multiplexing (the umbrella
decision; the channels layer has no `stream_type` concept, and no
identity concept either — both are above it)
- **ADR-026**: Forwarded-For Identity (Metadata, Not Authority) (why
the spoke caps the hub, not the browser — `forwarded_for` is
metadata; the direct caller's identity is the authority for the cap
just as it is for `AccessControl::check`)
- **ADR-042**: Hub Relay — Translate, Not Transparently Forward (the
relay path where the spoke sees the hub as the direct caller)
- **ADR-011**: Dynamic Resource Ownership for Runtime-Spawned
Resources (the parallel — a channel slot is a resource, the cap is
a quota check, same primitive as `OwnershipProvider::owns`)
- **ADR-025**: PeerEntry and Identity.id Decoupling (`PeerId` =
`Identity.id` — the stable key the per-identity cap counts against)

181
docs/architecture/decisions/042-hub-relay-translate-not-forward.md Normal file
View File

@@ -0,0 +1,181 @@
# ADR-042: Hub Relay — Translate, Not Transparently Forward
## Status
Accepted
## Context
The hub is the architectural role (ADR-024, ADR-034) that bridges peers and
browsers. With channels, the hub holds one channels connection per leg
(browser↔hub, hub↔spoke) and relays channels between them. The phase-0
research (`docs/research/alknet-channels/phase-0-findings.md` §OQ-CH-11,
§The hub relay) identified the key question: does the hub *translate*
`channel/open` (terminate channel 0 on both legs, re-issue the open on the
spoke leg) or *transparently forward* (pass the call operation through
unchanged)?
This is the most under-specified part of the research for something that is
the *primary motivation* for the channels crate (§Hub Motivation: the
multi-transport collapse). The research said "Phase 1 must specify whether
the hub translates or transparently forwards, and how the `channel_id`
mapping is maintained."
The answer is derivable from the existing machinery:
- The hub terminates channel 0 on both legs (it runs its own `CallAdapter`
per leg — ADR-036).
- The hub's `CallAdapter` receives the browser's `channel/open` as a call
operation, runs `AccessControl::check` with the browser's identity, then
forwards via `from_call` to the spoke (the hub as caller, the browser as
`forwarded_for` — ADR-026 §3).
- The spoke allocates its `channel_id` and returns it; the hub maps
browser-id ↔ spoke-id.
Transparent forwarding (passing the `channel/open` call operation through
without the hub's `CallAdapter` terminating it) would bypass the hub's
`AccessControl::check` and the `forwarded_for` auth chain — the hub would
not authenticate the open, and the spoke would see the browser as the direct
caller (not the hub), breaking the ADR-026/ADR-011 auth model. Translation
is the only option that preserves the auth model.
## Decision
### The hub translates, not transparently forwards
The hub's relay has two layers:
1. **Call-protocol layer (channel 0): translate.** The hub terminates
channel 0 on both legs. A `channel/open` from the browser is received by
the hub's `CallAdapter`, which:
1. Runs `AccessControl::check` on `channel/open` with the browser's
identity (bearer token resolved per ADR-034). If denied →
`channel:forbidden` to the browser.
2. Issues a *new* `channel/open` on the spoke's channel 0 via `from_call`,
with the hub as caller and the browser as `forwarded_for` (ADR-026
§3). The spoke's `AccessControl::check` sees the hub as the direct
peer (authorized per ADR-011) and the browser as `forwarded_for`.
3. The spoke allocates its `channel_id` and returns it.
4. The hub opens a matching channel on the browser's side (the hub is now
the *responder* for the browser leg, *initiator* for the spoke leg)
and records the `channel_id` mapping: `browser_id ↔ spoke_id`.
2. **Data-channel layer: byte-forward with `channel_id` rewrite.** Once the
mapping is established, the relay reads chunks for `browser_id` off the
browser's channels connection, rewrites the `channel_id` field to
`spoke_id`, and writes them onto the spoke's channels connection — and
vice versa. The relay does not parse the payload; it does not know if the
bytes are TTY chunks, SSH frames, or tunnel data. The channels layer on
each end does the chunk↔stream conversion; the relay just moves bytes
between two `AsyncRead + AsyncWrite` pairs with a 4-byte header rewrite.
### `channel_id` mapping
The hub maintains a `HashMap<channel_id, channel_id>` per (browser, spoke)
pair — the relay map. On `channel/open` (translated), the mapping is
inserted. On `channel/close` (translated the same way), the mapping is
removed. The relay task per channel reads the map to determine the rewrite
target.
`channel/control` operations on channel 0 carry `channel_id` in their JSON
payload (not in the chunk header). The hub's `CallAdapter` translates these
too: the browser's `channel/control` for `browser_id` is re-issued on the
spoke leg with `spoke_id` in the payload. The relay does not touch
`channel/control` — it's a call operation, translated by the hub's
`CallAdapter`, not byte-forwarded.
### What the hub runs
| Leg | What the hub runs |
|-----|-------------------|
| Browser leg | `ChannelsAdapter` (the relay's read/demux) + `CallAdapter` (channel 0, for the hub's own ops + translating the browser's ops) |
| Spoke leg | `ChannelsAdapter` + `CallAdapter` (same) |
| Relay | Per-channel byte-forward tasks with `channel_id` rewrite |
The hub never runs a handler for `alknet/tty`, `alknet/ssh`, or
`alknet/tunnel`. It runs `alknet/channels` (the relay) and `alknet/call`
(for its own hub-level operations + translation). The endpoints at each end
do the protocol work.
### What the hub still owns (unchanged from phase-0 §What the hub does still own)
- **Routing:** which spoke serves `container:abc123`? The hub's resource
registry / ownership store (ADR-011), queried via call operations on
channel 0. Channels doesn't touch this.
- **ACL at the hub:** does this browser's identity have `channel:open` scope
for `alknet/ssh` to `spoke-X`? `AccessControl::check` on `channel/open`,
run by the hub's `CallAdapter` before it forwards. Channels doesn't touch
this.
- **Relay lifecycle:** when a browser disconnects, the hub tears down the
spoke-side channels (and vice versa). `channel/close` on each channel, or
a transport-level close the channels layer observes (REQ-CH-02).
### Scope note: this is a hub-crate concern, not a channels-crate concern
This ADR defines the relay *contract* (translate channel 0, byte-forward
data channels with ID rewrite) so the channels crate's `ChannelManager`
exposes the interface the relay needs (`open_channel_stream(channel_id)
-> BiStream` for the byte-forward pumps). The relay *implementation*
lives in `alknet-hub` (or a downstream hub like alkapi), not in
`alknet-channels`. The channels crate is ALPN-blind and does not know it
is being relayed. The `channel_id` rewrite is a 4-byte field rewrite
within the 8-byte header (per ADR-035); the relay does not parse the
payload.
## Consequences
**Positive:**
- The auth model reuses cleanly: the hub's `AccessControl::check` +
`forwarded_for` (ADR-026) is the existing machinery, not a new one. The
spoke sees the hub as caller, the browser as `forwarded_for` — the
kernel/user-land + forwarded-for model from ADR-011.
- The relay is one pump function per channel, not per (protocol × transport)
cell. The hub's complexity is O(channels), not O(protocols × transports ×
spokes).
- The hub never runs protocol-specific handlers — it doesn't parse TTY
chunks, SSH frames, or tunnel data. It moves bytes and translates call
operations.
- `channel/resources/subscribe` (ADR-037) gives the hub a live view of each
spoke's resources, which the hub aggregates and exposes to the browser.
**Negative:**
- The hub maintains a `channel_id` mapping per (browser, spoke) pair. This
is per-channel state, not per-connection — a hub with many concurrent
browser sessions each with multiple channels has a non-trivial map. The
map is `HashMap<u32, u32>` per pair — cheap per entry, but the entry count
is (browsers × channels-per-browser). Bounded by `max_channels` (ADR-040)
per connection.
- The translate path adds one `channel/open` round-trip per relayed channel
(browser→hub, hub→spoke). This is the same cost as any hub-relayed call
operation and is not avoidable without transparent forwarding, which
breaks the auth model.
- `channel/control` translation requires the hub's `CallAdapter` to rewrite
`channel_id` in the JSON payload. This is a small but real translation
step — the hub is not a pure byte relay for channel 0.
## Door type
**One-way.** The translate-vs-forward decision is structural: transparent
forwarding would bypass the hub's `AccessControl::check` and the
`forwarded_for` chain, breaking the auth model. Reversing to transparent
forwarding after deployments exist would require re-architecting the hub's
auth path. The `channel_id` mapping strategy (`HashMap` per pair) is two-way
— an implementation detail that can change without breaking the contract.
## References
- ADR-024: peer-graph routing model (the hub's role)
- ADR-026: forwarded-for identity (the auth chain the translate path uses)
- ADR-034: outgoing-only X.509 and the three peer roles (browser identity
resolution)
- ADR-011: dynamic resource ownership (the ownership store the hub queries)
- ADR-036: channel 0 is pre-negotiated `alknet/call` (what the hub
terminates on each leg)
- ADR-037: channel lifecycle operations (what the hub translates)
- ADR-039: ChannelsAdapter and ChannelManager (the interface the relay uses)
- ADR-035: channels pure channel multiplexing (the 8-byte header the relay
reads/writes; the 4-byte `channel_id` rewrite; the `BiStream`-yielding
`open_channel_stream` interface)
- `docs/research/alknet-channels/phase-0-findings.md` §Hub Motivation,
§The hub relay, §OQ-CH-11
- `docs/architecture/crates/hub/README.md` — the hub crate (the relay
implementation's home)

300
docs/architecture/decisions/043-channelclient.md Normal file
View File

@@ -0,0 +1,300 @@
# ADR-043: ChannelClient — the Client Side of a Channels Connection
## Status
Accepted (amended 2026-07-12 — see "Amendment: transport-agnostic API"
below; amended 2026-07-16 — `connect_quic` removed per ADR-045 §5, see
"Amendment: `connect_quic` removed" below; **amended 2026-07-18 by
ADR-035 — `stream_types` field removed from `open_channel` and `Channel`;
the channels layer has no `stream_type` concept, see "Amendment
(ADR-035, 2026-07-18)" below**)
## Amendment (ADR-035, 2026-07-18)
The `stream_types: &[u8]` field is **removed** from `open_channel`'s
signature, and `pub stream_types: Vec<u8>` is **removed** from the
`Channel` struct. The channels layer has no `stream_type` concept
(ADR-035) — the handler owns its sub-stream multiplexing on the
`BiStream` it receives via `Channel.source` (a `ChannelBidiStreamSource`
whose `accept_bi` yields a `BiStream` per ADR-009). The handler's
sub-stream set is implicit in its ALPN's wire format (e.g., TTY's 5-byte
format declares its own `stream_type` set internally; the channels
layer carries the bytes transparently). The `into_sub_streams()` reference
in the `Channel.source` doc comment is moot — `into_sub_streams()` is
removed by ADR-035 (amending ADR-038).
The body below describes the **original** (with `stream_types`) shape;
the amendment above is the operative decision. See ADR-035 for the
resolution rationale and the cross-ADR impacts.
## Amendment: `connect_quic` removed (2026-07-16, per ADR-045 §5)
The `connect_quic(addr, credentials)` convenience constructor is
**removed**. Keeping it as a thin wrapper over
`AlknetClient::dial_quic` would make `alknet-channels-call` depend on
`alknet-client`, contradicting the dep graph (the protocol crates are
parallel to the dial, not downstream of it). Callers compose
`AlknetClient::dial_quic(...).await?` + `ChannelClient::from_connection(conn).await?`.
The `from_connection` primary constructor (the 2026-07-12 amendment
below) is unchanged and remains the one-way-door surface. The
`connect_quic` references in the body of this ADR are the historical
shape; they do not survive into the implementation. See ADR-045 §5 for
the full rationale and the breaking-change acknowledgment.
## Amendment: transport-agnostic API (2026-07-12)
The original Decision named `connect(addr: SocketAddr, credentials)` as the
primary constructor and framed it as "QUIC-only initially" — to be
generalized when a second transport's client exists. That framing welded
the client-side one-way-door API to QUIC, the same welding ADR-007 unwound
on the server side, and masked it as a two-way-door deferral
(anti-patterns #8, #9, #11). "Can be generalized later" meant "can be
rewritten later" — the expensive reversal the one-way-door classification
exists to prevent.
The channels protocol is transport-agnostic by design (ADR-034 substrate
modes; `Connection::from_stream`/`from_bidi`/`from_source` accept any
`AsyncRead + AsyncWrite`). The client side is half of that protocol and
must not be coupled to a transport. This amendment splits the constructor
surface:
- **`from_connection(connection: Connection)`** — the transport-agnostic
primary constructor and the one-way-door API. Takes a pre-established
`Connection` (produced by any transport — TCP+TLS via `from_bidi`,
WebTransport `BiStream`, SSH `direct-tcpip`, a quinn connection, a
WebSocket carrying `alknet/channels` per ADR-044), installs channel 0,
spawns the demux/mux, returns the client. Mirrors the server-side
`ChannelsAdapter::handle(Connection)` (substrate-agnostic) and the
existing `CallClient::spawn_dispatch(Connection)` pattern.
- **`connect_quic(addr, credentials)`** — a QUIC convenience constructor:
dial QUIC, then `from_connection`. Additive and two-way-door.
`connect_tcp_tls`, `connect_webtransport`, etc. join it as transports
are added, without touching the one-way-door surface.
The dial+TLS seam (the transport-specific work each dial helper does —
verifier selection per ADR-034, handshake, produce a `Connection`) is the
correct scope of OQ-55's deferral. `AlknetClient` is the eventual shared
*dial*; `from_connection` is the shared *channels-take-over*. Separating
them now, before the one-way-door API is cast, is the point — not a
deferral. The "QUIC-only initially" framing is removed; it was the
anti-pattern this amendment corrects.
The door-type classification is unchanged: `from_connection` is one-way
(the handler-facing surface), `connect_quic` is two-way (additive
convenience). The `AlknetClient` extraction remains deferred (OQ-55) — but
what is deferred is the shared *dial*, not a QUIC-welded client API.
## Context
Both sides of a channels connection do the demux/mux work. The server side
is a `ProtocolHandler` (`ChannelsAdapter::handle`, ADR-039). The client side
needs a symmetric type — `ChannelClient` — that takes over an established
transport `Connection`, runs the demux/mux, and exposes
`open_channel(alpn, params) -> Channel` to the application. This is the
channels analogue of `CallClient` (server: `CallAdapter`; client:
`CallClient`) in the call protocol.
The phase-0 research (`docs/research/alknet-channels/phase-0-findings.md`
§OQ-CH-14) clarified that there are two concerns here:
1. **`ChannelClient` (channels-specific):** the client type for channels
connections. Decision-ready — build it in `alknet-channels`, same shape
as `CallClient`. The *take-over* half is transport-agnostic
(`from_connection`); the *dial* half is transport-specific
(`connect_quic` and future transport helpers).
2. **`AlknetClient` (core, transport-polymorphic):** the shared *dial+TLS*
seam — the transport-specific work (open socket, TLS handshake, ADR-034
verifier selection, produce a `Connection`) that each transport's dial
helper rebuilds. Genuinely deferred — blocked on a second *transport's*
dial existing (OQ-55 tracks this correctly). A single QUIC dial
(`connect_quic`) does not give enough information to extract the
transport-polymorphic dial seam; two different transport dials do.
This ADR decides #1 — `ChannelClient`, with `from_connection` as the
transport-agnostic primary constructor and `connect_quic` as a transport-
specific dial helper. #2 (the shared `AlknetClient` dial+TLS seam) stays
deferred per OQ-55.
## Decision
### `ChannelClient` in `alknet-channels`
```rust
pub struct ChannelClient {
manager: ChannelManager,
// The transport-side demux/mux, running in a background task.
...
}
impl ChannelClient {
/// Transport-agnostic primary constructor. Takes a pre-established
/// `Connection` (any transport — TCP+TLS via `from_bidi`,
/// WebTransport BiStream, SSH direct-tcpip, a quinn connection, a
/// WebSocket per ADR-044), installs channel 0 (alknet/call), spawns
/// the demux/mux, and returns the client. Mirrors the server-side
/// `ChannelsAdapter::handle(Connection)`. This is the one-way-door
/// API surface — it must not be coupled to a transport (ADR-034,
/// ADR-007).
pub async fn from_connection(connection: Connection)
-> Result<Self, ChannelError>;
/// QUIC convenience constructor. Dials a QUIC connection to `addr`
/// on ALPN `alknet/channels` (credentials → TLS handshake,
/// ADR-034 verifier selection), then calls `from_connection`.
/// Additive and two-way-door — `connect_tcp_tls`,
/// `connect_webtransport`, etc. join it as transports are added.
///
/// **REMOVED per ADR-045 §5.** The dial is extracted into
/// `AlknetClient`; `connect_quic` is deleted, not delegated.
/// Callers compose `AlknetClient::dial_quic` + `from_connection`.
/// The `CallCredentials` parameter is moot — `CallCredentials` is
/// removed per ADR-012 (amended 2026-07-17); the dial consumes
/// `ConnectionCredentials` from `alknet-core`.
pub async fn connect_quic(
addr: SocketAddr,
credentials: CallCredentials, // REMOVED — CallCredentials is removed
) -> Result<Self, ChannelError>;
/// Open a data channel with the given ALPN and params. Sends
/// `channel/open` on channel 0, waits for the response, and returns
/// the channel's sub-streams.
pub async fn open_channel(
&self,
alpn: &str,
stream_types: &[u8],
params: Value,
direction: ChannelDirection,
) -> Result<Channel, ChannelError>;
/// Subscribe to the peer's resource updates. Returns a stream of
/// resource-set events (ADR-037 channel/resources/subscribe). Part of
/// the one-way-door handler-facing surface (see Door type below).
pub async fn subscribe_resources(&self)
-> Result<BoxStream<ResourceEvent>, ChannelError>;
/// The call-protocol connection on channel 0, for invoking channel
/// lifecycle operations and any other call ops the peer exposes.
pub fn call(&self) -> &CallConnection;
}
pub struct Channel {
pub channel_id: u32,
pub stream_types: Vec<u8>,
/// The sub-streams, accessible via the BidiStreamSource (accept_bi) or
/// into_sub_streams() — ADR-038.
pub source: ChannelBidiStreamSource,
}
```
### Transport-agnostic by construction
`ChannelClient` is the client side of the channels protocol, which is
transport-agnostic (ADR-034 substrate modes; ADR-007 `from_stream`/`from_bidi`). The primary constructor — `from_connection(connection: Connection)` — takes a pre-established `Connection` from any
transport and takes over channels establishment. This mirrors the
server-side `ChannelsAdapter::handle(Connection)`, which is
substrate-agnostic by the same mechanism: the server receives a
`Connection` (QUIC-native, TCP+TLS via `from_bidi`, WebTransport, SSH
`direct-tcpip`, …) and runs the demux loop unchanged; the client receives
a `Connection` the same way and runs the same logic from the dialing side.
`connect_quic(addr, credentials)` is a convenience over `from_connection`:
dial QUIC, then `from_connection`. It is additive and two-way-door.
Transport-specific dial helpers (`connect_tcp_tls`, `connect_webtransport`,
…) join it as transports are added — none of which touch the
`from_connection` contract. The dial helper set is open-ended by design.
This is the client-side analogue of the server-side generalization ADR-007
made. Welding the client's one-way-door API to QUIC would repeat the
welding ADR-007 explicitly unwound.
### Bidirectionality preserved
The channels protocol is bidirectional — either side can open a channel
(ADR-037 §direction semantics). `ChannelClient::open_channel` supports both
`ChannelDirection::InitiatorToResponder` and
`ChannelDirection::ResponderToInitiator`. The client is not "the client
side" in the sense of only initiating — it can also receive `channel/open`
requests from the peer (the peer initiates, the client's `ChannelManager`
responds). This mirrors the call protocol's operation overlay (each side
populates what operations they expose).
This means `ChannelClient` is not purely a "client" in the request/response
sense — it's one endpoint of a bidirectional channels connection. The name
`ChannelClient` follows the `CallClient` convention (the side that dialed),
not a request/response role.
### Relationship to `AlknetClient` (OQ-55 — deferred)
`ChannelClient`'s *API* is transport-agnostic — `from_connection` takes
a pre-established `Connection`. What is deferred (OQ-55) is the shared
*dial+TLS* seam (`AlknetClient`): the transport-specific work each dial
helper does — open a socket, run the TLS handshake, apply ADR-034's
verifier-selection rule, produce a `Connection`. That dial is genuinely
transport-specific (QUIC, TCP+TLS, WebTransport, raw TCP, SSH), and we have
one shape implemented (QUIC, in `connect_quic`). Extracting a QUIC-shaped
connector now and naming it `AlknetClient` would bake QUIC in as *the*
establishment shape — the same welding ADR-007 unwound on the server side.
This is why `from_connection` is the one-way-door surface and
`connect_quic` is a two-way-door convenience over it. `AlknetClient` (when
extracted, after a second transport's dial exists) becomes the shared
*dial*; `from_connection` stays the shared *channels-take-over*. The two
concerns are separated now, before the one-way-door API is cast.
The friction while `AlknetClient` is deferred is duplicated
verifier-selection boilerplate across dial helpers (~20 lines each) — not
duplicated capability and not a QUIC-welded client API.
## Consequences
**Positive:**
- `ChannelClient` gives the channels crate a symmetric client/server pair,
matching the call protocol's `CallAdapter`/`CallClient` shape.
- Bidirectionality is preserved — the client can both initiate and receive
`channel/open`.
- The `AlknetClient` deferral (OQ-55) is not blocked by `ChannelClient` —
they are independent concerns. `ChannelClient` builds standalone; the
core extraction happens later when the blocker clears.
**Negative:**
- Each transport-specific dial helper duplicates ~20 lines of
verifier-selection boilerplate (from `connect_quic`). This is the known
cost of not extracting `AlknetClient` yet (OQ-55). Acceptable until the
second transport's dial exists, at which point `AlknetClient` extracts
the shared dial+TLS seam. The `from_connection` API — the one-way-door
surface — is unaffected; only the dial helpers carry the duplication.
## Door type
**One-way.** The `ChannelClient::from_connection` / `open_channel` /
`call` / `subscribe_resources` API is the handler-facing surface;
changing it after consumers exist is a rewrite. `from_connection` is the
one-way-door primary constructor (transport-agnostic).
`connect_quic` (and future `connect_tcp_tls` / `connect_webtransport` /
…) are **two-way** doors — additive convenience constructors over
`from_connection`. Adding, removing, or changing a dial helper is cheap
and does not touch the one-way-door surface.
The `AlknetClient` extraction is a **deferred decision** (OQ-55,
deferred(scope)), not a door-type attribute. Its door type is two-way (the
extraction is a refactor, not a wire-format change), but it is not decided
in this ADR — see OQ-55 for the blocking condition. What is deferred is the
shared *dial+TLS* seam; `from_connection`'s transport-agnostic contract is
decided now.
## References
- ADR-037: channel lifecycle operations (`open_channel` sends `channel/open`)
- ADR-038: ChannelBidiStreamSource (what `Channel.source` wraps, as
amended by ADR-035 — `accept_bi` yields a `BiStream`)
- ADR-035: channels pure channel multiplexing (`stream_types` field
removed from `open_channel` and `Channel`; handler owns sub-stream
multiplexing)
- ADR-039: ChannelManager (the shared state `ChannelClient` holds)
- OQ-55: AlknetClient / client establishment extraction (the deferred core
concern this ADR does NOT block on)
- `docs/research/alknet-channels/phase-0-findings.md` §OQ-CH-14 (the
research-scope question this ADR carries forward)
- `docs/architecture/crates/call/client-and-adapters.md` — `CallClient` (the
shape `ChannelClient` mirrors)

251
docs/architecture/decisions/044-channels-subcrate-decomposition.md Normal file
View File

@@ -0,0 +1,251 @@
# ADR-044: alknet-channels Sub-Crate Decomposition
## Status
Accepted (amended 2026-07-18 by ADR-035 — the 9-byte wire format is now
8-byte; `ChannelSubStreams` / `SubStreamHandle` removed; the channels
layer has no `stream_type` concept — see "Amendment (ADR-035, 2026-07-18)"
below)
## Amendment (ADR-035, 2026-07-18)
The wire format in `channels-core` is now **8-byte** (not 9-byte); the
`ChannelSubStreams` / `SubStreamHandle` typed destructure accessor is
**removed** (the channels layer has no `stream_type` concept —
`accept_bi` yields a `BiStream`, and the handler owns its sub-stream
multiplexing). The channel 0 pre-negotiation in `channels-call` no
longer constructs "reassembly buffers with `stream_types` [0, 1]" — it
constructs one reassembly buffer for `channel_id = 0`, yielding a
`BiStream` to the `CallAdapter`. The "What moves where" table's "9-byte
wire format" row is now "8-byte wire format"; the `ChannelSubStreams`
row is removed. The two-crate split (`channels-core` /
`channels-call`), the dep graph, and the "hub and worker are consumers"
principle are unchanged. See ADR-035 for the resolution rationale.
## Context
The initial channels spec (ADR-034-080) framed `alknet-channels` as a
single crate depending on both `alknet-core` and `alknet-call`. The
dependency on `alknet-call` arises because channel lifecycle operations
(`channel/open`, `channel/close`, `channel/control`,
`channel/resources/subscribe`) register on the call protocol's
`OperationRegistry`, and channel 0 is pre-negotiated as `alknet/call`
(ADR-036).
This creates two issues:
1. **The dependency graph conflates the pure multiplexer with the call-
protocol coupling.** The wire format, demux/mux, and
`ChannelBidiStreamSource` are ALPN-blind and call-protocol-blind — they
depend on `alknet-core` only. The channel 0 pre-negotiation and lifecycle
op registrations are the call-protocol coupling. Baking both into one
crate means any consumer that wants the multiplexer also pulls in the
call-protocol coupling, even if they don't need channel 0 to be
`alknet/call`.
2. **The "no special-casing for downstream crates" principle is
violated.** The user's constraint: "we don't want to be doing anything
special for downstream crates unless it's needed/useful to the point of
they all kind of benefit." The call-protocol coupling is a channels-crate
concern (channels needs call for orchestration), not a downstream-crate
concern. Separating them makes the dependency graph honest: the pure
multiplexer has no opinion about channel 0; the call-protocol coupling
is isolated where it belongs.
The substrate simplification (ADR-034 revised) strengthened this: the wire
format is the same across all substrates, and the `ChannelsAdapter` is
substrate-agnostic. The pure multiplexer (`channels-core`) is cleanly
separable from the call-protocol orchestration (`channels-call`).
## Decision
### Two crates, not four
```
alknet-channels-core — the pure multiplexer (wire format, demux/mux,
│ ChannelBidiStreamSource, ChannelManager). Depends
│ on alknet-core only. ALPN-blind, call-protocol-blind,
│ transport-blind.
└── alknet-channels-call — channel 0 pre-negotiation + lifecycle op
registrations on the call protocol's
OperationRegistry. Depends on channels-core +
alknet-call.
```
There are no `channels-hub` or `channels-worker` sub-crates. The hub and
worker are **consumers** of channels, not sub-crates of it. The existing
`alknet-hub` crate (`docs/architecture/crates/hub/README.md`) IS the hub —
it depends on `channels-call` and uses the channels protocol as its
substrate. A worker is just a worker — it depends on `channels-call` and
uses `ChannelClient` (ADR-043) to dial. The hub relay logic (ADR-042) lives
in `alknet-hub`, not in a channels sub-crate.
### `alknet-channels-core`
The pure multiplexer. Contains:
- The 9-byte chunk wire format (`parse_header` / `write_header` — ADR-034)
- The demux/mux (`Demux`, `MuxHandle`/`MuxRunner` — ADR-039)
- `ChannelBidiStreamSource` (implements `BidiStreamSource` — ADR-038)
- `ChannelSubStreams` / `SubStreamHandle` (the typed destructure accessor)
- `ChannelManager` (the shared state — channel_id → ChannelState, but
**without** the `call_ops: Arc<OperationRegistry>` field; the manager is
ALPN-blind and call-protocol-blind)
- `ChannelsAdapter` (the `ProtocolHandler` on `alknet/channels` — the
read/demux loop, substrate-agnostic per ADR-034 revised)
Depends on `alknet-core` only. No `alknet-call` dependency. No opinion about
what channel 0 carries — that's the consumer's concern.
The `ChannelsAdapter::handle` in `channels-core` does NOT preinstall channel
0 as `alknet/call`. It runs the demux loop and routes chunks by
`channel_id`. Channel 0 is just another channel; what ALPN it carries is
determined by the consumer (the `channels-call` crate pre-negotiates it as
`alknet/call`; a hypothetical other consumer could pre-negotiate it
differently).
### `alknet-channels-call`
The call-protocol coupling. Contains:
- Channel 0 pre-negotiation as `alknet/call` (ADR-036) — the
`preinstall_channel_0` logic that constructs channel 0's reassembly
buffers with stream_types [0, 1] and hands the `Connection` to the
`CallAdapter`.
- The four lifecycle operations (ADR-037): `channel/open`,
`channel/close`, `channel/control`, `channel/resources/subscribe` —
registered on the call protocol's `OperationRegistry` at assembly time.
- `ChannelOperations` (the registration helper that closes over a
`ChannelManager` clone).
- `ChannelClient` (ADR-043) — the client-side type that dials a transport,
establishes the channels connection, and exposes `open_channel(alpn,
params) -> Channel`. This is the worker/client entry point; it lives here
because it needs channel 0 pre-negotiation (which is in `channels-call`).
Depends on `channels-core` + `alknet-call`. This is where the
call-protocol coupling lives, isolated from the pure multiplexer.
### Hub and worker are consumers, not sub-crates
The hub and worker are architectural roles, not channels sub-crates:
- **The hub** is the existing `alknet-hub` crate. It depends on
`channels-call` and uses the channels protocol as its substrate. The hub
relay logic (ADR-042 — translate `channel/open` on channel 0,
byte-forward data channels with `channel_id` rewrite) lives in
`alknet-hub`, alongside its existing peer lifecycle, aggregated env, and
service discovery responsibilities. There is no `channels-hub` sub-crate;
`alknet-hub` IS the channels hub.
- **A worker** is any crate that uses `ChannelClient` (ADR-043, in
`channels-call`) to dial a hub. There is no `channels-worker` sub-crate;
a worker depends on `channels-call` and uses `ChannelClient` directly.
The worker may be a CLI binary, a docker-side connector, an SSH-side
connector, or any other role that dials into a hub's channels connection.
This means the channels crate provides the substrate (`channels-core` +
`channels-call`); the hub and worker crates are consumers that build on it.
The dependency direction is: `alknet-hub` → `channels-call` →
`channels-core` → `alknet-core`; a worker → `channels-call` →
`channels-core` → `alknet-core`. The channels crate has no dependency on
`alknet-hub` or any worker crate.
### What moves where
| Component | Original (ADR-034-080) | Now |
|-----------|------------------------|-----|
| 9-byte wire format | `alknet-channels` | `channels-core` |
| Demux/Mux | `alknet-channels` | `channels-core` |
| `ChannelBidiStreamSource` | `alknet-channels` | `channels-core` |
| `ChannelSubStreams` | `alknet-channels` | `channels-core` |
| `ChannelManager` (without `call_ops`) | `alknet-channels` | `channels-core` |
| `ChannelsAdapter` (demux loop only) | `alknet-channels` | `channels-core` |
| Channel 0 pre-negotiation | `alknet-channels` (ADR-036) | `channels-call` |
| `channel/open`/`close`/`control`/`resources/subscribe` ops | `alknet-channels` (ADR-037) | `channels-call` |
| `ChannelOperations` registration helper | `alknet-channels` | `channels-call` |
| `ChannelClient` (ADR-043) | `alknet-channels` | `channels-call` |
| Hub relay (ADR-042) | `alknet-channels` (spec) | `alknet-hub` (the existing hub crate, consuming `channels-call`) |
### Relationship to `alknet-hub`
The existing `alknet-hub` crate (`docs/architecture/crates/hub/README.md`)
is the hub pattern: peer lifecycle, aggregated env, service discovery. With
channels as the substrate, `alknet-hub` gains a dependency on
`channels-call` and incorporates the relay logic (ADR-042). The hub spec
(`crates/hub/README.md`) will be updated to reflect that the hub uses
channels as its transport substrate — one channels connection per leg
(browser↔hub, hub↔spoke), with the relay translating `channel/open` and
byte-forwarding data channels. The hub's existing responsibilities (peer
lifecycle, aggregated env, service discovery, worker supervision) are
unchanged; channels is the substrate they run on.
This makes "channels hub" and "hub" the same thing — the hub IS built on
channels. There is no separate channels-hub concept.
## Consequences
**Positive:**
- The dependency graph is honest: `channels-core` is the pure multiplexer
with no call dependency; the call-protocol coupling is isolated in
`channels-call`. A consumer that wants the multiplexer without the
call-protocol orchestration can depend on `channels-core` only.
- The "no special-casing for downstream crates" principle is preserved:
`channels-core` doesn't know about `alknet-call`, `alknet-tty`, or any
handler crate. The call-protocol coupling is a channels-crate concern,
not a downstream-crate concern.
- Hub and worker are consumers, not sub-crates. The existing `alknet-hub`
crate IS the channels hub — it depends on `channels-call` and uses
channels as its substrate. A worker depends on `channels-call` and uses
`ChannelClient`. The channels crate has no dependency on `alknet-hub` or
any worker crate. This is the cleanest dependency direction: channels
provides the substrate; hub and worker consume it.
- The WASM and cross-platform story gets easier: `channels-core` is
WASM-compatible by construction (pure byte manipulation, no platform
deps); `channels-call` inherits the call protocol's WASM constraints; the
hub and worker crates are platform-specific as needed.
**Negative:**
- Two channels crates instead of one. The assembly layer must depend on
`channels-core` + `channels-call` instead of one `alknet-channels`. This
is the cost of the clean separation; the assembly layer already wires
multiple crates, so this is consistent with the existing pattern.
- The `ChannelsAdapter` in `channels-core` doesn't preinstall channel 0 —
the consumer does. This means `channels-core`'s `ChannelsAdapter::handle`
exposes a hook (callback or trait method) for the consumer to install
channel 0. `channels-call` provides the `preinstall_channel_0`
implementation; a different consumer could provide a different one. This
is a slightly more complex adapter shape than "channel 0 is always call,"
but it's the cost of the clean separation.
## Door type
**One-way (crate structure).** The two-crate split (`channels-core` /
`channels-call`) is one-way — once consumers depend on `channels-core`
without `channels-call`, re-merging them is a breaking change. The hub and
worker being consumers (not sub-crates) is also one-way — it establishes
the dependency direction (hub/worker → channels, not channels → hub/worker).
## References
- ADR-031: crate decomposition (no-handler-depends-on-another-handler —
preserved; the channels sub-crates depend on core/call, not on handlers)
- ADR-034: channels wire format (revised — substrate simplification; the
wire format is in `channels-core`; **amended by ADR-035 — 8-byte header,
no `stream_type`**)
- ADR-035: channels pure channel multiplexing (amends this ADR — 8-byte
wire format; `ChannelSubStreams` / `SubStreamHandle` removed; the
channels layer has no `stream_type` concept)
- ADR-036: channel 0 pre-negotiated (moves to `channels-call`)
- ADR-037: channel lifecycle operations (move to `channels-call`)
- ADR-038: ChannelBidiStreamSource (in `channels-core`; **amended by
ADR-035 — `into_sub_streams` removed, `accept_bi` yields `BiStream`**)
- ADR-039: ChannelsAdapter and ChannelManager (split: core demux in
`channels-core`, call coupling in `channels-call`)
- ADR-042: hub relay (in `alknet-hub` — the hub crate consumes
`channels-call`; there is no `channels-hub` sub-crate, per this ADR's
Decision §"No hub/worker sub-crates")
- ADR-043: ChannelClient (in `channels-call`; there is no
`channels-worker` sub-crate — a worker is any crate that uses
`ChannelClient` to dial)
- `docs/architecture/crates/hub/README.md` — the existing hub crate (the
relay's consumer)

557
docs/architecture/decisions/045-alknetclient-native-dial-seam.md Normal file
View File

@@ -0,0 +1,557 @@
# ADR-045: AlknetClient — the Native Client Dial Seam
## Status
Accepted (resolves OQ-55; §3 and §5 amended 2026-07-16 by ADR-012 —
the dial credential bundle is `ConnectionCredentials` (transport-level),
not `CallCredentials` (call-protocol-level); all three dial signatures
unify on `&ConnectionCredentials`; `dial_iroh`'s `node_id` parameter is
derived from `remote_identity`; the `auth_token` stays in the
call-protocol layer, not the dial; `CallCredentials` stays in
`alknet-call`, only `ConnectionCredentials`/`RemoteIdentity` move to
`alknet-core`; §5 further amended 2026-07-17 by ADR-012 —
`CallCredentials` is removed entirely (its `auth_token` field had no
reader); `auth_token` is a per-request payload field; the `from_call`
`credentials_auth_token` dead path is removed)
## Context
### The deferral and why it was valid
OQ-55 deferred `AlknetClient::dial()` — the transport-polymorphic client
dial seam — blocked on "a second transport's real dial existing." The
reasoning (recorded in the OQ-55 file and in
[channel-client.md](../crates/channels/channel-client.md) §"Relationship
to `AlknetClient`") was sound at the time: extracting a QUIC-shaped
connector and naming it `AlknetClient` would bake QUIC in as *the*
establishment shape — the same welding ADR-007 unwound on the server
side. Only one transport dial existed (`CallClient::connect` /
`ChannelClient::connect_quic`, both QUIC). The transport-polymorphic
seam was not extractable from two *different* transport implementations;
it was guessable from one.
### Why the deferral has collapsed
Three decisions landed since the deferral, each removing a blocker:
1. **ADR-086 (endpoint types)** named the native endpoint type and gave
it **two rustls-consuming transports**: QUIC (primary) and TCP+TLS
(fallback when UDP is blocked). Both consume `TlsClientConfig`; both
produce a `Connection` via `Connection::from_quinn_with_alpn` /
`Connection::from_bidi`. The native endpoint type also includes iroh
(key-based, not rustls-consuming). That is **three dial shapes within
one endpoint type** — two sharing `TlsClientConfig`, one using the
raw key directly. The OQ-55 blocking condition ("a second
transport's real dial existing") is met *within one endpoint type*,
not across two unrelated transports.
2. **ADR-087 (`TlsClientConfig`)** broke the circular hedge that linked
the TLS config to the dial. The client-side TLS config is extracted
and buildable today; it is a **prerequisite** for the dial, not a
consequence of it. Each transport-specific dial helper builds a
`TlsClientConfig` and passes it to its transport's connector. The
dial no longer waits on the TLS config; the TLS config is shared.
3. **ADR-083 (endpoint as accept-loop runner)** made the server side a
clean accept-loop runner that takes pre-built transports via
`with_quinn` / `with_iroh` / `with_tcp_tls`. The client-side analogue
— a dialer that takes pre-built transport handles and produces a
`Connection` — is now guessable by symmetry, not a shot in the dark.
The server side separates "build the transport" (assembly layer)
from "run the accept loop" (endpoint); the client side separates
"build the transport handle" (assembly layer) from "dial + produce
`Connection`" (`AlknetClient`).
The three together remove every blocker the deferral named. The dial
seam is extractable from two different rustls-consuming transport
implementations (QUIC + TCP+TLS) plus the key-based iroh path — three
real shapes, not one. The TLS config is shared. The server-side shape
gives the client-side shape by symmetry.
### The tangle this ADR also names
Three concept levels were conflated throughout the initial development,
contributing to the confusion that made `AlknetClient` hard to spec:
1. **Deployment role** — Hub / Worker / Hub-Worker. *Who accepts, who
dials, in the hub-and-spoke topology.* A hub accepts inbound and may
dial outbound (hub-as-client). A worker dials outbound and may
accept inbound (a hub-worker). A pure worker only dials.
2. **Establishment side** — `AlknetEndpoint` (server) / `AlknetClient`
(client). *Server-side accept vs. client-side dial.* The endpoint
accepts connections and resolves identity from the incoming
connection; the client dials and presents identity (client cert)
while verifying the remote (ADR-034).
3. **ALPN-level category** — endpoint ALPN / entry-point ALPN
(ADR-086 §2). *Identity-gated vs. bootstrap, at the TLS layer.*
These are orthogonal. A hub *uses* an `AlknetEndpoint` (server side) AND
*uses* an `AlknetClient` (client side, when dialing workers). A worker
*uses* an `AlknetClient` (client side) AND *may* use an `AlknetEndpoint`
(server side, if it accepts inbound). The role determines which side(s)
you instantiate, not what the side IS. `AlknetClient` is the client-side
establishment type — Layer 2 — independent of the deployment role that
uses it and of the ALPN-level category of the ALPN it dials.
## Decision
### 1. `alknet-client` is a new crate
`AlknetClient` lives in a new crate `alknet-client`, not in
`alknet-core` or `alknet-tls`. The dependency profile rules out the
alternatives:
- **`alknet-core` is ruled out by a cycle.** `AlknetClient` needs
`TlsClientConfig` from `alknet-tls`, and `alknet-tls` depends on
`alknet-core`. Putting `AlknetClient` in core creates
`alknet-core → alknet-tls → alknet-core` — a circular dependency.
- **`alknet-tls` is the wrong scope.** That crate is "TLS config + cert
sharing" (`TlsServerConfig` / `TlsClientConfig`), not "dial +
transport establishment." The dial calls `quinn::Endpoint::connect`,
`TcpStream::connect` + `TlsConnector::connect`, and
`iroh::Endpoint::connect` — transport-connection establishment, not
TLS config. Putting the dial in `alknet-tls` would weld transport
establishment to cert config, the same conflation ADR-082 untangled
on the server side.
- **Folding into `alknet-hub` / `alknet-worker`** would make both
depend on each other or duplicate the dial. Both roles need the dial;
the dial is not a role concern.
`alknet-client` depends on `alknet-core` (for `Connection`,
`CallCredentials`, `RemoteIdentity`, types) + `alknet-tls` (for
`TlsClientConfig`) + transport crates (quinn, tokio-rustls, iroh —
feature-gated). The DAG is clean:
`alknet-client → alknet-tls → alknet-core`. `alknet-hub` and
`alknet-worker` (and any assembly layer) depend on `alknet-client` for
the dial; `alknet-call` and `alknet-channels-call` do not — their
take-over APIs (`spawn_dispatch`, `from_connection`) consume the
`Connection` the dial produces, without knowing `AlknetClient` produced
it.
### 2. `AlknetClient` is the client-side analogue of `AlknetEndpoint`
`AlknetEndpoint` (ADR-083) is a multi-transport **accept-loop runner**:
it takes pre-built transport endpoints and runs their accept loops,
dispatching by ALPN. `AlknetClient` is a multi-transport **dialer**: it
takes pre-built transport handles and dials a remote endpoint on a
chosen ALPN, producing a `Connection` for the protocol take-overs to
consume.
The symmetry:
| Concern | `AlknetEndpoint` (server) | `AlknetClient` (client) |
|---------|---------------------------|-------------------------|
| Transports | `with_quinn` / `with_iroh` / `with_tcp_tls` — pre-built by the assembly layer | `with_quinn` / `with_iroh` / `with_tcp_tls` — pre-built by the assembly layer |
| Per-connection work | Accept → extract ALPN + fingerprint → `Connection` → `dispatch` | Dial → TLS handshake → `Connection` (ALPN + fingerprint carried) |
| Identity | Resolved *from* the incoming connection (fingerprint from client cert, or token on channel 0) | *Presented* (local `TlsIdentity` as client cert) + remote *verified* (ADR-034 — fingerprint pin or CA) |
| What it does NOT do | Run protocols — handlers do | Run protocols — `CallClient` / `ChannelClient` do |
| Config | `TlsServerConfig` (per endpoint type, built by assembly) | `TlsClientConfig` (per-dial, built from `CallCredentials`) |
`AlknetClient` produces a `Connection`; the protocol take-overs
(`CallClient::spawn_dispatch`, `ChannelClient::from_connection`) take
over from there. This is the exact analogue of `AlknetEndpoint`
producing a `Connection` for `ProtocolHandler::handle`.
### 3. Three dial methods, one per transport family
```rust
pub struct AlknetClient {
// Pre-built transport handles, all optional — the client dials
// with whichever transport the remote endpoint type implies.
#[cfg(feature = "quinn")]
quinn: Option<quinn::Endpoint>,
#[cfg(feature = "tcp")]
tcp_connector: Option<tokio_rustls::TlsConnector>,
#[cfg(feature = "iroh")]
iroh: Option<iroh::Endpoint>,
}
impl AlknetClient {
/// QUIC dial. Builds a `TlsClientConfig` from `credentials`
/// (ADR-034 verifier selection + ADR-084 provider), dials `addr`
/// on `alpn`, returns a `Connection` via
/// `Connection::from_quinn_with_alpn`. Feature-gated on `quinn`.
#[cfg(feature = "quinn")]
pub async fn dial_quic(
&self,
addr: SocketAddr,
server_name: &str,
alpn: &[u8],
credentials: &CallCredentials,
) -> Result<Connection, ClientDialError>;
/// TCP+TLS dial. Builds a `TlsClientConfig` from `credentials`,
/// connects `TcpStream`, wraps with `TlsConnector`, returns a
/// `Connection` via `Connection::from_bidi`. Feature-gated on `tcp`.
#[cfg(feature = "tcp")]
pub async fn dial_tcp_tls(
&self,
host: &str,
addr: SocketAddr,
alpn: &[u8],
credentials: &CallCredentials,
) -> Result<Connection, ClientDialError>;
/// Iroh dial. Dials `node_id` on `alpn` via the iroh endpoint. The
/// iroh path does NOT use `TlsClientConfig` — iroh has its own TLS
/// (shares the `Ed25519SecretKey`, not the rustls config —
/// ADR-087 §3). The verifier is iroh's `NodeId` match (fingerprint
/// pin by another name). Feature-gated on `iroh`.
#[cfg(feature = "iroh")]
pub async fn dial_iroh(
&self,
node_id: iroh::NodeId,
alpn: &[u8],
local_key: &alknet_core::config::Ed25519SecretKey,
) -> Result<Connection, ClientDialError>;
}
// NOTE: The signatures above are the ORIGINAL (pre-ADR-012) shapes.
// ADR-012 amends §3 — see the amendment note below.
```
The three dials share `CallCredentials` (the local identity + remote
identity + auth token bundle, from `Capabilities`). The two rustls dials
(QUIC, TCP+TLS) build a `TlsClientConfig` from the credentials; the
iroh dial uses the raw `Ed25519SecretKey` directly. This mirrors the
server side's "iroh shares the key, not the config" (ADR-082, ADR-087
§3) — the consistency is in the rule (ADR-034 verifier selection), not
in the type.
> **Amendment 2026-07-16 (ADR-012):** The dial credential bundle is
> `ConnectionCredentials`, not `CallCredentials`. `CallCredentials`
> couples the dial to the call protocol (its `auth_token` field is a
> call-protocol / hub-layer concept — bearer-token identity correlation
> for browsers and `alknet/register`, not a transport credential). The
> dial uses only the transport-identity dimensions (`local_identity` +
> `remote_identity`); those move to `ConnectionCredentials` in
> `alknet-core`. All three dial signatures unify on
> `&ConnectionCredentials`:
>
> ```rust
> dial_quic(addr, server_name, alpn, creds: &ConnectionCredentials) -> Connection
> dial_tcp_tls(host, addr, alpn, creds: &ConnectionCredentials) -> Connection
> dial_iroh(alpn, creds: &ConnectionCredentials) -> Connection
> ```
>
> The `node_id: iroh::NodeId` parameter on `dial_iroh` is removed — it
> is derived from `creds.remote_identity.fingerprint`
> (`ed25519:<hex>` → `NodeId::from_bytes`), the same extraction pattern
> the rustls dials use for the verifier. The consistency is now in both
> the rule (ADR-034) and the type. `CallCredentials` stays in
> `alknet-call` as the call-protocol credential bundle; the
> `auth_token` is a per-request field on `call.requested` payloads
> (set by the caller or the `from_call` forwarding handler, resolved by
> `Dispatcher::resolve_identity`), not a dial-level credential. See
> [ADR-012](012-connectioncredentials-decouple-dial-from-call.md).
### 4. The dial is transport-polymorphic across the native endpoint type
The native endpoint type (ADR-086) has QUIC + TCP+TLS (both
rustls-consuming) + iroh (key-based). `AlknetClient` dials all three.
The two rustls dials share `TlsClientConfig::new`; the iroh dial is the
exception. The dial is transport-polymorphic within the native endpoint
type — a native client can reach a native endpoint over QUIC, TCP+TLS
(when UDP is blocked), or iroh (relay-assisted p2p). The transport
choice is the caller's, driven by network conditions and the remote
endpoint's reachability.
### 5. `CallClient::connect` / `ChannelClient::connect_quic` are removed
The existing QUIC convenience constructors on `CallClient` and
`ChannelClient` (`connect` / `connect_quic`) are **removed**, not
delegated. Keeping them as thin wrappers over `AlknetClient::dial_quic`
would make `alknet-call` / `alknet-channels-call` depend on
`alknet-client` — contradicting the dep graph (§1: the protocol crates
are parallel to the dial, not downstream of it) and re-coupling every
`CallClient` user to `quinn` + `rustls` + the TLS verifier machinery,
the exact welding the extraction undoes. Duplicating the dial inline in
each convenience constructor would preserve the dep graph but defeats
the point of centralizing the dial.
The dial is a distinct concern from the protocol take-over.
`AlknetClient` is the single home for the dial; `CallClient` /
`ChannelClient` are the single home for the take-over. A caller that
wants the old one-liner shape composes two lines:
`client.dial_quic(...).await?` then
`CallClient::new(...).spawn_dispatch(conn)` (or
`ChannelClient::from_connection(conn).await?`). The one-way-door
surface is the `AlknetClient` dial + take-over pattern; the
per-protocol convenience constructors are gone, not retained as
two-way-door sugar.
This is a breaking change to `CallClient` / `ChannelClient`'s public
APIs. It is expected — the develop branch is a total rewrite addressing
issues not feasible to fix inline against `main`; there are no external
consumers to preserve compatibility for. The migration plan handles
the call-site updates.
**Consequence: `CallCredentials` / `RemoteIdentity` move to
`alknet-core`.** These types were in `alknet-call` because
`CallClient::connect` consumed them. With `connect` removed, the dial
(`AlknetClient`) is the consumer, and the dial must not depend on
`alknet-call` (§1). Both the call and channels clients need them (the
channels client takes `CallCredentials` for its own removed
`connect_quic`, and the dial takes them for all three dials). They move
to `alknet-core` — the shared-types crate, alongside `TlsIdentity` and
`AuthToken` which already live there. This is the cleaner of the two
options the original ADR-045 draft called a "two-way-door
implementation detail"; it is not implementation detail — it determines
the dep graph, and the dep graph requires it.
> **Amendment 2026-07-16 (ADR-012):** This consequence is superseded.
> `CallCredentials` does **not** move to `alknet-core` — it stays in
> `alknet-call` (it is the call-protocol credential bundle; its
> `auth_token` field is a call-protocol / hub-layer concept, not a
> transport credential). What moves to `alknet-core` is
> `ConnectionCredentials` (a new type carrying only the
> transport-identity dimensions: `local_identity` + `remote_identity`)
> and `RemoteIdentity`. The dial consumes `ConnectionCredentials`, not
> `CallCredentials`. All three dial signatures unify on
> `&ConnectionCredentials`. See
> [ADR-012](012-connectioncredentials-decouple-dial-from-call.md).
>
> **Further amendment 2026-07-17 (ADR-012):** `CallCredentials` is
> **removed**, not retained in `alknet-call`. The "stays in
> `alknet-call`" framing above is itself superseded: a trace of the code
> showed `CallCredentials.auth_token` had no reader (`connect()` read
> only `tls_identity` + `remote_identity`; `spawn_dispatch` takes no
> credentials; the `from_call` forwarding path's `auth_token` source was
> `OpSummary.credentials_auth_token: Option<String>`, always `None`,
> never connected to `CallCredentials.auth_token`). The original ADR-012
> rationale ("the `from_call` forwarding handler populates `auth_token`
> from `CallCredentials`") cited a code path that does not exist.
> `auth_token` is a per-request payload field — browsers send it in the
> WebSocket call payload; the HTTP gateway resolves bearer → `Identity`
> at its boundary (the call layer sees the identity, not the token);
> `Dispatcher::resolve_identity` reads `payload.get("auth_token")`.
> There is no call-protocol credential bundle. The `from_call`
> `credentials_auth_token` dead path is removed in the same pass. See
> [ADR-012](012-connectioncredentials-decouple-dial-from-call.md) §"`CallCredentials`
> is removed."
**Consequence: `FingerprintPinVerifier` moves to `alknet-tls`.** With
`connect` removed and the verifier-selection logic centralized in
`TlsClientConfig::new` (ADR-087), `FingerprintPinVerifier` has no
remaining home in `alknet-call`. It is a TLS concern (it implements
`rustls::client::danger::ServerCertVerifier`); moving it to
`alknet-tls` lets `alknet-call` shed its direct `rustls`,
`rustls-pemfile`, and `rustls-native-certs` deps entirely — `CallClient`
becomes a pure protocol crate (`{registry, identity_provider}` +
`spawn_dispatch`). See ADR-087 §5 (amended).
**Consequence: `ClientError` is removed.** The existing
`ClientError { Transport, TlsSetup, ConnectionClosed }` was produced
only by `connect` (`Transport` and `TlsSetup`) and by no current
`spawn_dispatch` path (`ConnectionClosed` is a `FrameError`/`StreamError`
variant internal to the dispatch loop, not a `CallClient` API error).
With `connect` gone, `ClientError` has no producing call site. It is
removed rather than left as a vestigial enum. If `spawn_dispatch` ever
gains a failure path, a fresh error type is cleaner than retrofitting
this one.
### 6. `alknet/register` is a dialable ALPN (entry point, wire protocol deferred)
`AlknetClient::dial_quic` / `dial_tcp_tls` can dial the `alknet/register`
ALPN — the native registration entry point, parallel to HTTP
registration (OQ-58) but without the HTTP layer. The connection is an
**entry point** (ADR-086 §2): accepted without an established peer
identity, authenticated per-request by the registration token (or open
for no-token registration). The dial is the same as any other ALPN; the
difference is the protocol that runs on the resulting `Connection`.
Two registration cases, both hub concerns and both optional:
- **Token registration** — a freshly-provisioned worker (docker,
vast.ai, runpod) generates its local identity, dials the hub on
`alknet/register`, presents the one-time registration token, and
enrolls its key. The hub creates a `PeerEntry` and returns a session
credential.
- **No-token (open) registration** — a hub that hosts public services
over channels, or a relay/gateway, accepts registration without a
token. The enrollment creates a `PeerEntry` with no token
requirement.
The `alknet/register` **wire protocol** (the handshake on the
`Connection` after the dial — what frames the client sends, what the
hub returns) ties into the call crate's ACL and the OQ-58 enrollment
model. It is **deferred** to a dedicated ADR — this ADR names the ALPN
and its entry-point role; it does not specify the wire protocol. The
HTTP registration endpoint (OQ-58) remains the first implementation;
`alknet/register` is the native analogue that removes the HTTP
dependency for workers that have no HTTP client.
### 7. OQ-55 is resolved for the native dial
OQ-55's blocking condition ("a second transport's real dial existing")
is met: the native endpoint type has two rustls-consuming transports
(QUIC + TCP+TLS) + iroh (key-based) — three dial shapes, two sharing
`TlsClientConfig`. The transport-polymorphic dial seam is extractable
from two different transport implementations. `AlknetClient` is that
seam, for the native case.
The **web/browser client** (WebSocket, HTTP — the browser bidirectional
path per ADR-044/048) was never what OQ-55 was about. The browser path
is a different client surface (the JS SDK / wasm), not a Rust dial. It
does not use `AlknetClient`; it negotiates TLS via the browser's
network stack and speaks the wire protocol over WebSocket. OQ-55
deferred the Rust transport-polymorphic dial; the browser path is out
of scope and always was. The non-Rust native clients (Node/Deno/Bun,
Python, wasm) that can negotiate TLS against an X.509 endpoint and
implement the wire protocols directly are also out of scope for
`AlknetClient` — `AlknetClient` is the Rust native client, one of
several possible native clients sharing the same wire protocols.
## What this does NOT change
- **`AlknetEndpoint` (ADR-083)** — the server side is unchanged. The
client is a new type, not a modification to the endpoint.
- **`TlsClientConfig` (ADR-087)** — the client-side TLS config is
unchanged. `AlknetClient` calls `TlsClientConfig::new` per-dial; the
config is a prerequisite, not a consequence of the dial (the
relationship ADR-087 established).
- **`CallClient::spawn_dispatch` / `ChannelClient::from_connection`**
— the take-over APIs are unchanged. They consume the `Connection`
the dial produces; they do not know `AlknetClient` produced it.
- **`RemoteIdentity`** — **moved to `alknet-core`** (see §5, as amended
by ADR-012). The location changes from `alknet-call` to `alknet-core`
so the dial does not depend on the call protocol. The call and
channels clients consume it from core. (`CallCredentials` is removed
per ADR-012's 2026-07-17 amendment — it is not moved, it is deleted;
its `auth_token` field had no reader. `ConnectionCredentials` is the
new transport-level credential bundle in core, carrying
`local_identity` + `remote_identity`.)
- **The channels substrate (ADR-034)** — unchanged. The dial produces a
`Connection`; the channels protocol runs on it.
- **ADR-086 (endpoint types / entry points)** — the endpoint-type model
is unchanged. `AlknetClient` is the client-side consumer of the
native endpoint type. The entry-point vs. endpoint ALPN distinction
(§2) governs which ALPNs the client can dial and whether identity is
required — `AlknetClient` dials both; the protocol on the resulting
`Connection` differs.
- **The hub's `supervise_worker` (hub README §"Dial")** — the hub's
supervision loop takes a `dial` closure that produces a
`Connection`. That closure can call `AlknetClient::dial_quic` /
`dial_tcp_tls` internally. The hub does not need to know
`AlknetClient` exists — the closure seam is preserved. The hub spec
is updated to note `AlknetClient` as the recommended dial producer
for the closure.
## Consequences
**Positive:**
- **OQ-55 is resolved.** The transport-polymorphic dial seam is
extracted, for the native case. The duplicated dial boilerplate
(the removed convenience constructors each rebuilt
`TlsClientConfig::new` + their transport's connector) is centralized
in `AlknetClient`. The friction the deferral accepted is removed.
- **`alknet-call` becomes a pure protocol crate.** With `connect`
removed and `FingerprintPinVerifier` moved to `alknet-tls` (§5),
`alknet-call` sheds its direct `quinn`, `rustls`, `rustls-pemfile`,
and `rustls-native-certs` deps. `CallClient` is `{registry,
identity_provider}` + `spawn_dispatch` — no TLS, no transport. Every
handler crate that uses `CallClient` stops transitively linking the
TLS/transport stack.
- **The client-side shape is symmetric with the server side.** A
reader who understands `AlknetEndpoint` (accept + dispatch) can
understand `AlknetClient` (dial + produce `Connection`) by
symmetry. The concept layers (role / side / ALPN-category) are
named, reducing the tangle that made the client hard to spec.
- **The hub-as-client case is first-class.** A hub that dials workers
(or another hub) uses `AlknetClient` — the same type a worker uses
to dial a hub. The role asymmetry (hub vs. worker) does not produce
a type asymmetry; both use the same client.
- **Transport selection is the caller's.** A native client that needs
QUIC-with-TCP+TLS-fallback dials QUIC first, falls back to TCP+TLS
on connection failure. `AlknetClient` provides both dials; the
fallback policy is a caller concern (or a future `dial_with_fallback`
helper — two-way-door).
- **`alknet/register` is named.** The native registration entry point
has a home in the ALPN registry, parallel to HTTP registration. The
wire protocol is deferred, but the ALPN and its entry-point role are
decided — a worker that has no HTTP client can register natively.
**Negative:**
- **Breaking change: `CallClient::connect` / `ChannelClient::connect_quic`
removed; `RemoteIdentity` / `FingerprintPinVerifier` relocated;
`CallCredentials` removed; `ClientError` removed.** Call sites that
used the convenience constructors must switch to `AlknetClient::dial_*`
+ `spawn_dispatch` / `from_connection`. Import paths for
`RemoteIdentity` change from `alknet_call` to `alknet_core`;
`CallCredentials` is removed (per ADR-012's 2026-07-17 amendment) —
callers pass `ConnectionCredentials` to the dial and, where needed,
`auth_token` as a per-request payload field. This is expected — the
develop branch is a total rewrite; there are no external consumers to
preserve compatibility for. The migration plan handles the call-site +
import updates.
- **A new crate.** `alknet-client` is one more crate in the workspace.
The cost is low (the dial is narrow), and the dependency profile
rules out the alternatives, but it is a new entry in the crate
graph.
- **The iroh dial is the exception.** It does not use
`TlsClientConfig` — iroh has its own TLS. The dial helper applies
the same ADR-034 rule via iroh's API (NodeId match). The
consistency is in the rule, not in the type. This is the same
exception as the server side (ADR-082, ADR-087 §3) — unavoidable,
and isolated to one dial method.
- **The `alknet/register` wire protocol is still deferred.** This ADR
names the ALPN and its role; the handshake protocol (token/no-token,
the frames, the `PeerEntry` creation, the session credential return)
is a separate ADR tied to OQ-58. A worker cannot register natively
until that ADR lands; the HTTP path (OQ-58) remains the first
implementation.
- **The ADR-086 entry-point/endpoint terminology is under-specified
as a general abstraction.** This ADR uses the current terms
(entry-point = no identity at TLS; endpoint = identity required) but
does not re-litigate them. The broader abstraction — that all
top-level ALPNs are "entry points to the endpoint," each handling
auth in its own way — is a separate conceptual refinement, not
this ADR's scope.
## Door type
**One-way (crate existence + dial seam).** `alknet-client` as the
shared client dial crate is structural — every outbound-dialing role
(hub, worker, hub-worker) depends on it. Reversing would mean
re-distributing the dial across crates, reintroducing the duplicated
boilerplate. The three-dial API (`dial_quic` / `dial_tcp_tls` /
`dial_iroh`) is one-way — changing the signatures after consumers exist
is a rewrite. The internal implementation (how `ConnectionCredentials`
feeds `TlsClientConfig::new`, how the iroh dial maps the
`Ed25519SecretKey`) is two-way. The `alknet/register` ALPN name is
one-way (wire compatibility); its wire protocol is two-way until the
dedicated ADR lands.
## References
- OQ-55 (resolved by this ADR) — `AlknetClient` / client establishment
extraction
- [ADR-083](083-endpoint-as-accept-loop-runner.md) — `AlknetEndpoint`
as multi-transport accept-loop runner; the server-side shape this
ADR mirrors on the client side
- [ADR-086](086-endpoint-types-and-entry-points.md) — endpoint types
(native has QUIC + TCP+TLS + iroh); entry-point vs. endpoint ALPN
distinction (§2)
- [ADR-087](087-tlsclientconfig-not-blocked-on-dial.md) —
`TlsClientConfig` not blocked on the dial seam; breaks the circular
hedge; the TLS config is a prerequisite for the dial
- [ADR-082](082-alknet-tls-extraction.md) — `TlsServerConfig` /
`TlsClientConfig` in `alknet-tls`; "iroh shares the key, not the
config"
- [ADR-007](007-connection-from-stream-generic-single-stream.md) —
`Connection::from_stream` / `from_bidi`; the server-side
generalization whose client-side analogue this ADR completes
- [ADR-034](034-outgoing-only-x509-and-three-peer-roles.md) —
client-side verifier selection (fingerprint pin vs CA vs fail-closed)
- [ADR-084](084-aws-lc-rs-crypto-provider.md) — aws-lc-rs crypto
provider on all paths
- [ADR-043](043-channelclient.md) — `ChannelClient::from_connection`
(the take-over `AlknetClient` feeds)
- [ADR-022](022-call-protocol-client-and-adapter-contract.md) —
`CallClient::spawn_dispatch` (the take-over `AlknetClient` feeds)
- OQ-58 — worker registration flow (the HTTP path; `alknet/register`
is the native analogue)
- `docs/architecture/crates/channels/channel-client.md` §"Relationship
to `AlknetClient`" — the deferral this ADR resolves

81
docs/architecture/open-questions.md Normal file
View File

@@ -0,0 +1,81 @@
---
status: draft
last_updated: 2026-08-12
---
# Open Questions
Open questions are tracked here, organized by theme. Each question has a
status, priority, and (when resolved) a resolution citing the ADR.
**Status values**: `open`, `resolved`, `deferred(scope)`, `deferred(unclear)`,
`partially resolved`, `dissolved`.
## Call Protocol
| OQ | Title | Status | Priority | Resolution |
|----|-------|--------|----------|------------|
| OQ-01 | Call protocol scope within a connection | resolved | medium | ADR-015 — stream model, multiplexing |
| OQ-02 | Operation path format and routing scope | resolved | medium | `/{service}/{op}` is the correct design |
| OQ-03 | Batch operation semantics | resolved | low | Correlated call.requested events |
| OQ-04 | Session-scoped operation registries | resolved | medium | ADR-019 — OperationEnv trait layering |
| OQ-05 | Abort cascade semantics for nested calls | resolved | high | ADR-020 |
| OQ-06 | Privilege model and authority context | resolved | high | ADR-017 |
| OQ-07 | Handler identity registration path and composition authority | resolved | high | ADR-018 |
| OQ-08 | Operation error schemas | resolved | high | ADR-016 |
| OQ-09 | Safe vault operations for call protocol exposure | resolved | high | ADR-010 — none exposed |
| OQ-10 | ~~Remote-safe marking shape~~ | dissolved | medium | ADR-024 — remote_safe/trusted_peer retired |
| OQ-11 | OperationAdapter error type (AdapterError variants) | resolved | medium | DiscoveryFailed, SchemaParse, Transport, Unauthorized, SamePeerCollision |
| OQ-12 | from_call re-import trigger | resolved | low | ADR-028 — manual free function |
| OQ-13 | from_call namespace collision | resolved | low | Same-peer = error; cross-peer dissolved (ADR-024) |
| OQ-14 | CallClient TLS client-auth | resolved | high | quinn client-auth; key-type-aware verification |
| OQ-15 | PeerRef::Any routing policy | resolved | low | Insertion-order first-match |
| OQ-16 | services/list-peers re-export semantics | resolved | low | Opt-in; services/list is own-ops-only |
| OQ-17 | Multi-hop federation | deferred(scope) | low | One-hop model is the commitment; multi-hop is a feature extension |
| OQ-18 | PeerId — crypto identity vs stable logical id | resolved | high | ADR-025 — PeerId = Identity.id (stable) |
| OQ-19 | Persistent peer registry | resolved | medium | ADR-025 — core trait + in-memory default; persistence adapters separate |
| OQ-20 | ~~API key asymmetry~~ | dissolved | medium | PeerEntry supports multiple credential paths |
| OQ-21 | X.509 outgoing-only case | resolved | medium | Three remote roles; PeerEntry asymmetry correct |
## Call Protocol — Pub/Sub (NEW)
| OQ | Title | Status | Priority | Resolution |
|----|-------|--------|----------|------------|
| OQ-22 | Call protocol pub/sub primitive — pub to go with sub | open | high | Not yet resolved. The call protocol has `subscribe` (consumer → producer: "send me a stream") but no `pub` (producer pushes to subscribers it didn't directly receive a call.requested from). Surfaced during channels spec work — `channel/resources/subscribe` (ADR-037) needs fan-out. See §"Pub/Sub Gap" below. |
### Pub/Sub Gap
The call protocol's `StreamingHandler` / `invoke_streaming()` path
(ADR-021) is point-to-point: a `call.requested` arrives, the handler
produces a stream of `call.responded` events back to that one caller.
There is no mechanism for a producer to push events to subscribers that
it didn't directly receive a `call.requested` from.
The first consumer that needs this is the channels `channel/resources/
subscribe` operation (ADR-037): the hub aggregates worker resources and
needs to push live updates to N browser subscribers when any worker's
resource set changes. Each browser's `subscribe` arrives on the hub's
channel 0, but the worker resource changes arrive on separate worker
connections. The hub needs to fan-out.
**Blocked on**: An ADR specifying the pub primitive's shape — topic-based
fan-out vs. producer-side push to existing subscriptions. This is the
next ADR to write (alkcall ADR-046).
## Channels
| OQ | Title | Status | Priority | Resolution |
|----|-------|--------|----------|------------|
| OQ-23 | Full channel-level flow-control windowing | deferred(scope) | low | Bounded-buffer decided (ADR-040); full windowing blocked on HOL-blocking deployment observation |
| OQ-24 | Channels add/strip API shape | open | low | Whether the 8-byte header add/strip is built into the read/write path or a standalone utility. The contract (ADR-035) is decided; the function surface is not |
## Core Types
| OQ | Title | Status | Priority | Resolution |
|----|-------|--------|----------|------------|
| OQ-25 | BiStream type definition | resolved | high | ADR-005 — trait, Connection parameter |
| OQ-26 | AuthContext resolution timing | resolved | high | ADR-003 — hybrid resolution |
| OQ-27 | ALPN string naming convention | resolved | medium | ADR-004 — alknet/ prefix |
| OQ-28 | Dynamic handler registration | resolved | low | ADR-019 — curated static, overlays dynamic |
| OQ-29 | Handler-level auth resolution observability | resolved | medium | set_identity() on Connection for observability |
| OQ-30 | Dynamic resource ownership | resolved | high | ADR-011 — OwnershipProvider, resource_id_path |

992
docs/architecture/operation-registry.md Normal file
View File

@@ -0,0 +1,992 @@
---
status: draft
last_updated: 2026-07-09
---
# Operation Registry
OperationSpec, Handler, OperationRegistry, AccessControl, service discovery, and the hand-rolled framing (no irpc — ADR-014).
## What
The operation registry maps operation names to specs and handlers. It is the dispatch core of the call protocol — when a `call.requested` event arrives, the registry looks up the operation by name, checks access control, invokes the handler, and returns the result.
The registry is **layered by trust boundary** (ADR-019): a static, immutable curated layer (`Local` provenance, registered at startup) plus dynamic overlays for session ops (`Session` provenance, per-session) and imported ops (`FromCall` etc., per-connection). The immutability claim that previously applied to the whole registry is now scoped to the curated layer — see ADR-019 for the layering model and the rationale for why immutability is the security control for composing ops but not for imported leaves.
## Why
The operation registry provides:
- **Discoverability**: Clients can query `/services/list` and `/services/schema` to learn what operations exist before calling them
- **Access control**: Each operation declares its required scopes and resources; the registry enforces ACL before invoking the handler
- **Type safety**: JSON Schema for input and output enables validation and client code generation
- **Composability**: Handlers can invoke other operations through `OperationEnv` (local dispatch — remote dispatch is a separate architectural concern, see Constraints)
The registry design is informed by the `@alkdev/operations` TypeScript package, which demonstrated the same capabilities in JavaScript runtimes. The Rust implementation in alknet-call is canonical — it preserves the behavioral contract (namespace + operation name → invoke with input, return output) while defining the adapter contract (from_*, to_*) in Rust (see ADR-033).
## Architecture
### OperationSpec
Every registered operation has a spec that declares its name, type, schemas, and access control:
```rust
pub struct OperationSpec {
pub name: String, // e.g., "fs/readFile", "agent/chat" (no leading slash)
pub namespace: String, // e.g., "fs", "agent"
pub op_type: OperationType, // Query, Mutation, Subscription
pub visibility: Visibility, // External (wire-callable) or Internal (composition-only)
pub input_schema: Value, // JSON Schema for input
pub output_schema: Value, // JSON Schema for output
pub error_schemas: Vec<ErrorDefinition>, // Declared domain errors (ADR-016)
pub access_control: AccessControl,
/// JSON pointer into the input for the resource ID, when
/// `access_control.resource_type` is set and the operation targets a
/// specific runtime-spawned resource (ADR-011). e.g., `"$.containerId"`
/// for `docker/container/exec`. Absent for no-specific-resource
/// operations (the `list` case — scope-gate + result-filter). The
/// dispatcher extracts the resource ID from the input using this path
/// and passes it to `AccessControl::check`. `None` for operations
/// with no `resource_type` or with static resource sets.
pub resource_id_path: Option<String>,
}
pub enum OperationType {
Query, // Read-only, idempotent (e.g., "fs/readFile", "services/list")
Mutation, // Side effects (e.g., "bash/exec", "github/authenticate")
Subscription, // Streaming (e.g., "agent/chat", "events/subscribe")
}
pub enum Visibility {
External, // Callable from the wire (call.requested from a client)
Internal, // Composition-only (env.invoke from a handler)
}
/// A declared operation-level error. See ADR-016.
pub struct ErrorDefinition {
pub code: String, // e.g., "FILE_NOT_FOUND", "RATE_LIMITED"
pub description: String, // Human-readable description
pub schema: Value, // JSON Schema for the error detail payload
pub http_status: Option<u16>, // HTTP status for adapter projection (from_openapi/to_openapi)
}
```
Operation names use slash-based paths without a leading slash, aligned with URL path conventions: `fs/readFile`, `agent/chat`, `services/list`. The leading slash is added when needed for display (`spec.path()` returns `/fs/readFile`) and for wire format (the `call.requested` payload uses `/fs/readFile`). See OQ-13 for the path format decision (single-node `service/op` vs head/worker `node/service/op`).
The `namespace` field is derived from the name: for `fs/readFile` it's `fs`, for `agent/chat` it's `agent`. It's a convenience accessor for ACL matching and service grouping.
Visibility (ADR-017) controls whether an operation is callable from the wire. `External` operations are wire-facing — they appear in `services/list` and accept `call.requested` from clients. `Internal` operations are composition-only — they return `NOT_FOUND` (not `FORBIDDEN`) when called from the wire, and do not appear in `services/list`. The assembly layer declares visibility at registration. All import adapters (`from_openapi`, `from_mcp`, `from_jsonschema`, `from_call`) register operations as `Internal` by default (they're composition material, not directly callable); the handler that composes them is `External`. (`from_jsonschema` is now a real HTTP-backed adapter in `alknet-http` per ADR-027, not the schema-only placeholder it was.)
### AccessControl
```rust
pub struct AccessControl {
pub required_scopes: Vec<String>, // AND-checked: caller must have ALL
pub required_scopes_any: Option<Vec<String>>, // OR-checked: caller must have at LEAST ONE
pub resource_type: Option<String>, // e.g., "service", "container"
pub resource_action: Option<String>, // e.g., "read", "exec"
}
```
`AccessControl::check` consults an ownership provider for runtime-spawned
resources (ADR-011). The signature:
```rust
impl AccessControl {
/// `ownership` is None when the operation has no `resource_type`
/// (pure scope check) or when no ownership provider is wired
/// (the static `Identity.resources` path — backward compatible).
/// `resource_id` is None for the `list` case (resource_type set,
/// `resource_id_path` absent — scope-gate + result-filter, ADR-011 §4a).
pub fn check(
&self,
identity: Option<&Identity>,
resource_id: Option<&str>,
ownership: Option<&dyn OwnershipProvider>,
) -> bool {
// 1. Scope check (unchanged): identity.scopes ⊇ required_scopes.
// If identity is None and scopes are required, deny here.
// 2. Resource check (only if self.resource_type is Some):
// a. resource_id Some + ownership Some:
// → p.owns(identity?, resource_type, resource_id, resource_action)
// b. resource_id None + ownership Some (the `list` case):
// → p.owns_any(identity?, resource_type) [scope-gate]
// c. ownership None → fall back to static
// identity.resources[resource_type] ∋ resource_action
// (backward compat for non-runtime resources)
}
}
```
The `OwnershipProvider` trait (read side, sync — called on the dispatch hot
path) and the `OwnershipStore` trait (write side, async — called by
handlers that manage resource lifecycles) are defined in `alknet-core` per
ADR-011's storage decision (fourth instance of the repo/adapter pattern,
ADR-033). See [auth.md](../core/auth.md) §"Ownership Provider and Store"
for the trait shapes and the in-memory default adapter.
**The ownership provider is carried on `OperationContext`** (or threaded
by the dispatcher), populated by the dispatch path from the registry's
wiring. When `ownership` is `None`, `check` falls back to the static
`Identity.resources` path — operations with static resource sets work
unchanged. The ownership provider is an additional check, not a
replacement.
**The `resource_id` parameter** is extracted by the dispatcher from the
operation input using `OperationSpec.resource_id_path` (ADR-011 §2a).
When the spec has no `resource_id_path` (the `list` case), the dispatcher
passes `resource_id: None`, and `check` takes the scope-gate path. The
handler is separately responsible for result-filtering via
`OwnershipProvider::owned_resources` (ADR-011 §4a).
When a `call.requested` event arrives:
1. The `CallAdapter` resolves the caller's `Identity` from `AuthContext` (and possibly an `AuthToken` in the payload)
2. The registry checks operation **visibility** — if the operation is `Internal`, returns `call.error` with code `NOT_FOUND` (does not leak existence)
3. The dispatcher extracts `resource_id` from the input via `spec.resource_id_path` (if present)
4. The registry checks `access_control.check(identity, resource_id, ownership)` — for external calls (`internal: false`), ACL runs against the **caller's identity**; for internal calls (`internal: true`), ACL runs against the **handler's identity** (ADR-017)
5. If access is denied, the adapter returns `call.error` with code `FORBIDDEN`
6. If the relevant identity is `None` and the operation has restrictions, the adapter returns `call.error` with code `FORBIDDEN` and message `"authentication required"`
Operations with empty `AccessControl` (no required scopes, no resource checks) are accessible to all callers, including unauthenticated ones.
**Internal calls and authority context**: When a handler invokes another operation through `OperationEnv`, the nested call is marked `internal: true`, meaning it originated from composition (not from a wire request). The `internal` flag switches the authority context: the ACL check runs against the composing handler's `handler_identity` (set at registration), not the caller's identity and not as a blanket skip. This prevents privilege escalation through composition — a handler can only compose operations its own identity is authorized for. See ADR-017.
**Composition and dynamic ownership (ADR-011 §4d)**: When a handler composes an operation that targets a runtime-spawned resource (e.g., a coordinator composing `docker/container/exec` against a specific container), two checks must pass: (a) the coordinator's `CompositionAuthority` has the `container:exec` scope (static, ADR-017/022 unchanged), and (b) the coordinator owns this specific container (dynamic, ownership provider). The composition authority stays static — it doesn't grow a dynamic path. The ownership store handles the dynamic resource-level check. Both must pass; they're orthogonal. ADR-017 and ADR-018 are unchanged.
### Handler
There are two handler types, one per dispatch shape — mirroring the
TypeScript prior art (`@alkdev/operations/src/types.ts:62-78`:
`OperationHandler` returns a single value; `SubscriptionHandler` returns an
`AsyncGenerator`). The split is locked by ADR-021.
```rust
/// Request/response handler — Query and Mutation operations.
pub type Handler = Arc<
dyn Fn(Value, OperationContext) -> Pin<Box<dyn Future<Output = ResponseEnvelope> + Send>>
+ Send + Sync,
>;
/// Streaming handler — Subscription operations. Returns a stream of
/// ResponseEnvelopes: each Ok(value) → call.responded, an Err → call.error
/// (terminal — stream ends), natural stream end → call.completed.
pub type StreamingHandler = Arc<
dyn Fn(Value, OperationContext)
-> Pin<Box<dyn Stream<Item = ResponseEnvelope> + Send>>
+ Send + Sync,
>;
/// Type alias for the boxed stream shape used by `invoke_streaming()` and
/// `StreamingHandler` return values. The concrete library
/// (`futures::stream::BoxStream<'static, T>` = `Pin<Box<dyn Stream<Item = T>
/// + Send>>`) is a two-way-door implementation detail (ADR-021); the alias
/// exists so the two spellings (the expanded form in `StreamingHandler` and
/// the short form in `invoke_streaming()`) refer to the same type.
pub type ResponseStream = Pin<Box<dyn Stream<Item = ResponseEnvelope> + Send>>;
```
Both handlers are async — many operations (file I/O, HTTP service calls,
LLM streaming) are inherently asynchronous. A handler (whether it wraps a
local function, an HTTP-backed OpenAPI operation, an LLM stream, or a
`from_call` remote) is a `Future` (for `Query`/`Mutation`) or a `Stream`
(for `Subscription`, ADR-021). The registry's `Handler` /
`StreamingHandler` trait objects (ADR-021) abstract over this. A handler
receives:
- `input: Value` — the deserialized `payload` from the `call.requested` event
(always `serde_json::Value`)
- `context: OperationContext` — request ID, identity, metadata, env
The **`Handler`** (request/response) returns a single `ResponseEnvelope`
containing the result or an error. `ResponseEnvelope` is defined in
[call-protocol.md](call-protocol.md#responseenvelope) — it carries the request
ID and a `Result<Value, CallError>`. Local dispatch produces it with no
serialization overhead; the `CallAdapter` converts it to `EventEnvelope` for
the wire.
The **`StreamingHandler`** (streaming) returns a `Pin<Box<dyn Stream<Item =
ResponseEnvelope> + Send>>` — the stream analogue of `Handler`'s
`Pin<Box<dyn Future<...>>>`. Each `Ok(value)` in the stream becomes a
`call.responded` event; an `Err` becomes a `call.error` event (terminal — the
stream ends after it); natural stream end becomes `call.completed`. The
dispatch path converts each `ResponseEnvelope` to `EventEnvelope` exactly as
it does for the single-response case — no new wire-format concept is
introduced. See ADR-021 and [call-protocol.md](call-protocol.md) §"CallAdapter
Stream Handling".
When a handler returns an error, the `CallError.code` is matched against the operation's declared `error_schemas` (ADR-016). If the code matches a declared `ErrorDefinition`, the `call.error` event carries that code and the error's detail payload. If it doesn't match, the `call.error` carries `INTERNAL`. This is how handler failures become typed errors on the wire instead of string-matched messages. The same matching applies to `Err` values yielded by a `StreamingHandler`.
A `make_streaming_handler()` helper (analogue of `make_handler()`) wraps a
stream-producing closure into a `StreamingHandler`:
```rust
pub fn make_streaming_handler<S, St>(f: S) -> StreamingHandler
where
S: Fn(Value, OperationContext) -> St + Send + Sync + 'static,
St: Stream<Item = ResponseEnvelope> + Send + 'static,
{
Arc::new(move |input, context| Box::pin(f(input, context)))
}
```
### OperationContext
```rust
pub struct OperationContext {
pub request_id: String,
pub parent_request_id: Option<String>,
pub identity: Option<Identity>, // Caller's identity (inbound — who invoked me)
pub handler_identity: Option<CompositionAuthority>, // Handler's composition authority (ADR-018)
pub forwarded_for: Option<Identity>, // Original caller when forwarded (ADR-026, metadata only — NOT used by AccessControl::check)
pub capabilities: Capabilities,
pub metadata: HashMap<String, Value>,
/// Reachability set — the operations this handler may compose.
/// Populated from the registration bundle's `scoped_env` (ADR-018).
/// The reachability check in `OperationEnv::invoke()` consults
/// `scoped_env.allows(&name)`. This is data, not a dispatch trait.
pub scoped_env: ScopedPeerEnv,
/// Composition dispatch trait. A handler calls `env.invoke(...)` to
/// compose child operations. This is `Arc<dyn OperationEnv>` (a trait
/// object), not a concrete struct — the trait-object design is what
/// enables registry layering (ADR-019): the CallAdapter composes the
/// root env per call from the active layers (curated base + connection
/// overlay + session overlay), and session/connection overlays wrap
/// the base via trait layering. Same pattern as `IdentityProvider`
/// (ADR-003). See ADR-019.
pub env: Arc<dyn OperationEnv + Send + Sync>,
/// Abort policy for this call's descendants (ADR-020 Decision 6).
/// Default `AbortDependents` — aborting this request aborts all
/// non-terminal descendants. `ContinueRunning` is an opt-in for
/// long-running work that should survive a parent's abort. Set by the
/// composing handler via `OperationEnv::invoke()` (or
/// `invoke_with_policy()`), not by the wire caller.
pub abort_policy: AbortPolicy,
/// Deadline for this call and all descendants. Set by `build_root_context`
/// to `now + CallAdapter.default_timeout` (default 30s). Composed calls
/// inherit the parent's deadline (children do not get a fresh 30s — the
/// root call's deadline bounds the entire call tree). A composed call
/// that exceeds the deadline is cancelled (future dropped, `Drop` guards
/// release resources). `None` means no deadline (unbounded — used for
/// long-running subscriptions). See call-protocol.md → Timeouts.
pub deadline: Option<Instant>,
/// Composition-origin flag. Set by `OperationEnv::invoke()` (true) or the
/// `CallAdapter` dispatch path (false) — never by handlers. Module-private
/// for writes; read via `is_internal()`. See ADR-017.
pub(crate) internal: bool,
}
/// Abort cascade policy for a call's descendants (ADR-020).
///
/// `AbortDependents` (default): aborting this call cascades to all
/// non-terminal descendants.
///
/// `ContinueRunning` (opt-in): descendants that have already started
/// continue to completion; descendants that haven't started are aborted;
/// no new descendants start.
pub enum AbortPolicy {
AbortDependents,
ContinueRunning,
}
impl Default for AbortPolicy {
fn default() -> Self { Self::AbortDependents }
}
impl OperationContext {
pub fn is_internal(&self) -> bool { self.internal }
}
```
- `request_id`: Correlates with the `call.requested` event's `id` field
- `parent_request_id`: Set when this call was initiated by another operation (via `OperationEnv`). Records the agency chain — the call tree is the principal→agent chain (ADR-017)
- `identity`: The authenticated caller (from `IdentityProvider`) — inbound auth (who is calling me). For external calls, this is who sent the `call.requested`. For internal calls, this is the parent handler's `handler_identity` (propagated through `OperationEnv::invoke()`)
- `handler_identity`: The composition authority of the handler processing this call. `None` for leaves (`FromOpenAPI`, `FromMCP`, `FromCall`) — they don't compose. `Some(...)` for `Local` and `Session` ops that can compose children. For internal calls (`internal: true`), the ACL check runs against this authority (ADR-017, ADR-018). This is NOT a peer `Identity` — it's a declared authority bundle set at registration by the assembly layer
- `forwarded_for`: The original caller when this call was forwarded by a `from_call` handler (ADR-026). **Metadata only** — `AccessControl::check` never reads it; the ACL always authorizes the direct caller's `identity`. Handlers may read it for logging, auditing, per-user rate limiting, or application context. Populated from `call.requested.forwarded_for` by the dispatch path; set to `None` for composed children (wire-ingress only). The forwarder's claim, not a verified identity — a malicious hub can lie (same property as HTTP `X-Forwarded-For`). See ADR-026.
- `capabilities`: Outbound credentials the handler may use (decrypted API keys, scoped vault access) — see [Capability Injection](#capability-injection) below
- `metadata`: Request-scoped context (tracing IDs, connection info). **Must not hold secret material** — see ADR-010. **Does not propagate through `OperationEnv::invoke()`** — nested calls get fresh metadata. The tracing link between parent and child is `parent_request_id`, not metadata propagation. Anything a handler needs to pass to a child goes in the call `input`.
- `scoped_env`: The reachability set — the operations this handler may compose. Populated from the registration bundle's `scoped_env` (ADR-018). The reachability check in `OperationEnv::invoke()` consults `scoped_env.allows(&name)`. This is *data* (a `ScopedPeerEnv` struct), not a dispatch trait. `None`/empty for leaves.
- `env`: The composition dispatch trait (`Arc<dyn OperationEnv + Send + Sync>`). A handler calls `context.env.invoke(...)` to compose child operations. This is a trait object, not a concrete struct — the trait-object design enables registry layering (ADR-019): the CallAdapter composes the root env per call from the active layers (curated base + connection overlay + session overlay), and overlays wrap the base via trait layering. Same pattern as `IdentityProvider` (ADR-003). See ADR-019.
- `internal`: When `true`, this call originated from composition (a handler calling another operation via `OperationEnv`), not from a wire request. This switches the authority context: ACL runs against `handler_identity`, not `identity`. The `internal` field uses module-private construction — handlers construct `OperationContext` through `OperationEnv::invoke()` which sets `internal: true`, or through the `CallAdapter` dispatch path which sets `internal: false`. The field is not `pub` for writes; only `pub fn is_internal(&self) -> bool` is exposed for reads. See ADR-017.
`identity` and `capabilities` are orthogonal: identity is inbound (who is calling me), capabilities are outbound (what credentials I can use). `identity` and `handler_identity` are the principal/agent pair: `identity` is the principal (who delegated), `handler_identity` is the agent (who is acting). See ADR-010 for capabilities, ADR-017 for the privilege model, and ADR-018 for the composition authority type.
### OperationRegistry
```rust
pub struct OperationRegistry {
operations: HashMap<String, HandlerRegistration>,
}
```
The registry maps operation names to `HandlerRegistration` bundles. The curated layer (Layer 0) is a `HashMap<String, HandlerRegistration>`; session and connection overlays (Layers 1 and 2) are separate maps that the `CallAdapter` composes into the per-call `OperationContext.env` (ADR-019). See ADR-018 for the full registration model and ADR-019 for the layering model. Key methods:
- `register(registration)`: Add an operation to the curated layer at startup. Validates `handler` is the right `HandlerKind` for `spec.op_type` (Once for Query/Mutation, Stream for Subscription — ADR-021). Mismatch is a startup error.
- `registration(name)`: Find a registration by operation name (checks active overlays first, then curated base — ADR-019). Returns spec, handler (`HandlerKind`), provenance, composition authority, scoped env, capabilities.
- `invoke(name, input, context)`: Look up, check ACL, invoke handler, return a single `ResponseEnvelope` (request/response path — Query/Mutation). **Errors with `INVALID_OPERATION_TYPE` if the op is a `Subscription`** — `invoke()` is the wrong dispatch path for streaming ops; use `invoke_streaming()` (ADR-021).
- `invoke_streaming(name, input, context)`: Look up, check ACL, invoke streaming handler, return a `ResponseStream` (the boxed stream alias — ADR-021) (streaming path — Subscription). Pre-handler errors (not-found, forbidden, `INVALID_OPERATION_TYPE` for a non-Subscription op) yield a single error `ResponseEnvelope` and end the stream. See ADR-021.
- `list_operations()`: Return all registered specs (for `/services/list` — returns curated + active overlay ops)
### Request ID Generation
Request IDs correlate `call.requested`/`call.responded` events and index the
abort-cascade tree (`PendingRequestMap` is keyed by request ID, ADR-020).
- **Wire calls**: the root `OperationContext.request_id` is the `id` field
from the wire `call.requested` event (generated by the client).
- **Composed calls**: `OperationEnv::invoke()` generates a new `request_id`
for each child via `generate_request_id()` — a UUID v4 (or
`parent_id + "-" + counter`). Deterministic IDs (e.g.
`format!("env-{name}")`) **must not** be used — they collide across
concurrent invocations of the same operation, corrupting
`PendingRequestMap` correlation and the abort-cascade tree.
- **Wire visibility**: composed child `request_id`s are **internal** — they
appear in `PendingRequestMap` for abort-cascade indexing but are not sent
as `call.requested` to any peer. The client only sees `call.aborted` for
the root ID it sent; the server cascades internally to descendants. The
exception is `from_call` ops, which generate their own wire ID when
forwarding to the remote node (the remote node's `PendingRequestMap`
indexes it).
### HandlerRegistration
The registration bundle carries everything the dispatch path needs to construct an `OperationContext`. See ADR-018 for the full rationale.
```rust
pub struct HandlerRegistration {
pub spec: OperationSpec,
pub handler: HandlerKind, // Once or Stream — validated against spec.op_type (ADR-021)
pub provenance: OperationProvenance,
pub composition_authority: Option<CompositionAuthority>, // None for leaves
pub scoped_env: Option<ScopedPeerEnv>, // None for leaves
pub capabilities: Capabilities,
// NOTE: ADR-023 added `remote_safe: bool` here; ADR-024 supersedes it and
// removes the field. Peer authorization is `AccessControl::check(peer_identity)`,
// not a per-op boolean. See ADR-024 §3.
}
/// Which dispatch path a handler uses — locked by ADR-021.
/// Validated against `spec.op_type` at registration:
/// Query/Mutation → Once; Subscription → Stream. Mismatch is a startup error.
pub enum HandlerKind {
Once(Handler),
Stream(StreamingHandler),
}
```
#### OperationProvenance
Where the op came from. Determines composition capability, default
visibility, and trust model. See ADR-018 for rationale.
```rust
pub enum OperationProvenance {
Local, // Assembly-written, trusted, can compose
FromOpenAPI, // HTTP forwarding stub (from_openapi), leaf
FromMCP, // MCP forwarding stub (from_mcp), leaf
FromCall, // call-protocol forwarding stub (from_call), leaf locally
FromJsonSchema, // HTTP forwarding stub (from_jsonschema, single endpoint), leaf
Session, // Agent-written, sandboxed, can compose within sandbox
}
```
| Provenance | Can compose? | Has composition authority? | Default visibility |
|-----------|-------------|---------------------------|-------------------|
| `Local` | Yes | Yes — scopes set by assembly layer | External or Internal (assembly declares) |
| `FromOpenAPI` | No (leaf) | No | Internal |
| `FromMCP` | No (leaf) | No | Internal |
| `FromCall` | No (leaf in local registry) | No | Internal |
| `FromJsonSchema` | No (leaf) | No | Internal |
| `Session` | Yes (within sandbox) | Yes — scopes set at sandbox creation | Internal always |
> **`FromJsonSchema` provenance.** `from_jsonschema` is an HTTP-backed
> single-endpoint adapter in `alknet-http` (ADR-027): a real reqwest
> forwarding handler, not a schema-only placeholder. `FromJsonSchema`
> is a leaf, same trust model as `FromOpenAPI` (HTTP endpoint trusted;
> handler is a forwarding stub). Schema validation without a handler is
> served by consuming `OperationSpec` directly, not by registering a
> placeholder op.
#### CompositionAuthority
The declared authority (label + scopes + resources) the handler operates
under when composing children. `None` for leaves. This replaces ADR-017's
`handler_identity: Identity` — it's not a peer identity, it's a declared
authority bundle. See ADR-018.
```rust
pub struct CompositionAuthority {
pub label: String, // e.g., "agent-chat" — not a peer id
pub scopes: Vec<String>, // e.g., ["llm:call", "fs:read"]
pub resources: HashMap<String, Vec<String>>, // e.g., {"service": ["vastai"]}
}
impl CompositionAuthority {
pub fn none() -> Option<Self> { None } // Convenience for leaves
pub fn new(label: &str, scopes: impl IntoIterator<Item = String>) -> Self { ... }
pub fn as_identity(&self) -> Option<Identity> { ... } // Synthetic Identity for ACL
}
```
- `provenance`: Determines composition capability. Only `Local` and `Session` ops can compose; leaves get `composition_authority: None` and `scoped_env: None`.
- `composition_authority`: The declared authority the handler operates under when composing children. `None` for leaves. See ADR-018.
- `scoped_env`: The set of operations this handler may reach via `env.invoke()`. `None` for leaves (empty env). The reachability control from ADR-017.
- `capabilities`: Outbound credentials (decrypted API keys, signing keys). Populated by the assembly layer from the vault at registration time. See [Capability Injection](#capability-injection).
The `OperationRegistryBuilder` provides a fluent API with convenience methods for common cases. The builder validates handler kind against `spec.op_type` at registration time — `with_local` / `with_leaf` accept `Handler` (for `Query`/`Mutation` ops), `with_local_streaming` / `with_leaf_streaming` accept `StreamingHandler` (for `Subscription` ops). Passing a `StreamingHandler` to `with_local` or a `Handler` to `with_local_streaming` is a registration-time error:
```rust
// with_local: Local provenance, full bundle — all 5 args required.
// Accepts Handler (for Query/Mutation ops). Validates op_type at registration.
// with_local(spec, handler, composition_authority, scoped_env, capabilities)
// with_local_streaming: Local provenance, full bundle — all 5 args required.
// Accepts StreamingHandler (for Subscription ops). Validates op_type at registration.
// with_local_streaming(spec, streaming_handler, composition_authority, scoped_env, capabilities)
// with_leaf: Leaf provenance (default FromOpenAPI), no composition authority.
// Accepts Handler (for Query/Mutation ops).
// with_leaf(spec, handler, capabilities)
// with_leaf_streaming: Leaf provenance (default FromOpenAPI), no composition authority.
// Accepts StreamingHandler (for Subscription ops).
// with_leaf_streaming(spec, streaming_handler, capabilities)
// with_leaf_provenance / with_leaf_streaming_provenance: explicit provenance variant.
let registry = OperationRegistryBuilder::new()
// Built-in service discovery (Local, no composition — empty authority, empty env, empty caps)
.with_local(services_list_spec(), Arc::new(services_list_handler),
CompositionAuthority::none(), ScopedPeerEnv::empty(), Capabilities::new())
.with_local(services_schema_spec(), Arc::new(schema_handler),
CompositionAuthority::none(), ScopedPeerEnv::empty(), Capabilities::new())
// Agent handler (Local, Subscription — streams call.responded as the
// LLM generates tokens; uses with_local_streaming for the StreamingHandler)
.with_local_streaming(agent_chat_spec(), Arc::new(agent_chat_streaming_handler),
CompositionAuthority::new("agent-chat", ["llm:call", "fs:read", "vastai:query"]),
ScopedPeerEnv::new(["fs/readFile", "vastai/listMachines", "llm/generate"]),
Capabilities::new().with_api_key("google", google_api_key))
// Imported ops (leaves — no authority, no scoped env; capabilities for outbound HTTP)
.with_leaf(vastai_listMachines_spec(), Arc::new(vastai_handler), vastai_credentials)
.build();
```
The CLI binary (or assembly layer) constructs the registry and passes it to the `CallAdapter`. Once built, the **curated layer** (Layer 0 — `Local` provenance ops) is immutable. Session and imported overlays are dynamic at their respective scopes (per-session, per-connection) per ADR-019. The `CallAdapter` composes the root `OperationContext.env` per incoming call from the active layers.
### OperationEnv
The `OperationEnv` trait is the universal composition mechanism. A handler calls `context.env.invoke("fs", "readFile", input, &context)` and gets a `ResponseEnvelope` back — regardless of whether the operation runs locally or on a remote node.
**`OperationEnv` is request/response-only** (ADR-021). It returns a single `ResponseEnvelope` — no streaming variant exists. Calling `invoke()` on a `Subscription` op produces `CallError { code: "INVALID_OPERATION_TYPE", ... }` — composition cannot truncate a stream to its first value. Stream composition (filter, map, combine, window, dedupe) is a handler-level concern, not a protocol composition concern; see ADR-021 for the rationale and the `@alkdev/pubsub` `operators.ts` prior art.
```rust
/// The composition dispatch trait. A handler composes child operations
/// through its `OperationContext.env` (which implements this trait).
///
/// This must remain a trait, not a concrete type — session-scoped
/// registries (OQ-19) depend on wrapping the global env via trait
/// layering. Making `OperationEnv` concrete or hardcoding the global
/// registry into the dispatch path would close the session-overlay
/// pattern.
#[async_trait]
pub trait OperationEnv: Send + Sync {
/// Compose a child operation. The child's `OperationContext` is
/// constructed with `internal: true`, inheriting the parent's
/// composition authority as the child's caller identity. The abort
/// policy defaults to the parent's (ADR-020 Decision 6, W19).
///
/// Default impl: delegates to `invoke_with_policy` with
/// `parent.abort_policy.clone()`. Impls only need to implement
/// `invoke_with_policy` — `invoke` is provided.
async fn invoke(
&self,
namespace: &str,
operation: &str,
input: Value,
parent: &OperationContext,
) -> ResponseEnvelope {
self.invoke_with_policy(namespace, operation, input, parent, parent.abort_policy.clone()).await
}
/// Compose a child with an explicit abort policy (ADR-020 Decision 6).
/// Use `AbortPolicy::ContinueRunning` for long-running work that
/// should survive a parent's abort. This is the required method —
/// `invoke()` delegates to it with the parent's policy.
async fn invoke_with_policy(
&self,
namespace: &str,
operation: &str,
input: Value,
parent: &OperationContext,
policy: AbortPolicy,
) -> ResponseEnvelope;
/// Does this env contain the named operation? Used by
/// `PeerCompositeEnv` to probe overlays before dispatching
/// (ADR-019 + ADR-024). The composite checks `session.contains()` →
/// each peer's sub-overlay (in `connection_order`) → base,
/// dispatching to the first overlay that contains the op. Default
/// impl returns `true` (a single-layer env like `LocalOperationEnv`
/// contains everything it can dispatch).
fn contains(&self, name: &str) -> bool { true }
/// Peer-routing composition (ADR-024 §2). Routes to a specific peer
/// (`PeerRef::Specific`) or to the first peer that serves the op
/// (`PeerRef::Any`). The default impl ignores the peer selector and
/// delegates to `invoke_with_policy`, preserving back-compat for
/// single-layer envs (`LocalOperationEnv`, `OverlayOperationEnv`)
/// that don't override it. `PeerCompositeEnv` overrides with real
/// peer-keyed routing.
async fn invoke_peer(
&self,
peer: &PeerRef,
namespace: &str,
operation: &str,
input: Value,
parent: &OperationContext,
policy: AbortPolicy,
) -> ResponseEnvelope {
// default: ignore peer selector, dispatch via invoke_with_policy
let _ = peer; // unused — single-layer envs don't route by peer
self.invoke_with_policy(namespace, operation, input, parent, policy).await
}
/// Does this env contain the named op *on the named peer*? Used by
/// `PeerCompositeEnv` to probe a specific peer's sub-overlay before
/// dispatching via `invoke_peer` with `PeerRef::Specific`. Default
/// impl delegates to `contains` (single-layer envs ignore the peer
/// dimension). `PeerCompositeEnv` overrides to check the specific
/// peer's sub-overlay.
fn peer_contains(&self, _peer: &PeerId, name: &str) -> bool { self.contains(name) }
}
```
The `parent` parameter propagates the calling context: the nested call gets `parent_request_id: Some(parent.request_id)`, inherits `parent.handler_identity` as the caller identity, and is marked `internal: true`.
The `invoke_peer` / `peer_contains` methods (ADR-024 §2) take a `PeerRef`
selector and a `PeerId`. These types are defined alongside the
`PeerCompositeEnv` struct — see [client-and-adapters.md](client-and-adapters.md#peer-keyed-composition-env-adr-029)
and [ADR-024](decisions/029-peer-graph-routing-model.md) §2:
```rust
pub enum PeerRef {
Specific(PeerId), // route to this peer; NOT_FOUND if it doesn't serve the op
Any, // first peer (insertion order) that serves it
}
pub type PeerId = String; // = Identity.id from IdentityProvider resolution
// = PeerEntry.peer_id (stable, not crypto material — ADR-025)
```
**Metadata does not propagate through composition.** Nested calls get fresh metadata (`HashMap::new()`), not the parent's metadata bag. This is a security constraint (ADR-010): `metadata: HashMap<String, Value>` accepts any `serde_json::Value`, including secret material. If metadata propagated through `env.invoke()`, a handler that accidentally placed a secret in metadata would leak it to every child operation — and if a child is a `from_call` operation (ADR-022), the metadata would cross the wire to the remote node. The tracing link between parent and child is `parent_request_id`, not metadata propagation. Anything a handler needs to pass to a child goes in the call `input`, not in ambient context.
**Local dispatch only.** The initial `OperationEnv` implementation for the
curated layer (Layer 0) dispatches directly through the local
`OperationRegistry`. The composite env (curated + session + peer-keyed
connection overlays) is a separate type built by the `CallAdapter` per call —
see ADR-019, ADR-024, and the `PeerCompositeEnv` sketch below.
```rust
/// Layer 0 dispatch — the curated registry. This is the base env that
/// overlays wrap. See ADR-019 for the layering model.
pub struct LocalOperationEnv {
registry: Arc<OperationRegistry>,
}
#[async_trait]
impl OperationEnv for LocalOperationEnv {
// `invoke` uses the default impl (delegates to `invoke_with_policy`
// with `parent.abort_policy.clone()`).
async fn invoke_with_policy(&self, namespace: &str, operation: &str, input: Value, parent: &OperationContext, policy: AbortPolicy) -> ResponseEnvelope {
let name = format!("{namespace}/{operation}");
// Reachability check (ADR-017, ADR-018): is this op in the parent's
// scoped env? If not, return NOT_FOUND. This bounds the
// parameterized-dispatch attack surface — a handler (or an LLM
// picking tools) can only reach declared ops. The reachability set
// is on `parent.scoped_env` (data), not on `parent.env` (dispatch
// trait) — see ADR-019 for the split.
if !parent.scoped_env.allows(&name) {
return ResponseEnvelope::not_found(name);
}
let registration = self.registry.registration(&name);
let context = OperationContext {
// Unique per invocation — a UUID v4 or parent_id + counter.
// A deterministic ID (e.g. format!("env-{name}")) collides across
// concurrent invocations of the same operation, which corrupts
// PendingRequestMap correlation and the abort-cascade tree
// (ADR-020), which is indexed by parent_request_id.
request_id: generate_request_id(),
parent_request_id: Some(parent.request_id.clone()),
// Parent's composition authority becomes the caller for the child.
// This is the authority switch: the child's ACL checks against
// the parent's authority, not the original wire caller's identity.
identity: parent.handler_identity.as_identity(),
// Child's own composition authority (from its registration).
// None for leaves — they don't compose, so this is never used
// for ACL on a grandchild.
handler_identity: registration.composition_authority.clone(),
// Composed children do not inherit forwarded_for — it's a
// wire-ingress field, not a composition-ingress field (ADR-026).
forwarded_for: None,
capabilities: parent.capabilities.clone(), // Inherit caller's capabilities
metadata: HashMap::new(), // Fresh — does NOT propagate parent metadata (ADR-010)
abort_policy: policy, // Explicit policy (from invoke() default or invoke_with_policy)
deadline: parent.deadline, // Inherit parent's deadline (children don't get a fresh 30s)
scoped_env: registration.scoped_env.clone()
.unwrap_or_else(ScopedPeerEnv::empty), // Child's own scoped env (empty for leaves)
// Dispatch trait: the child inherits the parent's env (the same
// composite of curated base + active overlays). See ADR-019.
env: parent.env.clone(),
internal: true, // Nested calls use handler authority
};
self.registry.invoke(&name, input, context).await
}
// `contains` uses the default impl (returns true — the curated registry
// contains everything it can dispatch). For a single-layer env, the
// reachability check in `invoke_with_policy` is the real gate.
}
```
The composite env (built by the `CallAdapter` per incoming call) wraps the
curated base and any active overlays. Per ADR-024, the connection overlay is
**peer-keyed** — a head node with N worker connections holds a
`HashMap<PeerId, connection_overlay>`, not one overlay. The singular-connection
case (one peer) is the degenerate case with a single-entry map.
```rust
/// Per-call composite env (ADR-019 + ADR-024). Built by the CallAdapter in
/// build_root_context from the active layers. The child inherits this by
/// Arc::clone through invoke(). The connection overlay is peer-keyed
/// (ADR-024 §1) to handle head→N-workers routing.
pub struct PeerCompositeEnv {
pub base: Arc<dyn OperationEnv + Send + Sync>, // Layer 0 curated
pub session: Option<Arc<dyn OperationEnv + Send + Sync>>, // Layer 1
pub connections: HashMap<PeerId, Arc<dyn OperationEnv + Send + Sync>>, // Layer 2, peer-keyed
connection_order: Vec<PeerId>, // insertion order for PeerRef::Any first-match
}
#[async_trait]
impl OperationEnv for PeerCompositeEnv {
// `invoke` uses the default impl (delegates to `invoke_with_policy`
// with `parent.abort_policy.clone()`).
async fn invoke_with_policy(&self, namespace: &str, operation: &str, input: Value, parent: &OperationContext, policy: AbortPolicy) -> ResponseEnvelope {
// PeerRef::Any routing (ADR-024 §2): session → peers in insertion
// order → curated base. First overlay that *contains* the op wins.
let name = format!("{namespace}/{operation}");
// Reachability check against parent.scoped_env (same as LocalOperationEnv).
if !parent.scoped_env.allows(&name) {
return ResponseEnvelope::not_found(name);
}
if let Some(session) = &self.session {
if session.contains(&name) {
return session.invoke_with_policy(namespace, operation, input, parent, policy).await;
}
}
// Peer-keyed overlay: iterate peers in insertion order, dispatch to
// the first peer whose sub-overlay contains the op. This is the
// head→N-workers fan-out primitive (ADR-024 §2, OQ-30: insertion-
// order first-match).
for peer_id in &self.connection_order {
if let Some(conn_env) = self.connections.get(peer_id) {
if conn_env.contains(&name) {
return conn_env.invoke_with_policy(namespace, operation, input, parent, policy).await;
}
}
}
self.base.invoke_with_policy(namespace, operation, input, parent, policy).await
}
// `invoke_peer` overrides the default impl with real peer-keyed
// routing (ADR-024 §2). `PeerRef::Specific` routes to the named peer's
// sub-overlay only (no fallthrough — NOT_FOUND if that peer doesn't
// serve the op). `PeerRef::Any` reuses `invoke_with_policy` (the
// insertion-order fan-out above).
async fn invoke_peer(
&self,
peer: &PeerRef,
namespace: &str,
operation: &str,
input: Value,
parent: &OperationContext,
policy: AbortPolicy,
) -> ResponseEnvelope {
let name = format!("{namespace}/{operation}");
if !parent.scoped_env.allows(&name) {
return ResponseEnvelope::not_found(name);
}
match peer {
PeerRef::Specific(peer_id) => {
// Route to this peer's sub-overlay only. No fallthrough —
// explicit routing must be honored or fail loudly (ADR-024 §2).
match self.connections.get(peer_id) {
Some(conn_env) if conn_env.contains(&name) => {
conn_env.invoke_with_policy(namespace, operation, input, parent, policy).await
}
_ => ResponseEnvelope::not_found(name),
}
}
PeerRef::Any => {
// Same as invoke_with_policy: session → peers in order → base.
self.invoke_with_policy(namespace, operation, input, parent, policy).await
}
}
}
fn contains(&self, name: &str) -> bool {
// The composite contains the op if any layer does (peer-agnostic).
self.session.as_ref().map_or(false, |s| s.contains(name))
|| self.connections.values().any(|c| c.contains(name))
|| self.base.contains(name)
}
fn peer_contains(&self, peer: &PeerId, name: &str) -> bool {
// Does the named peer's sub-overlay contain the op?
self.connections.get(peer).map_or(false, |c| c.contains(name))
}
}
```
The `contains()` method (review #003 C9) is the overlay-dispatch contract.
It replaces the previous "sentinel or contains check — two-way door" framing,
which was ambiguous enough to produce non-interoperable `OperationEnv` impls.
The structural decision (composite trait object, overlay order, `Arc::clone`
inheritance) is locked by ADR-019; the peer-keyed overlay extension
(`PeerCompositeEnv`, `invoke_peer`, `peer_contains`) is locked by ADR-024; the
dispatch contract (`contains` probe before `invoke_with_policy`) is locked by
both.
Two things happen in `invoke()`:
1. **Reachability check**: before constructing the child context, `invoke()` checks whether the requested op is in the parent's scoped env. If not, `NOT_FOUND`. This is the reachability control — a handler can only compose declared ops.
2. **Authority propagation**: the child's `identity` is the parent's `handler_identity` (the parent's composition authority becomes the caller). The child's `handler_identity` is the child's own registration's `composition_authority` — so if the child itself composes further, its children inherit the child's authority. This is the principal/agent chain from ADR-017, now wired via ADR-018.
Future work may add remote call protocol dispatch as an additional backend. The handler-facing API stays the same.
**`OperationEnv` must remain a trait.** This is a constraint, not a suggestion. The trait-based design enables registry layering (ADR-019): the CallAdapter composes the root env per call from the curated base + active peer-keyed connection overlays + session overlay, and overlays wrap the base via trait layering. Session-scoped registries (OQ-19) and connection-scoped remote imports (ADR-022 `from_call`) are both overlays on the same base, using the same mechanism. The peer-keyed extension (`PeerCompositeEnv`, `invoke_peer`, ADR-024) composes on top of the same trait — it overrides the new peer-routing methods, not the base dispatch. Making `OperationEnv` concrete or hardcoding the global registry into the dispatch path would close both the session-overlay and connection-overlay patterns, and would prevent the peer-keyed routing model from composing. This is the same integration-point pattern as `IdentityProvider` (ADR-003). See OQ-19, ADR-019, and ADR-024.
### Service Discovery
Two built-in operations expose what the node offers:
| Operation name | Display path | Type | Description |
|---------------|-------------|------|-------------|
| `services/list` | `/services/list` | Query | List registered operation names and metadata |
| `services/schema` | `/services/schema` | Query | Get the `OperationSpec` for a specific operation |
These are read-only — no admin operations are exposed through the call protocol itself.
`services/list` only returns `External` operations to remote callers. `Internal` operations are not part of the wire-facing API surface — they're implementation details of composition. A remote client cannot enumerate the internal call tree. See ADR-017.
`services/list` returns:
```json
{
"operations": [
{ "name": "fs/readFile", "namespace": "fs", "op_type": "query" },
{ "name": "agent/chat", "namespace": "agent", "op_type": "subscription" },
{ "name": "events/subscribe", "namespace": "events", "op_type": "subscription" }
]
}
```
`services/schema` accepts `{ "name": "fs/readFile" }` (no leading slash —
registry form, same as `OperationSpec.name`) and returns the full
`OperationSpec` including input/output JSON Schemas and declared
`error_schemas` (ADR-016). The `CallAdapter` normalizes the leading slash
from wire `operationId`s before lookup, so `services/schema` accepts both
`fs/readFile` and `/fs/readFile`. This enables client code generation: a
client reading the schema can produce typed error enums instead of generic
error handling.
### Operation Registry (hand-rolled, no irpc)
The operation registry is hand-rolled in alknet-call. ADR-013 accepted
"irpc as the call protocol foundation," but no `.rs` file in the workspace
ever imported irpc — the wire format (`wire.rs`), the operation registry,
and the dispatch are all hand-rolled. ADR-014 supersedes ADR-013 and
records the actual state. The table that previously contrasted "call
protocol (external, JSON)" with "irpc services (internal, postcard)" is
moot — there is no irpc layer.
If a handler internally uses a postcard/binary RPC for in-process calls,
that's a handler-internal choice, not an alknet-call integration. The
operation registry's external interface is always JSON (the `EventEnvelope`
wire format); the internal handler dispatch is a `Handler` /
`StreamingHandler` trait object (ADR-021), not an irpc `Service`.
### Operation Registration at Startup
The CLI binary (or assembly layer) constructs `HandlerRegistration` bundles with provenance, composition authority, scoped env, and capabilities (from the vault — see [Capability Injection](#capability-injection)), then registers them before starting the endpoint:
```rust
// Assembly layer: unlock vault, derive credentials
let vault = VaultServiceHandle::new();
vault.unlock(&mnemonic, passphrase.as_deref())?;
let google_api_key = vault.decrypt(&google_key_blob)?;
let github_signing_key = vault.derive_ed25519(PATHS::GITHUB_SIGNING)?;
let vastai_credentials = Capabilities::new().with_http_token("vastai", vastai_token);
// Register operations — vault operations are NOT registered here
let registry = OperationRegistryBuilder::new()
// Built-in service discovery (Local, no composition — empty caps)
.with_local(services_list_spec(), Arc::new(services_list_handler),
CompositionAuthority::none(), ScopedPeerEnv::empty(), Capabilities::new())
.with_local(services_schema_spec(), Arc::new(schema_handler),
CompositionAuthority::none(), ScopedPeerEnv::empty(), Capabilities::new())
// Agent handler (Local, Subscription — composes; streaming handler
// wrapped in HandlerKind::Stream by the builder per ADR-021)
.with(HandlerRegistration {
spec: agent_chat_spec(),
handler: HandlerKind::Stream(Arc::new(agent_chat_streaming_handler)),
provenance: OperationProvenance::Local,
composition_authority: Some(CompositionAuthority::new(
"agent-chat", ["llm:call", "fs:read", "vastai:query"])),
scoped_env: Some(ScopedPeerEnv::new(
["fs/readFile", "vastai/listMachines", "llm/generate"])),
capabilities: Capabilities::new().with_api_key("google", google_api_key),
})
// Vastai ops (FromOpenAPI, leaves — no authority, no scoped env)
.with_leaf(vastai_listMachines_spec(), Arc::new(vastai_listMachines_handler),
vastai_credentials.clone())
.build();
let call_adapter = CallAdapter::new(Arc::new(registry), identity_provider);
// Agent deployment: let call_adapter = CallAdapter::new(...).with_session_source(source);
```
The vault is used at construction time to populate `capabilities` in the registration bundle, not registered as call protocol operations. The curated layer (Layer 0) is immutable after construction — adding a `Local` op requires restarting the process. Session and imported overlays are dynamic at their respective scopes (ADR-019). This is consistent with OQ-04 (scoped to the `HandlerRegistry` by ADR-019), ADR-008, ADR-010, and ADR-018.
### Capability Injection
Handlers that need outbound credentials (LLM provider API keys, signing keys, HTTP service tokens) receive them through the `Capabilities` type on `OperationContext`, not by calling vault operations over the wire and not from environment variables. This is the mechanism that ADR-008 described in prose ("derived keys and decrypted credentials are injected into operation contexts at the assembly layer") and that ADR-010 specifies as a one-way door. ADR-018 specifies the registration path: capabilities live on the `HandlerRegistration` bundle, and the dispatch path populates `OperationContext.capabilities` from the bundle at call time.
The flow is:
```
Assembly layer (CLI startup):
1. Unlock vault (local, mnemonic from secure prompt or file)
2. Derive / decrypt the credentials each handler needs
3. Construct HandlerRegistration bundles with capabilities from the vault
4. Register the bundles in the OperationRegistry
5. Start the endpoint
Handler invocation (at call time):
call.requested → CallAdapter looks up registration by op name
→ build_root_context populates OperationContext.capabilities from registration.capabilities
→ handler reads context.capabilities → uses the credential for its outbound call
```
The handler closure does **not** capture capabilities — that was the pre-ADR-018 "Model A" that created a circular dependency with per-request `OperationContext.capabilities`. Capabilities live on the registration bundle, and the dispatch path populates the context from the bundle. One model, one wiring path. See ADR-018 Decision 6.
The `Capabilities` type holds non-serializable, zeroized secret material. It does not implement `Serialize` — it cannot cross the call protocol wire even by accident. The concrete shape of the type (a typed map, a struct with named fields, a trait object) is a two-way door for implementation. The one-way constraints are fixed by ADR-010:
- Capabilities are populated by the assembly layer at registration (on the `HandlerRegistration` bundle). They are never populated from call protocol inputs.
- Capabilities hold secret material that does not implement `Serialize` and does not appear in `EventEnvelope` payloads.
- The call protocol carries no secret material. See [call-protocol.md](call-protocol.md) for the wire-level constraint.
- **Capabilities are `Clone` and cloned through composition.** `OperationEnv::invoke()` calls `parent.capabilities.clone()` to pass capabilities to nested calls. This is intentional: a child handler needs the same outbound credentials as its parent (e.g., the `/agent/chat` handler composing `/fs/readFile` may need the same API key for an outbound LLM call). The security implication is that each composition step duplicates the secret material reference — but capabilities are scoped (the handler can only use what the assembly layer declared on the registration bundle), and children run under the parent's composition authority (ADR-017, ADR-018). A clone is the same scoped handle, not a widening of scope. The concrete cloning semantics (reference-counted `Arc` vs deep copy of zeroized material) is a two-way door for implementation, but `Capabilities: Clone` is required by the composition model.
- **Capabilities must be immutable after construction.** No interior mutability, no `Mutex<Map>`, no `RefCell`. This makes the clone-semantics two-way door genuinely two-way: Arc-based clone (shared immutable state) and deep-copy clone (isolated state) are behaviorally identical when neither supports mutation. Without this guard, a handler that mutates capabilities (e.g., adds a derived key for a child) would make the mutation visible to siblings and the parent under Arc-based clone — shared mutable state across the call tree, a security-relevant behavior. Once shipped, handlers may depend on shared mutation, and switching from Arc-shared to deep-copy-isolated later is a behavior change that breaks them. The immutability guard prevents the "two-way door" from becoming a future one-way door.
**No vault operations are registered in the call protocol.** The vault is assembly-layer only (ADR-008, ADR-010). A handler that needs a child key for a specific operation (e.g., signing for GitHub auth) receives a scoped capability that performs the derivation in-process — it never holds the master seed and never calls a network-exposed vault operation.
**Adapters take credential sources.** All import adapters (`from_openapi`, `from_mcp`, `from_jsonschema`, `from_call` — see ADR-022, constrained by ADR-010) register HTTP-backed, MCP-backed, or remote-call-backed operations. The credential each service needs (bearer token, API key, TLS identity for the remote connection) is provided by the assembly layer at registration time — the adapter receives a credential source, not a static token string. This is the integration point where the vault feeds credentials into backed operations, including LLM providers that expose OpenAPI-compatible endpoints. Adapter-registered operations are `Internal` by default (ADR-017) — they're composition material, not directly callable from the wire.
**`from_call` imports remote operations.** The `from_call` adapter (ADR-022) discovers operations on a remote call protocol endpoint via `services/list` and `services/schema`, then registers them with handlers that forward calls over the call-protocol connection (transport-agnostic — QUIC, TCP+TLS, or any `Connection::from_stream` source, ADR-007). This makes cross-node composition transparent — a handler calling `env.invoke("worker", "exec", ...)` doesn't know whether the operation is local or remote. Connection direction (who opened the connection) is independent of call direction (who calls whom) — both sides can call each other once connected.
**`from_call` trust is transitive.** A `from_call`-imported operation executes the remote node's code, not yours. The scoped env (ADR-017) bounds *which* operations are reachable, but not *what* they do. A compromised remote node can do anything its operations are declared to do (and anything its handler bugs allow). This is inherent to remote composition — same as trusting any RPC endpoint — but it must be explicit in the threat model. `from_call` means "I trust the remote node as much as my own handlers." The scoping protects the caller from reaching arbitrary ops; it does not protect against what the reached op does.
**Scoped composition env.** The `OperationEnv` given to a handler is scoped — it can only invoke a declared set of operations, set at registration on the `HandlerRegistration` bundle by the assembly layer (ADR-018). This bounds the parameterized-dispatch attack surface: a handler (or an LLM picking tools, or a quickjs sandbox) can only reach declared operations, not the entire registry. The scoped env is the reachability control; the composition authority is the authority control. Both are needed for least privilege. See ADR-017 and ADR-018.
**No-env-vars invariant.** No handler reads outbound credentials from any source other than `OperationContext.capabilities`. This is the dispatch-side corollary of the capability-injection flow above: because the dispatch path populates `OperationContext.capabilities` from the registration bundle (ADR-018 §6), and because the assembly layer constructs handlers with vault-derived credentials rather than calling `Default::default()`, downstream consumers' `std::env::var` credential reads are unreachable by construction. The full invariant, the credential injection path, and the downstream-consumer framing are recorded in [client-and-adapters.md](client-and-adapters.md); this section documents the dispatch-path mechanism that makes it enforceable.
## Constraints
- The registry is **layered by trust boundary** (ADR-019). The curated layer (`Local` provenance) is immutable after construction — adding a `Local` op requires restarting the process, which re-enters the startup trust boundary. Session (`Session`) and imported (`FromCall` etc.) ops are dynamic at their respective scopes (per-session, per-connection). The pre-ADR-019 blanket immutability claim was inherited by analogy from ADR-010's `HandlerRegistry` (ALPN-level) and did not apply to the operation registry — the TLS-config argument that justifies `HandlerRegistry` immutability does not touch the operation registry, which lives behind the single ALPN `alknet/call`.
- Operation specs use JSON Schema. The call protocol's external interface is always JSON. Internal handler dispatch is via `Handler` / `StreamingHandler` trait objects (ADR-021), not a binary RPC framework.
- `OperationEnv::invoke()` dispatches through the local registry. Remote dispatch (federation, head/worker routing) would be a separate mechanism at a different layer — not a prefix added to operation paths.
- The call protocol does not depend on any database. Operation specs are in-memory, populated at startup.
- `OperationContext.internal` is set by `OperationEnv`, not by callers. A handler cannot mark its own call as internal. The `internal` flag switches authority context (composition authority for ACL), it does not skip ACL — see ADR-017, ADR-018.
- **Operations have External/Internal visibility.** `Internal` operations return `NOT_FOUND` when called from the wire and are excluded from `services/list`. The assembly layer declares visibility at registration. See ADR-017.
- **The composition env is scoped.** A handler can only invoke operations declared in its scoped env (on the `HandlerRegistration` bundle). This bounds parameterized-dispatch attack surface. See ADR-017, ADR-018.
- **No vault operations are registered in the call protocol.** The vault is assembly-layer only (ADR-008, ADR-010). Handlers receive secret material through `OperationContext.capabilities`, not by calling vault operations over the wire.
- **The call protocol carries no secret material.** Secret material (private keys, API keys, mnemonics, decrypted credentials) must not appear in `call.requested` payloads, `call.responded` payloads, or `OperationContext.metadata`. See ADR-010.
- **Metadata does not propagate through composition.** `OperationEnv::invoke()` constructs fresh metadata for nested calls (`HashMap::new()`), not the parent's metadata. This prevents a handler that accidentally places a secret in metadata from leaking it to child operations — and if a child is a `from_call` operation (ADR-022), across the wire to a remote node. The tracing link is `parent_request_id`, not metadata propagation. See ADR-010.
- **Provenance determines composition capability.** Only `Local` and `Session` ops can compose. Leaves (`FromOpenAPI`, `FromMCP`, `FromCall`) get `composition_authority: None` and `scoped_env: None` — they don't compose, so they don't need authority or reachability bounds. See ADR-018.
- **`HandlerKind` matches `op_type`** (ADR-021). `Query`/`Mutation` ops register a `HandlerKind::Once(Handler)`; `Subscription` ops register a `HandlerKind::Stream(StreamingHandler)`. Mismatch is a startup error. `invoke()` on a `Subscription` and `invoke_streaming()` on a `Query`/`Mutation` both return `INVALID_OPERATION_TYPE`. `OperationEnv::invoke()` (composition) is request/response-only and errors with `INVALID_OPERATION_TYPE` on `Subscription` ops — stream composition is a handler-level concern, not a protocol composition concern.
## Design Decisions
| Decision | ADR | Summary |
|----------|-----|---------|
| Hand-rolled EventEnvelope framing (irpc never integrated) | [ADR-014](decisions/064-irpc-never-integrated-hand-rolled-framing.md) | Hand-rolled framing, registry, dispatch; supersedes ADR-013 |
| Call protocol stream model | [ADR-015](decisions/012-call-protocol-stream-model.md) | Bidirectional streams, EventEnvelope, ID-based correlation |
| Static handler registration | [ADR-010](decisions/010-alpn-router-and-endpoint.md) | `HandlerRegistry` (ALPN-level) immutable after construction; `OperationRegistry` layered by ADR-019 (curated immutable, session/imported dynamic) |
| Vault integration via assembly layer | [ADR-008](decisions/008-secret-service-integration.md) | Vault is a capability source, accessed at assembly time |
| Secret material flow and capability injection | [ADR-010](decisions/014-secret-material-flow-and-capability-injection.md) | Capabilities carry outbound credentials; call protocol carries no secret material |
| Privilege model and authority context | [ADR-017](decisions/015-privilege-model-and-authority-context.md) | `internal` = authority switch not ACL skip; External/Internal visibility; composition authority + scoped env |
| Handler registration, provenance, and composition authority | [ADR-018](decisions/022-handler-registration-provenance-and-composition-authority.md) | Registration bundle carries provenance, composition authority, scoped env, capabilities; dispatch path reads from bundle |
| Operation registry layering | [ADR-019](decisions/024-operation-registry-layering.md) | Curated (static, immutable) + session and connection overlays (dynamic); `OperationEnv` as trait-object integration point; `OperationContext.env` split into `scoped_env` (data) and `env` (dispatch trait) |
| Operation error schemas | [ADR-016](decisions/023-operation-error-schemas.md) | Operations declare domain errors; `call.error` carries typed `details`; adapter fidelity for `from_openapi`/`to_openapi` |
| Call protocol client and adapter contract | [ADR-022](decisions/017-call-protocol-client-and-adapter-contract.md) | `from_call`/`OperationAdapter` produce `HandlerRegistration` bundles; adapter-registered ops are `Internal` leaves. Surface specced in [client-and-adapters.md](client-and-adapters.md) |
| `from_jsonschema` as HTTP-backed single-endpoint adapter | [ADR-027](decisions/066-from-jsonschema-as-http-adapter.md) | Moved `from_jsonschema` from `alknet-call` (broken schema-only placeholder) to `alknet-http` as a real reqwest-backed single-endpoint adapter; `FromJsonSchema` provenance stays in `alknet-call` as a leaf (now handler-bearing, not "no handler") |
| Peer-graph routing model (supersedes ADR-023) | [ADR-024](decisions/029-peer-graph-routing-model.md) | Peer-keyed overlays + `PeerRef` routing; peer authorization via `AccessControl::check(peer_identity)`; retires `remote_safe`/`trusted_peer` (the field this doc's `HandlerRegistration` previously gained) |
| Forwarded-for identity | [ADR-026](decisions/032-forwarded-for-identity.md) | `forwarded_for` field on `OperationContext` and `call.requested`; metadata only — `AccessControl::check` never reads it; the `from_call` handler populates it |
| Streaming handler for subscriptions | [ADR-021](decisions/049-streaming-handler-for-subscriptions.md) | `StreamingHandler` type alongside `Handler`; `HandlerKind` enum on `HandlerRegistration` validated against `op_type`; `invoke_streaming()` on `OperationRegistry`; `invoke()` and `OperationEnv::invoke()` error with `INVALID_OPERATION_TYPE` on `Subscription` ops; composition stays request/response-only, stream composition is handler-level |
| Dynamic resource ownership for runtime-spawned resources | [ADR-011](decisions/050-dynamic-resource-ownership-for-runtime-spawned-resources.md) | `AccessControl::check` consults an `OwnershipProvider` (sync read trait, ADR-033 repo/adapter pattern); `OperationSpec` gains `resource_id_path` (JSON pointer into the input); proxy-only access pattern (spawner owns, proxy to share, teardown revokes); `list` = scope-gate + result-filter; teardown = automatic, handler-driven; composition = two orthogonal checks, ADR-017/022 unchanged |
## Open Questions
See [open-questions.md](open-questions.md) for full details.
- **OQ-13** (resolved): Operation path format is `/{service}/{op}`. Remote dispatch is a separate mechanism, not a path prefix.
- **OQ-14** (resolved): Batch is a client-side pattern of correlated `call.requested` events, not a protocol primitive.
- **OQ-16** (resolved by ADR-010): No vault operations are exposed over the call protocol for now.
- **OQ-19** (resolved): Session-scoped operation registries — agent-written operations overlaid on the curated registry via `OperationEnv` trait layering. Protocol doesn't need changes; `OperationEnv` must remain a trait. Session ops are `Session` provenance (ADR-018) — always `Internal`, compose under restricted authority scoped down at sandbox creation. Generalized by ADR-019 to cover connection-scoped overlays as well.
- **OQ-25** (dissolved by ADR-024): `remote_safe` marking shape — moot.
`remote_safe`/`trusted_peer` are retired; peer authorization is
`AccessControl::check(peer_identity)`, the existing mechanism. See
[client-and-adapters.md](client-and-adapters.md) and ADR-024 §3.
- **OQ-26** (resolved): `OperationAdapter` error type — `AdapterError`
variants: `DiscoveryFailed`, `SchemaParse`, `Transport`, `Unauthorized`,
`SamePeerCollision` (replaces flat `Conflict`). `#[non_exhaustive]`. See
[client-and-adapters.md](client-and-adapters.md).
- **OQ-27** (resolved): `from_call` re-import trigger — `from_call` is a
manual free function; the assembly layer calls it after the dial (in
`AlknetClient`). A `CallConnection::refresh()` method is a genuine
feature addition — non-breaking, additive. See
[ADR-028](decisions/069-from-call-manual-free-function.md).
- **OQ-28** (resolved): `from_call` namespace collision — same-peer
collision = error; cross-peer dissolved by ADR-024 (separate sub-overlays).
`namespace_prefix` is optional local-naming sugar. See
[client-and-adapters.md](client-and-adapters.md).
- **OQ-29..37** are tracked in [client-and-adapters.md](client-and-adapters.md)
(they concern the `CallClient` / adapter surface and peer-graph routing,
not the registry layering this document covers). In brief: OQ-29
(CallClient TLS client-auth) resolved; OQ-30 (`PeerRef::Any` routing
policy) resolved; OQ-31 (`services/list-peers` re-export semantics)
resolved; OQ-32 (multi-hop federation) open (feature extension);
OQ-33 (PeerId source) resolved by ADR-025; OQ-34 (persistent peer
registry) resolved by ADR-025+033; OQ-35 (API key asymmetry) dissolved;
OQ-36 (concrete persistence adapter shapes) resolved by ADR-035;
OQ-37 (X.509 outgoing-only) resolved by ADR-034.
- **OQ-42** (resolved by ADR-011): Dynamic resource ownership for
runtime-spawned resources — `AccessControl::check` consults an
`OwnershipProvider`; `OperationSpec` gains `resource_id_path`; proxy-only
access pattern; four edge specifics pinned (`list`, teardown, fleet,
composition). See [auth.md](../core/auth.md) §"Ownership Provider and
Store" for the trait shapes.
## References
- [call-protocol.md](call-protocol.md) — CallAdapter, EventEnvelope, stream model, PendingRequestMap
- ADR-014: Hand-rolled EventEnvelope framing (irpc never integrated; supersedes ADR-013)
- ADR-008: Vault integration point
- ADR-010: ALPN router and endpoint (static registration — applies to the `HandlerRegistry`, not the `OperationRegistry`; see ADR-019 for the distinction)
- ADR-015: Call protocol stream model
- ADR-019: Operation registry layering (curated + session/connection overlays; `OperationEnv` as trait-object integration point)
- ADR-024: Peer-graph routing model (peer-keyed overlays + `PeerRef` routing; `PeerCompositeEnv` supersedes the singular-connection `CompositeOperationEnv`)
- ADR-025: PeerEntry and Identity.id decoupling (`PeerId` source = `Identity.id` = `PeerEntry.peer_id`)
- ADR-026: Forwarded-for identity (`forwarded_for` on `OperationContext` and `call.requested`; metadata only)
- ADR-021: Streaming handler for subscriptions (`StreamingHandler`, `HandlerKind`, `invoke_streaming()`, `INVALID_OPERATION_TYPE`)
- ADR-011: Dynamic resource ownership for runtime-spawned resources (`OwnershipProvider` consulted by `AccessControl::check`; `OperationSpec.resource_id_path`; proxy-only access pattern; composition = two orthogonal checks, ADR-017/022 unchanged)
- Reference implementation: `/workspace/@alkdev/alknet-main/crates/alknet-core/src/call/`